Find your own place on the Red Planet: ASU, NASA and Google team up to give the public a way to suggest Mars sites to photograph, powered by Drupal

Arizona State University researchers and scientists have teamed with Google to create a new layer in Google Earth 5.0. The layer lets anyone, anywhere, recommend places on Mars to photograph with ASU's THEMIS camera on NASA's Mars Odyssey orbiter.

THEMIS is the Thermal Emission Imaging System, a multiband infrared and visual camera designed at ASU by Dr. Philip Christensen. A Regents Professor of Geological Sciences in the School of Earth and Space Exploration, Christensen is THEMIS' principal investigator and also director of the Mars Space Flight Facility on the Tempe campus.

Late on a Wednesday afternoon, Dr. Christensen approached software engineers Christian Yates and Eric Engle to develop an application that would let the public participate in Mars exploration by enabling them to suggest places for THEMIS to photograph.

By Friday morning, the team had a working application, with the help of Drupal.

Background

"We wanted to give the general public a way to suggest places on Mars for THEMIS to photograph," says Christensen. "Using the feature, people can recommend sites, and these recommendations go to mission scientists who will decide what areas THEMIS images. If a public suggestion matches what the researchers choose, we'll notify the person who suggested the site and let them see the image as soon as we do."

The challenge to the Mars Space Flight Facility engineers was to create a user-friendly application accessible from within Google Earth that engages the public and encourages interaction.

Additionally, the app had to withstand a potential onslaught of traffic from being embedded into millions of copies of Google Earth.

Overview

View from Google Earth
Google Earth screenshot

Most users will first encounter the "Suggest an Image" feature through the Google Earth application itself, where it resides in the Mars Gallery section. A click of a button loads the feature, and draws tracks on the planet's surface indicating where THEMIS will be able to take photos during the coming week.

Clicking on the track opens a "balloon" that lets the user add a comment, and their email address if they'd like to receive responses.

The Suggest an Image web site allows users a second method of using the layer. By registering with the site first, they can download a customized KML file to load into Google Earth that will let them enter suggestions without having to type their email address each time.

The Web Site
Website screen shot

For users that started directly from the Mars Gallery, the web site allows access to their past suggestions, images that have been successfully captured, and their preferences.

The site is also responsible for updating users when the status of their place suggestions change. Users can receive notifications via email when their suggestions are uploaded to the Mars Odyssey spacecraft, when the data is downlinked back to Earth, and when an image is fully processed and available for viewing on the web site.

Technical Details

With the release of version 5 of Google Earth, a number of new opportunities opened up. Most importantly for us, Google introduced a Mars layer, where users can explore images from instruments aboard a variety of past and present Mars spacecraft, including THEMIS.

Google Earth 5 also brings flexibility to developers by allowing greater use of HTML, CSS and Javascript within the application, which opens up doors for interacting with external services.

The Process
Connected applications

Drupal sits at the heart of the Suggest an Image application, and interacts with a variety of services:

Postgres and MySQL Databases

Store the data downloaded from the spacecraft, as well as geometry defining where Odyssey will be orbiting in the near future.

Java Applications

Query the databases to assemble KML files containing orbital tracks, and also determine when a user's suggestion has intersected with an image taken by THEMIS, returning data files to Drupal.

A processed image
A finished image
Python

A python web service written for the Mars Image Explorer (also a Drupal application) serves finished images in a variety of formats and sizes that users can download and share.

Drupal

Fulfills several roles in the Suggest an Image application:

  • User creation and preference management: Users can sign up for an account from the web site, or by entering an email address in Google Earth. An account is automatically created if a unique email address is given.
  • Notifications: The Notifications module powers interaction between the service and its users.
  • KML customization: A custom Drupal module takes a raw KML file generated by our Java applications, and adds user interface elements generated with the Drupal Forms API.
  • KML serving: The same custom module serves the finished KML file in both "generic" format, for inclusion in Google Earth, and a "customized" format, which is tailored to a specific user, and downloaded from the web site.
  • User transaction handling: Drupal processes suggestion submissions, manages sumbission quotas with the Quota by Role module, and ensures data security.

Drupal Environment

Drupal 6.10 was chosen as the core of the Suggest an Image service, and several contributed modules were used to complete the application a tight timeframe.

Module
Mars Suggest A proprietary module created for this project that manages the user interface, modifies and serves KML files and interacts with other modules to perform notifications, etc.
CCK Adds a Suggestion content type with fields for image status, ID, latitude and longitude.
Profile Adds opt-in buttons to user pages for notifications.
Notifications & Messaging frameworks Used to deliver notifications on suggestion status changes to users.
Lightbox2 Allows users to zoom into full-size versions of finished THEMIS images.
Login Destination and LoginToboggan Redirect users directly to their account pages on login, and allow immediate registration, respectively.
Panels2 Customizes user's account pages, allowing for inclusion of instructions, embedding of custom views, and custom KML download links.
Views2 Used to generate lists of finished images, users past suggestions, and to power generation of the front page "heat map" of user suggestion activity.
Service Links Creates links for sharing finished images via a number of social networks.
Quota by Role Allows limiting of the number of submissions from each user per week. Note that some functions of Quota by role had to be migrated to our Mars Suggest module in order to set specific date ranges on quotas.

Challenges

Performance

Infrastructure
Infrastructure

The Suggest an Image application had to be able to perform under the load imposed by direct inclusion into the Google Earth Mars Gallery layer. To accomplish this, both hardware and software architecture was carefully designed.

On the hardware side, a multi-layered architecture was devised, using layer-7 hardware load balancers from Coyote Point systems to direct traffic to a number of Squid cache servers and Lighttpd web servers. Both the Squid and Lighttpd layers scale well horizontally. PHP performance is augmented with the Alternative PHP Cache (APC).

While PHP code (Drupal core, modules, themes, etc.) is kept local to the web servers for best performance and synced with Rsync, the Drupal /files directory is served from our NetApp filers via NFS. THEMIS images themselves are served via a proprietary Python web service from our large AFS file system.

On the software side it was necessary to devise a way to minimize the load to both Drupal's MySQL server and our Postgres data servers. To do this, Drupal was utilized to generate and package HTML, CSS and images into a KMZ file (essentially, a ZIP file) which is served up by the web server layer and cached by our Squid farm for a week at a time.

Encapsulating all of the components this way consolidates the number of connections necessary to render the application's interface in Google Earth to one per session. Additional calls to Drupal are only made when a user actually submits a suggestion.

Design

While Google Earth 5.0 supports far more HTML, Javascript and CSS than its predecessor, some changes still need to be made to the application to optimize performance.

Since Google Earth does not support cookies, or even the POST method, an alternative way of submitting new suggestion nodes was devised, relying on keys rather than cookies or session ids.

Graphic design was also changed to reflect the limits of what types of Javascript and CSS could be implemented from within a KML file.

Next Steps

With the recent announcement of the Google Earth 5.0 browser plugin by our colleagues at Google, we'll be able to bring the Suggest an Image experience directly into Drupal.

We also have plans to increase the interactivity of the application by enabling a heatmap layer within Google Earth, and capturing the most popular areas each week with the THEMIS camera.

Credits

JAVA programming, KML generation and orbital geometry:
Eric Engle - eengle [at] mars [dot] asu [dot] edu

Drupal/Python development, architecture and UX design:
Christian Yates - cyates [at] mars [dot] asu [dot] edu

Graphic Design:
Chris "Scooby" Capages - ccapages [at] mars [dot] asu [dot] edu