ExoMars rover concept
A concept image for the ExoMars rover that is being developed for a 2018 mission to Mars.
updated 7/8/2012 1:17:47 PM ET 2012-07-08T17:17:47

It's a hot summer day, and your eyes spot an ice cream cart up ahead. Without even really thinking, you start walking that direction. Planetary scientists would like to give robots that kind of visual recognition — not for getting ice cream, but for finding scientifically interesting targets.

Currently, rovers and other space vehicles are still largely dependent on commands from their human controllers back on Earth. But to decide what commands to send, operators must wait to receive images and other pertinent information from the spacecraft. Because rovers don't have powerful antennas, this so-called downlink usually takes a lot of time.

The data bottleneck means rovers often "twiddle their thumbs" between subsequent commands.

"Our goal is to make smart instruments that can do more within each command cycle," says David Thompson of the Jet Propulsion Laboratory in Pasadena, Calif. 

Thompson is heading a project called TextureCam, which involves creating a computer vision package that can map a surface by identifying geological features. It is primarily envisioned for a rover, but it could also benefit a spacecraft visiting an asteroid or an aerobot hovering in the atmosphere of a distant world. [Curiosity - The SUV of Mars Rovers]

With funds from NASA's Astrobiology Science and Technology for Exploring Planets (ASTEP), Thompson's team is currently refining their computer algorithm, with an eventual plan to build a prototype instrument that can map an astrobiologically-relevant field site.

TextureCam analysis of Mars image
NASA / JPL / Caltech / Cornell
A TextureCam analysis of a Mars image is able to distinguish rocks from soil.

Roam rover, roam rover
Rovers have already made great advances in autonomy. Current prototypes can travel as much as a kilometer on their own using on-board navigation software. This allows these vehicles to cover a much larger territory.

But one concern is that a rover may literally drive over a potentially valuable piece of scientific real estate and not even realize it. Giving a rover some rudimentary visual identification capabilities could help avoid missing "the needle in the haystack," as Thompson refers to the hidden clues that astrobiologists hope to uncover on other planets.

"If the rover can make simple distinctions, we can speed up the reconnaissance," he says. As it drives along, the rover could snap several images and use on-board software to prioritize which images to downlink to Earth.

And while waiting for its next set of commands, it could pick a potentially interesting geological feature and then drive up close to take a detailed picture or even perform some simple chemical analysis.

"You could start the next day with the instrument sitting in front of a prime location," Thompson says.

Instead of spending time trying to get the rover from point A to point B, mission controllers could concentrate on doing the higher level scientific investigation that the rover can't do. At least, not yet. [NASA's Mars Rover Curiosity: 11 Amazing Facts]

"The field being investigated by David Thomson is vital to cope with the flood of remote sensing data returned from spacecraft," says Anthony Cook of Aberystwyth University in the UK, who is not involved with TextureCam.

There are a other projects working on computer vision for rovers. In 2010, the Mars rover Opportunity received a software upgrade called AEGIS that can identify scientifically interesting rocks. A project in the Atacama desert in Chile used a similar rock detector system on its rover called Zoë. And ESA's ExoMars mission is developing computer vision that can detect objects in the rover's vicinity.

TextureCam is unique from these other efforts in that it is mapping the surface, rather than trying to isolate particular objects. It's a more general strategy that can identify terrain characteristics, such as weathering or fracturing.

Stromatolite TextureCam analysis
A photo of a stromatolite (left) from Western Australia analyzed by TextureCam (right). The program assigns a color to each patch in the image according to how it matches the criteria for stromatolite rocks (red means good match, or high proba

Recognizing a rock face
The new approach by Thompson's group focuses on the "texture" of an image, which is computer vision terminology for the statistical patterns that exist in an array of  pixels. The same kind of image analysis is being used in more common day-to-day applications.

For example, the web is inundated with huge photo archives that haven't been sorted in any systematic way. Several companies are developing "search engines" that can identify objects in digital images. If you were looking for, say, an image with a "blue dog" or a "telephone booth," these programs could sift through a collection of photos to find those that match the particular criteria.

Additionally, many digital cameras detect faces in the camera frame and automatically adjust the focus depending on how far away the faces are. And some new video game consoles have sensors to detect the bodily pose of a game player. 

What all these technologies have in common is a sophisticated analysis of image pixels. The relevant software programs typically look for signals in the variations of brightness or the shades of color that are characteristic of a telephone or a face or a rock.

These signals often have little to do with the way we might describe these objects.

"The software identifies statistical properties that might not be obvious to the human eye," Thompson says.

  1. Space news from NBCNews.com
    1. KARE
      Teen's space mission fueled by social media

      Science editor Alan Boyle's blog: "Astronaut Abby" is at the controls of a social-media machine that is launching the 15-year-old from Minnesota to Kazakhstan this month for the liftoff of the International Space Station's next crew.

    2. Buzz Aldrin's vision for journey to Mars
    3. Giant black hole may be cooking up meals
    4. Watch a 'ring of fire' solar eclipse online

Let the computer do the guesswork
In the case of TextureCam, the computer program takes a small patch, or thumbnail, inside the image and performs a number of different pixel-to-pixel comparisons. Which comparisons? Actually, the computer decides.

"We train the system from examples," Thompson explains. They take images that were previously analyzed by a geologist as having an outcrop or a sediment or a rock of a particular variety. The computer program compares its pixel analysis to these labels and builds a decision tree (or a more elaborate "decision forest") that best discriminates between the different possibilities. 

"These decision trees can be quite efficient even after just a few branches," Thompson says.

This so-called "machine learning" has advantages over other techniques that construct a visual model of what the computer should be looking for.

"The disadvantage with visual models is that you have to build a new rule for every new thing you want to identify," Thompson says. It can be hard for humans to find reliable distinctions that can help a computer. It makes more sense to let the computer go out and explore the possibilities with trial and error.

"The system trains itself, so we don't have to anticipate," Thompson says.

The "training regimen" for TextureCam began with a set of images from Mars and is now moving onto photos from the Mojave Desert.

The team plans to integrate their algorithm into a field programmable gate array (FPGA), which is basically a special purpose computer that would connect directly to a rover camera. This would allow TextureCam to work faster, without relying on the rover's main computer.

"Computers and software are not ready to take over the interpretation tasks of human geologists, but they will help to pre-sort and pre-identify regions of interest, thus reducing the amount of remote sensing data that geologists must examine," Cook says.

This story was provided by Astrobiology Magazine, a Web-based publication sponsored by the NASA astrobiology program.

© 2013 Space.com. All rights reserved. More from Space.com.

Photos: Month in Space: January 2014

loading photos...
  1. Southern stargazing

    Stars, galaxies and nebulas dot the skies over the European Southern Observatory's La Silla Paranal Observatory in Chile, in a picture released on Jan. 7. This image also shows three of the four movable units that feed light into the Very Large Telescope Interferometer, the world's most advanced optical instrument. Combining to form one larger telescope, they are greater than the sum of their parts: They reveal details that would otherwise be visible only through a telescope as large as the distance between them. (Y. Beletsky / ESO) Back to slideshow navigation
  2. A balloon's view

    Cameras captured the Grandville High School RoboDawgs' balloon floating through Earth's upper atmosphere during its ascent on Dec. 28, 2013. The Grandville RoboDawgs’ first winter balloon launch reached an estimated altitude of 130,000 feet, or about 25 miles, according to coaches Mike Evele and Doug Hepfer. It skyrocketed past the team’s previous 100,000-feet record set in June. The RoboDawgs started with just one robotics team in 1998, but they've grown to support more than 30 teams at public schools in Grandville, Mich. (Kyle Moroney / AP) Back to slideshow navigation
  3. Spacemen at work

    Russian cosmonauts Oleg Kotov, right, and Sergey Ryazanskiy perform maintenance on the International Space Station on Jan. 27. During the six-hour, eight-minute spacewalk, Kotov and Ryazanskiy completed the installation of a pair of high-fidelity cameras that experienced connectivity issues during a Dec. 27 spacewalk. The cosmonauts also retrieved scientific gear outside the station's Russian segment. (NASA) Back to slideshow navigation
  4. Special delivery

    The International Space Station's Canadian-built robotic arm moves toward Orbital Sciences Corp.'s Cygnus autonomous cargo craft as it approaches the station for a Jan. 12 delivery. The mountains below are the southwestern Alps. (NASA) Back to slideshow navigation
  5. Accidental art

    A piece of art? A time-lapse photo? A flickering light show? At first glance, this image looks nothing like the images we're used to seeing from the Hubble Space Telescope. But it's a genuine Hubble frame that was released on Jan. 27. Hubble's team suspects that the telescope's Fine Guidance System locked onto a bad guide star, potentially a double star or binary. This caused an error in the tracking system, resulting in a remarkable picture of brightly colored stellar streaks. The prominent red streaks are from stars in the globular cluster NGC 288. (NASA / ESA) Back to slideshow navigation
  6. Supersonic test flight

    A camera looking back over Virgin Galactic's SpaceShipTwo's fuselage shows the rocket burn with a Mojave Desert vista in the background during a test flight of the rocket plane on Jan. 10. Cameras were mounted on the exterior of SpaceShipTwo as well as its carrier airplane, WhiteKnightTwo, to monitor the rocket engine's performance. The test was aimed at setting the stage for honest-to-goodness flights into outer space later this year, and eventual commercial space tours.

    More about SpaceShipTwo on PhotoBlog (Virgin Galactic) Back to slideshow navigation
  7. Red lagoon

    The VLT Survey Telescope at the European Southern Observatory's Paranal Observatory in Chile captured this richly detailed new image of the Lagoon Nebula, released on Jan. 22. This giant cloud of gas and dust is creating intensely bright young stars, and is home to young stellar clusters. This image is a tiny part of just one of 11 public surveys of the sky now in progress using ESO telescopes. (ESO/VPHAS team) Back to slideshow navigation
  8. Fire on the mountain

    This image provided by NASA shows a satellite view of smoke from the Colby Fire, taken by the Multi-angle Imaging SpectroRadiometer aboard NASA's Terra spacecraft as it passed over Southern California on Jan. 16. The fire burned more than 1,863 acres and forced the evacuation of 3,700 people. (NASA via AP) Back to slideshow navigation
  9. Where stars are born

    An image captured by NASA's Spitzer Space Telescope shows the Orion Nebula, an immense stellar nursery some 1,500 light-years away. This false-color infrared view, released on Jan. 15, spans about 40 light-years across the region. The brightest portion of the nebula is centered on Orion's young, massive, hot stars, known as the Trapezium Cluster. But Spitzer also can detect stars still in the process of formation, seen here in red hues. (NASA / JPL-Caltech) Back to slideshow navigation
  10. Cygnus takes flight

    Orbital Sciences Corp.'s Antares rocket rises from NASA's Wallops Flight Facility on Wallops Island, Va, on Jan. 9. The rocket sent Orbital's Cygnus cargo capsule on its first official resupply mission to the International Space Station. (Chris Perry / NASA) Back to slideshow navigation
  11. A long, long time ago...

    This long-exposure picture from the Hubble Space Telescope, released Jan. 8, is the deepest image ever made of any cluster of galaxies. The cluster known as Abell 2744 appears in the foreground. It contains several hundred galaxies as they looked 3.5 billion years ago. Abell 2744 acts as a gravitational lens to warp space, brightening and magnifying images of nearly 3,000 distant background galaxies. The more distant galaxies appear as they did more than 12 billion years ago, not long after the Big Bang. (NASA / NASA via AFP - Getty Images) Back to slideshow navigation
  12. Frosty halo

    Sun dogs are bright spots that appear in the sky around the sun when light is refracted through ice crystals in the atmosphere. These sun dogs appeared on Jan. 5 amid brutally cold temperatures along Highway 83, north of Bismarck, N.D. The temperature was about 22 degrees below zero Fahrenheit, with a 50-below-zero wind chill.

    Slideshow: The Year in Space (Brian Peterson / The Bismarck Tribune via AP) Back to slideshow navigation
  1. Editor's note:
    This image contains graphic content that some viewers may find disturbing.

    Click to view the image, or use the buttons above to navigate away.

  2. Editor's note:
    This image contains graphic content that some viewers may find disturbing.

    Click to view the image, or use the buttons above to navigate away.

  3. Editor's note:
    This image contains graphic content that some viewers may find disturbing.

    Click to view the image, or use the buttons above to navigate away.

  4. Editor's note:
    This image contains graphic content that some viewers may find disturbing.

    Click to view the image, or use the buttons above to navigate away.


Discussion comments


Most active discussions

  1. votes comments
  2. votes comments
  3. votes comments
  4. votes comments