Hyperspectral Principles from a Super Robot: Lessons in Perseverance

March 12, 2021

The Perseverance Rover delivers stunning images of a frozen Red Planet while showing the power of state-of-the-art optics to produce rapid, accurate and precise measurements. Explore how hyperspectral imaging and machine learning could enhance your analytical workflows.

At Opsyne, we closely monitor things at NASA to learn the latest and greatest in engineering and scientific instrumentation. Our instruments are somewhat instrumental to the effort to grow plants in space. Scientists aboard the ISS use our popular spectrometers to determine the cause-and-effect of plant growth in zero gravity.[1] This is helping humanity design solutions for sustainable human life on Mars. With much anticipation, we watched the landing on Thursday February 18th, 2021. The Perseverance Rover landed perfectly into heart of the Jezero Crater. The ancient lakebed may hold the secrets of past and future life on Mars.[2]

Space programs are widely celebrated for the power to inspire the imagination. They are also demonstrations of what is technologically possible and reliable. Missions to Mars, for instance, can help us better understand the principles of optics, machine learning and robotics. The constraints of space push human ingenuity and engineering to its limits. The exploding rocket engines and the cryptic parachute landing stole the headlines. The sensor array packed onto the Rover will help unlock the secrets and elucidate the parameters that may impact the future trajectory of humanity. Before astronauts can cultivate plants, we have to measure a few more things on Mars. These scientific instruments are the real stars of the show.

We live in a world awash in Big Data and Artificial Intelligence. Empirical science and analytical investigation have benefited from these powers for processing and interpreting data. Whether it is scientific investigation, material and content analysis or automation and robotics, it is sensors that enable and empower the ability to quantify observable characteristics in objects of interest. Typically, we aim to extract as much insight and detail as quickly, efficiency and accurately as possible. This is paramount in a medical clinic or food manufacturing, as examples. Missions to Mars clarify the benefits of extracting information with elegant precision.

There are three important lessons to download from Perseverance’s operating principles. First, is the power of artificial intelligence. The SuperCam, the set of cameras and lasers about the rover is capable of autonomous investigation.[3] This is called AEGIS or Autonomous Exploration for Gathering Increased Science. In autonomous target selection, the SuperCam identifies geological targets in images from the rover’s navigation cameras, choosing for itself targets that match the parameters specified by mission scientists, without Earth in the loop.[4]

Without AEGIS, Perseverance would sit idle much of the time. New image thumbnails are downloaded to the Jet Propulsion Laboratory (JPL) in Pasadena. The core team reviews the results and reverts to with further instructions to the rover. There were delays on both sides of the return journey. Limitations on interplanetary communications create operations latencies and slow progress in planetary surface missions. To avoid this, Perseverance uses its own brain. Images and instructions still bounce between rover and ground station via satellite. But the meantime is not lost. This is the full potential of sensing and artificial intelligence at the cutting-edge (of Space).[5]

Second, is the power of optical sensors. The sensor array on Perseverance can also be described as a remote astrobiological laboratory. Digging into the details of geological samples (aka. rocks) was once constrained to destructive analysis. In a laboratory, under a mass spectrometer, on Earth. Now, the chemical and mineralogical properties can be analyzed in-situ, on a site as far away as Mars, using spectral cameras. The hyperspectral PIXL or Planetary Instrument for X-ray Lithochemistry can scan the terrain and features of interest rapidly and non-invasively.[6] Hyperspectral describes a technology that is able to make reflectance intensity measurements over a continuous wavelength range while also doing imaging. Hyperspectral imaging or imaging spectroscopy performs spatially localised observations in two dimensions, typically. PIXL does this in X-rays, which is beyond the limit of human vision at high-energy wavelengths.

Third, though largely yet to be realised within the Perseverance mission is the relationship between number of sensors and the signal-to-noise ratio. This is a core principle at NASA. There is a powerful synergy that comes from fusion of the results from the different techniques. This data fusion can be applied within empirical investigations of all kinds. We expect the synergy experienced at both the human level, and with tools such as machine learning, will provide surprising discoveries that would not be possible without combining the results of the individual techniques. Hyperspectral imaging, for example, is one of many sensing and detection technologies that can enable more sensitive, specific and robust proactive continuous monitoring. The powerful sensor array shows us what is possible when you want to measure something quickly and remotely.

Think about this. A set of remote optical instruments can see into rocks. What could you do with such a suite of tools? Perhaps you want to map cancerous tissue in real-time during a surgery. Or determine the authenticity of a painting. The sensors and technological principles packed on the rover apply to a range of applications on Earth. We can look to the mission for inspiration. One of the great things of technology is that it allows us to revisit ways of working. Whatever your application, it is worth opening your mind: What do I not know when I would like to know it?  

Autonomous investigation. Local material analysis. Rapid assessment. The elucidation of stunning detail once reserved for destructive laboratory analysis. The principles behind these benefits are in full demonstration on Mars. Thanks to many technological breakthroughs in optic and image data processing, non-invasive real-time hyperspectral analysis is now available to researchers, operators and investigators of all stripes.

Often, we do not recognize the full potential of breakthrough technology. We make incremental improvements to the familiar baseline. But consider the principles and promises brought to life by the scientific instruments aboard Perseverance. Imagine if you could redesign your analytical workflows from scratch. If you could apply advanced sensors, artificial intelligence and robotics. Would you design them to look as they are at present? Probably not.

Here on Earth, we want to know as much as possible as fast as possible with the minimal effort. This means you want to scan your objects of interest and get an immediate answer. If you are a geologist, you want to find gold or quartz in your drill core samples and aerial surveys. If you are a physician or clinician, you want non-ionising screening and diagnostic tools to find disease. Or tools to enhance visibility into the surgical bed. If you are in agriculture and food, you want to know the quality and safety profile of every unit in your supply chain. From plants in the field to finished products packaged for distribution.

At Opsyne, we are engineering optical and machine learning solutions to provide the answers you need. Our hyperspectral imaging sensors, in concert with the full panoply of technologies discussed above, can scan non-invasively, rapidly and automatically. Perseverance is opening new horizons for humanity by saving time and uncovering more detail. Likewise, hyperspectral imaging can mitigate risk, unlock value and reduce costs. As we digitally transform, we can digitally evolve. If you are on Mars looking for signs of ancient life or on Earth looking for signs of disease, you want to look at a small scale and get detailed information about chemical elements present. Whatever your object of interest, contact us to learn how you can stay ahead of the curve with hyperspectral imaging solutions.

MastCam Panorama or the Jezero Crater on Planet Mars (NASA, 2021)


[1] Space plant science can also remind us of a fundamentally rewarding aspect of human life. Growing plants feels good. See our spectrometer in action here

[2] Functionally, dehydrated astronaut food could support human life on Mars. Though shelf-life still applies and reliance on deliveries from Earth adds an additional risk factor. NASA is prioritising plant growth on Mars. This reveals much about the value of living plants to our psychological well-being, wherever we are. If we can grow plants on Mars, we can probably grow humans there too.

[3] The SuperCam instrument suite provides the Mars 2020 rover, Perseverance, with a number of versatile remote-sensing techniques that can be used at long distance as well as within the robotic-arm workspace. These include laser-induced breakdown spectroscopy (LIBS), remote time-resolved Raman and luminescence spectroscopies, and visible and infrared (VISIR; separately referred to as VIS and IR) reflectance spectroscopy. A remote micro-imager (RMI) provides high-resolution colour context imaging, and a microphone can be used as a stand-alone tool for environmental studies or to determine physical properties of rocks and soils from shock waves of laser-produced plasmas. The SuperCam instrument responds to the Mars 2020 Science Definition Team’s (SDT) request to “combine mineralogy, texture, and ideally, chemistry observations at a scale comparable to that of the grains within rocks” By placing these capabilities within a single instrument, co-boresighted for remote sensing, the SuperCam meets all of its requirements and provides well-balanced scientific results covering elemental chemistry, mineralogy, and physical properties of the targets, allowing the best possible geological interpretations at a range of distances and size scales. The SuperCam can take photos and fire lasers at rocks to measure their chemical and mineralogical content. Awesome.

[4] The surface of Mars is covered by dust, which limits the ability of passive remote-sensing devices such as hyperspectral scanners to make observations of the mineralogy or chemistry of the underlying rocks. Geoscientists and exploration geologists regularly employ hyperspectral cameras now to scan basis with satellites and airplanes, open pit mines with UAVs, drill core samples with bench-top units to elucidate the relative concentration of minerals to support more precisely targeted efforts. Alas, here on Earth we are not as encumbered by the problems of dust. The SuperCam instrument overcomes this challenge by using a laser to help ablate the dust.

[5] Because of Curiosity’s relative lack of remote mineral-identification capabilities, the Mars 2020 Science Definition Team mandated that the next NASA rover should possess the ability to observe mineral compositions by remote sensing. The SuperCam instrument is a response to this requirement for remote mineralogy while preserving the ability to remove dust prior to making observations of nearby targets, and providing the same or better chemistry and high-resolution imaging as ChemCam. Read more here and here

[6] Read more about PIXL here and here

Book a consultation to learn more

We can walk you through how hyperspectral imaging can improve your workflows.