Recreating Mars In VR Thanks To Curiosity Rover

Category: 
Recreating Mars In VR Thanks To Curiosity Rover
September 13, 2019

About Us

Hello, we are Gwénaël Caravaca and Stéphane Le Mouélic, planetary scientists at the Laboratory of Planetology and Geodynamics (LPG) in Nantes, France. Our lab is a co-shared research unit of the CNRS (French National Centre for the Scientific Research) and the universities of Nantes and Angers. It includes one hundred of research actors working on four themes: diversity of icy worlds, terrestrial planets, planet Earth and changing marine systems.

 

Our lab is engaged in several NASA and ESA space missions on Mars and the outer planets and their icy moons (e.g. Titan). More specifically, we are working on the surface geology of Mars, using both orbital data (Mars Reconnaissance Orbiter and Mars Express spacecrafts) and ground-based data from rovers such as the Mars Science Laboratory rover Curiosity, which landed in 2012.

 

We are currently involved in the European Union H2020 project PlanMap with the goal of reconstructing Digital Outcrop Models (DOM) of Martian outcrops, integrating them in Virtual Reality (VR), and developing dedicated VR tools for geological characterization and mapping of reconstructed outcrops just as if we were on the field on Earth.

 

We have been interested since 2013 in 3D modelling and Virtual Reality integration for the planetary data. Our goal is to recreate VR representations of geological outcrops (on Earth and Planets such as Mars) for scientific purposes and teaching.

 

We are not 3D artists by background, but great enthusiasts of these techniques and VR. Most of our work relies on photogrammetry to reconstruct accurate and photorealistic depictions of the outcrops.

 

Why Use 3D/VR in Planetary Geology?

Planetary geology is by definition a remote science, which is quite paradoxical when knowing the field-based nature of geological studies. In that, most data we are using to study other planets’ surfaces are remote sensing data, such as images gathered by both orbital probes and ground-based landers and rovers. However, most raw data received from these spacecrafts are 2D images at the origin. This mode of representation prevents a good characterization of the geological features, such as the shape, the orientation of the sedimentary figures, etc. These characteristics are key to decipher the meaning of the rocks in terms of paleoenvironments in ancient times. Reconstructing 3D representations of the outcrops is therefore an important visualization step.

 

To recreate the 3D shape of the objects, we use Structure-from-Motion photogrammetry, using very high quality pictures sent back to Earth by different instruments on board planetary probes such as Curiosity.

Photogrammetric model of a sulfate vein network encountered at the Garden City outcrop (Gale crater, Mars), reconstructed using image data from the MSL rover Curiosity’s navigational and remote sensing mast cameras.

 

Using DOMs, we are able to observe the true shape of the geological features, which helps us a lot in our work. We occasionally use 3D printing at various scales mostly for outreach purposes: the image shows an orbital model of the Gale crater (left), and a few decimetre-scale outcrop studied by Curiosity on the floor of Gale crater. 3D printing is a wonderful way to “touch” an object that is literally millions of miles away! And this is very instructive for us as geologists, but also for our students and visitors during outreach activities.

3D printed models of the Gale Crater (left, white) from HiRISE orbital digital elevation model, and of a sulfate vein network at the Garden City outcrop (right, colored printing, courtesy of IRAP) from the above photogrammetric digital outcrop model.

 

Finally, the next step is the integration of our geological meshes into Virtual Reality. While a 3D reconstruction of an outcrop is a great thing for us, the ability to view it at real scale as if we were there is truly a fascinating experience! This capacity allows us to observe and describe with an unprecedented precision the reconstructed Martian outcrops traversed by Curiosity to assess their geological record.

Gwénaël experiencing a little field trip in the Gale crater, on Mars… in VR.

 

Recreating Mars: from Raw Data to VR Geology of the Red Planet

Recreating DOMs of Martian outcrops is both quite easy, and very complex. Easy, because we rely mainly on photogrammetry, and in that, the use of PhotoScan professional software helps us a lot. It is a commonly used software in the geosciences community. Complex, because we are dealing with a literally out of this world dataset. While usual photogrammetry workflow requests preferentially steady conditions and fixed optical parameters, in the case of Curiosity, we can use up to 5 different imagers, with varying fields of view, focal lengths, and even colorization (full colour vs greyscale)! Moreover, pictures taken by the rover are not optimized for photogrammetric reconstruction, with the exception of the navigational cameras that work as a stereoscopic pair. Lighting conditions and orientation can also vary significantly, making the production of a high resolution photogrammetric DOM sometimes challenging on Mars. We have focused our efforts on the reconstruction of the Kimberley outcrop, which was traversed by Curiosity in 2014, using ~2000 different images.

 

If using different instruments can make it problematic to reconstruct the DOM, it can also be a real advantage in making multiscale models, allowing us to finely study specific structures. For example, this micro-model of the Windjana drill hole (1.6 cm in diameter) made using ~30 images taken by the MAHLI microscope mounted on the arm of Curiosity has a resolution so high that is allows us to observe very tiny details, such as the sub-millimetric burn marks left by the LIBS, a powerful laser beam used by the ChemCam instrument to analyse the chemical composition of the rocks.

Photogrammetric micro-model of the Windjana drill hole on the Kimberley outcrop, reconstructed using image data from the MAHLI microscope instrument of the MSL rover Curiosity.

 

Finally, all the different models belonging to the same outcrop (e.g., the DOM and the drill for Kimberley) can be scaled and integrated into a VR environment, at real scale and orientation. You can visit our Kimberley DOM in VR using embedded function on Sketchfab to walk on Mars as if you truly were there!

And Then?

Reconstructing models of Martian outcrops and integrating them into virtual reality is an amazing opportunity for us geologists to study and understand the rocky environments of the red planet and their conditions of formation. However, simple observation, as great it can be, is not enough. So, as part of the PlanMap project we are developing in collaboration with VR2Planets (a company specialized in VR for geosciences) several tools dedicated to field exploration and geological mapping in VR. The objective is to be able to log a section, make measurements, take notes, anything we would do if we were on the field here on Earth. We are convinced that the use of VR for remote exploration of Earth and planets will strongly grow in the future, opening the way to exciting new discoveries!

Example of VR fieldwork on the Kimberley outcrop, with measurement of the sedimentary beds.

 

We want to thank the CNRS (French National Centre for the Scientific Research) and European Union H2020 project PlanMap (project # 776276) for supporting this work.

Related articles

VRrOOm Wechat