Long-Distance 3D Reconstructions Using Photogrammetry with Curiosity’s ChemCam Remote Micro-Imager in Gale Crater (Mars)

Abstract

The Mars Science Laboratory rover Curiosity landed in Gale crater (Mars) in August 2012. It has since been studying the lower part of the 5 km-high sedimentary pile that composes Gale’s central mound, Aeolis Mons. To assess the sedimentary record, the MSL team mainly uses a suite of imagers onboard the rover, providing various pixel sizes and fields of view from close to long-range observations. For this latter, we notably use the Remote Micro Imager (RMI), a subsystem of the ChemCam instrument that acts as 700 mm-focal length telescope, providing the smallest angular pixel size of the set of cameras on the Remote Sensing Mast. The RMI allows observations of remote outcrops up to a few kilometers away from the rover. As retrieving 3D information is critical to characterize the structures of the sedimentary deposits, we describe in this work an experiment aiming at computing for the first time with RMI Digital Outcrop Models of these distant outcrops. We show that Structure-from-Motion photogrammetry can successfully be applied to suitable sets of individual RMI frames to reconstruct the 3D shape and relief of these distant outcrops. These results show that a dedicated set of observations can be envisaged to characterize the most interesting geological features surrounding the rover.

1. Introduction

Gale crater on Mars records a large section of sedimentary rocks, mainly represented by the 5-km-thick sequence of Mount Sharp (Aeolis Mons). It is home to the Mars Science Laboratory (MSL) rover Curiosity, whose mission is to help characterize the geological record using its suite of instruments, seeking out past presence of habitable conditions at a time when liquid water was abundant at the surface of the red planet [1,2]. Its mission therefore critically relies on the capacity of the team in deciphering the multi-scale geomorphological and sedimentological features present there to characterize the past environments.

To that end, we are taking advantage of all the various imaging instruments at our disposal aboard the rover, from the navigation cameras (Navcam) [3], to the color science imagers Mastcam [4,5] and to the Remote Micro Imager (RMI) subsystem of the ChemCam instrument [6,7]. This suite of instruments with nested fields of view allows us to obtain multi-scale close-range to remote observations. This multi-scale imaging capacity is particularly important to characterize the natural rock exposures on Mars. Due to traversability issues and resources/time optimization constraints, the rover cannot access specific zones despite their scientific interest. Their exploration therefore solely relies on orbital data complemented by remote imaging by the rover from several hundreds of meters up to several kilometers, using onboard cameras (Mastcam, RMI [8]).

This is notably the case of the lower sulfates unit of lower Mount Sharp, an area of great interest due to the previous orbital detection of sulfate-bearing strata [9,10]. Orbital data combined with rover-based observations revealed that the sulfate-bearing unit coincides with a major change of depositional environments followed by alternation of wet and dry conditions rather than a monotonous aridification during the Hesperian epoch [11]. However, this distant area has not yet been reached by the rover, so current observations were notably supported using long-distance imaging capabilities of the RMI sub-system of the ChemCam instrument. With a focal length of 700 mm, the RMI telescope complements the color Mastcam imagers with higher spatial resolution views of rocks cropping out several hundred meters up to a few kilometers away from the rover. This long-range imaging capacity is also useful in planning guidance of the rover. While these long-distance observations pointed at the sulfate-bearing unit revealed the presence of sub-metric-scale cross-stratifications, deflation surfaces, and diagenetic features, the resulting images lack depth due to the distance. Yet such 3D information can be critical in characterizing and understanding the sedimentary processes which generated these structures. 3D views enable a more quantitative interpretation of the sedimentary structures (e.g., scale, transport directions, etc.). We thus highlight here the need for a way to better investigate the various readable but distant outcrops using existing imagery produced in situ by Curiosity.

In this work, we propose for the first-time to create Digital Outcrop Models (DOM) of remote outcrops using the long-distance imagery from RMI’s telescope. We use Structure-from-Motion photogrammetry, a method usually performed on close-range geologic features observed with other imagers onboard the rover (MAHLI, Mastcam, Navcam [12]). To conduct this experiment, we apply photogrammetric treatment to specific sets of repeated long-distance RMI observations that we specifically targeted on the sulfate-bearing units. We therefore demonstrate the feasibility of this concept. The use of this technique to produce 3D models of remote geological exposures might have critical implication in helping to assess and to characterize the paleoenvironmental record studied with the Curiosity rover.

2. Localization and Area of Interest

Gale crater is a ~155 km-wide impact crater (Figure 1a) situated near the crustal dichotomy boundary of Mars, separating southern cratered terrain and northern smoother lowlands. The Curiosity rover of the MSL mission landed in August 2012 in the northern part of this crater (Figure 1a) in the Aeolis Palus region. Since then, the rover explored the geological record of Aeolis Palus and the lower flanks of the Mount Sharp (Aeolis Mons) along a ~25 km-long traverse. This sedimentary sequence revealed the existence of past lacustrine to fluvial depositional systems [1,2,13,14,15,16,17], but also some levels deposited in dryer aeolian settings [18]. The rover is currently starting the exploration of the lower sulfates unit of Mount Sharp (Figure 1b,c), a part of the sedimentary pile that has only been characterized by orbital data [9,10] or by remote long-distance imaging using the onboard cameras of the rover [8,11]. These observations show the presence of large-scale sedimentary structures, possibly associated with ancient aeolian depositional settings [11], but also the presence of younger yardang structures linked to wind abrasion [19]. Here, we focus on long-distance observations performed using RMI on the lower sulfate unit buttes above the Glen Torridon region where the rover was situated at the time the images were taken (cf. Figure 1b,c).

3. Data and Methods

3.1. Instruments and Products

The Curiosity rover is equipped with 17 cameras serving multiple purpose from determining trafficability along its traverse to multi-scale scientific observations for scientific remote sensing investigations; we mainly use the five cameras situated on the 2-m high Remote Sensing Mast (RSM) of the rover: the Navcam pair, the Mastcam pair, and ChemCam’s RMI (Figure 2a). The Navcam pair provide wide-angle stereo greyscale imagery (

Table 1. Main optical parameters of the Navcam, Mastcam and RMI imagers onboard Curiosity.

Instrument	Common Designation	Colorization	Resolution	Focal Length	References
Navigation Cameras	Navcam	Greyscale	1024 × 1024 px	14.67 mm	[3]
Mast Camera (left)	Mastcam left (M-34)	RGB	1600 × 1200 px	34 mm	[4,5]
Mast Camera (right)	Mastcam right (M-100)	RGB	1600 × 1200 px	99.9 mm	[4,5]
Remote Micro Imager	RMI	Grayscale	1024 × 1024 px	700 mm	[6,7]

; Figure 2b) that can be used to produce close to mid-range 3D models of the rover’s surroundings [3,12,20]. The Mastcam pair provide high-resolution color imagery (Table 1; Figure 2b) to allow detailed scientific observations [4,5]. These cameras can be operated to obtain stereo mosaics [21] or even 3D DOM [12] but, contrary to the Navcam pair, their different focal length leads to a restricted field of acquisition of the 3D information. Finally, the RMI is a sub-system of the ChemCam instrument. Its main role is to image the targets analyzed by the Laser-Induced Breakdown Spectrometry [6,22], allowing close inspection of the analyzed rock (Figure 2b), its texture, and the markings left by the laser shots. Whereas RMI was not originally designed to observe at long distance, its 700 mm focal length appeared to be useful when pointing the telescope at remote objects (Table 1; [7]). In that role, the RMI can produce highly detailed greyscale views of outcrops up to several kilometers away (a ~60 km-record observation having recently been achieved [23]). As the pixel size of the RMI is 3.7× smaller than that of the M-100 [7], this instrument provides a complementary imaging capacity of the distant geological features with a very high resolution, but with a reduced field of view. The white box in the color Mastcam mosaic of Figure 2c (upper panel) highlights the area imaged by a composite RMI mosaic made from two merged observations (15 × 3 and 10 × 3 mosaics, lower panel in Figure 2c). With RMI, it is possible to see more details, such as meter-scale cross-stratifications and individual blocks highlighted in the detail boxes at the bottom of Figure 2c, that were not visible on the Mastcam image. This capacity is critical to characterize structures from long-distances and/or of outcrops inaccessible for the rover. However, unlike the Navcam and Mastcam pairs, the RMI is not natively capable of generating stereo-images to provide a 3D view of the imaged outcrops.

3.2. Digital Outcrop Modelling

When studying a geological outcrop, 3D information can be critical to characterize and understand the conditions that lead to the formation and/or alteration of the outcrop. This is particularly true in the case of the detrital depositional record encountered in Gale crater and the identification of sedimentary induced structures. However, this 3D information is not readily accessible when studying Martian outcrops from orbit and/or from rover-derived data. To compensate for the impossibility to observe these features “in person”, one solution is to digitally recreate the 3D shape of the explored outcrops from the data gathered by the rover.

To compute these DOMs, we use Structure-from-Motion (SfM) photogrammetry. This technique relies on a suite of algorithms able to recreate an accurate 3D representation of an object from a set of overlapping 2D images [24]. SfM photogrammetry is particularly well suited for geological purposes and is commonly used both on Earth [25,26,27,28,29], and in the planetary community [30,31]. In the case of the Gale crater, several successful examples of DOM reconstruction to characterize the sedimentary record have already been performed using Navcam [20], MAHLI [32], Mastcam [18] or a combination of these instruments [12,33].

In this work, we apply this method to images taken by the RMI telescope, in order to reconstruct the 3D shape of remote and inaccessible outcrops situated several hundred meters away from the rover. For the SfM process, we are using the commercial Agisoft Metashape Professional software v.1.7.1 (Agisoft LLC, St. Petersburg, Russia) [34].

3.3. Targets on the Lower Sulfates Unit

For this imaging experiment, we focus on two remote outcrops belonging to the lower sulfates unit, higher up the section of Mount Sharp (cf. Figure 2c and Figure 3a)). Each outcrop was imaged twice with RMI mosaics, taken from different viewpoints while the rover was situated in the Glen Torridon region (cf. Figure 2b and Figure 3a). To cope with the important distance, and with the necessity to obtain two widely different points of view for the overlapping images, we use a several hundred meters “virtual baseline”, represented by successive position of the rover along its traverse (Figure 3a). This baseline introduces parallax effects which allow the subsequent retrieval of 3D information.

The first outcrop (LD_Sulfates_2962a on Figure 3a, illustrated in Figure 3b) was imaged on Sol (mission Martian day) 2947 and on Sol 2962 by mosaics labeled LD_Sulfates_2947b and LD_Sulfates_2962a, respectively (Table 1). The outcrop is situated between ~510 and ~650 m away from the rover, with a virtual baseline of ~200 m (Table 1, Figure 3a).

The second outcrop (LD_Sulfates_2962b on Figure 3a, illustrated in Figure 3c) was imaged on Sol 2962 and on Sol 2979 by mosaics labeled LD_Sulfates_2962b and LD_Sulfates_2979a, respectively (Table 1). This second outcrop is situated between ~760 and ~875 m away from the rover, with a virtual baseline of ~175 m (Table 1, Figure 3a).

4. Processing Chain

4.1. Imagery Dataset

Individual images taken by ChemCam’s RMI can be downloaded either from the JPL’s MSL RAW images library (https://mars.nasa.gov/msl/multimedia/raw-images/, accessed on 5 October 2021) or from the Planetary Data System (PDS; https://pds-geosciences.wustl.edu/missions/msl/, accessed on 5 October 2021). Images from the PDS are provided in the IMG exchange format, and therefore need to be converted beforehand using the small IMG2PNG command-line software utility (available at: http://bjj.mmedia.is/utils/img2png/, accessed on 5 October 2021). Preprocessed mosaics are also provided on the PDS in PNG format at https://pds-geosciences.wustl.edu/msl/msl-m-chemcam-libs-4_5-rdr-v1/mslccm_1xxx/extras/rmi_mosaics/, accessed on 5 October 2021. The RMI individual frames used for the photogrammetric treatment have been radiometrically corrected following the pipeline described in [7]. Individual frames of each selected mosaic are imported in PNG format within the SfM software (Figure 4a).

As the RMI optics provide a circular field of view recorded by a square detector matrix, a generic circular mask is applied within the photogrammetric software to all entry images (Figure 4a). This mask (red area in Figure 4b) indicates that this part of the image should not be considered during the alignment and reconstruction process. In some cases, a small amount of stray light may cause the outer border of the RMI image to appear blurred and/or overexposed (cf. Figure 4a). To prevent using these altered data, this outer part of the entry image is included within the applied mask (Figure 4b).

4.2. Camera Alignment and Sparse Cloud Generation

Once each individual frame of the RMI mosaics is imported and masked, a first alignment is computed to obtain a sparse point cloud and a 3D representation of the calculated location and orientation of the original frames in a 3D space (Figure 4c). This alignment results in a cloud composed by a few thousand points (usually 4000 to 10,000, the higher, the better). This sparse cloud gives an idea of the general shape of the distant outcrop but cannot be used for further characterization in this state. Higher density of tie-points is obtained in areas where a greater number of entry images are overlapping, and consequential to the degree of overlap.

Given the distant nature of the targeted outcrops and non-optimal points of view, results at this stage may vary. If not all of the images are aligned, one or more additional iteration(s) of the alignment process can be performed until a satisfactory number of those frames are aligned and a recognizable sparse cloud obtained. However, additional iterations may still fail to produce a good alignment, or even lead to an impossibility of aligning the images. In this latter case, changing the quality setting of the software (and thus the number of tie-points), or adding Ground Control Points (GCP) markers by hand on the entry images could help in aligning the frames and creating a sparse cloud. In some cases, some outcrops just cannot be recreated in 3D due to poor imaging and/or insufficient overlap. Further work is required to propose recommendations for future acquisitions.

4.3. Dense Cloud Generation

With entry images aligned, a dense cloud can be generated. This cloud, composed of up to several millions of points (e.g., ~2 million for mesh #1, Figure 4d), gives a very precise representation of the reconstructed area. The sharpness of the 3D reconstruction is once more linked to the degree of overlapping and overall quality of the entry images but is also a function of the quality setting in the software. In the case displayed in Figure 4, we used a high precision setting in Metashape. The resolution of this dense cloud is sufficient to fulfill our scientific needs and to perform accurate characterization of the shape and structures on the outcrop.

4.4. Mesh Generation and Texturing

The next step in the photogrammetric reconstruction process is the generation of a 3D mesh. While the dense point cloud could be exported and used to perform measurements, a mesh is computed for easier handling and visual characterization, especially when associated with a photorealistic texture, providing an accurate representation of the actual geologic object.

The mesh is composed of several thousand faces or polygons (e.g., 500,000 in mesh #1, Figure 4e) representing the surface relief of the reconstructed object. These polygons can be generated either by triangulation of the dense cloud points (with each point tied to neighbors by a vertex, creating the polygons), or by computation of a depth map. Choice of either of these methods is left to the user, depending on the capacities of the SfM software.

To obtain a more accurate and realistic representation, a texture is computed for the mesh using the original entry images. In this case, a texture made from eight tiles of 4096 × 4096 pixels have been generated and UV-mapped onto the 3D mesh to obtain a photorealistic representation of the actual distant outcrop (Figure 4f). With RMI, the texture is greyscale due to the entry images being so. However, further work is planned to test if RMI frames colorized with Mastcam images using pan-sharpening [7] could be used to produce a full color texture and therefore improve the final 3D product.

Before exporting the DOM, the mesh is manually “cleaned” from any exotic or floating polygons, most being notably on the outer margins of the model [12]. The mesh can therefore be exported for visualization and share, for example in the widespread Wavefront OBJ format, associated with textures in either JPG or PNG format and accompanying MTL library file.

5. Case Studies: Reconstruction of Distant Sulfates Unit DOMs

5.1. Mesh #1 (LD_Sulfates_2947b-2962a)

Mesh #1 is the 3D reconstruction of an area imaged by RMI targets labelled LD_Sulfates_2947b and LD_Sulfates_2962a (

Table 2. List of targets used to compute 3D DOM of remote outcrops and associated data.

Mesh	Target	SeqID	Type	Sol Taken	Dist. from Rover	Virtual Baseline
#1	LD_Sulfates_2947b	ccam04947	12 × 1	2947	~650 m	~200 m
	LD_Sulfates_2962a	ccam04962	16 × 2	2964	~510 m
#2	LD_Sulfates_2962b	ccam05962	14 × 3	2964	~875 m	~175 m
	LD_Sulfates_2979a	ccam04979	11 × 3	2979	~760 m

, Figure 5a), that represents a blocky outcrop of cross-stratified sandstones of the lower sulfate unit, which has likely undergone important diagenetic processes. The DOM was reconstructed using 41 aligned images (out of 43 in total, ~95%), taken from two points of view separated by a virtual baseline of approximately 200 m (Figure 3a, Table 2). The resulting mesh is computed from a 2,264,005 points dense cloud, into a 452,976 polygons mesh (Figure 5b). The blocky expression of the outcrop is well reconstructed by the 3D mesh, as observed in the shaded model in Figure 5b. The model is textured by eight tiles of 4096 × 4096 pixels (Figure 5c). Scaling of the resulting mesh has been done by visual comparison and selection of GCPs against scaled and calibrated original 2D RMI and Mastcam mosaics (their scale being determined as a function of the pixel size and distance to the outcrop [7,21]). We also used orbital HiRISE orthoimage to better constrain the regional coverage, resulting in a sub-metric magnitude of error, which is similar to what is observed on 2D mosaics. The resulting mesh can be visualized on the Sketchfab online public repository at: https://skfb.ly/opAwH, accessed on 5 October 2021.

Thanks to the 3D reconstruction of the model, it is possible to see that sub-metric cross-stratification is quite widespread on the outcrop, and not affected by the several deflation surfaces. This area also presents some structures in the form of arcuate lineations protruding out of some blocks (orange arrows in Figure 5d,e). Here, the 3D mesh representation allows observation of the spatial distribution of these features, and they seem to crosscut the apparent stratal pattern of the entire outcrop. These structures might be interpreted as mineral veins likely of diagenetic origin. A close observation of the topographic expression of the outcrop rendered in 3D allows the viewer to identify the possibility that a contact, hardly seen on the mosaic, continues farther to the left. Indeed, the contact observed at the base of the right part of the outcrop is easily identifiable at the base of the biggest blocks (right Figure 5a,d). However, its continuity is more difficult to assess in the rubblier part on the left. The 3D mesh, however, lets us observe a topographic “step” in this rubblier part (as seen in differential shading in Figure 5b), allowing the viewer to trace the continuity of this contact down to the limits of the model (red line in Figure 5d).

5.2. Mesh #2 (LD_Sulfates_2962b-2979)

Mesh #2 is a 3D reconstruction of an area imaged by RMI targets LD_Sulfates_2962b and LD_Sulfates_2979a (Table 2, Figure 6a), that represents a hummocky outcrop of large-scale cross-stratified sandstones of the lower sulfates unit, possibly remnant of a fossilized dune field. The DOM was reconstructed using 64 aligned images (out of 73 in total, ~88%). Most non-aligned images are out of the optimum overlap area (see Figure 6a) and therefore cannot possibly be aligned as they are singletons, that is images of an object taken from an only point of view. The images were taken from two points of view separated by a virtual baseline of approximately 175 m (Figure 3a, Table 2). The resulting mesh is computed from an 836,381 points dense cloud, into a 180,000 polygons mesh (Figure 6b). The overall hummocky expression of the outcrop is well reconstructed by the 3D mesh, as observed in the shaded model in Figure 6b, despite the lack of marked relief in this specific area. The model is textured by eight tiles of 4096 × 4096 pixels (Figure 6c). The same scaling as described in Section 5.1 is applied for this mesh. The resulting mesh can be visualized on the Sketchfab online public repository at: https://skfb.ly/opAwI, accessed on 5 October 2021.

By observing the model, it is possible to see that metric-scale cross-stratifications are visible all over the outcrop (Figure 6c). Most buttes are made of those cross-stratified sandstones, and the 3D model here helps in characterizing these structures and their spatial distribution from different points of view. For example, the green line-drawing on Figure 6d,e helps identify several sets of large meter-scale sets of through cross-stratifications with their top truncated, which is common in aeolian settings. The direction of propagation implied by trough cross-stratifications is difficult to assess in simple plan/section view, therefore the different points of view permitted by the 3D DOM helps in determining the transport direction which appears to be roughly north-eastwards (Figure 6). The 3D representation also allows identification of several erosional surfaces truncating the cross-stratified structures (red line in Figure 6e), helping to decipher local bounding surfaces within this probable aeolian context.

6. Conclusions and Perspectives

While critical to understanding the sedimentary dynamics, assessing the 3D shape of the sedimentary structures in the Gale crater might be difficult relying only on 2D imagery, even of high quality. SfM photogrammetry has been recently used to propose high-resolution DOM of several areas studied by Curiosity [12,18,32,33], but these only concern outcrops that have been “physically” traversed by the rover. Many remote areas that are not accessible or not yet reached by the rover remain poorly documented in terms of 3D information. This is notably the case for the lower sulfates unit, which has been only characterized so far using orbital data [9] or observed from remote points of view using the rover’s M-100 and RMI telescope [8,11]. Still, this 3D information could be critical in helping us characterize with higher precision the sedimentary record we are observing from a distance, with higher resolution and detail than in usual orbital digital elevation models.

In this work we therefore tested the possibility of recreating the 3D shape of distant outcrop by using long-distance images obtained with the RMI telescope of Curiosity. Even if RMI was not specifically designed for long distance stereo imaging, it is the most powerful imager in terms of pixel size. We demonstrate for the first time that individual frames from overlapping long-distance RMI mosaics can be used to successfully compute DOMs of these distant outcrops on two test cases using SfM photogrammetry (performed by the Metashape software). To maximize the variation in point of view, we are using a “virtual baseline” implied by successive stops of the rover along its traverse, separated by several hundred meters.

The resulting models computed from these RMI mosaics allow examination of the overall accurate 3D shape of the geologic objects that are out of the rover’s reach. With such 3D visualization enabling a multi-point of view observation, it is therefore possible to remotely assess several critical features of the outcrops such as the transport direction represented by cross-stratified sedimentary structures, or the spatial distribution of diagenetic features with an unprecedented accuracy. Coupled with existing imagery such as Mastcam (stereo-)mosaics and usual RMI mosaics, this new original yet powerful way to characterize the 3D geological record remotely observed from several hundred meters will help the science team to characterize, for example, the distant sulfates unit of Mount Sharp, a key-objective of the MSL mission. The possibility of computing a DOM from afar with RMI could also serve to help planning the upcoming exploration of this sulfates interval before arrival of the rover on site to maximize the science return and designate high-interest targets in advance. A further evolution of this technique would be to use RMI frames that have been previously colorized using Mastcam color mosaics [7]. Adding color information to the DOM would be an important improvement that would help better characterization and understanding of the recreated structures. Finally, considering the similarity between ChemCam’s RMI and the enhanced SuperCam’s RMI aboard the newly arrived on Mars Perseverance rover [35,36,37], we could foresee a similar use to compute long-distance DOM of structures observed from the floor of the Jezero crater.