Squeezing Data from a Rock: Machine Learning for Martian Science

Abstract

Data analysis methods have scarcely kept pace with the rapid increase in Earth observations, spurring the development of novel algorithms, storage methods, and computational techniques. For scientists interested in Mars, the problem is always the same: there is simultaneously never enough of the right data and an overwhelming amount of data in total. Finding sufficient data needles in a haystack to test a hypothesis requires hours of manual data screening, and more needles and hay are added constantly. To date, the vast majority of Martian research has been focused on either one-off local/regional studies or on hugely time-consuming manual global studies. Machine learning in its numerous forms can be helpful for future such work. Machine learning has the potential to help map and classify a large variety of both features and properties on the surface of Mars and to aid in the planning and execution of future missions. Here, we outline the current extent of machine learning as applied to Mars, summarize why machine learning should be an important tool for planetary geomorphology in particular, and suggest numerous research avenues and funding priorities for future efforts. We conclude that: (1) moving toward methods that require less human input (i.e., self- or semi-supervised) is an important paradigm shift for Martian applications, (2) new robust methods using generative adversarial networks to generate synthetic high-resolution digital terrain models represent an exciting new avenue for Martian geomorphologists, (3) more effort and money must be directed toward developing standardized datasets and benchmark tests, and (4) the community needs a large-scale, generalized, and programmatically accessible geographic information system (GIS).

1. Introduction

The recent increase in the development and deployment of machine learning (ML) for Earth observation has led to a wealth of new data and insights about our planet. These ML methods—enabled by more numerous and advanced satellites, increasingly sophisticated detection algorithms, and faster computation resources—are required since the expansion of data collected on Earth has rapidly outpaced the speed of manual analysis. As more and more data are collected, automation of time-consuming tasks by algorithms has become more important. Examples of these tasks include sensor calibration, image preprocessing, image segmentation, feature extraction, and classification of features and land cover.

Given the difficulty and expense of imaging Mars, there has not been the same influx of new instruments nor the return of data at rates and quantities equal to those from Earth-orbiting satellites. That said, there are currently more instruments collecting Martian data than there have ever been in the past: of the 18 successful orbital missions around Mars, eight are currently active, and of the seven successful lander missions on Mars, three are still active. Furthermore, there are already large archives of Martian imagery data available in the National Aeronautics and Space Administration’s (NASA’s) Planetary Data System (PDS) [1,2] and the European Space Agency’s (ESA’s) Planetary Science Archive (PSA) [3,4]. While the quantity of the data is smaller, the Mars science community is relatively small given the lack of corporate profit incentive to analyze Mars data, leading to an outpacing of manual analysis.

Several interesting and unique problems make the application of ML approaches to Martian data an area of potentially significant value. The autonomy of rovers on the Martian surface has long been an area of great interest due to (1) the long time delay between signals and instructions sent between Earth and Mars, (2) an asynchronistic overlap between the Earth and Mars days, and (3) the desire for rovers to make mission progress even when they are not executing specific instructions from Earth. In recent years, robotic autonomy has progressed from overcoming driving navigation problems to including tasks such as selecting sample targets [5,6,7] and even guiding landers to safe landing areas [8,9,10,11,12,13,14,15]. While efficiently managing rover time is one area of focus, rover missions to Mars are expensive and require years of planning, but can only study an infinitesimally small fraction of the Martian surface. With these inherent limitations, there is a strong incentive to both (1) use satellite data to effectively identify locations of the greatest interest and suitability for a rover mission, and (2) use remote sensing wherever possible to learn as much as possible without the need for in situ rover observations [16,17]. Thus, the need to leverage the archives of Martian observations is an increasingly important focus for Mars researchers, and planetary scientists more broadly: “Mars is a uniquely enabling study target for investigations of surface processes active on other planetary bodies—both for extending terrestrial based process models and, in some areas, for serving as a superior comparative planetology basis over Earth” [16]. This is especially important given the range of major outstanding questions that ML could potentially help answer [18,19].

Scope and Audience

We do not intend this paper to be a review of methods currently employed in Earth observation as there are already many excellent review papers that cover that topic [20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36]. Similarly, rover navigation will not be our focus here as the topic has been covered elsewhere [37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54]. Instead, we aim to provide a novel summary of the state of ML for Mars observation, and to help further Martian research by highlighting under-explored domains. To that end, we set out with four goals: (1) to help to connect the planetary science and ML communities, (2) to collect and categorize the current state-of-the-art research methods used in Martian studies, (3) to offer a perspective on the future of automated methods, and (4) to identify opportunities for further studies. Our intended audience is twofold: (1) planetary scientists seeking to explore how ML could improve their research, or make their experiments more efficient, and (2) ML experts who can help collect, organize, and analyze data for Martian studies.

For the purposes of this paper, we use a broad definition of ML: a subdomain of artificial intelligence in which algorithms self-optimize to improve their results. While there are many further subdomains within the discipline, such as deep learning, active learning, computer vision, and natural language processing, here, we will use the umbrella term ML to capture the full breadth of these approaches as applied to Mars research.

2. Machine Learning Studies on Mars

Generally, there are two methods for structuring a description of different types of ML: learning-focused frameworks that center on the functioning of the algorithm in question, or task-focused frameworks that focus on the algorithm’s product rather than its method. For this paper, we have opted to use the learning-focused framework, with subsections for each type of task which contain research in that domain.

2.1. Supervised Learning

Supervised learning requires labeled data as both the input and target when fed into the ML algorithm; algorithms are fit to these data, and a trained model predicts targets for unknown inputs [55,56,57,58,59]. The two most common applications of supervised learning are classification and regression problems. In classification problems, the input data are mapped onto pre-defined categories, while in regression problems, the input data are mapped to numerical outputs. For example, a supervised classification network could be trained on images of cats and dogs, with each of the images having a corresponding label indicating whether the image contains a cat or a dog. The network could then be given an unlabeled image of a cat and—if trained well—identify that the image contains a cat. In contrast, a supervised regression network could be trained on several years worth of daily weather data. The model could then predict the temperature for any given day of the year.

2.1.1. Classification

Transfer Learning

Transfer learning is a popular method for decreasing the time required to train a model on a new set of data [60,61,62,63,64]. Transfer learning is simply the application of pre-learned weights or filters to a new dataset. In essence, a model trained for one task is applied to another, with the assumption that the two tasks are appropriately related. In image classification, this is intuitive: object recognition across domains typically requires pattern extraction, edge detection, and other standard image-processing steps. A model trained to recognize cats in images could quickly be adapted to recognize dogs without having to reset the model’s weights and begin training from scratch. Many complex models have been published that have been pre-trained on large datasets of text or images [65,66,67,68,69,70,71,72,73]. These off-the-shelf models allow researchers to take advantage of the complexity and depth of the models while reducing the time required to develop and train a model. The application of transfer learning is limited in Martian research, but its use is highlighted in the following sections.

Classification Applications on Mars

At present, supervised classification models are the most common ML applied to Mars. Classification assigns inputs to an output class, and many classification algorithms also include locators that allow the network to both classify the content of an image and locate the extent of the target, either by drawing a bounding box, masking, or segmenting the target [67,74,75,76]. The development of feature-pyramid networks [66,77,78,79,80,81] has been very powerful as they allow networks to be scale-invariant; the network can recognize an object such as a basketball whether it fills 90% of an image or just 5%. These network features are especially useful for Martian studies since many targets of interest are spatially distant from each other, and manually locating them is very labor-intensive.

By far the most common application of any ML method to Mars has been to segment and count craters on the surface. Crater counting is a well-established method for estimating the age of planetary surfaces [82,83,84,85,86,87], first applied to the Moon, and in more recent years, to Mars. There are many studies devoted to crater counting on Mars [88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105]. These papers can be divided in two ways: by the minimum crater detection size, or by the methodology used to extract craters. The most common datasets used in these crater counting studies are global in coverage and relatively coarse in resolution. Crater counting requires both a significant number of craters across a range of diameters and a representative sample of the surface, which explains why the existing emphasis has been on global datasets. Furthermore, smaller craters (up to 100 m in diameter) are less indicative of the age of the surface than larger craters, while simultaneously being more difficult to differentiate from secondary craters caused by a larger primary impact [86,106,107,108]. In some recent work, efforts have even been made to classify craters as “preexisting” or “new,” to identify craters that have formed in the last ~15 years [105].

The Mars Odyssey Thermal Emission Imaging System (THEMIS) daytime infrared (DIR) dataset, with a resolution of ~100 m per pixel [109], is the most comprehensive and commonly used imagery dataset [91,92,93,96,98], while the Mars Global Surveyor Mars Orbiter Laser Altimeter (MOLA) [110], with a resolution of 463 m/pixel, is the most commonly used digital terrain model (DTM) dataset [94,95,111,112]. Recently, data from MOLA and the Mars Express High-Resolution Stereo Camera (HRSC) have been blended to create a global DTM at a ~200 m/pixel resolution [113], which has begun to be used in studies [89]. Other recent work has explored the potential for using even higher-resolution data from the Mars Reconnaissance Orbiter (MRO) High Resolution Imaging Science Experiment (HiRISE) (~0.5 m/pixel) or Context Camera (CTX) sensors (~6 m/pixels) [101], with transfer learning to adapt pretrained crater locators to new datasets. The two most common methods for extracting craters can be simplified into: (1) looking for the characteristic circular crater patterns in imagery and fitting a diameter to each crater, or (2) using elevation data to look for circular depressions and fitting a diameter to each crater. Both methods have proved successful, helping to constrain the ages of large-scale Martian provinces.

The next most frequently used application of ML for Mars is segmenting features in Mars rover imagery. Many studies focus on segmenting or classifying rocks and surfaces nominally, which works toward the goal of optimizing traversability [114,115,116,117,118,119,120,121,122], while others further group terrain classes as safe or unsafe for rovers [123]. Beyond this, work has focused on segmenting dust storms using MOC imagery and neural networks [124,125]. Some studies have extracted volcanic features using elevation data [126] or HiRISE imagery [127]. Gravity-driven processes are also an area of focus, including recurring slope lineae [128,129,130] and rockfalls [131,132]. Aeolian features have been the subject of several studies using HiRISE or CTX data, including transverse aeolian ridges (TARs) [127,133] and dunes [134,135] (two of these projects leveraged transfer learning with pretrained versions of RetinaNet [133] and AlexNet [135]). Besides distinct features, efforts have also been made to develop general terrain classifiers for the surface of Mars with broad categories [119,133,136,137]. Looking below the surface, data from the MRO shallow radar (SHARAD) sensor have been used to train a neural network to locate discontinuities in the subsurface [138].

2.1.2. Regression

Unlike the discrete categorization of classification models, regression models connect inputs to continuous outputs. Supervised regressions have been used in studies of Mars, but to a much more limited extent than classification models.

One study [139] used a supervised regression model to predict the true weight percentages of SiO₂, Na₂O, and K₂O as derived from sample data from the laser-induced breakdown spectroscopy (LIBS) ChemCam instrument [140,141,142,143]. The samples collected from LIBS did not represent the true overall composition of the sampled rock so the model learned to correct for under- and oversampling caused by the data collection method. The corrected weight percentages were then used in a supervised classifier to predict the total alkali-silica classification of the sampled rocks. Another study used supervised regression models to hindcast the conditions of Mars’ thermal evolution using current observations and a large number of simulations [144,145]. This model approach modified parameters representing past conditions, which were then fed into the simulations. The results were compared to current observations, with the model attempting to modify past parameters to better fit the modern observations. By testing an extremely large number of simulations and parameter combinations, the past parameter space that could produce the current conditions could thus be indicated. A final study trained a model to predict five solar wind variables from three spacecraft-measured parameters using data from the Mars Atmosphere and Volatile EvolutioN (MAVEN) spacecraft [146]. In this study, the model was trained on a large amount of past data to learn the relationships between the five dependent and three measured variables.

2.2. Unsupervised Learning

Unsupervised learning uses unlabeled data to generate inferred—rather than explicitly defined—classifications or mappings of the input data, which can help provide insights into the data [147]. Perhaps the most common form of unsupervised learning comprises clustering algorithms such as K-nearest neighbors [148,149,150,151], self-organizing maps [152,153,154,155], and mixture models [156,157,158,159,160,161,162,163]. Other common applications are in anomaly detection, where a model can identify a data point that is significantly different from the rest [162,164,165,166,167].

Unsupervised learning is much less common in Martian research. This is likely due to the tradeoff in unsupervised learning: while it is easier to generate a training dataset (no manual labeling is required) and the algorithms are generally very fast, it is more challenging to align a research goal with the output of an unsupervised algorithm. For example, when an unsupervised algorithm is asked to create a number of classes, it may split or group the data in unexpected or unclear ways. An unsupervised algorithm could naively conclude that a Red Delicious apple is more similar to a tomato than to a green Granny Smith apple. Only with a priori information could the algorithm connect the two types of apple.

Unsupervised learning is used in popular dimensionality-reduction methods such as principal component analysis [168,169,170,171] or autoencoders—a type of neural network that can compress input data into a smaller but still representative version of the dataset [172,173,174,175]. These unsupervised methods are frequently used in preprocessing pipelines of supervised methods as they can reduce the computation time by reducing the number of parameters in the data and also help mitigate “the curse of dimensionality” [176,177,178,179,180].

2.2.1. Clustering

Most clustering on Mars is carried out using spectral applications [181,182,183]. In early work [181], unsupervised clustering was used to verify that the spectral responses of data formed two distinct populations, i.e., that the two different sampling areas had different mineral compositions. In subsequent work [182,183], a two-layer Gaussian mixture model was used to decompose spectral responses into the probable component minerals, resulting in the likely detection of rare mineral phases. A recent application [184] of clustering grouped similar rover MastCam [185,186,187] images.

2.2.2. Anomaly Detection

To date, four published papers [43,136,188,189] and one recent preliminary blog post [190] have focused on anomaly detection on the Martian surface. Some of the published work [43,136] does not use typical “anomaly detection” ontology, and instead, is similar to the general supervised surface classification outlined above. However, one study used anomaly-based detection to identify time-series changes in Martian imagery [188], and another used an anomaly-detection algorithm to discover a water-ice annulus around the north polar region of Mars [189]. The blog post was focused on detecting human-caused disturbances on the surface of Mars such as from rovers or lander impacts. It used a DCNN to locate six examples of anomaly disturbances among 10,000 sample images [43].

2.2.3. Dimensionality Reduction

An interesting application of dimensionality reduction is for testing whether manually extracted features contain meaningful, but disparate, information. A recent paper [191] tested different properties of landslides on Mars such as the slope angle, slide length, and thermal inertia, and similar work has been carried out on Earth-based DTMs, examining the characteristic properties of cirques [192]. Besides this, other Mars researchers have used principal component analysis and sensitivity analyses to identify the most influential features in their data as a preprocessing step, including identifying issues with data transmissions from Mars [193], segmenting Martian dust storms in satellite imagery [124,125], and a rover-based terrain classifier [116]. In the MastCam paper noted above [184], an autoencoder was used to project the rover images into latent space where a k-means classifier could be applied.

2.3. Semi-Supervised Learning

Semi-supervised learning is a hybrid form where both labeled and unlabeled data are used as inputs [194,195,196]. Typically, there are fewer labeled examples than unlabeled ones. At its core, semi-supervised learning includes human expertise in labeled data-processing without requiring that time be spent labeling every image or data point. Semi-supervised algorithms are stable and efficient, but sometimes suffer from lower accuracies than other methods [197,198].

Classification

At present, semi-supervised learning is not widely used to study Mars. Two published studies have applied this method [199,200], both using contrastive learning [201,202] and Curiosity rover imagery. Contrastive learning uses a unique optimization function to maximize the agreement between two disparate views of the same target, for example, an image of an entire dog and a cropped image of just a dog’s head [201,202]. Wang et al. [199] found that supplementing their labeled training data (Mars Science Laboratory (MSL) Surface Dataset [203]) with 34,000 unlabeled images allowed their image classification model to outperform a baseline model [135] trained on just the labeled data, with nearly 30% greater accuracy across 24 classes. A more recent paper [200] also used contrastive learning to segment MSL imagery. In this study, 16,000 images segmented in six classes were fed into a network that used a semi-supervised contrastive pretraining method. The contrastive pretraining approach boosted accuracy scores by 5–7% overall, but greatly improved the recall score by up to 70% for the smallest minority class that comprised only 0.03% of the dataset’s pixels. Large improvements on rare classes are difficult to achieve, so these results are very promising.

2.4. Self-Supervised Learning

Self-supervised learning is intermediate between supervised and unsupervised learning. In self-supervised learning, unlabeled data are the model input while the outputs are typically classified in some way. For example, natural learning processing (NLP) models may be fed huge amounts of text with random portions masked out to train a network to derive the content and structure of a language from context [204,205,206,207]. Perhaps the most well-known application of self-supervised learning in image processing is in generative adversarial networks (GANs) [208,209,210], which have become increasingly powerful in recent years with the development of user-friendly off-the-shelf networks such as StyleGAN [211,212]. These systems use two networks to generate synthetic data within the latent space of the training data: one network learns to generate synthetic data, while the other network learns to discriminate between real and synthetic data. This adversarial learning process improves both networks simultaneously. Common applications of GANs include the generation of photo-real artificial human faces [211,212,213,214,215], images based on input text descriptions [216,217,218,219,220], and “super-resolution” or upscaled images in many domains [221,222,223,224,225,226,227,228].

The training of these upscaling models is relatively simple: a downsampled image is used as the input and a higher-resolution image is used as the target. These image pairs are used to train the model to infer the upscaling needed to generate the target resolution. The trained network can then be given a base-resolution image, and the model will generate “missing” data to increase the input image’s resolution. This method can also be used to add color to panchromatic imagery or to fuse data from different sources [229,230].

Two separate projects have focused on generating super-resolution imagery of Mars, one focused on satellite imagery [231] and another on rover imagery [232]. In the satellite project, the network was trained on pairs of downsampled and corresponding full-resolution HiRISE images; in essence, the network learned to recreate higher-resolution HiRISE images from the lower-resolution alternatives. The model was then applied to the Color and Stereo Surface Imaging System (CaSSIS) instrument data (via transfer learning) to generate higher-resolution multi-spectral optical imagery with an approximately three times greater resolution. The rover work was similar in that pairs of degraded and original images were fed in, and the model then learned to generate higher-resolution surface images, which in theory, could be useful for further studies.

Another GAN, dubbed MADNet, has been developed to generate high-resolution digital terrain models (DTMs) from imagery [233,234,235]. The most recent version, MADNet 2.0 [233], can generate accurate DTMs from just HiRISE imagery with a 16 to 100 times greater resolution than dedicated DTM instruments can produce. The network was compiled with pairs of downsampled HiRISE images (input) and their corresponding DTMs derived from stereo HiRISE coverage (target). The model was then trained to recreate the full-resolution DTMs from the lower-resolution images. The final model could generate a native-resolution DTM for any HiRISE image.

One self-supervised project that did not use GANs was for a rover-based self-supervising terrain classifier that combined imagery and wheel vibration data to classify terrain as safe or unsafe based on the rover’s experience in the environment [236]. However, due to technical limitations, the success of the project was limited (precision scores of ~70%, 10–15% worse than a supervised classifier), and it is difficult to compare these early attempts to modern ML methods with more complex architectures and vastly greater computational power.

3. The State and Future of Machine Learning on Mars

3.1. Why Use Machine Learning at All?

When evaluating the possibilities for ML applications on Mars, it is important to first note a number of important differences between terrestrial geomorphic applications of ML and those found on Mars. These differences may in part explain why ML, using current approaches, may be difficult to apply to the range of Martian surface-feature identification issues, as well as to applications associated with spacecraft landing and traversing the Martian surface. Owing to the issues that will be presented in this section, geomorphic predictions from ML may have “low resolution”, and thus may offer weaker results than from ground-based sampling or mapping [237]. That said, many exciting potential applications may produce new insights into Martian geomorphic and climatic evolution, generating valuable information for future exploration work.

First—and probably most important—among the constraints on ML applications is how researchers are dealing with a surface and subsurface that cannot be exhaustively sampled in situ. There is, unlike terrestrial features, no true “ground truth” for Martian landforms. We begin with assumptions about what the features are without physically examining those features to determine the source processes. Therefore, many interpretations start with a level of uncertainty that is quite different from when studying terrestrial environments where scientists can physically examine the surface themselves. On Earth, human-derived samples or observations are often complex and ambiguous, and there can be problems with spatial and temporal extrapolation [238]. These issues are exacerbated on Mars, where the classification of features on the Martian surface tends to be poorly constrained relative to terrestrial research. Plus, in many, if not most, instances on Mars, geomorphic processes cannot be directly observed, and remnant features may reflect processes no longer active [239].

Second, physical processes on the Martian surface, like those on other planetary bodies, are controlled by basic physics and chemistry. Therefore, ultimately, the manifestation of Martian surface features should be predictable if the underlying physics and chemistry are well-known, meaning training datasets that reflect these physical properties can be developed. Balaji when discussing the use of ML in the physical sciences, noted that in recent research, it is assumed that everything can be derived from the first principles of classical physics [240]. Along these lines, Church noted that the intention of modern geomorphologists in seeking to understand landscapes is to “interpret features in terms of observables by the application of Newtonian mechanics.” [241]. However, currently on Mars, this is not directly possible, except at a few sites where rovers could sample in situ. Remotely sensed physical and chemical details of the Martian surface, while available and of value, are somewhat indeterminate and fuzzy. Learning algorithms, while capable of producing interesting results from these training datasets, must respect physical constraints, even if they are not present in the data. This may be one reason why unsupervised learning, relative to other types of ML, has been scarcely applied to Martian features. Unsupervised learning uses unlabeled data to generate inferred—rather than explicitly defined—classifications or mapping of the input data, which can help provide insights into the data. Unsupervised learning is, therefore, both a blessing, in that it may identify groupings of objects that might not occur to an observer with terrestrial analogs as a guide, and a curse, in that these groups may be meaningless or uninterpretable.

Third, even at sites where rovers could sample, there may be significant differences in the interpretations of geomorphic and geologic processes and resultant landforms. For example, studies of the source processes of deposits sampled by the Curiosity rover in the Gale crater produced conflicting hypotheses—lacustrine versus aeolian—about the formation of these sedimentary deposits [242,243,244,245]. In many cases, different processes can produce similarly appearing deposits that may be difficult, if not impossible, to differentiate. This equifinality, where the same end state may be achieved via many different paths or trajectories [238,246], produces great uncertainty. For example, rock glaciers on the north side of Mount Sharp in the Gale crater [247], while having a similar appearance to rock glaciers on Earth, could also be produced by subtle differences in driving processes (mudflow, lahar), leading to similarly appearing forms. As geomorphic researchers have noted [237,239,248], interpretation in geomorphology is inherently indeterminate, with “combinations of attributes, relationships, processes, drivers, legacy effects and sequences of events creating contingent circumstances that fashion complex arrays of responses” [237]. As such, considerable error can result from applying generalizations to specific cases, and a lack of objectivity may severely affect training dataset creation and subsequent interpretation [237].

Fourth, many Martian features have no known analog on Earth. Transverse aeolian ridges (TARs) are an excellent example where a feature that appears much like a transverse dune at a large scale, and a ripple at a small scale, is actually neither and without a clear analog on Earth [249,250,251,252,253,254,255,256,257,258,259,260]. This makes conceptualization of ML approaches difficult and interpretation of results unclear. As [240] noted, “one can have understanding of the system without the ability to predict; one can have skillful predictions innocent of any understanding. One can have a library of training data, and learn the trajectory of the system from that, at least in some approximate or probabilistic sense. If no analogue exists in the training data, no prediction is possible.” Held noted that to pass from simulation to understanding, ML-based modeling must learn from data comprising not just patterns but also simpler models [261]. In many cases, the requirement for models and associated training sets may necessitate the production of training sets with attributes that are more appropriate than simple bounding-boxed examples.

Finally, most landforms are typically not discrete objects and lack universally applicable definitions, with such a definition relying on an “I know it when I see it” type of expertise. Geomorphic features typically have fuzzy ill-defined boundaries that grade into other features (e.g., hill-to-valley). For example, cirques normally found in mountainous environments are generally defined as a semicircular feature with a steep head and sidewalls, an over-deepened central area, and a low sill that defines the lower edge of the feature. However, the poorly defined lower edge often grades into a talus slope, and the upper edges may take the form of a sharp arête or a poorly defined break in the slope [192]. Even when the prescribed features are present, the “edges” of the feature may be diffuse/transitional. Measures of cirque size and form can thus vary from observer to observer [192,262]. Nonetheless, even with these uncertainties, a relative novice mountain-goer can recognize a cirque. As such, while robust and verified algorithms can objectively and repeatably identify crisply defined objects, these identifications are currently lacking in the geomorphic realm.

3.2. Future Machine Learning Studies on Mars

It is important to note that until recently, only relatively low-resolution data were available for analysis of the Martian surface. This resulted in the creation of poorly constrained training datasets ill-suited to analyzing all but the largest surface features. Now though, ML approaches can be used to optimize data extraction from relatively new high-resolution sources, to explore properties of the surface and its change over time that were until relatively recently of sub-pixel resolution. As [16] suggested, “To acquire observational data related to mapping and timing of activity, continued high-resolution orbital imagery is key.” The following sections of this article highlight specific areas of research that could benefit from applying existing methods to the higher-resolution imagery.

3.2.1. Mission Planning and Landing

Mission planning considerations relate to fuel management, communications stability, guidance quality, the selection of geologically interesting landing sites, and landing site safety. ML has been increasingly applied to these issues, specifically in detecting telemetry issues [193,263,264,265], managing fuel/power [266,267], developing guidance systems [268,269,270,271], determining safe landing conditions (discussed in detail below), and selecting landing sites (discussed elsewhere in this review), among others [272].

Landing a spacecraft on a plain cluttered with boulders presents multiple problems, including threats to the lander safety and rover trafficability for subsequent scientific exploration [273,274,275,276,277,278,279,280,281,282]. This topic has been especially well-studied for Mars, where boulder fields have been a serious concern for landers. While multiple successful landings on Mars have taken place, an entire mission can be terminated due to rocks at the landing site. The challenge in overcoming this issue includes characterizing the boulder distribution (ideally, to identify areas with smaller boulder densities) and surface roughness during pre-mission site selection [275,279,283,284,285,286,287], and then, again during the entry, descent, and landing stages of a mission [280,286]. Enhanced recognition of boulders and other surface roughness obstacles is critical for mission success and subsequent exploration. Boulder fields are best resolved in HiRISE imagery, and the previous works cited in this section all used HiRISE data. In the future, more advanced supervised classifiers could be developed to help solve the problem of recognition, and data fusion between greater sensor types, such as those producing thermal inertia and elevation data, could be used as an additional feature to derive lander safety maps.

3.2.2. Boulder Sampling Strategies

Understanding the distribution of boulders and their properties is of significant importance since the boulder size-frequency distribution is critical for understanding the surficial geology, sedimentology, and erosional/depositional processes on Mars. The Perseverance rover landed in the Jezero crater in 2020 and will soon be sampling the deltaic deposits within the crater for sample return and evaluation of potential past life on Mars. A sampling strategy [288,289] was designed to collect samples that are both readily accessible in situ, as well as materials that may originate from boulder falls from the inaccessible upper surface of the delta, which contains a collection of boulder debris from the delta drainage catchment. [290] developed a customizable algorithm for boulder extraction under different brightness and contrast levels and illumination angles that can be used for sample selection, and various other methods for extracting boulders also exist [279,280,286]. Similar preplanning studies are necessary for future missions to Mars with other objectives and criteria [291,292,293]. In these studies, anomaly detection and supervised classification algorithms could be used to identify, locate, and then map boulders on the Martian surface using HiRISE data, to produce maps that could then be used in geological and geographic analyses.

3.2.3. Identify Shorelines and Deltas

Extracting high-water marks and deltas related to an ancient global ocean [294,295,296] and paleolake shorelines [297,298] is critical to understanding Mars’ climatic history. While the largest of these features are easily discernable in satellite imagery, most are typically subtle features on the landscape, subject to interpretation issues. A supervised classification algorithm could be trained to locate deltas and/or terraces in HiRISE imagery, but terraces would likely be easier to identify in high-resolution DTMs.

3.2.4. Map Unusual Aeolian Features

Transverse aeolian ridges (TARs) are wind-created landforms unique to Mars [254,255,257,258,259,299,300,301,302] that may provide insights into the past Martian climate [303], or more importantly, the nature of aeolian features in general across planetary bodies [250,252,304,305,306]. TARs are sparse but common, whereby TAR fields are small in scale, but frequently occur across the surface of Mars. Manually locating all of these features globally is nearly impossible, although some representative efforts have been made [256,301]. A preliminary study using a supervised classification algorithm identified TARs with high precision [133] in HiRISE imagery, but a global survey is yet to be completed. Information as basic as the density of TAR fields across Mars could provide greatly important insights into the role of the atmospheric density in shaping aeolian features [306,307], as well as the evolution of complex aeolian fields [304]. Other features of interest that could be mapped in high-resolution imagery include periodic bedrock ridges (PBRs) [308,309,310,311] and dark dune areas, which may have liquid water and astrobiological significance [312,313,314]. Supervised classification algorithms or anomaly detectors could be trained to map dark dune areas. Supervised classifiers can identify TARs in HiRISE imagery [133] and could likely be adapted to locate PBRs, as well. Both TARs and PBRs could easily be detected if high-resolution DTMs were more broadly available.

3.2.5. Map Inverted Channels

Inverted channels are raised features interpreted as exhumed fluvial paleochannel deposits exposed by erosion. The inverted terrain represents possible cemented deposits resistant to erosion [315]. Martian channel inversion is the result of long-term aeolian erosion [316,317]. The degree of exposure can be used to determine erosion rates [318,319,320]. Mapping these features is difficult since most are relatively small and at times indistinguishable from non-inverted surficial channels. These features would be best extracted by a supervised classifier in high-resolution DTMs, but such extraction may be possible with HiRISE data. Once extracted, further analyses of the width, length, or branching patterns of the channels could be performed to better understand historic flow patterns.

3.2.6. Map Small Channels

The amount of water, or discharge, that flowed through Martian rivers can be used to understand the planet’s former climate. While large channels have been mapped [321,322,323], smaller channels, which likely played an important role in past Martian hydrology, are poorly mapped [324]. Small channels are also important when identifying chloride deposits on Mars, which are important since they record the last surface water present at a given location on the surface, seeing as subsequent water events would dissolve them. Chloride deposits are found in local topographic depressions, sometimes with channel-like features that are often at the resolution limits of available imagery. Highly effective and simple tools to extract channels from DTM data have existed for decades, but the lack of high-resolution DTMs is again limiting. Given the likely characteristic pattern of illumination and shadow across channels, they could also potentially be mapped in imagery by a supervised classification algorithm. Then, similar to inverted channels, characteristics of past flow conditions can be inferred from these channels.

3.2.7. Locate and Characterize Chaotic Terrain

Chaos terrain refers to regions of a planetary landscape characterized by irregularly shaped, flat-topped, angular-sided blocks separated by steep valleys [325]. Several hypotheses exist to explain their formation, including association with flooding and outflow channels, crustal deformation, subsurface collapse, and groundwater release [326]. Mapping the distribution of these features through the recognition and classification of Martian chaos terrain could help distinguish theformation mechanisms [327] on Mars, as well as on similar planetary and planetary satellite surfaces [328,329]. The irregular nature of chaos terrain could prove challenging for a supervised classifier, but given the relatively large scale of chaos terrain on Mars, ML applications would not just be limited to HiRISE data; beyond these, classifiers could incorporate greater dimensionality into the dataset by using other instruments with a greater spectral resolution. Given recent advances in ML super-resolution methods [330,331,332,333] and standard panchromatic sharpening [334,335,336], these coarse datasets could be leveraged to help identify patterns of blocks and valleys.

3.2.8. Locate Ice/Water

Thermal and geomorphic mapping would be useful to locate accessible ice deposits within regions located near preferred landing-sites [337]. This ice is typically covered by a layer of dust or regolith, but is exposed in some locations by fresh impact craters or in erosional scarps [338,339]. Like small channels and inverted channels, exposed scarps are small features [338], often at the limits of imagery resolution. As such, supervised classification approaches may be useful to locate and analyze these features, especially since ice and water prompt different responses from Martian regolith, which may be recorded via a range of remote-sensing methods such as thermal, radar, and seismic sensing. Similar approaches, using a range of geophysical sensors, have suggested significant subsurface water exists across much of Mars outside of the polar regions [340].

3.2.9. Identify Glacial Landforms

To return to a previous example, a cirque, normally found in mountainous environments, is generally defined as a semicircular feature with a steep head and sidewalls, an overdeepened central area, and a low sill that defines the lower edge of the feature. However, many features recognized as cirques have one or more of these elements poorly defined or missing. Moreover, even when these features are present, the “edges” of the feature may be diffuse/transitional. Measures of cirque size and form can thus vary from observer to observer. As described above, however, even a relative novice mountain-goer can recognize a cirque. The presence of cirques on Mars indicates both past glaciation and areas of potential for locating subsurface remnant water or ice [341]. Other morphologically distinct glacial features on Mars include U-shaped valleys [192], lineated valley fills [342,343,344,345], concentric crater infills [342,343,346,347], and glaciers themselves [247,348]. There has already been success in extracting cirques from relatively coarse DTM data [192,262] with supervised classifiers and simple methods, as well as mapping debris-covered (rock) glaciers with a combination of coarse imagery and DTMs [349,350,351]. Similar methods should be tested on Mars.

3.2.10. Detect Novel or Rare Mineral Phases

Spectral data from Mars have been used to detect rare mineral phases in the Jezero crater [182,183]. Other rare mineral phases likely exist on Mars, and broader applications of ML methods to mine CRISM data should yield fruitful insights in novel geographic areas of interest for mineral researchers. In general, the application of multi-spectral methods to Mars remains underexploited given the efficacy these methods have demonstrated on Earth [24,352,353,354,355,356,357]. However, the recent development of an ML toolkit for analyzing CRISM data is a promising step in this direction [358].

3.3. Leverage Generative Adversarial Networks

3.3.1. Generating Synthetic Imagery

Machine learning is challenged by class-imbalance problems [359,360,361,362,363,364,365] and limited datasets. Some ML methods will not perform well if a single class is overrepresented within their training data. For example, a network that determines dogs versus cats in images will not work well if it is trained on a dataset composed of 80% cats and 20% dogs as the network can simply guess “cat” for every image and will be correct 80% of the time. There are well-established methods to mitigate this imbalance in quantitative data, such as the synthetic minority over-sampling technique (SMOTE) [366], but the options are much more limited when it comes to training with complex inputs like images. Recent developments such as focal loss [66] and semi-supervised contrastive training [200] can drastically reduce the effects of class imbalance, but the challenge remains, especially in small datasets where a minority class is very rare, perhaps with only a few dozen examples in a dataset of hundreds. Most ML methods rely on having a wealth of training data to create robust and generalized models, but sometimes a large number of samples is simply not available or is too time-intensive to collect.

Several studies have suggested that GANs could be used to help mitigate both these problems, by either generating synthetic data to supplement minority classes and reduce the imbalance, or by generating more training data from a small dataset [367,368,369]. Perhaps most relevant to Martian research are recent studies that demonstrate the efficacy of generating synthetic high-resolution remotely sensed data [370,371,372,373]. While it may seem counterintuitive to generate more Martian data when so much of the existing data remains unexplored, the ability to generate more data of a specific type or with specific characteristics is a promising tool for future research on Mars.

Synthetic data could help in two ways. First, the use of synthetic data could help mitigate spillage between training, testing, and validation datasets. For example, a terrain-classifying CNN trained on NavCam imagery may have images of the same area of Mars within its training and testing datasets. While the images themselves are unique and taken from different perspectives, there is overlap within the content of the images—perhaps a unique pattern or object can be seen in multiple images. This overlap could artificially increase the performance of the CNN during testing. To overcome this issue, a GAN could be trained to generate synthetic imagery with the same image characteristics as the NavCam imagery (resolution, field of view, optical distortion, etc.) from training data collected elsewhere on Mars by other rovers. Using this additional imagery in the training and/or testing datasets could reduce spillage and help create a more general classifier. To this end, a recent study showed that GANs can create realistic synthetic rover images [374]. This same method could also be applied to satellite-based terrain classifiers. Even if the training and testing datasets are derived from disparate images, there is a possibility that the images have been collected from the same area and may have similar or overlapping content. This risk is especially high with HiRISE data as the narrow coverage and targeted use of the camera means that some areas have extremely dense and/or repeat coverage. In that context, using GANs to generate synthetic terrain could be helpful in both training to make a classifier more general, and in testing to ensure independence between the datasets.

Second, class-imbalance problems and limited datasets are an issue on Mars, particularly the latter. HiRISE data only cover 3% of Mars, so features that are rare on Mars are likely to be extremely rare in high-resolution imagery. The small number of features could prohibit automated classification and location, though they support anomaly detection. GANs could be used to generate additional examples of the features to train a supervised classifier. For example, recurring slope lineae (RSL) were first described in 2011 in seven locations [375], and in the years since, 933 more locations have been manually identified [376]. There is little reason to believe that every imaged occurrence of an RSL has been identified, and GANs could help increase the training dataset for a CNN-based locator-classifier, which could then find further RSLs. In a promising study [377], Wang et al. showed that a classifier trained with ~900 examples of RSLs was highly (92%) accurate in locating RSLs in 136 other instances; looking ahead, this method could be improved and generalized with additional data augmentation. When testing the feasibility of the approach, the authors of this review found that StyleGAN could be trained to create realistic HiRISE data with just consumer hardware in a preliminary implementation (Figure 1) [378].

3.3.2. Feature Extraction from DTMs

DTMs of Mars are limited in either resolution or coverage, but are vital tools for planetary scientists. There are a wealth of Earth-based studies and methods for extracting geomorphological information from DTMs, but the domain transfer of these methods to Mars has been limited by the small number of high-resolution DTMs available. While early work in this field was promising [379,380], recent efforts have stalled.

The HiRISE instrument can create 0.5–1 m-per-pixel DTMs from stereo imagery, but at present, only ~0.3% of Mars has stereo coverage, and less than one-tenth of those scenes have DTMs generated. The recent development of high- and super-resolution synthesized Martian DTMs [233,234,235,381] has the potential to revolutionize Martian geomorphology in years to come.

A wide range of geomorphological features and landforms have already been shown to be identifiable and extractable from DTMs. The general methods were covered in various reviews [382,383,384,385,386,387,388,389,390,391,392,393], but those applicable to features specifically relevant to Mars (see Section 3.2) are as follows: rock glaciers [350], glacial terraces and ridges [394], glacial cirques [192,262], polygonal terrain [395,396,397], drumlins [386,398,399], glaciovolcanic features [400,401], landslides [402,403,404,405], deltaic features [406], watersheds [407], loess features [382,408], karst landscapes [409,410], relict patterned ground [411], fluvial terraces [412], and thalwegs [413].

Besides those established in terrestrial studies, a further range of potential applications is specific to the Martian domain. Boulders, for instance, could be automatically extracted from DTMs to help determine the rover-landing safety, classify ancient shorelines, or delineate geologic units [273,274,276,277,279,284,414,415,416]. More accurate profiles and measurements could be made of small-scale features such as TARs [299,300,301,417] PBRs [308], and models could be trained to map their locations and sizes automatically [133,259,417]. Inverted channels are important markers of past fluvial activity, and their distinct topographic characteristics [316,317,418,419,420] could be automatically detected. The mapping of channels too fine to be resolved at current DTM resolutions could be made possible in this same way.

Further theoretical applications could include the verification of human-imposed classification schemes by unsupervised classifiers. For example, an unsupervised classifier could group high-resolution data from craters to assess expert-estimated weathering and erosion patterns (i.e., young, old, and very old craters). If the classifier consistently grouped craters with low estimated ages separately from those with high estimated ages, this would support the use of those markers for age estimation. Similar verification experiments could be performed on other top-down classification schemes.

3.4. Develop Standardized Datasets

Standardized benchmark datasets are crucial to quantify the current state-of-the-art methods across studies. There are innumerable benchmark datasets in nearly every field in the literature, and many have been used in hundreds or even thousands of studies [421,422,423,424,425,426,427,428,429,430,431,432,433,434,435,436,437,438,439,440,441,442,443,444,445,446]. For example, the MNIST database of handwritten digits [443] has been cited in over 5700 publications at the time of writing. Besides serving as useful prototyping tools, these datasets serve a very important purpose in allowing one-to-one comparisons of different methods. At present, only a handful of standard datasets exist for Mars, and new methods are rarely directly compared to existing methods. Looking ahead, we propose that a standard testing dataset containing both imagery and DTMs should be created for crater counting methods. Then, as new algorithms or approaches are developed, their performance and characteristics can be directly compared. Currently, there is no easy way to determine which crater counting method performs the best out of the dozens outlined here. Without this comparison, every researcher has reason to believe that their method is the best, and a method may become the standard in related studies out of habit rather than by merit. As the use of ML expands in planetary science, the need for these resources will increase, and funding for their creation should be a priority.

3.4.1. Unlabeled Data

The most popular standardized dataset is Mars32k [447], which has been used in several studies, primarily those focused on rover navigation [123,232,374]. The dataset is relatively simple and is intended for unsupervised learning projects. As the name implies, it contains 32,000 images that have been downsampled via linear interpolation to a standard resolution of 560 × 500 pixels. The uniformity of the data and the dataset’s ease of use are commonly cited reasons for its popularity. As discussed above, unsupervised learning is not commonly utilized in Martian studies, but more unsupervised research is warranted; to advance unsupervised research in this way, the community needs more unlabeled datasets.

3.4.2. Expert Labeling

Most of the existing benchmark datasets on Mars are derived from expert knowledge, but few have achieved widespread use. The GSMRI dataset [374] used Mars32k and a GAN to generate synthetic images of Martian rocks, thereby doubling the size of the dataset, with a four-level classification structure used to label all rock images. Meanwhile, the DoMars16k [137] dataset consists of 16,150 samples across 15 general landforms in five categories, derived from 163 Context Camera images [448]. Further to this, AI4Mars [449] contains 35,000 images with 326,000 labeled objects intended for use in training autonomous rover navigation algorithms. Elsewhere, Rockfall Mars and Moon (RMaM) [450] contains a mixture of Martian and Lunar satellite images that can be used to train and test rockfall detectors. Then, the Mars orbital image (HiRISE) labeled data, version 3.2 [451], contains 10,815 samples broken down into eight landform classes. In another example, the MSL Surface Data Set (v2) contains 6800 images with 24 classes that are mostly focused on parts of the Curiosity rover as well as rover activities. The Mars day crater detection (MDCD) dataset, meanwhile, is composed of 500 images and 12,000 craters [88].

The datasets of rover imagery are at present very good; however, creating more satellite image-based datasets should be a priority. Satellite images cover a more diverse geographic distribution than rover images, and they are useful in wider domains. However, labeling satellite imagery is challenging given that a single satellite image can cover a large spatial extent and include many different types of features that might need to be re-labeled or reclassified for different projects. For example, a broad “aeolian” class might be suitable for one classifier model, but another might need subcategories [249] such as “dunes” and “ripples,” while a third project might need to identify different types of dunes. Thus, experts’ efforts to develop a more unified supervised classification hierarchy represent an important endeavor; to date, certain work has been done in this area of semantics by looking at DTMs on Earth [452]. Recent work also identified this problem and used semi-supervised learning as a solution [199,200,452], but a consensus structure should remain the priority.

3.4.3. Crowd-Sourced Data Labels

There have been a few projects seeking to leverage “the wisdom of the crowd” to classify Martian images: LabelMars [453,454] (as part of the Novelty or Anomaly Hunter (NOAH) project [455]), AI4Mars [449], Content-based Object Summarization to Monitor Infrequent Change (COSMIC) [456,457], Planet Four [458,459,460], and Planet Four: Ridges [461,462]. LabelMars was unable to recruit enough volunteers to succeed, and it reverted to an expert-labeling method. AI4Mars used the same dataset and successfully collected a significantly larger number of labels, likely due to the much simpler classification scheme of six classes, versus 20 in LabelMars. COSMIC generated 151,000 classifications from 4240 volunteers using a simple scheme to mark and classify araneiforms [463] in HiRISE imagery near the south polar region [456]. Planet Four generated 279,000 classifications from 16,734 volunteers searching for carbon dioxide jets in HiRISE imagery [458]. Planet Four: Ridges generated 514,000 classifications from 14,079 volunteers using a simple two-class system to locate polygonal and Meridiani-type ridges [461].

Given the huge quantity of data from Mars, using novice or amateur classifiers is an important direction for future work. Non-experts can help expand existing datasets or may contribute to generating novel ones. These datasets can then be used for research directly or work as training datasets for ML classifiers. The classification schemes used in the expert labeling section above were all relatively simple and accessible to non-geologists. The infrastructure for crowd-sourcing image labels and classifications is available on many web-based platforms such as Zooniverse [456,458,461] and LabelBox, and the growing public participation appears to demonstrate an appetite to help, making this an area ripe for further development.

3.5. Unify Geographic Data and Analysis Systems

The Planetary Data System (PDS) [1,2] is a distributed data storage and access system used to house NASA data from rovers, satellites, and other instruments. Data in seven categories (atmospheres, small bodies, etc.) are stored at different “nodes” across the United States, usually hosted by a university or other major institution. These nodes use different website structures, data formats, search queries, and tools—demanding practice and experience with each system. Further complicating the data navigation is the stratification of different products across different nodes. For example, the final one-meter DTM for the Gale crater can be found on the “cartography” node hosted by the United States Geological Survey, while the data the DTM is derived from are found there but also on the “geosciences” node hosted by Washington University in Saint Louis. Interactive maps showing the spatial distributions of these data are then hosted separately on the websites of Arizona State University (MOLA, HRSC), and the University of Arizona (CTX, HiRISE). If one wanted to use this DTM with the ground or atmospheric temperature in the Gale crater, the data could be found on the “atmospheres” node hosted by New Mexico State University, but there is no equivalent map visualization to locate ground or atmospheric samples that spatially overlap with the DTM in question. While there are certainly benefits of this distributed system, the disparities between different instruments’ workflows and data storage make cross-disciplinary research challenging to all but the most experienced users. As noted by [16], such research is key to advancing our understanding of planetary surface processes: “We need coupled surface, subsurface, and atmospheric/meteorological measurements so as to enable ’full system’ studies.”

As large corporations grow increasingly interested in planetary-scale data products (e.g., Google Earth and Microsoft’s Planetary Computer, among many smaller competitors), the disparity between these holistic solutions and the PDS grows. Google Mars is a popular platform that enables many Martian datasets and basemaps to be visualized at once. By using an extension of the Google Maps application programming interface (API), users can query several basemap types: elevation, visible, and infrared. However, the API is outdated compared to that of Google Earth Engine, and it lacks the high-resolution products that have revolutionized Martian science in recent years. JMARS [464] is another excellent geographic information system designed specifically for Martian data by Arizona State University. While JMARS works with most of the common geoscience data products, it does not yet include other data such as atmospheric information or raw imagery. Furthermore, it lacks a robust API for large-scale programmatic analyses. At present, neither platform could answer a query such as “return a list of all HiRISE DTMs that overlap >50% with CTX DEMs.” Funding to refine these existing tools, or develop new, more comprehensive platforms, should be a priority to enable further ML studies of Mars.

4. Conclusions

The collection of Martian data has outpaced the rate at which humans can meaningfully parse and interpret each record. Leveraging the novel automated analysis methods pioneered in other fields—Earth observation, most notably—presents a natural solution to this dilemma. As we have outlined in this article, ML studies are taking place on Mars, and there are many fruitful avenues via which these could be further applied. As [16] recently noted, “Mars is a uniquely enabling study target for investigations of surface processes active on other planetary bodies—both for extending terrestrial based process models and, in some areas, for serving as a superior comparative planetology basis over Earth.”

With greater knowledge transfer between Martian domain experts and machine learning specialists, insights from ML research could increase. Here, we outlined a range of specific future study routes, and we offered a few broad suggestions for the community and research funders. We conclude that improvements to our understanding of the physical processes on Mars, and potentially other non-terrestrial surfaces, can be achieved using ML. A few key findings should be noted: (1) semi-supervised and self-supervised learning are especially important methods for Martian applications, (2) creating synthetic Martian data (especially of high-resolution, accurate digital terrain models) using generative adversarial networks has the potential to revolutionize Martian geomorphology studies, (3) developing standardized datasets and benchmarks with which to test new methods should be both a research and funding priority, and (4) Martian research is hindered today by the lack of a scalable and programmatic analysis platform, such as Google Earth or Planetary Computer.

Challenges aside, existing and ongoing ML research on Mars has proven valuable, and the research and methods summarized here should be used as a basis to fund, encourage, and undertake further such studies.