Mapping Cones on Mars in High-Resolution Planetary Images with Deep Learning-Based Instance Segmentation

Abstract

Cones are among the significant and controversial landforms on Mars. Martian cones exhibit various morphological characteristics owing to their complex origin, and their precise origin remains an active research topic. A limited number of cones have been manually mapped from high-resolution images in local areas, and existing detection methods are only applicable to a single type of cone that has a similar morphology and spatial distribution, leading to the vast majority remaining unidentified. In this paper, a novel cone identification approach is proposed that is specially designed for adequately recognizing cones from different regions in high-resolution planetary images. First, due to the lack of a publicly available cone database for reference, we annotated 3681 cones according to the literature on manual interpretation and the cone information provided by the Lunar and Planetary Laboratory (IRL) in HiRISE images. Then, the cone identification problem was converted into an instance segmentation task, i.e., a cone identification approach was designed based on deep neural networks. The Feature Pyramid Network-equipped Mask R-CNN was utilized as the detection and segmentation model. Extensive experiments were conducted for fine recognition of Martian cones with HiRISE. The results show that the proposed approach achieves high performance; it especially efficiently detects multiple types of cones while generating accurate segmentation to describe the geometry contour of cones. Finally, a Martian cone dataset with deep learning-based instance segmentation (DL-MCD) was built, containing 3861 cones for exploring geological processes on the surface of Mars.

1. Introduction

Cones protruding from the surface of Mars are distinctive features of Martian geomorphology [1]. Compared to other typical landforms on Mars, cones exhibit various features in different morphologies and are widely distributed on the surface of Mars, including in high-latitude regions [2]. Cones can be classified into three types according to their origin, i.e., volcanic (scoria/cinder cones [3,4], rootless cones [5,6], and tuff cones/rings [7]), sedimentary (mud volcanoes [8,9,10,11]), and periglacial (pingos [12,13]). The formation of cones is evidence of recent water/ice, hydrothermal, and related ore-forming processes on Mars, and information has been recorded on the environment and climate change characteristics of Mars [14,15,16]. The effective identification of Martian cones is a necessary and important task for revealing the geological evolutionary history of Mars.

At present, the identification of cones is mainly focused on the utilization of the Mars Reconnaissance Orbiter (MRO) data. Manual visual interpretation has always been the main means of cone detection. As early as 1979, cones were first recognized by Allen and Frey on the northern plains of Mars using Viking Orbiter imagery [17,18]. Three years later, Frey and Jarosewich [19] found many small cones in Acidalia and parts of Utopia–Isidis–Elysium Planitia by interpreting more than 12,000 images transmitted back from the Viking orbiter. In 2005, Farrand et al. [20] marked 180 cones in Acidalia Planitia and Cydonia Mensae with the Mars Orbiter Camera (MOC), THEMIS images, and a TES-derived thermal inertia map. Hauber et al. [21] presented observations of 100 cones and mounds on the floor of Coprates Chasma, and the origin of these cones was inferred based on the crater/cone diameter ratios. In 2021, Dapremont and Wray [22] used orbital remote sensing data to assess 253 cones. On 15 May 2021, China’s Tianwen-1 successfully landed on the Martian Utopian Plain. The Tianwen-1 orbiter is conducting a detailed survey of the landing area via the High-Resolution Imaging Camera (HiRIC). More recently, Huang et al. [23] found a total of 272 well-preserved cones in the landing area with HiRIC data.

Recently, machine learning (ML) has gradually been applied to the automatic identification of cones [24,25,26,27] on two types of high-resolution MRO images, i.e., the High-Resolution Imaging Science Experiment (HiRISE, 0.3 m/pixel) and Context Camera (CTX; 6 m/pixel). Traditional machine learning methods such as artificial neural networks (ANNs) [24] and support vector machines (SVMs) [25] combined with features extracted based on well-established image processing techniques such as gradient features have been used for the automatic recognition of Martian landforms. However, the feature extracted via traditional image processing techniques is a hand-designed feature with weak robustness. With the great success of deep learning in the field of image processing, especially deep convolutional neural networks (CNNs), some researchers have tried to use CNNs for cone recognition owing to their powerful feature extraction capabilities. For example, Palafox et al. [25] demonstrated the utilization of a CNN and an SVM to classify volcanic rootless cone groups in northeastern Elysium Planitia on HiRISE, and the results show that the CNN is more advantageous for unknown cone detection. Palafox et al. [26] also classified volcanic rootless cone groups with an SVM and a CNN on HiRISE and CTX. Such methods, however, are still a two-stage model with a more laborious design process and lower performance compared to end-to-end models. Therefore, the researchers only used CNNs to accomplish cone recognition in an end-to-end manner. These types of methods can only determine cone groups, and information such as the number, size, and location of cones cannot be acquired. Pieterek et al. [27] automatically detected cones and impact craters with CTX global mosaic images using a CNN, with a detection rate of 90% (7/8) and 80% (71/90) for cones and impact craters, respectively. In general, little work has been performed to develop generalized recognizers to identify Martian cones. It is necessary to design models that can accurately recognize individual cones based on advanced artificial intelligence techniques.

According to the current literature, three factors, i.e., the spatial differentiation of cones for complex geological activities on Mars, various morphological characteristics owing to their complex origin, and the limited number of recognized cones, should be considered in the automatic identification of cones. The spatial distribution of Martian cones displays area differences. Existing techniques for the automatic identification of cones can only be applied to specific types in a small area. Meanwhile, the complex origin of cones leads to their irregular shapes and degraded morphology with blurred boundaries and them being connected to each other. This increases the difficulty of individual identification of cones, and complete cone information is crucial for future analysis. More importantly, there are relatively few marked Martian cones and no publicly available cone database for reference. Detection models use limited and single types of typical cones as training samples, as other types cannot be identified efficiently and comprehensively.

To address these issues, we annotated a Martian cone dataset that contains 3681 cones according to the literature on manual visual interpretation. The cone information was provided by the Lunar and Planetary Laboratory (IRL) at the College of Science, University of Arizona, from HiRISE images. Then, we converted the cone identification problem into an instance segmentation task and identified cones using the HiRISE images by means of deep neural networks. The identification of Martian cones with an instance segmentation approach can distinguish different individual cones in a pixel-to-pixel manner, which provides accurate information about the location and size of each cone, which is critically important for subsequent research. The main contributions of this article are as follows.

(1)

For the comprehensive identification of Martian cones, we use cones with different origins and distributed in different spatial locations as training samples. The Feature Pyramid Network-equipped Mask-RCNN model is introduced to Mars cone instance segmentation. This model not only effectively detects the cones in the image but also generates high-quality segmentation masks for each cone.

(2)

The Feature Pyramid Network fuses the features extracted by the convolutional network backbone at different levels in a top-down manner, which is suitable for the different scales of cones in high-resolution images, and thus provides a higher cone recognition performance.

(3)

Instance segmentation of the Martian cone dataset was carried out based on deep learning (DL-MCD) with 3681 cones, which is 180 more than the initial number of annotated cones. In this dataset, the locations and morphological characteristics, e.g., cone width and basal dip angle, are provided to explore geological processes on the surface of Mars.

2. Data and Methods

2.1. Description of Dataset

The imaging mode of the annotated Martian cone dataset is HiRISE images with a high resolution of 0.25 to 0.5 m/pixel. We collected and selected 43 and 12 HiRISE images according to Refs. [3,8,9,21,22,28,29,30,31,32,33] and the cone information provided by the IRL, in which 3355 and 326 cones were annotated, respectively. In this dataset, the origin of more than 40% of cones has been inferred, such as scoria cones [3,9], rootless cones (pseudocraters) [31], mud volcanoes [22,23,24,25], and tuff cones/rings [21], leading to the presentation of various morphological characteristics. The spatial distribution and morphological characteristics of Martian cones in the HiRISE images appear as isolated, clustered, chained, irregular, and destroyed cones. Among these cones, clustered and isolated cones account for 50% and 40%, respectively. Figure 1 shows the typical cones in the HiRISE images. It should be emphasized that, in this study, we do not distinguish among the different origins of cones, and all cones with different origins are identified as targets.

2.2. Identification of Cones with Deep Learning-Based Instance Segmentation

This subsection presents an intelligent Mars cone recognition method based on the Mask R-CNN [34] instance segmentation model. The Mask R-CNN is an advanced instance segmentation model extended from the Faster R-CNN [35] object detection model, which can simultaneously perform object detection and semantic segmentation. To adapt to the multi-scale characteristics of cones, the Mask R-CNN equipped with a Feature Pyramid Network (FPN) [36] was adopted for cone recognition. The model mainly includes multi-scale feature extraction, the region proposal network (RPN), the ROIAlign module, and the instance segmentation head. The framework of the Feature Pyramid Network-equipped Mask R-CNN is shown in Figure 2.

2.2.1. Multi-Scale Feature Extraction

The multi-scale feature extraction module uses a standard convolutional neural network (ResNet50 [37] was used in this study) to extract features at different levels from bottom to top and then fuses the features at different levels from top to bottom with the FPN module to obtain a multi-scale feature representation of the image.

2.2.2. Region Proposal Network (RPN)

The RPN generates candidate boxes with different sizes and scales based on the extracted multi-scale features and then classifies these potential candidate regions and generates bounding boxes. Candidate boxes that may contain targets are selected and integrated into the second stage framework for instance segmentation.

2.2.3. ROIAlign Module

The RoIAlign module processes the candidate frames into features of the same size, which are passed into the subsequent instance segmentation header. Specifically, RoIAlign uses bilinear interpolation to obtain the feature values of each point in the feature map, converting the feature aggregation operation into a continuous process instead of the two quantization processes in the RoIPooling operation in the Faster R-CNN model, which improves the detection and segmentation accuracy of the model.

2.2.4. Instance Segmentation Head

The instance segmentation head consists of two parts: the detection head and the segmentation head. The detection head is implemented using a fully connected layer and a Softmax classifier, which is used to accurately obtain the size and location of the cone, as well as the confidence level that the region is recognized as a cone. The segmentation head is implemented with fully convolutional networks (FCNs) [38]. It firstly up-samples the feature map via deconvolution to reduce the scale difference between the feature map and the original image, thus alleviating the error in mapping the target region to the original image, and then enhances the feature information of the object by utilizing multiple convolutional layers with a 3 × 3 kernel size and ultimately achieves cone segmentation by generating a prediction for every pixel of the feature map.

3. Experiments and Results

3.1. Experiment Design

3.1.1. Training Strategy

To effectively identify Martian cones with the deep learning-based instance segmentation cone identification approach, we adopted a cropping window size of 1024 × 1024, and the image area of interest was automatically cropped based on the position of the cones and the preset cropping window size for subsequent analysis. The cone images from different adjacent detection maps had a 50% overlap with each other. Then, we obtained a total of 2024 cone images, from which 1798 cone images were randomly selected for training and the remaining images for testing. It should be emphasized that the theoretical minimum detection size of the model is 32 × 32 pixels due to the fact that the feature extraction module of the model has five downsampling operations.

3.1.2. Evaluation Metrics

In this study, the normal instance segmentation criteria, i.e., the recall and precision of object detection and object segmentation, were used to evaluate the performance of the cone identification model.

In object detection, a bounding box (bbox) is usually used to describe the target location. The evaluation metrics of the location of cones, including recall and precision, can be formulated as follows:

$$\mathrm{Recall}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$$

(1)

$$\mathrm{Precision}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$$

(2)

where TP is the true positive cone, FP is the false positive cone, and FN is the number of false negative cones. In the experiment, if the intersection over union (IoU) of the detection result and the ground truth is more than 0.5, the sample is a TP. If the IoU is less than 0.5, the sample is an FP. When there is no ground truth, the sample is an FN. A high recall or a high precision suggests that the model has good performance.

In object segmentation, mAR and mAP were used to evaluate the segmentation results, which can be calculated as follows:

$$mAR=\frac{1}{c}{\displaystyle \sum _{i=0}^{c}A{R}_i}$$

(3)

$$mAP=\frac{1}{c}{\displaystyle \sum _{i=0}^{c}A{P}_i}$$

(4)

where mAR and mAP are the average recall and average precision of pixel segmentation, respectively, and c is the number of segmentation categories. The higher the mAR and mAP, the better the segmentation results.

3.1.3. Implementation Details and Parameter Settings

For the experiments in this study, the Mars cone recognition method was founded on the ResNet50 backbone pretrained on ImageNet [39] and the FPN. The stochastic gradient descent optimization algorithm was used to train the model, with momentum and weight decay set to 0.9 and 0.0001, respectively. The learning rate was 0.02, and the batch size was set to 2. The proposed method was implemented on a computer with an Intel(R) Core (TM) i5-13400F 2.50 GHz CPU, an NVIDIA RTX 3060 GPU, and 16 GB of RAM.

3.2. Results

For the evaluation metrics mentioned above, the performance of the deep learning-based instance segmentation cone identification achieved competitive object detection accuracies, i.e., 92.1% in recall and 84.8% in precision, and object segmentation accuracies, i.e., 92.2% in recall and 84.9% in precision (

Table 1. Results of the proposed cone identification model and the other mainstream methods.

Object Detection		Object Segmentation		Time (s)
Recall (%)	Precision (%)	mAR (%)	mAP (%)	0.52
92.1	84.8	92.2	84.9

). The average identification time required for each image was 0.52 s. For the recall, it indicated that almost all of the recognized cones in the test set can be recovered with the proposed cone identification mode, whereas from the precision, we could infer that the model identified many new Martian cones.

Figure 3 shows the visualization results of the Martian cones with the proposed deep learning-based instance segmentation approach. As can be seen from Figure 3, the identified cones have two indicators, i.e., the detection box and the target edge, which can clearly outline the cone base. For various spatial distributions and morphological characteristics of Martian cones, it showed good recognition performance.

However, due to the complex morphological characteristics of Martian cones, it might be expected that undetected and newly detected Martian cones are inevitable. Figure 4 shows undetected and newly detected Martian cones in the testing data. Green squares represent the newly detected cones, and the blue squares represent the undetected cones. One can observe that missed detection usually occurs when the shape of the cone is not obvious (Figure 4a,b,d), e.g., double cones, destroyed or degenerated cones, or flat isolated cones, and when the cones are densely distributed (Figure 4c,e), e.g., clustered cones, densely adjacent cones, and especially cones with small sizes. For double cones, the crater of the inner cone is relatively small, but destroyed or degenerated cones and flat isolated cones present features of wide and shallow craters. Among the clustered cones, densely distributed smaller cones exhibit cone chains or a pairwise distribution. Due to the limitation of the minimum detection size of the identification model, these smaller cones could not be detected. However, it should be noted that we could find many newly detected Martian cones in the clustered cones.

Deep Learning-Based Instance Segmentation Martian Cone Dataset (DL-MCD)

Due to the effectiveness of the cone identification model, more Martian cones can be recognized in the collected HiRISE images. The deep learning-based instance segmentation model for Martian cone identification based on a derivation of learned features was used to detect and segment cones simultaneously. This resulted in the identification of 3861 cones, which is approximately 5% greater than the initial number of labeled cones. Then, the newly identified cones, i.e., 180 cones that were not included in the labeled cones, were extracted for manual assessment of the identification accuracy. The false positive rate (FPRs) of the newly identified cones was 7.2 ± 3% and was derived with manual inspection by three domain experts. This illustrates the reliability and stability of the cone identification model. In conclusion, a deep learning-based instance segmentation Martian cone dataset (DL-MCD) of 3861 cones has been derived and made available (see Table S1 in Supplementary Materials).

In the DL-MCD, the geometry of the Martian cones, i.e., cone width (i.e., basal diameter), can be quantified with the segmentation results. It is worth noting that the bases of Martian cones often exhibit an irregular elliptical shape. To acquire accurate morphometric features of cones, the pixel edges of cones were extracted, and two cone width parameters, i.e., the long basal diameter (Lbd) and short basal diameter (Sbd), were measured in HiRISE images. Meanwhile, the basal dip angle (Da) of cones also determined the distributed direction of Martian cones. Moreover, derived parameters, i.e., flatness (Lbd/Sbd) of the basal parameters, were also considered. Figure 5 shows the diagram of the given geometry of the Martian cones.

For the DL-MCD, the range of Lbd is 14–2439 m, with a mean of 100 m, and the range of Sbd is 13–2130 m, with a mean of 88 m. Figure 6 shows the frequency histograms and cumulative percentages of basal and dig angle parameters. As can be observed from Figure 6a,b, approximately 90% of identified cones have Lbd < 100 m and Sbd < 100 m. The cone flatness (Lbd/Sbd) is also given in Figure 6c for measuring the shape of the Martian cone. Nearly half of the cones, i.e., 2017 cones, appear circular with flatness (Lbd/Sbd) ≤ 1.1 and present no special direction; a few cones exhibit an elongated elliptical shape, and three cones present obvious flatness (Lbd/Sbd) > 1.9. For the cone distributed direction with Da (Figure 6d), more than 55.24% of cones do not have apparent selectivity of direction, and their flatness (Lbd/Sbd) is close to 1, whereas 48 and 150 cones face toward the northeast or south-east, and 751 and 895 cones face toward the direction of north–south or east–west.

4. Discussion

4.1. Spatial Distribution Analysis

In this subsection, the spatial distribution of Martian cones in the DL-MCD is presented with the acquired spatial location information. In the DL-MCD, the cones were identified across six typical geographical regions on Mars. The distribution regions include (1) plains, such as Utopia Planitia, Amazonis Planitia, Isidis Planitia, Chryse Planitia, Elysium Planitia, and the area between the Amazonis Planitia and Elysium Planitia; (2) valleys and canyons, i.e., Athabasca Valles, Valles Marineris, and Coprates Chasma; (3) the Tharsis volcanic province, i.e., Olympus Mons and Pavonis Mons; (4) the southern highlands of Mars known as Terra Sirenum; (5) the areas near impact craters, such as Kamativi crater and Zunil crater; (6) ridges, i.e., Tartarus Montes; (7) fossae, i.e., Ulysses Fossae; and (8) other regions, i.e., the Aetheria Region, the Cydonia Region, Hydraotes Chaos, and the Phlegra Dorsa Region.

Figure 7 shows the distribution scale of the Martian cones in the DL-MCD. The widely distributed region of the identified cones is the Kamativi crater; nearly half of the cones are located near this area. More than 600 cones are in the region near the Zunil crater. For the plains, nearly 600 cones were identified in the western Elysium Planitia. A total of 273, 207, and 142 cones were located in the Athabasca Valles, Tartarus Montes, and Phlegra Dorsa, respectively. In other regions, there are fewer than 100 identified cones. Among these identified cones, 360 distributed in the western Elysium Planitia, Phlegra Dorsa, Utopia Planitia, Valles Marineris, and Terra Sirenum are not mentioned in the existing literature. The newly identified cones are mainly in the area of the Elysium Planitia and Phlegra Dorsa, where 157 and 142 cones are distributed, respectively.

4.2. Morphological Parameter Analysis

To verify the analysis of the newly identified Martian cones, we compared the three morphometric parameters, i.e., the long basal diameter (Lbd), short basal diameter (Sbd), and basal dip angle (Da), for the cones in the regions that had not been described in the existing literature. The western Elysium Planitia and Phlegra Dorsa are the two regions that have a large number of newly identified cones.

Figure 8 shows the distribution of newly identified cones in western Elysium Planitia and Phlegra Dorsa, also with the analogous cones in the existing literature [14,31]. In the western Elysium Planitia, the 152 newly identified cones are mostly circular with no direction, but elongated, irregular forms are also common. They are of varying sizes, with Lbd and Sbd ranging from 25.55 m to 219.18 m and from 23.27 m to 212.32 m, respectively. The analogous cones are located at Cerberus Palus within Elysium Planitia and determined as pseudocraters on Mars, with basal diameters ranging from 20 m to 300 m [14]. Compared with the middle upper image, the newly identified and analogous cones occur both in isolated clusters or bands of cones and in direct contact with lava flows or lava plains [14]. For the Phlegra Dorsa, the 142 newly identified cones and the analogous cones near the southern margin of Arcadia Planitia in the Tartarus Colles show unique characteristics, i.e., chain structures, and have mainly been interpreted as rootless cones produced by successive explosions as the crust was translating over a fixed steam source where the explosions overlapped [31,33,40]. The identified cones are usually below 120 m in Lbd and Sbd, and the diameter of rings ranges from 40 m to 120 m, similar to those in Ref [33]. Some cones in both Figure 8b and the lower middle of the image are elongated, with the south-east and north–south directions merging into parallel wakes.

4.3. Further Discussion on the Experiments

Due to the complex morphological characteristics of Martian cones and the limited HiRISE images, it is a challenge to identify Martian cone-like features. The DL-MCD is a powerful support for the geological and geomorphic analysis of Mars. However, the spatial distribution shows that the identified cones only account for a very small portion of Mars. Moreover, the small targets, also with a limited number of training samples, degrade the identification results. The identification of cones for expanding the DL-MCD will be a long-term task with the release and collection of HiRISE images.

In addition, the basal diameter and dip angle of the cones have been provided in the DL-MCD based on the cone identification model. There are other morphometric parameters, e.g., crater width and cone height, which are also crucial to studying the features of cones. The elevation data are crucial for the comprehensive analysis of cones, and the generation of elevation data will be investigated in the future. This can further improve the accuracy of cone identification.

From a data perspective, the field of view and coverage of HiRISE images are limited and cannot meet the requirements for the large-scale mapping of cones [41]. The HiRIC onboard China’s Tianwen-1 will be used to acquire new Martian cones [23].

5. Conclusions

This study focuses on mapping Martian cones in high-resolution images. To identify cones sufficiently, we collected Martian cones with various morphological characteristics in HiRISE images. Meanwhile, a deep learning-based instance segmentation approach, i.e., the Feature Pyramid Network-equipped Mask R-CNN, was utilized. To identify different scales of the Martian cones efficiently, the network was built by fusing the features extracted by the convolutional network backbone at different levels in a top-down manner. Meanwhile, the Martian cones were detected, and their basal outline was segmented. The cone identification model based on the Feature Pyramid Network-equipped Mask RCNN effectively and accurately detected and segmented conventional and isolated cones, clustered cones, cone chains, double cones, elongated cones, and destroyed or degenerated cones. The detection recall, precision, and segmentation mAR and mAP of the cone identification model were 92.1%, 84.8%, 92.2%, and 84.9%, respectively, and the average run-time was 0.52 s per image. Meanwhile, the morphometric parameters of cones were accurately calculated for building the DL-MCD. Finally, 3861 identified cones in the DL-MCD were analyzed with detailed surveys of the spatial distribution and morphological characteristics. These cones can be used in a comparative study to analyze their possible origin to advance the understanding of Martian geology.

In the future, we will collect more HiRISE images to acquire new cones to overcome the limitations of the deep learning-based instance segmentation approach and refine the cone identification model to obtain other crucial morphometric parameters. Additionally, we plan to study a cone detection model based on transfer learning so that it can recognize new cones in other high-resolution images that lack cone information, e.g., HiRIC images on Tianwen-1.