Robust Feature Matching of Multi-Illumination Lunar Orbiter Images Based on Crater Neighborhood Structure

Abstract

Lunar orbiter image matching is a critical process for achieving high-precision lunar mapping, positioning, and navigation. However, with the Moon’s weak-texture surface and rugged terrain, lunar orbiter images generally suffer from inconsistent lighting conditions and exhibit varying degrees of non-linear intensity distortion, which pose significant challenges to image traditional matching. This paper presents a robust feature matching method based on crater neighborhood structure, which is particularly robust to changes in illumination. The method integrates deep-learning based crater detection, Crater Neighborhood Structure features (CNSFs) construction, CNSF similarity-based matching, and outlier removal. To evaluate the effectiveness of the proposed method, we created an evaluation dataset, comprising Multi-illumination Lunar Orbiter Images (MiLOIs) from different latitudes (a total of 321 image pairs). And comparative experiments have been conducted using the proposed method and state-of-the-art image matching methods. The experimental results indicate that the proposed approach exhibits greater robustness and accuracy against variations in illumination.

1. Introduction

Lunar orbiter images and their derived mapping products are essential for both lunar scientific research and exploration missions [1,2,3,4]. Lunar orbiter cameras from various missions have observed the Moon’s surface for several decades, collecting extensive high-resolution, multi-scale, multi-temporal image data [5]. Some of these images have been used for 3D reconstruction [6,7,8], precise localization of points of interest [9,10], mapping and environmental analyses of potential landing areas [11,12,13]. The regional applications require synergistic use of multi-images, which makes accurate image matching essential [14]. Moreover, tasks such as lunar global mapping and global control network construction require more intensive demands for image matching [15,16]. With the acquisition of massive amounts of high-resolution lunar orbiter imagery and the growing demand for large-area or even global mapping, more reliable image matching methods have become indispensable.

Among the factors influencing lunar orbiter image matching, significant variations in shading and shadowing pose major challenges [17,18]. These variations are closely related to changes in the angles of sunlight, manifesting as a non-linear effect on image intensity, commonly known as non-linear radiometric distortion [19]. Specifically, the degree and direction of shadows cast by lunar terrain features are mainly influenced by the solar incidence and azimuth angles, further exacerbating the image intensity distortions. Some examples of multi-illumination lunar orbiter images (MiLOIs) are shown in Figure 1. The rugged terrain and weak-texture surface cause substantial differences in image intensity resulting from significant changes in sun direction over time. Low solar incidence angles (~0°) lead to overexposure and low contrast in images, generally in equatorial regions. Conversely, high incidence angles (~90°) result in severe shadowing, and even permanently shadowed regions near the lunar poles, due to the Moon’s low axial tilt relative to the ecliptic (approximately 1.54°). Therefore, for weakly-textured lunar surfaces, severe changes in illumination conditions predominantly may induce challenges in feature matching algorithms [20].

Feature-based matching algorithms have been widely studied in the fields of computer vision and photogrammetry. Typical feature-based methods include Harris (1988) [21], SIFT (2004) [22], ORB (2011) [23], AKAZE (2013) [24], and so on. SIFT is considered one of the most classical methods due to its invariance to scale, rotation, and brightness. However, it remains susceptible to intensity distortions caused by shadows during feature detection and is sensitive to non-linear radiometric distortions when using gradient information to describe features. To this end, Wu, et al. [25] improved the gradient orientation histogram of the SIFT algorithm to adapt it for matching planetary orbiter images with varying shadow direction. Yet, its robustness in matching images with large differences in both solar incidence and azimuth angles still requires further investigation, especially for images with severe shadows at low solar elevation angles. Overall, the performance of the above-mentioned spatial domain-based feature matching methods is limited in feature detection and description when dealing with non-linear radiometric distortions. Therefore, some researchers conducted feature detection and description over the frequency domain, effectively enhancing the matching performance of multimodal remote sensing images with non-linear radiometric distortions in Earth observation [19,26,27,28]. Similar approaches have also been applied to planetary remote sensing images with weak textures and illumination differences, such as FDAFT [17] and EPCFT [18]. Yet, the accuracy of the matching points remains relatively limited, and the dataset of images with illumination differences is not extensive.

Using geomorphological features for matching is a novel approach in lunar remote sensing image matching [15]. On the moon, geomorphological features, such as impact craters, are ubiquitous and exhibit distinct characteristics [29]. Using craters as cues infuses semantic prior knowledge to the matching problem, which enhances the ability to extract invariant features under extreme conditions [30]. With the recent advancements in deep learning, various algorithms have emerged that automatically detect craters from high-resolution lunar orbiter images using deep learning models [13,31,32,33], which also enhances the feasibility of the crater-based matching methods. Crater-based techniques have been developed for lunar autonomous visual navigation and position technology [33,34]. Researchers have achieved matching between navigation camera images and a base map or a crater database [35,36]. However, the methods are only applicable to matching scenarios involving a small number of craters, which is not suitable for matching orbiter images. For orbiter images, some studies used Hausdorff distance and mutual information to determine the correspondence between individual craters [30,37]. Whereas, these methods rely solely on the local information of individual craters, disturbing their robustness when craters are small or under varying illumination conditions.

Structural information between feature points has been used to improve matching performance of challenging images by reducing reliance on local grayscale descriptors and effectively eliminating outliers [38,39,40,41]. Considering the special challenges of lunar orbiter image matching, and inspired by the robustness of feature crater and structural information, we propose a robust feature matching method for MiLOIs called Crater Neighborhood Structure based Feature Matching (CNSFM) in this research. The approach combines automatic crater detection and topological characteristics of impact craters to address image matching issues arising from weak textures and multiple illumination conditions. Instead of traditional grayscale-based feature description, the proposed crater neighborhood structure features (CNSFs) were treated as matching units. A CNSF consists of a crater and its K-nearest neighboring craters. A clever approach was employed to eliminate non-corresponding craters within CNSFs, enabling the identification of similar CNSFs. Then, a CNSF distance measurement was established based on geometric topological parameters to determine the most similar CNSF pairs. Finally, local geometric transformation consistency was used to remove mismatched CNSFs. The matching process reducing reliance on single feature image patches by capitalizing on structural information between features, thereby enhancing robustness against weakly-textured regions and multi-illumination scenarios. And an evaluation dataset was constructed using lunar orbiter images captured under diverse illumination conditions from various latitudinal regions. This dataset was used to systematically assess the matching performance of recent state-of-the-art multi-modal image matching approaches, as well as the methodology proposed in this study.

This paper is structured as follows: Section 1 presents the research background and objectives. Section 2 details the proposed method. Section 3 provides the experimental results and analysis. Section 4 concludes with a summary of the paper.

2. Methods

This paper proposes a dedicated feature matching framework for robustly matching MiLOIs, called CNSFM. The methodological workflow is illustrated in Figure 2. Firstly, we constructed a crater dataset using MiLOIs by manual labelling and employed it for transfer learning on an advanced open-source deep learning-based object detection model, ensuring that enough corresponding features can be consistently extracted under varying illumination conditions. Secondly, the geometric structure (called CNSF) formed by combining a feature with multiple neighboring features is employed as the fundamental matching unit. This approach aims to reduce sole reliance on local image information from individual features during the matching process, thereby enhancing the robustness of feature correspondence. Whereas the omission of craters poses challenges for the identification of homologous CNSFs. The proposed method overcome them with the following steps. Specifically, based on the principle of similarity invariance, similar CNSF searches are conducted to remove the non-corresponding craters and identify CNSF pairs that satisfy the structural similarity judgement. Geometric parameters of CNSF are then utilized to construct distance measurements, enabling the selection of the most similar CNSF as matched CNSF. Finally, potential false correspondences among CNSFs are eliminated through outlier rejection grounded in the assumption of local geometric transformation consistency. Detailed descriptions will be provided in the subsequent sections.

2.1. Crater Feature Detection

To test the matching algorithm proposed in this paper, the feature detector was obtained by transfer learning based on an open-source deep learning model using a custom crater dataset. We selected the recent version of the YOLO series, YOLOv9 [42], an open-source, easy-to-implement, high-performance object detection network, to create the crater feature detector. YOLOv9 utilizes Programmable Gradient Information (PGI) and Generalized Efficient Layer Aggregation Network (GELAN) to extract key features more effectively, offering strong feature extraction capabilities, high computational efficiency, and fast detection speed. Moreover, since there are no suitable crater datasets available, we prepared a training dataset for transfer learning. We considered the following factors: (a) The image data should be representative, covering multi-illumination images of various geological units; (b) The annotated crater samples should be easily identifiable, with clear and preferably circular edges. These considerations aim to ensure that the crater detector achieves sufficient generalization capability while maintaining high localization accuracy of the detected crater features.

2.2. Crater Neighborhood Structure Based Feature Matching

This paper proposes a crater neighborhood structure feature matching method for MiLOIs. The approach is simple but efficient, leveraging the principle of similarity invariance to establish correspondences among feature-formed structures and thereby enabling robust matching for MiLOIs.

2.2.1. Rationale

Because of the weak textures on the lunar surface, crater centers are often used as tie points during manual image registration. The intensity characterization of the same crater on MiLOIs may vary considerably. Consequently, it is challenging to directly identify corresponding crater in another image based solely on the characteristics of a single crater. In the case of manual registration, it is often necessary to consider the surrounding craters collectively while identifying correspondences. Inspired by this experience, the geometric structure between craters within the neighborhood of a crater is worthing to be considered. The crater feature detector outlined in Section 2.1 has replaces manual impact crater identification. The automatic feature matching process requires comparing the geometric consistency of crater-formed structures. However, since automatic crater detection inevitably results in omissions or false detections, determining the similarity of geometric structures under these uncertain conditions is the core idea of the method proposed in this paper, which is primarily based on similarity invariants.

2.2.2. Similarity Invariant

A similarity transformation is a type of transformation in which the shape of a figure remains unchanged before and after the transformation. It can be decomposed into scaling, rotation, and translation. A similarity transformation $T$ is generally represented by,

$${\mathit{P}}^{\mathbf{\prime }}=T\left(\mathit{P}\right)=sR\mathit{P}+\mathit{t}$$

(1)

where $\mathit{P}$ is the observation; ${\mathit{P}}^{\mathbf{\prime }}$ is the observation after similarity transformation; $s$ is a scale parameter; $R$ is a rotation matrix; $\mathit{t}$ is a translation vector; For a two-dimensional plane, the rotation matrix $R\left(\theta \right)$ corresponding to a rotation angle $\theta$ can be expressed as:

$$R\left(\theta \right)=\left[\begin{array}{cc}\cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{array}\right]$$

(2)

The translation vector $\mathit{t}$ is formed by scale parameters $t_x$ and $t_y$,

$$\mathit{t}=\left[\begin{array}{c}{t}_x\\ t_y\end{array}\right]$$

(3)

Similarity invariants are quantities that remain unchanged when a geometric figure undergoes any similarity transformation. There are some basic invariants under similarity transformations, such as angles and similarity ratios (e.g., distance ratios). Figure 3 shows that the angles and the ratios of distance are invariant under similarity transformation, i.e., $\angle A_1{O}_1{B}_1=\angle A_2{O}_2{B}_2$, $\left|O_1{A}_1\right|/\left|O_1{B}_1\right|=\left|O_2{A}_2\right|/\left|O_2{B}_2\right|$ and ${\varphi }_{O_1}/{\varphi }_{A_1}={\varphi }_{O_2}/{\varphi }_{A_2}$.

Given features $O_1, A_1, B_1, C_1$, we transform them to $O_2, A_2, B_2, C_2$ by a similarity transformation. Based on the invariants, we also have the following equations:

$$\begin{gathered} \mathit{D}=s^2{\mathit{D}}^{\prime }, \mathrm{where} \\ \mathit{D}=\left[\begin{array}{c}\left|O_1{A}_1\right|\left|O_1{B}_1\right|\cos \mathrm{\angle }{A}_1{O}_1{B}_1\\ \left|O_1{A}_1\right|\left|O_1{C}_1\right|\cos \mathrm{\angle }{A}_1{O}_1{C}_1\\ \left|O_1{B}_1\right|\left|O_1{C}_1\right|\cos \mathrm{\angle }{B}_1{O}_1{C}_1\end{array}\right], \\ {\mathit{D}}^{\prime }=\left[\begin{array}{c}\left|O_2{A}_2\right|\left|O_2{B}_2\right|\cos \mathrm{\angle }{A}_2{O}_2{B}_2\\ \left|O_2{A}_2\right|\left|O_2{C}_2\right|\cos \mathrm{\angle }{A}_2{O}_2{C}_2\\ \left|O_2{B}_2\right|\left|O_2{C}_2\right|\cos \mathrm{\angle }{B}_2{O}_2{C}_2\end{array}\right] \end{gathered}$$

(4)

where s is a scale parameter of the transformation with $\left|O_1{A}_1\right|=s\left|O_2{A}_2\right|$.

For lunar orbiter images, the flight altitude of the orbiter is typically much greater than the variations in terrain elevation. As a result, deformations between images primarily exhibit rigid transformations [38]. Moreover, after orthorectification, local regions within the images generally satisfy similarity transformations. Therefore, the similarity invariants mentioned above are applicable to lunar orbiter images. By comprehensively utilizing these invariants, our method can identify similar structures, which details in the following sections.

2.2.3. Similar Crater Neighborhood Structure Feature Search

In the case of a pair of lunar orbiter images with disparate illumination conditions, the initial step is to perform crater feature detection utilizing the crater feature detector outlined in Section 2.1. In instances where images contain prior scale information, the diameters of the crater features are constrained so that the diameters of the crater features in both images fall within the same ground true scale range. This improves the adaptability of the proposed method to images with significant scale variations. As scale information is readily available for remote sensing images, prior scale information is employed in matching scenarios to closely simulate real-world conditions. And the following major work is to identify similar structures between the two crater feature sets. The representation of the crater feature sets is as follows:

$$\chi =\left\{\left(x_i,y_i,{\varphi }_i\right)|i\in I\right\}, \psi =\left\{\left(x_j,y_j,{\varphi }_j\right)|j\in J\right\}$$

(5)

where $\chi$ and $\psi$ are two distinct crater feature sets extracted from MiLOIs of the same region. $I$ and $J$ are separate index sets, representing the number of elements in sets $\chi$ and $\psi$, respectively. $x$ and $y$ are the coordinates of the crater center. $\varphi$ represents the diameter of the crater.

To prevent the computational complexity of similar structure search from becoming unmanageable, we define the structure formed by a central crater (Central Crater, CC) and its neighboring craters (Nearby Craters, NCs) as the foundational structural unit, which is designated as a CNSF. Specifically, the NCs of each crater could be searched quickly using KD-tree. The size of the neighborhood is determined by the K value, which specifies that only the K craters closest to the CC are considered NCs. Proximity is measured by Euclidean distance. The two sets of CNSFs are expressed as follows:

$$\begin{gathered} {\chi }_{CNSF}=\left\{\left({CNSF}_i\right)|i\in I\right\}, {CNSF}_i=\left\{\left(x_i,y_i,{\varphi }_i\right),\left(x_p,y_p,{\varphi }_p\right)|p\in K_i\right\} \\ {\psi }_{CNSF}=\left\{\left({CNSF}_j\right)|j\in J\right\},{CNSF}_j=\left\{\left(x_j,y_j,{\varphi }_j\right),\left(x_q,y_q,{\varphi }_q\right)|q\in K_j\right\} \end{gathered}$$

(6)

where ${\chi }_{CNSF}$ and ${\psi }_{CNSF}$ are the sets of CNSF. $K_i$ and $K_j$ are separate index sets of the K closest craters to the crater $\left(x_i,y_i,{\varphi }_i\right)$ and $\left(x_j,y_j,{\varphi }_j\right)$, respectively.

Determining the correspondence of crater features within a structure is not straightforward due to the omissions or false detections of features. When searching for similar CNSFs, we first assume that the CC of two CNSFs correspond to each other. Then, the neighborhood structure is decomposed into multiple angular structures, and an exhaustive comparison of all angular structures within the two CNSFs is performed to eliminate non-corresponding NCs. If enough NCs remain, the two CNSFs are identified as similar.

For a CNSF, the CC can form an angular structure with any two NCs in a clockwise direction. The geometric parameters of an angular structure include six parameters: the angle $\beta$, the two edge lengths $S_1$ and $S_2$, and the diameters of the three craters, ${\varphi }_0$, ${\varphi }_1$, and ${\varphi }_2$. According to the rule, each NC forms angle structures with the other $K-1$ NCs and a total of $K\left(K-1\right)/2$ angle structures can be formed when there are K NCs.

The direction for angle calculation is set to clockwise, whereas the initial edge is uncertain when splitting different CNSFs into angular structures. The geometric parameter values of an angular structure formed by the same pair of NCs and CC may differ when the initial edge is different, as illustrated in the Figure 4. Therefore, according to the principle of similarity invariants, if the six geometric parameters of an angular structure satisfy the following equations, it indicates that the two angular structures are similar.

$$\left\{\begin{array}{c}\beta ={\beta }^{\prime }\\ {\displaystyle \frac{S_2}{S_1}}={\displaystyle \frac{{S_2}^{\prime }}{{S_1}^{\prime }}}\\ {\displaystyle \frac{{\varphi }_1}{{\varphi }_0}}={\displaystyle \frac{{{\varphi }_1}^{\prime }}{{{\varphi }_0}^{\prime }}}\\ {\displaystyle \frac{{\varphi }_2}{{\varphi }_0}}={\displaystyle \frac{{{\varphi }_2}^{\prime }}{{{\varphi }_0}^{\prime }}}\end{array}\right.or\left\{\begin{array}{c}\beta =360-{\beta }^{\prime }\\ {\displaystyle \frac{S_2}{S_1}}={\displaystyle \frac{{S_1}^{\prime }}{{S_2}^{\prime }}}\\ {\displaystyle \frac{{\varphi }_1}{{\varphi }_0}}={\displaystyle \frac{{{\varphi }_2}^{\prime }}{{{\varphi }_0}^{\prime }}}\\ {\displaystyle \frac{{\varphi }_2}{{\varphi }_0}}={\displaystyle \frac{{{\varphi }_1}^{\prime }}{{{\varphi }_0}^{\prime }}}\end{array}\right.$$

(7)

In the equations, $\beta$ and ${\beta }^{\prime }$ represent the angles of the two similar angle structures, S indicates the length of the sides that form the angle, and $\varphi$ represents the diameter of the craters.

If a pair of CNSFs contains $\xi$ corresponding NCs, $\xi \left(\xi -1\right)/2$ similar triangular structures can be identified according to the split criteria. Conversely, when performing a brute-force comparison of angular structures on pairs of CNSFs and obtaining multiple similar angular structures, we only need to count the frequency of each NC pair’s occurrence among all the similar angular structures to identify the corresponding NCs. For all corresponding NCs, in the ideal case, each will appear $\xi -1$ times, so the mode of the frequency count is taken for $\xi -1$. And if a NC pair appears $\xi -1$ within these structures, it is considered a possibly correct correspondence; otherwise, it is deemed a fake correspondence. Based on this algorithm, non-corresponding NCs within a CNSF pair can be eliminated, and the correspondence between NCs can be established, thereby completing a single iteration of similar CNSF search.

Considering the presence of feature localization errors, the geometric parameters of similar structures do not strictly satisfy Equation (7). Assuming a tolerable feature localization error of $\delta$ pixels and a tolerable crater feature detection diameter error of $\eta$%, angular structures are considered similar if the equation residuals are less than the thresholds. Given that the central crater is fixed, the maximum angular deviation due to the localization error of the two neighboring craters can be derived as $\arcsin \left(\delta /S_1\right)+\arcsin \left(\delta /S_2\right)$; the maximum deviation in the side length ratio is: $\delta \left(S_2+S_1\right)/\left[S_1(S_1-\delta )\right]$; The maximum deviation in the crater diameter ratio is: ${2\varphi }_1\eta \%/\left[{\varphi }_0\left(1-\eta \%\right)\right]$. Taking the left-hand part of Equation (7) as an example, the maximum deviation expression is as follows:

$$\left\{\begin{array}{c}\Delta \beta \triangleq \left|\beta -{\beta }^{\prime }\right|\le \arcsin \left({\displaystyle \frac{\delta }{S_1}}\right)+\arcsin \left({\displaystyle \frac{\delta }{S_2}}\right)+\arcsin \left({\displaystyle \frac{\delta }{{S_1}^{\prime }}}\right)+\arcsin \left({\displaystyle \frac{\delta }{{S_2}^{\prime }}}\right)\\ \Delta R_S\triangleq \left|{\displaystyle \frac{S_2}{S_1}}-{\displaystyle \frac{{S_2}^{\prime }}{{S_1}^{\prime }}}\right|\le {\displaystyle \frac{\delta \left(S_2+S_1\right)}{\left[S_1\left(S_1-\delta \right)\right]}}+{\displaystyle \frac{\delta \left({S_2}^{\prime }+{S_1}^{\prime }\right)}{\left[{S_1}^{\prime }\left({S_1}^{\prime }-\delta \right)\right]}}\\ \Delta R_{{\varphi }_1}\triangleq \left|{\displaystyle \frac{{\varphi }_1}{{\varphi }_0}}-{\displaystyle \frac{{{\varphi }_1}^{\prime }}{{{\varphi }_0}^{\prime }}}\right|\le {\displaystyle \frac{2{\varphi }_1\eta \mathrm{\%}}{\left[{\varphi }_0\left(1-\eta \mathrm{\%}\right)\right]}}+{\displaystyle \frac{2{{\varphi }_1}^{\prime }\eta \mathrm{\%}}{\left[{{\varphi }_0}^{\prime }\left(1-\eta \mathrm{\%}\right)\right]}}\\ \Delta R_{{\varphi }_2}\triangleq \left|{\displaystyle \frac{{\varphi }_2}{{\varphi }_0}}-{\displaystyle \frac{{{\varphi }_2}^{\prime }}{{{\varphi }_0}^{\prime }}}\right|\le {\displaystyle \frac{2{\varphi }_2\eta \mathrm{\%}}{\left[{\varphi }_0\left(1-\eta \mathrm{\%}\right)\right]}}+{\displaystyle \frac{2{{\varphi }_2}^{\prime }\eta \mathrm{\%}}{\left[{{\varphi }_0}^{\prime }\left(1-\eta \mathrm{\%}\right)\right]}}\end{array}\right.$$

(8)

where $\mathsf{\Delta }\beta ,\mathsf{\Delta }{R}_S,\mathsf{\Delta }{R}_{{\varphi }_1}\mathsf{\Delta }{R}_{{\varphi }_2}$ are the equation residuals. Based on experience, setting the thresholds to one-third of the maximum deviations is appropriate.

According to the above strategy, the similar CNSFs can be searched and matched. A ground-truth corresponding CNSF with K = 10 in images under varying illumination conditions is shown in Figure 5. The crater indicated by the yellow line does not have a corresponding feature on the other side and will be excluded by similar CNSF search, whereas the crater indicated by the green line will be retained, with its correspondence clearly established. The results of the similar CNSF search can be expressed as follows:

$$\left\{\left({CNSF}_i^{\tau },{CNSF}_j^{\tau }\right)|i\in I,j\in J,and{CNSF}_i^{\tau }~{CNSF}_j^{\tau }\right\}$$

(9)

where $~$ indicates that the two CNSFs satisfy the similarity criteria, while the superscript $\tau$ denotes that the non-corresponding craters have been excluded from the two similar CNSFs and their correspondence has been established.

The similar CNSF search only ensures that the two CNSFs exhibit similar topological structures within a certain error threshold. Several potential corresponding CNSFs might be identify for a CNSF, with only one being the true match. The objective of the subsequent section is to further constrain the similarity of CNSFs and identify the most similar CNSF.

2.2.4. CNSF Distance Measurement

Measuring distance between features is crucial for feature matching. The measurement of CNSFs’ distance essentially quantifies the degree of similarity between two CNSFs that have been identified as similar in the previous section. Since the one-to-one correspondence of crater features within a CNSF has already been established, we can refer to the similarity invariance principle mentioned earlier. In an ideal similar geometric structure, the vectors $\mathit{D}$ and ${\mathit{D}}^{\mathbf{\prime }}$ should satisfy $\mathit{D}=s^2{\mathit{D}}^{\mathbf{\prime }}$ according to Equation (4). For two collinear vectors with all positive values, their directions are completely identical, and the cosine distance of them is 1. Thus, the specific calculation formula of CNSF distance measurement is shown in Equation (10).

$$d_{CNSF}=\left\{\begin{array}{c}\infty , dim\left(D\right)<{\xi }_{min}\\ 1-{\displaystyle \frac{{‖\mathit{D}‖}_2{‖{\mathit{D}}^{\prime }‖}_2-\mathit{D}·{\mathit{D}}^{\prime }}{{‖\mathit{D}‖}_2{‖{\mathit{D}}^{\prime }‖}_2}}, otherwise.\end{array}\right.$$

(10)

where $\mathit{D}$ and ${\mathit{D}}^{\mathbf{\prime }}$ are vector composed of all angular structures within the similar CNSFs, with a dimension of $\xi -1$. ${\Vert \Vert }_2$ represents the Euclidean norm of the vector, ∙ denotes the dot product of the vectors, and $\mathrm{d}\mathrm{i}\mathrm{m}\left(\right)$ represents the dimension of the vector. ${\xi }_{min}$ is the minimum number of corresponding NCs required in a pair of matching CNSFs. Specifically, the number of remaining NCs in a similar CNSF pair is less than ${\xi }_{min}$, the distance between the two CNSFs is considered infinite.

The strategy for determining matched CNSFs is the Nearest Neighbor Distance Ratio (NNDR). This means that if the ratio of the closest distance to the second closest distance, calculated using Equation (10), is less than a specified threshold (e.g., 0.1), the CNSFs are considered corresponding features. Additionally, the closest distance is also constrained within a threshold close to zero (e.g., 0.1).

2.2.5. Mismatched CNSF Removal

The outlier removal design for CNSFs ensures the reliability of the matching results. Structural similarity does not guarantee a genuine match. Inevitably, there are erroneous correspondences among the matched CNSFs generated in the previous session. Visualization reveals that these erroneous correspondences are easily identifiable and removable (as illustrated in Figure 6). Clearly, erroneous correspondences, while having similar structures, do not exhibit similar geometric transformations. Each corresponding CNSF can be used to compute a similarity transformation parameter. In correctly matched local regions, the similarity transformation parameters remain consistent, whereas mismatched CNSFs exhibit significant differences in their computed parameters. Therefore, mismatch elimination can be achieved by evaluating the consistency of similarity transformation parameters. Firstly, the similarity transformation between images is calculated based on a selected pair of corresponding CNSFs, which is then validated by the remaining CNSF pairs. If the number of CNSF pairs that satisfy this transformation exceeds a predefined threshold $\rho$, the selected pair of corresponding CNSFs is considered a correct correspondence. Whether a pair of CNSF $\left(\mathit{P},{\mathit{P}}^{\prime }\right)$ satisfies the transformation $T$ is determined by the following expression:

$$count\left\{‖T\left(\mathit{P}\right)-{\mathit{P}}^{\prime }‖<ϵ\right\}>⌊{\displaystyle \frac{2}{3}}nu{m}_P⌋$$

(11)

where $ϵ$ is the threshold of the transformation residual, $count$ represents the number of points that satisfy the inequality, and $nu{m}_P$ denotes the total number of points. The notation ∨ ∩ denotes the floor function, which rounds a value down to the nearest integer.

Finally, all correct CNSF correspondences are merged, duplicate corresponding craters are removed, and the final matching result is obtained.

3. Experimental Results and Analyses

3.1. Experimental Data and Metrics

The LROC Narrow Angle Camera (NAC) is composed of two linear pushbroom cameras (NAC-Left and NAC-Right) that offer highest-resolution (typically 0.5 to 2 m per pixel) lunar orbiter images so far [43]. Since 2009, the LROC NACs have been continuously observing the Moon for over 15 years, accumulating many repeated or overlapping observations that provide images of the lunar surface under a variety of illumination conditions. NAC images have been employed for numerous purposes [44], including large-area mapping, high-precision positioning, 3D reconstruction and landing site environment analysis. There is a pressing need for the co-registration of multi-illumination NAC images. Therefore, this paper uses NAC images as the experimental image data source to create the training dataset and the evaluation dataset.

The image data used for creating datasets were selected and downloaded from the Planetary Data System (PDS) Geosciences Node Lunar Orbital Data Explorer website (https://ode.rsl.wustl.edu/moon/index.aspx, accessed on 1 March 2024). Subsequently, relevant images were map projected using United States Geological Survey (USGS) Integrated Software for Imagers and Spectrometers (ISIS) [45,46], and then were converted into the GeoTiffs and cropped to the specified area using GDAL.

3.1.1. Training Dataset

Eight representative regions of global distribution were selected, including maria and highlands at different latitude on the both nearside and farside of the Moon. All available multi-illumination NAC images from each region were processed to create a training dataset, ensuring the crater feature detector’s generalization capability for various illumination conditions across the entire lunar surface. For each region, images were preprocessed, uniformly resampled to 0.5m resolution, and manually co-registered. Craters were then manually annotated using CraterTools [47], resulting in a total of 4682 annotated craters with a minimum diameter of 6 pixels. It is important to note that to minimize the impact of illumination on the accuracy of crater feature localization, only visually interpretable, perfectly circular craters were annotated. Finally, the images and CraterTools annotated craters were converted to the YOLO dataset format, with image blocks of 640 × 640 pixels, resulting in a training dataset comprising 1604 labeled image blocks.

3.1.2. Evaluation Dataset

To validate the effectiveness of the proposed method, multi-illumination images from three different latitude scenes (as shown in Figure 7 S1–S3) were selected for the experiments. S1 is the Chang’e 3 landing site (19.5117°W, 44.1214°N); S2 is the Apollo 11 landing site (23.4731°E, 0.6742°N); S3 is Peak Near Shackleton (123.7783°E, 88.8064°S), a region of interest for lunar south pole exploration [48,49]. All the selected scenes are 1000 × 1000 m. These regions have sufficient repeat observations, providing images under various illumination conditions. After preprocessing the images and removing images with similar illumination conditions, we obtained an image set with multiple scales and illumination that well represent the real scenarios of lunar orbiter image matching. The distribution of the solar azimuth, incidence and resolution for all 42 selected images is shown in Figure 7. For Scenes S1, S2, and S3, the maximum incidence angle differences are 41 degrees, 78 degrees, and 4 degrees, respectively, and the maximum solar azimuth angle differences are 164 degrees, 180 degrees, and 180 degrees, respectively. Sample image pairs with typical illumination differences are shown in Figure 8.

Quantitative evaluation of the matching results requires obtaining the true transformation between images. However, due to various errors, the exact ground truth geometric transformation is difficult to obtain. Therefore, approximate ground truth geometric transformation is commonly used for evaluation. Specifically, for each scene, we select a visually favorable image as the reference and determine five correspondences manually between it and other images to estimate the similarity transformations between all image pairs in the scene as approximate ground truth geometric transformations. And the fitting residuals are all within one pixel. The three scenes consist of 10, 10 and 22 images respectively, for a total of 321 multi-illumination image pairs.

3.1.3. Evaluation Metrics

The following five metrics were employed to evaluate the matching results:

• Number of Correct Matches (NCM)

Matches are considered correct if the residuals calculated using the approximate ground truth similarity transformation parameters are less than 5 pixels. Residuals are calculated as follows:

$${\sigma }_i={‖p_i-T_{true}·{p_i}^{\prime }‖}_2$$

(12)

where, $p_i$ and ${p_i}^{\prime }$ are a pair of matched points, $T_{true}$ represents the approximate ground truth similarity transformation parameters, ${\Vert \Vert }_2$ denotes the Euclidean norm, and · represents the dot product.

2.

Rate of Correct Matches (RCM)

The definition of RCM is the ratio of NCM to the total number of final matches (TNM).

$$RCM={\displaystyle \frac{NCM}{\mathrm{T}\mathrm{N}\mathrm{M}}}$$

(13)

3.

Root of Mean-Squared Error (RMSE)

RMSE is the root mean square of residuals calculated from the approximate true transformation. The RMSE of the correct matches is defined as Equation (14). ${\sigma }_i$ is derived from the application of Equation (12). In addition, the RMSE is set to 5 for failed matching pairs.

$$RMSE=\sqrt{{\displaystyle \frac{1}{NCM}}\sum _{i=1}^{NCM}{\sigma }_i^2}$$

(14)

4.

Success Rate (SR)

SR is the ratio of the number of successful matches to the number of matched image pairs. The definition of successful matching is when NCM between a pair of images exceeds 3.

5.

Matching Time (MT)

This metric is primarily used to evaluate algorithm performance during parameter analysis.

3.2. Custom Crater Feature Detector

To testing the proposed matching algorithm, we performed transfer learning on the YOLOv9 model using our custom dataset, resulting in a crater feature detector. The YOLOv9-C model and its pretrained weights from the MS COCO 2017 dataset were downloaded from https://github.com, accessed on 20 November 2024, and the training was conducted using a single NVIDIA GeForce RTX 3090 GPU (NVIDIA, Santa Clara, CA, USA) under PyTorch 3.9 framework. The custom training dataset created from MiLOIs were randomly subdivided into a training set and a validation set in a 4:1 ratio. The model was trained for 300 epochs with a batch size of 8 to obtain the crater feature detector. Since only easily recognizable craters were manually annotated, the results of the recall and precision are close to 1, indicating that the obtained crater detector has good detection capabilities. Figure 9 illustrates the crater detection results for a pair of MiLOIs. After cropping, the two images cover the same area, with an incidence difference of 41.1°, a solar azimuth difference of 76.7°, and a minor scale difference. 241 craters (labeled in green) were detected in the left image, 201 craters (labeled in blue) were detected in the right image, and 138 craters were successfully detected in both images (labeled in red), accounting for 57.3% and 68.6%, respectively. This demonstrates that the crater detector obtained through our training is capable of resisting illumination changes.

3.3. Matching Experiments and Results

Three state-of-the-art multimodal matching algorithms proposed in recent years, i.e., HAPCG [19], ML-HLMO [26], and WSSF [28], were tested for MiLOIs as comparison experiments. The implementations of these methods use publicly available code shared by the authors, with parameter settings as recommended by the authors. Additionally, the classic feature matching algorithm SIFT was also used in the experiments. The experiments were performed on a DELL T7920 workstation running Windows 10 x64 operating system, with hardware configurations including an Intel Silver 4216 CPU (2.10GHz), 64GB-RAM, and an NVIDIA GeForce RTX 3090 GPU. The evaluation dataset mentioned in Section 3.1.2 was utilized in the comparison experiments.

3.3.1. Qualitative Evaluation

Three pairs of images from each of the three scenes were selected respectively from the evaluation dataset, making a total of nine pairs of images (as shown in Figure 8), for a qualitative comparison. The matching results are shown in Figure 10, Figure 11 and Figure 12. Blue lines indicate incorrect matches, while yellow lines indicate correct matches.

Scene S1 is located at 44 degrees north latitude. The scale difference between the images in a selected pair is minimal. In the first pair, the incidence angle differs by 17 degrees, and the azimuth angle differs by 88 degrees. In the second pair, the incidence angle differs by 38 degrees, and the azimuth angle differs by 110 degrees. Only the WSSF and CNSFM methods successfully matched these two pairs of images. In the third pair, the incidence angle differs by 21 degrees, and the azimuth angle differs by 22 degrees, with the slightest illumination difference. All five methods successfully matched this pair of images. For the matched image pairs, the CNSFM and WSSF methods show no occurrences of blue lines, indicating higher reliability of the matching results.

Scene S2 is located near the equator where it is close to direct sunlight (very small incidence). A small incidence may also mean a loss of image contrast because of overexposure during imaging. Three pairs of images selected from Scene S2 are varied simultaneously in both scale and illumination, thereby presenting greater matching challenges. The incidence angle differences between the pair of images were 13°, 57°, and 44° respectively, with azimuth angle differences of 179°, 3°, and 176°, and scale differences of 1.7 times, 2.0 times, and 3.4 times. SIFT failed on these three pairs of images. The other four methods were successful in matching the first pair of images, as WSSF and CNSFM demonstrated higher reliability. The second pair with minimal azimuth angle difference but the greatest incidence angle difference was matched successfully using ML-HLMO, WSSF, and CNSFM. WSSF and CNSFM showed the reliable matching outcomes. The third pair of images exhibit significant differences in both illumination and scale, posing the greatest matching difficulty. Only ML-HLMO and CNSFM successfully matched them, but the results of ML-HLMO included numerous incorrect matches.

Scene S3 is near the South Pole, where the solar incidence angle remains consistently close to 90° and the terrain is rugged. The images in S3 exhibit extensive shadowing, and shadow directions vary with azimuth angles, thus presenting the most challenging matching scenario. For the selected three pairs of images from Scene S3, incidence angle differences are less than 2°, scale differences are less than 1.5 times, and azimuth angle differences are 62°, 95°, and 158° respectively. Among the five methods, only CNSFM successfully matched all three pairs and demonstrated high reliability in the matching results.

Based on the qualitative analysis of the multi-illumination lunar orbiter image matching results, among the five methods, SIFT is the most sensitive to illumination changes; HAPCG and ML-HLMO exhibit some resistance to illumination variations but are unstable and prone to mismatches. WSSF shows relatively good resistance to illumination changes, although its performance in polar regions is suboptimal. In contrast, the proposed CNSFM method consistently produces highly reliable matching results, demonstrating exceptional robustness.

3.3.2. Quantitative Evaluation

Table 1. Comparisons on SR Metric (%).

Region	SIFT	HAPCG	ML-HLMO	WSSF	Ours
S1	33.3	66.7	33.3	100	100
S2	24.4	48.9	88.9	66.7	100
S3	17.3	22.1	13.9	31.2	72.3

displays the SR of matching across 321 pairs of images from three scenes using five algorithms. The proposed algorithms achieved 100% SR in Scenes S1 and S2. In Scene S3, the SR is 72.3%, which is 4.2 times higher than SIFT, 3.3 times higher than HAPCG, 5.2 times higher than ML-HLMO, and 2.3 times higher than WSSF. It is evident that the method proposed in this paper is the most robust to illumination differences. Among the other methods, WSSF exhibits the best overall performance.

In feature matching, the positioning accuracy of invariant features is often challenging to determine directly. RMSE based on manually acquired approximate similarity transformation ground truth was used as a quantitative evaluation metric for matching accuracy.

Table 2. The average value of RMSE (pixel).

Region	SIFT	HAPCG	ML-HLMO	WSSF	Ours
S1	3.8	3.2	4.1	1.9	1.0
S2	4.2	3.8	3.1	3.0	1.5
S3	4.3	4.4	4.7	4.1	2.2

presents the average RMSE of five algorithms across datasets from three scenes, while Figure 13 illustrates the RMSE of the matching results for all 321 image pairs. The methods proposed in this paper consistently achieve lower average RMSE values compared to other algorithms.

As seen in the figure, aside from the failed matches, the RMSE of HAPCG and ML-HLMO is mostly concentrated between 2~3 pixels, while WSSF is around 2 pixels. In contrast, the RMSE of the matching results obtained by the proposed method is largely consistent with that of SIFT, generally around 1 pixel. This indicates that craters with lower feature localization accuracy were largely excluded, and the crater features matched by the proposed method exhibit high positioning accuracy.

Additionally, in lunar remote sensing image processing, the reliability of image matching results is crucial.

Table 3. The average value of RCM (%).

Region	SIFT	HAPCG	ML-HLMO	WSSF	Ours
S1	93.3	93.8	90.7	98.8	100
S2	93.2	88.1	62.5	98.6	99.3
S3	71.0	79.4	52.0	92.2	100

presents the average correct matching point ratio for all successfully matched pairs, while Figure 14 visualizes the specific distribution. It can be observed that the proposed method achieves exceptionally high RCM, reaching 100% in regions S1 and S3, and an average accuracy of 99.3% in region S2, outperforming the other four methods. This indicates that the proposed method can provide highly reliable matching results.

Table 4. The average value of NCM.

Region	SIFT	HAPCG	ML-HLMO	WSSF	Ours
S1	47	942	930	877	93
S2	65	2267	1332	567	150
S3	108	564	112	233	57

shows the average values of NCM across 321 pairs of images. Due to the limitation of lunar crater numbers, the NCM values obtained by the proposed method are relatively limited. The other algorithms exhibit significant fluctuations in NCM, ranging from tens to thousands for successfully matched image pairs, resulting in a relatively large average value.

From the matching experiment results, the proposed method demonstrates exceptional robustness to illumination differences and achieves highly reliable matching results. The WSSF method follows, exhibiting a certain degree of resistance to illumination variations, possibly due to its use of structural information. By fully leveraging the structural relationships between features, the proposed method significantly enhances matching robustness under extreme illumination differences.

3.3.3. Parameter Study

The value of parameter K is critical when matching using the proposed method. K determines the size of the neighborhood used to construct the CNSF, which significantly impacts algorithm performance. Other parameters are set based on existing experience. Qualitatively, we can initially analyze the impact of K: smaller K values result in faster computation but weaker robustness to gross errors, leading to mismatches due to similar structures. Increasing the K value increases the number of matches but also substantially increases matching computation time. Moreover, due to various potential errors and deformations, matching accuracy may also be affected.

To quantitatively analyze the impact of K on algorithm performance and determine the most suitable K value for practical applications, experiments with 6 different K values were conducted. The experiments used the evaluation dataset of region S1, evaluating NCM, SR, and MT as metrics. The experimental results are shown in

Table 5. Matching results with difference value of parameter K.

Metric	Values of Parameter K
	5	10	15	20	25	30
Average NCM	25	82	93	93	89	80
SR (%)	93.3	100	100	100	100	100
Average MT(s)	1.3	3.5	23.8	91.1	239.6	530.0

. Figure 15 provides a more intuitive visualization of the matching results for a sample image pair under different K values.

Experimental results indicate that setting the K value too low leads to matching failures, as the number of craters used to construct the CNSF is insufficient, resulting in poor structure robustness. Increasing the K value significantly enhances the matching performance (increased NCM). However, beyond a certain point, the matching performance stabilizes and may even deteriorate if the K value is set too high. Additionally, increasing the K value significantly increases MT. On the basis of these results, it is recommended that the K value be set at 15 or 20 and these settings were maintained in the experiments. And other parameters are set separately according to experience with $\delta =3$, $\eta =25$, ${\xi }_{min}=3$, $ϵ=5$, $\rho =3$.

3.3.4. Ablation Study

To further validate the effectiveness of the mismatched CNSF removal (MCR) algorithm in the proposed method, we conducted an ablation experiment using the dataset from region S1. By comparing the RCM of matching results with and without this algorithm, we assess its effectiveness. The experiment result is shown in

Table 6. Results of ablation experiment.

Metric	CNSFM Without MCR	Full CNSFM
Average RCM (%)	62.2	100

, with two visualization examples presented in Figure 16. The results indicate that the MCR algorithm effectively enhances the reliability of the matching results.

4. Discussion

All image pairs in region S1 and S2 were successfully matched, whereas the success rate in region S3 was not perfect. This is primarily due to the low solar elevation angles near the lunar south pole, where azimuthal differences approaching 180 degrees result in overlapping image regions predominantly covered by shadows (as shown in Figure 17), making direct matching extremely difficult. Since CNSFM relies on a certain number of crater features for structure construction, it may fail when the overlapping area between image pairs is small or heavily shadowed. In addition, in some geologically young units (e.g., light plains or impact melt ponds), craters might be relatively sparse, which can also result in matching failure. These are limitations common to structure-dependent crater feature matching algorithms.

In region S3, the images exhibit small differences in solar incidence angles, with variations primarily in solar azimuth. Figure 18 illustrates the relationship between azimuth differences and successful matching, providing insight into the resistance of different algorithms to changes in illumination azimuth. The proposed method demonstrates the strongest robustness to differences in solar azimuth (147.7°), followed by WSSF (62°) and HAPCG (47°). ML-HLMO (34°) and SIFT (33°) exhibit the weakest resistance.

As shown in Figure 18, successful matches still occur even when the azimuth angle difference exceeds these angles. For the proposed method, this can be attributed to the fact that topographic occlusion-induced shadows do not completely obscure the overlapping regions of the imagery. In contrast, other methods rely on the symmetry of local shadows under such conditions. However, their matching accuracy suffers significantly: at an azimuth difference of 159.1°, the correct matching rates are 78.6% (HAPCG), 34.9% (ML-HLMO), and 58.3% (WSSF), whereas the proposed method achieves 100% accuracy.

5. Conclusions

This paper presented a lunar orbiter image feature matching method that is robust to illumination variations, leveraging CNSF to achieve highly reliable matches. The performance of the proposed CNSFM method was evaluated using a dataset of 321 pairs of MiLOIs from three different latitude regions with maximum incidence angle difference of 78 degrees, maximum solar azimuth angle difference of 180 degrees. The results indicated that the proposed method exhibits superior robustness to illumination variations, achieving a perfect matching success rate of 100% in the equatorial and mid-to-high latitude regions. Even in the polar regions where there are full of shadows due to the ragged terrain and very low sun elevation angles, the method attains a matching success rate of 72.3%, outperforming the 31.2% at best delivered by other methods.

The CNSFM method will be highly suitable for applications in planetary remote sensing image processing applications, such as aligning overlapping images and constructing tie points for block adjustments. The method can be applied to tasks such as landing site mapping and other lunar scientific research.