MMARNet: Two-Stage Remote Sensing Image Registration with Multimodal Attention Mechanism

Highlights

What are the main findings?

• A geometric transformation prediction (GTP) module is developed by utilizing a dynamic adaptive sparse attention mechanism to capture prominent feature regions, thereby enabling accurate estimation and compensation for large-scale geometric transformations between the input images.
• A local feature refinement (LFR) module is constructed by leveraging a feature extraction network with a super token transformer attention mechanism. Therefore, high-precision keypoint-level features extracted by the module can be used to establish accurate correspondences across highly variable modalities.

What are the implications of the main findings?

• The proposed model can be applied in the field of multimodal remote sensing image matching and registration for pixel-level spatial coordinate alignment in multimodal image fusion and change detection.
• The proposed method can also be applied in the field of unmanned aerial vehicle navigation in restricted environments, providing a technical foundation for its cross-modal geographic positioning.

Abstract

Multimodal image registration is a fundamental yet challenging task, particularly in remote sensing scenarios involving cross-platform, multi-temporal, and cross-modal data. The primary difficulty arises from the coexistence of large-scale geometric distortions and complex local appearance variations across modalities, which makes it difficult for a single-stage model to achieve both global alignment and fine-grained correspondence simultaneously. To address this issue, we propose MMARNet, a task-driven coarse-to-fine registration framework that explicitly decomposes multimodal registration into global geometric alignment and local correspondence refinement. Instead of treating registration as a unified problem, the proposed framework sequentially resolves distinct sources of error, leading to improved robustness and accuracy under challenging conditions. In the first stage, MMARNet learns geometry-aware global alignment by identifying structurally reliable regions across modalities and estimating large-scale transformations, effectively reducing the initial misalignment and normalizing the geometric space. In the second stage, the model focuses on residual local discrepancies by learning context-enhanced feature representations, enabling robust keypoint-level matching even under severe modality differences and nonlinear distortions. The two stages are designed to work in a complementary manner, where global alignment significantly simplifies the subsequent local matching process. Extensive experiments on three challenging multimodal datasets demonstrate that MMARNet achieves superior performance in both accuracy and robustness compared to existing methods. The results validate the effectiveness of the proposed problem decomposition and highlight the advantage of the coarse-to-fine optimization strategy for multimodal remote sensing image registration.

1. Introduction

Multimodal image registration is the process of spatially aligning and superimposing images of the same scene acquired from different sensors, at different times, from various viewpoints, or across diverse platforms, aiming to achieve high-precision spatial alignment of multi-source information. This technology holds significant research value in the fields of image processing and computer vision and is widely applied in real-world scenarios, such as visual navigation [1], autonomous driving [2], image fusion [3], geographic information systems [4], and medical imaging analysis [5]. Due to the substantial nonlinear differences between multimodal images, including variations in spectral characteristics, geometric structure, scale, and resolution, multimodal image registration presents considerable challenges, making it a key area of focus in both academic research and engineering applications.

Currently, MRSI registration faces several significant challenges [6,7,8]. Remote sensing images acquired from different viewpoints or platforms (such as airborne or spaceborne sensors) often exhibit large-scale geometric distortions, making it extremely difficult to extract and maintain geometrically invariant features. Additionally, images captured at different times or by different sensors may have significant contrast differences, resulting in inconsistencies in brightness and tonal information, thereby affecting feature-based representations and further complicating precise registration. To address these challenges, researchers have developed various methods for multimodal remote sensing image registration, aiming to improve the robustness and accuracy of registration by adapting to the characteristics and variability inherent to multimodal images.

Traditional feature-based image registration methods [9,10,11,12] achieve geometric alignment of images by extracting edges, corner points, and other geometric features and then solving the matching relationships to complete the registration process. These methods typically consist of four main stages: feature detection, feature description [12,13], feature matching [9], and outlier removal [14]. In these stages, handcrafted feature descriptors are used to identify and align key points in the images. Traditional feature-based methods are effective in handling complex scenarios, such as rotation, scale changes, and illumination variations, by utilizing geometric features like corners and edges for image matching. However, in multimodal image registration, significant differences in contrast, texture, and other factors can cause local features at the same physical location to distort or disappear due to variations in lighting or imaging mechanisms, which increases the difficulty of image registration.

In contrast, deep learning-based methods [15,16,17,18,19,20,21,22] have significantly advanced image registration by learning discriminative representations and more robust transformation priors. The early approaches primarily improve feature extraction or matching efficiency through hybrid designs, such as combining handcrafted cues with CNN-based representations or introducing detector-free matching pipelines. Building on this line, LoFTR [15] formulates dense correspondence estimation in a coarse-to-fine manner using self- and cross-attention, while SE2-LoFTR [21] further improves robustness to rotation variations. More recent methods increasingly emphasize stronger contextual modeling and cross-modal adaptability, including TopicFM [16], XFeat [17], and XoFTR [18]. In multimodal remote sensing scenarios, SwinMatcher [23], DGIM [24], and MINIMA [25] further improve matching robustness through transformer-based interaction, data generation, and modality-invariant learning, respectively, while DS-MAR [26] and GPDRNet [27] explore semantic-level alignment and geometry-preserving refinement for more challenging registrations. Despite these advances, the existing methods still struggle to jointly handle large-scale geometric transformations and residual local misalignments in highly heterogeneous multimodal images.

In this paper, we propose MMARNet, a task-driven coarse-to-fine framework for multimodal remote sensing image registration. As illustrated in Figure 1, the proposed framework explicitly decomposes registration into two complementary stages: global geometric alignment and local correspondence refinement. In the first stage, MMARNet performs geometry-aware coarse alignment by leveraging multi-scale structural consistency and sparse attention to estimate large-scale transformations and reduce severe initial misalignment. In the second stage, the framework refines residual local discrepancies through context-enhanced feature learning and robust keypoint correspondence estimation, enabling accurate matching under complex modality variations. By progressively resolving distinct sources of registration error, MMARNet achieves improved robustness and accuracy for multimodal remote sensing image registration. To better summarize the distinctions between representative registration methods and our approach,

Table 1. Comparison of representative multimodal registration methods and the proposed MMARNet.

Category	Representative Methods	Main Idea	Strengths	Key Difference
Traditional region-based	CC [28], MI [29], and phase correlation [30]	Directly optimize image similarity at the region level	Simple and label-free; effective for roughly aligned pairs	Lack explicit feature modeling and are sensitive to modality gaps, noise, and large deformations
Traditional feature-based	RIFT [31], WSSF [32], and GSMIFT [33]	Use handcrafted detectors/descriptors and feature matching	Improved geometric robustness and radiation invariance	Rely on manually designed features and still struggle under severe appearance variation and large misalignment
Hybrid deep learning methods	Deep feature/similarity learning [34,35]	Replace part of the classical pipeline with learned representations	Improved robustness while preserving conventional optimization steps	Loosely coupled design may limit global consistency and end-to-end adaptability
End-to-end dense matching	MatchNet [36], LIFT [37], LoFTR [15], TopicFM [16], XFeat [17], XoFTR [18], and SwinMatcher [23]	Directly learn correspondences or dense matches from data	Strong automation and matching capability	Typically rely on unified matching pipelines and do not explicitly separate global correction from local refinement
Multimodal remote sensing methods	DGIM [24], MINIMA [25], DS-MAR [26], and GPDRNet [27]	Improve modality invariance, semantic alignment, or geometry- preserving refinement	Better suited for cross-modal remote sensing data	Usually emphasize one aspect of the problem, while MMARNet explicitly decomposes registration into global alignment and local correspondence refinement
Proposed MMARNet	Ours	Task-driven coarse-to- fine registration with global transformation prediction and local feature refinement	Handles both large geometric distortions and residual local misalignment	Explicitly separates global geometric correction from local correspondence refinement

compares their main strategies, strengths, and limitations. Our contributions are summarized as follows:

• We propose a task-driven coarse-to-fine framework for multimodal remote sensing image registration, which explicitly decomposes the registration process into global geometric alignment and local correspondence refinement to address the distinct error characteristics of heterogeneous image pairs.
• We introduce a geometry-aware global alignment mechanism that leverages multi-scale structural consistency and dynamic sparse attention to estimate large-scale transformations under severe modality discrepancies.
• We develop a context-enhanced local refinement mechanism based on super token aggregation to learn robust feature representations and recover accurate keypoint correspondences for fine-grained multimodal registration.

The remainder of this paper is structured as follows: Section 2 introduces related work on registration. Section 3 provides a detailed description of our method. Section 4 presents the experimental results and offers relevant analysis, while Section 5 discusses the limitations and future work. Finally, Section 6 concludes the paper.

2. Related Work

2.1. Image Rectification

Image rectification aims to reduce geometric discrepancies before feature matching and plays an important role in improving registration robustness under large viewpoint, scale, and rotation variations. In multimodal remote sensing scenarios, significant geometric distortions often coexist with severe radiometric differences, making accurate rectification particularly challenging. Traditional rectification methods mainly rely on geometric transformations and handcrafted priors. For example, ASIFT [38] simulates multiple affine views to achieve full affine invariance, significantly improving matching performance under viewpoint changes. MODS [39] further improves efficiency through an adaptive iterative framework that progressively generates intermediate views and transformation hypotheses. Although effective, these approaches often require extensive geometric sampling and may still suffer from reduced robustness in complex multimodal scenarios.

To overcome these limitations, learning-based rectification methods have been increasingly explored. Early studies focused on learning geometric normalization directly from image patches. Yi et al. [40] proposed learning canonical orientations for local patches to facilitate descriptor extraction, while AffNet [41] introduced a learned affine shape estimator that improves geometric normalization and matching repeatability. Rau et al. [42] further demonstrated that overlap-aware scale correction can effectively improve matching accuracy. More recently, Dai et al. [43] proposed an affine estimation unit (AEU) that predicts relative affine transformations for feature pairs, enhancing correspondence robustness. Park et al. [44] designed a two-stream symmetric network with bidirectional ensemble learning for aerial image matching, while ScaleNet [45] introduced scene-independent scale estimation through keypoint distance ratio supervision.

Meanwhile, affine correspondence-based geometric estimation has recently attracted considerable attention. Compared with point correspondences, affine correspondences encode additional local geometric information, providing stronger constraints for transformation recovery. Guan et al. [46] demonstrated that affine correspondences enable efficient relative pose estimation for multi-camera systems using significantly fewer correspondences. Hruby et al. [47] further combined affine correspondences with monocular depth cues for semi-calibrated pose estimation. Yu et al. [48] incorporated affine corrections into monocular depth priors to improve geometric estimation accuracy, while Sun et al. [49] integrated dense matching and geometric constraints to learn high-quality affine correspondences. More recently, Guan et al. [50] proposed a complete solution to generalized relative pose estimation from affine correspondences. These studies highlight the effectiveness of affine correspondences as compact geometric representations that bridge local feature matching and global transformation estimation, providing valuable insights for robust image rectification and registration.

Recent deep learning-based rectification frameworks further integrate geometric correction and registration into unified architectures. ADRNet [51] combines affine and deformable registration modules with spatial transformation learning to jointly address global and local misalignment. Sun et al. [49] further introduce a dense matching framework that incorporates geometric constraints together with learned scale and orientation estimation, achieving accurate affine correspondence generation. Despite these advances, the existing methods still face challenges when simultaneously handling severe modality gaps, large-scale geometric transformations, and residual local misalignments in multimodal remote sensing imagery.

2.2. Registration Methods

Multimodal image registration aims to establish spatial correspondence between images acquired from different sensors, viewpoints, or time instances. The existing methods can be broadly categorized into traditional methods and deep learning-based methods. Traditional methods include region-based and feature-based approaches, while deep learning-based methods can be divided into hybrid approaches and end-to-end methods.

Traditional registration methods rely on handcrafted priors to estimate geometric relationships. Region-based methods directly measure similarity between local patches or image regions using metrics such as cross-correlation (CC) [28], mutual information (MI) [29], and phase correlation [30]. These methods avoid explicit feature extraction but are sensitive to noise, local distortions, and illumination variations, and they are computationally expensive for high-resolution remote sensing images. Feature-based methods instead detect salient structures and establish correspondences via descriptors. Recent advances focus on improving robustness under nonlinear radiometric differences and geometric distortions. For example, RIFT [31] leverages phase congruency and an index map for radiation-invariant matching, while WSSF [32] constructs structural saliency maps using scale-space analysis and phase-aware filtering. Fan et al. [33] further improve robustness by introducing a modality-invariant feature transform with a global-to-local search strategy and adaptive rotation-invariant descriptors. Despite these improvements, handcrafted methods still rely on manually designed features and often struggle under large geometric deformations and severe appearance variations.

Deep learning has significantly advanced image registration by enabling data-driven feature representation and robust transformation estimation. The existing approaches can be divided into hybrid methods and end-to-end methods. Hybrid methods introduce deep models into specific components of conventional pipelines, such as feature extraction or similarity modeling, while retaining geometric optimization. Early works improve robustness by learning feature representations or similarity metrics [34]. More recent approaches further integrate geometric reasoning with deep features, such as learning task-specific keypoints or incorporating semantic-aware and multi-scale representations for multimodal registration [35]. However, their loosely coupled design may limit global consistency. End-to-end methods perform registration by directly predicting correspondences or transformation parameters. Early approaches such as MatchNet [36] and LIFT [37] demonstrate the effectiveness of joint feature learning and matching. More recent works incorporate attention mechanisms and transformer architectures to model long-range dependencies. LoFTR [15] proposes a detector-free coarse-to-fine framework for dense matching via self- and cross-attention. TopicFM [16] introduces topic-aware representations, while XFeat [17] focuses on efficient feature extraction. XoFTR [18] further enhances cross-modal matching through modality-aware pretraining and augmentation strategies. SwinMatcher [23] models cross-modal interactions using a Swin transformer, DGIM [24] improves robustness via dynamic data generation, and MINIMA [25] learns modality-invariant representations through synthetic cross-modal training pairs. More recently, DS-MAR [26] leverages dual-stream multiscale attention with adaptive deformation refinement for semantic-level alignment, while GPDRNet [27] introduces geometry-preserving dense registration to reduce structural distortions. Although these methods achieve improved performance under modality gaps, they still face challenges in jointly handling large-scale geometric transformations and residual local misalignments.

Motivated by these limitations, we propose a coarse-to-fine registration framework that explicitly separates global geometric correction from local correspondence refinement, enabling more accurate and stable multimodal remote sensing image registration.

3. Method

3.1. Coarse-to-Fine Registration Framework

To address the challenges in multimodal image registration, we propose a coarse-to-fine multimodal feature registration strategy named MMARNet, as illustrated in Figure 2. It consists of two main phases: the global transformation prediction (GTP) phase and the local feature refinement (lfr) phase. MMARNet employs a two-stage registration framework for multimodal images. The coarse registration phase predicts global geometric transformations through a GTP module, utilizing multi-scale local consistency features and dynamic sparse attention to achieve initial alignment, reducing initial alignment errors and laying a foundation for geometric consistency. The fine registration phase then applies a super token transformer-enhanced network for keypoint-level feature extraction and matching, resolving residual local discrepancies. By refining alignment from global to local, it effectively handles complex geometric variations in multimodal remote sensing while maintaining computational efficiency.

3.2. Geometric Transformation Prediction (GTP) Module

The top part of Figure 2 illustrates the two-stage process of the proposed geometric transformation prediction (GTP) module for multimodal remote sensing image registration. In the feature extraction stage, we employ a multi-scale feature extractor that captures both local and global features, enhancing the representation of spatial dependencies across different scales. This stage integrates sparsity attention mechanisms to dynamically adjust channel weights, amplifying meaningful features while suppressing irrelevant ones, allowing for more effective feature recalibration and spatial coherence. In the feature regression stage, the extracted features are progressively processed to estimate the geometric transformation parameters required for accurate registration, producing a geometrically consistent output image.

3.2.1. Feature Extraction

Let ${\mathbf{F}}_{\mathrm{in}}$ denote the input feature map of the GTP module. It is first processed by the SE-ResNeXt module [52], which introduces a squeeze-and-excitation (SE) block to dynamically adjust channel weights. The SE block enhances inter-channel dependencies through two steps: squeeze (global average pooling) and excitation (nonlinear mapping with sigmoid). The output feature map ${\mathbf{F}}_{\mathrm{se}}$ is computed as

$${\mathbf{F}}_{\mathrm{se}}={\mathbf{F}}_{\mathrm{in}}·\sigma \left(W_2·\mathrm{ReLU}(W_1·\mathrm{GAP}\left({\mathbf{F}}_{\mathrm{in}}\right))\right),$$

(1)

where GAP denotes global average pooling, and $W_1$, $W_2$ are learnable weights for dimensionality reduction and expansion.

Next, to enhance multi-scale feature extraction, we apply a multi-scale attention mechanism. The feature map ${\mathbf{F}}_{\mathrm{se}}$ is divided into G groups along the channel dimension, each group ${\mathbf{F}}_{\mathrm{se}}^{\left(g\right)}\in {\mathbb{R}}^{C/G\times H\times W}$. For each group, adaptive pooling along height and width yields spatial attention maps ${\mathbf{A}}_h^{\left(g\right)}$ and ${\mathbf{A}}_w^{\left(g\right)}$:

$${\mathbf{A}}_h^{\left(g\right)}=\mathrm{Softmax}\left({\mathrm{Pool}}_h\left({\mathbf{F}}_{\mathrm{se}}^{\left(g\right)}\right)\right),{\mathbf{A}}_w^{\left(g\right)}=\mathrm{Softmax}\left({\mathrm{Pool}}_w\left({\mathbf{F}}_{\mathrm{se}}^{\left(g\right)}\right)\right),$$

(2)

where ${\mathrm{Pool}}_h$ and ${\mathrm{Pool}}_w$ are adaptive pooling operations along height and width. The enhanced group features are obtained by element-wise multiplication:

$${\tilde{\mathbf{F}}}_{\mathrm{se}}^{\left(g\right)}={\mathbf{F}}_{\mathrm{se}}^{\left(g\right)}\odot ({\mathbf{A}}_h^{\left(g\right)}·{\mathbf{A}}_w^{\left(g\right)}).$$

(3)

All groups are concatenated to form the multi-scale feature map ${\mathbf{F}}_{\mathrm{multi}}$. To handle high-resolution images efficiently, we integrate a dynamic adaptive sparse attention mechanism (Figure 3). The input feature map ${\mathbf{F}}_{\mathrm{multi}}$ is first divided into M coarse regions. For each region $R_m$, we compute an importance score based on the aggregated attention response:

$$s_m={\displaystyle \frac{1}{|R_m|}}\sum _{x\in R_m}max{\left(\mathrm{Softmax}(\mathbf{Q}{\mathbf{K}}^{\top }/\sqrt{d})\right)}_x,$$

(4)

where $\mathbf{Q}$, $\mathbf{K}$, and $\mathbf{V}$ are the query, key, and value matrices derived from ${\mathbf{F}}_{\mathrm{multi}}$. We then select the top-k regions with the highest scores:

$${\mathcal{I}}^r=\mathrm{TopK}(s_1, s_2,\dots , s_M),k=\lceil \rho M\rceil ,$$

(5)

where $\rho$ is the retention ratio. In all experiments, we set $\rho =0.25$; i.e., the top 25% most informative regions are retained for fine-grained attention. The corresponding keys and values are gathered as

$${\mathbf{K}}^g=\mathrm{gather}(\mathbf{K}, {\mathcal{I}}^r),{\mathbf{V}}^g=\mathrm{gather}(\mathbf{V}, {\mathcal{I}}^r).$$

(6)

The final attention output is

$${\mathbf{F}}_{\mathrm{dynamic}}=\mathrm{Attention}(\mathbf{Q}, {\mathbf{K}}^g, {\mathbf{V}}^g)+\mathrm{LCE}\left(\mathbf{V}\right),$$

(7)

where $\mathrm{LCE}\left(\mathbf{V}\right)$ denotes a local convolutional embedding that enhances local dependencies.

3.2.2. Feature Regression

To establish dense correspondences between source and target images, we compute a dense correspondence map. Let ${\mathbf{F}}_s$ and ${\mathbf{F}}_t$ denote the source and target feature maps obtained from the extraction stage (both of size $h^{\prime }\times w^{\prime }\times C^{\prime }$). The correspondence map ${\mathbf{C}}_{\mathrm{geo}}\in {\mathbb{R}}^{h^{\prime }\times w^{\prime }\times (h^{\prime }\times w^{\prime })}$ is defined as

$${\mathbf{C}}_{\mathrm{geo}}(i, j, k)=\langle {\mathbf{F}}_s(i, k), {\mathbf{F}}_t(i, j)\rangle ,$$

(8)

where $\langle ·, ·\rangle$ is the inner product, and indices $(i, j)$ and $(i, k)$ denote spatial positions in the target and source feature maps, respectively. Each element of ${\mathbf{C}}_{\mathrm{geo}}$ represents the similarity between a source point and a target point.

The dense correspondence map is then fed into a regression network $\mathcal{R}$ that directly estimates the geometric transformation parameters. The regression network maps the correspondence map to the degrees of freedom (DoFs) of the transformation model:

$$\mathcal{R}:{\mathbb{R}}^{h^{\prime }\times w^{\prime }\times (h^{\prime }\times w^{\prime })}\to {\mathbb{R}}^{\mathrm{DoF}}.$$

(9)

This design enables the network to capture the geometric structure within the feature maps and infer the spatial transformation between images, providing a robust foundation for multimodal registration.

3.2.3. Loss Function for the GTP Module

During training, we adopt the transformed grid loss [53] as the basic supervision for the GTP module. Given the predicted transformation parameters $\widehat{\mathit{\theta }}$ and the ground-truth transformation parameters ${\mathit{\theta }}^{gt}$, the baseline loss is defined as

$${\mathcal{L}}_{\mathrm{grid}}(\widehat{\mathit{\theta }}, {\mathit{\theta }}^{gt})={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\displaystyle \sum _{j=1}^N}d{\left({\mathcal{T}}_{\widehat{\mathit{\theta }}}(x_i, y_j),{\mathcal{T}}_{{\mathit{\theta }}^{gt}}(x_i, y_j)\right)}^2,$$

(10)

where N denotes the number of grid points, ${\mathcal{T}}_{\widehat{\mathit{\theta }}}(·)$ and ${\mathcal{T}}_{{\mathit{\theta }}^{gt}}(·)$ are the transformation operations parameterized by $\widehat{\mathit{\theta }}$ and ${\mathit{\theta }}^{gt}$, respectively, and $d(·,·)$ denotes the Euclidean distance. To enable bidirectional learning, we further introduce supervision in both source-to-target and target-to-source directions. Let ${\widehat{\mathit{\theta }}}_{S\to T}$ and ${\widehat{\mathit{\theta }}}_{T\to S}$ denote the predicted transformations from source to target and from target to source, respectively. The bidirectional loss for the original image pair is defined as

$${\mathcal{L}}_{\mathrm{org}}={\mathcal{L}}_{\mathrm{grid}}({\widehat{\mathit{\theta }}}_{S\to T}, {\mathit{\theta }}_{S\to T}^{gt})+{\mathcal{L}}_{\mathrm{grid}}({\widehat{\mathit{\theta }}}_{T\to S}, {\left({\mathit{\theta }}_{S\to T}^{gt}\right)}^{-1}),$$

(11)

where the inverse transformation is obtained from the isomorphism of the affine transformation, so no additional ground-truth annotation is required for the reverse direction.

For regularization, we also exploit the augmented image pair generated during training. The corresponding loss is formulated as

$${\mathcal{L}}_{\mathrm{aug}}={\mathcal{L}}_{\mathrm{grid}}({\widehat{\mathit{\theta }}}_{S\to T}, {\mathit{\theta }}_{S\to T}^{gt})+{\mathcal{L}}_{\mathrm{grid}}({\widehat{\mathit{\theta }}}_{T\to S}, {\left({\mathit{\theta }}_{S\to T}^{gt}\right)}^{-1}),$$

(12)

where the augmented pair shares the same geometric relation as the original pair. In addition, we introduce an identity loss to encourage the predicted transformations for the original pair and the augmented pair to remain consistent:

$${\mathcal{L}}_{\mathrm{id}}={\mathcal{L}}_{\mathrm{grid}}({\widehat{\mathit{\theta }}}_{S\to T}, {\mathit{\theta }}_{S\to T})+{\mathcal{L}}_{\mathrm{grid}}({\widehat{\mathit{\theta }}}_{T\to S}, {\mathit{\theta }}_{T\to S}),$$

(13)

where ${\mathit{\theta }}_{S\to T}$ and ${\mathit{\theta }}_{T\to S}$ denote the identity transformations for the original and augmented pairs, respectively. The final loss function for the GTP module is defined as

$${\mathcal{L}}_{\mathrm{GTP}}=\alpha {\mathcal{L}}_{\mathrm{org}}+\beta {\mathcal{L}}_{\mathrm{aug}}+\gamma {\mathcal{L}}_{\mathrm{id}},$$

(14)

where $\alpha$, $\beta$, and $\gamma$ are the balance coefficients of the three loss terms. Following the original setting in [44], we set $\alpha =0.5$, $\beta =0.3$, and $\gamma =0.2$. These values are retained to maintain consistency with the baseline design and enable a fair comparison.

3.3. Local Feature Refinement (LFR) Module

The local feature refinement module enhances registration accuracy through three stages: feature point detection, descriptor extraction, and feature matching. The feature point detection layer leverages spatial selection mechanisms and multi-scale large kernels to capture key points while suppressing redundancy. The descriptor extraction layer employs clustering to strengthen global and local representations, making descriptors resilient to nonlinear radiation differences (NRDs). Finally, the feature matching layer predicts a similarity matrix between descriptors to achieve precise alignment.

3.3.1. Feature Detection Layer

The feature point detection layer combines SCconv [54] and LSKNet [55] to extract robust keypoints. Let ${\mathbf{G}}_{\mathrm{in}}$ be the input feature map to this layer.

SCconv Module: It consists of a spatial reconstruction unit (SRU) and a channel reconstruction unit (CRU). The SRU applies group normalization (GN) [56] to normalize spatially variant features:

$${\mathbf{G}}_{\mathrm{sc}}=\mathrm{GN}\left({\mathbf{G}}_{\mathrm{in}}\right)=\gamma {\displaystyle \frac{{\mathbf{G}}_{\mathrm{in}}-\mu }{\sqrt{{\sigma }^2+ϵ}}}+\beta ,$$

(15)

where $\mu$ and $\sigma$ are the mean and standard deviation, and $\gamma ,\beta$ are trainable parameters.

LSKNet Module: It captures multi-scale contextual information using large-kernel convolutions and a spatial selection mechanism. Starting from the input ${\mathbf{G}}_{\mathrm{sc}}$, a series of convolutions with varying kernel sizes k and dilations d produce multi-scale features,

$${\mathbf{G}}_{i+1}={\mathcal{F}}_i^{k,d}\left({\mathbf{G}}_i\right),{\tilde{\mathbf{G}}}_i={\mathcal{F}}_i^{1\times 1}\left({\mathbf{G}}_i\right),$$

(16)

where ${\mathcal{F}}_i^{k,d}$ denotes a convolution with kernel k and dilation d, and ${\tilde{\mathbf{G}}}_i$ is the result of a $1\times 1$ convolution. In our implementation, the large-kernel branch consists of a $5\times 5$ depth-wise convolution followed by a $7\times 7$ depth-wise convolution with dilation rate 3. A spatial attention map ${\mathbf{SA}}_i$ is computed for each scale, and the final output ${\mathbf{G}}_{\mathrm{lsk}}$ is obtained by weighted summation,

$${\mathbf{G}}_{\mathrm{lsk}}=\mathcal{F}\left({\displaystyle \sum _{i=1}^N}({\mathbf{SA}}_i·{\tilde{\mathbf{G}}}_i)\right),$$

(17)

where $\mathcal{F}$ is a convolution layer. The output ${\mathbf{G}}_{\mathrm{lsk}}$ serves as the detected feature points for subsequent descriptor extraction.

3.3.2. Feature Descriptor Layer

The architecture of the descriptor generator is shown in Figure 4. Inspired by the super token transformer (STT) [57], it consists of three sequential components: super token sampling (STS), multi-head self-attention (MHSA), and token upsampling (TU). STS compresses dense visual tokens into a compact set of super tokens to reduce redundancy and computational cost. MHSA captures global contextual dependencies among super tokens, while TU projects the enhanced representations back to the original token space. Through this process, both local details and global semantic information are incorporated into the generated descriptors.

Let ${\mathbf{X}}_{\mathrm{tok}}\in {\mathbb{R}}^{N\times C}$ be the input visual tokens (with $N=H\times W$). STS aggregates them into super tokens ${\mathbf{S}}_{\sup }\in {\mathbb{R}}^{m\times C}$, where $m={\displaystyle \frac{H}{h}}\times {\displaystyle \frac{W}{w}}$ (grid size $h\times w$). Initial super tokens ${\mathbf{S}}_{\sup }^0$ are obtained by averaging tokens within each grid cell. The association map from tokens to super tokens at iteration t is

$${\mathbf{A}}_{\mathrm{map}}^t=\mathrm{Softmax}\left({\displaystyle \frac{{\mathbf{X}}_{\mathrm{tok}}{\left({\mathbf{S}}_{\sup }^{t-1}\right)}^{\top }}{\sqrt{d}}}\right),$$

(18)

where d is the channel dimension. The super tokens are then updated as ${\mathbf{S}}_{\sup }^t={\mathbf{A}}_{\mathrm{map}}^t{\mathbf{X}}_{\mathrm{tok}}$.

In the MHSA stage, self-attention is applied on the super tokens ${\mathbf{S}}_{\sup }$ to capture global dependencies,

$${\tilde{\mathbf{S}}}_{\sup }=\mathrm{Softmax}\left({\displaystyle \frac{q\left({\mathbf{S}}_{\sup }\right)k{\left({\mathbf{S}}_{\sup }\right)}^{\top }}{\sqrt{d}}}\right)v\left({\mathbf{S}}_{\sup }\right),$$

(19)

where $q(·),k(·),v(·)$ are linear projections.

Finally, TU remaps the enhanced super tokens back to the original token space using the association map:

$${\mathbf{X}}_{\mathrm{enhanced}}={\mathbf{A}}_{\mathrm{map}}^t{\tilde{\mathbf{S}}}_{\sup }.$$

(20)

The output ${\mathbf{X}}_{\mathrm{enhanced}}$ is the refined descriptor map, which we denote as ${\mathbf{D}}_{\mathrm{enhanced}}$. This process integrates both local and global context, yielding robust descriptors for matching.

3.3.3. Feature Matching Layer

Given source descriptors ${\mathbf{D}}_s$ and target descriptors ${\mathbf{D}}_t$ (both derived from ${\mathbf{D}}_{\mathrm{enhanced}}$), we compute a similarity matrix ${\mathbf{M}}_{\mathrm{sim}}\in {\mathbb{R}}^{N_s\times N_t}$:

$${\mathbf{M}}_{\mathrm{sim}}^{i,j}=\langle {\mathbf{D}}_s^i,{\mathbf{D}}_t^j\rangle ,$$

(21)

where $N_s=N_t=(1/{\delta }^2)HW$, with $\delta$ being the downsampling factor. The similarity matrix is then normalized using dual-softmax [58] to obtain the correspondence matrix ${\mathbf{C}}_{\mathrm{match}}$:

$${\mathbf{C}}_{\mathrm{match}}^{i,j}=\mathrm{softmax}{\left({\mathbf{M}}_{\mathrm{sim}}^{i,:}\right)}_j·\mathrm{softmax}{\left({\mathbf{M}}_{\mathrm{sim}}^{:,j}\right)}_i.$$

(22)

This matrix serves as a supervisory signal to train the detection and descriptor modules, enabling high-precision multimodal image registration.

3.3.4. Loss Function for the LFR Module

The LFR module is supervised by multiple complementary objectives to jointly optimize keypoint localization, descriptor discriminability, and correspondence estimation. During training, given a source keypoint $p_s$, we warp it to the target image using the known geometric transformation ${\mathcal{T}}_{S\to T}$ and search for its nearest neighbor $p_t$ among the detected target keypoints. If the distance between them is smaller than a predefined threshold $ϵ$, the pair is regarded as a valid correspondence. Based on these pairs, the location loss is defined as

$${\mathcal{L}}_{\mathrm{loc}}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{∥{\mathcal{T}}_{S\to T}\left(p_s^i\right)-p_t^i∥}_2,$$

(23)

where N denotes the number of valid keypoint pairs. To learn robust and discriminative descriptors, we employ a pointwise triplet loss. For each anchor descriptor $D_i$, its matched descriptor $D_i^{+}$ is treated as the positive sample, while the hardest unmatched descriptor $D_i^{-}$ is selected as the negative sample. The loss is formulated as

$${\mathcal{L}}_{\mathrm{tri}}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\left[{∥D_i-D_i^{+}∥}_2-{∥D_i-D_i^{-}∥}_2+m\right]}_{+},$$

(24)

where m is the margin parameter and ${[·]}_{+}$ denotes the hinge function. Furthermore, the predicted correspondence matrix ${\mathbf{C}}_{\mathrm{match}}$ is optimized using a log-likelihood objective. Let ${\mathcal{M}}_{\mathrm{pos}}$ and ${\mathcal{M}}_{\mathrm{neg}}$ denote the sets of positive and negative matches, respectively. The correspondence loss is defined as

$${\mathcal{L}}_{\mathrm{corr}}=-\left({\displaystyle \frac{1}{|{\mathcal{M}}_{\mathrm{pos}}|}}\sum _{(i,j)\in {\mathcal{M}}_{\mathrm{pos}}}log{\mathbf{C}}_{\mathrm{match}}^{i,j}+\gamma {\displaystyle \frac{1}{|{\mathcal{M}}_{\mathrm{neg}}|}}\sum _{(i,j)\in {\mathcal{M}}_{\mathrm{neg}}}log\left(1-{\mathbf{C}}_{\mathrm{match}}^{i,j}\right)\right),$$

(25)

where $\gamma$ is a balancing coefficient. The final loss for the LFR module is given by

$${\mathcal{L}}_{\mathrm{LFR}}={\lambda }_{\mathrm{loc}}{\mathcal{L}}_{\mathrm{loc}}+{\lambda }_{\mathrm{tri}}{\mathcal{L}}_{\mathrm{tri}}+{\lambda }_{\mathrm{corr}}{\mathcal{L}}_{\mathrm{corr}},$$

(26)

The weighting coefficients are set to ${\lambda }_{\mathrm{loc}}=0.1$, ${\lambda }_{\mathrm{tri}}=4.0$, and ${\lambda }_{\mathrm{corr}}=0.5$, following the default configuration of the LFR backbone proposed in [59]. We retain these values to ensure consistency with the original design and facilitate fair performance evaluation.

4. Experiments

In this section, we will first provide a detailed description of the dataset used and the specific method for generating image pairs. Next, we conduct a systematic comparison and analysis between the proposed method and existing state-of-the-art approaches. Finally, through a series of ablation experiments, we thoroughly examine the importance of each strategy and module introduced in the model to verify their critical roles in enhancing overall performance.

4.1. Datasets and Implementation Details

Train Datasets

• Google Earth Dataset [60]: This dataset contains 9042 pairs of images with a size of 1080 × 1080 pixels. Each pair consists of a visible light remote sensing image and a high-resolution multispectral and panchromatic remote sensing image of the same location taken at different times. It is used to train our geometric transformation prediction network.
• MegaDepth Dataset [61]: The dataset is a large-scale image matching and 3D reconstruction dataset released in 2018. MegaDepth contains nearly one million internet photos covering multiple cities and landmark buildings around the world. These photos exhibit a wide range of viewpoints, lighting, scale differences, and occlusions, all of which pose significant challenges for image matching. Our feature matching network is trained on a subset of the MegaDepth dataset.

Test Dataset

• Google Earth Dataset: Used for both training and testing.
• High-Resolution Remote Sensing Dataset (PatternNet) [62]: PatternNet is a large-scale high-resolution remote sensing dataset specifically collected for remote sensing image retrieval. The dataset contains 38 categories, with 800 images per category, each of size 256 × 256 pixels. The images in PatternNet are sourced from Google Earth or collected via the Google Map API from certain cities in the United States.
• Visible Light–Infrared Image Dataset (VIS–NIR) [63]: This dataset contains 319 pairs of visible light and infrared images, mainly focusing on urban and coastal scenes.

Training Details

All the experiments were conducted on an Nvidia GeForce RTX 4090. The two modules are trained independently. For the geometric transformation prediction network, we implemented the model using PyTorch 2.0.1 and trained it with the ADAM optimizer [64] (learning rate $5\times {10}^{-4}$, batch size 8) for 20 epochs (about 2 days), applying random affine transformations as data augmentation and resizing input images to $540\times 540$. For the feature matching network, we used 118k images from COCO 2017 [65] without human annotations, applying spatial augmentation (scaling, rotation, and perspective) to generate self-supervised signals for Siamese training. Input images were resized to $240\times 320$, and the network was trained with ADAM (learning rate $3\times {10}^{-4}$, batch size 4) for 20 epochs (about 3 days); the learning rate was halved after 4 and 8 epochs. In the testing stage, image pairs are generated following the procedure shown in Figure 5. Specifically, an original image is randomly selected from the test dataset, and a random affine transformation (rotation $[-{90}^{\circ }, {90}^{\circ }]$, scaling $[0.85, 1.3]$) is applied to produce the corresponding target image. The generated image pair is then used as input for registration prediction, while the applied transformation serves as the ground-truth geometric relationship for quantitative evaluation. For evaluation, PCK is computed under a pixel threshold of 5 pixels, and MAE/RMSE are measured in pixel units. SSIM and PSNR are calculated between the warped image and the ground-truth target image.

4.2. Quantitative Comparisons

4.2.1. Image Similarity Evaluation

Table 2. Comparison of SSIM and PSNR metrics on different datasets. Bold values indicate the best result.

Method	Google Earth		PatternNet		VIS–NIR
	PSNR↑	SSIM↑	PSNR↑	SSIM↑	PSNR↑	SSIM↑
SIFT [9]	18.34	0.41	27.44	0.86	21.78	0.75
BRISK [12]	17.17	0.40	28.16	0.90	19.20	0.59
ORB [11]	13.55	0.29	24.48	0.80	10.45	0.23
ReDfeat [66]	14.36	0.27	13.01	0.26	9.36	0.08
XoFTR [18]	14.40	0.27	12.99	0.24	9.37	0.08
TopicFM [16]	22.82	0.60	21.16	0.59	16.90	0.45
LoFTR [15]	22.38	0.61	22.32	0.59	16.32	0.45
ASpanFormer [67]	22.88	0.60	22.60	0.62	17.24	0.47
XFeat [17]	21.18	0.52	20.44	0.58	16.57	0.50
MINIMA [25]	23.15	0.71	27.45	0.90	24.35	0.83
SwinMatcher [23]	23.40	0.73	27.20	0.88	25.10	0.86
DGIM [24]	23.70	0.75	27.80	0.91	24.00	0.81
MMARNet	24.75	0.79	28.24	0.93	26.19	0.92

reports the SSIM and PSNR results on the Google Earth, PatternNet, and VIS–NIR datasets. These two metrics evaluate the structural fidelity and visual quality of the registered images, and they provide a complementary assessment of registration performance.

The structural similarity index measure (SSIM) reflects the degree of structural preservation after registration. As shown in Table 2, traditional feature-based methods generally obtain lower SSIM values. For example, SIFT and BRISK achieve SSIM scores of 0.41 and 0.40 on Google Earth, respectively, while ORB only reaches 0.29. Although these methods perform better on PatternNet, their results on VIS–NIR remain limited, with ORB achieving only 0.23 and ReDfeat and XoFTR producing even lower values of 0.08. Recent learning-based methods show improvements over earlier approaches. TopicFM, LoFTR, ASpanFormer, and XFeat achieve moderate SSIM values, ranging from 0.45 to 0.62 on the three datasets. More recent methods further improve structural consistency: MINIMA achieves 0.71, 0.90, and 0.83 on Google Earth, PatternNet, and VIS–NIR, respectively; SwinMatcher obtains 0.73, 0.88, and 0.86; and DGIM reaches 0.75, 0.91, and 0.81. Among all the compared methods, MMARNet consistently achieves the best SSIM performance, with 0.79 on Google Earth, 0.93 on PatternNet, and 0.92 on VIS–NIR. These results indicate that MMARNet preserves structural information more effectively, especially in challenging multimodal scenarios.

The peak signal-to-noise ratio (PSNR) measures the visual quality and noise suppression capability of the registered results. Traditional feature-based methods produce relatively low PSNR values on the three datasets. For instance, ORB obtains only 13.55 on Google Earth and 10.45 on VIS–NIR, while ReDfeat and XoFTR yield even lower PSNR values of 14.36/14.40 on Google Earth and 9.36/9.37 on VIS–NIR, respectively. Compared with these methods, recent learning-based approaches exhibit clearer improvements. TopicFM, LoFTR, ASpanFormer, and XFeat reach PSNR values of 21.16–22.88 on Google Earth, 20.44–22.60 on PatternNet, and 16.32–17.24 on VIS–NIR. The newly added recent methods further strengthen the performance: MINIMA achieves 23.15, 27.45, and 24.35, SwinMatcher obtains 23.40, 27.20, and 25.10, and DGIM reaches 23.70, 27.80, and 24.00 on the three datasets, respectively. MMARNet still achieves the highest PSNR values across all the datasets, with 24.75 on Google Earth, 28.24 on PatternNet, and 26.19 on VIS–NIR. These results demonstrate that MMARNet provides stronger noise suppression and better visual consistency after registration.

In summary, MMARNet achieves the best overall performance in both SSIM and PSNR. The results show that the proposed method not only preserves structural information more effectively but also maintains higher image quality after registration, which confirms its robustness in multimodal remote sensing scenarios.

4.2.2. Registration Accuracy Evaluation

PCK (percentage of correct keypoints) [68], MAE (mean absolute error) [28] and RMSE (root mean squared error) are three widely used metrics for evaluating image matching performance [69]. To provide a more comprehensive assessment of geometric registration accuracy, the results are reported in

Table 3. Comparison of PCK, MAE and RMSE metrics on different datasets. Bold values indicate the best result, and underlined indicate the second-best result.

Method	Google Earth			PatternNet			VIS–NIR
	PCK↑	MAE↓	RMSE↓	PCK↑	MAE↓	RMSE↓	PCK↑	MAE↓	RMSE↓
SIFT	78.72	3.64	5.21	94.96	0.83	3.45	84.09	2.78	5.83
BRISK	60.21	10.63	14.70	96.43	0.85	3.52	67.83	6.29	8.96
ORB	22.70	16.82	18.31	93.59	1.21	4.98	42.15	12.35	14.22
TopicFM	99.70	0.80	2.28	42.20	13.01	15.02	36.05	11.76	14.31
LoFTR	99.49	0.85	2.36	44.12	11.42	13.89	41.35	12.02	13.88
ASpanFormer	99.69	0.81	2.30	59.84	8.23	9.56	46.81	10.86	12.26
XFeat	98.87	1.48	3.11	50.26	10.73	12.93	51.25	10.58	11.92
MINIMA	99.92	0.52	1.88	97.80	0.82	3.01	97.50	2.10	5.54
SwinMatcher	99.85	0.36	1.56	98.50	0.85	3.28	96.80	1.95	5.12
DGIM	99.90	0.38	1.49	99.11	0.78	2.92	97.20	2.20	5.85
MMARNet	99.99	0.32	1.28	99.11	0.75	2.35	98.00	0.96	3.78

. The results on the Google Earth, PatternNet, and VIS–NIR datasets demonstrate the effectiveness of MMARNet under both intra-modal and cross-modal scenarios.

Traditional methods such as SIFT [9] and BRISK [12] achieve moderate performance on the benchmark datasets. For example, SIFT obtains relatively high PCK scores on PatternNet (94.96) and VIS–NIR (84.09), but its MAE and RMSE on VIS–NIR remain 2.78 and 5.83, respectively, indicating limited robustness under severe modality variation. BRISK shows a similar tendency, with a PCK of 96.43 on PatternNet but a much larger MAE of 6.29 and RMSE of 8.96 on VIS–NIR. ORB [11] performs poorly on all three datasets, particularly on Google Earth, where it only achieves a PCK of 22.70 and the highest MAE/RMSE values among the traditional methods, confirming its limited effectiveness in multimodal registration.

Deep learning-based methods generally improve matching performance, but their results vary significantly across datasets. TopicFM [16], LoFTR [15], ASpanFormer [67], and XFeat [17] achieve strong performance on Google Earth, with PCK values above 98.00 and relatively low MAE/RMSE values. However, their performance drops substantially on PatternNet and VIS–NIR, especially for TopicFM and LoFTR, whose PCK values fall below 45.00 on PatternNet and below 42.00 on VIS–NIR. This indicates that, although these methods are effective for relatively easier alignment cases, their robustness remains limited when facing larger cross-modal discrepancies and more challenging geometric transformations.

Recent methods, including MINIMA [25], SwinMatcher [23], and DGIM [24], show improved robustness over earlier deep models. MINIMA achieves PCK scores of 99.92 on Google Earth and 97.50 on VIS–NIR, while SwinMatcher and DGIM further improve the overall registration quality across the three datasets. In particular, DGIM obtains the highest PCK on PatternNet (99.11), indicating strong performance on this more challenging dataset. Nevertheless, MMARNet still delivers the most consistent overall results. It achieves the best PCK on Google Earth (99.99) and VIS–NIR (98.00) and remains highly competitive on PatternNet (99.11), where it is only slightly lower than DGIM. More importantly, MMARNet achieves the lowest MAE on all three datasets, with values of 0.32, 0.75, and 0.96, respectively, and also the lowest RMSE, with 1.28 on Google Earth, 2.35 on PatternNet, and 3.78 on VIS–NIR. These results indicate that MMARNet not only improves the average alignment accuracy but also reduces large registration deviations, leading to more reliable geometric consistency.

Overall, the experimental results verify that MMARNet provides superior registration performance across all three datasets. Compared with both traditional methods and recent learning-based baselines, the proposed method achieves the most favorable trade-off between keypoint accuracy, average error, and overall geometric stability, demonstrating its robustness in multimodal remote sensing image registration.

4.2.3. Qualitative Comparison

Figure 6, Figure 7 and Figure 8 provide qualitative comparisons on the Google Earth, PatternNet, and VIS–NIR datasets. For clearer visualization, checkerboard overlays and zoomed-in regions are used to highlight local alignment details. Overall, MMARNet achieves the most consistent registration results across all three datasets, particularly in challenging cases involving large geometric transformations and strong modality differences.

On the Google Earth dataset, the geometric discrepancy between image pairs is relatively moderate, and the modality gap is also less pronounced. As a result, most methods are able to produce visually acceptable alignment, and the performance differences are comparatively limited. Nevertheless, several baseline methods still exhibit slight boundary shifts and local discontinuities in the enlarged regions, whereas MMARNet produces cleaner structural continuity and more accurate boundary alignment.

In contrast, the PatternNet dataset involves more significant rotation variations, which makes the registration task considerably more difficult. Under such conditions, traditional methods and earlier deep learning approaches begin to suffer from noticeable ghosting, edge distortion, and incomplete alignment. Although MINIMA, SwinMatcher, and DGIM improve the overall results to some extent, residual misalignment remains visible in the red-box regions. MMARNet, by comparison, better suppresses these artifacts and maintains more precise structural correspondence.

The VIS–NIR dataset is the most challenging one as it simultaneously presents substantial modality differences, scale variation, and rotation changes. On this dataset, most competing methods fail to achieve reliable alignment and produce obvious misregistration or severe structural inconsistency. Even the recent baselines still show clear ghosting and incomplete correspondence in the zoomed-in regions. In contrast, MMARNet consistently preserves geometric structure and achieves much better alignment quality. These results clearly demonstrate the effectiveness of the proposed coarse-to-fine strategy in progressively reducing global misalignment and refining local correspondence under highly complex multimodal conditions.

4.3. Ablation Study

4.3.1. Geometric Transformation Prediction Module

Table 4. Ablation results on Google Earth dataset.

Method	PCK (%)
	$\mathit{\tau }=\mathbf{0}.\mathbf{05}$	$\mathit{\tau }=\mathbf{0}.\mathbf{03}$	$\mathit{\tau }=\mathbf{0}.\mathbf{01}$
Baseline	94.5	82.4	31.5
Baseline + A	94.5	81.7	32.9
Baseline + B	96.2	86.2	37.1
Baseline + A + B	95.2	86.8	39.6

presents the ablation study results on the Google Earth dataset, using PCK as the evaluation metric to systematically measure the contributions of each improvement module at various thresholds ($\tau =0.05,0.03,0.01$). In this experiment, the multi-scale local consistency feature extraction module is denoted as “A”, the dynamic adaptive sparsity attention mechanism as “B” and the unmodified baseline network as “baseline”. Table 4 comprehensively illustrates the specific contributions of each module to the image matching task. Specifically, the baseline model shows a gradual decrease in PCK as the threshold decreases, reaching only 31.5% at $\tau =0.01$, indicating the limitations of the baseline model in high-precision registration tasks. When the multi-scale local consistency feature extraction module is introduced (baseline + A), the PCK at the lower threshold ($\tau =0.01$) slightly increases to 32.9%, demonstrating a certain effect on improving detail feature matching. When the dynamic adaptive sparsity attention mechanism (baseline + B) is introduced, the PCK significantly increases across all thresholds, especially reaching 86.2% and 37.1% at $\tau =0.03$ and 0.01, respectively. This suggests that the dynamic adaptive sparsity attention mechanism has a significant advantage in effectively focusing on key regions and suppressing redundant information.

Finally, by integrating both the multi-scale local consistency feature extraction module and the dynamic adaptive sparsity attention mechanism into the baseline model to form the complete MMARNet model, the best performance is achieved across all thresholds, especially reaching 39.6% at $\tau =0.01$, which significantly outperforms other configurations. This validates the synergy of the two modules, enabling MMARNet to achieve more accurate image matching.

4.3.2. Local Feature Refinement Module

Table 5. Ablation results on HPatches dataset.

Method	Re↑	LE↓	MS↑	H-1↑	H-3↑	H-5↑
Baseline	0.690	1.019	0.538	0.595	0.869	0.926
Baseline + A	0.689	1.015	0.561	0.536	0.866	0.919
Baseline + B	0.666	0.984	0.522	0.588	0.874	0.928
Baseline + A + B	0.689	0.951	0.575	0.617	0.881	0.929

presents the ablation study results on the HPatches dataset, evaluating the performance impact of incorporating the enhanced feature extraction module (denoted as “A”) and the super token transformer (denoted as “B”) on the baseline model. As shown in the table, the baseline model demonstrates moderate performance across all the metrics, with an Re value of 0.690, LE of 1.019, MS of 0.538, H-3 of 0.869, and H-5 of 0.926, indicating certain limitations when dealing with complex transformations. After introducing the feature extraction module (baseline + A) the LE metric improves by approximately 0.4% and MS improves by 4.3%, indicating that the feature extraction module helps to refine local features and enhance local matching accuracy. Further, adding the super token transformer (baseline + B) results in more substantial improvements across all the metrics. Compared to the baseline model, the Re value decreases by about 3.5%, the LE value decreases by 3.4%, the MS metric improves by 9.6%, and the H-3 and H-5 metrics increase by 0.6% and 0.2%, respectively. These results demonstrate that the super token transformer plays a significant role in focusing on key feature regions and enhancing matching consistency, thereby greatly improving robustness to complex transformations.

When both the feature extraction module and the super token transformer are combined, the model achieves optimal performance with notable overall improvement ratios. Compared to the baseline model, the LE metric decreases by 6.7% and MS improves by 6.9%. These results indicate that the synergistic effect of the two modules significantly enhances the accuracy and stability of image matching, achieving optimal registration results.

5. Discussion

The experimental results demonstrate that MMARNet effectively addresses multimodal remote sensing image registration through a coarse-to-fine strategy. By decomposing the task into global geometric alignment and local feature refinement, the framework achieves robust performance under large geometric distortions and significant cross-modal appearance variations. The global alignment stage compensates for large-scale transformations and reduces the search space for matching, while the local refinement stage further establishes accurate correspondences at the feature level. Compared with existing single-stage approaches, this hierarchical design improves both registration accuracy and robustness across different multimodal datasets.

Despite these advantages, several limitations remain. As a sequential framework, MMARNet is still sensitive to errors in the initial global transformation estimation, which may propagate to the local refinement stage and affect the final registration result. This issue becomes more pronounced in cases involving extremely large rotations, severe viewpoint changes, or limited overlap between image pairs. Future work will focus on jointly optimizing the two stages to reduce error propagation, improve robustness under extreme conditions, and extend the framework to more challenging multimodal and multi-temporal remote sensing applications.

6. Conclusions

In this paper, we introduce a new multimodal remote sensing image registration method called MMARNet. MMARNet is designed with a two-stage framework to achieve high-precision remote sensing image registration. In the first stage, MMARNet incorporates a geometric transformation prediction (GTP) module, which estimates and compensates for large-scale transformations, such as scaling and rotation, providing a geometrically consistent foundation for subsequent registration. In the second stage, MMARNet integrates a deep learning-based feature matching network (LFR) to perform pixel-level feature extraction and robust matching, enabling precise local alignment across modalities. Extensive experimental results demonstrate that MMARNet achieves state-of-the-art accuracy and robustness in multimodal image registration, validating its significant potential and superiority in the field of multimodal remote sensing image registration.