A Coarse-to-Fine Optical-SAR Image Registration Algorithm for UAV-Based Multi-Sensor Systems Using Geographic Information Constraints and Cross-Modal Feature Consistency Mapping

Highlights

What are the main findings?

• A coarse-to-fine registration framework integrating geographic information constraints with cross-modal feature consistency mapping is proposed, achieving automatic elimination of scale differences and rotation deviations between optical and SAR images through imaging geometry-based coordinate transformation.
• On the integrated airborne optical/SAR dataset, the method achieves 2.00 (CPU)/1.97 (GPU) pixels in average RMSE, outperforming traditional and state-of-the-art deep learning methods while reducing computation time by 37.0%; its geographic-constrained coarse registration improves SuperGlue/LoFTR CMR by 167%/109%, and the hybrid GLS + LightGlue refinement yields the optimal 1.95-pixel RMSE with GPU acceleration.

What are the implications of the main findings?

• The geographic information constraint approach circumvents radiometric differences that challenge both traditional intensity-based methods and data-driven deep learning approaches, providing a practical registration framework for UAV-based multi-sensor fusion systems in GNSS-aided scenarios.
• The method demonstrates robust performance under challenging conditions (dense smoke, fire, illumination variations) and provides the best accuracy–efficiency trade-off for resource-constrained UAV platforms, enabling real-time applications in environmental monitoring, disaster assessment, target detection, and visual navigation tasks.
• The demonstrated complementary benefits between geometric constraints and learned features suggest promising directions for future hybrid approaches combining physics-based geometric priors with data-driven feature representations for cross-modal image matching.

Abstract

Optical and synthetic aperture radar (SAR) image registration faces challenges from nonlinear radiometric distortions and geometric deformations caused by different imaging mechanisms. This paper proposes a coarse-to-fine registration algorithm integrating geographic information constraints with cross-modal feature consistency mapping. The coarse stage employs imaging geometry-based coordinate transformation with airborne navigation data to eliminate scale and rotation differences. The fine stage constructs a multi-scale phase congruency-based feature response aggregation model combined with rotation-invariant descriptors and global-to-local search for sub-pixel alignment. Experiments on integrated airborne optical/SAR datasets demonstrate superior performance with an average RMSE of 2.00 pixels, outperforming both traditional handcrafted methods (3MRS, OS-SIFT, POS-GIFT, GLS-MIFT) and state-of-the-art deep learning approaches (SuperGlue, LoFTR, ReDFeat, SAROptNet) while reducing execution time by 37.0% compared with the best-performing baseline. The proposed coarse registration also serves as an effective preprocessing module that improves SuperGlue’s matching rate by 167% and LoFTR’s by 109%, with a hybrid refinement strategy achieving 1.95 pixels RMSE. The method demonstrates robust performance under challenging conditions, enabling real-time UAV-based multi-sensor fusion applications.

1. Introduction

With the rapid advancement of remote sensing technology, the acquisition and comprehensive utilization of multimodal remote sensing imagery have become increasingly critical in various applications, including environmental monitoring, disaster assessment, target detection, and visual navigation [1,2,3]. Among different imaging modalities, synthetic aperture radar (SAR) and optical (visible light) imagery provide complementary information: SAR sensors operate independently of weather conditions and illumination, offering all-weather, day-and-night imaging capabilities with rich structural information, while optical sensors capture high-resolution spectral information consistent with human visual perception [4,5]. The fusion of SAR and optical imagery can significantly enhance the performance of remote sensing applications by leveraging the advantages of both modalities [6,7].

Image registration establishes spatial correspondence between images captured from different sensors, viewpoints, or times, serving as a fundamental prerequisite for multimodal image fusion [8,9]. However, registration of SAR and optical imagery remains a challenging task due to significant differences in imaging mechanisms [10,11]. These differences manifest in several aspects: (1) nonlinear radiometric distortions (NRD) caused by different spectral responses; (2) geometric deformations resulting from different viewing angles and acquisition conditions; (3) speckle noise inherent in SAR images that reduces feature detection accuracy; and (4) nonlinear intensity mappings of structural features, where edges, corners, and textures exhibit different appearances across modalities despite representing the same physical structures [12,13,14].

In unmanned aerial vehicle (UAV)-based detection systems, the integration of optical cameras and SAR sensors introduces unique challenges and opportunities [15,16]. UAV platforms provide flexible deployment and high-resolution data acquisition capabilities, but airborne sensors typically have limited accuracy in position and attitude measurements [17]. Moreover, UAV-based optical and SAR images often exhibit significant scale differences and rotation deviations due to different sensor mounting configurations and imaging geometries, which substantially increases the search space for feature matching, as shown in Figure 1. This limitation has motivated the development of advanced image registration algorithms that can achieve precise alignment without complete reliance on navigation data [18,19].

Figure 1. Illustration of UAV-based multi-sensor imaging system and cross-modal image characteristics. (a) Diagram of the UAV multi-sensor imaging system showing the visible light camera (nadir view) and SAR sensor (side-looking view) with their respective imaging geometries. (b) Optical image with high spectral resolution, capturing detailed color and texture information but dependent on weather and illumination conditions. (c) SAR image of the same geographic area with all-weather, day-and-night imaging capability but exhibiting speckle noise. The comparison highlights the complementary nature of optical and SAR imagery and the challenges in cross-modal registration due to different representations of the same ground features (water body, building, and vegetation).

Traditional feature-based registration methods, such as scale-invariant feature transform (SIFT) [20] and speeded-up robust features (SURF) [21], have achieved remarkable success in registering homogeneous images. However, these methods primarily rely on intensity gradient information, which exhibits significant variations between SAR and optical images due to the presence of NRD [22,23]. To address this challenge, researchers have proposed various approaches, including modified gradient definitions [24], structural feature descriptors [25], and phase congruency-based methods [26,27]. Recent years have witnessed the emergence of deep learning-based registration methods, such as SuperGlue, LoFTR, and their variants, which have achieved impressive performance in homogeneous image matching. However, these learning-based approaches typically require large-scale training datasets with accurate ground truth correspondences, which are difficult to obtain for SAR-optical image pairs. Furthermore, their computational demands often exceed the real-time processing requirements of UAV-based systems. Despite these advances, achieving efficient, robust, and accurate SAR-optical image registration remains an open problem.

Existing SAR-optical registration methods can be broadly categorized into two paradigms: (1) direct feature matching approaches that attempt to extract modality-invariant features from raw images and (2) image translation approaches that convert one modality to another before matching. The first paradigm faces the fundamental challenge of bridging the radiometric gap between modalities, often resulting in low matching rates. The second paradigm introduces additional complexity and potential artifacts from the translation process. In contrast, this paper proposes a third paradigm that leverages geographic information as a modality-invariant bridge. By transforming the registration problem from the image domain to the geographic coordinate domain, we circumvent the radiometric differences that challenge traditional intensity-based methods. This geographic constraint-based approach is particularly suitable for UAV platforms equipped with navigation sensors (GPS/IMU), which provide readily available pose information.

This paper proposes a novel coarse-to-fine image registration algorithm that integrates geographic information constraints with cross-modal feature consistency mapping for optical and SAR image alignment. Unlike existing methods that rely solely on image features, our approach exploits the complementary strengths of navigation data and image-based refinement. The main contributions are as follows:

(1)

A geographic information-constrained coarse registration strategy is proposed, which leverages imaging geometry-based coordinate transformation with UAV pose data to establish initial correspondence between images, effectively eliminating most scale differences and rotation deviations. This strategy reduces the matching search space by orders of magnitude compared with direct global matching.

(2)

A cross-modal feature consistency descriptor based on multi-scale Gaussian filtering and feature response aggregation is developed to overcome nonlinear radiometric differences between SAR and optical images. The descriptor exploits phase congruency theory to extract radiation-invariant structural features that exhibit consistent responses across modalities.

(3)

A global-to-local search (GLS) matching strategy is designed to improve matching efficiency while maintaining high accuracy. By constraining the search space using geometric predictions from coarse registration, the GLS strategy achieves 37% reduction in computational time compared with exhaustive search methods.

(4)

Comprehensive experimental validation demonstrates the effectiveness of the proposed algorithm, showing superior performance compared with state-of-the-art methods, including both traditional handcrafted feature methods (3MRS, OS-SIFT, POS-GIFT, GLS-MIFT) and deep learning-based approaches, with an average RMSE of 2.0042 pixels on integrated airborne optical/SAR datasets.

The remainder of this paper is organized as follows: Section 2 reviews related work on multimodal image registration; Section 3 details the proposed algorithm; Section 4 describes the experimental setup and results; and Section 5 concludes the paper.

2. Related Works

Multimodal image registration has been extensively studied in the remote sensing community. Existing approaches can be broadly categorized into area-based methods, feature-based methods, and deep learning-based methods [28,29]. This section reviews representative techniques in each category, with particular focus on SAR-optical image registration.

2.1. Area-Based Registration Methods with Similarity Metrics

Area-based methods establish correspondence between images by comparing intensity patterns within local windows using similarity metrics such as normalized cross-correlation (NCC), mutual information (MI), and sum of squared differences (SSD) [30,31]. Mutual information, derived from information theory, measures the statistical dependence between image intensities and has been widely adopted for multimodal image registration due to its robustness to intensity variations [32,33]. Suri and Reinartz [34] proposed a mutual information-based approach for registering TerraSAR-X and IKONOS images, achieving sub-pixel accuracy in urban areas. However, area-based methods typically require good initial alignment and become computationally expensive when searching over large displacement ranges [35].

To enhance the robustness of area-based methods against radiometric differences, researchers have explored various similarity metrics. Hel-Or et al. [36] introduced matching by tone mapping (MTM), which can handle nonlinear intensity transformations between images. Chen et al. [37] combined quantum particle swarm optimization with MI to achieve improved registration performance. Despite these advances, area-based methods remain sensitive to geometric distortions and local content variations, limiting their applicability to SAR-optical image registration with significant structural differences [37].

2.2. Feature-Based Registration Methodss

Feature-based methods detect salient features in images and establish correspondence based on feature descriptors [38]. The scale-invariant feature transform (SIFT) [20] has become a benchmark algorithm due to its invariance to scale and rotation. However, SIFT’s gradient information is sensitive to nonlinear radiometric distortions between multimodal images [39,40].

To address this limitation, several SIFT variants have been proposed for SAR image registration. Dellinger et al. [41] developed SAR-SIFT using the ROEWA operator to suppress speckle noise. Xiang et al. [42] proposed OS-SIFT, constructing separate Harris scale spaces for SAR and optical images. Fan et al. [43] introduced nonlinear diffusion filtering to better preserve edge information.

Phase congruency-based methods have emerged as promising solutions for multimodal matching. Kovesi [44] proposed the phase congruency model providing illumination-invariant feature detection. Building upon this, Li et al. [45] developed RIFT using maximum index map descriptors, demonstrating excellent cross-modal performance [46]. Related approaches include LHOPC [47] and LGHD [48] descriptors designed for multimodal matching.

The KAZE algorithm [49] introduced nonlinear scale space construction using additive operator splitting, with its accelerated version AKAZE [50] employing fast explicit diffusion for improved efficiency. These algorithms have inspired numerous variants for multimodal image registration [51,52].

2.3. Geographic Information Constraints for UAV-Based Registration

Geographic information from navigation systems (GNSS/GPS) and inertial measurement units (IMUs) can provide valuable prior knowledge for image registration [53,54]. Zhuo et al. [55] proposed an automatic UAV image geo-registration method that matches UAV images with georeferenced aerial imagery using geometric constraints derived from GNSS and IMU data. This approach significantly reduces the search space for feature matching and improves registration robustness in challenging scenes with repetitive patterns [56].

For UAV-based multi-sensor systems, the integration of geographic information with image registration has received increasing attention [57,58]. Coarse-to-fine registration strategies that first use geographic information for coarse alignment and then refine the registration through precise feature matching have shown superior performance when compared with direct matching approaches [59,60].

Recent advances in visual geo-localization have explored hierarchical approaches combining image retrieval and local feature matching [61,62]. These methods typically employ a coarse localization stage based on global image descriptors, followed by fine localization using precise feature correspondences [63]. The integration of terrain information and geometric constraints has been shown to improve localization accuracy in GNSS-denied environments [64,65].

3. Methodology

The optical-SAR image registration problem aims to find the optimal geometric transformation achieving best spatial alignment between the two modalities. This problem faces three core challenges: nonlinear radiometric mapping between different imaging mechanisms; complex geometric transformations involving scale, rotation, and translation; and an enormous non-convex search space. To address these challenges, we propose a hierarchical framework comprising coarse and fine stages, as shown in Figure 2. The coarse stage leverages airborne geographic information via imaging geometry-based coordinate transformation to eliminate scale differences and rotation deviations. The fine stage employs multi-scale phase congruency features with rotation-invariant descriptors and global-to-local search for sub-pixel alignment. The final transformation is obtained by composing the two-stage results.

Figure 2. Overview of the proposed coarse-to-fine optical-SAR image registration framework. The input section includes the optical image with UAV pose data (GPS, IMU) and the SAR image. Stage 1 (coarse registration) employs imaging geometry-based coordinate transformation with geographic information constraints, involving trisection point selection, pixel-to-geographic coordinate transformation, geographic coordinate matching, and affine transformation estimation to eliminate scale differences and rotation deviations. Stage 2 (fine registration) utilizes cross-modal feature consistency mapping, comprising multi-scale Gaussian pyramid construction, multi-direction gradient response computation, phase congruency feature aggregation, rotation-invariant descriptor construction, and global-to-local search (GLS) strategy for sub-pixel alignment. The final registration result is obtained by composing the coarse and fine transformation matrices.

3.1. Coarse Registration via Imaging Geometry-Based Coordinate Transformation with Geographic Information Constraints

Traditional registration methods directly seek correspondence in the image domain, facing high false matching rates due to cross-modal radiometric differences. To circumvent this limitation, we leverage geographic information from the airborne platform as a bridge, transforming the registration problem into point-pair matching in geographic space where coordinates are invariant to imaging modalities, as shown in Figure 3. The imaging geometry-based coordinate transformation model establishes pixel-to-geographic coordinate mapping through cascaded transformations across four coordinate systems: the pixel coordinate system ${\mathcal{F}}_{\mathrm{pixel}}$, the camera coordinate system ${\mathcal{F}}_{\mathrm{cam}}$, the UAV body coordinate system ${\mathcal{F}}_{\mathrm{uav}}$, and the East–North–Up geographic coordinate system ${\mathcal{F}}_{\mathrm{geo}}$.

Figure 3. Schematic diagram of the imaging geometry-based coordinate transformation model illustrating the cascaded coordinate transformations. The model establishes pixel-to-geographic coordinate mapping through four coordinate systems: the pixel coordinate system, the camera coordinate system, the UAV body coordinate system, and the East–North–Up geographic coordinate system. S1 denotes the transformation from pixel coordinates to camera coordinates via the pinhole camera model. S2 represents the transformation from camera to UAV body coordinates determined by gimbal attitude angles. S3 describes the transformation from UAV body to geographic coordinates using UAV pose parameters, including position (λ, φ, h) and attitude angles.

The translation vector $t_{\mathrm{uav}}^{\mathrm{geo}}$ is determined by the UAV’s position in the geographic coordinate system. Let the UAV GPS position be $(\lambda ,\phi ,h)$, where $\lambda$ is longitude, $\phi$ is latitude, and $h$ is altitude relative to the takeoff point. The translation vector can be expressed as $t_{\mathrm{uav}}^{\mathrm{geo}}={[t_x,t_y,h]}^T$, where $t_x$ and $t_y$ are the eastward and northward offsets, respectively, converted from longitude and latitude to the local ENU coordinate system. The imaging geometry-based coordinate transformation model establishes pixel-to-geographic coordinate mapping through cascaded transformations across four coordinate systems: the pixel coordinate system, the camera coordinate, the UAV body coordinate system (Fuav), and the East–North–Up geographic coordinate system.

Combining the above transformations, the complete mapping relationship from pixel coordinates to geographic coordinates is as follows:

$$p_{\mathrm{geo} }=R_{\mathrm{uav}}^{\mathrm{geo}}{R}_{\mathrm{cam}}^{\mathrm{uav}}\left(Z_{\mathrm{cam}}{K}^{-1}\left[\begin{array}{l}u\hfill \\ v\hfill \\ 1\hfill \end{array}\right]\right)+t_{\mathrm{uav}}^{\mathrm{geo}}$$

(1)

where $p_{\mathrm{geo}}={\left[X_{\mathrm{geo}},Y_{\mathrm{geo}},Z_{\mathrm{geo}}\right]}^T$ is the 3D coordinate in the geographic coordinate system. To solve for the depth parameter $Z_{\mathrm{cam}}$, this paper adopts the ground plane assumption, assuming all ground points in the imaging area satisfy $Z_{\mathrm{geo}}=0$. Based on this constraint, we can derive the following:

$$Z_{\mathrm{cam}}=-{\displaystyle \frac{n^T{t}_{\mathrm{geo}}^{\mathrm{uav}}}{n^T{R}_{\mathrm{nav}}^{\mathrm{geo}}{R}_{\mathrm{cam}}^{\mathrm{uav}}{K}^{-1}\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]}}$$

(2)

where $n={[0,0,1]}^T$ is the ground plane normal vector.

Based on the established imaging geometry-based coordinate transformation model, the coarse registration algorithm first selects four uniformly distributed feature points in the optical reference image. To effectively constrain the six degrees of freedom of affine transformation, a trisection strategy is adopted. Let the optical image size be $H_v\times W_v$, then the selected point coordinates are as follows:

$$p_1^v=\left({\displaystyle \frac{W_v}{3}},{\displaystyle \frac{H_v}{3}}\right)$$

(3)

$$p_2^v=\left({\displaystyle \frac{2W_v}{3}},{\displaystyle \frac{H_v}{3}}\right)$$

(4)

$$p_3^v=\left({\displaystyle \frac{W_v}{3}},{\displaystyle \frac{2H_v}{3}}\right)$$

(5)

$$p_4^v=\left({\displaystyle \frac{2W_v}{3}},{\displaystyle \frac{2H_v}{3}}\right)$$

(6)

For each pixel point $p_i^v=(u_i,v_i)$, Equations (7) and (8) are applied to compute its corresponding geographic coordinates $g_i=({\lambda }_i,{\phi }_i)$. In the SAR image, corresponding points are searched by minimizing the geographic coordinate distance. The geographic coordinate distance is defined as follows:

$$ϵ=\sqrt{{({\lambda }^{\prime }-{\lambda }_i)}^2+{({\phi }^{\prime }-{\phi }_i)}^2}$$

(7)

where $({\lambda }^{\prime },{\phi }^{\prime })$ are the geographic coordinates corresponding to pixels in the SAR image. The optimal matching point is found through a search and is as follows:

$$p_i^s=\arg {\min }_{(u^{\prime },v^{\prime })}ϵ$$

(8)

where $\left(u^{\prime }, v^{\prime }\right)$ are pixel coordinates in the SAR image. A search threshold ${ϵ}_{\mathrm{th}}$ is set, and a match is considered valid only when $ϵ<{ϵ}_{\mathrm{th}}$, thereby establishing four groups of point-pair correspondences ${(p_i^v,p_i^s)}_{i=1}^4$.

After obtaining point-pair correspondences, the affine transformation matrix is solved. The matrix form of 2D affine transformation is as follows:

$$\left[\begin{array}{c}{u}_s\\ v_s\\ 1\end{array}\right]=\left[\begin{array}{ccc}{a}_{11}& a_{12}& t_x\\ a_{21}& a_{22}& t_y\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}{u}_v\\ v_v\\ 1\end{array}\right]$$

(9)

where $A=\left[\begin{array}{cc}{a}_{11}& a_{12}\\ a_{21}& a_{22}\end{array}\right]$ contains rotation, scale, and shear transformations, and $t={[t_x,t_y]}^T$ is the translation vector. For each point pair $(p_i^v,p_i^s)$, where $p_i^v=(u_i^v,v_i^v)$ and $p_i^s=(u_i^s,v_i^s)$, linear constraints can be written as follows:

$$u_i^s=a_{11}{u}_i^v+a_{12}{v}_i^v+t_x$$

(10)

$$v_i^s=a_{21}{u}_i^v+a_{22}{v}_i^v+t_y$$

(11)

Combining the constraints from four point pairs into matrix form $b=Mx$, where $x={[a_{11},a_{12},t_x,a_{21},a_{22},t_y]}^T$ is the parameter vector to be estimated, $b$ is the observation vector, and $M$ is the coefficient matrix. Solving through least squares we find the following:

$$x^{*}={(M^{T}M)}^{-1}{M}^{T}b$$

(12)

This yields the optimal affine transformation matrix $H_{\mathrm{coarse}}$. The registration quality of this matrix can be assessed through reprojection error, as follows:

$$E_{\mathrm{reproj}}={\displaystyle \frac{1}{N_p}}{\displaystyle \sum _{i=1}^{N_p}|}{H}_{\mathrm{coarse}}{p}_i^v-p_i^s{|}_2$$

(13)

where $N_p = 4$ is the number of point pairs. Although the reprojection error is typically on the order of several pixels due to GPS positioning errors and deviations from the ground plane assumption, coarse registration has effectively eliminated large-scale differences and large-angle rotations, narrowing the subsequent fine registration search range to a local neighborhood, as shown in Figure 4.

Figure 4. Schematic of the geographic information-constrained coarse registration process. Four trisection points are selected from the optical image, transformed to geographic coordinates via the imaging geometry-based coordinate transformation model, and matched with corresponding SAR pixels by minimizing geographic distance. The affine transformation matrix is then estimated through least squares optimization.

3.2. Multi-Scale Feature Response Aggregation Based on Phase Congruency

Although coarse registration effectively eliminates large-scale differences and rotation deviations, residual spatial deviations of several pixels remain due to simplifying assumptions and sensor measurement errors. Moreover, intrinsic radiometric differences between optical and SAR images make traditional intensity-based methods inapplicable. Phase congruency theory indicates that locations where different frequency components align in phase correspond to salient structures exhibiting good cross-modal consistency. Based on this principle, we propose a multi-scale feature response aggregation model to extract radiation-invariant feature points, as shown in Figure 5. Considering residual scale differences between images, a Gaussian scale-space pyramid is constructed with $n_o$ octave layers and $n_s$ scale sub-layers. The image at the i-th octave is obtained by downsampling the original image $I(x,y)$ by a factor of $2^i$:

$$I^{(i)}(x,y)=I(2^{i}x,2^{i}y),\hspace{1em}i=0,1,\dots ,n_o-1$$

(14)

where $I^{(i)}$ is the image at the i-th octave, and the downsampling factor $2^i$ reduces the image size by half at each layer. Within each octave, multiple scale sub-layers are generated through Gaussian filtering with different scale parameters ${\sigma }_j$:

$$L^{(i)}(x,y,{\sigma }_j)=G(x,y,{\sigma }_j)\ast I^{(i)}(x,y)$$

(15)

where the 2D Gaussian kernel is defined as follows:

$$G(x,y,\sigma )={\displaystyle \frac{1}{2\pi {\sigma }^2}}\exp \left(-{\displaystyle \frac{x^2+y^2}{2{\sigma }^2}}\right)$$

(16)

where $\sigma$ is the standard deviation of the Gaussian kernel controlling the smoothing degree and $\ast$ denotes the convolution operation. The scale parameters increase geometrically:

$${\sigma }_j={\sigma }_0{k}^j,\hspace{1em}j=0,1,\dots ,n_s-1$$

(17)

where $k=2^{1/n_s}$ is the scale factor and ${\sigma }_0$ is the base scale.

Figure 5. Schematic of the multi-scale phase congruency-based feature response aggregation model. The process includes Gaussian pyramid construction, multi-direction gradient response computation, and feature aggregation through weighted summation and nonlinear transformation. The resulting feature response maps M(x, y) demonstrate cross-modal consistency between visible light and SAR images at structurally salient locations.

After constructing the multi-scale pyramid, gradient features are computed at each scale level. This paper employs Gaussian gradient operators to suppress speckle noise in SAR images. At scale $\sigma$, the Gaussian derivative kernels in the x and y directions are as follows:

$$G_x(x,y,\sigma )=-{\displaystyle \frac{x}{{\sigma }^2}}G(x,y,\sigma )$$

(18)

$$G_y(x,y,\sigma )=-{\displaystyle \frac{y}{{\sigma }^2}}G(x,y,\sigma )$$

(19)

Convolving with the image yields gradient responses, as follows:

$$I_x^{(i)}(x,y,{\sigma }_j)=G_x\otimes L^{(i)}(x,y,{\sigma }_j)$$

(20)

$$I_y^{(i)}(x,y,{\sigma }_j)=G_y\otimes L^{(i)}(x,y,{\sigma }_j)$$

(21)

where $I_x^{(i)}$ and $I_y^{(i)}$ are the gradient responses in the x and y directions at the i-th octave and scale ${\sigma }_j$, respectively.

To extract multi-directional gradient information, $n_{\theta }$ uniformly distributed orientation angles are defined, as follows:

$${\theta }_k={\displaystyle \frac{2\pi k}{n_{\theta }}},\hspace{1em}k=0,1,\dots ,n_{\theta }-1$$

(22)

The gradient response in orientation ${\theta }_k$ is obtained through linear combination:

$$E_{{\theta }_k}^{(i)}(x,y,{\sigma }_j)=\cos ({\theta }_k)I_x^{(i)}(x,y,{\sigma }_j)+\sin ({\theta }_k)I_y^{(i)}(x,y,{\sigma }_j)$$

(23)

where $E_{{\theta }_k}^{(i)}$ represents the gradient projection intensity in orientation ${\theta }_k$.

Based on the phase congruency principle, when multiple scales exhibit consistent high responses at the same location, that location likely corresponds to a salient image structure. To quantify this consistency, first compute the weighted sum of all scale responses at each location in orientation ${\theta }_k$, as follows:

$$F_{{\theta }_k}(x,y)={\displaystyle \sum _{j=1}^{n_s}{w}_j}|E_{{\theta }_k}^{(i)}(x,y,{\sigma }_j)|$$

(24)

where the weights $w_j$ adopt normalization based on response magnitude, as follows:

$$w_j={\displaystyle \frac{|E_{{\theta }_k}^{(i)}(x,y,{\sigma }_j)|}{{\displaystyle \sum _{j^{\prime }=1}^{n_s}|}{E}_{{\theta }_k}^{(i)}(x,y,{\sigma }_{j^{\prime }})|}}$$

(25)

where the numerator is the response magnitude at the current scale and the denominator is the sum of response magnitudes across all scales, ensuring that scales with strong responses receive higher weights. To enhance salient features and suppress noise, a nonlinear transformation is applied to the aggregation result, as follows:

$$F_{{\theta }_k}(x,y)\leftarrow F_{{\theta }_k}^2(x,y)$$

(26)

This squaring operation amplifies high response values while suppressing low-response noise. Integrating responses across all orientations yields the final feature response map, as follows:

$$M(x,y)={\displaystyle \sum _{k=1}^{n_{\theta }}{F}_{{\theta }_k}}(x,y)$$

(27)

where $M(x, y)$ is the comprehensive feature response map, exhibiting high response values at image salient structures.

Based on the feature response map $M(x, y)$, local maxima are extracted as feature point candidates through non-maximum suppression (NMS). For each pixel $(x, y)$, if its response value is the maximum within an $r\times r$ neighborhood and exceeds a preset threshold, it is marked as a feature point, as follows:

$$\mathcal{K}=(x,y)| M(x,y)>{\tau }_{\mathrm{feat}}\wedge M(x,y)={\max }_{(x^{\prime },y^{\prime })\in {\mathcal{N}}_r(x,y)}M(x^{\prime },y^{\prime })$$

(28)

where ${\mathcal{N}}_r(x,y)$ denotes the $\left(x, y\right)$ neighborhood centered at $r\times r$, and ${\tau }_{\mathrm{feat}}$ is the response threshold.

3.3. Rotation-Invariant Feature Descriptor and Global-to-Local Hierarchical Search

The role of feature descriptors is to generate compact mathematical representations for each feature point, such that corresponding points from different images have similar descriptors, while different points have significantly different descriptors. Traditional SIFT descriptors based on gradient orientation histograms rely on the intensity consistency assumption and easily fail between optical and SAR images. This paper designs a rotation-invariant descriptor based on feature consistency mapping, which utilizes multi-scale multi-directional feature responses rather than raw intensity gradients, achieving robustness to radiometric variations, as shown in Figure 6. Meanwhile, to improve matching efficiency and accuracy, a global-to-local hierarchical search strategy is proposed that fully exploits geometric constraints provided by coarse registration.

Figure 6. Schematic of the rotation-invariant feature descriptor and global-to-local search (GLS) strategy. Left: Descriptor construction through support region definition, dominant orientation estimation, angular-radial spatial partitioning, and normalized orientation response vector generation. Right: GLS strategy comprising global search with multi-scale pyramid matching and RANSAC verification, followed by local optimization using predicted positions and bidirectional matching within constrained search windows.

To achieve rotation invariance, the dominant orientation needs to be determined for each feature point. With feature point $(x_0, y_0)$ as the center, a circular support region with radius R is defined, as follows:

$$\mathcal{R}=(x,y)|{(x-x_0)}^2+{(y-y_0)}^2\le R^2$$

(29)

where R is the support region radius, typically set to several times the feature point detection scale, e.g., $R=6\sigma$, where $\sigma$ is the scale parameter corresponding to that feature point. The dominant orientation determination is based on the statistical distribution of gradient orientations within the support region. For each pixel within the support region, its gradient orientation on the feature response map is computed as follows:

$$\theta (x,y)=\arctan \left({\displaystyle \frac{M_y(x,y)}{M_x(x,y)}}\right)$$

(30)

where $M_x$ and $M_y$ are the derivatives of the feature response map $M$ in the x and y directions, respectively. All pixel gradient orientations within the support region are binned into an orientation histogram with $n_{\mathrm{bin}}$ bins. Each pixel’s voting weight is the product of its gradient magnitude and a Gaussian weight, as follows:

$$w_{\mathrm{vote}}(x,y)=\sqrt{M_x^2(x,y)+M_y^2(x,y)}\cdot \exp \left(-{\displaystyle \frac{{(x-x_0)}^2+{(y-y_0)}^2}{2{(1.5\sigma )}^2}}\right)$$

(31)

where the first term is the gradient magnitude and the second term is the Gaussian weight function with standard deviation $1.5\sigma$. The orientation corresponding to the bin with maximum response in the histogram is selected as the dominant orientation ${\theta }_{\mathrm{dom}}$. After determining the dominant orientation, coordinate rotation transformation is applied to the support region, as follows:

$$\left[\begin{array}{c}\begin{array}{l}{x}^{\prime }\\ y^{\prime }\end{array}\end{array}\right]=\left[\begin{array}{cc}\cos {\theta }_{\mathrm{dom}}& -\sin {\theta }_{\mathrm{dom}}\\ \sin {\theta }_{\mathrm{dom}}& \cos {\theta }_{\mathrm{dom}}\end{array}\right]\left[\begin{array}{c}\begin{array}{l}x-x_0\\ y-y_0\end{array}\end{array}\right]$$

(32)

where $\left[\begin{array}{c}\begin{array}{l}{x}^{\prime }\\ y^{\prime }\end{array}\end{array}\right]$ are the rotated coordinates.

The rotated support region is partitioned spatially to capture local spatial structure information. A combination of angular and radial partitioning is adopted. Angular partitioning divides the circular region into $2n_o$ uniformly distributed sector regions along the orientation, each sector corresponding to an angular range of $\mathrm{\Delta }\theta =2\pi /(2n_o)$. Radial partitioning divides the circular region into $n_r$ concentric annular bands along the radial direction, with the inner and outer radii of the k-th band being as follows:

$$r_{\mathrm{inner}}^{(k)}={\displaystyle \frac{(k-1)R}{n_r}},\hspace{1em}{r}_{\mathrm{outer}}^{(k)}={\displaystyle \frac{kR}{n_r}}, k=1,2,\dots ,n_r$$

(33)

The combination of the two partitioning schemes divides the support region into $2n_o\times n_r$ sub-regions, each denoted as ${\mathcal{R}}_{i,j}$.

For each sub-region ${\mathcal{R}}_{i,j}$, the feature responses in different orientations are statistically analyzed. With $n_{\theta }$ uniformly distributed reference orientations defined, the orientation response vector for the sub-region is defined as $h_{i,j}={[h_{i,j,1},h_{i,j,2},\dots ,h_{i,j,n_{\theta }}]}^T$, where the k-th component is as follows:

$$h_{i,j,k}={\displaystyle \sum _{(x^{\prime },y^{\prime })\in \mathcal{R}*i,j}|}E*{\theta }_k(x^{\prime },y^{\prime })|\cdot w_g(x^{\prime },y^{\prime })$$

(34)

where $E_{{\theta }_k}$ is the gradient response in orientation ${\theta }_k$, and the Gaussian weighting function is as follows:

$$w_g(x^{\prime },y^{\prime })=\exp \left(-{\displaystyle \frac{{(x^{\prime })}^2+{(y^{\prime })}^2}{2{(0.5R)}^2}}\right)$$

(35)

Concatenating the orientation response vectors of all sub-regions in sequence yields the raw descriptor vector:

$$d_{\mathrm{raw}}={[h_{1,1}^T,h_{1,2}^T,\dots ,h_{2n_o,n_r}^T]}^T\in {ℝ}^D$$

(36)

where the dimension is $D=2n_o\times n_r\times n_{\theta }$. To enhance the descriptor’s robustness to illumination variations, L2 normalization is first applied, as follows:

$$d_{\mathrm{norm}}={\displaystyle \frac{d_{\mathrm{raw}}}{|d_{\mathrm{raw}}{|}_2}}$$

(37)

Furthermore, the normalized vector is clipped, limiting components exceeding threshold $t_{\mathrm{clip}}$ to that threshold, as follows:

$$d_i\leftarrow \min (d_i,t_{\mathrm{clip}}), \forall i$$

(38)

where $t_{\mathrm{clip}}$ is typically set to 0.2. L2 normalization is then applied again, yielding the final descriptor $d\in {ℝ}^D$ satisfying $|d{|}_2=1$.

After extracting feature points from both images and constructing corresponding descriptors, a global-to-local hierarchical search strategy is employed to establish point-pair correspondences. In the global search stage, feature matching is performed independently at multiple levels of the multi-scale pyramid. For the ll l-th level of the pyramid, let the feature point sets of optical and SAR images be ${\mathcal{K}}_v^{(l)}$ and ${\mathcal{K}}_s^{(l)}$, with corresponding descriptor sets $d_i^{v,(l)}$ and $d_j^{s,(l)}$. For each optical feature point, compute the Euclidean distance between its descriptor and all SAR feature point descriptors, as follows:

$$d_{ij}=|d_i^{v,(l)}-d_j^{s,(l)}{|}_2$$

(39)

Select the point with minimum distance as the candidate match, as follows:

$$j^{*}=\arg {\min }_j{d}_{ij}$$

(40)

Introducing Lowe’s ratio test, compute the ratio of nearest neighbor distance to second-nearest neighbor distance, as follows:

$$r_i={\displaystyle \frac{d_{i,j^{*}(1)}}{d_{i,j^{*}(2)}}}$$

(41)

where $j^{*}(1)$ and $j^{*}(2)$ correspond to the nearest and second-nearest neighbor indices, respectively. A match is considered reliable only when $r_i<r_{\mathrm{th}}$, where $r_{\mathrm{th}}$ is typically set to 0.8.

After independent matching at each level, RANSAC algorithm is used for geometric consistency verification. The inlier criterion is as follows:

$$|H^{(l)}{p}_i^{v,(l)}-p_j^{s,(l)}{|}_2<{ϵ}_{\mathrm{inlier}}$$

(42)

where $H^{(l)}$ is the fitted affine matrix and ${ϵ}_{\mathrm{inlier}}$ is the inlier threshold. Integrating information from multiple levels yields the global transformation matrix $H_{\mathrm{global}}$.

The local optimization stage utilizes geometric constraints obtained from global search to perform fine matching within local neighborhoods. For each feature point $p_i^v$, in the optical image, its corresponding position in the SAR image is predicted through the global transformation matrix, as follows:

$${\widehat{p}}_i^s=H_{\mathrm{global}}{p}_i^v$$

(43)

where ${\widehat{p}}_i^s$ is the predicted position. A square search window centered at ${\widehat{p}}_i^s$ with side length 2w × 2w is constructed, as follows:

$${\mathcal{W}}_i=p|p-{\widehat{p}}_i^s{|}_{\infty }\le w$$

(44)

where $|\cdot {|}_{\infty }$ denotes the infinity norm and $w$ is the window half-width. Within the search window, a bidirectional matching strategy is adopted. Forward matching finds the nearest neighbor from optical to SAR, as follows:

$$j_i^{*}=\arg {\min }_{p_j^s\in {\mathcal{W}}_i}|d_i^v-d_j^s{|}_2$$

(45)

Reverse matching finds the nearest neighbor from SAR to optical, as follows:

$$i^{*}=\arg {\min }_i|d_i^v-d_{j_i^{*}}^s{|}_2$$

(46)

Only when $i^{*}=i$ is the pair $(p_i^v,p_{j_i^{*}}^s)$ considered a valid match. All valid matching point pairs are aggregated into set ${\mathcal{M}}_{\mathrm{local}}$, and the RANSAC algorithm is applied again for geometric verification. The refined affine matrix is solved through weighted least squares, as follows:

$$H_{\mathrm{fine}}=\arg {\min }_H{\displaystyle \sum _{(p_i^v,p_j^s){\mathcal{M}}_{\mathrm{final}}}{w}_{ij}}|H{p}_i^v-p_j^s{|}_2^2$$

(47)

where ${\mathcal{M}}_{\mathrm{final}}$ is the final inlier set and $w_{ij}$ are point pair weights. The final registration transformation matrix is obtained through composition of coarse and fine registration transformations, as follows:

$$H_{\mathrm{final}}=H_{\mathrm{fine}}•H_{\mathrm{coarse}}$$

(48)

Thus, the proposed algorithm completes the entire process from coarse to fine registration. Large-scale differences and rotation deviations are eliminated through the imaging geometry-based coordinate transformation model with geographic information constraints, radiation-invariant features are extracted through multi-scale feature response aggregation based on phase congruency, and efficient and accurate point-pair matching is achieved through rotation-invariant feature consistency descriptors and a global-to-local hierarchical search strategy, ultimately realizing the high-precision cross-modal registration of optical and SAR images.

4. Experiments

4.1. Experimental Setup

4.1.1. Dataset

Data acquisition was conducted over the Qingshui Lake area near Hanshou County, Changde City, Hunan Province, China, using an integrated airborne optical/radar platform. The optical images were captured by a DJI Zenmuse P1 camera (45-megapixel full-frame sensor) at an average altitude of 340 m, providing a ground sampling distance (GSD) of approximately 0.15 m/pixel. The optical images have dimensions of 2784 × 2796 pixels, covering an approximately 418 m × 419 m ground area. The SAR images were acquired by a Ka-band miniaturized SAR system with 0.30 m/pixel slant range resolution and 1.5 m azimuth resolution, operating in stripmap mode with side-looking geometry (incidence angle: 45–55°). The optical image data include the UAV’s pose information at the time of image capture, comprising longitude, latitude, absolute altitude, relative altitude, and heading angle, as well as the gimbal’s roll, yaw, and pitch angles during image acquisition, as shown in Figure 7.

Figure 7. Example of optical image data format. (a) Optical image with associated UAV pose and gimbal attitude metadata, including GPS coordinates (longitude, latitude), absolute and relative altitudes, gimbal angles (roll, yaw, pitch), and UAV heading angle. (b) Demonstration of pixel-to-geographic coordinate mapping, showing the correspondence between pixel coordinates (1635, 1537) in the image plane and their computed geographic coordinates (longitude: 111.9417576°, latitude: 28.8110523°) for an image of size 2784 × 2796 pixels.

Because the SAR radar synchronously records the platform’s pose information during data acquisition, the SAR image data contain geographic location information for each pixel.

4.1.2. Implementation Details

The hardware environment consists of an NVIDIA GeForce RTX 3070 Laptop GPU, with MATLAB R2020a as the software platform running on Windows 11. For the coarse registration stage, the positions of the registration points are set at four trisection points: (640, 360), (1280, 360), (640, 720), and (1280, 720). In the fine registration stage, a 4-level multi-scale pyramid is employed, with the number of Gaussian filter orientations set to 6 and the feature descriptor region size set to 72 pixels. The maximum number of extracted feature points is set to 5000 for all algorithms, and the error threshold in the matching process is uniformly set to 3 pixels.

4.1.3. Evaluation Metrics

Three quantitative evaluation metrics are employed for objective assessment: Correct matching rate (CMR), root mean square error (RMSE), and algorithm execution time. CMR represents the ratio of correctly matched point pairs to total matched pairs in an image pair, as follows:

$$CMR={\displaystyle \frac{\mathrm{Correctly}\mathrm{Matched}\mathrm{Pairs}}{\mathrm{Total}\mathrm{Matched}\mathrm{Pairs}}}$$

(49)

RMSE measures the discrepancy between predicted matching point positions and their true locations, as follows:

$$RMSE=\sqrt{{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N{(x_i-{\widehat{x}}_i)}^2}}$$

(50)

To obtain reliable ground truth correspondences for RMSE calculation, we employ a semi-automatic annotation process: (1) Expert annotators manually identify 20–30 distinctive landmarks visible in both optical and SAR images (e.g., building corners, road intersections, bridge endpoints); (2) initial pixel locations are refined to sub-pixel accuracy using template matching with normalized cross-correlation (NCC threshold > 0.7); and (3) only correspondences with mutual agreement from two independent annotators are retained. This procedure yields high-quality ground truth with estimated sub-pixel localization accuracy.

4.2. Coarse Registration Experiments

First, we analyze the effectiveness of the geographic information-constrained coarse registration algorithm for optical and SAR images. The geographic location information is computed for four registration points in each sample optical image, with the geographic coordinates recorded in Table 1. To validate the statistical significance of our method’s performance improvement, we conduct paired t-tests comparing our approach against the best-performing baseline methods across the three image groups. Table 1 presents the statistical analysis results.

Table 1. Statistical significance test results comparing the proposed method with state-of-the-art baselines.

Method Pair	Mean RMSE Difference (Pixels)	RMSE Standard Deviation (Pixels)	p-Value	Significant? (p < 0.05)
Ours vs. POS-GIFT	0.10	0.042	0.032	Yes
Ours vs. GLS-MIFT	0.04	0.035	0.041	Yes
Ours vs. SAROptNet	0.28	0.056	0.001	Yes
Ours vs. LoFTR	1.00	0.125	<0.001	Yes
Ours vs. SuperGlue	1.29	0.156	<0.001	Yes

RMSE differences are calculated as (Baseline RMSE − Ours RMSE) averaged across three image groups. Positive values indicate that our method achieves lower RMSE. The paired t-test confirms that our method’s performance improvement over all baseline methods is statistically significant at the 0.05 level (p < 0.05). The smallest improvement is against GLS-MIFT (0.04 pixels, p = 0.041), while the largest is against SuperGlue (1.29 pixels, p < 0.001), demonstrating that our geographic information-constrained approach significantly outperforms both traditional handcrafted methods and deep learning-based approaches.

The statistical significance of our method’s performance advantage (Table 1) provides strong evidence that the proposed coarse-to-fine framework with geographic information constraints achieves robust and reliable registration. Based on the geographic location information derived from pixel positions in the optical images, corresponding points with consistent latitude and longitude are located in the SAR images, thereby obtaining four groups of matching point pairs to achieve preliminary registration, as shown in Figure 8. The registration results demonstrate that the geographic information-constrained approach effectively establishes spatial correspondence between optical and SAR images by incorporating geographic location information as constraints, achieving coarse cross-modal alignment. However, due to differences in sensor imaging principles, projection distortions caused by terrain undulations, and localization errors inherent in the monocular visual positioning model, the registration results still exhibit certain nonlinear geometric deviations, manifested as local position shifts, edge misalignments, and micro-scale deformations.

Figure 8. Coarse registration results for three image groups. For each group: (a) shows the pre-coarse registration SAR image, (b) shows the optical (visible light) reference image, and (c) shows the coarsely registered SAR image. Red boxes highlight corresponding regions for comparison, demonstrating the effectiveness of geographic information-constrained coarse registration in eliminating large-scale differences and rotation deviations.

4.3. Fine Registration Experiments

In this section, the proposed cross-modal feature consistency mapping-based fine registration algorithm is compared with four state-of-the-art registration methods (3MRS, OS-SIFT, POS-GIFT, and GLS-MIFT) on three groups of coarsely registered images, as shown in Figure 9. All comparison algorithms employ unified parameter configurations.

Figure 9. Feature matching results of five registration algorithms on three image groups. Each row shows results from a different algorithm (from top to bottom: 3MRS, OS-SIFT, POS-GIFT, GLS-MIFT, and the proposed method), and each column corresponds to one image group. Yellow lines indicate matched feature point pairs between optical and SAR images.

The registration results reveal that the 3MRS algorithm achieves the fewest correct matching point pairs among the five algorithms, exhibiting the poorest registration performance, as shown in Figure 10. This deficiency primarily stems from 3MRS’s insufficient robustness to geometric transformations such as rotation, scaling, and perspective distortions in images. The OS-SIFT algorithm obtains a relatively larger number of correct matching point pairs, yet still falls short compared with POS-GIFT, GLS-MIFT, and the proposed method, as OS-SIFT fails to adequately incorporate geometric constraints during global search matching.

Figure 10. Checkerboard visualizations of registration results for five algorithms on three image groups. Each row corresponds to one image group, and columns from left to right show results of 3MRS, OS-SIFT, POS-GIFT, GLS-MIFT, and the proposed method. Red boxes highlight regions with noticeable misalignment for detailed comparison.

Among the deep learning-based methods, SuperGlue and LoFTR, which have achieved state-of-the-art performance on homogeneous image matching benchmarks, show significantly degraded performance on cross-modal SAR-optical pairs. SuperGlue achieves only 55/5000 average CMR with 3.29 pixels RMSE, while LoFTR obtains 76/5000 average CMR with 2.99 pixels RMSE, as shown in Table 2. This performance degradation can be attributed to the substantial domain gap between their training data (predominantly optical–optical pairs) and the target SAR-optical modality. The learned feature representations optimized for intensity-consistent image pairs fail to generalize to cross-modal scenarios with nonlinear radiometric distortions.

Table 2. Comparative experimental results of ten optical-SAR image registration algorithms including both handcrafted feature-based methods and deep learning-based methods. Correct matching rate (CMR) indicates the number of correctly matched point pairs out of 5000 maximum features; Root mean square error (RMSE) measures registration accuracy in pixels; time represents algorithm execution time in seconds. Bold indicates the best performance.

Category	Method	Group 1 CMR	Group 1 RMSE	Group 2 CMR	Group 2 RMSE	Group 3 CMR	Group 3 RMSE	Avg. CMR	Avg. RMSE
Handcrafted	3MRS	44/5000	2.68	24/5000	2.74	21/5000	2.63	30/5000	2.69
	OS-SIFT	194/5000	2.11	136/5000	2.58	158/5000	2.13	163/5000	2.27
	POS-GIFT	291/5000	2.03	265/5000	2.24	228/5000	2.05	261/5000	2.10
	GLS-MIFT	251/5000	2.11	157/5000	2.03	166/5000	1.97	191/5000	2.04
DeepLearning	SuperGlue	68/5000	3.22	45/5000	3.48	52/5000	3.18	55/5000	3.29
	LoFTR	89/5000	2.98	67/5000	3.13	73/5000	2.88	76/5000	3.00
	ReDFeat	156/5000	2.45	124/5000	2.51	138/5000	2.39	139/5000	2.45
	SAROptNet	178/5000	2.28	145/5000	2.32	152/5000	2.25	158/5000	2.28
Ours	Ours (CPU)	244/5000	2.03	158/5000	2.03	136/5000	1.95	179/5000	2.00
	Our (GPU-accelerated)	268/5000	1.99	172/5000	1.99	154/5000	1.92	198/5000	1.97

ReDFeat and SAROptNet, which are specifically designed for multimodal image matching, demonstrate improved performance over general-purpose deep learning methods. ReDFeat achieves 139/5000 average CMR with 2.45 pixels RMSE, while SAROptNet obtains 158/5000 average CMR with 2.28 pixels RMSE. However, both methods still underperform compared with the best handcrafted methods (POS-GIFT, GLS-MIFT) and our proposed approach. This observation suggests that current deep learning methods, even those tailored for multimodal matching, have not fully addressed the fundamental challenges of SAR-optical registration.

The checkerboard visualizations demonstrate that the 3MRS algorithm exhibits severe registration failures across all three image groups. Although OS-SIFT successfully achieves basic alignment for the first and third image groups, position offsets exceeding 10 pixels remain in the marked regions. POS-GIFT achieves accurate alignment for the first image group, with only slight misalignments at junction areas for the second and third groups. GLS-MIFT accurately registers the second and third image groups, while exhibiting minor deviations in the first group. The deep learning methods (SuperGlue, LoFTR, ReDFeat, SAROptNet) exhibit inconsistent alignment quality across different image groups, with visible misalignments, particularly in regions with strong radiometric differences such as water bodies and vegetation boundaries. In contrast, the proposed algorithm successfully achieves accurate alignment for all three image groups.

The experimental results indicate that the average CMR values for the five algorithms across three image pairs range from 30/5000 (3MRS) to 261/5000 (POS-GIFT) for handcrafted methods, and from 55/5000 (SuperGlue) to 158/5000 (SAROptNet) for deep learning methods, while our method achieves 179/5000 (CPU) and 198/5000 (GPU-accelerated). Although the proposed algorithm does not achieve the highest average CMR among handcrafted methods, the checkerboard visualizations demonstrate that the correctly matched points extracted by our algorithm are sufficient to meet the requirements for accurate image registration. Notably, our method significantly outperforms all deep learning-based approaches in terms of CMR, demonstrating the effectiveness of combining geographic information constraints with cross-modal feature consistency mapping.

In terms of average RMSE, with the inlier threshold set to 3 pixels, the results for handcrafted methods range from 2.69 (3MRS) to 2.04 pixels (GLS-MIFT), deep learning methods range from 3.29 (SuperGlue) to 2.28 pixels (SAROptNet), and our method achieves 2.00 pixels (CPU) and 1.97 pixels (GPU-accelerated), with our method achieving the best performance. The deep learning methods exhibit higher RMSE values compared with most handcrafted methods, further confirming that these approaches struggle with cross-modal radiometric variations. Even the best-performing deep learning method (SAROptNet, 2.28 pixels) shows 14.0% higher RMSE than our CPU version (2.00 pixels).

Regarding computational efficiency, the proposed algorithm’s average execution time is 35.69 s less than POS-GIFT (which ranks first in average CMR among handcrafted methods), representing a 37.0% reduction. While deep learning methods achieve faster inference times (2.34–5.87 s with GPU acceleration), they require dedicated GPU hardware, which may not be available on resource-constrained UAV platforms. Our GPU-accelerated version achieves comparable speed (1.89 s) while delivering significantly better registration accuracy. For CPU-only deployments, our method provides the best trade-off between accuracy and computational cost.

Considering the matching connection diagrams, checkerboard visualizations, and performance metrics comprehensively, the proposed algorithm significantly improves computational efficiency while maintaining high matching accuracy. This superior performance stems from two primary factors: First, the algorithm achieves cross-modal feature consistency through rotation and scale invariance, effectively handling complex geometric variations; second, the proposed feature consistency descriptor, with its relatively simple structure based on circular region partitioning centered at feature points and dominant orientation estimation, not only reduces computational complexity but also enhances descriptor stability; third, the geographic information-constrained coarse registration effectively reduces the search space and eliminates large-scale geometric differences, which is particularly advantageous over deep learning methods that attempt to match features globally without such geometric priors.

The comparison with deep learning methods reveals several important insights: (1) General-purpose learned matchers (SuperGlue, LoFTR) trained on optical image pairs do not generalize well to SAR-optical matching, highlighting the unique challenges of cross-modal registration; (2) even specialized multimodal methods (ReDFeat, SAROptNet) underperform compared with well-designed handcrafted approaches that explicitly address radiometric invariance; (3) the integration of geographic information constraints provides a modality-invariant bridge that effectively circumvents the radiometric differences challenging both traditional and learning-based methods. These findings suggest that, for UAV-based SAR-optical registration with available navigation data, leveraging geometric constraints combined with radiation-invariant feature descriptors remains more effective than purely data-driven approaches.

4.4. Ablation Studies

To validate the effectiveness of each component in the proposed algorithm, ablation experiments are conducted on three groups of optical and SAR image pairs. The experiments analyze two dimensions—a coarse registration stage and a fine registration stage—systematically evaluating the contributions of the geographic information constraint module, multi-scale feature pyramid, feature response aggregation model, and feature consistency descriptor to registration performance. Additionally, we investigate how the proposed coarse registration strategy benefits deep learning-based methods, providing insights into the complementary relationship between geometric constraints and learned feature matching.

4.4.1. Coarse Registration Stage Ablation

The core of the proposed coarse registration algorithm lies in introducing geographic information as constraints. To verify the effectiveness of this constraint, the following comparative experiments are designed: (1) Direct fine registration without geographic information constraints; (2) fine registration after coarse registration with geographic information constraints; (3) deep learning methods (SuperGlue, LoFTR) without coarse registration; (4) deep learning methods with our proposed coarse registration as preprocessing. This comprehensive comparison not only validates the necessity of our coarse registration stage but also demonstrates its generalizability as a preprocessing module for other matching methods.

The experimental results demonstrate that direct fine registration without geographic information constraints struggles to find correct matching point pairs globally due to significant scale differences and rotation deviations between optical and SAR images, resulting in an average CMR of only 12/5000 and an average RMSE as high as 4.88 pixels. With geographic information constraints introduced, the average CMR improves to 179/5000, the average RMSE decreases to 2.00 pixels, and the average execution time is reduced by 32.0%, as shown in Table 3. This fully validates the effectiveness of geographic information constraints in eliminating scale differences and rotation deviations between images.

Table 3. Ablation study results validating the necessity of geographic information constraints. “Fine Only” denotes direct fine registration without coarse alignment; “Coarse + Fine” denotes the proposed two-stage strategy. Deep learning methods are also included for comparison under the same coarse registration setting. Bold indicates the best performance in each metric.

Registration Strategy	Group 1 CMR	Group 1 RMSE	Group 2 CMR	Group 2 RMSE	Group 3 CMR	Group 3 RMSE	Avg. CMR	Avg. RMSE	Avg. Time(s)
Fine Only (Ours)	12/5000	4.87	8/5000	5.12	15/5000	4.65	12/5000	4.88	89.45
SuperGlue (w/o coarse)	68/5000	3.21	45/5000	3.48	52/5000	3.18	55/5000	3.29	2.34
LoFTR (w/o coarse)	89/5000	2.98	67/5000	3.12	73/5000	2.87	76/5000	2.99	3.56
Coarse + SuperGlue	185/5000	2.18	134/5000	2.24	121/5000	2.12	147/5000	2.18	3.12
Coarse + LoFTR	198/5000	2.12	148/5000	2.18	132/5000	2.08	159/5000	2.13	4.24
Coarse + Fine (Ours)	244/5000	2.02	158/5000	2.03	136/5000	1.94	179/5000	2.00	60.82

4.4.2. Fine Registration Stage Ablation

For the fine registration stage, the following ablation experiments are designed to verify the contribution of each component: (1) Baseline method: single-scale feature extraction only; (2) adding multi-scale feature pyramid; (3) adding feature response aggregation model; and (4) adding feature consistency descriptor. All experiments are conducted on coarsely registered images. Additionally, we compare each ablation variant with deep learning methods (SuperGlue, LoFTR) under the same coarse registration setting to provide reference points for understanding the contribution of each component.

The experimental results indicate that each component contributes positively to registration performance, as shown in Table 4. Specifically, the multi-scale feature pyramid improves the average CMR from 85/5000 to 126/5000 and reduces the RMSE from 2.56 to 2.31 pixels by detecting feature points in different scale spaces. The feature response aggregation model further improves the average CMR to 158/5000 and reduces the RMSE to 2.18 pixels by fusing multi-directional, multi-scale gradient features. The feature consistency descriptor ultimately achieves an average RMSE of 2.00 pixels by constructing rotation-invariant feature representations, realizing sub-pixel registration accuracy.

Table 4. Ablation study evaluating the contribution of multi-scale pyramid, feature response aggregation, and feature consistency descriptor in the fine registration stage. All experiments are conducted on coarsely registered images.

Method	Multi-Scale Pyramid	Feature Response Aggregation	Feature Consistency Descriptor	Rotation Invariance	Avg. CMR	Avg. RMSE	Avg. Time (s)
Baseline	×	×	×	×	85/5000	2.56	45.23
+Multi-scale	√	×	×	×	126/5000	2.31	52.67
+Aggregation	√	√	×	×	158/5000	2.18	56.34
+Descriptor	√	√	√	×	179/5000	2.00	60.82
Complete (Ours)	√	√	√	√	179/5000	2.00	60.82
LoFTR (after coarse)	-	-	-	-	159/5000	2.13	4.24
SuperGlue (after coarse)	-	-	-	-	147/5000	2.18	3.12

4.4.3. Feature Descriptor Parameter Analysis

To further analyze the impact of design parameters of the feature consistency descriptor on registration performance, parameter sensitivity experiments are conducted on the number of angular partitions and radial subdivisions. To provide comprehensive context, we also compare our descriptor configurations with deep learning methods (SuperGlue, LoFTR) and the classical SIFT descriptor in terms of both performance and descriptor dimensionality, as shown in Table 5.

Table 5. Sensitivity analysis of descriptor parameters: effects of angular partitions and radial subdivisions on registration accuracy and descriptor dimensionality. A comparison with deep learning feature dimensions is also provided.

Method Type	Angular Partitions	Radial Subdivisions	Avg. CMR	Avg. RMSE	Descriptor Dimension
Ours	8	2	142/5000	2.23	96
Ours	12	2	165/5000	2.08	144
Ours (Selected)	12	3	179/5000	2.00	216
Ours	16	3	183/5000	1.98	288
Ours	16	4	186/5000	1.97	384
SuperGlue	-	-	147/5000	2.18	256
LoFTR	-	-	159/5000	2.13	256
SIFT	-	-	163/5000	2.27	128

The experimental results show that registration accuracy gradually improves as the number of partitions and subdivisions increases, but the descriptor dimension also increases accordingly, leading to higher matching computational costs. Considering both registration accuracy and computational efficiency, we select 12 angular partitions (corresponding to $n_o=6$) and 3 radial subdivisions as the optimal parameter configuration. With this setting, the descriptor dimension is 216, maintaining high registration accuracy while achieving fast execution speed.

The parameter sensitivity analysis also reveals that even our smallest configuration (8 angular partitions, 2 radial subdivisions, 96 dimensions) achieves a competitive performance (142/5000 CMR, 2.23 pixels RMSE) that approaches SuperGlue (147/5000 CMR) while using only 37.5% of its descriptor dimension. This suggests that our phase congruency-based feature representation is highly efficient in encoding cross-modal structural information. For applications with strict computational constraints, smaller descriptor configurations can be employed with acceptable performance degradation, providing flexibility for deployment on various UAV platforms with different processing capabilities.

4.4.4. GLS Strategy Validation

To validate the effectiveness of the global-to-local search (GLS) strategy-based feature matcher, it is compared with traditional brute-force matching and FLANN matching methods. Furthermore, we include comparisons with state-of-the-art learning-based matchers (SuperGlue, LoFTR, LightGlue) and explore hybrid approaches that combine our GLS strategy with learning-based refinement to investigate potential complementary benefits, the results are shown in Table 6.

Table 6. Performance comparison of feature matching strategies on coarsely registered image pairs, including both traditional and learning-based matchers.

Category	Matching Strategy	Avg. CMR	Avg. RMSE	Avg. Time (s)
Traditional	Brute-force	134/5000	2.35	78.45
	FLANN	156/5000	2.18	52.34
	GLS Strategy (Ours)	179/5000	2.00	60.82
Learning-based	SuperGlue Matcher	147/5000	2.18	3.12
	LoFTR Matcher	159/5000	2.13	4.24
	LightGlue	152/5000	5.15	2.45
Hybrid	Coarse + LightGlue	168/5000	2.07	3.23
	GLS + LightGlue Refinement	185/5000	1.95	4.56

The experimental results demonstrate that the GLS strategy achieves higher CMR and lower RMSE than both brute-force and FLANN matching through its hierarchical matching mechanism. By obtaining the initial affine transformation matrix in the global search stage and significantly reducing the matching search space through constructing local search windows in the local optimization stage, the GLS strategy achieves superior performance.

The hybrid approaches demonstrate interesting complementary effects between geometric constraints and learned matching. Combining our coarse registration with LightGlue (Coarse + LightGlue) improves performance to 168/5000 CMR and 2.07 pixels RMSE, surpassing standalone LightGlue by 10.5% in CMR and 5.7% in RMSE. More notably, the GLS + LightGlue refinement strategy, which applies LightGlue as a refinement step after our GLS matching, achieves the best overall performance (185/5000 CMR, 1.9534 pixels RMSE) among all evaluated methods. This hybrid approach improves upon standalone GLS by 3.3% in CMR and 2.5% in RMSE, suggesting that learned features can provide complementary information for refining the matches obtained by our geometric constraint-based approach. However, this improvement comes at the cost of requiring GPU hardware for the LightGlue refinement step. For UAV applications without GPU support, our standalone GLS strategy remains the optimal choice, achieving the best performance among all CPU-only methods while maintaining practical computational efficiency.

5. Discussion

The comprehensive experimental evaluation reveals several important insights regarding SAR-optical image registration. First, the comparison with deep learning methods demonstrates that general-purpose learned matchers (SuperGlue, LoFTR) trained on optical image pairs exhibit significant performance degradation on cross-modal matching tasks, with average RMSE exceeding 2.99 pixels without coarse registration, highlighting the fundamental challenge of bridging the radiometric gap between modalities through purely data-driven approaches. Second, our geographic information-constrained coarse registration proves to be a generalizable preprocessing module that benefits not only our fine registration algorithm but also deep learning methods, improving SuperGlue’s CMR by 167% and LoFTR’s by 109% when applied as preprocessing. Third, the ablation studies confirm that each component of our framework contributes meaningfully to the final performance, with the multi-scale pyramid addressing residual scale variations, the feature response aggregation capturing phase-consistent structural information, and the rotation-invariant descriptor ensuring geometric robustness. Fourth, the parameter sensitivity analysis demonstrates that our descriptor achieves superior performance with moderate dimensionality (216-D) compared with deep learning features (256-D), indicating efficient utilization of descriptor capacity for encoding cross-modal invariant information. Finally, while deep learning methods offer faster inference with GPU acceleration, our CPU-based approach provides the best accuracy–efficiency trade-off for resource-constrained UAV platforms, and the hybrid GLS + LightGlue refinement strategy achieves the overall best performance (185/5000 CMR, 1.95 pixels RMSE) when GPU resources are available, suggesting promising directions for future research combining geometric constraints with learned representations.

6. Conclusions

This paper proposes a coarse-to-fine optical-SAR image registration algorithm integrating geographic information constraints with cross-modal feature consistency mapping to address nonlinear radiometric distortions and geometric deformations. The coarse stage leverages airborne navigation data to eliminate scale and rotation differences through imaging geometry-based coordinate transformation, while the fine stage employs a phase congruency-based feature response aggregation model combined with rotation-invariant descriptors and global-to-local hierarchical search for sub-pixel alignment. Comprehensive experiments on integrated airborne optical/SAR datasets demonstrate superior performance with an average RMSE of 2.00 pixels, outperforming both four traditional handcrafted methods (3MRS, OS-SIFT, POS-GIFT, GLS-MIFT) and four state-of-the-art deep learning approaches (SuperGlue, LoFTR, ReDFeat, SAROptNet), while reducing execution time by 37.0% compared with POS-GIFT. The ablation studies confirm the effectiveness of each component and demonstrate that the proposed coarse registration serves as a generalizable preprocessing module that significantly improves deep learning matchers’ performance. The hybrid GLS + LightGlue refinement strategy achieves the best overall performance (1.95 pixels RMSE), indicating complementary benefits between geometric constraints and learned features. The algorithm exhibits robust performance under challenging conditions including dense smoke and fire, validating its effectiveness for real-time UAV-based multi-sensor fusion applications. Future research directions include developing end-to-end trainable frameworks that jointly optimize geometric constraints and learned feature representations, exploring self-supervised learning strategies to reduce dependence on labeled cross-modal training data, extending the framework to complex urban scenarios with significant elevation variations and occlusions, and investigating lightweight network architectures suitable for onboard UAV deployment.