LiteSAM: Lightweight and Robust Feature Matching for Satellite and Aerial Imagery

Highlights

What are the main findings?

• LiteSAM, a lightweight satellite–UAV feature matching framework, employs unified feature representation and fine-grained matching strategies to achieve a superior trade-off between accuracy and efficiency in cross-view matching.
• Achieves state-of-the-art performance on multiple benchmarks while substantially reducing model complexity and inference latency.

What is the implication of the main finding?

• Enables real-time UAV visual localization in GPS-denied and resource-constrained environments, making it practical for deployment.
• Demonstrates strong generalization across datasets and scenarios, extending its applicability to remote sensing and natural image matching tasks.

Abstract

We present a (Light)weight (S)atellite–(A)erial feature (M)atching framework (LiteSAM) for robust UAV absolute visual localization (AVL) in GPS-denied environments. Existing satellite–aerial matching methods struggle with large appearance variations, texture-scarce regions, and limited efficiency for real-time UAV applications. LiteSAM integrates three key components to address these issues. First, efficient multi-scale feature extraction optimizes representation, reducing inference latency for edge devices. Second, a Token Aggregation–Interaction Transformer (TAIFormer) with a convolutional token mixer (CTM) models inter- and intra-image correlations, enabling robust global–local feature fusion. Third, a MinGRU-based dynamic subpixel refinement module adaptively learns spatial offsets, enhancing subpixel-level matching accuracy and cross-scenario generalization. The experiments show that LiteSAM achieves competitive performance across multiple datasets. On UAV-VisLoc, LiteSAM attains an RMSE@30 of 17.86 m, outperforming state-of-the-art semi-dense methods such as EfficientLoFTR. Its optimized variant, LiteSAM (opt., without dual softmax), delivers inference times of 61.98 ms on standard GPUs and 497.49 ms on NVIDIA Jetson AGX Orin, which are 22.9% and 19.8% faster than EfficientLoFTR (opt.), respectively. With 6.31M parameters, which is 2.4× fewer than EfficientLoFTR’s 15.05M, LiteSAM proves to be suitable for edge deployment. Extensive evaluations on natural image matching and downstream vision tasks confirm its superior accuracy and efficiency for general feature matching.

1. Introduction

Recent advances in UAV manufacturing and the enhanced computational capabilities of edge devices have rapidly broadened the scope of UAV applications. Unmanned aerial vehicles (UAVs) are now extensively deployed across diverse domains, including urban planning [1], border security [2], disaster management [3], precision agriculture [4], surveying and mapping [5], environmental conservation [6], military operations [7], infrastructure inspection [8], and construction [9]. Precise localization is critical for autonomous navigation and effective task execution. However, Global Navigation Satellite Systems (GNSSs) often suffer from degraded performance in challenging environments—such as urban canyons, dense forests, or regions affected by signal interference and spoofing.

In GNSS-degraded or GNSS-denied environments, absolute visual localization (AVL) provides a reliable alternative by matching aerial imagery obtained by UAVs with pre-stored reference satellite or aerial imagery to estimate global positions. In contrast, Relative Visual Localization (RVL) estimates a UAV’s motion state through changes between consecutive frames, enabling local navigation, but it susceptible to cumulative drift over time. By anchoring observations directly to a fixed global reference frame, AVL fundamentally eliminates drift errors. Serving as the foundation for environmental perception, precise navigation, and 3D reconstruction, AVL significantly enhances the reliability of UAV operations in complex scenarios by enabling them to comprehensively understand their surroundings and execute autonomous decisions. Consequently, AVL has emerged as a pivotal technology, endowing UAVs with robust autonomous flight capabilities.

Based on matching granularity, AVL can be classified into large-scale matching and fine-grained matching [10]. Large-scale matching focuses on image-level correspondences for initial localization, while fine-grained matching establishes pixel-level correspondences to support downstream tasks such as pose estimation, visual localization, and 3D reconstruction. Typically, fine-grained matching is performed after large-scale matching for high-precision localization. Because it closely resembles natural image matching, classical methods (e.g., template matching and hand-crafted features such as SIFT [11], ORB [12], and SURF [13]) are directly applicable to AVL. In satellite imagery, appearance and scale variations destabilize template matching. Moreover, keypoint-based methods often lack robustness in common AVL scenarios, such as repeated structures, low-texture regions, extreme scale differences, atmospheric distortions, or large viewpoint changes, resulting in diminish localization accuracy. To address these challenges, learning-based feature matching methods have been proposed. These methods learn to extract feature points and generate descriptors from massive-scale datasets, significantly improving the generalization and stability of matching, effectively mitigating complex environmental disturbances, and ultimately enhancing the reliability of AVL in high-precision localization tasks.

Directly applying feature matchers designed for natural images to AVL is challenging due to variations in illumination, seasonality, viewpoint, and scale, as well as motion blur in satellite–aerial images, which complicate matching and degrade performance. Semi-dense large models [14,15] are typically used to establish correspondences; however, their transformer-based processing of full-scale feature maps incurs high computational costs and is unsuitable for real-time edge deployment. However, low-resolution token selection can reduce some overhead [16], while inter- and intra-image correlations still pose bottlenecks. Sparse matchers lack sufficient accuracy [17,18]. To address this, we propose an edge-friendly semi-dense matcher with reduced model size and computation for AVL following a coarse-to-fine architecture [16].

In this work, we introduce LiteSAM, a lightweight and efficient feature matching framework that achieves a better trade-off between accuracy and inference latency. The framework is specifically designed for AVL applications. As illustrated in Figure 1, LiteSAM achieves state-of-the-art localization accuracy on satellite–aerial benchmarks, requiring significantly fewer parameters and lower computational costs while also performing competitively in natural image matching tasks. We employ a reparameterized feature extraction backbone to reduce inference latency while maintaining strong representation capability. In addition, we introduce a novel efficient feature fusion method, TAIFormer, which utilizes a convolutional token mixer (CTM) to effectively capture both global and local feature correlations. Additionally, we propose a learnable correspondence refinement strategy to improve matching accuracy and enhance generalization capability. LiteSAM can directly transfer from natural image datasets to satellite–aerial image datasets without fine-tuning on the target dataset, maintaining high accuracy across diverse domains. LiteSAM not only maintains high accuracy but also significantly reduces computational complexity, achieving an optimal balance between performance and efficiency for resource-constrained UAV visual localization tasks.

Our contributions can be summarized as follows:

(1)

We employ efficient multi-scale feature extraction that optimizes representation capacity and significantly reduces inference latency, enabling robust deployment on resource-constrained edge devices for real-time UAV localization tasks.

(2)

We introduce a Token Aggregation–Interaction Transformer (TAIFormer), a lightweight hybrid module leveraging a convolutional token mixer (CTM), to efficiently model inter- and intra-image correlations. This design achieves robust global–local feature fusion, enhancing matching accuracy across diverse appearance variations.

(3)

We develop a MinGRU-based dynamic subpixel refinement module that adaptively learns spatial offsets through local contextual modeling. This approach significantly improves subpixel-level matching accuracy and cross-scenario generalization for complex satellite–aerial scenes.

(4)

Extensive experiments validate that LiteSAM delivers competitive performance in accuracy, latency, and model size, particularly on the UAV-VisLoc dataset and NVIDIA Jetson edge devices. These results demonstrate its suitability for real-time UAV localization and general feature matching applications.

2. Related Work

2.1. Detector-Based Feature Matching

Detector-based methods perform image matching by identifying sparse keypoints and extracting their corresponding descriptors.

Learning-based approaches (e.g., D2D [25], SuperPoint [23]) leverage CNNs and large datasets to train robust keypoint detectors and feature descriptors, achieving superior matching performance under diverse appearance variations and adverse lighting conditions.

Detector-based methods still fail in extreme AVL conditions due to weak integration between detection and description stages. SuperGlue [17] mitigates this limitation by utilizing attention and graph neural networks (GNNs) for self- and cross-attention matching. However, its high computational and memory requirements hinder deployment on UAV edge devices.

In UAV-based AVL applications, substantial appearance discrepancies between current and reference images caused by variations in flight altitude, illumination, and seasonal conditions are common. Additionally, extensive low-texture regions, such as open terrains and water surfaces, exacerbate keypoint sparsity and descriptor degradation, leading to degraded matching performance.

To overcome these issues, we adopt detector-free matching, which avoids sparse keypoints and directly computes dense correspondences to improve accuracy and robustness in complex environments.

2.2. Detector-Free Feature Matching

Instead of relying on explicit keypoint detection, detector-free approaches establish dense or semi-dense correspondences directly at the pixel or patch level. They utilize convolutional neural networks (CNNs) to extract dense feature representations and construct 4D correlation volumes to model pairwise similarities. Nonetheless, these methods often exhibit limited robustness under severe illumination changes or wide baseline variations, which constrains their effectiveness in highly dynamic scenarios, such as satellite–aerial image matching.

LoFTR [15] proposes a transformer-based feature matching framework that leverages both self- and cross-attention, substantially improving the modeling of global context and long-range dependencies. Building upon this paradigm, several specialized frameworks have emerged. EfficientLoFTR [16] introduces an aggregated attention network for efficient local feature transformation and designs a two-stage correlation refinement module to enhance matching accuracy and subpixel alignment, thereby significantly enhancing overall efficiency and accuracy. ASpanFormer [14] implements an adaptive local attention design that dynamically adjusts the receptive field according to the complexity of local matching, thereby improving robustness in scenes with structural clutter or low texture. MatchFormer [26] utilizes a hierarchical transformer to jointly optimize feature extraction and correspondence estimation, which increases the distinctiveness of the learned features. In a different vein, TopicFM [27] segments features into multiple semantic topics and focuses computation on semantically aligned regions across images, leading to stronger cross-domain generalization. Its improved variant, TopicFM+ [28], further integrates sparse attention for fixed-size topics and fine-grained structures, substantially lowering the computational overhead while preserving matching accuracy. Deepmatcher [29] employs a vector-based attention to model the correlations among keypoints. GeoAT [30] utilizes the affine transformation matrix estimated during coarse matching to guide the cross-attention process in the intermediate matching stage, thereby enhancing correspondence accuracy. JamMa [31] replaces transformer with mamba architecture to achieve a well-balanced trade-off between matching precision and computational efficiency.

Despite improved accuracy and robustness, high computational costs hinder resource-constrained deployments, particularly in real-time AVL on UAVs with stringent latency and energy constraints. To overcome this, we propose TAIFormer, which enhances feature discriminability at a low cost, offering an efficient UAV-based AVL solution.

Moreover, recent dense matching frameworks such as DKM [32], RoMa [33], and HomoMatcher [34] incorporate probabilistic modeling and homography estimation to achieve high-precision pixel-level correspondences. These techniques perform well in applications demanding rigorous geometric consistency. However, their computational complexity escalates rapidly with image resolution, hindering their applicability in latency-sensitive environments. In contrast, the semi-dense matching strategy presented in this work maintains competitive accuracy with significantly reduced overhead, achieving a favorable trade-off between performance and efficiency in real-time UAV-based AVL applications.

2.3. Feature-Matching in AVL

Early approaches to AVL typically rely on template matching methods (e.g., [35,36,37]) to localize UAVs by identifying corresponding regions in satellite imagery. In recent years, learning-based aerial image matching techniques have advanced significantly and can be broadly categorized into large-scale and fine-grained matching methods. Large-scale matching methods rely on global descriptors for image retrieval, enabling the identification of candidate satellite views from massive databases. For instance, Sample4Geo [38] presents a lightweight contrastive learning framework for cross-view localization, while Deuser et al. [39] propose a compact orientation prediction module to enhance pose estimation. Although these methods offer strong scalability, they often fall short of the precision required for fine-grained localization.

Fine-grained matching methods address this limitation by aligning features at the pixel level to achieve high localization accuracy. WildNav [40] combines SuperPoint [23] and SuperGlue [17] to establish sparse correspondences across UAV and satellite images. VDFT [41] introduces a viewpoint-invariant transformation to mitigate geometric distortions, while MOMA [42] proposes a structure-consistent framework to handle scale variations. Notwithstanding their improved accuracy, these methods often involve bulky models and significant computational cost, making them difficult to deploy on UAV edge devices with strict real-time and resource constraints. SwinMatcher [43] introduces a cross-modal remote-sensing image matching framework that integrates cross-modal feature interaction with multi-scale contextual modeling, thereby enhancing robustness in complex environments. Similarly, ETQ-Matcher [44] employs a multi-transformer with channel attention to capture global representations, achieving superior performance in challenging aerial–ground urban scenarios.

In contrast, the fine-grained matching strategy proposed in this work achieves a more favorable trade-off between accuracy and efficiency, providing a practical solution for real-time AVL under constrained UAV platforms.

3. Methodology

Image matching is used to establish reliable correspondences between a given image pair. Formally, for an image pair $(I^A,I^B)$, where $I^A,I^B\in {\mathbb{R}}^{H\times W}$, a feature extractor is employed to obtain the corresponding feature sets ${\mathcal{F}}^A$ and ${\mathcal{F}}^B$. By computing the correlation between features, matching relationships $(k_i^A,k_j^B)$ are identified, where $k_i^A\in {\mathcal{F}}^A$ and $k_j^B\in {\mathcal{F}}^B$. Here, i and j denote the indices of individual feature points in the respective feature sets, representing potentially corresponding locations across the two images.

We focus on satellite–aerial image matching, which is particularly challenging due to significant viewpoint variations and low-texture regions. To tackle these challenges with minimal computational overhead, we employ a coarse-to-fine progressive matching strategy to achieve high-precision correspondences at the subpixel level.

LiteSAM is specifically designed to address these challenges in satellite–aerial image matching. By processing input image pairs $(I^A,I^B)$, the framework extracts multi-scale feature maps and progressively generates correspondence sets $\left\{(k_i^A,k_j^B)\right\}$ from coarse to subpixel-level precision. This design ensures high-precision matching while maintaining lightweight and efficient computation, suitable for real-time aerial visual localization tasks. As illustrated in Figure 2, the overall architecture consists of the following key modules:

• Section 3.1 introduces a reparameterizable backbone for compact and efficient feature representation;
• Section 3.2 presents the Token Aggregation–Interaction Transformer (TAIFormer), which adaptively integrates local and global cues to facilitate rich contextual interaction;
• Section 3.3 outlines a coarse-level matching strategy for generating initial correspondence hypotheses;
• Section 3.4 describes a learnable refinement module that progressively improves correspondence accuracy to the pixel and subpixel levels;
• Section 3.5 formulates the multi-level loss functions used to jointly supervise all components in an end-to-end training scheme.

3.1. Reparameterization-Based Feature Extraction

We adopt MobileOne as the backbone network due to its lightweight architecture and suitability for deployment in mobile and edge computing scenarios. During training, MobileOne integrates multiple linear branches, which are reparameterized into a unified feed-forward structure during inference. This architectural transformation effectively minimizes memory overhead and improves runtime efficiency, enabling real-time performance on resource-limited platforms.

In detail, we employ the MobileOne-S3 variant. After discarding the classification head and retaining only the convolutional feature extraction layers, the parameter count is significantly reduced from 10.1 million to 0.81 million. This substantial model compression lowers both memory footprint and computational overhead. On the ImageNet dataset, the original model achieves a top-1 accuracy of 78.1% and an inference latency of 1.53 ms on mobile devices, outperforming most reparameterized backbones of comparable size in terms of both effectiveness and speed.

For feature extraction, MobileOne produces $1/8$-scale feature maps, denoted as ${\mathbf{F}}_C^A,{\mathbf{F}}_C^B\in {\mathbb{R}}^{C\times {\displaystyle \frac{H}{8}}\times {\displaystyle \frac{W}{8}}}$, from the input image pair $(I^A,I^B)$. Additionally, we extract higher-resolution features at $1/4$ and $1/2$ scales to preserve finer-grained spatial information. This hierarchical feature representation provides essential contextual cues for subsequent refinement stages, enabling subpixel-level correspondence accuracy while maintaining a favorable balance between accuracy and efficiency.

3.2. Feature Information Propagation

While MobileOne efficiently extracts multi-scale features, it lacks the capacity to model inter-image correspondences, which are essential for accurate matching. To address this, we introduce a lightweight context module that enriches both global and local features.

We propose the Token Aggregation–Interaction Transformer (TAIFormer) as shown in Figure 3, a hybrid architecture that combines convolutional and transformer mechanisms to model intra- and inter-image correspondences. TAIFormer enhances feature representations for robust matching. It comprises two components: the convolutional token mixer (CTM), which integrates multi-head attention and convolution to capture global context and spatial details, and the convolutional feed-forward network (ConvFFN), which replaces fully connected layers with convolutional blocks to improve spatial coherence and reduce redundancy.

Through hierarchical attention and convolutional refinement, TAIFormer yields context-aware features for accurate and efficient correspondence refinement under resource constraints.

3.2.1. Convolutional Token Mixer (CTM)

Traditional token mixers effectively capture long-range dependencies and enable context-aware modeling, but their reliance on large matrix operations often incurs high computational costs and latency. To address this limitation, we employ depth-wise convolution (DWConv) and max pooling (MaxPool) for feature downsampling. This design reduces complexity while preserving essential global contextual cues. In particular, DWConv encodes local spatial structures, whereas MaxPool retains dominant activations by compressing feature responses. The query, key, and value matrices, denoted as $Q,K,V\in {\mathbb{R}}^{C\times {\displaystyle \frac{H}{8}}\times {\displaystyle \frac{W}{8}}}$, are initialized as follows:

$$\begin{array}{cc}\hfill Q& =W_Q\otimes \mathbf{DWConv}\left(f_i\right),\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill K& =W_K\otimes \mathbf{MaxPool}\left(f_j\right),\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill V& =W_V\otimes \mathbf{MaxPool}\left(f_j\right),\hfill \end{array}$$

(3)

where ⊗ denotes matrix multiplication; $W_Q$, $W_K$, and $W_V$ are linear projection layers; $f_i,f_j\in {\mathbb{R}}^{C\times {\displaystyle \frac{H}{8}}\times {\displaystyle \frac{W}{8}}}$ represents the 1/8-scale features obtained during feature extraction. For intra-image correlation, $f_i=f_j$, whereas for inter-image correlation, $f_i\ne f_j$.

• Global Contextual Feature Aggregation

The downsampled features are subsequently processed by multi-head attention (MHA) to enhance global representations, thereby yielding the global feature tensor ${\mathbf{F}}_{global}$ as follows:

$${\mathbf{F}}_{global}=\mathrm{softmax}\left({\displaystyle \frac{Q{K}^T}{\sqrt{d}}}\right)V,$$

(4)

where Q, K, and V denote the query, key, and value matrices, respectively, and $\sqrt{d}$ is a scaling factor used to stabilize gradients.

As shown in Figure 2, intra-image correlation is modeled via self-attention with Rotary Position Embedding (RoPE) [45], which captures dynamic relative dependencies within token sequences. The 2D RoPE is defined as follows:

$$R(x,y)=\left[\begin{array}{cccc}cos{\theta }_x& sin{\theta }_x& 0& 0\\ -sin{\theta }_x& cos{\theta }_x& 0& 0\\ 0& 0& cos{\theta }_y& sin{\theta }_y\\ 0& 0& -sin{\theta }_y& cos{\theta }_y\end{array}\right],{\theta }_x={\mathrm{10,000}}^{-2c/d}x,{\theta }_y={\mathrm{10,000}}^{-2c/d}y,$$

(5)

where c indexes channel pairs, and d is the feature dimension; $(x_i,y_i)$ and $(x_j,y_j)$ are the spatial coordinates of Q and K. Applying the rotations to Q and K, the attention product can be written as

$$Q_{\mathrm{rot}}{K}_{\mathrm{rot}}^{\top }=QR(x_i,y_i){\left(KR(x_j,y_j)\right)}^{\top }=QR(x_j-x_i,y_j-y_i)K^{\top },$$

(6)

showing that attention depends only on the relative positions between features. $Q_{\mathrm{rot}}$ and $K_{\mathrm{rot}}$ are then used in Equation (4) to integrate positional information into self-attention. In contrast, inter-image correlation adopts the same CTM structure with a cross-attention mechanism, omitting RoPE to avoid positional misalignment between different image domains.

• Local Contextual Feature Aggregation

In image matching tasks, point features are often ambiguous or indistinct, especially in low-texture or repetitive regions. Relying solely on global representations may compromise localization and matching accuracy. To address this, local context is incorporated to enhance spatial detail and improve keypoint detection and matching.

The CTM integrates Depthwise Convolution (DWConv) to capture fine-grained local structures and applies a Sigmoid function for adaptive modulation. This design increases sensitivity to subtle local variations and enhances matching reliability. The computation is formulated as follows:

$$\begin{array}{cc}\hfill {\mathbf{F}}_{local}& =\sigma \left(\mathbf{DWConv}\right(Q\left)\right)\odot \mathbf{DWConv}\left(Q\right),\hfill \end{array}$$

(7)

where $\sigma$ denotes the Sigmoid activation function, and ⊙ represents the Hadamard product.

• Dynamic Feature Interaction

To facilitate the effective integration of global and local representations, we design a dynamic feature interaction module that adaptively modulates local features using global contextual cues through Sigmoid. Notably, a weighting factor is computed from the global feature representation ${\mathbf{F}}_{global}$ via the Sigmoid function $\sigma (·)$ and applied to ${\mathbf{F}}_{local}$ through element-wise multiplication. The modulated local and global features are then fused and projected using a $1\times 1$ convolutional layer to produce the final output of the token mixer. The process is expressed as follows:

$$\begin{array}{cc}\hfill {\mathbf{F}}_{fuse}& ={\mathrm{Conv}}_{1\times 1}\left(({\mathbf{F}}_{local}\odot \sigma \left({\mathbf{F}}_{global}\right))\odot {\mathbf{F}}_{global}\right),\hfill \end{array}$$

(8)

where ${\mathbf{F}}_{\mathrm{global}}$ is derived from Equation (4), and ⊙ represents the Hadamard product.

Finally, ${\mathbf{F}}_{fuse}$ is upsampled to match the original input resolution and concatenated with the initial input feature $f_i$ to form the output of the convolutional token mixer (CTM). This fusion strategy effectively preserves both fine-grained spatial details and holistic semantic information for downstream processing.

3.2.2. Convolutional Feed-Forward Network

We employ a convolutional feed-forward network (ConvFFN) to improve spatial sensitivity and local contextual modeling. ConvFFN combines $1\times 1$ point-wise convolutions with DWConv to achieve efficient feature transformation.

The input feature map, ${\mathbf{F}}_{fuse}\in {\mathbb{R}}^{2C\times {\displaystyle \frac{H}{8}}\times {\displaystyle \frac{W}{8}}}$, is first passed through a $1\times 1$ convolutional layer to compress the channel dimension to C. A GELU activation and a subsequent DWConv layer are then applied to capture fine-grained spatial dependencies while minimizing computational overhead. A residual connection is adopted to preserve informative gradients and promote feature discriminability during training. The output is normalized via batch normalization and further processed by another $1\times 1$ convolution to produce the final representation ${\mathbf{F}}_c\in {\mathbb{R}}^{C\times {\displaystyle \frac{H}{8}}\times {\displaystyle \frac{W}{8}}}$, which constitutes the output of the Feature Information Propagation Module.

To ensure a balance between accuracy and efficiency, the channel compression is performed early using the $1\times 1$ convolution, maintaining a lightweight design while preserving the network’s expressive capacity.

This feature propagation module is designed to reinforce both intra-image feature distinctiveness and inter-image feature consistency, both of which are vital for robust visual matching. Accordingly, we utilize four stacked aggregation blocks, each composed of an intra-image TAIFormer layer and an inter-image TAIFormer layer.

Figure 4 presents comparative visualizations of feature outputs from LiteSAM and EfficientLoFTR. To validate the robustness of the findings and ensure fair comparison, LiteSAM was evaluated using both MobileOne and RepVGG—the default backbone of EfficientLoFTR—for feature extraction.

The initial feature maps predominantly emphasize salient foreground regions while suppressing irrelevant background areas such as the sky. After transformer-based TAIFormer refinement, LiteSAM’s features exhibit sharper, more localized attention on true correspondence regions, effectively suppressing background responses. This leads to a higher number of inlier matches and fewer outliers, whereas EfficientLoFTR maintains broad foreground attention, producing more outlier matches.

Furthermore, the refined features from LiteSAM show stronger cross-image correlation, with higher spatial coherence and cross-view similarity. This enhanced feature correspondence directly improves matching precision, ensuring that keypoint pairs align more faithfully with the true geometric relationships in challenging satellite–aerial image pairs. LiteSAM with TAIFormer demonstrates superior semantic region representation, higher inlier ratios, and reduced outliers compared to EfficientLoFTR.

3.3. Coarse-Level Matching Module

Following the feature propagation process, the resulting $1/8$-scale feature maps $({\mathbf{F}}_c^A,{\mathbf{F}}_c^B)\in {\mathbb{R}}^{C\times {\displaystyle \frac{H}{8}}\times {\displaystyle \frac{W}{8}}}$ are obtained from the two input images. A coarse matching score matrix $\mathbf{S}$ is subsequently computed based on these feature representations, which is defined as

$$\begin{array}{c}\hfill \mathbf{S}(i,j)={\displaystyle \frac{\langle {\mathbf{F}}_c^A\left(i\right),{\mathbf{F}}_c^B\left(j\right)\rangle }{\tau }},\end{array}$$

(9)

where $\langle ·,·\rangle$ denotes the inner product operation, and $\tau$ is a temperature coefficient that controls the sharpness of the similarity distribution. To obtain a valid matching probability distribution, the dual-softmax technique [15] is applied to normalize S along both row-wise and column-wise dimensions, yielding the coarse matching probability matrix $P_c$:

$$\begin{array}{c}\hfill P_c(i,j)=\mathrm{softmax}{\left(\mathbf{S}(i,·)\right)}_j·\mathrm{softmax}{\left(\mathbf{S}(·,j)\right)}_i,\end{array}$$

(10)

where $\mathbf{S}(i,·)$ denotes the i-th row of $\mathbf{S}$, reflecting the similarity between the i-th feature in image A and all features in image B, while $\mathbf{S}(·,j)$ represents the j-th column, indicating the reverse.

To ensure robust initial correspondences, a confidence-based thresholding strategy combined with mutual nearest neighbor (MNN) filtering is employed to construct a binary matching mask $M_c$, which is defined as

$$M_c(i,j)=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}\mathbf{S}(i,j)\ge {\theta }_c\mathrm{and}\mathrm{is}\mathrm{row}/\mathrm{column}\max ,\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(11)

where ${\theta }_c$ is a predefined confidence threshold used to eliminate low-probability matches. A value of $M_c(i,j)=1$ indicates that the feature pair $(i,j)$ satisfies both mutual nearest neighbor and confidence criteria, thereby constituting a valid coarse-level correspondence; otherwise, the pair is discarded.

Importantly, the dual-softmax operation plays a crucial role during training by imposing contextual constraints that emphasize the learning of discriminative features, thereby reinforcing the confidence of correct matches while suppressing ambiguous ones. Once the model attains sufficient discriminative power, this operation becomes optional during inference to alleviate computational overhead.

To speed up inference, especially for high-resolution images, LiteSAM (opt.) skips the dual-softmax step and directly applies the MNN strategy to the raw score matrix S. This adjustment simplifies the matching process while maintaining a balance between accuracy and efficiency.

3.4. Learnable Correspondence Refinement Module

To further refine the accuracy of feature correspondences, a learnable refinement module is devised, which systematically optimizes matching precision through a two-stage process: pixel-level refinement followed by subpixel-level adjustment. The first stage focuses on spatial alignment at the pixel scale, while the subsequent stage utilizes a learnable iterative strategy to achieve high-precision subpixel matching.

3.4.1. Pixel-Level Refinement

Building upon the feature pyramid fusion strategy, the propagated feature maps $({\mathbf{F}}_c^A,{\mathbf{F}}_c^B)$ from the Feature Information Propagation Module are upsampled to the original image resolution. Intermediate-resolution features (at 1/4 and 1/2 scales), obtained during the initial extraction phase, are integrated to reconstruct the pixel-level features $({\mathbf{F}}_f^A,{\mathbf{F}}_f^B)\in {\mathbb{R}}^{C/4\times H\times W}$, which retain rich spatial granularity essential for fine-grained matching.

For each coarse-level correspondence in $M_c$, a localized feature patch is cropped from the fine-resolution feature maps. Within this patch, a localized similarity matrix $S_f$ is formulated. Dual-softmax normalization is then conducted to derive a fine-grained matching probability matrix $P_f$. To ensure match reliability, a Global Nearest Neighbor (GNN) filtering strategy is subsequently applied to $P_f$, retaining only the most confident correspondence within each local region.

3.4.2. Subpixel-Level Refinement

To address the resolution limitations imposed by discrete pixel grids, this module models continuous feature distributions to achieve subpixel matching, thereby suppressing spurious correspondences and ensuring geometric consistency. This is in contrast to heatmap-based refinement methods [16], which exhibit limited nonlinear modeling capacity and are prone to super-linear errors in repetitive structures. We propose a learnable refinement module that dynamically captures local contextual dependencies through a stacked MinGRU [46] architecture. As illustrated in Figure 5, this module performs fine-grained corrections of initial matches by iteratively updating subpixel coordinates, thereby attaining high matching precision and robust generalization across diverse visual conditions.

Given an initial correspondence $(k_i^A,k_j^B)$, where $k_i^A$ and $k_j^B$ denote the i-th and j-th feature points in the reference image $I^A$ and target image $I^B$, respectively, and represent potentially corresponding locations across the two images, a $3\times 3$ search window is centered at $k_j^B$, restricting the candidate region to its local neighborhood. To refine the estimated location, a multi-layer MinGRU network is instantiated to encode the correlations between $k_i^A$ and each candidate point within the window. Through iterative updates, the position of $k_j^B$ is progressively adjusted toward sub-pixel accuracy.

Each MinGRU unit receives local correlation features and the hidden representation from the preceding layer, enabling context-aware updates. Unlike conventional GRUs that depend on temporal recurrence via $h_{t-1}$, the MinGRU formulation [46] eliminates this dependency, thereby facilitating parallel inferences without sacrificing modeling capacity. The update process is defined as follows:

$$\begin{array}{cc}\hfill z_t& =\sigma \left({\mathbf{Linear}}_{d_h}\left(C_f\right)\right),\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill {\tilde{h}}_t& ={\mathbf{Linear}}_{d_h}\left(C_f\right),\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill h_t& =(1-z_t)\odot h_{t-1}+z_t\odot {\tilde{h}}_t,\hfill \end{array}$$

(14)

where $C_f$ denotes the current correlation input, $\sigma (·)$ is the sigmoid activation function, and ⊙ represents the Hadamard product. All operations involving $z_t$ and ${\tilde{h}}_t$ are based solely on $C_f$, supporting the parallel derivation of $h_t$. The final subpixel offset is computed via the following:

$$\Delta =W_{\mathrm{offset}}\otimes \mathbf{ReLU}\left(h_t\right),$$

(15)

$$\delta =tanh(\Delta ),$$

(16)

where $W_{\mathrm{offset}}$ denotes a learnable projection matrix, and $tanh(·)$ constrains the predicted offset $\delta$ within $[-1,1]$ to ensure numerical stability.

This learnable correspondence refinement module independently optimizes both pixel-level and subpixel coordinates, effectively decoupling their estimation processes. By leveraging large-scale end-to-end supervision, it adaptively learns spatial offsets, thereby significantly improving the accuracy, robustness, and generalization capability of feature matching across various scenarios.

3.5. Loss Function

In this work, three auxiliary loss functions are utilized at different stages of the matching process: coarse, pixel-level, and subpixel-level matching. These loss functions serve to guide the model in learning precise matching relationships at various scales, thereby optimizing the overall matching accuracy at each stage.

3.5.1. Coarse-Level Matching Loss Function

During the coarse matching phase, the approach presented in [17] is adopted, where the relative pose between the depth map and images is leveraged to project grid points from the reference image, $I_A$, onto the target image $I_B$. This projection facilitates the construction of a ground-truth coarse matching set, denoted as ${\left\{{\mathcal{M}}_c\right\}}_{gt}$, which includes a total of $N_c$ matching points. As the feature maps are downsampled to 1/8 of the original image resolution, the matching points are annotated and supervised at this scale.

Furthermore, the model’s predicted coarse matching probability matrix, denoted as ${\mathcal{P}}_c$, is supervised with respect to the ground-truth coarse matching set. To optimize the coarse matching process, a negative log-likelihood loss function is employed. The objective of this loss function is to minimize the negative log-likelihood at the ground-truth matching positions, ${\left\{{\mathcal{M}}_c\right\}}_{gt}$, thereby enhancing the model’s ability to effectively capture and learn global matching relationships. The coarse matching loss function is defined as follows:

$$\begin{array}{c}\hfill {\mathcal{L}}_c=-{\displaystyle \frac{1}{N_c}}\sum _{(i,j)\in {\left\{{\mathcal{M}}_c\right\}}_{gt}}log{\mathcal{P}}_c(i,j)\end{array}$$

(17)

Here, ${\mathcal{P}}_c(i,j)$ denotes the matching probability at position $(i,j)$ during the coarse matching phase, and $N_c$ denotes the total number of ground-truth matches at the coarse matching stage.

3.5.2. Pixel-Level Matching Loss Function

In the pixel-level matching phase, feature point pairs obtained from the coarse matching stage undergo further refinement within corresponding $8\times 8$ pixel blocks at the original image resolution. Each coarse match is associated with an $8\times 8$ pixel region within the reference image $I_A$, and the objective of pixel-level matching is to identify the optimal pixel pair within the corresponding local blocks of both the reference and target images, thereby achieving precise feature alignment.

To facilitate this, projective geometry is employed in combination with camera extrinsics and depth map information [16]. Specifically, pixels within the $8\times 8$ blocks of the reference image $I_A$ are projected onto the target image $I_B$. This projection process constructs the ground-truth pixel-level matching set, denoted as ${\left\{{\mathcal{M}}_f\right\}}_{gt}$.

A negative log-likelihood loss function supervises the estimation of matching probabilities corresponding to the ground-truth positions in the probability matrix ${\mathcal{P}}_f$. The pixel-level matching loss function is defined as follows:

$$\begin{array}{c}\hfill {\mathcal{L}}_f=-{\displaystyle \frac{1}{N_f}}\sum _{(i,j)\in {\left\{{\mathcal{M}}_f\right\}}_{gt}}log{\mathcal{P}}_f(i,j)\end{array}$$

(18)

Here, ${\mathcal{P}}_f(i,j)$ represents the score at position $(i,j)$ within the fine-grained correlation score matrix, computed over a local matching window, while $N_f$ represents the number of valid ground-truth matches involved in the fine-grained matching process.

3.5.3. Subpixel-Level Matching Loss Function

The subpixel-level matching stage refines the pixel-level matching results, thereby enhancing the accuracy of the final matching positions to a subpixel resolution. To achieve this, depth maps and camera extrinsics are leveraged to precisely project pixel-level correspondences from the reference image $I_A$ onto the target image $I_B$ [16]. This projection generates continuous coordinate values for the resulting positions, enabling the construction of the ground-truth subpixel-level matching set, denoted as ${\left\{{\mathcal{M}}_s\right\}}_{gt}$, which includes not only integer pixel positions but also floating-point values that accurately capture subpixel-level correspondences.

To refine the model’s subpixel prediction accuracy, the Euclidean distance is used as the supervisory signal. This loss function quantifies the deviation between the predicted subpixel matching positions ${\mathcal{M}}_s$ and the ground-truth subpixel positions ${\left\{{\mathcal{M}}_s\right\}}_{gt}$. The subpixel matching loss is defined as follows:

$$\begin{array}{c}\hfill {\mathcal{L}}_s={\displaystyle \frac{1}{N}}\sum _{i=1}^{N_s}{\parallel {\left\{{\mathcal{M}}_s\right\}}^{\left(i\right)}-{\left\{{\mathcal{M}}_s\right\}}_{gt}^{\left(i\right)}\parallel }_2^2\end{array}$$

(19)

Here, $N_s$ represents the total number of matching points, ${\left\{{\mathcal{M}}_s\right\}}^{\left(i\right)}$ denotes the i-th predicted subpixel matching point, ${\left\{{\mathcal{M}}_s\right\}}_{gt}^{\left(i\right)}$ denotes the number of subpixel level ground-truth correspondences used for supervision, and ${|·|}_2$ denotes the Euclidean norm.

The final optimization objective is a weighted sum of the three loss functions. The overall loss function is expressed as follows:

$$\begin{array}{c}\hfill \mathcal{L}={\mathcal{L}}_c+\alpha {\mathcal{L}}_f+\beta {\mathcal{L}}_s\end{array}$$

(20)

where $\alpha$ and $\beta$ are weighting coefficients that regulate the contributions of each matching-level loss function. By jointly optimizing this multi-scale loss function, the model progressively refines its matching accuracy from coarse to pixel-level and subpixel-level precision, yielding robust and high-precision feature matching across various scales.

4. Experiments

We conduct a comprehensive evaluation of the proposed model across benchmark tasks involving natural scenes and satellite–aerial image matching. In natural scenes, the model is evaluated on downstream tasks including relative pose estimation, homography estimation, and visual localization. For satellite–aerial imagery, evaluation focuses on AVL. To further validate the effectiveness of each component, detailed ablation studies are carried out. Additionally, the framework is deployed on edge devices to demonstrate its real-time inference performance and localization accuracy under resource-constrained conditions.

4.1. Datasets

For relative pose estimation, we utilize the following datasets:

• MegaDepth [21] encompasses 196 outdoor landmark scenes, featuring over 130,000 images, along with high-quality depth ground truth derived from Structure-from-Motion and Multi-View Stereo techniques. We evaluate 1500 test image pairs, resized to a maximum dimension of 1200 pixels, to assess robustness to variations in illumination and viewpoint.
• ScanNet [22] comprises 1513 indoor scenes with over 2.5 million RGB-D frames, which provides precise 3D reconstruction ground truth. We select 1500 test pairs at a resolution of 640 × 480 pixels to evaluate performance in texture-limited indoor environments.

For absolute visual localization (AVL), which involves matching aerial images to satellite imagery, we employ the following:

• The Aerial Image Dataset [47] includes 500 test pairs of aerial images at 512 × 512 pixels, with ground-truth affine transformations to evaluate matching accuracy.
• UAV-VisLoc [19] encompasses 6744 unmanned aerial vehicle (UAV) images collected from 11 distinct locations across China, captured at altitudes ranging from 400 to 2000 m. These images span urban, rural, and agricultural settings, with real-time kinematic (RTK) positioning and heading angles provided as ground truth. Following the data augmentation pipeline detailed in Figure 6, the dataset yields 20,232 satellite images, each resized to 1184 pixels along the longer dimension.
• The Self-made Dataset includes 2284 UAV images acquired from 14 scenes in Harbin and Qiqihar, captured at altitudes between 100 and 500 m. Each image is annotated with RTK-based ground truth and heading angles. After applying the augmentation pipeline illustrated in Figure 6, the dataset generates 6852 satellite images.

For visual localization, which estimates 6 DoF camera poses within the HLoc framework, we carry out evaluations on the following datasets:

• Aachen Day-Night v1.1 [48] includes 6697 reference images and 1015 query images, featuring variations in viewpoint and illumination.
• InLoc [49] comprises 9972 reference images and 329 query images from texture-scarce indoor environments; this dataset incorporates depth supervision for evaluation.

For homography estimation, we use the following:

• HPatches [20] contains 580 image pairs across 116 scenes, with 285 pairs exhibiting illumination variations and 295 pairs with viewpoint changes. Images are resized to 640 × 480 pixels to test the accuracy of feature matching and homography estimation.

4.2. Implementation Details

The architecture, outlined in Section 3, is implemented in PyTorch using PyTorch Lightning for distributed training. MobileOne-S3 [50] extracts multi-scale features at 1/2, 1/4, and 1/8 resolutions. Features at 1/8 scale undergo $4\times 4$ convolution or max pooling, followed by four TAIFormer layers for coarse matching, with threshold ${\theta }_c$ in Section 3.3 set to 0.2 (20 in opt mode). Subpixel refinement uses four MinGRU units. Training uses 18,400 image pairs from 368 MegaDepth sub-scenes (50 pairs each) resized to 832 × 832 pixels with padding. We carry out training for 30 epochs using a batch size of 14, an AdamW optimizer (learning rate 0.002, weight decay 0.1), and loss weights $\alpha =1.0$ and $\beta =0.2$ on two NVIDIA A100 GPUs for 48 h. The model is evaluated on an NVIDIA RTX 3090 GPU.

4.3. Baseline

We evaluated LiteSAM against state-of-the-art feature matching methods on the datasets described in Section 4.1, covering sparse, semi-dense, and dense methods. Sparse methods include D2Net [51], R2D2 [52], DISK [53], and SuperPoint [23] (using nearest-neighbor matching), as well as SuperGlue [17], LightGlue [24], and OmniGlue [18]. Semi-dense methods comprise Sparse-NCNet [54], DRC-Net [55], LoFTR [15], MatchFormer [26], AspanFormer [14], TopicFM [27], RCM [56], SAM [57], EfficientLoFTR [16], DeepMatcher [29], GeoAT [30], Homomatcher [34], and Jamma [31]. Dense methods include DKM [32], ROMA [33], and PMatch [58]. Except for the results on the MegaDepth, Aerial Image Dataset, UAV-VisLoc, and Self-made Dataset, and the results of SP+LG and EfficientLoFTR (opt.) on the HPatches dataset, which were reproduced by the authors under a unified experimental environment, all other comparative results reported for different datasets are directly taken from the corresponding original publications to ensure fairness and reproducibility. Moreover, the runtime measurements reported for the MegaDepth dataset were obtained using the standardized testing protocol described in Section 4.2, and the reproduced results on the HPatches dataset are clearly indicated within the relevant table.

4.4. Relative Pose Estimation

4.4.1. Metric

Feature matching performance is evaluated through relative pose estimation, where correspondences are used to estimate the relative pose matrix via RANSAC. Accuracy is evaluated by measuring the angular errors in rotation and translation between the estimated and ground-truth poses. Following standard practice, we adopt the Area Under the Curve (AUC) at angular error thresholds of $5^{°}$, ${10}^{°}$, and ${20}^{°}$, using the greater of the two errors as the evaluation criterion.

4.4.2. Results

As shown in

Table 1. Relative pose estimation results on MegaDepth and ScanNet. All models were trained solely on the MegaDepth dataset and evaluated across all benchmarks to assess generalization performance. Reported metrics include pose estimation errors at multiple thresholds and inference speeds on the MegaDepth dataset at an input resolution of $1184\times 1184$ pixels. In the semi-dense category, the best result is in red, the second-best is in orange, and the third-best is in blue. The direction indicated by the arrows denotes superior performance.

Category	Method	MegaDepth				ScanNet			Times (ms)
		AUC@5$°\uparrow$	AUC@10$°\uparrow$	AUC@20$°\uparrow$		AUC@5$°\uparrow$	AUC@10$°\uparrow$	AUC@20$°\uparrow$
Sparse	SP+SG	42.2	61.2	76.0		10.4	22.9	37.2	99.4
	SP+LG	47.6	64.8	77.9		15.1	32.6	61.02	47.1
	SP+OG	47.4	65.0	77.8		14.0	28.9	44.3	414.9
Semi-Dense	LoFTR	52.8	69.2	81.2		16.9	33.6	50.6	314.82
	MatchFormer	53.3	69.7	81.8		15.8	32.0	48.0	691.0
	ASpanFormer	55.3	71.5	83.1		19.6	37.7	53.3	352.07
	TopicFM	54.1	70.1	81.6		17.3	35.5	50.9	270.69
	TopicFM+	58.2	72.8	83.2		20.4	38.5	54.5	220.37
	RCM	53.2	69.4	81.5		17.3	34.6	52.1	-
	DeepMatcher	55.7	72.2	83.4		-	-	-	-
	EfficientLoFTR	56.4	72.2	83.5		19.2	37.0	53.6	154.2
	EfficientLoFTR (opt.)	55.4	71.4	82.9		17.4	34.4	51.2	98.8
	GeoAT	53.6	70.3	82.2		-	-	-	-
	ASpan_Homo	57.1	73.0	84.1		22.0	40.5	57.2	>352
	LiteSAM	56.1	72.0	83.4		21.0	39.3	55.9	133.0
	LiteSAM (opt.)	56.3	72.1	83.4		19.5	37.8	54.7	79.9
Dense	DKM	60.4	74.9	85.1		26.64	47.07	64.17	528.21
	ROMA	62.6	76.7	86.3		28.9	50.4	68.3	3336.22

, LiteSAM exhibits robust generalization and competitive performance on MegaDepth and ScanNet. LiteSAM (opt.) outperforms EfficientLoFTR (opt.) while reducing inference time on MegaDepth, and closely matches LiteSAM, demonstrating high stability. On ScanNet, LiteSAM (opt.) surpasses SP+LG among sparse methods, with a moderate increase in latency. In comparison to dense methods, it significantly reduces inference times relative to DKM and ROMA, with modest performance reductions. This highlights LiteSAM’s effective balance of accuracy and efficiency for time-sensitive applications.

4.5. Absolute Visual Localization

4.5.1. Metric

To comprehensively evaluate matching and localization performance, multiple quantitative metrics are employed based on the ground truth from each dataset. Specifically, the Aerial Image Dataset [47] is evaluated using the Percentage of Correct Keypoints (PCK), whereas UAV-VisLoc and self-made datasets adopt a hit rate (HR) and a Root Mean Square Error within 30 m (RMSE@30) as the primary evaluation indicators.

(1)

Percentage of Correct Keypoints (PCK)

For the Aerial Image Dataset, PCK is applied to quantify the matching fidelity under affine transformations estimated through RANSAC. Each image pair is associated with a predicted transformation matrix ${\mathcal{T}}_{\theta }$ and the corresponding ground-truth matrix ${\mathcal{T}}_{\mathrm{gt}}$, both of which are used to warp 20 predefined keypoints. The proportion of correctly transformed points is computed across various pixel error thresholds $\tau \in 0.01,0.03,0.05$ as follows:

$$PCK={\displaystyle \frac{{\sum }_{i=1}^n{\sum }_{p_i}\mathbb{1}[d({\mathcal{T}}_{\theta }\left(p_i\right),{\mathcal{T}}_{\mathrm{gt}}\left(p_i\right))<\tau ·max(h,w)]}{{\sum }_{i=1}^n\left|p_i\right|}}$$

(21)

where $d(·,·)$ denotes the Euclidean distance, and $\mathbb{1}[·]$ is the indicator function.

(2)

Localization Error

For the UAV-VisLoc and self-made datasets, localization accuracy is evaluated by estimating the homography matrix H between UAV and satellite images using RANSAC. The UAV image’s center is projected onto the satellite view, and the localization error is quantified as the distance between predicted and ground-truth coordinates using the Haversine formula:

$$a={sin}^2\left({\displaystyle \frac{\Delta \phi }{2}}\right)+cos\left({\phi }_{\mathrm{pred}}\right)cos\left({\phi }_{\mathrm{gt}}\right){sin}^2\left({\displaystyle \frac{\Delta \lambda }{2}}\right),$$

(22)

$$d=R·2atan2(\sqrt{a},\sqrt{1-a})\times 1000,$$

(23)

where $R=6371\mathrm{km}$ is Earth’s radius, and d is the distance in meters. The RMSE@30 is calculated as follows:

$$\mathrm{RMSE}=\sqrt{{\displaystyle \frac{1}{N}}\sum _{i=1}^N{d}_i^2},$$

(24)

where $d_i$ is the error for the i-th sample, and N is the number of samples with errors below 30 m.

(3)

Hit Rate (HR)

To assess robustness in weak-texture or low-similarity scenarios, the hit rate (HR) is defined as follows:

$$\mathrm{HR}={\displaystyle \frac{N_{\mathrm{hit}}}{N_{\mathrm{total}}}},$$

(25)

where $N_{\mathrm{hit}}$ is the number of image pairs with localization errors below 30 m, and $N_{\mathrm{total}}$ is the total number evaluated. HR reflects the localization system’s reliability and the matching strategy’s robustness in challenging visual environments.

(4)

Efficiency Indicators

In addition to accuracy metrics, we report model complexity, computational load, and inference latency to highlight LiteSAM’s practical advantages. To ensure fair comparisons, FLOPs and runtime are averaged over correctly localized samples, accounting for variations in input resolution and image content.

4.5.2. Results

(1)

Aerial Image Dataset

As shown in

Table 2. Absolute visual localization results on the Aerial Image Dataset. All models were tested on $512\times 512$ images. The Percentage of Correct Keypoints (PCK), inference speed, FLOPs, and model parameters are shown. In the Semi-Dense category, the best result is in red, the second-best is in orange, and the third-best is in blue. The direction indicated by the arrows denotes superior performance.

Category	Method	DeepAerial			Time (ms)	FLOPs (G)	Params (M)
		PCK@1% ↑	PCK@3% ↑	PCK@5% ↑
Sparse	SP+NN	11.5	19.0	23.2	9.53	44.46	1.3
	SP+SG	8.4	16.5	19.8	66.85	108.13	13.32
	SP+LG	21.4	27.0	29.3	43.74	91.62	11.38
Semi-Dense	LoFTR	18.5	24.8	28.1	52.74	308.73	11.56
	AspanFormer	21.5	26.1	28.5	135.52	551.76	15.75
	EfficientLoFTR	23.3	30.3	33.2	36.55	189.40	15.05
	EfficientLoFTR (opt.)	21.4	29.9	32.8	32.08	189.28	15.05
	LiteSAM	23.7	30.2	32.7	33.29	92.39	6.31
	LiteSAM (opt.)	22.2	29.2	31.9	29.64	92.27	6.31
Dense	DKM	30.6	36.1	38.9	291.62	1543.86	70.21
	ROMA	38.0	50.8	56.4	243.88	1864.42	111.29

, LiteSAM achieves competitive matching accuracy among semi-dense approaches while offering clear advantages in model efficiency. Specifically, LiteSAM achieves a PCK@3% of 30.2%, closely matching the best-performing EfficientLoFTR but with significantly fewer parameters and lower FLOPs, resulting in faster inference.

Compared to traditional sparse methods such as SP+LG, LiteSAM improves accuracy by 11.3% under the PCK@3% metric while reducing inference latency by 23.9%. These gains highlight the benefit of its lightweight feature extraction and attention-based propagation. While dense models like ROMA yield the highest accuracy, they incur prohibitive computational costs, making them less suitable for UAV scenarios requiring real-time performance.

(2)

UAV-VisLoc and Self-made datasets

As shown in

Table 3. Absolute visual localization results on UAV-VisLoc dataset. Metrics include RMSE under a 30 m threshold, average hit rate, number of matched points, and per-frame inference time across Easy, Moderate, and Hard settings. In the Semi-Dense category, the best result is in red, the second-best is in orange, and the third-best is in blue. The direction indicated by the arrows denotes superior performance.

Category	Method	UAV-VisLoc Dataset (Easy/Moderate/Hard)			Time (ms) ↓
		RMSE@30 ↓	Avg.HR (%) ↑	Num of Point ↑
Sparse	SP+NN	18.71	37.15/29.54/19.89	819/820/804	19.82
	SP+SG	17.88	58.38/56.64/51.03	454/378/251	138.27
	SP+LG	17.81	60.34/59.57/54.32	473/402/276	44.15
Semi-Dense	LoFTR	17.99	59.96/59.96/50.59	548/422/254	135.09
	AspanFormer	17.87	62.92/59.10/49.96	1368/887/556	159.16
	EfficientLoFTR	17.87	65.78/63.62/57.65	1849/1483/971	112.60
	EfficientLoFTR (opt.)	17.94	63.85/57.78/42.30	3873/3311/2639	77.53
	LiteSAM	17.86	66.66/65.37/61.65	2096/1662/1065	83.79
	LiteSAM (opt.)	17.98	65.09/61.34/46.16	4227/3587/2826	60.97
Dense	DKM	17.87	59.11/57.29/52.52	10,000/10,000/10,000	498.88
	ROMA	17.81	64.13/60.67/52.70	10,000/10,000/10,000	688.32

and

Table 4. Absolute visual localization results on self-made dataset. Results include RMSE within 30 m, average hit rate, number of correspondences, inference time, and FLOPs across varying difficulty levels. In the Semi-Dense category, the best result is in red, the second-best is in orange, and the third-best is in blue. The direction indicated by the arrows denotes superior performance.

Category	Method	Self-Made Dataset (Easy/Moderate/Hard)			Time (ms)	FLOPs (G)
		RMSE@30 ↓	Avg.HR (%) ↑	Num of Point
Sparse	SP+NN	13.26	30.27/21.37/14.08	1123/1103/1071	20.64	148.7
	SP+SG	5.70	71.28/64.96/54.78	492/371/233	142.28	967.9
	SP+LG	6.76	78.85/70.03/58.31	622/482/322	49.49	942.16
Semi-Dense	LoFTR	8.74	63.03/51.71/41.09	288/147/87	145.61	1570.92
	AspanFormer	7.93	73.52/59.85/42.03	863/515/272	163.45	1738.81
	EfficientLoFTR	7.28	90.03/79.79/61.84	1540/1114/728	120.72	1036.61
	EfficientLoFTR (opt.)	8.43	88.95/75.84/54.17	4526/3953/3382	80.34	1033.54
	LiteSAM	6.12	92.09/87.88/77.30	1905/1406/888	85.31	588.51
	LiteSAM (opt.)	6.89	92.34/86.93/72.10	4356/3743/3128	61.98	586.06
Dense	DKM	5.86	77.09/75.15/71.06	10,000/10,000/10,000	512.71	3022.67
	ROMA	5.60	92.00/86.11/76.10	10,000/10,000/10,000	691.48	6750.08

, LiteSAM consistently outperforms existing methods in both accuracy and efficiency. Unlike methods that degrade in complex scenes, it maintains robustness. On the self-made dataset, it achieves competitive accuracy, with the optimized variant further balancing precision and computational cost. Figure 7 and Figure 8 qualitatively illustrate these advantages. Notably, LiteSAM produces more matched points and more consistent spatial distribution, which is also quantitatively reflected in the tables, demonstrating its superior matching quality.

LiteSAM integrates a reparameterized lightweight backbone, TAIFormer-based feature propagation, and iterative MinGRU refinement to achieve high-precision matching with low latencies and minimal overhead. Pretrained on MegaDepth [21], it generalizes well to cross-view settings. Compared to dense methods, it significantly reduces model size and runtime, even at high resolutions.

While some sparse baselines run faster, LiteSAM offers more reliable correspondences—beneficial for tasks like 3D reconstruction. Overall, it achieves a strong trade-off among accuracy, speed, and efficiency, making it well-suited for UAV localization and real-world deployment.

4.6. Visual Localization

4.6.1. Metric

The primary evaluation metric follows the HLoc protocol [59], which employs the Percentage of Correct Poses (PCP). Pose accuracy is quantified under three thresholds: $(0.25\mathrm{m},2^{°})$, $(0.5\mathrm{m},5^{°})$, and $(1.0\mathrm{m},{10}^{°})$. By comparing the predicted pose against the ground truth, this metric provides a comprehensive assessment of localization precision and robustness. Moreover, it indirectly reflects the quality of feature correspondences established by LiteSAM.

4.6.2. Results

On the Aachen Day-Night v1.1 benchmark, LiteSAM and LiteSAM (opt.) achieve robust accuracy across varying illumination, including night-time scenes (

Table 5. Visual localization results on the Aachen Day-Night v1.1 and InLoc datasets. The visual localization performance is evaluated on the Aachen Day-Night v1.1 [48] and InLoc [49] datasets using the HLoc framework [59]. In the Semi-Dense category, the best result is in red, the second-best is in orange, and the third-best is in blue. The direction indicated by the arrows denotes superior performance.

Category	Method	AachenV1.1			Inloc
		Day	Night		DUC1	DUC2
		(0.25 m, 2°)/(0.5 m, 5°)/(1.0 m, 10°)			(0.25 m, 2°)/(0.5 m, 5°)/(1.0 m, 10°)
Sparse	SP+SG	89.8/96.1/99.4	77.0/90.6/100.0		49.0/68.7/80.8	53.4/77.1/82.4
	SP+LG	89.2/95.4/98.5	87.8/93.9/100.0		49.0/68.2/79.3	55.0/74.8/79.4
Semi-Dense	LoFTR	88.7/95.6/99.0	78.5/91.1/99.0		47.5/72.2/85.5	54.2/74.8/85.5
	MatchFormer	-	-		46.5/73.2/85.9	55.7/71.8/85.5
	ASpanFormer	89.4/95.6/99.0	77.5/91.6/99.5		51.5/73.7/86.4	55.0/74.0/85.5
	TopicFM	90.2/95.9/98.9	77.5/91.1/99.5		52.0/74.7/87.4	53.4/74.8/83.2
	RCM	89.7/96.0/98.7	72.8/91.6/99.0		-	-
	SAM	89.7/95.8/99.0	78.6/91.8/100.0		51.8/73.9/87.8	56.0/75.8/83.1
	EfficientLoFTR	89.6/96.2/99.0	77.0/91.1/99.5		52.0/74.7/86.9	58.0/80.9/89.3
	LiteSAM	88.6/96.0/98.8	76.4/91.1/99.5		52.0/73.7/86.4	60.3/85.5/89.3
	LiteSAM (opt.)	88.6/95.5/98.8	76.4/90.6/99.5		54.0/74.2/86.9	55.7/80.9/85.5
Dense	DKM	-	-		51.5/75.3/86.9	63.4/82.4/87.8
	ROMA	-	-		60.6/79.3/89.9	66.4/83.2/87.8

). LiteSAM (opt.) offers faster inference with minimal accuracy trade-off. On InLoc, both outperform sparse and semi-dense methods in localization accuracy, leveraging efficient feature matching in complex indoor settings. LiteSAM’s lightweight backbone, TAIFormer propagation, and MinGRU-based refinement strike a balance between accuracy and efficiency, making it suitable for latency-sensitive applications such as UAV navigation and autonomous driving.

4.7. Homography Estimation

4.7.1. Metric

Following the evaluation protocol in [16], the primary metric is Corner Correctness, which measures the average deviation between the warped image corners (via estimated homography) and ground-truth projections. The proportion of image pairs with corner errors below thresholds of 3, 5, and 10 pixels is reported. All images are resized such that the shorter side is 480 pixels, ensuring uniform input resolution. Homography matrices are computed using 1000 matched correspondences through an RANSAC-based solver, with identical preprocessing across all evaluated methods.

4.7.2. Results

On the HPatches dataset, LiteSAM achieves competitive accuracy among semi-dense methods, outperforming others across all corner error thresholds (

Table 6. Homography estimation results on the HPatches dataset. Performance comparison of feature matching methods based on the percentage of image pairs with average corner error below predefined pixel thresholds. All images are resized to 480 pixels (short side), and homographies are estimated from 1000 correspondences using an RANSAC-based solver. In the Semi-Dense category, the best result is in red, the second-best is in orange, and the third-best is in blue. Methods marked with ^† are tested under the same experimental conditions as LiteSAM for a fair comparison. The direction indicated by the arrows denotes superior performance.

Category	Method	Homography est. AUC
		@3 px↑	@5 px↑	@10 px↑
Sparse	D2Net+NN	23.2	35.9	53.6
	R2D2+NN	50.6	63.9	76.8
	DISK+NN	52.3	64.9	78.9
	SP+SG	53.9	68.3	81.7
	SP+LG †	60.8	72.3	84.0
Semi-Dense	Sparse-NCNet	48.9	54.2	67.1
	DRC-Net	50.6	56.2	68.3
	LoFTR	65.9	75.6	84.6
	EfficientLoFTR	66.5	76.4	85.5
	EfficientLoFTR (opt.) †	65.1	75.2	84.8
	AspanFormer	67.4	76.9	85.6
	TopicM	67.3	77.0	85.7
	GeoAT	69.1	78.2	87.1
	ASpan_Homo	70.2	79.6	87.8
	JamMa	68.1	77.0	85.4
	LiteSAM	67.3	77.2	86.1
	LiteSAM (opt.)	65.8	76.2	85.6
Dense	DKM	71.3	80.6	88.5
	Pmatch	71.9	80.7	88.5

). Compared to EfficientLoFTR, it improves homography estimations at 3 px, 5 px, and 10 px with similar efficiency. The optimized variant, LiteSAM (opt.), slightly reduces accuracy but offers faster inference and lower computational cost. While dense methods like DKM and Pmatch achieve higher accuracy, they incur significantly greater resource demands. LiteSAM thus offers a favorable balance of precision and efficiency for real-world applications.

4.8. Ablation Study

To evaluate LiteSAM’s components, we conducted ablation studies on self-made and MegaDepth datasets at $1184\times 1184$ resolution, as shown in

Table 7. Ablation study on self-made dataset. AVL performance on the self-made dataset, reported in terms of RMSE@30m, average hit rate, number of correspondences, inference time, and computational cost (FLOPs) across different architectural variants of LiteSAM. The best result is in red, the second-best is in orange, and the third-best is in blue. The direction indicated by the arrows denotes superior performance.

Method	Self-Made Dataset(Easy/Moderate/Hard)			FLOPs (G)	Time (ms)
	RMSE@30 ↓	Avg.HR (%) ↑	Num of Point
Full	6.12	92.09/87.88/77.30	1905/1406/888	588.51	85.31
(1) Replace MobileOne with RepVGG	6.46	92.95/89.81/79.53	1520/1132/756	1066.24	100.77
(2) Replace TAIFormer to Agg. Attention	6.94	89.68/81.38/63.99	1477/1091/727	697.11	99.72
(3) Replace MinGRU to heatmap refine	6.39	91.75/86.67/73.45	1759/1317/859	588.49	86.49
(4) Remove MinGRU refinement	6.47	90.14/83.71/70.49	1864/1385/867	588.49	83.94
(5) Our Optimal (w/o dual softmax)	6.89	92.34/86.93/72.10	4356/3743/3128	586.06	61.98

and

Table 8. Ablation study on MegaDepth dataset. Relative pose estimation performance of LiteSAM variants, trained and tested on MegaDepth at $1184\times 1184$ resolution, reporting pose estimation AUC and inference times. The best result is in red, the second-best is in orange, and the third-best is in blue. The direction indicated by the arrows denotes superior performance.

Method	Pose Estimation AUC			Times (ms)
	AUC@5° ↑	AUC@10° ↑	AUC@20° ↑
Full	56.1	72.0	83.4	133.0
(1) Replace MobileOne with RepVGG	55.7	71.8	83.3	147.8
(2) Replace MobileOne with ResNet	55.4	71.7	83.2	162.4
(3) Replace TAIFormer to Agg. Attention	55.0	71.1	82.7	141.4
(4) Replace MinGRU to heatmap refine	55.6	71.9	83.4	133.4
(5) Remove MinGRU refinement	55.4	71.6	83.2	131.4
(6) Our Optimal (w/o dual softmax)	56.3	72.1	83.4	79.9

. LiteSAM comprises a reparameterizable MobileOne extractor, TAIFormer for feature propagation, and MinGRU-based subpixel refinement.

Replacing MobileOne with RepVGG slightly improves the hit rate but increases the RMSE, parameters, and latency, confirming MobileOne’s efficiency and generalization for edge devices. Substituting TAIFormer with EfficientLoFTR’s attention mechanism reduces accuracy and raises computational cost, highlighting TAIFormer’s effective global–local interaction. Omitting or modifying MinGRU results in marginal latency gains, underscoring its role in fine-grained matching, and precision is compromised. LiteSAM (opt.), without dual softmax, accelerates inferences by up to 39.9% while maintaining competitive accuracy, demonstrating a robust efficiency–accuracy trade-off.

4.9. Image Resolution

Experiments on the MegaDepth dataset using an NVIDIA RTX 3090Ti confirm LiteSAM’s balanced trade-off between accuracy and efficiency across resolutions. At 640 × 640, it delivers robust pose estimation, with AUC further improved at 1408 × 1408 alongside increased latency. LiteSAM opt achieves a similar AUC, with slightly reduced precision but significantly lower inference time, making it well-suited for latency-sensitive applications through efficient feature matching, as detailed in

Table 9. Impact of jmage resolution on localization performance on the MegaDepth dataset. AUC of pose estimation and inference time for LiteSAM (original) with dual softmax and LiteSAM (opt.) without dual softmax. The best result is in red, the second-best is in orange, and the third-best is in blue. The direction indicated by the arrows denotes superior performance.

Resolution	Dual Softmax	Pose Estimation AUC			Prec ↑	Num of Points	Times (ms)
		AUC@5° ↑	AUC@10° ↑	AUC@20° ↑
640 × 640	✓	50.7	67.5	79.9	97.8	1068.6	38.9
	-	50.7	67.5	79.9	96.6	1085.6	30.7
800 × 800	✓	53.2	69.9	81.9	98.0	1639.2	55.7
	-	53.5	69.8	81.7	96.7	1682.3	38.2
960 × 960	✓	55.0	71.1	82.9	98.2	2320.3	77.5
	-	55.3	71.3	82.9	96.7	2403.7	50.1
1184 × 1184	✓	56.1	72.0	83.4	98.0	3449.3	133.0
	-	56.3	72.1	83.4	96.1	3619.0	79.9
1408 × 1408	✓	56.3	72.4	83.9	98.0	4765.3	227.5
	-	56.8	72.6	83.9	95.5	5065.1	132.5

.

4.10. Performance on Edge Devices

We evaluated LiteSAM on the self-made dataset using an NVIDIA Jetson AGX Orin at 50 W with an input resolution of $1184\times 1184$ and AMP acceleration. As shown in

Table 10. Localization performance on the self-made dataset using NVIDIA Jetson AGX Orin. Results include RMSE within 30 m, average hit rate, number of correspondences, and inference time across difficulty levels. The best result is in red, the second-best is in orange, and the third-best is in blue. The direction indicated by the arrows denotes superior performance.

Method	Self-Made Dataset (Easy/Moderate/Hard)			Time (ms)
	RMSE@30 ↓	Avg.HR (%) ↑	Num of Point
EfficientLoFTR	7.53	90.76/78.63/59.24	2443/2011/1620	758.84
EfficientLoFTR (opt.)	8.31	88.87/75.67/53.44	4572/4011/3492	620.67
LiteSAM	6.17	92.69/89.08/77.21	2691/2113/1578	600.37
LiteSAM (opt.)	6.72	92.48/86.20/72.23	4388/3821/3220	497.49

, LiteSAM achieves the lowest RMSE@30 among full models while maintaining high hit rates and competitive latency. LiteSAM (opt.) achieves lower latency than other optimized models while maintaining strong accuracy and reliable matching across all difficulty levels. These results highlight LiteSAM’s suitability for real-time UAV localization on resource-constrained edge devices.

5. Conclusions

We propose LiteSAM, a lightweight feature matching framework for absolute visual localization (AVL) in UAV applications, designed to operate reliably on resource-constrained edge devices under spatiotemporal variations. LiteSAM integrates a reparameterizable multi-scale feature extractor to reduce inference latencies, a Token Aggregation–Interaction Transformer (TAIFormer) to efficiently model cross-view geometric deformations, and a learnable subpixel refinement module to enhance correspondence estimation and generalization.

Experiments on satellite–aerial benchmarks and the self-made dataset, as well as edge hardware, demonstrate that LiteSAM achieves competitive localization accuracies with reduced parameters and computational cost, enabling real-time UAV navigation in GNSS-denied environments. The implementation code is publicly available at: https://github.com/boyagesmile/LiteSAM, accessed on 20 September 2025.

Future research will focus on scaling datasets, leveraging self-supervised learning for cross-domain generalization, and integrating IMU data or temporal consistency to enhance long-term robustness.

6. Discussion

While LiteSAM demonstrates competitive performance and significant efficiency gains for satellite–aerial image matching, it exhibits certain limitations that warrant further exploration. Notably, LiteSAM relies on sufficient image overlap and consistent scales between satellite and aerial inputs, which may not always be guaranteed in real-world absolute visual localization (AVL) scenarios, such as those involving significant viewpoint changes or varying resolutions. This dependency could necessitate additional preprocessing steps, such as image retrieval or coarse alignment, to ensure robust matching. In future work, approaches such as the feature fusion wavelet attention-enhanced network [60] could be incorporated to adaptively enhance multi-scale discriminative features, thereby improving robustness and generalization across diverse AVL conditions, particularly under extreme scale variations or limited image overlap.