Lightweight Unsupervised Homography Estimation for Infrared and Visible Images Based on UAV Perspective Enabling Real-Time Processing in Space–Air–Ground Integrated Network

Highlights

What are the main findings?

• We propose a lightweight homography estimation method, LFHomo, for homography estimation of infrared and visible images in low-altitude scenarios.
• Experimental results show that the propose method significantly outperforms existing methods on infrared and visible datasets based on UAV perspective, with significant advantages in computational complexity and inference speed.

What is the implication of the main finding?

• LFHomo balances model accuracy and computational efficiency, demonstrating the potential of deep networks for low-altitude multimodal image registration tasks with excellent scalability.
• We constructed a novel unregistered UAV-based infrared and visible image dataset, which provides support for research on multimodal UAV remote sensing image registration and fusion.

Abstract

Homography estimation of infrared and visible light images is a key visual technique that enables drones to perceive their environment and perform autonomous localization in low-altitude environments. Its potential lies in integration with edge computing and 5G technologies, enabling real-time control of drones within air–ground integrated networks. However, research on homography estimation techniques for low-altitude dynamic viewpoints remains scarce. Additionally, images in low-altitude scenarios suffer from issues such as blurring and jitter, presenting new challenges for homography estimation tasks. To address these issues, this paper proposes a light-weight homography estimation method, LFHomo, comprising two components: two anti-blurring feature extractors with non-shared parameters and a lightweight homography estimator, LFHomoE. The anti-blurring feature extractors introduce in-verse residual layers and feature displacement modules to capture sufficient contextual information in blurred regions and to enable lossless and rapid propagation of feature information. In addition, a spatial-reduction-based channel shuffle and spatial joint attention module is designed to suppress redundant features introduced by lossless transmission, allowing efficient extraction and refinement of informative features at low computational cost. The homography estimator LFHomoE adopts a CNN–GNN hybrid architecture to efficiently model geometric relationships between cross-modal features and to achieve fast prediction of homography matrices. Meanwhile, we construct and annotate an unregistered infrared and visible image dataset from drone perspectives for model training and evaluation. Experimental results show that LFHomo maintains great registration accuracy while significantly reducing model size and inference time.

1. Introduction

In recent years, image perception systems have become increasingly important for unmanned aerial vehicles (UAV) in intelligent flight, emergency response, and autonomous cruising tasks [1,2]. Concurrently, the rapid development of 5G communication and edge computing technologies has enabled the deployment of real-time, efficient vision models directly on UAV platforms [3,4]. However, abrupt illumination changes and adverse weather conditions, such as rain and fog, significantly degrade the reliability of single-modal visual perception in complex environments. To address this issue, fusing multimodal information, such as infrared and visible spectra, and exploiting their complementary imaging properties has been shown to be an effective solution [5], achieving notable success in tasks including object tracking [6,7], semantic segmentation [8], anomaly detection [9,10], and depth estimation [11,12].

Due to differences in viewpoint and platform motion, images captured by different sensors often exhibit substantial geometric discrepancies. Furthermore, typical low-altitude scenes are affected by motion blur and low image resolution, thus rendering multimodal images difficult to use directly for information fusion. In this context, homography estimation between infrared and visible images serves as a fundamental visual task that models the geometric transformation between two images [13]. Thereby providing reliable preprocessing for higher-level tasks such as image registration [14,15], object detection [16], and localization or recognition [17]. Nonetheless, on UAV platforms, image quality degradation and constrained onboard computing resources impose stringent requirements on both the robustness and real-time performance of homography estimation methods.

Existing homography estimation approaches can be broadly divided into two categories: traditional methods and deep learning-based methods [5]. Traditional methods depend on feature descriptor operators [18,19] and utilize algorithms such as RANSAC [20] and MAGSAC++ [21] to remove outliers during matching, followed by solving the homography matrix using Direct Linear Transformations (DLT). These methods demonstrate efficacy in high-quality, single-source image scenarios, but tend to fail under multimodal conditions, low resolution, or strong noise. In contrast, deep learning-based methods regress the parameters required for homography estimation via a trainable end-to-end network [22,23,24,25,26], and then obtain the homography matrix using DLT. Such methods demonstrate greater robustness in multi-source image homography tasks. Conversely, this advantage is frequently accompanied by the disadvantage of larger model sizes and heavier computational costs, and their performance remains sensitive to factors such as noise and motion blur. Consequently, it remains difficult for these models to simultaneously satisfy both accuracy and real-time constraints on resource-limited low-altitude UAV platforms.

Recently, there has been increased focus on high-accuracy yet lightweight network designs, which hold considerable potential for scenarios with stringent real-time requirements. The MobileNet family [27,28,29] significantly reduces parameter counts and computational costs through depthwise separable convolutions and inverted residual structures. GhostNet [30] further demonstrates that equally rich feature representations can be obtained with fewer operations. Researchers have also begun to explore hybrid architectures that combine CNNs with Transformers or Graph Neural Networks (GNNs) to enhance feature representation while preserving compactness. Such as MobileViT [31], EfficientFormer [32], and MobileViG [33]. These works imply that appropriate architectural design can facilitate achieving a favorable balance between accuracy and efficiency.

Inspired by the above observations, this paper proposes a lightweight homography estimation method, LFHomo, for infrared–visible image homography estimation from a UAV perspective in low-altitude application scenarios. The model consists of two anti-blurring feature extractors with non-shared parameters and a lightweight homography estimator, LFHomoE. In the feature extractors, a Shift Module (SM) [34] is embedded into the inverted residual structure [28] to capture local contextual information and enable nearly lossless feature transmission. Furthermore, a Spatial-Reduction Channel-Sequential Shuffle Attention (SRCSSA) mechanism is designed to enhance the perception and integration of contextual information in blurred regions while suppressing redundant features at a low computational cost. LFHomoE draws on the lightweight design of MobileViG [33] and the graph attention mechanism of Vision GNN (ViG) [35] to build a CNN–GNN hybrid architecture, achieving a balance between performance and efficiency in homography matrix estimation. To alleviate the scarcity of unregistered UAV-based infrared and visible images datasets, we also construct a new dataset from a UAV perspective to support model training and validation.

Accordingly, this work formulates the following testable hypothesis: compared with representative multimodal homography estimation methods, LFHomo can reduce the average corner error (ACE) on the proposed UAV-based infrared and visible images dataset by approximately 5%, while maintaining lower model complexity and shorter inference time.

In summary, the main contributions of this paper are as follows:

• We introduce an Inverted Residual Shift Convolution (IRSC) block that embeds a shift module into an inverted residual structure to capture local contextual features in blurred low-altitude UAV images.
• We design a Spatial-Reduction Channel-Sequential Shuffle Attention (SRCSSA) module that suppresses redundant and enhances informative features through spatial reduction, channel grouping and shuffling, and attention-based fusion.
• We develop a lightweight CNN–GNN hybrid homography estimator, LFHomoE, which achieves a favorable trade-off between accuracy and efficiency and delivers fast inference on both synthetic benchmark datasets and unregistered infrared–visible UAV image pairs.

The structure of this paper is as follows: Section 2 briefly introduces existing related work. Section 3 details the proposed method. Section 4 describes the dataset construction process and presents experimental results on infrared and visible image datasets from the perspective of a drone. In Section 5, we discuss the limitations of our method and provide prospects for future work. Finally, we conclude the paper in Section 6.

2. Related Work

2.1. Homography Estimation for Multimodel Image

Homography estimation methods can be roughly divided into traditional approaches and deep learning–based approaches [13]. Traditional methods typically decompose homography estimation into three steps: feature descriptor extraction, descriptor matching, and homography matrix estimation. In practice, feature descriptors are first extracted using algorithms such as Scale Invariant Feature Transform (SIFT) [18], Speeded Up Robust Features (SURF) [36], Oriented FAST and Rotated BRIEF (ORB) [19], and BEBLID [37]. Then, matching algorithms are applied to pair descriptors, and incorrect correspondences are removed by robust estimators such as RANSAC [20] or MAGSAC++ [21]. Finally, the homography matrix is solved using the Direct Linear Transformation (DLT) algorithm. Some works also adopt trainable deep models to replace the local descriptor or matching steps in this pipeline, such as LIFT [38], SuperPoint [39], SuperGlue [40], and LoFTR [41]. Nevertheless, these methods generally require high-quality images and are easily affected by modality discrepancies, motion blur, noise, and low resolution.

In contrast, deep learning-based homography estimation methods convert the multi-step procedure of traditional approaches into a unified, trainable end-to-end framework [5] and have gradually become the mainstream direction for multi-source image homography estimation. We summarize representative multimodal homography methods and their characteristics in

Table 1. Deep learning-based methods for multimodal homography estimation. The table lists the core idea, advantages, and limitations of each method.

Method	Core Strategy	Advantages	Limitations
DADHN [22]	Designs a fine-grained feature extractor to support homography estimation.	Preserves channel and spatial information of multi-scale features, enhancing meaningful feature representations.	Limited shallow feature extraction capability; struggles in heavily blurred regions.
HomoMGAN [23]	Formulates the homography estimation process as a generative adversarial process.	Self-optimizes the homography matrix via GAN without explicitly optimizing images or attention maps.	Sensitive to illumination changes.
FCTrans [24]	Uses cross-image attention to explicitly model feature correlations.	Converts the multi-source homography problem into a single-source one.	Difficult to handle large-baseline image pairs.
LCTrans [25]	Adopts a coarse-to-fine strategy to iteratively refine the homography matrix.	Obtains multi-scale feature maps within a single network. Without additional matrix fusion.	Sensitive to noise.
homoViG [26]	First applies a graph attention mechanism to homography estimation.	Uses graph structures to mitigate modality discrepancies and strengthen feature matching.	Generalization to motion blur, adverse weather, and low-resolution UAV scenes remains unexplored.

. For example, Luo et al. [23] employ a Generative Adversarial Network (GAN) to estimate homographies. This approach, however, is difficult to generalize across varying illumination conditions. Transformer-based methods [24,25] also exhibit sensitivity to noise and blur, and their computational complexity remains high. Liao et al. [26] propose homoViG, which introduces a graph attention mechanism to alleviate the impact of modality differences. Nonetheless, its generalization ability in low-altitude UAV scenarios with motion blur, adverse weather, or low resolution has not been systematically validated.

Overall, these methods offer valuable insights into multimodal homography estimation. Yet they often require substantial computational resources and show limited robustness in complex real-world scenarios. In low-altitude UAV applications, lightweight and robust homography estimation methods for infrared and visible images are still lacking.

2.2. Lightweight Hybrid Backbones

With the increasing use of Transformers and GNNs across various vision tasks, the development of lightweight hybrid architectures that integrate CNNs with diverse network paradigms has emerged as a significant research direction. Mehta et al. [31] proposed MobileViT, a lightweight hybrid architecture combining CNNs and Transformers. It replaces part of the convolutional blocks in MobileNetV2 [28] with transformer blocks, enhancing global modeling capability while maintaining low computational cost. Yan et al. [32] introduced EfficientFormer, which achieves high inference speed on mobile devices through dimension-consistent and stage-wise design. Huang et al. [33] further proposed MobileViG, based on ViG [35], by constructing a CNN–GNN fusion framework and designing a Sparse Vision Graph Attention (SVAG) mechanism for message passing on local vision graphs, achieving a favorable efficiency–accuracy trade-off on image classification and object detection benchmarks.

Building on these developments, LFHomoE integrates the design principles of ViG [35] and MobileViG [33]. On one hand, ViG stacked a large number of ViG blocks [35] in the backbone. The frequent graph convolution and adjacency reconstruction introduce considerable computational overhead. In contrast, LFHomoE inserts only a few ViG blocks at the end of each stage. The remaining blocks are replaced by inverted residual structures, which significantly reduce the computational burden and bring the parameter size and FLOPs closer to those of a lightweight CNN backbone. On the other hand, MobileViG [33] introduces SVAG only at the lowest-resolution stage to control the cost. Nonetheless, it constructs the same static graph structure [42] for all images, making it difficult to adapt to geometric variations across different scenes and contents. LFHomoE replaces SVAG with ViG blocks and distributes them evenly across all stages, thereby strengthening multi-scale modeling of geometric relationships between multimodal features. As a result, the model can more flexibly describe local correspondences between the two modalities in different spatial regions and improve the accuracy of homography matrix estimation.

3. Method

3.1. Network Structure

LFHomo consists of two anti-blurring fast feature extractors with non-shared parameters and a lightweight homography estimator, LFHomoE. The overall network architecture is illustrated in Figure 1.

Given two input images, an infrared image $I_{inf}$ and a visible image $I_{rgb}$, both of size $H\times W\times 1$. First, employ the blur-resistant fast feature extractor to obtain two feature maps, $F_{inf}$ and $F_{rgb}$, of the same size $H\times W\times 1$. These feature maps are then concatenated along the channel dimension to form the fused feature map $F_c$, which is fed into LFHomoE to rapidly predict the homography matrix. The LFHomoE model is composed of an embedding layer, four sequentially stacked stages, and two fully connected layers. Each stage of the process comprises several inverted residual blocks [28] and one ViG block [35], collectively referred to as an LFHomoE Block. The embedding layer divides the fused feature map $F_c$ into patches and incorporates positional encoding. The patches are then processed through the four stages in sequence to establish the mapping between cross-modal features. The output of the final stage is passed via two fully connected layers and projected into an 8-dimensional offset vector. After that, employing the direct linear transformation algorithm, the offset vector is converted into a homography matrix $H_{inf\_r}$. During the training and testing phases, the order of the input image pairs is also reversed to generate another homography matrix $H_{rgb\_f}$. These two homography matrices are mutually inverse.

3.2. Anti-Blurring Feature Extractor

The anti-blurring feature extractor comprises four Inverse Residual Shift Convolution Blocks (IRSC) and a Spatial Reduction Channel Sequence Shuffling Attention (SRCSSA) module. Notably, the SRCSSA block contains a downsampling layer, a channel sequence shuffling attention block (CSSA) [43], a Global Spatial Attention block (GSA), and an upsampling layer. The layer structure and the output information of the feature extractor are displayed in

Table 2. The details of the anti-blurring feature extractor.

Layer	Type	Input Channels	Output Channels	Output Size
Layer-1	IRSC	1	8	$H\times W$
Layer-2	IRSC	8	16	$H\times W$
Layer-3	IRSC	16	32	$H\times W$
Layer-4	Downsampling	32	32	${\displaystyle \frac{H}{2}}\times {\displaystyle \frac{W}{2}}$
Layer-5	CSSA and GSA	32	32	${\displaystyle \frac{H}{2}}\times {\displaystyle \frac{W}{2}}$
Layer-6	Upsampling	32	16	$H\times W$
Layer-7	IRSC	16	1	$H\times W$

.

3.2.1. Inverse Residual Shift Convolution Block

The IRSC block contains an inverse residual convolutional block [28] and a shift module [34]. Its structure is shown in Figure 2a. Since the number of input and output channels per layer varies within the feature extractor, we did not incorporate residual connections within the IRSC. For each IRSC, adopting feature map $F$ as a paradigm, an inverse residual layer is first utilized to extract feature information. Specifically, the input feature map undergoes a $1\times 1$ pointwise convolution for dimensionality expansion. Subsequently, a $3\times 3$ depthwise separable convolution [27] is applied in the high-dimensional space to efficiently capture local spatial features. Finally, another $1\times 1$ pointwise convolution maps the number of channels back to the input dimension.

However, in drone imagery, blurred regions, particularly at edges, are still constrained by the limited receptive field of the depthwise convolution in the inverse residual block. This constraint easily leads to the loss of local contextual information in these regions, although such information often contains the true edge structure. To address this limitation, a shift module [34] is incorporated into the inverse residual block, enabling efficient transfer and supplementation of feature information.

The processing pipeline of the shift module is illustrated in Figure 3. Let the input feature map of the shift module be $F_{\mathrm{shift}}^{\mathrm{in}}\in {\mathbb{R}}^{H\times W\times C}$. First, it is evenly split along the channel dimension into four sub-tensors of equal size: $F_{shift}^{in}=\left[F_1, F_2, F_3, F_4\right], F_n\in {\mathbb{R}}^{H\times W\times {\displaystyle \frac{C}{4}}}$. Each sub-tensor is then circularly shifted by one pixel in the positive or negative direction along the x-axis or y-axis. We define the circular shift function in Equation (1):

$$F_n^{\prime }={CS}_{(\Delta x,\Delta y)}\left(F_n\right)$$

(1)

where ${\mathrm{C}\mathrm{S}}_{(\Delta x,\Delta y)}(\cdot )$ denotes the circular shift operation and $(\Delta x,\Delta y)$ specifies the shift along the x-axis and y-axis, respectively. After the four sub-tensors are shifted, they are concatenated in the original order to restore the feature map shape. The overall computation of the shift module can thus be written as Equation (2):

$$F_{shift}^{out}=Shift\left(F_{shift}^{in}\right)=Concat({CS}_{\left(+1,0\right)}\left(F_1\right), {CS}_{\left(-1,0\right)}\left(F_2\right), {CS}_{\left(0,+1\right)}\left(F_3\right), {CS}_{(0,-1)}(F_4\left)\right)$$

(2)

where $Shift(\cdot )$ denotes the shift module, $Concat(\cdot )$ is the concatenation operation. $F_{shift}^{out}$ is the output feature map of the shift module. Based on the above, the computation of the IRSC block can be expressed as Equation (3):

$$F_{IRSC}^{out}=Shift\left({PWConv}_{1\times 1}\right({DWConv}_{3\times 3}\left({PWConv}_{1\times 1}\right(F_{IRSC}^{in}\left)\right)\left)\right)$$

(3)

where $F_{IRSC}^{out}$ and $F_{IRSC}^{in}$ denote the output feature map and input feature map of each IRSC block, respectively. The ${PWConv}_{1\times 1}(\cdot )$ is a $1\times 1$ pointwise convolution, while the ${DWConv}_{3\times 3}(\cdot )$ is indicated $3\times 3$ depthwise separable convolution. In this way, the IRSC block can transfer feature information to the subsequent modules with minimal loss, while incorporating the shifted contextual information produced by the shift module.

The combination of the inverted residual block [28] and the shift module [34] exhibit a clear complementarity. The inverse residual block extracts local edge features by per-channel depth convolutions, while the shift module explicitly propagates these features at the spatial dimension. This integration facilitates the perception of motion-blurred boundaries. In other words, IRSC strengthens the complementarity and fusion among key features in blurred areas without introducing a significant increase in parameters or computational cost.

3.2.2. Spatial Reduction-Based Channel Sequence Shuffle Attention

The Spatial-Reduction Channel-Sequential Shuffle Attention suppresses interfering features in blurred regions at a low computational cost, and to better capture informative structures, the architecture is illustrated in Figure 4. To reduce the computational complexity of the attention mechanism and improve inference efficiency, the spatial size of the input feature map is first reduced from $H\times W$ to $R\cdot H\times R\cdot W$ by a downsampling operation, where $R$ denotes the spatial reduction ratio. The downsampled feature map is then divided into $G$ groups along the channel dimension, each group containing ${\displaystyle \frac{\mathrm{C}}{\mathrm{G}}}$ channels. Spatial average pooling is applied to each group to produce a descriptor of size $1\times 1\times {\displaystyle \frac{\mathrm{C}}{\mathrm{G}}}$. In our experiments, we set the number of groups to $G=4$ and the spatial reduction ratio to $R={\displaystyle \frac{1}{2}}$. This process can be expressed as Equation (4):

$$F=AvgPool\left(G\right(Down\left(F_{SRCSSA}^{in}\right)\left)\right)$$

(4)

where $F_{SRCSSA}^{in}$ denotes the input feature map of SRCSSA, $F$ represents the intermediate feature vector after grouping and pooling. $Down(\cdot )$ denotes the downsampling operation, $G(\cdot )$ is the grouping operation, and $AvgPool(\cdot )$ denotes average pooling. Then, a channel-sequence shuffling operation is used to mix channel features across different groups, as shown in Equation (5):

$$\overline{F}=f_{shuff}\left(F\right)=\left[X_1^1, X_2^1, \dots , X_G^1, X_1^2, \dots , X_G^2, X_1^3, \dots , X_G^3, \dots , X_1^{{\displaystyle \frac{C}{G}}}, \dots , X_G^{{\displaystyle \frac{C}{G}}}\right]$$

(5)

where $f_{shuff}\left(\cdot \right)$ denotes the sequence shuffling operation; $X_j^i$ denotes the pooled features of the i-th channel within the j-th group, where $j=\left\{1,2,\dots ,G\right\}$ and $i=\{1,2,\dots ,{\displaystyle \frac{C}{G}}\}$.

After shuffling, a group convolution with group size $G=4$ is applied to obtain channel-wise attention weights. A second shuffling operation is then used to restore the original channel order [43]. This procedure corresponds to the CSSA part in Figure 4 and can be written as Equation (6):

$${\overline{F}}_{CSSA}=f_{shuff}^T\left(Concat\right(GConv\left(f_{shuff}\right(F\left)\right)\left)\right)$$

(6)

where $GConv(\cdot )$ denotes group convolution, ${\overline{F}}_{CSSA}$ is the channel attention weight map. Finally, the attention weights are applied to the downsampled feature map to obtain the channel-weighted feature map ${\overline{F}}_{CSSA}^{out}$, as in Equation (7):

$${\overline{F}}_{CSSA}^{out}={\overline{F}}_{CSSA}⨀Down\left(F_{SRCSSA}^{in}\right)$$

(7)

where $⨀$ denotes element-wise multiplication. In addition, we use a Global Spatial Attention (GSA) branch to model long-range dependencies within the feature map, thereby enlarging the receptive field and enhancing global representation capability. The GSA is computed as Equation (8):

$${\overline{F}}_{GSA}=f_{GSA}\left({\overline{F}}_{CSSA}^{out}\right)=\sigma \left(Conv\right(Concat\left(MaxPool\right({\overline{F}}_{CSSA}^{out}), AvgPool({\overline{F}}_{CSSA}^{out}\left)\right)\left)\right)$$

(8)

where $MaxPool(\cdot )$ and $AvgPool(\cdot )$ represented max pooling and average pooling, respectively. $Conv(\cdot )$ is a convolution layer, and $\sigma (\cdot )$ signifies the sigmoid function. The final SRCSSA output is obtained by weighting the channel-attended feature map with the spatial attention map, as in Equation (9):

$$F_{SRCSSA}^{out}={\overline{F}}_{GSA}⨀{\overline{F}}_{CSSA}^{out}$$

(9)

where $F_{SRCSSA}^{out}$ denotes the weighted feature map produced by SRCSSA. Finally, $F_{SRCSSA}^{out}$ is upsampled back to the original resolution to match the input size required by the subsequent loss computation.

It is worth noting that, thanks to the introduction of the shift module [34], the feature maps learn richer local structural information across different channel groups. However, this cross-channel propagation may also introduce redundancy, such as background responses and image noise. SRCSSA enhances inter-channel interaction through channel shuffling combined with attention, thereby emphasizing complementary features from different directions while suppressing non-critical features.

3.3. Lightweight and Fast Homography Estimator

In this section, we develop a lightweight homography estimator, LFHomoE, which combines CNN and GNN structures to balance model accuracy, inference speed, and computational cost. LFHomoE takes the feature map $F_c$ as input and consists of an Embedding layer, four sequential LFHomoE blocks, and two fully connected layers. Each LFHomoE block is treated as one stage. The Embedding layer first performs patch embedding and positional encoding on $F_c$. Then, the four LFHomoE blocks are applied in order for feature modeling and matching. The configuration of each layer of information for LFHomoE is detailed in

Table 3. The structure of each stage of LFHomoE. Among them, Conv represents the convolutional layer, IRBlock represents the inverse residual block.

Stage	Type	Input Channels	Output Channels	Input Size	Output Size	Para (M)	FLOPs
Embedding	Conv × 3	2	42	$H\times W$	${\displaystyle \frac{H}{4}}\times {\displaystyle \frac{W}{4}}$	0.008	$1.94\times {10}^7$
Stage 1	$\left[\begin{array}{c}IRBlock\times 2\\ ViG Block\times 1\end{array}\right]$	42	42	${\displaystyle \frac{H}{4}}\times {\displaystyle \frac{W}{4}}$	${\displaystyle \frac{H}{4}}\times {\displaystyle \frac{W}{4}}$	0.047	$9.65\times {10}^7$
Downsampling	Conv	42	84	${\displaystyle \frac{H}{4}}\times {\displaystyle \frac{W}{4}}$	${\displaystyle \frac{H}{8}}\times {\displaystyle \frac{W}{8}}$	0.032	$1.63\times {10}^7$
Stage 2	$\left[\begin{array}{c}IRBlock\times 2\\ ViG Block\times 1\end{array}\right]$	84	84	${\displaystyle \frac{H}{8}}\times {\displaystyle \frac{W}{8}}$	${\displaystyle \frac{H}{8}}\times {\displaystyle \frac{W}{8}}$	0.182	$9.34\times {10}^7$
Downsampling	Conv	84	168	${\displaystyle \frac{H}{8}}\times {\displaystyle \frac{W}{8}}$	${\displaystyle \frac{H}{16}}\times {\displaystyle \frac{W}{16}}$	0.127	$1.62\times {10}^7$
Stage 3	$\left[\begin{array}{c}IRBlock\times 5\\ ViG Block\times 1\end{array}\right]$	168	168	${\displaystyle \frac{H}{16}}\times {\displaystyle \frac{W}{16}}$	${\displaystyle \frac{H}{16}}\times {\displaystyle \frac{W}{16}}$	1.413	$1.81\times {10}^8$
Downsampling	Conv	168	192	${\displaystyle \frac{H}{16}}\times {\displaystyle \frac{W}{16}}$	${\displaystyle \frac{H}{32}}\times {\displaystyle \frac{W}{32}}$	0.291	$9.29\times {10}^6$
Stage 4	$\left[\begin{array}{c}IRBlock\times 2\\ ViG Block\times 1\end{array}\right]$	192	192	${\displaystyle \frac{H}{32}}\times {\displaystyle \frac{W}{32}}$	${\displaystyle \frac{H}{32}}\times {\displaystyle \frac{W}{32}}$	0.935	$2.99\times {10}^7$
FC	Pooling & MLP	192	8	${\displaystyle \frac{H}{32}}\times {\displaystyle \frac{W}{32}}$	$1\times 1$	0.001	$3.01\times {10}^3$

.

The main difference between the inverted residual block in LFHomoE and the IRSC block lies in the presence of residual connections and the absence of the shift module. The structure of the inverted residual block in LFHomoE is shown in Figure 2b. Compared with standard convolutions, inverted residual blocks achieve deep nonlinear feature transformations with fewer parameters and lower computational cost, while providing skip connections in low-dimensional space to alleviate the vanishing-gradient problem. In both IRSC and LFHomoE, the channel expansion factor of the inverted residual block is set to 4. The shift module is not introduced into LFHomoE because the anti-blurring fast feature extractors already provide sufficiently accurate local contextual features. In addition, LFHomoE contains a relatively large number of inverted residual blocks. Adding shift modules to these blocks would introduce extra computation and degrade the overall inference speed.

In LFHomoE, each ViG block [39] consists of a maximum relative graph convolution [44] and a feed-forward network (FFN) formed by two fully connected layers. Initially, the input feature map is projected by a linear layer into a unified graph feature space, where each spatial position is regarded as a graph node. The maximum relative graph convolution then aggregates information from neighboring nodes to model their dependencies, providing cues for subsequent feature matching. The FFN applies nonlinear transformations to enhance the representation capacity and mitigate potential over-smoothing. To maintain topological structure consistency between the input and output, the ViG block reshapes features into sequences for graph operations and then restores them to their original layout.

As summarized in Table 3, LFHomoE adopts a top–down pyramid-style design. This differs from the RFViG backbone used in homoViG [26], where a large number of ViG blocks with Attention Feature Fusion (AFF) modules [45] are heavily stacked in each stage. In LFHomoE, most of these graph-based components are replaced with inverted residual convolution blocks, which preserve the ability to encode cross-modal geometric constraints while substantially reducing the overhead of repeated graph convolutions and reshape operations. The pyramid design also leads to a more balanced allocation of computation across stages. In the first two stages, the network processes feature maps with relatively high spatial resolution but fewer channels, focusing on extracting fine-grained geometric cues at larger spatial scales. In the last two stages, the spatial resolution is progressively reduced while the number of channels increases, enabling high-dimensional semantic and structural relationships to be modeled on more compact feature maps. This design effectively controls the growth of parameters and FLOPs. Quantitative results in Section 4.4 further confirm that the proposed method achieves lower parameter counts, peak memory usage, and inference time than existing counterparts, while maintaining competitive homography estimation performance.

Finally, we input the output of the last stage into two fully connected layers to obtain an 8-dimensional coordinate offset vector and use the DLT algorithm to convert it into the homography matrix of the corresponding image pair.

3.4. Loss Function

The loss function comprises three parts: Detail Feature Loss (DFL) [22], Feature Identity Loss (FIL) [46], and homography loss. The total loss $L_{total}$ is shown in Equation (10):

$$\begin{gathered} L_{total}=L_{DFL}(I_{inf}^{\prime }, I_{rgb})+L_{DFL}(I_{rgb}^{\prime }, I_{inf})+\lambda \left(L_{FIL}\right(I_{inf}, H_{inf\_r}) \\ +L_{FIL}(I_{rgb}, H_{rgb\_i}\left)\right)+\mu L_h \end{gathered}$$

(10)

where $L_{DFL}$ represents the loss of details feature between the distorted image and the target image, $L_{FIL}$ is the feature identity loss. $H_{inf\_r}$ denotes the homography matrix predicted by the model that maps image $I_{inf}$ to $I_{rgb}$. Contemporaneously, $H_{rgb\_i}$ is the homography matrix predicted after swapping the order of the input image pairs $I_{inf}$ and $I_{rgb}$. $L_h$ denotes the homography loss, which is employed to impose reversibility constraints. The hyperparameters $\lambda$ and $\mu$ are set to 1.0 and 0.01, respectively. We compared the effects of hyperparameters $\lambda$ and $\mu$ under different settings in Section 4.6.3.

3.4.1. Detail Feature Loss

Detail Feature Loss (DFL) proposed by Luo et al. [22], which calculates a triple loss on the feature map $F_{inf}$ and the warped feature map $F_{inf}^{\prime }$ obtained by the homography matrix. The original triple loss [47] computed on the input image $I_{inf}$ and the distorted image $I_{inf}^{\prime }$, may introduce irrelevant background information and noise during computation, thereby affecting network optimization. Conducting the calculation on feature maps enables the network to more effectively identify significant features for homography estimation, minimizing the impact of extraneous noise present in the original image. The formulation is given in Equation (11):

$$L_{DFL}\left(F_{inf}, F_{inf}^{\prime }, F_{rgb}\right)={\left|\right|F_{inf}^{\prime }-F_{rgb}\left|\right|}_1-{\left|\right|F_{inf}-F_{rgb}\left|\right|}_1$$

(11)

where $F_{inf}^{\prime }=f_{STN}\left(F_{inf}\right)$ is the result of the feature map $F_{inf}$ being distorted by the homography matrix $H_{inf\_r}$. $f_{STN}(·)$ represents the distortion operation using the Spatial Transform Network (STN) [48]. During the training process, as the input order of the images was reversed, the total loss function induced another feature detail loss $L_{DFL}\left(F_{rgb}, F_{rgb}^{\prime }, F_{inf}\right)$ between $I_{rgb}^{\prime }$ and $I_{inf}$.

3.4.2. Feature Identity Loss

However, during training, DFL does not necessarily minimize ${\left|\right|F_{inf}^{\prime }-F_{rgb}\left|\right|}_1$ while maximizing ${\left|\right|F_{inf}-F_{rgb}\left|\right|}_1$, which influences network optimization. The introduction of the Feature Identity Loss (FIL) serves to address this limitation [46]. FIL permits the network to disregard the order between image distortion and feature extraction during optimization. In other words, whether it extracts the feature map before applying distortion, or warps the original image first and then extracts the feature map, the resulting feature maps should be substantially similar. This facilitates more effective minimization of the DFL [24]. The computation of FIL is illustrated in Equation (12):

$$L_{FIL}\left(I_{inf}, H_{inf\_r}\right)={\left|\right|f_{STN}(f_{IR}\left(I_{inf}\right),H_{\mathrm{i}\mathrm{n}\mathrm{f}\_r})-f_{IR}\left(f_{STN}\right(I_{inf},H_{inf\_r}\left)\right)\left|\right|}_1$$

(12)

where $W(\cdot )$ is used to denote the operation of distortion. For the same reason, we also calculated another feature identity loss $L_{FIL}(I_{rgb}, H_{rgb\_i})$. Finally, the homography loss aims to add reversibility constraints to $H_{\mathrm{i}\mathrm{n}\mathrm{f}\_r}$ and $H_{rgb\_i}$ to ensure that they are reversible. As demonstrated in Equation (13):

$$L_h={\left|\right|H_{inf\_r}{H}_{rgb\_i}-E\left|\right|}_2^2$$

(13)

where $E$ is represents the third-order unit matrix.

4. Experimental Results

In this section, we first introduce the datasets used in our experiments and the implementation details. To evaluate the effectiveness of the proposed method in low-altitude application scenarios, we construct and relabel two infrared and visible datasets captured from a UAV perspective. The evaluation metrics employed in the experiments are then briefly described. Subsequently, we conduct qualitative and quantitative experiments on these two datasets and compare the proposed method with existing unsupervised end-to-end approaches for infrared and visible images homography estimation. It is worth noting that, although methods such as SuperGlue [40] and LoFTR [41] perform well in the feature-matching stage of traditional methods, they rely on local feature priors and are essentially used as matching modules for subsequent homography estimation. As a result, it is difficult to compare these methods with our proposed framework under strictly identical conditions. Finally, ablation studies are carried out to verify the effectiveness of each proposed component.

4.1. Dataset and Implementation Details

4.1.1. Dataset

Infrared and Visible Image Dataset Based on UAV Perspective. Currently, there is a lack of unregistered infrared and visible light image datasets from UAV perspectives that can be used for training. To address this gap, this paper constructs a new unregistered benchmark dataset for infrared and visible images from a UAV perspective, based on three real multimodal UAV datasets: VEDAI [49], DroneRGBT [50], and DroneVehicle [51], and re-annotates it. The comparison between the original datasets and the constructed UAV perspective infrared and visible image dataset is shown in

Table 4. Comparison of the original dataset and the infrared and visible images dataset based on UAV Perspective.

Type	DataSet	Original Dataset	Original		Unregistered Image
			Number	Resolution	Number	Resolution
Training Set	NIUHBD	VEDAI	1246	$512\times 512$	452	$150\times 150$
	UHBD	DroneRGBT	1807	$640\times 512$	293	$150\times 150$
		DroneVehicle	17,990	$840\times 712$	3705	$150\times 150$
Test Set	NIUHBD	VEDAI	1246	$512\times 512$	24	$150\times 150$
	UHBD	DroneRGBT	1800	$640\times 512$	6	$150\times 150$
		DroneVehicle	8980	$840\times 712$	23	$150\times 150$

.

Specifically, to construct datasets under identical conditions, we first uniformly and randomly crop all images to $320\times 240$. Subsequently, we randomly crop $150\times 150$ image patches from the RGB images and apply random offsets within a certain range to the four corner points of the cropped regions. This process yields the homography matrix $H$ and its inverse matrix. The infrared images are then warped using the inverse matrix, and a 150 × 150 image patch is cropped from the warped image at the corresponding position. The resulting image pairs, each of size $150\times 150$, are used as training or testing samples [22]. Considering that the infrared images in VEDAI [49] lie in the near-infrared band, we construct a separate Near-Infrared UAV Homography Benchmark Dataset (NIUHBD), comprising 452 training and 24 test pairs. The datasets generated from DroneRGBT [50] and DroneVehicle [51] are merged into a unified UAV Homography Benchmark Dataset (UHBD), comprising 3998 and 29 test pairs. The training–testing split for both datasets is determined based on the total number of samples and scene distributions, aiming to ensure sufficient diversity in the training set and independence in the test set. In practice, the splits are manually defined so that training and testing samples do not overlap, thereby improving the reliability of the evaluation. Since these datasets are mainly used to assess homography estimation performance, no category labels are involved. The process of generating the dataset is presented in Figure 5.

The newly established benchmark dataset is divided into three distinct categories: the visible images image set $I_{RGB}$, the infrared image set $I_{INF}$, and the ground truth infrared image set $I_{GT}$. Among these, $I_{RGB}$ and $I_{INF}$ are unaligned, while $I_{RGB}$ and $I_{GT}$ are aligned. The $I_{GT}$ only exists in the test set for the purpose of qualitative comparison.

Synthetic Benchmark Dataset. We adopt the same synthetic benchmark dataset as [22] to evaluate the proposed method. The dataset is constructed from several public datasets, including the OSU Color-Thermal Database, INO, and TNO, and covers a wide range of indoor and outdoor scenes with structured objects such as pedestrians, vehicles, buildings, and natural landscapes. Following the data partitioning protocol in [22], the synthetic benchmark consists of unregistered infrared–visible image pairs of size $150\times 150$, with 49,378 pairs for training, 29 pairs for validation, and 42 pairs for testing. For the synthetic benchmark dataset’s test set, each image pair provides the ground-truth infrared image $I_{GT}$ and four sets of corresponding ground-truth corner coordinates. The ground truth infrared image, $I_{GT}$, is utilized for qualitative comparisons, aiming to visually demonstrate the channel blending results between the predicted image and the true image. The corner coordinates are deployed for quantitative comparisons, enabling the computation of relevant evaluation metrics.

Real-world Dataset. To further evaluate the generalization ability of the proposed method, we adopt the CVC Multimodal Stereo Dataset [52] as a real-world benchmark. The CVC dataset primarily contains long-wave infrared and visible images captured under various environments, with a resolution of 506 × 408. Since accurate ground-truth corner coordinates are not provided in CVC, we manually annotate four matched corner points for each test image pair in order to compute the evaluation metrics.

4.1.2. Experimentation Details

The proposed method was implemented using the PyTorch framework (version number: 1.10.0) and trained on an NVIDIA GeForce RTX 3090 (NVIDIA, Santa Clara, CA, USA). The training was performed with the Adam optimizer with an initial learning rate of $1\times {10}^{-5}$ and a decay factor of 0.8 per epoch. The batch size of the experiment was 64, and the number of epochs was 50. In addition, to demonstrate the effectiveness of the proposed method, we compared its performance with that of the latest method under the same environment on various datasets.

4.2. Evaluation Metric

In the experiments, we employ a set of metrics to evaluate the performance of the proposed method, including Average Corner Error (ACE), Structural Similarity (SSIM), Adaptive Feature Registration Rate (AFRR) [22], average inference time, number of parameters, and peak memory usage.

4.2.1. Average Corner Error

ACE evaluates the accuracy of homography estimation by measuring the average $l_2$ distance between the four corner points obtained by the predicted homography and those obtained by the ground-truth homography. The definition is given in Equation (14). A lower ACE indicates a more accurate estimated homography matrix.

$$ACE={\displaystyle \frac{{\sum }_{i=1}^4{\left|\right|x_i-y_i\left|\right|}_2}{4}}$$

(14)

where $x_i$ represents the estimated corner point after the homography matrix transformation. $y_i$ signifies the actual corner point after the homography matrix transformation.

4.2.2. Structural Similarity

SSIM is used to measure the similarity between the warped source image and the target image after the homography transformation. A higher SSIM value indicates greater similarity and thus a more accurate homography. SSIM is defined in Equation (15):

$$SSIM\left(x,y\right)={\displaystyle \frac{\left(2m_x{m}_y+\alpha \right)\left(2C_{xy}+\beta \right)}{\left(m_x^2+m_y^2+\alpha \right)\left(s_x^2+s_y^2+\beta \right)}}$$

(15)

where $x$ and $y$ denote the warped image and the target image, respectively. $m_x$ and $m_y$ are the average values of all pixels of images. $s_x$ and $s_y$ are their standard deviations. $C_{xy}$ is the covariance between $x$ and $y$. $\alpha$ and $\beta$ represent constants used to maintain stability.

4.2.3. Adaptive Feature Registration Rate

AFRR [22] reflects homography accuracy by computing the number of correctly registered feature point pairs within an adaptive range. For each matched feature pair, the Euclidean distance ${\mathcal{D}}_i$ between the two points is first computed. A threshold $\chi$ is used to discard obvious outliers; matches with ${\mathcal{D}}_i<\chi$ are retained, and their distances are denoted as ${\mathcal{D}}_i^{\prime }$. A second threshold, $\delta$, is then applied to determine whether a retained match is accurate. The AFRR is defined in Equation (16):

$$AFRR={\displaystyle \frac{1}{N}}\sum _{i=1}^N\delta \left({\mathcal{D}}_i^{\prime }\right)$$

(16)

where $N$ is the number of matches that satisfy ${\mathcal{D}}_i<\chi$, and $\delta \left({\mathcal{D}}_i^{\prime }\right)$ represents the number of points that are judged to be accurate at the threshold $\delta$ (i.e., the match is regarded as accurate when ${\mathcal{D}}_i^{\prime }<\delta$).

4.2.4. Average Inference Time, Parameters, and Peak Memory Usage

To verify the effectiveness of the proposed method in terms of lightweight design and low computational cost, we also introduce the average inference time, number of parameters, and peak memory usage. In general, lower values of these metrics indicate a better trade-off between the method’s computational complexity and inference speed.

4.3. Qualitative Comparison

In this section, a qualitative comparison of the proposed method and existing approaches is conducted across three datasets. Firstly, our method is compared with feature-based homography estimation methods. These feature-based methods comprise three feature descriptors (SIFT [18], ORB [19], BRISK [53]) and two outlier removal algorithms (RANSAC [20] and MAGSAC++ [21]). The experimental results are shown in Figure 6. The images contained within rows 1 and 5 come from the synthetic benchmark dataset. Rows 2, 3, 6, and 7 feature images from the infrared and visible images dataset based on UAV perspective, UHBD and NIUHBD, constructed in this paper. The results demonstrate that feature-based methods display significant image distortion and ghosting, as well as match failures, across all datasets: the synthetic benchmark, UHBD, and NIUHBD. It may be posited that the significant modal differences between infrared and visible images likely make feature-based methods susceptible to errors during feature matching. Conversely, low-altitude UAV images often exhibit shake-induced blur, which hinders the extraction of sufficiently accurate features for matching using conventional methods. In contrast, the proposed method demonstrates more stable and accurate homography estimation on all three datasets.

Secondly, we compare the proposed method with existing deep learning-based homography estimation approaches, including DADHN [22], HomoMGAN [23], FCTrans [24], LCTrans [25], and homoViG [26]. The results are illustrated in Figure 7. To evaluate alignment accuracy, we blended the G and B channels of the predicted infrared image with the R channel of the real infrared image, and then visualized the degree of ghosting. The experimental results indicate that, on the synthetic benchmark dataset, where motion blur is minimal, the proposed method exhibits limited advantages in ghosting reduction. However, on the dataset based on the drone’s perspective, LFHomo demonstrated fewer ghosting artifacts and superior edge alignment performance under low-resolution and jitter-blurred conditions. This advantage primarily stems from the anti-blurring feature extractor, where the IRSC and SRCSSA blocks effectively compensate for feature information loss caused by motion blur or low resolution. This results in higher precision and robustness in UHBD and NIUHBD.

Analysis of Non-Advantageous Cases on UHBD

It is important to note that the visual difference between LFHomo and homoViG on the UHBD is indeed quite subtle. Nevertheless, it can be observed that, across all examples, after RGB channel blending, interpolation, and field-of-view cropping induced by the homography transformation, red artifacts often appear along the image boundaries. And in the UHBD example shown in Figure 7, when there is obvious red ghosting along the right edge, the visualization indicates a relatively large alignment error. As the red ghosting on the right edge weakens or disappears, the alignment quality improves. To explain this phenomenon, Figure 8 further illustrates the RGB fusion of the pre-warp infrared image and the ground-truth infrared image. The ghosting in the G and B channels appears mainly to the left of the red ghosting, suggesting that the true homography shifts the image to the right in this case. The homoViG result shows almost no red artifacts at either edge, whereas LFHomo retains slight red artifacts on the left edge. But both preserve a consistent spatial relationship between the G and B channels and the R-channel ghosting. This suggests that, although both methods are difficult to distinguish in visual images, LFHomo may provide finer geometric alignment that is not immediately evident from a single qualitative visualization. The results in Quantitative Comparison also confirm this.

4.4. Quantitative Comparison

In this section, we undertake a quantitative comparison of the proposed method with both feature-based and deep learning-based methods. Qualitative comparisons reveal that feature-based methods struggle to meet the accuracy requirements of low-altitude applications. To more precisely illustrate the accuracy gap between the proposed method and all other methods, we compare their corner error metrics on a synthetic benchmark dataset. Moreover, the results were categorized into three difficulty levels based on corner error: Easy (top 0–30%), Medium (top 30–60%), and Hard (top 60–100%) [42]. The detailed comparison results are shown in

Table 5. Comparison of corner errors between the proposed algorithm and all other methods on the synthetic benchmark dataset.

(1)	Method	Easy	Moderate	Hard	Average	Failure Rate
(2)	I3×3	4.59	5.71	6.77	5.79	0%
(3)	SIFT + RANSAC	50.87	-	-	50.87	93%
(4)	SIFT + MAGSAC++	131.72	-	-	131.72	93%
(5)	ORB + RANSAC	82.64	118.29	313.74	160.89	17%
(6)	ORB + MAGSAC++	85.99	109.14	142.54	109.13	19%
(7)	BRISK + RANSAC	104.06	126.8	244.01	143.2	24%
(8)	BRISK + MAGSAC++	101.37	136.01	234.14	143.4	24%
(9)	DADHN	3.84	5.01	6.09	5.08	0%
(10)	HomoMGAN	3.85	4.99	6.05	5.06	0%
(11)	FCTrans	3.75	4.70	5.94	4.91	0%
(12)	LCTrans	3.66	4.65	5.77	4.80	0%
(13)	homoViG	3.72	4.50	5.55	4.68	0%
(14)	LFHomo (Ours)	3.65	4.59	5.68	4.73	0%

The black bold number indicates the best result.

. Within this context, the term “-” signifies that the method was unsuccessful on multiple occasions during the testing phase. This led to the absence of data for that particular level. The entry “I_3×3” in row 2 represents the identity transformation, where the calculated corner error denotes the original distance between point pairs. Rows 2 to 8 represent the corner errors of the feature-based method at three levels, and rows 9 to 14 donate the results of the deep learning-based method at each level. The final two columns of Table 5 present the mean corner error and failure rate, respectively.

As evidenced in Table 4, all feature-based methods experienced algorithmic failures, and their corner errors were higher at all levels compared to those of deep learning-based methods. This indicates that the performance of feature-based homography estimation methods in multimodal image scenarios falls short of practical requirements, and we believe that this testing outcome can also be generalized to UAV-based infrared and visible image datasets. In contrast, deep learning-based methods demonstrate superior performance and robustness across all levels of analysis. The method proposed in this paper achieves the best result only at the first level, yet its final average corner error reaches 4.73, second only to 4.68 from homoViG [26].

To further evaluate the performance of the proposed method, we conduct quantitative comparisons between LFHomo and five deep learning-based approaches on three datasets, as reported in

Table 6. Quantitative comparison results of each deep learning-based methods on different datasets. The black bold number indicates the best result.

Dataset	Method	ACE	SSIM	AFRR
Synthetic Benchmark	DADHN [22]	5.08	0.97	0.74
	HomoMGAN [23]	5.06	0.97	0.72
	FCTrans [24]	4.91	0.97	0.72
	LCTrans [25]	4.80	0.97	0.71
	homoViG [26]	4.68	0.96	0.70
	LFHomo(Ours)	4.73	0.97	0.72
UHBD	DADHN [22]	7.03	0.95	0.87
	HomoMGAN [23]	6.69	0.95	0.85
	FCTrans [24]	6.59	0.96	0.88
	LCTrans [25]	6.41	0.95	0.87
	homoViG [26]	6.30	0.94	0.82
	LFHomo (Ours)	6.14	0.95	0.88
NIUHBD	DADHN [22]	6.59	0.94	0.87
	HomoMGAN [23]	6.54	0.94	0.88
	FCTrans [24]	6.39	0.95	0.87
	LCTrans [25]	6.36	0.92	0.85
	homoViG [26]	6.32	0.94	0.86
	LFHomo (Ours)	6.12	0.94	0.87

. It can be observed that, although LFHomo attains a slightly lower ACE than homoViG [26] on the synthetic benchmark dataset, it achieves the best ACE on both UAV datasets. Compared with the second-best method, its ACE is reduced by approximately 4–6% on average. For SSIM and AFRR, the differences are negligible for all methods. We speculate that this is because image warping introduces invalid pixel regions near the boundaries, which, in turn, affect the calculation of these two metrics. Therefore, in the subsequent experiments, ACE is adopted as the primary evaluation metric.

Meanwhile,

Table 7. Average inference time, parameters, and peak memory usage for each method. The black bold number denotes the best result.

Method	Time (s)	Parameter (M)	Memory (M)
DADHN [42]	0.36	85.23	85.32
HomoMGAN [32]	0.31	20.44	20.44
FCTrans [33]	0.36	20.12	20.12
LCTrans [16]	0.30	21.99	21.99
homoViG [17]	0.17	5.243	21.27
LFHomo (Ours)	0.09	3.233	19.35

compares the average inference time, number of parameters, and peak GPU memory usage during inference for the different methods. All experiments on the three datasets are conducted under a unified software–hardware configuration, with identical input and output sizes. Therefore, for a given model, the reported peak GPU memory usage is similar across the three datasets. As shown in Table 7, LFHomo achieves the lowest inference time, parameter count, and peak GPU memory usage among all competitors. This indicates that the method significantly improves the trade-off between computational efficiency and model complexity while maintaining competitive accuracy. Overall, these results are consistent with the hypothesis formulated in the introduction.

In addition, both the qualitative and quantitative comparisons show that the performance gap between homoViG and LFHomo is relatively small. To analyze whether these differences are statistically meaningful, we conduct paired t-tests on ACE across the three datasets and present the boxplot distributions of LFHomo and homoViG in Figure 9.

On the synthetic benchmark dataset, the ACE distributions of the two methods are highly similar, with a p-value of about 0.73, indicating that their accuracies are essentially at the same level in this scenario. By contrast, on the UHBD and NIUHBD UAV datasets, the ACE distributions of LFHomo are slightly shifted downward, with both the median and mean values being lower than those of homoViG. Although the corresponding p-values (0.14 and 0.27, respectively) do not reach the conventional 0.05 significance level under the current sample, LFHomo still shows a more consistent trend of error reduction while maintaining low computational complexity.

4.5. Comparison on the Real-World Dataset

We further conduct a quantitative comparison on the CVC real-world dataset [52], and the results are reported in

Table 8. Comparison of corner errors between the proposed algorithm and all other methods on the real-world dataset.

(1)	Method	Easy	Moderate	Hard	Average	Failure Rate
(2)	I3×3	2.56	4.13	6.17	4.28	-
(3)	SIFT + RANSAC	81.19	-	-	81.19	89%
(4)	SIFT + MAGSAC++	81.91	-	-	81.91	98%
(5)	ORB + RANSAC	112.31	-	-	112.31	81%
(6)	ORB + MAGSAC++	64.29	-	-	64.29	81%
(7)	BRISK + RANSAC	63.58	-	-	63.58	79%
(8)	BRISK + MAGSAC++	52.86	-	-	52.86	81%
(9)	DADHN	2.17	3.95	5.50	4.03	0%
(10)	HomoMGAN	2.47	3.89	5.52	4.12	0%
(11)	FCTrans	2.26	3.76	5.44	3.98	0%
(12)	LCTrans	2.14	3.68	5.39	3.90	0%
(13)	homoViG	2.03	3.64	5.36	3.82	0%
(14)	LFHomo(Ours)	2.17	3.31	4.59	3.57	0%

The black bold number denotes the best result.

. Similarly to the synthetic benchmark dataset, traditional methods also exhibit a large number of alignment failures on CVC, whereas deep learning-based approaches show more stable performance. Also, LFHomo does not achieve the best results on the Easy difficulty level, but attains the lowest errors on the other two difficulty levels. A plausible explanation is that, under the Easy setting, the image quality is relatively high, allowing methods such as homoViG [26] to exploit richer features and larger model capacity to achieve slightly better accuracy. In contrast, LFHomo sacrifices only a small amount of accuracy in these simple cases, while achieving significant error reductions on more challenging samples with poorer image quality. This suggests that the proposed method offers stronger robustness in more complex real-world scenarios.

4.6. Ablation Studies

4.6.1. Effectiveness of Proposed Components

To verify the contribution of each module to the model’s performance, we conducted a series of ablation experiments, and the results are shown in

Table 9. Ablation experiment results of different proposed components. Among them, ✓ and × represent whether the component is enabled.

Shift Module	SRCSSA	Backbone	ACE			Time (s)	Parameter (M)
			Synthetic Benchmark	UHBD	NIUHBD
✓	×	LFHomoE	5.02	6.33	6.23	0.09	3.232
×	✓	LFHomoE	5.01	6.31	6.21	0.09	3.233
✓	✓	ResNet-34 [54]	5.08	6.63	6.52	0.27	23.53
✓	✓	ViT [55]	5.11	6.49	6.35	0.13	20.45
✓	✓	ViG [35]	4.68	6.26	6.20	0.11	5.877
✓	✓	MobileViT [31]	5.17	6.64	6.45	0.09	1.848
✓	✓	MobileViG [33]	4.76	6.25	6.19	0.09	3.232
✓	✓	RFViG [26]	4.65	6.22	6.17	0.12	5.879
✓	✓	LFHomoE (Ours)	4.73	6.14	6.12	0.09	3.233

Black bold font represents the best results.

. Firstly, in the first two lines, we verified the complementarity between the shift module [34] and SRCSSA. As can be seen, when utilized separately, both the shift module and the SRCSSA module have been shown to yield limited improvement of the model’s performance. However, when employed in combination, they have been shown to enhance the homography estimation accuracy of the model, thus demonstrating a synergistic effect in capturing edge features.

Subsequently, to validate the benefit of the proposed homography estimators in achieving a balance between efficiency and accuracy, we compared the performance of four backbones, including ResNet-34 [54], ViT [55], ViG [35], MobileViT [31], MobileViG [33] and RFViG [16]. Among them, although MobileViT [31] has the smallest number of parameters, its homography estimation accuracy is relatively low. RFViG [26] achieves the best accuracy, but LFHomo attains a more favorable trade-off among inference speed, parameter count, and estimation accuracy.

4.6.2. Effectiveness of Attention

To further evaluate the effectiveness of different attention mechanisms in mitigating motion blur and vibration, we compared the proposed SRCSSA with several commonly used attention mechanisms, including SENet [56], CBAM [57], ECA [58], Coordinate Attention (CA) [59], Triple Attention (TA) [60], and Efficient Local Attention (ELA) [61], as presented in

Table 10. Comparison of average corner error and parameters of different attentions in feature extractors on three datasets.

Attention	ACE			Time (s)	Parameter (M)
	Synthetic Benchmark	UHBD	NIUHBD
SENet [56]	5.09	6.71	6.67	0.09	3.232
CBAM [57]	4.98	6.54	6.60	0.11	3.233
ECA [58]	5.02	4.56	6.64	0.09	3.232
CA [59]	4.85	6.48	6.41	0.10	3.234
TA [60]	4.80	6.25	6.29	0.10	3.233
ELA [61]	4.82	6.46	6.53	0.10	3.233
SRCSSA (Ours)	4.73	6.14	6.12	0.09	3.233

Black bold font indicates the best results.

. The results demonstrate that channel-wise attention mechanisms, such as SENet and ECA, yield a slightly lower computational cost and higher efficiency. Nonetheless, their accuracy is notably inferior to that of other attention mechanisms. This may be attributed to the fact that single-dimensional channel attention neglects the modeling of multi-dimensional feature relationships, leading to reduced model precision. In contrast, attention mechanisms such as CA and TA exhibit comparable computational efficiency and load to SRCSSA, but the average corner errors across the three datasets are still higher than SRCSSA.

Additionally, we investigate the impact of different group numbers and spatial reduction ratios on the performance of SRCSSA. The results are summarized in

Table 11. SRCSSA performance changes under different group numbers and space reduction rate settings. The best results are indicated in black bold.

$\mathit{G}$	$\mathit{R}$	ACE
		Synthetic Benchmark	UHBD	NIUHBD
S	0.5	4.99	6.34	6.25
1	0.5	5.08	6.58	6.51
2	0.5	5.06	6.53	6.44
4	0.5	4.73	6.14	6.12
8	0.5	5.02	6.42	6.37
4	1.0	4.76	6.17	6.16
4	0.25	4.76	6.33	6.28

. In the first row, $G$ is set to “S”, indicating a standard channel shuffle operation with four groups. The experiments show that SRCSSA achieves better performance than standard channel shuffle. As the number of groups varies, the ACE on all datasets first decreases and then increases, and the best performance is obtained when $G=4$ and the spatial reduction ratio is set to 0.5.

4.6.3. Hyperparameter Settings in Loss Function

To analyze the impact of the loss hyperparameters on model performance, we compare several combinations of the weights λ and μ in the loss function.

Table 12. The performance of LFHomo under different combinations of loss weights $\lambda$ and $\mu$. Bold values indicate the best result.

$\mathit{\lambda }$	$\mathit{\mu }$	ACE
		Synthetic Benchmark	UHBD	NIUHBD
0.1	0.001	4.93	6.41	6.32
1.0	0.001	4.86	6.24	6.14
1.0	0.01	4.73	6.14	6.12
1.0	0.05	5.11	6.62	6.56
0.1	0.01	5.06	6.42	6.27
2.0	0.001	4.80	6.35	6.24
2.0	0.01	5.10	6.44	6.40

reports the ACE values on the three datasets under different loss-weight settings. It can be observed that when $\mu$ becomes smaller (e.g., 0.001) or larger (e.g., 0.05), the error increases. A similar degradation occurs when $\lambda$ deviates from $1.0$. The best results on all three datasets are obtained when $\lambda =1.0$ and $\mu =0.01$, which is consistent with the hyperparameter configuration adopted in [26].

5. Discussion

5.1. Analysis of SRCSSA and Standard Attention Mechanisms

In this subsection, we compare SRCSSA with standard attention mechanisms from two perspectives: attention structure design and module synergistic effect. SENet [56] and ECA [58] compute global weights only along the channel dimension and do not explicitly model spatial structural information. CBAM [57] applies channel and spatial attention sequentially, but its channel branch still follows a global weighting paradigm, which limits its ability to fully exploit information across different channel subspaces. By contrast, SRCSSA performs attention computation at a lower spatial resolution. This effectively reduces the computational cost and the effect of blur and background noise. Furthermore, the channel-sequential shuffle operation facilitates enhanced cross-channel information exchange, thereby accentuating key features in the images.

Conversely, CA [59], TA [60], and ELA [61] further introduce the concept of cross-dimensional spatial interaction, providing richer contextual representations along different dimensions. Yet, these methods are designed primarily for general-purpose scenarios and are not specifically tailored to the motion-blur conditions considered in this work. In our design, the Shift Module explicitly introduces local contextual information in blurred regions through pixel-level shifts, while SRCSSA focuses and selectively enhances this information. The two modules act in a complementary manner and provide more accurate feature representations for homography estimation in infrared and visible images than generic attention mechanisms. As shown in Table 10 of Section 4.6.2, SRCSSA achieves better or more stable homography estimation performance than several representative attention mechanisms, with comparable parameter counts and inference times.

From the viewpoint of convolutional design, SRCSSA is also closely related to the depth-wise separable convolutions. Depth-wise separable convolutions are responsible for efficiently capturing local spatial patterns within each channel and enhances information flow. SRCSSA is not intended to replace these operators, but to act on top of them as a lightweight refinement module. By performing grouped channel attention and channel-sequential shuffle at a reduced spatial resolution, SRCSSA selectively reweights the features generated by depth-wise separable convolutions. This process redistributes information across channels, strengthening informative subspaces and suppressing redundant responses. In this way, the convolutional blocks provide efficient local feature extraction, whereas SRCSSA focuses on cross-channel interaction and feature selection.

5.2. Limitations and Future Work

Although the proposed method extends the applicability of homography estimation in low-altitude scenarios, several challenges remain to be addressed. Firstly, in real UAV applications, RGB and infrared images captured by onboard sensors often exhibit significant depth variations. However, most existing methods disregard this factor, resulting in inadequate adaptability in complex 3D scenes. Possible reasons include the lack of unregistered low-altitude datasets with depth information and the absence of model architectures that explicitly account for depth variation. Inspired by studies on spatial–spectral interaction in multispectral image processing [62], we are planning to construct a low-altitude multimodal dataset with depth annotations and to introduce interaction modules between depth and multimodal image features. We hypothesize that this is a promising research direction for mitigating depth information loss during registration.

Secondly, for real-time requirements in low-altitude applications, LFHomo achieves relatively fast inference, but direct deployment on UAV platforms with very limited onboard resources remains difficult. In future work, we plan to explore deployment schemes on edge servers by leveraging 5G communication and edge computing technologies. Existing studies have shown that the end-to-edge collaborative paradigm can effectively alleviate on-board computing bottlenecks. Using the “perception–transmission–computation–feedback” pipeline, computation-intensive tasks are offloaded from the UAV to edge nodes [63]. This paradigm particularly benefits from the high data rate and low latency of 5G/6G networks [64], which provides a foundation for building efficient low-altitude computing networks. On this basis, we will further incorporate model compression strategies such as network pruning, knowledge distillation, and low-bit quantization, and systematically measure changes in accuracy and inference time under different compression configurations.

Finally, because related research and publicly available data resources remain limited, LFHomo has been validated to date only in a subset of scenarios. Future work will expand the proposed UHBD and NIUHBDs to cover more complex environments, such as mountainous areas, forests, and sea surfaces. For each scene type and weather condition, the distributions of ACE, matching success rates, and mismatch ratios will be computed, enabling a quantitative analysis of model robustness across diverse environments. In addition, insights from multispectral imaging research on complex environmental perception [65] will be leveraged to more comprehensively assess the generalization capability of the proposed method. These are also expected to facilitate its application to environmental monitoring [66], disaster assessment [67,68], 3D reconstruction [69,70], and search-and-rescue tasks [71,72]. Also, we are interested in extending the method to hyperspectral image processing tasks [73,74] to evaluate its potential contribution to complex environmental perception and decision-making.

6. Conclusions

This paper addresses the problem of homography estimation between infrared and visible images from a UAV perspective, proposes a lightweight, unsupervised end-to-end model, LFHomo. Meanwhile, a corresponding UAV-based multimodal image dataset is also constructed to support model training and evaluation. Experimental results on multiple infrared–visible datasets show that LFHomo achieves competitive estimation accuracy while significantly reducing model complexity and inference time. At the same time, the model maintains good robustness under motion blur. We believe that this study provides a new perspective on efficient geometric registration of multimodal images in UAV scenarios, and suggests potential applications for real-time perception and multi-source information fusion in space–air–ground integrated networks.