UAV Detection in Low-Altitude Scenarios Based on the Fusion of Unaligned Dual-Spectrum Images

Highlights

What are the main findings?

• The SURF algorithm, improved based on the grayscale centroid method, can effectively match the feature points in visible light–infrared paired images.
• An enhanced YOLOv11 model with a modified loss function can achieve robust UAV detection in the fused images.

What are the implications of the main findings?

• The plane-adaptive pixel remapping algorithm can be used to get dual-spectrum fusion image based on the feature point matching result.
• The “Fusion and Detection” methodology enables stable image monitoring of UAV targets in complex environments.

Abstract

The threat posed by unauthorized drones to public airspace has become increasingly critical. To address the challenge of UAV detection in unaligned visible–infrared dual-spectral images, we present a novel framework that comprises two sequential stages: image alignment and object detection. The Speeded-Up Robust Features (SURF) algorithm is applied for feature matching, combined with the gray centroid method to remove mismatched feature points. A plane-adaptive pixel remapping algorithm is further developed to achieve images fusion. In addition, an enhanced YOLOv11 model with a modified loss function is employed to achieve robust object detection in the fused images. Experimental results demonstrate that the proposed method enables precise pixel-level dual-spectrum fusion and reliable UAV detection under diverse and complex conditions.

1. Introduction

With the rapid increase in drone usage, unauthorized operations have become more frequent, posing substantial risks to public safety. In airport environments, such activities may interfere with normal aircraft takeoff and landing. For example, in 2024, flight delays at Tianjin Airport were attributed to disruptions caused by unauthorized drones. In sensitive areas, drones equipped with cameras can be exploited for illicit imaging and real-time online video transmission, raising concerns regarding potential espionage [1].

Air surveillance technologies are essential for countering unmanned aerial vehicles (UAV) [2]. Among them, radar-based surveillance remains the most widely employed approach [3]. In 2019, the U.S. Department of Defense introduced the short-range “Spyglass” radar system, which integrates 3D radar with artificial intelligence (AI) to enhance the detection and identification of UAV swarms. However, drones are characterized as “low, slow, and small” targets, making detection technically challenging due to limited radar cross-sections, low-altitude flight, and frequent occlusion [4].

Electro-optical detection is commonly applied as a complementary surveillance technique [5,6]. The “Raven” counter-UAV system, developed by Indra Defense Systems in Spain, combines high-resolution imaging radar with an electro-optical detection subsystem to enable rapid UAV identification and tracking. In 2024, the AI-powered “Sky Eye” surveillance system was deployed in Washington, DC, USA, integrating advanced computer vision algorithms and capable of simultaneously monitoring up to 5000 aerial targets.

During the synchronous acquisition of visible and infrared images by photoelectric equipment, discrepancies in pixel alignment often occur. These misalignments result from several factors, including the difficulty in precisely synchronizing the imaging times of different sensors, challenges in adjusting the lens focal lengths to achieve consistent image scales, and relative angular deviations between optical centers caused by camera shake and mechanical installation errors. As illustrated in Figure 1, the position and size of the unmanned aerial vehicle differ in the dual-band images captured simultaneously.

This study introduces a novel UAV detection framework that employs visible–infrared image fusion under non-aligned conditions in various perspectives and scales. The main contributions are summarized as follows:

(a)

A feature matching and fusion method for non-aligned visible and infrared dual-spectrum images is proposed, applicable to the output processing of diverse dual-spectrum surveillance systems.

(b)

An enhanced YOLOv11-based detection algorithm is developed, designed to operate jointly with the proposed fusion method to achieve accurate UAV identification in fused imagery.

2. Research Status of UAV Detection

2.1. Visible Light-Based Method

This type of research primarily focuses on exploring the extension methods for object detection models. Liu et al. (2024) [7] provided a comprehensive review of the main challenges in drone detection, summarized existing solutions, and evaluated and compared commonly used datasets. Samadzadegan et al. (2022) [8] utilized the YOLOv4 model to detect two types of aerial targets—drones and birds—and implemented strategies to reduce bird-related interference. Yasmine et al. (2023) [9] enhanced the YOLOv7 architecture by incorporating the CSPResNeXt module, integrating the Transformer module with the C3TR attention mechanism, and adopting a decoupled head structure to improve detection performance. Hu et al. (2023) [10] proposed a modified YOLOv7-tiny-based detection model that optimized anchor box allocation based on the aspect ratios of actual bounding boxes and introduced the hard sample mining loss function (HSM Loss) to enhance the network’s focus on challenging samples.

2.2. Infrared-Based Method

The majority of research in this field is concentrated on the detection of infrared dim and small targets. Lu et al. (2022) [11] proposed a UAV detection model based on an enhanced adaptive feature pyramid network (EAFPAN) to address the issue of weakened feature representation in cross-scale infrared feature fusion. Zhao et al. (2023) [12] introduced a method using the Isolation Forest (iForest) algorithm. By constructing a global iForest they leveraged the inherent isolation tendency of UAVs, effectively mitigating detection difficulties caused by low signal-to-clutter ratios (SCRs). Xu et al. (2024) [13] developed the RMT-YOLOv9s model, which incorporated an improved multi-scale feature fusion network, RMTELAN-PANet, to effectively capture semantic information across feature maps. Additionally, Fang et al. (2024) [14] proposed the Spatial and Contrast Interactive Super-Resolution Network (SCINet), consisting of two branches: a spatial enhancement branch (SEB) and a contrast enhancement branch (CEB). Yang et al. (2023) [15] drew upon the visual processing mechanism of the human brain to propose an effective framework for UAV detection in low signal-to-noise ratio infrared images. In addition, Pan et al. (2024) [16] proposed the AIMED-Net, which incorporates a variety of feature enhancement techniques based on YOLOv7. Fang et al. (2022) [17] redefined infrared UAV detection as a residual image prediction task, developing an end-to-end U-shaped network that learns to map input images to residual images.

Some other studies focused on the pursuit of lightweight models. Cao et al. (2024) [18] introduced YOLO-TSL, which embeds the triplet attention mechanism into the YOLOv8n backbone and adopts the Slim-Neck architecture, achieving reduced computational complexity while maintaining high detection accuracy. Wang et al. (2024) [19] developed the PHSI-RTDETR model, combining a lightweight feature extraction module (RPConv-Block), a refined neck structure (SSFF), the HiLo attention mechanism, and the Inner-GIoU loss function to improve detection performance.

2.3. Dual-Spectrum Fusion-Based Method

This type of research usually simultaneously proposes methods for the fusion of dual-spectrum image features and object detection. Wang et al. (2025) [20] proposed RF-YOLO, an object detection method designed for both visible light and infrared images, which addresses the challenges of noise and uncertainty across different modalities by incorporating a dual feature fusion (DFF) module and a feature fusion corrector (FFC) module into the YOLOv5 framework. Gao et al. (2024) [21] proposed EfficientFuse, a multi-level cross-modal fusion network that captures both local dependencies and global contextual information at shallow and deep network levels. They further developed the AFI-YOLO detection framework to effectively reduce background interference in fused imagery. Chang et al. (2025) [22] introduced MPIFusion, an image fusion network that integrates a dual-channel shallow detail fusion module (DC-SDFM) and a deep feature fusion block (DFFB), effectively merging hierarchical features to produce high-quality fused images. Liu et al. (2025) [23] introduced a visible light and infrared image matching method based on a joint local attention mechanism, enabling precise positioning for UAVs. Jiang et al. (2024) [24] converted the cross-modal task into a single-modal framework by extracting and fusing features from infrared and visible light images through deep learning networks, thereby enabling target alignment for small UAVs. Zhu et al. (2025) [25] proposed the YOLOv9-CAG, an enhanced algorithm that demonstrated strong performance on both infrared and visible light datasets. Suo et al. (2025) [26] proposed a novel detection model, YOLOv9-AAG. The model’s detection head incorporates GSConv to efficiently capture edge and contour information, and integrates an Attention-based Feature Fusion (AFF) mechanism to enhance shape feature representation, thereby improving discrimination between birds and drones.

Most existing studies primarily concentrate on the design of image fusion and object detection network architectures, while overlooking the pixel-level coordinate discrepancies between visible light and infrared images, and have not yet incorporated more sophisticated object detection frameworks to address complex real-world scenarios.

3. Research Methods

The methodology flowchart of this paper is illustrated in Figure 2, which outlines the research methodology of “Fusion and Detection”, it also reflecting the thought of “Clarification—Recognition”. The workflow illustrated in Figure 2 can be summarized as follows: Firstly, the input visible-light and infrared images undergo a sharpening process. An improved speeded-up robust features (SURFs) algorithm is then employed to extract, match, and screen feature points from both modalities. Subsequently, pixel remapping and pixel reconstruction are performed to achieve feature-level image fusion. Finally, target detection is conducted on the fused image using an enhanced YOLOv11 model.

3.1. Feature Matching and Selection

3.1.1. Image Sharpening

Considering the distinct pixel distribution characteristics between visible light and infrared images, along with their similar edge features, the Laplacian operator is applied to perform differential computations within the pixel neighborhood. This method effectively captures abrupt changes in gray-level intensity and enhances the image’s edge features. The calculation is as follows:

$${\mathbf{I}}_{\mathrm{s}\mathrm{h}\mathrm{a}\mathrm{r}\mathrm{p}\mathrm{e}\mathrm{n}\mathrm{e}\mathrm{d}}=\mathbf{I}-k·(\mathbf{I}\ast {\nabla }^2)$$

(1)

Here, $\mathbf{I}$ denotes the original image, $k$ is the scaling coefficient that controls the sharpening intensity, and ${\nabla }^2$ represents the Laplacian convolution kernel.

3.1.2. Feature Matching

The SURF algorithm [27] was used to detect and match feature points between visible and infrared images. This approach enables the rapid detection of multi-scale feature points by constructing the Hessian matrix. Meanwhile, it improves computational efficiency through the use of integral images. To generate rotation-invariant descriptors, Haar wavelet responses are statistically analyzed within the local neighborhoods of the detected feature points. Finally, feature correspondences are established by matching these descriptors via the nearest-neighbor method.

3.1.3. Feature Selection

Due to the inherent differences in the color spaces of visible light and infrared images, mismatches are inevitable. After removing feature point pairs with excessively large spatial distances, the grayscale centroid method is used to compute the grayscale dissimilarity between corresponding feature points. The key principle of this method involves calculating an angular difference. This is done between the centroid vectors of the image regions that surround each feature point:

$$\theta =\arctan 2(m_{01}, m_{10})$$

(2)

Here, $\mathrm{a}\mathrm{r}\mathrm{c}\mathrm{t}\mathrm{a}\mathrm{n}2(·)$ denotes the four-quadrant inverse tangent function, while $m_{01}$ and $m_{10}$ correspond to the first-order moments of the image block. Feature point pairs exhibiting significant grayscale differences are excluded, and the remaining pairs are retained as the final matching results.

3.2. Feature Fusion

3.2.1. Pixel Remapping

The spatial variation in objects in low-altitude surveillance imagery, particularly the differences in scales between visible light and infrared images, presents a challenge for accurate pixel-level mapping. Traditional pixel coordinate mapping relies on homography transformation, which necessitates a large number of control points. To overcome this limitation, this study proposes an alternative approach that assigns each pixel to a distinct plane. By selecting the two closest sets of feature points: $A$ and $B$, identify their corresponding matching points in the infrared image: $A^{\prime }$ and $B^{\prime }$, as the control points for pixel $C$. The coordinate calculation method for point $C^{\prime }$ corresponding to point $C$ in the infrared image is as follows:

$$\left\{\begin{array}{c}{C}_x^{\prime }=A_x^{\prime }+a(C_x-A_x)-b(C_y-A_y)\\ C_y^{\prime }=A_y^{\prime }+a\left(C_y-A_y\right)+b(C_x-A_x)\end{array}\right.$$

(3)

$$\left\{\begin{array}{c}a={\displaystyle \frac{\left(B_x^{\prime }-A_x^{\prime }\right)\left(B_x-A_x\right)+\left(B_y^{\prime }-A_y^{\prime }\right)\left(B_y-A_y\right)}{{\left(B_x-A_x\right)}^2+{\left(B_y-A_y\right)}^2}}\\ b={\displaystyle \frac{\left(B_y^{\prime }-A_y^{\prime }\right)\left(B_x-A_x\right)+\left(B_x^{\prime }-A_x^{\prime }\right)\left(B_y-A_y\right)}{{\left(B_x-A_x\right)}^2+{\left(B_y-A_y\right)}^2}}\end{array}\right.$$

(4)

The above formulas indicate that the two corresponding pixels in dual-spectrum images share the same relative position as the control points. The control points in planes at varying depth are illustrated in Figure 3, demonstrating the feasibility of precise inter-image pixel matching.

3.2.2. Pixel Reconstruction

Based on the mapping relationship between visible light and infrared pixel coordinates, and leveraging the “black hot” feature of the drone target in the infrared image, the surrounding region of the target pixel is reconstructed. The calculation procedure is as follows:

$$\mathrm{RGB}\_\mathrm{E}\mathrm{n}\mathrm{h}\mathrm{a}\mathrm{n}\mathrm{c}\mathrm{e}\mathrm{d}(x, y)=\left[\mathrm{RGB}(x, y)+\epsilon \right]\times \mathrm{Infrared}(x^{\prime }, y^{\prime })/255$$

(5)

Here, $\epsilon$ denote the regulation parameter. Thus, the thermal values from the infrared image can be mapped onto the visible-light image. Each reconstructed pixel thereby carries combined information from both the visible and infrared spectral bands.

3.3. Object Detection

3.3.1. Model Structure

YOLOv11 [28], released in 2024, improves upon YOLOv10 [29] by addressing issues related to multi-scale object detection, small-object omissions, and bounding box overlap in complex scenes. The overall architecture is illustrated in Figure 4, which consists of three parts: Backbone, Neck, and Head. Its main innovation lies in the incorporation of the Bottleneck Block, SPPF, and C2PSA modules, which jointly strengthen feature extraction. We employ YOLOv11, incorporating an optimized loss function, as the drone target detection model.

3.3.2. Loss Function

The EIoU Loss [30] further decomposes the aspect ratio penalty term of the original CIoU Loss in YOLOv11, independently constraining the width–height disparity to enhance regression efficiency. The loss function is calculated as follows:

$$EIoU\_Loss=1-IoU+{\displaystyle \frac{p^2\left(b,b^{gt}\right)}{c^2}}+{\displaystyle \frac{p^2(w,w^{gt})}{c_w^2}}+{\displaystyle \frac{p^2(h,h^{gt})}{c_h^2}}$$

(6)

Here, $p^2\left(b,b^{gt}\right)$, $p^2(w,w^{gt})$ and $p^2(h,h^{gt})$ represent the square of the distance between the center points, the width and height differences between the predicted bounding box and the ground truth. $c$, $c_w$ and $c_h$ correspond to the diagonal length, width and height of the smallest enclosing rectangles, which are used to normalize the penalty term.

Based on Equation (5), focal loss [31] is introduced, which is specially designed to address the imbalance between positive and negative samples. It improves bounding box regression accuracy by dynamically adjusting the weights of samples with different IoU values, enabling the model to focus more effectively on hard-to-regress samples during training. The Focal-EIoU loss function is obtained as:

$$Loss=1-IoU+\alpha \times \left[{\displaystyle \frac{p^2\left(b,b^{gt}\right)}{c^2}}+{\displaystyle \frac{p^2\left(w,w^{gt}\right)}{c_w^2}}+{\displaystyle \frac{p^2\left(h,h^{gt}\right)}{c_h^2}}\right]+\beta \times \left[{{\displaystyle \frac{{Area}_{\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{s}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}}}{{Area}_{\mathrm{u}\mathrm{n}\mathrm{i}\mathrm{o}\mathrm{n}}+\sigma }}}^n\right]-\gamma$$

(7)

Here, ${Area}_{\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{s}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}}$ and ${Area}_{\mathrm{u}\mathrm{n}\mathrm{i}\mathrm{o}\mathrm{n}}$ refer to the intersection and union, respectively, of the predicted bounding box and the ground truth. $\sigma$ is a minimum value, $n=0.05$. Through a series of experimental evaluations conducted in this study, the optimal value of the parameters $\alpha$, $\beta$ and $\gamma$ are determined as 0.9, 0.1 and 0.065, respectively.

4. Dataset

This study utilizes the Anti-UAV drone object detection dataset [32] released by the University of Chinese Academy of Sciences, which includes over 300 visible–infrared image pairs featuring a variety of commonly available drone models currently on the market. The drones in the dataset exhibit a wide range of flight distances, orientations, lighting conditions (including backlighting), and background environments, with additional inclusion of nighttime scenarios. A key feature of the dataset is its ability to simulate misaligned images captured by a dual-spectrum camera under typical operational conditions. This characteristic makes the dataset particularly suitable for evaluating the performance of the proposed visible–infrared image fusion method.

A challenging subset of images was selected from the visible and infrared image database, covering multi-scale objects and complex architectural backgrounds. A total of 1570 image pairs along with the corresponding annotation files (accurately marking the UAV target locations) were collected to form the final dataset and will be used for dual-light fusion and object detection evaluations.

5. Experiment and Analysis

5.1. Dual-Spectrum Fusion

We selected three groups of dual-spectrum images from the dataset as the test images, which were sourced from representative scenarios. Among them, the sizes of the drones in the first group of images vary significantly, while the drones in the second and third groups are smaller and have different backgrounds.

According to Section 3.1, feature matching and selection were conducted on three representative images obtained under scenes of diverse background and drone distance, the results were presented in Figure 5, and further detailed with statistical data in

Table 1. Statistics of feature matching and selection results.

Scene	Before Selection			After Selection
	All Pairs	Matched	Precision	All Pairs	Matched	Precision
Scene 1	12	8	66.7%	6	6	100.0%
Scene 2	25	22	88.0%	14	13	92.9%
Scene 3	14	6	42.9%	6	6	100.0%

. It is evident that the proposed feature matching method effectively identifies feature points in both foreground and background regions, while eliminating incorrect feature correspondences.

Building upon the results of feature matching, further feature fusion was performed. This process is illustrated in Figure 6. As can be observed in the final visible light test image, the effective pixel regions—defined as those containing no fewer than two corresponding control points—are enhanced by the texture and brightness characteristics of the infrared image. Demonstrate that feature fusion significantly enhances the contrast of the foreground regions in the original image, thereby facilitating more effective feature extraction for the object detection network.

5.2. Object Detection

We initially conduct data preparation. For each pair in the dataset, the corresponding feature-fused image was generated using the proposed Dual-spectrum fusion method, with original labels preserved. These image pairs were then partitioned into training, validation, and test sets at a ratio of 7:2:1, respectively.

The model training was conducted using the Python 3.8 programming language. The experimental platform is a Lenovo brand AI workstation purchased from Shanghai, China, with a CPU (Intel Core i9-14900HX, 24 cores), 32 GB of RAM, and a GPU (NVIDIA GeForce RTX 4060). The model was trained with an initial learning rate of 0.01, a momentum of 0.937, a batch size of 6, and utilized 8 CPU workers. The training utilized the dataset described in Section 4.

To evaluate the model’s detection capability, an ablation experiment was conducted. The results are presented in

Table 2. Results of the ablation experiment.

Model	Precision	Recall	mAP50	mAP50-95
YOLOv11	0.990	0.990	0.994	0.675
YOLOv11-EIoU	0.995	0.995	0.994	0.671
Proposed method	0.995	0.995	0.995	0.673

. As shown in the table, the YOLOv11-EIoU model demonstrates significantly improved detection performance compared to the original YOLOv11 model, but the proposed method achieves the highest performance. Compared with state-of-the-art studies in this field, the YOLOv9-CAG architecture [25] and the YOLOv9-AAG architecture [26] achieved mean Average Precision (mAP) scores of 0.842 and 0.816, respectively, on their respective combined dataset of visible and infrared images. In contrast, the proposed method attained mAP of 0.995, demonstrating its superior learning capability.

The model was directly applied to UAV detection in fused dual-spectral images. Detection experiments were conducted using multispectral image data collected from diverse scenarios, encompassing variations in scale and background complexity. The results, presented in Figure 7, show that the detection confidence levels are all above 0.7, demonstrating that the detection model is capable of accurately localizing UAV targets across a range of challenging conditions.

The measured FPS (Frames Per Second) is 213.4, indicating that the model can achieve a relatively high detection speed under the existing computational resources. Furthermore, given that the proposed image fusion method is a traditional image processing approach, which consumes negligible computational resource, the “Fusion—Detection” methodology has good usability in real-time drone detection tasks.

6. Conclusions

This paper presents a spatially adaptive feature matching and fusion method for visible–infrared images, specifically designed to address the misalignment characteristics of dual-spectrum Low-altitude aerial monitoring images. The proposed method significantly enhances the feature representation capabilities of key regions in single-spectrum images. Additionally, the loss function of the YOLOv11 model is optimized and applied to improve the detection performance of drone targets in the fused images.

The method drawing on the widely adopted “Fusion–Detection” paradigm in related studies, this method proposes an image fusion approach that is computationally more efficient than conventional deep learning-based fusion networks. Through model lightweight optimization and format conversion, the detection framework can be deployed on lower-end systems or real-time applications. enabling work in conjunction with fixed surveillance cameras as well as airborne or vehicle-mounted mobile cameras.

In this study, the detection model was trained solely on the Anti-UAV dataset. Provided that sufficient image samples and scene diversity are available, the trained model can be effectively transferred to other drone platforms, including fixed-wing, hybrid drones and so on.

The proposed pixel reconstruction strategy relies heavily on the feature point combinations. Under extreme conditions such as low illumination or severe object occlusion, it may be difficult to obtain valid feature point groups to serve as control points, leading to potential failure of the feature fusion process. Future work will focus on further optimizing the algorithm to enhance its robustness in feature extraction under varied scales, lighting conditions, and complex environments.