DL-DEIM: An Efficient and Lightweight Detection Framework with Enhanced Feature Fusion for UAV Object Detection

Abstract

UAV object detection is still difficult to achieve due to large-scale variation, dense small objects, a complicated background, and resource constraints from onboard computing. To solve these problems, we develop a diffusion-enhanced detection network, DL-DEIM, tailored for aerial images. The proposed scheme generalizes the DEIM baseline across three orthogonal axes. First, we propose a lightweight backbone network called DCFNet, which utilizes a DRFD module and a FasterC3k2 module to maintain spatial information and reduce computational complexity. Second, we propose a LFDPN module, which can conduct bidirectional multi-scale fusion via frequency-spatial self-attention and deep feature refinement and largely enhance cross-scale contextual propagation for small objects. Third, we propose LAWDown, an adaptive-content-aware downsampling to preserve the discriminative representation with higher accuracy at lower resolutions, which can effectively capture the spatially-variant weights and group channel interactions. On the VisDrone2019 dataset, DL-DEIM achieves a mAP@0.5 of 34.9% and a mAP@0.5:0.95 of 20.0%, outperforming the DEIM baseline by +4.6% and +2.9%, respectively. The model maintains real-time inference speed (356 FPS) with only 4.64 M parameters and 11.73 GFLOPs. Ablation studies validate the fact that DCFNet, LFDPN, and LAWDown collaboratively contribute to the accuracy and efficiency. Visualizations also display clustered and better localized activation in crowded scenes. These results show that DL-DEIM achieves a good tradeoff between detection probability and computation burden and it can be used in practice on resource-limited UAV systems.

1. Introduction

Visual sensor-based unmanned aerial vehicles (UAV) have been widely deployed in various application scenarios, such as disaster emergency responding [1], urban traffic monitoring [2], air map, and military reconnaissance. The explosion of UAV technology has resulted in an increasing demand for effective and high-performance solutions for online object detection on aerial images. Compared with ground images, UAV images also exhibit several distinctive characteristics, which have posed strong challenges to the traditional computer vision methods: comprehensive coverage of the scene with cluttered background, overwhelmingly top view with occlusion, large variations in scale (orders of magnitudes), very small quantity of small objects in terms of image scales, and diversity in the illumination [3].

UAV platforms come with strict hardware constraints, which demand a detection algorithm to achieve millisecond-level inference at milliwatt-level power budgets. These severe resource constraints make researchers re-think the design paradigms of deep networks [4]. The imaging features resulting from the bird’s-eye view perspective pose very challenging pattern recognition problems: target scale changes on the order of a hundred-fold, small targets are highly cluttered and can be easily timbered by complex surrounding objects, and target shapes are distorted by flight attitude, causing further difficulty during recognition [5]. All of these obstacles have greatly promoted the technical difficulties of aerial intelligent perception and need to be urgently solved. Figure 1 shows an UAV aerial image.

Figure 1. Daytime drone-captured image.

In recent years, object detection techniques have seen a transition from the anchor-box-based methods to end-to-end learning. Two-stage approaches form region proposals (region-based convolutional neural network (R-CNN) and its faster versions, including fast R-CNN, faster R-CNN, and mask R-CNN [6,7,8,9,10]), achieve high accuracy but require a cascaded pipeline of components and, thus, are not real-time. Single-stage detectors (such as the You Only Look Once (YOLO) series [11,12,13,14,15]) make dense predictions and effectively enhance inference efficiency, thus striking a balance between processing speed and accuracy in general environments. However, these convolutional neural network (CNN)-based methods have intrinsic drawbacks for aerial image processing: the local receptive field of the convolutional operation restricts the modeling of global context, leading to difficulty in capturing long-range dependencies between objects in aerial scenes. Meanwhile, the non-maximum suppression (NMS) in the post-processing phase dramatically increases the computational complexity in dense target areas and becomes the performance bottleneck of real-time detection [16].

The introduction of the transformer architecture now provides a new route to overcome these constraints. Detection transformer (DETR) [17] was the first to use self-attention in object detection, and the Hungarian algorithm was used to match in a one-to-one manner between queries and targets, completely discarding anchor box design and NMS post-treatment, and achieving real end-to-end learning. This simple yet principled design reduces the overhead of detection pipelines and shows its merits in occlusion and crowded scenarios. While the original DETR can be trained to acceptable accuracy, it leads to slow convergence and high time consumption, which is not suitable for real-world resources such as smart devices and graphics processing unit (GPU) servers [18].

The effectiveness of diffusion models as generative models has motivated new investigations in the detection literature. Diffusion model with explicit instance matching (DEIM) [19] creatively introduces instance matching to diffusion denoising and iteratively performs optimization to progressively refine target estimations. Different from classical one-shot prediction, DEIM adopts a progressive optimization mechanism so that the model iteratively evolves from coarse to accurate detection. Most importantly, DEIM’s dense one-to-one (O2O) matching augments the number of targets in training images, to achieve supervision density as high as one-to-many (O2M) but with the benefits of one-to-one matching. Together with the tailored dual-endpoint matching loss (matchability-aware loss), DEIM makes a breakthrough in training efficiency: it achieves better detection performance while halving the amount of training time.

The backbone network in the DEIM model is implemented by using high-performance GhostNet version 2 (HGNetv2) to extract features. For real-time object detection tasks, the HGStage modules in HGNetv2 can obtain strong feature representation power, but the structurally complex HGStage units and the squeeze-and-excitation (SE) attention mechanism are computationally intensive, making it difficult to deploy the model on edge devices. Research by Chen et al. [20] showed that overly complex networks significantly increase inference latency and add barely any improvement in accuracy. Meanwhile, Liu et al. [21] pointed out that single feature extraction modules cannot cope with the simultaneous detection of objects of various sizes, such as in complex scenes containing multi-scale objects. Several recent works have attempted to improve HGNetv2 in different domains. For instance, an improved YOLOv11n segmentation model has been proposed for lightweight detection and segmentation of crayfish parts [22]; ELS-YOLO [23] and LEAD-YOLO [24] enhances UAV object detection performance in low-light conditions based on an improved YOLOv11 framework; and HCRP-YOLO, along with other HGNetv2-based approaches (LEAD-YOLO [24]), has been applied in agricultural and inspection tasks, demonstrating the backbone’s potential for real-world UAV deployments [25]. However, these improvements are often tailored for specific application scenarios or environmental constraints and do not provide a general solution to the challenges of multi-scale, cluttered, and resource-constrained UAV imagery, which motivates our proposed discriminative correlation filter network (DCFNet) design.

We propose a new DCFNet backbone network in this paper. DCFNet takes advantage of multi-scale features and background information for better performance. The network integrates the downsampling with reduced feature degradation (DRFD) module, which aims to improve standard downsampling techniques by maintaining higher spatial information during resolution reduction, leading to less feature loss. Moreover, we also propose an innovative feature enrichment module, FasterC3k2, by fusing the C3k2Block (as cross-stage partial network (CSPNet)) module and FasterBlock [26], effectively alleviating the superfluous calculations through parameter settings. The whole network follows a progressive cyclic structure, which not only guarantees full feature extraction but also saves a large amount of computational resources.

In the neck network, the fusion module of DEIM is in charge of multi-scale feature fusion. Object distribution and scale structure change with shooting angle in UAVs to some extent. Existing conventional feature pyramid networks adopt only simple residual concatenation and re-parameterization block (RepBlock [27]), which makes the model fail to fully express the complex features. To solve this problem, the paper proposes the lightweight feature diffusion pyramid network (LFDPN). The feature diffusion mode of LFDPN can effectively diffuse the context-rich features to different detection scales, through the bidirectional feature propagation paths from top-down to bottom-up. As shown in stage 1 of the network, the output of the feature focusing module is fused with high-resolution and low-resolution feature maps by downsampling and upsampling, respectively, to generate rough multi-scale feature representations. The second stage even realizes deep feature fusion together with complete context information diffusion by the second feature focusing (FocusFeature) and deep feature enhancement submodules (re-parameterized high-level multi-scale (RepHMS)). Moreover, this paper introduces lightweight adaptive weighted downsampling (LAWDown), which, together with LFDPN’s dual-stage feature spreading mechanism, successfully mitigates the loss of information caused by scale discrepancy in conventional feature pyramid networks, leading to a notable enhancement of consistent and expressive multi-scale features.

In summary, the main improvements and innovations of this paper are as follows:

1.

To minimize feature loss during the downsampling process and reduce redundant computations, we design a novel backbone network DCFNet for UAV object detection. It introduces the DRFD downsampling module and designs the FasterC3k2 feature enhancement module composed of C3k2Block and FasterBlock, which enhances feature extraction capability while achieving model lightweighting;

2.

To enable the model to fully express complex features, we design a novel neck network LFDPN, which effectively alleviates the information loss problem caused by scale differences in traditional feature pyramid networks through deep integration of cross-scale features and the design of the FocusFeature focusing module;

3.

To address the difficulty in balancing information preservation and computational efficiency in traditional convolutional downsampling, we design the LAWDown. It integrates adaptive weight mechanisms with grouped convolution, achieving new downsampling through dynamic feature selection and efficient channel reorganization, significantly reducing computational complexity while maintaining accuracy.

2. Materials and Methods

2.1. Overview

The full approach is described in this section, where DEIM framework is optimized for UAV object detection. The particular challenges presented by aerial images with widely varying scales, dense small target distributions, and complex backgrounds require dedicated architecture designs to reconcile detection accuracy and computational complexity. Our solution is designed to tackle these issues in a comprehensive manner by identifying and making incremental enhancements in several steps of the detection process.

Our proposed optimizations have a system-wide design, that golden, solid improvements to one component yield synergistic benefits for the system. We substitute the heavy HGNetv2 backbone with new designed DCFNet backbone, which is a shallower architecture and still has competitive feature learning ability with few more parameters. The feature fusion network of LFDPN adopts a novel bidirectional feature diffusion mechanism, so as to efficiently propagate semantic information as well as spatial details between scales. To deal with the needs for aerial object detection, we creatively compose dedicated modules, such as the DRFD module, the LAWDown sampler and the FocusFeature enhancer, to a systematic solution.

2.2. Base DEIM Framework

In this section, we first present the base DEIM framework, which serves as the foundation for our improved DL-DEIM model. The base framework establishes the overall detection pipeline and highlights the key modules that enable diffusion-based optimization. By understanding the architecture of the baseline system, we can better motivate the enhancements introduced later.

2.2.1. DEIM Architecture Overview

The base DEIM framework adopts a hierarchical architecture for end-to-end object detection, as illustrated in Figure 2. The framework consists of three main components: (1) HGNetv2 backbone for multi-scale feature extraction, (2) HybridEncoder for feature enhancement and fusion, and (3) DFINETransformer decoder with self-distillation for iterative detection refinement.

Figure 2. Structure of the base DEIM framework.

The pipeline is designed to perform progressive feature extraction, multi-scale fusion, and end-to-end detection of UAV images directly without any additional post-processing (e.g., NMS). The most crucial contribution is to integrate diffusion-based fine-tuning and dense supervision schemes.

2.2.2. HGNetv2 Backbone

The hierarchical gradient aggregation is realized on the HGNetv2 backbone, which consists of the specialized HGStage modules and the multi-HGBlock units responsible for dense feature extraction. The backbone gradually extracts features at five scales with continuously increasing receptive fields and semantic abstraction. Given the initial Stem block that downsamples the spatial resolution to H/4 × W/4, the network goes through four hierarchical stages (HGStage1–4) to produce feature maps of scales P2/4, P3/8, P4/16, and P5/32, respectively. At each stage, the proposed pipeline carefully trade-off resolution reduction and channel dimension expansion, to retain fine-grained local details for small object detection, as well as to create sufficient capacity for global context information for semantic understanding. This hierarchical structure guarantees complete multi-scale feature representation and facilitates the gradient flow through dense connections. Plan views of the HGStage and HGBlock modules are shown in Figure 3.

Figure 3. Structure of the HGStage and HGBlock.

2.3. Improved DEIM Framework

Building upon the base DEIM framework, we introduce an improved version termed DL-DEIM.

DL-DEIM Architecture

To overcome the deficiencies of the baseline DEIM without losing its main benefits, we suggest global architectural improvements in all components. The enhanced architecture substitutes the backbone HGNetv2 with our light-weight DCFNet, adds the neck LFDPN for better feature fusion, and keeps the powerful decoder DFINETransformer. The structure of the DL-DEIM architecture is illustrated in Figure 4.

Figure 4. Structure of the DL-DEIM architecture.

As shown in Figure 4, feature maps extracted by the DCFNet backbone are first refined by the DRFD–FasterC3k2 compound block, which jointly balances receptive field diversity and computation. These refined multi-scale features are then propagated to the LFDPN neck for bidirectional diffusion and cross-scale fusion. Finally, the fused representations are passed through the DFINETransformer decoder for detection. The LAWDown module is embedded between LFDPN stages as an adaptive downsampling operator to maintain spatial consistency. This pipeline ensures smooth information flow from shallow to deep layers while minimizing information loss for small-object detection.

The DL-DEIM follows a clear top-down flow: features are first extracted by the DCFNet backbone, where each stage combines a DRFD unit with a FasterC3k2 refinement block. The resulting multi-scale feature maps (p3–p5) are then passed into the LFDPN neck, which diffuses and fuses semantic and spatial information bidirectionally. Finally, the fused representations are decoded by the DFINETransformer head to produce bounding box and confidence predictions. This structure ensures a consistent data stream and enables interpretable interaction between modules.

2.4. DCFNet Backbone

As a fundamental contribution, we propose a new kind of backbone: DCFNet architecture, in sharp contrast to the computational complexity of HGNetv2, the DCFNet we develop is a lightweight architecture yet preserves the strong feature extraction ability with dramatically less computational cost. The backbone adopts a progressive feature extraction strategy in four hierarchical stages, producing multi-scale representations at resolutions ${\displaystyle \frac{H}{4}}\times {\displaystyle \frac{W}{4}}$, ${\displaystyle \frac{H}{8}}\times {\displaystyle \frac{W}{8}}$, ${\displaystyle \frac{H}{16}}\times {\displaystyle \frac{W}{16}}$ and ${\displaystyle \frac{H}{32}}\times {\displaystyle \frac{W}{32}}$, with the feature levels P2, P3, P4, and P5 for their levelsP2, P3, P4 and P5 respectively.

2.4.1. DRFD

Given an input feature $\mathbf{X}\in {\mathbb{R}}^{B\times C\times H\times W}$, the DRFD [28] block performs downsampling through three complementary branches—CutD, ConvD, and MaxD—followed by feature fusion. Here, GConv denotes grouped convolution with g groups (depth-wise when $g=C$), and CutD refers to pixel-rearrangement-based downsampling with $1\times 1$ fusion.

CutD Branch—The input is split into four spatially interleaved sub-tensors and fused by a $1\times 1$ convolution:

$$\begin{array}{cc}\hfill \mathbf{C}& =\mathrm{BN}\left({\mathrm{Conv}}_{1\times 1}\left(\mathrm{Concat}\left[\mathbf{X}\right[:,:,0::2,0::2],\mathbf{X}[:,:,1::2,0::2],\mathbf{X}[:,:,0::2,1::2],\mathbf{X}[:,:,1::2,1::2\left]\right]\right)\right),\hfill \end{array}$$

yielding $\mathbf{C}\in {\mathbb{R}}^{B\times C^{\prime }\times \frac{H}{2}\times \frac{W}{2}}$, where $C^{\prime }=2C$.

ConvD and MaxD Branches—For local context preservation, ConvD applies grouped convolutions with strides 1 and 2:

$${\mathbf{X}}_{\mathrm{conv}}=\mathrm{BN}\left({\mathrm{GConv}}_{3\times 3,s=2}^{g=C^{\prime }}\left({\mathrm{GConv}}_{3\times 3,s=1}^{g=C}\left(\mathbf{X}\right)\right)\right),$$

while MaxD captures salient responses via pooling:

$${\mathbf{X}}_{\max }=\mathrm{BN}\left({\mathrm{MaxPool}}_{2\times 2}({\mathrm{GConv}}_{3\times 3,s=1}^{g=C}\left(\mathbf{X}\right))\right).$$

The three outputs are concatenated and fused through a $1\times 1$ convolution:

$$\mathbf{Y}={\mathrm{Conv}}_{1\times 1}\left(\mathrm{Concat}[\mathbf{C},{\mathbf{X}}_{\mathrm{conv}},{\mathbf{X}}_{\max }]\right)\in {\mathbb{R}}^{B\times C^{\prime }\times \frac{H}{2}\times \frac{W}{2}}.$$

The three-branch design jointly preserves fine details (CutD), spatial coherence (ConvD), and high-response regions (MaxD), effectively mitigating information loss from conventional strided downsampling with only marginal computational overhead. The architecture of the DRFD module is illustrated in Figure 5.

Figure 5. Architecture of the DRFD module.

2.4.2. FasterC3k2

As illustrated by the orange arrows in Figure 4, each DRFD module is immediately followed by a FasterC3k2 block, forming a compound downsampling–refinement unit. The DRFD stage performs multi-branch feature reduction, generating a compact yet information-rich representation $\mathbf{Y}\in {\mathbb{R}}^{B\times C^{\prime }\times \frac{H}{2}\times \frac{W}{2}}$, where $C^{\prime }$ is typically set to $2C$. This output serves as the input to the FasterC3k2 block for efficient channel-wise refinement. Mathematically, the combined operation can be expressed as

$$\begin{array}{cc}\hfill {\mathbf{Y}}_{\mathrm{DRFD}}& ={\mathcal{F}}_{\mathrm{DRFD}}\left(\mathbf{X}\right),\hfill \\ \hfill {\mathbf{Z}}_{\mathrm{out}}& ={\mathcal{F}}_{\mathrm{FasterC}3\mathrm{k}2}\left({\mathbf{Y}}_{\mathrm{DRFD}}\right)=\mathrm{Concat}\left[\mathrm{Conv}\left({\mathbf{Y}}_{\mathrm{DRFD},1:\frac{C^{\prime }}{n_{\mathrm{div}}}}\right),{\mathbf{Y}}_{\mathrm{DRFD},\frac{C^{\prime }}{n_{\mathrm{div}}}:C^{\prime }}\right],\hfill \end{array}$$

where $n_{\mathrm{div}}=4$ denotes the channel division ratio used to perform partial convolution for efficiency.

This joint design allows DRFD to capture and preserve multi-scale contextual information while FasterC3k2 selectively refines and mixes feature channels with minimal computational overhead.

The compound structure ensures a balanced trade-off between receptive-field diversity and model compactness, leading to improved detection accuracy without increased complexity.

Following each DRFD module, the C3k2_Block incorporates Faster_Block components for efficient feature processing. The Faster_Block employs partial convolution strategies, processing only a fraction of channels ($n_{div}=4$) to reduce computational cost while maintaining representational capacity:

$${\mathbf{F}}_{out}=\mathrm{Concat}[\mathrm{Conv}\left({\mathbf{F}}_{1:C/4}\right),{\mathbf{F}}_{C/4:C}].$$

(1)

The architecture of the FasterC3k2 module is illustrated in Figure 6.

Figure 6. Architecture of the FasterC3k2 module and details.

2.5. Lightweight Feature Diffusion Pyramid Network (LFDPN)

The LFDPN introduces a novel bidirectional feature propagation mechanism that efficiently diffuses both semantic and spatial information across scales.

2.5.1. Architectural Overview

The LFDPN introduces a novel bidirectional feature propagation mechanism that efficiently diffuses both semantic and spatial information across scales. The network architecture follows a sophisticated encoder-decoder pattern with three main pathways: top-down semantic flow, bottom-up detail enhancement, and lateral connections for feature fusion.

2.5.2. FMetaFormerBlock

Although the transformer block effectively captures long-range dependencies through self-attention, its quadratic complexity with respect to token number makes it inefficient for high-resolution UAV images. To address this limitation, we adopt the FMetaFormer block, which retains the overall transformer-like token-mixing paradigm but replaces the attention operation with a lightweight frequency spatial mixing strategy for improved efficiency and better feature generalization.

The MetaFormer block serves as the foundational computational unit, implementing a generalized transformer-style architecture. The block follows a dual-branch design with residual connections:

$$\begin{array}{cc}\hfill {\mathbf{x}}^{\prime }& ={\mathrm{ResScale}}_1\left(\mathbf{x}\right)+{\mathrm{LayerScale}}_1\left(\mathrm{DropPath}\left(\mathrm{TokenMixer}\left(\mathrm{LayerNorm}\left(\mathbf{x}\right)\right)\right)\right),\hfill \\ \hfill {\mathbf{x}}_{out}& ={\mathrm{ResScale}}_2\left({\mathbf{x}}^{\prime }\right)+{\mathrm{LayerScale}}_2\left(\mathrm{DropPath}\left(\mathrm{MLP}\left(\mathrm{LayerNorm}\left({\mathbf{x}}^{\prime }\right)\right)\right)\right),\hfill \end{array}$$

(2)

where the TokenMixer is FSSA module, and LayerNorm employs generalized normalization across spatial and channel dimensions. The architecture of the FMetaformer module is illustrated in Figure 7.

Figure 7. Architecture of the FMetaformer module.

2.5.3. FSSA

The FSSA module uniquely combines frequency-domain and spatial-domain attention mechanisms to capture both global and local dependencies [29]. The frequency attention branch leverages the fast Fourier transform to operate in the frequency domain:

$${\mathbf{Q}}_f,{\mathbf{K}}_f,{\mathbf{V}}_f=\mathcal{F}\left(\mathbf{x}\right),$$

(3)

where $\mathcal{F}$ denotes the 2D FFT operation. The frequency attention scores are computed using complex-valued operations:

$${\mathrm{Attn}}_f=\mathrm{ComplexNorm}\left({\displaystyle \frac{{\mathbf{Q}}_f{\mathbf{K}}_f^H}{\sqrt{d}}}\odot \mathbf{T}\right),$$

(4)

where ${\mathbf{K}}_f^H$ represents the Hermitian transpose, $\mathbf{T}\in {\mathbb{R}}^{n_h\times 1\times 1}$ are learnable temperature parameters, and ComplexNorm applies specialized normalization:

$$\mathrm{ComplexNorm}\left(\mathbf{z}\right)=\mathrm{Softmax}\left(\mathrm{Re}\right(\mathbf{z}\left)\right)+i·\mathrm{Softmax}\left(\mathrm{Im}\right(\mathbf{z}\left)\right).$$

(5)

The spatial attention branch employs multi-scale depth-wise convolutions:

$$\begin{array}{cc}\hfill {\mathbf{Q}}_s& =[{\mathrm{DWConv}}_{3\times 3}\left(\mathbf{x}\right)\parallel {\mathrm{DWConv}}_{5\times 5}\left(\mathbf{x}\right)]\hfill \\ \hfill {\mathbf{K}}_s& =[{\mathrm{DWConv}}_{3\times 3}\left(\mathbf{x}\right)\parallel {\mathrm{DWConv}}_{5\times 5}\left(\mathbf{x}\right)]\hfill \\ \hfill {\mathbf{V}}_s& =[{\mathrm{DWConv}}_{3\times 3}\left(\mathbf{x}\right)\parallel {\mathrm{DWConv}}_{5\times 5}\left(\mathbf{x}\right)]\hfill \end{array}$$

(6)

$\mathbf{A}\in {\mathbb{R}}^{B\times C\times {\displaystyle \frac{H}{2}}\times {\displaystyle \frac{W}{2}}\times 4}$.

2.5.4. LAWDown

The LAWDown module realizes the attention-guided downsampling operation tailored for aerial imagery. The spatial-adaptive weights are generated dynamically by a global context-aware attention mechanism (average pooling and 1 × 1 convolution). This mechanism overcomes the limitation in traditional fixed-kernel downsampling, where the downsampling strategies can be adapted to the semantic content of input features, leading to high quality of feature preservation in different scenarios. We expanded the upsampled feature maps by channel rearrange operation, and then performed grouped convolution on these feature maps to reduce the dimension of each group, thus we can get even more groups. This architecture improves both feature representation ability and cross-channel semantic correlation maintaining using effective channel interactions. Reconstructing downsampled features from the s1 × s2 regions by softmax weights, LAWDown realizes fine-grained weighted information fusion in spatial dimension. The architecture of the LAWDown module is illustrated in Figure 8.

Figure 8. Architecture of the LAWDown module.

The module computes spatially adaptive weights for each $2\times 2$ region:

$$\mathbf{A}=\mathrm{Softmax}\left(\mathrm{Reshape}\left({\mathrm{Conv}}_{1\times 1}\left({\mathrm{AvgPool}}_{3\times 3}\left(\mathbf{x}\right)\right)\right)\right),$$

(7)

where $\mathbf{A}\in {\mathbb{R}}^{B\times C\times {\displaystyle \frac{H}{2}}\times {\displaystyle \frac{W}{2}}\times 4}$ assigns importance weights. The downsampling process is as follows:

$$\begin{array}{cc}\hfill {\mathbf{X}}_{ds}& =\mathrm{Reshape}\left({\mathrm{GConv}}_{3\times 3,s=2,g=C/16}\left(\mathbf{x}\right)\right)\hfill \\ \hfill {\mathbf{X}}_{out}& ={\mathrm{Conv}}_{1\times 1}\left(\sum _{i=1}^4{\mathbf{X}}_{ds}^{\left(i\right)}\odot {\mathbf{A}}^{\left(i\right)}\right).\hfill \end{array}$$

(8)

2.5.5. FocusFeature

The FocusFeature module performs adaptive multi-scale aggregation across three pyramid levels. The architecture of the FMetaformer module is illustrated in Figure 9. The input features $\{{\mathbf{f}}_1,{\mathbf{f}}_2,{\mathbf{f}}_3\}$ from scales P3, P4, and P5 are as follows:

$$\begin{array}{cc}\hfill {\mathbf{f}}_1^{\prime }& ={\mathrm{Conv}}_{1\times 1}\left({\mathrm{Upsample}}_{2\times }\left({\mathbf{f}}_1\right)\right)\hfill \\ \hfill {\mathbf{f}}_2^{\prime }& ={\mathrm{Conv}}_{1\times 1}\left({\mathbf{f}}_2\right)\mathrm{if}e\ne 1\hfill \\ \hfill {\mathbf{f}}_3^{\prime }& =\mathrm{ADown}\left({\mathbf{f}}_3\right).\hfill \end{array}$$

(9)

Figure 9. Architecture of the FocusFeature module.

The ADown operation combines average pooling and max pooling paths:

$$\mathrm{ADown}\left(\mathbf{x}\right)=[{\mathrm{Conv}}_{3\times 3,s=2}\left({\mathbf{x}}_{1:C/2}\right)\parallel {\mathrm{Conv}}_{1\times 1}\left(\mathrm{MaxPool}\left({\mathbf{x}}_{C/2:C}\right)\right)].$$

(10)

Multi-scale context is captured through parallel depth-wise convolutions with kernel sizes $\mathcal{K}=\{3,5,7,9\}$:

$${\mathbf{F}}_{enhanced}={\mathbf{F}}_{concat}+\sum _{k\in \mathcal{K}}{\mathrm{DWConv}}_k\left({\mathbf{F}}_{concat}\right).$$

(11)

2.5.6. MFM

The MFM module dynamically calibrates contributions from different scales using channel attention [30]. The architecture of the MFM module is illustrated in Figure 10. Given features ${\left\{{\mathbf{f}}_i\right\}}_{i=1}^N$ from multiple sources:

$$\mathbf{w}=\mathrm{Softmax}(\mathrm{MLP}(\mathrm{GAP}(\sum _{i=1}^N{\mathbf{f}}_i))).$$

(12)

Figure 10. Architecture of the MFM module.

2.5.7. RepHMS

The RepHMS module enhances feature representation through hierarchical processing with UniRepLKNet blocks [31]. The architecture of the RepHMS module is illustrated in Figure 11. The module employs cascaded DepthBottleneck blocks for multi-scale feature extraction:

$${\mathbf{y}}_{i,j}={\mathrm{DepthBlock}}_{i,j}({\mathbf{x}}_{i+1}+{\mathbf{cascade}}_j).$$

(13)

Figure 11. Architecture of the RepHMS module.

3. Results

3.1. Experimental Dataset and Experimental Setup

In this section, we have introduced the rationale for our dataset selection and have adopted the standard evaluation metrics described below to objectively quantify model performance under these conditions.

3.1.1. Experimental Dataset

All experiments are carried out on the publicly available VisDrone2019 dataset [32], which was released by the AISKYEYE team at Tianjin University. The dataset contains 6471 training images, 548 validation images, and 1580 testing images captured by unmanned aerial vehicles (UAVs) in various urban and suburban environments.

The VisDrone2019 dataset is challenging in several aspects for object detection algorithms: (1) Wide scale variations objects can vary in size from only a few pixels to hundreds of pixels; (2) complex crowded scenes are included with many object occlusions; and (3) varying viewpoints and angles. The images come from UAV platforms that provide vie wpoints not available in common object detection datasets. The distribution of different object scales in the VisDrone2019 dataset is presented in Figure 12.

Figure 12. Sample images illustrating the diversity and challenges of the VisDrone2019 dataset.

3.1.2. Evaluation Metrics

To conduct a fair comparison of our DL-DEIM model, we employ the common COCO evaluation metrics including the average precision (AP) and the average precision over classes, which is denoted as mAP.

Average Precision (AP): Average precision is calculated by computing the area under the precision–recall curve. For a specific class c and IoU threshold $\theta$:

$$A{P}_c^{\theta }={\int }_0^1{P}_c\left(r\right)dr,$$

(14)

where $P_c\left(r\right)$ represents the precision at recall rate r for class c.

AP@0.5 (AP50): AP50 specifically uses an IoU threshold of 0.5, which is calculated as

$$AP{50}_c={\displaystyle \frac{1}{11}}\sum _{r\in \{0,0.1,...,1.0\}}{P}_{interp}\left(r\right),$$

(15)

where $P_{interp}\left(r\right)$ is the interpolated precision:

$$P_{interp}\left(r\right)=\underset{r^{\prime }\ge r}{max}P\left(r^{\prime }\right).$$

(16)

Mean Average Precision (mAP): The mAP is computed by averaging the AP values across all object categories:

$$mAP@0.5={\displaystyle \frac{1}{C}}\sum _{c=1}^{C}A{P}_c^{0.5},$$

(17)

where C is the total number of object categories (10 for VisDrone2019).

mAP@0.5:0.95: This metric averages the mAP values computed at different IoU thresholds from 0.5 to 0.95 with a step size of 0.05:

$$mAP@0.5:0.95={\displaystyle \frac{1}{10}}\sum _{\theta \in \{0.5,0.55,...,0.95\}}mA{P}^{\theta }.$$

(18)

The choice of IoU thresholds follows the widely adopted COCO evaluation protocol. Specifically, AP@0.5 provides a relatively lenient criterion that emphasizes detection sensitivity, where predicted bounding boxes only need to overlap the ground truth by at least 50%. In contrast, AP@0.95 enforces a much stricter criterion, requiring almost perfect alignment between predicted and ground-truth boxes. By averaging mAP values across thresholds from 0.5 to 0.95 with a step size of 0.05, we obtain a balanced evaluation that simultaneously accounts for both coarse and precise localization. This combination enables a fair comparison with prior works and reflects real-world detection requirements where both recall and precise localization are important.

Evaluation Protocol: All results listed in this paper are based on the VisDrone2019 validation set (548 images). The validation set is used as the main evaluation benchmark, which is consistent with common practices in the VisDrone challenges, due to unavailability of the test set annotations. This is done to allow for a fair comparison with other current state-of-the-art approaches, which also give results on the validation set.

3.1.3. Experimental Setup

The developed DL-DEIM model was realized with the PyTorch deep learning library. Table 1 lists the computational environment for all experiments, which are important for reproducing our results.

Table 1. Experimental environment. The experiment use high-performance GPU, 16 GB memory, for the efficient training of our method, with bigger batch size and more faster convergence.

Parameter	Value
OS	Ubuntu 24.04
CPU	Intel Core i7-13700KF
GPU	NVIDIA RTX 4070 Ti SUPER(16 GB)
CUDA Version	CUDA 12.5
Memory	32 GB
Deep Learning Framework	PyTorch 2.3.0

The training parameters, determined through preliminary tuning on the validation set, are summarized in Table 2.

Table 2. Training parameters used in the experiments.

Parameter	Value
Epoch	160
Batch Size	4
Input Size	$640\times 640$
Initial Learning Rate	0.0004
Weight Decay Coefficient	0.0001
Optimizer	AdamW

3.2. Results and Comparison

3.2.1. Ablation Study

We carried out extensive ablation experiments to verify the effectiveness of each component in our proposed DL-DEIM model using the VisDrone2019 validation set. All performances in the tables below are generated on the 548 validation images unless otherwise indicated. The ablation study is performed to investigate how much each module contributes to the performance improvement.

We first consider the sensitivity of individual components of the LFDPN encoder. Table 3 shows the performance of the FMetaformer block and LAWDown module.

Table 3. Ablation study on LFDPN encoder components evaluated on the VisDrone2019 validation set.

DEIM-n	FMetaformer	LAWDown	mAP@0.5	mAP@0.5:0.95	GFLOPs	Params (M)
✓			30.3	17.1	7.12	3.73
✓	✓		33.9	19.5	10.26	4.46
✓		✓	32.6	18.7	10.28	4.34
✓	✓	✓	34.2	19.7	10.32	4.34

✓ indicates that the feature is supported.

Table 4 presents a comprehensive ablation study examining the contribution of DCFNet, LFDPN, and LAWDown modules both individually and in combination.

Table 4. Detection performance ablation study on the VisDrone2019 validation set. The table demonstrates the synergistic effect of combining different modules.

Model	DCFNet	LFDPN	LAWDown	P (%)	R (%)	0.5 1	0.5:0.95 2	GFLOPs	Params (M)	FPS
Base				38.7	31.5	30.3	17.1	7.12	3.73	311.3
A(+DCFNet)	✓			41.2	33.8	32.0	18.3	7.86	4.10	282.7
B(+LFDPN)		✓		43.5	35.9	33.9	19.5	10.26	4.46	336.3
C(+LAWDown)			✓	41.8	34.2	32.4	18.8	9.85	4.20	295.6
AB	✓	✓		42.8	35.2	33.0	19.5	10.91	4.39	330.1
AC	✓		✓	42.1	34.6	32.5	18.9	9.97	4.54	298.9
BC		✓	✓	44.2	36.4	34.2	19.7	10.32	4.34	300.1
ABC (Ours)	✓	✓	✓	45.6	37.3	34.9	20.0	11.73	4.64	356.0

✓ indicates that the feature is supported. ¹ mAP@0.5. ² mAP@0.5:0.95.

Figure 13 provides a visualization of the ablation studies presented in Table 4.

Figure 13. Analysis of ablation experiments.

We derive several important insights from the ablation study:

• Individual module contributions: All proposed modules contribute positively to detection, and LFDPN achieves the largest individual gain (+3.6% mAP@0. 5).
• Synergy effect: All the three modules combined (DCFNet + LFDPN + LAWDown) lead to their synergistic enhancements, outperforming the best overall results.
• Efficiency performance trade-off: Despite the increase in computational cost from 7.12 to 11.73 GFLOPs and number of parameters from 3.73 M to 4.64 M, the model remains real-time processing (356.0 FPS), which is appropriate for deployment on UAV.

3.2.2. Comparative Experiment

To further validate the effectiveness of our proposed DL-DEIM model, we conducted comparative experiments against different backbone networks and state-of-the-art (SOTA) object detection methods on the VisDrone2019 validation set. The comparisons aim to demonstrate three aspects: (1) the superiority of our proposed DCFNet backbone over other lightweight backbones (Table 5); (2) the per-class detection performance of DL-DEIM across challenging object categories (Table 6); and (3) the overall competitiveness of DL-DEIM compared with the latest detection frameworks (Table 7).

Table 5. Comparison of different backbone networks on the VisDrone2019 validation set.

Backbone	P (%)	R (%)	mAP@0.5	mAP@0.5:0.95	GFLOPs	Params (M)
MobileNetV3-Small [33]	38.7	31.2	26.8	15.2	2.9	1.5
ResNet18 [34]	42.1	34.5	29.6	16.8	11.1	11.2
EfficientNet-B0 [35]	41.5	33.8	28.9	16.4	3.8	4.0
ShuffleNetV2 [36]	37.9	30.6	25.4	14.7	1.3	1.4
DCFNet	41.2	33.8	32.0	18.3	7.86	4.10

Table 6. Per-class AP comparison of different models on the VisDrone2019 validation set.

Model	Ped 1	People	Bicycle	Car	Van	Truck	Tricycle	A-T 2	Bus	Motor	mAP@0.5
YOLOv8n [37]	31.2	29.8	8.4	72.1	37.2	21.3	20.8	10.2	45.6	25.4	30.2
RT-DETR-R18 [38]	30.6	28.9	7.8	70.5	35.8	20.1	19.5	9.8	44.2	18.8	28.6
DL-DEIM (Ours)	34.8	32.3	9.7	75.6	40.1	24.0	23.3	12.5	49.3	12.7	34.9

¹ Pedestrian. ² Awning-Tricycle.

Table 7. Performance comparison of different detection models on the VisDrone2019 validation set.

Model	P (%)	R (%)	mAP@0.5	mAP@0.5:0.95
YOLOv5n [15]	40.2	33.5	28.8	16.2
YOLOv8n [37]	41.8	34.6	30.2	17.3
YOLOv9n [39]	42.5	35.2	31.0	17.8
YOLOv10n [40]	42.1	34.9	30.8	17.6
YOLOv11n [41]	42.3	35.0	32.2	18.1
RT-DETR-R18 [38]	40.1	30.6	28.6	17.3
DFine-n [42]	41.5	33.8	29.7	17.1
DL-DEIM (Ours)	45.6	37.3	34.9	20.0

DCFNet achieves the highest detection accuracy (mAP@0.5 = 32.0%, mAP@0.5:0.95 = 18.3%) among all tested backbones, while keeping the model relatively lightweight (7.86 GFLOPs, 4.10 M parameters). Compared with MobileNetV3-Small and ShuffleNetV2, DCFNet significantly improves accuracy with only moderate computational cost, confirming its suitability for UAV-based applications.

Our DL-DEIM model shows consistent improvements across most object categories. This confirms the effectiveness of LFDPN and LAWDown in handling scale variations and complex backgrounds, which are common challenges in UAV imagery.

Compared with recent YOLO series (YOLOv5n–YOLOv11n), RT-DETR, and redefine regression task in DETRs as fine-grained distribution refinement (DFine [42]), our DL-DEIM achieves the best overall performance (mAP@0.5 = 34.9%, mAP@0.5:0.95 = 20.0%). This indicates that our model surpasses both transformer-based and CNN-based SOTA detectors while still maintaining real-time inference speed, making it highly suitable for UAV-based real-world applications. Figure 14 provides a visualization of the comparison experiment presented in Table 7.

Figure 14. Comparative experimental performance analysis.

3.3. Visualization Analysis

In order to make it more clear to see what improvements we have made to our proposed method, we show overall visualization results between the baseline and our accentuated approaches, as illustrated in Figure 15, Figure 16 and Figure 17. The detection performance, representation of feature maps, and activation map of the feature pyramid network (FPN) during detection are visualized in more details in the following subsections.

Figure 15. Detection results visualization comparison for two samples. (a) Sample 1; (b) Sample 2; (c) detection map (Sample 1); (d) detection map (Sample 2).

Figure 16. Feature-map responses for the same input using the baseline (top and middle) and the improved model (bottom). (a) Sample 1; (b) Sample 2; (c) Sample 3; (d) feature map before improvement (Sample 1); (e) feature map before improvement (Sample 2); (f) feature map before improvement (Sample 3); (g) feature map after improvement (Sample 1); (h) feature map after improvement (Sample 2); (i) feature map after improvement (Sample 3).

Figure 17. Heatmaps showing salient regions before and after the model improvements. (a) Sample 1; (b) Sample 2; (c) heatmap before improvement (Sample 1); (d) heatmap result before improvement (Sample 2); (e) heatmap result after improvement (Sample 1); (f) heatmap result after improvement (Sample 2).

3.3.1. Detection Comparison

Figure 15 shows the detection results on two exemplar samples of test dataset by our baseline method. The first row (Figure 15a,b) to original input images with the presence of target object in diverse situations and backgrounds. The corresponding detection maps in the bottom row (Figure 15c,d) visualizes the detected outputs of the model where the regions detected along with the confidence levels of detections are highlighted. These detection maps imply the effectiveness of the model in detecting and localizing objects of interest in complex environments. The brightness of the highlighted areas represents the confidence scores for detection where brighter portions indicate higher confidence of objects being present. The visualization shows that the baseline model can detect dominant objects, but there is space for it to be improved in the detecting precision and boundary clarity of components. These base results will be used as a baseline for comparison with our further model improvements and give first indications on the detection capabilities in the baseline setup.

3.3.2. Featuremap Comparison

We demonstrate the effectiveness of our enhancements by comparing the activation featuremaps before and after model enhancements. This consistency is shown in Figure 16 for three different samples. The maps of spatial transformed featuremaps in the original model (Figure 16d,f) contain more scattered attention patterns (i.e. activations are spread over various regions including background). Conversely, the feature maps of the enhanced model maps (Figure 16g,i) are due to noticeably much more focused and localized activation profiles. The improved model has stronger localizing ability that the activations are most focused on the object and noise pixels have been well suppressed. This enhancement of spatial attention indicates that the proposed modifications help the model to learn to be more discriminative and to have a better understanding of object boundaries. The compact and spiky activation regions in the better-refined model indicate better selectivity in feature and diminishment of uncertainty in target localization.

3.3.3. Heatmap Comparison

The last visualization plots the true heatmap results of the original and pruned models. Figure 17 demonstrates two challenging examples in which the superiority of the heatmaps is clear. The heatmaps of original model’s results (Figure 17c,d) have some drawbacks the missed heatmap, the inaccurate localization of the bounding box, and the occasionally false positive. In contrast, the results of the enhanced model (Figure 17e,f) are visibly significant if not visually significant in terms of detection quality. The bounding boxes obtained from the enhanced model are more tightly fitted to target objects with higher confidence scoring for true positives and less mis-detected ones. These advantages are particularly obvious in complicated scenes with a cluttered background or an occluded object. The visualization demonstrates that the improvements contributed by our proposed methods are not only reflected in the quantitative numbers but can be demonstrated in the quality of the heatmap results, which is more consistent for practical application. Both the decrease of false positive and the increase of the localization accuracy verify that our approach does alleviate the weakness in the baseline model.

4. Discussion

The proposed DL-DEIM framework demonstrates clear and consistent improvements over the baseline, achieving 34.9% mAP@0.5 and 20.0% mAP@0.5:0.95 on the VisDrone2019 dataset. These results validate the overall design philosophy that efficiency and accuracy in UAV object detection can be simultaneously enhanced through modular architectural optimization. This section discusses the individual contributions of each component and their synergistic effects on detection performance. An ablation analysis reveals the distinct roles of each major module:

• DCFNet. By integrating the lightweight DCFNet backbone with the DRFD downsampling unit, the framework effectively mitigates information loss that typically occurs when processing small aerial targets. This component alone contributes approximately 1.8% mAP improvement while reducing computation by nearly 30%.
• LFDPN. The bidirectional diffusion and frequency-domain attention in LFDPN enable more efficient semantic exchange across scales, strengthening object localization and recall—particularly for densely distributed small objects (+2.3% mAP).
• LAWDown. The adaptive downsampling strategy dynamically adjusts spatial weighting based on content importance, preserving boundary information and improving precision on small targets (+2.1% mAP).

These findings indicate that each component addresses a complementary aspect of the detection pipeline: DCFNet, LFDPN, and LAWDown. Their joint operation leads to a synergistic performance boost greater than the sum of their isolated effects.

Visualization analysis further supports these quantitative findings. DL-DEIM produces sharper activation maps and more distinct object boundaries compared with FPN-based detectors, effectively suppressing background noise and emphasizing true positives in cluttered aerial scenes. This suggests that the network learns to better separate meaningful object cues from the complex backgrounds that are typical in UAV imagery.

Despite these advances, certain limitations remain. The model still depends on high-resolution inputs, imposing memory constraints that restrict batch size during training. Moreover, the generalizability of the proposed modules to other aerial benchmarks and real-time UAV applications warrants further investigation. Future work will explore model compression, knowledge distillation, and hardware-aware optimization to enhance deployment efficiency without sacrificing detection quality.

5. Conclusions

This paper presents DL-DEIM, a diffusion-enhanced object detection network tailored for aerial imagery in UAV systems. Built upon the foundation of DEIM, DL-DEIM integrates three well-established modules:DCFNet, LFDPN, and LAWDown. DCFNet reduces computational complexity while preserving spatial information, LFDPN improves multi-scale feature fusion, and LAWDown ensures high-quality discriminative representations at lower resolutions. The proposed model achieves a mean average precision (mAP@0.5) of 34.9% and a mAP@0.5:0.95 of 20.0%, outperforming the DEIM baseline by 4.6% and 2.9%, respectively. Additionally, DL-DEIM maintains real-time inference speed (356 FPS) with only 4.64 M parameters and 11.73 GFLOPs, making it highly efficient for deployment in resource-limited UAV systems. Ablation studies demonstrate the critical contributions of each module to both accuracy and efficiency, while visualizations highlight the model’s ability to localize objects in crowded scenes. Despite these advancements, DL-DEIM’s performance may still degrade under extreme conditions, such as highly dynamic scenes or challenging weather environments. Future work will focus on further refining the model to handle these edge cases and explore adaptive strategies for real-time UAV deployment in diverse operational settings.