MPC-DETR: A Multi-Scale Patch Context Transformer for Small Object Detection in UAV Imagery

Highlights

What are the main findings?

• We develop MPC-DETR, a Multi-scale Patch Context Transformer that incorporates LGAF, DCFI, and PGMFF to improve small-object representation in UAV imagery.
• Experimental results on VisDrone2019 demonstrate the effectiveness of the proposed model, with mAP50 and mAP50–95 reaching 52.5% and 33.3%.

What are the implications of the main findings?

• MPC-DETR effectively tackles the challenges of weak feature responses, fine-grained information loss, dense object distributions, and complex background interference in UAV small-object detection.
• MPC-DETR is suitable for practical UAV visual perception tasks, as it maintains real-time inference and generalizes well across different imaging modalities, including visible-light and infrared UAV datasets.

Abstract

Small-object detection in unmanned aerial vehicle (UAV) imagery remains challenging because target objects often occupy only a few pixels, exhibit weak feature responses, and are easily obscured by complex backgrounds. These aspects significantly limit the effectiveness of end-to-end detection systems. To overcome these limitations and enhance the detection accuracy in challenging UAV settings, this paper proposes MPC-DETR, a Multi-scale Patch Context Transformer that is based on RT-DETR. To begin with, a Local-Global Attention Fusion Module (LGAF) is proposed to capture fine-grained local features and long-range semantic relations of small objects. LGAF enhances feature representation through a lightweight multi-branch synergistic attention mechanism while introducing limited computational overhead. Second, a Dilated Context-Aware Feature Interaction Module (DCFI) is proposed to enhance the discriminative capability of high-level features in cluttered backgrounds and densely populated small-object scenes. DCFI allows more efficient feature aggregation and contextual comprehension through multi-scale contextual modeling and scale-adaptive feature interaction. Third, a Patch-Guided Multi-scale Feature Fusion Module (PGMFF) is developed to create a patch-guided contextual fusion approach that combines shallow, high-resolution features with deeper semantic information. This process improves the maintenance and representation of fine object information and minimizes information loss in feature propagation. The experimental results on the VisDrone2019 dataset show that MPC-DETR has an mAP50 and mAP50–95 of 52.5% and 33.3%, respectively, which are 4.6 and 4.0 percentage points higher than the baseline model. Further analyses of the UAVDT and HIT-UAV datasets also support the high generalization potential of the suggested method to various UAV-based small-object detection problems. In general, the findings suggest that MPC-DETR provides precise, strong, and efficient small-object detection in complicated UAV images.

1. Introduction

With their ability to operate flexibly, respond quickly to field demands, and acquire imagery over large spatial ranges, unmanned aerial vehicles (UAVs) have become essential data acquisition platforms in remote sensing applications. Object detection using UAVs has found wide application in different applications, including traffic surveillance [1,2], emergency rescue [3,4], agricultural surveillance [5,6], and military surveillance [7,8]. Compared with conventional ground-level imagery, UAV imagery is generally captured from top-down or oblique perspectives. It is characterised by substantial object-scale variation, dense object distributions, background clutter, and frequent occlusion. These properties pose serious problems to proper object detection. Among these challenges, small-object detection is particularly critical in UAV remote sensing. When imaging under high-altitude conditions, targets can be represented by a few pixels and have low texture content, blurred edges, and low contrast with the background. In this study, small objects are quantitatively defined according to the COCO-style scale criterion as instances with a bounding-box area smaller than 32² pixels. Consequently, detection models often fail to learn discriminative enough feature representations, resulting in missed detections, false alarms, and inaccurate localization.

Over the past few years, both CNN-based [9] and Transformer-based [10] object detection methods have made impressive progress in object detection tasks. CNN-based detectors primarily rely on convolutional operations to extract local features, textures, and object boundaries. However, the limited receptive fields of convolution and repeated downsampling in deep networks may cause the loss of fine-grained details. In contrast to conventional detectors, Transformer-based methods use self-attention to model dependencies among spatially distant features. Despite these advantages, their application to small-object detection remains challenging because of the high computational cost. With its streamlined detection pipeline, RT-DETR [11] provides competitive detection accuracy without sacrificing inference efficiency. However, several limitations remain. First, downsampling operations that are repeated in the backbone network are likely to deteriorate the positional and boundary information of small objects. Second, insufficient cross-scale context modeling in the original feature interaction module restricts its capacity to separate small targets from complex backgrounds. Third, shallow fine-grained features are not effectively retained and transferred in the existing multi-scale fusion architecture, which weakens small-object representation. To overcome these challenges, MPC-DETR, a Multi-scale Patch Context Transformer, is introduced for UAV-based small-object detection. The major contributions are as follows:

• A Local-Global Attention Fusion Module (LGAF) is proposed to improve the network’s ability to capture fine-grained local features and long-range semantic dependencies associated with small objects. The module enhances feature representation by adding multi-branch synergistic attention to a lightweight feature extraction framework, without compromising computational efficiency.
• A Dilated Context-Aware Feature Interaction Module (DCFI) is proposed to enhance the discriminative ability of high-level features in complicated backgrounds and high-density small-object conditions. The module does this by using multi-scale contextual modeling and scale-adaptive feature interaction, which allows better aggregation and use of contextual information.
• A Patch-Guided Multi-scale Feature Fusion Module (PGMFF) is designed to establish a patch-guided contextual fusion strategy that combines shallow high-resolution features with deeper semantic representations. This design improves the conservation and expression of small-object fine-grained details during multi-scale feature fusion, and minimizes the loss of information during the process of feature propagation.

2. Related Work

2.1. CNN-Based Methods for Small-Object Detection in UAV Imagery

The evolution of modern object detection has been largely driven by detectors constructed with CNN-based architectures. Ren et al. [12] introduced Faster R-CNN, where an RPN first generates candidate regions, which are subsequently classified and localized through bounding-box regression. Even though Faster R-CNN has high localization accuracy, the two-stage detection pipeline is relatively complex, which restricts inference efficiency. To further enhance the quality of detection, Cascade R-CNN [13] was proposed to gradually increase the IoU thresholds across multiple detection stages, thus improving the quality of detected proposals. The multi-stage architecture, however, presents significant computational complexity and inference overhead. Lin et al. [14] introduced RetinaNet, which adopts Focal Loss to alleviate the severe foreground–background class imbalance in object detection. This design greatly enhances the detection performance and still has a reasonably high inference efficiency. To meet the demand for efficient object detection, the YOLO series [15] and SSD [16] have become common choices in UAV imagery analysis. Although these methods provide faster inference, they remain limited in detecting distant small objects in aerial imagery, partly because of the restricted receptive fields of convolutional operations. To overcome the limitations of small object sizes, high-density distributions of objects, and the interference of background features in UAV imagery, many studies have suggested specialized enhancements to small-object detection. Chen et al. [17] developed DTSSNet by integrating feature refinement and scale-sensitive learning strategies. In particular, an attention-enhanced feature module was developed to enhance object representations at various scales by combining channel information and object-related semantic cues. Wang et al. [18] introduced FLDet, a lightweight detection system that is aimed at enhancing efficiency in complex aerial settings. The method employs a scale-sparse detection head with two branches for multi-scale objects and a partially interactive neck to reduce redundant computations during feature fusion. Tian et al. [19] designed MFDAFF-Net by incorporating multi-scale frequency-domain perception and dual-attention-guided feature fusion. Xiao et al. [20] introduced FBRT-YOLO, incorporating a feature complementary mapping module that combines fine-grained spatial information from shallow layers with high-level semantic representations from deeper layers. By preserving texture, edge, and spatial cues that may be weakened during successive downsampling, this method enhances the detection of small objects.

In general, current CNN-based detection techniques have considerable benefits in the real-time inference efficiency and local feature representation. However, these methods remain inadequate for modeling contextual interactions across distant spatial regions. In addition, as features pass through successive downsampling stages in deep convolutional networks, small-object details such as boundaries, textures, and location-related cues are progressively weakened, making accurate detection more difficult.

2.2. Transformer-Based Methods for Small-Object Detection in UAV Imagery

The high-quality global modeling of Transformers has created new opportunities in detecting small objects. Carion et al. [21] suggested DETR, which removes manually designed elements that are typically present in traditional detectors. In order to reduce computational complexity, RT-DETR provides an efficient object detection framework by streamlining the overall detection pipeline within a unified architecture. Based on these developments, other researchers have gone on to enhance Transformer-based detectors in terms of feature extraction, multi-scale interaction, and adaptation to UAV-specific scenarios. Zheng et al. [22] proposed Focus-DETR, which employs a cross-scale foreground selection mechanism to filter out large amounts of redundant background information from high-resolution feature maps. This, together with a dual-attention architecture to model candidate foreground regions, makes the algorithm less expensive to compute, and more focused on small-object regions by the model. Wang et al. [23] proposed BAP-DETR, which integrates a bi-partition attention processing mechanism with a dual-fusion encoder to reduce semantic inconsistencies during multi-scale feature fusion. Wu et al. [24] introduced ED-DETR, which combines convolutional operations with Transformer architectures to capture multi-scale features and introduce an edge-guided branch to refine the contour and structural representations of small objects. Chen et al. [25] introduced Freq-DETR, which strengthens weak responses from small targets through frequency-domain processing while suppressing interference from complex background textures. Zuo et al. [26] suggested EAV-DETR that explicitly represents arbitrary viewpoints and pose variations in UAV imagery, thus improving the detector’s adaptability to viewpoint variations and spatially imbalanced object distributions. To further resolve the background interference and feature fusion issues, Wang et al. [27] proposed CFP-DETR to address background interference and feature-fusion limitations. Zhang et al. [28] proposed MPI-DETR that uses a dual-stream ranked self-attention mechanism to combine spatially dispersed small-object information based on the intensity of feature responses. Moreover, its prompt-based multi-granularity fusion strategy uses deep semantic features to guide the refinement of shallow features. This strategy suppresses background interference and improves small-object representation. While Transformer-based detectors are well suited for global context modeling, their ability to detect small objects is still affected by several unresolved issues.

Overall, Transformer-based approaches have achieved substantial progress, although several limitations remain. To begin with, most existing methods insufficiently preserve low-level fine-grained features in the initial phases of backbone feature extraction, causing essential discriminative information for small-object representation to be progressively weakened. Second, the majority of current multi-scale feature fusion methods are based on feature concatenation or element-wise addition. Even though these operations make it easier to integrate information at various levels of features, they can also add a lot of background noise to the process of adding shallow features, which can diminish the discriminative ability of small-object representations. Therefore, achieving an effective balance between preserving shallow details and modeling high-level context continues to pose a key challenge for small-object detection.

3. Methodology

3.1. Overview of MPC-DETR

We propose MPC-DETR, a detection framework tailored to small targets in challenging UAV scenes. As shown in Figure 1, MPC-DETR is designed with three major optimization directions, namely enhanced backbone feature extraction, improved feature interaction in the encoder, and more effective multi-scale feature fusion. First, the LGAF is constructed within the backbone network. By incorporating the proposed MSAFormerBlock, LGAF enables the model to better capture fine-grained local textures and global semantic dependencies related to small objects. Second, DCFI is introduced into the encoder to enhance high-level semantic feature interaction through dilated contextual modeling. Lastly, to better preserve small-object details, PGMFF combines high-resolution shallow features with semantically rich deep features. The synergy among these modules enables MPC-DETR to deliver more reliable and accurate detection results in challenging UAV-based aerial environments.

3.2. LGAF

Small objects in UAV imagery are characterised by limited spatial extent, weak texture information, and substantial background interference. When only conventional convolutional operations are used, the model cannot effectively capture long-range contextual dependencies. Consequently, small-object regions may exhibit insufficient semantic responses. We introduce LGAF to enhance small-object representation, as shown in Figure 2. LGAF retains the lightweight multi-branch design of C2f while replacing its Bottleneck units with the proposed MSAFormerBlock. This design maintains fine-grained local information and models long-range dependencies among spatial locations, thus enhancing the representation power of high-level semantic features of small objects.

MSAFormerBlock is built around the proposed Multi-branch Synergistic Attention (MSA) module, whose structure is illustrated in Figure 3. To address the challenges of weak feature responses in UAV imagery, MSA initially splits the input feature map into patches and flattens them into a feature sequence. Positional embeddings are then added to preserve spatial location information. The input feature is simultaneously processed by three local attention branches, namely channel attention, pixel attention, and spatial attention, while the position-enhanced feature sequence is fed into a global multi-head self-attention branch. In particular, the channel-attention branch focuses on informative semantic channels, the pixel-attention branch reinforces fine-grained local responses, and the spatial-attention branch highlights discriminative spatial regions. Meanwhile, the multi-head self-attention branch captures long-range contextual dependencies between spatially distant regions. The outputs of the pixel-attention and channel-attention branches are first aggregated through matrix multiplication and activated by a Sigmoid function. The resulting attention map is then multiplied element-wise with the spatial-attention feature to construct a locally fused representation. Subsequently, this representation is integrated with the multi-head self-attention output and the residual feature through element-wise summation, producing the final output feature.

Overall, MSAFormerBlock enables LGAF to integrate local details, positional information, and global contextual dependencies. The proposed module enhances semantic responses in small-object regions.

3.3. DCFI

To address the shortcomings of the AIFI, including high computational cost, inability to perceive local details, and inability to model multi-scale context in UAV small-object detection, we introduce DCFI, as shown in Figure 4. DCFI primarily consists of Multi-scale Convolutional Attention (MSCA) [29], the proposed Dilated Feed Forward Network (DFFN), and Monte Carlo Attention (MCAttn) [30]. DCFI can boost the fine-grained perception ability of the model in small-object regions by combining multi-scale spatial context modeling, local neighborhood information enhancement, and scale-adaptive attention selection.

In the first stage, the input feature $X$ is processed by MSCA for spatial context modeling. MSCA initially employs a 5 × 5 depth-wise convolution to extract basic local features. Subsequently, three groups of strip depth-wise convolutions with kernel sizes of 1 × 7 and 7 × 1, 1 × 11 and 11 × 1, and 1 × 21 and 21 × 1 are used to construct spatial receptive fields at various scales, thus capturing contextual information ranging from local details to global structures. A 1 × 1 convolution is then applied to generate a spatial attention map, which is utilized to reweight the input feature and obtain the output feature $X_1$. This MSCA-based modeling enhances the responses of small objects and their surrounding informative regions while maintaining relatively low computational overhead.

In the second stage, the feature $X_1$ is fed into DFFN for local contextual enhancement. Unlike a conventional FFN, DFFN introduces dilated convolutions into the feed-forward mapping process to enlarge the receptive field and strengthen the modeling capability for local neighborhood information. The detailed process is formulated in Equations (1) and (2):

$$H_1=\mathrm{GeLU}({\mathrm{Conv}}_{r=1}(\mathrm{LN}(X_1)))$$

(1)

$$X_2={\mathrm{Conv}}_{r=2}(H_1)$$

(2)

where $H_1$ denotes the intermediate feature, $X_2$ denotes the output feature, and $r$ denotes the dilation rate. ${\mathrm{Conv}}_{r=1}(\cdot )$ and ${\mathrm{Conv}}_{r=2}(\cdot )$ denote dilated convolution operations with dilation rates of 1 and 2, respectively. By first modeling information within local neighborhoods and subsequently enlarging the receptive field, DFFN further enriches the contextual information surrounding small objects, thereby providing more comprehensive semantic support for the subsequent attention computation.

In the third stage, MCAttn is introduced for scale-adaptive feature enhancement. In MCAttn, the input feature $X_2$ is first processed by average pooling with three output sizes, namely 1 × 1, 2 × 2, and 3 × 3, to generate multi-scale attention responses, as formulated in Equation (3):

$$B_i={\varphi }_i({\mathrm{AvgPool}}_{i\times i}(X_2)), i\in \{1,2,3\}$$

(3)

where ${\mathrm{AvgPool}}_{i\times i}(\cdot )$ denotes average pooling with an output size of $i\times i$, and ${\varphi }_i\left(\cdot \right)$ represents the operation used to transform the pooled feature into an attention response compatible with $X_2$.

Then, the multi-scale attention responses $B_1$, $B_2$, and $B_3$ are processed differently during training and inference. During training, one attention response is randomly selected from the three scales. The attention response during training is formulated in Equation (4):

$$A_m^{train}\left(X_2\right)=p_1{B}_1+p_2{B}_2+p_3{B}_3$$

(4)

where $p_1$,$p_2$, and $p_3$ are one-hot sampling indicators satisfying $p_i\in \left\{0,1\right\}$ and $p_1+p_2+p_3=1$.

During validation and testing, random sampling is disabled to ensure deterministic and reproducible inference. Instead, the three scale-specific attention responses are averaged, as formulated in Equation (5):

$$A_m^{test}\left(X_2\right)={\displaystyle \frac{1}{3}}\left(B_1+B_2+B_3\right)$$

(5)

In this manner, MCAttn uses stochastic scale selection during training to improve scale robustness, while employing deterministic multi-scale aggregation during inference. This strategy avoids dependence on a fixed pooling scale during training and ensures stable and reproducible predictions during testing. Since no stochastic scale selection is retained during inference, this deterministic aggregation eliminates the run-to-run variance attributable to random scale sampling.

Finally, the selected attention map is used to reweight the input feature, producing the enhanced feature $X_3$, as formulated in Equation (6):

$$X_3=X_2\odot \sigma \left(A_m\left(X_2\right)\right)$$

(6)

where $\odot$ represents element-wise multiplication and $\sigma \left(\cdot \right)$ indicates the sigmoid activation function. In this way, MCAttn avoids relying on a fixed pooling scale and improves the scale adaptability of the feature representation.

In the fourth stage, to further enhance the nonlinear representation capability and local contextual modeling ability of the output features, the MCAttn output feature $X_3$ is fed into the DFFN once again, producing the final output feature $F_{out}$ of DCFI.

In summary, through the coordinated integration of MSCA, DFFN, and MCAttn, DCFI achieves multi-scale spatial context modeling, enhanced local neighborhood information aggregation, and scale-adaptive attention selection. Compared with the original AIFI module, DCFI reduces the computational overhead associated with global self-attention while improving fine-grained feature perception and enhancing adaptability to object-scale variations in UAV imagery.

3.4. PGMFF

In UAV small-object detection, higher-level features contain rich semantic information but lack sufficient spatial details, whereas low-level features preserve abundant fine-grained information but exhibit relatively weak semantic representation. Directly fusing these features may introduce substantial background noise. We propose PGMFF, whose architecture is illustrated in Figure 5. First, SPDConv [31] is applied to the low-level feature $S_2$, thereby reducing the spatial resolution while preserving fine-grained features. Subsequently, the features $S_3$, $S_4$, and $F_5$ are progressively fused. Specifically, $F_5$ is first upsampled and concatenated with $S_4$. The resulting feature is then further upsampled and integrated with $S_3$ and the processed $S_2$ feature. By incorporating low-level detailed information and performing top-down multi-scale feature fusion, PGMFF enhances feature representation for small objects.

To further improve the fused features’ discriminative capability, we propose a Patch-Guided Fusion Block (PGFB) within PGMFF. The core component of PGFB is the Patch-Guided Context Attention (PGCA), whose architecture is displayed in Figure 6. Specifically, let the input feature be represented as $F$. At the initial stage, the channel dimension of the input feature is adjusted using a pointwise convolution, producing the intermediate feature $F_1$, as formulated in Equation (7):

$$F_1=\mathrm{PWConv}(F)$$

(7)

Subsequently, PGCA represents the input feature from three complementary views through local patch perception, global patch perception, and cascaded convolution. The local branch focuses on small patch regions, allowing it to capture edge structures and texture patterns. The global patch-aware branch adopts a larger patch size to incorporate broader contextual information surrounding the objects. Meanwhile, the cascaded convolution branch further enhances local spatial representation capability. The 3 branches’ outputs are denoted as $F_{\mathrm{local}}$, $F_{\mathrm{global}}$, and $F_{\mathrm{conv}}$, respectively, as formulated in Equations (8)–(10):

$$F_{\mathrm{local}}={\mathrm{PA}}_2(F_1)$$

(8)

$$F_{\mathrm{global}}={\mathrm{PA}}_4(F_1)$$

(9)

$$F_{\mathrm{conv}}=\mathrm{SC}(F_1)$$

(10)

where ${\mathrm{PA}}_2$ denotes the local perception branch with a patch size of 2, ${\mathrm{PA}}_4$ denotes the global context perception branch with a patch size of 4, and $\mathrm{SC}(\cdot )$ denotes the cascaded convolution branch composed of multiple 3 × 3 convolution layers.

For each patch-aware branch, the input feature is first split into fixed-size, non-overlapping patch regions. Feature aggregation and attention weight generation are then carried out within individual patches. By locating salient responses inside each patch, this process guides the model toward regions that contain useful small-object cues. The enhanced patch features are then restored to their original spatial positions to obtain scale-specific representations. The various branches’ outputs are combined through element-wise addition to create a multi-branch representation $F_m$ that integrates local details, contextual information, and convolutional spatial features, as defined in Equation (11):

$$F_m=F_{\mathrm{local}}+F_{\mathrm{global}}+F_{\mathrm{conv}}$$

(11)

After obtaining the fused feature $F_m$, PGCA applies the Global Channel-Spatial Attention (GCSA) module for further feature refinement. GCSA optimizes the fused feature in three dimensions, namely, channel relationship modeling, inter-channel information interaction and spatial region enhancement. On the one hand, the module enhances the responses of informative semantic channels and facilitates information exchange across channels. Spatial attention on the other hand is used to highlight areas with small objects and inhibit irrelevant responses of complex backgrounds. Finally, the improved feature $F_g$ is processed by Dropout, BN, and ReLU activation to obtain the final output $F_{\mathrm{out}}$, which is mathematically defined in Equation (12):

$$F_{\mathrm{out}}=\mathrm{ReLU}(\mathrm{BN}(\mathrm{Dropout}(F_g)))$$

(12)

Overall, PGMFF can effectively combine local texture, spatial positional, and global contextual information by adding low-level detailed features, a multi-scale patch-aware mechanism, and GCSA-based attention enhancement, which enhances the feature representation ability in complex aerial scenes.

4. Experiments

4.1. Datasets

As a large-scale benchmark for UAV vision, VisDrone2019 [32] provides images acquired from multiple real-world environments, covering residential, urban road, and rural scenarios. The dataset includes various object categories, including pedestrians, motorcycles, trucks, tricycles, bicycles, cars, and buses, and is defined by a rich set of object categories and a variety of scene types. Due to the relatively high imaging altitude of UAV platforms, the objects are represented by a limited number of pixels, which is why it is a good fit in the research of small-object detection. The official dataset split is used in the experiments, which includes 6471 images to train, 548 images to validate, and 1610 images to test.

UAVDT [2] mainly contains three types of objects, namely buses, trucks, and cars. It has 24,778 training images and 15,598 validation images. Characteristics of the dataset include complicated traffic scenes, large object-scale variations, and harsh occlusions. The high proportion of small targets makes UAVDT suitable for evaluating small-object detection performance.

HIT-UAV [33] primarily contains infrared UAV imagery with weak target thermal responses and strong background interference. These characteristics make it suitable for evaluating aerial small-object detection performance. The dataset contains 2898 images, 2029 of which are training, 290 validation, and 579 testing.

4.2. Experimental Settings

All experiments were conducted on the VirtAI cloud platform using a B1.medium instance equipped with 16 GB of system memory and 12 GB of GPU memory. The software environment included Ubuntu 22.04, Python 3.10, PyTorch 2.0.1, and CUDA 11.8 for GPU acceleration. During training, the number of epochs was set to 200, the batch size was set to 8, and four workers were employed for data loading. AdamW was adopted as the optimizer, with an initial learning rate of 1 × 10⁻⁴ and a weight decay of 1 × 10⁻⁴. All input images were uniformly resized to 640 × 640 pixels.

4.3. Evaluation Metrics

In order to fully assess the detection performance and computational efficiency, GFLOPs, Params, mAP50, mAP50–95, R, and P are used as evaluation metrics. Among these metrics, mAP measures the overall detection performance across all categories, whereas mAP50 and mAP50–95 evaluate detection accuracy at different IoU thresholds. P is the fraction of correctly predicted positive samples out of all samples that are predicted to be positive, and R is the fraction of correctly detected positive samples out of all ground-truth positive samples. Params is a measurement of the size and structural complexity of the model, and GFLOPs is a measure of the cost of inference. AP_i indicates the i-th category’s average precision. The mAP, P, and R are calculated using Equations (13)–(15):

$$\mathrm{mAP}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}A{P}_i$$

(13)

$$P={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}}$$

(14)

$$R={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}}$$

(15)

where FP and TP indicate the number of false and true positives, respectively; and FN denotes the number of false negatives.

To specifically evaluate the detection performance for small objects, we further report the Average Precision for small objects (AP_S) following the COCO evaluation protocol. Specifically, objects with a ground-truth bounding-box area smaller than 32² pixels are regarded as small objects.

4.4. Ablation Experiments

A series of ablation experiments on the VisDrone2019 dataset was designed to examine the influence of the proposed modules on small-object detection in UAV imagery. The LGAF, DCFI, and PGMFF were added to it separately and in combination.

Table 1. Results of the ablation experiments.

Number	LGAF	DCFI	PGMFF	P (%)	R (%)	mAP50 (%)	mAP50–95 (%)	APS (%)	Params (M)	FPS	GFLOPs
1	-	-	-	61.9	46.8	47.9	29.3	18.4	19.8	60	57.1
2	√	-	-	63.1	47.6	49.4	30.6	19.2	22.5	58	60.9
3	-	√	-	62.8	47.4	49.1	30.3	19.0	21.7	59	59.6
4	-	-	√	63.4	48.3	49.8	31.0	19.7	21.0	60	61.8
5	√	√	-	64.0	49.0	50.8	31.6	19.9	24.3	56	64.9
6	√	-	√	64.4	49.5	51.2	31.9	20.4	23.7	56	66.4
7	-	√	√	64.1	49.2	50.9	31.7	20.2	22.9	57	65.6
8	√	√	√	65.0	51.2	52.5	33.3	20.9	25.8	54	70.2

shows the ablation results of various combinations of modules, where √ means that the respective module is used and—means that it is not used.

The single-module results show that LGAF, DCFI, and PGMFF all improve the detection performance of the baseline model. LGAF enhances mAP50 by 1.5 percentage points, from 47.9% to 49.4%. Meanwhile, AP_S increases from 18.4% to 19.2%, representing an improvement of 0.8 percentage points. These results indicate that LGAF effectively improves the backbone’s representation of small objects by integrating fine-grained local details with global semantic information. DCFI enhances mAP50 to 49.1%, a 1.2 percentage point enhancement over the baseline. Its AP_S also increases to 19.0%, indicating that dilated contextual modeling and scale-adaptive feature interaction improve the extraction of contextual information associated with small objects in complex backgrounds. PGMFF is the most significantly improved individual module, with mAP50 improving to 49.8% and R improving to 48.3%. In addition, PGMFF increases AP_S to 19.7%, which is the highest result among the single-module configurations. These findings indicate that the use of shallow high-resolution features and patch-guided multi-scale fusion are effective in reducing the loss of fine-grained small-object information. The complementarity of the suggested components is also evidenced by the dual-module results. The combination of LGAF and DCFI gives a higher mAP50 of 50.8% than either of the configurations with either of the modules. This combination also achieves an AP_S of 19.9%. This indicates that backbone feature improvement and contextual interaction modeling offer complementary feature representation improvements. The combination of LGAF and PGMFF has the highest performance of the two-module settings, with a mAP50 of 51.2% and an R of 49.5%, which suggests that the combination of improved backbone capabilities and shallow high-resolution information is especially useful in small-object localization and recognition. The combination of DCFI and PGMFF results in mAP50 of 50.9%. Meanwhile, its AP_S reaches 20.2%, confirming that contextual information modeling and multi-scale feature fusion jointly improve the discrimination between small objects and complex backgrounds. The model has the highest overall detection performance when LGAF, DCFI, and PGMFF are used together. The full model increases mAP50 from 47.9% to 52.5%, outperforming the baseline by 4.6 percentage points. In the meantime, mAP50–95 rises by 4.0 percentage points. More importantly, AP_S increases from 18.4% to 20.9%. P and R also improve to 61.9% and 46.8% to 65.0% and 51.2%, respectively, which means that both detection accuracy and object retrieval ability are improved. Compared with the baseline, the complete model increases the computational complexity from 57.1 to 70.2 GFLOPs and reduces the inference speed from 60 to 54 FPS; nevertheless, it still maintains real-time inference while achieving substantial improvements in detection accuracy.

4.5. Comparative Experiments

4.5.1. Comparison with Mainstream Detectors

MPC-DETR was compared with mainstream detectors on VisDrone2019 to evaluate its advantages in UAV small-object detection. The compared methods are Cascade R-CNN, Faster R-CNN, RetinaNet, YOLOv11 [34], YOLOv12 [35], YOLOv13 [36] and a number of recently developed UAV object detectors, including RT-DETR, DDQ-DETR [37], DTSSNet, FLDet, MCIA-YOLO [38], MFDAFF-Net, DEIM [39], RTUAV-YOLO [40], UAV-DETR [41], and MSA-DETR [42]. Since MPC-DETR is directly developed from RT-DETR, both models were independently trained ten times using different random seeds under identical experimental settings, and their mean performance and standard deviations are reported in

Table 2. Comparative results on the VisDrone2019 dataset.

Model	P (%)	R (%)	mAP50 (%)	mAP50–95 (%)	APS (%)	Params (M)	FPS	GFLOPs
RT-DETR	61.9 ± 0.3	46.8 ± 0.3	47.9 ± 0.2	29.3 ± 0.1	18.4 ± 0.2	19.8	60	57.1
Faster R-CNN	49.5	36.7	39.2	23.2	9.6	41.2	31	208.4
Cascade R-CNN	50.2	36.4	39.5	23.8	10.2	69.3	26	236.7
RetinaNet	47.7	38.1	36.9	22.1	9.1	36.4	34	210.3
YOLOv11-L	54.0	42.3	43.8	26.7	13.8	25.3	67	95.3
YOLOv12-L	53.6	42.1	43.1	26.2	13.4	26.4	70	85.6
YOLOv13-L	54.2	41.9	43.7	26.5	13.7	27.6	63	80.3
DDQ-DETR	54.0	44.5	44.8	26.1	15.6	41.6	12	86.7
DTSSNet	55.6	43.8	41.1	25.2	13.9	10.1	91	46.8
FLDet-N	50.3	40.5	37.6	22.9	11.6	1.2	56	12.3
MCIA-YOLO	56.4	44.6	42.7	26.1	14.5	7.9	105	39.5
MFDAFF-Net	56.8	45.1	43.9	27.0	15.2	24.9	78	72.6
DEIM	57.2	49.1	50.5	30.9	18.9	19.2	56	80.5
RTUAV-YOLO	52.5	41.1	42.9	25.9	13.8	4.01	94	28.9
UAV-DETR	63.2	50.5	51.6	32.1	19.8	16.8	52	71.4
MSA-DETR	63.9	49.1	52.2	33.2	20.3	18.9	50	79.4
MPC-DETR (Ours)	65.0 ± 0.2	51.2 ± 0.3	52.5 ± 0.2	33.3 ± 0.1	20.9 ± 0.1	25.8	54	70.2

.

Table 2 demonstrates that the performance of conventional two-stage detectors is relatively low in UAV aerial scenes. Cascade R-CNN has mAP50 scores of 39.5, which is significantly lower than the proposed method. Their AP_S values are also relatively low, reaching only 10.2% and 9.6%, respectively, indicating their limited capability to detect small objects in UAV imagery. YOLOv11, YOLOv12, and YOLOv13 achieve relatively high inference speeds, but their detection accuracy remains limited. For example, YOLOv11-L achieves a mAP50 of 43.8% and an AP_S of 13.8%, which is still considerably lower than that of MPC-DETR. This finding suggests that traditional convolution-based feature extraction algorithms might not be able to effectively detect small objects in a complicated background. Compared with the RT-DETR baseline, MPC-DETR improves mAP50 from 47.9% to 52.5%, corresponding to a gain of 4.6 percentage points. Meanwhile, mAP50–95 increases from 29.3% to 33.3%, and AP_S improves from 18.4% to 20.9%, representing gains of 4.0 and 2.5 percentage points, respectively. The consistently small standard deviations obtained across ten independent runs demonstrate that MPC-DETR exhibits stable detection performance under different random initializations. MPC-DETR is also highly competitive in comparison to the recently developed Transformer-based detectors. In particular, it achieves higher mAP50 by 0.9 and 0.3 percentage points over UAV-DETR and MSA-DETR, respectively, which once again confirms its usefulness in detecting small objects using UAVs. Moreover, MPC-DETR achieves the highest P and R among the compared methods, indicating fewer false positives and missed detections. It also achieves the highest AP_S of 20.9%, confirming its superior detection and localization performance for small objects. Compared to other models, MPC-DETR has 25.8 M parameters, requires 70.2 GFLOPs, and has an inference rate of 54 FPS. Its computational cost is higher than that of some lightweight models, but it still has relatively high real-time performance. Overall, the proposed method delivers reliable detection performance with manageable computational cost, indicating its practical value in challenging UAV-based aerial environments.

4.5.2. Cross-Dataset Generalization Experiments

Using RT-DETR as the reference model, we further assessed the generalization performance of MPC-DETR across the UAVDT and HIT-UAV datasets.

Table 3. Performance comparison on the UAVDT dataset.

Model	P (%)	R (%)	mAP50 (%)	mAP50–95 (%)	APS (%)
RT-DETR	54.2 ± 0.3	37.6 ± 0.4	32.5 ± 0.2	16.5 ± 0.2	10.7 ± 0.2
MPC-DETR (Ours)	57.1 ± 0.2	40.6 ± 0.3	35.6 ± 0.2	18.7 ± 0.1	12.6 ± 0.1

and

Table 4. Performance comparison on the HIT-UAV dataset.

Model	P (%)	R (%)	mAP50 (%)	mAP50–95 (%)	APS (%)
RT-DETR	83.1 ± 0.2	71.4 ± 0.3	74.4 ± 0.2	48.0 ± 0.2	39.0 ± 0.2
MPC-DETR (Ours)	88.6 ± 0.2	72.1 ± 0.2	76.4 ± 0.1	50.8 ± 0.1	40.8 ± 0.1

summarize the comparison results. On each dataset, both RT-DETR and MPC-DETR were trained in ten independent runs with distinct random seeds while all other experimental conditions were kept unchanged, and their mean performance and standard deviations are reported in the corresponding tables. On the UAVDT dataset, MPC-DETR achieves an mAP50 of 35.6%, which is 3.1 percentage points higher than the 32.5% achieved by RT-DETR. Meanwhile, R increases from 37.6% to 40.6%, indicating that the proposed approach enhances the ability to detect small objects in complex aerial traffic scenarios. More importantly, AP_S increases from 10.7% to 12.6%, corresponding to an improvement of 1.9 percentage points. These results demonstrate that MPC-DETR retains its advantage in small-object detection on the UAVDT dataset. On the HIT-UAV dataset, MPC-DETR achieves an mAP50 of 76.4%, outperforming the 74.4% achieved by RT-DETR by 2.0 percentage points. Meanwhile, mAP50–95 and AP_S increase from 48.0% to 50.8% and from 39.0% to 40.8%, yielding gains of 2.8 and 1.8 percentage points, respectively. These findings demonstrate that the proposed approach improves both feature representation and localization accuracy for small targets in infrared UAV images. The consistently small standard deviations observed across the ten independent runs demonstrate that MPC-DETR exhibits consistent detection performance across different random initializations. Overall, the consistent improvements achieved by MPC-DETR on both UAVDT and HIT-UAV highlight its strong generalization capability across different UAV detection scenarios and imaging modalities.

4.5.3. Comparison with Deterministic Multi-Scale Feature Routing Methods

To further assess the contribution of the stochastic scale-selection mechanism in MCAttn, we compared it with several deterministic multi-scale feature weighting and routing methods, including equal averaging, iAFF-based attentional fusion [43], DyHead-style scale-aware routing [44], and DPConv-based dynamic scale routing [45]. In all experiments, the complete MPC-DETR architecture was retained, and only the scale-processing component between the two DFFN blocks in DCFI was replaced. The remaining network components, training settings, dataset split, and evaluation protocol were kept unchanged to ensure a fair comparison. Equal averaging assigns identical weights to the three contextual scale responses during both training and inference. The iAFF-based method progressively integrates different scale responses through input-conditioned attention. DyHead-scale predicts normalized importance weights for different contextual scales and performs deterministic soft routing. DPConv dynamically adjusts the contributions of feature branches with different receptive fields according to the input features. For a fixed model and input, all these comparison methods produce deterministic outputs. In contrast, MCAttn randomly selects one contextual scale during training and deterministically averages all scale responses during inference.

As shown in

Table 5. Performance comparison on VisDrone2019 dataset.

Method	P (%)	R (%)	mAP50 (%)	mAP50–95 (%)	APS (%)
Equal averaging	63.7	49.6	51.3	32.0	19.7
iAFF-based	64.1	50.1	51.7	32.4	20.0
DyHead-scale	64.6	50.7	52.1	32.9	20.5
DPConv-based	64.3	50.4	51.9	32.7	20.3
MCAttn	65.0	51.2	52.5	33.3	20.9

, the deterministic routing methods generally outperform equal averaging, indicating that adaptive scale weighting is beneficial for multi-scale feature representation. DyHead-scale performs best, achieving an mAP50 of 52.1% and an APS of 20.5%. In comparison, MCAttn achieves 52.5% mAP50 and 20.9% AP_S, exceeding DyHead-scale by 0.4 percentage points on both metrics. These results indicate that randomly exposing the network to different contextual scales during training can reduce its dependence on a particular scale and improve the representation of small objects without explicitly predicting input-dependent routing weights. To assess inference stability, the same trained MCAttn checkpoint was evaluated ten times under identical settings. Since stochastic scale selection is disabled during validation and testing, no variation was observed in the evaluation metrics, confirming that the stochastic operation in Equation (4) does not introduce inference instability.

4.6. Edge-Device Inference Efficiency Analysis

The computational cost of each proposed module was further assessed on an NVIDIA Jetson AGX Orin (NVIDIA Corporation, Santa Clara, CA, USA) by measuring its contribution to the overall inference latency of MPC-DETR. Each model was compiled into a TensorRT engine and benchmarked in FP16 mode with a batch size of 1 and an input size of (640 × 640). Following 100 warm-up iterations, the mean inference latency was measured across 1000 runs. Image preprocessing and post-processing were excluded to ensure a fair comparison of model-forward latency.

As shown in

Table 6. Module-wise inference latency on the edge computing platform.

Number	LGAF	DCFI	PGMFF	Latency (ms)	Latency Increase (ms)	FPS
1	-	-	-	16.7	-	60.0
2	√	-	-	17.2	0.5	58.1
3	-	√	-	17.5	0.8	57.0
4	-	-	√	17.1	0.4	58.5
5	√	√	√	18.5	1.9	54.0

, LGAF, DCFI, and PGMFF increase the inference latency by 0.5, 0.8, and 0.4 ms, respectively. Among them, DCFI introduces the largest latency increase because of its multi-scale attention and contextual feature-interaction operations, whereas PGMFF produces relatively limited additional overhead. After integrating all three modules, the inference latency increases from 16.7 to 18.5 ms, while MPC-DETR still maintains an inference speed of 54.0 FPS. Although the proposed modules introduce moderate computational overhead, MPC-DETR achieves gains of 4.6, 4.0, and 2.5 percentage points in mAP50, mAP50–95, and APS, respectively. The additional latency is accompanied by improvements in overall and small-object detection accuracy. These results suggest a reasonable accuracy–latency trade-off for UAV edge deployment.

4.7. Visualization Analysis

To qualitatively evaluate the proposed method, three representative aerial scenes were selected from the VisDrone2019 dataset. Figure 7 compares the visual detection performance of different methods, where Figure 7a gives the input images, and Figure 7b and Figure 7c correspond to RT-DETR and MPC-DETR, respectively. The red dashed boxes indicate the regions selected for detailed comparison. The first column presents a long-distance road scene in which the objects are relatively small and some targets are strongly affected by illumination conditions and background interference. RT-DETR exhibits several missed detections of distant pedestrians and vehicles, whereas MPC-DETR successfully detects more small-scale objects, demonstrating its stronger perception capability for distant small targets. The second column shows a dense multi-object scene at a road intersection, where pedestrians, vehicles, and non-motorized vehicles are densely distributed and exhibit partial occlusions and substantial scale variations. RT-DETR tends to miss objects in densely populated regions, whereas MPC-DETR detects small objects more completely in both the road and intersection areas. The third column presents a multi-scale object scene with a complex background. Distant road objects are small and often obscured by trees, buildings, and nearby vehicles. Compared with RT-DETR, MPC-DETR provides more stable detection results for objects at different scales, with a noticeable reduction in missed detections.

To provide a more balanced evaluation of cross-dataset generalization, we further present qualitative results on the UAVDT and HIT-UAV datasets. The red dashed boxes highlight representative challenging regions involving distant objects, dense distributions, occlusion, and low-contrast targets. As shown in Figure 8, the selected UAVDT samples cover a wide intersection, a long-distance urban road, and a nighttime traffic scene. RT-DETR produces several missed detections for distant and densely distributed vehicles, particularly near image boundaries and under low-illumination conditions. In comparison, MPC-DETR detects more small targets and provides more complete detection results across different traffic densities and illumination conditions. Nevertheless, a few extremely small or heavily occluded objects remain difficult to identify, indicating that further improvement is still needed for severe scale degradation and occlusion. Figure 9 presents representative infrared scenes from HIT-UAV, including moving vehicles, densely parked vehicles, and targets partially obscured by surrounding structures. Owing to weak thermal contrast and background interference, RT-DETR misses several small targets. MPC-DETR provides more complete detection results and improves the localization of low-contrast targets.

We further employed Grad-CAM on the VisDrone2019 dataset to visualize and compare the attention regions of RT-DETR and MPC-DETR, thereby validating MPC-DETR’s effectiveness from a feature-response standpoint. The results are shown in Figure 10. As can be observed from Figure 10, the feature response regions of RT-DETR are relatively dispersed, with certain ineffective activations appearing in background regions such as roads, buildings, and trees. In addition, RT-DETR does not focus sufficiently on distant small objects and densely distributed targets. In contrast, MPC-DETR exhibits more concentrated high-response regions and is able to cover target areas more accurately. In particular, in scenes involving distant road targets, dense object distributions, and complex backgrounds, MPC-DETR demonstrates stronger feature attention to small-scale objects. The Grad-CAM results further suggest that the proposed method improves feature discrimination and object localization.

To further evaluate the generalization of the feature-response capability, Grad-CAM visualizations on UAVDT dataset are provided in Figure 11. Figure 11a shows the original images, while Figure 11b and Figure 11c present the activation maps of RT-DETR and MPC-DETR, respectively. The selected samples cover three representative UAV traffic scenes, including a wide intersection, a long-distance road, and a nighttime intersection with dense traffic. As shown in Figure 11, RT-DETR mainly focuses on several visually salient vehicles, while its responses to distant, densely distributed, and marginal small targets are relatively weak or discontinuous. In contrast, MPC-DETR produces more compact and continuous activation regions around vehicles and pedestrians and suppresses irrelevant background responses more effectively. This advantage remains evident under different object distances, traffic densities, and illumination conditions, indicating that MPC-DETR provides more stable feature representations for small objects across different UAV scenes.

5. Discussion

The experimental results indicate that the performance improvement of MPC-DETR is mainly attributed to the complementary modelling of fine-grained details, multi-scale context, and shallow–deep feature interaction. Rather than relying on a single enhancement mechanism, the proposed framework improves small-object representation at different stages of the detection pipeline. The consistent performance gains on VisDrone2019, UAVDT, and HIT-UAV also suggest that these improvements are not limited to a specific dataset or imaging modality. Meanwhile, the edge-device results show that the increased computational cost remains within a real-time operating range, demonstrating a reasonable trade-off between detection accuracy and inference efficiency. Nevertheless, several limitations remain. Extremely small and heavily occluded objects are still difficult to detect because their discriminative visual information may be severely degraded. In addition, the current evaluation does not fully cover variations in weather, motion blur, flight altitude, illumination, and sensor domains. The increased model complexity may also restrict deployment on UAV platforms with limited computing resources. Future work will therefore focus on lightweight model design and improving robustness under more challenging real-world conditions.

6. Conclusions

To address the challenges of dense object distributions, complex background interference, and fine-grained information loss in UAV imagery, this paper proposed MPC-DETR, a small-object detection framework based on RT-DETR. MPC-DETR integrates LGAF, DCFI, and PGMFF to enhance local-global feature representation, multi-scale contextual interaction, and shallow–deep feature fusion. Experimental results on the VisDrone2019 dataset show that MPC-DETR achieves 52.5% mAP50 and 33.3% mAP50–95, outperforming the baseline by 4.6 and 4.0 percentage points, respectively. Experiments on UAVDT and HIT-UAV further demonstrate its generalization capability across different UAV scenes and imaging modalities. In addition, MPC-DETR maintains real-time inference performance, indicating its potential for practical UAV visual perception applications.