MPI-DETR: Multi-Grain Prompt and Intensity-Guided Transformer for Small-Object Detection in UAV Imagery

Highlights

What are the main findings?

• We propose MPI-DETR, a novel detection framework utilizing a Dual- Stream Ranked Self-Attention (DRSA) module that maps spatial features into ordered intensity sequences.
• The framework achieves state-of-the-art AP50 scores of 43.8%, 87.5%, and 92.5% on the highly challenging AI-TOD, DIOR, and NWPU VHR-10 datasets, respectively.

What are the implications of the main findings?

• MPI-DETR effectively overcomes severe background clutter and semantic misalignment challenges in UAV imagery through bounded noise filtering and prompt-driven feature fusion.
• It provides a highly competitive and computationally efficient visual perception solution, ideal for resource-constrained UAV platforms and edge deployment scenarios.

Abstract

Small object detection in unmanned aerial vehicle (UAV) remote sensing images remains challenging due to large-scale variations, dense object distributions, and complex background interference. Although Transformer-based detectors have improved global context modeling for remote sensing object detection, many existing designs still rely on spatial geometric relationships and conventional cross-level fusion, which may limit the aggregation of spatially scattered small-object features and introduce background interference. To address these issues, this paper proposes a Multi-granularity Prompt and Intensity Guidance Detection Transformer (MPI-DETR), an efficient end-to-end Transformer-based detector for small-object detection in UAV remote sensing images. MPI-DETR consists of three key components: a Dual-Stream Ranked Self-Attention (DRSA) module for intensity-ordered global feature aggregation, a Bilateral Tanh Gating and Cosine Attention Feature Alignment Module (BTC-FAM) for noise-resistant cross-level alignment, and a Prompt-driven Multi-granularity Fusion (PMGF) module for enhancing weak small-object details. Experiments on AI-TOD, DIOR, and NWPU VHR-10 demonstrate that MPI-DETR achieves $A{P}_{50}$ scores of $43.8\%$, $87.5\%$, and $92.5\%$, respectively. Compared with the RT-DETR-R18 baseline, MPI-DETR improves $A{P}_{50}$ by 3.0, 1.0, and 3.9 percentage points on the three datasets, respectively, and increases $A{P}_{\mathrm{S}}$ by 2.9, 2.9, and 11.6 percentage points. It also surpasses the strongest compared models by 1.7, 1.0, and 1.3 percentage points in $A{P}_{50}$ on AI-TOD, DIOR, and NWPU VHR-10, respectively. These results indicate that MPI-DETR provides a robust and efficient solution for small-object perception in complex UAV remote sensing applications, especially in scenes with dense targets, background interference, and limited computational resources.

1. Introduction

With the rapid development of unmanned aerial vehicle (UAV) remote sensing technology, its applications have become increasingly widespread in areas such as urban traffic monitoring [1,2], disaster emergency response [3,4], precision agriculture management [5,6], and military reconnaissance [7,8]. As one of the core tasks in UAV vision systems, object detection aims to accurately localize and identify objects of interest from high-resolution aerial imagery. However, unlike natural scene images, low-altitude UAV remote sensing images present numerous challenges, including drastic object scale variations caused by the top-down viewing perspective, extremely high object density, and interference from complex background textures [8]. These factors make accurate recognition of small objects within highly variable and cluttered backgrounds an extremely challenging problem.

In recent years, detectors represented by convolutional neural networks (CNNs) have dominated industrial applications due to their efficient inference speed. By constructing deep hierarchical feature pyramids and employing multi-scale fusion strategies, these algorithms have achieved competitive accuracy on general object detection benchmarks in natural scenes. However, due to the inherent limitation of the local receptive field in convolution operations, such models often struggle to capture global contextual information, resulting in relatively high false alarm rates in complex scenarios. To address this issue, Transformer-based detectors introduce global self-attention mechanisms, significantly enhancing the model’s capability to model long-range dependencies and providing a new solution for low-altitude UAV image detection. Nevertheless, the conventional self-attention mechanism they employ excessively relies on geometric neighborhood relationships [9]. While incurring high computational complexity, it assumes that adjacent pixels exhibit stronger correlations. However, in vast aerial scenes, objects of the same category are often spatially distant yet exhibit high consistency in spectral intensity or texture responses [10]. Spatial position-based scanning struggles to effectively capture such physical attribute associations characterized by “spectral similarity despite spatial separation,” leading to inefficient feature aggregation. Furthermore, existing multi-scale feature fusion networks usually adopt simple linear addition or concatenation [11]. During the transmission of deep semantic features to shallow layers, such a “passive” fusion strategy fails to effectively distinguish weak target signals from high-frequency background noise. As a result, the robustness of deep semantic features is often diluted by shallow background clutter, causing the edge information of small objects to gradually vanish during the fusion process.

To address the above challenges, this paper proposes a detection framework, termed MPI-DETR, based on multi-granularity prompting and intensity guidance. Unlike other general-purpose models, MPI-DETR abandons the spatial scanning strategy that relies on geometric neighborhoods and instead constructs an intensity ranking mechanism based on magnitude responses. At the same time, it breaks through the limitations of linear addition that lacks semantic interaction and adopts a semantics-driven prompt retrieval strategy, thereby effectively improving detection accuracy for low-altitude UAV images. Specifically, MPI-DETR consists of three core components. First, a Dual-Stream Ranked Self-Attention (DRSA) module is designed, which maps features into an ordered intensity sequence domain through an intensity reordering strategy. This not only reduces the computational complexity to linear scale, but also enables the model to transcend spatial distance and directly cluster and enhance objects according to their intrinsic physical attributes, such as intensity responses. Second, to address noise interference in the feature flow, we propose a Bilateral Tanh Gating and Cosine Attention Feature Alignment Module (BTC-FAM), which employs a Tanh gating mechanism to construct bounded residuals and dynamically filters high-amplitude background impulse noise during feature transmission. In addition, to resolve the challenge of cross-level feature alignment, we construct a Prompt-Driven Multi-Grain Fusion Module (PMGF). This module treats deep encoded features as “global semantic prompts” and actively retrieves and “activates” small-object textures overlooked by the backbone network in a multi-granularity space, thereby achieving precise complementarity between deep semantics and shallow details.

The main contributions of this paper are summarized as follows:

• We propose MPI-DETR, an end-to-end detection framework specifically designed for small-object detection in UAV remote sensing images. By incorporating intensity guidance and prompt learning mechanisms, the framework effectively alleviates the challenges posed by background clutter interference and the spatially scattered distribution of targets.
• We design the DRSA module, which overcomes the limitations of conventional spatial attention by introducing an intensity reordering strategy and a dual-stream interaction mechanism, thereby enabling efficient global aggregation of features from spatially dispersed targets.
• We propose the BTC-FAM and PMGF modules, which reconstruct the pathways of feature encoding and fusion from the perspectives of noise suppression and cross-level feature alignment, respectively, enhancing the model’s ability to perceive weak target signals.

2. Related Works

2.1. Object Detection in UAV Imagery

Object detection in UAV imagery is closely related to weak target perception in remote sensing scenarios, where targets often exhibit small spatial extent, ambiguous appearance, and strong background interference. In this context, several infrared and hyperspectral target perception studies have provided useful insights into robust appearance representation and background suppression. Zhao et al. [12] improved appearance modeling for infrared target tracking, showing the importance of discriminative target representation under low-contrast conditions. Recent hyperspectral video tracking methods further explored spectral-spatial angle mapping and state-aware template updating [13], spectral difference matching reduction and deep spectral target perception [14], as well as band correlation grouping and spatial-spectral information interaction [15]. In addition, local sub-block contrast and spatial-spectral gradient feature fusion have also been investigated for hyperspectral anomaly detection [16], indicating that spatial-spectral feature enhancement is valuable for identifying weak targets in complex remote sensing scenes. These studies suggest that fine-grained feature representation, background suppression, and cross-region feature interaction are essential for detecting small objects in UAV images.

Although traditional convolutional neural network (CNN)-based detectors have achieved considerable success in general scenarios, they are constrained by local receptive fields, and increasing network depth in UAV image processing can easily lead to the loss of fine-grained details of small objects. In contrast, DETR (Detection Transformer) introduces set prediction and global modeling mechanisms, capturing long-range dependencies. However, despite its strong detection capability, the DETR architecture often suffers from substantial computational overhead and parameter redundancy. To address these issues, current research has mainly focused on core computation lightweighting and multi-scale feature enhancement. Focus-DETR [17] removes redundant background computation and improves detection efficiency by combining a cross-scale foreground selector with a dual-attention mechanism. UAV-DETR [18], from another perspective, reduces model parameters by adopting an inverted residual structure and cascaded linear attention, while extending the receptive field through cross-channel dynamic sampling to preserve small-object details. E²-Former [19] introduces an edge-enhanced Transformer framework for UAV-based small-object detection, emphasizing edge-aware representation, dynamic multi-feature fusion, and fine-grained cross-scale integration under complex low-altitude scenes. To address the challenge of small objects under high-altitude UAV viewpoints, FBRT-YOLO [20] employs a feature complementary mapping module to deeply fuse shallow high-precision spatial location information with deep semantic information, thereby alleviating the loss of small-object features during deep-network downsampling. In addition, to cope with the challenge of dramatic object scale variation in aerial scenes, SED-DETR [21] proposes scale-enhanced deformable attention, which explicitly incorporates scale and shape priors through dynamic dilated sampling, improving the detection accuracy of dense and multi-scale objects in UAV images.

Although the above methods have made important progress in lightweight detection, edge-aware representation, scale-enhanced attention, and multi-scale feature enhancement, two issues remain insufficiently explored for small-object detection in complex UAV imagery. On the one hand, many existing attention or sampling strategies still rely on spatial geometric relationships when modeling feature interactions. This design is effective for neighboring or locally continuous targets, but it may be less efficient in aggregating spatially scattered small objects that share similar visual responses across large background regions. On the other hand, feature fusion modules such as complementary mapping, cross-level aggregation, or dynamic multi-feature fusion can enhance small-object details, but the direct interaction between shallow high-resolution features and deep semantic features may also introduce background interference when target responses are weak. In such cases, the key challenge is not only to fuse multi-level features, but also to selectively enhance target-relevant details while suppressing irrelevant high-frequency background textures. Therefore, this paper explores a detection architecture that combines response-aware global feature reorganization with noise-resistant cross-level feature alignment, aiming to improve small-object perception in UAV scenes with dense distributions and complex background clutter. Compared with closely related DETR-based UAV detectors, Focus-DETR mainly reduces redundant background computation through foreground selection and dual attention, while UAV-DETR improves lightweight modeling by combining inverted residual structures, cascaded linear attention, and dynamic sampling. These methods still organize feature interaction mainly from spatial foreground selection or spatial sampling perspectives. In contrast, DRSA first reorders tokens according to feature response intensity and then performs intra-bucket and inter-bucket interactions in the ranked sequence domain, which provides a different way to associate spatially separated small-object responses.

2.2. Global Context Modeling and Attention Mechanisms

Since the Vision Transformer (ViT) architecture was introduced into visual tasks, the ability of global context modeling has become a key factor in overcoming the bottleneck of object detection in complex scenes. Unlike convolutional neural networks that rely on local receptive fields, the self-attention mechanism can explicitly capture long-range dependencies between any two positions in an image. In recent years, numerous studies have focused on addressing the substantial computational cost of standard self-attention. Zhou et al. [22] proposed an adaptive sparse Transformer, which reduces computational overhead by filtering noisy interactions from irrelevant regions through a dual-branch sparse attention mechanism. Song et al. [23] explored an architecture that efficiently approximates global sparse attention through low-rank approximation, providing a solution to this problem. Chen et al. [24] further investigated an efficient global modeling scheme for Transformer in full-scene remote sensing images, demonstrating the necessity of global perception when handling ultra-high-resolution imagery. In addition, Pu et al. [25], from another perspective, effectively reduced spatial redundancy in multi-step feature interaction by introducing dynamic attention relays.

In the field of remote sensing and UAV image processing, extremely high resolution and complex background textures make efficient context modeling necessary. To address background interference in UAV images, Wang et al. [26] designed a multi-scale network based on dynamic sparse attention, which achieves efficient global feature aggregation while preserving high-resolution local details. To adapt to the arbitrary orientations and irregular distributions of objects in remote sensing images, Lin et al. [27] introduced a variant of deformable attention, enabling the network to adaptively focus on sparse foreground target regions. Beyond conventional RGB-based detection, hyperspectral target perception studies also demonstrate the importance of modeling non-local spatial-spectral relationships. For example, SiamSTU [13] enhances spectral-spatial angle representation for hyperspectral video tracking, while SiamBSI [15] models band correlation and spatial-spectral information interaction to strengthen target discrimination. These studies further indicate that weak target perception benefits from feature interaction beyond local spatial neighborhoods. However, when existing attention mechanisms attempt to match these discrete weak signals across extensive background regions, they not only suffer from low search efficiency but are also highly susceptible to interference from high-frequency background clutter encountered along the way. This misalignment between “spatial physical distance” and “semantic similarity” leads to difficulties in feature aggregation for existing global context modeling methods when handling discrete small objects in UAV imagery.

2.3. Feature Fusion and Pyramid Networks

Multi-scale feature fusion is a core paradigm for addressing the problem of object scale variation in object detection. Starting from the Feature Pyramid Network (FPN) [28] and its variants, cross-level information flow has alleviated the gap between deep semantics and shallow resolution. In recent years, research has focused on breaking static topologies and exploring dynamic feature interaction mechanisms. Cheng et al. [29] proposed a re-parameterized vision-language pyramid network, achieving multi-scale cross-modal feature guidance and fusion. Li et al. [30], in their investigation of the evolution of visual Transformer architectures, pointed out that dynamic and context-aware multi-scale feature aggregation has become crucial for improving the performance of dense prediction tasks.

In remote sensing and UAV vision tasks, given the small-object scale and complex background, the design of feature fusion pathways is critical. To address the problem that small objects are sparsely distributed in high-resolution images and often overwhelmed by the background, Liu et al. [31] proposed an efficient small-object detection framework that performs object searching and partitioning at the feature hierarchy, thereby avoiding ineffective fusion computation in redundant background regions. To cope with semantic confusion caused by multi-scale feature fusion under a large field of view, Zhang et al. [32] designed full-scale feature aggregation and grouped feature reconstruction modules, which actively reduce feature confusion and filter redundant information during cross-level interaction. Recent hyperspectral anomaly detection methods also provide relevant references for noise-resistant feature fusion. Zhao et al. [16] fused local sub-block contrast with spatial-spectral gradient features to strengthen weak anomaly responses. Huo et al. [33] proposed a multi-scale memory network with separation training to improve the discriminability of anomalous and background features. Furthermore, Huo et al. [34] designed a dual-stream background modeling network with anomaly suppression, where local and global branches are combined with attention mechanisms to enhance background modeling and suppress anomaly reconstruction. These works suggest that effective feature fusion should not simply combine multi-level features, but should selectively enhance weak target cues while suppressing background interference.

However, when facing UAV detection tasks involving small and dense objects, existing feature fusion paradigms still suffer from two limitations. On the one hand, most existing networks rely on element-wise addition or channel concatenation. In UAV images, shallow features contain high-resolution contours of small objects but are also filled with high-frequency background noise. Simple linear superposition causes shallow high-frequency noise to contaminate deep semantics, resulting in weak target signals being submerged in background clutter. On the other hand, deep high-level semantic features are often directly broadcast to shallow layers, without actively selecting or activating fine-grained textures in shallow features according to high-level semantics, which leads to “semantic misalignment” during cross-scale alignment.

3. Methods

3.1. Overall Architecture of MPI-DETR

The overall architecture of MPI-DETR is illustrated in Figure 1. Unlike other DETR-based object detection models, the proposed architecture is specifically optimized for UAV remote sensing images, where small-object features are prone to loss and background noise interference is severe. The improved architecture mainly consists of three core components: a backbone network, an Intensity-Prompt Hybrid Encoder, and a decoder. First, the input image is processed by the backbone to extract multi-scale features, denoted as $\{P_3,P_4,P_5\}$. To alleviate the computational redundancy of high-level features in long-range dependency modeling and enhance the perception of spatially scattered targets, we employ the DRSA module to refine deep features. Unlike attention mechanisms based on spatial grids, DRSA performs efficient feature interaction in the sequence domain through global sorting of feature intensities, thereby preserving critical semantic responses while reducing computational complexity. Second, along the multi-scale feature fusion path of the Intensity-Prompt Hybrid Encoder, we embed the BTC-FAM module. This module applies a bilateral gating mechanism together with Tanh-bounded activation to progressively filter deep features layer by layer. Such a design aims to suppress high-frequency noise impulses introduced by complex backgrounds during feature transmission, thereby ensuring the robustness of semantic features. Furthermore, to address the feature misalignment between deep semantics and shallow details, we place the PMGF module between the encoder and the decoder, where it receives shallow features from the backbone and deep outputs from the encoder in parallel. This module treats deep encoded features as “semantic prompts” and guides the selective enhancement of shallow features in a multi-granularity space by computing cosine similarity. Finally, the enhanced features are fed into a Transformer decoder with a fixed set of learnable object queries. The decoder performs query-feature interaction through multi-head attention layers and then uses prediction heads to generate the final object categories and bounding box coordinates. Through the collaborative operation of all modules, the overall architecture improves the detection performance of small and dense objects in UAV scenes while maintaining real-time capability.

3.2. Dual-Stream Ranked Self-Attention

DRSA is designed to efficiently encode global dependencies among dense small objects without relying on local geometric neighborhoods. In low-altitude UAV images, objects of the same category may be spatially scattered but share similar intensity or texture responses, while conventional self-attention is computationally expensive and often inefficient in associating such non-adjacent weak target patterns. To address these issues, we propose the DRSA module and apply it to deep features for efficient global feature encoding. As shown in Figure 2, given an input feature tensor $\mathbf{X}\in {\mathbb{R}}^{B\times C\times H\times W}$, DRSA discards the conventional spatial scanning paradigm based on geometric neighborhoods and adopts an “intensity reordering” strategy. This strategy maps features from the discrete two-dimensional spatial domain into an ordered one-dimensional intensity sequence domain. Specifically, DRSA first projects the input into a query $\mathbf{Q}$, key $\mathbf{K}$, and value $\mathbf{V}$ through linear transformations. Unlike conventional mechanisms, DRSA further introduces a numerical sorting index $\mathcal{I}$. To measure the response intensity of features, we first compute the $L_2$ norm of the value tensor $\mathbf{V}$ along the channel dimension, denoted as ${\parallel \mathbf{V}\parallel }_2$. Then, all feature pixels are globally sorted according to the magnitude of this norm to generate the index $\mathcal{I}$, and a spatial re-indexing operation (Gather) is used to synchronously rearrange $\mathbf{Q}$, $\mathbf{K}$, and $\mathbf{V}$. This operation thereby “clusters” features that are originally spatially dispersed but exhibit similar response intensities:

$$\mathcal{I}=argsort(\parallel \mathbf{V}{\parallel }_2)$$

(1)

$$[{\mathbf{Q}}_{rk},{\mathbf{K}}_{rk},{\mathbf{V}}_{rk}]=Gather([\mathbf{Q},\mathbf{K},\mathbf{V}],\mathcal{I})$$

(2)

where ${\mathbf{Q}}_{\mathrm{rk}}$, ${\mathbf{K}}_{\mathrm{rk}}$, and ${\mathbf{V}}_{\mathrm{rk}}$ denote the reordered feature sequences. On this basis, to capture intensity dependencies at different scales, we construct a dual-stream interaction mechanism. Specifically, we reshape the ordered sequence of length N into a matrix structure of $G\times M$, where M denotes the preset number of elements within each group and $G=\lceil N/M\rceil$ denotes the resulting number of groups. Two complementary views are then constructed:

• Intra-bucket dense stream (${\mathbf{\Phi }}_{\mathbf{intra}}$): Through consecutive grouping, attention is focused on adjacent elements in the sequence. Since the sequence has already been sorted, this stream concentrates on modeling the continuity of local fine-grained intensity variations.
• Inter-bucket sparse stream (${\mathbf{\Phi }}_{\mathbf{inter}}$): Through strided sampling, long-range connections are established across different intensity levels. This stream is intended to capture global statistical distribution relationships across hierarchical levels.

Both streams compute attention weights using smoothed exponential normalization. Formally, for the m-th stream ($m\in \{\mathrm{intra},\mathrm{inter}\}$), we first compute its attention affinity matrix ${\mathbf{A}}^{\left(m\right)}$, and then obtain the intermediate output feature ${\mathbf{H}}^{\left(m\right)}$ by weighted aggregation of the reordered value vectors:

$${\mathbf{A}}^{\left(m\right)}={\displaystyle \frac{exp(\tau ·{\mathbf{Q}}_{\mathrm{rk}}{\left({\mathbf{K}}_{\mathrm{rk}}\right)}^{\top })}{\sum exp(\tau ·{\mathbf{Q}}_{\mathrm{rk}}{\left({\mathbf{K}}_{\mathrm{rk}}\right)}^{\top })+1}}$$

(3)

$${\mathbf{H}}^{\left(m\right)}={\mathbf{A}}^{\left(m\right)}{\mathbf{V}}_{\mathrm{rk}}$$

(4)

where $\tau$ denotes a learnable temperature coefficient, and the constant term 1 in the denominator is introduced to ensure numerical stability and suppress the gradient response of non-salient features. In this manner, DRSA obtains ${\mathbf{H}}^{\left(\mathrm{intra}\right)}$, which represents local intensity consistency, and ${\mathbf{H}}^{\left(\mathrm{inter}\right)}$, which represents the global intensity distribution. Subsequently, DRSA fuses the dual-stream features through a mutual gating mechanism, in which the two branches calibrate each other’s response intensities via element-wise multiplication. Finally, to restore the spatial semantics of the features, the inverse sorting index ${\mathcal{I}}^{-1}$ is used to accurately remap the processed sequence back to the original spatial coordinates, yielding ${\mathbf{Y}}_{\mathrm{out}}$:

$${\mathbf{Y}}_{\mathrm{out}}=Scatter({\mathbf{H}}^{\left(\mathrm{intra}\right)}\odot {\mathbf{H}}^{\left(\mathrm{inter}\right)},{\mathcal{I}}^{-1})$$

(5)

where ⊙ denotes element-wise multiplication, and $Scatter(·)$ denotes the inverse mapping operation. It is worth noting that although DRSA introduces a global sorting operation, it remains hardware-friendly in terms of both computational overhead and gradient propagation. In terms of time complexity, the computational complexity of standard multi-head self-attention is $\mathcal{O}\left(N^{2}C\right)$. Here, $N=H\times W$ denotes the number of spatial tokens in a feature map with resolution $H\times W$, and C denotes the channel dimension. For simplicity, the batch size and the number of attention heads are omitted in the asymptotic notation because they introduce only constant scaling factors. In DRSA, the response score used for sorting is computed from the $L_2$ norm of $\mathbf{V}$ at each spatial position, which requires $\mathcal{O}\left(NC\right)$ operations. The global sorting operation is then performed once over the N response scores of each feature map to obtain the ranking index $\mathcal{I}$, leading to a theoretical complexity of $\mathcal{O}(NlogN)$. After sorting, the ordered sequence is divided into G groups with a preset group size M, where $G=\lceil N/M\rceil$. This grouping step only partitions the already sorted sequence and does not introduce an additional sorting operation. For the intra-bucket and inter-bucket streams, attention is computed within the grouped tokens rather than over the full $N\times N$ spatial token pairs. Therefore, the grouped interaction cost is approximately $\mathcal{O}\left(G{M}^{2}C\right)=\mathcal{O}\left(NMC\right)$. Since M is a fixed and small grouping hyperparameter in our implementation, this term can be written as $\mathcal{O}\left(NC\right)$ with respect to N. Therefore, the overall time complexity of DRSA is $\mathcal{O}(NlogN+NC)$. For multi-level feature maps, the complexity can be written as ${\sum }_l\mathcal{O}(N_{l}log{N}_l+N_l{C}_l)$, where $N_l=H_l\times W_l$ and $C_l$ denote the number of spatial tokens and channels at the l-th feature level, respectively. This complexity is not strictly linear because of the sorting term $\mathcal{O}(NlogN)$. However, compared with the quadratic token-to-token interaction in standard self-attention, DRSA removes the costly $N\times N$ attention matrix and achieves a strictly sub-quadratic complexity with respect to the number of spatial tokens. In practical high-resolution UAV feature maps, the sorting and grouped interaction operations introduce substantially lower computational overhead than full self-attention, while preserving global feature reorganization. Meanwhile, from the perspective of model optimization, although the sorting operation used to obtain the index matrix $\mathcal{I}=argsort(\parallel \mathbf{V}{\parallel }_2)$ is itself non-differentiable, this property does not hinder the end-to-end training of the network. In the forward propagation of DRSA, the index $\mathcal{I}$ and the inverse index ${\mathcal{I}}^{-1}$ participate in the computation only as static “routing masks” and are detached from the computational graph. During backpropagation, the error gradients are propagated back to the original query, key, and value feature maps ($\mathbf{Q},\mathbf{K},\mathbf{V}$) according to the recorded spatial re-indexing (Gather) and restoration (Scatter) paths. This hard-routing mechanism allows effective gradient propagation through the reordered feature tensors and avoids the memory burden caused by constructing a massive $N\times N$ attention matrix in conventional self-attention. Through this closed-loop process of “sorting–interaction–restoration,” DRSA reduces the quadratic dependency on the number of spatial tokens to a sub-quadratic form of $\mathcal{O}(NlogN+NC)$, while enabling the model to aggregate features according to response-intensity ordering rather than only local spatial neighborhoods. As a result, even in complex UAV scenes with severe background clutter and weak object responses, the model can more effectively capture discriminative small-object features across spatially scattered regions.

3.3. Bilateral Tanh Gating and Cosine Attention Feature Alignment Module

In object detection for UAV remote sensing images, as network depth increases, the features of small objects often face the dual challenges of high-frequency information loss and background noise interference. In the baseline detector, the activation maps in complex urban scenes tend to respond not only to target regions but also to building edges, road textures, and tree shadows.This observation indicates that cross-level fusion may transmit large-amplitude background responses together with weak target details. Therefore, the goal of BTC-FAM is not to fundamentally eliminate all background noise, but to mitigate such interference by bounding residual perturbations and aligning cross-level semantic responses. Specifically, BTC-FAM consists of a Bilateral Tanh Gating block (BTG Block) for local feature denoising and a cosine cross-attention mechanism for global semantic alignment. As shown in Figure 3, the preceding BTG Block adopts a bilateral nonlinear modulation mechanism to process the input feature ${\mathbf{X}}_{\mathrm{in}}\in {\mathbb{R}}^{C\times H\times W}$. Specifically, the BTG Block first decouples the input feature into two orthogonal feature streams, ${\mathbf{X}}^{\left(1\right)}$ and ${\mathbf{X}}^{\left(2\right)}$, through linear mapping. To introduce nonlinear constraints into the deep feature space, we define a bounded residual modulation mechanism. For the k-th feature stream ($k\in \{1,2\}$), the feature update process can be formulated as:

$${\mathbf{H}}^{\left(k\right)}={\mathbf{X}}^{\left(k\right)}+tanh\left({\mathcal{D}}_k({\mathbf{X}}^{\left(k\right)};{\mathrm{\Theta }}_k)\right)$$

(6)

where ${\mathcal{D}}_k(·)$ denotes a depthwise convolution transformation with spatial inductive bias, and ${\mathrm{\Theta }}_k$ represents learnable parameters.

In implementation, the two depthwise convolution branches adopt the same initialization strategy to avoid introducing branch preference at the beginning of training. Specifically, the convolutional weights in ${\mathcal{D}}_k(·)$ and the projection layer ${\mathcal{W}}_{\mathrm{out}}$ are initialized using the standard Kaiming initialization, while the bias terms are initialized to zero. No branch-specific manual initialization or freezing operation is adopted. The $tanh(·)$ function serves as a saturated nonlinear function. Unlike the unbounded ReLU, this saturation property constrains the residual perturbation of features within the interval $[-1,1]$, thereby flexibly suppressing large-amplitude noise fluctuations in the background while enhancing high-frequency details. During training, the two gating branches are optimized jointly with the whole detection network through end-to-end backpropagation. The gradient of the nonlinear residual branch is modulated by the derivative of the Tanh function, i.e., $1-{tanh}^2(·)$, which naturally bounds the gradient response of large-amplitude activations. Meanwhile, the identity residual connection in Equation (6) provides a direct gradient path for ${\mathbf{X}}^{\left(k\right)}$, preventing the Tanh gating branch from blocking feature propagation. Therefore, the BTG Block does not require an additional optimization strategy; its parameters are updated by the same loss function and optimizer as the rest of the detector.

Subsequently, by exploiting the distributional differences between the two streams, we generate the enhanced feature ${\mathbf{X}}_{\mathrm{enh}}$ through adaptive mutual gating:

$${\mathbf{X}}_{\mathrm{enh}}={\mathcal{W}}_{\mathrm{out}}({\mathbf{H}}^{\left(1\right)}\odot {\mathbf{H}}^{\left(2\right)})$$

(7)

where ⊙ denotes element-wise multiplication, and ${\mathcal{W}}_{\mathrm{out}}$ represents a linear projection used for feature aggregation. For the mutual gating operation, gradients are propagated to the two branches symmetrically through the element-wise multiplication. Specifically, the gradient received by one branch is weighted by the activation response of the other branch, which encourages the two streams to calibrate each other during optimization. This multiplicative interaction mechanism enables the two feature streams to dynamically weight and filter each other’s spatial responses. Based on the denoised feature ${\mathbf{X}}_{\mathrm{enh}}$ obtained from the BTG Block, to address the semantic misalignment caused by variations in feature magnitude, we model the feature alignment process on a unit hypersphere and establish cosine correlations with the reference feature $\mathbf{Y}\in {\mathbb{R}}^{C\times H\times W}$. Formally, this process can be described as cosine reweighting in a normalized feature space. The generation of the attention matrix $\mathbf{A}$ and the feature aggregation formula is expressed as follows:

$$\mathbf{Z}=Softmax\left(\tau ·{\displaystyle \frac{{\mathcal{P}}_q{\left({\mathbf{X}}_{\mathrm{enh}}\right)}^{\top }{\mathcal{P}}_k\left(\mathbf{Y}\right)}{\parallel {\mathcal{P}}_q\left({\mathbf{X}}_{\mathrm{enh}}\right){\parallel }_2{\parallel {\mathcal{P}}_k\left(\mathbf{Y}\right)\parallel }_2}}\right)·{\mathcal{P}}_v\left(\mathbf{Y}\right)$$

(8)

where ${\mathcal{P}}_{q,k,v}$ denotes the embedding projections for the query, key, and value, respectively, and flatten the spatial dimensions $H\times W$ into a sequence of length N; ${\parallel ·\parallel }_2$ denotes the $L_2$ norm; and $\tau$ is a temperature coefficient used to regulate the entropy of the distribution. In this way, BTC-FAM is able to achieve feature reconstruction for small objects within a unified framework, enhancing the robustness of the model in complex UAV scenarios.

3.4. Prompt-Driven Multi-Grain Fusion Module

During cross-level feature fusion in remote sensing images, small-object features are often weakened by background texture interference, and the semantic gap between different feature levels is difficult to eliminate through simple linear concatenation. To address this issue, we propose the PMGF module, which performs prompt-driven multi-granularity fusion by combining a main feature alignment path with parallel local–global attention branches.

As shown in Figure 4, PMGF receives two input features with the same spatial resolution but different channel dimensions, denoted as ${\mathbf{X}}_1\in {\mathbb{R}}^{B\times C_1\times H\times W}$ and ${\mathbf{X}}_2\in {\mathbb{R}}^{B\times C_2\times H\times W}$. The two features are first projected into a unified channel dimension by two independent convolutional projections:

$${\tilde{\mathbf{X}}}_1={\varphi }_1\left({\mathbf{X}}_1\right),{\tilde{\mathbf{X}}}_2={\varphi }_2\left({\mathbf{X}}_2\right),$$

(9)

where ${\varphi }_1(·)$ and ${\varphi }_2(·)$ are $1\times 1$ convolutional projections used for channel alignment. After projection, ${\tilde{\mathbf{X}}}_1$ and ${\tilde{\mathbf{X}}}_2$ have the same size of ${\mathbb{R}}^{B\times C\times H\times W}$. The main fusion path is then formulated as

$${\mathbf{F}}_{\mathrm{main}}={\varphi }_{\mathrm{m}}\left({\tilde{\mathbf{X}}}_1+{\tilde{\mathbf{X}}}_2\right),$$

(10)

where ${\varphi }_{\mathrm{m}}(·)$ denotes the convolutional transformation after feature summation.

To enhance target-relevant details at different spatial granularities, PMGF introduces parallel Local–Global Attention branches with patch sizes $p=2$ and $p=4$. For each aligned feature ${\tilde{\mathbf{X}}}_j$$(j\in \{1,2\left\}\right)$, the local–global attention branch at granularity p is denoted as

$${\mathbf{F}}_j^p={\mathcal{L}}_p\left({\tilde{\mathbf{X}}}_j\right),p\in \{2,4\},$$

(11)

where ${\mathcal{L}}_p(·)$ represents the Local–Global Attention operator with patch size p. Specifically, this operator partitions the input feature into local patches of size $p\times p$, models the interaction between local patch responses and global contextual information, and then restores the enhanced responses to the original spatial resolution. Thus, the $p=2$ branch focuses more on fine-grained local details, while the $p=4$ branch captures relatively larger contextual structures.

The outputs of the main path and the multi-granularity attention branches are then aggregated as

$${\mathbf{F}}_{\mathrm{fuse}}={\mathbf{F}}_{\mathrm{main}}+{\mathbf{F}}_1^2+{\mathbf{F}}_1^4+{\mathbf{F}}_2^2+{\mathbf{F}}_2^4.$$

(12)

Finally, the fused feature is further refined by a lightweight convolutional block with structural re-parameterization:

$${\mathbf{Z}}_{\mathrm{out}}={\varphi }_{\mathrm{o}}\left(RepConv\left({\varphi }_{\mathrm{f}}\left({\mathbf{F}}_{\mathrm{fuse}}\right)\right)\right),$$

(13)

where ${\varphi }_{\mathrm{f}}(·)$ and ${\varphi }_{\mathrm{o}}(·)$ denote the convolutional layers before and after the re-parameterized convolution, respectively. $RepConv(·)$ represents the structural re-parameterization block, which uses a multi-branch structure during training and can be equivalently converted into a single convolutional branch during inference. Through this design, PMGF selectively strengthens small-object details from different feature levels and granularities while maintaining efficient inference.

4. Experiments

4.1. Datasets

To comprehensively evaluate the performance of MPI-DETR in UAV and remote sensing scenarios, we selected three representative datasets: AI-TOD [26], DIOR [35], and NWPU VHR-10 [36]. The selection of these datasets is motivated by their complementary characteristics. AI-TOD contains a large number of extremely small objects with dense distributions, making it suitable for evaluating the small-object localization capability of the proposed method. DIOR covers diverse object categories, complex backgrounds, and significant scale variations, which can be used to assess the multi-class generalization ability and robustness of MPI-DETR in large-scale remote sensing scenes. NWPU VHR-10 consists of high-resolution aerial images with dense object distributions and complex spatial layouts, providing an additional benchmark for evaluating the adaptability of the model under high-resolution remote sensing conditions. Therefore, these three datasets jointly cover key challenges in UAV and remote sensing object detection, including extreme object scale, dense spatial distribution, background interference, and multi-class scene generalization. The details of the three datasets are described as follows.

• AI-TOD: This is an extremely challenging dataset specifically designed for small-object detection. It contains 28,036 images and 700,621 instances across 8 categories. Unlike conventional datasets, the average object size in AI-TOD is extremely small, at only 12.8 pixels, and the proportion of small objects (smaller than 16 pixels) is exceptionally high. We follow the official split, using 14,536 images for training, 4272 for validation, and 9228 for testing.
• DIOR: This is currently one of the largest and most category-rich remote sensing object detection datasets. It contains 23,463 images and 192,472 instances, covering 20 common object categories, such as airplanes, ships, and storage tanks. This dataset exhibits dramatic object scale variations together with complex background textures, including urban areas, ports, and wild fields. It can effectively evaluate the robustness and generalization performance of MPI-DETR when facing high-frequency background noise interference and cross-scale feature alignment. We divide the dataset into training, validation, and test sets with a ratio of 7:1:2.
• NWPU VHR-10: This is a classic high-resolution geospatial object detection dataset. It contains 800 ultra-high-resolution images covering 10 categories. Although the dataset is relatively small in scale, it includes rich high-resolution texture details and dense spatial distributions. The training, validation, and test sets contain 550, 100, and 150 images, respectively. We follow this split for model training and evaluation.

Table 1. Statistical details of the three datasets.

Dataset	Images	Instances	Classes	Resolution	Main Challenge
AI-TOD	28,036	700,621	8	$800\times 800$	Extremely Small Objects (≈12.8 px)
DIOR	23,463	192,472	20	$800\times 800$	Large Scale & Multi-class
NWPU VHR-10	800	3651	10	Variable sizes	High-Res & Dense Dist.

summarizes the statistical details of the above three datasets.

4.2. Experimental Setup

All experiments in this study were conducted on a Linux workstation running Ubuntu 20.04, equipped with four NVIDIA RTX A10 GPUs for accelerated computation. In terms of software, the model was implemented based on the PyTorch 2.1.0 deep learning framework, with Python 3.10 and CUDA 12.2 as the runtime environment. To ensure fairness in comparative experiments, ImageNet-pretrained ResNet-18 [37] was adopted as the backbone network for feature extraction. During training, the AdamW optimizer [38] was employed for parameter updates, with both the initial learning rate and weight decay set to $1\times {10}^{-4}$, and the momentum parameter set to 0.937. The model was trained for a total of 100 epochs with a batch size of 4. In addition, during data preprocessing, to enhance the model’s robustness to drastic scale variations under UAV viewpoints, a data augmentation strategy was adopted, including random photometric distortion to simulate different illumination conditions, as well as multi-scale geometric jittering and random cropping to improve adaptability to objects at different resolutions. These settings effectively ensure a fair experimental protocol for validating the effectiveness of the proposed MPI-DETR architecture. For reproducibility, the main structural hyperparameters of the proposed modules are as follows. In DRSA, the spatial token number is $N=H\times W$, the ranked sequence is divided into a preset group size M, and the group number is $G=\lceil N/M\rceil$. The query, key, and value tensors keep the same channel dimension as the input feature level. In BTC-FAM, the input channels are split into two branches with equal channel dimensions, and the two branches are initialized and updated as described in Section 3.3. In PMGF, the two input features are first aligned to the same channel dimension by $1\times 1$ convolution, and the Local–Global Attention branches use patch sizes $p=2$ and $p=4$.

4.3. Evaluation Indicators

To rigorously evaluate detection performance and ensure fair comparison with state-of-the-art methods, we follow the standard Microsoft COCO evaluation protocol [39]. The mean Average Precision (mAP) is adopted as the primary evaluation metric in this paper, which is computed by averaging AP values over IoU thresholds ranging from 0.50 to 0.95 with a step size of 0.05. Specifically, $A{P}_{50}$ and $A{P}_{75}$ denote the Average Precision at IoU thresholds of 0.50 and 0.75, respectively, while $A{P}_{50:95}$ denotes the mean AP averaged over IoU thresholds from 0.50 to 0.95. Meanwhile, considering the extreme scale variation in UAV images, we particularly emphasize scale-specific metrics, including $A{P}_{\mathrm{S}}$ (area $<{32}^2$ pixels), $A{P}_{\mathrm{M}}$ (${32}^2\le$ area $<{96}^2$ pixels), and $A{P}_{\mathrm{L}}$ (area $\ge {96}^2$ pixels). These three metrics represent the Average Precision for small, medium, and large objects, respectively, and are used to evaluate the model’s detection capability across different object scales. Furthermore, to assess the feasibility of deployment on resource-constrained UAV platforms, we also report the number of parameters (Params), floating-point operations (FLOPs), and inference latency measured by frames per second (FPS). Together, these metrics provide a comprehensive evaluation of detection accuracy, scale adaptability, and computational efficiency for UAV image detection tasks.

4.4. Comparative Experiment

To comprehensively evaluate the detection performance of MPI-DETR, this section compares MPI-DETR with various mainstream methods currently in use. These methods include CNN-based real-time detection models such as the YOLO series and its variant FBRT-YOLO, as well as Transformer-based end-to-end real-time detection models such as RT-DETR [40]. It is worth noting that, since MPI-DETR adopts ResNet-18 as its backbone, only the lightest versions of the compared models are selected in the experimental comparison of this paper. In addition, because the AI-TOD dataset mainly focuses on small objects and lacks large-object instances, the $A{P}_{\mathrm{L}}$ metric is not included in the evaluation on this dataset. Moreover, all compared models are trained according to their officially recommended training protocols rather than manually adjusted settings. The training epochs reported in

Table 2. Detection results of MPI-DETR and other mainstream models on the AI-TOD dataset. The bold represented the best performence of each metric.

Model	Epochs	Param (M)	FLOPs (G)	${\mathit{AP}}_{\mathbf{S}}$	${\mathit{AP}}_{\mathbf{M}}$	${\mathit{AP}}_{50}$	${\mathit{AP}}_{75}$	${\mathit{AP}}_{50:95}$
CNN-Based
YOLO8n [41]	200	3.1	8.9	0.321	0.152	0.386	0.185	0.170
YOLO10n [42]	200	2.8	8.7	0.302	0.148	0.365	0.178	0.164
YOLO11n [43]	200	2.6	6.6	0.325	0.155	0.389	0.188	0.169
YOLO12n [44]	200	2.6	6.6	0.331	0.158	0.393	0.192	0.173
YOLO13n [45]	200	2.4	6.4	0.335	0.160	0.396	0.196	0.175
FBRT-YOLO-N [20]	300	0.9	6.9	0.342	0.165	0.402	0.201	0.179
Transformer-Based
RT-DETR-R18 [40]	120	20.1	58.6	0.345	0.171	0.408	0.205	0.182
RT-DETRv2-R18 [46]	120	20.1	58.6	0.351	0.175	0.415	0.211	0.187
L-OWRT-DETR [47]	100	19.1	54.2	0.348	0.172	0.410	0.207	0.184
D-FINE-N [48]	160	3.7	7.3	0.326	0.162	0.395	0.193	0.173
DEIM [49]	160	3.7	7.3	0.355	0.178	0.421	0.216	0.190
MPI-DETR (Ours)	120	16.8	48.3	0.374	0.185	0.438	0.231	0.202

,

Table 3. Detection results of MPI-DETR and other models on the DIOR dataset. Bold text represents the best performance of each indicator.

Model	Epochs	Param (M)	FLOPs (G)	${\mathit{AP}}_{\mathbf{S}}$	${\mathit{AP}}_{\mathbf{M}}$	${\mathit{AP}}_{\mathbf{L}}$	${\mathit{AP}}_{50}$	${\mathit{AP}}_{75}$	${\mathit{AP}}_{50:95}$
CNN-Based
YOLO8n [41]	200	3.1	8.9	0.225	0.462	0.751	0.824	0.658	0.602
YOLO10n [42]	200	2.8	8.7	0.231	0.456	0.760	0.829	0.668	0.615
YOLO11n [43]	200	2.6	6.6	0.221	0.465	0.771	0.836	0.678	0.619
YOLO12n [44]	200	2.6	6.6	0.227	0.473	0.787	0.841	0.688	0.629
YOLO13n [45]	200	2.4	6.4	0.223	0.474	0.782	0.845	0.686	0.632
FBRT-YOLO-N [20]	300	0.9	6.9	0.208	0.433	0.723	0.792	0.628	0.572
Transformer-Based
RT-DETR-R18 [40]	120	20.1	58.6	0.276	0.517	0.803	0.865	0.705	0.651
RT-DETRv2-R18 [46]	120	20.1	58.6	0.277	0.520	0.791	0.861	0.704	0.647
L-OWRT-DETR [47]	100	19.1	54.2	0.293	0.505	0.792	0.854	0.698	0.643
D-FINE-N [48]	160	3.7	7.3	0.262	0.485	0.768	0.839	0.672	0.635
DEIM [49]	160	3.7	7.3	0.264	0.471	0.763	0.837	0.662	0.623
MPI-DETR (Ours)	120	16.8	48.3	0.305	0.531	0.815	0.875	0.718	0.662

and

Table 4. Detection results of MPI-DETR and other models on the NWPU VHR-10 dataset. Bold text represents the best performance of each indicator.

Model	Epochs	Param (M)	FLOPs (G)	${\mathbf{AP}}_{\mathbf{S}}$	${\mathbf{AP}}_{\mathbf{M}}$	${\mathbf{AP}}_{\mathbf{L}}$	${\mathbf{AP}}_{50}$	${\mathbf{AP}}_{75}$	${\mathbf{AP}}_{50:95}$
CNN-Based
YOLO8n [41]	200	3.1	8.9	0.319	0.526	0.619	0.904	0.623	0.552
YOLO10n [42]	200	2.8	8.7	0.149	0.477	0.582	0.840	0.543	0.512
YOLO11n [43]	200	2.6	6.6	0.322	0.513	0.624	0.896	0.628	0.556
YOLO12n [44]	200	2.6	6.6	0.295	0.484	0.595	0.859	0.591	0.524
YOLO13n [45]	200	2.4	6.4	0.301	0.483	0.594	0.852	0.587	0.529
FBRT-YOLO-N [20]	300	0.9	6.9	0.313	0.472	0.615	0.866	0.555	0.521
Transformer-Based
RT-DETR-R18 [40]	120	20.1	58.6	0.216	0.529	0.662	0.886	0.630	0.570
RT-DETRv2-R18 [46]	120	20.1	58.6	0.234	0.537	0.619	0.894	0.649	0.573
L-OWRT-DETR [47]	100	19.1	54.2	0.267	0.550	0.646	0.906	0.637	0.577
D-FINE-N [48]	160	3.7	7.3	0.283	0.547	0.632	0.904	0.633	0.570
DEIM [49]	160	3.7	7.3	0.275	0.556	0.647	0.912	0.644	0.580
MPI-DETR (Ours)	120	16.8	48.3	0.332	0.576	0.685	0.925	0.662	0.598

follow the original training recipes of the corresponding models, because different detector families adopt different optimization strategies and convergence schedules. For example, CNN-based YOLO detectors generally use longer training schedules with their official learning-rate settings, whereas Transformer-based detectors usually follow different optimization recipes designed for query-based feature interaction and encoder–decoder architectures. Therefore, forcing all methods to use an identical number of epochs may lead to under-training for some models or unnecessary over-training for others, which would not fairly reflect their representative performance. To ensure a fair and transparent comparison, all methods are evaluated under the same dataset splits, input resolution, evaluation metrics, and hardware environment, while their official training schedules are retained. Table 2 presents the test results of MPI-DETR and other mainstream methods on the AI-TOD dataset. MPI-DETR obtains the highest values among the compared methods on this dataset, with an $A{P}_{50}$ of 43.8% and an $A{P}_{\mathrm{S}}$ of 37.4%. Compared with the RT-DETR-R18 baseline, MPI-DETR improves $A{P}_{50}$ and $A{P}_{\mathrm{S}}$ by 3.0 and 2.9 percentage points, respectively. Although its parameter count and GFLOPs are not the smallest compared with extremely lightweight CNN models, MPI-DETR uses fewer parameters and FLOPs than RT-DETR-based Transformer variants, indicating a favorable trade-off between detection accuracy and computational cost among Transformer-based models. The detection results of MPI-DETR and other models on the AI-TOD dataset are illustrated in Figure 5. Meanwhile, Table 3 and Table 4 present the quantitative comparison results of MPI-DETR and other detection models on the DIOR and NWPU VHR-10 datasets, respectively. On the DIOR dataset, MPI-DETR achieves 87.5% and 30.5% in $A{P}_{50}$ and $A{P}_{\mathrm{S}}$, respectively, corresponding to improvements of 1.0–3.8 and 1.2–4.3 percentage points over other Transformer-based models. Figure 6 shows some detection results of MPI-DETR on the DIOR dataset. On the NWPU VHR-10 dataset, MPI-DETR achieves the highest $A{P}_{50}$ and $A{P}_{\mathrm{S}}$ among the compared methods, with gains of 1.3–8.5 percentage points in $A{P}_{50}$ and 1.0–18.3 percentage points in $A{P}_{\mathrm{S}}$. Figure 7 presents some qualitative results of MPI-DETR on the NWPU VHR-10 dataset. The three datasets emphasize different verification aspects. AI-TOD mainly evaluates extremely small and dense object localization, DIOR emphasizes multi-class remote sensing generalization under complex backgrounds and scale changes, and NWPU VHR-10 further examines adaptability on high-resolution aerial images with limited training samples. MPI-DETR obtains consistent $A{P}_{\mathrm{S}}$ gains on all three datasets, with improvements of 2.9, 2.9, and 11.6 percentage points over RT-DETR-R18 on AI-TOD, DIOR, and NWPU VHR-10, respectively. Since $A{P}_{\mathrm{S}}$ directly reflects small-object detection quality, these results further support the main objective of this work beyond the overall $A{P}_{50}$ metric.

Taken together, the above experimental results show that MPI-DETR achieves competitive detection accuracy while maintaining a relatively moderate computational cost among Transformer-based detectors. Compared with RT-DETR-R18, MPI-DETR reduces the number of parameters from 20.1 M to 16.8 M and the FLOPs from 58.6 G to 48.3 G, while improving $A{P}_{50}$ on AI-TOD, DIOR, and NWPU VHR-10 by 3.0, 1.0, and 3.9 percentage points, respectively. These results suggest that the collaborative design of DRSA, BTC-FAM, and PMGF contributes to small-object feature enhancement and cross-level feature alignment in UAV remote sensing images. Therefore, MPI-DETR provides a practical trade-off between accuracy and efficiency for UAV-based small-object detection, especially under complex backgrounds and resource-constrained deployment scenarios.

4.5. Ablation Experiment

To validate the effectiveness of each innovative module in MPI-DETR, we conducted progressive ablation experiments on the AI-TOD dataset. RT-DETR with a ResNet-18 backbone was selected as the baseline model, and all experimental settings for the ablation study were kept the same as those used in the comparative experiments. During the ablation process, the backbone, input resolution, optimizer, training schedule, data split, and evaluation protocol are kept unchanged. The only controlled variable is the insertion or replacement of the corresponding module, so the performance variation can be attributed to the investigated component under the same experimental protocol. The quantitative results are presented in

Table 5. Results of progressive ablation experiments for the core components of MPI-DETR on the AI-TOD dataset. All experiments use ResNet-18 as the backbone network for fair comparison.

Model	DRSA	BTC-FAM	PMGF	Params (M)	FLOPs (G)	${\mathit{AP}}_{\mathbf{S}}$ (%)	${\mathit{AP}}_{\mathbf{M}}$ (%)	${\mathit{AP}}_{50}$ (%)
Baseline	×	×	×	20.1	58.6	0.345	0.171	0.408
+DRSA	✓	×	×	18.5	53.0	0.358	0.176	0.421
+BTC-FAM	✓	✓	×	17.9	50.5	0.366	0.180	0.429
MPI-DETR	✓	✓	✓	16.8	48.3	0.374	0.185	0.438

.

• DRSA: As shown in Row 2 of Table 5, when the spatial attention module in the baseline model is replaced with the proposed DRSA, the $A{P}_{\mathrm{S}}$ of the model improves from 34.5% to 35.8%. More importantly, since DRSA abandons the computationally intensive $\mathcal{O}\left(N^2\right)$ spatial attention and instead adopts a parameter-free intensity sorting mechanism, the number of parameters and FLOPs are substantially reduced. This strongly demonstrates that, in UAV-view imagery, cross-region feature aggregation based on physical response intensity is not only more precise than conventional spatial search, but also more lightweight and efficient.
• BTC-FAM: On top of DRSA, we further embed BTC-FAM into the feature pyramid pathway. The experimental results show that this configuration improves $A{P}_{50}$ by 0.8% and $A{P}_{\mathrm{S}}$ by 0.8%. This performance gain validates our previous theoretical assumption that the high-frequency background noise prevalent in shallow features of UAV images severely interferes with the representation of weak targets. By employing a bilateral tanh gating mechanism, BTC-FAM successfully filters out these impulse noises before cross-level feature propagation, thereby providing a cleaner semantic environment. Meanwhile, because it replaces part of the heavy spatial convolutions, the computational cost of the model is further reduced.
• PMGF: Finally, we integrate PMGF into the network to form the complete MPI-DETR (Row 4 of Table 5). Compared with the previous version, $A{P}_{\mathrm{S}}$ achieves another 0.8 percentage point improvement, reaching the best result of 37.4%, while the model size is reduced to 16.8 M. This result indicates that PMGF changes the traditional passive feature concatenation paradigm by actively retrieving and activating the fine-grained textures of small objects that are submerged in the shallow network through deep global prompts, thereby thoroughly addressing the problem of “semantic misalignment” during cross-level fusion.

From a reverse-ablation perspective, comparing the complete MPI-DETR with the adjacent incomplete variants also indicates the independent contribution of each component. Removing PMGF from the complete model reduces $A{P}_{\mathrm{S}}$ from 37.4% to 36.6% and $A{P}_{50}$ from 43.8% to 42.9%. Removing BTC-FAM from the DRSA+BTC-FAM variant to the DRSA-only variant reduces $A{P}_{\mathrm{S}}$ from 36.6% to 35.8%. Removing DRSA and returning to the original baseline further reduces $A{P}_{\mathrm{S}}$ from 35.8% to 34.5%. This reverse-view analysis is based on the same controlled ablation table and provides a clearer view of the relative contribution of the three modules.

4.6. Necessity Analysis of the Dual-Flow Mechanism Within DRSA

To further investigate the respective roles of the intra-bucket dense stream and the inter-bucket sparse stream in DRSA, we conducted an internal decoupled ablation study of this module on the AI-TOD dataset (as shown in

Table 6. Internal ablation study of the Dual-Stream Ranked Self-Attention (DRSA) on the AI-TOD dataset.

Model Variant	Intra-Bucket Stream	Inter-Bucket Stream	${\mathit{AP}}_{\mathbf{S}}$ (%)	${\mathit{AP}}_{\mathbf{M}}$ (%)	${\mathit{AP}}_{50}$ (%)
Baseline (No DRSA)	×	×	34.5	17.1	40.8
DRSA (Intra-only)	✓	×	35.1	17.3	41.3
DRSA (Inter-only)	×	✓	35.3	17.4	41.6
DRSA (Full Dual-Stream)	✓	✓	35.8	17.6	42.1

). When only the intra-bucket dense stream is used, the model tends to capture the continuity of locally similar intensities, which is beneficial for modeling local contours of small objects but provides limited global span. When only the inter-bucket sparse stream is used, the model can establish response associations across broader spatial ranges, but may lose some local detail continuity. The full dual-stream design obtains the best $A{P}_{\mathrm{S}}$ in this comparison. Considering that the difference between the inter-only variant and the full dual-stream model is 0.5 percentage points in $A{P}_{\mathrm{S}}$, we interpret this result as supportive evidence rather than statistical proof of complementarity. Repeated trials and formal significance tests will be further explored in future work.

4.7. Visualization of Discriminative Regions

To more intuitively investigate the internal feature representation mechanism of MPI-DETR when handling complex UAV images, we employed the Grad-CAM [50] technique to visualize heatmaps of the deep feature maps before the output of the detection head. As shown in Figure 8, we selected typical aerial scenes with strong background interference and compared the response activation regions of the baseline model and MPI-DETR. It can be observed that, in urban scenes containing large buildings and tree-lined roads, the attention mechanism of the baseline model is severely distracted, with its high-response regions erroneously focusing on the complex textures of the high-rise buildings on the left, resulting in “semantic misalignment.” In contrast, MPI-DETR successfully separates the background noise of the buildings and precisely anchors the attention weights on the small vehicles on the road. This phenomenon visually verifies the effectiveness of BTC-FAM in filtering high-frequency background clutter through its bounded residual mechanism. Meanwhile, in densely built residential scenes, because the small targets are highly confused with the background rooftops in visual appearance, the activation regions of the baseline model exhibit an extremely scattered and diffused distribution, with the true target signals being completely submerged by the surrounding environment. By contrast, the heatmap distribution of MPI-DETR demonstrates a highly concentrated spatial pattern, accurately highlighting the dense target group in the lower-left corner. This fully indicates that, benefiting from DRSA, which discards the limitation of spatial neighborhoods, and the active detail retrieval capability of the PMGF module, the model can accurately reconstruct and enhance the feature representation of small targets across extensive interference regions, further confirming the robustness of the MPI-DETR architecture in complex scenes.

4.8. Analysis of Feature Representations in Dual-Stream Ranked Attention

To provide an illustrative analysis of the feature-alignment behavior, we selected a representative image containing multiple spatially scattered objects of the same category from the AI-TOD test set, extracted the attention weights between one target pixel in the deep network (the $Query$ pixel) and all other pixels in the image (the $Key$ pixels), and computed their Euclidean spatial distances.

Table 7. Statistics of average attention weights assigned to Key pixels at different spatial distances from the Query pixel.

Model	Avg. Weights (Dist. 0–20 px)	Avg. Weights (Dist. 50–100 px)	Avg. Weights (Dist. 150–300 px)	Avg. Weights (Dist. > 400 px)
Baseline (Standard Multi-Head Self-Attention)	0.85	0.22	0.08	0.02
DRSA (Intensity-Ranked)	0.65	0.61	0.58	0.55

summarizes the distribution of average attention weights across different spatial distance intervals for this case. Meanwhile, Figure 9 visualizes the attention-distance relationship. In the baseline model, the weights of standard multi-head self-attention (MHSA) show higher responses within local neighborhoods and decrease as the physical spatial distance increases. In contrast, in MPI-DETR equipped with DRSA, the attention weights are more evenly distributed across different distance intervals when the query and key positions share similar response intensity. This case study illustrates the intended behavior of intensity-based reordering, namely associating spatially separated small-object responses beyond local neighborhoods. Since this analysis is based on a representative sample rather than large-scale statistical testing, it is used as qualitative and illustrative evidence rather than confirmatory proof of a universal attention pattern.

To further provide intuitive visual evidence, we directly extracted the intermediate feature maps produced by the internal attention modules of the network, namely the standard MHSA in the baseline and the proposed DRSA, and mapped them into two-dimensional heatmaps for comparison. The visualization results in the $2\times 5$ layout of Figure 10 present two challenging UAV scenes. From the standard MHSA feature maps in the second column, the activated regions show scattered background responses and diffused feature activations. In contrast, the DRSA feature maps in the fourth column present more concentrated responses around annotated small-object regions and weaker responses in many background areas. These visual observations are consistent with the design motivation of DRSA, namely to reorganize features by intensity response and improve the aggregation of spatially separated small-object cues. Overall, the attention-distance example in Table 7 and the feature visualizations in Figure 10 provide intuitive evidence for the behavior of DRSA, while broader statistical validation across more scenes will be considered in future work.

4.9. Robustness and Discussion

In real UAV application scenarios, captured images are often affected by sensor thermal noise or transmission degradation. To verify the structural stability of MPI-DETR under adverse conditions, we conducted a controlled noise interference experiment. Without retraining the model, Gaussian noise with progressively increasing variance ${\sigma }^2\in \{0,0.01,0.05,0.1,0.2\}$ was injected into the test images, and the performance degradation trajectory of the model was recorded. As shown by the interference curves in Figure 11, the detection accuracy of both the baseline model RT-DETR and MPI-DETR decreases as the noise intensity increases. However, MPI-DETR exhibits a more gradual performance degradation rate. Specifically, under severe noise interference (${\sigma }^2=0.10$), the $A{P}_{\mathrm{S}}$ of the baseline model drops from 34.5% to 14.1%, whereas MPI-DETR decreases from 37.4% to 24.2%. This result indicates that the proposed architecture mitigates, rather than completely eliminates, the influence of noise interference. The bounded property of bilateral tanh gating in BTC-FAM can suppress part of the large-amplitude feature perturbations during cross-level feature fusion, thereby reducing the contamination of weak small-object semantics by high-frequency background responses.

4.10. Computational Complexity and Efficiency Analysis

To further validate the deployment feasibility of MPI-DETR on real edge devices, we measured the actual training video memory (VRAM) consumption and inference speed in frames per second (FPS) of both the baseline model and MPI-DETR on a single NVIDIA RTX A10 GPU, as reported in

Table 8. Empirical efficiency comparison of training VRAM and inference speed. Bold text represents the best performance of each indicator.

Model	Params (M)	FLOPs (G)	Training VRAM (GB)	Inference Speed (FPS)
Baseline	20.1	58.6	7.8	115
MPI-DETR (Ours)	16.8	48.3	5.4	132
Relative Change	$-16.4\%$	$-17.6\%$	$-30.8\%$	$+14.8\%$

.

The experimental results show that, by completely removing the massive $N\times N$ spatial similarity matrix in standard MHSA, the DRSA module enables MPI-DETR to reduce the peak training memory consumption from 7.8 GB to 5.4 GB. This low memory dependence substantially lowers the hardware requirements for model training. More importantly, in terms of inference speed, although the model includes a global sorting operation, the computational complexity along the sequence-length dimension is reduced to $\mathcal{O}(NlogN)$, so the slight latency introduced by sorting is offset by the substantial reduction in attention interaction overhead. The measured inference speed of MPI-DETR reaches 132 FPS, representing a 14.8% improvement over the baseline model. This fully demonstrates that the proposed intensity-ranking and bounded-gating mechanisms not only improve model accuracy, but also achieve a substantial breakthrough in low-level hardware execution efficiency, thereby fully satisfying the requirements of real-time UAV video stream detection.

5. Conclusions

To address the core challenges in UAV remote sensing images, including the susceptibility of small-object features to loss, their spatially discrete distribution, and their vulnerability to complex background clutter, we propose MPI-DETR, a lightweight end-to-end detection framework specifically designed for UAV visual perception. The framework incorporates multiple collaboratively designed components. By jointly integrating DRSA, BTC-FAM, and the prompt-driven PMGF, it successfully breaks the spatial geometric limitations of conventional attention mechanisms and effectively overcomes the problems of high-frequency noise contamination and semantic misalignment in cross-level fusion. Comprehensive experiments on three benchmark datasets, namely AI-TOD, DIOR, and NWPU VHR-10, verify the superior performance of MPI-DETR. Compared with existing state-of-the-art methods, MPI-DETR not only consistently pushes the detection performance limit across different scales, especially for small objects, but also achieves significant reductions in model parameters and computational complexity. In addition, the underlying feature visualization and Gaussian noise interference experiments further confirm the strong robustness of the proposed architecture when handling complex backgrounds and adverse imaging conditions, providing a highly competitive and efficient visual perception solution for resource-constrained UAV platforms and edge deployment scenarios in the Internet of Things (IoT). In future work, we will further investigate the deployment efficiency of MPI-DETR on resource-constrained UAV platforms and improve its adaptability to more diverse real-world imaging conditions. In addition, extending the proposed framework to video-based UAV perception and multi-modal remote sensing data will also be explored to further enhance its practical applicability.