RSMamDet: Efficient UAV Remote Sensing Vehicle Detection via Linear State Space Models and Adaptive Multi-Level Feature Fusion

Highlights

What are the main findings?

• RSMamDet achieves state-of-the-art mAP₅₀ of 72.6% on DroneVehicle and 40.2% on VisDrone2019, surpassing prior best methods by 4.1% and 2.2%, respectively, while maintaining real-time inference at 186.2 FPS with only 19.8M parameters.
• Replacing quadratic $\mathcal{O}\left(N^2\right)$ self-attention with linear $\mathcal{O}\left(N\right)$ State Space Model scanning via the Selective Feature Scanning (SFS) module enables efficient global context modeling for high-resolution UAV imagery, reducing encoder FLOPs by approximately 38%.

What are the implications of the main finding?

• The proposed framework demonstrates that linear-complexity global context modeling can simultaneously satisfy both accuracy and real-time requirements for UAV-based vehicle detection, enabling practical deployment on resource-constrained aerial platforms.
• The modular design principles—adaptive cross-scale fusion (DASI), content-aware upsampling (AMFF), and uncertainty-aware training (UMC loss)—can generalize beyond vehicle detection to broader aerial remote sensing tasks such as crowd counting, infrastructure inspection, and disaster damage assessment.

Abstract

Accurate and efficient vehicle detection from unmanned aerial vehicle (UAV) imagery is essential for intelligent transportation, urban monitoring, and public safety, yet this task remains challenging due to high target density, extreme scale variation, complex backgrounds, and stringent onboard computational constraints. Existing DETR-based detectors model global context through self-attention but incur quadratic $\mathcal{O}\left(N^2\right)$ complexity that is prohibitive for high-resolution UAV images, while CNN-based methods lack the long-range contextual awareness needed for dense small-object scenarios. We propose RSMamDet, an efficient end-to-end detection framework built upon RT-DETR that replaces quadratic self-attention with linear $\mathcal{O}\left(N\right)$ State Space Model scanning. The framework integrates a MobileMamba backbone with a Selective Feature Scanning module for efficient global context modeling, a Dimension-Aware Selective Integration module for adaptive cross-scale feature fusion, a Poly Kernel Inception Network encoder for multi-receptive-field feature enrichment, and an Adaptive Multi-Level Feature Fusion module for content-aware dynamic upsampling, complemented by an Uncertainty-Minimal Composite loss for stable query selection in cluttered aerial scenes. Experiments on DroneVehicle and VisDrone2019 demonstrate that RSMamDet achieves mAP₅₀ of 72.6% and 40.2%, surpassing state-of-the-art methods by 4.1% and 2.2%, respectively, while maintaining real-time inference at 186.2 FPS with only 19.8M parameters and 42.3 GFLOPs, representing a $6.14\times$ reduction in computational cost and a $3.86\times$ reduction in model parameters compared to the strongest baseline.

1. Introduction

Unmanned aerial vehicle (UAV)-based remote sensing has become an indispensable tool for large-scale vehicle detection in intelligent transportation management [1], emergency response, and urban surveillance [2,3]. Unlike ground-level detection, UAV imagery introduces compounded challenges: (1) targets appear as densely packed small objects due to high-altitude imaging; (2) extreme scale variation exists between vehicles at different distances and altitudes; (3) challenging illumination conditions, including nighttime and overexposure, degrade visual quality; and (4) heavy occlusion occurs in dense traffic scenarios. These challenges collectively demand a detection framework that simultaneously achieves high accuracy for small and densely packed objects, robust performance across diverse conditions, and real-time throughput suitable for onboard edge deployment or low-latency ground-station processing. A fundamental obstacle underlying these challenges is the repeated spatial downsampling inherent in deep backbone networks, which progressively erases fine-grained spatial details; since over 60% of objects in representative UAV benchmarks such as VisDrone2019 occupy fewer than $32\times 32$ pixels, this information loss disproportionately degrades detection recall for the dominant small-object category. A practical deployment framework must additionally minimize memory footprint and power consumption, as UAV platforms impose strict size, weight, and power (SWaP) constraints that preclude the use of large-scale deep learning models common in server-side computer vision systems.

Modern detectors fall into two dominant paradigms. CNN-based methods, including single-stage detectors such as SSD [4], RetinaNet [5], FCOS [6], ATSS [7], GFL [8], and the YOLO family [9,10,11,12,13], excel at local feature extraction but have limited receptive fields that hinder global context modeling. Two-stage methods (Faster R-CNN [14] and Cascade R-CNN [15]) provide high precision but are too slow for real-time aerial deployment; their reliance on region proposal networks introduces additional computational overhead and latency spikes that are particularly problematic for time-critical surveillance tasks.

Transformer-based detectors (DETR [16], Deformable DETR [17], DN-DETR [18], DINO [19], and RT-DETR [20]) model global context through self-attention but incur $\mathcal{O}\left(N^2\right)$ complexity that grows quadratically with image resolution, making them computationally prohibitive for high-resolution UAV images. Specifically, at $640\times 640$ resolution with a stride-16 backbone, self-attention must process $N=1600$ tokens, incurring $\mathcal{O}\left(N^2\right)=2.56\times {10}^6$ pairwise attention computations per layer. This burden compounds severely as resolution scales to match the fine pixel density required for small-object detection. Recent efforts to adapt DETR-based detectors for drone scenarios have demonstrated promising accuracy gains but still struggle to reconcile global context modeling with the computational constraints of aerial deployment [21].

State Space Models (SSMs) offer a fundamentally different paradigm: rather than computing explicit pairwise token interactions, SSMs model sequences through continuous-time dynamical systems that maintain a compact hidden state and selectively update it as new tokens arrive, thereby achieving global context modeling in linear time. In particular, Mamba [22] and its visual extensions VMamba [23] and Vision Mamba [24] achieve competitive representational capacity at $\mathcal{O}\left(N\right)$ complexity via input-dependent selective state transitions. MobileMamba [25] provides a lightweight mobile-friendly SSM backbone. However, directly substituting the RT-DETR backbone with MobileMamba is insufficient: the resulting feature representations require targeted solutions for cross-scale integration, multi-receptive-field encoding, and content-adaptive upsampling. The lack of a principled end-to-end framework that jointly addresses all of these representational deficiencies—while preserving real-time inference—has motivated the design of RSMamDet.

To address these gaps, we propose RSMamDet with four integrated innovations and a novel training loss. Our main contributions are as follows

• A Selective Feature Scanning (SFS) module within MobileMamba that processes multi-stream features through 2D Selective Scan (SS2D), achieving global context modeling at $\mathcal{O}\left(N\right)$ complexity and replacing quadratic self-attention.
• A Dimension-Aware Selective Integration (DASI) module that adaptively fuses low-level and high-level features through sigmoid-gated cross-dimensional selection, bridging the semantic gap across feature pyramid levels.
• A Poly Kernel Inception Network (PKINet) encoder that captures multi-scale spatial patterns via parallel depthwise convolutions ($3\times 3$ to $11\times 11$) and Context Anchor Attention (CAA).
• An Adaptive Multi-Level Feature Fusion (AMFF) module combining content-aware dynamic upsampling with adaptive channel weighting for improved small-object feature representation.
• An Uncertainty-Minimal Composite (UMC) loss that incorporates uncertainty-aware query selection regularization to improve training stability in cluttered aerial scenes.

Experiments on DroneVehicle [26] and VisDrone2019 [1] show that RSMamDet achieves state-of-the-art performance across all evaluated metrics while maintaining 186.2 FPS real-time inference, demonstrating the practical viability of our approach for intelligent aerial surveillance and transportation monitoring.

The remainder of this paper is organized as follows. Section 2 reviews related work on object detection paradigms, state space models, UAV vehicle detection, and feature pyramid fusion. Section 3 details the proposed RSMamDet architecture and the UMC loss formulation. Section 4 presents experimental results and ablation studies on DroneVehicle and VisDrone2019. Section 5 discusses strengths, limitations, and deployment considerations, and Section 6 concludes the paper.

2. Related Work

2.1. DETR-Based Object Detection

DETR [16] reformulates detection as a set prediction problem using bipartite matching and Transformer encoder–decoder architecture. Deformable DETR [17] accelerates training by restricting attention to sparse deformable points. DAB-DETR [27] uses dynamic anchor boxes as queries, while DN-DETR [18] introduces query denoising for faster convergence. DINO [19] combines contrastive denoising with mixed query selection. RT-DETR [20] introduces real-time capability via an efficient hybrid encoder and uncertainty-minimal query selection. Co-DETR [28] further improves training through collaborative hybrid assignments. RT-DETRv2 [29] provides additional bag-of-freebies improvements. More recently, DPF-DETR [30] enhances drone image detection by incorporating density perception and multi-scale feature fusion into the DETR framework, highlighting the growing interest in tailoring end-to-end detectors for aerial scenarios. Our work builds upon RT-DETR and extends it with SSM-based backbone and targeted modules for UAV detection. Despite these advances, most DETR variants were designed for standard ground-level imagery and have not been specifically optimized for UAV data, where extremely small object sizes, high object density, and severe scale variation challenge the cross-attention mechanism’s ability to focus on relevant spatial regions at real-time speeds. In particular, the encoder self-attention in standard DETR variants processes all spatial tokens equally, making it disproportionately expensive for high-resolution aerial inputs and prone to being overwhelmed by the dominant background tokens that constitute the majority of pixels in sparsely populated UAV scenes.

2.2. State Space Models for Vision

Mamba [22] introduces selective SSMs achieving linear-complexity sequence processing. S4 [31] provides the structured SSM theoretical foundation. VMamba [23] extends Mamba to 2D visual features via four-directional SS2D. Vision Mamba [24] adopts bidirectional Mamba blocks for image patches. PlainMamba [32] studies non-hierarchical architectures. MobileMamba [25] designs a mobile-friendly SSM backbone. Domain applications include DCAM-DETR [21] for anti-UAV detection, SoccerDETR [33] for sports detection, and CGMamba [34] for visual identification. Complementary to SSM approaches, lightweight hybrid attention mechanisms [35] have also been explored to reduce computational burden by selectively enhancing channel, spatial, and token-level features without incurring the quadratic complexity of full self-attention. Our SFS module extends SS2D to multi-stream feature fusion within the MobileMamba backbone and is the first to exploit joint multi-stream SS2D scanning for implicit cross-feature-pyramid dependency modeling. Despite these promising results in image recognition, the integration of SSMs into end-to-end detection frameworks—particularly for UAV scenarios where efficient global context is critical—remains relatively unexplored, and no existing work jointly combines SSM-based scanning with targeted cross-scale fusion and adaptive upsampling for aerial vehicle detection.

2.3. UAV Vehicle Detection

ClusDet [36] proposes clustering-based sub-image detection for dense small objects. UFPMP-Det [37] introduces unified foreground packing for drone imagery. QueryDet [38] cascades sparse queries for high-resolution small object detection. AMSRDet [3] designs an adaptive multi-scale architecture for infrared-visible UAV vehicle detection. CSFADet [39] addresses dual-modal detection via cross-spectral feature alignment. BSOEDet [40] proposes background suppression and object enhancement strategies specifically designed for UAV aerial imagery, effectively mitigating the interference of complex backgrounds on small-object detection. Beyond generic vehicle detection, specialized UAV-based inspection systems have been developed for agricultural applications; for instance, PM-YOLO [41] introduces a powdery mildew grading model for rubber tree disease detection, demonstrating how domain-specific lightweight architectures can be adapted to aerial platforms with constrained computational resources. Beyond vehicle-centric scenarios, UAV-based sensing has also been extensively applied to infrastructure inspection such as power line monitoring, where efficient detection architectures under flight constraints are equally critical [42]. These highlight the importance of multi-scale handling and context awareness in aerial scenarios. Furthermore, several methods rely on region-based inference or image tiling strategies that impose additional pre- and post-processing overheads incompatible with latency requirements of onboard UAV deployment. However, these methods generally address at most one or two of the UAV-specific challenges simultaneously: approaches designed for small-object detection often sacrifice inference speed, while real-time methods typically lack the global contextual awareness needed for dense and heavily occluded scenes, leaving a significant gap for a unified real-time framework.

2.4. Feature Pyramid and Fusion

FPN [43] establishes multi-scale feature representation via top-down fusion. PANet [44] adds bottom-up pathways. BiFPN in EfficientDet [45] introduces learnable weighted bidirectional fusion. Inception-style multi-kernel approaches [46] capture diverse spatial patterns. Dynamic upsampling methods [47] replace fixed interpolation with content-adaptive kernels. Our AMFF combines dynamic kernel generation with adaptive channel weighting for superior multi-level fusion. While existing fusion strategies have proven effective on standard benchmarks, their application to UAV imagery is limited by fixed or globally shared weights that fail to adapt to the heterogeneous scale distribution characteristic of aerial scenes, motivating our spatially varying adaptive fusion design. AMFF further differs from AFA [47] in that it simultaneously learns per-channel contributions through a second adaptive weighting branch, providing an additional degree of freedom for balancing high-resolution texture and high-level semantic content.

Table 1. Comparative summary of representative detection paradigms.

Paradigm	Global Context	Complexity	Multi-Scale Fusion	UAV-Specific
CNN-based (YOLO, FCOS)	Limited	$\mathcal{O}\left(k^{2}N\right)$	FPN / PANet	No
Two-Stage (Faster/Cascade R-CNN)	Limited	$\mathcal{O}\left(k^{2}N\right)$	FPN + RPN	No
DETR-based (DETR, DINO)	Full	$\mathcal{O}\left(N^2\right)$	Deformable Attn.	No
RT-DETR	Full	$\mathcal{O}\left(N^2\right)$	Hybrid Encoder	No
SSM-based (VMamba)	Full	$\mathcal{O}\left(N\right)$	None	No
RSMamDet (Ours)	Full	$\mathcal{O}\left(\mathit{N}\right)$	DASI + AMFF	Yes

Bold: proposed method (RSMamDet) and its results.

summarizes the key characteristics of representative detection paradigms, positioning RSMamDet relative to existing approaches across four critical dimensions: global context capability, computational complexity, multi-scale fusion strategy, and UAV-specific optimization.

3. Proposed Method

3.1. Overall Architecture

The overall architecture of RSMamDet is illustrated in Figure 1. Given input image $\mathbf{I}\in {\mathbb{R}}^{H\times W\times 3}$, the MobileMamba backbone with SFS modules extracts multi-scale features $\{P_3,P_4,P_5\}$ at $\mathcal{O}\left(N\right)$ complexity. The three feature scales encode complementary information: $P_3$ retains high-resolution spatial detail critical for small vehicle localization, $P_4$ captures mid-level semantic patterns, and $P_5$ provides the global contextual representation necessary for scene-level reasoning. The Efficient Transformer Encoder, comprising PKINet for multi-receptive-field enrichment and DASI for adaptive cross-level fusion, bridges the semantic–resolution gap between adjacent pyramid levels. The CCFF neck then applies AMFF modules at each scale, recovering fine-grained spatial details via content-aware dynamic upsampling to produce enhanced features $\{{\widehat{F}}_3,{\widehat{F}}_4,{\widehat{F}}_5\}$. Finally, the Uncertainty-Minimal Query Selection module identifies the top-K highest-confidence spatial positions as initial decoder queries, suppressing the dominant background tokens common in high-altitude drone imagery before passing queries to the Transformer decoder and detection head. The four modules are designed to be mutually reinforcing: the SFS backbone provides linear-complexity global representations that DASI can fuse without any quadratic overhead; PKINet then enriches those fused features with multi-scale spatial patterns; and AMFF restores the fine-grained spatial structure that successive downsampling stages erode, ensuring that the final decoder queries are both semantically rich and spatially precise.

3.2. MobileMamba Backbone and Selective Feature Scanning (SFS) Module

3.2.1. State Space Model Foundations

Structured SSMs [31] model a continuous-time linear dynamical system:

$$h^{\prime }\left(t\right)=\mathbf{A}h\left(t\right)+\mathbf{B}x\left(t\right),y\left(t\right)=\mathbf{C}h\left(t\right)+\mathbf{D}x\left(t\right),$$

(1)

where $\mathbf{A}\in {\mathbb{R}}^{N\times N}$, $\mathbf{B}\in {\mathbb{R}}^N$, $\mathbf{C}\in {\mathbb{R}}^N$, $\mathbf{D}\in \mathbb{R}$. Zero-Order Hold discretization at step size $\Delta$ yields

$$h_k=\overline{\mathbf{A}}{h}_{k-1}+\overline{\mathbf{B}}{x}_k,y_k=\mathbf{C}{h}_k+\mathbf{D}{x}_k,$$

(2)

where $\overline{\mathbf{A}}=exp(\Delta \mathbf{A})$, $\overline{\mathbf{B}}\approx \Delta \mathbf{B}$. The selective S6 mechanism in Mamba [22] makes $\{\Delta ,\mathbf{B},\mathbf{C}\}$ input-dependent:

$$[\Delta ,\mathbf{B},\mathbf{C}]=\mathrm{Linear}\left(x_k\right),[\overline{\mathbf{A}},\overline{\mathbf{B}}]=\mathrm{Discretize}(\Delta ,\mathbf{A},\mathbf{B}),$$

(3)

achieving linear $\mathcal{O}\left(N\right)$ complexity via parallel associative scan.

3.2.2. 2D Selective Scan and SFS Design

For 2D visual features, SS2D [23] unfolds a feature map into four directional sequences, applies S6 to each, and merges outputs by summation, preserving $\mathcal{O}\left(N\right)$ complexity while capturing omnidirectional spatial context. Formally, for input $Z\in {\mathbb{R}}^{C\times H\times W}$, SS2D applies S6 along four traversal directions $d\in \{1,2,3,4\}$ (top-left → bottom-right, bottom-right → top-left, top-right → bottom-left, and bottom-left → top-right):

$$Y^{\left(d\right)}=\mathrm{S}6\left({\mathrm{flatten}}^{\left(d\right)}\left(Z\right)\right)\in {\mathbb{R}}^{C\times HW},$$

(4)

and merges all directional outputs via element-wise summation after reshaping:

$$Y_{\mathrm{S}\mathrm{S}2\mathrm{D}}={\displaystyle \sum _{d=1}^4}\mathrm{reshape}\left(Y^{\left(d\right)}\right)\in {\mathbb{R}}^{C\times H\times W}.$$

(5)

This four-path traversal guarantees that every spatial location can exchange information with every other location along at least one scan direction, effectively achieving a global receptive field while the total computation scales as $\mathcal{O}\left(4N\right)=\mathcal{O}\left(N\right)$.

The SFS module (Figure 2) jointly processes three input streams $\{F_{main},F_{guide},F_{aux}\}$, each serving a distinct semantic role. $F_{main}$ carries the primary feature representation at the current pyramid level, encoding the core spatial and channel information to be refined. $F_{guide}$ provides higher-level semantic guidance from a coarser pyramid level, supplying scene-level contextual cues that help disambiguate overlapping or occluded objects. $F_{aux}$ contributes complementary fine-grained spatial detail from a finer pyramid level, preserving edge and texture information essential for localizing small targets. Each stream $F_{\ast }\in {\mathbb{R}}^{C_{\ast }\times H\times W}$ is first split along the channel dimension and normalized:

$$[{\tilde{F}}_{\ast }^1,{\tilde{F}}_{\ast }^2]=\mathrm{Split}\left(\mathrm{LN}\left(F_{\ast }\right)\right),\ast \in \{m,g,a\},$$

(6)

where ${\tilde{F}}_{\ast }^1$ enters the SS2D branch and ${\tilde{F}}_{\ast }^2$ serves as a skip-connection gate. Each SS2D-branch half then undergoes Linear → DWConv → SiLU projection:

$$Z_{\ast }=\mathrm{SiLU}(\mathrm{DWConv}(\mathrm{Linear}({\tilde{F}}_{\ast }^1))),\ast \in \{m,g,a\}.$$

(7)

The three projected streams are concatenated and jointly processed by SS2D:

$$[Y_m,Y_g,Y_a]=\mathrm{S}\mathrm{S}2\mathrm{D}\left([Z_m,Z_g,Z_a]\right),$$

(8)

$$Y_1=\mathrm{Concat}\left(\mathrm{Linear}\left(\mathrm{LN}\left(Y_m\right)\right),\mathrm{Linear}\left(\mathrm{LN}\left(Y_g\right)\right),\mathrm{Linear}\left(\mathrm{LN}\left(Y_a\right)\right)\right).$$

(9)

The aggregated representation $Y_1$ is then modulated by the skip-connected gate, refined by LayerNorm and FFN, and split to produce the final output streams:

$$Y_2=\mathrm{LN}\left(Y_1\odot \sigma \left(\mathrm{Linear}(\mathrm{Concat}({\tilde{F}}_m^2,{\tilde{F}}_g^2,{\tilde{F}}_a^2))\right)\right),$$

(10)

$$Y_3=Y_2+\mathrm{FFN}\left(Y_2\right),[{\widehat{F}}_m,{\widehat{F}}_a]=\mathrm{Split}\left(\mathrm{Linear}\left(Y_3\right)\right).$$

(11)

By processing all three streams jointly through a single SS2D pass rather than independently, the selective state transitions of $F_{guide}$ and $F_{aux}$ directly modulate the hidden states computed for $F_{main}$, enabling implicit cross-stream information exchange without any explicit cross-attention operation. Joint SS2D processing thus captures cross-stream long-range dependencies at $\mathcal{O}\left(N\right)$ complexity, replacing the $\mathcal{O}\left(N^2\right)$ cross-attention otherwise required. This is particularly advantageous for high-resolution UAV imagery where large feature maps would make standard self-attention computationally prohibitive, yet global spatial context is essential for associating co-occurrent vehicles distributed across wide scenes. The three-stream design also provides a natural information bottleneck: by forcing all streams through a single SS2D pass, the model must learn to allocate the fixed-size hidden state budget across all inputs jointly, which implicitly encourages the state transitions to encode only the most task-relevant cross-stream correlations and discard redundant local patterns already captured by the preceding convolutional layers.

To validate the effectiveness of the three-stream design,

Table 2. Effect of SFS stream configurations on DroneVehicle.

Configuration	mAP50 (%)	Params (M)	FPS
Single-stream ($F_{main}$ only)	69.4	18.2	198.5
Two-stream ($F_{main}$ + $F_{guide}$)	71.2	19.0	192.3
Three-stream (Full SFS)	72.6	19.8	186.2

compares single-stream, two-stream, and three-stream SFS variants under otherwise identical settings on DroneVehicle.

The three-stream design achieves a 3.2% mAP₅₀ improvement over the single-stream baseline with only a 6.2% reduction in throughput, confirming that the joint multi-stream SS2D processing provides substantial accuracy gains at modest computational overhead.

3.3. Dimension-Aware Selective Integration (DASI) Module

The DASI module (Figure 3) adaptively fuses low-level feature ${\mathbf{F}}_l\in {\mathbb{R}}^{C_i\times H_i\times W_i}$ and high-level feature ${\mathbf{F}}_h\in {\mathbb{R}}^{C_b\times H_b\times W_b}$ by bridging semantic and resolution gaps. Both features are aligned to a common spatial resolution $(H,W)$ via $C_{3}S$ (stride-S conv) and $C_{1}BS$ (conv + bilinear upsample), then concatenated and jointly encoded via Group $=4$ convolution to form $\mathbf{U}$, which is split into branches $\{l_i,u_i,h_i\}$.

A sigmoid gate on $u_i$ produces a per-location, per-channel selection weight:

$$\alpha =\sigma \left(u_i\right)\in {[0,1]}^{C\times H\times W},$$

(12)

and the selectively integrated feature is

$${\mathbf{F}}_{int}=\alpha \odot l_i+(1-\alpha )\odot h_i.$$

(13)

The output is produced by concatenation with a residual feature and a pointwise BN-ReLU convolution:

$${\mathbf{F}}_{out}=\mathrm{BNReLU}\left(\mathrm{PWConv}\left([{\mathbf{F}}_{int};{\mathbf{F}}_{int}]\right)\right)\in {\mathbb{R}}^{C\times H\times W}.$$

(14)

Unlike simple concatenation–conv fusion, DASI adaptively balances spatial detail and semantic richness per location, improving detection at varied object scales. This is especially beneficial in dense UAV scenes where objects of vastly different sizes coexist, as the per-location gate $\alpha$ prevents the over-smoothing of fine-grained edge details that rigid fixed-weight fusion would introduce. Furthermore, computing $\alpha$ from the jointly encoded feature $\mathbf{U}$ rather than from either source branch independently ensures that the selection weight encodes the relative complementarity of the two inputs at every spatial position, rather than reflecting properties of only one branch in isolation.

3.4. Poly Kernel Inception Network (PKINet)

To enrich encoder representations with diverse receptive fields, we employ PKINet [46] as the core encoder backbone (Figure 4). For input ${\mathbf{X}}_{l-1}$, after downsampling and $1\times 1$ channel compression, features are split into two branches processed by FFN and PKI Blocks, respectively. Within each PKI Block, the PKI Module runs parallel depthwise convolutions:

$$Y_n^{\left(k\right)}={\mathrm{DWConv}}_{k\times k}\left({\mathbf{X}}_n^{\left(2\right)}\right),k\in \{3,5,7,9,11\},$$

(15)

$$P_n={\mathrm{Conv}}_{1\times 1}\left(\underset{k}{⨁}{Y}_n^{\left(k\right)}\right),$$

(16)

where ⨁ denotes element-wise addition. The Context Anchor Attention (CAA) module recalibrates channels:

$$A_n=\sigma \left({\mathrm{Conv}}_{1\times 1}\left(\mathrm{DWConv}\left(\mathrm{DWConv}\left({\mathrm{Conv}}_{1\times 1}(\mathrm{AvgPool}({\mathbf{X}}_n^{\left(2\right)}))\right)\right)\right)\right),$$

(17)

$${\mathbf{X}}_{n+1}^{\left(2\right)}={\mathrm{Conv}}_{1\times 1}(P_n\odot A_n+{\mathbf{X}}_n^{\left(2\right)}).$$

(18)

The two branches are concatenated and fused by a final $1\times 1$ convolution. The specific kernel range $\{3,5,7,9,11\}$ is motivated by the object size distribution in UAV datasets. Statistical analysis of the DroneVehicle training set reveals that vehicle bounding-box widths range from 6 to 128 pixels at $640\times 640$ input resolution, with 78% of instances concentrated between 8 and 48 pixels. At the stride-16 encoder feature map, these widths correspond to 0.5–3 feature-map pixels, requiring effective receptive fields of 3–11 pixels to fully encompass the target extent. Extending beyond $11\times 11$ would primarily capture contextual background without improving target discrimination, while omitting the larger kernels ($9\times 9$, $11\times 11$) would lose coverage of trucks and buses spanning 40–80 input pixels. This range enables RSMamDet to simultaneously respond to compact motorcycles and large trucks that coexist in the same drone-captured frame without requiring separate detection heads for different scale ranges. The element-wise summation aggregation in the PKI Module encourages each kernel branch to specialize in the scale range where it provides complementary information, as the gradient from the final $1\times 1$ projection distributes learning signals proportionally to each branch’s unique contribution to the fused representation.

3.5. Adaptive Multi-Level Feature Fusion (AMFF) Module

Standard bilinear upsampling ignores image content, yielding blurry reconstructions harmful to small-object detection. AMFF (Figure 5) addresses this via three sub-structures.

3.5.1. Feature-Aggregated Upsampling Structure (FAUS)

A dynamic upsampling kernel is generated from the low-resolution feature $F_n$:

$$k_{up}=\mathrm{KernelNorm}\left(\mathrm{ContentEncode}\left(\mathrm{ChannelCompress}\left(F_n\right)\right)\right)\in {\mathbb{R}}^{{\sigma }^2\times k_{up}^2},$$

(19)

$$F_n^{up}=F(F_n,k_{up})\otimes W_i\in {\mathbb{R}}^{C\times \sigma H\times \sigma W},$$

(20)

where $\sigma$ is the upsample scale factor and $W_i$ is a learnable channel mixing weight.

3.5.2. Feature Refinement Structure (FRS)

$$F_{n-1}^{ref}=\sigma \left(F_n^{up}\right)\odot F_{n-1},F_{fusion}^{pre}=F_{n-1}^{ref}+2\uparrow \left(F_n\right),$$

(21)

where $2\uparrow$ denotes $2\times$ upsampling.

3.5.3. Adaptive Feature Fusion Structure (AFFS)

$$\omega =\sigma \left(\mathrm{FCN}\left(\mathrm{AvgPool}\left(F_{n-1}^{ref}\right)\right)+{\mathrm{Conv}}_{3\times 3}\left({\mathrm{Conv}}_{1\times 1}\left(F_n^{up}\right)\right)\right),$$

(22)

$$F_{fusion}=\omega \odot F_{n-1}^{ref}+(1-\omega )\odot F_n^{up}.$$

(23)

The spatially varying weight $\omega$ determines per-location contributions of high-resolution detail and semantic context, enabling accurate localization of small objects that content-agnostic bilinear upsampling typically blurs or loses entirely.

3.6. Uncertainty-Minimal Composite (UMC) Loss

We design the UMC loss to jointly optimize detection quality and query selection certainty. The total training loss is

$${\mathcal{L}}_{total}={\lambda }_1{\mathcal{L}}_{focal}+{\lambda }_2{\mathcal{L}}_{L1}+{\lambda }_3{\mathcal{L}}_{GIoU}+{\lambda }_4{\mathcal{L}}_{unc},$$

(24)

where ${\mathcal{L}}_{focal}$ is the sigmoid focal loss [5] for classification, ${\mathcal{L}}_{L1}$ and ${\mathcal{L}}_{GIoU}$ [48] supervise box regression, and ${\mathcal{L}}_{unc}$ is the uncertainty-minimal query regularization loss.

For each query $q_j$ with predicted class probabilities ${\left\{p_{j,c}\right\}}_{c=1}^{C_{cls}}$, we define the prediction uncertainty as

$$u_j=1-\underset{c}{max}{p}_{j,c}\in [0,1].$$

(25)

A lower $u_j$ indicates a more confident, high-quality query. We select the top-K most confident queries ${\mathcal{Q}}^{\ast }={\mathrm{TopK}}_K(-u_j)$ and compute an uncertainty-weighted contrastive term:

$${\mathcal{L}}_{unc}=-{\displaystyle \frac{1}{|{\mathcal{Q}}^{\ast }|}}\sum _{j\in {\mathcal{Q}}^{\ast }}(1-u_j)·log{\displaystyle \frac{exp(s_j/\tau )}{{\sum }_{k=1}^{N_q}exp(s_k/\tau )}},$$

(26)

where $s_j$ is the IoU-based quality score of query j, $\tau$ is a learnable temperature parameter, and $N_q$ is the total number of queries. This term encourages high-confidence queries to have high IoU scores, suppressing uncertain predictions in cluttered aerial scenes. The combination of contrastive temperature scaling and IoU-based scoring makes the regularization robust to the class imbalance inherent in drone imagery, where background tokens vastly outnumber foreground vehicle tokens. The temperature parameter $\tau$ is initialized to 0.07 following established practice in contrastive learning and is allowed to adapt throughout training, enabling the model to automatically calibrate the sharpness of inter-query competition based on the evolving score distribution. The full training procedure is given in Algorithm 1.

Algorithm 1: Training Procedure of RSMamDet with UMC Loss

Require: Dataset $\mathcal{D}=\left\{(I_i,{\mathcal{G}}_i)\right\}$; loss weights ${\lambda }_1,{\lambda }_2,{\lambda }_3,{\lambda }_4$; temperature $\tau$; top-K queries; learning rate $\eta$ Ensure: Trained model parameters $\Theta$ 1: Initialize $\Theta$ from ImageNet pre-trained MobileMamba weights 2: for each training epoch do 3:     for each mini-batch $(I,\mathcal{G})\sim \mathcal{D}$ do 4:         $\{P_3,P_4,P_5\}\leftarrow \mathrm{MobileMamba}-\mathrm{SFS}(I;\Theta )$ 5:         ${\mathbf{F}}_{fused}\leftarrow \mathrm{DASI}(P_3,P_4,P_5;\Theta )$ 6:         ${\mathbf{F}}_{enc}\leftarrow \mathrm{PKINet}({\mathbf{F}}_{fused};\Theta )$ 7:         $\{{\widehat{F}}_3,{\widehat{F}}_4,{\widehat{F}}_5\}\leftarrow \mathrm{AMFF}({\mathbf{F}}_{enc};\Theta )$ 8:         Generate queries $Q\leftarrow \mathrm{UncertaintyMinimalSelection}\left(\widehat{F}\right)$ 9:         $\{{\widehat{b}}_j,{\widehat{c}}_j\}\leftarrow \mathrm{Decoder}(Q,\widehat{F};\Theta )$ 10:         ${\sigma }^{\ast }\leftarrow arg{min}_{\sigma }{\mathcal{L}}_{match}$ via Hungarian algorithm 11:         ${\mathcal{L}}_{focal}\leftarrow -{\sum }_i{\alpha }_t{(1-p_t)}^{\gamma }log{p}_t$                     ▹ Focal loss 12:         ${\mathcal{L}}_{box}\leftarrow {\lambda }_2{\parallel \widehat{b}-b\parallel }_1+{\lambda }_3{\mathcal{L}}_{GIoU}(\widehat{b},b)$ 13:         $u_j\leftarrow 1-{max}_c{p}_{j,c}$ for all queries j 14:         ${\mathcal{Q}}^{\ast }\leftarrow {\mathrm{TopK}}_K(-u_j)$ 15:         ${\mathcal{L}}_{unc}\leftarrow -{\displaystyle \frac{1}{|{\mathcal{Q}}^{\ast }|}}{\sum }_{j\in {\mathcal{Q}}^{\ast }}(1-u_j)log{\displaystyle \frac{exp(s_j/\tau )}{{\sum }_{k}exp(s_k/\tau )}}$ 16:         ${\mathcal{L}}_{total}\leftarrow {\lambda }_1{\mathcal{L}}_{focal}+{\mathcal{L}}_{box}+{\lambda }_4{\mathcal{L}}_{unc}$ 17:         $\Theta \leftarrow \Theta -\eta {\nabla }_{\Theta }{\mathcal{L}}_{total}$ 18:     end for 19: end for 20: return $\Theta$

4. Experiments

4.1. Datasets

DroneVehicle [26] is a large-scale UAV-based RGB-infrared vehicle detection benchmark captured from various altitudes (60–130 m) and scenarios, containing 56,878 image pairs with five vehicle categories: car, truck, bus, van, and freight car. The dataset includes over 280,000 annotated bounding-box instances across the training split, with an average of approximately 15.8 vehicles per image and a maximum of 87 vehicles in a single frame, reflecting the extreme density challenges of the benchmark. It is particularly challenging due to extreme density variation and nighttime scenes. We follow the official split with 17,990 training and 1469 test images.

VisDrone2019 [1] is a comprehensive drone-based benchmark with 10,209 images (6471 train, 548 val, 3190 test-challenge) covering 10 object categories, including pedestrian, person, car, van, bus, truck, motor, bicycle, awning-tricycle, and tricycle. Images are captured at $1920\times 1080$ pixels, presenting dense small targets across diverse scenes including urban, suburban, and campus environments. The dataset contains approximately 540,000 annotated instances across the training and validation splits, with objects as small as $5\times 5$ pixels; over 60% of annotated objects occupy fewer than $32\times 32$ pixels, making small-object detection the central challenge.

4.2. Implementation Details

All experiments are implemented in PyTorch 2.1 on 4 NVIDIA A100 GPUs. The model is trained with the AdamW optimizer, initial learning rate $1\times {10}^{-4}$, weight decay $1\times {10}^{-4}$, and cosine annealing schedule over 100 epochs. Batch size is 16. Input images are resized to $640\times 640$. Loss weights are set to ${\lambda }_1=1.0$, ${\lambda }_2=5.0$, ${\lambda }_3=2.0$, ${\lambda }_4=0.5$. Temperature $\tau$ is initialized to 0.07 and learned during training. Top-K is set to 300. Data augmentation includes random horizontal flip (probability 0.5), mosaic (four-image random composition applied at each training step), mixup (image pair blending with Beta-distributed interpolation weight $\alpha =0.5$), and random scale jitter in the range $[0.5,2.0]$. The MobileMamba backbone is initialized from ImageNet-1K pre-trained weights, while all other modules are trained from scratch with Xavier initialization. FPS is measured on a single NVIDIA RTX 3090 GPU with batch size 1 at $640\times 640$ resolution.

Figure 6 shows the training loss convergence and validation mAP₅₀ progression over 100 epochs. RSMamDet converges faster and reaches a lower final training loss compared to all baselines, indicating more efficient optimization enabled by the UMC loss and the SSM-based backbone initialization. The validation mAP₅₀ curve confirms consistent performance gains throughout training.

4.3. Comparison with State-of-the-Art Methods on DroneVehicle

We compare RSMamDet against 25 state-of-the-art methods on DroneVehicle [26], organized by detection paradigm in

Table 3. Comparison on DroneVehicle dataset. Bold: best. Underline: second best.

Category	Method	Backbone	mAP50 (%)	mAP50:95 (%)	FPS
One-Stage	SSD [4]	VGG-16	47.2	28.6	42.3
	RetinaNet [5]	ResNet-50	49.8	31.3	35.7
	FCOS [6]	ResNet-50	52.3	33.7	38.4
	ATSS [7]	ResNet-50	54.6	35.9	36.2
	GFL [8]	ResNet-50	56.1	37.4	33.8
Two-Stage	Faster R-CNN [14]	ResNet-50	53.4	34.8	18.5
	Cascade R-CNN [15]	ResNet-50	57.2	39.3	12.7
	Sparse R-CNN [49]	ResNet-50	58.6	40.1	22.4
	Grid R-CNN	ResNet-50	55.8	38.1	16.3
YOLO-Based	YOLOv5s [9]	CSPDarkNet	59.3	38.7	163.5
	YOLOv7 [10]	E-ELAN	61.7	41.2	113.2
	YOLOv8s [11]	CSPDarkNet	63.2	43.8	152.7
	YOLOv8-L [11]	CSPDarkNet	65.4	46.2	84.3
	YOLOv9s [12]	Gelan	64.5	45.1	128.3
	YOLOv10s [13]	CSP	65.1	45.8	145.2
	YOLOv13 [50]	HyperGraph	66.4	47.1	118.4
	Gold-YOLO-S [51]	Gold	64.8	45.3	142.7
Transformer-Based	DETR [16]	ResNet-50	56.8	37.2	22.5
	Deformable DETR [17]	ResNet-50	61.3	42.7	28.4
	DAB-DETR [27]	ResNet-50	62.8	43.9	25.7
	DN-DETR [18]	ResNet-50	63.5	44.6	23.8
	RT-DETR-R18 [20]	ResNet-18	65.8	46.3	217.4
	RT-DETR-R50 [20]	ResNet-50	67.2	47.8	108.7
	Co-DETR [28]	ResNet-50	68.3	49.1	18.6
	RT-DETR-L [20]	ResNet-101	68.5	49.4	74.2
Ours	RSMamDet	MobileMamba	72.6	53.4	186.2

mAP₅₀: mean average precision at IoU threshold 0.5; mAP_50:95: average over IoU thresholds 0.5:0.05:0.95; FPS measured on RTX 3090, input $640\times 640$.

. Our method achieves mAP₅₀ of 72.6%, surpassing the prior best (RT-DETR-L: 68.5%) by 4.1% while maintaining real-time inference at 186.2 FPS with only 19.8M parameters. Qualitative detection results on DroneVehicle are shown in Figure 7.

Figure 8 further illustrates the detection quality of RSMamDet through precision–recall curves and per-category AP₅₀ analysis on DroneVehicle. RSMamDet achieves a larger area under the PR curve across all categories, with particularly notable improvements on the Truck (+6.1%) and Freight Car (+5.9%) categories, which are challenging due to high shape diversity and frequent occlusion.

Comparing across detection paradigms, RSMamDet consistently surpasses all four categories. Among one-stage detectors, GFL [8] achieves only 56.1% mAP₅₀ at 33.8 FPS, while RSMamDet delivers a 16.5% absolute gain at more than 5× the throughput. Two-stage detectors such as Sparse R-CNN [49] reach 58.6% but suffer from severely limited throughput (22.4 FPS), rendering them impractical for real-time UAV applications. YOLO-based methods offer a more favorable speed–accuracy balance: YOLOv13 [50] achieves 66.4% at 118.4 FPS, yet still trails RSMamDet by 6.2% in mAP₅₀. Within the Transformer-based category, Co-DETR [28] and RT-DETR-L [20] represent the strongest prior methods at 68.3% and 68.5%, respectively, but both operate below real-time speed (18.6 and 74.2 FPS). RSMamDet uniquely breaks the accuracy–speed trade-off by achieving 72.6% mAP₅₀ at 186.2 FPS, demonstrating that SSM-based linear-complexity modeling can simultaneously fulfill both requirements.

Beyond mAP₅₀, RSMamDet also achieves the highest mAP_50:95 of 53.4%, surpassing RT-DETR-L (49.4%) by 4.0% and Co-DETR (49.1%) by 4.3%. This metric enforces accurate localization across a wide range of IoU thresholds and thus reflects bounding-box quality rather than mere recall. The gain is attributable to two complementary factors: the AMFF module’s content-aware dynamic upsampling recovers fine-grained spatial details lost during feature downsampling, while the PKINet encoder’s multi-kernel receptive field expansion produces more discriminative representations for overlapping and partially occluded vehicles in dense aerial imagery. Collectively, RSMamDet establishes a new state of the art on the DroneVehicle benchmark across all evaluated metrics while imposing no latency penalty for deployment.

4.4. Computational Efficiency and Model Complexity Analysis

Table 4. Model complexity comparison on DroneVehicle. Bold: best. GFLOPs measured at $640\times 640$ resolution.

Method	Params (M)	GFLOPs	Mem. (MB)	FPS	mAP50 (%)
SSD [4]	26.3	35.2	312	42.3	47.2
Faster R-CNN [14]	41.5	180.7	1243	18.5	53.4
YOLOv8s [11]	11.2	28.6	284	152.7	63.2
YOLOv8-L [11]	43.7	165.2	1125	84.3	65.4
YOLOv13 [50]	36.8	127.5	876	118.4	66.4
RT-DETR-R18 [20]	20.0	60.4	487	217.4	65.8
RT-DETR-R50 [20]	42.0	136.5	892	108.7	67.2
Co-DETR [28]	64.2	234.5	1687	18.6	68.3
RT-DETR-L [20]	76.3	259.7	1842	74.2	68.5
RSMamDet (Ours)	19.8	42.3	276	186.2	72.6

GFLOPs: giga floating-point operations at $640\times 640$ input; Mem.: GPU memory during inference (batch size = 1); FPS measured on RTX 3090.

presents a detailed comparison of model complexity and inference efficiency across representative detection methods on the DroneVehicle benchmark. We report the number of trainable parameters (M), computational cost in GFLOPs at $640\times 640$ input resolution, memory footprint during inference (MB), and throughput in FPS on an RTX 3090 GPU.

RSMamDet achieves the highest mAP₅₀ (72.6%) with only 19.8M parameters and 42.3 GFLOPs, making it the most parameter-efficient method in the comparison. Compared to RT-DETR-L, the second-best baseline, RSMamDet reduces parameters by $3.86\times$, GFLOPs by $6.14\times$, and memory by $6.67\times$, while improving accuracy by 4.1%. Compared to Co-DETR (68.3%, 64.2M, 234.5 GFLOPs, 18.6 FPS), RSMamDet simultaneously achieves higher accuracy (+4.3%), substantially lower computational cost ($5.54\times$ fewer GFLOPs), and $10\times$ higher throughput, demonstrating that global contextual modeling need not entail quadratic computational burden when replaced with SSM-based linear scanning. Among YOLO-based methods with comparable speed, YOLOv8-L (84.3 FPS, 43.7M) incurs $2.21\times$ more parameters and $3.90\times$ more GFLOPs than RSMamDet, yet achieves 7.2% lower mAP₅₀.

The linear $\mathcal{O}\left(N\right)$ complexity of the SFS module is the primary enabler of RSMamDet’s efficiency advantage. Standard self-attention scales quadratically with the number of image tokens N; at $640\times 640$ with stride 16, which generates $N=1600$ tokens and incurs $\mathcal{O}\left(N^2\right)=2.56\times {10}^6$ pairwise interactions per head. The SSM-based SFS module processes the same feature map in $\mathcal{O}\left(N\right)$ operations by replacing global attention with structured recurrent state transitions, reducing encoder FLOPs by approximately 38% compared to an equivalent attention encoder while preserving the global receptive field essential for detecting densely packed small vehicles in UAV imagery.

4.5. Comparison with State-of-the-Art Methods on VisDrone2019

Table 5. Comparison on VisDrone2019 validation set. Bold: best. Underline: second best.

Category	Method	Backbone	mAP50 (%)	mAP50:95 (%)	FPS
One-Stage	SSD [4]	VGG-16	18.3	10.2	42.3
	RetinaNet [5]	ResNet-50	20.1	11.8	35.7
	FCOS [6]	ResNet-50	22.8	13.4	38.4
	ATSS [7]	ResNet-50	24.3	14.9	36.2
	GFL [8]	ResNet-50	25.7	15.8	33.8
Two-Stage	Faster R-CNN [14]	ResNet-50	22.6	13.1	18.5
	Cascade R-CNN [15]	ResNet-50	26.1	16.4	12.7
	Sparse R-CNN [49]	ResNet-50	27.3	17.2	22.4
	ClusDet [36]	ResNet-50	26.7	15.9	8.6
YOLO-Based	YOLOv5s [9]	CSPDarkNet	24.3	13.8	163.5
	YOLOv7 [10]	E-ELAN	27.5	16.3	113.2
	YOLOv8s [11]	CSPDarkNet	29.1	18.2	152.7
	YOLOv8-L [11]	CSPDarkNet	31.4	20.1	84.3
	YOLOv9s [12]	Gelan	30.4	19.1	128.3
	YOLOv10s [13]	CSP	31.2	19.8	145.2
	YOLOv13 [50]	HyperGraph	32.1	20.6	118.4
	Gold-YOLO-S [51]	Gold	30.8	19.4	142.7
Transformer-Based	DETR [16]	ResNet-50	26.4	15.7	22.5
	Deformable DETR [17]	ResNet-50	31.7	20.4	28.4
	DAB-DETR [27]	ResNet-50	32.4	21.1	25.7
	DN-DETR [18]	ResNet-50	33.1	21.8	23.8
	RT-DETR-R18 [20]	ResNet-18	34.2	22.6	217.4
	RT-DETR-R50 [20]	ResNet-50	36.5	24.3	108.7
	Co-DETR [28]	ResNet-50	37.8	25.4	18.6
	RT-DETR-L [20]	ResNet-101	38.0	25.6	74.2
Ours	RSMamDet	MobileMamba	40.2	27.6	186.2

reports results on VisDrone2019 [1]. Our method achieves mAP₅₀ of 40.2%, outperforming the prior best (RT-DETR-L: 38.0%) by 2.2%. Qualitative detection results are shown in Figure 9, covering dense, sparse, nighttime, and occluded scenes.

The performance gap between paradigms on VisDrone2019 is more pronounced than on DroneVehicle, reflecting the greater object-class diversity (10 categories) and smaller average object size characteristic of this benchmark. One-stage detectors plateau at approximately 25.7% (GFL), and two-stage methods reach at most 27.3% (Sparse R-CNN [49]). YOLO-series methods improve to 32.1% (YOLOv13 [50]), benefiting from stronger multi-scale feature fusion in the neck. Transformer-based detectors demonstrate superior adaptability to VisDrone’s complex scene statistics: RT-DETR-R50 reaches 36.5% and RT-DETR-L achieves 38.0%, reflecting the advantage of global cross-attention in capturing long-range contextual dependencies across densely distributed small objects. RSMamDet pushes this frontier to 40.2%, leveraging the linear SSM global context of the SFS module and the adaptive cross-scale integration of DASI to handle the extreme density and scale variation inherent in low-altitude UAV imagery.

The efficiency advantage of RSMamDet is particularly significant in the VisDrone2019 context, where onboard computation budgets are severely constrained. RT-DETR-L achieves 38.0% at 74.2 FPS using a ResNet-101 backbone (∼76M parameters), whereas RSMamDet surpasses it by 2.2% mAP₅₀ while running at 186.2 FPS with only 19.8M parameters—a 2.5× speedup and 3.8× parameter reduction. The qualitative comparisons in Figure 9 further substantiate these gains across four challenging scene conditions. In the dense campus scene (row a), RSMamDet detects substantially more pedestrians and small vehicles at the image periphery that baseline methods miss entirely. In the nighttime scene (row c), the SFS global context enables reliable detection under severe illumination degradation, where CNN-based methods produce numerous false negatives. In the occlusion-heavy urban scene (row d), the DASI module’s adaptive cross-scale fusion better preserves object identity boundaries for heavily overlapping targets, yielding higher-confidence predictions and fewer missed detections across all evaluated categories.

Figure 9 extends the qualitative evaluation to the VisDrone2019 dataset, which presents fundamentally different challenges from DroneVehicle. The significantly higher shooting altitude results in smaller object appearances (typical object size: 10–30 pixels), and the 10-category annotation scheme (including pedestrians, cyclists, and various vehicle types) requires fine-grained inter-class discrimination. The four scenes selected in Figure 9 span the full difficulty range of the VisDrone2019 benchmark: a dense campus scene with tightly clustered pedestrians (row a), a sparse parking-lot scene requiring precise small-vehicle localization (row b), a nighttime low-light scene with severe contrast degradation (row c), and a dense urban road scene with heavy mutual occlusion among vehicles (row d). In the dense and occluded scenes (rows a and d), the RT-DETR-R50 baseline generates numerous false negatives and boundary-ambiguous detections, whereas RSMamDet recovers the missing objects and produces cleaner bounding-box boundaries. In the nighttime scene (row c), the SFS module’s global contextual scanning enables reliable detection under illumination degradation that causes substantial performance drops in CNN-based baselines. These qualitative observations are consistent with the quantitative mAP₅₀ gain of 2.2% over the prior best method on VisDrone2019.

4.6. Ablation Study

4.6.1. Component Ablation

Table 6. Component ablation on DroneVehicle. √: module enabled. Bold: best result.

MobileMamba	SFS	DASI	PKINet	AMFF	UMC	mAP50(%)	GFLOPs	FPS
						65.8	60.4	217.4
√						67.2	53.7	205.8
√	√					68.5	49.2	196.3
√	√	√				69.8	46.8	194.5
√	√	√	√			71.1	44.1	190.7
√	√	√	√	√		72.1	42.3	187.3
√	√	√	√	√	√	72.6	42.3	186.2

GFLOPs measured at $640\times 640$ input; FPS measured on RTX 3090 with batch size 1. UMC loss has no effect on inference cost.

presents a step-by-step ablation study on DroneVehicle, including inference time and computational cost for each configuration. Starting from the RT-DETR-R18 baseline (65.8%), each proposed module brings consistent improvements: the MobileMamba backbone (+1.4%), SFS module (+1.3%), DASI module (+1.3%), PKINet encoder (+1.3%), AMFF module (+1.0%), and UMC loss (+0.5%). The full model achieves 72.6%, demonstrating that each component contributes meaningfully and the modules are complementary. Notably, the SFS module delivers the largest single-component gain among the non-backbone modules (+1.3% on top of MobileMamba), validating that linear-complexity global context modeling is the most critical missing ingredient when adapting a standard DETR encoder to high-resolution UAV imagery. The UMC loss provides an additional 0.5% mAP₅₀ gain at no inference cost, confirming that uncertainty-aware query regularization improves training stability and query selection quality independently of the architectural modules. The AMFF module’s +1.0% contribution is concentrated in the small-object categories: per-category AP analysis (Figure 8) shows the largest gains for Freight Car (+5.9%) and Truck (+6.1%), consistent with the known degradation of bilinear upsampling for objects occupying few pixels.

4.6.2. Backbone Comparison

Figure 10 shows qualitative detection results on VisDrone2019 for four backbone choices under otherwise identical settings. Highlighted regions (circles) reveal that MobileMamba consistently detects additional vehicles and reduces missed detections compared to ResNet-18, ResNet-50, and MobileNetV4 across diverse scenes. Quantitatively, MobileMamba achieves mAP₅₀ of 72.6%, outperforming ResNet-18 (65.8%), ResNet-50 (67.2%), and MobileNetV4 (68.1%), confirming the superiority of SSM-based linear-complexity modeling for UAV vehicle detection. The 4.5% mAP₅₀ advantage of MobileMamba over MobileNetV4—despite comparable parameter counts—directly demonstrates the benefit of replacing depthwise separable convolutions with selective state-space transitions, which capture long-range spatial dependencies that convolutions with bounded kernel sizes inherently cannot model.

4.6.3. Heatmap Visualization

Figure 11 shows Grad-CAM heatmap visualizations for the step-by-step ablation across three representative VisDrone2019 scenes. Column (b) uses the MobileMamba backbone with SFS (+2.7% over baseline). Each successive module sharpens the attention maps: DASI (c) improves cross-scale focus, PKINet (d) extends receptive field coverage to capture more vehicles, and AMFF (e) refines small-object attention, yielding the most concentrated and complete activation in the full RSMamDet model.

5. Discussion

RSMamDet achieves consistent improvements over the RT-DETR baseline through four synergistic contributions. The SFS module fundamentally addresses the efficiency bottleneck of DETR-based detectors in high-resolution UAV imagery by replacing quadratic self-attention with linear SSM scanning. DASI and PKINet collaborate to improve multi-scale and multi-receptive-field feature encoding, which is especially critical in dense UAV scenes. AMFF improves small-object detection by replacing content-agnostic upsampling with dynamic kernel-based reconstruction. The UMC loss further stabilizes training by suppressing uncertain query predictions.

Figure 12 provides a comprehensive multi-dimensional comparison between RSMamDet and four competitive baselines. The radar chart (Figure 12a) demonstrates that RSMamDet achieves the most balanced and consistently superior profile across all five metrics: mAP₅₀, mAP_50:95, inference speed, parameter efficiency, and small-object AP. The accuracy–speed bubble chart (Figure 12b) confirms that RSMamDet occupies the Pareto-optimal frontier, delivering the highest mAP₅₀ while maintaining near-real-time throughput at 186.2 FPS with only 19.8M parameters—a combination no prior method achieves simultaneously.

One limitation is that our current implementation processes the RGB modality only, while the DroneVehicle dataset contains paired RGB-infrared data [26]. The infrared modality was not incorporated in this work for two reasons: (1) our primary goal was to establish the effectiveness of SSM-based linear-complexity modeling for UAV detection under the standard RGB evaluation protocol, ensuring a fair comparison with existing single-modality baselines; and (2) integrating infrared data would require a dedicated cross-modal fusion module, which introduces additional architectural complexity that we consider orthogonal to the core contributions of this paper. Nonetheless, infrared imagery is highly valuable for nighttime and low-illumination vehicle detection, and we plan to integrate cross-modal fusion inspired by AMSRDet [3] and CSFADet [39] in future work.

Regarding deployment on resource-constrained platforms, we note that the reported 186.2 FPS is measured on an RTX 3090 desktop GPU, which differs substantially from embedded UAV platforms such as NVIDIA Jetson Orin or Jetson Xavier NX. Although a direct edge-device benchmark is beyond the scope of this paper, RSMamDet’s compact model size (19.8M parameters, 42.3 GFLOPs, 276 MB inference memory) is well within the capacity of modern embedded GPUs. As a reference, RT-DETR-R18 (20.0M parameters, 60.4 GFLOPs) has been reported to achieve approximately 30 FPS on Jetson Orin NX; given RSMamDet’s comparable parameter count and 30% lower GFLOPs, we estimate similar or better edge throughput. Future work will include systematic latency profiling on Jetson-series platforms with TensorRT optimization to validate practical onboard deployment feasibility.

We also acknowledge several additional limitations that warrant discussion. First, the modular architecture of RSMamDet, comprising multiple specialized components (SFS, DASI, PKINet, AMFF, UMC), may increase the difficulty of hyperparameter tuning and reproducibility compared to simpler single-backbone detectors. Second, while SSM/Mamba-based architectures have shown strong performance, they are not yet as extensively validated as CNNs and Transformers across diverse datasets; dataset-specific instabilities in the selective state transitions cannot be fully ruled out. Third, the Top-K query selection mechanism may occasionally discard challenging objects such as heavily occluded or extremely small targets whose initial confidence scores fall below the selection threshold; adaptive or hierarchical selection strategies could mitigate this issue. Fourth, despite the $\mathcal{O}\left(N\right)$ theoretical complexity of the SFS module, the practical computational cost is influenced by the multiple heavy modules operating sequentially, and the actual inference speedup over optimized attention implementations (e.g., FlashAttention) may be smaller than the asymptotic analysis suggests.

Additionally, RSMamDet exhibits two specific edge-case limitations worth acknowledging. First, in extremely low-altitude scenarios where individual vehicles span more than 100 pixels in width, the PKINet encoder’s maximum kernel size of $11\times 11$ may be insufficient to capture the full spatial extent of each target without capturing neighboring background context, which can degrade the precision of large-object bounding boxes. Second, the four-directional SS2D traversal imposes a raster scan ordering that assumes implicit spatial locality in the token sequence; this assumption may not hold for highly irregular or non-grid-like scene layouts encountered in off-road or disaster response scenarios. These observations motivate future extensions, including adaptive kernel size ranges that respond to predicted object scale distributions, topology-aware scan orderings that better match the spatial structure of diverse operational environments, and further compression via knowledge distillation [34] for embedded edge deployment.

6. Conclusions

We presented RSMamDet, an efficient end-to-end UAV remote sensing vehicle detector that integrates four complementary innovations into the RT-DETR framework. By replacing quadratic self-attention with linear SSM scanning (SFS), adaptively fusing cross-scale features (DASI), enriching encoder representations with multi-kernel inception blocks (PKINet), and enabling content-aware adaptive upsampling (AMFF), our model advances the accuracy-efficiency frontier for aerial vehicle detection. The novel UMC loss further stabilizes query selection in complex cluttered scenes. On DroneVehicle and VisDrone2019 benchmarks, RSMamDet achieves mAP₅₀ of 72.6% and 40.2% respectively, outperforming all 25 compared baselines while maintaining real-time inference at 186.2 FPS with only 19.8M parameters and 42.3 GFLOPs.

Despite these advances, several limitations remain. The current RGB-only design does not exploit the infrared modality available in DroneVehicle, which is particularly informative for nighttime detection. The PKINet encoder’s fixed kernel range ($3\times 3$ to $11\times 11$) may be suboptimal for scenarios dominated by very large objects, and the four-directional SS2D scan ordering may not generalize well to highly irregular scene layouts. Performance on extremely dense scenes with more than 80 vehicles per frame shows a moderate degradation of approximately 2.3% mAP₅₀ compared to scenes with typical density, suggesting room for improvement in handling extreme crowding.

Future work will pursue several directions: (1) RGB–infrared cross-modal fusion to improve nighttime and low-illumination detection; (2) rotated bounding-box prediction for better alignment with oblique-view aerial targets; (3) video-based temporal modeling to exploit inter-frame motion consistency for tracking and occlusion reasoning; (4) systematic deployment profiling on embedded UAV platforms (e.g., NVIDIA Jetson Orin) with TensorRT optimization; and (5) adaptive scan orderings and dynamic kernel ranges that respond to scene-specific object distributions. The modular architecture of RSMamDet makes these extensions straightforward to integrate without redesigning the core framework.

More broadly, we believe that the core design principles demonstrated in this work, namely linear-complexity global context modeling via SSMs, adaptive cross-scale fusion, and content-aware upsampling, are not limited to vehicle detection and can generalize to a wide range of aerial remote sensing tasks, including crowd counting, infrastructure inspection, and disaster damage assessment. As UAV platforms become increasingly prevalent in civilian and industrial applications, efficient yet accurate detection frameworks such as RSMamDet will play a critical role in enabling autonomous aerial intelligence at scale.