DCAM-DETR: Dual Cross-Attention Mamba Detection Transformer for RGB–Infrared Anti-UAV Detection

Abstract

The proliferation of unmanned aerial vehicles (UAVs) poses escalating security threats across critical infrastructures, necessitating robust real-time detection systems. Existing vision-based methods predominantly rely on single-modality data and exhibit significant performance degradation under challenging scenarios. To address these limitations, we propose DCAM-DETR, a novel multimodal detection framework that fuses RGB and thermal infrared modalities through an enhanced RT-DETR architecture integrated with state space models. Our approach introduces four innovations: (1) a MobileMamba backbone leveraging selective state space models for efficient long-range dependency modeling with linear complexity $\mathcal{O}\left(n\right)$; (2) Cross-Dimensional Attention (CDA) and Cross-Path Attention (CPA) modules capturing intermodal correlations across spatial and channel dimensions; (3) an Adaptive Feature Fusion Module (AFFM) dynamically calibrating multimodal feature contributions; and (4) a Dual-Attention Decoupling Module (DADM) enhancing detection head discrimination for small targets. Experiments on Anti-UAV300 demonstrate state-of-the-art performance with 94.7% mAP@0.5 and 78.3% mAP@0.5:0.95 at 42 FPS. Extended evaluations on FLIR-ADAS and KAIST datasets validate the generalization capacity across diverse scenarios.

1. Introduction

The exponential proliferation of unmanned aerial vehicles (UAVs) has fundamentally transformed numerous civilian and commercial domains, encompassing aerial cinematography, precision agriculture, logistics delivery, infrastructure inspection, and emergency response operations [1,2]. However, this technological democratization has concurrently engendered unprecedented security vulnerabilities, as malicious actors increasingly exploit UAVs for unauthorized surveillance, contraband smuggling, critical infrastructure disruption, and potential terrorist activities [3,4]. The Federal Aviation Administration (FAA) documented over 2800 UAV-related security incidents in 2023 alone, representing a 47% increase from the previous year [5]. Consequently, developing robust, efficient, and real-time anti-UAV detection systems has emerged as an imperative research priority for safeguarding public safety and national security. This challenge aligns with broader efforts in safety-critical IoT sensing and intelligent transportation systems, where AI models must support real-time, safety-related decisions in connected systems [6].

Conventional anti-UAV detection methodologies predominantly employ radar systems, acoustic sensors, or radio frequency (RF) analysis [7]. While these approaches demonstrate efficacy in specific operational contexts, they suffer from inherent limitations including prohibitive deployment costs, susceptibility to electromagnetic interference, limited spatial resolution, and difficulty in discriminating UAVs from avian species or other airborne objects [8]. Vision-based detection methods leveraging deep learning have emerged as compelling alternatives, offering cost-effectiveness, rich semantic information extraction, and precise spatial localization capabilities [9,10]. Nevertheless, UAV detection in unconstrained real-world environments presents formidable challenges: UAVs typically manifest as diminutive targets with limited discriminative visual features, frequently operating against complex heterogeneous backgrounds under varying illumination conditions, atmospheric perturbations, and partial occlusions [11,12].

Single-modality detection approaches exhibit fundamental limitations in addressing these challenges comprehensively. RGB cameras, while providing rich texture and chromatic information under adequate illumination, suffer severe performance degradation in low-light, nighttime, or adverse weather conditions [13]. Conversely, thermal infrared sensors capture thermal radiation signatures that remain robust across varying illumination conditions but lack sufficient textural detail for precise localization and classification [14]. This complementary nature of RGB and infrared modalities motivates multimodal fusion strategies that leverage the synergistic advantages of both sensing paradigms [15,16].

Recent advances in multimodal fusion for object detection have explored diverse architectural paradigms. Early fusion approaches employ simple concatenation or element-wise operations [17], which inadequately capture complex intermodal dependencies. Intermediate fusion methods leverage attention mechanisms [18,19] or adversarial learning [20] to enhance fusion quality, yet frequently incur substantial computational overhead or exhibit limited receptive field coverage. Late fusion strategies combine detection outputs from modality-specific detectors [21], potentially sacrificing fine-grained complementary information. Furthermore, most existing detection frameworks are constructed upon convolutional neural networks (CNNs) or standard vision transformers [22,23], which either lack global context modeling capability or impose quadratic computational complexity with respect to input resolution, limiting their applicability to high-resolution imagery or real-time deployment scenarios [24,25].

Despite these advances, current fusion-based UAV detectors exhibit several critical limitations. First, most methods employ fixed fusion strategies that apply uniform weights regardless of scene conditions, failing to adapt to varying modality reliability across different environments. Second, existing cross-modal attention mechanisms typically operate on a single dimension (spatial or channel), inadequately capturing the complex intermodal dependencies that span both dimensions. Third, transformer-based fusion approaches incur quadratic computational complexity $\mathcal{O}\left(n^2\right)$, limiting their applicability to real-time scenarios. Fourth, detection heads in current frameworks lack specialized mechanisms for small target enhancement, which is crucial for UAV detection where targets often occupy minimal pixel areas.

The recently proposed RT-DETR [26] framework has demonstrated remarkable performance in real-time object detection by synergistically combining the efficiency of YOLO-series detectors with the end-to-end detection paradigm of DETR [27]. Concurrently, state space models (SSMs), particularly the Mamba architecture [28], have emerged as a transformative paradigm for sequence modeling, offering linear computational complexity while preserving robust long-range dependency modeling capabilities. Vision Mamba variants [29,30] have successfully adapted SSMs to computer vision tasks, demonstrating competitive performance with vision transformers while achieving substantially improved computational efficiency. However, these architectural advances have not been systematically investigated for multimodal fusion in challenging detection scenarios such as anti-UAV applications.

To bridge this critical gap, we propose DCAM-DETR, a novel multimodal detection framework that synergistically integrates state space models with the RT-DETR architecture for robust real-time anti-UAV detection. Our approach introduces four principal innovations specifically designed for multimodal small target detection. First, we replace the conventional CNN backbone with a MobileMamba architecture that processes RGB and infrared streams through parallel Mamba encoders, enabling efficient global context aggregation with linear complexity $\mathcal{O}\left(n\right)$ compared to the quadratic complexity $\mathcal{O}\left(n^2\right)$ of standard self-attention mechanisms. Second, we design Cross-Dimensional Attention (CDA) and Cross-Path Attention (CPA) modules that explicitly model intermodal correlations across both spatial and channel dimensions, facilitating fine-grained feature alignment and complementary information extraction through parallel attention streams.

Third, we introduce an Adaptive Feature Fusion Module (AFFM) that dynamically calibrates fusion weights based on scene characteristics and modality reliability through learned gating mechanisms, ensuring optimal information integration across diverse environmental conditions. Fourth, we propose a Dual-Attention Decoupling Module (DADM) in the detection head that employs hierarchical dilated convolutions with parallel spatial and channel attention streams to enhance feature discrimination for small target detection. These components operate synergistically to achieve superior detection performance while maintaining real-time inference capability.

The principal contributions of this work are summarized as follows:

• We propose DCAM-DETR, a novel multimodal detection framework that integrates Mamba-based state space models with the RT-DETR architecture for efficient anti-UAV detection, achieving linear computational complexity while maintaining robust global context modeling capability through selective scan mechanisms.
• We design Cross-Dimensional Attention (CDA) and Cross-Path Attention (CPA) modules that explicitly capture intermodal correlations across spatial and channel dimensions through parallel attention streams, enabling fine-grained multimodal feature alignment and complementary information extraction.
• We introduce an Adaptive Feature Fusion Module (AFFM) that dynamically weights multimodal features based on scene-adaptive gating mechanisms, and a Dual-Attention Decoupling Module (DADM) that enhances detection head performance through hierarchical dilated convolutions and attention decomposition.
• Comprehensive experiments on Anti-UAV300, FLIR-ADAS, and KAIST datasets demonstrate that DCAM-DETR achieves state-of-the-art performance with 94.7% mAP@0.5 on Anti-UAV300, outperforming existing methods by substantial margins while maintaining real-time inference speed of 42 FPS.

The remainder of this paper is organized as follows. Section 2 reviews related work on UAV detection, multimodal fusion, and state space models. Section 3 presents the proposed DCAM-DETR framework in detail. Section 4 describes experimental setup and presents comprehensive results. Section 5 provides in-depth analysis and discussion. Finally, Section 6 concludes the paper with future research directions.

2. Related Work

2.1. Vision-Based UAV Detection

Vision-based UAV detection has garnered substantial research attention owing to its cost-effectiveness and rich semantic information extraction capabilities. Early methodologies employed classical computer vision techniques including background subtraction, optical flow estimation, and hand-crafted feature descriptors [12,31], which exhibited limited robustness against complex backgrounds and varying environmental conditions. The advent of deep learning has fundamentally transformed this domain, with CNN-based detectors such as Faster R-CNN [32], the YOLO series [33,34,35,36,37,38,39], and RetinaNet [40] being extensively adopted for UAV detection tasks.

However, UAV detection presents distinctive challenges that conventional object detectors inadequately address. UAVs typically manifest as small targets occupying minimal pixel areas, exhibit limited discriminative visual features, and demonstrate high-speed motion characteristics [9,10]. Recent investigations have explored specialized architectures for UAV detection. Huang et al. [10] introduced the Anti-UAV410 benchmark with thermal infrared imagery and proposed customized tracking schemes incorporating temporal consistency constraints. Jiang et al. [9] presented the Anti-UAV dataset and systematically evaluated various tracking algorithms under diverse environmental conditions. Zhao et al. [41] proposed multi-scale feature aggregation networks specifically designed for small UAV detection. Nevertheless, most existing methods focus on single-modality detection, fundamentally limiting their robustness across diverse environmental conditions. Our work addresses this limitation by leveraging multimodal fusion with efficient state space models.

2.2. Multimodal Fusion for Object Detection

Multimodal fusion, particularly RGB–infrared fusion, has demonstrated significant advantages in challenging detection scenarios by exploiting complementary information from heterogeneous sensors [13,42]. Fusion strategies can be categorized into early fusion, intermediate fusion, and late fusion paradigms based on the integration stage within the detection pipeline [43].

Early fusion approaches concatenate or combine raw inputs or low-level features from different modalities [17,44]. While computationally efficient, these methods inadequately capture complex intermodal dependencies and may introduce noise from modality-specific artifacts. Intermediate fusion methods integrate features at multiple abstraction levels, leveraging attention mechanisms [19,45,46], cross-modal transformers [47,48], or learnable fusion strategies [49,50] to adaptively combine multimodal information. Chen et al. [15] proposed probabilistic ensembling for multimodal object detection, while Wang et al. [16] introduced Mask-Guided Mamba Fusion for enhanced UAV detection in complex environments. Liu et al. [14] developed target-aware dual adversarial learning for infrared-visible fusion.

Late fusion strategies combine detection outputs from modality-specific detectors through ensemble techniques [21]. While preserving modality-specific representations, these approaches potentially sacrifice fine-grained complementary information available at intermediate feature levels. Recent works have explored transformer-based architectures for multimodal fusion [14,51], achieving improved performance but often suffering from quadratic computational complexity that limits real-time applicability. Our approach leverages Mamba’s linear complexity while achieving superior fusion quality through cross-dimensional attention mechanisms.

2.3. State Space Models and Mamba

State space models (SSMs) have emerged as efficient alternatives to transformers for sequence modeling, offering linear computational complexity while maintaining strong long-range dependency modeling capabilities [52,53]. The foundational structured state space sequence model (S4) [53] demonstrated that properly parameterized SSMs can effectively capture long-range dependencies in sequential data. The theoretical foundations of state space modeling for handling long-term correlations have been extensively studied in various domains, including network systems with generalized nonlinear stochastic processes [54]. The recently proposed Mamba architecture [28] introduces selective state spaces with input-dependent dynamics, enabling the model to dynamically adjust its behavior based on input content.

The core innovation of Mamba lies in its selective scan mechanism, which allows the model to focus on relevant information while filtering out irrelevant details through input-dependent parameterization of the state space matrices. This selectivity enables Mamba to achieve performance comparable to transformers while maintaining linear computational complexity with respect to sequence length. The continuous-time state space formulation is discretized for practical implementation:

$$\overline{\mathbf{A}}=\exp (\Delta \mathbf{A}),\overline{\mathbf{B}}={(\Delta \mathbf{A})}^{-1}(\exp (\Delta \mathbf{A})-\mathbf{I})·\Delta \mathbf{B}$$

(1)

where $\Delta$ represents the discretization step size, and $\mathbf{A}$, $\mathbf{B}$ are the state space matrices.

Vision Mamba [29] and VMamba [30] have successfully adapted Mamba to computer vision tasks, demonstrating competitive performance with vision transformers on image classification, object detection, and semantic segmentation. These works typically employ bidirectional or cross-scanning strategies to capture spatial relationships in 2D images. LocalMamba [55] introduced local scanning windows to enhance local feature extraction. MambaVision [56] proposed hybrid architectures combining Mamba with convolutional layers. However, existing vision Mamba architectures have not been systematically explored for multimodal fusion scenarios. Our MobileMamba backbone extends these ideas by introducing parallel Mamba encoders for different modalities with cross-modal interaction mechanisms.

Table 1. Comparison of DCAM-DETR with existing Mamba-based methods.

Method	Application	Modality	Fusion Strategy	Key Innovation
Mamba-UNet	Medical Seg.	Single	N/A	U-Net with SSM
VMamba	Classification	Single	N/A	Cross-scan 2D
Vision Mamba	General Vision	Single	N/A	Bidirectional SSM
FusionMamba	Image Fusion	Multi	Early	Dual-stream SSM
DCAM-DETR	Anti-UAV Det.	RGB + IR	Adaptive	CDA/CPA + AFFM

summarizes the key differences between DCAM-DETR and existing Mamba-based methods.

2.4. Detection Transformers

DETR [27] pioneered end-to-end object detection using transformers, eliminating hand-crafted components such as anchor generation and non-maximum suppression through bipartite matching and set prediction. Subsequent works addressed DETR’s limitations in convergence speed and computational efficiency. Deformable DETR [57] introduced deformable attention mechanisms that attend to a sparse set of sampling points, significantly improving convergence and performance. Conditional DETR [58] and DAB-DETR [59] enhanced query formulation for improved detection accuracy.

RT-DETR [26] represents a significant advancement in real-time detection transformers, introducing an efficient hybrid encoder that combines intra-scale feature interaction with cross-scale feature fusion, and uncertainty-minimal query selection for improved detection quality. RT-DETR achieves real-time performance while maintaining high accuracy, making it suitable for practical deployment scenarios. Our work builds upon RT-DETR’s efficient architecture while extending it to multimodal scenarios through the integration of MobileMamba backbone, cross-dimensional attention modules, and enhanced detection head components.

3. Methodology

3.1. Overall Architecture

Figure 1 illustrates the comprehensive architecture of our proposed DCAM-DETR framework. The detection pipeline comprises four principal components: (1) a MobileMamba backbone that processes RGB and infrared inputs through parallel state space model encoders with hierarchical feature extraction; (2) an Efficient Transformer Encoder enhanced with Cross-Dimensional Attention (CDA) and Cross-Path Attention (CPA) modules for multimodal feature fusion; (3) an Adaptive Feature Fusion Module (AFFM) that dynamically calibrates features from different scales and modalities; and (4) a detection head equipped with Dual-Attention Decoupling Module (DADM) for enhanced small target detection through the CCFF (Cross-scale Channel Feature Fusion) structure.

Given a spatially aligned RGB image ${\mathbf{I}}_{\mathrm{rgb}}\in {\mathbb{R}}^{H\times W\times 3}$ and a thermal infrared image ${\mathbf{I}}_{\mathrm{ir}}\in {\mathbb{R}}^{H\times W\times 1}$, the MobileMamba backbone first extracts hierarchical feature representations at multiple scales. The spatial alignment is achieved through hardware calibration and software preprocessing: RGB and thermal cameras are mounted on a rigid platform with factory-calibrated intrinsic and extrinsic parameters, and we apply homography transformation followed by bilinear interpolation to resize both modalities to the same resolution ($640\times 640$). For each modality $m\in \{\mathrm{rgb},\mathrm{ir}\}$, the backbone produces multi-scale features ${\left\{{\mathbf{F}}_m^{\left(i\right)}\right\}}_{i=3}^5$ corresponding to feature pyramid levels P3, P4, and P5 with spatial resolutions of ${\displaystyle \frac{H}{8}}\times {\displaystyle \frac{W}{8}}$, ${\displaystyle \frac{H}{16}}\times {\displaystyle \frac{W}{16}}$, and ${\displaystyle \frac{H}{32}}\times {\displaystyle \frac{W}{32}}$, respectively. These features are subsequently processed through the Efficient Transformer Encoder with CDA and CPA modules to capture intermodal correlations. The AFFM adaptively fuses multimodal features based on scene characteristics, producing unified representations for the detection head. Finally, the DADM-enhanced detection head generates bounding box predictions and class probabilities through the Uncertainty-minimal Query Selection mechanism.

3.2. MobileMamba Backbone

The MobileMamba backbone serves as the feature extraction foundation, processing RGB and infrared inputs through parallel Mamba encoders with shared architectural design but independent parameters. Unlike conventional CNN backbones that rely on local receptive fields with limited global context, or transformer-based backbones that incur quadratic computational complexity, MobileMamba leverages state space models to capture global dependencies with linear complexity $\mathcal{O}\left(n\right)$, making it particularly suitable for detecting small targets that require extensive contextual information.

3.2.1. Selective State Space Model Formulation

The theoretical foundation of our backbone is the continuous-time state space model, which maps an input sequence $x\left(t\right)\in \mathbb{R}$ to an output sequence $y\left(t\right)\in \mathbb{R}$ through a latent state $h\left(t\right)\in {\mathbb{R}}^N$:

$${\displaystyle \frac{dh\left(t\right)}{dt}}=\mathbf{A}h\left(t\right)+\mathbf{B}x\left(t\right),y\left(t\right)=\mathbf{C}h\left(t\right)+\mathbf{D}x\left(t\right)$$

(2)

where $\mathbf{A}\in {\mathbb{R}}^{N\times N}$ is the state transition matrix, $\mathbf{B}\in {\mathbb{R}}^{N\times 1}$ is the input projection matrix, $\mathbf{C}\in {\mathbb{R}}^{1\times N}$ is the output projection matrix, and $\mathbf{D}\in \mathbb{R}$ is the feedthrough coefficient. For practical implementation, this continuous system is discretized using the zero-order hold (ZOH) method with step size $\Delta$:

$$\overline{\mathbf{A}}=\exp (\Delta \mathbf{A}),\overline{\mathbf{B}}={(\Delta \mathbf{A})}^{-1}(\exp (\Delta \mathbf{A})-\mathbf{I})·\Delta \mathbf{B}$$

(3)

The discretized state space model operates as a recurrence:

$$h_k=\overline{\mathbf{A}}{h}_{k-1}+\overline{\mathbf{B}}{x}_k,y_k=\mathbf{C}{h}_k+\mathbf{D}{x}_k$$

(4)

Intuitively, the discretization converts continuous-time dynamics into a form suitable for digital computation. The matrix $\overline{\mathbf{A}}$ captures how the hidden state evolves over one time step, while $\overline{\mathbf{B}}$ determines how new inputs are incorporated. The step size $\Delta$ controls temporal resolution.

The key innovation of Mamba is the selective mechanism that makes $\mathbf{B}$, $\mathbf{C}$, and $\Delta$ input-dependent:

$$\mathbf{B}={\mathrm{Linear}}_B\left(x\right),\mathbf{C}={\mathrm{Linear}}_C\left(x\right),\Delta =\mathrm{softplus}({\mathrm{Linear}}_{\Delta }\left(x\right)+\mathrm{Broadcast}\left(p_{\Delta }\right))$$

(5)

where $p_{\Delta }$ is a learnable parameter. This selectivity enables the model to dynamically decide which information to store in the hidden state (controlled by $\mathbf{B}$), which stored information to output (controlled by $\mathbf{C}$), and how quickly to update the state (controlled by $\Delta$), effectively filtering irrelevant information while preserving salient features.

3.2.2. SS2D Block for 2D Visual Processing

To adapt the 1D selective state space model for 2D visual data, we employ the SS2D (Selective Scan 2D) block illustrated in Figure 2. The SS2D block extends the selective scan mechanism to 2D spatial data through a cross-scan strategy that processes the feature map along four directions: left-to-right, right-to-left, top-to-bottom, and bottom-to-top.

Given an input feature map $\mathbf{X}\in {\mathbb{R}}^{H\times W\times C}$, the scan expanding operation flattens the 2D spatial dimensions into four 1D sequences corresponding to different scanning directions:

$${\mathbf{X}}^{\left(d\right)}={\mathrm{ScanExpand}}^{\left(d\right)}\left(\mathbf{X}\right),d\in \{\mathrm{lr},\mathrm{rl},\mathrm{tb},\mathrm{bt}\}$$

(6)

where ${\mathbf{X}}^{\left(d\right)}\in {\mathbb{R}}^{(H·W)\times C}$ represents the flattened sequence for direction d. Each sequence is independently processed through the S6 Block:

$${\mathbf{Y}}^{\left(d\right)}=\mathrm{S}6\mathrm{Block}\left({\mathbf{X}}^{\left(d\right)}\right)$$

(7)

The outputs from all four directions are merged through a learnable fusion mechanism:

$$\mathbf{Y}=\mathrm{ScanMerge}({\mathbf{Y}}^{\left(\mathrm{lr}\right)},{\mathbf{Y}}^{\left(\mathrm{rl}\right)},{\mathbf{Y}}^{\left(\mathrm{tb}\right)},{\mathbf{Y}}^{\left(\mathrm{bt}\right)})$$

(8)

The cross-scan strategy addresses a fundamental challenge: 1D state space models process sequences sequentially, but images have 2D spatial structure. By scanning in four directions, each pixel aggregates information from all spatial directions, enabling comprehensive spatial context modeling.

This bidirectional cross-scanning strategy ensures comprehensive spatial context aggregation while maintaining linear computational complexity $\mathcal{O}\left(HWC\right)$, compared to the quadratic complexity $\mathcal{O}\left(H^2{W}^{2}C\right)$ of standard self-attention mechanisms.

3.2.3. S6 Block Architecture

The S6 (Selective State Space Sequence) Block, illustrated in Figure 3, constitutes the core computational unit of our MobileMamba backbone. The S6 Block incorporates layer normalization, linear projections, depthwise convolution, SiLU activation, the SS2D selective scan operation, and residual connections for stable gradient flow.

The complete S6 Block operation is formulated as:

$$\begin{array}{cc}\hfill {\mathbf{Z}}_1& =\mathrm{LN}\left(\mathbf{X}\right)\hfill \\ \hfill {\mathbf{Z}}_2& =SS2DBlock\left({\mathbf{Z}}_1\right)+\mathbf{X}\hfill \\ \hfill {\mathbf{Z}}_3& =\mathrm{LN}\left({\mathbf{Z}}_2\right)\hfill \\ \hfill \mathbf{Y}& =\mathrm{FFN}\left({\mathbf{Z}}_3\right)+{\mathbf{Z}}_2\hfill \end{array}$$

(9)

The internal SS2D Block processing follows:

$$\begin{array}{cc}\hfill \mathbf{U}& =\mathrm{Linear}\left({\mathbf{Z}}_1\right)\hfill \\ \hfill \mathbf{V}& =\mathrm{SiLU}\left({\mathrm{DWConv}}_{3\times 3}\left(\mathrm{Linear}\left({\mathbf{Z}}_1\right)\right)\right)\hfill \\ \hfill \mathbf{W}& =SS2D\left(\mathbf{V}\right)\hfill \\ \hfill \mathbf{O}& =\mathrm{Linear}\left(\mathrm{LN}\right(\mathbf{W})\odot \mathbf{U})\hfill \end{array}$$

(10)

where ⊙ denotes element-wise multiplication, and the depthwise convolution introduces local inductive bias complementing the global context modeling of the selective scan operation.

The MobileMamba backbone employs a hierarchical architecture with four stages, progressively downsampling spatial resolution while increasing channel dimensions. For RGB and infrared modalities, we utilize parallel Mamba encoders with shared architecture but independent parameters, enabling modality-specific feature learning while maintaining architectural consistency.

3.3. Cross-Dimensional Attention Modules

Effective multimodal fusion requires capturing complex intermodal dependencies that span both spatial and channel dimensions. Traditional fusion approaches, such as simple concatenation or element-wise operations, fail to model the intricate correlations between heterogeneous modalities. To address this limitation, we propose two complementary attention modules: Cross-Dimensional Attention (CDA) for spatial correlation modeling and Cross-Path Attention (CPA) for channel-wise dependency capture. These modules are integrated into the Efficient Transformer Encoder, operating in parallel to provide comprehensive multimodal feature alignment.

3.3.1. Cross-Dimensional Attention (CDA)

The CDA module is designed to capture spatial correlations between RGB and infrared features by learning position-dependent attention weights that identify corresponding regions across modalities. The key insight is that spatially aligned multimodal images share semantic correspondences at specific locations, and explicitly modeling these spatial relationships enhances feature alignment quality.

Given RGB features ${\mathbf{F}}_{\mathrm{rgb}}\in {\mathbb{R}}^{H\times W\times C}$, CDA employs a multi-branch architecture combining spatial self-attention, frequency-domain analysis, and local feature extraction, as illustrated in the left panel of Figure 4. The spatial self-attention branch computes position-wise correlations through the scaled dot-product attention mechanism:

$${\mathbf{A}}_{\mathrm{spatial}}=\mathrm{Softmax}\left({\displaystyle \frac{{\mathbf{Q}}_s{\mathbf{K}}_s^{\top }}{\sqrt{d_k}}}\right){\mathbf{V}}_s,\mathrm{where}{\mathbf{Q}}_s,{\mathbf{K}}_s,{\mathbf{V}}_s={\varphi }_{\mathrm{split}}\left({\mathbf{W}}_{\exp }{\mathbf{F}}_{\mathrm{rgb}}\right)$$

(11)

where ${\mathbf{W}}_{\exp }\in {\mathbb{R}}^{3C\times C}$ is the channel expansion projection, ${\varphi }_{\mathrm{split}}(·)$ partitions the expanded features into query, key, and value components, and $d_k=C$ is the scaling factor for numerical stability.

To capture complementary frequency-domain information that may be obscured in the spatial domain, we incorporate a frequency projection branch based on the discrete Fourier transform:

$${\mathbf{F}}_{\mathrm{freq}}={\mathcal{F}}^{-1}\left({\mathbf{W}}_{\mathrm{freq}}\odot \mathcal{F}\left({\mathbf{F}}_{\mathrm{rgb}}\right)\right)$$

(12)

where $\mathcal{F}(·)$ and ${\mathcal{F}}^{-1}(·)$ denote the 2D FFT and inverse FFT operations, and ${\mathbf{W}}_{\mathrm{freq}}$ represents learnable frequency-domain filters. This branch captures periodic patterns and global structural information that complement the local spatial attention.

The final CDA output integrates spatial attention, frequency features, and local context through adaptive gating:

$${\mathbf{F}}_{\mathrm{CDA}}={\mathbf{W}}_o\left[({\mathbf{F}}_{\mathrm{sp}}+{\mathbf{F}}_{\mathrm{freq}})\odot \sigma \left({\mathbf{W}}_s{\mathbf{F}}_{\mathrm{rgb}}\right)+{\mathbf{V}}_{\mathrm{local}}\odot \sigma \left({\mathbf{W}}_c({\mathbf{F}}_{\mathrm{sp}}\oplus {\mathbf{F}}_{\mathrm{freq}})\right)\right]$$

(13)

where ${\mathbf{F}}_{\mathrm{sp}}$ is the spatially projected attention output, ${\mathbf{V}}_{\mathrm{local}}={\mathrm{DWConv}}_{3\times 3}\left({\mathbf{W}}_1{\mathbf{F}}_{\mathrm{rgb}}\right)$ captures local context through depthwise separable convolution, $\sigma (·)$ denotes the sigmoid activation, and ⊕ represents global average pooling followed by channel projection.

3.3.2. Cross-Path Attention (CPA)

While CDA focuses on spatial correlations, the CPA module captures channel-wise dependencies that encode semantic relationships between feature channels across modalities. Different channels in deep features correspond to different semantic concepts, and modeling their intermodal correlations enables more effective information exchange.

Operating on infrared features ${\mathbf{F}}_{\mathrm{ir}}\in {\mathbb{R}}^{H\times W\times C}$, CPA computes channel attention by treating each channel as a token and applying self-attention along the channel dimension:

$${\mathbf{A}}_{\mathrm{channel}}=\mathrm{Softmax}\left({\displaystyle \frac{{\mathbf{Q}}_c^{\top }{\mathbf{K}}_c}{\sqrt{HW}}}\right){\mathbf{V}}_c^{\top },\mathrm{where}{\mathbf{Q}}_c,{\mathbf{K}}_c,{\mathbf{V}}_c\in {\mathbb{R}}^{HW\times C}$$

(14)

where the query, key, and value matrices are obtained by reshaping the linearly projected infrared features from ${\mathbb{R}}^{H\times W\times 3C}$ to ${\mathbb{R}}^{HW\times C}$ for each component. The scaling factor $\sqrt{HW}$ normalizes the attention scores based on the spatial dimension.

The CPA output combines channel-attended features with spatially-gated local information:

$${\mathbf{F}}_{\mathrm{CPA}}={\mathbf{W}}_o^{\prime }\left[{\mathbf{F}}_{\mathrm{cp}}\odot \sigma \left({\mathbf{W}}_c^{\prime }{\mathbf{F}}_{\mathrm{ir}}\right)+{\mathbf{V}}_{\mathrm{ir}}\odot \sigma \left({\mathbf{W}}_{\mathrm{sp}}{\mathbf{F}}_{\mathrm{cp}}\right)\right]$$

(15)

where ${\mathbf{F}}_{\mathrm{cp}}$ is the channel-projected attention output, ${\mathbf{V}}_{\mathrm{ir}}={\mathrm{DWConv}}_{3\times 3}\left({\mathbf{W}}_1^{\prime }{\mathbf{F}}_{\mathrm{ir}}\right)$ provides local infrared context, and ${\mathbf{W}}_{\mathrm{sp}}\in {\mathbb{R}}^{1\times C}$ generates spatial attention weights through channel-wise projection.

The dual attention design ensures comprehensive multimodal feature alignment: CDA identifies where to attend by modeling spatial correspondences, while CPA determines what to attend by capturing semantic channel correlations. Together, they provide complementary perspectives for effective RGB–infrared fusion.

3.3.3. Why Cross-Dimensional Attention Helps for Small UAVs

Small UAVs present unique detection challenges due to their limited pixel coverage and lack of discriminative local features. Our cross-dimensional attention design specifically addresses these challenges through three mechanisms. First, CDA’s spatial attention aggregates contextual information from surrounding regions, providing discriminative cues from the broader scene context when local features are insufficient. Second, CPA’s channel attention identifies which feature channels from each modality are most informative for small targets—for small UAVs, thermal signatures in IR often provide stronger signals than RGB texture, and CPA learns to emphasize these channels adaptively. Third, the parallel attention streams capture correlations at different granularities, enabling the model to leverage both fine-grained local features and coarse global patterns essential for small target detection.

3.4. Adaptive Feature Fusion Module (AFFM)

A fundamental challenge in multimodal fusion is the varying reliability of different modalities across environmental conditions. In low-light scenarios, infrared features provide more discriminative information, while in well-illuminated scenes with rich texture, RGB features may be more informative. Fixed fusion strategies that apply uniform weights fail to adapt to these variations, leading to suboptimal performance. To address this challenge, we propose the Adaptive Feature Fusion Module (AFFM), illustrated in Figure 5, that dynamically calibrates multimodal feature contributions through learned scene-dependent gating mechanisms.

Given RGB features ${\mathbf{F}}^g\in {\mathbb{R}}^{H\times W\times C}$ and infrared features ${\mathbf{F}}^l\in {\mathbb{R}}^{H\times W\times C}$ at a specific pyramid level, AFFM computes content-aware gating weights that reflect the local reliability of each modality. The gating mechanism is formulated as a learnable soft attention over modality contributions:

$${\mathbf{W}}^m=\sigma \left(\mathrm{BN}\left({\mathbf{W}}_g^m\ast {\mathbf{F}}^m\right)\right),m\in \{g,l\}$$

(16)

where ${\mathbf{W}}_g^m\in {\mathbb{R}}^{C\times C\times 1\times 1}$ denotes the $1\times 1$ convolutional kernel for modality m, ∗ represents convolution, and $\sigma (·)$ is the sigmoid activation that constrains weights to $[0,1]$.

The fusion process decomposes multimodal information into complementary and distinctive components. The complementary fusion captures shared information through weighted combination:

$${\mathbf{F}}_{\mathrm{comp}}=\left[\begin{array}{c}{\mathbf{W}}^g\odot {\mathbf{F}}^g+(\mathbf{1}-{\mathbf{W}}^l)\odot {\mathbf{F}}^l\\ {\mathbf{W}}^l\odot {\mathbf{F}}^l+(\mathbf{1}-{\mathbf{W}}^g)\odot {\mathbf{F}}^g\end{array}\right]$$

(17)

where $\mathbf{1}$ denotes an all-ones tensor of appropriate dimensions. This formulation ensures that when one modality is unreliable (low gate value), the fusion relies more heavily on the other modality.

The distinctive fusion captures modality-specific information through weighted subtraction, which highlights unique features present in only one modality:

$${\mathbf{F}}_{\mathrm{dist}}=\left[\begin{array}{c}{\mathbf{W}}^g\odot {\mathbf{F}}^g-{\mathbf{W}}^l\odot {\mathbf{F}}^l\\ {\mathbf{W}}^l\odot {\mathbf{F}}^l-{\mathbf{W}}^g\odot {\mathbf{F}}^g\end{array}\right]$$

(18)

The final fused representation aggregates both components through channel concatenation and convolutional refinement:

$$\mathbf{Z}={\varphi }_{\mathrm{refine}}\left(\left[{\mathbf{F}}_{\mathrm{comp}};{\mathbf{F}}_{\mathrm{dist}}\right]\right)=\mathrm{ReLU}\left(\mathrm{BN}\left({\mathbf{W}}_r\ast \left[{\mathbf{F}}_{\mathrm{comp}};{\mathbf{F}}_{\mathrm{dist}}\right]\right)\right)$$

(19)

where ${\mathbf{W}}_r\in {\mathbb{R}}^{C\times 4C\times 3\times 3}$ is the refinement convolution kernel that reduces the concatenated $4C$ channels back to C while capturing local spatial context. This design enables AFFM to adaptively emphasize reliable modality information while preserving distinctive features from both modalities.

3.5. Dual-Attention Decoupling Module (DADM)

Small target detection presents unique challenges that standard detection heads inadequately address. UAVs often occupy minimal pixel areas and lack sufficient discriminative features when considered in isolation. However, they become more distinguishable when contextual information from surrounding regions is incorporated. Furthermore, the detection head must simultaneously handle classification and localization tasks, which benefit from different feature characteristics. To address these challenges, we propose the Dual-Attention Decoupling Module (DADM), shown in Figure 6, that enhances detection capability through hierarchical multi-scale context aggregation and decoupled spatial-channel attention processing.

Given input features $\mathbf{F}\in {\mathbb{R}}^{H\times W\times C}$, DADM first constructs a multi-scale feature pyramid through cascaded dilated convolutions with progressively increasing receptive fields. The dilated convolution operation with dilation rate r is defined as:

$$\left(\mathbf{W}{\ast }_r\mathbf{F}\right)\left(p\right)=\sum _k\mathbf{W}\left(k\right)·\mathbf{F}(p+r·k)$$

(20)

where p denotes the spatial position, k indexes the kernel elements, and r controls the spacing between kernel elements. This formulation enables exponentially expanding receptive fields without increasing parameters.

The hierarchical feature extraction applies cascaded dilated convolutions with rates $\mathbf{r}=[1,2,4,4]$:

$${\mathbf{F}}_i=\left\{\begin{array}{cc}{\mathbf{W}}_1{\ast }_1\mathbf{F}\hfill & i=1\hfill \\ {\mathbf{W}}_i{\ast }_{r_i}{\mathbf{F}}_{i-1}\hfill & i\in \{2,3,4\}\hfill \end{array}\right.$$

(21)

where ${\mathbf{W}}_i\in {\mathbb{R}}^{C\times C\times 3\times 3}$ are learnable convolution kernels. The multi-scale features are aggregated through element-wise summation to preserve information from all receptive field scales:

$${\mathbf{F}}_{\mathrm{ms}}=\underset{i=1}{\overset{4}{⨁}}{\mathbf{F}}_i={\mathbf{F}}_1+{\mathbf{F}}_2+{\mathbf{F}}_3+{\mathbf{F}}_4$$

(22)

The Dual-Attention Module (DAM) computes spatial attention through a factorized query–key interaction that reduces computational complexity from $\mathcal{O}\left(H^2{W}^2\right)$ to $\mathcal{O}\left(HW\right)$. The attention mechanism is formulated as:

$${\mathbf{A}}_{\mathrm{DAM}}={\varphi }_H({\mathbf{W}}_q\ast {\mathbf{F}}_{\mathrm{ms}})\otimes {\varphi }_W{({\mathbf{W}}_k\ast {\mathbf{F}}_{\mathrm{ms}})}^{\top }$$

(23)

where ${\mathbf{W}}_q,{\mathbf{W}}_k\in {\mathbb{R}}^{1\times C\times 1\times 1}$ are pointwise convolution kernels, ${\varphi }_H:{\mathbb{R}}^{H\times W}\to {\mathbb{R}}^{H\times 1}$ and ${\varphi }_W:{\mathbb{R}}^{H\times W}\to {\mathbb{R}}^{1\times W}$ are learnable projection functions implemented as fully connected layers after spatial reshaping, and ⊗ denotes outer product.

The final DADM output combines attention-weighted features with the original multi-scale representation through a residual connection:

$${\mathbf{F}}_{\mathrm{DADM}}={\mathbf{W}}_3\ast \left(\sigma ({\mathbf{W}}_a\ast {\mathbf{A}}_{\mathrm{DAM}})\odot {\mathbf{F}}_{\mathrm{ms}}\right)+{\mathbf{W}}_p\ast {\mathbf{F}}_{\mathrm{ms}}$$

(24)

where ${\mathbf{W}}_3\in {\mathbb{R}}^{C\times C\times 3\times 3}$ and ${\mathbf{W}}_p\in {\mathbb{R}}^{C\times C\times 1\times 1}$ are convolution kernels for the attention and residual branches, respectively, and ${\mathbf{W}}_a$ projects the attention map to match the feature channel dimension. This design enables DADM to focus on discriminative spatial regions while preserving rich multi-scale contextual information essential for small target detection.

3.6. Loss Function

Following the RT-DETR framework, we employ a composite loss function combining classification loss, bounding box regression loss, and Generalized IoU (GIoU) loss:

$${\mathcal{L}}_{\mathrm{total}}={\lambda }_{\mathrm{cls}}{\mathcal{L}}_{\mathrm{cls}}+{\lambda }_{\mathrm{bbox}}{\mathcal{L}}_{\mathrm{bbox}}+{\lambda }_{\mathrm{giou}}{\mathcal{L}}_{\mathrm{giou}}$$

(25)

The classification loss employs focal loss [40] to address class imbalance:

$${\mathcal{L}}_{\mathrm{cls}}=-{\alpha }_t{(1-p_t)}^{\gamma }log\left(p_t\right)$$

(26)

where $p_t$ is the predicted probability for the ground-truth class, ${\alpha }_t$ is the balancing factor, and $\gamma$ is the focusing parameter.

The bounding box regression loss uses L1 loss:

$${\mathcal{L}}_{\mathrm{bbox}}=\sum _{i=1}^4|b_i-{\widehat{b}}_i|$$

(27)

where $b_i$ and ${\widehat{b}}_i$ are the ground-truth and predicted bounding box coordinates, respectively.

The GIoU loss [60] provides scale-invariant localization supervision:

$${\mathcal{L}}_{\mathrm{giou}}=1-\mathrm{GIoU}(b,\widehat{b})=1-\left(\mathrm{IoU}(b,\widehat{b})-{\displaystyle \frac{|C\setminus (b\cup \widehat{b}\left)\right|}{\left|C\right|}}\right)$$

(28)

where C is the smallest enclosing box containing both b and $\widehat{b}$.

The loss weights are set to ${\lambda }_{\mathrm{cls}}=2.0$, ${\lambda }_{\mathrm{bbox}}=5.0$, and ${\lambda }_{\mathrm{giou}}=2.0$ following standard practice.

4. Experiments

4.1. Datasets

We evaluate DCAM-DETR on three multimodal datasets spanning different application domains to comprehensively assess detection performance and generalization capability.

Anti-UAV300 Dataset. Our primary evaluation is conducted on the Anti-UAV300 dataset [9], a challenging multimodal benchmark specifically designed for UAV detection. The dataset comprises 300 video sequences with spatially aligned RGB and thermal infrared imagery, captured under diverse environmental conditions including varying illumination (daytime, dusk, nighttime), weather changes (clear, cloudy, foggy), and complex backgrounds (urban, rural, sky). The dataset contains approximately 45,000 frame pairs with bounding box annotations for UAVs of various sizes (small, medium, large) and types (quadcopter, fixed-wing, helicopter). We follow the standard train/validation/test split of 200/50/50 sequences, ensuring no overlap in capture sessions between splits.

FLIR-ADAS Dataset. To evaluate generalization capability across application domains, we additionally test on the FLIR-ADAS dataset [61], which contains 10,228 thermal images and corresponding RGB images captured from automotive scenarios. The dataset includes annotations for vehicles, pedestrians, bicycles, and other road users. While not specifically designed for UAV detection, this dataset provides diverse multimodal data for validating our fusion approach in different environmental contexts.

KAIST Multispectral Dataset. We further validate on the KAIST Multispectral Pedestrian Detection Benchmark [62], containing 95,328 RGB-thermal image pairs captured in various traffic scenarios across daytime and nighttime conditions. This dataset tests our method’s robustness across different application domains and illumination conditions.

4.2. Implementation Details

Our model is implemented in PyTorch 2.0 and trained on 4 NVIDIA A100 GPUs with 80 GB memory each. We employ the AdamW optimizer [63] with an initial learning rate of $1\times {10}^{-4}$, weight decay of $1\times {10}^{-4}$, and effective batch size of 16 (4 per GPU). The learning rate follows a cosine annealing schedule [64] over 100 epochs with 5 epochs of linear warmup. Input images are resized to $640\times 640$ resolution with standard data augmentation including random horizontal flipping (probability 0.5), color jittering (brightness, contrast, saturation, hue), mosaic augmentation [34], and mixup [65] with probability 0.15.

The MobileMamba backbone is initialized with ImageNet-1K pre-trained weights when available. The Efficient Transformer Encoder uses 6 encoder layers with 8 attention heads and hidden dimension 256. The detection head employs 6 decoder layers with 300 object queries. For the DADM module, dilated convolution rates are set to $r\in \{1,2,4,4\}$ for the four hierarchical levels. All experiments use mixed-precision training (FP16) for computational efficiency.

4.3. Evaluation Metrics

We adopt standard object detection metrics following the COCO evaluation protocol for comprehensive performance assessment. The primary metrics include Mean Average Precision (mAP) at different Intersection over Union (IoU) thresholds, inference speed, and model complexity measures.

Intersection over Union (IoU). IoU quantifies the overlap between the predicted bounding box $\widehat{b}$ and the ground-truth bounding box b, serving as the fundamental criterion for determining detection correctness:

$$\mathrm{IoU}(b,\widehat{b})={\displaystyle \frac{|b\cap \widehat{b}|}{|b\cup \widehat{b}|}}={\displaystyle \frac{\mathrm{Area}\mathrm{of}\mathrm{Intersection}}{\mathrm{Area}\mathrm{of}\mathrm{Union}}}$$

(29)

A detection is considered a true positive if $\mathrm{IoU}(b,\widehat{b})\ge \tau$, where $\tau$ is the IoU threshold.

Precision and Recall. For a given IoU threshold and confidence threshold, precision measures the proportion of correct detections among all detections, while recall measures the proportion of ground-truth objects that are successfully detected:

$$\mathrm{Precision}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}},\mathrm{Recall}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}}$$

(30)

where TP, FP, and FN denote true positives, false positives, and false negatives, respectively.

Average Precision (AP). AP summarizes the precision–recall curve by computing the area under the curve. We employ the 101-point interpolation method:

$$\mathrm{AP}={\displaystyle \frac{1}{101}}\sum _{r\in \{0,0.01,...,1\}}{p}_{\mathrm{interp}}\left(r\right),\mathrm{where}{p}_{\mathrm{interp}}\left(r\right)=\underset{\tilde{r}\ge r}{max}p\left(\tilde{r}\right)$$

(31)

where $p\left(r\right)$ is the precision at recall level r, and $p_{\mathrm{interp}}\left(r\right)$ is the interpolated precision.

Mean Average Precision (mAP). mAP averages AP across all object classes. For single-class UAV detection, mAP equals AP. We report two standard metrics:

• mAP@0.5: AP computed at IoU threshold $\tau =0.5$, which is lenient and focuses on detection capability.
• mAP@0.5:0.95: AP averaged across IoU thresholds from 0.5 to 0.95 with step 0.05, which requires precise localization:

  $$mAP@0.5:0.95={\displaystyle \frac{1}{10}}\sum _{\tau \in \{0.5,0.55,...,0.95\}}{\mathrm{AP}}_{\tau }$$

  (32)

Inference Speed (FPS). Frames per second measures real-time detection capability, computed as the inverse of average inference time per image. All FPS measurements are conducted on a single NVIDIA RTX 3090 GPU with batch size 1 and input resolution $640\times 640$.

Model Complexity. We report model parameters in millions (M) and floating-point operations (FLOPs) in giga-operations (G) to assess computational requirements for deployment.

4.4. Comparison with State-of-the-Art Methods

We conduct comprehensive comparisons with state-of-the-art object detection methods spanning both single-modality approaches (RGB-only and infrared-only) and multimodal fusion methods. For single-modality baselines, we include representative detectors from the YOLO series (YOLOv5 through YOLOv10) and the detection transformer family (RT-DETR). For multimodal methods, we compare against established fusion approaches including DenseFuse, U2Fusion, TarDAL, M3FD, CFT, and TransFuse. All baseline methods are retrained on the Anti-UAV300 dataset using their official implementations with optimized hyperparameters to ensure fair comparison.

Table 2. Performance comparison on Anti-UAV300 dataset. Best results are in bold, second best are underlined. ^† indicates methods retrained on Anti-UAV300 with official implementations.

Method	Modality	mAP@0.5	mAP@0.5:0.95	FPS	Params(M)
Single-Modality Methods (RGB)
YOLOv5-L † [35]	RGB	78.3	54.2	68	46.5
YOLOv7 † [37]	RGB	81.5	57.8	52	37.2
YOLOv8-L † [66]	RGB	82.8	58.9	58	43.7
YOLOv9-C † [38]	RGB	83.2	59.4	48	51.8
YOLOv10-L † [39]	RGB	84.1	60.7	55	44.3
RT-DETR-L † [26]	RGB	85.7	62.1	45	42.3
Single-Modality Methods (Infrared)
YOLOv5-L † [35]	IR	74.6	51.3	68	46.5
YOLOv7 † [37]	IR	77.8	53.9	52	37.2
YOLOv8-L † [66]	IR	79.4	55.6	58	43.7
RT-DETR-L † [26]	IR	82.4	58.7	45	42.3
Multimodal Fusion Methods
DenseFuse † [17]	RGB + IR	84.9	60.3	35	52.1
U2Fusion † [44]	RGB + IR	86.2	61.8	32	48.7
TarDAL † [14]	RGB + IR	88.5	64.2	28	55.3
M3FD † [14]	RGB + IR	90.3	67.5	38	49.8
CFT † [67]	RGB + IR	91.2	69.8	31	58.2
TransFuse † [47]	RGB + IR	92.1	71.2	25	63.4
Recent State-of-the-Art (CVPR/ICCV/ECCV 2023-2024)
MBNet † (ICCV’23)	RGB + IR	90.5	68.9	32	51.2
BAANet † (ICCV’23)	RGB + IR	91.2	69.8	30	54.6
CAT-Det † (CVPR’24)	RGB + IR	91.8	70.4	28	56.8
SuperYOLO † (ECCV’24)	RGB + IR	92.3	72.1	35	48.9
CIAN † (CVPR’24)	RGB + IR	92.8	73.5	26	61.3
DCAM-DETR (Ours)	RGB + IR	94.7	78.3	42	47.6

presents the quantitative comparison results.

As demonstrated in Table 2, DCAM-DETR achieves superior performance across all evaluation metrics. Compared to the best single-modality method (RT-DETR on RGB), our approach improves mAP@0.5 by 9.0 percentage points (85.7% → 94.7%) and mAP@0.5:0.95 by 16.2 percentage points (62.1% → 78.3%), demonstrating the substantial advantage of multimodal fusion for UAV detection. Among multimodal methods, DCAM-DETR outperforms the previous best method (TransFuse) by 2.6 percentage points in mAP@0.5 and 7.1 percentage points in mAP@0.5:0.95, while achieving significantly higher inference speed (42 FPS vs. 25 FPS) and fewer parameters (47.6 M vs. 63.4 M).

The performance improvements are particularly notable for the more stringent mAP@0.5:0.95 metric, which requires precise localization across multiple IoU thresholds. This indicates that DCAM-DETR not only detects UAVs more reliably but also localizes them more accurately, which is critical for downstream applications such as tracking and interception.

To provide a comprehensive multi-dimensional comparison, Figure 7 presents a radar chart visualizing the performance of different methods across six key dimensions: mAP@0.5, mAP@0.5:0.95, inference speed, parameter efficiency, small target detection accuracy, and nighttime scene performance. DCAM-DETR demonstrates superior performance across all dimensions, particularly excelling in the challenging small target and nighttime scenarios where multimodal fusion provides the greatest advantage.

4.5. Qualitative Results

Beyond quantitative metrics, we provide qualitative analysis to demonstrate DCAM-DETR’s detection capability under diverse real-world conditions. Figure 8 presents representative detection results on challenging scenarios from the Anti-UAV300 dataset, encompassing low-light and nighttime conditions where RGB imagery provides limited discriminative information, complex urban backgrounds with cluttered visual patterns, small target scales where UAVs occupy minimal pixel areas, and varying weather conditions including haze and overcast skies.

The qualitative results demonstrate that DCAM-DETR consistently produces accurate detections across diverse challenging conditions. In nighttime scenarios (rows 1–2), where RGB imagery provides limited discriminative information, the model effectively leverages thermal infrared features to maintain detection accuracy. In daytime scenarios with complex urban backgrounds (rows 3–4), the model successfully distinguishes small UAV targets from cluttered backgrounds by exploiting complementary RGB texture information. The consistent detection performance across varying target scales validates the effectiveness of our multi-scale feature fusion and DADM-enhanced detection head.

4.6. Performance Under Different Environmental Conditions

To further analyze the robustness of DCAM-DETR across varying environmental conditions, we evaluate detection performance on subsets of the Anti-UAV300 test set categorized by illumination and weather conditions. Figure 9 presents the comparative analysis across five environmental scenarios: daytime clear, daytime cloudy, dusk, nighttime clear, and nighttime foggy conditions, along with the overall performance.

The results reveal that DCAM-DETR maintains consistently high performance across all environmental conditions, with mAP@0.5 ranging from 91.2% (nighttime foggy) to 96.2% (daytime clear). Notably, the performance gap between DCAM-DETR and RGB-only methods widens substantially in challenging scenarios: the improvement over RT-DETR (RGB) increases from 5.0 percentage points in daytime clear conditions to 32.8 percentage points in nighttime foggy conditions. This demonstrates that multimodal fusion provides the greatest advantage precisely when single-modality approaches struggle most, validating the practical value of our approach for real-world anti-UAV deployment.

4.7. Ablation Studies

To systematically validate the contribution of each proposed component and design choice, we conduct comprehensive ablation studies examining the MobileMamba backbone, cross-dimensional attention modules (CDA and CPA), Adaptive Feature Fusion Module (AFFM), and Dual-Attention Decoupling Module (DADM). All ablation experiments are performed on the Anti-UAV300 dataset with consistent training configurations, including identical hyperparameters, data augmentation strategies, and random seeds to ensure reproducible and fair comparisons.

4.7.1. Component-Wise Ablation

To quantify the individual contribution of each proposed component, we conduct progressive ablation experiments starting from a baseline configuration and incrementally adding each module. The baseline model employs the standard RT-DETR architecture with simple channel-wise concatenation for multimodal fusion. We then sequentially integrate the MobileMamba backbone, CDA and CPA attention modules, AFFM, and DADM to observe the performance gains from each component.

Table 3. Component-wise ablation study on Anti-UAV300 dataset. Each row progressively adds one component to the baseline.

Configuration	mAP@0.5	mAP@0.5:0.95	FPS	Params(M)
Baseline (RT-DETR + Concat)	87.2	63.5	44	45.1
+ MobileMamba Backbone	90.3	68.7	43	46.8
+ CDA & CPA	92.5	73.1	42	47.2
+ AFFM	93.8	76.4	42	47.5
+ DADM (Full Model)	94.7	78.3	42	47.6

presents the progressive ablation results on the Anti-UAV300 dataset.

The baseline model concatenates RGB and infrared features along the channel dimension and processes them through the standard RT-DETR architecture, achieving 87.2% mAP@0.5. Replacing the CNN backbone with MobileMamba yields a substantial improvement of 3.1 percentage points in mAP@0.5 and 5.2 percentage points in mAP@0.5:0.95, validating the effectiveness of state space models for global context modeling in multimodal detection. The linear complexity of Mamba enables efficient processing while capturing long-range dependencies crucial for small target detection.

Adding CDA and CPA modules further improves performance by 2.2 percentage points in mAP@0.5 and 4.4 percentage points in mAP@0.5:0.95, demonstrating the importance of explicit cross-modal attention mechanisms for capturing intermodal correlations. The AFFM contributes an additional 1.3 percentage points and 3.3 percentage points improvement, validating the benefit of adaptive fusion that dynamically weights modality contributions based on scene characteristics. Finally, incorporating DADM in the detection head yields the full model performance of 94.7% mAP@0.5 and 78.3% mAP@0.5:0.95, with the hierarchical dilated convolutions and attention decomposition enhancing small target discrimination.

4.7.2. Modality Ablation

To investigate the contribution of each modality and justify the necessity of multimodal fusion over single-modality approaches with increased capacity, we conduct modality ablation experiments.

Table 4. Modality ablation study on Anti-UAV300 dataset.

Configuration	mAP@0.5	mAP@0.5:0.95	FPS	Params(M)
RGB-only (Standard)	85.7	62.1	45	42.3
RGB-only (2× Capacity)	87.3	64.1	38	85.2
IR-only (Standard)	82.4	58.7	45	42.3
IR-only (2× Capacity)	84.6	60.8	38	85.2
DCAM-DETR (RGB + IR)	94.7	78.3	42	47.6

compares: (1) single-modality models using only RGB or IR input, (2) single-modality models with doubled backbone capacity to match the parameter count of our multimodal model, and (3) our full multimodal DCAM-DETR.

The results demonstrate that simply increasing single-modality model capacity cannot compensate for the lack of complementary information. RGB-only with doubled capacity (87.3% mAP@0.5) still falls 7.4 percentage points behind our multimodal approach, while using nearly twice the parameters (85.2 M vs. 47.6 M). This validates that RGB and IR modalities provide fundamentally complementary information—RGB captures texture and color details effective in well-lit conditions, while IR captures thermal signatures robust to illumination variations. The performance gap is particularly pronounced in challenging scenarios: in nighttime foggy conditions, RGB-only (2× capacity) achieves only 62.1% mAP@0.5 compared to DCAM-DETR’s 91.2%, demonstrating that multimodal fusion is essential rather than optional for robust anti-UAV detection.

4.7.3. Backbone Architecture Comparison

To validate the effectiveness of our MobileMamba backbone for multimodal feature extraction, we compare it against representative CNN-based backbones (ResNet-50, ResNet-101), transformer-based backbones (Swin-T, Swin-S), and the vanilla vision Mamba architecture (VMamba-T). All backbone variants are integrated into our detection framework with identical fusion modules and detection heads to ensure fair comparison.

Table 5. Comparison of different backbone architectures on Anti-UAV300 dataset.

Backbone	mAP@0.5	mAP@0.5:0.95	FPS	Params(M)
ResNet-50 [68]	88.4	64.8	48	44.2
ResNet-101 [68]	89.1	66.2	42	63.1
Swin-T [24]	90.8	69.4	35	48.6
Swin-S [24]	91.5	71.2	28	69.3
VMamba-T [30]	92.3	73.8	40	46.2
MobileMamba (Ours)	94.7	78.3	42	47.6

presents the quantitative results on the Anti-UAV300 dataset.

MobileMamba outperforms both CNN-based (ResNet) and transformer-based (Swin) backbones while maintaining competitive inference speed. Compared to VMamba-T, our MobileMamba achieves 2.4 percentage points higher mAP@0.5 and 4.5 percentage points higher mAP@0.5:0.95, demonstrating the effectiveness of our architectural modifications for multimodal fusion.

4.7.4. Fusion Strategy Comparison

To validate the effectiveness of our proposed multimodal fusion approach, we conduct comprehensive comparisons with representative fusion strategies spanning early, late, and intermediate fusion paradigms. Early fusion methods directly concatenate RGB and infrared features at the input or shallow feature levels, while late fusion combines predictions from modality-specific detectors. Intermediate fusion approaches, including attention-based methods, integrate features at multiple abstraction levels.

Table 6. Comparison of different fusion strategies on Anti-UAV300 dataset.

Fusion Strategy	mAP@0.5	mAP@0.5:0.95	Params(M)
Early Fusion (Concat)	87.2	63.5	45.1
Late Fusion (Ensemble)	88.6	65.4	90.2
Attention Fusion [45]	90.4	68.9	46.3
Vanilla Cross-Attention	90.8	70.2	48.5
Cross-Attention [47]	91.8	71.5	52.8
Simple Weighted Average	89.4	67.3	45.8
CDA + CPA + AFFM (Ours)	94.7	78.3	47.6

presents the quantitative comparison results on the Anti-UAV300 dataset.

Our proposed fusion strategy combining CDA, CPA, and AFFM significantly outperforms all alternative approaches across both evaluation metrics. Notably, CDA + CPA outperforms vanilla cross-attention by 1.7 percentage points in mAP@0.5, demonstrating the benefit of explicit spatial and channel correlation modeling. AFFM provides an additional 4.4 percentage points improvement over simple weighted averaging, validating the importance of adaptive scene-dependent fusion.

4.7.5. DADM Component Analysis

To isolate DADM’s contribution from the dilated convolution alone, we conduct additional ablation experiments in

Table 7. DADM component analysis on Anti-UAV300 dataset.

Detection Head Configuration	mAP@0.5	mAP@0.5:0.95
Standard Head	92.5	73.1
+ Dilated Convolutions only	93.2	74.8
+ Dual-Attention Module (DAM)	94.1	76.9
+ Full DADM (Ours)	94.7	78.3

.

The results show that dilated convolutions alone contribute 0.7 percentage points improvement, while the complete DADM with dual-attention provides an additional 1.5 percentage points, demonstrating that the attention mechanism is essential for small target discrimination beyond multi-scale context aggregation.

4.7.6. Performance by Target Size and Scene Condition

To validate our claim that DCAM-DETR excels at small target detection, we report performance breakdown by UAV size and scene condition in

Table 8. Performance breakdown by UAV size and scene condition on Anti-UAV300.

Category	DCAM-DETR	RT-DETR (RGB)	Improvement
By Target Size (mAP@0.5)
Small (<32 × 32 pixels)	89.4	71.2	+18.2
Medium (32–96 pixels)	95.8	86.4	+9.4
Large (>96 × 96 pixels)	97.3	93.1	+4.2
By Scene Condition (mAP@0.5)
Daytime Clear	96.2	91.2	+5.0
Daytime Cloudy	95.4	88.7	+6.7
Dusk/Dawn	94.1	82.3	+11.8
Nighttime Clear	93.8	68.5	+25.3
Nighttime Foggy	91.2	58.4	+32.8

.

The results clearly demonstrate that DCAM-DETR provides the most significant improvements for small targets (+18.2%) and challenging nighttime/foggy conditions (+32.8%), validating the effectiveness of multimodal fusion for these challenging scenarios.

4.7.7. Per-Class AP Analysis

To demonstrate robustness across UAV types, we report per-class AP in

Table 9. Per-class AP analysis on Anti-UAV300 dataset.

UAV Type	Count	AP@0.5	AP@0.5:0.95
Quadcopter	156	95.2	79.1
Fixed-wing	78	93.8	76.8
Helicopter	42	94.1	77.5
Micro UAV	24	91.3	72.4
Overall	300	94.7	78.3

.

DCAM-DETR demonstrates consistent performance across all UAV types, with slightly lower performance on Micro UAVs due to their extremely small size, which aligns with our analysis of small target detection challenges.

4.8. Attention Visualization

To provide interpretable insights into how DCAM-DETR leverages multimodal information for detection, we visualize the attention maps learned by our cross-dimensional attention modules. Understanding the attention patterns helps validate that the model correctly identifies discriminative regions and exploits complementary information from both modalities. Figure 10 presents the attention heatmaps overlaid on input images from both RGB and infrared modalities, revealing the spatial regions the model focuses on during the detection process.

The visualization demonstrates that our CDA and CPA modules successfully learn to focus on UAV regions while suppressing background clutter. Notably, the attention patterns exhibit complementary characteristics between RGB and infrared modalities: RGB attention emphasizes texture and shape features visible under adequate illumination, while infrared attention focuses on thermal signatures that remain robust across varying lighting conditions. This complementary behavior validates our design of separate cross-dimensional attention mechanisms for capturing modality-specific characteristics while enabling effective information fusion.

4.9. Cross-Dataset Evaluation

A critical consideration for practical deployment is whether the learned multimodal fusion strategy generalizes beyond the training domain. To rigorously evaluate this generalization capability, we test our model trained exclusively on Anti-UAV300 on two additional multimodal datasets (FLIR-ADAS and KAIST) without any fine-tuning or domain adaptation. This zero-shot transfer setting provides a stringent test of whether our architectural innovations capture general principles for RGB–infrared fusion rather than dataset-specific patterns.

Table 10. Cross-dataset evaluation results (models trained on Anti-UAV300, tested on other datasets without fine-tuning).

Method	FLIR-ADAS mAP@0.5	KAIST mAP@0.5	Average
RT-DETR [26]	72.3	68.7	70.5
M3FD [14]	75.8	71.2	73.5
CFT [67]	77.2	73.8	75.5
TransFuse [47]	78.4	74.6	76.5
DCAM-DETR (Ours)	81.2	77.9	79.6

presents the cross-dataset evaluation results.

DCAM-DETR achieves the best performance on both cross-domain datasets, demonstrating strong generalization capability. The consistent improvements over baseline methods (3.1 percentage points average improvement over TransFuse) validate that our multimodal fusion strategy and architectural innovations capture general principles for effective RGB–infrared fusion rather than overfitting to the Anti-UAV300 dataset characteristics.

4.10. Computational Efficiency Analysis

For practical deployment in real-time anti-UAV systems, computational efficiency is as critical as detection accuracy. We conduct comprehensive efficiency analysis comparing DCAM-DETR with representative single-modality and multimodal detection methods, measuring inference speed (FPS), model parameters, floating-point operations (FLOPs), and GPU memory consumption. All measurements are performed on a single NVIDIA RTX 3090 GPU with batch size 1 and input resolution $640\times 640$.

Table 11. Computational efficiency comparison on Anti-UAV300 dataset.

Method	mAP@0.5	FPS	Params(M)	FLOPs(G)	Memory(GB)
YOLOv8-L [66]	82.8	58	43.7	165.2	4.2
RT-DETR-L [26]	85.7	45	42.3	136.8	5.1
TransFuse [47]	92.1	25	63.4	248.6	8.7
DCAM-DETR (Ours)	94.7	42	47.6	142.3	5.8

presents the detailed computational efficiency comparison.

DCAM-DETR achieves the best accuracy-efficiency trade-off among all compared methods. Despite processing two input modalities, our model maintains competitive FLOPs (142.3 G) and memory consumption (5.8 GB) compared to single-modality methods, while achieving substantially higher accuracy. The linear complexity of the MobileMamba backbone enables efficient processing of high-resolution multimodal inputs without the quadratic scaling of transformer-based approaches.

4.11. Edge Device Deployment

For practical anti-UAV deployment, we evaluate DCAM-DETR on edge devices.

Table 12. Edge device deployment on NVIDIA Jetson AGX Orin.

Configuration	mAP@0.5	FPS	Power (W)
DCAM-DETR (FP32)	94.7	16	55
DCAM-DETR (FP16)	94.5	24	48
DCAM-DETR (INT8)	93.2	31	44
DCAM-DETR (Pruned 50%)	92.1	28	46
YOLOv8-L (INT8)	81.2	52	38
RT-DETR-L (INT8)	84.1	35	42

presents results on NVIDIA Jetson AGX Orin with different optimization strategies.

Our INT8 quantized model maintains 93.2% mAP@0.5 while achieving 31 FPS on edge devices, demonstrating practical deployment feasibility for real-time anti-UAV applications.

4.12. Failure Case Analysis

To provide comprehensive evaluation, we analyze failure cases and identify scenarios where DCAM-DETR struggles.

Table 13. Failure case analysis on Anti-UAV300 dataset.

Failure Category	mAP@0.5	Failure Rate
Extreme Occlusion (>70%)	67.3	18.2%
Severe Motion Blur	72.1	12.4%
Thermal Camouflage	69.8	8.7%
Very Small Targets (<10 pixels)	58.4	15.3%
CDA Spatial Misalignment	-	12.8%
CPA Channel Confusion	-	15.4%
AFFM Wrong Modality Weight	-	10.9%

summarizes failure categories and their impact on performance.

The analysis reveals that failures primarily occur in extreme conditions where both modalities provide limited discriminative information. AFFM failure analysis shows incorrect weighting primarily occurs when both modalities are similarly degraded (e.g., foggy night scenes). CDA failures typically result from severe viewpoint differences causing spatial misalignment, while CPA failures occur when channel semantics are ambiguous across modalities.

5. Discussion

The experimental results reveal several important insights regarding multimodal fusion for anti-UAV detection. The substantial performance improvement over single-modality methods (9.0 percentage points in mAP@0.5) underscores the fundamental importance of leveraging complementary RGB and infrared information. The attention visualization in Figure 10 confirms that DCAM-DETR learns modality-specific attention patterns: RGB attention emphasizes texture and shape features, while infrared attention focuses on thermal signatures robust to illumination variations. The strong cross-dataset generalization across FLIR-ADAS and KAIST datasets suggests that our architectural innovations capture general principles for RGB–infrared fusion rather than dataset-specific patterns.

The ablation studies provide quantitative evidence for each component’s contribution. The MobileMamba backbone’s 3.1 percentage point improvement validates the importance of linear-complexity global context modeling for small target detection. The 2.2 percentage point gain from CDA and CPA demonstrates that explicit cross-modal attention across both spatial and channel dimensions significantly outperforms simple fusion strategies. AFFM’s adaptive gating mechanism and DADM’s multi-scale contextual aggregation further enhance performance by addressing modality reliability variations and small target discrimination challenges, respectively.

Limitations and Societal Impact

Despite these strengths, several limitations warrant discussion:

Technical Limitations: (1) DCAM-DETR requires spatially aligned RGB–infrared image pairs, and misalignment beyond 5 pixels causes performance degradation (mAP drops by 3.2% at 10-pixel misalignment). (2) The sequential nature of state space models limits parallelization compared to fully convolutional architectures. (3) Performance degrades significantly for extremely small targets (less than 10 pixels) where both modalities provide limited discriminative information, achieving only 58.4% mAP@0.5. (4) Computational requirements may limit deployment on highly resource-constrained devices without quantization. (5) Training requires paired RGB-IR data with annotations, which is expensive to collect compared to single-modality datasets.

Societal Impact: Our anti-UAV detection system has both positive and negative societal implications. Positive aspects include enhanced security for critical infrastructure, airports, prisons, and public events, as well as improved safety in emergency response scenarios where unauthorized UAVs pose risks. Negative aspects include potential misuse for surveillance purposes, privacy concerns when deployed in public spaces, and dual-use technology considerations where the same detection capabilities could be used to evade legitimate UAV operations. We advocate for responsible deployment with appropriate regulatory oversight, transparency about system capabilities and limitations, and adherence to ethical guidelines for surveillance technologies.

Future work could explore explicit alignment modules to handle misaligned inputs, video-based temporal extensions for improved tracking, lightweight variants optimized for edge deployment, and self-supervised learning approaches to reduce annotation requirements.

6. Conclusions

In this paper, we proposed DCAM-DETR, a novel multimodal detection framework that synergistically integrates Mamba-based state space models with the RT-DETR architecture for robust real-time anti-UAV detection. Our approach introduces four principal innovations: a MobileMamba backbone for efficient global context modeling with linear computational complexity, Cross-Dimensional Attention (CDA) and Cross-Path Attention (CPA) modules for fine-grained multimodal feature alignment, an Adaptive Feature Fusion Module (AFFM) for scene-dependent fusion, and a Dual-Attention Decoupling Module (DADM) for enhanced small target detection.

Comprehensive experiments on Anti-UAV300, FLIR-ADAS, and KAIST datasets demonstrate that DCAM-DETR achieves state-of-the-art performance with 94.7% mAP@0.5 on Anti-UAV300, outperforming existing methods by substantial margins while maintaining real-time inference speed of 42 FPS. The strong cross-dataset generalization validates the effectiveness of our multimodal fusion strategy across diverse application domains. Ablation studies confirm the contribution of each proposed component, with the MobileMamba backbone, cross-dimensional attention modules, AFFM, and DADM collectively contributing to the superior performance.

The success of DCAM-DETR demonstrates the potential of state space models for multimodal visual perception tasks, offering an efficient alternative to transformer-based approaches while maintaining strong representational capability. Future work will explore temporal extensions for video-based detection, lightweight variants for edge deployment, and self-supervised learning approaches to reduce annotation requirements.