MFE-DETR: Multimodal Feature-Enhanced Detection Transformer for RGB–Infrared Object Detection in Aerial Imagery

Abstract

Multimodal object detection utilizing RGB and infrared (IR) imagery has become a critical research area for unmanned aerial vehicle (UAV) surveillance applications, providing reliable perception under various lighting and environmental conditions. Nevertheless, current methods encounter three primary challenges: (1) insufficient utilization of frequency-domain properties in heterogeneous modalities, (2) restricted adaptability in crossmodal feature integration across different environmental scenarios, and (3) inadequate modeling of fine-grained spatial relationships for accurate object localization. To overcome these limitations, we introduce MFE-DETR, a novel Multimodal Feature-Enhanced Detection Transformer that achieves superior RGB-IR fusion through three complementary innovations. First, we present the Dual-Modality Enhancement Module (DMEM) with two specialized processing streams: the Haar wavelet decomposition stream (HWD-Stream) that conducts multi-resolution frequency-domain analysis to independently enhance low-frequency structural components and high-frequency textural information, and the Attention-guided Kolmogorov–Arnold Refinement Stream (AKR-Stream) that employs learnable spline-parameterized activation functions for adaptive nonlinear feature refinement. Second, we enhance the Cross-scale Channel Feature Fusion module by integrating an Adaptive Feature Fusion Module (AFAM) with complementary gating mechanisms that dynamically adjust modality contributions according to spatial informativeness. Third, we introduce the Bilinear Attention-Enhanced Detection Module (BADM) that models second-order feature interactions through factorized bilinear pooling, facilitating fine-grained crossmodal correlation analysis. Extensive experiments on the DroneVehicle benchmark show that MFE-DETR attains 78.6% ${\mathrm{mAP}}_{50}$ and 57.8% ${\mathrm{mAP}}_{50:95}$, outperforming state-of-the-art approaches by 5.3% and 3.7%, respectively. Additional evaluations on the VisDrone dataset further confirm the excellent generalization performance of our method, especially for small object detection with 18.6% ${\mathrm{AP}}_S$, achieving a 1.5% improvement over existing techniques. Comprehensive ablation studies and visualizations offer detailed insights into the effectiveness of each proposed component.

1. Introduction

The proliferation of unmanned aerial vehicles (UAVs) equipped with heterogeneous imaging sensors has catalyzed transformative advances in aerial surveillance, environmental monitoring, and intelligent transportation systems [1,2]. Among various sensor configurations, the combination of visible-spectrum (RGB) cameras and thermal infrared (IR) sensors has garnered substantial research attention due to their inherent complementarity: RGB imagery provides rich chromatic and textural information under favorable illumination conditions, while IR imagery captures thermal radiation signatures that remain discriminative in low-light environments, adverse weather conditions, and scenarios involving camouflaged or occluded objects [3,4,5]. This multimodal sensing paradigm offers unprecedented opportunities for developing robust object detection systems capable of maintaining consistent performance across the full spectrum of operational conditions encountered in real-world UAV deployments.

Despite the intuitive appeal of RGB-IR fusion, the effective integration of these heterogeneous modalities for object detection presents formidable technical challenges that existing methods have yet to adequately address. The fundamental difficulty stems from the substantial domain gap between visible and thermal imagery: RGB images encode reflectance properties governed by surface albedo and illumination geometry, whereas IR images capture emissivity patterns determined by object temperature and material thermal properties [6,7]. This intrinsic heterogeneity manifests in divergent statistical distributions, spatial resolution characteristics, and semantic content across modalities, rendering naive fusion strategies such as early concatenation or late decision-level combination suboptimal [8,9].

Contemporary approaches to RGB-IR object detection can be broadly categorized into three paradigms based on their fusion strategies. Early fusion methods [3] concatenate multi-spectral inputs at the image level, relying on subsequent convolutional layers to implicitly learn crossmodal representations. While computationally efficient, this approach fails to preserve modality-specific characteristics and struggles with the inherent misalignment between RGB and IR feature distributions. Late fusion methods [6] maintain separate processing streams for each modality and combine predictions at the decision level, preserving modality-specific information but sacrificing the potential for deep crossmodal interaction. Middle fusion approaches [7,8,10] represent the current state-of-the-art by fusing features at intermediate network layers, enabling learned crossmodal representations while maintaining some degree of modality-specific processing. However, these methods typically employ static fusion mechanisms that cannot adapt to the varying reliability and informativeness of each modality across different spatial locations and environmental conditions.

The emergence of transformer-based detection architectures, exemplified by DETR [11] and its efficient variants [12,13,14], has opened new avenues for multimodal object detection. The self-attention mechanism inherent in transformers provides a natural framework for modeling long-range dependencies and crossmodal interactions, while the set-based prediction paradigm eliminates the need for hand-crafted components such as anchor boxes and non-maximum suppression. The Real-Time Detection Transformer (RT-DETR) [14] has recently demonstrated that transformer-based detectors can achieve competitive speed-accuracy trade-offs through hybrid encoder designs and uncertainty-minimal query selection mechanisms. However, RT-DETR and similar architectures were designed for single-modality detection and lack specialized mechanisms for effective multimodal fusion.

From a symmetry perspective, RGB and IR modalities exhibit an intriguing duality: they share approximate symmetry in spatial geometric structures (boundaries, shapes, layouts) while demonstrating intrinsic asymmetry in imaging mechanisms, noise distributions, spectral characteristics, and salient regions. This observation suggests that effective multimodal detection is not merely about fusion, but rather about managing the tension between preserving shared structural consistency and exploiting modality-specific asymmetric differences. The key lies in achieving adaptive symmetry breaking and restoration that responds to varying scene conditions—leveraging symmetric structural information for robust detection while selectively utilizing asymmetric details for enhanced discrimination.

In this paper, we propose MFE-DETR (Multimodal Feature-Enhanced Detection Transformer), a comprehensive framework that extends the RT-DETR architecture with specialized modules designed to address the unique challenges of RGB-IR multimodal object detection from this symmetry-asymmetry perspective. Our approach is motivated by three key observations derived from systematic analysis of existing methods and the characteristics of multimodal aerial imagery.

First, we observe that RGB and IR modalities exhibit distinct frequency-domain characteristics that are inadequately exploited by existing spatial-domain fusion methods. RGB images typically contain rich high-frequency textural details arising from surface reflectance variations, while IR images often exhibit smoother intensity gradients dominated by low-frequency thermal patterns. This observation motivates our Haar wavelet decomposition stream (HWD-Stream), which performs multi-resolution frequency-domain analysis to decompose fused features into low-frequency structural components and high-frequency detail components, enabling specialized processing pathways that preserve and enhance modality-specific information.

Second, we recognize that the optimal fusion strategy varies spatially within an image and temporally across different environmental conditions. In well-illuminated daytime scenarios, RGB features may provide more discriminative information, while in nighttime or low-visibility conditions, IR features become dominant. This observation motivates our Adaptive Feature Fusion Module (AFAM), which employs learnable complementary gating mechanisms to dynamically balance modality contributions based on local feature informativeness.

Third, we identify that accurate object localization in cluttered aerial scenes requires modeling fine-grained spatial correlations that capture second-order feature interactions. Standard attention mechanisms compute first-order correlations through dot-product similarity, which may be insufficient for distinguishing objects with similar appearance but different spatial configurations. This observation motivates our Bilinear Attention-Enhanced Detection Module (BADM), which employs factorized bilinear pooling to capture higher-order feature interactions for improved localization accuracy.

Additionally, we introduce the Attention-guided Kolmogorov–Arnold Refinement Stream (AKR-Stream) as a complementary component to the HWD-Stream within our Dual-Modality Enhancement Module (DMEM). Inspired by the recently proposed Kolmogorov–Arnold networks (KANs) [15], the AKR-Stream replaces traditional fixed activation functions with learnable spline-parameterized univariate functions, providing superior function approximation capabilities for adaptive feature refinement. The combination of frequency-domain decomposition (HWD-Stream) and adaptive nonlinear transformation (AKR-Stream) enables comprehensive enhancement of multimodal features across different network stages.

We conduct comprehensive experiments on two challenging benchmark datasets to validate the effectiveness of our approach. The DroneVehicle dataset [5] serves as our primary evaluation benchmark, providing 28,439 aligned RGB-IR image pairs captured across diverse scenarios including urban roads, residential areas, parking lots, and highways under both daytime and nighttime conditions. Our MFE-DETR achieves 78.6%${\mathrm{mAP}}_{50}$ and 57.8% ${\mathrm{mAP}}_{50:95}$ on DroneVehicle, surpassing the previous state-of-the-art RT-DETR-X by 5.3% and 3.7%, respectively. To further demonstrate the generalization capability and robustness of our feature enhancement modules, we evaluate on the VisDrone dataset [1], a large-scale benchmark for small object detection in UAV imagery. Despite being designed primarily for multimodal detection, our approach achieves 42.8% ${\mathrm{mAP}}_{50}$ on VisDrone, with particularly strong performance on small objects (18.6% ${\mathrm{AP}}_S$), demonstrating the effectiveness of our proposed modules for general aerial object detection tasks.

The main contributions of this paper are summarized as follows:

• We propose MFE-DETR, a novel detection transformer framework for RGB-IR multimodal object detection that systematically addresses the challenges of heterogeneous modality fusion through frequency-domain analysis, adaptive gating mechanisms, and higher-order feature interaction modeling.
• We introduce the Dual-Modality Enhancement Module (DMEM) comprising two complementary processing streams: the HWD-Stream for multi-resolution wavelet-based feature decomposition and the AKR-Stream for Kolmogorov–Arnold network-based adaptive feature refinement, enabling comprehensive enhancement of multimodal representations.
• We design the Adaptive Feature Fusion Module (AFAM) with complementary gating mechanisms that dynamically balance modality contributions based on spatial informativeness, effectively addressing the modality imbalance problem in varying environmental conditions.
• We propose the Bilinear Attention-Enhanced Detection Module (BADM) that captures second-order feature interactions through factorized bilinear pooling, enabling fine-grained crossmodal correlation modeling for improved object localization.
• Extensive experiments on DroneVehicle and VisDrone datasets demonstrate state-of-the-art performance, with comprehensive ablation studies providing detailed insights into the contribution of each proposed component.

Table 1. Design rationale summary: Each module addresses a specific challenge in multimodal fusion while contributing to the unified goal of adaptive symmetry management between RGB and IR modalities.

Module	Challenge Addressed	Key Mechanism	Primary Benefit
DMEM	Crossmodal frequency-domain differences causing mutual information suppression during fusion	Haar wavelet decomposition separates structure/detail + KAN activation enables adaptive nonlinearity	Enhanced small object and nighttime robustness
AFAM	Varying modality reliability across different scenes and spatial locations	Complementary gating dynamically allocates weights based on spatial informativeness	Reduced interference from unreliable modality
BADM	First-order attention insufficient for modeling complex spatial relationships	Second-order bilinear attention captures pairwise feature interactions	Improved localization precision and reduced confusion

provides a unified overview of how each proposed module addresses specific challenges in multimodal aerial object detection, highlighting the complementary nature of our design choices within a coherent theoretical framework centered on managing the symmetry-asymmetry tension between modalities.

The remainder of this paper is organized as follows: Section 2 reviews related work on object detection, multimodal fusion, wavelet transforms in deep learning, and Kolmogorov–Arnold networks. Section 3 presents the detailed methodology of our MFE-DETR framework. Section 4 describes the experimental setup, presents quantitative and qualitative results, and provides comprehensive ablation studies. Section 5 discusses the implications and limitations of our approach. Section 6 concludes the paper and outlines future research directions.

2. Related Work

2.1. Deep Learning-Based Object Detection

Object detection has undergone a paradigm shift with the advent of deep learning, evolving from hand-crafted feature-based methods to end-to-end learnable architectures. Contemporary detectors can be broadly categorized into three families: two-stage detectors, single-stage detectors, and transformer-based detectors.

Two-stage detectors, pioneered by R-CNN [16] and its successors, decompose detection into region proposal generation and subsequent classification/refinement stages. Faster R-CNN [16] introduced the Region Proposal Network (RPN) for efficient proposal generation, while Feature Pyramid Networks (FPN) [17] addressed multi-scale detection through hierarchical feature aggregation. Cascade R-CNN [18] further improved localization accuracy through iterative bounding box refinement with progressively increasing IoU thresholds. Despite their strong performance, two-stage detectors incur computational overhead from the proposal generation stage, limiting their applicability in real-time scenarios.

Single-stage detectors eliminate the proposal generation stage by directly predicting object locations and categories from dense feature maps. The YOLO family [19,20,21,22] has been particularly influential, with successive versions introducing architectural innovations such as CSPNet backbones, path aggregation networks, and anchor-free prediction heads. YOLOv8 [20] achieved state-of-the-art speed-accuracy trade-offs through decoupled detection heads and advanced data augmentation strategies. YOLOv9 [21] introduced Programmable Gradient Information (PGI) to address information bottleneck issues in deep networks. YOLOv10 [22] eliminated the need for non-maximum suppression through consistent dual assignments, enabling true end-to-end detection. RetinaNet [23] addressed the class imbalance problem inherent in dense prediction through focal loss, while FCOS [24] demonstrated that anchor-free detection could achieve competitive performance through center-ness prediction.

Transformer-based detectors represent the latest evolution in object detection architectures. DETR [11] pioneered the application of transformers to detection, formulating the task as a set prediction problem solved through bipartite matching. Deformable DETR [12] addressed the slow convergence of vanilla DETR through deformable attention mechanisms that attend to a sparse set of sampling locations. DINO [13] further improved performance through contrastive denoising training and mixed query selection. RT-DETR [14] achieved real-time performance through a hybrid encoder design that decouples intra-scale interaction from cross-scale fusion, along with uncertainty-minimal query selection for improved decoder initialization. Our MFE-DETR builds upon the RT-DETR architecture, extending it with specialized modules for effective multimodal fusion.

2.2. Object Detection in Remote Sensing and UAV Imagery

Object detection in remote sensing and UAV imagery presents unique challenges distinct from natural image detection, including extreme scale variations, arbitrary object orientations, dense object distributions, and complex backgrounds [2,25]. The DOTA dataset [2] established benchmarks for oriented object detection in aerial images, spurring development of rotation-equivariant architectures. The VisDrone challenge [1] focused attention on small object detection in drone-captured imagery, revealing the limitations of standard detectors when applied to aerial perspectives.

Recent advances in UAV object detection have focused on addressing the small object challenge through enhanced feature fusion and attention mechanisms. SPD-YOLOv8 [26] introduced space-to-depth convolutions to preserve fine-grained spatial information during downsampling. SCA-YOLO [27] proposed spatial-channel attention modules for improved small object feature representation. BGF-YOLOv10 [28] combined bi-directional feature fusion with global context modeling for enhanced multi-scale detection. RemDet [29] rethought efficient model design specifically for UAV scenarios, achieving state-of-the-art results through carefully balanced architectural choices.

For remote sensing applications, specialized architectures have been developed to handle the unique characteristics of satellite and aerial imagery. Attention-based multi-level feature fusion [30] improved detection of objects at varying scales through adaptive feature aggregation. Enhanced RT-DETR models [31] demonstrated the potential of transformer-based architectures for remote sensing detection, particularly for small objects. These advances highlight the importance of domain-specific architectural innovations for achieving optimal performance in aerial object detection.

2.3. Multimodal Object Detection

Multimodal object detection aims to leverage complementary information from multiple sensor modalities to achieve more robust and accurate detection. RGB–thermal fusion has received particular attention due to its practical importance in surveillance, autonomous driving, and search-and-rescue applications [3,4].

Early approaches to RGB–thermal fusion employed simple concatenation strategies, either at the input level (early fusion) or at the decision level (late fusion) [6]. While computationally efficient, these methods fail to capture complex crossmodal interactions. Middle fusion approaches [7] fuse features at intermediate network layers, enabling learned crossmodal representations while maintaining some degree of modality-specific processing.

Attention-based fusion mechanisms have emerged as a powerful paradigm for adaptive multimodal integration. GAFF [8] introduced guided attentive feature fusion that leverages illumination information to weight modality contributions. CFT [9] proposed crossmodality feature aggregation through transformer-based attention. LRAF-Net [10] introduced long-range attention fusion for capturing global crossmodal dependencies. CAGT [32] addressed modality misalignment through cascade alignment-guided transformers with complementary fusion modules.

Recent transformer-based approaches have shown particular promise for multimodal detection. Calibrated transformers [33] address modality miscalibration through specialized alignment modules. Remedy dual-attention transformers [34] employ semantic alignment via contrastive learning for improved crossmodal feature matching. Multimodal transformers with cross-channel attention [35] enable early-stage modality alignment through learned channel relationships. Despite these advances, existing methods still struggle with the fundamental challenge of adaptively balancing modality contributions under varying environmental conditions while maintaining computational efficiency.

The DroneVehicle dataset [5] has emerged as a key benchmark for RGB-IR vehicle detection in UAV imagery, providing aligned multimodal image pairs across diverse scenarios. Methods evaluated on this dataset have demonstrated the importance of effective crossmodal fusion for robust detection under varying illumination conditions. Our MFE-DETR addresses the limitations of existing approaches through systematic innovations in frequency-domain feature processing, adaptive fusion mechanisms, and higher-order feature interaction modeling.

2.4. Wavelet Transforms in Deep Learning

Wavelet transforms provide multi-resolution analysis capabilities that decompose signals into frequency components at different scales, offering a principled framework for analyzing both local and global signal characteristics [36]. The Haar wavelet [37], as the simplest orthonormal wavelet basis, offers computational efficiency while effectively capturing local discontinuities and edge information through its piecewise constant basis functions.

The integration of wavelet transforms into deep learning architectures has gained increasing attention for various computer vision tasks. Wavelet-based downsampling has been proposed as an alternative to traditional pooling operations, preserving more information during feature map reduction by explicitly separating low-frequency approximation from high-frequency detail components [38]. WCNN3D [39] demonstrated the effectiveness of wavelet convolutions for 3D object detection in autonomous driving scenarios. MAWC-Net [40] combined multi-scale attention with wavelet convolutions for improved spectral feature representation.

For multimodal fusion applications, wavelet-based approaches offer unique advantages in separating modality-specific frequency characteristics. Recent work on RGB–thermal image fusion [41,42] has demonstrated that wavelet decomposition enables more effective preservation of complementary information from different spectral bands. The frequency-spatial attention transformer [42] combined wavelet analysis with attention mechanisms for improved infrared–visible image fusion. These advances motivate our HWD-Stream design, which leverages Haar wavelet decomposition to separately process low-frequency structural information and high-frequency textural details from fused multimodal features.

2.5. Kolmogorov–Arnold Networks

The Kolmogorov–Arnold representation theorem [43] establishes that any multivariate continuous function can be represented as a superposition of continuous univariate functions and addition operations. Formally, for any continuous function $f:{[0,1]}^n\to \mathbb{R}$, there exist continuous univariate functions ${\varphi }_{q,p}$ and ${\mathsf{\Phi }}_q$ such that

$$f(x_1,\dots ,x_n)=\sum _{q=0}^{2n}{\mathsf{\Phi }}_q\left(\sum _{p=1}^n{\varphi }_{q,p}\left(x_p\right)\right)$$

(1)

This theoretical foundation has recently inspired the development of Kolmogorov–Arnold networks (KANs) [15], which replace the fixed activation functions in traditional multilayer perceptrons with learnable univariate functions parameterized by B-splines. Unlike conventional neural networks where weights are learned on edges and activation functions are fixed at nodes, KAN places learnable activation functions on edges, enabling more expressive function approximation with fewer parameters.

The B-spline parameterization in KAN provides several advantages for feature refinement tasks. B-splines offer local control, meaning that modifying a control point only affects a limited region of the function, enabling fine-grained adaptation to local feature characteristics. Additionally, B-splines provide smooth interpolation with controllable continuity, avoiding the discontinuities that can arise with piecewise linear approximations. The learnable nature of spline-based activations allows the network to adaptively discover optimal nonlinear transformations for specific tasks, rather than relying on hand-designed activation functions such as ReLU or GELU.

Recent applications of KAN principles to computer vision have demonstrated promising results in various domains. KAN-based architectures have shown improved performance in image classification, semantic segmentation, and feature extraction tasks [15]. The ability of KAN to learn task-specific activation functions makes it particularly suitable for multimodal fusion scenarios where the optimal nonlinear transformation may vary across different feature channels and spatial locations. Our AKR-Stream incorporates KAN-inspired learnable spline activations within an attention-guided framework, enabling adaptive feature refinement that complements the frequency-domain processing of the HWD-Stream.

3. Methodology

This section presents the detailed architecture of our proposed MFE-DETR framework. We first provide an overview of the overall architecture, followed by detailed descriptions of each proposed module: the Dual-Modality Enhancement Module (DMEM), the Adaptive Feature Fusion Module (AFAM), and the Bilinear Attention-Enhanced Detection Module (BADM). Finally, we describe the loss function used for training.

3.1. Overall Architecture

The overall architecture of MFE-DETR is illustrated in Figure 1. Our framework extends the RT-DETR [14] architecture with specialized modules designed for effective RGB-IR multimodal fusion. The network consists of four main components: (1) dual-stream backbone networks for modality-specific feature extraction, (2) the Dual-Modality Enhancement Module (DMEM) for frequency-domain and adaptive feature enhancement, (3) the hybrid encoder with Cross-scale Channel Feature Fusion (CCFF) and Adaptive Feature Fusion Module (AFAM) for cross-scale feature aggregation, and (4) the transformer decoder with Bilinear Attention-Enhanced Detection Module (BADM) for final object detection.

3.1.1. Dual-Stream Feature Extraction

Given an aligned RGB-IR image pair $({\mathbf{I}}_{rgb},{\mathbf{I}}_{ir})\in {\mathbb{R}}^{H\times W\times 3}\times {\mathbb{R}}^{H\times W\times 1}$, we first apply a lightweight channel projection to the single-channel IR input to align it with the RGB channel dimension:

$${\mathbf{I}}_{ir}^{proj}=\mathrm{ChannelProj}\left({\mathbf{I}}_{ir}\right)={\mathrm{Conv}}_{1\times 1}\left({\mathbf{I}}_{ir}\right)\in {\mathbb{R}}^{H\times W\times 3}$$

(2)

where the channel projection layer maps the single-channel IR image to a 3-channel representation, enabling subsequent weight sharing between the dual-stream backbones while preserving the thermal information.

We then apply initial downsampling through a stem convolution layer to reduce computational complexity while preserving essential spatial information:

$${\mathbf{F}}_{rgb}^0=\mathrm{Stem}\left({\mathbf{I}}_{rgb}\right)\in {\mathbb{R}}^{{\displaystyle \frac{H}{4}}\times {\displaystyle \frac{W}{4}}\times C_2},{\mathbf{F}}_{ir}^0=\mathrm{Stem}\left({\mathbf{I}}_{ir}^{proj}\right)\in {\mathbb{R}}^{{\displaystyle \frac{H}{4}}\times {\displaystyle \frac{W}{4}}\times C_2}$$

(3)

where the Stem module consists of a $3\times 3$ convolution with stride 2 followed by batch normalization and SiLU activation, applied twice to achieve $4\times$ downsampling. The backbone parameters are shared between the RGB and IR streams after channel projection, which controls model complexity and ensures crossmodal structural consistency in feature extraction.

The backbone network follows a hierarchical structure with five stages, where each stage $s\in \{2,3,4,5\}$ progressively extracts features at different spatial resolutions. Such hierarchical architectures have been shown to exhibit favorable connectivity properties analogous to those studied in interconnection network theory [44,45]. The feature extraction at stage s can be formulated as

$${\mathbf{F}}_m^s={\mathcal{S}}_s\left(\mathrm{DownSample}\left({\mathbf{F}}_m^{s-1}\right)\right)\in {\mathbb{R}}^{{\displaystyle \frac{H}{2^s}}\times {\displaystyle \frac{W}{2^s}}\times C_s},m\in \{rgb,ir\}$$

(4)

where ${\mathcal{S}}_s(·)$ denotes the stage-specific feature transformation block, and the channel dimensions follow $C_2<C_3<C_4<C_5$ to capture increasingly abstract semantic representations.

3.1.2. Early Fusion with Element-Wise Operations

At each backbone stage, we perform early fusion of RGB and IR features through element-wise addition, which preserves the complementary information from both modalities while maintaining computational efficiency and training stability:

$${\mathbf{F}}_{fused}^s={\mathbf{F}}_{rgb}^s\oplus {\mathbf{F}}_{ir}^s={\mathbf{F}}_{rgb}^s+{\mathbf{F}}_{ir}^s$$

(5)

This additive fusion strategy is motivated by the observation that RGB and IR features exhibit complementary activation patterns: RGB features respond strongly to texture and color gradients, while IR features activate on thermal boundaries. The element-wise addition allows both types of information to be preserved without introducing additional learnable parameters at the fusion point.

It is important to clarify the distinction between early fusion and the fusion mechanism within DMEM. Early fusion employs element-wise addition for computational efficiency and gradient stability, serving as the primary crossmodal integration point. In contrast, the DMEM internal fusion adopts channel concatenation followed by $1\times 1$ convolution, which enables learnable crossmodal re-calibration for fine-grained feature enhancement rather than replacing the early fusion stage.

3.1.3. Stage-Wise DMEM Integration

The DMEM is strategically integrated at different backbone stages to leverage the distinct characteristics of features at various abstraction levels. As shown in Figure 1, we deploy DMEM-HWD blocks at stages 2–4 and DMEM-AKR blocks at stages 4–5:

$$\begin{array}{cc}\hfill {\widehat{\mathbf{F}}}^s& =\mathrm{DMEM}\text{-}\mathrm{HWD}\left({\mathbf{F}}_{fused}^s\right),s\in \{2,3,4\}\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill {\widehat{\mathbf{F}}}^s& =\mathrm{DMEM}\text{-}\mathrm{AKR}\left({\mathbf{F}}_{fused}^s\right),s\in \{4,5\}\hfill \end{array}$$

(7)

This design rationale stems from the frequency characteristics at different network depths. Early stages (2–3) contain rich high-frequency details that benefit from wavelet-based decomposition and enhancement. Later stages (4–5) capture more abstract semantic features that require adaptive nonlinear transformations provided by the KAN-inspired AKR-Stream.

3.1.4. Multi-Scale Feature Pyramid

The enhanced features from stages 3, 4, and 5 form a multi-scale feature pyramid $\{{\mathbf{S}}_3,{\mathbf{S}}_4,{\mathbf{S}}_5\}$ that serves as input to the feature fusion module:

$${\mathbf{S}}_l={\widehat{\mathbf{F}}}^{l+2}\in {\mathbb{R}}^{{\displaystyle \frac{H}{2^{l+2}}}\times {\displaystyle \frac{W}{2^{l+2}}}\times C_{l+2}},l\in \{3,4,5\}$$

(8)

These multi-scale features capture objects at different scales: ${\mathbf{S}}_3$ with higher spatial resolution is suitable for detecting small objects, while ${\mathbf{S}}_5$ with larger receptive fields captures contextual information for large objects.

3.1.5. Hybrid Encoder with CCFF and AIFT

The hybrid encoder follows the RT-DETR design philosophy of decoupling intra-scale and cross-scale feature interactions. The effectiveness of such hierarchical feature aggregation architectures has been theoretically supported by studies on network connectivity and information flow [46,47]. The Attention-based Intra-scale Feature Transformation (AIFT) module processes the highest-level features ${\mathbf{S}}_5$ through self-attention to capture long-range dependencies:

$${\mathbf{F}}_5^{\prime }=\mathrm{AIFT}\left({\mathbf{S}}_5\right)=\mathrm{Softmax}\left({\displaystyle \frac{\mathbf{Q}{\mathbf{K}}^T}{\sqrt{d_k}}}\right)\mathbf{V}+{\mathbf{S}}_5$$

(9)

where $\mathbf{Q}={\mathbf{S}}_5{\mathbf{W}}_Q$, $\mathbf{K}={\mathbf{S}}_5{\mathbf{W}}_K$, $\mathbf{V}={\mathbf{S}}_5{\mathbf{W}}_V$ are the query, key, and value projections, and $d_k$ is the key dimension for scaling.

The Cross-scale Channel Feature Fusion (CCFF) module then aggregates features across scales through our proposed AFAM:

$${\mathbf{F}}_{out}=\mathrm{CCFF}({\mathbf{S}}_3,{\mathbf{S}}_4,{\mathbf{F}}_5^{\prime })=\mathrm{AFAM}\left({\mathbf{S}}_3\right)\oplus \mathrm{AFAM}\left({\mathbf{S}}_4\right)\oplus \mathrm{AFAM}\left({\mathbf{F}}_5^{\prime }\right)$$

(10)

3.1.6. Query Selection and Decoder

Following RT-DETR, we employ uncertainty-minimal query selection to initialize decoder queries. Given the encoder output ${\mathbf{F}}_{out}$, we predict classification scores and localization uncertainties for each spatial position:

$$\begin{array}{cc}\hfill {\mathbf{P}}_{cls}& =\mathrm{Sigmoid}\left(\mathrm{Linear}\left({\mathbf{F}}_{out}\right)\right)\in {\mathbb{R}}^{N\times K}\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill {\mathbf{U}}_{loc}& =\mathrm{Softplus}\left(\mathrm{Linear}\left({\mathbf{F}}_{out}\right)\right)\in {\mathbb{R}}^{N\times 4}\hfill \end{array}$$

(12)

where N is the number of spatial positions, K is the number of object categories, and ${\mathbf{U}}_{loc}$ represents the predicted uncertainty for bounding box coordinates.

The top-M positions with highest confidence scores and lowest localization uncertainties are selected as initial queries:

$$\mathcal{Q}=\mathrm{TopK}\left(\underset{k}{max}\left({\mathbf{P}}_{cls}^{(n,k)}\right)-\lambda \sum _{i=1}^4{\mathbf{U}}_{loc}^{(n,i)},M\right)$$

(13)

where $\lambda$ is a hyperparameter balancing classification confidence and localization uncertainty.

The transformer decoder with BADM then refines these queries through cross-attention with encoder features:

$$\widehat{\mathbf{Q}}=\mathrm{Decoder}(\mathcal{Q},{\mathbf{F}}_{out})=\mathrm{BADM}\left(\mathrm{CrossAttn}(\mathcal{Q},{\mathbf{F}}_{out})\right)$$

(14)

The final detection outputs are obtained through prediction heads applied to the refined queries:

$$\begin{array}{cc}\hfill \widehat{\mathbf{c}}& ={\mathrm{Linear}}_{cls}\left(\widehat{\mathbf{Q}}\right)\in {\mathbb{R}}^{M\times K}\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill \widehat{\mathbf{b}}& =\mathrm{Sigmoid}\left({\mathrm{Linear}}_{box}\left(\widehat{\mathbf{Q}}\right)\right)\in {\mathbb{R}}^{M\times 4}\hfill \end{array}$$

(16)

where $\widehat{\mathbf{c}}$ and $\widehat{\mathbf{b}}$ represent the predicted class logits and normalized bounding box coordinates $(c_x,c_y,w,h)$, respectively.

3.2. Dual-Modality Enhancement Module (DMEM)

The Dual-Modality Enhancement Module (DMEM) is designed to comprehensively enhance multimodal features through two complementary processing streams: the Haar wavelet decomposition stream (HWD-Stream) and the Attention-guided Kolmogorov–Arnold Refinement Stream (AKR-Stream). The HWD-Stream performs frequency-domain analysis to separately process structural and textural information, while the AKR-Stream applies adaptive nonlinear transformations through learnable spline-based activations.

3.2.1. Haar Wavelet Decomposition Stream (HWD-Stream)

The HWD-Stream leverages the Haar wavelet transform to decompose fused features into frequency sub-bands, enabling specialized processing of low-frequency structural components and high-frequency detail components. Given the concatenated multimodal features ${\mathbf{F}}_{cat}=[{\mathbf{F}}_{rgb};{\mathbf{F}}_{ir}]\in {\mathbb{R}}^{H\times W\times 2C}$, we first apply a channel reduction convolution:

$${\mathbf{F}}_{fused}={\mathrm{Conv}}_{1\times 1}\left({\mathbf{F}}_{cat}\right)\in {\mathbb{R}}^{H\times W\times C}$$

(17)

The 2D Haar wavelet transform decomposes the fused features into four sub-bands through convolution with Haar filter banks. Let ${\mathbf{h}}_L={\displaystyle \frac{1}{\sqrt{2}}}{[1,1]}^T$ and ${\mathbf{h}}_H={\displaystyle \frac{1}{\sqrt{2}}}{[1,-1]}^T$ denote the low-pass and high-pass Haar filters, respectively. The four sub-bands are computed as

$$\begin{array}{cc}\hfill {\mathbf{F}}_{LL}& =\left({\mathbf{h}}_L{\mathbf{h}}_L^T\right)\ast {\mathbf{F}}_{fused}{\downarrow }_2\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill {\mathbf{F}}_{LH}& =\left({\mathbf{h}}_L{\mathbf{h}}_H^T\right)\ast {\mathbf{F}}_{fused}{\downarrow }_2\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill {\mathbf{F}}_{HL}& =\left({\mathbf{h}}_H{\mathbf{h}}_L^T\right)\ast {\mathbf{F}}_{fused}{\downarrow }_2\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill {\mathbf{F}}_{HH}& =\left({\mathbf{h}}_H{\mathbf{h}}_H^T\right)\ast {\mathbf{F}}_{fused}{\downarrow }_2\hfill \end{array}$$

(21)

where ∗ denotes convolution, ${\downarrow }_2$ represents downsampling by factor 2, and ${\mathbf{F}}_{LL},{\mathbf{F}}_{LH},{\mathbf{F}}_{HL},{\mathbf{F}}_{HH}\in {\mathbb{R}}^{{\displaystyle \frac{H}{2}}\times {\displaystyle \frac{W}{2}}\times C}$ represent the approximation (low–low), horizontal detail (low–high), vertical detail (high–low), and diagonal detail (high–high) sub-bands, respectively.

The low-frequency approximation sub-band ${\mathbf{F}}_{LL}$ captures the overall structural information and smooth intensity variations, which are particularly important for preserving the thermal signatures from IR imagery. The high-frequency detail sub-bands $\{{\mathbf{F}}_{LH},{\mathbf{F}}_{HL},{\mathbf{F}}_{HH}\}$ encode edge information and fine-grained textures, which are predominantly contributed by RGB imagery. We apply separate enhancement pathways for these components:

$$\begin{array}{cc}\hfill {\widehat{\mathbf{F}}}_{LL}& ={\mathbf{F}}_{LL}+\mathrm{SE}\left({\mathrm{Conv}}_{3\times 3}\left({\mathbf{F}}_{LL}\right)\right)\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill {\widehat{\mathbf{F}}}_H& =\mathrm{Concat}({\mathbf{F}}_{LH},{\mathbf{F}}_{HL},{\mathbf{F}}_{HH})\hfill \end{array}$$

(23)

$$\begin{array}{cc}\hfill {\widehat{\mathbf{F}}}_H& ={\widehat{\mathbf{F}}}_H+\mathrm{CBAM}\left({\mathrm{Conv}}_{3\times 3}\left({\widehat{\mathbf{F}}}_H\right)\right)\hfill \end{array}$$

(24)

where SE denotes the Squeeze-and-Excitation module [48] for channel-wise recalibration of structural features, and CBAM denotes the Convolutional Block Attention Module [49] for joint spatial-channel attention on textural details.

The enhanced sub-bands are then reconstructed through the inverse Haar wavelet transform:

$${\mathbf{F}}_{HWD}=\mathrm{IHWT}({\widehat{\mathbf{F}}}_{LL},{\widehat{\mathbf{F}}}_{LH},{\widehat{\mathbf{F}}}_{HL},{\widehat{\mathbf{F}}}_{HH})\in {\mathbb{R}}^{H\times W\times C}$$

(25)

where IHWT denotes the inverse Haar wavelet transform that upsamples and combines the sub-bands to reconstruct the full-resolution enhanced features.

3.2.2. Attention-Guided Kolmogorov–Arnold Refinement Stream (AKR-Stream)

The AKR-Stream complements the frequency-domain processing of HWD-Stream by applying adaptive nonlinear transformations through learnable spline-based activation functions. Inspired by Kolmogorov–Arnold networks [15], we replace fixed activation functions with B-spline parameterized univariate functions that can learn task-specific nonlinearities.

Given the fused features ${\mathbf{F}}_{fused}$, we first compute spatial attention weights to guide the refinement process:

$${\mathbf{A}}_{spatial}=\sigma \left({\mathrm{Conv}}_{7\times 7}\left([\mathrm{AvgPool}\left({\mathbf{F}}_{fused}\right);\mathrm{MaxPool}\left({\mathbf{F}}_{fused}\right)]\right)\right)$$

(26)

where $\sigma$ denotes the sigmoid function, and the attention map ${\mathbf{A}}_{spatial}\in {\mathbb{R}}^{H\times W\times 1}$ highlights spatially informative regions.

The core of AKR-Stream is the learnable spline-based activation layer. For each channel c, we define a B-spline function ${\varphi }_c\left(x\right)$ parameterized by a set of control points ${\left\{{\mathbf{p}}_c^{\left(k\right)}\right\}}_{k=0}^K$ on a uniform grid:

$${\varphi }_c\left(x\right)=\sum _{k=0}^K{\mathbf{p}}_c^{\left(k\right)}{B}_k^{\left(d\right)}\left(x\right)$$

(27)

where $B_k^{\left(d\right)}\left(x\right)$ denotes the k-th B-spline basis function of degree d, and $K+1$ is the number of control points. The B-spline basis functions are computed recursively using the Cox–de Boor formula:

$$\begin{array}{cc}\hfill B_k^{\left(0\right)}\left(x\right)& =\left\{\begin{array}{cc}1\hfill & \mathrm{if}{t}_k\le x<t_{k+1}\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.\hfill \end{array}$$

(28)

$$\begin{array}{cc}\hfill B_k^{\left(d\right)}\left(x\right)& ={\displaystyle \frac{x-t_k}{t_{k+d}-t_k}}{B}_k^{(d-1)}\left(x\right)+{\displaystyle \frac{t_{k+d+1}-x}{t_{k+d+1}-t_{k+1}}}{B}_{k+1}^{(d-1)}\left(x\right)\hfill \end{array}$$

(29)

where $\left\{t_k\right\}$ denotes the knot vector defining the B-spline parameterization.

The attention-guided KAN refinement is applied as

$${\mathbf{F}}_{AKR}={\mathbf{A}}_{spatial}\odot \mathsf{\Phi }\left({\mathbf{F}}_{fused}\right)+(1-{\mathbf{A}}_{spatial})\odot {\mathbf{F}}_{fused}$$

(30)

where $\mathsf{\Phi }(·)$ applies the learnable spline activation channel-wise, and ⊙ denotes element-wise multiplication. This formulation allows the network to selectively apply nonlinear refinement to informative regions while preserving original features in less salient areas.

The final output of DMEM combines the outputs from both streams:

$${\mathbf{F}}_{DMEM}={\mathrm{Conv}}_{1\times 1}\left([{\mathbf{F}}_{HWD};{\mathbf{F}}_{AKR}]\right)+{\mathbf{F}}_{fused}$$

(31)

where the residual connection ensures stable gradient flow during training.

3.3. Adaptive Feature Fusion Module (AFAM)

The Adaptive Feature Fusion Module (AFAM) is designed to dynamically balance the contributions of RGB and IR modalities based on their local informativeness. Unlike static fusion approaches that apply uniform weights across the entire feature map, AFAM employs complementary gating mechanisms that adaptively adjust modality weights at each spatial location. The design of AFAM is inspired by the complementary gating mechanism proposed in CGMamba [50], which demonstrated the effectiveness of adaptive feature selection for multimodal learning tasks.

As illustrated in Figure 2, the proposed AFAM employs complementary gating mechanisms to adaptively balance RGB and IR modality contributions.

Given the enhanced RGB and IR features ${\mathbf{F}}_{rgb}^{enh},{\mathbf{F}}_{ir}^{enh}\in {\mathbb{R}}^{H\times W\times C}$ from the DMEM, AFAM first computes modality-specific informativeness scores through parallel attention pathways. For each modality $m\in \{rgb,ir\}$, we compute

$$\begin{array}{cc}\hfill {\mathbf{g}}_m^{channel}& =\sigma \left(\mathrm{MLP}\left(\mathrm{GAP}\left({\mathbf{F}}_m^{enh}\right)\right)\right)\in {\mathbb{R}}^{1\times 1\times C}\hfill \end{array}$$

(32)

$$\begin{array}{cc}\hfill {\mathbf{g}}_m^{spatial}& =\sigma \left({\mathrm{Conv}}_{3\times 3}\left({\mathrm{Conv}}_{1\times 1}\left({\mathbf{F}}_m^{enh}\right)\right)\right)\in {\mathbb{R}}^{H\times W\times 1}\hfill \end{array}$$

(33)

where GAP denotes global average pooling, MLP is a two-layer perceptron with reduction ratio r, and $\sigma$ is the sigmoid activation.

The complementary gating mechanism ensures that the modality weights sum to unity at each spatial location, preventing feature magnitude explosion while enabling adaptive modality selection. This constraint is analogous to the matching conditions studied in combinatorial graph theory [51]:

$$\begin{array}{cc}\hfill {\mathbf{w}}_{rgb}& ={\displaystyle \frac{exp({\mathbf{g}}_{rgb}^{channel}\odot {\mathbf{g}}_{rgb}^{spatial})}{exp({\mathbf{g}}_{rgb}^{channel}\odot {\mathbf{g}}_{rgb}^{spatial})+exp({\mathbf{g}}_{ir}^{channel}\odot {\mathbf{g}}_{ir}^{spatial})}}\hfill \end{array}$$

(34)

$$\begin{array}{cc}\hfill {\mathbf{w}}_{ir}& =1-{\mathbf{w}}_{rgb}\hfill \end{array}$$

(35)

The fused features are computed as the weighted combination:

$${\mathbf{F}}_{AFAM}={\mathbf{w}}_{rgb}\odot {\mathbf{F}}_{rgb}^{enh}+{\mathbf{w}}_{ir}\odot {\mathbf{F}}_{ir}^{enh}$$

(36)

To further enhance crossmodal interaction, we apply a cross-attention mechanism that allows each modality to query information from the other:

$$\begin{array}{cc}\hfill {\mathbf{Q}}_{rgb}& ={\mathbf{F}}_{rgb}^{enh}{\mathbf{W}}_Q^{rgb},{\mathbf{K}}_{ir}={\mathbf{F}}_{ir}^{enh}{\mathbf{W}}_K^{ir},{\mathbf{V}}_{ir}={\mathbf{F}}_{ir}^{enh}{\mathbf{W}}_V^{ir}\hfill \end{array}$$

(37)

$$\begin{array}{cc}\hfill {\mathbf{F}}_{rgb\leftarrow ir}& =\mathrm{softmax}\left({\displaystyle \frac{{\mathbf{Q}}_{rgb}{\mathbf{K}}_{ir}^T}{\sqrt{d_k}}}\right){\mathbf{V}}_{ir}\hfill \end{array}$$

(38)

where ${\mathbf{W}}_Q,{\mathbf{W}}_K,{\mathbf{W}}_V$ are learnable projection matrices and $d_k$ is the key dimension. A symmetric operation computes ${\mathbf{F}}_{ir\leftarrow rgb}$.

The final AFAM output combines the gated fusion with crossmodal attention:

$${\mathbf{F}}_{out}={\mathbf{F}}_{AFAM}+\alpha ·({\mathbf{F}}_{rgb\leftarrow ir}+{\mathbf{F}}_{ir\leftarrow rgb})$$

(39)

where $\alpha$ is a learnable scaling parameter initialized to a small value (0.1) to ensure stable training.

3.4. Bilinear Attention-Enhanced Detection Module (BADM)

The Bilinear Attention-Enhanced Detection Module (BADM) is designed to capture second-order feature interactions for improved object localization in cluttered aerial scenes. Standard attention mechanisms compute first-order correlations through dot-product similarity, which may be insufficient for distinguishing objects with similar appearance but different spatial configurations. BADM addresses this limitation through factorized bilinear pooling [52] that models pairwise feature interactions.

As illustrated in Figure 3, the proposed BADM captures second-order feature interactions through factorized bilinear pooling for enhanced object localization.

Given the fused encoder features ${\mathbf{F}}_{enc}\in {\mathbb{R}}^{N\times C}$ (where $N=H\times W$ is the number of spatial positions) and object queries $\mathbf{Q}\in {\mathbb{R}}^{M\times C}$ (where M is the number of queries), BADM computes bilinear attention as follows.

First, we project the features and queries into a shared embedding space:

$$\begin{array}{cc}\hfill \mathbf{X}& ={\mathbf{F}}_{enc}{\mathbf{W}}_X\in {\mathbb{R}}^{N\times d}\hfill \end{array}$$

(40)

$$\begin{array}{cc}\hfill \mathbf{Y}& =\mathbf{Q}{\mathbf{W}}_Y\in {\mathbb{R}}^{M\times d}\hfill \end{array}$$

(41)

where ${\mathbf{W}}_X,{\mathbf{W}}_Y\in {\mathbb{R}}^{C\times d}$ are learnable projection matrices.

The bilinear interaction between queries and encoder features is computed using factorized bilinear pooling. Instead of computing the full bilinear tensor ${\mathbf{Y}}^T{\mathbf{W}}_B\mathbf{X}$ with ${\mathbf{W}}_B\in {\mathbb{R}}^{d\times d\times d}$, which would be computationally prohibitive, we employ a low-rank factorization:

$${\mathbf{B}}_{ij}=\sum _{k=1}^K\left({\mathbf{y}}_i^T{\mathbf{U}}_k\right)·\left({\mathbf{x}}_j^T{\mathbf{V}}_k\right)$$

(42)

where ${\mathbf{U}}_k,{\mathbf{V}}_k\in {\mathbb{R}}^{d\times r}$ are low-rank factor matrices, r is the rank of factorization, and K is the number of bilinear factors. This factorization reduces the computational complexity from $O\left(d^3\right)$ to $O\left(Kdr\right)$.

The bilinear attention weights are computed by combining first-order and second-order interactions:

$${\mathbf{A}}_{bilinear}=\mathrm{softmax}\left({\displaystyle \frac{\mathbf{Y}{\mathbf{X}}^T+\lambda \mathbf{B}}{\sqrt{d}}}\right)$$

(43)

where $\lambda$ is a learnable parameter that balances the contribution of bilinear interactions.

The attended features are computed as

$${\mathbf{F}}_{att}={\mathbf{A}}_{bilinear}{\mathbf{V}}_{enc}$$

(44)

where ${\mathbf{V}}_{enc}={\mathbf{F}}_{enc}{\mathbf{W}}_V$ is the value projection of encoder features.

To further enhance localization accuracy, BADM incorporates a spatial refinement branch that predicts offset corrections for bounding box coordinates:

$$\Delta \mathbf{bbox}={\mathrm{MLP}}_{loc}({\mathbf{F}}_{att}\oplus {\mathbf{F}}_{bilinear})$$

(45)

where ${\mathbf{F}}_{bilinear}=\mathrm{Hadamard}(\mathbf{Y}\mathbf{U},{\mathbf{A}}_{bilinear}\mathbf{X}\mathbf{V})$ captures the bilinear feature representation, and ⊕ denotes concatenation.

The final detection head predicts class probabilities and refined bounding boxes:

$$\begin{array}{cc}\hfill {\mathbf{p}}_{cls}& =\mathrm{softmax}\left({\mathrm{MLP}}_{cls}\left({\mathbf{F}}_{att}\right)\right)\hfill \end{array}$$

(46)

$$\begin{array}{cc}\hfill \mathbf{bbox}& ={\mathrm{MLP}}_{box}\left({\mathbf{F}}_{att}\right)+\Delta \mathbf{bbox}\hfill \end{array}$$

(47)

3.5. Loss Function

Following the DETR paradigm [11], we employ a set-based loss function that performs bipartite matching between predicted and ground-truth objects. The total loss consists of three components:

$${\mathcal{L}}_{total}={\lambda }_{cls}{\mathcal{L}}_{cls}+{\lambda }_{box}{\mathcal{L}}_{box}+{\lambda }_{giou}{\mathcal{L}}_{giou}$$

(48)

The classification loss ${\mathcal{L}}_{cls}$ is computed using focal loss [23] to address class imbalance:

$${\mathcal{L}}_{cls}=-\sum _{i=1}^M{\alpha }_t{(1-p_t)}^{\gamma }log\left(p_t\right)$$

(49)

where $p_t$ is the predicted probability for the target class, ${\alpha }_t$ is the class balancing factor, and $\gamma$ is the focusing parameter.

The bounding box regression loss ${\mathcal{L}}_{box}$ combines L1 loss and generalized IoU loss [53]:

$$\begin{array}{cc}\hfill {\mathcal{L}}_{box}& =\sum _{i=1}^M{\parallel {\mathbf{b}}_i-{\widehat{\mathbf{b}}}_{\sigma \left(i\right)}\parallel }_1\hfill \end{array}$$

(50)

$$\begin{array}{cc}\hfill {\mathcal{L}}_{giou}& =\sum _{i=1}^M(1-\mathrm{GIoU}({\mathbf{b}}_i,{\widehat{\mathbf{b}}}_{\sigma \left(i\right)}))\hfill \end{array}$$

(51)

where ${\mathbf{b}}_i$ and ${\widehat{\mathbf{b}}}_{\sigma \left(i\right)}$ denote the ground-truth and predicted bounding boxes, respectively, and $\sigma$ is the optimal assignment from Hungarian matching.

4. Experiments

4.1. Datasets

We evaluated our proposed MFE-DETR on two challenging benchmark datasets for aerial object detection.

DroneVehicle [5] is a large-scale RGB-IR multimodal dataset specifically designed for vehicle detection from UAV perspectives. The dataset contains 28,439 aligned RGB-IR image pairs captured across diverse scenarios including urban roads, residential areas, parking lots, and highways. Images are captured under both daytime and nighttime conditions, providing a comprehensive evaluation of multimodal fusion effectiveness. The dataset includes five vehicle categories: car, truck, bus, van, and freight car. Following the official split, we used 17,990 image pairs for training and 10,449 pairs for testing.

VisDrone [1] is a large-scale benchmark for object detection in drone-captured imagery, containing 10,209 images with over 540,000 annotated instances across 10 categories. The dataset is particularly challenging due to extreme scale variations, dense object distributions, and complex backgrounds. We used VisDrone to evaluate the generalization capability of our approach, particularly for small object detection. Following the standard protocol, we used 6,471 images for training and 548 images for validation.

4.2. Implementation Details

Our MFE-DETR was implemented using PyTorch 2.1 [54] and trained on 4 NVIDIA A100 GPUs. We adopted ResNet-50 [55] pretrained on ImageNet as the backbone network. The input images were resized to $640\times 640$ pixels with standard data augmentation, including random horizontal flipping, color jittering, and mosaic augmentation [20].

We used the AdamW optimizer [56] with an initial learning rate of $1\times {10}^{-4}$ and weight decay of $1\times {10}^{-4}$. The learning rate was decayed by a factor of 0.1 at epochs 80 and 120. The total training duration was 150 epochs with a batch size of 16. For the loss function, we set ${\lambda }_{cls}=2.0$, ${\lambda }_{box}=5.0$, and ${\lambda }_{giou}=2.0$. The focal loss parameters are set to $\alpha =0.25$ and $\gamma =2.0$.

For the DMEM module, we used cubic B-splines ($d=3$) with $K=8$ control points for the AKR-Stream. The Haar wavelet decomposition was applied at a single level. For AFAM, the channel reduction ratio was set to $r=16$. For BADM, we used $K=4$ bilinear factors with rank $r=64$.

Efficiency Measurement Protocol. To ensure reproducibility and fair comparison, we reported the efficiency metrics under a standardized protocol. All FPS and FLOPs measurements were conducted with input resolution $640\times 640$, batch size 1, using PyTorch inference on a single NVIDIA A100 GPU with FP32 precision. The cudnn benchmark mode was enabled for optimized convolution operations. FPS values were computed as the average over 1000 inference iterations after 100 warmup iterations to ensure stable measurements. FLOPs were calculated using the fvcore library with the same input configuration.

4.3. Evaluation Metrics

We adopted the standard COCO-style evaluation metrics [57] for comprehensive performance assessment. The primary metrics included:

• ${\mathrm{mAP}}_{50}$: Mean Average Precision at IoU threshold 0.5.
• ${\mathrm{mAP}}_{50:95}$: Mean Average Precision averaged over IoU thresholds from 0.5 to 0.95 with step 0.05.
• ${\mathrm{AP}}_S$, ${\mathrm{AP}}_M$, ${\mathrm{AP}}_L$: Average Precision for small (<${32}^2$ pixels), medium (${32}^2$–${96}^2$ pixels), and large (>${96}^2$ pixels) objects.

4.4. Comparison with State-of-the-Art Methods

We compared our MFE-DETR with state-of-the-art methods across three categories: two-stage detectors, single-stage detectors, and transformer-based detectors. For fair comparison, all methods were trained and evaluated using the same data splits and evaluation protocols.

4.4.1. Results on DroneVehicle Dataset

Table 2. Comparison with state-of-the-art methods on the DroneVehicle dataset. The best results are shown in bold. † indicates multimodal methods using both RGB and IR inputs.

Method	Year	${\mathrm{mAP}}_{50}$	${\mathrm{mAP}}_{50:95}$	${\mathrm{AP}}_{\mathit{S}}$	${\mathrm{AP}}_{\mathit{M}}$	${\mathrm{AP}}_{\mathit{L}}$
Two-Stage Detectors
Faster R-CNN [16]	2015	58.2	38.4	21.3	42.6	55.8
Cascade R-CNN [18]	2018	61.5	41.2	23.8	45.3	58.7
FPN [17]	2017	60.3	40.1	22.5	44.1	57.2
Single-Stage Detectors
RetinaNet [23]	2017	57.8	37.9	20.8	41.5	54.6
FCOS [24]	2019	59.4	39.2	21.9	43.2	56.3
YOLOv5-L	2020	64.2	43.5	25.1	47.8	61.2
YOLOv8-L [20]	2023	68.5	47.3	28.4	51.6	64.8
YOLOv9-E [21]	2024	70.2	49.1	30.2	53.4	66.5
YOLOv10-L [22]	2024	71.8	50.8	31.5	54.9	68.2
Transformer-Based Detectors
DETR [11]	2020	55.6	36.2	18.5	39.8	52.4
Deformable DETR [12]	2020	62.8	42.5	24.6	46.2	59.8
DINO [13]	2022	69.4	48.6	29.8	52.7	65.9
RT-DETR-L [14]	2024	71.2	51.3	32.1	55.4	68.7
RT-DETR-X [14]	2024	73.3	54.1	34.5	58.2	71.3
Multimodal Fusion Methods${}^{†}$
GAFF [8] ${}^{†}$	2021	65.8	44.2	26.3	48.5	62.4
CFT [9] ${}^{†}$	2022	68.2	46.8	28.1	50.9	64.7
LRAF-Net [10] ${}^{†}$	2023	70.5	49.3	30.4	53.6	67.2
CAGT [32] ${}^{†}$	2024	74.1	53.2	33.8	57.4	70.8
MFE-DETR (Ours)${}^{†}$	2026	78.6	57.8	37.2	62.1	75.4

presents the comprehensive comparison results on the DroneVehicle dataset. We organized the comparison methods into four categories based on their architectural paradigms.

The results demonstrate that our MFE-DETR achieves state-of-the-art performance across all evaluation metrics. Compared to the best-performing single-modality method RT-DETR-X, our approach achieves improvements of 5.3% in ${\mathrm{mAP}}_{50}$ and 3.7% in ${\mathrm{mAP}}_{50:95}$, validating the effectiveness of our multimodal fusion strategy. The improvements are particularly pronounced for small objects (${\mathrm{AP}}_S$: 37.2% vs. 34.5%), demonstrating that our frequency-domain feature enhancement effectively preserves fine-grained details crucial for detecting small vehicles in aerial imagery. To provide a more intuitive comparison, Figure 4 presents a comprehensive dual-panel visualization: a radar chart showing normalized performance metrics across multiple dimensions, and a line chart illustrating the progressive performance improvement with module additions.

Among multimodal fusion methods, MFE-DETR outperforms the previous state-of-the-art CAGT by 4.5% in ${\mathrm{mAP}}_{50}$ and 4.6% in ${\mathrm{mAP}}_{50:95}$. This significant improvement can be attributed to three factors: (1) the HWD-Stream effectively separates and enhances frequency-specific information from both modalities, (2) the AKR-Stream provides adaptive nonlinear feature refinement that complements the frequency-domain processing, and (3) the BADM captures second-order feature interactions that improve localization accuracy in cluttered scenes.

The two-stage detectors (Faster R-CNN and Cascade R-CNN) show relatively lower performance compared to modern single-stage and transformer-based methods, primarily due to their limited capacity for handling the extreme scale variations present in aerial imagery. Single-stage detectors from the YOLO family demonstrate progressively improving performance with each generation, with YOLOv10-L achieving competitive results through its end-to-end design. However, these methods still fall short of transformer-based approaches that can model long-range dependencies through self-attention mechanisms.

4.4.2. Results on VisDrone Dataset

To evaluate the generalization capability of our approach, we conducted experiments on the VisDrone dataset using only RGB inputs. This evaluation demonstrates that our proposed modules (DMEM, AFAM, and BADM) provide benefits beyond multimodal fusion, improving feature representation for general aerial object detection.

Module Adaptation for Single-Modality Input. When applied to RGB-only scenarios, our modules gracefully degrade to single-modality operation: (1) AFAM degenerates to identity mapping with single-branch gating, as the complementary gating mechanism naturally assigns full weight to the available modality; (2) the HWD-Stream and AKR-Stream within DMEM continue to function as general-purpose feature enhancement modules, performing frequency-domain decomposition and adaptive nonlinear transformation on single-modality features; (3) BADM operates as a detection head enhancement module that is inherently modality-agnostic, capturing second-order feature interactions regardless of input modality.

As summarized in

Table 3. Comparison with state-of-the-art methods on the VisDrone dataset. All methods use RGB inputs only.

Method	Year	${\mathrm{mAP}}_{50}$	${\mathrm{mAP}}_{50:95}$	${\mathrm{AP}}_{\mathit{S}}$	${\mathrm{AP}}_{\mathit{M}}$	${\mathrm{AP}}_{\mathit{L}}$
Two-Stage Detectors
Faster R-CNN [16]	2015	28.4	16.2	8.5	24.3	38.6
Cascade R-CNN [18]	2018	31.2	18.5	10.2	27.1	41.8
Single-Stage Detectors
RetinaNet [23]	2017	26.8	15.1	7.8	22.6	36.4
FCOS [24]	2019	29.5	17.3	9.4	25.2	39.7
YOLOv8-L [20]	2023	37.2	22.8	14.1	32.5	48.3
YOLOv9-E [21]	2024	38.6	24.1	15.3	33.8	49.7
YOLOv10-L [22]	2024	39.4	25.2	16.1	34.6	50.8
SPD-YOLOv8 [26]	2024	40.1	25.8	16.8	35.2	51.4
SCA-YOLO [27]	2024	40.5	26.1	17.2	35.6	51.9
Transformer-Based Detectors
DETR [11]	2020	25.3	14.2	6.8	21.4	35.2
Deformable DETR [12]	2020	33.5	19.8	11.5	29.2	44.6
DINO [13]	2022	38.2	23.6	14.8	33.1	49.2
RT-DETR-L [14]	2024	40.2	25.4	16.5	35.1	51.2
RT-DETR-X [14]	2024	41.5	26.8	17.1	36.4	52.8
MFE-DETR (Ours)	2026	42.8	28.3	18.6	37.9	54.2

, our method achieves superior performance compared with state-of-the-art detectors on the VisDrone dataset.

On the VisDrone dataset, MFE-DETR achieves 42.8% ${\mathrm{mAP}}_{50}$ and 28.3% ${\mathrm{mAP}}_{50:95}$, outperforming the baseline RT-DETR-X by 1.3% and 1.5%, respectively. The improvement is most significant for small objects (${\mathrm{AP}}_S$: 18.6% vs. 17.1%), demonstrating that our wavelet-based feature decomposition effectively preserves high-frequency details crucial for detecting small objects in complex aerial scenes.

4.5. Ablation Studies

We conducted comprehensive ablation studies on the DroneVehicle dataset to analyze the contribution of each proposed component.

4.5.1. Component Analysis

Table 4. Ablation study on the contribution of each proposed component on DroneVehicle dataset.

DMEM	AFAM	BADM	${\mathrm{mAP}}_{50}$	${\mathrm{mAP}}_{50:95}$	${\mathrm{AP}}_{\mathit{S}}$	${\mathrm{AP}}_{\mathit{M}}$	${\mathrm{AP}}_{\mathit{L}}$
			71.2	51.3	32.1	55.4	68.7
✓			74.5	54.2	34.6	58.3	71.5
	✓		73.8	53.5	33.9	57.6	70.8
		✓	73.2	52.8	33.2	56.9	70.2
✓	✓		76.4	55.9	35.8	60.2	73.4
✓		✓	75.8	55.3	35.2	59.5	72.7
	✓	✓	75.1	54.6	34.5	58.8	72.1
✓	✓	✓	78.6	57.8	37.2	62.1	75.4

presents the ablation results for each proposed module. Starting from the RT-DETR-L baseline, we progressively added each component to evaluate its individual contribution.

The results reveal several important findings. First, DMEM provides the largest individual improvement (+3.3% ${\mathrm{mAP}}_{50}$), demonstrating the effectiveness of frequency-domain feature enhancement for multimodal fusion. The improvement is particularly pronounced for small objects (+2.5% ${\mathrm{AP}}_S$), validating our hypothesis that wavelet decomposition helps preserve fine-grained details. Second, AFAM contributes +2.6% ${\mathrm{mAP}}_{50}$ by enabling adaptive modality balancing, which is crucial for handling varying illumination conditions in the DroneVehicle dataset. Third, BADM improves ${\mathrm{mAP}}_{50}$ by +2.0% through better localization via second-order feature interactions. Finally, the combination of all three modules achieves synergistic improvements, with the full model outperforming the baseline by +7.4% ${\mathrm{mAP}}_{50}$. Figure 5 provides a comprehensive dual-panel visualization: (a) individual module contributions with color-coded bars, and (b) progressive performance trends across different module combinations.

4.5.2. Analysis of DMEM Components

Table 5. Ablation study on DMEM components.

HWD-Stream	AKR-Stream	${\mathrm{mAP}}_{50}$	${\mathrm{mAP}}_{50:95}$	${\mathrm{AP}}_{\mathit{S}}$	${\mathrm{AP}}_{\mathit{M}}$	${\mathrm{AP}}_{\mathit{L}}$
		71.2	51.3	32.1	55.4	68.7
✓		73.4	53.1	33.8	57.2	70.4
	✓	72.8	52.5	33.2	56.5	69.8
✓	✓	74.5	54.2	34.6	58.3	71.5

analyzes the contribution of the two streams within DMEM: the HWD-Stream and the AKR-Stream.

Both streams contribute positively to the overall performance, with HWD-Stream providing slightly larger improvements (+2.2% ${\mathrm{mAP}}_{50}$) compared to AKR-Stream (+1.6% ${\mathrm{mAP}}_{50}$). The combination of both streams achieves the best performance, indicating that frequency-domain decomposition and adaptive nonlinear transformation provide complementary benefits for multimodal feature enhancement.

4.5.3. Impact of Wavelet Decomposition Level

We investigated the impact of wavelet decomposition levels in the HWD-Stream.

Table 6. Impact of wavelet decomposition levels in HWD-Stream.

Levels	${\mathrm{mAP}}_{50}$	${\mathrm{mAP}}_{50:95}$	${\mathrm{AP}}_{\mathit{S}}$	${\mathrm{AP}}_{\mathit{M}}$	${\mathrm{AP}}_{\mathit{L}}$	FLOPs (G)
1	78.6	57.8	37.2	62.1	75.4	142.3
2	78.2	57.4	36.8	61.7	75.1	156.8
3	77.5	56.8	36.1	61.0	74.5	171.4

shows the results with different decomposition levels.

Single-level decomposition achieves the best performance while maintaining computational efficiency. Deeper decomposition levels introduce additional computational overhead without corresponding accuracy improvements, likely because the increased abstraction loses fine-grained spatial information important for object detection.

4.5.4. Impact of B-Spline Parameters in AKR-Stream

Table 7. Impact of B-spline parameters in AKR-Stream.

Degree d	Control Points K	${\mathrm{mAP}}_{50}$	${\mathrm{mAP}}_{50:95}$	${\mathrm{AP}}_{\mathit{S}}$	${\mathrm{AP}}_{\mathit{M}}$	${\mathrm{AP}}_{\mathit{L}}$
2	6	77.8	57.1	36.5	61.4	74.7
2	8	78.1	57.4	36.8	61.7	75.0
3	6	78.2	57.5	36.9	61.8	75.1
3	8	78.6	57.8	37.2	62.1	75.4
3	10	78.4	57.6	37.0	61.9	75.2
4	8	78.3	57.5	36.9	61.8	75.1

analyzes the impact of B-spline parameters (degree d and number of control points K) in the AKR-Stream.

Cubic B-splines ($d=3$) with 8 control points achieve the best balance between expressiveness and regularization. Higher-degree splines or more control points do not provide additional benefits, suggesting that the learned activation functions do not require excessive flexibility for this task.

4.5.5. Computational Efficiency Analysis

Table 8. Computational efficiency comparison.

Method	Params (M)	FLOPs (G)	FPS	${\mathrm{mAP}}_{50}$
YOLOv8-L	43.7	165.2	78.5	68.5
YOLOv10-L	24.4	120.3	92.3	71.8
RT-DETR-L	32.0	108.0	114.2	71.2
RT-DETR-X	67.4	234.7	74.8	73.3
CAGT	58.2	198.5	52.6	74.1
MFE-DETR (Ours)	45.8	142.3	68.4	78.6

compares the computational efficiency of MFE-DETR with other methods.

DME-DETR achieves the best accuracy while maintaining reasonable computational efficiency. Compared to CAGT, our method achieves 4.5% higher ${\mathrm{mAP}}_{50}$ with 28% fewer FLOPs and 30% faster inference speed. The efficiency gains come from our streamlined module designs that avoid redundant computations while maximizing feature enhancement effectiveness. Figure 6 provides a visual analysis of the accuracy-efficiency trade-off, where bubble size represents inference speed (FPS) and the optimal region highlights methods achieving high accuracy with moderate computational cost.

4.6. Qualitative Results

Figure 7 presents qualitative detection results on the DroneVehicle dataset, comparing our MFE-DETR with RT-DETR-X and CAGT under various challenging conditions.

The visualizations demonstrate several advantages of our approach. In nighttime scenes (rows 1–2), MFE-DETR successfully detects vehicles that are missed by single-modality methods, benefiting from the effective fusion of RGB and IR information through AFAM. In dense traffic scenarios (rows 3–4), our method achieves more accurate localization with fewer false positives, attributed to the second-order feature interactions captured by BADM. For small objects (rows 5–6), MFE-DETR maintains high detection accuracy due to the frequency-domain feature enhancement provided by the HWD-Stream. To further analyze the detection performance across different vehicle categories and evaluation metrics, Figure 8 presents a comprehensive dual-panel comparison.

Figure 9 shows detection results on the VisDrone dataset, demonstrating the generalization capability of our approach for small object detection.

On VisDrone, MFE-DETR demonstrates superior performance in detecting small pedestrians and vehicles in crowded urban scenes. The wavelet-based feature decomposition effectively preserves high-frequency details that are crucial for distinguishing small objects from complex backgrounds.

5. Discussion

The experimental results demonstrate that MFE-DETR achieves state-of-the-art performance on both DroneVehicle and VisDrone datasets, validating the effectiveness of our proposed modules for multimodal aerial object detection. In this section, we discuss the key insights and limitations of our approach.

It is important to emphasize that our three proposed modules (DMEM, AFAM, and BADM) are not independent engineering additions but rather form a unified framework addressing the symmetry-asymmetry tension in multimodal fusion. DMEM handles the frequency-domain asymmetry between modalities, AFAM manages the spatial-temporal asymmetry in modality reliability, and BADM captures the higher-order symmetric correlations for precise localization. The ablation studies in Table 4 demonstrate that these modules provide complementary rather than redundant benefits, with the full combination achieving synergistic improvements (+7.4% ${\mathrm{mAP}}_{50}$) that exceed the sum of individual contributions.

The success of DMEM can be attributed to its ability to separately process frequency-specific information from multimodal features. The HWD-Stream effectively separates low-frequency structural information (predominantly from IR imagery) from high-frequency textural details (predominantly from RGB imagery), enabling specialized enhancement pathways that preserve modality-specific characteristics. The AKR-Stream complements this by providing adaptive nonlinear transformations that can learn task-specific activation functions, offering greater flexibility than fixed activations like ReLU or GELU. The choice of Kolmogorov–Arnold network-inspired activations is motivated by their theoretical foundation in universal function approximation and their demonstrated effectiveness in learning complex nonlinear mappings, as validated by the ablation study in Table 5.

The AFAM module addresses a fundamental challenge in multimodal fusion: the varying reliability of different modalities across spatial locations and environmental conditions. By employing complementary gating mechanisms, AFAM can dynamically suppress unreliable modality contributions while amplifying informative features. This adaptive behavior is particularly important for the DroneVehicle dataset, which contains images captured under diverse illumination conditions where the relative informativeness of RGB and IR modalities varies significantly. Figure 10 illustrates the adaptive fusion weights learned by AFAM under different environmental conditions.

The BADM module improves localization accuracy by capturing second-order feature interactions through factorized bilinear pooling. Standard attention mechanisms compute first-order correlations that may be insufficient for distinguishing objects with similar appearance but different spatial configurations. The bilinear attention in BADM enables more discriminative feature matching, leading to improved bounding box predictions.

Despite the strong performance, our approach has several limitations that warrant future investigation, which we discuss as actionable boundary conditions for practical deployment.

Alignment Assumption. The current implementation assumes well-aligned RGB-IR image pairs, which may not always be available in practical deployments due to sensor mounting variations, temporal offsets, or parallax effects. While our AFAM can partially compensate for minor misalignments through adaptive weighting, significant geometric discrepancies would degrade fusion quality. The reliability of multimodal systems under such conditions relates to broader questions of network diagnosability and fault tolerance [58,59]. Future work could incorporate learnable registration modules or develop weakly-aligned fusion mechanisms that explicitly model and compensate for crossmodal geometric transformations.

Thermal Saturation and IR Noise Robustness. Our method’s performance under extreme thermal conditions (e.g., thermal saturation in high-temperature industrial environments or severe IR noise in adverse weather) remains to be thoroughly characterized. The HWD-Stream’s frequency decomposition may be sensitive to thermal artifacts that manifest as spurious high-frequency components. Developing robust preprocessing or adaptive filtering mechanisms for handling thermal saturation and sensor noise would enhance the practical applicability of our approach.

Edge Deployment Constraints. While MFE-DETR achieves real-time performance (68.4 FPS) on high-end GPUs, deployment on resource-constrained edge devices commonly used in UAV systems requires further optimization. The wavelet decomposition and bilinear attention operations, while effective, introduce computational overhead that may be prohibitive for embedded platforms. Future work should explore model compression techniques including structured pruning, knowledge distillation, and quantization-aware training to enable efficient edge deployment without significant accuracy degradation.

Future work could explore several promising directions. Extending MFE-DETR to handle temporal information from video sequences could improve detection consistency and enable tracking capabilities. Incorporating uncertainty estimation could provide confidence measures for detection outputs, which is important for safety-critical applications. Additionally, exploring self-supervised pretraining strategies for multimodal representations could reduce the reliance on large-scale annotated datasets.

6. Conclusions

In this paper, we proposed MFE-DETR, a novel detection transformer framework for RGB-IR multimodal object detection in aerial imagery. Motivated by the observation that RGB and IR modalities exhibit approximate symmetry in spatial structures while demonstrating intrinsic asymmetry in imaging characteristics, our approach addresses three fundamental challenges in multimodal fusion through three synergistic innovations: the Dual-Modality Enhancement Module (DMEM) for frequency-domain and adaptive feature enhancement, the Adaptive Feature Fusion Module (AFAM) for dynamic modality balancing, and the Bilinear Attention-Enhanced Detection Module (BADM) for second-order feature interaction modeling.

The DMEM comprises two complementary processing streams: the HWD-Stream that leverages Haar wavelet decomposition for multi-resolution frequency analysis, and the AKR-Stream that employs Kolmogorov–Arnold network-inspired learnable spline activations for adaptive nonlinear transformation. The AFAM employs complementary gating mechanisms to dynamically balance modality contributions based on spatial informativeness, effectively addressing the modality imbalance problem. The BADM captures second-order feature interactions through factorized bilinear pooling, enabling fine-grained crossmodal correlation modeling for improved localization accuracy.

Comprehensive experiments on the DroneVehicle and VisDrone datasets demonstrate that MFE-DETR achieves state-of-the-art performance. On DroneVehicle, our method achieves 78.6% ${\mathrm{mAP}}_{50}$ and 57.8% ${\mathrm{mAP}}_{50:95}$, surpassing previous methods by significant margins. On VisDrone, MFE-DETR demonstrates superior generalization capability with 42.8% ${\mathrm{mAP}}_{50}$ and particularly strong performance on small objects (18.6% ${\mathrm{AP}}_S$). Extensive ablation studies validate the contribution of each proposed component and provide insights into optimal design choices.

Our work demonstrates the potential of combining frequency-domain analysis, adaptive nonlinear transformations, and higher-order feature interactions for effective multimodal fusion in aerial object detection. We believe that the proposed techniques can be extended to other multimodal perception tasks and contribute to the development of more robust autonomous systems.