ISDG-Net: Efficient RGB–Infrared Object Detection for Remote Sensing Imagery

Highlights

What are the main findings?

• We have proposed the ISDG-Net, a lightweight RGB–infrared detection framework, which integrates modules such as IBC-Conv, DySparse, Detect-SASD, and GIS to achieve efficient cross-modal fusion.
• This framework demonstrated extremely high accuracy on the VEDAI, M3FD and LLVIP datasets, while using only 42 million parameters. An extremely low cost of computing resources, with 1.13 billion floating-point operations.

What are the implications of the main findings?

• This tailored, integrated architecture successfully resolves the differences between various modes and reduces the excessive suppression of densely arranged small objects in complex and dimly lit environments.
• By achieving high-precision detection with extremely low computational overhead, this model makes it possible to deploy remote sensing object detection on resource-constrained edge platforms.

Abstract

In all-weather Earth observation and complex unstructured environments, traditional single-modal remote sensing object detection often fails due to low illumination and strong background interference. While RGB–infrared fusion provides complementary information, existing methods are typically computationally intensive and struggle with dense small objects and modality discrepancies, limiting their deployment on resource-constrained platforms. To address these challenges, we propose ISDG-Net, a lightweight and efficient visible-infrared dual-modal object detection framework specifically tailored for edge deployment. ISDG-Net integrates four core components: (1) a channel-separated inverted bottleneck backbone (IBC-Conv) that reduces parameter redundancy while preserving modality-specific semantics; (2) a dynamic sparse attention module (DySparse) based on Bi-Level Routing Attention, enabling long-range dependency modeling with low computational cost; (3) an adaptive spatial fusion detection head (Detect-SASD) that aligns visible and infrared features at the pixel level to resolve semantic inconsistency and scale mismatch; and (4) a geometry-aware IoU selector (GIS) that mitigates over-suppression in crowded scenes by incorporating multi-dimensional geometric constraints into post-processing. Extensive experiments on the VEDAI, M3FD, and LLVIP datasets demonstrate the effectiveness and efficiency of ISDG-Net. It achieves 55.1% and 77.1% mAP@0.5 on VEDAI and M3FD, respectively, and 93.7% mAP@0.5 with 89.7% recall on LLVIP, while maintaining a compact model size of 4.2 M parameters and 11.3 GFLOPs. These results validate that accurate RGB–infrared detection is achievable under strict resource constraints, making ISDG-Net well-suited for deployment in edge-based remote sensing systems.

1. Introduction

1.1. Background and Significance

In key fields such as homeland security monitoring, disaster emergency response, and resource exploration and assessment, remote sensing object detection technology serves as the core support for information extraction, and its detection accuracy and robustness directly determine decision-making efficiency and reliability [1]. With the iteration of remote sensing platforms and sensor technologies, the capacity to obtain high-definition remote sensing data has seen remarkable enhancement. However, in practical applications, low-visibility conditions such as cloud and fog obscuration, sudden illumination changes, and night-time imaging appear frequently, causing traditional target recognition techniques to encounter substantial obstacles [2,3].

Aiming to tackle the object localization difficulties under low-visibility conditions, early research was mainly dedicated to single-modal image enhancement technologies. Researchers attempted to introduce Retinex theory, histogram equalization (HE) and its improved algorithms (exemplified by Contrast Limited Adaptive Histogram Equalization (CLAHE)), or dehazing approaches relying upon the dark channel prior (DCP), attempting to recover object details by improving the contrast and clarity of the images [4,5]. However, these methods have essential limitations: image enhancement is essentially a statistical adjustment of existing pixel values, and it cannot recover the physical information completely lost due to insufficient illumination. In extremely low signal-to-noise ratio environments, enhancement algorithms often introduce a large amount of background noise and artifacts while amplifying the target signal, which instead destroys the distribution patterns of the original features. Furthermore, these preprocessing steps increase additional computational overhead, making it difficult to meet the real-time processing requirements of remote sensing platforms. Therefore, relying solely on single-modal algorithm optimization has reached a performance ceiling, and there is an urgent need to introduce complementary modalities such as infrared or Synthetic Aperture Radar (SAR) to break through the perception bottleneck.

To overcome the limitations of optical sensors in visually degraded environments, Synthetic Aperture Radar (SAR) has been widely utilized as a robust single-modal alternative for all-weather remote sensing. In the SAR domain, overcoming complex backgrounds and scale variations heavily relies on efficient feature representation. For instance, the Multi-scale Rotation-Invariant Haar-like feature integrated CNN (MSRIHL-CNN) effectively addressed multi-scale challenges in dense multi-target environments [6], while the Multi-kernel-size Feature Fusion-based Convolutional Neural Network (MKSFF-CNN) significantly improved target classification accuracy through multi-scale kernel integration [7]. Furthermore, to handle complex missing-modality scenarios, the Light-weight Modality Compensation Network (LMCNet) provided a highly efficient compensation strategy [8]. Although these remarkable methods primarily focus on the SAR modality, their underlying algorithmic philosophies—specifically lightweight architecture design, multi-scale feature aggregation, and robust fusion strategies—are universal. These concepts provide profound inspiration for the broader remote sensing community to overcome similar scale and modality heterogeneity bottlenecks by exploring another highly complementary paradigm: visible-infrared multi-modal detection.

Visible light optical imagery serves as the primary input for recognition tasks due to its abundant color and spatial details, yet it remains highly vulnerable to variations in lighting intensity and weather conditions; infrared images achieve imaging by recording thermal disparities separating the objects of interest from their surroundings, possessing all-weather working capability, but they lose key detailed texture features. When either is applied alone, it often fails to fulfill the identification demands of intricate environments. Therefore, integrating the distinct advantages of both optical and thermal spectrums to build resilient detection architectures represents a pivotal trend in smart remote sensing analysis.

Overcoming the limitations of remote sensing target localization in degraded visibility has driven recent efforts toward both optical-only algorithms [9] and cross-modal fusion architecture.

By thoroughly exploiting the cross-modal synergies between optical and thermal imagery, dual-modal fusion has emerged as the primary mechanism for overcoming the limitations of remote sensing detection in poor visibility, prompting a surge of recent literature in this area. Chen et al. [10] proposed an Alignment-Assisted Frequency Fusion Network (AFFNet), achieving 75.6% mAP@0.5 on the FLIR dataset, which effectively solved the dual-modal data misalignment problem and strengthened the cross-domain feature complementarity of vehicle targets. Zhao et al. [11] proposed an Edge Feature Enhanced Transformer Network (EFETN), achieving 96.4% mAP@0.5 on the LLVIP dataset, which strengthened the target edge features and the capture of cross-modal long-range dependencies through learnable edge extraction operators and Transformer global modeling units. Hao et al. [12] proposed a fusion strategy driven by energy scores for feature augmentation (EEF), achieving 96.9% mAP@0.5 on the LLVIP dataset, which realized adaptive feature enhancement by calculating the modal feature energy scores to dynamically allocate fusion weights. Meng et al. [13] proposed a Fusion-enhanced Quad-head Network (FQDNet), achieving 88.8% mAP@0.5 on the M3FD dataset, which improved the multi-scale object recognition accuracy by using four dedicated detection heads to process single-modal features and fused features, respectively. Yuan et al. [14] proposed a Cascaded Alignment-Guided Transformer (CAGT), achieving 74.57% mAP@0.5 on the DroneVehicle dataset, which optimized cross-modal feature consistency through multi-stage modal alignment and Transformer cross-attention modules.

These methods have achieved significant results in specific scenarios, but there are still common limitations: they cannot simultaneously account for the global context and local details of dual-modal features, the target recognition robustness is relatively low under complex backgrounds, while the bounding box localization deviation increases, and the missed detection problem of dense remote sensing targets is noticeable. At the same time, the models tend to fall into the dilemma of parameter redundancy and low computational efficiency, and the discriminability of dual-modal features is insufficient. Multi-scale remote sensing targets are prone to scale conflicts, and the recognition performance for miniature targets remains severely limited.

Despite these substantial advancements, the domain of aerial target recognition continues to face two primary bottlenecks:

(1)

Inherent limitations of single-modal image object detection: The visible light modality depends on illumination conditions and is prone to feature blurring or even target loss in scenes such as night-time or rainy weather; although the infrared modality is unaffected by illumination, it cannot capture key discriminative information such as color and texture, resulting in increased difficulty in distinguishing similar targets and a significant drop in recognition accuracy.

(2)

Multi-modal detection accuracy needs to be improved: Existing fusion methods mostly adopt fixed weights or simple concatenation to process dual-modal features, making them inflexible to the fluctuating distributions of modal information in different scenes; some models ignore the global-local information synergy during the course of generating and blending cross-domain representations, thereby deteriorating the recognition accuracy for diminutive targets and under complex backgrounds.

1.2. Main Contributions

Motivated by the inherent vulnerabilities of unimodal systems under adverse weather and the current bottlenecks in fusion efficiency, cross-modal alignment, and crowded scene analysis, we introduce ISDG-Net: a streamlined and highly effective RGB-thermal detection architecture. Our model establishes a novel standard for seamlessly integrating optical and thermal signals under lightweight constraints, and adapts dynamic sparse attention, adaptive spatial fusion, and geometry-aware post-processing strategies, achieving accurate detection of all-weather, multi-scale remote sensing objects. Our main contributions are as follows:

(1)

We proposed ISDG-Net, a two-stream target recognition architecture tailored for complex, low-illumination remote sensing environments. Overcoming single-sensor limitations, it significantly enhances multi-scale object localization. Evaluations on VEDAI, M3FD, and LLVIP datasets demonstrate that ISDG-Net surpasses state-of-the-art models, achieving 55.1% mAP@0.5 on VEDAI and 93.7% on LLVIP, proving its exceptional accuracy, rapid convergence, and robust cross-domain generalization.

(2)

We design an inverted bottleneck module based on channel separation (IBC-Conv) to construct an efficient dual-modal feature extraction backbone, addressing edge deployment challenges. By utilizing inverted residuals and depthwise separable convolutions, it independently extracts visible textures and infrared characteristics, preventing mutual interference. This lossless semantic transmission significantly reduces parameter redundancy, improving cross-modal fusion and detection accuracy in complex scenes.

(3)

We integrate and adapt DySparse, a dynamic sparse Transformer module designed to alleviate the computational burden of global modeling in high-resolution imagery. By incorporating Bi-Level Routing Attention, DySparse employs a query-aware strategy to sparsely connect only the most semantically relevant regions. This approach reduces complexity while effectively suppressing background noise, thereby enhancing global context perception for small objects.

(4)

We develop a detection head integrating adaptive spatial feature fusion (Detect-SASD) to overcome modality heterogeneity and scale deviations. Utilizing a learnable adaptive weight map, it dynamically adjusts pixel-level dual-modal fusion ratios for accurate multi-scale alignment. This mechanism significantly enhances feature robustness under low illumination and complex backgrounds, effectively solving missed detections of multi-scale objects.

(5)

We adapt the CIoU and Soft-NMS mechanisms to construct a geometry-aware greedy IoU selector (GIS, distinct from Geographic Information Systems) to address the false suppression problem of traditional NMS in densely arranged scenes. By introducing multi-dimensional CIoU metrics (overlap area, center distance, aspect ratio), GIS reconstructs the suppression strategy without increasing training costs. This effectively distinguishes highly overlapping adjacent objects, significantly improving recall rates in dense vehicle scenes.

To outline the remainder of this article: Section 2 provides a comprehensive survey of prior work concerning conventional target recognition and cross-modal RGB–infrared techniques. Section 3 details the architectural design of the ISDG-Net model proposed in this paper, covering the theoretical design and specific implementation of the overall network architecture and core components such as the IBC-Conv backbone, the Detect-SASD detection head, the DySparse attention module, and the GIS post-processing strategy. Afterward, in Section 4, an extensive empirical evaluation across several standard aerial datasets is provided, where the performance superiority and computational efficiency of our approach are rigorously assessed via qualitative and quantitative comparative analysis, ablation experiments, and generalization ability verification. Finally, Section 5 summarizes the work of the entire paper and discusses the deployment potential of this technology in actual complex environments and future research directions.

2. Related Works

2.1. Traditional Object Detection Algorithms

The development of deep learning object detection algorithms has long been dominated by two-stage and one-stage architectures based on Convolutional Neural Networks (CNN). Dual-stage frameworks like Faster R-CNN [15] operate on a mechanism that initially extracts region proposals before optimizing categorization and bounding box coordinates, and have established a benchmark status in high-precision recognition under complex backgrounds through the Region Proposal Network (RPN). Nevertheless, the cumbersome proposal extraction steps and the requisite Non-Maximum Suppression (NMS) procedures inherent in two-stage methods limit their potential for end-to-end optimization and inference speed. Subsequently, the emergence of architectures such as Vision Transformer (ViT) [16] and Detection Transformer (DETR) [17] leveraged global self-attention to successfully address the limitations of standard CNNs in capturing distant spatial correlations [18,19]. However, the high quadratic computational complexity of the standard Transformer leads to extremely high GPU memory consumption [20], which severely hinders its practical application where strict low-latency edge computing is required on remote sensing edge devices. Therefore, the industry has gradually turned to more efficient one-stage detection schemes.

As a typical prominent single-stage framework, the You Only Look Once (YOLO) family [21] has secured an excellent balance of inference velocity and precision through a streamlined architecture. Consequently, it has been continuously iterating in recent years [22,23]: YOLOv8 introduced the C2f module and dynamic label assignment strategy, significantly improving the efficiency of feature reuse; YOLOv10 [24] combined a lightweight architecture with a hybrid attention mechanism, optimizing the recognition capability for miniature targets while reducing the number of parameters; YOLOv11 [25] utilized the Dynamic Feature Pyramid Network (Dynamic FPN) to fortify the algorithm’s robustness in occluded and blurred scenes. Aiming at cross-domain and edge-side applications, YOLOv12 [26] and YOLOv13 [27] achieved breakthroughs in cross-modal feature fusion and sparse inference, respectively, laying a technical foundation for efficient detection in open-world scenarios by introducing modality-adaptive attention and large-model-assisted training. Concurrently, researchers have developed various YOLO adaptations specifically tailored for remote sensing imagery, such as lightweight designs [28], optimized small object detection [29], and general aerial target recognition frameworks [30]. Even so, when processing multi-modal data under extremely low-visibility conditions, the above methods still face the challenges of multi-scale feature alignment difficulties and computational redundancy, which provides an opportunity for this paper to propose a lightweight and globally aware detection framework.

2.2. Visible-Infrared Object Detection

Although single-modal detectors perform robustly under ideal illumination conditions, when exposed to adverse scenarios like darkness, fog, or intense light interference, the signal-to-noise ratio (SNR) of optical sensors degrades sharply [31], while infrared thermal imaging has limited feature discriminability because they inherently omit pigmentation and detailed visual patterns [32,33]. To break through the perception bottleneck of a single physical sensor, visible-infrared (RGB-IR) multi-modal fusion strategies have emerged as the primary solution for enhancing the reliability of detection systems. According to the processing stage where fusion occurs, current integration techniques are generally categorized into three distinct paradigms: pixel-, feature-, and decision-level fusions.

Both pixel- and decision-based integrations, as early exploration paths in this field, have significant theoretical and practical limitations [34]. Pixel-level fusion attempts to synthesize a “super image” at the input end through weighted average or Laplacian pyramid transform, but this forced spatial alignment often performs poorly and inevitably destroys the unique statistical distribution characteristics of each modality [35]; although decision-level fusion preserves modality independence by performing weighted voting on the outputs of independent detectors, due to the lack of semantic interaction of deep features, coupled with double the inference computational overhead, it is difficult to meet the dual constraints of real-time performance and accuracy of remote sensing platforms.

In view of this, feature-level fusion—that is, performing semantic interaction within the deep feature space of neural networks—has evolved into the current mainstream paradigm [36,37]. In this field, researchers have proposed various architectural strategies to address the challenge of modality heterogeneity. Early two-stream architectures mostly adopted linear operations such as channel concatenation or element-wise addition to aggregate features [38]. However, this static fusion strategy ignores the complementary differences in different modalities in complex scenes and easily leads to modality-specific background noise polluting the fused features, reducing the effectiveness of feature expression.

To overcome the limitations of static strategies, recent research has been dedicated to introducing attention mechanisms and alignment modules to enhance the dynamic adaptability of features [39]. For example, AFFNet proposed by Chen et al. [10] utilized frequency domain analysis to effectively alleviate the spatial misalignment problem of dual-modal data; EFETN by Zhao et al. [11] and CAGT by Yuan et al. [14] innovatively introduced Transformer into the fusion stage, utilizing its long-range dependency modeling capability to strengthen edge feature extraction and cross-modal consistency; EEF by Hao et al. [12] achieved adaptive feature enhancement by calculating feature energy scores to dynamically allocate weights.

Although the above methods have made some progress on specific benchmarks, when facing practical remote sensing edge deployment tasks, they still face severe challenges: First, the problems of computational complexity and parameter redundancy are noticeable. In pursuit of extreme accuracy, networks such as FQDNet often rely on complex multi-branch topologies or dense attention computations, leading to inference delays that are difficult to meet real-time requirements. Second, feature alignment and scale adaptability are insufficient. Existing fusion modules are often difficult to balance global context and local texture details, resulting in increased localization deviation in dense small object scenes, and it is difficult to adaptively process severe scale changes in remote sensing images. This is exactly the core driving force for this paper to propose the ISDG-Net framework: aiming to construct an efficient fusion paradigm that combines lightweight design, dynamic sparse perception capability, and multi-scale adaptive alignment.

3. Method

3.1. Overall Architecture

ISDG-Net is an end-to-end RGB–infrared detector built on a dual-stream pipeline with three tightly connected parts (shown in Figure 1), including a dual-modal feature extraction backbone (IVE Backbone), a global-local context aggregation neck (Hyper ACE & Full PAD Neck), and an adaptive spatial fusion detection head (Detect-SASD Head). At the input stage, strictly aligned visible and infrared images are fed into two parallel backbone branches to extract modality-specific representations. In the shallow layers, BiFormer is introduced to extend the effective receptive field and capture early global dependencies that standard convolutions may miss. In deeper layers, the backbone integrates the proposed channel-separation inverted bottleneck (IBC-Conv), which combines depthwise separable computation with an inverted residual pathway to reduce redundancy while preserving high-level semantics [40]. The backbone finally produces three multi-scale feature maps ($P_1,P_2, P_3$) with progressively lower spatial resolution, which are then passed to the neck for subsequent context aggregation and fusion.

3.2. IBC-Conv: Mid-Level Dual-Modal Feature Extraction and Fusion Backbone

In RGB–infrared remote sensing detection, the backbone is expected to produce discriminative high-level semantics while keeping the computational budget low for edge deployment. To meet this requirement, we design an inverted bottleneck module with channel separation, referred to as IBC-Conv (see Figure 2). The module follows the “expand-transform-project” pipeline and combines an inverted residual pathway with depthwise separable computation, which reduces redundancy while preserving modality-specific representations.

Traditional Residual Blocks usually adopt an hourglass structure of “dimensionality reduction—convolution—dimensionality increase”, which easily causes the compression loss of feature information in low-dimensional space. On the contrary, IBC-Conv adopts an inverted structure, first utilizing a $1 \times 1$ convolution to expand the number of channels of the input feature map $X_{in}\in {\mathbb{R}}^{\mathbb{H}\times \mathbb{W}\times \mathbb{C}}$ to high-dimensional space (usually 4 times the input channels) to generate expanded features $X_{exp}$. This high-dimensional mapping enables subsequent convolution operations to decouple spatial and channel information in a richer feature space, effectively avoiding information loss caused by nonlinear activation.

To solve the problem of excessive computational load of standard convolution, IBC-Conv introduces depthwise separable convolution technology. On the expanded high-dimensional features, a $3 \times 3$ Depthwise Convolution is applied. Through the “channel separation” strategy, this operation makes each convolution kernel only responsible for the spatial filtering of one channel, thereby cutting off the interaction between channels and reducing the computational complexity to $1/\mathrm{C}$ of standard convolution. Subsequently, the module embeds an SE (Squeeze-and-Excitation) [41] attention unit to adaptively calibrate feature responses by explicitly modeling the dependencies between channels, enhancing the model’s sensitivity to key features in remote sensing images.

Finally, a $1 \times 1$ Pointwise Convolution is used to project the feature map back to low-dimensional space, completing the cross-dimensional interaction and fusion of channel information. To ensure smooth propagation of gradients, when the input and output dimensions are consistent, a residual connection is introduced to add the input directly to the output end. The calculation process of the entire IBC-Conv can be formalized as the formula:

$$Y={\mathcal{P}}_{1\times 1}\left(SE\left(\mathcal{D}{\mathcal{W}}_{3\times 3}\left({\mathcal{E}}_{1\times 1}\left(X_{in}\right)\right)\right)\right)+X_{in}$$

(1)

Among them, ${\mathcal{E}}_{1\times 1}$ represents the dimensionality-increasing convolution, $\mathcal{D}{\mathcal{W}}_{3\times 3}$ represents the depthwise convolution, and ${\mathcal{P}}_{1\times 1}$ represents the dimensionality-reducing projection convolution. Unlike the standard inverted bottlenecks in MobileNetV2 [40] that target single-modal tasks, IBC-Conv is specifically designed to handle the heterogeneity of visible and infrared data. Standard dense convolutions often cause early mixing of different modalities, leading to feature interference. To avoid this, IBC-Conv applies depthwise convolution in a high-dimensional space to perform spatial filtering independently for each channel. This separation prevents the premature blending of optical textures and thermal contours. Subsequently, the SE attention module recalibrates these separated channels, preserving essential modality-specific cues before the final 1 × 1 projection fuses them. Through this design, IBC-Conv successfully achieved efficient feature extraction and transmission in a lightweight architecture.

3.3. Detect SASD (Learned Adaptive Spatial Detection)

In the visible-infrared dual-modal object detection task, the collaborative optimization of modal heterogeneity and object scale diversity has always been the core challenge to improving detection performance. Visible light images possess rich texture details and scene semantic information, while infrared images can accurately capture the thermal radiation contours of objects in complex environments such as low light and occlusion; the complementary characteristics of the two provide a foundation for complex scene detection. However, traditional detection heads usually adopt fixed-mode feature concatenation or weighted fusion strategies [42], which are not only difficult to dynamically balance the contribution of dual-modal features at different spatial locations with but also are unable to effectively solve the problem of representation misalignment between multi-scale feature levels. At the same time, the differences in channel dimensions and semantic distributions of dual-modal features easily lead to the dilution of useful information during the fusion process; especially in small object and low signal-to-noise ratio scenarios, detection accuracy and robustness are significantly constrained.

To address the above problems, we designed a detection head (Detect-SASD) integrating the Learned Adaptive Spatial Detection (SASD) mechanism (as shown in Figure 3). Targeting the three-layer feature maps output by the dual-modal feature pyramid, this detection head constructs an end-to-end processing flow containing three major modules: feature scale unification, spatial adaptive fusion, and detection task decoupling.

First, feature scale unification (Scale Unification) is performed: At the output end of the feature pyramid, the network provides three dual-modal feature maps with different spatial resolutions and semantic levels, denoted as $F_0,F_1,F_2$, and their corresponding numbers of channels are $c{h}_0, c{h}_1, c{h}_2$, respectively. To enable features of different levels to interact in the same spatial dimension, the SASD unit first unifies their scales and channels through a differentiated alignment function $\mathcal{T}\left(\cdot \right)$:

For high-level low-resolution features (Level 0): After using a $1\times 1$ convolution to compress the channel dimension, an upsampling operation (Scale factor is 2 or 4) is used to match the target spatial scale.

$$F_0^{\prime }=UpSample\left(Con{v}_{1\times 1}\left(F_0\right)\right),$$

(2)

For mid-level features (Level 1): Combined with a $1 \times 1$ convolution to achieve channel compression, the adjusted number of feature channels can be flexibly controlled by the multiplier parameter (such as taking 0.5 or 1) to balance representation capability and computational overhead.

$$F_1^{\prime }=Con{v}_{1\times 1}\left(F_1\right),$$

(3)

For low-level high-resolution features (Level 2): Through a $3\times 3$ convolution with a stride of 2 or max pooling downsampling, channel compression and resolution alignment are completed synchronously.

$$F_2^{\prime }=DownSample\left(Con{v}_{3\times 3}\left(F_2\right)\right),$$

(4)

Through the above operations, all features are mapped to a unified spatial dimension $F_i^{\prime }\in {\mathbb{R}}^{\mathbb{H}\times \mathbb{W}\times \mathbb{C}}$.

Then, Spatial Adaptive Fusion is performed: In order to dynamically aggregate dual-modal complementary information at the pixel level, Detect-SASD introduces learnable three-dimensional spatial weight maps. First, the aligned three-layer features are respectively mapped to an extremely low-dimensional space (compressed number of channels is 8 or 16) through a $1 \times 1$ convolution, and then they are concatenated in the channel dimension, and a normalized weight tensor $W\in {\mathbb{R}}^{3\times \mathbb{H}\times \mathbb{W}}$ is output through convolution and the Softmax function. This process can be represented as:

$$W=Softmax\left(Conv\left(\left[Con{v}_{sq}\left(F_0^{\prime }\right),Con{v}_{sq}\left(F_1^{\prime }\right),Con{v}_{sq}\left(F_2^{\prime }\right)\right]\right)\right),$$

(5)

The weight tensor $W$ is sliced in the channel dimension into three independent spatial weight matrices $\mathsf{\alpha }, \mathsf{\beta }, \mathsf{\gamma }$, which respectively represent the feature importance of $F_0^{\prime },F_1^{\prime },F_2^{\prime }$ at various spatial locations. This mechanism strictly guarantees that the constraint condition is satisfied at any pixel coordinates $\left(i,j\right)$:

$${\mathsf{\alpha }}_{\left(i,j\right)}+{\mathsf{\beta }}_{\left(i,j\right)}+{\mathsf{\gamma }}_{\left(i,j\right)}=1,\hspace{1em}\mathsf{\alpha },\mathsf{\beta },\mathsf{\gamma }\in \left[0,1\right],$$

(6)

Finally, the fused feature $F_{fused}$ is generated through pixel-level weighted summation ($\odot$ represents the Hadamard product), enabling the model to adaptively strengthen complementary features in target regions and suppress redundant noise in background regions:

$$F_{fused}=\mathsf{\alpha }\odot F_0^{\prime }+\mathsf{\beta }\odot F_1^{\prime }+\mathsf{\gamma }\odot F_2^{\prime }$$

(7)

Ultimately, Detection Task Decoupling is performed: The fused high-quality feature $F_{fused}$ is sent into two parallel lightweight convolutional branches (cv2 and cv3) to complete the final detection task decoupling. The Regression Branch cv2 branch outputs a 16-dimensional feature vector through the Distribution Focal Loss (DFL) [43] layer, achieving fine regression of the bounding box coordinates $\left(x,y,w,h\right)$. The Classification Branch cv3 outputs the predicted probability distribution of each target category.

The overall architecture possesses core advantages such as strong modal adaptability, lightweight structure, and outstanding generalization capability, achieving end-to-end precise dual-modal detection without significantly increasing computational complexity.

3.4. Dynamic Sparse Transformer (DySparse)

In modern object detection architectures, Vision Transformer (ViT) shows significant advantages in capturing long-range dependencies through the global self-attention mechanism. However, the computational complexity of standard Multi-Head Self-Attention (MHSA) has a quadratic relationship $\mathcal{O}\left({\left(HW\right)}^2\right)$ with the number of input tokens [16,20]. For high-resolution remote sensing images, the massive number of tokens leads to unacceptable memory overhead and inference latency. Some existing sparse attention mechanisms attempt to limit the receptive field through fixed windows or dilation rates, but such static hand-crafted designs lack adaptability to the content of the input image.

To alleviate the contradiction between global context modeling and computational efficiency, this paper constructs a dynamic sparse Transformer (DySparse) module. This module integrates the existing Bi-Level Routing Attention (BRA) mechanism and adapts it for remote sensing imagery [44]. Drawing inspiration from the progressive screening philosophy of the recent “Glance-Focus-Gaze” paradigm [45], our adaptation specifically leverages its core idea of sparse attention allocation to handle complex scenes. By constructing a “Query-aware” dynamic sparse connection graph, it effectively filters out irrelevant background noise while only computing attention for regions with high semantic relevance.

The forward computation process of the DySparse module mainly consists of four cascaded steps: implicit positional encoding, coarse-grained region routing, fine-grained feature aggregation, and local context enhancement:

First is implicit positional encoding and feature mapping. Given the input feature map $X\in {\mathbb{R}}^{\mathbb{H}\times \mathbb{W}\times \mathbb{C}}$, to avoid the resolution adaptability issues caused by introducing explicit absolute positional encoding, the module first utilizes a $3 \times 3$ Depthwise Convolution to perform implicit spatial positional encoding:

$$X_{pe}=X+DWCon{v}_{3\times 3}\left(X\right),$$

(8)

Through a linear projection matrix, $X_{pe}$ is mapped into query (Q), key (K), and value (V) tensors, with dimensions all maintained at ${\mathbb{R}}^{\mathbb{H}\times \mathbb{W}\times \mathbb{C}}$.

Next is Coarse Region Partition & Routing. To achieve dynamic sparsity, the feature map is uniformly divided along the spatial dimension into $S \times S$ non-overlapping coarse-grained regions, with each region containing $R={\displaystyle \frac{HW}{S^2}}$ tokens. By performing an average pooling operation on the tokens within each region, the region-level representations $Q^r,K^r\in {\mathbb{R}}^{{\mathbb{S}}^2\times \mathbb{C}}$ are obtained. Then, the region-level semantic adjacency matrix $A^r\in {\mathbb{R}}^{{\mathbb{S}}^2\times {\mathbb{S}}^2}$ is calculated to quantify the correlation strength between different regions:

$$A^r=Q^r{\left(K^r\right)}^T,$$

(9)

For the $i$-th query region, a greedy routing strategy is used to retain the Top-$k$ regions with the highest correlation, generating the dynamic routing index matrix $I^r\in {\mathbb{N}}^{{\mathbb{S}}^2\times \mathbb{k}}$:

$$I_i^r=TopK\left(A_i^r,k\right),$$

(10)

This process constructs a dynamic sparse graph at the macroscopic scale, actively discarding irrelevant regions containing massive redundant ground objects or background noise.

Next is fine-grained attention and local context enhancement (Fine-grained Attention & LCE). After entering the fine-grained computation stage, the algorithm utilizes the Gather operation to dynamically collect the corresponding key and value tensors across regions according to the routing index $I_i^r,$ forming highly compact $K_i^g,V_i^g\in {\mathbb{R}}^{\mathbb{k}\mathbb{R}\times \mathbb{C}}$. The sparse attention computation for the $i$-th region is formalized as:

$$O_i=Softmax\left({\displaystyle \frac{Q_i{\left(K_i^g\right)}^T}{\sqrt{d}}}\right)V_i^g,$$

(11)

where $d$ is the channel dimension of a single attention head.

In addition, since sparse sampling inevitably breaks the physical continuity of local features, DySparse introduces a depthwise convolution-based Local Context Enhancement (LCE) branch that acts directly on the original value tensor $V$. The final feature output $O_{fused}$ is the combination of long-range sparse attention and local context features:

$$O_{fused}=Concat\left(O_1,O_2,\dots ,O_{S^2}\right)+DWConv\left(V\right),$$

(12)

As shown in Figure 4, the fused feature $O_{fused}$ finally passes through a two-layer Multi-Layer Perceptron (MLP) to complete the nonlinear feature transformation in the channel dimension.

Overall, DySparse adopts a four-stage pyramid architecture design. Benefiting from the bi-level routing mechanism, DySparse successfully reduces the computational complexity of the self-attention mechanism from $\mathcal{O}\left({\left(HW\right)}^2\right)$ to $\mathcal{O}\left({\left(HW\right)}^{1.5}\right)$. This content-adaptive sparse modeling approach enables the network to highly focus limited computing power on salient tiny object regions in remote sensing images, significantly improving the model’s object detection accuracy in complex, large-scale scenes.

3.5. GreedyIoU Selector (GIS)

In the post-processing stage of object detection, Non-Maximum Suppression (NMS) is a crucial step for eliminating redundant bounding boxes and retaining the optimal detection results. Traditional NMS algorithms adopt a simple suppression strategy, that is, once the Intersection over Union (IoU) between a candidate box and the box with the highest score exceeds a set threshold, the candidate box will be directly discarded. However, in dense remote sensing object detection tasks, this greedy hard-filtering strategy has significant flaws: when multiple targets of the same class occlude each other or are densely arranged, correct detection boxes are often mistakenly suppressed due to high overlap, leading to a decrease in recall and missed detection issues. To address this issue, this paper proposes an improved Greedy IoU Selector (GIS, as shown in Figure 5). (Please note: throughout this manuscript, the acronym GIS refers exclusively to our proposed algorithmic bounding box selection module, distinguishing it from the traditional geospatial concept of Geographic Information Systems).

The core idea of GIS is to negate the simple deletion operation of traditional NMS, and instead adopt a smooth score decay strategy based on geometric perception. For a candidate box $b_i$ whose overlap with the highest-scoring box $M$ is greater than the threshold, GIS does not directly remove it, but instead lowers its confidence score according to its geometric similarity to $M$. In order to more accurately measure the geometric relationship between targets, GIS abandons the traditional IoU, which only considers the overlap area, and introduces the CIoU metric, which includes overlap area, Euclidean distance between center points, and aspect ratio consistency. This improvement enables the algorithm to more acutely perceive the true spatial distribution of targets when processing arbitrarily oriented targets (such as dense vehicles and long bridge structures) from an overhead remote sensing perspective. GIS uses a Gaussian function as the re-scoring kernel, inspired by Soft-NMS [46], and dynamically adjusts the candidate box scores through the following formula:

$$s_i=s_i\exp \left(-{\displaystyle \frac{CIoU{\left(M,b_i\right)}^2}{\mathsf{\sigma }}}\right),\forall b_i\notin \mathcal{D},$$

(13)

where $\mathcal{D}$ is the set of selected final detection boxes, and $\mathsf{\sigma }$ is a hyperparameter controlling the decay strength. The calculation of CIoU, following its original definition [47], is as the formula, covering the IoU term, the center point distance penalty term, and the aspect ratio impact factor:

$$CIoU=IoU-\left({\displaystyle \frac{{\rho }^2\left(M,b_i\right)}{c^2}}+\alpha v\right),$$

(14)

Through this mechanism, when the candidate box and the highest-scoring box not only have a large overlap area, but also a close center point distance and highly similar shapes, GIS will determine that it has an extremely high probability of being a redundant box, thereby imposing a stronger score decay; conversely, if the two only have partial edge overlap (common in densely arranged vehicles), the CIoU value is relatively small, and the decay strength is lighter, thereby largely retaining the correct object boxes. Without the need for additional model training, GIS effectively balances detection accuracy and recall rate, significantly reducing false detections and missed detections in complex dense scenes, and the computational complexity remains at $\mathcal{O}\left(N^2\right)$, ensuring the overall inference efficiency of the algorithm.

4. Experiments

4.1. Datasets and Evaluation Metrics

4.1.1. Dual-Modal Image Object Detection Dataset

In order to rigorously validate the performance and transferability of our approach, we perform extensive experiments across three widely recognized benchmarks: VEDAI, M3FD, and LLVIP. The above datasets all contain precisely paired visible and infrared imagery data and corresponding standardized annotation information, providing solid data support for the comprehensive evaluation of model performance in multi-modal scenes.

(1)

VEDAI

The VEDAI (Vehicle Detection in Aerial Imagery) collection serves as a standard reference for identifying vehicles from overhead perspectives [48]. It was developed to support research on UAV and satellite image analysis by providing training and testing samples with diverse and challenging backgrounds. The dataset contains 1268 high-resolution aerial captures featuring ground sample distances between 0.5 m and 2 m, covering rural roads, urban areas, and suburban open spaces. It includes 10,216 annotated vehicle instances of different categories, such as cars, trucks, and buses. In addition to object locations, the annotations provide detailed information, including orientation and occlusion levels. Owing to its large-scale variation, background complexity, and dense target distribution, VEDAI has been widely used to evaluate detection methods for aerial imagery, especially in vehicle detection and small-object recognition tasks.

(2)

M3FD

The M3FD (Multi-scale Multi-scene Multi-sensor Small Object Detection) dataset is designed for multi-scale locating miniature targets amidst challenging backgrounds [49]. It overcomes the constraints of conventional datasets in terms of small-object representation and scene diversity. The dataset integrates imagery collected from multiple sensors, including satellites, UAVs, and ground surveillance systems, and contains 4200 images across 12 representative scenarios such as urban areas, mountains, oceans, and airports. Small objects with pixel areas below 32 × 32 make up a substantial fraction of the corpus, while medium- and large-scale objects are also included to support multi-scale evaluation. The annotations are produced under a strict double-checking procedure and include object categories, bounding boxes, confidence information, and sensor-related parameters. In addition, local feature descriptions are provided for objects that are difficult to distinguish or localize. Because of its multi-sensor and multi-scene characteristics, M3FD is well suited for evaluating the resilience and domain-transfer proficiencies of detection algorithms.

(3)

LLVIP

The LLVIP (Low-Light Vehicle and Pedestrian) corpus stands as a prominent testbed for target recognition under low-light environments [50]. It was introduced to address the difficulty of detecting vehicles and pedestrians under challenging illumination conditions, such as nighttime, dusk, and tunnel scenes. The dataset contains 15,488 low-light images, encompassing diverse transportation scenes such as city streets, expressways, and neighborhoods. All images have a resolution of 1920 × 1080 and cover a variety of transportation scenes, encompassing city streets, expressways, and neighborhoods. In addition, the archive incorporates dark-scene data acquired during harsh meteorology like precipitation, haze, and dust, which further increases scene complexity. A total of 35,620 objects are annotated, including multiple vehicle categories and pedestrian targets. The annotations provide bounding boxes as well as auxiliary information such as illumination level and blur degree. These characteristics make LLVIP a valuable benchmark for assessing recognition capabilities in poorly illuminated settings and for exploring the combined enhancement of image enhancement and object detection.

To ensure fair evaluation and strict reproducibility, we adhered to standard partition protocols using the pre-defined subdirectories for training and testing splits. Specifically, the VEDAI dataset was partitioned into 935 training images and 311 testing images, corresponding to a 75%:25% ratio. For the M3FD dataset, which contains diverse scenarios and sensor types, we employed a standard 80%:20% split across the entire dataset. This results in 3360 training pairs and 840 testing pairs, ensuring that the varied sensor characteristics are representatively distributed in both sets. Lastly, the LLVIP dataset utilizes its official split, consisting of 12,025 image pairs for training and 3463 pairs for testing (a ratio of approximately 77.6% to 22.4%).

4.1.2. Evaluation Metrics

For a rigorous numerical assessment regarding both the recognition efficacy and architectural efficiency of ISDG-Net in visually degraded and intricate aerial scenes, we employ a holistic testing protocol. Specifically, following standard object detection evaluation metrics [51], the adopted indicators encompass Precision, Recall, mean Average Precision (mAP), total parameter count (Params), and floating-point operations (FLOPs).

The classification and localization performance of an object detector is commonly assessed using Precision $P$ and Recall $R$, which are defined based on the confusion matrix as follows:

$$P={\displaystyle \frac{TP}{TP+FP}},$$

(15)

$$R={\displaystyle \frac{TP}{TP+FN}},$$

(16)

Here, $TP$ (True Positives) indicates the count of accurately localized instances where the Intersection over Union (IoU) against the manual annotation surpasses a specific threshold; $FP$ (False Positives) represents erroneous predictions—either nonexistent objects or bounding boxes failing to meet the IoU criterion; meanwhile, $FN$ (False Negatives) stands for the actual entities that the algorithm failed to capture.

To jointly evaluate Precision and Recall, Average Precision ($AP$) corresponds to the integrated area beneath the Precision–Recall ($PR$) trajectory. Mathematically, the continuous $AP$ for an individual category can be written as:

$$AP={\int }_0^{1}P\left(R\right)dR,$$

(17)

In practical offline evaluation, the $PR$ curve is usually discretized and smoothed to reduce fluctuations. Using all-point interpolation as an example, $AP$ can be expressed as:

$$AP={\sum }_{i=1}^{n-1}\left(R_{i+1}-R_i\right)\underset{\tilde{R}\ge R_{i+1}}{\max }P\left(\tilde{R}\right),$$

(18)

Given an aerial dataset comprising N distinct classes, the mean Average Precision ($\mathrm{m}AP$) calculates the unweighted average of the $AP$ scores across all types, serving as a comprehensive indicator of the model’s overall multi-category recognition proficiency:

$$mAP={\displaystyle \frac{1}{N}}{\sum }_{k=1}^{N}A{P}_k,$$

(19)

In the experiments of this paper, we focus on two $\mathrm{m}AP$ metrics with different degrees of stringency:

mAP@0.5: This metric constrains the strict lower bound of the decision threshold for positive target samples to $\mathsf{\tau }= 0.5$. For a detection task containing $N$ target categories, given a category $k$, its average precision (AP) under the threshold $\mathsf{\tau }$ can be expressed as the integral of precision $P$ with respect to the change in recall $R$. The global analytical expression of $mA{P}_{50}$ is as follows:

$$mA{P}_{50}={\displaystyle \frac{1}{N}}{\sum }_{k=1}^N{\int }_0^1{P}_k\left(R,\mathsf{\tau }=0.5\right)dR,$$

(20)

where $P_k\left(R,\mathsf{\tau }\right)$ denotes the precision of category $k$ under IoU threshold $\mathsf{\tau }$. This metric mainly reflects the model’s basic classification performance and coarse localization ability.

mAP@[0.5:0.95]: Compared with single-threshold evaluation, this metric introduces a more stringent dynamic threshold penalty mechanism. It systematically and comprehensively evaluates different levels of localization accuracy by performing discrete summation and averaging of $mA{P}_{\mathsf{\tau }}$ over a preset discrete threshold set $\mathcal{T}=\{0.50,0.55,0.60,\dots ,0.95\}$. Its complete formula definition is:

$$mA{P}_{50:95}={\displaystyle \frac{1}{\left|\mathcal{T}\right|}}{\sum }_{\mathsf{\tau }\in \mathcal{T}}\left({\displaystyle \frac{1}{N}}{\sum }_{k=1}^N{\int }_0^1{P}_k\left(R,\mathsf{\tau }\right)dR\right)={\displaystyle \frac{1}{10N}}{\sum }_{i=0}^9{\sum }_{k=1}^{N}A{P}_k^{\mathsf{\tau }=0.5+0.05i},$$

(21)

This metric uses an IoU step size of 0.05 and covers ten evaluation thresholds. Since higher IoU thresholds impose stricter localization requirements, even small coordinate deviations may cause a predicted box to be counted as a false positive. Therefore, $mA{P}_{50:95}$ provides a more rigorous measure of localization accuracy and is particularly suitable for evaluating model robustness in dense scenes and small-object detection tasks.

Considering the stringent computational and storage constraints of remote sensing edge devices, such as UAV platforms, this paper further introduces Params and FLOPs to quantify model complexity.

For a standard two-dimensional convolutional layer, assume that the number of input channels is $C_{in}$, the number of output channels is $C_{out}$, the kernel size is $K \times K$, and the spatial resolution of the output feature map is $H \times W$. When a bias term is included, the number of parameters is calculated as:

$$Params=\left(C_{in}\times K^2+1\right)\times C_{out},$$

(22)

The total number of parameters of the entire network is the linear accumulation of the number of parameters of each layer. Correspondingly, the floating-point operations (FLOPs) used to measure the computational cost mainly count the Multiply–Accumulate Operations (MACs) in the forward propagation process, and the theoretical calculation formula for the FLOPs of a single-layer standard convolution is:

$$FLOPs=2\times H\times W\times \left(C_{in}\times K^2+1\right)\times C_{out},$$

(23)

The coefficient 2 represents one multiplication and one addition operation. Lower Params means the model occupies smaller memory space, while lower FLOPs directly determines that the model can achieve a higher inference frame rate (FPS) on mobile hardware.

4.2. Implementation Details

The configurations used for training and testing are shown in

Table 1. Training and testing environment configuration.

Environment	Parameters
CPU	Intel Core Ultra 9-285K
GPU	NVIDIA GeForce RTX 5090
Memory	64 GB
Language	Python 3.12
Framework	PyTorch 2.8.0
CUDA Version	12.8

and

Table 2. Training parameter settings.

Epochs	Batch Size	Workers	Optimizer	Amp	Initial Learning Rate	LR Schedule	Input Resolution
200	24	20	SGD	False	0.01	Linear Decay	512 × 512

.

Other parameters are explicitly detailed as follows: all models are trained for 200 epochs with a batch size of 24. We utilize the Stochastic Gradient Descent (SGD) optimizer with Nesterov momentum set to 0.937 and a weight decay of 0.0005. The initial learning rate is set to 0.01. To ensure stable convergence, we apply a linear decay learning rate schedule, reaching a final learning rate of 0.0001, which is preceded by a 3-epoch warmup phase. Regarding the input resolution, the original image dimensions vary across the datasets (1280 × 1024 for LLVIP, 1024 × 768 for M3FD, and 512 × 512 for VEDAI). To prevent object distortion during scaling, we employ a standard letterbox resizing strategy. This scales the longest edge of all images to 512 pixels while maintaining the original aspect ratio, padding the shorter edge with gray pixels. Consequently, the final unified input resolution fed into the network is strictly 512 × 512 across all datasets, ensuring standardized training and fair evaluation of computational metrics (e.g., FLOPs and FPS). The number of worker processes is set to 20. These hyperparameter settings are consistently applied across the VEDAI, M3FD, and LLVIP datasets, and are summarized in Table 2.

To rigorously ensure the statistical significance and robustness of our evaluation, all reported experimental metrics (including the ablation studies and main results) are the average values obtained from three independent runs under the same configuration.

4.3. Ablation Experiments

In order to ascertain the individual contributions and synergistic dynamics among the newly introduced IBC-Conv, SASD (detection head), and DySparse (attention module) on the VEDAI dataset, we set up 8 groups of experiments using the original baseline model (Baseline) as a control. Specifically, this baseline is the YOLOv13-mid-fusion architecture. Compared with the standard single-modal YOLOv13, this dual-stream baseline simply concatenates (cascades) the relevant backbone feature layers from the visible and infrared modalities along the channel dimension, without introducing any advanced fusion enhancement modules. As shown in

Table 3. Component analysis involving IBC-Conv, SASD, and DySparse evaluated on VEDAI. Increments or decrements compared with the reference model are denoted within brackets, with red representing increased values and green representing decreased values. × indicates the absence of this item, √ indicates the presence of this item, ↓ represents a decrease in the indicator, and ↑ indicates an increase in the indicator.

Method	IBC-Conv	SASD	DySparse	Param (M)	FLOPs (G)	mAP@0.5 (%)	mAP@[0.5:0.95] (%)
Baseline	×	×	×	3.6	9.9	50.8	31.1
1	√	×	×	2.4 (↓1.2)	8.5 (↓1.4)	52.3 (↑1.5)	31.3 (↑0.2)
2	×	√	×	5.2 (↑1.6)	13.1 (↑3.2)	50.0 (↓0.8)	31.0 (↑0.1)
3	×	×	√	3.6	9.2 (↓0.7)	48.9 (↓1.9)	29.5 (↓1.6)
4	×	√	√	5.3 (↑1.7)	12.6 (↑2.7)	50.4 (↓0.4)	29.7 (↓1.4)
5	√	×	√	2.5 (↓1.1)	7.8 (↓2.1)	48.0 (↓2.8)	29.2 (↓1.9)
6	√	√	×	4.1 (↑0.5)	11.7 (↑1.8)	51.9 (↑1.1)	31.8 (↑0.7)
7	√	√	√	4.2 (↑0.6)	11.3 (↑1.4)	55.1 (↑4.3)	33.8 (↑2.7)

, this unenhanced baseline achieves a performance of 50.8% mAP@0.5. This represents the raw performance of the model without any of our proposed enhancement modules (IBC-Conv, DySparse, SASD, or GIS), providing a fundamental benchmark for evaluating the effectiveness of our subsequent components. Detailed settings and metrics are shown in Table 3.

The experimental results first verified the contribution of IBC-Conv to the lightweight design of the model. To quantitatively demonstrate the performance gains of the ‘channel separation + SE attention’ combination, we compared Method 1 with the standard convolution Baseline. As shown in Method 1, simply replacing the original backbone with IBC-Conv reduced the number of parameters from 3.6 M to 2.4 M (a 33% reduction) and decreased the computational cost from 9.9 G to 8.5 G FLOPs. Notably, this reduction in model complexity did not impair detection performance; instead, the mAP@0.5 improved from 50.8% to 52.3%. This result suggests that the channel separation strategy and inverted residual structure of IBC-Conv can effectively preserve critical semantic information whilst filtering out extraneous noise, yielding a highly streamlined descriptor for downstream target localization.

In contrast, introducing SASD alone (method 2) or DySparse alone (method 3) didn’t improve performance. Instead, the mAP@0.5 dropped to 50.0% and 48.9%, respectively. A possible reason is that both modules mainly operate on late-stage features and emphasize global dependency modeling or feature refinement. When the baseline backbone fails to provide sufficiently clean and discriminative representations, SASD may struggle to generate reliable alignment weights, while DySparse may be unable to establish effective sparse routing. Moreover, the parameter count of Method 2 increased to 5.2 M, yet its detection accuracy declined. This suggests that, without the support of an efficient backbone, merely increasing the complexity of the detection head may introduce parameter redundancy and even lead to overfitting.

It is worth noting that combining only IBC-Conv and SASD (Method 6) results in a slight mAP decrease to 51.9% compared with using IBC-Conv alone (52.3%). This occurs because directly feeding highly condensed local features from IBC-Conv into the spatially sensitive SASD head creates a representational mismatch, making the alignment weights susceptible to local fusion noise. However, the introduction of the third component, DySparse (Method 7), perfectly resolves this specific mismatch. By acting as an intermediate global bridge, DySparse effectively eliminates the fusion noise by supplying the essential long-range dependencies that the SASD head requires. The compact and informative features produced by IBC-Conv improved DySparse’s ability to capture long-range dependencies and simultaneously provided more reliable alignment cues for the multi-scale fusion process in SASD. Experimental results show that the full ISDG-Net achieved 55.1% mAP@0.5, representing a 4.3-point improvement over the baseline, while the stricter metric mAP@[0.5:0.95] also reached 33.8%. Although the final model has slightly higher complexity than the baseline, with 4.2 M parameters and 11.3 G FLOPs, the substantial gain in detection accuracy indicates that this trade-off is justified.

4.4. Comparative Experiments

To benchmark the operational viability and superiority of ISDG-Net, extensive comparative trials were executed across two public datasets, VEDAI and M3FD, against several representative dual-modal mid-fusion baselines from the YOLO family. The compared methods cover recent YOLO variants from YOLOv10 to YOLOv13. In addition to detection accuracy under different evaluation criteria, such as mAP@0.5 and mAP@[0.5:0.95], we simultaneously evaluated structural bulk via Params and FLOPs to determine our algorithm’s readiness for implementation on edge devices.

The VEDAI dataset is characterized by multi-scale aerial imagery and complex geographical scenes. As shown in the comparison results (

Table 4. Performance and structural footprint assessment across various established network architectures evaluated on VEDAI.

Method	Param (M)	FLOPs (G)	Precision (%)	Recall (%)	mAP@0.5 (%)	mAP@[0.5:0.95] (%)
YOLOv10-mid-fusion [24]	3.7	11.3	56.1	52.0	52.5	32.3
YOLOv11-mid-fusion [25]	3.7	9.6	57.7	52.7	51.0	31.2
YOLOv12-mid-fusion [26]	4.0	9.1	57.4	50.6	50.6	30.6
YOLOv13-mid-fusion [27]	3.5	9.9	63.2	50.5	50.8	31.1
iRMB-fusion [52]	4.1	11.8	52.9	52.1	49.3	28.7
MSGA-fusion [53]	4.7	14.4	57.8	53.5	50.8	30.9
ISDG-Net (Ours)	4.2	11.3	63.7	52.9	55.1	33.8

), ISDG-Net achieved the best overall detection performance among all competing methods. Specifically, our model obtained 55.1% mAP@0.5 and 33.8% mAP@[0.5:0.95], outperforming the previously strongest baseline, YOLOv10-mid-fusion, which achieved 54.5% and 32.3%, respectively. Although the parameter count of ISDG-Net (4.2 M) is slightly higher than that of the highly lightweight YOLOv13-mid-fusion (3.5 M), its computational cost remains well controlled at 11.3 G FLOPs, which is comparable to YOLOv10-mid-fusion. This is mainly attributed to the efficient feature compression enabled by the proposed IBC-Conv module. These results indicate that ISDG-Net can capture complex aerial targets more accurately while introducing only marginal additional inference cost.

To further verify the model performance quantitatively, Figure 6 shows the convergence curves of the core metrics, including Recall, mAP@0.5, mAP@[0.5:0.95], and Precision for ISDG-Net and other baseline models during 200 training epochs.

As shown by the training trends, the advantages of the model proposed in this paper are mainly reflected in the following three dimensions: Higher performance upper limit and sustained learning ability. From the mAP@0.5 and mAP@[0.5:0.95] curves, it can be seen that most of the compared models fall into a performance bottleneck (plateau) around 100–120 epochs. In contrast, ISDG-Net (red line) maintains a strong growth momentum throughout the training period. This indicates that the compact high-dimensional feature space provided by the IBC-Conv module effectively mitigates overfitting, enabling the model to ultimately reach the highest levels on mAP@0.5 and mAP@[0.5:0.95], respectively (55.1% and 33.8%).

More balanced optimization of Precision and Recall. In object detection, Precision and Recall often exhibit a trade-off. However, the leftmost and rightmost sub-figures show that ISDG-Net consistently outperforms the baseline models on both metrics. This trend further supports the effectiveness of the proposed architecture. In particular, the sparse routing mechanism of DySparse helps suppress redundant or false responses, thereby improving Precision, while the dynamic modal interaction in Detect-SASD enhances target coverage and reduces missed detections, contributing to higher Recall.

Stronger robustness under stricter localization criteria. Among the four sub-figures, the advantage of ISDG-Net is most pronounced on the more demanding mAP@[0.5:0.95] metric, where the red curve maintains the clearest margin over the competing methods. This result indicates that the proposed geometry-aware selection mechanism is particularly effective in improving localization quality. Beyond increasing the number of correct detections, it also helps the model generate bounding boxes with higher spatial alignment to the ground-truth targets, thereby improving performance under stricter IoU threshold.

To intuitively evaluate the robustness of the proposed ISDG-Net in complex real-world remote sensing environments, Figure 7 (visible modality) and Figure 8 (infrared modality) present qualitative comparisons between our method and several representative baselines, including MSGA, iRMB Fusion, and the mid-fusion variants of the YOLO series. The red annotations highlight typical failure cases of the baseline models, including missed detections (MISS), false detections (Wrong object), and over-detection (Over-detection), under challenging conditions such as low contrast, background interference, and dense target distributions. The qualitative results show that the proposed model provides more robust detection performance across diverse scenarios.

In the first, second, and fifth groups of examples, small targets are difficult to distinguish under low-light shadow occlusion, extremely low signal-to-noise ratios, or weak infrared thermal responses, which leads to frequent missed detections in baseline methods such as MSGA and iRMB Fusion. In contrast, ISDG-Net is able to preserve more complete target responses in these challenging cases. This result suggests that the proposed Detect-SASD head improves cross-modal feature interaction by assigning adaptive fusion weights to heterogeneous features. When the information quality of one modality degrades, the model can make better use of complementary cues from the other modality, thereby improving feature alignment and enhancing recall in difficult scenes.

In the third and sixth groups of examples, background textures such as building edges and ground patches are easily confused with vehicle targets, resulting in obvious false detections in methods such as YOLOv10-mid-fusion and YOLOv13-mid-fusion. By contrast, the predictions of ISDG-Net remain cleaner and less affected by background interference. This advantage is likely related to the DySparse module, whose sparse routing mechanism reduces unnecessary responses to low-correlation background regions and encourages the model to focus context aggregation on more informative target-related areas. As a result, false responses caused by redundant background structures are effectively suppressed.

In the fourth and seventh groups of examples, densely distributed vehicles cause most compared methods to produce over-detection, i.e., multiple overlapping boxes for the same target. This limitation is related to the conventional NMS strategy, which relies on hard suppression based mainly on box overlap and may struggle to distinguish highly adjacent targets from redundant predictions in crowded scenes. To address this issue, the proposed GIS module incorporates geometric information such as center distance, aspect ratio, and overlap, and replaces direct suppression with a smoother Gaussian-based score decay strategy. This design improves the model’s ability to handle dense targets and reduces redundant bounding boxes more effectively.

Overall, with the support of the efficient feature representations generated by IBC-Conv, ISDG-Net achieves more accurate multi-scale target detection while showing stronger robustness to background noise and denser object layouts than the compared methods.

In addition, comparative experiments were also conducted on the M3FD dataset. As a challenging benchmark containing a large number of multi-sensor small objects, M3FD places stricter demands on model robustness and localization accuracy. On this dataset (as shown in

Table 5. Comparison of detection accuracy and model complexity among different mainstream baseline models on the M3FD dataset, with the optimal solutions indicated in bold.

Method	Param (M)	FLOPs (G)	Precision (%)	Recall (%)	mAP@0.5 (%)	mAP@[0.5:0.95] (%)
YOLOv10-mid-fusion [24]	3.7	11.3	83.9	67.3	73.4	49.0
YOLOv11-mid-fusion [25]	3.7	9.6	83.1	64.2	71.1	46.6
YOLOv12-mid-fusion [26]	4.0	9.1	84.9	61.4	70.0	46.3
YOLOv13-mid-fusion [27]	3.5	9.9	84.5	65.9	72.7	48.6
ISDG-Net (Ours)	4.2	11.3	88.8	69.7	77.1	55.8

), ISDG-Net achieved 77.1% mAP@0.5, outperforming YOLOv10-mid-fusion (74.4%) by 2.7 percentage points. More importantly, on the stricter metric mAP@[0.5:0.95], ISDG-Net reached 55.8%, substantially exceeding the second-best result of 49.0% obtained by YOLOv10-mid-fusion. This stronger performance under high-IoU thresholds suggests that the proposed framework improves both fine-grained feature fusion and localization quality, especially in small-object and dense-scene settings.

From the perspective of Precision and Recall, the baseline models generally maintain relatively high precision (all above 83%) but show comparatively limited recall, with the best recall reaching only 67.3%. In contrast, although ISDG-Net yields a slightly lower precision (78.8%), it achieves a higher recall of 69.7%. This trade-off is valuable in remote sensing applications where missed detections are particularly costly. The result further indicates that the proposed architecture is more effective at recovering challenging small-object targets from complex low-light or infrared backgrounds.

4.5. Generalization Experiments

In addition to the primary benchmark evaluations, we further assessed the generalization ability of ISDG-Net on more challenging remote sensing and low-light detection datasets. The results demonstrate that the proposed model maintains strong robustness under diverse imaging conditions and scene distributions.

On the M3FD dataset, ISDG-Net showed strong adaptability to complex multi-sensor scenarios involving a large number of small objects (as shown in Figure 9 and

Table 6. Comparison of experimental results between different SOTA models and our model on the M3FD dataset.

Method	Param (M)	FLOPs (G)	Precision (%)	Recall (%)	mAP@0.5 (%)	mAP@[0.5:0.95] (%)
MSGA-fusion [53]	4.7	13.7	85.4	65.3	72.06	48.50
iRMB-fusion [52]	4.1	11.8	83.3	66.7	73.31	48.67
PSFM-fusion [54]	4.7	14.4	86.2	64.4	72.71	48.5
Ours	4.2	11.3	88.8	69.7	77.10	55.8

). Although the model made a slight trade-off in Precision, it achieved clear gains in Recall and delivered the best overall detection performance. In particular, ISDG-Net achieved a clear advantage on the stricter mAP@[0.5:0.95] metric, indicating that the proposed framework is effective in improving localization quality under challenging small-object conditions. These results suggest that the ISDG module can extract more robust and transferable feature representations under complex backgrounds and varying environmental conditions. As a result, the model is well suited for real-world Earth observation tasks that require accurate object localization, such as vehicle counting and infrastructure monitoring.

To further verify cross-dataset generalization, we conducted additional experiments on the LLVIP dataset, which differs substantially from the main remote sensing benchmarks in terms of scene characteristics and imaging conditions. As shown in

Table 7. Comparison of experimental results between different SOTA models and our model on the LLVIP dataset.

Method	Param (M)	FLOPs (G)	Precision (%)	Recall (%)	mAP@0.5 (%)	mAP@[0.5:0.95] (%)
MSGA-fusion [53]	4.7	13.7	91.3	89.4	93.1	62.2
iRMB-fusion [52]	4.1	11.8	91.5	88.7	92.2	61.3
PSFM-fusion [54]	4.7	14.4	93.4	87.4	93.4	63.5
Ours	4.2	11.3	91.5	89.7	93.7	63.6

, ISDG-Net demonstrated strong adaptability on this dataset. Specifically, it achieved the highest mAP@0.5 (93.7%) and Recall (89.7%), while maintaining low model complexity with only 4.2 M parameters and 11.3 G FLOPs. These results indicate that the proposed model can generalize effectively across different data domains without sacrificing deployment efficiency.

It is also worth noting that ISDG-Net exhibited faster convergence during training than the compared methods. The model reached its best performance within approximately 100 epochs, whereas most baseline methods required the full 200-epoch training schedule. These findings indicate that our introduced framework goes beyond merely boosting recognition performance; it additionally enables smoother training convergence and extracts highly generalizable representations suitable for diverse data domains.

4.6. Evaluation Experiment

To provide an intuitive assessment of ISDG-Net under challenging real-world conditions, Figure 10 presents qualitative visualizations from several representative difficult scenarios. The figure includes the ground-truth annotations and the final detection results produced by the model, together with the predictions from the visible branch (Ours [RGB]) and the infrared branch (Ours [IR]). The selected cases cover heavy smoke occlusion (e.g., a–c), distant small targets (e.g., d and f), and nighttime scenes with low illumination and strong glare (e.g., e and g–h).

The visual results indicate that ISDG-Net remains stable when one modality is severely degraded. In cases (a)–(c), dense smoke largely obscures pedestrians in the visible images, whereas the infrared modality preserves clear thermal silhouettes. With spatially adaptive fusion in Detect-SASD, the detector leverages complementary infrared cues to recover the occluded targets and outputs accurate bounding boxes. Similarly, in the nighttime cases (e) and (g), the visible modality suffers from low contrast and strong light interference, yet the model still generates outputs exhibiting strong consistency with the actual labels. In the cluttered scene shown in (f), ISDG-Net suppresses background distraction and localizes the target reliably. In the urban road scene in (h), where vehicles of very different sizes coexist, the model maintains consistent recognition across categories and scales. Overall, these qualitative examples support that ISDG-Net can exploit RGB–infrared complementarity to mitigate the physical limitations of a single sensor and to improve robustness under abrupt illumination changes and adverse visibility conditions.

5. Discussion

Existing dual-modal object detection research (including the benchmark datasets used in this paper, such as VEDAI, M3FD, and LLVIP) is mostly built on the ideal assumption that visible light and infrared images have completed strict pixel-level registration. However, in real physical deployment environments—especially when the system is mounted on practical remote sensing platforms like aerial or wheeled drones—limited by the physical baseline distance between camera lenses, mechanical vibration, reference instability during the picturing process, and time synchronization errors in dynamic scenes, spatial disparity and dynamic edge misalignment inevitably exist between different sensors. Although the Detect-SASD module designed in this paper can achieve a certain degree of adaptive fault tolerance and semantic alignment at the feature level through a learnable three-dimensional spatial weight map, when the magnitude of physical misalignment is too large and exceeds the effective receptive field of the high-level feature maps, the fusion network still faces the risk of misjudging the visible light and infrared features of the same object as two independent entities. Therefore, exploring the network’s tolerance for imperfect data and bridging the gap between highly scientific theoretical models and complex real-world hardware constraints is an important subject for improving the engineering practicality of multi-modal perception systems.

Beyond spatial misalignment, another critical dimension of engineering practicality involves the model’s universality across different data sources and its robustness against objective difficulties. While ISDG-Net demonstrates strong adaptability to challenging scenarios like low illumination (e.g., nighttime) and partial occlusion across the VEDAI, M3FD, and LLVIP datasets, this performance inherently assumes a relatively high baseline of original image quality from the specific sensors. We objectively acknowledge that, at its current stage, the architecture cannot fully overcome extreme objective difficulties, such as severe image quality degradation (e.g., heavy sensor noise) or highly incomplete image content. Furthermore, regarding dataset universality, the proposed method is not completely “plug-and-play.” When applied to datasets originating from entirely different sensor platforms—such as transitioning from low-altitude UAVs to high-altitude satellite imagery with vastly different Ground Sample Distances (GSD)—the statistical distribution shifts will inevitably require further research and adaptation (e.g., transfer learning or unsupervised domain adaptation) to align with the new appropriate conditions.

In future research, we will address these limitations from multiple technical perspectives. First, we will be dedicated to breaking through the assumption limits of strict data registration, and explore introducing weakly aligned or un-aligned cross-modal feature interaction mechanisms (such as feature matching based on deformable cross-attention) into this detection framework. Second, to tackle the objective difficulties of poor image quality and incompleteness, we plan to explicitly integrate frontend image processing technologies—such as image completion, dehazing, and denoising. By restoring high-quality imagery prior to feature extraction, we aim to perform object detection on the enhanced inputs, thereby systematically elevating the detection efficacy under extreme conditions. Furthermore, given the outstanding lightweight characteristics and all-weather robust perception capabilities demonstrated by ISDG-Net, in the future, we not only plan to apply it to high-altitude remote sensing platforms but also expect to deploy it on resource-constrained edge computing platforms (e.g., Jetson modules) as the underlying core visual foundation for robotic systems executing tasks in unstructured environments. By deeply integrating this powerful multi-modal anti-interference perception capability with embodied intelligence architectures, we aim to overcome hardware-induced instabilities and achieve high-precision dynamic visual servoing and object interaction under harsh low-visibility conditions, thereby further promoting the substantive implementation of dual-modal object detection algorithms in the complex real physical world.

6. Conclusions

Aiming at the difficulties of low illumination, strong background interference, and multi-scale object detection faced by remote sensing images in all-weather earth observation and complex unstructured environments, this paper proposes a visible-infrared dual-modal object detection framework, ISDG-Net, that balances high accuracy and lightweight design. In the feature extraction stage, this framework designs an inverted bottleneck module based on channel separation (IBC-Conv), which achieves the lossless extraction and independent representation of high-dimensional semantic features of different modalities whilst significantly cutting down the parameter volume and processing burden via separated operational steps; in terms of global context modeling, it integrates a dynamic sparse Transformer module (DySparse), utilizing the bi-level routing attention mechanism to dynamically filter background noise, reducing the computational complexity to $\mathcal{O}\left({\left(HW\right)}^{1.5}\right)$ and achieving the efficient capture of long-range dependencies for tiny objects; in the cross-modal interaction link, it constructs a detection head integrating adaptive spatial feature fusion (Detect-SASD), dynamically aligning dual-modal information at the pixel level through learnable three-dimensional spatial weights, overcoming the semantic misalignment and scale conflicts between different modalities; in the post-processing stage, it adapts a geometry-aware greedy IoU selector (GIS), replacing the traditional hard suppression with smooth Gaussian score decay, which significantly improves the recall rate and spatial localization accuracy of densely overlapping objects under the premise of not increasing additional computational costs.

Systematic experiments on three highly challenging authoritative multi-modal datasets, VEDAI, M3FD, and LLVIP, show that ISDG-Net completely surpasses existing mainstream state-of-the-art (SOTA) baseline models in both basic recognition and small object detection metrics. Under the lightweight conditions of 4.2 M parameters and 11.3 G FLOPs, the model’s mAP@0.5 on the VEDAI and M3FD datasets reached 55.1% and 77.1%, respectively, and the mAP@[0.5:0.95] reached 33.8% and 55.8%, respectively. Furthermore, on the LLVIP low-illumination dataset, the model only needs 100 epochs to converge, and achieves an mAP@0.5 of 93.7% and a recall rate of 89.7%. These data indicate that while maintaining a relatively fast convergence speed and good environmental adaptability, the model can effectively satisfy the stringent prerequisites for executing cross-modal recognition on resource-constrained edge platforms.