RAPT-Net: Reliability-Aware Precision-Preserving Tolerance-Enhanced Network for Tiny Target Detection in Wide-Area Coverage Aerial Remote Sensing

Highlights

What are the main findings?

• The RAPT-Net framework performs exceptionally well on two representative aerial remote sensing datasets (VEDAI and RGBT-Tiny), outperforming existing state-of-the-art methods in detection accuracy. It achieves mAP values of 62.22% and 18.52%, respectively, improving over the best competing methods by 4.3% and 10.3%. Notably, for extremely tiny targets (<8 × 8 pixels), APet reaches 10.86% on RGBT-Tiny, representing a 17.3% relative improvement.
• The framework demonstrates strong cross-platform adaptability, being applicable to both satellite (12.5–25 cm/pixel resolution) and UAV (60–100 m altitude) remote sensing scenarios without architectural modification. Comprehensive ablation experiments verify the synergistic effectiveness of its three core modules (MRAAF, CMFE-SRP, DS-STD), with combined gains (6.07% mAP improvement) exceeding the sum of individual contributions, enabling robust detection under varying illumination conditions and complex backgrounds while significantly reducing missed detections and false positives.

What are the implications of the main findings?

• It targets and addresses three coupled core challenges of existing aerial remote sensing small object detection methods: spatial heterogeneity of modality reliability under scene diversity, irreversible spatial information degradation from progressive downsampling, and annotation ambiguity conflicting with IoU-based training. It provides reliable technical support for precision detection in critical applications such as wide-area surveillance, traffic monitoring, maritime security, and search-and-rescue operations.
• The proposed spatial tolerance supervision strategy (DS-STD) with 4× positive sample expansion offers a generalizable solution to mitigate the inherent 1–2 pixel boundary localization uncertainty in remote sensing imagery. The hierarchy-specific feature processing paradigm (shallow layers for spatial preservation, deep layers for semantic aggregation) provides a new design principle for subsequent research on multi-scale feature extraction in tiny object detection, promoting the deployment of related technologies on operational Earth observation platforms.

Abstract

Multi-platform aerial remote sensing supports critical applications including wide-area surveillance, traffic monitoring, maritime security, and search and rescue. However, constrained by observation altitude and sensor resolution, targets inherently exhibit small-scale characteristics, making small object detection a fundamental bottleneck. Aerial remote sensing faces three unique challenges: (1) spatial heterogeneity of modality reliability due to scene diversity and illumination dynamics; (2) conflict between precise localization requirements and progressive spatial information degradation; (3) annotation ambiguity from imaging physics conflicting with IoU-based training. This paper proposes RAPT-Net with three core modules: MRAAF achieves scene-adaptive modality integration through two-stage progressive fusion; CMFE-SRP employs hierarchy-specific processing to balance spatial details and semantic enhancement; DS-STD increases positive sample coverage to 4× through spatial tolerance expansion. Experiments on VEDAI (satellite) and RGBT-Tiny (UAV) demonstrate mAP values of 62.22% and 18.52%, improving over the state of the art by 4.3% and 10.3%, with a 17.3% improvement on extremely tiny targets.

1. Introduction

Multi-platform aerial remote sensing has become a core technology for geospatial information acquisition, with multi-level platforms ranging from satellites to unmanned aerial vehicles (UAVs) serving critical applications including wide-area surveillance and reconnaissance, traffic monitoring, maritime and airspace security, and search-and-rescue operations [1,2]. However, remote sensing observation is jointly constrained by platform altitude and sensor capabilities, resulting in an inherent trade-off between coverage area and spatial resolution. When observation altitude is increased to achieve wide-area coverage, the distance between the sensor and the observation area enlarges, causing the projected size of targets on the imaging plane to decrease correspondingly, with features being compressed into increasingly smaller pixel footprints. This scale compression is not a limitation of specific sensors or platforms, but rather an intrinsic characteristic of remote sensing geometry [3,4]. Consequently, small object detection performance has become a critical challenge constraining the effectiveness of wide-area remote sensing observation, directly affecting the reliability of downstream tasks such as land cover classification, change detection, and emergency response decision-making [5,6].

Modern Earth observation relies on multi-level platforms with distinct characteristics. As shown in Figure 1, satellite and high-altitude platforms [7] acquire visible and infrared imagery at 12.5–25 cm/pixel resolution, covering diverse land surface scenes suitable for large-scale urban monitoring and land use analysis [8,9]. Low-altitude UAV platforms [10] enable flexible observation at altitudes of 60–100 m [11], covering scenes such as roads, bridges, and airports. These platforms typically carry visible and infrared sensors to acquire complementary information. Visible imaging provides rich spatial texture details but is susceptible to illumination conditions and atmospheric scattering, while infrared imaging possesses inherent robustness to illumination variations and can provide effective target–background contrast even under low-light or darkness conditions [12,13,14,15]. Effective fusion of these complementary modalities can significantly enhance small object detection capability [16,17]. However, multi-platform remote sensing applications face common challenges. Targets occupy extremely low proportions in images with complex backgrounds posing severe interference [18,19,20,21], altitude variations lead to extreme scale changes [10], and modality quality fluctuates significantly with scene and illumination conditions. How to effectively fuse multi-source heterogeneous remote sensing information to improve small object detection performance remains a critical problem [22,23].

Researchers have explored small object detection in aerial remote sensing from three directions. In multimodal fusion, early methods adopted simple concatenation strategies [24,25], while recent attention-based methods such as GAFF [26], CFT [27], and C²Former [28] have achieved adaptive fusion through dynamic weight adjustment, cross-modal dependency modeling, and dual-branch interaction, respectively. In network architecture, FPN [29] and its variants PANet [30] and BiFPN [31] have been widely introduced for multi-scale feature fusion, while SuperYOLO [14], and ALCNet [32] address remote sensing-specific challenges including feature degradation and weak infrared target detection. In training strategies, Focal Loss [33], ATSS [34], and FCOS [35] have been transferred to remote sensing scenarios, mitigating sample imbalance from perspectives of loss reweighting, adaptive sample assignment, and low-quality prediction suppression, respectively.

Despite significant progress, existing methods suffer from three fundamental limitations. First, scene diversity and dynamic imaging conditions cause modality reliability to exhibit spatial heterogeneity [36,37], where visible modality dominates in well-illuminated regions while infrared remains robust under vegetation shadows, low-light, or darkness conditions. However, existing fusion methods [26,27,28] assume spatially uniform reliability, failing to capture pixel-level differences. Second, wide-area coverage requirements compress targets to extremely low pixel proportions, causing severe spatial information degradation in deep networks [38]. Traditional feature pyramids [29,30,31] apply uniform processing across all levels, neglecting scale-specific requirements and resulting in degraded performance under complex backgrounds [32]. Third, physical limitations such as sensor point spread function, atmospheric turbulence, and platform attitude variations introduce 1–2 pixel boundary localization uncertainty [39], creating fundamental conflicts with IoU threshold-based positive sample assignment strategies [34,35]. These three challenges form a coupled propagation chain: modal reliability heterogeneity introduces fusion noise → spatial information degradation amplifies this noise → annotation uncertainty conflicts with precise matching requirements. Existing works [26,27,28,32,34] address these problems independently, failing to recognize their interdependencies. This paper proposes RAPT-Net as a systematic solution with three synergistic modules designed to break this propagation chain through feedback mechanisms.

To address the aforementioned common technical challenges in small object detection for aerial remote sensing, this paper proposes RAPT-Net (Reliability-Aware Precision-Preserving Tolerance-Enhanced Network), an adaptive multimodal small object detection framework for multi-platform remote sensing applications. The main contributions of this paper include the following four aspects.

(1)

To address the spatial heterogeneity of modality reliability caused by scene diversity and dynamic imaging conditions, we propose the Modality Reliability Assessment and Adaptive Fusion module (MRAAF). MRAAF models modality reliability as a scene-condition joint function, achieving pixel-level adaptive modality integration through two-stage progressive fusion and reliability-guided feedback mechanisms, effectively suppressing the interference of atmospheric disturbances and illumination variations on fusion quality, This differs fundamentally from existing single-stage fusion methods [26,27,28] that lack reliability feedback mechanisms to adjust fusion quality dynamically.

(2)

To address target scale compression and spatial information degradation caused by wide-area coverage observation requirements, we design the Cross-Modal Feature Enhancement with Spatial Resolution Preservation module (CMFE-SRP). The CMFE-SRP module employs a hierarchy-specific processing strategy: shallow layers preserve geometric contour details, middle layers fuse multi-source complementary features, and deep layers aggregate geospatial context, achieving a balance between spatial detail preservation and semantic enhancement. Unlike existing methods [14,31,32] that apply uniform processing across all pyramid levels, CMFE-SRP employs differentiated designs adapted to scale-specific requirements at each hierarchy.

(3)

To address the fundamental conflict between annotation ambiguity introduced by remote sensing physical limitations and IoU-based training strategies, we propose the Dense Supervision with Spatial Tolerance Domain strategy (DS-STD). DS-STD establishes a spatial tolerance mechanism targeting the 1–2 pixel boundary localization uncertainty introduced by imaging physical limitations: boundary proximity extension tolerates grid assignment ambiguity of target centers, and aspect ratio tolerance matching accommodates target shape variations caused by observation angle changes, jointly increasing positive sample coverage by 4×. Distinct from data-driven adaptive thresholding methods [34], DS-STD is physics-driven, directly modeling the inherent 1–2 pixel imaging uncertainty [39] of remote sensing platforms.

(4)

Addressing the coupled nature of these challenges, we design a systematic framework where modules interact through feedback loops: MRAAF’s reliability assessment guides CMFE-SRP’s hierarchical processing, and both influence DS-STD’s spatial tolerance design. Comprehensive ablation studies demonstrate that the synergistic gain (6.07% mAP total improvement) results from this systematic interaction rather than the simple addition of independent contributions. Comprehensive validation on VEDAI [7] and RGBT-Tiny [10] demonstrates mAP values of 62.22% and 18.52%, respectively, improving existing state-of-the-art methods by 4.3% and 10.3%, with a 17.3% improvement on extremely tiny targets, validating the effectiveness of the proposed method across different remote sensing platforms.

The remainder of this paper is organized as follows. Section 2 reviews related work. Section 3 elaborates the network architecture and module design of RAPT-Net. Section 4 presents experimental results and analysis. Section 5 concludes the paper.

2. Materials and Methods

2.1. Characteristics of Small Objects in Remote Sensing Scenarios

Small objects in remote sensing originate from the inherent geometric constraints of aerial observation. The projected size of targets on the imaging plane is inversely proportional to the sensor–target distance: when the observation distance increases from 100 m to 1000 m, the pixel proportion of targets with identical physical dimensions decreases by approximately 100 times. This scale compression is an intrinsic characteristic of imaging geometry, fundamentally distinguishing it from controlled close-range imaging environments.

The academic community primarily defines small objects based on pixel area. The MS COCO standard [40] defines small objects as those with area smaller than 32 × 32 pixels, which has been widely adopted as a reference benchmark. Addressing the unique scale distribution in remote sensing, Ying et al. [10] further subdivided the MS COCO small object category into three sub-levels—extremely tiny objects ($1^2-8^2$ pixels), tiny objects ($8^2-{16}^2$ pixels), and small objects (${16}^2-{32}^2$ pixels)—to more precisely characterize the dominant distribution of tiny objects in aerial imagery. The infrared detection domain adopts classifications of extremely tiny ($0-36$ pixels), tiny ($36-81$ pixels), and small ($81-256$ pixels) [41]. Satellite surveillance typically encounters targets smaller than $20\times 20$ pixels [42]. The relative area criterion defines small objects as those occupying less than 1% of image area [43], such as objects smaller than $30\times 30$ pixels in a $2048\times 2048$ pixel image [44].

2.2. Multi-Platform Aerial Remote Sensing Datasets

Aerial remote sensing visible–infrared datasets reflect platform-specific imaging geometry, with target scale distributions directly determined by observation altitude. As discussed in Section 2.1, the sensor–target distance fundamentally determines target pixel footprint, which explains the systematic scale differences between high-altitude and low-altitude platforms.

High-altitude and low-altitude platforms exhibit complementary observation characteristics and challenges. High-altitude platforms (primarily satellites) achieve wide-area coverage at the cost of severe scale compression. For example, VEDAI [7] collects 1210 pairs of vehicle images at 12.5–25 cm/pixel resolution, with small and medium targets comprising 40.7% and 58.3%, respectively. Low-altitude platforms (primarily UAVs) introduce extreme scale variations due to flight dynamics, leading to datasets such as DVTOD [45], DroneVehicle [46], and RGBTDronePerson [47]. Among these, RGBT-Tiny [10], as the first large-scale benchmark specifically designed for visible–infrared tiny object detection, contains 115 sequences, 93,000 frames, and 1.2 million annotations, with 81.5% of targets smaller than $16\times 16$ pixels, providing an unprecedented platform for algorithm validation under extreme scale conditions.

Table 1. Target size distribution of multi-platform aerial remote sensing datasets. Size categories are defined as follows: extremely tiny ($1^2-8^2$ pixels), tiny ($8^2-{16}^2$ pixels), small (${16}^2-{32}^2$ pixels), medium (${32}^2-{96}^2$ pixels), and large ($>{96}^2$ pixels) following [10].

Dataset	Platform	Extremely Tiny	Tiny	Small	Medium	Large
VEDAI	Satellite	0.3%	0.0%	40.7%	58.3%	0.7%
DroneVehicle	UAV	0.0%	0.0%	11.6%	84.9%	3.5%
DVTOD	UAV	0.0%	0.0%	2.4%	31.8%	65.8%
RGBTDronePerson	UAV	7.7%	84.3%	7.9%	0.1%	0.0%
RGBT-Tiny	UAV	36.7%	44.8%	15.9%	2.5%	0.1%

summarizes the target scale distributions of typical multi-platform aerial remote sensing datasets, adopting the classification scheme proposed by Ying et al.: extremely tiny ($1^2\le A\le 8^2$), tiny ($8^2<A\le {16}^2$), small (${16}^2<A\le {32}^2$), medium (${32}^2<A\le {96}^2$), and large ($A>{96}^2$) pixels. Platform-dependent patterns are clearly visible: high-altitude datasets concentrate on small-medium categories due to stable observation geometry, while UAV datasets exhibit bimodal distributions—DVTOD and DroneVehicle primarily contain medium-large targets due to lower flight altitudes, whereas RGBTDronePerson and RGBT-Tiny shift significantly toward extremely tiny and tiny categories. This diversity validates the applicability of the proposed framework under different aerial observation conditions.

2.3. Multimodal Small Object Detection Algorithms for Aerial Remote Sensing

Research on multimodal small object detection in aerial remote sensing can be divided into two categories: methods transferred from general object detection, and methods specifically designed for remote sensing challenges.

Transfer of general methods to remote sensing. In multimodal fusion, early strategies such as channel concatenation [24] and element-wise operations [25] were directly applied to remote sensing scenarios. Although attention mechanisms including GAFF [26], CFT [27], C²Former [28] improved fusion adaptability, they were originally designed for ground-view scenarios such as pedestrian detection, without considering the spatial heterogeneity of modality reliability caused by atmospheric transmission and illumination conditions in aerial remote sensing. In network architecture, multi-scale frameworks such as FPN [29], PANet [30], BiFPN [31] have been widely introduced, but their homogeneous processing strategy across all levels struggles to accommodate the highly concentrated scale distribution of remote sensing targets. In training strategies, methods such as Focal Loss [33], ATSS [34], FCOS [35], and PAA [48] mitigate imbalance issues from loss weighting and sample assignment perspectives, but their assumption of precise annotations fundamentally conflicts with the boundary ambiguity introduced by remote sensing imaging physical limitations. Recent advances in RGB-thermal fusion have explored fine-grained feature alignment strategies. Li et al. [49] proposed pseudo visible feature fusion to handle modality discrepancies in thermal object detection. Zhao et al. [50] recently reconsidered multimodal detection from a mono-modality learning perspective, demonstrating that preserving modality-specific characteristics can enhance fusion quality.

Specialized designs for remote sensing. Addressing ground sampling distance constraints in aerial imagery, SuperYOLO [14] introduces a super-resolution branch to recover feature degradation caused by downsampling. Targeting the low signal-to-noise ratio characteristics of weak infrared small targets, ALCNet [32] leverages local contrast priors to enhance target response. Addressing modality registration difficulties on UAV platforms, QFDet [47] designs quality-aware detection heads, and GLFDet [51] fuses global-local features to enhance tiny target representation. For multi-source information fusion in remote sensing, CMAFF [16] and ICAFusion [52] designed cross-modal attention mechanisms. The aforementioned methods improve remote sensing small object detection performance from specific perspectives, but have not yet formed a systematic solution addressing the three core challenges of modality reliability heterogeneity, spatial information degradation, and annotation physical ambiguity. Recent efforts have also focused on computational efficiency. Liu et al. [53] explored lightweight architectures for tiny object detection in remote sensing, achieving competitive accuracy with reduced computational complexity.

3. Methodology

Aerial remote sensing detection algorithms face unique challenges at three stages. At the information acquisition stage, visible imaging relies on reflected solar radiation and degrades severely under low-light conditions, while infrared imaging responds to target radiation characteristics with inherent robustness to illumination variations—this complementarity exhibits spatially varying distributions depending on scene conditions. At the feature extraction stage, wide-area coverage requirements combined with progressive downsampling cause irreversible degradation of small target spatial information. At the model training stage, extremely tiny targets commonly exhibit 1–2 pixel boundary localization uncertainty [39], causing IoU to be overly sensitive to localization deviations. To address the challenges at these three stages, this paper proposes RAPT-Net, comprising three modules, MRAAF, CMFE-SRP, and DS-STD, which systematically address the core problems of multimodal small object detection in aerial remote sensing from three perspectives: fusion strategy, feature extraction, and training optimization. The overall architecture of RAPT-Net is illustrated in Figure 2.

3.1. Modality Reliability Assessment and Adaptive Fusion (MRAAF)

As discussed in Section 3, the differences in imaging mechanisms between visible and infrared sensors cause modality reliability to exhibit spatial heterogeneity. Existing fusion methods assume spatially uniform modality reliability, i.e., ${\mathcal{R}}_{\mathrm{RGB}}(x)={\mathcal{R}}_{\mathrm{IR}}(x)=\mathrm{const}$, failing to capture such pixel-level reliability differences.

MRAAF models modality reliability as a scene-dependent confidence function ${\mathcal{R}}_m(x,s)$, where $x$ denotes spatial position and $s$ encodes local scene conditions. This module employs a two-stage progressive fusion strategy: the first stage performs coarse-grained global reliability estimation, and the second stage conducts fine-grained reliability modulation based on feedback from the first stage.

(1) Coarse-grained Global Reliability Estimation (Stage One): Input raw images $I_{\mathrm{RGB}}\in {ℝ}^{H\times W\times 3}$ and $I_{\mathrm{IR}}\in {ℝ}^{H\times W\times 1}$; generate initial modality response maps through a lightweight convolutional network:

$$R_{\mathrm{RGB}}^{(1)}=\sigma ({\mathrm{Conv}}_3^{(2)}(\mathrm{ReLU}({\mathrm{Conv}}_3^{(1)}(I_{\mathrm{RGB}}))))\in {[0,1]}^{H\times W\times 3}$$

(1)

$$R_{\mathrm{IR}}^{(1)}=\sigma ({\mathrm{Conv}}_3^{(2)}(\mathrm{ReLU}({\mathrm{Conv}}_3^{(1)}(I_{\mathrm{IR}}))))\in {[0,1]}^{H\times W\times 1}$$

(2)

Response maps are applied to original images with residual connections:

$${\widehat{I}}_{\mathrm{RGB}}^{(1)}=I_{\mathrm{RGB}}+I_{\mathrm{RGB}}\odot R_{\mathrm{RGB}}^{(1)}$$

(3)

$${\widehat{I}}_{\mathrm{IR}}^{(1)}=I_{\mathrm{IR}}+I_{\mathrm{IR}}\odot R_{\mathrm{IR}}^{(1)}$$

(4)

Enhanced modality features are projected to an intermediate space via 3 × 3 convolution with asymmetric channel allocation (RGB: 48 channels, IR: 16 channels). The concatenated 64-channel features undergo channel response recalibration through squeeze-and-excitation mechanism:

$$F_{\mathrm{concat}}^{(1)}=[F_{\mathrm{RGB}}^{(1)},F_{\mathrm{IR}}^{(1)}]\in {ℝ}^{H\times W\times 64}$$

(5)

$$F_{\mathrm{fused}}^{(1)}=F_{\mathrm{concat}}^{(1)}\odot \sigma (W_2\cdot \mathrm{ReLU}(W_1\cdot \mathrm{GAP}(F_{\mathrm{concat}}^{(1)})))$$

(6)

where $W_1\in {ℝ}^{64\times 4}$ and $W_2\in {ℝ}^{4\times 64}$ are squeeze–excitation layers with reduction ratio 16.

(2) Fine-grained Reliability Feedback Modulation (Stage Two): The first-stage fusion result $F_{\mathrm{fused}}^{(1)}$ is used to evaluate modality utilization quality. We define the reliability indicator vector:

$$C^{(2)}=\sigma (W_c\cdot \mathrm{GAP}(F_{\mathrm{fused}}^{(1)}))\in {[0,1]}^{1\times 1\times 4}$$

(7)

This 4-dimensional vector encodes modality balance, spatial consistency, and complementarity utilization. We decompose it into modality-specific signals:

$$C_{\mathrm{RGB}}^{(2)}=C^{(2)}[0:3,:,:]\in {[0,1]}^{1\times 1\times 3}$$

(8)

$$C_{\mathrm{IR}}^{(2)}=C^{(2)}[3:4,:,:]\in {[0,1]}^{1\times 1\times 1}$$

(9)

To achieve bidirectional adjustment based on reliability feedback, MRAAF introduces an additive modulation mechanism:

$${\tilde{R}}_{\mathrm{RGB}}^{(2)}=R_{\mathrm{RGB}}^{(2)}\odot (1+C_{\mathrm{RGB}}^{(2)})$$

(10)

$${\tilde{R}}_{\mathrm{IR}}^{(2)}=R_{\mathrm{IR}}^{(2)}\odot (1+C_{\mathrm{IR}}^{(2)})$$

(11)

where $R_{\mathrm{RGB}}^{(2)}$ and $R_{\mathrm{IR}}^{(2)}$ are generated by a convolutional network with the same architecture as stage 1. As shown in Figure 3, the modulation factor $(1+C^{(2)})$ varies in [1,2], enabling both maintenance and adaptive enhancement.

The modulated responses are applied to original inputs:

$${\widehat{I}}_{\mathrm{RGB}}^{(2)}=I_{\mathrm{RGB}}+I_{\mathrm{RGB}}\odot {\tilde{R}}_{\mathrm{RGB}}^{(2)}$$

(12)

$${\widehat{I}}_{\mathrm{IR}}^{(2)}=I_{\mathrm{IR}}+I_{\mathrm{IR}}\odot {\tilde{R}}_{\mathrm{IR}}^{(2)}$$

(13)

(3) Two-stage Weighted Aggregation: To fully utilize fusion information from both stages, we employ a learnable weighted aggregation strategy:

$$F_{\mathrm{fused}}={\displaystyle \sum _{s=1}^2{w}_s}\cdot F_{\mathrm{fused}}^{(s)},\hspace{1em}{w}_s={\displaystyle \frac{\exp ({\alpha }_s)}{{\displaystyle \sum _{k=1}^2\exp }({\alpha }_k)}}$$

(14)

where weight parameters $\left\{{\alpha }_s\right\}$ are automatically optimized through backpropagation, initialized as ${\alpha }_1=\log (0.4)$ and ${\alpha }_2=\log (0.6)$ following the weighted fusion principle in [31]. These empirically tuned values reflect a 40–60% contribution ratio for the two-stage progressive fusion, enabling MRAAF to achieve optimal balance between computational efficiency and fusion accuracy.

(4) Physical Interpretation of Reliability Modeling: To address the distinction between reliability modeling and conventional attention mechanisms, we provide visualization analysis in Figure 3. Unlike attention mechanisms that learn feature-level importance weights, MRAAF models scene-dependent modality reliability driven by physical imaging conditions. As shown in Figure 4, in well-illuminated regions (upper-left area), RGB modality dominates with higher reliability, yielding sharper detection results. Conversely, in shadowed areas (lower-left buildings) and regions with complex backgrounds, infrared modality maintains stable thermal contrast. The two-stage feedback mechanism progressively refines this reliability assessment: Stage 1 (Figure 4c) performs coarse-grained global estimation, while Stage 2 (Figure 4d) incorporates feedback from Stage 1 fusion quality to achieve fine-grained spatial modulation. This physics-driven design fundamentally differs from data-driven attention in that reliability weights explicitly correspond to observable imaging conditions rather than abstract feature statistics.

3.2. Cross-Modal Feature Enhancement with Spatial Resolution Preservation (CMFE-SRP)

As discussed in Section 3, wide-area coverage requirements combined with progressive downsampling cause irreversible degradation of small target spatial information. Unlike natural scenes where small objects can be treated as negligible background elements, small targets in aerial remote sensing are often core monitoring objects, such as vehicles in traffic monitoring, vessels in maritime surveillance, and personnel in search-and-rescue operations—their detection accuracy directly affects the execution of downstream tasks.

Small targets maintain relatively complete geometric contours in shallow feature maps (stride 4–8), but are compressed to sub-pixel-level representations in deep feature maps (stride 32), with spatial details completely lost. However, standard feature pyramids employ uniform feature extraction strategies across all levels, neither prioritizing the protection of shallow-layer spatial details nor providing differentiated designs for the concentrated scale characteristics.

To address these challenges, CMFE-SRP designs hierarchy-specific processing units: P2/P3 (stride 4/8) use lightweight Resolution Preserving Units (RPUs) for spatial detail preservation, P4 (stride 16) employs Cross-modal Semantic Alignment Units (CSAUs) for semantic alignment, and P5 (stride 32) adopts Geographic Spatial Aggregation Units (GSAUs) for global context aggregation.

(1) Resolution Preserving Unit (RPU): RPU for P2/P3 levels adopts depthwise separable convolution as its core to preserve detail independence of RGB-IR modalities:

$$F_{\mathrm{hidden} }=\mathrm{ReLU}\left(\mathrm{BN}\left({\mathrm{Conv}}_{1\times 1}^{\mathrm{in} \to \alpha \cdot \mathrm{out} }\left(F_{\mathrm{in}}\right)\right)\right)$$

(15)

$$F_{\mathrm{spatial}}=\mathrm{Re}\mathrm{LU}(\mathrm{BN}({\mathrm{DWConv}}_{3\times 3}(F_{\mathrm{hidden}})))$$

(16)

$$F_{\mathrm{recal}}=F_{\mathrm{spatial}}\odot \sigma (W_{\mathrm{fc}2}\cdot \mathrm{ReLU}(W_{\mathrm{fc}1}\cdot \mathrm{GAP}(F_{\mathrm{spatial}})))$$

(17)

$$F_{\mathrm{out} }=\mathrm{ReLU}\left(\mathrm{BN}\left({\mathrm{Conv}}_{1\times 1}^{\alpha - \mathrm{out} \to \mathrm{out} }\left(F_{\mathrm{recall}}\right)\right)\right)+F_{\mathrm{in}}$$

(18)

The bottleneck ratio $\alpha =0.5$ balances parameter efficiency and expressive capability, and the CSP structure prevents gradient vanishing. P2 uses 3 RPU blocks to avoid overfitting, while P3 uses 9 blocks to ensure sufficient feature extraction. Depthwise separable convolution is crucial for maintaining the independence of multi-source remote sensing information, because RGB and infrared images are complementary at the semantic level but spatial details may be inconsistent—shadow regions appear as dark areas in RGB, but may still exhibit thermal features in infrared. By independently processing each channel in the spatial dimension, this design avoids cross-modal interference and maintains geometric accuracy before fusion, which is crucial for precise localization of small targets.

(2) Cross-modal Semantic Alignment Unit (CSAU): CSAU for P4 layer employs sequential dual-attention mechanism. Channel attention selectively enhances modality-specific channels:

$$F_{\mathrm{channel}}=F_{\mathrm{proj}}\odot \sigma (W_{\mathrm{c}2}\cdot \mathrm{ReLU}(W_{\mathrm{c}1}\cdot \mathrm{GAP}(F_{\mathrm{proj}})))$$

(19)

where RGB channels encode visual features (texture, edges, gradients) and IR channels capture material spectral responses (NIR) or temperature gradients (TIR). Spatial attention follows for target–background discrimination:

$$M_{\mathrm{spatial} }=\sigma ({\mathrm{Conv}}_{7\times 7}([\mathrm{AvgPool}\left(F_{\mathrm{channel}}\right),\mathrm{MaxPool}\left(F_{\mathrm{channel}}\right)]))$$

(20)

$$F_{\mathrm{spatial}}=F_{\mathrm{channel}}\odot M_{\mathrm{spatial}}$$

(21)

Average pooling captures smooth regional variations while max pooling captures sharp edge features. The $7\times 7$ kernel provides sufficient receptive field to cover targets and surrounding context. Adaptive fusion is achieved through learnable gating $\gamma \in [0,2]$:

$$F_{\mathrm{out}}=\gamma \cdot F_{\mathrm{conv}}+F_{\mathrm{in}}$$

(22)

As shown in Figure 5, the network automatically adjusts $\gamma$ based on regional characteristics. The range [0,2] enables both feature suppression (γ → 0) and adaptive amplification (γ > 1) when cross-modal complementarity is strong, enhancing weak tiny target responses.

(3) Geographic Spatial Aggregation Unit (GSAU): GSAU for P5 layer employs spatial pyramid pooling to capture multi-scale geospatial context:

$$F_{\mathrm{reduced}}={\mathrm{Conv}}_{1\times 1}(F_{\mathrm{in}})\in {ℝ}^{H\times W\times C/2}$$

(23)

$$F_{\mathrm{pool}-k}={\mathrm{MaxPool}}_{k\times k}^{\mathrm{stride}=1}(F_{\mathrm{reduced}}),\hspace{1em}k\in \{5,9,13\}$$

(24)

$$F_{\mathrm{SPP}}={\mathrm{Conv}}_{1\times 1}([F_{\mathrm{reduced}},F_{\mathrm{pool}-5},F_{\mathrm{pool}-9},F_{\mathrm{pool}-13}])$$

(25)

Reliable identification of targets in remote sensing images relies on contextual cues—vehicles in parking lots, vessels beside docks—requiring the combination of local neighborhood, medium-range, and wide-area background information. Different pooling kernels (k = 5, 9, 13) capture geospatial context at these three levels, respectively, effectively reducing false alarm rates against complex ground surface backgrounds.

As shown in Figure 6, different pooling kernels simulate different geographic scales: $k=5$ captures local neighborhoods, $k=9$ captures medium-range contexts, and $k=13$ captures wide-area contexts. This multi-scale context is crucial for reducing false detections of small targets. P5 uses standard bottleneck rather than lightweight RPU as the feature map size is small with low overfitting risk, and high-level semantics require stronger nonlinear transformation capability.

(4) Complete CMFE-SRP Architecture: The forward propagation follows progressive downsampling with differentiated processing:

$$F_{P2}={\mathrm{CSP}-\mathrm{RPU}}_{n=3}({\mathrm{Conv}}^{\mathrm{stride}=2}(F_{\mathrm{fused}}))\in {ℝ}^{H/4\times W/4\times 128}$$

(26)

$$F_{P3}={\mathrm{CSP}-\mathrm{RPU}}_{n=9}({\mathrm{Conv}}^{\mathrm{stride}=2}(F_{P2}))\in {ℝ}^{H/8\times W/8\times 256}$$

(27)

$$F_{P4}={\mathrm{CSP}-\mathrm{CSAU}}_{n=9}({\mathrm{Conv}}^{\mathrm{stride}=2}(F_{P3}))\in {ℝ}^{H/16\times W/16\times 512}$$

(28)

$$F_{P5}={\mathrm{CSP}-\mathrm{Standard}}_{n=3}(\mathrm{SPP}({\mathrm{Conv}}^{\mathrm{stride}=2}(F_{P4})))\in {ℝ}^{H/32\times W/32\times 1024}$$

(29)

This design balances spatial–semantic trade-off by prioritizing resolution preservation in shallow layers and semantic enhancement in deep layers. Model capacity is matched to feature map size and target occupancy, while channel independence preserves modality-specific information.

3.3. Dense Supervision with Spatial Tolerance Domain (DS-STD)

Extremely tiny targets exhibit 1–2 pixel boundary localization uncertainty due to physical limitations of remote sensing imaging [39], causing IoU to be overly sensitive to deviations. For an 8 × 8 pixel target, 1-pixel deviation on each side drops IoU from 1.0 to 0.64, easily falling below the typical 0.5 positive sample threshold. Moreover, such targets mapped to stride-8 feature maps occupy only 1 × 1 grid cells, producing positive-to-negative sample ratios exceeding 1:10,000 that overwhelm localization gradients with background signals. To address these problems, DS-STD proposes the spatial tolerance domain concept: the boundary proximity extension rule tolerates grid assignment ambiguity between adjacent cells, while the aspect ratio tolerance matching rule accommodates pixel-level annotation deviations, jointly extending positive sample regions around target centers.

(1) Boundary Proximity Extension Rule: This rule addresses the quantization effect when continuous target coordinates are mapped to discrete feature map grids. We calculate the relative position of the target center within the feature map grid:

$$g_x={\displaystyle \frac{x_c}{s}}\hspace{0.17em}\mathrm{mod}\hspace{0.17em}1$$

(30)

$$g_y={\displaystyle \frac{y_c}{s}}\hspace{0.17em}\mathrm{mod}\hspace{0.17em}1$$

(31)

where $x_c$ and $y_c$ are the target center coordinates, and $s$ is the feature stride. The values $g_x$ and $g_y$ represent the fractional position within the grid cell, ranging from 0 to 1.

As shown in Figure 7, this extension can increase positive samples from 1 to up to 4 per target. When $g_x<0.2$ or $g_x>0.8$ (similarly $g_y<0.2$ or $g_y>0.8$), the target center lies close to the grid boundary. In such cases, the adjacent grid cell also contains significant target information. We therefore extend the positive sample assignment to include these neighboring units. This extension can increase positive samples from 1 to up to 4 per target, representing a 300% improvement in positive sample coverage.

(2) Aspect Ratio Tolerance Matching Rule: Traditional anchor matching requires strict IoU thresholds, which penalizes small targets disproportionately due to their limited pixel coverage. We propose a shape-based matching criterion that focuses on aspect ratio compatibility:

$$\mathrm{Match} \mathrm{if}: \max \left({\displaystyle \frac{w_{\mathrm{gt}}}{w_a}},{\displaystyle \frac{h_{\mathrm{gt}}}{h_a}},{\displaystyle \frac{w_a}{w_{\mathrm{gt}}}},{\displaystyle \frac{h_a}{h_{\mathrm{gt}}}}\right)<4$$

(32)

where ($w_{gt}$, $h_{gt}$) are ground truth dimensions and ($w_a$, $h_a$) are anchor dimensions. This rule allows a maximum 4:1 aspect ratio difference, accommodating natural shape variations in remote sensing targets while avoiding extreme mismatches that would degrade regression quality.

(3) Extended Prediction Range: To complement the spatial tolerance domain, we extend the detection head’s prediction range beyond the traditional [0,1] grid-relative coordinates:

$$x=(\sigma (\Delta x)\cdot 2-0.5+c_x)\cdot s$$

(33)

$$w={(\sigma (\Delta w)\cdot 2)}^2\cdot w_a\cdot s$$

(34)

The center prediction range extends from [0,1] to [−0.5, 1.5], allowing predictions to slightly exceed the current grid cell. This design echoes the spatial tolerance domain expansion, ensuring that targets near grid boundaries can be accurately localized by either the current cell or its neighbors.

4. Experiments

This section is organized as follows: Section 4.1 describes the datasets and evaluation metrics; Section 4.2 details the experimental setup; Section 4.3 presents comparisons with state-of-the-art methods; Section 4.4 conducts ablation studies to validate each proposed component.

4.1. Datasets and Evaluation Metrics

4.1.1. Datasets

VEDAI [7]. VEDAI is an aerial vehicle detection dataset containing 1210 high-resolution (1024 × 1024) RGB-infrared image pairs with 3706 annotated targets across 8 vehicle categories. As shown in Figure 7, small and medium targets comprise nearly 99% of all instances (40.7% and 58.3%, respectively). This dataset represents typical vehicle monitoring scenarios from medium-to-high altitude aerial remote sensing platforms, with applications in urban traffic flow statistics, parking lot occupancy analysis, and road network vehicle distribution monitoring, providing a reliable benchmark for evaluating algorithm potential in practical traffic monitoring applications.

RGBT-Tiny [10]. RGBT-Tiny is the first large-scale benchmark for visible-thermal tiny object detection, containing 115 sequences, 93,000 frames, and 1.2 M annotations across 7 categories and 8 scene types. As shown in Figure 8, over 81% of targets are smaller than 16 × 16 pixels (36.7% extremely tiny, 44.8% tiny). This dataset represents fine-grained monitoring scenarios from low-altitude UAV platforms, covering diverse urban functional areas including campuses, streets, parking lots, and playgrounds. The extreme scale distribution corresponds to typical imaging conditions for pedestrians and vehicles at 60–100 m flight altitudes, making it an ideal platform for evaluating algorithm performance in personnel search, security surveillance, and emergency response applications. Based on the original RGBT-Tiny dataset [10], we re-partitioned the 115 sequences into 55 for training (20,300 frames), 30 for validation (13,433 frames), and 30 for testing (12,968 frames), expanding from the original binary split to enable proper model validation.

4.1.2. Metrics

We employed the Scale Adaptive Fitness (SAFit) metric [10] as our core criterion. SAFit addresses IoU’s low tolerance to bounding box perturbations for small targets through scale-adaptive weighted fusion:

$$\mathrm{SAFit}={\displaystyle \frac{1}{1+e^{-(\sqrt{A}/C-1)}}}\times \mathrm{IoU}+\left(1-{\displaystyle \frac{1}{1+e^{-(\sqrt{A}/C-1)}}}\right)\times \mathrm{NWD}(C)$$

(35)

where $A$ is the ground truth box area (pixels²) and $C$ is the scale parameter controlling the weights of IoU and NWD; in this paper, $C=32$.

We report: $AP$ ($mAP$) over SAFit thresholds $[0.5:0.05:0.95]$; $A{P}_{50}/A{P}_{75}$ at thresholds 0.50/0.75; $AR$ (average recall); and scale-specific metrics ($A{P}_{et}$, $A{P}_t$, $A{P}_s$, $A{P}_m$, $A{P}_l$) for extremely tiny ($(1^2,8^2]$), tiny ($(8^2,{16}^2]$), small ($({16}^2,{32}^2]$), medium ($({32}^2,{96}^2]$), and large ($({96}^2,\infty )$) targets, respectively.

4.2. Experimental Setup

4.2.1. Implementation Details

All experiments used NVIDIA RTX 4090 GPU (24 GB) (NVIDIA Corporation, Santa Clara, CA, USA) with PyTorch 2.4.1 (CUDA 12.1). Input resolutions: VEDAI at 1024 × 1024, RGBT-Tiny at 640 × 512. Batch size was 4. We used SGD optimizer (momentum = 0.937, weight decay = 0.0005) with cosine annealing (0.01→0.01%, 300 epochs) and 5-epoch warmup. Early stopping patience was 30.

Data augmentation: random horizontal flip (p = 0.5), scaling (0.5–1.5×), translation (±10%), brightness/contrast/saturation (±0.2, RGB only), Mosaic (p = 0.5), and MixUp (p = 0.15). Augmentation was disabled during testing.

4.2.2. Training Strategy Configuration and Fair Comparison

To comprehensively evaluate the performance of RAPT-Net on visible–infrared small object detection tasks in aerial remote sensing, comparative experiments were conducted on two benchmark datasets: VEDAI and RGBT-Tiny. The compared methods cover three categories: (1) single-modality detectors, including visible and infrared versions of YOLOv8 [54] and YOLOv10 [55], to establish performance baselines; (2) general multimodal fusion methods, including CFT [27], ICAFusion [52], and CMA-Det [45]; (3) multimodal detectors optimized for remote sensing small objects, including SuperYOLO [14], QFDet [47], and GLFDet [51]. To ensure fair comparison, all baseline methods retain their original training strategies as reported in their respective publications: YOLOv8/YOLOv10 employ standard center-point assignment, SuperYOLO uses resolution-aware sample assignment, while CFT and QFDet adopt IoU threshold-based adaptive assignment. The proposed DS-STD strategy was specifically designed to address the inherent 1–2 pixel boundary localization uncertainty in remote sensing imagery [39], which arises from physical constraints including sensor point spread function, atmospheric turbulence, and platform attitude variations—fundamentally differing from annotation ambiguity in general object detection scenarios.

To address potential concerns regarding training strategy differences and ensure the fairness of performance comparison, we adopted a modular evaluation approach that allows for the independent assessment of each component’s contribution. Table in Section 4.4.4 presents comprehensive ablation studies designed to isolate the impact of DS-STD from other modules (MRAAF and CMFE-SRP), enabling quantification of how much improvement stems from modified training strategies versus novel architectural designs. Furthermore, the evaluation was conducted on two datasets with substantially different characteristics: VEDAI represents satellite platform scenarios with relatively stable imaging geometry, while RGBT-Tiny reflects UAV platform conditions with extreme scale variations and dynamic illumination changes. Additionally, scale-specific analysis across extremely tiny, tiny, and small target categories further validated whether DS-STD addresses a fundamental matching problem inherent to remote sensing imagery or merely exploits dataset-specific characteristics.

4.3. Comparison with State-of-the-Art Methods

4.3.1. On the VEDAI Dataset

The VEDAI dataset, as a classic aerial vehicle detection benchmark, provides a standardized platform for evaluating algorithm performance in medium-complexity remote sensing scenarios.

As shown in

Table 2. Quantitative comparison of different methods on the VEDAI dataset. Methods are categorized into three groups: single-modality detectors (YOLOv8, YOLOv10), general multimodal fusion methods (CFT, ICAFusion, CMA-Det, QFDet, GLFDet), and remote sensing-optimized multimodal detectors (SuperYOLO, proposed RAPT-Net). Evaluation metrics include mean Average Precision (mAP) averaged over SAFit thresholds [0.5:0.05:0.95], scale-specific Average Precision for small targets (AP_s) and medium targets (AP_m), and Average Recall (AR). The best results are highlighted in red bold, second-best in blue bold, and third-best in purple bold. RAPT-Net achieves favorable accuracy–efficiency trade-off for practical deployment.

Methods	Modality	mAP	mAP50	mAP75	APs	APm	AR	Params (M)	GFLOPs	FPS
YOLOv8	RGB	0.5428	0.7885	0.6860	0.4437	0.5521	0.1919	3.01	8.9	230.83
YOLOv8	IR	0.5640	0.8002	0.7092	0.4652	0.5708	0.1991	3.01	8.9	231.08
YOLOv10	RGB	0.4908	0.7128	0.6272	0.3924	0.4919	0.1793	2.71	8.4	138.43
YOLOv10	IR	0.5155	0.7410	0.6590	0.4191	0.5123	0.1937	2.71	8.4	137.55
CFT	RGB-IR	0.5842	0.8162	0.7157	0.4846	0.6061	0.2160	32.4	89.6	23.71
ICAFusion	RGB-IR	0.5781	0.8094	0.6961	0.4768	0.5962	0.2066	35.4	98.2	45.00
SuperYOLO	RGB-IR	0.5967	0.8380	0.7404	0.5056	0.6119	0.2237	5.04	54.3	21.04
CMA-Det	RGB-IR	0.5717	0.7964	0.6909	0.4701	0.5839	0.2085	27.6	141.3	15.16
QFDet	RGB-IR	0.5846	0.8151	0.7040	0.4877	0.5924	0.2158	51.2	168.5	26.30
GLFDet	RGB-IR	0.5760	0.8077	0.6911	0.4732	0.5886	0.2069	29.3	156.8	15.00
RAPT-Net (ours)	RGB-IR	0.6222	0.8483	0.7595	0.5268	0.6452	0.2439	37.8	64.9	28.95

, RAPT-Net achieves optimal performance across all evaluation metrics. Specifically, RAPT-Net’s $mAP$ reaches 62.22%, improving by 2.55 and 5.82 percentage points over the second-best method SuperYOLO (59.67%) and the best single-modality method YOLOv8-IR (56.40%), respectively. For $mA{P}_{50}$ and $mA{P}_{75}$ metrics, RAPT-Net achieves 84.83% and 75.95%, demonstrating advantages in both detection and precise localization. Regarding scale-specific performance, RAPT-Net achieves 52.68% for small targets ($A{P}_s$) and 64.52% for medium targets ($A{P}_m$), outperforming SuperYOLO by 2.12 and 3.33 percentage points, respectively. These results validate that RAPT-Net achieves comprehensive performance improvement by systematically addressing the three core challenges of modality fusion, spatial resolution preservation, and scale matching. Regarding efficiency, RAPT-Net achieves 28.95 FPS with 37.8 M parameters and 64.9 GFLOPs, demonstrating favorable accuracy–efficiency balance. Compared to QFDet (51.2 M parameters, 168.5 GFLOPs) and CFT (32.4 M parameters, 89.6 GFLOPs), RAPT-Net provides superior accuracy with reasonable computational cost.

Notably, YOLOv10 exhibits lower performance than YOLOv8 on VEDAI despite architectural advances in natural scene detection. This performance gap likely stems from YOLOv10’s optimization for general object detection scenarios, where its reduced model capacity (2.71 M vs. 3.01 M parameters) and architecture choices may not effectively capture the specific characteristics of aerial vehicle detection at high altitudes. This observation underscores the necessity of remote sensing-specific designs as implemented in RAPT-Net.

Figure 9 presents the Precision–Recall (PR) curves of all compared methods on the VEDAI dataset. Overall, RAPT-Net achieves the best performance in terms of the area under the curve, though GLFDet shows competitive or slightly higher precision in certain recall ranges. The curves exhibit approximate hierarchical stratification: RAPT-Net, SuperYOLO, and GLFDet form the top tier with frequent crossovers among them; CFT, QFDet, and ICAFusion constitute the middle tier; single-modality detectors (YOLOv8, YOLOv10) occupy the bottom tier, showing rapid precision decay as recall increases. Notably, in the high recall region (>0.6), RAPT-Net maintains a more gradual descent compared to most other methods, reflecting its superior capability in retrieving difficult samples without introducing excessive false positives.

Figure 10 provides qualitative detection results across five representative scenes from the VEDAI dataset. Each row displays the same scene processed by different methods, with ground truth shown in the first column. In the scene of Row 1, RAPT-Net generates clean and accurate bounding boxes on vessels, while YOLOv8 exhibits obvious missed detections, and GLFDet has a serious problem with false positives. In the scene of Row 2, RAPT-Net’s detection boxes precisely cover all vehicles with consistent box sizes, whereas other methods show missed detections or redundant boxes. The scene in Row 3 presents a background with cluttered line patterns; both GLFDet and SuperYOLO exhibit false detections, while RAPT-Net maintains clean outputs focusing only on actual targets. In the scene of Row 4, RAPT-Net consistently generates correct detection boxes, while competing methods all exhibit false detections. Across all visualizations, RAPT-Net demonstrates the most consistent detection behavior with minimal background interference and precise target localization.

4.3.2. On the RGBT-Tiny Dataset

The RGBT-Tiny dataset, as a cutting-edge benchmark for extreme small target detection, served as the primary validation platform for the core innovations of this paper. The dataset contains 81.5% extremely tiny and tiny targets (<16 × 16 pixels), providing an ideal testing scenario for evaluating algorithm detection capability at extreme scales.

As shown in

Table 3. Quantitative comparison of different methods on the RGBT-Tiny dataset. This dataset serves as the first large-scale benchmark specifically designed for visible-thermal tiny object detection, containing over 81% targets smaller than 16 × 16 pixels, providing an ideal platform for evaluating algorithm performance under extreme scale conditions. Methods are categorized into three groups: single-modality detectors (YOLOv8, YOLOv10), general multimodal fusion methods, and remote sensing-optimized multimodal detectors (SuperYOLO and the proposed RAPT-Net). Evaluation metrics include mean Average Precision (mAP) averaged over SAFit thresholds [0.5:0.05:0.95]; mAP at IoU thresholds of 0.50 and 0.75; scale-specific Average Precision for extremely tiny targets (AP_et), tiny targets (AP_t), and small targets (AP_s); as well as Average Recall (AR). The best results are highlighted in red bold, second-best in blue bold, and third-best in purple bold. All methods are trained and evaluated under identical experimental settings for fair comparison.

Methods	Modality	mAP	mAP50	mAP75	APet	APt	APs	AR	Params (M)	GFLOPs	FPS
YOLOv8	RGB	0.1084	0.1987	0.1251	0.0524	0.0992	0.1567	0.1428	3.01	2.78	230.94
YOLOv8	IR	0.0862	0.1581	0.0976	0.0342	0.0718	0.1236	0.1135	3.01	2.78	229.48
YOLOv10	RGB	0.1139	0.2064	0.1317	0.0571	0.1047	0.1639	0.1482	2.71	2.62	138.56
YOLOv10	IR	0.0904	0.1645	0.1024	0.0378	0.0761	0.1297	0.1188	2.71	2.62	138.92
CFT	RGB-IR	0.1254	0.2276	0.1465	0.0624	0.1178	0.1854	0.1647	32.4	27.9	45.74
ICAFusion	RGB-IR	0.1376	0.2454	0.1618	0.0746	0.1324	0.2076	0.1823	35.4	30.6	103.50
SuperYOLO	RGB-IR	0.1425	0.2558	0.1684	0.0714	0.1337	0.1984	0.1886	5.04	16.9	52.60
CMA-Det	RGB-IR	0.1205	0.2178	0.1379	0.0593	0.1138	0.1745	0.1571	27.6	44.0	30.32
QFDet	RGB-IR	0.1679	0.3015	0.1949	0.0926	0.1631	0.2267	0.2174	51.2	52.5	35.94
GLFDet	RGB-IR	0.1296	0.2347	0.1484	0.0648	0.1213	0.1924	0.1697	29.3	48.9	37.50
RAPT-Net (ours)	RGB-IR	0.1852	0.3287	0.2142	0.1086	0.1859	0.2412	0.2369	37.8	20.3	58.77

, RAPT-Net demonstrates significant performance advantages on the RGBT-Tiny dataset. RAPT-Net’s $mAP$ reaches 18.52%, improving by 1.73 and 7.13 percentage points over the second-best method QFDet (16.79%) and the best single-modality method YOLOv10-RGB (11.39%), respectively. For $mA{P}_{50}$ and $mA{P}_{75}$ metrics, RAPT-Net achieves 32.87% and 21.42%, demonstrating advantages in both detection and precise localization. Regarding scale-specific performance, RAPT-Net achieves 10.86% for extremely tiny targets ($A{P}_{et}$), 18.59% for tiny targets ($A{P}_t$), and 24.12% for small targets ($A{P}_s$), outperforming QFDet by 1.60, 2.28, and 1.45 percentage points, respectively. Notably, the performance gap between RAPT-Net and other methods is significantly amplified on RGBT-Tiny compared to VEDAI, confirming that the smaller the target size, the greater the value of modality reliability assessment and spatial resolution preservation mechanisms. RAPT-Net achieves 58.77 FPS with 20.3 GFLOPs, outperforming QFDet (+1.73% mAP) with 63% higher FPS and 26% fewer parameters, confirming superior efficiency for UAV-based tiny target detection.

Figure 11 presents the Precision–Recall (PR) curves on the RGBT-Tiny dataset. Notably, the effective recall range is constrained to [0, 0.6] rather than [0, 1.0] as in VEDAI, reflecting inherent limitations in feature representation for extremely tiny targets. In the low recall region (<0.45), GLFDet, QFDet, and SuperYOLO exhibit higher precision than RAPT-Net; however, in the high recall region (>0.45), RAPT-Net surpasses all other methods and maintains a more gradual descent, demonstrating stronger capability in retrieving difficult samples. In terms of area under the curve, RAPT-Net achieves the best overall performance. The curves exhibit clear stratification: multimodal fusion methods (RAPT-Net, QFDet, GLFDet, SuperYOLO, ICAFusion) occupy the upper region, while single-modality detectors cluster at the bottom. Particularly, YOLOv8_RGB terminates before reaching 0.2 recall, and other single-modality methods approach near-zero precision beyond recall of 0.3, visually demonstrating the essential role of multimodal fusion for extremely tiny target detection.

Figure 12 provides qualitative detection results across five representative scenes from the RGBT-Tiny dataset. Each row displays the same scene processed by different methods, with ground truth shown in the first column. In Row 1, YOLOv8 exhibits obvious false detections and missed detections, while CFT, SuperYOLO, and GLFDet all fail to detect the bus target; in contrast, RAPT-Net produces comprehensive coverage with detection boxes closely matching ground truth annotations. In Row 2, targets become dense and detection difficulty increases. YOLOv8 shows numerous missed detections, while CFT, SuperYOLO, and GLFDet exhibit obvious false detections. Although RAPT-Net also has some missed detections, its detection accuracy is significantly improved compared to other methods. In Row 3, under extremely challenging conditions with very tiny targets, RAPT-Net still demonstrates precise detection capability. In Row 4, illumination becomes poor. YOLOv8 completely fails to detect any targets, CFT and SuperYOLO also suffer from severe missed detections, while GLFDet exhibits more severe false detections. In comparison, RAPT-Net shows neither false detections nor missed detections. Across all visualizations, RAPT-Net demonstrates the highest detection density with minimal background interference, validating its superior capability for extremely tiny target detection.

4.4. Ablation Studies

To systematically validate the effectiveness of each proposed module in RAPT-Net, we conducted comprehensive ablation studies on the RGBT-Tiny dataset. The baseline architecture (A0, B0, D0, E0 in

Table 4. Ablation study of MRAAF fusion strategy on the RGBT-Tiny dataset. A0: baseline with simple concatenation fusion. A1: single-stage MRAAF with coarse-grained reliability estimation. A2: two-stage fusion without feedback mechanism. A3: complete MRAAF with learnable aggregation and reliability-guided feedback. Metrics include mAP and scale-specific AP_et, AP_t, AP_s. Results demonstrate progressive improvement from each component.

ID	Fusion Strategy	Guidance	mAP	APet	APt	APs
A0	Baseline	×	0.1245	0.0629	0.1164	0.1732
A1	Single stage	×	0.1368	0.0718	0.1285	0.1896
A2	Two stage	×	0.1456	0.0804	0.1378	0.2014
A3	Two stage	√	0.1521	0.0865	0.1452	0.2108

,

Table 5. Ablation study of CMFE-SRP hierarchy-specific processing units on the RGBT-Tiny dataset. B0: baseline with standard C3 blocks at all levels. B1: RPU at P2/P3 for spatial preservation. B2: CSAU at P4 for semantic alignment. B3: GSAU at P5 for context aggregation. B4: complete CMFE-SRP integrating all units. Metrics include mAP and scale-specific AP_et, AP_t, AP_s. Results confirm that RPU at shallow layers contributes the most significant improvement for tiny target detection.

ID	P2/P3	P4	P5	mAP	APet	APt	APs
B0	C3	C3	C3+SPP	0.1245	0.0629	0.1164	0.1732
B1	RPU	C3	C3+SPP	0.1586	0.0924	0.1518	0.2103
B2	C3	CSAU	C3+SPP	0.1412	0.0758	0.1337	0.1894
B3	C3	C3	GSAU	0.1328	0.0694	0.1251	0.1806
B4	RPU	CSAU	GSAU	0.1712	0.1015	0.1647	0.2256

,

Table 6. Ablation study of DS-STD dense supervision strategy on the RGBT-Tiny dataset. D0: baseline with standard positive sample assignment. D1: Boundary Proximity Extension for grid assignment tolerance. D2: Aspect Ratio Tolerance Matching for shape variation accommodation. D3: complete DS-STD combining both strategies with up to 4× positive sample expansion. Metrics include mAP and scale-specific AP_et, AP_t, AP_s. Results demonstrate synergistic improvement exceeding the sum of individual contributions.

ID	Boundary Extension	Aspect Ratio Tolerance	mAP	APet	APt	APs
D0	×	×	0.1437	0.0768	0.1359	0.1975
D1	√	×	0.1521	0.0845	0.1446	0.2068
D2	×	√	0.1498	0.0821	0.1423	0.2041
D3	√	√	0.1612	0.0934	0.1548	0.2187

and

Table 7. Comprehensive ablation study of module combinations on the RGBT-Tiny dataset. E0: baseline without proposed modules. E1 to E3: individual modules (MRAAF, CMFE-SRP, DS-STD). E4 to E6: pairwise combinations. E7: complete RAPT-Net. Metrics include mAP and scale-specific AP_et, AP_t, AP_s. Results show CMFE-SRP provides the largest individual improvement, and the complete framework achieves 6.07 percentage point mAP improvement, with synergistic gains exceeding the sum of individual contributions.

ID	MRAAF	CMFE-SRP	DS-STD	mAP	APet	APt	APs
E0	×	×	×	0.1245	0.0629	0.1164	0.1732
E1	√	×	×	0.1521	0.0865	0.1452	0.2108
E2	×	√	×	0.1712	0.1015	0.1647	0.2256
E3	×	×	√	0.1437	0.0768	0.1359	0.1975
E4	√	√	×	0.1798	0.1084	0.1731	0.2348
E5	√	×	√	0.1643	0.0947	0.1576	0.2186
E6	×	√	√	0.1826	0.1017	0.1764	0.2391
E7	√	√	√	0.1852	0.1086	0.1859	0.2412

) adopted YOLOv5 with simple element-wise addition for RGB-infrared fusion, using standard C3 blocks uniformly across all feature pyramid levels without reliability assessment or spatial tolerance mechanisms.

The experiments were organized into four parts: MRAAF fusion strategy ablation, CMFE-SRP hierarchy-specific processing unit ablation, DS-STD dense supervision strategy ablation, and comprehensive module combination ablation.

4.4.1. MRAAF Fusion Strategy Ablation

The MRAAF module addresses the spatial heterogeneity of modality reliability through a two-stage progressive fusion strategy. The baseline (A0) employs simple concatenation fusion. Configuration A1 implements single-stage MRAAF with only coarse-grained global reliability estimation. Configuration A2 extends to two-stage fusion without reliability-guided feedback. Configuration A3 represents the complete MRAAF design with both learnable weight aggregation and reliability-guided feedback mechanism.

As shown in Table 4, the results demonstrate progressive improvement from each MRAAF component. Single-stage MRAAF (A1) improves $mAP$ by 1.23 percentage points over baseline, indicating that coarse-grained reliability estimation provides meaningful guidance. Two-stage fusion (A2) further improves $mAP$ by 0.88 percentage points. The complete MRAAF (A3) achieves a $mAP$ value of 15.21%, with $A{P}_{et}$ improving from 8.04% to 8.65%. The reliability-guided feedback mechanism contributes an additional 0.65 percentage points in $mAP$, confirming that the combination of learnable aggregation and reliability-guided feedback enables optimal balance between efficiency and accuracy.

As shown in Figure 13, baseline A0 exhibits diffuse feature activations with weak target–background discrimination due to unresolved modality conflicts. Configurations A1 and A2 show progressively concentrated activation patterns with improved spatial localization. The complete MRAAF in A3 produces the sharpest feature responses, achieving the most prominent target highlighting through reliability-guided adaptive fusion. These results demonstrate that the combination of two-stage progressive fusion and reliability feedback mechanism is essential for addressing the spatial heterogeneity of modality reliability caused by land cover diversity and dynamic imaging conditions in aerial remote sensing imagery.

4.4.2. CMFE-SRP Hierarchy-Specific Processing Unit Ablation

The CMFE-SRP module addresses spatial information loss through hierarchy-specific processing units. Baseline B0 uses standard C3 blocks at all levels. B1 introduces Resolution Preserving Units (RPUs) at P2/P3. B2 employs Cross-modal Semantic Alignment Units (CSAUs) at P4. B3 substitutes Geographic Spatial Aggregation Units (GSAUs) at P5. B4 represents the complete design integrating all units.

As shown in Table 5, the RPU at P2/P3 (B1) contributes the most significant improvement, achieving a $mAP$ value of 15.86% with $A{P}_{et}$ increasing from 6.29% to 9.24%, confirming that maintaining spatial detail independence is crucial for extremely small targets—for targets occupying less than 0.04% of image area, geometric information in shallow feature maps is fundamental for precise localization. The CSAU at P4 (B2) and GSAU at P5 (B3) improve $mAP$ by 1.67 and 0.83 percentage points, respectively. The complete CMFE-SRP (B4) achieves a $mAP$ value of 17.12% and $A{P}_{et}$ of 10.15%, with the combined gain (4.67 percentage points) exceeding individual improvements, suggesting synergistic interactions among hierarchy-specific units.

As shown in Figure 14, B0 is not displayed as it shares identical visualization with baseline A0. B1 with RPU shows relatively uniform activation distribution with moderate target responses. B2 with CSAU produces stronger activation intensity with prominent high-response regions. B3 with GSAU exhibits diffuse activation patterns. The complete B4 achieves the most balanced distribution with clearly concentrated target responses and suppressed background interference.

4.4.3. DS-STD Dense Supervision Strategy Ablation

The DS-STD module addresses extreme positive–negative sample imbalance through the spatial tolerance domain concept. D0 serves as the baseline using standard positive sample assignment. D1 implements Boundary Proximity Extension. D2 applies Aspect Ratio Tolerance. D3 combines both strategies.

As shown in Table 6, the Boundary Proximity Extension (D1) improves $mAP$ by 0.84 percentage points, with $A{P}_{et}$ increasing from 7.68% to 8.45%, confirming that extremely tiny targets benefit most from increased positive sample coverage. The Aspect Ratio Tolerance (D2) contributes 0.61 percentage points improvement with consistent gains across all target sizes. The combined strategy (D3) achieves a $mAP$ value of 16.12% and $A{P}_{et}$ value of 9.34%, with the total improvement (1.75 percentage points) exceeding the sum of individual gains, suggesting synergistic interaction between the two components.

As shown in Figure 15, D0 exhibits scattered high-response regions with substantial background noise interference. D1 with boundary proximity extension shows reduced background activation and more concentrated target responses. D2 with aspect ratio tolerance achieves similar improvement in activation focus. The complete D3 produces the cleanest feature map with minimal background noise and the most precise target highlighting, demonstrating synergistic effects of both strategies.

The effectiveness of DS-STD confirms the impact of inherent ambiguity in aerial remote sensing small target annotation on training: boundary proximity expansion and aspect ratio tolerant matching effectively mitigate the unstable training signal caused by pixel-level annotation uncertainty by increasing positive sample coverage.

4.4.4. Comprehensive Ablation Experiments

To evaluate overall contribution and interaction effects, we conduct comprehensive experiments with all module combinations. E0 represents baseline, E1–E3 evaluate individual modules, E4–E6 test pairwise combinations, and E7 represents complete RAPT-Net.

As shown in Table 7, among individual modules, CMFE-SRP (E2) provides the largest improvement ($mAP$, 17.12%), followed by MRAAF (E1, 15.21%) and DS-STD (E3, 14.37%), aligning with challenge severity where spatial information loss is the most critical bottleneck. The pairwise combinations demonstrate positive synergistic effects: MRAAF with CMFE-SRP (E4) achieves 17.98%, and CMFE-SRP with DS-STD (E6) achieves 18.26%. The complete RAPT-Net (E7) achieves a $mAP$ value of 18.52%, $A{P}_{et}$ value of 10.86%, $A{P}_t$ value of 18.59%, and $A{P}_s$ value of 24.12%, with total improvement reaching 6.07 percentage points in $mAP$ and 4.57 percentage points in $A{P}_{et}$ (72.7% relative improvement).

As shown in Figure 16, E0 exhibits weak feature activations with minimal target–background discrimination. Since E0, A0, and D0 share identical baseline configuration without any proposed modules, their visualizations are consistent. E1 with only MRAAF shows similar activation patterns to A3, as both employ the complete MRAAF module. E2 with only CMFE-SRP produces significantly enhanced activation intensity with prominent high-response regions. E3 with only DS-STD generates strong but relatively scattered activations. E4–E6 with pairwise module combinations demonstrate progressively refined activation patterns. The complete E7 achieves the most optimal feature representation, combining concentrated target responses with effective background suppression, validating the synergistic interaction among all three modules.

The comprehensive ablation study reveals the relative importance of the three core challenges in aerial remote sensing small object detection: spatial information loss (CMFE-SRP) is the most critical bottleneck, followed by modal fusion (MRAAF) and sample imbalance (DS-STD). The synergistic gain of the three modules combined (6.07 percentage points) exceeds the sum of their individual contributions, indicating that these three challenges are coupled in remote sensing scenarios and require a systematic solution.

5. Discussion

The experimental results reveal several important insights into the proposed framework. First, the performance gap between RAPT-Net and competing methods is substantially larger on RGBT-Tiny (10.3% relative improvement) compared to VEDAI (4.3%), which can be attributed to the different target scale distributions—RGBT-Tiny contains 81.5% targets smaller than 16 × 16 pixels, where the benefits of reliability-aware fusion and spatial preservation become more pronounced. Second, the ablation studies demonstrate that CMFE-SRP contributes the largest individual improvement, followed by MRAAF and DS-STD, aligning with our analysis that spatial information loss represents the most fundamental bottleneck for tiny target detection. Third, the synergistic gain of the complete framework (6.07 percentage point mAP) exceeds the arithmetic sum of individual module contributions, providing empirical evidence for the coupled nature of the three challenges and validating our systematic design philosophy. Unlike existing attention-based fusion methods that learn data-driven feature importance weights, MRAAF explicitly models scene-dependent modality reliability driven by physical imaging conditions, offering more interpretable fusion behavior. Similarly, rather than attempting to recover lost spatial details through learned upsampling as in super-resolution approaches, CMFE-SRP prioritizes preservation of existing spatial information, which proves more effective for extremely tiny targets.

Despite the demonstrated improvements, several limitations warrant acknowledgment. The current implementation requires co-registered RGB-IR image pairs, which may not always be available in operational scenarios; future work could explore fusion strategies robust to moderate misregistration. While RAPT-Net achieves competitive inference speed, the 37.8 M parameter count may pose challenges for deployment on resource-constrained edge platforms such as onboard UAV processors, motivating lightweight architecture design through knowledge distillation or quantization techniques. The current evaluation is limited to visible–infrared modality combinations, and extension to other remote sensing modalities, such as SAR or hyperspectral imagery, could potentially enhance detection robustness but would require modality-specific adaptations. Nevertheless, the demonstrated cross-platform generalization capability and the modular architecture design provide practical value for operational Earth observation applications, enabling incremental deployment pathways where organizations can adopt individual components independently before full system integration.

6. Conclusions

This paper proposes RAPT-Net, an adaptive multimodal small object detection framework addressing three coupled challenges in aerial remote sensing. The MRAAF module achieves scene-adaptive modality integration through two-stage progressive fusion with reliability-guided feedback. The CMFE-SRP module employs hierarchy-specific processing to balance spatial preservation and semantic enhancement. The DS-STD strategy establishes spatial tolerance mechanisms to accommodate annotation physical ambiguity, increasing positive sample coverage by 4×.

Comprehensive experiments on VEDAI and RGBT-Tiny datasets demonstrate that RAPT-Net achieves mAP values of 62.22% and 18.52%, respectively, improving existing state-of-the-art methods by 4.3% and 10.3%, with a 17.3% improvement on extremely tiny targets (<8 × 8 pixels). The framework maintains practical inference speeds (28.95–58.77 FPS) while demonstrating cross-platform adaptability without architectural modification.

Future work will focus on lightweight architecture design for edge computing platforms and extension to additional remote sensing modalities, including SAR and hyperspectral imagery.