HSFANet: Hierarchical Scale-Sensitive Feature Aggregation Network for Small Object Detection in UAV Aerial Images

Abstract

Small object detection in aerial images, particularly from Unmanned Aerial Vehicle (UAV) platforms, remains a significant challenge due to limited object resolution, dense scenes, and background interference. However, existing small object detectors often overlook making full use of hierarchical features and inevitably introduce noise interference because of hierarchical upsampling operations, and commonly used loss metrics lack sensitivity to scale information; these two issues jointly lead to performance deterioration. To address these issues, we propose Hierarchical Scale-Sensitive Feature Aggregation Network (HSFANet), a novel framework that conducts robust cross-layer feature interaction to perceive the small objects’ position information in hierarchical feature pyramids and enforces the model to balance the multi-scale prediction heads for accurate instances localization. HSFANet introduces a Dynamic Position Aggregation (DPA) module to explicitly enhance the object area in both shallow and deep layers, which is capable of exploiting the complementarily salient representation of the small objects. Additionally, an efficient Scale-Sensitive Loss (SSL) is proposed to balance the small object detection outputs in hierarchical prediction heads, thereby effectively improving the performance of small object detection. Extensive experiments on two challenging UAV benchmarks, VisDrone and UAVDT, demonstrate that HSFANet achieves state-of-the-art (SOTA) results, with a 1.3% gain in overall average precision (AP) and a notable 2.2% improvement in AP for small objects on VisDrone. On UAVDT, HSFANet outperforms previous methods by 0.3% in overall AP and 16.7% in small object AP. These results highlight the effectiveness of HSFANet in enhancing small object detection performance in complex aerial imagery, making it well suited for practical UAV-based applications.

1. Introduction

Object detection is a fundamental task in computer vision with numerous real-world applications [1,2,3,4,5,6,7,8,9,10,11,12,13], such as intelligent video surveillance, autonomous driving, and medical image analysis. This task focuses on identifying and localizing instances of semantic objects within an image, which plays a pivotal role in enabling machines to perceive and interact with their surroundings. Among the various detection scenarios, object detection in Unmanned Aerial Vehicle (UAV) aerial images is garnering growing attention in recent years due to the swift advancement of drone technology and the increasing demand for aerial scene understanding [14,15,16]. However, this task is inherently more demanding than traditional object detection tasks [17,18]. As shown in Figure 1, this is primarily because UAV-based aerial images are often captured from high altitudes with wide fields of view, resulting in a multitude of small-scale objects that are densely distributed and easily conflated with background textures [19,20]. Moreover, during the process of convolution and pooling in standard convolutional neural networks (CNNs), the features of these small objects are frequently weakened, blurred, or even lost due to resolution reduction, making it exceedingly difficult for detectors to accurately recognize and localize them. Consequently, how to improve the detection performance for small objects in aerial images remains a foundational yet unresolved problem in the computer vision community, and it has become an active research focus in recent years [21,22,23].

To tackle these unique challenges in aerial image object detection, researchers have made considerable efforts in the fusion of small object features. A series of advanced detection frameworks, such as UFPMP-Det [24], QueryDet [25], and HRDNet [26], have achieved marked improvements by introducing multi-level feature enhancements or adopting transformer-based query mechanisms. These models have shown encouraging results on public benchmarks and have extended the limits of detection accuracy. Nevertheless, many of them still are deficient in fully capturing the complex characteristics of small targets, particularly in the context of aerial scenes with complex backgrounds and scale variations. A prevalent limitation is that these detectors often fail to conduct effective hierarchical feature interactions across different semantic layers, resulting in inadequate fine-grained representation of small-scale instances. Additionally, the widespread use of traditional Feature Pyramid Networks (FPNs) in these detectors typically involves multiple stages of downsampling and upsampling to align feature resolutions. Although this hierarchical fusion strategy enables coarse-to-fine detection, it also introduces considerable noise and information redundancy, especially when dealing with low-resolution small objects. As discussed in [27], such noise interference has a disproportionately adverse effect on small object detection, making precise localization more difficult and diminishing overall performance.

In addition, researchers have proposed a series of loss functions specifically designed to improve the performance of small object detection. Furthermore, traditional small object detection methods typically employ IOU-based [28,29,30] metrics or Gaussian distribution distance measures [31,32] as regression losses. These metrics inherently exhibit limitations regarding the relevant target scale range: IOU-based approaches are generally more suitable for medium to large-sized objects, whereas Gaussian distribution distance measures are better suited for extremely small objects. Traditional small object detectors are deficient in scale-awareness and thus cannot utilize appropriate regression losses customized for objects of varying scales. Given that target scale distribution in UAV aerial imagery is highly imbalanced, this shortcoming results in suboptimal detection performance in UAV aerial scenarios.

In this paper, we introduce a novel small object detection framework tailored for aerial images termed HSFANet. The proposed HSFANet seeks to address the limitations of existing methods by promoting stronger cross-layer feature interactions and explicitly improving the model’s sensitivity to small-scale targets through a new scale-aware supervision strategy. Specifically, we first devise a DPA module that is integrated into a standard CSPDarknet backbone to facilitate effective spatial-aware fusion between shallow and deep features. The DPA module adaptively upsamples and aligns the spatial structures of low-level features and combines them with high-level semantic features, thereby enhancing the localization cues of small targets across multiple resolutions. This hierarchical aggregation not only retains fine-grained spatial details from early layers but also utilizes semantic context from deeper layers, mitigating the adverse impact of feature noise and enabling the detector to more accurately detect and identify small objects. Furthermore, we introduce an effective SSL that supervises predictions across multiple output heads. The SSL dynamically readjusts the learning process to highlight difficult and small-scale samples, prompting the model to focus more on these hard examples. The integration of DPA and SSL leads to a more resilient and fine-grained representation of small objects, ultimately resulting in better detection performance.

Extensive experiments are conducted on two widely used and challenging aerial object detection benchmarks, namely VisDrone and UAVDT, to verify the effectiveness of our proposed approach. The results demonstrate that HSFANet regularly surpasses existing state-of-the-art detectors in terms of small object detection accuracy and localization precision, exhibiting its robustness and generalization capability under varied aerial imaging scenarios. Our proposed method not only advances the performance frontier but also offers a practical and efficient solution for real-world UAV-based applications.

In summary, the main contributions of this paper are presented as follows:

• We introduce a novel DPA module. This module enables resilient and flexible cross-layer feature interaction by fusing spatial and semantic features from different levels and mitigating the effect of noise interference introduced during feature compression, allowing the detector to effectively discern small object position information within hierarchical feature pyramids for precise localization.
• We devise an effective SSL function that supervises predictions across multiple output scales. This loss function prompts the model to dedicate more learning resources to small and difficult objects by adaptively reweighting the losses, thus improving detection robustness.
• We propose a HSFANet and conduct comprehensive experiments on two public aerial object detection datasets, VisDrone and UAVDT, to verify the effectiveness and generalization capability of our method. The results clearly demonstrate that HSFANet considerably enhances small object detection performance and attains new state-of-the-art results in this challenging domain.

The organization of this paper is as follows: Section 2 reviews related research work in the field of small object detection. Section 3 provides a detailed explanation of the proposed HSFANet. Section 4 presents experimental validations demonstrating the effectiveness of HSFANet. Finally, Section 5 summarizes the findings of this study.

2. Related Work

2.1. Small Object Detection in Aerial Images

Small object detection in aerial imagery [33] has become a major research focus due to its broad range of real-world applications, including traffic monitoring, urban planning, and disaster response. Dissimilar to natural scene images, aerial images captured by Unmanned Aerial Vehicles (UAVs) or satellites often exhibit ultra-high resolutions, complex backgrounds, and densely packed small targets [34]. These characteristics introduce distinct challenges [19], such as considerable scale variation, cluttered scenes, and extreme class imbalance. In particular, small targets usually occupy only a few pixels and are susceptible to feature loss during convolution and pooling operations, making accurate localization and classification challenging [35].

To address these issues, multiple strategies have been explored. Some approaches introduce scale-specific branches or resolution-preserving structures to better preserve small object features [36]. Others incorporate contextual modeling or global semantic reasoning to enrich the representations of low-resolution instances [37,38]. Despite these efforts, small object detection in aerial images remains impeded by the low signal-to-noise ratio, especially under occlusion, illumination variation, or atmospheric interference. Moreover, traditional loss functions and post-processing strategies such as IoU-based regression often are unable to offer sufficient supervision or suppression behavior for small instances [39]. Therefore, formulating effective architectures and learning objectives that are robust to noise and scale variation is critical for improving performance in this domain.

Although the aforementioned works have improved small object detection to some extent, they often disregard cross-level feature structures. The hierarchical upsampling operations unavoidably introduce noise interference, and frequently used loss metrics lack scale-awareness. As a result, detection performance in UAV aerial scenarios remains suboptimal. In this work, we fully leverage the hierarchical feature structure and adopt dynamic upsampling to mitigate sampling noise. Moreover, we introduce a scale-aware function into the loss to balance the regression loss across objects of different scales.

2.2. Hierarchical Feature Representation for Small Objects

Small object detection is intrinsically difficult due to the loss of detailed spatial information and the prevalence of background noise in high-resolution images [40]. Feature Pyramid Networks (FPNs) and their variants have demonstrated considerable success in multi-scale object detection by constructing hierarchical feature maps [41]. However, the intrinsic trade-off between spatial resolution and semantic richness still presents difficulties for accurately detecting small objects [42]. High-level features offer stronger semantics but are deficient in spatial precision, while low-level features preserve spatial details but are semantically weak [43]. This semantic gap often leads to imprecise or missed detections of small objects.

To address this disparity, a range of enhanced FPN structures have been introduced. Shi et al. [44] proposed the High-frequency and Spatial-aware FPN (HS-FPN), which utilizes high-pass filters to extract high-frequency signals and incorporates spatial dependency modules to capture spatial cues for small object enhancement. Yang et al. [45] introduced the Asymptotic FPN (AFPN), which merges non-adjacent feature levels to lessen semantic inconsistency and employs adaptive spatial fusion to decrease multi-object interference. Guo et al. [46] created AugFPN with consistency supervision, residual enhancement, and soft RoI selection to improve multi-scale representations. Additionally, Deng et al. [47] proposed EFPN, which adds a super-high-resolution pyramid to retain finer detail and enhances feature propagation across scales.

The aforementioned methods have eased the difficulty of small object feature representation to a degree; most of them still depend on sequential feature fusion, which often results in new problems such as gradual degradation of feature quality across layers. Conversely, our approach further enhances the representation of small objects by utilizing cross-level feature aggregation and a feature extraction module specifically designed to better capture small object characteristics.

2.3. Loss Function Design for Small Object Detection

The optimization process of object detection models is heavily dependent on the choice of loss function. For small objects, traditional IoU-based losses such as GIoU [28], DIoU, and CIoU [29] often are unable to offer robust gradients due to the minimal overlap between predictions and ground truth boxes. This results in less than optimal localization performance, particularly for minuscule instances.

To tackle this shortcoming, a number of scale-aware and distribution-based loss functions have been introduced. $\alpha$-IoU loss [30] includes an exponential term to boost sensitivity in low-IoU regions, which is especially advantageous for small object alignment. Meanwhile, distribution-based losses such as the Normalized Wasserstein Distance (NWD) [31] and Gaussian-based Receptive Field Label Assignment (RFLA) [32] quantify the spatial distribution mismatch between predicted and ground truth boxes, offering improved localization signals for small targets. These methods represent spatial uncertainty and geometric variance more efficiently than hard-threshold IoU metrics. Additionally, loss designs that jointly optimize classification and regression, such as Generalized Focal Loss (GFL) [48] and Focal Loss [49], help alleviate sample imbalance and enhance detection robustness.

While the preceding methods have improved the effectiveness of small object loss metrics to some extent, the highly imbalanced target scale distribution in UAV aerial scenarios often surpasses the relevant scale range of these single, static metrics. To resolve this problem, we design a loss function that effectively discerns target scales, thereby alleviating the problems caused by scale imbalance.

3. Methods

In this section, we provide a comprehensive and detailed description of the proposed HSFANet architecture. The overall structure and design of HSFANet are illustrated clearly in Figure 2, which helps to visualize the network’s key components and their interconnections. Initially, the network employs a well-established and widely used backbone, CSPDarknet, as the foundational feature extractor. This backbone is responsible for generating multi-scale feature pyramids that capture hierarchical information from the input images at different resolutions and semantic levels. Following the backbone, we introduce a novel DPA module. This module is specifically designed to effectively fuse and integrate high-level semantic information, which carries rich contextual cues, with low-level detailed feature information that contains fine-grained spatial details.

The purpose of the DPA module is to enable a more robust and adaptive fusion of features across multiple scales and layers, enhancing the representation capability of the network for objects of varying sizes. To further address the common problem of feature information loss that often occurs during upsampling operations in HSFANet, a dynamic upsampling operator is carefully incorporated. This operator adaptively adjusts the upsampling process, reducing the distortion and degradation of feature details that can negatively impact detection accuracy, especially for small objects. Moreover, in order to boost the detection performance on small-scale targets, a SSL function is integrated into the overall loss computation. This specialized loss formulation enhances the network’s sensitivity to small objects by dynamically emphasizing their importance during training, thereby significantly improving detection precision and robustness in challenging scenarios.

3.1. Dynamic Position Aggregation

In UAV target detection tasks, small targets constitute a significantly high proportion of the overall objects present in aerial scenes. However, due to their inherently limited pixel count and insufficient distinctive feature information, small targets are much more vulnerable to severe feature degradation and information loss during the downsampling stages of convolutional neural networks. Compared to medium and large targets, which inherently contain richer and more detailed visual semantics, small targets tend to be compressed excessively after multiple downsampling operations, sometimes shrinking to pixel-level or even sub-pixel-level representations. This extreme compression causes a drastic reduction in useful discriminative information, rendering small target detection particularly challenging when relying on high-level feature maps that tend to focus on coarse semantic cues.

To effectively address this critical challenge, this paper proposes a novel DPA module, specifically designed to explicitly enhance and emphasize the object-related regions across both shallow and deep network layers. The DPA module leverages complementary salient representations by dynamically aggregating position-aware feature information that is crucial for the accurate localization and identification of small targets. In particular, DPA extracts and preserves effective feature details from lower-level feature maps, which usually contain richer spatial and texture information. This extraction helps mitigate the otherwise inevitable negative effects of feature loss during the feature extraction and downsampling processes, thus substantially improving the network’s ability to detect small-scale objects.

Moreover, the DPA module employs a cross-level feature fusion architecture, which plays a pivotal role in alleviating the semantic gap that often arises from continuous hierarchical fusion of features at multiple levels. By effectively integrating cross-level information, the module introduces valuable low-level detailed features into the fusion process, providing crucial fine-grained guidance that benefits small object detection. This integration enables the network to better combine spatial and positional information, which are key to accurately delineating small objects that are otherwise prone to being overlooked. The DPA module thereby organically fuses high-level global contextual information with low-level local detail cues, further strengthening the overall representational capacity of the network for small target detection.

In addition to the DPA module, we design the Convolution to Feature with Mish (C2FM) module and the Convolution to Feature with Mish Compact Inverted Block (C2FMCIB) to further enhance the sensitivity of the DPA towards subtle variations in feature maps. The C2FM and C2FMCIB modules adaptively modulate channel-wise features by focusing on informative feature channels and suppressing irrelevant or noisy ones, which significantly improves the discrimination power of the aggregated features. This complementary design allows the DPA module to better capture minute changes and subtle characteristics in feature maps that are often crucial for distinguishing small targets from complex backgrounds or clutter.

The feature fusion process at the ${\mathbf{P}}_2^{\prime }$ in the DPA module is formulated as follows:

$${\mathbf{F}}_3^{\prime }=\mathrm{Cat}\left({\mathbf{F}}_3,\mathrm{Dysample}\left({\mathbf{F}}_5\right)\right)$$

(1)

$${\mathbf{P}}_2^{\prime }=\mathrm{Cat}\left({\mathbf{F}}_2,\mathrm{Dysample}\left({\mathbf{F}}_3^{\prime }\right)\right)$$

(2)

where $\mathrm{Cat}\left(·\right)$ denotes the concatenation operations. $\mathrm{Dysample}$ denotes the dynamic upsample operations [50]. And ${\mathbf{P}}_2^{\prime }$ denotes the output feature map corresponding to the second layer in the DPA module.

The feature fusion process at the ${\mathbf{P}}_3^{\prime }$ in the DPA module is formulated as follows:

$${\widehat{\mathbf{F}}}_3=\mathrm{Downsample}\left({\mathbf{P}}_2^{\prime }\right)$$

(3)

$${\mathbf{P}}_3^{\prime }=\mathrm{Cat}\left({\mathbf{F}}_3^{\prime },{\widehat{\mathbf{F}}}_3\right)$$

(4)

where $\mathrm{Downsample}\left(·\right)$ denotes the downsampling operation is implemented using CBS. And ${\mathbf{P}}_3^{\prime }$ denotes the output feature map corresponding to the third layer in the DPA module.

The feature fusion process at the ${\mathbf{P}}_5^{\prime }$ in the DPA module is formulated as follows:

$${\widehat{\mathbf{F}}}_5=\mathrm{Downsample}\left(\mathrm{Cat}\left({\mathbf{F}}_3^{\prime },{\widehat{\mathbf{F}}}_3\right)\right)$$

(5)

$${\mathbf{P}}_5^{\prime }=\mathrm{Cat}({\mathbf{F}}_5,{\widehat{\mathbf{F}}}_5)$$

(6)

where $\mathrm{Downsample}\left(·\right)$ denotes the downsampling operation is implemented using SCDown [51]. And ${\mathbf{P}}_5^{\prime }$ denotes the output feature map corresponding to the fifth layer in the DPA module.

In the DPA module, high-magnification upsampling is necessary to align and match the spatial dimensions of low-level feature maps, ensuring effective cross-level feature fusion. However, commonly used traditional upsampling methods such as nearest-neighbor interpolation and bilinear interpolation tend to introduce undesirable sampling noise during this process. This noise can lead to distortions in the reconstructed feature maps, particularly impacting the delicate and limited feature representations of small targets. Such perturbations may degrade the network’s ability to accurately identify and localize small objects, as the added noise can obscure subtle spatial details crucial for small target detection.

To effectively alleviate the negative impact caused by high-magnification upsampling noise, this paper proposes the use of a dynamic upsampling operator within the DPA module. Unlike static interpolation methods that rely on fixed kernels and uniform sampling patterns, dynamic upsampling adaptively learns sampling offsets and weights based on the local feature context. This learned approach enables more precise and content-aware feature reconstruction, significantly reducing the introduction of sampling artifacts and preserving the integrity of small target features. By dynamically adjusting the upsampling behavior to the underlying feature structure, the network achieves a more faithful and noise-robust spatial alignment, thereby enhancing the overall detection accuracy for small-scale objects.

In Figure 3, the weights are computed using 1 × 1 convolutional layers (’linear1’/’linear2’), with input channels equal to that of the corresponding FPN feature map, output channels set to $2\times g\times {\mathrm{scale}}^2$, and groups = 1. Here, g denotes the number of groups for dynamic computation (set to 4 in our experiments), while $sH$ and $sW$ represent the height and width of the local feature patch, corresponding to the spatial size of the input FPN feature map. The initial positional offsets are generated using the specified ‘scale’ factor. These settings are consistent across all datasets and are chosen to balance computational cost and representational power.

And the process of dynamic upsampling is shown in Figure 3. The use of cross-hierarchical dynamic upsampling effectively enhances the overall detection performance of HSFANet. The computation process of dynamic upsampling [50] is as follows in Equation (7):

$${\mathcal{X}}^{\prime }=\mathrm{gridsample}\left(\mathcal{X},\mathcal{S}\right),$$

(7)

where $\mathcal{X}$ and ${\mathcal{X}}^{\prime }$ represent the input feature map and upsampled feature map. Additionally, $\mathcal{S}$ is the sampling set in the equation that can be represented by Equation (8):

$$\mathcal{S}=\mathcal{G}+\mathcal{O},$$

(8)

then the sampling set $\mathcal{S}$ is the sum of the offset $\mathcal{O}$ and the original sampling grid $\mathcal{G}$. And the $\mathcal{O}$ is defined as shown in Equation (9):

$$\mathcal{O}=\mathrm{sigmoid}\left({\mathrm{linear}}_1\left(\mathcal{X}\right)\right)·{\mathrm{linear}}_2\left(\mathcal{X}\right).$$

(9)

3.2. Scale-Sensitive Loss

The CIoU loss demonstrates strong performance when applied to medium and large objects, effectively guiding the regression of bounding boxes in these cases. However, small objects exhibit much higher sensitivity to slight perturbations in their bounding box coordinates due to their limited pixel size and fewer distinctive features. Consequently, the CIoU loss, while suitable for larger objects, tends to be less effective for accurately regressing the positions of small objects. This limitation poses minimal impact in conventional detection scenarios dominated by medium and large targets, where minor localization errors are often negligible relative to object size.

In contrast, in UAV aerial imagery scenarios, small objects constitute a substantially higher proportion of the detected targets. In such contexts, even slight perturbations or inaccuracies in bounding box regression can lead to significant performance degradation, as these small deviations represent a larger fraction of the object’s spatial extent. This sensitivity to bounding box fluctuations makes accurate localization for small objects a critical challenge, one that traditional CIoU loss struggle to overcome.

To address this issue, we introduce the Normalized Wasserstein Distance (NWD) metric [31] as a complementary measure for bounding box regression loss. NWD assesses the similarity between predicted and ground truth bounding boxes by considering their spatial distributions as probability measures, making it inherently more robust to minor positional shifts and scale variations typical in small object detection. We then design a combined regression loss function that integrates NWD with the CIoU loss [52], leveraging the strengths of both metrics.

Furthermore, a scale-aware adaptive weighting function $\lambda \left(s\right)$ is introduced to dynamically balance the relative contribution of the NWD and CIoU components within the combined loss. This adaptive weighting mechanism allows the model to place more emphasis on NWD when learning from small objects, effectively increasing the proportion of positive samples influenced by the SSL during training. As a result, the model can more effectively learn the subtle features and precise localization cues of small targets, leading to improved regression accuracy and overall detection performance in challenging UAV aerial imagery environments.

According to the definition of the Wasserstein distance in optimal transport theory [53], the Gaussian distributions ${\mathcal{N}}_a$ and ${\mathcal{N}}_b$ modeled by the bounding boxes $\mathbf{a}=\left(c{x}_a,c{y}_a,{\displaystyle \frac{w_a}{2}},{\displaystyle \frac{h_a}{2}}\right)$ and $\mathbf{b}=\left(c{x}_b,c{y}_b,{\displaystyle \frac{w_b}{2}},{\displaystyle \frac{h_b}{2}}\right)$, the Wasserstein distance between two distributions can be expressed as

$$W_2^2\left({\mathcal{N}}_a,{\mathcal{N}}_b\right)={∥\left({\mathbf{a}}^{\mathrm{T}},{\mathbf{b}}^{\mathrm{T}}\right)∥}_2^2,$$

(10)

where ${∥·∥}_F$ is the Frobenius norm.

Since the range of $W_2^2\left({\mathcal{N}}_a,{\mathcal{N}}_b\right)$ is $[0,+\infty ]$, it cannot be directly used as a similarity measure for label assignment. To align its range with the IoU range, which is between 0 and 1, we choose to use an exponential nonlinear transformation function to map the Gaussian Wasserstein distance to another space, thus normalizing its range to $(0,1]$. This gives us the NWD, as shown in Equation (11):

$$\mathrm{NWD}\left({\mathcal{N}}_a,{\mathcal{N}}_b\right)=exp\left(-{\displaystyle \frac{\sqrt{W_2^2\left({\mathcal{N}}_a,{\mathcal{N}}_b\right)}}{C}}\right),$$

(11)

where C is a constant.

Therefore, the loss function based on NWD can be defined as shown in Equation (12):

$${\mathcal{L}}_{nwd}=1-\mathrm{NWD}\left({\mathcal{N}}_p,{\mathcal{N}}_g\right).$$

(12)

The definition of CIoU [52] is shown in Equation (13):

$${\mathcal{L}}_{CIoU}=1-IoU+{\displaystyle \frac{{\rho }^2\left(b,b^g\right)}{c^2}}+\alpha v,$$

(13)

where $\alpha$ is defined as Equation (14):

$$\alpha ={\displaystyle \frac{v}{\left(1-IoU\right)+v}},$$

(14)

and v is denoted as Equation (15):

$$v={\displaystyle \frac{4}{{\pi }^2}}{\left(arctan{\displaystyle \frac{w^g}{h^g}}-arctan{\displaystyle \frac{w^p}{h^p}}\right)}^2.$$

(15)

The regression loss function in this paper is defined as shown in Equation (16):

$$\mathcal{L}=\lambda \left(s\right){\mathcal{L}}_{CIoU}+\left(1-\lambda \left(s\right)\right){\mathcal{L}}_{nwd},$$

(16)

where $\lambda \left(s\right)$ is a scale-aware adaptive weighting function that adjusts the weight between the CIoU and NWD losses. To adapt the loss function to object size, we design a smooth task-adaptive weighting function defined as

$$\lambda \left(s\right)={\displaystyle \frac{1}{1+e^{-\alpha (s-s_0)}}}$$

(17)

where $s=\sqrt{wh}$ denotes the object size computed from the width w and height h of the bounding box. The threshold $s_0$ serves as the boundary between small and large objects, and $\alpha$ controls the sharpness of the transition. A larger $\alpha$ approximates a step function while maintaining gradient continuity, which facilitates stable training.

In Equation (17), $s=\sqrt{wh}$ denotes the absolute pixel-based size of the object, where w and h are the width and height of the bounding box in image pixels. It is not normalized relative to the input resolution or FPN level. While our method leverages multi-scale features from the FPN, the Scale-Sensitive Loss (SSL) automatically handles scale variations across pyramid levels, allowing small objects to receive higher relative weighting without manually adjusting for different feature scales.

4. Experiments

This section first introduces the benchmark datasets, evaluation metrics, and specific implementation details used in our experiments. Then, we conduct a series of ablation studies to thoroughly evaluate the individual contributions and effectiveness of the proposed components, including the feature aggregation mechanism, backbone architecture, and loss function design. To further verify the effectiveness of the proposed method, we perform comprehensive comparisons with state-of-the-art object detection approaches. Additionally, to assess the generalization capability of our algorithm, we conduct experiments across different datasets. The results, presented from both quantitative evaluation metrics and qualitative visualizations, comprehensively demonstrate the superior performance of our method in complex aerial object detection scenarios.

4.1. Datasets

The experiments in this paper use two aerial image datasets: VisDrone and UAVDT.

The VisDrone [54] dataset comprises videos and images captured by various drone platforms operating across fourteen different cities, covering a wide range of real-world scenarios, such as urban streets, residential areas, and public squares. Specifically, the VisDrone subset includes a total of 10,209 high-resolution static images, among which 6471 images are designated for training, 548 for validation, and 3190 for testing. The dataset spans 10 object categories, including pedestrians, vehicles, bicycles, and other common targets observed from aerial perspectives. In Figure 4, the distributions of instance counts across different categories and object sizes are presented. The image resolutions vary significantly, ranging from 960 × 540 to 2000 × 1500 pixels, reflecting the diversity in scene complexity and altitude.

The UAVDT [55] dataset comprises a total of 38,327 images with an average resolution of 1080 × 540 pixels. It primarily focuses on vehicle detection tasks and includes three object categories: car, bus, and truck.

Figure 5 presents an overview of the number of instances per category and the distribution of objects across different size ranges. The dataset is divided into 23,258 images for training and 15,069 images for testing. Captured from real-world urban traffic surveillance scenes under various weather and lighting conditions, UAVDT presents considerable challenges such as small object sizes, frequent occlusions, and dynamic backgrounds. These characteristics make it a valuable benchmark for evaluating the robustness and generalization ability of object detection algorithms, especially in aerial scenarios.

4.2. Evaluation Metrics

We use AP, ${\mathrm{AP}}_{50}$, and ${\mathrm{AP}}_{75}$ as evaluation metrics. AP represents the average precision, while ${\mathrm{AP}}_{50}$ and ${\mathrm{AP}}_{75}$ denote the average precision calculated at IoU thresholds of 0.5 and 0.75, respectively. To assess detection performance for objects of different sizes, we use ${\mathrm{AP}}_s$, ${\mathrm{AP}}_m$, and ${\mathrm{AP}}_l$ to evaluate the average precision for small, medium, and large objects, respectively. Additionally, we provide the average precision for each object category to evaluate the model’s detection performance for individual classes. In order to gain deeper insights into the model’s performance across various object categories within the dataset, we incorporate precision (P) and recall (R) to evaluate the detection effectiveness. True Positives (TPs) represent the cases where the model correctly predicts a positive instance. False Negatives (FNs) occur when the actual label is positive, but the model incorrectly predicts it as negative. False Positives (FPs) refer to instances where the actual label is negative, but the model mistakenly classifies it as positive. True Negatives (TNs) correspond to situations where both the actual label and the model’s prediction are negative. The formulas for computing precision and recall are given as follows:

$$\mathrm{P}={\displaystyle \frac{TP}{TP+FP}}$$

(18)

$$\mathrm{R}={\displaystyle \frac{TP}{TP+FN}}$$

(19)

4.3. Implementation Details

We conducted the HSFANet experiments using 8 NVIDIA GeForce GTX 3090 GPUs (NVIDIA Corporation, Santa Clara, CA, USA) with 24 GB of memory. The experiments were implemented using Python version 3.9.19, with PyTorch 2.0.1 as the deep learning framework, and accelerated using CUDA 11.7.0. The specific configurations of the experimental environment are shown in

Table 1. Specific configuration of the experimental environment.

Environment	Configuration Information
Operating system	Ubuntu 22.04.2
CPU	AMD EPYC 7773X 64-Core Processor
CPU number	2
GPU	NVIDIA Corporation GA102 [GeForce RTX 3090]
GPU memory size	24G
GPU number	8
GPU calculate platform	CUDA 11.7.0
Python version	Python 3.9.19
Deep learning framework	PyTorch 2.0.1

. The input images were RGB images with dimensions of 1024 × 1024 pixels. The model was trained using the Adam optimizer with a batch size of 16 for 200 epochs. We applied a cosine learning rate decay schedule, starting from 0.001 and decreasing to $1\times {10}^{-6}$, with a linear warm-up for the first 3 epochs. Common data augmentation techniques were employed, including random horizontal flipping, random scaling, random cropping, color jittering, and mosaic augmentation, which improve generalization, particularly for small objects. Our method follows an anchor-free design with SimOTA label assignment, standard NMS (IoU threshold 0.5), and a confidence score threshold of 0.2. No test-time augmentation (TTA) was used, ensuring fair comparison across methods. These settings are fully specified to facilitate reproducibility of our experiments. The key hyperparameters of the proposed SSL were set as follows: $C=1.6$ (Equation (11)), $\alpha =13$, and $s_0=32$ (Equation (17)). These values were applied consistently across both VisDrone and UAVDT datasets, without dataset-specific tuning, as they yielded stable performance in all experiments. Reporting these hyperparameters explicitly ensures reproducibility and facilitates fair comparisons.

For the results of other comparison methods, we follow the numbers directly cited from their original papers, since official codes or complete training configurations are not always publicly available. As a result, the input resolution, data augmentation strategies, and training schedules are consistent with those reported in the respective works.

We acknowledge that differences in the original experimental setups may lead to minor inconsistencies across methods. However, all cited approaches are widely recognized and evaluated on the same benchmarks (VisDrone and UAVDT), which ensures the fairness and validity of the comparison.

4.4. Main Results

VisDrone Results. The performance of the proposed algorithm is comprehensively evaluated and compared against several state-of-the-art related methods on the challenging VisDrone dataset, with results summarized in

Table 2. Performance comparison on VisDrone. Best results are in bold.

Method	Year & Venue	Backbone	AP	${\mathbf{AP}}_{50}$	${\mathbf{AP}}_{75}$	${\mathbf{AP}}_{\mathit{s}}$	${\mathbf{AP}}_{\mathit{m}}$	${\mathbf{AP}}_{\mathit{l}}$
FRCNN [56]	2015 NIPS	ResNeXt101	24.4	47.8	21.8	17.8	34.8	34.3
ClusDet [57]	2019 ICCV	ResNeXt101	32.4	56.2	31.6	-	-	-
DREN [58]	2019 ICCVW	ResNeXt152	30.3	-	-	-	-	-
DMNet [59]	2020 CVPRW	ResNeXt101	29.4	49.3	30.6	21.6	41.0	56.7
CDMNet [60]	2021 ICCVW	ResNeXt101	30.7	51.3	32.0	22.2	42.4	44.7
AMRNet [61]	2020 Arxiv	ResNeXt101	32.1	-	-	23.2	43.9	60.5
GLSAN [62]	2021 TIP	ResNet101	32.5	55.8	30.0	-	-	-
HRDNet [26]	2021 ICME	ResNeXt50+101	35.5	62.0	35.1	-	-	-
FCOS+SAHI [63]	2022 ICIP	-	-	38.5	-	-	-	-
VFNet+SAHI [63]	2022 ICIP	-	-	42.2	-	-	-	-
TOOD+SAHI [63]	2022 ICIP	-	-	43.5	-	-	-	-
QueryDet [25]	2022 CVPR	ResNet50	28.3	48.1	28.8	-	-	-
CEASC [64]	2023 CVPR	ResNet18	28.7	50.7	28.4	-	-	-
UFPMP-Det [24]	2022 AAAI	ResNeXt101	40.1	66.8	41.3	-	-	-
CenterNet [65]	2019 arXiv	Hourglass104	27.8	47.9	27.6	21.3	42.1	49.8
YOLOX [66]	2021 arXiv	CSPv5-M	27.6	47.7	27.5	17.6	41.0	46.1
YOLOv9 [67]	2024 ECCV	GELAN	29.5	49.9	29.4	20.2	41.7	47.8
YOLC [68]	2024 TITS	ResNeXt101	36.3	60.1	37.4	28.9	47.5	51.8
GLSDet [69]	2025 TGRS	CSPv5-M	32.3	54.2	32.8	23.3	42.9	50.2
HSFANet	Ours	CSPDarknet53	41.4	63.2	44.3	31.1	51.5	55.4

Note. Results of other methods are directly cited from their original papers, as official codes or complete training details are not always publicly available. Therefore, the reported input resolution, augmentation, and training settings follow those specified in the respective works. While slight inconsistencies may exist across methods, all results are obtained on the same benchmarks (VisDrone), ensuring a fair and valid comparison.

. We focus on a variety of widely accepted evaluation metrics, including AP, ${\mathrm{AP}}_{50}$, ${\mathrm{AP}}_{75}$, as well as ${\mathrm{AP}}_s$, ${\mathrm{AP}}_m$, and ${\mathrm{AP}}_l$, to provide a detailed and nuanced comparison across different object scales.

As shown in Table 2, our proposed method achieves superior detection results across multiple metrics. Specifically, it surpasses the second-best performing algorithm by 1.3% in overall AP, indicating a consistent improvement in detection accuracy. Furthermore, notable gains are observed in ${\mathrm{AP}}_{75}$, where our method exceeds the runner-up by 3.0%, reflecting its enhanced capability for precise localization. Improvements in scale-specific metrics are also significant: ${\mathrm{AP}}_s$ is increased by 2.2% and ${\mathrm{AP}}_m$ by 4.0%, demonstrating that HSFANet is particularly effective in detecting small and medium-sized objects, which are prevalent in UAV imagery. The visualization of HSFANet’s detection results on the VisDrone dataset is shown in Figure 6.

A key strength of HSFANet lies in its ability to substantially enhance small object detection performance, which is a challenging aspect for many existing detectors. Importantly, our approach achieves these gains without relying on excessively deep or computationally heavy backbone networks. This indicates that HSFANet effectively mitigates feature information loss and narrows the semantic gap between high-level semantic features and low-level detailed feature maps, facilitating more accurate and robust multi-scale feature fusion.

In

Table 3. Performance of HSFANet on various categories of the VisDrone dataset.

Class	Instances	P	R	AP	${\mathbf{AP}}_{50}$	${\mathbf{AP}}_{75}$
pedestrian	8844	73.7	68.7	40.4	73.7	39.2
people	5125	71.0	57.7	29.8	62.0	24.7
bicycle	1287	49.8	47.9	25.5	47.2	23.2
car	14,064	83.7	88.3	70.3	91.4	80.0
van	1975	65.8	62.6	48.8	64.5	56.4
truck	750	65.4	56.4	43.1	59.3	49.2
tricycle	1045	60.5	54.4	34.0	53.6	37.9
awning-tricycle	532	43.4	28.8	20.3	29.9	22.7
bus	251	81.9	72.0	61.2	77.8	70.1
motor	4886	67.3	72.5	40.1	72.5	39.8

, we present the performance of HSFANet across different categories on the VisDrone dataset. It is evident that the categories with relatively low AP are awning-tricycle, bicycle, and tricycle, which are typically composed of densely distributed small targets. This observation highlights that small object detection remains a challenging aspect in object detection tasks. Therefore, improving the detection performance for small objects is of great significance to advancing overall object detection capabilities.

Through both quantitative and qualitative evaluations of HSFANet’s performance, we find that it is highly adapted to UAV aerial scenarios. This contributes to improving the accuracy of object detection and recognition in UAV applications and holds significant importance for enhancing UAVs’ autonomous understanding of ground targets and scenes from high altitudes.

UAVDT Results. We conduct a thorough comparison of the proposed HSFANet algorithm against several recent and relevant methods on the UAVDT dataset, with detailed results presented in

Table 4. Performance comparison on UAVDT. Best results are in bold.

Method	AP	${\mathbf{AP}}_{50}$	${\mathbf{AP}}_{75}$	${\mathbf{AP}}_{\mathit{s}}$	${\mathbf{AP}}_{\mathit{m}}$	${\mathbf{AP}}_{\mathit{l}}$
R-FCN [70]	7.0	17.5	3.9	4.4	14.7	12.1
SSD [71]	9.3	21.4	6.7	7.1	17.1	12.0
FRCNN [56]	5.8	17.4	2.5	3.8	12.3	9.4
FRCNN [56]+FPN	11.0	23.4	8.4	8.1	20.2	26.5
ClusDet [57]	13.7	26.5	12.5	9.1	25.1	31.2
DMNet [59]	14.7	24.6	16.3	9.3	26.2	35.2
CDMNet [60]	16.8	29.1	18.5	11.9	29.0	15.7
DREN [58]	17.1	-	-	-	-	-
AMRNet [61]	18.2	30.4	19.8	10.3	31.3	33.5
GLSAN [62]	17.1	28.3	18.8	-	-	-
CEASC [64]	17.1	30.9	17.8	-	-	-
CenterNet [65]	13.2	26.7	11.8	7.8	26.6	13.9
UFPMP-Det [24]	24.6	38.7	28.0	-	-	-
YOLOX [66]	15.8	27.7	16.3	10	27.3	24.8
YOLOV9 [67]	16.1	28.1	16.5	10.6	27.9	25.3
YOLC [68]	19.3	30.9	20.1	10.9	32.2	35.5
GLSDet [69]	18.3	29.8	17.6	11.8	29.3	26.8
HSFANet (Ours)	24.9	40.7	28.0	28.6	26.7	20.0

Note. Results of other methods are directly cited from their original papers, as official codes or complete training details are not always publicly available. Therefore, the reported input resolution, augmentation, and training settings follow those specified in the respective works. While slight inconsistencies may exist across methods, all results are obtained on the same benchmarks (UAVDT), ensuring a fair and valid comparison.

. To maintain a rigorous and consistent evaluation protocol, we adopt the widely used standard from the COCO dataset for object size definitions: small objects are defined as having an area less than 32 × 32 pixels, medium objects as having an area between 32 × 32 and greater than or equal to 96 × 96 pixels, and large objects as having an area 96 × 96 pixels. Our algorithm consistently outperforms the second best performing approach by 0.3% in overall AP, 2% in ${\mathrm{AP}}_{50}$, and an impressive 16.7% increase in ${\mathrm{AP}}_s$. The most significant improvement is observed in the detection of small objects, which highlights the effectiveness of the HSFANet for enhancing small target representations and overcoming the inherent challenges posed by limited pixel information and feature loss.

HSFANet demonstrates significant advantages in object detection tasks from UAV perspectives, achieving notably superior performance on the UAVDT benchmark dataset. The network effectively addresses inherent challenges in UAV-captured imagery—such as severe scale variations (ranging from extremely small to relatively large objects), dense object distributions, and complex background clutter—through its unique Hierarchical Scale-aware Feature Aggregation mechanism. Evaluated against the UAVDT benchmark, HSFANet exhibits outstanding results across key metrics significantly outperforming numerous mainstream object detection models. These results robustly validate the efficacy of HSFANet’s design in accurately localizing and identifying diverse targets within UAV aerial images. Consequently, HSFANet provides a powerful and reliable technical foundation for practical UAV-based applications such as traffic monitoring and security surveillance. The visualization of HSFANet’s detection results on the UAVDT dataset is shown in Figure 7.

Furthermore, the superior results obtained on the UAVDT dataset demonstrate not only the excellent performance of the proposed architecture but also its strong generalization ability across different aerial imagery scenarios. The consistent improvements in small object detection indicate that HSFANet can effectively handle the diverse and complex conditions typical of UAV-captured datasets, making it a promising solution for real-world aerial surveillance and monitoring applications.

4.5. Ablation Studies

To systematically verify the effectiveness of each component in our proposed HSFANet architecture, we conducted a comprehensive ablation study on the VisDrone dataset. The experimental results, detailed in

Table 5. Ablation study on the VisDrone dataset. ’Baseline’ denotes the baseline model, ’DPA’ stands for Dynamic Position Aggregation, and ’SSL’ represents Scale-Sensitive Loss. A check mark (✓) indicates that the corresponding feature or configuration is used. Best results are in bold.

Baseline	DPA	SSL	AP	${\mathbf{AP}}_{50}$	${\mathbf{AP}}_{75}$	${\mathbf{AP}}_{\mathit{s}}$	${\mathbf{AP}}_{\mathit{m}}$	${\mathbf{AP}}_{\mathit{l}}$
✓	-	-	36.4	56.8	34.8	25.1	47.7	58.8
✓	✓	-	41.3	63.0	43.2	29.9	50.2	58.9
✓	-	✓	38.4	59.4	40.8	26.8	50.4	54.3
✓	✓	✓	41.4	63.2	44.3	31.1	51.5	55.4

, were obtained by incrementally integrating the DPA module and the SSL into the baseline network.

From the results, it is evident that the addition of the DPA module led to a substantial improvement in the detection of small objects, with the ${\mathrm{AP}}_s$ metric increasing significantly by 4.8%. This demonstrates the crucial role of DPA in enhancing feature fusion and preserving fine-grained spatial details necessary for small target detection.

Furthermore, when SSL was incorporated, the ${\mathrm{AP}}_s$ further increased by 1.7%, indicating that the scale-aware loss function complements the feature-level improvements by better guiding the model to learn small object representations during training. In total, these two modules collectively contributed to a remarkable 6.0% gain in ${\mathrm{AP}}_s$, alongside a 5.0% improvement in the overall AP.

As shown in Table 5, the final model exhibits a slight decrease in large-object AP (${\mathrm{AP}}_l$) from 58.8 to 55.4 compared to the baseline. This reflects an inherent trade-off in scale-sensitive designs: the SSL emphasizes smaller objects by assigning them higher relative weight, and the DPA module enhances local feature aggregation for small targets. Consequently, features for large objects may receive slightly less attention, leading to a modest reduction in ${\mathrm{AP}}_l$. Potential strategies to mitigate this include adjusting the slope of $\lambda \left(s\right)$, balancing losses across different FPN levels, and applying multi-scale feature regularization. These considerations will be explored in future work to better balance performance across all object scales.

The visual detection results comparing the baseline and HSFANet models are presented in Figure 8. Within the zoomed-in windows, it is clearly observed that the baseline model fails to detect two small targets, whereas HSFANet accurately localizes all present objects. This qualitative comparison vividly illustrates the enhanced regression precision and robustness achieved by our method. Such improvements highlight the ability of HSFANet to effectively capture and preserve crucial spatial information during feature aggregation.

Additionally, Figure 9 displays the normalized confusion matrices of both the baseline model and HSFANet. Examining the diagonal elements, it is clear that HSFANet attains higher classification accuracy across all object categories. This improvement underscores the efficacy of the DPA module combined with the customized SSL function in mitigating feature information loss and semantic discrepancies. Consequently, the classification confidence for each category is significantly increased, contributing to the overall superior detection results.

As illustrated by the comparison of the PR curves in Figure 10, the proposed HSFANet significantly outperforms the baseline model across most object categories, particularly for typical small objects such as pedestrians, bicycles, and motorcycles. The PR curves of HSFANet are consistently higher and smoother, indicating better precision–recall trade-offs and more stable performance across different confidence thresholds. These improvements are attributed to the enhanced capability of the DPA module in detail preservation and efficient feature aggregation, as well as the SSL function’s effectiveness in mitigating the issue of target scale imbalance. Overall, HSFANet demonstrates superior detection performance and robustness in complex UAV and remote sensing scenarios, validating the effectiveness of the proposed design.

Specifically, HSFANet contains 63.9M parameters, requires 365.5 GFLOPs for an input resolution of 1024 × 1024, runs at 19.7 FPS on an NVIDIA RTX 3090 GPU (NVIDIA Corporation, Santa Clara, CA, USA), and consumes approximately 3.8 GB GPU memory during inference.

Although our laboratory does not currently have onboard UAV devices to directly measure deployment performance, the above results demonstrate that HSFANet achieves a practical balance between accuracy and efficiency on high-resolution UAV datasets. We will further explore onboard deployment in future work to provide more comprehensive real-world runtime evaluations.

In summary, HSFANet achieves leading performance in object detection by effectively integrating the DPA module and the SSL loss function. This integration enables efficient cross-level aggregation of deep and shallow feature information and mitigates the issue of target scale imbalance. As a result, the model’s ability to focus on and accurately detect small objects is significantly enhanced while maintaining effective loss measurement across targets of varying scales.

5. Conclusions

In this paper, we propose HSFANet, a novel small object detection framework specifically designed for UAV aerial imagery. To address the challenges faced by small objects in terms of limited resolution, feature degradation, and scale imbalance, HSFANet integrates two key components: the DPA module and the SSL function. The DPA module facilitates effective cross-level feature fusion by combining high-level semantic information with low-level spatial details, significantly enhancing the localization capability for small objects. Meanwhile, the SSL introduces a scale-aware regression strategy balance the small object detection outputs in hierarchical prediction heads, thereby effectively improving the performance of small object detection. Extensive experiments conducted on the VisDrone and UAVDT benchmarks demonstrate the superior performance of HSFANet, which is state-of-the-art (SOTA). These results confirm that HSFANet offers an effective and practical solution for small object detection in complex aerial scenarios. Its enhanced detection capability, efficiency, and robustness make it a promising candidate for real-world applications in UAV-based surveillance, monitoring, and remote sensing tasks.

In future work, we plan to extend HSFANet to multi-modal scenarios such as RGB-Thermal fusion and LiDAR-visual fusion, enabling more robust performance under adverse conditions like low light or occlusion. In addition, we will explore lightweight deployment on edge devices, such as UAV onboard systems, to further improve inference speed and adapt the framework for real-time applications.