Vehicle Detection in UAV Images via Background Suppression Pyramid Network and Multi-Scale Task Adaptive Decoupled Head

Abstract

Vehicle detection based on unmanned aerial vehicle (UAV) aerial images plays a significant role in areas such as traffic monitoring and management, disaster relief, and more, garnering extensive attention from researchers in recent years. However, datasets acquired from UAV platforms inevitably suffer from issues such as imbalanced class distribution, severe background interference, numerous small objects, and significant target scale variance, presenting substantial challenges to practical vehicle detection applications based on this platform. Addressing these challenges, this paper proposes an object detection model grounded in a background suppression pyramid network and multi-scale task adaptive decoupled head. Firstly, the model implements a long-tail feature resampling algorithm (LFRA) to solve the problem of imbalanced class distribution in the dataset. Next, a background suppression pyramid network (BSPN) is integrated into the Neck segment of the model. This network not only reduces the interference of redundant background information but also skillfully extracts features of small target vehicles, enhancing the ability of the model to detect small objects. Lastly, a multi-scale task adaptive decoupled head (MTAD) with varied receptive fields is introduced, enhancing detection accuracy by leveraging multi-scale features and adaptively generating relevant features for classification and detection. Experimental results indicate that the proposed model achieves state-of-the-art performance on lightweight object detection networks. Compared to the baseline model PP-YOLOE-s, our model improves the ${AP}_{50:95}$ on the VisDrone-Vehicle dataset by 1.9%.

1. Introduction

In recent years, drones have played a pivotal role in numerous practical engineering applications, such as traffic monitoring [1,2,3], intelligent traffic management [4], orchard pest detection [5], water resource monitoring [6], and disaster relief [7,8]. Compared to traditional remote sensing platforms, drone platforms equipped with cameras not only offer flexible deployment and a broad field of view but also possess commendable real-time capabilities in image transmission and processing. This makes them highly promising for vehicle detection tasks [9]. Consequently, vehicle detection based on drone platforms has garnered extensive attention from researchers.

In the realm of vehicle detection, the use of object detection methods based on deep neural networks has become mainstream. Broadly, these methods can be categorized into two-stage and single-stage object detection techniques. Two-stage object detection methods first generate region proposals in the initial stage, followed by classification and bounding box regression of these proposals in the second stage. The typical representatives of this approach are R-CNN [10] and its derivatives, such as SPPnet [11], Fast R-CNN [12], Faster R-CNN [13], R-FCN [14], and Mask R-CNN [15]. The advantages of this approach lie in its high precision in object detection. However, its downsides include a higher computational complexity and limited real-time capabilities, making it less suitable for deployment in resource-constrained detection hardware platforms. In contrast, single-stage detection methods, unlike their two-stage counterparts, directly predict the category of the objects and position on the feature map using preset anchor points, accomplishing object detection in a single step and thereby enhancing computational speed. Notable instances of this methodology include SSD [16], RetinaNet [17], and YOLO [18,19,20]. Single-stage methods, when compared to two-stage ones, excel in real-time performance but usually lag slightly in accuracy. Although the aforementioned deep learning-based object detection models have shown commendable performance on generic large-scale datasets such as COCO [21], ImageNet [22], and VOC2007/2012 [23], the unique context of data collected from drone platforms renders these datasets distinctively different. First, drone-taken images are often large and have a wide view, with a complicated background and many different objects. This makes it easy to confuse vehicles with similar-looking items, making vehicle detection and classification harder. Second, due to changes in drone height and differences in vehicle sizes, the same object might look different in size in pictures. Then, because of factors like where and when the photo is taken and the environment, there is a significant imbalance in the types of objects found in the database. Lastly, capturing images from an elevated perspective with a drone results in objects occupying a smaller portion of the image, leading to reduced image quality. Consequently, applying existing models based on standard datasets directly for vehicle detection from drone images tends to yield suboptimal performance.

To address these challenges, many researchers have attempted improvements in the workflow of object detection from both data and model perspectives. Regarding data-side enhancements, data augmentation techniques [24,25,26,27] are commonly used to address issues like multi-scale targets and uneven category distribution. For handling multi-scale targets, Li et al. [28] divided the original image into two separate pictures for feature extraction, solving the problem of vehicle small target distortion caused by image reduction, thereby effectively enhancing the accuracy of subsequent models in multi-scale target detection; Zhou et al. [29] employed an adaptive interpolation method, which first categorizes images using object relative scale (ORS) and then uses bilinear interpolation or SRGAN to enlarge and crop images with numerous small vehicle targets to the appropriate size. To improve the imbalance in category distribution, Li et al. [30] proposed a new definition for positive samples and data augmentation algorithms, ameliorating the imbalance between vehicle targets and the background as well as among vehicle samples. Pandey et al. [31] selected images with less frequent category occurrences and expanded them back into the dataset through rotation.

On the model front, considering the limited edge computing capabilities of drone platforms and the real-time requirement for algorithms deployed on them, models applied in actual projects often evolve from single-stage object detection models. A prevalent technique involves combining shallow and deep feature maps [32,33,34,35], primarily targeting low-resolution (small objects) and background interference issues. For small object detection, Xie et al. [36] extended YOLOv2 to DYOLO, enhancing accuracy through deep-shallow upsampling feature map fusion. Tayara et al. [37] developed a Full Convolutional Regression Network (FCRN) tailored for drone image-based small vehicle detection, building on Faster-RCNN. Liang et al. [38] introduced the FS-SSD network, a fusion of SSD and Feature Pyramid Network, which, through average pooling layers and deconvolution layers, bolstered small object detection precision. For addressing background interference, Xi et al. [39] proposed a fine-grained target focusing network (FiFoNet), which effectively selects multi-scale feature combinations during feature fusion, further enhancing multi-scale feature expressiveness and thus suppressing background noise. Ma et al. [40] presented an aerial visual scene-based object detection method (AVS-YOLO) that filters out redundant background information by introducing a dense connection in the feature pyramid section. Notably, while many studies have improved generic methods from various angles to better fit the unique context of drone aerial imagery, no single approach has yet provided a comprehensive solution framework to holistically address the aforementioned challenges.

To solve the aforementioned issues, this article proposes an object detection model based on the Background Suppression Pyramid Network and Multi-scale Task Adaptive Decoupled Head, built upon the PP-YOLOE framework. This model consists of four parts: data preprocessing, backbone network, feature fusion network, and detection head network. In the data preprocessing part, a long-tail feature resampling algorithm was designed to address the uneven distribution of target categories. The backbone network part employs the CSPResNet architecture from the PP-YOLOE framework. In the feature fusion network part, a background suppression module and a Lightweight Transformer module are added to the original network to reduce background information interference and enhance the capability of detecting small targets. In the detection head network, a newly designed multi-scale task adaptive decoupled head replaces the original detection head to deal with severe changes in object scale. This model design effectively improves detection accuracy in unmanned aerial vehicle target detection. The main contributions of this article include:

• Long-tail feature resampling algorithm: This algorithm generates target prediction density maps and resamples small-sample targets based on these maps to alleviate the issue of imbalanced target category distribution.
• Background suppression module: This module integrates spatial and channel attention mechanisms. Spatial attention focuses on key image regions to highlight targets, while channel attention emphasizes important features. The combined use of these attention methods can reduce the interference of the background in object detection.
• Lightweight Transformer module: This module uses the concept of lightweight to divide input features into blocks, each then processed by a transformer. This design not only improves the ability of the model to detect small targets but also reduces computational requirements.
• Multi-scale task adaptive decoupled head: This module processes multi-scale features from different receptive fields through dynamic convolution to extract target scale features, selecting the most optimal features. This approach addresses the problem of drastic target scale changes in unmanned aerial vehicle target detection.

2. Related Work

In this section, this paper will first review our baseline model and then delve into the detailed design of PP-YOLOE [41], from the overall network architecture and feature fusion network, to the detection head network.

The overall architecture of PP-YOLOE is built upon PP-YOLO v2 [42]. A series of upgrades and optimizations have been applied, as depicted in Figure 1. The design of the PP-YOLOE model showcases its uniqueness in several aspects. Firstly, the model employs a novel and unified backbone and neck design, which flexibly supports configurations of various sizes. Secondly, to effectively address the imbalance issue between classification and regression in object detection tasks, PP-YOLOE adopts the dynamic matching strategy of task alignment learning (TAL). This strategy significantly improves detection accuracy with an efficient label assignment method. Lastly, PP-YOLOE introduces the design of the efficient task-aligned head (ET-Head). Although this induces a minor speed reduction, it markedly enhances the detection precision of the model.

In the feature fusion (neck) network of PP-YOLOE, the model adopts the CSPRepResBlock module, as illustrated in Figure 2. This design utilizes residual connections, effectively addressing the vanishing gradient problem while simultaneously serving as a form of model ensembling to boost performance. Additionally, the implemented dense connection fusion strategy integrates intermediate-layer features with varied receptive fields, offering significant advantages in tasks like object detection, and showcasing commendable results.

In the detection head network of the PP-YOLOE model, the ET-Head module is specifically designed for classification and regression tasks, as depicted in Figure 3. The primary improvements encompass four facets. Firstly, the time-consuming task interaction feature module in the original task-aligned head (T-Head) is discarded. Secondly, while ensuring accuracy, the channel attention module is streamlined into an effective squeeze and extraction block (ESE). Furthermore, to enhance the speed, the classification task alignment module has been simplified to a shortcut. Lastly, addressing the intricate and deployment-unfriendly regression task alignment module in the T-Head, the model draws inspiration from the integration module in GFL [43] to model the detection bounding box.

In summary, PP-YOLOE demonstrates remarkable advantages in terms of performance, hardware compatibility, scalability, and flexible model configuration, outperforming YOLOv5 and YOLOX in the trade-off between speed and accuracy. Although PP-YOLOE, as a universal object detection model, has exhibited superior performance in numerous application scenarios, its direct application to the domain of drones might encounter some limitations. On the one hand, images captured by drones often encompass intricate backgrounds, potentially challenging the detection accuracy of the model. On the other hand, the feature distribution of drone-captured images diverges from typical image datasets, with more pronounced scale variations of objects from a drone’s perspective, necessitating specific model refinements. Thus, it is imperative to suitably adjust and enhance the PP-YOLOE model to cater to the unique demands of the drone realm.

Our model has been adapted and optimized, with PP-YOLOE serving as the benchmark. At the outset of the backbone network, we introduce a data augmentation module that aims to boost the representation of minority samples. Within the neck network, we drew inspiration from the path aggregation network (PAN) structure, incorporating a background suppression block during the top-down phase to mitigate the influence and interference of background noise on the model. In the subsequent bottom-up phase, a lightweight transformer block is integrated to further refine the performance of the model. At the detection head, we conceptualized and integrated a multi-scale task adaptive decoupled head.

The primary objective of this module is to address the drastic scale variations of a single object, thereby enhancing the recognition and handling capabilities of the model for targets of varying scales. Through such strategic design, our model not only retains its foundational performance but further elevates its ability to identify targets amidst complex backgrounds and sparse samples, simultaneously optimizing its adaptability to diverse object scale variations.

3. Proposed Approach

3.1. Overall Framework of the Model

In this study, we introduce a novel vehicle detection method based on the background suppression pyramid network and multi-scale task adaptive decoupled head. The overall architecture of the model, as illustrated in Figure 4, comprises four main components: data enhancement, backbone network, neck network, and detector head network.

Within the data enhancement segment, we employ a long-tailed feature resampling strategy. This approach, by generating drone images through the cropping and re-pasting of density maps, ameliorates the imbalance of vehicle target categories present within the dataset. In the backbone network, we continue to leverage the cross stage paritial network (CSPDarknet) architecture from the PP-YOLOE [41] for image feature extraction. Within the Neck network, image features first pass through a background suppression block. This block generates attention feature maps in both the channel and spatial dimensions, mitigating the interference of image backgrounds on target objects. Subsequently, features are processed by the lightweight transformer block, which, through multi-head attention and a fully connected approach, enhances the capability to capture varying local information, bolstering the detection prowess for smaller targets. In the detector head network, we design the multi-scale task adaptive decoupled head. Within this module, three distinct branches are designed, each extracting image features from different receptive fields, addressing the challenges posed by the pronounced scale variations in the to-be-detected vehicle targets.

In the sections that follow, we delve into the refinements made in the domains of data enhancement, the neck network, and the detector head network in this study.

3.2. Data Enhancement

In this paper, the data enhancement network is illustrated in Figure 5, centered around the long-tailed feature resampling algorithm for data augmentation. This algorithmic flow consists of three main components: density map generation, cropping based on the density map, and image processing, followed by re-stitching. Initially, the input image undergoes a trained network to produce a predictive density map, which is then used in series-wound with a sliding window and a threshold to create a density mask, resulting in cropped images. These cropped images are subsequently flipped and adjusted randomly before being stitched onto drone-view background images to generate new images.

In the density map generation segment, the image is channeled into three distinct paths. Each path employs varying convolution kernel sizes to capture features of different scales. Within each path, the image undergoes convolution followed by a $2\times 2$ pooling operation, a process that is repeated twice, ultimately reducing the output feature map size to a quarter of the original image. To retain the initial resolution, while circumventing the memory issues related to very high-resolution images, this paper employs a nearest-neighbor interpolation method to upscale the produced density map by a factor of four.

For the segment involving cropping based on the density map, the areas with densely packed vehicle targets exhibit a higher pixel intensity. Therefore, we utilize the segmentation algorithm from reference [44] to set thresholds within these regions. This aids in estimating the number of vehicle targets while filtering out pixels void of any target vehicles.

During the image processing and re-stitching phase, to ensure that the cropped images serve as a means for data augmentation of tail-end vehicle targets without altering the relative size of the vehicle targets to the original image, the cropped images are first horizontally flipped and subsequently subjected to contrast and brightness adjustments. To prevent image distortion, the adjustments are based on random values within a specified range. Depending on the dimensions of image, it is then spaced at specific intervals against a naturally captured drone background, resulting in the final composited image.

3.3. Background Suppression Pyramid Network

The structure of the background suppression pyramid network employed in this study is depicted in Figure 6. It is constituted of two branches, top-down and bottom-up, with lateral connections that fuse the feature maps from both branches. Within the top-down branch, the deeper network layers convey strong semantic information, providing a more abstract and global understanding crucial for tasks like target class identification and rough localization. Conversely, in the bottom-up branch, the shallower network layers communicate robust localization feature information, capturing the fine structure and positioning of the target. This type of information is vital for precise target localization and recognition tasks. The fusion of these feature maps amalgamates parameters from different backbone layers to their respective detection layers, significantly enhancing vehicle target recognition and localization. Building upon this, our study innovatively incorporates a background suppression block within the top-down branch. This ensures that, during the fusion phase, semantic information emphasizes vehicle target features while mitigating the influence of irrelevant background information. Additionally, within the bottom-up branch, we introduce the lightweight transformer block, leveraging multi-head self-attention mechanisms to bolster spatial localization for smaller vehicle targets. Subsequent sections delve into the background suppression block and lightweight transformer block in detail.

3.3.1. Background Suppression Block

In this paper, the background suppression block (BS Block) is introduced, as illustrated in Figure 7. The structure of this module is divided into two main components: channel attention and spatial attention. Both components operate independently on the original feature map, yielding channel and spatial weights, respectively. Each of these weights is then multiplicatively combined with the original input feature map to achieve adaptive feature refinement, producing two distinct feature maps. Ultimately, these maps are combined via pixel-wise addition and undergo normalization, resulting in a feature map enriched with both channel and spatial weights.

Given a primary feature map $F^B$ from the backbone network, it is concurrently processed through both channel attention and spatial attention mechanisms. This results in the channel attention weight $M^c$ and spatial attention weight $M^s$. These weights are then element-wise multiplied with the original input feature map, yielding the channel-informed feature map $F^c$ and the spatial-informed feature map $F^s$, respectively. Ultimately, a pixel-wise summation of these two maps is conducted, followed by normalization, producing a feature map $F^{sc}$, encapsulating both channel and spatial information.

The channel attention mechanism used in this paper employs channel average pooling and channel max pooling to capture global and significant features within the channel dimension. By integrating these two methods, the model effectively extracts discriminative features closely associated with vehicle targets, thereby enhancing its capability for vehicle detection. The resultant features are then fed into a shared, fully connected network. The outputted features from this network undergo pixel-wise summation and a subsequent sigmoid normalization to derive the channel attention weight $M^c$. The channel-informed feature map $F^c$ is defined as:

$$F^c=F^B\times M^c$$

(1)

where $F^B$ is the input feature map from the backbone. The channel attention weight $M^c$ is expressed as:

$$M^c=sigmoid\left(MLP\left(AvgPool\left(F^B\right)\right)+MLP\left(MaxPool\right(F^B\left)\right)\right)$$

(2)

where $sigmoid(·)$ denotes the sigmoid activation function, $MLP(·)$ signifies the shared fully connected operation, $AvgPool(·)$ and $MaxPool(·)$ stand for average pooling and max pooling, respectively.

The spatial attention mechanism adopted in this paper, through the application of average pooling and max pooling operations on feature maps, effectively extracts global and prominent features in the spatial dimension. The combination of these two operations not only emphasizes key areas in the image but also highlights features crucial for vehicle target detection, thereby effectively enhancing the capability of the model to detect vehicle objects. After the concatenation of this information, convolution and normalization operations are applied to consolidate into the spatial attention weight $M^s$. The spatial-informed feature map $F^s$ is then defined as:

$$F^s=F^B\times M^s$$

(3)

where $F^B$ is the input feature map from the backbone. The spatial attention weight $M^s$ is given by:

$$M^s=sigmoid\left({Conv}_{7\times 7}\left(concat\left(AvgPool\left(F^B\right),MaxPool\left(F^B\right)\right)\right)\right)$$

(4)

where ${Conv}_{7\times 7}$ represents the convolution operation with a $7\times 7$ kernel size, and $concat$ signifies the concatenation operation.

Upon obtaining both the channel-informed feature map $F^c$ and the spatial-informed feature map $F^s$, the final feature map $F^{sc}$ can be represented as:

$$F^{sc}=sigmoid\left(F^c+F^s\right)$$

(5)

3.3.2. Lightweight Transformer Block

The lightweight transformer block designed in this study is depicted in Figure 8. Given an input image feature $F^p$, it is first equally divided into $n\times n$ local image features ${\widehat{F}}_i$, $0<i\le n^2$. Subsequently, for each local image feature, a token ${\widehat{F}}_i^t$ is generated. Following an attention operation on the token, it is concatenated with its corresponding local image feature to form a new local image feature ${\widehat{F}}_i^w$. Each of these new local features is subjected to a distinct attention operation. The final operation involves reconstructing and concatenating these processed local features to generate $F^{wt}$, which retains the same feature size as $F^p$.

Initially, the transformer operation of global image features is translated into transformer operations for local image features. Employing a transformer on local image features efficiently enhances computational efficiency. By partitioning the input feature map of size $H\times W$ uniformly into $n\times n$ local windows, and for simplicity letting $H=W$, $n=H^2/k^2$, where $k$ is the size of the local image feature. The local image feature ${\widehat{F}}_i$ is represented as:

$${\widehat{F}}_i={Split}_{k\times k}\left(F^p\right)$$

(6)

where ${Split}_{k\times k}$ divides the image feature by $k\times k$. Each local image feature initializes a token to capture its comprehensive information. The token initialization is given by:

$${\widehat{F}}_i^t={AvgPool}_{k^2\to 1}\left({Conv}_{3\times 3}\left(F^p\right)\right)$$

(7)

where ${Conv}_{3\times 3}$ effectively extracts from the image feature and ${AvgPool}_{k^2\to 1}$ pools the $k^2$ sized local image feature into a size 1 token. This token represents the holistic information of its respective local image feature. These local tokens ${\widehat{F}}_i^t$ are concatenated to produce ${\widehat{F}}^t$, which undergoes an Attention & Conv layer to facilitate local image feature interactions via tokens, enriching the information capture. The Attention & Conv operation is expressed as:

$${\widehat{F}}^{At}=Conv\left(BN\left({\widehat{F}}^t+{\gamma }_1·MHSA\left(Conv\left(BN\left({\widehat{F}}^t\right)\right)\right)\right)\right)$$

(8)

where ${\gamma }_1$ is a hyperparameter and $MHSA$ stands for multi-head self-attention. ${\widehat{F}}^{At}$ is the result after the tokens have undergone Attention & Conv. These tokens are then separated:

$${\widehat{F}}_i^{At}={Split}_{1\times 1}\left({\widehat{F}}^{At}\right)$$

(9)

where ${Split}_{1\times 1}$ divides ${\widehat{F}}^{At}$ into $1\times 1$ sizes, yielding $n\times n$ tokens ${\widehat{F}}_i^{At}$ that have undergone information exchange.

Next, interactions between local image features and exchanged information tokens are computed to model both short and long-range spatial information. Local features ${\widehat{F}}_i$ and tokens ${\widehat{F}}_i^{At}$ are concatenated to form ${\widehat{F}}_i^w$. Each of these localized features ${\widehat{F}}_i^w$ enters an Attention & Conv to produce ${\widehat{F}}_i^{Aw}$:

$${\widehat{F}}_i^{Aw}=Conv\left(BN\left({\widehat{F}}_i^w+{\gamma }_1·MHSA\left(Conv\left(BN\left({\widehat{F}}_i^w\right)\right)\right)\right)\right)$$

(10)

where ${\widehat{F}}_i^{Aw}$ is split back into tokens after local attention ${\widehat{F}}_i^{Awt}$ and local image features ${\widehat{F}}_i^A$. A reconstruction and concatenation operation yields the feature map $F^{wt}$:

$$F^{wt}=Concat\left(reshape\left({\widehat{F}}_i^A\ast {\widehat{F}}_i^{Awt}+{\widehat{F}}_i^A\right)\right)$$

(11)

3.4. Detection Head Network

In this work, we inherit the decoupled detection head design from PP-YOLOE and further refine it to enrich image features of multi-scale objects. This addresses the issue of dramatic scale variations of objects from drone perspectives, consequently enhancing the accuracy of object detection. The proposed multi-scale task adaptive decoupled head (MTAD Head) is illustrated in Figure 9. This network channels feature maps from three pyramid levels into distinct classification and regression branches. These branches extract classification and regression features, ultimately pinpointing the location and category of vehicle targets.

3.4.1. Multi-Scale Task Adaptive Decoupled Head

As depicted in Figure 9, the multi-scale task adaptive decoupled head (MTAD Head) is divided into classification and regression branches. Each branch encompasses features from three pyramid levels. These branches have different receptive fields, capturing multi-scale features, which enhances the capability of the model to detect objects with severe scale variations. Concurrently, the classification task requires features that are spatially coarse yet semantically rich to infer the category of the object. In contrast, the regression task necessitates high-resolution features imbued with edge details for a more precise object boundary regression. Our MTAD Head is designed to allow the model to adaptively learn features required for both classification and regression tasks. Each pyramid level is intrinsically linked to its two neighboring levels, ensuring that the features from the three pyramid levels cater to the demands of both classification and regression tasks.

3.4.2. Dynamic Convolutional Attention Block

The structure of the dynamic convolutional attention block (DCA) designed in this study is illustrated in Figure 10. Given a network input $F_n$, the feature map first undergoes a linear layer to yield ${\overline{F}}_n$. Both $F_n$ and ${\overline{F}}_n$ are then fed into the group deformable convolutional network (DCN). The multi-group DCN can learn richer information from various positions and distinct representational subspaces. Each DCN adaptively adjusts the sampling position and shape of its convolution kernel based on the deformation of the object, thereby enhancing the perceptual capability of the model towards the target. For common issues in drone imagery, such as object deformation and scale variation, the DCN can better adapt to feature variations of the target, broadening the coverage of the receptive field. The DCN passes $F_n$ through a convolutional and a linear layer to obtain the mask $m_n$ and the offset $∆p_n$. The offset leads to deformed convolution, which, when multiplied by the mask, procures the weight of each sampling point, mitigating interference from background factors. The deformable convolution formula is as follows:

$$DCN\left(p_0\right)=\sum _{p_n\in R}w\left(p_n\right)·\left(p_0+p_n+∆p_n\right)·m_n$$

(12)

where $p_0$ corresponds to the central coordinate in the convolution kernel. $p_n$ signifies other points’ relative positional coordinates concerning the center of the kernel during convolution. $R$ represents the relative positions of all sampling points in the kernel, and $m_n$ is the weight of each sampling point. Outputs from multiple DCNs are channeled into the SimAM [45] to capture essential features, yielding $F^{SD}$:

$$F^{SD}=SimAM\left(\left[{DCN}_0\left(F_n\right),{DCN}_1\left(F_n\right),\dots ,{DCN}_n\left(F_n\right)\right]\right)$$

(13)

3.5. Loss Function

Section 3.5 of this study introduces two types of loss functions, which play a vital role in the density map generation network and the object detection network. Loss functions are crucial in the training process of deep learning models, as they assess the discrepancy between the output of the model and the actual results. The model can learn how to accurately predict and detect targets by minimizing these loss functions.

3.5.1. Density Map

The loss function for the density map generation network is based on pixel-wise mean absolute error and can be represented as:

$$L\left(\theta \right)=\frac{1}{2N}\times \sum _{i=1}^N{‖D\left(X_i;\theta \right)-D_i‖}^2$$

(14)

where $N$ denotes the total number of training samples, $\theta$ represents the parameters of the data augmentation network, $X_i$ and $D_i$ are the input image and the ground truth density map, respectively. $D(X_i;\theta )$ represents the density map generated through the density map network. The method to obtain the ground truth density map $D_i$ is similar to reference [46]. Initially, a geometry-adaptive Gaussian kernel is used to blur the annotation for each object, generating the actual target density map for drone imagery.

3.5.2. Object Detection

The loss function for the object detection network can comprehensively evaluate target location, scale, category, and confidence. It can be formulated as:

$$Loss=\frac{1}{{\sum }_i^{N_{pos}}\widehat{t}}\left(\alpha ·{loss}_{VFL}+\beta ·{loss}_{GIoU}+\gamma ·{loss}_{DFL}\right)$$

(15)

where ${\sum }_i^{N_{pos}}\widehat{t}$ where $N_{pos}$ is the number of positive samples and $\widehat{t}$ represents the target score (equivalent to target confidence). ${loss}_{VFL}$ denotes the classification loss, ${loss}_{DFL}$ indicates regression loss, and ${loss}_{GIoU}$ represents the target localization loss. The hyperparameters $\alpha$, $\beta$ and $\gamma$ are used to balance the individual losses.

4. Experiments

In the experimental section, we first constructed the VisDrone-Vehicle dataset based on the VisDrone-DET2019 dataset. Subsequently, by comparing the performance of our proposed method with relevant methods on the drone object detection task using the VisDrone-Vehicle dataset, we further confirmed and highlighted the practical value and potential impact of our approach.

4.1. VisDrone-Vehicle Dataset

The VisDrone-Vehicle dataset we utilized is built upon the VisDrone-DET2019 dataset. Based on the detection task target categories, we narrowed down the original categories to 5, namely cars, vans, trucks, buses, and motorcycles. The entire dataset comprises 6471 training images and 548 validation images. Specifically, the number of objects for cars, vans, trucks, buses, and motorcycles are 172,940, 15,534, 30,727, 8866, and 35,492, respectively. It is evident that the car category is the most prevalent, accounting for 65.6% of the total number of objects, while the bus category represents only 3.3%. There is a substantial inequality in the number of objects across categories, indicating a long-tail distribution in the data. The article defines large, medium, and small targets according to the COCO standard [21], objects with a bounding box area smaller than 32 × 32 pixels as small objects, those with an area greater than 32 × 32 pixels but smaller than 96 × 96 pixels as medium objects, and those with an area larger than 96 × 96 pixels as large objects. The distribution of large, medium, and small objects across each category is shown in

Table 1. Distribution of object counts across categories in the VisDrone-Vehicle dataset.

Object Size	Total	Category	Number
Small	132,484	Car	84,913
		Van	13,643
		Truck	4761
		Bus	4984
		Motor	26,808
Medium	110,752	Car	74,976
		Van	14,212
		Truck	8169
		Bus	4984
		Motor	8411
Large	20,323	Car	13,051
		Van	2872
		Truck	2604
		Bus	1523
		Motor	273

. The dataset predominantly consists of small objects, but there is also a significant presence of medium and large objects. Scenes with objects of varying sizes appearing simultaneously are common, signifying considerable variation in object scales.

4.2. Experimental Setup

The experiments in this study were conducted on a tower server platform, SYS-4028GR-TR, provided by Supermicro, a company based in San Jose, California. For GPU, we employed a Nvidia RTX 2080Ti graphics card. The software environment was established on the Ubuntu 18.04 operating system, utilizing mmdetection and mmyolo as the object detection frameworks. During the training process, to achieve lightweight object detection, images of size 2000 × 1500 were resized to 640 × 640. We recommend using the stochastic gradient descent (SGD) method, initializing the learning rate at 0.005, setting momentum at 0.9, and weight decay at 0.0001.

As primary model evaluation metrics, this paper employs various average precisions ($AP$) based on different IoU thresholds. These include $AP$ at an IoU threshold of 0.5 (denoted as ${AP}_{50}$), $AP$ at an IoU threshold of 0.75 (denoted as ${AP}_{75}$), and $AP$ at IoU thresholds ranging from 0.5 to 0.95, with an interval of 0.05 (denoted as ${AP}_{50:95}$). In addition, we considered the impact of area scale on model performance, introducing AP metrics based on different object scales. Collectively, these metrics offer a comprehensive and detailed perspective on model performance evaluation. The ($AP$ metric can be calculated as:

$$AP={\int }_0^{1}P\left(R\right)dR$$

(16)

where $P\left(R\right)$ denotes the curve formed by Precision $P$ and Recall $R$. Precision measures the accuracy of predictions, while Recall assesses the coverage of actual positive samples by the model, indicating how many actual positive samples the model can correctly detect. Precision and Recall can be expressed as:

$$P=\frac{TP}{TP+FP}$$

(17)

$$R=\frac{TP}{TP+FN}$$

(18)

where FN, FP, and TP represent the numbers of false negatives, false positives, and true positives, respectively.

4.3. Ablation Study

To validate the efficacy of the proposed modules in addressing the challenges of class imbalance, a plethora of small vehicle objects, significant object scale variations, and complex background information in drone-captured images, we conducted an ablation study on key components of our method. Specifically, we examined the long-tail feature resampling algorithm, the Background Suppression Pyramid Network (BSPN), and the multi-scale task-adaptive decoupled head. The results are presented in

Table 2. Ablation study results of the proposed method on the VisDrone-Vehicle dataset.

Method	LFRA	BSPN	MTAD	${\mathit{A}\mathit{P}}_{50:95}$	${\mathit{A}\mathit{P}}_{50}$	${\mathit{A}\mathit{P}}_{75}$	${\mathit{A}\mathit{P}}_{\mathit{s}\mathit{m}\mathit{a}\mathit{l}\mathit{l}}$	${\mathit{A}\mathit{P}}_{\mathit{m}\mathit{e}\mathit{d}\mathit{i}\mathit{u}\mathit{m}}$	${\mathit{A}\mathit{P}}_{\mathit{l}\mathit{a}\mathit{r}\mathit{g}\mathit{e}}$
Baseline				30.9	49.7	33.3	13.8	40.7	52.4
M1	$\surd$			31.6	50.2	34.0	14.3	41.3	53.3
M2		$\surd$		31.5	50.0	33.9	14.0	41.6	53.5
M3			$\surd$	31.7	50.4	34.5	14.1	41.8	53.6
M4	$\surd$		$\surd$	32.4	51.0	35.2	14.5	42.5	54.2
M5	$\surd$	$\surd$		32.0	50.6	34.8	14.2	41.9	53.9
M6		$\surd$	$\surd$	32.1	50.8	34.9	14.3	42.3	53.8
M7	$\surd$	$\surd$	$\surd$	32.8	51.7	36.0	14.6	43.4	54.7

. In this context, the baseline model is the PP-YOLOE-s model without the inclusion of the aforementioned modules. Models M1, M2, M3, and M4, respectively, incorporate the long-tail feature resampling algorithm, the background suppression pyramid network, the multi-scale task-adaptive decoupled head, and a combination of the long-tail feature resampling algorithm and the multi-scale task-adaptive decoupled head. Model M5 represents the complete method proposed in this study.

As can be observed from Table 2, across all performance metrics, M1 outperforms the baseline, indicating that the long-tail feature resampling algorithm has enhanced the capability of the model to recognize small-sample targets, effectively improving the performance of the object detection model. Comparing the results of the baseline model (baseline) with M2, it is evident that adopting the background suppression pyramid network significantly boosts the performance of object detection. Particularly, there is an improvement of 0.9% and 1.1% in detecting medium and large targets, respectively. Notably, since we resized the input images, the area of medium-sized targets shrunk by an average of 7.3 times, placing them in the category of small objects in general object detection. Hence, the performance enhancement of our model in ${AP}_{medium}$ slightly large, small objects essentially mirrors its prowess in handling smaller targets. This is attributed to our model effectively leveraging the pyramid network structure with background suppression capabilities, leading to a notable accuracy boost in small object detection. Comparing the results of the baseline and M3, there is an improvement across all performance metrics, with ${AP}_{small}$, ${AP}_{medium}$ and ${AP}_{large}$ improving by 0.3%, 1.1%, and 1.2%, respectively. This indicates that our modules enhance detection performance when facing drastic target scale variations, validating the effectiveness of our model. When comparing the results between the baseline and M4, it is evident that the combined use of the modules can further boost the performance of the model. From the results of M4, it is evident that combining LFRA with MTAD has significantly improved the overall detection capability of the model, compared with M1, an increase of 0.8% in ${AP}_{50:95}$. This indicates that M4 enhances the detection ability for small samples and shows strong adaptability to the frequent and drastic target scale changes commonly observed from a drone’s perspective. Comparing M5 with M1, it is found that the addition of a BSPN on the basis of M1 improves the detection of medium and large targets, suggesting that the model effectively suppresses background noise in object detection and better extracts features of medium and large targets. The comparison of M6 with M2 shows that the integration of an MTAD into the base of M2 leads to a 0.7% increase in ${AP}_{50:95}$. The combination of these two modules effectively enhances the overall detection ability of the model, demonstrating the targeted effectiveness of this combination for UAV-based vehicle object detection tasks. Comparing the outcomes of the baseline and M7 (the model introduced in this paper), the ${AP}_{50:95}$ increases by 1.9% compared to the baseline. This further attests to the efficacy of each module in enhancing the performance of the model. Additionally, it suggests that when these modules are employed collectively, they synergize, leading to even better model capabilities.

The detection results from the ablation study of our proposed method on the VisDrone-Vehicle dataset are shown in Figure 11. White boxes in the image denote areas that the model failed to predict, while boxes of other colors indicate correctly detected targets. A comparison between Figure 11c,d reveals that upon the inclusion of the long-tail feature resampling algorithm, the recognition ability of the model for low-sample targets, namely buses and motorcycles, was enhanced. Four misdetected targets are seen in Figure 11c, specifically located in boxes 1, 3, 4, and 7. After incorporating the long-tail feature resampling algorithm, the four misdetected cases in Figure 11c were improved, indicating that the long-tail feature resampling algorithm is effective in addressing sample distribution imbalance. By contrasting Figure 11c,e, it can be observed that in Figure 11c, the unrelated background was misidentified as the detection target in boxes 1, 4, 7, and 9. However, in Figure 11e, this background noise is suppressed and no longer misinterpreted as a target. This suggests that the feature pyramid module based on transformer can suppress intricate background information, thereby reducing the false detection rate of the model. Comparing Figure 11c,f, after incorporating MTAD, the detection rate of the model for distant small targets was heightened, effectively recognizing distant car targets. Surprisingly, even some unannotated distant car targets were detected accurately. This demonstrates that the module effectively enhances the ability of the model to detect targets of the same category with drastic shape variations. When contrasting Figure 11c,g, it is evident that the enhanced model could proficiently recognize near-range low-sample motorcycle targets and distant car targets. This indicates a significant improvement in recognition precision for low-sample targets by the model and its capability to recognize targets with drastic shape variations. Lastly, a comparison between Figure 11c,h showcases that our proposed model, when compared with the baseline model, not only detects more distant small targets but also exhibits commendable recognition ability for low-sample motorcycle and bus targets amidst complex backgrounds. Overall, the detection capability of the model has been significantly uplifted.

4.4. Overall Model Performance Analysis

To further validate the detection performance of our proposed model on UAV vehicle object detection tasks, we carried out a comparative analysis between our model and other classic models on the VisDrone-Vehicle dataset. The detection results of various object detection methods on the VisDrone-Vehicle dataset are presented in

Table 3. Detection results (%) of different methods on the VisDrone-Vehicle Dataset.

Method	${\mathit{A}\mathit{P}}_{50:95}$	${\mathit{A}\mathit{P}}_{50}$	${\mathit{A}\mathit{P}}_{75}$	${\mathit{A}\mathit{P}}_{\mathit{s}\mathit{m}\mathit{a}\mathit{l}\mathit{l}}$	${\mathit{A}\mathit{P}}_{\mathit{m}\mathit{e}\mathit{d}\mathit{i}\mathit{u}\mathit{m}}$	${\mathit{A}\mathit{P}}_{\mathit{l}\mathit{a}\mathit{r}\mathit{g}\mathit{e}}$	Param	GFlops	FPS
Faster RCNN [13]	26.2	41.4	29.5	10.3	36.5	45.9	41.1	202	11.6
Cascade RCNN [47]	27.9	42.7	31.6	11.2	38.5	50.4	68.9	230	16.3
FSAF [48]	19.4	35.4	19.1	6.8	26.3	39.5	36.0	201	20.7
GFL [43]	20.4	35.1	21.5	6.9	27.8	42.0	32.0	203	20.6
FCOS [49]	18.4	32.2	18.8	4.9	25.6	40.1	31.9	196	21.4
Fovea [50]	17.5	30.5	18.2	3.3	25.1	41.2	36.0	201	21.1
YOLOX [51]	23.7	40.1	25.2	9.3	32.2	33.9	8.94	13.4	46.2
YOLOv6-n [52]	24.9	40.7	26.7	9.3	33.7	47.5	4.30	5.49	33.4
YOLOv6-t [52]	28.4	45.6	31	11.1	38.1	49.2	9.67	12.3	34.8
YOLOv7-t [53]	25.4	43.3	26.7	9.9	33.7	46.9	6.02	6.89	65.5
YOLOv8-n	25.3	41.4	27.1	9.5	34.5	45.6	3.01	4.40	64.7
YOLOv8-s	30.2	48.2	32.7	12.7	40.4	51.8	11.1	14.3	63.2
Faster RCNN NWD [54]	30.3	32.8	49.8	15.9	39.5	46.5	41.14	202	11.2
CZ Det [55]	24.8	26.3	41.3	12.2	30.9	35.2	45.9	210	7.0
RT-DETR-s [56]	28.9	31.8	46.0	11.2	39.1	51.3	8.87	14.8	16.7
PPYOLOE-s [41]	30.9	49.7	33.3	13.6	40.4	51.5	7.46	7.95	45.1
Our method	32.8	51.7	36	14.6	43.4	54.7	10.3	13.1	20.1

.

From Table 3, it can be observed that the model proposed in this study, as a single-stage detection model, not only leads in overall performance but also boasts a parameter size of just 9.48 M, making it highly suitable for deployment on UAV platforms. Compared to the current top-performing single-stage model, YOLOv8-s, our proposed model shows an improvement of 2.6% on ${AP}_{50:95}$. Relative to the best-performing two-stage model, Cascade RCNN, our model sees an elevation of 4.9% in ${AP}_{50:95}$. Compared to the best model in this field, Faster RCNN NWD, our proposed model has achieved a 1.5% increase in ${AP}_{50:95}$, indicating that the overall performance of the model is superior in UAV vehicle detection tasks, with significant advancements across other metrics as well. Specifically, the ${AP}_{small}$ of our model is 1.9% and 3.4% higher than YOLOv8-s and Cascade RCNN, respectively, highlighting that our model exhibits superior performance in UAV detection tasks, which typically involve smaller targets in the images, making it especially suited for UAV vehicle detection tasks. Furthermore, for ${AP}_{medium}$ and ${AP}_{large}$, our model surpasses the top-performing single-stage model, YOLOv8-s, by 3% and 2.9%, respectively. When contrasted with the best-performing two-stage model, Cascade RCNN, our model records increments of 4.9% and 4.3% in these two metrics. When compared to the best model in this field, Faster RCNN NWD, these two metrics are higher by 3.9% and 8.2%, respectively. This underscores that our model is likely to exhibit superior performance in scenarios intrinsic to UAV vehicle detection tasks where there is a significant variance in target sizes.

In the evaluation of unmanned aerial vehicle (UAV) target detection systems, besides detection performance, the real-time nature, size, and computational efficiency of the model are also key indicators. In the UAV vehicle target detection field, the real-time nature of a model is typically measured by frames per second (FPS). Considering the flying altitude of UAVs, the field of view range, the average speed of ground targets, and actual engineering requirements, a model is generally considered to have high real-time performance when its FPS reaches 15. Such processing speed not only sufficiently covers the monitoring area but also ensures effective capture and analysis of the dynamics of targets within the field of view. As seen in Table 3, the model proposed in this article meets the real-time requirements on the FPS indicator and outperforms the latest UAV detection research like RT-DETR-s, CD Det, and Faster RCNN NWD. Additionally, when evaluating UAV vehicle target detection models, the size of the model and computational efficiency are commonly measured in GFlops, particularly important in hardware resource-limited application scenarios, such as on mobile devices or embedded systems. Usually, a higher GFlops value implies a larger, more complex model requiring more computational resources. The GFlops value of the model proposed in this paper is only 13.1, showing lighter weight and more computation-friendly characteristics compared to other Yolo series models currently used in actual engineering. These indicators collectively reflect the comprehensive performance advantages of our model in the field of UAV vehicle target detection.

The detection outcomes of the optimal single-stage and two-stage models in typical UAV vehicle detection scenarios are depicted in Figure 12. As can be discerned from Figure 12d, YOLOv8-s demonstrates commendable detection capabilities for smaller targets commonly found in UAV detection tasks. However, being a general detection model, it does not adequately account for the challenges posed by the complex backgrounds inherent in UAV detection, nor does it consider the class imbalances present within the dataset. As a result, the image exhibits numerous false detections, and it struggles with detecting under-represented objects such as buses and motorcycles. Observing Figure 12e, it becomes evident that the Cascade RCNN model, much like YOLOv8-s, faces difficulties in detecting smaller objects and exhibits a limited proficiency in recognizing under-sampled objects like buses and motorcycles. In contrast, our proposed model showcases superior detection performance in these typical UAV vehicle detection scenarios.

The relationship between ${AP}_{50:95}$ and model parameters across various models is depicted in Figure 13. The x-axis represents the parameters of the model, while the y-axis illustrates the detection performance metric ${AP}_{50:95}$ on the VisDrone-Vehicle dataset. In the context of UAV-based detection tasks covered in this paper, models situated closer to the top-left corner are relatively superior. Two-stage models, such as Faster RCNN and Cascade RCNN, exhibit better detection precision compared to most single-stage models. However, their parameter count is an order of magnitude larger than many single-stage models, thus limiting their practical deployment on UAV platforms. Our approach not only boasts smaller parameters and superior detection precision compared to classic single-stage methods such as FCOS, GFL, Fovea, and Fsaf but also exhibits relatively higher detection precision when compared to the latest single-stage detection techniques with comparable parameters. Taking both detection performance and model parameter count into consideration; our model is optimal for UAV vehicle detection tasks compared to all benchmark models.

5. Conclusions

To address issues in UAV vehicle detection, such as background interference, drastic target scale changes, and high mis-detection rates for small-sample targets, we introduced an object detection model based on the Background Suppression Pyramid Network and the Multi-scale Task Adaptive Decoupled Head. This method integrates long-tail feature resampling, a background suppression pyramid network, and a multi-scale task adaptive decoupled head. Long-tail feature resampling addresses the class imbalance in the dataset, and the background suppression pyramid network minimizes the interference of background information on targets while enhancing the feature extraction capabilities of the model for vehicles, and the multi-scale task adaptive decoupled head reduces the sensitivity of the model to target scale. Experimental Results: The results demonstrate that our approach achieves the current best performance on small-parameter object detection networks, meeting both the performance and accuracy demands of UAV platforms. Compared to the baseline model PP-YOLOE-s, our model ${AP}_{50:95}$ reached 32.8%, an improvement of 1.9%. The average precision for three different sizes of targets ${AP}_{small}$, ${AP}_{medium}$ and ${AP}_{large}$ increased by 1%, 3%, and 3.2%, respectively, with the model parameter being 9.48 M. In the future, our work will continue to focus on dense small object detection and real-world deployment and testing on UAV platforms.