YOLOv7-UAV: An Unmanned Aerial Vehicle Image Object Detection Algorithm Based on Improved YOLOv7

Abstract

Detecting small objects in aerial images captured by unmanned aerial vehicles (UAVs) is challenging due to their complex backgrounds and the presence of densely arranged yet sparsely distributed small targets. In this paper, we propose a real-time small object detection algorithm called YOLOv7-UAV, which is specifically designed for UAV-captured aerial images. Our approach builds upon the YOLOv7 algorithm and introduces several improvements: (i) removal of the second downsampling layer and the deepest detection head to reduce the model’s receptive field and preserve fine-grained feature information; (ii) introduction of the DpSPPF module, a spatial pyramid network that utilizes concatenated small-sized max-pooling layers and depth-wise separable convolutions to extract feature information across different scales more effectively; (iii) optimization of the K-means algorithm, leading to the development of the binary K-means anchor generation algorithm for anchor allocation; and (iv) utilization of the weighted normalized Gaussian Wasserstein distance (nwd) and intersection over union (IoU) as indicators for positive and negative sample assignments. The experimental results demonstrate that YOLOv7-UAV achieves a real-time detection speed that surpasses YOLOv7 by at least 27% while significantly reducing the number of parameters and GFLOPs to 8.3% and 73.3% of YOLOv7, respectively. Additionally, YOLOv7-UAV outperforms YOLOv7 with improvements in the mean average precision (map (0.5:0.95)) of 2.89% and 4.30% on the VisDrone2019 and TinyPerson datasets, respectively.

1. Introduction

With the decrease in the cost of drones, the civilian drone market has entered a period of rapid development. At the same time, target detection technology based on deep learning has also made remarkable progress in recent years, which has made the combination of drones and target detection technology more closely related. The integration of the two can play an important role in many fields, such as crop detection [1], intelligent transportation [2], and search and rescue [3]. However, most target detection models are designed based on natural scene image datasets, and there are significant differences between natural scene images and drone aerial images. This makes it a meaningful and challenging task to design a target detection model specifically suitable from the aerial drone perspective.

In practical application scenarios, real-time target detection of the unmanned aerial vehicle (UAV) aerial video stream places a high demand on the detection speed of the algorithm model. Furthermore, unlike natural scene images, due to the high altitude of UAV flights and the existence of a large number of small targets in aerial images, there are fewer extractable features for these targets. In addition, the UAVs actual flight altitude often varies greatly, leading to drastic changes in object proportions and a low detection accuracy. Finally, complex scenes are often encountered during actual flight shooting, and there may be a large amount of occlusion between densely packed small targets, making them easily obscured by other objects or the background. In general, generic feature extractors [4,5,6] downsample the feature maps to reduce spatial redundancy and noise while learning high-dimensional features. However, this processing inevitably leads to the representation of small objects being eliminated. Additionally, in real-world scenarios, the background exhibits diversity and complexity, characterized by various textures and colors. Consequently, small objects tend to be easily confounded with these background elements, resulting in an increased difficulty of their detection. In summary, there is a need to design a real-time target detection model for UAV aerial photography that is suitable for dense small target scenarios in order to meet practical application requirements.

Object detection algorithms based on neural networks can generally be divided into two categories: two-stage detectors and one-stage detectors. Two-stage detection methods [7,8,9] first use region proposal networks (RPNs) to extract object regions, and then detection heads use region features as input for further classification and localization. In contrast, one-stage methods directly generate anchor priors on the feature map and then predict classification scores and coordinates. One-stage detectors have a higher computational efficiency but often lag behind in accuracy. In recent years, the YOLO series of detection methods has been widely used in object detection of UAV aerial images due to their fast inference speeds and good detection accuracies. YOLOv1 [10] was the first YOLO algorithm, and subsequent one-stage detection algorithms based on its improvements mainly include YOLOv2 [11], YOLOv3 [12], YOLOv4 [13], YOLOv5 [14], YOLOx [15], YOLOv6 [16], YOLOv7 [17], and YOLOv8 [18]. YOLO algorithms directly regress the coordinates and categories of objects, and this end-to-end detection approach significantly improves the detection speed without sacrificing much accuracy, which meets the basic requirements of real-time object detection for unmanned systems.

Previous improvement methods for target detection in UAV aerial images can be categorized into three types: (i) utilizing more shallow feature information, such as adding small target detection layers [19]; (ii) enhancing the feature extraction capability of the target detection network, such as improving the Neck network [20] or introducing attention mechanisms [21]; and (iii) increasing input feature information, such as generating higher resolution images [22], image copying [23], and image cropping [24,25].

Taking into consideration the aforementioned discussion, we propose a high-precision real-time algorithm, namely YOLOv7-UAV, for aerial image detection in unmanned aerial vehicles (UAVs). In summary, the contributions of this paper are as follows:

(1) We have optimized the overall architecture of the YOLOv7 model by removing the second downsampling layer and introducing an innovative approach to eliminate its final neck and detection head. This modification significantly enhances the utilization efficiency of the detection model in capturing shallow-level information.

(2) We present the DpSPPF module as an alternative to the SPPF module. It replaces the original max pooling layers with a concatenation of smaller-sized max pooling layers and depth-wise separable convolutions. This design choice enables a more detailed extraction of feature information at different scales.

(3) We propose the binary K-means anchor generation algorithm, which avoids the problem of local optimal solutions and increases the focus on sparse-sized targets by reasonably dividing the anchor generation range into intervals and assigning different numbers of anchors that need to be generated in each interval.

(4) Extensive experiments were conducted on both the VisDrone dataset and the TinyPerson dataset to validate the superiority of our proposed method over state-of-the-art real-time detection algorithms.

2. Related Work

2.1. YOLOv7

YOLOv7 is one of the most advanced single-stage object detection algorithms that satisfies both real time and high precision requirements. YOLOv7 incorporates several trainable bag-of-freebies, which can significantly enhance the detection accuracy without increasing the inference cost. It uses the “ex-tend” and “compound scaling” methods to improve the utilization of parameters and computational resources. YOLOv7 also incorporates improved re-parametrization modules and label assignment strategies. The YOLOv7 model is mainly composed of three parts: a backbone network (Backbone), a bottleneck layer network (Neck), and a detection network (Head). The backbone network includes standard convolutional layers, max pooling layers, Extended Efficient Layer Aggregation Networks (ELAN) modules, and SPPCSPC modules. The backbone network performs feature extraction, where the ELAN module increases the cardinality of newly added features using group convolution without altering the original gradient propagation path. It merges features from different groups by mixing and merging their cardinalities, which enhances the learned features from different feature maps and improves the usage of parameters and computations. The SPPCSPC module performs feature extraction through max-pooling with different pooling kernel sizes, which expands the model’s receptive field. To fuse feature information on different scales, the neck uses three different sized feature maps extracted from the backbone for feature fusion. This part still uses the PANet [26]structure based on FPN, which adds channels from shallow to deep networks. The model’s head can be viewed as the YOLOv7 classifier and regressor.

However, the YOLOv7 algorithm was not specifically designed to address small object datasets so it cannot be directly applied to the detection of aerial images from unmanned aerial vehicles (UAVs).

2.2. Spatial Pyramid Pooling

Spatial pyramid pooling (SPP) was proposed by Kaiming He et al. [27]. It aggregates features of different sizes by using pooling layers of different scales and produces an output with a fixed size. In the YOLO series, YOLOv4 was the first to incorporate the SPP structure. YOLOv5 replaced the three parallel maximum pooling layers in SPP with three concatenated maximum pooling layers of smaller sizes to obtain Spatial Pyramid Pooling-Fast (SPPF). The impacts of SPPF and SPP on neural network output results are nearly identical, but SPPF has a faster processing speed. YOLOv7 uses the SPPCSPC module, which is a fusion of SPP and CSPNet [28] modules. Compared to the SPP module, the SPPCSPC module can extract richer feature information, but it has a higher number of parameters and a higher computational complexity.

2.3. Anchor Generation Algorithm

Anchors were first introduced in Fast-RCNN as pre-defined bounding boxes that are used to label regions in an image that may contain objects. It aids in the precise and efficient localization of targets for object detection algorithms. During detection, anchor-based object detection models adjust the size of anchors and filter them to obtain the final predicted boxes. In the past, there have been two main approaches to obtaining anchors: one involves manual design, while the other involves clustering algorithms such as K-means and K-means++. In the YOLO series, the anchor mechanism was first introduced in YOLOv2. YOLOv3, YOLOv4, YOLOv5, and YOLOv7 object detection models employ a genetic algorithm to refine the anchor generated by the K-means algorithm.

2.4. Bounding Box Regression Loss Function

The bounding box regression loss function is an important component in object detection tasks which measures the difference between predicted detection boxes and true boxes. In early object detection methods, the Mean Square Error (MSE) loss function was a common choice, which calculates the squared error between the predicted coordinates of the detection box and the true coordinates of the box. However, the MSE loss function is highly sensitive to outliers. To address this issue, Fast R-CNN introduced the Smooth L1 loss function, which uses a square function when the error is small and a linear function when the error is large, while also possessing robustness.

IoU Loss is a loss function based on intersection over union (IoU), which optimizes the model by minimizing the IoU distance between the detection box and the true box, thereby more directly considering the degree of overlap between the detection boxes. GIoU Loss [29] is an improved version of IoU Loss, which not only considers the intersection and union of the two boxes but also considers the distance between their bounding boxes. Zhaohui Zheng et al. [30] proposed DIoU and CIoU. DIoU Loss is an improvement of GIoU Loss, which uses a more accurate distance metric in the calculation of the distance. CIoU Loss further considers the difference in aspect ratios based on DIou loss. Compared to CIoU Loss, EIoU Loss [31] directly considers the difference in length and width and SIoU Loss [32] adds considerations for the angle of the bounding box regression. Jinwang Wang et al. [33] pointed out that IoU is too sensitive to small object position deviations; thus, they designed an evaluation metric (nwd, normalized Gaussian Wasserstein distance) for small objects based on the Wasserstein distance.

3. Methods

3.1. YOLOv7-UAV

YOLOv7 is one of the most advanced single-stage object detection models, which comprises seven distinct versions: YOLOv7-tiny, YOLOv7, YOLOv7-X, YOLOv7-W6, YOLOv7-E6, YOLOv7-D6, and YOLOv7-E6E. Considering the trade-off between detection accuracy and speed, we selected the YOLOv7 model as the foundation for constructing the YOLOv7-UAV network architecture.

The overall structure of the YOLOv7-UAV model is illustrated in Figure 1, which differs from YOLOv7 in four aspects. In the following four subsections, we will introduce each of these four modifications separately in detail. It should be noted that in order to ensure a fair comparison, we performed an overall scaling of the channel numbers on the modified model to ensure the compared models had similar GFLOPs. The scaling of channel numbers is accomplished through the approach described in Equation (1). We denote the scaling factor for the portion of YOLOv7 that is located before the second downsampling layer as $W_1$, and the scaling factor for the remaining portion as $W_2$. The GFLOP calculation formula for the CNN neural network is shown in Equation (2). Removing the second downsampling layer in the YOLOv8 model results in a two-fold increase in the height and width of feature maps following the layer, leading to a significant rise in model GFLOPs. However, the model’s GFLOPs can be substantially reduced by reducing the number of feature map channels using a scaling factor W. Additionally, since removing the second downsampling layer does not affect the size of feature maps preceding it, different values of parameter W can be assigned to the the feature maps before and after the second downsampling layer. It is worth noting that the settings of W1 and W2 in this paper not only make the GFLOPs of the model similar before and after adjustment but also adhere to the ratio of GFLOPs between the second downsampling layer of the YOLOv7 model before and after the adjustment. Clearly, there are infinite combinations of W1 and W2 that satisfy this condition. However, due to experimental constraints, we only compared a few of them.

$$C_2=⌈C_1\times \frac{W}{8}⌉\times 8$$

(1)

where $C_1$ and $C_2$ are the number of channels in a neural network layer before and after scaling, respectively.

$$\mathrm{GFLOPs}=2\ast \mathrm{H}\ast \mathrm{W}\ast {\mathrm{k}}^2\ast {\mathrm{C}}_i\ast {\mathrm{C}}_o$$

(2)

where H and W represent the height and width of the output feature map, k denotes the size of the convolutional kernel, and $C_i$ and $C_o$ correspond to the channel numbers of the input and output feature maps, respectively.

3.1.1. Reducing the Receptive Field of YOLOv7

We removed the second downsampling layer in YOLOv7 in order to reduce the receptive field and mitigate the loss of fine-grained feature information caused by downsampling.

Despite the fact that deep-level feature information is beneficial for object classification, there exists a semantic gap between feature information extracted from different layers, and the overly large receptive field of deep networks is not conducive to detecting small objects. Thus, we removed the third detection head of the YOLOv7 model to enhance its utilization of fine-grained feature information. We then adjusted the model’s channel numbers by scaling them using $W_1$ = 0.75 and $W_2$ = 0.5.

3.1.2. Replacing SPPCSPC with DpSPPF

We believe that a large-sized maximum pooling layer will result in the loss of fine-grained feature information, which is detrimental to small object detection. Therefore, we proposed the DpSPPF moudle which replaces the maximum pooling layer (Kernel_size = 5) in the SPPF module with interconnected smaller depth-wise separable convolution (Kernel_size = 3) and max pooling layers (Kernel_size = 3). The structure of the DpSPPF module is illustrated in Figure 2. Subsequently, we incorporated the DpSPPF module into the deepest layer of the YOLOv7 model backbone that had undergone two modifications to aggregate the feature information of different scales.

3.1.3. Binary k-Means Anchor Generation Algorithm

In UAV-based image detection tasks, there exist targets of different sizes, and these targets are imbalanced in both the dataset and real-world scenarios. YOLOv7 initially generates anchors using the K-means algorithm and then applies the standard genetic algorithm to mutate these anchors based on their fitness, which is determined by the overlap between the generated anchors and the dimensions of all the targets in the training set. However, the k-means algorithm is highly influenced by initial points and outliers, which may result in the clustering results being only locally optimal. In addition, when the k-means algorithm is combined with the genetic algorithm in the anchor clustering process, it often focuses on samples with common sizes, while some samples with rare sizes may significantly deviate from the clustering results. To address these issues, we propose an improved anchor generation algorithm referred to as the “binary k-means anchor generation algorithm”.

The binary K-means prior anchor generation algorithm first obtains K cluster centers on the dataset using k-means and the genetic algorithm. Based on the width and height of the cluster anchor with the largest area, the algorithm divides the target size distribution interval for the dataset into three intervals to generate an anchor, requiring each region to generate at least one prior anchor. This helps the generated anchors to be closer to some rare sizes and reduces the probability of them becoming local optimal solutions. In addition, the algorithm determines the number of prior anchors to be generated in each interval based on the ratio between the number of cluster centers contained in each interval, so that more attention can be paid to samples with common sizes during the process of prior anchor generation. Its steps are shown in Algorithm 1.

The selection of a K value affects the degree of attention paid by the target detection algorithm to targets of different sizes, so the selection of a K value needs to be within an appropriate range. Through experiments in Section 3.1.2, we found that when generating six anchors on the VisDrone2019 dataset, as long as the value of K is within [11, 19], the effect of the binary K-means prior anchor generation algorithm is better than that of the K-means prior anchor generation algorithm. The default value of K for YOLOv7-UAV is 12, which was determined as the optimal value through testing on the VisDrone2019 dataset.

To illustrate more clearly the difference in clustering performance between the binary prior anchor generation algorithm and the approach that sequentially uses k-means and the genetic algorithm, we present in Figure 3a comparison of the two algorithms on the VisDrone2019 dataset and the TinyPerson dataset. From the figure, it can be observed that the anchors generated by the binary K-means prior box generation algorithm are more widely dispersed, yet they also place a greater emphasis on objects of common sizes.

Algorithm 1 Binary K-means anchor generation algorithm

Input: $T=[(w_1,h_1),(w_2,h_2),\dots (w_n,h_n)],K,k$

1: T: The object bounding box sizes collection of the dataset.

2: n: The number of objects in the dataset.

3: K: The number of clusters at the first cluster.

4: k: The number of anchors needed.

Output: $anchor=(a_1,a_2,\dots ,a_k)$

5: Consecutively using k-means and genetic algorithm to obtain K cluster centers on T,

listed in ascending order by their respective areas as $A_1\dots A_K$.

6: $N_1\leftarrow 0,N_2\leftarrow 0,N_3\leftarrow 0$. $N_1,N_2, and N_3$ represent the number of anchors allocated to

each interval to generate anchor, initially set to 0

7: $b\leftarrow (A_K\left[0\right]+A_K\left[1\right])/6$. It serves as the partition threshold for generating three

intervals to generate anchor.

8: for $i=1$ to K do

9:       if $A_i\left[0\right]<2\ast b\wedge A_i\left[1\right]<2\ast b$ then

10:           if $A_i\left[0\right]<b\wedge A_i\left[1\right]<b$ then

11:              $N_1\leftarrow N1+1$

12:           else

13:              $N_2\leftarrow N2+1$

14:           end if

15:       else

16:           $N_3\leftarrow N3+1$

17:       end if

18: end for

19: $T_1\leftarrow \left[\right],T_2\leftarrow \left[\right],T_3\leftarrow \left[\right]$. $T_1,T_2,T_3$ represent the sets of object bounding box

sizes within the three interval to generate anchor.

20: for $i=1$ to n do

21:       if $T\left[i\right]\left[0\right]<2\ast b\wedge T\left[i\right]\left[1\right]<2\ast b$ then

22:           if $T\left[i\right]\left[0\right]<b\wedge T\left[i\right]\left[1\right]<b$ then

23:              $T_1.append\left(T\left[i\right]\right)$

24:           else

25:              $T_2.append\left(T\left[i\right]\right)$

26:           end if

27:       else

28:           $T_3.append\left(T\left[i\right]\right)$

29:       end if

30: end for

31: for $i=1$ to 3 do

32:       if $Ni\ast k/K<1$ then

33:           $ki\leftarrow 1,k\leftarrow k-1,K\leftarrow K-Ni$

34:       end if

35:       if $ki\ne 1$ then

36:           $ki\leftarrow \lfloor Ni\ast k/K+0.5\rfloor$

37:           $k\leftarrow k-ki,K\leftarrow K-Ni$

38:       end if

39: end for

40: Successively apply K-means and genetic algorithm for clustering on the

$T_1(k=k_1)$,$T_2(k=k_2)$, and $T_3(k=k_3)$ to get the anchor, respectively.

3.1.4. Nwd and Positive/Negative Sample Allocation Strategy

YOLOv7 determines the number of positive samples required (k) for each ground truth object by summing the top 10 IoU scores. The model then selects the top k samples with the smallest cost (the cost is the sum of the classification loss and the regression loss, added in a ratio of 1:3) for each ground truth object as positive samples. Due to the excessive sensitivity of the IoU to size deviations of small objects, this approach leads to an insufficient number of positive samples being assigned to small target ground truths during the training process of object detection networks.

The normalized Wasserstein distance (nwd) is a novel method for evaluating small object detection, which models the bounding boxes as two-dimensional Gaussian distributions and measures the similarity between predicted and ground truth objects, regardless of their overlap. The nwd is less affected by the scale of objects, making it particularly suitable for evaluating small objects. In YOLOv7’s positive/negative sample allocation strategy, we use the weighted nwd metric and IoU metric instead of the IoU metric. The IoU loss is retained, as it is more suitable for medium- to large-sized objects. The computation process of the nwd is shown in Equation (3). After experimental tuning, we set nwd:IoU = 0.5:0.5 for the TinyPerson dataset, and nwd:IoU = 0:1.0 for the VisDrone2019 dataset.

$$nwd\left({\mathcal{N}}_a,{\mathcal{N}}_b\right)=exp\left(-\frac{\sqrt{W_2^2\left({\mathcal{N}}_a,{\mathcal{N}}_b\right)}}{C}\right)$$

(3)

where C is a constant related to the dataset (we adopted the same setting of C = 12.8 as in the original paper [33]), $W_2^2\left({\mathcal{N}}_a,{\mathcal{N}}_b\right)$ is a distance metric which is computed using Equation (4), and $N_a$ and $N_b$ are Gaussian distributions modeled by $B_a=(x_a,c_a,w_a,h_a)$ and $B=(x_b,y_b,w_b,h_b)$.

$$\begin{array}{c}\hfill W_2^2\left({\mathcal{N}}_a,{\mathcal{N}}_b\right)={∥\left({\left[x_a,y_a,\frac{w_a}{2},\frac{h_a}{2}\right]}^{\mathrm{T}},{\left[x_b,y_b,\frac{w_b}{2},\frac{h_b}{2}\right]}^{\mathrm{T}}\right)∥}_2^2\end{array}$$

(4)

4. Experiments

4.1. Datasets and Evaluation Metrics

4.1.1. Datasets

The VisDrone2019 [34] dataset consists of a large number of annotated images captured by drones, with a total of 7019 images divided into 10 classes. The training and validation sets contain 6471 and 548 images, respectively. This dataset mainly contains small- and medium-sized targets.

The TinyPerson [35] dataset consists of 1610 images with a total of 72,651 annotated bounding boxes, mainly focusing on small objects. The images in this dataset were mainly captured by unmanned aerial vehicles and are categorized into two groups, namely sea_person and earth_person. The training and testing sets of the dataset comprise 794 and 816 images, respectively. There are a few annotation boxes in TinyPerson that can be ignored, including densely packed crowds that are difficult to separate, ambiguous regions, and shadow regions in water. These annotation boxes were replaced by the mean value of the image region in [35], and we simply ignore them.

4.1.2. Evaluation Metrics

We evaluated the performance of the object detection algorithm using four metrics, namely mean average precision (map), GFLOPs, Frames Per Second (FPS), and parameters. Map is computed using Equation (5), map0.5 denotes the map calculated at an IoU threshold of 0.5 and map(0.5:0.95) denotes the map scores across 10 IoU thresholds ranging from 0.5 to 0.95 with a step size of 0.05. GFLOPs can quantify the computational complexity of the model, parameter can measure the size of the model, and FPS represents the actual inference speed of the model.

$$mAP=\frac{{\sum }_{n=1}^N{\int }_0^{1}P\left(R\right)dR}{N}$$

(5)

where N represents the total number of categories, P represents precision, and R represents recall.

4.2. Experimental Details

Unless otherwise stated, all experiments were performed with NVIDIA RTX3080ti GPU, a Linux operating system, Pytorch1.12.1, and cuda11.6.

In the experiment conducted on the TinyPerson dataset, we set the number of epochs, batch size, and image input dimensions to 150, 4, and 960 ∗ 960, respectively. In the experiment conducted on the VisDrone2019 dataset, we set the number of epochs, batch size, and image input dimensions to 300, 4, and 640 ∗ 640, respectively. The iteration numbers of the K-means algorithm and genetic algorithm were set to 30 and 1000, respectively. The object detection model was evaluated for FPS on the corresponding test set of the dataset on which it was trained.

4.3. Ablation Experiments

We have divided the construction process of YOLOv7-UAV into four steps: In step 1, we removed the second downsampling layer and the deepest detection head of YOLOv7. Then, we reduced the number of channels in the model using Equation (1) with weights $W_1=0.75$ and $W_2=0.5$. In step 2, we introduced the DpSPP module. In step 3, we utilized the binary K-means anchor generation algorithm. In step 4, we used the weighted nwd and IoU instead of the IoU in the positive and negative sample allocation strategy. In the tables of each subsection in this chapter, the experimental settings highlighted in bold will serve as the baseline for the subsequent subsection.

Table 1. Impact of each step evaluated on VisDrone2019.

Steps	map(0.5:0.95)	map0.5	FPS	GFLOPs
YOLOv7	27.79%	48.94%	70	103.3
Step 1	29.74%	51.24%	89	72.5
Step 2	30.38%	51.94%	87	76.8
Step 3	30.68%	52.21%	89	76.8

and

Table 2. Impact of each step evaluated on TinyPerson.

Steps	map(0.5:0.95)	map0.5	FPS
YOLOv7	1.74%	7.12%	64
Step 1	4.98%	16.98%	90
Step 2	5.18%	17.67%	86
Step 3	5.40%	18.27%	86
Step 4	6.04%	20.56%	86

illustrate the changes in the performance of the object detection model during the construction process of YOLOv7-UAV on the VisDrone2019 and TinyPerson datasets, respectively. The results shown in these two tables indicate that each step of improvement we made to YOLOv7 was effective. In the following four sections, we will further introduce the ablation experiments conducted for each step and compare YOLOv7-UAV with other advanced object detection algorithms. Due to the presence of a large number of intermediate-sized objects in VisDrone2019, we did not apply step 4 to the models trained on VisDrone2019.

4.3.1. Comparison of the Effects of Transforming the Model Structure

We present the impact of changes to the model architecture on the detection model in

Table 3. Impact of changes in the YOLOv7s architecture evaluated on VisDrone2019 (‘dsp’ refers to the second downsampling layer).

Change	${\mathit{W}}_1$, ${\mathit{W}}_2$	map(0.5:0.95)	map0.5	FPS	GFLOPs	Params/M
-dsp	1.0, 0.5	30.68%	52.19%	67	99.8	9.18
-dsp-head	0.825, 0.55	30.74%	52.42%	87	96.1	3.49
-dsp	0.87, 0.435	30.04%	51.50%	71	79.8	7.06
-dsp-head	0.75, 0.5	29.74%	51.24%	89	72.5	2.74
-dsp-head+sppf	0.75, 0.5	30.03%	51.44%	87	76.8	3.07
-dsp-head+sppcspc	0.75, 0.5	30.35%	51.92%	84	96.0	4.57
-dsp-head+dpsppf	0.75, 0.5	30.38%	51.94%	87	76.8	3.07

and

Table 4. Impact of changes in the YOLOv7s architecture evaluated on TinyPerson (‘dsp’ refers to the second downsampling layer).

Change	${\mathit{W}}_1$, ${\mathit{W}}_2$	map(0.5:0.95)	map0.5	FPS
-dsp	1.0, 0.5	4.09%	14.75%	57
-dsp-head	0.825, 0.55	5.09%	17.15%	80
-dsp	0.87, 0.435	3.45%	12.21%	58
-dsp-head	0.75, 0.5	4.98%	16.98%	90
-dsp-head+sppf	0.75, 0.5	4.97%	17.15%	88
-dsp-head+sppcspc	0.75, 0.5	4.81%	16.65%	85
-dsp-head+dpsppf	0.75, 0.5	5.18%	17.67%	86

. In order to facilitate a fair comparison, we ensured that the experimental setups had similar GFLOP levels by adjusting the overall channel numbers of the models.

The comparative results of the first and third rows, as well as those of the second and fourth rows in both tables, demonstrate that reducing the detection head located at the deepest position of YOLOv7 not only significantly increases the speed of model detection, but also greatly improves the accuracy of detecting unmanned aerial vehicle (UAV) images. The comparative results from the fifth to the seventh rows in both tables indicate that, in terms of both the detection speed and accuracy, the DpSPPF module achieves the best performance compared to the popular SPPF and SPPCSPC modules. The experimental setup in the seventh row of both tables corresponds to step 2 as described in Table 1 and Table 2.

4.3.2. The Impact of the K Value in the Binary k-Means Anchor Generation Algorithm

We conducted an empirical study on the impact of the choice of the K value in the first iteration of the binary K-means anchor generation algorithm on the performance of the object detection algorithm. The results, as shown in

Table 5. The impact of the K value in the binary k-means anchor generation algorithm on VisDrone2019.

K = ?	map(0.5:0.95)	Anchor
step 2	30.38%	[4,5, 5,10, 11,8, 10,18, 24,14, 34,33]
10	30.22%	[4,5, 5,9, 12,11, 14,24, 25,14, 45,32]
11	30.60%	[4,5, 6,12, 11,7, 14,15, 27,22, 57,39]
12	30.68%	[4,5, 6,12, 12,7, 14,15, 27,23, 59,40]
13	30.61%	[4,5, 6,12, 12,8, 15,16, 31,24, 64,43]
14	30.58%	[4,5, 6,12, 13,8, 16,17, 34,26, 69,46]
15	30.47%	[4,5, 7,12, 14,8, 19,19, 41,30, 78,51]
17	30.48%	[4,5, 7,12, 14,8, 19,20, 42,31, 85,55]
19	30.39%	[4,6, 12,8, 8,15, 22,20, 48,34, 91,56]

, indicate that as the value of K increases, the generated range of anchors tends to become larger. Furthermore, the binary K-means anchor generation algorithm performs better when the value of K falls within the range of [11, 19]. Specifically, the results in the table also indicate that the object detection algorithm achieves the highest map (0.5:0.95) with K = 12 (i.e., step 3 in Table 1 and Table 2).

In

Table 6. Impact of the binary K-means anchor generation algorithm on several detection models.

Dataset	Model	map(0.5:0.95)	Binary K-Means Anchor
VisDrone	YOLOv5-l	+0.84%	[3,5, 7,7, 8,15, 17,9, 20,20, 42,20, 23,42, 45,40, 80,52]
	YOLOv7	+0.26%	[3,5, 7,7, 8,15, 17,9, 20,20, 42,20, 23,42, 45,40, 80,52]
	Step2	+0.30%	[4,5, 6,12, 12,7, 14,15, 27,23, 59,40]
TinyPerson	YOLOv5-l	+0.11%	[2,3, 3,6, 5,5, 4,9, 8,9, 7,16, 15,18, 20,40, 38,81]
	YOLOv7	+0.19%	[2,3, 3,6, 5,5, 4,9, 8,9, 7,16, 15,18, 20,40, 38,81]
	Step2	+0.22%	[2,4, 4,6, 5,10, 10,14, 15,32, 32,67]

, we present the impact of the binary K-means anchor generation algorithm on several popular object detection models. For the object detection algorithm with nine anchors, we simply set K to 18 by taking $12\times K_1/K_2=12\times 9/6=18$. As demonstrated in this table, all detection algorithms exhibited better performances on both datasets, indicating the excellent generalization ability of the binary K-means anchor generation algorithm.

4.3.3. The Impact of Different Weights of nwd and IoU

Table 7. Impact of using different weights of nwd and IoU.

Dataset	nwd:IoU	map(0.5:0.95)	map0.5
VisDrone	0:1	30.68%	52.21%
	0.05:0.95	−0.02%	−0.18%
	0.1:0.9	−0.02%	−0.27%
	0.2:0.8	−0.44%	−0.28%
TinyPerson	0:1	5.40%	18.27%
	0.2:0.8	+0.18%	+0.61%
	0.35:0.65	+0.28%	1.14%
	0.5:0.5	+0.64%	+2.29%
	0.65:0.35	+0.45%	+1.24%
	0.8:0.2	+0.31%	+1.34%

presents the impact of using different weights of nwd and IoU in the positive–negative sample allocation strategy on the detection performance. The results in the table indicate that on the TinyPerson dataset, setting nwd:IoU = 0.5:0.5 (i.e., step 4 in Table 2) yields the best performance, while the VisDrone dataset, which contains a large number of medium-sized objects, does not require the use of the nwd.

4.3.4. Algorithm Comparison

The detection performance comparison of YOLOv7-UAV and other state-of-the-art real-time object detection algorithms on a UAV dataset is presented in

Table 8. Performance of the YOLOv7-UAV algorithm and other object detection algorithms.

Dataset	Model	map(0.5:0.95)	map0.5	FPS	GFLOPs	Parameter/M
VisDrone	YOLOv5l	22.83%	39.62%	55	107.8	46.16
	YOLOv6m	21.65%	31.70%	56	82.0	34.24
	tph-YOLOv5	27.63%	46.39%	32	145.7	60.43
	YOLOv7	27.79%	48.94%	70	103.3	36.53
	YOLOv8	27.88%	45.25%	73	164.9	46.61
	YOLOv8m-p2	29.76%	48.29%	68	98.0	25.04
	YOLOv7-UAV	30.68%	52.21%	89	76.8	3.07
TinyPerson	YOLOv5l	2.08%	8.93%	52	107.7	46.11
	YOLOv6m	2.87%	9.27%	51	82.0	34.23
	tph-YOLOv5	2.33%	9.57%	34	145.3	60.36
	YOLOv7	1.74%	7.12%	64	103.2	36.49
	YOLOv8	4.73%	14.4%	68	164.7	46.58
	YOLOv8m-p2	5.81%	17.87%	63	97.9	25.03
	YOLOv7-UAV	6.04%	20.56%	86	76.7	3.07

. The algorithm tph-YOLOv5 in the table is specifically designed for detecting small targets in UAV imagery, while YOLOv8m-p2 is a version of YOLOv8 specifically designed for small object detection. The results indicate that YOLOv7-UAV outperforms its counterparts in both detection speed and accuracy.

Figure 4 illustrates the detection results of YOLOv7 and YOLOv7-UAV. Both models were trained on the training set of the VisDrone2019 dataset. It can be observed that YOLOv7-UAV has a lower false negative rate and generally higher confidence levels compared to YOLOv7.

5. Conclusions

There exists a significant challenge for existing detection algorithms in detecting a large number of small targets with diverse shooting angles in unmanned aerial vehicle (UAV) images. This paper proposes an algorithm, YOLOv7-UAV, which can detect UAV images in real time. Firstly, the algorithm reduces the loss of feature information and improves the model’s utilization efficiency of fine-grained feature information by removing the second downsampling layer and the deepest detection head of the YOLOv7 model. Then, the algorithm replaces the maximum pooling layer in the SPPF module with concatenated smaller depth-wise separable convolution and maximum pooling layers, optimizing its ability to extract fine-grained feature information while retaining the ability to aggregate multi-scale feature information. Subsequently, the paper proposes a binary K-means anchor generation algorithm that reasonably divides the anchor box generation interval and retains a focus on common sizes to obtain better anchors. Finally, YOLOv7-UAV introduces the weighted nwd and IoU as evaluation metrics in the label assignment strategy on the TinyPerson dataset. Results on the VisDrone2019 and TinyPerson datasets demonstrate that YOLOv7-UAV outperforms most popular real-time detection algorithms in terms of the detection accuracy, detection speed, and memory consumption.

Despite the excellent performance of our proposed method on the unmanned aerial vehicle object detection datasets, there are still some limitations. Specifically, the effectiveness of YOLOv7-UAV on low-power platforms, such as embedded devices, requires further testing and optimization. Moreover, the determination of the K value in the binary K-means anchor generation algorithm is solely based on experimental results, without a sufficient analysis of the factors influencing K. In real-world unmanned aerial vehicle tasks, adverse weather conditions such as fog and darkness may be encountered, which our method has not been specifically optimized for. Additionally, unmanned aerial vehicles can be equipped with camera systems that have a larger field of view (FOV), which introduces radial and barrel distortions that significantly impact the detection of small targets, an aspect that our proposed algorithm has not specifically addressed.

In future work, we plan to optimize the performance of YOLOv7-UAV on low-power platforms by employing model compression techniques such as model pruning and distillation. Moreover, we aim to improve the process of generating anchors using the binary K-means algorithm by directly determining an appropriate K value based on the distribution of data within the dataset and the receptive fields of the target detection model. We plan to incorporate advanced generative networks or construct more abundant datasets to enhance the performance of the YOLOv7-UAV algorithm in challenging environments such as those with dense fog or low light conditions. Additionally, we intend to investigate the rectification issues of wide-angle cameras and devise targeted data augmentation methods to enhance the detection performance of object detection algorithms on images captured with a larger FOV.