An Effective and Lightweight Full-Scale Target Detection Network for UAV Images Based on Deformable Convolutions and Multi-Scale Contextual Feature Optimization

Abstract

Currently, target detection on unmanned aerial vehicle (UAV) images is a research hotspot. Due to the significant scale variability of targets and the interference of complex backgrounds, current target detection models face challenges when applied to UAV images. To address these issues, we designed an effective and lightweight full-scale target detection network, FSTD-Net. The design of FSTD-Net is based on three principal aspects. Firstly, to optimize the extracted target features at different scales while minimizing background noise and sparse feature representations, a multi-scale contextual information extraction module (MSCIEM) is developed. The multi-scale information extraction module (MSIEM) in MSCIEM can better capture multi-scale features, and the contextual information extraction module (CIEM) in MSCIEM is designed to capture long-range contextual information. Secondly, to better adapt to various target shapes at different scales in UAV images, we propose the feature extraction module fitting different shapes (FEMFDS), based on deformable convolutions. Finally, considering low-level features contain rich details, a low-level feature enhancement branch (LLFEB) is designed. The experiments demonstrate that, compared to the second-best model, the proposed FSTD-Net achieves improvements of 3.8%, 2.4%, and 2.0% in AP₅₀, AP, and AP₇₅ on the VisDrone2019, respectively. Additionally, FSTD-Net achieves enhancements of 3.4%, 1.7%, and 1% on the UAVDT dataset. Our proposed FSTD-Net has better detection performance compared to state-of-the-art detection models. The experimental results indicate the effectiveness of the FSTD-Net for target detection in UAV images.

1. Introduction

Target detection represents one of the most important fundamental tasks in the domain of computer vision, attracting substantial scholarly and practical interest in recent years. The core objective of target detection lies in accurately identifying and classifying various targets within digital images, such as humans, vehicles, or animals, and precisely determining their spatial locations. The advent and rapid evolution of deep learning technologies have significantly propelled advancements in target detection, positioning them as a focal point of contemporary research with extensive applications across diverse sectors [1,2]. Notably, drones, leveraging their exceptional flexibility, mobility, and cost-effectiveness, have been integrated into a wide array of industries. These applications span emergency rescue [3], traffic surveillance [4], military reconnaissance [5], agricultural monitoring [6], and so on.

For natural images, target detection technology has made considerable progress. This technology can be divided into single-stage and two-stage detection methods. Two-stage detection methods primarily include region-based target detection approaches such as R-CNN [7], Fast R-CNN [8], Faster R-CNN [9], and so on. Single-stage detection methods encompass techniques like YOLO, SSD, and DETR [10,11]. Natural images are typically captured from a ground-level perspective, with a relatively fixed viewpoint and minimal changes in the shapes and sizes of targets. In contrast, drone images are usually taken from high altitudes, offering a bird’s-eye view. Under this perspective, especially when the drone’s flight altitude varies, the shapes and sizes of targets can change significantly. As shown in Figure 1, we analyzed the number of small, medium, and large targets in the VisDrone 2019 and UAVDT datasets (the latter being divided into training, testing, and validation sets for clarity). The results indicate that the training, testing, and validation sets of both the VisDrone 2019 and UAVDT datasets contain targets of various sizes, showing significant differences in target scale distribution.

Moreover, targets in the drone images often have more complex backgrounds due to variations in lighting, weather conditions, and surrounding environments. As shown in Figure 2a, the buildings, trees, and other non-target categories presented in the images, along with the dim lighting conditions, make target detection in UAV images more challenging. For Figure 2b, the occlusion by trees, shadows, and adverse weather conditions introduces more challenges to the target detection task. Consequently, the significant scale differences and complex backgrounds make it challenging to directly apply existing detection models designed for natural images to UAV images.

In this paper, to address the challenges of significant scale variability of targets and complex backgrounds in UAV images, we propose the UAV target detection network, FSTD-Net. The design of FSTD-Net is based on three principal aspects. Firstly, to capture more effective multi-level features and avoid the interference of complex backgrounds, the multi-scale and contextual information extraction module (MSCIEM) is designed. Secondly, to extract full-scale object information that can better represent targets of various scales, we designed the feature extraction module fitting different shapes (FEMFDS), which is guided by the deformable convolutions. The positions of the sampling points in deformable convolutions can be dynamically adjusted during the learning process; the FEMFDS is positioned before the detection heads for different scales, enabling it to effectively capture full-scale target features of different target shapes. Finally, to fully utilize the detailed information contained in the low-level features, we propose the low-level feature enhancement branch (LLFEB), which uses a top-down and bottom-up feature fusion strategy to incorporate low-level semantic features. In summary, the main contributions are as follows:

(1)

The MSCIEM is proposed to optimize the semantic features of targets on different scales while avoiding the interference of complex backgrounds. MSIEM employs parallel convolutions with different kernel sizes to capture the features of targets on different scales. The CIEM embedded in MSCIEM is to capture long-range contextual information. Thus, the MSCIEM can effectively capture multi-scale and contextual information.

(2)

We design the FEMFDS, which is guided by the deformable convolutions. Due to the FEMFDS breaking through the conventional feature extraction paradigm of standard convolution, the FEMFDS can dynamically match full-scale target shapes.

(3)

Due to the rich, detailed features in low-level features, such as edges and textures, we designed the LLFEB. The LLFEB uses a top-down and bottom-up feature fusion strategy to incorporate low-level and high-level features, which can effectively utilize the detailed information in low-level features.

(4)

Experiments on the VisDrone2019 and UAVDT dataset show that the proposed FSTD-Net surpasses the state-of-the-art detection models. Our proposed FSTD-Net can achieve more accurate detection of targets at various scales and in a complex background. In addition, FSTD-Net is lightweight. These experiments demonstrate the superior performance of the proposed FSTD-Net.

This paper is structured into six distinct sections. Section 2 reviews the related work pertinent to this work. Section 3 elaborates on the design details of the FSTD-Net. Section 4 presents the parameter settings of the model training process, training environment, and experimental results. Section 5 provides an in-depth discussion of the advantages of FSTD-Net and the impacts of the cropping and resizing methods designed for the experiments. Finally, Section 6 summarizes the current work and proposes directions for future research.

2. Related Work

2.1. YOLO Object Detection Models and New Structure Innovations

The YOLO series of target detection models has played a significant role in advancing the task of target detection in natural images. YOLOv1 initially proposed an end-to-end target detection framework, treating target detection as a regression problem [12]. Subsequent versions, such as YOLOv2 [13], YOLOv3 [14], YOLOv4 [15], and so on, continually optimized network architectures and training methods, leading to improvements in detection accuracy and speed. YOLOv2 improves YOLOv1 by using anchor boxes for better target size and shape handling, higher resolution feature maps for improved localization, and batch normalization for faster convergence and enhanced detection performance. YOLOv3 enhances YOLOv2 by using multi-scale predictions, a ResNet-based architecture, and logistic regression classifiers, improving small target detection, feature extraction, and suitability for multi-label classification, thus boosting accuracy and speed. YOLOv4 introduced the Bag of Specials and Bag of Freebies to enhance model expressiveness. YOLOv5 absorbed state-of-the-art structures and added modules that are easy to study. YOLOv6 customized networks of different scales for various applications and adopted a self-correcting strategy [16]. YOLOv7 addressed some issues through trainable free packets and expanded the approach [17]. YOLOv8 employed the latest C2f structure and PAN structure, with a decoupled form for the detection head [18]. The YOLOv9 further improved the model structure, resolving the issue of information loss [19]. The latest YOLOv10 further advances the performance-efficiency boundary of YOLOs through improvements in both post-processing techniques and model architecture [20]. These enhancements have significantly advanced the YOLO series models in target detection tasks.

Feature Pyramid Networks (FPN) are widely adopted in target detection frameworks, including the YOLO series, and have been proven to significantly enhance detection performance [21]. Recently, there have been several studies focused on improving FPN structure. DeNoising FPN (DN-FPN), proposed by Liu et al, employs contrastive learning to suppress noise at each level of the feature pyramid in the top-down pathway of the FPN [22]. GraphFPN is capable of adjusting the topological structure based on different intrinsic image structures, achieving feature interaction at the same and different scales in contextual layers and hierarchical layers [23]. AFPN enables feature fusion across non-adjacent levels to prevent feature loss or degradation during transmission and interaction. Additionally, through adaptive spatial fusion operations, AFPN suppresses information conflicts between features at different levels [24].

Furthermore, several enhancements to standard convolutional feature extraction have been proposed recently, with deformable convolution garnering significant attention. Standard convolution has a fixed geometric structure, which is detrimental to feature extraction. However, the Deformable Convolution series can overcome this limitation of the CNN building blocks. When first introduced, Deformable Convolution (DCNv) introduced deformable convolution kernels and deformable ROI pooling to enhance the model’s modeling capability, and it can automatically learn offsets in target tasks without additional supervision [25]. DCNv2 addresses the issues of DCNv by further introducing a modulation mechanism to adjust the spatial support regions, avoiding irrelevant information affecting the features, and achieving better results compared to the original DCNv [26]. DCNv3 further extends DCNv2 by weight sharing among convolutional neurons and introducing multiple sets of mechanisms and per-sample normalization modulation scalars [27]. DCNv4 addresses the limitations of DCNv3 by eliminating soft-max normalization in spatial aggregation to enhance its dynamic properties and expressive power. Then, it optimizes memory access to minimize redundant operations and improve speed [28].

2.2. Methods of Mitigating the Impact of Complex Backgrounds on Target Detection

To mitigate the impact of complex backgrounds on target detection, Liu et al. designed a center-boundary dual attention (CBDA) module within their network, employing dual attention mechanisms to extract attention features from target centers and boundaries, thus learning more fundamental characteristics of rotating targets and reducing background interference [29]. Shao et al. developed a spatial attention mechanism to enhance target feature information and reduce interference from irrelevant background information [30]. Wang et al. introduced the target occlusion contrast module (TOCM) to improve the model’s ability to distinguish between targets and non-targets [31]. To alleviate interference caused by complex backgrounds, Zhang et al. designed a global-local feature guidance (GLFG) module that uses self-attention to capture global information and integrate it with local information [32]. Zhou proposed a novel encoder-decoder architecture, combining a transformer-based encoder with a CNN-based decoder, to address the limitations of CNNs in global modeling [33]. Zhang et al. constructed a spatial location attention (SLA) module to highlight key targets and suppress background information [34]. Gao et al. introduced a multitask enhanced structure (MES) that injects prominent information into different feature layers to enhance multi-scale target representation [35]. Gao et al. also designed the hidden recursive feature pyramid network (HRFPN) to improve multi-scale target representation by focusing on information changes and injecting optimized information into fused features [36]. Furthermore, Gao et al. designed an innovative scale-aware network with a plug-and-play global semantic interaction module (GSIIM) embedded within the network [37]. Dong et al. developed a novel multiscale deformable attention module (MSDAM), embedded in the FPN backbone network, to suppress background features while highlighting target information [38].

2.3. Methods of Addressing the Impact of Significant Scale Variations on Target Detection

To address the impact of significant scale variations on target detection, Shen et al. combined inertial measurement units (IMU) to estimate target scales and employed a divide-and-conquer strategy to detect targets of different scales [39]. Jiang et al. enhanced the model’s ability to perceive and detect small targets by designing a small target detection head to replace the large target detection head [40]. Liu et al. designed a widened residual block (WRB) to extract more features at the residual level, improving detection performance for small-scale targets [41]. Lan et al. developed a multiscale localized feature aggregation (MLFA) module to capture multi-scale local feature information, enhancing small target detection [42]. Mao et al. proposed an efficient receptive field module (ERFM) to extract multi-scale information from a variety of feature maps, mitigating the impact of scale variations on small target detection performance [43]. Hong et al. developed the scale selection pyramid network (SSPNet) to enhance the detection ability of small-scale targets in images [44]. Cui et al. created the context-aware block network (CAB Net), which establishes high-resolution and strong semantic feature maps, thereby better detecting small-scale targets [45]. Yang et al. introduced the QueryDet scheme, which utilizes a novel query mechanism to speed up the inference of feature pyramid-based target detectors by first predicting rough locations of small targets on low-resolution features and then using these rough locations to guide high-resolution features for precise detection results [46]. Zhang et al. proposed the semantic feature fusion attention (FFA) module, which aggregates information from all feature maps of different scales to better facilitate information exchange and alleviate the challenge of significant scale variations [47]. Nie et al. designed the enhanced context module (ECM) and triple attention module (TAM) to enhance the model’s ability to utilize multi-scale contextual information, thereby improving multi-scale target detection accuracy [48]. Wang et al. introduced a lightweight backbone network, MSE-Net, which better extracts small-scale target information while also providing good feature descriptions for targets of other scales [49]. Cai et al. designed a poly kernel inception network (PKINet) to capture features of targets more effectively at different scales, thereby addressing the significant variations in target scales more efficiently [50].

3. Methods

In this section, the baseline model YOLOv5s and the proposed FSTD-Net will be introduced. Firstly, the overall framework of the baseline model YOLOv5s will be introduced. Then, we will give an overall introduction to the FSTD-Net. Finally, we will discuss the components of FSTD-Net in detail: the LLFEB, which can provide sufficient detailed information from low-level features; the MSCIEM, which can effectively capture target features at different scales; and long-range context information. The FEMFDS, which is based on deformable convolution, is sensitive to full-scale targets of different scales.

3.1. The Original Structure of Baseline YOLOv5s

YOLOv5 is a classic single-stage target detection algorithm known for its fast inference speed and good detection accuracy. It comprises five different models: n, s, m, l, and x, which can meet the needs of various scenarios. As shown in Figure 3, the YOLOv5s architecture consists of a backbone for feature extraction, which includes multiple CBS modules composed of convolutional layers, batch normalization layers, and SiLU activation functions, as well as C3 modules for complex feature extraction (comprising multiple convolutional layers and cross-stage partial connections). At the end of the backbone, global feature extraction is performed using the SPPF module. The SPPF (Spatial Pyramid Pooling-Fast) module in YOLOv5 serves the purpose of enhancing the receptive field and capturing multi-scale spatial information without increasing the computational burden significantly. By applying multiple pooling operations with different kernel sizes on the same feature map and then concatenating the pooled features, SPPF effectively aggregates contextual information from various scales. This helps in improving the model’s ability to detect targets of different sizes and enhances the robustness of feature representations, contributing to better overall performance in target detection tasks.

The Neck incorporates both FPN and PANet [51] structures, employing concatenation and nearest-neighbor interpolation for upsampling to achieve feature fusion. FPN is utilized to build a feature pyramid that enhances the model’s ability to detect targets at different scales by combining low-level and high-level feature maps. PANet further improves this by strengthening the information flow between different levels of the feature pyramid, thus enhancing the model’s performance in detecting small and large targets.

The Head consists of several convolutional layers that transform the feature maps into prediction outputs, providing the class, bounding box coordinates, and confidence for each detected target. In the YOLOv5s, the detection head adopts a similar structure to YOLOv3 due to its proven efficiency and effectiveness in real-time target detection. This structure supports multi-scale predictions by detecting targets at three different scales, handles varying target sizes accurately, and utilizes anchor-based predictions to generalize well across different target shapes and sizes. Its simplicity and robustness make it easy to implement and optimize, leading to faster training and inference times while ensuring high performance and reliability across diverse applications.

3.2. The Overall Framework of FSTD-Net

In this paper, our proposed FSTD-Net is based on the YOLOv5s architecture, as illustrated in Figure 4. The structure of FSTD-Net includes three main components: The Backbone, the Neck, and the Head, all integrated into an end-to-end framework for efficient and precise target detection. The Backbone is responsible for feature extraction, leveraging YOLOv5s for its lightweight design, which balances performance with computational efficiency. The Neck further processes and fuses these features from different levels of the backbone, enhancing the network’s ability to recognize targets of varying scales and complex backgrounds through multi-scale feature fusion. The Head, employing a detection mechanism similar to YOLOv3, accurately identifies and localizes target categories and positions using regression methods, ensuring high accuracy and reliability. Overall, FSTD-Net’s design combines lightweight architecture with multi-scale feature integration, achieving high detection accuracy while maintaining low computational complexity and high processing efficiency, making it suitable for UAV target detection tasks.

For FSTD-Net, the Backbone adopts the CSP-Darknet network structure, which effectively extracts image features. In the Neck part, to better suppress complex background information in UAV images and extract information from targets of different scales, the MSCIEM replaces the original C3 module. The MSCIEM is mainly composed of MSIEM and CIEM. MSIEM utilizes multiple convolutional kernels with different receptive fields to better extract information from targets on different scales. Meanwhile, the context information extraction module (CIEM) is embedded in the MSCIEM to extract long-distance contextual semantic information. To improve the detection of targets of different scales in UAV images, the FEMFDS module breaks through the inherent feature extraction method of standard convolution. It can better fit the shapes of targets of different scales and extract more representative features. The designs of MSCIEM and FEMFDS aim to enhance the detection performance of FSTD-Net for targets of different sizes. Furthermore, considering that low-level features contain rich, detailed features such as edges, corners, and texture features, the LLFEB is separately designed to better utilize low-level semantic features and enhance the target detection performance of FSTD-Net.

3.3. Details of MSCIEM

Since UAV images are captured from a high-altitude perspective, they introduce complex background information and contain various types of targets. The complexity of the background and the scale differences can significantly impact the performance of target detection. To mitigate the introduction of complex background information and address the issue of scale differences in targets, MSCIEM is designed. As shown in Figure 5, the structural composition of MSCIEM is illustrated. MSCIEM consists of an independent shortcut connection branch (SCB), MSIEM, and CIEM, where features from these three paths are fused as the final output feature. The MSIEM, designed with convolutional kernels of various sizes, effectively captures feature information from targets of different scales in feature maps. Meanwhile, the CIEM is employed to capture contextual information, enhancing the overall feature representation capability.

The inputs and outputs of MSCIEM are respectively $f_{in}\in {ℝ}^{C_{in}\times H_{in}\times W_{in}}$ and $f_{out}\in {ℝ}^{C_{out}\times H_{out}\times W_{out}}$. For the independent branch SCB, it consists of a convolutional layer with a kernel size of 3 × 3 and a stride of 1. The input is $f_{in}\in {ℝ}^{C_{in}\times H_{in}\times W_{in}}$. The output feature after processing by SCB is $SC{B}_{out}$, as shown in Equation (1).

$$SC{B}_{out}=SCB\left(f_{in}\right)\in {ℝ}^{C_{in}\times H_{in}\times W_{in}}$$

(1)

For MSIEM and CIEM, before being fed into MSIEM and CIEM, $f_{in}$ undergoes a convolutional layer with a 1 × 1 kernel size to reduce the number of channels to 1/4 of the original, resulting in the inputs $f_{in}\in {ℝ}^{C_{in}/4\times H_{in}\times W_{in}}$ for MSIEM and CIEM, respectively. For MSIEM, it follows an inception-style structure [52,53]. It first undergoes 3 × 3 depth-wise convolutions (DWConvs) to extract local information and is then followed by a group of parallel DWConv with kernel sizes of 5 × 5, 7 × 7, and 9 × 9 to extract cross-scale contextual information. Let $L$ represent the local features extracted by the 3 × 3 DWConv, and $Z_i\in {ℝ}^{C_{in}/4\times H_{in}\times W_{in}},i=1,2,3$ represent the cross-scale contextual features extracted by the DWConvs with kernel sizes of 5 × 5, 7 × 7, and 9 × 9, respectively. The contextual features $Z_i$ and the local features $L$ are fused using 1 × 1 convolutions, resulting in the features $P_{out}$, as shown in Equation (2).

$$P_{out}=Con{v}_{1\times 1}\left(L+{\displaystyle {\sum }_{i=1}^3{Z}_i}\right)$$

(2)

For CIEM, its main role is to capture long-range contextual information. CIEM is integrated into MSIEM to strengthen the central features while further capturing the contextual interdependencies between distant pixels. As shown in Figure 5, by using an average pooling layer followed by a 1 × 1 convolutional layer on $f_{in}\in {ℝ}^{C_{in}/4\times H_{in}\times W_{in}}$, the local features are gained. $Avg$ represents the average pooling operation, followed by two depth-wise strip convolutions applied to $F_{local}$, and then the long-range contextual information $F_{long}$ is extracted. Finally, the sigmoid activation function is used to extract the final attention feature map $F_{att}$.

$$F_{att}=Sigmoid\left(Conv1\times 1\left(DW{C}_{11\times 1}\left(DW{C}_{1\times 11}\left(F_{local}\right)\right)\right)\right)$$

(3)

In Equation (3), $DW{C}_{11\times 1}$ and $DW{C}_{1\times 11}$ represent depth-wise strip convolutions with kernel sizes of 11 × 1 and 1 × 11, respectively. $Sigmoid$ denotes the activation function. $F_{att}$ is used to enhance the contextual expression of features extracted by MSIEM. After the features extracted by MSIEM and CIEM are processed by a 1 × 1 convolution, they are fused with the features extracted by SCB to obtain the final features.

$$f_{out}=Conv1\times 1\left(P_{out}+P_{out}\otimes F_{att}\right)+SC{B}_{out}$$

(4)

In Equation (4), $\otimes$ represents element-wise multiplication.

3.4. The Design of FEMFDS

Standard convolutions have fixed geometric structures and receptive fields; thus, they have certain limitations in feature extraction. Standard convolutions cannot effectively model full-scale targets of different shapes present in UAV images. However, deformable convolutions, by adding additional offsets based on sampling positions within the module and learning these offsets during the target task, can better model full-scale targets of various shapes present in UAV images. Thus, we designed the FEMFDS to better fit different full-scale target shapes. As shown in Figure 6, The FEMFDS consists of two branches: one branch is used to extract local features from the image, and the other branch is dedicated to extracting features that can better fit targets of different sizes. The features extracted by these two branches are then fused to obtain the final feature representation.

The deformable convolution used in FEFMDS is deformable convolution v4 (DCNv4). The reason for choosing DCNv4 is that, based on DCNv3, it removes the soft normalization to achieve stronger dynamic properties and expressiveness. Furthermore, as shown in Figure 7, memory access is further optimized. DCNv4 uses one thread to process multiple channels in the same group that share sampling offset and aggregation weights. Thus, DCNv4 has a faster inference speed. When the input is $x\in {ℝ}^{C\times H\times W}$, with height H, width W, and C channels. The process of deformable convolution is as follows:

$$\begin{array}{l}{y}_g={\displaystyle \sum _{k=1}^K{m}_{gk}{x}_g\left(p_0+p_k+\Delta p_{gk}\right)}\\ \\ y=concat\left(\left[y_1,y_2,\dots ,y_g\right],axis=-1\right)\end{array}$$

(5)

In Equation (5), g represents the number of groups for spatial grouping along the channel dimension. For the g-th group, the $x_g\in {ℝ}^{C\times H\times W}$ and $y_g\in {ℝ}^{C\times H\times W}$ respectively represents the feature maps for the g-th group. The dimension of the feature channels is represented by $c^{*}=c/g$. $m_{gk}\in R$ represents the spatial aggregation weight of the k-th sampling point in the g-th group. $p_k$ represents the k-th position of the predefined grid sampling, similar to standard convolution $\left\{(-1,-1),\left(-1,0\right),\dots ,\left(0,+1\right),\dots ,\left(+1,+1\right)\right\}$. $\Delta p_{gk}$ represents the offset corresponding to the position $p_k$ of the grid sampling in the g-th group.

3.5. The Structure of LLFEB

In convolutional neural networks, low-level semantic features include rich details such as edges, corners, textures, and so on. These features provide the foundation for subsequent high-level features, thereby enhancing the model’s detection performance on images. In the YOLOv5s baseline model, the feature maps obtained after two down-sampling operations are not directly used. Instead of directly fusing these low-level semantic features with the down-sample features, we adopt a top-down and bottom-up feature fusion approach. The fusion of low-level and high-level features combines local, detailed information with global semantic information, forming a more comprehensive feature representation. The multi-level information integration helps FSTD-Net better understand and parse image content, thereby further enhancing the detection performance of FSTD-Net on UAV images.

As shown in Figure 8, the specific implementation process of LLFEB is as follows. The low-level feature map is $F_2$, a convolutional layer that first operates on the corresponding higher-level semantic feature $F_2^{high}$, followed by upsampling on the processed $F_2^{high}$, then the $F_2^{LE\prime }$ is gained. Then, $F_2^{LE\prime }$ and $F_2$ are merged with the Concat operation, the fused feature map is further processed with FEMFDS and a convolutional layer, and finally, the enhanced low-level semantic feature $F_2^{LE}$ is obtained.

The overall process of LLFEB is as follows:

$$\begin{array}{l}{F}_2^{LE\prime }=Up\left(CBS\left(F_2^{high}\right)\right)\\ \\ F_2^{LE}=Concat\left(FE\left(Concat\left(F_2^{LE\prime },F_2\right)\right),CBS\left(F_2^{high}\right)\right)\end{array}$$

(6)

In Equation (6), $Up$ represents the upsampling operation, $FE$ represents the FEMFDS module.

3.6. Loss Function

The loss function of FSTD-Net consists of three components, bounding box regression loss, classification loss, and objectness loss. For bounding box regression loss, the Complete Intersection over Union (CIoU) is used. The CIoU loss considers not only the IoU of the bounding box but also the differences in centroid distances and aspect ratios, which makes the prediction of the bounding box more accurate.

$$\begin{array}{l}CIoU=1-IoU+{\displaystyle \frac{{\rho }^2\left(b,b^g\right)}{c^2}}+\alpha v\\ \\ v={\displaystyle \frac{4}{{\pi }^2}}{\left(arctan{\displaystyle \frac{w^{gt}}{h^{gt}}}-arctan{\displaystyle \frac{w}{h}}\right)}^2\end{array}$$

(7)

In Equation (7), $IoU$ means the Intersection and juxtaposition ratio of predicted and true frames. As shown in Figure 9, we use two different colors of boxes to represent the ground truth boxes and the predicted boxes, $w^{gt}$, $h^{gt}$, $w$, $h$ represent the width and height of the ground truth boxes and predicted boxes, respectively. ${\rho }^2\left(b,b^g\right)$ represents the Euclidean distance between the center point of the predicted box and the center point of the ground truth box. $c$ represents the length of the diagonal of the smallest enclosing box that contains both the predicted box and the ground truth box. $\alpha$ represents the weight parameter used to balance the impact of the aspect ratio. $v$ represents the consistency measure of the aspect ratio.

For classification loss, the classification loss (CLoss) measures the difference between the predicted and true categories of the target within each predicted box. The Binary Cross-Entropy Loss (BCEloss) is used to calculate the classification loss.

$$CLoss=-{\displaystyle \sum _{c=1}^C\left[y_c\log \left(p_c\right)+\left(1-y_c\right)\log \left(1-p_c\right)\right]}$$

(8)

In Equation (8), $C$ represents the total number of categories. $y_c$ represents the true label of class c (0 or 1). $p_c$ represents the predicted probability of class c.

For Objectness Loss (OLoss), the objectness loss measures the difference between the predicted confidence and the true confidence regarding whether each predicted box contains a target.

$$OLoss=-\left[y\log \left(p\right)+\left(1-y\right)\log \left(1-p\right)\right]$$

(9)

In Equation (9), $y$ represents true confidence (1 if the target is present, otherwise 0). $p$ is the predicted confidence.

The final total loss function (TLoss) is obtained by weighting and combining the three aforementioned loss components:

$$TLoss={\lambda }_{box}\times CIoU+{\lambda }_{cls}\times CLoss+{\lambda }_{obj}\times OLoss$$

(10)

In Equation (10), ${\lambda }_{box}$, ${\lambda }_{cls}$ and ${\lambda }_{obj}$ represent the weight coefficients for the bounding box regression loss, classification loss, and objectness loss, respectively. In this experiment, the weights of the box regression loss, classification loss, and objectness loss are 0.05, 0.5, and 1.0, respectively.

4. Experiments and Analysis

In this section, the VisDrone2019 and UAVDT [54] dataset used in this paper will be introduced first, including its characteristics and relevance to the experiment. The experimental setup is then elaborated, encompassing the training configuration, the computational platform used for training, and the evaluation metrics employed to assess the performance of the proposed FSTD-Net and other state-of-the-art models.

4.1. Dataset and Experimental Settings

4.1.1. Dataset Introduction

The dataset used in this experiment is the large-scale VisDrone2019 dataset and UAVDT dataset. For the UAVDT dataset, it contains 6471 images for training, 548 images for validation, and 3190 images for testing. However, since 1580 images are reserved for challenges and do not provide the ground truth, we use 1610 images for testing in this experiment. The VisDrone2019 dataset includes a large number of targets of different scales and complex background information, making it suitable for evaluating the performance of the proposed algorithm. As shown in Figure 10, it shows the statistical information on the VisDrone2019 dataset, including the number of instances for different categories, the distribution of bounding box centers, the normalized coordinates of bounding box center points in the image, and the distribution of bounding boxes with different width-to-height ratios. During the training process, all images are cropped into 800 × 800 pixels with a stride of 700. Additionally, to further validate the performance of the proposed network, we conducted experiments on the UAVDT dataset. The UAVDT dataset consists of 50 videos, comprising a total of 40,375 images, each with a resolution of 1024 × 540 pixels. The UAVDT has three categories: car, truck, and bus. Due to the high similarity among image sequences within each video, we selected image sequences from each video to construct the train and test sets. The train set comprises 6566 images, and the test set includes 1830 images.

4.1.2. The Details of Experimental Environment

In this experiment, as shown in

Table 1. Experimental environment configuration.

Parameter	Configuration
Operation Environment	Windows
GPU	GeForce RTX 2080ti
Programming Language	Python 3.8
Integrated Development Environment	Pycharm
Cuda	12.2
Deep Learning Framework	Pytorch

, we utilized an operational environment running on Windows, equipped with a GeForce RTX 2080ti GPU. The programming was conducted using Python 3.8 within the PyCharm integrated development environment, leveraging CUDA 12.2 for GPU support. For deep learning tasks, the Pytorch framework was employed, ensuring efficient model development and training processes.

4.1.3. The Setting of Model Training

In this training session, as shown in

Table 2. The basic parameters of training settings on a different dataset.

Parameter	VisDrone 2019/UAVDT
Input Image Size	800 × 800/640 × 640
Batch-size	2
Epochs	80
Learning rate	0.01
Warmup epochs	3
Momentum	0.937
Weight-decay	0.0005

, the batch size for each epoch was set to 2, and the input image size was 800 × 800 for VisDrone 2019 dataset and 640 × 640 for the UAVDT dataset. The initial learning rate was set at 0.01 and gradually decayed to 0.001. A momentum of 0.937 was applied to accelerate the convergence process, and a weight decay of 0.0005 was employed to prevent overfitting. The more training epochs, the higher the risk of overfitting; thus, our FSTD-Net are trained for 80 epochs on both VisDrone 2019 and UAVDT dataset. The first three epochs involved a warm-up stage, during which a lower learning rate and momentum were applied to stabilize the training process. Data augmentation techniques included changes in hue, saturation, and brightness, as well as geometric transformations such as rotation, translation, scaling, and shearing. Additionally, to improve model generalization and avoid overfitting, horizontal and vertical flips, as well as advanced augmentation methods like Mosaic, Copy-Paste, and so on, are applied.

4.1.4. Evaluation Metrics

To better evaluate the detection performance of the model, we selected six metrics: average precision (AP), AP₅₀, AP₇₅, AP_s, AP_m, and AP_l. AP represents the average precision across all ten IoU thresholds, ranging from 0.5 to 0.95. AP₅₀ is the average precision calculated at an IoU threshold of 0.5, while AP₇₅ is calculated at an IoU threshold of 0.75. AP_s, AP_m, and AP_l refer to the precision of target detection with areas in the ranges of 32 × 32, 32 × 32 to 96 × 96, and above 96 × 96, respectively. The process for calculating the metrics is as follows:

$$\begin{array}{l}P={\displaystyle \frac{TP}{TP+FP}}\\ \\ \\ R={\displaystyle \frac{TP}{TP+FN}}\\ \\ \\ AP={\displaystyle \sum _{i=1}^{n-1}\left(R\left(i+1\right)-R\left(i\right)\right)\times P\left(i+1\right)}\\ \\ \\ A{P}_{50}={\displaystyle \sum _{i=1}^{n-1}\left(R_{0.5}\left(i+1\right)-R_{0.5}\left(i\right)\right)\times P_{0.5}\left(i+1\right)}\\ \\ \\ A{P}_{75}={\displaystyle \sum _{i=1}^{n-1}\left(R_{0.75}\left(i+1\right)-R_{0.75}\left(i\right)\right)\times P_{0.75}\left(i+1\right)}\\ \\ \\ A{P}_s={\displaystyle \sum _{i=1}^{n-1}\left(R_s\left(i+1\right)-R_s\left(i\right)\right)\times P\left(i+1\right)}\\ \\ \\ A{P}_m={\displaystyle \sum _{i=1}^{n-1}\left(R_m\left(i+1\right)-R_m\left(i\right)\right)\times P_m\left(i+1\right)}\\ \\ \\ A{P}_l={\displaystyle \sum _{i=1}^{n-1}\left(R_l\left(i+1\right)-R_l\left(i\right)\right)\times P_l\left(i+1\right)}\\ \end{array}$$

(11)

True Positive (TP), False Positive (FP), and False Negative (FN) are important metrics in evaluating the performance of classification and detection models. TP refers to the number of correctly identified positive instances, meaning the targets that are correctly detected and match the ground truth. FP refers to the number of instances incorrectly identified as positive, meaning the targets that are detected but do not match any ground truth. FN refers to the number of positive instances that were missed by the model, meaning the targets that are present in the ground truth but were not detected by the model. To calculate the specific value of each AP in the above formula, we first need to calculate the Precision (P) and Recall (R) values. Then, AP is obtained by computing the weighted sum of the precision at different recall levels, where the weight is the difference in recall multiplied by the corresponding precision. Specifically, AP₅₀, AP₇₅, APs, AP_m, and AP_l correspond to the average precision at different IOU thresholds (such as 0.5 and 0.75) and different target scales (small, medium, and large targets). The calculation method is the same, but the precision and recall used are under the corresponding conditions.

4.2. Experimental Results on VisDrone2019 Dataset

4.2.1. Detecting Results on the VisDrone2019 Dataset

To validate the detection performance of our proposed FSTD-Net, we compared it with several state-of-the-art target detection models, including the YOLOv3, SSD, YOLOv6, YOLOv7, YOLOv8, YOLOv10, Faster R-CNN, and so on, on the VisDrone2019 dataset. As shown in

Table 3. Detection results of FSTD-Net with state-of-the-art models on the VisDrone dataset.

Method	AP50	AP	AP75	APs	APm	APl	Params
Faster-RCNN [9]	13.4	4.9	2.5	0.3	8.3	25.7	41.2M
SSD [55]	24.1	12.1	11.3	6.8	19.7	19.8	26.3M
YOLOv3-Tiny [36]	26.2	13.0	11.4	6.1	20.5	30.6	8.85M
YOLOv6s [38]	33.2	18.8	19.1	10.7	29.8	32.1	18.5M
YOLOv7-Tiny [39]	32.4	17.6	17.0	9.6	26.7	35.8	6.2M
MFO-Net [25]	34.3	18.4	17.9	11.3	28.0	32.6	35.2M
YOLOv5s	37.7	20.5	20.1	11.2	31.5	38.4	7.2M
YOLOv10s [20]	35.0	19.9	20.3	11.6	30.5	33.8	7.2M
YOLOv8s [40]	36.5	20.8	21.0	11.8	32.0	35.7	11.2M
FSTD-Net (Ours)	41.5	23.2	23.0	13.2	35.1	42.1	11.3M

. From the evaluation metrics we selected, it can be observed that FSTD-Net outperforms other models and achieved the best accuracy values across all evaluation metrics. Specifically, FSTD-Net achieves a 2.4% higher AP, 3.8% higher AP₅₀, and 2.0% higher AP₇₅ compared to the second-best model. Moreover, in the AP_s, AP_m, AP_l metrics, FSTD-Net achieves 1.4%, 3.1%, and 3.7 % higher scores compared to the second-best model. FSTD-Net not only achieves the best detection performance overall among all models but also attains the best detection results in AP_s, AP_m, and AP_l. This indicates that FSTD-Net is more sensitive to targets of different scales in UAV images and can better detect targets with significant scale variability in UAV images. At the same time, the overall number of params in the FSTD-Net remains relatively low. The FSTD-Net maintains the same params as YOLOv8, but it achieves better detection performance. FSTD-Net achieves 5% higher AP₅₀, 2.4% higher AP, 2% higher AP₇₅, 1.4% higher AP_s, 3.1% higher AP_m, and 6.4% higher AP_l compared to YOLOv8s. Our proposed FSTD-Net can achieve better detection performance with a lightweight architecture.

4.2.2. Visual Comparisons of Detection Results

To better demonstrate the superior detection performance of FSTD-Net compared to other state-of-the-art models, we selected several images from the VisDrone 2019 dataset to visualize the detection results, which allows us to directly compare and evaluate the performance differences between FSTD-Net and other models in practical applications. As shown in Figure 11, we use blue boxes to highlight the detection areas for comparison, allowing us to evaluate the detection performance of our proposed FSTD-Net and other advanced models in these specific areas. For better visibility, we enlarge the area outlined by the blue box and move it to the upper right corner of the image. In the first column of images, the blue box areas contain two “motor” targets in the drone imagery. Comparative models, such as YOLOv6 and YOLOv8, failed to detect these two targets, with YOLOv5 detecting only one of them. Due to the proposed LLEFB, FEFMDS, and MCIEM, they have enhanced FSTD-Net’s ability to learn features of targets at different scales; thus, FSTD-Net can precisely detect these two targets. For the second column of images, YOLOv5, YOLOv6, and YOLOv8 all failed to detect the car target highlighted by the blue box. However, our model successfully detected this car target. This is attributed to LLEFB, which enhances FSTD-Net’s capability to learn detailed features of certain target types, such as texture and edge features. In the third column of images, YOLOv5, YOLOv6, and YOLOv8 all failed to detect the human target highlighted by the blue box. However, our FSTD-Net successfully detected this human target, demonstrating FSTD-Net’s superior capability in detecting some difficult targets. In the fourth column of images, YOLOv6 and YOLOv8 all failed to fully detect the bicycle and the two motor targets highlighted in the blue box area. YOLOv5 mistakenly identified the bicycle as a motor. However, our FSTD-Net successfully detected all these targets, demonstrating FSTD-Net’s superior detection capabilities. In the fifth column of images, YOLOv6 and YOLOv8 both failed to fully detect the “van” target highlighted by the blue box, which is situated in a relatively complex background area. However, both our FSTD-Net and YOLOv6 successfully detected the entire target. Thanks to MSCIEM, FSTD-Net can effectively mitigate the interference from the complex background, successfully detecting the target.

Overall, our proposed FSTD-Net has better detection results for the presence of the different scale targets in the UAV image or the presence of some targets that are interfered with by complex backgrounds. LLEFB can provide FSTD-Net with sufficient detailed features, enabling FSTD-Net to detect targets that are difficult to detect. MCIEM can provide FSTD-Net with sufficient multi-scale information and, to some extent, avoid interference from complex backgrounds. FEFMDS can better enhance the model’s ability to learn targets on different scales. Both MSCIEM and FEFMDS can better enhance FSTD-Net’s detection of targets on different scales. Our FSTD-Net is better than the compared detection effect of the compared state-of-the-art models; the proposed FSTD-Net can correctly identify the majority of targets in the VisDrone2019 image.

4.2.3. Experimental Results of Different Categories on the VisDrone2019 Test Set

To further validate the effectiveness of the proposed FSTD-Net, we compare the detection performance of the proposed FSTD-Net and the Yolov8s on all categories on the VisDrone2019 dataset. As shown in

Table 4. Experimental results of different categories on the VisDrone2019 test set.

Methods	Category	AP50	AP	AP75	APs	APm	APl
FSDT-Net (ours)	pedestrian	36.8	14.5	8.7	11.2	42.8	51.4
	people	21.8	7.1	2.6	6.4	21.2	23.0
	bicycle	21.4	8.7	5.2	6.9	18.3	26.7
	car	79.8	48.9	52.9	34.3	64.8	77.5
	van	50.6	33.3	38.1	19.5	47.2	54.1
	truck	48.2	30.3	33.7	8.0	36.6	52.7
	tricycle	26.7	14.6	14.5	6.7	21.5	36.5
	awning-tricycle	25.2	14.9	15.6	9.6	20.4	20.0
	bus	61.9	42.2	48.3	14.2	48.6	63.0
	motor	43.2	17.4	10.2	15.0	29.4	16.1
YOLOv8s	pedestrian	31.4	12.6	7.6	9.3	40.2	47.0
	people	17.4	6.1	2.5	5.4	19.1	21.8
	bicycle	16.2	6.8	4.1	6.2	11.8	21.1
	car	76.7	47.5	51.7	33.0	63.8	75.2
	van	46.5	30.8	35.5	18.0	44.2	46.1
	truck	38.9	24.8	27.5	5.9	31.2	41.5
	tricycle	21.3	11.7	12.0	6.0	17.4	31.2
	awning-tricycle	23.4	13.8	15.7	7.8	20.5	12.1
	bus	55.8	38.1	43.9	12.8	44.8	56.4
	motor	37.8	15.5	9.7	13.5	27.0	4.7

, compared to the proposed FSTD-Net, Yolov8s surpasses it by only 0.1% in AP₇₅ and AP_m for the awning-tricycle target. For the remaining categories, the proposed FSTD-Net all outperforms Yolov8s in detection performance. As shown in Table 3, the params of the FSTD-Net are comparable to those of Yolov8s. However, compared to Yolov8s, FSTD-Net exhibits overall better detection performance, providing more accurate detection for targets of varying scales present in the UAV images. This indicates the superior detection capability of the proposed FSTD-Net, demonstrating its ability to effectively detect various target categories present in the UAV dataset.

4.2.4. Ablation Studies of FSTD-Net on the VisDrone 2019 Dataset

To further explore the effectiveness of the proposed method, we conducted ablation experiments to investigate the impact of the LLFEB, MSCIEM, and FEFMDS modules on FSTD-Net detection performance on the VisDrone2019 dataset.

Table 5. Detailed settings of ablation experiment.

	YOLOv5s	LLFEB	FEFMDS	MSCIEM	Params
baseline	√				7.2M
a	√	√			7.3M
b	√	√	√		8.6M
c	√	√	√	√	11.3M

presents the improvements based on the baseline YOLOv5s model when performing ablation experiments.

Table 6. Detailed comparison of the proposed FSTD-Net in the ablation studies.

Method	AP50	AP	AP75	APs	APm	APl
baseline	37.7	20.5	20.1	11.2	31.5	38.4
a	38.6	21.2	20.8	11.9	32.3	38.3
b	40.7	22.7	22.7	12.9	34.3	41.4
c	41.5	23.2	23.0	13.2	35.1	42.1

shows the detection performance of the baseline with the addition of the different modules. As shown in Table 5, √ indicated that the current module is embedded in the baseline.

In the baseline model, the feature maps obtained after two down-sampling operations are not used; thus, the detailed features of the low-level feature have not been fully utilized. The LLFEB enhances the baseline’s learning and understanding of target features in UAV images by providing low-level semantic features such as edges, corners, and textures as a basis for subsequent high-level feature extraction. After introducing LLFEB to baseline, AP₅₀, AP, AP₇₅, AP_s, and AP_m on the VisDrone2019 increase by 0.9%, 0.7%, 0.7%, 0.7%, and 0.8%, the AP_l remains comparable after the introduction of LLFEB. With the integration of LLFEB, the overall target detection accuracy of the model is improved, particularly for medium and small targets.

The FEFMDS overcomes the limitations of standard convolutions in extracting features according to fixed geometric patterns, providing more representative features for the detection of targets of various scales. After introducing FEFMDS, it can be observed that there are improvements of 1% in AP_s, 2% in AP_m, and 3.1% in AP_l. AP₅₀, AP, and AP₇₅ on VisDrone2019 increased by 2.1%, 1.5%, and 1.9%, respectively. The designed model can better detect targets of different scales in drones under the guidance of the FEFMDS.

The MSCIEM suppresses complex backgrounds in UAV images while enhancing the feature extraction of targets of different scales by utilizing multiple convolutions with different kernel sizes. The CIEM further enhances the baseline’s ability to capture long-range semantic information. With the introduction of MSCIEM, we observed improvements of 0.3% in AP_s, 0.8% in AP_m, and 0.7% in AP_l. MSCIEM can guide the model to detect targets of different scales in UAV images more accurately.

In summary, the proposed MSCIEM and FEFMDS modules enhance the baseline’s detection of targets of different scales, while LLFEB improves the baseline’s learning and understanding of target features in UAV images.

4.3. Experimental Results on UAVDT Dataset

To further validate the effectiveness of the proposed FSTD-Net, we further conducted experiments on the UAVDT dataset and compared the detection performance of the FSTD-Net with that of YOLOv6, YOLOv7, YOLOv8, YOLOv10, and so on.

Table 7. Detection results of FSTD-Net with state-of-the-art models on the UAVDT dataset.

Methods	AP50	AP	AP75	APs	APm	APl	Params
YOLOv3-Tiny [36]	35.4	18.6	17.3	17.3	19.5	26.3	8.85M
YOLOv6s [38]	37.4	22.5	25.4	23.2	22.3	17.0	18.5M
YOLOv7-Tiny [39]	35.9	17.3	14.0	15.8	18.6	17.7	6.2M
YOLOv10s [20]	36.8	21.4	22.1	23.4	21.0	20.6	7.2M
YOLOv8s [40]	37.0	22.0	24.4	23.8	20.8	30.2	11.2M
YOLOv5s	37.3	21.7	22.6	20.7	22.5	35.7	7.2M
FSTD-Net (Ours)	40.8	24.2	26.4	23.2	24.6	32.0	11.3M

presents the AP, AP₅₀, AP₇₅, and the detection precision for small, medium, and large targets (AP_s, AP_m, AP_l) on the UAVDT dataset. Compared to other algorithms, FSTD-Net achieved the highest detection precision in AP₅₀, AP, AP₇₅, and AP_m. Specifically, our method improved by 3.4%, 1.7%, 1%, and 2.1%, respectively, over the second-best model in these metrics. As for the AP_s and AP_l, FSTD-Net achieved relatively good results. These results demonstrate the effectiveness of our FSTD-Net in handling large-scale variations in object detection and achieving a good detection accuracy.

To further explore the effectiveness of the proposed method on the UAVDT dataset, we conducted ablation experiments to investigate the impact of the LLFEB, MSCIEM, and FEFMDS modules on FSTD-Net detection performance on the UAVDT dataset. As shown in

Table 8. Ablation studies of FSTD-Net on the UAVDT dataset.

Method	AP50	AP	AP75	APs	APm	APl
baseline	37.3	21.7	22.6	20.7	22.5	35.7
a	37.6	22.2	24.1	19.6	23.2	36.5
b	38.8	23.5	26.4	23.9	22.6	28.6
c	40.8	24.2	26.4	23.2	24.6	32.0

, LLFEB can better improve the detection performance of medium and large targets. FEFMDS can help the model better learn small targets. MSCIEM enhances the model’s ability to comprehensively learn targets of various scales, effectively balancing the learning of small and large targets while significantly improving the learning of medium-sized targets. Overall, due to the design of the three modules, FSTD-Net achieves relatively optimal performance in object detection across various scales.

To further validate the effectiveness of the proposed FSTD-Net, we conducted a comparison of its detection performance with that of baseline YOLOv5s across all categories on the UAVDT dataset.

Table 9. Experimental results of different categories on the UAVDT dataset.

Methods	Category	AP50	AP	AP75	APs	APm	APl
FSTD-Net	car	76.2	46.6	52.9	40.5	53.2	-
	truck	22.8	13.7	15.0	4.5	17.0	25.1
	bus	23.2	12.3	11.4	24.7	3.5	38.9
baseline	car	76.2	45.4	50.0	39.9	51.4	-
	truck	14.9	8.0	7.1	3.1	10.2	32.9
	bus	21.0	11.6	10.7	19.2	5.8	38.5

presents the detection performance of FSTD-Net across various categories in the UAVDT dataset. It can be seen that YOLOv5s outperforms FSTD-Net only in the AP_m of the bus category and the AP_l of the truck category. In the other metric of all target categories, FSTD-Net all achieves better results. FSTD-Net can achieve a better detection result for all categories of targets in general. Overall, FSTD-Net achieves better detection performance on the UAVDT dataset.

4.4. Extended Experimental Results on VOC Dataset

To validate the generalization capability of the FSTD-Net, we conducted further experiments on the Pascal VOC datasets [56], which include the VOC 2007 and VOC 2012 datasets. The training and validation sets from VOC 2007 and 2012 are combined to form new training and validation sets, respectively, while the test set from VOC 2007 is used as the new test set. The training set consists of 8218 images; the test set contains 4952 images; and the validation set comprises 8333 images. FSTD-Net was trained for a total of 300 epochs, and the results are presented in

Table 10. Detection results of FSTD-Net with baseline on the VOC dataset.

Method	AP50	AP	AP75	APs	APm	APl
baseline	66.7	41.2	43.4	11.5	29.2	49.8
FSTD-Net	68.2	44.1	47.0	11.2	29.7	54.1

. As shown in Table 10, FSTD-Net achieves an AP₅₀ of 68.2%, an AP of 44.1%, and an AP₇₅ of 47.0%. Compared to the baseline, our proposed FSTD-Net achieves improvements of 1.5%, 2.9%, 3.6%, 0.5%, and 4.3% in AP₅₀, AP, AP₇₅, AP_m, and AP_l, respectively. It indicates that our proposed model is well-suited for generic image applications.

As shown in

Table 11. The detection results of FSTD-Net with other state-of-the-art models on the VOC dataset.

Methods	Category	AP50	AP	AP75	APs	APm	APl
FSDT-Net	airplane	76.3	49.0	52.7	23.1	41.9	58.1
	bicycle	77.5	50.7	56.2	5.1	36.1	61.2
	bird	60.8	36.4	36.1	14.0	31.3	46.5
	boat	54.8	28.7	25.8	14.6	23.9	44.4
	bottle	44.1	25.7	26.3	8.4	27.2	43.0
	bus	78.1	61.2	67.9	15.1	24.5	74.8
	car	80.3	55.1	58.8	21.2	43.6	76.7
	cat	78.5	55.0	58.0	0.0	18.6	58.8
	chair	50.8	28.7	28.2	3.8	25.8	37.3
	cow	70.4	44.1	49.6	19.6	44.6	52.9
	diningtable	61.6	41.3	43.7	0.0	3.5	49.3
	dog	71.0	47.3	50.5	0.7	30.5	51.6
	horse	79.5	52.9	56.1	5.2	21.6	61.2
	motorbike	79.3	50.1	54.9	9.0	30.9	60.1
	person	79.3	46.9	47.6	14.1	37.5	60.6
	pottedplant	42.9	19.6	14.7	6.2	15.6	30.0
	sheep	65.5	43.6	47.6	18.4	45.1	54.3
	sofa	63.9	43.3	46.8	-	21.2	45.3
	train	80.0	54.2	60.6	-	31.1	58.0
	tvmonitor	69.7	47.4	57.4	23.2	38.8	57.6
baseline	airplane	74.7	45.3	48.3	23.5	39.4	53.2
	bicycle	76.2	49.3	54.6	5.0	36.8	58.9
	bird	62.1	35.1	36.4	17.2	28.5	45.3
	boat	49.9	25.1	20.3	10.7	23.2	36.2
	bottle	45.2	24.3	22.6	6.8	27.1	38.4
	bus	75.2	57.2	62.6	13.4	23.4	69.8
	car	78.6	52.8	56.2	19.3	43.4	73.5
	cat	76.9	50.0	53.5	0.0	20.3	52.7
	chair	49.6	27.1	26.9	5.5	25.7	33.3
	cow	74.3	45.6	48.1	24.4	48.7	52.2
	diningtable	58.5	35.8	38.2	0.0	3.4	42.7
	dog	70.2	43.0	45.7	0.4	30.3	46.8
	horse	75.5	48.2	51.9	4.0	20.9	55.9
	motorbike	75.8	45.6	48.2	13.1	27.8	55.2
	person	77.8	44.0	43.7	13.1	36.4	56.9
	pottedplant	41.1	19.3	14.0	7.4	16.2	27.3
	sheep	64.6	42.3	46.8	17.0	47.8	49.7
	sofa	61.9	39.3	41.1	-	20.5	40.9
	train	77.4	50.1	54.9	-	26.6	53.9
	tvmonitor	69.0	45.4	53.3	26.8	38.5	53.6

, the results demonstrate the detection accuracy of FSTD-Net compared to the baseline across various categories on the VOC dataset. It can be seen that our proposed model outperforms the baseline in most categories, indicating that our method also improves detection performance in general image applications.

5. Discussion

5.1. The Effectiveness of FSTD-Net

Targets in UAV images are more complex compared to those in natural images due to significant scale variability and complex background conditions. Target detection models suitable for natural images, such as the YOLO series of neural networks, cannot be directly applied to UAV images. To address this issue, we designed FSTD-Net, providing some insights on how to adapt models such as the YOLO series for use in UAV image detection.

Overall, the structure of FSTD-Net is essentially consistent with the YOLO series, including the backbone, neck, and head parts, maintaining a relatively lightweight structure. Its parameter count is 11.3M, which is roughly equivalent to that of YOLOv8s. However, compared to the YOLO series of neural networks, FSTD-Net is better suited for UAV image detection tasks.

Experimental results on the VisDrone 2019 and UAVDT datasets demonstrate that the methods proposed in this paper can enhance the applicability of the YOLO model for detection tasks involving UAV images. Additionally, we conducted further experiments on the general datasets VOC2007 and VOC2012. Experiments on the general datasets indicate that the proposed methods can also improve the baseline’s detection performance on natural images. Overall, FSTD-Net not only performs well on UAV image datasets but also achieves commendable results on general datasets.

5.2. The Impact of Image Resizing and Cropping on Target Detection Performance

We further conduct the experiments on VisDrone 2019, we cropped the train, test, and val sets into images of 640 × 640 pixels. The original training set contained 6471 images, and after cropping, the number of images in the training set increased to 44,205. We then compared the results of training on two datasets: one with the original images resized to 640 × 640, containing 6471 images, and the other with the cropped images, containing 44,205 images. FSTD-Net was trained for 80 epochs on both datasets. The detection results are shown in

Table 12. Detection results of FSTD-Net with resizing and cropping methods.

Method	AP50	AP	AP75	APs	APm	APl
FSTD-Net (resized)	25.6	12.8	11.5	6.3	19.6	25.9
FSTD-Net (with crop)	40.9	23.1	22.9	14.4	34.7	39.7

.

As shown in Table 12, it can be observed that the detection performance on the datasets with cropping methods is significantly better across all metrics compared to the performance on the datasets with resizing methods. On one hand, the cropping operation increases the number of data samples in the training set, allowing the network to undergo more iterations during training, which leads to improved detection performance on the cropped dataset. On the other hand, resizing methods change the original shape of the targets in the UAV images, which may significantly degrade detection performance compared to the cropped dataset.

6. Conclusions

In this paper, to address the target scale variability and complex background interference presented in UAV images, we propose the FSTD-Net, which effectively detects multi-scale targets in UAV images and avoids interference caused by complex backgrounds. Firstly, to better capture the features of targets at different scales in UAV images, MSCIEM utilizes a multi-kernel combination approach, and the CIEM in MSCIEM can capture long-range contextual information. Due to MSCIEM, the model can effectively capture features of targets at different scales, perceive long-range contextual information, and be sensitive to significant variations in scale. Secondly, to further account for the targets of different shapes on different scales, FEFMDS breaks through the conventional method of feature extraction by standard convolution, providing a more flexible approach to learning full-scale targets of different shapes. The FEFMDS can better fit full-scale targets of different shapes in UAV images, providing more representative features for the final detection of FSTD-Net. Finally, LLFEB is used to efficiently utilize low-level semantic features, including edges, corners, and textures, providing a foundation for subsequent high-level feature extraction and guiding the model to better understand various target features. Experimental results demonstrate that FSTD-Net outperforms the selected state-of-the-art models, achieving advanced detection performance. FSTD-Net achieves better detection results in terms of overall detection accuracy while maintaining a relatively lightweight structure. In the future, we will further explore more efficient and lightweight target detection models to meet the requirements for direct deployment on UAVs.