A Lightweight Object Detection Algorithm for Remote Sensing Images Based on Attention Mechanism and YOLOv5s

Abstract

The specific characteristics of remote sensing images, such as large directional variations, large target sizes, and dense target distributions, make target detection a challenging task. To improve the detection performance of models while ensuring real-time detection, this paper proposes a lightweight object detection algorithm based on an attention mechanism and YOLOv5s. Firstly, a depthwise-decoupled head (DD-head) module and spatial pyramid pooling cross-stage partial GSConv (SPPCSPG) module were constructed to replace the coupled head and the spatial pyramid pooling-fast (SPPF) module of YOLOv5s. A shuffle attention (SA) mechanism was introduced in the head structure to enhance spatial attention and reconstruct channel attention. A content-aware reassembly of features (CARAFE) module was introduced in the up-sampling operation to reassemble feature points with similar semantic information. In the neck structure, a GSConv module was introduced to maintain detection accuracy while reducing the number of parameters. Experimental results on remote sensing datasets, RSOD and DIOR, showed an improvement of 1.4% and 1.2% in mean average precision accuracy compared with the original YOLOv5s algorithm. Moreover, the algorithm was also tested on conventional object detection datasets, PASCAL VOC and MS COCO, which showed an improvement of 1.4% and 3.1% in mean average precision accuracy. Therefore, the experiments showed that the constructed algorithm not only outperformed the original network on remote sensing images but also performed better than the original network on conventional object detection images.

1. Introduction

In the era of big data, due to the development and progress of machine learning technology represented by deep learning, artificial intelligence can better cope with complex environments and tasks through continuous learning and adaptation. By processing and analyzing large-scale and high-dimensional data [1], it can reveal the patterns and patterns behind the data, supporting more accurate predictions and decisions [2,3,4]. By exploring potential patterns in data, new knowledge and innovation can be discovered, supporting the development of various sciences and technologies. Therefore, in recent years, artificial intelligence has been gradually applied to speech recognition, natural language processing, computer vision, and other fields. In the field of artificial intelligence, computer vision technology has found widespread application in diverse fields such as intelligent security [5], autonomous driving [6], remote sensing monitoring [7,8], medical and pharmaceuticals [9,10], agriculture [11], intelligent transportation [12], and information security [13]. Computer vision tasks can be categorized into image classification [14], object detection [15], and image segmentation [16]. The core task of object detection is to determine the categories and positions of multiple objects in the image and to give corresponding detection boxes and object categories for each object. Remote sensing images contain a wealth of detailed information, which can intuitively reflect the shape, color, and texture of ground targets. Remote sensing target detection, as a fundamental technique, is widely applied in various fields, such as urban planning [17], land use [18], traffic guidance [19], and military surveillance [20]. With the development of ground observation technology, the scale of high-resolution remote sensing image data has been continuously increasing. High-resolution remote sensing images provide higher image quality and more abundant, detailed information, which presents greater opportunities for the development of target detection in the field of remote sensing. Target detection can be divided into two main categories: traditional object detection algorithms and deep learning-based object detection algorithms. Traditional object detection algorithms mainly rely on traditional feature extractors [21] and use sliding windows to generate object candidate regions. Representative algorithms include the Viola–Jones detector (VJ-Det) [22], the Histogram of Oriented Gradient (HOG) detector [23], and the deformable part model detector (DPM) [24]. With the development of deep learning, convolutional neural networks (CNNs) have gradually been applied to object detection tasks. Based on deep learning, object detection technology can use multi-structured network models and powerful training algorithms to adaptively learn high-level semantic information from images. Image features are extracted and fed into a classification network to complete the tasks of object classification and localization, thereby effectively improving the accuracy and efficiency of object detection tasks.

According to the detection principle, target detection algorithms based on deep learning can be divided into two categories: (1) Two-stage target detection algorithms based on candidate regions. Representative algorithms include R-CNN [25], Fast R-CNN [26], and Faster R-CNN [27]. This algorithm first generates sample candidate boxes [28,29,30], then encodes the extracted feature vectors using deep convolutional neural networks [31,32,33], and finally performs regression on the class and location of the target object within the candidate box [34]. By employing a two-stage operation, the target detection algorithm achieves high detection accuracy at the expense of slower speed and non-real-time detection. (2) One-stage object detection algorithms based on direct regression, represented by algorithms such as SSD [35] and the YOLO series [36,37,38]. This algorithm abandons the stage of generating candidate bounding boxes and directly outputs the position and category of the target through regression, which improves the detection speed.

The YOLO series is currently a classical one-stage object detection algorithm. Redmon et al. [39] proposed the YOLO algorithm, which represented a significant breakthrough in real-time object detection. However, the training of each component in YOLO needs to be conducted separately, leading to a slow inference speed. A strategy of jointly training the components was proposed, which not only improved the inference speed but also reduced the complexity of network training while enhancing the detection performance of YOLO. The YOLOv3 algorithm [37] utilizes Darknet-53 as the backbone network and fuses up-sampled feature maps with shallow feature maps to retain the semantic information of small objects and enhance the detection performance of such objects. The YOLOv4 algorithm [38] adopts CSPDarknet53 [40] as the backbone network and introduces Spatial Pyramid Pooling (SPP) [41] to optimize the receptive field of deep feature maps, thereby further improving detection accuracy. Ultralytics released the YOLOv5 algorithm, which incorporates CSPNet as its backbone network. The neck component employs a feature pyramid network (FPN) [42] to enable top-down semantic information transmission, leveraging both low-level features with high resolution and high-level features with semantic information. In addition, the algorithm utilizes a path aggregation network (PAN) [43] for bottom-up localization transmission, which facilitates the propagation of low-level information to the top level. The YOLOv5 model proposes five types—YOLOv5n, YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x—based on the differences in network structure depth and width. While the parameters and performance of the models increase sequentially, the detection speed gradually decreases. The YOLOv5n/s models have a small backbone feature extraction network and are lightweight, but their target bounding box regression is not sufficiently accurate for practical applications. The YOLOv5m/l/x models have better detection and recognition performance with increasing network depth and width, but they struggle to meet real-time detection requirements on hardware-limited embedded devices. To address this problem, this paper proposes a lightweight object detection algorithm based on the YOLOv5s network, which combines the shuffle attention (SA) module, the depthwise-decoupled head (DD-head) module, the content-aware reassembly of features (CARAFE) module, the GSConv module, and the spatial pyramid pooling cross-stage partial GSConv (SPPCSPG) module to improve the detection accuracy of the model while meeting real-time requirements. The main contributions and innovations of the constructed model can be summarized as follows:

(1)

In this paper, depthwise convolution is used to replace the standard convolution in the decoupled head module to construct a new detection head, the DD-head, which can improve the negative impact of classification and regression task conflicts while reducing the parameter volume of the decoupled head.

(2)

Based on the SPPCSPC module, this paper utilizes the design principle of the GS bottleneck and replaces the CBS module in the SPPCSPC module with the GSConv module to design a lightweight SPPCSPG module, which is introduced into the backbone structure to optimize the YOLOv5s network model.

(3)

The effect of embedding the SA module in the network backbone, neck, and head regions is studied, and the SA module is ultimately embedded in the head region to enhance the spatial attention and channel attention of the feature map, thereby improving the accuracy of multi-scale object detection.

(4)

The CARAFE module is used to replace the nearest neighbor interpolation up-sampling module to reassemble feature points with similar semantic information in a content-aware manner and aggregate features in a larger receptive field, achieving the up-sampling operation.

(5)

In this study, the Conv module in the variety of view-GS cross-stage partial (VoV-GSCSP) module is replaced by the GSConv module to reconstruct a new VoV-GSCSP module to further reduce the model’s parameters. The GSConv module and the improved VoV-GSCSP module are embedded in the neck structure to maintain the model’s detection accuracy while reducing the parameter volume.

Experimental results show that the proposed algorithm outperforms the original YOLOv5s algorithm in multi-scale object detection performance while meeting real-time requirements.

2. Related Work

2.1. Object Detection Algorithms for Remote Sensing Images

Traditional remote sensing image object detection algorithms are based on handcrafted feature design. The detection process typically includes candidate region extraction [44], feature extraction [45], classifier design [46], and post-processing. First, potential target regions are extracted from the input image using candidate region extraction. For each region, features are extracted; then, the extracted features are classified. Finally, post-processing, such as filtering and merging, is applied to all candidate boxes to obtain the final detection results. Candidate region extraction requires the setting of a large number of sliding windows, which results in a high time complexity and a significant amount of redundant computation. Handcrafted features are mainly extracted based on target visual information (such as color [47], texture [48], edges [49], context [50], etc.), giving them strong interpretability. However, handcrafted features have weak feature expression capability, poor robustness, limited adaptability, and are difficult to apply in complex and changing environments.

With the development of deep learning, the deep features extracted by neural networks have stronger semantic representation and discriminative ability. However, due to the characteristics of remote sensing images, such as large image size, significant directional changes, large-scale small targets, dense target distribution, significant scale variations, target blurring, and complex backgrounds, existing detection algorithms cannot achieve satisfactory performance on remote sensing images. To address the problem of large image size, R²-CNN [51] was designed using a lightweight backbone network, Tiny-Net, for feature extraction and used the approach of judging first and then locating to filter out sub-image blocks without targets, thereby reducing the computational burden of subsequent detection and recognition. For the problem of significant directional changes, the approaches of data augmentation [52] or adding rotation-invariant sub-modules [53] are typically used to solve this problem. Cheng et al. [54] explicitly increased rotation-invariant regularizers on CNN features by optimizing a new objective function to force the feature representation of training samples before and after rotation to be closely mapped to achieve rotational invariance. For the small target problem, Yang et al. [55] increased the number and scale of shallow feature pyramids to improve the detection accuracy of small targets and used a dense connection structure to enhance the feature expression ability of small targets. Zhang et al. [56] improved small target detection by up-sampling and enlarging the feature map size of each candidate region in the first stage of the two-stage Faster R-CNN. To address the problem of dense target distribution, DAPNet [57] used an adaptive region generation strategy based on the density of targets in the image. For the problem of significant scale variations, Guo et al. [58] and Zhang et al. [59] directly used a multi-scale candidate region network and a multi-scale detector to detect targets of different scales. For the problem of target blurring, Li et al. [60] proposed a dual-channel feature fusion network that can learn local and contextual attributes along two independent paths and fuse features to enhance discriminative power. Finally, for the problem of complex backgrounds, Li et al. [61] extracted multi-scale features and used an attention mechanism to enhance each feature map individually, thereby eliminating the influence of background noise.

2.2. Attention Mechanism

To extract effective information from massive and complex data, researchers have proposed attention mechanisms to obtain the importance differences of each feature map. In the visual system, attention mechanisms are considered as dynamic selection processes that adaptively weigh the features based on their importance differences in the input [62]. Currently, attention mechanisms have achieved good performance in tasks such as image classification [63], object detection [64], semantic segmentation [65], medical image processing [66], super-resolution [67], and multimodal tasks [68]. Attention mechanisms can be classified into the following six categories. (1) Channel attention: In deep neural networks, different channels in various feature maps typically represent different objects. Channel attention adaptively adjusts the weight of each channel to increase the importance of focused objects. Hu et al. [69] were the first to propose the concept of channel attention and to introduce SENet. The SENet module collects global information [70] using squeeze and excitation modules, captures channel relationships, and improves the representation power of the network. (2) Spatial attention: Spatial attention is an adaptive mechanism for selecting spatial regions. The representative algorithms of spatial attention include RAM [71], based on the RNN method; STN, which uses subnetworks to explicitly predict relevant regions [72]; GENet [73], which uses subnetworks implicitly to predict soft masks for selecting important regions; and GCNet [74], which uses a self-attention mechanism [75]. (3) Temporal attention: Temporal attention can be regarded as a dynamic temporal selection mechanism for determining when to focus attention, typically achieved by capturing short-term and long-term cross-frame feature dependencies [76,77]. Li et al. [76] proposed a global–local temporal representation (GLTR) to utilize multi-scale temporal information in video sequences. GLTR consists of a dilated temporal pyramid (DTP) for local temporal context learning and a temporal self-attention module for capturing global temporal interaction. (4) Branch attention: Branch attention can be regarded as a dynamic branch selection mechanism; it is often used in conjunction with multi-branch structures. Representative networks include highway networks [78], Selective Kernel (SK) convolution [79], CondCov operator [80], and dynamic convolution [81]. (5) Channel and spatial attention: Channel and spatial attention combines the advantages of channel and spatial attention and can adaptively select important objects and regions [82]. Based on the ResNet network [83], the Residual Attention Network [84] pioneers the research of channel and spatial attention by combining attention mechanism and residual connection, emphasizing the importance of information features in spatial and channel dimensions. Woo et al. [63] proposed the convolutional block attention module (CBAM) by concatenating channel and spatial attention; it decouples channel and spatial attention to improve computational efficiency. (6) Spatial and temporal attention: Combining the advantages of spatial and temporal attention, it can adaptively select important regions and key frames. Song et al. [85] proposed a joint spatial and temporal attention network based on LSTM [86], which enables the adaptive discovery of discriminative features and key frames.

2.3. Multi-Scale Feature Fusion

In object detection tasks, feature maps at different levels represent varying information about the detection targets. High-level feature maps encode semantic information about the objects which can be used for classification, while low-level feature maps encode positional information about the objects which can be used for regression [87]. The YOLO algorithms fuse multiple features obtained from neural networks to extract more information about small targets to improve detection accuracy. The Feature Pyramid Network (FPN) [42] enhances semantic feature representation through a top-down pathway and fuses features with more precise location information. However, FPN fails to propagate accurate localization information from lower-level feature maps to higher-level semantic feature maps, and the feature transfer between non-adjacent layers is limited. In addition, for masks generated for large targets, the redundant and lengthy spatial transfer path hinders the effective integration of high-level and low-level information, leading to information loss. Liu et al. [43] proposed the PANet network, which incorporates a bottom-up pathway enhancement structure and integrates shallow network features with FPN features. To improve upon the suboptimal fusion performance of manually designed feature pyramids, Ghiasi et al. [88] introduced the neural architecture search-feature pyramid network (NAS-FPN), which utilizes neural network architecture search methods to automatically design the feature network. The bidirectional feature pyramid network (BiFPN) [89] improves upon the PANet by introducing contextual [90] and weight information to balance features of different scales, resulting in a larger receptive field and richer semantic information. To address the issue of inconsistent feature scales in pyramid-based methods, Liu et al. [91] proposed a data-driven pyramid feature fusion strategy called adaptive spatial feature fusion (ASFF), which enables the network to learn how to directly filter out features from other levels in space to preserve useful information for combination. Subsequent research has shown the effectiveness of BiFPN [92,93,94] and ASPP [95,96,97] in improving the detection performance of YOLO algorithms.

3. YOLOv5 Algorithm

Among the existing object detection algorithms, the YOLOv5 algorithm has gained wide popularity in various applications due to its fast detection speed, high accuracy, and good flexibility. Based on differences in network depth and width, the proposed YOLOv5 model can be categorized into five types: YOLOv5n, YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x. These models demonstrate a progressive increase in number of parameters and level of performance, but with a corresponding decrease in detection speed. Considering both detection performance and speed, this research selected the YOLOv5s model. This model comprises four main components: input, backbone, neck, and head. Figure 1 illustrates the network structure diagram.

3.1. Input

YOLOv5 performs adaptive image scaling and Mosaic data augmentation, as well as optimized anchor box calculations at the input end for image data [38].

Mosaic is a data augmentation method based on Cutmix [98]. By combining four training images into one image and scaling the resulting image to the standard size before training, Mosaic can effectively improve object detection beyond normal backgrounds. During training, each batch of data contains a large number of images, and data augmentation increases the number of images in each batch fourfold, which reduces the requirement for large quantities when estimating mean and variance. Adaptive image scaling is only performed during the model inference stage. First, the scaling ratio is calculated based on the original image and the input network image size. Then, the scaled image size is determined by multiplying the original image size by the scaling ratio. Finally, the image is scaled to fit the input size of the network. In the YOLOv2–4 algorithms [36,37,38], prior box dimensions need to be extracted using K-means clustering [99]. To train on different datasets, a separate program is required to obtain the initial anchor boxes to meet specific size requirements. YOLOv5 embeds adaptive anchor box calculation into its code, which automatically calculates the optimal anchor boxes during each training session based on the dataset.

3.2. Backbone

The backbone network primarily extracts feature information from input images. The C3 module and SPPF module are mainly used in the YOLOv5 network. The C3 module reduces model computation and improves inference speed, while the SPPF module extracts multi-scale information from feature maps, which is beneficial for improving model accuracy.

The C3 module consists of three standard convolutional layers and multiple bottleneck modules, the number of which is determined by the parameters specified in the configuration file. The C3 module is the main module for learning residual features. Its structure consists of two branches: one branch uses the specified multiple bottleneck modules stacked and standard convolutional layers, while the other branch only passes through standard convolutional layers. Finally, the two branches are concatenated and passed through standard convolutional layers to output the final feature map.

SPP [41] can fuse feature maps of different scales and sizes by performing fixed-size pooling on feature maps of any scale to obtain a fixed number of features. Then, each pooled feature is concatenated to obtain a fixed-length feature map. The principle of the SPPF module is similar to SPP, with a slightly different structure. In YOLOv5, SPP uses three scales of features [5,9,13] to fuse with the input feature. The results further improve the scale invariance of input images with different scales and aspect ratios. On the other hand, SPPF only uses a 5 × 5 pooling kernel. After the input image passes through a standard convolutional layer, it goes through three stacked 5 × 5 pooling kernels. Each scale feature after pooling is fused with the scale feature after passing through the standard convolutional layer to obtain the final feature map. Compared with the SPP module, the computational complexity of the SPPF module is greatly reduced, and the model speed is improved.

3.3. Neck

The neck network of YOLOv5 consists of a feature pyramid network (FPN) [42] and a path aggregation network (PAN) [43]. FPN always uses the semantic information of high-level features and high-resolution location information of low-level features simultaneously by using a top-down approach to propagate semantic information. On the other hand, PAN uses a bottom-up approach to facilitate the propagation of low-level information to the top level for better localization. The three sizes of feature maps output by the backbone network are aggregated by the neck network to enhance semantic information and localization features, which helps to improve the ability to detect objects of different sizes.

3.4. Head

As the detection component of the object detection model, the head predicts objects of different sizes by processing multi-scale feature maps. The anchor box mechanism at the head extracts prior box scales through clustering and constrains the predicted box positions.

The model outputs three scale tensors, with the first scale having an eight-fold down-sampling compared to the input image, resulting in a smaller receptive field that preserves high-resolution features from the bottom layers and is beneficial for detecting small objects. The second scale has a 16-fold down-sampling, resulting in a moderately sized receptive field that is beneficial for detecting medium-sized objects. The third scale has a 32-fold down-sampling, resulting in a larger receptive field that is beneficial for detecting large objects.

3.5. Loss Function

The loss function (${\mathcal{L}}_{total}$) of YOLOv5 consists of three components defined as Equation (1), which covers several necessary loss function modules in object detection such as confidence loss function, class prediction loss function, and bounding box prediction loss function [100].

$${\mathcal{L}}_{total}={\mathcal{L}}_{obj}+{\mathcal{L}}_{class}+{\mathcal{L}}_{bbox}$$

(1)

where ${\mathcal{L}}_{obj}$ represents the target confidence loss of the model, ${\mathcal{L}}_{class}$ represents the target class prediction loss of the model, and ${\mathcal{L}}_{bbox}$ represents the bounding box loss of the model.

$$\begin{gathered} {\mathcal{L}}_{obj}={\lambda }_{obj}{\displaystyle \sum }_{i=0}^{S^2}{\displaystyle \sum }_{j=0}^B{I}_{ij}^{obj}\left[-{\widehat{C}}_i^j\ln \left(C_i^j\right)-\left(1-{\widehat{C}}_i^j\right)\ln \left(1-C_i^j\right)\right] \\ +{\lambda }_{nobj}{\displaystyle \sum }_{i=0}^{S^2}{\displaystyle \sum }_{j=0}^B{I}_{ij}^{nobj}\left[-{\widehat{C}}_i^j\ln \left(C_i^j\right)-\left(1-{\widehat{C}}_i^j\right)\ln \left(1-C_i^j\right)\right] \end{gathered}$$

(2)

$${\mathcal{L}}_{class}={\displaystyle \sum }_{i=0}^{S^2}{\displaystyle \sum }_{j=0}^B{\displaystyle \sum }_{c\subset classes}\left[-{\widehat{p}}_i\left(c\right)\ln \left(p_i\left(c\right)\right)-\left(1-{\widehat{p}}_i\left(c\right)\right)\ln \left(1-p_i\left(c\right)\right)\right]$$

(3)

where $S$ represents the grid size, $B$ represents the number of predicted boxes per grid, and $I_{ij}^{obj}$ represents whether the $i_{th}$ predicted box in the $j_{th}$ grid contains an object. If the overlap between the predicted box and the ground truth box exceeds the threshold, $I_{ij}^{obj}$ is set to 1, indicating the presence of an object to be predicted, and it is included in the calculation of the loss function; otherwise, $I_{ij}^{obj}$ is set to 0. $I_{ij}^{nobj}$ represents whether the $i_{th}$ predicted box in the $j_{th}$ grid contains a background object. If the overlap between the predicted box and the ground truth box is less than the threshold, $I_{ij}^{nobj}$ is set to 1; otherwise, $I_{ij}^{nobj}$ it is set to 0. ${\lambda }_{obj}$ and ${\lambda }_{nobj}$ are balance coefficients used to adjust the balance between the confidence loss in the presence and absence of objects. $C_i^j$ represents the confidence of the predicted box, ${\widehat{C}}_i^j$ represents the confidence of the ground truth box, $p_i\left(c\right)$ represents the predicted probability of the $c$ class when the $i_{th}$ network detects an object, and ${\widehat{p}}_i\left(c\right)$ represents the true probability of the $c$ class when the $i_{th}$ network detects an object.

The YOLOv5 bounding box prediction loss function utilizes the CIOU loss function (${\mathcal{L}}_{CIOU}$) [101], and its definition is as follows:

$${\mathcal{L}}_{CIOU}={\displaystyle \sum }_{i=0}^{S^2}{\displaystyle \sum }_{j=0}^{B}1-IOU+\frac{{\rho }^2\left(b,b^{gt}\right)}{c^2}+\alpha v$$

(4)

$$IOU==\frac{\left|B\cap B^{gt}\right|}{\left|B\cup B^{gt}\right|}$$

(5)

where ${\rho }^2\left(b,b^{gt}\right)$ represents the Euclidean distance between the centers of the predicted box and the ground-truth box; $c$ represents the diagonal distance of the minimum enclosing region that can simultaneously contain the predicted and ground-truth boxes; $B$,$B^{gt}$, and $\alpha$ respectively denote the predicted box, the ground-truth box, and the weight coefficient; $v$ is used to measure the similarity of aspect ratios; and the formulas for $\alpha$ and $v$ are given by Equations (6) and (7), respectively:

$$\alpha =\frac{v}{\left(1-IOU\right)+v}$$

(6)

$$v=\frac{4}{{\pi }^2}{\left(\arctan \frac{w^{gt}}{h^{gt}}-\arctan \frac{w^p}{h^p}\right)}^2$$

(7)

where $w^{gt}$ and $h^{gt}$ respectively represent the width and height of the ground-truth box; and $w^p$ and $h^p$ respectively represent the width and height of the predicted box.

4. Improved YOLOv5s Algorithm

4.1. Shuffle Attention Module

As the hierarchical depth of the network increases, the information extracted from the head of the YOLOv5s network becomes increasingly abstract which will lead to missed or false detection of small objects in the image. In this study, an attention mechanism was incorporated into the YOLOv5s network to address this issue.

The attention mechanism can be mainly divided into spatial attention and channel attention, which are used to capture pixel relationships in space and dependencies between channels, respectively. The combination of these two attention mechanisms, such as in CBAM, can achieve better results but inevitably increases the computational complexity of the model. The SGE attention mechanism module [102] is a classic attention module. Its core idea is to group feature maps, with each group of feature maps representing a semantic feature. By utilizing the similarity between local and global features, the attention mask is generated to guide the spatial distribution of enhanced semantic features. Based on the design concept of the SGE attention mechanism, the shuffle attention (SA) mechanism [103] introduces the channel shuffle operation, which uses both spatial and channel attention mechanisms in parallel, efficiently combining the two. As shown in Figure 2, the SA module first groups the c × h × w feature map obtained by convolution, and the grouped feature map serves as the SA unit. Each SA unit is divided into two parts, with the upper part using the channel attention mechanism and the lower part using the spatial attention mechanism. The processed two parts are stacked by channel numbers to achieve information fusion within the SA unit. Finally, the channel shuffle operation is applied to all SA units to realize information communication between different sub-features and obtain the final output feature map.

The SA module has the convenient feature of being plug-and-play and has been applied in some networks. However, there is currently no absolute theoretical basis for determining which part of the YOLOv5s network is best in terms of integration. The YOLOv5s network can be divided into four modules: input, backbone, neck, and head. The input module mainly performs preprocessing operations on images; it does not perform feature extraction processing on images. Therefore, in this study, fusion network models that incorporate the SA module into the backbone, neck, and head modules of YOLOv5s were designed and named YOLOv5s-SA-A, YOLOv5s-SA-B, and YOLOv5s-SA-C, respectively.

The SA module was embedded into the backbone structure to form the YOLOv5s-SA-A network. The backbone extracts the feature information from images through a relatively deep convolutional network. As the network layers deepen, the resolution of the feature map decreases. The SA module can be used for spatial attention enhancement and channel attention reconstruction of feature maps at different locations. The C3 module aggregates features at different levels. In this study, the SA module was placed after the C3 module. The network structure is shown in Figure 3A. The SA module was embedded into the neck structure to form the YOLOv5s-SA-B network. The FPN and PAN structures in the neck module can transmit semantic information from top to bottom and positional information from bottom to top, thereby enhancing the aggregation of semantic information and positioning features. This module uses four Concat operations to fuse deep and shallow information. Therefore, the SA module was placed after the Concat operation to enhance the spatial attention and channel reconstruction of the fused feature map. The network structure is shown in Figure 3B. The SA module was embedded into the head structure to form the YOLOv5s-SA-C network. The YOLOv5s network predicts targets using three feature maps of different scales. Large targets are predicted on small feature maps, while small targets are predicted on large feature maps. In this study, the SA module was embedded before the prediction head to enhance the spatial attention and channel reconstruction of each feature map. The network structure is shown in Figure 3C.

4.2. DD-Head Module

Traditional YOLO algorithms use the coupled head, which utilizes the same convolutional layer for both classification and regression tasks at the head of the network. However, classification and regression tasks have different focuses. Classification is more concerned with the texture of each sample, while regression is more focused on the edge features of object images. Studies [104,105] have pointed out that there is a conflict between classification and regression tasks in object detection, and using a coupled head for both tasks may lower the model’s performance.

The YOLOX algorithm [106] is the first algorithm to apply the decoupled head module, achieving more significant results than the coupled head. The network structure of the decoupled head is shown in Figure 4. For the input feature map, the decoupled head first uses a 1 × 1 convolution to reduce its dimensionality, mapping the feature maps of P3, P4, and P5 with different dimensions of the feature fusion network output into feature maps with a unified number of channels. Then, two parallel channels are used to perform object regression and target box coordinate regression tasks. To reduce the complexity of the decoupled head and improve model convergence speed, each channel uses two 3 × 3 convolutions. Cls., Reg., and Obj. output values can be obtained through processing, where Cls. represents the category of the target box, Reg. represents the position information of the target box, and Obj. represents whether each feature point contains an object. The final prediction information is obtained by fusing the three output values. In summary, the decoupled head improves model performance by separately addressing classification and regression tasks.

Although introducing a decoupled head can effectively improve the detection performance of a network, it undoubtedly increases the model’s parameter count and decreases the detection speed. In this study, the 3 × 3 convolution in each branch of the decoupled head network was replaced with a 3 × 3 depthwise convolution [107], reducing the parameter count. The network architecture is shown in Figure 5, which is named the DD-head. The original coupled head in the YOLOv5s model was replaced with the DD-head to mitigate the negative impact of the classification and regression task conflict.

4.3. Content-Aware Reassembly of the Features Module

The YOLOv5s model utilizes the FPN module to achieve top-down semantic information transfer and multi-scale object detection through feature fusion. The multi-scale feature fusion is achieved by nearest-neighbor interpolation up-sampling to unify the feature map size. However, this up-sampling operation presents two limitations: (1) the interpolation up-sampling operation only considers the spatial information of the feature map and ignores its semantic information, resulting in simultaneous up-sampling of target and noise positions; (2) the receptive field of the interpolation up-sampling operation is usually small, leading to insufficient use of global feature information. Another adaptive up-sampling operation is the deconvolution operation; however, it also has two limitations: (1) the deconvolution operator uses the same convolution kernel across the entire feature map, regardless of the underlying information, which limits its ability to respond to local changes; (2) the parameter volume of the deconvolution operation is large, which reduces the detection speed of the network. To address the problem of low semantic correlation in up-sampling in object detection models, this study adopts the Content-Aware Reassembly of Features (CARAFE) module [108] to replace the nearest-neighbor interpolation up-sampling module. The CARAFE module recombines feature points with similar semantic information in a content-aware manner and aggregates features in a larger receptive field to achieve the up-sampling operation.

The up-sampling process of the CARAFE module mainly consists of two steps: up-sampling kernel prediction and content-aware reassembly. Firstly, channel compression is performed by the up-sampling kernel prediction module to reduce the number of input feature channels. Then, the compressed feature map is encoded with content and the reassembly kernel is predicted according to the content of each target location. Finally, the content-aware reassembly module performs a dot product between the reassembly kernel and the corresponding region of the original feature map to complete the up-sampling process.

4.4. GSConv Module

Although the introduction of the decoupled head and SPPCSPC modules can improve the detection performance of the YOLOv5s network model, these modules increase the parameter count of the model, which is unfavorable for creating lightweight networks. To design lightweight networks, deep separable convolution (DSC) modules are typically used instead of conventional convolutional modules. The advantage of DSC modules is their efficient computational capability, as their parameter count and computational workload are approximately one-third of those of traditional convolutional modules. However, during the feature extraction process, the channel information of the input image is separated in the calculation process, which can result in lower feature extraction and fusion capabilities compared to standard convolutional modules. To effectively utilize the computational capability of DSC and ensure that its detection accuracy reaches the level of standard convolution (SC), the GSconv [109] module is proposed based on SC, DSC, and shuffle modules. The network structure is shown in Figure 6. Firstly, the feature map with C₁ channels is split into two parts, where half of the feature map is used for deep separable convolution and the remaining part is used for standard convolution. Then, the two-channel feature maps are combined for feature concatenation. Shuffle is a channel mixing technique that allows information from the SC module to completely mix with the DSC output by transmitting its feature information across various channels, thus achieving channel information interaction.

During the convolution process, the spatial information of the feature map gradually shifts to channels, where the number of channels increases when the width and height of the feature map decrease, resulting in stronger semantic information. However, each spatial compression and channel expansion of the feature map can lead to partial loss of semantic information, which affects the accuracy of object detection. The SC module largely preserves the hidden connections between each channel, which can reduce information loss to some extent, but with high time complexity. In contrast, the DSC module cuts off these hidden connections, resulting in the complete separation of channel information during the calculation process. The GSconv module retains as many connections as possible while maintaining lower time complexity, thereby reducing information loss and achieving faster operations and thus unifying SC and DSC.

Based on the GSConv module, the network structure of the GS bottleneck and VoV-GSCSP module is shown in Figure 7. Compared with the bottleneck module in the original YOLOv5s network, the GS bottleneck replaces the two 1 × 1 convolutions in the bottleneck module with the GSConv module and adds new skip connections. Therefore, the two branches of the GS bottleneck perform separate convolutions without weight sharing, propagating channel information through different network paths by dividing the number of channels. As a result, the information propagated by the GS bottleneck shows greater correlation and diversity, resulting in more accurate information and reduced computational workload. The VoV-GSCSP module is designed by using the GS bottleneck instead of the bottleneck in the C3 module. In the VoV-GSCSP module, the input feature map is also divided into two parts based on channel numbers. The first part is processed by a convolutional module and features are extracted by stacking the GS bottleneck, while the other part serves as the residual connection and is convolved by a convolutional module. The two feature maps are then concatenated based on channel numbers and passed through a convolutional module for output. The VoV-GSCSP module inherits the advantages of both the GSConv module and the GS bottleneck. With the new skip connection branch, the VoV-GSCSP module has a stronger nonlinear representation, effectively addressing the problem of gradient vanishing. At the same time, the split-channel method of VoV-GSCSP achieves rich gradient combinations, solving the problem of redundant gradient information and improving learning ability [109]. Experimental results have shown that the VoV-GSCSP module not only reduces computational workload but also improves model accuracy [110,111]. In this study, the Conv module in the VoV-GSCSP module was replaced with the GSConv module to further reduce model parameters. The GSConv module and the improved VoV-GSCSP module were embedded into the neck structure of the model, so as to maintain detection accuracy while reducing parameter count.

4.5. SPPCSPG Module

The SPPCSPC module [112] is built on the basis of the SPP module and the CSP structure, as shown in Figure 8. The module first divides the features extracted by the C3 module of the backbone into two parts: the SPP and conventional convolution operations. The SPP structure consists of four branches, corresponding to max-pooling operations with pool kernel sizes of 1, 5, 9, and 13. These four different pool kernels allow the SPPCSPC structure to handle objects with four different receptive fields, better distinguishing small and large targets. Finally, the SPP operation and conventional convolution operation are merged together by Concat to achieve faster speed and higher accuracy.

Although the SPPCSPC module can improve the detection performance of the model to a certain extent, it also increases the model’s parameter count. Therefore, in this study, a lightweight SPPCSPG module was proposed based on the design principles of the GS bottleneck and the GSConv module. The SPPCSPG module was integrated into the backbone structure to optimize the YOLOv5s network model.

Incorporating the improvements to the YOLOv5s backbone, this study replaced the SPPF module with the SPPCSPG module to enhance detection accuracy. In the neck structure, all Conv modules were replaced with GSConv modules and the improved VoV-GSCSP module was introduced to reduce the parameters and computation brought about by feature pyramid structure upgrades. To address the issue of limited semantic information and receptive fields caused by nearest-neighbor interpolation up-sampling operations in the original network model, this study adopted the CARAFE module to replace the nearest-neighbor interpolation up-sampling module, which reorganizes feature points with similar semantic information in a content-aware way and aggregates features in a larger receptive field to perform up-sampling operations. To address the problem of inaccurate target localization and weak feature expression capability in the original network model, this study introduced the SA attention module in the head module. Finally, to improve the performance metrics of the model, the DD-head was used in the detection layer of the YOLOv5s model to accomplish classification and regression tasks. The improved network model structure is shown in Figure 9.

5. Experimental Results and Analysis

5.1. Experimental Platform and Dataset

The dataset used in the experiment includes two remote sensing datasets, the RSOD dataset [113,114] and DIOR dataset [115], as well as two general target detection datasets, the PASCAL VOC dataset [116,117] and MS-COCO dataset [118].

The RSOD dataset includes four types of detection object categories, aircraft, oil tank, playground, and overpass, with a total of 976 images. The aircraft category includes 446 images, with a total of 4993 targets. The oil tank category includes 165 images, with a total of 1586 targets. The playground category includes 189 images, with a total of 191 targets. The overpass category contains 176 images and a total of 180 targets. The partition ratio between the training set to the test set is 4 to 1.

The DIOR dataset is a large-scale benchmark dataset for object detection in remote sensing images. The dataset includes 23,463 images of different seasons and weather patterns, with a total of 190,288 targets. The unified image size is 800 × 800, with a resolution of 0.5 m to 30 m. DIOR datasets include 20 categories: airplane (AL), airport (AT), baseball field (BF), basketball court (BC), bridge (B), chimney (C), dam (D), expressway service area (ESA), expressway toll station (ETS), golf course (GC), ground track field (GTF), harbor (HB), overpass (O), ship (S), stadium (SD), storage tank (ST), tennis court (TC), train station (TS), vehicle (V), and windmill (W). According to the original settings in the DIOR dataset, the number of images in the training, validation, and testing sets is 5863, 5862, and 11,738, respectively. This study combines the training set and validation set as the training set.

The PASCAL VOC dataset includes the PASCAL VOC 2007 and 2012 datasets, which can be used for tasks such as image classification, object detection, semantic segmentation, and motion detection. The PASCAL VOC dataset includes a total of 20 common objects in daily life. In this study, the training and validation sets of the PASCAL VOC 2007 and VOC 2012 datasets were used as the model’s training set, while the testing set of VOC 2007 was used as the model’s testing set. The MS-COO dataset is currently the most challenging target detection dataset which includes a total of 80 types of detection objects. The MS-COO dataset includes more small objects (with an area smaller than 1% of the image) and more dense localization objects than the PASCAL VOC dataset.

The experiments were conducted on a system with Ubuntu 18.04, CUDA 11.1, and a GeForce RTX A5000 graphics card. The network development framework used was Pytorch 1.9, and the integrated development environment was Pycharm. The training was uniformly set to 300 epochs, with a batch size of 64.

5.2. Evaluation Metrics

In this study, precision, mean average precision (mAP), and detection frames per second (FPS) were used as performance evaluation metrics for the object detection method. Precision (P) and recall (R) were calculated using Equations (8) and (9), respectively:

$$P=\frac{TP}{TP+FP}$$

(8)

$$R=\frac{TP}{TP+FN}$$

(9)

where TP represents the number of positive samples that are correctly identified as positive, FP represents the number of negative samples that are incorrectly identified as positive, and FN represents the number of positive samples that are incorrectly identified as negative. By selecting different precision and recall values, the precision-recall (PR) curve can be drawn, and the area under the PR curve is defined as the AP. The mean AP (mAP) is calculated by taking the mean of the AP for all detection categories. The calculation of the performance evaluation metrics AP and mAP is shown in Equations (10) and (11), respectively:

$$AP={{\displaystyle \int }}_0^{1}p\left(r\right)dr$$

(10)

$$mAP=\frac{1}{n}{\displaystyle \sum }_{i=1}^{n}A{P}_i$$

(11)

where $p\left(r\right)$ represents the precision value at a certain recall value r, and n represents the number of detection categories.

5.3. Experimental Results and Analysis

The experimental results are divided into seven sections based on two different datasets: the comparison experiment of SA module embedding, the effect experiment of the DD-head module, the effect experiment of the SPPCSPG module, the RSOD dataset experiment, the DIOR dataset experiment, the PASCAL VOC dataset experiment, and the MS COCO dataset experiment.

5.3.1. Performance Evaluation of the SA Module Embedded Model

To explore the best SA module embedding model and investigate the detection performance changes brought by embedding the SA module in different structures of the YOLOv5s network, three proposed models (YOLOv5s-SA-A, YOLOv5s-SA-B, and YOLOv5s-SA-C) were evaluated using the RSOD dataset to achieve better optimization design of the network. The detection performance of the original YOLOv5s and improved models were compared, and the experimental results are shown in

Table 1. Performance evaluation of shuffle attention module embedded model.

Method	Data Size	Param.	GFLOPs	Precision (%)	Recall (%)	[email protected] (%)	FPS
YOLOv5s	640 × 640	7.02 M	15.8	0.942	0.932	0.950	119.0
YOLOv5s-SA-A	640 × 640	7.02 M	15.8	0.930	0.920	0.948	105.8
YOLOv5s-SA-B	640 × 640	7.02 M	15.8	0.937	0.935	0.948	112.3
YOLOv5s-SA-C	640 × 640	7.02 M	15.8	0.939	0.930	0.952	113.9

.

From Table 1, it can be seen that not all networks with integrated SA modules can improve detection performance. YOLOv5s-SA-A had significantly decreased precision, recall, mAP, and FPS compared to the original YOLOv5s network. YOLOv5s-SA-B showed improved recall, but the other three indicators were decreased compared to the original network. YOLOv5s-SA-C had increased mAP, but the other three indicators were decreased compared to the original network. The reason for the different experimental results when embedding attention mechanisms at different positions in the network is that the feature maps extracted by the backbone have rich semantic features, while the feature maps extracted by the neck and head have larger receptive fields, which play a crucial role in improving object detection performance. In the backbone module, the feature maps retain the shallow texture and contour information of the targets, with poor semantic information; thus, embedding attention mechanisms cannot effectively learn semantic information. The YOLOv5s-SA-C algorithm is superior to the YOLOv5s-SA-B algorithm in both detection accuracy and speed. Therefore, considering the principle of balancing accuracy and speed, the YOLOv5s-SA-C algorithm was finally chosen as the model for embedding SA modules.

5.3.2. Effect Experiment of the DD-Head Module

Experiments were conducted on the proposed DD-head module, decoupled head module, and coupled head module in the RSOD dataset to investigate the impact of the constructed DD-head module on the detection accuracy and speed of the model.

Table 2. Performance evaluation of depthwise-decoupled head module embedded model.

Method	Data Size	Param.	GFLOPs	Precision (%)	Recall (%)	[email protected] (%)	FPS
YOLOv5s	640 × 640	7.02 M	15.8	0.942	0.932	0.950	119.0
YOLOv5s + Decoupled head	640 × 640	14.33 M	56.2	0.937	0.969	0.956	94.9
YOLOv5s + DD-head	640 × 640	7.28 M	16.7	0.960	0.929	0.955	106.8

shows that both the decoupled head and the proposed DD-head modules can improve the mean average precision accuracy of the model, with an increase of 0.6% and 0.5%, respectively. Although the DD-head module has a 0.1% lower mean average precision accuracy compared to the decoupled head module, it has a 7.05 M lower parameter count and an 11.9 increase in FPS value compared to that of the decoupled head module.

5.3.3. Effect Experiment of the SPPCSPG Module

Experiments were conducted on the proposed SPPCSPG module, SPPCSPC module, and SPPF module in the RSOD dataset to study the impact of the constructed SPPCSPG module on the detection accuracy and speed of the model.

Table 3. Performance evaluation of spatial pyramid pooling cross-stage partial with GSConv module embedded model.

Method	Data Size	Param.	GFLOPs	Precision (%)	Recall (%)	[email protected] (%)	FPS
YOLOv5s	640 × 640	7.02 M	15.8	0.942	0.932	0.950	119.0
YOLOv5s + SPPCSPC	640 × 640	13.45 M	20.9	0.947	0.931	0.953	117.0
YOLOv5s + SPPCSPG	640 × 640	9.95 M	18.1	0.942	0.954	0.955	113.1

indicates that both the SPPCSPC module and the proposed SPPCSPG module can improve the mean average precision accuracy of the model, with an increase of 0.3% and 0.5%, respectively. Moreover, the SPPCSPG module has a 0.2% higher mean average precision accuracy than the SPPCSPC module and has 3.5 M fewer parameters. Therefore, based on the evaluation of parameter count and detection accuracy, the proposed SPPCSPG module outperforms the SPPCSPC module.

5.3.4. Performance Comparison in the RSOD Dataset

The detection performance of the original YOLOv5s model and the constructed model was compared on the RSOD dataset to verify the effectiveness of the constructed model. The results were shown in

Table 4. RSOD test detection results.

Method	Param.	GFLOPs	[email protected] (%)	Aircraft	Oil Tank	Playground	Overpass	FPS
YOLOv5s	7.02 M	15.8	0.950	0.983	0.986	0.839	0.994	119.0
Ours	9.67 M	18.4	0.964	0.981	0.987	0.893	0.992	88.8

. The mean average precision accuracy and FPS of the model constructed in this article with the original YOLOv5s model were compared on the RSOD dataset, and the detection accuracy of each category was compared. It can be seen that the mean average precision accuracy of the model constructed in this article is 96.4%, which is an improvement of 1.4% compared to the original model, and the mean average precision accuracy on categories oil tank and playground were improved by 0.1% and 5.4%, respectively.

In order to further visually demonstrate the effectiveness of the constructed models, the detection results of each model were shown in Figure 10.

As shown in Figure 10, from rows 1 to 4, it could be seen that the detection boxes of the model constructed in this article were closer to the actual detection boxes of targets, with a higher recall rate. From row 4, it can be seen that the model constructed in this article had a lower false alarm rate on the overpass. Therefore, the effectiveness of the model constructed in this article is superior to the original model.

5.3.5. Performance Comparison in the DIOR Dataset

In order to further validate and evaluate the effectiveness of the improved YOLOv5s in improving the detection accuracy of the model, experimental comparisons were conducted on the DIOR dataset with other methods specified in the literature. The experimental results are shown in

Table 5. DIOR test detection results.

Method	mAP	AL	AT	BF	BC	B	C	D	ESA	ETS	GC
HawkNet [119]	72.0	65.7	84.2	76.1	87.4	45.3	79.0	64.5	82.8	72.4	82.5
CANet [120]	74.3	70.3	82.4	72.0	87.8	55.7	79.9	67.7	83.5	77.2	77.3
Yao et al. [121]	75.8	91.0	74.5	93.3	83.2	47.4	91.9	63.3	68.0	61.4	80.0
MFPNet [122]	71.2	76.6	83.4	80.6	82.1	44.3	75.6	68.5	85.9	63.9	77.3
FSoD-Net [123]	71.8	88.9	66.9	86.8	90.2	45.5	79.6	48.2	86.9	75.5	67.0
ASDN [124]	66.9	63.9	73.8	71.8	81	46.3	73.4	56.3	73.4	66.2	74.7
MSFC-Net [125]	70.1	85.8	76.2	74.4	90.1	44.2	78.1	55.5	60.9	59.5	76.9
Xue et al. [126]	80.5	95.2	84.2	94.8	85.2	54.0	90.5	71.0	75.3	70.7	82.0
DFPN-YOLO [127]	69.33	80.2	76.8	72.7	89.1	43.4	76.9	72.3	59.8	56.4	74.3
AC-YOLO [128]	77.1	93.1	80.9	79.9	84.4	76.0	81.7	77.1	67.6	70.0	66.7
SCRDet++ [129]	75.1	71.9	85.0	79.5	88.9	52.3	79.1	77.6	89.5	77.8	84.2
MSSDet [130]	76.9	70.7	88.6	81.8	90.4	56.5	82.5	73.0	90.1	78.6	86.6
Gao et al. [131]	72.5	78.1	83.9	73.0	89.0	48.2	79.4	65.6	63.9	61.9	80.6
MDCT [132]	80.5	92.5	85.0	93.5	84.7	53.7	90.2	74.3	79.9	68.2	68.6
YOLOv5s	80.4	87.2	86.9	86.2	92.3	55.5	83.0	72.6	91.1	83.0	81.6
Ours	81.6	87.9	91.1	84.9	91.7	55.8	80.7	78.9	92.8	82.6	86.6
Method	mAP	GTF	HB	O	S	SD	ST	TC	TS	V	W
HawkNet [119]	72.0	74.7	50.2	59.6	89.7	66.0	70.8	87.2	61.4	52.8	88.2
CANet [120]	74.3	83.6	56.0	63.6	81.0	79.8	70.8	88.2	67.6	51.2	89.6
Yao et al. [121]	75.8	82.8	57.4	65.8	80.0	92.5	81.1	88.7	63.0	73.0	78.1
MFPNet [122]	71.2	77.2	62.1	58.8	77.2	76.8	60.3	86.4	64.5	41.5	80.2
FSoD-Net [123]	71.8	77.3	53.6	59.7	78.3	69.9	75.0	91.4	52.3	52.0	90.6
ASDN [124]	66.9	75.2	51.1	58.4	76.2	67.4	60.2	81.4	58.7	45.8	83.1
MSFC-Net [125]	70.1	73.7	49.6	57.2	89.6	69.2	76.5	86.7	51.8	55.2	84.3
Xue et al. [126]	80.5	82.1	70.6	67.3	95.0	94.3	83.8	91.6	61.2	79.8	81.8
DFPN-YOLO [127]	69.33	71.6	63.1	58.7	81.5	40.1	74.2	85.8	73.6	49.7	86.5
AC-YOLO [128]	77.1	75.7	75.5	76.7	87.0	65.8	70.1	88.7	63.5	81.2	80.5
SCRDet++ [129]	75.1	83.1	64.2	65.6	71.3	76.5	64.5	88.0	70.9	47.1	85.1
MSSDet [130]	76.9	85.6	63.5	66.5	82.5	82.0	63.3	88.7	71.7	46.7	89.2
Gao et al. [131]	72.5	76.6	63.5	61.6	89.6	68.7	76.4	87.0	66.4	57.0	78.7
MDCT [132]	80.5	92.9	68.4	83.8	92.9	77.4	83.0	92.8	64.7	77.4	83.0
YOLOv5s	80.4	86.4	66.5	67.3	91.8	81.0	80.4	93.2	69.7	60.3	92.2
Ours	81.6	86.4	68.7	67.3	91.7	81.5	80.3	93.0	77.1	60.9	91.4

Note: In the table, 20 categories are divided into 2 rows, with 10 detection results for each row. The boldface values represent the maximum value in the column.

, with bold values indicating the optimal results in each column.

From the experimental results in

Table 6. Object detection results on PASCAL VOC2007 test dataset.

Method	Backbone	[email protected] (%)	FPS	GPU
Faster R-CNN [27]	VGGNet	73.2	7	Titan X
SSD 300 [35]	VGGNet	74.1	46	Titan X
ASSD 300 [133]	VGGNet	79.1	39.6	GTX 1080Ti
MFFAMM 300 [134]	VGG16	80.7	26	-
Zhe et al. 300 [135]	VGG16	80.1	42.2	RTX 2080Ti
FESSD 300 [136]	ResNet-50	82.2	41.3	RTX 3090
YOLOv3 320 [37]	Darknet53	74.5	45.5	Titan X
GC-YOLOv3 320 [137]	Darknet53	81.3	39	GTX 1080Ti
DSP-YOLO 416 [138]	Darknet53	82.2	56	Titan Xp
Zhang et al. 416 [139]	MobileNetv2	81.67	44.18	RTX 2080Ti
He et al. 416 [140]	ECA-CSPNet	78.6	94	RTX 2080Ti
SSD 512 [35]	VGGNet	76.8	19.0	Titan X
ASSD 512 [133]	VGGNet	81.0	20.8	GTX 1080Ti
PDS-Net 512 [141]	CSPDarknet-53	84.9	32.2	RTX 2070
SLMS-SSD 512 [142]	VGG16	81.2	17.4	RTX 2080Ti
YOLOv3 544 [37]	Darknet53	78.6	40	Titan X
GC-YOLOv3 544 [137]	Darknet53	83.7	31	GTX 1080Ti
RON384++ [143]	VGG16	77.6	-	Titan X
STDN 513 [144]	DenseNet169	80.9	28.6	Titan Xp
Zhong et al. [145]	BottleneckCSP	84.3	85.2	RTX 2080Ti
YOLO-T 640 [146]	CSPDarknet-53	85.2	65.7	RTX 3090
BFBG-YOLO [147]	CSPDarknet-53	80.3	99.0	RTX 3090
SL-YOLO [148]	ShuffleNet v2	81.2	17.8	Tesla P40
YOLOv5s 640	CSPDarknet-53	83.7	143	RTX A5000
Ours 640	CSPDarknet-53	85.1	90.2	RTX A5000

The boldface values represent the maximum value in the column.

, it can be seen that compared with HawkNet [119], CANet [120], MFPNet [122], FSoD-Net [123], ASDN [124], MSFC-Net [125], DFPN-YOLO [127], AC-YOLO [128], SCRNet++ [129], MSSNet [130], MDCT [132], and YOLOv5s, the improved algorithm proposed in this paper based on YOLOv5s significantly improves detection accuracy. On the DIOR dataset, the mean average precision accuracy of the improved YOLOv5s network is 1.2% higher than that of the original YOLOv5s network, indicating that the network constructed in this article not only outperforms the original YOLOv5s network on the RSOD dataset but also achieves better performance than the original network on the more complex large-scale remote sensing dataset, DIOR.

In order to further visually demonstrate the effectiveness of the constructed models, the detection results of each model were shown in Figure 11.

As shown in Figure 11, from rows 1, 2, 5 and 6, it could be seen that the detection boxes of the model constructed in this article were closer to the actual detection boxes of targets, with a higher recall rate. From rows 1, 3, 4, and 6, it can be seen that the model constructed in this article had a lower false alarm rate. Therefore, the effectiveness of the model constructed in this article is superior to the original model.

5.3.6. Performance Comparison in the PASCAL VOC Dataset

To further validate and evaluate the effectiveness of the proposed algorithm to YOLOv5s in improving detection accuracy, the proposed algorithm was compared with several advanced object detection algorithms that have emerged in recent years. The training set and test set used in the experiments were consistent with those used in the study. The experimental results are shown in Table 6, where the bold values indicate the best results in each column.

From Table 6, it can be seen that the algorithm proposed in this article meets the real-time requirements, with a mean average precision accuracy of 85.1%, which is 1.4% higher than the original YOLOv5s algorithm. Compared to one-stage target detection algorithms, such as the SSD series algorithm and its improved algorithm, it has advantages in terms of precision and detection speed. Compared to two-stage target detection algorithms, such as the Fast R-CNN algorithm, it has a much higher detection accuracy and detection rate.

To further verify whether the algorithm can effectively improve the accuracy of small target detection,

Table 7. PASCAL VOC2007 test detection results.

Method	mAP	Aero	Bike	Bird	Boat	Bottle	Bus	Car	Cat	Chair	Cow
Faster R-CNN [27]	73.2	76.5	79	70.9	65.5	52.1	83.1	84.7	86.4	52	81.9
SSD 300 [35]	74.1	74.6	80.2	72.2	66.2	47.1	82.9	83.4	86.1	54.4	78.5
ASSD 300 [133]	79.1	85.4	84.1	78.7	71.8	54.0	86.2	85.3	89.5	60.4	87.4
FESSD 300 [136]	82.2	89.4	86.2	84.3	78.2	57.8	91.6	91.5	91.7	62.2	90.4
Zhe et al. 300 [135]	80.1	84.6	87.6	80.1	73.0	50.4	89.3	88.3	90.9	60.2	87.8
He et al. 416 [140]	78.6	86.1	86.2	76.5	66.5	66.4	86.6	91.3	80.7	64.3	84.4
Zhang et al. 416 [139]	81.6	88.5	87.5	83.1	75.2	67.1	85.3	90.2	88.9	60.9	89.7
DSP-YOLO 416 [138]	82.2	88.5	89.5	79.1	74.0	68.7	89.7	90.6	89.9	66.7	84.4
SSD 512 [35]	76.8	82.4	84.7	78.4	73.8	53.2	86.2	87.5	86.0	57.8	83.1
ASSD 512 [133]	81.0	86.8	85.2	84.1	75.2	60.5	88.3	88.4	89.3	63.5	87.6
PDS-Net 512 [141]	84.2	93.3	98.0	80.2	73.8	70.2	90.9	96.3	87.1	65.0	87.3
SLMS-SSD 512 [142]	81.2	88.5	87.1	83.2	76.4	59.2	88.3	88.4	89.0	66.6	86.9
STDN513 [144]	80.9	86.1	89.3	79.5	74.3	61.9	88.5	88.3	89.4	67.4	86.5
DSP-YOLO 608 [138]	83.1	91.0	90.7	81.8	75.6	73.8	91.3	92.7	91.2	66.9	86.9
RON384++ [143]	77.6	86. 0	82.5	76.9	69.1	59.2	86.2	85.5	87.2	59.9	81.4
SFGNet [149]	81.2	82.2	83.9	80.3	71.5	78.2	89.6	86.9	90.0	65.7	87.9
SL-YOLO [148]	81.2	86.4	85.7	77.9	75.5	72.5	85.4	87.8	86.2	85.9	72.1
YOLOv5s 640	83.7	91.6	91.9	81.1	75.0	78.5	91.2	92.9	87.3	67.4	88.0
Ours 640	85.1	92.2	92.0	82.9	74.4	78.1	92.6	93.7	91.1	68.8	89.3
Method	mAP	Table	Dog	Horse	Mbike	Person	Plant	Sheep	Sofa	Train	Tv
Faster R-CNN [27]	73.2	65.7	84.8	84.6	77.5	76.7	38.8	73.6	73.9	83	72.6
SSD 300 [35]	74.1	73.9	84.4	84.5	82.4	76.1	48.6	74.3	75.0	84.3	74.0
ASSD 300 [133]	79.1	77.1	87.4	86.8	84.8	79.5	57.8	81.5	80.1	87.4	76.9
FESSD 300 [136]	82.2	74.4	89.4	90.5	87.7	83.7	52.4	88.6	81.6	91.0	81.7
Zhe et al. 300 [135]	80.1	81.4	87.1	89.1	87.9	82.1	54.6	80.4	80.5	89.2	78.1
He et al. 416 [140]	78.6	73.7	77.6	85.3	85.9	86.1	52.4	80.8	75.5	84.5	80.7
Zhang et al. 416 [139]	81.6	78.4	89.5	89.5	84.9	84.8	55.1	86.9	74.3	90.8	82.0
DSP-YOLO 416 [138]	82.2	75.0	89.2	89.3	89.8	85.8	56.6	84.4	81.1	89.1	81.6
SSD 512 [35]	76.8	70.2	84.9	85.2	83.9	79.7	50.3	77.9	73.9	82.5	75.3
ASSD 512 [133]	81.0	76.6	88.2	86.7	85.7	82.8	59.2	83.6	80.5	87.5	80.8
PDS-Net 512 [141]	84.2	74.5	86.5	91.8	91.9	89.7	59.9	92.8	79.3	89.1	84.6
SLMS-SSD 512 [142]	81.2	74.6	87.3	88.6	86.5	82.2	54.8	85.5	80.9	87.9	81.0
STDN513 [144]	80.9	79.5	86.4	89.2	88.5	79.3	53.0	77.9	81.4	86.6	85.5
DSP-YOLO 608 [138]	83.1	75.5	89.0	90.4	88.6	87.3	55.1	87.3	80.0	86.9	80.0
RON384++ [143]	77.6	73.3	85.9	86.8	82.8	79.6	52.4	78.2	76.0	86.2	78.0
SFGNet [149]	81.2	72.4	90.3	89.9	83.5	82.5	67.8	79.0	81.6	86.7	75.7
SL-YOLO [148]	81.2	79.5	78.8	88.2	86.5	81.1	71.2	84.4	79.2	82.7	76.3
YOLOv5s 640	83.7	79.1	86.0	91.4	89.3	89.5	60.5	86.1	76.0	86.8	84.9
Ours 640	85.1	76.7	88.1	92.6	92.2	90.0	61.2	89.4	79.0	90.6	84.7

Note: In the table, 20 categories are divided into 2 rows, with 10 detection results for each row. The boldface values represent the maximum value in the column.

compares the accuracy of the improved YOLOv5s and other advanced target detection algorithms in the 20 categories in the PASCAL VOC2007 test set. The results show that compared to the original YOLOv5s algorithm, the improved YOLOv5s algorithm improves the detection accuracy of the model in almost every category, especially for small target categories such as birds.

5.3.7. Performance Comparison in the MS COCO Dataset

In order to further demonstrate the advantages of this method in detecting small and dense targets, an experimental comparison between this method and other methods in the literature was conducted on the MS COCO test dataset. From the experimental results in

Table 8. MS COCO test-dev detection results.

Method	AP	AP50	AP75	AP Small	AP Medium	AP Large
R-FCN [150]	29.2	51.5	-	10.3	32.4	43.3
SSD 300 [35]	25.1	43.1	25.8	6.6	25.9	41.4
FESSD 300 [136]	28.3	-	29.6	-	-	-
Zhe et al. 300 [135]	29.9	49.9	31.3	10.6	24.5	47.6
YOLOv3 416 [37]	31.0	55.3	32.3	15.2	33.2	42.8
GC-YOLOv3 416 [137]	-	55.5		-	-	-
Mini-YOLOv4-tiny 416 [151]	23.4	42.2	23.4	-	-	-
TRC-YOLO 416 [152]	18.4	38.4	15.6	6.3	17.6	27.2
Trident-YOLO 416 [153]	18.8	37.0	17.3	20.9	25.1	29.3
He et al. 416 [140]	23.6	43.8	26.8	8.4	27.1	42.3
SSD 512 [35]	28.8	48.5	30.3	10.9	31.8	43.5
SLMS-SSD 512 [142]	30.8	52.4	32.0	16.1	33.7	44.0
BANet_S 640 [154]	40.2	58.6		23.5	44.6	53.2
STDN 513 [144]	31.8	51.0	33.6	14.4	36.1	43.4
YOLOv3 608 [37]	33.0	57.9	34.4	18.3	35.4	41.9
SFGNet [149]	32.3	54.1	-	-	-	-
Zhong et al. [145]	38.4	56.2	42.1	21.6	43.0	52.4
YOLO-T 640 [146]	42.0	58.3	44.1	-	-	-
SL-YOLO [148]	36.8	51.3	37.3	11.7	37.7	48.4
YOLOv5s 640	37.4	56.8	40.7	21.2	42.3	49.0
Ours 640	40.8	59.9	43.7	21.4	44.2	52.9

The boldface values represent the maximum value in the column.

, it can be seen that compared with R-FCN [150], SSD [35], FESSD [136], YOLOv3 [37], GC-YOLOv3 [137], Mini-YOLOv4-tiny [151], TRC-YOLO [152], Trident-YOLO [153], SLMS-SSD [142], BANet_S [154], STDN [144], SFGNet [149], YOLO-T [146], SL-YOLO [148], and YOLOv5s, the improved algorithm proposed in this article based on YOLOv5s significantly improved detection accuracy. Compared to the original YOLOv5s algorithm, the overall detection accuracy of the algorithm proposed in this article was improved by 3.1%, and the detection accuracy of small, medium, and large targets was improved by 0.2%, 1.9%, and 3.9%, respectively. Experimental results show that the proposed algorithm outperforms the original YOLOv5s algorithm in small target detection, medium target detection, and large target detection.

6. Discussion

In this section, the contribution of the constructed module to the proposed network was explored through ablation experiments.

Ablation experiments were conducted on the RSOD dataset to study the effects of CARAFE, SA attention, SPPCSPG, GSConv, and DD-head modules on both model accuracy and detection speed. These models were trained on the RSOD dataset and tested on an RTXA5000 GPU. The input size of the test images for the ablation experiment was 640 × 640, and the experimental results are presented in

Table 9. Ablation experiment on RSOD dataset.

Model	Model 1	Model 2	Model 3	Model 4	Model 5	Model 6	Model 7	Model 8	Model 9	Model 10
CARAFE		√					√	√	√	√
SA			√				√	√	√	√
SPPCSPG				√				√	√	√
GSConv					√				√	√
DD-head						√				√
Params(M)	7.02 M	7.15 M	7.02 M	9.95 M	6.35 M	7.28 M	7.15 M	10.09 M	9.41 M	9.67 M
FLOPs(G)	15.8	16.3	15.8	18.1	14.6	16.7	16.3	18.6	17.5	18.4
[email protected] (%)	0.950	0.952	0.952	0.955	0.953	0.955	0.955	0.958	0.961	0.964
[email protected]:0.95 (%)	0.653	0.659	0.652	0.676	0.649	0.674	0.671	0.672	0.669	0.648
FPS	119.0	110.9	113.9	113.1	107.8	106.8	109.9	96.2	91.2	88.8

.

In Table 9, CARAFE represents the replacement of the nearest-neighbor interpolation up-sampling module in the original YOLOv5s network with the CARAFE module. SA represents the embedding of the SA attention module in the head structure of the YOLOv5s network. SPPCSPG represents the replacement of the SPPF module in the original YOLOv5s network with the SPPCSPG module. GSConv represents the replacement of the Conv module in the neck structure of the YOLOv5s network with the GSConv module and the replacement of the C3 module with the improved VoV-GSCSP module. DD-head represents the replacement of the coupled head module in the original YOLOv5s network with the DD-head module. The presence or absence of a checkmark (√) indicates whether the proposed improvement module was incorporated into the YOLOv5s network.

Model 1 is the original YOLOv5s network, while models 2–10 are the corresponding improved YOLOv5s networks. Analysis of the results in Table 2 shows that embedding the CARAFE, SA attention, SPPCSPG, GSConv, and DD-head modules separately into the original YOLOv5s network can improve the detection accuracy of the network. The detection performance measured by [email protected] is improved by 0.2%, 0.2%, 0.5%, 0.5%, and 0.3%, respectively, compared with the original YOLOv5s network. Analysis of models 7–10 reveals that the combination of multiple improved modules performs better than individual improved modules, indicating that each introduced module contributes to the effective improvement of the model’s detection performance.

In order to verify the improvement effect of the proposed model on the accuracy of general object detection, ablation experiments were conducted on the PASCAL VOC dataset, and the experimental results are shown in

Table 10. Ablation experiment on PASCAL VOC dataset.

Model	Model 1	Model 2	Model 3	Model 4	Model 5	Model 6	Model 7	Model 8	Model 9	Model 10
CARAFE		√					√	√	√	√
SA			√				√	√	√	√
SPPCSPG				√				√	√	√
GSConv					√				√	√
DD-head						√				√
Params(M)	7.06 M	7.20 M	7.06 M	10.00 M	6.39 M	7.32 M	7.20 M	10.13 M	9.46 M	9.71 M
FLOPs(G)	15.9	16.4	15.9	18.3	14.8	16.9	16.4	18.8	17.6	18.6
[email protected] (%)	0.837	0.840	0.839	0.847	0.841	0.842	0.841	0.848	0.850	0.851
[email protected]:0.95 (%)	0.585	0.595	0.588	0.606	0.603	0.596	0.595	0.615	0.619	0.619
FPS	143.0	125.4	141.9	122.5	125.2	128.6	120.8	110.8	98.4	90.2

.

Similarly, analyzing the results of models 2–6 in Table 10, it can be seen that embedding the CARAFE module, SA attention module, SPPCSPG module, GSConv module, and DD-head module separately in the original YOLOv5s network can improve the mean average precision accuracy of the network to levels higher than the detection indicators of the original YOLOv5s network; [email protected] increased by 0.3%, 0.2%, 1.0%, 0.4%, and 0.5% respectively. Analyzing models 7–10, it is apparent that the results of multiple improved module combinations are better than those of a single improved module, indicating that the introduced improved models have effectively improved the detection performance of the model.

7. Conclusions

This paper proposes a lightweight target detection algorithm based on YOLOv5s to improve the detection performance of the model while meeting the real-time detection requirements. Specifically, this article constructs a DD-head to replace the coupled head of YOLOv5s based on a decoupled head and depthwise convolution to improve the negative impact of classification and regression task conflicts. An SPPCSPG module based on the SPPCSPC module and GSConv module is constructed to replace the SPPF module of YOLOv5s, which improves the utilization of multi-scale information. An SA attention mechanism is introduced in the head structure to enhance spatial attention and reconstruct channel attention. A CARAFE module is introduced in the up-sampling operation to reassemble feature points with similar semantic information in a content-aware manner and aggregate features in a larger receptive field to fully fuse semantic information. In the neck structure, the GSConv module and the reconstructed VoV-GSCSP module are introduced to maintain detection accuracy while reducing the number of parameters. The experiments show that the constructed algorithm performs better than the original network not only for remote sensing images but also for conventional object detection images. The model built in this research is for target detection in remote sensing datasets, and the equipment used is a high-performance GPU. The detection performance of the model built on resource-constrained edge computing devices has not been tested. The next research work will focus on how to build a real-time detection model for edge computing devices.