Enhancing Object Detection with Shape-IoU and Scale–Space–Task Collaborative Lightweight Path Aggregation

Abstract

We propose a novel target detection algorithm that addresses the issues of ignoring shape attributes in regression loss and the inability of the high-parameter PAFPN to jointly perceive scale–space–task information. Specifically, we construct a Lightweight Path Aggregation Feature Pyramid Network (LPAFPN) to reduce model parameters by shuffling and fusing features across channels. To further enhance its perception ability, we augment LPAFPN with a scale–space–task joint-perception enhancement module, terming the resulting network ALPAFPN, which can adaptively process joint information of scale, space, and task. Finally, we introduce a shape-scale bounding box regression loss method that focuses on the target’s intrinsic attributes to optimize the regression measurement, thereby boosting the detection accuracy. Experimental results show that the proposed algorithm outperforms state-of-the-art algorithms in terms of F1 score, Precision, and Mean Average Precision (mAP) on the PASCAL VOC and VisDrone2019-DET datasets.

1. Introduction

As one of the downstream tasks of computer vision, object detection aims to localize and recognize objects of interest from images or videos. It is now widely used in various fields, such as autonomous driving [1], robotic vision [2], and video surveillance [3].

Traditional object detection methods [4] typically employ manually designed sliding windows and feature extractors to predict object categories and locations. However, these methods may fail to retain accuracy and robustness with diverse objects and complex backgrounds [5]. Moreover, in real-world applications, factors like sensor quality and environmental conditions directly influence the accuracy and reliability of the acquired data, thereby significantly impacting the performance of detection models evaluated on the dataset [6].

In comparison with traditional methods, deep learning-based object detection methods demonstrate superior robustness by extracting deeper semantic features. Existing deep learning-based object detection methods [7] can be broadly categorized into two types: Transformer-based [8] models and CNN-based models [9]. Both paradigms effectively support parallel computation, thereby overcoming the inherent limitations of recurrent neural network (RNN) [10] models in parallel processing. In most cases, Transformer-based methods demonstrate superior performance in capturing global contextual information [11,12], whereas CNN-based object detection approaches exhibit lower computational complexity, rendering them more suitable for real-time applications with stringent latency requirements. CNN-based models are primarily divided into two-stage and one-stage paradigms. The former is typically represented by algorithms such as Faster R-CNN [13] and Mask R-CNN [14], while the latter includes frameworks like SSD [15] and RetinaNet [16]. In these classical approaches, there often exists spatial misalignment in predictions between classification and localization tasks. To overcome these shortcomings, the TOOD algorithm [17] designs a Task-Aligned Head to enhance the interaction of two tasks, thereby optimizing the spatial distribution of features learned for object detection. TOOD and YOLOv8 [18] adopt FPN [19] and PAFPN [20] as their feature fusion networks, respectively. Compared with FPN, PAFPN incorporates a bottom–up path augmentation to enhance multiscale feature fusion capabilities. However, PAFPN struggles to synergistically integrate scale, spatial, and task-specific information, while its reliance on standard convolutions may increase parametric complexity.

To address the limitations of PAFPN and the neglect of target shape attributes in existing regression losses, we propose a novel object detection algorithm via Shape-IoU Loss and Lightweight Path Aggregation Feature Pyramid Network featuring scale–space–task collaborative enhancement. The approach not only designs a Lightweight Path Aggregation Feature Pyramid Network with scale–space–task collaborative enhancement, but also introduces a shape-scale bounding box regression loss method, thereby improving detection accuracy while reducing model parameters. The principal contributions of this work are summarized as follows:

(1)

We construct Lightweight Path Aggregation Feature Pyramid Network (LPAFPN), which integrates and shuffles information from standard convolutions and depth-wise separable convolutions to reduce Parameter Count.

(2)

We design Lightweight Path Aggregation Feature Pyramid Network with scale–space–task collaborative enhancement (ALPAFPN), which enhances the perceptual capabilities of the model by synergistically processing features from distinct feature layers, spatial locations, and task-specific channels.

(3)

We introduce a shape-scale bounding box regression loss method that incorporates target shape attributes to optimize the regression loss function, thereby improving the detection accuracy of the model.

(4)

We conduct extensive experiments on the Pascal VOC dataset and VisDrone2019-DET dataset, and the experimental results reveal that the F1 score, Precision, and Mean Average Precision of LSCA are superior to those of the state-of-the-art methods.

2. Related Work

2.1. CNN-Based Methods

Existing CNN-based methods can be broadly classified into two main types: two-stage methods and one-stage methods [21].

The two-stage object detection approaches extract features of interest from these bounding boxes to detect objects after generating candidate bounding boxes. Faster R-CNN enhances detection speed by introducing the Region Proposal Network (RPN) [22]. Dynamic R-CNN [23] proposes a dynamic IoU threshold adjustment mechanism that adapts to the demand for high-quality samples during training by progressively optimizing the threshold. Sparse R-CNN [24] employs a fixed number of learnable sparse proposal boxes to perform classification and localization through a dynamic instance interactive head.

One-stage object detection methods directly predict the location and category of objects from images without generating candidate boxes. SSD introduces multireference and multiresolution detection techniques, thereby enhancing detection accuracy. YOLOv7 [25] achieves significant performance improvements by employing strategies such as dynamic label assignment and model structure reparameterization. YOLOv8 reduces the prediction of redundant bounding boxes by improving the anchor-free detection head, which achieves a balance between accuracy and speed in object detection tasks. YOLOv9 [26] designs the GELAN architecture to reduce computational complexity by optimizing model parameters. YOLOv10 [27] employs an NMS-free dual-label assignment strategy during training that maintains computational efficiency while capturing key detection features.

2.2. Path Aggregation Feature Pyramid Network

The Path Aggregation Feature Pyramid Network integrates multiscale features through top–down and bottom–up path aggregation. However, the PAFPN exhibits a relatively weaker capability in collaboratively processing scale, space, and task information. Additionally, the extensive number of input and output channels in the 3 × 3 convolutional layers of this architecture may lead to a substantial increase in the number of parameters. In response to the aforementioned issues, LSCA has developed a Lightweight Path Aggregation Feature Pyramid Network with scale–space–task collaborative enhancement. This innovation not only reduces the Parameter Count of the model but also significantly improves its perceptual capabilities related to scale, space, and task information.

2.3. Loss Function

IoU Loss [28] describes the degree of match between the prediction box and the GT box. However, it encounters limitations when the IoU is zero, as it fails to differentiate the positional relationship between the two boxes due to the vanishing gradient problem. To mitigate this issue, Generalized IoU (GIoU) Loss [29] introduces the minimum enclosing box, which alleviates the vanishing gradient problem to some extent when IoU equals zero. Center IoU (CIoU) Loss [30] enhances the accuracy of object detection by incorporating constraints related to the normalized distance between the centroid of the prediction box and the GT box. In comparison to the aforementioned loss functions, the Scale-Invariant IoU (SIoU) Loss [31] provides a more precise measurement of the matching degree between the predicted and ground truth boxes by integrating angle-aware geometric constraints. While these loss functions primarily focus on the geometric relationship between the predicted and ground truth boxes, they fail to explicitly account for the actual shape of the target, thus neglecting the influence of inherent attributes of the target on bounding box regression. The Shape-IoU bounding box regression loss method is introduced to solve the above problems. This method takes into account the impact of geometric attributes such as the shape of the target on the positioning accuracy, so as to optimize the regression loss metric.

3. Method

3.1. Lightweight Path Aggregation Feature Pyramid Network

The Path Aggregation Feature Pyramid Network (PAFPN) partially enhances the expression ability of the model by integrating multiscale features through top–down and bottom–up paths. The 3 × 3 convolution in PAFPN maximally preserves hidden connections between channels, but this increases the number of parameters in the model. To address this problem, GSConv is introduced into the proposed model to construct LPAFPN, as shown in Figure 1 Through the shuffle operation, GSConv fuses the feature information generated by the 1 × 1 standard convolution and the depth-wise separable convolution, which reduces the Parameter Count while preserving the detection accuracy.

Figure 1. Network Structure of LPAFPN.

Let $F_{\mathrm{in}}\in R^{\mathrm{h}\times w\times {\mathrm{C}}_1}$ and $F_{\mathrm{out}}\in R^{h\times w\times C_2}$ denote the input and output feature maps of GSConv, respectively. As can be seen from Equation (1), the feature maps $X\in R^{\mathrm{h}\times w\times {\mathrm{C}}_2}$ are obtained by applying standard convolution, depth-wise separable convolution, and concatenation operations to $F_{\mathrm{in}}$,

$$X=Concat({Conv}_{(1,1)}(F_{\mathrm{in}}),{DSC}_{(5,1)}({Conv}_{(1,1)}(F_{\mathrm{in}}))),$$

(1)

where ${Conv}_{(1,1)}(•)$ is a 1 × 1 convolution with a stride of 1, and ${DSC}_{(5,1)}(•)$ is a 5 × 5 depth-wise separable convolution using a stride of 1.

Let $X=(X_1,\dots ,X_{\mathrm{g}},\dots ,X_{\mathrm{G}})$, $X_{\mathrm{g}}=\left(\begin{array}{ccc}{x}_{1,[(g-1)(C_2//G)+1]}& \cdots & x_{1,g(C_2//G)}\\ ⋮& \ddots & ⋮\\ x_{\mathrm{w},[(g-1)(C_2//G)+1]}& \cdots & x_{w,g(C_2//G)}\end{array}\right)$, $X_{\mathrm{g}}\in R^{h\times w\times (C_2//G)}$, $x_{i,[(g-1)(C_2//G)+j]}\in R^{\mathrm{h}}$, $i\in \left\{1,2,\dots ,\mathrm{w}\right\}$, $j\in \left\{1,2,\dots ,({\mathrm{C}}_2//\mathrm{G})\right\}$, then $F_{\mathrm{out}}$ can be formalized as Equation (2):

$$F_{\mathrm{out}}=(X_1^{\mathrm{T}},\dots ,X_{\mathrm{g}}^{\mathrm{T}},\dots ,X_{\mathrm{G}}^{\mathrm{T}})$$

(2)

The number of model parameters is reduced without sacrificing detection accuracy, since the depth-wise separable convolution reduces model complexity, while the shuffle operation fuses cross-channel feature information in GSConv.

3.2. Scale–Space–Task Co-Enhanced Lightweight Path Aggregation Feature Pyramid Network

GSConv serves as a basic building block in LPAFPN, where the shuffle operation is introduced to integrate the features extracted by the depth-wise separable convolution and the standard convolution. To solve the problem that LPAFPN may neglect the diversity of feature scales, spatial locations, and channel tasks, we construct ALPAFPN by introducing the scale–space–task attention mechanism (SSTA), which jointly processes scale, space and task information through cascading scale-aware, spatial-aware, and task-aware attention mechanisms. The scale-aware attention mechanism focuses on the scale size of the target, facilitating the learning of the feature importance from different layers, and enhancing the features of the layer of interest. The spatial-aware attention mechanism captures the spatial location information within the same layer and across different layers to extract more discriminative spatial features. The task-aware attention mechanism is designed to adaptively process feature data on different channels, which enables the achievement of objectives in multiple detection tasks.

3.2.1. Scale-Aware Attention Mechanism

Set $m=\left[{\displaystyle \frac{L}{2}}\right]$. Let ${\left\{F_i\in R^{H_i\times W_i\times C_i}\right\}}_{i=1}^L$ and $G\in R^{H_{\mathrm{m}}\times W_m\times \left(L\times C_m\right)}$ be the input and output feature maps of the scale-aware attention mechanism, respectively. As illustrated in Equations (3) and (4), $F_i^{\prime }\in R^{H_m\times W_m\times C_m}$, which is obtained by resizing $F_i$, is further processed via pooling, convolution, and activation operations to acquire the weight vector $W_i\in R^{C_m}$. If $F_i^{\prime }=\left(\begin{array}{c}{F}_{i1}^{\prime }\\ ⋮\\ F_{i\mathrm{c}}^{\prime }\\ ⋮\\ F_{i{C}_{\mathrm{m}}}^{\prime }\end{array}\right)$, $W_i=\left(\begin{array}{c}{W}_{i1}\\ ⋮\\ W_{i\mathrm{c}}\\ ⋮\\ W_{i{C}_{\mathrm{m}}}\end{array}\right)$, $F_{i\mathrm{c}}^{\prime }\in R^{H_m\times W_m}$, then $G$ can be attained from Equations (5) and (6):

$$\left\{\begin{array}{c}{F}_i^{\prime }=\mathrm{Downsample}(F_i),i\in \{1,2,\dots ,m-1\}\\ F_m^{\prime }=F_m\\ F_i^{\prime }=\mathrm{Upsample}(F_i),i\in \{m+1,\dots ,L\}\end{array}\right.$$

(3)

$$W_i=\text{Hard\_sigmoid}(\mathrm{Relu}({\mathrm{Conv}}_{(1,1)}(\text{avg\_pool}(F_i^{\prime }))))$$

(4)

$$G_i=F_i^{\prime }\odot W_i=\left(\begin{array}{c}{F}_{i1}^{\prime }\\ ⋮\\ F_{i\mathrm{c}}^{\prime }\\ ⋮\\ F_{i{C}_m}^{\prime }\end{array}\right)\odot \left(\begin{array}{c}{W}_{i1}\\ ⋮\\ W_{i\mathrm{c}}\\ ⋮\\ W_{i{C}_m}\end{array}\right)\in R^{H_m\times W_m\times C_m}$$

(5)

$$G=\mathrm{Concat}(G_1,\dots ,G_l,\dots ,G_L)$$

(6)

where $\mathit{\text{Hard\_sigmoid}}(x)=\max (0,\min (1,{\displaystyle \frac{x+1}{2}}))$.

3.2.2. Spatial-Aware Attention Mechanism

Let $\left\{G(c)\in R^{L\times H\times W},\mathrm{c}\in \left\{1,\dots ,C\right\}\right\}$ and $\left\{Q(c)\in R^{L\times H\times W},\mathrm{c}\in \left\{1,\dots ,C\right\}\right\}$ be the input and output feature maps of the spatial-aware attention mechanism, respectively. Set $W=\left(\begin{array}{ccccc}{W}_{11}& \cdots & W_{1h}& \cdots & W_{1H}\\ ⋮& \ddots & ⋮& \ddots & ⋮\\ W_{l1}& \cdots & W_{lh}& \cdots & W_{lH}\\ ⋮& \ddots & ⋮& \ddots & ⋮\\ W_{L1}& \cdots & W_{Lh}& \cdots & W_{LH}\end{array}\right)\in R^{L\times H}$, $\Delta m=(\Delta m_1,\dots ,\Delta m_h,\dots ,\Delta m_H)\in R^H$. Suppose that the weight matrix $W$ is divided into blocks according to Equation (7). Then $W_{*}\in R^{L\times H}$ can be acquired from Equation (8). If $G\left(c\right)=\left(\begin{array}{ccccc}{g}_{11}^c& \cdots & g_{1h}^c& \cdots & g_{1H}^c\\ ⋮& \ddots & ⋮& \ddots & ⋮\\ g_{l1}^c& \cdots & g_{lh}^c& ˯& g_{lH}^c\\ ⋮& \ddots & ⋮& \ddots & ⋮\\ g_{L1}^c& \cdots & g_{Lh}^c& \cdots & g_{LH}^c\end{array}\right)$, $W_{*}=\left(\begin{array}{ccccc}{W}_{11}^{\prime }& \cdots & W_{1h}^{\prime }& \cdots & W_{1H}^{\prime }\\ ⋮& \ddots & ⋮& \ddots & ⋮\\ W_{l1}^{\prime }& \cdots & W_{lh}^{\prime }& \cdots & W_{lH}^{\prime }\\ ⋮& \ddots & ⋮& \ddots & ⋮\\ W_{L1}^{\prime }& \cdots & W_{Lh}^{\prime }& \cdots & W_{LH}^{\prime }\end{array}\right)$, then $g\prime \left(c\right)\in R^{L\times H\times W}$ can be obtained by the dot product of $G\left(c\right)$ and $W_{*}$, as demonstrated by Equation (9). Let $g^{\prime }\left(c\right)={\left(q_{lh}^c\right)}_{L\times H}$, $q_{lh}^c\in R^W$, then $W_{*}^{\prime }\left(\mathrm{c}\right)\in R^W$ can be attained from Equation (10). Assume that $W_{*}^{\prime }\left(\mathrm{c}\right)$ and $G(c)$ are divided into blocks as derived from Equations (11) and (12), then $Q(c)$ can be acquired from Equation (13).

$$W=\left(W_1\hspace{1em}\cdots \hspace{1em}{W}_h\hspace{1em}\cdots \hspace{1em}{W}_H\right),W_h=\left(\begin{array}{c}{W}_{1h}\\ ⋮\\ W_{lh}\\ ⋮\\ W_{Lh}\end{array}\right)\in R^L$$

(7)

$$W_{*}=(W_1,\dots ,W_{\mathrm{h}},\dots ,W_H)\odot (\Delta {\mathrm{m}}_1,\dots ,\Delta {\mathrm{m}}_h,\dots ,\Delta {\mathrm{m}}_H)$$

(8)

$$g^{\prime }\left(c\right)=G\left(c\right)\odot W_{*}$$

(9)

$$W_{*}^{\prime }(\mathrm{c})={\displaystyle \frac{1}{L}}{\displaystyle \sum _{l=1}^L{\displaystyle \sum _{h=1}^H{q}_{lh}^c}}$$

(10)

$$W_{*}^{\prime }(\mathrm{c})=\left(\begin{array}{c}{\xi }_1^c\\ ⋮\\ {\xi }_w^c\\ ⋮\\ {\xi }_W^c\end{array}\right)$$

(11)

$$G\left(c\right)=\left(\begin{array}{c}{G}_1^{\mathrm{c}}\\ ⋮\\ G_w^{\mathrm{c}}\\ ⋮\\ G_W^{\mathrm{c}}\end{array}\right),G_w^{\mathrm{c}}\in R^{L\times H}$$

(12)

$$Q(\mathrm{c})=W_{*}^{\prime }(\mathrm{c})\odot G(\mathrm{c})=\left(\begin{array}{c}{\xi }_1^c\\ ⋮\\ {\xi }_w^c\\ ⋮\\ {\xi }_W^c\end{array}\right)\odot \left(\begin{array}{c}{G}_1^{\mathrm{c}}\\ ⋮\\ G_w^{\mathrm{c}}\\ ⋮\\ G_W^{\mathrm{c}}\end{array}\right)$$

(13)

3.2.3. Task-Aware Attention Mechanism

Let $Q\in R^{C\times H\times W}$ and $Y\in R^{C\times H\times W}$ be the input and output feature maps of the task-aware attention mechanism, respectively. Set $C\times H\times W=D$. Then $Q$ and $Y$ are vectorized to generate $q=\left(\begin{array}{c}{q}_1\\ ⋮\\ q_e\\ ⋮\\ q_D\end{array}\right)\in R^D$ and $y=\left(\begin{array}{c}{y}_1\\ ⋮\\ y_e\\ ⋮\\ y_D\end{array}\right)\in R^D$, respectively. If $M=\left(\begin{array}{cc}A& b\end{array}\right)\in R^{\left(D\times k\right)\times \left(D+1\right)}$, $A=\left(\begin{array}{cccccc}{a}_1& & & & & \\ & a_2& & & & \\ & & \ddots & & & \\ & & & a_{\mathrm{e}}& & \\ & & & & \ddots & \\ & & & & & a_D\end{array}\right)\in R^{\left(D\times k\right)\times D}$, $a_{\mathrm{e}}=\left(\begin{array}{c}{a}_e^1\\ ⋮\\ a_e^k\end{array}\right)\in R^k$, $b=\left(\begin{array}{c}{b}_1\\ ⋮\\ b_{\mathrm{e}}\\ ⋮\\ b_D\end{array}\right)\in R^{\left(D\times k\right)\times 1}$, $b_{\mathrm{e}}=\left(\begin{array}{c}{b}_e^1\\ ⋮\\ b_e^k\end{array}\right)\in R^{\mathrm{k}}$, then $Y^{\prime }\in R^{\left(D\times k\right)\times 1}$ can be derived from Equation (14). Let $Y^{\prime }=\left(\begin{array}{c}{y}_1^{\prime }\\ ⋮\\ y_k^{\prime }\\ ⋮\\ y_{(\mathrm{e}-1)k+1}^{\prime }\\ ⋮\\ y_{ek}^{\prime }\\ ⋮\\ y_{(D-1)k+1}^{\prime }\\ ⋮\\ y_{D\times k}^{\prime }\end{array}\right)$, then the vectorized output feature map $y$ can be formalized as Equation (15).

$$Y^{\prime }=\left(\begin{array}{cc}A& b\end{array}\right)\left(\begin{array}{c}q\\ 1\end{array}\right)$$

(14)

$$y_{\mathrm{e}}=\max \left(y_{(e-1)k+1}^{\prime }\hspace{1em}, \dots ,\hspace{1em}{y}_{ek}^{\prime }\right)$$

(15)

SSTA facilitates the cooperative processing of scale, space and task information, thereby enhancing the perceptual capabilities of the model and improving its detection performance.

3.3. Shape-IoU Loss

The bounding box regression losses represented by CIoU, DIoU, and SIoU, which are integrated into the optimizer, play an important role in the localization task of object detection. However, the losses ignore the influence of the intrinsic properties such as the shape and scale of the target, as shown in Figure 2.

Figure 2. GT frames and prediction frames.

In Figure 2, for A and B, all bounding boxes maintain the same scale, while the ground truth boxes between A and B have different shapes. In A and C, the bounding boxes have the same shape, but the GT boxes differ in scale. The data presented in A and B illustrate that the deviation along the short side has a larger impact on the IoU values in comparison to the long side. From A and C, it can be observed that at smaller scales, the influence on the IoU values is more sensitive than at larger scales. To address this shortcoming, we introduce the shape-scale loss for bounding box regression (Shape-IoU Loss) that focuses on the intrinsic properties of the target, as shown in Figure 3. Shape-IoU Loss is computed by utilizing the shape and scale of the bounding boxes, which optimizes the metric of the regression loss to improve detection accuracy.

Figure 3. Schematic diagram of Shape-IoU Loss.

Let scale represent the scale factor. Suppose that $w^{GT}$ and $h^{GT}$ are the width and height of the ground truth box, respectively. Then $w_{shape}$ and $h_{shape}$ can be computed by Equations (16) and (17):

$$w_{shape}={\displaystyle \frac{2\times {(w^{GT})}^{scale}}{{(w^{GT})}^{scale}+{(h^{GT})}^{scale}}}$$

(16)

$$h_{shape}={\displaystyle \frac{2\times {(h^{GT})}^{scale}}{{(w^{GT})}^{scale}+{(h^{GT})}^{scale}}}$$

(17)

where $w_{shape}$ and $h_{shape}$ indicate the shape of the GT box, and scale denotes the target size.

Let $B$ and $B^{GT}$ denote the predicted box and the target box, respectively. Assume that w and h denote the width and height of the prediction box, respectively. Suppose that ${(\mathrm{x}}_{\mathrm{c}}{,\mathrm{y}}_{\mathrm{c}})$ and $(x_c^{GT},y_c^{GT})$ are the coordinates of the center points in the predicted box and the labeled box, respectively. Let us assume that c is the diagonal length of the smallest rectangle enclosing both the prediction box and the GT box. Then the formulas for Shape-IoU Loss are as follows.

$$D^{shape}={\displaystyle \frac{h_{shape}\times {(x_c-x_c^{GT})}^2}{c^2}}+{\displaystyle \frac{w_{shape}\times {(y_c-y_c^{GT})}^2}{c^2}}$$

(18)

$${\Omega }^{shape}={{\displaystyle \sum _{t=w,h}(1-e^{-{\omega }_t})}}^{\theta },\theta =4$$

(19)

$${\omega }_w=h_{shape}\times {\displaystyle \frac{\left|w-w^{GT}\right|}{\max (w,w^{GT})}}$$

(20)

$${\omega }_h=w_{shape}\times {\displaystyle \frac{\left|h-h^{GT}\right|}{\max (h,h^{GT})}}$$

(21)

$$L_{\mathit{\text{Shape-IoU}}}=1-{\displaystyle \frac{\left|B\cap B^{GT}\right|}{\left|B\cup B^{GT}\right|}}+D^{shape}+0.5\times {\Omega }^{shape}$$

(22)

The regression accuracy is improved, since Shape-IoU Loss not only includes the geometric relationship between the target box and the predicted box, but also integrates the intrinsic properties including the shape and scale of the target.

3.4. Overall Architecture of LSCA

The overview of LSCA is illustrated in Figure 4. First, after the input images are fed into the backbone to extract features, the LPAFPN module is constructed to integrate feature information across different channels and reduce the number of parameters in the model. Second, SSTA is designed to improve the representational capacity of the model by coordinating the processing of scale, space, and task information. Then, Shape-IoU Loss is introduced to focus on the shape and scale of the bounding box itself, thereby improving detection accuracy. Finally, we achieve the outcomes that are regions estimated to be objects from the detection head.

Figure 4. Overall architecture of LSCA.

4. Experimental Results and Analysis

4.1. Dataset and Evaluation Metrics

We evaluated the proposed algorithm’s performance on the Pascal VOC dataset and VisDrone2019-DET dataset. To comprehensively validate the efficacy of the LSCA model, we not only calculated F1 score, Average Precision (AP), Mean Average Precision (mAP), Precision (P), Recall (R) and Parameter Count, but also plotted the P-R curves and F1 bar charts.

4.2. Experimental Platform and Training Details

The proposed model is trained and tested under Ubuntu18.04 OS with Intel(R) Xeon(R) Platinum 8255C CPU @ 2.50 GHz and NVIDIA GeForce RTX 3080 GPU. Pytorch 1.9.0 is employed as a deep learning framework.

In the training stage, the detection models are trained using SGD as the optimizer, with the epoch set to 100 and the batch size set to 8. The momentum factor and weight decay factor are set to 0.937 and 0.0005, respectively. The initial learning rate is set to 0.01, and the learning rate is dynamically adjusted via cosine annealing strategy. Additionally, L in Section 3.2 is set to 3.

4.3. Quantitative Analysis

4.3.1. Quantitative Analysis on Pascal VOC Dataset

In this section, Table 1 shows the comparison of Mean Average Precision (mAP) and Average Precision (AP) for each category of nine algorithms. It is noted that LSCA reaches the best AP in 13 classes, obtains suboptimal AP in 3 classes, and achieves 84.12% mAP, which is 10.04%, 9.09%, 10.92%, 11.71%, 8.93%, 1.14%, 6.61%, and 1.05% higher than those of ATSS [32], TOOD, Faster-RCNN, Dynamic Head [33], NAS-FCOS [34], YOLOv7, YOLOv8, and YOLOv9-c, respectively.

Table 1. AP and mAP of the nine algorithms.

Method	mAP (%)	AP of 20 Classes (%)
ATSS	74.08	Aero	Bicycle	Bird	Boat	Bottle	Bus	Car
		70.78	83.55	74.89	62.37	51.64	82.36	79.56
		Cat	Chair	Cow	D-table	Dog	Horse	M-bike
		89.10	59.20	80.06	70.59	85.37	83.71	79.15
		Person	P-plant	Sheep	Sofa	Train	Tv
		76.58	48.21	75.21	72.08	80.89	76.21
TOOD	75.03	Aero	Bicycle	Bird	Boat	Bottle	Bus	Car
		68.88	85.14	74.16	61.78	50.30	85.32	79.91
		Cat	Chair	Cow	D-table	Dog	Horse	M-bike
		87.74	62.75	81.53	70.09	84.62	86.05	83.33
		Person	P-plant	Sheep	Sofa	Train	Tv
		77.74	50.04	73.01	77.68	83.94	76.59
Faster R-CNN	73.20	Aero	Bicycle	Bird	Boat	Bottle	Bus	Car
		73.31	83.90	73.30	60.59	53.20	83.21	83.40
		Cat	Chair	Cow	D-table	Dog	Horse	M-bike
		86.71	42.61	78.80	68.91	84.72	82.01	76.61
		Person	P-plant	Sheep	Sofa	Train	Tv
		69.92	31.81	70.12	74.80	80.41	70.42
Dynamic Head	72.41	Aero	Bicycle	Bird	Boat	Bottle	Bus	Car
		79.13	71.78	80.40	51.32	46.40	79.77	66.31
		Cat	Chair	Cow	D-table	Dog	Horse	M-bike
		90.08	56.01	78.54	71.20	85.95	76.48	77.73
		Person	P-plant	Sheep	Sofa	Train	Tv
		75.88	51.61	77.42	73.78	83.52	74.04
NAS-FCOS	75.19	Aero	Bicycle	Bird	Boat	Bottle	Bus	Car
		74.48	82.29	77.43	64.33	49.42	84.98	79.43
		Cat	Chair	Cow	D-table	Dog	Horse	M-bike
		87.02	62.24	82.13	72.46	87.21	83.76	79.43
		Person	P-plant	Sheep	Sofa	Train	Tv
		75.20	54.85	74.38	72.92	84.69	75.12
YOLOv8	77.51	Aero	Bicycle	Bird	Boat	Bottle	Bus	Car
		85.47	79.91	68.30	68.26	67.55	85.78	83.12
		Cat	Chair	Cow	D-table	Dog	Horse	M-bike
		91.70	59.93	78.41	66.79	83.50	89.77	87.61
		Person	P-plant	Sheep	Sofa	Train	Tv
		86.05	58.14	76.82	75.46	88.30	68.88
YOLOv7	82.98	Aero	Bicycle	Bird	Boat	Bottle	Bus	Car
		92.47	77.38	91.13	67.64	66.17	91.60	88.65
		Cat	Chair	Cow	D-table	Dog	Horse	M-bike
		95.59	71.48	88.71	77.64	94.28	83.01	86.27
		Person	P-plant	Sheep	Sofa	Train	Tv
		87.74	64.06	88.90	79.48	88.94	82.96
LSCA	84.12	Aero	Bicycle	Bird	Boat	Bottle	Bus	Car
		93.31	77.44	92.18	70.44	66.04	93.28	88.11
		Cat	Chair	Cow	D-table	Dog	Horse	M-bike
		96.36	70.43	90.07	82.01	94.80	83.72	88.27
		Person	P-plant	Sheep	Sofa	Train	Tv
		87.93	65.20	90.70	79.39	91.68	81.04
YOLOv9-c	83.07	Aero	Bicycle	Bird	Boat	Bottle	Bus	Car
		92.31	79.03	89.64	68.52	64.10	92.17	89.97
		Cat	Chair	Cow	D-table	Dog	Horse	M-bike
		96.31	68.49	89.51	80.05	94.10	84.13	88.24
		Person	P-plant	Sheep	Sofa	Train	Tv
		87.82	60.50	88.78	78.36	89.78	80.98

Note: The bold font indicates the best value, and the second-best value is underlined.

As shown in Figure 5 and Figure 6, we performed P-R curve analysis and F1 bar chart analysis to evaluate the performance of LSCA.

Figure 5. Comparison of P-R curves of the nine algorithms on the PASCAL VOC dataset.

Figure 6. Comparison of F1 bar chart of the nine algorithms on the PASCAL VOC dataset.

As is well known, the closer the P-R curve is to the top right corner, the better the detection performance. As can be seen from Figure 5, the P-R curve of the proposed algorithm is closest to the upper right corner, suggesting that LSCA is superior to the compared algorithms.

Moreover, it is well known that the higher the location of the F1 bar chart, the better the overall performance. Following Figure 6, the F1 bar chart of LSCA is the highest, which shows that our presented method outperforms other state-of-the-art algorithms.

As shown in Table 2, the proposed LSCA achieves optimal results in terms of mAP, F1 score, and Precision when compared with twelve state-of-the-art object detection models, including ATSS, TOOD, Faster R-CNN, Dynamic Head, NAS-FCOS, YOLOv8, YOLOv7, YOLOv9-c, Lite YOLO-ID [35], YOLO-RACE [36], MLCA [37], and EfficientDet-D0 [38].

Table 2. Comparison of objective evaluation indexes of the nine algorithms on the PASCAL VOC dataset.

Method	mAP (%)	Params (M)	F1 Score (%)	Precision (%)	Recall (%)	FPS
LSCA	84.12	35.10	80.03	81.16	80.78	68.1
ATSS	74.08	31.93	72.01	60.83	88.16	49
TOOD	75.03	31.84	68.97	56.51	88.60	22.8
Faster R-CNN	73.20	41.16	63.04	49.21	87.50	7
Dynamic Head	72.41	38.79	76.89	67.01	90.49	46.5
NAS-FCOS	75.19	32.08	78.1	70.08	87.18	42.1
YOLOV8	77.51	105.97	75.93	80.95	68.79	143.2
YOLOV7	82.98	37.62	79.21	78.91	79.68	62.3
YOLOv9-c	83.07	51.04	77.87	79.84	77.60	99.7
Lite YOLO-ID	78.48	3.76	75.35	78.26	72.63	137.2
YOLO-RACE	77.31	3.2	73.02	76.44	69.87	153.6
MLCA	56.59	3.01	57.17	60.37	54.31	82.1
Efficientdet_d0	30.18	3.84	21.36	62.33	13.43	128.4

Note: The bold font indicates the best value, and the second-best value is underlined.

4.3.2. Quantitative Analysis on VisDrone2019-DET Dataset

Table 3 lists the results of six evaluation indicators for eleven algorithms. It can be seen from Table 3 that the Mean Average Precision, F1 score, Precision, and Recall of the LSCA algorithm all perform better than those of the other ten mainstream comparative algorithms.

Table 3. Comparison of objective evaluation indexes of the nine algorithms on VisDrone2019-DET dataset.

Method	mAP (%)	F1 Score (%)	Precision (%)	Recall (%)	Params (M)	FPS
Faster-RCNN	32.9	39.17	44.9	34.2	41.39	28.5
SSD	24.1	26.03	20.8	35.9	28.37	93.7
YOLOv5l	38.7	39.98	44.2	36.1	44.82	31.5
YOLOv7	42.0	48.07	57.2	41.9	36.5	38.3
YOLOv8s	40.9	45.1	52.6	39.6	11.1	124.1
YOLOv9c	39.7	46.12	52.7	40.7	8.04	53.4
YOLOv10l	43.8	46.97	55.8	41.6	25.7	50.6
YOLOv11l	44.3	48.08	55.6	42.3	25.2	63.4
YOLO-RACE	31.67	36.47	41.95	32.11	3.2	103.8
YOLO-MMS	27.16	45.11	51.08	40.14	3.28	100.5
LSCA	44.7	49.12	58.1	43.1	33.9	47.3

Note: The bold font indicates the best value, and the second-best value is underlined.

4.4. Qualitative Analysis

To qualitatively analyze the detection performance of the proposed algorithm, we selected six images from the Pascal VOC 2007 test set to compare with the algorithms of ATSS, Dynamic Head, Faster-RCNN, NAS-FCOS, TOOD, YOLOv7, and YOLOv8. Figure 7 shows the images to be detected, and Figure 8, Figure 9, Figure 10, Figure 11, Figure 12 and Figure 13 present the comparison of detection results from the eight algorithms.

Figure 7. These tested samples are labeled as 1 to 6, from left to right.

Figure 8. The detected results of image 1.

Figure 9. The detected results of image 2.

Figure 10. The detected results of sample 3.

Figure 11. The detected results of image 4.

Figure 12. The detected results of sample 5.

Figure 13. The detected results of image 6.

Sample 1 contains three person labels, one motorbike label, and one potted plant label, and the scene in this sample is complex, with occlusion and incomplete targets. The experimental results are shown in Figure 8, which reveals that ATSS and YOLOv8 exhibit issues with duplicate detections for the motorbike and person targets, respectively. ATSS, Faster-RCNN, and YOLOv8 fail to detect the potted plant target. Dynamic Head, NAS-FCOS, and TOOD fail in detecting the person and potted plant targets. Only YOLOv7 and LSCA have no problems with duplicate detection and missed detection, and therefore yield the best detection performance.

The second sample to be detected consists of two persons and one bicycle. The scene of this sample is characterized by darkness and insufficient lighting, and the detection results are shown in Figure 9. It can be seen from Figure 9 that ATSS, Dynamic Head, and Faster-RCNN fail to detect the person target. TOOD incorrectly detects the person target. Even worse, NAS-FCOS exhibits issues of missed detection, false detection, and duplicate detection. It is worth noting that YOLOv7, YOLOv8, and the LSCA successfully avoid missed detection, false detection, and duplicate detection, while LSCA demonstrates more accurate localization of the person target.

The targets of test sample 3 are composed of three persons and two chairs. Overlap and occlusion occur in this sample, as shown in Figure 10. The results indicate that ATSS, Dynamic Head, Faster-RCNN and TOOD fail to detect the chair targets, while NAS-FCOS and YOLOv7 miss the detection of both chair and person targets. Additionally, YOLOv8 not only misses the detection of the two chair targets, but also suffers from duplicate detection. It is noted that LSCA achieves optimal detection performance without any issues of missed detection, duplicate detection, or false detection.

The labels of the fourth tested image include four persons, two bottles, one chair, and one dining table. The sample scenario is complex, with a large number of targets. The results are shown in Figure 11, from which we can observe that ATSS, Faster-RCNN, NAS-FCOS and TOOD fail to detect bottle or chair targets while Dynamic Head exhibits missed detection of person, bottle, and chair targets. In addition, YOLOv7 and YOLOv8 suffer from inaccurate localization and duplicate detection issues, respectively. In a word, LSCA reaches the best detection performance without the problems of duplicate detection, missed detection, and poor localization accuracy.

The fifth image to be detected consists of 2 person labels, 1 TV monitor label, and 1 potted plant label. The scene of this sample is characterized by a complex background and the presence of nested objects. Figure 12 shows the results which demonstrate that ATSS, Faster R-CNN, and NAS-FCOS are unable to detect the person targets while ATSS and Dynamic Head fail to detect the potted plant target and the TV monitor target, respectively. Meanwhile, TOOD exhibits missed detection and false detection for the potted plant and person targets, correspondingly. YOLOv8 not only misses the detection of the person target, but also has generated false positives for the potted plant and chair targets. Additionally, YOLOv7 exhibits inaccurate localization of the potted plant target. It is noteworthy that LSCA avoids false detection, missed detection, and poor localization accuracy, demonstrating the highest detection accuracy.

Test image 6 contains 2 person labels, 2 chair labels, and 1 dining table label, and the scene exhibits significant variations in the target scale with regard to small targets. By examining the detection results in Figure 13, we can see that Dynamic Head has the situation of missing detection of person targets, while YOLOv7 and YOLOv8 do not detect both chair and person targets, and ATSS, NAS-FCOS and TOOD struggle with missed detection of person targets, chair targets and dining table targets. Moreover, Faster-RCNN not only misses detecting the person targets and dining table targets, but also falsely identifies a potted plant target. In contrast, only LSCA achieves the best detection performance, committing neither missed nor false detection.

4.5. Ablation Study

To evaluate the effectiveness of GSConv, SSTA, and Shape-IoU, extensive ablation experiments are performed, and the results are shown in Table 4.

Table 4. Ablation experiment results of the LSCA algorithm.

Group	GSConv	SSTA	Shape-IoU	mAP (%)	Params (M)
1				82.98	37.6
2	√			82.94	35.9
3		√		84.06	36.9
4			√	83.74	37.6
5	√	√		83.44	35.1
6	√		√	83.17	35.9
7		√	√	84.21	36.6
8	√	√	√	84.12	35.1

Table 4 illustrates the results of the ablation experiments. A comparison between rows 5 and 8 in Table 4 suggests that the absence of Shape-IoU results in a 0.68% decrease in mAP, which indicates that Shape-IoU improves the accuracy of bounding box regression loss by focusing on the intrinsic properties of targets. As can be seen from the sixth and eighth rows in Table 4, the mAP decreases by 0.95% due to the absence of SSTA, which enables adaptive processing of joint information related to scale, space, and task to enhance the model’s perception capabilities. From the detection results in row 7 and row 8 of Table 4, it can be seen that the mAP improves by 0.09% while the parameters increase by 1.5 M without GSConv. This shows that GSConv effectively shuffles and fuses the outputs from standard convolution and depth-wise separable convolution, reducing Parameter Count while almost maintaining model accuracy. Furthermore, comparing row 1 and row 8 in Table 4 reveals that the proposed algorithm achieves both reduced complexity and improved detection accuracy. Specifically, the integration of GSConv, SSTA, and Shape-IoU can bring a 1.14% improvement in mAP, and reduce parameters by 2.5 M.

As we can see from rows 1, 3, and 4 in Table 4, the introduction of SSTA and Shape-IoU into the baseline model can bring an mAP improvement of 1.08% and 0.76%, respectively. Rows 1 to 3 show that the incorporation of GSConv and SSTA reduces the Parameter Count by 1.7 M and 0.7 M, respectively. This demonstrates that SSTA contributes most to the improvement in network performance, while GSConv contributes to the greatest reduction in Parameter Count.

The results of the ablation study on GSConv are reported in Table 5. As illustrated in Table 5, GSConv achieves suboptimal results in both mAP and the number of parameters among the four convolution methods. Although GSConv exhibits an mAP value that is 0.04% lower than that of Conv, it has 1.7 M fewer parameters than Conv. Conversely, while the Parameter Count of GSConv is 0.2 M higher than that of PConv, its mAP is 0.76% higher than that of PConv. Compared with the other three convolution methods, GSConv reduces the Parameter Count while maintaining nearly the same detection accuracy. This validates the effectiveness of GSConv.

Table 5. Ablation experimental results of GSConv.

Method	Params (M)	mAP (%)
Conv	37.6	82.98
PConv	35.7	82.18
DySnakeConv	41.8	82.87
GSConv	35.9	82.94

Note: The bold font indicates the best value, and the second-best value is underlined.

The effectiveness of the proposed feature fusion network can be verified through the ablation experimental results with different numbers of SSTA modules in Table 6. As shown in Table 6, when the number of SSTA modules is greater than one, the performance of the proposed feature fusion network surpasses that of the baseline network. Notably, when two SSTA modules are employed, not only does the network achieve the optimal mAP, but its Parameter Count is also 0.73 M fewer than that of the baseline network. Therefore, it validates the effectiveness of the proposed feature fusion method.

Table 6. Ablation experimental results of SSTA.

Dynamic Head	Params (M)	mAP (%)
0	37.61	82.98
1	34.54	82.90
2	36.88	84.06
3	38.33	83.91
4	40.29	83.71

Note: The bold font indicates the best value, and the second-best value is underlined.

In the experiment, the SSTA module is integrated into several popular object detectors to evaluate its generalization capability. As shown in Table 7, the introduction of the SSTA module improves the mAP of the baseline detector by 0.92%, 0.93%, 1.08%, 0.92%, and 0.84% for Faster R-CNN, ATSS, YOLOv7, YOLOv8, and NAS-FCOS, respectively. This consistent improvement across diverse architectures confirms the module’s strong generalization performance.

Table 7. Ablation study of the SSTA module on different algorithms.

Method	mAP (%)
Faster R-CNN	73.20
Faster R-CNN + SSTA	74.12
ATSS	74.08
ATSS + SSTA	75.01
YOLOv7	82.98
YOLOv7 + SSTA	84.06
YOLOv8	77.51
YOLOv8 + SSTA	78.43
NAS-FCOS	75.19
NAS-FCOS + SSTA	76.03

The performance of the detection algorithm varies with different scale factors of the Shape-IoU loss function. As shown in Table 8, a scale factor of 1.2 yields the highest mAP value. Consequently, the proposed algorithm sets the scale factor at 1.2.

Table 8. The selection of scale factors.

Group	Method	Scale	mAP (%)
1	Shape-IoU	0	83.41
2	Shape-IoU	0.5	83.55
3	Shape-IoU	1.0	83.53
4	Shape-IoU	1.2	83.74
5	Shape-IoU	1.3	83.56
6	Shape-IoU	1.4	83.44

Note: The bold font indicates the best value.

Table 9 presents the ablation experiment results of Shape-IoU. As shown in Table 9, DIoU, Wise-IoU, SIoU, and Shape-IoU achieve mAP improvements of 0.24%, 0.36%, 0.67%, and 0.76%, respectively, compared with CIoU. This indicates that Shape-IoU performs the best among the five methods, validating the effectiveness of the Shape-IoU method.

Table 9. Ablation experiment results of Shape-IoU.

Group	Method	mAP (%)
1	CIoU	82.98
2	DIoU	83.22
3	Wise-IoU	83.34
4	SIoU	83.65
5	Shape-IoU	83.74

Note: The bold font indicates the best value, and the second-best value is underlined.

5. Conclusions

In this work, we present a novel one-stage object detection algorithm via Shape-IoU and a Lightweight Path Aggregation Feature Pyramid Network featuring scale–space–task collaborative enhancement. Specifically, we construct an LPAFPN that uniformly exchanges local feature information across different channels to fully integrate the information output from both standard convolution and depth-wise separable convolution, reducing the number of model parameters. Furthermore, ALPAFPN is designed to enhance the network’s representation capability by simultaneously perceiving scale information across feature layers, spatial information at different locations, and task information among different output channels. Moreover, the detection performance of the model is improved by introducing Shape-IoU Loss, which focuses on the shape and scale of targets to optimize the regression loss function. The experimental results demonstrate that, when compared with the state-of-the-art algorithms used on the Pascal VOC dataset and VisDrone2019-DET dataset, our approach achieves superior performance in terms of F1 score, Precision, and Mean Average Precision.