LssDet: A Lightweight Deep Learning Detector for SAR Ship Detection in High-Resolution SAR Images

Abstract

Synthetic aperture radar (SAR) ship detection has been the focus of many previous studies. Traditional SAR ship detectors face challenges in complex environments due to the limitations of manual feature extraction. With the rise of deep learning (DL) techniques, SAR ship detection based on convolutional neural networks (CNNs) has achieved significant achievements. However, research on CNN-based SAR ship detection has mainly focused on improving detection accuracy, and relatively little research has been conducted on reducing computational complexity. Therefore, this paper proposes a lightweight detector, LssDet, for SAR ship detection. LssDet uses Shufflenet v2, YOLOX PAFPN and YOLOX Decopuled Head as the baseline networks, improving based on the cross sidelobe attention (CSAT) module, the lightweight path aggregation feature pyramid network (L-PAFPN) module and the Focus module. Specifically, the CSAT module is an attention mechanism that enhances the model’s attention to the cross sidelobe region and models the long-range dependence between the channel and spatial information. The L-PAFPN module is a lightweight feature fusion network that achieves excellent performance with little computational effort and a low parametric count. The Focus module is a low-loss feature extraction structure. Experiments showed that on the Sar ship detection dataset(SSDD), LssDet’s computational cost was 2.60 GFlops, the model’s volume was 2.25 M and $AP@[0.5:0.95]$ was 68.1%. On the Large-scale SAR ship detection dataset-v1.0 (LS-SSDD-v1.0), LssDet’s computational cost was 4.49 GFlops, the model’s volume was 2.25 M and $AP@[0.5:0.95]$ was 27.8%. Compared to the baseline network, LssDet had a 3.6% improvement in $AP@[0.5:0.95]$ on the SSDD, and LssDet had a 1.5% improvement in $AP@[0.5:0.95]$ on the LS-SSDD-v1.0. At the same time, LssDet reduced Floating-point operations per second (Flops) by 7.1% and Paraments (Params) by 23.2%. Extensive experiments showed that LssDet achieves excellent detection results with minimal computational complexity. Furthermore, we investigated the effectiveness of the proposed module through ablation experiments.

1. Introduction

Synthetic aperture radar (SAR) is a high-resolution imaging radar with long-range, all-day, all-weather operation [1,2,3]. It plays an essential role in ship detection, environmental inspection, geological mapping and other applications [4,5,6]. As a fundamental maritime task [7,8,9], SAR ship detection has received significant attention in areas such as marine traffic control, maritime distress rescue and maritime defence warning [7,8,9]. The traditional and deep learning methods are the main methods of SAR ship detection [10].

Traditional methods generally complete SAR ship detection tasks by manually extracting the geometric structure features, the electromagnetic scattering features, the transformation features and the local invariant features. The constant false alarm rate (CFAR) is the most widely and deeply applied traditional SAR ship detection method. Novak et al. [11] proposed a two-parameter CFAR approach for modelling sea clutter using a Gaussian model. Gao et al. [12] proposed a method based on Parzen-window-kernel density estimation as an SAR image ship detection method based on non-parametric modelling. Sugimoto et al. [13] combined Yamaguchi decomposition theory with the CFAR method to accomplish SAR ship detection. Lang et al. [14] proposed a hierarchical SAR ship recognition scheme by extracting texture descriptors to construct a robust ship classification and recognition representation. Leng et al. [15] combined the SAR ship’s intensity and location information and proposed a bilateral CFAR method. Dai et al. [16] proposed a CFAR SAR ship detection method based on the supervised learning approach based on object candidate regions. However, the traditional methods rely excessively on manual feature extraction, so this one has a tedious computational process and poor generalisation, which cannot meet the needs of SAR ship detection. With the rise of deep learning technology, SAR ship detection algorithms based on deep learning have been widely studied in recent years [17,18,19,20].

Deep learning SAR ship detection methods have shown excellent performance. Kang et al. [17] used the object proposals generated by Faster R-CNN for the guard window of the CFAR algorithm to enhance the detection performance for small targets. Kang et al. [4] also proposed a multi-layer fused convolutional neural network based on contextual regions, fusing deep semantic features and shallow high-resolution features to improve detection performance for small ships. Chang et al. [18] developed a reduced YOLOv2 algorithm that has fewer layers than regular YOLOv2, and they achieved speed and accuracy improvements. Tang et al. [21] developed a noise level classifier (NLC), an SAR target potential area extraction module (STPAE) and a YOLOv5-based detection module, N-YOLO, and achieved competitive performance. Cui et al. [22] proposed a method based on CenterNet, which is based on spatial shuffle-group enhancement (SSE) attention, for large-scale SAR image ship detection. Zhu et al. [20] proposed an R-FCOS method that can detect dim and small ships in large-scale SAR images with higher accuracy than other that of methods.

Current SAR ship detection research is mainly based on computationally intensive neural networks. Although achieving high detection accuracy, the computational complexity of such models is unsatisfactory. However, computational complexity and detection accuracy are equally essential for SAR ship detection tasks. Yu et al. [23] designed the ASIR-Block, Focus-Block, SPP-Block and CAPE-Block modules based on the attention mechanism and feature pyramid network and constructed FASC-Net. Zhou et al. [24] proposed the CSPMRes2 module for improving feature representation and the FC-FPN module for adapting feature maps, and constructed MSSDNet based on YOLOv5s. Liu et al. [25] used YOLOV4-LITE and MobileNetv2 as the baseline network, proposed an improved receptive field block (RFB) structure to enhance the network’s feature extraction ability and proposed a sliding-window-block method to detect the whole SAR image, which can solve the problem of image input. Lightweight SAR ship detectors have excellent applications, especially in seaside defence warnings and sea distress rescue, which should not be neglected.

For this reason, we propose LssDet, an SAR ship detection method based on a lightweight convolutional neural network. LssDet implements a lightweight, anchor-free, one-stage SAR ship detection method which has sufficient robustness for multi-scale targets, complex backgrounds and large-scene SAR images; and it has a low computational cost and low parametric counts.

The main contributions of this paper are as follows:

(1) A new attention module, the cross sidelobe attention (CSAT) module, is constructed. This module enhances the model’s attention to the cross sidelobe region and models the long-range dependence of the channel and spatial information, improving the method’s effectiveness for SAR ship detection.

(2) A new fusion network module, the lightweight path aggregation feature pyramid network (L-PAFPN) module, is constructed. This module uses the L-CSP module based on the ghost block design to replace the CSP module in YOLOX PAFPN. By reducing the network’s redundancy, the computational load is reduced, the number of parameters of the method is reduced and the method’s performance for SAR ship detection is improved.

(3) A new feature extraction structure, the Focus module, is introduced. Based on the baseline network, this paper introduces the Focus module to improve the backbone network. It increases the number of feature channels in the feature extraction process, improves the feature extraction capability of the backbone network and enhances the SAR ship detection effect of the method.

(4) A lightweight SAR ship detector, LssDet, is constructed. Based on the baseline network and the CSAT module, the L-PAFPN module and the Focus module, we proposed LssDet, which achieves competitive performance with low computational complexity and a low number of model parameters. The detector is proposal-free and anchor-free.

The remainder of the paper is organised as follows: In Section 2, we present LssDet’s baseline network and our proposed architecture. In Section 3, we present the experiments’ details and results and analyse the experimental results. In Section 4, we discuss the experimental results and the shortcomings of the experiment. In Section 5, we draw some conclusions and give an outlook on future research. In addition,

Table A1. The abbreviations and corresponding full terms.

Abbreviation	Full Name
AP	Average precision
Avg Pool	Adaptive average pooling
BAM	Bottleneck attention module
CBAM	Convolutional block attention module
CNN	Convolutional neural network
BN	Batch normalisation
CenterNet	Keypoint triplets for object detection
CFAR	Constant false alarm rate
Conv	Convolution
CSAT	Cross sidelobe attention
CSP	Cross stage partial
CV	Computer vision
DWconv	Depthwise separable convolution
DL	Deep learning
FLOPs	Floating-point operations per second
FN	False negatives
FCOS	Fully convolutional one-stage object detection
FP	False positives
FPN	Feature pyramid network
LssDet	Lightweight SAR ship detector
L-PAFPN	Lightweight path aggregation feature pyramid network
LS-SSDD-v1.0	Large-scale SAR ship detection dataset-v1.0
P	Precision
PAFPN	Path aggregation feature pyramid network
Params	Paraments
R	Recall
R-CNN	Region-based convolutional neural networks
SAR	Synthetic aperture radar
SE	Squeeze-and-excitation networks
SGD	Stochastic gradient descent
TN	True negatives
TP	True positives
YOLO	You only look once

offers all the abbreviations and corresponding full terms involved for the convenience of reading.

2. Method

This paper proposes a lightweight neural network, LssDet, for SAR ship detection tasks. This section details the main ideas of LssDet. In particular, Section 2.1 introduces the baseline network of LssDet, and Section 2.1 introduces specific improvements to LssDet.

2.1. Baseline

This paper proposes a lightweight neural network, LssDet, for SAR ship detection tasks. In this section, we detail the main ideas of LssDet. Section 2.1 presents the baseline network architecture of LssDet. Section 2.2 presents the proposed architecture of LssDet.

LssDet’s backbone network is based on the lightweight network Shufflenet v2 [26]. Low computational computer vision (CV) tasks widely use lightweight networks. Due to the limited computational resource, the lightweight network only uses a limited number of feature channels. Shufflenet v2 enables an efficient and accurate backbone network through extensive feature reuse, which significantly increases the number of feature channels used by the network with limited computational resources. Specifically, Shufflenet v2 uses a depthwise separable convolution (DWconv) [27] and hopping structure to design the block unit. At the same time, the channel shuffle operation shuffles each group of channels evenly, enabling information communication between different channel groups, significantly increasing the number of network feature channels and improving the network’s performance at a fraction of the cost of a full-sized network. The details of Shufflenet v2 are shown in Figure 1. The left subfigure shows the structure of Shufflenet v2, the middle subfigure shows the basic unit of Shufflenet v2 and the right subfigure shows the spatial down sampling unit (2×) of Shufflenet v2.

LssDet’s feature fusion network is based on YOLOX PAFPN [28]. In CV tasks, the backbone network has a hierarchical structure, outputting features at different levels. The feature fusion network achieves complementary strengths by optimally combining features at different levels. Specifically, the backbone network extracts features from shallow to deep layers. Shallow features have high resolution and contain more locational and detailed information but less semantic information; deeper features have richer semantic information but lower resolution and less perception of details. The feature pyramid network (FPN) [29] enhances the feature representation capability of the network by introducing top-down paths that fuse features from adjacent levels based on bottom-up feature extraction from the backbone network. Based on the FPN, the path aggregation feature pyramid network (PAFPN) [30] introduces bottom-up paths to enhance the information transfer between features at different levels, further improving the network’s feature extraction capability. Based on the PAFPN, YOLOX PAFPN refers to a cross stage partial network (CSPNet) [31] that has a CSP Block as a cross-stage information fusion module to improve performance and decrease the computational cost. The structure of YOLOX PAFPN is shown in Figure 2.

LssDet’s prediction network is based on the YOLOX Decopuled Head [28]. Most existing detection models use a coupled head network for their prediction networks [32,33,34], which achieves classification and regression tasks with less computational cost by sharing network parameters. However, the coupled head network can lead to performance losses due to the inconsistent focus of the classification and regression tasks [35,36]. Therefore, YOLOX achieves the classification and regression tasks through different network branches, thereby reducing the adverse effects of parameter sharing and improving network performance. Specifically, YOLOX first reduces the channel’s number of input data by 1 × 1 convolution. Then, it splits data into two parallel branches, each containing two 3 × 3 convolutions. The classification branch calculates the classification loss, and the regression branch calculates the regression loss and IoU loss. The structure of the prediction network is shown in Figure 3 [28]. The top subfigure shows the structure of the coupled head network, and the bottom subfigure shows the structure of YOLOX.

2.2. Proposed Architecture

LssDet has improved the baseline network primarily through the CSAT module, the L-PAFPN module and the Focus module. This section describes the details of these modules.

2.2.1. CSAT

Ships generally consist of cockpits, sides, decks, storage compartments, railings and other structures. The backscattering of radar incident waves varies significantly among the different structures. When the backscattering is surface scattering on decks and in storage compartments, the scattering intensity is weak. In contrast, in the two-sided angular regions formed by areas such as decks, sides and cockpits, the backscatter is angular reflection, and the scattering intensity is intense.

According to the SAR imaging principle, the echo signal of the focused target is a two-dimensional $Sinc$ function, expressed as a gradually decaying signal in azimuthal and distance directions. Equation (1) shows the $Sinc$ function.

$$\begin{array}{c}\hfill Sinc\left(t\right)=\left\{\begin{array}{c}sin\left(\pi t\right)/\pi t,t\ne 0\hfill \\ 1,t=0\hfill \end{array}\right.\end{array}$$

(1)

When the backscattering caused by the ship’s structure is extreme, or the image is poorly focused, although the echo sidelobe signal is much weaker than the primary sidelobe signal, it will still be more substantial than the surface signal or other weakly scattering parts of the echo signal. Thus, there can be an apparent cross sidelobe effect by coherent superposition. In SAR imaging, it appears as two straight lines crossing vertically in the azimuthal and distance directions. Due to the imaging mechanism of the cross sidelobe, they generally occur in areas of the ocean where there are angular reflections, such as ships, harbours and drilling platforms, and rarely on land or elsewhere. Figure 4 shows examples of cross sidelobes in the LS-SSDD-v1.0 [37].

In SAR ship detection tasks, the appearance of a cross sidelobe is closely related to the ship’s position. In particular, the cross sidelobe is more evident for some poorly imagable ships. The imaging area of the cross sidelobe will be significantly larger than the imaging area of the ship. Therefore, increasing the attention to the cross sidelobe region to improve SAR ship detection performance is meaningful.

The attention mechanism is a resource allocation mechanism that highlights the impact of necessary information by adaptively assigning attention weights to the input information and is widely used in neural networks [38]. In this paper, we use the attention mechanism to assign attention weights to the cross sidelobe region, which improves the detection performance of SAR ships. However, most attention mechanism methods are computationally intensive, which makes them difficult to apply to lightweight neural networks. At present, squeeze-and-excitation (SE) attention [39] is still the most widely used attention mechanism in lightweight neural networks. However, in the feature space, SE attention only considers the channel information but ignores the spatial information. To consider the spatial information in the feature space, the bottleneck attention module (BAM) [40] and the convolutional block attention module (CBAM) [41] combine the channel and spatial information in the feature space by sequentially computing channel and spatial attention. However, this approach can only model regional dependencies and cannot model long-range dependencies. However, long-range dependencies are essential for computer vision tasks [42]. For this reason, we designed the CSAT module to model the long-range dependence of the channel and spatial information.

In order to make better use of the cross sidelobe region features and model the long-range dependence of the channel and spatial information, we designed the CSAT module. The CSAT module focuses the weight of the input data on the square region where the cross sidelobe is located by decomposing the conventional attention mechanism into two parallel one-dimensional feature encoding processes along the horizontal and vertical directions. In addition, through the parallel one-dimensional encoding process, the CSAT module can effectively integrate feature spatial coordinates for long-range modelling dependence on the channel and spatial information. The structure of the CSAT module is shown in Figure 5.

Specifically, for the given input $X\in {\mathbb{R}}^{c\times h\times w}$, where c is the channel number (input data channel), and h and w are the height and width of the input data. Firstly, we split X into x and y along the channels, $x\in {\mathbb{R}}^{\left(c/2\right)\times h\times w}$, $y\in {\mathbb{R}}^{\left(c/2\right)\times h\times w}$. Then, we encode x and y along the horizontal and vertical directions using two spatially scoped pool kernels $(H,1)$ and $(1,W)$. Therefore, the output data at height h of the c channel can be formulated as:

$$\begin{array}{c}\hfill z_c^h\left(i\right)=\frac{1}{W}\sum _{0\le i<W}{x}_c(h,i)\end{array}$$

(2)

Similarly, the output at the width w of the c-th channel can be formulated as:

$$\begin{array}{c}\hfill z_c^w\left(j\right)=\frac{1}{H}\sum _{0\le j<H}{y}_c(j,w)\end{array}$$

(3)

The above transform produces aggregated features for x and y along the horizontal and vertical directions, producing a pair of direction-aware feature maps.

Next, the feature mappings $z^h$ and $z^w$ are sent to the $1\times 1$ convolutional transform functions $F_{h1}$ and $F_{w1}$.

$$\begin{array}{c}\hfill {\mathbf{f}}^{h1}=\delta \left(F_{h1}\left(z^h\right)\right)\end{array}$$

(4)

$$\begin{array}{c}\hfill {\mathbf{f}}^{w1}=\delta \left(F_{w1}\left(z^w\right)\right)\end{array}$$

(5)

where $\delta$ represents the nonlinear activation function Hswish; ${\mathbf{f}}^{h1}\in {\mathbb{R}}^{(C/2·r)\times H}$ and ${\mathbf{f}}^{w1}\in {\mathbb{R}}^{(C/2·r)\times W}$ are intermediate feature mappings that encode spatial information in the vertical and horizontal directions. Next, we concatenate ${\mathbf{f}}^{h1}$ and ${\mathbf{f}}^{w1}$ along the channels and send them to the $1\times 1$ convolutional transform function $F_2$.

$$\begin{array}{c}\hfill {\mathbf{f}}_{\mathbf{2}}=\delta \left(F_2\left([{\mathbf{f}}^{h1},{\mathbf{f}}^{w1}]\right)\right)\end{array}$$

(6)

where $[·,·]$ denotes the concatenating operation along the channel, ${\mathbf{f}}_{\mathbf{2}}\in {\mathbb{R}}^{C/r\times \left[\right(H+W)/2]}$ is the intermediate feature mapping encoding spatial information in the vertical and horizontal directions, r is the scaling ratio used to control the number of channels. Next, channel shuffling is performed, splitting ${\mathbf{f}}_{\mathbf{2}}$ into ${\mathbf{f}}^{h3}$ and ${\mathbf{f}}^{w3}$ along the channel, where ${\mathbf{f}}^{h3}\in {\mathbb{R}}^{(C/2·r)\times H}$ and ${\mathbf{f}}^{w3}\in {\mathbb{R}}^{(C/2·r)\times W}$. Then, ${\mathbf{f}}^{h3}$ and ${\mathbf{f}}^{w3}$ are sent to the $1\times 1$ convolutional transform functions $F_{h3}$ and $F_{w3}$.

$$\begin{array}{c}\hfill g^h=\sigma \left(F_{h3}\left({\mathbf{f}}^{h3}\right)\right)\end{array}$$

(7)

$$\begin{array}{c}\hfill g^w=\sigma \left(F_{w3}\left({\mathbf{f}}^{w3}\right)\right)\end{array}$$

(8)

where $\sigma$ represents the nonlinear activation sigmoid function, $g^h\in {\mathbb{R}}^{C/2\times H}$, $g^w\in {\mathbb{R}}^{C/2\times W}$. Finally, the output of the CSAT module can be expressed as:

$$\begin{array}{c}\hfill y_c(i,j)=[x_{c/2}(i,j)\times g_c^h\left(i\right),y_{c/2}(i,j)\times g_c^w\left(j\right)]\end{array}$$

(9)

The CSAT module improves the SAR ship detection performance of LssDet by enhancing the model’s focus on the cross sidelobe region and modelling the long-range dependence of the channel and spatial information. Figure 6 shows the position of the CSAT module inserted in the Shufflenet v2 block unit. The left part shows the position of the CSAT module inserted into the basic unit of Shufflenet v2, and the right part shows the position of the CSAT module inserted into the spatial down sampling unit (2×) of Shufflenet v2.

2.2.2. L-PAFPN

How to better utilise the features extracted from images by the backbone network is a fundamental problem in CV tasks. By introducing top-down paths to fuse features from adjacent layers output by the backbone network, the FPN [29] enhances the semantic information in shallow features to adapt to targets at different scales in object detection tasks. Based on the FPN, the PAFPN [30] introduces bottom-up paths, fusing shallow resolution information into deeper features in a bottom-up manner, thereby enhancing the network’s ability to exploit features. Further, YOLOX PAFPN introduces the CSP block [31] as the information fusion module to improve the detection performance of the network.

YOLOX PAFPN performs cross-layer information fusion via bottom-up and top-down paths, significantly improving detection performance. However, YOLOX PAFPN uses the CSP block as the cross-layer information fusion module, which has complex computational and redundant information, making it computationally burdensome for applications in lightweight neural networks.

For this reason, inspired by the GhostNet [43], we propose L-PAFPN, which uses the L-CSP block based on the ghost block design instead of the CSP block in YOLOX PAFPN. Compared to the CSP block, the L-CSP block requires less computational effort. Figure 7 shows the organisation of the L-PAFPN module’s components. The left part shows the structure of the ghost block, the middle part shows the structure of the ghost bottleneck and the right part shows the structure of the L-PAFPN module. Figure 8 shows the structure of the L-PAFPN module.

We first introduce the ghost block, part of the L-PAFPN. For the given input $X\in {\mathbb{R}}^{c\times h\times w}$, where c is the channel number of the input data, and h and w are the height and width of the input data. The regular convolutional layer operation for generating n feature maps can be expressed as:

$$\begin{array}{c}\hfill {\mathbf{f}}_{\mathbf{1}}=\delta \left(F_1\left(\mathbf{X}\right)\right)\end{array}$$

(10)

where $F_1$ represents the convolutional transform function, and ${\mathbf{f}}_{\mathbf{1}}\in {\mathbb{R}}^{n\times h\times w}$ represents the intermediate feature mapping.

However, due to the characteristics of the convolutional transform function, ${\mathbf{f}}_{\mathbf{1}}$ usually contains a large amount of redundancy, which places a burden on the network computation. Therefore, we can split n. Specifically, a part of the feature mapping is generated using the convolutional transform function, and a low-computational-cost linear operation generates another portion of feature mapping. The convolution operation used to generate the m feature mapping can be expressed as:

$$\begin{array}{c}\hfill {\mathbf{f}}_{\mathbf{2}}=\delta \left(F_2\left(\mathbf{X}\right)\right)\end{array}$$

(11)

where $F_2$ represents the 1 × 1 convolutional transform function, ${\mathbf{f}}_{\mathbf{2}}\in {\mathbb{R}}^{m\times h\times w}$, $m\le n$.

To obtain the required number of feature maps, n, we apply a further operation on ${\mathbf{f}}_{\mathbf{2}}$ to generate s features, which can be expressed as:

$$\begin{array}{c}\hfill {\mathbf{f}}_{\mathbf{3}}=\delta \left(F_3\left({\mathbf{f}}_{\mathbf{2}}\right)\right)\end{array}$$

(12)

where $F_3$ represents the 3 × 3 depthwise separable convolutional transform function, ${\mathbf{f}}_{\mathbf{3}}\in {\mathbb{R}}^{s\times h\times w}$, $s=n-m$.

At this point, concatenate ${\mathbf{f}}_{\mathbf{2}}$ and ${\mathbf{f}}_{\mathbf{3}}$ along the channel and get the ghost block output ${\mathbf{f}}_{\mathbf{4}}$.

$$\begin{array}{c}\hfill {\mathbf{f}}_{\mathbf{4}}=[{\mathbf{f}}_{\mathbf{2}},{\mathbf{f}}_{\mathbf{3}}]\end{array}$$

(13)

We next introduce the ghost bottleneck, which is also one of the components of the L-PAFPN. It consists of a series of operations carried out by the ghost block.

Specifically, for the given input $Y\in {\mathbb{R}}^{c\times h\times w}$, we first send Y into the ghost block transform function $Ghostblock$, and get the output ${\mathbf{g}}_{\mathbf{1}}$, and then send ${\mathbf{g}}_{\mathbf{1}}$ into the ghost block transform function $Ghostblock$ to get the output ${\mathbf{g}}_{\mathbf{2}}$, which can be formulated as:

$$\begin{array}{c}\hfill {\mathbf{g}}_{\mathbf{1}}=Ghostblock\left(\mathbf{Y}\right)\end{array}$$

(14)

$$\begin{array}{c}\hfill {\mathbf{g}}_{\mathbf{2}}=Ghostblock\left({\mathbf{g}}_{\mathbf{1}}\right)\end{array}$$

(15)

At the same time, we send the input data Y into the 3 × 3 depthwise separable convolutional transform function $G_3$ and get the output ${\mathbf{g}}_{\mathbf{3}}$, and then send ${\mathbf{g}}_{\mathbf{3}}$ into the 1 × 1 convolution transform function $G_4$ and get the output ${\mathbf{g}}_{\mathbf{4}}$, which can be formulated as:

$$\begin{array}{c}\hfill {\mathbf{g}}_{\mathbf{3}}=\sigma \left(G_3\left(\mathbf{Y}\right)\right)\end{array}$$

(16)

$$\begin{array}{c}\hfill {\mathbf{g}}_{\mathbf{4}}=\sigma \left(G_4\left({\mathbf{g}}_{\mathbf{3}}\right)\right)\end{array}$$

(17)

Finally, the output of the ghost bottleneck is $\mathbf{f}={\mathbf{g}}_{\mathbf{2}}+{\mathbf{g}}_{\mathbf{4}}$, where + represents the adding operation.

We finally introduce the L-CSP module, which consists of the ghost bottleneck. Specifically, for the input data Z, $Z\in {\mathbb{R}}^{c\times h\times w}$.

The input Z is first to split into $z_1$ and $z_2$ along the channel dimensions, $z_1\in {\mathbb{R}}^{(c/2)\times h\times w}$ and $z_2\in {\mathbb{R}}^{(c/2)\times h\times w}$. $z_1$ is sent to the $1\times 1$ convolutional transform function $Z_1$, and $z_2$ is sent to the ghost bottleneck transform function $Ghostbottleneck$.

$$\begin{array}{c}\hfill {\mathbf{z}}_{\mathbf{3}}=\delta \left(Z_1\left(z_1\right)\right)\end{array}$$

(18)

$$\begin{array}{c}\hfill {\mathbf{z}}_{\mathbf{4}}=Ghostbottleneck\left(z_2\right)\end{array}$$

(19)

where ${\mathbf{z}}_{\mathbf{3}}$ and ${\mathbf{z}}_{\mathbf{4}}$ represent the intermediate feature mapping, ${\mathbf{z}}_{\mathbf{3}}\in {\mathbb{R}}^{(n/2)\times h\times w}$ and ${\mathbf{z}}_{\mathbf{4}}\in {\mathbb{R}}^{(n/2)\times h\times w}$. We concatenate ${\mathbf{z}}_{\mathbf{3}}$ and ${\mathbf{z}}_{\mathbf{4}}$ along the channel and send the result to the 1 × 1 convolution $Z_2$. Finally, the output of the L-CSP block can be expressed as:

$$\begin{array}{c}\hfill \mathbf{f}=\delta \left(Z_2[{\mathbf{z}}_{\mathbf{3}},{\mathbf{z}}_{\mathbf{4}}]\right)\end{array}$$

(20)

where $\mathbf{f}\in {\mathbb{R}}^{n\times h\times w}$. Finally, the L-CSP module is applied to YOLOX PAFPN, replacing the original CSP module to obtain our proposed L-PAFPN module.

2.2.3. Focus

In lightweight neural networks, due to their limited computational resources, only a limited number of feature channels can be used, which limits the network’s performance. To reduce the impact of the limited number of feature channels on the network performance, we use the Focus [28,34] module for feature extraction instead of the first convolutional layer in the backbone network.

The Focus module splits the input image into four equal parts and sends them into the backbone network. Through these operations, the Focus module increases the number of feature channels, reduces the information loss during feature extraction and improves network performance.

Specifically, for the given input $X\in {\mathbb{R}}^{c\times h\times w}$, the Focus module is split $\mathbf{X}$ into ${\mathbf{f}}_{\mathbf{1}}$, ${\mathbf{f}}_{\mathbf{2}}$, ${\mathbf{f}}_{\mathbf{3}}$ and ${\mathbf{f}}_{\mathbf{4}}$ by inter-pixel fetching.

$$\begin{array}{c}\hfill {\mathbf{f}}_{\mathbf{1}}\in {\mathbb{R}}^{c\times (h/2)\times (w/2)},\\ \hfill {\mathbf{f}}_{\mathbf{2}}\in {\mathbb{R}}^{c\times (h/2)\times (w/2)},\\ \hfill {\mathbf{f}}_{\mathbf{3}}\in {\mathbb{R}}^{c\times (h/2)\times (w/2)},\\ \hfill {\mathbf{f}}_{\mathbf{4}}\in {\mathbb{R}}^{c\times (h/2)\times (w/2)}.\end{array}$$

(21)

Next, ${\mathbf{f}}_{\mathbf{1}}$, ${\mathbf{f}}_{\mathbf{2}}$, ${\mathbf{f}}_{\mathbf{3}}$ and ${\mathbf{f}}_{\mathbf{4}}$ are concatenated along the channel dimension and sent to the $3\times 3$ convolutional transform function $F_1$.

$$\begin{array}{c}\hfill \mathbf{f}=\delta \left(F_1\left([{\mathbf{f}}_{\mathbf{1}},{\mathbf{f}}_{\mathbf{2}},{\mathbf{f}}_{\mathbf{3}},{\mathbf{f}}_{\mathbf{4}}]\right)\right)\end{array}$$

(22)

where $\mathbf{f}$ is the output of the Focus module, $\mathbf{f}\in {\mathbb{R}}^{n\times h\times w}$, $n=4c$.

3. Experiments

This section validates our proposed SAR ship detection method’s performance. Firstly, we introduce the hardware and software environment for the experiments, the dataset used for experiments, the details of the experiments and the experimental evaluation metrics. Next, we introduce the experiments’ results on the SAR Ship Detection Dataset (SSDD) [44] and the Large-Scale SAR Ship Detection Dataset v1.0 (LS-SSDD-v1.0) [37] and compare them with the latest detection methods’ results. Finally, we introduce the ablation experiments’ results to verify the validity of each module we proposed.

3.1. Experimental Environment

All experiments used the same environment to run. The environment’s configuration details are shown in

Table 1. Experimental environment.

Configuration	Parameter
CPU	Intel(R) Xeon(R) Gold 6140 CPU @ 2.30 GHz
GPU	Tesla P100-PCIE-16GB GPU
Operating system	Ubuntu 18.04
Development tools	PyTorch 1.8, python3.7, CUDA11.1, CuDNN 8.0.5

.

3.2. Dataset

In the experiments, we used two SAR datasets to evaluate our proposed method, the SSDD and the LS-SSDD-v1.0. The SSDD includes 1160 SAR images with 500 × 500 pixels, with various resolutions, from 1 to 15 m. The LS-SSDD-v1.0 includes 15 large-scale images with 24,000 × 16,000 pixels, and the 15 large-scale images were directly cut into 9000 sub-images with 800 × 800 pixels, with various resolutions from 5 m to 20 m.

Table 2. Details of the SSDD.

Parameter	Result
Satellite	RadarSat-2, TerraSAR-X, Sentinel-1
Sensor mode	–
Location	Yantai, Visakhapatnam
Resolution (m)	1–15
Polarization	HH, HV, VV, VH
Image size (pixel)	500 × 500
Cover width (km)	10
Image number	1160

and

Table 3. Details of the LS-SSDD-v1.0.

Parameter	Result
Satellite	Sentinel-1
Sensor mode	IW
Location	Tokyo, Adriatic Sea, etc.
Resolution (m)	5-20
Polarization	VV, VH
Image size (pixel)	24,000×16,000
Cover width (km)	250
Image number	15

show the details of the SSDD and the LS-SSDD-v1.0.

To eliminate the impact of a different dataset division on the experimental results, we used the same dataset division as the dataset publisher. Specifically, the SSDD’s training subset contains 928 images, and the test subset contains 232 images. The LS-SSDD-v1.0’s training subset contains 6000 images, and the test subset contains 3000 images.

3.3. Experimental Setup

All experiments used stochastic gradient descent (SGD) with the learning rate lr = 0.01, momentum m = 0.937 and weight decay = 0.0005 as the optimisation algorithm. Furthermore, the batch size was 16; and the input size was $608\times 608$ in the SSDD and $800\times 800$ in the LS-SSDD-v1.0. All experiments were constructed based on the MMdetection [45], and the unmentioned parameters were the default parameters of MMdetection.

3.4. Evaluation Criteria

All experiments used $AP@[0.5:0.95]$, $AP@0.5$, $AP@0.75$, $APs$, $APm$ and $APl$ index in the COCO evaluation [46], floating-point operations per second (FLOPs) and paraments (Params) to measure the detection performance of the proposed method. $AP@[0.5:0.95]$, $AP@0.5$, $AP@0.75$, $APs$, $APm$ and $APl$ measure the detection accuracy of the method; $Flops$ measures the computational complexity of the method; and $Params$ measures the parameter size of the method.

$$\begin{array}{c}\hfill AP={\int }_0^{1}P\left(R\right)dR\end{array}$$

(23)

$$\begin{array}{c}\hfill AP@[0.5:0.95]=\frac{{\sum }_{x=0.5:0.05}^{0.95}AP@x}{10}\end{array}$$

(24)

$$\begin{array}{c}\hfill APs:area<322pixels\end{array}$$

(25)

$$\begin{array}{c}\hfill APm:322pixels<area<962pixels\end{array}$$

(26)

$$\begin{array}{c}\hfill APl:962pixels<area\end{array}$$

(27)

where,

$$\begin{array}{c}\hfill P=\frac{TP}{TP+FP}\end{array}$$

(28)

$$\begin{array}{c}\hfill R=\frac{TP}{TP+FN}\end{array}$$

(29)

TP, FP, TN, and FN are shown in

Table 4. Confusion matrix.

	Actual Positive	Actual Negative
Predicted Positive	TP	FP
Predicted Negative	FN	TN

.

3.5. Compared with the Latest Detectors in SAR Ship Detection

According to the experimental setup and evaluation criteria, we conducted experiments on the SSDD and LS-SSDD. Furthermore, we compared the SAR ship detection performance of LssDet with that of the latest methods, such as ATSS [47], AutoAssign [48], FCOS [49], YOLO V3 [32], YOLOX-s [28] and YOLOX-tiny [28]. The experiment results on the SSDD are shown in

Table 5. Comparison of the performances of different methods on the SSDD.

Method	Backbone	AP@[0.5:0.95]	[email protected]	[email protected]	Aps	Apm	Apl	FLOPs	Params
ATSS	ResNet50	64.5	94.0	76.6	65.0	65.8	35.3	72.74	31.89
AutoAssign	ResNet50	63.7	93.6	77.3	64.4	63.2	44.7	71.45	35.97
FCOS	ResNet50	62.4	93.6	73.3	62.4	65.2	52.8	71.03	31.84
YOLO V3	MobileNet v2	61.2	94.1	74.0	63.1	55.7	34.2	5.93	3.67
YOLOX-s	CSP Darknet53	67.3	96.4	81.7	68.5	65.0	33.3	6.97	5.92
YOLOX-tiny	CSP Darknet53	62.9	93.6	76.6	65.4	55.2	4.8	4.00	3.34
OurModel	OurNet	68.1	96.7	82.1	69.7	65.3	38.0	2.60	2.25

, and the experimental results on the LS-SSDD-v1.0 are shown in

Table 6. Comparison of the performances of different methods on the LS-SSDD-v1.0.

Method	Backbone	AP@[0.5:0.95]	[email protected]	[email protected]	Aps	Apm	FLOPs	Params
ATSS	ResNet50	24.3	66.5	8.9	23.3	35.2	125.95	31.89
AutoAssign	ResNet50	20.1	60.1	5.9	19.0	31.6	123.7	35.97
FCOS	ResNet50	23.5	66.0	7.8	22.4	35.0	122.98	31.84
YOLO V3	MobileNet v2	22.2	64.5	7.6	21.2	33.4	10.27	3.67
YOLOX-s	CSP Darknet53	27.7	76.2	10.5	26.5	38.4	12.06	5.92
YOLOX-tiny	CSP Darknet53	26.8	74.3	9.4	25.8	36.9	6.92	3.34
Our Model	Our Net	27.8	74.8	11.2	26.9	37.7	4.49	2.25

.

The experimental results on the SSDD and the LS-SSDD-v1.0 demonstrate the superiority of the method proposed in this paper. From a quantitative perspective, the $AP@[0.5:0.95]$ of LssDet was the optimal algorithm on the SSDD. Specifically, LssDet had an $AP@[0.5:0.95]$ of 68.1%, which is 3.6%, 4.4%, 5.7%, 6.9%, 0.8% and 5.2% higher than those of ATSS, AutoAssign, FCOS, YOLO V3, YOLOX-s and YOLOX-tiny, respectively. On the LS-SSDD-v1.0, the $AP@[0.5:0.95]$ of LssDet was optimal. Specifically, LssDet had an $AP@[0.5:0.95]$ of 27.8%, which was 3.5%, 7.7%, 4.3%, 5.6%, 0.1% and 1.0% higher than those of ATSS, AutoAssign, FCOS, YOLO V3, YOLOX-s and YOLOX-tiny, respectively. Both $Flops$ and $Params$ of LssDet were optimal. The $Flops$ of LssDet were 3.57%, 3.63%, 3.65%, 43.72%, 37.23% and 64.88% for ATSS, AutoAssign, FCOS, YOLO V3, YOLOX-s and YOLOX-tiny, respectively. The $Params$ of LssDet were 7.06%, 6.26%, 7.07%, 61.31%, 38.01% and 67.37% for ATSS, AutoAssign, FCOS, YOLO V3, YOLOX-s and YOLOX-tiny, respectively.

To visualise the detection performance of LssDet, Figure 9 shows the detection results of LssDet on the SSDD and the LS-SSDD-v1.0. The first and second columns show the label visualisation results and LssDet’s detection results on the SSDD. The third and fourth columns show the label visualisation results and LssDet’s detection results on the LS-SSDD-v1.0.

3.6. Ablation Experiments

To validate the effectiveness of LssDet, we designed two sets of ablation experiments on the SSDD and the LS-SSDD, each containing six sub-experiments.

The first experiment used Shufflenet v2, YOLOX PAFPN and YOLOX Decopuled Head as the baseline networks for comparison in subsequent experiments. The second experiment used the CSAT module inserted into the Shufflenet v2 bottleneck structure primarily to verify the effectiveness of the proposed CSAT module. The third experiment used the L-PAFPN module to replace the YOLOX PAFPN module, mainly to verify the performance of the proposed lightweight feature fusion network. In the fourth experiment, we used the Focus module to replace the first layer of convolution in Shufflenet v2, mainly to verify the effect of feature extraction on the network’s performance. The fifth experiment was a superimposition of the second and third experiments, primarily to verify the effectiveness of the CSAT and L-PAFPN module superimpositions. The sixth experiment was a superposition of the second, third and fourth experiments, mainly to verify the effectiveness of the CSAT, L-PAFPN and Focus module superpositions. The six experiments were carried out step by step to validate our proposed method’s effectiveness and superiority. The datasets and parameter settings were kept constant in all experiments.

The results of the two sets of ablation experiments are shown in

Table 7. Ablation experiment on the SSDD.

CSAT	L-PAFPN	Focus	AP@[0.5:0.95]	[email protected]	[email protected]	Flops	Params
F	F	F	64.5	94.9	78.1	2.80	2.93
T	F	F	66.2	96.6	81.3	2.80	2.98
F	T	F	66.3	96.0	81.7	2.41	2.20
F	F	T	65.9	96.0	81.7	2.98	2.94
T	T	F	67.1	96.1	82.4	2.42	2.25
T	T	T	68.1	96.7	82.1	2.60	2.25

and

Table 8. Ablation experiment on the LS-SSDD-v1.0.

CSAT	L-PAFPN	Focus	AP@[0.5:0.95]	[email protected]	[email protected]	Flops	Params
F	F	F	26.3	72.8	9.6	4.85	2.93
T	F	F	26.8	73.8	9.3	4.85	2.98
F	T	F	26.6	73.9	10.3	4.18	2.20
F	F	T	26.7	73.4	10.0	5.16	2.94
T	T	F	27.1	74.3	10.4	4.18	2.25
T	T	T	27.8	74.8	11.2	4.49	2.25

. Table 7 shows the results of the ablation experiments on the SSDD, and Table 8 shows the results of the ablation experiments on the LS-SSDD-v1.0. In Experiment 2, the CSAT module was used to assign attention weights and simulate the long-range dependence of channel and spatial information on the cross sidelobe region of the SAR ship, thereby obtaining a 1.7% $AP@[0.5:0.95]$ boost on the SSDD and a $AP@[0.5:0.95]$ boost of 0.5% on the LS-SSDD-v1.0. In Experiment 3, L-PAFPN was used to reduce the computational load and parametric counts of YOLOX PAFPN, thereby obtaining a 1.8% $AP@[0.5:0.95]$ boost on the SSDD and a $AP@[0.5:0.95]$ boost on the LS-SSDD-v1.0 by 0.3%. Meanwhile, $FLOPs$ were reduced by 13.9% and $Params$ was reduced by 24.9%. In Experiment 4, the Focus module was used to improve the backbone network’s feature extraction, thereby obtaining a 1.4% $AP@[0.5:0.95]$ improvement on the SSDD and a $AP@[0.5:0.95]$ improvement of 0.4% on the LS-SSDD-v1.0. In Experiment 5, the introduction of both the CSAT module and the L-PAFPN module resulted in a 2.6% $AP@[0.5:0.95]$ boost on the SSDD and a $AP@[0.5:0.95]$ boost of 0.8% on the LS-SSDD-v1.0. In Experiment 6, the CSAT module, the L-PAFPN module and the Focus module were introduced simultaneously, resulting in an $AP@[0.5:0.95]$ boost of 3.6% on the SSDD and a 1.5% $AP@[0.5:0.95]$ boost on the LS-SSDD-v1.0. Compared to the baseline, $FLOPs$ were reduced by 7.1% and $Params$ were reduced by 23.2%. The quantitative analysis of the performance evaluation metrics confirmed the superiority of our proposed SAR ship detection method and the effectiveness of the individual modules.

4. Discussion

Experimental results on the SSDD and the LS-SSDD-v1.0 demonstrated the superiority of the method proposed in this paper. Based on Shufflenet v2, YOLOX PAFPN and YOLOX Decopuled Head, we proposed the CSAT module and the L-PAFPN module and introduced the Focus module. The CSAT module improves the SAR ship detection performance by assigning attention weights to SAR ship cross sidelobe regions and models the long-range dependence of the channel and spatial information. The L-PAFPN module improves the SAR ship detection performance by lightening the YOLOX PAFPN module. The Focus module improves the SAR ship detection performance by enhancing the feature extraction capability of the backbone network. We demonstrated the effectiveness of each module through ablation experiments. The SSDD’s imaging results were better with fewer cross sidelobe phenomena. However, the LS-SSDD-v1.0 imaging results were poor with more cross sidelobe phenomena. Therefore, on the SSDD, the CSAT module improved performance mainly by modelling the long-range dependence of channel and spatial information. On the LS-SSDD-v1.0, the CSAT module obtained performance gains by assigning attention weights and modelling the long-range dependence of channel and spatial information. Thus, the L-PAFPN module most significantly improved performance on the SSDD, and the CSAT module most significantly improved performance on the LS-SSDD-v1.0.

However, the CSAT module is based on Shufflenet v2, and the effectiveness of applying it to other backbone networks needs further study. LssDet achieved excellent performance gains on both $AP@[0.5:0.95]$ and $AP@0.75$, but the performance gain on $AP@0.5$ was not as good as the former. The research in this paper was mainly concerned with backbone and feature fusion networks, and other parts of the detection method, such as prediction networks, were not further investigated. As the existing datasets only include factors such as ships, ports and land, we did not find other objects with cross sidelobes identified as ships on the existing dataset. Misidentification may occur if future datasets include more samples with cross sidelobe phenomena.

5. Conclusions

In this paper, we proposed a lightweight, anchor-free detector, LssDet, for SAR ship detection tasks. We used Shufflenet v2, YOLOX PAFPN and YOLOX Decopuled Head as the baseline models. Moreover, we proposed the CSAT module and the L-PAFPN module, and introduced the Focus module, to improve the detection performance. Among them, the CSAT module was responsible for cross sidelobe region feature enhancement and modelling the long-range dependence of channel and spatial information. The L-PAFPN module is responsible for lightly modifying the feature fusion network. The Focus module is responsible for enhancing the feature extraction capability. Experimental results on the SSDD showed that LssDet achieves an $AP@[0.5:0.95]$ of 68.1%, a $FLOPs$ of 2.60 G and a $Params$ of 2.25 M. Experimental results on the LS-SSDD-v1.0 showed that LssDet achieved an $AP@[0.5:0.95]$ of 27.8%, a $FLOPs$ of 4.49 G and $Params$ of 2.25 M. LssDet had superior detection performance from both quantitative and qualitative perspectives. Comparison with other recent detection methods showed that LssDet achieved optimal detection results with minimal computational effort and minimal parameters. Further, the results of ablation experiments demonstrated the effectiveness of each module of LssDet.

In the future, we will further investigate the effectiveness of CSAT on different backbone networks, find methods for improving the performance gains of LssDet on $AP@\left[0.5\right]$ and investigate improvements to other structures in the detection method to achieve better detection results in SAR ship detection tasks.