R-SABMNet: A YOLOv8-Based Model for Oriented SAR Ship Detection with Spatial Adaptive Aggregation

Abstract

Synthetic Aperture Radar (SAR) is extensively utilized in ship detection due to its robust performance under various weather conditions and its capability to operate effectively both during the day and at night. However, ships in SAR images exhibit various characteristics including complex land scattering interference, variable scales, and dense spatial arrangements. Existing algorithms are insufficient in effectively addressing these challenges. To enhance detection accuracy, this paper proposes the Rotated model with Spatial Aggregation and a Balanced-Shifted Mechanism (R-SABMNet) built upon YOLOv8. First, we introduce the Spatial-Guided Adaptive Feature Aggregation (SG-AFA) module, which enhances sensitivity to ship features while suppressing land scattering interference. Subsequently, we propose the Balanced Shifted Multi-Scale Fusion (BSMF) module, which effectively enhances local detail information and improves adaptability to multi-scale targets. Finally, we introduce the Gaussian Wasserstein Distance Loss (GWD), which effectively addresses localization errors arising from angle and scale inconsistencies in dense scenes. Our R-SABMNet outperforms other deep learning-based methods on the SSDD+ and HRSID datasets. Specifically, our method achieves a detection accuracy of 96.32%, a recall of 93.13%, and an average level of accuracy of 95.28% on the SSDD+ dataset.

1. Introduction

As a case of microwave-based imaging radar, SAR produces remote sensing images with high resolution [1]. It facilitates Earth observation through active microwave irradiation [2,3]. In contrast to optical imagery, SAR images provide some penetration capability [4,5]. Due to its exceptional observational capabilities [6], SAR is extensively utilized in military reconnaissance, marine resource management, and rescue missions [7]. In particular, SAR has demonstrated considerable potential for ship target monitoring [8,9].

With improvements in data quality [10] and the increasing diversity of imaging scenarios, SAR ship target detection still faces several technical challenges [11,12]. First, Figure 1a shows that ship detection is challenged by the similar scattering characteristics of land areas and ship targets in nearshore scenes [13,14]. It tends to cause many false detections in land areas [15,16]. Secondly, as illustrated in Figure 1b, ship targets exhibit variations in scattering characteristics and scales [17,18], requiring detection models to handle multi-scale targets. Additionally, due to the dense arrangement of ships shown in Figure 1a, horizontal-frame-based models fail to describe the shape and position of these targets accurately, resulting in more false positives and missed detections [19,20]. Consequently, oriented bounding boxes represent the information of the target better, resulting in higher detection accuracy [21].

This paper aims to solve these problems by proposing R-SABMNet, which utilizes YOLOv8 as the baseline. It integrates several improvements to strengthen the model’s accuracy and robustness. The proposed method includes the following improvements:

• The SG-AFA module is proposed to aggregate global spatial information. This module enhances the sensitivity to ship features and improves the feature representation of key regions. As a result, it effectively suppresses land scattering interference and boosts the accuracy in intricate scenarios.
• The BSMF method is proposed, using a balanced shifted-window attention mechanism. This module enhances local detail and establishes global dependencies, enabling the handling of scale-variant targets.
• The GWD is introduced as the regression loss to perform the computation of the differentiable Rotated IOU. It addresses localization errors due to angle and scale inconsistencies in dense scenes.
• The experiments are conducted on various public SAR datasets. The results demonstrate that our method surpasses existing mainstream methods across various detection metrics, particularly in complex coastline scenes and multi-scale scenarios.

The following gives a description of the contributions of this paper. A thorough analysis of work related to the topic at hand is given in Section 2, which covers classical models and the progress made in addressing key challenges. The methodology of R-SABMNet is presented in Section 3. Subsequently, the experimental setup is described, the results are analyzed, and the findings are also visualized in Section 4. Finally, Section 5 concludes by summarizing the paper’s contributions.

2. Related Works

To mitigate the challenges in detecting targets within SAR images, several techniques have been proposed by researchers [22,23]. Based on their technical characteristics, these approaches can be classified into two primary types: traditional approaches and deep learning-driven approaches [24].

2.1. Traditional Detection Methods

Classical SAR target detection approaches principally leverage the contrast between the target and the surrounding clutter. Many commonly used models are based on Constant False Alarm Rate (CFAR), with several improvements built upon it [25,26]. In addition, there are other widely applied algorithms, such as matching templates by comparing similarity measures [27], entropy [28], and wavelet transform [29]. However, most of these algorithms are scene-specific. They rely heavily on predefined distributions or manually designed features. This results in shallow feature representations and limited generalization capability.

2.2. Deep Learning-Driven Detection Methods

Deep learning has experienced substantial growth and innovation in recent years [30]. Various network models, particularly CNNs [31], have greatly enhanced object detection performance. In contrast to traditional methods, deep learning approaches eliminate dependency on complex, manually designed feature engineering. They automatically extract and learn deep features from images, enhancing detection accuracy.

The relevant methods can be classified into two fundamental types: two-stage and one-stage methods [32]. The former proposes candidate regions for potential ship targets using predefined anchor boxes, which are then refined for precise target localization. Notable examples include Faster R-CNN [33], Libra R-CNN [34], and Mask R-CNN [35]. The latter performs both classification and regression on anchor boxes in one step. Prominent single-stage algorithms include the YOLO series [36,37,38,38], SSD [39], and FCOS [40]. Overall, two-stage-based methods generally demonstrate superior detection accuracy but require more computational resources, leading to slower speeds. One-stage-based detectors provide an improved harmony between precision and effectiveness.

2.3. Research on the Related Key Points

SAR images vary in terms of sensors, resolution, and ship types. Consequently, the size difference between the diverse pixel regions of ships within a single dataset can vary by up to a factor of a thousand [41]. To deal with the problem of identifying targets with diverse scales, several algorithms have been proposed, including multi-scale feature processing methods [42], scale-invariant feature detectors like SIFT [42], and pyramid-based approaches that capture both global and local information at various resolutions [43]. In addition, an efficient detection approach based on the Hessian matrix has been proposed [44]. It uses integral images for faster computation and ensures scale and rotation invariance. Bi-DFFM [45] integrates top-down and bottom-up pathways to improve multi-scale ship recognition. Based on CenterNet [46], CenterNet++ [47] was developed to address small-scale SAR ship detection.

Ship detection is challenged by the similar scattering characteristics of land areas and ship targets in nearshore scenes. Additionally, removing the vast ocean and coastline backgrounds in SAR images captured at varying heights is a significant challenge. The MLBR-YOLOX method [48] introduces a new SAR ship recognition network using SSPD and DSFD modules. Also inspired by the advantages of the YOLOX framework, AFSar [49] was introduced as an effective anchorless detection method for complex SAR scenes. Based on the FCOS framework, an anchorless SAR vessel detection approach was introduced [50]. This method modifies the assignment strategy for samples, effectively reducing the likelihood of background regions being incorrectly labeled as positive samples.

2.4. Baseline Model Selection

The YOLO series of models has gained widespread popularity among researchers due to its exceptional detection accuracy and inference efficiency. For instance, YOLOv5 and YOLOv7 have demonstrated remarkable performance across various tasks. YOLOv8, introduced by Ultralytics after extensive optimization, represents a significant advancement over the classic YOLOv5. It incorporates substantial innovations in network architecture and loss function design, while also exhibiting superior capabilities in multi-scale object detection and handling complex backgrounds. Notably, in SAR image processing scenarios, YOLOv8 demonstrates enhanced adaptability and robustness in the precise detection of ship targets.

In contrast, YOLOv11, the latest iteration in the YOLO series, primarily focuses on adjustments to model depth and width parameters. However, its core components, such as the loss function, remain largely unchanged, offering only limited performance improvements for SAR-based ship detection tasks. Moreover, YOLOv8 has been extensively applied and rigorously validated, supported by a well-established community and comprehensive technical documentation. These factors ensure its reliability and stability in practical research applications.

Considering the model’s innovation, task suitability, and maturity of application, this study adopts YOLOv8 as the baseline model. On this foundation, we propose R-SABMNet, which aims to further enhance the accuracy and robustness of ship detection in SAR imagery.

3. Proposed Methods

This section presents detailed information regarding R-SABMNet. The overall architecture of R-SABMNet is first described. Then, the implementation principles of the SG-AFA module, the BSMF module, and the improved loss function are presented.

3.1. Model Structure of R-SABMNet

R-SABMNet integrates a Spatial-Guided Adaptive Feature Aggregation (SG-AFA) module and a Balanced Shifted Multi-Scale Fusion (BSMF) module. This approach combines rich and detailed information from different feature layers to achieve high-precision ship target detection. The overall architecture of R-SABMNet is shown in Figure 2. The architecture includes three primary parts: a feature extraction network with the SG-AFA module (SA-FEN), a feature pyramid network with the BSMF module (BSMF-FPN), and an improved detection head. First, the input image is resized to dimensions of 640 × 640 × 3 before feature extraction. It is then passed through the enhanced backbone, which generates three feature maps at different scales: $F3$, $F4$, and $F5$. For multi-scale feature fusion, these maps are provided to the BSMF-FPN. At the final stage, the fused features are passed into the detection head, which generates bounding boxes and classification results.

3.2. Feature Extraction Network Based on SG-AFA Module

The YOLOv8-based backbone designed for feature extraction captures semantic features at different levels. The key improvement in YOLOv8 is the Spatial Pyramid Pooling Fast (SPPF), which enhances the model’s performance by enabling better handling of multi-scale information. This improvement allows for more efficient feature extraction, leading to better accuracy and faster inference, especially when detecting objects at different scales in images. In SAR images, scattered facilities in harbors resemble ship hulls in grayscale and texture. This similarity causes strong interference, making it challenging for the original model to effectively extract ship features, resulting in misdetections and omissions. To address this, the SA-FEN is proposed. It enhances global perception, addressing the limitations of the original feature extraction network.

Figure 3 illustrates the structural composition of the SA-FEN. This network consists of a standard convolution layer, the SG-AFA module, a bottleneck layer, and a cascade of the SPPF module. First, the input image passes through the convolutional layer for initial feature extraction. This layer includes a sequence of three modules: Conv2d, BN, and SiLU activation. The features then pass through the SG-AFA module for spatially adaptive aggregation, enhancing the representation of key regions and focusing on important areas. Next, feature extraction is further refined through multiple convolutional layers and the C2f module. This fuses rich features from both shallow and deep layers, capturing multi-scale information. Finally, high-level features are output through the SPPF module, which performs multi-scale pooling on the feature map. This module aggregates both global and local information, improving the model’s capability to detect target areas at multiple scales.

Clutter and scattering interference from land regions result in a significant number of false positives. To address these issues, the SG-AFA module is proposed. Figure 4 provides an overview of the overall structure. The core idea is to consider not only the value of a single pixel but also the surrounding pixel structure and location during feature extraction. This approach supports the model so it can better catch the spatial relationships between features and enhances feature representation in key areas. This module is introduced after the standard convolutional layer of the backbone to generate the aggregated features completely. It retains the ship outline information while suppressing land scattering interference, thereby improving ship detection accuracy when handling complex backgrounds.

First, the feature matrix F is obtained using the convolutional block. The features are then evenly distributed to the adaptive average pooling (adaptive-avg-pool2d) layer to generate the initial aggregation center matrix $C_k=(C_{x,k},C_{f,k})$. $C_{x,k}$ represents the spatial coordinate and _f,k is a feature vector (e.g., color and depth). The features are normalized by the BN layer.

Second, the features are aggregated using the QS Block. The specific computational process is as follows: To assign pixels to corresponding centers, a combined spatial and feature distance metric is used:

$$D_{i,k}=\sqrt{\left({\displaystyle \frac{\parallel x_i-c_{x,k}{\parallel }^2}{s^2}}\right)+\left({\displaystyle \frac{\parallel f_i-c_{f,k}{\parallel }^2}{m^2}}\right)}$$

(1)

where $x_i$ is the spatial coordinates of pixel i, and $f_i$ is the feature vector of pixel i. In addition, the parameters s and m are used to balance the influence of spatial and feature distances, respectively. A smaller s emphasizes local spatial regions, while a larger s enhances global connections. Similarly, a smaller m highlights feature similarity, whereas a larger m reduces sensitivity to feature differences. Based on multiple experimental validations, this study selects $s=15$ and $m=0.85$ as the final parameter configuration to balance local details and global characteristics while ensuring the stability and accuracy of feature aggregation.

Next, the aggregation weight matrix W is computed to update the superpixel centers. The initial weight matrix is computed as follows:

$$w_{i,k}=e^{-D_{i,k}}$$

(2)

To reduce the interference of land regions, a background suppression strategy is introduced during weight calculation. The modified weight calculation formula is as follows:

$$w_{i,k}=e^{-\alpha D_{i,k}-\beta R_i}$$

(3)

where $R_i$ is the size of the region containing pixel i, and $\alpha$ and $\beta$ are adjustable parameters for balancing the contributions of distance and background suppression. A larger $\alpha$ enhances the effect of distance, emphasizing local features, while a smaller $\alpha$ improves global aggregation capability. A larger $\beta$ effectively suppresses background interference, whereas a smaller $\beta$ focuses more on feature similarity. Given initial values of ${\alpha }_0=4$ and ${\beta }_0=2$, the parameters $\alpha$ and $\beta$ are adaptively adjusted according to the formulas $\alpha ={\alpha }_0·(1+\mathrm{Grad})$ and $\beta ={\displaystyle \frac{{\beta }_0}{1+\mathrm{Grad}}}$, where Grad represents the gradient information at the corresponding position in the image. This strategy adjusts the weight contributions of background regions by reducing their influence, ensuring robust feature aggregation. Then, the weight matrix W is normalized to ensure consistent feature aggregation within each region:

$$w_{i,k}^{\prime }={\displaystyle \frac{w_{i,k}}{{\sum }_{j\in N}{w}_{j,k}}}$$

(4)

Then, W is used to recalculate the aggregation center matrix C:

$$c_{f,k}={\displaystyle \frac{{\sum }_{i\in N_k}{w}_{i,k}^{\prime }{f}_i}{{\sum }_{i\in N_k}{w}_{i,k}^{\prime }}},c_{x,k}={\displaystyle \frac{{\sum }_{i\in N_k}{w}_{i,k}^{\prime }{x}_i}{{\sum }_{i\in N_k}{w}_{i,k}^{\prime }}}$$

(5)

where $N_k$ is the set of pixels that belong to the aggregated region given by k.

Finally, the aggregated matrix $F^{*}$ is computed through the assignment mapping of the central matrix C, and resized to the original dimensions and concatenated with the initial features F. The result is then passed through the BN layer.

Figure 5 illustrates a comparison of feature heatmaps with and without the SG-AFA module. The SG-AFA module utilizes spatial information to adaptively aggregate features, effectively enhancing the representation of target regions while suppressing land interference. Compared to the YOLOv8 without the SG-AFA module, it adaptively adjusts feature weights based on spatial distributions. This significantly enhances the saliency of target regions and suppresses land scattering interference. Under complex backgrounds, the SG-AFA module demonstrates superior spatial consistency and feature aggregation capability. Experimental results indicate that the module enhances the sensitivity to ship features and distinguishes targets from the surrounding land regions.

3.3. Feature Pyramid Network with BSMF Module

A network must be capable of processing multi-scale features due to ships’ multi-scale characteristics. The FPN can fuse features at different scales using a multi-level network structure, making it suitable for multi-scale scenarios. However, FPNs lose some internal structural information during pooling operations. This weakens the model’s ability to reconstruct small targets and capture spatial hierarchical information.

To enable effective cross-scale fusion, we propose an enhanced feature fusion network, as observed in Figure 6. The extracted features $(F3,F4,F5)$ from the backbone are input into the BSMF-FPN module to boost the feature representation capability. In convolutional networks, small target features in SAR images often fade during deep feature extraction. This can lead to missed detections. To address this, the BSMF-FPN introduces the BSMF module into the shallow feature channel $F3$. This enhances local information and feature representation, providing the model with richer semantic data.

The BSMF module consists of two key components: window-based multi-self-attention (W-MSA) and balanced shifted multi-self-attention (BS-MSA). It includes a cascade of LayerNorm and MLP layers. Figure 7 illustrates the model’s specific structure. It splits the image into multiple windows and performs self-attention within each window. This mechanism captures local features and reduces computational complexity. The sliding window mechanism enables information exchange between neighboring windows. This improves its ability to integrate both local and global information.

First, the W-MSA is applied within a local window. The input feature map is decomposed into multiple separate windows. Then, self-attention is applied within each window to capture local features. This reduces computational cost by eliminating global computation to enhance detailed local information.

In BS-MSA, the balanced shifted operation is introduced. The window division shifts relative to the previous layer, adjusting the window size and shift step based on local feature map details. This helps the model establish associations between local windows and capture a broader range of contextual information. By alternating between W-MSA and BS-MSA, this model captures global information and cross-window dependencies while maintaining low computational complexity.

Figure 8 illustrates the implementation of balanced shifted windows. In layer l (left side), an average window division strategy is applied. Then, self-attention is calculated individually for each window. In layer $l+1$ (right side), the windows are shifted according to the current configuration, forming new windows. Smaller window sizes and shift steps are used at target edges or in complex regions (with high gradients) to preserve important details and edge information. In contrast, larger shift steps in background or low-gradient areas facilitate a broader range of information interaction across windows, enhancing the global semantic understanding of the image. The self-attention operation in the new window extends across the boundaries of the windows in layer l, enabling cross-window connectivity.

Compared to the original FPN, the BSMF-FPN integrates the BSMF module into the shallow feature channels. This approach mitigates information loss and enhances the fusion capability of deeper features in FPN, ultimately advancing the model’s capacity to detect multi-scale objects.

3.4. Improved Detect Head with GWD

In SAR images, ships are often not fixed in direction and are densely arranged. The horizontal frame provides only a rectangular bounding box with a fixed orientation, which is insufficient for accurately enclosing vessels. This increases the likelihood of false positives and missed detections. Especially in the case of closely spaced ships, they may be misclassified as a single target. In contrast, the rotating frame can align with the ship’s actual orientation. This tightens the bounding box around the target, reducing background interference. Additionally, the rotating frame adapts better to small or irregularly shaped targets. Therefore, oriented detection with a rotating frame offers distinct advantages in SAR vessel detection. It better handles the multi-angle and elongated shapes of vessels, improving both accuracy and efficiency. A branch for angle prediction is added to the YOLOv8 model for oriented detection. This forms three decoupled detection heads: the classification, regression, and angle prediction branches, each optimized with different loss functions. However, the computation of RIOU for two rotating frames in OBB detection is non-differentiable. This prevents efficient gradient computation during backpropagation. To address this, this paper introduces the GWD function for regression. This function measures the difference between rotating frames by calculating the Wasserstein distance between two Gaussian distributions.

It is assumed that each rotated bounding box can be approximated by a 2D Gaussian distribution as presented in Figure 9. The red rectangle represents the rotated bounding box, while the inside depicts the corresponding Gaussian distribution. The mean (m) of this distribution represents the center of the bounding rectangle, while the covariance matrix (∑) captures the box’s dimensions and rotation. The rotated bounding box $B(x,y,h,w,\theta )$ is first transformed into the distribution $N(m,\sum )$.

$${\sum }^{1/2}=RS{R}^T=\left[\begin{array}{cc}cos\theta & -sin\theta \\ sin\theta & cos\theta \end{array}\right]\left[\begin{array}{cc}{\displaystyle \frac{w}{2}}& 0\\ 0& {\displaystyle \frac{h}{2}}\end{array}\right]\left[\begin{array}{cc}cos\theta & sin\theta \\ -sin\theta & cos\theta \end{array}\right]$$

(6)

$$m=(x,y)$$

(7)

Once the rotated rectangle is converted to a Gaussian distribution, the Wasserstein distance can be used to evaluate the distance between the two distributions. And the Wasserstein distance can be computed using the following formula:

$$d^2={\parallel m_1-m_2\parallel }_2^2+Tr({\sum }_1+{\sum }_2-2{\left({\sum }_1^{1/2}{\sum }_2{\sum }_1^{1/2}\right)}^{1/2})$$

(8)

where $m_1$ and $m_2$ are the mean values of the two Gaussian distributions that correspond to the centers of the rotated bounding boxes, and ${\sum }_1$ and ${\sum }_2$ are the covariance matrices, representing the dimensions and orientations of the rotated boxes. By calculating the Wasserstein distance, we can measure the geometric difference between the two rotated bounding boxes. This method removes the requirement for intricate IoU calculations.

The Generalized Wasserstein Distance (GWD) is mapped to a form that is similar to an IoU-based loss function using a nonlinear transformation function f. The Gaussian function helps focus more on densely located targets, effectively reducing the influence of background regions. It is especially well suited for scenarios where targets are concentrated and close to each other. This enhances the representation of local target information. Therefore, it realizes the nonlinear transformation as f. The IoU-like loss function is as follows:

$$L_{gwd}=1-{\displaystyle \frac{1}{\tau +f\left(d^2\right)}},\tau \ge 1.$$

(9)

where $d^2$ represents the squared distance between the target and predicted regions and $\tau$ is a hyperparameter for controlling the sensitivity of the transformation. Using cross-validation, different values of $\tau$ were tested during training on the validation set. The results showed that $\tau =2$ achieved the best performance. Therefore, $\tau =2$ was chosen as the value for this hyperparameter in this study.

Therefore, GWD simplifies the model’s learning by representing rotated frames as 2D Gaussian distributions. This approach effectively handles challenges related to inconsistent angles and scales.

4. Results

The effectiveness of R-SABMNet is evaluated through ablation comparative experiments and experiments.

4.1. Datasets

We chose the SSDD+ [51] and HRSID [52] datasets as the research data for this paper. These datasets include both offshore and onshore ships, as well as a variety of complex marine scenes. They are labeled with rotated bounding boxes. The first dataset contains 2456 ship targets. The primary sources of the images in this dataset are TerraSAR-X, Sentinel-1, and RadarSat-2. The image resolutions range from 1 m to 15 m. The second dataset includes 5604 images. Satellite sensors, including TanDEM-X and Sentinel-1B, were used to obtain these images. The image resolutions range from 0.5 m to 3 m.

Table 1. Detailed information concerning datasets.

	SSDD+	HRSID
Number of Images	1160	5640
Number of Ships	2551	16,965
Image Size (Pixels)	500 × 500	800 × 800
Resolution (m)	1–15	0.5–3
Polarization Mode	VV, VH, HH, HV	VV, HV, HH

outlines comprehensive information about each dataset.

4.2. Performance Evaluation Metrics

To quantitatively evaluate the detection performance of the R-SABMNet, we use three widely adopted metrics: recall, precision, and AP, which are calculated using the following formulas:

$$Recall={\displaystyle \frac{TP}{TP+FN}}$$

(10)

$$Precision={\displaystyle \frac{TP}{TP+FP}}$$

(11)

$$AP={\int }_0^{1}P\left(R\right)dR$$

(12)

where the number of missed targets is denoted by $FN$, the number of correctly detected targets is $TP$, and $FP$ indicates the number of falsely detected targets.

4.3. Experiment Details

The experiments in this paper were conducted within a network built on PyTorch 2.1.0, running on a Windows 11 system. YOLOv8n was utilized as the baseline model. After multiple rounds of experimental validation, we set the number of training epochs to 150 and the batch size to eight. The optimization algorithm used was SGD, and the initial rate of learning was 0.0025. The hardware configuration included a 13th Gen Intel i5-13400F CPU (2.50 GHz), an NVIDIA RTX 4060 GPU with 8 GB of video memory, and 64 GB of RAM.

4.4. Comparison with Other Classical Methods

To evaluate the effectiveness of R-SABMNet, this paper selects nine popular directional detection methods for comparison on the SSDD+ dataset. As shown in

Table 2. Comparison of detection results from different models on SSDD+ dataset.

	Module	Precision (%)	Recall (%)	AP (%)
Two-Stage	R-FasterRCNN	90.74	91.49	89.62
	O-RCNN	92.83	91.36	90.15
	ROI	88.42	89.28	87.65
One-Stage	R-FCOS	83.55	87.73	82.91
	R3Det	84.41	85.64	83.72
	R-RetinaNet	86.56	88.19	87.61
	R-YOLOv7	83.55	87.73	82.91
	R-LRBPNet	94.93	92.54	94.86
	R-YOLOv8 (baseline)	94.17	91.34	94.06
	Ours	96.32	93.13	95.28

, the comparison results are summarized.

Our comparison experiments include both classical two-stage [53,54] and single-stage detectors [55,56]. In addition, we selected the latest YOLOv8-based oriented ship detection model, R-LRBPNet [57], as a comparison model to validate the effectiveness of the proposed approach. It integrates advanced components such as bottleneck transformers, angle decoupling, and lightweight receptive field feature convolution to enhance feature extraction and detection accuracy. Among all detection models, the proposed R-SABMNet outperforms the others across all metrics. Compared to the baseline model in this experiment, R-SABMNet improves accuracy by 2.15%, recall by 1.79%, and AP by 1.22%. Furthermore, compared to R-LRBPNet, the proposed network’s SG-AFA module significantly enhances the model’s sensitivity to ship targets and its capability to represent key regions. As a result, our model achieves a notable improvement in accuracy, outperforming R-LRBPNet by 1.39%. Additionally, by representing oriented bounding boxes as two-dimensional Gaussian distributions, GWD effectively mitigates challenges caused by angular and scale inconsistencies, leading to a 0.59% increase in recall for our model.

To further demonstrate the effectiveness of R-SABMNet, this paper visualizes the results from different methods, as shown in Figure 10. These visualizations clearly demonstrate the ship detection capabilities of each method in complex scenarios. From the comparison, it is evident that our method has a significant advantage, while the other models show varying degrees of false alarms and missed detections. Specifically, in Scene 1, two land areas are mistakenly detected as ships by O-RCNN. The same is the case with Scene 2: sea surface clutter and port facilities are incorrectly identified as ships by O-RCNN, R-YOLOv7, and even R-YOLOv8. In contrast, our method detects all targets correctly, with no false positives or missed detections. This is attributed to the SG-AFA module, which suppresses land scattering interference, enhancing the model’s sensitivity to ship targets and its ability to express key regions. Additionally, a ship on Scene 4’s right side is missed by YOLOv7. Additionally, in Scene 4, dense ships lead to redundant detection boxes by the R-YOLOv8. By contrast, our method performs better, which can be attributed to the GWD, which effectively solves the problem of localization error due to angle and scale inconsistency in dense scenes. In summary, the comparison results clearly confirm that our algorithm achieves superior detection performance.

Since the HRSID provides higher image quality and includes more targets than the SSDD+ dataset, it is well suited for complex target detection tasks. To evaluate the generalization and robustness of R-SABMNet, this paper conducts the same comparison experiments on this dataset. The comparison results in

Table 3. Comparison of detection results from different models on HRSID dataset.

	Module	Precision (%)	Recall (%)	AP (%)
Two-Stage	R-FasterRCNN	80.62	81.04	77.87
	O-RCNN	85.35	84.61	80.2
	ROI	84.19	82.48	78.76
One-Stage	R-FCOS	78.77	74.91	73.15
	R3Det	79.86	75.29	77.45
	R-RetinaNet	83.72	81.45	80.23
	R-YOLOv7	89.6	85.37	84.38
	R-LRBPNet	91.35	87.59	88.74
	R-YOLOv8 (baseline)	90.72	86.35	87.8
	Ours	92.56	89.43	90.69

show that R-SABMNet far exceeds all competing methods in overall performance. Specifically, our model achieves an accuracy improvement of 1.84%, a recall increase of 3.08%, and a 2.89% enhancement in AP compared to the baseline R-YOLOv8. Similarly, compared to R-LRBPNet, a model also improved from YOLOv8, our model achieves higher precision (92.56% vs. 91.35%), recall (89.43% vs. 87.59%), and AP (90.69% vs. 88.74%) on the HRSID dataset. These improvements further confirm the effectiveness and generalizability of R-SABMNet across different datasets.

To further validate the effectiveness of the model, we selected densely arranged and irregularly oriented scene images for inference testing on the HRSID dataset. Meanwhile, we selected YOLOv8 and R-LRBPNet as comparison methods to verify the improvements of our approach over YOLOv8. Comparative experiments were conducted, with a focus on highlighting the angular differences of the rotated detection boxes. The experimental results are shown in Figure 11. Unlike previous comparisons, this result comparison includes P cases, where the number of rotated bounding boxes is incorrect or the rotation angles differ significantly, highlighted with purple boxes in the figure.

As shown in subfigures (a–d), both the baseline and R-LRBPNet exhibit angular and area errors in the detection results of adjacent targets. In contrast, our method incorporates GWD, which provides a more accurate distance metric for rotated bounding boxes, thereby reducing the errors in RIoU and rotation angles calculations and improving the model’s recall. Subfigures (e–h) illustrate densely packed ship regions with irregular orientations. Due to the high target density, YOLOv8 suffers from significant missed and false detections, with R-LRBPNet performing similarly. In comparison, our model not only substantially reduces both missed and false detections but also mitigates large angular errors present in the baseline. These results demonstrate that by integrating the GWD module, our model optimizes the computation of the intersection-over-union for rotated bounding boxes, effectively reducing false and missed detections, thereby enhancing both detection accuracy and recall.

4.5. Ablation Studies

To thoroughly validate the function of each improvement of R-SABMNet, ablation studies were conducted. The original R-YOLOv8 model serves as the baseline. We conducted three ablation experiments for both quantitative and qualitative analysis. All experiments were conducted under consistent conditions with identical parameter settings to ensure the reliability and validity of the results. We examined the effects of three improvements combined with the baseline model: the SG-AFA module, the BSMF module, and the GWD function.

Table 4. Results of ablation experiments on SSDD+ dataset.

Experiment	SG-AFA	BSMF	GWD	P (%)	R (%)	AP (%)
1	-	-	-	94.17	91.34	94.06
2	✓	-	-	95.64	91.96	94.71
3	✓	✓	-	96.21	91.99	94.83
4	✓	✓	✓	96.32	93.13	95.28

displays the results. When the SG-AFA module is added to the baseline, precision increases by 1.47%, recall improves by 0.62%, and AP rises by 0.65%. This improvement is due to the spatial feature aggregation, which enhances the model’s sensitivity to ship targets. Adding the BSMF module further enhances multi-scale feature fusion, increasing precision by 0.64%. Additionally, incorporating the GWD loss function for regression addresses the size, shape, and orientation of rotating boxes. This improvement boosts recall from 91.99% to 93.13%, as shown in row 4 of Table 4.

4.5.1. Effect of SG-AFA Module

Due to the strong similarity in grayscale values and texture features between harbor infrastructure and ship hulls in SAR images, significant interference is introduced. This makes it difficult for the original model to accurately extract ship features, leading to misdetections. To address this, we integrate the SG-AFA module into the baseline model. It enhances feature representation in key regions by incorporating spatial location information, enabling the model to effectively distinguish ship targets while suppressing land scattering interference. As a result, ship detection accuracy improves in cases with complex backgrounds.

We evaluated the performance of the proposed method in complex scenarios, and the comparison results are shown in Figure 12. Subplots (a–c) represent the intersection of nearshore and ocean regions, a scene characterized by both land scattering interference and severe ocean clutter. As shown in subplot (b), YOLOv8 falsely detected two port facilities with scattering characteristics similar to ship targets, along with two significant ocean noise regions that were misclassified as targets. This could be attributed to the lack of mechanisms for capturing the spatial characteristics of ships, leading to severe interference from land scattering and ocean noise in the detection results. With the addition of the SG-AFA module to the baseline model, the model effectively enhanced the representation of key regions by adaptively aggregating spatial features, thus improving detection accuracy. As shown in subplot (c), the improved model accurately identified all targets with zero false positives. Subplots (d–f) depict a nearshore port scene where severe land scattering interference is present. In subplot (e), the baseline model demonstrates a high sensitivity to land scattering, resulting in some false positives. However, the improved detection results, as shown in subplot (f), reveal a significant reduction in false positives in areas with dense land interference. These experimental results indicate that the SG-AFA module not only mitigates the impact of land scattering but also enhances the representation of key features in complex nearshore environments, further validating the effectiveness of the module.

4.5.2. Effect of BSMF Module

During deep feature extraction in convolutional networks, a significant amount of information from small targets in SAR images is lost, leading to missed detections. To deal with this, we proposed the BSMF module to be added into the baseline model during the feature fusion stage. This module enhances the information flow between features of different scales, reduces shallow feature loss, and improves feature fusion in the FPN by capturing global dependencies across spatial locations. This ultimately boosts the performance of YOLOv8.

Figure 13 demonstrates the performance of the proposed method in multi-scale ship detection scenarios. Subfigures (a–c) present the detection results of densely distributed ships in nearshore scenes, while subfigures (d–f) display the detection results of multi-scale ships docked near port facilities. As illustrated in subfigures (b,e), due to the dense arrangement of ships and the severe interference from port facilities, the baseline model exhibits missed detections of targets and false detections caused by land scattering interference. This highlights the baseline model’s insufficient sensitivity to variations in multi-scale features. The limitations become particularly evident when the spatial features of ships are obscured by surrounding ships or complex backgrounds.

After integrating the BSMF module, the model’s sensitivity to multi-scale features is significantly improved. As shown in Figure 13f, the improved method successfully detects ships that were previously missed. Furthermore, as illustrated in Figure 13c, false detections caused by land scattering interference are also reduced. These results demonstrate that the BSMF module effectively preserves shallow features and combines them with deep features. By incorporating the fusion-balanced shifted cross-window multi-head attention mechanism, the module substantially reduces both missed and false detections, further validating its effectiveness in enhancing overall detection performance.

4.5.3. Effect of GWD

Ships in SAR images are typically not fixed in direction and are densely arranged, making the calculation of RIOU between two rotating frames in OBB detection a complex task. The backpropagation process cannot efficiently compute the gradient, resulting in suboptimal model optimization and increasing the likelihood of misdetections and omissions. The following issues often arise: (1) in dense ship scenarios, adjacent ship targets are misclassified as a single target, as shown by the blue box in Figure 14e; (2) redundant detection frames are generated when ships are closely arranged, as shown in Figure 14b. To address these issues, we introduce the GWD into our method. This function measures the difference between rotated frames using the Wasserstein distance of corresponding Gaussian distributions. It effectively handles the size, shape, and orientation of rotated bounding boxes while improving the performance and efficiency of the detection model.

As shown in Figure 14, we evaluated the performance of the proposed method in scenarios where ships are densely arranged and oriented irregularly. Figure 14a,b depict the ground truth of ship detection in two port scenarios with densely arranged ships. From the comparison results, it is evident that the baseline method exhibits significant limitations in such complex scenarios. Specifically, in Figure 14b, redundant detection boxes are concentrated near densely distributed ships, while Figure 14e demonstrates merged detections where a single bounding box covers multiple distinct ships. These issues highlight the baseline method’s inability to distinguish spatially proximate targets and its limited capacity for optimizing rotated bounding boxes. In contrast, as shown in Figure 14c,f, the proposed method significantly mitigates these problems. Redundant detection boxes are effectively eliminated, and densely arranged ship targets are accurately separated. This improvement can be attributed to the incorporation of GWD, which calculates the differences between rotated bounding boxes based on the Wasserstein distance of their corresponding Gaussian distributions. GWD enables precise modeling of the geometric and spatial characteristics of ships, allowing the network to capture subtle distinctions between overlapping or densely arranged targets. The experimental results validate the superiority of the proposed method in detecting densely arranged ships, demonstrating its capability to improve the robustness and detection efficiency of the model in complex SAR image scenarios.

4.6. Generalization Ability Test

To assess the generalization performance of R-SABMNet, we selected a Gaofen-3 SAR image captured in 2022 near the southern mouth of the Yangtze River. The image, with a resolution of 1 m, is located at approximately $(122.{20}^{°}\mathrm{E},29.{95}^{°}\mathrm{N})$, in an archipelago off the northeastern coast of Zhejiang Province, China. We use the weights trained on the SSDD+ dataset to test the method’s generalization ability. First, two representative regions were selected from this image for examination, as shown in Figure 15. Region 2 covers a mixed nearshore and offshore scene, while Region 1 represents an offshore scene. The effectiveness of the R-SABMNet is compared to the baseline. The detection results of the baseline model are displayed in Figure 15A,B, located in the upper left and right corners, respectively. In contrast, Figure 15C,D in the lower left and right corners display the detection results of our method.

In Region 1, where a large number of ship targets are distributed in an offshore scene, the YOLOv8 model misses five ship targets, illustrated in Figure 15A. In contrast, R-SABMNet accurately detects all the ships, achieving a high detection accuracy. Furthermore, in Region 2, the complex port area exhibits densely arranged ships and facility scattering interference. The baseline model generates several missed detections and two false positives in land areas, as seen in Figure 15B. In comparison, R-SABMNet reduces false positives, effectively suppresses interference from land areas, and significantly improves detection performance, with a substantial reduction in missed targets, as depicted in Figure 15D. This also highlights the superior ability of our method to handle land scattering interference.

4.7. Comparison of Model Inference Speed

The above experiments effectively validate that the proposed model achieves promising detection performance. However, computational complexity and runtime are equally critical for practical deployment. To further assess the practicality and feasibility of the proposed R-SABMNet, experiments were conducted across different hardware platforms.

The experimental platforms included the RTX 4060 GPU and a 13-inch MacBook Pro. The RTX 4060, featuring 3072 CUDA cores, 8 GB of GDDR6 memory, and a floating-point performance of over 20 TFLOPS, excels in high-performance tasks. In contrast, the 13-inch MacBook Pro, with its M2 GPU and 8 GB of unified memory, delivers a lower floating-point performance of 3.6 TFLOPS, but offers enhanced integration and efficiency. These hardware differences influence the models’ performance across platforms.

In addition, YOLOv8 (baseline) and YOLOv11 were selected as comparison models to evaluate the inference speed of each model on various hardware platforms, thereby validating the inference speed advantage of our model. In the model inference process, the total time is divided into three main components: pre-process (Pre), inference (Infer), and post-process (Post). We conducted comparative experiments on two different devices to observe the variations across these three metrics. The results are shown in

Table 5. Comparison of different models’ inference speed across different hardware platforms.

Model	MacBook Pro (13-Inch)			RTX4060 GPU
	Pre (ms)	Infer (ms)	Post (ms)	Pre (ms)	Infer (ms)	Post (ms)
YOLOv8	0.7	6.5	6.9	0.4	4.8	5.3
YOLOv11	0.8	6.3	6.0	0.5	3.9	4.2
R-SABMNet	0.8	6.7	5.3	0.5	3.6	3.7

.

Despite achieving superior accuracy, our model maintains inference speeds comparable to the YOLOv8 and YOLOv11 models. On both the MacBook Pro (13-inch) and RTX 4060 GPU, R-SABMNet shows similar inference times, achieving 3.6 ms (RTX 4060) and 5.3 ms (MacBook Pro) for inference, values which are very close to the YOLOv8 and YOLOv11 models. Notably, our model excels in post-processing speed, achieving 3.7 ms (RTX 4060) and 5.3 ms (MacBook Pro), while other models have higher post-processing times. This demonstrates that R-SABMNet achieves a balance between high accuracy and efficient inference speeds, making it both effective and fast in real-world applications.

5. Conclusions

This paper proposes a new oriented detection model, R-SABMNet, for ship target detection in SAR images. This network first utilizes a feature extraction module with the SG-AFA module to enhance sensitivity to ship features and improve the representation of key regions. It significantly suppresses the scattering interference from land areas. The BSMF module is then introduced to refine local detail and establish global dependencies, thereby enhancing the model’s adaptability to handle multi-scale targets. Additionally, the GWD is employed to improve the regression loss, enabling the computation of differentiable RIOU. This effectively addresses localization errors caused by inconsistencies in angle and scale in dense scenes. A series of comparative and ablation experiments conducted on the HRSID and SSDD+ datasets demonstrate the superior detection performance of the proposed method. The results show that R-SABMNet outperforms nine other methods. Furthermore, tests on Gaofen-3 satellite imagery validate the model’s strong generalizability.

Although the experiments in this study are based on YOLOv8, the modular design ensures that the proposed approach can be flexibly integrated into other network architectures, such as ResNet, Swin Transformer, and other YOLO models. For instance, the SG-AFA module focuses on the unique characteristics of SAR images, such as edge contour enhancement and background noise suppression, significantly improving the representation of ship targets during the feature extraction stage. Meanwhile, the BSMF module incorporates a balanced offset multi-head attention mechanism, which optimizes the capture of local details and effectively models global dependencies during the multi-scale feature fusion stage. This enhances the model’s performance in multi-scale target detection. In future work, we will explore integrating the proposed approach into various backbone networks to further investigate improvements in detection performance and computational efficiency. Special attention should be given to addressing compatibility issues related to feature map resolution and optimizing computational efficiency to accommodate diverse practical application scenarios.