Oriented SAR Ship Detection Based on Edge Deformable Convolution and Point Set Representation

Abstract

Ship detection in synthetic aperture radar (SAR) images holds significant importance for both military and civilian applications, including maritime traffic supervision, marine search and rescue operations, and emergency response initiatives. Although extensive research has been conducted in this field, the interference of speckle noise in SAR images and the potential discontinuity of target contours continue to pose challenges for the accurate detection of multi-directional ships in complex scenes. To address these issues, we propose a novel ship detection method for SAR images that leverages edge deformable convolution combined with point set representation. By integrating edge deformable convolution with backbone networks, we learn the correlations between discontinuous target blocks in SAR images. This process effectively suppresses speckle noise while capturing the overall offset characteristics of targets. On this basis, a multi-directional ship detection module utilizing radial basis function (RBF) point set representation is developed. By constructing a point set transformation function, we establish efficient geometric alignment between the point set and the predicted rotated box, and we impose constraints on the penalty term associated with point set transformation to ensure accurate mapping between point set features and directed prediction boxes. This methodology enables the precise detection of multi-directional ship targets even in dense scenes. The experimental results derived from two publicly available datasets, RSDD-SAR and SSDD, demonstrate that our proposed method achieves state-of-the-art performance when benchmarked against other advanced detection models.

1. Introduction

Ships are vital tools and carriers in maritime activities, playing a crucial role in ensuring maritime safety and facilitating maritime transportation. Synthetic aperture radar (SAR) serves as a primary data source for maritime ship detection, offering continuous operation under adverse conditions such as cloud cover or nighttime, along with high-resolution imaging capabilities [1,2,3,4,5]. The development of a reliable SAR ship detection method not only enhances the real-time monitoring capabilities of maritime traffic conditions, illegal fishing, and smuggling activities but also provides essential technical support to safeguard national maritime rights and interests, as well as for maritime law enforcement and rescue operations [6,7,8,9]. Consequently, research on SAR image-based ship detection methods holds significant practical value in maintaining marine security and promoting the sustainable use of marine resources.

Although researchers have conducted extensive studies on ship detection in SAR images and proposed numerous detection models [10,11,12], the technology for detecting ships in SAR imagery still encounters several challenges [13,14,15]. The imaging process is influenced by various factors, including imaging geometry, sensor parameters, and electromagnetic wave scattering. Consequently, the resulting images often exhibit significant speckle noise, leading to blurred or incomplete target edges. The shapes of ships within these images can appear distinctly discrete and intermittent, posing substantial difficulties for feature extraction. Furthermore, ships in SAR images display characteristics such as multi-directionality, multi-scale variations, and proximity to port facilities. These factors can easily lead to deviations in the predicted bounding boxes generated by detection algorithms, resulting in missed detections and false alarms, as well as failing to meet the requirements for the precise localization of densely arranged ship targets within ports and maritime environments. Figure 1 illustrates two representative scenarios of SAR ship detection, where the green boxes denote the ground-truth, the yellow circles indicate potential false alarms, and the red circles represent possible missed detections. The ships depicted on the left in Figure 1a exhibit a discontinuity in target representation, whereas the ship targets shown in Figure 1b are presented from multiple angles at various scales and are densely arranged. Due to the aforementioned challenges, it is imperative to develop an effective strategy for extracting ships from SAR images and to establish a reliable correlation between target features and algorithmic prediction boxes.

To address these challenges, we propose a ship detection method for SAR images that leverages edge deformable convolution and point set representation. Specifically, to tackle the issues of discontinuous characteristics and speckle noise interference in target blocks within SAR images, we integrate the strengths of traditional convolution with deformable convolution techniques. This leads to the introduction of edge deformable convolution, which facilitates an exploration of the correlations among discontinuous target blocks in SAR imagery. Our approach enables the learning of overall deformation characteristics while effectively suppressing speckle noise, resulting in a preliminary feature map. Building upon this foundation, we present a multi-directional ship detection model based on radial basis function (RBF) point set representation. By constructing an RBF point set transformation function, we establish efficient geometric alignment between the point set and the predicted rotation box. Furthermore, we impose constraints on the penalty term associated with point set transformation to ensure accurate mapping between point set features and directed prediction boxes. This methodology achieves the precise detection of multi-directional ship targets even in dense scenes. The main contributions of this work can be summarized as follows:

• An SAR ship detection framework via edge deformable convolution and point set representation is proposed, which can achieve the accurate detection of densely arranged multi-directional ships in port scenes.
• A feature extraction module based on edge deformable convolution is proposed, which explores the correlation between discontinuous target blocks in SAR images, suppressing speckle noise while learning the overall deformation features of ship targets.
• The RBF point set transformation function and the associated point set transformation penalty term are introduced, establishing efficient and accurate mapping between point set features and predicted rotation boxes. This approach enables the precise detection of ship target direction and position in complex scenes.

The subsequent sections of this paper are structured as follows: Section 2 gives a thorough analysis of work related to the topic; Section 3 elaborates on the proposed methodology; Section 4 presents the experimental outcomes; and Section 5 is the conclusion.

2. Related Work

2.1. Feature Extraction of Ship Targets in SAR Images

In SAR images, methods for ship feature extraction can be systematically categorized into three primary approaches: statistical feature-based methods, frequency-domain analysis, and deep learning techniques. Statistical methods exploit differences in gray-level distributions, textural patterns, and gradient characteristics between targets and backgrounds. A representative algorithm within this category is the gray-level co-occurrence matrix (GLCM), which quantifies textural features through metrics such as energy, contrast, and correlation. Khesali et al. [16] enhanced this approach by employing variable window sizes to compute co-occurrence matrices, thereby capturing spatial relationships between pixels while analyzing gray-level distributions. Hanbay et al. [17] further improved feature discrimination by integrating Gaussian derivative filters to calculate gradient magnitudes for GLCM construction.

Frequency-domain methods offer significant advantages in computational efficiency and background suppression, primarily through the application of Fourier transforms and wavelet analysis. For instance, He et al. [18] developed an automatic ship detection system utilizing a modified hypercomplex Fourier transform (MHFT) saliency model, which effectively suppresses clutter while preserving target signatures. Wavelet-based approaches excel in multi-scale feature extraction, as demonstrated by Wu et al. [19], who proposed a wavelet-driven feature enhancement network that incorporates wavelet cascade residual (WCR) modules to improve detection capabilities in complex backgrounds.

Recent advancements have increasingly focused on end-to-end deep learning frameworks. Convolutional neural networks (CNNs) are at the forefront of this domain, and Yang et al. [20] proposed a lightweight CNN architecture featuring split bidirectional feature pyramids (S-BiFPN) to optimize multi-scale feature representation. Li et al. [21] introduced an attention-guided balanced feature pyramid network (A-BFPN) with enhanced refinement modules (ERMs) to learn channel-dependent and spatial-dependent features more effectively. Further contributions by Zhang et al. [22] advanced this area by proposing a quadruple feature pyramid network (Quad-FPN) to strengthen SAR ship feature extraction. In addition to CNNs, Generative Adversarial Networks (GANs) have emerged as a significant research frontier, particularly in applications related to data augmentation and domain adaptation. Song et al. [23] addressed cross-modal discrepancies by employing a domain transfer adversarial metric (DTAM) network, which integrates adversarial training with metric learning techniques to enhance feature extraction in SAR ship imagery.

Compared with traditional methods, deep learning networks have significant advantages in feature extraction. Unlike other deep learning networks, we propose an edge deformable convolution approach to extract ship features from SAR images. By flexibly adjusting the position and distribution of the convolution kernel sampling points, we improve the adaptive ability of SAR image ship feature extraction and accurately capture the overall contour information of ships, effectively solving problems such as speckle noise and the segmentation of ship targets into multiple discontinuous feature blocks in the image.

2.2. Ship Detection Method in SAR Images

In the field of SAR ship detection, detection methods can be categorized into traditional algorithms and deep learning-based approaches. Traditional SAR ship detection relies on handcrafted feature extraction, with key steps including image preprocessing, candidate region extraction, target recognition, and post-processing. Due to the distinct scattering distributions of ships and sea backgrounds, constant false alarm rate (CFAR) detection is widely employed. For instance, Leng et al. [24] proposed a bilateral CFAR algorithm that integrates the intensity and spatial distributions of SAR images to mitigate the effects of SAR ambiguity and sea clutter, introducing a novel kernel density estimation method for spatial distribution modeling. Gao and Shi [25] developed a statistical model for the Geometric Perturbation-Polarization Notch Filter (GP-PNF), using the inverse gamma distribution to characterize sea clutter texture and enabling CFAR detection. Later, Li et al. [26] introduced an adaptive CFAR algorithm based on fused intensity–texture features and an attention contrast mechanism, enhancing detection performance in complex scenarios through local feature enhancement and generalized gamma distribution modeling. Despite its widespread use, CFAR exhibits limitations in complex sea conditions, low signal-to-noise ratios, and small target detection.

With advances in deep learning, data-driven SAR ship detection methods have emerged as a research focus. Li et al. [27] pioneered the application of deep learning in SAR ship detection by refining feature fusion in Faster R-CNN and incorporating transfer learning and hard negative mining to improve performance. Chai et al. [28] enhanced the Cascade R-CNN algorithm for small ship detection in complex backgrounds by introducing a multi-scale backbone network and a spatial enhancement module, achieving higher accuracy for densely distributed targets. While two-stage detectors excel in performance, their serial processing of region proposal generation and classification–regression leads to high computational costs, limiting real-time applicability. In contrast, single-stage networks, with their end-to-end architecture, are better suited for real-time applications. Zhou et al. [29] improved the Single Shot MultiBox Detector (SSD) by redesigning anchor box aspect ratios and leveraging saliency maps to augment training data, boosting performance in few-shot scenarios. Guo et al. [30] proposed a lightweight convolutional unit and an adaptive spatial feature fusion module for YOLO (You Only Look Once) to enable multi-scale feature fusion.

Anchor-based methods often struggle with the diverse scales and complex backgrounds in remote sensing imagery. Anchor-free approaches address this by directly regressing target locations, eliminating predefined anchor constraints and adapting flexibly to targets of varying scales, shapes, and densities. For example, Guo et al. [31] localized ships via center points and predicted their size and shape, reducing background interference. Chen et al. [32] proposed the anchor-free box detection method SAD-Det based on Transformer and adaptive features. Long-range dependency and context modeling are enhanced by pooling the pyramid Transformer backbone in ship space, and multi-scale feature fusion is optimized by combining the adaptive feature pyramid neck network. Additionally, a deformable detection head is designed to dynamically adapt to the spatial sampling position of the target, achieving optimal detection performance on the SAR ship detection dataset, significantly improving the robustness and accuracy of multi-directional ships in complex scenarios. Shen et al. [33] proposed an efficient and lightweight network, ELLK-Net, for SAR ship detection. It fuses global features and long-range dependencies through large kernel convolution decomposition and the adaptive multi-scale attention mechanism (LKMA), and it optimizes the reasoning speed by combining the structural reparameterization technique. It provides a high-precision and low-latency solution for edge computing deployment in complex SAR scenarios. Li et al. [34] proposed a lightweight edge feature perception and fusion method, combining the multi-scale channel attention module (EFA) and the pruned optimized bidirectional feature pyramid (FP-BiFPN), achieving efficient model compression through an adaptive bit-width selection quantization strategy. The detection accuracy and reasoning speed are balanced. Zhao et al. [35] introduced a supervised guidance paradigm using Kullback–Leibler Divergence (KLD) loss to integrate the structural and positional attributes of scatter points in RepPoints.

Unlike existing methods, we propose a point set-based multi-oriented ship detection model. By adaptively transforming point sets, our approach learns and regresses ship contours or keypoint layouts, enabling the accurate detection of densely arranged, multi-oriented ships in complex scenarios.

3. Methodology

3.1. Method Overview

A flowchart of the proposed method is shown in Figure 2. Similar to the classic Oriented Reppoints model [36], the proposed method consists of two parts. In order to achieve the accurate detection of ship targets in SAR images, we propose a ship detection model based on edge deformable convolution (EDConv) and point set representation. It mainly consists of two parts: a feature extraction module based on EDConv and a multi-directional ship detection module based on point set transformation. Through the first module, we can explore the correlation between discontinuous target blocks in SAR images and learn the overall deformation features of the target while suppressing speckle noise, thus generating preliminary feature maps. On this basis, in order to improve the detection performance of densely arranged ships, a multi-directional ship detection model based on point set transformation is proposed. By establishing mapping relationships between point sets and prediction boxes and building loss constraints, high-probability and low false alarm detection of targets in complex scene images can be achieved.

3.2. Feature Extraction Module via Edge Deformable Convolution

Due to the limitations of SAR imaging systems and resolution, ship targets typically exhibit strong local reflections and incomplete overall contours in SAR images. Moreover, a ship may appear as multiple discontinuous feature blocks in the image. These characteristics can easily lead to missed or false detections by ship detection algorithms. The limited local structure perception capability of typical convolution operations makes it difficult to effectively capture and extract highly discrete and fragmented local features, thereby restricting further improvements in detection performance.

To address these issues, we propose an edge deformable convolution (EDConv) structure to enhance the backbone network’s ability to focus on local target features. By organically integrating the advantages of both standard and deformable convolutions, EDConv enables the flexible and efficient extraction of local target features, thereby significantly improving feature representation quality. A schematic diagram of the proposed edge deformable convolution structure is illustrated in Figure 3.

For a standard 2D convolution with a 3 × 3 kernel, the computation formula for each position $p_0$ on the output feature map is given by

$$y\left(p_0\right)=\sum _{\begin{array}{c}{p}_n\in {\mathcal{R}}_s\end{array}}w\left(p_n\right)·x(p_0+p_n)$$

(1)

where $p_n$ enumerates the positions in $R=\left\{\right(-1,-1),(-1,0), \dots ,(1,0),(1,1\left)\right\}$.

To enhance the convolution kernel’s ability to flexibly perceive the complex geometric features of ships, we draw inspiration from deformable convolutions [37] and introduce deformation offsets. In traditional deformable convolutions, the deformation offsets allow the kernel to freely adjust within the entire receptive field. While this fully flexible deformation mechanism improves model adaptability, it may also cause the receptive field to deviate from the target region, especially when the target’s weak features are unevenly distributed, thereby affecting the effectiveness of feature extraction. To better capture the local features of ships, we propose a hybrid strategy that combines standard convolution and deformable convolution.

Specifically, we design an edge deformable convolutional kernel structure consisting of both standard and deformable convolution components. In the central cross-shaped region of the kernel, we retain standard convolution operations to ensure the stable linear extraction of local cross-shaped features from the input image. In contrast, the remaining regions of the kernel are assigned deformable convolution properties, enabling them to flexibly capture the geometric features of targets through adaptive spatial transformations. This design not only enhances the perception of local target features but also mitigates the potential divergence of the receptive field caused by fully flexible deformations.

In EDConv, we consider a 3 × 3 convolution kernel and introduce offsets $\Delta p$ at its four corner positions. In other words, we sample at both regular positions $p_i$ and irregular offset positions $p_j+\Delta p_j$. This calculation process can be characterized as

$$y\left(p_0\right)=\sum _{p_i\in {\mathcal{R}}_s}w\left(p_i\right)·x(p_0+p_i)+\sum _{p_j\in {\mathcal{R}}_d}w\left(p_j\right)·x(p_0+p_j+\Delta p_j)$$

(2)

where ${\mathcal{R}}_s$ represents the central cross-shaped region, and ${\mathcal{R}}_d$ denotes the edge region. The term $\Delta p_j$ represents the learnable offset at the j-th position. To prevent excessive displacement, we constrain $\Delta p_j$ within a predefined range of [−2, 2]. Since $K=p_0+p_n+\Delta p_j$ is generally not an integer, the computation of $x\left(K\right)$ is performed using bilinear interpolation:

$$x\left(K\right)=\sum _{K^{\prime }}B\left(K^{\prime },K\right)·x\left(K^{\prime }\right)$$

(3)

where $K^{\prime }$ enumerates all integer spatial positions in the feature map x, and B represents the bilinear interpolation kernel, which can be decomposed into two one-dimensional kernels:

$$B(K^{\prime },K)=b(K_{x^{\prime }},K_x)·b(K_{y^{\prime }},K_y)=max\left(0,1-\left|K_{x^{\prime }}-K_x\right|\right)·max\left(0,1-\left|K_{y^{\prime }}-K_y\right|\right).$$

(4)

As illustrated in Figure 3, to generate offsets, we introduce an additional convolution layer that takes the input feature map and outputs offsets with the same spatial resolution. The channel dimension of the output offsets is $2k^{\prime }$, corresponding to a total of $k^{\prime }$ two-dimensional offsets in both the x and y directions. These offsets are then weighted by a channel attention mechanism, followed by batch normalization and tanh activation function processing to obtain the normalized output offsets. This computation not only ensures that the offset values remain within a controlled range but also enhances the model’s adaptability to discontinuous local features of the target.

After implementing the EDConv convolution operation, we mainly apply it to the feature extraction stage of SAR ship detection models to enhance the network’s ability to extract intermittent local features of ship targets in SAR images. Specifically, EDConv is applied to multiple stages of the detection model backbone network. In each stage, the input features are first split, and then one branch performs feature extraction through bottleneck and standard convolution calculations. The other branch is processed by EDConv to adaptively adjust the receptive field and achieve the flexible and efficient modeling of local discontinuous features of the ship. Finally, the features of the two branches are weighted and fused at the element level to obtain the output features of this stage.

It should be noted that we embed this module into the third, fourth, and fifth layers of the ResNet extraction branch, mainly because deep features emphasize the completeness of semantic information and context dependence, making it easier to capture the large-scale structure and potential location distribution of the target. EDConv can optimize the modeling of target deformation and scale changes in these branches.

Through the above modules, we can enable the extraction branches to learn discontinuous target feature blocks and output feature maps for the subsequent detection head.

3.3. Multi-Directional Ship Detection Module via Point Set Transformation

After processing the previous module, we can obtain a feature map of ship targets. Given that ships may be densely arranged in port scenes, to enhance target representation and minimize missed detections, we propose a multi-directional ship detection module based on point set transformation, which can capture the shape and direction changes of densely arranged ships more flexibly.

The point set representation-based method enables the direct learning and regression of polygonal contours or keypoint layouts for ships. This approach not only captures the shape features and docking angles of various ships but also enhances localization accuracy for complex ship outlines. Unlike classical point set representation methods, traditional point sets often lack clear semantic information, such as bounding boxes and masks, due to their discrete nature. This limitation frequently results in issues related to non-convergence and susceptibility to outliers when attempting to translate into target locations or regions. To address these challenges, we propose an RBF (radial basis function) point set transformation function that facilitates efficient geometric alignment between the point set and the predicted rotation box. Furthermore, a penalty term is introduced for point set transformation to ensure accurate mapping between the features of the point set and directed prediction boxes. The proposed module for transforming point sets primarily consists of four steps.

(1) Point set representation

The point set representation method achieves refined feature extraction by generating point sets on the surface or edges of objects, enabling a more precise capture of local structures and orientations of ships than traditional bounding boxes. Given a feature map $X\in {\mathbb{R}}^{H\times W\times C}$, a deformable convolutional network (DCN) is dynamically employed to generate a point set comprising adaptive sample points $R={\left\{(x_k,y_k)\right\}}_{k=1}^n$, where $n=9$ is specifically designed to align with a 3 × 3 convolution kernel. By iteratively refining the candidate point set positions using the predicted offsets $\Delta$ from the DCNs, this approach efficiently aligns semantic features with geometric deformations. This mechanism enhances both the precision and robustness of ship detection in remote sensing imagery.

(2) Point set transformation function

To map any shaped point set $P=\{p_1,p_2, \dots ,p_n\}$ to a standard oriented box while ensuring geometric consistency, we propose a nonlinear mapping model based on the radial basis function (RBF) to achieve efficient geometric alignment between point sets and oriented boxes.

First, we define an RBF mapping model to project input points into a new feature space:

$$\varphi \left(x\right)=\sum _{i=1}^k{\alpha }_i\rho \left({\displaystyle \frac{x-c_i}{r_i}}\right)$$

(5)

where $c_i$ represents the core point, $r_i$ denotes the kernel radius, ${\alpha }_i$ corresponds to the weight coefficient, and $\rho$ signifies the radial basis function. By adjusting the kernel parameters $\{c_i,r_i,{\alpha }_i\}$, the RBF network can learn the nonlinear mapping relationship between the input point set and the target ground-truth rotated bounding box.

According to the definition of the target rotated bounding box, its geometric properties are characterized by the center coordinate $c_0$, rotation angle $\theta$, and semi-axis lengths $w/2$ and $h/2$.

The coordinates of the predicted points in dataset P relative to the oriented bounding box must satisfy the following constraint equations:

$$\left\{\begin{array}{c}{(x-c_0)}^{\top }R\left(\theta \right)R{\left(\theta \right)}^{\top }(x-c_0)\le 1\hfill \\ \mathrm{where}R\left(\theta \right)=\left[\begin{array}{cc}cos\theta & -sin\theta \\ sin\theta & cos\theta \end{array}\right]\hfill \end{array}\right.$$

(6)

(3) Adaptive point set quality assessment metric

To address the issue of ground-truth annotation deficiency in point set representation learning, we propose an adaptive point set quality assessment framework based on multidimensional feature fusion. Since ground-truth annotated point sets remain directly inaccessible during training, conventional methods encounter significant challenges in quantifying the confidence levels of predicted point sets. This limitation directly impacts the model’s ability to efficiently acquire high-quality point set prediction capabilities. To overcome this fundamental obstacle, our approach constructs a comprehensive quality metric system encompassing three core evaluation dimensions.

• Classification quality assessment
  We measure the classification quality by utilizing the classification loss between the point set $R_i^{\mathrm{cls}}$ and the ground-truth $b_j^{\mathrm{cls}}$. The specific formula is as follows:

  $$Q_{\mathrm{cls}}(R_i,b_j)=L_{\mathrm{cls}}\left(R_i^{\mathrm{cls}}\left(\theta \right),b_j^{\mathrm{cls}}\right).$$

  (7)
• Localization quality assessment
  The localization quality is measured by the localization loss between the polygons generated from point sets $O{B}_i^{\mathrm{loc}}$ and the ground-truth $b_j^{\mathrm{loc}}$. The formula for localization quality is as follows:

  $$Q_{\mathrm{loc}}(R_i,b_j)=L_{\mathrm{loc}}\left(O{B}_i^{\mathrm{loc}}\left(\theta \right),b_j^{\mathrm{loc}}\right).$$

  (8)
  The location quality of a point set primarily focuses on the overlap extent between the polygon generated by the point set and the ground-truth while being insensitive to directional changes. This phenomenon is particularly evident in ship targets within remote sensing images.
• Oriented quality assessment
  We utilize the MinAeraRect point set transformation function to convert the predicted point set into oriented bounding boxes. Subsequently, we perform equidistant sampling along each edge of the oriented bounding boxes. Based on the sampled points, we compute the corner distance between the predicted box and the ground-truth box to determine the oriented quality of the point set. This process can be expressed as follows:

  $$Q_{\mathrm{ori}}(R_i,b_j)=CD\left(R_i^{\mathrm{v}}\left(\theta \right),R_{b_j}^{\mathrm{g}}\right)$$

  (9)

  where $CD$ denotes the chamfer distance between two point sets.

To sum up, based on classification quality $Q_{\mathrm{cls}}$, localization quality $Q_{\mathrm{loc}}$, and orientation quality $Q_{\mathrm{ori}}$, we derived an adaptive point set quality assessment metric:

$$Q=Q_{\mathrm{cls}}+{\mu }_1{Q}_{\mathrm{loc}}+{\mu }_2{Q}_{\mathrm{ori}}$$

(10)

where $\mu$ represents the weighted factor.

(4) Loss function

The ship detection algorithm based on point sets involves two optimization stages. In the network initialization stage, adaptive point sets are generated from the object centers in the feature map, and a set of point sets is allocated to each object. In the refinement stage, high-quality predicted point sets are selected as positive samples based on a sample allocation scheme derived from quality assessment metrics of the adaptive point sets, followed by further refinement adjustments. These two point set optimization processes are implemented by minimizing the following loss function:

$$L=L_{\mathrm{cls}}+{\lambda }_1{L}_{\mathrm{s}1}+{\lambda }_2{L}_{\mathrm{s}2}$$

(11)

where ${\lambda }_1$ and ${\lambda }_2$ denote the balance coefficients, $L_{\mathrm{cls}}$ represents the classification loss, and $L_{\mathrm{s}1}$ and $L_{\mathrm{s}2}$ denote the spatial localization losses during the network initialization phase and the refinement phase, respectively.

The calculation formula for the classification loss $L_{\mathrm{cls}}$ is as follows:

$$L_{\mathrm{cls}}={\displaystyle \frac{1}{N_{\mathrm{cls}}}}\sum _i{F}_{\mathrm{cls}}\left(R_i^{\mathrm{cls}}\left(\theta \right),b_j^{\mathrm{cls}}\right)$$

(12)

where $N_{\mathrm{cls}}$ denotes the number of point sets, $R_i^{\mathrm{cls}}$ denotes the classification confidence of the point set, and $b_j^{\mathrm{cls}}$ denotes the true class label.

Since both $L_{\mathrm{s}1}$ and $L_{\mathrm{s}2}$ terms are involved, they can be unified into a single formula. Here, $L_{\mathrm{loc}}$ represents the loss between the oriented bounding boxes derived from the transformation function and the ground-truth boxes.

$$L_{\mathrm{loc}}={\displaystyle \frac{1}{N_{\mathrm{loc}}}}\sum _i[b_j^{\mathrm{cls}}\ge 1]F_{\mathrm{loc}}\left({OB}_i^{\mathrm{loc}}\left(\theta \right),b_j^{\mathrm{loc}}\right)$$

(13)

where $N_{\mathrm{loc}}$ denotes the number of positive samples in the point set, $b_j^{\mathrm{cls}}$ represents the ground-truth located label, and $F_{\mathrm{loc}}$ signifies the GIoU loss.

Additionally, to optimize the RBF mapping model, we design a loss function that minimizes the distance from the mapped points to the bounding box boundaries.

$$L=\sum _{i=1}^{n}min\left({\displaystyle \frac{w}{2}}·\left|q_{ix}\right|,{\displaystyle \frac{h}{2}}·\left|q_{iy}\right|,{\displaystyle \frac{w}{2}}·\left|q_{iy}cos\theta +q_{ix}sin\theta \right|,{\displaystyle \frac{h}{2}}·\left|q_{iy}sin\theta -q_{ix}cos\theta \right|\right).$$

(14)

To ensure that the point set forms a convex polygon after transformation via the RBF mapping model, a combination of geometric constraint optimization and post-processing strategies is required. Specifically, during the optimization of RBF kernel parameters, convexity constraints are introduced by incorporating a convexity penalty term. This term calculates the average squared distance of the points to their convex hull, thereby enforcing the mapped point set to lie within the convex hull. The convexity penalty term is formulated as follows:

$$L_{\mathrm{convexity}}={\displaystyle \frac{1}{n}}\sum _{q\in \mathcal{Q}}\underset{h\in \mathrm{ConvexHull}\left(\mathcal{Q}\right)}{min}{\parallel q-h\parallel }^2.$$

(15)

The mathematical expression for incorporating a convexity penalty term in the optimization of the RBF mapping model can be represented as

$$L_{\mathrm{rbf}}=L+\lambda L_{\mathrm{convexity}}$$

(16)

where the $\lambda$ regularization parameter, used to control the strength of the convexity constraint, must be tuned based on the specific application scenario.

By combining the above losses and training the network, we can obtain the final ship detection result, achieving the accurate detection of targets in dense scenes.

4. Experiments

4.1. Datasets and Experimental Settings

To verify the effectiveness and superiority of the proposed method, we conducted extensive experiments on two publicly available datasets, RSDD-SAR [38] and SSDD [39]. Detailed information of the two datasets is shown in

Table 1. Detailed descriptions of SSDD and RSDD-SAR.

Dataset	SSDD	RSDD-SAR
Image number	1160	7000
Ship number	2587	10,263
Image size	190–668	$512\times 512$
Number of sensors	3	2
Resolution (m)	1–15	2–20
Frequency bands covered	$C,X$	$C,X$

.

The RSDD-SAR dataset consists of images from the Gaofen-3 satellite and the European Space Agency’s TerraSAR-X satellite. Specifically, this dataset consists of data from 127 scenes, namely, 84 scenes from Gaofen-3 data, 41 scenes from TerraSAR-X data slices, and 2 uncropped large images. It includes 7000 slices with multiple imaging modes, polarization modes, and resolutions, as well as 10,263 ship instances. A total of 7000 data samples were divided into 5000 training samples and 2000 test samples. The RSDD-SAR dataset has the characteristics of an arbitrary rotation direction, a large aspect ratio, a high proportion of small targets, and rich scenes. It covers features such as multi-scale, multi-azimuth, and dense distributions of targets, as well as backgrounds of different complexity levels, such as sea surfaces and ports.

SSDD is a small-scale SAR ship detection dataset. The data sources are Radarsat-2, TerraSAR-X, and Sentinel-1 satellites. The polarization modes are HH, VV, HV, and VH, and the image resolutions range from 1 m to 15 m. SSDD contains 1160 SAR image slices, with a total of 2587 ships. The ratio of the training set to the test set is 8:2, that is, 928 training samples and 232 test samples.

The experiments are conducted on a workstation equipped with 64 GB of RAM and a Quadro RTX 4090 GPU, utilizing the Pytorch framework. Each model is trained for 36 epochs, with a batch size of 4. We use stochastic gradient descent (SGD) with a weight decay of 0.0005 and a momentum of 0.9. Additionally, mosaic data augmentation is employed, which combines four training samples into a single one, reducing the requirement for a large mini-batch size. The intersection over union (IoU) value of the prediction box is 0.5.

4.2. Evaluation Metrics

In order to measure the performance of different detection models, classic metrics such as precision, recall, and average precision (AP) are selected to measure the overall effectiveness of the algorithm. Precision (P) measures the proportion of correct positive predictions among all detections identified by the model. It is defined as the ratio of true-positive detections to the total number of detections made by the detector.

$$\mathrm{P}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}}$$

(17)

where TP and FP represent the true positive and false positive, respectively.

Recall (R) measures the proportion of correctly identified positive instances out of all actual positive instances in the dataset.

$$\mathrm{R}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}}$$

(18)

where FN represents the false negative.

The AP is a widely used metric to evaluate the overall performance of object detection algorithms. Specifically, the AP is computed as the mean of the average precision (AP) values for one class, where the AP is obtained by calculating the area under the precision–recall curve:

$$\mathrm{AP}={\int }_0^{1}P\left(R\right)\mathrm{d}R.$$

(19)

4.3. Ablation Experiments

To demonstrate the contribution of the proposed modules, ablation experiments are first conducted on the RSDD-SAR dataset. For a fair comparison, the original Oriented Reppoints method is chosen as the baseline.

Table 2. Results of ablation experiment on the RSDD-SAR dataset. Bold indicates the optimal value.

Method	EDConv	Point Set Transformation	P (%)	R (%)	AP (%)
Baseline	×	×	90.24	89.12	89.48
	✓	×	92.10	90.09	90.23
	×	✓	93.22	90.23	90.74
Proposed	✓	✓	94.91	91.35	91.62

shows the experimental results of multiple image scenes on the RSDD-SAR dataset. It can be observed that, when Oriented Reppoints are directly used, the AP value obtained is 89.48%. Based on this situation, when the EDConv module is assembled on the original architecture, the AP value is 90.23%. When configuring the point set transformation module separately, the AP value is 90.74%. Furthermore, when two modules are configured simultaneously in the original network, the AP can reach 91.62%, an increase of 2.14% compared to the baseline network.

We also conducted ablation experiments on the SSDD dataset.

Table 3. Results of ablation experiment on the SSDD dataset. Bold indicates the optimal value.

Method	EDConv	Point Set Transformation	P (%)	R (%)	AP (%)
Baseline	×	×	92.25	88.33	88.40
	✓	×	93.47	90.40	90.66
	×	✓	95.02	91.70	91.85
Proposed	✓	✓	95.45	92.12	92.81

shows the ablation experiment results on this dataset. It can be observed that, when Oriented Reppoints are directly used, the AP value obtained is 88.40%. Based on this situation, when the EDConv module is assembled on the original architecture, the AP value is 90.66%. When configuring the point set conversion module separately, the AP value is 91.85%, an increase of 3.45%. Furthermore, when two modules are simultaneously configured in the baseline network, the AP can reach 92.81%, an increase of 4.41%. From the above results, it can be found that both of the proposed modules have positive gains in detection performance, and the gain of the point set transformation module is more significant than that of the EDConv module.

The results of the ablation experiment detection are illustrated in Figure 4. It is evident that the EDConv module plays a crucial role in minimizing false alarms, thereby demonstrating its effectiveness in interference suppression. Additionally, the point set transformation module significantly reduces missed detections and exhibits excellent detection performance for densely arranged ships.

4.4. Comparison Experiments

In order to comprehensively evaluate the effectiveness and advancement of the proposed method in applying images to different scenes, seven advanced object detection models are selected for comparative experiments, namely, Rotated-RetinaNet [40], Oriented Faster R-CNN [41], S2A-Net [42], R3Det [43], Oriented Reppoints [36], Rotated-RTMDet-s [44], and LSR-Det [45].

In order to quantitatively evaluate the performance of the proposed method in comparison with that of other algorithms, we present the quantitative index results for different methods on the RSDD-SAR and SSDD datasets in

Table 4. Results of comparison experiments on the RSDD-SAR dataset. Bold indicates the optimal value.

Method	P (%)	R (%)	AP (%)
Rotated-RetinaNet	89.77	84.30	85.22
Oriented Faster R-CNN	90.20	85.45	86.98
S2A-Net	89.82	86.45	88.37
R3Det	90.61	86.50	88.86
Oriented Reppoints	90.24	89.12	89.48
Rotated-RTMDet-s	91.60	88.50	90.16
LSD-Det	92.45	90.25	90.34
Proposed	94.91	92.35	91.62

and

Table 5. Results of comparison experiments on the SSDD dataset. Bold indicates the optimal value.

Method	P (%)	R (%)	AP (%)
Rotated-RetinaNet	89.35	83.52	85.58
Oriented Faster R-CNN	90.53	86.79	87.36
S2A-Net	88.80	90.39	89.78
R3Det	90.76	88.46	89.62
Oriented Reppoints	92.25	88.33	88.40
Rotated-RTMDet-s	92.43	89.94	90.65
LSD-Det	93.86	91.02	91.45
Proposed	95.45	92.12	92.81

, respectively. The performance distribution of these algorithms across both datasets is largely consistent, primarily due to the similarities in image resolution and scene types between the two datasets, which leads to comparable robustness in algorithm application. Specifically, Rotated RetinaNet exhibits the poorest overall performance, while S2A-Net and R3Det demonstrate performance that is similar to that of our proposed method. Additionally, LSR-Det’s performance is also close to that of our approach. Our method outperforms other algorithms across several indices. Based on an analysis of both the quantitative and qualitative results presented above, it is evident that our method achieves superior detection performance compared to alternative approaches and effectively identifies multi-directionally distributed ship targets with dense docking amidst complex port facility interference.

In order to visually demonstrate the applicability of the proposed method to various image scenes, we show the detection results of several typical port scene images in the RSDD-SAR dataset in Figure 5. In order to show the distribution of the prediction box more clearly, we do not display the confidence label of the prediction box. It can be observed in Figure 5 that the port scene is selected to cover the image of intensive docking of multiple ship targets in multiple directions, the interference of port facilities on the shore is complex and changeable, and the characteristics of some facilities are highly similar to those of ship targets. It should be noted that we use the green box to represent the true position of the target, the red box to represent the true-positive prediction box of the algorithm, the yellow box to represent the false detection of the algorithm, and the blue circle to indicate missed detections.

In the first scene of Figure 5, ships dock side by side near the port. The Oriented Faster R-CNN and Oriented Reppoints methods correctly detect most ships, but there are multiple false detections, mainly because the extraction of target features is not precise enough. It is difficult to effectively distinguish between targets and shore facilities. Although the overall detection result of LSR-Det is close to that of the proposed method, it fails to detect targets with occluded edges. However, our method uses the idea of point set to focus on and extract the features of important parts of ships, which can better detect targets with incomplete structures. In the second scenario, the shape, brightness, and other characteristics of port facilities and ship targets are highly pixellated. Oriented Faster R-CNN and Oriented Reppoints still produce multiple errors in detection, and both miss small-sized ship targets that are densely docked. LSR-Det detects most of the targets but easily misidentifies two approaching ships as one target. Our method misses one small target but picks up the rest. The third scenario is mainly the condition of multi-target multi-direction distribution. Oriented Faster R-CNN and Oriented Reppoints miss targets with less obvious edge profiles, while LSR-Det misses targets with limited adaptability to targets with partially obscured edges. The fourth scenario is the condition that the scene scattering interference is more serious. Oriented Faster R-CNN, Oriented Reppoints, and LSR-Det cannot distinguish the docked ships seriously affected by scattering interference, and the first two methods show many false detections. Our method uses edge deformable convolution to better adapt to deformed and discontinuous targets. In addition, the point set conversion strategy adopted can better focus the core features of the target and capture the direction more accurately, showing the best detection performance in the above scenarios. The experimental results of different scenarios show that the proposed method is superior to other methods in detecting the target and reducing the false alarms and missed detections as much as possible.

We also show the detection results of different algorithms on the SSDD dataset in Figure 6. Similar to the performance of the RSDD-SAR dataset, Oriented Faster R-CNN and Oriented Reppoints are prone to false detection. The overall detection result of LSR-Det is close to that of the proposed method, but it is poor in distinguishing targets with dense docking, and it is easy to identify two targets as one target. By incorporating missed detection labels, we observe that the proposed method has significantly improved detection outcomes for densely arranged nearshore ships and effectively mitigates instances of missed detections.

5. Conclusions

In this article, we propose an SAR ship detection method based on edge deformable convolution and point set representation, which can achieve the accurate detection of densely arranged ships in complex port scenes. The proposed method mainly consists of two parts. Firstly, a structure of edge deformable convolution is proposed to learn the highly discrete and fragmented local features of ship targets, solving the interference of speckle noise and discontinuous target blocks in SAR images. Afterwards, a multi-directional ship detection module based on point set transformation is proposed, which generates high-quality prediction boxes by constructing point set transformation functions and loss constraints, thus flexibly capturing the shape and direction changes of densely arranged ships. The experimental results on two publicly available datasets, RSDD-SAR and SSDD, show that the proposed ship detection method exhibits better detection performance than seven typical detection models.