Instance Segmentation of Ship Images Based on Multi-Branch Adaptive Feature Fusion and Occluded Region Decoupling in Occluded Scenes

Simple Summary

In busy harbors and coastal waters, ships frequently block one another, making it challenging for existing computer vision systems to detect and separate them accurately. To address this, this study developed a novel instance segmentation method. The method effectively extracts ship features across various scales and utilizes these features to restore occluded semantics. Furthermore, it can localize occluded regions and refine ship contours. Experiments on a newly collected dataset show that the method outperforms current state-of-the-art techniques, achieving higher accuracy while maintaining fast processing speeds. This work has the potential to improve maritime safety, enable more efficient port management, and support autonomous ship navigation.

Abstract

Instance segmentation accurately extracts the position and outline of ships, serving as the foundation for maritime safety tasks such as multi-object tracking, sensor fusion, and collision warning. This study focuses on single-frame segmentation and aims to address the challenge of multi-scale ship occlusion in congested ports, providing reliable observational data through high-precision recognition to ensure navigation safety. Existing methods suffer from performance degradation in complex maritime environments due to factors such as multi-scale distribution, low resolution of distant targets, and frequent occlusions. Among these, ship occlusion is particularly problematic as it leads to feature confusion between adjacent instances and inaccurate boundary segmentation. To address these challenges, we propose a novel instance segmentation algorithm (MAF-ORDNet) based on Multi-branch Adaptive Feature Fusion and Occluded Region Decoupling. Firstly, a multi-branch adaptive feature fusion module is designed to capture contextual information through different receptive fields and dynamically fuse multi-scale features, thereby restoring occluded semantics and enhancing robustness. Secondly, an occlusion region decoupling module is constructed to accurately localize occluded regions and enhance contour responses via adaptive sampling, achieving refined boundary processing. In addition, we constructed and annotated the Occlusion ShipSeg dataset, which contains 1969 real occlusion images, 2150 simulated occlusion images, and 1132 images under adverse weather conditions, totaling 17,352 fine instance annotations. Experimental results show that, compared with PatchDCT, YOLOv11s, and Mask2Former, our method improves AP by 2.7%, 3.2%, and 2.4%, respectively, while maintaining a comparable inference speed to YOLOv8s. These results confirm that MAF-ORDNet achieves a favorable balance between accuracy and efficiency in multi-scale occluded ship segmentation tasks.

1. Introduction

In maritime scenes, precise acquisition of ship position and shape is of vital importance for reliable traffic management and safety control in dense ports. Ship instance segmentation technology holds significant research value in this field due to its ability to simultaneously provide precise location and contour information [1]. Unlike practical challenges such as small object visibility or data scarcity, the dense occlusion problem in port scenes caused by convoy navigation and berthing operations leads to inter-object feature penetration, loss of discriminative regions, and blurred physical boundaries. This makes it difficult for models to accurately distinguish individual instances, thereby limiting the application of existing instance segmentation methods in port scenarios. Various image sources are used for ship segmentation, including optical remote sensing [2], SAR [3], infrared [4], and visible light images [5]. Among these, visible light images are characterized by unique advantages such as high spatial resolution and rich texture information. Consequently, visible light data serves as an ideal source for high-precision instance segmentation. Despite the aforementioned advantages of visible light images, accurate segmentation of ship targets in complex port and nearshore waters remains a significant challenge. Severe mutual occlusion degrades the performance of edge localization, making the separation of overlapping objects difficult. Specifically, large ships often occlude small ships, causing fragmented geometric structures and blurred contours. Furthermore, occluded areas suffer from uneven lighting conditions driven by variable maritime environments including fog, rain, snow, and sea surface reflections. The coupling of occlusion and adverse weather conditions results in blurred target boundaries and difficulty in feature extraction, severely compromising mask integrity and segmentation accuracy. Therefore, it is of significance to overcome these challenges to achieve rapid and accurate ship segmentation.

Traditional instance segmentation methods for ships rely heavily on low-level features such as edges and textures but lack global context modeling, resulting in poor generalization and low segmentation accuracy [6]. However, due to its advantages in both accuracy and speed, deep learning-based instance segmentation has become the mainstream research approach. Existing studies on ship instance segmentation mainly focus on multi-scale perception, small object enhancement, overlapping object processing, and multi-feature fusion. To address challenges of large variations in ship scale and the tendency to overlook small-target features, Sharma et al. [7] proposed the MASSNet algorithm aiming at strengthening multi-scale feature representation via attention mechanisms, which effectively enhances contextual awareness and improves segmentation accuracy. Yuan et al. [8] designed the GR R-CNN network by introducing a recurrent enhanced pyramid structure and a fine-grained global mask head to collaboratively optimize multi-scale feature fusion. Wang et al. [9] constructed ALF-YOLO and incorporated an asymptotic feature pyramid network (AFPN) to integrate multi-level features and thereby mitigate environmental interference. Similarly, Yasir et al. [10] embedded a feature optimization module into the YOLOv7 backbone to expand the receptive field and enhance multi-scale target capture capability. To solve the issue of low segmentation accuracy for small-scale ships, Sun et al. [11] proposed DANet to enhance feature responses of small targets through the combination of a pyramid refinement structure and a dual-branch design. Zhou et al. [12] combined wavelet transform with a P2 detection head to enlarge the receptive field and facilitate feature interaction, improving the robustness of small ship segmentation against complex marine backgrounds. Sharma et al. [13] refined key features using an enhanced ASPP module to improve small-target ship segmentation accuracy. To overcome the limitations of single visual features, Ren et al. [14] fused semantic structures extracted by shallow CNNs with traditional handcrafted features such as HOG and LBP to enhance ship segmentation accuracy. Guo et al. [15] proposed a discriminative information enhancement method based on cross-modal domain knowledge to ensure detection robustness under adverse weather conditions. Xu et al. [16] developed MM-ShipNet to enhance detection and classification performance through the fusion of radar and AIS information. Furthermore, Chen et al. [17] designed the EU-Net architecture to extract maritime ship features through an encoder, substantially improving the reliability of ship detection and segmentation.

Although existing CNN-based ship segmentation algorithms perform well in general scenarios, most of them struggle to effectively handle occlusion problems. As illustrated in Figure 1, common occlusions mainly include overlapping ships, small targets occluded by large ones, environmental interference such as adverse weather conditions, and complex navigation scenarios such as double overtaking. These occlusions hinder the extraction of key features such as contours, shapes, and textures, thereby affecting the accuracy of ship instance segmentation in complex marine scenes.

Figure 1. Common occluded ship images.

Currently, several methods have attempted to address the target occlusion problem. To tackle imaging issues caused by fog, rain, scattering, and noise, Yuan et al. [18] proposed the FA YOLO framework, which employs an adaptive defogging module and multi-scale feature aggregation to effectively alleviate image blurring caused by fog. Sun et al. [19] introduced a dynamic contour learning mechanism along with an interference suppression module to mitigate haze effects, effectively improving segmentation accuracy in low-visibility environments. Zhang et al. [20] developed SwinSeg, combining Swin Transformer with a lightweight MLP to enhance segmentation performance in adverse weather while maintaining computational efficiency. Wei et al. [21] designed LFG-Net with a low-level feature-guided pyramid structure to enhance segmentation detail quality in complex scenes. To handle dense occlusion and background interference, Qu et al. [22] proposed DSONet to enhance ship detail through an adaptive upsampling module and a dual-feature detection structure. Sun et al. [23] developed a spatial attention-based algorithm with a location guidance module to suppress background noise. Xue et al. [24] proposed RFFSCPNet to capture fine boundaries via a contour predictor, achieving higher instance segmentation accuracy in occluded scenarios. Among these methods, FA YOLO, SwinSeg, and DSONet have been progressively integrated into practical maritime surveillance and autonomous navigation systems, particularly for real-time ship detection and environmental perception in adverse weather or high-density port environments.

Existing studies are primarily designed for specific scenarios such as fog interference, complex background noise, or general occlusion, demonstrating effectiveness under specific conditions. Consequently, these methods may struggle to accurately segment ships in more complex and diverse occlusion scenarios. Furthermore, high-quality segmentation data for marine scenes remains scarce due to the high cost of mask annotation. Existing publicly available ship instance segmentation datasets mostly focus on single, complete ships and lack coverage of multi-scale and mutually occluded cases. To address this, we perform data augmentation on collected ship images to expand the set of occlusion samples.

The contributions of our work can be summarized as follows:

(1)

The multi-branch adaptive feature fusion (MAF) module is proposed to solve the feature confusion and detail loss caused by the occlusion of multi-scale ships in dense scenarios. This module employs a synergistic design of progressive pooling and multi-branch dilated convolutions to achieve adaptive fusion of multi-scale features, reinforcing fine-grained details of small targets while preserving the global semantic integrity of large targets. Experimental results demonstrate that MAF significantly improves the detection of small objects, contributing to a 2.5% increase in AP_S compared to the YOLOv8s.

(2)

The occlusion region decoupling (ORD) module is introduced to tackle the blurred boundaries and inaccurate localization caused by multi-scale ship occlusion. This module first injects shallow texture and contour information into deep semantic features, and employs an Occlusion Region Localization Module to achieve precise localization of occluded regions. The integration of ORD leads to a 3.3% improvement in AP compared to the YOLOv8s, effectively enhancing localization accuracy in high-overlap scenarios.

(3)

A spatial-adaptive RFA detection head is designed to address the incomplete segmentation caused by texture degradation and edge breakage under dense occlusion and severe weather scenarios. This module integrates depthwise separable convolutions with denoised prototype masks to enhance the extraction of local texture features.

(4)

In response to the scarcity of multi-scale occlusion scenarios, the Occlusion ShipSeg dataset is constructed. The dataset comprises 5251 occluded ship images of varying scales and 17,352 fine instance mask annotations, covering multiple ship types in marine and port environments.

The remainder of this paper is organized as follows. The details of our proposed method are introduced in Section 2. The segmentation performance of the proposed network and comparisons with other methods are presented in Section 3. Finally, the conclusions are drawn in Section 4.

2. Materials and Methods

2.1. MAF-ORDNet

To address the challenges of multi-scale feature confusion, blurred occlusion region localization, and edge fragmentation in ship instance segmentation under dense occlusion scenarios, MAF-ORDNet is proposed to segment ship images. The core idea of the proposed method is to construct a collaborative framework for feature restoration, region localization, edge refinement, and mask generation. As shown in Figure 2, MAF-ORDNet consists of four main components: the backbone network, the MAF module, the ORD module, and the spatially adaptive RFA detection head. The backbone network utilizes C2f-CSP to extract multi-scale feature maps layer by layer, providing essential semantic and spatial information. The MAF module achieves adaptive fusion of local details and global semantics through lightweight multi-dilation convolutions combined with dynamic weighting. Subsequently, the ORD module focuses on occluded regions via spatial localization and strengthens edge responses using deformable convolutions. Finally, the spatially adaptive RFA detection head integrates these enhanced features and edge information to generate continuous and complete instance masks. Specifically, MAF-ORDNet takes an image of size $H\times W\times C$ as input. First, the C2f-CSP backbone extracts multi-scale features, generating five feature maps denoted as $C_1$ to $C_5$. Subsequently, the $C_3$, $C_4$, and $C_5$ feature maps are fed into the neck network for cross-layer fusion. After being processed by the MAF module, $C_5$ is fused bottom-up with $C_4$ and $C_3$ through concatenation, generating feature maps at three scales: $P_3$, $P_4$, and $P_5$. These three feature maps are then separately processed by the ORD module to decouple occluded regions and subsequently fed in parallel into the spatially adaptive RFA detection head. Through decoupled convolutions, the head predicts object categories, bounding boxes, and instance masks, producing the final segmentation output.

Figure 2. The MAF-ORDNet framework for ship instance segmentation in occlusion scene.

2.2. Muti-Branch Adaptive Feature Fusion Module

In dense scenes, mutual occlusion among multi-scale ships frequently leads to feature confusion and missed detections. Traditional ASPP modules [25] extract multi-scale context using a fixed dilation rate, which hinders adaptive receptive field adjustment for occluded regions. Such a design inevitably introduces redundant information, hindering the recovery of complete semantics. To address these issues, we propose a multi-branch adaptive feature fusion module. This module first expands the base receptive field through hierarchical pooling. It then extracts multi-scale information using three parallel dilated convolutions with different dilation rates, and finally introduces a dynamic weighting mechanism to adaptively adjust the fusion ratio across branches. The synergy of these three components enables the MAF module to achieve an adaptive receptive field that varies with the target scale, as illustrated in Figure 3. The input feature F undergoes dual-branch processing, where the left branch enhances texture details through average pooling and channel attention. The main branch employs cascaded max-pooling followed by three-branch dilated convolutions for multi-scale feature extraction. Finally, the outputs of both branches are concatenated to generate the enhanced feature map $F^{*}$.

Figure 3. The architecture of MAF. Given an input feature map $F\in {\mathbb{R}}^{C\times H\times W}$, a channel attention branch captures global context via AvgPool and PWConv, while a multi-scale spatial branch employs MaxPool and T-DilatedConv for multi-granularity feature extraction. The fused features are concatenated and refined by a $1\times 1$ convolution to generate the final refined feature map $F^{*}\in {\mathbb{R}}^{C\times H\times W}$.

To resolve the problem of semantic loss in ship segmentation, the MAF module adopts a synergistic mechanism combining multi-branch dilated convolutions with adaptive weighted fusion. Specifically, small-dilation convolutions focus on preserving fine-grained local details within occluded regions, while large-dilation convolutions capture global contextual information across occlusions. The two branches are adaptively fused through weighted aggregation to enhance feature completeness. Meanwhile, the module incorporates global attention to suppress background noise and integrates progressive pooling to aggregate multi-scale context, thereby effectively recovering the semantics of occluded regions and boosting recognition accuracy for multi-scale occluded ships.

Given the input feature map, the MAF module employs three cascaded max-pooling operations and concatenates them with the original feature map to form four branches. This design adopts a gradual receptive field expansion strategy using multiple small-sized pooling operations instead of a single large-kernel pooling. While achieving a comparable receptive field, this approach significantly reduces the number of parameters and improves computational efficiency. The three max-pooling operations can be formulated as follows:

$$X_i=M{P}^{\left(i\right)}\left(F\right),i=1,2,3,$$

(1)

$$N_i=C_{1\times 1}\left(MP\left(X_i\right)\right),i=1,2,3,$$

(2)

where $X_i$ denotes the feature map after the i-th max-pooling operation, $MP$ represents the max-pooling operation, $C_{1\times 1}$ indicates the $1\times 1$ convolution operation, and $N_i$ is the corresponding feature map after dimensionality reduction via the $1\times 1$ convolution.

To extract features of ships at different scales within occluded regions, we design a dilated convolution structure, as illustrated in Figure 4. The structure is composed of three parallel convolutional branches with different dilation rates ($r=1,2,3$). Each branch processes only one-twelfth of the input channel dimension, reducing parameters and computational cost while achieving effective receptive fields of $5\times 5$, $7\times 7$, and $11\times 11$, respectively. This enables the parallel capture of local texture details of small targets and global contour features of large targets, thereby optimizing the multi-scale feature representation capability of the model. Subsequently, learnable parameters $w_1$, $w_2$, and $w_3$ are introduced to construct a dynamic weighted fusion mechanism, which adaptively integrates features from different scales and generates context-rich feature maps with low computational overhead.

Figure 4. Structure of dilated convolutions for extracting multiscale features.

Furthermore, to address the issues of reduced image contrast and blurred details caused by complex marine meteorological environments, we employ PConv and PWConv [26] to model channel dependencies. By aggregating spatial context through global average pooling, an adaptive channel weight matrix is generated. After applying a Sigmoid activation function, redundant channels are suppressed, enhancing the distinguishability of foreground features. This approach effectively strengthens critical details such as contours and textures while suppressing background noise. The computational process of feature maps in the MAF module is formulated as follows:

$$F_i=C_{1\times 1}\left(\mathit{cat}{\left({\mathit{DC}}_r\left(N_i\right)\right)}_{r=2,3,4}\right),i=1,2,3,$$

(3)

$$F_{c1}=\mathit{cat}\left(w_1\ast F_1,w_2\ast F_2,w_3\ast F_3,{\mathit{MP}}^{\left(3\right)}\left(F\right)\right)$$

(4)

$$F^{*}=cat\left(\sigma \left(PW\left(PC\left(Avg\left(F\right)\right)\right)\right)\odot F,F_{c1}\right)$$

(5)

where $\left(3\right)$ denotes the number of max pooling operations, while $Avg$ represents the average pooling operation. $C_{1\times 1}$ indicates the $1\times 1$ convolution operation, and $D{C}_r$ refers to the dilated convolution operation where r represents the dilation rate. $F_i$ denotes the feature map generated by concatenating the three dilated convolution branches, and $F_{c1}$ represents the feature map after concatenating four branches. $PC$ and $PW$ denote the PConv and PWConv operations, respectively. Finally, $\sigma$ indicates the Sigmoid activation function.

Figure 5 presents the visualization results of the feature map P5 processed by the MAF module, where the second column shows the feature maps without MAF processing and the third column shows those processed by MAF. It illustrates the module’s capability to represent features of ships at different scales, while also reducing background noise to a certain extent. In the heatmaps, high-brightness regions (yellow/green) indicate strong feature responses of the network, whereas low-brightness regions (purple) correspond to weak responses. Specifically, the ship hull contours and superstructures (e.g., masts, bridges) exhibit the highest response intensity (bright yellow). Decks and ship sides show moderate responses (green), and background areas such as water surfaces and buildings appear as low-response dark purple. In the first row, the network without MAF struggles to accurately identify large-scale ships due to a mismatch between the fixed receptive field and the target scale, whereas the MAF module effectively captures complete ship features. In the second row, the network without MAF fails to capture small target features, leading to missed detections, while the MAF module clearly delineates the contours of large ships and effectively extracts features of small ships. In the third row, the network without MAF suffers from confusion between ship features and the background due to a lack of multi-scale contextual modeling, whereas the MAF module effectively suppresses background interference and improves the discriminability of the feature maps. These results demonstrate that the MAF module effectively alleviates feature confusion, reduces missed detections of small targets, and enhances feature discriminability.

Figure 5. Visualization of features output by MAF.

2.3. Occluded Region Decoupling Module

To address the issues of boundary blurring and localization difficulties caused by mutual occlusion among ships in dense scenarios, we propose an occlusion region decoupling module. This module integrates shallow texture and contour information into deep semantic features, localizes occluded regions via position-wise normalization, and enhances contour extraction through an edge-aware mechanism, thereby enabling precise separation of occluded instances and refined boundary delineation. The ORD module consists of an Occlusion Region Localization (ORL) module and an Edge Feature Enhancement (EFE) module, as illustrated in Figure 6. The ORL module utilizes XNorm to compute independent statistics for each spatial position as the normalization basis, and further highlights differences in occluded regions through learnable scaling and shifting to achieve precise localization. The EFE uses deformable convolution to predict sampling point offsets, adaptively shifting them toward the contour when edge gradients change abruptly to enhance edge response. The synergistic effect of these two modules significantly improves segmentation accuracy in complex occlusion scenarios.

Figure 6. The architecture of the occlusion region decoupling module.

As shown in Figure 7, the occlusion region localization module adopts a bottleneck architecture. It enhances feature representation through channel compression and nonlinear mapping to highlight ship structures, while controlling computational complexity via dimension reduction. To improve the ability to distinguish occlusion boundaries, the module employs XNorm for dynamic feature calibration in the spatial dimension. It captures spatial distribution in a fine-grained manner by computing statistics independently for each position, and adaptively adjusts features at each position using learnable parameters $\gamma$ and $\beta$. This mechanism effectively amplifies the differences between occluded and non-occluded regions, enhances the distinguishability of overlapping target boundaries, and achieves effective separation of occluded instances. In addition, the module introduces a learnable parameter $\alpha$ in the residual connection to adaptively fuse the enhanced features with the original input, thereby avoiding the loss of structural information. The formulas of the occlusion region localization module are provided as follows:

$$X_1=W_2\ast \left(\mathrm{ReLU}(W_1\ast X)\right)$$

(6)

$$Y=X+\alpha ·\mathit{XNorm}(X,X_1,X_1)$$

(7)

where X denotes the input feature map with dimensions ($2C\times H\times W$), while $W_1$ represents the first $1\times 1$ convolution kernel with size ($2C/4\times 2C\times 1\times 1$). $ReLU$ denotes the ReLU activation function, and $W_2$ is the second $1\times 1$ convolution kernel with size ($2C\times 2C/4\times 1\times 1$). This signifies the output feature map ($2C\times H\times W$) processed by the bottleneck transform structure, which consists of a $1\times 1$ convolution, a ReLU layer, and another $1\times 1$ convolution. Additionally, $XNorm$ stands for Cross-Normalization, and $\alpha$ represents the feature map processed by the Ship Occlusion Region Localization module. The parameter $\alpha$ is a learnable parameter with a range of $[0,1)$ and an initial value set to 0. Smaller values of $\alpha$ encourage the model to preserve more original structural features to avoid information loss, while larger values gradually increase the contribution of enhanced features to the final output.

After obtaining the feature map Y, the module integrates shallow contour information with deep semantic features by applying a Sigmoid activation function to Y. This activated output is then multiplied element-wise with the original features. Subsequently, the resulting feature map is added to the original features from another branch, and a final concatenation operation is performed. The specific expression is formulated as follows:

$$F_l^{*}=F_l+w_h\ast F_h,F_h^{*}=F_h+w_l\ast F_l$$

(8)

$$F_c=C\left(F_l+w_h\ast F_h,F_h+w_l\ast F_l\right)$$

(9)

where $w_h$ and $w_l$ represent the weights for the deep and shallow feature maps, respectively, with a value range of $[0,1]$. $F_h^{*}$ indicates the deep feature map injected with shallow information, while $F_l^{*}$ signifies the shallow feature map injected with deep information. Additionally, $F_c$ refers to the concatenated feature map, and C represents the concatenation operation.

Figure 7. The architecture of ORL.

Figure 7. The architecture of ORL.

When ship targets occlude each other, traditional edge detection methods often struggle to align accurately with the true boundaries due to fixed sampling positions, resulting in weakened edge responses and blurred occlusion boundaries. To address this issue, we design a ship edge enhancement module, as illustrated in Figure 8, aimed at improving the ability of model to perceive ship edges. This module introduces a learnable offset $\Delta p_n$ for each sampling point $p_n$, allowing the convolution kernel to adaptively adjust its sampling positions based on the true ship contours, thereby obtaining higher quality contour information. Within the sampling region, the input features $x(p_0+p_n+\Delta p_n)$ are weighted and summed with the convolution weights $w\left(p_n\right)$, significantly enhancing the edge feature response. The specific formulas of the EFE are expressed as follows:

$$Y_1\left(p_0\right)=\sum _{p_n\in R}w\left(p_n\right)·x(p_0+p_n+\Delta p_n)$$

(10)

$$Y_1=DCN\left(\sigma \left(C_{3\times 3}\left(\sigma \left(C_{1\times 1}\left(x\right)\right)\right)\right)\right)+x$$

(11)

where $Y_1\left(p_0\right)$ denotes the value at position $p_0$ in the output feature map. $w\left(p_n\right)$ represents the weight of the convolution kernel at position $p_n$, and x indicates the input feature map. R represents the sampling region of the convolution kernel. $\Delta p_n$ is the learnable offset corresponding to the n-th sampling point, which is represented as a 2D vector $(\Delta x,\Delta y)$. $(p_0+p_n+\Delta p_n)$ indicates the actual reading position of the n-th offset sampling point for the current output position $p_0$, and $x(p_0+p_n+\Delta p_n)$ is the feature value corresponding to this position. Additionally, $C_{1\times 1}$ is the $1\times 1$ convolution kernel, and $C_{3\times 3}$ is the $3\times 3$ convolution kernel. $\sigma (·)$ denotes the ReLU activation function. $DCN$ represents the Deformable Convolution kernel. Finally, $Y_1$ is the output feature map. To avoid edge information loss caused by over-reliance on a single path, we assign a weight w to one feature channel and $1-w$ to the other. The final feature fusion process can be formulated as

$$F_h^{*}=cat\left(w{F}_h^{*}+(1-w)F_l^{*}\right)$$

(12)

Figure 9 presents the visualization results of the feature map $F_h^{*}$ generated by the ORD module, where the second column shows feature maps without ORD processing and the third column shows those processed by ORD. It displays the feature maps of adjacent targets under occlusion, demonstrating that ORD can accurately locate occlusion boundaries to ensure clearer separation between ships. In the first row, the network with ORD successfully captures the complete features of three ships, whereas the network without ORD suffers from incomplete feature extraction due to the lack of spatial localization capability. In the second row, the network without ORD confuses the contours of two large ships and misses a small ship. In the third row, the network without ORD struggles to distinguish adjacent ships from the background, resulting in blurred boundaries. In contrast, the ORD module significantly improves the discriminability of occluded regions and boundary clarity through the collaborative mechanism of occlusion region localization and edge enhancement. These experimental results demonstrate that the ORD module effectively enhances the distinction between ship contours and the background, leading to more complete and accurate segmentation results.

Figure 8. The architecture of EFE.

Figure 8. The architecture of EFE.

Figure 9. Visualization of features output by ORD.

Figure 9. Visualization of features output by ORD.

2.4. Spatial Adaptive RFA Detection Head Module

Conventional convolutional detection heads with fixed receptive fields struggle to adapt to texture degradation and edge fragmentation caused by the coupling of dense occlusion and adverse weather conditions, thereby compromising mask integrity. To this end, we design a spatially adaptive detection head based on RFAConv and integrate it with ProtoNet. First, leveraging the spatial feature transformation mechanism of RFAConv [27], the detection head unfolds the input features into a non-overlapping structure to break the constraint of weight sharing, preserving fine-grained textures while generating spatial mask coefficients, thereby laying a foundation for repairing mask fragmentation. Subsequently, ProtoNet introduces depthwise separable convolution to eliminate channel interference, producing robust prototype masks. Finally, the mask coefficients and prototype masks are linearly combined to repair fragmented textures and output complete instance masks.

The spatially adaptive RFA detection head adopts a multi-task parallel decoupled structure, as shown in Figure 10, where tasks such as classification, bounding box regression, and mask coefficient generation have independent prediction branches. Unlike conventional designs that stack standard convolutions, this module introduces two cascaded RFAConv modules at the front of each branch. It partitions the feature map into independent local structures within the receptive field to break the constraints of weight sharing, and applies dynamic reweighting to adaptively enhance features at discontinuities, thereby generating mask coefficients conducive to shape restoration.

Figure 10. The architecture of adaptive RFA detection head.

In the prototype mask generation stage, to reduce computational overhead and noise interference, ProtoNet employs 4-layer depthwise separable convolution instead of standard $3\times 3$ convolutions, as illustrated in Figure 11. By performing convolution independently on each channel, it avoids cross-channel noise interference and generates denoised prototype masks. Finally, the mask coefficient mask_coef from the spatially adaptive RFA detector is multiplied by the prototype masks to obtain instance masks. The calculation is formulated as follows:

$$M_i=\sum _{i=1}^k{\alpha }_i{P}_i$$

(13)

where $M_i$ denotes the instance mask after morphological repair, while $P_i$ represents the prototype mask generated by ProtoNet, and ${\alpha }_i$ indicates the corresponding mask coefficient predicted by the spatial adaptive RFA detection head. Through the spatial attention mechanism, the prototype mask coefficients enable each $P_i$ to precisely capture the local semantic structures of ships, alleviating segmentation discontinuity under dense occlusion and producing more complete ship masks.

Figure 11. The architecture of ProtoNet.

Figure 11. The architecture of ProtoNet.

2.5. Occlusion ShipSeg Dataset

Existing publicly available datasets exhibit significant limitations for ship instance segmentation tasks. General-purpose datasets such as COCO and PASCAL VOC contain very few ship samples, accounting for less than 1% of the total instances. Specialized datasets like SeaShips [28] support only object detection and lack instance segmentation annotations, while ShipRSImageNet [29] is designed for remote sensing scenarios and differs substantially from visible-light imagery. More critically, existing datasets generally lack coverage of multi-scale, mutually occluded ship images, which represent a typical challenge in real-world port and waterway environments. To address this gap, we construct the Occlusion ShipSeg dataset. This dataset is composed of images from field collection, publicly available datasets, and occlusion samples generated through data augmentation. This dataset includes images of ships at various scales under different occlusion relationships, encompassing both real and simulated occlusions, and covers a variety of sea and weather conditions, thereby providing a reliable data foundation for ship instance segmentation research in densely occluded scenes.

Due to the scarcity of real-world data on occluded ships, this paper adopts an occlusion simulation method based on Mixup [30] to generate ship occlusion images, thereby expanding the training samples and improving model generalization capability.

$$x_1=x_c-\alpha ·w/2,x_2=x_c+\alpha ·w/2$$

(14)

$$y_1=y_c-\alpha ·h/2,y_2=y_c+\alpha ·h/2$$

(15)

First, the region of interest (ROI) of a ship is cropped from the source image, with its bounding box coordinates denoted as $(x_c,y_c)$ and its width and height as w and h, respectively. The horizontal and vertical ranges of the ROI are defined by $x_1,x_2$ and $y_1,y_2$. Subsequently, a scaling factor $\alpha \in (0,1]$ is introduced to resize the ROI, enabling the simulation of occluding objects at different scales. The transformed ROI is then pasted onto another background image, and the new bounding box information is recorded to generate the corresponding annotations. Finally, the synthesized ship occlusion images are obtained, as illustrated in Figure 12.

Figure 12. Examples of occlusion-enhanced images. The first and second columns show the original images, and the third column presents the occlusion effects generated by the simulation.

To enhance model generalization and mitigate overfitting, a generative adversarial network (GAN) [31] is introduced to simulate adverse weather conditions such as fog, rain, and snow, automatically generating ship images with diverse weather effects along with their corresponding annotations. By incorporating illumination variations and simulated occlusions, this method improves the model’s ability to perceive local textures and geometric edges, thereby enhancing the discriminability between ships and complex backgrounds. Examples of the resulting weather-augmented images are shown in Figure 13.

Figure 13. Examples of enhanced images with different weather effects. The first column displays the original images, while the second, third, and fourth columns correspond to the results after fog, snow, and rain processing, respectively.

To construct a ship instance segmentation dataset tailored for multi-scale occlusion scenarios, this paper collects ship images with varying scales from multiple sources, including field-captured visible-light images, public datasets, and existing maritime datasets such as SeaShips, Mariboats [32], and Singapore Maritime [33]. The dataset is further enriched through simulated occlusions and weather-based augmentation, ultimately forming the Occlusion ShipSeg dataset, which covers multi-scale occlusion scenarios. This dataset includes a wide range of common ship types, including speedboats, sailboats, fishing vessels, cruise ships, cargo ship, and military ships, and encompasses diverse imaging environments, such as ports and terminals, adverse weather conditions, open seas, and nearshore scenes.

The Occlusion ShipSeg dataset consists of 5251 multi-scale occluded ship images, including 1969 real occlusion images and 3282 simulated occlusion images generated through data augmentation (https://gitee.com/Occlusion_Shipseg/Occlusion_ShipSeg.git, accessed on 20 March 2026). Images in the Occlusion ShipSeg dataset were collected from multiple high-density shipping areas, including major seaports such as Singapore Port, Shanghai Port, and Zhoushan Port, as well as busy coastal waters and frequently used navigation channels. These areas are characterized by high ship density and frequent inter-ship occlusion. All images are annotated using the ISAT semi-automatic annotation tool. The dataset contains a total of 17,352 instance masks, with an average of 3.3 ship targets per image. The scene sources for this dataset include typical navigational environments such as large ports, offshore waters, coastal areas, and densely populated ship waterways. Regarding ship type distribution, speedboats account for 25%, sailboats 12%, fishing vessels 10%, cargo ships 25%, cruise ships 20%, and military ships 8%. In terms of target size, small targets are defined as those with fewer than $32\times 32$ pixels, medium targets as those between $32\times 32$ and $96\times 96$ pixels, and large targets as those exceeding $96\times 96$ pixels, with small targets accounting for 26% of all instances. Detailed statistics of the dataset are presented in Table 1. The Occlusion-test includes 420 real-world occluded images and 250 images generated through data augmentation.

Table 1. The detailed information of Occlusion ShipSeg dataset.

Dataset	Ship Seg
Images	5251
Real—Occlusion; instances	1969; 6694
Simulated—Occlusion; instances	2150; 7310
Weather—Augmentation; instances	1132; 3348
Occlusion—Train; Instances	2592; 8553
Occlusion—Test; Instances	690; 2105
Number of instances	17,352
Small objects	4511
Medium objects	6366
Large objects	5781
Type	Visual images
Task	Instance segmentation

3. Results

3.1. Experimental Parameters

The experiments were conducted on a Windows 11 platform using an GTX 4060 GPU with PyTorch 1.12.1, accelerated by CUDA 11.6 and cuDNN 11.0. No pre-trained weights were loaded during training. The Adam optimizer was employed with a batch size of 16, an initial learning rate of 0.001, and a weight decay of $5\times {10}^{-5}$. Input images were uniformly resized to $640\times 640\times 3$. The models were trained for 600 epochs with automatic mixed precision enabled to improve training efficiency. All comparative experiments were conducted using the same experimental settings.

The experiments adopt the Occlusion ShipSeg dataset for training and evaluation, which comprises 1969 real occlusion images, 2592 training images, and 690 test images, with the training and test sets split in an 8:2 ratio. Evaluation metrics include AP, as well as AP₅₀, AP₇₅, AP_S ($\mathrm{area}<{32}^2$ pixels), AP_M (${32}^2<\mathrm{area}<{96}^2$ pixels), and AP_L ($\mathrm{area}>{96}^2$ pixels) from the COCO evaluation protocol. Additionally, to ensure a comprehensive assessment of algorithm performance and efficiency, we also report the number of parameters (Para), frames processed per second (FPS), Time per frame (T), and floating point operations (FLOPs).

3.2. Ablation Experiments

To validate the effectiveness of the proposed method, ablation experiments are conducted on the Occlusion ShipSeg dataset using YOLOv8s with CSP-C2f backbone as the baseline, with results presented in Table 2. Introducing the MAF module alone leads to improvements of 2.5% and 2.4% in AP_S and AP_M, respectively, indicating that the module effectively enhances segmentation accuracy for small and medium targets by leveraging multi-dilation convolutions and a global attention mechanism to preserve fine-grained ship details. Incorporating the ORD module alone yields gains of 3.3%, 4.2%, and 6.3% in AP, AP₅₀, and AP₇₅, respectively, with a 2% increase in AP_L, demonstrating that the edge enhancement mechanism of ORD refines contour modeling and improves boundary precision for large targets. When both modules are integrated, segmentation accuracy improves across all scales, with AP_S, AP_M, and AP_L increasing by 4.1%, 3.4%, and 2.5%, respectively. This suggests that the two modules work synergistically to extract multi-scale features and contextual information, thereby enhancing the model’s capability for segmenting ships under multi-scale occlusion scenarios.

Table 2. Performance contribution of different modules.

MAF	ORD	AP	AP50	AP75	APS	APM	APL	FPS
✕	✕	39.3	64.3	38.1	22.7	36.3	64.3	118.2
√	✕	42.3	67.3	43.0	25.2	38.7	64.7	111.7
✕	√	42.6	68.5	44.4	24.7	38.4	66.3	111.4
√	√	43.5	69.9	46.1	26.8	39.7	66.8	110.2

3.3. Comparison Experiments

We conducted comparative experiments on the Occlusion ShipSeg dataset with several mainstream instance segmentation methods, and the results are shown in Table 3. When trained on a conventional dataset, Mask R-CNN achieved an AP of 35.7% on the occlusion test set; after training on the occlusion dataset, its AP improved to 39.1%, verifying the effectiveness of occlusion-specific training. Under the same occlusion training settings, CondInst, PatchDCT, and Cascade R-CNN achieved AP scores of 39.6%, 40.8%, and 42.0%, respectively, all outperforming Mask R-CNN. These results demonstrate that simulated occlusion and weather augmentation strategies can effectively enhance the model’s ability to recognize multi-scale ships.

Table 3. The result of different models on the occlusion-train dataset.

Method	Training Set	Testing Set	Backbone	AP	AP50	AP75	APS	APM	APL
Mask R-CNN [34]	Normal-Train	Occlusion-Test	R-50-FPN	35.2	60.3	37.5	23.6	29.1	55.9
Mask R-CNN [34]	Normal-Train	Occlusion-Test	R-101-FPN	35.7	61.2	40.1	23.2	32.6	58.0
Mask R-CNN [34]	Occlusion-Train	Occlusion-Test	R-50-FPN	38.3	60.8	40.9	22.9	33.3	61.9
Mask R-CNN [34]	Occlusion-Train	Occlusion-Test	R-101-FPN	39.1	62.8	41.5	23.6	37.9	64.8
CondInst [35]	Occlusion-Train	Occlusion-Test	R-50-FPN	39.6	62.7	42.0	26.0	34.7	65.7
PatchDCT [36]	Occlusion-Train	Occlusion-Test	R-101-FPN	40.8	68.2	41.9	21.8	36.8	66.4
BoxInst [37]	Occlusion-Train	Occlusion-Test	R-50-FPN	30.3	56.4	27.4	18.0	23.8	54.7
Cascade R-CNN [38]	Occlusion-Train	Occlusion-Test	R-101-FPN	42.0	68.2	43.9	22.9	33.3	61.9
SOLOv2 [39]	Occlusion-Train	Occlusion-Test	R-50-FPN	36.5	62.5	38.2	20.7	29.7	61.0
Yolact [40]	Occlusion-Train	Occlusion-Test	R-50-FPN	41.9	66.9	41.3	21.3	37.4	67.8
Mask2former [41]	Occlusion-Train	Occlusion-Test	R-50-FPN	41.1	68.3	42.0	22.2	36.9	65.3
YOLOv8s	Occlusion-Train	Occlusion-Test	CSP-C2f	39.3	64.3	38.1	22.7	36.3	64.3
YOLOv11s	Occlusion-Train	Occlusion-Test	CSP-C3k2	40.3	64.3	41.6	24.6	37.1	63.8
Ours	Occlusion-Train	Occlusion-Test	CSP-C2f	43.5	69.9	46.1	26.8	39.7	66.8

The proposed MAF-ORDNet achieves an AP of 43.5% under the same experimental settings, representing a 3.8% improvement over the baseline YOLOv8s. Compared to CondInst, SOLOv2, and BoxInst, its AP is 3.9%, 7.0%, and 13.2% higher, respectively, primarily due to the ORD module’s enhancement of edge details. Compared to PatchDCT, AP improves by 2.7%. This is because PatchDCT relies on local block transformations, which can easily lose the overall contour, whereas MAF-ORDNet maintains feature integrity through multi-scale fusion. In terms of multi-scale performance, MAF-ORDNet achieves 4.1%, 3.4%, and 2.5% improvements in AP_S, AP_M, and AP_L over YOLOv8s, respectively, and 4.6%, 2.8%, and 1.5% improvements over Mask2former, indicating that the MAF module enhances feature representation for ships at different scales. Notably, although MAF-ORDNet achieves slightly lower AP_L than Yolact, it outperforms Yolact by a substantial margin of 5.5% in AP_S. This is because Yolact relies on deep low-resolution features, which benefit large-object semantics but compromise small-object details. In contrast, MAF-ORDNet leverages the synergy between weighted fusion of shallow and deep features and edge enhancement, improving small-object representation while maintaining stable segmentation of large objects, thereby achieving more balanced multi-scale performance.

Table 4 presents a comparison of computational complexity and inference efficiency across different models. MAF-ORDNet has only 12.4 M parameters, significantly fewer than models such as Mask R-CNN, SOLOv2, and Cascade R-CNN. Its FLOPs are only 35.9 G, which is approximately $1/4$ of SOLOv2, $1/5$ of Mask R-CNN, and $1/8$ of BoxInst, demonstrating considerably lower computational complexity.

Table 4. Comparison of Para, FPS, T, and FLOPs.

Method	Backbone	Para (M)	FPS	T (ms)	FLOPs (G)
Mask R-CNN [34]	R-50-FPN	44	14	71.2	196
Mask R-CNN [34]	R-101-FPN	63	13.8	72	245
CondInst [35]	R-50-FPN	33	11.5	87.0	305
PatchDCT [36]	R-101-FPN	60.3	11.6	86.2	266
BoxInst [37]	R-50-FPN	34	11	90.7	343
Cascade R-CNN [38]	R-101-FPN	95	9.1	110.4	309
SOLOv2 [39]	R-50-FPN	46	36.9	27.1	186
Yolact [40]	R-50-FPN	34	31.6	31.6	61
Mask2former [41]	R-50-FPN	47	10.2	97.6	178
YOLOv8s	CSP-C2f	11.8	118.2	8.4	32.1
YOLOv11s	CSP-C3k2	10.7	121.6	8.2	31.5
Ours	CSP-C2f	12.4	110.2	9.0	35.9

Specifically, two-stage methods such as Mask R-CNN and Cascade R-CNN achieve relatively high accuracy but come with parameter counts of 44 M and 63 M, FLOPs of 196 G and 245 G, and inference speeds of only 14 FPS and 13.8 FPS, respectively, making it difficult to meet real-time requirements. CondInst enhances multi-scale perception through conditional convolutions but incurs substantial computational costs, with FLOPs reaching 305 G and a speed of only 11.5 FPS. PatchDCT performs well in conventional scenes but has limited boundary fitting capability under occlusion, with FLOPs reaching 266 G. Yolact achieves relatively high inference speed but lacks sufficient edge detail capture, resulting in limited accuracy under occlusion. Mask2former attains an AP of 41.1%, but its Transformer-based global attention leads to heavy computational demands, achieving only 10.2 FPS. YOLOv11s has a lightweight architecture with a speed of 121.6 FPS, but its accuracy is 3.2% lower than that of MAF-ORDNet. In contrast, MAF-ORDNet achieves an AP of 43.5% while maintaining a high inference speed of 110.2 FPS, striking a favorable balance between accuracy and efficiency, and demonstrating comprehensive advantages in real-time, multi-scale occluded ship segmentation tasks.

To further verify the robustness of our model, we conducted supplementary experiments on the real-world occluded dataset. Specifically, we utilized 1969 real occluded images, which were partitioned into training and validation sets at an 8:2 ratio to evaluate the model’s generalization capability in authentic occluded scenarios. The experimental results are presented in Table 5.

Table 5. The result of different models on the real-occlusion dataset.

Method	Backbone	AP	AP50	AP75	APS	APM	APL
Mask R-CNN	R-50-FPN	36.1	58.5	38.6	21.0	31.5	59.8
Mask R-CNN	R-101-FPN	37.0	60.4	39.2	21.8	35.8	62.7
Yolact	R-50-FPN	39.8	64.5	39.2	19.8	35.2	65.5
YOLOv8s	CSP-C2f	37.4	62.1	36.2	20.9	34.5	62.0
YOLOv11s	CSP-C3k2	38.5	62.2	39.5	22.7	35.3	61.6
Ours	CSP-C2f	41.2	67.5	42.8	24.9	37.9	64.5

Under these authentic occluded scenarios, Mask R-CNN R-101 and Yolact achieved AP scores of 37.0% and 39.8%, respectively, while the latest YOLOv11s reached 38.5% AP. Our proposed MAF-ORDNet achieves the highest performance with an AP of 41.2%, representing a significant improvement of 3.8% over the baseline YOLOv8s and 2.7% over YOLOv11s.The advantage of our method is particularly evident in localization accuracy and multi-scale detection. Specifically, MAF-ORDNet achieves an AP₇₅ of 42.8%, outperforming YOLOv8s by 6.6%, which suggests that the ORD module effectively refines mask boundaries even in complex real-world occlusions. In terms of multi-scale performance, our model improves AP_S and AP_M by 4.0% and 3.4% over YOLOv8s, respectively. Although Yolact shows competitive performance on large objects with an AP_L of 65.5%, it lags behind our model in small-object detection by a margin of 5.1% in AP_S. These results consistently demonstrate that the synergy between multi-scale feature fusion and edge enhancement allows MAF-ORDNet to exhibit good potential for generalization in ship occlusion scenes within harbor or navigation channel areas. However, its performance remains limited by the distribution of the training data and the similarity of the test environment.

As shown in Figure 14, we compare the segmentation performance of Mask2former, Mask R-CNN, Yolact, YOLOv8s, YOLOv11s, and the proposed MAF-ORDNet under occlusion scenarios. Column 1 illustrates instances of mutual occlusion between large-scale ships and smaller crafts, while Column 2 displays varying degrees of overlap among ships of diverse dimensions. Column 3 features scenarios characterized by dense ship clusters and severe obscuration, and Column 4 captures inter-ship masking effects occurring across a range of distances, spanning from background to foreground.

Figure 14. The visualization results on occlusion-test. Note that occlusion images and their annotated instance masks are listed in the first and second rows. From the third row to the end, the results are from Mask2former, Mask R-CNN, Yolact, YOLOv8s, YOLOv11s, and MAF-ORDNet. The yellow dashed line and red solid line represent the false segmentation and missed segmentation results, respectively.

In the first column, Mask2former misclassifies background as ships, reflecting its insufficient suppression of background noise; Mask R-CNN fails to completely segment large targets, indicating its limited capability in large-scale contour modeling. Yolact misclassifies background as small ships, revealing bias in small-target feature extraction. YOLOv8s misses two small ships due to scale overlap, while YOLOv11s misses one small target. In contrast, MAF-ORDNet accurately identifies one large ship and four small ships. This is because the MAF module enhances multi-scale feature extraction through multi-ratio dilated convolutions and a global attention mechanism, effectively suppressing background noise.

In the second column, Mask2former misses an occluded ship. Mask R-CNN suffers from instance over-segmentation, splitting three mutually occluded ships into four instances. Yolact exhibits instance fragmentation and misses a small ship; both YOLOv8s and YOLOv11s fail to detect the occluded small targets. By contrast, MAF-ORDNet accurately segments all instances. This is because the ORD module strengthens contour information in occluded regions through an edge enhancement mechanism, effectively distinguishing ship boundaries. In the fourth column, Mask2former misses a small occluded ship. Mask R-CNN fails to identify the centrally occluded ship. Yolact misses both the central ship and the occluded small target. YOLOv8s erroneously segments the mast and hull of the same ship into two separate instances. YOLOv11s missegments the occluded small target into two instances. MAF-ORDNet achieves complete segmentation without any missed detections or false segmentations.

Overall, MAF-ORDNet maintains high segmentation accuracy under occlusion conditions and significantly outperforms the compared methods. The visualization results demonstrate that our proposed network has a better segmentation performance in occluded scenes.

To evaluate the robustness of the model under adverse weather conditions, we compare the segmentation performance of various algorithms under complex meteorological conditions such as rain and fog, as shown in Figure 15. Column 1 presents the segmentation performance of multi-scale ships in dense fog, while Column 2 illustrates the mutual occlusion of ships of different scales under foggy conditions. Column 3 depicts ship occlusion scenarios in rainy weather, and Column 4 shows the instance segmentation results for multi-scale ships in snowy conditions.

Figure 15. The visualization results under adverse weather conditions. Note that occlusion images and their annotated instance masks are listed in the first and second rows. From the third row to the end, the results are from Mask2former, Mask R-CNN, Yolact, YOLOv8s, YOLOv11s, and MAF-ORDNet. The yellow dashed line and red solid line represent the false segmentation and missed segmentation results, respectively.

In the first column of foggy samples, Mask2former misses a small ship and confuses a large ship with the background, indicating a lack of compensation for texture loss caused by fog. Mask R-CNN misses three ships, reflecting insufficient feature extraction under low visibility. Yolact misses two ships and exhibits mask fragmentation for large targets. YOLOv8s misses a small ship and misclassifies a building as part of a ship. YOLOv11s incorrectly merges two ships into one instance. In contrast, MAF-ORDNet accurately segments two large ships and three small ships. The MAF module restores local textures lost due to haze via global attention, effectively preserving semantic coherence.

In the third column of rainy samples, Mask2former fails to distinguish occluded boundaries; Mask R-CNN over-segments one ship into two; Yolact, YOLOv8s, and YOLOv11s all suffer from incomplete segmentation due to background interference. However, MAF-ORDNet accurately segments all three ships. This is primarily due to the adaptive RFA detection head, which generates more precise prototype masks and maintains the continuity and integrity of segmentation even under interference. In the fourth column of snowy samples, Mask2former and Mask R-CNN each miss one ship; Yolact and YOLOv8s each miss two ships, with the latter also confusing small targets with the background. YOLOv11s misses one small target affected by environmental interference. MAF-ORDNet correctly identifies all ships, demonstrating its comprehensive robustness in scenarios with multiple types of interference.

In summary, MAF-ORDNet exhibits strong robustness under adverse weather conditions, effectively restoring complete edges and textures of ships. This performance is attributed to the synergy of the MAF, ORD, and RFA detection head modules, enabling high segmentation accuracy in complex interference scenarios.

4. Conclusions

In this paper, we propose a ship instance segmentation framework MAF-ORDNet for densely occluded ship scenarios. First, a multi-branch adaptive feature fusion module is developed to construct multi-scale feature extraction branches by integrating progressive pooling with multi-dilation convolutions, and to achieve adaptive feature fusion through dynamic weighting and a global attention mechanism, thereby enhancing the model’s multi-scale perception capability. Second, an occlusion region decoupling module is introduced to accurately localize occluded regions via cross-layer feature concatenation and spatially independent normalization, while deformable convolutions strengthen edge responses for precise boundary refinement. Third, the spatially adaptive RFA detection head overcomes parameter-sharing constraints through spatial feature transformation. Combined with ProtoNet, it generates robust prototype masks, yielding continuous and complete ship instance masks. Finally, the Occlusion ShipSeg dataset is constructed, comprising 5251 images with 17,352 fine mask annotations covering various ship types and simulating typical occlusion scenarios in dense port and nearshore navigation environments. Our proposed method achieves an AP of 43.5% with an inference speed of 110.2 FPS. Compared to Mask R-CNN, Mask2former, and YOLOv8s, it improves AP by 4.4%, 2.4%, and 4.2%, respectively. These numerical results further validate the effectiveness of the proposed approach. This study uses a large amount of synthetic occlusion and weather-augmented data during training. Although we have validated the effectiveness on an independent real-world dataset, domain discrepancy between synthetic and real data may still affect generalization performance. Future work requires larger-scale and more diverse real annotated data to further verify the robustness of the model. Furthermore, while this study focuses on maritime occlusion, the underlying mechanisms of feature decoupling and spatial adaptation in MAF-ORDNet possess inherent extensibility. Future research could explore the adaptation of this framework to other high-density logistics environments, such as aircraft parking segmentation in airports or container stack monitoring in smart ports, where inter-object occlusion remains a critical challenge.