MPFT-UNet: A Boundary-Refined and Multi-Scale Dynamic Fusion Network for UAV-Based Port Ship Segmentation

Abstract

Ship semantic segmentation based on unmanned aerial vehicle (UAV) imagery has important application value in maritime scenarios such as marine surveillance, port management, and maritime safety. However, UAV images often contain large scale variations of ships, a high proportion of small targets, and complex background interference, including sea surface reflections, waves, and clouds. These factors make accurate segmentation and boundary localization difficult. To address these issues, this paper proposes a UAV-based ship semantic segmentation network, termed MPFT-UNet. The network introduces a Multi-scale Dynamic Sparse Cross-gating (MDSC) module to improve the representation of small targets. A Boundary Supervision Refinement (BSR) module is used to enhance boundary delineation. In addition, a Transformer-based Feature Fusion (FFT) module is applied at the bottleneck layer to strengthen global semantic representation. Experimental results show that MPFT-UNet achieves better performance than existing methods across multiple evaluation metrics. The model obtains an IoU of 0.8365, Dice coefficient of 0.9028, Recall of 0.8881, and AP of 0.95731. These results indicate stable segmentation performance under complex maritime conditions. Compared with the baseline U-Net model, the IoU is improved by approximately 5.1%.

1. Introduction

With the rapid development of the marine economy and the increasing demand for maritime governance, unmanned aerial vehicles (UAVs) have been widely applied in maritime surveillance, port management, and search and rescue operations, gradually becoming an important tool for marine target perception and situational awareness [1,2]. Their flexibility and high-resolution imaging capabilities provide effective support for large-scale maritime data acquisition [3].

Traditional UAV-based ship detection methods are mainly built on object detection frameworks, in which targets are localized using bounding boxes [4,5]. This type of representation is coarse and cannot accurately describe the true shape of ships. In addition, performance is easily affected by complex background conditions. In scenarios with small-scale or densely distributed targets, detection-based methods are prone to missed detections and target confusion [6]. These problems become more obvious when targets are closely spaced or partially occluded. In contrast, semantic segmentation performs pixel-level classification and separates ship regions from the background [7]. This representation allows more accurate delineation of object contours and structural details. It also provides more reliable information for downstream tasks, including ship identification, behavior analysis, and maritime traffic statistics [8].

Although semantic segmentation has achieved remarkable progress in fields such as medical imaging and urban scene analysis [9], its performance in UAV-based maritime scenarios remains limited. First, the boundary between ships and seawater often has low contrast and irregular shapes. It is also affected by illumination changes and wave disturbances, which can cause blurred or fragmented boundary predictions [8]. Second, ships show large scale variation across different scenes. Small targets may be lost during feature downsampling, which leads to incomplete segmentation and confusion between adjacent targets [5,10]. In addition, complex sea surface backgrounds, including reflections, waves, and shadows, often have visual patterns similar to ships. This makes it difficult for convolutional networks to capture global semantic relationships and increases the risk of misclassification [11].

To address these challenges, existing approaches for UAV-based ship perception can be divided into two categories: detection-based methods and segmentation-based methods. Detection-based methods, such as Faster R-CNN [12], SSD [13], and YOLO [14], are effective in target localization. However, they use bounding box representations, which cannot meet the requirements of fine-grained segmentation. In contrast, segmentation networks, including FCN [15], SegNet [16], and U-Net and its variants [17,18], show better performance in ship segmentation tasks. This is because they combine low-level spatial details with high-level semantic information [19]. However, these methods have a limited ability to capture long-range dependencies. As a result, they still have limitations in boundary delineation, small-target preservation, and global contextual modeling [20,21,22].

This paper proposes a boundary-aware multi-scale fusion network, termed MPFT-UNet, for UAV-based ship segmentation tasks. The model is built on the U-Net architecture and is designed to address several challenges in UAV maritime scenarios. These include blurred boundaries, a high proportion of small targets, and complex background interference. The network is improved from three aspects: boundary modeling, multi-scale feature selection, and global semantic modeling. These components work together to enhance feature extraction, cross-scale fusion, and boundary refinement. The main contributions are summarized as follows:

• A Boundary Supervision Refinement (BSR) module is introduced to address the low contrast and irregular characteristics of ship boundaries under complex maritime conditions. This module enhances boundary-aware feature representation and improves the continuity of object contours.
• A Multi-scale Dynamic Sparse Cross-gating (MDSC) module is designed to cope with significant scale variation and the predominance of small-scale targets in UAV imagery. By incorporating a dynamic sparse attention mechanism, the module enables effective selection of informative cross-scale features while reducing the influence of background noise.
• A Transformer-based Feature Fusion (FFT) module is embedded at the bottleneck layer to mitigate the limitations of conventional convolutional networks in capturing long-range dependencies. This design contributes to improved global semantic consistency in complex maritime scenes.
• Extensive experiments conducted on the LSRS-Ship dataset demonstrate that the proposed MPFT-UNet achieves superior performance compared with existing state-of-the-art methods. The results indicate notable improvements in segmentation accuracy and robustness under complex UAV-based maritime conditions.

The remainder of this paper is organized as follows. Section 2 reviews the related work. Section 3 presents the overall architecture and key modules of MPFT-UNet. Section 4 describes the experimental setup and reports the results of comparative and ablation studies. Finally, Section 5 concludes the paper and discusses future research directions.

2. Related Work

This section reviews techniques related to this study. The content is organized into three aspects: boundary-aware learning and contour refinement, multi-scale feature fusion, and Transformer-based attention mechanisms.

2.1. Boundary-Aware Segmentation and Contour Refinement

In semantic segmentation, object boundaries and contour information are important for modeling fine structures. In UAV-based ship images, the boundary between ships and seawater often has low contrast and is affected by waves, reflections, and shadows. This leads to blurred boundaries and fragmented contours in the segmentation results [20]. Poor boundary quality weakens the geometric representation of ships and reduces the reliability of subsequent analysis tasks [23].

Early studies mainly used traditional edge detection operators and methods based on geometric constraints to model object contours. Typical examples include the Sobel operator, Canny edge operator, and Laplacian operator [24], as well as geometric approaches such as active contour models and level set methods for boundary extraction [25]. These methods obtain boundary information by modeling image gradient changes. However, under complex maritime conditions, wave textures and specular reflections introduce strong interference. This often produces a large number of pseudo edges and reduces the stability of boundary extraction.

With the development of deep learning, boundary-aware mechanisms have been progressively introduced into semantic segmentation networks. One category of methods explicitly supervises contours by constructing dedicated boundary branches and integrating them with backbone features to improve boundary localization accuracy [26]. Another category focuses on enhancing the ability of the model to discriminate boundary regions by jointly optimizing segmentation objectives and boundary constraints. In addition, optimization strategies based on distance-based constraints, such as Boundary Loss [27] and Hausdorff Distance Loss [28], have also been employed to improve contour accuracy. Furthermore, attention-based methods combined with fine-grained supervision, such as channel–spatial attention fusion and point-level supervision, further enhance boundary modeling capability [29,30].

However, existing methods mainly rely on implicit boundary constraints or loosely coupled auxiliary branches, which still suffer from boundary instability under complex maritime conditions. In contrast, the proposed BSR module integrates handcrafted edge priors with deep feature representations in a more direct manner. Gradient-consistent edge cues are embedded into intermediate feature maps, and residual enhancement is applied. This process preserves semantic consistency and strengthens structural continuity. This design effectively suppresses pseudo edges caused by ocean dynamics and improves the clarity and integrity of ship contours in challenging UAV scenarios.

2.2. Multi-Scale Feature Fusion and Small-Target Modeling

In UAV-based maritime scenarios, ship targets exhibit significant scale variations. A single image typically contains large ships at close range and small vessels at long distances, where small targets occupy only a limited number of pixels [5]. In addition, in areas such as ports and busy waterways, ship targets are densely distributed, and the spatial distances between targets are small, which further increases the difficulty of segmenting small targets [31].

To improve the adaptability of the model to scale variations, multi-scale feature fusion has been widely adopted in computer vision tasks. For example, the Feature Pyramid Network (FPN) enhances multi-scale representation through multi-level feature fusion [32]. PSPNet aggregates contextual information using a pyramid pooling structure [33]. The DeepLab series expands the receptive field via atrous convolution to enhance multi-scale modeling capability [34]. In addition, some studies further improve the utilization of multi-scale information through adaptive fusion mechanisms. For instance, MANet enhances feature representation via multi-scale context extraction and dynamic fusion [35], while other studies improve segmentation performance in complex backgrounds by integrating features of different resolutions [36]. These studies demonstrate that multi-scale fusion helps alleviate the loss of small-target information and improves segmentation performance.

However, direct aggregation or concatenation of multi-scale features tends to introduce information redundancy and scale interference, thereby reducing the ability of the model to discriminate small targets and fine-grained structures [37]. Therefore, the key to effective multi-scale modeling lies in achieving efficient information selection and noise suppression. To this end, a MDSC module is proposed, which employs a dynamic sparse attention mechanism to enable adaptive interaction and selection of cross-scale features, improving segmentation performance in small-target and densely distributed scenarios.

2.3. Transformer-Based Global Context Modeling

In UAV-based ship semantic segmentation, sea surface backgrounds often contain similar textures over large areas. Relying solely on the local receptive field of convolutional networks can easily lead to semantic confusion between ships and the background [38]. Therefore, introducing global context modeling plays an important role in improving the consistency and robustness of segmentation results.

Transformer models capture feature dependencies globally through a self-attention mechanism [39] and have been gradually introduced into the field of computer vision. Vision Transformer (ViT) represents images as sequences and demonstrates strong global modeling capability across various visual tasks [40]. Subsequently, a series of hybrid models combining Transformer and convolutional networks have been proposed to balance local detail extraction and global semantic modeling [41,42]. In semantic segmentation tasks, methods such as the SETR and Segmenter have improved context modeling capability by incorporating Transformers and have achieved promising results in urban scene and remote sensing segmentation tasks [43]. Furthermore, recent studies indicate that incorporating Transformer into remote sensing and UAV image segmentation can alleviate semantic confusion among multi-scale targets by modeling cross-region semantic relationships [11].

However, pure Transformer architectures struggle to effectively capture small-scale targets and complex contour structures in local detail representation and boundary modeling [44]. Meanwhile, existing hybrid approaches typically introduce Transformer modules in a loosely coupled manner, which limits the interaction between global context and multi-scale features, resulting in suboptimal performance under complex sea conditions. To address these issues, this paper proposes an FFT module embedded at the bottleneck of the U-Net architecture. By jointly modeling long-range dependencies and enhancing cross-scale feature interaction, FFT reduces semantic ambiguity between ships and background, thereby improving global semantic consistency and structural integrity in complex maritime environments.

3. Proposed Methodology

In this section, three core modules of MPFT-UNet are mainly introduced: (1) the BSR module is designed to improve the contour representation accuracy of ship images; (2) the MDSC module aims to address significant scale variations and the high proportion of small targets in UAV-based ship imagery; and (3) the FFT module is used to fuse low-level detailed features with high-level semantic features, thereby improving the overall consistency of segmentation results.

3.1. Overall Framework

The U-Net architecture consists of an encoder, a bottleneck, a decoder, and skip connections. Both the encoder and decoder adopt Double Convolution Blocks (DCB), and multi-scale feature modeling is achieved through downsampling and upsampling operations. To address challenges such as blurred boundaries, small targets, and complex background interference in UAV-based ship segmentation, three key modules are incorporated into this architecture to construct MPFT-UNet, as illustrated in Figure 1.

Given an input UAV ship image $X\in {\mathbb{R}}^{H\times W\times C}$, where H, W, and C denote the height, width, and number of channels of the input image, respectively ($C=3$ in this study). The encoder extracts multi-scale features layer by layer through downsampling and DCB. At the n-th layer, the output feature of the DCB is defined as

$$F_n^{\mathrm{DCB}}\in {\mathbb{R}}^{{\displaystyle \frac{H}{2^{n-1}}}\times {\displaystyle \frac{W}{2^{n-1}}}\times 2^{n-1}C},$$

(1)

where $F_n^{\mathrm{DCB}}$ denotes the output feature of the DCB at the n-th layer of the encoder, and $n\in \{2,3,4,5\}$ represents the layer index of the network. As the network deepens, the spatial resolution of features decreases progressively, while the number of channels increases accordingly.

To enhance feature representation, the MDSC module is introduced into the skip connections at layers $n=2,3,4$ to enable adaptive interaction and selection of cross-scale features. Meanwhile, the BSR module is applied after the DCB at layers $n=2,3,4,5$ to strengthen boundary feature representation, producing boundary-enhanced features denoted as $F_n^{\mathrm{BSR}}$. During the skip connection process, the MDSC module utilizes the deepest encoder feature as global semantic guidance to perform adaptive interaction and selection on shallow multi-scale features, enhancing semantic representation while preserving detailed information. At the bottleneck layer $n=5$, the deepest feature $F_5^{\mathrm{BSR}}$ is processed by the FFT module, which models global contextual relationships via self-attention.

Through the collaborative operation of these modules, MPFT-UNet effectively integrates multi-scale feature modeling, boundary refinement, and global semantic representation.

3.2. Boundary Supervision Refinement Module (BSR)

To address blurred boundaries, fragmented contours, and inconsistent scale structures of ship targets in UAV-based maritime imagery under complex sea conditions, a BSR module is proposed. The module extracts boundary cues from shallow and intermediate features and suppresses background interference via explicit boundary supervision, thereby enhancing the structural representation capability of the model. The position of the proposed BSR module in the network is illustrated in Figure 1, and its detailed structure is shown in Figure 2.

For the input feature at the n-th layer, the input of the BSR module is the encoder output feature $F_n^{\mathrm{DCB}}$ (as defined in Equation (1)), where $n\in \{2,3,4,5\}$. First, a convolution operation is applied to enhance the local edge response of the feature, which is formulated as:

$$E_{\mathrm{conv}}=\mathrm{ReLU}\left(\mathrm{BN}\left({\mathrm{Conv}}_{3\times 3}\left(F_n^{\mathrm{DCB}}\right)\right)\right).$$

(2)

Relying solely on convolutional features is still susceptible to high-frequency noise such as wave textures and strong reflections. Therefore, edge prior information is introduced, and gradient features are extracted using the Sobel operator

$$G_x=F_n^{\mathrm{DCB}}*S_x^{\mathrm{marine}},G_y=F_n^{\mathrm{DCB}}*S_y^{\mathrm{marine}},$$

(3)

where $S_x^{\mathrm{marine}}$ and $S_y^{\mathrm{marine}}$ denote the horizontal and vertical Sobel operators optimized for maritime ship imagery, respectively. $G_x$ and $G_y$ represent the gradient responses of the feature map $F_n^{\mathrm{DCB}}$ along the horizontal and vertical directions. The operator ∗ denotes the convolution operation. These are further combined with an adaptive Canny edge detector to obtain the boundary map:

$$E_{\mathrm{canny}}=\mathrm{AdaptiveCanny}(G_x,G_y,{\theta }_{\mathrm{ship}}),$$

(4)

where ${\theta }_{\mathrm{ship}}$ denotes the adaptive threshold. This process improves the continuity and stability of the extracted boundaries.

To avoid noise amplification caused by directly aggregating edge features, a residual module is introduced before feature fusion to perform smooth modeling of edge information

$$E_{\mathrm{res}}={\mathrm{ResBlock}}_3\left(E_{\mathrm{conv}}+E_{\mathrm{canny}}\right),$$

(5)

thereby improving the continuity of edge features. On this basis, an Edge-guided Supervision Attention (ESA) mechanism is constructed to perform weighted modeling of boundary regions

$$\begin{array}{c}\hfill P=\sigma \left({\mathrm{Conv}}_{3\times 3}\left(E_{\mathrm{canny}}\right)\right)\\ \hfill E_{\mathrm{esa}}=P\odot E_{\mathrm{res}}\end{array}.$$

(6)

The boundary-enhanced features are injected into the original semantic features via a residual connection to obtain the output of the BSR module

$$F_n^{\mathrm{BSR}}=F_n^{\mathrm{DCB}}+\alpha E_{\mathrm{esa}},$$

(7)

where $\alpha$ is a learnable scalar parameter used to balance the contribution between boundary information and semantic information. This design enhances the model’s ability to represent ship contour regions.

Finally, $F_n^{\mathrm{BSR}}$ is propagated to the encoder downsampling layers and the MDSC module, enabling effective supplementation of multi-scale features with boundary information and enhancing fine-grained details.

3.3. Multi-Scale Dynamic Sparse Cross-Gating Module (MDSC)

In UAV-based maritime scenarios, ship targets typically exhibit significant scale variations and are affected by complex background interference. Multi-level feature fusion may introduce redundant information, thereby reducing the discriminative capability of the model. To address this issue, an MDSC module is proposed to achieve selective enhancement of cross-scale features. The position of the MDSC module within the overall network is illustrated in Figure 1, and its internal structure is shown in Figure 3.

3.3.1. Cross-Scale Multi-Level Fusion

To ensure that cross-scale feature interaction is performed at a unified spatial scale, the BSR-enhanced features at each level are first upsampled to obtain scale-aligned features:

$${F\prime }_n=\mathrm{Upsample}\left(F_n^{BSR}\right),n\in \{2,3,4,5\}.$$

(8)

During cross-scale interaction, the deepest feature is selected as the semantic guidance branch ${F\prime }_5$, while the shallow features are defined as the detail complementary branch ${F\prime }_5$.

To enhance the multi-scale representation capability of shallow features, multi-scale pooling is performed on the shallow features to obtain context at different scales

$$Y_1={Pool}_{k_1}\left(Y\right),Y_2={Pool}_{k_2}\left(Y\right),Y_3={Pool}_{k_3}\left(Y\right),k_1<k_2<k_3,$$

(9)

where ${Pool}_k(·)$ denotes an average pooling operation with a window size of $k\times k$, and appropriate padding is applied to maintain the output size consistent with the input.

Features obtained at different scales are fused to obtain an enhanced shallow multi-scale representation

$$\widehat{Y}=Y_1+Y_2+Y_3,$$

(10)

where $\widehat{Y}$ integrates structural information from different scales and is used for subsequent attention computation.

3.3.2. Dynamic Sparse Cross-Scale Fusion

To fully utilize the detailed structural information in shallow features while avoiding interference from wave reflections, wave textures, and shadow noise on the deep-shallow fusion process, a dynamic sparse cross-scale fusion mechanism is proposed. This mechanism uses deep features X as semantic guidance and shallow multi-scale features $\widehat{Y}$ as complementary information. Cross-scale attention is employed to model feature correlations, while a sparsity strategy is introduced to suppress low-relevance responses.

First, the input features X and $\widehat{Y}$ are projected into a unified feature space to construct the Query, Key, and Value representations

$$\begin{array}{c}\hfill Q=\mathrm{Flatten}\left(X\right)W_Q,\\ \hfill K,V=\mathrm{Split}\left(\mathrm{LN}\left(\mathrm{Flatten}\left(\widehat{Y}\right)\right)W_{KV}\right).\end{array}$$

(11)

Subsequently, scaled dot-product attention is used to compute the correlation matrix between spatial positions:

$$M=\mathrm{Softmax}\left({\displaystyle \frac{Q{K}^{\top }}{\sqrt{c}}}\right),$$

(12)

the matrix M represents the similarity relationships between deep semantic features and shallow multi-scale features across different spatial positions.

A Dynamic Dual-TopK sparsification mechanism is proposed to address irrelevant information introduced by background noise. When the maritime conditions are stable, high-level semantics are highly consistent with shallow structures, and a lower sparsity rate is preferred to retain more information. In contrast, under complex or noisy conditions, a higher sparsity rate is adopted for stricter filtering. Two learnable parameters $sigma\left({\theta }_1\right)$ and $sigma\left({\theta }_2\right)$ are used to generate sparsity ratios:

$$r_1=\sigma \left({\theta }_1\right),r_2=\sigma \left({\theta }_2\right),$$

(13)

where $\sigma (·)$ denotes the Sigmoid function, and $r_1$ and $r_2$ correspond to sparsity strategies with different selection intensities. Based on these ratios, TopK selection is performed row-wise on the attention matrix M:

$$M_1=\mathrm{Softmax}\left(\mathrm{TopK}(M,\lceil r_{1}N\rceil )\right)M_2=\mathrm{Softmax}\left(\mathrm{TopK}(M,\lceil r_{2}N\rceil )\right),$$

(14)

where $\mathrm{TopK}(·)$ denotes the operation of selecting the Top-k largest elements for sparsification.

To further enhance fusion stability and flexibility, learnable weights $\alpha$ and $\beta$ are introduced to combine the two sparse attention maps, yielding the dynamic sparse attention map $M^{\prime }$

$$M^{\prime }=\alpha M_1+\beta M_2,$$

(15)

where $\alpha ,\beta \in [0,1]$ are learnable coefficients. Finally, the fused attention matrix is applied to the Value features to obtain the output feature A:

$$A=M^{\prime }V,$$

(16)

this process selects key structural information from shallow features and suppressing noise.

The MDSC module injects shallow detailed information into deep semantic features. This improves the model’s ability to distinguish small targets, dense targets, and complex backgrounds.

3.4. Transformer-Based Feature Fusion Module (FFT)

In UAV-based maritime scenarios, convolution operations alone cannot capture local structural details and long-range dependencies at the same time. To address this limitation, an FFT module is introduced at the bottleneck layer of the U-Net architecture. The module uses a parallel dual-branch structure. One branch models local details, and the other captures global semantic information. This design improves tthe ability of the network to represent fine structures and long-range relationships. The position of the FFT module within the overall network is illustrated in Figure 1, and its internal structure is shown in Figure 4.

After processing by the encoder and the BSR module, the deepest feature is denoted as $F_5^{\mathrm{BSR}}\in {\mathbb{R}}^{H_5\times W_5\times C_5}$. First, the feature is evenly split along the channel dimension into two parts. These are used as the inputs of the local branch $F_{\mathrm{local}}\in {\mathbb{R}}^{N\times {\displaystyle \frac{C}{2}}}$ and the global branch $F_{\mathrm{global}}\in {\mathbb{R}}^{N\times {\displaystyle \frac{C}{2}}}$, where $N=H_5\times W_5$.

3.4.1. Local and Global Path Modeling

Considering the significant shape variations of ship targets in UAV imagery, deformable convolution is introduced in the local path to enhance local feature modeling capability. Meanwhile, to reduce the computational cost of standard self-attention, a window-based attention mechanism is adopted to perform feature interaction within local windows. The output of the local path is formulated as:

$${F\prime }_{\mathit{local}}=\mathrm{WindowAttn}\left(F_{local}\right)+\mathrm{DeformConv}\left(F_{local}\right),$$

(17)

where $\mathrm{WindowAttn}(·)$ denotes the window attention operation, and $\mathrm{DeformConv}(·)$ represents the deformable convolution operation.

The global path is designed to model long-range dependencies among ships and capture large-scale contextual semantic information, thereby enhancing the perception of overall ship distribution and sea surface background relationships. To reduce computational complexity, a lightweight self-attention mechanism is adopted in this paper. And key–value pairs are generated via soft pooling, enabling global dependency modeling in a low-dimensional space. This approach focuses on low-frequency semantic components, enabling the model to obtain a more stable global semantic representation and reducing the interference of local noise on the overall semantic understanding.

3.4.2. Feature Fusion and Enhancement

After modeling the local and global paths, the two feature branches are concatenated along the channel dimension to obtain the fused representation

$$F_{\mathit{fusion}}=\mathrm{Concat}({F\prime }_{\mathit{local}},{F\prime }_{\mathit{global}}).$$

(18)

An External Attention mechanism is introduced to replace the multi-layer perceptron (MLP) in the standard Transformer. This mechanism reconstructs features using external memory units and reduces parameter complexity and computational cost to a certain extent.

Through the above design, the FFT module effectively integrates local detailed information with global semantic features, thereby improving the consistency and robustness of segmentation results.

4. Experiments and Results

4.1. Dataset

4.1.1. LSRS-Ship Dataset

To evaluate the performance of the proposed method in UAV low-altitude remote sensing ship segmentation tasks, a self-built UAV ship semantic segmentation dataset is constructed, named LSRS-Ship (Low-altitude Remote Sensing Ship Dataset). Representative samples are shown in the first row of Figure 5. Under UAV perspectives, ship targets exhibit significant multi-scale characteristics, while port and waterway scenes present complex backgrounds.

A DJI Mavic 3 Pro UAV was used to collect 1283 ship images in port areas at altitudes ranging from 50 to 200 m. The acquisition process includes both vertical overhead and oblique views, enabling the images to retain standard remote sensing geometric characteristics while preserving perspective variations encountered in real-world scenarios.

The original dataset is divided into training, validation, and test sets with a ratio of 7:2:1. Due to the high resolution of the original UAV images (5280 × 2970), a split-before-cropping strategy is adopted to prevent data leakage. Subsequently, a sliding-window strategy is applied within each subset to crop the images into multiple local regions, which preserves fine details while reducing computational complexity and increasing the number of small-scale samples. After preprocessing, the dataset consists of 14,997 images.

Based on the ratio between target area and image area, objects are categorized into three scales: small-scale (<0.5%), medium-scale (0.5–2%), and large-scale (>2%). The distribution is illustrated in the left panel of Figure 6. Statistical results show that small-scale targets account for 75.78%, while medium-scale and large-scale targets account for 18.55% and 5.67%.

4.1.2. iSAID-Ship Dataset

To evaluate the generalization ability of the model under different imaging conditions and scene distributions, ship category samples are introduced from the publicly available remote sensing dataset iSAID, and an iSAID-Ship dataset consisting of 549 images is constructed. Representative samples are presented in the second row of Figure 5. This dataset is derived from high-altitude remote sensing imagery, where the imaging perspective, target scale distribution, and background structure differ significantly from those of low-altitude UAV data. Ship targets in high-altitude remote sensing images are typically smaller and more densely distributed, which poses greater challenges for accurate segmentation.

During preprocessing, ship categories in iSAID are uniformly converted into binary semantic segmentation labels (ship and background) to ensure consistency with the LSRS-Ship dataset in terms of task formulation. In addition, a statistical analysis of the target scale distribution in the iSAID-Ship dataset is conducted, as shown in the right panel of Figure 6. Specifically, small-scale targets account for 90.24% of the dataset, while medium-scale and large-scale targets account for 7.23% and 2.53%.

The iSAID-Ship dataset is not involved in the training process but is solely used for generalization evaluation to evaluate the performance of the model under cross-dataset conditions.

4.2. Experimental Environment and Settings

All comparative methods are trained and evaluated under a unified experimental environment, with key configuration parameters listed in

Table 1. Experimental settings of the proposed MPFT-UNet.

Category	Setting
Framework	PyTorch 2.8.0
GPU	NVIDIA RTX 5090 (32 GB)
Input Size	$512\times 512$
Optimizer	Adam
Learning Rate	$5\times {10}^{-5}$ (Cosine Annealing)
Batch Size	8
Epochs	80
Loss Function	$\mathrm{BCE}+\mathrm{Dice}+\lambda \mathrm{Edge}$ ($\lambda =0.5$)

.

The experiments are implemented based on the PyTorch framework and conducted on an NVIDIA GeForce RTX 5090 platform with 32 GB of memory. During the data preprocessing stage, a split-before-cropping strategy is adopted to crop all input images into $512\times 512$ patches as network inputs. In the training phase, the Adam optimizer is employed to update model parameters, with an initial learning rate of $5\times {10}^{-5}$ and a cosine annealing learning rate schedule for dynamic adjustment. The batch size is set to 8, and the model is trained for 80 epochs. The loss function is defined as a combination of Binary Cross-Entropy (BCE) loss and Dice loss to jointly constrain pixel-level classification accuracy and region overlap. The detailed definitions are provided in Section 4.3. During the testing phase, the model outputs are converted into probability maps using a sigmoid function and binarized using multiple thresholds. The final performance is determined by selecting the threshold that yields the optimal IoU through threshold searching.

4.3. Loss Function

This study adopts a composite loss function, including BCE loss, Dice loss, and a boundary constraint term.

The BCE loss is used to constrain pixel-wise classification errors, which is defined as

$$L_{\mathrm{BCE}}=-{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}\left[y_{i}log\left(p_i\right)+(1-y_i)log(1-p_i)\right],$$

(19)

where $p_i\in [0,1]$ denotes the predicted probability of the i-th pixel, $y_i\in \{0,1\}$ represents the ground-truth label, and N is the total number of pixels.

To measure the overlap between the predicted region and the ground-truth region, the Dice loss is introduced, defined as

$$L_{\mathrm{Dice}}=1-{\displaystyle \frac{2{\sum }_{i=1}^N{p}_i{y}_i+ϵ}{{\sum }_{i=1}^N{p}_i+{\sum }_{i=1}^N{y}_i+ϵ}},$$

(20)

where $ϵ$ is a smoothing term to avoid numerical instability.

To enhance the ability of the model to represent ship boundaries, a boundary constraint term is introduced. Boundary regions are extracted by performing local difference analysis on ground truth annotations and assigning higher supervision weights to the boundary locations. The formulation is given as follows:

$$L_{\mathrm{edge}}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}\left|p_i-y_i\right|·w_i,$$

(21)

where $w_i$ denotes the boundary weight.

Finally, the overall loss function is defined as:

$$L=L_{\mathrm{BCE}}+L_{\mathrm{Dice}}+\lambda L_{\mathrm{edge}},$$

(22)

where $\lambda$ is a weighting coefficient, which is set to 0.5 in this paper. This composite loss balances the optimization between boundary regions and overall regions, thereby improving the stability of segmentation results.

4.4. Comparative Experiments

To evaluate the performance of the proposed method in low-altitude remote sensing ship segmentation tasks, several representative samples are selected from the LSRS-Ship test set for comparative analysis. The comparison includes several mainstream models, such as U-Net [17], U-Net++ [45], DeepLabV3+ [46], Attention U-Net [47], TransUNet [48], and Mamba U-Net [49]. The selected samples cover complex scenarios such as small-scale targets, dense distribution, and strong reflections and haze interference. The model performance is evaluated from both qualitative and quantitative perspectives. And Intersection over Union (IoU), Dice coefficient, Precision, Recall, and Average Precision (AP) are adopted as evaluation metrics. In addition, True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN) visualizations are introduced for qualitative error analysis. TP and TN regions denote correctly classified ship and background pixels, while FP and FN regions represent falsely detected ship areas and missed ship regions, respectively.

4.4.1. Qualitative Evaluation

The segmentation results of different models are shown in Figure 7. Under complex background conditions, all comparative methods suffer from issues such as blurred boundaries, missed detection of small targets, and background misclassification. In contrast, MPFT-UNet performs better in terms of ship outline integrity, detail representation, and adaptability to complex backgrounds.

In terms of boundary details, models such as U-Net and DeepLabV3+ often produce broken or blurred contours along ship boundaries. This issue is more evident in regions with strong reflections and wave interference, as highlighted by the yellow circles in Figure 8. In comparison, MPFT-UNet produces more continuous and complete ship contours. This improvement comes from the BSR module. The module introduces explicit edge information and boundary-guided attention, which improves boundary localization accuracy.

For small-target segmentation, models such as U-Net++ and Attention U-Net often miss distant or small objects, or produce incomplete segmentation results, as highlighted by the yellow circles in Figure 9. In comparison, MPFT-UNet recovers the structure of small targets more completely. This improvement is related to the MDSC module. The module uses cross-scale feature fusion and dynamic sparse selection to strengthen the representation of small targets.

Under complex background conditions, such as wave textures, cloud shadows, and strong reflections, some methods are more prone to missed detections. But MPFT-UNet effectively maintains responses to target regions in such scenarios, significantly preserving the structural integrity of ships. This improvement is mainly attributed to the Transformer-based FFT module, which enhances long-range dependency modeling through global context modeling.

4.4.2. Quantitative Evaluation

The experimental results are presented in

Table 2. Quantitative comparison of different segmentation models on the LSRS-Ship dataset. The best performances are illustrated in bold.

Model	IoU	Dice	Precision	Recall	AP	Inference Time (ms)
UNet	0.7854	0.8591	0.9030	0.8384	0.9371	7.00
UNet++	0.6857	0.7663	0.8525	0.7336	0.8712	22.51
DeepLabV3+	0.7891	0.8648	0.9076	0.8488	0.9406	4.07
Attention UNet	0.6263	0.7149	0.8245	0.6782	0.8531	10.40
TransUNet	0.7347	0.8168	0.8810	0.7922	0.9028	13.28
Mamba UNet	0.6912	0.7731	0.8533	0.7408	0.8878	4.01
MPFT-UNet	0.8365	0.9028	0.9346	0.8881	0.9573	8.52

. To provide an intuitive comparison of performance across multiple metrics, radar charts are used for visualization, as shown in Figure 10. MPFT-UNet achieves the best performance across all evaluation metrics.

In terms of IoU, MPFT-UNet achieves 0.8365, representing an improvement of approximately 5.1% over U-Net and 4.7% over DeepLabV3+. And compared with the Transformer-based TransUNet, MPFT-UNet achieves an improvement of over 10%. Methods relying solely on traditional convolution or global modeling have limitations in their feature representation capabilities for complex sea surface scenes. The proposed method improves segmentation accuracy through the synergistic integration of boundary modeling, multi-scale feature fusion, and global contextual modeling.

In terms of region overlap, MPFT-UNet achieves a Dice coefficient of 0.9028. For Precision and Recall, MPFT-UNet achieves 0.9346 and 0.8881, respectively. Notably, the improvement in Recall is particularly significant, indicating that the model maintains more stable responses in small-scale targets and low-contrast regions. Furthermore, MPFT-UNet achieves an AP of 0.9573, demonstrating stable performance across different thresholds. And it requires 8.52 ms per image, indicating a moderate inference speed. Although additional modules increase computational cost, the model still achieves good segmentation performance.

4.5. Ablation Experiments

To evaluate the contribution of each module to the overall model performance, ablation experiments are conducted on the LSRS-Ship dataset for the BSR, MDSC, and FFT modules. The results are presented in

Table 3. Ablation study of different modules in MPFT-UNet on the LSRS-Ship dataset. The best performances are illustrated in bold.

BSR	MDSC	FFT	IoU	Dice	Precision	Recall	AP
×	×	×	0.7854	0.8591	0.9030	0.8384	0.9371
✓	×	×	0.8181	0.8874	0.9184	0.8762	0.9441
×	✓	×	0.8032	0.8727	0.9009	0.8664	0.9403
×	×	✓	0.8247	0.8909	0.9230	0.8757	0.9471
✓	✓	×	0.8095	0.8776	0.9160	0.8630	0.9445
✓	×	✓	0.8261	0.8926	0.9207	0.8816	0.9448
×	✓	✓	0.8296	0.8952	0.9254	0.8818	0.9504
✓	✓	✓	0.8365	0.9028	0.9346	0.8881	0.9573

.

Without introducing any modules, U-Net achieves an IoU of 0.7854. After incorporating the BSR module, the IoU increases to 0.8181, and the Dice coefficient improves to 0.8874, demonstrating that the boundary supervision mechanism effectively enhances the response of the model to structural variations, resulting in more continuous and accurate boundaries. With the introduction of the MDSC module, the IoU improves to 0.8032 and Recall reaches 0.8664, indicating that this module enhances the representation of small-scale targets and mitigates the omission of small targets. After introducing the FFT module, the IoU reaches 0.8247, achieving the best performance among single-module configurations, which indicates that global contextual modeling helps improve semantic consistency.

In terms of module combinations, additional dual-module ablation experiments are conducted. When the BSR and MDSC modules are jointly introduced, the IoU reaches 0.8095 and the Dice coefficient reaches 0.8776. Although slightly lower than using the BSR module alone, this configuration still outperforms the baseline U-Net, indicating that local feature enhancement without sufficient global constraints may introduce feature redundancy. When the BSR and FFT modules are jointly applied, the IoU improves to 0.8261 and the Recall reaches 0.8816, indicating that combining boundary refinement with global contextual modeling helps maintain structural continuity and semantic consistency. In addition, the combination of MDSC and FFT achieves an IoU of 0.8296 and a Recall of 0.8818, indicating the complementarity between multi-scale feature fusion and global contextual modeling.

When all three modules are used, the model achieves the best performance across all metrics. The IoU reaches 0.8365, the Dice is 0.9028, the Precision is 0.9346, the Recall is 0.8881, and the AP is 0.9573. Compared with the baseline U-Net, the IoU improves by about 5.1%. This result indicates that the three modules work together in boundary modeling, multi-scale feature representation, and global semantic modeling.

4.6. Parameter Sensitivity Analysis

To evaluate the influence of the boundary loss weight ${\lambda }_{\mathrm{Edge}}$, sensitivity experiments were conducted on the LSRS-Ship dataset with ${\lambda }_{\mathrm{Edge}}$ set to 0.3, 0.5, and 0.7, respectively. The experimental results are presented in

Table 4. Sensitivity analysis of the boundary loss weight ${\lambda }_{\mathrm{Edge}}$. The best performances are illustrated in bold.

${\mathit{\lambda }}_{\mathbf{Edge}}$	IoU	Dice
0.3	0.8317	0.8991
0.5	0.8365	0.9028
0.7	0.8228	0.8943

.

As shown in Table 4, the proposed model achieved the best performance when ${\lambda }_{\mathrm{Edge}}=0.5$, obtaining the highest IoU and Dice values among all settings. When ${\lambda }_{\mathrm{Edge}}$ was set to 0.3 or 0.7, the segmentation performance decreased slightly.

4.7. Cross-Dataset Generalization Experiments

This paper designs cross-dataset experiments to evaluate the robustness and generalization ability of the model under different data distributions. All models are trained exclusively on the LSRS-Ship dataset and directly tested on the iSAID-Ship dataset to evaluate their adaptability under cross-domain conditions.

Table 5. Generalization performance of different models on the cross-dataset evaluation. The best performances are illustrated in bold.

Model	IoU	Dice	Precision	Recall	AP	Inference Time (ms)
UNet	0.6112	0.7245	0.8012	0.6724	0.8410	7.00
UNet++	0.5887	0.7084	0.7796	0.6513	0.8295	22.51
DeepLabV3+	0.6320	0.7550	0.8157	0.6832	0.8578	4.06
Attention UNet	0.5902	0.7226	0.7823	0.6481	0.8263	10.40
TransUNet	0.5843	0.7128	0.7714	0.6372	0.8209	13.28
Mamba UNet	0.6095	0.7296	0.7964	0.6480	0.8315	4.01
MPFT-UNet	0.6927	0.7793	0.8414	0.7552	0.8679	8.52

presents the quantitative evaluation results of all models on the iSAID-Ship dataset, while Figure 11 shows the corresponding qualitative comparisons. Due to the distribution discrepancy between training and testing data, all models exhibit performance degradation on the iSAID-Ship dataset.

U-Net and its variants show significant performance degradation on iSAID-Ship, with frequent missed detections in densely distributed and small-scale target regions. DeepLabV3+ demonstrates certain advantages in preserving overall structure but still suffers from missing targets in complex background regions. TransUNet exhibits unstable performance under cross-dataset conditions, with discontinuous structures, indicating that such methods are sensitive to distribution shifts. In contrast, MPFT-UNet achieves the best performance on iSAID-Ship, with IoU, Dice, Precision, Recall, and AP of 0.6927, 0.7793, 0.8414, 0.7552, and 0.8679, respectively, outperforming all comparative methods. Notably, the Recall performance is particularly strong, indicating that the model effectively reduces missed detections of small-scale targets under cross-domain conditions. As shown in Figure 11, MPFT-UNet maintains stable responses in dense and small-scale scenarios. It produces fewer missed detections and more complete structures. Meanwhile, the inference time remains unchanged under cross-dataset evaluation, indicating stable inference efficiency.

Overall, the cross-dataset experiments show that MPFT-UNet remains robust and generalizes well under different data distributions.

5. Conclusions

In this paper, a multi-module collaborative semantic segmentation network, MPFT-UNet, was proposed to address the challenges of large-scale variation, blurred boundaries, and complex background interference in UAV-based remote sensing ship segmentation. Built upon the U-Net framework, the proposed model incorporated a BSR module to enhance contour representation, an MDSC module to improve small-target modeling, and an FFT module to strengthen global semantic consistency, thereby achieving fine-grained ship segmentation. In addition, a UAV-based ship segmentation dataset, LSRS-Ship, was constructed to provide effective data support for ship segmentation research in complex maritime scenarios. Experimental results on the LSRS-Ship dataset show that MPFT-UNet achieves the best performance across multiple metrics, with an IoU of 0.8365. The model also performs well on the iSAID-Ship dataset. This indicates good generalization under different imaging conditions. However, the multi-module design increases computational cost. Future work will focus on lightweight model design, improved small-target representation, and cross-domain adaptation. These directions aim to improve efficiency and generalization in real-world maritime monitoring tasks.