A Multi-Scale Attention U-Net for Water Body Extraction Under Urban Shadow Interference

Abstract

With the continuous improvement of spatial resolution in remote sensing imagery, the representation of ground object details has been significantly enhanced. However, shadows generated by complex objects such as buildings and vegetation have become increasingly prominent, severely interfering with water body spectral characteristics and posing challenges for the automatic extraction of narrow and fragmented water bodies. To address shadow interference, blurred boundaries, and multi-scale water body recognition in high-resolution remote sensing images, this paper investigates water body extraction based on a commonly used dataset in water body shadow interference research. On the basis of the U-Net architecture, this paper proposes a Multi-Scale Attention U-Net (MSA-UNet) by integrating multi-scale feature extraction and attention mechanisms. The proposed model enhances the representation of water bodies at different scales through multi-scale feature fusion, while the attention mechanism emphasizes discriminative water body features and suppresses shadow-induced interference, thereby improving segmentation accuracy under complex background conditions. Experimental results demonstrate that MSA-UNet achieves performance comparable to or superior to mainstream semantic segmentation networks, including U-Net, U²-Net, and TransUNet, in terms of Overall Accuracy (OA), F1-score, Intersection over Union (IoU), and the Kappa coefficient. These results confirm the effectiveness and robustness of the proposed method for high-precision water body extraction under shadow interference conditions.

1. Introduction

Water resources are a critical natural foundation for the sustainable development of human society, and their spatial distribution patterns and dynamic variations are directly related to water security, ecological environmental protection, and regional development decision-making [1]. With the increasing impacts of climate change and intensified human activities, refined monitoring and dynamic sensing of water bodies have become important research directions in the fields of water resources management and ecological monitoring.

Traditional water body monitoring mainly relies on manual field surveys and point-based sampling. Although such approaches can achieve high accuracy at local scales, they suffer from limitations including long monitoring cycles, limited spatial coverage, and difficulties in generating continuous spatial data, making them inadequate for large-scale and rapidly changing water body monitoring. In contrast, remote sensing technology, with its wide spatial coverage, short data acquisition cycles, and strong spatial continuity, has become an essential tool for regional-scale water body monitoring.

In recent years, the rapid development of high-resolution remote sensing imagery has significantly improved the representation of water body boundaries and fine-scale details, while simultaneously introducing new technical challenges. Shadow effects generated by complex ground objects, such as buildings, vegetation, and terrain variations, become more pronounced in high-resolution imagery and are easily confused with water bodies in terms of spectral characteristics and spatial morphology. This issue poses severe interference to the identification of narrow and fragmented small water bodies. Consequently, existing water body extraction methods still suffer from widespread misclassification and omission errors under complex shadowed backgrounds, and the robustness of current models to multi-scale water bodies and shadow interference remains to be further enhanced.

With the advancement of deep learning techniques, U-Net and its variants have been widely applied to water body semantic segmentation tasks and have achieved notable improvements in water body extraction accuracy. However, most existing studies primarily focus on increasing network depth or expanding feature channels, while insufficient attention has been paid to the collaborative modeling of multi-scale water body feature representation and shadow region suppression. In particular, there remains a research gap in the design of network architectures specifically tailored to mitigate interference caused by ground-object shadows.

To address the challenges of shadow interference and multi-scale water body recognition in high-resolution remote sensing imagery, the main contributions of this paper are summarized as follows:

(1)

A water body semantic segmentation model integrating multi-scale feature extraction and attention mechanisms, termed Multi-Scale Attention U-Net (MSA-UNet), is proposed. Built upon the U-Net architecture, the proposed model incorporates multi-scale feature representation and attention-guided learning to achieve high-precision segmentation of water bodies under complex background conditions.

(2)

A multi-scale feature extraction module is designed to enhance the representation of water bodies at different scales. Through a multi-scale feature fusion strategy, the proposed module strengthens the model’s ability to characterize water bodies with varying sizes and shapes, effectively improving the recognition of narrow and fragmented water bodies in high-resolution remote sensing imagery.

(3)

An attention mechanism is constructed to suppress interference caused by ground-object shadows. By adaptively weighting feature responses, the attention mechanism highlights discriminative water body information while attenuating the influence of shadow regions generated by buildings and other ground objects, thereby improving the accuracy and stability of water body extraction under complex shadowed backgrounds.

2. Materials and Methods

2.1. Data Sources

The dataset used in this study is derived from an existing remote sensing water body extraction dataset adopted in previous research [2]. As shown in Figure 1, the study area for water body extraction is Liuhe District, Nanjing City, Jiangsu Province, China. Liuhe District is located on the northern bank of the lower reaches of the Yangtze River, with geographic coordinates approximately ranging from 32°20′ to 32°50′ N latitude and 118°30′ to 119°10′ E longitude. The district covers an area of about 1500 km² and is characterized primarily by flat terrain, with hilly areas distributed locally, representing typical geomorphological features of the Yangtze River Delta.

Figure 1. Study Area.

The complex natural geographic conditions and dense water system of Liuhe District, Nanjing, make it an ideal study area for remote sensing-based water body extraction. The region contains a wide variety of water body types, including natural rivers, lakes, reservoirs, ponds, and artificial canals, which exhibit both the spatial continuity of natural water bodies and the geometric regularity of artificial ones. This diversity provides abundant samples for investigating the spectral characteristics and spatial distribution of water bodies. In addition, the land cover types in the study area are highly heterogeneous, with farmland, forest land, residential areas, and industrial land interwoven. Multi-source remote sensing data, including Landsat, Sentinel series, and domestic Gaofen satellite imagery, are readily available, providing strong data support for this study. However, the spectral similarity between water bodies and dark ground objects such as shadows and asphalt roads increases the difficulty of accurate water body extraction. The basic information of the remote sensing imagery used in this study is summarized in Table 1. The spatial resolution of the GF-2 imagery is referenced to the official mission specifications provided by the eoPortal database (see Reference [3]).

Table 1. Image Information.

	Parameter
Image	Gaofen-2 (GF-2) high-resolution satellite imagery
Number of Bands	4
Image Size	7300 × 6900
Spatial Resolution (m)	0.8 m PAN/3.2 m MSS
Coverage	118°22′ E–119°14′ E, 31°14′ N–32°37′ N

In this dataset, representative sample areas covering buildings, shadows, water bodies, and other land cover types were selected from the remote sensing images. High-precision vector label data were generated through manual visual interpretation, in which water body regions were annotated as the target class with a pixel value of 1 and filled in white, while non-water regions were treated as background with a pixel value of 0 and filled in black, enabling accurate binary classification. After completing the vector sample construction, the labels were converted into standard raster formats to ensure data compatibility. Furthermore, data augmentation was applied to the original samples, resulting in a total of 4000 augmented samples. These samples were divided into training and validation sets at a ratio of 3:1, providing reliable data support for model optimization and performance evaluation. The dataset images and their corresponding labels are shown in Figure 2.

Figure 2. Selected Remote Sensing Images and Label Maps from the Dataset.

To ensure an unbiased evaluation, the test set used in this study is spatially independent from the training and validation samples. As illustrated in Figure 3, the orange rectangular box denotes the training area and the validation area, while the yellow rectangular box represents the test area. The training, validation, and test areas are geographically separated, with no spatial overlap between the test region and the samples used for model training or parameter tuning. This spatial separation helps prevent information leakage and supports the validity of the reported performance.

Figure 3. Spatial distribution of the training, validation, and test areas.

2.2. Methods

In the early stages of water body information extraction using optical remote sensing data, manual visual interpretation was the primary approach. This method can achieve relatively high extraction accuracy by relying on expert knowledge and experience in land-cover discrimination. However, manual interpretation is highly labor-intensive and time-consuming, involves substantial economic costs, and suffers from low processing efficiency, making it unsuitable for large-scale or long-term water body monitoring tasks. With the rapid advancement of remote sensing technologies and computational capabilities, water body extraction methods have gradually evolved from traditional manual visual interpretation toward intelligent and automated processing, providing effective technical support for regional- and large-scale water body monitoring.

Water body extraction methods can generally be categorized into five major types. The first category comprises traditional methods based on spectral characteristics, which extract water bodies by exploiting their distinct spectral properties, such as differences in band reflectance. These methods mainly include spectral thresholding approaches and water indices-based approaches. Spectral thresholding methods identify water bodies by setting thresholds on band reflectance or absorption characteristics. Among them, single-band threshold methods segment water bodies using reflectance thresholds from individual bands, such as the near-infrared (NIR) band. Zhao et al. demonstrated through experiments conducted in mountainous river channels during the wet season that single-band threshold methods achieved higher accuracy than other approaches under typical monthly conditions [4]. The LBV transformation analysis method constructs a discrimination model based on three elements: radiance (L), band ratio (B), and radiance variation vector (V). Wei et al. proposed an algorithm based on LBV transformation analysis, which effectively eliminates the influence of roads, building shadows, and cloud cover on water body extraction, and enables accurate extraction of small water bodies such as ponds, lakes, and reservoirs [5]. Water index-based methods enhance water body features by constructing multi-band combination indices. For example, the Normalized Difference Water Index (NDWI) is derived from the combination of the green and near-infrared bands. Huang et al. showed that NDWI outperformed other models in water body extraction accuracy [6]. Nie et al. proposed a Modified Automatic Water Extraction Index (MAWEI) for water body information extraction, and experimental results demonstrated that MAWEI can be used for rapid and effective extraction of large-area water bodies [7]. In addition, the Enhanced NDWI (ENDWI) and the Modified NDWI (MNDWI) introduce mid-infrared or shortwave infrared bands to reduce interference from non-water features. Zhang et al. developed the ENDWI specifically for glacial-dammed lake extraction, achieving higher accuracy and faster processing efficiency than other methods, thereby demonstrating its practical value in emergency monitoring applications [8].

The second category consists of supervised classification methods based on machine learning, which distinguish water bodies from non-water areas by training classifiers using labeled samples. Representative methods include Support Vector Machines (SVM), decision trees, and maximum likelihood estimation (MLE). SVM effectively handle complex land surface interference through multi-feature integration and parameter optimization. Zhang et al. combined SVM with visual interpretation in experiments conducted in the Three Gorges Reservoir Area, achieving an overall classification accuracy exceeding 90%, and demonstrating particular advantages in the refined extraction of elongated river boundaries [9]. Duan et al. reported that SVM outperformed other classification methods in terms of accuracy in a comparative study of Poyang Lake [10]. Decision tree methods perform classification or regression through a hierarchical tree structure and recursive data partitioning. Li et al. conducted a comparative study in Weishan County, Shandong Province, and found that the decision tree method achieved the highest extraction accuracy among the evaluated approaches [11]. Zheng et al. demonstrated that a decision tree method based on the Shadow Water Index (SWI) effectively eliminated interference from buildings and bright land cover types, while clearly distinguishing shadows from water bodies, making it particularly suitable for extracting small water bodies at the watershed scale [12]. Wang et al. investigated the Bakhaleike Reservoir in the Tekes River Basin, Xinjiang, and showed that a decision tree classification method combining single-band thresholding with the Multiband Shadow Water Index (MSWI) achieved superior performance, yielding high overall accuracy [13]. Maximum likelihood estimation (MLE) assumes that the spectral features of each class follow a normal distribution. By calculating the likelihood probability of each pixel belonging to predefined classes, the pixel is assigned to the class with the highest probability. Song et al. applied supervised classification based on MLE to urban inundation areas caused by heavy rainfall in Langfang City, demonstrating that this method can effectively extract floodwater extents in urban environments [14].

The third category comprises deep learning-based methods, which automatically learn water body features from remote sensing images using deep neural networks. These methods are particularly suitable for complex scenes and high-resolution imagery. Representative models include Convolutional Neural Networks (CNNs), U-Net and its variants, multi-scale networks, Transformer-based models, CoNFM-Net (ConvNeXt-based Nested Feature Mining Network), and HRNet (High-Resolution Network). Convolutional neural networks are well suited for processing high-dimensional data with strong spatial correlations. Li et al. constructed an automated water extraction model using lightweight CNN-based image classification and semantic segmentation networks, and comparative experiments demonstrated high extraction accuracy [15]. Chen et al. showed that combining CNNs with object-oriented methods effectively preserved water body boundary integrity and suppressed mountain shadow interference, achieving improved extraction of narrow rivers [16]. Li et al. proposed a dense local feature compression network that integrates spatial and spectral information from remote sensing images [17]. Farooq et al. introduced a CNN architecture named WaterNet, which improved prediction accuracy and ensured boundary pixel continuity while reducing computational complexity [18]. U-Net and its variants are particularly effective for small-sample learning and high-precision segmentation. Li et al. proposed a Laplacian edge-enhancement-based water extraction method, achieving superior performance for large water bodies, small lakes, and narrow rivers [19]. Cai et al. designed an improved U-Net model with dual down-sampling and up-sampling operations, outperforming the original U-Net in land cover discrimination and resistance to confusing features [20]. Li et al. proposed KWEnet (Kernel-Wise Attention Enhanced Network), which introduced an attention mechanism into the up-sampling process of U-Net to enhance water-related features and suppress interference from building shadows [21]. Li et al. found through comparative experiments that U-Net++ achieved the highest training accuracy and exhibited strong boundary extraction capability [22]. Zhao et al. proposed MAP-UNet (Multi-scale Atrous Pyramid U-Net), which integrates multi-level atrous convolution and pooling structures, and verified its effectiveness and advancement in water body extraction [23]. Wang et al.proposed a Res2Net-UNet-based method for small water body extraction, effectively addressing issues such as shoreline discontinuities and salt-and-pepper noise, thereby improving extraction accuracy [24]. Liu et al. introduced DMAM-UNet (Dual Multiplicative Attention Module U-Net), demonstrating that the model achieved higher extraction accuracy [25]. Multi-scale networks capture features at different spatial scales through multi-branch architectures or hierarchical processing. Liang et al. proposed a multi-scale attention-based water segmentation network that enhanced feature learning across different water body scales while reducing training time [26]. Yin et al. demonstrated that the Deeplab V3+ model based on dilated residual networks effectively extracted water information from high-resolution remote sensing images [27]. Ma et al. proposed a multi-scale feature densely connected network based on deep feature encoding and water identification decoding, addressing edge detail loss and low accuracy issues [28]. Zhang et al. showed that an improved Deeplab V3+ model effectively extracted river water bodies, enhanced small river detection, and reduced omission errors [29]. Miao et al. proposed a network integrating multi-scale features and global contextual information, which effectively suppressed noise, improved water extraction accuracy in complex backgrounds, and enhanced boundary segmentation [30]. Liu et al. proposed a multi-scale feature extraction network for water body extraction, significantly improving the performance for small water bodies [31]. Transformer-based models enhance long-range dependency modeling through global attention mechanisms. Wang et al. constructed a high-resolution urban water body dataset and employed an attention-based SegFormer semantic segmentation model, which can be extended to large-area water extraction and mapping [32]. Liu et al. proposed an improved Swin Transformer-based method for water extraction from ZY-3 satellite imagery, demonstrating its effectiveness and reliability [33]. CoNFM-Net (Context-aware Nested Feature Mining Network) is a water extraction method based on ConvNeXt with dual feature extraction branches. Zhou et al. proposed CoNFM-Net based on PSPNet (Pyramid Scene Parsing Network), and experimental results showed that the method exhibited certain advantages [34]. HRNet (High-Resolution Network) is suitable for tasks requiring precise localization, such as remote sensing image segmentation and keypoint detection. Zhang et al. found that HRNet demonstrated superior performance in noise suppression and shadow resistance [35]. OCNet (Object Context Network) is designed for high-precision segmentation in complex scenes. Weng et al. employed OCNet as a feature extraction backbone to improve baseline semantic segmentation models, achieving excellent performance on water extraction tasks in high-resolution, UAV, and true-color remote sensing imagery. SCR-Net (Spectral-Contextual Residual Network) is specifically designed for multispectral remote sensing imagery [36]. Weng et al. proposed SCR-Net, which enhanced feature learning for lakes and ensured segmentation accuracy [37]. Guo et al.proposed a multi-scale water extraction convolutional neural network (MWEN), and experimental results demonstrated that MWEN outperformed other comparative methods across multiple scenarios [38].

The fourth category consists of multi-source data fusion-based methods, which improve water extraction accuracy by integrating heterogeneous remote sensing data, such as optical imagery, radar data, and elevation information. Li adopted a rule-based object-oriented approach to extract narrow linear rivers developed under special climatic and geomorphological conditions in Xinjiang. By incorporating ASTER DEM data with a spatial resolution of 30 m, the proposed method effectively suppressed interference from mountainous shadows and dark land-cover features, while preserving the continuity and integrity of linear water bodies [39]. Wang et al. applied an optimized data fusion strategy for water information extraction and conducted comparative analyses based on overall classification accuracy and the Kappa coefficient [40]. Their results indicated that the Pansharp transformation was the most suitable fusion method for water extraction from GF-1 imagery.

The fifth category includes hybrid methods, which integrate traditional spectral features with machine learning techniques or adopt multi-step optimization strategies to improve water extraction performance. A common approach combines spectral indices with threshold segmentation. Gu et al. demonstrated through comparative experiments that the normalized difference water index (NDWI) combined with thresholding could effectively and completely extract water body information [41]. Lv et al. further showed that NDWI-based threshold segmentation achieved the best extraction results [42]. Khurram et al. proposed a method combining the Automatic Water Extraction Index (AWEI) with the Otsu thresholding algorithm implemented via the Google Earth Engine (GEE) API to capture subtle temporal changes in coastlines [43]. Their study demonstrated the global applicability of GEE for automated coastline extraction and target identification using satellite imagery. Another widely used hybrid strategy involves combining decision tree models with threshold segmentation. Chen et al. applied a decision tree model integrating a single-band threshold method (near-infrared band) and the Shadow Water Index (SWI) in arid regions of northwestern China [44]. This approach effectively eliminated interference from snow cover and mountainous shadows, achieving an overall accuracy of 92.34%. Object-based methods enhance classification accuracy through image segmentation and multi-feature fusion. Huang et al. employed a fractal network evolution algorithm for image segmentation combined with rule-based classification to extract water bodies, achieving higher overall accuracy than other methods [45]. Jia Bo reported that object-based approaches tend to miss small water bodies but generally maintain high overall accuracy [46]. Zhao et al. proposed a multi-scale segmentation-based extraction method, which effectively separated water and non-water information and achieved optimal extraction performance [47]. Principal Component Analysis (PCA)-based methods construct water enhancement models by fusing NDWI, NDVI, and water-sensitive bands such as Band 4 (near-infrared). Shen Mengjun found that PCA-based methods achieved the highest accuracy in comparative studies [48]. Ke Lijuan et al. demonstrated that a PCA-based integrated approach could extract complete and accurate water boundaries while suppressing more than 95% of non-water information, significantly improving water extraction accuracy [49]. Optimal index factor (OIF)-based methods evaluate band combinations using statistical indicators such as standard deviation and correlation to identify optimal spectral combinations. Li et al. showed that the OIF-based comprehensive method produced the most complete water extraction results, with accuracy reaching 95.62% [50].

Automatic multi-feature water extraction (MFWE) integrates spectral and spatial features in an unsupervised manner for water extraction from GF-1 multispectral imagery. Zhang et al. described a method that removes pixels with extremely low PRI feature values—considered potential boundary pixels—and then applies a region-growing-like strategy to achieve high-precision and spatially continuous water extraction [51]. RF-ANN (Random Forest combined with Artificial Neural Networks) integrates neural networks and thermal infrared data to extract river surface information. Shi demonstrated that the RF-ANN method effectively eliminated interference from mountainous shadows and other land-cover types, ensuring high extraction accuracy in terms of water body width [52]. In addition, Hu et al. proposed a water extraction method based on a joint probabilistic model combining convolutional neural networks and spectral information [53]. Compared with other methods, the proposed approach significantly improved the Kappa coefficient and overall accuracy, indicating a higher recognition capability.

In summary, to address the challenges of widely distributed multi-scale water bodies, blurred boundaries, and severe background interference in high-resolution remote sensing-based water body recognition, this study proposes an improved U-Net model, termed Multi-Scale Attention U-Net (MSA-UNet), which integrates a multi-scale convolutional expansion module with a spatial attention mechanism.

Specifically, the MSA-UNet adopts a symmetric encoder–bottleneck–decoder architecture. In each encoding stage, multi-scale convolutional operations are employed to enhance feature diversity and representation capability. During decoding, attention modules are introduced to reweight features in the skip connections, thereby improving boundary delineation and spatial consistency of the segmentation results. The MSA-UNet is specifically optimized for remote sensing image segmentation tasks, achieving a favorable balance between global semantic understanding and local detail preservation while maintaining a lightweight network architecture. The network architecture of the model is shown in Figure 4 and the overall MSA-UNet architecture consists of five main components:

Figure 4. MSA-UNet Network Architecture Diagram.

• Input Layer: The input images have a size of 256 × 256 × 3, corresponding to three-channel RGB images. All images are normalized to scale pixel values to the range [0, 1], improving training stability.
• Encoder: The encoder consists of three encoding layers. Each layer contains a multi-scale convolutional block for feature extraction, followed by max pooling for downsampling.
• Bottleneck: The bottleneck layer serves as a feature bridge between the encoder and decoder, performing multi-scale feature extraction.
• Decoder: In the decoding stage, two-dimensional upsampling is used to gradually restore the spatial resolution of feature maps. A spatial attention mechanism is introduced to selectively enhance and filter skip connection features from the encoding stage. At each decoding step, the attention-modulated encoding features are concatenated with the current decoding features, and the fused features are then passed through a multi-scale convolutional module to further extract and integrate contextual information at different scales, thereby enhancing the model’s representation of boundaries and fine details.
• Output Layer: A 1 × 1 convolution is applied to reduce the number of channels to 1, producing a segmentation probability map. The Sigmoid activation function is used to map arbitrary real numbers into the range (0, 1), making it suitable for binary segmentation tasks.

2.2.1. Multi-Scale Convolution Module

In the MSA-UNet model proposed in this paper, a multi-scale convolutional module is incorporated in the encoder and bottleneck layers to enhance the model’s adaptability to targets of varying scales. The same multi-scale convolutional module with an identical structural configuration is consistently applied across all encoder stages and the bottleneck layer. Without excessively increasing computational complexity, this module employs three parallel 3 × 3 convolutional kernels with different dilation rates (1, 2, and 4) to simulate feature extraction at multiple receptive fields. These dilation rates correspond to progressively enlarged receptive fields, which are approximately equivalent to 3 × 3, 5 × 5, and 9 × 9 convolutional kernels, enabling the extraction of both local and contextual features, as illustrated in Figure 5, where dark blue indicates the original baseline sampling points and light blue indicates the additional sampling points introduced by dilation. In the context of remote sensing imagery, different dilation rates in the multi-scale module enable the network to distinguish narrow linear water bodies from large-area water surfaces by capturing features at complementary spatial ranges. The three sets of features are then concatenated and integrated into a unified feature dimension through a 1 × 1 convolution, producing the final fused feature map. This design maintains a controllable number of parameters while effectively enlarging the receptive field, thereby improving the model’s ability to recognize small-scale targets. The multi-scale convolution module is shown in Figure 6.

Figure 5. 3 × 3 Convolutional Kernels with Different Dilation Rates.

Figure 6. Schematic of the multi-scale convolution module.

2.2.2. Attention Mechanism Module

In the MSA-UNet model proposed in this paper, a spatial attention module is introduced before the fusion of each decoder layer with its corresponding encoder features to enhance feature utilization and spatial localization accuracy during decoding. As illustrated in Figure 7,the attention module guides the current decoder features and the encoder outputs through convolution and normalization operations to generate an attention mask, which dynamically modulates the spatial weight distribution of the encoder features. Based on this, the encoder features are weighted on a per-pixel basis, suppressing redundant background interference and emphasizing regions of interest. This attention mechanism effectively improves the selectivity of skip connection information, ensuring that high-resolution features transmitted through the network are more focused on target edges and fine details. While retaining the advantages of the original U-Net architecture, MSA-UNet enhances spatial consistency and boundary recognition during decoding, thereby improving the accuracy and robustness of segmentation results. Figure 8 presents the visualization of feature responses from different convolutional layers of the network, including shallow layers from early encoder stages, intermediate layers from the middle decoding stages, and deep layers corresponding to high-level semantic representations. It is evident that, as the network depth increases, the model progressively enhances its responses to water regions while suppressing activations in background and shadow areas, indicating that the attention mechanism enables the network to gradually extract discriminative features of water bodies. The visualization further reveals that intermediate-level features exhibit more intricate spatial patterns. This complexity is likely related to the integration of multi-scale convolutional features and information from both the encoder and decoder, allowing the network to simultaneously capture the main water body, shoreline boundaries, and local texture variations. In contrast, shallow and deep features primarily reflect low-level textures and high-level semantic information, respectively, resulting in more localized response regions.

Figure 7. Attention Mechanism.

Figure 8. Visualization of Feature Responses at Different Network Levels. Warm colors highlight regions with strong network responses (e.g., water bodies), while cooler colors indicate weaker responses (e.g., background or shadows).

In the MSA-UNet model proposed in this paper, the output layer features are compressed into a single channel using a 1 × 1 convolution, and a Sigmoid activation function is applied to generate a pixel-wise probability map, enabling end-to-end binary segmentation prediction. The model is trained using the binary cross-entropy loss function, which computes the difference between the predicted and ground truth labels on a per-pixel basis, enhancing the distinction between foreground and background regions. The Adam optimizer is employed to stabilize the training process and accelerate convergence due to its adaptive learning rate characteristics. Input data are uniformly processed as 256 × 256 RGB images with corresponding binary labels. An independent validation set is used to monitor model performance during training, and a checkpoint mechanism is applied to save the best-performing parameters, effectively preventing overfitting and ensuring the model’s generalization capability.

3. Results

3.1. Experimental Setup and Evaluation Metrics

The proposed method is implemented using the TensorFlow deep learning framework. Binary cross-entropy is adopted as the loss function, and model parameters are optimized using the Adam optimizer with an initial learning rate of 1 × 10⁻⁴. The training process is conducted for a total of 50 epochs with a batch size of 8, following commonly adopted settings in recent studies [54]. This parameter configuration ensures training efficiency while maintaining model performance stability. To ensure a fair comparison, all comparative models were trained under the same training strategy, including identical data splits, input resolution, loss function, optimizer, and training schedule. The experimental platform configuration is summarized in Table 2.

Table 2. Experimental Platform Configuration.

Computer Hardware	Corresponding Version
CPU	12th Gen Intel (R) Core (TM) i5-12490F 3.00 GHz
RAM	32.0 GB (31.9 GB available)
GPU	NVIDIA GeForce RTX 3060 (12 G/NVIDIA)

In this paper, four widely used metrics were selected to comprehensively evaluate the performance of the model in binary semantic segmentation tasks:

• Overall Accuracy (OA): OA represents the overall classification accuracy across all pixels and is defined as

  $$OA={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

  (1)

  where TP, TN, FP and FN represent the numbers of true positives, true negatives, false positives, and false negatives, respectively.
• F1-score: The F1-score measures the model’s comprehensive ability to identify the positive class (i.e., water pixels) and is the harmonic mean of precision and recall, calculated as

  $$F1={\displaystyle \frac{2TP}{2TP+FP+FN}}$$

  (2)
• Intersection over Union (IoU): IoU represents the overlap between the predicted region and the ground truth, and is an important metric for evaluating performance in semantic segmentation tasks, defined as

  $$IoU={\displaystyle \frac{TP}{TP+FP+FN}}$$

  (3)
• Kappa Coefficient: The Kappa coefficient measures the agreement between classification results and ground truth while excluding the influence of random chance. It is calculated as

  $$Kappa={\displaystyle \frac{p_o-p_e}{1-p_e}}$$

  (4)

  where

  $$p_o={\displaystyle \frac{TP+TN}{TN+FP+FN+TP}},p_e={\displaystyle \frac{\left(TN+FP\right)\left(TN+FN\right)+\left(FN+TP\right)\left(FP+TP\right)}{{\left(TN+FP+TN+FN\right)}^2}}$$

All evaluation metrics were calculated on the same test set and compared with U-Net, U²-Net, TransUNet, and Swin-UNet.

3.2. Comparative Experiments

The proposed MSA-UNet model is compared with several widely used state-of-the-art models, including U-Net, U²-Net, TransUNet, and Swin-UNet, through comparative experiments conducted on three selected test regions. OA, F1-score, IoU, and the Kappa coefficient are employed as evaluation metrics to quantitatively assess the segmentation performance of each model.

The comparative results are illustrated in Figure 9, which presents the test images, corresponding ground truth water masks, and the prediction results of the five models for the same test regions. In the visualized results, white regions denote water bodies, while black regions represent non-water areas, facilitating intuitive observation of each model’s performance in terms of boundary delineation and detail preservation.

Figure 9. Prediction Results of Comparative Experiments.

Through systematic comparison and analysis across different test regions, the adaptability and robustness of the five models are evaluated, thereby demonstrating the effectiveness and superiority of the proposed MSA-UNet. According to the experimental results, MSA-UNet achieves consistently improved extraction performance compared to the other four models, producing predictions that are closest to the ground truth water masks, as summarized in Table 3.

Table 3. Accuracy Comparison in Comparative Experiments.

	OA	F1	IoU	Kappa
MSA-UNet	0.9846	0.9077	0.8311	0.8993
U-Net	0.9794	0.8823	0.7894	0.8710
U2-Net	0.9668	0.8281	0.7066	0.8101
TransUNet	0.9381	0.7257	0.5695	0.6934
Swin-UNet	0.9639	0.8165	0.6899	0.7970

3.3. Ablation Experiments

To validate the effectiveness of the multi-scale convolution module and the attention mechanism introduced in the proposed MSA-UNet model, an ablation study is conducted using four network variants: the baseline U-Net, U-Net with a spatial attention mechanism (U-Net + Attention), U-Net with a multi-scale convolution module (U-Net + Multiscale), and the complete MSA-UNet model. The ablation experiments are performed on the test set, where each test image has a spatial resolution of 500 × 500.

Figure 10 presents the test images, the corresponding ground truth labels, and the prediction results of the four models for the same test regions. The quantitative comparison of segmentation accuracy among the four models is summarized in Table 4.

Figure 10. Prediction Results of Ablation Experiments.

Table 4. Accuracy Comparison in Ablation Study.

	OA	F1	IoU	Kappa
U-Net	0.9794	0.8823	0.7894	0.8710
U-Net + Attention	0.9790	0.8801	0.7858	0.8686
U-Net + Multiscale	0.9831	0.8991	0.8167	0.8899
MSA-UNet	0.9846	0.9077	0.8311	0.8993

4. Discussion

As shown in Table 3, the comparative results indicate that MSA-UNet demonstrates significant performance advantages in water body extraction from remote sensing images containing shadows. MSA-UNet achieved an OA of 0.9846, an F1-score of 0.9077, an IoU of 0.8311, and a Kappa coefficient of 0.8993, all of which are clearly superior to the four classical semantic segmentation models: U-Net, U²-Net, TransUNet, and Swin-UNet. Among these metrics, F1-score and IoU are core indicators for evaluating target extraction accuracy and regional consistency. Compared with the second-best performing model, U-Net, MSA-UNet improved F1-score and IoU by 2.54% and 4.17%, respectively, demonstrating its stronger target segmentation capability and spatial consistency while maintaining high precision, effectively mitigating the influence of shadows in the imagery.

Further analysis of the prediction results shown in Figure 9 indicates that MSA-UNet can effectively suppress the interference from complex background factors, such as building shadows, and accurately extract continuous water body regions. Moreover, it demonstrates higher fidelity in recovering water body edge contours. In contrast, the Transformer-based models, TransUNet and Swin-UNet, exhibit relatively blurred edge information and are more susceptible to the influence of building shadows.

The red rectangular boxes in Figure 9 and Figure 10 are used to highlight regions where misclassification occurs in the comparative methods. These elongated or small lens-shaped features do not represent actual water bodies, but are mainly caused by building shadows or other non-water objects with similar spectral and spatial characteristics. In contrast, the proposed method does not produce such misclassifications in the same regions, indicating its improved robustness under shadow-affected conditions.

As shown in Table 4 and Figure 10, adding the multi-scale convolution module to the U-Net model significantly improves its prediction results compared with the baseline U-Net, with all accuracy metrics increased: OA improved by 0.37%, F1-score by 1.68%, IoU by 2.73%, and Kappa coefficient by 1.89%. This demonstrates that the multi-scale convolution module enhances the model’s ability to extract water bodies in the presence of building shadows.

Under the current experimental setting, when the attention mechanism was introduced into the U-Net model alone, the performance metrics slightly decreased, and the extraction results were somewhat worse than the baseline U-Net. However, the MSA-UNet model, which integrates both the multi-scale convolution module and the attention mechanism, outperformed the baseline U-Net across all metrics, achieving improvements of 0.52% in OA, 2.54% in F1-score, 4.17% in IoU, and 2.83% in Kappa coefficient. This indicates that introducing the attention mechanism alone may suppress effective features and inadvertently emphasize non-informative features in areas affected by building shadows. In MSA-UNet, the attention mechanism operates on the multi-scale features, effectively selecting key regions and mitigating the influence of shadows. The multi-scale convolution module provides rich feature representations, while the attention mechanism optimizes feature selection, forming a complete “feature generation + selection” process.

Thus, the two modules introduced on top of U-Net work collaboratively to improve extraction accuracy and effectively suppress the influence of building shadows and vegetation. By integrating multi-scale feature extraction and attention mechanisms while maintaining a lightweight structure, MSA-UNet achieves a good balance between semantic perception and localization accuracy, demonstrating its effectiveness and robustness in processing large, heterogeneous targets such as water bodies in remote sensing scenarios, and showing promising application potential. Overall, MSA-UNet shows consistent performance improvements across all test areas, although the magnitude of the gains varies slightly among different scenes.

The experiments in this paper were conducted using a single study area. This area includes urban regions affected by building shadows, as well as water bodies of varying shapes and sizes, providing a representative and challenging experimental setting for evaluating the robustness of water body extraction methods under shadow-affected conditions. The primary focus of this study is to validate the effectiveness of the proposed MSA-UNet in suppressing shadow-induced interference and improving water body segmentation accuracy. However, beyond the characteristics of the study area itself, uncertainty in the manually annotated ground-truth data should also be considered when interpreting the experimental results.

Although the dataset size used in this study is sufficient to support model training and evaluation, potential labeling uncertainty should be acknowledged. The ground-truth water body masks were manually annotated, and ambiguities may arise during the labeling process, particularly in regions affected by building shadows, mixed pixels, and fuzzy water boundaries. Such labeling inaccuracies may introduce noise into the training process and consequently lead to false positives or missed detections in the model predictions. This labeling uncertainty may, to some extent, influence the evaluation results, and therefore should be taken into consideration when interpreting the model performance.

Nevertheless, several limitations of the MSA-UNet model remain. While it performs well in segmenting medium to large water bodies, its ability to extract small or narrow water bodies in high-resolution images is still limited. For complex water structures such as narrow channels and small ponds, missed detections may occur, and boundary delineation is less precise, indicating challenges in preserving fine spatial details. In addition, due to limitations in the size and diversity of the training data, the model’s performance stability across different sensors or geographic regions remains to be further validated.

All experiments in this paper were conducted using a fixed training–validation–test split to ensure consistency and comparability among different models. All comparative methods were trained and evaluated under the same experimental settings and data partitioning scheme, which helps reduce potential bias introduced by random initialization. While this experimental design enables a fair comparison across models, cross-validation or repeated experiments were not performed, and thus random effects cannot be completely eliminated. This issue will be further investigated in future work.

5. Conclusions

This paper focuses on the automatic extraction of water bodies from high-resolution remote sensing imagery and proposes an improved semantic segmentation model, termed MSA-UNet. By integrating multi-scale feature extraction and attention mechanisms into the U-Net architecture, the proposed model effectively enhances the representation of water bodies at different spatial scales and significantly improves segmentation robustness in scenes with prominent non-water interference, particularly those affected by building shadows.

Experimental results across multiple test areas demonstrate that MSA-UNet consistently outperforms U-Net, U²-Net, TransUNet, and Swin-UNet in terms of OA, F1-score, IoU, and Kappa coefficient. In regions with evident shadow interference, the proposed method effectively reduces misclassification of non-water objects and produces extraction results that are more consistent with the ground-truth water body distributions. Ablation experiments further confirm that the collaborative effect of the multi-scale module and the attention mechanism is the key factor driving the performance improvements.

Although the proposed model exhibits strong practical effectiveness, limitations remain in extracting small and elongated water bodies, as well as in generalization across different sensors and geographic regions. Future work will focus on enhancing small-target feature representation and improving cross-sensor generalization to further extend the applicability of the proposed approach.