DSFNet: A Directional Statistical Fusion Network for Cloud and Cloud Shadow Segmentation

Highlights

What are the main findings?

• DSFNet integrates directional structural enhancement, robust statistical modulation, and adaptive grouped multi-scale fusion for joint cloud and cloud shadow segmentation.
• By incorporating directional refinement and adaptive grouped multi-scale fusion, DSFNet handles elongated shadow structures and achieves 76.97% mIoU on the GF1_WHU dataset, a 1.51% improvement over the DeepLabV3+ baseline.

What are the implications of the main findings?

• The proposed framework demonstrates the value of combining directional structural priors with robust statistical modeling for remote sensing segmentation under heterogeneous background interference.
• By preserving boundary details and semantic consistency, DSFNet enables more reliable preprocessing for downstream agricultural and climate monitoring.

Abstract

Accurate cloud and cloud shadow segmentation is a critical prerequisite for remote sensing image preprocessing. However, this task remains challenging due to the directional continuity of projected cloud shadows, the radiometric ambiguity between low-reflectance shadows and other dark surfaces, and the difficulty of preserving semantic consistency and fine boundaries in complex scenes. To address these issues, this paper proposes a Directional Statistical Fusion Network (DSFNet) based on an enhanced DeepLabV3+ architecture. Specifically, a Directional Scale Refinement Module (DSRM) is introduced in parallel with Atrous Spatial Pyramid Pooling to strengthen the representation of direction-sensitive cloud-shadow structures and multi-scale cloud regions. An Adaptive Statistical Context Attention (ASCA) module is further designed to perform robust feature modulation by jointly exploiting global statistics, edge-aware statistics, and median-based normalization, thereby suppressing anomalous responses under heterogeneous backgrounds. In the decoder, an Adaptive Grouped Multi-scale Fusion (AGMF) module is employed to adaptively fuse shallow detail features and high-level semantic features through discrepancy-guided grouped gating, improving structural consistency and boundary recovery. In addition, a hybrid loss is adopted to further optimize segmentation. Experiments on the GF1_WHU dataset show that DSFNet achieves 76.97% mIoU, demonstrating strong effectiveness and robustness in complex remote sensing scenes.

1. Introduction

With the rapid advancement of remote sensing technologies, optical imagery has become indispensable for global terrestrial monitoring. However, cloud cover and the associated shadows frequently obscure the land surface, degrade radiometric consistency, and reduce the reliability of subsequent image interpretation. Such interference poses substantial challenges to downstream applications, including land cover mapping, object detection, and environmental monitoring. Therefore, accurate segmentation of clouds and cloud shadows has become a fundamental prerequisite for remote sensing image preprocessing, as it is essential for mitigating atmospheric contamination and improving the usability and analytical reliability of remote sensing data in applications such as agricultural monitoring, solar energy assessment, and climate change research.

Traditional cloud and cloud shadow detection methods primarily rely on physical models and statistical features, which can generally be categorized into three main categories. The first category consists of threshold-based detection methods, which construct physical discrimination rules based on radiometric characteristics in the visible and thermal infrared bands [1,2]. These methods are simple and efficient, but they are often sensitive to thin clouds and high-reflectance backgrounds. The second category comprises geometry- and texture-based detection methods, which aim to mitigate spectral confusion by incorporating spatial geometric relationships and textural information [3,4]. However, their performance is usually unstable in complex scenes with weak boundaries or heterogeneous surface backgrounds. The third category includes time-series and statistical-learning-based methods, which enhance robustness by exploiting multi-temporal variation or statistical classifiers [5,6,7]. Despite their improved robustness, time-series and statistical-learning-based methods still depend heavily on handcrafted features and prior assumptions [8], making it difficult to capture complex structures and fine boundary details.

Since 2015, deep learning has fundamentally reshaped semantic segmentation in remote sensing. Early representative models, such as the Fully Convolutional Network (FCN) [9], established the paradigm of end-to-end pixel-wise prediction, while subsequent architectures mainly advanced along three directions: preserving spatial details, strengthening multi-scale contextual modeling, and capturing long-range dependencies. Specifically, encoder–decoder networks such as U-Net [10] and SegNet [11], as well as high-resolution architectures such as HRNet [12], emphasized the recovery and preservation of fine spatial structures. To better handle scale variations and contextual aggregation, the DeepLab series [13,14,15,16] and PSPNet [17] introduced dilated convolutions, ASPP, and pyramid pooling to enlarge receptive fields and enhance multi-scale representation. More recently, Transformer-based models, including Swin-Unet [18], SegFormer [19], and DA-TransUNet [20], have further strengthened global dependency modeling through self-attention mechanisms. These advances have laid an important foundation for cloud and cloud shadow segmentation in remote sensing, where the targets often exhibit large scale variation, weak boundaries, and strong interference from complex land-surface backgrounds.

In recent years, increasing attention has been paid to the integration of direction-aware enhancement, attention-based modulation, and cross-scale feature fusion for cloud and cloud shadow segmentation in remote sensing imagery. For instance, MSFANet [21] improves the anisotropic structural representation of cloud shadows through multi-scale strip pooling, while AGASMA [22] introduces axial hybrid attention within a parallel CNN–Transformer framework to enhance fine-grained segmentation. In terms of multi-scale and multi-modal interaction, HyCloudX [23], Wang et al. [24], and Feng et al. [25] further improve segmentation performance from the perspectives of spectral–spatial collaboration, channel–spatial attention, and adaptive feature fusion, respectively. Collectively, these studies indicate the importance of direction-sensitive structural representation, robust feature modulation, and effective cross-level interaction.

Nevertheless, several critical challenges remain in joint cloud and cloud shadow segmentation. First, cloud shadows are not merely dark regions, but projection-induced structures formed under solar illumination. Their spatial morphology is closely related to solar elevation, solar azimuth, and local projection geometry, and therefore often exhibits strong structural continuity along specific directions [3,26,27]. Meanwhile, cloud bodies usually contain multi-scale textures and irregular local boundaries, making it difficult for mainstream isotropic convolutions and pooling operations to simultaneously preserve cloud details and shadow continuity. Second, from a radiometric perspective, cloud shadows usually appear as low-reflectance regions and are easily confused with water bodies, building shadows, and other dark surfaces [28], while snow, bright buildings, and highly reflective water may introduce anomalous responses. Consequently, attention mechanisms based on mean statistics remain sensitive to noise and outliers, leading to weight bias and misclassification in high-interference backgrounds [29,30]. Third, in the decoding stage, conventional multi-scale fusion strategies still rely mainly on direct concatenation or summation, without explicitly exploiting discrepancy information between features of different levels to guide aggregation, which may result in boundary degradation and structural discontinuity [31,32]. Therefore, a more task-oriented framework is still needed to jointly address projection-aware structural modeling, robust radiometric modulation, and selective cross-scale aggregation in a unified manner.

These observations directly motivate the proposed framework. Specifically, the direction-sensitive continuity of cloud-shadow projection motivates the Directional Scale Refinement Module (DSRM), the need to suppress radiometric ambiguity and anomalous responses motivates the Adaptive Statistical Context Attention (ASCA), and the limitation of direct cross-scale aggregation motivates the Adaptive Grouped Multi-scale Fusion (AGMF) in the decoder. From this perspective, DSFNet is not a simple combination of existing techniques, but a task-oriented framework that reorganizes these three methodological cues around the specific challenges of joint cloud and cloud shadow segmentation. On this basis, this paper proposes a Directional Statistical Fusion Network (DSFNet) for precise cloud and cloud shadow segmentation in complex remote sensing imagery. The main contributions of this work are summarized as follows:

• We propose a Directional Statistical Fusion Network that enhances directional structural representation and improves statistical robustness, thereby alleviating cross-scale feature imbalance and background interference in cloud and cloud-shadow segmentation.
• A Directional Scale Refinement Module is designed to enhance direction-aware feature modeling and improve representation of directionally continuous cloud shadows and anisotropic cloud structures without compromising multi-scale receptive fields.
• The Adaptive Statistical Context Attention module is developed based on a median-centered robust modulation principle to suppress noise-sensitive feature responses in complex remote sensing environments.
• To mitigate cross-scale feature imbalance, an Adaptive Grouped Multi-scale Fusion module is introduced to facilitate fine-grained fusion between deep semantic abstractions and shallow spatial details.
• The training strategy integrates a hybrid loss function combining Logit-Adjusted Focal Cross-Entropy and Multi-class Focal Tversky Loss. This combination guides the model to prioritize cloud shadow boundaries and minority classes, reducing omission errors and boosting overall accuracy.

2. Materials and Methods

2.1. Network Structure

The overall architecture of the proposed DSFNet is illustrated in Figure 1. In the encoding stage, the backbone first extracts hierarchical features from the input image, where shallow features preserve rich boundary, texture, and local semantic information, while high-level features provide stronger abstract semantic representations. The high-level features are then fed into the ASPP module for multi-scale context aggregation, while a Directional Scale Refinement Module (DSRM) is introduced in parallel to further enhance the representation of elongated cloud-shadow structures and multi-scale cloud regions. The aggregated features are subsequently refined by the Adaptive Statistical Context Attention (ASCA) module, which suppresses anomalous responses under complex backgrounds and improves the stability of semantic features. In the decoding stage, instead of directly concatenating multi-level features, the network employs an Adaptive Grouped Multi-scale Fusion (AGMF) module to spatially align features and perform discrepancy-guided fusion of shallow detail features, high-level semantic features, and the enhanced semantic feature from the encoder. On this basis, the fused semantic feature produced by AGMF is further concatenated with the shallow feature to complement stable high-level semantics with low-level boundary and texture details, thereby reinforcing the structural consistency of cloud-shadow regions while preserving fine details, and finally generating the cloud and cloud shadow segmentation result.

The proposed framework is built upon two closely related ideas, namely directional modeling and statistical fusion. In DSRM, directional modeling does not simply refer to elongated objects, but to the representation of 2D spatial context as aggregated responses along predefined directional bases, thereby capturing direction-sensitive structural continuity in the feature space. This is particularly relevant to cloud shadows, whose spatial morphology is closely related to solar illumination and projection geometry and therefore often exhibits locally extended and direction-consistent structures. For ASCA, statistical fusion is reflected in robust feature modulation through the joint use of global statistics, edge-aware statistics, and median-based location estimation, which reduces the sensitivity of mean-dominated responses to outliers and anomalous background interference. Furthermore, AGMF extends statistical fusion to cross-level feature aggregation by explicitly exploiting discrepancy information between features of different levels to guide adaptive fusion, rather than relying solely on direct concatenation or summation. This design is consistent with the radiometric characteristics of remote sensing imagery, where cloud shadows usually appear as low-reflectance regions and are easily confused with other dark surfaces, while strong reflections from bright objects may introduce anomalous responses. By integrating direction-aware structural enhancement, robust statistical modulation, and discrepancy-guided cross-scale fusion, DSFNet improves the discrimination of cloud and cloud-shadow regions in complex scenes.

2.2. Directional Scale Refinement Module (DSRM)

In cloud and cloud shadow segmentation, cloud shadows often exhibit direction-sensitive structural continuity due to projection geometry, while cloud bodies usually involve multi-scale variations and irregular local textures. Such structural heterogeneity makes it difficult for conventional isotropic feature extraction to simultaneously preserve shadow continuity and cloud details.

Although ASPP enlarges the receptive field through multi-scale atrous convolutions, its sampling pattern remains spatially sparse. For elongated shadows and low-contrast transition regions, such sparse sampling may produce discontinuous feature responses, thereby leading to fragmented shadow boundaries or local missed detections. To alleviate this limitation without disturbing the original multi-scale receptive-field distribution, we introduce a Directional Scale Refinement Module as a parallel branch to ASPP, as shown in Figure 2. DSRM is designed to supplement ASPP with explicit directional context modeling and complementary local multi-scale refinement.

Formally, for an input feature map $X\in {\mathbb{R}}^{B\times C\times H\times W}$, its directional representation is defined as:

$$D\left(X\right)=\left\{P_{\theta }\right(X)\mid \theta \in \mathsf{\Theta }\}$$

(1)

where $\mathsf{\Theta }=\{0, \pi /2\}$ denotes the predefined orthogonal directional bases corresponding to the horizontal and vertical directions, and $P_{\theta }$ denotes the strip-wise aggregation operator along direction $\theta$. Unlike conventional anisotropic convolutions that still operate within local receptive fields, this formulation performs global aggregation along one spatial dimension and therefore emphasizes long-range directional structural continuity. From a physical perspective, DSRM does not explicitly encode solar elevation or azimuth angles, but implicitly responds to the projection-induced structural regularity of cloud shadows in image space.

Figure 3 illustrates the complementary multi-scale feature refinement branch used in DSRM. Although directional aggregation is effective in capturing long-range structural continuity, it is insufficient for fully describing local boundary transitions, fragmented textures, and scale variations. To compensate for these missing cues, a multi-scale feature refinement branch [33] is first introduced to generate the complementary feature $F_m$. This branch employs two parallel depthwise convolution paths with kernel sizes $3\times 3$ and $7\times 7$ to capture local and mesoscale contexts, respectively, and fuses their responses to form $F_m$.

With this local complement, the directional context branch shown in Figure 2 extracts orientation-sensitive responses through Strip Average Pooling and Strip Maximum Pooling along the horizontal and vertical directions. Specifically, aggregation along width $W$ yields horizontal strip descriptors:

$$s_h^{avg}\left(c,h,1\right)={\displaystyle \frac{1}{W}}\sum _{w=1}^W X\left(c,h,w\right),$$

(2)

$$s_h^{max}\left(c,h,1\right)=\underset{w}{max} X\left(c,h,w\right)$$

(3)

while aggregation along height $H$ yields vertical strip descriptors:

$$s_w^{avg}\left(c,1,w\right)={\displaystyle \frac{1}{H}}\sum _{h=1}^H X\left(c,h,w\right)$$

(4)

$$s_w^{max}(c,1,w)=\underset{h}{max} X(c,h,w)$$

(5)

These strip-wise responses are concatenated and projected to produce the directional context feature $F_d$, which mainly captures long-range structural continuity and directional consistency:

$$F_d=\varphi \left(BN\left(Con{v}_{1\times 1}\left(Concat\left[\mathcal{E}\left(s_h^{avg}\right),\mathcal{E}\left(s_h^{max}\right),\mathcal{E}\left(s_w^{avg}\right),\mathcal{E}\left(s_w^{max}\right)\right]\right)\right)\right)$$

(6)

Finally, the directional context feature $F_d$ is fused with the multi-scale refined feature $F_m$ through channel concatenation and a $1\times 1$ convolution, followed by residual addition with the input feature:

$$F_{\mathrm{o}\mathrm{u}\mathrm{t}}={\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}}_{1\times 1}\left(\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}\right[F_d,F_m\left]\right)+X$$

(7)

In this way, DSRM establishes a direction-aware structural prior for cloud-shadow projection and complements it with local multi-scale refinement. By jointly enhancing global directional continuity and local structural variability, the proposed module effectively mitigates the discontinuity introduced by sparse atrous sampling, while improving shadow continuity preservation and cloud-detail representation in complex remote sensing scenes.

2.3. Adaptive Statistical Context Attention Module (ASCA)

Traditional attention mechanisms usually rely on global averaging to estimate feature importance. However, in remote sensing imagery, mean-based statistics are highly sensitive to radiometric ambiguity and extreme responses. In particular, low-reflectance cloud shadows are easily confused with other dark surfaces, while highly reflective objects may significantly bias the global mean, which is especially detrimental to thin clouds, faint shadows, and low-contrast transition regions. To alleviate this limitation, we introduce an Adaptive Statistical Context Attention module. The overall architecture of ASCA and the LGCG operator are illustrated in Figure 4 and Figure 5, respectively. Instead of relying on a single mean-based descriptor, ASCA incorporates multiple statistical cues and a median-centered robust spatial reference [34,35] to construct more stable and discriminative attention responses under complex radiometric conditions.

The LGCG operator first generates a statistically guided channel descriptor. Given an input feature tensor $\mathrm{F}\in {\mathbb{R}}^{B\times C\times H\times W}$, global semantic statistics are first extracted by Global Average Pooling:

$$g_{\mathrm{g}\mathrm{l}\mathrm{o}\mathrm{b}\mathrm{a}\mathrm{l}}=\mathrm{G}\mathrm{A}\mathrm{P}\left(F\right)\in {\mathbb{R}}^{B\times C}$$

(8)

To further incorporate structural variation cues, a depthwise convolution is applied to obtain a smoothed response, and the absolute difference between the input and the smoothed feature is then used to characterize edge-aware variation:

$$F_{edge}=\left|F-{DWConv}_{3\times 3}\left(F\right)\right|$$

(9)

$$g_{\mathrm{e}\mathrm{d}\mathrm{g}\mathrm{e}}=\mathrm{G}\mathrm{A}\mathrm{P}\left(F_{\mathrm{e}\mathrm{d}\mathrm{g}\mathrm{e}}\right)\in {\mathbb{R}}^{B\times C}$$

(10)

The two statistics are then fused to generate a channel-wise guidance vector:

$$\mathrm{g}=\sigma \left({\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}}_{1\times 1}\left({\mathrm{g}}_{\mathrm{g}\mathrm{l}\mathrm{o}\mathrm{b}\mathrm{a}\mathrm{l}}+{\mathrm{g}}_{\mathrm{e}\mathrm{d}\mathrm{g}\mathrm{e}}\right)\right),\mathrm{g}\in (\mathrm{0,\; 1}{)}^{B\times C}$$

(11)

where $g\in (\mathrm{0,1}{)}^{B\times C}$. In this way, LGCG jointly encodes global semantic context and local variation information, making the channel guidance less susceptible to radiometric bias caused by complex backgrounds.

Based on the guidance signal produced by LGCG, ASCA further performs robust spatial modulation, as shown in Figure 4. The guided response $Q_{guided}$ is obtained by modulating the depthwise-convolved feature with the LGCG-generated channel guidance vector $g$. The spatial saliency map $W$ is then obtained by channel-wise aggregation:

$$W=\sum _{c=1}^C Q_{guided}^{\left(c\right)}$$

(12)

To suppress the influence of extreme responses, ASCA introduces a median-based spatial normalization strategy. Specifically, the saliency map is flattened along the spatial dimension, and its median is used as a robust location statistic:

$$m_{map}=\mathrm{M}\mathrm{e}\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{n}\left(W\right)\in {\mathbb{R}}^{B\times 1\times 1\times 1}$$

(13)

where $\mathrm{M}\mathrm{e}\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{n}(\cdot )$ denotes median estimation over the flattened spatial dimension. The normalized spatial attention is then formulated as:

$$\alpha =\sigma \left(\beta (W-m_{map})\right)$$

(14)

where $\beta$ is a learnable scaling factor and $\alpha \in (\mathrm{0,1}{)}^{B\times 1\times H\times W}$ denotes the robust spatial attention map. Compared with conventional average-based attention, this design is less sensitive to outliers and therefore more suitable for remote sensing scenes with radiometric heterogeneity.

Finally, the channel guidance and the robust spatial attention are jointly used to modulate the feature response, followed by a $1\times 1$ convolution and residual addition with the input feature to obtain the output of ASCA.

$$Y=F+\gamma \cdot {\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}}_{1\times 1}(V\odot \alpha \odot g)$$

(15)

where $V$ denotes the depthwise-convolved feature response and $\gamma$ is a learnable scaling coefficient that balances the contributions between the attention branch and the identity pathway.

By jointly exploiting global statistics, edge-aware statistics, and median-based robust normalization, ASCA effectively reduces the ambiguity between cloud shadows and other dark surfaces, suppresses interference from extreme radiometric responses, and enhances cloud-shadow discrimination in complex remote sensing backgrounds.

2.4. Adaptive Grouped Multi-Scale Fusion Module (AGMF)

In the decoding stage, accurate cloud and cloud shadow segmentation requires both stable semantic abstraction and sufficient boundary detail. However, these cues are not equally preserved across feature levels: shallow features retain richer local structures and boundary information, whereas deeper features encode more reliable semantic context. Conventional decoding strategies usually merge them through direct concatenation or summation, which may obscure the discrepancy between semantic abstraction and local texture responses. In complex remote sensing scenes, this cross-level discrepancy is especially informative for ambiguous cloud-shadow regions. Therefore, rather than relying on uniform fusion, the decoder should explicitly exploit such discrepancy as a statistical indicator to guide adaptive aggregation, so as to reduce ambiguity and preserve structural integrity.

To address this issue, we introduce an Adaptive Grouped Multi-scale Fusion Module, whose overall structure is shown in Figure 6. Let the deep semantic feature refined by ASCA and the subsequent $1\times 1$ convolution be denoted as $F_{sem}$, the high-level feature from the backbone be denoted as $F_{high}$, and the shallow feature be denoted as $F_{low}$. To facilitate pixel-level fusion, taking the spatial resolution of the shallow feature as the baseline, $F_{sem}$ and $F_{high}$ are first upsampled to ${\displaystyle \frac{H}{4}}\times {\displaystyle \frac{W}{4}}$ by bilinear interpolation. Subsequently, a $1\times 1$ convolution is applied to project the three branches into the same channel dimension $C$, yielding ${\widehat{F}}_{sem},{\widehat{F}}_{high},{\widehat{F}}_{low}\in {\mathbb{R}}^{B\times C\times {\displaystyle \frac{H}{4}}\times {\displaystyle \frac{W}{4}}}$.

To avoid redundant coupling caused by full-channel fusion [36], the concatenated feature $X\in {\mathbb{R}}^{3C\times H\times W}$ is processed by grouped convolution. Formally, $X$ is split into $G$ channel groups $X=\{X_1,X_2,\dots ,X_G\}, X_g\in {\mathbb{R}}^{{\displaystyle \frac{3C}{G}}\times H\times W},$ and each group is independently transformed as:

$$Y_g=\varphi \left({\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}}^{\left(g\right)}\left(X_g\right)\right)$$

(16)

The grouped responses are then concatenated to form the fused representation, thereby reducing redundant channel coupling. The grouped convolution structure used in this work is illustrated in Figure 7.

Based on the grouped-convolution design, an Adaptive Grouped Gating (AGG) unit is introduced as the basic fusion operator in AGMF. For each dual-input feature pair, AGG is responsible for performing discrepancy-aware adaptive gating before the subsequent multi-branch aggregation. Its detailed procedure is summarized in Algorithm 1.

Algorithm 1 Adaptive Grouped Gating (AGG)

Input: $H,L\in {\mathbb{R}}^{C\times H\times W}$

Output: $F_{agg}$

1: Compute discrepancy $D\leftarrow \mid H-L\mid$

2: Concatenate $X\leftarrow \mathrm{C}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}[H,L,D]$

3: Apply grouped $3\times 3$ convolution to obtain local grouped representation

4: Apply grouped $1\times 1$ convolution and Sigmoid to generate gate $\beta$

5: Fuse the two inputs as

$F_{agg}\leftarrow \beta \odot H+(1-\beta )\odot L$

6: Return $F_{agg}$

Accordingly, AGMF performs pairwise fusion on three aligned feature pairs: $AGG\left({\widehat{F}}_{low},{\widehat{F}}_{high}\right)$, $AGG\left({\widehat{F}}_{low},{\widehat{F}}_{sem}\right)$, $AGG\left({\widehat{F}}_{high},{\widehat{F}}_{sem}\right)$. The three fusion outputs are then concatenated along the channel dimension and compressed by a $1\times 1$ convolution:

$$F_{mix}=\varphi \left({\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}}_{1\times 1}\left(\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}\right({AGG\left({\widehat{F}}_{low},{\widehat{F}}_{high}\right)}_1,AGG\left({\widehat{F}}_{low},{\widehat{F}}_{sem}\right),AGG\left({\widehat{F}}_{high},{\widehat{F}}_{sem}\right)\left)\right)\right)$$

(17)

where $\varphi (\cdot )$ denotes a nonlinear transformation consisting of Batch Normalization followed by ReLU activation. To further enhance local spatial consistency, a depthwise separable $3\times 3$ convolution is applied to obtain the final output:

$$F_{out}=\varphi \left(DWCon{v}_{3\times 3}\left(F_{mix}\right)\right)$$

(18)

In this way, AGMF can be understood as a discrepancy-guided cross-scale fusion mechanism. By explicitly introducing cross-level discrepancy into the gating process, the proposed module better preserves semantic consistency and local structural details, while reducing the ambiguity between low-reflectance cloud shadows and visually similar dark surfaces in complex remote sensing scenes.

2.5. Loss Function

In cloud and cloud shadow segmentation tasks, cloud shadow pixels are typically far fewer than background pixels, and their boundaries exhibit characteristics of weak contrast and elongated structures. Such significant class imbalance and boundary blurriness tend to bias the model toward the background class during training, subsequently leading to missed detections of small-scale shadow targets and structural fragmentation. To address the aforementioned issues, this paper proposes a hybrid loss function that balances pixel-level discriminative stability and region-level structural consistency. This function is composed of a weighted combination of the Logit-Adjusted Focal Cross-Entropy Loss [37] and the multi-class Focal Tversky Loss (mFTL) [38]. For convenience, the proposed hybrid loss is denoted as FJFL.

At the pixel level, to alleviate the prior bias caused by the long-tail distribution, Logit prior adjustment is introduced into the Softmax layer, combined with a Focal mechanism to mine hard-to-classify samples. The pixel-level loss is defined as follows:

$$L_{LA-FocalCE}=-\sum _{k=1}^C{\eta }_k{y}_k(1-p_k{)}^{\gamma }\mathit{log}(p_k)$$

(19)

where $C$ denotes the total number of classes, $y_k\in \left\{\mathrm{0,\; 1}\right\}$ represents the ground-truth label for class $k$, and ${\eta }_k$ is the class balancing weight. The focusing parameter $\gamma$ controls the penalty for hard-to-classify samples. The probability $p_k$ is computed based on the adjusted logit $z_k^{\prime }=z_k+\tau \mathit{log}({\pi }_k)$, where $z_k$ is the original network output logit, ${\pi }_k$ represents the empirical class frequency, and $\tau$ is the temperature scaling factor. By explicitly correcting the class priors, this mechanism effectively enhances the model’s discriminative capability for minority-class cloud shadow pixels.

At the region level, to further reduce the missed detection rate of cloud shadows and maintain the integrity of elongated structures, this paper introduces mFTL as a region-level constraint. By adjusting the penalty weights (${\alpha }_k,{\beta }_k$) for false positives and false negatives in the Tversky index and incorporating a focusing parameter, this loss function is able to concentrate more on difficult regions with low overlap:

$$L_{mFTL}=\sum _{k=1}^C{w}_k{\left(1-{\displaystyle \frac{T{P}_k}{T{P}_k+{\alpha }_{k}F{P}_k+{\beta }_{k}F{N}_k}}\right)}^{\gamma }$$

(20)

where $w_k$ is the assigned weight for class $k$. $T{P}_k$, $F{P}_k$, and $F{N}_k$ denote the number of true positive, false positive, and false negative pixels for class $k$, respectively. The exponent $\gamma$ further scales the loss to emphasize difficult-to-segment regions.

The final total loss function is integrated via a weighted fusion controlled by a balancing coefficient $\lambda$:

$$L=\lambda L_{LA-FocalCE}+(1-\lambda )L_{mFTL}$$

(21)

where $\lambda$ serves as the balancing coefficient used to adjust the contribution ratio between pixel-level supervision and region-level supervision. This hybrid loss guarantees training stability while significantly improving the recall rate of cloud shadow regions.

During the experiments, multiple hyperparameter combinations were tested, as shown in

Table 1. Segmentation performance of the proposed model under different hyperparameter $\lambda$ values.

$\mathit{\lambda }$	MPA (%)	MIoU (%)
0.2	84.98	76.21
0.4	85.24	76.46
0.6	85.42	76.97
0.8	85.13	76.48

Note: Bold indicates the best result.

. The best segmentation performance was observed when $\lambda = 0.6$. Therefore, this parameter setting was adopted for subsequent experiments.

3. Experiments and Analysis

3.1. Experimental Datasets

To comprehensively evaluate the performance and generalization capability of the proposed DSFNet in pixel-level cloud and cloud shadow semantic segmentation tasks, extensive experimental analyses were conducted on three publicly available benchmark datasets: GF1_WHU, HRC_WHU and Cloud and Cloud Shadow Dataset. These datasets present significant challenges, including multi-scale targets, complex surface backgrounds, and variable lighting conditions, making them highly valuable for assessing the robustness of our network.

The GF1_WHU dataset, produced by the RS IDEA team at Wuhan University, serves as the primary benchmark for this study. It contains 108 Level-2A images captured by the Wide Field of View (WFV) sensor onboard the Gaofen-1 (GF-1) satellite. These images feature a spatial resolution of 16 m and encompass four multispectral bands: red, green, blue, and near-infrared. The dataset provides extensive global coverage across diverse land cover types—including urban areas, water bodies, vegetation, and deserts—under varying cloud cover conditions. The ground truth masks were meticulously annotated by experts into three distinct categories: background (0), cloud shadow (128), and cloud (255). The diversity of cloud morphologies and the potentially confusing background features make it an ideal standard for validating segmentation accuracy. Representative samples and labels are illustrated in Figure 8.

Due to GPU memory constraints, the original large-scale images were divided into non-overlapping patches of size 256 × 256, and patches containing invalid background or severe blur were discarded. The resulting patches were then randomly divided into training, validation, and test subsets with a ratio of 8:1:1. To enhance robustness to multi-scale and multi-directional cloud structures, data augmentation techniques, including horizontal flipping, vertical flipping, and random rotation, were applied during training. Ultimately, 5428 patches were used for training, 680 for validation, and 680 for testing. The validation set was used for hyperparameter tuning and model selection, while the test set was reserved exclusively for the final performance evaluation.

To evaluate the generalization robustness of the model across multi-source data, this paper further employed the HRC_WHU dataset and the Cloud and Cloud Shadow Dataset. The HRC_WHU dataset consists of 150 high-resolution RGB images sourced from Google Earth, with spatial resolutions ranging from 0.5 to 15 m. It covers five typical land surface types: water, vegetation, urban, ice/snow, and bare land, which often exhibit spectral similarities to clouds, thereby increasing segmentation difficulty. This dataset primarily provides binary annotations for cloud and non-cloud regions. Following the same protocol, the original images were cropped into 3200 patches of 256 × 256 pixels and then divided into 2560 training patches, 320 validation patches, and 320 test patches. The Cloud and Cloud Shadow Dataset, also sourced from Google Earth, comprises high-resolution remote sensing imagery collected by professional meteorologists across various geographical regions, including the Yunnan-Guizhou Plateau, the Qinghai–Tibet Plateau, and the Yangtze River Delta. It covers five typical backgrounds: water, forest, farmland, towns, and deserts, with refined manual annotations for clouds, cloud shadows, and background. Considering the larger variation in spatial extent and aspect-ratio distribution of the original images in this dataset, as well as the fact that the effective target regions are relatively concentrated in some samples, the images were cropped into 224 × 224 patches rather than 256 × 256. In our preliminary preprocessing comparisons, 256 × 256 patches were observed to introduce more boundary redundancy and lower effective region utilization for this dataset, which was particularly unfavorable for preserving thin clouds and subtle edge details. After removing invalid samples, a total of 2560 patches were obtained and further divided into 2048 training patches, 256 validation patches, and 256 test patches. It should be emphasized that, within each dataset, all compared methods used exactly the same patch size and preprocessing strategy to ensure a fair comparison.

3.2. Experimental Details

All experiments were conducted using the PyTorch 2.2.2 framework on an NVIDIA RTX 4090 GPU, with CUDA 11.8 for acceleration. For training, this paper employed the Adam optimizer due to its superior convergence properties. The initial learning rate was set to 0.001, utilizing a cosine annealing strategy for dynamic decay to facilitate model convergence. The learning rate was adjusted according to the following formula:

$${\eta }_t={\eta }_{min}+{\displaystyle \frac{1}{2}}\left({\eta }_{max}-{\eta }_{min}\right)\left(1+\cos \left({\displaystyle \frac{t}{T_{max}}}\pi \right)\right)$$

(22)

where ${\eta }_{max}$ denotes the initial learning rate, the minimum learning rate ${\eta }_{min}$ is set to $0.01{\eta }_{max}$, and $T_{max}$ represents the total number of training epochs, and $t$ denotes the current epoch. Considering hardware memory limits and convergence characteristics, the batch size was set to 16, and the model was trained for a total of 200 epochs. Notably, the proposed DSFNet was optimized using a hybrid loss function, while compared models utilized standard cross-entropy loss. During training, hyperparameters were determined based on the validation set, and the checkpoint achieving the best validation MIoU was selected as the final model for test evaluation.

To quantitatively evaluate the segmentation accuracy and computational efficiency of DSFNet across the three datasets, seven core metrics were selected: Pixel Accuracy (PA), Mean Pixel Accuracy (MPA), F1-score, Mean Intersection over Union (MIoU), Frequency Weighted Intersection over Union (FWIoU), single-frame inference time (Time) and Floating-Point Operations (FLOPs).

Calculation variables were defined based on the confusion matrix: for a dataset with $k+1$ classes (including background), $p_{i,j}$ denotes the number of pixels belonging to class $i$ but predicted as class $j$. Therefore, for a specific class $i$, $p_{i,i}$ represents the true positives (TP). The false positives (FP) and false negatives (FN) are calculated as $\sum _{j\ne i} p_{j,i}$ and $\sum _{j\ne i} p_{i,j}$, respectively.

PA is defined as the ratio of correctly classified pixels to the total pixels:

$$PA={\displaystyle \frac{\sum _{i=0}^k p_{i,i}}{\sum _{i=0}^k \sum _{j=0}^k p_{i,j}}}$$

(23)

In pixel-level semantic segmentation, PA is essentially equivalent to Overall Accuracy (OA), as both quantify the proportion of correctly classified pixels over the entire image.

MPA calculates the average accuracy across all categories:

$$MPA={\displaystyle \frac{1}{k+1}}\sum _{i=0}^k {\displaystyle \frac{p_{i,i}}{\sum _{j=0}^k p_{i,j}}}$$

(24)

F1-score is the harmonic mean of Precision (P) and Recall (R):

$$P={\displaystyle \frac{TP}{TP+FP}}$$

(25)

$$R={\displaystyle \frac{TP}{TP+FN}}$$

(26)

$$\begin{array}{c}{F}_1=2\times {\displaystyle \frac{P\times R}{P+R}}\end{array}$$

(27)

The Intersection over Union (IoU) for class $i$ is defined as:

$$Io{U}_i={\displaystyle \frac{p_{i,i}}{\sum _{j=0}^k p_{i,j}+\sum _{j=0}^k p_{j,i}-p_{i,i}}}$$

(28)

MIoU measures the overlap between predicted cloud/shadow regions and ground truth by averaging the IoU of each class:

$$MIoU={\displaystyle \frac{1}{k+1}}\sum _{i=0}^k {\displaystyle \frac{p_{i,i}}{\sum _{j=0}^k p_{i,j}+\sum _{j=0}^k p_{j,i}-p_{i,i}}}$$

(29)

FWIoU evaluates overall segmentation performance by weighting the IoU of each class according to its frequency in the dataset:

$$FWIoU={\displaystyle \frac{1}{\sum _{i=0}^k \sum _{j=0}^k p_{i,j}}}\sum _{i=0}^k {\displaystyle \frac{p_{i,i}\cdot \sum _{j=0}^k p_{i,j}}{\sum _{j=0}^k p_{i,j}+\sum _{j=0}^k p_{j,i}-p_{i,i}}}$$

(30)

For the GF1_WHU and the Cloud and Cloud Shadow Dataset, three classes (cloud, cloud shadow, and background) were used for calculation. For the HRC_WHU dataset, two classes (cloud and background) were used.

3.3. Ablation Experiment

In this section, we first utilized DeepLabV3+ as the backbone network and, while keeping the overall network structure unchanged, replaced our proposed modules with an auxiliary module consisting of standard convolutions and upsampling, subsequently concatenating them for output. Next, we progressively integrated the designed modules (DSRM, ASCA, AGMF, and FJFL) into the network to replace the auxiliary module, thereby validating the feasibility of each module and the entire network.

Table 2. Ablation results for different module combinations.

DeepLabV3+	DSRM	ASCA	AGMF	FJFL	MPA (%)	MIoU (%)	F1 (%)
✓					84.26	75.46	85.69
✓	✓				84.50	76.02	85.75
✓	✓	✓			84.72	76.24	86.47
✓	✓	✓	✓		85.07	76.56	86.09
✓	✓	✓	✓	✓	85.42	76.97	86.72

Note: Bold indicates the best result, and the underline indicates the second-best result.

details the improvement in network prediction accuracy contributed by each module.

• Ablation of DSRM: Standard dilated convolutions in traditional ASPP struggle to capture directional geometric features. DSRM introduces strip pooling and multi-scale depthwise separable convolutions to effectively establish long-range directional dependencies while preserving local textures. Results show that DSRM improved MPA and MIoU by 0.24% and 0.56%, respectively, demonstrating its efficacy in enhancing directional consistency and textural detail.
• Ablation of ASCA: At the end of feature aggregation, overlapping multi-scale context often results in an uneven distribution of spatial responses. ASCA utilizes semantic guidance vectors and a median-centered spatial reweighting mechanism to adaptively emphasize significant spatial positions while suppressing anomalous noise. Its inclusion improved MPA and MIoU by 0.22% each, indicating enhanced spatial discrimination and feature robustness in complex cloud shadow regions.
• Ablation of AGMF: The feature fusion method in the decoding stage determines boundary precision. AGMF replaces simple concatenation with a dynamic grouping mechanism and local context gating, adaptively adjusting fusion granularity based on feature complexity. AGMF contributed a 0.32% increase in MIoU, validating its superiority in recovering spatial details. Although the F1-score slightly fluctuates, the consistent improvement in MIoU indicates that AGMF contributes to better overall regional overlap.
• Ablation of FJFL: Cloud shadow segmentation suffers from severe class imbalance and weak contrast, particularly for elongated and small-scale shadow structures. To address this issue, FJFL integrates Logit-Adjusted Focal Cross-Entropy to mitigate long-tail distribution bias and incorporates a multi-class Focal Tversky Loss to emphasize hard-to-classify regions while preserving structural continuity. This hybrid loss formulation jointly enhances pixel-level discriminative robustness and region-level structural consistency. The inclusion of FJFL improves MPA, MIoU, and F1-score by 0.35%, 0.41%, and 0.63%, respectively, effectively reducing missed detections and improving overall segmentation robustness.

3.4. Comparative Experiment

To verify the superiority and robustness of DSFNet in complex cloud shadow scenarios, we performed a rigorous benchmark comparison against several representative methods in the field. The comparison covers three mainstream architectural paradigms: classic CNN architectures (Unet, FCN, DeepLabV3+), advanced Transformers (SegFormer, Swin-Unet), and high-resolution or hybrid architectures designed specifically for cloud segmentation (HRNet, EDFF-Unet, MFAFNet). Experimental results (

Table 3. Comparison of segmentation results on the GF1_WHU dataset.

Networks	PA (%)	MPA (%)	MIoU (%)	FWIoU (%)	FLOPs (G)	Time (ms)
BiSeNetV2 [39]	86.15	80.20	67.85	75.72	10.01	2.13
Unet [10]	89.71	83.11	74.62	80.15	30.77	9.04
U2net [40]	89.30	83.11	72.72	78.93	25.92	8.36
DeepLabV3+ [16]	89.46	84.26	75.46	81.48	20.69	35
FCN [9]	89.12	82.13	73.51	80.51	37.13	11.43
SegNet [11]	88.22	81.36	72.11	79.09	28.70	7.23
PSPNet [17]	89.12	82.90	74.21	79.88	28.74	10.36
HRNet [12]	89.30	83.95	74.69	80.99	14.80	14.05
SegFormer [19]	88.97	84.01	74.28	80.39	19.98	8.39
Swin-Unet [18]	89.79	85.21	75.63	81.83	32.87	5.78
DATransunet [20]	90.06	82.90	75.11	81.43	44.56	71
EDFF-Unet [24]	89.92	84.43	76.13	82.71	28.55	7.71
MFAFNet [25]	90.31	84.80	76.38	82.95	32.60	8.94
DSFNet (ours)	90.62	85.42	76.97	83.41	29.60	8.1

Note: Bold indicates the best result, and the underline indicates the second-best result.

) indicate that while advanced models like HRNet and SegFormer perform well on certain metrics, DSFNet demonstrates consistent superiority across multiple evaluation metrics. Specifically, our model reached the highest scores in PA (90.62%), MPA (85.42%), and MIoU (76.97%), proving its significant advantage in complex feature extraction and boundary refinement.

To further assess the class-specific performance of the proposed model, evaluation was conducted for the cloud and cloud-shadow classes, as shown in

Table 4. Comparison of segmentation results of different models in cloud and cloud shadow.

Networks	Cloud				Cloud Shadow
	P (%)	R (%)	F1 (%)	IoU (%)	P (%)	R (%)	F1 (%)	IoU (%)
BiSeNetV2	87.13	89.93	88.51	79.38	77.26	52.13	62.25	45.20
Unet	89.65	92.18	90.92	83.31	78.62	64.43	70.81	54.82
U2net	89.10	91.20	90.14	82.05	77.80	62.50	69.32	53.04
DeepLabV3+	90.95	92.16	91.55	84.42	81.45	64.53	72.01	56.26
FCN	90.62	92.44	91.52	84.37	79.04	61.82	69.31	53.11
SegNet	88.89	92.54	90.68	82.95	77.17	61.17	68.24	51.80
PSPNet	90.01	93.12	91.53	84.40	78.12	63.52	70.01	53.93
HRNet	92.55	90.44	91.48	84.30	75.56	69.12	72.19	56.49
SegFormer	91.06	91.38	91.21	83.86	75.62	70.34	72.88	57.34
Swin-Unet	91.58	93.18	92.37	85.83	74.29	71.52	72.88	57.33
DATransunet	92.11	92.57	92.34	85.77	81.81	62.79	71.05	55.10
EDFF-Unet	91.82	93.23	92.52	86.08	80.50	66.04	72.53	56.93
MFAFNet	92.08	93.52	92.79	86.56	81.07	67.21	73.34	58.09
DSFNet(ours)	92.36	94.01	93.18	87.23	82.15	70.84	75.99	61.39

Note: Bold indicates the best result, and the underline indicates the second-best result.

. For cloud detection, DSFNet achieved the highest recall (94.01%), F1 score (93.18%), and IoU (87.23%), while its precision (92.36%) was slightly lower than that of HRNet. For the more challenging cloud-shadow class, DSFNet attained 82.15% precision, 70.84% recall, 75.99% F1 score, and 61.39% IoU, outperforming the second-best method by 3.3 percentage points in IoU. These results indicate that the proposed method not only performs strongly on the relatively easier cloud class, but also yields more substantial gains on the harder cloud-shadow class, which is more susceptible to background interference and boundary ambiguity.

Based on an in-depth comparative analysis of the visual results in Figure 9, classical models exhibit significant limitations when handling complex cloud shadow scenes. In the figure, black represents the background, white denotes clouds, and gray indicates cloud shadows, with red boxes highlighting regions of significant discrepancy. Although HRNet maintains high-resolution features and produces relatively smooth boundaries, it still exhibits instability when processing thin cirrus clouds with translucent characteristics or irregular broken clouds, leading to partial false positives and loss of detail in complex scenes (as shown in the first row of Figure 9). While Transformer architectures like SegFormer and Swin-Unet outperform traditional CNNs in capturing global contexts, they fall short in restoring local textures of cloud layers: Swin-Unet displays distinct jagged artifacts on segmentation edges, whereas SegFormer suffers from extensive false negatives in fragmented cloud regions (as shown in the second row of Figure 9). Despite DeepLabV3+ demonstrating powerful multi-scale capture capabilities via atrous convolutions, its pursuit of a large receptive field limits its perception of high-frequency details; consequently, high-level semantics dominate the prediction results, causing subtle targets in the extremely faint cloud shadow region in the bottom-left corner of the fourth row to be missed. Building upon the strong semantic representation capabilities inherited from DeepLabV3+, DSFNet introduces DSRM to strengthen the extraction of directional broken clouds and strip-like structures. It leverages the statistical attention mechanism of ASCA to effectively suppress complex background noise, significantly reducing false negatives for small cloud shadows. Furthermore, by adaptively aggregating multi-scale features through AGMF in the decoding stage, it sharply defines boundaries and generates high-quality segmentation masks with tight contours and smooth edges.

To better demonstrate the generalization capability of our model in different scenarios, we compared the segmentation results of various models across eight different scenes as shown in Figure 10, including grassland, desert, rocky terrain, urban, mountainous areas, water bodies, snow/ice, and barren land. In the first set of images, HRNet shows relatively coarse edge recognition for irregular cloud shadows; while other models perform better in edge segmentation, their detailed rendering is still inferior to that of our model.

In the second set of images, HRNet and SegFormer experience structural distortion when processing irregular cloud shadow edges. Although DeepLabV3+ and Swin-Unet successfully recover the overall shape and contours of the cloud shadows, our proposed method achieves the best overall performance in preserving the morphological integrity and edge sharpness of the cloud shadows, achieving segmentation results that are most highly consistent with the ground truth labels.

In the third set of images, the red box highlights a cloud shadow with a ring-like topological structure. DeepLabV3+, SegFormer, and Swin-Unet tend to fill in the central hole, resulting in a loss of the topological structure. In contrast, our proposed method successfully preserves the hole features within the cloud shadow, reflecting its capability to maintain complex geometric structures.

In the fourth set of images, all comparative models merge adjacent independent cloud shadow patches into connected regions. Only our model successfully segments the intervals between cloud shadows, maintaining clear physical boundaries between targets.

In the fifth set of images, Swin-Unet confuses the terrain shadows caused by topographic variations in mountainous areas with cloud shadows. Our model, however, effectively distinguishes the shadows cast by clouds from the inherent terrain shadows of the surface.

In the sixth set of images, water body scenes are typically accompanied by spectral variations caused by waves or turbidity, making thin clouds extremely difficult to observe over water. For the extremely thin cloud layers above the water surface, HRNet and SegFormer almost completely miss them, whereas the proposed method successfully captures these low-contrast thin-cloud targets.

In the seventh set of images, HRNet was largely unable to distinguish between snow-capped mountain peaks and cloud pixels. Other competing methods suffered from interference due to different objects with similar spectra, misclassifying portions of snow-covered summits as clouds. Our model exhibited the lowest false positive rate, demonstrating superior feature discrimination and robustness in complex backgrounds.

In the eighth set of images, DeepLabV3+ failed to adequately identify the fragmented black background areas within cloud layers, resulting in the loss of internal topological details. Our model, however, fully preserved the internal cavities within the cloud layers.

Figure 11 visualizes the segmentation performance of different methods for cloud and cloud-shadow regions with varying optical thicknesses and morphological heterogeneity. Cumulus clouds typically exhibit complex geometric boundaries and evident internal brightness variations, which increase the difficulty of maintaining consistent segmentation within the cloud body. In such cases, the proposed method produces more coherent predictions across both bright and dark cloud regions by effectively modeling long-range contextual dependencies. Broken clouds display highly discontinuous spatial distributions, and these fine-scale structures are easily smoothed out or omitted during feature extraction. As shown in the highlighted regions, DSFNet preserves these fragmented cloud components more effectively and reduces local omissions.

The interior of thick clouds is usually easier to identify due to the strong reflectance associated with large optical thickness. However, near the cloud margins, optical thickness gradually decreases and reflectance drops sharply, leading to blurred transitions between the cloud and the background. In these low-contrast regions, DSFNet yields cloud contours that are sharper and more consistent with the ground truth. Thin clouds, by contrast, are semi-transparent and allow background radiative signals to partially penetrate the cloud layer, so that the observed pixel values become a mixture of cloud and surface spectra. Over highly reflective backgrounds such as deserts, snow/ice, or urban areas, thin clouds are therefore particularly prone to omission. In these challenging cases, DSFNet suppresses background interference more effectively and improves the completeness of thin-cloud segmentation.

Cloud-shadow regions exhibit even stronger ambiguity under complex backgrounds. Although their overall shapes are not always regular or strictly elongated, they often show local directional extension, boundary continuity, and projection-consistent structural coherence, especially in weak-response regions and near irregular cloud boundaries. These characteristics are particularly difficult to preserve under low contrast and strong background interference. As further illustrated by the samples in rows 6–9 of Figure 11, such cloud-shadow regions may appear fragmented, locally stretched, or weakly contrasted rather than forming regular stripe-like patterns. Compared with the competing methods, DSFNet produces cloud-shadow predictions with better regional continuity, tighter boundary adherence, and fewer structural breaks.

Overall, the proposed method yields more complete cloud regions, more continuous cloud-shadow predictions, and more accurate boundaries across diverse cloud and cloud-shadow scenarios.

3.5. Generalization Performance Analysis

3.5.1. Evaluation on Additional Datasets

To further validate the cross-dataset generalization performance of the proposed DSFNet, comparative experiments were conducted on two benchmark datasets, namely the HRC_WHU dataset (Dataset 1) and the Cloud and Cloud Shadow Dataset (Dataset 2). The segmentation accuracy was quantitatively evaluated using three widely adopted metrics, including PA, MPA, and MIoU. The quantitative results are reported in

Table 5. Comparison of segmentation results across different models on additional datasets.

Networks	HRC_WHU			Cloud and Cloud Shadow Dataset
	PA (%)	MPA (%)	MIoU (%)	PA (%)	MPA (%)	MIoU (%)
BiSeNetV2	92.71	91.55	85.62	94.42	92.36	86.91
Unet	93.49	92.62	86.53	96.83	95.61	91.83
U2net	93.28	92.41	86.31	96.54	95.23	91.46
DeepLabV3+	93.17	91.98	86.29	97.05	95.64	92.03
FCN	92.85	91.67	85.93	94.31	92.17	87.12
SegNet	92.94	91.88	86.01	94.61	92.52	87.39
PSPNet	93.20	92.07	86.42	95.08	93.41	89.57
HRNet	93.08	91.91	86.15	96.52	94.88	91.08
SegFormer	93.71	93.12	86.74	97.14	96.03	92.28
Swin-Unet	93.79	93.21	87.05	97.26	96.18	92.61
DATransunet	94.43	93.38	87.12	97.39	96.31	92.74
EDFF-Unet	94.18	93.44	87.28	97.57	96.47	92.89
MFAFNet	94.28	93.51	87.47	97.48	96.49	92.97
DSFNet(ours)	94.57	93.76	87.81	97.78	96.73	93.31

Note: Bold indicates the best result, and the underline indicates the second-best result.

.

Experimental results indicate that our model achieves the best overall performance on both datasets. On Dataset 1, our model comprehensively leads all compared models, achieving the highest scores of 94.57% in PA, 93.76% in MPA, and 87.81% in MIoU. Similarly, on Dataset 2, it exhibits the best overall performance with a PA of 97.78%, MPA of 96.73%, and MIoU of 93.31%. Notably, despite the distribution discrepancy between the two datasets, DSFNet maintains consistent superiority, highlighting its strong domain adaptability. Overall, although models like DATransunet and MFAFNet achieve competitive results on certain metrics, our model consistently outperforms them across both datasets. This robustly demonstrates the strong generalization capability and adaptability of our proposed model in complex cloud and cloud shadow segmentation tasks.

It is worth noting that differences in absolute performance across datasets should be interpreted with caution. Such differences largely reflect variations in dataset setting and intrinsic segmentation difficulty, rather than inconsistency in the evaluation protocol. The absolute performance differences across datasets are mainly attributable to differences in task setting, spatial resolution, and scene complexity. In particular, HRC_WHU is a binary cloud/background segmentation task, whereas GF1_WHU and the Cloud and Cloud Shadow Dataset adopt a three-class setting including cloud shadow, which introduces additional inter-class confusion. Moreover, GF1_WHU has a coarser spatial resolution, making boundary localization more difficult due to mixed pixels and blurred transitions. It also contains more heterogeneous backgrounds and more complex scene variations, which further increase the segmentation difficulty. Therefore, the relatively lower absolute metrics on GF1_WHU mainly reflect the higher intrinsic difficulty of this dataset rather than any inconsistency in the experimental protocol.

3.5.2. Cross-Dataset Transfer Evaluation

To further examine the transferability of the proposed model under domain shift, bidirectional cross-dataset experiments were conducted between the GF1_WHU dataset and the Cloud and Cloud Shadow Dataset (Dataset 2). Specifically, the model was first trained on GF1_WHU and evaluated on the Cloud and Cloud Shadow Dataset, after which the training and testing domains were reversed. The quantitative results are summarized in

Table 6. Cross-dataset transfer performance between GF1_WHU and the Cloud and Cloud Shadow Dataset.

Train	Test	Method	PA (%)	MPA (%)	F1 (%)	MoU (%)
GF1_WHU	Dataset 2	DeepLabV3+	86.99	78.06	79.05	67.39
GF1_WHU	Dataset 2	DSFNet	89.60	81.97	83.63	73.01
Dataset 2	GF1_WHU	DeepLabV3+	79.11	69.34	70.56	57.23
Dataset 2	GF1_WHU	DSFNet	82.46	73.84	75.92	61.76

Note: Bold indicates the better result between DeepLabV3+ and DSFNet.

.

Under both transfer settings, DSFNet consistently outperforms DeepLabV3+. When trained on GF1_WHU and evaluated on the Cloud and Cloud Shadow Dataset, DSFNet improves PA, MPA, F1, and MIoU by 2.61%, 3.91%, 4.58%, and 5.62%, respectively. When the training and testing domains are reversed, the corresponding improvements remain 3.35%, 4.50%, 5.36%, and 4.53%. These results indicate that, although both models suffer from performance degradation under cross-dataset testing, the proposed method maintains stable advantages in the presence of distribution shifts.

The superior transfer robustness of DSFNet can be attributed to the complementary design of the proposed framework. The DSRM enhances direction-sensitive structural continuity, which is beneficial for preserving projected cloud-shadow morphology under varying scene layouts. The ASCA improves robustness to radiometric ambiguity and anomalous responses, thereby alleviating confusion between cloud shadows and visually similar dark regions. Meanwhile, the AGMF strengthens cross-level interaction by explicitly exploiting feature discrepancies, which helps preserve boundary details when the source and target distributions are inconsistent. As a result, compared with the baseline model, DSFNet exhibits stronger robustness under cross-dataset transfer and better tolerance to domain shifts.

4. Discussion

4.1. Advantages of the DSFNet

The advantages of DSFNet mainly arise from the consistency between its design and the intrinsic characteristics of cloud and cloud-shadow segmentation in remote sensing imagery. In complex scenes, cloud shadows often exhibit weak responses, local directional extension, and strong confusion with visually similar dark surfaces, while cloud regions usually involve multi-scale structures and irregular boundaries. In addition, overexposed snow/ice regions may introduce abnormally strong responses and further increase the difficulty of distinguishing clouds from bright surface objects. Such characteristics are difficult to handle using conventional frameworks that rely mainly on isotropic context modeling and direct cross-level fusion.

In this framework, DSRM enhances direction-sensitive structural representation and helps preserve the continuity of elongated cloud-shadow regions. ASCA improves robustness to radiometric heterogeneity by suppressing anomalous responses through statistically guided feature modulation. AGMF further strengthens boundary recovery and local consistency by explicitly introducing cross-level discrepancy into adaptive fusion. Through the complementary action of these modules, DSFNet shows particular advantages in challenging scenarios involving low contrast, complex backgrounds, and ambiguous boundaries. At the same time, these improvements are achieved within a relatively conventional DeepLabV3+ based framework, which makes the proposed method both effective and practical for remote sensing image segmentation.

4.2. Outlook of the DSFNet

Despite achieving a competitive MIoU of 76.97%, DSFNet may still exhibit prediction errors and boundary ambiguity in highly challenging scenarios, particularly when cloud or cloud-shadow regions present low contrast, complex background interference, or confusing spectral responses with visually similar surface objects. From a physical perspective, remote sensing observations are inherently affected by atmospheric scattering, illumination variation, surface reflectance heterogeneity, and scale inconsistency, which make thin clouds, shadow transitions, and blurred boundaries difficult to separate in a strictly deterministic manner. From a mathematical perspective, the current framework still mainly relies on discriminative feature learning under pixel-level supervision, and thus may suffer from insufficient inter-class separability and elevated predictive uncertainty in structurally ambiguous or low-confidence regions. As a result, pixels around difficult boundaries are more likely to be oversmoothed toward dominant categories during optimization, leading to local omission and error accumulation.

To alleviate these limitations, future work will focus on developing a dynamic error-aware optimization framework that more closely couples physical prior information with uncertainty-guided representation learning. Specifically, an image-level performance evaluation repository can be established during training to continuously track the segmentation quality of individual samples and identify persistently difficult cases. Based on this repository, the network can adaptively prioritize low-confidence or poorly segmented samples for targeted hard example mining and iterative refinement, thereby improving its sensitivity to ambiguous structures and boundary transitions [41]. In addition, physically meaningful constraints, such as illumination consistency, boundary transition regularity, and cross-scale contextual coherence, may be further introduced to regularize the optimization process and enhance robustness in challenging regions. This direction is also supported by recent studies showing that global–local dependency modeling, cross-level feature interaction, and token-enhanced contextual representation are effective for capturing complex structures and difficult boundaries in remote sensing interpretation tasks [42,43,44,45].

5. Conclusions

Accurate segmentation of clouds and cloud shadows is a critical prerequisite for remote sensing image preprocessing, particularly in complex scenarios with diverse morphological variations and terrestrial interference. To address the inherent limitations of traditional methods in preserving fine edge details and semantic consistency, we proposed DSFNet, a novel framework built upon an enhanced DeepLabV3+ architecture.

DSFNet incorporates several targeted structural innovations to achieve high-precision segmentation. DSRM enhances the capture of elongated shadows and multi-scale clouds, while ASCA effectively suppresses terrestrial noise and bolsters feature robustness. Furthermore, AGMF, guided by a grouped gating mechanism, adaptively recovers critical boundary information during the decoding stage. Experimental results explicitly demonstrate the superiority of our approach: evaluated on the GF1_WHU dataset, DSFNet achieves a Mean Intersection over Union (MIoU) of 76.97%, outperforming the second-best method by a margin of 0.59%.

DSFNet provides significant practical value in meteorological monitoring and remote sensing analysis by ensuring robust segmentation in intricate atmospheric conditions. However, similar to other high-performance multi-scale fusion networks, the computational complexity introduced by the parallel modules remains a limitation, potentially restricting its direct deployment on resource-constrained satellite edge devices. Future research will prioritize lightweight design strategies—such as knowledge distillation or model pruning—to minimize the network’s parameter footprint while preserving its high segmentation accuracy. Ultimately, these advancements will facilitate low-latency, real-time remote sensing image processing for broader earth observation applications.