Cross-Modality Spectral Expansion Combined with Physical–Semantic Dual Priors for Cloud Detection in GF-1 Imagery

Highlights

What are the main findings?

• A cross-modality spectral expansion framework is proposed to reconstruct pseudo-Sentinel-2 spectral representations from four-band GF-1 imagery.
• Physical NDSI-based priors and LLM-derived semantic priors are jointly integrated to improve cloud–snow discrimination in high-resolution GF-1 cloud detection.

What are the implications of the main findings?

• The reconstructed SWIR-like bands provide task-relevant spectral cues for NDSI-guided cloud detection under spectrally limited sensor conditions.
• The proposed CMSE-DPNet achieves improved cloud detection performance on GF-1 imagery, with an F1-score of 94.41% and an IoU of 89.40%.

Abstract

Cloud detection in high-resolution Gaofen-1 (GF-1) imagery is challenging due to the absence of short-wave infrared (SWIR) bands, which prevents the use of physically interpretable indices such as the Normalized Difference Snow Index (NDSI) and often leads to severe cloud–snow confusion. To address this limitation, we propose a unified framework, termed the Cross-Modality Spectral Expansion and Dual-Prior Network (CMSE-DPNet), that integrates cross-modality spectral expansion with physical–semantic dual priors. First, an improved CycleGAN reconstructs 13-band pseudo-Sentinel-2 spectra from four-band GF-1 imagery, enabling the computation of snow-sensitive physical indices. Second, a Snow-Aware Feature Attention Guidance Module (SAFAGM) introduces pixel-level physical priors derived from NDSI, while a Label-Guided Channel Attention Module (LG-CAM) injects scene-level semantic priors inferred from geographic metadata using a large language model. These complementary priors guide the network to better distinguish clouds from spectrally similar backgrounds. Experiments on the GF-1 dataset show that the proposed method achieves an F1-score of 94.41% and an Intersection over Union (IoU) of 89.40%, outperforming several state-of-the-art cloud detection methods. The results indicate that cross-modality spectral expansion combined with physical–semantic prior guidance effectively improves cloud detection performance in complex cloud–snow coexistence scenarios.

1. Introduction

Cloud detection is a fundamental preprocessing step in optical remote sensing, and its accuracy directly affects downstream applications such as land-cover classification, environmental monitoring, and disaster response. High-resolution Gaofen-1 (GF-1) imagery provides valuable spatial details for regional observation; however, GF-1 WFV data contain only visible and near-infrared bands and lack shortwave infrared (SWIR) information [1,2]. This spectral limitation weakens the separability between clouds and high-reflectance surfaces, especially snow and ice, resulting in frequent false alarms in complex cloud–snow coexistence scenes [3,4].

Existing studies on cloud detection have evolved from rule-based thresholding and machine-learning classifiers to deep segmentation networks with multi-scale feature fusion, attention mechanisms, and boundary-aware modeling [5,6,7,8,9]. Meanwhile, image translation and spectral reconstruction methods have shown the potential to infer richer spectral representations from spectrally limited observations [10,11], while recent large language models (LLMs) provide a new way to extract high-level geographic and semantic context [12,13]. A detailed review of these three research lines is provided in Section 2.

Despite these advances, several limitations remain for GF-1 cloud detection. First, most cloud detection networks are constrained by the spectral bands available in the input imagery, making it difficult to use physically interpretable SWIR-dependent indices such as the Normalized Difference Snow Index (NDSI) [3,14]. Second, existing image translation and spectral reconstruction methods often focus on visual or spectral fidelity, but their reconstructed bands are not always explicitly connected to downstream cloud–snow discrimination. Third, scene-level semantic information, such as whether snow, desert, vegetation, or large water bodies are likely to appear at a given location and season, is rarely incorporated into dense cloud segmentation in a structured manner.

To address these gaps, this study proposes a Cross-Modality Spectral Expansion and Dual-Prior Network (CMSE-DPNet) for cloud detection in GF-1 imagery. The core idea is to first reconstruct pseudo Sentinel-2A spectral representations from four-band GF-1 images, thereby enabling SWIR-sensitive physical cues, and then combine pixel-level physical priors with scene-level semantic priors for robust segmentation. In this work, physical priors refer to information derived from physically interpretable spectral indices that capture reflectance differences between clouds and other surfaces, whereas semantic priors describe high-level contextual information inferred from geographic metadata.

Specifically, after reconstructing the 13-band spectra through cross-modality transfer, NDSI values can be computed to derive pixel-level physical guidance. In parallel, an LLM extracts semantic labels from geographic metadata to provide global contextual guidance. These two complementary priors are then injected into the cloud detection network through attention-based feature modulation, enabling the model to better distinguish clouds from spectrally similar backgrounds. These observations motivate our dual-prior cloud detection framework:

• Cross-modality spectral expansion generates SWIR proxy bands for GF-1, enabling SWIR-dependent physical cues for downstream processing.
• The Normalized Difference Snow Index Module (NDSIM) and Snow-Aware Feature Attention Guidance Module (SAFAGM) inject pixel-level physical priors to suppress cloud–snow confusion.
• An LLM performs scene pre-classification from geographic metadata and provides a semantic label vector.
• The Spatial-Spectral Fusion Module (SSFM) and Label-Guided Channel Attention Module (LG-CAM) fuse spectral–spatial features with physical and semantic priors for robust segmentation.

This causal chain forms the core rationale of our proposed method and unifies all components into a coherent design.

The main contributions of this paper are summarized as follows:

• Cross-Modality Spectral Expansion: We propose a transfer network based on an improved CycleGAN to translate 4-band GF-1 images into 13-band pseudo-Sentinel-2A images. By incorporating Spectral Self-Attention and structural consistency constraints, this method effectively reconstructs missing spectral information, thereby bridging the capability gap between spectrally limited sensors and the requirements for high-precision cloud detection. By reconstructing the SWIR band, we establish the necessary condition for utilizing physical snow indices (NDSI) in high-resolution imagery.
• Spatial-Spectral Fusion Module (SSFM): We design a Spatial-Spectral Fusion Module to enhance feature discriminability. By processing spatial and spectral features in parallel and utilizing additive and subtractive fusion paths, the SSFM effectively integrates spatial details with spectral cues, significantly improving the network’s ability to capture comprehensive and discriminative features.
• Dual Pre-classification Guidance Framework: We propose a novel detection paradigm that integrates pixel-level physical priors and scene-level semantic priors. We utilize the Normalized Difference Snow Index Module (NDSIM) and Snow-Aware Feature Attention Guidance Module (SAFAGM) to explicitly model reflectance divergence at the pixel level, while simultaneously employing the LLM-driven Label-Guided Channel Attention Module (LG-CAM) for scene-level semantic guidance. This “Pre-classify, then Detect” strategy effectively guides the network in complex scenarios.

2. Related Work

This section reviews three groups of studies most closely related to the proposed framework: cross-modality image translation and spectral reconstruction, cloud detection in remote sensing imagery, and LLM-driven semantic priors for remote sensing interpretation.

2.1. Image Translation

Most current style transfer algorithms employ autoencoders. The earliest approach was proposed by Li et al. with Whitening and Coloring Transform (WCT [15]). After the encoder extracts image features, whitening removes the style information from the input image while preserving its global structure, whereas coloring serves as the inverse process of whitening, incorporating style image information during coloring to achieve style transfer. However, this method still suffers from spatial distortions that degrade output quality, such as image artifacts. Subsequent researchers developed PhotoWCT [16] and WCT² [17] based on WCT to address spatial alignment and artifact issues, significantly improving the photorealism of generated images.

PhotoWCT improves upon WCT by replacing traditional upsampling with unpooling operations and introducing a MaxPooling mask to recover details lost during feature extraction. Since max pooling compresses feature map information, conventional encoder-based upsampling fails to effectively restore the original image structure. WCT² enhances the WCT framework by integrating wavelet transforms into neural networks, enabling features to preserve both structural information and VGG-space statistical characteristics. Additionally, WCT²-based methods improve temporal stability in stylized outputs.

To better inject style characteristics, researchers have proposed various regularization approaches including AdaIN [18] and SPADE [19]. AdaIN (Adaptive Instance Normalization) introduces the mean and variance of style features to content features after encoding, enabling style fusion before decoding. SPADE (Spatially Adaptive Normalization), a conditional generative model, enhances structural and textural realism by integrating semantic information into the generation process, enabling precise region-wise feature control.

In recent advancements, Wu et al. proposed StyleFormer [20], incorporating Transformer architectures into style transfer networks to achieve both fine-grained style diversity and semantic content consistency. Lin et al.’s LapStyle [21] first transfers global style patterns through a draft network at low resolution, then refines local details using a revision network that hallucinates textures based on the draft and Laplacian-filtered image components. By stacking revision networks with multiple Laplacian pyramid levels, high-resolution details can be progressively generated.

Beyond cross-modality image translation based on autoencoders, GANs also have extensive applications in this field. Mirza et al. proposed CGAN [22], which introduces conditional information to control the image generation process of GANs. Building on this, Isola et al. developed Pix2pix [23], where the input is modified to images, and paired datasets are used to train networks to learn the mapping between input and output images. cGAN has been widely applied in remote sensing image processing, particularly in SAR-to-optical image translation. SAR images are grayscale and less affected by weather or time factors, but their speckle noise and geometric distortions make them challenging for human interpretation, especially for non-experts. In contrast, optical images align with human visual perception but are susceptible to environmental conditions. To combine the advantages of both modalities, researchers have explored SAR-to-optical translation (S2O). M. Schmitt et al. collected and released the open-source SEN1-2 dataset [24], containing 282,384 matched patches of SAR and optical images from Sentinel-1 and Sentinel-2 satellites. Since then, numerous studies have improved CGAN architectures for more effective SAR-to-optical translation. For example, Bermudez et al. used classical CGAN to synthesize missing or corrupted multispectral optical images from SAR data of the same scene [25]. Later works employed CycleGAN to translate SAR images of varying resolutions into grayscale optical images. The atrous-CGAN architecture incorporated dilated convolutions and atrous spatial pyramid pooling to enhance fine details in output optical images by leveraging multi-scale spatial contexts [26]. Zhang et al. adopted feature-guided methods to improve generated image quality [27]. Further efforts to refine optical image quality include structural and loss function optimizations. Yang et al. enhanced network architectures with feature fusion generators, multi-scale discriminators, and chromatic loss functions to improve contour sharpness, texture granularity, and color fidelity [28]. Sun et al. proposed a generator with symmetric architecture and non-uniform convolutional kernels to reduce background clutter and edge blurring [29].

Due to the high dependency on paired training data in earlier methods, Zhu et al. introduced CycleGAN [10], which uses cycle-consistency loss to enable cross-modality translation between unpaired datasets from different domains. Building on CycleGAN, Chen et al. proposed NICEGAN [30], achieving high-quality image generation by reusing discriminators to encode target domain images. Yang et al. developed SwitchGAN [31], utilizing AdaIN for medical CT image reconstruction. Torbunov et al. combined U-Net with Transformer in a cycle-consistent GAN [32] for high-fidelity image synthesis. Han et al. proposed a contrastive learning framework with dual encoders to infer effective mappings between unpaired data, bridging the gap between unsupervised and supervised methods [33]. Cheng et al. introduced the Visual Concept Translator, a novel framework capable of handling diverse image-to-image translation tasks with only a reference image while delivering state-of-the-art results [34]. Recently, with the rise of multimodal learning, text-guided cross-modality image translation has gained traction. Tumanyan et al. proposed a framework integrating text-to-image synthesis into image-to-image translation: given a guidance image and a target text prompt, it leverages pre-trained text-to-image diffusion models to generate new images that align with the text while preserving the semantic layout of the guidance image [35].

Beyond cross-modality image translation within the same channel domain, spectral reconstruction (SR) in remote sensing focuses on translating RGB three-channel images to multispectral or hyperspectral images. Current SR research primarily targets RGB-to-hyperspectral image (HSI) mapping. Traditional SR methods rely on handcrafted hyperspectral priors. For example, Parmar et al. proposed a data sparsity expansion method for HSI reconstruction [36]. Arad et al. introduced a sparse coding approach that constructs dictionaries for HSI signals and their RGB projections [37]. Aeschbacher et al. suggested using relatively shallow spectral-specific learning models prior to spectral super-resolution [38]. However, these model-based methods suffer from limited representation capacity and poor generalization. Recently, inspired by the success of deep learning in natural image restoration, convolutional neural networks (CNNs) have been adopted to learn the underlying RGB-to-HSI mapping function. Xiong et al. proposed a unified HSCNN framework for HSI reconstruction from both RGB and compressed inputs [39]. Shi et al. built HSCNN-R, a deep residual network with adaptive residual blocks [40]. Zhang et al. designed a pixel-aware deep hybrid network to simulate RGB-to-HSI mappings [41]. While these CNN-based SR methods achieve impressive results, they exhibit limitations in capturing non-local self-similarities and long-range interdependencies. Addressing this, Cai et al. recently proposed a Transformer-based spectral reconstruction model that leverages inter-spectral self-attention. This approach significantly reduces computational complexity and parameter counts while winning the CVPR 2022 Spectral Reconstruction Challenge [11].

2.2. Cloud Detection

In recent years, deep learning methods have demonstrated exceptional performance in computer vision tasks such as object detection, image segmentation, and image super-resolution. For cloud detection segmentation tasks, deep learning approaches have also been widely adopted. Traditional cloud detection methods include threshold-based approaches such as Saunders et al.’s improved method for detecting clear sky and cloudy radiances from AVHRR data [5], and various adaptive threshold methods. Machine learning approaches have also been explored, including support vector machine based methods [6] and hybrid methods [42].

Ronneberger et al. proposed the U-Net [43] framework, an encoder-decoder network that significantly improved medical image segmentation accuracy and has since been influential across other domains. Jeppesen et al. adapted U-Net for remote sensing applications, introducing RSNet [7] to enhance cloud detection precision. The evolution of deep learning-driven cloud detection technologies has focused on multi-scale feature fusion and network architecture innovation. Ji et al. developed a cascaded CNN framework integrating cloud detection and mask generation, employing cross-layer multi-scale feature aggregation to improve recognition of thin and fragmented clouds [44]. Luotamo et al. designed a multi-scale feature pyramid module with parallel dilated convolutions tailored for high-resolution remote sensing data, mitigating underfitting in small-sample scenarios [45]. Segal-Rozenhaimer et al. proposed an adversarial domain-adaptive CNN model that enhances cross-sensor generalization through feature alignment [46]. Li et al. introduced GCDB-UNet [47], which innovatively incorporates dual-path feature extraction units: a sparse cloud feature capture module using dilated convolutions to expand receptive fields, and a channel interaction module leveraging self-attention for spectral feature reorganization. Experiments validated its superior performance in detecting thin and fragmented clouds.

Recent advances in cloud detection include specialized architectures such as CDNet [48] and its extension CDNetV2 [49] for handling cloud-snow coexistence scenarios. Zhang et al. proposed deep network architectures using wavelet transform and multi-scale spatial attention [50]. He et al. developed DABNet with deformable contextual and boundary-weighted features [51]. Zhao et al. proposed boundary-aware bilateral fusion networks [8]. Dong et al. developed MCDNet with multilevel cloud detection capabilities [9]. Li et al. created lightweight methods fusing multiscale spectral and spatial features [52]. Wu et al. employed self-attentive generative adversarial networks [53].

2.3. LLM-Driven Semantic Priors in Remote Sensing

In recent years, Large Language Models (LLMs) have made rapid progress in understanding and generating natural language, and have further evolved toward multimodal reasoning with the ability to align vision and language. Representative systems such as GPT, Qwen, and DeepSeek demonstrate strong instruction-following and general reasoning capabilities, which makes them promising “semantic engines” to process unstructured text and provide high-level priors for downstream perception tasks [54,55,56].

Beyond general-domain applications, the geospatial and remote sensing (RS) communities have started to explore how LLMs and large vision-language models (LVLMs) can benefit RS understanding. Recent works adapt conversational LVLMs to RS imagery for open-ended captioning, question answering, and multi-turn dialogue, showing that language-driven interaction can complement conventional purely-visual pipelines [12,57]. Meanwhile, LLMs can also operate purely on non-image geospatial context (e.g., place names, latitude/longitude, acquisition time, season, and map tags) and convert such metadata into semantically rich descriptions or categories. This ability provides a new route to construct semantic priors for RS interpretation, where high-level context can disambiguate visually similar patterns [13,58].

However, injecting LLM-derived priors into dense, pixel-level RS tasks remains challenging. For example, cloud detection must separate clouds from visually confusing phenomena such as snow/ice, bright sand, where local textures alone may be insufficient. This motivates methods that leverage LLM-produced semantic cues as complementary global context, while still preserving the fine-grained spatial accuracy required by segmentation-style prediction.

3. Materials and Methods

To enhance the discriminative power of low-spectral remote-sensing images for cloud detection, we propose an end-to-end framework that couples cross-modality spectral expansion with semantic guidance. The pipeline consists of two synergistic stages. First, a cross-modality expansion network, built upon an improved CycleGAN [10], expands the four-band GF-1 WFV imagery into 13-band pseudo-multispectral data aligned with Sentinel-2A, thereby substantially enriching the spectral cues available for subsequent discrimination. It is important to note that this stage performs spectral expansion rather than spatial super-resolution. The generated pseudo-Sentinel-2A representation has 13 spectral channels but preserves the spatial grid, patch size, and ground sampling distance of the input GF-1 WFV image. In our experiments, each generated patch remains $512\times 512$ pixels on the GF-1 WFV grid, corresponding to the original 16-m GF-1 WFV spatial resolution. Second, a dedicated cloud-detection network based on U-Net [43] processes these expanded bands through a series of spectral–spatial feature extraction modules to produce pixel-level cloud masks. To ensure precise segmentation, the model is optimized using a binary cross-entropy loss function, a standard objective derived from cross-entropy-based supervised learning [59], which minimizes the divergence between the predicted cloud probability maps and the ground truth labels.

The reconstructed SWIR bands generated by the transfer network provide the essential spectral information required for computing NDSI [14]. Therefore, NDSIM becomes a transfer-enabled physical prior, and cannot be used on the original GF-1 data. This physical prior forms the foundation for subsequent attention-guided feature modulation.

In addition, a LLM-generated Label-Guided Channel Attention Module (LG-CAM) injects high-level semantic priors derived from geographic metadata (e.g., latitude–longitude). The motivation follows recent studies showing that LLMs and remote-sensing vision-language models can extract useful geographic or scene-level semantic knowledge from non-image and image-related context [12,13,58]. These LLM-produced land-cover descriptors guide channel-wise feature modulation in skip connections, enabling the network to selectively amplify cloud-relevant spectral responses while suppressing misleading activations from snow, water, or other background types. This semantic-aware reweighting significantly enhances robustness in complex surface conditions where pixel-level visual cues are insufficient.

Together with the Spatial-Spectral Fusion Module (SSFM) and the Snow-Aware Feature Attention Guidance Module (SAFAGM), these components collaboratively improve accuracy under challenging scenarios such as cloud–snow coexistence, water-induced confusion, and thin-cloud ambiguity. Figure 1 provides an overview of the full architecture, illustrating how spectral expansion and semantic-enhanced detection are integrated into a unified workflow.

Figure 1. Overview of the Cross-Modality Spectral Expansion and Dual-Prior Network (CMSE-DPNet), including the Spectral Expansion Generator (SEG), Spatial-Spectral Fusion Module (SSFM), Normalized Difference Snow Index Module (NDSIM), Snow-Aware Feature Attention Guidance Module (SAFAGM), and Label-Guided Channel Attention Module (LG-CAM). The arrows indicate the main feature and prior-information flow. SEG generates pseudo-Sentinel-2A spectral representations from GF-1 inputs, while NDSIM/SAFAGM and LG-CAM inject pixel-level physical priors and scene-level semantic priors into the cloud detection network, respectively.

3.1. Cross-Modality Transfer Network

The standard CycleGAN framework [10] has achieved strong performance in unpaired image-to-image translation, but it is not tailored to remote-sensing spectral expansion (e.g., 4-band GF-1 → 13-band Sentinel-2A). When directly applied to band expansion, two key issues arise: (1) conventional ResNet-style generators may lose high-frequency spatial details that are critical for small targets and sharp boundaries; and (2) purely convolutional mappings provide limited explicit modeling of inter-band dependencies, which is essential for low- to high-dimensional spectral transfer.

To address these limitations, we propose an improved Cross-Modality Transfer Network (Figure 2) built upon CycleGAN’s bidirectional adversarial learning and cycle consistency between Domain X (GF-1) and Domain Y (Sentinel-2A) [10]. The core changes include (i) a Spectral Expansion Generator (SEG) with cascaded spectral self-attention modules inspired by self-attention-based dependency modeling [11,60], and (ii) cloud-mask guidance with an edge-aware regularization to enhance cloud integrity and boundary preservation.

Figure 2. Overview of the proposed Cross-Modality Transfer Network. Yellow arrows indicate the forward data flow, teal and red circular arrows denote the two generator mappings, blue rectangles indicate image or mask outputs, red rectangles indicate discriminators, yellow rectangles indicate loss terms, and dashed gray lines link generated outputs to their corresponding loss functions.

During image translation, the differences between GF-1 and Sentinel-2A are balanced as follows. The spatial-resolution difference is handled at the preprocessing and network-input level: Sentinel-2A samples are spatially normalized to the same patch size as GF-1 samples, and the generator is formulated as $G:X\in {\mathbb{R}}^{H\times W\times 4}\to \tilde{Y}\in {\mathbb{R}}^{H\times W\times 13}$, where H and W are unchanged. Thus, the model learns a Sentinel-2A-like spectral distribution without changing the GF-1 spatial geometry. The spectral-band difference is handled by the channel-alignment layer and the Spectral Expansion Generator, which map four GF-1 bands to a 13-channel pseudo-Sentinel-2A representation. To avoid spatial distortion during this spectral translation, the residual branch, structural–style separation loss, and edge-guided cloud-mask constraint are used to preserve GF-1 spatial structures and cloud boundaries while allowing the spectral statistics to approach the Sentinel-2A target domain.

It should be noted that the reconstructed spectral bands are not intended to perfectly reproduce the physical radiance measurements of Sentinel-2 sensors. Instead, the objective of the spectral expansion network is to generate spectral representations that approximate the statistical distribution and discriminative characteristics of the target spectral space. Such representations provide sufficient spectral cues for downstream tasks, enabling indices such as NDSI to function as effective discriminators between clouds and snow.

3.1.1. Spectral Expansion Generator (SEG)

Let $X\in {\mathbb{R}}^{H\times W\times C_1}$ be the source-domain image and $C_2$ be the target spectral dimension. The SEG first aligns channels using a $1\times 1$ convolution, then performs spectral expansion via cascaded Spectral Self-Attention (SSA) networks composed of a U-shaped module (SAU) and a dual-branch module (DSAU). The U-shaped design follows the general encoder–decoder principle of U-Net-like networks [43], while the spectral attention branch is newly designed in this work to model inter-band dependencies after channel expansion. The DSAU produces two outputs: the expanded image and an auxiliary cloud-mask prediction. The expanded image is refined by a $3\times 3$ convolution and fused with an input residual branch to preserve spatial details. The overall SEG architecture and the detailed SAU/DSAU structures are illustrated in Figure 3 and Figure 4, respectively. For the mask branch, we use the soft mask $\overline{M}$ for optimization and apply a fixed threshold only for inference/visualization:

$$X_1={\varphi }_{1\times 1}\left(X\right),$$

(1)

$$(\widehat{Y},\widehat{M})=SSA(X_1),$$

(2)

$$\tilde{Y}={\varphi }_{3\times 3}(\widehat{Y})+Res(X),$$

(3)

$$\overline{M}=\sigma (\widehat{M}),$$

(4)

$$M=255·\mathcal{T}(\overline{M};\tau ),$$

(5)

where ${\varphi }_{·}$ denotes convolution, $SSA(·)$ denotes cascaded SAU+DSAU, $\sigma (·)$ is Sigmoid, and $\mathcal{T}(·;\tau )$ is a fixed thresholding operator with threshold $\tau$ (used only for inference/visualization). The U-shaped encoder–decoder structure with skip connections helps retain fine structures, which is crucial for cross-sensor transfer.

Figure 3. Architecture of the Spectral Expansion Generator (SEG). Here, $1\times 1$ Conv denotes channel alignment, SSA denotes cascaded spectral self-attention, Res denotes the residual branch for spatial-structure preservation, and the mask branch outputs a soft mask for training and a thresholded hard mask for inference.

Figure 4. Architecture of the SAU and DSAU modules. (a) U-shaped Spectral Self-Attention Unit (SAU). (b) Dual-branch variant (DSAU) for joint spectral expansion and cloud-mask prediction. Downsampling and upsampling operators change feature resolution, skip connections transfer shallow structural details, and DSAU uses two output branches to jointly predict the expanded spectral image and auxiliary cloud mask.

3.1.2. Spectral Self-Attention Block (SAB)

SAB is the basic unit inside SAU/DSAU (Figure 5). Different from spatial self-attention, SAB performs attention along the spectral/channel dimension to model inter-band dependencies with low overhead. This design is motivated by the self-attention mechanism [60] and by spectral-wise attention used in hyperspectral reconstruction [11], but here the attention is adapted to GF-1-to-Sentinel-2A spectral expansion. Given a feature tensor F, $SAB$ is written compactly as

$$SAB(F)=F+\psi \left(softmax\left(K{(F)}^{\top }Q(F)\right)V(F)\right),$$

(6)

where $Q(·),K(·),V(·)$ are learnable linear projections and $\psi (·)$ is a linear output projection.

Figure 5. Architecture of the Spectral Self-Attention Block (SAB). Q, K, and V denote learnable query, key, and value projections. The softmax operation computes spectral/channel-wise attention weights, and the residual connection preserves the original feature response. Yellow arrows indicate the feature flow, colored feature stacks denote projected feature representations, circled cross symbols denote attention-based multiplication/fusion, and the circled plus symbol denotes residual addition.

3.1.3. Discriminator

We adopt the convolutional discriminator from the original CycleGAN [10]. It contains one input convolution (channel $\to 64$), followed by three $5\times 5$ convolution blocks (each with Instance Normalization and ReLU), and a final $5\times 5$ convolution to output the authenticity prediction.

3.1.4. Loss Function

The total objective is

$$\mathcal{L}={\mathcal{L}}_{adv}+{\lambda }_{cyc}{\mathcal{L}}_{cyc}+{\lambda }_S{\mathcal{L}}_S+{\lambda }_b{\mathcal{L}}_b.$$

(7)

While ${\mathcal{L}}_{adv}$ and ${\mathcal{L}}_{cyc}$ follow the standard CycleGAN objectives [10], the latter two terms (${\mathcal{L}}_S$ and ${\mathcal{L}}_b$) are proposed in this paper to better suit remote-sensing spectral expansion. Specifically, ${\mathcal{L}}_S$ regularizes the generator to preserve spatial structure (cloud geometry and boundaries) while allowing spectral/style statistics to change, thereby reducing structure hallucination during band expansion. In addition, ${\mathcal{L}}_b$ leverages the auxiliary cloud-mask branch to introduce a differentiable edge-guided constraint, encouraging sharp and coherent cloud boundaries (especially for thin clouds).

Adversarial Loss

We use the standard bidirectional adversarial losses following CycleGAN’s adversarial formulation [10]:

$${\mathcal{L}}_{adv}={\mathcal{L}}_{GAN}(G,D_Y,X,Y)+{\mathcal{L}}_{GAN}(F,D_X,Y,X),$$

(8)

where $G:X\to Y$ and $F:Y\to X$.

Cycle-Consistency Loss

The cycle-consistency term follows CycleGAN and constrains the translated image to be reconstructable back to its source domain [10]:

$${\mathcal{L}}_{cyc}={\parallel F(G(X))-X\parallel }_1+{\parallel G(F(Y))-Y\parallel }_1.$$

(9)

Structural–Style Separation Loss

We propose a structural–style separation regularizer to discourage structural hallucination and better decouple structure (cloud shapes/edges) from style/spectral statistics during band expansion. Specifically, we use a structure-aware similarity decomposition. We keep the formulation compact by using SSIM-style components derived from structural similarity analysis [61]:

$${\mathcal{L}}_S=\left(1-l(\tilde{Y},Y)c(\tilde{Y},Y)\right)+{\lambda }_{dc}\left(1-s(\tilde{Y},\mathrm{DC}\left(X\right))\right),$$

(10)

where $l(·,·)$ and $c(·,·)$ denote luminance and contrast terms for style/spectral alignment, $s(·,·)$ denotes the structure term, $\mathrm{DC}(·)$ is the dark channel used as a structural anchor inspired by the dark channel prior [62], and ${\lambda }_{dc}$ is a weighting coefficient.

Gaussian Edge Loss (Cloud-Mask Guidance)

We further introduce a Gaussian edge loss for cloud-mask guidance to better preserve cloud boundaries. Specifically, we define an edge-processing operator $sg(·)$ that: (i) applies the Sobel operator [63] to compute horizontal and vertical gradient magnitudes, (ii) filters out weak gradient responses using a threshold $\eta$ (we set $\eta =300$) to suppress noise and retain salient edges, and (iii) smooths the retained edge responses with a Gaussian kernel to obtain coherent boundary maps. The loss is computed as the mean squared error (MSE) between the processed edge maps of the predicted soft masks and those of the ground-truth masks, and is applied in both translation directions in our bidirectional framework:

$$\begin{array}{cc}\hfill L_b& = \mathrm{MSE}\left(sg\left(255{\overline{M}}_x\right),sg\left(M_x^{*}\right)\right)\hfill \\ \hfill & +\mathrm{MSE}\left(sg\left(255{\overline{M}}_y\right),sg\left(M_y^{*}\right)\right).\hfill \end{array}$$

(11)

where ${\overline{M}}_x$ and ${\overline{M}}_y$ denote the predicted soft cloud masks in domains X and Y, respectively, and $M_x^{*}$ and $M_y^{*}$ are the corresponding ground-truth masks. The factor 255 is used to match the dynamic range for gradient computation.

The SSFM architecture used in the cloud detection network is shown in Figure 6.

Figure 6. Architecture of the Spatial-Spectral Fusion Module (SSFM). The module processes spatial and spectral features in parallel and integrates them through additive and subtractive fusion paths. The $3\times 3$ branch extracts local spatial features, the $1\times 1$ branch models spectral/channel interactions, and ⊕ and ⊖ denote additive and subtractive fusion paths. The operator cr denotes convolution followed by ReLU activation.

3.2. Spatial-Spectral Fusion Module (SSFM)

Prior to spectral expansion, GF-1 contains only four spectral channels, limiting the usefulness of spectral modeling. After cross-modality transfer reconstructs 13 bands, spectral relationships become significantly richer. Spatial-Spectral Fusion Module (SSFM) is therefore introduced to fully exploit these expanded spectral dimensions through parallel spatial and spectral pathways. SSFM is newly designed in this work, while its separation of spatial and spectral processing follows the general motivation of multispectral spatial–spectral feature modeling [11,52].

The SSFM is designed to effectively extract and integrate both spatial and spectral information from the multispectral data. Traditional convolutional networks often underutilize the rich spectral information in multispectral imagery. Unlike generic convolutional blocks, SSFM is proposed here as a task-specific fusion module for pseudo-13-band cloud detection. Our SSFM addresses this limitation through parallel processing of spatial and spectral features.

Given an input feature map $X_{in}\in {\mathbb{R}}^{H\times W\times C}$, the module performs:

$$X_{spa}={\mathrm{conv}}_{3\times 3}(X_{in}),X_{spe}={\mathrm{conv}}_{1\times 1}(X_{in})$$

(12)

where $X_{spa}$ represents spatial features and $X_{spe}$ represents spectral features. These features are then processed through two fusion paths:

$$X_{add}=\mathrm{cr}(X_{spa})+\mathrm{cr}(X_{spe}),X_{sub}=\mathrm{cr}(X_{spa})-\mathrm{cr}(X_{spe})$$

(13)

where cr denotes convolution followed by ReLU activation. The additive fusion preserves complete spatial and spectral information, while the subtractive fusion emphasizes discriminative features, particularly useful for detecting thin clouds and cloud boundaries.

The final output is obtained by:

$$X_{out}=\mathrm{cr}(X_{add})+\mathrm{cr}(X_{sub})$$

(14)

This design enables the network to capture both comprehensive and discriminative features, significantly improving detection performance in complex scenarios.

3.3. Snow-Aware Feature Attention Guidance Module (SAFAGM)

To reduce confusion between clouds and snow, we propose a snow-aware feature guidance scheme that integrates a Normalized Difference Snow Index Module (NDSIM) with a Snow-Aware Feature Attention Guidance Module (SAFAGM). Thanks to the cross-modality spectral expansion described earlier, the reconstructed bands—including the SWIR wavelength absent in original GF-1 imagery—enable the use of a snow-sensitive physical prior within our network.

NDSIM exploits the reflectance contrast between clouds and snow in the visible and SWIR regions, following the physical basis of NDSI [14], providing a pixel-level guidance map. SAFAGM then incorporates this guidance when fusing encoder and decoder features, allowing the attention mechanism to focus on ambiguous regions where cloud–snow confusion is likely. By combining spatial and channel attention with this prior, the module selectively enhances cloud-relevant responses and suppresses snow-induced interference, drawing on the general idea of channel and spatial attention mechanisms [64,65] while using NDSI-derived guidance as a task-specific prior. The detailed SAFAGM architecture is shown in Figure 7.

Figure 7. Architecture of the Snow-Aware Feature Attention Guidance Module (SAFAGM). The module combines channel attention and spatial attention mechanisms to emphasize discriminative features for cloud-snow separation. NDSIM provides the pixel-level physical-prior map. The ms and cs branches generate channel and spatial attention maps, respectively, and ⊗ denotes element-wise multiplication used to reweight cloud–snow ambiguous features.

This concise integration of reconstructed spectral cues and attention refinement improves robustness in cloud–snow coexistence scenarios.

First, to explicitly model the spectral divergence, we calculate the Normalized Difference Snow Index (NDSI) [14]. Clouds typically exhibit higher reflectance in both visible and Short-Wave Infrared (SWIR) bands compared to snow, which has lower SWIR reflectance. The index is calculated as:

$$\mathrm{NDSI}={\displaystyle \frac{Green-SWIR}{Green+SWIR+ϵ}}$$

(15)

where Green represents the reflectance in the green band (550 nm) and SWIR represents the reflectance in the short-wave infrared band (1630 nm). We enhance this basic formulation within the network by introducing a learnable parameter $\alpha$ to create the NDSIM, improving the robustness of the feature generation:

$$\mathrm{NDSIM}=\alpha ·{\displaystyle \frac{Green-SWIR}{Green+SWIR+ϵ}}$$

(16)

Building upon the transfer-enabled physical prior, the Snow-Aware Feature Attention Guidance Module (SAFAGM) explicitly incorporates the pixel-level physical-prior map produced by NDSIM. Since NDSIM highlights snow-like regions where cloud–snow confusion is most likely to occur, SAFAGM uses this map to guide its spatial attention mechanism, allowing the network to focus on spectrally ambiguous areas.

To incorporate this physical prior, SAFAGM combines the high-resolution cloud–snow cues (NDSIM) with deeper, semantically richer features ($Y_{in}$), and modulates them using attention cues derived from NDSIM. This design enables SAFAGM to selectively enhance cloud-related responses while suppressing snow-induced activations, thereby improving discrimination in challenging cloud–snow coexistence regions. In this way, SAFAGM serves as the pixel-level physical-prior branch in the physical–semantic dual-prior design, specifically addressing cloud–snow ambiguity in high-reflectance areas.

The module first fuses these features through a convolution and pooling operation:

$$X_{hc}={\mathrm{Conv}}_{3\times 3}\left(NDSIM\right)+Y_{in}.$$

(17)

To capture both spectral and spatial dependencies, the module computes parallel attention maps, following the general channel–spatial attention paradigm [64,65] but conditioned here on the NDSIM physical-prior feature:

$$X_c=\mathrm{ms}(X_{hc})$$

(18)

$$X_s=\mathrm{cs}(X_{hc})$$

(19)

where ms represents a sequence of reshape, multi-layer perceptron (MLP), and sigmoid operations for channel attention, while cs represents convolution and sigmoid operations for spatial attention. These mechanisms allow the network to selectively emphasize discriminative spectral channels and spatially important regions.

The attention-weighted features are subsequently computed as:

$$A_s=X_s\otimes X_{hc}$$

(20)

$$A_c=X_c\otimes X_{hc}$$

(21)

where ⊗ denotes element-wise multiplication. The final output of the combined module is obtained through a residual connection that preserves original feature information while injecting the attention-guided refinements:

$$X_{out}=X_{hc}+\mathrm{conv}(A_s+A_c)$$

(22)

By coupling the explicit spectral priors from NDSIM with the learnable attention mechanisms of SAFAGM, this architecture significantly improves performance in complex cloud-snow coexistence scenarios where visual appearance alone is insufficient.

3.4. Label-Guided Channel Attention Module (LG-CAM)

To further enhance the discriminative capacity of skip-connection features, we introduce a Label-Guided Channel Attention Module (LG-CAM). Unlike conventional attention mechanisms that rely solely on feature activations, LG-CAM injects scene-level semantic priors inferred from geographic metadata through an LLM, motivated by geospatial knowledge extraction and remote-sensing language-vision studies [12,13,58]. Together with the pixel-level physical prior (NDSIM), LG-CAM forms a physical–semantic dual-prior design: NDSIM addresses local cloud–snow ambiguity, while LG-CAM provides global semantic cues such as snow presence, desert terrain, or dense vegetation.

Unlike conventional vision-only cloud detection approaches, the proposed LG-CAM module incorporates contextual information derived from geographic metadata. These metadata, including acquisition time and geographic coordinates, provide complementary information that is independent of image appearance. By converting such metadata into semantic labels using a large language model, the network gains additional environmental context that helps reduce ambiguities caused by visually similar surfaces.

This design is motivated by the observation that traditional cloud detection methods often suffer from a “semantic gap”: they attempt to segment clouds at the pixel level without prior knowledge of the underlying surface type (e.g., snow, desert, or vegetation). This lack of context makes identifying clouds in complex scenarios inefficient and prone to errors, particularly in cloud-snow coexistence regions.

To bridge this gap, we reframe the LG-CAM not merely as an attention mechanism, but as a precise semantic prior inference module. By leveraging the vast knowledge base of Large Language Models (LLMs), we perform global scene classification before pixel-level processing utilizing only geographic metadata (latitude, longitude, and time). The LLM acts as a high-level reasoner, explicitly answering prompts such as “Is there snow cover at this location during this season?”.

This pre-classification yields a semantic label vector that encodes the environmental context. LG-CAM then translates these high-level semantic priors into channel-wise modulation weights. The structure of LG-CAM is shown in Figure 8. Consequently, the network is “informed” of the surface type in advance, allowing it to adaptively suppress background noise (e.g., snow reflectance) and focus on cloud-specific features. This “Pre-classify, then Detect” paradigm offers a more natural and robust solution compared to traditional blind feature extraction.

Figure 8. Structure of the Label-Guided Channel Attention Module (LG-CAM). By processing geographic metadata via an LLM, this module generates semantic labels to modulate feature channels, explicitly injecting environmental priors (e.g., snow cover) into the network. The metadata-derived semantic label vector is projected by a linear layer and Sigmoid activation to generate channel-wise modulation weights, which are then applied to skip-connection features through residual-style channel reweighting.

LLM-Based Semantic Label Generation and Validation

In terms of implementation, the semantic label vector is constructed through a constrained metadata-driven question-answering pipeline. For each satellite image, we first extract the available metadata, including latitude and longitude coordinates, acquisition time, imaging angle, and sensor-related information. These metadata are then processed by DeepSeek-R1 [66] to infer scene-level semantic attributes. When online geographic information is available, it is used only as auxiliary context for the same binary decision process.

Instead of asking the LLM to produce free-form scene descriptions, we query each semantic attribute independently using a fixed binary prompt. Taking snow-cover inference as an example, the prompt template is:

This is the metadata of a satellite image, including latitude and longitude coordinates, acquisition time, imaging angle, and other available information. Based on the above metadata and relevant geographic knowledge, please answer the following question: Is there snow cover on the land surface of this satellite image? If yes, output 1; if no, output 0. Please output only one number, either 1 or 0.

After receiving the LLM response, an automatic format-checking step is performed. If the response contains only a valid binary value, i.e., 0 or 1, the value is accepted as the semantic label for the current attribute, and the procedure moves to the next predefined question. If the response contains explanatory text, missing values, or non-binary answers, the same question is re-issued until a valid binary output is obtained. This validation step ensures that the LLM output can be directly converted into structured labels rather than uncontrolled natural-language descriptions.

The same sequential procedure is applied to the remaining semantic attributes according to the following predefined label schema:

• Is there snow cover? Yes → 1, No → 0
• Is there dense vegetation cover? Yes → 1, No → 0
• Is the terrain desert? Yes → 1, No → 0
• Are there large water bodies in the background? Yes → 1, No → 0

To further reduce semantic label noise, the accepted labels are checked against the corresponding metadata context and representative image samples. The final validated binary answers are concatenated into a scene-level semantic label vector. This yields a 4-dimensional label vector $\mathbf{y}\in {\{0,1\}}^L$ (with $L=4$), representing semantic priors for the current scene.

Given a label vector $\mathbf{y}\in {\{0,1\}}^L$, a learnable linear projection followed by a Sigmoid is applied to generate channel attention coefficients. This label-conditioned modulation is inspired by feature-wise conditioning mechanisms such as FiLM [67], but is adapted here for channel reweighting in cloud-detection skip connections:

$$\gamma =\sigma (W\mathbf{y}+b),\gamma \in {(0,1)}^C,$$

(23)

where $W\in {\mathbb{R}}^{C\times L}$ and $b\in {\mathbb{R}}^C$ are trainable parameters. The vector $\gamma$ is reshaped to $\tilde{\gamma }\in {\mathbb{R}}^{C\times 1\times 1}$ and broadcast to match the skip-connection feature map $X\in {\mathbb{R}}^{C\times H\times W}$.

Channel-wise modulation is then performed via element-wise multiplication:

$$X^{\prime }=X\odot (1+\tilde{\gamma }),$$

(24)

where $X^{\prime }$ denotes the reweighted skip connection. This residual-style gating strengthens semantically relevant channels while avoiding overly suppressing the original features. LG-CAM is inserted before each skip-connection feature concatenation in the decoder, except at the shallowest layer, allowing semantic priors to guide the multi-scale feature fusion process.

3.5. Datasets and Experimental Settings

3.5.1. Datasets

This study utilizes two datasets: GF-1 WFV and Sentinel-2A, sourced from the Gaofen-1 satellite and the Sentinel-2 mission, respectively. For GF-1, we adopt a preprocessed patch dataset derived from the GF1_WHU dataset: the original scenes are tiled into $512\times 512$ patches, patches containing NoValue (invalid background) pixels are discarded, and the original labels are remapped [1,68].

Launched in 2013, Gaofen-1 is the first satellite of China’s high-resolution Earth observation system [2]. It is equipped with two 2-m panchromatic cameras, an 8-m multispectral camera (PMS), and four 16-m multispectral wide-field cameras (WFV) [2,69]. The PMS sensor has a swath width of 60 km, while each WFV sensor covers up to 800 km, enabling global coverage every four days [2,69]. Its multispectral bands span the visible to near-infrared range (0.45–0.89 μm), designed to support land cover classification and change detection [2]. However, its limited spectral bands—lacking critical wavelengths such as shortwave infrared—restrict its application in complex land cover analysis [2,3].

The Sentinel-2 mission, deployed by the European Space Agency (ESA), is a high-resolution multispectral Earth observation system designed to provide detailed monitoring data for various surface features and environmental changes [3]. With the two-satellite constellation, Sentinel-2 provides a 5-day global revisit capability, supporting environmental monitoring and disaster response [3]. Compared to Gaofen-1, Sentinel-2 offers significant advantages in spectral dimensionality: it provides 13 spectral bands covering visible, near-infrared, and shortwave infrared ranges, whereas Gaofen-1 WFV has only four bands [2,3]. Additionally, Sentinel-2 provides multiple spatial resolutions (10 m, 20 m, and 60 m); in this study, we use the 10-m resolution products for experiments [3].

Therefore, GF-1 WFV and Sentinel-2A differ in three main aspects. First, their spatial resolutions are different: GF-1 WFV provides 16-m multispectral imagery, whereas Sentinel-2A provides bands at 10 m, 20 m, and 60 m. Second, their spectral dimensionalities differ substantially: GF-1 WFV contains four visible/near-infrared bands, whereas Sentinel-2A contains 13 bands. Third, their wavelength coverage is different: GF-1 WFV covers approximately 0.45–0.89 μm, while Sentinel-2A extends from visible and near-infrared wavelengths to shortwave infrared wavelengths. Before cross-modality training, the Sentinel-2A bands are organized into 13-channel target patches and spatially normalized to the same tensor size as the GF-1 patches. This preprocessing allows the translation network to learn spectral-domain correspondence while using the GF-1 image grid as the structural reference for downstream cloud detection.

3.5.2. Experimental Environment

The hardware configuration consists of an AMD EPYC 7302 16-Core Processor, 128 GB DDR4 RAM (8 × 16 GB modules), and dual NVIDIA GeForce RTX 4090 GPUs. On the software side, the server runs Ubuntu 22.04 LTS, with experiments executed in a PyTorch 2.5.1 environment [70] accelerated by CUDA 12.2.2.

In Stage 1 training, we jointly train on the GF-1 and preprocessed Sentinel-2A training sets after spatial-size normalization. The batch size is set to 4, with the initial learning rate configured as 0.0001. The total number of epochs is set to 20.

In Stage 2 training, we freeze SEG and continue training based on the Stage 1 model. The batch size is set to 4, and the initial learning rate is also 0.0001. After the 50th epoch, the learning rate is reduced to one-tenth of the initial value. The total number of epochs for this stage is set to 100.

Both stages employ the Adam optimizer [71] for stable and efficient training.

3.5.3. Baseline Setting

For the cross-modality transfer experiment, we selected RS-Net as the baseline cloud detection model. RS-Net is a U-Net-based deep learning architecture tailored for remote sensing image processing, which has been verified in existing studies [7] to exhibit strong performance in multi-scale feature extraction and cloud boundary preservation, two capabilities that are essential for accurate cloud detection. Its compatibility with both single-band and multi-band remote sensing inputs also aligns with our experimental design, where cloud detection performance must be evaluated on both original GF-1 WFV (4-band) data and cross-modality translated (13-band) data. By using RS-Net as the unchanged baseline, we isolate the contribution of cross-modality spectral expansion and ensure that the observed metric gains are attributable to enhanced spectral information rather than architectural changes to the detection model.

3.5.4. Evaluation Metrics

In the objective evaluation of cloud detection, five core metrics from the image segmentation field were adopted following common classification and segmentation evaluation protocols [72,73]: Precision, Recall, False Alarm Rate (FAR), F1-score, and Intersection over Union (IoU). Among these metrics, Precision measures the ratio of true positives to all predicted positives, Recall measures the ratio of true positives to all actual positives, FAR quantifies the ratio of false positives to all background pixels, F1-score summarizes the balance between Precision and Recall, and IoU evaluates the overlap between predicted and reference cloud regions. Together, these metrics characterize cloud detection performance in terms of positive prediction quality, coverage, false-alarm control, and boundary agreement.

$$Precision={\displaystyle \frac{TP}{TP+FP}}$$

(25)

$$Recall={\displaystyle \frac{TP}{TP+FN}}$$

(26)

$$FAR={\displaystyle \frac{FP}{FP+TN}}$$

(27)

$$F1\text{-}score={\displaystyle \frac{2\times Precision\times Recall}{Precision+Recall}}$$

(28)

$$IoU={\displaystyle \frac{TP}{TP+FP+FN}}$$

(29)

Here, TP (true positives) denotes correctly predicted cloud pixels, TN (true negatives) denotes correctly identified non-cloud pixels, FP (false positives) denotes non-cloud pixels incorrectly labeled as clouds, and FN (false negatives) denotes cloud pixels that were missed by the model.

4. Results

4.1. Cross-Modality Transfer Results

To evaluate the impact of cross-modality spectral expansion on cloud detection, we trained RS-Net using the spectrally translated GF-1 data and conducted performance assessment on the corresponding test set. As shown in Table 1, without introducing additional annotations or modifying the RS-Net architecture, multiple key metrics exhibit clear improvements: the F1-score increases from 91.02% to 92.84% (+1.82%), IoU improves by 3.02 percentage points, and FAR decreases to 3.33% (−0.64 percentage points). These results demonstrate that cross-modality spectral expansion provides more discriminative spectral cues, thereby enhancing the separability of thin clouds and cloud boundaries beyond what visual enhancement alone can offer.

Table 1. Cloud detection performance before and after cross-modality transfer.

Baseline	SEG	SSFM	SAFAGM	LG-CAM	Precision (%)	Recall (%)	FAR (%)	F1-Score (%)	IoU (%)
✓	✕	✕	✕	✕	94.34	87.92	3.97	91.02	83.39
✓	✓	✕	✕	✕	94.48	91.26	3.33	92.84	86.41
✓	✓	✓	✕	✕	95.77	91.85	3.85	93.77	88.02
✓	✓	✓	✓	✕	96.53	91.21	2.80	93.79	88.16
✓	✓	✕	✓	✓	95.91	92.35	2.40	94.10	88.85
✓	✓	✓	✓	✓	96.94	92.00	2.02	94.41	89.40

Abbreviations: SEG, Spectral Expansion Generator; SSFM, Spatial-Spectral Fusion Module; SAFAGM, Snow-Aware Feature Attention Guidance Module; LG-CAM, Label-Guided Channel Attention Module; FAR, False Alarm Rate; IoU, Intersection over Union. Bold values indicate the best performance for each metric.

Qualitative inspection further shows that the translated images preserve structural integrity and produce smoother cloud edges. The generated images exhibit reduced color deviation, clearer textures, and improved stability in high-reflectance scenarios such as snow and building rooftops. Overall, the results demonstrate that cross-modality translation effectively enhances spectral discriminability for cloud detection, especially in thin-cloud and cloud-edge regions.

4.2. Spectral Fidelity Evaluation of Reconstructed SWIR-like Bands

To evaluate whether the reconstructed SWIR-like bands preserve meaningful spectral cues, we conducted an auxiliary Sentinel-2 band-holdout reconstruction experiment. Specifically, the Sentinel-2 bands most similar to the GF-1 WFV spectral configuration, including B2 (Blue), B3 (Green), B4 (Red), and B8 (NIR), were used as four-band inputs to reconstruct the SWIR bands B11 and B12. The reconstructed SWIR-like bands were then compared with the real Sentinel-2 SWIR observations using root mean square error (RMSE), Pearson correlation, and spectral angle mapper (SAM).

The error, correlation, and spectral-angle metrics are defined as follows:

$$\mathrm{RMSE}(A,\widehat{A})=\sqrt{{\displaystyle \frac{1}{N}}\sum _{i=1}^N{\left(A_i-{\widehat{A}}_i\right)}^2},$$

(30)

$$\mathrm{MAE}(A,\widehat{A})={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}\left|A_i-{\widehat{A}}_i\right|,$$

(31)

$$\mathrm{Corr}(A,\widehat{A})={\displaystyle \frac{{\sum }_{i=1}^N(A_i-\overline{A})({\widehat{A}}_i-\overline{\widehat{A}})}{\sqrt{{\sum }_{i=1}^N{(A_i-\overline{A})}^2}\sqrt{{\sum }_{i=1}^N{({\widehat{A}}_i-\overline{\widehat{A}})}^2}}},$$

(32)

$$\mathrm{SAM}(\mathbf{s},\widehat{\mathbf{s}})=arccos\left({\displaystyle \frac{\mathbf{s}·\widehat{\mathbf{s}}}{{\parallel \mathbf{s}\parallel }_2{\parallel \widehat{\mathbf{s}}\parallel }_2}}\right),$$

(33)

where A and $\widehat{A}$ denote the real and reconstructed band values, respectively, and $\mathbf{s}$ and $\widehat{\mathbf{s}}$ denote the real and reconstructed spectral vectors.

Furthermore, to verify whether the reconstructed B11 can support NDSI-based cloud–snow discrimination, we computed NDSI using both real B11 and reconstructed B11:

$${\mathrm{NDSI}}_{real}={\displaystyle \frac{B_3-B_{11}}{B_3+B_{11}+ϵ}},{\mathrm{NDSI}}_{pseudo}={\displaystyle \frac{B_3-{\widehat{B}}_{11}}{B_3+{\widehat{B}}_{11}+ϵ}}.$$

(34)

As shown in Table 2, the reconstructed B11 and B12 achieved correlation coefficients of 0.812641 and 0.852589, respectively, with RMSE values of 0.086228 and 0.064325. Since the reflectance values were normalized to [0,1], these results indicate that the reconstructed SWIR-like bands have acceptable numerical errors and maintain relatively strong spatial correlation with the real Sentinel-2 SWIR bands. The SAM value of 7.159606° further suggests that the overall spectral shape is reasonably preserved, although the reconstruction is still not equivalent to direct physical SWIR measurement.

Table 2. Spectral fidelity evaluation of reconstructed SWIR-like bands on the Sentinel-2 band-holdout experiment.

Metric	Value
B11 RMSE ↓	0.086228
B11 Corr ↑	0.812641
B12 RMSE ↓	0.064325
B12 Corr ↑	0.852589
SAM ↓	7.159606°
NDSI RMSE ↓	0.227730
NDSI MAE ↓	0.145995
NDSI Corr ↑	0.830303

The upward arrow indicates that a higher value is better, whereas the downward arrow indicates that a lower value is better. SAM is reported in degrees; RMSE, MAE, Corr, and NDSI-based metrics are unitless because the reflectance values are normalized.

For NDSI, the RMSE and MAE were 0.227730 and 0.145995, respectively, indicating that the absolute NDSI values derived from reconstructed B11 contain non-negligible deviations. However, the NDSI correlation still reached 0.830303, demonstrating that the reconstructed NDSI preserves a relatively strong spatial distribution trend compared with the real NDSI. Therefore, the reconstructed SWIR-like bands should not be interpreted as exact physical replacements for real SWIR measurements, but they preserve task-relevant SWIR-like cues that can support NDSI-based cloud–snow discrimination.

4.3. Ablation Results on the Cloud Detection Network

4.3.1. Effectiveness of Spectral–Spatial Fusion Module

To verify whether SSFM improves multispectral processing, we replaced every plain convolution in the encoder of the baseline with SSFM and conducted a comprehensive evaluation on the same 22-scene test set. As reported in Table 1, the upgraded network outperforms the baseline by a clear margin: Precision rises by 1.29%, F1-Score rises by 0.93%, and IoU improves by 1.61%. These gains confirm that SSFM is beneficial for multispectral imagery. Traditional CNNs are mainly tuned for RGB inputs and tend to under-explore the abundant spectral dimensions; SSFM explicitly disentangles spatial and spectral features, fuses them through additive and subtractive interaction paths, and thus enriches the representation. Consequently, the module simultaneously strengthens spatial detail recovery and spectral discriminability, leading to consistent performance improvements on complex multispectral scenes.

To further isolate the contribution of SSFM under the full dual-prior framework, we additionally evaluated a leave-one-out variant that removes SSFM while retaining SEG, SAFAGM, and LG-CAM. As shown in Table 1, this variant obtains 95.91% Precision, 92.35% Recall, 2.40% FAR, 94.10% F1-Score, and 88.85% IoU. Compared with the complete model, removing SSFM decreases Precision, F1-Score, and IoU by 1.03, 0.31, and 0.55 percentage points, respectively, and increases FAR by 0.38 percentage points, although it slightly increases Recall by 0.35 percentage points. This indicates that SSFM improves the balance between false-alarm suppression and boundary-overlap quality by enhancing spatial–spectral representation.

These results confirm that integrating spectral cues with spatial clues is crucial for robust cloud identification.

4.3.2. Effectiveness of Snow-Aware Feature Attention Guidance Module

To evaluate the contribution of Snow-Aware Feature Attention Guidance Module, we inserted it between the shallowest encoder stage and the decoder of the network. Comparative experiments on the 22-scene test set (Table 1) show consistent gains: Precision by 0.76% and IoU rises by 0.14%. These increments indicate that SAFAGM successfully focuses the network on discriminative regions. By exploiting the additional snow/cloud contrast provided by NDSIM, the module learns spatial–channel attention weights that suppress snow confusion and highlight cloud boundaries, leading to more accurate and robust cloud masks in complex multispectral scenes.

To provide a more direct view of SAFAGM’s contribution under snow-interference conditions, we further report the quantitative comparison before and after enabling SAFAGM on a test set containing prominent snow-covered scenes in Table 3. Introducing SAFAGM increases Precision from 96.73% to 98.39% (+1.66%), Recall from 91.97% to 93.73% (+1.76%), reduces FAR from 2.54% to 1.26% (−1.28 percentage points), and improves F1-Score and IoU by 1.71 and 3.12 percentage points, respectively. These results indicate that SAFAGM effectively suppresses snow-induced false alarms in snow–cloud confusing conditions, thereby improving the reliability of the predicted cloud masks.

Table 3. Ablation comparison of SAFAGM under snow interference conditions (before vs. after enabling).

Variant	Precision (%)	Recall (%)	FAR (%)	F1-Score (%)	IoU (%)
Without SAFAGM	96.73	91.97	2.54	94.29	89.19
With SAFAGM	98.39	93.73	1.26	96.00	92.31

Bold values indicate the best performance for each metric.

Figure 9 employs color-coded regions to highlight different types of classification outcomes. The cyan regions denote cloud pixels that are successfully identified, whereas the black regions represent correctly detected non-cloud pixels. The purple regions correspond to cloud pixels that should have been detected but were incorrectly classified as non-cloud, while the yellow regions indicate non-cloud pixels that were wrongly predicted as cloud. Through this visual decomposition, it can be clearly observed that the module markedly reduces the probability of snow being misclassified as cloud, providing intuitive evidence of its improved behavior in complex snow–cloud mixed scenes.

Figure 9. The top-left shows the original image, the top-right shows the ground-truth labels, the bottom-left shows the prediction without SAFAGM, and the bottom-right shows the prediction with SAFAGM applied. Cyan indicates correctly detected cloud pixels, black indicates correctly detected non-cloud pixels, purple indicates missed cloud pixels, and yellow indicates false-alarm non-cloud pixels predicted as cloud.

4.3.3. Effectiveness of LG-CAM

To further evaluate the contribution of the proposed Label-Guided Channel Attention Module (LG-CAM), we insert it into all skip connections of the cloud detection network and conduct experiments on the 22-scene test set. The quantitative results in Table 1 reveal a remarkable improvement across all major metrics, demonstrating that LG-CAM enhances not only localized features but also the overall discriminative capability of the entire architecture.

Compared with the variant equipped with SAFAGM, SSFM, and SEG, the introduction of LG-CAM leads to the highest Precision (96.94%), best F1-score (94.41%), and best IoU (89.40%) among the main configurations. The leave-one-out variant without SSFM achieves a slightly higher Recall (92.35%), but its lower Precision, higher FAR (2.40%), lower F1-score, and lower IoU indicate that the complete model provides a more balanced segmentation performance. Furthermore, FAR remains low at 2.02%, reflecting a significant reduction of false alarms in challenging cloud–snow mixtures.

These consistent gains verify that LG-CAM leverages semantic label priors to refine channel-wise responses in skip-connection features. By adaptively reweighting feature channels according to their relevance to cloud regions, the module suppresses snow-like noise, improves boundary coherence, and reinforces multi-scale contextual representation.

Overall, the results demonstrate that LG-CAM substantially strengthens the network’s comprehensive performance, making it particularly effective for cloud detection in complex multispectral scenes with thin clouds, cloud edges, and cloud–snow coexistence.

4.4. Comparison with State-of-the-Art Methods

Table 4 presents the quantitative comparison results against state-of-the-art methods, including RS-Net, CDNetV2, BABFNet, and MCDNet. For completeness, we also include published GF1_WHU results of CloudMamba; however, this method does not report all five metrics under the same evaluation protocol, and unavailable entries are marked with “-”. The proposed framework significantly outperforms all baselines that are evaluated under our unified experimental setting across the five evaluation metrics. Specifically, CMSE-DPNet (ours) achieves the highest Precision (96.94%) and Recall (92.00%), yielding a superior F1-score of 94.41% and IoU of 89.40%. Compared to the second-best model (BABFNet), our approach achieves improvements of 2.13% in F1-score and 3.79% in IoU.

Table 4. Quantitative comparison results (Precision, Recall, FAR, F1-Score, IoU).

Method	Precision (%)	Recall (%)	FAR (%)	F1-Score (%)	IoU (%)
RS-Net [7]	94.34	87.92	3.97	91.02	83.39
CDNetV2 [49]	94.64	89.94	3.08	92.23	85.45
BABFNet [8]	93.30	91.29	4.14	92.28	85.61
MCDNet [9]	94.78	89.60	3.22	92.12	85.39
CloudMamba [74]	-	-	-	94.33	89.27
CMSE-DPNet (ours)	96.94	92.00	2.02	94.41	89.40

Metrics unavailable in the original paper are denoted by “-”. Bold values indicate the best performance for each metric.

Most notably, the FAR is drastically reduced to 2.02%, whereas competing methods range between 3.08% and 4.14%. This substantial reduction is primarily attributed to the Label-Guided Channel Attention Module (LG-CAM), which utilizes an LLM to inject global semantic priors. Specifically, the module generates scene-level labels for four key environmental conditions–snow cover, dense vegetation, desert terrain, and large water bodies–based on geographic metadata. By explicitly identifying these background types, the network effectively suppresses interference from spectrally similar regions (e.g., distinguishing snow from clouds), thereby preventing false positives in complex scenarios.

To intuitively demonstrate this superiority, Figure 10 visualizes the cloud detection results on representative challenging scenes. As shown in the figure, traditional methods such as RS-Net (c) and CDNetV2 (d) exhibit noticeable false alarms in high-reflectance background regions. While advanced models like BABFNet (e) and MCDNet (f) reduce some misclassifications, their performance remains suboptimal. In contrast, the CMSE-DPNet (ours) (g) generates cloud masks that are highly consistent with the ground truth (b). By leveraging explicit scene-level classification priors via LG-CAM to guide segmentation, our model effectively eliminates snow-induced false positives while preserving fine-grained cloud details.

Figure 10. Comparison of subjective results from different algorithms. (a) Original RGB image; (b) Ground truth; (c) RS-Net; (d) CDNetV2; (e) BABFNet; (f) MCDNet; (g) CMSE-DPNet (ours). The visual comparison highlights that our method in panel (g) aligns best with the ground truth in panel (b), significantly reducing false alarms in snow-covered areas compared to the baselines in panels (c–f). In the color-coded prediction panels, cyan and black denote correctly classified cloud and non-cloud regions, respectively, while purple and yellow denote missed cloud regions and false-alarm regions, respectively.

5. Discussion

The results consistently show that the proposed framework benefits from combining expanded spectral cues with explicit physical and semantic priors. Cross-modality spectral expansion improves the separability of clouds from confusing high-reflectance backgrounds by reconstructing pseudo-Sentinel-2 spectral information, while the subsequent cloud detection network further exploits these cues through spatial–spectral fusion and prior-guided attention. This staged design explains why the gains are not limited to a single metric: Precision, F1-score, and IoU all improve together, indicating better discrimination, fewer false alarms, and more coherent cloud boundaries.

From a methodological perspective, the results suggest that the physical and semantic prior branches play complementary roles. The NDSIM–SAFAGM pathway mainly addresses local cloud–snow ambiguity by injecting pixel-level physical guidance derived from reconstructed SWIR-sensitive information. In contrast, LG-CAM contributes scene-level contextual knowledge inferred from geographic metadata, which is particularly useful when visually similar land-cover types create ambiguity beyond the capability of local appearance features alone. The substantial reduction in FAR compared with competing methods further supports the interpretation that semantic guidance helps suppress systematic false positives in challenging environments.

The auxiliary spectral fidelity experiment further clarifies the interpretation of the NDSI branch. Although the reconstructed SWIR-like bands show high correlation with real Sentinel-2 SWIR bands, they are not equivalent to physically measured SWIR reflectance. Therefore, the NDSI branch is interpreted as a task-oriented physical prior that preserves cloud–snow discrimination cues, rather than as a strict sensor-level physical measurement.

This study also has several strengths. First, it offers a practical solution for spectrally limited GF-1 imagery without requiring hardware modification. Second, the framework is modular: Spectral expansion, physical priors, and semantic priors can be analyzed separately through ablation experiments, which makes the design easier to interpret than a purely monolithic architecture. Third, both quantitative and qualitative comparisons indicate that the approach remains effective in difficult cloud–snow coexistence scenarios, where many existing methods degrade.

At the same time, several limitations should be acknowledged. The semantic prior branch depends on the availability and reliability of geographic metadata, and the current experiments are conducted on a specific sensor setting and dataset. In addition, the spectral expansion and segmentation stages are optimized sequentially rather than jointly, which may limit the final end-to-end performance. Future work could therefore explore larger and more diverse datasets, adaptation to additional satellite platforms, and tighter joint optimization between spectral reconstruction and cloud segmentation. More advanced multimodal language–vision models may also provide richer and more adaptive scene priors for operational cloud detection.

6. Conclusions

In this paper, we presented a unified framework for cloud detection in GF-1 imagery that couples Cross-Modality Spectral Expansion with Physical–Semantic Dual Priors Feature Modulation. An improved CycleGAN-based transfer network reconstructs 13-band pseudo Sentinel-2A spectra from 4-band GF-1 WFV data, equipped with U-shaped Spectral Self-Attention blocks and structure-aware constraints to better preserve both spectral fidelity and geometric details. By explicitly recovering SWIR information, the transfer stage establishes the necessary spectral basis for computing NDSI and enables the use of physical snow indices that are otherwise unavailable for GF-1 sensors.

On top of the spectrally expanded data, we design a cloud-detection network that jointly exploits spatial, spectral, and semantic cues. The Spatial–Spectral Fusion Module (SSFM) disentangles and recombines spatial textures and spectral responses, enhancing the representation of thin clouds and cloud boundaries. The Normalized Difference Snow Index Module (NDSIM) provides a transfer-enabled pixel-level physical prior, which is embedded into the Snow-Aware Feature Attention Guidance Module (SAFAGM) to form snow-aware spatial–channel attention. In parallel, the LLM-driven Label-Guided Channel Attention Module (LG-CAM) injects scene-level semantic priors derived from geographic metadata, implementing a “Pre-classify, then Detect” paradigm that bridges the gap between pixel-level features and global environmental context.

Despite these promising results, our method still has limitations. The LLM-based semantic priors depend on the availability and accuracy of geographic metadata, and the current framework is trained and evaluated on a specific sensor configuration and region. In future work, we plan to extend the cross-modality transfer and dual pre-classification framework to other satellite sensors and larger-scale datasets, explore end-to-end joint optimization of the transfer and detection stages, and investigate more advanced multimodal language–vision models to provide richer, temporally adaptive scene priors for robust cloud detection in diverse operational scenarios.