Evaluation of Cloud Mask Performance of KOMPSAT-3 Top-of-Atmosphere Reflectance Incorporating Deeplabv3+ with Resnet 101 Model

Abstract

Cloud detection is a crucial task in satellite remote sensing, influencing applications such as vegetation indices, land use analysis, and renewable energy estimation. This study evaluates the performance of cloud masks generated for KOMPSAT-3 and KOMPSAT-3A imagery using the DeepLabV3+ deep learning model with a ResNet-101 backbone. To overcome the limitations of digital number (DN) data, Top-of-Atmosphere (TOA) reflectance was computed and used for model training. Comparative analysis between the DN and TOA reflectance demonstrated significant improvements with the TOA correction applied. The TOA reflectance combined with the NDVI channel achieved the highest precision (69.33%) and F1-score (59.27%), along with a mean Intersection over Union (mIoU) of 46.5%, outperforming all the other configurations. In particular, this combination was highly effective in detecting dense clouds, achieving an mIoU of 48.12%, while the Near-Infrared, green, and red (NGR) combination performed best in identifying cloud shadows with an mIoU of 23.32%. These findings highlight the critical role of radiometric correction and optimal channel selection in enhancing deep learning-based cloud detection. This study demonstrates the crucial role of radiometric correction, optimal channel selection, and the integration of additional synthetic indices in enhancing deep learning-based cloud detection performance, providing a foundation for the development of more refined cloud masking techniques in the future.

1. Introduction

In the field of satellite remote sensing, detecting cloud masks, including shadows, is crucial. This is not only vital for analyzing atmospheric cloud properties [1,2] but also significantly impacts higher-value-added products related to the land surface, such as vegetation indices, surface reflectance, water, and land use [3,4,5]. Clouds and their shadows limit the potential applications of satellite imagery in extracting accurate land surface information [6]. In addition, cloud detection is also one of the key variables for accurately estimating renewable solar energy outputs [7,8]. With the advent of the New Space era characterized by the launch and utilization of numerous satellites, the number of researchers who have increasingly focused on extracting clouds and cloud shadows from satellite imagery has markedly increased recently [9]. Therefore, improving our ability to accurately detect clouds and cloud shadows is crucial for ensuring the quality of satellite products and for successfully achieving objectives in various fields of application, including renewable solar energy potential.

The initial stages of developing cloud detection algorithms are focused on simple thresholding techniques of multispectral satellite bands, known as rule-based classification [10,11]. These methods utilize the physical properties of clouds in different bands, particularly the visible and infrared channels. With advances in satellite technology, researchers have developed rule-based cloud detection algorithms to leverage higher spatiotemporal resolution and more spectral bands. The MODIS cloud mask product [12] utilizes a variety of tests, including brightness temperature differences, reflectance thresholds, and spatial variability, to enhance cloud detection accuracy. The most well-known cloud detection method for Landsat satellite imagery is Fmask [6,13,14], which achieves high accuracy in most circumstances, even for mountainous areas and bright land surfaces. Despite these advancements, detecting thin cirrus clouds and distinguishing cloud shadows remain challenging. Furthermore, Fmask requires auxiliary data such as elevation, along with imagery from all channels, and its computational process can take several minutes, making it difficult to apply to large-scale satellite datasets.

Traditional rule-based thresholding techniques for cloud detection have evolved over time, demonstrating their potential to improve accuracy through key spectral indices. For example, Ge et al. (2023) utilized TOA reflectance, NDVI, and several derived values to classify clouds, applying fixed thresholds to distinguish clouds from vegetation and other land surface features [15]. This approach effectively reduced the false detections caused by similar spectral characteristics. However, these methods still rely on manually set thresholds, which limits their adaptability under varying atmospheric and surface conditions. As the need for robust detection systems grew, machine learning and deep learning methods gained traction [16,17,18], improving cloud detection by learning complex patterns and better distinguishing clouds, shadows, and the Earth’s surface. Despite these improvements, these methods require large amounts of labeled training data.

Building on the potential of deep learning, Hughes and Hayes (2014) developed a neural-network-based model that utilizes both the spectral band data and pre-computed spatial features from the images to identify clouds and their shadows [19]. Weiland et al. (2019) extended this by developing a model that uses convolutional neural networks (CNNs) [20], which can process 3D inputs and simultaneously integrate both spatial and spectral information. This approach has demonstrated reliable performance in cloud detection [16,21]. As a result, numerous cloud detection training datasets have been established to support the development of cloud detection models for various satellites. These datasets vary in their data processing levels: some are derived from Top-of-Atmosphere (TOA) reflectance or radiance, while others use digital numbers (DNs) from multispectral bands. For instance, cloud detection training datasets from satellites such as Sentinel-2A and 2B (Sen2Cor) [10], Landsat 8 (L8-Biome, L8-SPARCS, L8-95 Cloud [22]; https://landsat.usgs.gov/landsat-8-cloud-cover-assessment-validation-data (accessed on 7 April 2025)), and PeruSat [23] are provided as TOA reflectance or radiance, which are radiometrically corrected products. In contrast, other studies provide training datasets in a DN (Digital Number) format for deep learning-based cloud detection models.

For instance, Kim et al. (2022) evaluated the performance of cloud detection techniques using a modified DeepLabV3+ model with heterogeneous sensors [21], such as the Landsat and KOMPSAT-3 satellites. They found that, despite improvements to the DeepLabV3+ network, which allowed the model to extract and learn channel features from two distinct training datasets, the Jaccard index for clouds remained low due to differences in the datasets. Specifically, the KOMPSAT-3 dataset was distributed in a DN format, while the Landsat L8-Biome training dataset consisted of radiometrically calibrated image products.

In other words, even though the deep learning model applied normalization, training on DN data from KOMPSAT-3 and applying the model to radiometrically calibrated Landsat data lowered accuracy. The KOMPSAT-3 data were used without additional radiometric correction because the Korea Aerospace Research Institute (KARI) provided the training datasets in a DN format rather than in TOA (Top-of-Atmosphere) reflectance or radiance. Although the normalization process of the matchup dataset was expected to reduce the discrepancies between the datasets, it proved insufficient in this case. The absence of standardized training datasets has led to a lack of clear criteria for determining the optimal data format for training deep learning models. While several studies have demonstrated that deep learning approaches using either digital numbers (DNs) or Top-of-Atmosphere (TOA) reflectance provide more reliable cloud masking results compared to those of the physical or empirical models, inconsistencies in data formats can affect model training and performance evaluations.

In conventional cloud mask algorithms, the input for cloud detection should ideally be TOA reflectance rather than DN values to reduce radiometric differences between images [6,24,25]. However, radiometrically calibrated satellite data have not always been applied in deep learning algorithms, as mentioned above. The existing deep learning models are considered robust to pixel value variations due to their complex architecture; however, the impact of converting DNs to TOA reflectance or using various derived indices on cloud segmentation accuracy remains uncertain. Thus, a systematic evaluation is required to determine the impact of these transformations on model performance. However, it is essential to conduct a comprehensive evaluation to establish whether these transformations are necessary for reliable cloud segmentation. Additionally, the lack of defined criteria for developing training datasets in DNs or radiometrically calibrated formats from satellite data may affect other fields, such as building segmentation, road detection, and change detection. Therefore, further analysis is needed to evaluate the performance of deep learning models when radiometric corrections are applied or omitted. Also, this would enhance the development of deep learning-based cloud detection models.

In this study, we tested which training dataset is more effective when applying deep learning technology to the Korea Multi-Purpose Satellite (KOMPSAT), specifically by using TOA reflectance and derived indices during model training, compared to using the default DN values. As mentioned, the KARI distributed a training dataset of cloud masks as DN images with corresponding label files. Therefore, we newly calculated the TOA reflectance from KOMPSAT-3 and the 3A DN training dataset by using the solar geometry and ESUN of the selected images. In addition, we also tested the performance of a deep learning algorithm changing the composite of channels for the DNs and TOA reflectance because the combination of satellite channels affects the accuracy of the cloud mask results [26].

2. Material and Methods

2.1. Overview of KOMPSAT-3 and KOMPSAT-3A Satellite Datasets

The Korea Aerospace Research Institute (KARI, Daejeon, Republic of Korea) developed and launched the high-resolution KOMPSAT-3 and KOMPSAT-3A Earth observation satellites in 2012 and 2015, respectively. KOMPSAT-3 is equipped with 2.8 m multispectral channels and a 0.7 m Panchromatic (PAN) channel from the optical camera (AEISS; Advanced Earth Imaging Sensor System) capable of providing precise optical images of the Earth’s surface, which are used for purposes such as environmental monitoring, disaster response, and resource management. KOMPSAT-3A, an upgraded version of KOMPSAT-3, features the existing AEISS sensor along with a newly developed infrared (IR) sensor, enabling Earth observation at night or in cloudy weather conditions.

Table 1. Characteristics of satellite sensors from KOMPSAT-3 and 3A for cloud masks and whether TOA radiance was applied or not.

Satellite	KOMPSAT 3	KOMPSAT 3A
Blue-band wavelength (um)	450–520	450–520
Green-band wavelength (um)	520–600	520–600
Red-band wavelength (um)	630–690	630–690
NIR-band wavelength (um)	760–900	760–900
Resolution (m)	2.8	2.2
Publicly opened cloud detection dataset	KOMPSAT-3	KOMPSAT-3A
Sensor	PAN, MSI	PAN, MSI
Launch year	2015	2017
Data acquisition period	3 days	3 days

presents the spectral channels and specifications of KOMPSAT-3 and 3A. In this study, we used datasets that the KARI refined and constructed from KOMPSAT-3 and KOMPSAT-3A for training and preparing cloud detection data, which are publicly available on AI Hub (https://www.aihub.or.kr/ (accessed on 7 April 2025)).

2.2. KOMPSAT-3 and KOMPSAT-3A Dataset Construction Process

The cloud mask dataset used in this study was provided by the Korea Aerospace Research Institute (KARI) and was designed using a multi-step approach to ensure high reliability and accuracy. Initially, a Land Use Land Cover (LULC) map provided by the European Space Agency (ESA) was utilized to achieve balanced diversity across various land cover types in the KOMPSAT-3 and KOMPSAT-3A datasets. Subsequently, the research team gathered cloud detection training data from satellite imagery that the KARI multi-purpose satellites acquired between 2019 and 2022, focusing on images with cloud cover between 20% and 50%. This selection was made because images with cloud cover exceeding 50% can adversely affect image registration quality, making images with 20% to 50% cloud cover more suitable for dataset construction.

Artificial intelligence (AI) techniques initially labeled approximately 20–30% of the data to automate the process. After this, a team of annotation workers, who had undergone specialized training, performed a second round of manual annotations following the KARI’s established criteria. The data preprocessing team and the AI model development team further refined the dataset through an iterative review process involving collaboration. This rigorous process ensured the dataset’s reliability, making it suitable for cloud mask development and validation for KOMPSAT-3 and KOMPSAT-3A.

In this study, the cloud detection dataset consists of multi-purpose satellite imagery, including red, green, blue (RGB) and Near-Infrared (NIR) channel data (as shown in Table 1), along with previously generated cloud detection labels. The cloud mask labels in the dataset are categorized into four classes: clear sky, thick cloud, thin cloud, and cloud shadow. In Figure 1d, the colors used to represent these classes are as follows: red for thick clouds, green for thin clouds, yellow for cloud shadows, and black for clear sky. Clear sky refers to cloud-free areas where the surface is fully visible. Thick clouds are characterized by high reflectance values in optical imagery, completely obscuring surface features beneath them. Thin clouds are commonly observed surrounding thick clouds, where surface features such as buildings or vehicles remain partially visible but appear more blurred than in clear sky regions. Additionally, pixels affected by aerosols or fog, which are often difficult to distinguish from clouds in optical imagery, are also classified as thin clouds. Cloud shadow is strictly defined as the shadow cast by clouds, where sunlight is blocked, creating darkened areas on the surface. Shadows caused by terrain or other obstructions are not included in this category.

In this study, the multi-purpose satellite images used were provided by the Korea Aerospace Research Institute (KARI) and consisted of a total of 177 images. Of these, we used 130 images for training, 16 for validation, and the remaining 31 for testing. The resolution of each image used for training and validation was approximately 6000 × 6000 pixels, with slight variations in size. To address these size inconsistencies, we divided the satellite images into patches of 800 × 800 pixels, with a 400-pixel overlap between patches to minimize discontinuities. (Figure 2). This overlap ensures seamless transitions between patches during model application, helping the model generalize effectively across diverse conditions.

As a result, the number of training images increased from 130 to 22,890, and the validation images increased from 16 to 2772, bringing the total to 25,662 images. However, of these, 12,504 images contained only clear sky without any clouds or shadows, leading to an imbalance in the dataset. To improve model performance, data augmentation techniques such as horizontal and vertical flipping, rotation, and brightness adjustment were applied during preprocessing to balance the number of samples across different classes (Figure 3).

In this study, we evaluated various satellite channel combinations for digital numbers (DNs) and Top-of-Atmosphere (TOA) reflectance to assess the accuracy of each dataset. Figure 4 shows sampled images from KOMPSAT with different channel combinations, including RGB, NRG, and RGB with NIR. Additionally, we tested RGBN combined with the NDVI (Normalized Difference Vegetation Index). TOA-based channel combinations were also evaluated, specifically TOA RGB and TOA RGBN with the NDVI, to determine their performance in deep learning models.

For the TOA analysis, only the channel combinations that achieved the highest accuracy in the DN-based evaluations, namely RGBN and RGBN with the NDVI, were further assessed. The model performance for each channel combination was then compared to identify the most effective setup for accurate data interpretation.

2.3. The Network Structure of the Deep Learning Model

Clouds in satellite imagery have amorphous, unclear shapes that differ significantly from typical objects. These characteristics make cloud detection difficult and highlight the need for pixel-wise labeling, which is widely used in image segmentation [21,23]. Deep learning has significantly advanced cloud detection networks. Researchers have moved from early CNN-based segmentation methods to more sophisticated architectures, including ResNet-based and U-Net-based models, which leverage large parameter sets to effectively handle the complex nature of clouds [27,28]. Among these, advanced models like CPNet and DeepLabV3+ have shown notable performance [29,30]. CPNet, also known as Cloud-Net, builds on the U-Net framework and integrates ResNet to improve gradient flow and feature extraction. It uses an encoder–decoder architecture specifically designed for cloud detection [31].

In contrast, researchers widely use DeepLabV3+ because of its strong performance in image segmentation. It originally employs Xception as its encoder instead of ResNet, thereby reinforcing the concept that channel and spatial correlations can be considered separately [32]. However, this experiment showed that the Xception module was less effective at capturing the amorphous and variable nature of cloud pixels, leading to the adoption of a deeper ResNet101 module as the encoder. ResNet101 utilizes residual learning through skip connections and a bottleneck architecture that applies 1 × 1, 3 × 3, and again 1×1 convolutions to effectively regulate the number of input channels, thereby reducing the parameter count and enhancing both computational efficiency and stability.

DeepLabV3+ employs an Atrous Spatial Pyramid Pooling (ASPP) module to effectively extract multi-scale features while preserving spatial resolution [30].

$${\mathrm{y}}_{\mathrm{r}}\left[\mathrm{i},\mathrm{j}\right]={\sum }_{\mathrm{m}}{\sum }_{\mathrm{n}}\mathrm{x}\left[\mathrm{i}+\mathrm{r}\cdot \mathrm{m},\mathrm{j}+\mathrm{r}\cdot \mathrm{n}\right]\cdot {\mathrm{w}}_{\mathrm{r}}\left[\mathrm{m},\mathrm{n}\right]$$

(1)

where there are the following:

• $y_r\left[i,j\right]$: the output value at coordinate $(i,j)$ after applying atrous convolution;
• $x\left[i+r\cdot m,j+r\cdot n\right]$: the pixel value in the input feature map, sampled at intervals defined by the atrous rate $r;$
• $w_r\left[m,n\right]$: the weights of the convolutional filter (kernel);
• $r$: the atrous rate, which determines the spacing between the sampled pixels;
• $m,n$: the indices of the kernel dimensions.

For instance, setting $r$ = 1, 2, 4 progressively expands the receptive field, enabling the network to capture both fine details and broader contextual information.

The ASPP module utilizes multiple parallel atrous convolutions with different dilation rates to capture features at various resolutions. In addition, a global context is captured via average pooling followed by a 1$\times$1 convolution as IN (2).

$$y_{pool}=Con{v}_{1\times 1}\left(AvgPool\left(x\right)\right)$$

(2)

The model generates the final ASPP output by concatenating the outputs from atrous convolutions with different rates (e.g., $r$ = 1, 6, 12, 18) along with $y_{pool}$ and processing the result through another 1$\times$1 convolution:

$$y_{ASPP}=Con{v}_{1\times 1}\left(Concat\left(y_1,y_6,y_{12},y_{18},y_{pool}\right)\right)$$

(3)

DeepLabV3+ ResNet101’s encoder–decoder structure preserves positional information, which is crucial for semantic segmentation tasks. By aggregating multi-scale features, ASPP enables the network to interpret the image from diverse perspectives, making it particularly effective for detecting clouds of varying sizes and shapes in satellite imagery.

2.4. Specifications of Hyperparameters

Originally introduced by Milletari et al. (2016) in their seminal work on semantic segmentation tasks, Dice Loss has proven to be an effective tool for tackling complex segmentation challenges [33]. In cloud segmentation tasks, class imbalance presents a significant hurdle, as the background (clear sky class) typically dominates the pixel distribution, vastly outnumbering the cloud regions. This imbalance often biases models toward over-predicting clear sky areas, yielding deceptively high accuracy metrics while failing to adequately detect clouds, which are the primary objects of interest. Dice Loss mitigates this issue by automatically balancing the foreground (clouds) and background (clear sky) without the need for manual class weight adjustments, a feature that distinguishes it from traditional approaches. This capability allows Dice Loss to outperform conventional loss functions, such as clear sky and multi-class logistic loss, particularly in imbalanced segmentation scenarios where minority classes require greater emphasis. Milletari et al. (2016) first demonstrated its effectiveness in medical image segmentation, a domain where similar class imbalance problems such as distinguishing small regions of interest like tumors from large backgrounds are prevalent [33]. Building on these findings, we extend the application of Dice Loss to cloud segmentation, anticipating comparable advantages in tackling class imbalance due to the structural similarities between these tasks. Recent studies further suggest that combining Dice Loss with pixel-weighting strategies can enhance its performance in satellite imagery, improving the delineation of cloud boundaries—an insight that reinforces its potential in this context. Dice Loss is mathematically derived from the Dice Similarity Coefficient, a metric that quantifies the overlap between the predicted segmentation and the ground truth, as expressed in Equation (1), making it particularly adept at optimizing for the precise segmentation of underrepresented classes like clouds. This derivation ensures that the loss function prioritizes overlap, offering a robust solution for improving detection in imbalanced datasets.

$$\mathrm{Dice} \mathrm{Loss}=1-{\displaystyle \frac{2\ast |GT\cap Pred|}{\left|\mathrm{G}\mathrm{T}\right|+\left|Pred\right|}}$$

(4)

• GT: ground truth from labeled image;
• Pred: predicted cloud detection image from model.

In addition, we determined the optimal settings for our deep learning model for cloud detection through extensive hyperparameter tuning, as shown in

Table 2. Hyperparameter range used to select optimized deep learning model structure for cloud detection.

Parameters	Configuration
Learning rate for Adam	0.0003
Learning rate factor for scheduling	0.5
Momentum	0.937
Weight decay	0.0005

. Our selection of hyperparameters was guided by prior studies on cloud detection in satellite imagery, particularly those utilizing AI algorithms for Kompsat-3/3A satellite images [34]. Building upon these existing techniques, we incorporated experimental approaches to further refine the model. We utilized an Adam optimizer with a learning rate of 0.0003, which was adjusted by a scheduling factor of 0.5 to enhance convergence. To maintain consistent updates, the model incorporated a momentum of 0.937, and a weight decay of 0.0005 was applied to prevent overfitting. These parameters were carefully selected to optimize the model’s performance in accurately detecting clouds.

2.5. The Performance Metrics of Cloud Detection

To evaluate the accuracy of cloud masks generated for KOMPSAT-3 and 3A imagery, this study applied a focused set of performance metrics to the test dataset, extending beyond basic accuracy to capture the nuanced aspects of segmentation quality. We calculated the selected metrics, F1-score, and Intersection over Union (IoU), using the true-positive (TP), true-negative (TN), false-positive (FP), and false-negative (FN) values obtained from the pixel-wise comparisons. In this context, TP represents correctly identified cloud pixels, FP indicates non-cloud pixels misclassified as clouds, TN denotes correctly identified non-cloud pixels, and FN signifies cloud pixels missed by the model.

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}={\displaystyle \frac{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}}}$$

(5)

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}}$$

(6)

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}}$$

(7)

$$\mathrm{F}1\mathrm{S}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}={\displaystyle \frac{2\ast \mathrm{p}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\ast \mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}+\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}}$$

(8)

$$\mathrm{IoU}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}}}$$

(9)

These metrics collectively provide a robust framework for assessing the model’s performance in cloud segmentation. Accuracy, as shown in Equation (5), offers a baseline measure of the overall correctness across all pixels, while precision and recall, defined in Equations (6) and (7), respectively, break down the model’s ability to correctly classify cloud pixels and avoid errors. The F1-score, derived from Equation (8), integrates precision and recall into a single metric, emphasizing the balance between detecting clouds and minimizing false positives—a critical consideration for applications like meteorological analysis where both over- and under-detection can impact outcomes. In contrast, the IoU, as presented in Equation (9), evaluates the spatial overlap between the predicted and actual cloud regions, providing a geometric perspective on segmentation accuracy that is particularly relevant for tasks requiring precise cloud boundary delineation, such as atmospheric correction in satellite imagery processing. To extend this evaluation across the entire test dataset, the mean IoU (mIoU) was introduced, calculated as the average of the individual IoU scores across all the images, as shown in Equation (10).

$$\mathrm{m}\mathrm{I}\mathrm{o}\mathrm{U}={\displaystyle \frac{1}{\mathrm{N}}}{\sum }_{\mathrm{i}=0}^{\mathrm{n}}{\mathrm{I}\mathrm{o}\mathrm{U}}_{\mathrm{i}}$$

(10)

The mIoU complements the IoU by offering a holistic measure of the model’s consistency and reliability, capturing its ability to maintain effective cloud detection across the diverse imaging conditions and scene complexities inherent in KOMPSAT-3 and 3A data. By incorporating the mIoU, this study ensures a comprehensive assessment that not only evaluates the localized segmentation quality but also verifies the model’s generalizability.

3. Results

3.1. Training and Validation Loss Analysis for Epochs

To evaluate the cloud segmentation model’s performance across different channel inputs, this study analyzed the training and validation loss per epoch for the various channel combinations derived from the KOMPSAT-3 and 3A imagery. Figure 5 illustrates these loss curves, with the left side depicting the training data and the right side representing the validation data.

For all the channel configurations, the training loss decreased steadily with increasing epochs, as depicted in Figure 6, which illustrates the training and validation loss per epoch for different channel combinations using KOMPSAT-3 and 3A data, with training data on the left and validation data on the right. Correspondingly, validation loss also declined consistently without rising, even at higher epochs, suggesting that the model avoids overfitting to the training data. However, certain configurations, particularly those incorporating multiple spectral bands, exhibited fluctuating validation loss curves. In the context of cloud segmentation, such variability may indicate instability in convergence, indicating that the model failed to consistently adapt to the data during training and exhibited erratic performance. As shown in Figure 7, the RGBNNDVI combination displayed the largest fluctuations and yielded the lowest detection accuracy.

3.2. Evaluation and Analysis of the Generalization Performance of Datasets

Figure 7 shows the accuracy and mIoU scores achieved by the model for various multispectral channel combinations using a test dataset excluded from the training process. As these data were neither included nor disclosed during training, they serve as a strong benchmark for assessing the model’s generalization and real-world applicability. The results indicate that simply altering the combination of multispectral channels can influence performance. Accuracy ranges from a minimum of 86.5% to a maximum of 87.5%, while the mIoU scores vary from 43.3% to 46.5%. Figure 7 further demonstrates that the TOA reflectance-based channel combinations consistently outperform those using DN values. Notably, incorporating the NDVI with TOA yields the highest accuracy and mIoU among all the tested configurations.

Table 3. Parameter count for each input channel configuration.

Channel	RGB	NGR	RGBN	RGBN+NDVI	TOA	TOA+NDVI
Parameters	45,670,484	45,670,484	45,673,620	45,676,756	45,673,620	45,676,756

summarizes the number of parameters for each channel combination. While the total parameter count varies slightly depending on the input data configuration, these differences are relatively small compared to the overall model size. These results indicate that adding the NDVI as an additional channel slightly increases the parameter count but does not significantly affect the overall model complexity.

3.3. Evaluation by Channel Configuration

As a result of the comprehensive analysis of the data presented in

Table 4. The performance metrics of the cloud masks for each of the channel combination cases of the KOMPSAT-3 and 3A test datasets whether TOA reflectance was applied or not.

Experiment	Class	Precision	Recall	F1-Score	mIoU
RGB	Clear Sky	90.75%	97.26%	93.89%	88.49%
	Thick Cloud	48.84%	95.76%	64.68%	47.80%
	Thin Cloud	68.35%	34.63%	45.97%	29.84%
	Cloud Shadow	56.18%	22.28%	31.91%	18.98%
	Overall	66.03%	62.48%	59.11%	46.28%
NGR	Clear Sky	89.75%	97.20%	93.33%	87.49%
	Thick Cloud	48.33%	90.75%	63.07%	46.06%
	Thin Cloud	64.88%	27.60%	38.73%	24.02%
	Cloud Shadow	62.92%	27.85%	38.61%	23.92%
	Overall	66.47%	60.85%	58.43%	45.37%
RGBN	Clear Sky	89.76%	97.86%	93.63%	88.02%
	Thick Cloud	49.92%	92.76%	64.91%	48.05%
	Thin Cloud	70.76%	26.79%	38.87%	24.12%
	Cloud Shadow	61.80%	27.24%	37.82%	23.32%
	Overall	68.06%	61.16%	58.80%	45.88%
RGBN+NDVI	Clear Sky	89.87%	96.84%	93.22%	87.31%
	Thick Cloud	44.19%	92.79%	59.87%	42.73%
	Thin Cloud	65.55%	27.43%	38.67%	23.97%
	Cloud Shadow	50.87%	23.76%	32.39%	19.33%
	Overall	62.62%	60.20%	56.04%	43.33%
TOA	Clear Sky	90.12%	97.91%	93.85%	88.42%
	Thick Cloud	48.27%	88.40%	62.44%	45.39%
	Thin Cloud	71.75%	29.99%	42.30%	26.82%
	Cloud Shadow	59.43%	24.68%	34.87%	21.12%
	Overall	67.39%	60.24%	58.37%	45.44%
TOA+NDVI	Clear Sky	90.19%	97.58%	93.74%	88.22%
	Thick Cloud	51.89%	86.88%	64.97%	48.12%
	Thin Cloud	69.18%	35.16%	46.62%	30.40%
	Cloud Shadow	66.06%	20.89%	31.75%	18.87%
	Overall	69.33%	60.13%	59.27%	46.40%

, six channel combinations RGB, NGR, RGBN, RGBN+NDVI, TOA, and TOA+NDVI were used to evaluate the performance of the cloud detection model. Performance was measured based on the precision, recall, F1-score, and mIoU and analyzed for four classes: clear sky, thick cloud, thin cloud, and cloud shadow, as well as for overall performance. The experimental results showed that for the clear sky class, all the channel combinations achieved F1-scores greater than 93%, with RGB (93.89%) and TOA (93.85%) demonstrating particularly strong performance. In the thick cloud class, TOA+NDVI achieved the highest F1-score at 64.97%, slightly outperforming RGBN (64.91%) and RGB (64.68%), while RGBN+NDVI (59.87%) showed relatively lower performance. In the thin cloud class, TOA+NDVI (46.62%) slightly outperformed RGB (45.97%) to achieve the best performance, while the other channels remained in the 38–42% range. In contrast, the cloud shadow class showed relatively high performance with NGR (38.61%) and RGBN (37.82%), while TOA+NDVI (31.75%) performed worse than TOA (34.87%) and RGBN+NDVI (32.39%). For overall performance, TOA+NDVI (59.27%) slightly outperformed RGB (59.11%) and showed superior results compared to the other combinations (56.04–58.80%). This indicates that TOA+NDVI provides a balanced performance across various classes.

To statistically verify the provided experimental results, an ANOVA test was performed on the F1-scores of each class. The results showed F-statistics of 1228.9363 for Clear Sky_F1, 595.9905 for Thick Cloud_F1, 2164.7038 for Thin Cloud_F1, and 380.6453 for Cloud Shadow_F1, with corresponding p-values of 0.0000. This indicates statistically significant differences in the mean F1-scores across the channel combinations, allowing the rejection of the null hypothesis (H₀: no difference in mean F1-scores between all channels). The analysis was conducted at a 95% confidence level. Additionally, as shown in

Table 5. The Tukey HSD post hoc analysis of F1-scores across channel combinations for each class. The Mean Difference (MD), defined as $\mathrm{MD}={\overline{x}}_{group1}-{\overline{x}}_{group2}$, quantifies the difference in F1-scores between the two channel pairs. A positive MD indicates that the first channel (group1) has a higher F1-score than the second (group2), while a negative MD suggests the opposite. The adjusted p-value (p-adj) indicates the statistical significance of the difference, with values below 0.05 marked in Bold Italic to denote significance.

group1	group2	Clear Sky		Thick Cloud		Thin Cloud		Cloud Shadow
		MD	p-adj	MD	p-adj	MD	p-adj	MD	p-adj
NGR	RGB	0.002	0	0.0507	0	0.0723	0	−0.0432	0
NGR	RGBN	0.008	0	−0.0024	0.0966	−0.015	0	−0.0011	0.9723
NGR	RGBNNDVI	0	0	0.018	0	−0.016	0	−0.0204	0
NGR	TOA	0.01	0	−0.0305	0	0.014	0	−0.022	0
NGR	TOANDVI	0.005	0	−0.0354	0	0.0741	0	−0.058	0
RGB	RGBN	0.007	0	−0.0531	0	−0.088	0	0.0422	0
RGB	RGBNNDVI	0	0	−0.0327	0	−0.089	0	0.0228	0
RGB	TOA	0.008	0	−0.0812	0	−0.058	0	0.0212	0
RGB	TOANDVI	0.004	0	−0.0861	0	0.0018	0.58	−0.0148	0
RGBN	RGBNNDVI	−0.01	0	0.0204	0	−0.001	0.9495	−0.0194	0
RGBN	TOA	0.001	0	−0.0281	0	0.0294	0	−0.0209	0
RGBN	TOANDVI	0	0	−0.033	0	0.0895	0	−0.057	0
RGBNNDVI	TOA	0.011	0	−0.0485	0	0.0304	0	−0.0016	0.858
RGBNNDVI	TOANDVI	0.007	0	−0.0534	0	0.0905	0	−0.0376	0
TOA	TOANDVI	0	0	−0.0049	0	0.0601	0	−0.036	0

, post hoc pairwise comparisons using Tukey HSD (honest significant difference) revealed that TOA+NDVI showed significant differences from other channels in several classes. For instance, in Clear Sky_F1, the average difference between NGR and TOA+NDVI was 0.0072 (p-adj = 0.0), which was statistically significant but with a minimal practical difference. In Thick Cloud_F1, TOA+NDVI showed average differences of 0.0294 and 0.0136 higher than NGR and RGB, respectively (p-adj = 0.0). In Thin Cloud_F1, TOA+NDVI showed significant differences from most channels except RGB, where the average difference was 0.0017 (p-adj = 0.7708), showing no significant difference. In Cloud Shadow_F1, TOA+NDVI showed average differences of −0.0517 and −0.0312 from NGR and TOA, respectively (p-adj = 0.0), reflecting relatively lower performance. These statistical analyses confirm that TOA+NDVI excels in the clear sky, thick cloud, and thin cloud classes but shows some limitations in the cloud shadow class.

When compared to the ground truth cloud labels (g) shown in Figure 8, the results in (c) reveal that all the models except the model trained with NGR data mistakenly classified a clear sky area as a light cloud in the lower-left portion of the satellite image. In contrast, (e) and (f), where TOA correction was applied, exhibited less over-detection of dense clouds compared to that of the other datasets. This indicates that the TOA correction contributed to improving cloud detection accuracy and reducing unnecessary false positives.

4. Conclusions

In this study, we evaluated the performance of cloud detection using a deep learning model, focusing on whether TOA reflectance was applied or not. Although our evaluation was relatively straightforward, the results demonstrated that incorporating TOA reflectance—commonly used as an input parameter in conventional cloud mask approaches—significantly enhanced the detection of cloud classes. The main contribution of this research lies in proposing a novel input data configuration that can improve model performance in cloud detection. Through the comprehensive analysis of various dataset combinations, we confirmed that the TOA+NDVI combination generally outperforms other configurations. This combination achieved particularly high accuracy and precision, effectively minimizing false detections. While its recall value was relatively low—resulting in more undetected cases—the highest recorded F1-score highlights that it strikes a strong balance overall. In contrast, the RGBN+NDVI combination exhibited limited performance, underscoring that some data configurations may be inherently constrained by their input characteristics. Class-by-class evaluations further supported these findings. The TOA+NDVI combination delivered superior results for both the dense and thin cloud classes, reflecting the physical properties of clouds more accurately, even when their boundaries were ambiguous. Meanwhile, the NGR combination performed well in detecting cloud shadows, indicating that different channel configurations may better capture distinct atmospheric and environmental nuances.

In conclusion, our study identifies the TOA+NDVI combination as a highly effective dataset configuration for cloud detection across a range of classes. Because the effectiveness of each combination depends on the underlying data characteristics and the selected variables, choosing the most suitable configuration for a specific class is essential. Generalizability can be enhanced through the application of diverse sensors. Future research should explore other model architectures, deep learning techniques, and variations in atmospheric conditions or temporal factors to further enhance cloud detection accuracy. These findings not only advance the field of cloud detection but can also be applied to other remote sensing domains. For instance, this approach could be valuable in disaster monitoring and response, enabling more efficient and error-free cloud detection, which makes it an important application. By using TOA or surface reflectance inputs, rather than relying solely on DN values, improved performance may be achieved in various Earth observation applications [29,30].