Optimizing Cloud Mask Accuracy over Snow-Covered Terrain with a Multistage Decision Tree Framework

Highlights

What are the main findings?

• Optimized a multi-level decision tree cloud detection algorithm for cloud-snow discrimination.
• Utilized the distinct bright temperature difference at 3.7 μm and 11 μm to enhance cloud detection performance.

What are the implications of the main findings?

• Outperforms existing algorithms and significantly reduces the false cloud detection in snow-covered areas.
• Provides accurate and efficient cloud detection in the Northern Hemisphere to support related cryospheric studies.

Abstract

High-resolution optical remote sensing imagery plays a crucial role in monitoring the Earth’s surface. However, cloud obstruction and spectral confusion between clouds and snow significantly compromise data quality and application reliability, leading to persistent cloud overestimation in optical remote sensing products. To address this challenge, this study developed an enhanced multi-threshold cloud detection algorithm based on AVHRR surface reflectance data, which incorporates dynamic threshold optimization within a multi-level decision tree framework. Utilizing Landsat 5 SR as reference data, the algorithm demonstrated superior cloud-snow discrimination capability, achieving an overall accuracy (OA) of 82.08%, with the user’s accuracy (UA) and F-score reaching 79.41% and 82.55%. Comparative evaluation demonstrates that the proposed algorithm outperforms two existing algorithms, with OA improvements of 17.42% and 7.93%, respectively. A particularly notable enhancement is the significant reduction in cloud misidentification, as reflected by UA increases of 21.02% and 13.21%. These improvements are most pronounced in high-altitude mountainous regions with snow cover. The algorithm maintains computational efficiency while providing reliable cloud masking, thereby offering enhanced support for snow cover monitoring and broader environmental applications.

1. Introduction

High-resolution optical remote sensing imagery has become an indispensable data source for monitoring dynamic changes on the Earth’s surface, widely applied in climate research, hydrological modeling, and environmental change analysis [1,2]. However, due to atmospheric conditions and sensor performance limitations, these images often suffer from data loss and quality deterioration caused by phenomena such as cloud cover, cloud shadows, and polar night [3,4]. This issue is particularly pronounced in regions with seasonally accumulated snow, where clouds and snow exhibit high spectral similarity in the visible to near-infrared bands, complicating the differentiation between the two [5]. Inaccurate cloud detection can lead to effective pixels being incorrectly obscured, reducing data availability, and may also cause overestimation of surface reflectance due to residual sub-pixel clouds or cloud shadows, thereby distorting time series analysis results and posing a significant threat to the accuracy of key geographical parameters, such as snow cover area [6].

To achieve effective cloud masking, various algorithms have been developed. Among them, threshold-based spectral methods distinguish clouds from the surface by setting reflectance thresholds. Although straightforward to implement, their accuracy is limited under complex atmospheric conditions, such as in mixed-pixel scenarios [7,8]. Other approaches, including the CMAPROB algorithm based on naive Bayesian theory [9] and optimal estimation (OE) techniques [10], offer more flexible identification strategies that can effectively handle different cloud types. However, these methods require higher data quality and computational resources. In recent years, intelligent processing techniques such as machine learning have improved recognition accuracy to some extent by training models to learn cloud features, while also providing long-term and comprehensive cloud type distribution information [11]. Nevertheless, class imbalance in datasets—where certain cloud types occur less frequently—can lead to model bias toward more common types. Additionally, inconsistencies among satellite products may affect the reliability of the results.

For large-scale surface parameter retrieval in the Northern Hemisphere, the long-term surface reflectance data acquired by the Advanced Very High-Resolution Radiometer (AVHRR) sensor are of irreplaceable value [12,13]. With over forty years of temporal coverage and near-global spatial availability, these data provide a critical foundation for monitoring land surface dynamics such as vegetation change, land cover classification, and snow/ice monitoring [14,15]. A cloud identification algorithm and threshold system based on AVHRR radiometric calibration data was proposed by Hori et al. [16] and has been widely applied in Northern Hemisphere snow product generation. Although this algorithm employed conventional cloud detection tests with enhanced strictness, it still inadequately addressed the spectral confusion between clouds and snow, resulting in both misclassification and omission errors. In response, Hao et al. [17] optimized the cloud masking thresholds within Hori’s framework using the National Oceanic and Atmospheric Administration (NOAA) Climate Data Record (CDR) of AVHRR Surface Reflectance Version 4 (AVHRR SR V4) over China, achieving an overall accuracy (OA) exceeding 84.76% and demonstrating the value of region-specific adaptive optimization. However, since the algorithm was developed and validated exclusively over China using AVHRR SR V4, its applicability to other geographical regions, diverse surface types, and subsequent CDR versions has not been verified, which substantially limits its utility for global-scale parameter retrieval.

The recently released NOAA CDR of AVHRR Surface Reflectance Version 5 (AVHRR SR V5) [18], jointly developed by NASA’s Goddard Space Flight Center (GSFC) and the University of Maryland (UMD), incorporates significant improvements in radiometric calibration and atmospheric correction. However, studies have indicated that the built-in cloud detection in this version exhibits biases, particularly a notable overestimation of cloud coverage. As shown in Figure 1 (taking 18 November 1988, as an example), obvious over-identification of clouds is observed in the AVHRR SR V5 cloud mask. Such biases can severely impact the quality of various land surface parameter retrieval products. The data exhibit pronounced band fluctuations before and after 2000, while the period from 1981 to 2000 demonstrates relatively minor fluctuations and greater stability. Given this early stability, alongside the availability of higher-quality satellite data from sensors such as MODIS, VIIRS, and Sentinel after 2000, and considering that AVHRR was among the earliest satellite systems to offer extensive spatial coverage with relatively high spatiotemporal resolution, there is an urgent need to develop a high-accuracy cloud detection algorithm adapted to the characteristics of AVHRR SR V5 with applicability in the Northern Hemisphere from 1981 to 2000, to enhance the reliability and consistency of long-term land surface parameter products.

This study proposes an improved cloud detection algorithm based on multi-source data fusion to address misidentification issues caused by cloud-snow spectral confusion in northern regions. Using Landsat high-resolution imagery as “truth” reference, combined with an improved SNOMAP algorithm [19] and cloud mask information from the QA_PIXEL quality band [20], a cloud-snow sample dataset integrating spectral features and topographic parameters was constructed. By analyzing the spectral response characteristics of clouds and snow from visible to shortwave infrared bands, a multi-level decision tree algorithm was established through threshold optimization strategies to accurately identify cloud pixels in the Northern Hemisphere from 1981 to 2000. Finally, the performance of the proposed algorithm was evaluated, and its uncertainties as well as future application prospects were analyzed.

2. Materials and Preprocessing

2.1. AVHRR Surface Reflectance CDR

The National Oceanic and Atmospheric Administration (NOAA) Climate Data Record (CDR) of Advanced Very High-resolution Radiometer (AVHRR) Surface Reflectance Version 5 (AVHRR SR V5) images serve as the foundational dataset for this study. Produced by the NASA Goddard Space Flight Center (Greenbelt, MD, USA) and the University of Maryland (College Park, MD, USA), this dataset is archived at the NOAA National Centers for Environmental Information (Keesler Air Force Base, MS, USA) and encompasses gridded daily surface reflectance, brightness temperatures, and quality control flags (QA) from 24 June 1981 to 31 December 2000. The data, derived from AVHRR sensors onboard eight NOAA polar-orbiting satellites, was generated using AVHRR GAC Level 1B data through geolocation, calibration, and atmospheric correction, offering a global coverage at a spatial resolution of 0.05° [18]. Compared to Version 4, Version 5 incorporates significant scientific enhancements, including higher resolution ancillary data and improved methodologies for bidirectional reflectance distribution function (BRDF) correction, calibration, compositing, and QA. Additionally, it rectifies known errors in temporal, latitudinal, and longitudinal variables while enhancing global and variable attribute definitions. This dataset provides essential model input, including surface reflectance data, brightness temperature data, angle data, and quality control information, with spectral band details outlined in

Table 1. AVHRR SR V5 product band information.

Band Name	Units	Scale	Wavelength	Description
SR1		0.0001	640 nm	Bidirectional surface reflectance
SR2		0.0001	860 nm	Bidirectional surface reflectance
SR3		0.0001	3.75 um	Bidirectional surface reflectance
BT3	K	0.1	3.75 um	Brightness temperature
BT4	K	0.1	11.0 um	Brightness temperature
BT5	K	0.1	12.0 um	Brightness temperature
RAA	deg	0.01		Relative sensor azimuth angle
SZA	deg	0.01		Solar zenith angle
VZA	deg	0.01		View zenith angle, scale 0.01
QA				Quality control bit flags

. The quality control flags are summarized in

Table 2. AVHRR SR V5 product QA bitmask information.

Bitmask Value	Description	0	1
Bit 0	Unused	No	Yes
Bit 1	Pixel is cloudy	No	Yes
Bit 2	Pixel contains cloud shadow	No	Yes
Bit 3	Pixel is over water	No	Yes
Bit 4	Pixel is over sunglint	No	Yes
Bit 5	Pixel is over dense dark vegetation	No	Yes
Bit 6	Pixel is at night (high solar zenith)	No	Yes
Bit 7	Channels 1–5 are valid	No	Yes
Bit 8	Channel 1 value is invalid	No	Yes
Bit 9	Channel 2 value is invalid	No	Yes
Bit 10	Channel 3 value is invalid	No	Yes
Bit 11	Channel 4 value is invalid	No	Yes
Bit 12	Channel 5 value is invalid	No	Yes
Bit 13	RHO3 value is invalid	No	Yes
Bit 14	BRDF correction is invalid	No	Yes
Bit 15	Polar flag, latitude over 60 degrees (land) or 50 degrees (ocean)	No	Yes

. The processing of all AVHRR SR V5 images was conducted using the Google Earth Engine (GEE) cloud platform (Google Inc., Mountain View, CA, USA) [21].

Cloudy pixels are identified by comparing the spectrally adjusted AVHRR data—adjusted by a factor of 0.97—to a MODIS monthly average red band (MODIS Channel 1) BRDF corrected climatology, computed using a decade of MODIS data. A pixel is classified as cloudy if the difference exceeds 0.03. Furthermore, cloud height is estimated based on temperatures derived from AVHRR Channels 4 and 5, while shadow pixels are determined by the projection of cloudy pixels onto the surface, retaining only the cloudy flag for those identified as both cloudy and shadowed [20].

2.2. Landsat 5 Reference Snow and Cloud Maps

The Landsat data utilized in this study were primarily obtained from the Landsat 5 Level 2, Collection 2, Tier 1 Surface Reflectance (SR) products distributed by the United States Geological Survey (Reston, VA, USA) [22]. This dataset is generated from the Thematic Mapper (TM) sensor onboard the Landsat 5 satellite and includes atmospherically corrected surface reflectance and land surface temperature data. The Landsat 5 satellite, the fifth satellite in the Landsat program, was launched from Vandenberg Air Force Base in California on 1 March 1984 and was decommissioned on 5 June 2013. The image data comprises four visible and near-infrared (VNIR) bands and two shortwave infrared (SWIR) bands, all processed to generate orthorectified surface reflectance, along with one thermal infrared (TIR) band processed for orthorectified surface temperature. Detailed band information is presented in

Table 3. Landsat 5 SR product band information.

Name	Units	Scale	Wavelength (μm)	Spatial Resolution	Description
SR_B1		2.75 × 10−5	0.45~0.52	30 m	Band 1 (blue) surface reflectance
SR_B2		2.75 × 10−5	0.52~0.60	30 m	Band 2 (green) surface reflectance
SR_B3		2.75 × 10−5	0.63~0.69	30 m	Band 3 (red) surface reflectance
SR_B4		2.75 × 10−5	0.77~0.90	30 m	Band 4 (near-infrared) surface reflectance
SR_B5		2.75 × 10−5	1.55~1.75	30 m	Band 5 (shortwave infrared 1) surface reflectance
ST_B6	K	3.418 × 10−3	10.40~12.50	120 m	Band 6 surface temperature.
SR_B7		2.75 × 10−5	2.08~2.35	30 m	Band 7 (shortwave infrared 2) surface reflectance
QA_PIXEL					Pixel quality attributes generated from the CFMASK algorithm

. The dataset also includes intermediate bands and quality control (QA) bands utilized for generating land surface temperature products, alongside pixel quality attributes derived from the CFMASK algorithm [23] for cloud information extraction, as outlined in

Table 4. Landsat 5 SR product QA bitmask information.

Bitmask Value	Description	0	1	2
Bit 0	Fill
Bit 1	Dilated Cloud
Bit 2	Unused
Bit 3	Cloud
Bit 4	Cloud Shadow
Bit 5	Snow
Bit 6	Clear	Cloud or Dilated Cloud bits are set	Cloud and Dilated Cloud bits are not set
Bit 7	Water
Bit 8–9	Cloud Confidence	None	Low	Medium
Bit 10–11	Cloud Shadow Confidence	None	Low	Medium
Bit 12–13	Snow/Ice Confidence	None	Low	Medium
Bit 14–15	Cirrus Confidence	None	Low	Medium

. The Landsat 5 Surface Reflectance (SR) products are generated using the Landsat Ecosystem Disturbance Adaptive Processing System (LEDAPS) algorithm (version 3.4.0), while the Collection 2 Surface Temperature (ST) products are derived from a single-channel algorithm jointly developed by the Rochester Institute of Technology (RIT, Rochester, NY, USA) and the Jet Propulsion Laboratory (JPL, Pasadena, CA, USA) of NASA. Data are organized through a standardized reference grid and packaged into overlapping “scenes” covering approximately 170 km × 183 km.

Landsat imagery is widely regarded as a “true” reference dataset for medium and low-resolution remote sensing data due to its rich spectral information, high spatial resolution, long time series, and free accessibility [24,25]. In this study, snow and cloud samples within the multi-level decision tree cloud detection algorithm reference the Landsat 5 imagery as ground truth.

To generate the reference snow cover data, an improved “SNOMAP” algorithm was employed. This method addresses the limitations of the traditional NDSI-based approach (NDSI ≥ 0.4 and NIR reflectance ≥ 0.11), which often underestimates snow cover in forested areas due to canopy obstruction. The improved algorithm incorporates multiple criteria, specifically NDFSI > 0.4 and NDVI < 0.6, to effectively identify snow pixels in forests where NDSI values fall below 0.4, resulting in a 30 m binary snow cover map [19].

Cloud reference data were obtained from the “QA_PIXEL” quality assessment band of Landsat 5 imagery, which provides reliable cloud masking information. To match the spatial resolution of AVHRR products, both the binary snow maps and cloud masks were aggregated to a 5 km resolution. Each 5 km pixel contains a value indicating either snow or no snow, and cloud or no cloud, serving as reference truth data for subsequent analysis.

Figure 2 illustrates the generation process of the reference snow data. Using Landsat 5 imagery (path/row 045024) from 30 April 1998, as an example, it displays the 30 m false-color composite, the binary snow cover map derived from the improved algorithm, and the aggregated 5 km resolution reference map. Similarly, Figure 3 presents the 30 m false-color image alongside its corresponding cloud mask derived from the “QA_PIXEL” band, as well as the aggregated 5 km cloud reference data. These aggregated datasets provide essential ground truth for the development and validation of cloud and snow discrimination models.

2.3. Ancillary Data

The GTOPO30 dataset is a global digital elevation model (DEM) developed by the U.S. Geological Survey (USGS, Reston, VA, USA), offering a horizontal resolution of 30 arc-seconds (approximately 1 km) and a vertical accuracy of ±30 m [26]. It provides essential elevation data as input to the cloud detection model. To align with the spatial resolution of AVHRR products, the GTOPO30 data were resampled to 5 km using bicubic interpolation.

The MOD44W Version 6 product provides a global land-water mask at 250 m resolution, generated through a decision-tree classification applied to MODIS data and validated against its predecessor, MOD44W V5. The classification incorporates multiple ancillary masks to mitigate errors caused by terrain shadows, burn scars, clouds, and oceanic ice cover. This mask was resampled to 5 km using nearest-neighbor interpolation to maintain consistency with the AVHRR grid [27].

3. Methods

A multi-layer decision tree-based framework was designed to enhance cloud masking accuracy over snow-covered terrain, which consists of four main steps (Figure 4).

(1) Selection of reference truth samples. The selection of reference truth samples was conducted on the Google Earth Engine (GEE) platform using Landsat 5 imagery to acquire high-confidence reference data for clouds and snow. Based on predefined criteria, a large set of reference truth labels was generated to provide a reliable truth reference for subsequent model training and validation.

(2) Build training dataset. Building on the existing AVHRR cloud discrimination algorithm framework, spectral and topographic parameters were derived from the reference truth data. Feature screening was then applied to yield a representative and quality-controlled training dataset as input parameters for the decision tree algorithm construction.

(3) Algorithm improvement based on threshold optimization. The cloud detection algorithm was enhanced through the application of threshold optimization strategies. Thresholds for ten cloud detection schemes were specifically improved, focusing on the key feature BT3-BT4 for distinguishing between cloud and snow. This led to the development of a high-precision cloud recognition algorithm capable of effectively differentiating cloud pixels from snow.

(4) Accuracy Assessment. Model performance was quantitatively evaluated using accuracy assessment metrics, and potential sources of uncertainty in cloud-related studies were analyzed. Optimization suggestions were provided for the algorithms and models developed in this chapter to enhance their reliability and stability under complex scenarios. The performance was further assessed by examining algorithmic uncertainties and exploring future application prospects.

3.1. Selection of Truth Reference Samples

High-resolution Landsat 5 SR product serves as a “true” reference for mid-resolution data in the classification of clouds and snow in this study. The specific methods for obtaining “true” maps for cloud and snow can be found in Section 2.2. To select the most representative cloud and snow sample images, the following screening criteria were applied: the period from 1 June to 1 August was designated as the training selection date for cloud samples, while 1 October to 1 May of the following year was allocated for snow sample selection. This choice is primarily due to the generally stable climatic conditions during the June to August period, which favor consistent cloud formation and distribution patterns. Additionally, the infrequent snow events during this timeframe help reduce confusion between clouds and snow, enhancing the representativeness and effectiveness of the samples. Conversely, the period from 1 October to 1 May is characterized by lower temperatures and frequent snowfall, creating favorable natural conditions for the formation and accumulation of snow.

Based on these criteria, a total of 17,851 cloud samples and 5992 snow samples were screened. During the sample selection process, relatively lenient standards were adopted to ensure diversity and quantity in the samples. This approach aims to improve the cloud detection algorithm through the use of extensive sample data. The advantage of this strategy lies in its ability to enhance the generalization capability of the algorithm by encompassing a wide range of climatic and environmental conditions, thereby ensuring more stable performance in practical applications. Furthermore, a diverse dataset can effectively improve the algorithm’s ability to distinguish between clouds and snow against complex backgrounds.

3.2. Build Training Dataset

To enhance the accuracy and reliability of cloud detection algorithms, a new algorithm was developed based on traditional combinations of existing methods [7,8,16,28,29]. During the construction process, surface reflectance (SR), brightness temperature (BT), Index and band combination data (I&B), and terrain data (TD) were utilized, comprising nine key variables: SR1, SR2, SR3, BT4, the brightness temperature difference between BT3 and BT4 (BT3-BT4), the brightness temperature difference between BT4 and BT5 (BT4-BT5), the reflectance difference between SR1 and SR2 (SR1-SR2), the Normalized Difference Vegetation Index (NDVI), and elevation (DEM), as summarized in

Table 5. The multi-level decision tree cloud detection algorithm input parameters.

Classification	Parameter
Surface reflectance data (SR)	SR1	SR2	SR3
Bright temperature data (BT)	BT4
Index and band combination data (I&B)	NDVI	SR1-SR2	BT3-BT4	BT4-BT5
Terrain data (TD)	DEM

.

The selection of these variables aims to comprehensively reflect the spectral and thermal characteristics of the cloud layer and the underlying surface. Surface reflectance data (SR1, SR2, SR3) provide spectral information across different bands, aiding in the identification of cloud types and their distribution [30]. The BT series indicators offer critical data support for the thermal characteristics of clouds, particularly the brightness temperature difference between BT3 and BT4, which serves as a primary testing metric to effectively distinguish clouds from other surface covers [31]. The reflectance difference (Ref1-Ref2) and brightness temperature difference (BT4-BT5) capture subtle variations between cloudy and cloud-free areas. Additionally, the NDVI and elevation (DEM) provide vegetation cover information and terrain characteristics, further enhancing the discriminative capacity of cloud-snow differentiation [32].

To enhance sample representativeness, invalid pixels were classified as gap pixels during the preprocessing stage and subsequently removed. These gap pixels included (1) pixels with poor quality control, (2) pixels with surface reflectance values outside the range [0, 1], (3) pixels exhibiting anomalous brightness temperatures, (4) water bodies, and (5) pixels with no data.

3.3. Algorithm Improvement Based on Threshold Optimization

3.3.1. Existing Algorithm Framework

Currently, the cloud detection algorithm based on AVHRR data was proposed by Hori et al. [16] and employs a multi-level threshold classification method to effectively identify cloud pixels. The core of this algorithm lies in the hierarchical classification of cloud pixels through stringent threshold settings, thereby enhancing the accuracy and reliability of the cloud detection process.

Initially, a preliminary classification is conducted based on specific thresholds of elevation DEM and the brightness temperature band BT4 at a wavelength of 11 μm: when the DEM value exceeds 300 m and the BT4 is below 260 K, the cloud pixels are categorized as the high or cold land (Target A); otherwise, they are classified as the plains or normal-temperature land (Target B).

Building on the preliminary classification, a secondary classification is performed using multiple variables, including SR1, SR2, SR3, BT4, BT4-BT5, SR1-SR2, and NDVI. In this process, clouds in high-altitude cold regions are further subdivided into four subclasses (A1, A2, A3, A4), while flat temperate regions are divided into six subclasses (B1, B2, B3, B4, B5, B6). Among these classifications, the brightness temperature difference between BT3 and BT4 (BT3-BT4) serves as a crucial third-level discriminative indicator, effectively distinguishing between different types of clouds. The specific framework of the algorithm is illustrated in Figure 5.

Despite the thresholds employed by this cloud detection algorithm being largely based on traditional cloud detection test combinations from previous studies, certain limitations persist. In particular, the setting of the BT3-BT4 threshold, while utilizing a more stringent cloud recognition algorithm, has not sufficiently addressed the confusion between clouds and snow, leading to potential misclassification and omission in practical applications. Therefore, optimization of the algorithm is urgently needed to enhance its accuracy and reliability under different environmental conditions.

In this context, the research team has built upon the aforementioned framework, utilizing AVHRR CDR SR V4 data to improve and evaluate the cloud detection algorithm thresholds specifically for the Chinese region [17]. Results demonstrate that the modified algorithm achieves an overall accuracy (OA) of 84.76% for cloud detection in the study area, further validating the effectiveness of the framework. However, it is important to note that the construction and application of this algorithm are primarily focused on the Chinese region and are based on AVHRR CDR SR V4, which somewhat limits its applicability in other geographical areas or with different versions of AVHRR CDR products.

3.3.2. Proposed Algorithm Improvement Strategy

Based on the AVHRR CDR SR V5 dataset and utilizing cloud and snow samples extracted from Landsat 5 SR maps as reference truth values, an improved threshold-optimized cloud detection algorithm was developed within the framework established by Hori et al. The study area was divided into two distinct regions: high-altitude mountainous areas (Target A) and temperate plains (Target B). Target A encompasses four cloud detection scenarios (A1–A4), while Target B includes six cloud detection scenarios (B1–B6) along with four clear-sky detection scenarios.

To address cloud-snow confusion over complex terrain, a third-level decision rule based on the BT3-BT4 threshold was incorporated into the original framework, specifically for cloud type A4 in high-altitude regions and type B6 in temperate plains. The multi-level decision tree cloud detection algorithm was implemented as follows: after data preprocessing, first- and second-level features were used to classify valid pixels into the ten cloud detection schemes (A1–A4 and B1–B6). Subsequently, frequency distributions of BT3-BT4 under different cloud categories were derived from cloud and snow training samples. Based on the distribution patterns of cloud and snow samples in the AVHRR SR V5 data, the overall accuracy (OA) was computed at intervals of 0.01, and the threshold corresponding to the maximum OA was selected as optimal.

Figure 4 illustrates the optimal determination scheme for the brightness temperature difference (BT3-BT4), using Target A1 as an example. The left panels show the frequency distributions of BT3-BT4BT3-BT4 for cloud and snow training samples from before the year 2000. The right panels display the variation in overall accuracy across different BT3-BT4 thresholds. For cloud type A1, the highest accuracy (84.81%) was achieved at the intersection of the frequency distributions of snow and cloud, corresponding to a BT3-BT4 threshold of 20 K. This value reflects a trade-off between the omission error (9.43%) and the commission error (19.08%). Thus, the final threshold was set to 20 K, whereby pixels with BT3-BT4 > 20 K are classified as cloud. Optimal thresholds for other cloud types were determined using the same methodology.

3.4. Accuracy Assessment Metrics

To comprehensively analyze the accuracy of classification results and the types of errors, this study employs a confusion matrix as a fundamental tool. The confusion matrix compares the predicted values with the actual values to provide a detailed display of the model’s misclassification across different categories. The specific form of the confusion matrix is shown in

Table 6. Confusion matrix.

Confusion Matrix	Prediction/Product
Reference		Positive	Negative
	Positive	TP	FN
	Negative	FP	TN

.

In this matrix, TP represents the number of samples that are actually positive and predicted as positive, while FN indicates the number of actual positives predicted as negatives; FP denotes the number of actual negatives predicted as positives, and TN represents the number of actual negatives predicted as negatives. In the multi-level decision tree cloud detection algorithm, the positive class refers to the identified clouds. Based on the confusion matrix, this study employs overall accuracy (OA), producer’s accuracy (PA), user’s accuracy (UA), and F-score (FS) metrics for accuracy assessment. OA indicates the proportion of correctly predicted values among all predictions, PA reflects the proportion of correctly predicted positives among actual positive cases, UA signifies the proportion of actual positives among predicted positives, and FS considers both UA and PA to evaluate the overall performance of the binary classification model. The specific calculation formulas are as follows:

$$\mathrm{OA}={\displaystyle \frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}}$$

(1)

$$\mathrm{PA}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}}$$

(2)

$$\mathrm{UA}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}}$$

(3)

$$\mathrm{FS}=2\times {\displaystyle \frac{\mathrm{UA}\times \mathrm{PA}}{\mathrm{UA}+\mathrm{PA}}}$$

(4)

4. Results

4.1. Improved Thresholds

Table 7. Improved multi-level decision tree cloud detection algorithm and thresholds.

Target	Target Serial Number	Switch	Elevation (m)	SR1	SR2	SR3	SR1-SR2	NDVI	BT4	BT3-BT4	BT4-BT5
A: high or cold land (DEM > 300 and BT4 < 260 K)	A1	On	<3000						≥240	>20
	A2	On	≥3000						≥240	>23.6
	A3	On							<240	>32.8
	A4	On				>0.1	>0.02			>31.4
B: plains or normal-temperature land (other land) (DEM < 300 and BT4 ≥ 260 K)	B1	On							<260	>21.4
	B2	On					>−0.02		<310	>16
	B3	On		>0.3			>−0.02		<293	>16
	B4	On			>0.4		>−0.03		<293	>16.8	>−1
	B5	On			>0.4				<278	>16.4	>−1
	B6	On		>0.3		>0.02				>16.4
	B7	Off						>0.5	>288
	B8	Off							>310
	B9	Off	>1000	<0.4			<−0.04		>275
	B10	Off					<−0.04		>300

¹ BT3-BT4 are the primary improved band thresholds.

presents the improved cloud detection scheme for the Northern Hemisphere based on AVHRR CDR SR V5 data, which includes 10 cloud detection schemes and 4 clear sky schemes. Target A represents the high or cold land, comprising four types: A1–A4. Target B indicates the plains or normal-temperature land (other land), including ten cloud detection schemes: B1–B10. For each target, cloud detection tests are conducted from the top of the list to the bottom. If the cloud flag switch is set to “on” pixels are classified as cloudy when the threshold tests meet the conditions listed on the right. If the switch is set to “off” pixels previously identified as cloudy in earlier tests are reset to clear.

4.2. Accuracy Assessment

4.2.1. Overview of the Algorithm Accuracies

The average OA of the algorithm is 82.08%, with a PA of 86.35%, a UA of 79.41%, and an FS of 82.55%. These results, indicate that while the enhanced cloud detection algorithm still experiences some misclassifications and omissions in distinguishing between clouds and snow, it achieves a relatively high accuracy overall. The algorithm maintains a good balance between precision and recall, as evidenced by the F-Score. This reflects the effectiveness of the threshold optimization strategy applied to the BT3-BT4 spectral features, significantly improving the distinguishability between cloud and snow pixels.

4.2.2. Algorithm Accuracies of Different Cloud Detection Schemes

Figure 6 presents the accuracy evaluation results for ten cloud detection schemes, revealing a significant difference in overall accuracy between the high or cold land (Target A) and the plains or normal-temperature land (Target B). The target A1 achieves an OA of 84.81% and a PA of 90.57%, indicating strong cloud recognition capabilities; however, the UA of 80.92% suggests some misclassification issues. The OA for target A2 decreases slightly to 79.44%, with an increase in both omission and commission errors, reflecting the complexity of cloud-snow confusion in high-altitude regions. The target A3 exhibits a significantly lower OA of 67.43%, demonstrating insufficient recognition ability, with high rates of both omission and commission errors, particularly the latter. Although the target A4 shows some improvement over A3, further enhancements are still needed.

In contrast, the cloud detection schemes in Target B generally perform better. The target B1 achieves an OA of 83.27%, demonstrating good performance in avoiding omissions, although the commission rate remains relatively high. The target B2 reports an OA of 80.50%, slightly lower than target B1, while the target B3 reaches an OA of 83.64%, showing a favorable balance between omission and commission errors. The OAs for Target B4 and Target B5 are 87.16% and 86.09%, respectively, both reflecting high cloud recognition capabilities. Notably, the target B6 achieves an OA of 90.93%, with minimal omission errors and relatively few commission errors, highlighting its superiority in cloud detection.

The accuracy of Target B is significantly better than that of Target A, which can primarily be attributed to environmental factors. Target A faces complex climatic conditions and variable topographic features. For instance, the severe climatic fluctuations in high-altitude cold regions dramatically influence the formation and dissipation of clouds, affected by factors such as temperature, humidity, and wind speed [33,34]. This complexity complicates the spectral characteristics of clouds and snow, increasing the likelihood of cloud-snow confusion [35]. Additionally, the intricate terrain, characterized by ridges and valleys, results in significant variations in radiative properties across different areas [36,37], affecting the applicability of cloud detection algorithms.

Conversely, the plain or normal-temperature land in Target B present a more stable environment. This area experiences relatively minor climatic fluctuations, resulting in a simplified process of cloud formation and evolution, enhancing the consistency of spectral features between clouds and snow [38]. Moreover, the flatter terrain in the plains results in minimal variation in surface reflectance, improving the comparability of features across different regions and thereby enhancing the effectiveness of classification algorithms.

4.3. Performance Comparison with Existing Algorithms

As clearly illustrated in Figure 1, the operational cloud mask embedded within the AVHRR SR V5 product exhibits substantial overestimation. Consequently, it was excluded from this comparative analysis, which focuses on two established cloud detection algorithms specifically designed for AVHRR.

4.3.1. Overview of the Algorithm Performance Comparison

A quantitative intercomparison of the three algorithms is presented in Figure 7. Evaluation based on the average accuracy across cloud detection schemes reveals a significant performance enhancement achieved by the newly proposed algorithm. Compared to the methods introduced by Hori et al. and Hao et al., the improved algorithm demonstrates an increase in OA of 17.42% and 7.93%, and a notable increase in UA of 21.02% and 13.21%, respectively. These results highlight the main advantage of the improved algorithm, which is its effectiveness in alleviating the common problem of cloud overestimation in historical AVHRR data. The substantial gain in UA indicates a marked reduction in commission errors, meaning a significant decrease in false detection where bright surfaces like snow are misclassified as cloud. This effectively alleviates the inherent limitation of the AVHRR SR V5 product shown in Figure 1. The observed moderate decrease in PA by 8.12% and 9.21% reflects a deliberate trade-off in the algorithm design. By adopting a more conservative strategy that prioritizes the correct identification of clear-sky pixels, even at the cost of a slight increase in cloud omission, the algorithm provides a more reliable foundation for generating high-quality surface reflectance products. The concurrent improvement in the FS by 8.61% and 3.05% confirms that the algorithm achieves a superior balance between commission and omission errors. This balance is crucial for ensuring the continuity and reliability of long-term land surface parameter retrieval from time-series data.

4.3.2. Algorithm Performance Comparison of Different Detection Schemes

Further analysis categorized by schemes indicates that the performance gain is particularly pronounced in the high or cold land (Target A). Specifically, compared to the Hori algorithm, the OA for classes A1, A2, and A3 increased by 31.60%, 16.97%, and 43.42%, respectively, with FS improvements ranging from 7.23% to 16.27%. Compared to the Hao algorithm, OA improvements ranged from 5.06% to 43.34%, with FS gains between 2.46% and 9.76%. Figure 8 presents the newly developed optimal threshold determination scheme for BT3-BT4, along with the threshold distributions from two existing analogous algorithms. This comparative visualization provides an intuitive demonstration of the classification performance across the three methodologies. This notable advancement stems from the algorithm’s efficacy in mitigating cloud-snow confusion, a frequent challenge in high-latitude and mountainous regions. Traditional threshold-based methods often struggle with high-reflectance surfaces like snow, leading to high commission errors. The improvement was achieved by refining thresholds for the brightness temperature difference between the 3.7 µm and 11 µm channels, a critical spectral feature for cloud-snow discrimination, through the integration of spectral and topographic information.

The plain or normal-temperature land (Target B), the algorithm also demonstrates consistent and widespread improvement, as shown in Figure 9. The OA increased within ranges of 1.09–35.83% and 2.01–9.86% against the Hori and Hao algorithms, respectively, while FS improvements ranged from 0.54 to 17.51% and 0.46 to 5.13%. The most significant improvement within Target B was observed for class B1, where OA increased by 35.83% and 9.86%, and FS increased by 17.51% and 5.13% compared to the Hori and Hao algorithms, further validating the algorithm’s superiority in snow-/ice-covered scenarios. Furthermore, the newly developed classifiers for classes A4 and B6, which address surface types not effectively handled by the Hori algorithm, show improved OA of 6.21% and 3.14%, respectively, compared to the Hao algorithm, indicating broader applicability and enhanced robustness.

4.3.3. Algorithm Performance Comparison in Typical Regions

Figure 10 presents a cloud mask schematic of the optimized algorithm on 18 November 1988, along with a visual comparison of cloud detection results across four representative regions, such as the Alps, the Qinghai Tibet Plateau (QTP), Northeast China (NCN), and the Rocky Mountains (Rockies). For each region, the figure shows a false-color composite of the original AVHRR imagery alongside cloud masks generated by the original AVHRR SR V5 product, the Hori algorithm, the Hao algorithm, and the newly proposed multistage decision tree framework. The comparative visualization clearly demonstrates that the improved algorithm significantly reduces cloud overestimation over bright snow-covered surfaces, which is a persistent issue in the original AVHRR SR V5 product and remains partially unresolved in the two existing algorithms. In snow-abundant high-altitude regions such as the Tibetan Plateau, the Alps, and the Rocky Mountains, as well as frequently snow-covered areas of Northeast China, the new algorithm remains capable of accurately distinguishing snow from clouds, thereby minimizing commission errors while maintaining robust cloud detection. This enhanced discrimination capability stems from the refined thresholding of the BT3-BT4 brightness temperature difference, combined with integrated spectral and topographic contextual analysis, which improve the distinction between cool, bright snow and cloud.

5. Discussion

5.1. Analysis of Algorithm Uncertainty

5.1.1. Uncertainty of Truth Reference

Landsat 5 imagery, used as the truth reference for distinguishing between clouds and snow, has inherent limitations that may introduce significant uncertainty, particularly in complex scenarios (such as thin clouds and mixed pixels), where the cloud mask may suffer from omissions or misclassifications [39]. The morphology and distribution of clouds change rapidly over time, and the temporal difference between Landsat 5 and AVHRR may lead to inconsistencies in cloud cover. Although the improved “SNOMAP” algorithm achieves an accuracy exceeding 90% in forested and non-forested areas, limitations remain in regions with low snow cover (such as heterogeneous surfaces with uneven snow), where these errors may further propagate into the reference snow. Overall, despite some uncertainties, the algorithm’s overall accuracy remains satisfactory. Additionally, the significant differences in spatial scale between the high-resolution data of Landsat 5 and the low-resolution data of AVHRR may introduce errors during the scale transformation process.

5.1.2. Spectral Characteristic Overlap

Clouds and snow exhibit similar spectral characteristics in visible and near-infrared bands, particularly in the presence of thin clouds or shallow snow, where spectral overlap becomes more pronounced [40]. This similarity may lead to confusion in the algorithm’s ability to distinguish between clouds and snow. Furthermore, variations in atmospheric conditions (such as aerosols and water vapor) may exacerbate spectral similarity, especially in AVHRR data due to its lower spectral resolution [41].

5.1.3. Limitations of the Algorithm

The multi-threshold method relies on preset thresholds, which may not adapt well to complex surface conditions and variable atmospheric environments, particularly in areas with overlapping spectral characteristics or high surface heterogeneity [42]. Nevertheless, the algorithm demonstrates high accuracy in many scenarios.

5.2. Analysis of Application Prospects

5.2.1. Application and Characteristics of the Algorithm

This study initially identified overestimation of cloud detection due to cloud-snow spectral confusion in the AVHRR SR V5 product during the development of long-term snow cover products specifically for the Northern Hemisphere. To address this issue, an improved cloud detection algorithm was optimized for accuracy over snow-covered terrain using a multistage decision tree framework.

The algorithm employs a multistage decision tree architecture, significantly enhancing the differentiation between clouds and snow through a dynamic threshold optimization strategy. Quantitative assessments indicate that the overall accuracy of the new algorithm improves by 17.42% compared to the Hori algorithm and by 7.93% compared to the Hao algorithm. This improvement is particularly pronounced in areas with extensive snow cover in the Northern Hemisphere, effectively addressing the problem of snow surfaces being misclassified as clouds in the original product. By reducing the misclassification rate of snow surfaces, the algorithm effectively enhances the usability of cloud detection. Moreover, precise initial cloud detection reduces the computational burden of subsequent ground object recovery under the clouds.

Compared to machine Learning or deep learning, the advantages of this algorithm lie in its foundation as a traditional method based on physical rules and threshold optimization, offering strong interpretability and high computational efficiency. The algorithm can directly generate binary cloud masks, supports cloud-based processing on platforms such as Google Earth Engine without downloading raw spectral data, and can also be deployed locally, providing significant advantages in terms of time and storage costs. However, compared to deep learning, this method is somewhat less capable of capturing the complex nonlinear characteristics of cloud and snow pixels. Additionally, it is more sensitive to fluctuations in spectral features and faces challenges in threshold adjustment and generalization when applied across different sensors.

5.2.2. Extended Applications of Products

The improved cloud mask product demonstrates significant value in snow monitoring across the Northern Hemisphere. Snow cover detection primarily relies on optical remote sensing satellites [43], which are greatly affected by clouds. To obtain comprehensive surface snow information, it is necessary to conduct land cover recovery on remote sensing imagery, essentially performing cloud removal [44].

As illustrated in Figure 10a, the cloud mask product generated by the algorithm accurately represents the spatial distribution characteristics of cloud layers, providing a reliable data foundation for snow cover extraction. Moreover, the enhanced cloud mask has broad application potential in long-term time series of optical remote sensing parameter retrieval. For instance, in studies related to the Normalized Difference Vegetation Index (NDVI) [45], Leaf Area Index (LAI) [46], soil moisture [47], crop growth assessment [48], and land surface temperature product generation [49] in the Northern Hemisphere, the accurate cloud mask effectively differentiates cloud and non-cloud areas, significantly reducing the interference of clouds on remote sensing data and enhancing the accuracy of parameter retrieval. This not only optimizes snow monitoring and the preparation of related products but also provides more reliable data support for vegetation dynamics monitoring, agricultural resource management, and ecological environment assessments.

6. Conclusions

An enhanced multi-threshold cloud detection algorithm was developed for the AVHRR surface reflectance data to address the significant overestimation issue present in existing AVHRR cloud mask products, particularly due to cloud-snow spectral confusion. The algorithm was constructed and evaluated using Landsat 5 imagery as truth reference maps, where its inherent cloud mask and snow cover were identified by an improved “SNOMAP” algorithm, which served as cloud and snow samples, respectively. The model’s accuracy, uncertainties, and future development prospects were assessed.

The improved algorithm enhances cloud detection by adjusting the brightness temperature difference threshold between bands BT3 and BT4. Experimental results show an OA of 82.08%, with a PA of 86.35%, UA of 79.41%, and FS of 82.55%. Significant differences in detection accuracy were observed between high or cold land (Target A) and plains or normal-temperature land (Target B). Target A experienced greater cloud-snow confusion, particularly in subclass A3, which had a low OA of 67.43%. In contrast, Target B achieved a much higher OA of 90.93% in subclass B6 due to its environmentally stable and flat terrain.

Compared to the existing two algorithms, the improved algorithm effectively mitigates cloud overestimation in the AVHRR SR V5 product, showing improvements in OA by 17.42% and 7.93% and in UA by 21.02% and 13.21%. It achieves a better balance between commission and omission errors, indicated by FS enhancements of 8.61% and 3.05%. Optimizing the brightness temperature difference threshold between the 3.7 µm and 11 µm channels significantly reduces cloud-snow confusion in Target A, with a 43.42% OA improvement for Target A3. Target B also shows consistent performance, particularly with a 35.83% OA improvement for Target B1. The newly developed schemes for Targets A4 and B6 further extend the algorithm’s applicability, providing reliable technical support for generating high-quality, long-term cloud-free data products from AVHRR.

In summary, the enhanced cloud detection algorithm effectively mitigated the misclassification caused by cloud-snow spectral confusion in AVHRR data, significantly improving cloud detection accuracy, particularly over snow-covered regions in the Northern Hemisphere. Through a multi-level decision tree architecture and dynamic threshold optimization, the algorithm maintained high accuracy while enabling efficient computation, offering a robust solution for large-scale remote sensing data processing. The improved cloud mask product not only enhances snow cover monitoring quality but also provides a superior data foundation for applications such as vegetation analysis and agricultural management, holding significant importance for global environmental monitoring.