A Machine-Learning-Based Study on All-Day Cloud Classification Using Himawari-8 Infrared Data

Abstract

Clouds are diverse and complex, making accurate cloud type identification vital in improving the accuracy of weather forecasting and the effectiveness of climate monitoring. However, current cloud classification research has largely focused on daytime data. The lack of visible light data at night presents challenges in characterizing nocturnal cloud attributes, leading to difficulties in achieving continuous all-day cloud classification results. This study proposed an all-day infrared cloud classification model (AInfraredCCM) based on XGBoost. Initially, the latitude/longitude, 10 infrared channels, and 5 brightness temperature differences of the Himawari-8 satellite were selected as input features. Then, 1,314,275 samples were collected from the Himawari-8 full-disk data and cloud classification was conducted using the CPR/CALIOP merged cloud type product as training data. The key cloud types included cirrus, deep convective, altostratus, altocumulus, nimbostratus, stratocumulus, stratus, and cumulus. The cloud classification model achieved an overall accuracy of 86.22%, along with precision, recall, and F1-score values of 0.88, 0.84, and 0.86, respectively. The practicality of this model was validated across all-day temporal, daytime/nighttime, and seasonal scenarios. The results showed that the AInfraredCCM consistently performed well across various time periods and seasons, confirming its temporal applicability. In conclusion, this study presents an all-day cloud classification approach to obtain comprehensive cloud information for continuous weather monitoring, ultimately enhancing weather prediction accuracy and climate monitoring.

1. Introduction

Cloudiness, a consequence of the presence of minuscule water vapor particles or ice crystals in the atmosphere, is closely intertwined with numerous climatic phenomena [1]. According to the International Satellite Cloud Climatology Project [2,3], the global annual average cloud cover covers two-thirds of the Earth’s surface. As the primary regulators of the Earth’s radiation balance, the water cycle, and biogeochemical cycles [4,5], distinct cloud types produce varying radiative effects. Therefore, the precise categorization of cloud types and understanding of their distribution patterns hold significant practical importance [6,7]. The pursuit of effective and precise cloud classifications remains a prominent focus in meteorological research [8].

In past decades, cloud classification has predominantly been carried out using traditional methods and machine-learning-based techniques. Traditional methods include thresholds and statistical mathematical methods. However, these methods rely heavily on empirical knowledge when dealing with data, resulting in low classification efficiency. With continuous developments in computer science, several machine learning methods have emerged. Machine learning methods do not require extensive prior knowledge and offer high computational efficiency and excellent classification performance [9]. Therefore, conventional machine learning methods such as Random Forest (RF) and Support Vector Machine (SVM) approaches have been widely used in cloud classification research [10]. Wohlfarth et al. [11] used data from three visible channels and one infrared (IR) channel obtained from the Landsat-8 satellite. They classified clouds into nine different cloud classes and four subsurface classes using the SVM algorithm, achieving an impressive classification accuracy of up to 95.4%. Yu et al. [12] proposed a cloud classification method based on an RF algorithm for FY-4A. This method was combined with CloudSat’s 2B-CLDCLASS cloud product to classify 8 single-layer cloud types and 12 multilayer cloud types. As the volume of data continues to increase, deep learning models such as deep neural networks (DNN) and artificial neural networks (ANN) are increasingly being used in cloud classification research. Cai et al. [13] employed FY-2C’s infrared channel 1 (10.3–11.5 μm) data in combined with a Convolutional Neural Network (CNN) model for cloud classification. Their classification divided clouds into 5 types, achieving an impressive average recognition rate of 90.6%. In addition, Gorooh et al. [14] proposed the Deep Neural Network Cloud-Type Classification (DeepCTC) model that can classify clouds into 8 different types with an overall classification accuracy of 85%. Machine learning techniques have surpassed traditional cloud classification methods in terms of both classification speed and accuracy. However, it is important to note that these methods are generally applicable to daytime cloud classification. Nighttime visible light data are often noisy and cannot be used for nighttime cloud classification. Therefore, one of the research questions addressed in this study is how to achieve nighttime cloud classification.

Researchers have commenced extensive studies using infrared data to more accurately monitor cloud evolution at night, eliminating the need for visible data and achieving successful research outcomes [15]. Tan et al. [16] proposed a nighttime cloud classification algorithm based on Himawari-8 satellite channel data and machine learning algorithms. They utilized data from 5 infrared channels, 3 brightness temperature difference (BTD) datasets, and latitude/longitude information as training features. By employing the RF algorithm, this approach achieved an overall accuracy of 0.79, classifying clouds into clear-sky, single-layer, and multilayer clouds. Li et al. [17] used infrared data from the Himawari-8 satellite to classify clouds. Their classification scheme included five types: clear, single-ice clouds, single-mixed clouds, single-water clouds, and multilayer clouds. They achieved an impressive overall classification accuracy of 0.81. Despite significant progress in nighttime cloud classification research, a critical analysis of existing research results revealed a limited number of nighttime cloud classifications. Typically, cloud classification is constrained to primary cloud categories without considering the finer distinctions between clouds.

This study addresses the challenges of nighttime cloud classification and the limited number of classifiers using the latitude/longitude, five brightness temperature differences (BTD), and ten IR channels from the Himawari-8 satellite. The combined Cloud-Profiling Radar (CPR) and Cloud-Aerosol LIDAR with Orthogonal Polarization (CALIOP) cloud-type product 2B-CLDCLASS-LIDAR were used as labels, which classified clouds into nine types: clear-sky (Clear), cirrus (Ci), deep convective (Dc), altostratus (As), altocumulus (Ac), nimbostratus (Ns), stratocumulus (Sc), stratus (St), and cumulus (Cu). By comparing various models using various metrics, such as overall accuracy, precision, recall, and F1-scores, this study investigated the viability of all-day cloud classification and the selection of optimal parameters. The goal was to improve both the quantity and accuracy of cloud classification, thus providing valuable additional reference data for nighttime meteorological observations.

Section 2 introduces the experimental data and methodology used in this study. Section 3 presents the experimental results and their implications. Finally, Section 4 and Section 5 present the discussion and conclusion, respectively.

2. Materials and Methods

2.1. Data Collection

This study used Level-1 infrared channel data from the Himawari-8 satellite and the cloud type product 2B-CLDCLASS-LIDAR, which is a joint project of CloudSat and CALIPSO. Additionally, longitude, latitude, and solar zenith angle data were employed for support.

(1)

Himawari-8 data

The Japan Meteorological Agency’s Geostationary Meteorological Satellite, Himawari-8, was launched on 7 July 2015. The data from this satellite can be accessed through the Japan Aerospace Exploration Agency (JAXA) (http://www.eorc.jaxa.jp/ptree/index.html, accessed on 15 May 2023) [18]. Compared with MTSAT-2 (Himawari-7), Himawari-8 is equipped with an Advanced Himawari Imager (AHI) sensor that has expanded from the original 5 bands to 16 bands, including 3 visible light bands, 3 near-infrared bands, and 10 infrared bands [19,20]. It covers the area from 60°S to 60°N and from 80°E to 200°E (Figure 1). The observation frequency of Himawari-8 has been increased to once every 10 min, providing abundant data for meteorological studies and continuous cloud observations [21]. This study used 10 infrared bands from the Himawari-8 satellite.

Table 1. Himawari-8 band parameters and applications.

Bands	Channel Type	Center Wavelength (μm)	Spatial Resolution (km)	Main Applications
7	Midwave IR	3.9	2	Natural disasters, low cloud (fog) observation
8	Water vapor	6.2	2	Observation of water vapor volume in the upper and middle layers
9		6.9	2	Observations of water vaporization in the mesosphere
10		7.3	2
11	Longwave IR	8.6	2	Cloud phase identification and SO2 detection
12		9.6	2	Measurement of total ozone
13		10.4	2	Observation of cloud images and cloud top conditions
14		11.2	2	Observation of cloud images and sea surface water temperature
15		12.4	2	Observation of cloud images and sea surface water temperature
16		13.3	2	Measurement of cloud height

displays the parameters of the 10 infrared bands of the Level-1 products of Himawari-8 and their main applications.

The Level-2 (L2) cloud product of the Himawari-8 satellite provides a comprehensive set of cloud-related parameters, such as cloud type (CLTYPE), cloud top height, cloud top temperature, and cloud optical thickness. However, it is important to note that this product solely provides cloud information for observations made during the daytime period. The spatial resolution is 5 km and the temporal resolution is 10 min. A more detailed description of AHI Level-1/2 data is available at https://www.eorc.jaxa.jp/ptree/userguide.html, accessed on 20 June 2023.

This study used 10 infrared bands as basic data, ranging from band 7 to band 16. In addition, auxiliary data, including longitude, latitude, sun zenith angle, and CLTYPE, were incorporated. Figure 2 displays the cloud images of Himawari-8 satellite for its 10 infrared channels at UTC 03:20 on 1 July 2019.

(2)

CloudSat data

CloudSat and CALIPSO are part of the A-train that crosses the equator in the afternoon [22]. Both satellites provide near global views of clouds from sun-synchronous orbits and are available at https://www.cloudsat.cira.colostate.edu, accessed on 23 May 2023. CloudSat’s onboard CPR and CALIOP (a vision and near-infrared LIDAR) are 2 powerful onboard active instruments [23]. Currently, these are the only instruments capable of accurately detecting the vertical structure of clouds [24]. Owing to the different working wavelengths of CPR and CALIOP, they exhibit different sensitivities to different types of clouds. The millimeter-wave radar CPR has the advantage of detecting optically thick clouds and precipitation systems [25]. Combining their advantages, a joint product of CPR/CALIOP was developed to provide the most reliable vertical cloud information; this was named 2B-CLDCLASS-LIDAR. This product provides information on up to 10 cloud layers. These layers are characterized by a horizontal resolution of 1.4 × 1.8 km and a vertical resolution of 0.24 km. This dataset contains valuable details, including cloud layers, cloud heights, and cloud phases [26]. The CloudLayerType band elements range from 0 to 8 and represent different cloud types: Clear, Ci, As, Ac, St, Sc, Cu, Ns, and Dc. In this study, CloudLayerType band data were utilized as the labeling criteria. For a more comprehensive understanding of 2B-CLDCLASS-LIDAR, please refer to the CloudSat product brochure [27].

(3)

Cloud type of this study

In this study, 130 days of data (

Table A1. The statistics of AInfraredCCM classification results for the four seasons and corresponding CLTYPE of Himawari-8.

Season	No.	Data ID	No.	Data ID
Spring	1	20190301_0400	17	20190317_0400
	2	20190302_0400	18	20190318_0400
	3	20190303_0400	19	20190319_0400
	4	20190304_0400	20	20190320_0400
	5	20190305_0400	21	20190321_0400
	6	20190306_0400	22	20190322_0400
	7	20190307_0400	23	20190323_0400
	8	20190308_0400	24	20190324_0400
	9	20190309_0400	25	20190325_0400
	10	20190310_0400	26	20190326_0400
	11	20190311_0400	27	20190327_0400
	12	20190312_0400	28	20190328_0400
	13	20190313_0400	29	20190329_0400
	14	20190314_0400	30	20190330_0400
	15	20190315_0400	31	20190331_0400
	16	20190316_0400
Summer	32	20190601_0400	51	20190620_0400
	33	20190602_0400	52	20190622_0400
	34	20190603_0400	53	20190623_0400
	35	20190604_0400	54	20190624_0400
	36	20190605_0400	55	20190625_0400
	37	20190606_0400	56	20190626_0400
	38	20190607_0400	57	20190627_0400
	39	20190608_0400	58	20190628_0400
	40	20190609_0400	59	20190630_0400
	41	20190610_0400	60	20190701_0400
	42	20190611_0400	61	20190702_0400
	43	20190612_0400	62	20190703_0400
	44	20190613_0400	63	20190704_0400
	45	20190614_0400	64	20190705_0400
	46	20190615_0400	65	20190706_0400
	47	20190616_0400	66	20190707_0400
	48	20190617_0400	67	20190708_0400
	49	20190618_0400	68	20190709_0400
	50	20190619_0400	69	20190710_0400
Autumn	70	20181101_0400	85	20181116_0400
	71	20181102_0400	86	20181117_0400
	72	20181103_0400	87	20181118_0400
	73	20181104_0400	88	20181119_0400
	74	20181105_0400	89	20181120_0400
	75	20181106_0400	90	20181121_0400
	76	20181107_0400	91	20181122_0400
	77	20181108_0400	92	20181123_0400
	78	20181109_0400	93	20181124_0400
	79	20181110_0400	94	20181125_0400
	80	20181111_0400	95	20181126_0400
	81	20181112_0400	96	20181127_0400
	82	20181113_0400	97	20181128_0400
	83	20181114_0400	98	20181129_0400
	84	20181115_0400	99	20181130_0400
Winter	100	20190601_0400	116	20190617_0400
	101	20190602_0400	117	20190618_0400
	102	20190603_0400	118	20190619_0400
	103	20190604_0400	119	20190620_0400
	104	20190605_0400	120	20190621_0400
	105	20190606_0400	121	20190622_0400
	106	20190607_0400	122	20190623_0400
	107	20190608_0400	123	20190624_0400
	108	20190609_0400	124	20190625_0400
	109	20190610_0400	125	20190626_0400
	110	20190611_0400	126	20190627_0400
	111	20190612_0400	127	20190628_0400
	112	20190613_0400	128	20190629_0400
	113	20190614_0400	129	20190630_0400
	114	20190615_0400	130	20190631_0400
	115	20190616_0400

) were selected from November 2018, January 2019, March 2019, June 2019, and July 2019. The Himawari-8 CLTYPE products encompass 10 types (0–9 represent Clear, Ci, Cs, Dc, Ac, As, Ns, Cu, Sc, and St, respectively) and CPR/CALIOP products consist of only 9 types. To ensure consistency, the clouds were finally classified into 9 types based on the one-to-one mapping between the CPR/CALIOP and Himawar-8 clouds. Notably, in Himawari-8, Ci and Cs were merged into a one type called Ci clouds. The classification criteria are listed in

Table 2. Cloud type of this study.

Cloud Label	Label of CPR/CALIOP	Label of CLTYPE	Name of Cloud
0	0 (Clear)	0 (Clear)	Clear
1	1 (Ci)	1, 2 (Ci, Cs)	Ci (Ci, Cs)
2	8 (Dc)	3 (Dc)	Dc
3	3 (Ac)	4 (Ac)	Ac
4	2 (As)	5 (As)	As
5	7 (Ns)	6 (Ns)	Ns
6	6 (Cu)	7 (Cu)	Cu
7	5 (Sc)	8 (Sc)	Sc
8	4 (St)	9 (St)	St

.

2.2. Method

In this study, 10 different IR bands from Himawari-8 and 5 BTDs were used to select potential donor pixels. Longitude and latitude were used as additional constraints. In the following section, the proposed algorithm is referred to as the All-day Infrared Cloud Classification Model (AInfraredCCM). Figure 3 shows a conceptual diagram of the method used in this study and its comparison.

(1) Data collection and processing: This study utilized various sources, such as CloudLayerType, Himawari-8 Level-1 IR, latitude and longitude, and solar zenith angle data from the 2B-CLDCLASS-LIDAR product [26]. To obtain BTD information, pairwise differences were calculated between all the infrared channels. Subsequently, feature selection was performed using the feature analysis function within the machine learning module [28]. The final input feature set included 5 BTDs, 10 infrared channels, and latitude and longitude data. BTD (11.2–7.3 μm) can be employed to detect high- and mid-level clouds over land during the night. BTD (3.9–11.2) is known as useful BTD to detect low-level clouds. BTD (11.2–12.4 μm) is useful for distinguishing thin clouds from clear sky. BTD (12.4–10.4 μm) can serve as a substitute for visible light to describe cloud optical thickness. BTD (7.3–10.4 μm) is utilized for nighttime cloud detection by differencing the water vapor channel and the infrared channel [10,29]. After obtaining the BTD data, a series of data preprocessing steps were applied to both the Himawari-8 and CPR/CALIOP datasets. These steps include spatiotemporal matching, label extraction, data resampling, and cropping. Consequently, a training dataset consisting of 1,182,688 samples and a testing dataset consisting of 131,587 samples were created following a random 9:1 split ratio. The specific features contained in the dataset are listed in

Table 3. Information of dataset.

Dimension	Number	Variables
Predictor	BTs (10)	BT (3.9 μm), BT (6.2 μm), BT (6.9 μm), BT (7.3 μm), BT (8.6 μm), BT (9.6 μm), BT (10.4 μm), BT (11.2 μm), BT (12.4 μm), and BT (13.3 μm)
	BTDs (5)	BTD (11.2–7.3 μm), BTD (3.9–11.2 μm), BTD (11.2–12.4 μm), BTD (12.4–10.4 μm), and BTD (7.3–10.4 μm)
	Auxiliary data (2)	Latitude and Longitude
Prediction	1	Cloud label from 2B-CLDCLASS-LIDAR and CLTYPE

.

(2) Label extraction: To obtain cloud type samples for training, Himawari-8 and 2B-CLDCLASS-LIDAR products were subjected to temporal and spatial matching. Each pixel in the CloudLayerType dataset contained information on 10 cloud types, and each Himawari-8 pixel corresponded to a cloud type. During data matching, the first step was to compress the 10-layer data. If the CPR point corresponded to multilayer clouds with different cloud types in each layer, it was defined as a multilayer cloud; otherwise, it corresponded to the cloud type in the first layer. The second step involved a temporal and spatial matching process due to the difference in the spatial and temporal resolutions between AHI and CPR. This entailed selecting AHI data points within a 5 min time window of the CPR scan point and within a 5 km radius of the CPR point location [30,31]. In the third step, considering that the AHI observations did not include multilayer cloud types, the CPR points corresponding to multilayer clouds were eliminated. Finally, cloud type labels were assigned by majority rule based on the matching of AHI and CPR points [32,33].

(3) Model building: The processed dataset was used to train the XGBoost model, and the optimal model parameters were determined by Bayesian optimization. This step selection was made by sampling using the Bayesian optimization algorithm after estimating the distribution of the objective function in the parameter space through a Gaussian process model. It evaluated the overall classification accuracy of related models while searching for the best parameter combination across a finite number of iterations. Finally, the parameter combination with the highest overall classification accuracy was obtained and represented as the optimal model parameters.

(4) Model evaluation: The overall accuracy (OA), precision, recall, and F1-score were calculated based on ten-fold cross-validation [34]. If the final model achieved sufficient accuracy, it was selected; otherwise, the parameters were modified.

This study used the parameters shown in

Table 4. Classification results of the model on dataset C.

		Number of Ever Category		Total Number
		Number of category A clouds	Number of category B clouds
Model classification result	Number of category A clouds	TA	FB	T1
	Number of category B clouds	FA	TB	T2
Total number		AS	BS	T

as an example to better explain the metrics used for accuracy assessment. Assume that a dataset C consists of T samples; here, AS is the number of samples for class A clouds and BS is the number of samples for class B clouds. Table 4 shows the specific classification results from dataset C, which were used for model classification prediction.

The overall accuracy (OA) is a crucial measure, representing the proportion of correctly classified samples to the total number of samples [12]. A higher overall accuracy indicates a more reliable classification outcome. The overall accuracy is calculated as follows:

$$OA=\frac{TA+TB}{TA+FB+FA+TB}$$

(1)

where TB represents an accurately classified cloud type B.

Precision is a metric that quantifies the proportion of correctly identified clouds in the target class, indicating fewer classification errors. Precision can be expressed as follows:

$$Precision=\frac{TA}{TA+FA}$$

(2)

where TA denotes correctly identified class A clouds and FA denotes the misdiagnosis of class A clouds as class B clouds.

The recall pertains to the proportion of correctly identified class A clouds within the total number of identified class A clouds [35]. Recall can be represented as follows:

$$Recall=\frac{TA}{TA+FB}$$

(3)

where TA denotes the correct identification of class A clouds and FB denotes the misdiagnosis of class B clouds as class A clouds.

The F1-score combines precision and recall, with values ranging from 0 to 1. A higher score indicated that the model was more accurate. Compared with that using only precision or recall metrics, this score provides a more comprehensive evaluation of model accuracy [36]. The F1-score can be written as follows:

$$F1=\frac{2PrecisionRecall}{Precision+Recall}$$

(4)

(5) The AInfraredCCM was conducted through all-day cloud classification for the entire research area using the test data.

XGBoost is an ensemble learning method that employs boosting principles. Unlike traditional boosting methods, XGBoost uses tree models (decision trees) as weak learners. It combines multiple weak learners to create a strong learner, gradually improving the model’s performance. XGBoost is adaptive; in each iteration, it introduces a new weak classifier that attempts to correct the errors of the previous round until the stopping condition is satisfied. The key characteristic of XGBoost is the integration of multiple weak learners into a strong learner, gradually improving the model accuracy through iterative error correction [37].

Bayesian optimization approach was employed to tune the model and identify the optimal parameters of the AInfraredCCM model. The n_estimators parameter was set within a range from 100 to 300, the learning_rate parameter was set within a range from 0.01 to 0.999, the max_depth parameter was set within a range from 10 to 100, and the min_child_weight parameter ranges from 1 to 10 during optimization. After 100 iterations, the AInfraredCCM was developed, and the model parameters are displayed in

Table 5. Parameters of the AInfraredCCM.

Parameter	Meaning	Value
n_estimators	Number of trees	204
learning_rate	Magnitude of the iterative model update	0.2122
max_depth	Maximum tree depth	26
min_child_weight	Minimum number of samples required in a leaf node	3

.

3. Results

3.1. Comparison with Other Methods

The results are based on an independent test dataset of approximately 130,000 pixels, which is divided into clear sky and eight different cloud forms.

Table 6. Precision, recall, and F1-score of the AInfraredCCM.

Cloud Type	Precision	Recall	F1-Score
Clear	0.85	0.89	0.87
Ci	0.90	0.88	0.89
Dc	0.93	0.87	0.90
Ac	0.82	0.74	0.78
As	0.89	0.88	0.89
Ns	0.95	0.93	0.94
Cu	0.68	0.57	0.60
Sc	0.88	0.93	0.91
St	0.98	0.90	0.94

shows the precision, recall, and F1-score of the AInfraredCCM for each form of cloud in the training dataset. The confusion matrix for various types of clouds is shown in Figure 4, wherein the vertical coordinate represents the true value and the horizontal coordinate represents the predicted value.

Based on the model’s classification results for the all-day data, the average overall accuracy for all cloud types was 86.22%, with a precision of 0.88, a recall of 0.84, and an F1-score of 0.86. Notably, Cu clouds exhibited considerably lower identification rates than the other cloud types, and both recall and F1-score are significantly lower than those of other cloud types (Table 6). The confusion matrix shown in Figure 4 graphically displays that most of Cu is misclassified as clear sky (34.252%) and Sc (8.962%). These phenomena can be attributed to the unique properties of Cu, which include two possibilities: one resembles Cu in clear weather and appears as thin, fragmented clouds that closely resemble clear in terms of reflectance, making them easy to misidentify; the alternative scenario is continuous Cu, which is denser and more vertically developed than clear-sky Cu and is frequently categorized as Sc.

To enhance the credibility of the model, the cross-validation results were compared with those obtained using various other models, including RF, LightGBM, AdaBoost, and GradientBoost. RF is a conventional machine learning model known for its versatility in handling both classification and regression tasks. LightGBM has been optimized for large datasets, offering high processing speed and low memory usage. This makes it particularly suitable for scenarios involving substantial amounts of data and high dimensional features [38]. Gradient Boosting is a boosting-based integrated learning technique known for its robustness to outliers and good performance in a variety of classification and regression problems. Similarly, AdaBoost is an integrated learning method that employs weighted voting to combine several weak classifiers. The weights are updated after each iteration based on the outcome of the previous round, thereby reducing overfitting and improving the management of sample imbalances. The best parameter combinations for each model were obtained after running several Bayesian optimizations.

Table 7. Optimal parameter combinations for the model.

Algorithm	Parameted Range
Random Forest	1. max_depth = 73 2. n_estimators = 280
LightGBM	1. learning_rate = 0.095 2. max_depth = 22 3. n_estimators = 252 4. num_leaves = 35
AdaBoost	1. learning_rate = 0.4224 2. max_depth = 74 3. n_estimators = 458 4. min_samples_leaf = 1
GradientBoost	1. learning_rate = 0.4749 2. max_depth = 37 3. n_estimators = 10

provides more information about the optimal parameter combinations for the four models.

All models were trained using the selected parameters and tested for the classification of the training data. As demonstrated in

Table 8. Evaluation for different models.

Algorithm	Accuracy	Precision	Recall	F1-Score
Random Forest	82.53%	0.83	0.76	0.79
LightGBM	74.60%	0.70	0.64	0.66
GradientBoost	80.96%	0.78	0.77	0.78
AdaBoost	85.83%	0.87	0.83	0.85
AInfraredCCM	86.22%	0.88	0.84	0.86

and Figure 5, the AInfraredCCM achieved the highest overall accuracy of 86.22% for the test datasets. Following was the AdaBoost classification model, which attained an accuracy of 85.83%. However, the AInfraredCCM outperformed AdaBoost across all three criteria, i.e., precision, recall, and the F1-score. The four indices of overall accuracy outside precision, recall, and F1-score were highest for the AInfraredCCM in the full analysis (OA—86.22%; precision—0.88; recall—0.84; F1-score—0.86). Table 8 lists the categorization metrics of the test datasets for the models.

Figure 6 illustrates the classification results of different models on 29 June 2019. A comparison between the images in the figure reveals that Gradient Boosting yields better classification results but with notable inconsistencies. The classification results of LightGBM, AdaBoost, and RF contained a relatively large number of errors and were ineffective for accurately classifying the Ci clouds. While all these models possess certain advantages, they failed to fully harness the dataset’s features for precise cloud classification. In contrast, the classification results derived from the AInfraredCCM exhibited relatively smooth boundaries and accurate classification results.

Table 9. Cloud classification model statistical table.

Model	Category	Feature	Time	OA	Sample	Reference
RF	Dc, Ns, Cu, Sc, St, Ac, As, Ci, and multi	REF and BT of 13 channels, cloud top height, cloud optical thickness, cloud effective radius	Day	0.67	272414	Yu et al. (2021) [12]
BP	Clear, low cloud, middle cloud, thick cirrus clouds, thin cirrus cloud, deep convective	IR1 (10.3–11.3), IR2 (11.5–12.5), WV (6.3–7.6), IR1-IR2, IR1-WV, IR2-WV	Day	0.86	2449	Zhang et al. (2012) [39]
CNN	Clear, Ci, Ac, As, Sc, Dc, Ns, Cu	All channel of FY-4A	Day	0.95	15780	Wang et al. (2023) [40]
RF	Clear, low cloud, middle cloud, thin cloud, thick cloud, multilayer cloud, cumulonimbus	R (0.64), R (1.6), BT (11.2 μm), BTD (11.2–3.9 μm), BTD (11.2–7.3 μm), BTD (11.2–8.6 μm), BTD (11.2–12.3 μm)	Day	0.88	127192	Wang et al. (2023) [41]
RF	Clear, low cloud, middle cloud, thin cloud, thick cloud, multilayer cloud, cumulonimbus	BT (11.2 μm), BTD (11.2–3.9 μm), BTD (11.2–7.3 μm), BTD (11.2–8.6 μm), BTD (11.2–12.3 μm)	Night	0.79	72934	Wang et al. (2023) [41]
RF	Clear, single, multi	BT (3.9 um), BT (7.3 m), BT (8.6 μm), BT (11.2 μm), BT (12.4 μm), BTD (3.9–11.2 μm), BTD (8.6–11.2 μm), BTD (11.2–12.4 μm), latitude, longitude	Day and night	0.79	12553889	Tan et al. (2022) [16]
DNN	Clear, single-ice, single-mixed, single-water, multi	BT (3.9–13.3 μm), cosine of satellite zenith angle, simulated clear-sky radiances	Day and night	0.81	1114591	Li et al. (2022) [17]
AInfraredCCM	Clear, Ci, Dc, Ac, As, Ns, Cu, Sc, St	BT (3.9–13.3 μm), BTD (11.2–9.6 μm), BTD (3.9–11.2 μm), BTD (11.2–12.4 μm), BTD (12.4–10.4 μm), BTD (7.3–10.4 μm), latitude, longitude	Day and night	0.86	1314275	This study

provides a statistical summary of the cloud classification models, featuring columns for the employed model, the number and specific types of cloud classifications, features, time, overall accuracy, sample size, and references. When comparing this study with those of other researchers, the AInfraredCCM demonstrated superior cloud classification performance (Table 9). While the CNN and the Backpropagation Neural Network (BP) achieved higher overall accuracy (0.95 and 0.86, respectively), it is noteworthy that these models were exclusively designed for daytime data and did not incorporate nighttime data. Furthermore, the CNN and BP models covered only eight and six cloud types (including clear sky), respectively, whereas nine cloud type classifications were retrieved in this study. Considering both daytime and nighttime data, this study achieved an overall accuracy of 0.86. In contrast, Yu et al. also categorized clouds into nine types, but their approach was exclusively applicable to the daytime, with a significantly lower overall accuracy compared to this study. Similarly, Tan and Li conducted research throughout the day; however, their approach did not exceed the overall accuracy or number of cloud types in this study. In summary, the AInfraredCCM has notable advantages in cloud classification.

3.2. Comparison with Himawari-8 Cloud Classification Production

In this study, a comparative analysis approach was employed to compare the cloud classification results of AInfraredCCM with CLTYPE using test data. To facilitate this comparative analysis in this study, the CLTYPE data were mapped to those of 2B-CLDCLASS-LIDAR. Notably, since Himawari-8 products officially cover cloud daytime evaluations, comparison and evaluation were limited to the daytime scenarios. The comparison was conducted in three scenarios, and the results are summarized in

Table 10. Accuracy rate of Himawari-8 CLTYPE and AInfraredCCM.

	Himawari-8 CLTYPE	AInfraredCCM
Full area	0.48	0.86
Cloudy area	0.36	0.87
Clear sky	0.77	0.85

.

The three scenarios (all-sky, cloudy, and clear-sky) produced correct identification probabilities of 0.48, 0.36, and 0.77, respectively. The results of AInfraredCCM are presented in Table 7 and Table 8. The model demonstrated superior performance, with overall accuracies of 86.22%, 87%, and 85% for all-sky, cloudy, and clear sky, respectively, outperforming the CLTYPE. The comparison data results are presented in Table 10; for further reference, Figures S1–S130 illustrate the disk classification results in contrast to those of the CLTYPE.

3.3. Effects of Day and Night on Cloud Classification

The results were meticulously examined individually under both daytime and nighttime conditions to assess the classification performance. An analysis of the efficacy of the AInfraredCCM during both the diurnal and nocturnal phases was carried out by assessing the overall precision, recall, and F1-score. Pixels in the test dataset with a solar zenith angle exceeding 80° were designated as nighttime, whereas the rest were designated as daytime [42].

Table 11. Cloud classification results of the AInfraredCCM for daytime and nighttime.

Time	Cloud Type	Clear	Ci	Dc	Ac	As	Ns	Cu	Sc	St
Accuracy = 85.82%
Daytime	Precision	0.85	0.90	0.92	0.82	0.89	0.95	0.68	0.89	0.98
	Recall	0.89	0.88	0.86	0.72	0.88	0.93	0.54	0.92	0.90
	F1-score	0.87	0.89	0.89	0.77	0.88	0.94	0.60	0.91	0.94
Accuracy = 91.45%
Nighttime	Precision	0.90	0.92	0.99	0.87	0.92	0.96	0.77	0.93	0.97
	Recall	0.90	0.91	0.92	0.85	0.93	0.97	0.56	0.96	0.96
	F1-score	0.90	0.91	0.95	0.86	0.93	0.96	0.65	0.94	0.96

displays cloud classification results of the AInfraredCCM for daytime and nighttime.

Figure 7 presents the bar charts and provides a visual representation of the evalumetrics for the AInfraredCCM in daytime and nighttime. During the daytime, the AInfraredCCM achieved an average accuracy of 85.82% for all cloud types. All cloud classifications, except Cu, had accuracy levels above 0.8, accompanied by high recall and F1 scores, which indicated that the classification results were reliable. The AInfraredCCM performed better in nighttime than in daytime, attaining an impressive accuracy of 91.45%. In contrast, Cu had an overall accuracy of 0.77, whereas Dc, Ac, and As exhibited classification accuracies above 0.87. The confusion matrices presented in Figure 7 indicate that Cu was the cloud type with the weakest classification performance during both the daytime and the nighttime. This trend can be attributed to the presence of various types of Cu that exhibit complex characteristics. Some fragmented Cu clouds possess limited thickness and area along with high radiation transmittance, so their reflectance is very similar to that of the ground surface. These properties make them easy to be misclassified as clear sky, thus resulting in relatively low recognition accuracy for Cu throughout both during the daytime and nighttime evaluations.

On 15 May 2019, at 04:20 UTC, a cloud map covering southeastern Australia and the South Pacific Ocean was classified using the AInfraredCCM. The data corresponding to the CPR/CALIOP orbit are visually depicted in Figure 8a. The brightness temperature data collected from the 10.4 μm channel of the CPR/CALIOP orbit are displayed in Figure 8b. The AInfraredCCM classification results for this specific region are presented in Figure 8c. The radar track for this scenario extends from the lower-right corner to the upper-left corner of the cloud map, encompassing both daytime and nighttime hours. In this case, the CLTYPE encompasses both daytime and nighttime data, with data in the latitude range from −52.45°S to −56.56°S representing the nighttime region, as depicted in Figure 8d. As part of this comprehensive study, nighttime data, in addition to daytime data, underwent cloud classification. The classification results (Figure 8e) exhibit a significant degree of alignment with the single-layer cloud data (Figure 8f) obtained from the 2B-CLDCLASS-LIDAR. Figure 8f illustrates the outcomes of the combined cloud products of the CPR/CALIOP and Himawari-8 data. Figure 8g provides an insight into the vertical profile of the cloud types along the orbit for CPR/CALIOP.

3.4. Effects of Different Seasons on Cloud Classification

The model was evaluated across all four seasons to determine its suitability.

Table 12. Results of the four seasonal classifications.

Season	Cloud Type	Clear	Ci	Dc	Ac	As	Ns	Cu	Sc	St
Accuracy = 86.61%
Spring	Precision	0.85	0.90	0.93	0.83	0.90	0.96	0.69	0.90	0.97
	Recall	0.90	0.87	0.87	0.75	0.87	0.94	0.56	0.93	0.90
	F1-score	0.88	0.88	0.90	0.79	0.89	0.95	0.62	0.92	0.93
Accuracy = 85.60%
Summer	Precision	0.84	0.91	0.95	0.82	0.87	0.95	0.66	0.88	0.97
	Recall	0.88	0.89	0.89	0.74	0.88	0.93	0.50	0.92	0.93
	F1-score	0.86	0.90	0.92	0.78	0.88	0.94	0.57	0.90	0.95
Accuracy = 85.87%
Autumn	Precision	0.85	0.90	0.93	0.81	0.88	0.95	0.68	0.88	0.99
	Recall	0.89	0.87	0.90	0.74	0.88	0.93	0.54	0.93	0.90
	F1-score	0.87	0.89	0.91	0.77	0.88	0.94	0.60	0.90	0.94
Accuracy = 87.27%
Winter	Precision	0.87	0.91	0.86	0.84	0.91	0.95	0.69	0.89	0.99
	Recall	0.91	0.89	0.88	0.75	0.88	0.93	0.55	0.93	0.87
	F1-score	0.89	0.90	0.87	0.79	0.90	0.94	0.60	0.91	0.92

presents the performance of AInfraredCCM in different seasons. Figure 9 shows the confusion matrices for the four seasons. The test dataset incorporates data from the four seasons and includes samples from daytime and nighttime observations as well as diverse subsurface information. During spring, summer, autumn, and winter, the overall accuracy was consistently >85% (86.61%, 85.60%, 85.87%, and 87.27% for spring, summer, autumn, and winter, respectively), which is consistent with the overall accuracy of the model. Ci, Dc, Ns, and St exhibited accuracies greater than 90%, coupled with recall and F1-scores of approximately 0.9. However, the classification of Cu remains challenging. Based on the characteristics of Cu, in subsequent experiments, we divided it into continuous and fragmented forms to improve the detection rate of Cu.

Figure 10 shows a comparative analysis of the results for all four seasons, with individual cases (a)–(d) corresponding to spring, summer, autumn, and winter, respectively. The three columns represent the brightness temperature for each case and the classification results of the AInfraredCCM and the CLTYPE. In general, the brightness temperature of a cloud is inversely proportional to its altitude, with lower brightness temperatures indicating higher cloud cover.

Taking the spring scenario (a) as an example, the first image is a brightness temperature map captured on 15 May 2019 at 02:50 UTC. In this figure, the AInfraredCCM effectively classifies the entire cloud image into regions with lower brightness temperature values. However, CLTYPE struggles to identify nighttime areas. In the summer scenario (b) taken at the typhoon center at UTC 05:00 on 5 August 2017, the CLTYPE identified Ci and Dc within the core of the typhoon but encountered difficulties in distinguishing these clouds from clear skies over most of the region. This study is able to finely classify clouds based on their brightness and temperature characteristics. It also categorized fill areas in CLTYPE, thereby enhancing the coherence of the classification results. Owing to the complex cloud composition during typhoons, the classification results of AInfraredCCM are consistent with the actual conditions. In contrast, the CLTYPE exhibited inaccuracies such as mislabeling as Ac in autumn (c). Similarly, the winter scenario (d) resulted in Cu being erroneously classified as clear, with many small cloud structures being misclassified. This study employed a pixel-level cloud classification method that facilitated the identification of more fragmented clouds and cloud pixels that resemble clear skies.

4. Discussion

This study utilized Himawari-8 satellite infrared data and CPR/CALIOP cloud products for conducting comprehensive research on all-day cloud classification. The input features included latitude, longitude, brightness temperature, and brightness temperature difference data derived from Himawari-8. Cloud type labels were extracted from the CloudSat/CALIOP joint secondary product 2B-CLDCLASS-LIDAR. Following this, a cloud classification model was developed to handle all-day observations by leveraging satellite detection. Five machine learning models were employed for model development. The results highlight the superior performance of the AInfraredCCM based on the XGBoost model compared to that of the other models. It achieved impressive overall accuracy, precision, recall, and F1 scores of 86.22%, 0.88, 0.84, and 0.86, respectively.

The AInfraredCCM surpassed many previous models in terms of overall accuracy and cloud classification diversity (Table 9). Unlike previous studies that primarily focused on daytime data due to limited nighttime cloud product data, this study included both daytime and nighttime data. The model performed well at night, yielding an overall classification accuracy of 91.45%. Previous studies on nighttime cloud classification often had a limited range of cloud types. In this study, satellite data were categorized into nine types—clear, Ci, Dc, Ac, As, Ns, St, Sc, and Cu. Compared with other nighttime classification models, this model achieved classification of a broader range of cloud types while maintaining a good overall classification performance. However, due to the limited amount of nighttime data, it may result in a more favorable classification performance. In future research, it is advisable to increase the volume of nighttime data. Through the AInfraredCCM, continuous cloud classification was realized throughout the day with a time resolution of up to 10 min. These high-resolution and continuous cloud classification results provide a richer dataset for exploring various cloud parameters and support all-day meteorological monitoring.

After conducting a comparative and analytical evaluation of the classification results across all-day, daytime, nighttime, and four-season scenarios, it was evident that the classification performance for Cu clouds was relatively subpar. The confusion matrix (Figure 4, Figure 7 and Figure 9) clearly depicts that Cu clouds are predominantly misclassified as clear skies and Sc. This phenomenon is closely related to the developmental stages of Cu. Cu clouds were classified into two types: fragmented and continuous Cu. Fragmented Cu are usually small, which makes their detection using satellite observations challenging. These clouds are easily dispersed by wind and have a short lifespan. Consequently, satellite-based systems may mistakenly identify clear skies as broken Cu clouds. Additionally, Cu clouds, as a convective system forming in the lower atmosphere, can transition into Sc clouds. Cu exhibited distinct characteristics when the convection intensified. If their convection development is suppressed, then Cu clouds may transform into Sc, leading to frequent misidentifications between the two cloud types.

There are some limitations that need to be further investigated. First, the cloud type labels used in this research were derived from the 2B-CLDCLASS-LiDAR product, which is based on data from CPR/CALIOP. During the label screening process, only pixels with a single layer of clouds were selected for comparison with the CLTYPE. Moreover, the majority principle was employed for cloud type screening during filtering labels. It is important to note that this screening method may result in the omission of certain data with limited feature information, particularly in the case of fragmented Cu and optically thin Ci. To improve the quality of the dataset, more refined label-generation techniques should be explored in future studies. Second, errors can arise when observing the same object using the Himawari-8 and CloudSat satellites from different observation positions. Spatiotemporal matching errors may be more pronounced, especially in regions with large observation angles [43,44]. Therefore, error correction is vital. In future research, the cloud positions in the AHI data could be computed using cloud information, including longitude, latitude, and cloud top height, collected by CALIPSO satellites. The corrected cloud positions can then be used to create a more accurate dataset [34,45].

5. Conclusions

This study describes an algorithm for all-day cloud classification, referred to as AInfraredCCM, which is based on machine learning and utilizes Himawari-8 data. AHI and CPR/CALIOP data collected in November 2018, January 2019, March 2019, and June–July 2019 were used to develop the algorithm. After preprocessing, the data were randomly divided according to a ratio of 9:1. Of the data, 90% were allocated to the training and validation datasets for algorithm development, while the remaining 10% were reserved for testing. The predictors employed in this algorithm include 10 IR channels, 5 BTDs channels, and latitude and longitude information. The conclusions of this study are detailed here:

(1)

The overall accuracy, precision, recall, and F1-score of AInfraredCCM cloud classification were 86.22%, 0.88, 0.84, and 0.86, respectively. Notably, the here-proposed model outperformed the other models selected for this study (Table 8) and those proposed by other researchers (Table 9). These results indicate that it is an efficient all-day cloud classification method.

(2)

The model performed well when used for all-day cloud classification or when used separately for daytime and nighttime classification, which suggests that the AInfraredCCM provides continuous data for cloud classification research throughout the day.

(3)

The model was applied to both day and night scenarios as well as to four seasons and produced good classification results. In addition to Cu, this study demonstrated efficacy in classifying other cloud types. More emphasis should be laid on Cu in future studies.