Research on Modeling Weighted Average Temperature Based on the Machine Learning Algorithms

Abstract

In response to the nonlinear fitting difficulty of the traditional weighted average temperature (T_m) modeling, this paper proposed four machine learning (ML)-based T_m models. Based on the seven radiosondes in the Yangtze River Delta region from 2014 to 2019, four forecasting ML-based T_m models were constructed using Light Gradient Boosting Machine (LightGBM), Support Vector Machine (SVM), Random Forest (RF), and Classification and Regression Tree (CART) algorithms. The surface temperature (T_s), water vapor pressure (E_s), and atmospheric pressure (P_s) were identified as crucial influencing factors after analyzing their correlations to the T_m. The ML-based T_m models were trained using seven radiosondes from 2014 to 2018. Then, the mean bias and root mean square error (RMSE) of the 2019 dataset were used to evaluate the accuracy of the ML-based T_m models. Experimental results show that the overall accuracy of the LightGBM-based T_m model is superior to the SVM, CART, and RF-based T_m models under different temporal variations. The mean RMSE of the daily LightGBM-based T_m model is reduced by 0.07 K, 0.04 K, and 0.13 K compared to the other three ML-based models, respectively. The mean RMSE of the monthly LightGBM-based T_m model is reduced by 0.09 K, 0.04 K, and 0.11 K, respectively. The mean RMSE of the quarterly LightGBM-based T_m model is reduced by 0.09 K, 0.04 K, and 0.11 K, respectively. The mean bias of the LightGBM-based T_m model is also smaller than that of the other ML-based T_m models. Therefore, the LightGBM-based T_m model can provide more accurate T_m and is more suitable for obtaining GNSS precipitable water vapor in the Yangtze River Delta region.

1. Introduction

Global Navigation Satellite System (GNSS) precipitable water vapor (PWV) can be obtained through the inversion of several meteorological parameters and the zenith total delay (ZTD) of the GNSS signal. The process of PWV calculation also involves critical parameters such as zenith hydrostatic delay (ZHD), zenith wet delay (ZWD), and weighted average temperature (T_m) [1]. It is evident that T_m is a crucial conversion parameter for obtaining high-precision PWV. T_m is the continuous integration of water vapor pressure and temperature in the atmosphere from the Earth’s surface to the top of the troposphere. Water vapor pressure and temperature can be obtained from atmospheric reanalysis data or directly from radiosondes [2]. However, the low temporal resolution of atmospheric reanalysis data and radiosondes prevents users from accessing real-time information on meteorological parameters [3]. Therefore, a high-accuracy T_m model is essential to improve the accuracy and practicality of obtaining real-time GNSS-PWV.

The traditional T_m model is an empirical model depending on the periodic variation of the T_m phase and amplitude [4,5]. According to Bevis et al., based on 8718 radiosonde records covering a range of 27° to 65° latitude in North America, a global T_m model was constructed using one-dimensional linear regression based on the surface temperature [6]. Yao et al. combined the Bevis model with the global pressure and temperature (GPT) to construct the global weighted mean temperature (GWMT) model, which provides global T_m estimates [7]. Guo, et al. constructed a local T_m model that exhibited overall higher accuracy compared to the Bevis model based on seven radiosondes data from 2015 to 2017 in the Yangtze River Delta region [8].

The abovementioned empirical T_m models are not in consideration of the nonlinear relationship between T_m and spatiotemporal meteorological factors. Researchers have created some nonlinear T_m models to address the inaccurate calculation of linear empirical T_m models [9]. Machine learning methods, renowned for their potent nonlinear fitting capabilities, have been extensively used to improve the accuracy of T_m models. Thus, numerous scholars have successfully constructed ML-based T_m models, which have demonstrated more accuracy than traditional T_m models in extensive experiments [9,10,11,12,13,14]. Ding et al. demonstrated that their back propagation neural networks (BPNN) T_m model outperformed conventional models [15]. Sun et al. constructed T_m models based on the random forest (RF), BPNN, and generalized regression neural network (GRNN) algorithms. These T_m models are able to obtain high-accuracy T_m [10]. Cai et al. used artificial neural networks to develop a high-accuracy hybrid T_m model for the China region [16].

In addition to the T_m model, machine learning algorithms have demonstrated significant success in wind power prediction, air temperature estimation, and landslide disaster prevention. It is possible to accurately predict the fluctuations and trends in wind power and prevent landslide disasters. Yun et al. combined the light gradient boosting machine (LightGBM) model with a convolutional neural network model to construct an ultra-short-term wind power prediction model, from which they obtained high-precision wind power prediction [17]. Mohamed Saber and his colleagues developed a river valley flash flood prediction model based on the LightGBM algorithms. Their multiple field tests confirmed the effectiveness of the LightGBM-based model in predicting flash floods [18]. Wind speed modeling was performed by Morshed-Bozorgdel and his team utilizing the stacking ensemble machine learning (SEML) method. The results revealed that the performance of the base algorithms was significantly affected by the incorporation of the SEML method in wind speed modeling. The highest correlation coefficient (R) achieved in wind speed modeling at the sixteen stations using the SEML was 0.89. It was observed that the implementation of the SEML method led to an increase in the accuracy of wind speed modeling by more than 43% [19]. A modified air temperature estimation model was proposed by Xu, et al. [20], which combined the temperature–vegetation index method with a multiple regression model. Statistical results demonstrated that the modified method yielded higher accuracy in estimating air temperature for the winter wheat planted area than the traditional regression method. The aforementioned achievements demonstrate the broad prospects and potential of machine learning algorithm models in forecasting applications. The machine learning methods are of great significance for T_m modeling and improving their accuracy in GNSS-PWV studies [9,17,18,21,22].

The accuracy of the traditional T_m models is limited because the nonlinear relationships between T_m and spatiotemporal meteorological factors have not been considered in previous empirical T_m models. In this paper, seven radiosondes in the Yangtze River Delta region from 2014 to 2018 will be utilized to construct the ML-based T_m models, which are based on the RF, LightGBM, support vector machine (SVM), and classification and regression tree (CART) algorithms. Then, the accuracy of ML-based T_m models will be assessed using 2019 radiosondes. Ultimately, the most appropriate ML-based T_m model will be selected for the Yangtze River Delta region.

2. Data and Methods

2.1. Data

The study area is the Yangtze River Delta region of China. The region is located between 114° E and 124° E and 26° N to 36° N. Figure 1 depicts the topography of the region. The radiosondes were obtained at http://weather.uwyo.edu/ (accessed on 28 May 2023). The data covers the period from 2014 to 2019, with a time resolution of 12 h.

Table 1. The location information of radiosondes in the Yangtze River Delta region.

Region	Site Number	Location [°]	Elevation [m]
Hangzhou	58457	(30.23° N, 120.16° E)	43.00
Quzhou	58633	(28.96° N, 118.86° E)	71.00
Shanghai	58362	(31.40° N, 121.46° E)	4.00
Anqing	58424	(30.53° N, 117.05° E)	20.00
Fuyang	58203	(32.86° N, 115.73° E)	33.00
Nanjing	58238	(32.00° N, 118.80° E)	7.00
Sheyang	58150	(33.76° N, 120.25° E)	7.00

provides the location information of the radiosondes. Four main variables, such as water vapor pressure (E_s), surface temperature (T_s), atmospheric pressure (P_s), and T_m, will be used in the ML-based T_m models for data training and accuracy analysis.

2.2. Methods

2.2.1. Machine Learning Algorithms

1.

LightGBM

The LightGBM is an efficient machine learning framework that utilizes gradient lifting algorithms to train scalable models quickly. The exclusive feature bundling technology is used to reduce the number of features and speed up training in the experiment. In addition, the gradient-based one-side sampling technique is used to filter out the samples with small prediction errors to minimize the training time. Compared to the traditional level-wise growth strategy, LightGBM prioritizes the selection of leaf nodes with the highest splitting gain for growth after finding an optimal split node. This approach allows for faster identification of leaf nodes that contribute to minimizing the loss function but may potentially lead to overfitting. Therefore, LightGBM provides parameters to control the growth strategy. LightGBM supports multi-threaded parallel learning and can also operate in a distributed environment to handle large-scale datasets.

Therefore, LightGBM has fast efficiency, high accuracy, low memory usage, custom loss function, and scalability advantages. It is suitable for discovering the nonlinear relationships between T_m and other meteorological influencing factors, such as temperature, humidity, wind speed, and precipitation [23].

A predictive LightGBM-based model can be developed to forecast future meteorological variables by leveraging historical meteorological observations, geographical features, and seasonal variations. Moreover, the LightGBM-based model enables the strength and likelihood of weather phenomena prediction, such as heavy rain, typhoons, and tornadoes. The LightGBM adeptly captures intricate relationships among diverse influencing factors, encompassing meteorological conditions, geographic topography, and environmental data, thereby providing timely alerts and decision-making support [24,25,26].

2.

RF

The Random Forest is an ensemble learning technique that utilizes bagging to create multiple distinct training datasets and employs multiple CART for prediction. The prediction outcome is determined by either the highest voting score or the average value. The core concept behind this method is that the collective judgments of multiple classifiers yield superior results compared to those of a single classifier, as illustrated in Equation (1).

$$\widehat{y}=\frac{1}{B}{\sum }_{i=1}^B{f}_i(X)$$

(1)

where $B$ represents the number of classification regression trees in the RF. The segmentation points of the regression tree in the model are determined by the minimum regression error, which is the weighted sum of the subset regression error, as shown in Equations (2) and (3).

$$K=\frac{M_L}{M}\times K\left(B_L\right)+\frac{M_R}{M}\times K(B_R)$$

(2)

$$M\left(B\right)=\frac{\sum _{i=1}^M{(y_i-\overline{y})}^2}{M}$$

(3)

where $K(B_L)$ and $K(B_R)$ represents the regression error for left and right subsets. $M_L$ and $M_R$ are the left and right subsets, respectively. $M$ represents the total number of samples. The $K(B)$ refers to the regression error.

The RF provides accurate prediction by combining historical meteorological data and processing complex relationships and features between them to construct models. The RF application in meteorology fully utilizes its advantages in adaptability to complex nonlinear relationships and robustness to outliers. It can be a powerful tool in weather forecasting, model evaluation, and meteorological data analysis [27,28,29].

3.

SVM

The SVM is a machine learning algorithm derived from statistical theory primarily applied to classification and regression problems. It maps the original data from the input space to a high-dimensional feature space through nonlinear mapping. Its purpose is to construct an optimal classification hyperplane in the feature space, separating samples of different categories. It maximizes the distance between the classification hyperplane and the support vectors, thereby achieving better generalization capability, as Equation (4) shows.

$$f\left(x\right)=w^{T}x+b$$

(4)

where $w$ is the weight vector, $b$ is the bias term, and $x$ is the input variable. As a predictive model, the SVM can solve nonlinear problems between variables and demonstrates good generalization ability and robustness to outliers. The model has broad potential and offers promising prospects for forecasting meteorological variables [21,30,31].

4.

CART

The CART is a common decision tree algorithm. It is frequently employed for regression and classification problems. It aims to split the input space into different regions and assign each region a category or regression value. When the CART is applied to the regression analysis of meteorological variables, it selects a feature as the root node, and the entire training dataset is used as the node’s dataset. The result of Equation (5) can be used to evaluate the splitting effectiveness of each feature in the current node.

$$MSE=\frac{\sum _{i=1}^n{(Y_i-Y_{mean})}^2}{n}$$

(5)

where $MSE$ is the mean squared error, $n$ is the sample size of the current node, $Y_i$ is the target variable, $Y_{mean}$ is the mean of current node sample. The MSE of each target variable at different thresholds was calculated by CART and selects the target variable and threshold corresponding to the minimum MSE as the optimal split. Then, repeat the process continuously until the stopping criteria are met. Finally, the mean or median is chosen as its regression value in the node.

The CART demonstrates robust performance and flexibility in regression tasks, rendering it extensively applied for predictions across diverse domains. It solves nonlinear problems and disposes of outliers present within multiple features [32,33,34].

While the ML-based T_m model can address the issue of nonlinear fitting models, it also exhibits certain limitations. For instance, machine learning algorithms are susceptible to the quality of input data. The presence of noise, missing values, or outliers in the input data can affect the model’s performance and accuracy. There are various ML algorithms, and selecting an appropriate model is crucial for predictive performance. Additionally, model parameter tuning is necessary to obtain the best prediction results. The problems of overfitting and underfitting may arise in ML-based models. Overfitting occurs when the model performs well on the training data but poorly on unseen data, while underfitting indicates that the model fails to capture the complex relationships within the data, leading to suboptimal performance. To mitigate the impact of such issues on model accuracy, rigorous data preprocessing and model optimization are required.

2.2.2. Evaluations

The bias and root mean square error (RMSE) are used as the accuracy evaluation metrics in this experiment.

$$X_{BIAS}=\frac{{\sum }_{i=1}^N\left|x_{obs,i}-x_{model,i}\right|}{n}$$

(6)

The bias measures the gap between predicted values (${\mathrm{X}}_{\mathrm{m}\mathrm{o}\mathrm{d}\mathrm{e}\mathrm{l},\mathrm{i}}$) and the observed values (${\mathrm{X}}_{\mathrm{o}\mathrm{b}\mathrm{s},\mathrm{i}}$). It represents the average error between the observed value (${\mathrm{X}}_{\mathrm{o}\mathrm{b}\mathrm{s},\mathrm{i}}$) and the predicted values (${\mathrm{X}}_{\mathrm{m}\mathrm{o}\mathrm{d}\mathrm{e}\mathrm{l},\mathrm{i}}$), as Equation (6) shows.

$$X_{RMSE}=\sqrt[]{\frac{{\sum }_{i=1}^N(x_{obs,i}-x_{model,i}{)}^2}{N}}$$

(7)

The RMSE serves as a metric for assessing the accuracy of regression models. It measures the bias between the predicted values (${\mathrm{X}}_{\mathrm{m}\mathrm{o}\mathrm{d}\mathrm{e}\mathrm{l},\mathrm{i}}$) and the observed values (${\mathrm{X}}_{\mathrm{o}\mathrm{b}\mathrm{s},\mathrm{i}}$). By computing the square root of bias, RMSE provides a single value to represent the model’s overall performance. As depicted in Equation (7), a lower RMSE signifies higher accuracy, while a higher RMSE implies more significant prediction errors.

3. Results and Discussion

Figure 2 illustrates the modeling workflow of ML-based algorithms. The data was first preprocessed, and then modeling was carried out using four distinct ML algorithms. Subsequently, modeling optimization is performed. Finally, the accuracy of the models is evaluated using the RMSE and bias.

3.1. Feature Selection

3.1.1. Correlation

The T_S, P_s, and E_s serve as indispensable input data in the T_m calculation process. Their spatiotemporal variations significantly influence the weighted distribution of the data. The T_S variability determines the weighting temperature from different locations, with higher T_S exerting a more significant influence on the result. Moreover, when considering the vertical temperature distribution, the changes in P_s are also employed as essential weights in the T_m computation, assigning different weights to the temperature at various pressure levels. Additionally, the impact of water vapor pressure and humidity on temperature is particularly notable under humid climatic conditions, potentially assigning higher weights to T_m from moist air.

To select input features of models, the Pearson coefficient was used for the correlation analysis between T_m and T_s, E_s, and P_s [35,36,37,38].

$$R=\frac{{\sum }_{i=1}^n(X_i-\overline{X})\times (Y_i-\overline{Y})}{\sqrt{{\sum }_{i=1}^n(X_i-\overline{X}{)}^2\times \sqrt{{\sum }_{i=1}^n(Y_i-\overline{Y}{)}^2}}}$$

(8)

where $n$ represent the sample number, and $X$ and $Y$ represent different variables. The correlation criteria are as follows. $\left|R\right|$ ≥ 0.81 represents an extremely strong correlation; 0.61–0.80 represents a strong correlation; 0.41–0.60 represents a moderately correlated; 0.21–0.40 represents a weakly correlated; and 0.0–0.20 represents a minimal correlated or no correlation. Figure 3a–c shows the R of T_m and E_s, T_s, P_s are 0.96, 0.95, and −0.87, indicating an extremely strong correlation between T_m and E_s, T_s, P_s.

3.1.2. Collinearity

As shown in

Table 2. Correlations, tolerances, and variance inflation factors (VIF) between variables.

Dependent Variable	Independent Variable	R	Independent Variable	Independent Variable	R	R2	Tol	VIF
Tm	Ts	0.95	Ts	Ps	−0.85	0.68	0.32	3.1
Tm	Ps	−0.87	Ts	Es	0.91	0.83	0.17	5.9
Tm	Es	0.96	Ps	Es	−0.85	0.70	0.30	3.3

, there is a strong correlation among the independent variables, which implies severe multicollinearity. The multicollinearity can introduce problems in the model, such as unstable coefficient estimates and difficulties in interpreting the impact of individual variables.

The tolerance $(Tol=1-R^2 )$ was used to evaluate multicollinearity among independent variables in the experiment. The $R^2$ represents the square of the correlation coefficient between two different variables and is referred to the determination coefficient. Table 2 shows that the $R^2$ and $Tol$ between the T_s and P_s are 0.68 and 0.32. Also, the $R^2$ and $Tol$ between T_s and E_s are 0.83$\mathrm{a}\mathrm{n}\mathrm{d} 0.17$, and the $R^2$ and $Tol$ between P_s and Es are 0.70 and 0.30. If the $Tol$ is greater than 0.2, we can conclude that there is no multicollinearity among the independent variables. However, the $Tol$ between T_s and E_s is less than 0.2. It suggests the possible presence of severe multicollinearity. The $VIF$ can be used to further verification of their multicollinearity, which is calculated by $VIF=1/(1-R^2)$. From Table 2, the VIF between T_s and E_s is 5.9. It can be concluded that there is no severe multicollinearity between T_s and E_s since their $VIF$ is less than 10 [39,40].

Based on the correlation analysis, it is evident that the key input variables for the ML-based models are T_s, P_s, and E_s in the study, whereas the output variable is T_m.

3.2. The ML-Based T_m Modeling

3.2.1. Data Preprocessing

After determining the modeling variables, the dataset mentioned in Section 2.1 must be partitioned accordingly. The training dataset consists of radiosondes collected in the Yangtze River Delta region during 2014–2018. The validation dataset consists of radiosondes in 2019.

To ensure the modeling accuracy, the preprocessing of outliers and missing data in the dataset is necessary. Two approaches were used to replace the outliers in the study. One method is interpolation when the outliers are moderately outliers [39]. Another one is the k-nearest neighbor (KNN) algorithm used for extreme outliers. The KNN relies on the distances between samples in the feature space to perform classification and prediction, which can improve data reliability. Interpolation was employed based on the numerical range to fill in the gaps of missing data.

Considering the different value ranges among variables. To avoid focusing more on features with a larger range during the training procedure of modeling, data standardization is necessary. It can improve the convergence speed, robustness, and interpretability of models [41,42]. The linear normalization was chosen for standardization, which maps the original dataset to the range of [0, 1] using the following formula.

$$x^{\prime }=\frac{(x-\mathrm{M}\mathrm{I}\mathrm{N}(x))}{(\mathrm{MAX}\left(x\right)-\mathrm{M}\mathrm{I}\mathrm{N}(x))}$$

(9)

3.2.2. The Model Optimization

Model optimization is the most crucial step in the modeling procedure. It includes the selections of cross-validation and model parameters, such as learning rate, maximum depth, and iteration number.

The GridSearch algorithm is employed to automatically search for the best training parameters. It iteratively trains to select the optimal parameters of ML-based T_m models. After a series of iterations, the optimal learning rate, maximum depth, and iteration number for the LightGBM-based T_m model are determined to be 0.03, 30, and 700, respectively. The optimal kernel function and penalty factor for the SVM-based T_m model are determined to be radial basis function (RBF) kernel and 1, respectively. The optimal number of classifiers and maximum depth for the RF and CART-based T_m models are 100 and 30, respectively.

The main step of K-fold cross-validation is dividing the dataset into 15 groups. Subsequently, one group is randomly chosen as the validation dataset, while the remaining K-1 groups would be the training dataset. The training dataset was used for the model’s training, and the validation dataset was used to evaluate the model’s accuracy. The procedure will repeat K times; the average accuracy rate is the final evaluation metric of the models. After completing optimization, the optimized models will be employed to predict T_m during 2019 in the Yangtze River Delta region.

3.3. Accuracy Analysis

In this section, to ensure the rigors of accuracy analysis, the ML-based T_m models were evaluated at various temporal resolutions (daily, monthly, and quarterly) to assess their performances comprehensively.

3.3.1. The Daily Models

Figure 4 shows the daily bias and RMSE of the four ML-based T_m models. As shown in Figure 4, the average RMSE of the LightGBM-based T_m model is 1.85 K, which is 0.07 K, 0.04 K, and 0.13 K lower than the SVM, CART, and RF-based T_m models, respectively. Moreover, the maximum RMSE of the LightGBM-based T_m model is lower than the SVM, CART, and RF-based T_m models by 0.15 K, 0.13 K, and 0.15 K. The LightGBM-based T_m model shares the same minimum RMSE with the SVM and CART-based T_m models but outperforms the RF model. As for the bias, the LightGBM, CART, and RF-based models demonstrate relatively small variations, while the SVM-based model exhibits more significant fluctuations. Clearly, the LightGBM, CART, and RF-based models are more stable and suitable for obtaining T_m in the Yangtze River Delta region.

The left side of Figure 5 illustrates that the predicted ML-based T_m does not exactly match the true T_m values. However, they exhibit a similar overall trend, with the predicted T_m from the LightGBM-based T_m model being closer to the true value. On the right side of Figure 5, it is evident that the LightGBM-based T_m model demonstrates a smaller overall deviation compared to other ML-based T_m models and is more proximate to zero.

3.3.2. Quarterly and Monthly Models

To further investigate the impact of temporal changes on model accuracy. The accuracy of the monthly and quarterly ML-based T_m models was analyzed in this section. The radiosondes in 2019 were divided into 12 validation datasets to analyze the accuracy of monthly ML-based T_m models. Similarly, when the temporal resolution is quarterly, the 2019 radiosonde data were divided into four validation datasets to analyze the accuracy of quarterly ML-based T_m models.

Figure 6 and Figure 7 show the average bias and RMSE for the quarterly and monthly ML-based T_m models. It shows that the RMSE of four quarterly ML-based T_m models fluctuates between 1.46 K and 2.58 K, while the bias varies between 0.05 K and 0.77 K. The RMSE of four monthly ML-based T_m models fluctuates between 1.27 K and 2.85 K, while their bias varies from 0.01 K to 1.29 K.

The result shows that the LightGBM-based T_m model is more accurate and stable than the other three ML-based T_m models in any spatiotemporal conditions. In summary, the LightGBM-based T_m model is more stable in providing accurate T_m and more suitable for GNSS-PWV research in the Yangtze River Delta region.

4. Conclusions

The purpose of the paper was to solve the nonlinear fitting problem of traditional T_m models based on the four machine learning algorithms. It aimed to select the most suitable ML-based T_m model in the Yangtze River Delta region.

Due to the strong correlations between T_m and T_s, E_s, and P_s, which were selected as the features of ML-based models. The optimization procedure of ML-based models includes the cross-validation and optimal parameters identification, such as learning rate, maximum depth, and iteration counts, to ensure the reliability and accuracy of models.

The comparisons among the LightGBM, SVM, CART, and RF-based T_m models revealed that the RMSE of the daily LightGBM-based T_m model decreased by 0.07 K, 0.04 K, and 0.13 K than the other three ML-based T_m models. Similarly, the RMSE of the monthly LightGBM-based T_m model decreased 0.09 K, 0.04 K, and 0.11 K, while the RMSE of the quarterly LightGBM-based T_m model decreased 0.09 K, 0.04 K, and 0.11 K. It was evident that the LightGBM-based T_m model was more stable and superior to other ML-based T_m models in different temporal variations. In summary, the LightGBM-based T_m model can be used to get more high-precision T_m and GNSS-PWV in the Yangtze River Delta region.

This study cannot demonstrate the suitability of the ML-based T_m models for other regions because it only validates the applicability in the Yangtze River Delta region. In order to further validate the applicability of ML-based T_m models, future research will involve constructing a comprehensive T_m model for the China region by algorithm fusion incorporating more ML-based algorithms.