Estimating Daily Dew Point Temperature Using Machine Learning Algorithms

In the current study, the ability of three data-driven methods of Gene Expression Programming (GEP), M5 model tree (M5), and Support Vector Regression (SVR) were investigated in order to model and estimate the dew point temperature (DPT) at Tabriz station, Iran. For this purpose, meteorological parameters of daily average temperature (T), relative humidity (RH), actual vapor pressure (Vp), wind speed (W), and sunshine hours (S) were obtained from the meteorological organization of East Azerbaijan province, Iran for the period 1998 to 2016. Following this, the methods mentioned above were examined by defining 15 different input combinations of meteorological parameters. Additionally, root mean square error (RMSE) and the coefficient of determination (R2) were implemented to analyze the accuracy of the proposed methods. The results showed that the GEP-10 method, using three input parameters of T, RH, and S, with RMSE of 0.96°, the SVR-5, using two input parameters of T and RH, with RMSE of 0.44, and M5-15, using five input parameters of T, RH, Vp, W, and S with RMSE of 0.37 present better performance in the estimation of the DPT. As a conclusion, the M5-15 is recommended as the most precise model in the estimation of DPT in comparison with other considered models. As a conclusion, the obtained results proved the high capability of proposed M5 models in DPT estimation.


Introduction
Dew point temperature (DPT) is defined as the temperature in which air becomes liquid water due to the high concentration of water molecules.Precise and accurate estimation of DPT has a significant role in solving agricultural problems, such as calculating the amount of available moisture in the air and estimating the near surface humidity [1].DPT and relative humidity are commonly used to measure the air humidity level [2].The DPT can also be used to estimate the temperature of crops considering glaciation [3].Many studies have paid attention to the accurate estimation of DPT using regression methods.However, data-driven methods such as Gene Expression Programming (GEP) and Neuro-Fuzzy Inference System (ANFIS) have been developed to identify optimal functions and modeling for complex phenomena.In this regard, several studies have been carried out on the application of the mentioned methods in meteorological studies [4][5][6][7][8][9][10][11][12][13].Shiri [2] compared the capabilities of the artificial neural network (ANN) and GEP to estimate the DPT using meteorological parameters at Seoul and Inchon stations, located in South Korea.They used two management scenarios: In the first scenario, the meteorological information of each station was used to estimate the DPT of the same station; in the second scenario, they used the meteorological information of adjacent stations.Their results showed that in both scenarios GEP was more accurate than ANN.Also, the application of the second scenario showed that GEP had more accurate results in estimating the DPT values of Seoul stations using Incheon station parameters.They also reported that the DPT values at Seoul Station could be estimated using the average temperature and relative humidity of the Incheon station with proper accuracy.Deka et al. [14] examined the ability of a support vector machine (SVM), ANN, and Extreme Learning Machine (ELM) to estimate DPT at two stations in Iran.They showed that the results of the ELM model were more similar to observed DPT at the two mentioned stations.In other research, Zounemat-Kermani [15] implemented two methods of multiple linear regression (MLR) and Levenberg-Marquardt algorithm (LMA) in the artificial neural network (LMA-ANN) in order to estimate DPT values at Ontario Station, Canada.The results of the LMA-ANN model had an appropriate match with observational data.Additionally, Jia et al. [16] investigated dew formation.For this purpose, they used meteorological data of average temperature, sunny hours, wind speed, saturated vapor pressure, relative humidity, and DPT values of three stations of Dagot, Pohang, and Ulsan, South Korea.They reported that the effects of sunny hours, wind speed, and saturated vapor pressure were lower than other parameters.Therefore, it was possible to estimate the DPT using average temperature and relative humidity.Attar et al. [17] used GEP, multivariate adaptive regression splines (MARS), and SVM models to estimate the DPT in arid regions of Iran.Using the meteorological data of 13 synoptic stations during the 55 years (1996 to 2014), and by defining 50 different scenarios.They concluded that the MARS model offers more accurate results than other studied models.In a similar study, Mehdizadeh et al. [18] estimated the DPT values in Tabriz and Urmia cities, in the northwest of Iran, using the GEP method.They defined three scenarios: A parameters-based scenario, a temperature-based scenario, and a periodicity-based scenario considered the meteorological parameters of minimum, maximum, and mean air temperature, actual vapor pressure, and atmospheric pressure.Their results showed that the actual vapor pressure is the most effective meteorological parameter in estimating the DPT in the study area.
Therefore, over the last decade, researchers have tried to estimate DPT values with suitable accuracy.For this reason, the main purpose of the current study was to implement three data-driven methods of GEP, M5, and SVR in order to improve the estimation accuracy and develop some mathematical formulations for obtaining precise estimations of DPT values using explicit formulations.To the best of our knowledge, the application of M5 has not been reported in the literature.In other words, the goals of the study were (i) evaluating the performance of the models above in the estimation of DPT, and (ii) investigating the role of climatic parameters estimation DPT values.The rest of the paper is structured as follows: Section 2 describes implemented methods, evaluation parameters, and characteristics of the study area.Additionally, Section 3 discussed the obtained results and, finally, the conclusion is presented in Section 4.

Study Area
The Tabriz synoptic station is one of the oldest meteorological stations belonging to the Iranian Meteorological Organization, located in East Azerbaijan province with a latitude of 38 • 05 N and longitude of 46 • 17 E and an elevation of 1364 m above sea level.In the current study, the DPT values of Tabriz station in the period of 1998 to 2016 were utilized to evaluate the precision of the considered models.The geographic location of the study area is shown in Figure 1.
Water 2019, 11, x FOR PEER REVIEW 3 of 12

Study Area
The Tabriz synoptic station is one of the oldest meteorological stations belonging to the Iranian Meteorological Organization, located in East Azerbaijan province with a latitude of 38° 05′ N and longitude of 46° 17′ E and an elevation of 1364 m above sea level.In the current study, the DPT values of Tabriz station in the period of 1998 to 2016 were utilized to evaluate the precision of the considered models.The geographic location of the study area is shown in Figure 1.

Support Vector Regression (SVR)
SVM, which is established by statistical learning theory, has been broadly applied for identifying complex patterns of different environmental phenomena.Moreover, SVR, as one of the types of SVM, has been used previously for regression problems [20,21].The basics of the support vector regression are presented briefly below.
If the available data can be linearly split, they can be distinguished using cloud-computing system data.In some cases, data cannot be linearly separated.In these cases, the data is mapped to a larger dimensional space, after which, the data are separated.Figure 2B shows the non-linear mapping of φ (0): R n → R n h, in which, the nonlinear training data are mapped to a higher-dimensional space.Following that, a linear relationship is obtained between the input and output data in the converted space.The general form of the linear relation is as follows: (1) where, f(x) is the estimated variable, W T is the transpose of the vector of coefficients and b is a constant coefficient.The SVM is based on minimizing the value of empirical error.(2) where, Θε (y, f(x)) is the ε-insensitive error function defined as Equation (3): Location of the study area [19].

Support Vector Regression (SVR)
SVM, which is established by statistical learning theory, has been broadly applied for identifying complex patterns of different environmental phenomena.Moreover, SVR, as one of the types of SVM, has been used previously for regression problems [20,21].The basics of the support vector regression are presented briefly below.
If the available data can be linearly split, they can be distinguished using cloud-computing system data.In some cases, data cannot be linearly separated.In these cases, the data is mapped to a larger dimensional space, after which, the data are separated.Figure 2B shows the non-linear mapping of ϕ (0): R n → R n h, in which, the nonlinear training data are mapped to a higher-dimensional space.Following that, a linear relationship is obtained between the input and output data in the converted space.The general form of the linear relation is as follows: where, f(x) is the estimated variable, W T is the transpose of the vector of coefficients and b is a constant coefficient.The SVM is based on minimizing the value of empirical error.
Water 2019, 11, 582 where, Θ ε (y, f(x)) is the ε-insensitive error function defined as Equation (3): optimal separator plate will maximize the distance between training data.Following this, the SVR model minimizes the general error concerning the constraints.These limitations presented in Equations ( 4) to (8).
where, ξi is an error greater than -ε, ξi* is the error greater than +ε, and c is a constant.After obtaining the optimal separator plate, the vector of the coefficients w obtained as Equation ( 9).

𝑦 − 𝑊 𝜑 𝑥 − 𝑏 𝜀 + 𝜉
in which, βi* and βi are calculated by applying quadratic programming and Lagrange coefficients, respectively.Finally, the SVR function is obtained as Equation ( 10) in a two-dimensional space, as follows: =  * −    ,  +  (10) K (Xi, Xj) is called the Kernel function and is equal to the inner product of the two vectors of Xi and Xj in a space with a high dimension.In the SVR method, several Kernel functions are used, including the polynomial Kernel function, the normalized polynomial Kernel function, the radialbase Kernel function, and the Pearson Kernel function [22].

Gene Expression Programming
GEP, which is a variant of genetic programming (GP), is a generalized genetic algorithm.GEP is considered to be a circular method based on Darwin's theory of evolution.GEP at the beginning of the process does not take into account the functional relationship and can optimize the structure of In addition, the function Θ ε (y, f(x)) is used to find the optimal separator plain in a high dimension space (which may have infinitive dimensions).In a space with a high dimension, an optimal separator plate will maximize the distance between training data.Following this, the SVR model minimizes the general error concerning the constraints.These limitations presented in Equations ( 4) to (8).
where, ξ i is an error greater than -ε, ξ i * is the error greater than +ε, and c is a constant.
After obtaining the optimal separator plate, the vector of the coefficients w obtained as Equation (9).
in which, β i * and β i are calculated by applying quadratic programming and Lagrange coefficients, respectively.Finally, the SVR function is obtained as Equation ( 10) in a two-dimensional space, as follows: K (X i , X j ) is called the Kernel function and is equal to the inner product of the two vectors of X i and X j in a space with a high dimension.In the SVR method, several Kernel functions are used, including the polynomial Kernel function, the normalized polynomial Kernel function, the radial-base Kernel function, and the Pearson Kernel function [22].

Gene Expression Programming
GEP, which is a variant of genetic programming (GP), is a generalized genetic algorithm.GEP is considered to be a circular method based on Darwin's theory of evolution.GEP at the beginning of the process does not take into account the functional relationship and can optimize the structure of the model and its components [23].Unlike the genetic algorithm, GEP acts on the tree structure of formulas rather than a series of binary numbers.The tree structures created from the set of functions (mathematical operators used in formulas) and terminals (problem variables and constant numbers).Before the implementation of GEP, the following factors are determined in the following: 1.
The mathematical operators used in formulas, 3.
Select the fitness function (RMSE, MSE, MAE, . . . ) to measure the fitness of the formulas, 4.
Select the parameters controlling the implementation of the program (population size, the probability associated with the use of genetic operators and other details related to the implementation of the program), 5.
The completion benchmark and the presentation of the results of the program implementation (the number of new population production, the determination of the specified amount for the fitness of the formulas if the fitness level is equal to or greater than that value stopped) [24].The outlines of the mentioned steps are shown in Figure 3.Moreover, the parameters used in the implementation of the GEP presented in Table 1.
Water 2019, 11, x FOR PEER REVIEW 5 of 12 the model and its components [23].Unlike the genetic algorithm, GEP acts on the tree structure of formulas rather than a series of binary numbers.The tree structures created from the set of functions (mathematical operators used in formulas) and terminals (problem variables and constant numbers).Before the implementation of GEP, the following factors are determined in the following: (the number of new population production, the determination of the specified amount for the fitness of the formulas if the fitness level is equal to or greater than that value stopped) [24].The outlines of the mentioned steps are shown in Figure 3.Moreover, the parameters used in the implementation of the GEP presented in Table 1.

M5 Model Tree
The M5 model tree is a subset of the machine learning and data mining methods developed by Quinlan [25].Data mining refers to the process of searching and discovering various models, summarizing and obtaining quantities from the collections.Learning machine and data mining methods have the ability to discover data patterns, semi-automatically.The main reasons for using a model tree are as follows: 1.The model tree is directly related to estimative variables; therefore, the results of the model are easy to understand.

M5 Model Tree
The M5 model tree is a subset of the machine learning and data mining methods developed by Quinlan [25].Data mining refers to the process of searching and discovering various models, Water 2019, 11, 582 6 of 13 summarizing and obtaining quantities from the collections.Learning machine and data mining methods have the ability to discover data patterns, semi-automatically.The main reasons for using a model tree are as follows: 1.
The model tree is directly related to estimative variables; therefore, the results of the model are easy to understand. 2.
Model trees are non-parametric, and there is no user intervention on them.

3.
The output of the model has a high degree of accuracy that can be compared to other models.
The structure of the model trees includes roots, branches, nodes, and leaves.The nodes are represented by a circle and the branches represent the connection between the nodes [26].The generation of the model tree structure consists of different steps of creating a tree and pruning it.In the first step, an inferential algorithm or division criterion is used for the production of the tree.The decision criterion for the M5 is the standard deviation of the class values, which is calculated as a quantity of error to a node and calculates the expected reduction in this error as the result of the test of each attribute in that node.The standard deviation ratio (SDR) is calculated as follows (Equation ( 11)): In which, T is a collection of input samples to each node, T i is a subset of the samples that have the i output of the potential series and sd denotes the standard deviation [27].As a result of the branching process, the data in the child nodes has a lower standard deviation than the parent node and is purer.After maximizing all possible branches, the M5 selects an attribute that maximizes the expected reduction.This division forms a large quasi-tree structure, which causes over-fitting.To overcome the mentioned problem, the tree should be pruned.This is done by replacing a sub-tree with a leaf; therefore, the second step in the design of a model tree is to prune the grown tree and to replace the sub-trees with linear regression functions.This method for creating the model tree divides the space of the input parameters into smaller areas or sub-areas.In each area, a linear regression model is fitted.After obtaining a linear model, the simplification of the model can minimize the estimation error by deleting the model parameters [28].

Evaluation Criteria
Error values between observed and estimated data were determined by RMSE and coefficient of determination (R 2 ).The RMSE was used to evaluate the accuracy of the estimations.The consistent estimated values of the model lead to the minimization of the statistical index.Furthermore, R 2 is a statistical tool for determining the type and degree of the relationship of a variable with other variables.This coefficient varies from 0 to 1; when there is no relationship between two variables, its value is equal to zero [29,30].Furthermore, Taylor diagrams [31] were used to check the accuracy of the mentioned models.It is noteworthy that Taylor suggested a diagram, in which measured parameters and some characteristics of the model are summed up, coincidentally.Surprisingly, Taylor diagrams utilize several points on a polar plot for comparing the accuracy of measured and estimated values.In these diagrams, the coefficient of determination and normalized standard deviation are represented by an azimuth angle and radial distances from the base point, respectively [31,32].

Results and Discussion
In order to reach the research objectives, daily average temperature, relative humidity, actual vapor pressure, wind speed, and sunny hours at the Tabriz synoptic station were collected from the Meteorological Organization of East Azerbaijan province, Iran during the period 1998 to 2016.The statistical characteristics of the implemented data are presented in Table 2.There is no basic way of separating training and testing data.For example, the study of Kurup and Dudani [33] used a total of 63% of their data for model development, whereas Samadianfard et al. [4] and Samadianfard et al. [7] used 67% of total data, and Deo et al. [34] used 70% of total data to develop their models.Thus, to develop the studied GEP, M5, and SVR models for estimation DPT, we divided the data into training (67%) and testing (33%).Therefore, the accuracy of the models in estimating DPT evaluated through Taylor diagrams.Additionally, the effects of considered meteorological parameters were inspected by defining 15 different input combinations (Table 3).After performing the computations for different input combinations, the accuracy of the considered models was determined in the testing phase based on the statistical criteria (Equations ( 9) and ( 10)) and Taylor diagrams.The obtained results are presented in Table 4.As can be seen in Table 4, GEP-10 with RMSE of 0.96 degrees and R 2 equal to 0.902 with the parameters of T, RH, and S shows better performance compared to GEP models.However, SVR-6 with RMSE of 0.44 degree and R 2 of 0.996 presents more accurate estimation compared to the SVR models.Furthermore, the best estimation of the DPT, based on M5 models, was related to M5-15 with RMSE of 0.37 degree and R 2 of 0.996 and using all considered meteorological parameters as the input.In other words, a comprehensive comparison between the mentioned models exhibited that M5-15 had the best performance in estimation DPT values by using input combinations of T, S, RH, W, V p .After selecting the most accurate models for estimation DPT values, the time series plots and scatterplots are finalized and illustrated in the Figures 4 and 5.   (SVR-6) It can be comprehended from Table 4 and Figure 4 that the estimation accuracy of the M5-15 was higher than the GEP-10 and SVR-6.The above-mentioned conclusion, regarding the high accuracy of the M5-15 model in estimation the DPT for Tabriz station, can be deduced from Figure 5.In this figure, it can be seen that the distribution of the points around the bisector line in the M5-15 model was less than the corresponding points of GEP-10 and SVR-6.It can be comprehended from Table 4 and Figure 4 that the estimation accuracy of the M5-15 was higher than the GEP-10 and SVR-6.The above-mentioned conclusion, regarding the high accuracy of the M5-15 model in estimation the DPT for Tabriz station, can be deduced from Figure 5.In this figure, it can be seen that the distribution of the points around the bisector line in the M5-15 model was less than the corresponding points of GEP-10 and SVR-6.
Furthermore, Taylor charts were used to examine the standard deviation and correlation values among estimated and measured DPT values for the GEP, M5, and SVR models with different input parameters.Taylor diagrams for models mentioned above are shown in Figure 6.The length of the space from the reference point (a green color point) to each point is defined as centered RMSE [31].Therefore, the most accurate model has a minimum distance between the green point and its corresponding point.According to Figure 6 Furthermore, Taylor charts were used to examine the standard deviation and correlation values among estimated and measured DPT values for the GEP, M5, and SVR models with different input parameters.Taylor diagrams for models mentioned above are shown in Figure 6.The length of the space from the reference point (a green color point) to each point is defined as centered RMSE [31].Therefore, the most accurate model has a minimum distance between the green point and its corresponding point.According to Figure 6  One of the advantages of GEP and the M5 models, in comparison with other data-driven methods, is their ability to provide explicit relationships to calculate the output parameter.Therefore, for the current study, Equation ( 12) was obtained for estimation DPT values using GEP-10 as the most accurate GEP model.(12) Additionally, the list of linear equations (presented in Table 6) was the outcome of M5-15 estimation of the DPT values using meteorological parameters of T, S, RH, W, Vp.One of the advantages of GEP and the M5 models, in comparison with other data-driven methods, is their ability to provide explicit relationships to calculate the output parameter.Therefore, for the Additionally, the list of linear equations (presented in Table 5) was the outcome of M5-15 estimation of the DPT values using meteorological parameters of T, S, RH, W, V p .As previously mentioned, Deka et al. [14] used SVM, ANN, and ELM and implemented meteorological parameters of minimum, maximum, and average temperatures, relative humidity, atmospheric pressure, water vapor pressure, sunny hours, and solar radiation in order to estimate the DPT in two cities of Kerman province, Iran.The minimum RMSE reported in the mentioned study was 0.49, related to the ELM method by using minimum temperature and water vapor pressure data as input parameters.In the present study, with the application of the M5 and applying meteorological parameters of average temperature, relative humidity, actual vapor pressure, wind speed, and sunny hours, the RMSE decreased to 0.37, indicating the high accuracy of the M5 model tree for estimation DPT values.The output of the M5 was a simple linear relationship that can be used to calculate the DPT values easily, while the ELM does not have such a capability.Additionally, in another study by Baghban et al. [35] the maximum estimation accuracy of the DPT was reported using an SVM model with an RMSE value of 0.4.
Furthermore, the precision of the M5-15 in the current study was more than the accuracy of the proposed SVM method by Baghban et al. [35].To investigate the influence of input parameters on the DPT estimation, the RMSE and R 2 were utilized for different groupings of input variables.For this purpose, all utilized models, including GEP, M5, and SVR were selected for sensitivity analysis (Table 6).Each model confirmed the extent to which the eliminated variable would affect the model accuracy.As shown in Table 6, the precision of all models decreased if each of T, RH, V p , W, and S input parameters were removed from the modeling.Furthermore, it can be comprehended that T had the greatest effect in increasing the prediction accuracy.In other words, eliminating T caused a sharp increase in RMSE values in all studied models.

Conclusions
In the current study, three data-driven methods including GEP, M5, and SVR were used to estimate DPT values at Tabriz synoptic station, Iran.For this purpose, the meteorological parameters were collected from the Meteorological Organization of East Azerbaijan province in the period 1998 to 2016.Also, 15 different input combinations were defined to study the effect of meteorological parameters on the estimation of DPT values.The results of this study revealed that the SVR-6, using two input parameters of T and RH, and GEP-10 using three parameters of T, RH, and S, had appropriate performance in the estimation of DPT values.Furthermore, the overall analysis of the studied methods showed that the M5-15 using five parameters of T, S, RH, W, and V p had the best performance in the estimation of DPT values at Tabriz station in comparison with all considered models with different input combinations.To conclude, M5-15 is proposed as the most accurate method for the estimation of DPT values at the Tabriz synoptic station, Iran.

1 .
Terminal set (problem variables, randomized constant numbers), 2. The mathematical operators used in formulas, 3. Select the fitness function (RMSE, MSE, MAE,...) to measure the fitness of the formulas, 4. Select the parameters controlling the implementation of the program (population size, the probability associated with the use of genetic operators and other details related to the implementation of the program), 5.The completion benchmark and the presentation of the results of the program implementation
, M5-15 (a blue color point) offered the most accurate estimations of DPT values at Tabriz station.Water 2019, 11, x FOR PEER REVIEW 9 of 12 , M5-15 (a blue color point) offered the most accurate estimations of DPT values at Tabriz station.

Figure 6 .
Figure 6.Taylor diagrams of estimated DPT values in the test period.

Figure 6 .
Figure 6.Taylor diagrams of estimated DPT values in the test period.

Table 1 .
Parameters used in the GEP method.

Table 1 .
Parameters used in the GEP method.

Table 2 .
Statistical characteristics of the meteorological data.

Table 3 .
Different combinations of input parameters in the estimation of dew point temperature.

Table 4 .
Evaluation of the performance of GEP, M5 model tree (M5), and support vector regression (SVR) models in the testing phase.
Equation (12)was obtained for estimation DPT values using GEP-10 as the most accurate GEP model.

Table 5 .
Obtained Equation from the M5 model tree for scenario No. 15.

Table 6 .
Effect of removing input variables on the utilized model's accuracy for predicting DPT values.