An Improved LSSVM Model for Intelligent Prediction of the Daily Water Level

Abstract

Daily water level forecasting is of significant importance for the comprehensive utilization of water resources. An improved least squares support vector machine (LSSVM) model was introduced by including an extra bias error control term in the objective function. The tuning parameters were determined by the cross-validation scheme. Both conventional and improved LSSVM models were applied in the short term forecasting of the water level in the middle reaches of the Yangtze River, China. Evaluations were made with both models through metrics such as RMSE (Root Mean Squared Error), MAPE (Mean Absolute Percent Error) and index of agreement (d). More accurate forecasts were obtained although the improvement is regarded as moderate. Results indicate the capability and flexibility of LSSVM-type models in resolving time sequence problems. The improved LSSVM model is expected to provide useful water level information for the managements of hydroelectric resources in Rivers.

1. Introduction

Time series forecasting has been recognized as one of the classical problems in the fields of both energy engineering and science [1], among which daily water level forecasting is closely related to the hydroelectric resource utilization [2]. In order to obtain accurate and reliable water level forecasting, great efforts have been paid and fruitful achievements have been accomplished in various ways.

In view of the complex intrinsic mechanism and multiple influencing factors, artificial intelligence methods, e.g., adaptive network based fuzzy inference system [1,2] and neural network [3,4] have been accepted and extensively applied to resolve time series forecasting problems. The definition of membership function and rule system is important with regard to the model reliability [5] and accuracy [6]. In contrast to the artificial neural network models, grey models use only a small amount of historical data and mathematical relationship between variables is not required [7]. However, long term characteristics of hydrological data, such as seasonality and cyclical variations, need to be considered and more carefully handled. Successful experience has also been obtained by using artificial neural network (ANN) methods. Palani et al. [8] applied an ANN model for water quality estimation. Nourani et al. [9] established an ANN model for groundwater level prediction. Ivan and Gilja [10] showed good performance of ANNs for hydraulic parameter prediction. However, the model accuracy differs with neuron structures and parameter calibrations might be time-consuming.

The support vector machine (SVM) has been used to address short-term forecasting problems since the 90s of the 20th century, on the basis of which LSSVM (least squares support vector machine) is put forward to overcome drawbacks (e.g., computation cost [11], uncertainties in structural parameter determination [12]) of SVM. LSSVM models solve a linear matrix equation with fewer constraint conditions and have been utilized in a variety of applications, e.g., forecasting of groundwater level fluctuations [13], river stage [14], and watershed runoff [15]. In the case of monthly flow forecasting, Noori et al. [16] discussed the influence of parameter selections on the model performance. Hybrid models have also been proved to be effective ways, such as SVM-Wavelet transform [17].

Although the LSSVM models provide favorable solutions in hydrological forecasting problems, issues such as the kernel function and unbalanced features need to be carefully explored. Cheng et al. [18] improved LSSVM by integrating an adaptive time function. Thereby, the dynamic nature of the time series is considered by assigning an appropriate weight in the cash flow prediction for construction projects. To cope with low efficiency, Cong et al. [19] incorporated the fruit fly optimization algorithm (FOA) for appropriate parameter values of LSSVM. Comparison between LS-SVM-FOA and other models indicated the superiority of the improved model. Ghorbani et al. [20] modelled river discharge time series using SVM and ANN. The authors conclude that SVM and ANN have an edge over the results by the conventional RC (Rating Curve) and MLR (Multiple Linear Regression) models. This is more obvious for peak value predictions. The authors also presented a critical view on inter-comparison studies through a model establishment process and uncertainty analysis. Guo et al. [21] proposed an improved SVM model with an adaptive insensitive factor. Meanwhile, the wavelet de-noising method and phase-space reconstruction theory are applied to eliminate noise and determine the structure of the prediction model. The feasibility and performance of the model is evaluated through a case study of monthly streamflow forecasting. LS-SVM combined with self-organizing maps (SOM) has been applied in time series forecasting [22]. The two-stage architecture of LSSVM and SOM provides a promising tool for resolving the time series forecasting problem.

The present study focuses on the short-term forecasting of the daily water level by using an improved LSSVM model. The data source and pre-processing is presented in Section 2, followed by a detailed description of the improved LSSVM methodology in Section 3. Predictive capability of the improved LSSVM is verified and compared with the classic version in Section 4. Concluding remarks are finally drawn in Section 5.

2. Data Source

The blooming of water transport in the middle reaches of the Yangtze River has resulted in the rapid increase of vessel traffic flow. Meanwhile, human activities (e.g., sand mining, waterway regulation projects) and operations of upstream dams contribute to the complexity of the temporal characteristics of the daily water level [23]. The accurate forecasting of the daily water level is essential for waterway capacity evaluation as well as maritime risk assessment. Historic hydrological data obtained from the Changjiang Waterway Bureau (MOT, China) are applied for model training and testing. The time series of the daily water level span from 2010 to 2016 and the layout of the research region is presented in Figure 1. The flow runs from the Yichang station (upstream) to the Hankou station (downstream).

As aforementioned, the temporal variations of the daily water level in the middle reaches of the Yangtze River are affected by both natural factors and human activities. Seasonal and cyclical characteristics are readily observed in the raw time sequence of daily water level (as shown in Figure 2). It is necessary to eliminate the noise among the sample data before LSSVM model training. To eliminate noises (e.g., high-frequency fluctuations) in the hydrological time series, wavelet decomposition and reconstruction theory [24] were adopted in the data pre-processing for all five stations. The threshold is determined by the unbiased likelihood estimation method during the de-noising process.

Since missing data can occasionally occur due to technical failures, a linear interpolation was applied to ensure integrity of the input data. The de-noised daily water level data are thus proceeded by the model training of LSSVM_c (subscript c denotes conventional LSSVM model) and LSSVM_i (subscript i denotes improved LSSVM).

3. Methodology

3.1. Conventional LSSVM Model

The conventional support vector machine (SVM) is one of the machine learning methods [25]. The principle of machine learning is to minimize structural risk and achieve data classification or regression by applying kernel function and high-dimensional data simplification schemes (as presented in Equation (1)). On the other hand, least squares support vector machine (LSSVM) utilizes the least squares results as a basic algorithm to pursue structural risk minimization. Therefore, the basic equations of LSSVM_c are written as Equation (2).

$$\min J\left(w,e\right)=\frac{1}{2}{w}^2+\gamma {\sum }_{i=1}^m{X}_i$$

(1)

$$\mathrm{s}.\mathrm{t}.Y_i\left(w^{\prime }{X}_i+b\right)+{\xi }_i\ge 1;{\xi }_i\ge 0;i=1,2,\dots m$$

$$\min J\left(w,e\right)=\frac{1}{2}{w}^{T}w+\frac{1}{2}\gamma {\sum }_{i=1}^m{e}_i^2$$

(2)

$$\mathrm{s}.\mathrm{t}.Y_i=w^T\phi \left(X_i\right)+b+e_i;i=1,2,\dots m$$

where J is the risk bound, X_i is the slack variable, Y_i is binary target, w is the weight matrix, b is the bias, ξ_i is the slack variable, and e_i is the error variable; γ denotes a regularization constant; φ(X _i) is the kernel function.

3.2. Improved LSSVM Model

In order to obtain the unbiased estimation for the forecasting model, an extra bias error control term ($\frac{1}{2}a{b}^2$) is added in the objective function of LSSVM_i and the aforementioned Equation (2) is re-organized as follows:

$$\min J\left(w,e\right)=\frac{1}{2}{w}^{T}w+\frac{1}{2}\gamma {\sum }_{i=1}^m{e}_i^2+\frac{1}{2}a{b}^2$$

(3)

$$\mathrm{s}.\mathrm{t}.Y_i=w^T\phi \left(X_i\right)+b+e_i;i=1,2,\dots m$$

where a is a penalty factor for bias b exceeding the allowable range. To solve the optimization problem, the Lagrangian function is obtained as [16]:

$$L\left(w,b,e;\alpha \right)=J\left(w,e\right)-{\sum }_{i=1}^N{\alpha }_i\left\{w^T\phi \left(x_i\right)+b+e_i-y_i\right\}$$

(4)

where α_i are Lagrangian multipliers. By taking derivatives of w, b, e, α respectively and setting all derivatives as zero (i.e., Equation (5)), the following equations are thus derived.

$$\frac{\partial L}{\partial w}=\frac{\partial L}{\partial b}=\frac{\partial L}{\partial e_i}=\frac{\partial L}{\partial {\alpha }_i}=0$$

(5)

$$w={\sum }_{i=1}^N{\alpha }_i\phi \left(X_i\right)$$

(6)

$${\sum }_{i=1}^N{\alpha }_i-ab=0$$

(7)

$${\alpha }_i=\gamma e_i$$

(8)

$$w^T\phi \left(X_i\right)+b+e_i-Y_i=0;i=1,2,\dots m$$

(9)

A linear system of functions is therefore obtained.

$$\left[\begin{array}{cccc}-a& 1& \dots & 1\\ 1& {\phi }^T\left(X_1\right)\phi \left(X_1\right)+\frac{1}{\gamma }& \dots & {\phi }^T\left(X_1\right)\phi \left(X_i\right)\\ \dots & \dots & \dots & \dots \\ 1& {\phi }^T\left(X_i\right)\phi \left(X_1\right)& \dots & {\phi }^T\left(X_i\right)\phi \left(X_i\right)+\frac{1}{\gamma }\end{array}\right]\left[\begin{array}{c}b\\ {\alpha }_1\\ \dots \\ {\alpha }_i\end{array}\right]=\left[\begin{array}{c}0\\ Y_i\\ \dots \\ Y_i\end{array}\right]$$

(10)

where the kernel function is defined as:

$$K\left(X_i,X_j\right)={\phi }^T\left(X_i\right)\phi \left(X_i\right)=X_i^T{X}_j;i,j=1,2\dots N$$

(11)

The least squares method is introduced to solve the above equation, on the basis of which the least squares regression function is therefore derived.

$$f\left(x\right)={\sum }_{i=1}^N{\alpha }_{i}K\left(X_i,X_j\right)+b$$

(12)

Both LSSVM_c and LSSVM_i are trained by using historical daily water level data (Year 2010–2015), on the basis of which short-term forecasting is achieved for the year 2016. The training results for stations Jianli and Chenglingji have been presented in Figure 3. The error rate of model training is further presented and discussed in Section 4.

3.3. Model Performace Metrics

To evaluate the forecasting accuracy of both LSSVM_c and LSSVM_i, three metrics were employed as the root mean square error (RMSE, Equation (13)), the mean absolute percentage error (MAPE, Equation (14)), and the index of agreement (d, Equation (15) by Willmott, [26]). RMSE is a frequently used estimator of the difference between observations and model predictions. Meanwhile, MAPE quantifies the ratio between the deviation and observations, thus being scale independent. The index of agreement (d) was developed as a standardized measure of the model forecasting error and varies between 0 (no agreement at all) and 1 (perfect match). Suppose the water level observation is $\left\{X_{o1},X_{o2},\dots X_{on}\right\}$ and the corresponding model prediction is $\left\{X_{p1},X_{p2},\dots X_{pn}\right\}$. $\overline{X_o}$ is the mean value of the observed time sequence. All metrics are calculated as follows:

$$RMSE=\sqrt{{\sum }_{i=1}^n\frac{{\left(X_{oi}-X_{pi}\right)}^2}{n}}$$

(13)

$$MAPE=\frac{{\sum }_{i=1}^n\left|100\times \left(X_{oi}-X_{pi}\right)/X_{oi}\right|}{n}$$

(14)

$$d=1.0-\frac{{\sum }_{i=1}^n\left|X_{oi}-X_{pi}\right|}{{\sum }_{i=1}^n\left(\left|X_{pi}-\overline{X_o}\right|+\left|X_{oi}-\overline{X_o}\right|\right)}$$

(15)

4. Result and Discussion

4.1. LSSVM_i Forecasting of Daily Water Level

The water level forecasting by the LSSVM_i is presented for different stations together with field observations and LSSVM_c predictions (Figure 4 and Figure 5). It was found that the model forecasting is overall satisfactory. Some minor deviations were noted in the June and October for the station Shashi, which locates downstream of the Three Gorge Dam and Gezhou Dam. This could be attributed to the joint operations of multi-reservoir system, especially during the summer seasons when the rainfall generally increases. Besides, the influence of the river confluence was evident, e.g., Chenglingji, which is situated downstream of the Yangtze River–Dongting Lake confluence reaches. The majority of the discrepancies between LSSVM_i predictions and field observations appear in the summer seasons (e.g., June–August).

4.2. Model Performance Evaluation

The tuning parameters of LSSVM_i are determined by using the cross-validation method (

Table 1. Parameters of the LSSVM_i (least squares support vector machine) model with regard to different forecast lead times.

Lead Time	Parameter	Yichang	Shashi	Jianli	Chenglingji	Hankou
1-day	$a$	6.655e7	5.488e8	7.59e8	8.43e7	8.41e8
	$\gamma$	1.75e6	3.489e2	2.688e7	6.93e2	4.85e3
2-day	$a$	6.555e7	5.889e9	6.95e8	8.43e7	8.43e8
	$\gamma$	2.75e6	3.689e2	1.388e7	5.99e2	2.97e5
3-day	$a$	6.455e7	5.889e6	6.59e8	8.43e7	7.41e9
	$\gamma$	3.75e6	1.789e3	1.188e7	5.95e3	3.07e5

). By adopting the performance metrics introduced in Section 3.3, the model performance was investigated. Generally, three metrics are computed and tabulated (

Table 2. Computation results of RMSE, MAPE and d for LSSVM_c and LSSVM_i.

Stations	RMSE [m]		MAPE [%] [-]		D [-]
	LSSVMc	LSSVMi	LSSVMc	LSSVMi	LSSVMc	LSSVMi
Yichang	0.1394	0.1384	10.0872	8.6710	0.9848	0.9863
Shashi	0.1727	0.1740	20.7350	20.5456	0.9796	0.9794
Jianli	0.3196	0.2222	6.1984	2.4874	0.9552	0.9736
Chenglingji	0.1482	0.1449	1.3801	1.3232	0.9852	0.9857
Hankou	0.1536	0.1546	1.5315	1.4613	0.9862	0.9865

). It was found that the LSSVM_i provides more accurate forecasting of daily water level although the improvement is generally moderate. Moreover, RMSE has been calculated for model training results and the comparison is shown in Figure 6. The model residual is comparable for both training and testing stages, indicating the LSSVM_i does not suffer from an over-fitting problem.

The hydrological processes always show seasonal fluctuation features. $RMSE$ was calculated in terms of monthly data and presented in Figure 7. It is of note that the forecasting accuracy is improved by LSSVM_i. Similar temporal variation patterns are observed at Chenglingji while it is quite different at Jianli station.

The qualified rate which is defined as the proportion of the predicted values with relative error below 20% is widely used in practice [21]. As one-day forecasting of daily water level, the qualified rate is therefore calculated for both LSSVM_i and LSSVM_c (

Table 3. Computations of qualified rate R_q by using LSSVM_i and LSSVM_c.

Stations	Yichang	Shashi	Jianli	Chenglingji	Hankou
LSSVMi	0.9726	0.9235	1.0000	1.0000	1.0000
LSSVMc	0.9671	0.9207	0.9644	1.0000	1.0000

). The results show clear increases of qualified rate for the stations of Yichang, Shashi and Jianli, while full qualified forecasting is obtained for Chenglingji and Hankou.

4.3. Influence of Forecast Lead Time

The characteristics of the model accuracy are further explored when the forecast lead time increases. Examples with different forecast lead times are presented for Jianli (Figure 8) by using LSSVM_i. Computations of RMSE, MAPE and d are also presented and compared in

Table 4. Computations of RMSE, MAPE and d for different forecast lead time (LSSVM_i).

Stations	RMSE [m]			MAPE [%]			D [-]
	1-day	2-day	3-day	1-day	2-day	3-day	1-day	2-day	3-day
Jianli	0.2222	0.3755	0.4175	2.4874	3.1273	4.7808	0.9736	0.9603	0.9489
Chenglingji	0.1448	0.3262	0.4056	1.3232	1.9215	2.1845	0.9857	0.9737	0.9681

. The model accuracy is overall acceptable. Although it decreases gradually as the forecast lead time increases, the LSSVM_i model results in relatively high accuracy. This also implies that the proposed LSSVM_i model should be further improved in order to yield reliable and effective forecast of the daily water level in the Yangtze River, such as alternative types of kernel functions (e.g., RBF: radial basis function) or integrated algorithm (e.g., Wavelet-LSSVM).

5. Conclusions

The daily water level forecasting is of significant importance for the maritime administration and water transport safety. The temporal and spatial variations of the daily water level have been recognized as non-linear and non-stationary processes while the least square support vector machine (LSSVM) models have proved to be an effect tool. In the present study, an improved LSSVM_i model was proposed through a bias error control scheme.

The model performance of LSSVM_i in short term forecasting of the daily water level was evaluated and compared with the conventional LSSVM_c model. Both models were trained by using historical hydrological data (Year 2010–2015) to provide forecasting results of Year 2016. It was found that the result yielded by the LSSVM_i model is generally satisfactory, although the precision is inevitably affected by the seasonality and forecast lead time. Meanwhile, the influence of joint operations of the multi-reservoir system and river confluence was noted at Shashi station and Chenglingji station respectively. Although the forecasting accuracy decreases gradually as the forecast lead time increases, it is improved most of the time by LSSVM_i. The present study indicates the capability and flexibility of LSSVM-type models in resolving time series problems. The LSSVM_i proves to be a promising alternative in the daily water level forecasting of the Yangtze River (China) while optimization in forecast extrapolation and error control scheme is still required in future research.