Comparison of Multiple Linear Regression, Artificial Neural Network, Extreme Learning Machine, and Support Vector Machine in Deriving Operation Rule of Hydropower Reservoir

Abstract

Operation rule plays an important role in the scientific management of hydropower reservoirs, because a scientifically sound operating rule can help operators make an approximately optimal decision with limited runoff prediction information. In past decades, various effective methods have been developed by researchers all the over world, but there are few publications evaluating the performances of different methods in deriving the hydropower reservoir operation rule. To achieve satisfying scheduling process triggered by limited streamflow data, four methods are used to derive the operation rule of hydropower reservoirs, including multiple linear regression (MLR), artificial neural network (ANN), extreme learning machine (ELM), and support vector machine (SVM). Then, the data from 1952 to 2015 in Hongjiadu reservoir of China are chosen as the survey case, and several quantitative statistical indexes are adopted to evaluate the performances of different models. The radial basis function is chosen as the kernel function of SVM, while the sigmoid function is used in the hidden layer of ELM and ANN. The simulations show that three artificial intelligence algorithms (ANN, SVM, and ELM) are able to provide better performances than the conventional MLR and scheduling graph method. Hence, for scholars in the hydropower operation field, the applications of artificial intelligence algorithms in deriving the operation rule of hydropower reservoir might be a challenge, but represents valuable research work for the future.

1. Introduction

As a classical tool for adjusting natural runoff, reservoirs play an increasingly important role in the human society [1]. In practice, reservoirs need to satisfy a variety of practical requirements from various administrative departments, such as flood control, power generation, agricultural irrigation, water supply, and ecological protection [2]. In addition, booming socio-economic development has caused an unprecedented imbalance between water supply and water demand [3], and it is of great necessity to make the utmost of the regulation abilities of all the reservoirs [4]. As a result, the reservoir operation optimization has become one of the most significant tasks in water resources and power system over past decades [5]. In general, when the inflow per scheduling period is known, the global optimal solution for the reservoir operation problem can be easily obtained using the dynamic programming or other optimization methods [6]. Traditionally, this dispatching pattern is identified as the deterministic optimization and the corresponding scheduling result denotes the best solution found in this scenario [7]. Nevertheless, it is difficult to capture the perfect future runoff information because of the limitation of existing runoff forecasting technology. That is to say, the deterministic optimization is just a potential reflection for the fixed runoff case, but is not suitable for uncertain environments. In recent years, the fast-growing computer technology has markedly promoted the collection, processing, and storage of multi-source heterogeneous data produced in the entire life-cycle of a hydropower reservoir, which indicates that abundant data information is available to provide potential technical support for operators. Hence, a natural idea for handling the above issue is to examine the reservoir operation rule with actual data and planning data [8].

Implicit stochastic optimization (ISO) is a tool developed to achieve this goal. The key idea hidden in the ISO method is to derive the near-optimal reservoir operation rule from the long-term historical data [9]. Since its origin, ISO has attracted intensive attention from researchers all over the world and many effective methods have been developed to enhance the practicality of ISO [10]. Thus far, all of the existing methods can be roughly divided into two different groups [11]: the first includes traditional techniques like scheduling graph method (SGM) and multiple linear regressions (MLR); and the other is artificial intelligence (AI) approaches represented by artificial neural network (ANN), extreme learning machine (ELM), and support vector machine (SVM). The former involves classical methods, but they often fail to consider the latest operation data and deal with the complex nonlinearity between dependent variable and independent variables [12], while the latter can not only effectively alleviate the above defects, but also scientifically analyze large-scale dataset [13]. Over the past few decades, extensive applications of the AI-based methods have been published, because the AI-based methods can produce accurate results for a variety of engineering problems.

ANN is inspired by the working mechanism of the human brain and nervous system and has been widely applied to solve a variety of practical engineering problems. ANN can be treated as a special signal processing system with numerous interconnected layers linked by weight vectors between two neighboring layers. For instance, the authors of [14] used a particle swarm optimization model to train the parameters of ANN in stage prediction of Shing Mun River; the authors of [15] verified the feasibilities of support vector regression and ANN in river stage prediction; the authors of [16] developed a hybrid ANN method based on quantum-behaved particle swarm optimization for the daily runoff forecasting; the authors of [17] used ANN to forecast the ice conditions of the Yellow River in the inner Mongolia reach; the authors of [18] compared the performances of several AI-based methods (like ANN, and SVM) in monthly discharge predication; the authors of [19] made full use of ANN to forecast concurrent flows in a river system; and, based on ANN and SVM, the authors of [20] developed a hybrid forecasting method to effectively improve the forecast accuracy of monthly streamflow. Therefore, the above literatures indicate that ANN can provide reasonable results in water resources problems.

ELM is a novel training method for single-hidden layer feed-forward neural networks. After randomly determining the input-hidden weights and hidden biases, ELM can directly obtain the hidden-output weights by calculating the Moore–Penrose generalized inverse of the hidden output matrix. ELM has better generalization ability and a faster learning rate than the gradient-based method, promoting its widespread application in practice. For instance, the authors of [21] used wavelet neural networks and ELM to forecast monthly discharge; the authors of [22] used ELM and quantum-behaved particle swarm optimization to predict daily runoff; the author of [23] proposed a robust ELM method and then verified its feasibility in indoor positioning; the authors of [24] developed a weighted ELM for imbalance learning; the authors of [25] used a base-flow separation, binary-coded swarm optimization, and ELM for neural network river forecasting; the authors of [26] used binary-coded particle swarm optimization and ELM to develop a data-driven input variable selection method for rainfall-runoff modeling; and the authors of [27] developed a hybrid ELM model for multi-step short-term wind speed forecasting. Thus, existing simulations have fully demonstrated that ELM is a promising tool to address complicated regression and classification problems.

SVM is a supervised machine learning method based on the Vapnik–Chervonenkis dimension theory and structural risk minimization principle. It was proven in theory that SVM is able to guarantee global optimization for regression or classification problems. Recently, growing attention has been paid to the SVM method because it can produce satisfactory results in many engineering problems. For instance, the authors of [28] verified the predictability of monthly streamflow using SVM coupled with discrete wavelet transform and empirical mode decomposition; the authors of [29] used support vector machines for long-term discharge prediction; the authors of [30] developed a multi-objective ecological reservoir operation model based on an improved SVM model in which meteorological and hydrological data are used as the input information; the authors of [31] proposed an artificial bee colony method optimized SVM for system reliability analysis of slopes; and the authors of [32] used a modified SVM model based on the ensemble empirical mode decomposition to forecast the annual rainfall-runoff. Thus, various reports have fully proven feasibility of SVM in solving engineering problems.

Although a variety of reports on reservoir operation rule derivation has been published has been published over the past few decades, there are few publications evaluating the performances of different methods in deriving the hydropower reservoir operation rule thus far. Hence, in order to fill this gap, the primary goal of this paper is to compare the performances of several famous methods in deriving the reservoir operation rule, including the conventional scheduling graph method (SGM), MLR, ANN, ELM, and SVM. The Hongjiadu reservoir located in southwest China is chosen as the study area, and the effectiveness of five methods with different indexes is compared. The simulations show that three artificial intelligence methods (ANN, ELM, and SVM) are promising tools in deriving the reservoir operation rule when compared with SGM and MLR.

This rest of this paper is organized as follows. The deterministic hydropower reservoir operation is given in Section 2. Section 3 briefly presents the theories of several methods adopted in this study. The quantitative indexes, experimental results, and discussions are presented in Section 4, and the conclusions are given in Section 5.

2. Deterministic Hydropower Reservoir Operation to Produce Dataset

2.1. Objective Function

The scheduling process obtained from the deterministic optimization model is used to evaluate the performance of the derived reservoir operation rule. Considering that power generation is an important indicator to compare the management levels of different hydropower enterprises in a market environment, the objective function is often chosen to maximize of the multi-year average electric energy production in the target hydropower reservoir [33], which can be expressed as follows:

$$\max E={\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^M{P}_{i,j}{t}_{i,j}}}-g\left(P_{i,j}\right)$$

(1)

where $E$ is the value of the objective function; $N$ is the number of years; $M$ is the number of periods per year (month here, i.e., $M=12$); $P_{i,j}$ is the reservoir’s power output at the jth period of the ith year; $t_{i,j}$ is the total hours at the jth period of the ith year; and $g\left(P_{i,j}\right)$ denotes the penalty function, which can be described as below:

$$g\left(P_{i,j}\right)=\{\begin{array}{l}a{\left[P_{i,j}-P_{i,j}^{\min }\right]}^b\mathrm{if}\left(P_{i,j}<P_{i,j}^{\min }\right)\\ 0\mathrm{otherwise}\end{array}$$

(2)

where $P_{i,j}^{\min }$ is the preset minimum power output, and $a$ and $b$ are two positive coefficients.

2.2. Operation Constraints

To ensure that all the variables vary in the feasible zones, the following equality or inequality constraints are considered in the modeling process [34,35,36], including the water balance equation, storage volume limits, water spillage limits, turbine discharge, and power output limits.

$$V_{i,j}=V_{i,j-1}+\left[I_{i,j}-\left(q_{i,j}+s_{i,j}\right)\right]·t_{i,j};i\in \left[1,N\right],j\in \left[1,M\right]$$

(3)

$$V_{i,j}^{\min }\le V_{i,j}\le V_{i,j}^{\max };i\in \left[1,N\right],j\in \left[1,M\right]$$

(4)

$$q_{i,j}^{\min }\le q_{i,j}\le q_{i,j}^{\max };i\in \left[1,N\right],j\in \left[1,M\right]$$

(5)

$$s_{i,j}^{\min }\le s_{i,j}\le s_{i,j}^{\max };i\in \left[1,N\right],j\in \left[1,M\right]$$

(6)

$$P_{i,j}^{\min }\le P_{i,j}\le P_{i,j}^{\max };i\in \left[1,N\right],j\in \left[1,M\right]$$

(7)

where $V_{i,j}$, $I_{i,j}$, $q_{i,j}$, and $s_{i,j}$ are the storage volume, local inflow, turbine discharge, and abandoned spillage at the jth period of the ith year, respectively. $V_{i,j}^{\max }$ and $V_{i,j}^{\min }$ are the maximum and minimum storage volume at the jth period of the ith year, respectively. $q_{i,j}^{\max }$ and $q_{i,j}^{\min }$ are the maximum and minimum turbine discharge at the jth period of the ith year, respectively. $s_{i,j}^{\max }$ and $s_{i,j}^{\min }$ are the maximum and minimum water spillage at the jth period of the ith year, respectively. $P_{i,j}^{\max }$ and $P_{i,j}^{\min }$ are the maximum and minimum power output at the jth period of the ith year, respectively.

2.3. Optimization Methods

When the long-term inflow series, initial storage, and terminal storage are known, the above optimization model will become a deterministic operation problem that can be easily resolved by the famous dynamic programming method [37,38]. Then, the corresponding dynamic programming recursive equation is given as below:

$$E_{i,j}^{\ast }\left(V_{i,j}\right)=\max \left\{e_{i,j}\left(V_{i,j},V_{i,j-1}\right)+E_{i,j-1}^{\ast }\left(V_{i,j-1}\right)\right\}$$

(8)

where $e_{i,j}\left(V_{i,j},V_{i,j-1}\right)$ is the objective function value at the jth period of the ith year and $E_{i,j}^{\ast }\left(V_{i,j}\right)$ denotes the optimal cumulative return from the jth period of the ith year to the first period.

3. Brief Introductions of the Adopted Methods

Brief information of the four adopted methods is given in this section, including multiple linear regress (MLR), artificial neural network (ANN), extreme learning machine (ELM), and support vector machine (SVM). Because of its simple principle and easy implementation, MLR is seen as one of the most classical method in the reservoir operation rule field; with strong generalization and self-learning abilities, ANN is chosen as to derive reservoir operation rule; with faster training rate and better regression ability, ELM is also used for reservoir operation rule derivation; because of the merits of less computation parameters and theoretical completeness, SVM is also an alternative tool in deriving the reservoir operation rule. Besides, numerous mature software packages have been developed to achieve those methods, which can make an obvious improvement in the workload and execution efficiency.

3.1. Multiple Linear Regression (MLR)

Multiple linear regression (MLR) is a classical statistical tool develop to formulate the complex input–output relationship [39]. The key goal of MLR is to find out an approximation linear function between a set of independent variables and the dependent variable. Without loss of generality, the regression line in MLR can be expressed as follows:

$$y={\beta }_0+{\beta }_1{x}_1+\cdots +{\beta }_i{x}_i+\cdots +{\beta }_k{x}_k+\epsilon$$

(9)

where $y$ is the dependent variable, $x_i$ is the ith independent variable, ${\beta }_i$ is the polynomial coefficients of $x_i$, $k$ is the number of independent variables, and $\epsilon$ is the possible variation form.

Then, the above equation for a set of samples can be rewritten in a compact matrix form, which can be described as below:

$$\mathit{Y}=\mathit{X}\mathit{\beta }+\mathit{\epsilon }$$

(10)

where

$$\mathit{Y}=\left[\begin{array}{c}{y}_1\\ y_2\\ ⋮\\ y_n\end{array}\right],\mathit{\epsilon }=\left[\begin{array}{c}{\epsilon }_1\\ {\epsilon }_2\\ ⋮\\ {\epsilon }_n\end{array}\right],\mathit{\beta }=\left[\begin{array}{c}{\beta }_0\\ {\beta }_1\\ ⋮\\ {\beta }_k\end{array}\right]\mathrm{and}\mathit{X}={\left[\begin{array}{cccc}1& x_{1,1}& \cdots & x_{1,k}\\ 1& x_{2,1}& \cdots & x_{2,k}\\ 1& ⋮& x_{m,i}& ⋮\\ 1& x_{n,1}& \cdots & x_{n,k}\end{array}\right]}_{n\times \left(k+1\right)}$$

(11)

where $n$ is the number of samples, $x_{m,i}$ is the value of the ith independent variable in the mth sample, and ${\epsilon }_i$ is the ith residual error in the mth sample.

Based on the classical matrix operation theory, the standard least-square method can be used to calculate the coefficient vector $\mathit{\beta }$ associated in the MLR model, which is described as below:

$$\mathit{\beta }={\left({\mathit{X}}^{\mathrm{T}}\mathit{X}\right)}^{-1}{\mathit{X}}^{\mathrm{T}}\mathit{Y}$$

(12)

In such a way, the coefficient vector $\mathit{\beta }$ is known and the obtained MLR model can be adopted to predict the possible dependent variable related with the newly input vector.

3.2. Artificial Neural Network (ANN)

ANNs have been widely used to alleviate the shortcomings of the conventional algorithms to deal with complex problems. Without knowing the accurate mathematical description about the underlying process to be addressed, ANNs can learn hidden knowledge from the assigned data samples via establishing an input–output mapping for simulations. By far, there are many different types of ANN variants in previous literatures [40]. Here, the feed-forward network based on the back propagation training method is the choice of this paper. The sketch map of the feed-forward ANN model is drawn in Figure 1. In the feed-forward ANN, there are often three kinds of layers that are composed of multiple interconnected neurons, including the input layer receiving the external signal, the hidden layer or layers processing data in an order way, and the output layer exporting the predictive result.

Two key procedures are involved in the training process of the feed-forward ANN: The first is the feed-forward procedure in which the information is delivered from the input layer to the output layer via all the hidden layers, and the other is the reverse procedure in which the overall derivatives of the objective function in terms of weights are scattered among all the nodes of the neural network, which means that the weights and biases of all the nodes are dynamically adjusted based on the error between the simulated values of the network and the target outputs. For any one node per layer, the transfer function is adopted to obtain the accumulated result by calculating the inner product of the input vector and the weight vector, which can be expressed in Equation (13). Then, the accumulated result is directly delivered to the next layer. In addition, the neurons in the previous layer are often linked with all the neurons in the next layer, whereas the connections for any two neurons in the same layer do not exist.

$$y=f\left\{{\displaystyle \sum \mathit{w}·\mathit{x}}+b\right\}$$

(13)

where $y$ is the output of the node; $f$ is the transfer function of the node; $b$ is the bias value of the node; and $\mathit{w}$ and $\mathit{x}$ denote the input vector and weight vector of the node, respectively.

3.3. Extreme Learning Machine (ELM)

Extreme learning machine (ELM) is an emerging optimization technique developed to train the single-hidden layer feed-forward neural networks (SLFNs) [41]. In ELM, after randomly generating the input weights and hidden biases in the preset range, the hidden-output weights can be obtained via the matrix multiplication of the generalized inverse of hidden output matrix and the targeted output matrix. For a set of training samples $\left\{\left({\mathit{x}}_t,{\mathit{y}}_t\right),{\mathit{x}}_t\in {\mathit{R}}^n,{\mathit{y}}_t\in {\mathit{R}}^m,t=1,2,\cdots ,N\right\}$, the hidden outputs of the ELM model can be expressed as below:

$$f_t={\displaystyle \sum _{i=1}^L{\mathit{\beta }}_{i}g\left({\mathit{\alpha }}_i·{\mathit{x}}_t+b_i\right)}={\mathit{O}}_t,t=1,2,\cdots ,N$$

(14)

where ${\mathit{\alpha }}_i\in {\mathit{R}}^n$ is the weight vector linking the input layer and the ith hidden node, ${\mathit{\beta }}_i\in {\mathit{R}}^m$ is the weight vector linking the ith hidden node and the output layer, $b_i\in \mathit{R}$ is the bias value of the ith hidden node, $g(·)$ is the nonlinear activation function of the hidden node, L is the number of neurons in the hidden layer, and ${\mathit{O}}_t\in {\mathit{R}}^m$ is the simulated output vector of the neural network.

Then, the above equation can be rewritten as follows:

$$\mathit{H}\mathit{\beta }=\mathit{T}$$

(15)

where

$$\mathit{H}=\left[\begin{array}{c}\mathit{h}\left({\mathit{x}}_1\right)\\ ⋮\\ \mathit{h}\left({\mathit{x}}_N\right)\end{array}\right]={\left[\begin{array}{ccc}g\left({\mathit{a}}_1·{\mathit{x}}_1+b_1\right)& \cdots & g\left({\mathit{a}}_L·{\mathit{x}}_1+b_L\right)\\ ⋮& \cdots & ⋮\\ g\left({\mathit{a}}_1·{\mathit{x}}_N+b_1\right)& \cdots & g\left({\mathit{a}}_L·{\mathit{x}}_N+b_L\right)\end{array}\right]}_{N\times L}$$

(16)

$$\mathit{\beta }={\left[\begin{array}{c}{\mathit{\beta }}_1^{\mathrm{T}}\\ ⋮\\ {\mathit{\beta }}_L^{\mathrm{T}}\end{array}\right]}_{L\times m}\mathrm{and}\mathit{T}={\left[\begin{array}{c}{\mathit{y}}_1^{\mathrm{T}}\\ ⋮\\ {\mathit{y}}_L^{\mathrm{T}}\end{array}\right]}_{N\times m}$$

(17)

where $\mathit{H}$ denotes the output matrix of the hidden layer.

The optimization objective of ELM is to find appropriate parameters making ${\displaystyle \sum _{t=1}^N\Vert {\mathit{O}}_t-{\mathit{y}}_t\Vert }=0$ hold. Then, the coefficient matrix $\mathit{\beta }$ can be obtained by analytically determining the least-squared solution of the above-mentioned linear system $\underset{\mathit{\beta }}{\min }\Vert \mathit{H}\mathit{\beta }-\mathit{T}\Vert$, and then the special solution can be expressed as follows:

$$\mathit{\beta }={\mathit{H}}^{†}\mathit{T}$$

(18)

where ${\mathit{H}}^{†}$ denotes the Moore–Penrose generalized inverse of the hidden layer output matrix.

Then, the learning procedures for the ELM method are summarized as below:

Step 1:

Define the amount of hidden neurons and the activation function of each neuron.

Step 2:

Produce the input-hidden weights as well as the hidden biases.

Step 3:

Use all the data samples to obtain the output matrix of the hidden layer.

Step 4:

Choose the suitable method to calculate the hidden-output weights.

Step 5:

Use the optimized ELM network to produce the simulated output for new samples.

3.4. Support Vector Machine (SVM)

As a famous technology based on statistical learning theory, the support vector machine (SVM) makes full use of the principle of structural risk minimization, rather than the classical empirical risk minimization in conventional methods, to guarantee the generalization capability of the regression model [42]. Figure 2 shows he sketch map of the SVM model. Supposing that the ith sample has a D-dimensional input vector ${\mathit{x}}_i\in {\mathit{R}}^D$ and a scalar output ${\mathit{y}}_i\in \mathit{R}$, then the following regression function can be employed to express the nonlinear input–output relationship in the SVM model:

$$f\left({\mathit{x}}_i\right)={\mathit{w}}^{\mathrm{T}}\phi \left({\mathit{x}}_i\right)+b,i=1,2,\cdots ,l$$

(19)

where $f\left({\mathit{x}}_i\right)$ denotes the predicated value of the SVM model, $\phi \left({\mathit{x}}_i\right)$ is the nonlinear mapping function, and $\mathit{w}$ and $b$ are the parameters of the SVM model to be optimized.

For the training dataset with l samples, the v-SVM optimization model for can be expressed as follows:

$$\{\begin{array}{l}\min R\left(\mathit{w},\mathit{\xi },{\mathit{\xi }}^{\ast },\epsilon \right)=\frac{1}{2}{\Vert \mathit{w}\Vert }^2+C\left[v\epsilon +\frac{1}{l}{\displaystyle \sum _{i=1}^l\left({\xi }_i+{\xi }_i^{\ast }\right)}\right]\\ \mathrm{subjective}\mathrm{to}:y_i-{\mathit{w}}^{\mathrm{T}}\phi \left({\mathit{x}}_i\right)-b\le \epsilon +{\xi }_i\\ {\mathit{w}}^{\mathrm{T}}\phi \left({\mathit{x}}_i\right)+b-y_i\le \epsilon +{\xi }_i\\ {\mathit{\xi }}^{\ast },\epsilon \ge 0\end{array}$$

(20)

where $C$ is the parameter used to balance the empirical risk and model complexity term ${\Vert \mathit{w}\Vert }^2$, and ${\xi }_i^{\ast }$ is the slack variable to denote the distance of the ith sample outside of the $\epsilon$-tube.

As a standard nonlinear constrained optimization problem, the above problem can be resolved by constructing the dual optimization problem based on the Lagrange multipliers technique:

$$\{\begin{array}{l}\max R\left(a_i,a_i^{*}\right)={\displaystyle \sum _{i=1}^l{y}_i\left(a_i^{*}-a_i\right)}-\frac{1}{2}{\displaystyle \sum _{i=1}^l{\displaystyle \sum _{j=1}^l\left(a_i-a_i^{*}\right)\left(a_j-a_j^{*}\right)K\left({\mathit{x}}_i,{\mathit{x}}_j\right)}}\\ \mathrm{subjective}\mathrm{to}:{\displaystyle \sum _{i=1}^l\left(a_i-a_i^{*}\right)}=0\\ 0\le a_i,a_i^{*}\le C/l\\ {\displaystyle \sum _{i=1}^l\left(a_i+a_i^{*}\right)}\le C·v\end{array}$$

(21)

where $K\left({\mathit{x}}_i,{\mathit{x}}_j\right)$ is the kernel function satisfying the Mercer’s condition; and $a_i$ and $a_i^{*}$ are the nonnegative Lagrange multipliers, respectively.

After obtaining the best solution for the dual optimization problem, the parameters of the SVM model are known and the regression form for an unknown input vector $\mathit{x}$ is expressed as follows:

$$f\left(\mathit{x}\right)={\displaystyle \sum _{i=1}^l\left(a_i^{\ast }-a_i\right)K\left({\mathit{x}}_i,\mathit{x}\right)}+b$$

(22)

4. Experimental Results

4.1. Study Area and Dataset

Here, the Hongjiadu reservoir located on the mainstream of Wu River in southwest China is chosen as the study site. This reservoir has a total drainage area of 9900 km² and an average annual runoff of 4.89 billion m³. The dead water level is 1076 m and the dead storage is 1.14 billion m³; the normal water level is 1140 m and the corresponding storage volume is 4.5 billion m³. In Hongjiadu, the flood control level is 1138 m from 1 June to 1 September, while its regulation storage is about 3.4 billion m³. Obviously, the active-storage volume of the Hongjiadu reservoir is rather large in comparison with its annual inflow volume, meaning it plays a large role in determining the efficiencies to be achieved by any operation rules. Besides, the Hongjiadu reservoir has three mixed-flow turbine generating units with 200 MW per unit and its total installed capacity is 600 MW. Under normal circumstances, almost all of the flow of Hongjiadu is through the hydropower turbines. As a leading carry-over storage reservoir on the trunk stream of Wu River, the Hongjiadu reservoir begins to provide comprehensive benefits to promote the healthy and orderly development of Guizhou Province since being put into operation, like power generation, ecological protection, water supply, flood control, and environment governance. In practice, various scheduling purposes can be well addressed in the derived operating rule by setting the necessary constraints on some variables, like water levels, power outputs, or discharge rates [43,44,45,46].

The actual monthly streamflow data from January 1952 and December 2015 are collected from the watershed management organization of Wu River. Then, dynamic programing is employed to calculate the deterministic optimization results for Hongjiadu reservoir, and the minimum power output is defined as 150 MW. The optimal results (water level, inflow, and outflow) are drawn in Figure 3. For the optimized scheduling results, the first 50 years’ data are used to train the model, while those of the last 13 years are employed for testing. In addition, for the artificial intelligence algorithms (like ANN, ELM, and SVM), the numerical problem is often unavoidable if the smaller attribute values are dominated by the large ones. In order to effectively avoid the numerical difficulties in the modeling process, the normalization process in Equation (23) is adopted to make all the attribute values scale to the range of 0 and 1. All the results are obtained on a desktop computer with the Windows 7 operating system, Intel-core i7-3770 processor, and 4GB random access memory (RAM).

$${\tilde{x}}_i=\frac{x_i-\underset{1\le i\le n}{\min }\left\{x_i\right\}}{\underset{1\le i\le n}{\max }\left\{x_i\right\}-\underset{1\le i\le n}{\min }\left\{x_i\right\}}$$

(23)

where $x_i$ and ${\tilde{x}}_i$ denote the original and normalized value of the target factor, respectively.

To be mentioned, the actual, rather than runoff-prediction, data are used by reservoir operators in deriving the candidate operation rules, in which the actual monthly inflow is an important input component and the monthly outflow is the key decision variable. When the obtained operation rule is used for production guidance, the future monthly runoff obtained by the real-time inflow rates available on a daily basis for the past months are used to determine the flows through the turbines and the abandoned flows through the spillway.

4.2. Performance Criterion

Here, two quantitative indicators are used to test the feasibility of different methods, including average power generation (APG) and generation guarantee rate (GGR). APG shows the simulated generation benefit of the target method in the long run, while GGR measures the assurance degree of the simulated power output larger than the preset minimum. Generally, the method with a larger value of the two indexes has better performance. The definitions of the two indexes are given as below:

$$APG=\frac{1}{N}{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^M{\tilde{P}}_{i,j}{t}_{i,j}}}$$

(24)

$$GGR=\frac{1}{N\times M}{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^M{c}_{i,j}}},c_{i,j}=\{\begin{array}{l}1\mathrm{if}\left({\tilde{P}}_{i,j}\ge P_{i,j}^{\min }\right)\\ 0\mathrm{otherwise}\end{array}$$

(25)

where ${\tilde{P}}_{i,j}$ is the simulated power output of the target method at the jth period of the ith year, and $c_{i,j}$ is the intermediate variable.

4.3. Model Development

4.3.1. MLR Model Development

Because of its simplicity and easy implementation, the linear operation rule is used for the purpose of comparison. The total discharge is chosen as the dependent variable, while the initial water level and inflow per period are chosen as two independent variables that are related to the dependent variable. The linear operation rule for Hongjiadu reservoir per month is expressed in Equation (26). Then, the parameters involved in the linear operation rule of Hongjiadu reservoir are obtained by the MLR method mentioned in the Section 3.1.

Table 1. Some parameters involved in the linear operation rule of Hongjiadu reservoir.

Coefficient	Month
	1	3	5	7	9	11
a	740.9	966.6	−205.9	−7001.2	2698.6	6297.8
b	−0.54	−0.73	0.30	6.30	−2.34	−5.49
c	−0.04	0.02	0.58	0.50	0.73	0.84

shows the obtained coefficients for the linear operation rule per month. It can be observed that three coefficients in 12 months are totally different from each other, demonstrating the complexity of reservoir operation.

$$O_t=a+b\times Z_{t-1}^{}+c\times I_t,t=1,2,\cdots ,12$$

(26)

where $O_t$ is the total discharge at the tth month; $I_t$ is the total inflow at the tth month; $Z_{t-1}^{}$ is the initial water level at the tth month; and $a$, $b$, and $c$ are three different parameters.

4.3.2. ANN Model Development

Here, the three-layer ANN model based on the back-propagation training method is used to derive the operation rule of Hongjiadu reservoir. All the hidden nodes use the sigmoid activation function, while the linear function is used in the output layer. Given that the number of nodes in the hidden layer has an important effect on the performance of the ANN model, the trial and error strategy is used to choose the best network structure. The training process will be terminated when the root-mean-square error (RMSE) of all the testing samples reaches the minimum. Figure 4 shows the performances of the testing dataset with the change of hidden nodes from 3 to 18. It can be found that the model performance is affected by the hidden neurons. When there are seven nodes in the hidden layer, the best performance in the testing dataset will be achieved. Thus, the number of hidden nodes is set as seven for Hongjiadu reservoir.

4.3.3. ELM Model Development

Similar to the above ANN model, the sigmoid and linear activation functions are adopted in the hidden layer and output layer of the ELM model, respectively. The amount of hidden nodes is two times of the number of input layers, while the quantum-behaved particle swarm optimization (QPSO) [16,22] is employed to search for the appropriate network parameters (including the input-hidden weights and hidden bias). The number of individuals and iterations in QPSO is set as 100, while the RMSE value is chosen as the indicator to compare the model parameters. Figure 5 illustrates the simulation results of the ELM model for Hongjiadu reservoir in 10 runs. It can be found that the ELM model in the fourth run has the optimal performance in both generation guarantee rate and average power generation. Thus, the corresponding model is chosen to derive the operation rule of Hongjiadu reservoir.

4.3.4. SVM Model Development

In general, the kernel function plays an important role in enhancing the SVM performance. Based on the previous publications, the radial basis function is seen as one of the most commonly used kernel functions because it has better generalization ability compared with other kernel functions. Hence, the radial basis function (RBF) in Equation (27) is chosen as the Kernel function. Obviously, there are three parameters ($C,\gamma ,\epsilon$) in the RBF function, as used in the SVM model. In order to obtain satisfying performance, the above-mentioned QPSO method is used to optimize those parameters. Based on the simulation results, the optimal parameter combination in the SVM model is set as (10.768, 0.456, 0.784) for operation rule derivation in Hongjiadu reservoir.

$$K\left({\mathit{x}}_i,{\mathit{x}}_j\right)=\exp \left(-\gamma {\Vert {\mathit{x}}_i-{\mathit{x}}_j\Vert }^2\right)$$

(27)

where $\gamma$ is the kernel parameter to be optimized.

4.4. Comparison and Discussion

For the purpose of comparison, the traditional scheduling graph method (SGM) was chosen as the benchmark yardstick.

Table 2. Comparison of different methods in Hongjiadu reservoir. DP—dynamic programming; MLR—multiple linear regression; ANN—artificial neural network; ELM—extreme learning machine; SVM—support vector machine; SGM—scheduling graph method; GGR—generation guarantee rate; APG—average power generation.

Method	DP	SGM	MLR	ANN	ELM	SVM
APG (108 kWh)	23.38	21.03	21.36	22.41	23.11	22.71
Gap (%)	-	−10.05	−8.64	−4.15	−1.15	−2.87
GGR (%)	98.18	89.84	92.97	95.83	97.66	97.40
Gap (%)	-	−8.49	−5.31	−2.39	−0.53	−0.79

Note: Gap = (Method − DP)/DP × 100%; Gap denotes the gap between method and DP.

and Figure 6 show the detailed results of different approaches in Hongjiadu reservoir. It can be clearly observed that the dynamic programing method can obtain the best scheduling results in the deterministic case; four simulation-based methods (MLR, ANN, ELM, and SVM) are able to provide suboptimal results when compared with the dynamic programing method, but outperform SGM with respect to two statistical measures. On the other hand, as compared with SGM, MLR, ANN, and SVM, the ELM method can generate the best solution with approximately 9.00%, 7.57%, 3.03%, and 1.73% improvements in APG, respectively, while the generation assurance rate is improved by about 8.01%, 4.80%, 1.87%, and 0.27%, respectively. Hence, it can be concluded that the three AI-based methods can provide better results than the traditional SGM and MLR methods, and the operation rule derived by ELM has the best performance in the long-term simulations.

Figure 7 shows the average power output obtained by different methods for Hongjiadu reservoir. Figure 8 shows the water level of different methods for Hongjiadu reservoir in the testing samples. It can be found that the dynamic programming (DP) method can provide the most power generation in wet season but the least power generation in the dry season. This case indicates that in the ideal scheduling process, Hongjiadu reservoir can reduce the power generation in the dry season, and then use the abundant runoff to keep the reservoir operating at a high level, enhancing the operation efficiency of hydroelectric generators in the long run. Besides, the SGM method tends to smooth the power output in the long run because it fails to raise the water level in the wet season; the ELM method has a stronger capability than MLR, ANN, and SVM in mimicking the optimal scheduling process. Thus, the feasibilities of the solutions obtained by several methods are fully proven in this case.

Figure 9 draws the graphic models (outflow–inflow–water level) of four algorithms for Hongjiadu reservoir in August. The following conclusions for four methods can be deduced: when the water level is fixed, there is a positive relationship between power output and inflow; when the inflow is fixed, the reservoir tends to increase the power output with the increase of water level. On the other hand, there are obvious differences in the graphic models, while the gap among the average annual power generation of four methods is relatively small, demonstrating the equivalence for different combinations of parameters in the hydropower reservoir operation rule. Thus, operators should take the actual working condition of the hydropower reservoir into consideration when making the scheduling plan for production guidance.

From the above analysis, it can be clearly observed that the dynamic programming has the best performance, while the three artificial intelligence algorithms (ANN, ELM, and SVM) can provide better simulation results than SGM and MLR. The dynamic programing method divides the complicated multistage reservoir operation optimization problem into a series of relatively simple subproblems to be solved sequentially, and then seeks for the global optimal solution in the discrete state space, providing the best scheduling results for simulation [47,48,49]. In general, for the reservoir operation rule per month, there is often a strong nonlinearity between the independent variables (like water level and inflow) and dependent variables (like outflow). The conventional MLR method based on the simulation and optimization strategy can only handle conventional linearity, rather than the inherent nonlinear relationship in this problem, leading to lower generation benefit of the hydropower reservoir. The SGM approach based on the historical data and engineering experience cannot well consider the dynamic variation of reservoir runoff caused by climate change and human activities, reducing the overall operational efficiency of Hongjiadu reservoir. Three artificial intelligence algorithms get the utmost out of the mapping functions to map the training samples into the high-dimensional feature space, and then carefully choose the appropriate optimization strategies to find the solution that minimizes the total training error. As a result, three artificial intelligence algorithms have some unique merits in comparison with the SGM and MLR methods, including self-learning ability (training network parameters to simulate the complex nonlinear input–output relationship), generalization ability (possessing satisfying performance for new data samples), and fault-tolerant ability (behaving well for the partially damaged system), producing better performances than the two traditional methods. On the other hand, it seems that the ELM method is capable of obtaining the best performance among all the methods used to derive the reservoir operation rule. The difference in the adopted optimization principle per method is the key point leading to the fact that the performance of ELM is superior to both SVM and ANN. Specifically speaking, with a strong generalized ability for a variety of feature mappings, ELM is able to approximate any continuous functions by determining the global optima of the training samples [50], while the QPSO optimizer can effectively enhance the network compactness by carefully choosing the necessary parameters [51]; the traditional gradient-based ANN training method tends to fail into local optima with a relatively long learning time; and the SVM method can only provide the suboptimal solution with a higher computational complexity and more compact constraints. In addition, it should be pointed out that the performances of the three artificial intelligence methods may be different along with the change in the problem characteristics or research objects. To sum up, it can be concluded that in the reservoir operation rule derivation field, future research efforts can be directed to those artificial intelligence methods with promising simulation ability.

5. Conclusions

This study investigates the performances of four effective methods in deriving the operation rule of a hydropower reservoir, including MLR, ANN, ELM, and SVM. For the purpose of comparison, the conventional SGM approach was chosen as the benchmark yardstick. The historical streamflow data of Hongjiadu reservoir optimized by the dynamic programming method are adopted to develop those models. Two indexes are adopted to evaluate the performance of different methods, including average power generation and generation guarantee rate. The results indicate that three artificial intelligence algorithms (ANN, ELM, and SVM) provide better simulation performances than SGM and MLR. Therefore, the results show that the artificial intelligence methods are promising tools in deriving the operation rule of a hydropower reservoir. To be mentioned, the performances of ANN, ELM, and SVM vary with the change of parameter combination, and it is of great importance to develop effective tools to choose appropriate model parameters.

Besides, the amount of reservoir operation data will increase with the passage of time, which directly affects the major decision (monthly outflow that is equivalent to the abandoned spillage and turbine discharge) and the core function (like generation benefit) involved in the operation rules. Thus, the update frequency of reservoir operating rules can be set to a monthly basis in practice. On the other hand, in many parts of the world, climate change is creating non-stationary conditions in the flow-rate and streams. These non-stationary conditions show up as time trends in the annual flow-rate of streams and change over time in the pattern of month-by-month contributions to the annual streamflow [52,53,54,55], which will lead to certain significant impacts on reservoir operation. Because of the limited time and energy, the check for time trends in the inflow data of the survey region was not made, but this work is necessary for any future application of the methods presented here for operating-rule selection. Thus, in the future, we will deepen the research on the operation optimization of a hydropower reservoir in the changing environment.