A Gradient-Based Cuckoo Search Algorithm for a Reservoir-Generation Scheduling Problem

In this paper, a gradient-based cuckoo search algorithm (GCS) is proposed to solve a reservoir-scheduling problem. The classical cuckoo search (CS) is first improved by a self-adaptive solution-generation technique, together with a differential strategy for Lévy flight. This improved CS is then employed to solve the reservoir-scheduling problem, and a two-way solution-correction strategy is introduced to handle variants’ constraints. Moreover, a gradient-based search strategy is developed to improve the search speed and accuracy. Finally, the proposed GCS is used to obtain optimal schemes for cascade reservoirs in the Jinsha River, China. Results show that the mean and standard deviation of power generation obtained by GCS are much better than other methods. The converging speed of GCS is also faster. In the optimal results, the fluctuation of the water level obtained by GCS is small, indicating the proposed GCS’s effectiveness in dealing with reservoir-scheduling problems.


Introduction
The penetration of renewable power generation has increased substantially in recent years [1].The use of renewable energy such as wind power and hydropower can reduce greenhouse gas emissions from fossil fuels.Wind power is one of the fastest-growing sources, however it is uncertain and cannot be scheduled due to its intrinsic dependence on varying weather conditions over many years [2,3].Solar power is less uncertain, but has higher variability compared to wind power [4].Unlike wind energy, hydropower can be scheduled and has already played an important role in electricity supply all over the world.The long-term hydropower-generation scheduling (LHGS) of cascade reservoirs is of great importance to improving power grid stability.A suitable scheduling scheme can effectively increase the economic benefit of cascade reservoirs.The purpose of LHGS is to determine the water release of all reservoirs in a time period in order to maximize the benefit [5].
Many studies have been carried out [6][7][8] indicating that the LHGS is a non-linear and non-convex problem with very complicate constraints including water balancing, hydraulic connection, water-level limitations, water-release limitations and output capacity limitations [9].In order to solve this problem, classical studies have taken many kinds of approaches, including dynamic programming (DP) [10,11], linear programming (LP) [12,13], the progressive optimality algorithm (POA) [14,15], and discrete differential dynamic programming (DDDP) [16,17].However, these methods have their own shortcomings in dealing with the problems of large-scale cascade reservoirs [18].
In recent years, meta-heuristic algorithms have been popular in the search for a global optimum in non-convex problems [19].Metaheuristics and their applications have wide application in artificial intelligence, e.g., water-distribution systems [20], transit network design problems [21], reservoir-operation optimization [22], and so forth.Many metaheuristics have included some form of stochastic optimization, so the searching efficiency is affected by the set of random variables generated [23].By searching over a large numbers of feasible solutions, metaheuristics can often find good solutions near to the optimum solution with less computational effort [24].
Cuckoo search (CS) is one kind of metaheuristics developed by Xin-she Yang and Suash Deb in 2009 [25].It was inspired by the behavior of some cuckoo species by laying eggs in the nests of other host birds.Compared to other artificial intelligence algorithms such as the genetic algorithm (GA) [26] and particle swarm optimization (PSO) [27], CS has the characteristics of higher solution quality, shorter computational time, and higher success rates [28].It has been applied to various optimization problems.For example, Maiya Din et al. introduced it for the analysis of linear feedback shift register-based cryptosystems [29], Bhateja et al. used CS for the cryptanalysis of vigenere [30], Naik et al. used CS for intrinsic discriminant analysis-based face recognition [31].
Although CS has shown its advantages and has wide application in many areas, it still has some drawbacks.Proper computational time cannot be guaranteed in large-scale systems [32].In this paper, some improvements are proposed to the original CS using four strategies: (1) a dynamic parameter adjustment of the parameter; (2) a boundary value perturbation strategy for boundary value adjustment; (3) a differential strategy for Lévy flight; (4) a different way to update solutions.When applying CS to the LHGS problem, we find that the converge speed and the results are not satisfactory.What's more, solutions generated by a random search have the characteristic of large fluctuations in water level.Thus, a gradient strategy is designed especially for the LHGS problem.This gradient strategy improves the local search ability of the algorithm by fine-tuning the water level process.In case studies, comparisons of benchmark function tests among different algorithms has been carried out.Results show that the improved cuckoo search (ICS) performs much better than the improved harmony search (IHS) and CS.Finally, a case study of the Jinsha River is put forward to show the application of the LHGS model.The advantage of the proposed GCS is very obvious: it can effectively obtain satisfactory results in all wet years, normal years and dry years.
This paper is organized as follows: Section 2 describes the objective function and constraints of LHGS model; Section 3 gives the details of improved CS; Section 4 presents the gradient-based CS algorithm for LHGS; Section 5 is the experiments and the case study in the Jinsha River; and Section 6 is the summary of this paper.

Reservoir-Scheduling Problem
The main purpose of LHGS is to improve the power generation of cascaded reservoirs.The objective function can be listed as follows.

Long-Term Hydropower-Generation Scheduling (LHGS) Model
The LHGS model is used to find a set of water releases or storage volumes that maximizes the generating benefit for cascade hydropower system.The objective formula of hydropower optimization problem is given as follows: where E is the total energy production of the cascade hydropower system; T is the count of periods; N is the number of reservoirs; k i is the output coefficient of reservoir i; Q i,t is the generation flow which outflows through hydropower units of the reservoir i in time t; H i,t is the net water head of hydropower reservoir i in time t; and ∆t is the length of time interval.

Constraints
To complete the scheduling model, the constraints of the scheduling problem are defined as: (1) Hydraulic Connection I i,t is the inflow of reservoir i in time t; O i−1,t is the outflow of reservoir i − 1 in time t; R i,t is the interval inflow between reservoir i − 1 and i; S i−1,t is spillage of the upstream reservoir i − 1; Q i−1,t is the generation flow of the upstream reservoir i − 1.
(2) Water-Balance Constraint V i,t is the reservoir storage of reservoir i in time t.(3) Water-Level Constraints Z min i,t and Z max i,t are, respectively, the lower and upper bounds of the water level; is the limitation of water-level variation.(4) Outflow Constraint O min i,t and O max i,t are, respectively, the minimum and maximum outflows of reservoir i. (5) Output Constraint N min i,t and N max i,t are, respectively, the minimum and maximum power output of reservoir i. (6) Boundary Condition Z i,start and Z i,end are, respectively, the initial and terminal water levels of reservoir i.

Basic Cuckoo Search Algorithm
Cuckoo search (CS) is a meta-heuristic algorithm inspired by the obligate brood parasitism of some cuckoo species [25].It is a population-based evolutionary algorithm and can be applied for various optimization problems.The procedures of CS are explained as follows: 1.
Each cuckoo lays one egg at a time, and dumps its egg in randomly chosen nest; 2.
The best nests with a high quality of eggs will carry over to the next generation; 3.
The number of available host nests is fixed, and the egg laid by a cuckoo is discovered by the host bird with a probability p a .Discovering operates on some set of the worst nests, and discovered solutions are dumped from further calculations.
For a minimum function, the basic steps of the CS are summarized as the pseudo code in Algorithm 1:

Algorithm 1. Cuckoo search via Lévy flights
Objective function f (x), x = (x 1 , ..., x d ) T Generate initial population of n host nests x i (i =

Improvement for Cuckoo Search Algorithm
The basic cuckoo search algorithm (CS) is capable of finding the optimum function, but after several iterations, we found that it still needs improvements.In this section, three strategies are presented to improve the CS.

Dynamic Parameter Adjustment Strategy
In the CS algorithm, the abandonment of the nest is determined by its abandon probability p a .If p a is large, this nest is considered as a bad solution and would be replaced with new randomly generated solutions.The converging speed of the algorithm will decline due to the replacement, but the population diversity can be improved.If p a is small, the characteristics of the population can be reserved and it can benefit the convergence.According to this feature, a dynamic adjustment strategy is introduced to decide p a showing as Equation (9).With this strategy, there would be better global search capabilities in the early stage evolution, and the convergence becomes faster in the later stage: where CE is the current evaluation times; NE is the maximum evaluation times; and p s a and p e a are start and end p a , respectively.

A Boundary Value Perturbation Strategy
When a new solution is generated, the values of some dimensions may be no longer in the proper range.The common method is to adjust it to the boundary value, which may result in unexpected accumulation of solutions on the border.Thus, a boundary value perturbation strategy is carried out, as shown in Equation ( 10): where mod(a, b) is modulo operation to find the remainder after division of a by b; and UB d and LB d are the upper and lower boundary limits, respectively.

Differential Strategy for Lévy Flight
In CS, an important problem is the implementation of Lévy flights, as shown in the original paper [25]: Unfortunately, the original paper only described the basic idea of Lévy flights, and the details of its implementation were not given.Later studies had different interpretations of Lévy flight such as [33,34].In this paper, the strategy used for Lévy flight is given as: where i and j are the numbers of randomly chosen nests; sl is the step size parameter; Levy(u,c) is sample value from the Lévy distribution, and its probability density function is:

Solution Updates Strategy Changes
The CS generates new solutions by Lévy flights and compares it with another randomly chosen nest.However, when replacing the randomly chosen solution, the source solution and the target solution are not relevant, so the useful information from the target solution is missing.Thus, if the new generated solution is better, instead of replacing another randomly chosen nest, we replace the source nest that is used for performing Lévy flights.
Additionally, in the proposed algorithm, the guide information from the best solution or current best solution is not used, and we should not find the optimal solution in each iteration.

Implementation of Improved Cuckoo Search (ICS)
With the improvements above, the improved CS can be described as follows: Step 1 Initialize the algorithm parameters This step specifies the desired parameters, including size of population (ns), abandon probability (p a ), disturbance weight (w), step size (sl), and Lévy distribution parameters u and c.

Step 2 Initialize the population
In this step, new random nests (x 1 , ..., x ns ), whose total number is ns, are generated as in Equation ( 14): Then each vector will be evaluated by the objective function and stored in the population.
Step 3 Generate a new solution by Lévy flights Here a source nest x i and target nest x j from the population are randomly chosen.A new cuckoo from nest i flight to nest j obeys the features of Lévy flight.Then a new solution is generated from Equation (12).
If the new generated solution is out of proper range, we would adjust the solution from Equation (10).
Step 4 Update the host nest Evaluate the fitness of the new solution f (x i ) and compare it with the fitness of source nest f (x i ).If the new solution is better, replace the source nest with the new solution.
Step 5 Abandon the worst nest If the random value is smaller than p a , a new solution is generated randomly and it will replace the worst nest.
Step 6 Check the stop criterion Repeat Step 3 to Step 5 until the termination criterion is satisfied.In this paper, the termination criterion is the maximum number of evaluation times.
The pseudo-code of the ICS algorithm is presented in Algorithm 2.

Algorithm 2. Improved cuckoo search
Objective function f (x), x = (x 1 , ..., x d ) T Initialize default parameters Generate initial population of n host nests x i (i = 1, 2, ..., n) While (t < MaxEvaluation) or (stop criterion) Select two solution x i , x j from host nests randomly For d=1,...,D do Init the worst nest x worst End if End while

Gradient Cuckoo Search for Reservoir Scheduling
Generally, when the heuristic algorithm is applied on the reservoir-scheduling problem, the process of water levels is selected as the decision variable, as shown in Equation ( 15): where N is the number of reservoirs; and T is the number of time intervals.

Constraints Handling
Cascade reservoir operation is a complicated optimization problem with varieties of constraints.According to ICS, the initial population is generated following Equation ( 16): where x min i and x max i are, respectively, the lower and upper boundaries limit of reservoir i; and rand is a random number between 0 and 1.However, there is a high probability that the randomly generated solutions do not satisfy all the constraints displayed in Section 2.2.In the initialization stage, if the generated solution is not feasible, it needs to be replaced by a new feasible solution.
In the random search stage, the newly generated unfeasible solution cannot be discarded.It is adjusted to the feasible range as follows: The feasible range of water level x i+1 is defined as: x min i+1 = max where Q min t and Q max t are, respectively, the minimum and maximum outflows of the reservoir; Q Pmin t is the minimum outflow for a guaranteed output; z min and z max are minimum and maximum water levels; ∆z is the limitation of the water-level change; and Z(V) and V(Z) are the relationship between the water level and the storage.
In the adjustment of the unfeasible solutions, a two-way solution correction strategy is employed.First, the water level is adjusted to the feasible range from period 1 to T. If the correction fails at some unpredictable period, then the water level is adjusted from period T to the breakpoint.If the correction is still not successful, the solution would be considered as an infeasible solution.

Gradient-Based Search Strategy
In the random search phase, new solutions are randomly generated from the heuristics algorithms.However, after many numerical experiments, it is found that the performances of the heuristic algorithms fail to meet the requirements.On the one hand, the variance of the calculation results is large, indicating that the algorithms are not very stable.On the other hand, the converging speed of the algorithm is quite slow when dealing with the complex cascade reservoirs problem.Hence in this section, a gradient-based search strategy is designed especially for LHGS.
For a random generated solution, the water level is adjusted by a small gradient ∆l i,t as shown in Figure 1, which would result in the power generation of the entire cascade reservoir changing.If we change the end water level of reservoir i by ∆l at time period t, the power generation of reservoir i at time period t is: where V i (Z) is the fitting function of water level and storage curve of reservoir i; and Zd i (Q) is the fitting function of the downstream water level and outflow curve of reservoir i.
Then the power generation of reservoir i at the next time period t + 1 can be given by: ) When the outflow of reservoir i is changed and the water level of the downstream reservoirs remains the same, the inflow of all reservoirs in the downstream reservoir would be changed by the outflow of reservoir i.For one of the reservoirs downstream, the power generation at time period t is: The power generation of the next reservoir i + 1 at time period t + 1 is: ∆H j,t+1 = Zd j Q j,t+1 − Zd j Q j,t+1 + ∆Q j,t+1 ∆P j,t+1 = k j Q j,t+1 + ∆Q j,t+1 H j,t+1 + ∆H j,t+1 − k j Q j,t+1 H j,t+1 (24) For the solution adjustment, we need to adjust this to the better direction.Instead of a time-consuming fitness calculation of the incremental power generation ∆E, we only need to determine whether the incremental power generation ∆E is positive.If ∆E > 0, the adjustment will be made to give a better solution.Otherwise the previous solution is kept without change.
In this paper, we determine whether ∆E is positive from the calculation of the partial derivative of power generation E with respect to the gradient ∆l, as given below.For the reservoir i at time period t shown in Equation ( 25), the partial derivative of power output is: For the reservoir i at time period t + 1 shown in Equation ( 26), the partial derivative of power output is: For the reservoir downstream at time period t shown in Equation ( 27), the partial derivative of power output is: ∂l + H j,t ∂Q j,t ∂l (27) For the reservoir downstream at time period t + 1 shown in Equation ( 28), the partial derivative of power output is: ∂Q j,t+1 ∂l (28) According to the formulas above, the partial derivative of power generation E with respect to the gradient ∆l is: If ∂E ∂l > 0, adjust the water level by ∆l, otherwise adjustment is rejected.For the cascade system, the flowchart of the gradient-based search strategy is shown in Figure 2.

Implementation of Gradient-Based Cuckoo Search (GCS) for Long-Term Hydropower Generation Scheduling (LHGS)
According to the description above, the steps to solve long-term scheduling problem using the cuckoo search algorithm with the gradient-based search strategy (GCS) are shown in Figure 3: Step 1: Randomly generate feasible initial solutions.
Step 2: Evaluate fitness of the solutions.
Step 3: Generate new solutions by Lévy flights.
Step 4: If the new solution is infeasible, adjust it by two-way solution correction strategy.
Step 5: Adjust the new solution by gradient-based search strategy.
Step 6: Update the host nest.
Step 7: Abandon the worst nest.
Step 8: Repeat Steps 3 to 7 until the stop criteria is reached.

Study Area
The Jinsha River is the upper reach of the Yangtze River in south-west China.It flows through Qinghai, Sichuan and Yunnan provinces.The Jinsha River is rich in water resources, and is being heavily developed for hydroelectric power.In this case study, we discuss four hydropower stations with total installed capacity of 42,960 MW on the river, including Wudongde, Baihetan, Xiluodu and Xiangjiaba.The locations of these hydropower stations are signed in Figure 4, and the major parameters of these hydropower stations are listed in Table 1.The aim of long-term hydropower generation scheduling is to determine the water release process of all hydropower stations at each period.The total power generation of different schemes will be different, thus we need a suitable method to solve this problem.In this case, we do the scheduling simulation of the cascade hydropower stations using the proposed GCS algorithm.
The topology of the four reservoirs in the Jinsha River is shown in Figure 5.The historical observed streamflow of Pingshan hydrological station is used as the input of the LHGS model.

Benchmark Function Tests
To evaluate the performance of the proposed ICS, we chose several famous typical benchmark functions, shown in Table 2. Results obtained by proposed ICS algorithms are compared with that simulated by improved harmony search (IHS) [35], the Kbest gravitational search algorithm (KGSA) [36] and basic CS.Of the functions in Table 2

Formula Search Domain Optimum
Ackley Dimensions of the benchmark functions are set to 10.The maximum evaluation count is set to 100,000.The default parameters of ICS are ns = 30, p s a = 0.3, p e a = 0.1, sl = 0.01, u = 0, c = 1.5.The parameters of IHS and GSA are set to the default values in [35,36].All of the experiments are performed using a computer with 2.7 GHz Intel i7-4800MQ with 8 GB RAM.The source code is compiled with Java SE8.
Table 3 shows the results of the algorithms including means and standard deviation based on 100 independent tests for each benchmark function.As demonstrated in Table 3, ICS performs superior to IHS, KGSA and CS in most functions except the shifted Rosenbrock function, shifted Happy Cat function, and rotated and shifted Ackley function.Especially on the Sphere, Bent Cigar and Discus, the means and standard deviations of ICS are extremely close to the optimal solutions, much better than other algorithms.While on Rosenbrock function, shifted Happy Cat function, rotated and shifted Ackley function, KGSA performs slightly better than ICS.Overall, ICS can find a satisfactory solution, meaning that the improvement is effective.

Reservoir Scheduling in Wet Years
To evaluate the gradient strategy for reservoir scheduling, the optimal scheduling scheme of the cascade hydropower stations were obtained by using GCS and compared with those obtained by IHS, KGSA and CS.The parameter of GCS are ns = 40, p s a = 0.3, p e a = 0.1, sl = 0.01, u = 0, c = 1.5, and the maximum evaluation times is set to 12,000.The initial and end water levels of the reservoirs are set to their normal levels as shown in Table 1.Each month is divided into three periods of approximately 10 days each, resulting in 36 time intervals in a year.Each method is run 100 times, independently.
To evaluate the performance of GCS on reservoir scheduling in wet years, the streamflow of Pingshan station in Figure 6 was chosen as the input of the LHGS model.As shown in Equation (15), the water levels of the reservoirs at different time intervals are the optimization parameters.Because the water level at the 36th time interval is set to the normal water level, the optimization parameters of each reservoir are 35 and the number of reservoirs is 4, meaning that it is a 140-dimensional problem.Equation ( 1) is the fitness function of LHGS, and the larger power generation means a better scheduling solution.Table 4 clearly shows that GCS is superior to the other three methods.The average power generation obtained by GCS is 232.2, 223.7, 226.3 and 248.4 billion kWh in the four wet years.Compared with IHS, KGSA and CS, the average increases are 4.2%, 8.6% and 2.6%, respectively.It is also noted that the standard deviation obtained by GCS is at least 10 times smaller than that obtained by the other three methods, thus indicating GCS is more stable.Figure 7 shows the convergence process of the four algorithms in the case of the year 1987 as an example.It can be seen from Figure 7 that GCS has the fastest convergence speed, IHS and ICS have not converged yet, while KGSA has fallen into a local optimum.Figure 8 shows the optimal results of the four hydropower stations in the year 1987.As shown in Figure 6, the streamflow in 1987 increased significantly in July, August and September.The water levels also dropped below the flood limit water level during the flooding season.In the non-flooding season, the water levels remained at the normal level from November to March.The fluctuation of the water level obtained by GCS is small, indicating that it is an effective way to solve LHGS.

Reservoir Scheduling in Normal Years
In this section, we evaluate the performance of GCS in normal years.Streamflows of the Pingshan station in Figure 9 were chosen as the input of the LHGS model.The parameters and other conditions were the same as those in Section 5.3.
As shown in Table 5, GCS gains more power generation than the other three methods.The average power generation obtained by GCS is 210.5, 220.5, 218.1 and 215.9 billion kWh in the four normal years.Compared with IHS, KGSA and CS, the average increases are 5.8%, 8.6% and 3.3%, respectively.The standard deviation obtained by GCS is also smaller than that obtained by the other three methods.Figure 10 shows the convergence process of the four algorithms in the case of the year 1997 as an example.It can be seen from Figure 10 that GCS has the fastest convergence speed.As in Section 5.3, KGSA does not perform well compared with the other methods.

Reservoir Scheduling in Dry Years
In this section, we evaluate the performance of GCS in dry years.The streamflows of Pingshan station in Figure 11 were chosen as the input of the LHGS model.The parameters and other conditions were also the same as those in Section 5.3.As in Sections 5.3 and 5.4, GCS performed best in these four methods.The mean values obtained by KGSA are the smallest, while the standard deviations obtained by IHS are the largest.As shown in Table 6, the average power generation obtained by GCS is 199.2, 201.9, 194.4 and 197.7 billion kWh in the four normal years.Compared with IHS, KGSA and CS, the average increases are 5.0%, 7.7% and 3.2%, respectively.Figure 12 also shows that the convergence speed of GCS is the fastest.From Sections 5.3-5.5, it is clear that GCS can effectively obtain satisfactory results in different streamflow input scenarios.

Conclusions
The long-term hydropower generation of cascade reservoirs is one of the active research areas in water resources management.According to the operating characteristics of the reservoirs, we propose a gradient strategy to improve the performance of the search algorithm when dealing with the LHGS problem.Moreover, the original CS algorithm is adapted with four improvements: a dynamic parameter adjustment of the parameter, a boundary value perturbation strategy for boundary value adjustment, a differential strategy for Lévy flight, and a different way to update solutions.
In the experimental work, the performance of the improvement is shown by several benchmark tests.Results show that the improved CS achieves better results than the other three methods.Finally, the improved CS is hybridized with the gradient strategy and is applied to the LHGS problem in the Jinsha River.The case study shows that the simulation results obtained by GCS are more reliable and stable than those obtained by IHS, KGSA and CS.Compared with IHS, KGSA and CS, the average increases are 4.2%, 8.6% and 2.6% in wet years, 5.8%, 8.6% and 3.3% in normal years, and 5.0%, 7.7% and 3.2% in dry years.The standard deviation obtained by GCS is smaller than that obtained by the other three methods.Overall, GCS can effectively obtain satisfactory results in different streamflow input scenarios.
However, simulation results show that ICS still has drawbacks such as local convergence when dealing with some functions.The uncertainty in reservoir generation scheduling is also not considered yet.Further research is underway to solve these problems.

Figure 1 .
Figure 1.Adjustment in water level by a small gradient.

Figure 5 .
Figure 5. Topology of reservoirs in the Jinsha River.
, the Ackley function, Griewank function, Rastrigin function, Rosenbrock function and Happy Cat function are multimodal functions.The Sphere function, Bent Cigar function, Discus function and Schwefel 2.22 function are unimodal functions.These functions will be tested under sophisticated features such as shifted and rotated features.

Figure 6 .
Figure 6.Streamflow of Pingshan station in wet years.

Figure 9 .
Figure 9. Streamflow of Pingshan station in normal years.

Figure 10 .
Figure 10.Convergence process of the algorithms in the year 1987.

Figure 12 .
Figure 12.Convergence process of the algorithms in year 1977.

Table 1 .
Main parameters of the four cascade hydropower stations in the Jinsha River.

Table 3 .
Errors of test results of the algorithms.
* PS: The number in bold represent the minimum value.

Table 4 .
Comparison of power generation (10 8 kWh) in wet years.
Figure 7. Convergence process of the algorithms in the year 1987.

Table 5 .
Comparison of power generation (10 8 kWh) in normal years.

Table 6 .
Comparison of power generation (10 8 kWh) in dry years.