# NN-Based Implicit Stochastic Optimization of Multi-Reservoir Systems Management

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Methods

#### 2.1. Problem Definition

- Deterministic solution of many different open loop-problems each corresponding to one of the scenarios;
- Fitting of a suffcienctly general class of functions to reproduce as much as possible the behaviour of the open-loop solution;
- Testing of the identified rules with different scenarios on the complete system by new simulations.

#### 2.2. Deterministic Open-Loop Problem

- Allow solving fairly quickly nonlinear problems with lots of dimensions. They are faster than methods that discretize the decisions space and perform an exhaustive search;
- Guarantee a better performance than other random optimization methods, requiring an acceptable increasing of the computational effort;
- Behave better than gradient-based methods when the objective function has lots of local optima.

#### 2.3. Policy Identification

- radial basis functions, since it has been recently empirically proved that they can be more appropriate in some contexts similar to the one considered here [19].

#### 2.4. Rule Testing

## 3. Case Study Description

#### 3.1. The Nile River basin

#### 3.2. Model Formulation

#### 3.2.1. System Variables

#### (1) State Variables

#### (2) Input Variables

- Water demand in the respective five agricultural district, whose values are known at time t when the manager decides the control (green arrows in Figure 3):$${\mathbf{w}}_{t}=\left|w{d}_{t}^{\mathrm{Tana}},w{d}_{t}^{\mathrm{Ros}},w{d}_{t}^{\mathrm{KeG}},w{d}_{t}^{\mathrm{Nas}},w{d}_{t}^{\mathrm{Egypt}}\right|$$

- Random flows, whose subscripts t indicate that their values are known at the end of the time step $\left[t-1,t\right)$.$${\mathbf{i}}_{t}=\left|{n}_{t}^{\mathrm{Tana}},{e}_{t}^{\mathrm{Ros}},{e}_{t}^{\mathrm{KeG}},{e}_{t}^{\mathrm{Nas}},{l}_{t}^{\mathrm{Nas}},{i}_{t}^{\mathrm{Kes}},{i}_{t}^{\mathrm{Bor}},{i}_{t}^{\mathrm{Din}},{i}_{t}^{\mathrm{Rah}},{i}_{t}^{\mathrm{WN}},{i}_{t}^{\mathrm{u},\mathrm{At}},{i}_{t}^{\mathrm{l},\mathrm{At}}\right|$$
- -
- ${n}_{t}^{\mathrm{Tana}}$ is the net inflow to Lake Tana;
- -
- ${e}_{t}^{\mathrm{Ros}}$, ${e}_{t}^{\mathrm{KeG}}$ and ${e}_{t}^{\mathrm{Nas}}$ are the evaporation rates of Roseires, Girba and Nasser reservoirs;
- -
- ${l}_{t}^{\mathrm{Nas}}$ is the seepage and bank loss at Nasser;
- -
- ${i}_{t}^{\mathrm{Kes}}$, ${i}_{t}^{\mathrm{Bor}}$, ${i}_{t}^{\mathrm{Din}}$, ${i}_{t}^{\mathrm{Rah}}$, ${i}_{t}^{\mathrm{u},\mathrm{At}}$ and ${i}_{t}^{\mathrm{l},\mathrm{At}}$ are the inflows to Kessie, Border, Dinder, Rahad, upper and lower Atbara;
- -
- ${i}_{t}^{\mathrm{WN}}$ represents the White Nile discharge at Khartoum.

#### (3) Decision Variables

#### (4) Internal Variables

- the natural outflow from Lake Tana at Bahir Dahr, ${q}_{t}^{\mathrm{Bahir}}$;
- the discharges downstream of Border, ${q}_{t}^{\mathrm{Bor}}$, and Dongola, ${q}_{t}^{\mathrm{Don}}$, which are the inflows to Roseires reservoir and Lake Nasser;
- others discharges at some significant points of the network such as Tamaniat (${q}_{t}^{\mathrm{Tam}}$) and Hassanab (${q}_{t}^{\mathrm{Has}}$) on the Main Nile, the discharge of the Blue Nile at Khartoum (${q}_{t}^{\mathrm{BN}}$) and of the river Atbara just before the junction with the Main Nile (${q}_{t}^{\mathrm{l},\mathrm{At}}$);
- the hydraulic head of the power plants, computed as the difference, if positive, between the water level and the tailwater: ${H}_{t}^{\mathrm{Tana}}$, ${H}_{t}^{\mathrm{Bahir}}$, ${H}_{t}^{\mathrm{Ros}}$, ${H}_{t}^{\mathrm{KeG}}$ and ${H}_{t}^{\mathrm{Nas}}$.

#### 3.2.2. Operating Objectives

#### (1) Hydropower Production

- $\eta $ is the overall efficiency of the power plant;
- $\gamma $ is the specific weight of water, equal to $9810\phantom{\rule{0.277778em}{0ex}}\frac{N}{{m}^{3}}$;
- ${H}_{t}^{\mathrm{i}}$ is the hydraulic head, expressed in m;
- the fourth factor represents the flow rate that will be used for power production, expressed in $\left[{\mathrm{km}}^{3}/\mathrm{month}\right]$, computed as the minimum between the release from the reservoir and the maximum flow that can go through the turbines (${mtf}^{\mathrm{i}}$);
- $\psi $ is a coefficient of dimensional conversion, which contains the duration of the time step, 1 month, and whose value is $\frac{1\mathrm{month}}{3600\phantom{\rule{0.277778em}{0ex}}\mathrm{s}/\mathrm{h}}$ in order to obtain an energy value expressed in $\left[\mathrm{GWh}\right]$.

#### (2) Water Supply to Agricultural Districts

#### (3) Objective Space Definition

#### 3.2.3. Constraints

#### (1) White Nile

#### (2) Blue Nile

#### (3) Atbara

#### (4) Main Nile

#### 3.3. Deterministic Open-Loop Problem

## 4. Results

#### 4.1. Control Laws

#### 4.2. System Performances

- a trivial policy, which releases the maximum feasible flow at each time step;
- a closed-loop linear policy, which takes as input the same variables of the neural policy. Since we considered four different cases depending on the available information (DI, DF, CI, and CF), four different linear policies have been identified;
- the open-loop control sequence computed with the GA. The performance obtained using this control sequence can be considered as a utopia point, in the sense that it is the target to which we wish to get close. This point represents the best performance which is possible to obtain when all the future inflows are deterministically known.

## 5. Concluding Remarks

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## Abbreviations

CF | Centralized-full hydrological |

CI | Centralized-incomplete hydrological |

DEM | Digital elevation model |

DF | Decentralized-full hydrological |

DI | Decentralized-incomplete hydrological |

ESO | Explicit stochastic optimization |

GA | Genetic Algorithm |

ISO | Imolicit stochastic optimization |

Nile-DST | Nile decision support tool |

NN | Neural network |

Probability density function | |

ReLU | Rectified Linear Unit |

SDP | Stochastic dynamic programming |

## References

- Becker, L.; Yeh, W.W.G. Optimization of real time operation of a multiple-reservoir system. Water Resour. Res.
**1974**, 10, 1107–1112. [Google Scholar] - Murray, D.M.; Yakowitz, S.J. Constrained differential dynamic programming and its application to multireservoir control. Water Resour. Res.
**1979**, 15, 1017–1027. [Google Scholar] - Turgeon, A. Optimal operation of multireservoir power systems with stochastic inflows. Water Resour. Res.
**1980**, 16, 275–283. [Google Scholar] [CrossRef] - Soncini-Sessa, R.; Weber, E.; Castelletti, A. Integrated and Participatory Water Resources Management-Theory; Elsevier Science: Amsterdam, The Netherlands, 2007. [Google Scholar]
- Bertsekas, D.P.; Tsitsiklis, J.N. Neuro-dynamic programming: An overview. In Proceedings of the 34th IEEE Conference on Decision and Control, New Orleans, LA, USA, 13–15 December 1995; Volume 1, pp. 560–564. [Google Scholar]
- Cervellera, C.; Chen, V.C.; Wen, A. Optimization of a large-scale water reservoir network by stochastic dynamic programming with efficient state space discretization. Eur. J. Oper. Res.
**2006**, 171, 1139–1151. [Google Scholar] [CrossRef] - Castelletti, A.; De Rigo, D.; Rizzoli, A.E.; Soncini-Sessa, R.; Weber, E. Neuro-dynamic programming for designing water reservoir network management policies. Control Eng. Pract.
**2007**, 15, 1031–1038. [Google Scholar] [CrossRef] - Nandalal, K.; Bogardi, J.J. Dynamic Programming Based Operation of Reservoirs: Applicability and Limits; Cambridge University Press: Cambridge, UK, 2007. [Google Scholar]
- Xu, X.; Bin, L.; Pan, C.; Ding, A.; Chen, D. Optimal reoperation of multi-reservoirs for integrated watershed management with multiple benefits. Water
**2014**, 6, 796–812. [Google Scholar] [CrossRef] - Giuliani, M.; Li, Y.; Cominola, A.; Denaro, S.; Mason, E.; Castelletti, A. A Matlab toolbox for designing Multi-Objective Optimal Operations of water reservoir systems. Environ. Model. Softw.
**2016**, 85, 293–298. [Google Scholar] [CrossRef] - Jiang, Z.; Qin, H.; Wu, W.; Qiao, Y. Studying operation rules of cascade reservoirs based on multi-dimensional dynamics programming. Water
**2017**, 10, 20. [Google Scholar] - Jiang, Z.; Qin, H.; Ji, C.; Feng, Z.; Zhou, J. Two dimension reduction methods for multi-dimensional dynamic programming and its application in cascade reservoirs operation optimization. Water
**2017**, 9, 634. [Google Scholar] [CrossRef] - Liu, P.; Li, L.; Chen, G.; Rheinheimer, D.E. Parameter uncertainty analysis of reservoir operating rules based on implicit stochastic optimization. J. Hydrol.
**2014**, 514, 102–113. [Google Scholar] [CrossRef] - Labadie, J.W. Optimal operation of multireservoir systems: State-of-the-art review. J. Water Resour. Plan. Manag.
**2004**, 130, 93–111. [Google Scholar] [CrossRef] - Tilmant, A.; Pinte, D.; Goor, Q. Assessing marginal water values in multipurpose multireservoir systems via stochastic programming. Water Resour. Res.
**2008**, 44. [Google Scholar] [CrossRef] - Celeste, A.B.; Billib, M. Evaluation of stochastic reservoir operation optimization models. Adv. Water Resour.
**2009**, 32, 1429–1443. [Google Scholar] [CrossRef] - Liu, P.; Guo, S.; Xu, X.; Chen, J. Derivation of aggregation-based joint operating rule curves for cascade hydropower reservoirs. Water Resour. Manag.
**2011**, 25, 3177–3200. [Google Scholar] [CrossRef] - Giuliani, M.; Herman, J.; Castelletti, A.; Reed, P. Many-objective reservoir policy identification and refinement to reduce policy inertia and myopia in water management. Water Resour. Res.
**2014**, 50, 3355–3377. [Google Scholar] [CrossRef] - Giuliani, M.; Castelletti, A.; Pianosi, F.; Mason, E.; Reed, P.M. Curses, tradeoffs, and scalable management: Advancing evolutionary multiobjective direct policy search to improve water reservoir operations. J. Water Resour. Plan. Manag.
**2015**, 142, 04015050. [Google Scholar] [CrossRef] - Liu, P.; Guo, S.; Xiong, L.; Li, W.; Zhang, H. Deriving reservoir refill operating rules by using the proposed DPNS model. Water Resour. Manag.
**2006**, 20, 337–357. [Google Scholar] [CrossRef] - Celeste, A.B.; Curi, W.F.; Curi, R.C. Implicit stochastic optimization for deriving reservoir operating rules in semiarid Brazil. Pesqui. Oper.
**2009**, 29, 223–234. [Google Scholar] [CrossRef] - Chen, J.; Zhong, P.A.; Zhao, Y.F.; Xu, B. Risk analysis for the downstream control section in the real-time flood control operation of a reservoir. Stoch. Environ. Res. Risk Assess.
**2015**, 29, 1303–1315. [Google Scholar] [CrossRef] - Adeyemo, J.A.; Otieno, F.O. Maximization of hydropower using strategies of differential evolution. OIDA Int. J. Sustain. Dev.
**2010**, 1, 33–37. [Google Scholar] - Xu, B.; Zhong, P.A.; Zambon, R.C.; Zhao, Y.; Yeh, W.W.G. Scenario tree reduction in stochastic programming with recourse for hydropower operations. Water Resour. Res.
**2015**, 51, 6359–6380. [Google Scholar] [CrossRef] - Neboh, N.; Adeyemo, J.; Enitan, A.; Olugbara, O. A Review on Applications of Evolutionary Algorithms to Reservoir Operation for Hydropower Production. Int. J. Environ. Chem. Ecol. Geol. Geophys. Eng.
**2015**, 9, 1115–1121. [Google Scholar] - Kang, L.; Zhang, S.; Ding, Y.; He, X. Extraction and preference ordering of multireservoir water supply rules in dry years. Water
**2016**, 8, 28. [Google Scholar] [CrossRef] - Xu, B.; Zhong, P.A.; Huang, Q.; Wang, J.; Yu, Z.; Zhang, J. Optimal hedging rules for water supply reservoir operations under forecast uncertainty and conditional value-at-risk criterion. Water
**2017**, 9, 568. [Google Scholar] [CrossRef] - Jia, B.; Zhong, P.; Wan, X.; Xu, B.; Chen, J. Decomposition–coordination model of reservoir group and flood storage basin for real-time flood control operation. Hydrol. Res.
**2015**, 46, 11–25. [Google Scholar] [CrossRef] - Sordo-Ward, A.; Gabriel-Martin, I.; Bianucci, P.; Garrote, L. A Parametric Flood Control Method for Dams with Gate-Controlled Spillways. Water
**2017**, 9, 237. [Google Scholar] [CrossRef] - Lian, J.; Yao, Y.; Ma, C.; Guo, Q. Reservoir operation rules for controlling algal blooms in a tributary to the impoundment of Three Gorges Dam. Water
**2014**, 6, 3200–3223. [Google Scholar] [CrossRef] - Momtahen, S.; Dariane, A. Direct search approaches using genetic algorithms for optimization of water reservoir operating policies. J. Water Resour. Plan. Manag.
**2007**, 133, 202–209. [Google Scholar] [CrossRef] - Zeng, X.; Hu, T.; Xiong, L.; Cao, Z.; Xu, C. Derivation of operation rules for reservoirs in parallel with joint water demand. Water Resour. Res.
**2015**, 51, 9539–9563. [Google Scholar] [CrossRef] - Li, X.; Guo, S.; Liu, P.; Chen, G. Dynamic control of flood limited water level for reservoir operation by considering inflow uncertainty. J. Hydrol.
**2010**, 391, 124–132. [Google Scholar] [CrossRef] - Jeuland, M. Planning Water Resources Development in an Uncertain Climate Future: A Hydro-Economic Simulation Framework Applied to the Case of the Blue Nile. PhD Thesis, The University of North Carolina at Chapel Hill, Chapel Hill, NC, USA, 2009. [Google Scholar]
- Whittington, D.; Guariso, G. Water Management Models in Practice: A Case Study of the Aswan High Dam; Elsevier Scientific Publishing Company: Amsterdam, The Netherlands, 1983. [Google Scholar]
- Guariso, G.; Whittington, D. Implications of Ethiopian water development for Egypt and Sudan. Int. J. Water Resour. Dev.
**1987**, 3, 105–114. [Google Scholar] [CrossRef] - Georgakakos, A.P.; Marks, D.H. A new method for the real-time operation of reservoir systems. Water Resour. Res.
**1987**, 23, 1376–1390. [Google Scholar] [CrossRef] - Whittington, D.; McClelland, E. Opportunities for regional and international cooperation in the Nile basin. Water Int.
**1992**, 17, 144–154. [Google Scholar] [CrossRef] - Levy, B.S.; Baecher, G.B. NileSim: A Windows-based hydrologic simulator of the Nile River Basin. J. Water Resour. Plan. Manag.
**1999**, 125, 100–106. [Google Scholar] [CrossRef] - Whittington, D.; Waterbury, J.; Jeuland, M. The Grand Renaissance Dam and prospects for cooperation on the Eastern Nile. Water Policy
**2014**, 16, 595–608. [Google Scholar] - Whittington, D.; Wu, X.; Sadoff, C. Water resources management in the Nile basin: The economic value of cooperation. Water Policy
**2005**, 7, 227–252. [Google Scholar] - Jeuland, M. Economic implications of climate change for infrastructure planning in transboundary water systems: An example from the Blue Nile. Water Resour. Res.
**2010**, 46, 2387–2392. [Google Scholar] [CrossRef] - Jeuland, M.; Whittington, D. Water resources planning under climate change: Assessing the robustness of real options for the Blue Nile. Water Resour. Res.
**2014**, 50, 2086–2107. [Google Scholar] [CrossRef] - Ding, N.; Erfani, R.; Mokhtar, H.; Erfani, T. Agent based modelling for water resource allocation in the transboundary Nile River. Water
**2016**, 8, 139. [Google Scholar] [CrossRef] - Jeuland, M.; Wu, X.; Whittington, D. Infrastructure development and the economics of cooperation in the Eastern Nile. Water Int.
**2017**, 42, 121–141. [Google Scholar] [CrossRef] - Liersch, S.; Koch, H.; Hattermann, F.F. Management Scenarios of the Grand Ethiopian Renaissance Dam and Their Impacts under Recent and Future Climates. Water
**2017**, 9, 728. [Google Scholar] - Nagesh Kumar, D.; Janga Reddy, M. Multipurpose reservoir operation using particle swarm optimization. J. Water Resour. Plan. Manag.
**2007**, 133, 192–201. [Google Scholar] [CrossRef] - Liu, P.; Cai, X.; Guo, S. Deriving multiple near-optimal solutions to deterministic reservoir operation problems. Water Resour. Res.
**2011**, 47, 2168–2174. [Google Scholar] [CrossRef] - Ahn, J.M.; Jung, K.Y.; Shin, D. Effects of coordinated operation of weirs and reservoirs on the water quality of the Geum River. Water
**2017**, 9, 423. [Google Scholar] - Ahn, J.M.; Yang, D.S.; Jung, K.Y.; Shin, D.S. Assessing the Coordinated Operation of Reservoirs and Weirs for Sustainable Water Management in the Geum River Basin under Climate Change. Water
**2018**, 10, 30. [Google Scholar] - Taormina, R.; Chau, K.W.; Sivakumar, B. Neural network river forecasting through baseflow separation and binary-coded swarm optimization. J. Hydrol.
**2015**, 529, 1788–1797. [Google Scholar] [CrossRef] - Olyaie, E.; Banejad, H.; Chau, K.W.; Melesse, A.M. A comparison of various artificial intelligence approaches performance for estimating suspended sediment load of river systems: A case study in United States. Environ. Monit. Assess.
**2015**, 187, 189. [Google Scholar] [CrossRef] [PubMed] - Chen, X.Y.; Chau, K.W. A hybrid double feedforward neural network for suspended sediment load estimation. Water Resour. Manag.
**2016**, 30, 2179–2194. [Google Scholar] [CrossRef] - Cybenko, G. Approximation by superpositions of a sigmoidal function. Math. Control Signals Syst.
**1989**, 2, 303–314. [Google Scholar] [CrossRef] - Hornik, K.; Stinchcombe, M.; White, H. Multilayer feedforward networks are universal approximators. Neural Netw.
**1989**, 2, 359–366. [Google Scholar] [CrossRef] - UNEP (United Nations Environment Programme). Adaptation to Climate-Change Induced Water Stress in the Nile Basin: A Vulnerability Assessment Report; E. Kironde-Gowa; United Nations: New York, NY, USA, 2013; p. 164. [Google Scholar]
- Camberlin, P. Nile basin climates. In The Nile; Springer: Berlin, Germany, 2009; pp. 307–333. [Google Scholar]
- El-Fadel, M.; El-Sayegh, Y.; El-Fadl, K.; Khorbotly, D. The Nile River Basin: A case study in surface water conflict resolution. J. Nat. Resour. Life Sci. Educ.
**2003**, 32, 107. [Google Scholar] - Yao, H.; Georgakakos, A. Nile Decision Support Tool River Simulation and Management; Georgia Institute of Technology: Atlanta, GA, USA, 2003. [Google Scholar]
- Woodward, J.; Macklin, M.; Fielding, L.; Millar, I.; Spencer, N.; Welsby, D.; Williams, M. Shifting sediment sources in the world’s longest river: A strontium isotope record for the Holocene Nile. Quat. Sci. Rev.
**2015**, 130, 124–140. [Google Scholar] [CrossRef] - Foucault, A.; Stanley, D.J. Late Quaternary palaeoclimatic oscillations in East Africa recorded by heavy minerals in the Nile delta. Nature
**1989**, 339, 44–46. [Google Scholar] [CrossRef] - Padoan, M.; Garzanti, E.; Harlavan, Y.; Villa, I.M. Tracing Nile sediment sources by Sr and Nd isotope signatures (Uganda, Ethiopia, Sudan). Geochim. Cosmochim. Acta
**2011**, 75, 3627–3644. [Google Scholar] [CrossRef] - Wale, A. Hydrological Balance of Lake Tana Upper Blue Nile Basin, Ethiopia; ITC: Enschede, The Netherlands, 2008; pp. 159–180. [Google Scholar]
- Sangiorgio, M. A Neural Approach for Multi-Reservoir System Operation. Master’s Thesis, Politecnico di Milano, Milano, Italy, 2016. [Google Scholar]
- Taormina, R.; Chau, K.W.; Sethi, R. Artificial neural network simulation of hourly groundwater levels in a coastal aquifer system of the Venice lagoon. Eng. Appl. Artif. Intell.
**2012**, 25, 1670–1676. [Google Scholar] [CrossRef] - Glorot, X.; Bordes, A.; Bengio, Y. Deep sparse rectifier neural networks. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA, 11–13 April 2011; pp. 315–323. [Google Scholar]
- Hornik, K. Approximation capabilities of multilayer feedforward networks. Neural Netw.
**1991**, 4, 251–257. [Google Scholar] [CrossRef]

**Figure 1.**Schematic representation of the ISO procedure (modified from [14]). First, the optimal open-loop decisions are computed. Then, the obtained dataset is used for the identification of the closed-loop policy.

**Figure 3.**Schematic representation of the Nile River system. White Nile upstream Khartoum and Atbara before Khasm el Girba Dam have been considered as unregulated flows.

**Figure 5.**Objective space for a dry (

**a**) and a wet (

**b**) scenario. Averages of inflows for the considered scenarios are reported in Table 1.

**Figure 7.**Error distribution for the identified control laws. Errors are computed as the difference between targets and model outputs and normalized between $-1$ and 1.

**Figure 9.**Control laws for Lake Nasser with decentralized and incomplete information: the release is expressed as function of the storage.

**Table 1.**Yearly average of inflows (${\mathrm{km}}^{3}/\mathrm{month}$) and objectives improvement with respect to the trivial policy, obtained using the optimal open-loop control sequence for two synthetic scenarios.

Inflows | Dry | Wet |
---|---|---|

${n}_{t}^{\mathrm{Tana}}$ | $0.809$ | $0.830$ |

${i}_{t}^{\mathrm{Kes}}$ | $1.008$ | $1.173$ |

${i}_{t}^{\mathrm{Bor}}$ | $2.504$ | $2.972$ |

${i}_{t}^{\mathrm{Din}}$ | $0.177$ | $0.215$ |

${i}_{t}^{\mathrm{Rah}}$ | $0.090$ | $0.095$ |

${i}_{t}^{\mathrm{WN}}$ | $2.214$ | $2.356$ |

${i}_{t}^{\mathrm{u},\mathrm{At}}$ | $0.849$ | $1.063$ |

${i}_{t}^{\mathrm{l},\mathrm{At}}$ | $0.036$ | $0.020$ |

$\sum \mathrm{inflows}$ | $7.687$ | $8.724$ |

Energy increase $\left[\%\right]$ | $8.31$ | $11.48$ |

Water deficit decrease $\left[\%\right]$ | $56.14$ | $51.22$ |

**Table 2.**Variables given as input to the control laws for the four considered cases: centralized-full (CF), centralized-incomplete (CI), decentralized-full (DF), and decentralized-incomplete (DI).

C | D | F | I | Variables Composing ${\mathit{I}}_{\mathit{t}}$ |
---|---|---|---|---|

• | • | Tana: ${s}_{t}^{\mathrm{Tana}}$, $w{d}_{t}^{\mathrm{Tana}}$ | ||

Roseires: ${s}_{t}^{\mathrm{Ros}}$, $w{d}_{t}^{\mathrm{Ros}}$ | ||||

Girba: ${s}_{t}^{\mathrm{KeG}}$, $w{d}_{t}^{\mathrm{KeG}}$ | ||||

Nasser: ${s}_{t}^{\mathrm{Nas}}$, $w{d}_{t}^{\mathrm{Egypt}}$ | ||||

• | • | Tana: ${s}_{t}^{\mathrm{Tana}}$, $w{d}_{t}^{\mathrm{Tana}}$, ${n}_{t}^{\mathrm{Tana}}$ | ||

Roseires: ${s}_{t}^{\mathrm{Ros}}$, $w{d}_{t}^{\mathrm{Ros}}$, ${e}_{t}^{\mathrm{Ros}}$, ${i}_{t}^{\mathrm{Ros}}$ | ||||

Girba: ${s}_{t}^{\mathrm{KeG}}$, $w{d}_{t}^{\mathrm{KeG}}$, ${e}_{t}^{\mathrm{KeG}}$, ${i}_{t}^{\mathrm{KeG}}$ | ||||

Nasser: ${s}_{t}^{\mathrm{Nas}}$, $w{d}_{t}^{\mathrm{Egypt}}$, $w{d}_{t}^{\mathrm{Nas}}$, ${e}_{t}^{\mathrm{Nas}}$, ${l}_{t}^{\mathrm{Nas}}$, ${i}_{t}^{\mathrm{Nas}}$ | ||||

• | • | ${s}_{t}^{\mathrm{Tana}}$, ${s}_{t}^{\mathrm{Ros}}$, ${s}_{t}^{\mathrm{KeG}}$, ${s}_{t}^{\mathrm{Nas}}$, $w{d}_{t}^{\mathrm{Tana}}$, $w{d}_{t}^{\mathrm{Ros}}$, $w{d}_{t}^{\mathrm{KeG}}$, $w{d}_{t}^{\mathrm{Egypt}}$ | ||

• | • | ${s}_{t}^{\mathrm{Tana}}$, ${s}_{t}^{\mathrm{Ros}}$, ${s}_{t}^{\mathrm{KeG}}$, ${s}_{t}^{\mathrm{Nas}}$, $w{d}_{t}^{\mathrm{Tana}}$, $w{d}_{t}^{\mathrm{Ros}}$, $w{d}_{t}^{\mathrm{KeG}}$, $w{d}_{t}^{\mathrm{Egypt}}$, $w{d}_{t}^{\mathrm{Nas}}$ | ||

${n}_{t}^{\mathrm{Tana}}$, ${e}_{t}^{\mathrm{Ros}}$, ${i}_{t}^{\mathrm{Ros}}$, ${e}_{t}^{\mathrm{KeG}}$, ${i}_{t}^{\mathrm{KeG}}$, ${e}_{t}^{\mathrm{Nas}}$, ${l}_{t}^{\mathrm{Nas}}$, ${i}_{t}^{\mathrm{Nas}}$ |

Reservoir | Function | Case | Training | Validation | Testing | All |
---|---|---|---|---|---|---|

Ros | Sigmoid | DI | $89.1$ | $89.0$ | $88.7$ | $89.0$ |

DF | $90.5$ | $90.4$ | $90.1$ | $90.4$ | ||

CI | $89.7$ | $89.5$ | $89.3$ | $89.6$ | ||

CF | $90.7$ | $90.5$ | $90.3$ | $90.6$ | ||

Radial Basis | DI | $89.1$ | $89.0$ | $88.7$ | $89.0$ | |

DF | $90.5$ | $90.4$ | $90.1$ | $90.4$ | ||

CI | $89.7$ | $89.6$ | $89.2$ | $89.6$ | ||

CF | $90.8$ | $90.6$ | $90.3$ | $90.6$ | ||

KeG | Sigmoid | DI | $90.1$ | $89.7$ | $89.3$ | $89.9$ |

DF | $91.8$ | $91.4$ | $91.3$ | $91.6$ | ||

CI | $91.0$ | $90.6$ | $90.3$ | $90.8$ | ||

CF | $92.4$ | $92.1$ | $91.9$ | $92.2$ | ||

Radial Basis | DI | $90.1$ | $89.7$ | $89.4$ | $89.9$ | |

DF | $91.7$ | $91.4$ | $91.2$ | $91.6$ | ||

CI | $91.0$ | $90.5$ | $90.3$ | $90.8$ | ||

CF | $92.4$ | $92.0$ | $91.9$ | $92.2$ | ||

Nas | Sigmoid | DI | $78.1$ | $77.3$ | $77.4$ | $77.8$ |

DF | $91.2$ | $90.7$ | $90.8$ | $91.0$ | ||

CI | $78.7$ | $77.8$ | $78.0$ | $78.4$ | ||

CF | $91.2$ | $90.7$ | $90.7$ | $91.0$ | ||

Radial Basis | DI | $78.1$ | $77.3$ | $77.4$ | $77.8$ | |

DF | $91.2$ | $90.7$ | $90.8$ | $91.0$ | ||

CI | $78.8$ | $77.9$ | $78.0$ | $78.4$ | ||

CF | $91.2$ | $90.7$ | $90.7$ | $91.0$ |

**Table 4.**Improvements over the solution obtained with the trivial closed-loop policy for all the considered cases, expressed as percentage of the distance between this solution and the optimal open-loop release sequence.

Control | Case | Distance from the Trivial Policy [%] | |
---|---|---|---|

Hydropower Producers | Agricultural Districts | ||

Trivial policy | $0.00$ | $0.00$ | |

Linear policy | DI | $-\phantom{\rule{0.277778em}{0ex}}13.73$ | $+\phantom{\rule{0.277778em}{0ex}}61.28$ |

DF | $+\phantom{\rule{0.277778em}{0ex}}7.95$ | $+\phantom{\rule{0.277778em}{0ex}}67.97$ | |

CI | $+\phantom{\rule{0.277778em}{0ex}}19.41$ | $+\phantom{\rule{0.277778em}{0ex}}65.99$ | |

CF | $+\phantom{\rule{0.277778em}{0ex}}48.82$ | $+\phantom{\rule{0.277778em}{0ex}}93.60$ | |

Neural policy | DI | $+\phantom{\rule{0.277778em}{0ex}}51.52$ | $+\phantom{\rule{0.277778em}{0ex}}91.98$ |

DF | $+\phantom{\rule{0.277778em}{0ex}}52.46$ | $+\phantom{\rule{0.277778em}{0ex}}93.79$ | |

CI | $+\phantom{\rule{0.277778em}{0ex}}50.64$ | $+\phantom{\rule{0.277778em}{0ex}}94.44$ | |

CF | $+\phantom{\rule{0.277778em}{0ex}}56.47$ | $+\phantom{\rule{0.277778em}{0ex}}96.13$ | |

OL sequence | $+\phantom{\rule{0.277778em}{0ex}}100.00$ | $+\phantom{\rule{0.277778em}{0ex}}100.00$ |

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Sangiorgio, M.; Guariso, G. NN-Based Implicit Stochastic Optimization of Multi-Reservoir Systems Management. *Water* **2018**, *10*, 303.
https://doi.org/10.3390/w10030303

**AMA Style**

Sangiorgio M, Guariso G. NN-Based Implicit Stochastic Optimization of Multi-Reservoir Systems Management. *Water*. 2018; 10(3):303.
https://doi.org/10.3390/w10030303

**Chicago/Turabian Style**

Sangiorgio, Matteo, and Giorgio Guariso. 2018. "NN-Based Implicit Stochastic Optimization of Multi-Reservoir Systems Management" *Water* 10, no. 3: 303.
https://doi.org/10.3390/w10030303