# Machine Learning, Urban Water Resources Management and Operating Policy

## Abstract

**:**

## 1. Introduction

_{s}

^{n}N

_{d}

^{n}), where N

_{s}and N

_{d}are the number of elements in the discretized state, and release decision sets respectively. Furthermore, if the inflow uncertainty is considered, the resulting Bayesian stochastic dynamic programming [12] carries an even higher computational cost, since the number of the state variables increases by the number of inflows and forecasts [13]. Chaves and Kojiri [14] tried to address this issue by employing stochastic fuzzy neural networks, which intrinsically consider the stochastic nature of the problem and can be trained directly, without dynamic programming, to minimize operating costs. This method, though much faster than the two-step optimization (first the application of the dynamic programming and then the training of the neural network), is still compute-intensive for two reasons. First, an extra loop within the training process is introduced to account for the stochastic variable—the inflow. Second, the genetic algorithm, which is used for optimization, is much slower than any gradient-based optimization method, which is used in FFN training.

## 2. Materials and Methods

- Instead of dynamic programming, network flow programming (NFP) is used for optimizing the water supply system. In NFP, a water supply system is represented with a graph of N nodes (e.g., points where two aqueducts join, reservoirs, etc.) and A ⊂ N × N links between the nodes (including the carry-over artificial arcs that simulate the storage elements). The computational effort of an efficient NFP algorithm, e.g., RELAX4, is proportional to T N card(A) log(N C), where C is the range of fluctuation of the penalty functions’ values [17]. Therefore, it is feasible to optimize multi-reservoir water supply systems over a continuous state space. Furthermore, NFP simultaneously performs simulation and optimization of the whole water supply system, and there are programs that offer friendly GUI and CAD environments to define the topology of the water supply system [18].
- To successfully manage water resources, it is important to properly analyze the stochastic structure of the reservoir inflows. Bras and Rodriguez-Iturbe [19] have highlighted that the autocovariance estimator employed in classical statistics underestimates the autocovariance terms with large lags. This can underestimate the duration of the droughts. To overcome this issue, Koutsoyiannis suggested an approach combining a generalized autocovariance function with a coupling of stochastic models of different time scales [20,21]. This approach is capable of reproducing the persistence of multivariate processes (i.e., ‘the tendency of annual average streamflows to stay above or below their mean value for long periods’ [20]), essential for reliably estimating the long-term fluctuations of the reservoir inflows.
- The typical output of multi-reservoir management tools is the abstraction or releases from the available water resources. However, this output does not directly assist in system operations. The releases/abstractions need to be routed to the demand locations. This may be straightforward in the case of simple water supply systems (e.g., single reservoir and a single-line aqueduct); however, in the case of more complicated systems, a model may be required. To avoid this additional effort, a demand-oriented approach is employed in this study (see UWOT pull-signals in [22]). In this approach, the generation and routing of the water demand are simulated, instead of the flows. Instead of training the FFN to approximate the releases/abstractions, this approach trains it to approximate the ratio of the flows wherever two or more aqueducts join. With these ratios available at each time step for the entire water supply system, the calculation of releases/abstractions is simply a matter of multiplying the incoming demand signal by the appropriate sequence of ratios on the path from the demand point to each resource.

## 3. Case Study

_{it}for Mornos and Hyliki (priority 4000, see Equation (4) of [23]), and −10,010 f

_{it}for Marathon (priority 3999), where fit is the storage in the reservoir i during the time step t. In this study, 4 alternative operating policies were assessed to verify the applicability of the suggested methodology. These were implemented employing 4 different sets of energy consumption penalty functions (Table 1) assigned to the appropriate links.

**O**= f

_{out}(f

_{hid}(

**J**

**W**

_{hid}+

**b**

_{hid})

**W**

_{out}+

**b**

_{out}),

**O**(with dimensions 1 × 4 for the topology of Figure 3 and for a single time step) is the neural network output, f

_{out}and f

_{hid}are the activation functions of the output and hidden layers respectively,

**J**(1 × 9) is the FFN input,

**W**

_{hid}(9 × 21) are the weights of the hidden layer,

**b**

_{hid}is the vector with the bias terms (1 × 21) of the hidden layer,

**W**

_{out}(21 × 4) are the weights of the output layer, and

**b**

_{out}(1 × 4) is the vector with the bias terms of the output layer.

## 4. Results

## 5. Discussion

^{3}of the energy-intensive resources can be found in the first row of Table 1, which is also the penalty values of operating policy No. 1). Mavrosouvala remains inactive in all 4 operating policies. Systematic usage of the Vassilika boreholes is suggested by operating rules No. 1 and 2, whereas the usage of this resource is not suggested by the other two rules. The contribution of the energy-intensive resources in each operating policy is summarized in Table 3.

^{3}respectively. These gradually increasing abstractions result in the gradually lower storage levels during the dry period when going from operating policy No. 1 to operating policy No. 4 (see Figure 7 and Figure 9). In all cases, the FFN reproduced with very good accuracy the abstractions suggested by the optimization of the water supply system with MODSIM. It should be noted this optimization was not included in the FFN training, which was based exclusively on the simulations with synthetic data.

## 6. Conclusions

- The optimization is very fast and accounts for the long-term persistence of the system stresses (i.e., ‘the tendency of annual average streamflows to stay above or below their mean value for long periods’)
- Enabled by the demand-oriented approach, the suggested methodology does not require any specialized software for the decision support model. The ‘how-to-operate’ instructions via a simple algebraic formula can moderate the required level of expertise to use a decision support tool because the algebraic formula can be easily wrapped in a user-friendly interface (a simple desktop application, a web-based application, a GIS-based application, even a spreadsheet).

## Funding

## Conflicts of Interest

## References

- Philbrick, C.R.; Kitanidis, P. Limitations of deterministic optimization applied to reservoir operations. J. Water Resour. Plan. Manag.
**1999**, 125, 135–142. [Google Scholar] [CrossRef] - Grigg, N.S. Water Resources Management: Principles, Cases, and Regulations; McGraw-Hill Professional: New York, NY, USA, 1996. [Google Scholar]
- Ramsundar, B.; Zadeh, R.B. TensorFlow for Deep Learning, 1st ed.; O’Reilly: Sebastopol, CA, USA, 2018. [Google Scholar]
- French, M.N.; Krajewski, W.F.; Cuykendal, R.R. Rainfall forecasting in space and time using a neural network. J. Hydro.
**1992**, 137, 1–37. [Google Scholar] [CrossRef] - Raman, H.; Chandramouli, V. Deriving a general operating policy for reservoirs using neural network. J. Water Resour. Plan. Manag.
**1996**, 122, 342–347. [Google Scholar] [CrossRef] - Chandramouli, V.; Raman, H. Multireservoir Modeling with Dynamic Programming and Neural Networks. J. Water Resour. Plan. Manag.
**2001**, 127, 89–98. [Google Scholar] [CrossRef] - Cancelliere, A.; Giuliano, G.; Ancarani, A.; Rossi, G. A Neural Networks Approach for Deriving Irrigation Reservoir Operating Rules. Water Resour. Manag.
**2002**, 16, 71–88. [Google Scholar] [CrossRef] - Chang, Y.; Chang, L.; Chang, F. Intelligent control for modeling of real-time reservoir operation, part II: Artificial neural network with operating rule curves. Hydrol. Process.
**2005**, 19, 1431–1444. [Google Scholar] [CrossRef] - Mansouri, R.; Pudeh, H.T.; Yonesi, H.A.; Haghiabi, A.H. Dynamic programming model for hydraulics and water resources simulating and optimizing water transfer system (a case study in Iran). J. Water Supply Res. Technol.
**2017**, 66, 684–700. [Google Scholar] [CrossRef] - Bellman, R. Dynamic Programming, 1st ed.; Princeton University Press: Princeton, NJ, USA, 1957. [Google Scholar]
- Castelletti, A.; Galelli, S.; Restelli, M.; Soncini-Sessa, R. Tree-based reinforcement learning for optimal water reservoir operation. Water Resour. Res.
**2010**, 46, W09507. [Google Scholar] [CrossRef] - Karamouz, M.; Vasiliadis, H.V. Bayesian stochastic optimization of reservoir operation using uncertain forecasts. Water Resour. Res.
**1992**, 28, 1221–1232. [Google Scholar] [CrossRef] - Rani, D.; Moreira, M.M. Simulation–Optimization Modeling: A Survey and Potential Application in Reservoir Systems Operation. Water Resour. Manag.
**2010**, 24, 1107–1138. [Google Scholar] [CrossRef] - Chaves, P.; Kojiri, T. Deriving reservoir operational strategies considering water quantity and quality objectives by stochastic fuzzy neural networks. Adv. Water Resour.
**2007**, 30, 1329–1341. [Google Scholar] [CrossRef] - Lobbrecht, A.H.; Solomatine, D.P. Machine learning in real-time control of water systems. Urban Water
**2002**, 4, 283–289. [Google Scholar] [CrossRef] - Brandmeyer, J.E.; Karimi, H.A. Coupling methodologies for environmental models. Environ. Model. Softw.
**2000**, 15, 479–488. [Google Scholar] [CrossRef] - Bertsekas, D.; Tseng, P. RELAX-IV: A Faster Version of the RELAX Code for Solving Minimum Cost Flow Problems, Completion Report under NSF Grant CCR-9103804; Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology: Cambridge, MA, USA, 1994. [Google Scholar]
- Rozos, E. An assessment of the operational freeware management tools for multi-reservoir systems. Water Supply
**2018**, 19, 995–1007. [Google Scholar] [CrossRef] - Bras, R.L.; Rodriguez-Iturbe, I. Random Functions and Hydrology, 1st ed.; Dover Publications: New York, NY, USA, 1993. [Google Scholar]
- Koutsoyiannis, D. A generalized mathematical framework for stochastic simulation and forecast of hydrologic time series. Water Resour. Res.
**2000**, 36, 1519–1534. [Google Scholar] [CrossRef] - Koutsoyiannis, D. Coupling stochastic models of different time scales. Water Resour. Res.
**2001**, 37, 379–392. [Google Scholar] [CrossRef] - Rozos, E.; Butler, D.; Makropoulos, C. An integrated system dynamics–cellular automata model for distributed water-infrastructure planning. Water Sci. Technol. Water Supply
**2016**, 17, 995–1007. [Google Scholar] [CrossRef] - Labadie, J.W.; Larson, R. MODSIM 8.1: River Basin Management Decision Support System, User Manual and Documentation, 1st ed.; Colorado State University: Fort Collins, CO, USA, 2010. [Google Scholar]
- US Army Corps of Engineers. HEC-ResPRM Prescriptive Reservoir Model, Quick Start Guide, 1st ed.; US Army Corps of Engineers, Hydrologic Engineering Center: Davis, CA, USA, 2011. [Google Scholar]
- Castalia. A Computer System for Stochastic Simulation and Forecasting of Hydrologic Processes; National Technical University of Athens: Athens, Greece, 2012; Available online: http://itia.ntua.gr/en/softinfo/2/ (accessed on 30 August 2012).
- Koutsoyiannis, D. Probability and Statistics for Geophysical Processes, 1st ed.; National Technical University of Athens: Athens, Greece, 2008; Available online: https://doi.org/10.13140/RG.2.1.2300.1849/1 (accessed on 1 May 2008).
- Dennis, J.E.; Schnabel, R.B. Numerical Methods for Unconstrained Optimization and Nonlinear Equations, 1st ed.; Prentice-Hall: Englewood Cliffs, NJ, USA, 1983; p. 227. [Google Scholar]
- Eaton, J.W.; Bateman, D.; Hauberg, S.; Wehbring, R. GNU Octave Version 5.1.0 Manual: A High-Level Interactive Language for Numerical Computations; Free Software Foundation, Inc.: Boston, MA, USA, 2019. [Google Scholar]
- Rozos, E.; Makropoulos, C. Source to tap urban water cycle modelling. Environ. Model. Softw.
**2013**, 41, 139–150. [Google Scholar] [CrossRef][Green Version]

**Figure 4.**Abstractions from the Mornos reservoir obtained with MODSIM, and FFN with UWOT for operating policies No. 1 (

**a**), No. 2 (

**b**), No. 3 (

**c**), and No. 4 (

**d**).

**Figure 5.**Abstractions from the Hyliki reservoir obtained with MODSIM, and FFN with UWOT for operating policies No. 1 (

**a**) and 3 (

**b**).

**Figure 6.**Groundwater abstractions from Vassilika obtained with MODSIM, and FFN with UWOT for operating policies No. 1 (

**a**) and 2 (

**b**).

**Figure 7.**Storage in the Mornos reservoir obtained with MODSIM, and FFN with UWOT for operating policies No. 1 (

**a**), No. 2 (

**b**), No. 3 (

**c**), and No. 4 (

**d**).

**Figure 8.**Storage in Hylilki (

**a**) and Marathon (

**b**) reservoirs obtained with MODSIM, and FFN with UWOT.

**Figure 9.**Energy/storage trade-off of operating policies No. 1 to 4. The square markers indicate the solutions obtained by MODSIM and the markers with a cross indicate the solutions obtained by the FFN. The red line is the trendline.

**Table 1.**Operating policies (OP) as implemented with the penalty values assigned to the appropriate network links.

OP | Link Hy-Kl ^{1} | Link Ma-Kl ^{1} | Link Va-Di ^{1} |
---|---|---|---|

1 | 45 | 150 | 23 |

2 | 19,565 | 65,217 | 10,000 |

3 | 45 | ∞ | ∞ |

4 | ∞ | ∞ | ∞ |

^{1}Link Hy-Kl is between Rsv_Hyliki and Cnf_Klidi, link Ma-Kl is between GW_Mavrosouvala and Cnf_Klidi, link Va-Di is between GW_Vassilika and Cnf_Distomo, see Figure 2.

OP1 | OP2 | OP3 | OP4 | |
---|---|---|---|---|

Correlation coefficient | 0.92 | 0.91 | 0.95 | 0.95 |

MSE | 30.1 | 19.1 | 24.7 | 23.7 |

Operating Policy | Hyliki Abstractions | Vassilika Abstractions | Mavrosouvala Abstractions |
---|---|---|---|

1 | ✓ | ✓ | – |

2 | – | ✓ | – |

3 | ✓ | – | – |

4 | – | – | – |

© 2019 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Rozos, E.
Machine Learning, Urban Water Resources Management and Operating Policy. *Resources* **2019**, *8*, 173.
https://doi.org/10.3390/resources8040173

**AMA Style**

Rozos E.
Machine Learning, Urban Water Resources Management and Operating Policy. *Resources*. 2019; 8(4):173.
https://doi.org/10.3390/resources8040173

**Chicago/Turabian Style**

Rozos, Evangelos.
2019. "Machine Learning, Urban Water Resources Management and Operating Policy" *Resources* 8, no. 4: 173.
https://doi.org/10.3390/resources8040173