Next Article in Journal
Silicon Fractionation of Soluble Silicon in Volcanic Ash Soils That May Affect Groundwater Silicon Content on Jeju Island, Korea
Next Article in Special Issue
Application of the Systems Approach to the Management of Complex Water Systems
Previous Article in Journal
Effect of Inclination Angles on the Local Scour around a Submerged Cylinder
Previous Article in Special Issue
Socio-Hydrology: A New Understanding to Unite or a New Science to Divide?
Open AccessArticle

Aggregation–Decomposition-Based Multi-Agent Reinforcement Learning for Multi-Reservoir Operations Optimization

1
Princeton Institute for International and Regional Studies (PIIRS), Princeton University, Princeton, NJ 08544, USA
2
School of Petroleum, Civil and Mining Engineering, Amirkabir University of Technology, Tehran 1591634311, Iran
3
School of Industrial Engineering, Amirkabir University of Technology, Tehran 1591634311, Iran
4
Department of Systems Design Engineering, University of Waterloo, Waterloo, ON N2L 3G1, Canada
*
Author to whom correspondence should be addressed.
Water 2020, 12(10), 2688; https://doi.org/10.3390/w12102688
Received: 13 June 2020 / Revised: 17 September 2020 / Accepted: 23 September 2020 / Published: 25 September 2020
Stochastic dynamic programming (SDP) is a widely-used method for reservoir operations optimization under uncertainty but suffers from the dual curses of dimensionality and modeling. Reinforcement learning (RL), a simulation-based stochastic optimization approach, can nullify the curse of modeling that arises from the need for calculating a very large transition probability matrix. RL mitigates the curse of the dimensionality problem, but cannot solve it completely as it remains computationally intensive in complex multi-reservoir systems. This paper presents a multi-agent RL approach combined with an aggregation/decomposition (AD-RL) method for reducing the curse of dimensionality in multi-reservoir operation optimization problems. In this model, each reservoir is individually managed by a specific operator (agent) while co-operating with other agents systematically on finding a near-optimal operating policy for the whole system. Each agent makes a decision (release) based on its current state and the feedback it receives from the states of all upstream and downstream reservoirs. The method, along with an efficient artificial neural network-based robust procedure for the task of tuning Q-learning parameters, has been applied to a real-world five-reservoir problem, i.e., the Parambikulam–Aliyar Project (PAP) in India. We demonstrate that the proposed AD-RL approach helps to derive operating policies that are better than or comparable with the policies obtained by other stochastic optimization methods with less computational burden. View Full-Text
Keywords: multireservoir operations; optimization; multi-agent reinforcement learning; aggregation–decomposition; neural networks multireservoir operations; optimization; multi-agent reinforcement learning; aggregation–decomposition; neural networks
Show Figures

Figure 1

MDPI and ACS Style

Hooshyar, M.; Mousavi, S.J.; Mahootchi, M.; Ponnambalam, K. Aggregation–Decomposition-Based Multi-Agent Reinforcement Learning for Multi-Reservoir Operations Optimization. Water 2020, 12, 2688.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Search more from Scilit
 
Search
Back to TopTop