Next Article in Journal
Steady-State Methodology for Activated Sludge Model 1 (ASM1) State Variable Calculation in MBR
Next Article in Special Issue
Evaluating Efficiency Improvement of Deep-Cut Curb Inlets for Road-Bioretention Stripes
Previous Article in Journal
Taking into Account both Explicit Conduits and the Unsaturated Zone in Karst Reservoir Hybrid Models: Impact on the Outlet Hydrograph
Previous Article in Special Issue
Anthropogenic Activity Effects on Canals Morphology, Case Study: Nile Delta, Egypt
Article

Deep Reinforcement Learning with Uncertain Data for Real-Time Stormwater System Control and Flood Mitigation

1
Department of Engineering Systems and Environment, University of Virginia, P.O. Box 400747, Charlottesville, VA 22904, USA
2
Link Lab, University of Virginia, P.O. Box 400259, Charlottesville, VA 22904, USA
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Water 2020, 12(11), 3222; https://doi.org/10.3390/w12113222
Received: 19 October 2020 / Revised: 5 November 2020 / Accepted: 12 November 2020 / Published: 17 November 2020
(This article belongs to the Special Issue Urban Rainwater and Flood Management)
Flooding in many areas is becoming more prevalent due to factors such as urbanization and climate change, requiring modernization of stormwater infrastructure. Retrofitting standard passive systems with controllable valves/pumps is promising, but requires real-time control (RTC). One method of automating RTC is reinforcement learning (RL), a general technique for sequential optimization and control in uncertain environments. The notion is that an RL algorithm can use inputs of real-time flood data and rainfall forecasts to learn a policy for controlling the stormwater infrastructure to minimize measures of flooding. In real-world conditions, rainfall forecasts and other state information are subject to noise and uncertainty. To account for these characteristics of the problem data, we implemented Deep Deterministic Policy Gradient (DDPG), an RL algorithm that is distinguished by its capability to handle noise in the input data. DDPG implementations were trained and tested against a passive flood control policy. Three primary cases were studied: (i) perfect data, (ii) imperfect rainfall forecasts, and (iii) imperfect water level and forecast data. Rainfall episodes (100) that caused flooding in the passive system were selected from 10 years of observations in Norfolk, Virginia, USA; 85 randomly selected episodes were used for training and the remaining 15 unseen episodes served as test cases. Compared to the passive system, all RL implementations reduced flooding volume by 70.5% on average, and performed within a range of 5%. This suggests that DDPG is robust to noisy input data, which is essential knowledge to advance the real-world applicability of RL for stormwater RTC. View Full-Text
Keywords: real-time control; reinforcement learning; smart stormwater systems; urban flooding real-time control; reinforcement learning; smart stormwater systems; urban flooding
Show Figures

Figure 1

MDPI and ACS Style

Saliba, S.M.; Bowes, B.D.; Adams, S.; Beling, P.A.; Goodall, J.L. Deep Reinforcement Learning with Uncertain Data for Real-Time Stormwater System Control and Flood Mitigation. Water 2020, 12, 3222. https://doi.org/10.3390/w12113222

AMA Style

Saliba SM, Bowes BD, Adams S, Beling PA, Goodall JL. Deep Reinforcement Learning with Uncertain Data for Real-Time Stormwater System Control and Flood Mitigation. Water. 2020; 12(11):3222. https://doi.org/10.3390/w12113222

Chicago/Turabian Style

Saliba, Sami M.; Bowes, Benjamin D.; Adams, Stephen; Beling, Peter A.; Goodall, Jonathan L. 2020. "Deep Reinforcement Learning with Uncertain Data for Real-Time Stormwater System Control and Flood Mitigation" Water 12, no. 11: 3222. https://doi.org/10.3390/w12113222

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Search more from Scilit
 
Search
Back to TopTop