Next Article in Journal
Community Response of Cladocera to Trophic Stress by Biomanipulation in a Shallow Oxbow Lake
Next Article in Special Issue
Efficacy of Inactivation of Human Enteroviruses by Dual-Wavelength Germicidal Ultraviolet (UV-C) Light Emitting Diodes (LEDs)
Previous Article in Journal
Second Cheese Whey Treatment Using Zeolite under Continuous Flow Mode and Its Application on Wheat Growth
Previous Article in Special Issue
Application of the Mathematical Simulation Methods for the Assessment of the Wastewater Treatment Plant Operation Work Reliability
Article Menu
Issue 5 (May) cover image

Export Article

Open AccessArticle

Intelligent Control/Operational Strategies in WWTPs through an Integrated Q-Learning Algorithm with ASM2d-Guided Reward

State Key Laboratory of Urban Water Resource and Environment, Harbin Institute of Technology, Harbin 150000, China
*
Author to whom correspondence should be addressed.
Water 2019, 11(5), 927; https://doi.org/10.3390/w11050927
Received: 10 March 2019 / Revised: 29 April 2019 / Accepted: 30 April 2019 / Published: 1 May 2019
(This article belongs to the Special Issue Advances in Water and Wastewater Monitoring and Treatment Technology)
  |  
PDF [3664 KB, uploaded 17 May 2019]
  |  

Abstract

The operation of a wastewater treatment plant (WWTP) is a typical complex control problem, with nonlinear dynamics and coupling effects among the variables, which renders the implementation of real-time optimal control an enormous challenge. In this study, a Q-learning algorithm with activated sludge model No. 2d-guided (ASM2d-guided) reward setting (an integrated ASM2d-QL algorithm) is proposed, and the widely applied anaerobic-anoxic-oxic (AAO) system is chosen as the research paradigm. The integrated ASM2d-QL algorithms equipped with a self-learning mechanism are derived for optimizing the control strategies (hydraulic retention time (HRT) and internal recycling ratio (IRR)) of the AAO system. To optimize the control strategies of the AAO system under varying influent loads, Q matrixes were built for both HRTs and IRR optimization through the pair of <max reward-action> based on the integrated ASM2d-QL algorithm. 8 days of actual influent qualities of a certain municipal AAO wastewater treatment plant in June were arbitrarily chosen as the influent concentrations for model verification. Good agreement between the values of the model simulations and experimental results indicated that this proposed integrated ASM2d-QL algorithm performed properly and successfully realized intelligent modeling and stable optimal control strategies under fluctuating influent loads during wastewater treatment. View Full-Text
Keywords: machine learning; Q-learning algorithm; optimized control strategies; activated sludge model No. 2d (ASM2d), enhanced nutrients removal; integrated ASM-QL algorithm machine learning; Q-learning algorithm; optimized control strategies; activated sludge model No. 2d (ASM2d), enhanced nutrients removal; integrated ASM-QL algorithm
Figures

Graphical abstract

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).

Supplementary material

SciFeed

Share & Cite This Article

MDPI and ACS Style

Pang, J.; Yang, S.; He, L.; Chen, Y.; Ren, N. Intelligent Control/Operational Strategies in WWTPs through an Integrated Q-Learning Algorithm with ASM2d-Guided Reward. Water 2019, 11, 927.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Water EISSN 2073-4441 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top