Application of LSTM Networks for Water Demand Prediction in Optimal Pump Control
- How does an LSTM behave compared with methods that are already in operational use, in detail persistence and auto-regressive models?
- How does incremental or online learning of LSTM models help to improve the prediction accuracy?
- When no historic data are available, can an LSTM model from a different water distribution network be used instead as a starting point?
- The prediction accuracy for the next day is analyzed, in particular, the predictive performance of different LSTMs.
- The applicability of LSTMs in the process of operation planning is investigated.
2. Review of Methods for Time Series Prediction
2.1. Naïve Predictors
|Algorithm 1: Pseudocode for the naïve water consumption prediction model|
2.2. Vector Autoregressive Models (VARs)
2.3. Long Short-Term Memory Network (LSTM)
2.4. Online Learning with LSTMs
2.5. Transfer Learning with LSTMs
3. Application: Decision Support for Water Supply Systems
3.1. Water Suppply System
3.2. Decision Support Software
- The evaluated demand forecasting models are used to predict sequences of 24 h consumption profiles for one year. Each profile starts at midnight.
- The resulting profiles are fed into the decision support software and operation plans are computed.
- Beyond the simulation based on the predicted water demand and the computed operation plan, another simulation is performed based on the realized water consumption and the same operation plan.
- The latter simulation must be checked for feasibility. It stays feasible if the water level inside the storage container neither falls below the lower bound given by the firefighting reserve nor exceeds the upper bound, meaning that there is an overflow and water is wasted.
- Finally, both simulations are compared by means of the end-of-day container level of the water tank. The difference measures the prediction quality of the method for each day.
3.3. Use Cases for the Prediction of Water Demand
4. Experimental Approach
4.1. Model Parameters
4.2. Evaluation Metric for Model Comparison
5. Results for Consumption Profile Prediction
5.1. Scenario 1: Batch Learning
5.2. Online Learning
6. Results for Optimal Pump Control
Conflicts of Interest
- Roclawski, H.; Knapp, A.; Geil, C.; Böhle, M.; Krieg, H.; Nowak, D. H2Opt—Eine Software zur Entscheidungsunterstützung für die Planung und den Betrieb von Trinkwasserversorgungsanlagen. Energ. Wasser Prax. 2018, 3, 30–33. [Google Scholar]
- Adhikari, R.; Agrawal, R. An Introductory Study on Time Series Modeling and Forecasting; LAP Lambert Academic Publishing: Düsseldorf, Germany, 2013. [Google Scholar]
- Du, H.; Zhao, Z.; Xue, H. ARIMA-M: A New Model for Daily Water Consumption Prediction Based on the Autoregressive Integrated Moving Average Model and the Markov Chain Error Correction. Water 2020, 12, 760. [Google Scholar] [CrossRef][Green Version]
- Benitez, R.; Ortiz-Caraballo, C.; Preciado, J.; Conejero, J.; Sanchez, F.; Largo, A. A Short-Term Data Based Water Consumption Prediction Approach. Energies 2019, 12, 2359. [Google Scholar] [CrossRef][Green Version]
- Sutskever, I.; Vinyals, O.; Le, Q. Sequence to sequence learning with neural networks. Adv. Neural Inf. Process. Syst. 2014, 3104–3112. [Google Scholar]
- Antunes, A.; Andrade-Campos, A.; Sardinha-Lourenço, A.; Oliveira, M.S. Short-term water demand forecasting using machine learning techniques. J. Hydroinform. 2018, 20, 1343–1366. [Google Scholar] [CrossRef][Green Version]
- Poornima, P.; Boyapati, S. Prediction of Water Consumption Using Machine Learning Algorithm. In ICCCE 2020; Springer: Singapore, 2021; pp. 891–908. [Google Scholar]
- Rahim, M.S.; Nguyen, K.A.; Stewart, R.A.; Giurco, D.; Blumenstein, M. Machine Learning and Data Analytic Techniques in Digital Water Metering: A Review. Water 2020, 12, 294. [Google Scholar] [CrossRef][Green Version]
- An, A.J.; Chan, C.; Shan, N.; Cercone, N.; Ziarko, W. Applying knowledge discovery to predict water-supply consumption. IEEE Expert Intell. Syst. Appl. 1997, 12, 72–78. [Google Scholar] [CrossRef][Green Version]
- Arampatzis, G.; Perdikeas, N.; Kampragou, E.; Scaloubakas, P.; Assimacopoulos, D. A water demand forecasting methodology for supporting day-to-day management of water distribution systems. In Proceedings of the 12th International Conference “Protection & Restoration of the Environment, Skiathos Island, Greece, 29 June–3 July 2014. [Google Scholar]
- Mamo, T.G.; Juran, I.; Shahrour, I. Urban Water Demand Forecasting Using the Stochastic Nature of Short Term Historical Water Demand and supply Pattern. J. Water Resour. Hydraul. Eng. 2013, 2, 92–103. [Google Scholar]
- Adamowski, J.K.; Christina Comparison of Multivariate Regression and Artificial Neural Networks for Peak Urban Water-Demand Forecasting. Evaluation of Different ANN Learning Algorithms. J. Hydrol. Eng. 2010, 15, 729–743. [Google Scholar] [CrossRef][Green Version]
- Bougadis, J.; Adamowski, K.; Diduch, R. Short-term municipal water demand forecasting. Hydrol. Process. 2005, 19, 137–148. [Google Scholar] [CrossRef]
- Candelieri, A.; Soldi, D.; Archetti, F. Short-term forecasting of hourly water consumption by using automatic metering readers data. Procedia Eng. 2005, 119, 844–853. [Google Scholar] [CrossRef]
- Manuel, H.; Luís, T.; Joaquín, I.; Rafael, P. Predictive models for forecasting hourly urban water demand. J. Hydrol. 2010, 387, 141–150. [Google Scholar]
- Rangel, H.R.; Puig, V.; Farias, R.L.; Flores, J.J. Short Term Demand Forecast using a Bank of Neural Network Models Trained using Genetic Algorithms for the Optimal Management of Drinking Water Networks. J. Hydroinform. 2016, 19, 1–16. [Google Scholar] [CrossRef][Green Version]
- Perez, R.; Kivalov, S.; Schlemmer, J.; Hemker, K.; Renn, D.; Hoff, T.E. Validation of short and medium term operational solar forecasts in the US. Sol. Energy 2010, 84, 2161–2172. [Google Scholar] [CrossRef]
- Takeyosi, K. Prediction of photovoltaic power generation output and network operation. Integr. Distrib. Energy Resour. Power Syst. 2016, 15–20. [Google Scholar] [CrossRef]
- Kaushik, S.; Choudhury, A.; Sheron, P.; Dasgupta, N.; Natarajan, S.; Pickett, L.; Dutt, V. AI in Healthcare: Time-Series Forecasting Using Statistical, Neural, and Ensemble Architectures. Front. Big Data 2020, 3, 4. [Google Scholar] [CrossRef][Green Version]
- Shen, Y.; Giannakis, G.B.; Baingana, B. Nonlinear Structural Vector Autoregressive Models with Application to Directed Brain Networks. IEEE Trans. Signal Process. 2019, 67, 5325–5339. [Google Scholar] [CrossRef] [PubMed]
- Melnyk, I.; Matthews, B.; Valizadegan, H.; Banerjee, A.; Oza, N. Vector autoregressive model-based anomaly detection in aviation systems. Procedia Eng. 2015, 119, 442–449. [Google Scholar] [CrossRef][Green Version]
- Demir, E.; Gozgor, G.; Lau, C.; Vigne, S. Does economic policy uncertainty predict the Bitcoin returns? An empirical investigation Financ. Res. Lett. 2018, 26, 145–149. [Google Scholar] [CrossRef][Green Version]
- Konar, A.; Bhattacharya, D. Time-Series Prediction and Applications: A Machine Intelligence Approach, Intelligent Systems Reference Library; Springer International Publishing: Berlin, Germany, 2017. [Google Scholar]
- Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural Comput. 1997, 9, 1735–1780. [Google Scholar] [CrossRef]
- Carbune, V.; Gonnet, P. Fast multi-language LSTM-based online handwriting recognition. Int. J. Doc. Anal. Recognit. 2020, 1–14. [Google Scholar] [CrossRef][Green Version]
- He, T.; Droppo, J. Exploiting LSTM Structure in Deep Neural Networks for Speech Recognition. In Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China, 20–25 March 2016. [Google Scholar]
- Li, J.; Zhao, R.; Sun, E.; Wong, J.H.; Das, A.; Meng, Z.; Gong, Y. High-Accuracy and Low-Latency Speech Recognition with Two-Head Contextual Layer Trajectory LSTM Model. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, 12 October 2020; pp. 7699–7703. [Google Scholar]
- Yang, H.; Ding, K.; Qiu, R.C.; Mi, T. Remaining Useful Life Prediction Based on Normalizing Flow Embedded Sequence-to-Sequence Learning. IEEE Trans. Reliab. 2020. [Google Scholar] [CrossRef]
- Chimmula, V.K.R.; Zhang, L. Time series forecasting of COVID-19 transmission in Canada using LSTM networks. Chaos Solitons Fractals 2020, 135, 109864. [Google Scholar] [CrossRef] [PubMed]
- Xiang, Z.; Yan, J.; Demir, I. A Rainfall-Runoff Model With LSTM-Based Sequence-to-Sequence Learning. Water Resour. Res. 2020, 56, 1. [Google Scholar] [CrossRef]
- Elsheikh, A.; Katekar, V.; Muskens, O. Utilization of LSTM neural network for water production forecasting of a stepped solar still with a corrugated absorber plate. Process. Saf. Environ. Prot. 2021, 148, 273–282. [Google Scholar] [CrossRef]
- Barzegar, R.; Aalami, M.T.; Adamowski, J. Short-term water quality variable prediction using a hybrid CNN–LSTM deep learning model A Short-Term Data Based Water Consumption Prediction Approach. Stoch. Environ. Res. Risk Assess. 2020, 1–19. [Google Scholar] [CrossRef]
- Xu, Z.; Ying, Z.; Li, Y.; He, B.; Chen, Y. Pressure prediction and abnormal working conditions detection of water supply network based on LSTM. Water 2020, 3, 963–974. [Google Scholar] [CrossRef]
- Olah, C. Understanding LSTM. Available online: Http://colah.github.io/posts/2015-08-Understanding-LSTMs/ (accessed on 22 October 2020).
- Zilka, L.; Jurcicek, F.; Guruprasad, N.; Gerber, J.; Carlson, K. Predict Land Covers with Transition Modeling and Incremental Learning. In Proceedings of the International Conference on Data Mining (SIAM), Houston, TX, USA, 27–29 April 2017; pp. 171–179. [Google Scholar]
- Xiaowei, J.; Khandelwal, A. Incremental LSTM-based dialog state tracker. In Proceedings of the 2015 Ieee Workshop on Automatic Speech Recognition and Understanding (Asru), Scottsdale, AZ, USA, 13–17 December 2015; pp. 757–762. [Google Scholar]
- Lee, J.; Dernoncourt, F.; Szolovits, P. Transfer learning for named-entity recognition with neural networks. arXiv 2017, arXiv:1705.06273. [Google Scholar]
- Coutinho, E.; Deng, J.; Schuller, B. Transfer learning emotion manifestation across music and speech. In Proceedings of the International Joint Conference on Neural Networks (IJCNN), Beijing, China, 6–11 July 2014; pp. 3592–3598. [Google Scholar]
- Luna, T.; Ribau, J.; Figueiredo, D.; Alves, R. Improving energy efficiency in water supply systems with pump scheduling optimization. J. Clean. Prod. 2019, 213, 342–356. [Google Scholar] [CrossRef]
- Marchi, A.; Simpson, A.; Lambert, M. Pump Operation Optimization Using Rule-based Controls. Procedia Eng. 2017, 186, 210–217. [Google Scholar] [CrossRef]
- Mala-Jetmarova, H.; Sultanova, N.; Savic, D. Lost in Optimisation of Water Distribution Systems? A Literature Review of System Operation. Environ. Model. Softw. 2017, 93, 209–254. [Google Scholar] [CrossRef][Green Version]
- Krieg, H.; Nowak, D.; Bortz, M.; Knapp, A.; Geil, C.; Roclawski, H.; Böhle, M. Decision support for planning and operation of drinking water supply systems. Water Solut. 2018, 3, 49–60. [Google Scholar]
- Thomas, J.A. Optimisation Method for the Clear Sky PV Forecast Using Power Records from Arbitrarily Oriented Panels. In Proceedings of the 2018 7th International Conference on Renewable Energy Research and Applications (ICRERA), Paris, France, 14–17 October 2018; pp. 117–123. [Google Scholar]
|Label||Use Case 1||Use Case 2|
|Measurement||Flow in||Flow in|
|Sampling Rate||1 h||1 h|
|Days for Training||365||113|
|Days for Validation||365||111|
|Days for Test||365||111|
|Day||Naïve Predictor||LSTM 1||LSTM 2||LSTM 3||LSTM 4||VAR|
|Naïve Predictor||LSTM 1||LSTM 2||LSTM 3||LSTM 4||VAR|
|Feasible days after re-simulation||days||228||287||215||209||240||270|
|End-of-day container level differences|
|All days without additional outliers||m|
|April-September without additional outliers||m|
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Kühnert, C.; Gonuguntla, N.M.; Krieg, H.; Nowak, D.; Thomas, J.A. Application of LSTM Networks for Water Demand Prediction in Optimal Pump Control. Water 2021, 13, 644. https://doi.org/10.3390/w13050644
Kühnert C, Gonuguntla NM, Krieg H, Nowak D, Thomas JA. Application of LSTM Networks for Water Demand Prediction in Optimal Pump Control. Water. 2021; 13(5):644. https://doi.org/10.3390/w13050644Chicago/Turabian Style
Kühnert, Christian, Naga Mamatha Gonuguntla, Helene Krieg, Dimitri Nowak, and Jorge A. Thomas. 2021. "Application of LSTM Networks for Water Demand Prediction in Optimal Pump Control" Water 13, no. 5: 644. https://doi.org/10.3390/w13050644