# Application of Machine Learning in Water Resources Management: A Systematic Literature Review

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Systematic Literature Review

## 3. Major Application of ML in WRM

#### 3.1. Prediction

#### 3.1.1. Essential Data Processing in ML

#### 3.1.2. Algorithms and Metrics for Evaluation

#### 3.1.3. Applications and Challenges

#### 3.2. Clustering

#### 3.2.1. Algorithms and Metrics for Evaluation

#### 3.2.2. Clustering in the Field of WRM

#### 3.2.3. Clustering Applications and Challenges in WRM

#### 3.3. Reinforcement Learning

## 4. Less-Studied Areas for ML Applications in WRM

## 5. Challenges and Future Research

## 6. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## Nomenclature

A2c-A3c | Asynchronous Advantage Actor-Critic |

ABM | Agent-based modeling |

AGNES | Agglomerative nesting |

AI | Artificial intelligence |

AR | Autoregressive |

ARIMA | AR-integrated moving average |

BART | Bayesian additive regression tree |

BNN | Bayesian neural network |

BP-ANN | Back-propagation artificial neural network |

C51 | Categorical 51-Atom DQN |

CAS | Complex adaptive systems |

CLARA | Clustering Large Applications |

CLARANS | Clustering Large Applications based on RANdomized Search |

CNN | Convolutional neural networks |

DBCLASD | Distribution-based clustering of large spatial databases |

DBSCAN | Density-based spatial clustering of applications with noise |

DDPG | Deep Deterministic Policy Gradient |

DDQN | Double DQN |

DIANA | Divisive clustering analysis |

DL | Deep learning |

DM | Data management |

DM | Decision maker |

DNN | Deep neural networks |

DP | Dynamic programming |

DQN | Deep Q-Network |

DRL | Deep reinforcement learning |

ELM | Extreme learning machine |

EMD | Empirical mode decomposition |

GA | Genetic algorithm |

GNG | Growing neural gas |

GRU | Gated recurrent unit |

GWL | Groundwater level |

HER | Hindsight Experience Replay |

I2A | Imagination-Augmented Agents |

IHP | Intergovernmental Hydrological Programme |

KNN | K-nearest neighbors |

LSTF | Long sequence time-series forecasting |

LSTM | Long and short-term memory |

LSTNet | Long- and Short-term Time-series network |

MBMF | Model-Based RL with Model-Free Fine-Tuning |

MBVE | Model-Based Value Expansion |

MC | Monte Carlo |

MDP | Markov Decision Process |

ML | Machine learning |

MLP | multi-layer perceptron |

MODWT | Maximal Overlap Discrete Wavelet Transform |

MTS | Multivariate time series |

PCSA | parallel cooperation search algorithm |

PPO | Proximal Policy Optimization |

QR-DQN | Quantile Regression DQN |

RF | Random forest |

RL | Reinforcement learning |

RNN | Recurrent neural networks |

SAC | Soft Actor-Critic |

SARSA | State–action–reward–state–action |

STDM | Serial triple diagram model |

SVM | Support vector machine |

SWOT | Strengths, weaknesses, opportunities, and threats |

TD | Temporal Difference |

TD3 | Twin Delayed DDPG |

TRPO | Trust Region Policy Optimization |

## References

- Razavi, S.; Hannah, D.M.; Elshorbagy, A.; Kumar, S.; Marshall, L.; Solomatine, D.P.; Dezfuli, A.; Sadegh, M.; Famiglietti, J. Coevolution of Machine Learning and Process-Based Modelling to Revolutionize Earth and Environmental Sciences A Perspective. Hydrol. Process.
**2022**, 36, e14596. [Google Scholar] [CrossRef] - Reinsel, D.; Gantz, J.; Rydning, J. The Digitization of the World From Edge to Core; International Data Corporation: Framingham, MA, USA, 2018; Available online: https://www.seagate.com/files/www-content/our-story/trends/files/idc-seagate-dataage-whitepaper.pdf (accessed on 7 October 2022).
- UNESCO IHP-IX: Strategic Plan of the Intergovernmental Hydrological Programme: Science for a Water Secure World in a Changing Environment, Ninth Phase 2022-2029. 2022. Available online: https://unesdoc.unesco.org/ark:/48223/pf0000381318 (accessed on 7 October 2022).
- Ibrahim, K.S.M.H.; Huang, Y.F.; Ahmed, A.N.; Koo, C.H.; El-Shafie, A. A Review of the Hybrid Artificial Intelligence and Optimization Modelling of Hydrological Streamflow Forecasting. Alexandria Eng. J.
**2022**, 61, 279–303. [Google Scholar] [CrossRef] - Mosaffa, H.; Sadeghi, M.; Mallakpour, I.; Naghdyzadegan Jahromi, M.; Pourghasemi, H.R. Application of Machine Learning Algorithms in Hydrology. Comput. Earth Environ. Sci.
**2022**, 585–591. [Google Scholar] [CrossRef] - Ahmadi, A.; Olyaei, M.; Heydari, Z.; Emami, M.; Zeynolabedin, A.; Ghomlaghi, A.; Daccache, A.; Fogg, G.E.; Sadegh, M. Groundwater Level Modeling with Machine Learning: A Systematic Review and Meta-Analysis. Water
**2022**, 14, 949. [Google Scholar] [CrossRef] - Bernardes, J.; Santos, M.; Abreu, T.; Prado, L.; Miranda, D.; Julio, R.; Viana, P.; Fonseca, M.; Bortoni, E.; Bastos, G.S. Hydropower Operation Optimization Using Machine Learning: A Systematic Review. AI
**2022**, 3, 78–99. [Google Scholar] [CrossRef] - Tao, H.; Hameed, M.M.; Marhoon, H.A.; Zounemat-Kermani, M.; Heddam, S.; Sungwon, K.; Sulaiman, S.O.; Tan, M.L.; Sa’adi, Z.; Mehr, A.D.; et al. Groundwater Level Prediction Using Machine Learning Models: A Comprehensive Review. Neurocomputing
**2022**, 489, 271–308. [Google Scholar] [CrossRef] - Dikshit, A.; Pradhan, B.; Santosh, M. Artificial Neural Networks in Drought Prediction in the 21st Century–A Scientometric Analysis. Appl. Soft Comput.
**2022**, 114, 108080. [Google Scholar] [CrossRef] - Ewuzie, U.; Bolade, O.P.; Egbedina, A.O. Application of Deep Learning and Machine Learning Methods in Water Quality Modeling and Prediction: A Review. Curr. Trends Adv. Comput. Intell. Environ. Data Eng.
**2022**, 185–218. [Google Scholar] [CrossRef] - Zhu, M.; Wang, J.; Yang, X.; Zhang, Y.; Zhang, L.; Ren, H.; Wu, B.; Ye, L. A Review of the Application of Machine Learning in Water Quality Evaluation. Eco-Environment Heal.
**2022**, 1, 107–116. [Google Scholar] [CrossRef] - Saha, S.; Mallik, S.; Mishra, U. Groundwater Depth Forecasting Using Machine Learning and Artificial Intelligence Techniques: A Survey of the Literature. Lect. Notes Civ. Eng.
**2022**, 207, 153–167. [Google Scholar] [CrossRef] - Hamitouche, M.; Molina, J.-L. A Review of AI Methods for the Prediction of High-Flow Extremal Hydrology. Water Resour. Manag.
**2022**, 36, 3859–3876. [Google Scholar] [CrossRef] - Nourani, V.; Paknezhad, N.J.; Tanaka, H. Prediction Interval Estimation Methods for Artificial Neural Network (ANN)-Based Modeling of the Hydro-Climatic Processes, a Review. Sustain
**2021**, 13, 1633. [Google Scholar] [CrossRef] - Guo, Y.; Zhang, Y.; Zhang, L.; Wang, Z. Regionalization of Hydrological Modeling for Predicting Streamflow in Ungauged Catchments: A Comprehensive Review. Wiley Interdiscip. Rev. Water
**2021**, 8, e1487. [Google Scholar] [CrossRef] - Gupta, D.; Hazarika, B.B.; Berlin, M.; Sharma, U.M.; Mishra, K. Artificial Intelligence for Suspended Sediment Load Prediction: A Review. Environ. Earth Sci.
**2021**, 80, 346. [Google Scholar] [CrossRef] - Ahansal, Y.; Bouziani, M.; Yaagoubi, R.; Sebari, I.; Sebari, K.; Kenny, L. Towards Smart Irrigation: A Literature Review on the Use of Geospatial Technologies and Machine Learning in the Management of Water Resources in Arboriculture. Agronomy
**2022**, 12, 297. [Google Scholar] [CrossRef] - Kikon, A.; Deka, P.C. Artificial Intelligence Application in Drought Assessment, Monitoring and Forecasting: A Review. Stoch. Environ. Res. Risk Assess.
**2021**, 36, 1197–1214. [Google Scholar] [CrossRef] - Ghobadi, F.; Kang, D. Multi-Step Ahead Probabilistic Forecasting of Daily Streamflow Using Bayesian Deep Learning: A Multiple Case Study. Water
**2022**, 14, 3672. [Google Scholar] [CrossRef] - Ghobadi, F.; Kang, D. Improving Long-Term Streamflow Prediction in a Poorly Gauged Basin Using Geo-Spatiotemporal Mesoscale Data and Attention-Based Deep Learning: A Comparative Study. J. Hydrol.
**2022**, 615, 128608. [Google Scholar] [CrossRef] - Ikram, R.M.A.; Hazarika, B.B.; Gupta, D.; Heddam, S.; Kisi, O. Streamflow Prediction in Mountainous Region Using New Machine Learning and Data Preprocessing Methods: A Case Study. Neural Comput. Appl.
**2022**, 1–18. Available online: https://link.springer.com/article/10.1007/s00521-022-08163-8 (accessed on 7 October 2022). [CrossRef] - Granata, F.; Di Nunno, F.; de Marinis, G. Stacked Machine Learning Algorithms and Bidirectional Long Short-Term Memory Networks for Multi-Step Ahead Streamflow Forecasting: A Comparative Study. J. Hydrol.
**2022**, 613, 128431. [Google Scholar] [CrossRef] - Granata, F.; Di Nunno, F.; Najafzadeh, M.; Demir, I. A Stacked Machine Learning Algorithm for Multi-Step Ahead Prediction of Soil Moisture. Hydrology
**2022**, 10, 1. [Google Scholar] [CrossRef] - Zhang, Y.; Li, C.; Jiang, Y.; Sun, L.; Zhao, R.; Yan, K.; Wang, W. Accurate Prediction of Water Quality in Urban Drainage Network with Integrated EMD-LSTM Model. J. Clean. Prod.
**2022**, 354, 131724. [Google Scholar] [CrossRef] - Li, Z.; Zhang, C.; Liu, H.; Zhang, C.; Zhao, M.; Gong, Q.; Fu, G. Developing Stacking Ensemble Models for Multivariate Contamination Detection in Water Distribution Systems. Sci. Total Environ.
**2022**, 828, 154284. [Google Scholar] [CrossRef] [PubMed] - Yang, Z.; Zou, L.; Xia, J.; Qiao, Y.; Cai, D. Inner Dynamic Detection and Prediction of Water Quality Based on CEEMDAN and GA-SVM Models. Remote Sens.
**2022**, 14, 1714. [Google Scholar] [CrossRef] - Lee, S.S.; Lee, H.H.; Lee, Y.J. Prediction of Minimum Night Flow for Enhancing Leakage Detection Capabilities in Water Distribution Networks. Appl. Sci.
**2022**, 12, 6467. [Google Scholar] [CrossRef] - Kizilöz, B. Prediction of Daily Failure Rate Using the Serial Triple Diagram Model and Artificial Neural Network. Water Supply
**2022**, 22, 7040–7058. [Google Scholar] [CrossRef] - Zanfei, A.; Brentan, B.M.; Menapace, A.; Righetti, M.; Herrera, M. Graph Convolutional Recurrent Neural Networks for Water Demand Forecasting. Water Resour. Res.
**2022**, 58, e2022WR032299. [Google Scholar] [CrossRef] - Kim, J.; Lee, H.; Lee, M.; Han, H.; Kim, D.; Kim, H.S. Development of a Deep Learning-Based Prediction Model for Water Consumption at the Household Level. Water
**2022**, 14, 1512. [Google Scholar] [CrossRef] - Zhang, D.; Wang, D.; Peng, Q.; Lin, J.; Jin, T.; Yang, T.; Sorooshian, S.; Liu, Y. Prediction of the Outflow Temperature of Large-Scale Hydropower Using Theory-Guided Machine Learning Surrogate Models of a High-Fidelity Hydrodynamics Model. J. Hydrol.
**2022**, 606, 127427. [Google Scholar] [CrossRef] - Drakaki, K.K.; Sakki, G.K.; Tsoukalas, I.; Kossieris, P.; Efstratiadis, A. Day-Ahead Energy Production in Small Hydropower Plants: Uncertainty-Aware Forecasts through Effective Coupling of Knowledge and Data. Adv. Geosci.
**2022**, 56, 155–162. [Google Scholar] [CrossRef] - Sattar Hanoon, M.; Najah Ahmed, A.; Razzaq, A.; Oudah, A.Y.; Alkhayyat, A.; Feng Huang, Y.; kumar, P.; El-Shafie, A. Prediction of Hydropower Generation via Machine Learning Algorithms at Three Gorges Dam, China. Ain Shams Eng. J.
**2022**, 14, 101919. [Google Scholar] [CrossRef] - Park, K.; Jung, Y.; Seong, Y.; Lee, S. Development of Deep Learning Models to Improve the Accuracy of Water Levels Time Series Prediction through Multivariate Hydrological Data. Water
**2022**, 14, 469. [Google Scholar] [CrossRef] - Khosravi, K.; Golkarian, A.; Tiefenbacher, J.P. Using Optimized Deep Learning to Predict Daily Streamflow: A Comparison to Common Machine Learning Algorithms. Water Resour. Manag.
**2022**, 36, 699–716. [Google Scholar] [CrossRef] - Sun, J.; Hu, L.; Li, D.; Sun, K.; Yang, Z. Data-Driven Models for Accurate Groundwater Level Prediction and Their Practical Significance in Groundwater Management. J. Hydrol.
**2022**, 608, 127630. [Google Scholar] [CrossRef] - Sharghi, E.; Nourani, V.; Zhang, Y.; Ghaneei, P. Conjunction of Cluster Ensemble-Model Ensemble Techniques for Spatiotemporal Assessment of Groundwater Depletion in Semi-Arid Plains. J. Hydrol.
**2022**, 610, 127984. [Google Scholar] [CrossRef] - Liu, J.; Yuan, X.; Zeng, J.; Jiao, Y.; Li, Y.; Zhong, L.; Yao, L. Ensemble Streamflow Forecasting over a Cascade Reservoir Catchment with Integrated Hydrometeorological Modeling and Machine Learning. Hydrol. Earth Syst. Sci.
**2022**, 26, 265–278. [Google Scholar] [CrossRef] - Cho, K.; Kim, Y. Improving Streamflow Prediction in the WRF-Hydro Model with LSTM Networks. J. Hydrol.
**2022**, 605, 127297. [Google Scholar] [CrossRef] - Apaydin, H.; Taghi Sattari, M.; Falsafian, K.; Prasad, R. Artificial Intelligence Modelling Integrated with Singular Spectral Analysis and Seasonal-Trend Decomposition Using Loess Approaches for Streamflow Predictions. J. Hydrol.
**2021**, 600, 126506. [Google Scholar] [CrossRef] - Peng, L.; Wu, H.; Gao, M.; Yi, H.; Xiong, Q.; Yang, L. TLT: Recurrent Fine-Tuning Transfer Learning for Water Quality Long-Term Prediction. Water Res.
**2022**, 225, 119171. [Google Scholar] [CrossRef] - Feng, Z.K.; Shi, P.F.; Yang, T.; Niu, W.J.; Zhou, J.Z.; Cheng, C. tian Parallel Cooperation Search Algorithm and Artificial Intelligence Method for Streamflow Time Series Forecasting. J. Hydrol.
**2022**, 606, 127434. [Google Scholar] [CrossRef] - Nguyen, D.H.; Le, X.H.; Anh, D.T.; Kim, S.H.; Bae, D.H. Hourly Streamflow Forecasting Using a Bayesian Additive Regression Tree Model Hybridized with a Genetic Algorithm. J. Hydrol.
**2022**, 606, 127445. [Google Scholar] [CrossRef] - Liu, Y.; Hou, G.; Huang, F.; Qin, H.; Wang, B.; Yi, L. Directed Graph Deep Neural Network for Multi-Step Daily Streamflow Forecasting. J. Hydrol.
**2022**, 607, 127515. [Google Scholar] [CrossRef] - Adnan, R.M.; Mostafa, R.R.; Elbeltagi, A.; Yaseen, Z.M.; Shahid, S.; Kisi, O. Development of New Machine Learning Model for Streamflow Prediction: Case Studies in Pakistan. Stoch. Environ. Res. Risk Assess.
**2022**, 36, 999–1033. [Google Scholar] [CrossRef] - Adnan, R.M.; R. Mostafa, R.; Kisi, O.; Yaseen, Z.M.; Shahid, S.; Zounemat-Kermani, M. Improving Streamflow Prediction Using a New Hybrid ELM Model Combined with Hybrid Particle Swarm Optimization and Grey Wolf Optimization. Knowledge-Based Syst.
**2021**, 230, 107379. [Google Scholar] [CrossRef] - Ikram, R.M.A.; Ewees, A.A.; Parmar, K.S.; Yaseen, Z.M.; Shahid, S.; Kisi, O. The Viability of Extended Marine Predators Algorithm-Based Artificial Neural Networks for Streamflow Prediction. Appl. Soft Comput.
**2022**, 131, 109739. [Google Scholar] [CrossRef] - Chavent, M.; Lechevallier, Y.; Briant, O. DIVCLUS-T: A Monothetic Divisive Hierarchical Clustering Method. Comput. Stat. Data Anal.
**2007**, 52, 687–701. [Google Scholar] [CrossRef] - Tokuda, E.K.; Comin, C.H.; Costa, L. da F. Revisiting Agglomerative Clustering. Phys. Stat. Mech. Its Appl.
**2022**, 585, 126433. [Google Scholar] [CrossRef] - Caliñski, T.; Harabasz, J. A Dendrite Method Foe Cluster Analysis. Commun. Stat.
**1974**, 3, 1–27. [Google Scholar] [CrossRef] - Chou, C.H.; Su, M.C.; Lai, E. A New Cluster Validity Measure and Its Application to Image Compression. Pattern Anal. Appl.
**2004**, 7, 205–220. [Google Scholar] [CrossRef] - Dunn, J.C. A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters. J. Cybern
**2008**, 3, 32–57. [Google Scholar] [CrossRef] - Wang, J.S.; Chiang, J.C. A Cluster Validity Measure with Outlier Detection for Support Vector Clustering. IEEE Trans. Syst. Man, Cybern. Cybern.
**2008**, 38, 78–89. [Google Scholar] [CrossRef] [PubMed] - Kim, M.; Ramakrishna, R.S. New Indices for Cluster Validity Assessment. Pattern Recognit. Lett.
**2005**, 26, 2353–2363. [Google Scholar] [CrossRef] - Rousseeuw, P.J. Silhouettes: A Graphical Aid to the Interpretation and Validation of Cluster Analysis. J. Comput. Appl. Math.
**1987**, 20, 53–65. [Google Scholar] [CrossRef] - Marsili, F.; Bödefeld, J. Integrating Cluster Analysis into Multi-Criteria Decision Making for Maintenance Management of Aging Culverts. Mathematics
**2021**, 9, 2549. [Google Scholar] [CrossRef] - Javed, A. Cluster Analysis of Time Series Data with Application to Hydrological Events and Serious Illness Conversations. Graduate College Dissertations and Theses, The University of Vermont and State Agricultural College, Burlington, Vermont, 2021. [Google Scholar]
- Hamed Javadi, S.; Guerrero, A.; Mouazen, A.M. Clustering and Smoothing Pipeline for Management Zone Delineation Using Proximal and Remote Sensing. Sensors
**2022**, 22, 645. [Google Scholar] [CrossRef] - Zhang, R.; Chen, Y.; Zhang, X.; Ma, Q.; Ren, L. Mapping Homogeneous Regions for Flash Floods Using Machine Learning: A Case Study in Jiangxi Province, China. Int. J. Appl. Earth Obs. Geoinf.
**2022**, 108, 102717. [Google Scholar] [CrossRef] - Tshimanga, R.M.; Bola, G.B.; Kabuya, P.M.; Nkaba, L.; Neal, J.; Hawker, L.; Trigg, M.A.; Bates, P.D.; Hughes, D.A.; Laraque, A.; et al. Towards a Framework of Catchment Classification for Hydrologic Predictions and Water Resources Management in the Ungauged Basin of the Congo River: An a priori approach. In Congo Basin Hydrology, Climate, and Biogeochemistry: A Foundation for the Future; Wiley: New York, NY, USA. [CrossRef]
- Kebebew, A.S.; Awass, A.A. Regionalization of Catchments for Flood Frequency Analysis for Data Scarce Rift Valley Lakes Basin, Ethiopia. J. Hydrol. Reg. Stud.
**2022**, 43, 101187. [Google Scholar] [CrossRef] - Sikorska-Senoner, A.E. Clustering Model Responses in the Frequency Space for Improved Simulation-Based Flood Risk Studies: The Role of a Cluster Number. J. Flood Risk Manag.
**2022**, 15, e12772. [Google Scholar] [CrossRef] - Li, K.; Huang, G.; Wang, S.; Baetz, B.; Xu, W. A Stepwise Clustered Hydrological Model for Addressing the Temporal Autocorrelation of Daily Streamflows in Irrigated Watersheds. Water Resour. Res.
**2022**, 58, e2021WR031065. [Google Scholar] [CrossRef] - Bajracharya, P.; Jain, S. Hydrologic Similarity Based on Width Function and Hypsometry: An Unsupervised Learning Approach. Comput. Geosci.
**2022**, 163, 105097. [Google Scholar] [CrossRef] - Zhong, C.; Wang, H.; Yang, Q. Hydrochemical Interpretation of Groundwater in Yinchuan Basin Using Self-Organizing Maps and Hierarchical Clustering. Chemosphere
**2022**, 309, 136787. [Google Scholar] [CrossRef] [PubMed] - He, B.; He, J.T.; Zeng, Y.; Sun, J.; Peng, C.; Bi, E. Coupling of Multi-Hydrochemical and Statistical Methods for Identifying Apparent Background Levels of Major Components and Anthropogenic Anomalous Activities in Shallow Groundwater of the Liujiang Basin, China. Sci. Total Environ.
**2022**, 838, 155905. [Google Scholar] [CrossRef] [PubMed] - Karypis, M.S.G.; Kumar, V.; Steinbach, M. A comparison of document clustering techniques. In TextMining Workshop at KDD2000; Boston, MA, USA, 2000; pp. 428–439. [Google Scholar]
- Eskandarnia, E.; Al-Ammal, H.M.; Ksantini, R. An Embedded Deep-Clustering-Based Load Profiling Framework. Sustain. Cities Soc.
**2022**, 78, 103618. [Google Scholar] [CrossRef] - Gao, K.; Khan, H.A.; Qu, W. Clustering with Missing Features: A Density-Based Approach. Symmetry
**2022**, 14, 60. [Google Scholar] [CrossRef] - Ezugwu, A.E.; Ikotun, A.M.; Oyelade, O.O.; Abualigah, L.; Agushaka, J.O.; Eke, C.I.; Akinyelu, A.A. A Comprehensive Survey of Clustering Algorithms: State-of-the-Art Machine Learning Applications, Taxonomy, Challenges, and Future Research Prospects. Eng. Appl. Artif. Intell.
**2022**, 110, 104743. [Google Scholar] [CrossRef] - Arsene, D.; Predescu, A.; Truica, C.-O.; Apostol, E.-S.; Mocanu, M.; Chiru, C. Consumer Profiling Using Clustering Methods for Georeferenced Decision Support in a Water Distribution System. In 2022 14th International Conference on Electronics, Computers and Artificial Intelligence (ECAI); IEEE: Toulouse, France, 2022; pp. 1–6. [Google Scholar] [CrossRef]
- Arsene, D.; Predescu, A.; Truica, C.O.; Apostol, E.S.; Mocanu, M.; Chiru, C. Clustering Consumption Activities in a Water Monitoring System. In Proceedings of the 2022 IEEE International Conference on Automation, Quality and Testing, Robotics (AQTR), Cluj-Napoca, Romania, 19–21 May 2022. [Google Scholar] [CrossRef]
- Bodereau, N.; Delaval, A.; Lepage, H.; Eyrolle, F.; Raimbault, P.; Copard, Y. Hydrological Classification by Clustering Approach of Time-Integrated Samples at the Outlet of the Rhône River: Application to Δ14C-POC. Water Res.
**2022**, 220, 118652. [Google Scholar] [CrossRef] [PubMed] - Rami, O.; Hasnaoui, M.D.; Ouazar, D.; Bouziane, A. A Mixed Clustering-Based Approach for a Territorial Hydrological Regionalization. Arab J Geosci.
**2022**, 15, 75. [Google Scholar] [CrossRef] - Arsene, D.; Predescu, A.; Pahont, B.; Gabriel Chiru, C.; Apostol, E.-S.; Truică, C.-O. Advanced Strategies for Monitoring Water Consumption Patterns in Households Based on IoT and Machine Learning. Water
**2022**, 14, 2187. [Google Scholar] [CrossRef] - Morbidelli, R.; Prakaisak, I.; Wongchaisuwat, P. Hydrological Time Series Clustering: A Case Study of Telemetry Stations in Thailand. Water
**2022**, 14, 2095. [Google Scholar] [CrossRef] - Nourani, V.; Ghaneei, P.; Kantoush, S.A. Robust Clustering for Assessing the Spatiotemporal Variability of Groundwater Quantity and Quality. J. Hydrol.
**2022**, 604, 127272. [Google Scholar] [CrossRef] - Wainwright, H.M.; Uhlemann, S.; Franklin, M.; Falco, N.; Bouskill, N.J.; Newcomer, M.E.; Dafflon, B.; Siirila-Woodburn, E.R.; Minsley, B.J.; Williams, K.H.; et al. Watershed Zonation through Hillslope Clustering for Tractably Quantifying Above-and below-Ground Watershed Heterogeneity and Functions. Hydrol. Earth Syst. Sci.
**2022**, 26, 429–444. [Google Scholar] [CrossRef] - Yin, J.; Medellín-Azuara, J.; Escriva-Bou, A. Groundwater Levels Hierarchical Clustering and Regional Groundwater Drought Assessment in Heavily Drafted Aquifers. Hydrol. Res.
**2022**, 53, 1031–1046. [Google Scholar] [CrossRef] - Akstinas, V.; Šarauskienė, D.; Kriaučiūnienė, J.; Nazarenko, S.; Jakimavičius, D. Spatial and Temporal Changes in Hydrological Regionalization of Lowland Rivers. Int. J. Environ. Res.
**2021**, 16, 1–14. [Google Scholar] [CrossRef] - Eskandari, E.; Mohammadzadeh, H.; Nassery, H.; Vadiati, M.; Zadeh, A.M.; Kisi, O. Delineation of Isotopic and Hydrochemical Evolution of Karstic Aquifers with Different Cluster-Based (HCA, KM, FCM and GKM) Methods. J. Hydrol.
**2022**, 609, 127706. [Google Scholar] [CrossRef] - Lee, Y.; Toharudin, T.; Chen, R.-C.; Kuswanto, H.; Noh, M.; Mariana Che Mat Nor, S.; Milleana Shaharudin, S.; Ismail, S.; Aimi Mohd Najib, S.; Leong Tan, M.; et al. Statistical Modeling of RPCA-FCM in Spatiotemporal Rainfall Patterns Recognition. Atmosphere
**2022**, 13, 145. [Google Scholar] [CrossRef] - Akbarian, H.; Gheibi, M.; Hajiaghaei-Keshteli, M.; Rahmani, M. A Hybrid Novel Framework for Flood Disaster Risk Control in Developing Countries Based on Smart Prediction Systems and Prioritized Scenarios. J. Environ. Manag.
**2022**, 312, 114939. [Google Scholar] [CrossRef] - James, N.; Bondell, H. Temporal and Spectral Governing Dynamics of Australian Hydrological Streamflow Time Series. J. Comput. Sci.
**2022**, 63, 101767. [Google Scholar] [CrossRef] - Xu, M.; Wang, G.; Wang, Z.; Hu, H.; Kumar Singh, D.; Tian, S. Temporal and Spatial Hydrological Variations of the Yellow River in the Past 60 Years. J. Hydrol.
**2022**, 609, 127750. [Google Scholar] [CrossRef] - Giełczewski, M.; Piniewski, M.; Domański, P.D. Mixed Statistical and Data Mining Analysis of River Flow and Catchment Properties at Regional Scale. Stoch. Environ. Res. Risk Assess.
**2022**, 36, 2861–2882. [Google Scholar] [CrossRef] - Tafvizi, A.; James, A.L.; Yao, H.; Stadnyk, T.A.; Ramcharan, C. Investigating Hydrologic Controls on 26 Precambrian Shield Catchments Using Landscape, Isotope Tracer and Flow Metrics. Hydrol. Process.
**2022**, 36, e14528. [Google Scholar] [CrossRef] - Hung, F.; Son, K.; Yang, Y.C.E. Investigating Uncertainties in Human Adaptation and Their Impacts on Water Scarcity in the Colorado River Basin, United States. J. Hydrol.
**2022**, 612, 128015. [Google Scholar] [CrossRef] - Saber, J.; Hales, R.C.; Sowby, R.B.; Williams, G.P.; James Nelson, E.; Ames, D.P.; Dundas, J.B.; Ogden, J. SABER: A Model-Agnostic Postprocessor for Bias Correcting Discharge from Large Hydrologic Models. Hydrology
**2022**, 9, 113. [Google Scholar] [CrossRef] - Tang, T.; Jiao, D.; Chen, T.; Gui, G. Medium- and Long-Term Precipitation Forecasting Method Based on Data Augmentation and Machine Learning Algorithms. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.
**2022**, 15, 1000–1011. [Google Scholar] [CrossRef] - Rahman, A.T.M.S.; Kono, Y.; Hosono, T. Self-Organizing Map Improves Understanding on the Hydrochemical Processes in Aquifer Systems. Sci. Total Environ.
**2022**, 846, 157281. [Google Scholar] [CrossRef] [PubMed] - Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A.A.; Veness, J.; Bellemare, M.G.; Graves, A.; Riedmiller, M.; Fidjeland, A.K.; Ostrovski, G.; et al. Human-Level Control through Deep Reinforcement Learning. Nature
**2015**, 518, 529–533. [Google Scholar] [CrossRef] [PubMed] - Géron, A. Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow SECOND EDITION Concepts, Tools, and Techniques to Build Intelligent Systems; O’Reilly Media, Inc.: Sebastopol, CA, USA, 2022. [Google Scholar]
- Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT press: Cambridge, MA, USA, 2016; Volume 521, ISBN 978-0262035613. [Google Scholar]
- Agha-Hoseinali-Shirazi, M.; Bozorg-Haddad, O.; Laituri, M.; DeAngelis, D. Application of Agent Base Modeling in Water Resources Management and Planning. Springer Water
**2021**, 177–216. [Google Scholar] [CrossRef] - Jang, B.; Kim, M.; Harerimana, G.; Kim, J.W. Q-Learning Algorithms: A Comprehensive Classification and Applications. IEEE Access
**2019**, 7, 133653–133667. [Google Scholar] [CrossRef] - Zhu, Z.; Hu, Z.; Chan, K.W.; Bu, S.; Zhou, B.; Xia, S. Reinforcement Learning in Deregulated Energy Market: A Comprehensive Review. Appl. Energy
**2023**, 329, 120212. [Google Scholar] [CrossRef] - Nurcahyono, A.; Fadhly Jambak, F.; Rohman, A.; Faisal Karim, M.; Neves-Silva, P.; Cruz Foundation, O.; Horizonte, B. Shifting the Water Paradigm from Social Good to Economic Good and the State’s Role in Fulfilling the Right to Water. F1000Research
**2022**, 11, 490. [Google Scholar] [CrossRef] - Damascene123, N.J.; Dithebe, M.; Laryea, A.E.N.; Medina, J.A.M.; Bian, Z.; Gilbert, M.A.S.E.N.G.O. Prospective Review of Mining Effects on Hydrology in a Water-Scarce Eco-Environment; North American Academic Research: San Francisco, CA, USA, 2022; Volume 5, pp. 352–365. [Google Scholar] [CrossRef]
- Yan, B.; Jiang, H.; Zou, Y.; Liu, Y.; Mu, R.; Wang, H. An Integrated Model for Optimal Water Resources Allocation under “3 Redlines” Water Policy of the Upper Hanjiang River Basin. J. Hydrol. Reg. Stud.
**2022**, 42, 101167. [Google Scholar] [CrossRef] - Xiao, Y.; Fang, L.; Hipel, K.W.; Wre, H.D.; Asce, F. Agent-Based Modeling Approach to Investigating the Impact of Water Demand Management; American Society of Civil Engineers (ASCE): Reston, VA, USA, 2018. [Google Scholar] [CrossRef]
- Lin, Z.; Lim, S.H.; Lin, T.; Borders, M. Using Agent-Based Modeling for Water Resources Management in the Bakken Region. J. Water Resour. Plan. Manag.
**2020**, 146, 05019020. [Google Scholar] [CrossRef] - Berglund, E.Z.; Asce, M. Using Agent-Based Modeling for Water Resources Planning and Management. J. Water Resour. Plan. Manag.
**2015**, 141, 04015025. [Google Scholar] [CrossRef] - Tourigny, A.; Filion, Y. Sensitivity Analysis of an Agent-Based Model Used to Simulate the Spread of Low-Flow Fixtures for Residential Water Conservation and Evaluate Energy Savings in a Canadian Water Distribution System. J. Water Resour. Plan.
**2019**, 145, 1. [Google Scholar] [CrossRef] - Giacomoni, M.H.; Berglund, E.Z. Complex Adaptive Modeling Framework for Evaluating Adaptive Demand Management for Urban Water Resources Sustainability. J. Water Resour. Plan. Manag.
**2015**, 141, 11. [Google Scholar] [CrossRef] - Tensorforce: A TensorFlow Library for Applied Reinforcement Learning—Tensorforce 0.6.5 Documentation. Available online: https://tensorforce.readthedocs.io/en/latest/ (accessed on 26 October 2022).
- Plappert, M. keras-rl. “GitHub—Keras-rl/Keras-rl: Deep Reinforcement Learning for Keras.” GitHub Repos. 2019. Available online: https://github.com/keras-rl/keras-rl (accessed on 26 October 2022).
- Guadarrama, S.; Korattikara, A.; Ramirez, O.; Castro, P.; Holly, E.; Fishman, S.; Wang, K.; Gonina, E.; Wu, N.; Kokiopoulou, E.; et al. TF-Agents: A Library for Reinforcement Learning in Tensorflow. GitHub Repos. 2018. Available online: https://github.com/tensorflow/agents (accessed on 26 October 2022).
- Caspi, I.; Leibovich, G.; Novik, G.; Endrawis, S. Reinforcement Learning Coach, December 2017. [CrossRef]
- Hoffman, M.W.; Shahriari, B.; Aslanides, J.; Barth-Maron, G.; Nikola Momchev, D.; Sinopalnikov, D.; Stańczyk, P.; Ramos, S.; Raichuk, A.; Vincent, D.; et al. Acme: A Research Framework for Distributed Reinforcement Learning. arXiv
**2020**, arXiv:2006.00979. [Google Scholar] - Castro, P.S.; Moitra, S.; Gelada, C.; Kumar, S.; Bellemare, M.G.; Brain, G. Dopamine: A Research Framework for Deep Reinforcement Learning. arXiv
**2018**, arXiv:1812.06110. [Google Scholar] - Liang, E.; Liaw, R.; Nishihara, R.; Moritz, P.; Fox, R.; Gonzalez, J.; Goldberg, K. Ray RLlib: A Composable and Scalable Reinforcement Learning Library. arXiv
**2017**, preprint. arXiv:1712.09381. [Google Scholar] - Raffin, A.; Hill, A.; Gleave, A.; Kanervisto, A.; Ernestus, M.; Dormann, N. Stable-Baselines3: Reliable Reinforcement Learning Implementations. J. Mach. Learn. Res.
**2021**, 22, 12348–12355. [Google Scholar] - Van Hasselt, H.; Guez, A.; Silver, D. Deep Reinforcement Learning with Double Q-Learning. Proc. AAAI Conf. Artif. Intell.
**2016**, 30, 2094–2100. [Google Scholar] [CrossRef] - Wang, Z.; Schaul, T.; Hessel, M.; Hasselt, H.; Lanctot, M.; Freitas, N. Dueling Network Architectures for Deep Reinforcement Learning 2016. In Proceedings of the 33rd International Conference on Machine Learning, New York, NY USA, 19–24 June 2016; pp. 1995–2003. [Google Scholar]
- PFRL, a Deep Reinforcement Learning Library — PFRL 0.3.0 Documentation. Available online: https://pfrl.readthedocs.io/en/latest/ (accessed on 26 October 2022).
- Mnih, V.; Badia, A.P.; Mirza, L.; Graves, A.; Harley, T.; Lillicrap, T.P.; Silver, D.; Kavukcuoglu, K. Asynchronous Methods for Deep Reinforcement Learning. arXiv
**2016**, arXiv:1602.01783v2. [Google Scholar] - Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; Klimov, O. Proximal Policy Optimization Algorithms. arXiv
**2017**, arXiv:1707.06347. [Google Scholar] - Racanière, S.; Weber, T.; Reichert, D.P.; Buesing, L.; Guez, A.; Rezende, D.; Badia, A.P.; Vinyals, O.; Heess, N.; Li, Y.; et al. Imagination-Augmented Agents for Deep Reinforcement Learning. arxiv
**2017**, preprint. arXiv:1707.06203. [Google Scholar] - Feinberg, V.; Wan, A.; Stoica, I.; Jordan, M.I.; Gonzalez, J.E.; Levine, S. Model-Based Value Estimation for Efficient Model-Free Reinforcement Learning. arxiv
**2018**, arXiv:1803.00101. [Google Scholar] - Silver, D.; Hubert, T.; Schrittwieser, J.; Antonoglou, I.; Lai, M.; Guez, A.; Lanctot, M.; Sifre, L.; Kumaran, D.; Graepel, T.; et al. Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm. arXiv
**2017**, arXiv:1712.01815. [Google Scholar] - Schulman, J.; Levine, S.; Moritz, P.; Jordan, M.; Abbeel, P. Trust Region Policy Optimization. arXiv
**2015**, arXiv:1502.05477. [Google Scholar] - Lillicrap, T.P.; Hunt, J.J.; Pritzel, A.; Heess, N.; Erez, T.; Tassa, Y.; Silver, D.; Wierstra, D. Continuous Control with Deep Reinforcement Learning. arXiv
**2015**, arXiv:1509.02971v6. [Google Scholar] - Fujimoto, S.; Van Hoof, H.; Meger, D. Addressing Function Approximation Error in Actor-Critic Methods. arXiv
**2018**, arXiv:1802.09477. [Google Scholar] - Haarnoja, T.; Zhou, A.; Abbeel, P.; Levine, S. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. arXiv
**2018**, arXiv:1801.01290. [Google Scholar] - Mnih, V.; Kavukcuoglu, K.; Silver, D.; Graves, A.; Antonoglou, I.; Wierstra, D.; Riedmiller, M. Playing Atari with Deep Reinforcement Learning. arXiv
**2013**, preprint. arXiv:1312.5602. [Google Scholar] - Bellemare, M.G.; Dabney, W.; Munos, R. A Distributional Perspective on Reinforcement Learning. arXiv
**2017**, arXiv:1707.06887. [Google Scholar] - Dabney, W.; Rowland, M.; Bellemare, M.G.; Munos, R. Distributional Reinforcement Learning with Quantile Regression. arXiv
**2017**, arXiv:1710.10044. [Google Scholar] - Andrychowicz, M.; Wolski, F.; Ray, A.; Schneider, J.; Fong, R.; Welinder, P.; McGrew, B.; Tobin, J.; Abbeel, P.; Zaremba, W. Hindsight Experience Replay. arXiv
**2017**, arXiv:1707.01495. [Google Scholar] - Anshuka, A.; Chandra, R.; Buzacott, A.J.V.; Sanderson, D.; van Ogtrop, F.F. Spatio Temporal Hydrological Extreme Forecasting Framework Using LSTM Deep Learning Model. Stoch. Environ. Res. Risk Assess
**2022**, 36, 3467–3485. [Google Scholar] [CrossRef] - Chen, C.; Jiang, J.; Liao, Z.; Zhou, Y.; Wang, H.; Pei, Q. A Short-Term Flood Prediction Based on Spatial Deep Learning Network: A Case Study for Xi County, China. J. Hydrol.
**2022**, 607, 127535. [Google Scholar] [CrossRef] - Marco, Z.; Elena, A.; Anna, S.; Silvia, T.; Andrea, C. Spatio-Temporal Cross-Validation to Predict Pluvial Flood Events in the Metropolitan City of Venice. J. Hydrol.
**2022**, 612, 128150. [Google Scholar] [CrossRef] - Wahla, S.S.; Kazmi, J.H.; Sharifi, A.; Shirazi, S.A.; Tariq, A.; Smith, H.J. Assessing Spatio-Temporal Mapping and Monitoring of Climatic Variability Using SPEI and RF Machine Learning Models. Geocarto Int.
**2022**, 1–20. [Google Scholar] [CrossRef] - Roushangar, K.; Ghasempour, R.; Nourani, V. Spatiotemporal Analysis of Droughts Over Different Climate Regions Using Hybrid Clustering Method. Water Resour. Manag.
**2021**, 36, 473–488. [Google Scholar] [CrossRef] - Liu, J.; Xu, L.; Chen, N. A Spatiotemporal Deep Learning Model ST-LSTM-SA for Hourly Rainfall Forecasting Using Radar Echo Images. J. Hydrol.
**2022**, 609, 127748. [Google Scholar] [CrossRef] - Adnan, R.M.; Liang, Z.; Parmar, K.S.; Soni, K.; Kisi, O. Modeling Monthly Streamflow in Mountainous Basin by MARS, GMDH-NN and DENFIS Using Hydroclimatic Data. Neural Comput. Appl.
**2020**, 33, 2853–2871. [Google Scholar] [CrossRef] - Senatore, A.; Furnari, L.; Mendicino, G. Impact of High-Resolution Sea Surface Temperature Representation on the Forecast of Small Mediterranean Catchments’ Hydrological Responses to Heavy Precipitation. Hydrol. Earth Syst. Sci.
**2020**, 24, 269–291. [Google Scholar] [CrossRef] - Barzegar, R.; Aalami, M.T.; Adamowski, J. Short-Term Water Quality Variable Prediction Using a Hybrid CNN–LSTM Deep Learning Model. Stoch. Environ. Res. Risk Assess.
**2020**, 34, 415–433. [Google Scholar] [CrossRef] - Gu, K.; Vosoughi, S.; Prioleau, T. Feature Selection for Multivariate Time Series via Network Pruning. In 2021 International Conference on Data Mining Workshops (ICDMW); IEEE: Toulouse, France, 2021. [Google Scholar]

**Figure 1.**Reported keyword occurrences in the literature on the implementation of ML models within the research domain of WRM.

**Figure 6.**Reinforcement learning (RL) categorization (all the abbreviations are provided in the nomenclature).

**Figure 7.**Schematic representation of ML–driven for (

**a**) spatiotemporal model, and (

**b**) geo-spatiotemporal model.

Research Domain | Description | Ref. |
---|---|---|

Streamflow forecasting | A comprehensive review of artificial intelligence models used in the review domain is presented, with the goal of optimizing reservoir operations. | [4] |

Flood, precipitation estimation, water quality, and groundwater | An in-depth review of machine-learning applications in the review domain is presented. | [5] |

Groundwater level modeling | An in-depth review of the ability of ML models in monitoring and predicting different aspects of the review domain is presented. | [6] |

Hydropower operation | A systematic review of hydropower operation optimization using ML is presented. | [7] |

Groundwater level prediction | The state-of-the-art ML models implemented in the review domain and the milestones achieved in this domain are presented. | [8] |

Drought prediction | The most used architectures in the review domain during the last two decades are evaluated. | [9] |

Water quality modeling | The state-of-the-art applications of machine learning and deep learning in the review domain are presented. | [10] |

Water quality evaluation | The applications of ML in various water environments such as surface and ground water, drinking water, seawater, and sewage are described. | [11] |

Groundwater level prediction | Various ML and AI techniques and their corresponding methodologies in the review domain are discussed. | [12] |

High-flow extremal hydrology | A comprehensive review is presented including an overview of the state-of-the-art AI techniques and examples of their applications, followed by a SWOT analysis to benchmark their predictive capabilities. | [13] |

Hydro-climatic processes | An in-depth review of the different techniques of prediction interval development in the review domain is presented. | [14] |

Streamflow forecasting | An in-depth review of decadal progress in the regionalization of hydrological modeling for predictions in ungauged basins from 2000 to 2019 is presented. | [15] |

Suspended sediment load prediction | Three popular artificial intelligence-based models are described, mainly focusing on the research between January 2015 and November 2020 in the review domain. | [16] |

Water resources in arboriculture | An overview of the application of new technologies in the analysis of crop water status to improve irrigation management, with a focus on arboriculture is presented. | [17] |

Drought prediction | Various artificial intelligence techniques used in the review domain are presented. | [18] |

Learning Types | Type of Data | Training | Used for | Algorithms | |
---|---|---|---|---|---|

Supervised Learning | Labeled data | Trained using labeled data (extra supervision) | Regression for nowcasting and forecasting | Classification in binary and multiple classes | Linear regression, logistic regression, RF, SVM, KNN, RNN, DNN, etc. |

Unsupervised Learning | Unlabeled data | Trained using unlabeled data without any guidance (no supervision) | Clustering | K—Means, C—Means, Agglomerative Hierarchical Clustering, DBSCAN, Gaussian Mixture Models, OPTICS, etc. | |

Reinforcement Learning | Without predefined data | Works based on the interaction between agent and environment (no supervision) | Decision making | Q—Learning, SARSA, DQN, double DQN, dueling DQN, etc. |

Metric | Equation | Description |
---|---|---|

Mean normalized bias error | $MBE=\frac{1}{n}{\displaystyle {\displaystyle \sum}_{i=1}^{n}}\frac{{Y}_{i}-{\widehat{Y}}_{i}}{{Y}_{i}}$ | Estimation of the average bias of the prediction approach used to decide on measures for correcting the approach bias |

Mean percentage error | $MPE=\frac{1}{n}{\displaystyle {\displaystyle \sum}_{i=1}^{n}}\frac{{Y}_{i}-{\widehat{Y}}_{i}}{{Y}_{i}}\times 100$ | The computed average of the percentage errors between a model’s forecasts and the actual values of the quantity being forecast |

Mean absolute error | $MAE=\frac{1}{n}{\displaystyle {\displaystyle \sum}_{i=1}^{n}}\left|{\widehat{Y}}_{i}-{Y}_{i}\right|$ | A statistic to assess the average magnitude of errors in a set of forecasts without considering the direction of the errors |

Mean absolute percentage error | $MAPE=\frac{1}{n}{\displaystyle {\displaystyle \sum}_{i=1}^{n}}\left|\frac{{Y}_{i}-{\widehat{Y}}_{i}}{{Y}_{i}}\right|\times 100$ | An accuracy rating metric that measures accuracy as a percentage of the average absolute percentage error minus the real amounts divided by the real amounts |

Relative absolute error | $RAE=\frac{{{\displaystyle \sum}}_{i=1}^{n}\left|{Y}_{i}-{\widehat{Y}}_{i}\right|}{{{\displaystyle \sum}}_{i=1}^{n}\left|{Y}_{i}-\overline{Y}\right|}$ | A relative metric for evaluating a prediction model’s performance |

Weighted mean absolute percentage error | $WMAPE=\frac{{{\displaystyle \sum}}_{i=1}^{n}\left|{Y}_{i}-{\widehat{Y}}_{i}\right|}{{{\displaystyle \sum}}_{i=1}^{n}\left|{Y}_{i}\right|}$ | A measure of a forecasting method’s prediction accuracy that is a weighted version of the MAPE |

Normalized mean absolute error | $NMAE=\frac{MAE}{\frac{1}{n}{{\displaystyle \sum}}_{i=1}^{n}\left|{Y}_{i}\right|}$ | Intended to make it easier to compare datasets with different scales in terms of MAE |

Mean squared error | $MSE=\frac{1}{n}{\displaystyle \sum}_{i=1}^{n}{\left({Y}_{i}-{\widehat{Y}}_{i}\right)}^{2}$ | Measures the variation between the mean squares of the real amount and forecast values |

Root mean square error | $RMSE=\sqrt{\frac{1}{n}{\displaystyle \sum}_{i=1}^{n}{\left({Y}_{i}-{\widehat{Y}}_{i}\right)}^{2}}$ | An estimation of the mean amount of error |

Coefficient of variation | $CV=\frac{\sqrt{\frac{{{\displaystyle \sum}}_{i=1}^{n}{\left({Y}_{i}-{\widehat{Y}}_{i}\right)}^{2}}{n}}}{\overline{Y}}$ | Known as the relative standard deviation, it is a standardized measurement of the dispersion of a probability distribution |

Normalized root mean square error | $NRMSE=\frac{RMSE}{{Y}_{i}{}_{max}-{Y}_{i}{}_{min}}$ | A normalized RMSE to facilitate comparisons between datasets and models with different scales |

Coefficient of determination | ${R}^{2}=1-\frac{{{\displaystyle \sum}}_{i=1}^{n}{\left(\widehat{Y}-\overline{Y}\right)}^{2}}{{{\displaystyle \sum}}_{i=1}^{n}{\left({Y}_{i}-\overline{Y}\right)}^{2}}$ | Measurement of the variance ratio of a dependent variable using an independent variable |

Willmott’s index agreement | $WI=1-\left[\frac{{{\displaystyle \sum}}_{i=1}^{n}{({Y}_{i}-{\overline{Y}}_{i})}^{2}}{{{\displaystyle \sum}}_{i=1}^{n}{(\left|{Y}_{i}-{\overline{Y}}_{i}\right|+\left|{\widehat{Y}}_{i}-{\overline{\widehat{Y}}}_{i}\right|)}^{2}}\right]$ | Measurement of the ratio of the mean square error and the potential error |

Legates–McCabe’s | $LM=1-\left[\frac{{{\displaystyle \sum}}_{i=1}^{n}\left|{Y}_{i}-{\widehat{Y}}_{i}\right|}{{{\displaystyle \sum}}_{i=1}^{n}\left|{Y}_{i}-{\overline{Y}}_{i}\right|}\right]$ | A useful alternative goodness-of-fit or relative error that overcomes many of the limitations of correlation-based metrics |

Kling–Gupta efficiency | $KGE=\sqrt{\left[r-1{]}^{2}+\right[\alpha -1{]}^{2}+{[\beta -1]}^{2}}$ $r=\frac{cov(\widehat{Y},Y)}{\sigma \left(\widehat{Y}\right).\sigma \left(Y\right)},\alpha =\frac{\sigma \left(\widehat{Y}\right)}{\sigma \left(Y\right)},\beta =\frac{\overline{\widehat{Y}}}{\overline{Y}}$ | Measures the model efficiency with accuracy, precision, and consistency components. Here, r, α, and β represent the correlation coefficient, the bias (ratio between the standard deviations of the predicted and observed values), and the ratio of variances, respectively. σ denotes the standard deviation. |

Akaike information criterion | $AIC=-2log(L\left(\widehat{{\theta}^{ML}}|Y\right))+2\mathrm{i}$ | A measure of a model performance that accounts for model complexity. $\widehat{{\theta}^{ML}}$ represents the vector of maximum likelihood estimates of the model parameters, and i denotes the number of the observed values. |

Probabilistic Metric | ||

Continuous ranked probability score | $\mathrm{CRPS}={{\displaystyle \int}}_{-\infty}^{+\infty}{\left[P\left({\widehat{Y}}_{i}\right)-H\left({\widehat{Y}}_{i}-{Y}_{i}\right)\right]}^{2}d{\widehat{Y}}_{i}$ | A quadratic measure of the difference between forecast and empirical cumulative distribution functions (CDF), $P\left({\widehat{Y}}_{i}\right)$ is the prediction CDF, and $H$ is the Heaviside step function, which is equal to 0 if ${\widehat{Y}}_{i}<{Y}_{i}$ and 1 otherwise. |

Average width of the prediction intervals | $\mathrm{AWPI}=\frac{1}{n}{\displaystyle {\displaystyle \sum}_{i=1}^{n}}\left({\widehat{Y}}_{i}{}^{u}-{\widehat{Y}}_{i}{}^{l}\right)$ | An estimate of an interval in which a future observation will fall, with a certain confidence level, based on previous observations. $u$ and $l$ denote the upper and lower bounds of the 95 % prediction interval, respectively. |

Prediction interval coverage | $\mathrm{PICP}=\frac{1}{n}{\displaystyle {\displaystyle \sum}_{i=1}^{n}}{c}_{i},{c}_{i}=\{\begin{array}{c}1,if{Y}_{i}\in \left[{\widehat{Y}}_{i}{}^{l},{\widehat{Y}}_{i}{}^{u}\right]\\ 0,if{Y}_{i}\notin \left[{\widehat{Y}}_{i}{}^{l},{\widehat{Y}}_{i}{}^{u}\right]\end{array}$ | The percentage of the time the prediction interval covers the actual value in a holdout set |

Prediction interval normalized average width | $\mathrm{PINAW}=\frac{1}{n}{\displaystyle {\displaystyle \sum}_{i=1}^{n}}\frac{{\widehat{Y}}_{i}{}^{u}-{\widehat{Y}}_{i}{}^{l}}{R}$ | Measures the wide degree of the prediction interval. R denotes the range of variation of the observed value. |

Research Field | Algorithm | Goal | Ref. |
---|---|---|---|

Streamflow | LSTM, BNN, LSTM-MC, BLSTM | Developing a probabilistic forecasting model that addresses the relevant subproblem of univariate time series models for multistep ahead daily streamflow forecasting in order to quantify both epistemic and aleatory uncertainty. | [19] |

Streamflow | SVR, TSVR, ELM, Huber loss function-based ELM | Adopting new data preprocessing techniques to capture the data noise and enhance prediction accuracy. | [21] |

Streamflow | Bi-directional LSTM, Stacking of RF and MLP | Introducing a novel streamflow forecasting model. | [22] |

Soil Moisture | MLP, RF, SVR | Applying machine learning with novel structures for the estimation of daily volumetric soil water content. | [23] |

Water quality in an urban drainage network | EMD-LSTM | Combining a data preprocessing mechanism based on empirical mode decomposition (EMD) with an LSTM neural network prediction module. | [24] |

Contamination in water distribution systems | ANN, SVM with linear kernels, SVM with radial basis function kernels (RBF), linear regressor, decision tree, extra trees, gradient boosting regressor, RF, KNN, and uniform weighted KNN | Introducing a new stacking ensemble model for contamination detection based on several water quality metrics. | [25] |

Urban water quality | RF, SVM | Creating an integrated decomposition-reclassification-prediction technique for water quality by combining the CEEMDN and RF methods with the genetic algorithm-support vector machine model (GA-SVM). | [26] |

Leakage detection | LSTM | Constructing a model using multi-layer perceptron (MLP) and LSTM to predict MNF (minimal night flow). | [27] |

Failure rate | ANN | Identifying the most effective serial triple diagram model (STDM) methodologies for predicting the daily failure rates of a water distribution system. | [28] |

Water demand | Graph convolutional recurrent neural network (GCRNN) | Building a graph-based model capable of capturing the dependence among the different water demand time series in both spatial and temporal terms. | [29] |

Water consumption | ARIMA, LSTM | Developing a water consumption prediction model for individual customers using a deep learning-based LSTM approach. | [30] |

Hydropower | SVR, LSTM | Developing a theory-guided ML framework and validating the model’s performance for a reservoir located in Southwest China. | [31] |

Hydropower | DNN | Exploring if it is desirable to use the known energy production of previous days as a predictor or to predict the day ahead inflows and then simulate the consequent energy production. | [32] |

Hydropower | ANN, ARIMA, SVM | Investigating the capabilities of different ML algorithms in predicting the power production of a reservoir in China using data from 1979 to 2016. | [33] |

Water levels | LSTM, GRU | Developing an efficient and precise data model for predicting water levels with extreme temporal variations. | [34] |

Streamflow | CNN-BAT | Demonstrating the prediction accuracy of a convolutional neural network (CNN) using BAT metaheuristic algorithm. | [35] |

Groundwater | ARIMA, a back-propagation artificial neural network (BP-ANN), LSTM | Investigating the accuracy of the model in forecasting the GWL at the monthly and daily scales by using three widely used data-driven models: an autoregressive integrated moving average (ARIMA), a back-propagation artificial neural network (BP-ANN), and an LSTM network. | [36] |

Groundwater | Ensemble learning (FFNN, ANFIS, GMDH), LASTM | Forecasting multi-step ahead GWL of each cluster’s piezometer centroid. | [37] |

Streamflow | LSTM | Integrating meteorological forecasts, land surface hydrological model simulations, and ML to forecast hourly streamflow. | [38] |

Precipitation | LSTM | Proposing a combination of the weather research and forecasting hydrological modeling system (WRF-Hydro) and LSTM network to improve streamflow prediction. | [39] |

Streamflow | ANN, CNN, LSTM | Evaluating the possibilities of singular spectral analysis (SSA), seasonal-trend decomposition utilizing loess (STL), and attribute selection preprocessing approaches with neural network techniques for predicting monthly streamflow. | [40] |

Streamflow | ConvLSTM, LSTNet, 3D-CNN, TD-CNN, transformer | Enhancing the multi-step ahead prediction capability by using mesoscale hydroclimate data as booster predictors and employing attention-based DNNs. | [20] |

Water quality | Deep transfer learning based on transformer (TLT) | Introducing a transfer learning approach to water quality prediction in order to improve prediction performance in data constrained environments. | [41] |

Streamflow | ANN, ELM, SVM, EMD, EEMD | Developing a hydrological forecasting model based on parallel cooperation search algorithm (PCSA) and extreme learning machine (ELM). | [42] |

Streamflow | BART | Developing a novel hybrid model, GA-BART, that combines a genetic algorithm (GA) and the Bayesian additive regression tree (BART) model for hourly streamflow forecasting. | [43] |

Streamflow | DGDNN | Introducing a DL model and directed graph DNN for multi-step streamflow prediction. | [44] |

Streamflow | ANFIS-GBO | Improving the accuracy of prediction in a mountainous river basin. | [45] |

Streamflow | ELM-PSOGWO | Introducing a hybrid model based on heuristic optimization and extreme learning machine algorithms for monthly runoff prediction. | [46] |

Streamflow | ANN-EMPA | Introducing a hybrid model based on extended marine predators algorithm (EMPA)-based ANN | [47] |

Metric | Equation | Description | Ref. |
---|---|---|---|

Calinski-Harabasz | $CH=\frac{n-k}{k-1}\frac{{{\displaystyle \sum}}_{i=1}^{k}\left|{C}_{i}\right|d\left({v}_{i},\overline{x}\right)}{{{\displaystyle \sum}}_{i=1}^{k}{{\displaystyle \sum}}_{{x}_{j\in {C}_{i}}}d\left({v}_{i},\overline{x}\right)}$ | The numerator measures the distance between the cluster centroids and the global centroid, whereas the denominator measures the distances between centroids within each cluster. Clearly, a valid optimal partition is indicated by the largest CH. | [50] |

Chou-Su-Lai | $CS=\frac{{{\displaystyle \sum}}_{i=1}^{k}\left\{\frac{1}{\left|{C}_{i}\right|}{{\displaystyle \sum}}_{{x}_{j\in {C}_{i}}}ma{x}_{{x}_{l\in {C}_{i}}}d\left({v}_{i},\overline{x}\right)\right\}}{{{\displaystyle \sum}}_{i=1}^{k}\left\{mi{n}_{j:j\ne i}d\left({v}_{i},\overline{x}\right)\right\}}$ | The numerator is the sum of the average maximum distances between points within each cluster, and the denominator is the sum of the minimum distances between clusters. The partition with the smallest CS is valid and optimal. | [51] |

Dunn’s index | $DI=\underset{i\ne \mathrm{j}\in \left[k\right]}{min}\left\{\frac{min\left\{\left(d\left({x}_{u},{x}_{v}\right)|{x}_{u}\in {C}_{i},{x}_{v}\in {C}_{i}\right)\right\}}{ma{x}_{l\in \left[k\right]}max\left\{\left(d\left({x}_{u},{x}_{v}\right)|{x}_{u},{x}_{v}\in {C}_{l}\right)\right\}}\right\}$ | The numerator represents the minimum between-cluster distance, whereas the denominator represents the maximum within-cluster distance. The largest DI indicates an optimally valid partitioning. | [52] |

Davies-Bouldin’s index | $DB=\frac{1}{k}{\displaystyle {\displaystyle \sum}_{i=1}^{k}}\underset{j\in \left[k\right]\backslash \left\{i\right\}}{max}\left\{\frac{{S}_{i,q}+{S}_{j,q}}{{({{\displaystyle \sum}}_{s=1}^{p}{\left|{v}_{i,s}-{v}_{j,s}\right|}^{t})}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$t$}\right.}}\right\}$ | The partition with the smallest DB is the optimal partition. | [53] |

Davies-Bouldin’s Index* | $D{B}^{*}=\frac{1}{k}{\displaystyle {\displaystyle \sum}_{i=1}^{k}}\frac{\underset{j\in \left[k\right]\backslash \left\{i\right\}}{max}\left\{{S}_{i,q}+{S}_{j,q}\right\}}{\underset{j\in \left[k\right]\backslash \left\{i\right\}}{min}\left\{{({{\displaystyle \sum}}_{s=1}^{p}{\left|{v}_{i,s}-{v}_{j,s}\right|}^{t})}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$t$}\right.}\right\}}$ | The smallest DB* denotes a valid optimal partition, similar to the original DB. | [54] |

Silhouette Coefficient | $SC=\frac{1}{N}{{\displaystyle \sum}}_{i=1}^{n}\begin{array}{c}{s}_{i},{s}_{i}=\{\begin{array}{c}\frac{{b}_{i}-{a}_{i}}{max\left\{{b}_{i},{a}_{i}\right\}}\\ 0if\left|{C}_{l}\right|=1\end{array}if\left|{C}_{l}\right|1\\ \end{array},$ $\{\begin{array}{c}{a}_{i}=\frac{1}{\left|{C}_{l}\right|-1}{\displaystyle {\displaystyle \sum}_{j:{x}_{j}\in {C}_{l}}}d\left({x}_{i},{x}_{j}\right)\\ {b}_{i}=\underset{s\ne l}{min}\frac{1}{\left|{C}_{s}\right|}{\displaystyle {\displaystyle \sum}_{j:{x}_{j}\in {C}_{s}}}d\left({x}_{i},{x}_{j}\right)\end{array}$ | The largest SC denotes an optimal partition which is valid. | [55] |

Hybrid validity index | $SCD=\frac{SC}{\left(0.5CS+0.5DB\right)}$ | SCD is a collection of three robust measures of cluster validity (SC, CS, and DB). | [37] |

Research Field | Cluster No. | Algorithm | Goal | Ref. |
---|---|---|---|---|

Water distribution system | 3–5 | OPTICS/K-means | Determine different customer patterns based on the geographic locations of households | [71] |

Water monitoring system | 3 | k-means | Monitor water consumption in a household to improve WRM | [72] |

Sediment | 5 | Fuzzy C-means | Classify the Rhône River hydrology according to the main hydrological events | [73] |

Hydrological regionalization | 6 | Hierarchical and K-means | Delineate the homogeneous clusters of watersheds | [74] |

Water consumption patterns | - | k-means | Observe the consumption patterns with regard to their variability. | [75] |

Hydrological time series clustering | 4 | Hierarchical/DBSCAN | Develop a clustering framework | [76] |

Flood risk | 1–3, 5, 7, 10, 20, 50, 100, 200, 500 | k-means | Choosing the optimal number of clusters and associated parameter sets for a hydrologic model. model | [62] |

Groundwater | 2–10 | K-Means/hierarchical (WARD)/self-organizing neural network (GNG) | Promote remediation measures for groundwater depletion and contamination | [77] |

Groundwater | 4, 5, 6 | K-means/FCM/GNG/Cluster ensemble | Identify the patterns of groundwater level (GWL) over the Ghorveh-Dehgolan plain (GDP) located in western Iran | [37] |

Watershed zonation | 1–7 | K-means/hierarchical/Gaussian mixture | Characterize the organization and functions of the watershed in a more tractable manner by integrating multiple spatial data layers | [78] |

Groundwater | 5 | agglomerative hierarchical | Classify wells in the San Joaquin River Basin, California | [79] |

Hydrological regionalization | 3 | K-value | Regionalize lowland rivers using long data series and selected hydrological characteristics based on the example of Lithuanian rivers. | [80] |

Karstic aquifers | 1–5 | K-means, fuzzy logic, fuzzy C-mean genetic K-means | Improve protection in a vulnerable karstic region | [81] |

Rainfall | 2, 3, 4, 7, 11 | K-Means/Fuzzy C-Means | Determining the spatiotemporal patterns of torrential rainfall along the East Coast of Peninsular Malaysia | [82] |

Flood | 8 | K-Means | Design and implementation of real-time monitoring | [83] |

Streamflow | 3 | Hierarchical | Study the collective similarity in periodic phenomena | [84] |

River | 5 | Hierarchical | Identify the primary causes of flow and sediment load variations | [85] |

River | 4 | k-means | Analysis of climatic and physiographic catchment properties | [86] |

Catchment hydrologic control | 3 | Hierarchical | Investigate landscape controls on hydrologic response through catchment classification | [87] |

Water scarcity | 3 | k-means | Investigate how human adaptation affects water scarcity uncertainty | [88] |

Flow regime | 2–10 | k-means | Stream analysis for bias estimation and reduction | [89] |

Precipitation | 3 | k-means | Improve both medium- and long-term precipitation forecasting accuracy | [90] |

Aquifer system | 11 | Hierarchical | Classify non-linear hydrochemical data | [91] |

Research Field | Data/Period | Location | Algorithm | Ref. |
---|---|---|---|---|

Hydrological extreme | Satellite rainfall estimates and sea surface temperature (SST) anomalies/1980–2020 | Fiji’s islands | LSTM | [130] |

Flood | Six-hour precipitation based on Himawari-8 and ground station data | Xi County, China | ConvLSTM | [131] |

Flood | 60 historical events occurred in the area/1995–2020 | Venice, Italy | LR, NN, RF | [132] |

Droughts | Meteorological data from the openly accessible climate dataset PMD, which contains land-based observations/1980–2020 | Cholistan, Punjab, Pakistan | RF | [133] |

Droughts | Standardized precipitation index series with timescales of 3, 6, and 12 months during the 1951–2016 period | 31 stations in Iran | Maximal overlap discrete wavelet transform (MODWT) and K-means | [134] |

Precipitation | Radar station Z9270/2016–2020 | Wuhan, China | ST-LSTM-SA | [135] |

Groundwater | GWL, rainfall, runoff/1989–2018 | Iran | Ensemble learning (FFNN, ANFIS, GMDH), LASTM | [37] |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Ghobadi, F.; Kang, D.
Application of Machine Learning in Water Resources Management: A Systematic Literature Review. *Water* **2023**, *15*, 620.
https://doi.org/10.3390/w15040620

**AMA Style**

Ghobadi F, Kang D.
Application of Machine Learning in Water Resources Management: A Systematic Literature Review. *Water*. 2023; 15(4):620.
https://doi.org/10.3390/w15040620

**Chicago/Turabian Style**

Ghobadi, Fatemeh, and Doosun Kang.
2023. "Application of Machine Learning in Water Resources Management: A Systematic Literature Review" *Water* 15, no. 4: 620.
https://doi.org/10.3390/w15040620