Leak Management in Water Distribution Networks Through Deep Reinforcement Learning: A Review

Abstract

Leak management in water distribution networks (WDNs) is essential for minimising water loss, improving operational efficiency, and supporting sustainable water management. However, effectively identifying, preventing, and locating leaks remains a major challenge owing to the ageing infrastructure, pressure variations, and limited monitoring capabilities. Leakage management generally involves three approaches: leakage assessment, detection, and prevention. Traditional methods offer useful tools but often face limitations in scalability, cost, false alarm rates, and real-time application. Recently, artificial intelligence (AI) and machine learning (ML) have shown growing potential to address these challenges. Deep Reinforcement Learning (DRL) has emerged as a promising technique that combines the ability of Deep Learning (DL) to process complex data with reinforcement learning (RL) decision-making capabilities. DRL has been applied in WDNs for tasks such as pump scheduling, pressure control, and valve optimisation. However, their roles in leakage management are still evolving. To the best of our knowledge, no review to date has specifically focused on DRL for leakage management in WDNs. Therefore, this review aims to fill this gap and examines current leakage management methods, highlights the current role of DRL and potential contributions in the water sector, specifically water distribution networks, identifies existing research gaps, and outlines future directions for developing DRL-based models that specifically target leak detection and prevention.

1. Introduction

On Earth, life does not exist without water, a basic need for the survival of all life forms on the planet. This is undoubtedly the key reason why Earth is the only planet known to support life. Worldwide, water scarcity occurs because of excessive extraction, growing populations, and climate change. This shortage makes it challenging to satisfy the water requirements of industrial sectors, communities, and agricultural activities [1]. High-quality water is vital for human health and economic development. Worldwide, two billion people lack access to potable water, whilst 3.6 billion people lack effectively managed sanitation facilities [2]. Projections indicate that urban areas will house 68% of the global population by 2050 [3]. A major issue associated with water scarcity is the occurrence of leaks in WDNs, which can lead to substantial water wastage, particularly in urban areas, where approximately 30% of the total water consumption is attributed to losses from leakage [4]. Across England and Wales, approximately 2.5 billion litres of water are lost each day due to leakage [5].

Non-revenue water (NRW) refers to water that is produced but not billed, including both real losses (e.g., leaks) and apparent losses (e.g., theft or meter errors) [6,7,8]. Leakage is the major contributor to NRW and often occurs at joints, valves, and service lines owing to ageing pipes, poor installation, soil conditions, pressure fluctuations, and external stresses [9,10]. These losses place additional strain on freshwater sources that are already affected by overuse and climate change.

Leakage management is a comprehensive process involving three core components: assessment, detection, and prevention [11]. Assessment quantifies losses using methods such as Minimum Night Flow (MNF) and the Burst and Background Estimation (BABE) model. Detection strategies range from manual inspection to sensor-based systems and data-driven models. Prevention focuses on proactive pressure control and system optimisation [6,10,11,12,13]. Conventional methods may detect leaks after significant water loss, lack the ability to handle complex and uncertain network environments in real time, or struggle with scalability, cost, and adaptability to changing conditions [6,9,10,14,15]. This requires dynamic and intelligent approaches capable of operating under uncertainty [16].

In recent years, the integration of AI and ML has shown promise in addressing the challenges faced by WDNs [17]. Among these, DRL has emerged as a particularly promising approach that combines the perception capabilities of deep learning with the decision-making framework of reinforcement learning to address complex dynamic control problems. DRL has shown promise for managing complex nonlinear control problems and has yielded promising results in fields such as robotics, control systems, and smart infrastructure [16,18]. It has also been utilised to optimise WDNs and pre-vent leakage [16,19,20,21]. DRL can optimise operations such as valve control, pump scheduling, and pipe maintenance with the aim of preventing leaks while enhancing the overall efficiency [18,21,22,23,24]. Although DRL has proven highly effective in optimising WDN operations, its specific application in leak detection and prevention remains underexplored which presents a valuable opportunity for further research.

Although AI and ML are increasingly used in water systems, there is still no focused review that examines how DRL can support leak management in water distribution networks. Most existing reviews by [16] discussed broader challenges and opportunities for DRL in urban water systems. However, these studies do not provide a structured review focused solely on leakage management. This paper fills that gap by bringing together recent DRL research and highlighting how it can be adapted and improved for effective leak management in water networks. This review aims to explore the current landscape of water leakage management, with a particular focus on the transformative role of DRL in optimising WDN operations. This paper also examines the strengths and limitations of existing leakage detection techniques, highlights recent advancements in DRL applications, and discusses the challenges and future directions for leveraging this technology in water leakage management. By providing a comprehensive analysis, this review seeks to contribute to ongoing efforts to achieve sustainable and efficient water distribution systems, addressing one of the most pressing global challenges of the 21st century. The remainder of this paper is structured as follows. Section 2 details the method used to identify the relevant literature. Section 3 presents a comprehensive review of leakage management approaches in water distribution networks, including assessment, detection, and prevention methods, as well as their respective advantages and limitations. Section 4 introduces the core principles of deep reinforcement learning (DRL), explores its current applications within the water sector, particularly in WDNs, and discusses the associated challenges. Section 5 provides potential future directions, and lastly, Section 6 concludes the review paper.

2. Methodology

This review comprehensively examines the current state of applications of leak management and DRL for (WDNs). The methodology followed established guidelines for literature review, and the review protocol was structured into two main phases: paper identification and screening. Figure 1 illustrates this process, which is further detailed in the subsequent section.

Paper Identification and Screening

The Web of Science core collection database was selected for literature retrieval owing to its coverage of peer-reviewed publications in the engineering, computer science, and water domains. A topic-based search was conducted on all documents for 2020–2024. A four-phase keyword approach progressively narrowed down water leakage topics to identify research gaps. The search strategy yielded results across phases: phase 1 with 2369, phase 2 with 341, phase 3 with 976, and phase 4 with 643 publications.

Inclusion criteria were peer-reviewed journal articles, review papers, and conference proceedings from 2020 to 2024. The search included leakage detection, prevention and assessment techniques, leakage management techniques, AI and ML applications, foundational papers on water distribution management, and studies on RL and DRL applications in water systems. Exclusion criteria were non-peer-reviewed studies, papers unrelated to water leakage management, non-English publications, duplicates, and studies not addressing the review’s key theme.

The screening process consisted of four stages for identifying the most relevant papers. First, the titles and abstracts of all papers from the four-phase keyword search were reviewed. Only those related to leak detection, prevention, AI/ML in water networks, and RL and DRL applications were retained. Full-text screening removed papers with vague methods, unclear objectives, or low relevance to DRL or leak management. To ensure that important studies were not missed, a snowballing strategy was used. This included checking references (backward snowballing) and newer papers that cited key studies (forward snowballing). Some older foundational papers were also included based on expert knowledge. Finally, the quality, relevance, and contribution of each remaining study were assessed. This process led to the selection of high-quality papers and some relevant articles which addressed the key theme of the paper. This approach helped shape a focused body of literature that reflected the current landscape of DRL and leakage management in water systems.

Four keyword phases were designed to map the research landscape from foundational topics to specific intersections. Phase 1 established the baseline of water leakage research across detection, prevention, and assessment. The keyword “water leakage detection” gave a total publication of 4085 which further reduced to 1657 after selecting the range between 2020 and 2024. As shown in Figure 2, the annual breakdown is 268 in 2020, 303 in 2021, 334 in 2022, 351 in 2023, and 401 in 2024, which shows a growing interest in water leakage detection. Upon refining the search to “water leakage detection AND water distribution networks”, 297 results were identified, with 171 publications appearing between 2020 and 2024:14 in 2020, 26 in 2021, 45 in 2022, 46 in 2023, and 40 in 2024. In contrast, the search for “water leakage prevention” yielded 1470 publications in total, with 429 of these published within the 2020–2024 timeframe: 72 in 2020, 68 in 2021, 103 in 2022, 82 in 2023, and 104 in 2024. Within the context of water distribution networks “water leakage prevention AND water distribution networks” only yielded 28 publications overall, with 15 from the period 2020–2024: four in 2020, three in 2021, two in 2022, two in 2023, and four in 2024. Finally, the query “water leakage assessment AND water distribution networks” resulted in 234 total publications, with 97 recent ones: 20 in 2020, 15 in 2021, 20 in 2022, 23 in 2023, and 19 in 2024.

Phase 2 explored how artificial intelligence and machine learning have been integrated into leakage-related studies. As shown in Figure 3, the query “water leakage detection AND (Artificial Intelligence OR Machine Learning)” resulted in 386 publications, with 295 between 2020 and 2024: 28 (2020), 30 (2021), 62 (2022), 69 (2023), and 106 (2024). For “water leakage prevention AND (Artificial Intelligence OR Machine Learning)”, there were 23 total, 19 of which were recent: 1 (2020), 2 (2021), 3 (2022), 3 (2023), and 10 (2024). Data on “water leakage assessment AND Artificial Intelligence” were not available.

Phase 3 examined the broader application of reinforcement learning in water systems. As shown in Figure 4, the search for “Reinforcement Learning AND water” returned 11,032 total publications, with 8198 from 2020 to 2024: 777 (2020), 1011 (2021), 1697 (2022), 1888 (2023), and 2823 (2024). To narrow down, we selected research areas related to water resources, which left only 139 publications: 16 papers for 2020, 15 for 2021, 35 for 2022, 34 for 2023 and 39 for 2024. The query “Deep Reinforcement Learning AND water” yielded a total of 4210, including 3601 recent publications—323 (2020), 445 (2021), 777 (2022), 846 (2023), and 1210 (2024) which, when introducing the research area of water resources, further reduced to 39 publications. The next keyword combination, Reinforcement Learning AND urban water,” had 624 results, with 482 in the recent period: 37 (2020), 60 (2021), 96 (2022), 110 (2023), and 179 (2024). Similarly, “Deep Reinforcement Learning AND urban water” returned a total of 374, 307 of which were published from 2020 to 2024: 22 (2020), 34 (2021), 62 (2022), 72 (2023), and 117 (2024).

Finally, phase 4 directly targeted the review’s central research gap, as shown in Figure 5. The phrase “Reinforcement Learning AND water distribution networks” produced 439 total publications, with 374 in the 2020–2024 range: 30 (2020), 34 (2021), 63 (2022), 96 (2023), and 151 (2024). For “Deep Reinforcement Learning AND water distribution networks”, there were 299 total results, 267 of which were recent: 23 (2020), 20 (2021), 46 (2022), 72 (2023), and 106 (2024). In contrast, “Deep Reinforcement Learning AND leakage detection” appeared only once in 2022, and “Deep Reinforcement Learning AND water leakage detection” returned zero results across all years. A similar lack of activity was found for “Deep Reinforcement Learning” AND “pressure optimisation Or Leakage prevention” with only one relevant publication in 2024.

A systematic four-phase search revealed a significant research gap. Although substantial research exists on water leakage management (2369 papers in Phase 1), AI/ML adoption in leak management (341 papers in Phase 2), and DRL applications across water systems (976 papers in Phase 3), there is virtually no research on the intersection of DRL and leak management in water distribution networks. Phase 4 showed that despite 641 papers on RL and DRL applications in water distribution networks, only two papers address DRL for leakage-related applications. This represents a missed opportunity, given that leakage management is a pressing challenge in water distribution systems, and DRL has shown success in related infrastructure optimisation. This gap indicates minimal effort to bridge traditional leak management and DRL applications in water systems. Therefore, this review examines how DRL methodologies can enhance leak detection and prevention in WDNs, potentially revolutionising current approaches to this persistent water industry challenge.

3. Leakage Management in Water Distribution Networks

Managing and mitigating leaks are crucial for sustainable water use and economic efficiency. Leakage in WDNs occurs owing to a combination of intrinsic and extrinsic factors [25]. Intrinsic causes stem from the aging and deterioration of infrastructure components, where material fatigue, corrosion, and poor maintenance worsen the issue. Extrinsic causes include environmental conditions such as soil movement and vibrations that stress the network. Hydraulic conditions, such as pressure surges, can widen cracks, while construction activities may damage pipelines [6,10,26,27]. The chartered institution of water and environmental management (CIWEM) policy explains that inefficient pressure management and a lack of active control of leakage can further increase leakage in WDNs. High pressures can cause existing cracks to widen, and temperature variations can cause pipes to expand or shrink, which may lead to more frequent breakage [6,13].

Refs. [10,11,28] classified leakages as reported leakages that are easy to find, and unreported leakages that are difficult to find. Unreported leakages occur mainly underground, with moderate flow, and are a major source of water loss [28,29]. Unreported leakages include Burst and Background types. Background leakage consists of small, continuous leaks from aging pipes that are hard to detect, while burst leakage involves sudden, high-volume releases requiring urgent repair [10,28,30,31]. Ref. [32] further added that background leakages are small leaks (<0.5 m³/h) that run continuously and are uneconomical to fix, while bursts are larger leaks (>0.5 m³/h) with varying durations.

To manage leaks, water utilities follow a structured approach that involves assessment, detection, and prevention. Leakage assessment quantifies water losses and differentiates between real and apparent losses [31,33,34,35]. Detection involves three phases: identification, localisation, and pinpoint (ILP) [8,10,14,36]. Ref. [36] The identification phase differentiates leak signals from other network signals, while localisation narrows the leak to a general area such as a District Metered Area (DMA). Pinpointing determines the exact location within 20 cm, replacing the previous 30 cm locating phase [10,37]. The final phase of leakage prevention focuses on minimising future leaks through interventions such as pressure management, pump scheduling, pipe upgrades, and maintenance strategies to improve network resilience and enable sustainable leak control [10,11,13,14,38]. The following sections discuss these approaches in detail.

3.1. Leakage Assessment Methods

Leakage assessment is generally performed in two steps: a top-down approach and a bottom–down approach [8,11,39]. The top–down approach is the most straightforward and begins by calculating the NRW using the IWA water balance sheet [8,11,39]. NRW is determined by subtracting Billed Authorised Consumption (BAC) from the System Input Volume (SIV) [33,40]. It is divided into Apparent Losses (AL) and Real Losses (RL). Apparent losses typically arise from meter inaccuracies, billing errors, or unauthorised consumption, whereas real losses refer to actual physical leakages in the network. Using informed assumptions or field data, the volume of AL is first estimated, and then RL is calculated as the remaining portion of NRW. This approach offers a high-level overview with minimal field investigation but depends heavily on input data quality and assumptions.

The bottom-up approach is succeeded by a top-down approach for further leakage assessment of the selected District Metered Area (DMA). One widely adopted method is Minimum Night Flow (MNF) analysis, which estimates leakage by measuring flow during low-demand hours (typically 2:00–4:00 AM) and subtracting expected legitimate use [8,11,39,40,41].

The Burst and Background Estimate (BABE) method further breaks down losses into short-term bursts and long-term background leaks based on flow trends [28]. Another approach, the water–wastewater balance, estimates apparent losses by comparing supply data with wastewater outflows, accounting for outdoor use and infiltration [11,35,39,42]. Although bottom-up techniques provide higher spatial and temporal accuracy, they require more instrumentation and field effort. Once losses are quantified, utilities can proceed to and localisation, to diagnose the specific sources of real losses within the system.

3.2. Leakage Detection Methods

In the literature, the simplest categorisation of leakage detection techniques can be divided into hardware- and software-based methods. The criteria for classifying hardware-based methods are determined by the characteristics of the devices employed, whereas those for software-based methods are based on model-based and data-driven approaches. A detailed hierarchical classification system for leakage detection methods can be found in [10,14].

3.2.1. Hardware-Based Leakage Detection Methods

Hardware-based leak detection techniques are vital for identifying leaks across water distribution networks. These approaches use sensors and technologies to detect leaks and provide real-time data for a prompt response. Examples include inside-pipe robotic devices and outside-pipe sensors like acoustic devices, radar, infrared, and remote sensing [15,43].

Table 1. Summary of comparison of hardware leakage detection methods.

Method	Principle	Applications	Benefits	Drawbacks	Reference
Acoustic Emission	Detects leaks by listening to specific sounds inside the pipe via instruments like hydrophones, listening sticks, accelerometers, and correlators. Combines microphones or correlators for precise leak location.	Leak detection in pressurised systems.	Real-time detection of leaks.	Not suitable for large leaks.	[10,15]
Fibre Optics	Detects changes in temperature, pressure, or strain in pipe walls.	Monitoring pipelines for leakage.	Monitors pressure and temperature simultaneously. Covers kilometres of pipeline in hours. Survives harsh conditions.	Expensive to install. Can only be used in linear pipelines.	[15,44]
Negative-Pressure Waves	Detect leaks by measuring sudden pressure drops, using sensors to localise leaks efficiently.	Leak detection in pipelines.	High accuracy and can localise leaks.	Accuracy depends on pipe material, diameter, and network complexity.	[6]
Closed Circuit Television (CCTV)	Use of cameras to detect leaks.	Inspection of pipelines.	Direct pipe condition observation.	Time-consuming and highly costly. Low level of reliability.	[15]
Ground Penetration Radar (GPR)	Uses radio waves to image the subsurface and detect reflections, identifying soil gaps to locate pipe leaks.	Detects leaks in buried pipes.	Able to detect leaks in most of the pipe via high-resolution images.	Penetration depth varies with soil type. Requires an experienced operator and has high operational costs.	[15,25]
Infrared Thermography	Uses infrared cameras to detect leaks by capturing images of cooler surfaces and identifying temperature differences.	Used to detect leaks under various surface conditions.	Less costly and can be utilised in real-time applications. Non-contact, non-destructive technique.	Requires operator experience. Detection depends on soil type and accuracy is affected by weather conditions.	[10,25,44]
RFID (Radio Frequency Identification)	Detects environmental changes and transfers signals to an RFID reader; requires installation.	Monitoring pipelines in areas which are hard to access.	Low power usage and wireless communication.	Limitations because of the range of RFID.	[9,15]
Leakage Pinpointing Methods	Combines microphones or correlators for precise leak detection. Main approaches include leak noise correlators (LNCs), the Tracer Gas Technique (TGT), and Pig-Mounted Acoustic (PMA).	Provides precise location of leaks.	High precision; can detect leaks in pipes of various sizes.	Costly and dependent on the leak location in the pipe. Water quality issues can occur during indoor pipe access.	[15]

summarises hardware-based methods reviewed in the literature with their working principles, applications, pros, and cons, whereas Figure 6 depicts several hardware-based methods found in the literature.

3.2.2. Software-Based Leakage Detection Methods

According to [10,15,45], software-based methods use algorithms and data analysis to identify WDN leaks through model-based or data-driven approaches. Model-based methods use hydraulic models to detect and locate leakage. Software for modelling includes EPANET, LOOP, Info Works WS, and WaterGEMS for single-phase flow, whereas PIPEPHASE, OLGA, and Aspen PIPE™ handle multi-phase flow [14], allowing instantaneous and long-term tests, and it may take some steps to calibrate and modify the model. After the model is successfully calibrated, various leakage detection strategies can be employed to predict and identify potential leak locations and their respective sizes [10]. These strategies can be classified into negative pressure wave (NPW) detection, pressure residual vector, sensitivity-based, classifier-based, and calibration methods [10,14,45,46].

As in Refs. [46,47,48], data-driven methods use ML and DL techniques to identify leakages without requiring a model. Data acquisition, pre-processing, and transformation are required before classifying and predicting leakages. ML techniques, such as supervised, unsupervised, or reinforcement learning, and DL are data-based. DRL combines RL with DL for sequential decision-making [16,49,50,51,52,53]. Based on these techniques, data-driven approaches to leak detection in WDNs can be grouped into four primary subcategories: prediction, classification, statistical, signal processing, and classification [10,14,54]. Data-driven approaches use distinct algorithms to detect and classify leakages. Prediction-classification methods predict system behaviour using algorithms such as Support Vector Regression (SVR), Kalman Filter (KF), Mixed Density Networks (MDN), Fuzzy Inference Systems (FIS), Bayesian Inference Systems (BIS), Recurrent Neural Networks (RNN), and Long Short-Term Memory (LSTM) networks. Statistical methods detect anomalies through Statistical Process Control (SPC), Principal Component Analysis (PCA), and Independent Component Analysis (ICA). Classification methods distinguish leak patterns using Artificial Neural Networks (ANN), Convolutional Neural Networks (CNN), time-delay neural networks, self-organising maps (SOM), and Support Vector Machines (SVM). Signal processing methods analyse sensor signals using Short-Time Fourier transform (STFT), Wavelet transform (WT, DWT), Multi-Layer perceptron (MLP), and Dempster-Shafer (D-S) fusion [10,14,15,46,47,54].

Refs. [18,21,22,55] introduced a hybrid technique combining model-based and data-driven methods like EPANET for hydraulic modelling with Deep Reinforcement Learning. This integration leverages simulations and real-time data adaptation, improves water quality, reduces energy consumption, optimises pump efficiency, and enhances leak prevention. This approach shows the potential of AI in addressing infrastructure challenges and promoting sustainable urban water management. Model-based approaches are advantageous when network data are limited, but hydraulic models are accessible. Conversely, data-driven methods suitcases with substantial historical network data. The hybrid method capitalises on the strengths of both approaches [16].

Table 2. Summary of comparison of software-based leakage detection methods.

Method	Principle	Benefits	Drawbacks	Reference
Statistical Methods	Monitor sensor readings to ensure normal operations, as deviations from standard patterns may indicate the presence of leaks.	Intelligent algorithms are used. Capable of multi-leakage detection.	A large database is needed for better performance. Not suitable for pipelines lacking database entries.	[45]
Sensitivity Matrix-Based	Based on pressure measurement and sensitivity analysis of WDNs.	Easy to perform leak---age detection under ideal conditions.	Uncertainties in nodal demands and noise measurement.	[47]
Volume Balancing	Looks for difference between input and output to detect leakage.	Simple and based on the principle of conservation of mass.	Less effective in complex systems where demand is variable.	[45,48]
Negative Pressure Wave (NPW)	Performing the analysis of pressure wave signals generated to detect leaks.	Low equipment investment and maintenance cost.	Poor detection accuracy and requires strategic sensor placement.	[10,45,48]
Real-Time Transient Method	Employing simulations that model the dynamic behaviour of water distribution networks to identify leaks.	Real-time leak detection and localisation.	The simulations are not easy to calibrate because of their complexity and require high computational power.	[10,27,45]
Pressure Point Analysis	Identify pipeline leaks by observing pressure reductions that fall beneath a specified threshold throughout the system.	Easy to use and quick to install.	Cannot detect small leaks.	[45,56]
Fuzzy Methods	Using fuzzy logic fundamentals to interpret uncertain data.	High accuracy and outputs are easy to interpret.	High computational complexity.	[15]
Computational Fluid Dynamics	Utilises numerical analysis and data structures to model free-stream flow.	Effective in high dimensional spaces and does not require explicit statistical models.	Hard to predict the leaks in pipelines.	[15]
Water Balance Method	The water balance method calculates water losses by comparing the total input volume with accounted-for consumption and uses.	Effective in quantifying total water loss.	Hard to pinpoint leaks.	[6]
Support Vector Machine (SVM)	Utilises classification and regression methodologies to classify data into leak and non-leak categories.	Can process noisy data with significant unpredictability.	Could be less effective for overlapping data points.	[15,46,48]
Kalman Filtering	Employs a series of observations to estimate the system’s condition and identify leaks based on the collected data.	Simple to understand and implement.	Assumes that the system might exhibit linear behaviour, which may not always be the case.	[46,48]
K-Nearest Neighbours (KNN)	Detects leakages through comparison of present-day information with historical leak trends.	Can model complex nonlinear relationships with flexibility.	May struggle when handling high-dimensional data and is particularly sensitive to noise within the data.	[15,46]
Artificial Neural Networks (ANNs)	Mimic the human brain to process the prediction and detection of leaks. Based on input data.	Accurately detect patterns and anomalies in image data.	Require careful design as performance is dependent on many variables.	[15,48]
Conventional Neural Networks (CNNs)	Detect leaks from spatial data, such as images from sensors.	Easy to perform leak---age detection under ideal conditions.	Need significant computational resources with large datasets for training.	[15,48]

summarises the principles, benefits, and drawbacks, while Figure 7 illustrates various software-based methods reviewed in literature.

3.3. Leakage Prevention (Control)

Leakage control mitigates the impact on water distribution systems by reducing the likelihood and severity of leaks through operational changes. Asset management and pressure management are key tools for prevention, although asset renewal is costly and time-consuming [31]. Pressure management is widely recognised as the most effective and economical technique for proactive control. Pressure drives background leakage and burst events, with empirical studies showing strong correlation between leakage rates and pressure head [57,58]. A more comprehensive relationship was derived from the fixed area and variable area (FAVAD) concept by May (1994) [57,59,60,61]. Reducing excessive pressure not only lowers leakage rates, but also decreases the risk of new bursts, reduces energy use, and contributes to lower operational costs and carbon emissions [29,58]. Pressure management (PM) is achieved through a range of actuating components such as Pressure-Reducing Valves (PRVs), Pressure-Sustaining Valves (PSVs), and Pumps as Turbines (PATs) and These devices allow for both static and dynamic control of pressures within DMAs and across the network [58,62]. Modern pressure management has evolved from fixed settings to more dynamic, real-time control strategies, including Fixed Outlet Pressure Control (FOPC), Time-Modulated Pressure Control (TMPC), Flow-Modulated Pressure Control (FMPC), Closed-Loop Pressure Control (CLPC), parameter-less controllers and optimisation-based approaches. These strategies differ in their complexity, responsiveness, and implementation costs [13,62].

Pressure management (PM) techniques in (WDNs) include pressure-regulating valves (PRVs), district metered areas (DMAs), and variable-speed pumps (VSPs) to minimise leakage. Optimisation methods are mathematical and metaheuristic. Mathematical optimisation defines problems using hydraulic equations, employing approaches such as linear programming (LP), nonlinear programming (NLP), mixed-integer linear programming (MILP), and mixed-integer nonlinear programming (MINLP). Metaheuristic techniques, including the Genetic Algorithm (GA) and simulated annealing (SA), operate independently of hydraulic simulators. GA uses stochastic evolution, refining solution populations based on fitness functions, while SA navigates solutions by reducing acceptance of less optimal outcomes [63,64].

One of the initial efforts by [65] demonstrated the practical benefits of PRVs by achieving an 18% leakage reduction in the Benchmark WDS through the installation of four to six valves. Building on this, Ref. [66] introduced a multi-objective NSGA-II algorithm for optimising PRV location and control settings, leading to a 19% reduction in leakage. To reduce the computational overhead, Ref. [67] proposed the Pressure Reference Method (PRM), a rule-based hydraulic simulation approach that simplifies PRV site selection based on pressure thresholds. Further refinement was offered in [68] by applying a Genetic Algorithm (GA) to optimise PRV placement in the WDS of Buja, Italy, resulting in considerable water savings and a 15.76% average pressure reduction. Around the same time [69], a hybrid approach combining GA and Linear Programming (LP) has been proposed for valve placement and pipe replacement. More complex formulations have been developed for large-scale networks. Ref. [70] applied a mixed-integer nonlinear programming (MINLP) model for PRV localisation in a WDS comprising 2643 pipelines and 1890 nodes, successfully mitigating excess pressure across the system. The authors in [71] focused on achieving multi-objective optimisation using NSGA-II, effectively identifying optimal valve placements while minimising water loss. De Paola et.al [72] applied a Harmony Search (HS) algorithm to determine optimal PRV control settings, which yielded better outcomes than traditional GA approaches in terms of system pressure reduction. This method effectively balanced valve installation costs against leakage volume, outperforming the GA alone in terms of computational efficiency. Ref. [73] extended the PRM framework by integrating it with NSGA-II to enhance PRV setting optimisation, achieving a 20.64% reduction in leakage, an improvement over earlier techniques. Ref. [74] utilised Particle Swarm Optimisation (PSO) to fine-tune PRV settings in large-scale WDNs, demonstrating the continued relevance and adaptability of metaheuristic approaches in modern leakage prevention efforts. Ref. [75] modified the reference pressure algorithm with nodal matrix analysis to optimise PRV placement in large-scale WDNs. Applied to the Campos do Conde II network in Brazil, the proposed method reduced candidate valve locations from 22 to 4 and achieved a 20.08% reduction in leakage, demonstrating improved pressure management and network efficiency [76] compared to Sequential Addition (SA) and NSGA-II in optimising valve locations for stormwater management. Their study confirmed the higher accuracy of NSGA-II, albeit with increased computational demands. Ref. [77] developed a graph-based algorithm using EPANET to identify optimal PRV locations and setpoints by targeting pipes with extensive downstream networks and high pressures, thereby achieving efficient pressure control and leakage reduction.

In drought-affected Central Chile, Ref. [78] advanced pressure management was implemented in La Calera to optimise existing PRVs. The intervention reduced the minimum night flow by 10.12% and improved the service by lowering the sub target pressure events by 52%, demonstrating effective leakage control and service enhancement. Moreover, Ref. [79] integrated energy recovery turbines with PRVs offer the dual benefits of leakage prevention and renewable energy generation. Such optimisation-based pressure control frameworks allow utilities to operate their systems more sustainably while simultaneously reducing water loss and energy usage. Ref. [80] proposed a two-stage optimisation approach in which PRVs are optimally placed and dynamically operated, particularly during low-demand periods, to minimise background leakage without compromising service levels. Their use of simulated annealing enabled the fine-tuning of valve settings across multiple pressure zones, leading to substantial reductions in the average nighttime leakage. Ref. [63] proposed a sequential optimisation approach combining PRVs, variable-speed pumps, and DMAs to manage the pressure in WDNs. By evaluating benefit–cost ratios and applying multi-criteria analysis, the method identified optimal sequences for leakage reduction, balancing economic, hydraulic, and environmental objectives.

One of the earlier contributions to pump optimisation was from [81], who demonstrated the effectiveness of variable-speed pumps in managing pressure levels and reducing leakage. However, their findings cautioned that reducing pressure during peak demand hours could compromise service adequacy. Building upon this, Ref. [82] employed a Genetic Algorithm (GA) to optimise water tank levels in the Tehran WDS, successfully reducing leakage from 11% to 5%. Ref. [83] added further sophistication using fuzzy linear programming to improve the estimation of demand patterns, thereby refining the pressure control responses. Later developments focused on improving the performance of pressure controllers using optimisation algorithms. A more advanced real-world implementation was conducted by [84], who introduced the OPIR software system in Poznan. This tool predicted the pressure and flow demand for the next 48 h using historical data, enabling dynamic pump scheduling and PRV control. The application resulted in a 20% reduction in background leakage and significant energy savings of EUR 21,500 per year. A study by [85] proposed a PAT-based pressure control strategy for WDNs, replacing PCVs with variable-speed pumps. Using EPANET simulations, the method-maintained pressure and flow while recovering 270,192.19 kWh/year. Variable-speed operation ensured higher efficiency (0.62–0.64) with a 27-month payback NPV of USD 64,476.18, and IRR of 63%, demonstrating both hydraulic and economic viability.

3.4. Evaluation Criteria and Performance of Leak Management Methods

The effectiveness of various leak management methods can be evaluated based on multiple criteria, including accuracy, localisation capabilities, real-time monitoring potential, cost, and false-alarm rates. These criteria are essential for determining the suitability of a particular method for specific applications in WDNs.

For leakage detection, methods such as leak noise correlators and loggers offer good accuracy and reliable localisation capabilities, with the added advantage of enabling real-time monitoring. However, their performance can vary in terms of cost, with correlators being more expensive and loggers having relatively low-cost alternatives [6,11,14,86]. Traditional tools, such as listening sticks, provide better accuracy but lack real-time monitoring features, limiting their applicability in dynamic environments [6,37]. Advanced techniques, such as fibre optics and infrared thermography, deliver better accuracy and low false alarm rates, making them highly effective for detecting leaks in WDNs. However, their installation and operational costs are prohibitive [6,14,15,27]. Similarly, ground-penetrating radar (GPR) and tracer gas methods provide excellent localisation, but are often constrained by high costs and medium false alarm rates, reducing their appeal for large-scale implementation [10,14,15,25].

In the realm of software-based methods, artificial neural networks (ANNs) and convolutional neural networks (CNNs) offer superior accuracy and real-time monitoring capabilities, coupled with low false alarm rates [6,15,48]. However, their high computational costs and data requirements may pose challenges for their practical deployment. Techniques such as Support Vector Machines (SVM) and fuzzy methods also perform well in terms of accuracy and localisation, with fuzzy methods exhibiting notably high precision and low false-alarm rates [6,15,37,48]. Other approaches, such as Kalman filtering and k-nearest neighbours (KNN), are versatile in achieving good accuracy and low false alarm rates but vary in computational and implementation costs. Simpler techniques, including the water balance method and pressure point analysis, are cost-effective but exhibit limitations in accuracy and localisation, which can hinder their broader application [15,46,48]. Methods such as computational fluid dynamics (CFD) strike a balance by offering better accuracy and localisation but are associated with higher costs and computational demands [15,87]. On the more traditional end, water balance methods and pressure point analysis are low-cost and easy to implement but suffer from limited accuracy and localisation, making them less suitable for complex WDNs [6,10,46,88].

In addition to detection, leakage prevention strategies are critical. Despite the proven benefits of pressure control, limitations exist, including reliance on accurate models, high computational demands, limited adaptability to changing network conditions, and the need for substantial infrastructure and data. These methods often struggle with real-time applications, particularly in large-scale or dynamically varying systems [89,90,91,92,93]. More advanced optimisation algorithms, such as Genetic Algorithms (GAs), are computationally intensive and do not guarantee the best outcome. They also require a substantial number of samples to obtain better results and can consume a lot of time [92,94,95,96].

To overcome the limitations of these traditional methods, recent focus has shifted toward emerging ML techniques, such as RL. Several studies have combined RL with DL, giving rise to DRL [49,50,53,97,98]. These approaches can offer scalable, model-free solutions that learn optimal control policies through interaction with the environment, reducing dependence on predefined parameters, and allowing real-time adaptability [16,18,21,22,89]. DRL merges deep learning’s representational strength with reinforcement learning’s trial-and-error strategy, which has shown promise in managing complex, nonlinear control problems [18] and has been successfully applied in domains such as robotics, control systems, and smart infrastructure [16]. In the context of WDNs, DRL have been used for system optimisation and leakage prevention through pressure regulation, valve control, and pump optimisation [16,18,21,22,23,99,100,101]. Despite these advancements, the application of DRL specifically for comprehensive leakage management remains underexplored and presents a promising avenue for future research.

4. Deep Reinforcement Learning Background

As shown in Figure 8, Artificial Intelligence (AI) includes key subfields such as Machine Learning (ML) and Deep Learning (DL), which have been widely adopted in water system management [102,103,104,105,106]. ML focuses on algorithms that learn patterns from data to make predictions or decisions without explicit programming [103,106]. DL, as a subset of ML, uses multi-layered neural networks inspired by the human brain to process large and complex datasets [105,107,108]. ML is typically divided into supervised, unsupervised, semi-supervised, and reinforcement learning, each with distinct roles in classification, clustering, and control tasks [106,109]. RL is suited to dynamic decision-making problems, where an agent learns to act within an environment to maximise cumulative rewards [110]. DRL builds on RL by integrating deep neural networks to handle large state and action spaces and can be categorised into value-based, policy-based, and actor–critic approaches [106,111].

4.1. Reinforcement Learning

Reinforcement Learning is a type of machine learning in which an agent learns to make decisions by interacting with an environment to achieve a defined goal [50,112,113,114,115,116]. The agent starts without prior knowledge and improves its strategy by receiving feedback in the form of rewards or penalties based on its actions. This trial-and-error learning enables the agent to optimise its behaviour over time by maximising the cumulative rewards [117]. The standard RL framework, illustrated in Figure 9, is typically modelled as a Markov decision process and is closely related to dynamic programming techniques RL has been applied across diverse fields, such as robotics, healthcare, and game-playing, and continues to evolve with recent developments in deep and quantum reinforcement learning [49,98,113].

As shown in Figure 10, RL can be considered a Markov decision process (MDP) that involves several elements [50,53,97,117,118,119,120]: a set of states (S) representing possible situations in the environment, a set of actions (A) that the agent can take, the environment dynamics and transition probabilities (P) that determine how actions change the state, the reward function (R) that signals the desirability of state-action pairs, and the discount factor (γ) that determines the weight given to immediate versus future rewards. At each step, the agent is in a state (s0,s1,s2) takes an action (a0,a1) based on its policy, receives a reward (r0,r1), and moves to the next state. The transitions between states are governed by the environment’s dynamics, and the rewards provide feedback for learning. This loop of state → action → reward → next state forms the basis of reinforcement learning.

In general, the reward is a signal of how “good” or “bad” a situation is, which helps the agent learn to differentiate more promising from less desirable decision trajectories. A trajectory is a sequence of states and actions performed by an agent. Reward (r) is a crucial identifier that determines whether an agent’s actions are beneficial or harmful. The cumulative reward over a trajectory is called the return (R(τ)), and includes a discount factor γ ∈ [0, 1], as shown in Equation (1) [16,51,97,117,119,120,121,122].

$$\mathrm{R}\left(\mathsf{\tau }\right)={\mathrm{r}}_{\mathrm{t}}+{\gamma \mathrm{r}}_{\mathrm{t}+1}+{\mathsf{\gamma }}^2{\mathrm{r}}_{\mathrm{t}+2}+\cdots =\sum _{k=0}^{\infty }{\mathsf{\gamma }}^{\mathrm{k}} {\mathrm{r}}_{\mathrm{t}+\mathrm{k}}$$

(1)

A process in reinforcement learning that fulfils the Markov property is termed a Markov decision process [97,122]. RL algorithms can be model-based, in which the agent learns the model or transition function of the environment directly; however, there is no exploration required and model-free, in which the agent learns either a value or policy function through trial and error [18,121,122], and a detailed discussion of RL algorithms can be found in these studies.

The three main approaches used in RL are Monte Carlo (MC), Temporal Difference (TD),and Dynamic Programming (DP) [16,119,121]. DP solves complex problems by breaking them into subtasks with defined states, actions, and rewards, which are effective for MDPs, while MC relies on trial and error and learning from experience, without requiring a model. This method is suitable for uncertain environments. TD combines MC and DP by updating value estimates after each step, thereby supporting ongoing tasks and incremental learning.

4.2. Deep Reinforcement Learning (DRL)

DRL integrates RL concepts with DL techniques, as shown in Figure 11 [16,49,53,98,115,122,123]. It Involves agent learning through interaction with the environment by taking action and receiving feedback in the form of rewards or penalties. This process helps the agent identify valuable actions and develop optimal policies to maximise cumulative rewards. The agent must perceive the current state, evaluate the actions, and understand the consequences to effectively guide decision-making [53,98,114,115]. It helps agents learn effective behaviours through trial and error, using repeated actions, feedback, and rewards to guide their learning. For example, a robot can be trained by attempting tasks, receiving rewards for successful outcomes, refining its actions to maximise rewards, and eventually mastering how to achieve optimal results [16,50,52,108,122]. DRL involves several elements: observed spaces (S), action spaces (At), environment dynamics and transition probabilities (P), reward function (Rt), and discount factor (γ), which balance immediate and future rewards [49,50,53,98,115]. The agent observes the initial environment state (S1), selects a policy, performs an action, and receives a reward. This process is iterated to determine the optimal policy through trial and error. A visual illustration of this process is shown in Figure 12. Unlike traditional RL, which uses tabular data, DRL incorporates a deep neural network, which is a key distinguishing feature [22]. Further details on the DRL architecture and processes are available in various studies [16,49,50,51,52,53,98,115,122,123].

4.3. RL and DRL Algorithms

Deep Reinforcement Learning algorithms are primarily divided into two categories: model-based, where the agent learns the model or transition function of the environment directly, and model-free, where the focus is on learning a value or policy function. These algorithms are further classified into value-based, policy-based, and a combination of both [120].

• Value-based DRL algorithms: These methods learn action values in different states and derive a policy from these values. Examples include Deep Q-Networks (DQNs) and its variants like Double DQN, Duelling DQN, and Rainbow DQN. Value-based methods excel at discrete action spaces but may struggle with continuous action spaces [16,18,52,121].
• Policy-based DRL algorithms: These approaches directly learn policy-mapping states to actions without an intermediate-value function. Methods such as REINFORCE, Trust Region Policy Optimisation (TRPO), and Proximal Policy Optimisation (PPO) fall into this category. Policy-based methods naturally handle continuous action spaces and typically have better convergence properties but may suffer from high variance [114].
• Actor–critic DRL algorithms: These hybrid methods combine value-based and policy-based approaches by maintaining both a policy (actor) and a value function (critic). The actor selects actions, whereas the critic evaluates these actions by estimating the value function. Examples include Advantage Actor–Critic (A2C), Deep Deterministic Policy Gradient (DDPG), and Soft Actor–Critic (SAC). This synergy enables more effective learning, as the policy adjusts based on feedback from the value function [16,51,114,122].

The selection of an appropriate DRL algorithm depends on the specific characteristics of the problem, including whether the action space is discrete or continuous, dimensionality of the state space, exploration–exploitation balance required, and computational constraints. In water distribution systems, where actions such as valve adjustments and pump controls may be continuous or discrete, the choice of the algorithm has significant implications for performance and practicality.

4.4. Applications of DRL in Water Sector

DRL has had a transformative impact across numerous fields by effectively solving complex problems and advancing sequential decision-making capabilities. Compared to other industries, DRL is gaining popularity in the urban water sector. Figure 13 provides a summary of the reviewed literature where DRL has been utilised in urban water networks.

4.4.1. DRL in Stormwater

Ref. [124] applied DRL for the real-time control of urban stormwater systems. The authors developed a DRL-based control algorithm using Deep Q-Networks to autonomously manage stormwater infrastructure. The model was trained using a Python-based co-simulation framework integrating SWMM and TensorFlow, allowing the agent to interact with the simulated watersheds under thousands of storm scenarios. Through various reward functions, this paper evaluated control policies across configurations, showing that structured rewards and proper neural network design enhance performance. The approach outperformed uncontrolled systems under intense storms while highlighting the challenges of scaling DRL to complex water networks.

Ref. [125] utilised the Deep Deterministic Policy Gradient (DDPG) algorithm with historical rainfall data, achieving a 70.5% reduction in flooding volume compared to passive systems, demonstrating robustness against noisy data. Ref. [126] developed a Model Predictive Control (MPC) algorithm using EPA SWMM5 and swmm_mpc library, reducing flooding by 32% and maximum node flood volume by 52% during sea-level rise conditions. Both DDPG and MPC effectively mitigated flooding, with DDPG excelling in real-time control and MPC being more efficient in sea-level rise scenarios. Ref. [127] trained a deep-RL agent in a physics-based model with two controllable retention ponds, achieving 32% less flooding than passive systems, matching the MPC’s reduction within 3% at 88× speedup, and outperforming rule-based control by 19%. Ref. [128] designed an integrated multi-RL and MPC framework for real-time control in CSO and flooding mitigation, comparing five RL algorithms, implementing a security system for control decisions, developing a hybrid “voting” system, and conducting robustness tests.

4.4.2. DRL in Water Distribution Networks

In the context of water distribution systems (WDS), Ref. [22] stated that DRL can optimise pump operations in real time, achieving performance comparable to that of conventional methods, but at twice the speed. They employed a Duelling Deep Q-Network (D-DQN) to optimise the pump speeds for hydraulic efficiency under random demand conditions. The algorithm minimises the tank inflow/outflow while maintaining an acceptable pressure across all nodes. The reward combines consumer satisfaction (based on problematic nodes), pump efficiency (actual-to-peak performance ratio), and supply balance (pump versus reservoir contribution). Tested on both small (Anytown) and large (D-town) networks, D-DQN performed comparably to Differential Evolution (DE) and outperformed other methods such as PSO and random search. Unlike other approaches, it uses live data for real-time control, making it highly suitable for practical deployment.

Ref. [89] explored DRL for optimising pressure-reducing valves (PRVs) in a WDS, focusing on a model-free approach with minimal elaboration of the DRL methodology, aiming to predict optimal pressure distributions. Furthermore, in another study [26], the authors addressed the challenge of inefficient scheduling in water distribution systems, which leads to energy wastage, overflow, and failure to satisfy demand. To address the complexity of real-time system management, they proposed using deep reinforcement learning DRL with function approximation to handle the state explosion problem. They trained a long short-term memory (LSTM)-based RL agent to optimise energy usage while maintaining safe operations. The agent’s performance was compared with of that a baseline controller and a fuzzy logic controller, both of which reflect human experience. The results indicate that the RL agent outperformed both in terms of energy efficiency and operational safety. Ref. [21] applied a baseline reinforcement learning approach using Q-learning to optimise valve set points for active pressure control in a water distribution network. Implemented within an EPANET-Python simulation environment, the agent learned to adjust pressure-reducing valves to lower the average system pressure while maintaining compliance with the OFWAT’s regulatory threshold of 10 m. This study focused on the D-Town network, where the RL agent demonstrated continuous performance improvement, ultimately identifying an optimal set point of 26 m. This led to a 2% reduction in the average system pressure through minimal changes in only two valve settings. These findings highlight the potential of simple RL algorithms for real-time control and infrastructure resilience, with the authors suggesting that further gains can be achieved through the application of deep reinforcement learning techniques.

Ref. [18] developed the DRL-EPANET framework by integrating a Branching Duelling Q-Network (BDQ) with EPANET to optimise pressure control in water distribution systems. An enhanced variant, BDQ with Fixed Actions (BDQF), improves the performance in sparse action scenarios. Validated on 10 real-world networks, the framework achieved up to a 26% improvement in mean pressure and handled uncertainties, such as pipe bursts and demand fluctuations. Although focused on pressure control, the method is adaptable to pump optimisation, energy management, and water quality applications. Additionally, Ref. [23] proposed an enhanced DRL framework, E-PPO, for real-time pump scheduling that mitigates traditional limitations by learning suboptimal schedules and incorporating a tank level penalty to minimise energy costs by up to 11.14% while maintaining optimal water levels. Furthermore, Ref. [99] modelled real-time pump and valve scheduling in WDNs as a cooperative multi-agent MDP and applied a multi-agent deep deterministic policy gradient (MADDPG) showing that water loss, driven by consumer demand patterns rather than energy tariffs, dominates optimisation and achieves significantly faster and more robust solutions than GA, PSO, and DE. Ref. [129] proposed a knowledge-assisted RL framework (KA-PPO) that integrates historical WDN data into a proximal policy optimisation algorithm using only nodal pressures to optimise pump scheduling in arbitrarily shaped networks under time-varying demand, outperforming the Nelder–Mead method on a benchmark 22-node two-pump system. Moreover, Ref. [24] utilised hybrid reinforcement learning for pump sustainability in real-world water networks. These studies illustrate the broad potential of DRL in optimising water systems, enhancing operational efficiency, and reducing costs.

4.4.3. DRL in Water Quality Applications

In Ref. [130], a parity-Q DQN algorithm was introduced to identify vulnerable zones based on water quality resilience in São Paulo. They created an RL-compatible testbed using a Deep-Sea Treasure (DST) environment and proposed objective relation mapping (ORM) to guide policy learning. Testing two DST scenarios and a water quality resilience model, it outperformed MP-DQN, MO-MCTS, and MPQ in terms of generational distance, IGD, and hypervolume. Ref. [131] developed a Deep Reinforcement Learning (DQN)-based autonomous system to optimize aeration in a membrane bioreactor (MBR) wastewater treatment plant. It reduced energy use by over 33% while maintaining effluent quality, outperforming manual and conventional RL-based systems by dynamically adjusting dissolved oxygen levels based on influent conditions. Ref. [132] developed a pH controller for electroplating wastewater using RL with PSO-based tuning. In Ref. [133], DDPG was used for pH and level regulation in stirred tank reactors, showing better stability than PI controllers. Ref. [134] applied DRL to control raw water pumping under high sediment loads, thereby reducing energy and suspended solids.

Ref. [135] implemented an actor–critic DRL approach with transfer learning for aeration control in wastewater treatment, outperforming traditional strategies. Ref. [136] applied multi-agent autonomous surface vehicles to water quality monitoring. They combined local Gaussian processes with deep reinforcement learning to achieve fleet control. Local Gaussian processes capture spatial variations in water quality better than global models. A deep convolutional policy uses the mean and variance of the model, guided by an information gain reward. The agents were trained using double deep Q-learning with consensus-based heuristics. Simulations showed a 24% reduction in mean absolute error, outperforming state-of-the-art methods by 20–24% in monitoring water quality. Ref. [137] integrated DRL with a CNN-based global sensing model for water quality monitoring in river basins. Their framework uses DRL-guided static sensor deployment to optimise the placement of spatiotemporal patterns. The CNN model with the attention mechanism infers TN concentrations at unmonitored locations using existing sensor data, outperforming conventional interpolation techniques.

In summary, Deep Reinforcement Learning holds transformative potential for the water sector; however, its application in WDNs is still in its early stages. WDNs are complex systems that require relentless monitoring, swift decision-making, and the ability to navigate uncertain and dynamic conditions. The integration of DRL into WDNs is not only beneficial but also essential, especially for critical tasks such as leak detection and prevention. However, to unlock the full potential of DRL in this domain, it is imperative to address existing challenges. By overcoming these hurdles, we can revolutionise the management of WDNs and ensure more efficient, reliable, and sustainable water distribution systems in the future.

4.5. Challenges of DRL in WDNs

Based on a literature review, the successful implementation of Deep Reinforcement Learning in Water Distribution Networks (WDNs) presents several key challenges that must be addressed for effective water leakage management.

• Data Availability and Quality: One major challenge is the availability and quality of data. Incomplete or inaccurate data can significantly affect the performance of DRL algorithms. High-quality labelled data are crucial for training DRL models; however, acquiring these data is difficult because of sparse and noisy measurements that are influenced by inaccuracies and environmental factors. The lack of historical data further complicates the model training, making it more difficult to create robust models [16,24,99]. This scarcity can hinder the training and accuracy of DRL models.
• Simulation-to-Real Gap: Most RL research on water systems has been confined to simulation-based environments. Although simulations offer safety and control platforms for model development, they often fail to capture the full complexity and unpredictability of real-world systems. Consequently, a significant simulation-to-reality gap exists, which impedes the operational deployment of RL agents. Bridging this gap will require robust techniques for policy generalisation, domain adaptation, and transfer learning to ensure that RL models can function reliably in practical water distribution scenarios [116].
• Hydraulic Simulator Integration: Efficient integration with hydraulic simulators such as EPANET is crucial for modelling real-world WDN behaviour; however, this integration introduces latency issues owing to the computational overhead of the simulations [16,18,129].
• Scalability in Large Networks: Scalability is also a significant concern for DRL applications. Conventional DRL algorithms may experience a decline in performance as the network size increases, with the number of possible states and actions growing exponentially. This makes it challenging for DRL algorithms to effectively manage larger WDNs. To address this issue, there is a demand for algorithms that can handle more intricate environments, such as hierarchical DRL algorithms, which enable learning in multiple abstractions [53].
• Computational Complexity: The computational complexity involved in training DRL agents is another challenge, particularly in complex systems such as WDNs. Training involves exploring several possible states and actions that require substantial computational resources. Without sufficient computational power, the training may be limited, thereby affecting the generalisation capabilities of the model [16,26,124,138].
• Sensitivity to Network Architecture: Sensitivity to the neural network architecture also presents a significant challenge in achieving optimal control performance. Architecture, including layers, neurones, and hyperparameters, plays a crucial role in effective learning. However, determining the best structure often requires trial and error, which is time-consuming and lacks systematic methods [16,124].
• Reward Function Design: Another challenge is the manual and often heuristic-based approach to reward function design, which involves significant trial and error and domain expertise. This not only prolongs the development cycle but also risks suboptimal learning outcomes. Future efforts should explore automated reward shaping using techniques such as inverse RL, meta-learning, or preference elicitation from domain experts to enable more structured and adaptive reward engineering [116].
• Uncertainty Handling: WDNs face various forms of uncertainty, such as fluctuating demand, leak occurrences, and sensor failures, which are difficult to accurately model. DRL systems must be robust to these fluctuations and capable of detecting anomalies to maintain reliability [16,24,99].
• Credit Assignment Problem: In dynamic environments, such as WDNs, rewards often occur long after a causative action, making it difficult to attribute outcomes accurately. This temporal credit assignment problem complicates learning and necessitates architectures capable of managing long-term dependencies [16,115].
• Exploration vs. Exploitation Trade-off: The exploration versus exploitation dilemma is a significant challenge for most RL and DRL algorithms, as agents often act in a manner that prioritises immediate rewards. Because an agent’s observations are influenced by its actions and its actions are driven by the rewards it receives, RL agents can become trapped in a cycle around a local optimum instead of discovering the global optimum through exploitation. To address this issue, introducing randomness into the agent’s behaviour is essential, as it allows the agent to gather new observations and potentially explore global optima [16].
• Leak-specific Challenges: Leakage events, particularly background leaks, present a unique challenge in DRL-based water network management because of their sparse, prolonged, and often undetectable nature. Unlike burst leaks, which have more identifiable hydraulic signatures, background leaks often persist silently and offer little or no immediate feedback from which DRL agents can learn. This creates a sparse reward environment that complicates agent training and may cause overfitting in normal (non-leak) scenarios. Additionally, DRL-based prevention strategies, such as pressure management, must overcome the complexity of minimising leakage while maintaining regulatory pressure thresholds without the benefit of clear detection signals. Therefore, developing effective DRL policies for leak-specific tasks requires careful reward shaping, environmental design, and policy generalisation [16,18,99].

5. Future Directions

Despite the emerging role of DRL in water systems, its application in leakage management remains at an early stage. Based on the reviewed studies, several research directions are recommended to guide future work.

• Domain-specific DRL Environment Development: There need to develop benchmarked, open-source DRL environments tailored to leakage detection and prevention tasks in WDNs. These should integrate realistic hydraulic models (e.g., EPANET and WNTR) and simulate various leak scenarios, sensor configurations, and operational constraints.
• Improved State Representation and Reward Design: Effective use of DRL depends on how well the environmental states and objectives are defined. Future work should explore encoding spatial–temporal patterns of pressure, flow, and demand into state vectors that reflect real-time network conditions. Similarly, multi-objective reward functions should be designed to capture the trade-offs between leak detection accuracy, computational efficiency, and operational feasibility. Incorporating domain knowledge, such as hydraulic principles, can further constrain learning toward physically meaningful and efficient solutions [18,129].
• Transferability Across Networks: Most DRL models are network-specific and require retraining when applied to new systems. To increase generalisability, researchers should develop methods that support transfer learning across different network topologies, demand patterns, and boundary conditions. This can involve domain adaptation, hybrid synthetic–real datasets, or curriculum learning strategies. Such approaches would reduce the dependence on labelled data and allow faster adaptation of models across utilities.
• Integration with Real-Time Systems: Bridging simulation-based DRL with real-time leak management systems remains a key challenge. Studies should explore hybrid DRL + heuristic control, online learning, or human-in-the-loop frameworks.
• Multi-Agent and Multi-Objective DRL: Given the distributed nature of WDNs, multi-agent reinforcement learning (MARL) can be explored to allow decentralised decision-making. Similarly, multi-objective formulations can balance leak detection with pressure control and energy savings [16].
• Explainability and Trustworthiness: Future models should include interpretability features to support decision makers, particularly in safety-critical scenarios involving large-scale leak events or system failures [16].
• Pilot Studies: Real-world pilot studies are essential for validating simulation-trained agents under field conditions. These studies should include rigorous monitoring protocols and allow researchers to assess both technical performance and operational viability. Hybrid model- and data-driven systems are particularly promising in this context, offering sensor-free yet hydraulically realistic environments for training and evaluation [16].

6. Conclusions

Advanced leak management techniques have been developed for water distribution networks because of ongoing concerns regarding substantial water losses, operational inefficiencies caused by ageing infrastructure, and limited monitoring capabilities. This review has revealed a clear and growing interest in applying artificial intelligence, particularly DRL, to leakage management in water distribution networks. Through structured keyword-based exploration, we observed strong research activity in traditional leak detection and prevention, followed by a gradual integration of AI and ML techniques. Although DRL has gained momentum in broader water system control, its direct application to leakage detection and prevention remains limited.

Our analysis indicates that most existing DRL studies focus on general pump scheduling and pressure optimisation, with only a few addressing leakage as a primary objective. Recurring gaps in the literature include the lack of benchmark environments, limited transferability across network topologies, and minimal real-world implementations. Overall, the review underscores the potential of DRL to transform leakage management, but also emphasises that current research is still in its early stages. More targeted efforts are required to develop specialised DRL models, integrate them with real-time systems, and test them under diverse hydraulic scenarios.