A Machine Learning Approach to Predicting the Turbidity from Filters in a Water Treatment Plant

Abstract

Rapid sand filtration is a critical step in the water treatment process, as its effectiveness directly impacts the supply of safe drinking water. However, optimising filtration processes in water treatment plants (WTPs) presents a significant challenge due to the varying operational parameters and conditions. This study applies explainable machine learning to enhance insights into predicting direct filtration operations at the Ålesund WTP in Norway. Three baseline models (Multiple Linear Regression, Support Vector Regression, and K-Nearest Neighbour (KNN)) and three ensemble models (Random Forest (RF), Extra Trees (ET), and XGBoost) were optimised using the GridSearchCV algorithm and implemented on seven filter units to predict their filtered water turbidity. The results indicate that ML models can reliably predict filtered water turbidity in WTPs, with Extra Trees models achieving the highest predictive performance (R² = 0.92). ET, RF, and KNN ranked as the three top-performing models using Alternative Technique for Order of Preference by Similarity to Ideal Solution (A-TOPSIS) ranking for the suite of algorithms used. The feature importance analysis ranked the filter runtime, flow rate, and bed level. SHAP interpretation of the best model provided actionable insights, revealing how operational adjustments during the ripening stage can help mitigate filter breakthroughs. These findings offer valuable guidance for plant operators and highlight the benefits of explainable machine learning in water quality management.

1. Introduction

Water treatment plants are under increasing stress from increasing levels of contaminants of public health significance in their raw water sources due to climate-induced runoffs and increasing anthropogenic activities in their catchment areas [1]. Removing these contaminants to safeguard public health requires cost-effective drinking water treatment processes. Granular filtration is an important step in all water treatment plants and plays a critical role in the removal of chemical and microbial contaminants of public health concern [2]. Depending on the media properties, granular media filters also play the role of adjusting/correcting physicochemical parameters such as pH, colour, odour, and taste. The contaminant removal process involved in granular media filtration includes straining larger particles from the raw water and adsorption and/or precipitation of dissolved heavy metals by redox reactions. It also includes removal mechanisms such as interception, sedimentation, and the Brownian motion of particles within the flow streamlines. Particle dynamics, such as deposition kinetics [3] and particle detachment mechanisms impacted by hydraulic gradients and backwashing [4], were investigated in early research on granular media filtration. Turbidity monitoring has been pivotal in understanding filter performance, detecting particle breakthroughs, and optimising water treatment processes [5].

Hendricks, Clunie William [6] demonstrated that turbidity measurements are useful for assessing filter performance and for monitoring particle retention and microbial releases during filtration in rapid sand filters. Due to the dynamic nature of rapid filter behaviour, particularly during ripening and breakthrough, short-term turbidity fluctuation may indicate an increased risk of microbial breakthrough. Guideline values for filtered water turbidity, ranging from 1 to 2 NTU, are recommended. For example, certain Drinking Water Treatment Plant (DWTP) configurations require turbidity below 1 NTU after settling and coagulation-flocculation steps [7]. However, modern best practices recommend a stricter target of ≤0.1 NTU to minimise the risk of pathogen breakthroughs [8]. These developments have been accompanied by a regulatory shift towards mandatory continuous turbidity measurements using in-line sensors for plant capacities exceeding 10,000 m³/day, thereby enabling real-time operational oversight with Supervisory Control and Data Acquisition (SCADA) systems.

Filter operations play a critical role in maintaining water quality. For instance, during the backwash recovery (ripening) period, the recommended duration is under 15 min. If turbidity exceeds the safe limits, operators can intervene with corrective measures such as: (i) starting the filter slowly, (ii) delaying filtration, (iii) filtering to waste, or (iv) adding coagulation to the setup to aid the filtration process. Post-ripening, filtered water quality typically stabilises, but turbidity spikes during the breakthrough phase ultimately necessitate ending the filter cycle [8].

Drinking water treatment plants must also account for dynamic influent conditions, operational variability, and confounding external effects. Confounding effects of holidays, day-of-week effects, seasons, temperature and precipitation have been documented by Hsieh, Nguyen [9]. For optimal operation of granular filters, understanding the dynamics of breakthrough and the impact on turbidity in filtered water is critical. A key tool for managing turbidity is the filter breakthrough curves theory, which helps to predict particle breakthrough based on headloss progression during filtration. Therefore, by monitoring filter headloss, the filters can be isolated by valve action and backwashed before the breakthrough occurs. In other cases, the filter runtime is set to a limit beyond which the breakthrough is anticipated, considering the raw water quality and operation settings, without directly being triggered by the headloss [10,11].

With the increasing availability of treatment plant data, data-driven methods, particularly machine learning (ML) are increasingly applied to optimise particle removal in filtration systems. Various studies leverage ML algorithms, such as Artificial Neural Networks (ANNs), Decision Trees (DTs), Random Forests (RFs), Extreme Gradient Boosting (XGBoost), and Improved Random Forest (IRF), which have been successfully implemented to predict the efficiency of nanofiltration, membrane filtration, and rapid sand filtration in water treatment plants [12,13,14]. These models enhance operational decision-making by identifying patterns in turbidity, headloss, and breakthrough behaviour, ultimately improving filter performance [12,15,16]. However, a key limitation of existing models is that they typically treat filters in isolation, overlooking the variability between filters and the operational interdependencies among them. This is a critical gap, as full-scale drinking water treatment plants operate with multiple filters whose performance is interconnected [17,18]. This study addresses this gap by introducing models that capture the variabilities and interdependencies of multiple filters in a real-life treatment plant.

The primary objective of this study is to apply machine learning (ML) techniques to predict filtered water turbidity in a drinking water treatment plant in Ålesund, Norway. By exploring these predictive insights and accounting for operational factors, this study aims to optimise filter operation parameters, thereby enhancing water quality and operational efficiency. Additionally, we explore how data-driven decision-making can improve plant resilience and resource management, contributing to the broader digital transformation of water treatment systems.

This study is organised as follows: Section 1 presents the introduction; Section 2 details the methodology used, including a description of the case study plant and the framework for implementing machine learning (ML) models. Section 3 presents the results and discussion, incorporating SHapley Additive exPlanations (SHAP) analysis of ML models’ interpretability along with the study’s broader implications. Finally, Section 4 provides the conclusions and suggests directions for future research.

2. Materials and Methods

2.1. Overview of Drinking Water Treatment Plant

Figure 1 shows an overview of the drinking water treatment plant. The drinking water plant draws its raw water from Lake Brusdalsvatnet in Ålesund, a city on the coast of western Norway, with an annual rainfall of approximately 1830 mm [19], contributing to the dilution of dissolved organic matter (DOM) in surface water sources. According to the Norwegian water (Vann-Nett) classification, this lake has a high ecological and good chemical status [20]. Abstracted from a stable depth to the Ålesund waterworks, the treatment process involves dosing CO₂ in-line to reduce the pH before chlorination. After this, the pre-treated water is passed through seven mono-media marble direct filters in a parallel arrangement. The combined effluent of the seven filter beds is redistributed to seven ultraviolet (UV) lamps for disinfection, which is collected in a clear water reservoir and transported to high-level reservoirs for distribution.

The inflow to the filter beds transitions to an open-channel flow with seven lateral-feeding manifolds with rectangular weir openings, which split to feed a gallery of seven in-house filters. The filters are housed to prevent freezing during winter and protect the influent water quality from deterioration. Each filter has a 20 m² bed area, a 2 m depth, and a media effective diameter of 3 mm (L/d ratio = 666.7). A typical filter runtime is 7 days; therefore, a one-week backwashing cycle corresponds to filters 1–7 from Monday to Sunday. Backwashing involves pressurised air and clean water to fluidise the filter bed and flush trapped particles into the backwash trough. A typical backwashing duration is 15 ± 2 min. During backwash, the filter bed is isolated using the valves at both the inflow and outflow ends of the filter. The backwash water is collected into the backwash trough above the filter bed, retained, and later discharged into a local stream.

The filtration setup for this DWTP comprises marble sand media for pH correction and corrosion control for the water distribution network since the lake has a slightly lower pH (6.6–6.8), like most natural lakes in Norway, with low pH and low DOM in surface water. The filter serves as an auxiliary function for particle removal, which affects the turbidity of the raw water. Dissolved heavy metals in the raw water precipitate during a rapid pH drop during the pre-treatment stage and increase when they pass through the marble (CaCO₃) media. The media is routinely replenished annually to maintain the bed level, which is affected by operational factors such as the total water filtered and the filter’s backwashing time. The backwashing process is initiated by shutting the outlet valve of the isolated filter. After closing the valve, pressurised air and clean water are flushed upwards to fluidise the filter bed, which enables trapped particles to float up and be collected by the backwashing trough.

2.2. Conceptual Framework of the Study

Figure 2 shows the conceptual framework adopted for ML development. The framework illustrates the workflow for developing and evaluating machine learning models to predict filtered water in DWTP filters. It comprises the interlinked steps of data collection and preprocessing from the plant, feature selection, model development and evaluation, and finally, model interpretation. These steps are elucidated in the sections below:

2.3. Data Collection, Structuring and Descriptive Analysis

The datasets for this study were collected from the Supervisory Control and Data Acquisition (SCADA) system via Application Programming Interface (API). Each data instance corresponds to a specific filter unit (Filter nr. ∈ [1, 2, 3, 4, 5, 6, 7]) within the Ålesund Drinking Water Treatment Plant. The study analysed four months (1 August 2023–1 December 2023) of high-resolution data (1 min interval) from the DWTP SCADA system, capturing valve opening (%), filtered water turbidity (NTU), flow rate (L/s), and bed level (m). This period was strategically selected to capture seasonal hydrodynamic effects, particularly lake upwelling and vertical mixing as predicted by the study lake’s hydrodynamic models [21], which have a marginal influence on the intake water quality at the abstraction point [22]. The filter runtime (in minutes) was derived from valve status (ON and OFF) and validated with the bed water level and operational reports. To assess the overall filtration performance, the combined filtered water turbidity from the gallery was computed as a flow-weighted average, providing a representative measure of treated water quality. Understanding the impact of unit filters on the collective system performance is essential for operational changes to optimise treatment efficiency. The Average Weighted Turbidity (AWT) at time $t$ is computed using Equation (1) below.

$$\mathrm{A}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{g}\mathrm{e} \mathrm{W}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}\mathrm{e}\mathrm{d} \mathrm{T}\mathrm{u}\mathrm{r}\mathrm{b}\mathrm{i}\mathrm{d}\mathrm{i}\mathrm{t}\mathrm{y}={\displaystyle \frac{ \sum _{i=1}^7(T_i\times Q_{i })}{\sum _{i=1}^7{Q}_i}}$$

(1)

where $T_i$ is the turbidity of the ${filter}_i$, and $Q_i$ is the flow rate (L/s) of the ith filter.

2.4. Machine Learning Models Development

2.4.1. Data Preprocessing and Feature Selection

The raw data preprocessing steps included imputation of missing values, where necessary. The NAN (Not a Number) turbidity readings, comprising 9.6% of the raw turbidity dataset, were addressed using the forward-fill imputation method. This method involves forward propagating the last valid observation to replace the missing values. These occurred intermittently, with a maximum of 20 consecutive rows affected during different time intervals as observed in a similar SCADA pattern [23]. The features selected for the machine learning models predicting the filtered water turbidity (NTU) of the individual filters were runtime (min.), flow (L/s), and level (m), based on operational relevance (validated by the plant manager) and supported by Pearson’s correlation. The valve opening (%) parameter was excluded from the input set to prevent data leakage, as it was directly used to derive the filter runtime data.

The machine learning models were developed to predict filtered water turbidity (NTU) for each individual filter using the following predictor variables: flow rate (L/s), bed level (m), and runtime (min). A total of 175,738 data rows were used for each filter, representing a monitoring period of 4 months. The dataset was partitioned into training (80%) and testing (20%) subsets for the machine learning models. The implementation and performance evaluation of the data-driven models are detailed in the subsequent sections.

2.4.2. Model Families and Evaluation

We evaluated a diverse suite of supervised machine learning model families to address the complexities and non-linearities of the DWTP filter datasets. These included three baseline models (Multiple Linear Regression (MLR), Support Vector Regression (SVR), K-Nearest Neighbours (KNN)) and three ensemble methods (Bagged Trees (Random Forest (RF) and Extra Trees (ET)) and Boosted Trees (Extreme Gradient Boosting (XGBoost))). All the features were pre-processed before model training to ensure optimal performance across the algorithm families. For baseline models that require scale-invariant comparisons, the features were standardised to have a zero mean and a unit variance scale using Scikit-learn’s StandardScaler (version 1.7.2). This preprocessing step is critical for: (i) enabling gradient-based optimisation in MLR, (ii) ensuring proper kernel weighting in SVR, and (iii) maintaining meaningful distance metrics in k-NN. In contrast, tree-based ensemble methods (Random Forest, Extra Trees, and XGBoost), which are generally insensitive to input feature scale, were trained on the raw feature values. The differential preprocessing approach was adopted to respect the mathematical assumptions of each algorithm family, as detailed below.

Baseline Models

Multiple Linear Regression (MLR)

Multiple linear regression (MLR) extends simple linear regression to model a continuous dependent variable and multiple independent variables. As a classical statistical technique widely used for predictive modelling, MLR models serve as foundational tools for supervised machine learning. MLR uniquely assumes no multicollinearity among independent variables in addition to the four key assumptions shared with simple linear regression models: linearity, independence of observations, homoscedasticity (constant variance of errors) and normally distributed residuals [24]. Although MLR is simple, it remains valuable for baseline studies and rapid prototyping. However, the technique performs poorly on non-linear relationships in real-world data. The general mathematical expression for the response variable, y and the independent variables is represented by:

$$y={\beta }_0+{\beta }_1{x}_n+\cdots +{\beta }_n{x}_n+ \epsilon$$

(2)

where ${\beta }_0$ is the intercept, ${\beta }_i$ denotes a coefficient that quantifies the change in response to a unit change in the independent variables, and $\epsilon$is the error term.

Support Vector Regression (SVR)

Support Vector Regression (SVR) is an adaptation of Support Vector Machines (SVMs), a robust class of supervised machine learning algorithms for handling both classification tasks and regression tasks. Unlike traditional regression methods that minimise error, SVR finds a function that deviates from the actual observed target within a specified margin (ε), while maintaining simplicity (flatness) [25].

In solving non-linear problems, the kernel functions (support vectors) map the data into a higher-dimensional space, where it becomes easier to fit an optimal hyperplane that captures the relationship between features and targets. The relationship between the predictors and predicted variables for an SVR can be mathematically expressed as

$$f\left(x\right)=\sum _{i=1}^n({\alpha }_i -{\alpha }_i^{*})K\left(x, x_1\right)+b$$

(3)

where ${\alpha }_i, {\alpha }_i^{*}$ are Lagrange multipliers found during training, $K\left(x, x_1\right)$ is the kernel function user-defined, and $b$ is the bias term [26]. The most popular kernels are the Radial Basis Function (RBF), polynomial, and sigmoid kernels; each is suited for various data complexities and structures as commonly implemented in scikit-learn.

K-Nearest Neighbour (KNN)

K-Nearest Neighbour (KNN) is a classical, non-parametric regression machine learning algorithm used for both regression and classification tasks. According to Györfi, Kohler [27], KNN is preferable for low-dimensional feature variables, as high-dimensional problems often lead to the curse of dimensionality. KNN predicts the target value for a query point $x$, by averaging the responses of its $k$ nearest neighbours in the training set, based on a chosen distance metric (such as Euclidean or Manhattan).

The mathematical formulation of the KNN regression for a predicted value is given by:

$$\widehat{y}\left(x\right)={\displaystyle \frac{1}{K}} \sum _{i \in {\mathcal{N}}_K\left(x\right)}{y}_i$$

(4)

where $\widehat{y}\left(x\right)$ is the predicted value for the input of $x$, $K$ is the number of nearest neighbours, ${\mathcal{N}}_K\left(x\right)$is the set of indices corresponding to the k nearest training samples.

Ensemble Models

Random Forest (RF)

The Random Forest algorithm’s principle is to construct multiple decision tree models during training and output the average prediction of the regressors. This principle uses “divide and conquer”, resampling, aggregation, and random search of the feature space. The strengths of random forest models include handling missing data using the proximity matrix, which measures the proximity between pairs of observations in the forest to estimate missing values [28]. The Random Forest prediction is the average of T decision tree outputs as expressed in Equation (5) below:

$$\widehat{y}\left(x\right)={\displaystyle \frac{1}{T}}\sum _{t=1}^T{h}_t\left(x\right)$$

(5)

where T is the number of trees, and $h_t\left(x\right)$ is the prediction from the t-(th) tree. Key hyperparameters that influence the behaviour of the multiple models are the number of trees (n_estimators), which determines the ensemble size, and the maximum depth of the trees (max_depth), which controls complexity to mitigate overfitting. Other parameters include the minimum samples required to split a node (min_samples_split) and sampling with replacement (bootstrap), which introduces randomness into the model. The residuals used to evaluate the prediction errors are defined in Equation (6) below:

$$r_i=y_i-\widehat{y}\left(x_i\right)$$

(6)

where $r_i$ is the residual for the i-th observation, $y_i$ is the actual observed value and $\widehat{y}\left(x_i\right)$is the predicted value from the model for the input $x_i$.

Extra Trees (ET)

Extra Trees (ET) regressor or extremely randomised trees, is an ensemble learning algorithm that aggregates the predictions of randomised decision trees to improve regression performance. Developed by [29], the ET algorithm shares many similarities with Random Forest algorithm but distinguishes itself with two primary mechanisms; (i) ET splits points for features completely at random, and (ii) it typically utilises the entire training sample, without bootstrapping. This increased randomisation serves to significantly reduce variance in aggregated predictions compared to Random Forest, which relies on both bootstrapping and random feature selection. As a result, ET can often achieve comparable or superior predictive performance with higher computation efficiency [30].

The mathematical formulation for the ET regressor’s prediction value is expressed in Equation (7) below.

$$\widehat{y}(x)={\displaystyle \frac{1}{T}}\sum _{t=1}^T{\widehat{y}}_t\left(x\right)$$

(7)

where (T) is the number of trees, and $\widehat{y}\left(x\right)$ is the prediction of the t-th tree in the ensembles.

Extreme Gradient Boosting (XGBoost)

The Extreme Gradient Boosting (XGBoost) algorithm is designed for high performance on structured data, supporting parallel tree boosting and regularisation [31]. Unlike RF’s independent trees, XGBoost builds trees sequentially, using previous residuals to correct the next prediction and completing tasks faster than the RF models. Equation (8) below computes the prediction value of XGBoost.

$$\widehat{y}(x)=\sum _{t=1}^T{f}_t\left(x\right)$$

(8)

where T is the number of trees, and $f_t\left(x\right)$ is the contribution of the (t)-th tree, iteratively fitted to residuals of prior predictions. Its design is based on these core parameters: the number of boosting rounds (n_estimators), the maximum depth of trees (max_depth), and shrinkage (learning rate), which scales each tree to prevent overfitting. The residuals are similarly computed with Equation (6).

2.5. Model Training, Cross Validation and Hyperparameter Tuning

We utilised k-fold cross-validation, a widely recognised machine learning technique, to assess the performance of our models and fine-tune the hyperparameters associated with the predicted outputs. Specifically, 5-fold cross-validation (k = 5) was implemented using Scikit-learn’s GridSearchCV (version 1.7.2), which explores combinations of specified hyperparameter values. This approach partitions the training data into 5 equal subsets and iteratively trains the models on 4 folds while the validation is performed on the remaining fold. Finally, the performance metrics are computed as an aggregate across the 5 folds. The hyperparameter search space used for tuning KNN, RF and ET is summarised below in

Table 1. Hyperparameter search space for model prediction.

Model Type	Hyperparameters
K-Nearest Neighbour	‘n_neighbors’: [3, 5, 7, 9], ‘weights’: [‘uniform’, ‘distance’], ‘p’: [1, 2]
Random Forest	‘n_estimators’: [50, 100, 200], ‘max_depth’: [None], ‘min_samples_split’: [2, 5], ‘min_samples_leaf’: [1, 2], ‘bootstrap’: [True, False]
Extra Trees	‘n_estimators’: [50, 100, 200], ‘max_depth’: [None], ‘min_samples_split’: [2, 5], ‘min_samples_leaf’: [1, 2], ‘bootstrap’: [False]

.

Alternative search strategies, including Random Search and HalvingGridSearch, were evaluated over the same parameter space. However, these approaches did not improve performance and computational efficiency compared to Grid Search. Hence, they were not used in the final model selection.

2.6. Model Evaluation

The performance of the models was assessed using Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and R-squared (R²) values, as detailed in Equations (9)–(11). Lower RMSE and MAE values indicate better model performance, with these metrics indicating the model’s predictive accuracy in terms of error. MAE, expressed in the same units as the target variable, quantifies the average deviation of the predictions; for example, in this case, MAE informs us of the prediction error in Nephelometric Turbidity Units (NTU). For R², values closer to 1 suggest a model that better explains the variability in the data. In our context, models with an R² value greater than 0.7 are considered adequate, reflecting the specific requirements and standards of our study.

$$\mathrm{M}\mathrm{A}\mathrm{E}={\displaystyle \frac{1}{n}}\sum _{t=1}^n\left|{\chi }_t-{\widehat{\chi }}_t\right|$$

(9)

$$\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}=\sqrt{{\displaystyle \frac{1}{n}}\sum _{t=1}^n{\left({\chi }_t-{\widehat{\chi }}_t\right)}^2}$$

(10)

$${\mathrm{R}}^2=1-{\displaystyle \frac{\sum _{t=1}^n{\left({\chi }_t-\widehat{{\chi }_t}\right)}^2}{\sum _{t=1}^n{\left({\chi }_t-\overline{\chi }\right)}^2}}$$

(11)

where ${\chi }_t$ is the actual value of the output, ${\widehat{\chi }}_t$ is the predicted value of the output, $\overline{\chi }$ is the mean value of the output, and n is the number of data points in the training or testing dataset.

The overall best-performing algorithm among the six algorithms was determined using Alternative Technique for Order of Preference by Similarity to Ideal Solution (A-TOPSIS) [32]. Each algorithm was evaluated under its performance evaluation metrics (R², MAE, RSME) [32,33]. For each criterion, the mean and the standard deviation pair were obtained thereafter from the 6 × 21 decision matrix (6 algorithms, 7 datasets and 3 evaluation metrics). R² was defined as the primary criterion for the algorithm ranking, while MAE and RMSE were treated as secondary.

To ensure objective weighting, the entropy method was applied to normalised decision matrix. The A-TOPSIS algorithm was applied. The criteria were normalised and weighted with the vector (1, −1, −1) to give full importance to R² while penalising larger error values to compute the ideal and anti-ideal solutions, Euclidean distances to these solutions were calculated, and closeness coefficients were derived for the ranking of the six algorithms [34].

2.7. TreeSHAP for ML Model Explainability

We employed TreeSHAP, an efficient variant of the SHAP method, to interpret how operational features influence filtered water turbidity in our ML models. The SHAP analysis represents an innovative approach in the field of Explainable Artificial Intelligence (xAI), to make the predictions of ML models more transparent to humans based on game theory [35]. TreeSHAP provides both local explanations (feature contributions for individual predictions) and global insights (aggregate feature importance across the dataset). The TreeSHAP value output prediction for one prediction (x) is composed as:

$$f\left(x\right)= {\varphi }_{0 }\left(f\right)+ \sum _{i=1}^M{\varphi }_i\left(f,x\right)$$

(12)

where $f\left(x\right)$ is the output of the prediction for the input $x$, ${\varphi }_{0 }\left(f\right)$is the baseline value over the training dataset, and M is the number of input features. The term ${\varphi }_{i }\left(f,x\right)$ is the SHAP value for the feature $i,$ representing the contribution to the i-th instance of $x$.

The global interpretation of the SHAP values over the dataset aggregates the absolute SHAP values across all instances as:

$$Importance\left(i\right)= {\displaystyle \frac{1}{N}}\sum _{j=1}^N\left|{\varphi }_{i }\left(f,x^{\left(j\right)}\right)\right|$$

(13)

where N is the number of samples and ${\varphi }_{i }\left(f,x^{\left(j\right)}\right)$ is the SHAP value for the feature $i$ for instance $j$ [35,36].

The TreeSHAP explainer is a state-of-the-art explainable AI method that quantifies the contribution of each feature to the model’s prediction of the filtered water turbidity in the DWTP.

3. Results and Discussion

3.1. Feature Characteristics and Correlations

Table 2. Summary data for Filters 1–7.

Parameter	Unit	Min	Max	Mean	Std.
Filtered Water Flow Rate	L/s	0	121.97	46.43	9.65
Bed Level	m	3.56	5.58	4.40	0.26
Filtered Water Turbidity	NTU	0.01	10.0	0.13	0.12
Valve opening	%	0.00	100.0	69.4	20.9
Filter Runtime	min	1	10,283	4806	2904

below presents the summary descriptive statistics of the SCADA data from the Ålesund Drinking Water Treatment Plant, which were employed in the ML models. The filtered water flow rate ranged from 0 to 121.9 L/s, averaging 46.4 L/s over the study period. The filter bed level remained stable and only increased up to an additional metre during backwashing when the filter bed is fluidized. The turbidity (NTU) of the filtered water was similarly stable across the seven filters. However, high values were observed in some unique instances. The valve opening (%) and filter runtime (min) exhibited significant variability, with minimum values indicating when the valve is shut and the beginning of a filter cycle.

Figure 3 shows the Pearson correlation coefficients among the operational parameters and filtered water turbidity for Filters 1–7. A consistent positive correlation was observed between filter flow rate and bed level across all filters. This relationship was stronger between bed levels and filter flow rate was observed in Filters 1–3 compared to Filters 4–7, which may indicate flow maldistribution in these filters as a result of the influent lateral manifold [37].

Negative correlations were found between filter runtime and the filtered water turbidity, as well as flow rate and bed level for all the filters. Additionally, filters with stronger correlation between flow rate and runtime also tend to exhibit stronger correlations between runtime and bed level, suggesting interdependent operational behaviour.

Regarding filtered water turbidity, Filters 1–3 showed positive correlation with both flow rate and bed level. In contrast, Filters 4–7 exhibited a negative correlation between these parameters. Overall, there was a negative correlation between flow rate and filtered water across all filters. Notably, the magnitude of the correlation between flow rate and filtered water turbidity in Filters 4–7 is comparable to that between runtime and bed level observed in Filters 1–3.

Effect of Filters on Turbidity

The individual filter turbidity is summarised in Figure 4 below and compared with the combined filtered water turbidity expressed as flow-weighted turbidity from Equation (1). The weighted filtered water turbidity ranged from 0.07 to 0.18, remaining well below the United States Environmental Agency (USEPA) provisions for combined filtered water [8,38]. Notably, Filters 1–3 exhibited mean filtered water turbidity slightly above the weighted average, whereas Filters 4–7 demonstrated lower individual turbidity values relative to the combined filtered water turbidity. Among all the units, Filter 7 consistently produced the lowest turbidity values, indicating the most effective performance over the study period. Figure 4 below presents a box plot showing the distribution of the turbidity in the filtered water.

This indicates that, at any point, the turbidity of the combined filtered water is reduced by the contributions of the other filters, which maintain a low turbidity, referred to here as the weighted turbidity.

Filter Flow Rate and Turbidity

The patterns of the diurnal flow rate (a) and filtered water turbidity (b) are illustrated for Filter 1 in Figure 5 below. Typically, the flow rate at midnight is below the median line (45.6 L/s), along the decline from the evening maximum flow (~51 L/s) until it dips to its lowest (~41 L/s) at 4:00. The morning flow momentarily peaks (~49 L/s) at 6:00 and declines till 8:00. The flow rate increases steadily and is stabilised between 16:00 and 18:00 before reaching the maximum flow rate at 19:00.

In Figure 5b, the filtered water turbidity curve starts at midnight slightly above the median (~0.15 NTU), increasing sharply to the first peak (>0.20 NTU) between 6:00 and 7:00, followed by an abrupt decline (~0.16 NTU) and minimal fluctuations until the next backwashing day. This pattern remains consistent across all filters due to the weekly backwashing routine process at 6:00, causing the observed spike in filtered water turbidity. During backwashing, the filter bed is fluidised, and particles are temporarily dislodged [39], causing an increase in the filtered water turbidity at the beginning of the new filtration cycle. The subsequent turbidity reduction from 0.21 NTU to 0.16 NTU (above the conventional 0.10 NTU threshold [8]) indicates the filter’s recovery after backwashing. This is primarily due to a decrease in the overall pore volume of the bed, which increases particle capture by interception and the contribution of other particle capture mechanisms in filters [40,41]. Additionally, the shape of the media also influences particle capture; therefore, media sharpness and the condition of the filter media, as well as replenishment, can significantly affect turbidity removal [39,42]. The second peak in flow rate at 19:00, however, corresponds to very minimal fluctuations in turbidity, indicating that turbidity is indirectly impacted by flow rate. The shaded regions in the respective figures (left and right) highlight the period of the highest increase in both flow rates and filtered water turbidity. This occurrence is due to the filter’s backwashing routine; the full effect is captured in Figure 6 showing a typical seven-day runtime for Filter 1.

3.2. Model Performance Evaluation for Filtered Water Turbidity Prediction

Six machine learning algorithms were evaluated to assess the effect of operational parameters and filtered water turbidity for the seven filters. For this, the performance using R², MAE and RMSE of the filters after the baseline (MLR, SVR, KNN) and the ensembles (XGBoost, RF and ET) algorithms. The results for the assessment are presented in

Table 3. (a,b) Performance comparison of baseline models and ensemble models.

(a) Baseline Models
	R2			MAE			RMSE
Filter	MLR	SVR	KNN	MLR	SVR	KNN	MLR	SVR	KNN
Filter 1	0.010	0.182	0.61	0.033	0.046	0.024	0.112	0.102	0.069
Filter 2	0.013	0.228	0.79	0.039	0.047	0.025	0.163	0.144	0.081
Filter 3	0.032	0.163	0.70	0.040	0.053	0.025	0.142	0.133	0.073
Filter 4	0.004	0.185	0.63	0.035	0.043	0.028	0.102	0.092	0.061
Filter 5	0.011	0.141	0.56	0.010	0.036	0.027	0.109	0.102	0.076
Filter 6	0.017	0.470	0.82	0.034	0.045	0.022	0.122	0.089	0.054
Filter 7	0.011	0.196	0.81	0.032	0.045	0.019	0.101	0.091	0.047
(b) Ensemble Models
	R2			MAE			RMSE
Filter	XGBoost	RF	ET	XGBoost	RF	ET	XGBoost	RF	ET
Filter 1	0.61	0.81	0.86	0.025	0.017	0.016	0.071	0.048	0.016
Filter 2	0.65	0.85	0.90	0.027	0.018	0.017	0.098	0.062	0.017
Filter 3	0.66	0.89	0.92	0.025	0.016	0.016	0.084	0.048	0.016
Filter 4	0.35	0.79	0.80	0.029	0.019	0.019	0.082	0.047	0.019
Filter 5	0.49	0.79	0.82	0.029	0.018	0.017	0.078	0.052	0.017
Filter 6	0.74	0.89	0.91	0.023	0.016	0.016	0.063	0.040	0.016
Filter 7	0.75	0.87	0.87	0.021	0.014	0.014	0.051	0.037	0.014

a,b below.

Table 3a,b shows a comprehensive comparison baseline models (MLR, SVR, KNN) and ensemble models (XGBoost, RF, ET) for predicting turbidity across seven different filter datasets, using R², MAE, and RMSE as performance metrics. Best performing models for each filter in bold.

The results from Table 3 shows that the ensemble models consistently outperform baseline models across all filters and evaluation criteria due to their robustness in non-linear datasets [14,29,31]. Among the ensemble methods, Extra Trees (ET) is distinguished with the highest R² scores and the lowest MAE and RMSE values, indicating its superior predictive performance. For instance, for Filter 3, ET achieves R² = 0.92, MAE = 0.016 and RMSE = 0.016, which is notably better than the best performing baseline model—KNN; R² = 0.70, MAE = 0.025, and RMSE = 0.073.

The baseline models present a mixed performance. Both MLR and SVR consistently yield low R² values (all ≤ 0.23), and higher MAE and RMSE, indicating poor model fit and limited suitability for this regression task. KNN performs moderately well, achieving its best results on Filter 6 (R² = 0.82, MAE = 0.022, RMSE = 0.054) and Filter 7 (R² = 0.81, MAE = 0.019, RMSE = 0.047), comparable to the ensemble model’s performance. This observation is consistent with established literature, which indicates that non-parametric methods, such as KNN, generally perform better in low-dimensional spaces, although they struggle with more features or noise [43].

The ensemble methods, particularly ET and RF showed robustness and reliable performance across all seven filters. RF and ET maintained R² values consistently above 0.79, with ET achieving the highest R² score in four out of seven filters. For example, ET achieves R² = 0.91 and MAE = 0.016 on Filter 6, compared to KNN’s R² = 0.82, and MAE = 0.022. XGBoost also performs well, although it scores lower than RF and ET on some filters, emphasising its effectiveness in modelling the data’s underlying complexity [29]. The ensemble models, particularly RF and ET, provide superior and more consistent predictive performance for complex, non-linear relationships within this dataset [29,30].

The results demonstrate that ET and RF ensemble models offer superior and more consistent predictive performance for filter turbidity modelling compared to baseline approaches. These findings underscore the value of ensemble learning in modelling complex environmental datasets and support their adoption for robust turbidity prediction in water treatment applications.

Table 4. A-TOPSIS ranking of the algorithms.

Algorithm	Ratio	Total	Ranking
ET	0.29 ± 0.25	1.00	1
RF	0.24 ± 0.23	0.83	2
KNN	0.20 ± 0.16	0.64	3
XGBoost	0.18 ± 0.14	0.53	4
SVR	0.06 ± 0.11	0.10	5
MLR	0.03 ± 0.11	0.01	6

presents the A-TOPSIS (an approach based on the Alternative Technique for Order Preference by Similarity to Ideal Solution) ranking results for all the machine learning algorithms evaluated. The A-TOPSIS ranking reflects the overall performance, with lower ranks indicating better performance.

The A-TOPSIS ranking in Table 4 confirms that the ensemble models, ET and RF are the best in ranking, first and second, respectively, with their total scores (ET =1.00, RF = 0.83). SVR and MLR are the lowest ranking, indicating poorer overall performance, while KNN ranks third, outperforming XGBoost in the regression task.

Table 5. Overall performance of the top three models.

Metric	Extra Trees	Random Forest	KNN
Accuracy	High	High	Moderate
Stability	Consistent	Consistent	Filter-dependent
MAE	~0.014–0.018	~0.014–0.018	~0.020–0.030
RMSE	Lowest (~0.036–0.045)	Low (~0.04–0.05)	Higher (~0.07)
Training	Medium (~620 s)	Slow (~2100–2500 s)	Fast (~13 s)
Prediction	Medium (~1 s)	Medium (~0.4 s)	Fast

presents an aggregate comparison of the key performance characteristics of the three best-performing models: KNN, Extra Trees (ET) and Random Forest (RF).

Extra Trees and Random Forest demonstrate high predictive accuracy, consistently outperforming KNN, which gives moderate accuracy on the datasets. By comparing the computational performance of the algorithms on the datasets, KNN was the fastest model to train (~13 s), making it ideal for real-time applications with limited computational resources. However, this speed comes at the expense of lower accuracy and stability, since KNNs are known to be lazy learners [27]. ET achieves a balance with moderate training time (~620 s) and prediction time (~1 s) while maintaining the highest accuracy. Although RF delivers high accuracy, it is the slowest to train (~2100–2500 s), which may be considered for large-scale or time-sensitive applications. ET models, besides their accuracy, are faster because the algorithm is based on random splits, in contrast with RF models which find the optimal split by looking at all possible nodes [30].

3.3. Model Interpretation and Explainability

The TreeSHAP analysis discussed here only pertains to the ET models for the filters, as they demonstrated the best performance across all three metrics used to evaluate the models’ performance. Figure 7 below shows the global interpretations based on the average impact of the features on the model’s output. The left column bar plots, representing feature importance, display the mean absolute SHAP value for each feature in the ET models’ output for turbidity prediction, ranked by their importance. The right column scatter plots (summary plot) illustrate the individual predictions. The colour of the dots indicates the feature value, ranging from low (blue) to high (red) [44]. Positive SHAP values indicate a positive effect on the predicted variable and vice versa. Long tails for a feature suggest that a specific feature significantly impacts the model output [45].

From the bar plots, the generalised parameter trend is runtime > flow rate > bed level for all the ET filter models, except for Filter 6, where the bed levels are second to the filter runtime in feature importance. However, each ET model exhibits slight variations in the magnitude of their respective SHAP values and the corresponding impact, as demonstrated in the scatter plots. The effects of the features on the filters are discussed below.

The significance of feature importance fluctuates depending on the filter’s position, from 1 to 7. The most apparent differences among the three features were observed in Filter 3 (the highest R² = 0.92). The summary plot in the right column illustrates that (i) lower runtimes positively influence the filtered water turbidity values; (ii) high flow rates also increase the filtered water turbidity, although only to half the degree of lower runtimes; and (iii) the bed water levels affect the filter’s performance in either direction, potentially increasing or decreasing the filtered water turbidity. On the contrary, the ET model for filter 4 is the worst-performing filter (R² = 0.80) with average SHAP impact values of 0.010, approximately half that of Filter 3 (0.020) and 0.004 less than the other filters in the gallery, thus showing the weakest contribution to the filter. At the same time, the Flow and level parameters for the Filter 4 depict outlier situations as shown in the corresponding scatter plot [35].

Generally, increasing the runtime of the filters reduced the turbidity values and vice versa. Higher flow rates in Filter 6 have increased the predicted turbidity values. The bed level of Filter 3 has a negligible effect on the turbidity prediction. This implies a highly stable bed level throughout most filtration cycles. Therefore, while runtime is the dominant factor for effluent turbidity in Filter 3, increasing the flow rate impacts this trend, while the influence of bed level is negligible. In contrast with the other six filters, Filter 3 exhibits uniqueness in the trend of flow rates. The summary plot shows that the low flow rates in Filter 3 are as critical as the filter runtime in predicting turbidity.

The feature importance for all seven ET models of the filters indicates that the runtime feature is the most influential parameter, contributing to low filtered water turbidity in the mono-media filters.

For a local explanation of the factors, the predicted value, denoted in the force plot as f(x), is the contribution of the individual factors as arrows to the base value (expected output) of the entire dataset. The force plots for filters 1 to 7 (F1–F7) are shown in Figure 8 below. The force plots for the ET models regarding filters of the input features (runtime, flow rate, bed level) explain the predicted filtered water turbidity at a local level.

The runtime has a significant influence on the final prediction and is least affected by the bed level in the filter. For instance, using Filter 3 again for the explanation, the average prediction for the filtered water turbidity has a baseline ≈ 0.12 NTU, the runtime feature forces the predictions to the left by 0.01 NTU, and the filtered water flow rate further lowers the prediction in the same direction to 0.10 NTU by approximately similar units. The bed level makes a small push to the right to give the final prediction, and f(x) is 0.10 NTU.

By comparing the forced plots, we observe certain uniqueness in all seven filters. Filters 1, 2, and 3, have predicted filtered water turbidity values slightly higher than 0.10 NTU, whereas Filters 4, 5, 6 and 7 have predicted values ≤ 0.01 NTU.

At lower flow rates, the SHAP values for runtime are high, indicating that the filtration rate is crucial for the direct filters. An increased filtration rate may hinder effective particle removal, resulting in higher predicted filtered water turbidity [46]. As the flow rate increases, the SHAP values for runtime decrease, suggesting that the impact of runtime on turbidity prediction lessens at higher flow rates. This could be attributed to the fact that, at higher flow rates, particles are not effectively captured by the media [41]. This phenomenon relates to the properties of the filter media, such as diameter or material, which remain unaddressed in the machine learning models. Elevated turbidity in filtered water increases the risk of virus shielding and ineffective removal in the disinfection stage with UV light [47]. The elevated turbidity after backwashing can be improved by operating the filters based on predicted turbidity levels.

3.4. Implication of Predicted Filter Performance on Operation Routines

The WTP’s filtration operations encompass essential routines, such as backwashing and planned maintenance. The diurnal pattern of the Ålesund DWTP indicates that the plant has higher production during the evening compared to the mornings. This pattern occurs because the flow rate is reduced during backwashing routines, which involve isolating one of the seven filters. Consequently, there is a temporary decrease in the flow rate, which impacts the filtered water turbidity from the newly backwashed filter due to the disturbance during the backwashing period, as backwashing does not eliminate trapped particles completely. To address increased turbidity after backwashing, the start of the backwash filter is delayed for a few minutes before initiating the next filtration cycle. Although the SCADA data indicate that the system’s operation has experienced some flow reduction compared to the diurnal flow pattern, the corresponding flow data show that the flow is not significantly diminished. Furthermore, the SHAP interpretation for the best models in the case study indicates that the operational runtime for the filters can be extended by over seven days under the current operation of the plant.

Theoretically, the Length-to-Depth (l/d) ratio for this filter assembly is significantly lower than that of most rapid sand filters described in the literature. This emphasises the filters’ main purpose: pH correction, while enabling us to increase the l/d ratio. Reducing the effective diameter of the current filter media should be prioritised to enhance particle capture. Furthermore, the ultimate benefits would involve maintaining the upstream raw water turbidity < 5 NTU, as recommended for direct filtration setups. Nevertheless, the WTP’s downstream interventions, such as mixing the effluent from the seven filters and UV, are sufficient to meet today’s demands.

In applications, this study is applicable for developing soft sensors for each filter in DWTPs as either the main or a backup [16]. These soft sensors can be built as a stand-alone and updatable online platform to leverage data from the DWTP SCADA system. This online platform will provide early warnings and recommended actions to plant operators on the impact of specific operational conditions on filtered water turbidity levels.

4. Conclusions

The study employs three baseline and three ensemble machine learning models to investigate the prediction of filtered water turbidity in mono-bed filters at a drinking water treatment plant. The models were built using bed level, flow rate, and filter runtime as the primary parameters affecting the filtered water turbidity for a seven-unit filter bed. The filters generally behave similarly to the ripening stages of rapid media filters, typically expanding following backwashing. This resulted in increased turbidity for the unit filters immediately after the backwashing routine. The results from the Extra Trees (ET) ensembles achieved higher R² values for all the filters (0.92) and the lowest RMSE (0.016 NTU), demonstrating their capacity to model complex relationships, although with slower computation times, compared to XGBoost models. The best-performing baseline model was KNN (R² = 0.82 and RMSE = 0.054), ranking third after ET and RF models by the A-TOPSIS method. The SHAP feature analysis using the ET models highlighted significant operational parameters influencing filtered water turbidity, including filter runtime and filtered flow rate.

The predictive model developed in this study can help optimise mono-bed filter operations by providing early warnings of high turbidity levels. For instance, integrating AI in digital monitoring systems could enhance decision-making, reduce operational costs, and improve water treatment efficiency. It also fosters innovation in the management and optimisation of drinking water systems.

The approach presented offers a practical trade-off between accuracy and data availability in operational environments where long-term data collection may not be feasible. Since the models are trained on four months of operational data, their generalisability over a more extended period may be limited. However, they remain useful for seasonal use, provided they are regularly updated. Moreover, the study’s assumptions simplify the filtration process to the flow rate of the filtered water, bed level, and runtime of the filters, which may overlook critical physical complexities of filter dynamics. These include the length-to-depth ratio of the filters, flow regime within the filters, influent manifold correction factors, filter media roughness factors, and raw water quality. These complexities can be addressed by deploying hybrid models that incorporate fundamental filtration physics. For instance, the Tufenkji-Elimelech equation can be used to model particle capture and transport [41], and the Kozeny-Carman equation can account for operational headloss, bed level and filter characteristics. By using the outputs of these equations as augmented features in the model, the predictive performance of the data-driven methods can be substantially enhanced. Future studies will be evaluated using online SCADA data in real-time operations to assess their robustness and scalability while accounting for the physics of filtration.