Machine Learning Models for Water Quality Prediction: A Comprehensive Analysis and Uncertainty Assessment in Mirpurkhas, Sindh, Pakistan

Abstract

Groundwater represents a pivotal asset in conserving natural water reservoirs for potable consumption, irrigation, and diverse industrial uses. Nevertheless, human activities intertwined with industry and agriculture contribute significantly to groundwater contamination, highlighting the critical necessity of appraising water quality for safe drinking and effective irrigation. This research primarily focused on employing the Water Quality Index (WQI) to gauge water’s appropriateness for these purposes. However, the generation of an accurate WQI can prove time-intensive owing to potential errors in sub-index calculations. In response to this challenge, an artificial intelligence (AI) forecasting model was devised, aiming to streamline the process while mitigating errors. The study collected 422 data samples from Mirpurkash, a city nestled in the province of Sindh, for a comprehensive exploration of the region’s WQI attributes. Furthermore, the study probed into unraveling the interdependencies amidst variables in the physiochemical analysis of water. Diverse machine learning classifiers were employed for WQI prediction, with findings revealing that Random Forest and Gradient Boosting lead with 95% and 96% accuracy, followed closely by SVM at 92%. KNN exhibits an accuracy rate of 84%, and Decision Trees achieve 77%. Traditional water quality assessment methods are time-consuming and error-prone; a transformative approach using artificial intelligence and machine learning addresses these limitations. In addition to WQI prediction, the study conducted an uncertainty analysis of the models using the R-factor, providing insights into the reliability and consistency of predictions. This dual approach, combining accurate WQI prediction with uncertainty assessment, contributes to a more comprehensive understanding of water quality in Mirpurkash and enhances the reliability of decision-making processes related to groundwater utilization.

1. Introduction

Water, as an indispensable resource, plays a fundamental role in sustaining life and supporting various human activities. Among its many sources, groundwater stands as a crucial reservoir essential for drinking, agriculture, and industrial processes in Pakistan. However, the escalating impact of human interventions, particularly in industrial and agricultural sectors, poses a substantial threat to the quality of this invaluable resource [1,2,3,4]. The condition of groundwater in Pakistan, including areas like Mirpurkhas in the province of Sindh, has faced mounting challenges due to extensive usage, urbanization, and agricultural runoff, leading to contamination concerns and a decline in overall quality. The region’s reliance on groundwater for daily consumption and agricultural needs amplifies the urgency for effective water quality assessment measures [2,5,6,7]. Contamination of groundwater due to these anthropogenic activities has heightened concerns regarding its suitability for consumption and irrigation purposes, necessitating robust methods for accurate evaluation and monitoring [8,9,10,11].

Traditionally, assessing water quality, especially the determination of the Water Quality Index (WQI), relied heavily on manual calculations and established formulas based on a set of parameters [9,12,13,14,15]. These methods often entail time-consuming processes and are prone to human errors, particularly in complex calculations involving multiple interdependent factors [16,17,18]. In recent years, the integration of artificial intelligence and machine learning techniques, implemented using programming languages like Python (version 3.10), alongside specialized libraries such as scikit-learn (version 0.24), XGBoost (version 1.5), and pandas (version 1.3), has emerged as a transformative approach to overcome the limitations of traditional methods. Machine learning models, such as Random Forest, Gradient Boosting, Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Decision Trees, were developed and trained using these tools. They offer the advantage of learning patterns and relationships from vast datasets, enabling more accurate and efficient prediction of WQI [19,20,21].

This study, conducted in the geographic area of Mirpurkhas in Sindh, collected an extensive dataset of 422 samples to comprehensively understand the region’s water quality characteristics. Leveraging Python and various machine learning libraries such as scikit-learn, XGBoost, and pandas, the research employed these tools to preprocess data, build, train, and evaluate machine learning classifiers for predicting WQI [22,23]. The results indicate that Random Forest and Gradient Boosting outperformed other algorithms, achieving an exceptional accuracy rate of 99%. Following closely were SVM and XGBoost, scoring approximately 95% and 93% accuracy, respectively, while KNN and Decision Trees demonstrated accuracy rates of 88% and 87%, respectively. These findings underscore the efficacy of Python-based machine learning techniques implemented with specialized libraries in accurately predicting WQI, showcasing their potential for advancing water quality assessment methods, particularly in groundwater evaluation [24,25,26]. In a similar vein, Reza Mohammadpour’s article [27] employed Support Vector Machine (SVM) and two artificial neural network (ANN) methods, feed forward back propagation (FFBP) and radial basis function (RBF), for Water Quality Index (WQI) prediction in a constructed wetland. The SVM model outperformed, achieving a high coefficient of correlation (R²) of 0.9984 and a low mean absolute error (MAE) of 0.0052, demonstrating its effectiveness in streamlining WQI calculations and optimizing computational efforts in free surface constructed wetland environments. Afaq Juna’s article [28] supports our experimental findings, where Random Forest (RF) and XGBoost both attain an 80% accuracy for Water Quality Index (WQI). RF demonstrates precision, recall, and an F1 score of 80%, while XGBoost achieves 80% precision and recall, with an F1 score slightly lower at 79%. In contrast, KNN and SGDC exhibit the lowest WQI accuracy at 59%. Mehedi Hassan’s article [19] in WQI prediction demonstrates outstanding accuracy, with Kappa, Accuracy Lower, and Accuracy Upper scores reaching 99.83, 99.17, and 99.07, respectively. These results underscore the crucial role of machine learning in precisely categorizing water quality, highlighting its significance for effective water management and corroborating our high accuracy in machine learning models for Water Quality Index prediction.

Through an interdisciplinary approach integrating environmental science and machine learning, this research aims to contribute to the advancement of accurate and efficient water quality assessment methods, utilizing the potential of artificial intelligence and predictive modeling implemented through Python-based tools and specialized libraries. Beyond WQI prediction, this study integrates an uncertainty analysis using the R-factor, providing a nuanced perspective on the reliability and consistency of our predictive models. The combined approach of accurate WQI prediction and uncertainty assessment contributes to a more holistic understanding of water quality dynamics in Mirpurkash. Ultimately, this research aims to inform robust decision-making processes regarding groundwater utilization, considering both the accuracy of predictions and the inherent uncertainties associated with them. The integration of artificial intelligence (AI) forecasting models, specifically machine learning classifiers, such as Random Forest, Gradient Boosting, SVM, XGBoost, KNN, and Decision Trees, has proven to be instrumental in predicting the Water Quality Index (WQI) with remarkable accuracy [19,29,30,31,32]. However, the accuracy of predictions alone does not provide a complete picture, and understanding the structure of these models is essential for a comprehensive assessment of uncertainty.

This research paper is structured to encompass several key sections. Beginning with an Introduction that highlights the significance of groundwater, particularly in the context of Mirpurkhas in Sindh, it emphasizes the challenges of water quality and the need for advanced assessment methods, summarizing previous studies on groundwater quality, traditional Water Quality Index (WQI) determination methods, their limitations, and existing research on applying machine learning in water quality assessment. The Section 2.2 outlines the steps undertaken, including data collection of 422 samples, data preprocessing, feature selection, and the utilization of machine learning algorithms such as Random Forest, Gradient Boosting, SVM, XGBoost, KNN, and Decision Trees and evaluation of uncertainty in the above machine learning algorithms. The subsequent Section 3 presents the performance metrics of these models in predicting WQI accuracy rates. Following this, the Discussion interprets the outcomes, compares model performances, addresses limitations, and suggests further research avenues. Finally, a Conclusion summarizes the key findings, reinforces the significance of employing machine learning in water quality assessment, and suggests future implications.

2. Materials and Methods

2.1. Study Area

Mirpurkhas, situated in the Sindh province of Pakistan, experiences an arid to semi-arid climate characterized by scorching summers with temperatures often exceeding 40 degrees Celsius (104 degrees Fahrenheit) from April to September Figure 1. Monsoons, occurring between July and September, bring moderate to heavy rainfall, providing relief from the intense heat. Winters are relatively mild, ranging from around 10 to 20 degrees Celsius (50 to 68 degrees Fahrenheit). Geographically, Mirpurkhas is located near the Indus River in the southern part of Pakistan and is renowned for its agricultural activities (Figure 1) [7,33]. Wells play a crucial role in providing groundwater for various purposes, including drinking water supply and agricultural irrigation, supporting the local livelihoods within this semi-arid region.

Mirpurkhas, a town located in the Sindh province of Pakistan, relies heavily on well water for various purposes. The inhabitants of Mirpurkhas primarily utilize well water for drinking, agricultural irrigation, and domestic needs [34,35]. Wells in the region serve as a primary source of groundwater, supplying water to the local community. Well water in Mirpurkhas is crucial for sustaining daily activities and agricultural practices. However, like many areas reliant on groundwater, the water quality in wells can be susceptible to contamination from various sources such as agricultural runoff, industrial activities, and natural factors [36,37,38].

2.2. Methodology

The research methodology involved the collection of 422 water samples from multiple sites across Mirpurkhas, Sindh, Pakistan, during the period from April to May 2022, covering various locations deemed significant for groundwater extraction and consumption. Parameters, including pH levels, temperature, dissolved oxygen, turbidity, nitrates, and other physiochemical characteristics, were measured using standardized water testing procedures and equipment [39,40,41]. The dataset comprises a total of 422 samples, filtered to 0.45 µm for further analysis, with their locations recorded using a global positioning system (GPS). Standard methods outlined by the American Public Health Association [42] were followed for analysis. The well depths vary widely, ranging from 5.7 m to 590 m, indicating a diverse dataset that includes samples from both shallow and deep aquifers. The variation in well depths is essential to consider, as it may influence groundwater characteristics, impacted by geological and hydrological factors associated with different depth ranges. Following data collection, a rigorous preprocessing phase was conducted to ensure data accuracy and suitability for machine learning analysis Figure 2. This stage encompassed handling missing values through imputation methods, outlier removal, and normalization or scaling to ensure uniformity across parameters. Feature engineering was performed to extract pertinent features and reduce dimensionality for enhanced model performance. Feature selection techniques were employed, including Variance Inflation Factor (VIF) and Information Gain (IG), to identify influential parameters affecting water quality. These methods aimed to reduce redundancy and select the most informative features for modeling [43,44].

The evaluation of groundwater suitability for human consumption involved the computation of the Water Quality Index (WQI) based on the standards established by the World Health Organization (WHO). The WQI calculation comprised a three-step procedure. Initially, an individual weight (wi) was assigned to each parameter, encompassing TDS, Sodium, Calcium, Magnesium, Bicarbonate, Sulfate, Chloride, pH, EC, Nitrate (NO³⁻), Well Depth, and Potassium. Subsequently, the relative weight (W_i) for each parameter was determined. Lastly, quality-rating scales (q_i) and sub-indices (${SI}_i$) were computed for each parameter, and the overall WQI was derived by summing the sub-indices. The resulting classification into five groups, ranging from Group 1 (0–25), indicating Excellent water quality, to Group 5 (above 100), signifying Very Poor to Unacceptable water quality, was employed in collaboration with machine learning models for a more comprehensive analysis.

For model development, thePython programming language was utilized, along with machine learning libraries such as scikit-learn, XGBoost, pandas, and numpy. Supervised learning algorithms, including Random Forest, Gradient Boosting, Support Vector Machine (SVM), XGBoost, K-Nearest Neighbors (KNN), and Decision Trees, were implemented and trained using the preprocessed dataset [45,46]. Hyperparameter tuning through techniques like grid search and cross-validation optimized the models. The performance of the developed models was evaluated using common metrics such as accuracy, confusion matrix, Friedman test, and Nemenyi test [47,48]. The uncertainty of model predictions has been evaluated using R-factor and bootstrapping.

The data underwent resampling using a cross-validation technique to assess model robustness and generalizability. The training and testing ratios, as evidenced by our confusion matrices across all classifiers, fall within the range of 20% to 33%. XGB, Random Forest, and SVC are generally regarded as robust and less susceptible to overfitting, enabling them to perform effectively with a smaller testing set (20%). In contrast, KNN, Gradient Boosting, and Decision Tree models may exhibit greater sensitivity to the nuances of the training data, suggesting potential benefits from a larger testing set (30%) for a more thorough evaluation [49]. Results interpretation involved comparing and analyzing the outcomes of various machine learning classifiers to identify the most accurate models for predicting the Water Quality Index (WQI). Models demonstrating the highest accuracy rates were further analyzed to understand the impact of different parameters on WQI prediction and water quality assessment.

The variables that have been used in our research to determine the Water Quality Index are shown in Figure 3.

The VIF analysis

Table 1. Variance Inflation Factor (VIF) values indicating multicollinearity among water quality assessment features in Mirpurkhas, Sindh, Pakistan.

Feature	VIF
TDS (mg/L)	4209.78
Sodium (mg/L)	1137.34
Calcium (mg/L)	425.13
Magnesium (mg/L)	380.55
Bicarbonate (mg/L)	58.74
Sulfate (mg/L)	39.68
Chloride (mg/L)	31.69
pH	20.16
EC (us/cm)	10.20
Nitrate (NO3-N) (mg/L)	5.45
Well Depth (m)	1.70
Potassium (mg/L)	1.43

highlights varying degrees of multicollinearity among the features considered for water quality assessment in Mirpurkhas, Sindh, Pakistan. Notably, certain parameters, such as ‘TDS’, ‘Sodium’, ‘Calcium’, and ‘Magnesium’, exhibited notably high VIF values, indicative of strong multicollinearity among these variables. Conversely, ‘Potassium’, ‘Well Depth’, and ‘Nitrate (NO³⁻)’ demonstrated relatively lower VIF values, suggesting lower levels of multicollinearity in comparison [50].

The Variance Inflation Factor (VIF) values, obtained from the assessment of water quality parameters in Mirpurkhas, Sindh, reveal varying degrees of multicollinearity among features considered for predicting the Water Quality Index (WQI). Features such as ‘TDS’, ‘Sodium’, ‘Calcium’, and ‘Magnesium’ exhibit notably high VIF values, suggesting strong interdependencies among these variables. This significant multicollinearity potentially impacts the accuracy of predictive models developed for water quality assessment [51]. Parameters with lower VIF values, including ‘Potassium’, ‘Well Depth’, and ‘Nitrate (NO³⁻)’, indicate weaker correlations, potentially posing less influence on multicollinearity issues within predictive models. Addressing high multicollinearity, particularly among variables with elevated VIF values, becomes crucial in enhancing the reliability and precision of predictive models for more accurate water quality assessment in the Mirpurkhas region.

Tree-based models (Decision Trees, Random Forest, Gradient Boosting, XGBoost) and K-Nearest Neighbors (KNN) are generally less sensitive to multicollinearity compared to linear models like linear regression or logistic regression. Support Vector Machine (SVM) can be sensitive to multicollinearity to some extent, depending on the kernel used; therefore, we use a linear kernel with SVM [52]. The linear kernel computes the dot product between two observations. It is less sensitive to multicollinearity because it effectively works in the original feature space without introducing non-linear transformations.

Although elevated VIF values may signal multicollinearity and potential challenges in linear models, opting to include all variables based on Information Gain remains a viable strategy, particularly when employing tree-based models such as KNN, RF, Gradient Boosting, XGBoost, and Decision Trees. In addition, to make SVM less sensitive to multicollinearity, we use a linear kernel in our research. Nevertheless, it is crucial to empirically validate this decision by evaluating the model’s performance on independent datasets or employing robust cross-validation techniques.

The Information Gain (IG) analysis

Table 2. Information Gain (IG) values indicating corresponding information gain for each water quality assessment feature in Mirpurkhas, Sindh, Pakistan.

Feature	IG
Nitrate (NO3-N) (mg/L)	0.876
Calcium (mg/L)	0.869
Sodium (mg/L)	0.869
Sulfate (mg/L)	0.869
Chloride (mg/L)	0.869
Potassium (mg/L)	0.869
Magnesium (mg/L)	0.869
TDS (mg/L)	0.816
EC (us/cm)	0.784
Well Depth (m)	0.525
Bicarbonate (mg/L)	0.520
pH	0.509

highlights the relevance of various features in predicting the Water Quality Index (WQI) in Mirpurkhas, Sindh, Pakistan. Features such as ‘Nitrate (NO₃-N)’, ‘Calcium’, ‘Sodium’, ‘Sulfate’, ‘Chloride’, ‘Potassium’, and ‘Magnesium’ exhibit higher IG values, indicating their considerable relevance in predicting WQI. Conversely, ‘pH’, ‘Bicarbonate’, ‘Well Depth’, ‘EC’, and ‘TDS’ present relatively lower IG values, suggesting comparatively lesser impact in predicting the WQI. Understanding the relevance of these features assists in selecting the most influential variables for the development of accurate predictive models for water quality assessment.

However, it is important to note that while IG values help identify influential features, the absolute value of IG alone might not necessarily determine the direct impact or importance of a feature in predicting the WQI [53]. Other factors, such as domain knowledge, the nature of the dataset, and the specific context of the water quality assessment, should also be considered when selecting influential variables for building accurate predictive models. Therefore, while IG values provide valuable insights, the selection of the most influential variables should involve a comprehensive analysis that integrates multiple factors beyond IG values alone.

2.3. Uncertainty Analysis

2.3.1. R-Factor

While various factors contribute to the uncertainty in predicting Water Quality Index (WQI), including modeling, sampling errors, data preparation, and pre-processing, this study specifically addresses the uncertainty linked to individual model structures and input parameter selection. To assess model structure uncertainty, the analysis involves examining a set of three predicted WQI values during the testing phase for each observed WQI. These predictions are generated by the aforementioned predictive models.

The mean and standard deviation are computed for each predicted set, serving as parameters for a designated normal distribution function. Employing the ‘Monte Carlo’ simulation method, 1000 WQI values are generated for each observed value based on this distribution. While other methods like Latin Hypercube [54], Lagged Average [55], and Multimodal Nesting [56] are utilized for sample generation, the Monte Carlo technique has demonstrated greater applicability, especially in hydrology and water-related sciences [57]. To quantify the uncertainty associated with WQI prediction, the 95% prediction confidence interval (i.e., the interval between the 97.5% and 2.5% quantiles), known as the prediction uncertainty of 95% (95PPI), is determined using the generated WQI values for each observed WQI. Specifically, the uncertainty is computed using the defined R-factor (Equation (1)).

The formula for the calculation of the R-factor is expressed as:

$$\mathrm{R-factor}=\frac{s_p}{s_x}$$

(1)

Here, $s_x$ represents the standard deviation of the observed values, and $s_p$ is determined using Equation (2):

$$s_p=\frac{\sum _{i=1}^J(U_{Li}-L_{Li})}{J}$$

(2)

In this equation, J denotes the number of observed data points, while $U_{Li}$ and $L_{Li}$ correspond to the i-th values of the upper quartile (97.5%) and lower quartile (2.5%) of the 95% prediction confidence interval band (95PPI).

Other approaches, such as the Coefficient of Variation (CV), Prediction Interval Coverage Probability (PICP), and Prediction Interval Normalized Root-mean-square Width (PINRW), have been proposed as substitutes for the R-factor method [58]. Nevertheless, these alternative methods solely rely on either observed or predicted data. In contrast, the R-factor method takes into account both observed and predicted data, making it a more comprehensive metric for characterizing prediction uncertainty [59,60]. The inherent uncertainty in predictive models arises from various sources, including the complexity of the underlying data and the dynamic nature of water quality parameters. The structure of machine learning models contributes significantly to this uncertainty, and exploring their characteristics sheds light on the reliability of predictions.

2.3.2. Bootstrapping

In the uncertainty analysis of predictive models for Water Quality Index, generating prediction intervals is crucial for understanding the range of possible values for each prediction. This step involves using bootstrapping, a resampling technique that provides a measure of the uncertainty associated with the model’s predictions. Bootstrapping involves creating multiple bootstrap samples by randomly drawing observations with replacements from the original dataset. For each bootstrap sample, the model is trained, and predictions are made on the test set. This process is repeated numerous times (in our case, 1000 iterations), resulting in a distribution of predicted values for each data point Figure 4.

The Mean Squared Error on the test set (0.108) indicates the average squared difference between the actual Water Quality Index values and the predicted values. A lower MSE generally suggests better model performance, demonstrating that the model’s predictions are, on average, close to the true values. However, the MSE alone may not provide a complete picture, as it does not account for the uncertainty in the predictions. This is where prediction intervals come into play. The generated prediction intervals using bootstrapping offer insights into the variability and uncertainty associated with the model’s predictions. The lower and upper bounds of the intervals (calculated at the 2.5th and 97.5th percentiles, respectively) represent the plausible range within which the true Water Quality Index values are likely to fall. The scatter plot (Figure 4) of actual versus predicted values, along with the shaded gray area representing the prediction intervals, provides a clear visualization of the model’s performance and the associated uncertainty. The narrower the prediction intervals, the more confident we can be in the model’s predictions.

A narrow prediction interval suggests that the model has a high degree of certainty in its predictions. A wider prediction interval indicates higher uncertainty, emphasizing the need for caution when relying on specific predictions in these regions. By incorporating bootstrapping to generate prediction intervals, we not only assess the model’s accuracy through MSE but also gain a comprehensive understanding of the uncertainty inherent in the Water Quality Index predictions. This holistic approach enhances the reliability and robustness of the predictive modeling process, making it more applicable and informative for water quality management and decision-making.

2.3.3. Random Forest and Gradient Boosting

These ensemble methods aggregate predictions from multiple decision trees, which individually capture different patterns in the data. The robustness of Random Forest and Gradient Boosting lies in their ability to mitigate overfitting and enhance predictive accuracy. However, the ensemble nature introduces uncertainty due to the variability in individual tree predictions [61,62].

2.3.4. Support Vector Machine (SVM) and XGBoost

SVM focuses on finding the hyperplane that best separates data into classes, while XGBoost optimizes the performance of weak learners through boosting. The structural complexity of SVM and the iterative refinement process of XGBoost contribute to their predictive power but also introduce uncertainty, particularly in capturing non-linear relationships and intricate patterns [63].

2.3.5. K-Nearest Neighbors (KNN) and Decision Trees

KNN relies on proximity-based classification, and Decision Trees partition the data based on feature splits. These models are interpretable and less complex, but their simplicity can lead to uncertainty when faced with intricate relationships in the data. KNN’s reliance on neighbors introduces variability, while Decision Trees’ sensitivity to data changes may affect stability [64].

Understanding the interplay between model structure and uncertainty is crucial for reliable water quality assessments. The ensemble nature of Random Forest and Gradient Boosting, along with the iterative optimization in SVM and XGBoost, contributes to their robust performance but introduces variability. Simpler models like KNN and Decision Trees may be more interpretable but can exhibit uncertainty in capturing complex relationships. The uncertainty associated with each model’s structure emphasizes the importance of a nuanced approach to water quality prediction. Integrating uncertainty analysis, such as the R-factor, alongside accurate predictions allows for a more informed and cautious interpretation of water quality assessments, fostering a holistic understanding for effective decision-making, as shown in

Table 3. R-factor obtained for all the machine learning algorithms in WQI prediction.

Classifier	R-Factor
K-Nearest Neighbors	0.83
Decision Trees	0.77
Gradient Boosting	0.83
Random Forest	0.83
SVM	0.83
XGBoost	0.83

.

3. Results

3.1. AUC-Based Performance Evaluation

The AUC values, as presented in

Table 4. Performance Evaluation of Machine Learning Algorithms in WQI Prediction.

Algorithm	AUC
Decision Trees (DTs)	0.77
Random Forest (RF)	0.95
Gradient Boosting	0.96
K-Nearest Neighbors (KNN)	0.84
Support Vector Machine (SVM)	0.92
XGBoost	0.95

and Figure 5, offer valuable insights into the performance of various machine learning models in predicting the Water Quality Index. Decision Trees (DTs) exhibit reasonable discriminatory power with an AUC of 0.77, while the Random Forest (RF) and XGBoost models outperform, showcasing high AUC values of 0.95 and 0.96, respectively. These results underscore their robust performance in accurately categorizing water quality. The Gradient Boosting model also demonstrates excellent discriminatory power, with an AUC of 0.95. The Support Vector Machine (SVM) performs admirably with an AUC of 0.92, indicating effective classification. K-Nearest Neighbors (KNN) exhibits good discriminatory power, though slightly lower compared to some other models, with an AUC of 0.84. These varying AUC values emphasize the importance of selecting models with superior discriminatory capabilities when predicting the Water Quality Index, contributing to informed decision-making in environmental management.

The ROC curve for Class 5 closely resembles that of Class 1, positioned near the ideal top-left corner Figure 6. The AUC score of 0.99 highlights exceptional performance in distinguishing Class 5 from other classes. The ROC curves provide a visual representation of the trade-off between sensitivity and specificity for each class, showcasing that all the classifiers performed well to make accurate predictions regarding WQI predictions in our experiment.

3.2. Statistical Analysis Using Friedman Test

The Friedman Test was employed to assess the overall performance variation among multiple machine learning algorithms utilized for predicting the Water Quality Index (WQI) in the Mirpurkhas region of Sindh, Pakistan

Table 5. The results of the Friedman Test, indicating an F-value of 5.0 and a corresponding p-value of 0.4159.

Test	Value
Friedman Test—F-value	5.0
Friedman Test—p-value	0.4159

. The computed Friedman Test statistic yielded an F-value of 5.0 with a corresponding p-value of 0.4159 [65]. This analysis examines whether there exists a statistically significant difference in the performance of the various machine learning models employed for water quality assessment. The obtained p-value of 0.4159, exceeding the conventional significance level of 0.05, indicates insufficient evidence to reject the null hypothesis. Therefore, based on this statistical test, there appears to be no significant difference observed in the predictive performance of the machine learning algorithms utilized for WQI prediction in the Mirpurkhas region [66].

3.3. Nemenyi Test for Pairwise Comparisons

Each value in the matrix represents the critical distance between pairs of algorithms. In this matrix [67], rows and columns correspond to the XGB Classifier, Random Forest Classifier, Support Vector Classifier, K-Nearest Neighbors Classifier, Gradient Boosting Classifier, and Decision Tree Classifier, respectively (labeled 1 to 6)

Table 6. The critical distance values obtained from the Nemenyi Test for pairwise comparisons among six machine learning classifiers (XGB Classifier, Random Forest Classifier, Support Vector Classifier, K-Nearest Neighbors Classifier, Gradient Boosting Classifier, Decision Tree Classifier).

	XGB Classifier	Random Forest Classifier	Support Vector Classifier	K-Nearest Neighbors Classifier	Gradient Boosting Classifier	Decision Tree Classifier
1	1.0	NaN	NaN	NaN	NaN	NaN
2	NaN	1.0	NaN	NaN	NaN	NaN
3	NaN	NaN	1.0	NaN	NaN	NaN
4	NaN	NaN	NaN	1.0	NaN	NaN
5	NaN	NaN	NaN	NaN	1.0	NaN
6	NaN	NaN	NaN	NaN	NaN	1.0

. The value of 1.0 along the diagonal signifies the comparison of an algorithm with itself, showing a critical distance of zero (as expected). The NaN (Not a Number) values outside the diagonal indicate that there is no significant difference between those pairs of algorithms based on the Nemenyi Test at a specific significance level.

The critical distance values are used in post hoc tests like the Nemenyi Test to compare the average ranks of different algorithms and determine which pairs of algorithms exhibit statistically significant differences in performance [68]. If the difference in the average ranks of two algorithms exceeds the critical distance value, this suggests a statistically significant difference in their performance.

3.4. Confusion Matrix

A confusion matrix serves as a tabular tool employed in machine learning and classification endeavors to assess the efficacy of a classification algorithm. It condenses the model’s predictions on a dataset, contrasting them with the actual labels [69]. This matrix proves instrumental in discerning the nature of errors made by a model, including instances of false positives and false negatives.

Table 7. The table represents a confusion matrix detailing the classification results for a multi-class classification for XGB Classifier.

True\Predicted	Class 1	Class 2	Class 3	Class 4	Class 5
Class 1	2 (TP)	3 (FN)	0	0	0
Class 2	0	3 (TP)	0	1 (FP)	0
Class 3	0	3 (FP)	1 (TP)	2 (FP)	1 (TN)
Class 4	0	2 (FP)	0	1 (TP)	1 (TN)
Class 5	0	0	0	2 (FP)	62 (TP)

,

Table 8. The table represents a confusion matrix detailing the classification results for a multi-class classification for Random Forest Classifier.

True\Predicted	Class 1	Class 2	Class 3	Class 4	Class 5
Class 1	4 (TP)	1 (FN)	0	0	0
Class 2	2 (FP)	1 (TP)	0	0	1 (FN)
Class 3	0	1 (FP)	1 (TP)	2 (FP)	3 (TN)
Class 4	0	1 (FP)	0	0	3 (TN)
Class 5	0	0	0	0	64 (TP)

,

Table 9. The table represents a confusion matrix detailing the classification results for a multi-class classification for SVC (Support Vector Classifier with probability = True).

True\Predicted	Class 1	Class 2	Class 3	Class 4	Class 5
Class 1	4 (TP)	1 (FN)	0	0	0
Class 2	2 (FP)	1 (TP)	0	0	1 (FN)
Class 3	1 (FP)	0	0	0	6 (FN)
Class 4	0	0	0	0	4 (FN)
Class 5	0	0	0	0	64 (TP)

,

Table 10. The table represents a confusion matrix detailing the classification results for a multi-class classification for KNN classifier.

True\Predicted	Class 1	Class 2	Class 3	Class 4	Class 5
Class 1	3 (TP)	2 (FN)	0	0	0
Class 2	3 (TP)	0	0	0	1 (FN)
Class 3	0	2 (FP)	1 (TP)	1 (FP)	3 (TN)
Class 4	0	0	0	0	4 (TN)
Class 5	0	1 (FP)	1 (TP)	2 (FP)	60 (TP)

,

Table 11. The table represents a confusion matrix detailing the classification results for a multi-class classification for Gradient Boosting Classifier.

True\Predicted	Class 1	Class 2	Class 3	Class 4	Class 5
Class 1	4 (TP)	1 (FN)	0	0	0
Class 2	2 (FP)	1 (TP)	0	1 (FP)	0
Class 3	0	2 (FP)	2 (TP)	2 (FP)	1 (TN)
Class 4	0	2 (FP)	0	1 (TP)	1 (TN)
Class 5	0	0	0	0	64 (TP)

and

Table 12. The table represents a confusion matrix detailing the classification results for a multi-class classification for Decision Tree Classifier.

True\Predicted	Class 1	Class 2	Class 3	Class 4	Class 5
Class 1	4 (TP)	1 (FN)	0	0	0
Class 2	2 (FP)	2 (TP)	0	0	0
Class 3	0	1 (FP)	3 (TP)	2 (FP)	1 (TN)
Class 4	0	1 (FP)	1 (FP)	1 (TP)	1 (TN)
Class 5	0	1 (FP)	0 (FP)	2 (FP)	61 (TP)

in our study showcase the confusion matrices for all machine learning algorithms utilized, offering a comprehensive overview of their performance.

These tables exhibit the counts of true positives (TP), false positives (FP), false negatives (FN), and true negatives (TN) for each class predicted by each respective classifier in a five-class classification problem for predicting the Water Quality Index (WQI) in the specified region.

The Water Quality Index (WQI) ranges in

Table 13. Water quality ranges and their corresponding classes.

Classes	WQI Range	Water Quality
Class 1	0–25	Excellent water quality
Class 2	26–50	Good water quality
Class 3	51–75	Fair water quality
Class 4	76–100	Poor water quality
Class 5	Above 100	Very poor to unacceptable water quality

were calculated based on a general classification scheme. These ranges are commonly used in water quality assessments, and the specific values may vary depending on the guidelines or standards adopted by environmental agencies.

Total count of Class 5 instances among all classifiers:

$$62+64+64+60+64+61=375$$

(3)

Calculating the percentage of WQI values that belong to Class 5:

$$\mathrm{P}\mathrm{e}\mathrm{r}\mathrm{c}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{a}\mathrm{g}\mathrm{e} \mathrm{o}\mathrm{f} \mathrm{C}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s} 5 \mathrm{W}\mathrm{Q}\mathrm{I}=\frac{\mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l} \mathrm{c}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t} \mathrm{o}\mathrm{f} \mathrm{i}\mathrm{n}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{c}\mathrm{e}\mathrm{s}}{\mathrm{C}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t} \mathrm{o}\mathrm{f} \mathrm{C}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s} 5 \mathrm{i}\mathrm{n}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{c}\mathrm{e}\mathrm{s}}\times 100$$

(4)

$$\mathrm{P}\mathrm{e}\mathrm{r}\mathrm{c}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{a}\mathrm{g}\mathrm{e} \mathrm{o}\mathrm{f} \mathrm{C}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s} 5 \mathrm{W}\mathrm{Q}\mathrm{I}=\frac{375}{422}\times 100\approx 88.63\mathrm{\%}$$

(5)

Approximately 88.63% of the Water Quality Index (WQI) values fall into Class 5, which represents very poor to unacceptable water quality.

Table 14. Testing sample results for RF classifier.

TDS	EC	Well Depth	pH	Sulfate	Chloride	Sodium	Potassium	Magnesium	Calcium	Bicarbonate	Nitrate (NO3-N)	Rescaled_WQI	Model Predicted_WQI_Code
281	2892	24	8.5	17.12571	30.90157	17.18087	0.380653	6.112794	57.36129	82.53007	23.6361	134	5
285	2455	33	7.99	10.50744	25.27189	15.79175	0.880966	10.69318	73.59643	233.2372	5.283148	104	5
293	1266	47.1	7.39	15.33482	14.97501	35.97985	1.158209	14.11058	54.82109	275.1002	3.197757	6	1
302	2111	25	7.56	30.84742	22.94148	12.19949	1.193796	12.96383	78.09345	358.8264	5.686334	80	4
310	2000	19	8.37	67.98364	36.64451	20.84443	2.2859	15.15061	69.56399	358.8264	0.239171	76	4
370	1712	33	7.37	50.69821	23.40693	9.955348	0.890684	9.830066	103.6547	358.8264	10.54824	43	2
405	1122	300	8.21	768.8918	27.44628	15.49567	1.111356	14.22923	98.52735	358.8264	2.862759	66	3
406	1242	60	6.91	54.48075	152.4355	14.49937	1.267128	16.98422	106.119	358.8264	27.40252	26	2
406	1242	40	6.91	55.73943	30.83817	8.220457	1.118785	21.9048	105.0737	358.8264	7.057716	17	1
409	1829	101	10.94	59.5349	12.51032	5.80349	0.708562	10.05297	125.124	358.8264	8.928545	67	3
479	1972	265	9.18	54.17818	62.93912	23.93549	0.597271	21.97534	119.45	358.8264	3.133473	98	4

below shows the testing sample results for the RF classifier.

4. Discussion

This study focused on predicting the Water Quality Index (WQI) in the Mirpurkhas region, Sindh, Pakistan, utilizing various machine learning algorithms and exploring the significance of model structures and variable importance. The extensive analysis encompassed data collection, preprocessing, model development, performance evaluation, statistical tests, and uncertainty analysis, providing a comprehensive understanding of the water quality assessment process.

The AUC-based performance evaluation shed light on the efficacy of machine learning models in discriminating between different water quality classes. Notably, Random Forest and XGBoost demonstrated high AUC values of 0.99 and 0.95, respectively, indicating robust discriminatory power. Gradient Boosting and SVM also exhibited excellent performance, with AUC values of 0.95 and 0.93. Decision Trees, while showing reasonable discriminatory power (AUC of 0.87), stood out as a viable model. The findings of this study shed light on the effectiveness of machine learning models in predicting the Water Quality Index (WQI) for the Mirpurkhas region in Sindh, Pakistan. Notably, XGBoost and Gradient Boosting demonstrated remarkable accuracy rates of 95%, outperforming other models. Random Forest closely followed suit, showcasing its effectiveness in WQI prediction. These outcomes align with the growing body of literature emphasizing the potential of machine learning in water quality assessment. The high accuracy of XGBoost, Gradient Boosting, and Random Forest models suggests their robust performance in capturing the intricate relationships among water quality parameters. Such findings resonate with studies conducted in various regions, where ensemble methods and tree-based models have shown superiority in water quality prediction [70]. The ability of these models to handle non-linear relationships and complex patterns in water quality data enhances their utility in environmental monitoring. Support Vector Machine (SVM) also exhibited commendable performance, with an AUC of 0.93, indicating effective classification. This aligns with studies that have highlighted the versatility of SVM in handling diverse datasets and its efficacy in water quality modeling [71,72]. However, it is essential to acknowledge the variations in model sensitivity to multicollinearity, as indicated by the Variance Inflation Factor (VIF) analysis. Features such as ‘TDS’, ‘Sodium’, ‘Calcium’, and ‘Magnesium’ exhibited high VIF values, suggesting strong interdependencies among these variables. While tree-based models are generally less sensitive to multicollinearity, addressing high VIF values remains crucial for enhancing the reliability of predictive models, especially in linear models like SVM. Information Gain analysis highlighted the relevance of specific physiochemical variables, such as ‘Nitrate (NO₃-N)’, ‘Calcium’, and ‘Sodium’, in WQI prediction. The study recommended future research to address limitations, including dataset size and variable scope. Advanced strategies like feature engineering, ensemble methods, and integration of remote sensing data were proposed to enhance predictive accuracy and provide a nuanced understanding of water quality dynamics. The Information Gain (IG) analysis provided insights into the relevance of different features in predicting WQI. Variables such as ‘Nitrate (NO₃-N)’, ‘Calcium’, ‘Sodium’, ‘Sulfate’, ‘Chloride’, ‘Potassium’, and ‘Magnesium’ exhibited higher IG values, underscoring their considerable impact on water quality assessment. This finding aligns with existing literature emphasizing the importance of specific physiochemical variables in influencing water quality [73,74].

The Friedman Test, employed to assess overall performance variation, yielded an F-value of 5.0 with a p-value of 0.4159. The non-significant p-value suggests consistent performance across machine learning algorithms, emphasizing their similarity in predictive accuracy for WQI in the Mirpurkhas region. The lack of significant differences supports the reliability and consistency of the models. The Nemenyi Test for pairwise comparisons provided critical distance values, offering insights into statistically significant differences in algorithm performance. The absence of significant differences between certain pairs of algorithms highlighted their comparable performance. While the critical distance values can guide algorithm ranking, the overall consistency observed in the Friedman Test aligns with the notion that various algorithms perform similarly in WQI prediction. Confusion matrices detailed the classification results for each machine learning algorithm, presenting true positives, false positives, false negatives, and true negatives for each water quality class. These matrices provide a granular view of model errors and successes, aiding in the interpretation of classification performance. The consistently high true-positive rates and low false-positive rates across classifiers reflect the models’ abilities to accurately predict water quality classes. Defined water quality ranges and corresponding classes facilitated the interpretation of model predictions. Approximately 88.63% of WQI values fell into Class 5, representing very poor to unacceptable water quality. This distribution underscores the predominance of deteriorated water quality in the Mirpurkhas region, emphasizing the urgency of effective water resource management. The uncertainty analysis, incorporating the R-factor and bootstrapping, added a crucial layer of insight into the reliability of model predictions. The R-factor addressed structural uncertainty, while bootstrapping provided prediction intervals, aiding in understanding the range of possible values for each prediction. The approach acknowledged and quantified uncertainty, contributing to a more informed interpretation of water quality assessments. Uncertainty analysis, including the R-factor and bootstrapping, contributed to a nuanced understanding of predictive model reliability. The Monte Carlo simulation method provided a robust approach to assessing the uncertainty associated with WQI predictions. The incorporation of bootstrapping not only assessed model accuracy through Mean Squared Error (MSE) but also provided valuable insights into prediction intervals, offering a more comprehensive understanding of uncertainty in the models.

5. Conclusions

In conclusion, this study demonstrates the efficacy of machine learning models, particularly XGBoost, Gradient Boosting, Random Forest, and SVM, in predicting the Water Quality Index for the Mirpurkhas region. The high accuracy rates of these models underscore their potential for precise water quality assessment. Feature importance analysis highlights the critical role of specific variables, emphasizing the need for targeted monitoring and management. This study’s findings contribute to the broader discourse on machine learning applications in environmental science. The identified variables and models can serve as valuable tools for water resource management, aiding in informed decision-making. Despite the promising results, it is crucial to acknowledge the study’s limitations, including dataset size and variable scope. Future research should explore advanced strategies and incorporate additional parameters for a more comprehensive understanding of water quality dynamics in the region. Overall, this study not only showcases the capabilities of machine learning in water quality prediction but also underscores the importance of considering uncertainties for robust environmental assessments.