Analysis of Prediction Confidence in Water Quality Forecasting Employing LSTM

Abstract

Water quality prediction serves as an important foundation for risk control and the proactive management of the aquatic environment, and the Long Short-Term Memory (LSTM) network has gained recognition as an effective approach for achieving high-precision water quality predictions. However, despite its potential, there is a significant gap in the literature regarding the confidence analysis of its prediction accuracy and the underlying causes of variability across different water quality indicators and basins. To address this gap, the present study introduces a novel confidence evaluation method to systematically assess the performance of LSTM in predicting key water quality parameters, including ammonia nitrogen (AN), biochemical oxygen demand (BOD), chemical oxygen demand (COD), dissolved oxygen (DO), hydrogen ion concentration (pH), and total phosphorus (TP). This evaluation was conducted across three basins with distinct geographical, climatic, and water quality conditions: the Huangshui River Basin (HSB), the Haihe River Basin (HRB), and the Yangtze River Basin (YRB). The results of the confidence evaluation revealed that LSTM exhibited higher credibility in the Haihe River Basin compared to the Yangtze River Basin. Additionally, LSTM demonstrated greater accuracy and stability in predicting total phosphorus (TP) compared to other water quality indicators in both basins, with median NSE values of 0.71 in the HRB and 0.73 in the YRB. Additionally, the research demonstrated a linear relationship between the ability of LSTM models to predict the water quality and temporal autocorrelation as well as the cross-correlation coefficients of the water quality parameters. The coefficients of determination (R²) ranged from 0.59 to 0.85, with values of 0.59 and 0.79 for the YRB and 0.85 and 0.80 for the HRB, respectively. This finding underscores the importance of considering these correlation metrics when evaluating the reliability of LSTM-based predictions.

1. Introduction

Over the past few decades, artificial intelligence (AI) has emerged as an effective alternative approach for modeling complex nonlinear systems [1]. Extensive research has been conducted on river water quality prediction, monitoring, and management using various AI models, including artificial neural networks (ANNs) [1,2,3,4], adaptive neuro-fuzzy inference systems (ANFISs) [5,6,7], and support vector machines (SVMs) [8,9], among others. However, recurrent neural networks (RNNs) are particularly well suited for sequential data due to their inherent structure [10], and they have been utilized for water quality prediction [11,12]. The Long Short-Term Memory (LSTM) network addresses the vanishing or exploding gradient issues associated with RNNs [13], making it even more effective for handling sequential data, such as water quality information [10,14]. Recently, LSTM-based models have demonstrated promising performance in water quality prediction [14,15,16,17,18,19,20].

Recent studies have demonstrated that Long Short-Term Memory (LSTM) networks hold significant value in many time-series predictions and exhibit excellent simulation capabilities. For instance, in the field of runoff prediction, LSTM models have been shown to provide accurate and reliable results, especially when combined with time-frequency analysis methods. However, it is crucial to recognize that LSTM models have their own limitations and are not universally applicable. They are designed to handle specific types of sequential data and may not perform well in all scenarios. For example, the Root Mean Square Errors (RMSEs) of LSTM models for dissolved oxygen (DO) differ across various basins: 0.0396 in Jiangsu Wuxi of the Yangtze River, China; 0.55 in the Prespa Basin in southeastern Europe; 0.07 in Tai Lake, China; and 0.067 in Victoria Bay, China [14,15,21]. Similarly, within the same basin, LSTM models exhibit different performances for various indicators. In the Beilun Estuary of the Guangxi Autonomous Region, China, the Mean Absolute Errors (MAEs) for predicting the weekly pH, DO, chemical oxygen demand (COD), and ammonia nitrogen (AN) were 0.58, 0.51, 0.71, and 0.68, respectively [20]. Moreover, the performance of LSTM is influenced by various factors, including data characteristics, model parameters, and training methods. Despite these considerations, there is a notable gap in the literature regarding the credibility analysis of LSTM predictions. While numerous studies have focused on comparing different models or incrementally improving existing ones, there is a lack of in-depth research on evaluating the confidence level of LSTM predictions in diverse applications [1,18]. For example, in water quality prediction, LSTM models show varying performances across different basins and indicators. The Root Mean Square Errors (RMSEs) of LSTM models for dissolved oxygen (DO) differ significantly across various basins: 0.04 in Jiangsu Wuxi of the Yangtze River, China; 0.545 in the Prespa Basin in southeastern Europe; 0.07 in Tai Lake, China; and 0.067 in Victoria Bay, China [22,23]. Similarly, within the same basin, LSTM models exhibit different performances for various indicators. In the Beilun Estuary of the Guangxi Autonomous Region, China, the Mean Absolute Errors (MAEs) for predicting the weekly pH, DO, chemical oxygen demand (COD), and ammonia nitrogen (AN) were 0.58, 0.51, 0.71, and 0.68, respectively [23,24,25]. However, the underlying causes of these variations remain largely unexplored. There is a striking deficiency in the literature regarding the feasibility and applicability of data-driven models for various water quality indicators and across diverse basins. Although it is evident that LSTM performance fluctuates with different indicators and basins, the root causes of these discrepancies remain largely unexplored. The absence of comprehensive credibility analysis significantly hampers the reliable application and broader adoption of LSTM models in practical scenarios.

The Yangtze River Basin (YRB), the Haihe River Basin (HRB), and the Huangshui River Basin (HSB) are three different basins with distinctive features in China, located in the southern, northern, and northwestern regions, respectively. The YRB is characterized by its humid climate, abundant rainfall, and rich water resources, whereas the HRB is relatively dry, with scarce water resources and frequent water shortages [18,26,27,28,29]. It is worth highlighting that the Haihe River Basin (HRB) experiences the most critical levels of water scarcity and pollution when compared to the other six major river basins in China [30]. Accurate water quality forecasts are essential for early warning and timely intervention to mitigate pollution in these basins [31]. The water shortage problem is more serious in the HSB; the water quantity is scarce, and the water pollution level is serious [32]. Various models, including process-based and machine learning models, have been applied to predict water quality in the HRB [18,33,34,35,36,37] and YRB [21,38,39,40,41]. However, LSTM-based models have been underutilized in these regions [18,21]. Moreover, LSTM applications have been limited to predicting the dissolved oxygen (DO) in both basins, while other critical pollution indicators, such as the biochemical oxygen demand (BOD) and total phosphorus (TP), urgently need to be assessed. Further research is needed to analyze the prediction accuracy of LSTM for these additional water quality indicators. Additionally, comparative analysis of LSTM performance in areas with different water resource availability and pollution conditions is essential to evaluate its applicability.

Comparing LSTM performance between the HRB and YRB can not only reveal the characteristics and influencing factors of LSTM in different regions but also provide valuable insights for similar studies worldwide. As a result, an analytical comparison of the effectiveness of water quality forecasting using LSTM models was conducted across the Yangtze and Haihe River Basins. Our aims are twofold: (1) to demonstrate the diverse performance of LSTM in predicting water quality across different basins and elucidate the underlying causes and (2) to establish the connection between LSTM performance and the data characteristics of the modeled region.

2. Methodology

2.1. Study Area

The Yangtze River Basin (YRB), Haihe River Basin (HRB), and Huangshui River Basin (HSB) were selected as the study area (shown in Figure 1).

The YRB is the largest river in China and the third longest river in the world [42,43,44]. Most of the YRB is in the subtropical monsoon climate zone [29] and is characterized by a warm/humid climate but with an irregular distribution of annual average temperature and precipitation in both the spatial and temporal dimensions. The Haihe River is the largest river catchment in Northern China [45].

The Haihe River Basin (HRB) has about 1.05 × 10⁵ m³/km² (less than one-fifth of that in the YRB) of water resources, covering an area of about 2.6 × 10⁵ km². Located in a continental monsoon climate zone, the HRB belongs to semi-humid and semi-arid region [46]. With the rapid urbanization and economic development, the HRB receives large amount of sewage and waste discharged from varied origins [45,47]. Accordingly, the HRB suffers the most water shortages and water pollution among all seven river basins in China.

The Huangshui River Basin (HSB) plays a crucial role in the socio-economic advancement of Qinghai Province, China. However, due to its typical alpine and arid climate, the basin faces significant challenges, including drought, water shortages, and soil erosion. The average annual precipitation in the HSB ranges from 300 to 500 mm, while the average annual evaporation rate is as high as 800 to 1500 mm, indicating a severe water scarcity situation [30].

2.2. Data Sources

The water quality dataset encompasses monthly records for 23 monitoring sites within the Yangtze River Basin (YRB) spanning from 2003 to 2018, 76 monitoring sites in the Huai River Basin (HRB) from 2010 to 2020, and 68 monitoring sites in the Huangshui River Basin (HSB) between 2011 and 2021 (Figure 1). The monthly concentrations for biochemical oxygen demand (BOD), chemical oxygen demand (COD), dissolved oxygen (DO), ammonia nitrogen (AN), total phosphorus (TP), and hydrogen ion concentration (pH) were collected.

The mean, minimum, maximum, standard deviation (SD), and coefficient of variation (CV) were used for a comparative evaluation of model performance in the three basins, as shown in

Table 1. Basic statistical analysis for water quality indicators of three basins.

Basins	Indicators	Unit	Mean	Minimum	Maximum	SD	CV
YRB	AN	mg/L	0.178	0.025	1.340	0.184	1.035
	BOD	mg/L	1.100	0.500	2.500	0.400	0.300
	COD	mg/L	2.200	0.500	4.100	0.500	0.200
	DO	mg/L	8.530	4.400	13.10	1.510	0.180
	pH	-	7.970	6.930	8.920	0.330	0.040
	TP	mg/L	0.072	0.005	0.250	0.051	0.706
HRB	AN	mg/L	8.104	0.012	122.00	14.554	1.796
	BOD	mg/L	12.300	0.200	220.00	24.700	2.000
	COD	mg/L	11.000	0.600	127.00	16.00	1.400
	DO	mg/L	6.750	0.020	18.80	3.500	0.520
	pH	-	7.890	6.420	8.990	0.380	0.050
	TP	mg/L	0.730	0.005	8.880	1.243	1.703
HSB	AN	mg/L	0.572	0.011	10.80	0.929	1.622
	BOD	mg/L	2.100	0.200	24.00	1.700	0.800
	COD	mg/L	2.100	0.200	13.00	1.000	0.500
	DO	mg/L	8.190	3.220	12.700	1.200	0.150
	pH	-	8.220	6.490	9.290	0.310	0.040
	TP	mg/L	0.081	0.005	1.190	0.103	1.270

.

The Maximal Information Coefficient (MIC) can be utilized as a versatile tool for detecting and quantifying complex dependencies between variables, going beyond traditional correlation measures, which often fail to capture nonlinear relationships [48]. The MIC was computed and employed to determine both temporal autocorrelation and cross-correlation coefficients for water quality indicators across various lag periods, ranging from 1 to 12 months, as illustrated in Figure 2. The measurement MIC is symmetric and normalized into a range $\left[0,1\right]$. A high MIC value indicates a strong dependency between the investigated variables, whereas MIC = 0 describes the relationship between two independent variables.

2.3. Model Development Based on LSTM Models

2.3.1. Principle of the Model

As an advanced recurrent neural network (RNN), Long Short-Term Memory (LSTM) has a specialized neuron structure. This neuron structure has a selective memory function [14,49] by means of a special model structure design, a memory block, for information filtering and conversion [16,50]. A memory block is composed of a forget gate, an input gate, a memory cell, and an output gate [13,51], with its state at a given time depicted in Figure 3.

In the Long Short-Term Memory (LSTM) model, at the preceding time step (t − 1), the memory block retains crucial information in the form of both the cell state (C_t−1) and the output (h_t−1). The initial values for the cell state (C₀) and the hidden state (h₀) are typically set to zero, serving as the starting point for the LSTM cell’s memory. As it progresses to time step t, the current inputs (X_t) become accessible. The process begins with deriving the hidden state (h_t) through a nonlinear transformation. This transformation utilizes the output gate and the newly updated cell state, as described in Equation (1). This hidden state (h_t) encapsulates the information learned from the previous time steps and the current input, forming the basis for subsequent calculations. Next, four multi-layer perceptrons (MLPs) are employed to compute key gates and states within the LSTM cell. Specifically, these MLPs are formulated as Equations (2)–(4) and (6), and they calculate the forget gate (f_t), the candidate cell state ($\widetilde{c_t})$, the input gate (i_t), and the output gate (o_t). These calculations are based on the previous hidden state (h_t−1) and the current input (X_t). The forget gate (f_t) decides what information to discard from the previous cell state, the input gate (i_t) determines what new information to add, and the candidate cell state ($\widetilde{c_t})$ represents the potential new values for the cell state. The output gate (o_t) controls the output based on the updated cell state. Finally, the updated cell state (C_t) is calculated using the relationship outlined in Equation (5). This new cell state incorporates the forgotten information, the new input, and any other modifications decided by the gates, effectively updating the LSTM cell’s memory for the current time step.

$$h_t=o_t\ast {tanh(c}_t)$$

(1)

$$f_t=\sigma \left(W_{xf}{X}_t+W_{hf}{h}_{t-1}+b_f\right)$$

(2)

$$\widetilde{c_t}=\mathit{tanh}\left(W_{xc}{X}_t+W_{hc}{h}_{t-1}+b_c\right)$$

(3)

$$i_t=\sigma \left(W_{xi}{X}_t+W_{hi}{h}_{t-1}+b_t\right)$$

(4)

$$c_t=f_t\ast c_{t-1}+i_t\ast \widetilde{c_t}$$

(5)

$$o_t=\sigma \left(W_{xo}{X}_t+W_{ho}{h}_{t-1}+b_0\right)$$

(6)

In Equations (2)–(4) and (6), $W$ denotes the matrices of weights for the gates or cells with the corresponding subscripts, and $b$ represents learnable biases. Additionally, $\sigma$ and $tanh$ denote the sigmoid function and the tanh function, respectively.

Long Short-Term Memory (LSTM) networks are powerful tools in the realm of deep learning, distinguished by their ability to retain and manipulate information over extended temporal periods. By incorporating specialized memory cells, LSTM networks can “remember” the most pertinent input values, making them exceptionally well suited for sequential learning tasks [49]. This capability enables LSTM models to capture long-term dependencies and patterns within data sequences, which is crucial for accurate predictions and understanding of temporal dynamics.

In the context of the present study, LSTM models were meticulously developed to forecast future one-month values of a specific water quality indicator. To achieve this, two distinct configurations of input variables were employed:

(1) Univariate inputs: In this configuration, the LSTM models solely rely on historical data of the targeted water quality indicator. This approach is straightforward and focuses on understanding the temporal evolution of the single variable of interest.

(2) Multivariate inputs: Alternatively, the LSTM models were also trained using historical data from all available water quality indicators. This multivariate approach provides a more comprehensive view of the water quality system, potentially capturing interdependencies and correlations between different indicators.

For both input configurations, the time steps (k) for the model inputs were varied from 1 to 12 months. This means that the LSTM models were fed with time-series data spanning from the previous 1 to k months.

2.3.2. Model Training and Testing

The data from each station were randomly divided into three subsets—80% for training, 10% for validation, and 10% for testing—to facilitate model training. And same kind of subsets from the same basin were combined to form the training, validation, and test dataset. Then, all water quality indicators were standardized according to Equation (1) to ensure that the model’s input variables remained on the same scale and to guarantee a stable convergence of parameters in LSTM [52].

$$\widetilde{x_i}={\displaystyle \frac{x_i-\overline{x}}{std}}$$

(7)

Here, $\overline{x}$ and $std$ signify the average and standard deviation of the training dataset, respectively; $x_i$ defines every single value of the raw data, while $\widetilde{x_i}$ represents the standardized value.

2.3.3. Model Optimization

For each LSTM model, Keras Tuner (https://keras-team.github.io/keras-tuner/, accessed on 31 March 2025) was employed to optimize the model hyperparameters via a 150-time random search (10 repetitive times training for each random search) from the searching space of hidden layers and neurons [53]. Other hyperparameters of LSTM were determined using trial and error. The batch size and the dropout rates were 256 and 0.5, respectively. The maximum number of epochs for model training was established at 200, signifying that the training phase would terminate when convergence was reached or when the 200 predefined epochs had elapsed. Furthermore, an initial learning rate of 0.1 was assigned to facilitate the learning process. This rate undergoes automatic adjustment downwards if there is no notable improvement in the loss function on the validation dataset for five straight epochs.

During the training phase, models are developed using the training dataset, with Keras Tuner being instrumental in pinpointing the best model architectures through thorough performance assessments on the validation dataset. After identifying these optimal architectures, the LSTM models are fine-tuned through a retraining process and subsequently assessed on both training and testing datasets. To mitigate the variability caused by the random initialization of model parameters, the entire cycle of training, optimization, retraining, and evaluation is carefully repeated 100 times, ensuring the creation of robust, reliable, and generalizable LSTM models.

2.4. Confidence Analysis of LSTM Models

In the field of deep learning and machine learning, confidence is an important measure of the predictive reliability of a model and a key factor in determining whether a model is usable. Confidence usually refers to the degree to which a model is certain of its predictions, and this degree of confidence can be quantified. The applicability of the LSTM model is a key issue that needs to be evaluated for practical application scenarios, such as watershed water quality prediction. Although LSTM performs well in many time-series prediction tasks, its scope of application is limited and cannot be arbitrarily applied to all scenarios. Therefore, this paper constructs a set of model confidence evaluation methods.

2.4.1. Model Accuracy Calculation

By comparing the predicted value and the real value, the accuracy of the model base on the Nash–Sutcliffe Efficiency (NSE; Equation (8)) [50] will be judged. The Nash–Sutcliffe Efficiency (NSE; Equation (8)) [50], a widely recognized metric for hydrological model evaluation [2,51], is employed as the model assessment criterion in the current study:

$$NSE=1-{\displaystyle \frac{{\sum }_{i=1}^n{\left(y_{m,i}-y_{p,i}\right)}^2}{{\sum }_{i=1}^n{\left(y_{m,i}-\overline{y_m}\right)}^2}}$$

(8)

where n is the number of observations; $y_{m,i}$ and $y_{p,i}$ define the ith observations and the corresponding values predicted using LSTM, respectively; and $\overline{y_m}$ represents the average of observations. In a lot of studies, the NSE ranges from $-\mathrm{\infty }$ to 1. The model is acceptable when the NSE ≥ 0.60 [54] and good when the NSE ≥ 0.75 [2,55].

2.4.2. Confidence Analysis

The confidence level and confidence interval were used for confidence analysis based on the NSE. The confidence value under confidence level Z is:

$$c_Z=\overline{X}\pm {\displaystyle \frac{Z\times \sigma }{\sqrt{n}}}$$

(9)

The confidence coefficient, defined as $\delta$, can be simulated as:

$$\delta =max(0,\overline{X}-d_Z)$$

(10)

$$d_Z={\displaystyle \frac{max\left(c_Z\right)-min\left(c_Z\right)}{max\left(c_Z\right)}}$$

(11)

where $\overline{X}$ is the mean of the NSE, $\sigma$ is the standard deviation, n is the sample size, $Z$ is the coefficient under the confidence level (usually 95%), and $d_Z$ is the confidence width of the confidence interval under Z. According to the accuracy evaluation basis of the NSE, the classification criteria are divided into four levels and determined as follows:

$$L\delta =\left\{\begin{array}{cc}L1& bad, \delta <0.4\\ L2& acceptable, 0.4\le \delta <0.6\\ \begin{array}{c}L3\\ L4\end{array}& \begin{array}{c}\begin{array}{cc}& good,\end{array} 0.6\le \delta <0.7\\ \begin{array}{cc}& excellent\end{array},\delta \ge 0.7\end{array}\end{array}\right.$$

(12)

3. Results and Discussion

3.1. Model Accuracy Evaluation of Water Quality Prediction with Different Indexes

The accuracy of the model for various water quality indicators across different basins was examined and compared, considering the input variables and their respective lag times, as illustrated in Figure 4 and Figure 5.

For models with different input conditions (shown in Figure 4), when inputting more variables, most of the median NSEs of the YRB decreased by nearly 0.25 (Figure 4b), while, except for the pH, there was little deterioration in the performances of the models of the HRB (Figure 4a). However, LSTM models to predict the DO in the HRB performed better after inputting data for more water quality indicators (Figure 4a). Moreover, with more time-step data inputs, most LSTM models improved to some extent. Models for predicting the AN and TP of the YRB with multivariate inputs improved significantly when inputting more time-step data.

Figure 5 shows the differences in the performance of LSTM in the HRB and in the YRB under different inputs conditions. In the univariate input condition, LSTM models for the pH in the YRB are better than those for the pH in the HRB, while little difference between the YRB and HRB for predicting the DO or TP was found; the performances for predicting the AN, BOD, and COD in the HRB are much better than those for the YRB. As for the multivariate input condition, LSTM models for all water quality indicators of the HRB perform significantly better than those for the YRB. Nevertheless, the prediction accuracy gap between the HRB and YRB for the AN or TP was narrowed with increasing input time steps.

Table 2. Average ($\overline{X}$) and standard deviation ($\sigma$) of Nash–Sutcliffe Efficiency (NSE) values with the LSTM model.

Index	YRB		HRB		HSB
	$\overline{\mathit{X}}$	$\mathit{\sigma }$	$\overline{\mathit{X}}$	$\mathit{\sigma }$	$\overline{\mathit{X}}$	$\mathit{\sigma }$
BOD	0.060	0.038	0.773	0.070	0.523	0.167
COD	0.041	0.047	0.785	0.047	0.420	0.166
DO	0.375	0.174	0.623	0.055	0.074	0.110
NH3-N	0.233	0.038	0.631	0.082	0.706	0.043
TP	0.463	0.033	0.644	0.056	0.478	0.048
Ph	0.382	0.108	0.471	0.092	0.421	0.103

presents the average Nash–Sutcliffe Efficiency (NSE) values of the LSTM model for five key water quality indicators across three different basins: the Yangtze River Basin (YRB), the Haihe River Basin (HRB), and the Huangshui River Basin (HSB).

This table provides a comprehensive comparison of the model’s performance in predicting these indicators within each basin, highlighting the variability in prediction accuracy across different regions and parameters. The NSE values in the Haihe River Basin (HRB) are generally higher than those in the Yangtze River Basin (YRB) and the Huangshui River Basin (HSB). For instance, the HRB achieves NSE values of 0.773 for BOD, 0.785 for COD, 0.623 for DO, 0.631 for NH₃-N, and 0.644 for TP, indicating relatively high prediction accuracy. In contrast, the YRB shows much lower NSE values, such as 0.060 for BOD, 0.041 for COD, and 0.375 for DO, suggesting that the LSTM model performs less effectively in this basin. The HSB has intermediate NSE values, with 0.523 for BOD, 0.420 for COD, and 0.540 for DO. Notably, the HRB has the highest NSE for NH₃-N (0.631) and TP (0.644), while the YRB has the lowest NSE for most indicators, except for TP, where the HSB has the lowest value (0.478). The significant differences in NSE values across the three basins highlight the importance of considering regional characteristics when applying the LSTM model for water quality prediction. The higher NSE values in the HRB suggest that the model may be more suitable for this region due to specific environmental conditions or data quality. Conversely, the lower NSE values in the YRB indicate potential limitations of the model in this basin, possibly due to more complex hydrological processes or higher data variability. These findings emphasize the need for further investigation into the factors influencing model performance and the importance of tailoring model applications to specific basin conditions. Overall, while the LSTM model shows promise in some regions, its applicability and reliability vary significantly, necessitating careful evaluation and adaptation for different environments.

3.2. Confidence Analysis of LSTM for Water Quality Prediction in the Three Basins

We calculated the mean and standard deviation of the Nash–Sutcliffe Efficiency (NSE) for the predictions of BOD, COD, DO, NH₃-N, TP, and pH across all models in three watersheds. Based on these values, we determined the 95% confidence intervals. Subsequently, we obtained the widths of these confidence intervals under different confidence levels. The results are presented in

Table 3. Confidence interval (C_i) and confidence width $d_Z$ of three basins under confidence level of 95%.

Index	YRB		HSB		HRB
	Ci	${\mathit{d}}_{\mathit{Z}}$	Ci	${\mathit{d}}_{\mathit{Z}}$	Ci	${\mathit{d}}_{\mathit{Z}}$
BOD	[0.058, 0.062]	0.071	[0.769, 0.777]	0.011	[0.514, 0.532]	0.035
COD	[0.038, 0.044]	0.123	[0.782, 0.788]	0.007	[0.411, 0.429]	0.043
DO	[0.365, 0.385]	0.053	[0.620, 0.626]	0.010	[0.068, 0.080]	0.156
NH3-N	[0.231, 0.235]	0.019	[0.626, 0.636]	0.015	[0.704, 0.708]	0.007
TP	[0.461, 0.465]	0.008	[0.641, 0.647]	0.010	[0.475, 0.481]	0.011
Ph	[0.376, 0.388]	0.033	[0.465, 0.477]	0.023	[0.415, 0.427]	0.028

.

For all water quality indicators, as the confidence level increases, the width of the confidence intervals gradually expands, indicating increased prediction uncertainty at higher confidence levels. As shown in the table, at a 95% confidence level, the HRB demonstrates high prediction certainty for multiple key indicators. Particularly for biochemical oxygen demand (BOD), chemical oxygen demand (COD), dissolved oxygen (DO), and pH, the HRB exhibits the narrowest confidence intervals, implying relatively more stable prediction results. In contrast, the YRB shows lower prediction certainty for BOD, COD, and DO, with wider confidence intervals, while the HSB has relatively lower certainty in predicting DO and ammonia nitrogen (NH₃-N). Analyzing the specific performance for different water quality indicators, the HRB performs best in terms of the BOD, COD, and DO, with high prediction certainty and narrow confidence intervals. For the prediction of NH₃-N, the HSB performs optimally, with the smallest confidence interval width. Regarding the prediction of total phosphorus (TP), the YRB also demonstrates relatively high certainty. Overall, the HRB exhibits the best prediction stability, surpassing that of the HSB and YRB. Further analysis of the mean NSE values and prediction reliability levels for the three watersheds is presented in the Figure 6.

From Figure 6a, it can be seen that in terms of model simulation accuracy, the HRB performs the best, followed by the HSB, with the YRB performing the worst. Further analysis of the simulation reliability grades of the LSTM model for different indicators in the three watersheds in Figure 6b reveals that in the YRB, except for TP, the simulation reliability of the other indicators is at grade L1, indicating poor simulation performance and suggesting that the LSTM model may not be suitable for water quality prediction in the YRB; whereas in the HRB, the simulation performance of BOD and COD reaches grade L4 (excellent), and the simulation reliability of indicators such as DO and NH₃-N is also at grade L3, indicating good simulation performance and suggesting that the LSTM model is highly suitable for water quality prediction in the HRB; as for the HSB, the simulation performance of NH₃-N is good, and the simulation performance of other indicators is acceptable, indicating that with certain optimizations, the LSTM model can also be used for water quality prediction in the HSB watershed, and the prediction results are basically reliable.

3.3. Influencing Factors of Model Performance in Different Basins

Most of these phenomena in Section 3.1 and Section 3.2 can be associated with temporal autocorrelation and cross-correlation coefficients, as mentioned in Section 2.2. For water quality indicators of the three basins, their autocorrelation coefficients are generally bigger than their cross-correlation coefficients with other indicators. In the YRB, the cross-correlation coefficients are much smaller (most MICs smaller than 0.2). As the volume of input data expanded in the LSTM model, water quality variables that exhibited low correlation with the output variables introduced significant amounts of redundant information. This influx of redundant data acted as a barrier, impeding the model’s capacity to discern meaningful and coherent patterns among the variables. Consequently, the prediction performance of the model suffered, as it struggled to differentiate between relevant and irrelevant information, leading to a decline in its overall accuracy and reliability [56,57,58]. Therefore, models with multivariate inputs of the YRB will be worse than those with univariate inputs of the YRB, which is consistent with the results. Similarly, TP has the biggest autocorrelation coefficients (MICs more than 0.6, 0.75, and 0.7 in the YRB, HRB, and HSB, respectively) among water quality indicators. However, except for the pH, water quality indicators in the HRB have bigger cross-correlation coefficients (MICs generally bigger than 0.5). In the YRB, LSTM performs the best in predicting the TP. And in the HRB, the prediction performance for the pH is the worst.

Therefore, the correlations between LSTM performances of water quality prediction and temporal autocorrelation/cross-correlation coefficients of water quality indicators warrant further exploration (as shown in Figure 7). In all scenarios, model performances are linearly dependent on the maximum values of temporal autocorrelation/cross-correlation coefficients. In addition, the linear relation is more stable and significant with the coefficients of determination for MICs ranging from 0.59 to 0.85. Different scenarios have almost the same slope in Figure 7. Significant variations are observed in the intercepts of the models, which can be attributed to various factors. One possible reason is the inherent differences across different basins, such as geographical characteristics, climate conditions, and water flow patterns. Another contributing factor could be the distinct input patterns used in the models, which may reflect variations in data collection methods, preprocessing steps, or the specific variables included in the analysis. These differences in intercepts highlight the complexity and variability inherent in modeling water quality across diverse basins and underscore the importance of considering basin-specific characteristics and input patterns when developing and applying such models.

Furthermore, pollutant concentrations of the HRB and HSB are much higher than those of the YRB (Table 1), which could be attributed to large geographical variations [1]. Additionally, the Haihe River Basin is a typical sluice-controlled river. The construction and operation of sluice dams change the flow and other hydrological factors of rivers, which have adverse effects on water [30]. The high pollutant concentration and poor fluidity of water in the HRB and HSB may be one of the reasons why water quality indicators in these two basins are more related than those in the YRB.

Thus, the relationship between concentration and variability of water quality indicators (described by the mean value and coefficient of variation (CV), respectively) and prediction performance (exhibited by the median NSE) has been investigated, as shown in Figure 8. The results revealed that little linear correlation exists between the mean concentration and prediction performance of water quality indicators with coefficients of determination (R²) as low as 0.10. Additionally, there is a certain correlation between model performances and CVs of water quality indicators with an R² of 0.44 and p-value < 0.05. Thus, the performance of the model demonstrates a pronounced and robust linear dependence on the maximum values of the temporal autocorrelation and cross-correlation coefficients of the water quality indicators. This suggests that the model’s predictive accuracy and reliability are significantly influenced by the strength and patterns of temporal relationships among the water quality indicators. Specifically, as the maximum values of these correlation coefficients increase, the model’s performance tends to improve, indicating that it is better able to capture and utilize the underlying temporal dynamics of the water quality data. This finding underscores the importance of considering temporal autocorrelation and cross-correlation in the development and optimization of water quality models.

In this study, forecasts for water quality in the YRB, a large, water-rich basin, were less accurate than those for the HRB and HSB, which are smaller, water-scarce basins. Considering that the HRB and HSB have smaller watershed areas and more intensive monitoring networks than the YRB, more data are available. The variation in water quality is small, which makes the autocorrelation and cross-correlation of water quality variables bigger. This makes the LSTM model more applicable in both basins. It also shows, from the other hand, that increasing the density of the water quality monitoring network and the amount of available water quality data for training can significantly improve the surface water quality prediction performances of the learning models [23].

4. Conclusions

In this study, we developed a comprehensive confidence evaluation method for assessing the performances of LSTM models in water quality prediction. This method was applied to three major river basins in China. By comparing the prediction performance of the LSTM model across these basins for various water quality indicators, including the AN, BOD, COD, DO, pH, and TP, we found that the LSTM model exhibited superior performance in the Haihe River Basin and the Huangshui River Basin. Specifically, the prediction accuracy was higher in these two basins, and the confidence intervals for water quality prediction were also wider, indicating greater reliability. Our analysis revealed that for indicators with higher temporal autocorrelation, such as TP and COD, the prediction accuracy was significantly higher. Further analysis revealed a linear association between the predictive accuracy of the LSTM model and the temporal autocorrelation as well as the inter-variable correlation coefficients of the water quality indicators. This finding underscores the importance of considering the specific data characteristics of the target indicators when applying LSTM models for water quality prediction. While the LSTM model is a powerful tool for time-series prediction, its application must be tailored to the specific characteristics of the basin and the data features of the target indicators. A thorough analysis of the data features and the underlying temporal relationships of the water quality indicators is essential to maximize the effectiveness of the LSTM model in different river basins.