Water Quality Predictions Based on Grey Relation Analysis Enhanced LSTM Algorithms

Abstract

With the growth of industrialization in recent years, the quality of drinking water has been a great concern due to increasing water pollution from industries and industrial farming. Many monitoring stations are constructed near drinking water sources for the purpose of fast reactions to water pollution. Due to the relatively low sampling frequencies in practice, mathematic prediction models are clearly needed for such monitoring stations to reduce the delay between the time points of pollution occurrences and water quality assessments. In this work, 2190 sets of monitoring data from automatic water quality monitoring stations in the Qiandao Lake, China from 2019 to 2020 were collected, and served as training samples for prediction models. A grey relation analysis-enhanced long short-term memory (GRA-LSTM) algorithm was used to predict the key parameters of drinking water quality. In comparison with conventional LSTM models, the mean absolute errors (MAEs) to predict the four parameters of water quality, i.e., dissolved oxygen (DO), permanganate index (COD), total phosphorus (TP), and potential of hydrogen (pH), were reduced by 23.03%, 10.71%, 7.54%, and 43.06%, respectively, using our GRA-LSTM algorithm, while the corresponding root mean square errors (RMSEs) were reduced by 24.47%, 5.28%, 6.92%, and 35.89%, respectively. Such an algorithm applies to predictions of events with small amounts of data, but with high parametric dimensions. The GRA-LSTM algorithm offers data support for subsequent water quality monitoring and early warnings of polluting water sources, making significant contributions to real-time water management in basins.

1. Introduction

As a source of life, drinking water with considerable quality has always been of great concern. Especially in recent decades, disqualified drinking water has become more and more serious with the advancement of technology and industrialization [1,2]. Water quality risk assessment and pre-warning has become a new challenge for various countries [3]. In order to evaluate the water quality level, numerous multi-parameter monitoring stations have been built. However, there is always a time delay between the water pollution occurrence and water quality assessments, due to the long sampling period in most monitoring stations. For example, the sampling period is one month for many of the monitoring stations in Malaysia [4], and one or even two weeks in China [5]. Thus, mathematical models to predict the evolution of water pollution in practice are clearly necessary.

Several prediction models are commonly used in practice, including Artificial Neural Networks (ANNs), Support Vector Regression (SVR), and Recurrent Neural Networks (RNNs) [6,7,8,9,10,11]. ANNs are advantageous in treating the system as a ‘black box’, regardless of underlying physical mechanisms [12,13]. Thus, it is difficult to address dominant factors and give hints as to the type of data inputs, data division, and data validations during processing [14]. Compared with ANN, SVR exhibits better performance in data dispersion for water parameter predictions [15]. However, the accuracy of SVR relies strongly on users’ experiences, because various setting parameters are required to be manually defined [2]. RNN enables long-term series learning and shows superior performance in dealing with the sequential data of text, stock information, and water quality data [16,17]. However, as there is an error back-propagation process in RNN, divergences might occur during iterations due to gradient explosions or gradient disappearances of searching routes. Long Short-Term Memory (LSTM), as an improved RNN, solves this problem, because LSTM draws forgetting gates into the memory and truncates the gradient [18,19]. Thus, LSTM has become one of the most popular methods for time serial predictions [20]. For example, Rasheed Abdul Haq [21] and Liu [22] used LSTM to predict the water quality with acceptable accuracy. Like any ANNs methods, in order to increase the accuracy of the solution, large datasets of good quality should be fed to the networks of LSTM, causing massive computational loads. Thus, LSTM is restricted to coping with the problem using intensive feeding datasets and low dimensionality.

Being a multi-response correlation method, Grey Relation Analysis (GRA) can obtain efficient solutions to uncertainty, multi-input, and discrete data problems [23]. It has been successfully used to analyze the weight factors of various water parameters [24,25], as well as to determine the dominant factor during water quality assessments [26,27,28,29]. However, limited studies use the GRA to predict accurate temporal responses of water quality. In this work, we propose a GRA-enhanced LSTM (GRA-LSTM) algorithm to enable predictions of the events with a limited amount of feeding datasets, making full use of multiple simultaneous relevant events. Combining LSTM networks and multidimensional GRA, the GRA-LSTM algorithm is demonstrated here to be able to relieve the quantity requirements of feeding datasets, and thus increase computational efficiency and accuracy for water quality predictions.

2. Methods and Materials

2.1. GRA Formulations

GRA measures the correlation degree of vectors by measuring the vector distance. For the time series, it is commonly used to predict event trends [30,31]. To eliminate the unit effect, all data are normalized by the feature engineering method [30]. All values are scaled to [0, 1].

The format of the data array is $X_i=\left({X_1}^{*},{X_2}^{*},\dots ,{X_k}^{*}\right)$, where i = 1, 2, …, m; k = 1, 2, …, n; m is the number of arrays; and n is the dimension of the array. The symbol ‘*’ indicates the original data.

Three steps are normally involved in calculating correlation degrees between any two data arrays.

Step 1: Scale the original data by the first data array:

$$X_i^{\prime }\left(k\right)=\frac{X_i{(k)}^{*}}{X_1{(k)}^{*}}$$

(1)

or by max-min normalization:

$$X_i^{\prime }\left(k\right)=\frac{X_i\left(k{)}^{*}-\mathit{min}(X_i\right)}{\mathit{max}(X_i)-\mathit{min}(X_i)}$$

(2)

Step 2: Calculate the correlation coefficients by:

$${\zeta }_i\left(k\right)=\frac{\underset{i}{min}\underset{k}{min}\left|X_j^{\prime }\left(k\right)-X_i^{\prime }\left(k\right)\right|+\rho \cdot \underset{i}{max}\underset{k}{max}\left|X_j^{\prime }\left(k\right)-X_i^{\prime }\left(k\right)\right|}{\mid X_j^{\prime }\left(k\right)-X_i^{\prime }\left(k\right)\left|+\rho \cdot \underset{i}{max}\underset{k}{max}\right|X_j^{\prime }\left(k\right)-X_i^{\prime }\left(k\right)\mid }$$

(3)

where $X_j^{\prime }\left(k\right)$ is the reference data array, j = 1, 2, …, m, and j ≠ i. If we define ${\Delta }_i\left(k\right)=\left|X_j^{\prime }\left(k\right)-X_i^{\prime }\left(k\right)\right|,\underset{i}{min}\underset{k}{min}\left({\Delta }_i\left(k\right)\right)$ is equivalent to min{min(${\Delta }_i\left(k\right)$)}.

A similar indication holds for $\underset{i}{max}\underset{k}{max}\left({\Delta }_i\left(k\right)\right)$ and ρ is the resolution coefficient. The value of the correlation coefficient is between 0 and 1. The closer to 1 the value is, the stronger the potential correlation between two arrays.

Step 3: Evaluate the correlation degree by:

$$r_i=\frac{1}{n}{{\displaystyle \sum }}_{k=1}^n{\zeta }_i\left(k\right)$$

(4)

2.2. LSTM Structure

LSTM was first proposed by Hochreiter and Schmidhuber [32]. It can overcome the long computational time, large errors, explosions, or disappearances of data gradients issues of standard RNN networks. The schematic structure of an LSTM network is shown in Figure 1. C_t−1 and C_t are the cell state of the prior nod and the current nod, respectively; $\widetilde{C_t}$ is temporary variable of the cell state; H_t−1 and H_t are the hidden layer state of the prior layer and the current layer state; X_t is the input variable; σ and tanh represent activation functions of neurons; the red crosses represent pointwise multiplication operations; the black cross represents the totaling operation; the two lines with one arrow represent concatenate operation; and the two arrows represent copy operation.

Activation functions σ and tanh are used to scale the input vector $x$ into a range of [0, 1], and are expressed mathematically as:

$$\sigma =sigmoid(\omega x+b)$$

(5)

$$\mathit{tanh}=\mathit{tanh}\left({\omega }^{T}x+b\right)$$

(6)

where ω and b are the transfer and bias matrices, respectively.

The main difference between an LSTM structure and a standard RNN structure lies in the introduction of memory cell C and gate structures. The gate structures consist of three parts: a forgetting gate, an updating gate, and an output gate. The forgetting gate is expressed using Equation (7), which determines how much information is forgotten in the memory cell by the state of hidden layers at the previous node and at the current input.

$$f_t=\sigma \left({\omega }_f\cdot \left[H_{t-1},X_t\right]+b_f\right)$$

(7)

where $H_{t-1}$ is the hidden state matrix of the last time stamp; sigmoid function σ maps the values to [0, 1]; 1 represents forgetting all; and 0 represents forgetting null.

The update gate is described in Equations (8)–(10). Equation (8) is the input gate of the current memory cell. Equation (9) is the information brought by the current input. The current memory cell is updated with Equation (10), in accordance with the past and present states of the input cell.

$$i_t=\sigma \left({\omega }_i\cdot \left[H_{t-1},X_t\right]+b_i\right)$$

(8)

$$\widetilde{C_t}=tanh\left({\omega }_c\cdot \left[H_{t-1},X_t\right]+b_c\right)$$

(9)

$$C_t=f_t·C_{t-1}+i·C_t$$

(10)

where $i_t$ is the coefficient that controls the candidate memory cell $\widetilde{C_t}$ to update the memory cell; the candidate memory cell $\widetilde{C_t}$ holds short-term dependency information computed from the input matrix $X_t$.

The output gate is described using Equations (11) and (12). The output current cell state is determined by the hidden layer state of the previous time node and the input node.

$$O_t=\sigma \left({\omega }_o\cdot \left[H_{t-1},X_t\right]+b_o\right)$$

(11)

$$H_t=O_t·tanh\left(C_t\right)$$

(12)

2.3. Materials

The algorithm of GRA-LSTM is used to predict the water quality of Qiandao Lake, Zhejiang Province, China, based on historical data collected by a monitoring station. Figure 2 shows the geographical location of the monitoring station. The fundamental parameters to assess the water quality include pH, dissolved oxygen (DO), permanganate index (COD), ammonia nitrogen (NH₃-N), and total phosphorus (TP). The data were collected for 4 h each day during 2019 and 2020.

Table 1. Water quality information statistics.

Water Quality Parameters	DO (mg/L)	NH3-N(mg/L)	pH	COD (mg/L)	TP (mg/L)
Minimum value	4.05	0	6.68	0.8	0
Maximum value	11.05	0.36	7.58	6.90	0.39
Average value	7.24	0.048	7.16	1.62	0.025
Standard deviation	1.261	0.044	0.206	0.684	0.024
Skewness	0.819	2.762	−0.394	4.082	6.111

shows the statistic information of measured parameters. Due to occasional malfunctions of some sensors, some data are either abnormal or lost. Such a database is used to verify our GRA-LSTM algorithms.

3. Modeling Flow

The modeling platform in this study was based on TensorFlow, and the development environment was Python 3.7. Figure 3 shows the flow chat of GRA-LSTM model, the details of which are described in the following four steps.

Step 1: Data preprocesses. The missing data from the original dataset are replenished by Lagrange interpolations of adjacent values (missing points are less than 6) or the averaged values of the previous seven (missing points are more than 6). Figure 4 shows the preprocessed data. From Figure 4, we can observe that the concentrations of DO and COD are almost one order of magnitude higher those that of TP and NH₃-N. The concentration of TP is close to zero, with the exception of noise-like fluctuations. The sharp rises and falls in the preprocessed data indicate significant changes in water quality, which should be captured by the prediction model and then trigger the warning in advance for water managers.

Step 2: GRA model establishment. The preprocessed data are subjected to feature scaling in accordance with Equation (1) or Equation (2), eliminating the magnitudes and units. The grey correlation coefficients and degrees are then calculated according to Equation (3) and Equation (4), respectively. Figure 5 shows calculated grey correlation coefficients using two feature scaling methods. When the original data are scaled by the first data array, as shown in Figure 5a, the coefficients almost reach beyond 0.9. However, when the original data are scaled by the max-min normalization, as shown in Figure 5b, the coefficients span between 0.5 and 1, and the correlations between different quality parameters becomes much more distinguishable. The concentrations of NH₃-N, TP, and COD are strongly relevant, while DO and pH are more relevant. Thus, max-min normalization will be used in the GRA process.

Step 3: LSTM model establishment. The data are first divided into three subsets: a training set, a development set, and a validation set, following a typical ratio of 6:2:2 [14]. Since there are 2190 sets of data in total for each water parameter we observed, the sizes of the three datasets are 1314, 438, and 438, respectively, as shown in Figure 4. The measurement data on the previous 12 nodes are used as input data, and the data of the 13th node is used as output data. The output data is further weighted by the grey correlation degree of the other water quality parameters calculated in Step 2. Different optimizers, such as Stochastic Gradient Descent, AdaGrad, and Adaptive Moment Estimation, are applied to adjust the model combining the training set and the development set. Hyper-parameters of the network, including total number of LSTM layer, number of neurons, attenuation coefficient, learning rate, patience value, epoch, and batch side are adjusted according to the root mean square errors (RMSEs) of the water quality parameters. The final hyper-parameters are listed in

Table 2. Hyper-parameter settings.

Hyperparameters	DO	NH3-N	pH	COD	TP
Total number of LSTM layers	4	4	4	4	4
Number of neurons	100	100	100	100	100
Attenuation coefficient	0.8	0.1	0.6	0.6	0.1
Learning rate	0.0001	0.0001	0.0001	0.0001	0.001
Patience values	2	5	2	15	5
Epoch	200	200	200	200	200
Batch size	128	8	16	64	8

.

Step 4: Figure of merit (FOM) evaluations. The fitted model is verified by the verification set. The FOM of the model performance which we used is the RMSE between the verification set and the predicted data from the LSTM networks.

4. Results and Discussions

Figure 6 shows the present water quality predictions and their comparisons with the original data. The configurations of the hardware and software, as well as the calculation costs of LSTM and GRA-LSTM, are listed in

Table 3. Hardware and software configurations.

Algorithm	LSTM	GRA-LSTM
Processor	Core i7-6700HQ CPU: 8	Core i7-6700HQ CPU: 8
Configurations	Windows 10 + python3.7	Windows 10 + python3.7
Calculation time	220.5 s	219.4 s

. From Table 3, it can be found that the GAR-LSTM model slightly saves computational time, even if the same processor and platform are used. The computation efficiency could be considerable if a significant amount of feeding data were needed.

The parameters of water quality include DO (a), COD (b), TP (c), pH (d), and NH₄-N (e), which are predicted using conventional LSTM (red) and GRA-LSTM (yellow) methods. The upper panels in Figure 5 show the comparisons of the predicted data with the original data (black lines), and the lower panels show the residual errors relative to original data using two prediction methods. It should be noted that the regions filled with orange color are the intersections of residual errors using the conventional LSTM (red) and GRA-LSTM (yellow) methods. As can be clearly seen from the upper panels, the predictions generally follow the tendencies of original data, regardless of noise-like fluctuations and sharp changes, validating the feasibility of prediction models. It can also be observed that the residual errors of conventional LSTM are globally larger than those of GRA-LSTM, since the yellow regions in the lower panels of Figure 6a–e are generally covered by the red regions.

In order to quantify the residual errors from the two prediction models, we calculate the mean absolute errors (MAEs), root mean square errors (RMSEs) and Nash–Sutcliffe efficiency coefficients (NSEs) for each parameter of water quality. As shown in the bar diagram in Figure 7a, the MAEs of GRA-LSTM are globally lower than those of conventional LSTM. The MAEs of five parameters are DO, COD, TP, pH, and NH₃-N, and these are reduced by 23.03%, 10.71%, 7.54%, 43.06%, and 1.62% by using the GRA-LSTM algorithm. As for RMSEs, as can be observed in Figure 7b, with the exception of a comparable level in NH₃-N, the RMSEs of LSTM are generally higher than those of GRA-LSTM, which can be reduced by 24.47%, 5.28%, 6.92%, and 35.89% for the four parameters: DO, COD, TP, and pH, respectively. We also calculate the NSEs for each water parameter, which statistically denote the relative magnitude of the residual variance compared to the measured data variance. As seen from Figure 7c, with the exception of a slightly lower level in NH₃-N, NSEs of the other parameters are higher and closer to 1 when using the GRA-LSTM algorithm, indicating the higher estimation skill of the prediction model. We note that the NSEs of TP are negative, indicating that the two prediction models are unreliable. This is due to the low concentration of TP in the water, which is quite close to the noise level. Based on the above error analyses, we can conclude that GRA-LSTM demonstrates a higher precision and robustness in comparison with conventional LSTM.

Similarly to other kinds of prediction methods based on artificial neural networks [33], GRA-LSTM treats temporal response of multiple parameters as a black box, regardless of its internal physical mechanisms. This implies that GRA-LSTM can treat universe data series, which might show sharp variances and even random distributions. Being different from other neural network-based models, GRA-LSTM does not need a large amount of feeding data. This is because additional correlations of other events occurring in the same time range are implemented. This saves considerable computational resources and time, allowing the possibility of the realizations of real-time predictions. In addition, due to the considerations of other relevant events or factors, the GRA-LSTM algorithm normally shows better immunity to background noises, indicating a higher stability and robustness compared with other neural network-based methods. This has been demonstrated in the comparison of NSEs. On the other hand, each data series predicted by GRA-LSTM is dependent and are limited to the situations in which the measurements are multiple dimensionalities.

5. Conclusions

By combining conventional LSTM networks with GRA correlation models, an improved GRA-LSTM algorithm was developed in order to better predict the real-time quality of drinking water. A total of 2190 sets of monitoring data were collected from monitoring stations in the Qiandao Lake, China, from 2019 to 2020, and these served as feeding datasets. Each dataset was divided into a ratio of 6:2:2, each of which served as training set, development set, and validation set, for the purpose of dynamic self-corrections. GRA degree coefficients between different water quality parameters were used to correct the weight factors of LSTM networks. Compared with conventional LSTM models, the MAEs of four key parameters of water quality, DO, COD, TP, and pH, were reduced by 23.03%, 10.71%, 7.54%, and 43.06%, respectively, while corresponding RMSEs were reduced by 24.47%, 5.28%, 6.92%, and 35.89%, respectively. Combining GRA and conventional LSTM algorithms, the GRA-LSTM is proposed to treat discrete events, using a small amount of feeding datasets and multiple correlated events in the same time range. The prediction precision and robustness are further enhanced by the implementation of correlation coefficients to serve as the weight factors at each node of neural networks. We envision that other mathematical models, such as Markov chains and Kalman recursions, could be further implemented into the node of our GRA-LSTM or for more generalized artificial neural networks, in order to better cope with discrete events with sequential correlations.