Prediction of Water Level and Water Quality Using a CNN-LSTM Combined Deep Learning Approach

Abstract

A Convolutional Neural Network (CNN)-Long Short-Term Memory (LSTM) combined with a deep learning approach was created by combining CNN and LSTM networks simulated water quality including total nitrogen, total phosphorous, and total organic carbon. Water level and water quality data in the Nakdong river basin were collected from the Water Resources Management Information System (WAMIS) and the Real-Time Water Quality Information, respectively. The rainfall radar image and operation information of estuary barrage were also collected from the Korea Meteorological Administration. In this study, CNN was used to simulate the water level and LSTM used for water quality. The entire simulation period was 1 January 2016–16 November 2017 and divided into two parts: (1) calibration (1 January 2016–1 March 2017); and (2) validation (2 March 2017–16 November 2017). This study revealed that the performances of both of the CNN and LSTM models were in the “very good” range with above the Nash–Sutcliffe efficiency value of 0.75 and that those models well represented the temporal variations of the pollutants in Nakdong river basin (NRB). It is concluded that the proposed approach in this study can be useful to accurately simulate the water level and water quality.

1. Introduction

One of the main sources of the freshwater supply for the uses of domestic and industrial water and agricultural water are rivers. However, these water sources are often limited in many regions. The optimization of water resources management should take into account both quantity and quality. Not only optimizing water distribution to various sectors such as domestic, agricultural, and industrial sectors but also maintaining pollution levels within permissible limits is critical for optimization.

To predict surface water quality, process-based models such as the Soil and Water Assessment Tool (SWAT [1]) and Storm Water Management Model (SWMM [2]) have been widely used. For example, Baek et al. [3] improved the low-impact development module in the SWMM model to accurately simulate total suspended solids (TSS), chemical oxygen demand (COD), total nitrogen (TN) and total phosphorus (TP) in an urban watershed in the Republic of Korea (hereafter South Korea). Even though these conventional process-based models are capable of accurately simulating water quality, large input data and parameters that require high computational costs are often required. However, these datasets are not always available [4]. Furthermore, these limitations may become substantially larger for a river basin with complex hydraulic structures and various water uses, because input data and parameters for these all processes in a complex basin are practically not possible to obtain.

Recently, a deep learning approach has received more attention in water quality modeling. A deep learning technique is one of machine learning techniques and has neural network architectures generally consisting of an input layer, more than one hidden layer, and one output layer [5]. Liu et al. [6] developed a drinking-water quality model using the long short-term memory (LSTM) network for the Yangtze River basin. They concluded that the proposed LSTM network has promise as a tool in predicting the drinking-water quality including pH, dissolved oxygen (DO), chemical oxygen demand (COD), and NH₃-N. The LSTM network was also used to predict other water quality parameters such as water temperature [7]. Barzegar et al. [8] proposed a hybrid convolutional neural network (CNN)-LSTM model and predicted DO and chlorophyll-a (Chl-a) in the Small Prespa Lake in Greece. They found that the hybrid CNN-LSTM model outperformed the standalone machine learning models including CNN, LSTM, support-vector regression (SVR) and decision tree.

Due to the Nakdong river basin (NRB) having various land uses and hydraulic infrastructures, process-based hydrologic and water quality models may have limitations that accurately reflect all hydrologic and hydraulic dynamics including the dam operations. The Nakdong river basin is one of the largest river basins in South Korea and a complex basin with five multi-objective dams (Andong, Imha, Hapcheon, Namgang, and Milyang) and eight weirs (Sangju, Nakdan, Gumi, Chilgok, Gangnjeong-Goryreong, Dalseong, Hapcheon-Changryeong, Changryeong-Haman), indicating a very complicated and complex hydrological and hydraulic processes. The basin supplies various water uses including domestic, agricultural, and industrial water use. The supply water of the five multi-objective dams accounts for domestic and industrial water (58.7%), followed by environmental flow (20.0%) and agricultural water (21.3%) [9]. These operations of those multi-objective dams contribute to higher complexity in the basin. For such a complex basin, the collection of all input data and parameters for process-based models is often limited.

To the best of our knowledge, few studies on water level and water quality in NRB have used the CNN-LSTM combined deep learning approach. We proposed a CNN-LSTM combined deep learning approach by combining CNN and LSTM networks to predict water quality including TN, TP, and TOC.

2. Materials and Methods

2.1. Study Area and Data Acquisition

The NRB is one of the major rivers in South Korea and has been developed and urbanized (Figure 1). This basin is the largest river in South Korea and the drainage area of this basin is about 23,817 km². The NRB has about 200 km and 120 km as the length and width of the basin, respectively [10]. This basin has multiple weirs since the implementation of the Four Major Rivers Restoration Project [11]. The NDB has a monsoon climate with an average annual temperature and rainfall of 14.7 °C and the 1519 mm, respectively. About seven million people have lived near the NDB and more than 10 million people used this river as a drinking water [12]. Over the several decades, the rapid growth of population by industrial and urban development has caused the water quality deterioration in the NRB. Major pollution sources are the industrial wastewater, the livestock, and the urban and agricultural runoff [13].

Water level data were collected from the Water Resources Management Information System (WAMIS) in South Korea, while water quality data were obtained from the Real-Time Water Quality Information (RTWQI) [14]. The water sampling and water level monitoring sites are displayed in Figure 1. The obtained water quality data included TP, TN and TOC. Additionally, the rainfall radar image and operation information of estuary barrage were acquired from the Korea Meteorological Administration (KMA) and the WAMIS, respectively. The entire simulation period was 1 January 2016–16 November 2017 and divided into two parts: (1) calibration period (1 January 2016–1 March 2017); and (2) validation period (2 March 2017–16 November 2017).

2.2. Water Level and Quality Simulation

In this study, CNN and LSTM network were combined for predicting the water levels and water quality concentrations, respectively. CNN and LSTM are the most common algorithms among deep learning (DL) models and have applied to the various fields (e.g., image recognition, transition, and speech analysis) [15,16]. Specifically, CNN has been developed to recognize patterns of image features [17], while LSTM has been widely used for identifying patterns in sequential data such as time series [18]. Figure 2 shows our CNN model architecture with two inputs having different shapes: multi-dimensional and single vector data. The multi-dimensional data consisted of the rainfall radar image with the dimension 251 × 141 (Figure 2a), while the single vectors were applied as the additional information such as the water level in the previous day, the averaged water level for the past three days, the temperature, the operation information of estuary barrage and the evaporation (Figure 2b). Based on assumption that the water level of the Nakdong River Estuary Barrage can influence the water level at the water level monitoring site (Figure 1), because the Nakdong River Estuary Barrage is closed to the site, water levels of one control structure (the Nakdong River Estuary Barrage) were used for this study. The CNN model consisted of three convolutional layers, two max pooling and two fully connected layers. The output image from the convolutional layer and single vector data were fed into a fully connected layer that converts a one-dimensional feature vector (Figure 2c) [19]. The output from the fully connected layer was the water level. The more detailed descriptions for each layer in CNN are found in Section 2.3. A schematic diagram for LSTM is shown in Figure 3. The input data of this model adopted the water level and water quality concentrations in previous time step. This structure comprised LSTM and fully connected layers. The output from the LSTM layers transferred a fully connected layer, resulting in generating the concentrations of the water quality. More details on the LSTM layers are given in Section 2.4. Both CNN and LSTM models used the mean square error (MSE) as the loss function during the model training. The CNN and LSTM model were implemented using TensorFlow (1.4, Google brain, Mountain View, CA, USA) environment based on Python.

2.3. Convolutional Neural Network (CNN)

CNN recognizes the patterns to represent image features by utilizing the convolutional layers [17]. CNNs can receive images or a multi-dimensional matrix, and the neurons in CNN are connected to a smaller feature from the previous layer. This algorithm can reduce computations and prevent overfitting problems [20]. Therefore, CNN has been adopted in numerous studies focusing on the image objects from digital images [21]. A convolutional layer consists of the filter size, padding, and stride as the layer parameters [21]. The filter having a specific size (e.g., FH: filter height, FW: filter width) moves around the input image [22]. The padding inserts the zero values around the input image, which prevents the loss for the feature extraction [23]. The stride can define the step size of the filter in convolutions [24]. In each convolutional layer, the output size is calculated as:

$$\mathrm{OH}=\frac{\mathrm{IH}+2\mathrm{PH}-\mathrm{FH}}{\mathrm{SH}}+1$$

(1)

$$\mathrm{OW}=\frac{\mathrm{IW}+2\mathrm{PW}-\mathrm{FW}}{\mathrm{SW}}+1$$

(2)

where, OH is the height of output, IH is the height of input, FH is the height of the filter, SH is the height of the stride, OW is the width of output, IW is the width of input, PH is the height of the padding, PW is the width of the padding, FW is the width of the filter, and SW is the width direction of the stride [22].

In general, the convolutional layer needs the activation function to transform the signal from linear to non-linear. The rectified linear unit (ReLU) is employed as the activation function in this study. This function improves the computational speed and accuracy compared with the other activation functions (e.g., tangent sigmoid function) [25]. Especially, the ReLU function prevents the vanishing gradient problem by an exponentially decreasing the training gradient. The ReLU function is defined as:

$$\mathrm{f}\left(x\right)=\max \left(0,x\right)$$

(3)

where, $\mathrm{f}\left(x\right)$ is the output of ReLU and x is the input signal.

The max-pooling layer was used to extract the invariant features with an efficient convergence rate. This layer can eliminate the non-maximal values by the non-linear downsampling that can reduce the computational sampling during the CNN process [26]. The fully connected vector connects a loss function to calculate errors between the observed and simulated values by the vectorizing the input signal [27]. The MSE is used as a loss function in our study [28,29]. This calculates errors between simulated and observed values. The mathematical equation of the MSE is as follows:

$$\mathrm{MSE}=\frac{1}{N}{\displaystyle \sum }_{i=1}^N{\left(Y_i-O_i\right)}^2$$

(4)

where, $Y_i$ is the simulated result, $O_i$ is the observed data, and N is the number of the dataset.

The stochastic gradient descent (SGD) optimization was applied to train a CNN network. SGD optimizes the parameters of a CNN network by reducing the loss function, as:

$$\vartheta =\mathrm{argmin}\frac{1}{N}{\displaystyle \sum }_{i=1}^N\ell \left(x_i,\varnothing \right)$$

(5)

where, $\vartheta$ is the network parameter, $x$ is the training dataset, $N$ is the number of the dataset, and $\ell$ is the loss function.

The deep learning models such as CNN and LSTM require e an epoch number, a batch size, and a learning rate as the hyperparameters for the model training. The epoch number is the number of the learning in the entire training dataset, while the batch size is the number of samples that worked in the training at a time [30]. The learning rate is the step size at each iteration to minimize the loss function. In this study, the assigned epoch number and mini batch of CNN were 1000 and 16, respectively, and the applied learning rate was 0.001.

2.4. Long Short-Term Memory (LSTM)

The LSTM network is an extension of the recurrent neural network (RNN). RNN adopts a directed cycle structure that transfers the output of a hidden layer to the same hidden layer [31]. This structure can identify features of time-series by receiving the signal of the previous time. However, RNN has encountered the vanishing gradient problem, resulting in the unacceptable accuracy [18]. This vanishing gradient problem was overcome in LSTM suggested by Hochreiter and Schmidhuber [32]. The cell states can be updated by a gating regulation consisting of three different gates (forget gate, input gate, and output gate) and cells which are connected to each element. The following equations were used in LSTM:

$$C_c^{\langle t\rangle }=tanh(W_c[a^{\langle t\rangle },x^{\langle t\rangle }]+b_c)$$

(6)

$${\Gamma }_i=\delta (W_i[c^{\langle t-1\rangle },x^{\langle t\rangle }]+b_i)$$

(7)

$${\Gamma }_f=\delta (W_f[c^{\langle t-1\rangle },x^{\langle t\rangle }]+b_f)$$

(8)

$${\Gamma }_o=\delta (W_o[c^{\langle t-1\rangle },x^{\langle t\rangle }]+b_o)$$

(9)

$$c^{\langle t\rangle }={\Gamma }_i\ast C_c^{\langle t\rangle }+{\Gamma }_f\ast c^{\langle t-1\rangle }$$

(10)

$$a^{\langle t\rangle }={\Gamma }_o\ast tanh c^{\langle t\rangle }$$

(11)

where, $C_c^{\langle t\rangle }$ is the cell state vector, $a^{\langle t-1\rangle }$ is the activation function at time step t, $x^{\langle t\rangle }$ is the input at current step t, $\delta$ is an element-wise non-linear activation function, ${\Gamma }_i$ is the input gate, ${\Gamma }_f$ is the forget gate, ${\Gamma }_o$ is the output gate, and $c^{\langle t\rangle }$ is a cell state at current step t. The bias and weight matrices are represented as b and W, respectively.

2.5. Performance Evaluation

The accuracies of the predicted water level, TN, TP, and TOC were evaluated using coefficient of determination (R²), Nash–Sutcliffe efficiency (NSE) and mean square error (MSE). The equation of R² and NSE is defined as follows:

$${\mathrm{R}}^2= \left[\frac{{\sum }_{i=1}^n\left(O_i-\overline{O_{ı}}\right)\left(P_i-\overline{P_{ı}}\right)}{{\sum }_{i=1}^n{\left(O_i-\overline{O_{ı}}\right)}^2{\sum }_{i=1}^N{\left(P_i-\overline{P_{ı}}\right)}^2}\right]$$

(12)

$$\mathrm{NSE}=1-\frac{{\sum }_{i=1}^n{\left(O_i-P_i\right)}^2}{{\sum }_{i=1}^n{\left(O_i-\overline{O_{ı}}\right)}^2}$$

(13)

where n is the number of datasets that have the water level (m), TN (mg/L), TP (mg/L) and TOC (mg/L), P_i indicates the predicted results, $\overline{P_{ı}}$ is mean of observed ones and O_i represents the observed data, $\overline{O_{ı}}$ is mean of observed ones.

3. Results and Discussion

3.1. Monitoring of Water Level and Water Quality

The results of the descriptive statistical analyses for the water level, TN, TP, and TOC are summarized in

Table 1. Descriptive statistics of water level, total nitrogen (TN), total organic carbon (TOC), and total phosphorus (TP).

Periods	Descriptive Statistics		Water Level (m)	TN (mg/L)	TP (mg/L)	TOC (mg/L)
Total	Min		1.19	1.104	0.003	2.100
	Max		3.11	4.383	0.061	5.900
	Mean		1.65	2.465	0.021	3.202
	Median		1.64	1.917	0.011	3.100
	Quantile	Q2 (25%)	1.60	2.410	0.016	2.800
		Q3 (75%)	1.69	3.002	0.028	3.500
	Standard deviation		0.12	0.666	0.013	0.577
	CoV		0.07	0.270	0.646	0.180
Training	Min		1.19	1.274	0.003	2.100
	Max		3.11	4.383	0.059	5.900
	Mean		1.68	2.706	0.023	3.100
	Median		1.67	2.660	0.020	2.900
	Quantile	Q2 (25%)	1.63	2.252	0.013	2.700
		Q3 (75%)	1.71	3.096	0.031	3.300
	Standard deviation		0.11	0.589	0.013	0.636
	CoV		0.07	0.218	0.564	0.205
Validation	Min		1.36	1.104	0.004	2.400
	Max		2.71	3.473	0.061	4.600
	Mean		1.62	2.105	0.017	3.366
	Median		1.61	1.866	0.012	3.300
	Quantile	Q2 (25%)	1.57	1.681	0.009	3.100
		Q3 (75%)	1.64	2.671	0.017	3.600
	Standard deviation		0.11	0.610	0.013	0.417
	CoV		0.07	0.290	0.762	0.124

. In this study, the maximum values of water level, TN, TP and TOC were 1.19 m, 1.104, 0.003, and 2.100 mg/L, respectively, while the minimum values of those were 3.11 m, 4.383, 0.061, and 5.900 mg/L. The ratio of TN to TP (hereafter TN:TP ratio) was calculated using the minimum and Q2 values of TN and TP, respectively. The TN:TP ratio using those minimum values was 368 and that using the Q2 values was 150.62. The TN:TP ratio is an indicator of phytoplankton nutrient limitation [33]. These values were much higher than 22 of TN:TP ratio, indicating that NRB was in phosphorus-limited conditions [34,35]. The mean water level, TN, TP, and TOC were 1.65 m, 2.465, 0.021, and 3.202 mg/L, respectively. The median values of water level and TOC were close to the mean values of water level and TOC, while the median values of TN and TP were appreciably different from the mean values of those. The standard deviation of TN was the highest among the pollutants and TP had the highest coefficient of variation. Both statistics are commonly used to quantify the variation of data. However, the coefficient of variation is more proper to the comparison between the variations of each pollutant in that the coefficient of variation is useful to determine the variation of the independent data without considering the unit [36,37]. The water level of Q1 and Q3 were 1.60 and 1.69 (m), while the maximum water level was 3.11 (m), indicating that our data existed at the extreme value. This might be caused by the heavy rainfall that can provoke floods [38]. The validation set of water level and TOC smaller ranged than the training set of those, while these of TN and TP had a similar range to the training set. The standard deviation of the training set was larger than the validation set without TSS.

3.2. Water Level Simulation

Figure 4 presents a comparison between the observed and simulated water levels. The simulated water levels by the CNN model showed good agreement with the observed water levels. The R² values between simulation and observation were 0.934 and 0.923 for the training and validation steps, respectively, while MSE between them were 0.001 and 0.001 (m²) (

Table 2. Performance index of water level and water quality simulation.

Periods	Index	Water Level (m)	TN (mg/L)	TP (mg/L)	TOC (mg/L)
Training	R2	0.934	0.950	0.92	0.860
	MSE	0.001	0.017	1.37 × 10−5	0.055
	NSE	0.926	0.951	0.921	0.864
Validation	R2	0.923	0.970	0.87	0.793
	MSE	0.001	0.010	2.08 × 10−5	0.041
	NSE	0.933	0.987	0.899	0.832

). The NSE values in the training and validation steps were 0.926 and 0.933, which is within the “very good” performance range (0.75 to 1) proposed by Moriasi et al. [39]. These values are in substantial agreement with those of Bustami et al. [40] and Panda et al. [41]. Bustami et al. [42] simulated water levels of the Bedup river in Malaysia using an artificial neural network (ANN) technique which resulted in an R² value of 0.92. Panda et al. [41] produced the water levels of the Mahanadi delta using MIKE and ANN, by showing to R² value of 0.921.

The water level fluctuated in the rainy season that is from June to October, while the variation of the water level was low in the dry season. This can be explained by considering that the rainfall was one of the most influential factors to the water level in that the increment of rainfall increased the water level [42]. Specifically, in the rainy season in 2016, the water level showed 3.11 m that was the highest value for the entire study period. The highest rainfall (407.7 mm) occurred in September of the year 2017. This heavy rainfall could result in a higher peak flow [38]. The CNN model in this study well captured this phenomenon, indicating that this model can simulate extreme water levels. The simulated results also showed relatively higher water levels in the rainy season in 2017 which were very similar to the observations. The water levels between the end of September and early October in 2016 were much higher than those for the same period in 2017. One possible explanation is that the period in the year 2016, typhoon Chava—one of the strongest tropical cyclones that made landfall in South Korea—had a great impact on the Korean peninsula with a large amount of precipitation.

3.3. Water Quality Simulation

Figure 5 shows the comparison between the observed pollutant values and the simulated results of the LSTM model. The R² of TP and TN for the training period were 0.92 and 0.95, respectively, while those in the validation period were 0.87 and 0.97, respectively (Table 2). TOC had the lowest R² value among the pollutants for both of the training and validation periods, with 0.86 and 0.79, respectively. The MSE values for TOC, TN and TP for the training period were 1.37 × 10⁻⁵ 0.017 and 0.055 respectively, while those in the validation period were 2.08 × 10⁻⁵, 0.010 and 0.041, respectively. The NSE values of the LSTM model for both of the training and validation periods were above 0.75 which is within the “very good” performance range (0.75 to 1) in all the pollutants (e.g., TOC, TN and TP) [39]. As shown in Figure 5, the LSTM model in this study well simulated the temporal variations of those pollutants. Since these temporal variations may result from pollutant transport characteristics, this result implies that the LSTM model can properly reflect the transport characteristics of each pollutant. These temporal variations have been well simulated in previous studies. For example, Zhang et al. [43] predicted the temporal variations of DO in the Burnett river using the PCA-RNN model with the R² value of 0.908. Choubin et al. [44] used the CART model to simulate the suspended solids in the Haraz River with an R² value of 0.67. These studies focused on simulating a single pollutant, while our study simulated the concentrations of multiple pollutants (i.e., TOC, TN and TP).

The fluctuations of temporal variations in TOC and TP were higher in the rainy season (June to October) than those in the dry season. This can be explained by considering the rainfall patterns in South Korea. Most of the precipitation in South Korea falls in the summer monsoon season (June to September). TOC and TP were easily washed off by the rainfall resulting in higher concentrations of these pollutants in the rainy season [45,46]. Schrumrf et al. [47] demonstrated that TOC increased with rainfall. Park et al. [48] also showed that TP was higher in the rainy season compared with the dry season. However, the patterns of temporal variations for TN were different from the two pollutants. The TN concentrations increase in the period from February to June. We surmised that the nitrogen fertilizer application contributed to this increase. The fertilizers in South Korea are usually applied in spring and contain a large amount of nitrogen [49,50,51]. NRB has a broad agricultural area that can influence the variation of TN. Karlen et al. [52] reported that higher TN in water was generally found after fertilizer applications.

4. Conclusions and Future Work

In this study, we combined the two deep learning models (CNN and LSTM) to simulate the water level and the three water quality parameters (TN, TP and TOC) in NRB. Among the deep learning models, the CNN model was adopted to simulate the water level, while the LSTM model was selected to simulate the concentration of the pollutants. We found the following in this study:

(1)

The water level from the CNN model produced the NSE value of 0.933 that can be regarded as acceptable model performance. The water levels increased in the rainy season, while those were low in the dry season.

(2)

For all of the pollutants, the NSE values of the LSTM model for the training and validation periods were above 0.75 which is within the “very good” performance range. The LSTM model in this study well represented the different temporal variations of each pollutant type.

(3)

The TOC and TP concentrations had similar temporal variations in that the concentrations of the pollutants were highly fluctuated in the rainy season, while TN increased in the spring season.

This study suggests that the combined approach of the two deep learning techniques proposed in this study has promise as a tool in accurately simulating the water level and water quality and that this approach can contribute to developing effective strategies for better water sustainability and management. Although our model showed the acceptable model performance, only the three different pollutants were investigated in this study. However, most process-based models can simulate a lot more water quality including the three pollutants (e.g., chlorophyll, algae, dissolved oxygen, and fecal bacteria). A further study is recommended to develop deep learning models so that more pollutants including chlorophyll, algae, dissolved oxygen, and fecal bacteria can be simulated. In addition, further study on the deep learning model with “visual explanations” is required, such as Gradient-weighted-Class Activation Mapping (Grad-CAM) [53] and CAM [54], because the deep learning model is a black-box model that has general difficulty in identifying physical features. In addition, the approach outlined in this study should be replicated with other datasets.