# A Spatial-Reduction Attention-Based BiGRU Network for Water Level Prediction

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Study Area Description

#### 2.2. Comparison of RNN Variations

#### 2.3. Bidirectional RNN Structure

#### 2.4. Spatial-Reduction Attention

_{i}is the head number of the Stage i attention layer. Therefore, each head’s dimension is equal to $\frac{{C}_{i}}{{N}_{i}}$. The notation for $\mathrm{SR}(\xb7)$, which reduces the spatial dimension of the input sequence, is as follows:

_{i}) W

^{S})

_{i}, $\mathrm{Norm}(\xb7)$ is the same as the original transformer, and $\mathrm{Attention}(\xb7)$ is calculated as follows:

#### 2.5. Overall Model

## 3. Comparative Experiments and Results

#### 3.1. Data Processing

_{1}, x

_{2}, x

_{3}, x

_{4}, x

_{5}, and x

_{6}, where x

_{4}was the empty item, and the filling of x

_{4}was (x

_{1}+ x

_{2}+ x

_{3}+ x

_{5}+ x

_{6}+ x

_{7})/6. Finally, data outliers such as extremely large and small water-level-observation values that clearly deviate from the average level of the series were eliminated and filled in using the data missing processing method [39]. Furthermore, to hasten the convergence of the proposed model and improve its accuracy, maximum–minimum normalization was used so that all values are compressed within the interval [0, 1] [30].

#### 3.2. Hyperparametric Optimization

#### 3.3. Evaluation Index

_{i}represents the actual value of water level, ${\widehat{y}}_{i}$ represents the predicted water level, and $\overline{y}$ is the average value of actual values.

#### 3.4. Results of Comparative Experiments

#### 3.4.1. Comparative Experiment Results Based on the Bidirectional RNN Structure

#### 3.4.2. Comparative Experiment Results Based on the Spatial-Reduction Attention Mechanism

#### 3.4.3. Overall Comparative Experiment Results

## 4. Discussion

## 5. Conclusions

- GRU and LSTM, both excellent variants of the recurrent neural networks, can use their strong fitting ability in capturing nonlinear characteristics and fully consider the time series of water-level data. Moreover, in this experiment, GRU outperforms LSTM in terms of water-level-prediction accuracy and training speed;
- The bidirectional GRU structure enables the modeling of the potential relationship between past and future water-level data and current data, thereby improving the accuracy of prediction;
- The introduction of spatial-reduction attention based on BiGRU can actively learn the correlation of hidden vectors of BiGRU and highlight the influence of important features on the prediction results, thereby solving the problems of insufficient utilization of spatial information and long time span in water-level-prediction tasks, which lead to the decline of prediction accuracy. Particularly, due to its unique structure, spatial-reduction attention reduces the overhead of multi-head attention mechanism computation and memory;
- All evaluation index values of comparative experiments confirm that the SRA-BiGRU model has higher prediction accuracy in the water-level-prediction task, indicating that it is a high availability, high accuracy, and high robustness water-level-prediction model.

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Guo, Y.H.; Liu, J.L.; Liu, M.J. Research on the relationship between water level and channel navigability. J. Wuhan Univ. Technol.
**2010**, 34, 842–844. [Google Scholar] - Liu, M.J.; Gong, D.P. Analysis on changes of the Changjiang navigation environment. Mar. Eng.
**2004**, 2, 36–39. [Google Scholar] - Xiao, Z.Y.; Chen, Y.L.; Chen, C.H. Web-oriented “one-stop” service system for Yangtze River hydrological data. People’s Yangtze River
**2018**, 49, 111–116. [Google Scholar] [CrossRef] - Sang, L.Z.; Mao, Z.; Zhang, W.J.; Yan, X.P. Realization of early warning system for safe navigation of ships in multi-bridge waters of inland rivers. China Navig.
**2014**, 37, 34–39. [Google Scholar] - Feng, N.C.; Chen, X.N.; Wu, J.B. Construction and operation of water surface evaporation monitoring station in Danjiangkou Reservoir. People’s Yangtze River
**2018**, 49, 29–34. [Google Scholar] [CrossRef] - Wu, Q.; Mei, J.Y.; Du, Y.D.; Yuan, H. Practice and understanding of water resources monitoring in the Yangtze River basin. People’s Yangtze River
**2017**, 48, 12–15. [Google Scholar] [CrossRef] - Bian, N. Application of improved gray system in prediction of channel water level. China Water Transp. Channel Technol.
**2018**, 5, 75–80. [Google Scholar] [CrossRef] - Pan, M.; Zhou, H.; Cao, J.; Liu, Y.; Hao, J.; Li, S.; Chen, C.H. Water level prediction model based on GRU and CNN. IEEE Access
**2020**, 8, 60090–60100. [Google Scholar] [CrossRef] - Pierini, N.A.; Vivoni, E.R.; Robles-Morua, A.; Scott, R.L.; Nearing, M.A. Using observations and a distributed hydrologic model to explore runoff thresholds linked with mesquite encroachment in the Sonoran Desert. Water Resour. Res.
**2014**, 50, 8191–8215. [Google Scholar] [CrossRef][Green Version] - Kaya, C.M.; Tayfur, G.; Gungor, O. Predicting flood plain inundation for natural channels having no upstream gauged stations. J. Water Clim. Chang.
**2019**, 10, 360–372. [Google Scholar] [CrossRef][Green Version] - Lai, X.J.; Jiang, J.H.; Huang, Q. The impact pattern and mechanism of the water storage of the Three Gorges Project on the water regime of Dongting Lake. Lake Sci.
**2012**, 24, 178–184. [Google Scholar] - Zhou, H.; Mao, D.H.; Liu, P.L. Analysis of the influence of the operation of the Three Gorges on the water level of East Dongting Lake. Mar. Limnol. Bull.
**2014**, 4, 180–186. [Google Scholar] [CrossRef] - Zhou, B.D.; Liu, Y.H. Influence of Three Gorges Operation on flood level of Dongting Lake. Hunan Water Conserv. Hydropower
**2003**, 1, 27–29. [Google Scholar] - Liang, C. Water level prediction of Dongting Lake Based on Long-Short-Term Memory Network and the Impact of the Three Gorges Project on the Water Level of Dongting Lake. Ph.D. Thesis, Wuhan University, Wuhan, China, 2019. [Google Scholar]
- Wang, C.H. Application of wavelet neural network model in groundwater prediction. Water Sci. Eng. Technol.
**2016**, 3, 44–46. [Google Scholar] [CrossRef] - Adamowski, J.; Chan, H.F. A wavelet neural network conjunction model for groundwater level forecasting. J. Hydrol.
**2011**, 407, 28–40. [Google Scholar] [CrossRef] - Seo, Y.; Kim, S.; Kisi, O.; Singh, V.P. Daily water level forecasting using wavelet decomposition and artificial intelligence techniques. J. Hydrol.
**2015**, 520, 224–243. [Google Scholar] [CrossRef] - Cortes, C.; Vapnik, V. Support-vector networks. Mach. Learn.
**1995**, 20, 273–297. [Google Scholar] [CrossRef] - Behzad, M.; Asghari, K.; Coppola, E.A. Comparative study of SVMs and ANNs in aquifer water level prediction. J. Comput. Civ. Eng.
**2010**, 24, 408–413. [Google Scholar] [CrossRef] - Dai, L.; Mao, J.; Wang, Y.; Dai, H.; Zhang, P.; Guo, J. Optimal operation of the Three Gorges Reservoir subject to the ecological water level of Dongting Lake. Environ. Earth Sci.
**2016**, 75, 1111. [Google Scholar] [CrossRef] - Zakaria, M.N.A.; Abdul Malek, M.; Zolkepli, M.; Najah Ahmed, A. Application of artificial intelligence algorithms for hourly river level forecast: A case study of Muda River, Malaysia. Alex. Eng. J.
**2021**, 60, 4015–4028. [Google Scholar] [CrossRef] - Ren, T.; Liu, X.; Niu, J.; Lei, X.; Zhang, Z. Real-time water level prediction of cascaded channels based on multilayer perception and recurrent neural network. J. Hydrol.
**2020**, 585, 124783. [Google Scholar] [CrossRef] - Rumelhart, D.E.; Hinton, G.E.; Williams, R.J. Learning representations by back-propagating errors. Nature
**1986**, 323, 533–536. [Google Scholar] [CrossRef] - Potter, C. RNN based MIMO channel prediction. In Differential Evolution in Electromagnetics. Evolutionary Learning and Optimization; Qing, A., Lee, C.K., Eds.; Springer: Berlin/Heidelberg, Germany, 2020; Volume 4, pp. 177–206. [Google Scholar] [CrossRef]
- Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural Comput.
**1997**, 9, 1735–1780. [Google Scholar] [CrossRef] [PubMed] - Gers, F.A.; Schmidhuber, J.; Cummins, F. Learning to forget: Continual prediction with LSTM. Neural Comput.
**2000**, 12, 2451–2471. [Google Scholar] [CrossRef] - Cho, K.; Van Merriënboer, B.; Gulcehre, C.; Bahdanau, D.; Bougares, F.; Schwenk, H.; Bengio, Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv
**2014**, arXiv:1406.1078. [Google Scholar] [CrossRef] - Le, X.H.; Ho, H.V.; Lee, G.; Jung, S. Application of long short-term memory (LSTM) neural network for flood forecasting. Water
**2019**, 11, 1387. [Google Scholar] [CrossRef][Green Version] - Xu, G.Y.; Zhou, X.Y.; Si, C.Y.; Hu, W.B.; Liu, F. Water level time series prediction model based on GRU and LightGBM feature selection. Comput. Appl. Softw.
**2020**, 37, 25–31. [Google Scholar] - Bao, K.; Bi, J.; Gao, M.; Sun, Y.; Zhang, X.; Zhang, W. An improved ship trajectory prediction based on AIS data using MHA-BiGRU. J. Mar. Sci. Eng.
**2022**, 10, 804. [Google Scholar] [CrossRef] - Li, X.; Lu, X.L. Short-term load forecasting model based on dual attention mechanism and GRU network. Comput. Eng.
**2022**, 48, 291–296. [Google Scholar] [CrossRef] - Li, H.J.; Fang, X.; Dai, H.R. Deep knowledge tracking optimization model based on self-attention mechanism and bidirectional GRU neural network. Appl. Res. Comput.
**2022**, 39, 732–738. [Google Scholar] [CrossRef] - Hu, H.X.; Sui, H.C.; Hu, Q.; Zhang, Y.; Hu, Z.Y.; Ma, N.W. Runoff forecasting model based on graph attention network and two-stage attention mechanism. Comput. Appl.
**2022**, 42, 1607–1615. [Google Scholar] - Sudriani, Y.; Ridwansyah, I.; Rustini, H.A. Long short term memory (LSTM) recurrent neural network (RNN) for discharge level prediction and forecast in Cimandiri river, Indonesia. Earth Environ. Sci.
**2019**, 299, 012037. [Google Scholar] [CrossRef] - Ma, R.X.; Zhao, P.; Zhu, J.; Li, Z.L. Development and application of Wujiang water safety prediction and early warning system. People’s Yangtze River
**2019**, 50, 211–216. [Google Scholar] [CrossRef] - Zhang, G.; Tan, F.; Wu, Y. Ship motion attitude prediction based on an adaptive dynamic particle swarm optimization algo-rithm and bidirectional LSTM neural network. IEEE Access
**2020**, 8, 90087–90098. [Google Scholar] [CrossRef] - Agarap, A.F.; Grafilon, P. Statistical analysis on e-commerce reviews, with sentiments classification using bidirectional recurrent neural network (RNN). arXiv
**2018**, arXiv:1805.03687. [Google Scholar] [CrossRef] - Bahdanau, D.; Cho, K.; Bengio, Y. Neural machine translation by jointly learning to align and translate. Statistics
**2014**, 3, 62–68. [Google Scholar] - Cho, M.; Kim, C.; Jung, K.; Jung, H. Water level prediction model applying a long short-term memory (lstm)–gated recurrent unit (gru) method for flood prediction. Water
**2022**, 14, 2221. [Google Scholar] [CrossRef] - Kingma, D.; Ba, J. Adam: A method for stochastic optimization. arXiv
**2014**, arXiv:1412.6980v9. [Google Scholar] [CrossRef] - Wang, W.; Xie, E.; Li, X.; Fan, D.P.; Song, K.; Liang, D.; Lu, T.; Luo, P.; Shao, L. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. In Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 10–17 October 2021; pp. 548–558. [Google Scholar]
- Cryer, J.D.; Chan, K.-S. Time Series Analysis: With Applications in R; Springer: New York, NY, USA, 2008. [Google Scholar]

**Figure 8.**Loss value of comparative experiment based on the BiRNN structure. (

**a**) BiGRU vs. GRU; (

**b**) BiLSTM vs. LSTM.

**Figure 9.**Loss value of comparative experiment based on the spatial-reduction attention. (

**a**) SRA-BiGRU vs. BiGRU; (

**b**) SRA-LSTM vs. BiLSTM; (

**c**) SRA-GRU vs. GRU; (

**d**) SRA-LSTM vs. LSTM.

Methods | MAE | RMSE | NSE |
---|---|---|---|

SRA-BiGRU | 0.54383 | 0.69723 | 0.91097 |

SRA-BiLSTM | 0.55497 | 0.71105 | 0.90324 |

SRA-GRU | 0.72584 | 0.93057 | 0.88365 |

BiGRU | 0.80069 | 1.02653 | 0.86651 |

BiLSTM | 0.87764 | 1.12519 | 0.85985 |

SRA-LSTM | 0.94777 | 1.21509 | 0.84885 |

GRU | 1.32214 | 1.69505 | 0.83982 |

LSTM | 1.41426 | 1.81302 | 0.83179 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Bao, K.; Bi, J.; Ma, R.; Sun, Y.; Zhang, W.; Wang, Y.
A Spatial-Reduction Attention-Based BiGRU Network for Water Level Prediction. *Water* **2023**, *15*, 1306.
https://doi.org/10.3390/w15071306

**AMA Style**

Bao K, Bi J, Ma R, Sun Y, Zhang W, Wang Y.
A Spatial-Reduction Attention-Based BiGRU Network for Water Level Prediction. *Water*. 2023; 15(7):1306.
https://doi.org/10.3390/w15071306

**Chicago/Turabian Style**

Bao, Kexin, Jinqiang Bi, Ruixin Ma, Yue Sun, Wenjia Zhang, and Yongchao Wang.
2023. "A Spatial-Reduction Attention-Based BiGRU Network for Water Level Prediction" *Water* 15, no. 7: 1306.
https://doi.org/10.3390/w15071306