# A Hydrological Data Prediction Model Based on LSTM with Attention Mechanism

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

- The model uses an LSTM as an encoding layer to encode the historical water flow sequence into a context vector, and another LSTM as a decoding layer to decode the context vector to predict the target runoff.
- The model explores the usage of an attention mechanism to improve the prediction accuracy.
- Based on the analysis of normality, smoothness, homogeneity and trend of different runoff data, we demonstrate the positive prediction effect of the model on runoff data with different characteristic properties.

## 2. Related Works

## 3. Model Construction

#### 3.1. Model Structure

#### 3.2. Data Preprocessing and Feature Analysis

#### 3.2.1. Data Preprocessing

#### 3.2.2. Analysis of Features

#### 3.3. Model Setting and Evaluation Index

#### 3.3.1. Model Setting

#### 3.3.2. Model Evaluation Metrics

## 4. Experiment and Result Analysis

#### 4.1. Experimental Environment and Data Set

#### 4.2. Impact of Statistical Characteristics

#### 4.3. Prediction Accuracy Comparison

#### 4.4. Convergence Comparison

#### 4.5. Algorithm Robustness

## 5. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## References

- Yuan, X.; Chen, C.; Lei, X.; Yuan, Y.; Muhammad Adnan, R. Monthly runoff forecasting based on LSTM–ALO model. Stoch. Environ. Res. Risk Assess.
**2018**, 32, 2199–2212. [Google Scholar] [CrossRef] - Adamowski, J.; Sun, K. Development of a coupled wavelet transform and neural network method for flow forecasting of non-perennial rivers in semi-arid watersheds. J. Hydrol.
**2010**, 390, 85–91. [Google Scholar] [CrossRef] - Sang, Y.F.; Wang, Z.G.; Liu, C.M. Research progress of hydrological time series analysis methods. Prog. Geogr. Sci.
**2013**, 32, 20–30. [Google Scholar] - Xiong, L.H.; Jiang, C.; Du, T.; Guo, S.L.; Xu, Z.Y. A review of studies on incongruent hydrological frequency analysis in changing environments. J. Water Resour. Res.
**2015**, 4, 310. [Google Scholar] [CrossRef] - Machiwal, D.; Jha, M.K. Hydrologic Time Series Analysis: Theory and Practice; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2012. [Google Scholar]
- Yang, B. Comparison of several methods of normality test. Stat. Decis. Mak.
**2015**, 4, 72–74. [Google Scholar] - Li, Y.Z.; Yue, C.F. Stability test of time series of runoff extremum in Jingou River Basin. Hydropower Energy Sci.
**2019**, 37, 21–24. [Google Scholar] - Jiang, Y.; Xu, Z.X.; Wang, J. Performance comparison of five trend detection methods based on annual runoff series. J. Hydraul. Eng.
**2020**, 51, 845–857. [Google Scholar] - Liu, J.; Ma, Z.F.; Fan, G.Z.; You, Y. A comparative study of various uniformity test methods. Weather
**2012**, 38, 1121–1128. [Google Scholar] - Hsu, K.; Gupta, H.V.; Sorooshian, S. Application of a recurrent neural network to rainfall-runoff modeling. In Aesthetics in the Constructed Environment; ASCE: Reston, VA, USA, 1997. [Google Scholar]
- Hochreiter, S. The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int. J. Uncertain. Fuzziness Knowl. Based Syst.
**1998**, 6, 107–116. [Google Scholar] [CrossRef] - Graves, A. Long short-term memory. In Supervised Sequence Labelling with Recurrent Neural Networks; Springer: Berlin/Heidelberg, Germany, 2012; pp. 37–45. [Google Scholar]
- Chung, J.; Gulcehre, C.; Cho, K.; Bengio, Y. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv Prepr.
**2014**, arXiv:1412.3555. [Google Scholar] - Kratzert, F.; Klotz, D.; Brenner, C.; Schulz, K.; Herrnegger, M. Rainfall–runoff modelling using long short-term memory (LSTM) networks. Hydrol. Earth Syst. Sci.
**2018**, 22, 6005–6022. [Google Scholar] [CrossRef] - Yin, Z.K.; Liao, W.H.; Wang, R.J.; Lei, X.H. Rainfall-runoff modelling and forecasting based on long short-term memory (LSTM). South North Water Transf. Water Sci. Technol.
**2019**, 17, 1–9. [Google Scholar] - Zuo, G.; Luo, J.; Wang, N.; Lian, Y.; He, X. Decomposition ensemble model based on variational mode decomposition and long short-term memory for streamflow forecasting. J. Hydrol.
**2020**, 585, 124776. [Google Scholar] [CrossRef] - Tian, Y.; Xu, Y.P.; Yang, Z.; Wang, G.; Zhu, Q. Integration of a parsimonious hydrological model with recurrent neural networks for improved streamflow forecasting. Water
**2018**, 10, 1655. [Google Scholar] [CrossRef] - Zhang, J.; Zhu, Y.; Zhang, X.; Ye, M.; Yang, J. Developing a Long Short-Term Memory (LSTM) based model for predicting water table depth in agricultural areas. J. Hydrol.
**2018**, 561, 918–929. [Google Scholar] [CrossRef] - Hu, C.; Wu, Q.; Li, H.; Jian, S.; Li, N.; Lou, Z. Deep learning with a long short-term memory networks approach for rainfall-runoff simulation. Water
**2018**, 10, 1543. [Google Scholar] [CrossRef] - Feng, J.; Pan, F. A LSTM-BP multi-model combined hydrological forecasting method. Comput. Mod.
**2018**, 7, 82–85+92. [Google Scholar] - Fang, K.; Shen, C.; Kifer, D.; Yang, X. Prolongation of SMAP to spatiotemporally seamless coverage of continental US using a deep learning neural network. Geophys. Res. Lett.
**2017**, 44, 11–030. [Google Scholar] [CrossRef] - Zhang, D.; Lindholm, G.; Ratnaweera, H. Use long short-term memory to enhance Internet of Things for combined sewer overflow monitoring. J. Hydrol.
**2018**, 556, 409–418. [Google Scholar] [CrossRef] - Yuan, M.X.; Wei, S.K.; Sun, M.; Zhao, J.D. Seq2Seq Water quality prediction model based on wavelet denoising and LSTM. Comput. Bus. Syst.
**2022**, 31, 38–47. [Google Scholar] - Sun, Y.J.; Tang, W.H.; Wang, C.; Li, Y.D. Prediction method of river level interval based on CNN-Seq2seq. J. Zhejiang Univ. Technol.
**2022**, 50, 381–392+405. [Google Scholar] - Liu, X.L.; Zhao, W.F.; Tang, W. The CNN-Seq2seq PM2.5 one-hour concentration prediction model was applied. Small Microcomput. Syst.
**2020**, 41, 1000–1006. [Google Scholar] - Zheng, Z.S.; Liu, M.; Hu, C.Y.; Fu, Z.P.; Lu, P.; Jiang, X.Y. Typhoon classification prediction based on Seq2Seq and Attention time series satellite cloud images. Remote Sens. Inf.
**2020**, 35, 16–22. [Google Scholar] - Liu, Y.; Zhang, T.; Kang, A.Q.; Li, J.Z.; Lei, X.H. Short-term water level prediction by Seq2Seq model. Adv. Water Conserv. Hydropower Technol.
**2022**, 42, 57–63. [Google Scholar] - Adeloye, A.J.; Montaseri, M. Preliminary streamflow data analyses prior to water resources planning study/analyses préliminaires des données de débit en vue d’une étude de planification des ressources en eau. Hydrol. Sci. J.
**2002**, 47, 679–692. [Google Scholar] [CrossRef] - Aklilu, E.G.; Adem, A.; Kasirajan, R.; Ahmed, Y. Artificial neural network and response surface methodology for modeling and optimization of activation of lactoperoxidase system. S. Afr. J. Chem. Eng.
**2021**, 37, 12–22. [Google Scholar] [CrossRef] - Kim, T.; Yang, T.; Gao, S.; Zhang, L.; Ding, Z.; Wen, X.; Gourley, J.J.; Hong, Y. Can artificial intelligence and data-driven machine learning models match or even replace process-driven hydrologic models for streamflow simulation?: A case study of four watersheds with different hydro-climatic regions across the CONUS. J. Hydrol.
**2021**, 598, 126423. [Google Scholar] [CrossRef] - Nash, J.E.; Sutcliffe, J.V. River flow forecasting through conceptual models part I—A discussion of principles. J. Hydrol.
**1970**, 10, 282–290. [Google Scholar] [CrossRef] - Wang, H.W.; Meng, J. Multiple linear regression predictive modeling method. J. Beijing Univ. Aeronaut. Astronaut.
**2007**, 4, 500–504. [Google Scholar] - Zhang, S.Q.; Wang, W.; Wang, J. Application of ARIMA model in urban annual electricity consumption forecast. Power Demand Side Manag.
**2010**, 12, 31–34. [Google Scholar] - Dong, S.S.; Huang, Z.X. Brief analysis of Random predict theory. Integr. Technol.
**2013**, 2, 1–7. [Google Scholar] - He, X.H.; Duan, Q.C.; Yan, L. Probabilistic prediction of short-term wind speed based on DeepAR. J. Railw.
**2022**. [Google Scholar] - Wang, D.M.; Wang, L.; Zhang, G.M. Short-term wind speed prediction model based on genetic BP neural network. J. Zhejiang Univ.
**2012**, 46, 837–841+904. [Google Scholar]

**Table 1.**Feature verification algorithm [5].

Stability | Students T test Simple T test Mann–Whitney test |

Normality | Kolmogorov–Smirnov test Jarque Bera test Geary’s test |

Tendency | Kendall’s Rank Correlation test Mann–Kendall test SROC test |

Uniformity | Bayesian test Dunnett test Von Neumann test |

Processor | 11th Gen Intel(R) Core(TM) i5-1155G7 @ 2.50GHz 2.50 GHz |

Operating system | Windows10 |

Deep learning Framework | Pytorch |

Integrated development environment | Pycharm2021 |

Programming language | Python3.6 |

Time Stamp | Sensor Number | Water Level Value/m |
---|---|---|

1 January 2015 00:00:00 | 12,911,060 | 60.510 |

11 March 2015 13:00:00 | 12,910,420 | 55.576 |

22 April 2015 12:55:00 | 62,916,750 | 5.580 |

26 June 2015 14:45:00 | 60,403,200 | 7.210 |

Data Set | Purpose of Experiment |
---|---|

Chuhe River water level data | The impact of the statistical analysis on the prediction effect of the model on hydrological data with different characteristics. |

Runcheng, Wuzhi, White Horse Temple, Longmen town, East Bay and Lu’s water flow data | The prediction accuracy of LSTM-seq2seq model comparison to 3 machine learning models and 4 deep learning models. |

Water flow data at Tongguan | The algorithm robustness and convergence speed comparison. |

Model | NSE | |||
---|---|---|---|---|

12,911,060 | 12,910,420 | 60,403,200 | 62,916,750 | |

Stationarity | Normality | Tendency | Uniformity | |

LSTM-BP | 0.81 | 0.79 | 0.75 | 0.78 |

LSTM-seq2seq | 0.83 | 0.81 | 0.79 | 0.79 |

LSTM | 0.71 | 0.77 | 0.69 | 0.76 |

BP | 0.69 | 0.76 | 0.68 | 0.74 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Dai, Z.; Zhang, M.; Nedjah, N.; Xu, D.; Ye, F.
A Hydrological Data Prediction Model Based on LSTM with Attention Mechanism. *Water* **2023**, *15*, 670.
https://doi.org/10.3390/w15040670

**AMA Style**

Dai Z, Zhang M, Nedjah N, Xu D, Ye F.
A Hydrological Data Prediction Model Based on LSTM with Attention Mechanism. *Water*. 2023; 15(4):670.
https://doi.org/10.3390/w15040670

**Chicago/Turabian Style**

Dai, Zhihui, Ming Zhang, Nadia Nedjah, Dong Xu, and Feng Ye.
2023. "A Hydrological Data Prediction Model Based on LSTM with Attention Mechanism" *Water* 15, no. 4: 670.
https://doi.org/10.3390/w15040670