# Deep Learning with a Long Short-Term Memory Networks Approach for Rainfall-Runoff Simulation

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Methods

#### 2.1. Artificial Neural Network

#### 2.2. RNN

#### 2.3. LSTM

- The forget gate defines what information is removed from the cell state.
- The input gate specifies what information is added to the cell state.
- The output gate specifies what information from the cell state is used

- Input node$${g}^{(t)}=tanh({W}_{gx}{x}^{(t)}+{W}_{gh}{h}^{(t-1)}+{b}_{g})$$
- Input gate$${i}^{(t)}=\sigma ({W}_{ix}{x}^{(t)}+{W}_{ih}{h}^{(t-1)}+{b}_{i})$$
- Forget gate$${f}^{(t)}=\sigma ({W}_{fx}{x}^{(t)}+{W}_{fh}{h}^{(t-1)}+{b}_{f})$$
- Output gate$${o}^{(t)}=\sigma ({W}_{ox}{x}^{(t)}+{W}_{oh}{h}^{(t-1)}+{b}_{o})$$
- Cell state$${s}^{(t)}={g}^{(t)}\odot {i}^{(t)}+{s}^{(t-1)}\odot {o}^{(t)}$$
- Hidden gate$${h}^{(t)}=tanh({s}^{(t)})\odot {o}^{(t)}$$
- Output layer$${y}^{(t)}=({W}_{hy}{h}^{(t)}+{b}_{y})$$

#### 2.4. Performance Evalution Criteria

#### 2.5. The Approach and Modelling Process

## 3. Case-Study

## 4. Results

## 5. Discussion and Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Mlv, M.; Todini, E.; Libralon, A. A Bayesian decision approach to rainfall thresholds based flood warning. Hydrol. Earth Syst. Sci. Discuss.
**2006**, 2, 413–426. [Google Scholar] [CrossRef] - Bartholmes, J.C.; Thielen, J.; Ramos, M.H.; Gentilini, S. The european flood alert system EFAS-Part 2: Statistical skill assessment of probabilistic and deterministic operational forecasts. Hydrol. Earth Syst. Sci.
**2009**, 13, 141–153. [Google Scholar] [CrossRef] - Park, D.; Markus, M. Analysis of a changing hydrologic flood regime using the variable infiltration capacity model. J. Hydrol.
**2014**, 515, 267–280. [Google Scholar] [CrossRef] - Meng, C.; Zhou, J.; Tayyab, M.; Zhu, S.; Zhang, H. Integrating artificial neural networks into the VIC Model for rainfall-runoff Modeling. Water
**2016**, 8, 407. [Google Scholar] [CrossRef] - Lee, H.; Mcintyre, N.; Wheater, H.; Young, A. Selection of conceptual models for regionalisation of the rainfall-runoff relationship. J. Hydrol.
**2005**, 312, 125–147. [Google Scholar] [CrossRef] - Calver, A. Calibration, sensitivity and validation of a physically-based rainfall-runoff model. J. Hydrol.
**1988**, 103, 103–115. [Google Scholar] [CrossRef] - Kan, G.; Li, J.; Zhang, X.; Ding, L.; He, X.; Liang, K.; Jiang, X.; Ren, M.; Li, H.; Wang, F.; et al. A new hybrid data-driven model for event-based rainfall-runoff simulation. Neural Comput. Appl.
**2016**, 28, 2519–2534. [Google Scholar] [CrossRef] - Talei, A.; Chua, L.H.C.; Quek, C. A novel application of a neuro-fuzzy computational technique in event-based rainfall-runoff modeling. Expert Syst. Appl.
**2010**, 37, 7456–7468. [Google Scholar] [CrossRef] - Taormina, R.; Chau, K.W. Data-driven input variable selection for rainfall-runoff modeling using binary-coded particle swarm optimization and Extreme Learning Machines. J. Hydrol.
**2015**, 529, 1617–1632. [Google Scholar] [CrossRef] - Hsu, K.; Gupta, H.V.; Sorooshian, S. Artificial Neural Network Modeling of the Rainfall—Runoff Process. Water Resour. Res.
**1995**, 31, 2517–2530. [Google Scholar] [CrossRef] - Radfar, A.; Rockaway, T.D. Captured runoff prediction model by permeable pavements using artificial neural networks. J. Infrastruct. Syst.
**2016**, 22, 04016007. [Google Scholar] [CrossRef] - Salas, J.D.; Markus, M.; Tokar, A.S. Streamflow forecasting based on artificial neural networks. Artif. Neural Netw. Hydrol.
**2000**, 36, 23–51. [Google Scholar] - Tokar, A.S.; Johnson, P.A. Rainfall-runoff modeling using artificial neural networks. J. Hydrol. Eng.
**1999**, 4, 232–239. [Google Scholar] [CrossRef] - Chang, T.K.; Talei, A.; Alaghmand, S.; Ooi, M.P.L. Choice of rainfall inputs for event-based rainfall-runoff modeling in a catchment with multiple rainfall stations using data-driven techniques. J. Hydrol.
**2017**, 545, 100–108. [Google Scholar] [CrossRef] - Yu, Y.; Zhang, H.; Singh, V. Forward prediction of runoff data in data-scarce basins with an improved ensemble empirical mode decomposition (EEMD) Model. Water
**2018**, 10, 388. [Google Scholar] [CrossRef] - Chen, Y.; Fan, R.; Yang, X.; Wang, J.; Latif, A. Extraction of urban water bodies from high-resolution remote-sensing imagery using deep learning. Water
**2018**, 10, 585. [Google Scholar] [CrossRef] - Fischer, T.; Krauss, C. Deep learning with long short-term memory networks for financial market predictions. Eur. J. Oper. Res.
**2018**, 270, 654–669. [Google Scholar] [CrossRef] [Green Version] - LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature
**2015**, 521, 436–444. [Google Scholar] [CrossRef] [PubMed] - Zhang, D.; Martinez, N.; Lindholm, G.; Ratnaweera, H. Manage sewer In-Line storage control using hydraulic model and recurrent neural network. Water Resour. Manag.
**2018**, 32, 2079–2098. [Google Scholar] [CrossRef] - Kratzert, F.; Klotz, D.; Brenner, C.; Schulz, K.; Herrnegger, M. Rainfall-Runoff modelling using Long-Short-Term-Memory (LSTM) networks. Hydrol. Earth Syst. Sci.
**2018**. [Google Scholar] [CrossRef] - Wan, R.; Shan, G. Progress in the hydrological impact and flood response of watershed land use and land cover change. J. Lake Sci.
**2004**, 16, 258–264. [Google Scholar] - Tayfur, G.; Singh, V.P. ANN and fuzzy logic models for simulating event-based Rainfall-Runoff. J. Hydraul. Eng.
**2006**, 132, 1321–1330. [Google Scholar] [CrossRef] - Zhang, D.; Lindholm, G.; Ratnaweera, H. Use long short-term memory to enhance Internet of Things for combined sewer overflow monitoring. J. Hydrol.
**2018**, 556. [Google Scholar] [CrossRef] - Hochreiter, S.; Schmidhuber, J. Long Short-Term Memory. Neural Comput.
**1997**, 9, 1735–1780. [Google Scholar] [CrossRef] [PubMed] - Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. Comput. Sci.
**2014**, arXiv:1412.6980. [Google Scholar] - Dawson, C.; Wilby, R. An artificial neural network approach to rainfall-runoff modelling. Int. Assoc. Sci. Hydrol. Bull.
**1998**, 43, 47–66. [Google Scholar] [CrossRef] [Green Version] - Jhong, Y.D.; Chen, C.S.; Lin, H.P.; Chen, S.T. Physical hybrid neural network model to forecast typhoon floods. Water
**2018**, 10, 632. [Google Scholar] [CrossRef] - Hu, C.; Guo, S.; Xiong, L.; Peng, D. A Modified Xinanjiang Model and Its Application in Northern China. Hydrol. Res.
**2005**, 36, 175–192. [Google Scholar] [CrossRef] - Li, Q. Analysis and discussion related to the hydrological watershed models used in the first hydrological forecasting technology competition of China. Adv. Water Sci.
**1998**, 9, 191–195. [Google Scholar] - Lin, G.F.; Jhong, B.C.; Chang, C.C. Development of an effective data-driven model for hourly typhoon rainfall forecasting. J. Hydrol.
**2013**, 495, 52–63. [Google Scholar] [CrossRef] - Kan, G.; Yao, C.; Li, Q.; Li, Z.; Yu, Z.; Liu, Z.; Ding, L.; He, X.; Liang, K. Improving event-based rainfall-runoff simulation using an ensemble artificial neural network based hybrid data-driven model. Stoch. Environ. Res. Risk Assess.
**2015**, 10, 1345–1370. [Google Scholar] [CrossRef] - Huang, G.; Rui, X.; Shi, P. Analysis of rainfall-runoff characteristics of Jing-Luo-Wei river basin. Adv. Sci. Technol. Water Resour.
**2004**, 24, 21–23. [Google Scholar]

**Figure 1.**ANN architecture with one hidden layer (typical three-layer feed forward artificial neural networks) [10].

**Figure 2.**A simple RNN architecture with one hidden layer (recurrence using the previous hidden state).

**W**,

**U**,

**V**are parameters for weights [23].

**Figure 3.**The architecture of LSTM memory block [17].

**Figure 4.**Location of the study site and the gauge stations. (

**a**) Description of Fen River basin in Shanxi Province of China; (

**b**) Description of Shanxi Province in Chian; (

**c**) The control catchment of Jingle Station in Fen River and distribution of rainfall gauge stations.

**Figure 5.**Box-plots (

**a**) and cumulative distribution function (

**b**) of observed and estimated discharge for the 12 flood events of validation using ANN and LSTM models.

**Figure 6.**The observed and estimated hydrographs (12 flood events of validation) using ANN and LSTM models.

**Figure 7.**Scatter plot of the observed and the simulated runoff during 12 validation flood events. (

**a**) ANN model; (

**b**) LSTM model.

**Figure 8.**Observed and estimated hydrographs of the ANN and LSTM model at the validation stage in 12 flood events.

Event No. | Date | Total Rainfall (mm) | Rainfall Duration (h) | Rainfall Center | Peak Discharge (m${}^{3}$/s) |
---|---|---|---|---|---|

1 | 1 July 1971 | 8.86 | 36 | Ninghuabao | 164.50 |

2 | 23 July 1971 | 63.40 | 69 | Chunjingwa | 261.21 |

3 | 31 July 1971 | 10.44 | 12 | Dongzhai | 286.00 |

4 | 7 August 1971 | 21.07 | 42 | Ninghuabao | 184.14 |

5 | 15 August 1971 | 7.60 | 16 | Chunjingwa | 145.00 |

6 | 27 August 1971 | 15.71 | 36 | Chunjingwa | 112.00 |

7 | 31 July 1972 | 11.98 | 15 | Huaidao | 142.43 |

... | ... | ... | ... | ... | |

92 | 10 October 2007 | 43.88 | 57 | Chashang | 106.00 |

93 | 23 September 2008 | 70.49 | 88 | Qidongzi | 132.00 |

94 | 10 August 2010 | 70.50 | 24 | Songjiaya | 67.00 |

95 | 11 July 2011 | 41.88 | 24 | Dujiacun | 54.35 |

96 | 26 July 2012 | 40.57 | 41 | Ninghuabao | 134.00 |

97 | 30 July 2012 | 41.95 | 41 | Chashang | 61.90 |

98 | 17 July 2013 | 29.91 | 32 | Jingle | 74.40 |

**Table 2.**Comparison of performances of ANN and LSTM models for runoff prediction (lead time = 1 h) at calibration (86 flood events) and validation (12 flood events) periods.

Events | Modes | ${\mathit{R}}^{2}$ | $\mathit{RMSE}$ (m${}^{3}$ s${}^{-1}$) | $\mathit{NSE}$ | $\mathit{MAE}\phantom{\rule{3.33333pt}{0ex}}($m${}^{3}$ s${}^{-1}$) | ${\mathit{ET}}_{\mathit{p}}$ (h) | ${\mathit{EQ}}_{\mathit{p}}$ |
---|---|---|---|---|---|---|---|

Calibration | |||||||

86 events series | ANN | 0.81 | 124.21 | 0.83 | 47.23 | 5.4 | 12% |

LSTM | 0.95 | 45.12 | 0.97 | 12.4 | 2.6 | 4% | |

Validation | |||||||

12 events series | ANN | 0.83 | 35.6 | 0.83 | 23.6 | 3.7 | 14% |

LSTM | 0.96 | 12.4 | 0.96 | 6.3 | 1.4 | 3% |

**Table 3.**The performances of runoff forecasting at different lead times (1–6 h) by ANN and LSTM model for series flood events.

Lead Time (h) | Data | Models | ${\mathit{R}}^{2}$ | $\mathit{RMSE}$ (m${}^{3}$ s${}^{-1}$) | $\mathit{NSE}$ | $\mathit{MAE}$ (m${}^{3}$ s${}^{-1}$) | ${\mathit{ET}}_{\mathit{p}}$ (h) | ${\mathit{EQ}}_{\mathit{p}}$ |
---|---|---|---|---|---|---|---|---|

1 | Calibration | ANN | 0.81 | 124.21 | 0.83 | 47.23 | 5.4 | 12% |

LSTM | 0.95 | 45.12 | 0.97 | 12.4 | 2.6 | 4% | ||

Validation | ANN | 0.83 | 35.6 | 0.83 | 23.6 | 3.7 | 14% | |

LSTM | 0.96 | 12.4 | 0.96 | 6.3 | 1.4 | 3% | ||

2 | Calibration | ANN | 0.83 | 132.2 | 0.86 | 42.13 | 11.4 | 13% |

LSTM | 0.95 | 42.12 | 0.94 | 13.4 | 2.4 | 7% | ||

Validation | ANN | 0.79 | 23.6 | 0.85 | 23.1 | 2.7 | 12% | |

LSTM | 0.93 | 15.4 | 0.95 | 6.3 | 1.8 | 13% | ||

3 | Calibration | ANN | 0.78 | 164.21 | 0.79 | 56.23 | 14.4 | 11% |

LSTM | 0.91 | 47.12 | 0.91 | 13.4 | 2.8 | 6% | ||

Validation | ANN | 0.81 | 25.6 | 0.78 | 23.6 | 4.2 | 15% | |

LSTM | 0.92 | 14.4 | 0.91 | 7.3 | 1.4 | 16% | ||

4 | Calibration | ANN | 0.81 | 144.21 | 0.82 | 48.23 | 11.4 | 12% |

LSTM | 0.91 | 65.12 | 0.91 | 15.4 | 2.8 | 12% | ||

Validation | ANN | 0.72 | 37.8 | 0.81 | 25.6 | 3.1 | 11% | |

LSTM | 0.91 | 13.4 | 0.93 | 11.3 | 1.6 | 15% | ||

5 | Calibration | ANN | 0.78 | 135.21 | 0.81 | 48.23 | 11.4 | 12% |

LSTM | 0.87 | 49.12 | 0.81 | 17.4 | 4.6 | 8% | ||

Validation | ANN | 0.74 | 38.6 | 0.79 | 24.6 | 5.7 | 16% | |

LSTM | 0.84 | 22.4 | 0.91 | 6.3 | 1.4 | 17% | ||

6 | Calibration | ANN | 0.71 | 144.21 | 0.73 | 67.23 | 18.4 | 17% |

LSTM | 0.84 | 48.12 | 0.96 | 13.4 | 2.7 | 12% | ||

Validation | ANN | 0.75 | 25.6 | 0.79 | 23.6 | 3.7 | 14% | |

LSTM | 0.83 | 14.4 | 0.85 | 8.3 | 2.4 | 18% |

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Hu, C.; Wu, Q.; Li, H.; Jian, S.; Li, N.; Lou, Z.
Deep Learning with a Long Short-Term Memory Networks Approach for Rainfall-Runoff Simulation. *Water* **2018**, *10*, 1543.
https://doi.org/10.3390/w10111543

**AMA Style**

Hu C, Wu Q, Li H, Jian S, Li N, Lou Z.
Deep Learning with a Long Short-Term Memory Networks Approach for Rainfall-Runoff Simulation. *Water*. 2018; 10(11):1543.
https://doi.org/10.3390/w10111543

**Chicago/Turabian Style**

Hu, Caihong, Qiang Wu, Hui Li, Shengqi Jian, Nan Li, and Zhengzheng Lou.
2018. "Deep Learning with a Long Short-Term Memory Networks Approach for Rainfall-Runoff Simulation" *Water* 10, no. 11: 1543.
https://doi.org/10.3390/w10111543