# Transformer Based Water Level Prediction in Poyang Lake, China

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Study Area

^{2}during the dry season to around 3000 km

^{2}in the wet season [38]. The lake bottom elevation increases from north to south, and there are five hydrological gauging stations: Hukou, Xingzi, Duchang, Tangyin, and Kangshan stations.

#### 2.2. Research Data

#### 2.3. Transformer-Based Water Level Prediction

#### 2.3.1. Modelling

#### 2.3.2. Training and Testing

_{i}is the observation value, $x$

_{max}and $x$

_{min}are the maximum and minimum of the data set, respectively.

^{0}, we then input them into several transformer layers to conduct feature representation learning according to the following formulas:

#### 2.4. LSTM-Based Water Level Prediction

#### 2.5. Implementation Details

^{−5}. The batch size is 32 and we train the model up to 150 epochs. We evaluate the model every one epoch and select the model that performs best on the validation set for producing the final results. All the training and testing operations are conducted on a GeForce RTX 3090 GPU card with 24 G memory.

#### 2.6. Performance Evaluation

^{2}), Nash–Sutcliffe efficiency coefficient (NSE). and the root mean square error (RMSE). The value of R

^{2}is from 0 to 1, and the larger the value, the better the model simulation effect. NSE is a parameter that determines the relative importance of the residual variance compared to the observed variance. It takes a value from negative infinity to 1. The closer to 1, the better the simulation effect and the higher the credibility of the model; close to 0, the simulation result is close to the mean values of the observations, the root mean square error (RMSE), but the process simulation error is large. An RMSE value of 0 indicates that the observed value is in perfect agreement with the predicted value.

^{2}, RMSE, and NSE are listed as follows:

## 3. Results and Discussion

#### 3.1. Model Performance at Lake Water Level Simulation

^{2}of 0.84–0.91 at four hydrological stations in training stage and an R

^{2}of 0.76–0.84 in testing stage. The RMSE is 1.12 m, 1.02 m, 0.96 m, and 0.94 m in training stage and 1.60 m, 1.63 m, 1.66 m, and 1.28 m in testing stage in Xingzi, Duchang, Tanyin, and Kangshan stations, respectively. The NSE of S1 is 0.83–0.91 and 0.69–0.79 in training stage and testing stages, respectively. Scenario 2 (S2) produced a much better simulation at all four stations. R

^{2}values increased from 0.76–0.91 for S1 to 0.97–1.0 for S2, which showed the highest fit between observations and simulations. The RMSE values of S2 are 0.16 m, 0.15 m, 0.14 m, and 0.14 m for four stations in the training stage respectively, and this is much lower than that of S1 (0.94–1.12 m) at the same stage. Similarly, in the testing stage, the RMSE values decreased from 1.28–1.66 m in S1 to 0.26–0.70 m in S2. NSE values of S2 are higher than 0.94 in both training and testing stages at all four stations, and are higher than that of S1. Moriasi [44] showed that NSE values around 0.75 mean the model simulation is effective. Therefore, the simulation accuracy of S2 is high and can reflect the temporal water level variation characteristics in Poyang Lake. The better performance of S2 indicated that the discharge of the Yangtze River has non-negligible effects on the Poyang Lake water level variations. Li [37] found similar results using the back-propagation neural network, and Yu 2022 [45] also found that the Yangtze River plays the most important role in lake water level variations, followed by five rivers inflowing, and lake rainfall effects were the weakest.

#### 3.2. The Contribution of the Yangtze River on Simulation Performance in Different Stages

#### 3.3. Effects of Window Size, Model Layers, and Lead Time on Model Performance

## 4. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## References

- Zhang, Q.; Ye, X.C.; Werner, A.D.; Li, Y.L.; Yao, J.; Li, X.H.; Xu, C.Y. An investigation of enhanced recessions in Poyang Lake: Comparison of Yangtze River and local catchment impacts. J. Hydrol.
**2014**, 517, 425–434. [Google Scholar] [CrossRef] - Cusell, C.; Lamers, L.P.; van Wirdum, G.; Kooijman, A. Impacts of water level fluctuation on mesotrophic rich fens: Acidification vs. eutrophication. J. Appl. Ecol.
**2013**, 50, 998–1009. [Google Scholar] [CrossRef] - Wang, P.; Zhang, Q.; Xu, Y.S.; Yu, F.H. Effects of water level fluctuation on the growth of submerged macrophyte communities. Flora
**2016**, 223, 83–89. [Google Scholar] [CrossRef] - Khanal, R.; Uk, S.; Kodikara, D.; Siev, S.; Yoshimura, C. Impact of water level fluctuation on sediment and phosphorous dynamics in Tonle Sap Lake, Cambodia. Water Air Soil Pollut.
**2021**, 232, 139. [Google Scholar] [CrossRef] - Li, B.; Yang, G.; Wan, R.; Lai, X.; Wagner, P.D. Impacts of hydrological alteration on ecosystem services changes of a large river-connected lake (Poyang Lake), China. J. Environ. Manag.
**2022**, 310, 114750. [Google Scholar] [CrossRef] - Ye, X.; Li, Y.; Li, X.; Zhang, Q. Factors influencing water level changes in China’s largest freshwater lake, Poyang Lake, in the past 50 years. Water Int.
**2014**, 39, 983–999. [Google Scholar] [CrossRef] - Li, M.; Li, Y. On the hydrodynamic behavior of the Changed River–Lake relationship in a large floodplain system, Poyang Lake (China). Water
**2020**, 12, 626. [Google Scholar] [CrossRef] - Shankman, D.; Keim, B.D.; Song, J. Flood frequency in China’s Poyang Lake region: Trends and teleconnections. Int. J. Climatol. J. R. Meteorol. Soc.
**2006**, 26, 1255–1266. [Google Scholar] [CrossRef] - Zhang, Z.; Chen, X.; Xu, C.Y.; Hong, Y.; Hardy, J.; Sun, Z. Examining the influence of river–lake interaction on the drought and water resources in the Poyang Lake basin. J. Hydrol.
**2015**, 522, 510–521. [Google Scholar] [CrossRef] - Li, Y.; Zhang, Q.; Werner, A.D.; Yao, J.; Ye, X. The influence of river-to-lake backflow on the hydrodynamics of a large floodplain lake system (Poyang Lake, China). Hydrol. Process.
**2017**, 31, 117–132. [Google Scholar] [CrossRef] - Wang, W.; Wang, Y.; Hou, J.; Ouyang, S. Flooding influences waterbird abundance at Poyang Lake, China. Waterbirds
**2019**, 42, 30–38. [Google Scholar] - Zhou, H.; Zhou, W.; Liu, Y.; Yuan, Y.; Huang, J.; Liu, Y. Meteorological drought migration in the Poyang Lake Basin, China: Switching among different climate modes. J. Hydrometeorol.
**2020**, 21, 415–431. [Google Scholar] [CrossRef] - Liu, Y.; Song, P.; Peng, J.; Fu, Q.; Dou, C. Recent increased frequency of drought events in Poyang Lake Basin, China: Climate change or anthropogenic effects. Hydro-Climatol. Var. Chang. IAHS Publ.
**2011**, 344, 99–104. [Google Scholar] - Jianping, B.; Pengxin, D.; Xiang, Z.; Sunyun, L.; Marani, M.; Yi, X. Flood coincidence analysis of Poyang Lake and Yangtze River: Risk and influencing factors. Stoch. Environ. Res. Risk Assess.
**2018**, 32, 879–891. [Google Scholar] [CrossRef] - Lai, X.; Jiang, J.; Liang, Q.; Huang, Q. Large-scale hydrodynamic modeling of the middle Yangtze River Basin with complex river–lake interactions. J. Hydrol.
**2013**, 492, 228–243. [Google Scholar] [CrossRef] - Nourani, V.; Baghanam, A.H.; Adamowski, J.; Kisi, O. Applications of hybrid wavelet–artificial intelligence models in hydrology: A review. J. Hydrol.
**2014**, 514, 358–377. [Google Scholar] [CrossRef] - Li, Y.; Zhang, Q.; Yao, J.; Werner, A.D.; Li, X. Hydrodynamic and hydrological modeling of the Poyang Lake catchment system in China. J. Hydrol. Eng.
**2014**, 19, 607–616. [Google Scholar] [CrossRef] - Altunkaynak, A.; Şen, Z. Fuzzy logic model of lake water level fluctuations in Lake Van, Turkey. Theor. Appl. Climatol.
**2007**, 90, 227–233. [Google Scholar] [CrossRef] - Buyukyildiz, M.; Tezel, G.; Yilmaz, V. Estimation of the change in lake water level by artificial intelligence methods. Water Resour. Manag.
**2014**, 28, 4747–4763. [Google Scholar] [CrossRef] - Çimen, M.; Kisi, O. Comparison of two different data-driven techniques in modeling lake level fluctuations in Turkey. J. Hydrol.
**2009**, 378, 253–262. [Google Scholar] [CrossRef] - Das, M.; Ghosh, S.K.; Chowdary, V.M.; Saikrishnaveni, A.; Sharma, R.K. A probabilistic nonlinear model for forecasting daily water level in reservoir. Water Resour. Manag.
**2016**, 30, 3107–3122. [Google Scholar] [CrossRef] - Kakahaji, H.; Banadaki, H.D.; Kakahaji, A.; Kakahaji, A. Prediction of Urmia Lake water-level fluctuations by using analytical, linear statistic and intelligent methods. Water Resour. Manag.
**2013**, 27, 4469–4492. [Google Scholar] [CrossRef] - Khan, M.S.; Coulibaly, P. Application of support vector machine in lake water level prediction. J. Hydrol. Eng.
**2006**, 11, 199–205. [Google Scholar] [CrossRef] - Zaji, A.H.; Bonakdari, H. Robustness lake water level prediction using the search heuristic-based artificial intelligence methods. ISH J. Hydraul. Eng.
**2019**, 25, 316–324. [Google Scholar] [CrossRef] - Güldal, V.; Tongal, H. Comparison of recurrent neural network, adaptive neuro-fuzzy inference system and stochastic models in Eğirdir Lake level forecasting. Water Resour. Manag.
**2010**, 24, 105–128. [Google Scholar] [CrossRef] - Zhu, S.; Lu, H.; Ptak, M.; Dai, J.; Ji, Q. Lake water-level fluctuation forecasting using machine learning models: A systematic review. Environ. Sci. Pollut. Res.
**2020**, 27, 44807–44819. [Google Scholar] [CrossRef] - Targ, S.; Almeida, D.; Lyman, K. Resnet in resnet: Generalizing residual architectures. arXiv
**2016**, arXiv:1603.08029. [Google Scholar] - Yu, Y.; Si, X.; Hu, C.; Zhang, J. A review of recurrent neural networks: LSTM cells and network architectures. Neural Comput.
**2019**, 31, 1235–1270. [Google Scholar] [CrossRef] - Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. Adv. Neural Inf. Process. Syst.
**2017**, 30, 5998–6008. [Google Scholar] - Liang, C.; Li, H.; Lei, M.; Du, Q. Dongting lake water level forecast and its relationship with the three gorges dam based on a long short-term memory network. Water
**2018**, 10, 1389. [Google Scholar] [CrossRef] - Hrnjica, B.; Bonacci, O. Lake level prediction using feed forward and recurrent neural networks. Water Resour. Manag.
**2019**, 33, 2471–2484. [Google Scholar] [CrossRef] - Guo, Y.; Lai, X.-J. Water level prediction of Lake Poyang based on long short term memory neural network. J. Lake Sci.
**2020**, 32, 865–876. [Google Scholar] - Dai, Z.; Yang, Z.; Yang, Y.; Carbonell, J.; Le, Q.V.; Salakhutdinov, R. Transformer-xl: Attentive language models beyond a fixed-length context. arXiv
**2019**, arXiv:1901.02860. [Google Scholar] - Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. An image is worth 16 × 16 words: Transformers for image recognition at scale. arXiv
**2020**, arXiv:2010.11929. [Google Scholar] - Yin, H.; Guo, Z.; Zhang, X.; Chen, J.; Zhang, Y. RR-Former: Rainfall-runoff modeling based on Transformer. J. Hydrol.
**2022**, 609, 127781. [Google Scholar] [CrossRef] - Dai, X.; Wan, R.; Yang, G. Non-stationary water-level fluctuation in China’s Poyang Lake and its interactions with Yangtze River. J. Geogr. Sci.
**2015**, 25, 274–288. [Google Scholar] [CrossRef] - Li, Y.L.; Zhang, Q.; Werner, A.D.; Yao, J. Investigating a complex lake-catchment-river system using artificial neural networks: Poyang Lake (China). Hydrol. Res.
**2015**, 46, 912–928. [Google Scholar] [CrossRef][Green Version] - Hui, F.; Xu, B.; Huang, H.; Yu, Q.; Gong, P. Modelling spatial-temporal change of Poyang Lake using multitemporal Landsat imagery. Int. J. Remote Sens.
**2008**, 29, 5767–5784. [Google Scholar] [CrossRef] - Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural Comput.
**1997**, 9, 1735–1780. [Google Scholar] [CrossRef] - Graves, A.; Schmidhuber, J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw.
**2005**, 18, 602–610. [Google Scholar] [CrossRef] - Kratzert, F.; Klotz, D.; Brenner, C.; Schulz, K.; Herrnegger, M. Rainfall–runoff modelling using long short-term memory (LSTM) networks. Hydrol. Earth Syst. Sci.
**2018**, 22, 6005–6022. [Google Scholar] [CrossRef] - Qi, Y.; Li, Q.; Karimian, H.; Liu, D. A hybrid model for spatiotemporal forecasting of PM
_{2.5}based on graph convolutional neural network and long short-term memory. Sci. Total Environ.**2019**, 664, 1–10. [Google Scholar] [CrossRef] - Qin, J.; Liang, J.; Chen, T.; Lei, X.; Kang, A. Simulating and Predicting of Hydrological Time Series Based on TensorFlow Deep Learning. Pol. J. Environ. Stud.
**2019**, 28, 795–802. [Google Scholar] [CrossRef] [PubMed] - Moriasi, D.N.; Arnold, J.G.; Van Liew, M.W.; Bingner, R.L.; Harmel, R.D.; Veith, T.L. Model evaluation guidelines for systematic quantification of accuracy in watershed simulations. Trans. ASABE
**2007**, 50, 885–900. [Google Scholar] [CrossRef] - Yu, M.; Liu, X.; Wood, P.; Wei, L.; Wang, G.; Zhang, J.; Li, Q. Simulation of Poyang Lake water levels and outflow under historical extreme hydrological scenarios. J. Flood Risk Manag.
**2022**, 15, e12806. [Google Scholar] [CrossRef] - Zhu, S.; Hrnjica, B.; Ptak, M.; Choiński, A.; Sivakumar, B. Forecasting of water level in multiple temperate lakes using machine learning models. J. Hydrol.
**2020**, 585, 124819. [Google Scholar] [CrossRef] - Ye, X.; Zhang, Q.; Bai, L.; Hu, Q. A modeling study of catchment discharge to Poyang Lake under future climate in China. Quat. Int.
**2011**, 244, 221–229. [Google Scholar] [CrossRef] - Shankman, D.; Keim, B.D.; Nakayama, T.; Li, R.; Wu, D.; Remington, W.C. Hydroclimate analysis of severe floods in China’s Poyang Lake region. Earth Interact.
**2012**, 16, 1–16. [Google Scholar] [CrossRef] - Huang, S.; Xia, J.; Zeng, S.; Wang, Y.; She, D. Effect of Three Gorges Dam on Poyang Lake water level at daily scale based on machine learning. J. Geogr. Sci.
**2021**, 31, 1598–1614. [Google Scholar] [CrossRef]

**Figure 1.**Poyang Lake area (

**A**) and rivers, hydrological stations and gauging stations in Poyang Lake (

**B**).

**Figure 2.**The Transformer Model architecture [29].

**Figure 9.**Effects of lead time on model accuracy in Xingzi Station. The red vertical line split the results into two groups to indicate the reason why the difference gap suddenly increased from day 7 to day 15: the left group are the prediction results for the coming 1 day to 7 days correspondingly; the right group are the prediction results for day 15 and day 30 respectively.

Data Description | Gauging Station | Duration | Application |
---|---|---|---|

Meteorological data | 14 national meteorological stations | 1960–2013 | Input |

Catchment inflow | Waizhou (Ganjiang River) | 1960–2013 | Input |

Lijiadu (Fuhe River) | 1960–2013 | Input | |

Meigang (Xinjiang River) | 1960–2013 | Input | |

Dufengkeng (Raohe River) Hushan (Raohe River) | 1960–2013 1960–2013 | Input Input | |

Wanjiabu (Xiushui River) | 1960–2013 | Input | |

Yangtze River water level | Hukou | 1960–2013 | Input |

Poyang lake water level | Xingzi | 1960–2013 | Output |

Duchang | 1960–2013 | Output | |

Tangyin | 1960–2013 | Output | |

Kangshan | 1960–2013 | Output |

Stage | Index | Xingzi | Duchang | Tangyin | Kangshan |
---|---|---|---|---|---|

Scenario 1 | |||||

Training | R^{2} | 0.91 | 0.91 | 0.89 | 0.84 |

RMSE | 1.12 | 1.02 | 0.96 | 0.94 | |

NSE | 0.91 | 0.91 | 0.89 | 0.83 | |

Testing | R^{2} | 0.84 | 0.84 | 0.81 | 0.76 |

RMSE | 1.60 | 1.63 | 1.66 | 1.28 | |

NSE | 0.79 | 0.76 | 0.69 | 0.66 | |

Scenario 2 | |||||

Training | R^{2} | 1.00 | 1.00 | 1.00 | 1.00 |

RMSE | 0.16 | 0.15 | 0.14 | 0.14 | |

NSE | 1.00 | 1.00 | 1.00 | 1.00 | |

Testing | R^{2} | 0.99 | 0.97 | 0.97 | 0.98 |

RMSE | 0.40 | 0.70 | 0.43 | 0.26 | |

NSE | 0.99 | 0.94 | 0.96 | 0.98 |

Study Area | Temporal Scale | Model Input | Training Period | Testing Period | Performance Evaluation Criteria | Reference | |
---|---|---|---|---|---|---|---|

LSTM | Poyang Lake | Daily | River flows | 1956–1980 | 1981–2000 | R^{2}, RMSE, NSE | Guo and Lai 2020 [32] |

LSTM, SVM | Dongting Lake | Daily | Water level, water inflow, rainfall | 2003–2012 | 2011–2013 | R^{2}, RMSE | Liang 2018 [30] |

FFNN, LSTM | Vrana Lake | Monthly | Water level | 1978–2016 | 12 months | R, RMSE | Hrnjica and Bonacci 2019 [31] |

FFNN, DL | 69 lakes in Poland | Monthly | Water level | 1984–2014 (2/3rd) | 1984–2014 (1/3rd) | R, RMSE | Zhu 2020 [46] |

Transformer | Poyang Lake | Daily and monthly | Water level, meteorological factors | 1960–2000 | 2001–2013 | R^{2}, RMSE, NSE | This study |

Station | Model | Training Stage | Testing Stage | ||||
---|---|---|---|---|---|---|---|

R_{2} | RMSE/m | NSE | R_{2} | RMSE/m | NSE | ||

Xingzi | LSTM | 0.98 | 0.81 | 0.94 | 0.98 | 0.55 | 0.97 |

Transformer | 1.00 | 0.16 | 1.00 | 0.99 | 0.40 | 0.99 | |

Duchang | LSTM | 0.92 | 0.91 | 0.85 | 0.97 | 0.79 | 0.93 |

Transformer | 1.00 | 0.15 | 1.00 | 0.97 | 0.70 | 0.94 | |

Tangyin | LSTM | 0.99 | 0.76 | 0.95 | 0.96 | 0.48 | 0.95 |

Transformer | 1.00 | 0.14 | 1.00 | 0.97 | 0.43 | 0.96 | |

LSTM | 0.97 | 0.77 | 0.89 | 0.96 | 0.37 | 0.95 | |

Kangshang | Transformer | 1.00 | 0.14 | 1.00 | 0.98 | 0.26 | 0.98 |

Period | RMSE Contribution Rate (%) | NSE Contribution Rate (%) | ||||||
---|---|---|---|---|---|---|---|---|

Xingzi | Duchang | Tangyin | Kangshan | Xingzi | Duchang | Tangyin | Kangshan | |

Dry period | 38.0 | 2.8 | 58.8 | 65.0 | 50.1 | 8.6 | 223.5 | 81.4 |

Rising period | 70.8 | 62.4 | 79.3 | 72.5 | 19.1 | 24.8 | 36.9 | 23.7 |

Flood period | 85.2 | 84.8 | 79.7 | 83.9 | 514.6 | 506.6 | 507.8 | 803.1 |

Receding period | 92.0 | 80.3 | 82.6 | 86.4 | 505.5 | 550.6 | 477.0 | 274.1 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Xu, J.; Fan, H.; Luo, M.; Li, P.; Jeong, T.; Xu, L. Transformer Based Water Level Prediction in Poyang Lake, China. *Water* **2023**, *15*, 576.
https://doi.org/10.3390/w15030576

**AMA Style**

Xu J, Fan H, Luo M, Li P, Jeong T, Xu L. Transformer Based Water Level Prediction in Poyang Lake, China. *Water*. 2023; 15(3):576.
https://doi.org/10.3390/w15030576

**Chicago/Turabian Style**

Xu, Jiaxing, Hongxiang Fan, Minghan Luo, Piji Li, Taeseop Jeong, and Ligang Xu. 2023. "Transformer Based Water Level Prediction in Poyang Lake, China" *Water* 15, no. 3: 576.
https://doi.org/10.3390/w15030576