Landslide Deformation Prediction Based on a GNSS Time Series Analysis and Recurrent Neural Network Model

: The prediction of landslide displacement is a challenging and essential task. It is thus very important to choose a suitable displacement prediction model. This paper develops a novel Attention Mechanism with Long Short Time Memory Neural Network (AMLSTM NN) model based on Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN) landslide displacement prediction. The CEEMDAN method is implemented to ingest landslide Global Navigation Satellite System (GNSS) time series. The AMLSTM algorithm is then used to realize prediction work, jointly with multiple impact factors. The Baishuihe landslide is adopted to illustrate the capabilities of the model. The results show that the CEEMDAN-AMLSTM model achieves competitive accuracy and has significant potential for landslide displacement prediction.


Introduction
Landslide disaster is one of the crucial topics in geological research [1]. The sustainable development of economies and society is seriously threatened as a result of landslide disasters [2]. Reliable early warning systems are a reasonable approach for landslide risk reduction [3,4]. The mechanisms analysis and prediction of landslide movements are the key components of landslide early warning [5][6][7]. Therefore, it is judicious to carry out landslide displacement prediction.
Landslide displacement prediction models can be divided into two categories: physical models and numerical models [8,9]. Traditional physical models provide a physical explanation for the prediction work according to geological theory [10]. Saito established a three-stage theory of landslide creep failure in 1968 [11,12], and Hoek proposed the extension line method to predict the time-displacement curve of Chilean landslides in 1977 [13]. However, physical models are deficient in their ability to meet the demands of dynamic large landslide prediction [14][15][16]. With the rapid development of mathematical statistical theory and intelligent algorithms, numerical models have become more popular [5]. Numerical models fully consider the complexity and nonlinearity of the landslide evolution process and have higher prediction accuracy [5,17].
Advances in machine learning provide a powerful tool for numerical landslide model research. Zhou et al. [17] used kernel extreme learning for landslide displacement prediction. Zhu et al. [18] proposed a least squares support vector model and applied it to prediction of the Shuping landslide. Among them, Recurrent Neural Networks (RNNs) have particular advantages in dealing with sequential data [19,20]. Different from other neural networks, RNNs are the deepest algorithms [21], and they can effectively process data information with higher dimensions [22]. As a variant of RNNs, Long Short Term Memory (LSTM) networks perform better at storing and transferring historical information than RNNs [23][24][25][26]. The utility of the LSTM in landslide research has been confirmed by many scholars [27][28][29][30]. Thus, we choose an LSTM network for landslide displacement prediction in this paper.
The Attention Mechanism (AM) is currently a powerful deep learning toolkit [31]. AM is similar to the human visual observation mechanism that can transfer key information from the input information [32]. AM has been successfully applied in several tasks, such as natural language processing [31], translation [33], and image recognition [34]. Li et al. [35] added the Attention Mechanism to the LSTM model and successfully realized the prediction of personal mobility. Ding et al. [36] proposed a spatio-temporal attention LSTM model for flood forecasting. Thus, we incorporate an Attention Mechanism with an LSTM neural network to capture significant variation and improve the model's performance.
Therefore, a novel model based on time series analysis and Attention Mechanism with Long Short Term Memory (AMLSTM) was proposed to predict landslide displacement. The Baishuihe landslide in China, Hubei province, is utilized for the experiment area. First, we use the Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN) algorithm to divide the total displacement into the trend term, the periodic term, and the residual term. By analyzing the corresponding relationship between displacement and external factors, a multiple factors AMLSTM model, is applied to predict the displacement, and it is compared with a further four machine learning models. A series of contrastive analyses are conducted to evaluate the performance of all of the models. The results indicate that the proposed CEEMDAN-AMLSTM model performs best in the experiment.

Landslide Evolution Analysis
The evolution of landslides is the result of the interaction of geological conditions and external factors [37]. The non-linear and non-stationary landslide displacement series are particularly complex and changeable. Therefore, it is necessary to decompose the landslide time series and forecast each component separately. The corresponding time series of the landslide displacement can be expressed by the additive model: where is the cumulative displacement, T is the trend term, is the period term, and is the residual term.

Decomposition of Displacement Time Series
Many approaches have been recognized as being powerful tools for decomposing landslide displacement time series, and they include moving average [38], wavelet analysis [39], Variational Mode Decomposition (VMD) [40], and Empirical Mode Decomposition (EMD) [41]. The EMD method is an adaptive method that is used to analyze nonlinear signals [42]. However, the model mixing problem constitutes an obstacle when using EMD. To address this problem, the Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN) method has been proposed in recent years [43]. Compared to the more commonly used EMD method, it has a better separation effect and is noise free. It has many applications in the fields of biological signal processing [44] and engineering [45], but its application in the geological field still needs to be explored.
The CEEMDAN decomposes the complex signal into a finite number of Intrinsic Mode Functions (IMFs). The basic process of the CEEMDAN is as follows [46]: 1. White Gaussian noises is added onto the lines of EEMD. The first IMF can be expressed as: where n is the number of decomposition, x is the original signal, is a fixed coefficient, is the noise, and E(·) is the decomposition operator. 2. The first residual, , is calculated: 3. For k= 2,3…, K, the and the kth residual can be calculated by: 4. The process is calculated until the last residual, R, does not have more than two extrema points; the original signal can be expressed as:

LSTM
Long Short Time Memory (LSTM) was proposed by Hochreiter and Schmidhuber in 1997 [23]. The LSTM can learn information through a well-designed structure called a "gate". The gate can store and control the flow of information so that the state of the previous time step can be transferred to the next time step. The LSTM algorithm has three gates-update gate, forget gate, and output gate-to protect and control the cell state explosion in training [25]. The internal structure of the unit memory is as shown in Figure 1. The ⨂ represents the element-wise product and ⨁ is the element-plus product. The forget gate represents how much of the previous moment unit sate, ct−1 , is retained by the current moment, ct. The input gate determines how much of the current moment input, xt, is saved in the unit state, ct. The output gate controls how much of the unit state, ct, is transferred to the output value, ht, of the LSTM. Equations (7)(8)(9)(10)(11)(12) show the calculation process of LSTM： where ， , and are gating vectors that respectively store the forgotten, updated, and output information of the storage unit memory; is the vector for the cell state; is the hidden state vector; is the sigmoid function; and is the input vector. , , , and are linear transformation matrices whose parameters need to be learned, and , , , and are corresponding bias vectors.
Through the connection of several unit memories, the information flow can be transferred as shown in Figure 2.

Attention Mechanism
The Attention Mechanism is based on the visual Attention Mechanism found in human observation [32]. This mechanism helps the model focus on the salient information. The schematic of the Attention Mechanism layer is illustrated in Figure 3. The purpose of the attention layer is to enable the model to pay more attention to the significant information. Raffel et al. [47] proposed a reduced Feed-Forward Attention model, which was calculated as follows: s = * where the score is the attention score, a is the state vector, v is the learnable function, w is the weight, and s is the context vector.

Attention Mechanism-LSTM Model
Based on the previous discussion, this paper applied the Attention Mechanism with LSTM (AMLSTM) model for landslide displacement prediction. The AMLSTM model includes an input vector, LSTM hidden layers, an attention layer, a fully connected layer, and output predicted values. The architecture of the AMLSTM model is shown in Figure  4.

Prediction Process with the Proposed Model
The basic flow of the proposed CEEMDAN-AMLSTM model is shown in Figure 5. Firstly, the landslide cumulative displacement is decomposed into three components: the trend term, the periodic term, and the residual term. The three terms are then predicted separately. The trend displacement is expressed as a monotone increasing function under the influence of internal geological factors. The prediction of the trend term can be carried out by fitting the growth curve with the univariate AMLSTM model. During the construction of the model, the displacement time series is put into the model only. The periodic displacement fluctuates under the influence of two external triggers: rainfall and reservoir water level. Therefore, a multivariable AMLSTM model is established and used to predict the periodic term. Three time series, the historical periodic displacement, rainfall, and reservoir water level are put into the model. Furthermore, the residual displacement affected by random factors shows smooth fluctuation function. The univariate AMLSTM model is adopted for the prediction work.
In the prediction experiments, the majority dataset is used to train the model. The original time series should be normalized and reshaped to meet the requirements of the model. After the AMLSTM model is constructed, the prediction ability is tested and demonstrated with the rest of the dataset.
Ultimately, the cumulative prediction displacement is obtained by adding the trend, the periodic, and the residual prediction displacements. The prediction results should be compared with the actual value to verity the performance.

Evaluation of Model Accuracy
Quantitative analysis were carried out to access the performance of the model. Three criterions-Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and R 2 -were employed to evaluate the prediction work. These metrics are described as follows: where is the measured value, is the prediction value, and is the average value.

Study Area
The experimental area is located in Baishuihe, Zigui County, the Three Gorges Reservoir area of the Yangtze River in China. The Baishuihe landslide is located on the south bank of the Yangtze River, with a longitude of 110°32'09" and a latitude of 31°01'34" (Figure 6a). The slope is located on the south bank of the Yangtze River, spreading towards the Yangtze River in a ladder shape. The elevation of the back edge of the landslide is 410 m, bounded by the rock-soil boundary, and the front edge is about 70 m. It has been submerged below the reservoir water level. The east and west sides are bounded by bedrock ridges, and the overall slope is about 30°. The length of the north-south direction is 600 m, the width of the east-west direction is 700 m, the average thickness of the sliding body is about 30 m, and the volume is 1.26×10 7 m 3 . Six Global Navigation Satellite System (GNSS) deformation monitoring points were installed on the surface of the landslide to form three longitudinal monitoring profiles (Figure 6b). The displacement was monitored once a month. Figure 7 shows the calculated displacement results from December 2006 to December 2012.    are considered to be the trigger factors of the Baishuihe landslide, leading to the occurrence of the periodic term displacement.

GNSS Time Series Analysis
According to landslide analysis theory, the cumulative displacement can be decomposed into trend displacement, periodic displacement, and residual displacement using the CEEMDAN algorithm. The results are as follows (Figure 9):

Trend Displacement Prediction
Trend displacement is driven by geological conditions. Therefore, the univariate AM-LSTM NN model is used to predict the trend displacement. In order to verify the validity of the proposed model, the experiment will be benchmarked with LSTM, Random Forest(RF), RNN, and Support Vector Machine(SVM). The prediction results of the test dataset are shown in Figure 10. It can be seen in Figure 10 that the trend displacement of the ZG118 and XD01 points represent a smooth monotonically properties. The prediction work by the SVM shows the worst, and the prediction values of the AMLSTM, LSTM, RNN, and RF models show high agreement with the measured true value. The relative error analysis in Table 1 indicates that the AMLSTM, LSTM, and RF have excellent performance in trend term prediction work.

. Periodic Displacement Prediction
Periodic term is a key component for displacement prediction. According to the analysis in Section 4.1, the external periodic rainfall and reservoir water level both have an important influence. In this section, the periodic displacement will be predicted by the multivariate AMLSTM, and the multivariate LSTM, the SVM, the RF, and the RNN are used as benchmarks. The predictive periodic displacements by the five models are shown in Figure 11 and Table 2.   As shown in Figure 11, the predictions of the AMLSTM and LSTM methods are clearly better than the others, and the quantitative analysis suggest that the AMLSTM achieved the best performance, along with RMSE, MSE, and R 2 , in periodic displacement prediction.

Residual Displacement Prediction
Traditionally, the residual term can be regarded as the noise, which is removed during the decomposition procedure. Throughout the test, the residual term does not belong to the white noise. Therefore, the prediction work of this term is necessary. In this experiment, the univariate AMLSTM, LSTM, SVM, RF, and RNN models are used to predict the residual displacement prediction.
Compared with the trend and the periodic term, the residual term is harder to adopt in a model because of its random characteristic. As shown in Figure 12 and Table 3, the AMLSTM offers a better prediction effect than the other four models.

. Total Displacement Prediction
The predicted cumulative displacements can be obtained by taking the sum of the trend, period, and residual displacements. The results are shown in Figure 13 and Table  4.   The results show that, although some of the prediction values slightly deviate from the real measured data, the AMLSTM model shows the best performance, because this model not only considers multiple external factors, but also optimizes the LSTM algorithm by adding an attention layer. It can better reflect the response relationship between displacement and trigger factors. Moreover, the cumulative displacements are predicted badly by the SVM and RF models.
From a quantitative point of view, the RMSE and MAE of the AMLSTM model are lower than the LSTM, RNN, SVM, and RF models. These results reveal that the AMLSTM shows the most stable prediction performance. Secondly, the R 2 of the AMLSTM are higher than the others. The results indicate that the AMLSTM model has done the best accuracy prediction work. Therefore, the superiority of the AMLSTM can be proved.

Conclusions
The traditional landslide prediction model directly deletes the residual items. Moreover, most classic deep learning prediction models do not highlight the impact of important information on the results, so they cannot accurately predict the displacement. This paper used the CEEMDAN and the Attention Mechanism, combined with the LSTM NN to establish a dynamic prediction model for landslide displacement prediction. To corroborate its feasibility and applicability, the proposed model was applied to the Baishuihe landslide area, and joint multiple impact factors were considered here for prediction. By comparing to the prediction effects of other models, the prediction accuracy demonstrated a competitive performance. The results strongly suggest the effectiveness and feasibility of the AMLSTM model in landslide displacement prediction. This novel CEEMDANAM-LSTM strategy can be recommended to other landslide prediction works and has great potential in landslide risk assessment.  Acknowledgments: The dataset we used in this paper includes the GNSS time series, rainfall and reservoir water level data set of Baishuihe landslide provided by Chinese National Cryosphere Desert Data Center(http://www.crensed.ac.cn/portal/). The authors acknowledge Google Earth for providing the map and Origin software. Thanks to the editor Aguero Gui and the anonymous reviewers.

Conflicts of Interest:
The authors declare no conflict of interest.