Landslide Displacement Prediction Combining LSTM and SVR Algorithms: A Case Study of Shengjibao Landslide from the Three Gorges Reservoir Area

Abstract

Displacement predictions are essential to landslide early warning systems establishment. Most existing prediction methods are focused on finding an individual model that provides a better result. However, the limitation of generalization that is inherent in all models makes it difficult for an individual model to predict different cases accurately. In this study, a novel coupled method was proposed, combining the long short-term memory (LSTM) neural networks and support vector regression (SVR) algorithm with optimal weight. The Shengjibao landslide in the Three Gorges Reservoir area was taken as a case study. At first, the moving average method was used to decompose the cumulative displacement into two components: trend and periodic terms. Single-factor models based on LSTM neural networks and SVR algorithms were used to predict the trend terms of displacement, respectively. Multi-factors LSTM and SVR models were used to predict the periodic terms of displacement. Precipitation, reservoir water level, and previous displacement are considered as the candidate factors for inputs in the models. Additionally, ensemble models based on the SVR algorithm are used to predict the optimal weight to combine the results of the LSTM and SVR models. The results show that the LSTM models display better performance than SVR models; the ensemble model with optimal weight outperforms other models. The prediction accuracy can be further improved by also considering results from multiple models.

1. Introduction

Since the impoundment of the Three Gorges Reservoir area (TGRA) in June 2003, many new landslides have occurred and some existing landslides have been reactive. The Shanshucao landslide took place on 2 September 2014, interrupting the power supply for the town and damaged around $20\times {10}^4$ m² area of orange planting [1]. The Honyanzi landslide took place on 24 June 2015, initiating a reservoir tsunami that resulted in two deaths and significant damage to shipping facilities [2]. As an essential component for landslide early warning systems, displacement predictions become ever more important in the TGRA.

In general, landslide displacement predictions involve the following steps: (i) the decomposition of total displacement, (ii) the selection of candidate factors, (iii) the establishment of predicting models, and (iv) the evaluation of prediction results. For the decomposition of total displacement (step i), the moving average method (MAM) [3], the empirical mode decomposition [4], the wavelet analysis [5,6], and the variational mode decomposition [7] have been proposed to obtain sub-sequences of total displacement. The obtained sub-sequences are usually defined as trend terms and periodic terms. The trend term of the original total displacement represents the component mainly caused by the state factors and the periodic term represents the component mainly caused by the influence factors. For the selection of input factors (step ii), the mutual information, the Pearson cross-correlation coefficient [4], and the grey correlation analysis method (GCAM) [8,9] have been used. In the TGRA, the landslide deformation is the consequence of the state and the influence factors, and these controlling factors are very diverse [10,11,12]. Thus, it is necessary to select factors before establishing models.

Since Saito [13] proposed the empirical formula for landslide prediction, many approaches to landslide displacement prediction have been developed [14], including those based on creep theory, statistical theory, time-series analysis, and artificial intelligence methods [15,16]. For artificial intelligence methods that have been applied in landslide displacement, the Gaussian process model [17], the neural network model [18,19,20], the Support Vector Machine (SVM) model [9,21,22], the Extreme Learning Machine (ELM) model [8,23,24], and the LSTM model [4,25] have become popular. Moreover, these approaches can also be divided into two categories which are univariate forecasting methods and multivariate forecasting methods. Univariate forecasting methods only consider previous displacement, while the multivariate forecasting methods consider triggering factor as well.

The current studies mainly focus on choosing single prediction models from the numerous artificial intelligence methods. Since some models many be only suitable for specific landslide, a specific type of landslides, or a set of landslides with specific local geological conditions, periods, and variations, it is necessary to study how to accurately and comprehensively different models predict landslides by considering advantages from different models [15,16]. Moreover, the linear combination is perhaps most well developed and mature [26]. Therefore, this study mainly focuses on an Ensemble model with the weight coefficients based on linear combination theory.

In this study, the accumulated displacements of the selected monitoring points on the Shengjibao landslide were decomposed into trend term and periodic term using the MAM. The trend terms were predicted using the univariate forecasting methods based on the LSTM and SVR algorithm, respectively. The periodic terms were predicted by the multivariate forecasting methods based on the LSTM and SVR algorithm with analyzing the relationship between landslide displacement and the triggering factors separately. The predictive total displacements of each model were obtained by adding their predictive trend and periodic terms. The final total predicting displacements were obtained by adding the predictive total displacements of both models with the weight coefficients obtained from the prediction of the optimal weight.

2. Methodology

2.1. Decomposition of the Displacement Time Series into Trend and Periodic Terms

Landslide displacement was modeled using two components: a trend component and a periodic component [3,27]. In the long-term, the deformation trend is controlled by internal geological conditions, such as lithology, geological structure, and progressive weathering. The short-term deformation in the TGRA is influenced by two main external triggering factors: rainfall and reservoir water level. The short-term displacement represents the periodic component in the model [25]. The accumulated displacement time series can be decomposed as follows:

$$S(t)=\varphi (t)+\eta (t)$$

(1)

where $t$ is the time step, $S(t)$ is the accumulated displacement, $\varphi (t)$ is the trend term, and $\eta (t)$ is the periodic term.

2.2. Moving Average Methods

The decomposition of the trend and periodic terms forms the basis of the model. If the period of the time series and influencing factors can be determined accurately, the fluctuation can be smoothed by the moving average method, and the trend term of the displacement can be extracted. The function of the single moving average method is shown as follows:

$${\overline{S}}_t=\frac{S_t+S_{t-1}+\cdots +S_{t-n+1}}{n}(t=n,n+1,\cdots ,T)$$

(2)

where $\overline{S_t}$ is the trend term of the displacement at the time step $t$, $S_t$ is the accumulative displacement of the landslide at the time step $t$, $n$ is the moving average cycle. Considering the scheduling cycle of the TGRA, the moving average cycle was selected as 12 months [21].

2.3. Long Short-Term Memory Neural Networks

Recurrent neural networks (RNNs) (Figure 1) have been successfully applied to a variety of problems, such as speech recognition, language modeling, translation, and image captioning [28,29]. RNNs can utilize historical information and apply it to the current output, which allows for feedback in the networks [30]. The recurrent connections in the RNN implement passing information from one step of the network (time step $t-1$) to the next one (time step $t$). For a given input sequence $x=(x_1,x_2,\cdots ,x_T)$, the hidden state vector sequence is $h=(h_1,h_2,\cdots ,h_T)$, and the output sequence is $y=(y_1,y_2,\cdots ,y_T)$. The output sequence y can be obtained by iterating the following equations from time $t=1$ to $T$:

$$h_t=\Re (W_{xh}{x}_t+W_{hh}{h}_t{}_{-1}+b_h)$$

(3)

$$y_t=W_{hy}{h}_t+b_y$$

(4)

where $x_t$ and $y_t$ are the input and output at time step $t$; $h_t$ and $h_t{}_{-1}$ are the hidden states at time step t and $t-1$, respectively; $T$ is the prediction period; $W_{xh}$ is the weight matrix between input and hidden vectors; $W_{hh}$ represents the weight matrix between different time steps of the hidden vectors; $W_{hy}$ represents the weight matrix that connects the hidden information to the output vectors; $b_h$ and $b_y$ are the corresponding bias vectors of $W_{hh}$ and $W_{hy}$, respectively; and $\Re$ is the nonlinear activation function for hidden nodes.

However, in practice, RNNs are not capable of handling a problem as long-term dependencies because of the vanishing gradient or exploding gradient problems [31]. The LSTM neural networks are a special type of RNN, which are capable of learning long-term dependencies [32]. LSTMs replace every single unit in an RNN with a memory block (Figure 2). There are three types of gate functions in this memory block [33]. The input gates control the flow of input activations into the memory cell, and the output gates control the output flow into other cells or as the final results [34]. The forget gate controls whether the value from the previous moment is forgotten or remembered. The combination of the three types of gates controls the output state. The key difference between RNN and LSTM lies in the formulation of the hidden vector ($h_t$). The hidden vector ($h_t$) in LSTM neural network can be obtained as follows:

$$\left\{\begin{array}{c}{i}_t=\sigma (W_{xi}{x}_t+W_{hi}{h}_t{}_{-1}+W_{ci}{c}_t{}_{-1}+b_i)\\ f_t=\sigma (W_{xf}{x}_t+W_{hf}{h}_t{}_{-1}+W_{cf}{c}_t{}_{-1}+b_f)\\ c_t=f_t{c}_t{}_{-1}+i_t\tanh (W_{xc}{x}_t+W_{hc}{h}_t{}_{-1}+b_c)\\ o_t=\sigma (W_{xo}{x}_t+W_{ho}{h}_t{}_{-1}+W_{co}{c}_t{}_{-1}+b_o)\\ h_t=o_t\tanh (c_t)\end{array}\right.$$

(5)

where $i_t$, $f_t$, $o_t$, and $c_t$ are the values of the input gate, the forget gate, the output gate, and the memory cell in the memory block; $b_i$, $b_f$, $b_o$, and $b_c$ are their corresponding bias values; $\sigma$ is the sigmoid function; $W_x$ represents the weights between input nodes and hidden nodes; $W_h$ represents the weights between hidden nodes and memory cell; $W_c$ represents the weights that connect memory cell to output nodes. The output $y_t$ can then be obtained by Equation (2). The output sequence $y=(y_1,y_2,\cdots ,y_T)$ will be obtained by iterating Equations (2) to (7) from times $t=1$ to $T$.

2.4. Support Vector Regression

The support vector regression (SVR) model was proposed by Vapnik et al. in 1995 and has been widely used in nonlinear problem solving [35,36]. In SVR models, sample data are divided into a fitting sample and predicting sample. The fitting sample is then mapped to a high-dimensional feature space. The best fitting effect is obtained in the space of the optimal decision function model and the validation sample is used to validate the analytical model results. The regression function for SVR is:

$$f(x)=〈W\cdot \mathsf{\Phi }(x)〉+b$$

(6)

where $W$ is the weight vector, $\mathsf{\Phi }(x)$ is a nonlinear mapping from the input space to the output space, $b$ is bias. Transforming the estimation function into a function minimization problem by the $\epsilon$ insensitive loss function:

$$R_{\min }=\frac{1}{2}{‖W‖}^2+C{\displaystyle \sum _{i=1}^m({\xi }_i+{\xi }_i{}^{\ast })}$$

(7)

The constraint conditions are as follows:

$$\left\{\begin{array}{l}{W}^T\mathsf{\Phi }(x_i)+b_i-y_i\le \epsilon +{\xi }_i;\\ y_i-W^T\mathsf{\Phi }(x_i)-b_i\le \epsilon +{\xi }_i{}^{\ast };\\ {\xi }_i,{\xi }_i{}^{\ast }\ge 0,i=1,\dots ,l.\end{array}\right.$$

(8)

where $C$ is the penalty factor; ${\xi }_i$ and ${\xi }_i{}^{\ast }$ are relaxation factors. The Lagrange multiplier is introduced at last, and converted into an equivalent dual problem based on the Wolf duality theory:

$$\min \frac{1}{2}{({\alpha }_i-{\alpha }_i{}^{\ast })}^{T}Q({\alpha }_i-{\alpha }_i{}^{\ast })+\epsilon {\displaystyle \sum _{i=1}^l({\alpha }_i+{\alpha }_i{}^{\ast })}+{\displaystyle \sum _{i=1}^l{y}_i({\alpha }_i-{\alpha }_i{}^{\ast })}$$

(9)

The constraint conditions are expressed in Equation (10):

$$\left\{\begin{array}{l}{\displaystyle \sum _{i=1}^l({\alpha }_i+{\alpha }_i{}^{\ast })}=0\\ 0\le {\alpha }_i,{\alpha }_i{}^{\ast }\le C,I=1,2\dots l.\end{array}\right.$$

(10)

where $Q_{ij}=k(x_i,x_j)=\mathsf{\Phi }{(x_i)}^T\mathsf{\Phi }(x_j)$. The SVR prediction model can be obtained by quadratic programming:

$$f(x,{\alpha }_i{}^{\ast },{\alpha }_i)={\displaystyle \sum _{i=1}^l({\alpha }_i{}^{\ast }-{\alpha }_i)}K(x_i,x)+b$$

(11)

where $K(x_i,x)$ is the kernel function of SVR.

Linear kernel, polynomial kernel, radial basis kernel function (RBF), and sigmoid function [37] are four commonly used kernel functions for SVR algorithm. With a wide convergence domain, the RBF is widely used [21,38,39]. Therefore, the RBF is adopted as the kernel function for the SVR model in this paper.

2.5. Ensemble Model

We used ensemble models to reduce the instability of a single prediction model. The core idea of ensemble model is to assign different weight coefficients to the predictive results of different predicting models, to extract effective information from each model system, utilize the forecasting results of each model synthetically, disperse forecasting risks and errors, and improve the accuracy and reliability of predicting [15].

The displacement forecasts based on the LSTM model and the SVR model at the time step $t$ were $L(t)$ and $S(t)$, the total displacement forecast of the ensemble algorithm was $E(t)$, the original displacement in the validation dataset at the time step $t$ was $O(t)$.

$$E(t)=WL(t)+(1-W)S(t),t=1,2,\dots ,N$$

(12)

where $W$ is the dynamic weight coefficient obtained from the prediction of optimal weight.

2.6. Reliability Evaluation of the Model

After tuning the hyperparameters of the LSTM and SVR models with the fitting datasets, the training datasets and optimal hyperparameters were adopted to train the LSTM models and SVR models. To verify the prediction accuracy of the LSTM models, SVR models and the ensemble models, the root mean square error (RMSE) (Equation (13)) and mean absolute percentage error (MAPE) (Equation (14)) were calculated. The overall process is shown in Figure 3.

$$RSME=\sqrt{\frac{1}{N}{{\displaystyle \sum _{i=1}^N({\widehat{x}}_i-x_i)}}^2}$$

(13)

$$MAPE=\frac{1}{N}{\displaystyle \sum _{i=1}^N\left|x_i-\frac{{\widehat{x}}_i}{x_i}\right|}$$

(14)

where $x_i$ is the measured value; ${\widehat{x}}_i$ is the predicted value; $N$ is the number of predicted values.

3. Study Site

3.1. Shengjibao Landslide

The Shengjibao landslide is located on the South Bank of the Yangtze River in Fengjie County (Figure 4). The Yangtze River flows through the front of the landslide area from S72W. The overall landform type of the landslide is structure-erosion and denudation low-mountain valley. The Yangtze River Valley extends eastward in this section. The width of the river in this area is 350–400 m; the valley is V-shaped with asymmetrical banks. The topographic slope of the northern bank is steep, generally 30–50 degrees. The southern bank is a consequent slope with an overall slope of 15–20 degrees. The slope surface is ladder-like and gullies on the slope are well-developed. The slope gradient at the bottom of the gully is similar to that on the slope surface. The front elevation of the Shengjibao landslide is 81–85 m, the escarpment elevation is 370 m. The gradient range is 15–20 degrees and the longitudinal length is 1500 m. The lateral average width is about 1160 m, the average thickness is 25 m, the total area of landslide is about 2.45 km², and the total volume of the landslide is 3998 × 10⁴ m³.

3.2. Landslide Activity

The initial monitoring network of Shengjibao landslide was established in March 2007, which includes 16 horizontal displacement monitoring points (Figure 5). In May 2007, Shengjibao landslide began to deform after the first 156 m impoundment of the TGRA, which was affected by the drop of the reservoir water level and seasonal rainfall. Movement has been small-scale controlled by the topography and generally occurred in the loose soil of the surface layer.

The main manifestations are the deformation of some houses, discontinuous cracks on the ground, and ground tilt (Figure 6). The main slide direction of the landslide is 348 degrees and a typical schematic geological cross-section is shown in Figure 7. Based on data collected from March 2007 to December 2015 (Figure 8), four stages of surface displacement along the II-II’ profile could be identified (

Table 1. Deformation stages of the Shengjibao landslide.

Deformation Stage	Time Range	Remarks
1	March 2007–May 2007	Landslide deformation starting stage.
2	June 2007–October 2009	Increased deformation, the first fluctuation of 172 m.
3	November 2009–August 2011	Steady deformation stage, the first fluctuation of 175 m.
4	September 2011–December 2015	Deformation has periodic and trend characteristics except August and September 2013.

).

According to the reservoir water level data, the Shengjibao landslide did not show an obvious displacement during the period from March 2007 to May 2007, which was the discharge period after the first 156 m experimental impoundment of the TGRA. The operation period of the low water level (145 m) in the flood season of the reservoir was from 1 June 2007 to 20 September 2007. Reservoir water level fluctuations were not significant during this period. However, landslide deformation, characterized by increased displacement of the monitoring points, was obvious during this period, with rainfall being the dominant control on landslide deformation. From October 2007 to January, 2008, the reservoir water level rose again to 156 m. But the displacement of landslide monitoring points did not change significantly during this period.

The displacement data showed a sharp increase in surface displacement in August and September 2013 (Figure 8). However, a similar pattern was not detected in August and September 2014, despite comparable rainfall levels. According to the precipitation data showed in Figure 9, there was a rainfall pattern that lasted for 15 days during the period in 2013, and a rainfall pattern only lasted for 9 days in 2014. This indicates that under the same fluctuation conditions of reservoir water level, a longer duration of rainfall has a more important influence on landslide deformation.

4. Prediction Process

4.1. Point Selection and Data Processing

As shown in Figure 5, there were three types of deformation zones on the Shengjibao landslide: strong deformation zone, obvious deformation zone, and weak deformation zone. We selected ZK2-1 and ZK2-3 from the obvious deformation zone and the strong deformation zone, respectively, to represent two types of deformation points on the Shengjibao landslide.

Eighty-four measured values from January 2009 to December 2015 were adopted for establishing predictive models from the GPS monitoring points as ZK2-1 and ZK2-3. First, the adopted values were decomposed with MAM into trend terms and periodic terms. After decomposition, as suggested by Yang [25], 72 values were set as the fitting dataset, 48 values were set as the training dataset to fit the models, and 24 values were set as validation dataset to tune the hyperparameters of the models. Twelve values were set as predicting dataset to evaluate the capability of the models (Figure 10).

4.2. Factors Selection

Selecting the proper triggering factors is essential to guarantee the accuracy of a prediction. First, the candidate factors were determined with landslide deformation characteristics. Since displacement rates mainly increased during the rainy season and reservoir water level fluctuation stages (Figure 11), rainfall and reservoir water level were considered the main triggering factors that caused the deformation of the Shengjibao landslide. Additionally, the Wenchuan earthquake that occurred in May 2008 had an insignificant impact on the pattern of the landslide displacement according to the monitoring records shown in Figure 8 and Figure 11. Thus, the events of the earthquake were not considered as candidate factors for prediction. As the values of GPS monitoring data were obtained at the end of each month, the candidate factors should include the precipitation during the current month, the precipitation during the past two months, the average reservoir level during the current month, and the change of the reservoir level during the current month, the change of the reservoir level during the past two months. These candidate factors were considered as $f_1$, $f_2$, $f_3$, $f_4$, $f_5$.

Slopes in different evolutionary states may respond differently to the same external trigger factors. When the slope is stable, even strong influence factors are usually not enough to cause large deformation. In contrast, if a slope is marginally stable or even unstable, the slightest increase in external “loads” may cause disequilibrium and large deformation [40]. Therefore, the state factors were also considered as input values. The candidate factors should include the change of the displacement during the last month, during the last two months, during the last three months, and the average displacement during the last 12 months (annual displacement rate). These candidate factors were considered as $f_6$, $f_7$, $f_8$, $f_9$.

A resolution coefficient of 0.5 was obtained using the GCAM. The relational degrees $r_k$ between the periodic term and the candidate factors are shown in

Table 2. Relation degree between the periodic terms and the candidate factors.

Candidate Factors	Description	ZK2-1	ZK2-3
f1	the precipitation during the current month	0.65	0.66
f2	the precipitation during the past two months	0.67	0.63
f3	the average reservoir level	0.63	0.60
f4	the change of the reservoir level during the current month	0.72	0.66
f5	the change of the reservoir level during the past two months	0.71	0.69
f6	the change of the displacement that during the last month	0.68	0.76
f7	the change of the displacement that during the last two months	0.71	0.75
f8	the change of the displacement that during the last three months	0.74	0.74
f9	the average displacement during the last 12 months	0.67	0.72

. The candidate factors are closely related to the periodic term, $r_k>0.6$, suggesting that the parameters were properly selected [41].

4.3. Normalization and Inverse Normalization

LSTM is a type of Deep Learning Algorithms, while SVR is a type of Instance-Based Algorithm. Additionally, they are all Machine Learning (ML) Algorithms, while for ML Algorithms, normalization makes it easier to find optimal parameters and reduces the chances of getting stuck in local optimal [42,43]. To maximally preserve the distributions of the dataset, each triggering factor is normalized to [−1, 1] using linear normalization.

As stated in Section 4.1, the whole fitting datasets were assumed to be known, and the predicting datasets were considered unknown. To prevent information leakage, maximum and minimum values of inputs and outputs for normalization and inverse normalization were obtained from the fitting datasets, respectively.

$$x^{*}=2\times \frac{x-x_{\min }}{x_{\max }-x_{\min }}-1$$

(15)

where $x^{*}$ is the normalized value, $x$ is the original value, $x_{\max }$ is the maximum value of the samples, and $x_{\min }$ is the minimum value of the samples.

4.4. Parameters of LSTMs and SVRs

Keras framework [44] was used to establish LSTM models, while TensorFlow was used as the back end, and Python language was adopted to write the codes. As a time series forecasting experiment, hyperparameters of LSTM models were tuned with the Grid Search (GS) method [44]. Its process can be described as follows: plausible values for each parameter are combined, and then all the combination values are used in the training process of the model. The performance metric of each combination is typically measured by validation on the validation dataset. When all the parameter combinations have been tried, an optimal parameter combination with the best performances is returned automatically.

For LSTM models establishment, batch-size, and numbers of layers of LSTM models were determined with the GS method successively. In GS processing, the gird range of batch-size is [1, 20] with a grid step of 1, the gird range of layers is [2,8] with a grid step of 1. The optimal epochs were obtained with an early-stopping set during the algorithmic process, and the patience of early-stopping was set as 20 which means the algorithmic process will stop when the result has no improvement within 20 steps.

The SVR models were also operated in the Python language environment. The GS method was used to search for the optimal parameters of the SVR model. As stated in Section 2.4, the RBF was adopted as kernel function for the SVR model. The values of gamma (the kernel coefficient for RBF) and the values of $C$ (the penalty parameter) were determined with GS. In GS processing, the search range of gamma is [0.0075, 1] with a grid step of 0.0075, the search range of C is [1, 20] with a grid step of 1. The best parameters are shown in

Table 3. Optimal hyperparameters of LSTMs and support vector regressions (SVRs) used for the dataset.

Point	LSTMs				SVRs
	Numbers of Layers	Numbers of Epochs	Numbers of Batch-Size	Numbers of Layers	C	Gamma
Trend term of ZK2-1	3	70	2	3	14.0	0.015
Periodic term of ZK2-1	4	107	13	4	3.0	0.015
Trend term of ZK2-3	3	54	4	3	17.0	0.0075
Periodic term of ZK2-3	2	355	2	2	15.0	0.015

.

5. Results

5.1. LSTM Models and SVR Models

Figure 12,

Table 4. Accuracy of the predicted displacement with the LSTM and SVR models of ZK2-1.

Time	Original Displacement (mm)	LSTMs			SVRs
		Predicted Displacement (mm)	Absolute Error (mm)	Relative Error (%)	Predicted Displacement (mm)	Absolute Error (mm)	Relative Error (%)
2015-01	1340.71	1336.37	4.33	0.32	1340.30	0.41	0.03
2015-02	1343.48	1339.88	3.60	0.27	1343.47	0.01	0.00
2015-03	1349.82	1342.94	6.89	0.51	1348.05	1.77	0.13
2015-04	1353.57	1361.07	7.51	0.55	1364.93	11.36	0.84
2015-05	1378.87	1367.57	11.30	0.82	1373.37	5.50	0.40
2015-06	1428.34	1406.19	22.15	1.55	1404.04	24.30	1.70
2015-07	1474.93	1469.89	5.05	0.34	1453.27	21.66	1.47
2015-08	1493.43	1502.84	9.41	0.63	1492.91	0.52	0.03
2015-09	1489.41	1490.28	0.87	0.06	1495.64	6.23	0.42
2015-10	1484.74	1475.99	8.75	0.59	1477.60	7.14	0.48
2015-11	1483.27	1469.68	13.58	0.92	1465.96	17.31	1.17
2015-12	1500.79	1466.85	33.95	2.26	1467.16	33.63	2.24
Min			0.87	0.06		0.01	0.00
Max			33.95	2.26		33.63	2.24
Mean			10.61	0.74		10.82	0.74
RMSE		13.79			15.12

and

Table 5. Accuracy of the predicted displacement with the LSTM and SVR models of ZK2-3.

Time	Original Displacement (mm)	LSTMs			SVRs
		Predicted Displacement (mm)	Absolute Error (mm)	Relative Error (%)	Predicted Displacement (mm)	Absolute Error (mm)	Relative Error (%)
2015-01	2222.73	2217.40	5.33	0.24	2209.50	13.23	0.60
2015-02	2225.82	2223.07	2.75	0.12	2217.39	8.43	0.38
2015-03	2232.97	2228.60	4.37	0.20	2225.04	7.93	0.36
2015-04	2236.89	2258.60	21.71	0.97	2249.45	12.56	0.56
2015-05	2277.32	2277.21	0.11	0.00	2270.75	6.57	0.29
2015-06	2342.75	2350.48	7.73	0.33	2330.80	11.95	0.51
2015-07	2401.47	2391.93	9.54	0.40	2383.81	17.66	0.74
2015-08	2432.98	2428.25	4.73	0.19	2424.24	8.74	0.36
2015-09	2450.97	2424.41	26.56	1.08	2436.17	14.80	0.60
2015-10	2463.37	2448.69	14.68	0.60	2438.27	25.10	1.02
2015-11	2465.66	2453.88	11.78	0.48	2446.16	19.50	0.79
2015-12	2477.34	2463.61	13.73	0.55	2453.10	24.24	0.98
Min			0.11	0.00		6.57	0.29
Max			26.56	1.08		25.10	1.02
Mean			10.25	0.43		14.23	0.60
RMSE		12.74			15.43

present the comparison between measured and predicted displacement of ZK2-1 and ZK2-3 using LSTM and SVR models. For ZK2-1, the values of RMSE and MAPE with the LSTM model were 13.79 mm and 0.74%, while the accuracy factors of the SVR model were 15.12 mm and 0.74%. For ZK2-3, the values of RMSE and MAPE with the LSTM model were 12.74 mm and 0.43%, while the accuracy factors of the SVR model were 15.43 mm and 0.60%.

In trend terms, the values of RMSE with the LSTM model and SVR model in ZK2-1 were 3.38 and 3.17 mm, and the values of RMSE with LSTM model and SVR model in ZK2-3were 5.23 and 6.26 mm. In periodic terms, the values of RMSE with the LSTM model and SVR model in ZK2-1 were 12.95 and 13.63 mm, and the values of RMSE with the LSTM model and SVR model in ZK2-3 were 10.92 and 10.62 mm (

Table 6. Accuracy of the prediction in trend term and periodic term of ZK2-1 and ZK2-3.

Model	RMSE in Trend Term(mm)		RMSE in Periodic Term(mm)
	ZK2-1	ZK2-3	ZK2-1	ZK2-3
LSTM	3.38	5.23	12.95	10.92
SVR	3.17	6.26	13.63	10.62

).

The LSTM models and SVR models both had worse performances after May 2015. LSTM models had significantly better performance in step a, period c, step d, and period e than SVR models. The performances of the models in step a, period b, period c, step d, and period e in the periodic terms were reflected in the total displacement (Figure 12).

5.2. Ensemble Models

Eight models were established by tuning hyperparameters and training for prediction. In this section, the whole datasets with 84 time-steps of ZK2-1 and ZK2-3 were applied to make predictions with the LSTM models and SVR models. The total prediction displacements of ZK2-1 and ZK2-3 from January 2009 to December 2015 with LSTM models and SVR models can be obtained with this process. An optimal weight can be obtained with setting the original total displacements as $E(t)$ in Equation (12).

The optimal weights were set as outputs, and all the inputs that show in Table 2, the obtained total prediction displacements with LSTM models, the obtained total prediction displacements with SVR models, and the difference between these two total prediction displacements were set as candidate input factors. The latter 3 new candidate factors were considered as $f_{10}$, $f_{11}$, $f_{12}$.

An SVR model using the GS method was adopted to predict the optimal weight for an ensemble model. As stated in Section 4.1, 84 measured values were adopted for finding a better model between LSTMs and SVRs. For the ensemble model runs, 72 time-steps were set as the fitting dataset, 48 time-steps were set as a training dataset to fit the models, and 24 time-steps were set as validation dataset to tune the hyperparameters of the SVR models and to determine the optimal inputs (

Table 7. Optimal hyperparameters and inputs of SVRs used for optimal weight prediction.

Point	Inputs	Hyperparameters
		C	Gamma
ZK2-1	f12	1.0	0.0075
ZK2-3	f1, f2, f3, f4, f5, f6, f7, f8, f9	11.0	0.105

). Twelve time-steps were set to evaluate the models. The optimal weights were set as 1 when the predicting values were more than 1, and were set as 0 when the predicting values were less than 0.

Figure 13,

Table 8. Accuracy of the predicted displacement with the Ensemble models of ZK2-1.

Time	Original Displacement (mm)	Predicted Displacement (mm)	Optimal Weight	Absolute Error (mm)	Relative Error (%)
2015-01	1340.71	1337.31	0.76127	3.40 *	0.25
2015-02	1343.48	1340.73	0.76283	2.75 *	0.20
2015-03	1349.82	1344.19	0.75586	5.63 *	0.42
2015-04	1353.57	1361.99	0.76160	8.42 ^	0.62
2015-05	1378.87	1369.00	0.75274	9.87 *	0.72
2015-06	1428.34	1405.74	0.78928	22.60 ^	1.58
2015-07	1474.93	1467.50	0.85624	7.43 ^	0.50
2015-08	1493.43	1501.11	0.82538	7.68 *	0.51
2015-09	1489.41	1491.60	0.75473	2.19 ^	0.15
2015-10	1484.74	1476.36	0.77193	8.38 *	0.56
2015-11	1483.27	1468.93	0.79659	14.34 ^	0.97
2015-12	1500.79	1466.92	0.77788	33.87 *	2.26
Min				2.19	0.15
Max				33.87	2.26
Mean				10.55 *^	0.73
RMSE		13.76 *^

* Better than LSTM model, ^ better than SVR model.

and

Table 9. Accuracy of the predicted displacement with the LSTM and SVR models of ZK2-3.

Time	Original Displacement (mm)	Predicted Displacement (mm)	Optimal Weight	Absolute Error (mm)	Relative Error (%)
2015-01	2222.73	2216.74	0.93060	5.88 ^	0.26
2015-02	2225.82	2223.06	1.0	2.75 ^	0.12
2015-03	2232.97	2228.46	0.97188	4.47 ^	0.20
2015-04	2236.89	2255.25	0.65265	18.53 *	0.83
2015-05	2277.32	2271.63	0.14902	5.61 ^	0.25
2015-06	2342.75	2350.48	1.0	7.73 ^	0.33
2015-07	2401.47	2391.93	1.0	9.54 ^	0.40
2015-08	2432.98	2428.25	0.96782	4.86 ^	0.20
2015-09	2450.97	2435.83	0	14.80 *	0.60
2015-10	2463.37	2448.69	1.0	14.68 ^	0.60
2015-11	2465.66	2453.88	1.0	11.78 ^	0.48
2015-12	2477.34	2463.61	1.0	13.73 ^	0.55
Min				2.75	0.12
Max				18.53	0.83
Mean				9.53 *,^	0.40
RMSE		10.70 *,^

* Better than LSTM model, ^ better than SVR model.

present the comparison between measured and predicted displacement of ZK2-1 and ZK2-3 using LSTM, SVR, and ensemble models. For the total displacement term of ZK2-1, the values of RMSE and MAPE with the Ensemble model were 13.76 mm and 0.73%, while the accuracy factors of the LSTM model were 13.79 mm and 0.74%, and the accuracy factors of the SVR model were 15.12 mm and 0.74%.

For the total displacement term of ZK2-3, the values of RMSE and MAPE with the Ensemble model were 10.70 mm and 0.40%, while the accuracy factors of the LSTM model were 12.74 mm and 0.43%, and the accuracy factors of the SVR model were 15.43 mm and 0.60%.

The total displacements predicted by the Ensemble model were in better agreement with the measured values than those with the LSTM or SVR models. As shown in Figure 13, in point a, point b, point c, and period d, the results of the Ensemble models tend to be closer to the results of original values than at least one of the LSTM or SVR models. This suggests that when the results of the LSTM and SVR models are different, the Ensemble model gives more weight to the results of the model that are closer to the real values. As shown in Table 8, in 12 predicted points of ZK2-1, 7 of them were had lower values of absolute error than the LSTM model, 5 of them had lower values of absolute error than the SVR model. As shown in Table 9, in 12 predicted points of ZK2-3, 2 of them were had lower values of absolute error than LSTM model, 10 of them had lower values of absolute error than SVR model.

6. Discussion

Considering predicting total displacement, the LSTM models showed a more satisfactory performance than the SVR models in ZK2-1 and ZK2-3, respectively, and the proposed Ensemble models yield relatively better results (Table 8 and Table 9). In predicting trend terms, the LSTM models showed better performance than SVR models in ZK2-3 and worse performance in ZK2-1. In predicting periodic terms, the LSTM models showed better performance than SVR models in ZK2-1 and worse performance in ZK2-3. This suggests that the accuracy of the total displacement prediction does not depend in the case of the Shengjibao landslide on the accuracy of only one sub-sequence.

Meanwhile, the generalization ability of the models directly affects the performance of the test dataset. The models established based on the performance on the validation dataset do not always perform as expected on the predicting dataset. As shown in Table 6 and

Table 10. Accuracy of the prediction in trend term and periodic term of ZK2-1 and ZK2-3.

Model	RMSE in Trend Term(mm)		RMSE in Periodic Term(mm)
	ZK2-1	ZK2-3	ZK2-1	ZK2-3
LSTM	2.84	36.29	14.95	56.30
SVR	3.39	35.58	15.11	60.53

, the LSTM models did not achieve the expected performance in predicting the trend term of ZK2-1 or the periodic term of ZK2-3, while the SVR model did not achieve the expected performance in predicting the trend term of ZK2-3.

There have been few reports on the research of decomposing the original landslide displacement into trend term, periodic term, and stochastic term [7,9] up to now. The stochastic displacement is caused by some random factors such as wind load and vehicle load. In decomposing original displacement, we chose to ignore the stochastic term. One should consider means to develop optimal models for predicting the stochastic displacement in the future. Additionally, the stability of the location of the GPS monitoring control point is the key controlling factors of the reliability of the monitoring data. Thus, the evaluation of the stability of the GPS monitoring control point is an important content that cannot be ignored in the processing of displacement observation data.

As stated in Section 3.2, there is a sharp increase in the displacement rates in August and September 2013. Although the proposed method achieved good results in this study, if factors as the precipitation or water level suddenly change, or an unusual rainfall pattern occurs, the error in point-based displacement predictions will inevitably increase. Thus, solving this problem becomes highly necessary [9]. As the Shengjibao landslide is a wading landslide in the TGRA, the factors considered including precipitation, water level and displacement. One should, in the future, consider means to develop prediction approaches for landslides with different triggering factors. There are three types of deformation zones divided on Shengjibao landslide. Any single point cannot represent the displacement characteristics of the entire landslide. To differentiate spatial zones of comparably the deformation characteristics and select the monitoring data of representative points are within these zones are vital preparatory steps for displacement prediction studies.

According to the existed research on the application of ML techniques to landslide displacement prediction up to now, many hybrid models were established for prediction. However, this research mainly focus on the hybrid models of the coupled single ML models and models used to obtain optimal hyperparameters for these proposed single ML models. In this study, an ensemble model of the coupled LSTM and SVR algorithm with the weight coefficients based on linear combination theory was established for prediction, and the ensemble model performed well. Meanwhile, many approaches to landslide susceptibility mapping and landslide risk analysis have been developed [45,46], including those based on artificial neural network [47,48], support vector machine [49], alternative decision tree [36]. In order to achieve an accurate result in landslide susceptibility mapping and landslide risk analysis, it is necessary to research more advanced ML algorithms. Thus, the novel method proposed in this paper is recommended for conducting landslide susceptibility mapping, landslide risk analysis, and other fields.

7. Conclusions

The monitoring data are fundamental for establishing landslide displacement prediction models, and the results of prediction are significant for landslide early warning. In this study, the Shengjibao landslide in the TRGA was taken as an example. Based on the monitoring data, the response relationship between landslide displacement and factors is analyzed, an ensemble model of the coupled LSTM and SVR algorithm with the weight coefficients based on linear combination theory was established. The dynamic model based on the deep learning algorithm of LSTM and the static SVR model were also used to predict landslide displacement for comparison. The main conclusions are as follows: (1) Under the same fluctuation conditions of reservoir water level, a longer duration of rainfall has a more important influence on landslide deformation; (2) Overall, the LSTM models perform better than the SVR models, but the results of LSTM models are not closer to the original value than the results of SVR models at every time step in predicting dataset; (3) The proposed Ensemble model integrated both the advantages of LSTM and SVR algorithms, where the Ensemble model has higher prediction performance than both of LSTM and SVR models. In general, by combing the advantages of different ML algorithms, the ensemble model proposed in this paper can effectively construct the response relationship between landslide displacement and factors.

Overall, the proposed method, which applies time series analysis, LSTM and SVR, can achieve accurate predictions in case of slow and the step-like deformation periods. This method of establishing an Ensemble model has the potential for application to predict landslide displacement in the TGRA and other landslide-prone regions.