Predicting Surface Subsidence in Northern Huainan Based on a Hybrid LSTM–Transformer Model

Abstract

As one of the world’s primary energy sources, coal has driven economic development but has also led to severe surface subsidence. Currently, many regions around the world face significant ground deformation risks due to ongoing or legacy mining activities. Accurate monitoring and trend prediction are critical for enhancing subsidence early-warning capabilities and urban resilience. The northern region of Huainan City exhibits a spatial pattern characterized by the coexistence of mining areas, urban areas, and decommissioned mining sites, among which the mining areas show more pronounced surface deformation due to prolonged mining activities. To fully understand the subsidence evolution characteristics and differences across various regions, an LSTM–Transformer prediction model was constructed based on SBAS-InSAR monitoring technology to predict the surface subsidence processes in the three types of areas separately. The results indicated that the subsidence rate and cumulative subsidence in the mining areas were significantly greater than those in the urban and decommissioned areas, demonstrating more intense deformation activity. The average subsidence rates for the mining areas, urban areas, and decommissioned mining sites were −57.42 mm/yr, −5.37 mm/yr, and −3.21 mm/yr, respectively. The model’s prediction results demonstrated good accuracy across different regions, with the root mean square errors (RMSEs) for the mining areas, urban areas, and decommissioned mining sites being 2.16 mm, 1.03 mm, and 0.22 mm, respectively. The study shows that the constructed LSTM–Transformer hybrid model not only possesses strong capability in fitting subsidence trends but will also provide a scientific basis for future monitoring and early warning of surface subsidence hazards.

1. Introduction

Huainan is a typical resource-based city in China, with abundant coal reserves. In the northern part of Huainan City, mining areas are densely distributed, and the urban population is highly concentrated. Due to coal mining activities, the mining areas and urban areas exhibit distinct subsidence characteristics. The mining areas exhibit an average annual subsidence rate exceeding −50 mm/yr, with some regions reaching as high as −148.40 mm/yr [1]. In contrast, surface deformation in the urban areas is relatively stable, with an average annual subsidence rate around −23 mm/yr [2]. The mining areas have a significant impact on the surrounding regions. Therefore, developing predictive models for surface deformation in mining and urban areas holds significant scientific importance for surface subsidence early warning.

Traditional surface deformation monitoring methods generally suffered from limitations such as point-based measurements, insufficient spatial resolution, and high resource consumption. However, Interferometric Synthetic Aperture Radar (InSAR) technology effectively addressed these issues. In 1969, the team led by Rogers [3] pioneered the development of InSAR technology. With its advantages of centimeter-level accuracy, cost-effectiveness, wide-area coverage, and high spatial resolution, this innovative technique has been successfully applied in the field of geological hazard monitoring, including early detection of potential landslides [4], inversion of seismic deformation fields [5], and urban surface subsidence monitoring [6]. It is worth noting that Differential InSAR(D-InSAR), which was developed based on InSAR, has enhanced deformation monitoring capabilities. However, its practical effectiveness is constrained by spatiotemporal baseline decorrelation effects, which often lead to phase unwrapping difficulties and a decline in data quality. In 2002, the team led by Berardino [7] first proposed the Small Baseline Subset Interferometric Synthetic Aperture Radar (SBAS-InSAR) technique. By optimizing the spatiotemporal baseline configuration and adopting a multi-master image combination strategy, SBAS-InSAR effectively overcame the phase decorrelation issues caused by long baselines in D-InSAR [8]. However, it was limited to the inversion of historical deformation and lacked predictive capability. Most existing studies employed single machine learning models [9] to process large-scale subsidence data, overlooking the differences in subsidence-driving mechanisms of mining areas affected by the dynamics of goaf areas and urban built-up areas affected by building loads, resulting in insufficient generalization of the models. At the prediction method level, most researchers commonly employed Long Short-Term Memory (LSTM) neural networks [10] to process time-series deformation data; however, the single network structure struggled to capture long-range dependencies in sudden subsidence events.

In recent years, SBAS-InSAR [11,12,13,14,15,16,17,18,19,20,21] technology has been widely applied to monitor surface deformation in both mining areas and urban regions. However, most existing studies have primarily focused on the identification and analysis of historical deformation, while research on the prediction of future subsidence remains relatively limited. Furthermore, traditional approaches often fail to account for the differing driving mechanisms between mining and urban environments, and single-network models have difficulty capturing sudden deformation variations. In addition, many existing prediction methods rely on conventional statistical or machine learning models, which are insufficient to accurately characterize the nonlinear and temporal dependency features inherent in deformation time series. Therefore, LSTM–Transformer hybrid prediction model was designed, which utilized LSTM [22,23] to extract local temporal features and combined the Transformer’s self-attention mechanism [24,25] to capture long-term spatiotemporal dependencies in subsidence events. The two types of features were dynamically fused through gating units. Finally, a unified surface subsidence prediction model was constructed to simultaneously model three types of regions. This approach effectively overcomes the limitations of traditional statistical, physical [26,27], and machine learning models in terms of prediction accuracy and their ability to handle complex geological conditions.

2. Overview of the Study Area and Research Data

2.1. Overview of the Study Area

The northern part of Huainan City is located in the central–northern region of Anhui Province, upstream of the Huai River, with coordinates ranging from 32°30′08″ N to 33°00′26″ N, and from 116°21′05″ E to 117°12′30″ E. Situated in East China, this area is a major energy base with abundant natural resources. The topography of the northern region is predominantly characterized by plains and hills, with the Tianjiaan District being notably urbanized. However, extensive and ongoing mineral extraction activities have led to the formation of numerous mining subsidence zones, disrupting the structure of underground aquifers and exacerbating surface subsidence. Furthermore, the surface subsidence in urbanized areas has become more pronounced due to various anthropogenic factors, including building load effects, disturbances from underground construction, and the over-extraction of shallow groundwater. For subsequent experimental analysis, the study area was divided into three types of regions-mining areas, urban areas, and abandoned mining areas-based on administrative boundaries and mining rights. The research area is depicted in Figure 1.

2.2. Research Data

The data used in this study comprise 90 Sentinel-1A images collected between January 2022 and December 2024, along with the ASTER GDEM Digital Elevation Model (DEM) at a 30 m resolution, which was used to mitigate the terrain-induced phase effect. Detailed parameter information is provided in

Table 1. Parameter information of Sentinel-1A images.

Sensor	Orbit Direction	Revisit Period	Polarization Mode	Operating Mode	Number of Images/Scenes
Sentinel-1A	Ascending	12d	VV	IW	90

.

3. Methodology

3.1. SBAS-InSAR Processing

The SBAS-InSAR technique selects interferometric pairs whose temporal and spatial baselines satisfy certain thresholds and processes them using multi-temporal spatial analysis to maximize the coherence of the interferometric phase, making it suitable for large-scale time-series deformation analysis. Here, the temporal baseline refers to the time interval between two image acquisitions, while the spatial baseline represents the vertical distance between satellite orbits at different acquisition times. To reduce decorrelation effects and improve interferogram coherence, interferometric pairs are screened based on the small-baseline criterion. In this study, 90 Sentinel-1 images were used. After performing image co-registration, differential interferometry, and other preprocessing steps, temporal and spatial baseline connection maps were generated. Combined with the ASTERGDEM Digital Elevation Model, the resulting time series were analyzed to invert nonlinear surface deformation, ultimately producing deformation time series, cumulative deformation maps, and annual mean deformation rate maps.

3.2. Model Architecture Choice

Temporal CNN is a type of neural network designed for processing time-series data and is highly efficient in terms of computation. Essentially, it is a temporal extension of CNN, employing one-dimensional convolutions to replace the stepwise modeling of traditional RNNs. The convolutional layers extract local temporal patterns, which allows Temporal CNN to perform well in short-term prediction. However, for long-term time-series data such as surface subsidence, its modeling performance is limited. Moreover, Temporal CNN lacks a global attention mechanism, making it prone to overlooking abrupt temporal changes. In addition, subsidence patterns in mining areas are often nonlinear and unstable, and Temporal CNN shows inadequate fitting capability for such complex data.

Gradient Boosting is a method that models nonlinear relationships by sequentially fitting multiple decision trees to the residuals. Although it offers fast training speed, it lacks the ability to model temporal dependencies and is therefore unable to capture long-term subsidence evolution. Moreover, the model relies on static feature splits, which prevents it from dynamically adjusting weights over time.

In this study, a hybrid LSTM–Transformer model was employed. The LSTM component captures long-term dependencies in the time series, effectively mitigating the vanishing gradient problem, while the Transformer module introduces a self-attention mechanism that can dynamically identify critical stages in the time series, such as abrupt deformation caused by mining activities or seasonal variations. This hybrid architecture not only preserves the continuity of temporal information but also enhances the model’s ability to perceive global temporal correlations.

3.3. LSTM Long Short-Term Memory Network

LSTM is a specialized type of Recurrent Neural Network (RNN). The core components of LSTM are three gates (forget gate, input gate, and output gate) and a cell state. The cell state flows through the entire time series and is responsible for transmitting long-term memory with only a small number of linear operations, making it less susceptible to gradient vanishing.

The forget gate decides which information to discard from the cell state, outputting values between 0 and 1 via a sigmoid function (0 indicating complete forgetting and 1 indicating complete retention).

$$f_t=\sigma (W_f\cdot [h_{t-1},x_t]+b_f)$$

(1)

In the formula, $f_t$ represents the output of the forget gate; $\sigma$ represents the $\mathrm{s}\mathrm{i}\mathrm{g}\mathrm{m}\mathrm{o}\mathrm{i}\mathrm{d}$ activation function, which outputs values between 0 and 1; $W_f$ represents the weight matrix of the forget gate; $h_{t-1},x_t$ represents the concatenated matrix of the previous hidden state and the current input; and $b_f$ represents the bias term of the forget gate.

The input gate controls the new information to be added to the cell state, with the sigmoid layer determining which parts will be updated, and the Tanh layer generating the candidate value ${\tilde{C}}_t$.

$$i_t=\sigma (W_i\cdot [h_{t-1},x_t]+b_i)$$

(2)

In the formula, $i_t$ represents the output of the input gate, which determines the new information to be added to the cell state; $W_i$ represents the weight matrix of the input gate; $h_{t-1},x_t$ represents the concatenated matrix of the previous hidden state and the current input; and $b_i$ represents the bias term of the input gate.

$${\tilde{C}}_t=tanh(W_c\cdot [h_{t-1},x_t]+b_c)$$

(3)

In the formula, ${\tilde{C}}_t$ is the candidate value, generated through the $tanh$ activation function, with values ranging from −1 to 1; $W_C$ represents the weight matrix of the candidate value; $h_{t-1},x_t$ represents the concatenated matrix of the previous hidden state and the current input; and $b_C$ represents the bias term of the input gate.

The cell state is updated by combining the results of the forget gate and the input gate.

$$C_t=f_t\odot C_{t-1}+i_t\odot {\tilde{C}}_t$$

(4)

In the formula, $C_t$ represents the cell state at the current time step, $\odot$ represents element-wise multiplication, and $C_{t-1}$ represents the cell state at the previous time step.

The output gate determines the output at the current time step, based on the updated cell state and the filtering by the output gate.

$$o_t=\sigma (W_o\cdot [h_{t-1},x_t]+b_o)$$

(5)

In the formula, $o_t$ represents the output of the output gate, which determines the output at the current time step; $W_o$ represents the weight matrix of the output gate; $h_{t-1},x_t$ represents the concatenated matrix of the previous hidden state and the current input; and $b_o$ is the bias term of the output gate.

$$h_t=o_t\odot tanh(C_t)$$

(6)

In the formula, $h_t$ represents the hidden state at the current time step, while $tanh\left(C_t\right)$ represents the cell state generated by the $tanh$ activation function.

3.4. LSTM–Transformer Model

In the constructed model, sinusoidal position encoding is used to inject absolute position information, enabling the model to capture information from each position in the sequence. This helps the model better capture information from each position in the sequence, thereby improving its performance in processing sequential data.

$$P{E}_{(pos,2i)}=\sin \left({\displaystyle \frac{pos}{{\mathrm{10,000}}^{\frac{2i}{d}}}}\right)$$

(7)

$$P{E}_{(pos,2i+1)}=\cos \left({\displaystyle \frac{pos}{{\mathrm{10,000}}^{\frac{2i}{d}}}}\right)$$

(8)

In the formula, pos represents the position index, i represents the dimension index, and d represents the embedding dimension.

The encoder consists of N identical layers stacked together, each containing multi-head self-attention mechanisms and residual connections. In the multi-head self-attention module, the input is divided into h parallel attention heads (where h = 4 in this experiment), and each head independently computes scaled dot-product attention:

$$\mathrm{Attention}(Q,K,V)=\mathit{softmax}\left({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}}\right)V$$

(9)

In the formula, $\mathit{Q},\mathit{K},\mathit{V}\in {\mathit{R}}^{\mathit{T}\times {\mathit{d}}_{\mathit{k}}}$ represent the Query, Key, and Value matrices, respectively; ${\mathit{d}}_{\mathit{k}}$ represents the dimensions of the key and value; ${\displaystyle \frac{\mathit{Q}{\mathit{K}}^{\mathit{T}}}{\sqrt{{\mathit{d}}_{\mathit{k}}}}}$ is used to compute the similarity between the query and the key; and the division by $\sqrt{{\mathit{d}}_{\mathit{k}}}$ helps prevent the dot product from becoming too large, which could cause the gradient vanishing in the $\mathit{softmax}$ function.

The outputs from all attention heads are concatenated and undergo a linear transformation to produce the final attention output:

$$\mathrm{MultiHead}(Q,K,V)=Concat(hea{d}_1,\dots ,hea{d}_h)W^O$$

(10)

In the formula, $\mathit{h}\mathit{e}\mathit{a}{\mathit{d}}_{\mathit{i}}$ represents the output of the i attention head, ${\mathit{W}}^{\mathit{O}}$ represents the output weight matrix, and $\mathit{C}\mathit{o}\mathit{n}\mathit{c}\mathit{a}\mathit{t}$ can concatenate the outputs of all heads together.

The model structure is shown in Figure 2. The matrices V1-Vn, containing time-series subsidence values, are processedthrough the LSTM branch for feature extraction. The extracted features are thenfurther refined by the Transformer’s multi-head attention mechanism to captureglobal dependencies, ultimately generating the final predictions.

3.5. Module Ablation Test

In constructing an ablation test for the LSTM–Transformer model, we focus on analyzing the impact of different components on model performance. The ablation test systematically evaluates the contribution of key components by removing or replacing them step-by-step. First, we removed the Transformer module and compared the results with the baseline model. Second, we ablated the LSTM component to evaluate its substantive contribution to capturing long-term cumulative trends. It is crucial that all other hyperparameters and thetraining process remain consistent during the tests to ensure fairness and that any variations in the results are a true reflection of the specific components’ influence.

3.6. Prediction Accuracy Evaluation Metrics

This paper evaluates the prediction performance and accuracy of the model using two common error metrics: Mean Absolute Error (MAE) and Root Mean Square Error (RMSE).

$$S_{MAE}={\displaystyle \frac{1}{m}}{\displaystyle \sum _{i=1}^m|}{Y}_i-y_i|$$

(11)

$$S_{RMSE}=\sqrt{{\displaystyle \frac{1}{m}}{\displaystyle \sum _{i=1}^m{(Y_i-y_i)}^2}}$$

(12)

In the formula, m represents the total number of samples, ${\mathit{Y}}_{\mathit{i}}$ is the actual value of the $\mathit{i}$ point, ${\mathit{y}}_{\mathit{i}}$ is the predicted value of the $\mathit{i}$ point. The smaller the values of ${\mathit{S}}_{\mathit{M}\mathit{A}\mathit{E}}$ and ${\mathit{S}}_{\mathit{R}\mathit{M}\mathit{S}\mathit{E}}$, the better the model’s prediction accuracy.

4. Results

4.1. Subsidence Rate Analysis

Figure 3 illustrates the subsidence rate in the study area from 2022 to 2024. The annual average deformation rate in the study area mainly ranges from −5 to 10 mm/yr, while the mining areas show significant deformation. The southeastern part of the study area, which is the urban area, experiences a deformation rate stable between −5 and 10 mm/yr, influenced by factors such as municipal construction, groundwater extraction, and natural settlement. In contrast, the mining areas exhibit larger deformation rates, and some regions show decorrelation. The maximum deformation rate in the northern part of the study area reaches −85.70 mm/yr. The yellow boundary line indicates the decommissioned mining sites. During the mining period, underground voids caused surface subsidence and settlement. After mining ceased, these areas gradually stabilized, with subsidence slowing down, and the subsidence rate decreased each year, eventually stabilizing at around −2 mm/yr. From the figure, it is clear that location with deformation rates less than −30 mm/yr largely coincide with the mining areas. In the southern mining area, no significant settlement has been observed in recent years due to the cessation of mining, and the surface deformation has remained relatively stable, as shown in Figure 3.

4.2. Accuracy Validation

Based on the settlement values obtained from SBAS-InSAR technology, this study established eight research points (A–H), as shown in Figure 3, to analyze the time Variation In Cumulative deformation at these points. The data reveals that settlement has consistently increased at these 8 points, though the rate of increase varies. Among them, the settlement at points A–D, located in the mining area, is more significant, with point B showing the largest cumulative settlement, exceeding −200 mm. Points E and F, located in the urban area, show relatively stable settlement, with a range within −20 mm. Points G and H, located in the decommissioned mining sites, show stable settlement, both within −5 mm. The settlement characteristics of these 8 points reflect the settlement patterns in the mining area, urban area, and decommissioned mining sites. Therefore, these 8 research points will be used for subsequent experiments. The variation in settlement values over time for these 8 points is shown in Figure 4.

To ensure consistency in the temporal steps during model training, the original Sentinel-1 deformation time series was interpolated. The raw dataset consisted of 90 SAR images acquired at irregular time intervals. To obtain an evenly spaced temporal sequence and enhance the model’s ability to resolve the temporal evolution of deformation, linear interpolation was applied, resulting in 360 uniformly distributed time steps. No additional temporal smoothing was performed during interpolation to avoid introducing artificial autocorrelation or weakening the true fluctuation characteristics of ground subsidence. To verify the performance and accuracy of the model constructed in this study for actual predictions, the interpolated time-series subsidence values from eight study points were used as input data, with a feature dimension of 1 (time dimension). The dataset was divided into training, validation, and test sets according to an 8:1:1 ratio. Considering the characteristics of the time-series data and the prediction task, the input window was set to 32 and the output length to 7. After multiple adjustments, a learning rate of 0.001 was found to provide the best convergence speed and accuracy. The Adam optimizer was used to accelerate convergence. After testing different batch sizes, 16 was selected as the optimal batch size, balancing training speed and resource utilization. To prevent overfitting, a dropout rate of 0.1 was applied, which was experimentally found to provide the best training stability. The Tanh activation function was chosen to avoid the vanishing gradient problem. Mean Squared Error (MSE) was used as the loss function to evaluate model performance, and the number of training iterations was set to 200. Detailed model parameters are provided in

Table 2. The parameters of the LSTM–Transformer model.

Parameters	Settings
Number of LSTM Layers	64
Number of Transformer Layers	64
Number of Transformer Encoder Layers	2
Number of Attention Heads	4
Input Windows	32
Output Length	7
Learning Rate	0.001
Optimizer	Adam
Batch Size	16
Dropout Rate	0.1
Activation Function	Tanh
Loss Function	MSE
Epochs	200

. Root Mean Square Error (RMSE)and Mean Absolute Error (MAE) were used as evaluation metrics to assess prediction performance. In addition, an ablation study was conducted to further validate the contributions of different model components.

Based on the results of the ablation study, from a quantitative perspective, as shown in

Table 3. Prediction accuracy of feature points (among them, A–D represent research points in the mining area, E, F in the urban area, and G, H in the decommissioned mining area).

Research Point	Subsidence Rate mm/yr	LSTM		Transformer		LSTM–Transformer
		RMSE/mm	MAE/mm	RMSE/mm	MAE/mm	RMSE/mm	MAE/mm
A	−52.52	2.39	2.02	2.25	1.98	2.20	1.71
B	−68.14	3.73	3.35	3.59	3.36	3.52	3.08
C	−51.28	2.94	2.48	2.87	2.37	2.84	2.34
D	−58.38	2.61	2.10	2.59	2.01	2.16	1.67
E	−3.60	1.43	1.21	1.25	1.08	1.03	0.81
F	−5.38	1.74	1.15	1.53	1.41	1.39	1.12
G	−2.62	1.13	1.24	0.88	0.79	0.31	0.26
H	−1.25	0.87	0.79	0.74	0.63	0.22	0.17

, the LSTM–Transformer model achieved the highest prediction accuracy compared to the individual LSTM and Transformer models. This superior performance was consistent across mining areas, urban areas, and abandoned mining areas. Notably, at point H, the model achieved the lowest prediction errors, with a root mean square error (RMSE) of 0.22 mm and a mean absolute error (MAE) of 0.17 mm. Therefore, the LSTM–Transformer model demonstrates significant advantages in predicting surface subsidence across mining, urban, and abandoned mining regions.

4.3. Prediction Results and Analysis

After validating the accuracy and reliability of the proposed model, it was further applied to predict future surface subsidence at the eight established study points. The LSTM–Transformer model’s predicted subsidence values for the next seven times steps were compared with the corresponding measurements obtained from the SBAS-InSAR technique to evaluate the model’s predictive performance. The prediction results are presented in Figure 5.

As shown in the figure above, for the research points in the mining area, urban area, and decommissioned mining sites, the predictions from the LSTM–Transformer model closely follow the trends of the actual values, thereby validating the effectiveness of the model proposed in this paper.

5. Conclusions and Discussion

In this study, SBAS-InSAR technology was used to obtain surface deformation rates and cumulative settlement time series for the northern part of Huainan City. The results revealed that the mining area experiences more severe surface settlement compared to the urban and decommissioned mining sites, with a maximum deformation rate of −85.70 mm/yr and the maximum cumulative settlement reaching −243.81 mm. In the urban area, the maximum deformation rate was −31.47 mm/yr, and the largest cumulative settlement amounted to −29.72 mm. For the decommissioned mining sites, the maximum deformation rate was −5.76 mm/yr, with a maximum cumulative settlement of −6.21 mm. The model can provide a quantitative basis for identifying high-risk areas of surface instability, with in mining areas, the predicted results can be used to optimize excavation sequences and adjust mining plans to mitigate surface damage. In urban areas, the predictions can support infrastructure maintenance planning and risk zoning. In abandoned mining areas, the results can provide scientific guidance for ecological restoration strategies. Based on the monitoring results from SBAS-InSAR, the LSTM–Transformer model was constructed to predict settlement in the mining area, urban area, and decommissioned mining sites, leading to the following conclusions:

• By utilizing time-series settlement data obtained through SBAS-InSAR technology, effective classification and management of the mining area, urban area, and decommissioned mining sites were achieved, providing a solid data foundation for differentiated modeling. The LSTM–Transformer model demonstrated high accuracy in predicting settlement across all three areas, with the RMSE consistently maintained between 1 and 2 mm. These results underscore the feasibility and practical application of this method in the fine monitoring and prediction of surface settlement.
• A comparison of the LSTM–Transformer model’s predictions with the actual data demonstrates a high degree of consistency, highlighting the model’s strong predictive performance and reliability. The integration of SBAS-InSAR technology with the LSTM–Transformer model not only enhances the accuracy of surface settlement monitoring and prediction but also provides a solid scientific foundation and technical support for early warning and mitigation of settlement-related disasters.

However, this study also has some limitations. The temporal interpolation of the Sentinel-1 dataset, while ensuring uniform time steps, may introduce uncertainty in long-term deformation trends. In addition, external driving factors such as rainfall, groundwater extraction, or mining depth were not incorporated into the current model. Future work will focus on developing multi-source data fusion approaches and interpretable prediction frameworks to enhance model generalization and reliability.