Next Article in Journal
Evaluation and Optimization Strategies for Forest Landscape Stability in Different Landform Types of the Loess Plateau
Previous Article in Journal
Exploring the Advantages of Multi-GNSS Ionosphere-Weighted Single-Frequency Precise Point Positioning in Regional Ionospheric VTEC Modeling
Previous Article in Special Issue
Enhanced Phase Optimization Using Spectral Radius Constraints and Weighted Eigenvalue Decomposition for Distributed Scatterer InSAR
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Integrating InSAR Data and LE-Transformer for Foundation Pit Deformation Prediction

Surveying Engineering, School of Civil and Transportation Engineering, Guangdong University of Technology, Guangzhou 510006, China
*
Author to whom correspondence should be addressed.
Remote Sens. 2025, 17(6), 1106; https://doi.org/10.3390/rs17061106
Submission received: 10 January 2025 / Revised: 14 March 2025 / Accepted: 18 March 2025 / Published: 20 March 2025

Abstract

:
The rapid development of urban infrastructure has accelerated the construction of large foundation pit projects, posing challenges for deformation monitoring and safety. This study proposes a novel approach integrating time-series InSAR data with a multivariate LE-Transformer model for deformation prediction. The LE-Transformer model integrates Long Short-Term Memory (LSTM) to capture temporal dependencies, Efficient Additive Attention (EAA) to reduce computational complexity, and Transformer mechanisms to model global data relationships. Deformation monitoring was performed using PS-InSAR and SBAS-InSAR techniques, showing a high correlation coefficient (0.92), confirming the reliability of the data. Gray relational analysis identified key influencing factors, including rainfall, subway construction, residential buildings, soil temperature, and hydrogeology, with rainfall being the most significant (correlation of 0.838). These factors were incorporated into the LE-Transformer model, which outperformed univariate models, achieving a mean absolute percentage error (MAPE) of 2.5%. This approach provides a robust framework for deformation prediction and early warning systems in urban infrastructure projects.

1. Introduction

In the process of urban and rural development, the expansion and modernization of infrastructure have led to a significant increase in the number of large foundation pit projects. The safety of these foundation pit projects plays a pivotal role in maintaining societal stability, as any safety incidents could result in project delays, economic losses, and even severe harm to the surrounding environment and personnel [1]. Traditional monitoring methods for foundation pit projects, such as total stations, inclinometers, and theodolites, face several limitations [2,3]. These limitations include a restricted number of monitoring points, which prevent comprehensive coverage of the entire foundation pit area, and an inability to provide a full and accurate depiction of deformation patterns. Furthermore, these methods are often affected by baseline point shifts during excavation, leading to compromised monitoring accuracy and timeliness [4]. Therefore, there is an urgent need for new monitoring techniques that can enhance monitoring efficiency and overcome the limitations of current technologies. In contrast, InSAR technology provides large-scale, high-precision surface deformation monitoring, making it a promising tool for addressing these challenges. Many researchers have utilized InSAR technology for deformation monitoring [5,6]. For example, Jinhua Zhang et al. employed the PS-InSAR method to acquire ground settlement data along the Shanghai subway lines and validated these results with leveling measurement data. Their findings demonstrated good agreement between the two datasets, highlighting the reliability of InSAR-based monitoring [7].
Surface deformation prediction is critical for determining the development trends and deformation modes of foundation pit projects [8]. Traditional prediction methods include empirical models, semi-empirical models, numerical models, and artificial neural network models [9]. For instance, Juan Du et al. utilized multiple neural network models combined with various influencing factors to predict landslide displacement in the accumulation layers of the Three Gorges Reservoir area. However, these models are only applicable to regions with well-defined deformation mechanisms and are unsuitable for complex study areas [10].
With the development of deep learning methods, models such as LSTM and Transformer have demonstrated outstanding performance in areas like computer vision and time-series forecasting [11,12,13]. For example, Chen Y et al. studied the Hong Kong International Airport and used the LSTM model to predict deformation, comparing the results with InSAR data. While the LSTM model performed well at the point-level scale, it did not account for influencing factors such as meteorological conditions [14]. The Transformer model, however, has proven to be an effective solution to this issue, demonstrating excellent performance in time-series prediction tasks due to its efficient parallel computation capabilities and self-attention mechanism. Mukesh Kumar Bharti et al. focused their research on Transformer models, exploring multivariate time-series prediction and achieving superior results [15]. Jing Wang et al. proposed a deep transformer-based model for InSAR time-series deformation prediction to accurately forecast surface deformation trends around Salt Lake in the permafrost areas of the Qinghai–Tibet Plateau [16]. Therefore, Transformer models can be leveraged for multi-feature pit deformation prediction and deformation mechanism analysis.
Previous research has primarily focused on InSAR time-series deformation prediction using GRU or LSTM models. Although Transformer-based models have demonstrated strong capabilities in InSAR time-series deformation prediction, they still require improvements. A key limitation of the Transformer model is its computational complexity and sensitivity to long-term dependencies, which impacts both the accuracy and efficiency of predictions [17,18]. This issue becomes particularly prominent in the ability to evaluate deformation trends in foundation pit areas, where long-term forecasting accuracy is critical. To address this gap, we propose the LE-Transformer model, which enhances time-series forecasting performance by integrating the strengths of LSTM, Efficient Additive Attention (EAA), and Transformer mechanisms. This model improves computational efficiency by optimizing time and space complexity, refining self-attention computation, and enhancing feature extraction capabilities, resulting in more efficient and accurate long-sequence data predictions. The proposed LE-Transformer model is applied to InSAR time-series deformation prediction for foundation pit deformation in the Nansha District of Guangzhou. The model incorporates environmental factors such as rainfall, subway construction, and hydrogeology to forecast deformation trends, providing a valuable tool for risk assessment and management in foundation pit engineering.

2. Materials and Methods

2.1. Study Area

This study focuses on the 220 kV Port Substation foundation pit project in Nansha District, Guangzhou, located at approximately 113°34′33″E and 22°46′14″N (Figure 1). The foundation pit is part of the region’s power infrastructure, influenced by various geological and environmental factors, including soil composition, groundwater conditions, rainfall, and nearby human activities like subway construction and residential buildings.
The geological conditions consist of a mixture of sand, clay, and peat. Peat and clay layers, with lower shear strength, are particularly prone to deformation when saturated. Sand provides some resistance, but its interaction with clay and peat leads to significant surface settlement, especially during heavy rainfall.
Environmental factors, such as rainfall and groundwater fluctuations, play a major role in soil behavior. Groundwater levels rise during the rainy season, leading to soil saturation, which weakens its strength and accelerates deformation. In the dry season, lower groundwater levels reduce moisture content, increasing soil temperature and causing shrinkage and settlement.
Fluctuating groundwater levels are a key risk factor for foundation pit stability. High groundwater levels weaken the soil, while low levels cause consolidation, but also increase brittleness, leading to cracking and damage. Monitoring these fluctuations is crucial to mitigate their impact.
Human activities, such as subway construction and residential buildings, add extra load to the soil, further contributing to deformation. The interaction of these factors creates a complex environment, requiring careful monitoring and analysis for surface stability management.

2.2. Dataset

The study used C-band Sentinel-1A imagery from the European Space Agency (ESA) as the primary Synthetic Aperture Radar (SAR) data source. A total of 72 SAR images were collected between 20200108 and 20240907, with a time interval of 24 days between each acquisition. The images were captured in IW mode with VV polarization and an incidence angle of 39.73°. PixelSpacingRg was 2.33 m, and PixelSpacingAz was 13.95 m. The study incorporated Precise Orbit Data (POD) to enhance satellite positioning accuracy, ensuring precise alignment of the SAR images with ground coordinates. To mitigate the impact of terrain-induced phase variations, a 30 m resolution Digital Elevation Model (DEM) generated from ALOS PALSAR data was applied. Additionally, the study incorporated tropospheric delay correction products from GACOS.
Beyond the primary SAR data, a range of complementary datasets were integrated to enhance the understanding of the factors influencing foundation pit deformation. These included subway construction data, which provided information on the spatial distribution of subway lines, the locations of subway stations, and other related factors. Residential building data were also incorporated, offering insights into the types, heights, and spatial distribution of buildings surrounding the pit. Rainfall data, obtained from the Nansha District meteorological station, were used to explore the relationship between precipitation and deformation, particularly during the rainy season. Soil temperature data, collected from monitoring stations within and around the foundation pit, were averaged on a monthly basis to assess the effects of temperature variations on soil behavior. Finally, hydrogeological data from local geological surveys provided crucial information on groundwater levels, water content, and other subsurface conditions, all of which were essential for a comprehensive analysis of the deformation process.

2.3. Deformation Monitoring and Model Development

Figure 2 illustrates the comprehensive workflow followed in this study, which consists of three main stages: InSAR data processing, impact factor extraction, and model development and forecasting. Initially, SAR images, DEM data, and POD are collected and processed using SARscape 5.7, with PS-InSAR and SBAS-InSAR techniques to extract deformation features. Then, ArcGIS 10.8 is used for visualization analysis and accuracy verification to ensure reliability. In the second stage, impact factors such as subway construction, housing construction, soil temperature, and rainfall, along with hydrogeological data, are analyzed through gray correlation analysis to evaluate their association with deformation. Finally, the data undergo model development, which includes preprocessing, model architecture design, and hyperparameter tuning, followed by training, inference, and deformation prediction. A subsequent evaluation assesses the model’s accuracy and forecasting capability.

2.3.1. InSAR Technology

(1)
PS-InSAR
The PS-InSAR technology was proposed by Ferretti et al. in 2001 [19]. By analyzing the amplitude and phase information of the images, it identifies permanent scatterers (PS), such as rocks and buildings, with high coherence and strong scattering characteristics in long time-series. The core principle is to establish a differential interferometric phase model for PS points, separating the temporal and spatial characteristics of each phase component, thereby obtaining both linear and nonlinear deformation information for each point. This study adopts an enhanced PS-InSAR technology, which performs adaptive filtering on the interferometric phase to more accurately identify PS and distributed scatterer (DS) pixels. This method effectively extracts surface deformation information, improving the accuracy and coverage of ground deformation monitoring [20,21].
Acquire N + 1 SAR images covering the same region, using one of them as the reference image. After registration, resampling, atmospheric correction, and removal of flat earth and topographic phases, N interferograms are obtained. The interferograms are then subjected to terrain correction using DEM and satellite orbit data. The phase at position x in the i-th interferogram, denoted as ϕ x , i , can be expressed as follows:
ϕ x , i = ϕ d e f o , x , i + ϕ t o p o , x , i + ϕ c o n s t , x , i + ϕ s l o p e , x , i + ϕ a t m , x , i + ϕ n o i s e , x , i
where the sum of the three components, ϕ c o n s t , x , i , ϕ s l o p e , x , i , and ϕ a t m , x , i , is referred to as the atmospheric phase shift (APS). These components represent the constant phase value, the linear phase shift induced by atmospheric effects, and the nonlinear atmospheric effects, respectively. ϕ d e f o , x , i and ϕ t o p o , x , i represent the ground deformation phase along the radar line of sight and the terrain phase caused by DEM errors, while ϕ n o i s e , x , i represents the noise phase [22,23].
The acquired images are first subjected to differential interferometric processing and registered to the master image from 21 May 2022 (Figure 3). Then, adaptive filtering is applied to identify the DS points from the interferogram. High amplitude pixels are selected as candidate PS points using the amplitude deviation index. Subsequently, the linear deformation rate and elevation errors are inverted based on the linear deformation model. The residual errors are then decomposed to obtain nonlinear deformation information, ultimately providing surface deformation data. The specific process is shown in Figure 4.
(2)
SBAS-InSAR
The SBAS-InSAR technique was proposed by Berardino et al. [24]. Its core idea is to group all SAR images into small baseline sets based on spatial and temporal baselines, ensuring that the baseline distances within each subset are small. Ground surface deformation time-series within each subset are computed using the least squares method, and the singular value decomposition (SVD) method is employed to address possible ill-posed parameter solutions between subsets. By combining multiple interferograms, a linear deformation model is established based on high-coherence points, allowing the inversion of linear deformation rates and elevation errors. Residual errors are then decomposed to obtain nonlinear deformation information. This study adopts an enhanced SBAS-InSAR technique, integrating PS and SBAS methods, and simultaneously processes both PS and DS pixels. This approach enables the measurement of nonlinear displacement trends in both PS and DS pixels, providing more comprehensive surface deformation information [25,26].
Obtain N + 1 SAR images covering the same area, select one image as the reference image, and then perform registration, resampling, and removal of flat earth and topographic phases. Based on reasonable temporal and spatial baseline thresholds, M interferograms are generated, where M interferometric pairs must satisfy the inequality (N + 1)/2 < M < N(N + 1)/2. The phase value of the pixel (x, y) in the i-th (i = 1, …, M) differential interferogram is given by the following:
ϕ x ( x , y ) = ϕ ( t B , x , y ) ϕ ( t A , x , y ) ϕ d e f o , ( x , y ) , i + ϕ t o p o , ( x , y ) , i + ϕ a t m , ( x , y ) , i + ϕ n o i s e , ( x , y ) , i
where tA and tB (tA > tB) represent the acquisition times of the SAR images corresponding to the i-th interferogram, ϕ d e f o , ( x , y ) , i is the deformation phase component corresponding to the displacement in the slant range direction between tB and tA, ϕ t o p o , ( x , y ) , i is the topographic phase error, ϕ a t m , ( x , y ) , i is the atmospheric phase error, and ϕ n o i s e , ( x , y ) , i is the noise phase error [27,28].
The baseline estimation of the image is first performed, using the image from 21 May 2022, as a master image (Figure 5). It is then registered and resampled with all other images. After applying terrain flattening, interferometry, and phase unwrapping, differential interferograms and unwrapped phase images are obtained. Subsequently, residual phases are removed through orbit refinement and further terrain flattening. A linear deformation model is constructed based on high-coherence points, and the linear deformation rate and elevation errors are inverted. The residual errors are decomposed to extract nonlinear deformation information. Surface deformation time-series data are obtained using the SVD method. The flowchart of this process is shown in Figure 6.

2.3.2. The Gray System Theory

The gray system theory, first proposed by renowned scholar Professor Deng Julong, is a scientific theory that includes various analysis methods. One such method is gray relational analysis, which performs quantitative analysis on the development of dynamic processes. By comparing the geometric relationships of statistical data within the system, the method determines the gray relational degree between a reference sequence and each comparative sequence. Finally, based on the magnitude of the relational degrees, the evaluation indicators are ranked. The specific steps are as follows.
First, assume that {xt(0)} is the reference sequence (also known as. the parent sequence), {xt(i)} is the comparison sequence (also known as. the subsequence), (t = 1, 2, …, n, i = 1, 2, …, m), constructs the matrix x.
x i 0 = x 1 0 x 1 1 x 1 m x 2 0 x 2 1 x 2 m x n 0 x n 1 x n m
where i = 0, 1, 2, …, m, are the labels of the reference factor (i = 0) and the m comparison factors; t = 1, 2, …, n, are the sample numbers; and xt(i) is the dimensionless quantized function value of the ith indicator for the tth sample point.
ε t i = m i n + ρ m a x t i + ρ m a x
where ε t i is the correlation coefficient between the i-th comparison sequence and the reference sequence at the t-th sample point; and ρ is the discrimination coefficient, with a value in the range of (0, 1), and usually takes the value of 0.1 to 0.5.
t i = x t i x t 0
m a x = m a x t m a x i x t i x t 0
m i n = m i n t m i n i x t i x t 0
The correlation coefficient ε t i obtained from the above calculation and the correlation between the comparison sequence and the reference sequence is calculated by the following equation:
r i = 1 n i = 1 n ε t i
where r i is the correlation between the i-th comparison sequence and the reference sequence [29,30].

2.3.3. Surface Deformation Prediction Model

(1)
LSTM
LSTM (Long Short-Term Memory) improves upon traditional RNNs by effectively addressing the issues of gradient explosion and vanishing gradients, making it particularly successful in the field of time-series prediction [31,32]. As shown in Figure 7, LSTM introduces three gating mechanisms—forget gate, input gate, and output gate—along with a memory cell to regulate the flow of information. These mechanisms enable LSTM to selectively retain or discard information, enhancing the model’s ability to process and learn from long-sequence data. This architecture empowers LSTM to capture both short-term and long-term dependencies, making it a robust tool for sequential data modeling [33].
The data are calculated in the forget gate as shown in Equation (9).
f t = σ W f h t 1 , x t + b f
where σ is the sigmoid activation function. ht−1 and xt are the hidden state and input vector at the moment t − 1, respectively, and Wf, as well as bf, represent the weight value and bias value, respectively. When the value of ft output is close to 0, it means that the previous data are forgotten, while when it is close to 1, it indicates that the previous data are memorized.
The data are calculated in the input gate as shown in Equations (10) and (11).
i t = σ W i h t 1 , x t + b i
C ~ t = t a n h W C h t 1 , x t + b c
where σ is the sigmoid activation function. Wi and Wc are the updated weights, bi is the bias of the input gate, and bc is the deviation.
After the computation of the forget gate and the input gate is completed and the memory cell is updated, we have the following:
C t = f t × C t 1 + i t × C ~ t
The data are first output at the output gate via the sigmoid layer output Ot, which is used to determine which parts of the cell state values will be output. The cell state is then processed by the tanh function, by which operation of the cell state values are scaled to between −1 and 1. Finally, the result is output by multiplying it with the output of the sigmoid gate. The calculation formula is given below.
o t = σ W o h t 1 , x t + b o
h t = o t × t a n h C t
where ht represents the new output value and Wo and bo are the deviations of the updated weights and of the output gate, respectively [34,35].
(2)
Transformer
The Transformer is an encoder–decoder architecture, with both the encoder and decoder consisting of multiple stacked Transformer blocks [36,37,38]. At the heart of the Transformer module lies the multi-head attention mechanism, which is composed of multiple independent and parallel attention mechanisms. This design allows the model to focus on information from different subspaces, enabling the attention mechanism to capture the multidimensional features of input data effectively [39,40,41]. Additionally, the Transformer leverages layer normalization (LN) and skip connections to enhance training efficiency and predictive performance. These structural innovations contribute to the model’s robustness and ability to handle complex data dependencies [42,43]. The core computational formula of the Transformer model is as follows:
Attention Q , K , V = softmax Q K T / d K + B V
h e a d i = A t t e n t i o n Q W i Q , K W i K , V W i V
M u l t i H e a d Q , K , V = C o n c a t h e a d 1 , , h e a d h W O
where Q, K, and V represent query vector, key vector, and value vector, respectively; WQ, WK, WV, and WO are trainable convolutional kernels; headi denotes the ith attention head, where i = 1, 2, …, h; h is the number of attention heads; d K denotes the dimensionality of the key vectors; B is the relative positional bias, which serves to enhance the performance of the network model; softmax denotes the activation function; Concat denotes the splicing function; Attention denotes the attention function; and MultiHead denotes the multi-head self-attention function [44,45]. Figure 8 shows the overall structure of the Transformer network model.
(3)
Efficient Additive Attention
Efficient Additive Attention (EAA) is the core technique employed to improve model performance in this study. Its key innovation lies in replacing traditional quadratic matrix multiplication with linear element-level multiplication to reduce computational complexity. In conventional Transformer models, the self-attention operation is computationally expensive due to the quadratic complexity (O(n2)) with respect to the input sequence length. EAA introduces an additive attention mechanism to compute the global context through the interaction of the query and key matrices via linear transformation, thus reducing this high computational cost.
In the EAA mechanism, the input embedding matrix is first transformed into query (Q) and key (K) matrices using separate learnable weight matrices Wq and Wk. The query matrix is then multiplied by a learnable vector Wa to obtain the attention weight of the query, generating a global attention query vector. This attention vector is pooled to produce a single global query vector, which is then used to encode the interaction between the query and key matrices through element-wise multiplication to form a global context. This global context can effectively capture the global information while maintaining linear complexity in relation to the sequence length, making it more efficient than the multi-head self-attention (MHSA) mechanism, which has quadratic complexity.
Specifically (see Figure 9), the input embedding matrix x is transformed into a query matrix Q and a key matrix K via two matrices Wq and Wk, where Q, KRn×d and Wq, WkRd×d, n is the labeling length and d is the dimension of the embedding vector. Then, the query matrix Q is multiplied by the learnable parameter vector WaRd to learn the attention weight of the query, generating the global attention query vector αRn, computed as follows:
Q = x W q ,   K = x W k
α = Q W a d
A pooling operation on the query matrix based on the learned attention weights yields a single global query vector qRd, i.e., the following:
q = i = 1 n α i     Q i
The interaction between the global query vector q and the key matrix K is encoded through element-wise multiplication, and the resulting context vector is then transformed using a linear transformation to form the global context C.
C = K q
The interaction between the global query vector qRd and the key matrix KRn×d is encoded using element-by-element multiplication to form the global context Rn×d. This matrix is similar to the attention matrix in MHSA in terms of its ability to capture information from each token and its flexibility to learn the relevance of the input sequence. It is computationally less expensive compared to MHSA and has linear complexity with respect to the marker length [46]. Inspired by the Transformer architecture, a hidden representation of tokens is learned using a linear transformation layer employed for query key interactions. The output of Efficient Additive Attention can be described as follows:
x ^ = Q ^ + T K     q
where Q ^ denotes the normalized query matrix and T denotes the linear transformation.
The LE-Transformer model, enhanced with the EAA mechanism, predicts foundation pit deformation using time-series InSAR data. By integrating environmental factors such as rainfall, subway construction, and soil temperature, the model effectively captures both local and global contexts. The EAA mechanism enables the model to process large data sequences quickly, making it suitable for real-time foundation pit stability monitoring. By replacing traditional attention mechanisms, the LE-Transformer improves prediction accuracy and reduces computational overhead, providing a robust framework for foundation pit deformation prediction and early warning systems in urban infrastructure projects.
(4)
LE-Transformer Model Architecture
The LE-Transformer network is a deep learning architecture that integrates LSTM, Efficient Additive Attention (EAA), and Transformer modules, designed to process complex sequential data and perform effective feature extraction and prediction. As illustrated in Figure 10, the architecture combines the strengths of its three core components. Below is a detailed description of the network architecture.
(1)
Data preprocessing module: The data preprocessing module normalizes the input data to a range between 0 and 1, improving the training efficiency and accuracy of the model. This operation ensures that the data can be effectively handled in subsequent modules, laying the foundation for accurate feature learning by the model.
(2)
LSTM module: The LSTM module initializes hidden states and cell states before performing forward propagation through LSTM layers. It outputs features that contain rich temporal information about the input sequences, providing a robust representation for the subsequent attention mechanism.
(3)
EAA Module: The EAA module transforms the input embedding matrix into query and key matrices through specific matrix operations, followed by normalization. The query matrix is multiplied by a learnable vector to generate attention weights, producing a global attention query vector. This query vector is pooled into a single global query vector, and element-wise multiplication is applied to encode its interaction with the key matrix to form a global context. EAA replaces traditional matrix multiplications with linear operations, reducing computational complexity. By precisely focusing on key factors in multivariate data, this module provides weighted features to the Transformer module, enabling the model to capture critical information and enhance the accuracy of pit deformation prediction.
(4)
Transformer module: The Transformer module processes the weighted features provided by the EAA module. Using its multi-head self-attention mechanism, it captures global dependencies in the data. The module further employs a feedforward network for nonlinear transformations, while residual connections and layer normalization help mitigate gradient issues during the training of deep networks.
(5)
Output layer: The features processed by the Transformer module are passed to a fully connected layer. This layer adjusts weight based on the training data, mapping the input features to the final predicted values. The output layer generates accurate predictions for pit deformation.
Figure 10. LE-Transformer model architecture.
Figure 10. LE-Transformer model architecture.
Remotesensing 17 01106 g010
To update the parameters of a deep learning network model, the loss function is utilized to evaluate the error between the true values and the predicted values. In this study, which focuses on time-series prediction tasks, the goal is to minimize the error between the true values and the predicted values. Therefore, the mean squared error (MSE) function is chosen as the loss function for model training. The mathematical expression for the MSE loss function is as follows:
M S E = 1 r g = 1 r d g 2
where r is the sample size of the training samples; and dg denotes the error between the predicted and true values.
(5)
Training and Evaluation
After constructing the LE-Transformer model, it was trained and evaluated to ensure its validity and accuracy. The dataset used in this study consisted of data related to the 220 kV Port Substation foundation pit project in Nansha District, Guangzhou, including InSAR deformation rates and multiple influencing factors. The dataset was divided into training (70%), validation (20%), and test (10%) sets. The data were preprocessed and normalized to 0–1 to improve the training efficiency, and the Adam optimizer was used to minimize the prediction and true value errors. During training, the model learns the prediction based on historical data, calculates the prediction value with forward propagation on the training set, calculates the loss with the loss function, updates the parameters with backward propagation, and calculates the loss and the validation loss on the training and validation sets at the same time to monitor the training effect and the generalization ability. After training, the model is evaluated in the test set with RMSE, MAE, MAPE, R2, and other indicators, and the model prediction results are back-normalized and compared with the real values to judge its prediction ability for pit deformation.

3. Results

3.1. InSAR Monitoring Results

The foundation pit project commenced in May 2023 and was completed in December 2023. Using 72 ascending-track Sentinel-1A synthetic aperture radar images, combined with PS-InSAR and SBAS-InSAR techniques, vertical ground deformation monitoring results for the pit site from 2020 to 2024 were obtained. The final deformation results were primarily based on SBAS-InSAR monitoring, which is due to its ability to process multiple small baseline subsets of synthetic aperture radar images. This provides unique advantages in capturing the spatial variation in large-scale ground deformation. Such dense spatial coverage allows for a more comprehensive and detailed presentation of deformation patterns, which is crucial for understanding the complex deformation processes in the foundation pit and its surrounding regions, especially in areas with significant spatial heterogeneity. The ground deformation time-series was analyzed as follows.
Early Stage (January–April 2020): During the preparatory stage, the site was selected for power infrastructure construction, and the region was generally stable. The observed deformation, as shown in the SBAS-InSAR results (Figure 11a), indicated a cumulative settlement ranging from 1 to 4 mm. This minor deformation can be attributed to factors such as soil consolidation, slight groundwater table fluctuations, and initial disturbances from preparatory construction activities, including equipment movements and site clearing. Despite the absence of significant excavation work at this stage, these factors contributed to the observed surface settlement.
Middle Stage (May–December 2023): The soil composition in the pit area consisted of sand, clay, and peat, which are prone to wet collapse and deformation during rainfall. During this period, the total rainfall was 1775 mm, with distinct dry and wet seasons. The rainy season (May–October) accounted for 85% of the annual rainfall. To investigate the impact of rainfall on surface deformation, five subsidence time-series points (P1–P5) were selected within the settlement zones (Figure 11c). The relationship between settlement and monthly rainfall (Figure 11d) shows the following: All five points exhibited nonlinear settlement, which was strongly correlated with rainfall. Deformation fluctuated with rainfall, and higher rainfall led to accelerated deformation. The sudden increase in subsidence during the rainy season was primarily caused by the saturation of the soil, which significantly reduced its strength and load-bearing capacity. In particular, the peat and clay layers, which are highly susceptible to wet collapse, experienced a reduction in shear strength due to the intense rainfall. The excess moisture from the rainfall decreased the friction between soil particles, resulting in accelerated surface deformation. Additionally, soil temperature played a role in influencing the soil’s moisture retention and strength. During the rainy season, higher temperatures, particularly in May and June, increased evaporation rates, affecting the soil’s ability to retain moisture. This caused fluctuating soil moisture levels, further complicating the deformation processes. The combined effects of temperature and moisture fluctuations exacerbated the deformation, especially in the peat and clay layers, which are sensitive to both temperature and moisture changes.
While the effects of rainfall on deformation were noticeable, they were temporary. After the rainfall stopped, surface deformation gradually returned to a normal settlement rate. Notably, points P3 and P5 exhibited the fastest settlement, with maximum settlements of −32.1 mm and −16.6 mm, respectively. This rapid settlement was largely due to the combined effects of nearby subway construction, residential buildings, and the high amount of rainfall. The subway construction added additional load to the soil, compounding the deformation caused by the saturated soil. Residential buildings contributed to local settlement by exerting extra pressure on the soil. Furthermore, rainfall exacerbated the deformation by weakening the soil structure.
In the months of July and August, heavy rainfall did not cause dramatic settlement. This can be explained by the infiltration of rainwater raising the groundwater table, which temporarily reduced the pore water pressure. As a result, the soil became more stabilized, preventing substantial surface settlement in the short term.
Point P4, which exhibited slow settlement under normal conditions, showed accelerated deformation during August–September. This was largely due to the heavy rainfall during this period, which saturated the soil, and its relative distance from the subway lines and high-rise buildings. The absence of nearby structural loading (such as subway tunnels or high buildings) meant that rainfall had a more significant impact on the settlement at this point.
Following the end of the rainy season, reduced rainfall and rising temperatures during November caused higher soil temperatures and surface drought conditions. The increase in soil temperature, combined with the decrease in groundwater levels, led to accelerated evaporation and further reduction in soil moisture content. While the soil’s strength and load-bearing capacity did decline, this did not result in a re-acceleration of surface settlement. Instead, surface deformation stabilized after the rainy season, and the settlement rate gradually returned to normal levels as the soil regained some stability despite the reduced moisture content.
Late Stage (January–September 2024): During the late stage and after the completion of the foundation pit project, the cumulative deformation in the overall region decreased and tended toward stability (Figure 11b). Most of the cumulative settlement in the pit area ranged between −3 mm and 1 mm, indicating that the foundation structure provided support and constraint. Additionally, the cumulative deformation in the surrounding area also decreased and stabilized, possibly due to the foundation structure reducing the influence of external factors on pit deformation.
To evaluate the accuracy of InSAR monitoring results, a correlation analysis was performed between the monitoring outcomes of the PS-InSAR and SBAS-InSAR methods. The results demonstrated a high level of consistency, with a correlation coefficient of 0.92, indicating good monitoring precision. As shown in Figure 12, most data points are concentrated in the 5–15 mm range, with a few scattered between −10 and 5 mm. The differences may arise from variations in the implementation processes of the two methods, such as differences in the construction of connection graphs and the selection of Ground Control Points (GCPs), which lead to variations in deformation rate values. In this study, the SBAS-InSAR pixel values were extracted using highly coherent PS points from PS-InSAR. Since SBAS-InSAR provides a denser distribution of monitoring points, the coherence of PS points may introduce slight biases in the SBAS pixel values. Overall, the mutual validation between PS-InSAR and SBAS-InSAR results supports the reliability of InSAR monitoring to a certain extent. This demonstrates the robustness of InSAR techniques for monitoring surface deformation with high precision.

3.2. Analysis of Influencing Factors

The analysis of ground settlement causes at the 220 kV Port Substation foundation pit in Nansha District, Guangzhou, reveals that the deformation is influenced by both anthropogenic factors (subway construction and residential buildings) and natural factors (rainfall, soil temperature, and hydrogeology). This classification aligns with the findings of scholars such as MA Yuanyuan [47] and LI Hong-xia [48] regarding the causes of ground settlement. To facilitate input into the prediction model, qualitative factors (subway construction and residential buildings) were quantitatively processed. By creating buffer zones of varying widths to cover settlement areas, these qualitative factors were converted into raster data. The processed quantitative data for subway construction and residential buildings are shown in Figure 13.
Settlement points within the pit area and its surroundings were selected, and the corresponding data for subway construction, residential buildings, rainfall, soil temperature, and hydrogeology were extracted to these settlement points using the multi-value extraction-to-point technique. To evaluate whether these five influencing factors are suitable as predictors for urban ground settlement, a gray relational analysis was performed between each factor and the settlement values.
The deformation point sample values of the five main controlling factors—subway construction, residential buildings, rainfall, soil temperature, and hydrogeology—were selected as comparative sequences, with settlement values serving as the reference sequence. The results are shown in Table 1. It is evident that the selected five influencing factors exhibit a high degree of correlation with settlement values, aligning with the settlement driving factors proposed by scholars. This verifies the rationality and reliability of using these factors as predictors for pit settlement.

3.3. LE-Transformer Prediction Results

The Inverse Distance Weighted (IDW) interpolation method was applied to process pit deformation data to more reasonably estimate the values of unknown points. This method assigns weights based on the distance between the point to be interpolated and known data points, with closer distances receiving higher weights [49]. Using this approach, deformation rate maps from 2020 to 2024 were generated, providing intuitive spatiotemporal variation information for subsequent analysis. Additionally, the settlement time-series of 1556 observation points were selected as training samples.
The grid search method exhaustively evaluates all possible parameter combinations to find the optimal configuration for improving model prediction accuracy [50]. In this study, grid search was applied to optimize the number of network layers (K) and the number of hidden layer nodes (S). Figure 14 presents the prediction heatmap, illustrating the distribution of model prediction errors under different combinations of network layers and hidden layer nodes. The color intensity in the heatmap represents the magnitude of prediction errors. The analysis revealed that the number of layers and hidden nodes were not the only factors influencing prediction accuracy. For a configuration of five network layers and 128 hidden layer nodes, the model achieved a good balance between capturing complex features in the data and avoiding overfitting. Fewer layers might fail to fully capture deep-level information, while excessive layers could lead to overfitting, diminishing the model’s generalization ability. Therefore, the optimal configuration for the model in this study was determined to be five layers with 128 hidden nodes.
To evaluate the model’s prediction performance in various deformation areas, random points under different conditions (including the pit site and surrounding areas) were selected within the study area for analysis. A comparison between the model’s prediction results and the SBAS-InSAR method results (Figure 15) shows that the two exhibit consistent overall trends, although differences were observed in finer details. The model effectively captures the overall deformation trends, while SBAS-InSAR, due to its technical characteristics, is more sensitive to minor deformations, leading to slight deviations. However, the model demonstrates advantages in handling continuous time-series data and integrating the effects of multiple factors, providing a more comprehensive prediction of deformation trends.
This study compares the impact of univariate and multivariate data on deformation prediction results. Univariate data consist solely of InSAR time-series deformation data, while multivariate data incorporate additional factors, such as rainfall. The temporal span of the input deformation monitoring data is from 8 January 2020 to 25 September 2023, with the model’s prediction period extending from 19 October 2023 to 7 September 2024. As shown in Figure 16, the prediction errors between the forecasted and actual cumulative deformation reveal that the multivariate model outperforms the univariate model in regions with significant deformation. It provides a more accurate reflection of deformation trends at key points in major deformation zones and aligns more closely with the SBAS-InSAR results. In contrast, the univariate model, due to the absence of additional influencing factors, often exhibits an “inertia” effect. This results in challenges when adjusting predictions under changing environmental conditions, leading to either overestimation or underestimation of deformation. In areas distant from the pit site, where deformation is minor and the impact of influencing factors is limited, the differences between the two models are negligible. Nevertheless, the univariate model can still partially capture deformation in these regions. However, in areas with complex deformation, the multivariate model demonstrates significant advantages. This analysis underscores the importance of integrating multivariate data to achieve accurate predictions of pit-induced deformation. The selected factors play a critical role in significantly improving the model’s predictive capability in regions experiencing complex deformation.

3.4. Comparison with Other Models

The EAA mechanism offers unique advantages within the LE-Transformer combination model. Unlike traditional quadratic matrix multiplications, EAA uses linear element-wise multiplication, avoiding complex computations. By calculating global context through linear transformations of query and key interactions, it efficiently captures global information. When processing pit deformation data, EAA quickly focuses on critical features and accurately identifies relevant factors in multivariate data, enhancing prediction accuracy in complex deformation regions. Comparative studies were conducted to evaluate its impact on dataset training. As shown in Table 2, EAA’s performance is comparable to Scaled Dot-Product Attention and Transformer’s multi-head self-attention mechanism, with similar MSE and MAE values. However, EAA demonstrates practical improvements in training datasets. Moreover, the computational complexity of EAA, Scaled Dot-Product Attention, and multi-head attention remains efficient, enabling the prediction of extended time-series data under similar hardware conditions.
To assess the superiority of the introduced prediction model, LSTM, Transformer, GRU, and Informer models were trained and evaluated for multivariate sequence deformation prediction. In addition to MSE, other performance metrics such as MAPE, RMSE, and R2 were employed to compare model accuracy. The LE-Transformer model demonstrated predictions closest to the SBAS-InSAR results, with an MAPE of only 2.5%, proving its effectiveness. As shown in Table 3, the LE-Transformer model outperformed other models in accuracy on the multivariate dataset. This indicates that the inclusion of the EAA mechanism enhances the model’s capability in handling multivariate sequences, effectively processing long input sequences while achieving high prediction accuracy at the point scale. Overall, the LE-Transformer model exhibited superior performance, making it a reliable tool for predicting surface deformation in foundation pit projects.

4. Discussion

This study successfully applied the LE-Transformer model to predict foundation pit deformation, showing its effectiveness in capturing both temporal and spatial dependencies in time-series data. With an MAPE of 2.5%, the model outperforms traditional methods, including LSTM, GRU, and Transformer, by integrating multiple factors such as rainfall, subway construction, and soil temperature.
The LE-Transformer’s use of EAA improved computational efficiency, enabling the model to process multivariate data with reduced complexity. This approach addresses limitations found in traditional multi-head self-attention mechanisms, which often suffer from quadratic complexity.
However, the model’s generalizability may be affected by data sparsity or extreme environmental conditions. Future studies should focus on enhancing its robustness by incorporating more diverse data sources and improving its performance in less data-rich scenarios. Additionally, the model’s accuracy could be further improved by exploring nonlinear interactions between influencing factors and testing its applicability across varied geological conditions.
Comparison with existing research demonstrates that, unlike traditional empirical and neural network models, the LE-Transformer offers a more comprehensive solution for deformation prediction by integrating InSAR time-series data with environmental factors. This makes it a promising tool for urban infrastructure projects where deformation prediction and early warning systems are crucial.
Future work should focus on improving the model’s integration with real-time monitoring systems and expanding its application to other areas, such as geological disaster monitoring and urban planning. Incorporating factors like vegetation cover and slope orientation could also further enhance its predictive accuracy.

5. Conclusions

This study introduced an innovative approach for monitoring and predicting foundation pit deformation by integrating time-series InSAR data with the LE-Transformer model, a deep learning framework combining LSTM, Efficient Additive Attention (EAA), and Transformer mechanisms. Using the 220 kV Port Substation foundation pit in Guangzhou’s Nansha District as a case study, the key findings and contributions of this research are summarized as follows.
(1)
Deformation data for the 220 kV Port Substation foundation pit in Guangzhou’s Nansha District were obtained using two InSAR techniques, PS-InSAR and SBAS-InSAR. The cumulative deformation ranged from −10 mm to +15 mm, with a high correlation coefficient of 0.92 between the results from both methods. This demonstrates the effectiveness of InSAR-based monitoring for large-scale urban infrastructure projects, offering superior coverage and precision compared to traditional methods.
(2)
Gray relational analysis identified key factors affecting deformation, including rainfall, subway construction, residential buildings, soil temperature, and hydrogeology. Rainfall was the most influential factor, with a correlation of 0.838. These findings underscore the importance of considering both environmental and infrastructural factors in foundation pit settlement, which is crucial for risk management in urban development projects.
(3)
The LE-Transformer model outperformed traditional prediction models, achieving a mean absolute percentage error (MAPE) of 2.5%. By incorporating both spatial and temporal data, the model demonstrated superior capability in predicting complex deformation trends. Its ability to handle multivariate data provides a more comprehensive understanding of the deformation mechanisms in foundation pit projects.
This integrated methodology provides a reliable framework for monitoring foundation pit deformation and can enhance risk prevention and early warning systems. Future work should explore the integration of more factors into the model to further improve prediction accuracy.

Author Contributions

Conceptualization, Y.L.; Methodology, B.H.; Formal analysis, R.L.; Investigation, W.L. (Weifeng Lu) and F.Z.; Resources, B.H.; Writing—original draft, W.L. (Wen Li); Writing—review & editing, B.H.; Supervision, B.H. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data Availability Statement

The original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Zhou, Y.; Su, W.; Ding, L.; Luo, H.; Love, P.E. Predicting safety risks in deep foundation pits in subway infrastructure projects: Support vector machine approach. J. Comput. Civ. Eng. 2017, 31, 04017052. [Google Scholar] [CrossRef]
  2. Xie, Z.; Tu, F. Reliability analysis of electronic total station and inclinometer in deformation monitoring of foundation pit. Geotech. Investig. Surv. 2009, 37, 81–84. [Google Scholar]
  3. Zhang, W.; Xiang, D.; Yu, Z.; Hua, X. Horizontal displacement monitoring and results analysis of pit surveyed by total station. Surv. Mapp. Geol. Miner Resour. 2009, 25, 4–6. [Google Scholar]
  4. Wu, J.; Peng, L.; Li, J.; Zhou, X.; Zhong, J.; Wang, C.; Sun, J. Rapid safety monitoring and analysis of foundation pit construction using unmanned aerial vehicle images. Autom. Constr. 2021, 128, 103706. [Google Scholar]
  5. Wang, Y.; Feng, G.; Li, Z.; Luo, S.; Wang, H.; Xiong, Z.; Zhu, J.; Hu, J. A Strategy for Variable-Scale InSAR Deformation Monitoring in a Wide Area: A Case Study in the Turpan–Hami Basin, China. Remote Sens. 2022, 14, 3832. [Google Scholar] [CrossRef]
  6. Zhang, G.; Xu, Z.; Chen, Z.; Wang, S.; Liu, Y.; Gong, X. Analyzing surface deformation throughout China’s territory using multi-temporal InSAR processing of Sentinel-1 radar data. Remote Sens. Environ. 2024, 305, 114105. [Google Scholar] [CrossRef]
  7. Zhang, J.; Ke, C.; Shen, X.; Lin, J.; Wang, R. Monitoring Land Subsidence along the Subways in Shanghai on the Basis of Time-Series InSAR. Remote Sens. 2023, 15, 908. [Google Scholar] [CrossRef]
  8. Sun, L.; Liu, S.; Zhang, L.; He, K.; Yan, X. Prediction of the displacement in a foundation pit based on neural network model fusion error and variational modal decomposition methods. Measurement 2025, 240, 115534. [Google Scholar] [CrossRef]
  9. Li, Q.; Yao, C.; Yao, X.; Zhou, Z.; Ren, K. Time Series Prediction of Reservoir Bank Slope Deformation Based on Informer and InSAR: A Case Study of Dawanzi Landslide in the Baihetan Reservoir Area, China. Remote Sens. 2024, 16, 2688. [Google Scholar] [CrossRef]
  10. Du, J.; Yin, K.; Lacasse, S. Displacement prediction in colluvial landslides, Three Gorges Reservoir, China. Landslides 2012, 10, 203–218. [Google Scholar] [CrossRef]
  11. Shi, J.; Wang, S.; Qu, P.; Shao, J. Time series prediction model using LSTM-Transformer neural network for mine water inflow. Sci. Rep. 2024, 14, 18284. [Google Scholar] [CrossRef] [PubMed]
  12. Li, W.; Law, K.L.E. Deep Learning Models for Time Series Forecasting: A Review. IEEE Access 2024, 12, 92306–92327. [Google Scholar] [CrossRef]
  13. Li, Y.; Zhu, Z.; Kong, D.; Han, H.; Zhao, Y. EA-LSTM: Evolutionary attention-based LSTM for time series prediction. Knowl.-Based Syst. 2019, 181, 104785. [Google Scholar] [CrossRef]
  14. Chen, Y.; He, Y.; Zhang, L.; Chen, B.; He, X.; Pu, H.; Cao, S.; Gao, L.; Yang, W. Surface deformation prediction based on TS-InSAR technology and long short-term memory networks. Natl. Remote Sens. Bull. 2022, 26, 1326–1341. [Google Scholar] [CrossRef]
  15. Bharti, M.K.; Wadhvani, R.; Gyanchandani, M.; Gupta, M. Transformer-Based Multivariate Time Series Forecasting. In Proceedings of the 2024 IEEE International Students’ Conference on Electrical, Electronics and Computer Science (SCEECS), Bhopal, India, 24–25 February 2024; pp. 1–6. [Google Scholar]
  16. Wang, J.; Li, C.; Li, L.; Huang, Z.; Wang, C.; Zhang, H.; Zhang, Z. InSAR time-series deformation forecasting surrounding Salt Lake using deep transformer models. Sci. Total Environ. 2023, 858, 159744. [Google Scholar] [CrossRef]
  17. Tong, G.; Ge, Z.; Peng, D. RSMformer: An efficient multiscale transformer-based framework for long sequence time-series forecasting. Appl. Intell. 2024, 54, 1275–1296. [Google Scholar] [CrossRef]
  18. Zhang, J.; Luo, Z.; Yang, Z. Research on Air Quality Prediction Based on LSTM-Transformer with Adaptive Temporal Attention Mechanism. In Proceedings of the 2023 2nd International Conference on Artificial Intelligence and Intelligent Information Processing (AIIIP), Hangzhou, China, 27–29 October 2023; pp. 320–323. [Google Scholar]
  19. Ferretti, A.; Prati, C.; Rocca, F. Permanent scatterers in SAR interferometry. IEEE Trans. Geosci. Remote Sens. 2001, 39, 8–20. [Google Scholar] [CrossRef]
  20. Ferretti, A.; Fumagalli, A.; Novali, F.; Prati, C.; Rocca, F.; Rucci, A. A New Algorithm for Processing Interferometric Data-Stacks: SqueeSAR. IEEE Trans. Geosci. Remote Sens. 2011, 49, 3460–3470. [Google Scholar] [CrossRef]
  21. Fornaro, G.; Verde, S.; Reale, D.; Pauciullo, A. CAESAR: An Approach Based on Covariance Matrix Decomposition to Improve Multibaseline–Multitemporal Interferometric SAR Processing. IEEE Trans. Geosci. Remote Sens. 2015, 53, 2050–2065. [Google Scholar] [CrossRef]
  22. Liu, Y.; Fan, H.; Wang, L.; Zhuang, H. Monitoring of surface deformation in a low coherence area using distributed scatterers InSAR: Case study in the Xiaolangdi Basin of the Yellow River, China. Bull. Eng. Geol. Environ. 2020, 80, 25–39. [Google Scholar] [CrossRef]
  23. Ye, K.; Wang, Z.; Wang, T.; Luo, Y.; Chen, Y.; Zhang, J.; Cai, J. Deformation Monitoring and Analysis of Baige Landslide (China) Based on the Fusion Monitoring of Multi-Orbit Time-Series InSAR Technology. Sensors 2024, 24, 6760. [Google Scholar] [CrossRef] [PubMed]
  24. Berardino, P.; Fornaro, G.; Lanari, R.; Sansosti, E. A new algorithm for surface deformation monitoring based on small baseline differential SAR interferograms. IEEE Trans. Geosci. Remote Sens. 2002, 40, 2375–2383. [Google Scholar] [CrossRef]
  25. Lanari, R.; Mora, O.; Manunta, M.; Mallorquí, J.J.; Berardino, P.; Sansosti, E. A small-baseline approach for investigating deformations on full-resolution differential SAR interferograms. IEEE Trans. Geosci. Remote Sens. 2004, 42, 1377–1386. [Google Scholar] [CrossRef]
  26. Parizzi, A.; Brcic, R.; De Zan, F. InSAR performance for large-scale deformation measurement. IEEE Trans. Geosci. Remote Sens. 2020, 59, 8510–8520. [Google Scholar]
  27. Chen, Y.; Yu, S.; Tao, Q.; Liu, G.; Wang, L.; Wang, F. Accuracy Verification and Correction of D-InSAR and SBAS-InSAR in Monitoring Mining Surface Subsidence. Remote Sens. 2021, 13, 4365. [Google Scholar] [CrossRef]
  28. Du, Q.; Li, G.; Zhou, Y.; Chai, M.; Chen, D.; Qi, S.; Wu, G.; Zhao, X.D. Deformation Monitoring in an Alpine Mining Area in the Tianshan Mountains Based on SBAS-InSAR Technology. Adv. Mater. Sci. Eng. 2021, 2021, 9988017. [Google Scholar] [CrossRef]
  29. Sun, Y.; Liu, S.; Li, L. Grey Correlation Analysis of Transportation Carbon Emissions under the Background of Carbon Peak and Carbon Neutrality. Energies 2022, 15, 3064. [Google Scholar] [CrossRef]
  30. Tsai, M.-S.; Hsu, F.-Y. Application of Grey Correlation Analysis in Evolutionary Programming for Distribution System Feeder Reconfiguration. IEEE Trans. Power Syst. 2010, 25, 1126–1133. [Google Scholar] [CrossRef]
  31. Sherstinsky, A. Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) network. Phys. D Nonlinear Phenom. 2020, 404, 132306. [Google Scholar] [CrossRef]
  32. Waqas, M.; Humphries, U.W. A critical review of RNN and LSTM variants in hydrological time series predictions. MethodsX 2024, 13, 102946. [Google Scholar] [CrossRef]
  33. Liu, J.; Xu, T.; Lu, C.; Yang, J.; Xie, Y. Variational mode decomposition coupled LSTM with encoder-decoder framework: An efficient method for daily streamflow forecasting. Earth Sci. Inform. 2024, 18, 38. [Google Scholar] [CrossRef]
  34. Singh, U.; Saurabh, K.; Trehan, N.; Vyas, R.; Vyas, O.P. GA-LSTM: Performance Optimization of LSTM driven Time Series Forecasting. Comput. Econ. 2024, 1–36. [Google Scholar] [CrossRef]
  35. Tamilselvi, C.; Paul, R.K.; Yeasin, M.; Paul, A.K. Novel wavelet-LSTM approach for time series prediction. Neural Comput. Appl. 2024, 1–10. [Google Scholar] [CrossRef]
  36. Meng, X.; Yang, Y.; Wang, L.; Wang, T.; Li, R.; Zhang, C. Class-Guided Swin Transformer for Semantic Segmentation of Remote Sensing Imagery. IEEE Geosci. Remote Sens. Lett. 2022, 19, 6517505. [Google Scholar] [CrossRef]
  37. Qiu, Y.; Liu, Y.; Zhang, L.; Lu, H.; Xu, J. Boosting Salient Object Detection with Transformer-Based Asymmetric Bilateral U-Net. IEEE Trans. Circuits Syst. Video Technol. 2024, 34, 2332–2345. [Google Scholar] [CrossRef]
  38. Wang, S.; Wang, H.; She, S.; Zhang, Y.; Qiu, Q.; Xiao, Z. Swin-T-NFC CRFs: An encoder–decoder neural model for high-precision UAV positioning via point cloud super resolution and image semantic segmentation. Comput. Commun. 2023, 197, 52–60. [Google Scholar] [CrossRef]
  39. Xiao, T.; Liu, Y.; Huang, Y.; Li, M.; Yang, G. Enhancing Multiscale Representations with Transformer for Remote Sensing Image Semantic Segmentation. IEEE Trans. Geosci. Remote Sens. 2023, 61, 5605116. [Google Scholar] [CrossRef]
  40. Xiao, X.; Guo, W.; Chen, R.; Hui, Y.; Wang, J.; Zhao, H. A Swin Transformer-Based Encoding Booster Integrated in U-Shaped Network for Building Extraction. Remote Sens. 2022, 14, 2611. [Google Scholar] [CrossRef]
  41. Ye, X.; Fang, S.; Sun, F.; Zhang, C.; Xiang, S. Meta Graph Transformer: A Novel Framework for Spatial–Temporal Traffic Prediction. Neurocomputing 2022, 491, 544–563. [Google Scholar] [CrossRef]
  42. Zheng, Z.; Zhong, Y.; Tian, S.; Ma, A.; Zhang, L. ChangeMask: Deep multi-task encoder-transformer-decoder architecture for semantic change detection. ISPRS J. Photogramm. Remote Sens. 2022, 183, 228–239. [Google Scholar] [CrossRef]
  43. Fan, J.; Gao, B.; Ge, Q.; Ran, Y.; Zhang, J.; Chu, H. SegTransConv: Transformer and CNN Hybrid Method for Real-Time Semantic Segmentation of Autonomous Vehicles. IEEE Trans. Intell. Transp. Syst. 2024, 25, 1586–1601. [Google Scholar] [CrossRef]
  44. Ahmed, S.; Nielsen, I.E.; Tripathi, A.; Siddiqui, S.; Ramachandran, R.P.; Rasool, G. Transformers in Time-Series Analysis: A Tutorial. Circuits Syst. Signal Process. 2023, 42, 7433–7466. [Google Scholar] [CrossRef]
  45. Kim, G.h.; Kim, E. Stacked encoder–decoder transformer with boundary smoothing for action segmentation. Electron. Lett. 2022, 58, 972–974. [Google Scholar] [CrossRef]
  46. Shaker, A.; Maaz, M.; Rasheed, H.; Khan, S.; Yang, M.-H.; Khan, F.S. Swiftformer: Efficient additive attention for transformer-based real-time mobile vision applications. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 1–6 October 2023; pp. 17425–17436. [Google Scholar]
  47. Ma, Y.Y.; Chen, Y.B.; Zuo, X.Q.; Ma, W.F.; Wu, W.H. Analysis of land subsidence in Kunming based on SentineFlA data and SBAS technolog. Sci. Surv. Mapp. 2019, 44, 59–66+95. [Google Scholar] [CrossRef]
  48. Li, H.X.; Zhao, X.H.; Chi, H.Y.; Zhang, J.J. Prediction and Analysis of Land Subsidence Based on Improved BP Neural Network Model. J. Tianjin Univ. 2009, 42, 60–64. [Google Scholar]
  49. Mueller, T.G.; Pusuluri, N.B.; Mathias, K.K.; Cornelius, P.L.; Barnhisel, R.I.; Shearer, S.A. Map Quality for Ordinary Kriging and Inverse Distance Weighted Interpolation. Soil Sci. Soc. Am. J. 2004, 68, 2042–2047. [Google Scholar] [CrossRef]
  50. Ma, F.; Sui, L.; Lian, W. Prediction of Mine Subsidence Based on InSAR Technology and the LSTM Algorithm: A Case Study of the Shigouyi Coalfield, Ningxia (China). Remote Sens. 2023, 15, 2755. [Google Scholar] [CrossRef]
Figure 1. Geographical location of foundation pit.
Figure 1. Geographical location of foundation pit.
Remotesensing 17 01106 g001
Figure 2. The workflow of foundation pit deformation prediction based on LE-Transformer.
Figure 2. The workflow of foundation pit deformation prediction based on LE-Transformer.
Remotesensing 17 01106 g002
Figure 3. PS-InSAR spatiotemporal baseline diagram.
Figure 3. PS-InSAR spatiotemporal baseline diagram.
Remotesensing 17 01106 g003
Figure 4. PS-InSAR basic processing flow.
Figure 4. PS-InSAR basic processing flow.
Remotesensing 17 01106 g004
Figure 5. SBAS-InSAR spatiotemporal baseline diagram.
Figure 5. SBAS-InSAR spatiotemporal baseline diagram.
Remotesensing 17 01106 g005
Figure 6. SBAS-InSAR basic processing flow.
Figure 6. SBAS-InSAR basic processing flow.
Remotesensing 17 01106 g006
Figure 7. LSTM neural network structure.
Figure 7. LSTM neural network structure.
Remotesensing 17 01106 g007
Figure 8. Schematic diagram transformer structure.
Figure 8. Schematic diagram transformer structure.
Remotesensing 17 01106 g008
Figure 9. Efficient Additive Attention.
Figure 9. Efficient Additive Attention.
Remotesensing 17 01106 g009
Figure 11. Deformation and related relationships in foundation pit area and surrounding across early, middle, and late stages. (a) Early stage cumulative deformation. (b) Late stage cumulative variables. (c) Middle stage deformation rate. (d) Relationship between settlement values and monthly rainfall during middle stage.
Figure 11. Deformation and related relationships in foundation pit area and surrounding across early, middle, and late stages. (a) Early stage cumulative deformation. (b) Late stage cumulative variables. (c) Middle stage deformation rate. (d) Relationship between settlement values and monthly rainfall during middle stage.
Remotesensing 17 01106 g011
Figure 12. Correlation of PS-InSAR and SBAS-InSAR results.
Figure 12. Correlation of PS-InSAR and SBAS-InSAR results.
Remotesensing 17 01106 g012
Figure 13. Quantitative processing of qualitative factors. (a) Subway buffer zone. (b) Housing construction buffer zone.
Figure 13. Quantitative processing of qualitative factors. (a) Subway buffer zone. (b) Housing construction buffer zone.
Remotesensing 17 01106 g013
Figure 14. Grid search area heat map.
Figure 14. Grid search area heat map.
Remotesensing 17 01106 g014
Figure 15. Comparison of model prediction results with those of SBAS-InSAR method. (a) Training set training results. (b) Test set prediction results.
Figure 15. Comparison of model prediction results with those of SBAS-InSAR method. (a) Training set training results. (b) Test set prediction results.
Remotesensing 17 01106 g015
Figure 16. Comparison chart of prediction errors. (a) Univariate prediction error. (b) Multivariate prediction error.
Figure 16. Comparison chart of prediction errors. (a) Univariate prediction error. (b) Multivariate prediction error.
Remotesensing 17 01106 g016
Table 1. Impact factor correlation for foundation pit settlement.
Table 1. Impact factor correlation for foundation pit settlement.
Controlling FactorSubway ConstructionHousing ConstructionRainfallSoil TemperatureHydrogeology
Relatedness0.7310.7250.8380.7960.824
Table 2. Training effects of different mechanisms.
Table 2. Training effects of different mechanisms.
Self-Attention TypeMSEMAEComplexity
EAA0.0850.122O(n × d)
Multi-Head Attention0.1560.232O(n2 × d)
Scaled Dot-Product Attention0.1830.283O(n2 × d)
Table 3. Effect of different model training.
Table 3. Effect of different model training.
ModelsMSEMAPERMSER2
GRU0.2380.0480.4120.835
LSTM0.2150.0420.3840.857
Transformer0.1950.0390.3690.886
LE-Transformer0.1250.0250.2920.934
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Hu, B.; Li, W.; Lu, W.; Zhao, F.; Li, Y.; Li, R. Integrating InSAR Data and LE-Transformer for Foundation Pit Deformation Prediction. Remote Sens. 2025, 17, 1106. https://doi.org/10.3390/rs17061106

AMA Style

Hu B, Li W, Lu W, Zhao F, Li Y, Li R. Integrating InSAR Data and LE-Transformer for Foundation Pit Deformation Prediction. Remote Sensing. 2025; 17(6):1106. https://doi.org/10.3390/rs17061106

Chicago/Turabian Style

Hu, Bo, Wen Li, Weifeng Lu, Feilong Zhao, Yuebin Li, and Rijun Li. 2025. "Integrating InSAR Data and LE-Transformer for Foundation Pit Deformation Prediction" Remote Sensing 17, no. 6: 1106. https://doi.org/10.3390/rs17061106

APA Style

Hu, B., Li, W., Lu, W., Zhao, F., Li, Y., & Li, R. (2025). Integrating InSAR Data and LE-Transformer for Foundation Pit Deformation Prediction. Remote Sensing, 17(6), 1106. https://doi.org/10.3390/rs17061106

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop