A Comparative Study on Structural Displacement Prediction by Kernelized Regressors under Limited Training Data ^†

Abstract

An accurate prediction of the structural response in the presence of limited training data still represents a big challenge if machine learning-based approaches are adopted. This paper investigates and compares two state-of-the-art kernelized supervised regressors to predict the structural response of a long-span bridge retrieved from spaceborne remote sensing technology. The kernelized supervised procedure is either based on a support vector regression or on a Gaussian process regression. A small set of displacement time histories and corresponding air temperature data are fed into the regressors to predict the actual structural response. Results demonstrate that the proposed regression techniques are reliable, even when only 30% of the training data are used at the learning stage.

1. Introduction

Structural health monitoring (SHM) has brought a practical methodology for ensuring the safety and integrity of civil structures [1,2,3,4,5]. This methodology is based on sensor deployment over the structure to be monitored, data acquisition, modeling, feature extraction, and feature analysis [6,7,8]. The modeling stage can be either physics- or data-based [9,10,11,12]. Sensors are obviously important to any SHM process because the acquired data from the structures provide information on their behavior and current state. Recently, spaceborne remote sensing has become an emerging and practical technology for monitoring large-scale civil structures [13,14] by using synthetic aperture radar (SAR) images [15,16,17,18,19]. Despite some limitations such as speckle noise and low spectral and resolution information, SAR images have become important data for the SHM process to rely on [20,21]. The main product of the remote sensing for SHM is the extraction of structural displacements from said SAR images [22,23,24].

Even if recent progress in SAR-based SHM using the aforementioned displacement responses can be exploited, especially for huge civil structures, some limitations cause obstacles to fully take advantage of this methodology. First, as for any SHM program, the in situ/field measurements are not always trivial. In most cases, field testing and measurements entail high costs, low efficiency, impact on traffic, and damage to the structures. Although the use of non-contact-based sensors, particularly spaceborne remote sensing, significantly copes with the limitations of contact-based sensing and its difficulties regarding in situ measurement, SAR images produce Big Data of huge sizes (in the unit of GB), leading to issues related to memory storage. Second, the displacement is a feature extracted from SAR images. This means that such information is not provided directly from sensor recordings, and feature extraction techniques (like interferometric approaches) appear therefore necessary. In some cases, this results in poor and unreliable displacement data. Third, spaceborne remote sensing cannot provide as rich data/features as other contact-based sensing methods, which may be installed permanently. More precisely, in most practical and long-term SHM projects based on installed contact sensors, it is possible to supply large datasets measured hourly; however, it is difficult to provide so much data from remote sensing. Fourth, it is probable that any in situ/field measurement may contain missing data, which leads to incomplete information for SHM purposes. To address these limitations, the most appropriate solution is to predict the structural response based on measured excitation or input parameters.

In machine learning, data prediction is a well-known problem. To this purpose, it is possible to exploit various regression models based on predictor (independent or input) and response (dependent or output) data [25]. In relation to the SAR-based SHM strategy, the major challenge to face is that structural displacements extracted from the SAR images are limited. In other words, due to the size of such images, few observations are often considered to extract displacement responses. In this case, the use of any regression model for small data may be problematic. The best solution is thus to leverage data expansion techniques. From the viewpoint of regression modeling, support vector regression (SVR) and Gaussian process regression (GPR) are two supervised regressors developed from the concept of kernel trick that expand a low-dimensional feature space to a high-dimensional one with a different kernel function [26,27]. However, the performance of these techniques in the presence of small datasets and the consideration of a limited training ratio have not been explored properly for SAR-based SHM.

This paper mainly intends to compare the SVR and GPR methods for predicting structural responses obtained from a few SAR images, under a tiny and unusual ratio of training data. For this goal, data related to the structural response of a long-span bridge have been considered along with ambient temperature recordings with contact-based sensors. The structural responses have been considered for some areas of the bridge, exploiting 29 SAR images only of Sentinel-A1 in a long-term monitoring scheme. Accordingly, the temperature and displacement samples are divided into training and test sets with a ratio of 30:70. The recorded ambient temperature stems as the major predictor datum, while the structural displacements are considered as the main response for prediction. Results demonstrate that SVR outperforms GPR.

2. Supervised Regression Techniques

2.1. Support Vector Regression

The fundamental principle of SVR is to map the original training data to a higher-dimensional space, and then apply an optimization approach to find a hyperplane that can separate the training data in the transformed space. This hyperplane resembles a function that can predict a target value within a tolerance margin or a decision boundary based on the training data points [28]. Given the predictor data x = {x₁,…,x_n} and response data y = {y₁,…,y_n}, which, in the present case, gather the temperature and displacement points, respectively, the general form of the SVR model can be expressed as $y=w^{T}x+b$, where w denotes the weight vector and b is the bias. Based on this general form, SVR intends to exploit the training data to predict the response data, moving through the following steps: (i) separating the training data into support vectors, (ii) mapping the support vectors into high-dimensional space via a kernel function, and (iii) developing a regression model containing estimated parameters through an optimization process. Based on Mercer’s theorem, the mapping procedure is performed by using different kernel functions. The procedure aims to minimize a convex function subject to constraints; for more details, readers are referred to [28].

To deal with the nonlinear regression problem, the low-dimensional parameter space needs to be mapped into the high-dimensional one by a kernel function ϕ(x), which computes inner product values of mapped points in the feature space stored in a matrix. Therefore, the final SVR model based on any kernel function can be expressed as $y=w^T\varphi \left(x\right)+b$.

2.2. Gaussian Process Regression

GPR is a supervised regression model that predicts data based on the development of a kernel-based probabilistic algorithm and the theory of Gaussian processes [29]. Given the predictor and response data x and y, GPR predicts a response point using new predictor data by introducing latent variables, L(x₁),…,L(x_n) from a GP, and an explicit basis function. For each i = 1,…,n, if L(x_i) and x_i conform to this process, the joint distribution of the random variables L(x₁),…, L(x_n) is Gaussian. If these variables are from a zero mean Gaussian process, one can derive the GPR model as $h{\left(x\right)}^{T}a+L\left(x\right)$, where h(x) denotes a basis function that transforms the predictor data {x₁,…,x_n} into a new vector, and a is the set of the coefficients of this function. As the GPR model is based on the probability theory, $h{\left(x\right)}^{T}a+L\left(x\right)$ can be re-written as

$$\mathrm{Pr}\left(y|L\left(x\right),x\right)\sim N\left(x|h{\left(x\right)}^{T}a+g\left(x\right),{\sigma }^2\right)$$

(1)

where $g\left(x\right)\sim GP\left(0,\varphi \left(x\right)\right)$ is equivalent to a zero-mean GP and ϕ(x) denotes the kernel function (matrix) of the predictor data. Once the GPR model has been developed via the training data, it can predict any new response by means of the conditional distribution in Equation (1).

3. Application

The Dashengguan Bridge is a long-span high-speed railway steel bridge that crosses the Yangtze River in Nanjing, China [20]. The bridge features a large-span continuous steel arch truss with a length of 1615 m. This work focuses on the six main parts of the bridge, with a total length of 1272 m, as shown in Figure 1. The arches consist of three truss planes above the deck. The main truss has a welded, monolithic joint. The members and gusset plates were welded together in the fabrication yard and then transported to the site and spliced outside the joint with high-strength bolts.

3.1. Predictor and Response Data

In a long-term monitoring program between 25 April 2015 and 5 August 2016, 29 SAR images from Sentinel-A1 were used to extract displacement responses in the unit of mm at some critical areas of the bridge, including piers 4–6 and 8–10; see Figure 1. Figure 2 shows the mentioned structural responses at the six piers of the bridge, which were extracted with the persistent scatterer interferometry technique [20]. The ambient temperature was also recorded by contact-based sensors and is shown in Figure 3.

3.2. Prediction Results

To predict the displacement responses, the predictor and response data are divided into the training and test sets. For the training process, a small ratio of 30% is considered so that the training data consist of nine samples, including both the recorded temperature and the extracted displacement points. The remaining 20 samples are incorporated into the test dataset. The Bayesian hyperparameter optimization is adopted to tune the unknown elements of the SVR and GPR, especially the kernel function. On this basis, the linear and squared exponential kernel functions are selected for SVR and GPR, respectively, to map the limited training points into the high-dimensional feature space.

The results of displacement response prediction via SVR and GPR are shown in Figure 4 and Figure 5, respectively. These charts display the scatter plots of the predicted displacements versus their real values, as extracted from the SAR images. When the scatter points are close to the reported straight line, representing a perfect match between the two datasets, one can infer that the prediction process operates well. The comparison between Figure 4 and Figure 5 testifies that SVR outperforms GPR in predicting the displacement data in the case of a small training ratio. For further investigation,

Table 1. Performance evaluation of the kernelized supervised regressor.

Pier No.	Metrics
	R2	RMSE	R2	RMSE
	SVR	GPR	SVR	GPR
4	0.9732	0.8413	10.6044	25.8231
5	0.9475	0.8231	12.3346	22.6547
6	0.9210	0.7603	9.5984	16.7244
8	0.9680	0.9632	6.1634	6.6125
9	0.9875	0.8155	6.7403	25.9005
10	0.9548	0.9701	14.1607	11.5007

compares the numerical outputs of the regression modeling based upon the R-squared (R²) and root-mean-square-error (RMSE) measures; an R-squared value close to 1 is indicative of a good prediction. In Table 1, one can observe that R² and RMSE values relevant to the SVR model are closer to 1 and smaller than the corresponding values concerning the GPR model. Therefore, both the graphical and numerical assessments confirm a better performance of SVR compared to GPR.

4. Conclusions

This paper has compared two kernelized supervised regressors, namely SVR and GPR, to predict the structural displacements extracted from a few SAR images obtained with remote sensing technology in the case of a small ratio of training data. Bayesian hyperparameter optimization has also been applied to tune the unknown elements of the SVR and GPR models, especially their kernel functions.

A limited number of displacement responses of a long-span steel bridge coupled with the relevant ambient temperature values have been considered, to evaluate the capability of the regressors. The results have demonstrated that SVR provides better prediction results than GPR, in the case of only a small training dataset used in the data analysis stage of the SHM strategy.