Vector Quantized Variational Autoencoder-Based Compressive Sampling Method for Time Series in Structural Health Monitoring

Abstract

The theory of compressive sampling (CS) has revolutionized data compression technology by capitalizing on the inherent sparsity of a signal to enable signal recovery from significantly far fewer samples than what is required by the Nyquist–Shannon sampling theorem. Recent advancement in deep generative models, which can represent high-dimension data in a low-dimension latent space efficiently when trained with big data, has been used to further reduce the sample size for image data compressive sampling. However, compressive sampling for 1D time series data has not significantly benefited from this technological progress. In this study, we investigate the application of different architectures of deep neural networks suitable for time series data compression and propose an efficient method to solve the compressive sampling problem on one-dimensional (1D) structural health monitoring (SHM) data, based on block CS and the vector quantized–variational autoencoder model with a naïve multitask paradigm (VQ-VAE-M). The proposed method utilizes VQ-VAE-M to learn the data characteristics of the signal, replaces the “hard constraint” of sparsity to realize the compressive sampling signal reconstruction and thereby does not need to select the appropriate sparse basis for the signal. A comparative analysis against various CS methods and other deep neural network models was performed in both synthetic data and real-world data from two real bridges in China. The results have demonstrated the superiority of the proposed method, with achieving the smallest reconstruction error of 0.038, 0.034 and 0.021, and the highest reconstruction accuracy of 0.882, 0.892 and 0.936 for compression ratios of 4.0, 2.66, and 2.0, respectively.

1. Introduction

With the increasing number of large-scale public infrastructure, structural health monitoring (SHM) technology has been more and more widely used in civil infrastructure in recent years, such as outlier diagnosis [1,2], damage identification [3,4], response prediction [5,6], and condition assessment [7,8], in order to reduce the risk of irreversible structural damage by continuous tracking of structural condition. However, a typical SHM system requires a large number of sensors to collect enough data for accurate inference of structural conditions. The need of collecting a massive amount of data leads to a tradeoff between the limited storage and data transmission capacity of the SHM system and the monitoring accuracy. Efficient data compression technology is the key to solve this difficult problem. In 2006, the compressive sampling (CS) method was proposed by Candes et al. [9] and Donoho [10], which has revolutionized the field of signal processing because of its ability to efficiently sense and compress signal simultaneously in bandwidth-constrained scenarios like SHM. If a signal can be represented by a sparse vector with a set of known basis functions, the CS method mathematically guarantees a high probability of reconstructing a signal compressed at a rate far less than the Nyquist sampling frequency. Its basic principle is to utilize the sparsity of a signal to constrain the ill-posed linear inverse problem. In addition, research about the robustness to uncertainty has always been a research priority in SHM due to environmental noise and insufficient data problems. For example, Zhao et al. [11] proposed a new paradigm of the recursive modeling strategy of Bayesian multiple linear regression for bridge deflection to deal with the interference from the data time lag and abnormal signal. Zhang et al. [12] presented a new modal parameter identification process based on the Gaussian mixture model of the data envelope and bandpass filtering. Motived by this, Bayesian probabilistic methods, which have been widely used in SHM data modeling [5,6,13] have also been extensively investigated in CS signal reconstruction, in which the confidence of the reconstructed signal can be quantified and the signal reconstruction robustness can be effectively enhanced [14,15].

In real-word applications, it is often difficult to find a sparse representation of many SHM signals. For example, while many time series signals are sparse after wavelet transform, this result does not apply to acceleration signals measured from some real structures [15], limiting the application of the CS method in many SHM applications. In recent years, many researchers have attempted to take advantage of the strong feature-learning ability of deep neural networks (DNNs) [16,17,18,19] to relieve the signal sparsity constraint in conventional CS, e.g., Bora et al. [20] proposed to use well-trained deep generative networks that capture the high-dimension image signals in a low-dimension space as an implicit regularizer for the ill-posed CS problem; Huang et al. [21] and Zhang et al. [22] successfully applied this idea to achieve high segmentation accuracy on building crack images with a high compression ratio; Dave et al. [23] proposed to use the architecture of an untrained deep convolutional generative adversarial networks as a prior to solve any differentiable linear inverse problem for image data, assuming such an architecture has already provided enough constraints to capture the underlying distribution of natural images [24].

Deep-learning-based CS methods have been successfully applied to many practical problems related to image or video signals [25,26,27,28], but 1D time series signals have not benefited from these advancements in CS. Ni et al. [29] directly used neural networks for compression and reconstruction of signals, which differs from the linear signal projection setup in the CS problem. There are studies on data loss recovery in SHM [30,31] using specially designed network structures that are not suitable for the general CS situation. To the best of our knowledge, however, there are seldom reports on significant improvement of CS for time series data based on deep learning. We speculate that the reason is the lack of efficient feature capturing neural network structures for time series. For example, the popular recurrent neural network (RNN) does not necessarily reduce the time series input to a very low-dimensional space. In fact, there is exponentially less volume for compression in 1D space than in higher dimensional spaces. Most of the successful time series models focus on modeling correlation structure at different length scales, such as the traditional autoregressive models.

In this paper, instead of following the standard approach to compress a signal in a single batch, we propose a new modeling approach based on block CS and VQ-VAE model with a multitask learning paradigm (VQ-VAE-M) for the 1D CS problem, which leverages the hierarchical structure of short-term and long-term correlations in time series. This method uses the VQ-VAE-M to learn the data characteristics of the signal, and replaces the “hard constraint” of sparsity to realize the compressive sampling signal reconstruction. Block CS has been studied by many researchers to speed up the reconstruction process [32,33,34,35]. Here, we investigate the potential of improving the reconstruction accuracy of the CS signal under high compression ratios by combining the idea of block CS and projection of short time series on a discrete space. The combination preserves the spatial position features between the elements in the signal effectively in the compression and reconstruction stages.

The remainder of this paper is organized as follows: Section 2 provides a concise explanation of conventional CS and block CS. The technical details of the proposed method are elaborated in Section 3. Then, we demonstrate the effectiveness of different DNNs applied to the 1D CS problem on both synthetic and real SHM data, including the basic multilayer perceptron (MLP) network, waveform transposed convolution neural network (WTCNN) (see Appendix A), waveform generative adversarial network (WaveGAN) (see Appendix B) [36], vector quantized–variational autoencoder (VQ-VAE) (see Appendix C) [37,38], and the proposed method in Section 4. Finally, we remark some concluding results and potential issues of our method in Section 5.

2. Compressive Sampling and Block Compressive Sampling

The conventional CS problem is basically solving the ill-posed inverse linear problem with $l_1$ regularization. Let us consider a linear transformation of N-dimensional discrete signal $\mathsf{{\rm X}}={\left[{\mathbf{x}}_1,{\mathbf{x}}_2,\dots ,{\mathbf{x}}_N\right]}^T$ to $\mathbf{W}$, which has only K (<<N) non-zero coefficients ${\mathbf{w}}_i$, based on a set of standard orthogonal basis vectors $\mathsf{\Psi }=\left[{\mathsf{\psi }}_1,{\mathsf{\psi }}_2,\dots ,{\mathsf{\psi }}_N\right]$:

$$\mathsf{{\rm X}}={\sum }_{i=1}^N{\mathsf{\psi }}_i{\mathbf{w}}_i=\mathsf{\Psi }\mathbf{W}$$

(1)

Next, we compress the signal $\mathsf{{\rm X}}$ to $\mathbf{y}$ by a random projection matrix:${\mathsf{\Phi }}_t\in R^{M\times N}\left(M\ll N\right):$

$$\mathbf{y}={\mathsf{\Phi }}_t\mathsf{{\rm X}}={\mathsf{\Phi }}_t\mathsf{\Psi }\mathbf{W}=\mathsf{\Theta }\mathbf{W}$$

(2)

where $\mathsf{\Theta }={\mathsf{\Phi }}_t\mathsf{\Psi }\in R^{M\times N}$ and $\mathbf{y}={\left[{\mathbf{y}}_1,{\mathbf{y}}_2,\dots ,{\mathbf{y}}_N\right]}^T\in {\mathbb{R}}^{M\times N}$. If ${\mathsf{\Phi }}_t$ satisfies the Restricted Isometry Property (RIP) condition, Candes et al. showed that we can reconstruct $\mathsf{{\rm X}}$ from $\mathbf{y}$ with a high probability of success [9] by solving a non-linear optimization for $\widehat{\mathbf{X}}$:

$$\widehat{\mathbf{X}}=\mathsf{\Psi }\widehat{\mathbf{W}} \mathrm{s}.\mathrm{t}. \widehat{\mathbf{W}}=\mathrm{m}\mathrm{i}\mathrm{n}{‖\mathbf{W}‖}_{l_1}$$

(3)

${\mathsf{\Phi }}_t$ is typically taken as a Gaussian matrix with independent and identically distributed elements [9], but N can be very large for image signals, causing potential issues on the storage of the matrix and computation time of CS.

Gan et al. [39] studied a block CS method where the main idea is to divide X into ${\rm P}$ short sequences ${\mathbf{x}}_{\mathit{i}}$ of length $b$, and compress them into a sequence of short measurements ${\mathbf{y}}_{\mathit{i}}$ of length $a$ by the same projection matrix ${\mathsf{\Phi }}_b$ of size $a\times b$. The reconstruction of X is achieved by concatenating the separately reconstructed ${\widehat{\mathbf{X}}}_{\mathit{i}}$:

$$\widehat{\mathbf{X}}={\mathsf{\Psi }}_b\widehat{\mathbf{W}} \mathrm{s}.\mathrm{t}. \widehat{\mathbf{W}}=\mathrm{m}\mathrm{i}\mathrm{n}{‖\mathbf{W}‖}_1$$

(4)

Mathematically, it is equivalent to solving the original CS problem by designing ${\mathsf{\Phi }}_t$ as

$${\mathsf{\Phi }}_t=\left[\begin{array}{ccc}\begin{array}{cc}{\mathsf{\Phi }}_b& 0\\ 0& {\mathsf{\Phi }}_b\end{array}& \begin{array}{c}\mathbf{L}\\ \mathbf{L}\end{array}& \begin{array}{cc}0& 0\\ 0& 0\end{array}\\ \begin{array}{cc}\mathbf{M}& \mathbf{M}\end{array}& \mathrm{O}& \begin{array}{cc}\mathbf{M}& \mathbf{M}\end{array}\\ \begin{array}{cc}0& 0\\ 0& 0\end{array}& \begin{array}{c}\mathbf{L}\\ \mathbf{L}\end{array}& \begin{array}{cc}{\mathsf{\Phi }}_b& 0\\ 0& {\mathsf{\Phi }}_b\end{array}\end{array}\right]$$

(5)

Recently, deep learning techniques have been applied on the block CS problem to improve computational accuracy and efficiency [40].

The hardware of the projection matrix in Equation (5) can be easily constructed by implementing a random 2D filter bank [39]. In the basic formulation of the block CS method, we assume that the small measurements are independent of each other. As a result, block artifacts usually appear at the boundaries of separations caused by non-smooth signal values as they are reconstructed separately. In our proposed method, we adopt the compression process of block CS to obtain smaller segments of time series signals. However, the reconstruction is not separated for each segment to avoid the block artifacts. Our main goal of using the block CS compression step is not to reduce storage size. As explained in the next section, the small signal segments match well with the neural network structure we picked to reconstruct the signals under a multitask framework.

3. Modified Block CS with VQ-VAE

The fundamental challenge of the CS problem is how to efficiently constrain the solution space of the ill-posed linear inverse problem based on the distribution of data. DNN can be seen as a tool to learn a representation of the underlying data distribution that can constrain the solution space through a set of data samples. This idea is typically implemented in two ways: (1) training a discriminative neural network that takes y as input and x as output; (2) training a generative neural network that transforms a low-dimensional distribution into the high-dimensional data distribution. In the former case, the trained DNN can be directly used as the signal decompressor, assuming that the learned patterns of decompressing signals from the data distribution are embedded in the trained network parameters. In the latter case, the low-dimensional space of the generative network input is optimized to decompress the signals, and the generative network is treated as an implicit constraint, replacing the sparsity constraint in the conventional CS [17].

In the next section, we demonstrate the unsatisfactory performance of these approaches on synthetic and real SHM data. We hypothesize that the state-of-the-art DNN architectures are not suitable for the 1D CS problem because it is difficult to find the compact representation of time series data in a low-dimension space. Therefore, instead of searching for a compact representation of the high dimension time series data directly, we try to take advantage of the hierarchical structure of short-term and long-term correlations in time series. Our proposed method is motivated by two main ideas: (1) using the block CS compression step to compress signal information within a short length scale, and (2) training a neural network that can assemble learned hidden features in the compressed signal segments to reconstruct the complete signal taking into account of the correlations across the short length scale segments. In particular, we implement the latter idea using a VQ-VAE model, which is originally designed as a generative model that projects a high dimension distribution into a discrete space. Different from the convention, our VQ-VAE model encodes the compressed data y into a discrete space and decodes them back into the original signal segment x directly.

3.1. VQ-VAE Model with a Naïve Multitask Implementation

A vector quantized–variational autoencoder (VQ-VAE) [37,38] is basically a variational autoencoder (VAE) model with a discrete hidden space. The embedding space in a VQ-VAE model can be seen as a dictionary or codebook for the signals to be encoded. Once the model is trained, signals can be collected by drawing samples from the hidden space and passing them through the decoder part of the model. Here, we apply this model to decompress y back to x directly, i.e., a compressed signal y is fed into the encoder and decoder sequentially and the output is expected to be the decompressed signal x. This method is similar to the idea of designing over-complete dictionaries to solve the conventional CS problem [41,42,43,44], except that the codebooks in VQ-VAE are learned by training a DNN. However, applying VQ-VAE directly to CS does not lead to satisfactory results, as shown in the next section. Instead, we use block CS to compress a time series and input the concatenated vector $\mathbf{y}=\left[{\mathbf{y}}_1,\dots ,{\mathbf{y}}_i,\dots ,{\mathbf{y}}_a\right]$ of the compressed short data segments to VQ-VAE. This design is based on our speculation that it is easier to learn the decompression pattern if the short- and long-length scale correlations are separated. If we consider the decompression of each short segment ${\mathbf{y}}_i$ as a single CS task, the concatenation of ${\mathbf{y}}_i$ can be seen as using a naïve multitask learning approach to solve the block CS problem, allowing us to avoid the block artifact issue. At the same time, long-term correlation in the time series is less likely to be destructed during the compression step of block CS, i.e., implicitly separating the short-term and long-term correlation. We call our model VQ-VAE-M in this paper.

We adopt a 2-level VQ-VAE structure in this study according to the suggestions of Razavi et al. [38] (see Figure 1). First, the concatenated vector $\mathbf{y}$ is encoded into the top and bottom latent representations ${\mathbf{Z}}_{\mathbf{e}\mathbf{t}}$ and ${\mathbf{Z}}_{\mathbf{e}\mathbf{b}}$ by two encoders. Then, vector quantization is carried out according to the top codebook $E_t$ and bottom codebook $E_b$, respectively, to obtain the top latent discrete representations ${\mathbf{Z}}_{\mathbf{q}\mathbf{t}}$ and the bottom latent discrete representations ${\mathbf{Z}}_{\mathbf{q}\mathbf{b}}$ (the quantization process of ${\mathbf{Z}}_{\mathbf{e}\mathbf{b}}$ is guided by ${\mathbf{Z}}_{\mathbf{q}\mathbf{t}}$ as well). Finally, ${\mathbf{Z}}_{\mathbf{q}\mathbf{t}}$ and ${\mathbf{Z}}_{\mathbf{q}\mathbf{b}}$ are passed to the decoder to reconstruct the original signal x. The vector quantization step simply replaces each value in ${\mathbf{Z}}_{\mathbf{e}\mathbf{t}}$ and ${\mathbf{Z}}_{\mathbf{e}\mathbf{b}}$ by the nearest neighbor value in the corresponding codebook $E_t$ and $E_b$ to form ${\mathbf{Z}}_{\mathbf{q}\mathbf{t}}$ and ${\mathbf{Z}}_{\mathbf{q}\mathbf{b}}$. We employ convolutional layers in the encoder and decoder. Figure 2 shows the detailed structure and tensor sizes of the network, where $B$ is the batch size of input, $L_c$ is the length of a single compressed signal in the input batch, $L_{eb}$ is the coding length of the bottom latent space, $L_{et}$ is the coding length of the top latent space, $C_e$ is the number of latent coding channels, $C_{RB}$ is the number of residual block channels, $L_{qt}$ is the top latent discrete coding length, $L_{qb}$ is the bottom latent discrete coding length, $C_q$ is the number of latent discrete coding channels, and $L$ is the original signal length. We note that our encoder and decoder are not symmetrical because the input and output are not in the same size. In order to extract deeper signal features and accelerate convergence, residual blocks and skip connections are used in both the encoder and the decoder.

3.2. DNN-Based CS Algorithm

In the proposed algorithm, we use the block CS compression step to linearly project a 1D signal x to y based on the matrix in Equation (5), and use VQ-VAE-M to reconstruct the original signal by feeding y into the input of VQ-VAE-M and taking the output as the reconstructed signal x directly. To train the VQ-VAE-M model, pairs of the original signal x and compressed signal y are collected to form a set of training data. In the case of SHM, time series data x from the sensors on a structure are collected and compressed into y. Following the typical neural network training scheme, the training dataset is separated into multiple batches, each of size B. During the backpropagation training phase, all batches of data are sequentially applied to a VQ-VAE-M model with randomly initialized parameters in a random order. The procedure of training with all batches is repeated $N_{ep}$ times (denoted as $N_{ep}$ epochs), until the parameter values converge. We adopt the same loss function as the original VQ-VAE model [38] in the backpropagation step:

$$Loss\left(\mathbf{x},D\left(\mathbf{e}\right)\right)={‖\mathbf{x}-D\left(\mathbf{e}\right)‖}_2^2+{‖sg\left[E\left(\mathbf{x}\right)\right]-\mathbf{e}‖}_2^2+\beta {‖sg\left[\mathbf{e}\right]-E\left(\mathbf{x}\right)‖}_2^2$$

(6)

where $\mathbf{e}$ denotes the quantized code of y, $E$ denotes the encoder function, $D$ denotes the decoder function, and $sg$ refers to a stop-gradient operation.

In addition to the usual DNN hyperparameters, such as learning rate, batch size, and number of epochs, number of blocks in block CS (or the length of each data segment) is also an important hyperparameter in our algorithm, which controls the maximum length scale of information being compressed into y. The shorter the length of each data segment, the less expected amount of pattern we can observe from the data. As a result, it is easier for VQ-VAE-M to find a good representation in the discrete hidden space for the signal reconstruction task. However, the data compression space is also less, i.e., the minimum compression ratio is higher. As shown in the next section, our model favors shorter data segments for better reconstruction accuracy. Therefore, we choose the shortest possible segment length based on the desired compression ratio $\mathrm{C}\mathrm{R}=N/M$ in most of our tests.

4. Results

In order to verify the reconstruction performance of the VQ-VAE-M method on CS signals, we test it on a set of synthetic times series constructed by superimposing sinusoidal signals and real acceleration time series obtained from SHM sensors on two real bridges in China (Tianjin Yonghe Bridge and Hangzhou Bay Bridge). Using the real bridge data, we compare our method with five conventional CS algorithms and four deep-learning-based CS algorithms to demonstrate the important contributions of block CS and DNN in the 1D CS problem. The five conventional CS algorithms based on sparse constraints are Basis Pursuit (BP) [9], Compressive sampling matching pursuit (CoSaMP) [45], Gradient Projection for Sparse Reconstruction (GPSR) [46], Bayesian compressive sensing (BCS) [47] and Bayesian compressive sensing integration over the prediction–error precision parameter (BCS-IPE) [14]. Among them, CoSaMP is a Greedy iterative algorithm, and BP and GPSR are two Convex optimization algorithms, all of which are well known and classic deterministic CS methods. BCS and BCS-IPE are two Bayesian probabilistic methods, which utilize sparse Bayesian learning to quantify the posterior uncertainty of the signal model and so can improve the signal reconstruction robustness. The four deep-learning-based CS algorithms are MLP, WTCNN, WaveGAN [36] and VQ-VAE [37,38]. The first two are discriminative-model-based methods, in which the MLP method is known to be simple and fast in calculation, and the WTCNN model has the advantages of fast training speed. WaveGAN and VQ-VAE are two most popular generative-model-based methods, as explained in the previous section. All four models are from our original design because of the lack of publicly available pretrained models for time series data using a deep-learning-based CS algorithm. The details of the structures and training procedures and advantages of these DNNs are shown in Appendix A, Appendix B and Appendix C. During the training procedure, early stopping [48] is employed to avoid overfitting and to select the network hyperparameters. In addition, data normalization and the two-tier network structure, which can improve the depth of the network structure and help to capture more characteristic information, are also employed to avoid underfitting.

4.1. Performance Metric

To evaluate the performance of signal reconstruction evaluation, the reconstruction error of a given signal is defined as follows:

$$E_r=\frac{{‖\widehat{\mathbf{X}}-\mathbf{X}‖}_2^2}{{‖\mathbf{X}‖}_2^2}$$

(7)

where $\mathbf{X}$ is the original signal and $\widehat{\mathbf{X}}$ is the corresponding reconstructed signal. If the reconstruction error $E_r$ is less than or equal to a certain threshold $\delta$, the reconstruction of the single signal is considered successful; otherwise, the reconstruction is considered failed. The ratio of the number of successfully reconstructed signals $L_s\left(\delta \right)$ to the total number of signals $L$ is used to quantify the reconstruction quality, which is called the reconstruction success rate:

$$S_r=\frac{L_s\left(\delta \right)}{L}$$

(8)

Various indicators are employed in the following tests and their results will be tabulated in

Table 1. Performance of different CS methods under compression ratio of $N/M=8.0$.

Methods	Emean	E1	E2	t (s)	ttrain (min)
VQ-VAE-M	0.003	1.0	1.0	0.038	2.967
WaveGAN	0.155	0.0938	0.8516	4.743	56.15
VQ-VAE	0.02	1.0	1.0	0.329	2.333

,

Table 2. Influence of the length of each signal segment $D_B$ on reconstruction performance of VQ-VAE-M under compression ratio of $N/M=4.0$.

DB	Emean	E1	E2	t (s)
8	0.039	0.882	0.985	0.014
16	0.100	0.641	0.849	0.014
32	0.110	0.662	0.818	0.013
64	0.116	0.613	0.803	0.014
128	0.306	0.156	0.344	0.013
256	0.580	0.072	0.162	0.013
512	0.768	0.056	0.133	0.013

,

Table 3. Influence of the number of blocks $N_B$ on the reconstruction performance of VQ-VAE-M under compression ratio of $N/M=4.0$.

${\mathit{N}}_{\mathit{B}}$	Emean		E1
	Multi-Blocks	Block	Multi-Blocks	Block
1	0.136	0.136	0.699	0.699
2	0.129	0.096	0.653	0.737
4	0.066	0.076	0.791	0.766
8	0.066	0.068	0.790	0.786
16	0.058	0.064	0.821	0.800
32	0.053	0.062	0.831	0.800
64	0.040	0.060	0.882	0.815

,

Table 4. Ablation experiments of three models under compression ratio of $N/M=4.0$.

Methods	Emean	E1	E2	t (s)	ttrain (min)
VQ-VAE-M	0.038	0.882	0.987	0.013	18.501
(VQ-VAE-M)E	0.043	0.867	0.964	0.011	20.362
(VQ-VAE-M)D	0.049	0.838	0.956	0.011	21.034

,

Table 5. Ablation experiments of three models under compression ratio of $N/M=2.66$.

Methods	Emean	E1	E2	t (s)	ttrain (min)
VQ-VAE-M	0.034	0.892	0.977	0.013	14.312
(VQ-VAE-M)E	0.038	0.877	0.946	0.011	16.031
(VQ-VAE-M)D	0.043	0.862	0.941	0.011	16.647

,

Table 6. Ablation experiments of three models under compression ratio of $N/M=2.0$.

Methods	Emean	E1	E2	t (s)	ttrain (min)
VQ-VAE-M	0.021	0.936	1.000	0.013	12.148
(VQ-VAE-M)E	0.025	0.911	0.972	0.011	14.025
(VQ-VAE-M)D	0.029	0.895	0.964	0.011	14.507

,

Table 7. Performance indexes of each model under compression ratio of $N/M=4.0$.

Methods	Emean	E1	E2	t (s)
BP	0.475	0	0	0.172
CoSaMP	3.501	0	0	0.191
GPSR	0.437	0	0.020	0.042
BCS	0.499	0	0.050	0.067
BCS-IPE	0.466	0.010	0.060	1.251
MLP	0.051	0.854	0.944	0.016
WTCNN	0.050	0.862	0.964	0.012
VQ-VAE-M	0.038	0.882	0.987	0.013
WaveGAN	0.553	0.110	0.190	4.732
VQ-VAE	0.453	0.569	0.685	2.574

,

Table 8. Performance indexes of each model under compression ratio of $N/M=2.66$.

Methods	Emean	E1	E2	t (s)
BP	0.364	0	0	0.178
CoSaMP	0.417	0.020	0.170	0.250
GPSR	0.310	0	0.250	0.022
BCS	0.338	0.040	0.290	0.095
BCS-IPE	0.325	0.060	0.330	1.504
MLP	0.048	0.859	0.913	0.017
WTCNN	0.043	0.874	0.956	0.011
VQ-VAE-M	0.034	0.892	0.977	0.013
WaveGAN	0.434	0.170	0.210	6.722
VQ-VAE	0.460	0.149	0.526	2.586

and

Table 9. Performance indexes of each model under compression ratio of $N/M=2.0$.

Methods	Emean	E1	E2	t (s)
BP	0.304	0	0.060	0.190
CoSaMP	0.293	0.070	0.360	0.301
GPSR	0.250	0.020	0.390	0.013
BCS	0.268	0.150	0.390	0.097
BCS-IPE	0.266	0.150	0.410	2.000
MLP	0.032	0.895	0.979	0.016
WTCNN	0.023	0.918	0.997	0.011
VQ-VAE-M	0.021	0.936	1.000	0.013
WaveGAN	0.365	0.180	0.210	7.005
VQ-VAE	0.194	0.595	0.733	2.623

. E_mean is the average reconstruction error of all test signals; E₁ and E₂ refer to the reconstruction success rate S_r for $\delta =0.1$ and $\delta =0.2$, respectively; in addition, the average reconstruction time t of each signal and the training time t_train for deep-learning-based methods are introduced; for cutting time series in the compression stage, the two cases of cutting sequence data into different blocks and no cutting are denoted as Multi-Blocks and Block, respectively. Among these indicators, we employ E_mean to quantify the overall signal reconstruction accuracy at a certain compression ratio; E₁ and E₂ are used to measure how many signals can be reconstructed successfully. We also quantify the efficiency of the method by the indicators of average reconstruction time t and training time t_train.

For conventional CS methods, the selection of sparse basis is often a necessary step. In this study, we try both cosine and wavelet bases for each method, and select the better result for comparison in order to demonstrate an advantage of the conventional methods.

4.2. Test on Synthetic Data

We synthesized a total of 27,300 superimposed sinusoidal signals x of length 512 by

$$\mathbf{x}=\sum _{i=1}^{10}{A}_i\sin (w_{i}t+{\phi }_i)$$

(9)

where $A_i$, $w_i$ and ${\phi }_i$ were sampled from $U(0.1, 0.5)$, $U(1, 20)$, $U\left(-\pi , \pi \right), \mathrm{r}\mathrm{e}\mathrm{s}\mathrm{p}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{l}\mathrm{y}$ and $U(a,b)$ denotes a uniform distribution between the interval of a and b. These signals were compressed to y by block CS with a compression ratio CR of $N/M=8$.0. We randomly selected 23,400 pairs of the x and y data to train the DNN models and tested them on the remaining signals along with various CS algorithms. Figure 3 shows a comparison of a synthetic signal and its reconstruction based on our proposed method. It can be seen that the two signals are consistent with each other very well, demonstrating the effectiveness of the method.

To investigate the robustness of the method, we repeated the experiment on each test signal 100 times and recorded the mean values of the performance metrics for three deep generative model-based methods VQ-VAE-M, WaveGAN and VQ-VAE. Table 1 lists the performance results of different CS methods. In terms of reconstruction quality, as expected, the reconstruction results of VQ-VAE-model-based algorithms are good because of their ability to learn from the data. The average reconstruction errors (E_mean) of the two algorithms are significantly smaller compared with WaveGAN, where the error of VQ-VAE-M is only 0.003; in addition, the reconstruction success rates of both VQ-VAE-M and VQ-VAE methods are close to 1.

In terms of reconstruction speed, the VQ-VAE-M and VQ-VAE algorithms are fast on average. For the best performing model, VQ-VAE-M, it takes 0.038 s on average to reconstruct each signal and less than 3 min to train the model.

4.3. Test on Real SHM Data

In practical SHM applications, sensor signals often exhibit complex data characteristics, such as non-periodicity or contain various interference components due to environmental excitation, human disturbance, etc. Using two sets of real acceleration signals from Tianjin Yonghe Bridge and Hangzhou Bay Bridge, we inspected the performance of the proposed method and its sensitivity to the hyperparameters in block CS, noise contamination, and residual blocks and skip connections. Figure 4 shows the acceleration signals from the two bridges. We divided each time series into subsequences of length 512, yielding a total of 23,400 pieces of signals, and chose 23,010 pieces of signals as the training data set. Here, we judge whether the training data is sufficient based on the convergence of the loss function. In order to ensure that the model performs stably and reliably, the remaining signals were used as the validation data set. The validation data set should be selected randomly but cannot contain data from the training dataset [49]. Figure 5 shows a sample of signal from Tianjin Yonghe Bridge in different bases. We note that the signal is not very sparse in either cosine or wavelet basis, illustrating the difficulty of selecting a suitable basis for a conventional CS algorithm.

4.3.1. Sensitivity to Hyperparameters in Block CS

To understand the impact of the number of blocks $N_B$ and the length of each signal segment $D_B$ in block CS on the reconstruction performance, we compared the performance of VQ-VAE-M models with different $D_B$ and $N_B$. The results in Table 2 and Table 3 show that a lower $D_B$ or higher $N_B$ is always preferred. This may indicate that finding a good representation of a long time series for solving the CS problem is difficult. Convolutional layers in DNNs (including VQ-VAE) are probably more effective for vectors with spatial correlation. Block CS with a high $N_B$ or low $D_B$ implies that most of the correlations in the time series are likely to be conserved. However, $D_B$ is the highest possible compression ratio in this case. Therefore, we suggest choosing the lowest possible $D_B$ corresponding to the desired compression ratio of a CS application in the applications.

4.3.2. Sensitivity to Noise Contamination

Noise contamination is a common issue in SHM signals. Sensitivity to noise is also related to the robustness of VQ-VAE-M and the possible overfitting problem. Here, we tested our method on the acceleration signal of Hangzhou Bay Bridge with a small signal-to-noise ratio. With a compression ratio of $N/M=4.0$, Figure 6 demonstrates the distribution of reconstruction errors as a function of $\delta$. In general, the performance is not very satisfactory. Although the model cannot reconstruct complex signals completely, the low frequency components of complex signals can be reconstructed accurately. As shown in Figure 7, the reconstruction quality of signals in the time domain is poor and some peak values are not well reconstructed; however, in the frequency domain, the reconstruction quality of the low-frequency part is relatively good, while the reconstruction quality of the high-frequency part is relatively poor. This is because the neural network will pay more attention to the low-frequency components during the training phase, as has been previous suggested that the network fits low frequency before high frequency [50]. In fact, the natural frequency of bridge structure is generally low, as the high-frequency components are usually environmental noise. Therefore, even the proposed method cannot completely reconstruct the signal under particularly complex environmental excitation, it also has certain application potential in the field of SHM.

4.3.3. Sensitivity to Residual Blocks and Skip Connections

In order to extract signal features and accelerate convergence effectively, residual blocks and skip connections are used in both the encoder and decoder, as stated previously. We designed and performed ablation experiments on the acceleration signals from Tianjin Yonghe Bridge to demonstrate the effectiveness of these two modules. The compressive sampling signals were reconstructed at the compression ratios of 4.0, 2.66, 2.0, and the experiments were repeated 390 times for each compression ratio. The performance metric results of three methods were recorded in Table 4, Table 5 and Table 6, in which (VQ-VAE-M)_E and (VQ-VAE-M)_D show that residual blocks and skip connections are only used in the encoder or decoder, respectively.

From the results, it is concluded that the residual blocks and skip connections in both the encoder and decoder are conducive to reducing the reconstruction error and improving the reconstruction accuracy. Meanwhile, the residual blocks and skip connections in both the encoder and decoder can accelerate the model training convergence.

4.3.4. Comparison of Different CS Algorithms

As stated previously, we compared our proposed method with five conventional CS algorithms and four deep-learning-based CS algorithms on the acceleration signals of Tianjin Yonghe Bridge, which have a higher signal-to-noise ratio than those of Hangzhou Bay Bridge. At the compression ratios of 4.0, 2.66, 2.0, the compressive sampling signals were reconstructed by running the algorithms, and the experiments were repeated 390 times for each compression ratio. Table 7, Table 8 and Table 9 recorded the performance metrics of all 10 algorithms.

In terms of reconstruction quality, the deep-learning-based algorithms generally have better reconstruction results at the three compression ratios, showing that the compression patterns of the test data are successfully learnable from the training data. The E_mean of MLP, WTCNN and VQ-VAE-M are all very small, where VQ-VAE-M has the smallest errors of 0.038, 0.034 and 0.021 at all three compression ratios, while the E_mean of the traditional compressive sampling algorithms are larger than 0.2, which are relatively high. In terms of E₁ and E₂, the indices values of deep-learning-based algorithms are also much higher than traditional methods. Among them, VQ-VAE-M algorithm has the highest E₁ accuracy of more than 0.88 when the compression ratio $N/M$ is 4.0, while the highest E₁ accuracy of traditional algorithm is 0.10. The situation is similar in E₂, where the VQ-VAE-M algorithm obtains the highest E₂ accuracy of more than 0.98. Similar conclusion can be observed in Table 7, Table 8 and Table 9 when the compression ratios $N/M$ are 2.66 and 2.0, respectively.

In terms of signal reconstruction speed, the three deep-learning-based algorithms MLP, WTCNN and VQ-VAE-M are much faster on average. For the best performing model, VQ-VAE-M, it takes only 0.013 s on average to reconstruct each signal, which is much faster than WaveGAN and VQ-VAE, and most traditional methods.

In the application, if E₁ or E₂ equals to 1.0, we consider the signal reconstruction is successful. The results in Table 7, Table 8 and Table 9 show that the maximum compression ratio of our method is around 2.0 for the acceleration signals from Tianjin Yonghe Bridge. It is also observed that the traditional compressive sampling methods and other deep-learning-based compressive sampling methods fail to reconstruct all signals successfully at this compression ratio. Therefore, our method achieves the highest compression ratio compared to other methods.

It is concluded that compared with the traditional compressive sampling methods based on sparse constraint, the compressive sampling signal reconstruction method based on VQ-VAE-M significantly improves the reconstruction quality and the reconstruction speed, while being able to achieve a high-quality reconstruction of one-dimensional SHM compressive sampling signal at a high compression ratio (Figure 8).

5. Conclusions

The traditional compressive sampling methods require the signal to satisfy the “hard constraint” of sparsity in a certain transform domain. However, the civil structural response is not only related to the sparsity of modes, but also affected by the structural excitation. The sparsity is sometimes difficult to be satisfied in the case of complex environmental interference. In this paper, based on block CS and the vector quantized–variational autoencoder model with a naïve multitask paradigm (VQ-VAE-M), a data driven-based compressive sampling signal reconstruction method for one-dimensional time series signals in SHM is studied. The main contents can be summarized as follows:

(1)

Based on the vector quantized–variational autoencoder model with a naïve multitask paradigm (VQ-VAE-M), a one-dimensional compressive sampling signal reconstruction method is established. This method uses VQ-VAE-M to learn the data characteristics of the signal, replaces the “hard constraint” of sparsity to realize the compressive sampling signal reconstruction and so eliminate the need to select the appropriate sparse basis for the signal. VQ-VAE-M embeds and maintains one or more codebooks in the latent space to reconstruct signal. The unique bottleneck structure improves the depth of the network, and the two-tier network structure enriches the details of the reconstructed signal. In addition, the model combined with the block random Gaussian projection matrix can preserve the spatial position features between the elements in the signal as much as possible in both the compression and reconstruction stages, which greatly reduces the difficulty of decoupling in decompression and reconstruction, and ensures the quality and speed of signal reconstruction.

(2)

The superimposed sinusoidal synthetic signal (sparse enough in the frequency domain) and Yonghe Bridge acceleration signal (not sparse enough on both the cosine basis and wavelet basis) are used to verify the performance of the proposed method, which is compared with the other compressive sampling methods. The results show that the proposed method obviously has better performance, that is, higher reconstruction quality, faster reconstruction speed, enabling to reconstruct signal at a higher compression ratio, and no need to select the appropriate sparse basis for the signal.

(3)

The characteristics and advantages of the block random Gaussian projection matrix are analyzed. The matrix can preserve the spatial features between the elements in the signal, which is not only conducive to decoupling, but also to extracting the overall characteristics of adjacent signals. In addition, the reconstruction performance of this method in complex environment is also investigated. Even if the environment excitation is more complex, the method can still reconstruct the information of the low-frequency components of the signal effectively.

This method also has some limitations: a large number of data are required to train the neural network model aiming at different compressive sampling signal types, and the reconstruction performance of complex and high-dimensional signals can be further improved. In future study, it will be useful to combine the proposed method with a Transfomer model and Bayesian probabilistic method to address these issues.