A Multi-Fidelity Deep Neural Network Approach to Structural Health Monitoring ^†

Abstract

The structural health monitoring (SHM) of civil structures and infrastructures is becoming a crucial issue in our smart and hyper-connected age. Due to structural aging and to unexpected loading conditions, partially linked to extreme events caused by the climate change, reliable and real-time SHM schemes are currently facing a burst in development and applications. In this work, we propose a procedure that relies upon a surrogate modeling scheme based on a multi-fidelity (MF) deep neural network (DNN), which has been conceived to sense and identify a structural damage under operational (and possibly environmental) variability. By exploiting the sensor recordings from a densely deployed network within a fully stochastic framework, the MF-DNN model is adopted to feed a Markov chain Monte Carlo (MCMC) sampling procedure and update the probability distribution of the structural state, conditioned on noisy observations. As information regarding the health of real structures is usually rather limited, the datasets to train the MF-DNN are generated with physical (e.g., finite element) models: high-fidelity (HF) and low-fidelity (LF) models are adopted to simulate the structural response under the mentioned varying conditions, respectively, in the presence or absence of a structural damage. As far as the architecture of the DNN is concerned, the MF approach is obtained by merging a fully connected LF-DNN and a long short-term memory HF-DNN. The LF-DNN mimics the output of the sensor network in the undamaged condition, while the HF-DNN is exploited to improve the LF model and appropriately catch the structural response in the presence of a pre-defined set of damaged patterns. Thanks to the adaptive enrichment of the LF signals carried out by the MF-DNN, the proposed model updating strategy is reported capable of locating (and possibly quantifying) a damage event.

1. Introduction

In civil engineering, numerical models are often used to approximate the structural response of real-life structures, accounting also for uncertainties related to their geometrical and material properties. Within this field, a relevant challenge concerns the optimal management of deteriorating structural systems. The evolution from classical time-based maintenance practices with scheduled inspections toward condition-based ones has been recently put forth, to increase the system safety and availability at the same time [1,2]. This is made possible by permanent data collecting systems and by diagnostic activities, to feed a digital twin of the structural system. SHM has thus become a standard and widespread framework for the identification of damage inception and evolution under varying loading conditions. Vibration-based SHM techniques exploit dynamic response data collected by a sensor network, to assess the damage; this perspective looks therefore well suited for an automated monitoring and to substitute non-destructive tests [3,4].

Approaches to SHM can be classified as either data-driven [5,6,7,8] or model-based [9,10]: the former, which is the one we focus on in this work, exploits the data to dig into the relationship between the structural response, in terms of, e.g., selected features, and the damage pattern to be identified [11,12]; the latter exploits instead physics-based numerical models to assess the damage via model updating [13]. To deal with uncertainties in a proper way and sampling solutions from the relevant probability density functions (pdfs), MCMC methods are frequently adopted [14,15]. Such procedures can incur in high computational costs, mainly linked to repeated evaluations of the nunerical model they rely upon, thus preventing real-time applications. The computational efficiency can be improved by replacing computationally demanding numerical models with data-driven surrogate models, obsersve, e.g., [16,17,18,19]. In the case of multiple models available to simulate the structural response with a different accuracy, namely in case of a MF framework, the simultaneous use of LF and HF model responses can provide a reduction of the overall computational burden, also keeping a selective accuracy of the global output high enough to allow the localization and quantification of damage.

Along the aforementioned research line, in this work we propose a MF-DNN surrogate model, observe, e.g., [20,21]. Such a model is characterized by a multi-level architecture based on two DNNs, which are, respectively, the LF and HF parts of the entire model. A fully-connected DNN is used to mimic sensor recordings in the undamaged structural state, whilst a long short-term memory (LSTM)-based model is used to approximate the effect of damage on the structural response. The two DNNs are trained on data related to the vibration responses, within a simulation-based paradigm. The performance of the proposed method is assessed on a L-shaped cantilever beam, to obtain insights into its accuracy and efficiency.

2. MF-DNN Surrogate Modeling

As already pointed out, the LF and HF datasets ${\mathbf{D}}_{\mathtt{LF}}$ and ${\mathbf{D}}_{\mathtt{HF}}$, respectively, collect the structural responses in the absence and presence of the damage. As customarily assumed, the initial SHM phase refers to the undamaged state. Datasets ${\mathbf{D}}_{\mathtt{LF}}$ and ${\mathbf{D}}_{\mathtt{HF}}$ consist of $I_{\mathtt{LF}}$ and $I_{\mathtt{HF}}<I_{\mathtt{LF}}$ instances, so that:

$${\mathbf{D}}_{\mathtt{LF}}={\{({\mathbf{x}}_i^{\mathtt{LF}},{\mathbf{U}}_i^{\mathtt{LF}})\}}_{i=1}^{I_{\mathtt{LF}}},{\mathbf{D}}_{\mathtt{HF}}={\{({\mathbf{x}}_j^{\mathtt{HF}},{\mathbf{U}}_j^{\mathtt{HF}})\}}_{j=1}^{I_{\mathtt{HF}}},$$

(1)

where: ${\mathbf{x}}_i^{\mathtt{LF}}\in {\mathbb{R}}^{{\mathtt{N}}_{\mathtt{par}}^{\mathtt{LF}}}$ represents the operational conditions inducing the LF vibration recordings ${\mathbf{U}}_i^{\mathtt{LF}}({\mathbf{x}}_i^{\mathtt{LF}})={[{\mathbf{u}}_1^{\mathtt{LF}},\dots ,{\mathbf{u}}_{N_u}^{\mathtt{LF}}]}_i\in {\mathbb{R}}^{N_u\times L}$; the $N_u$ time series are assumed made of L measurements in the time interval $(0,T)$; the damage is accounted for in the HF recordings ${\mathbf{U}}_j^{\mathtt{HF}}({\mathbf{x}}_j^{\mathtt{HF}})={[{\mathbf{u}}_1^{\mathtt{HF}},\dots ,{\mathbf{u}}_{N_u}^{\mathtt{HF}}]}_j\in {\mathbb{R}}^{N_u\times L}$; the ${\mathtt{N}}_{\mathtt{par}}^{\mathtt{HF}}$ input parameters ${\mathbf{x}}_j^{\mathtt{HF}}\in {\mathbb{R}}^{{\mathtt{N}}_{\mathtt{par}}^{\mathtt{HF}}}$ control instead the operational and the damage conditions in the current state.

The HF model describes the dynamic response of the structure under the applied loadings, assuming a linearized kinematic. The semi-discretized equations of the motion thus read:

$${\mathbf{M}}_{\mathtt{HF}}{\ddot{\mathbf{d}}}^{\mathtt{HF}}(t)+{\mathbf{C}}_{\mathtt{HF}}({\mathbf{x}}^{\mathtt{HF}}){\dot{\mathbf{d}}}^{\mathtt{HF}}(t)+{\mathbf{K}}_{\mathtt{HF}}({\mathbf{x}}^{\mathtt{HF}}){\mathbf{d}}^{\mathtt{HF}}(t)={\mathbf{f}}_{\mathtt{HF}}(t,{\mathbf{x}}^{\mathtt{HF}}),$$

(2)

to be supplemented by the relevant initial conditions ${\mathbf{d}}^{\mathtt{HF}}(0)={\mathbf{d}}_0^{\mathtt{HF}}$ and ${\dot{\mathbf{d}}}^{\mathtt{HF}}(0)={\dot{\mathbf{d}}}_0^{\mathtt{HF}}$ in terms of nodal displacements and velocities, respectively. Here: $t\in (0,T)$ denotes time; ${\mathbf{d}}^{\mathtt{HF}}(t),{\dot{\mathbf{d}}}^{\mathtt{HF}}(t),{\ddot{\mathbf{d}}}^{\mathtt{HF}}\in {\mathbb{R}}^{\mathcal{M}}$ are the vectors of nodal displacements, velocities and accelerations; $\mathcal{M}$ is the number of system degrees of freedom (dofs); ${\mathbf{M}}_{\mathtt{HF}}\in {\mathbb{R}}^{\mathcal{M}\times \mathcal{M}}$ is the mass matrix; ${\mathbf{C}}_{\mathtt{HF}}({\mathbf{x}}^{\mathtt{HF}})\in {\mathbb{R}}^{\mathcal{M}\times \mathcal{M}}$ is the damping matrix; ${\mathbf{K}}_{\mathtt{HF}}({\mathbf{x}}^{\mathtt{HF}})\in {\mathbb{R}}^{\mathcal{M}\times \mathcal{M}}$ is the stiffness matrix; ${\mathbf{f}}_{\mathtt{HF}}(t,{\mathbf{x}}^{\mathtt{HF}})\in {\mathbb{R}}^{\mathcal{M}}$ is the vector of external forces.

The structural damage is modeled as a localized degradation of the material stiffness, which is dealt with by means of a parametrization of the stiffness matrix ${\mathbf{K}}_{\mathtt{HF}}$ through the variables $\mathfrak{y}\in {\mathbb{R}}^3$ and $\delta \in \mathbb{R}$, collected in ${\mathbf{x}}^{\mathtt{HF}}$, that denote position and magnitude of the stiffness reduction.

The considered LF model is a projection-based reduced order model (ROM) of the HF one, observe, e.g., [22,23,24], obtained by linearly combining ${\mathcal{M}}_{\mathtt{LF}}\ll \mathcal{M}$ proper orthogonal decomposition (POD) basis functions ${\mathbf{w}}_k\in {\mathbb{R}}^{\mathcal{M}}$, $k=1,\dots ,{\mathcal{M}}_{\mathtt{LF}}$, as ${\mathbf{d}}^{\mathtt{LF}}(t,{\mathbf{x}}^{\mathtt{LF}})\approx \mathbf{W}\mathbf{r}(t,{\mathbf{x}}^{\mathtt{LF}})$, where $\mathbf{W}=[{\mathbf{w}}_1,\dots ,{\mathbf{w}}_{{\mathcal{M}}_{\mathtt{LF}}}]\in {\mathbb{R}}^{\mathcal{M}\times {\mathcal{M}}_{\mathtt{LF}}}$ is the projection matrix and $\mathbf{r}(t,{\mathbf{x}}^{\mathtt{LF}})\in {\mathbb{R}}^{{\mathcal{M}}_{\mathtt{LF}}}$ are the unknown POD coefficients. The following ${\mathcal{M}}_{\mathtt{LF}}$-dimensional ROM is so obtained:

$${\mathbf{M}}_{\mathtt{r}}\ddot{\mathbf{r}}(t)+{\mathbf{K}}_{\mathtt{r}}\mathbf{r}(t)={\mathbf{f}}_{\mathtt{r}}(t,{\mathbf{x}}^{\mathtt{LF}}),$$

(3)

wherein the damping effects are disregarded, and:

$${\mathbf{M}}_{\mathtt{r}}\equiv {\mathbf{W}}^{\top }{\mathbf{M}}_{\mathtt{HF}}\mathbf{W},{\mathbf{K}}_{\mathtt{r}}\equiv {\mathbf{W}}^{\top }{\mathbf{K}}_{\mathtt{LF}}\mathbf{W},{\mathbf{f}}_{\mathtt{r}}(t,{\mathbf{x}}^{\mathtt{LF}})\equiv {\mathbf{W}}^{\top }{\mathbf{f}}_{\mathtt{HF}}(t,{\mathbf{x}}^{\mathtt{LF}}),$$

(4)

with the projection matrix $\mathbf{W}$ obtained with the so-called method of snapshots.

To set the entries of ${\mathbf{D}}_{\mathtt{LF}}$ and ${\mathbf{D}}_{\mathtt{HF}}$, ${\mathbf{x}}^{\mathtt{LF}}$ and ${\mathbf{x}}^{\mathtt{HF}}$ are assumed to be uniformly distributed within their input spaces. The instances are generated by sampling ${\mathbf{x}}^{\mathtt{LF}}$ and ${\mathbf{x}}^{\mathtt{HF}}$ via latin hypercube rule and picking up the measurables from the system state vectors through appropriate Boolean operations.

Moving on to the deep learning stage of the procedure, the MF-DNN surrogate ${\mathcal{NN}}_{\mathtt{MF}}$ is composed as well of the LF and HF parts, respectively termed ${\mathcal{NN}}_{\mathtt{LF}}$ and ${\mathcal{NN}}_{\mathtt{HF}}$. ${\mathcal{NN}}_{\mathtt{LF}}$ is a fully-connected DNN used to mimic the vibration recordings for any given LF input data ${\mathbf{x}}^{\mathtt{LF}}$, according to:

$${\widehat{\mathbf{U}}}^{\mathtt{LF}}({\mathbf{x}}^{\mathtt{LF}})={\mathtt{vec}}_{N_u\times L}^{-1}[\mathbf{Y}(\frac{1}{\omega }\odot {\mathcal{NN}}_{\mathtt{LF}}({\mathbf{x}}^{\mathtt{LF}}))],{\mathcal{NN}}_{\mathtt{LF}}({\mathbf{x}}^{\mathtt{LF}})=\omega \odot \widehat{\mathbf{h}}({\mathbf{x}}^{\mathtt{LF}}),$$

(5)

where: $\mathtt{vec}:{\mathbb{R}}^{m\times n}\to {\mathbb{R}}^{mn}$ denotes the vectorization operation, and ${\mathtt{vec}}_{m\times n}^{-1}:{\mathbb{R}}^{mn}\to {\mathbb{R}}^{m\times n}$ is its inverse; $\mathbf{Y}=[{\mathbf{y}}_1,\dots ,{\mathbf{y}}_{L_{\mathtt{LF}}}]\in {\mathbb{R}}^{L_{\mathtt{concat}}\times L_{\mathtt{LF}}}$, with $L_{\mathtt{concat}}=L{N}_u$, is a matrix of $L_{\mathtt{LF}}\ll L_{\mathtt{concat}}$ POD basis functions used to reduce the number of trainable parameters of ${\mathcal{NN}}_{\mathtt{LF}}$; ${\mathbb{R}}^{L_{\mathtt{LF}}}\ni \mathbf{h}({\mathbf{x}}^{\mathtt{LF}})=[h_1({\mathbf{x}}^{\mathtt{LF}}),\dots ,h_{L_{\mathtt{LF}}}({\mathbf{x}}^{\mathtt{LF}})]={\mathbf{Y}}^{\top }\mathtt{vec}[{\mathbf{U}}^{\mathtt{LF}}]$ are the POD coefficients; ⊙ is the Hadamard product; $\omega \in {\mathbb{R}}^{L_{\mathtt{LF}}}$ is a vector used to weight the relative importance of the POD basis functions.

${\mathcal{NN}}_{\mathtt{LF}}$ is a 9-layers fully-connected DNN with residual connections. All layers feature 30 neurons, except the last two that are equipped with $2L_{\mathtt{LF}}$ and $L_{\mathtt{LF}}$ neurons, respectively. A couple of double-layer identity residual connections [25] are also exploited, to perform an iterative refinement of the outcome. No activation is used for the last layer, while the PReLU [26] one is employed elsewhere. During training, the weights ${\Omega }_{\mathtt{LF}}$ are tuned by minimizing the following loss function:

$${\mathcal{L}}_{\mathtt{LF}}({\Omega }_{\mathtt{LF}},{\mathbf{D}}_{\mathtt{LF}})=\frac{1}{I_{\mathtt{LF}}}\sum _{i=1}^{I_{\mathtt{LF}}}\frac{1}{L_{\mathtt{LF}}}{\parallel \omega \odot ({\mathbf{Y}}^{\top }\mathtt{vec}[{\mathbf{U}}_i^{\mathtt{LF}}])-{\mathcal{NN}}_{\mathtt{LF}}({\mathbf{x}}_i^{\mathtt{LF}})\parallel }_2^2+{\lambda }_{\mathtt{LF}}{\parallel {\Omega }_{\mathtt{LF}}\parallel }_2^2,$$

(6)

where ${\lambda }_{\mathtt{LF}}$ plays the role of a regularization parameter. The loss function is minimized using the Adam algorithm for a maximum of $\mathrm{10,000}$ allowed epochs. The learning rate ${\eta }_{\mathtt{LF}}$ is initially set to $0.005$, and decreased for $4/5$ of the allowed training steps using a cosine decay schedule with weight decay equal to $0.02$. An early-stopping strategy is used to interrupt the learning process, whenever overfitting shows up.

${\mathcal{NN}}_{\mathtt{HF}}$ is instead built upon the LSTM scheme, that is used to mimic the time correlation between the two fidelity levels. Recurrent neural networks (RNNs), such as the adopted LSTM one, can handle the sequences of the inputs of variable length thanks to a hidden state variable that acts as a data memory. As RNNs can fail in addressing long-term dependencies due to the vanishing/exploding gradients issue arising during the back propagation through time [27], the LSTM cell model [28] can better capture long-term dependencies by means of an internal gating mechanism [27]. The ${\mathcal{NN}}_{\mathtt{HF}}$ consists of four LSTM layers, with cell states of size $16,16,32,N_u$, and of a time distributed fully-connected output layer with $N_u$ neurons. No activation is applied to the dense layer, while the LSTM layers have standard cells with sigmoidal gating functions and hyperbolic tangent activated states. The weights ${\Omega }_{\mathtt{HF}}$ are tuned by minimizing the following loss function:

$$\begin{array}{cc}\hfill {\mathcal{L}}_{\mathtt{HF}}({\Omega }_{\mathtt{HF}},{\mathbf{D}}_{\mathtt{HF}})& {\displaystyle =\frac{1}{I_{\mathtt{HF}}}\frac{1}{N_{\mathtt{C}}}\sum _{j=1}^{I_{\mathtt{HF}}}\sum _{\tau =1}^{N_{\mathtt{C}}}\frac{1}{N_u}\frac{1}{L_{\mathtt{C}}}\parallel \mathtt{vec}[{\mathbf{U}}_{j,\tau :\tau +L_{\mathtt{C}}}^{\mathtt{HF}}-}\hfill \\ \hfill & {\mathcal{NN}}_{\mathtt{HF}}({\mathbf{x}}_j^{\mathtt{HF}},{\widehat{\mathbf{U}}}_{\tau :\tau +L_{\mathtt{C}}}^{\mathtt{LF}}({\mathbf{x}}^{\mathtt{LF}}({\mathbf{x}}_j^{\mathtt{HF}})),t_{\tau :\tau +L_{\mathtt{C}}}){]\parallel }_1+{\lambda }_{\mathtt{HF}}{\parallel {\Omega }_{\mathtt{HF}}\parallel }_2^2.\hfill \end{array}$$

(7)

The optimization is carried out by using the Adam algorithm again. The learning rate is decreased as the training advances using a cosine decay schedule, and an early stopping strategy is adopted to prevent overfitting issues.

3. Results

To assess the performance of the proposed MF-DNN surrogate model, the cantilever beam depicted in Figure 1 is considered. The structure is made of two arms having a length of $4\mathrm{m}$, a width of $0.3\mathrm{m}$ and a height of $0.4\mathrm{m}$. The structure is assumed to be made of concrete, with mechanical properties: Young’s modulus $E=30\mathrm{G}\mathrm{P}\mathrm{a}$, Poisson’s ratio $\nu =0.2$, density $\rho =2500\mathrm{k}\mathrm{g}/{\mathrm{m}}^3$. The structure is excited by a distributed vertical load $q(t)$, acting on an area of $(0.3\times 0.3){\mathrm{m}}^2$ close to the beam tip and varying according to $q(t)=Qsin(2\pi ft)$, with $Q\in [1,5]\mathrm{k}\mathrm{P}\mathrm{a}$ and $f\in [10,60]\mathrm{H}\mathrm{z}$ being the load amplitude and frequency. We assume that f and Q are uniformly distributed in their ranges.

Displacement time histories ${\mathbf{U}}^{\mathtt{LF}}({\mathbf{x}}^{\mathtt{LF}})=[{\mathbf{u}}_1^{\mathtt{LF}},\dots ,{\mathbf{u}}_{N_u}^{\mathtt{LF}}]$ and ${\mathbf{U}}^{\mathtt{HF}}({\mathbf{x}}^{\mathtt{HF}})=[{\mathbf{u}}_1^{\mathtt{HF}},\dots ,{\mathbf{u}}_{N_u}^{\mathtt{HF}}]$ are recorded from $N_u=8$ sensors placed along the bottom surface of the structure. The recordings are provided for the time interval $(0,T=1\mathrm{s})$ with a $200\mathrm{H}\mathrm{z}$ sampling rate. The HF numerical model is obtained from a finite element discretization using linear tetrahedral elements and resulting in $\mathcal{M}=4659$ dofs. Damage is simulated via a $25\%$ reduction of the material stiffness within a subdomain ${\Omega }_{\mathfrak{y}}$, whose size is $(0.3\times 0.3\times 0.4){\mathrm{m}}^3$, and whose position is parametrized by the coordinates of its center of mass $\mathfrak{y}=(x_{\Omega },y_{\Omega })$, with $x_{\Omega },y_{\Omega }\in [0.15,3.85]\mathrm{m}$. The ROM is obtained by prescribing a high accuracy, so that its order is set to ${\mathcal{M}}_{\mathtt{LF}}=14$, leading to a speed-up in the analyses of about $25.8$ times. $I_{\mathtt{LF}}=\mathrm{10,000}$ LF instances are collected to train ${\mathcal{NN}}_{\mathtt{LF}}$, while only $I_{\mathtt{HF}}=1000$ additional HF instances are exploited to train ${\mathcal{NN}}_{\mathtt{HF}}$.

To show the capabilities of the surrogate model to match the ground truth solution, some exemplary signals provided by ${\mathcal{NN}}_{\mathtt{MF}}$ are reported in Figure 2, together with the corresponding LF approximations and target signals, at varying input parameters. It can be observed that the ${\mathcal{NN}}_{\mathtt{MF}}$ model always match very well the target signal, in relation to both the low and high frequency components. Though not shown, preventing ${\mathcal{NN}}_{\mathtt{HF}}$ from exploiting the correlation between HF and LF recordings spoils the approximation capabilities of ${\mathcal{NN}}_{\mathtt{MF}}$, as ${\mathcal{NN}}_{\mathtt{LF}}$ seems to play a crucial role in helping ${\mathcal{NN}}_{\mathtt{HF}}$ to maintain correct solutions in time. Intuitively, this loss of accuracy is due to the fact that, if ${\mathcal{NN}}_{\mathtt{LF}}$ is removed, ${\mathcal{NN}}_{\mathtt{HF}}$ needs to learn not only how to enrich the LF signals with the effects of damage and structural damping but to reproduce the entire input–output behavior.

4. Conclusions

In this work, we have proposed a strategy to build MF surrogate models for structural health monitoring in a non-intrusive way, exploiting DNNs. The framework allows to map damage and operational parameters onto recordings linked to the structural response to the external loading. The modeling scheme is based on a two-level architecture, characterized by the relevant DNNs to be trained separately: a fully-connected model is used to mimic sensor recordings in the undamaged state; a long short-term memory model is used to enrich the approximation with the effect of damage. The effects of damage are accounted for without the need of model order reduction techniques, prone to losing damage-sensitive components.

The strategy has been tested against a cantilever beam, and the results have demonstrated that a (structure-independent) remarkable accuracy can be obtained. Though not shown, it has also outperformed the single-fidelity counterpart in mimicking the structural response. The computing time to obtain results from the surrogate model has been demonstrated to be up to three orders of magnitude smaller than that corresponding to the full-order model.

In future activities, the capability of the strategy will be assessed while feeding MCMC algorithms empowered by a Siamese DNN, allowing for measurement noise and varying operational conditions. Some preliminary results have demonstrated that learnable features used in place of the raw vibration recordings would enable improvements in the parameter identification outcomes, especially in terms of damage detection and localization.