Probabilistic Seismic Response Prediction of Three-Dimensional Structures Based on Bayesian Convolutional Neural Network

Seismic response prediction is a challenging problem and is significant in every stage during a structure’s life cycle. Deep neural network has proven to be an efficient tool in the response prediction of structures. However, a conventional neural network with deterministic parameters is unable to predict the random dynamic response of structures. In this paper, a deep Bayesian convolutional neural network is proposed to predict seismic response. The Bayes-backpropagation algorithm is applied to train the proposed Bayesian deep learning model. A numerical example of a three-dimensional building structure is utilized to validate the performance of the proposed model. The result shows that both acceleration and displacement responses can be predicted with a high level of accuracy by using the proposed method. The main statistical indices of prediction results agree closely with the results from finite element analysis. Furthermore, the influence of random parameters and the robustness of the proposed model are discussed.


Introduction
An earthquake is a natural disaster which causes catastrophic damage to structures [1,2]. To accurately compute the seismic dynamic response and evaluate the safety of structures, the randomness and uncertainty of structure parameters need to be taken into consideration [3][4][5][6][7][8]. To solve this problem, various methods have been proposed by researchers since the 1960s, such as the Monte-Carlo simulation [9,10], the orthogonal polynomial expansion method [11] and the probability density evolution method [12][13][14][15]. The analytic results using these methods are given in the form of probability distributions. However, for these methods, the random parameter probability distributions need to be obtained before the calculation. If the probability distributions are unknown or change, these methods are difficult to carry out.
Recently, the development of neural network and deep learning [16] has provided a powerful approach for seismic response prediction. Different types of neural networks or deep learning models are currently used to predict the dynamic response of structures. The earliest work utilizing this approach aimed at response analysis of the linear response of O i,j,k = (X * V) i,j,k = ∑ l,m,n X (i−1)s+m,(j−1)s+n,l V m,n,l,k , where in Equation (1), x is the 1-D input, v is the convolutional kernel and o is the output. Similarly, in Equation (2), X is the input of the 2-D convolutional operation, and V and O are the corresponding kernel and output. The strides of convolutional operation, s, refers to how many data points are skipped between the two operations. The process of 1-D and 2-D convolution is expressed in Figure 1.
where in Equation (1), x is the 1-D input, v is the convolutional kernel and o is the output. Similarly, in Equation (2), X is the input of the 2-D convolutional operation, and V and O are the corresponding kernel and output. The strides of convolutional operation, s, refers to how many data points are skipped between the two operations. The process of 1-D and 2-D convolution is expressed in Figure 1. Pooling operation is another important operation used in CNN. The main function of pooling is to compress the data. There are mainly two types of pooling; mean pooling and maximum pooling. As its name implies, mean pooling outputs the mean value in the pooling region while maximum pooling outputs the maximum value.
A 1-D CNN model is used to predict the seismic response of the building. The architecture of the proposed CNN model is described in Figure 2. The input is the earthquake acceleration, while the output is the displacement or the acceleration response. There are five hidden convolutional layers and one fully connected layer between input and output layers. For the convolutional layer, rectified linear unit (ReLU) is used as the activation function.

Training of Bayesian Deep Learning Model
When the deep learning model is built, the model parameters, such as weight matrix and bias vector, can be estimated by using the samples of input and output. This process is the so-called 'model training'. From the perspective of probability theory, when the point estimation is utilized, the trained deep learning model has deterministic values. However, if Bayesian estimation is applied, the probability distribution can be obtained. Pooling operation is another important operation used in CNN. The main function of pooling is to compress the data. There are mainly two types of pooling; mean pooling and maximum pooling. As its name implies, mean pooling outputs the mean value in the pooling region while maximum pooling outputs the maximum value.
A 1-D CNN model is used to predict the seismic response of the building. The architecture of the proposed CNN model is described in Figure 2. The input is the earthquake acceleration, while the output is the displacement or the acceleration response. There are five hidden convolutional layers and one fully connected layer between input and output layers. For the convolutional layer, rectified linear unit (ReLU) is used as the activation function.
where in Equation (1), x is the 1-D input, v is the convolutional kernel and o is the output. Similarly, in Equation (2), X is the input of the 2-D convolutional operation, and V and O are the corresponding kernel and output. The strides of convolutional operation, s, refers to how many data points are skipped between the two operations. The process of 1-D and 2-D convolution is expressed in Figure 1. Pooling operation is another important operation used in CNN. The main function of pooling is to compress the data. There are mainly two types of pooling; mean pooling and maximum pooling. As its name implies, mean pooling outputs the mean value in the pooling region while maximum pooling outputs the maximum value.
A 1-D CNN model is used to predict the seismic response of the building. The architecture of the proposed CNN model is described in Figure 2. The input is the earthquake acceleration, while the output is the displacement or the acceleration response. There are five hidden convolutional layers and one fully connected layer between input and output layers. For the convolutional layer, rectified linear unit (ReLU) is used as the activation function.

Training of Bayesian Deep Learning Model
When the deep learning model is built, the model parameters, such as weight matrix and bias vector, can be estimated by using the samples of input and output. This process is the so-called 'model training'. From the perspective of probability theory, when the point estimation is utilized, the trained deep learning model has deterministic values. However, if Bayesian estimation is applied, the probability distribution can be obtained.

Training of Bayesian Deep Learning Model
When the deep learning model is built, the model parameters, such as weight matrix and bias vector, can be estimated by using the samples of input and output. This process is the so-called 'model training'. From the perspective of probability theory, when the point estimation is utilized, the trained deep learning model has deterministic values. However, if Bayesian estimation is applied, the probability distribution can be obtained. In Bayesian deep learning, the variational inference is used to realize the Bayesian estimation.
(1) Variational inference Let us consider a deep learning model with model parameter W. The set of training data includes M samples, such as: S = {(x 1 , y 1 ), (x 2 , y 2 ), · · · , (x M , y M )}. According to the Bayes formula [38], the model parameters are estimated as: where p(W) is the prior distribution, which is determined by assumption, prior knowledge or experience. The distribution of train samples is p(S), p(S|W) is the likelihood function and p(W |S) is the estimated distribution of model parameter W. Since p(S) is difficult to obtain, the Bayes formula cannot be used to obtain the parameter estimation directly.
To solve this problem, a new distribution q(W) is introduced to approximate p(θ|S). The difference between q(W) and p(W|S) can be measured through the concept of Kullback-Leibler (KL) divergence [39]. KL divergence can be expressed as: The aim of variational inference is to obtain the optimal q(θ) by minimizing KL divergence, or, in essence, maximizing the second term in the left as: In order to convert the variational problem to an optimization problem, the q(W) can be assumed to be a joint Gaussian distribution and every parameter W i in the parameter matrix W obeys an independent Gaussian distribution as: where µ and σ refer to the mean value matrix and the standard deviation. The optimal distribution q(W) can be obtained by finding the optimal value of parameter matrix µ and σ as: (2) Bayes back propagation algorithm The Bayes back propagation algorithm [40,41] is used to train the proposed Bayesian deep learning model. This algorithm combines variational inference with the back propagation algorithm used in conventional deep learning model training. The object function in Equation (7) is used as an additional loss function L KL to update the model parameters. The process of the Bayes back propagation algorithm can be expressed as follows: Step 1: Initialize the mean and standard deviation value of random parameters as µ and σ.
Step 2: Select a batch of data from the training data.
Step 4: Calculate the total loss function as L total = L mse + L KL , where L mse is the mean square error function while L KL is the KL divergence loss function as the object function in Equation (7).
Step 5: Calculate the g mse W , g KL µ and g KL σ which are the gradient to L mse and L KL by back propagation algorithm.
Step 6: Update the random parameter by gradient descent as: Step 7: Repeat steps 2 to 6 until the deep learning model is well trained.
Where α is the learning rate; some adaptive gradient descent, such as Adam or RMSprop [42], can also be applied to train the proposed Bayesian deep learning model to make the model convergence in less training epochs.

The Framework of Proposed Method
The framework of the proposed method is presented in Figure 3. A finite element model of a 3D building with random characteristics is built. Through Monte-Carlo simulation, samples of input ground motion acceleration and output structural response are generated. The training data are utilized to train the deep convolutional neural network through Bayes back propagation algorithm. The trained Bayesian deep learning model is used to predict the random response of the 3D building. The results are given in the form of probability distribution functions (PDFs) and the main statistical indices are expressed as mean value and standard deviation.

Three Dimensional Building with Random Characteristics
As shown in Figure 4, a three dimensional (3D) building structure with ran rameters is utilized to validate the performance of the Bayesian deep learning mo this building, the story height for each floor is 3.0 m. The section dimension of and beams are 500 mm × 500 mm and 500 mm × 300 mm, respectively. The slab t is equal to 20 cm. In addition to the self-weight of beams and slabs, a distribut load of 2.5 kN/m 2 and a live load with nominal value of 2.0 kN/m 2 are considere analysis. The material parameters of concrete and steel are considered as random eters, as shown in Table 1.

Three Dimensional Building with Random Characteristics
As shown in Figure 4, a three dimensional (3D) building structure with random parameters is utilized to validate the performance of the Bayesian deep learning model. For this building, the story height for each floor is 3.0 m. The section dimension of columns and beams are 500 mm × 500 mm and 500 mm × 300 mm, respectively. The slab thickness is equal to 20 cm. In addition to the self-weight of beams and slabs, a distributed dead load of 2.5 kN/m 2 and a live load with nominal value of 2.0 kN/m 2 are considered in the analysis. The material parameters of concrete and steel are considered as random parameters, as shown in Table 1.
rameters is utilized to validate the performance of the Bayesian deep learning model. For this building, the story height for each floor is 3.0 m. The section dimension of columns and beams are 500 mm × 500 mm and 500 mm × 300 mm, respectively. The slab thickness is equal to 20 cm. In addition to the self-weight of beams and slabs, a distributed dead load of 2.5 kN/m 2 and a live load with nominal value of 2.0 kN/m 2 are considered in the analysis. The material parameters of concrete and steel are considered as random parameters, as shown in Table 1.

The Samples of Ground Motion Acceleration
In this paper, generalized the Kanai-Tajimi (GKT) [43][44][45] model is used to generate the samples of random ground motion acceleration. The GKT model can be expressed by Equations (8) and (9) as: .. ..
where X f (t) is the filtered response, ω g is the characteristic ground frequency, ς g is the effective ground damping coefficient, is the output ground acceleration, e(t) is the amplitude envelope function, and wn(t) is a stationary white noise process. In this paper, the amplitude envelope function can be written as: where t n is the duration time of the earthquake, ε t is normalized time when ground motion peak appears, and η is the value of ground motion at 90% of the duration time.
The 3D building is considered to be located in a 2nd class site and the design earthquake group belongs to group 2. According to the Chinese code for seismic design of buildings and related literature [46,47], the parameters ω g and ς g are selected as ω g = 15.7 rad/s and ς g = 0.72. A frequent earthquake with intensity of VII is considered. Therefore, the spectral intensity of the stationary white noise process S 0 is equal to 15.21 cm 2 /s 3 . The parameters of amplitude envelope function are given as: η = 0.1, ε t = 0.2 and t n = 40 s. Figure 5 presents a sample generated by the GKT model.
The 3D building is considered to be located in a 2nd class site and the design earthquake group belongs to group 2. According to the Chinese code for seismic design of buildings and related literature [46,47], the parameters g ω and g ς are selected as ωg = 15.7 rad/s and ςg = 0.72. A frequent earthquake with intensity of VII is considered. Therefore, the spectral intensity of the stationary white noise process S0 is equal to 15.21 cm 2 /s 3 . The parameters of amplitude envelope function are given as: =0.1 η , = 0.2 t ε and tn = 40 s. Figure 5 presents a sample generated by the GKT model.

Modeling and Dynamic Response Analysis of 3D Building
The 3D building is modeled by OpenSees [46] for further analysis. The analytic model in OpenSees can be described as in Figure 6. Every column and beam in the building is modeled through fiber approach [47]. Each fiber in the column or beam section can be assigned as a concrete or steel bar in order to simulate their combined action. The earthquake wave is applied in the direction of North to South (NS). The displacement and acceleration response of point A to C (as shown in Figure 6) is presented in Figure 7.

Modeling and Dynamic Response Analysis of 3D Building
The 3D building is modeled by OpenSees [46] for further analysis. The analytic model in OpenSees can be described as in Figure 6. Every column and beam in the building is modeled through fiber approach [47]. Each fiber in the column or beam section can be assigned as a concrete or steel bar in order to simulate their combined action. The earthquake wave is applied in the direction of North to South (NS). The displacement and acceleration response of point A to C (as shown in Figure 6) is presented in Figure 7.

Model Training
To validate the random response prediction capacity of the proposed Bayesian deep learning model, 1000 samples were generated by 3D building FE model through Monte-Carlo simulation. Out of these samples, 500 were utilized for model training while the rest were used for validation and analysis. Bayes back propagation algorithm and RMSprop algorithm were used to train the proposed Bayesian deep learning model and the initial learning rate was equal to 0.0001. The training process is presented in Figure 8 which describes the loss function, varied with training epochs. As shown, the total losses of Bayesian deep learning were reduced during modeling and after 80 epochs; the total losses arrived at the level of 10 −4 . This means the proposed model can be well trained by using the back propagation algorithm.

Model Training
To validate the random response prediction capacity of the proposed Bayesian deep learning model, 1000 samples were generated by 3D building FE model through Monte-Carlo simulation. Out of these samples, 500 were utilized for model training while the rest were used for validation and analysis. Bayes back propagation algorithm and RMSprop algorithm were used to train the proposed Bayesian deep learning model and the initial learning rate was equal to 0.0001. The training process is presented in Figure 8 which describes the loss function, varied with training epochs. As shown, the total losses of Bayesian deep learning were reduced during modeling and after 80 epochs; the total losses arrived at the level of 10 −4 . This means the proposed model can be well trained by using the back propagation algorithm.

Model Training
To validate the random response prediction capacity of the proposed Bayesian deep learning model, 1000 samples were generated by 3D building FE model through Monte-Carlo simulation. Out of these samples, 500 were utilized for model training while the rest were used for validation and analysis. Bayes back propagation algorithm and RMSprop algorithm were used to train the proposed Bayesian deep learning model and the initial learning rate was equal to 0.0001. The training process is presented in Figure 8 which describes the loss function, varied with training epochs. As shown, the total losses of Bayesian deep learning were reduced during modeling and after 80 epochs; the total losses arrived at the level of 10 −4 . This means the proposed model can be well trained by using the back propagation algorithm.

Response Prediction Results
The response prediction results are provided in the form of PDF, illustrated in Figure 9. Since the shapes of time history response curvatures at A, B and C points were similar, we only discuss the response at point A as our example. As shown in Figure 10, the samples of random response can be included in the section of [µ − 3σ, µ + 3σ], where µ is the mean value of response and σ is the standard deviation at point A. This result coincides with the conclusion of Gaussian distribution in probability theory. Figure 11 further compares the difference between Bayesian deep learning prediction results and Monte-Carlo simulation results in the mean value and standard deviation. The PDF of prediction result was close to the PDF from FEM based Monte-Carlo simulation. The normalized mean square errors (MSEs) of mean value were 1.92 × 10 −5 (acceleration) and 5.05 × 10 −5 (displacement), while the normalized MSEs of standard deviation were 1.17 × 10 −4 (acceleration) and 5.10 × 10 −4 (displacement). The peak errors of mean value were 2.56% and 5.99%, while the standard deviations were 5.34% and 10.23%. It was found that the prediction results of acceleration response had higher accuracy. This might be due to the weaker correlation between ground motion acceleration and structural displacement response.

Response Prediction Results
The response prediction results are provided in the form of PDF, illustrated in Figure  9. Since the shapes of time history response curvatures at A, B and C points were similar, we only discuss the response at point A as our example. As shown in Figure 10, the samples of random response can be included in the section of [μ − 3σ, μ + 3σ], where μ is the mean value of response and σ is the standard deviation at point A. This result coincides with the conclusion of Gaussian distribution in probability theory. Figure 11 further compares the difference between Bayesian deep learning prediction results and Monte-Carlo simulation results in the mean value and standard deviation. The PDF of prediction result was close to the PDF from FEM based Monte-Carlo simulation. The normalized mean square errors (MSEs) of mean value were 1.92 × 10 −5 (acceleration) and 5.05 × 10 −5 (displacement), while the normalized MSEs of standard deviation were 1.17 × 10 −4 (acceleration) and 5.10 × 10 −4 (displacement). The peak errors of mean value were 2.56% and 5.99%, while the standard deviations were 5.34% and 10.23%. It was found that the prediction results of acceleration response had higher accuracy. This might be due to the weaker correlation between ground motion acceleration and structural displacement response.         be used as a surrogate model for seismic reliability analysis when it is combined wit Monte-Carlo simulation and first passage probability theory.

Discussion
In this section, the influence of structural parameter randomness and the input nois level are discussed. The PDF of peak acceleration and peak displacement are compared as well as their mean value and standard deviation value. Furthermore, two indices, be low, are introduced as: where ( ) t X is the building response at point A, calculated by FEM, and ˆ( ) t X is the pre diction response. Peak ( ) is the peak value function, which selects the absolute maximum value from input. Mean ( ) is used to calculate the mean value of the input and Std ( calculates the standard deviation of the input. Equations (12) and (13) present the predic tion errors of mean value and standard deviation value.

Influence of Parameter Randomness
To discuss the influence of parameter randomness, we considered three different lev els of randomness. The variable coefficients of all random parameters were 0.5, 1 and times the original coefficients, respectively. Slight randomness corresponds to 0.5 time variable coefficients, while 1.0 and 2.0 times correspond to medium and strong random ness. Figure 13 presents and compares the Errormean and Errorstdv of acceleration and dis placement responses at point A under different randomness levels. It can be conclude

Discussion
In this section, the influence of structural parameter randomness and the input noise level are discussed. The PDF of peak acceleration and peak displacement are compared, as well as their mean value and standard deviation value. Furthermore, two indices, below, are introduced as: |Mean(Peak(X(t)))| (12) Error stdv = Std(Peak(X(t))) − Std(Peak(X(t))) |Std(Peak(X(t)))| , where X(t) is the building response at point A, calculated by FEM, andX(t) is the prediction response. Peak ( ) is the peak value function, which selects the absolute maximum value from input. Mean ( ) is used to calculate the mean value of the input and Std ( ) calculates the standard deviation of the input. Equations (12) and (13) present the prediction errors of mean value and standard deviation value.

Influence of Parameter Randomness
To discuss the influence of parameter randomness, we considered three different levels of randomness. The variable coefficients of all random parameters were 0.5, 1 and 2 times the original coefficients, respectively. Slight randomness corresponds to 0.5 times variable coefficients, while 1.0 and 2.0 times correspond to medium and strong randomness. Figure 13 presents and compares the Error mean and Error stdv of acceleration and displacement responses at point A under different randomness levels. It can be concluded that, as the randomness of the structure parameters increases, the prediction errors increase. Due to the uncertainty simulation capacity of Bayesian deep learning, the increases in errors are considered to be limited.

Influence of Input Noise
Robustness is the anti-noise capacity of the proposed model. In engineering practice, the input ground motion acceleration signal is polluted by noise from different sources. Therefore, robustness is very important to a deep learning model for response prediction. To discuss the influence of noise in input ground motion and the robustness of the deep learning model, ground motion data were added with 1%, 2%, 5%, 10% and 20% Gaussian white noise [48]. Figure 14 shows the Errormean and Errorstd of acceleration and displacement prediction results at point A. The mean value prediction result remained stable under different noise levels but the standard deviation results fluctuated with the noise level. However, there was no evidence that noise would influence the standard deviation prediction results, since Errorstd does not increase with noise level. Therefore, the proposed Bayesian deep learning model is robust, even in the case of a high noise level. This is an advantage of the proposed model.

Influence of Input Noise
Robustness is the anti-noise capacity of the proposed model. In engineering practice, the input ground motion acceleration signal is polluted by noise from different sources. Therefore, robustness is very important to a deep learning model for response prediction. To discuss the influence of noise in input ground motion and the robustness of the deep learning model, ground motion data were added with 1%, 2%, 5%, 10% and 20% Gaussian white noise [48]. Figure 14 shows the Error mean and Error std of acceleration and displacement prediction results at point A. The mean value prediction result remained stable under different noise levels but the standard deviation results fluctuated with the noise level. However, there was no evidence that noise would influence the standard deviation prediction results, since Error std does not increase with noise level. Therefore, the proposed Bayesian deep learning model is robust, even in the case of a high noise level. This is an advantage of the proposed model.

Influence of Input Noise
Robustness is the anti-noise capacity of the proposed model. In engineering practice, the input ground motion acceleration signal is polluted by noise from different sources. Therefore, robustness is very important to a deep learning model for response prediction. To discuss the influence of noise in input ground motion and the robustness of the deep learning model, ground motion data were added with 1%, 2%, 5%, 10% and 20% Gaussian white noise [48]. Figure 14 shows the Errormean and Errorstd of acceleration and displacement prediction results at point A. The mean value prediction result remained stable under different noise levels but the standard deviation results fluctuated with the noise level. However, there was no evidence that noise would influence the standard deviation prediction results, since Errorstd does not increase with noise level. Therefore, the proposed Bayesian deep learning model is robust, even in the case of a high noise level. This is an advantage of the proposed model.

Conclusions
In this paper, a Bayesian deep learning method was proposed to predict the seismic response of a structure with random characteristics. A deep convolutional neural network, with random parameters, was established to predict the structural response. Bayes

Conclusions
In this paper, a Bayesian deep learning method was proposed to predict the seismic response of a structure with random characteristics. A deep convolutional neural network, with random parameters, was established to predict the structural response. Bayes back propagation algorithm was introduced to train the deep learning model. To validate the proposed model, a 3D building structure model with random parameters was constructed in OpenSees to obtain the random response under a GKT earthquake. The Bayesian deep learning model was utilized to predict the PDF of the response. The influence of randomness of structural parameters and the noise due to ground motion were discussed. The following conclusions can be drawn: (1) The proposed Bayesian deep learning model can reliably predict the random response of a 3D building structure with random parameters and obtain the PDF of response. The prediction of mean value is better than the prediction of standard deviation because of the parameter randomness. (2) As the parameter randomness increases, the effective performance of Bayesian deep learning model deteriorates. Compared with mean value prediction, the standard deviation prediction is more seriously affected by parameter randomness. (3) Input noises have very little influence on the proposed Bayesian deep learning model.
The proposed model is robust and can work under a high noise level of 20%. Therefore, the proposed model has promising potential for practical applications in civil, mechanical, and aerospace engineering.