Hybrid Wavelet Scattering Network-Based Model for Failure Identification of Reinforced Concrete Members

Abstract

After earthquakes, qualified inspectors typically conduct a semisystematic information gathering, physical inspection, and visual examination of the nation’s public facilities, buildings, and structures. Manual examinations, however, take a lot of time and frequently demand too much work. In addition, there are not enough professionals qualified to assess such structural damage. As a result, in this paper, the efficiency of computer-vision hybrid models was investigated for automatically detecting damage to reinforced concrete elements. Data-driven hybrid models are generated by combining wavelet scattering network (WSN) with bagged trees (BT), random subspace ensembles (RSE), artificial neural networks (ANN), and quadratic support vector machines (SVM), named “BT-WSN”, “RSE-WSN”, “ANN-WSN”, and “SVM-WSN”. The hybrid models were trained on an image database containing 4585 images. In total, 15% of images with different sorts of damage were used to test the trained models’ robustness and adaptability; these images were not utilized in the training or validation phase. The WSN-SVM algorithm performed best in classifying the damage. It had the highest accuracy of the hybrid models, with a value of 99.1% in the testing phase.

1. Introduction

Structural damage diagnosis has drawn a lot of attention as an essential subject in the area of structural health monitoring (SHM) over the years [1,2]. Data-driven damage diagnosis of structures has steadily evolved into one of the primary research areas in the SHM domain [3], particularly in recent years with the accelerated growth of statistical machine learning (ML), deep neural networks, and other mathematical methodologies. As an example, He et al. [4] proposed a unique hybrid model based on echo state and convolutional neural networks to efficiently derive the time-frequency characteristics of structures for damage identification. Shamila Ebenezer et al. [5] developed an ensemble model to determine both bridge and road damage using a transfer learning-based method. Their ensemble model was created using a proprietary convolutional neural network (CNN) as well as trained Xception and AlexNet architectures to increase accuracy rate and efficiency. Fallahian et al. [6] constructed a majority-voting based ensemble model based on the decomposition of vibration data using discrete wavelet transforms to overcome uncertainty conditions such as noise. Liu et al. [7] used 1-D CNN and structural transmissibility functions to create a framework for vibration-based damage diagnosis. They claimed the suggested model displayed more notable damage-sensitive characteristics and higher robustness under excitation interference. In particular, regarding using 1-D CNN, several distinct advantages over other machine learning algorithms in terms of computation efficiency, generalization capability, and noise tolerance were reported. To identify corrosion from drone photos, Forkan et al. [8] offered an ensemble deep learning technique supported by CNNs. They offered an empirical assessment utilizing actual photographs of a complex structure (like a telecommunications tower) taken by drones, which is a common occurrence for engineers. Their analysis showed that, in terms of classification accuracy, the ensemble technique performs noticeably better than the traditional algorithms. Convolutional neural network-based crack detection method, GoogLeNet-Inception V3, was presented by Wu et al. [9]. They utilized a crack image database that consisted of almost 3600 images. After that, the GoogLeNet-Inception V3 architecture was retrained using the database and a learning method called “transfer learning.” This made it better at finding crack images.

In addition, Zawad et al. [10] conducted a comprehensive literature review and a critical examination of the crack detection by image processing concept to comprehend the potential of this approach. They suggested that a universal CNN-based algorithm with camera pictures for crack identification could be a useful strategy with greater accuracy. However, numerous flaws, such as the inability to distinguish between cracks and noises, and the difficulty in spotting tiny fractures, have made this technique extremely difficult. Additionally, finding transverse cracks in concrete structures is another critical problem for some algorithms [11]. In order to distinguish between photos of cracked concrete surfaces and images without cracks, Padsumbiya [12] suggested a straightforward CNN-based model employing pixel-level information, which does away with the requirement for expensive digital image capture equipment. Pham et al. [11] investigated the new notion of training an image-based model for loosened bolt identification by utilizing synthetic images of bolts produced from a graphical model. They created a methodology for bolt-loosening identification based on computer graphics and deep learning. The bolt-loosening tracking of a lab-scaled bolted joint model serves as evidence of the viability of the suggested architecture. Additionally, the model’s effectiveness is assessed using the actual bolted connections of a historic truss bridge. Li et al. [13] introduced a CNN-based model for the multiple damage identification of concrete structures using 2750 images. The variables of the model are initialized during the training phase using a transfer learning technique. Their model could distinguish cracks, spalling, and efflorescence with an acceptable degree of accuracy. Wang et al. [14] created a CNN-based model for identifying building structure deterioration using a time-frequency plot and the signal’s marginal spectrum. To provide superior CNN efficiency, the intrinsic parameters of the CNN model are also adaptively tuned by the particle swarm optimization algorithm (PSOT). The experimental findings demonstrate that the novel introduced damage identification technique has considerable performance advantages over existing conventional methods (artificial neural network and support vector machine). The proposed CNN model’s accuracy increased by over 10% following PSOT’s optimization.

Furthermore, Zhan et al. [15] investigated the impact of structural variable randomness on CNN’s capacity to accurately identify damage, as well as the coupled effect of structural variable unpredictability and sampling noise on CNN’s ability to withstand noise. The findings demonstrate that the randomness of structural characteristics will have an impact on CNN’s ability to accurately identify damage, and this impact cannot be neglected. A deep-learning classifier-based model with generated transmittance deviations (GTD) was introduced by Seventekidis and Giagopoulos [16]. They create a comprehensive training dataset in their work by approximating the GTD for each scenario in the finite element model by decreasing the stiffness of the associated components while simulating various uncertainties simultaneously. Ogunjinmi et al. [17] demonstrate how deep learning improves the evaluation of earthquake damage from the Pohang earthquake. Using transfer learning, six traditional pretrained CNN models (Inception, ResNet, MobileNet, Xception, VGG16, and VGG19) are applied to a dataset of 1780 photos exhibiting structural damage. On the datasets, feature extraction and fine-tuning techniques were applied. They compared the performances of multiple CNN models. The MobileNet fine-tuned model had the greatest result. Bai et al. [18] suggested that it is impracticable to capture correct nature frequency of the structure before filtering the camera movements since there is more than one peak from the camera movements. They came up with a new technique to filter the impact of camera motions using background templates. The frequencies of the target motions were effectively determined using the established filtering technique. According to the findings, filtering out camera movements greatly increased the efficiency of displacement monitoring by roughly 40%. It is important to note that these ML models have been successfully used in different applications of civil engineering [19,20,21,22,23,24,25,26,27,28,29,30,31,32,33].

A swift and precise evaluation of the type of structural damage is essential for emergency response and recovery design following a seismic event [34]. The typical methods for determining the damage to concrete structures are mostly associated with manual data collection, visual identification, and subjective judgment and interpretation. In addition, it is challenging to install contact conventional tools to monitor strains and displacements for the SHM on bridges and high-rise structures. On the other hand, the development of new methods for identifying damage has received more attention in recent years because existing approaches are time-consuming and prone to human error. As applying vision-based methods to structural engineering issues is still a relatively new research topic, there are still some major issues that need to be resolved. For structural damage diagnosis, for instance, there is an absence of a defined automated identification approach or framework based on available knowledge with sufficient accuracy. The purpose of this study is to determine whether hybrid wavelet scattering network (WSN)-based models could be used to determine damage to reinforced concrete (RC) elements. WSN is utilized to extract reliable features from images since they have a lot of advantages, including stable to small deformation, rotation invariant, and so on [35]. Bagged trees with WSN (BT-WSN), random subspace ensemble with WSN (RSE-WSN), artificial neural network and WSN (ANN-WSN), and quadratic support vector machine with WSN (SVM-WSN) are the hybrid models that were explored in this study.

2. Dataset of Damaged and Undamaged RC Elements

The damage type has a direct impact on the mechanical properties of the RC members [36]. Failures of RC elements can be broadly categorized into three major groups (flexural, shear, and flexural-shear). The Pacific Earthquake Engineering Research Center (PEER) provides a four-class classification benchmark for the damage type of the RC members [37]. The database contains 4585 color images (three-channels) with a resolution of 224 × 224 pixels.

Table 1. Four-class classification for damage type.

State	Undamaged	Flexural Damage	Flexural-Shear (Combined) Damage	Shear Damage	Total
Symbol	UN	FD	CD	SD	-
Sample number	1813	522	1325	925	4585

presents the corresponding sample numbers of the three-damage and undamaged types in the database. Additionally, [37] provides additional detail, such as how features used to label the same type of damage. Here, the training, testing, and validation datasets contain the 70%, 15%, and 15% of the samples, respectively. Figure 1 displays sample images of FD damage, SD damage, and CD damage.

3. Hybrid Models’ Details

WSNs are frequently used to identify and manage nonlinear systems. The capacity of WSN to map complex nonlinear functions is greatly improved because the wavelets exhibit time-precise characteristics in high-frequency regions and frequency-precise characteristics in low-frequency regions as a result of the dilation and translation of the mother wavelet. The wavelet analysis yields a collection of values that show how closely the signal resembles a specific basis function. Equation (1) is used to conduct such analysis.

$$WaveletTransform(WT)=\frac{1}{\sqrt{\left|d\right|}}{\displaystyle {\int }_{-\infty }^{+\infty }x(s){\psi }^{*}(\frac{s-t}{d})ds}$$

(1)

where $*$, $x(s)$, $\psi$, $t$, and $d$ denote complex conjugate, signal, mother wavelet, dilation, and translation parameters, respectively. Convolution of the inputs with a mother wavelet that has been rotated by $\theta$ and dilated by $2^j$ produces the wavelet transform (WT):

$${\psi }_{j,\theta }=\frac{\psi (2^{-j}\cdot R)}{2^j}=2^j\psi (2^{-j}\cdot \theta x)$$

(2)

where $R$ is the rotation matrix and $j$ is the scale. $\lambda =(j,k)$ is defined as where $k$ indexes $\theta$ to give $K$ angles in $(0,\pi )$; thus, the WT can be expressed as:

$$\begin{array}{l}{\psi }_{j,\theta }\to {\psi }_{\lambda }\\ W{T}_{x(u)}=\{x(u)*{\varphi }_J(u),x(u)*{\psi }_{\lambda }(u)\}\\ takingmodulus|.|:\tilde{W}{T}_{x(u)}=\{x(u)*{\varphi }_J(u),|x(u)*{\psi }_{\lambda }(u)\left|\right\}\end{array}$$

(3)

where $u$ is coordinates of the pixels and ${\varphi }_J(u)$ represents a low pass filter. For a path in which some $\lambda s$ ($p=({\lambda }_1,{\lambda }_2,\dots ,{\lambda }_m)$) exist, by defining modulus terms as $U[\lambda ]x=|x*{\psi }_{\lambda }|$ and acting on $p$, we get:

$$U[p]x=U[{\lambda }_m]\dots U[{\lambda }_2]U[{\lambda }_1]x=\left|\right|\dots |x*{\psi }_{{\lambda }_1}|*{\psi }_{{\lambda }_2}|\dots *{\psi }_{{\lambda }_m}|$$

(4)

The scattering coefficients are calculated as follows:

$$S[p]x(u)=U[p]x*{\varphi }_J(u)$$

(5)

The structure of the hybrid WSN-based algorithms is shown in Figure 2. In this study, four wavelets per octave in the first and second layers are used. Two-dimensional Morlet wavelets are used as the mother wavelet. Scattering coefficients are averaged over channels and are fed into a metalearner to predict the type of structural damage. Four different metalearners are considered, including bagged trees, random subspace ensemble, artificial neural network, and quadratic support vector machine (SVM). Regarding the selection of metalearner models, using models developed with different intellectual approaches has been attempted. For instance, the data are sent into a higher dimensional space in the SVM so that they can be linearly separated using the kernel approach [39]. An array of linear combinations blended with (often) activation functions is carried out across a number of layers in a neural network [39]. Neuronal networks adjust parameters to convert the input and, inadvertently, control the activity of neighboring neurons. Decision trees directly adapt parameters to control the information stream. Additionally, ensemble learning integrates predictions from various neural network models to lower forecast variability and generalization error.

Typically, a set of input variables is used to characterize the training data. Different input variable groups, also known as subspaces, offer various perspectives on the data. Individual weak learners who have been instructed in several subspaces are therefore often heterogeneous. By selecting random subsets of the input features, a basis is provided for an ensemble learning process where a model can be trained on each random input variable subspace. This method is also referred to as the Random Subspace Ensemble (RSE). The classifier is made up of several k-NN classifications that were built in a systematic manner by pseudorandomly choosing subsets of the feature vector’s components. Here, the cross-validation method is utilized to determine parameters for both the weak learner (nearest neighbors) and the ensemble (RSC) using a training set. Figure 3 depicts the number of nearest neighbors in relation to the classification error. The lowest error occurs when the number of nearest neighbors is three. Now, ensembles for the three-nearest neighbor classification are made with different numbers of dimensions (number of input features), and their cross-validated loss is examined. Figure 4 shows the effect of various numbers of dimensions using training set. It can be seen from the figure that using five predictors per learner has the lowest cross-validated error.

The last stage is to identify the ensemble’s minimal number of learners who can still provide accurate classification. Figure 5 illustrates the number of learners in ensemble versus classification error using the training set. It is evident from Figure 5 that the classification error declines dramatically to approximately 0.15 and seems to be no advantage in an ensemble with more than 150 or so learners. Therefore, 150 learners are utilized.

Shallow artificial neural network (ANN) typically consists of three layers: an input layer, a hidden layer, and an output layer. Regression and classification tasks can both be accomplished using neural networks. Just like biological neurons do, artificial neurons contain an input and an output. Artificial neurons receive input, multiply them by weights, add them all up, and then send the results to a nonlinear function. The output of a node is defined as a function of an input or group of inputs by the activation function of that node. ANNs are made up of linked artificial neurons, much like biological neuron networks. They only have one or two hidden layers since adding more layers makes the computation more difficult.

The ANN used in this study had one hidden layer. The activation functions of the hidden layer and output layer were “Relu” and “Softmax”, respectively. The ideal number of neurons was identified by varying the number of neurons in the hidden layers and analyzing the error for various cases using the training set. As observed in Figure 6, the hidden layer with 10 neurons had the lowest error value. Therefore, the neural network with ten neurons in the hidden layer was considered for further study.

SVM is a supervised learning method within the field of machine learning. Tasks involving data classification can be resolved with it. The aim of SVM is to identify a function $f(x)$ (Equation (6)) that predicts the output value with a maximum epsilon divergence from the reference value. To put it another way, an inaccuracy greater than epsilon is not acceptable.

$$f(x)={\displaystyle \sum _{i=1}^n({\alpha }_i-{\alpha }_i^{*})}K+b$$

(6)

where ${\alpha }_i$ and ${\alpha }_i^{*}$ are the Lagrange multipliers, $K$ is a quadratic kernel function, and $b$ is bias term.

The key concept behind bagged trees (BT) is that the model relies on numerous decision trees rather than just one, which enables the model to take advantage of the wisdom of multiple decision trees. A decision tree’s large volatility is generally considered to be its major drawback. This is a problem since even a small change in the data can have a significant impact on the model and forecasts for the future. One of the advantages of bagged trees is that they reduce variation while maintaining bias consistency [23]. The key parameter in BT is the number of classification trees. By increasing the number of classification trees and using training sets, the error value of the model is stored each time to examine the impact of the number of classification trees on the model’s performance. The findings are displayed in Figure 7. It is evident from Figure 7 that the classification error declines dramatically to approximately 0.05 and seems to be no advantage in a BT with more than 160 or so classification trees. Therefore, 160 classification trees are utilized.

4. Results

To evaluate the efficacy and reliability of each of the hybrid models more thoroughly, a confusion matrix (CM) was employed. The CM is a table that contrasts actual damage with damage that was anticipated. The following measures are used to evaluate the reliability of the hybrid models:

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}=\frac{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}+\mathrm{T}\mathrm{N}}$$

(7)

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}$$

(8)

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}$$

(9)

where true positive (TP): a real damage is predicted correctly. False positive (FP): a real nondamage is predicted as a damage. False negative (FN): a real damage is predicted as a nondamage. True negative (TN): a real nondamage is predicted correctly. Recall indicates the percentage of accurate predictions for each real (actual) damage, whereas precision is the percentage of correctly predicted outcomes for each anticipated damage type. On the other hand, accuracy (the lowest cell on the right side of the matrix) is a general performance indicator that shows the proportion of reliable predictions across all four failure categories across the full dataset. Models were trained with a computer with an Intel (R) Intel(R) Core (TM) i7-6700HQ CPU @ 2.60GHz, 16 GB RAM, and an NVIDIA GTX960M. Extracting features using the WSN-based algorithm took almost 4 h, but training metalearners with the extracted features took almost less than 15 min for each metalearner.

The performance of the proposed hybrid models on the training set is shown in Figure 8. As it can be seen, all the models, except for the WSN-SVM algorithm, in the training phase have obtained the highest score (100%) in terms of all criteria. Only the WSN-SVM algorithm has 99.8% accuracy, and there was one error in SD type failure detection. From the CMs in Figure 9, it can be observed that, in the testing phase, the WSN-SVM algorithm performed best in classifying the damage. It has the highest accuracy of the hybrid models, with a value of 99.1% in the testing phase. The WSN-BT algorithm and WSN-ANN algorithm are ranked second and third, after the WSN-SVM algorithm, in terms of damage classification accuracy. While the recognition of CD damage is challenging in other research studies [40,41], the WSN-SVM algorithm herein achieved an almost 99.1% recall and 99.5% precision in the CD failure mode prediction in the test phase. However, it seems that the identification of the CD damage is difficult for the other proposed hybrid models. The WSN-RSE algorithm showed the lowest efficiency in terms of a higher number of incorrect failure mode recognition, with over 80 incorrect classifications.

Table 2. Comparison of models’ F1 score.

Models	Accuracy (%)
WSN-ANN	0.943
WSN-BT	0.953
WSN-RSE	0.953
WSN-SVM	0.991

presents the computed F1 score using the micro averaging method to consider the imbalanced dataset effect (multiclass/multilabel case). This investigation demonstrates that the WSN-SVM algorithm was superior compared to the WSN-ANN, WSN-BT, and WSN-RSE algorithms in terms of F1 score. Both the WSN-BT and WSN-RSE algorithms have the same F1 score. The WSN-ANN Algorithm had the lowest F1 score. Figure 10 shows the learning curves of the algorithms.

5. Comparative Study

In this section, the best proposed model (WSN-SVM algorithm) is compared with other models that have previously been developed using the same database (Section 2, PEER benchmark). Gao and Mosalam [38] employed the transfer learning method, using the pertinent components of a machine learning model that has already been trained to solve a different but related problem and certain CNN (benchmark) models for damage identification, including VGG-16 [42], VGG-19 [42], and ResNet-50 [43] models. In addition, Barkhordari et al. [44] identified structural damage using a hybrid ensemble model, which combines the weighted average ensemble of the pretrained models (MobileNetV2 and NASNetMobil) with differential evolution (WAE-DE).

Table 3. Comparison of previous models and the WSN-SVM algorithm.

Models	Accuracy (%)
VGG-16 [38]	70.53
VGG-19 [38]	72.36
ResNet-50 [38]	68.09
WAE-DE [44]	94.04
WSN-SVM	99.1

provides an overview of the models’ performance in terms of overall accuracy. WSN-SVM performs better than other ML models employed to detect structural damage. The model based on ResNet-50 [24] has the lowest accuracy value. Moreover, it is evident that there is little difference in the accuracy value of the three models, ResNet-50, VGG-16, and VGG-19. Gao and Mosalam [38] only reported the accuracy value of their models. However, Barkhordari et al. [44] also mentioned the recall and precision values of the WAE-DE, which are shown in

Table 4. Comparison of models’ recall and precision.

	Models	Failure Mode
		CD	FD	UD	SD
Recall (%)	WAE-DE	96.17	91.57	97.3	87.73
	WSN-SVM	99.1	98.6	99.6	98.5
Precision (%)	WAE-DE	93.12	96.7	91.97	96.62
	WSN-SVM	99.5	97.3	99.3	99.2

. Table 4 reveals that the proposed model (WSN-SVM) achieved the highest performance in terms of recall and precision for all failure modes.

6. Conclusions

This paper investigated applications of hybrid wavelet scattering network (WSN)-based models with a special focus on determining damage to reinforced concrete (RC) elements. The hybrid models investigated in this study are bagged trees with WSN (BT-WSN), random subspace ensemble with WSN (RSE-WSN), artificial neural network and WSN (ANN-WSN), and quadratic support vector machine with WSN (QSVM-WSN). The following conclusions can be drawn:

• WSNs were great for extracting domain features from image data.
• In the training stage, all models, apart from the WSN-SVM algorithm, received the highest score (100%) across all criteria. Only the WSN-SVM technique had a 99.8% accuracy rate and had one error for SD type failure.
• The good performance of a model in the training phase does not mean it will have a good performance on new data.
• The WSN-SVM algorithm classified the damage the most accurately throughout the testing phase. With a value of 99.1 percent during testing, it had the hybrid models’ highest accuracy. The differences in the performance evaluation criteria for this model in the two phases of training and testing were very close.

The proposed methodology is not only entirely data-driven, i.e., it does not rely on directed signal processing, but it also extracts useful characteristics from the data and applies machine learning to them to build the model. The proposed models require a considerable amount of data to work well, so increasing the quantity and quality of data is crucial. Future studies can expand the limited data sets. Additionally, future studies can focus on minimizing the impact of unstructured and potentially anomalous data sets. In addition, machine learning models must be made more robust. This can be accomplished using the bagging and bootstrap method, which is frequently employed for building machine learning models.