Deep Learning Enriched Automation in Damage Detection for Sustainable Operation in Pipelines with Welding Defects under Varying Embedment Conditions

Abstract

Welded joints in metallic pipelines and other structures are used to connect metallic structures. Welding defects, such as cracks and lack of fusion, are vulnerable to initiating early-age cracking and corrosion. The present damage identification techniques use ultrasonic-guided wave procedures, which depend on the change in the physical characteristics of waveforms as they propagate to determine damage states. However, the complexity of geometry and material discontinuity (e.g., the roughness of a weldment with or without defects) could lead to complicated wave reflection and scatters, thus increasing the difficulty in the signal processing. Artificial intelligence and machine learning exhibit their capability for data fusion, including processing signals originally from ultrasonic-guided waves. This study aims to utilize deep learning approaches, including a convolutional neural network (CNN), Long-short term memory network (LSTM), or hybrid CNN-LSTM model, to demonstrate the capability in automation for damage detection for pipes with welded joints embedded in soil. The damage features in terms of welding defect types and severity as well as multiple defects are used to understand the effectiveness of the hybrid CNN-LSTM model, which is further compared to the two commonly used deep learning approaches, CNN and LSTM. The results showed the hybrid CNN-LSTM model has much higher classification accuracy for damage states under all scenarios in comparison with the CNN and LSTM models. Furthermore, the impacts of the pipelines embedded in different types of materials, ranging from loose sand to stiff soil, on signal processing and data classification were further calibrated. The results demonstrated these deep learning approaches can still perform well to detect various pipeline damage under varying embedment conditions. However, the results demonstrate when concrete is used as an embedding material, high attention to absorbing the signal energy of concrete could pose a challenge for the signal processing, particularly under high noise levels.

1. Introduction

Welding is often used to connect metallic structures [1,2,3], including connecting metallic parts for oil and gas pipelines and other civil structures [4]. Different forms of welding faults, including partial penetration, lack of fusion, cracking, and undercut are frequently observed due to the complicated properties of the welding procedure used in shops and on building sites [4]. As a result, even seemingly minor welding errors frequently lead to early-age damage in materials and structures, such as corrosion caused by cracks [4,5,6]. Thus, it is essential to inspect the weldment for structural health monitoring (SHM) [7,8,9] during fabrication, construction, and afterward during the in-service phase to ensure welding quality [10,11,12,13,14,15].

Nondestructive testing methods, including ultrasonic, dye penetration, magnetic particle, and eddy current are widely used in pipeline damage detection [16,17,18]. But for online testing, dye penetrant and magnetic particle testing typically necessitate removing the tested portion [16]. Eddy current testing cannot be used on non-standard surfaces [16]. However, ultrasonic-guided wave testing [4,19,20] can be used to carry out large-scale and quantitative damage detection because of its benefits of non-contact, a large working area, and high sensitivity [16,21,22].

Table 1. Comparison of ultrasonic-guided wave testing with other non-destructive testing techniques for detecting pipe welding defects.

Method	Advantages	Disadvantages	Accuracy	Cost	Ease of Use
Ultrasonic-Guided Waves	Non-invasive	Equipment setup can be complex	High	Moderate	Moderate
X-ray Radiography	Excellent defect visualization	Ionizing radiation, requires safety precautions	Very High	High	Complex
Magnetic Particle	Portable and cost-effective	Only detects surface defects	Moderate	Low	Easy
Eddy Current Testing	Detects surface and some subsurface defects	Requires skilled operators	Moderate	Moderate	Moderate

compares ultrasonic-guided wave testing with other non-destructive testing techniques for detecting pipe welding defects.

Weldment is a part of pipelines and is vulnerable to damage [23,24,25,26]. Rattanawangcharoen et al. [27] used the finite element approach and wave expansion function to simulate the propagation of ultrasonic-guided waves with various weld shapes in thin-walled cylinders and studied the dispersion of axisymmetric guided waves in the area where the cylinders’ bonding materials were located. The resonance peak of the reflection coefficient was more pronounced as the joint thickness increased. The proposed method could be used to evaluate weld faults quantitatively and without causing any damage. Zhuang et al. [28] also simulated the propagation of a symmetrical guided wave mode in welded steel pipes based on finite element analysis and analyzed the difference between weld joints with and without faults in the reflection coefficient.

Deep learning approaches have demonstrated their robustness in the signal processing of ultrasonic-guided waves [29,30,31,32]. CNN is one of the representative deep learning algorithms and it has the properties of local perception and parameter sharing, which enables CNN to efficiently learn the associated features from many samples [16,33]. Xu et al. [34] used features extracted from the guided wave on several monitoring paths to train CNN and determined the length of the fatigue fracture. Su et al. [35] established a damage classification model based on CNN. Frequency-domain characteristics of the ultrasonic-guided wave were used to train the model. Feng et al. [36] developed a CNN model to detect damage based on image detection. Kumaresan et al. [37] used CNN for transfer learning to classify welding defects. But, the CNN model did not consider the temporal connection of the collected signals. LSTM networks, a subset of Recurrent Neural Networks (RNNs), are renowned for their prowess in time series analysis, adept at capturing intricate temporal patterns. They have excelled in multiple fields: financial markets [38,39], forecasting stock prices [40], healthcare, predicting patient outcomes from health records [41], and natural language processing [42]. LSTM’s adaptability and ability to tackle sequential data make them indispensable in a broad range of domains. LSTM has also been used to identify the damage information in the vibration signal damage because it can maintain the temporal correlation of vibration signals [16]. Zhao et al. [43] proposed the CNN and LSTM networks to achieve early damage detection and identify the cantilever beam breathing crack. Choe et al. [44] used gated recurrent unit neural networks with LSTM to identify damage related to structures, which resulted in high damage identification performance. The time series features obtained by LSTM can be used to improve the damage detection accuracy, as demonstrated by the literature mentioned above.

In this study, the CNN-LSTM network was used to detect pipeline damage with different types of welding defects in a notch. Firstly, twenty-nine feature parameters were calculated and compared based on the training performance of different deep learning models, including CNN, LSTM, and CNN-LSTM models. The CNN-LSTM hybrid model was expected to achieve the highest performance because of its complex structure. It combined CNN and LSTM networks; as a result, it could help the hybrid network to extract temporal information comprehensively, which reduces the side effects of the CNN network. The LSTN network is followed the CNN network; the compressed information from the CNN network could be input into the LSTM network directly, and it could improve the training efficiency of the CNN-LSTM hybrid model. Furthermore, noise interference, different types of defects, and different types of pipeline embedment were designed to verify the effectiveness of the CNN-LSTM hybrid model.

2. Deep Learning Enriched Automation in Damage Detection

The structure of methodologies is shown in Figure 1, including feature extraction, model training and testing, and classification. Different types of features were chosen, calculated, and trained by deep learning models. The most effective features were selected to express the signals’ information. Three deep learning approaches, including CNN, LSTM, and CNN-LSTM models, were then used to perform the data classification for detecting welding defects (type and severity). Noises were introduced to the original signals to discuss the robustness of the deep learning approaches.

2.1. CNN Model

CNN is a widely accepted deep learning approach with a deep neural network, consisting of convolutional, pooling, and a fully linked layer with a rectified linear activation function (ReLU), for data processing [45,46]. In this study, a one-dimensional (1D) signal was used as an input for the CNN network. The input data of the 1D signal vector is represented by $x=x_1, x_2, x_3, \dots , x_n, C$ where $x_n \in R^d$ stands for features (i.e., time series signal data) and C stands for a class label. From a collection of features $f$, the following new feature map $fm$ is created [45].

$${h{l}_i}^{fm}=\tanh \left(w^{fm}{x}_{i:i+f-1}+b\right),$$

(1)

where a feature map $hl=\left[h{l}_1, h{l}_2, \dots , h{l}_{i+f-1}\right]\in R^{i+f-1}$. The kernel $hl$ is applied to each array of features $f$ within the input data specified as $\{ x_{1:f}, x_{2:f+1}, x_{3:f+2}, \dots , x_{i:i+f-1} \}$, and $b \in R$ signifies a bias term [45].

The average pooling layer receives the output of the convolutional layer and down samples the data [47], which employs the ReLU activation function applying $aver(0, x)$ to each input to the ReLU represented by x [45]. Here, each feature map is subjected to the average-pooling process using the formula $\overrightarrow{hl}=aver \left\{hl\right\}$, which yields the most important features [45]. The fully connected layer, which contains the $softmax$ function and provides the probability distribution across each class, receives these chosen features as input [45]. As a result, the CNN network’s fully connected layer (FC) calculates the classes that make up its final output [45].

2.2. LSTM Model

LSTM was developed to overcome recurrent neural networks (RNNs) with consideration of long-term memory of time-dependent information [45]. This implies LSTMs possess the capacity to retain and establish connections between preceding data—often considerably distant in time from the current moment—to the present context [45]. With the progression of LSTM research, enhancements such as the introduction of forget gates and peephole connections became integrated into the LSTM network [45]. The forget gate replaces the constant error carousel (CEC) and aids in forgetting or resetting the states of memory cells [45].

The functioning of the LSTM is as described below. The LSTM architecture receives an input sequence of data with length $x=(x_1, x_2, x_3, \dots , x_n)$ that can be any length [45]. Within the recurrent concealed stratum of the LSTM framework, the resultant sequence $x=\left( x_1, x_2, x_3, \dots , x_n\right)$ is computed in an iterative manner, progressing from t = 1 to T. This is achieved through consistent write, read, and reset actions executed via the memory cell (me) of the input gate ($in$), forget gate ($for$), and output gate ($out$) [45,48]. The operation sequence at the time t can be described as follows [45].

$${in}_t=\mathsf{\sigma }\left(w_{xin}{x}_t+w_{hin}{h}_{t-1}+w_{mein}{me}_{t-1}+b_{in}\right)$$

(2)

$${for}_t=\mathsf{\sigma }\left(w_{xfor}{x}_t+w_{hfor}{h}_{t-1}+w_{mein}{me}_{t-1}+b_{for}\right)$$

(3)

$${me}_t={\mathit{for}}_t \odot {\mathit{me}}_{t-1}+{\mathit{in}}_t\odot \mathit{tanh}\left(w_{xme}{x}_t+w_{hme}{h}_{t-1}+b_{me}\right)$$

(4)

$${out}_t=\mathsf{\sigma }\left(w_{xout}{x}_t+w_{hout}{h}_{t-1}+w_{meout}{me}_t+b_{out}\right)$$

(5)

$$h_t={\mathit{out}}_t \odot \mathit{tanh}\left({me}_t\right)$$

(6)

The forget gate is essential for removing self-recurrent values no longer useful and maintaining them for the next time step by multiplying them with the memory cell. Additionally, peephole connections allow each gate and memory cell to determine the exact timings of their outputs [45]. This comprehensive LSTM architecture, with the forget gate and peephole connections, enhances the model’s ability to capture and utilize long-term dependencies in time series data.

2.3. CNN-LSTM Hybrid Model

A CNN-LSTM model was developed, as shown in Figure 2. The proposed CNN-LSTM model used the convolution1D and average pooling1D layers to extract features from a number of variables affecting the categorization of defect types and to reduce the data distribution [49,50]. The following LSTM layer receives input from the average pooling1D layer’s output.

$$y_i=\mathit{CNN}\left(x_i\right)$$

(7)

The original input vector of the CNN network, along with its corresponding class label, is denoted as $x_i$. The resulting output of the CNN network, represented by $y_i$, serves as the input for the subsequent LSTM network. In order to learn the long-range temporal relationships, the LSTM is fed with the feature vector created by the average pooling procedure in CNN.

In the context of detecting pipeline damage with various welding defects, the CNN-LSTM network serves as a powerful tool, offering efficient temporal feature extraction. In this study, a comprehensive analysis of deep learning models, including CNN, LSTM, and the CNN-LSTM hybrid, was conducted to determine the most suitable approach.

Table 2. The structure of CNN, LSTM and CNN-LSTM networks.

Network	Number of Layers	Layer Types	Hyperparameters
CNN	3	Sequence Input, Convolution, Average Pooling	Filters: 16, Padding: ‘same’, Dilation Factor: 1
LSTM	4	Sequence Input, Sequence Folding, LSTM, Dropout	Number of Units: 100
CNN-LSTM	7	Sequence Input, Sequence Folding, Convolution, Average Pooling, Sequence Unfolding, Flatten, LSTM, Dropout, Fully Connected, SoftMax	Filters: 16, Padding: ‘same’, Dilation Factor: 1, LSTM Units: 100

illustrates and compares the structure of CNN, LSTM and CNN-LSTM networks. In our MATLAB training process, the learning rate is set to 0.1, the mini-batch size is set to 16, and the number of LSTM units is 100. The chosen activation functions are ReLU for the CNN layers and hyperbolic tangent (tanh) for the LSTM layers.

The CNN-LSTM hybrid model was expected to outperform others due to its intricate structure. This model combines the strengths of both CNN and LSTM networks, resulting in the comprehensive extraction of temporal information. In the CNN, LSTM, and CNN-LSTM models, we included batch normalization layers to standardize the outputs of each layer. This was done to reduce the chances of overfitting and enhance the robustness of the optimization process. The effectiveness of batch normalization layers has been supported by prior research [51]. By integrating CNN before LSTM, the hybrid network can efficiently process spatial and sequential data. It captures essential spatial features via CNN and subsequently feeds this compressed information into LSTM for in-depth temporal analysis. This not only optimizes feature extraction but also reduces the potential side effects associated with using CNN in isolation.

The CNN-LSTM model was then put to the test under challenging conditions, including noise interference, various defect types, and distinct pipeline embedment scenarios. Its effectiveness in handling these complex and real-world situations was assessed. The combination of CNN and LSTM, offering a balance between spatial and temporal feature extraction, demonstrated its capability to robustly detect pipeline damage and welding defects in notches, making it a promising approach for real-world applications.

2.4. Features Extraction

Definition of Features

To define the fault characteristics in various types of damaged pipelines, 29 time- and frequency-domain feature parameters, comprising a total of 16 feature parameters in the time domain and 13 feature parameters in the frequency-domain, were chosen for this work. The detailed information is shown in

Table 3. Time-/frequency-domain feature indicators.

Time-Domain Features (16 Features)
Index of Characteristics	Formulations	Index of Characteristics	Formulations
$\mathrm{Mean} \mathrm{value} (\overline{X}$)	$\overline{X}=\frac{1}{N}{\displaystyle \sum _{i=1}^N}{x}_i$	$\mathrm{Kurtosis} (\beta$)	$\beta =\frac{1}{N}{\displaystyle \sum _{i=1}^N}{x}_i^4$
$\mathrm{Root} \mathrm{mean} \mathrm{square} \mathrm{value} (X_{rms}$)	$X_{rms}=\sqrt{\frac{1}{N}{\displaystyle \sum _{i=1}^N}{x}_i^2}$	$\mathrm{Variance} ({\sigma }_x^2$)	${\sigma }_x^2=\frac{1}{N-1}{\displaystyle \sum _{i=1}^N}{(x_i-\overline{X})}^2$
$\mathrm{Square} \mathrm{root} \mathrm{amplitude} (X_r$)	$X_r={\left[\frac{1}{N}{\displaystyle \sum _{i=1}^N}\sqrt{\left|x_i\right|}\right]}^2$	$\mathrm{maximum} \mathrm{value} (X_{max}$)	$X_{max}=max\left\{\left|x_i\right|\right\}$
$\mathrm{absolute} \mathrm{mean} \mathrm{amplitude} (\left|\overline{X}\right|$)	$\left|\overline{X}\right|=\frac{1}{N}{\displaystyle \sum _{i=1}^N}\left|x_i\right|$	$\mathrm{minimum} \mathrm{value} (X_{min}$)	$X_{min}=min\left\{x_i\right\}$
$\mathrm{Skewness} (\propto$)	$\propto =\frac{1}{N}{\displaystyle \sum _{i=1}^N}{x}_i^3$	peak-to-peak value $(X_{p-p}$)	$X_{p-p}=\max \left(x_i\right)-\mathrm{m}\mathrm{i}\mathrm{n}\left(x_i\right)$
$\mathrm{Waveform} \mathrm{Index} (S_f$)	$S_f=\frac{X_{rms}}{\left|\overline{X}\right|}$	$\mathrm{peak} \mathrm{index} (C_f$)	$C_f=\frac{X_{max}}{X_{rms}}$
$\mathrm{pulse} \mathrm{index} (I_f$)	$I_f=\frac{X_{max}}{\left|\overline{X}\right|}$	$\mathrm{margin} \mathrm{index} ({CL}_f$)	${CL}_f=\frac{X_{max}}{X_r}$
$\mathrm{kurtosis} \mathrm{index} (K_v$)	$K_v=\frac{\beta }{x_{rms}^4}$	$\mathrm{Skewness} \mathrm{Index} (S$)	$S=\frac{\propto }{x_{rms}^3}$
Frequency-domain features (13 features)
1	$p_1=\frac{{\sum }_{k=1}^{K}s\left(k\right)}{K}$	8	$p_8=\sqrt{\frac{{\sum }_{k=1}^K{f_k}^{4}s\left(k\right)}{{\sum }_{k=1}^K{f_k}^{2}s\left(k\right)}}$
2	$p_2=\frac{{\sum }_{k=1}^K{(s\left(k\right)-p_1)}^2}{K}$	9	$p_9=\frac{{\sum }_{k=1}^K{f_k}^{2}s\left(k\right)}{\sqrt{{\sum }_{k=1}^{K}s\left(k\right){\sum }_{k=1}^K{f_k}^{4}s\left(k\right)}}$
3	$p_3=\frac{{\sum }_{k=1}^K{(s\left(k\right)-p_1)}^3}{K{\left(\sqrt{p_2}\right)}^3}$	10	$p_{10}=\frac{p_6}{p_5}$
4	$p_4=\frac{{\sum }_{k=1}^K{(s\left(k\right)-p_1)}^4}{K{p}_2^2}$	11	$p_{11}=\frac{{\sum }_{k=1}^K{(f_k-p_5)}^{3}s\left(k\right)}{K{p}_6^2}$
5	$p_5=\frac{{\sum }_{k=1}^K{f}_{k}s\left(k\right)}{{\sum }_{k=1}^{K}s\left(k\right)}$	12	$p_{12}=\frac{{\sum }_{k=1}^K{(f_k-p_5)}^{4}s\left(k\right)}{K{p}_6^4}$
6	$p_6=\sqrt{\frac{{\sum }_{k=1}^K{(f_k-p_5)}^{2}s\left(k\right)}{K}}$	13	$p_{13}=\frac{{\sum }_{k=1}^K{(f_k-p_5)}^{0.5}s\left(k\right)}{K{p}_6}$
7	$p_7=\sqrt{\frac{{\sum }_{k=1}^K{f_k}^{2}s\left(k\right)}{{\sum }_{k=1}^{K}s\left(k\right)}}$

. These parameters were chosen in accordance with the findings of Chen’s study [52]. In this study, feature extraction was used as a signal preprocessing method.

2.5. Evaluation of Model Performances

The confusion matrix serves as a robust tool for assessing the classification performance, enabling the quantification of overlaps in categorization [53]. This numerical framework is pivotal in analyzing error distributions within classification tasks [54]. It is utilized extensively in various machine learning contexts, including neural networks, decision trees, Bayesian methods, and support vector machines [55]. Ahmad et al. applied modularized induction techniques using the confusion matrix for pretrained CNNs [56]. This matrix can be used to determine the classification accuracy as follows:

$$Accuracy=\frac{A+D}{A+B+C+D}$$

(8)

where $A$ signifies the ratio of correct negative predictions, $B$ stands for the ratio of incorrect positive predictions, $C$ denotes the ratio of precise negative predictions, and $D$ indicates the ratio of precise positive predictions.

The Area Under the Receiver Operating Characteristic Curve (AUC), a crucial assessment tool in classification tasks, is also introduced. The AUC quantifies the model’s proficiency in distinguishing between classes by measuring its capacity to assign higher probabilities to positive instances. A higher AUC score, closer to 1, signifies superior discrimination and overall model performance. This inclusion broadens the scope of our evaluation, offering deeper insights into the model’s classification ability and its adaptability to class imbalances, ensuring a more comprehensive analysis of its effectiveness and reliability.

3. Datasets Generated from Lamb Wave Approaches

3.1. Model Construction from COMSOL

Figure 3 depicts the COMSOL model with soil-embedded pipelines with notch and welding flaws. The pipeline has a 2000 mm length, an exterior diameter ($D_{out}$) of 76 mm, and an interior diameter of 68 mm ($D_{in}$). Figure 4 shows four kinds of welding defects, including lack of penetration, lack of fusion, undercut, and cracks. There are four kinds of defects discussed in the COMSOL model, including defect 1 (welding defects of lack of fusion with 10% severity and notch damage at ${5\times D}_{out}$ placement), defect 2 (welding defects of cracks with 10% severity and notch damage at ${5\times D}_{out}$ placement), defect 3 (welding defects of undercut with 10% severity and notch damage at ${5\times D}_{out}$ placement), and defect 4 (welding defects of lack of penetration with 10% severity and notch damage at ${5\times D}_{out}$ placement).

Table 4. Four kinds of pipeline defects.

Defects	Description
Defect 1	$\mathrm{Welding} \mathrm{defects} \mathrm{of} \mathrm{lack} \mathrm{of} \mathrm{fusion} \mathrm{with} 10\% \mathrm{severity} \mathrm{and} \mathrm{notch} \mathrm{damage} \mathrm{at} {5\times D}_{out}$ placement
Defect 2	$\mathrm{Welding} \mathrm{defects} \mathrm{of} \mathrm{cracks} \mathrm{with} 10\% \mathrm{severity} \mathrm{and} \mathrm{notch} \mathrm{damage} \mathrm{at} {5\times D}_{out}$ placement
Defect 3	$\mathrm{Welding} \mathrm{defects} \mathrm{of} \mathrm{undercut} \mathrm{with} 10\% \mathrm{severity} \mathrm{and} \mathrm{notch} \mathrm{damage} \mathrm{at} {5\times D}_{out}$ placement
Defect 4	$\mathrm{Welding} \mathrm{defects} \mathrm{of} \mathrm{lack} \mathrm{of} \mathrm{penetration} \mathrm{with} 10\% \mathrm{severity} \mathrm{and} \mathrm{notch} \mathrm{damage} \mathrm{at} {5\times D}_{out}$ placement

presents a comprehensive depiction of the dataset’s particulars.

Table 5. Experimental array for computational modeling.

Case Design	Label	Damage Location	Damage Size	Damage Depth (mm)	Welding Defects Type	Severity of Welding Defects	Noise Interference
Base	State #1	/	/	/	/	/
Case 1: variance due to the variety of welding defects	State #2	${5\times D}_{out}$	${0.1\times D}_{out}$	4	Defect 1	10%
	State #3	${5\times D}_{out}$	${0.1\times D}_{out}$	4	Defect 2	10%	From 3 dB to 15 dB
	State #4	${5\times D}_{out}$	${0.1\times D}_{out}$	4	Defect 3	10%
	State #5	${5\times D}_{out}$	${0.1\times D}_{out}$	4	Defect 4	10%
Case 2: variance due to severity of welding defects	State #6	${5\times D}_{out}$	${0.1\times D}_{out}$	4	Defect 4	1%	From 3 dB to 15 dB
	State #7	${5\times D}_{out}$	${0.1\times D}_{out}$	4	Defect 4	5%
	State #8	${5\times D}_{out}$	${0.1\times D}_{out}$	4	Defect 4	10%

shows the experimental array for computational modeling. The dataset used in this study was divided into training, validation, and test sets. Specifically, 70% of the data was allocated for training, 15% for validation, and 15% for testing. The evaluations included previously unseen data in the validation and test sets, ensuring the models were assessed on data they had not been exposed to during training. This approach helped in gauging the models’ generalization capabilities to new, unseen data.

The welding substance utilized is Ti-6Al-4V, featuring a Poisson ratio of 0.40, a density measuring 4453 kg/m³, a Young’s modulus reaching 125.8 GPa, and a shear modulus amounting to 40 GPa. Similarly, the embedding material, a firm soil, showcases a density of 2600 kg/m³, a Young’s modulus registering at 20 MPa, and a Poisson ratio equivalent to 0.2. Four symmetrical receivers were circumferentially positioned at the end of pipelines (15 mm from the left side), with the excitation nodes situated at the left side. A Hanning window-operated 5-cycle sinusoidal signal at 100 kHz was employed for the excitation. The wave moved at a rate of about 5241 m/s. Figure 5 shows the excited guided wave. The time-domain waveforms corresponding to defect 1, defect 2, defect 3, and defect 4 are shown in Figure 6.

3.2. Signals with Noise Interference

Signals collected from construction sites are contaminated with serious noise, which is totally different from signals collected from a simulation environment. A simulation environment is an ideal environment, which has no noise interference on signals. To better understand the effectiveness of the method used in this study, different levels of noise would be added to simulation signals to better express the actual situation in real life. Taking defect 4 with 10% severity as an example, Figure 7 depicts the signals affected by varying levels of noise interference. The original signal is collected from a simulation environment, and it is an ideal signal without noise interference. As shown in Figure 7, when SNR is 3 dB, the signal is hard to differentiate from the noise. The signal becomes much clearer and stronger as the noise levels decreased. At SNR = 15 dB, the adulterated signal closely resembles the original signal.

4. Results and Discussion

4.1. Impacts of Features on the Performance of the Deep Learning Models (Case 1)

To evaluate the influence of feature selection on the training performance of deep learning models, time-domain features, frequency-domain features and time- and frequency-domain features were used as input to train CNN, LSTM, and CNN-LSTM models. The training data is from Case 1. Accuracy was used as the evaluation index, as shown in Figure 8. Firstly, it is clear the classification performance keeps rising as the noise levels reduce for three deep learning models (CNN, LSTM, and CNN-LSTM models). When the noise levels reduce to 15 dB, the classification performance is up to 100%. It is also demonstrated in Figure 7, when the noise level is 15 dB, the signal is clear without noise interference, which is why the accuracy is the highest. It means eliminating noise interference can improve the training performance of deep learning models.

In addition, the accuracy of three models with time- and frequency-domain features is much higher than models with time-domain features and frequency-domain features when the noise level is from 3 dB to 15 dB, as shown in Figure 8. For example, when SNR is 3 dB, the accuracies of three models with time- and frequency-domain features improve by 38–45% compared to models with time-domain features and improve by 26–39% compared to models with frequency-domain features. The results demonstrated time- and frequency-domain features have much more comprehensive information about signals than time-domain features and frequency-domain features and they can help increase the classification performance of deep learning models. However, when SNR is 15 dB, the accuracies of three models have the same value (100%), which has no relationship with feature types. It reflects that when noise interference is eliminated, three types of features can express signals’ information uniquely and can achieve the best classification performance.

Comparison of the Performance of Three Deep Learning Models (Case 1)

To demonstrate the superiority of the CNN-LSTM model, CNN and LSTM models were selected and trained with the same inputs. Accuracy and the confusion matrix were used as evaluation indices, as shown in Figure 8 and Figure 9. Figure 8 shows the accuracies of the CNN-LSTM model are always higher than those of the CNN and LSTM models. When the SNR level is equal to 3 dB, the CNN-LSTM model has an increase in accuracy by 19% compared to the LSTM model and an increase of 22% compared to the CNN model when time-domain and frequency-domain features were used as input. Meanwhile, the accuracies of the CNN-LSTM model are improved by 26% compared to the LSTM model and by 34% when time- and frequency-domain features were used as input. The results prove that time- and frequency-domain features can express more signals information than time-domain features and frequency-domain features, and that the CNN-LSTM model has better prediction performance than the CNN and LSTM models. The CNN-LSTM model can extract temporal features better compared to the CNN model, and the input data is first processed and compressed by the CNN and then input into the LSTM model, which can improve the processing efficiency of the LSTM. That is the reason why the CNN-LSTM model can achieve better performance than the CNN and LSTM models.

Furthermore, when SNR is equal to 12 dB, the accuracies of the CNN-LSTM model has a 3% improvement over the LSTM model and a 5% improvement over the CNN model. The results demonstrate the reduction of noise interference can narrow the difference between different feature inputs.

4.2. Classification Performance of CNN, LSTM and CNN-LSTM Models (Case 2)

To verify the robustness of the CNN-LSTM model, the CNN, and LSTM models were trained, and the training data came from case 2. Time- and frequency-domain features were used as the models’ input, as it had been testified in Case 1 that time- and frequency-domain features included the most comprehensive information of signals. Accuracy and the confusion matrix were used as evaluation indices, as shown in

Table 6. Performance of three models on different levels of noise interference.

Input	SNR (dB)	Accuracy
		CNN	LSTM	CNN-LSTM
Time- and frequency-domain features	NAN	100.0%	100.0%	100.0%
Time- and frequency-domain features	3	32.8%	34.2%	36.7%
	6	51.0%	53.0%	55.0%
	9	71.0%	73.0%	75.0%
	12	89.3%	90.5%	93.3%
	15	100.0%	100.0%	100.0%

and Figure 10. Table 6 shows the same trend as in Case 1, the CNN-LSTM model has the best classification performance, the LSTM model is the second, and the CNN model is the worst. For instance, when SNR is 3 dB, the CNN-LSTM model has a 2.5% improvement over the LSTM model and a 3.7% improvement over the CNN model. The difference among the three models is decreasing as the noise interference is reduced. When the SNR level reaches 15 dB, there is no difference among the three models; all of them can achieve 100% accuracy. The results demonstrate the effectiveness of the proposed CNN-LSTM model.

For the CNN-LSTM model, when SNR is equal to 3 dB, its accuracy is 36.7%, where the misclassification is mainly on the labels of base, 5% and 10%, with 86.7%, 80.0%, and 86.7% misclassification rates, respectively. When SNR is equal to 12 dB, the classification performance is 93.3%, where the misclassification is mainly on the labels of 5% and 10%, with 6.7% and 20.0% misclassification rates, respectively. When SNR is equal to 15 dB, there is no misclassification. In our experimental setup, all pipelines have multiple faults, both with damage and welding defects. The confusion matrix results show the efficacy of the CNN-LSTM model has no clear relationship with damage categories in this study.

The area under the curve (AUC) values for all three models across various levels of noise interference were utilized to further elucidate our findings, as summarized in

Table 7. The AUC values of three models on different levels of noise interference.

Input	SNR (dB)	Accuracy
		CNN	LSTM	CNN-LSTM
Time- and frequency-domain features	NAN	1.000	1.000	1.000
Time- and frequency-domain features	3	0.330	0.345	0.369
	6	0.515	0.530	0.555
	9	0.710	0.725	0.755
	12	0.896	0.910	0.935
	15	1.000	1.000	1.000

and Figure 11. The AUC values were pivotal for assessing a model’s ability to distinguish between classes, with higher values indicating enhanced discriminatory power. These AUC values reinforced the consistency of the CNN-LSTM model’s performance as they increased with decreasing noise levels. Notably, at an SNR of 15 dB, the AUC reached 1.000. And, at an SNR of 3 dB, the AUC value of 0.369 was lower than the threshold (0.750) suggested by Fan’s research [57], which indicated unacceptable classification accuracy. The increasing AUC values alongside accuracy further validated the robustness of our CNN-LSTM model in handling varying noise levels.

5. Further Discussion of Pipelines under Different Embedment

To further evaluate the robustness of the effectiveness of the CNN-LSTM model, CNN and LSTM models were trained, and more complex models were constructed in COMSOL to produce more training data. The established parameter of the COMSOL model was kept the same as in Case 1, as shown in Figure 3. The only difference was the pipeline embedding materials. The soil embedding materials [34] were changed to soft clay, stiff clay, loose sand, dense sand, and concrete [58,59,60].

Table 8. The properties of embedding materials [34].

Embedding Materials	Young’s Modulus	Poisson	Density (kg/m3)
Soft clay	3.5 MPa	0.42	1.4 × 103
Stiff clay	20 MPa	0.20	2.6 × 103
Loose sand	10.35 MPa	0.30	1.5 × 103
Dense sand	50 MPa	0.40	1.6 × 103
Concrete	32.5 GPa	0.16	25 × 103

shows the properties of the embedding materials.

5.1. Signal Characteristics of the Pipes under Different Embedment Materials

Figure 12 shows the pipeline waveforms with defect 4 with 10% severity under soft clay, stiff clay, loose sand, dense sand, and concrete embedment. It can be seen the signals of soft clay, stiff clay, loose sand, and dense sand have the most obvious defect reflection area and boundary reflection area, while the signals of concrete embedment have no such trend. It was mostly due to the concrete’s substantially higher attenuation of sand and concrete, which results from the directed waves’ higher energy leakage to the embedment and energy loss absorbed by concrete and sand in comparison to soil. Leinov’s research also confirmed this [61].

5.2. Impacts of Embedment Conditions on Classification Performance of Deep Learning Models

As shown in Figure 13, the performance for the cases with embedding soil, soft clay, stiff clay, loose sand, and dense sand is almost the same. It can be also demonstrated in Figure 12, the waveforms of embedding soil, soft clay, stiff clay, loose sand, and dense sand are almost the same. And, the CNN-LSTM model has the best training performance and the highest accuracy in comparison with the CNN and LSTM models under high noise interference for the cases of embedding soil, soft clay, stiff clay, loose sand, and dense sand. For instance, when the noise level rises to 6 dB for the cases with embedding sand, the performance of the CNN-LSTM model improves by 12.2% compared to the CNN model and 17.9% compared to the LSTM model. At the same noise level for the cases with embedding concrete, the CNN-LSTM model outperforms the CNN and LSTM models by 12.1% and 6.1% higher, respectively. The results show the CNN-LSTM hybrid has the best classification performance.

Furthermore, the accuracies of the three models for the cases with embedding soil, soft clay, stiff clay, loose sand, and dense sand are much higher than the accuracies of the three models for the cases with embedding concrete. When SNR is equal to 12 dB, the performance of the three models for the cases with embedding soil improves by 8–11% compared to that for the cases with embedding concrete, which is consistent with Zhang’s early findings [4].

5.3. Further Discussion about the Applicability to Different Metallic Materials

Our proposed ultrasonic-guided wave testing method has shown promise for the detection of welding defects in pipelines. While the study primarily focuses on a specific material, Ti-6Al-4V, which is commonly used in pipelines, it is essential to consider the broader applicability of this method to a range of metallic materials, including steel, aluminum, and copper, which are frequently employed in various industrial settings.

Advantages of the Proposed Method for Different Metallic Materials: firstly, the method’s non-destructive nature makes it adaptable to a variety of metallic materials without causing damage [62]. Secondly, ultrasonic-guided waves have demonstrated sensitivity to material variations, enabling the detection of defects in different metals. Thirdly, the method’s ability to operate at various frequencies allows for versatility when dealing with different materials, each having its own acoustic characteristics.

However, there are still limitations and challenges with non-ferrous metals. Non-ferrous metals, such as aluminum and copper, have distinct acoustic properties that may require specific calibration and signal processing techniques. Furthermore, materials with high electrical conductivity, like copper, can affect the propagation of ultrasonic waves. Mitigating this influence is an ongoing challenge. Last but not least, non-ferrous metals may exhibit higher signal attenuation compared to ferrous materials, impacting the range and quality of defect detection.

As a result, to broaden the scope of our method’s applicability, further research is needed to investigate and address the specific challenges associated with non-ferrous metals. This includes the development of material-specific calibration techniques and signal processing algorithms to enhance the accuracy and reliability of defect detection.

6. Conclusions

This study demonstrated the effectiveness of the developed CNN-LSTM hybrid model for damage detection. The training data was collected from different cases based on COMSOL models. Different types of features were used as the input to testify the CNN-LSTM model. Different levels of noise interference were used to evaluate the robustness of the CNN-LSTM model. The following conclusions can be drawn:

(a)

Time- and frequency-domain features have the most comprehensive information about signals. In this study, for most of the cases (noise levels from 3 to 15 dB), the accuracies of the three models (CNN, LSTM and CNN-LSTM models) with time- and frequency-domain features are much higher than the three models’ time-domain and frequency-domain features. It means time-frequency features have more signal information than time difference features.

(b)

When the noise interference can be ignored (e.g., 15 dB), three types of features, including time-domain features, frequency-domain features, and time- and frequency-domain features, can be used to express signals’ information and can achieve the best classification performance.

(c)

The CNN-LSTM hybrid model has a better performance for automated damage detection than the CNN and LSTM models, because the hybrid model can make up the shortcomings of CNN and combine the advantages of LSTM to better process the time series signal.

(d)

Embedding materials could impact signal processing, and results reveal variances in different types of soil or sand did not affect the accuracy of the deep learning approaches significantly. However, when concrete is used as an embedding material, all deep models (CNN, LSTM, and CNN-LSTM models) have much lower classification, particularly with an increase in noise interference.