# Real-Time Detection of Weld Defects for Automated Welding Process Base on Deep Neural Network

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Experiments

#### 2.1. Material

#### 2.2. Equipment and Experimental Procedure

_{2}. A ER70S-3 grade welding wire with a diameter of 1.2 mm was used. For a reliable analysis, two replicates were performed at each condition.

## 3. Results and Discussion

#### 3.1. Relationship between Feature Variables and Porosity Using Welding Voltage Signal

_{1}, X

_{2}, X

_{8}, X

_{9}, and X

_{12}have a positive correlation with porosity ratio and were approximately from 0.6 to 0.7. Although these values do not represent extremely strong positive correlations, they certainly show strong correlations. Furthermore, the correlation coefficients of X

_{7}and X

_{11}, which have negative correlations, are −0.666, and −0.468, respectively. Thus, the correlation coefficient of X

_{7}represents a strong correlation, whereas the correlation coefficient of X

_{11}represents a moderately weak correlation. Meanwhile, X

_{3}, X

_{4}, X

_{5}, X

_{6}, and X

_{10}show very weak correlations with the porosity ratio.

_{1}($s\left[V\right]$), X

_{2}($s\left[{V}_{s}\right]$), X

_{7}(${\overline{T}}_{a}$), X

_{8}($N\left[T\right]$), X

_{9}($\mathrm{s}\left[{T}_{s}\right]$), and X

_{12}($\mathrm{s}\left[\mathrm{V}\left({T}_{a}\right)\right]$), having positive or negative correlation coefficient value of 0.6 are closely related to porosity.

#### 3.2. Porosity Detection and Prediction Algorithm based on Deep Learning Techniques

#### 3.2.1. Deep Neural Network (DNN)

#### 3.2.2. Porosity Prediction Model based on Artificial Intelligence Techniques

_{1}($s\left[V\right]$), X

_{2}($s\left[{V}_{s}\right]$), X

_{7}(${\overline{T}}_{a}$), X

_{8}($N\left[T\right]$), X

_{9}($\mathrm{s}\left[{T}_{s}\right]$), and X

_{12}($\mathrm{s}\left[\mathrm{V}\left({T}_{a}\right)\right]$), were selected as input values for the input layer, and one hidden layer consisting of 24 nodes was selected.

#### 3.2.3. Prediction model evaluation for detecting porosity

#### 3.2.4. Porosity Detection and Prediction Systems for Field Application

## 4. Conclusions

- (1)
- The welding current and arc voltage signals generated in the GMAW process were measured in real-time, and the feature variables were extracted through preprocessing. In addition, the correlation between the feature variable and the porosity was analyzed to select the feature variable that is considered to be the defect signal.
- (2)
- An artificial intelligence technique suitable for nonlinear arc welding process prediction was used, and a model was developed to detect and predict porosity by comparing DNN and ANN models.
- (3)
- The predictive performance of ANN model and DNN model was evaluated. The evaluation result shows that the predictive performance of the DNN model is 15.2% higher than the ANN model on average.
- (4)
- An experiment was performed using the developed system to evaluate it for field application. The results indicated that all the pits generated in the welded part were detected.
- (5)
- It has the advantage of easy site application and low initial equipment cost because it is an NDT system that detects and predicts porosity defects in the weld by using only arc welding process signals without additional devices.

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Kodama, S.; Ishida, Y.; Furusako, S.; Saito, M.; Miyazaki, Y.; Nose, T. Arc welding technology for automotive steel sheets. Nippon Steel Tech. Rep.
**2013**, 103, 83–90. [Google Scholar] - Mukai, Y.; Nishimura, A.; Oku, K. CO2 welding of galvanized steel. Weld. Int.
**1990**, 4, 123–127. [Google Scholar] [CrossRef] - Uchihara, M. Joining technologies for automotive steel sheets. Weld. Int.
**2011**, 25, 249–259. [Google Scholar] [CrossRef] - Shin, S.; Rhee, S. Porosity characteristics and effect on tensile shear strength of high-strength galvanized steel sheets after the gas metal arc welding process. Metals
**2018**, 8, 1077. [Google Scholar] [CrossRef][Green Version] - Kang, M.J.; Rhee, S. The statistical models for estimating the amount of spatter in the short circuit transfer mode of GMAW. Weld. J.
**2001**, 80, 1–8. [Google Scholar] - Kang, M.J.; Rhee, S. A study on the development of the arc stability index using multiple regression analysis in the short-circuit transfer region of gas metal arc welding. Proc. Inst. Mech. Eng. Part B: J. Eng. Manuf.
**2001**, 215, 195–205. [Google Scholar] [CrossRef] - Kang, B.Y.; Lee, C.H.; Kim, H.J.; Chang, H.S. Relationship between spatter generation and waveform factors in transitional condition of CO2 welding. J. Weld. Join.
**1998**, 16, 39–46. [Google Scholar] - Kang, S.K.; Moon, H.S.; Na, S.J. A study on determining arc stability using weight of spatter. J. Weld. Join.
**1997**, 15, 41–48. [Google Scholar] - Wu, C.; Gao, J.Q.; Hu, J.K. Real-time sensing and monitoring in robotic gas metal arc welding. Measure Sci. Technol.
**2007**, 18, 303–310. [Google Scholar] [CrossRef] - Zhang, Z.; Chen, X.; Chen, H.; Zhong, J.; Chen, S. Online welding quality monitoring based on feature extraction of arc voltage signal. Int. J. Adv. Manuf. Technol.
**2014**, 70, 1661–1671. [Google Scholar] [CrossRef] - Na, S.J.; Moon, H.S. Signal processing algorithm for analysis of welding phenomena. J. Weld. Join.
**1996**, 14, 24–32. [Google Scholar] - Sin, J.S.; Kim, J.W.; Na, S.J. A study on monitoring of welding waveforms in gas metal arc welding. J. Weld. Join.
**1991**, 9, 34–40. [Google Scholar] - Quinn, T.P.; Smith, C.; McCowan, C.N.; Blachowiak, E.; Madigan, R.B. Arc sensing for defects in constant voltage gas metal arc welding. Weld. J.
**1999**, 78, 59–73. [Google Scholar] - Adolfsson, S.; Bahrami, A.; Bolmsjo, G.; Claesson, I. On-line quality monitoring in short-circuit gas metal arc welding. Weld. J.
**1999**, 78, 59–73. [Google Scholar] - Chu, Y.X.; Hu, S.J.; Hou, W.K.; Wang, P.C.; Marin, S.P. Signature analysis for quality monitoring in short-circuit GMAW. Weld. J.
**2004**, 83, 336–343. [Google Scholar] - Shin, S.; Kim, M.S.; Rhee, S. Prediction of weld porosity (pit) in gas metal arc welds. Int. J. Adv. Manuf. Technol.
**2019**, 104, 1109–1120. [Google Scholar] [CrossRef] - Suyama, F.M.; Delgado, M.R.; da Silva, R.D.; Centeno, T.M. Deep neural networks based approach for welded joint detection of oil pipelines in radiographic images with Double Wall Double Image exposure. NDT E Int.
**2019**, 105, 46–55. [Google Scholar] [CrossRef] - Bacioiu, D.; Melton, G.; Papaelias, M.; Shaw, R. Automated defect classification of SS304 TIG welding process using visible spectrum camera and machine learning. NDT E Int.
**2019**, 107, 102139. [Google Scholar] [CrossRef] - Chen, Y.; Chen, B.; Yao, Y.; Tan, C.; Feng, J. A spectroscopic method based on support vector machine and artificial neural network for fiber laser welding defects detection and classification. NDT E Int.
**2019**, 108, 102176. [Google Scholar] [CrossRef] - Kingma, D.P.; Lei Ba, J. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, USA, 7–9 May 2015; pp. 1–15. [Google Scholar]

**Figure 3.**Welding signal, bead appearance, and matching of x-ray images. (

**a**) and (

**b**)-Welding speed: 600 mm/min, WFR: 3 m/min, Gap: 0 mm.

**Figure 4.**Welding signal, bead appearance, and matching of x-ray images. (

**a**) and (

**b**)-Welding speed: 600 mm/min, WFR: 3 m/min, Gap: 0.5 mm.

**Figure 5.**Comparison of voltage waveform with or without porosity occurrence. (

**a**) no porosity signal and (

**b**) porosity signal.

**Figure 6.**Feature variables of arc voltage waveform for short-circuit transfer mode in the 0.1 s region.

**Figure 15.**Matching of X-ray images with weld appearance according to voltage signals using test data. (

**a**) test data 1 and (

**b**) test data 2.

**Figure 16.**Training results for loss function and accuracy. (

**a**) Loss function and accuracy of train data set 1. (

**b**) Loss function and accuracy of train data set 2.

**Figure 17.**Test results of ANN and DNN models. (

**a**) Results of the test data set 1; (

**b**) Results of the test data set 2.

Base Metal | Chemical Composition (wt. %) | Mechanical Properties | |||||||
---|---|---|---|---|---|---|---|---|---|

GA 590 | C | Si | Mn | P | S | Fe | YS (MPa) | TS (MPa) | EL (%) |

0.0817 | 0.136 | 1.440 | 0.013 | 0.002 | Bal. | 583 | 629 | 25 |

Welding Conditions | |
---|---|

Gap (mm) | 0, 0.5 |

CTWD (mm) | 15 |

WFR (m/min) | 3 |

Welding speed (mm/min) | 600 |

Shielding gas | Ar (90%)-CO_{2} (10%) |

Feature Variable | Description | Symbol |
---|---|---|

X_{1} | Standard deviation of voltage | $s\left[V\right]$ |

X_{2} | Standard deviation of instantaneous short-circuit voltage | $s\left[{V}_{s}\right]$ |

X_{3} | Standard deviation of short-circuit peak voltage | $s\left[{V}_{p}\right]$ |

X_{4} | Average voltage during short-circuit time | $\overline{V}\left({T}_{s}\right)$ |

X_{5} | Average voltage during arc time | $\overline{V}\left({T}_{a}\right)$ |

X_{6} | Average short-circuit time | ${\overline{T}}_{s}$ |

X_{7} | Average arc time | ${\overline{T}}_{a}$ |

X_{8} | Number of short-circuit periods | $N\left[T\right]$ |

X_{9} | Standard deviation of short-circuit time | $\mathrm{s}\left[{T}_{s}\right]$ |

X_{10} | Standard deviation of arc time | $\mathrm{s}\left[{T}_{a}\right]$ |

X_{11} | Standard deviation of voltage during short-circuit time | $\mathrm{s}\left[\mathrm{V}\left({T}_{s}\right)\right]$ |

X_{12} | Standard deviation of voltage during arc time | $\mathrm{s}\left[\mathrm{V}\left({T}_{a}\right)\right]$ |

Region no. | Signal Section (s) | Total Weld Length (mm) | Bead Width (mm) | Bead Area (mm^{2}) | Porosity Area (mm^{2}) | Porosity Ratio (%) |
---|---|---|---|---|---|---|

1 | 1.40–2.30 | 9.0 | 4.5 | 40.50 | 2.96 | 7.3 |

2 | 2.60–3.50 | 9.0 | 4.5 | 40.50 | 3.56 | 8.8 |

3 | 4.00–4.25 | 9.3 | 4.5 | 41.85 | 3.71 | 8.9 |

4.95–5.28 | ||||||

5.38–5.73 | ||||||

4 | 8.30–8.55 | 8.4 | 4.5 | 37.80 | 5.66 | 15.0 |

9.18–9.32 | ||||||

9.63–9.75 | ||||||

11.68–11.84 | ||||||

12.78–12.95 | ||||||

5 | 2.85–3.20 | 8.8 | 5.2 | 45.76 | 3.16 | 6.9 |

3.90–4.25 | ||||||

5.60–5.78 | ||||||

6 | 6.25–7.15 | 9.0 | 5.2 | 46.80 | 2.55 | 5.4 |

7 | 7.50–8.40 | 9.0 | 5.2 | 46.80 | 5.18 | 11.1 |

8 | 8.63–9.53 | 9.0 | 5.2 | 46.80 | 3.11 | 6.7 |

9 | 15.00–15.30 | 8.9 | 5.2 | 46.28 | 7.05 | 15.2 |

15.65–15.80 | ||||||

16.08–16.20 | ||||||

16.38–16.70 |

Region no. | X_{1}$\mathit{s}\left[\mathit{V}\right]$ | X_{2}$\mathit{s}\left[{\mathit{V}}_{\mathit{s}}\right]$ | X_{3}$\mathit{s}\left[{\mathit{V}}_{\mathit{p}}\right]$ | X_{4}$\overline{\mathit{V}}\left({\mathit{T}}_{\mathit{s}}\right)$ | X_{5}$\overline{\mathit{V}}\left({\mathit{T}}_{\mathit{a}}\right)$ | X_{6}${\overline{\mathit{T}}}_{\mathit{s}}$ | X_{7}${\overline{\mathit{T}}}_{\mathit{a}}$ | X_{8}$\mathit{N}\left[\mathit{T}\right]$ | X_{9}$\mathbf{s}\left[{\mathit{T}}_{\mathit{s}}\right]$ | X_{10}$\mathbf{s}\left[{\mathit{T}}_{\mathit{a}}\right]$ | X_{11}$\mathbf{s}\left[\mathbf{V}\left({\mathit{T}}_{\mathit{s}}\right)\right]$ | X_{12}$\mathbf{s}\left[\mathbf{V}\left({\mathit{T}}_{\mathit{a}}\right)\right]$ | Porosity (%) |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|

1 | 6.8 | 0.48 | 0.35 | 10.39 | 18.69 | 0.0025 | 0.0095 | 7.5 | 0.00033 | 0.0023 | 8.5724 | 5.17 | 7.3 |

2 | 7.2 | 0.65 | 0.48 | 10.26 | 18.77 | 0.0025 | 0.0082 | 8.3 | 0.00036 | 0.0023 | 8.5307 | 5.54 | 8.8 |

3 | 6.8 | 0.30 | 0.36 | 9.85 | 18.87 | 0.0026 | 0.0101 | 6.8 | 0.00026 | 0.0016 | 8.4170 | 4.99 | 8.9 |

4 | 6.6 | 0.60 | 0.53 | 10.36 | 18.74 | 0.0025 | 0.0101 | 6.9 | 0.00034 | 0.0029 | 8.5166 | 5.01 | 15.0 |

5 | 6.6 | 0.12 | 0.52 | 10.33 | 19.08 | 0.0026 | 0.0110 | 6.4 | 0.00029 | 0.0024 | 8.8207 | 4.88 | 6.9 |

6 | 6.5 | 0.13 | 0.68 | 9.89 | 19.09 | 0.0027 | 0.0119 | 5.8 | 0.00022 | 0.0025 | 8.6466 | 4.66 | 5.4 |

7 | 6.3 | 0.22 | 0.44 | 10.01 | 19.11 | 0.0027 | 0.0125 | 5.6 | 0.00034 | 0.0024 | 8.5702 | 4.56 | 11.1 |

8 | 6.3 | 0.08 | 0.56 | 9.81 | 19.19 | 0.0028 | 0.0130 | 5.3 | 0.00027 | 0.0025 | 8.5740 | 4.44 | 6.7 |

9 | 6.2 | 0.22 | 0.38 | 10.14 | 19.01 | 0.0027 | 0.0126 | 5.5 | 0.00043 | 0.0040 | 8.4627 | 4.57 | 15.2 |

10 | 6.3 | 0.09 | 0.25 | 10.41 | 18.99 | 0.0025 | 0.0124 | 5.8 | 0.000152 | 0.0022 | 8.9922 | 4.57 | 0.0 |

11 | 6.2 | 0.06 | 0.40 | 10.58 | 19.15 | 0.0024 | 0.0127 | 5.7 | 0.000171 | 0.0007 | 9.1050 | 4.49 | 0.0 |

12 | 6.3 | 0.09 | 0.35 | 10.31 | 19.10 | 0.0025 | 0.0128 | 5.6 | 0.000175 | 0.0028 | 9.0368 | 4.55 | 0.0 |

13 | 6.1 | 0.09 | 0.39 | 10.14 | 19.09 | 0.0026 | 0.0137 | 5.2 | 0.000187 | 0.0022 | 8.8822 | 4.33 | 0.0 |

14 | 5.4 | 0.06 | 0.55 | 9.98 | 19.00 | 0.0024 | 0.0172 | 4.1 | 0.000211 | 0.0021 | 8.6851 | 3.78 | 0.0 |

15 | 5.9 | 0.07 | 0.51 | 9.46 | 18.98 | 0.0027 | 0.0152 | 4.6 | 0.000232 | 0.0032 | 8.4889 | 4.10 | 0.0 |

16 | 5.9 | 0.08 | 0.34 | 9.57 | 18.94 | 0.0026 | 0.0145 | 4.8 | 0.000196 | 0.0040 | 8.5004 | 4.21 | 0.0 |

17 | 5.8 | 0.08 | 0.49 | 9.58 | 18.96 | 0.0026 | 0.0159 | 4.4 | 0.000147 | 0.0038 | 8.5599 | 4.06 | 0.0 |

18 | 5.7 | 0.14 | 0.39 | 9.71 | 18.88 | 0.0026 | 0.0161 | 4.4 | 0.000222 | 0.0046 | 8.5876 | 4.16 | 0.0 |

No. | Porosity | X_{1} | X_{2} | X_{3} | X_{4} | X_{5} | X_{6} | X_{7} | X_{8} | X_{9} | X_{10} | X_{11} | X_{12} |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|

X_{1} | 0.626 | 1 | - | - | - | - | - | - | - | - | - | - | - |

X_{2} | 0.683 | 0.696 | 1 | - | - | - | - | - | - | - | - | - | - |

X_{3} | 0.053 | −0.146 | 0.045 | 1 | - | - | - | - | - | - | - | - | - |

X_{4} | 0.248 | 0.488 | 0.321 | −0.206 | 1 | - | - | - | - | - | - | - | - |

X_{5} | −0.317 | −0.349 | −0.759 | 0.138 | −0.048 | 1 | - | - | - | - | - | - | - |

X_{6} | 0.116 | −0.149 | −0.187 | 0.264 | −0.684 | 0.269 | 1 | - | - | - | - | - | - |

X_{7} | −0.666 | −0.979 | −0.706 | 0.177 | −0.582 | 0.390 | 0.269 | 1 | - | - | - | - | - |

X_{8} | 0.603 | 0.966 | 0.776 | −0.175 | 0.587 | −0.498 | −0.351 | −0.977 | 1 | - | - | - | - |

X_{9} | 0.733 | 0.369 | 0.623 | 0.274 | 0.128 | −0.337 | 0.343 | −0.370 | 0.363 | 1 | - | - | - |

X_{10} | −0.135 | −0.471 | −0.112 | 0.162 | −0.545 | −0.170 | 0.545 | 0.5 | −0.493 | 0.243 | 1 | - | - |

X_{11} | −0.468 | −0.054 | −0.420 | −0.283 | 0.621 | 0.492 | −0.581 | 0.012 | −0.025 | −0.592 | −0.473 | 1 | - |

X_{12} | 0.631 | 0.98 | 0.767 | −0.167 | 0.563 | −0.463 | −0.244 | −0.977 | 0.986 | 0.421 | −0.405 | −0.051 | 1 |

Activation Function | Equation | Graph |
---|---|---|

Sigmoid | $\mathrm{f}\left(\mathrm{z}\right)=\frac{1}{1+{e}^{-z}}$ | |

ReLU | $\mathrm{f}\left(\mathrm{z}\right)=\{\begin{array}{c}z(z>0)\\ 0\left(z\le 0\right)\end{array}$ |

Model | ANN | DNN | ||||
---|---|---|---|---|---|---|

Structure | Input Layer | Hidden Layer | Output Layer | Input Layer | Hidden Layer | Output Layer |

node | 6 | 1–24 | 1 | 6 | 4–24 | 1 |

Learning rate | 0.01 | |||||

Epoch | 100,000 | |||||

Activation function | Sigmoid | ReLU | ||||

Function of output layer | Softmax | |||||

Optimizer | Gradient descent | Adam optimizer |

Item | Training Dataset 1 | Training Dataset 2 | ||
---|---|---|---|---|

ANN | DNN | ANN | DNN | |

Loss function | 0.126 | 0.002 | 0.131 | 0.018 |

Accuracy | 0.825 | 0.998 | 0.816 | 0.990 |

Item | Test dataset 1 | Test dataset 2 | ||
---|---|---|---|---|

ANN | DNN | ANN | DNN | |

Prediction accuracy | 0.659 | 0.858 | 0.790 | 0.895 |

Welding Conditions | |
---|---|

Gap (mm) | No Gap |

CTWD (mm) | 15 |

WFR (m/min) | 3 |

Welding speed (mm/min) | 600 |

Shielding gas | Ar (90%)-CO_{2} (10%) |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Shin, S.; Jin, C.; Yu, J.; Rhee, S. Real-Time Detection of Weld Defects for Automated Welding Process Base on Deep Neural Network. *Metals* **2020**, *10*, 389.
https://doi.org/10.3390/met10030389

**AMA Style**

Shin S, Jin C, Yu J, Rhee S. Real-Time Detection of Weld Defects for Automated Welding Process Base on Deep Neural Network. *Metals*. 2020; 10(3):389.
https://doi.org/10.3390/met10030389

**Chicago/Turabian Style**

Shin, Seungmin, Chengnan Jin, Jiyoung Yu, and Sehun Rhee. 2020. "Real-Time Detection of Weld Defects for Automated Welding Process Base on Deep Neural Network" *Metals* 10, no. 3: 389.
https://doi.org/10.3390/met10030389