Penetration State Identification of Aluminum Alloy Cold Metal Transfer Based on Arc Sound Signals Using Multi-Spectrogram Fusion Inception Convolutional Neural Network
Abstract
:1. Introduction
2. The Proposed Method
2.1. Arc Sound Feature Extraction
- Pre-emphasis
- 2.
- Short-time Fourier Transform (STFT)
- 3.
- Filtering and spectrogram fusion
2.2. Inception CNN Classification Model
3. Experiment
3.1. Welding Experiments
3.2. Building and Dividing the Dataset
3.3. Welding Defect Recognition
4. Results and Discussion
4.1. Arc Sound Frequency Analysis
4.2. Effect of Number of Filters on Recognition Accuracy
4.3. Multilingual Spectrogram Fusion Analysis
4.4. Comparison with Other Classification Methods
5. Conclusions
- (1)
- Welding arc acoustic signals contain a wealth of information related to the state of weld penetration.
- (2)
- The dataset generated by the filter bank, containing 60 filters as the input to the model, yields a maximum recognition accuracy of 97.7%. Adding more filters does not improve the recognition accuracy of the model.
- (3)
- The multi-spectrogram fusion method for arc sound feature extraction increases the recognition accuracy of the weld penetration state to 97.7% (MGB-60), which is 0.56%, 0.41% and 0.75% higher than the recognition accuracy of the Mel spectrogram (Mel-60, 97.14%), GT spectrogram (GT-60, 97.29%) and Bark spectrogram (Bark-60, 96.95%), respectively.
- (4)
- The recognition accuracy of Inception CNN, proposed in this paper, is 0.93% better than that of GoogleNet, and the recognition results are more stable. It also has better recognition results and greater stability compared to AlexNet and ResNet.
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Selvi, S.; Vishvaksenan, A.; Rajasekar, E. Cold metal transfer (CMT) technology—An overview. Def. Technol. 2018, 14, 28–44. [Google Scholar] [CrossRef]
- Pickin, C.G.; Williams, S.W.; Lunt, M. Characterisation of the cold Metal transfert (CMT) process and its application for low dilution cladding. J. Mater. Process. Technol. 2011, 211, 496–502. [Google Scholar] [CrossRef]
- Furukawa, K. New CMT arc welding process—Welding of steel to aluminium dissimilar metals and welding of super-thin aluminium sheets. Weld. Int. 2006, 20, 440–445. [Google Scholar] [CrossRef]
- González, J.; Rodríguez, I.; Prado-Cerqueira, J.L.; Diéguez, J.L.; Pereira, A. Additive manufacturing with GMAW welding and CMT technology. Procedia Manuf. 2017, 13, 840–847. [Google Scholar] [CrossRef]
- Derekar, K.S.; Addison, A.; Joshi, S.S.; Zhang, X.; Lawrence, J.; Xu, L.; Melton, G.; Griffiths, D. Effect of pulsed metal inert gas (pulsed-MIG) and cold metal transfer (CMT) techniques on hydrogen dissolution in wire arc additive manufacturing (WAAM) of aluminium. Int. J. Adv. Manuf. Technol. 2020, 107, 311–331. [Google Scholar] [CrossRef]
- Ji, T.; Nor, N.M. Deep Learning-Empowered Digital Twin Using Acoustic Signal for Welding Quality Inspection. Sensors 2023, 23, 2643. [Google Scholar] [CrossRef] [PubMed]
- Wang, Q.; Gao, Y.; Huang, L.; Gong, Y.; Xiao, J. Weld bead penetration state recognition in GMAW process based on a central auditory perception model. Measurement 2019, 147, 106901. [Google Scholar] [CrossRef]
- Wu, J.; Shi, J.; Gao, Y.; Gai, S. Penetration Recognition in GTAW Welding Based on Time and Spectrum Images of Arc Sound Using Deep Learning Method. Metals 2022, 12, 1549. [Google Scholar] [CrossRef]
- Gao, Y.; Wang, Q.; Xiao, J.; Zhang, H. Penetration state identification of lap joints in gas tungsten arc welding process based on two channel arc sounds. J. Mater. Process. Technol. 2020, 285, 116762. [Google Scholar] [CrossRef]
- Cui, Y.; Shi, Y.; Hong, X. Analysis of the frequency features of arc voltage and its application to the recognition of welding penetration in K-TIG welding. J. Manuf. Process. 2019, 46, 225–233. [Google Scholar] [CrossRef]
- Cui, Y.; Shi, Y.; Zhu, T.; Cui, S. Welding penetration recognition based on arc sound and electrical signals in K-TIG welding. Meas. Measurement 2020, 163, 107966. [Google Scholar] [CrossRef]
- Lu, J.; Xie, H.; Chen, X.; Han, J.; Bai, L.; Zhao, Z. Online welding quality diagnosis based on molten pool behavior prediction. Opt. Laser Technol. 2020, 126, 106126. [Google Scholar] [CrossRef]
- Lu, J.; He, H.; Shi, Y.; Bai, L.; Zhao, Z.; Han, J. Quantitative prediction for weld reinforcement in arc welding additive manufacturing based on molten pool image and deep residual network. Addit. Manuf. 2021, 41, 101980. [Google Scholar] [CrossRef]
- Lv, N.; Xu, Y.; Li, S.; Yu, X.; Chen, S. Automated control of welding penetration based on audio sensing technology. J. Mater. Process. Technol. 2017, 250, 81–98. [Google Scholar] [CrossRef]
- Zhao, Z.; Lv, N.; Xiao, R.; Liu, Q.; Chen, S. Recognition of penetration states based on arc sound of interest using VGG-SE network during pulsed GTAW process. J. Manuf. Process. 2023, 87, 81–96. [Google Scholar] [CrossRef]
- Ren, W.; Wen, G.; Xu, B.; Zhang, Z. A Novel Convolutional Neural Network Based on Time-Frequency Spectrogram of Arc Sound and Its Application on GTAW Penetration Classification. IEEE Trans. Ind. Inform. 2021, 17, 809–819. [Google Scholar] [CrossRef]
- Liu, L.; Chen, H.; Chen, S. Quality analysis of CMT lap welding based on welding electronic parameters and welding sound. J. Manuf. Process. 2022, 74, 1–13. [Google Scholar] [CrossRef]
- Yang, G.; Guan, K.; Zou, L.; Sun, Y.; Yang, X. Weld Defect Detection of a CMT Arc-Welded Aluminum Alloy Sheet Based on Arc Sound Signal Processing. Appl. Sci. 2023, 13, 5152. [Google Scholar] [CrossRef]
- Salvati, D.; Drioli, C.; Foresti, G.L. A late fusion deep neural network for robust speaker identification using raw waveforms and gammatone cepstral coefficients. Expert Syst. Appl. 2023, 222, 119750. [Google Scholar] [CrossRef]
- Liang, R.; Kong, F.; Xie, Y.; Tang, G.; Cheng, J. Real-Time Speech Enhancement Algorithm Based on Attention LSTM. IEEE Access 2020, 8, 48464–48476. [Google Scholar] [CrossRef]
- Ancilin, J.; Milton, A. Improved speech emotion recognition with Mel frequency magnitude coefficient. Appl. Acoust. 2021, 179, 108046. [Google Scholar] [CrossRef]
- Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A.; Liu, W.; et al. Going deeper with convolutions. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015; pp. 1–9. [Google Scholar] [CrossRef]
- Li, J.; Song, G.; Zhang, M. Occluded offline handwritten Chinese character recognition using deep convolutional generative adversarial network and improved GoogLeNet. Neural Comput. Appl. 2020, 32, 4805–4819. [Google Scholar] [CrossRef]
- Mondal, S.; Barman, A.D. Speech activity detection using time-frequency auditory spectral pattern. Appl. Acoust. 2020, 167, 107403. [Google Scholar] [CrossRef]
- Liu, S.; Li, R.; Li, Q.; Zhao, J. Porn streamer audio recognition based on deep learning and random Forest. Appl. Intell. 2023, 53, 18857–18867. [Google Scholar] [CrossRef]
- Malayath, N.; Hermansky, H. Data-driven spectral basis functions for automatic speech recognition. Speech Commun. 2003, 40, 449–466. [Google Scholar] [CrossRef]
- Toyoshima, I.; Okada, Y.; Ishimaru, M.; Uchiyama, R.; Tada, M. Multi-Input Speech Emotion Recognition Model Using Mel Spectrogram and GeMAPS. Sensors 2023, 23, 1743. [Google Scholar] [CrossRef] [PubMed]
- Tao, H.; Wang, P.; Chen, Y.; Stojanovic, V.; Yang, H. An unsupervised fault diagnosis method for rolling bearing using STFT and generative neural networks. J. Frankl. Inst. 2020, 357, 7286–7307. [Google Scholar] [CrossRef]
- Nagarajan, S.; Nettimi, S.S.S.; Kumar, L.S.; Nath, M.K.; Kanhe, A. Speech emotion recognition using cepstral features extracted with novel triangular filter banks based on bark and ERB frequency scales. Digit. Signal Process. 2020, 104, 102763. [Google Scholar] [CrossRef]
- Mondal, S.; Barman, A.D. Human auditory model based real-time smart home acoustic event monitoring. Multimed. Tools Appl. 2022, 81, 887–906. [Google Scholar] [CrossRef]
- Hermansky, H. Perceptual linear predictive (PLP) analysis of speech. J. Acoust. Soc. Am. 1990, 87, 1738–1752. [Google Scholar] [CrossRef]
- Zhang, Z.; Wen, G.; Chen, S. Weld image deep learning-based on-line defects detection using convolutional neural networks for Al alloy in robotic arc welding. J. Manuf. Process. 2019, 45, 208–216. [Google Scholar] [CrossRef]
- Wu, D.; Huang, Y.; Zhang, P.; Yu, Z.; Chen, H.; Chen, S. Visual-Acoustic Penetration Recognition in Variable Polarity Plasma Arc Welding Process Using Hybrid Deep Learning Approach. IEEE Access 2020, 8, 120417–120428. [Google Scholar] [CrossRef]
- Ma, G.; Yu, L.; Yuan, H.; Xiao, W.; He, Y. A vision-based method for lap weld defects monitoring of galvanized steel sheets using convolutional neural network. J. Manuf. Process. 2021, 64, 130–139. [Google Scholar] [CrossRef]
- Gaba, S.; Budhiraja, I.; Kumar, V.; Garg, S.; Kaddoum, G.; Hassan, M.M. A federated calibration scheme for convolutional neural networks: Models, applications and challenges. Comput. Commun. 2022, 192, 144–162. [Google Scholar] [CrossRef]
- Zhang, Q.; Zhang, M.; Chen, T.; Sun, Z.; Ma, Y.; Yu, B. Recent advances in convolutional neural network acceleration. Neurocomputing 2019, 323, 37–51. [Google Scholar] [CrossRef]
- Ioffe, S.; Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on Machine Learning, Lille, France, 6–11 July 2015; Volume 1, pp. 448–456. [Google Scholar]
- Reyad, M.; Sarhan, A.M.; Arafa, M. A modified Adam algorithm for deep neural network optimization. Neural Comput. Applic 2023, 35, 17095–17112. [Google Scholar] [CrossRef]
No. | Component | Output Size |
---|---|---|
0 | Input | C × H × W |
1 | BasicConv2D (C, 64, 7, 1, 3) | 64 × H × W |
2 | MaxPool (64, 64, 3, 2, 1) | 64 × ⎡H/2⎤ × ⎡W/2⎤ |
3 | BasicConv2D (64, 64, 1, 1, 0) | 64 × ⎡H/2⎤ × ⎡W/2⎤ |
4 | BasicConv2D (64, 64, 3, 1, 1) | 64 × ⎡H/2⎤ × ⎡W/2⎤ |
5 | MaxPool (64, 192, 3, 2, 1) | 192 × ⎡H/4⎤ × ⎡W/4⎤ |
6 | InceptionModel (192, 64, (96, 128), (16, 32), 32) | 256 × ⎡H/4⎤ × ⎡W/4⎤ |
7 | InceptionModel (256, 128, (128, 192), (32, 128), 64) | 512 × ⎡H/4⎤ × ⎡W/4⎤ |
8 | MaxPool (512, 3, 2, 1) | 512 × ⎡H/8⎤ × ⎡W/8⎤ |
9 | InceptionModel (512, 256, (160, 320), (32, 128), 128) | 832 × ⎡H/8⎤ × ⎡W/8⎤ |
10 | InceptionModel (832, 256, (192, 512), (64, 128), 128) | 1024 × ⎡H/8⎤ × ⎡W/8⎤ |
11 | AdaptiveAvgPool2d (1024, (1, 1)) | 1024 × 1 × 1 |
13 | FullConnection (1024, 3) | 3 |
0 | Input | C × H × W |
No. | Current (A) | Voltage (V) | Welding Speed (m/min) | Wire Feed Speed (m/min) | Gap (mm) | Penetration State |
---|---|---|---|---|---|---|
1 | 78 | 12.6 | 40 | 5.8 | 0 | non-penetration, full penetration |
2 | 78 | 12.6 | 40 | 5.8 | 1 | non-penetration, full penetration |
3 | 95 | 13.5 | 40 | 6.7 | 1 | excessive penetration |
4 | 98 | 13.7 | 35 | 6.9 | 1 | excessive penetration |
5 | 85 | 13 | 40 | 6.2 | 1 | full penetration |
6 | 85 | 13 | 40 | 6.2 | 0 | full penetration |
7 | 78 | 12.6 | 40 | 5.8 | 0 | non-penetration, full penetration |
Dataset | MGB-40 | MGB-50 | MGB-60 | MGB-70 | MGB-80 | MGB-90 | MGB-100 |
Standard Deviation | 0.23 | 0.29 | 0.12 | 0.10 | 0.36 | 0.30 | 0.30 |
Dataset | Bark-60 | GT-60 | Mel-60 | MGB-60 |
Standard Deviation | 0.005 | 0.002 | 0.004 | 0.001 |
Model | AlexNet | Inception CNN | ResNet | GoogleNet |
Standard Deviation | 0.033 | 0.001 | 0.003 | 0.004 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Yang, G.; Guan, K.; Yang, J.; Zou, L.; Yang, X. Penetration State Identification of Aluminum Alloy Cold Metal Transfer Based on Arc Sound Signals Using Multi-Spectrogram Fusion Inception Convolutional Neural Network. Electronics 2023, 12, 4910. https://doi.org/10.3390/electronics12244910
Yang G, Guan K, Yang J, Zou L, Yang X. Penetration State Identification of Aluminum Alloy Cold Metal Transfer Based on Arc Sound Signals Using Multi-Spectrogram Fusion Inception Convolutional Neural Network. Electronics. 2023; 12(24):4910. https://doi.org/10.3390/electronics12244910
Chicago/Turabian StyleYang, Guang, Kainan Guan, Jiarun Yang, Li Zou, and Xinhua Yang. 2023. "Penetration State Identification of Aluminum Alloy Cold Metal Transfer Based on Arc Sound Signals Using Multi-Spectrogram Fusion Inception Convolutional Neural Network" Electronics 12, no. 24: 4910. https://doi.org/10.3390/electronics12244910
APA StyleYang, G., Guan, K., Yang, J., Zou, L., & Yang, X. (2023). Penetration State Identification of Aluminum Alloy Cold Metal Transfer Based on Arc Sound Signals Using Multi-Spectrogram Fusion Inception Convolutional Neural Network. Electronics, 12(24), 4910. https://doi.org/10.3390/electronics12244910