Line-Structured Light Fillet Weld Positioning Method to Overcome Weld Instability Due to High Specular Reflection
Abstract
:1. Introduction
2. System Model Visualization
2.1. Weld Positioning System
2.2. Establishment of the Mathematical Model
2.3. Calibration of Camera
3. Image Processing
3.1. The DeeplabV3+ Neural Network Model, Based on MobileNetV2
3.2. Image Preprocessing
3.3. Centerline Extraction
3.3.1. Skeleton Extraction
3.3.2. Line Extraction
3.4. Extraction of the Key Feature Points
4. Welding Torch Attitude
5. Experiment and Analysis
- Check the surrounding environment of the welding robot, confirm that the barrier is free, and begin the experiment. Install the auxiliary positioning tool shown in Figure 17; the vertex of the positioning tool is the focus position of the laser beam and it is also the most ideal position for the laser to hit the weld. At this time, images of weld are collected to obtain the initial weld feature points. If the acquisition fails at this point, repeat the previous steps.
- Set the moving speed of the welding robot and start the program. During operation, the laser head laser should always move with the welding seam and change the robot’s attitude in real time. When moving to the end of the workpiece, the program ends; save the position coordinates of the feature points during the program, while it is running, for experimental analysis.
6. Conclusions
- (1)
- A fillet weld positioning method is proposed that can effectively overcome the uneven reflection of linear structure light intensity;
- (2)
- A DeeplabV3+ model was used for the first time to identify linear structural light and to filter reflective noise, while MobileNetV2 was used as the backbone network to improve the efficiency of model recognition, compared with the traditional network model;
- (3)
- Based on the closed operation used in the TMM to eliminate the effect of binarization, the expansion, combined with the Zhang–Suen method, was used to extract the skeleton of the discontinuous region, which improved the ability to filter negative pulse noise. Compared with other mainstream skeleton extraction methods, the improved method was proven to be superior to the traditional method;
- (4)
- The attitude correction of a welding torch was studied, the three-dimensional reconstruction mathematical model of the system was established, and the coordinates of the welding seam and welding torch angle were obtained, according to the deduced formula;
- (5)
- The positioning experiment was carried out on stainless steel and highly reflective irregular fillet welds. The experimental results showed that the identified weld feature points were continuous, and the welding torch angle was satisfactory, without collision. The average detection error of the weld feature point P in the Y-axis and Z-axis directions was 0.1709 mm and 0.1768 mm, respectively. The average angle error of the angle between sheets was 0.1836°. The accuracy of the weld identification model meets the requirements and verifies the effectiveness of the system.
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Rout, A.; Deepak, B.; Biswal, B.B. Advances in weld seam tracking techniques for robotic welding: A review, Robot. Comput. Integr. Manuf. 2019, 56, 12–37. [Google Scholar] [CrossRef]
- Lei, T.; Rong, Y.; Wang, H.; Huang, Y.; Li, M. A review of vision-aided robotic welding. Comput. Ind. 2020, 123, 103326. [Google Scholar] [CrossRef]
- Lee, D.; Ku, N.; Kim, T.W.; Kim, J.; Lee, K.Y.; Son, Y.S. Development and application of an intelligent welding robot system for shipbuilding, Robot. Comput. Integr. Manuf. 2011, 27, 377–388. [Google Scholar] [CrossRef]
- Xia, C.; Pan, Z.; Polden, J.; Li, H.; Xu, Y.; Chen, S.; Zhang, Y. A review on wire arc additive manufacturing: Monitoring, control and a framework of automated system. J. Manuf. Syst. 2020, 57, 31–45. [Google Scholar] [CrossRef]
- Yang, L.; Li, E.; Long, T.; Fan, J.; Liang, Z. A high-speed seam extraction method based on the novel structured-light sensor for arc welding robot: A review. IEEE Sens. J. 2018, 18, 8631–8641. [Google Scholar] [CrossRef]
- Wang, B.; Hu, S.J.; Sun, L.; Freiheit, T. Intelligent welding system technologies: State-of-the-art review and perspectives. J. Manuf. Syst. 2020, 56, 373–391. [Google Scholar] [CrossRef]
- Xue, K.; Wang, Z.; Shen, J.; Hu, S.; Zhen, Y.; Liu, J.; Wu, D.; Yang, H. Robotic seam tracking system based on vision sensing and human-machine interaction for multi-pass MAG welding. J. Manuf. Processes 2021, 63, 48–59. [Google Scholar] [CrossRef]
- Maldonado-Ramirez, A.; Rios-Cabrera, R.; Lopez-Juarez, I. A visual path-following learning approach for industrial robots using DRL, Robot. Comput. Integr. Manuf. 2021, 71, 102130. [Google Scholar] [CrossRef]
- Zou, Y.; Wang, Y.; Zhou, W.; Chen, X. Real-time seam tracking control system based on line laser visions. Opt. Laser. Technol. 2018, 103, 182–192. [Google Scholar] [CrossRef]
- Bračun, D.; Sluga, A. Stereo vision based measuring system for online welding path inspection. J. Mater. Processing Technol. 2015, 223, 328–336. [Google Scholar] [CrossRef]
- Ding, Y.; Huang, W.; Kovacevic, R. An on-line shape-matching weld seam tracking system. Robot. Comput. Integr. Manuf. 2016, 42, 103–112. [Google Scholar] [CrossRef]
- Li, X.; Li, X.; Khyam, M.O.; Ge, S.S. Robust welding seam tracking and recognition. IEEE Sens. J. 2017, 17, 5609–5617. [Google Scholar] [CrossRef]
- Xue, B.; Chang, B.; Peng, G.; Gao, Y.; Tian, Z.; Du, D.; Wang, G. A vision based detection method for narrow butt joints and a robotic seam tracking system. Sensors 2019, 19, 1144. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Dinham, M.; Fang, G. Autonomous weld seam identification and localisation using eye-in-hand stereo vision for robotic arc welding. Robot. Comput.-Integr. Manuf. 2013, 29, 288–301. [Google Scholar] [CrossRef]
- Ma, H.; Wei, S.; Sheng, Z.; Lin, T.; Chen, S. Robot welding seam tracking method based on passive vision for thin plate closed-gap butt welding. Int. J. Adv. Manuf. Technol. 2010, 48, 945–953. [Google Scholar] [CrossRef]
- HShah, N.M.; Sulaiman, M.; Shukor, A.Z.; Kamis, Z.; Rahman, A.A. Butt welding joints recognition and location identification by using local thresholding. Robot. Comput.-Integr. Manuf. 2018, 51, 181–188. [Google Scholar] [CrossRef]
- Shao, W.; Liu, X.; Wu, Z. A robust weld seam detection method based on particle filter for laser welding by using a passive vision sensor. Int. J. Adv. Manuf. Technol. 2019, 104, 2971–2980. [Google Scholar] [CrossRef]
- Dinham, M.; Fang, G. Detection of fillet weld joints using an adaptive line growing algorithm for robotic arc welding. Robot. Comput.-Integr. Manuf. 2014, 30, 229–243. [Google Scholar] [CrossRef]
- Du, R.; Xu, Y.; Hou, Z.; Shu, J.; Chen, S. Strong noise image processing for vision-based seam tracking in robotic gas metal arc welding. Int. J. Adv. Manuf. Technol. 2019, 101, 2135–2149. [Google Scholar] [CrossRef]
- Yu, W.; Li, Y.; Yang, H.; Qian, B. The Centerline Extraction Algorithm of Weld Line Structured Light Stripe Based on Pyramid Scene Parsing Network. IEEE Access 2021, 20, 105144–105152. [Google Scholar] [CrossRef]
- Zhao, Z.; Luo, J.; Wang, Y.; Bai, L.; Han, J. Additive seam tracking technology based on laser vision. Int. J. Adv. Manuf. Technol. 2021, 116, 197–211. [Google Scholar] [CrossRef]
- Chen, S.; Liu, J.; Chen, B.; Suo, X. Universal fillet weld joint recognition and positioning for robot welding using structured light. Robot. Comput.-Integr. Manuf. 2022, 74, 102279. [Google Scholar] [CrossRef]
- Chen, L.C.; Zhu, Y.; Papandreou, G.; Schroff, F.; Adam, H. Encoder-decoder with Atrous separable convolution for semantic image segmentation. In Proceedings of the 15th European Conference on Computer Vision, Munich, Germany, 8–14 September 2018; pp. 833–851. [Google Scholar]
- Sandler, M.; Howard, A.; Zhu, M.; Zhmoginov, A.; Chen, L.C. MobileNetV2: Inverted residuals and linear bottlenecks. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018. [Google Scholar]
- Zhang, T.Y.; Suen, C.Y. A fast parallel algorithm for thinning digital patterns. Commun. ACM 1984, 27, 236–239. [Google Scholar] [CrossRef]
- Mei, L.; Yan, D.; Chen, G.; Wang, Z.; Chen, S. Influence of laser beam incidence angle on laser lap welding quality of galvanized steels. Opt. Commun. 2017, 402, 147–158. [Google Scholar] [CrossRef]
Parameter | Result |
---|---|
Focal Length | [582.801412347346, 583.065607745483] |
Principal Point | [311.284435316149, 243.748894966768] |
Radial Distortion | [0.119887527023955, −0.149684597695317, −0.840199958301362] |
Tangential Distortion | [0.00236751860870302, −0.00598240042060791] |
Pixel Error | 0.0874304210396761 |
Type | PA (%) | MIoU (%) | TIME (s) |
---|---|---|---|
Segnet | 98.1 | 84.7 | 0.176 |
Unet | 97.3 | 81.2 | 0.145 |
DeeplabV3-MobilenetV2 | 98.9 | 87.9 | 0.172 |
Type | Morphological Erosions | Steger | Zhang–Suen | Closing Operation Steger | Dilation Zhang–Suen |
---|---|---|---|---|---|
Time (s) | 0.0128 | 0.0877 | 0.0361 | 0.1035 | 0.0477 |
Discontinuous area identification length | _______ | 285.2087 | 336.9275 | 319.8760 | 354.5796 |
Methods | Normal Fillet Weld | Highly Reflective Fillet Weld | Time per Picture | Maximum Error of Weld Feature Points | Maximum Angle between Plates |
---|---|---|---|---|---|
weld likelihood method | 100% | 100% | 0.18 s | 0.52 mm | ______ |
method in this paper | 100% | 98.9% | 0.23 s | 0.35 mm | 0.3657° |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wang, J.; Zhang, X.; Liu, J.; Shi, Y.; Huang, Y. Line-Structured Light Fillet Weld Positioning Method to Overcome Weld Instability Due to High Specular Reflection. Machines 2023, 11, 38. https://doi.org/10.3390/machines11010038
Wang J, Zhang X, Liu J, Shi Y, Huang Y. Line-Structured Light Fillet Weld Positioning Method to Overcome Weld Instability Due to High Specular Reflection. Machines. 2023; 11(1):38. https://doi.org/10.3390/machines11010038
Chicago/Turabian StyleWang, Jun, Xuwei Zhang, Jiaen Liu, Yuanyuan Shi, and Yizhe Huang. 2023. "Line-Structured Light Fillet Weld Positioning Method to Overcome Weld Instability Due to High Specular Reflection" Machines 11, no. 1: 38. https://doi.org/10.3390/machines11010038
APA StyleWang, J., Zhang, X., Liu, J., Shi, Y., & Huang, Y. (2023). Line-Structured Light Fillet Weld Positioning Method to Overcome Weld Instability Due to High Specular Reflection. Machines, 11(1), 38. https://doi.org/10.3390/machines11010038