Automated Fillet Weld Inspection Based on Deep Learning from 2D Images

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The paper detects fillet weld defects using CNNs trained on 2D images.

My comments are as follow:

 

The paper is poorly written, poorly organized, and has a lot of typos.

Example: page 7 line 198: To carry out the experiments of this study, a series of steps were carried out as can you seen in Figure 6.

 

The method is unclear, and its novelty seems to be very limited for a journal paper.

The reason for choosing the you only look once (YOLO) tool is not clear in the paper. As it is a journal paper, more details and discussion need to be added in the manuscript.

 

 

The author focused on detecting two main types of defects: lack of penetration and underbite, categorizing other defects under a general class. This narrow focus may overlook other significant defects that could be critical in a comprehensive weld inspection process.

 

As the complexity of the detection task increases, the performance of the model tends to decrease. Hence a question arises how the YOLO works for more complex detection scenarios?

 

Model performance may vary if images come from different systems or conditions, as discrepancies in quality can reduce detection accuracy, limiting its consistency across environments.

 

Author Response

We would like to express our sincere gratitude for the comments that have been provided with the aim of enhancing and elucidating certain issues that have been raised. Following a thorough review of the evaluations of our work, we will now address each of the comments and suggestions that have been made. The changes have been introduced to the article that is highlighted in yellow, and the parts of the document that have been replaced and therefore will not appear in the final version have been crossed out. 

 

Comment 1: 

The paper is poorly written, poorly organized, and has a lot of typos.

Example: page 7 line 198: To carry out the experiments of this study, a series of steps were carried out as can you seen in Figure 6.

 

Response 1: 

This appreciation is acknowledged, and consequently, a comprehensive review of the document has been conducted, resulting in amendments made not only to the attached example but also to other expressions deemed suitable for refinement. The ensuing section delineates the alterations that have been effected, together with the sections of the document in which said alterations have been implemented. In these sections, the parts that have been substituted are retained, whilst those that have been eliminated are indicated by being crossed out.

 

 

• In recent years, a range of methods and models based on the concept of neural networks, in its various variants, have been proposed by other researchers to address the tasks of inspection, monitoring or diagnosis of a weld. These methods have been employed in various welding processes across multiple sectors. A study presented in Liu [8] employed an artificial neural network (ANN) for the purpose of predicting welding residual stress and deformation in the context of electro-gas welding. Section 1.2 line 65.

 

• The following stage of the study will involve the acquisition of a series of images of the aforementioned welds, with the objective of training a neural network. of the study will entail the acquisition of a series of images depicting the aforementioned welds, with the objective of facilitating the training of a neural network. Section 2.1 line 307

 

• The experiments in this study were conducted in accordance with a series of steps, as illustrated in Figure 6. Section 2.3 line 301.

 

• As is universally acknowledged within the field of training, it is imperative to employ the most accurate data possible. Consequently, the primary focus was directed towards the acquisition of 2D images of the welding seams, characterised by optimal quality and precision, in alignment with the experimental protocol that was to be executed. Section 2.3.1 line 223.

 

• As illustrated in Figure 8, a steel plate that has been welded and appropriately labelled is presented. This plate serves as an example to the system, indicating that welds which have been manufactured to a high standard are labelled 'good label', while welds that exhibit defects are labelled 'bad label'. Section 2.3.2 line 354

 

• In the interest of optimising the system's convergence, it is imperative to implement a uniform size adjustment to each image. To this end, a recalibration of image dimensions has been executed. Concurrently, a self-orientation has been applied to each image, thereby facilitating the system's training with a more robust pattern. Section 2.3.3 line 371

 

• In the context of a YOLO system, a series of hyperparameters are configurable in accordance with the training to be performed. Section 2.3.3 line 397.

 

• The labels in each experiment were awarded in accordance with the visual assessment of two welding experts. These experts analysed each of the welded plates and determined, based on their experience and criteria in visual inspection of welding beads, whether a weld was well manufactured. The welds were categorised as follows: GOOD, BAD, FCAW or GMAW, DEFECT (undercut (UNDER), lack of penetration (LOP), or rest of defects (OP)). Section 3.1 line 533.

 

• This paper presents the findings of an investigative study conducted to detect fillet weld beads in a series of 2D images, which were also captured during the course of this study. The images were subjected to a rigorous treatment process, resulting in the creation of multiple datasets. These datasets were then used to conduct a series of experiments aimed at detecting various types of welds, assessing the quality of the weld bead fabrication, and identifying defects. The object detection process has been focused using the YOLOv8 algorithm, with appropriate configuration of its hyperparameters and application of a specific methodology developed for this study. Section 4 line 680.

 

Comment 2: 

The method is unclear, and its novelty seems to be very limited for a journal paper. As it is a journal paper, more details and discussion need to be added in the manuscript. As it is a journal paper, more details and discussion need to be added in the manuscript. 

Response 2:

• We are grateful for your perspective on this matter, however, we respectfully submit that there may be a slight inaccuracy in your viewpoint, which we would like to address:

 

• The present study employs a methodology previously applied in a study on machine learning [1], in which one of the authors participated, and which was found to be both interesting and appropriate for adaptation for use in research focused on deep learning, as presented. This publication was made in the journal Applied Science in 2020 and currently has more than 130 citations, as recorded in Google Scholar.

 

• Conversely, there are publications within the scientific community that have been accepted for publication and possess a similarity to the present study, including those that have been accepted by this publisher. In our opinion, these publications have not been disproven to the same extent as the methodological aspect, which is characterised by an absence of clarity regarding the steps taken and the development of the methodology. This is evidenced by the works cited in [2] and [3].

 

 

[1] Towards a Reliable Comparison and Evaluation of Network Intrusion Detection Systems Based on Machine Learning Approaches. Applied Sciences 2020, Vol. 10, Page 1775 2020, 10, 1775. 755. https://doi.org/10.3390/APP10051775.

 

[2] Oh, S.J.; Jung, M.J.; Lim, C.; Shin, S.C. Automatic detection of welding defects using faster  R-CNN . Applied sciences 2020, 10, 1–10. https://doi.org/10.3390/app10238629 .

 

[3] Chang, Y.; Wang, W. A Deep Learning-Based Weld Defect Classification Method Using Radiographic Images with a Cylindrical Projection. IEEE Transactions on Instrumentation and Measurement 2021, 70. https://doi.org/10.1109/TIM.2021.3124053

 

• It is indeed the case that the rationale behind the selection of the YOLO algorithm for the execution of the experiment, as opposed to other analogous algorithms, may not be entirely evident. In order to address this proposal, Section 2.3.6 has undergone a partial revision, wherein the deep learning model employed is elucidated. Furthermore, additional references have been incorporated to substantiate the selection, drawing upon extant research.

 

• It is indeed the case that the rationale behind the selection of the YOLO algorithm for the execution of the experiment, as opposed to other analogous algorithms, may not be entirely evident. In order to address this proposal, Section 2.3.6 has undergone a partial revision, wherein the deep learning model employed is elucidated. Furthermore, additional references have been incorporated to substantiate the selection, drawing upon extant research.

 

• In sections 3.2.1, 3.2.2 and 3.2.3, corresponding to the exposition and discussion of the experiments, they have been expanded with a more detailed discussion.

 

Comment 3:

The author focused on detecting two main types of defects: lack of penetration and underbite, categorizing other defects under a general class. This narrow focus may overlook other significant defects that could be critical in a comprehensive weld inspection process.

Response 3: 

• We are grateful for the commentary and acknowledgement. To make clear to the reader of the article the initial reason for this research, a paragraph has been added to the Conclusions Section, which records this motivation, in addition to clearing the way for this research in a more in-depth way.

 The results demonstrate the efficacy of the approach, indicating the potential for further research in this domain. It is evident that further refinement and expansion of this research are possible, as the scenarios can be rendered more complex to assess the efficacy of the algorithm in question. Additionally, conducting a wider range of experiments that encompass additional classes, such as welding defects, would further augment the dataset and facilitate more comprehensive investigation.

 

• In the third experiment, Section 3.2.3, reference is also made to the authors' intention with this study.

Comment 4: 

As the complexity of the detection task increases, the performance of the model tends to decrease. Hence a question arises how the YOLO works for more complex detection scenarios?

Response 4: 

• We would like to express our gratitude for your consideration of this matter. It has been demonstrated that the efficacy of the model is diminished when confronted with a more intricate scenario involving the YOLO algorithm. This outcome is predicable, provided that the loss does not reach a substantial threshold. As illustrated in Table 3, the mAP metric is the most impacted, attaining a performance of 77% within a designated class. This is believed to be a consequence of the reduced number of samples, as evidenced in Table 2, which were the lowest among all the classes utilised in the experiment.

 

• This reflection and references will be added to section 3, Results and Discussion, of this work.

 

• A bibliographic search was conducted, the results of which indicated the presence of other authors who have utilised the YOLO algorithm for scenarios of greater complexity than those proposed in this study. Illustrative examples can be found in [1] and [2].

 

[1] Liu, Y.; Shi, G.; Li, Y.; Zhao, Z. M-YOLO: Traffic Sign Detection Algorithm Applicable to Complex Scenarios. Symmetry 2022, 14, 952. https://doi.org/10.3390/SYM14050952.

[2] Xu, W.; Fang, H.; Yu, S.; Yang, S.; Yang, H.; Xie, Y.; Dai, Y. RSNC-YOLO: A Deep-Learning-Based Method for Automatic Fine-Grained Tuna Recognition in Complex Environments. Applied Sciences 2024, 14, 10732. https://doi.org/10.3390/APP142210732.

 

Comment 5: 

Model performance may vary if images come from different systems or conditions, as discrepancies in quality can reduce detection accuracy, limiting its consistency across environments.

Response 5:

• Expressions of gratitude are extended for the appreciation received. It is imperative to provide a comprehensive account of the methodology employed in acquiring the images, with the objective of enabling other research teams to replicate the experiment or evaluate its quality. While a degree of detail regarding the image acquisition process was provided in Section 2.3.1, this information has been further elaborated in the same section through the incorporation of an additional paragraph. The content of this paragraph is as follows:

 

The distance at which the images were captured fell within the operational range recommended by the camera manufacturer, with a minimum distance of 270 mm and a maximum distance of 3000 mm. Consequently, images were captured from a variety of positions and distances, ranging from 1200 mm to 1700 mm. The dimensions of the images were consistent across all captures, aligning with the size recommended by the camera for optimal quality. The three experiments that have been conducted differ in terms of the classes to be recognised and the manner in which the images are treated. In the initial experiment, weld seams were extracted in great detail, with two classes being attempted to be identified. In the second and third experiments, the complete image was used, with all welding seams, resulting in the detection of two classes in experiment two and four classes in experiment three.

 

 

 

 Additionally:

 

• Furthermore, content has been appended to Section 1.1, Background.

 

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

 The paper presents an utomated fillet weld inspection based on deep learning YOLOv8 from 2D weld images. Especially for two type of the common welding processes, the Flux-Cored Arc Welding (FCAW) and Gas Metal Arc Welding (GMAW) .The application is in practical, but The method is limited.

1 When capiture the images of welding, how the distance is used? And the image size, espscially the weld region rate in the image should be statisticed. We all know, the small objects are difficult the be detected.

2 The dataset size is not mentioned. How many images is captured? How about the training and testing set?

3 In the contribution, the second is "The development of a methodology that can be used in other works based on image detection". What does that mean? Three are three expriments are perfermed, but did not show the second contribution.

Author Response

We would like to express our sincere gratitude for the comments that have been provided with the aim of enhancing and elucidating certain issues that have been raised. Following a thorough review of the evaluations of our work, we will now address each of the comments and suggestions that have been made. The changes have been introduced to the article that is highlighted in yellow, and the parts of the document that have been replaced and therefore will not appear in the final version have been crossed out. 

 

Comment 1:

When capiture the images of welding, how the distance is used? And the image size, espscially the weld region rate in the image should be statisticed. We all know, the small objects are difficult the be detected.

Response 1: 

• It is true that the distance at which the images of the weld seams were taken have not been mentioned.  Please find the aforementioned assessment added to Section 2.2.1.

  The distance at which the images were captured fell within the operational range recommended by the camera manufacturer, with a minimum distance of 270 mm and a maximum distance of 3000 mm. Consequently, images were captured from a variety of positions and distances, ranging from 1200 mm to 1700 mm. The dimensions of the images were consistent across all captures, aligning with the size recommended by the camera for optimal quality. The three experiments that have been conducted differ in terms of the classes to be recognised and the manner in which the images are treated. In the initial experiment, weld seams were extracted in great detail, with two classes being attempted to be identified. In the second and third experiments, the complete image was utilised, with all welding seams, resulting in the detection of two classes in experiment two and four classes in experiment three.

 

• Furthermore, all images were resized to a standard size of 320x320, as detailed in Section 2.2.5.

 

• The weld zone was labeled with roboflow by assigning labels of non-uniform size, adjusted to the perspective of the weld bead, although these had a similar size, between 5 and 7 cm, as mentioned in Section 2.1.

Comment 2:

The dataset size is not mentioned. How many images is captured? How about the training and testing set?

Response 2: 

• We do not really add the number of images we use in the different datasets, we are grateful for the commentary and acknowledgement.

 

• In order to demonstrate that the steps proposed in the methodology can be used in different experiments, it was decided to create an independent dataset for each experiment. Thus, although the same images were used, the labeling was different in each of them, the number of images included for each class in each experiment varies due to the aforementioned factors.

 

• The proportion of data was consistent across all three experiments, comprising an 80% training set, 10% validation set, and 20% test set.

 

• Please refer to Table 2 for a detailed account of the aforementioned alterations.

 

Comment 3:

 In the contribution, the second is "The development of a methodology that can be used in other works based on image detection". What does that mean? Three are three expriments are perfermed, but did not show the second contribution.

 

Response 3: 

Thank you for your comment. We made a mistake in the wording when defining. What was meant was that a methodology had been created, a series of steps, which can be used to recognize different objects. The three experiments were conducted using different approaches to image processing and classification. In the first experiment, weld seams were extracted in great detail, and two classes were identified. In the second and third experiments, we worked with the complete image, detecting two classes in experiment two and four classes in experiment three. The aforementioned alterations were incorporated into Section 1.3.

This contribution is rewritten again in the paper to make its intention clearer…

The development of a methodology with a series of steps that can be used in other research dealing with the detection of objects through images.

 

Additionally: 

Furthermore, content has been appended to Section 1.1, Background.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

Interesting article. Very extensive research methodology, where all assumptions made in the analysis were specified.

The material for learning were two types of welds made using the GMAW and FCAW methods. The methods differ in the type of filler metal, and therefore in the heat distribution during welding. As a result of welding with appropriately selected process parameters, the shape of the weld should be identical from the point of view of observing photos. The difference is only in defects, but they are not justified in the case of the analyzed methods. In the case of flux-cored wires, the key factor is the silicon oxide deposited on the surface, which causes poor paint adhesion. Why was this factor not taken into account when analyzing the images.

On the other hand, the correctness of the fusion into the sheet metal in fillet welds is an important factor. How can the correctness of the execution of the opposite side be revealed by viewing the face?

The fillet weld is arranged spatially. Welds with a length of 50-70 mm were analyzed, but there is no key parameter here, i.e. the thickness of the weld. Why was it not included in the algorithm? When performing tests, a welding expert should first check the thickness and shape of the weld. Otherwise, all the defects (welding imperfections) revealed are irrelevant. To illustrate the situation, I will give an example - the aim of the process was to make a ship, and a car was created. What does it matter that it is sporty and red, since it cannot sail?

However, leaving aside the methodological errors indicated above and taking advantage of the opportunity that you have good material for analysis, I propose:

1. Shorten the methodological introduction - especially in the scope of general information

2. Specify the essential criteria for the assessment of the Fillet weld in the scope of the assessment of the weld face.

3. From the essential criteria, select those that are subject to assessment in the applied algorithm.

4. And only now can you start analyzing the applied assessment system.

Author Response

We would like to express our sincere gratitude for the comments that have been provided with the aim of enhancing and elucidating certain issues that have been raised. Following a thorough review of the evaluations of our work, we will now address each of the comments and suggestions that have been made. The changes have been introduced to the article that is highlighted in yellow, and the parts of the document that have been replaced and therefore will not appear in the final version have been crossed out. 

 

Comment 1: 

The material for learning were two types of welds made using the GMAW and FCAW methods. The methods differ in the type of filler metal, and therefore in the heat distribution during welding. As a result of welding with appropriately selected process parameters, the shape of the weld should be identical from the point of view of observing photos. The difference is only in defects, but they are not justified in the case of the analyzed methods. In the case of flux-cored wires, the key factor is the silicon oxide deposited on the surface, which causes poor paint adhesion. Why was this factor not taken into account when analyzing the images.

Response 1:

• In the study, a combination of CO₂ (enhanced penetration) and argon (improved protection) was employed, taking into account the material being welded (carbon steel). Prior to initiating the experiments, it was believed that this combination would suffice to achieve the objective, namely the detection of a weld bead according to a search factor: fcaw or gmaw welding, well-made or poorly-made bead, defect found. The performance of the system in this detection was highly satisfactory.

 

• A concise account of the proportion of gases employed in the experiment is provided in Section 2.1., The following illustration demonstrates this point.

 

This combination of gases was employed with the objective of combining the optimal performance of each gas in a welding process, namely CO₂ (enhanced penetration) and argon (improved protection). This approach was informed by the specific characteristics of the material to be welded (carbon steel). Furthermore, the reliability of the weld and its final quality are more balanced.

 

Comment 2: 

On the other hand, the correctness of the fusion into the sheet metal in fillet welds is an important factor. How can the correctness of the execution of the opposite side be revealed by viewing the face?

Response 2:

• It should be noted that the document does not include details of the standards that have been followed. However, the suggestion to include them has been appreciated. Therefore, the ISO 15792-3:2011 standard was followed for the welding face and the rest of the preparation. Details of how the suggestions in the final part of the list (Proposals 2, 3 and 4) were addressed can be found in the end of the document.

Comment 3:

The fillet weld is arranged spatially. Welds with a length of 50-70 mm were analyzed, but there is no key parameter here, i.e. the thickness of the weld. Why was it not included in the algorithm? When performing tests, a welding expert should first check the thickness and shape of the weld. Otherwise, all the defects (welding imperfections) revealed are irrelevant. To illustrate the situation, I will give an example - the aim of the process was to make a ship, and a car was created. What does it matter that it is sporty and red, since it cannot sail?  It is accurate to conclude that the article in question does not provide a substantial amount of information on the subject matter. Accordingly, the specific type of welding wire utilized in each welding process (GMAW, FCAW) and the type of steel plate employed are also included.

Response 3:

• We would like to express our gratitude for your submission of this suggestion. In response to your valuable input, the following text has been incorporated into Section 2.1 of the document, providing comprehensive details on the consumables and steel plates employed in the study.

 

The SC-420MC titania flux-cored wire is suitable for all-position welding with either 100% CO₂ shielding gas or Ar-20%CO₂ shielding gas. The reduced spattering and enhanced slag detachability result in a shorter bead grinding operation. On the other hand, the SM-70 eco wire, a general-purpose AWS ER70S-6 copper-clad solid wire suitable for manual and semi-automatic applications, was utilised in this investigation for GMAW weld seams. Its extensive utilisation in a multitude of industrial sectors, including structural fabrication, automotive, heavy machinery and shipbuilding, attests to its versatility and reliability. Moreover, the steel employed in the experiments was 6 mm thick carbon steel, designated S275JR. Both the welding wires and the steel utilized are commonly utilized in the shipbuilding industry.

 Comment 4: 

Shorten the methodological introduction - especially in the scope of general information

 Response 4:

• For the sake of convenience, Section 2.2 has been abridged.

 Comment 5-6-7 

Specify the essential criteria for the assessment of the Fillet weld in the scope of the assessment of the weld face.

From the essential criteria, select those that are subject to assessment in the applied algorithm.

And only now can you start analyzing the applied assessment system.

 

 Response 5-6-7: 

In order to respond to this comments, the ISO 15792-3:2011 standard was consulted in order to ascertain the aspects that must be taken into account when preparing the welding piece for testing. Subsequently, the elements that were taken into account in the experiments were selected. These have been incorporated into a new subsection in the paper, Section 2.2 (the current Section 2.2, entitled 'Methodology', will be renamed Section 2.3).

 

 

Additionally:

Furthermore, content has been appended to Section 1.1, Background.

Author Response File: Author Response.pdf

Round 2

Reviewer 3 Report

Comments and Suggestions for Authors

Thank you for your answers and changes.

I don't have any more comments.

Author Response

Comments 1: 

Thank you for your answers and changes.

I don't have any more comments.

 

Response 1:

Thank you for your feedback. It has allowed us to improve the article, making it more complete.