An Efficient Hybrid Framework for Weld Defect Detection Using GAN, CNN and XGBoost ^†

Abstract

Automated detection of defects in welds are inevitable in the assurance of structural integrity, but this faces serious challenges due to the microscopic characteristics of the discontinuities, low visual contrast and infrequent occurrence of defect samples. Conventional deep learning methods, while accurate, often lack interpretability and exhibit low recall for rare defects. This paper proposes a novel hybrid system combining a Generative Adversarial Network (GAN), a Convolutional Neural Network (CNN), and Extreme Gradient Boosting (XGBoost 2.0.0) to enhance weld defect classification performance and transparency. Firstly, a Deep Convolutional GAN (DCGAN) creates synthetic images of the minority classes; thus, the problem of class imbalance is resolved. Then, a pretrained ResNet50V2 CNN is used to extract features of the deep layers from the original images as well as from the generated ones. After that, these features are fed into an XGBoost classifier, which uses tree-based learning to optimize classification results and make the process more understandable to the user. Furthermore, interpretation is also facilitated by Grad-CAM rendering of the CNN regions of interest and SHAP analysis to measure the involvement of the features in XGBoost. Experiments using the available LoHi-WELD datasets show that the overall accuracy is significantly improved, the per-class recall of the rare defects is also enhanced, and the robustness is also improved. The proposed hybrid method not only achieves better results but also generates visual/explainable output, which is very valuable when the system is implemented in industrial welding inspection systems. This paper serves as a liaison between the latest AI technology and the practical interpretability requirements of the mechanical and welding engineering fields.

1. Introduction

Welding is known as one of the primary fabrication processes in many industries, such as construction, aerospace, manufacturing, and energy. The main focus of any welded structure is its stability and durability over time. The dataset consists of four types of weld defects that are characterized by a specific physical appearance and formation processes. Cracks take the shape of linear metallic fractures, which occur when the strains at a point exceed the final strength of a material, and they develop either due to rapid cooling or high restraint during solidification. They are considered the most problematic defect because they are prone to spreading; to prevent this, low-hydrogen-content electrodes on base metal are used to minimize the degree of thermal shock. Deposits are hard lodgings, i.e., slag or foreign contaminants that have been lodged in the molten weld pool. These are typically caused by inappropriate cleaning between passes or the mishandling of the electrode and can be prevented by appropriate inter-pass grinding and by having a clean weld area. Structural continuity defects (discontinuities) are a broader group of structural defects, such as incomplete penetration or lack of fusion, where the weld metal fails to bond with the base material due to insufficient heating or due to incorrect travel speed. Mitigation is essential to ensure suitable current settings and travel rates. Finally, stains are oxidation or discoloration of the surface due to the lack of shielding gas cover. They are normally cosmetic, though they can herald impaired resistance to corrosion; they can be prevented by maximizing the rate of gas flow and eliminating drafts in the welding area.

Traditionally, the chief method of checking the quality of a weld is through Nondestructive Testing (NDT) procedures like radiographic testing (RT) or ultrasonic testing (UT), which are then reviewed by certified human personnel. Nevertheless, this traditional approach is accompanied by a number of downsides. Human inspection is naturally very slow and thus not suitable for high-volume production environments where it can create bottlenecks. More importantly, the reading of radiographic images is a subjective process in which different inspectors may come to different conclusions. This inconsistency, together with the possibility of human mistakes caused by tiredness or lack of attention, makes the conventional technique far from ideal for providing stable quality assurance at high speed. The worldwide shift towards Industry 4.0, smart manufacturing, and interconnected systems has turned the use of automated, reliable, and intelligent NDT tools into a must-have condition. In line with this, computer vision has become one of the main factors enabling this transition. DL, especially with an architecture like CNN, has changed the way image-based tasks are handled, as it has an outstanding capability of automatically deriving complex, hierarchical feature representations from raw image data. For weld defect analysis, DL provides the technology to achieve complete automation of the defect detection and classification process at an unprecedented speed and with very high accuracy, thereby making industrial quality control automation-friendly in the first place.

The primary objective of this project is the development of a powerful automated system capable of classifying weld defect images. While typical CNN models have shown promising results in the field of weld defect analysis, obtaining the best performance turns out to be very challenging, especially when there is a class imbalance problem. This project not only uses a regular CNN but also experiments with a new hybrid strategy that combines the advantages of different AI paradigms: the feature-extraction ability of top-tier Deep Neural Networks (DNNs) and the strong classification ability of standard boosting algorithms, in particular, Extreme Gradient Boosting (XGBoost). The reason for this research to explore such a combination is to achieve higher accuracy as well as to be able to interpret the results, which is extremely important in the case of implementation in industrial sectors where safety is critical.

2. Literature Survey

Weld defect analysis has been a primary focus of research, primarily aimed at maintaining structural integrity and reducing reliance on human inspection in industrial quality control. A ResNet50-based CNN was used for classifying the four most common weld defect types from the RIAWELC dataset and reached a 98.75% accuracy level [1]. Similarly, machine vision-assisted CNN models were applied to an industrial dataset and achieved high classification accuracy, thus proving the great potential of CNN-based systems in welding quality control for reducing human labor [2]. A smart recognition system that combined CNN-based feature extraction with a Random Forest classifier was developed to recognize welding surface defects, which demonstrated excellent performance even in noisy environments [3]. To locate defects precisely in automotive brake joint laser weld seams, YOLOv8 was used extensively, thereby establishing a method that was not only fast but also accurate for defect localization, which made it suitable for real-time industrial applications [4].

Earlier methods based on traditional image processing and manually engineered features, as outlined in foundational signal processing studies [5], were found to be less efficient in the presence of noise, changing welding conditions, and highly complex defect patterns. An unsupervised system was developed that uses both audio and video signals for detecting 11 different types of welding defects, making it more suitable for use in real-time production lines, as it is designed to be fully automated and scalable [6]. Standard documentation and reporting practices in engineering research [7] have also guided the systematic presentation of such inspection methodologies. The LoHi-WELD dataset, a recent industrial dataset for weld defect detection and classification, was introduced [8]. XGBoost was employed to predict the seam tensile strength online during laser welding, confirming the potential of ensemble learning models in monitoring and optimizing welding processes [9]. The great scalability and efficiency of XGBoost as a foundational framework was further presented [10].

Techniques for interpreting models like SHAP [11] and Grad-CAM [12] have provided greater understanding and trust in decision-making based on deep learning. The GDXray dataset, which is a widely used benchmark for radiographic weld inspection, was introduced [13]. Surveys, such as the comprehensive review of deep learning for unsupervised anomaly localization in industrial images [14], have extensively reviewed recent developments and identified main issues, including generalization, data imbalance, and explainability. It is also worth noting that power electronics innovations, such as high-frequency step-down converter designs, have similarly benefited from intelligent control and optimization techniques in adjacent engineering domains [15]. A conditional GAN-based method combined with transfer learning was developed to extract contrast-enhanced features and solve the class imbalance in weld defect detection [16].

Therefore, methods based on deep learning, especially those using Convolutional Neural Networks (CNNs), have become the most popular methods for weld defect analysis. With deeper neural networks becoming popular, Residual and very deep CNN models shortly became mainstream. The foundational architectures of deep models such as ResNet [17], VGGNet [18], and EfficientNet [19] account for the great success of such methods. There are still significant problems of data scarcity and class imbalance in weld defect analysis, and generative models and data augmentation techniques have been suggested to alleviate these problems. Following the foundational introduction of Generative Adversarial Networks (GANs) [20], researchers have adapted these models for industrial use. Overall, current papers show that CNN-based and hybrid deep learning models can reach high accuracy levels for the classification and detection of welding defects. Nonetheless, unresolved issues such as the lack of data, explainability, consistent performance under various welding conditions, and the feasibility of real-time use still exist in the research area. Thus, these limitations call for continuing research into intelligent weld inspection systems that are not only robust but also explainable and scalable.

3. Background

The automation of weld quality assessment is already becoming a key issue in many industries that rely on consistent joint integrity, including large-scale manufacturing, construction, and aerospace fabrication. Traditionally, weld images—either generated by radiographic inspection or by any visual NDT procedure—are reviewed by trained human specialists. While this method has proven effective, it is often very slow and inconsistent, being subject to the variable nature of human judgment, fatigue, and the increasing levels of data generated in contemporary production environments. With the development of fully integrated industrial systems embracing digital technologies and autonomous workflows, the demands placed on reliable, high-throughput computer vision applications for weld assessment keep increasing.

Deep neural networks, especially convolutional models, have demonstrated excellent ability in learning representative spatial patterns from weld radiographs without relying on manually crafted descriptors. Indeed, several works report very good classification results using CNN-based approaches. However, there are some persistent problems, including the inherent difficulty of interpreting deep models, limited transparency of decision-making, and the frequent presence of imbalanced datasets that prevent robust training. Within this context, recent studies have focused their attention on hybrid architectures and generative models as further steps toward enriching diversity and balance in training data.

This paper proposes a novel hybrid system comprising a Generative Adversarial Network (GAN) to produce synthetic data, a Convolutional Neural Network (CNN) to extract features, and an XGBoost model to perform the final classification. These three methods each have a specific role to play. The GAN is used to address the issue of class imbalance in this dataset, since one or more of the types of defects is underrepresented. The GAN will be useful in generating more homogeneous training data by producing synthetic samples of the type of defect that has been underrepresented in the dataset (the minority class). The CNN will extract a consistent, high-level feature representation of the defect types from both original raw images and the newly generated synthetic images. The XGBoost approach introduces an additional contextual interpretation and identifies the point at which the decision boundary may be considered stable. The adoption of this combination of methods enhances prediction performance and eliminates the ambiguity that is associated with the identification of weld defects. Such a combination not only results in improved classification of weld defects but also allows practical use in challenging industrial inspection environments where a high priority is given to reliable and repeatable results.

4. Gaps in the Existing System

Limited Datasets and Generalization: The scarcity of small, proprietary data is a fundamental weakness in the existing research on weld defect detection. CNNs and other deep learning models require large amounts of varied data to learn strong features and feature representations; in the case of small amounts of training data, the networks are likely to overfit and memorize noise patterns instead of learning defect morphologies that can be generalized across cases. Moreover, small datasets are not able to represent the stochastic variations in an industrial setting such as dissimilar weld geometry, varying X-ray levels, and infrequent defect types such as cracks, leading to low efficiency when models are applied to unknown data (domain shift). Lastly, the absence of large-scale, publicly distributed benchmarks makes it impossible to perform standardized comparison of methodologies, since high accuracy on small, balanced datasets does not necessarily translate to reliability in class-imbalanced, real-world manufacturing settings due to the lack of synthetic data augmentation. Only a very few studies have been found to use traditional augmentations; however, none of them has implemented advanced synthetic data generation (e.g., using GAN models), the absence of which has been identified as the major bottleneck in most studies and thus makes it difficult to develop robust models in cases where datasets are small or imbalanced.

Limited Interpretability: The majority of the high-accuracy models referred to are deep CNNs that function as ”black boxes.” The absence of interpretability in these models constitutes a significant drawback in industrial applications where safety is critical, as acknowledged by [2]. Narrow Scope: The research focus is sometimes limited to a narrow range of defect types (for example, only four classes), a specific imaging domain (e.g., surface defects only), or a certain task (e.g., segmentation without classification).

Past research on the classification of welding defects has mainly been based on the use of monolithic, standalone CNN architectures that are accompanied by simple preprocessing methods such as Gaussian filtering. However, when these end-to-end deep learning techniques are used in complex real-world industrial environments, they exhibit serious limitations. The major flaw in the earlier studies is that they depend on shallow networks that not only try to perform feature extraction but also classification at the same time and, as a result, in most cases, they do not manage to optimize the decision boundary if the training data are limited. In addition, the researchers conducted their experiments on small, artificially balanced datasets without taking into consideration that there is a severe class imbalance problem in the manufacturing industry, which leads to the creation of models that overfit. Furthermore, the methods preceding this work almost completely overlooked the utilization of modern hybrid strategies; not only did they lack a generative model that would help generate the missing data, but the ensemble classifiers that could potentially be used for the final stage of prediction to improve the results were also ignored. The traditional models discussed here, in addition to the above, operated as ’black boxes’ with no provision for explainability mechanisms, thus not allowing any form of visual or feature-level explanation for their decisions. This, unquestionably, is a critical limitation as far as safety-critical NDT applications are concerned. Most noteworthy, however, are the previous evaluation methods, which were commonly confined to small internal datasets with little, if any, validation on different industrial samples. The research presented here goes beyond these limitations by introducing an advanced, industrial-grade hybrid CNN–XGBoost paradigm. This work departs from the previous monolithic approaches by splitting the architecture; in other words, the deep pretrained CNN backbones (ResNet50 and VGG19) are used only for exhaustive feature extraction in order to capture the complicated radiographic patterns. The deep features of these high dimensions are then given to XGBoost, which is an efficient gradient-boosted ensemble classifier and hence is able to produce better results than the standard fully connected layers while simultaneously managing feature importance and non-linear relationships. The amalgamation of these two technologies is what leads to an increase in classification accuracy to a much higher level. Additionally, this research has adopted a transparent AI solution where model transparency and reliability are guaranteed by employing technologies such as Grad-CAM for spatial visualization and SHAP for feature-level interpretability. As a matter of fact, the proposed system has been tested thoroughly on the dataset kept aside for the LoHi-WELD, making it perfect for deployment in the real-world industrial field.

5. The Proposed System

In order to overcome these shortcomings, this paper proposes a hybrid model combining GANs, CNNs, and XGBoost to ensure high accuracy and interpretability of weld defect classification. The layout of this system is such that it is capable of dealing with data insufficiency and model non-interpretability issues simultaneously.

• GAN: To overcome the problem of data scarcity and class imbalance, a GAN will be trained to generate realistic, synthetic radiographic images of weld defects, particularly for defect classes that have been less studied. The artificially created data will be utilized for extending the training dataset, and thus the model’s ability will be enhanced.
• CNN: The major role of a pre-trained CNN (such as ResNet50 or VGG19) is to serve as an automatic, very efficient feature extractor. Instead of its own fully connected layers being fed with features for classification, the CNN will find the weld images and produce feature vectors of very high dimension, which represent the complex spatial patterns of defects.
• XGBoost: The main classifier, a very simple yet powerful tree-based algorithm, i.e., XGBoost, will be the one to perform the main work. It will be trained on the CNN-derived feature vectors. This hybrid CNN–XGBoost scheme is generally more accurate and faster in training than a normal deep learning classifier, and this is mainly because of the tabular nature of the data (which feature vectors are) on which the method is carried out.
• Interpretability: Two local explanation methods, SHAP and LIME, will be the ones used with the trained XGBoost model. After providing the rationale for a certain classification (e.g., ”this is a ’crack’ because of these particular features”), the system can reveal the internal workings and, therefore, grant trust in the model’s decisions.

6. Methodology

As the first step, standard transfer learning methods have been applied to a few well-known trained CNN architectures. The upper classification layers of these networks were changed to custom dense layers that fit our particular weld classes. The models that were considered are:

The VGG19 deep learning model is a straightforward model, known for its clarity and depth. ResNet50 has residual connections that allow the training of networks of very deep layers without the problem of gradients vanishing. EfficientNet-B0 is an up-to-date architecture that maintains a balance between accuracy and computational power usage by compound scaling. Finally, these models were fine-tuned end-to-end on the weld dataset.

Public domain industrial datasets are used to assess the performance and generalization capability of the proposed framework: The LoHi-WELD dataset comprises 3022 images of weld beads recorded in robotic MAG welding; it contains a variety of resolutions, different imaging conditions, and four main defect categories: pores, deposits, discontinuities, and stains. The dataset has large internal variation within each class, and many classes of defects are represented only very sparsely—a combination of factors that give strong motivation for the use of synthetic augmentation.

The datasets were standardized beforehand through a preprocessing pipeline to ensure compatibility across components in the hybrid framework. All images were resized to a common resolution suitable for CNN input, such as 224 × 224 pixels, and their pixel intensity values were normalized to maintain consistent dynamic ranges across samples. Thereafter, the LoHi-WELD data were split into training, validation, and test sets in a manner that preserved the relative proportions of the four categories of defects. Due to the fact that several classes in LoHi-WELD are of relatively small size, additional representations of those minority defect types were created by training a GAN. Synthetic samples created in this way were combined with real images to balance the dataset. Both authentic and generated images were used as input to a CNN that extracted deep feature embeddings in a common representation space. The extracted embeddings were fed into an XGBoost classifier, which constituted the last decision layer of the system. Thus, the proposed preprocessing pipeline standardizes the input, adds balance between classes, and enables consistent feature encoding for subsequent classification.

The datasets were pre-processed with a standardization pipeline to make them compatible across the elements of the hybrid framework. It was also necessary to resize the images from different sources to the standard size of 224 × 224 pixels. The reason behind its selection is that this dimension corresponds to the native input size of the three pre-trained CNN models employed in this study (VGG19, ResNet50V2, and EfficientNetB0). With this resolution, it is still possible to use the transfer learning weights that have been trained on ImageNet by the model without affecting the learned feature hierarchy at various spatial resolutions. Such a radical change in the input size would imply that the preliminary layers would need to be retrained from scratch, or that extremely small textural features that are invaluable in the detection of tiny defects in welds like cracks or pores would be lost [1]. Additionally, pixel intensity values were scaled in order to achieve the same dynamic ranges across samples. Subsequently, the LoHi-WELD data were divided into the training, validation, and test sets in such a way that the proportionality of the four categories of defects was maintained [2]. Since a few of the classes in LoHi-WELD are of relatively small size, further representations of those minority defect types were obtained by training a GAN [3]. Artificial samples prepared in this manner were mixed with actual images to balance the dataset [4]. Authentic and generated images were introduced as input into a CNN to produce deep feature embedding in a common space [5]. These embeddings were extracted and input into an XGBoost classifier, which was the final decision layer of the system [6]. In this way, the suggested preprocessing pipeline is able to standardize the input, introduce balance between classes, and provide uniform encoding of features for further classification.

Input Acquisition: Import weld inspection images from the LoHi-WELD dataset as the primary source and GDXray as an external test set. Each sample should include the radiographic image and its corresponding defect label (pore, deposit, discontinuity, or stain). Preprocessing: Resize all images to a fixed resolution (e.g., 224 × 224), normalize pixel intensities, and convert to a consistent color space if required. Split the data into training, validation, and test subsets while preserving class distribution. GAN-Based Data Augmentation: Train a Deep Convolutional GAN (DCGAN) on the training subset with a focus on minority defect classes. Generate synthetic images after convergence and merge them with the real data to form a balanced dataset. Although more modern generative architectures like WGAN or StyleGAN have enhanced features to synthesize semantic images with high resolution, DCGAN was chosen to be used in this work because of the nature of weld defects, both structurally and texturally. In contrast to facial or scenic imagery, where StyleGAN has been shown to be particularly successful in handling complex and high-level semantic variations, weld defects have localized, relatively low-level geometric shapes and intensity differences. DCGAN is more computationally efficient, with a stable convergence point during training, than other GAN structures in the case of this particular application and size of a dataset, as it does not require a large quantity of resources and is not prone to instability when training larger and more complex generative models on small pools of industrial data. Feature Extraction with CNN: as stated in Figure 1. The pre-trained backbones used to obtain the feature vectors were the fixed feature extractors of VGG19, ResNet50V2 and EfficientNetB0. The features were specifically acquired at the Global Average Pooling (GAP) layer immediately before the final fully connected classification head. This is the result of the final max-pooling block, flattened to a vector in the VGG19 architecture, and the final global pooling block of ResNet50V2 and EfficientNetB0 This particular layer is chosen because it contains the most semantically abstracted information in the input image, including a few detailed structural patterns, e.g., the shape of the cracks or the concentration of the porosity, and it does not carry spatial redundancy, as this was eliminated in the earlier convolutional layers. These are mini high-density feature vectors, which will prove to be an efficient representation for the next XGBoost classifier. Feature Set Construction: Aggregate all extracted features into a tabular feature matrix and prepare label mappings for classification. XGBoost-Based Classification: Train an XGBoost classifier on the CNN-derived features using optimized hyperparameters and validation feedback. Explainability Module (XAI): Apply Grad-CAM to interpret spatial CNN activations and apply SHAP (or LIME) to compute feature importance for XGBoost predictions. Classification and Training Procedure: During each training iteration, update the GAN when needed, extract CNN features, train XGBoost on the updated dataset and tune using learning rate schedules and validation feedback. Output: For a new test image, compute the predicted defect class and provide attribution maps (Grad-CAM + SHAP). Evaluate performance using Accuracy, Precision, Recall, F1-score, AUC-ROC, and the Sankey Diagram.

Figure 1. CNN architecture diagram.

There are multiple equations:

• GAN Min-Max Objective

  $$\underset{G}{\min } \underset{D}{\max }V\left(D,G\right)={\mathbb{E}}_{x \sim p_{\mathrm{data}}}\left[\log D\left(x\right)\right]+{\mathbb{E}}_{z \sim p_Z}\left[\log \left(1-D\left(G\left(z\right)\right)\right)\right]$$

  (1)
• Generator and Discriminator Losses

  $$L_D=-{\mathbb{E}}_{\left(\mathit{x}\sim p_{\mathrm{data}}\right)}\left[logD\left(\mathit{x}\right)\right]-{\mathbb{E}}_{\left(\mathit{z}\sim p_{\mathit{z}}\right)}\left[log\left(1-D\left(G\left(\mathit{z}\right)\right)\right)\right]$$

  (2)

  $$L_G=-{\mathbb{E}}_{\left(z\sim p_z\right)}\left[log\left(D\left(G\left(z\right)\right)\right)\right]$$

  (3)
• CNN Feature Extraction Mapping

  $$f={\varphi }_{\theta }\left(x\right)$$

  (4)
• XGBoost Prediction Function

  $$\widehat{y}=F\left(f\right)=\sum {\left(t=1\right)}^T{f}_t\left(f\right)$$

  (5)
• XGBoost Objective with Regularization

  $$L=\sum {\left(i=1\right)}^{N}l\left(y_i,\widehat{y_i}\right)+\sum {\left(t=1\right)}^T\Omega \left(f_t\right)$$

  (6)

  $$\Omega \left(f_t\right)=\gamma T_t+\lambda /2\sum {\left(j=1\right)}^{\left(I_t\right)}{w}_{\left(t,j\right)}^2$$

  (7)
• Evaluation Metrics

  $$\mathrm{Accuracy}={\displaystyle \frac{\left(TP+TN\right)}{\left(TP+TN+FP+FN\right)}}$$

  (8)

  $${\mathrm{Precision}}_k={\displaystyle \frac{\left(T{P}_k\right)}{T{P}_k+F{P}_k}}$$

  (9)

  $${\mathrm{Recall}}_k={\displaystyle \frac{\left(T{P}_k\right)}{T{P}_k+F{N}_k}}$$

  (10)

  $$F{1}_k={\displaystyle \frac{\left(2\cdot {\mathrm{Precision}}_k\cdot {\mathrm{Recall}}_k\right)}{{\mathrm{Precision}}_k+{\mathrm{Recall}}_k}}$$

  (11)
• Grad-Cam Equation

  $${\alpha }_k^c=1/Z{\sum }_i{\sum }_j\left(\partial y^c\right)/\left(\partial A_{\left(ij\right)}^k\right)$$

  (12)

  $$L_{\mathrm{Grad}-\mathrm{CAM}}^c=\mathrm{ReLU}\left({\sum }_k{\alpha }_k^c{A}^k\right)$$

  (13)
• SHAP Feature Decomposition

  $$\widehat{y}={\varphi }_0+{\sum }_{j=1}^d{\varphi }_j$$

  (14)

7. Architecture Diagram

Table 1 presents the performance results of individual pre-trained CNNs on the LoHi-WELD dataset. The EfficientNetB0 model achieved the best classification accuracy of 82.33% and the lowest loss value of 17.67, while also requiring the fewest parameters among all baseline models, implying its ability to capture fine-grained weld features effectively. VGG19 followed with an accuracy of 82.15% and a loss of 17.85. In contrast, ResNet50V2 exhibited a comparatively lower accuracy of 81.24% along with a higher loss of 18.76. Although these models provide a strong baseline, their performance plateau around 82% highlights the necessity of the proposed hybrid framework to further address complex inter-class ambiguities.

Table 1. Model loss and accuracy.

Model	Accuracy %	Loss %
VGG19	82.15	17.85
ResNet50_V2	81.24	18.76
EfficientNetB0	82.33	17.67

8. Experimental Setup

This work lays out the experimental setup for evaluating the proposed hybrid learning framework against conventional deep learning baselines in weld defect classification. The baseline configuration, which stands for a baseline approach, makes use of a regular end-to-end Convolutional Neural Network (CNN) as the core of the system. This method involves the fine-tuning of a pre-trained model, e.g., ResNet50, on the target dataset. The training of the last fully connected layer of the model is executed to perform the classification. Even though this method has been able to achieve relatively high accuracy, it still has considerable drawbacks, among them the fact that it heavily depends on large, balanced datasets and that it is inherently non-transparent, operating as a kind of “black box”, which is not appropriate for industrial auditing of a high-stakes nature. By means of a modular design for data augmentation, high-performance classification, and total interpretability, the hybrid framework, which is a sophisticated approach, aims to overcome these weaknesses. First, a Generative Adversarial Network (GAN) is used to address the problem of data scarcity and class imbalance, which are usually the main challenges of defect datasets. The GAN learns to produce the most realistic synthetic radiographic images of the under-represented defect classes, which are then used for the expansion of the training dataset. The framework’s centerpiece is a pre-trained CNN, e.g., ResNet50 or VGG16, implemented in PyTorch 2.11.0 or TensorFlow/Keras 2.21.0. This network serves only a feature extractor, as its final classification layers are discarded. It goes through the augmented image dataset to create high-dimensional feature vectors that represent intricate spatial patterns. These feature vectors serve as input to an XGBoost (Extreme Gradient Boosting) classifier, a highly efficient tree-based model, which carries out the final defect classification. This CNN–XGBoost hybrid is a chain in which the CNN part is responsible for feature extraction while the XGBoost model performs the classification. In order to make the system interpretable to the human user, several special Explainable AI (XAI) components are incorporated in the framework. The SHAP (SHapley Additive exPlanations) library is used after training on the XGBoost model. In this way, global feature importance and local, per-prediction explanations are obtained. These explanations identify the visual features on which the CNN attention was focused and which were the most contributive to the given classification. Additionally, LIME (Local Interpretable Model-agnostic Explanations) is used to give visual, model-agnostic justifications of individual predictions, thereby enhancing auditability. All parts of the system are coded in Python 3.13.12, with the model training being accelerated by an NVIDIA T4 Tensor Core GPU. The Scikit-learn 1.8.0 library is employed for the preprocessing of data as well as the computation of evaluation metrics. All computational experiments were carried out on the cloud-based Google Colab platform, using an NVIDIA T4 Tensor Core GPU with 4 GB of VRAM and 8 GB of system RAM. The proposed hybrid framework was developed in Python; the deep learning components (GAN and CNN backbones) were implemented using TensorFlow/Keras, while the Scikit-learn and XGBoost libraries were utilized for the final classification stage. The dataset was split into training, validation, and test sets to allow robust performance evaluation, and the defect class distribution was maintained as described in the preprocessing stage. The CNN feature extractors were trained with the Adam optimizer and an initial learning rate of 0.01 to enable rapid convergence. A batch size of 50 was chosen to maximize GPU throughput without the memory constraints being exceeded. The training time was limited to between 50 and 100 epochs, depending on model convergence; to guard against overfitting, an Early Stopping mechanism was used to monitor the validation loss and stop training if no improvement was recorded for 10 consecutive epochs. The usual Categorical Cross-Entropy loss function was applied for the multiclass classification tasks during the CNN fine-tuning phase. Lastly, the verification of the applicability of the system in real time was performed; thus, the average inference time of around 65 milliseconds per image on the T4 GPU was achieved, which is a confirmation of its suitability for industrial deployment.

9. Evaluation Dataset

The primary dataset for the current study is LoHi-WELD, a novel, large-scale public industrial dataset, which was introduced by [8]. With 3022 real weld bead images captured directly from a Metal Active Gas (MAG) robotic welding industrial process, it stands out as a nearly perfect match for the project. A major feature of the LoHi-WELD dataset is its resolution diversity, which includes both high-resolution (e.g., 2048 × 1080 pixels) and low-resolution images; thus, it represents the different imaging scenarios in industrial settings. Firstly, the dataset is annotated manually and geared towards the classification of four typical defect types: pores, deposits, discontinuities, and stains. With a total of 3022 images, the volume is still inadequate for deep learning models to be trained from scratch, and the dataset may also face the problem of class imbalance (certain defects appear less frequently). The shortage of such data requires the employment of a Generative Adversarial Network (GAN). This GAN, which is the pivotal element of the proposed system, produces a large number of plausible synthetic defect images in accordance with the LoHi-WELD distribution to both equalize and increase the dataset for classifier training. As a means of measurement, only the LoHi-WELD dataset is used in the study. Instead of using different data, the efficiency of our framework is measured through the use of a separate test set of authentic images drawn from the LoHi-WELD dataset. This method guarantees that the model is extensively verified against the particular visual features and fluctuations of the MAG welding process, thus providing an immediate and concentrated measure of industrial environment suitability.

10. Evaluation Metrics

Standard accuracy will not provide a full account of the performance of the model in the testing of welds since the data are imbalanced with predominantly good welds rather than defects. This implies that a naive model, which predicts the absence of a defect, can have high accuracy (e.g., 95 percent) but still fail to identify a defect that can be critical, and this is not useful in real-world quality control. To critically examine the framework, this study prioritizes metrics that are developed specifically for imbalanced data. Accuracy is used to determine the consistency of defect identification and to minimize false alarms, and Recall (Sensitivity) ensures the detection of real defects, which is an undeniable requirement for industrial safety. The F1-Score forms a harmonic mean of Accuracy and Recall to provide a balanced measure of the efficiency of the model. The ability of the model to distinguish between the classes, with a score of 1.0 as ideal and 0.5 as random guessing, is also measured using the Area Under the Receiver Operating Characteristic (AUC-ROC). Finally, to visualize the mistakes of classification in a more intuitive form than that of standard tables, Sankey diagrams are utilized: Every flow band is represents the number of samples transferred between the true and predicted classes, in order to instantly determine any patterns of misclassification.

11. Results and Discussion

This section provides a comparative performance study of baseline CNNs versus our proposed hybrid framework. The evaluation was carried out using the LoHi-WELD dataset. The models were assessed based on Accuracy, Precision, Recall, F1-Score, and the Area Under the Receiver Operating Characteristic (AUC-ROC) curve.

A.

Baseline Model Performance:

The primary phase of our experiment was to determine benchmark performance by testing three diverse pre-trained CNN architectures, i.e., ResNet50, VGG19, and EfficientNetB0. These were trained end-to-end on the weld defect classification task. The outcomes, which are summarized in Table 1, give a clear indication of a performance ceiling for those standalone models. Each of the three architectures converged to a similar classification accuracy that was able to reach around 82% in the best performances. Although per-class metrics analysis and the ensuing Sankey diagrams demonstrated significant issues, the ability of the models to isolate high-level features was clear. The misclassification streams in the baseline were high, and the models confused defect classes that were visually similar, such as the one with the name “cracks” and the one with the name “deposits,” or the one with the name “discontinuities” and the one with the name “stains.” This can be interpreted to mean that, although all CNNs learn a strong feature representation, none of the models was able to individually learn the full and finer visual information needed to resolve such complex ambiguities.

Proposed Hybrid Framework Performance:We conjecture that the feature representations learned by the individual models are not redundant but rather complementary, which is why they have limited performance. To verify this conjecture, we created a hybrid, two-stage framework. Therefore, the three CNNs were incorporated as parallel feature extractors in this model, i.e., ResNet50, VGG19, and EfficientNetB0. From each network, three high-dimensional feature vectors were taken from the next-to-last layers and were then concatenated to form a single feature vector of the highest dimension. This concatenated vector, representing diverse hierarchical features learned by all three architectures, was used as the input for training an XGBoost classifier. This indeed led to more conclusive results through the hybrid approach. With that, the CNN–XGBoost framework proposed achieved a classification accuracy of 92.15%. This represents a 10-point increase in accuracy over the best baseline model, indicating a major improvement in overall predictive performance. The qualitative improvement can also be illustrated by the Sankey diagrams shown in Figure 2, Figure 3, Figure 4 and Figure 5. The Sankey diagram of the proposed hybrid model (Figure 5) has very streamlined flows and little divergence. The prevailing streams relate true labels with their corresponding prediction in such a way that visually confirms that the level of inter-class confusion has been minimized. The corresponding changes in the macro-averaged F1-score (0.81 to 0.92) and AUC-ROC support this finding. The training and validation accuracy and loss trends over the epochs for the baseline models—ResNet50, VGG19, and EfficientNetB0—are presented in Figure 6, Figure 7 and Figure 8, respectively.

Figure 2. ResNet50 Sankey diagram.

Figure 3. VGG19 Sankey diagram.

Figure 4. EfficientNetB0 Sankey diagram.

Figure 5. Hybrid model Sankey diagram.

Figure 6. ResNet50 accuracy graph.

Figure 7. VGG19 accuracy graph.

Figure 8. EfficientNetB0 accuracy graph.

12. Limitations

Although the proposed hybrid framework has proven to be more effective than standalone deep learning models, its performance also has certain limitations that should be acknowledged. The first weakness is that it has even greater computational complexity: Feature extraction that runs three parallel CNN backbones (ResNet50, VGG19, and EfficientNetB0) requires much more memory and computing power on the GPU than single-model architectures. Some industrial applications may find it difficult to deploy this resource-intensive system on low-power edge devices. Moreover, the model presupposes the use of GAN-generated synthetic data, meaning that there might be a source of bias; although DCGAN is effective in balancing the dataset, when the synthetic images fail to completely capture the stochastic variations of real-world defects, there is a risk that the classifier will overfit to the synthetic data, which may have a minor impact on overall generalization in highly novel imaging settings.

13. Conclusions

This paper demonstrates a robust hybrid deep learning framework that is capable of dealing with the challenges of automated weld defect classification in industrial environments. The proposed system, by combining deep Convolutional Neural Networks (CNN) for feature extraction with the XGBoost ensemble algorithm for classification, is able to efficiently surpass the limitations of performance that are frequently encountered with monolithic deep learning models. The main contribution of this paper is the separation of the feature learning process from the final decision-making stage. The experiments on the LoHi-WELD dataset show that classification performance improves significantly when the fully connected dense layers of a traditional CNN are replaced with a gradient-boosted decision tree. Comparison of this work with recent state-of-the-art methods indicates that the hybrid approach has better accuracy, precision, and recall than the use of CNN alone (e.g., standard VGG19 or ResNet50 implementations). The XGBoost classifier was effective at handling the high-dimensional feature vectors generated by the backbone, thereby making the model not only more accurate but also computationally efficient for discriminating the defect classes that are closely related, such as cracks and discontinuities. In essence, this research provides evidence that a hybrid CNN–XGBoost model can be a better choice than traditional end-to-end deep learning NDT methods. The results reveal that the interaction between deep spatial feature extraction and ensemble learning provides the accuracy and precision needed for deployment in automated manufacturing quality control systems in real-world settings. This paper introduces a hybrid solution to the ongoing issues of class imbalance and insufficient interpretability within automated weld defect detection, comprising GAN, Multi-CNN feature fusion, and XGBoost. With the application of DCGAN to targeted data augmentation and the fusion of hierarchical features of different architectures, the proposed system was quite effective in resolving the ambiguities between visually similar defects. Experimental findings on the LoHi-WELD dataset showed that the hybrid model had a classification accuracy of 92.15%, which significantly outperformed the typical transfer learning baselines that stagnated at about 82%. The fact that the F1-Score of the macro-averaged scores increased considerably (0.81 to 0.92) and produced a very skewed confusion matrix, which was very diagonalized, further confirm the effectiveness of this approach alongside a substantial decrease in false positives. In addition to quantitative measures, the combination of XGBoost and explainable AI modules ensures that the system is not a black box, providing transparent and interpretable results, which are essential in safety-critical manufacturing settings. Future research will be devoted to enhancing the level of computational efficiency of the hybrid feature extractor to support real-time execution on resource-constrained hardware.