Classification and Contour Recognition of Welding Defects in Magneto-Optical Images

Abstract

In the field of magneto-optical imaging nondestructive testing for welding defects, multi-angle detection of welding defects has already been achieved. However, research on automatic defect recognition and contour extraction remains insufficient. Therefore, to enable automatic detection of welding defects using magneto-optical imaging technology, it is essential to address the key issues of defect recognition and contour extraction in magneto-optical images. The dataset in this article includes five types of images: defect-free, lack-of-fusion, cracks, pits, and Weld reinforcement. Firstly, the Mask R-CNN detection method is used to perform defect recognition and contour segmentation on the original magneto-optical image dataset. The detection results indicate that the recognition rate of lack-of-fusion and Weld reinforcement in the original magneto-optical image is not high, and the recognition accuracy of pits and cracks is extremely low. Subsequently, the magneto-optical image dataset was preprocessed using the differential level set method, and the mask R-CNN algorithm was used to identify defect types and segment defect contours. Comparing the results of two experiments, it was found that the detection accuracy of the preprocessed dataset was higher, and the overall recognition accuracy increased by $30\%$.

1. Introduction

With the rapid advancement of science, technology, and the national economy, the manufacturing industry has entered a phase of accelerated development [1]. Welding technology plays a crucial role in modern manufacturing, especially in high-precision assembly and electronic component manufacturing [2]. Being one of the key technologies in manufacturing, welding is involved in nearly every field of the national economy, including the automotive industry, aerospace, shipbuilding, and machinery manufacturing [1,3]. Based on differences in heating levels and process characteristics during welding, welding methods can be categorized into three main types: (1) fusion welding–including gas welding, arc welding, electroslag welding, plasma arc welding, electron beam welding, and laser welding; (2) pressure welding–including resistance welding, friction welding [4,5], cold pressure welding, diffusion welding, and explosion welding; (3) brazing. Welded structures often constitute significant industrial assets and are also characterized by their environmental friendliness.

Statistics indicate that during the 1990s, failures in welds and solder joints led to a $5\%$ decline in the Gross National Product (GNP) of the United States. Therefore, accelerating the research and development of effective non-destructive testing (NDT) technologies for welded products is a critical challenge that must be overcome to ensure the safety and orderliness of social production and daily life. Major catastrophic accidents caused by welding defects often result from internal volumetric defects within the weld that were not fully detected. For example, the presence of porosity in a weld can compromise the density of the welded product, subsequently reducing its strength and toughness. Microscopic cracks within the weld can lead to stress concentration; once the component is in service, uneven stress distribution can cause these cracks to propagate, eventually forming macroscopic cracks and potentially leading to the direct fracture of the welded product. Consequently, defect detection is a vital step for ensuring the quality of welded products and engineering projects. An effective NDT method for welding defects is paramount to guaranteeing the quality of welded products [6]. Beyond visual inspection of surface defects (such as cracks, undercut, concavity, lack of fusion, and incomplete penetration), it is essential to employ non-destructive testing techniques to detect internal weld defects (such as cracks and porosity) that are not visible to the naked eye [6]. The quality of a welded product is generally assessed based on the type, size, quantity, distribution, and impact of any defects present. The tolerance for defects in a welded product depends on its intended application: products intended for high-stakes applications may permit no defects whatsoever, while others may allow certain non-critical defects, and some products may even be repairable through rewelding. Failure to effectively monitor and inspect the quality of welded components, and to promptly repair or even reject substandard itemsdual line interpolation can significantly impact a company’s production efficiency, severely undermine its economic performance, and, in more serious cases, impede broader social development.

Conventional non-destructive testing techniques include visual inspection, penetrant testing, structured light testing, magnetic flux leakage testing, magnetic particle testing, radiographic testing, eddy current testing, ultrasonic testing, and potential difference measurement [7,8]. Among these, magnetic particle testing [9], penetrant testing [10], potential difference measurement [11], eddy current testing [12], and structured light testing [13] are limited to detecting surface and near-surface defects, while radiographic [14,15] and ultrasonic testing [16] offer a broader detection range. Within the field of non-destructive testing (NDT), magneto-optical imaging (MOI) represents a relatively new domain. In 1993, Fitzpatrick et al. designed one of the earliest magneto-optical imaging devices, which produced magneto-optical (MO) images capable of revealing defect contours [17,18]. This team also investigated the feasibility of applying MOI technology to aircraft defect detection [18]. Since then, researchers have focused on improving the imaging performance of MOI sensors through both hardware and software enhancements, while continuously expanding the application domains of MOI-based NDT.

Welding defect magnetic-optical imaging detection is a relatively new non-destructive testing method, and its research and application are still under continuous exploration [19].

In 2014, scholars began exploring the application of MOI technology in the welding field, including weld seam tracking and weld defect detection [20]. In 2017, Chen Yuquan applied Kalman Filter (KF) techniques and Radial Basis Function (RBF) neural networks to weld detection using magneto-optical sensors, achieving tracking of the weld center position [21]. Also in 2017, research commenced on the classification of magneto-optical images of welding defects. Researchers such as Lan Chongzhou conducted MOI experiments utilizing both permanent magnet excitation and AC-driven electromagnets, validating the feasibility of using MOI systems for weld defect detection [22]. Concurrently, they proposed a pattern recognition method comprising three steps. This method was capable of classifying typical welding conditions, such as incomplete penetration, overfill (weld reinforcement), cracks, and defect-free states, achieving a classification accuracy of $93.5\%$ [22].

In 2019, Zheng Qiaoqiao utilized Principal Component Analysis (PCA) to extract and reduce the dimensionality of grayscale features from image column pixels. By combining this with an AdaBoost algorithm integrated with a Back-Propagation (BP) neural network, she established a BP-AdaBoost classification model for welding defects. In 2021, Xiang He proposed a novel method for automatic defect detection and classification in low-carbon steel Wire Arc Additive Manufacturing (WAAM) products, employing improved residual magnetism/magneto-optical imaging and a cost-sensitive Convolutional Neural Network (CNN) [23]. A CNN model was deployed to detect defects in MO images. The prediction results obtained from the enhanced image dataset exceeded those from the original dataset by over $7\%$, thereby validating the effectiveness of the image enhancement technique based on optimal light intensity thresholding.

In 2024, Li Yanfeng’s research demonstrated that the classification accuracy of a combined GLCM (Gray-Level Co-occurrence Matrix) + Tamura-BP model surpassed that of a GLCM-BP model, achieving an overall classification accuracy of $91.1\%$. This model proved effective and accurate in classifying naturally invisible weld defects. In 2025, the same scholar employed PCA to extract grayscale features from fused image column pixels and utilized GLCM to extract textural features from MO images [24]. A BP-AdaBoost neural network model and a Support Vector Machine (SVM) model were subsequently developed to identify these defect features. Experimental results indicated that, under rotating magnetic field excitation, the classification accuracies of the BP-AdaBoost and SVM models reached $98.2\%$ and $98.6\%$, respectively.

Also in 2025, Guohua He introduced a multi-modal fusion classification model termed the Attention-Enhanced Trustworthy Multimodal Classification model based on Fusion of Convolutional Neural Network and Vision Transformer (AETMC-FCVT), which is an end-to-end deep learning framework [25]. The trained AETMC-FCVT model demonstrated effectiveness and robustness in identifying four typical welding states in resistance welding joints. Evaluation metrics for the four-class classification model exceeded $96\%$, while those for the two-class classification model surpassed $99\%$. Furthermore, results from ablation studies and comparative experiments confirmed that the AETMC-FCVT model yielded significant performance improvements.

The identification and classification of welding defects using magneto-optical imaging detection are particularly important [24]. At present, there are some algorithms based on Kalman filtering, support vector machine, etc., for magneto-optical image processing to achieve weld seam tracking and welding defect classification [21,24]. However, there has been no research on contour extraction in magneto-optical imaging of welding defects. This paper employs the Mask R-CNN object detection method to perform object detection on both the original magneto-optical image dataset and the preprocessed magneto-optical image dataset (which includes five categories of images: defect-free, lack of fusion, cracks, pits, and Weld reinforcements). The method identifies the types of defects and segments their contours. The detection results demonstrate that the accuracy rates for pits and cracks in the original magneto-optical images are relatively low, while the preprocessed dataset yields higher detection accuracy. Therefore, the research findings of this paper will have broad application prospects.

2. Mask R-CNN Detection of Original Magneto-Optical Images of Welding Defects

2.1. Detection Model of Mask R-CNN

The Mask R-CNN algorithm is employed for instance segmentation of magneto-optical images of welding defects, aiming to both identify the types of welding defects and extract their contours. The Mask R-CNN algorithm primarily builds upon the structure of Faster R-CNN, utilizing the ResNet-FPN (Feature Pyramid Network) architecture for feature extraction, and adds a Mask branch for predictive segmentation [26,27,28]. Backbone version is ResNet 101. The main components of the Faster R-CNN and Mask R-CNN algorithms are illustrated in Figure 1. The framework of Faster R-CNN is shown by the black parts in Figure 1, and the combination with the red parts (modifications based on the Faster R-CNN framework) constitutes the Mask R-CNN algorithm. Firstly, the RoI Pooling layer in the Faster R-CNN framework is replaced with the RoIAlign layer. Then, a parallel FCN layer (mask layer) is added.

This structured approach ensures accurate detection and segmentation of welding defects in magneto-optical images. The ResNet-FPN architecture consists of three main components: a bottom-up pathway, a top-down pathway, and lateral connections. The key elements of the ResNet-FPN algorithm are illustrated in Figure 2. This pyramid feature structure enables the fusion of features from different levels of the magneto-optical images, ensuring that the feature maps possess both strong semantic information and detailed spatial information.

As shown in Figure 2, ResNet-FPN is a typical encoder-decoder structure. The leftmost part of the structure consists of the “bottom-up” sampling layers (Bottom-up Layers), the middle part comprises the “top-down” sampling layers (Top-down Layers), and the rightmost part integrates the sampling layers from different depths to ultimately produce multi-scale feature layers. For example, if the size of the original magneto-optical image is $400\times 400\times 3$, it is first resized to $512\times 512\times 3$, which serves as the input to the ResNet-FPN, denoted as $512\times 512\times 3$. In Figure 2, the left side uses ResNet as the backbone network, dividing the feature maps into 5 layers based on their sizes, with each layer having a stride of 2, labeled as C1 to C5. Consequently, the resolution of each layer is halved, resulting in downsampling rates of $\left[2,4,8,16,32\right]$. Thus, the resolutions of the layers are $\left[256,128,64,32,16\right]$, and the number of channels for the magneto-optical image is 3.

The recognition algorithm requires the extraction of welding defect contours from magneto-optical images. If the RoI pooling method from Faster R-CNN is used for feature extraction, the rounding operation introduced during the process can lead to misalignment between the extracted feature maps and the original magneto-optical images. This misalignment can negatively impact the accuracy of defect contour detection. To address this issue, the RoIAlign method can be employed to replace RoI pooling. RoIAlign uses bilinear interpolation to calculate pixel values, eliminating the rounding operation and thereby preserving the approximate spatial location of the defect contours. This approach enhances the precision of defect contour detection in the recognition process.

The use of bilinear interpolation to calculate the output coordinates of RoIAlign can effectively improve the accuracy of defect contour extraction in magneto-optical image welding defect detection. The calculation process is shown in Figure 3. The dashed line in Figure 3 represents the feature map of the magneto-optical image, and the solid line represents the ROI, which is set to $2\times 2$ sized cells. If the number of sampling points for each cell is set to 4 and the cell is divided into four equal parts, then the center point of each part is the sampling point. The points in Figure 3 represent the sampling points of each cell. From Figure 3, it can be seen that the coordinates of these sampling points are floating-point numbers, and bilinear interpolation (as indicated by the four arrows) is required to obtain the pixel values of the sampling points. Then, max pooling is performed on the four sampling points within each cell to obtain the final RoIAlign result.

2.2. Mask R-CNN Detection

Under the current magneto-optical imaging detection conditions, defect-free magneto-optical image samples were obtained under different magnetic poles and magnetic field intensities, while magneto-optical images of cracks, pits, lack of fusion, and weld reinforcements were acquired under various magnetic field strengths and excitation fields (constant, alternating, rotating, and combined) [24]. The dataset comprises 180 defect-free images, 655 crack images, 398 pit images, 727 lack-of-fusion images, and 267 weld reinforcement images. Representative magneto-optical images are shown in Figure 4.

The magneto-optical image dataset was divided into training and testing sets at a ratio of 7:3, with the maximum number of iterations set to 100,000. The test results were evaluated at various iteration intervals (5000; 10,000; 15,000; 20,000; 25,000; 30,000; 35,000; and 40,000). Comparative analysis revealed that the optimal detection performance was achieved at 30,000 iterations. Consequently, all subsequent analyses in this study are based on the Mask R-CNN detection results obtained at this iteration count. The input resolution was set to $400\times 400$. Standard data augmentation techniques, including random horizontal flipping, random scaling, random cropping, and random color jittering, were adopted to reduce overfitting. The batch size was taken to be nine, and it was trained using three NVIDIA TITAN Xp graphics processing units (GPUs). The network was trained for 30,000 iterations with an initial learning rate of $2.5\times {10}^{-4}$, which was reduced to $2.5\times {10}^{-5}$ during the final 10000 iterations. The average operation time per image was 97.7 ms on a TITAN Xp GPU.

Figure 5 lists the segmentation results of the Mask R-CNN model for some magnetic optical images. From Figure 5, it can be seen that in the test results of the Mask R-CNN model, the shapes of the defects are segmented on the magnetic optical images, and a rectangular box is used to identify the type of the defect. The confidence (degree of certainty) of the defect category is given in the upper left corner of the rectangular box. From Figure 5, it can be observed that the defect contours segmented by the model are similar to the actual defect contours, but the classification results for some defects are chaotic (different colored rectangular boxes represent different classification results, and different colored arcs represent different extracted defect contours). For example, in the first image of the crack, it is classified as a pit, and the confidence is 1. This will lead to disorder in the classification results. A critical limitation stems from the inherent intensity gradient variations in magneto-optical images, which introduce significant challenges for precise visual boundary demarcation during manual annotation. This optical artifact consequently compromises the accuracy of human-drawn defect contours. As exemplified in Figure 5, the weld reinforcement images exhibit pronounced bimodal intensity distribution (bright-dark contrast) along the defect midline, substantially impeding reliable visual boundary assessment. Notably, since Mask R-CNN’s detection performance is contingent upon training label precision, these annotation inaccuracies propagate directly into the model’s predictive reliability. This observation underscores the necessity for advanced annotation protocols or computational preprocessing to mitigate optical interference in magneto-optical defect characterization.

2.3. Evaluation of Mask R-CNN Model Detection Results

In the field of object detection, the most important evaluation criteria are the average accuracy, which mainly involves Intersection over Union (IoU), True Positive (TP), False Positive (FP), False Negative (FN), Precision, Recall, and Confidence. an Intersection over Union IoU is used to measure the degree of overlap between two different rectangular boxes, and its calculation formula is as follows:

$$\begin{array}{c}\hfill {IoU}_{AB}={\displaystyle \frac{A\cap B}{A\cup B}}\end{array}$$

(1)

A and B in the formula represent rectangular boxes. In object detection, for all bounding boxes of the same category in the same image, if the predicted bounding box A and a bounding box B in the label are greater than a specified threshold, then bounding box A is considered a correct prediction, otherwise it is an incorrect prediction. $TP$ refers to the number of correctly predicted bounding boxes. $FP$ refers to the number of prediction errors in the predicted bounding box. $FN$ refers to the number of bounding boxes in the label that have not been predicted.

Precision refers to the percentage of correctly recognized objects A to the total number of recognized objects n, that is, the probability of predicting correctly in a positive case, expressed by the following equation:

$$\begin{array}{c}\hfill Precision={\displaystyle \frac{TP}{(TP+FP)}}\end{array}$$

(2)

Recall refers to the percentage of correctly identified objects A to the total number of objects A in the test set, which is the probability of correctly predicting all positive cases:

$$\begin{array}{c}\hfill Recall={\displaystyle \frac{TP}{(TP+FN)}}\end{array}$$

(3)

The average precision $AP$ represents the area enclosed by the precision recall curve and the coordinate axis, and is calculated using the following formula:

$$\begin{array}{c}\hfill AP={\int }_0^{1}P\left(r\right)dr\end{array}$$

(4)

In the formula, r represents Recall and $P\left(r\right)$ represents Precision. $AP$ is for a single category specific intersection ratio threshold. For all categories, calculate the $AP$ for each intersection to union ratio threshold and take the average to obtain the average accuracy mean mAP. As the most important indicator for measuring the performance of object detection algorithms, the range of $mAP$ values is [0,1], and the larger the better.

The average recall rate(AR) of rectangular box and contour segmentation under different IoU parameters in the original magneto-optical image dataset is shown in

Table 1. AR of bbox and segm under different IOU parameters of original magneto-optical images.

	IoU	mD	LF	WR	Crack	Pit	Total
RB	0.50:0.95	1	0.483	0.258	0.294	0.096	0.345
	0.50:0.95	10	0.505	0.474	0.299	0.128	0.394
	0.50:0.95	100	0.505	0.474	0.299	0.128	0.394
CS	0.50:0.95	1	0.502	0.214	0.13	0.058	0.321
	0.50:0.95	10	0.522	0.384	0.132	0.095	0.362
	0.50:0.95	100	0.522	0.384	0.132	0.095	0.362

. In Table 1, RB represents “Rectangular box”, CS represents “Contour segmentation”, mD represents “maxDets”, LF represents “Lack-of-fusion”, and WR represents “Weld reinforcement”. According to Table 1, when IoU is 0.5, the total AR value for rectangular box and contour segmentation is optimal. At this point, the probability of predicting correctly in all positive cases of the rectangular box is $39.4\%$, and the probability of predicting correctly in all positive cases of contour segmentation is $36.2\%$. The classification test found that the AR values of rectangular boxes and contour segmentation for pits and cracks were lower than the total AR value: the probability of correct prediction in all positive cases of crack rectangular boxes was $29.9\%$, and the probability of correct prediction in all positive cases of contour segmentation was $13.2\%$; the probability of correct prediction in all positive cases of concave rectangular boxes is $12.8\%$, and the probability of correct prediction in all positive cases of contour segmentation is $9.5\%$. When IoU is at other values, the AR values of rectangular boxes and contour segmentation show the same situation: the AR values of rectangular boxes and contour segmentation for pits and cracks are both lower.

The mAP and AP for rectangular box and contour segmentation under different IoU parameters in the original magneto-optical image dataset are shown in

Table 2. AP of bbox and segm under different IOU parameters of original magneto-optical images.

	AP	LF	WR	Crack	Pit	Total
RB	$mAP$	0.441	0.336	0.1808	0.0738	0.2762
	$A{P}_{50}$	0.756	0.7064	0.5126	0.1504	0.5203
	$A{P}_{75}$	0.495	0.3211	0.0814	0.0628	0.2797
CS	$mAP$	0.491	0.2857	0.066	0.0454	0.2688
	$A{P}_{50}$	0.7439	0.6764	0.218	0.1383	0.5192
	$A{P}_{75}$	0.5866	0.2483	0.013	0.0206	0.2497

. According to Table 2, the average AP value of the rectangular box and contour segmentation calculated when an Intersection over Union is 0.5 is optimal: the probability of correctly predicting the rectangular box in the positive case is $52.03\%$, and the probability of correctly predicting the contour segmentation in the positive case is $51.92\%$. The classification test found that the average AP values of the rectangular box and contour segmentation of the pit were both low, and the probability of correctly predicting the rectangular box as a positive example was $15.04\%$. The probability of correctly predicting contour segmentation in positive cases is $13.83\%$. When IoU is set to other values, the AP values for the classification test rectangular box and contour segmentation show the same situation: the average AP values for the concave rectangular box and contour segmentation are both lower. As the most important indicator for measuring the performance of object detection algorithms, mAP has a test value of $27.62\%$ for rectangular boxes and $26.88\%$ for contour segmentation. This indicates that the performance of the Mask R-CNN model is poor in the current situation. The classification test found that the mAP values for rectangular boxes and contour segmentation of pits and cracks were lower than the total mAP value: the mAP probability for rectangular boxes with cracks was $18.08\%$, and the mAP probability for contour segmentation was $6.60\%$; the mAP probability of concave rectangular frame is $7.38\%$, and the mAP probability of contour segmentation is $4.54\%$.

3. Preprocessing of Magneto-Optical Images of Welding Defects

As shown in Figure 4, the brightness of defect-free magneto-optical (MO) images varies under different excitation conditions. From Figure 4, it can be observed that the brightness distribution in MO images of radiation-type defects significantly differs from that of seam weld defects, which is closely related to the relative positioning between the magnetic poles and the defects. Figure 4 reveals that in MO images of pits, the upper contour appears dark while the lower contour is bright, and the left contour is dark whereas the right contour is bright. According to Figure 4, the lack-of-fusion defects exhibit a noticeable brightness contrast on both sides of the defect, with the MO image reaching saturation. Figure 4 demonstrates that in the MO images of Weld reinforcements, the upper contour of the defect is dark, and the lower contour is bright. During the detection process, the relative distribution between the magnetic poles and defects is highly random, suggesting that these brightness variations may play a critical role in the accuracy of defect detection.

As illustrated in Section 2.2, when annotating the contours of defects in MO images, challenges such as the indistinct boundaries of cracks and the inaccurate delineation of pits lead to imprecise labeling. The accuracy of annotations is also likely to significantly influence the performance of defect detection. Section 2.3 indicates an imbalanced distribution of defect types in the MO image dataset: the number of crack and lack-of-fusion images is twice that of pit and Weld reinforcement images, while defect-free images are the least represented. Such dataset imbalances may further degrade detection performance. Therefore, we hypothesize that the suboptimal model performance stems from three primary factors: (1) inaccurate image annotations, (2) brightness variations in MO images under different magnetic fields, and (3) dataset imbalance.

As reported in reference [29], the pixel value distribution in the R-channel of magneto-optical (MO) images exhibits a higher dynamic range, making it prone to saturation and consequent loss of essential defect information. In contrast, the G-channel and B-channel demonstrate lower pixel value distributions with reduced saturation tendencies. However, the B-channel data shows significantly lower signal-to-noise ratio (SNR) compared to the G-channel.

Since the leakage magnetic field intensity reaches extreme values at welding defect edges, we performed central difference operations on the G-channel signals of MO images along both x- and y-directions to enhance boundary detection. This process generated images with more pronounced contour boundaries. Subsequently, the differential images from both directions were weighted and fused, producing a composite differential image with improved grayscale uniformity. As illustrated in Figure 6, Figure 6a presents the original MO image of the defect, Figure 6b displays the x-direction central difference result, Figure 6c shows the y-direction central difference result, and Figure 6d demonstrates the final fused differential image.

Due to the presence of substantial noise in the images, which significantly interferes with further extraction of defect contours, it is necessary to perform filtering operations on the differential-processed images. Among common frequency-domain low-pass filters—including the ideal low-pass filter, Gaussian low-pass filter, and Butterworth filter—the Gaussian low-pass filter was employed to filter the differential fusion image, as demonstrated in Figure 7.

A comparison between Figure 6a and Figure 7b demonstrates that the processed image, after differential fusion and filtering, exhibits significantly enhanced defect contours compared to the original magneto-optical (MO) image. This improvement effectively addresses the inaccuracy issues in MO image annotation. Furthermore, comparative analysis between the original MO images and the filtered differential fusion images reveals that the proposed processing method, incorporating differential operations and filtering, can substantially mitigate the impact of illumination variations caused by different magnetic field intensities on detection performance. This advancement significantly facilitates the contour annotation of welding defects. Representative examples comparing original MO images with differential fusion images are presented in Figure 8.

As illustrated in Figure 8, the welding defect contours become markedly clearer after implementing differential operations and Gaussian low-pass filtering. For instance, while the first and last crack images in the original MO images merely indicate the presence of defects, their boundaries remain ambiguous. In contrast, the differential fusion-filtered images distinctly reveal the precise contour locations. Similarly, when comparing the original MO images of lack-of-fusion defects with their processed counterparts, the filtered differential fusion images effectively eliminate the confounding effects of brightness variations, resulting in sharper defect delineation. These findings confirm that differential fusion-filtered images provide superior substrates for contour annotation.

4. Detection of Welding Defects in Preprocessed Magneto-Optical Images Using Mask R-CNN

The workflow of the welding defect detection model based on Mask R-CNN for magneto-optical (MO) images is shown in Figure 9. The processing pipeline consists of four main stages. First, the original MO image is decomposed into its constituent R, G, and B channel components. The G-channel image is then selected for subsequent processing due to its optimal signal-to-noise characteristics. Next, a central differential fusion operation is applied to enhance defect boundaries in the G-channel image. The fused image undergoes Gaussian low-pass filtering to suppress noise while preserving critical edge information. The filtered output is then used for precise contour annotation. Finally, the preprocessed and annotated images are fed into the Mask R-CNN network for automated defect recognition and segmentation.

4.1. Mask R-CNN Detection of Preprocessed Magneto-Optical Images

As specified in Section 2.2, the magneto-optical (MO) image dataset was divided into training and testing sets with a ratio of 7:3, and the model was trained for 30,000 iterations. Representative segmentation results of preprocessed MO images are presented in Figure 10.

In Figure 10a, the first three images successfully segmented crack contours with a confidence score of 1. The fourth image yielded two detection results: a pit with confidence 0.69 and a crack with confidence 0.72, with the latter being ultimately classified as the correct detection. Figure 10b demonstrates accurate segmentation of pit contours in all images (confidence = 1), with the fourth image containing two objects: a pit (confidence = 1) and a crack (confidence = 0.95), confirming the model’s capability to identify multiple defects within a single image. All lack-of-fusion defects were correctly segmented in Figure 10c, although the fourth image misclassified one instance as a crack (confidence = 0.85 versus the correct classification confidence of 1). Similarly, Figure 10d shows perfect segmentation of Weld reinforcements, despite one instance in the fourth image being misidentified as a crack (confidence = 0.81 versus the correct confidence of 1).

As evidenced in Figure 10, the proposed processing pipeline enables Mask R-CNN to accurately identify defect types and segment their contours, even in images containing multiple defect classes. However, the results reveal a systematic tendency for pits, lack-of-fusion, and Weld reinforcements to be occasionally misclassified as cracks. This phenomenon stems from two fundamental characteristics of crack detection in MO imaging: (1) the inherently weak and unpredictable leakage magnetic fields associated with cracks make them particularly challenging imaging targets, and (2) variations in sensor lift-off distance during inspection can produce diverse crack morphologies and leakage field characteristics, leading to similarities between crack signatures and other defect types in MO images. While these factors contribute to a certain percentage of misclassifications, the contour segmentation accuracy remains consistently high throughout all defect categories.

4.2. Analysis of Mask R-CNN Detection Results for Preprocessed Magneto-Optical Images

The average recall rate AR of rectangular box and contour segmentation under different IoU parameters in the preprocessed magneto-optical image dataset is shown in

Table 3. AR of bbox and segm under different IOU parameters.

-	IoU	mD	LF	WR	Crack	Pit	Total
RB	0.50:0.95	1	0.720	0.434	0.733	0.447	0.589
	0.50:0.95	10	0.720	0.691	0.733	0.551	0.689
	0.50:0.95	100	0.720	0.691	0.733	0.551	0.689
CS	0.50:0.95	1	0.637	0.391	0.563	0.429	0.529
	0.50:0.95	10	0.637	0.627	0.563	0.526	0.624
	0.50:0.95	100	0.637	0.627	0.563	0.526	0.624

. According to Table 3, when IoU is 0.50:0.95 and maxDets is 10 or 100, the total AR value for rectangular box and contour segmentation is optimal. At this point, the overall prediction accuracy of the rectangular box is $68.9\%$, which is $29.5\%$ higher than the detection results of the original Mask R-CNN model ($39.4\%$ to $68.9\%$); the overall prediction accuracy of contour segmentation is $62.4\%$, which is $26.2\%$ higher than the detection results of the original model ($36.2\%$ to $62.4\%$). The classification test found that the AR values of rectangular boxes and contour segmentation for pits and cracks increased by more than $40\%$: the prediction accuracy of crack rectangular boxes was $73.3\%$ ($29.9\%$ to $73.3\%$), and the prediction accuracy of contour segmentation was $56.3\%$ ($13.2\%$ to $56.3\%$); the prediction accuracy of concave rectangular boxes is $55.1\%$ ($12.8\%$ to $55.1\%$), and the prediction accuracy of contour segmentation is $52.6\%$ ($9.5\%$ to $52.6\%$). During classification testing, the AR values of defects in other categories showed varying degrees of improvement. Under other parameters, the AR values for rectangular box and contour segmentation showed the same situation. The results show that the preprocessed magneto-optical image effectively overcomes the influence of brightness changes on image detection.

The mAP and AP for rectangular box and contour segmentation under different IoU parameters in the original magneto-optical image dataset are shown in

Table 4. AP of bbox and segm under different IOU parameters.

	AP	LF	WR	Crack	Pit	Total
RB	$mAP$	0.6886	0.5982	0.6605	0.4776	0.6077
	$A{P}_{50}$	0.8240	0.9162	0.9592	0.7932	0.8710
	$A{P}_{75}$	0.7907	0.6437	0.7421	0.4670	0.6723
CS	$mAP$	0.6160	0.5426	0.4737	0.4541	0.5470
	$A{P}_{50}$	0.8245	0.9042	0.9345	0.7808	0.8592
	$A{P}_{75}$	0.7843	0.5950	0.4569	0.4506	0.6277

.

According to Table 4, the average AP value of rectangular box and contour segmentation calculated when an Intersection over Union is 0.5 is optimal: the overall prediction accuracy of rectangular box is $87.10\%$, which is $35.07\%$ higher than the detection result of the original Mask R-CNN model ($52.03\%$ to $87.10\%$); the overall prediction accuracy of contour segmentation is $85.92\%$, which is $34\%$ higher than the detection results of the original model ($51.92\%$ to $85.92\%$). The classification test found that the average AP values for rectangular boxes and contour segmentation of pits were both below $80\%$: the prediction accuracy of rectangular boxes was $79.32\%$, and the probability of correct contour segmentation prediction was $78.08\%$. And the average AP values for rectangular boxes and contour segmentation in other classes are all above $80\%$. When IoU is other values, the AP values of the rectangular box and contour segmentation in the classification test show the same situation: the average AP values of the rectangular box and contour segmentation for pits are lower than those of the other classes.

As the most important indicator for measuring the performance of object detection algorithms, mAP, The test value of the rectangular box is $60.77\%$, which is $33.15\%$ higher than the detection result of the original Mask R-CNN model ($27.62\%$ to $60.77\%$); the test value for contour segmentation is $54.70\%$, which is $27.82\%$ higher than the detection result of the original model ($26.88\%$ to $54.70\%$). This indicates a significant improvement in the performance of the Mask R-CNN model in the current situation. Classification testing found that the mAP values for both concave rectangular boxes and contour segmentation were lower than the total mAP value: the mAP value for rectangular boxes was $47.76\%$, which was $40.38\%$ higher than the detection results of the original Mask R-CNN model ($7.38\%$ to $47.76\%$); the mAP for contour segmentation is $45.41\%$, which is $40.87\%$ higher than the detection results of the original model ($4.54\%$ to $45.41\%$). Although the mAP value of crack contour segmentation is lower than the total mAP value by $54.70\%$, which is only $47.37\%$, it still improves by $40.77\%$ ($6.6\%$ to $47.37\%$) compared to the detection results of the original Mask R-CNN model. The mAP value of the rectangular box with cracks was $66.05\%$, but compared to the detection results of the original Mask R-CNN model, it still increased by $47.97\%$ ($18.08\%$ to $66.05\%$). The mAP value of the lack-of-fusion rectangular box was $68.86\%$, but compared to the detection results of the original Mask R-CNN model, it still improved by $24.76\%$ ($44.1\%$ to $68.86\%$); The mAP of its contour segmentation is $61.60\%$, which is $12.5\%$ higher than the detection results of the original model ($49.1\%$ to $61.60\%$). The mAP value of the rectangular box of the Weld reinforcement is $59.82\%$, but compared to the detection results of the original Mask R-CNN model, it still improved by $26.22\%$ ($33.6\%$ to $59.82\%$); the mAP of its contour segmentation is $54.26\%$, which is $26.13\%$ higher than the detection results of the original model ($28.57\%$ to $54.70\%$). These results demonstrate that the performance of the optimized Mask R-CNN magneto-optical image defect recognition model has been significantly improved.

5. Conclusions

This paper proposes a magneto-optical imaging classification and recognition model for welding defects based on the Mask R-CNN intelligent recognition framework, designed to identify defect types and segment defect contours. In order to optimize the recognition results of Mask R-CNN, pre-differential fusion operation is performed on the magneto-optical images based on the magneto-optical imaging characteristics of welding defects. Then, the Mask R-CNN model was used to intelligently recognize the preprocessed magneto-optical images, obtaining recognition results for different types of welding defects in magneto-optical images. The categories and contours of the preprocessed welding defects were compared with the recognition results of the original model, and the average accuracy was used to measure the quality of the model recognition results.

The experimental results showed that when an Intersection over Union (IoU) was 0.5, the accuracy of Mask R-CNN in recognizing rectangular boxes in the model increased by $35.07\%$, and the accuracy of segmentation increased by $34\%$. As the most important indicator for measuring the performance of object detection algorithms, mAP, the test values of rectangular boxes increased by $33.15\%$ (from $27.62\%$ to $60.77\%$) compared to the detection results of the original Mask R-CNN model, and the test values of contour segmentation increased by $27.82\%$ (from $26.88\%$ to $54.70\%$) compared to the detection results of the original model. From this, it can be seen that the performance of the optimized Mask R-CNN magneto-optical image defect recognition model has been greatly improved.

The distribution of defect magneto-optical images in this dataset is imbalanced: the number of magneto-optical images for cracks and lack of fusion is twice that of pits and weld reinforcement, while defect-free magneto-optical images are the least numerous. The imbalance of the dataset can also affect detection results. Therefore, dataset imbalance may be a contributing factor to the suboptimal performance of the model. Future research will focus on mitigating the impact of dataset imbalance on defect recognition and contour extraction.