Insulated Gate Bipolar Transistor Solder Layer Defect Detection Research Based on Improved YOLOv5

Abstract

The expanding market scale of the insulated gate bipolar transistor as a new type of power semiconductor device has higher insulated gate bipolar transistor soldering requirements. However, there are some small bubbles difficult to detect. The accuracy and speed of existing detection algorithms are difficult to meet the requirements of automated quality monitoring. For solving these problems, a detection data set of solder layer images captured by X-ray and labeled was made and an improved algorithm based on YOLOv5 was proposed, which can detect defects accurately and at a fast speed. The main contributions of this research are as follows: (1) a tiny bubble detection layer that further integrates the deep feature information and shallow feature information is added to improve the model’s ability to detect small bubbles; (2) to speed up model convergence by optimizing anchor frame parameters; (3) we change the EIoU loss function as the bounding box loss function to solve the sample imbalance of the dataset; (4) combine the Swin Transformer structure to improve the convolution module and form a new feature extraction module, and introduce it into the backbone layer to improve the detection accuracy. The results of the experiment show that the overall performance of the improved network is better than the original and mainstream detection algorithms. The accuracy of the improved YOLOv5_SEST has reached 94.5% and 5.6% improvement in mAP for common bubble defect detection compared to the original algorithm. Our model size is only 5.3 MB, and the detection speed reaches 110 f/s. Therefore, the improved YOLOv5_SEST can well meet the requirements of automated quality monitoring of insulated gate bipolar transistors.

1. Introduction

In recent years, the rapid development of advanced technology is inseparable from the support of power electronics technology, the use of commutation technology can efficiently achieve the conversion of electrical energy, the Insulated Gate Bipolar Transistor (IGBT) is a variety of electronic power equipment undertaking the conversion of electrical energy and transmission control of the core components [1,2,3,4], it is also the most failure-prone component in converters [5,6]. With the rise of new energy industries, the efficiency and lifetime of IGBT have become the key to the global green energy-saving economy [7]. IGBT modules are subjected to relatively repetitive alternating high and low temperature and thermal cycles in actual operation for long periods, which makes it imperative that they have excellent reliability in terms of temperature. However, it has been found that the reliability of IGBT modules in high-reliability high power places is not high [8,9], and that defects in their solder layer interfaces can not only affect the overall model heat transfer but can even lead to the failure of the entire module. Its failure can lead to the failure of electronic and electrical equipment in minor cases, or it can lead to the overall system being unexpectedly plunged into downtime, resulting in serious economic losses [10,11,12,13]. Therefore, IGBT solder layer defect detection is of great importance to ensure the reliability and stability of IGBT modules.

The traditional methods for defect detection in IGBT modules include destructive testing sampling with microscopic and nondestructive testing. For nondestructive testing, a series of detection methods were proposed, such as ultrasonic inspection [14], and X-ray detection using digital radiography (DR) and computed tomography (CT). However, in traditional methods, due to the high cost and low efficiency of manual testing, it is difficult to complete the task of large-scale quality testing. As deep learning-based defect detection relies on images, imaging-friendly X-rays become the nexus for detection using deep learning.

Object detection algorithms are divided into three categories: single-stage algorithms, two-stage algorithms, and anchor-free algorithms. The two-stage algorithms generate candidate regions in advance with high precision but low speed such as Faster R-CNN [15], Mask R-CNN [16], and Cascade R-CNN [17], etc. The one-stage algorithms that synchronize the target prediction category with the regression position with high speed and high practicality meet the requirements of industrial real-time quality inspection such as YOLO family series [18,19], SSD [20], RetinaNet [21], etc. The anchor-free algorithms avoid hand-designed anchor frames, such as CenterNet [22], FCOS [23], etc.

In recent years, the rapid development of hardware technology has led to the rapid growth of computer computing power, laying the foundation for long development in various fields. Menelaos et al. [24] proposed the implementation of an unsupervised image classification task using a time-multiplexed spiking convolutional neural network based on VCSELs; Kim et al. [25] proposed AI2O3/TiOx-based resistive random access memory (RRAM) in response to the lack of consideration of errors occurring in the program during transfer in previous studies; Indranil et al. [26] proposed a technology-aware training algorithm to address the problem that various non-idealities in the crossbar implementations of synaptic arrays can degrade the performance of neural networks.The above hardware development cases offer the possibility and feasibility of deep learning to implement industrial defect detection.

With the rapid development of computer computing power, deep learning has contributed to defect detection in industries such as printed circuit boards, steel, capsules, batteries, textiles, fruits and vegetables, and smart cars. For example, Xu et al. [27] proposed a joint training strategy of Faster R-CNN and Mask R-CNN to achieve intelligent pavement crack detection. Compared with YOLOv3, it improves the performance but leads to a decrease in Mask R-CNN bounding box detection. Guo et al. [28] proposed a method to improve the detection accuracy by first preprocessing the tile images with uneven illumination, complex surface texture, and low contrast using an adaptive histogram homogenization method with restricted contrast, and then using Mask R-CNN for defect detection. Guo et al. [29] proposed a scheme of adding a guided anchor frame algorithm to produce an anchor based on Faster R-CNN to achieve intelligent detection of surface defects of three parts, which substantially improved the detection accuracy of the model.

Compared with other algorithms, YOLO series algorithms have non-negligible advantages. Yao et al. [30] proposed a YOLOv5-based model for the detection of surface defects in kiwifruit, which provides an efficient and intelligent way of post-production quality inspection for agriculture. Wang et al. [31] proposed a YOLOv5-based algorithm to achieve unmanned quality inspection of tile surface defects on production lines for the current problem that manual quality inspection cannot be avoided on tile surfaces. The results show that the accuracy of the YOLOv5 model is higher than that of Faster R-CNN, SSD, and YOLOv4. Li et al. [32] used the DBSCAN+K-means clustering algorithm based on YOLOv3 to re-cluster Anchor under Avg IOU criterion, add residual units, and introduce SE attention mechanism to increase the model-specific feature extraction capability to achieve the improvement of YOLOv3 for PCB defect detection accuracy and speed. The above case study provides the feasibility of choosing YOLOv5 for defect detection in IGBT solder layers.

There is a lack of research on the use of deep learning for the detection of bubble defects in the solder layer of IGBT modules, so this paper collects a dataset of labeled IGBT solder layer bubble defects. Considering that the more complex detection process of the two-stage algorithm and the network structure leads to a slower detection speed, which cannot meet the requirements of industrial quality inspection in real-time. Low confidence in the prediction results of the anchor-free frame algorithm. The regression-based, one-stage algorithm is faster and more practical to meet the requirements of industrial real-time unmanned quality control. This paper presents an improved YOLOv5-based defect detection model for IGBT solder layers. The main contributions of this research are as follows:

• A tiny bubble detection layer that integrates deep feature information and shallow feature information is added to improve the model’s ability to detect small bubbles.
• To speed up model convergence by optimizing anchor frame parameters.
• We change the EIoU loss function to the bounding box loss function to solve the sample imbalance of the dataset.
• Combine the Swin Transformer structure to improve the convolution module, form a new feature extraction module and introduce it into the backbone layer to improve the detection accuracy.

In the remainder of this paper, Section 2 introduces the YOLOv5 base model. We explain our proposed methodology in Section 3, while experimental details and results are discussed in Section 4. Finally, a brief conclusion is given in Section 5.

2. Related Work

YOLOv5

Among the series of object detection models, the most classic one-stage detection model is the YOLO series, of which YOLOv5 [33] was released by Ultralytics in 2020. After continuous updates and upgrades, there are five versions of the YOLOv5 algorithm in the order of increasing the weight, width, and depth of the model, which are YOLOv5n, YOLOv5s, YOLOv5m, YOLOv5m YOLOv5l, and YOLOv5x. The overall structure of the YOLOv5 network is shown in Figure 1, which consists of four main parts: input side, backbone, neck, and head.

Data augmentation, self-adaptive anchor frame calculation, and self-adaptive image scaling are implemented on the input side. The data augmentation process, in which Mosaic data augmentation cite18 is used in addition to the basic data augmentation method, as shown in Figure 2, randomly takes four images from the dataset and rotates, flips, scales, and splices them to obtain new training data, adding small target samples to enrich the dataset and further enhance the training speed of the network. In the self-adaptive anchor frame calculation process, the set anchor frame is input to the network to calculate the predicted frame compared with the real frame to reverse update the network parameters and iterate to reach the best. In the self-adaptive image scaling process, calculating the scaling ratio, size, and black border fill values to add the least amount of black border and retain the most features.

Backbone consists of a 6 × 6 Conv module equivalent to Focus, four Conv modules, four C3 modules, and the SPPF module. The input image is firstly sliced through the 6 × 6 Conv module by taking a value for every pixel interval in the horizontal and vertical directions, halving the image size and quadrupling the number of channels, followed by a series of convolution, pooling, Cross Stage Partial Network (CSPNet), and finally extracting high-level semantic information through SPPF to obtain the feature map.

Neck follows the structure of Feature Pyramid Networks (FPN) combined with the Path Aggregation Network (PAN) [34], which achieves the fusion of high-level semantic information with shallow location information to improve the model detection accuracy while preserving the original image information.

Head obtains high-quality candidate frames by using the non-maximum suppression method to remove low-quality predicted frames from the target anchor frames of corresponding sizes detected from the Neck of three different sizes of feature maps.

The loss of a target detection class model generally consists of target confidence loss, classification loss, and bounding box loss, which is shown as Equation (1), where ${\lambda }_1,{\lambda }_2,{\lambda }_3$ are the equilibrium coefficients.

$$\mathrm{Loss}={\lambda }_1{L}_{cla}+{\lambda }_2{L}_{loc}+{\lambda }_3{L}_{conf}$$

(1)

The target confidence of YOLOv5 and the loss of classification are adopted from BCEWithLogitsLoss, as shown in Equations (2) and (3).

$$\begin{array}{cc}\hfill & \hfill BCEL=-\sum _{i=1}^N\left[y_i^{*}ln\left(\sigma \left(x_i\right)\right)+\left(1-y_i^{*}\right)ln\left(1-\sigma \left(x_i\right)\right)\right]\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill & \hfill \sigma \left(x_i\right)=Sigmoid\left(x_i\right)=\frac{1}{1+e^{-x_i}}\hfill \end{array}$$

(3)

where N represents the number of target categories, $x_i$ represents the predicted value of the i category, $\sigma \left(x_i\right)$ represents the probability of the i category after the sigmoid function, and $y_i^{*}$ represents the true value of the i category.

Bounding box loss is calculated using CIoU Loss [35], as shown in Equations (4)–(6). Where $\alpha$ is the weight coefficient, v is used to measure the consistency of the relative proportions of two rectangular boxes, $w^{gt},h^{gt}$ represents the width and height of the real box, $w,h$ represents the width and height of the predicted box, and $IoU$ represents the intersection ratio of the predicted box and the real box. Even though CIoU Loss increases the penalty for aspect ratio to improve the evaluation accuracy, there are problems with using inverse trigonometric functions for the calculation to increase the arithmetic power consumption, reduce the training speed and not consider the sample distribution.

$$\begin{array}{c}\hfill \alpha =\frac{v}{(1-IoU)+v}\end{array}$$

(4)

$$\begin{array}{c}\hfill v=\frac{4}{{\pi }^2}{\left(arctan\frac{w^{gt}}{h^{gt}}-arctan\frac{w}{h}\right)}^2\end{array}$$

(5)

$$\begin{array}{c}\hfill \mathrm{CIoULoss}=1-IoU+\frac{\rho \left(b,b^{gt}\right)}{c^2}+\alpha v\end{array}$$

(6)

3. Methodology

YOLOv5 is a high-performance model that locates and classifies defects in one step. Therefore, YOLOv5 is selected for the corresponding network structure adjustment and improvement to achieve the detection of IGBT solder layer bubble defects. Due to the small size, large number, irregular distribution, and different shapes of defective bubbles in IGBT modules, the detection of solder layer bubble defects by using object detection algorithms such as YOLOv5 are not effective. To improve the detection capability of solder layer defect bubbles, the overall structure of the improved YOLOv5 in this paper is shown in Figure 3: (1) adding a small target detection layer that fits the tiny bubbles of the IGBT solder layer and adjusting the anchor frame parameters; (2) improving the box regression loss function; (3) adding a new convolution module to the backbone network.

3.1. Add a Small Object Detection Layer

The original YOLOv5 model has three detection layers. They are a detection layer of size 80 × 80 for detecting objects with a size of 8 × 8 and above, a detection layer of 40 × 40 for detecting objects with a size of 16 × 16 and above, and a 20 × 20 size detection layer for detecting targets of size above 32 × 32 [36]. A larger feature map size detects small targets and a smaller feature map size detects large targets [37].

As the relatively high number of tiny defects in IGBT solder layer defect detection, the model is required to improve the detection capability for tiny targets. Therefore, in this paper, based on the original YOLOv5, a set of a priori anchor frames (5,6), (8,14), (15,11) for tiny targets is added to improve the convergence speed during model training; while retaining the original map information, the feature map is further extended by adding a set of convolution modules, upsampling modules for tiny targets. To reduce the loss of location information, the deep semantic information is fused with the shallow information, and the minimum feature map obtained in the backbone network is spliced with the further extended feature map to obtain a new feature map. A detection layer for detecting small targets is added to match the new feature map detection.

3.2. Loss Function

Box regression loss for YOLOv5 calculated by CIoU loss function [38]. Although CIoU improves the convergence speed and regression accuracy of the network, the v in the formula only reflects the difference in aspect ratio and does not show the true difference between the aspect ratio and its confidence level. Moreover, as inferred from the prediction frame and the prediction formula, w and h cannot be increased and decreased at the same time, so there may be a situation that prevents the model from optimizing the similarity effectively. To address the above problem, EIoU solves theoretically the problem that w and h cannot be scaled up or down simultaneously in CIoU Loss by directly penalizing the loss function of the prediction results of w and h, as shown in Equation (7).

$$\begin{array}{cc}\hfill & \hfill \mathrm{EIoULoss}=L_{IoU}+L_{dis}+L_{asp}\hfill \\ \hfill & \hfill =1-IoU+\frac{{\rho }^2\left(b,b^{gt}\right)}{c^2}+\frac{{\rho }^2\left(w,w^{gt}\right)}{C_w^2}+\frac{{\rho }^2\left(h,h^{gt}\right)}{C_h^2}\hfill \end{array}$$

(7)

It can be seen that the EIoU loss function consists of the intersection ratio loss of the predicted frame and the real frame, the center distance loss of the predicted frame and the real frame, and the width and height loss of the predicted frame and the real frame, where $C_w$ and $C_h$ are the width and height of the two rectangular closures, respectively. Bounding box regression also suffers from the problem of unbalanced training samples, that is, the number of low-quality anchor boxes with large regression errors in a single image is much larger than the number of low-quality samples with small errors, and such samples can cause excessive gradients to affect the training effect. To solve this problem, Focal Loss is introduced to solve the problem of sample imbalance based on the original one, which makes the EIoU Loss loss function more consistent with the detection situation in this study.

3.3. Improvement of C3 Module

The traditional Convolutional Neural Network (CNN) [39] has been very successful in extracting features as well as structure for image classification with good fitting ability, and the structure has been optimized with the addition of residual structure and the deep neural network has led to a better combination of context and corresponding generalization ability. From the perspective of human vision, the Transformer [40] based on the self-attention mechanism is more in line with human visual logic, but in the application process, it is found that it cannot adapt to a large range of target size changes and the attention mechanism brings a correspondingly large amount of computation, which makes the accuracy and speed less than that of CNN. Swin Transformer [41] proposes a hierarchical approach to construct multi-layer feature maps to solve the above problems, as well as to introduce a sliding window operation to restrict attention to a finite window to make the computational complexity linearly related to the input image size.

Swin Transformer is similar to ViT in that the input image with size $H\times H\times 3$ is first chunked, then the feature dimension of the image block is changed by linear embedding in stage1, and then it is fed into Swin Transformer Block and the input features are calculated; stage2-stage4 is similar in that the $2\times 2$ adjacent image blocks are merged and stitched together, and then the convolutional network is used to downscale them to achieve the desired feature dimension.

Swin Transformer Blocks are shown in Figure 4, and the normalized data first enter the Windows Multi-head Self Attention (W-MSA) module that further divides the image blocks into different regions and performs self-attentive calculations on them; immediately into the Multilayer Perception (MLP) machine using the GeLU function as the activation function to implement the nonlinear transformation of the data. Finally, the introduction of the Shifted Windows Multi-head Self Attention (SW-MSA) mechanism, which enables the interaction of information within partial windows between different levels; and connected using residual modules after each MSA module and MLP, as shown in Equations (8) to (11).

$$\begin{array}{c}\hfill {\widehat{Z}}^l=W-MSA\left(LN\left(Z^{l-1}\right)\right)+Z^{l-1}\end{array}$$

(8)

$$\begin{array}{c}\hfill Z^l=MLP\left(LN\left({\widehat{Z}}^l\right)\right)+{\widehat{Z}}^l\end{array}$$

(9)

$$\begin{array}{c}\hfill {\widehat{Z}}^{l+1}=SW-MSA\left(LN\left(Z^l\right)\right)+Z^l\end{array}$$

(10)

$$\begin{array}{c}\hfill Z^{l+1}=MLP\left(LN\left({\widehat{Z}}^{l+1}\right)\right)+{\widehat{Z}}^{l+1}\end{array}$$

(11)

The C3 module, which incorporates Swin Transformer Blocks, is named the C3STR module, and its specific structure is shown in Figure 3. It is experimentally demonstrated that the C3STR module, which introduces the Swin Transformer module, can achieve the same effect of introducing an attention mechanism to improve model accuracy with minimal additional overhead.

4. Experiments

4.1. Experimental Environment and Parameter Settings

To verify the effectiveness of the improved method in this paper, corresponding experiments will be conducted on the homemade IGBT data set. The experimental environment is Ubuntu 18.04 operating system, 2 Inter Xeon Silver(R) 4010R processors, Nvidia Geforce RTX 3080 graphics card, and deep learning framework Pytorch version 1.10. To ensure the fairness and reliability of the experimental results, all training parameters in this paper are fully set as the parameters in

Table 1. Parameter settings.

Name	Value
Learning rate	0.01
Momentum	0.937
Weight decay	0.0005
Batch size	16
Epochs	2000
size of picture	640 × 640

, which contain learning rate, momentum, weight decay, batch size, number of iterative rounds, and picture size.

4.2. Dataset Construction

4.2.1. Acquisition of Raw Images

The quality of the dataset is highly relevant to the training and improvement of the model [42]. Since there is no publicly available dataset for IGBT solder layer defects, this paper establishes a standardized IGBT production line solder layer defect detection dataset. The images of this dataset were randomly collected from the standardized IGBT production line using an industrial X-ray device with a collection size of 1536 × 1536 images and a total of 400 images with high image quality. To ensure the diversity and completeness of defect types, the data images are randomly collected from different production batches. The principle is to use the penetrating property of X-ray for precise imaging to detect the internal solder layer structure, solder layer defect location, size, and dimension of IGBT without damaging the IGBT.

4.2.2. Image Pre-Processing

The image format of X-rays acquisition is 16-bit, which cannot be processed by the algorithm directly, so it needs to be converted to an 8-bit image by window width and window bit processing, and the principle is to find the area corresponding to the image that needs to be displayed and processed, and do histogram equalization on this part to convert it to the 8-bit image. Figure 5a is the original image of the IGBT solder layer acquired by X-ray, which can be seen to be unusable for algorithmic calculations. Figure 5b shows the IGBT display image after preprocessing. It can be seen that in the X-rays acquisition image of the IGBT after processing, the solder layer defects are visible.

4.2.3. Annotation of the Dataset

LabelImg software was used to manually grade and label the 400 sheets of preprocessed IGBT solder layer bubble defects to produce inspection labels and avoid boxing irrelevant content as much as possible during the labeling process. The label format is chosen from the YOLO data set format, a total of two types of solder defects are marked, and the labels are divided into the bubble, bubble_edge. The image annotation is shown in Figure 5c, where the bubble is annotated with regular bubble defects, while bubble_edge is used to annotate the edge bubble with the irregular shape at the edge of the solder layer. The obvious sample imbalance can be seen in Figure 6a, where the first type of ordinary bubbles is much larger than the second type of edge irregularity bubbles, Figure 6b where the central point can be seen to be relatively concentrated, and Figure 6c where the defects are of different sizes.

4.3. Evaluation Indicators

In this paper, Precision, Mean Average Precision (mAP), GFLOPs, and Weight are used as model evaluation metrics for the IGBT solder layer defect detection task. The accuracy rate represents the ratio of the number of defects (TP) correctly predicted by the model to the total number of defects (TP + FP) predicted for a certain category, as shown in Equation (12). The mAP, which combines the accuracy and completeness rates, is calculated as follows Equations (13) to (15), where n is the total number of categories, $AP\left(i\right)$ is the detection accuracy of a certain category of defects, can effectively express the recognition ability of the detection model. The computational volume and model size reflect the complexity of the model from different aspects, respectively, and all the above four metrics are positively correlated with the detection effect.

$$\begin{array}{c}\hfill P={\displaystyle \frac{TP}{TP+FP}}\end{array}$$

(12)

$$\begin{array}{c}\hfill R={\displaystyle \frac{TP}{TP+FN}}\end{array}$$

(13)

$$\begin{array}{cc}\hfill AP& \hfill ={\int }_0^{1}P\left(R\right)dR\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill mAP& \hfill =\frac{1}{n}\sum _{i=0}^{n}AP\left(i\right)\hfill \end{array}$$

(15)

4.4. Analysis of Experimental Results

4.4.1. Comparison of Improved Model Results for YOLOv5 Base Model

To further verify the universality and effectiveness of the improved method proposed in this paper, the following five groups of comparison experiments were designed for the improved detection of defects in the solder layer of IGBT modules, and the comparison results are shown in

Table 2. Comparison of YOLOv5 basic network model and improved model results.

Model	Precision%	mAP%	GFLOPs/M	Weight/M
YOLOv5n	81.3	57.9	4.2	3.9
YOLOv5n_SEST	85.4	63.5	17.7	5.3
YOLOv5s	81.7	58.2	15.8	14.5
YOLOv5s_SEST	74.1	63.2	69.8	17.5
YOLOv5m	83.5	59.9	48.0	42.2
YOLOv5m_SEST	77.8	62.9	272.5	52.7
YOLOv5l	75.9	60.7	107.8	92.9
YOLOv5l_SEST	80.0	62.7	690.6	118.7
YOLOv5x	75.1	61.3	204.0	173.2
YOLOv5x_SEST	84.8	63.5	1401.6	225.4

.

According to the results of previous public dataset tests, the complexity of the target detection network structure is positively correlated with the detection effect. However, the experimental results show that too much convolution operation and too deep a network structure are not suitable for solder layer bubble defects with non-uniform size and irregular distribution. As can be seen from Table 2, the improvement strategies proposed in this paper are optimized in all five base models, and in terms of mAP in YOLOv5n, YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x are improved by 5.6%, 5%, 3%, 2%, and 2.2%, respectively, proving that the improvement methods mentioned in this paper effectively enhance the detection performance of the models. Overall, YOLOv5n_SEST performs better in all aspects of industrial application conditions where accuracy, recognition performance, and lightness are required.

4.4.2. Improved Model Comparison Experiments

To better demonstrate the benefits of the improved model on the IGBT dataset, each improvement point will be validated separately below:

• To verify the superiority of the C3STR module, the following comparison experiments were carried out between it and the classical attention module. To ensure fairness in the experiments, all experiments were carried out at the same location based on the addition of a small target detection layer and the results are shown in

  Table 3. Performance comparison of C3STR with common attention modules.

  Model	Precision%	mAP%	GFLOPs/M	Weight/M
  YOLOv5n6_CA	80.1	56.1	5.6	6.8
  YOLOv5n6_SE	78.4	57.8	4.3	4.1
  YOLOv5n6_CBAM	68.8	57.3	4.23	4.0
  YOLOv5n6_C3CBAM	71.2	53.6	57.0	8.0
  YOLOv5_SEST	85.4	63.5	17.7	5.3

  . The results show that the CSTR module proposed in this paper improves the detection accuracy by 7.4%, 5.7%, 6.2% and 9.9% over the CA [43] module, SE module [44], CBAM [45] module and C3CBAM module, respectively, which is because C3STR combines the advantages of Swin Transformer Blocks and retains the maximum features based on Improved detection.
• Performance analysis of the number of C3STR settings was carried out to compare the effect of different numbers on the performance of the IGBT solder layer defect detection model, and the experimental design gradually replaced the C3 module when the detection results were compared, and the results are shown in

  Table 4. Performance comparison of test results with different numbers of C3STR.

  Model	Number of C3STR%	Precision%	mAP%	GFLOPs/M	Weight/M
  YOLOv5n_C3STR	1	85.4	63.5	17.7	5.3
  YOLOv5n_C3STR	2	74.1	61.1	18.1	8.0
  YOLOv5n_C3STR	3	71.4	53.6	57.0	10.0
  YOLOv5n_C3STR	4	78.2	60.1	71.3	20.2

  . Experimental results show that replacing a C3 module gives the greatest accuracy gain at a minimal cost.
• In order to verify the effectiveness of the EIoU loss function for the dataset of this paper, the following experiments will be compared with the loss functions of DIoU, GIoU, CIoU and AlphaIoU, and the results are shown in

  Table 5. Performance comparison of different loss function detection results.

  Model	Precision%	mAP%	GFLOPs/M	Weight/M
  YOLOv5n_DIoU	64.8	59.5	7.8	5.1
  YOLOv5n_GIoU	70.6	60.2	7.8	5.1
  YOLOv5n_CIoU	70.7	61.5	7.8	5.1
  YOLOv5n_AlphaIoU	66.1	59.7	7.8	5.1
  YOLOv5n_EIoU	78.4	62.5	7.8	5.1

  . The experimental results show that the EIoU loss function improves the detection accuracy by 3%, 2.3%, 1% and 2.8%, respectively, compared with other loss functions, proving its indispensable advantage for sample imbalanced data sets.

4.4.3. Ablation Experiments

To further prove the effectiveness of the proposed improvement method in this paper and to explore the effectiveness and relevance of each improvement method, seven sets of ablation experiments were designed based on YOLOv5n, and each set of experiments uses the same initial conditions as well as data sets, the results of the ablation experiments are shown in

Table 6. Ablation experiment results. Where, Small_D indicates the small target detection layer structure proposed in this paper, EIOU indicates the change to the EIoU loss function applicable to the sample imbalance data, and C3STR indicates the introduction of the new C3 module that combines the C3 module with Swin Transformer in this paper. “✓” indicates that the improvement scheme is cited, and blank indicates that it is not cited.

Group	Small_D%	EIoU%	C3STR%	Precision%	mAP%	GFLOPs/M	Weight/M
1	✓			82	61.5	7.8	5.1
2		✓		81.4	59.7	4.2	3.9
3			✓	81.7	60.2	14.1	4.1
4	✓	✓		82.1	61.5	7.8	5.1
5	✓		✓	82.5	61.3	17.7	5.4
6		✓	✓	81.8	60.7	14.1	4.1
7	✓	✓	✓	85.4	63.5	17.7	5.4

.

As can be seen from Table 6, the addition of the small target detection layer improves mAP by 3.6% and accuracy by 0.7%, proving a large improvement in the detection of small target bubbles, as well as dense edge bubbles. Modifying the bounding box loss function has almost no change in the size of the model and the amount of computation, but different bounding box loss functions for different datasets have different effects, as well as for the sample imbalance. The EIoU loss function for the dataset used in this paper improves the mAP by 1.8%; after introducing the C3STR module, the mAP improves by 2.3%, which strongly demonstrates the feasibility of the Swin Transformer backbone network in the field of target detection; finally, by fusing the three improvements, the mAP is improved by 5.6%, which is a great improvement for the detection of small target dense bubble defects in IGBT module solder layers.

4.4.4. Performance Comparison Experiments

To further reflect the detection performance of the improved model in this paper, it is compared with various mainstream target detection models on the data set collected in this paper, and the comparison results are shown in

Table 7. Comparison of different model experiments.

Model	mAP%	Weight/M	FPS (f/s)
Faster_rcnn_r50	42.67	125.0	4.83
SSD_r34	24.2	47.4	15.46
Fcos	12.3	127.4	11.66
PPYOL_Tiny	19.23	3.8	36.28
Mask_rcnn	36.2	133.0	0.78
Cascade_rcnn	37.5	262.8	11.96
YOLOv7	46.12	74.8	71.42
YOLOv5n_SEST	63.5	5.4	110.00

.

As can be seen from Table 7, YOLOv5n_SEST has a significant advantage over most models not only in terms of accuracy but also in terms of detection speed. Although the model size compared to the PPYOLO_Tiny model is slightly larger, it has an irreplaceable advantage in terms of detection accuracy and speed. To further verify the superiority of the performance of the improved model, the top four models in Table 7, Faster R-CNN, Mask R-CNN, Cascade R-CNN, YOLOv7 [46], and YOLOv5n are selected for visual comparison with YOLOv5_SEST, and the comparison results are shown in Figure 7, it shows that the improved method based on the anchor box can better distinguish the background from the target, reduce the interference of background information, and lower the rate of missed detection; the other models have more or less missed detection and false detection.

Since YOLOv5n itself is a very small model, the addition of the small target detection layer and the fused C3STR module does not increase its size to an unacceptable level, and its model time and space complexity makes the model less demanding for the hardware environment when detecting so that it can be well applied to the IGBT industrial inspection site.

Table 8. YOLO5n_SEST specific performance.

Model	Precision %(Bubble)	Precision% (Bubble_Edge)	Precision% (All)	mAP%
YOLO5n_SEST	94.5	76.3	85.4	63.5

shows the YOLOv5n_SEST model-specific performance details, it can be seen that for the first category of ordinary bubbles accuracy reached 94.5%, the second category of unconventional bubbles only reached 76.3%, which shows that the second category of unconventional bubbles is a breakthrough point. From Figure 7a for the YOLOv5n_SESTR detection effect graph, it can be seen that the effect near the edge of the general because the first category and the second category have similar places difficult to distinguish, but the overall effect is suitable for industrial detection. The specific performance analysis of the model in this paper is shown in Figure 8, where (a) shows the relationship between the reconciled average functions of model accuracy and recall and the confidence level, (b) represents the graph line of the relationship between accuracy and confidence level, (c) represents the relationship between model accuracy and recall, and (d) represents the relationship between model recall and confidence level.

4.4.5. NEU-DET Dataset Validation

We selected the open-source dataset NEU-DET to verify the generality of the improved algorithm proposed in this paper. The model training parameters are set as in Section 4.1, and the experimental results are shown in

Table 9. Comparison of detection results in the NEU-DET dataset.

Model	AP%						mAP%	Weight/M
	Cr	In	Pa	Ps	Rs	SC
YOLOv5n	44.2	82.7	92.3	84.3	62.9	88.1	75.8	3.9
YOLOv5n_SEST	50.4	86.6	91.6	84.3	59.9	92.7	77.6	4.5

. The NEU-DET dataset is a strip surface defect dataset published by Northeastern University, which collects six typical surface defects of the hot-rolled strip: crazing, inclusion, patches, pitted, rolled, and scratches. A total of 1800 grayscale images, containing 300 samples for each type of defect. Table 9 shows that the improved model proposed in this paper improves the mAP of all defects by 1.8% compared with the original YOLOv5n, which verifies the generality of the proposed model.

5. Conclusions

In this paper, for the IGBT module solder layer bubble defect detection task, two types of IGBT module solder layer bubble defect data sets are collected and labeled, and an improved algorithm based on YOLOv5, the YOLOv5n_SEST model, is proposed. A detection layer for small target bubbles is added, which effectively detects more tiny bubbles. For the problem of unbalanced samples in the data set, the bounding box loss function is improved to improve the comprehensive detection effect of the model. For improving the detection capability capacity of bubble defects capability, the C3STR module is added to the backbone network; Swin Transformer Blocks are used to construct multi-layer feature maps; and the self-attention is restricted to the moving window to ensure that the computational effort is within the controllable range to achieve efficient feature extraction. The experimental results show that the improved model achieves 94.5% accuracy for common bubble defects, 85.4% accuracy for all defects, 5.6% improvement in mAP, and 110 f/s speed. The algorithm can well perform the task of real-time detection of industrial IGBT solder layer defects, ensuring the detection speed of the one-stage algorithm and improving the accuracy of the YOLOv5 model. The algorithm can well perform the task of real-time detection of industrial IGBT solder layer defects, ensuring the detection speed of the one-stage algorithm and improving the accuracy of the YOLOv5 model.

The degradation of the detection performance for irregular and dense bubbles at the edges is a limitation of the model. Therefore, we will continue to explore and optimize the performance of this model for edge bubbles, study the algorithms that are universally applicable to various industrial fields, and promote the application of artificial intelligence in unmanned quality inspection and intelligent inspection processes in the future.