Chip Appearance Inspection Method for High-Precision SMT Equipment

: In order to meet the defect-detection requirements of chips in high-precision surface mount technology (SMT) equipment widely used in the electronic industry, a chip appearance defect-detection method based on multi-order fractional discrete wavelet packet decomposition (DWPD) is proposed in this paper. First, lead and body regions were extracted from chip images using the image segmentation algorithm with asymmetric Laplace mixture model and connected-component labelling algorithm; then, the texture feature of the region to be inspected was extracted with the multi-order fractional DWPD algorithm and the geometric and gradient features were combined to form image features of the region to be inspected before the subset of features was selected from image features with the feature selection algorithm based on the variational Bayesian Gaussian mixture model; and ﬁnally, the support vector machine was used to determine whether the region to be inspected was defective. An experiment was conducted on a data set captured in high-precision SMT equipment. The accuracy of the proposed chip appearance defect-detection method is about 93%, which is more accurate than existing ones.


Introduction
High-precision surface mount technology (SMT) equipment plays an important role in the electronic manufacturing field. The high-precision placement machine, a typical high-precision SMT equipment, is used to mount the chip onto the print circuit board (PCB) accurately so that electronic equipment can realize the designed functions. Integrating software, algorithm, machine, control, optical, and other technologies, it is the most important and complex equipment in the manufacturing of electronic equipment [1,2]. Recently, more and more researchers are paying attention on the key technologies of high-precision placement machine [3,4].
With the rapid development of SMT placement machines, the size of components is becoming smaller and the assembly density of SMT patch is becoming higher. Among the various problems in placement machines, inspecting appearance defects on the electronic chip before the chip is mounted onto the PCB are receiving increasing attention. As shown in Figure 1, common appearance defects of chips in electronic manufacturing include contaminated areas on lead surfaces, burrs on lead edges, and scratches on chip bodies. The contaminated area on a lead surface refers to the oil or rusty stain on the lead. Lead with such defects cannot be fixed to the PCB bonding pad by soldering, leading to poor contact of the chip or the chip peeling off from the PCB. A burr on the lead edge will concentrate soldering around the burr, short-circuiting the PCB. A scratch on the chip body shows that the chip has frictions with other objects and that the resultant static electricity may have damaged the internal circuit of the chip. Any electronic chip with such a defect used in production will cause serious failure of the electronic equipment or even serious potential risks. Therefore, a chip appearance defect-detection method is one of the key technologies urgently needed in the development of placement machines.  Researchers have conducted a lot of work on electronic chip appearance defect detection. A lead defect-detection algorithm based on template matching and colour contrast was proposed in [5]. With this algorithm, lead positions and colour defects can be inspected, but it is impossible to determine whether there is any contaminated area on the lead surface of a chip and whether there is any scratch on the chip body. Furthermore, a defect-detection method based on image feature and support vector machine was presented in [6,7]. The image features include geometric, grey, and texture features. However, the texture feature cannot reflect the change in image texture using multiple scales. A work by [8] investigated an electronic chip inspection system based on stereoscopic vision, by which the stereoscopic information of an electronic chip can be obtained with a complex three-axis control platform. The work in [9] developed several methods to inspect defects in the flip chip solder joints. In [10], some problems when inspecting defects in industrial applications were reviewed, and the texture based method is shown to be widely used to solve those problems. Also, the work in [11] investigated a segment-based method to detect grain surface defects in the chip industry. The algorithms proposed by the researches mentioned above meet the requirements of the industrial electronic manufacturing field for surface defect detection to some extent. Among these algorithms, the chip appearance defect detection based on a wavelet is more popular with higher accuracy [12,13], which remains unsolved.
On the other hand, wavelet transform technology and wavelet packet decomposition (WPD) technology are widely utilized in texture feature extraction. The work in [14] processed texture images with wavelet transform and applied mean and standard deviation as wavelet statistical characteristics. Furthermore, WPD was adopted in [15] instead ofwWavelet transform in texture feature extraction, and the daubechies filter, symlets filter, and biorthogonal filter were compared in wavelet transform in [16]. The work in [17] spread 2D wavelet transform into 3D wavelet transform to extract spectral characteristics from hyper-spectral remote sensing images. Furthermore, the singular value decomposition (SVD) algorithm was used to enhance images before wavelet transforming in [18]. However, the chip appearance defect detection based on multi-order fractional WPD remains unsolved.
Contributions of this paper. The contributions of our work are listed as follows:

1.
A multi-order fractional DWPD algorithm is proposed to extract features from chip images.

2.
The clustering algorithm based on the variational Bayesian Gaussian mixture model is adopted to select image features.

3.
A chip appearance defect-detection method based on the multi-order fractional DWPD is conducted.
Organization of this paper. The rest of this paper is organised as follows. The background on DWPD and multi-order fractional DWPD are presented in Section 2. The main results on chip appearance defect detection based on multi-order fractional DWPD are proposed in Section 3. Section 4 presents the experiment on a high-precision SMT equipment to estimate the effectiveness of the proposed method, and Section 5 concludes this paper.

Discrete Wavelet Packet Decomposition
The 1D discrete wavelet transform (DWT) can be realized with a multi-layer transform structure. High-pass filters and low-pass filters are used to process the input signal in each layer of transform, with the high-pass filter outputting a precise detailed part of the input signal and with the low-pass filter outputting the rough approximation part. According to the Nyquist sampling theorem, the highest frequency of the filtered signal is only half of its original signal. Therefore, it is necessary to halve the data length by downsampling after the signal is processed by two groups of filters. In the 1D DWT, the input signal is taken as the input at the first layer and the rough approximation part of the previous layer is taken as the input at each subsequent layer. A layer of DWT is expressed as [19]: where x is the input signal, y h is the precise detail part, y l is the rough approximation part, h is the high-pass filter coefficient, and l is the low-pass filter coefficient. h and l are a group of quadrature mirror filters meeting the following condition: where N is the length of the filter coefficient. The 2D DWT can be realized with a similar multi-layer transform structure. The layer of wavelet transform defined by Equation (1) is conducted in the row and column directions successively in each layer of transform. Equation (1) discomposes the input signal into two parts. Therefore, the input signal is decomposed into four parts by the single layer transform in 2D DWT, including the output processed by the low-pass filter in the row and column directions successively as the rough approximation part as well as all other three outputs as the precise detail part. Similar to 1D DWT, the input signal is taken as the input at the first layer, and the rough approximation part of the previous layer is taken as the input at each subsequent layer.
What should be mentioned is that only the rough approximation part of the previous layer in the above 1D DWT and 2D DWT is processed at each layer, while all other precise detail parts are ignored. Therefore, too much detailed information is lost in the transform process. To address this problem, the discrete wavelet packet decomposition (DWPD) algorithm is adopted to extract features. What should be mentioned is that the multi-layer transform structure in 1D DWPD and 2D DWPD is similar to that in DWT and the filters in 1D DWPD and 2D DWPD are the same as that in DWT at each layer. In the multi-layer transform structure, the input signal is taken as the input at the first layer, while each output of the previous layer is taken as the input at each subsequent layer.

Multi-Order Fractional Discrete Wavelet Packet Decomposition
Before presenting the main results on chip appearance defect detection, the basic definitions on multi-order fractional DWPD are given in this section. By adding an extra transform-order parameter to the Fourier transform, fractional Fourier transform is defined as follows [20,21]: where p is the order of fractional Fourier transform and K p (u, t) is the kernel function of fractional Fourier transform, which is defined as follows: where n is an integral and σ(·) is the Dirac function that meets the following conditions: and where A(u, t, p) is defined as follows: The period of the order p of the fractional Fourier transform defined by Equation (4) is 2, usually restricting the order within p ∈ (−1, 1). Furthermore, it is not difficult to find that the fractional Fourier transform has the property of FrFT p, f (u) = FrFT p+1, f (−u), further restricting the order within p ∈ (0, 1).
Based on the discrete sampling method proposed in [22], 1D discrete fractional Fourier transform is defined as follows: where I is the N long input discrete signal and W N is the matrix of N × N. The element with a label of [m, n] is defined as follows: and Λ N p,u and Λ N p,t are the diagonal matrix of N × N. Elements with the label [n, n] are defined as follows: where ∆t and ∆u p are the sampling intervals of the time domain and fractional Fourier transform domain, respectively, and meet the following condition: Similar to the 2D discrete Fourier transform, the 1D discrete fractional Fourier transform defined by Equation (7) is conducted in the row and column directions successively to obtain the results of 2D discrete fractional Fourier transform. In this paper, if the orders in both directions are p, then the 1D discrete fractional Fourier transform is called a p-order 2D discrete fractional Fourier transform.
What should be mentioned is that the 2D fractional DWPD algorithm [23] can be obtained by combining 2D discrete fractional Fourier transform and 2D DWPD. In 2D fractional DWPD, the input signal is processed by the p-order 2D discrete fractional Fourier transform first, and then, the results of the previous steps are processed by the 2D DWPD to obtain the results of p-order 2D fractional DWPD. According to the properties of discrete fractional Fourier transform, the order of 2D fractional DWPD is within p ∈ (0, 1).

Steps of the Appearance Defect-Detection Method
As shown in Figure 2, the appearance defect-detection method researched in this paper is carried out according to following steps: (1) Extract the lead and body regions from chip images using the image segmentation algorithm with the asymmetric Laplace mixture model [24] and connected-component labelling algorithm;

Extraction Algorithm of the Region to Be Inspected
In this section, the region to be inspected is extracted using the image segmentation algorithm with the asymmetric Laplace mixture model and connected-component labelling algorithm. As shown in Figure 1, a chip image includes three regions: the lead region, the body region, and the background region. The body and lead of an electronic chip consist of opaque plastic and bright-colored sheet metal, respectively, and are displayed as a black region and a white region on the chip image, respectively. The background of the chip image is displayed as black in the placement machine but with possible small white jamming regions. That is, when a complete chip image is processed by the chip defect-detection method directly, the inspection method may mistake jamming regions in the background of the chip image as chip defects, which will affect the accuracy of the detection method. To handle this problem, the lead and body regions are extracted from the chip image as the regions to be processed in subsequent steps before inspecting chip defects.
In the high-precision placement machine, the chip is often not in the center of the image and has a certain rotation angle; therefore, it is necessary to locate the position of the chip image. In addition, there may be jamming regions in the background of the chip image under the influence of ambient light and background region, which will lower the accuracy of the chip positioning. Therefore, the algorithm extracting the region to be inspected from the chip image is carried out according to the following steps: (1) Classify pixels of the chip image into two categories using the image segmentation algorithm with the asymmetric Laplace mixture model, among which the one with a larger average grey value is the potential lead pixel of the chip image. (2) Calculate the connected components of potential lead pixels in the chip image using the connected-component labelling algorithm, and take these regions as the potential lead regions. (3) Estimate the bounding rectangles with rotation angles for each potential lead region, and calculate their rectangularity using the following equation: where A lead region is the area of the potential lead region, i.e., pixels, and A bounding rectangle is the area of the bounding rectangle with a rotation angle. (4) Identify the regions for which the size does not meet the standard or for which the rectangle is too small in the potential lead regions as jamming regions, and then, eliminate these regions from the potential lead regions. If the number of screened lead regions is inconsistent with that of the standard chip, the chip is considered to have the defect "the number of lead regions does not match", and the subsequent defect-detection steps are stopped. Then, the first step of the proposed chip appearance defect-detection method is accomplished.

Texture Feature Extraction Based on Multi-Order Fractional DWPD
The second step of the proposed chip appearance defect-detection method presented above is the texture feature extraction of the region to be inspected. The time-frequency domain analysis method based on WPD is a common texture feature extraction method. In the appearance defect detection for high-precision SMT, the appearance texture of the chip is quite complex, requiring expansion of the traditional transform-based feature extraction method to meet the requirements of an actual application.
The fractional DWPD algorithm introduced in Section 2.2 was applied in some image processing applications, including image encryption [23], image watermark increasing [27], etc. In these applications, the traditional DWPD was replaced by the fractional DWPD and the order was selected by the order estimation link. The fractional DWPD method performed well in the applications mentioned above. For example, in the application of image encryption, the unknown order of fractional DWPD was used to increase the difficulty of image decryption. However, image textures cannot be increased by this method.
For the problem mentioned above, a multi-order fractional DWPD structure was conducted in this paper to extract image features as shown in Figure 3. As shown in Figure 3, two methods were used for decomposition: (1) The first case: The chip image was decomposed using the traditional 2D DWPD algorithm.
(2) The second case: The chip image was decomposed with the 2D multi-order fractional DWPD algorithm. Since the value range of the order of 2D multi-order fractional DWPD algorithm was within p ∈ (0, 1), the chip image was decomposed by the 2D fractional DWPD algorithm of 0.2-order, 0.4-order, 0.6-order, and 0.8-order. After the chip image was processed by the above multi-order fractional DWPD structure, the sub-frequency bands of the chip image were obtained and the energy of all the sub-frequency bands was combined as the texture features of the chip image. Then, the average value and standard deviation of the sub-frequency band were taken as the energy of the sub-frequency band in the texture feature extraction algorithm proposed in this paper. The average value and standard deviation of the sub-frequency band are given as following: where C is the matrix of the sub-frequency band, and M and N are the length and width of C, respectively.

Image Feature Selection Based on Variational Bayesian Algorithm
The third step of the proposed chip appearance defect-detection method presented above is image feature selection. As mentioned above, in order to excavate the features of defects deeply, texture features, geometric features, and gradient features are combined in the process of image feature extraction. Some of these features may be correlated, which is ineffective for appearance defect detection or even causes "dimensional disaster" [28]. Therefore, it is necessary to select a subset of features from image features using the feature selection algorithm before classification with the support vector machine [29][30][31][32].
As is known, the clustering-based algorithm is a common image feature selection algorithm [33,34] and consists of two steps: (1) Cluster features with clustering algorithms such as k-means and minimum spanning tree so that the features of different classes are mutually independent. (2) Take the feature closest to the center of the class as the representative feature in each class, and compose the final feature subset of all the representative features of the class.
What should be mentioned is that features in the subset obtained through the above algorithm are independent of each other, providing better results for subsequent support vector machine algorithms.
In traditional clustering-based feature selection algorithms, it is required to provide the number of subsets of features. If the number of subsets of features given is wrong in the feature selection algorithm, the representative features are ignored, which affects the effect of chip detection. To solve this problem, the method proposed in this paper adopts a clustering algorithm based on the variational Bayesian Gaussian mixture model [21]. In the process of clustering, the algorithm determines the optimal number of subsets of features automatically instead of trying to obtain the optimal number of subsets of features by cross verification.
To present the process of a chip appearance defect-detection method based on multiorder fractional DWPD better, bluethe pseudocode are given in Algorithm 1. Filter connected-components C based on as effective components C *

5:
Rotate components C * to generate regions to be inspected R 6: for r in R do 7: Extract feature f of region r Select feature f * from f based on variational Bayesian algorithm 13: Determine whether region r is defective based on Support Vector Machine 14: if r is defective then 15: return The chip is defective 16: end if 17: end for 18: return The chip is non-defective

Experimental Design
In order to verify the accuracy of the chip appearance defect-detection method proposed in this paper, an experiment is conducted on a high-precision SMT equipment.
As shown in Figure 4, the chip inspection module of the high-precision SMT equipment consists of a camera, lens, and light source. The camera is equipped with a 1/3-inch black and white charge coupled device (CCD) with a resolution of 1028 × 1024. The prime lens with a resolution of 1 million pixels is used for the lens. The light source consists of three ring light sources of 0 • , 45 • , and 60 • . The chip detection module controls these three ring light sources (each having 16 brightness levels) by the Pulse Width Modulation (PWM) module. Different details of the chip can be reflected in the image by adjusting the brightness level of ring light source: The ring light source of 0 • mainly reflects the flat part of the chip, that of 45 • mainly reflects the edge of the chip, and that of 60 • mainly reflects the uneven area of the chip. During the experiment, 30 chips with relevant appearance defects and 20 chips without defects in the electronic equipment production were collected. Chip images were collected under 12 light configurations with the high-precision SMT equipment, and the configurations of three ring light sources are shown in Table 1. As presented in Section 3, the lead and body regions are extracted from these chip images with image segmentation and connected-component labelling algorithms. Then, the regions obtained are manually labelled, and the lead region was marked as three categories: no defect, pollutant on the surface of the chip, and burr on the edge of the chip. The body region was marked as two categories: no defect and scratch on the body. Finally, the lead data set and the body data set were formed according to the results of manual labelling. What should be mentioned is that there are three types of data in the lead data set, each consisting of 200 samples, and there are two types of data in the body data set, each consisting of 200 samples. These types are splited into the training sets and test sets, which contain 120 samples and 80 samples, respectively. These training sets and test sets are gathered to generate training sets and test sets for the lead data set and body data set.  To evaluate the effectiveness of the proposed method, contrast experiments with the DWPD-based method and the cooccurence matrix based method were conducted with the same lead data set and body data set mentioned above. In the experiment, lead defect detectors and body defect detectors were trained with the lead train data set and body train data set and the detection results were estimated with the lead test set and body test set. In the test period, classification accuracy was calculated as follows: ACCU category = Number of correctly classified samples of category Number of samples of category .

Results and Discussion
The experiment results of these three methods are demonstrated in Tables 2 and 3. As shown in Tables 2 and 3, the performance of the proposed method outperforms the DWPD-based method and the cooccurrence matrix-based method. Furthermore, the accuracy of the proposed defect-detection method for lead without defects and pollutants on the lead surface and a body without defects and scratches on the body is more than 91%; specifically, the detection rate of a body without defects is more than 93%. However, the recognition rate is low for the defect with a burr on the edge of the pin. The main reason for these phenomena is that a chip with a larger burr size will be eliminated in the step extracting the region to be detected in the detection method, and this chip will be identified as the defect "the number of pin areas does not conform". That is, only chips with small burr sizes will be tested for appearance defects, and the texture of these samples but will not differ much from those without defects. However, the small size of these burrs will not cause problems of electronic chip offset and pin short circuit, so there is almost no impact on product quality.
A small data set consisting of 200 samples, which contains 120 samples for the training set and 80 samples for the test set, was used. In many cases, large data sets are not available in practical industrial electronic applications in which the relatively low accuracy is feasible and tolerated. If the number of samples for test data set is larger, the accuracy of the proposed method is higher and the cost of testing increases. The accuracy of chip defect detection may be up to 99% if the test set is large enough. That is, the accuracy of the proposed method is about 93%, which is higher than existing results with the same data set.
Furthermore, the proposed chip defect-detection method based on multi-order fractional DWPD cannot be applied to appearance defect-detection problems based on big data, which will be future work.

Conclusions
The requirement for high stability of the electronic equipment puts forward a high standard for chip defect detection in a high-precision placement machine. To solve this problem, a chip defect-detection method based on multi-order fractional DWPD was proposed in this paper. First, the image segmentation algorithm and connected-component labelling algorithm were utilized to extract the lead and body regions of the chip. Then, the complete region features were constructed by texture features based on the multi-order fractional DWPD algorithm, the geometric features were constructed based on the Hu invariant moment algorithm, and the gradient features were constructed based on the HoG algorithm. Finally, features were classified with the support vector machine to inspect the appearance defects of the chip. The proposed method was verified by experiments with pictures taken by actual equipment. The accuracy of the proposed method is about 93%, which is higher than existing results with the same data set. Furthermore, chip appearance defect detection based on a deep learning method will be considered in future work.

Data Availability Statement:
The source codes and datasets used to support the findings of this study are available from the corresponding author upon request via email: sunhaohit0@gmail.com.

Conflicts of Interest:
The authors declare no conflict of interest.

Abbreviations
The following abbreviations are used in this manuscript: