The Performance Evaluation of Classic ICA Algorithms for Blind Separation of Fabric Defects

Abstract

Independent Component Analysis (ICA) is a blind source separation technique that has been broadly used in signal and image separation. In order to verify the feasibility of ICA algorithms which will be used for the detection of fabric defect, four kinds of classic ICA algorithms have been chosen and compared in terms of their algorithm performances. The results of simulation experiments show that the separation performances of these algorithms are different and FastICA algorithm has the best separation performance than others.

1. Introduction

In production and manufacture of fabric and garment, quality control usually is a crucial aspect to improve enterprises’ economic benefit and level of management. Inspection of fabric defect is an important process in fabric quality control. Defect inspection, a process of subjective, tedious, and time-consuming work, is usually carried out by human operators. With the development of machine vision and compute technology, Automated Image-based Inspection System (AIIS) offers an attractive alternative to human inspection. In image recognition, finding the shape of object or component is a rather general problem and various solutions are available, depending on the nature of data. Blind Sources Separation (BSS), which does not depend on any prior knowledge of source signals, is the process of estimating unknown source signals from observed signals. Usually BSS take an algorithm of ICA to accomplish source separation. The goal of ICA algorithm is to find a linear representation of non-Gaussian data so that the components are statistically independent.

Currently ICA has an extensive application in communication signal processing, speech signal separation, biomedical signal processing, financial data analysis, array signal processing and general signal analysis ect [1]. As so many domestic and foreign scholars studied ICA algorithms, the application of ICA algorithms in image processing aspect is becoming more and more popular. At present, the applications of ICA algorithms in image processing are mainly focused on image feature extraction [2,3], face recognition and detection [4], image separation, medical im-age processing [5], image watermarking [6], image retrieval, character recognition, remote sensing image processing [7] ect. A few studies have also been reported with the use of ICA for texture analysis. Serzer [8] et al. presented a new method for texture classification, using ICA extracts feature of texture image from non-overlapping window and classifying a subwindows as defective or non-defective based on the Euclidean distance. Tsai Du-Ming [9] et al. proposed a constrained version of the ICA model for detecting defect in low-contrast surface images. In recent years, researchers have proposed various methods on fabric defects, such as Fourier transform, wavelet analysis and neural network [10]. In this paper, we introduced ICA as a computation technique for use in fabric defects detection.

2. ICA Algorithm

2.1. Preprocessing for ICA

To simplify the solution of ICA the observed data should be preprocessed appropriately to make it better meet the basic assumptions of ICA. So before applying an ICA algorithm on the data, it usually undergoes preprocessing [1]. In this section we discuss some preprocessing techniques, such as centering, whitening, that make the problem of ICA estimation simpler and better conditioned. Sometimes PCA may also be used to remove the second-order correlation.

2.1.1. Centering

Centralization is the most basic and necessary preprocessing which is used to center the observed signal. Supposed observation vector set as X and the mean of X is m = E(X), the center can expressed as X’ = X−m, X’→X. After estimating centered data, E(X) = E(X’) = E(X−m) = E(X)−m = 0. From the basic ICA model: X = AS, we can obtain E(X) = E(AS) = AE(S), then E(S) = A⁻¹E(X) = A⁻¹m, where m is the mean that was subtracted in the preprocessing.

2.1.2. Whitening

Another useful preprocessing strategy in ICA is whitening the observed data. Applying a linear transformation to make any multidimensional signal into a whited signal process called whitening, which is also known as sphering. One popular method for whitening is to use the eigenvalue decomposition (EVD) of the covariance matrix E(XX^T) = EDE^T, where E is the orthogonal matrix of eigenvalues of E(XX^T) and D is the diagonal matrix of its eigenvectors. The corresponding matrix is called whitening matrix Q = D^−1/2E^T. We can solve half of the problem by whitening.

2.1.3. PCA

Traditional Principal Component Analysis (PCA) can be used as the preprocessing method of ICA to remove the second order redundancy information. PCA is a kind of orthogonal transformation, but there is still the possibility that the existence of some higher-order redundant information which can influence the data that is transformed.

2.2. Classic ICA Algorithm

2.2.1. Natural Gradient Algorithm

The Natural Gradient algorithm (NG) was first proposed by Amari et al. [11], and the cost function is defined as

where W is de-mixing matrix, E is expectation and p_i(y_i(t)) is probability density function. The iterative formula of the separation matrix in Eq. (1) is proposed as follows:

W (t + 1) = W (t) + µ[I − f (y(t))y^T (t)]W (t)

(2)

where µ is the step factor, Eq. (2) updates the amplitude control separation matrix W in each iteration, it affects the convergence speed and stability. Typically, µ generally use the fixed step size. The nonlinear function of separated signal is

Compared with the general gradient, gradientis called natural gradient. Natural gradient algorithm can avoid matrix inverse operation in computational phase, so the algorithm has low computational complexity. Numerical stability is also greatly enhanced while the convergence rate of the algorithm is strengthened to a certain extent.

2.2.2. Infomax Algorithm

The Infomax algorithm [12] was first put forward by the United States Salk Institute Computational Neuroscience Laboratories researchers. In the basic model, the separated mixture signals y(t) can be expressed as

y(t) = Bx(t)

(5)

where B is de-mixing matrix and x(t) is source signal. The model lead into a nonlinear function r_i = g_i(y_i) to replace the estimation of higher order statistics after obtained mixing components y(t) by de-mixing matrix B. The criterion: given appropriate g_i(y_i), we can find the total entropy weight H(r) of r = [r₁, r₂, · · · , r_M ] maximum. The maximum entropy output form as follows: L = H(G₁(y₁), G₂(y₂), · · · , G_n(y_n)). where G_i(y_i) is a class of nonlinear function.

2.2.3. Extended Infomax Algorithm

An extension of the Infomax algorithm of Bell and Sejnowski based on information theory is able to blindly separate mixed signals with sub-Gaussian and super-Gaussian source distributions. This was achieved by using a simple type of learning rule first derived by Girolami by choosing negative entropy as a projection pursuit index [13]. The algorithm is updated according to the following formula:

∆W ∝ [I − K₄ tanh(y)y^T − yy^T ]W

(6)

where K₄ is equal to diag(sign(k₄(y_i)), k₄(y_i) is Kurtosis value, sign( ) is sign function, diag( ) put vector into a diagonal matrix.

The Fast ICA algorithm [14] based on the principle of non-Gauss maximization, using fixed point iteration theory in order to seek the maximum non Gauss of W^T X, which was proposed by Hyvarinen et al. from University of Helsinki in Finland. An algorithm uses Newton’s iterative algorithm to batch process a large number of sampling points of the measured variable X. The approximate Newton iteration formulated as:

W^* = W − [E{Xg(W^T X)} − βW ]/[E{g^'(W^T X)} − β]

(7)

where E[ ] is mean operation, g( ) is nonlinear function, g( ) can be interpreted as g₁(y) = tanh(a₁y), or g₂(y) = y exp(−y²/2), or g₃(y) = y³ etc. Here, 1 ≤ a₁ ≤ 2, we generally take a₁=1.

3. Simulation

This section is made up of two parts: One is about the separation of mixed signal; the other is about the separation of mixed image. The simulation is completed on a computer whose CPU Type is Intel Corei5 2350M and frequency 2.3 GHz.

3.1. Signal Separation

The separation performance of the algorithms mixed in the case of the random mixing matrix was compared by the separation experiment of the mixed sub-Gaussian (s1), Gaussian (s2) and super-Gaussian signal (s3). From top to bottom as shown in Figure 1 the signals are, in order, s1, s2 and s3, which kurtosis value are −0.3738, −0.0096, 2.5145, respectively. In order to effectively compare arithmetic performance, we use the same random mixed matrix W1, which is given by W1=[0.8654 0.5227 0.2278; 0.6126 0.4795 0.4981; 0.9900 0.8013 0.9009]. Figure 2 shows the mixed signals of s1, s2 and s3 with mixed matrix W1. Figure 3, Figure 4, Figure 5, Figure 6 show, respectively, the separated signals y1, y2 and y3 using natural gradient algorithm, Infomax algorithm, extended Infomax algorithm, and FastICA algorithm.

Figure 1. Source Signals (s1, s2, s3).

Figure 2. Mixed Signals.

Figure 3. Separation by NG.

Figure 4. Separation by Infomax.

Figure 5. Separation by ExInfomax.

Figure 6. Separation by FastICA.

It can be observed from the separation results that the Fast ICA algorithm can well recover the original source signal, whereas the other three algorithms cannot finish it well. Although theseparated sequence and source signals sequence are not consistent, the separated signal waveform of the Fast ICA algorithm is conformed to the originals. Compared with the originals, the wave-forms separated by ExInfomax algorithm have a smaller difference, while waveforms by natural gradient and Infomax algorithm have great difference. The running time of the algorithms is figured out by MATLAB which is given in Table 1 so that we can better compare the algorithms separation performance for mixed signal. The kurtosis value of the signal before and after the separation are also worked out and displayed in Table 2.

Table 1. The running time of the algorithms.

Table 2. Kurtosis value of the signal before and after the separation.

Kurtosis Value	Kurtosis Value of s1	Kurtosis Value of s2	Kurtosis Value of s3
Originals	−0.3738	−0.0096	2.5145
NG	−0.1614	0.0030	0.1599
Infomax	−9.0350	1.4640	1.7253
ExInfomax	−1.1937	−1.4835	2.3099
FastICA	−1.4996	−1.0303	2.3614

As seen in Table 2, the algorithms have different performances to cope with sub-Gaussian (s1), Gaussian (s2) and super-Gaussian signals (s3). It shows NG algorithm have better ability to separate it from the mixed signals while ExInformax and Fast ICA make wrong results and Informax tend to give an unsatisfactory separating results when dealing with the sub-Gaussian signal which kurtosis value equal to −0.3738. Besides, the kurtosis value of s2 before the separation is equal to −0.0096, the result of NG equals 0.0030 is closest. The kurtosis value of s3 before the separation is equal to 2.5145, the result of four algorithm is 0.1599, 1.7253, 2.3099, 2.3614, respectively, after the separation, so we can draw a conclusion that the result of Fast ICA is closest. In general, according to the information shown in Table 1 and Table 2, the Infomax algorithm and natural gradient algorithm work more efficiently than Fast ICA algorithm, but their separation effect are worse than it. Furthermore, the effect of the Infomax algorithm is not ideal when sub-Gaussian signal was separated. Compared with several other algorithms, kurtosis values of the separated signals used Fast ICA algorithm are closer to the originals.

3.2. Image Separation

This section is divided into two parts: the first part is the separation of natural gray images; the second part is the separation of fabric defect images. In order to evaluate the performance of the algorithm, the performance index is used.

Performance index [15] can be calculated using the equation as follow:

where g_ij is the i-th row and j-th column element of the global matrix G=WA, max_j|g_ij| represents the maximum value of the i-th row vector of |G|, and max_j|g_ji| represents the maximum value of the i-th column vector of |G|. The performance index is zero, ideally, the separation performance achieved perfect. In fact, PI values around 10⁻² show that the algorithm has a very good performance.

3.2.1. Natural Gray Images

The sizes of test images, from natural image benchmarks, are of 256 pixel × 256 pixel and mixed by random matrix. We used the same random mixing matrix W2 = [0.7922 0.0357 0.6787; 0.9594 0.8491 0.7577; 0.6557 0.934 0.7431] to validly compare the algorithm performance. Figure 7 shows three natural images and Figure 8 mixed images. The separation results are displayed in Figure 9, Figure 10, Figure 11 and Figure 12, respectively. The running time and the performance index are depicted in Table 3.

Figure 7. Natural images.

Figure 8. Mixed images.

Figure 9. Separation by NG.

Figure 10. Separation by Infomax.

Figure 11. Separation by ExInfomax.

Figure 12. Separation by FastICA.

Table 3. The separation performance index and the running time.

Algorithm	Run time (s)	PI
NG	1.24	0.0074
Infomax	0.87	1.0988
ExInfomax	4.41	0.7745
FastICA	0.38	0.0648

From those images, it can be observed that the separated images using natural gradient algo-rithm, extended Infomax algorithm, and FastICA algorithm respectively are consistent with the source image; however, the separation of Infomax algorithm is not well done as shown in Figure 10.

The data in Table 3 shows that running time for separating the natural mixed images with Fast ICA algorithm requires 0.38 s, compared with 4.41 s with ExInfomax algorithm. As is clearly shown above, PI value of Fast ICA algorithm is smaller than Infomax and ExInfomax algorithm. So Fast ICA algorithm separation is excellent. In spite of the best separation effect of natural gradient algorithm for natural image, the calculation time is longer than Fast ICA.

3.2.2. Fabric Defect Images

Three fabric defect images, i.e., coarse pick, cracked end, gout, are selected, which size is all the same as 256×256 and mixed by random matrix. The same random mixing matrix W3=[0.8143 0.3500 0.6160; 0.2435 0.1966 0.4733; 0.9293 0.2511 0.3517] are employed. The original Fabric defect images are shown in Figure 13 and mixed images a in Figure 14. Separated by the algorithm of NG, Infomax, ExInfomax and FastICA, the results are given in Figure 15, Figure 16, Figure 17 and Figure 18. The running time of the algorithms as well as the performance index are presented in Table 4.

Figure 13. Fabric defect images.

Figure 14. Mixed images.

Figure 15. Separation by NG.

Figure 16. Separation by Infomax.

Figure 17. Separation by ExInfomax.

Figure 18. Separation by FastICA.

Table 4. The separation performance index and the running time.

Algorithm	Run Time (s)	PI
NG	4.40	0.1574
Infomax	1.92	0.5393
ExInfomax	10.7	0.8152
FastICA	0.45	0.1387

It can be observed from the experimental results that the Fast ICA algorithm and NG can well recover the original image, whereas the other two algorithms cannot finish it well. Although the separated sequence and source signals sequence are not consistent, the separation image of the Fast ICA and NG are conformed to the originals.

From Table 4, we observed the Fast ICA provides better execution time compared to Infomax, NG and ExInfomax. PI value of the Fast ICA algorithm shows that Fast ICA’s separation is alsoexcellent. The image separation simulation test results of Section 3.2.1 and Section 3.2.2 can be concluded that the Fast ICA algorithm outperforms ExInfomax algorithm, Infomax and natural gradient in terms of image separation.

4. Conclusion

This paper makes a comparative research using four ICA algorithms on signal separation and image separation performance. Experimental results show that the separation performances of four algorithms are different. In order to effectively compare arithmetic performance, we used the same random mixed matrix, respectively. In the aspect of signal separation, Fast ICA can well separate the three mixed signal, while Infomax algorithm for sub-Gaussian signal separation is not ideal. In terms of image separation, Fast ICA separation performance is still the best, and Infomax algorithm can separate the fabric defect image, whereas the natural image cannot be separated completely. Although this paper only introduced the ICA in signal processing and image processing application, the application of ICA is utilized widely. The experiment was conducted to evaluate the performance of those algorithms. At present, there are many ways for defect detection. Compared to other detection method, ICA is a method based on the high order statistics and this method for fabric defects detection is worth further study. Using ICA with other intelligent optimization algorithms for fabric defection is the next thing we need to do and further research is underway. With the constant improvement of ICA theory and further promotion, the ICA technology will surely make more contribution to the fabric defects detection in the future.