A Novel High Recognition Rate Defect Inspection Method for Carbon Fiber Plain-Woven Prepreg Based on Image Texture Feature Compression

Carbon fiber plain-woven prepreg is one of the basic materials in the field of composite material design and manufacturing, in which defect identification is an important and easily neglected part of testing. Here, a novel high recognition rate inspection method for carbon fiber plain-woven prepregs is proposed for inspecting bubble and wrinkle defects based on image texture feature compression. The proposed method attempts to divide the image into non-overlapping block lattices as texture primitives and compress them into a binary feature matrix. Texture features are extracted using a gray level co-occurrence matrix. The defect types are further defined according to texture features by k-means clustering. The performance is evaluated in some existing computer vision and machine learning methods based on fiber recognition. By comparing the result, an overall recognition rate of 0.944 is achieved, which is competitive with the state-of-the-arts.


Introduction
Carbon fiber reinforced plastics are widely used in aerospace due to their good fatigue and corrosion resistance coupled with high strength-to-weight and stiffness-to-weight ratios [1]. The predominant material when it comes to composites in structural aircraft components is pre-impregnated carbon fibers [2]. The preparation process of composites means that it is easy to produce bubble and wrinkle defects, and timely inspection of these defects can help improve both the performance of composites and their subsequent service performance.
The inspection methods for examining wrinkle and bubble defects in carbon fiber prepreg are not like the common non-destructive testing methods for composite materials, which are similar to surface texture inspection methods. Unidirectional prepreg and woven prepreg are the most popular raw materials for fabricating composite materials. Unidirectional prepreg is the most commonly utilized material thanks to automatic manufacturing processes such as automated tape laying and automatic fiber placement. With improvement in the degree of automation, defect inspection technologies for unidirectional prepreg have gained popularity as well; these include ultrasonic testing [3], radiographic testing [4], and thermal imaging testing [5]. The structure of woven prepreg is more complicated than that of unidirectional prepreg, and the quality inspection of its preparation process is more difficult. At present, defect inspection for carbon fiber woven prepreg relies heavily on manual inspection, which has the disadvantages of low detection efficiency dependence on the experience of the inspector. Moreover, due to the complex structure of carbon fiber woven prepreg, when defect inspection technologies for unidirectional prepreg are used to examine defects in carbon fiber woven prepreg, the recognition rate of the inspection technologies used with unidirectional prepreg cannot meet the demands of examining carbon fiber woven prepreg. Therefore, it is particularly important to propose a new defect inspection method with a high recognition rate for use with carbon fiber woven prepreg, in particular to allow for online inspection during future automated manufacturing processes.
From a woven fabric texture feature point of view, there are typically four classes [6,7]: structural analysis (SA) [8,9], spectral methods [10][11][12], model-based methods [13][14][15], and statistics-based classification approaches [16][17][18]. Among these, SA assumes that the surface texture is generated by following a placement rule and that defect-frees and defects are respectively composed of overlapped texture primitives and nonoverlapped texture primitives. Therefore, SA is one of the most suitable methods for defect inspection of woven prepreg; its core texture primitives are defined differently in the literature. Bodnarova et al. [10] introduced the definition of texture primitive as texture blobs surrounded by rectangular regions to form an overlapping binarized grid. Ng tested the performance of the valley-emphasis method on common defect detection applications [19]. Jia and Liang [20] proposed a texture blob location-based method which does not directly assume that it conforms to a rigid grid, rather inferring the placement rule dynamically. As stated in [6,7], the requirement of the texture pattern is regular; thus, the locations of defects can be identified through structural analysis.
From the perspective of training strategy, woven prepreg inspection methods are divided into two classes based on either supervised or unsupervised training. Unlike unsupervised machine learning methods, woven prepregs may have unpredictable defect forms in different production processes; thus, manufacturing inspection methods are different from traditional machine learning methods, e.g., image decomposition (ID) [21], motif-based (MB) methods [22], Bollinger bands (BB) [23], regular bands (RB) [24], Elo rating (ER) methods [25], and wavelet-pre-processed golden image subtraction (WGIS) [26]. The ID method inspects defects using the integrating the image decomposition method, which allows for the possibility of removing repeated texture primitives completely and removing segment defects directly [27][28][29]. Another special method, MB [22], is rarely mentioned in the literature on plain-woven fabric inspection; its distinguishing characteristic is the preprocessing step of lattice segmentation. The lattice segmentation method [30] divides a plain-woven fabric image into non-overlapping block lattices which consist of the same textures. The block lattice is similar to the texture primitives in SA, and the lattice segmentation method derives a unified placement rule for images of the same woven fabric patterns which share roughly the same texture for segmented block lattices. The two categories (defect-frees and defects) generalize the categorization of motif-based and non-motif-based methods.
The vision testing and lattice segmentation methods are exploited here to develop a novel plain-woven fabric image analysis method including texture feature compression, which is proposed for synthetically analyzing the texture patterns of carbon fiber plainwoven prepreg and classify them into defect-free, bubble defect, and wrinkle defect classes. Texture features are extracted using a gray level co-occurrence matrix (GLCM). The types of defects are further defined according to their texture features. Finally, k-means clustering (KMC) is used to confirm the types of defects, then detect the quality of prepreg for effective processing. This method can accurately extract the texture feature information of carbon fiber plain-woven prepreg, making inspection results more reliable and realistic.

Image Preprocessing
Prepreg images acquired from via digital cameras are embedded with errors such as noise, fickle shadows, and illumination changes; these can appear similar to defective objects caused in the manufacturing process and affect image quality. Thus, image preprocessing can dampen the bad effects caused by such errors, which is vital in feature extraction [31,32]. Three kinds of preprocessing methods were used here to enhance captured images as well as to determine crossover points or floats (the technical details of these calculations can be found in Appendix A). The main preprocessing methods used were presented as follows.
Step 1. Histogram equalization: The plain-woven texture of the images was equalized by cumulative distribution function (CDF), which is helpful for enhancing their contrast.
Step 2. Gray-level morphology: The bottom-hat image was subtracted from the sum of the original and top-hat images in order to maximize the contrast between the objects and the gaps and distinguish them from each other.
Step 3. Steerable filters: Image edges were obtained by deconvolving the image with templates generated in different directions. Then, Hough transform was used to strengthen the weft yarn in the image, and the image was corrected according to the inclination angle. Meanwhile, performance evaluation metrics were utilized to compare the results of the vertical and horizontal filters.

Image Compression
The woven pattern of carbon fiber plain-woven prepreg has different describable symmetry features depending on the warp and weft yarns. Based on this symmetrical woven pattern, a new image compression algorithm which can remove noise data, retain texture features, and compress and simplify the original image data is proposed here. At the same time, the texture primitive of plain-woven patterns can be extracted from defect-free images. The texture primitive is a unit that can form a whole pattern via simple translation without rotation [33]. In addition, it is a marker for recognizing the woven pattern of carbon fiber plain-woven prepreg.
As shown in Figure 1, taking a defect-free carbon fiber plain-woven prepreg as an example, the image compression process here was carried out as follows. Step 1. Preparation: The original matrix O org can be obtained from a preprocessing image, and the pixel width w and pixel height h can be determined as well. In the process of capturing the surface image of each prepreg, we determined the focal length, FOV, and working distance of the camera and the morphological structure of the prepreg did not change. Therefore, the numbers of warps and wefts in each image could be determined, and are defined here as n x and n y , respectively.
Step 2. Segment horizontally: the original matrix, O org , is segmented horizontally according to the value of the weft, n y . As a result, n y weft-block images corresponding to the matrix of n y weft-block images can be obtained as shown in Equation (2). The number of rows and columns in each matrix is respectively r h and w, where r h = h/n y and x represents the gray value in the weft-block image. Then, the weft-block and original matrices can be expressed as shown in Equations (3) and (4): Step 3. Compute the binary threshold value, t; the elements (gray value) of each column in the matrix W i are added up together and then multiplied by a correction coefficient, λ, as shown in Equation (5). The weft-block matrix W i becomes a new matrix, V i , as shown in Equation (6).
Afterwards, the element sequence of V i is used as the x-coordinate and the element value as the y-coordinate to draw the grayscale transformation graph; in other words, the variations in the element sequence of V i in terms of grayscale values are plotted against the number of pixels. A sub-threshold value, t i , can be easily obtained from the grayscale transformation graph. Similarly, the binary threshold value, t, can be obtained by taking the average of the corresponding thresholds of each obtained V i , as expressed in Equation (7): Step 4. Binary operation: the binary threshold, t, is used to divide the element value of each column vector x i j into binary data (0 or α) in matrix W i , as shown in Equation (8). Thus, the original matrix, O org , and the warp-block matrix W i become the binary matrices O bnr and W i bnr , respectively.
Generally, the pixel values 0 and 255 represent black and white in a grayscale image. For a good data visualization effect, the pixel value α is usually defined as 255.
Step 5. Segment vertically: the binary matrix W i bnr is segmented vertically according to the value of the warp, n x . As a result, n x crossing-block images and the corresponding n x crossing-block matrix C il in each W i bnr can be obtained as shown in Equation (9), with C il being a binary matrix. The number of rows and columns of each matrix C il is r h and r w , respectively, where r h = h/n y , r w = w/n x . Then, the binary warp-block and binary original matrices can be expressed as shown in Equations (10) and (11): Step 6. Black/White classification: the binary proportions of each element are calculated for the crossing-block matrix C il ; the color with a smaller proportion will be replaced by the color with a larger proportion. In terms of a grayscale image, the matrix C il is constituted by either a black block or a white block. A new matrix, O cls , can be obtained by updating the values of the elements of O bnr .
Step 7. Extract the output matrix F: Convolution operations are performed between O cls and convolution kernel K. The kernel K is an r w × r h matrix in which the values of all elements are the same as ζ calculating by Equation (12); the horizontal step is r w and the vertical step is r h . The output matrix F can be obtained from the following Equation (13): where ⊗ is the convolution operator. As a result, the output matrix F is obtained, in which the number of rows is equal to the number of weft yarns and the number of columns is equal to the number of warp yarns; it is a binary matrix that the element value is either zero or one. Thus, the texture feature of a defect-free carbon fiber plain-woven prepreg is clearly mapped by matrix F with the least amount of data.

Texture Feature Extraction
The GLCM is a texture feature extraction method based on gray-level spatial dependence. Each element in the matrix represents the occurrence of a grayscale combination. Assuming that f (x, y) is a two-dimensional digital image with a size of M × N and a gray level of h, the different pixel spacing modeling of GLCM with a certain spatial relationship is expressed as Equation (14) [34]: where # denotes the number of elements in the set, d is the distance between (x 1 , y 1 ) and (x 2 , y 2 ), and θ is the angle between the vector and the axis of the coordinate. In general, d selects 1~8, while θ selects 0 • , 45 • , 90 • , and 135 • . According to the output matrix F provided by the texture feature compression algorithm, only one kind of the texture feature of weft and warp yarns can be extracted in a case where the angle of θ is 45 • or 135 • , as shown in Figure 2. Thus, the values of θ with 45 • and 135 • can be selected in order to improve the contrast of the feature data, thereby facilitating the detection of defect and defect-free images. The co-occurrence matrices of the two conditions are tabulated in Table 1.   [34].

Function
Position Conditions Therefore, texture features can be represented by the values of contrast and homogeneity along with the angular second moment [35,36]. These three secondary statistics can reflect the textural features of carbon fiber plain-woven prepreg.
The schematic for defect-free fiber texture feature extraction is shown in Figure 2; the different colors of the square sections represent the different values of different textural features. The value of the white section is 1, the value of the black section is 0, and the values of dark gray and light gray sections are equal to the calculation results. Taking the defect-free image as an example, whether θ is 45 • or 135 • the size of a matrix is 2 × 2 and one diagonal value is 0. Therefore, the contrast, homogeneity, and angular second moment can be easily obtained using basic arithmetic (the specific operation is shown in Appendix A).

Defect Inspection
Bubble and wrinkle defects are two kinds of typical defects in prepreg during the laying up process. The KMC algorithm was adopted to inspect the textural features in defect-free, winkle defect, and bubble defect cases. By calculating the center of clustering for each condition, the KMC algorithm looked for the best way of grouping images with a minimal value of the mean similarity [37,38]. The accuracy of the KMC algorithm can be improved and verified using training and validation sets with images for the three conditions. Moreover, the KMC algorithm can consolidate the defined defect-type partition [39].

Materials and Experiments
Carbon fiber plain-woven prepreg (WP-3011, Guangwei Composite Material Co., Ltd, Weihai, China) was used for this study. The prepreg was laid on the mold with assistance from a machine hand (KRC4, KUKA Robotics Co., Ltd, Shanghai, China). Then, images were captured using a digital camera (MV-CH050-10UM, HIKROBOT Co., Ltd, Hangzhou, China). The camera captured images of other areas by coordinate movement and stored them in the computer. The computer configuration used was a Windows 7 64x operating system and Inter Core i7-8565U (1.8 GHz) CPU with 8 GB running memory. The process of defect inspection for carbon fiber plain-woven prepreg is shown in Figure 3. The working distance of the camera was 450 mm. Defect sizes were no more than 150 mm × 180 mm, and not smaller than the size of the texture primitive. Representative original images of the defect-free, bubble defect, and wrinkle defect samples are shown in Figure 4.  As a result, 1200 prepreg surface images (200 defect-free images, 500 images with bubbles, and 500 images with wrinkles) were collected as the total dataset and used for improving the proposed plain-woven prepreg defect automatic inspection method. Python 3.7 and OpenCV module were used for the development of the proposed defect automatic inspection method; a flowchart is shown in Figure 5. The enhanced grayscale images of the prepreg were obtained using three preprocessing algorithms. A texture feature compression algorithm was used to compress and simplify the preprocessed images. Then, the texture features of the compressed output matrix F, consisting of contrast, homogeneity, and the angular second moment, were extracted by the GLCM algorithm (1-pixel and 45 • , 135 • directions). The mean value of two directions was utilized as the texture feature. These three features formed a three-element array as the input of the KMC algorithm. In the end, the results of clustering recognition were obtained.

Results and Discussion
As shown in Figure 6, a series of preprocessed defect-free images were used to describe the image preprocessing steps. The original grayscale image is shown in Figure 6a. To enhance the contrast in the original grayscale image, the plain-woven texture of the image was equalized by CDF, as shown in Figure 6b. Then, the histogram-equalized image was filtered through a box filtering function to reduce noise, as shown in Figure 6c. After that, a top-hat transform (as shown in Figure 6d) and bottom-hat transform (as shown in Figure 6e) were applied to the filtered image to obtain the gray-level morphology, as shown in Figure 6f. The gray-level morphology algorithm allowed us minimize the effects of tensile fiber on fiber texture extraction. Finally, a steerable filtering algorithm, which included a horizontal steerable filter (as shown in Figure 6g) and a vertical steerable filter (as shown in Figure 6h), was used to obtain the clear edges of each yarn, with the templates generated in different directions by the deconvolving method. With this comparison, the vertical steerable image was selected for the Gaussian filter to complete the whole image preprocessing step, as shown in Figure 6i.
The comparisons between the original gray and the histogram-equalized images and histograms in three conditions (defect-free, bubble defect, and wrinkle defect) are shown in Figure 7. By comparing the images, it can easily be seen that the histogram equalization equalizes the brightness of the original grayscale image caused by the reflective resin on the prepreg surface and improves the clarity of the plain-woven texture. By comparing the histograms, it can be seen that a uniform distribution of the gray level was obtained, which is helpful for extraction of texture features.  The gray-level morphological operation processing is shown in Figure 8. An area of 10 (warp yarns) × 10 (weft yarns) was intercepted for image observation. The weave points of warp and weft yarns in the image were regarded as mountains and valleys. The contrast was improved by minimizing the number of valleys. The top-hat and bottom-hat images contained the mountains and the valleys of the yarns, respectively. Thus, gray-level morphological operation processing was carried out by adding the top-hat operated image to the original grayscale image and then subtracting the bottom-hat operated image from the sum. Following this gray-level morphology process, the gray-level morphological image was obtained, which was similar to the ideal image. A steerable filter was adopted to filter the gray-level morphological image in order to emphasize the borders of warp and weft yarns in the final preprocessing step. The effects of vertical filtering and horizontal filtering are compared in Figure 9. The performance evaluation metrics for the vertical and horizontal steerable filtered images are shown in Table 2. From Table 2, it can be seen that (i) the ACC values of the vertical steerable filter are much higher than those of the horizontal steerable filter; (ii) the TPR values of the horizontal steerable filter in the full-size image are very low, specifically, 41.64% below the maximum value; and (iii) the FPR values of the vertical steerable filters are much lower than those of the horizontal steerable filters. Therefore, the vertical steerable filtered images were selected to improve recognition accuracy before the final preprocessing step.  An image compression algorithm was proposed to compress and simplify the preprocessed image data while retaining the texture features for further analysis; the image compression process is shown in Figure 10. A preprocessed image (shown in Figure 10a) was segmented horizontally (shown in Figure 10b) based on the number of weft yarns, then used to compute the binary threshold value for threshold classification (shown in Figure 10c). The binary sub-image arrays were then segmented vertically based on the number of warp yarns to obtain crossing-block image arrays as the preliminary texture pattern, which are shown in Figure 10d. Then, a dichotomy operation was carried out for the Black/White classification process, as shown in Figure 10e. After that, the crossingblock image arrays were merged to obtain the final texture pattern, as shown in Figure 10f. Using a convolution operation, the final texture pattern was converted into the output matrix, F (obtained by Python and shown in Figure 10g), which was used as a standard defect-free template for further analysis. Following the same image compression process, the texture patterns of the original images of carbon fiber plain-woven prepreg with and without defects (corresponding to Figure 4) were obtained, and are shown in Figure 11.  Table 3. Three texture feature parameters, namely, the contrast and homogeneity along with the angular second moment, made up a three-dimensional array which was used to present the texture features.
The KMC algorithm classified 180 samples by three conditions, defect-free (60 samples), with bubble defect (60 samples), and with wrinkle defect (60 samples), by recognizing different representative defect-free, bubble, and wrinkle texture features of carbon fiber plain-woven prepreg samples randomly selected from the 1200 total samples. The results were plotted as a three-dimensional scatter plot, shown in Figure 12. Each texture feature of these three conditions has distinguishable differences, which are defined according to their locations in the plot. Figure 11. The texture patterns of the original images of carbon fiber plain-woven prepreg with and without defects, corresponding to Figure 4.  The KMC algorithm classified 180 samples by three conditions, defect-free ples), with bubble defect (60 samples), and with wrinkle defect (60 samples), by r ing different representative defect-free, bubble, and wrinkle texture features of c ber plain-woven prepreg samples randomly selected from the 1200 total samples sults were plotted as a three-dimensional scatter plot, shown in Figure 12. Each feature of these three conditions has distinguishable differences, which are def cording to their locations in the plot. Moreover, a total of 1000 samples based on the same three conditions (def bubble, and wrinkle) with proportions of 2:3:5, 2:4:4, and 2:5:3, respectively, were verify the recognition rate. The plots for each proportion are shown in Figure 13. tial and final center points of the clustering results are shown in Tables 4 and 5 tively. The overall recognition accuracy (IE) listed in Table 6 is 94.41%, where TN, TW are the number of defect-free, bubble and wrinkle samples, RN, RB, and RW number of correctly recognized data elements, IN, IB, and IW are the recognition a and IM is their average. Moreover, a total of 1000 samples based on the same three conditions (defect-free, bubble, and wrinkle) with proportions of 2:3:5, 2:4:4, and 2:5:3, respectively, were used to verify the recognition rate. The plots for each proportion are shown in Figure 13. The initial and final center points of the clustering results are shown in Tables 4 and 5 respectively. The overall recognition accuracy (IE) listed in Table 6 is 94.41%, where TN, TB, and TW are the number of defect-free, bubble and wrinkle samples, RN, RB, and RW are the number of correctly recognized data elements, IN, IB, and IW are the recognition accuracy, and IM is their average.  When texture feature extraction and defect type inspection was performed only by GLCM and KMC the recognition accuracy was 91.96%, as shown in Table 7. At the same time, the image processing time is also slower than that of the above method, as shown in Table 8. In contrast with general image preprocessing and the GLCM algorithm, this paper further refines the processing of texture features. Therefore, the recognition accuracy can be significantly improved in different proportions, as shown in Figure 14. Furthermore, the proposed method was compared with several existing computer vision and machine learning methods based on fiber recognition, with the results shown in Figure 15. Despite the different data sets employed by the above methods, their final purpose is ultimately to effectively recognize fiber morphologies and defect types. The proposed method can describe the morphological characteristics of defects in carbon fiber prepreg more comprehensively and more simply, and shows a remarkable improvement recognition accuracy; the results show that the proposed methodology is competitive or better than these traditional techniques. The three kinds of surface morphologies (defectfree, bubble, and wrinkle) in carbon fiber prepreg were all recognized effectively.    [40], SDAE [41], LSTS [30], BB [23], ER [25], RB [24], WGIS [26], NN [42]).

Conclusions
Here, we have proposed a novel defect inspection method to improve the efficiency pf automatic online detection of defects in carbon fiber plain-woven prepregs, which can help in developing an automatic laying-up process in their production. The proposed method compresses a plain-woven prepreg image while reserving additional texture features, which are calculated using GLCM. These features can be regarded as a three-dimensional array and used as the input of the KMC algorithm, which is then applied to define the defect types and realize defect inspection. It should be noted that the performance of the proposed method was compared to several different inspection methods, it showed significant improvement; the recognition rate and performance time of the proposed method reached 94.41% and 136 ms, respectively. Moreover, this proposed method could be used for surface defect identification in other woven patterns, such as twill and satin.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author.

Appendix A
Let us recall that when the prepreg images captured in the process of automatic placement need to be enhanced by three preprocessing methods: (1) histogram equalization; (2) gray-level morphology; (3) steerable filters; (4) Performance evaluation metrics; (5) Gray level co-occurrence matrix. The specific algorithm implementation of these methods is as follows.
(1) Histogram equalization: Firstly, the global image is equalized directly. Suppose that the histogram distribution of the captured image A is H A (D). The monotone nonlinear mapping is used to change image A into image B, that is, function transformation f is applied to each pixel in image A, and the histogram of image B is obtained as H B (D).
The whole process can be understood as changing all the D A in image A into D B as follows: In order to achieve histogram equalization, the following are in particular: Because the target is histogram uniformly distributed, we obtain H B (D) = A 0 /L, where A 0 is the number of pixels, and L is the grayscale depth (usually 256). And then we get: Therefore, let us define f : The discrete form is: Then the histogram distribution in the local area window is used to construct the mapping function to equalize the local image area.
Finally, in order to avoid the discontinuity and excessive enhancement of the image, the interpolation method is used to accelerate the histogram equalization. Bilinear interpolation formula is as followed. (2) Gray-level morphology: The operation object of gray-level mathematical morphology is not a set, but an image function. The dilation and erosion operations of the input image f(x, y) with structure element b(x, y) are respectively defined as: Opening and closing operations are respectively defined as: The operations of top-hat and bottom-hat are respectively defined as: Therefore, we obtain the final gray-level morphology preprocessing formula: (3) Steerable filters: Considering the woven pattern, we use the steerable vertical filters with the following basic 2-D Gaussian function in this paper.
f (x, y) = 1 2πσ x σ y exp − x 2 2σ x 2 + y 2 2σ y 2 (A14) where σ x and σ y are the standard deviation in the x and y directions, respectively.
Rotation matrix (Equation (A15)) is used to rotate the axis in accordance with a necessary angle θ into the function.
Therefore, the steerable vertical filter algorithm is derived from Equation (A16). After defining appropriate parameters of σ x , σ y and θ, the filter is convolved with abovementioned morphology enhanced image.
(5) Gray level co-occurrence matrix: All values of the co-occurrence matrices are given normalized treatment before the texture features are calculated. The co-occurrence matrices, M ij , is shown as Equation (A22).
Contrast, homogeneity and angular second moment are used to describe the carbon fiber texture features.
Contrast: The contrast reflects the sharpness of the image and the depth of the texture patterns. The pattens is deep, the image is clear, and the contrast is large. On the contrary, the patterns are shallow, the image is fuzzy, and the contrast is small.
Homogeneity: The homogeneity measures how much the image texture changes locally. The larger the value is, the less variation of different regions of the image texture is, and the local texture is very uniform.
Angular second moment (ASM): ASM is the sum of squares of element values in the gray co-occurrence matrix. It reflects the degree of texture thickness and the degree of evenness of image gray-level distribution. When the value is larger, gray-level distribution is uniform and image is closed-grain. Otherwise, it is smaller.