Image-Based Scratch Detection by Fuzzy Clustering and Morphological Features

Abstract

Detection of scratch defects on randomly textured surfaces remains challenging due to their unnoticeable visual features. In this paper, an algorithm for piezoelectric ceramic plate surface scratch defects based on the combination of fuzzy c-means clustering and morphological features is proposed. Foreground membership of each gray value is calculated firstly on a reference set of training images by fuzzy c-means clustering and the interpolation method, then an enhanced image is obtained by multiplying the foreground membership function and gray image. The location relationship between regions and the gradient direction of regions is extracted from the binary image of the enhanced image. Based on the morphological features, isolated non-scratched defects are filtered out and the intermittent scratches are merged. Experiments show that the algorithm can be used to detect scratch defects on the surface of a piezoelectric ceramics plate with randomly textured surfaces.

1. Introduction

Detection of piezoelectric ceramic plate surface defects is essential to maintain performance in a range of applications. Scratching is a common surface defect, which takes a lot of manpower to repeatedly detect by the naked eye [1]. Scratches are small in scale and can be different in shape. The detection process is easily affected by illumination and background [2]. It is very difficult to extract features. Therefore, scratch defect detection has always been a difficult problem in product surface quality detection [3]. In order to solve this problem, researchers have done a lot of work [4,5,6]. In machine vision technology, scratch detection has been addressed mainly in the field of image processing by utilizing an optical microscope under good lighting conditions [7,8,9,10,11,12,13], Convolutional Neural Networks (CNN) [14,15,16], and mathematical morphology [17,18,19,20,21,22].

Good and uniform illumination from an external light source is essential for machine vision applications to function. The optimal position of the light source is the lighting incident angle [7,8]. Puntous et al. proposed two thresholds that characterize the visualization of scratches: a visibility threshold as well as an acceptability threshold. The total optical contrast is a combination of the size of the scratch and the contrast in specular and off-specular scattering. A lighting transform method based on a low pass Butterworth filter was first developed to convert the non-uniform intensity distribution on spherical oranges into a uniform intensity distribution over the whole fruit surface [9]. Another dedicated optical system technology is designed for the measurement of surface states by the method of variation of focus. A comparative analysis of the merits and lack of diffractive optics and refractive optics in the problems of the formation of nonparaxial Bessel beams has shown the superior ability of diffractive optics applications in crystal optics [10]. A method with the advantage of being insensitive to both object rotations and pattern orientation has been put forward for solving anomaly detection problems by using reflectance transformation imaging data based on a local assessment of the reflectance across the inspected surface [11].

A deep learning algorithm for surface scratch detection has also been proposed. A SpiralNet framework of a deep learning and optimization method to train the network was proposed for the task of automatic crack detection on highly imbalanced training samples [12]. The algorithm, which automatically detects defects on randomly textured surfaces, was designed by employing three different architectures of convolutional neural networks [13]. A fully connected convolutional encoder–decoder for defect detection in archived video was presented. The proposed method handles the detection of two of the most common archived video-related defects, namely blotches and scratches [14].

Morphological operators are widely used in binary image processing for several purposes, such as removing noise, detecting contours or particular structures, and regularizing shapes [15]. A method with high detection accuracy with extremely fast speed detecting tiny surface defects was proposed based on normal maps of metal parts [16]. The proposed edge detection method based on a two-dimensional (2D)-direction filter and a modified adaptive threshold method was used to automatically detect the edge of the scratches on a filtered image [17]. A multi-scale line detection method was used to efficiently extract shallow scratches [18]. Morphological operations with arc-shaped SEs were proposed without polar transformation, which can achieve high detection accuracy for various small defects, including scratches, bumps, and edge bursts [19]. The method, based on the adaptive sector scanning algorithm (ASC) and cascading mean variance threshold algorithm (MVTH), was designed to detect intermittent scratches on a polished surface [20].

In this paper, the light-field surface detection system is shown in Figure 1. The hardware part is composed of two strip light sources, a lens with little distortion, an industrial camera, and the piezoelectric ceramic plates on a conveyor belt. The industrial camera and lens are mounted directly above the conveyor belt. In order to enhance the quality of images of surface scratch defects, the length of the strip light source is twice as long as that of the ceramic chip, and the strip light source irradiates the ceramic plate at a low angle. A new detection method utilizing the fuzzy c-means (FCM) algorithm and morphological characteristics is presented to extract piezoelectric ceramic plate surface scratches. The method aims to enable automated recognition and a simple offline learning process. The research is focused on detecting surface scratches on a piezoelectric ceramic plate.

In the preparation process of a piezoelectric ceramic plate, when there are improper processing methods, scratches will form on its surface, which degrade the piezoelectric, dielectric, and elastic properties. When working under high voltage, it is easy to crack, which affects the service life and utilization value of the product. In the method proposed in this paper, FCM was used to cluster the foreground and background of the ceramic image offline, and the foreground membership degree of the gray value was calculated. Online, the membership degree and a normalization operation were used to enhance the ceramic image, then the scratch was segmented according to the morphological analysis method. This provides a new method for surface scratch detection on piezoelectric ceramics.

The structure of this paper is as follows. In Section 2, the FCM algorithm and its application in image enhancement are briefly reviewed. Section 3 is devoted to the analysis and extraction of morphological features for binary enhanced images. In Section 4, the outline of the scratch detection working steps and experimental results are presented. Finally, this article ends with a conclusion in Section 5.

2. The FCM Algorithm and Scratch Detection by Growth Area

2.1. The FCM Algorithm

The fuzzy c-means (FCM) algorithm proposed by Dunn [21] and Bezdek [22] is the best known and most used detection method. It is a method for finding groups within data with the most similarity in the same cluster and the most dissimilarity between different clusters [23].

The piezoelectric ceramic plate image I is an ($M$, $N$) 2D matrix. Let $x=\left(x_1,x_2,\cdots ,x_n\right)\prime$ be the dataset constituted by gray values of image I, with $n=M\times N$. The FCM objective function is defined as follows:

$$J\left(u,v\right)={{\displaystyle \sum }}_{k=1}^2{{\displaystyle \sum }}_{i=1}^n{u}_{ik}^m{\left(x_i-v_k\right)}^2$$

(1)

subject to

$${{\displaystyle \sum }}_{k=1}^2{u}_{ik}=1\mathrm{with}{u}_{ik}\in \left[0,1\right],1\le i\le n,1\le k\le 2$$

(2)

where the weighting exponent $1<m<+\infty$ presents the degree of fuzziness, and $u_{ik}$ and $v=\left(v_1,v_2\right)$ are the fuzzy membership of the value $x_i$ in the kth cluster and the cluster center vector, respectively. The optimum solution for minimizing $J\left(u,v\right)$ can be calculated by updating the equations as follows:

$$u_{ik}=\frac{{\left(x_i-v_k\right)}^{2/1-m}}{{{\displaystyle \sum }}_{t=1}^2{\left(x_i-v_t\right)}^{2/1-m}}$$

(3)

$$v_k=\frac{{{\displaystyle \sum }}_{i=1}^n{u}_{ik}^m{x}_i}{{{\displaystyle \sum }}_{i=1}^n{u}_{ik}^m}.$$

(4)

In industrial visual defect detection, the illumination environment is relatively stable. This means that the spatial distribution of the gray value of the same kind of piezoelectric ceramic plate image is relatively fixed, as in Figure 1. Therefore, a dataset composed of gray images can be collected to analyze the foreground membership of the gray value in the image when the light environment and camera are fixed. This process can be completed offline. Figure 2 shows the scratch images under the imaging system shown in Figure 1. Among them, Figure 2a contains four narrow scratches with different lengths, and Figure 2b contains two wide scratches and one narrow scratch. In the online detection process, if 256 (0–255) gray values are not fully covered, the foreground membership of new gray values can be approximately calculated by the interpolation method.

Define the scratch area as the foreground and the non-scratch area as the background. Then, the background and foreground membership of each pixel in the image can be obtained from Formulas (3) and (4). In Figure 3, the FCM algorithm was used to cluster the foreground and background images with scratches in Figure 2. Figure 3a shows the changes in the clustering centers of foreground and background gray values with the number of iterations. It can be found that the clustering centers are basically stable after two iterations. Figure 3b shows the foreground membership of different gray values in the clustering results. It can be found that the membership degree is about 0.5 when the gray value is 117–118.

Let the foreground membership vector of the gray value calculated by the FCM be $f\in R^{256}$, $f_i\in \left[0,1\right], i=1,\cdots ,256$. A new image $IF$, obtained by preprocessing, was composed of the product of the gray value of each pixel of the image and its corresponding foreground membership coefficient $\left(2f_i-1\right)$, such as in Figure 3b. In order to further enhance the foreground gray value in the image, we replaced the product of the gray value and the corresponding foreground membership value by Formula (5), as follows:

$$IF\left(i,j\right)=\{\begin{array}{c}I\left(i,j\right)·(2f_q-1), 1/2\le f_q\le 1, q=I\left(i,j\right)+1\\ 0, 0\le f_q<1/2, q=I\left(i,j\right)+1\end{array}$$

(5)

where $i=1,\cdots ,M, j=1,\cdots ,N$. We standardized image $IF$ by

$$IF\left(i,j\right)=IF\left(i,j\right)\ast 255/\max \left(IF\right).$$

(6)

In Figure 4, the enhanced results for images in Figure 2 were obtained by FCM clustering results and Formulas (4)–(5).

2.2. Scratch Region Segmentation

Binary image $IFB$ came from the image $IF$ with a threshold segmentation parameter of 0.6. The experimental results show that segmentation parameters with 0.5–0.7 had little effect on the later detection results. We then removed all connected components that had fewer than $P$ pixels from the binary image $IFB$. The threshold $P$ was selected according to the actual detection target and accuracy requirements. In the detection of scratches on the surface of piezoelectric ceramic plates, $P=30$ was selected.

3. Scratch Area Growth Based on Multiple Features

In binary image $IFB$, some connected regions are isolated and are non-scratched, which need to be further distinguished. In addition, some shallow scratches are not segmented because of the small gray value. In fact, because scratches are usually caused by cutting tools or other sharp objects passing through the surface of mechanical parts, their shapes show obvious strips and maintain strong directional consistency. The direction gradient can be approximated by the mean slope of two pairs of connected domains and the orientation angle of the long axis of each region. The scratch depth varies with the magnitude of the force applied. Therefore, scratches in the image will appear intermittently and have a non-linked morphology. Some isolated areas may be part of the scratch, while discontinuous areas may be missed.

In order to improve the accuracy of scratch detection, it was necessary to extract the location relationship between regions, the gradient direction of regions, and the local gray distribution around regions [24]. These features will be used to further determine whether the segmentation area is scratched or not.

First, calculate the minimum distance between regions and constitute distance matrix $D$, $D\in R^{s\times s}$, where s is the number of regions in binary image $IFB$, merging the two regions into one if the distance between them is less than distance threshold DT ($D\left(i,j\right)<\mathrm{DT}, i=1,\cdots ,s, j=1,\cdots ,s, i\ne j$).

Second, when the scratches on the surface of the piezoelectric ceramic plate are discontinuous and the discontinuous distance is large, it is not enough to merge the regions depending only on the Euclidean distance. According to the linear characteristics of scratches, the discontinuous scratches are approximately distributed on a line as shown in Figure 5a. In Figure 5a, k represents the slope and b represents the intercept. The line coefficients $\left(k_i,b_i\right),i=1,2,\cdots ,N$ can be calculated by the least square method on every region. Then, some isolated non-scratched regions are excluded by the critical distance clustering method on the set of regional linear parameters. The clustering results are shown in Figure 5b, where the blue part represents the area reserved and the red part represents the area eliminated.

If a region is far away from all other regions, it is judged as an isolated region. Isolated regions may be part of the background or scratch, which needs to be further judged by the gray value and membership distribution in the extended area of its outer rectangle. The extended area is a sector-shaped area along the long and short axis of the outer rectangle, with the center point as the circle point, as shown in Figure 5c. In Figure 5c, R represents the major axis of the circumscribed rectangle, and d represents the minor axis of the circumscribed rectangle. To further determine whether the isolated area is part of the scratch, the search area is expanded. The enlarged area is parallel to the outer rectangular long axis, and the spacing is twice as long as the outer rectangular short axis. If there are isolated areas in the enlarged area, and the statistical expectations and variances of the combined regions exceed the prescribed range, they will be regarded as scratches [25].

To further confirm whether the area is part of the scratch, extract the size of each area, fitting line, mean, skewness, kurtosis, etc. In this study, the length and width of the smallest outer rectangle in the region were chosen as the sizes, and the long axis straight line of the smallest outer rectangle was chosen as the fitting line.

4. Experiments

The algorithm consists of three steps: image enhancement utilizing FCM, detection of abnormal regions based on the morphological features, and judging scratch defects, as in Figure 6. Image enhancement was realized by a membership table lookup and normalization. The membership table was composed of the foreground membership degree and interpolation method of offline FCM clustering. Abnormal region detection was obtained for the enhanced image by the threshold technique, eliminating small areas, and merging and deleting regions based on the mathematical morphology. The area size, average value, and skewness were used as criteria to judge whether the abnormal area was a scratch defect.

In the experiments, the images were gray scale images employing 400 × 800 pixels. The enhanced image was converted to binary by the threshold 179. Then, the area less than 16 in the binary image was deleted. Some regions were merged by the morphological method. Some non-scratched areas needed to be further excluded, as shown in Figure 7. It can be found that the region with scratches can be distinguished by two parameters: the size and the ratio of mean to skewness, as in Figure 8 and Figure 9. According to the application requirements of piezoelectric ceramic plate surface scratch detection, the size and ratio of mean to skewness thresholds were set to 100. According to the size, regions 1, 6, and 8 were scratch areas. According to the skewness, regions 2, 9, 10, and 11 were scratch areas. Figure 10 shows the scratch detection results of the piezoelectric ceramic chip by the method proposed in this paper.

5. Conclusions

In this paper, an algorithm is proposed for the detection of scratch defects on a piezoelectric ceramic plate surface based on a combination of the FCM method and morphological characteristics. The method proposed in this paper has invariance of rotation and translation for the detection results. Through collecting 20 images from different angles and positions of the same ceramic chip and processing them, it was found that the scratch detection results were consistent. The surface of the piezoelectric ceramic chip has the characteristics of an uneven texture and a random gray value, which affected the accuracy of scratch detection. Experiments show that the algorithm can accurately detect the scratch defects from images with an uneven texture and a random gray value. In order to verify the accuracy of the proposed method for scratch detection, image acquisition and processing were carried out on 30 piezoelectric ceramic slices in 10 positions and directions, respectively. The results show that the proposed method achieves a 100% detection rate and a false alarm rate of 5.63%.

Future directions of this work could lead to the development of more types of defect detection. Furthermore, the efficiency of the algorithm will be improved by parallel processing of morphological features extraction.