Automatic Defect Inspection for Coated Eyeglass Based on Symmetrized Energy Analysis of Color Channels

: Nowadays, eyeglassesare used for vision correction as well as in the fashion industry. Eyeglasses have become more expensive and the eyewear industry has grown rapidly, thereby requiring the development of advanced coating technologies. However, defect detection by visual inspection in the manufacturing process of eyeglass coatings is difficult. To solve this problem, we propose the coated eyeglass defect detection system framework based on machine vision for real-time inspection. First, we locate and extract regions of interest (ROI) ofthe coated eyeglass by adopting cross-projection based on symmetrizedenergy analysis. Next, we propose an efficient method based on the symmetrized energy analysis of color channelsto enhance defectsin each color channel of the ROI of the coated eyeglass. Then, we adoptsymmetrized cross-projection energy analysis for locating defective areas inside the ROI of the coated eyeglass. Finally, we compare the defect detection resultsfor the coated eyeglass with the standard manufacturingquality. An experiment is conducted using real data collected froma Taiwanese eyeglass factory to validate the performance of the proposed framework. This framework achieves a 100% defect detection rate,demonstrating that it is valid and useful for inspecting coated eyeglasses in industries.


Introduction
In Industry 4.0, an automatic optical inspection (AOI) system is a key technique in the manufacturing and testing of products to ensure that the products leaving the production line are high-quality and defect-free. Nowadays, manual inspection is not viable because of the increasing complexity and size minimization of products. Although this approach is widely accepted, obviously, it is not particularly effective, as inspectors soon become tired and may let inferior goods pass through quality control. Because businesses now require high-end products to be brought to the market rapidly and reliably in high volumes, an effective method is required to ensure product quality. AOI is an indispensable tool that uses optics to capture images of the product being tested. This integrated test strategy minimizes costs by detecting faults early in the production line.
AOI has been increasingly used for automatic defect detection in industry product quality control. One mature application of AOI is the inspection of the printed circuit board [1][2][3][4]. Recently, the use of AOI-based defect inspection systems has been extended to several industries, such as steel [5][6][7][8][9][10], textile and garment [11][12][13], solar cells [14][15][16][17][18], plastic [19,20], and liquid crystal display (LCD) [21][22][23][24] industries. Recently, glass is being used increasingly in various applications, and thus, the glass industry is growing rapidly. The human eye cannot keep up with a machine's running speed to identify defects in time, and the reflectivity and transparency of glass make it difficult to examine. Therefore, some researchers have used AOI systems for defect inspection to obtain results faster and more effectively. AOI systems have been developed for different types of glass products, such as float glass [25], satin glass [26], glass bottle bottoms [27], and glass containers [28]. Although automated inspection systems have successfully replaced manual ones, the analysis process must be enhanced to improve assessments and reduce processing times.
Glasses or eyeglasses are used to enable people with poor vision to see clearly. They typically consist of a metal or a plastic frame and two lenses to improve vision. Furthermore, eyeglasses must also be light, thin, and anti-reflective to protect the wearer's eyes from harmful ultraviolet (UV) radiation or glare and provide impact and abrasion resistance. Lightness and thinness can be achieved by using specific materials such as polycarbonates (PCs) to produce eyeglasses. Antireflectivity can be achieved by applying a specific coating on the surface of the eyeglass. Such coatings are made of a very thin film that is layered on the eyeglass; the film has arefractive index between that of air and of glass. Such coatings make eyeglasses more expensive, and therefore, the eyewear industry has actively focused on coating technologies. Various coating technologies exist,however, all require a high heat process, and the coating process still produces many unwanted defects. If a defect is detected early in the high heat process, the product can be corrected by repeating this process until it is satisfactory. However, once exposure starts in the high heat process, an inspector cannot detect defects and manually intercede, making it impossible to recover a coating defect. If a coating defect were to be detected during exposure in the high heat process, the coating process could be stopped immediately. In this study, we propose a coated eyeglass defect detection system (CEDDS) that can efficiently detect defects occurring in the coating process, as shown in Figure 1. If a coating is defective or is scratched, the coating process could be re-performed until the piece passes the quality examination. CEDDS can be used to examine the coating quality in each stage of the coating process, thereby avoiding reprocessing and unnecessary waste. In experiments on 50 pieces of each of the four types of eyeglasses, our system achieved 100% defect detection accuracy, demonstrating its efficiency. The remainder of this paper is organized as follows. Section 2 briefly reviews eyeglass types and their coating processes. Section 3 describes the image acquisition system. Section 4 describes our defect detection method. Section 5 presents the experimental results. Finally, Section 6 presents the conclusions of this study.

Overview of Eyeglass Types and Their Coating Process
Here, we perform experiments on 50 pieces of each of the four types of spherical eyeglasses, for a total of 200 eyeglasses. Each eyeglass type has a different edge thickness (ET) and diameter (Dia.), as shown in Figures2 and 3. Different manufacturing processes can be used to apply a coating onto the surface of an eyeglass. One method involves coating on the eyeglass and then exposing it to high heat for an extended period, thereby causing the liquid solution to harden and adhere to the eyeglass. Another method involves a type of vacuum application in which the eyeglass is placed inside a large machine and then coated with a liquid. Then, the final vacuum procedure combined with high heat helpsadhere the coated layer to the eyeglass. In this manner, the coating is actually built into the material and distributed throughout the eyeglass. The eyeglass coating defect detection standard is defined by two tests before and after applying the eyeglass coating. Before the coating is applied, an inspection is performed and the number of acceptable defects is recorded. After the coating is applied, another inspection is performed to detect defects; if the number of defects is increased, the coated eyeglass is considered substandard and recoating is considered to be required.

Image Acquisition System
This section presents the system architecture of the coated eyeglass inspection apparatus and discusses specific considerations for the illumination scheme. The image properties and the challenges faced in defect detection are then investigated.
In every image acquisition device, lighting conditions play a major role in deciding the quality of the acquired images and therefore in the inspection task. When light meets a glass surface, some of it is reflected, depending on the angle of incidence, the refractive indices of the glass, and the medium the light is originating from (e.g., air). The light passing through the glass is reflected off both the front and the back surfaces. In fact, light may be reflected back and forth several times. When inspecting an eyeglass surface to find defects, the glass background is misleading for the camera. Therefore, the camera should not simply focus on the glass surface. To normalize the illumination variation and increase the contrast between defects and the glass, different lighting systems such as background lighting [29], parallel lighting [30], and infrared lighting [31] have been proposed. However, these methods are unsuitable for application to spherical eyeglasses.
According to our study, the quality of a spherical eyeglass image is influenced by three factors: (1) purity of the background, (2) curvature of the eyeglass, (3) and focal length of the photographic lens, which varies depending on the thickness and curvature of the eyeglass. Furthermore, an uneven coating does not change the brightness of the light owing to the influence of the illumination source. Therefore, to overcome the first two aforementioned factors, a high-brightness projector is adopted as a light source, and then, the detection is performed. The projector lamp minimizes light diffraction and leakage and focuses the maximum possible brightness onto the glasses. With the strong blinding light from the lamp, the interference of ambient light can be avoided completely and the projection color can be changed easily through computer settings to achieve uniformity. The camera takes photographs of the eyeglass on a pure white background screen, as shown in Figure 4. To overcome the abovementioned factor without changing the distances among the projector, screen, and camera, we vary the exposure time of the camera on the basis of the thickness of the eyeglass. Figure 5 shows the proposed image acquisition system, which has been equipped with a servomechanism to overcome the variation of angles since there is curvature on the eyeglass surface. In general, eyeglass defects are small and difficult to detect for the human eye. Therefore, image acquisition is a critical step in the defect inspection system. Although a high-resolution image can help make defects clearer, it increases the processing time required. To solve this problem, we extract the region of interest (ROI) of the coated eyeglass with a quasi-circular shape and then extract defects within this circle. Figure 6 shows examples of images captured using our system for the four types of eyeglasses examined in this study; the images have a resolution of 2330 × 1750 pixels and are stored in 24-bit BMP format. These images are used for defect detection to optimize the running time and ensure synchronization with the factory's production line.

CEDDS
First, the ROI with a quasi-circular shape is extracted automatically to remove background interference and therefore shorten the defect detection time. Next, symmetrized energy analysis is used to enhance the defect contour within the ROI, and then, partial cross-projection is used to cut out the defect image. The defect number and size are finally calculated to determine whether the manufacturer's specifications are met.

Eyeglass Extraction
To extract the correct position of the eyeglass in the image, we detect the contour of the eyeglass by following its quasi-circular shape. Traditional contour detection is based on edge detection approaches [32], such as the Sobel or Canny operator, which commonly extract edges by adopting a specific template or combining a smoothing function. A Sobel edge detector can be employed to find vertical and horizontal edges in an image. Nevertheless, the Sobel edge detector is highly sensitive to noise. Besides, the kernel filters' size and its coefficients are unaltered and cannot be modified to a given image. The Canny edge detector has been developed as an optimal edge detector that provides good performance for detection and localization, and a unique solution for a true edge. It performs better than the Sobel detector and gradient-based operators in most cases. Even so, its performance strongly depends on adjustable parameters, similar to the Gaussian filter's standard deviation and the threshold value. Studies have recently developed various edge detection operators based on the different types of edges [33,34]. However, these approaches still depend heavily on the contrast of the image. They require different parameters to adjust the sensitivity based on the associated content in an image. The symmetrized energy-based edge detector proposed in this study cannot only detect edge pixels equally well in all directions but also afford the concernment of no parameter-tuning, low sensitivity to noise, and isotropy.
Contour detection is usually used to detect important discontinuities for feature extraction and object recognition in digital image analysis. Color is a powerful and strong visual cue to recognize an object from others. Recently, color segmentation has been used widely in computer vision fields, such as visual tracking, object recognition, and vision-based robotics. However, color segmentation exposes color variation incurred by uneven illumination and the position of the camera. Especially, the variation of light and the surface materials of the object is the major problem, limiting color segmentation application to visual inspection tasks. The changing illumination conditions alter the color characteristics, while complex environments increase false-positive pixels. In digital image processing, the YCbCr color space is often used to exploit the lower resolution capability of the human visual system for color with respect to luminosity. Y is the luma component and Cb and Cr are the blue-and red-difference chroma components, respectively. Defects on the eyeglass are relatively more sensitive to illumination. Therefore, we use the luma component Y of the YCbCr color space as a gray level for contour detection. It is more useful for defect detection and can greatly reduce the processing time. Rather than processing the 24-bit coated eyeglass color image, we process the 8bit Y-channel image in the YCbCr color space to reduce the running time of the proposed CEDDS system.
Without generality loss, is assumed to be a coated eyeglass color image with a resolution of M × N, and , ∈ , , represents the RGB color channels. Therefore, ∈ ℜ × × and , , ∈ ℜ × . The luma component offin the YCbCr color space can be calculated as Before contour object extraction is perfumed, a smoothing filter is used to smooth the image and enhance the desired local edge. The local standard average and energy defined by the mask with 3 × 3 pixels are respectively expressed as where = 9 is a normalizing constant, the average value of the pixels in the mask, and ( , )the luma component of the input image in the YCbCr color space.
To extract the object of interest from the background, many methods based on the binary image have been used. For example, the fuzzy binarization has been used successfully in arrow detection in biomedical image problems [35][36][37]. In this study, the binary image based on an automatic threshold proposed by Otsu [38] is adopted to extract the object of interest from the background. Otsu's algorithm for the automatic binary threshold is as follows: where tis the current histogram level value from 0 to 255; ( )the cumulative probability from 0 to t-1, ( )the cumulative probability from t to 255, ( )the cumulative expected average from 0 to t-1, ( )the cumulative expected average from t to 255, and p(i) the probability of distribution in the image. The binary image based on an automatic threshold value , denoted as , is selected from the energy as follows: where pixel values labeled 255 are objects of interest, whereas pixel values labeled 0 are undesired ones. Figure 7 illustrates the resulting binary images for contour detection. The binary image in Figure  7dis used for contour detection. To detect the exact location of the circular coated eyeglass with a circular shape, this study adopts the cross-projection method to determine the detection range. The cross-projection includes two projection mirrors interchangeably to detect marginal points on the contour of the eyeglass in the image. For the binary image , the first projection, called the forward projection, uses the mask

Defect Detection Based on Symmetrized Energy Analysis of Three Color Channels
In the production line, various types of defects can occur on the coated eyeglass. Most defects appear in the B channel, and the rest are located in the G and R channels, as shown in Figure 9. In our experiment, defect enhancement is first performed on each color channel by using local energy analysis. Three energy images of the color channels are then superimposed to obtain the enhanced image for defect detection. The ROI of the coated eyeglass image is first separated into the R, G, and B color channels, and then, the defects are enhanced by symmetrized energy analysis on each color channel separately. Because the ROI of the coated eyeglass with circular shape and the defect may lie near the contour of the coated eyeglass, a circular mask is adopted to solve this problem. The circular mask can not only detect the defect but also localize the contour of the coated eyeglass. The symmetrized energy image _ of each color channel is calculated as where is the number of pixels, the average value of the pixels in the circular mask , d the diameter of the circular mask and an odd number ≥3, and = the radius of the circular mask.
(17) Figure 10a shows the energy image for the ROI of the coated eyeglass. Figures10b-d provides the energy image for the R, G, and B color channels of Figures9b-d, respectively. As shown in Figure  10, the defects are detected accurately. To locate and extract defects within the ROI of the coated eyeglass, we use the cross-projection method as discussed in Section 4.1. First, the contour of the ROI of the coated eyeglass is removed, and then, cross-projection is adopted with the energy analysis result to locate and extract defects. Then, based on the cross-projection result, defects are automatically extracted to detect their numbers, sizes, and coordinates. The manufacturer's specifications are then used to determine whether the defects are acceptable. Figure 11a shows the symmetrized cross-projection result for the ROI of the coated eyeglass, and Figure 11b shows the final defect detection result for the GO/NG decision. As shown, our method can efficiently locate and extract defects on the coated eyeglass.

Experimental Results and Discussion
The defect detection algorithm was tested on the four types of coated eyeglasses. The 200 eyeglass samples had diameters of 70-80mm and edge thicknesses of 2.9-13mm. Each type of eyeglass had a different curvature, as shown in Figures 2 and3.Figure 4 illustrates the optical architecture of our automatic defect detection system and Figure 5 shows the acquisition equipment. The proposed system was implemented in Microsoft Visual Studio C# Express 2010. The experiments were conducted on a PC with an Intel Core i7-4790 CPU @3.60GHz and 8GB of DDR3 RAM running the Windows 10 operating system.
The system used a Basler avA2300-30kc camera with a Camera-Link interface. The camera was equipped with a KAI-4050 CCD sensor delivering 31 frames per second at 4 MP resolution. The sensor size was12.8 × 9.6 mm 2 and resolution (H × V) was 2330 × 1750 pixels, with a pixel size (H × V) of 5.5 × 5.5µm 2 . The camera had a Nikon 60mm f/2.8 microlens. This lens is an excellent normal and short telephoto lens as well as an excellent close-up lens; it can focus on life-size objects, implying that objects as small as an inch across can fill the frame. The light source uses the Epson EMP-1700 projector. The camera, projector, and eyeglass were placed at 680, 715, and 0 mm from the screen, respectively, as listed in Table 1. Table 2 shows the balanced ratio of the light source and the exposure time of the camera based on different thicknesses of the eyeglass.
To inspect whether the coated eyeglass satisfies the manufacturer's requirements, the ROI on the coated eyeglass was first located and extracted. To reduce the processing time, the color image was then transformed to the YCbCr color space to obtain the Y-channel image for contour extraction. On the basis of the symmetrized energy analysis of the Y-channel image, defects were extracted in sequence using cross-projection in both horizontal and vertical directions. The size and number of defects were then calculated against the manufacturer's requirement to judge whether the eyeglass could be passed or whether it required recoating. Section IV discusses the details of this algorithm and Figure 12 shows a diagram of the overall algorithm. Figure 13 illustrates the defect detection results obtained using the proposed method. To reduce the processing time, which has a crucial impact on the production line, we extracted the ROI of the coated eyeglass image, as shown in Figure 13a, and then detected defects in this area. Next, the ROI of the coated eyeglass was enhanced by symmetrized energy analysis of each color channel separately; the result is shown in Figure 13b. Then, the arc contours of the enhanced ROI circles wereremoved for defect detection, as shown in Figure 13c. Next, the defects were accurately detected by applying symmetrized cross-projection for ROI. Defects were correspondingly mapped back to the original image in Figure 13a to obtain the defect detection result on the coated eyeglass, as shown in Figure 13d,e. Table 3 lists the type and size of typical defects on each of the four types of eyeglasses. Before locating and extracting defects for the ROI of the coated eyeglass by using the crossprojection method, three color channels of the coated eyeglass image were computed by symmetrized energy analysis. This pre-process has an important role in our system. To show the effectiveness of the proposed method, we did experiments by usingPugin and Zhiznyakovmethod [39]. Instead of performing a transfer to fuzzy sets in the late stages of processing and analysis, Pugin and Zhiznyakov translated the original image into a fuzzy view in the first step. Table 4 summarizes the average defect detection rates and average running times for each eyeglass piece. Through experimental tests on 200 eyeglass samples, our method has the detection rate of 100%, whilePugin and Zhiznyakov's method is achieved with a detection rate of 98.6%. The difference in results is because some defects have a low contrast between the defect and background and even a tiny defect. Some defects are lost throughout the binarization process. The results confirm that our system can efficiently detect defects on the coated eyeglass,even if they have an extremely small size and cannot be seen clearly by the human eye. Moreover, the running time is short, and therefore, the proposed framework fully meets the manufacturer's requirement.
In this experiment, the defects were provided by the cooperative manufacturer. Therefore, there are maybe some types of defects which we have not yet done. On the other hand, the fuzzy binarization [35][36][37] could be adopted to help produce multiple image layersand we could find more hidden defects in the coated eyeglass to help the manufacturer improve eyeglass product quality. Table 1. Distances among components in the image acquisition system.

Object
The Distance between Two Objects Projector to Screen CCD Camera to Screen Sample to Screen Distance 680 mm 715 mm 0 mm

Conclusions
In this study, we have proposed an effective CEDDS, which consists of two parts, including (1) an optical structure for image acquisition that can remove the environmental lighting interference and then detect a defect and (2) an efficient algorithm based on symmetrized energy analysis to detect defects on the eyeglass. Experimental results for 200 coated eyeglass pieces provided by a manufacturer demonstrated that our system could achieve a 100% defect detection rate. Moreover, inspection of each eyeglass piece required only 6s on average.