First, the ROI with a quasi-circular shape is extracted automatically to remove background interference and therefore shorten the defect detection time. Next, symmetrized energy analysis is used to enhance the defect contour within the ROI, and then, partial cross-projection is used to cut out the defect image. The defect number and size are finally calculated to determine whether the manufacturer’s specifications are met.
4.1. Eyeglass Extraction
To extract the correct position of the eyeglass in the image, we detect the contour of the eyeglass by following its quasi-circular shape. Traditional contour detection is based on edge detection approaches [
32], such as the Sobel or Canny operator, which commonly extract edges by adopting a specific template or combining a smoothing function. A Sobel edge detector can be employed to find vertical and horizontal edges in an image. Nevertheless, the Sobel edge detector is highly sensitive to noise. Besides, the kernel filters’ size and its coefficients are unaltered and cannot be modified to a given image. The Canny edge detector has been developed as an optimal edge detector that provides good performance for detection and localization, and a unique solution for a true edge. It performs better than the Sobel detector and gradient-based operators in most cases. Even so, its performance strongly depends on adjustable parameters, similar to the Gaussian filter’s standard deviation and the threshold value. Studies have recently developed various edge detection operators based on the different types of edges [
33,
34]. However, these approaches still depend heavily on the contrast of the image. They require different parameters to adjust the sensitivity based on the associated content in an image. The symmetrized energy-based edge detector proposed in this study cannot only detect edge pixels equally well in all directions but also afford the concernment of no parameter-tuning, low sensitivity to noise, and isotropy.
Contour detection is usually used to detect important discontinuities for feature extraction and object recognition in digital image analysis. Color is a powerful and strong visual cue to recognize an object from others. Recently, color segmentation has been used widely in computer vision fields, such as visual tracking, object recognition, and vision-based robotics. However, color segmentation exposes color variation incurred by uneven illumination and the position of the camera. Especially, the variation of light and the surface materials of the object is the major problem, limiting color segmentation application to visual inspection tasks. The changing illumination conditions alter the color characteristics, while complex environments increase false-positive pixels. In digital image processing, the YCbCr color space is often used to exploit the lower resolution capability of the human visual system for color with respect to luminosity. Y is the luma component and Cb and Cr are the blue- and red-difference chroma components, respectively. Defects on the eyeglass are relatively more sensitive to illumination. Therefore, we use the luma component Y of the YCbCr color space as a gray level for contour detection. It is more useful for defect detection and can greatly reduce the processing time. Rather than processing the 24-bit coated eyeglass color image, we process the 8-bit Y-channel image in the YCbCr color space to reduce the running time of the proposed CEDDS system.
Without generality loss,
is assumed to be a coated eyeglass color image with a resolution of
M ×
N, and
represents the RGB color channels. Therefore,
and
. The luma component
of
f in the YCbCr color space can be calculated as
Before contour object extraction is perfumed, a smoothing filter is used to smooth the image and enhance the desired local edge. The local standard average
and energy
defined by the mask with 3 × 3 pixels are respectively expressed as
where
is a normalizing constant,
the average value of the pixels in the mask, and
the luma component of the input image in the YCbCr color space.
To extract the object of interest from the background, many methods based on the binary image have been used. For example, the fuzzy binarization has been used successfully in arrow detection in biomedical image problems [
35,
36,
37]. In this study, the binary image based on an automatic threshold proposed by Otsu [
38] is adopted to extract the object of interest from the background. Otsu’s algorithm for the automatic binary threshold
is as follows:
where
t is the current histogram level value from 0 to 255;
the cumulative probability from 0 to
t−1,
the cumulative probability from
t to 255,
the cumulative expected average from 0 to
t−1,
the cumulative expected average from
t to 255, and
p(
i) the probability of distribution in the image. The binary image based on an automatic threshold value
, denoted as
, is selected from the energy
as follows:
where pixel values labeled 255 are objects of interest, whereas pixel values labeled 0 are undesired ones.
Figure 7 illustrates the resulting binary images for contour detection. The binary image in
Figure 7d is used for contour detection.
To detect the exact location of the circular coated eyeglass with a circular shape, this study adopts the cross-projection method to determine the detection range. The cross-projection includes two projection mirrors interchangeably to detect marginal points on the contour of the eyeglass in the image. For the binary image
, the first projection, called the forward projection, uses the mask
M1 from the top left to the bottom right of the image to obtain the maximum value coordinate
in the projection. The second projection, called the reverse projection, uses the mask
M2 from the bottom right point to the top left point of image to obtain the minimum value coordinate
in the projection. The masks
M1 and
M2 are defined in Equations (10) and (11), respectively.
Figure 8 presents the results with both forward and backward projections. The center point of the coated eyeglass, denoted as
O(
x0,
y0), is then calculated as
On the basis of the center point
O of the overlapping circles shown in yellow color, as shown in
Figure 8d, we can precisely extract the ROI of the coated eyeglass with a circular shape shown in aqua blue color, as shown in
Figure 8e.
4.2. Defect Detection Based on Symmetrized Energy Analysis of Three Color Channels
In the production line, various types of defects can occur on the coated eyeglass. Most defects appear in the
B channel, and the rest are located in the
G and
R channels, as shown in
Figure 9. In our experiment, defect enhancement is first performed on each color channel by using local energy analysis. Three energy images of the color channels are then superimposed to obtain the enhanced image for defect detection.
The ROI of the coated eyeglass image is first separated into the
R,
G, and
B color channels, and then, the defects are enhanced by symmetrized energy analysis on each color channel separately. Because the ROI of the coated eyeglass with circular shape and the defect may lie near the contour of the coated eyeglass, a circular mask is adopted to solve this problem. The circular mask can not only detect the defect but also localize the contour of the coated eyeglass. The symmetrized energy image
of each color channel is calculated as
where
is the number of pixels,
the average value of the pixels in the circular mask
,
d the diameter of the circular mask and an odd number ≥3, and
the radius of the circular mask.
The circular mask with diameter of 5 pixels is presented as
After enhancing defects in each color channel based on
, the energy image of the ROI of the coated eyeglass, denoted as
, is calculated using the OR operator as
Figure 10a shows the energy image for the ROI of the coated eyeglass.
Figure 10b–d provides the energy image for the
R,
G, and
B color channels of
Figure 9b–d, respectively. As shown in
Figure 10, the defects are detected accurately.
To locate and extract defects within the ROI of the coated eyeglass, we use the cross-projection method as discussed in
Section 4.1. First, the contour of the ROI of the coated eyeglass is removed, and then, cross-projection is adopted with the energy analysis result to locate and extract defects. Then, based on the cross-projection result, defects are automatically extracted to detect their numbers, sizes, and coordinates. The manufacturer’s specifications are then used to determine whether the defects are acceptable.
Figure 11a shows the symmetrized cross-projection result for the ROI of the coated eyeglass, and
Figure 11b shows the final defect detection result for the GO/NG decision. As shown, our method can efficiently locate and extract defects on the coated eyeglass.