An Automatic Foreign Matter Detection and Sorting System for PVC Powder

Abstract

In the present study, an automatic defect detection system has been assembled and introduced for Polyvinyl chloride (PVC) powder. The average diameter for PVC powder is approximately 100 μm. The system hardware includes a powder delivery device, a sieving device, a circular platform, an image capture device, and a recycling device. A defect detection algorithm based on YOLOv4 was developed using CSPDarkNet53 as the backbone for feature extraction, spatial pyramid pooling (SPP) and path aggregation network (PAN) as the neck, and Yoloblock as the head. An auto-annotation algorithm was developed based on a digital image processing algorithm to save time in feature engineering. Several hyper-parameters have been employed to improve the efficiency of detection in the process of training YOLOv4. The Taguchi method was utilized to optimize the performance of detection, in which the mean average precision (mAP) is the response. Results show that our optimized YOLOv4 has a test mAP of 0.9385, compared to 0.8653 and 0.7999 for naïve YOLOv4 and Faster RCNN, respectively. Additionally, with the optimized YOLOv4, there is no false alarm for images without any foreign matter.

1. Introduction

Polyvinyl chloride (PVC) is a polymer material obtained by the additional polymerization of vinyl chloride. It is the third most widely synthetic polymer after polyethylene and polypropylene [1]. PVC is synthesized with the addition of pure water, liquefied vinyl chloride monomer (VCM), and dispersant to the reactor following the addition of the initiator and other auxiliary agents in the suspension polymerization process. The PVC powder is generated as the VCM undergoes radical polymerization when a high temperature of the reactor is reached. During the formation process, continuous stirring is required to obtain a uniform particle size suspended in water. After removal of residual VCM through a stripper, PVC slurry is subjected to a drying process, followed by sieving before being loaded into packages. A flow diagram of the PVC production process is presented in Figure 1.

However, some issues remain in the PVC manufacturing process. For examples, the presence of choking occurs due to local high temperature, or rust in the piping system during the manufacturing process. If these faults are not excluded, the yield of related polymer products could be affected. Usually, two-step quality inspections, magnetic and visual selections, are performed before packing and shipping. After the magnetic selection of rust (black color) is carried out by pouring a certain amount of PVC powder into a funnel and passing it through a magnetic bar (as shown in Figure 2a), non-magnetic slags (dark pink color) are inspected with visual observation by pouring the same powder onto a white paper and viewing through a magnifying glass with a brush and puffed air under strong backlight. Non-magnetic slags are picked out with a marker on white paper as an inspection report, as shown in Figure 2b. Because it is difficult for human eyes to identify non-magnetic slags in PVC powder with a particle size ranging from 120 to 150 μm, the inspection time required is about 30 min for a pack of 500 g of samples. In addition, the inspectors will be troubled by persistence of vision (POV) due to constant staring at similar background textures, which may cause eye injuries in the long term.

In the manufacturing process of PVC powder mentioned above, a solution that helps to reduce the overall cost is to change the inspection with human eyes to machine vision inspection before packaging and shipment, so that there will be an opportunity to improve the quality of inspection and reduce work injuries and other related problems. Root cause analysis can be conducted by tracing the time of occurrence of foreign matter and analyzing its characteristics. These results can be utilized as feedback to the processing personnel for machine parameter adjustment.

Studies of foreign matter detection in powder can be found from over 30 years ago [2,3,4]. However, there were few related investigations after those patents had been issued. For the last decade, research on the detection of foreign matter in powders has been widely conducted in the field of crushed materials such as minerals, plastics, metals, and dry powders. These investigations were probably performed due to the expiration of the patent rights for these early inventions and the conversion of related knowledge into public property. In these studies, the powders vary in size and color. Nevertheless, the main objective is to detect foreign particles or pollutants during the manufacturing process. In addition, food powder inspection is another issue that is a concern of food safety. The purpose of food powder inspection is to identify environmental hormones or additives harmful to the human body mixed into the food. Shimazu and Morikawa [5] proposed a continuous supply of quartz powder raw material to take the color images for quartz using a charge-coupled device (CCD) with a scattered light in a certain wavelength. They found that quartz powder showed a polygonal glass-like surface. Additionally, quartz powder can be identified by the shadow of the polygon edge with the existence of light-yellow foreign matter. Huang et al. [6] used near-infrared spectroscopy (NIRS) to take images of skimmed milk powder to investigate the distribution of inorganic nutritional additives, zinc sulfate and lactose, and melamine with principal component analysis (PCA), correlation analysis, regression analysis, and other techniques. Prabhu and Mascles [7] took images of ultra-high molecular weight polyethylene (UHMWPE) powder with CCD to distinguish foreign matter from UHMWPE powder based on the characteristics of color, size, and density, and they removed this foreign matter through a vacuum or mechanical sorting system. Li et al. [8] analyzed the Compton scattering image with X-Ray scattering imaging technology. They found that the normalized density value of foreign matter is higher than that of milk powder, such that the images of foreign matter appear to be brighter. Mauruschat et al. [9] used NIRS to capture hyperspectral images for classification with PCA and soft independent modeling of class analogy (SIMCA). In their experiment, wood plastic mixed with contaminants was tested. A second sorting step for the purified fraction helped to further reduce the contamination level, and a final purity of 98.9% was obtained. Ding et al. [10] utilized images of freeze-dried powder captured by a CCD in a back-propagation neural network (BPNN) and support vector machine (SVM) for foreign matter classification with PCA for feature extraction. Their experimental results showed that the recognition rate based on the PCA and SVM algorithms is higher than that based on PCA and the third-nearest neighbor classification algorithm. Later, Zhang et al. [11] employed the same digital image technology as Ding et al. [10], with 3 × 3 median filtering for pre-processing, and they demonstrated that integrating PCA and SVM could be used to classify foreign bodies in lyophilized powder. Zhou et al. [12] utilized computed tomography (CT) to take image of metal powder in an inclusion module receiving samples from a powder source. These 2D images could then be inspected for foreign objects through the segmentation process. After the reconstruction process, 3D images could be established to measure and mark the size or shape of foreign objects. Based on the literature mentioned above, there have been several applications in the development of imaging technology. In addition to observing the distribution of foreign matter in powder particles, the counting or measuring of foreign matter, as well as identification, are performed.

However, there will be problems encountered because the operator employs the existing powder inspection mechanism based on the patents mentioned above, or on research articles and existing commercial powder inspection machines (Tarrach [13]; QICPIC, Sympatec GmbH, Clausthal-Zellerfeld, Germany):

• Manual annotation is a time-consuming process, and the criteria for foreign matter are inconsistent. Before training the model, the collected foreign matter samples need to be manually annotated as the ground truth of the training sample. However, the operators usually do not have enough time to perform time-consuming annotation work in the on-site environment. Additionally, the annotation process varies from person to person due to the variations of foreign objects, inconsistent particle sizes, or similar characteristics between categories.
• Manual feature characterization is time-consuming and a poor adaption: all the methods mentioned earlier utilize the features of the object to identify whether or not the foreign matter is embedded in the powder. Therefore, it takes a significant amount of time for the operator to find the most distinguishing feature combinations, such as the diameter, shape, and color of the foreign matter. The operator has to restart the feature engineering process as long as the particles and foreign objects are being altered. Usually, the manual features are not capable of handling exceptions such as lighting changes, distortion, occlusion, and deformation, leading to a low adaptability of the algorithm.
• Model training takes longer, and finding superior hyper-parameters is less efficient: there are many hyper-parameters that need to be set in training a model. These hyper-parameters are directly related to the elasticity of the model learning and affect the degree of generalization of the model in inference. However, due to the time-consuming nature of training a single deep learning model, finding the best hyper-parameter combination of a deep learning model manually would take a significant amount of time.

This present research will take PVC as an example to develop the integration mechanism of a powder-grain imaging machine and a foreign matter detection algorithm. For the three practical problems mentioned previously, the following mechanisms will be developed. The main objective of the present investigation, taking PVC powder as an example, is to develop a powder imaging machine integrated with the foreign matter detection algorithm. There are three features of the machine required to solve the three practical problems mentioned earlier:

• First, the auto-annotation and correction mechanism: before performing supervised learning in defect detection applications, it is often necessary for several operators to manually annotate samples. This is a tedious and time-consuming task. Therefore, most of the annotation work is outsourced to other companies for processing, or existing datasets are purchased, which makes the cost of deep learning research extremely high. There are some tools that can be used to annotate images based on pre-trained models, such as Anno-Mage and easyDL intelligent labeling. These methods are categorized as predictive auto-annotation generation. The general procedures are as follows: First, a preliminary model, trained with a small batch dataset, is used to test a small batch of unlabeled samples. Secondly, after the detection results have been corrected by human intervention, a new model is trained, established with the corrected dataset. Thirdly, the new model is used to detect a larger batch of datasets. Through the repeated calculation of the above steps, the accuracy of the annotation can be gradually refined. In addition, Chandra et al. [14] developed an auto-annotation module for the classification of normal and abnormal brain MRI images. In their study, a gray-scale histogram of an unlabeled image was employed to depict brain MRI image features, and then the most similar labeled image was searched for through image correlation analysis as the category of the unlabeled image. However, they also mentioned that the quality of auto-annotation is still far from desired expectations. Bharara [15] proposed an auto-annotation system for the location of abnormal patterns in 3D visual images. His system used historical data to identify normal and abnormal patterns after the sensor receives the data. The position of the abnormal pattern in the generated visual image is then annotated and exported to the display. There is also another method called generative auto-annotation generation that generates the objects that need to be detected through the help of simulation software or algorithms. Zhang et al. [16] developed a detection mechanism for wild forest fire smoke. A synthetic smoke image is generated by inserting real smoke or virtual smoke generated by Blender into the forest background to solve the problem of lack of fire data. In addition, the problem of annotation is solved because the position of the smoke is identified during the process of inserting the smoke. Li et al. [17] developed a component segmentation mechanism for printed circuit board (PCB) identification. Their training image set is automatically synthesized through rendering based on a 3D model of the PCB. Although the color labeling of components still requires manual intervention, automatic color labeling is realized through CAD secondary development so that the image training in industrial production can be completely automatically generated in the future. On the other hand, there is also research studying generative auto-annotation generation with generated virtual images for the sake of saving annotation time. Chen et al. [18] detected a patch of particle defects on a wafer die using a few manual annotation files, and they used the defect patch after removing the background into the generative adversarial network (GAN) to generate realistic virtual defects. These virtual defects were then randomly pasted onto a die image without defects. Because the size and position of the virtual defects were known, an annotation file could then be automatically generated. Chen and Tasi [19] also expanded the number of virtual images through existing manual annotation files, especially for surface defects of SMD LEDs for image block extraction. The virtual defects were generated through performing random morphological and affine transformations with the defect batches after removing the background. These virtual defects were then randomly sampled and pasted onto defect-free LED chips and automatically annotated. From the literatures mentioned above, auto-annotation can be applied to real or virtual images. Although auto-annotation can greatly reduce the workload, these processes still require human participation. In the present investigation, an automatic annotation program is developed for real defect images, saving the time spent annotating real defects using a series of digital image processing (DIP).
• Foreign matter detection with deep learning model: The convolutional neural network (CNN) in deep learning frameworks is impressive regarding its ability of self-taught feature learning. It is also able to cope, to a certain degree, with the exceptions mentioned earlier. Therefore, CNN has received widespread attention in the field of defect detection because of its application in defect classification, defect detection, and defect segmentation. The most popular application of CNN is defect detection from the perspective of patch level annotation, the time for training the model, and information feedback. Li et al. [20] performed defect detection for four types of defects on the surface of canned containers. After image augmentation, they fed these images into SSD with MobileNet as the backbone. They found that the lightweight MobileNet backbone improved the accuracy and rate of identification. Li et al. [21] performed the surface detection of rails with YOLOv3 using DarkNet53 as the backbone, along with the concept of feature pyramid networks (FPNs) for multi-scale detection in order to detect small defects. Lin et al. [22] employed CNN to identify three types of defect on LED chips and utilized class activation mapping (CAM) to locate the defects. Sun et al. [23] carried out surface detection on a wheel hub with modified Faster R-CNN by using a ZF network as the backbone and replacing the maximum pooling layer with the sliding convolution layer. Dai et al. [24] proposed an integrated inspection framework for the solder joint defects of printed circuit boards. YOLOv3 was used to localize individual solder joints, and an SVM classifier was used based on the small portion of the labeled samples after small joints were transformed into a well-classified representation space. Wang et al. [25] proposed a cloud computing system that used Faster R-CNN to detect defects on the turbine blades of automobile engines. Their method could effectively detect defects in complex product images. It can be clearly seen from the above literature that Faster RCNN, SSD, and YOLO, etc., are popular for defect detection and have all been applied in various fields in recent years. However, it is still rare to see related research on defect detection for powders using deep learning methods. In this investigation, a defect detection algorithm for PCV powder based on YOLOv4 (Bochkovskiy et al. [26]) is proposed. The category, location, and size of defects (iron powder and slag) can be detected without performing feature engineering.
• Model optimization of hyper-parameters through the Taguchi method in the design of experiment (DOE): There are many hyper-parameters in the training foreign matter detection model that need to be set. However, when the number of experimental groups is too large, it will lead to a significant training time. Therefore, some studies have introduced the Taguchi method for the selection and sensitivity test of hyper-parameters, so less experimental combinations are used to obtain the approximate optimal hyper-parameter combinations (Medan et al. [27]). Lin et al. [28] established a two-layer convolutional layer CNN to classify the existence of cancer in lungs using CT images. They used the L36 (24 × 34) Taguchi method to find the hyper-parameter combinations of number of convolution core, kernel size, stride, and padding in the two-layer convolutional layer. Lin et al. [29] proposed a Taguchi-based AlexNet structure for gender classification with face images. They found the optimal hyper-parameter combination with the L18 (21 × 35) Taguchi experiment, including convolution kernel size, stride, and filling hyper-parameters of the first and fifth convolutional layers of AlexNet. Lee et al. [30] used the L16 (45) Taguchi method, including the number of convolutional layers, the number of fully connected layers, the number of convolution kernels, the size, and the number of neurons in the fully connected layer to determine the CNN structure. This CNN structure was applied to handwritten digit recognition. Chen and Tasi [19] used the YOLOv3-dense model for SMD LEDs defect detection. There were several hyper-parameters in the YOLOv3-dense model, including image amplification, size of anchor box determined with k-means, size of the mini-batch, the size of the input image, and the number of frozen layers. They employed the L32 (23 × 32) Taguchi method to determine the hyper-parameters in the YOLOv3-dense model.

This paper reports an automatic foreign matter detection system for PVC powder. The second section explains the methodology, which introduces the hardware structure of the image capture machine, the automatic annotation mechanism, and the model of detection with YOLOv4. The demonstration of the experimental results of the hyper-parameter recommended by YOLOv4 is described, as well as the foreign matter detection results and comparative analysis in Section 3. Finally, brief summaries conclude the paper.

2. Methods

The overall process of the present investigation is schematically presented in Figure 3. The images of PVC powder and foreign matter were taken after the image capture machine has been developed and assembled. These images of foreign matter were located and marked through the automatic annotation mechanism based on the DIP algorithm. Afterwards, foreign matter was identified and classified with the YOLOv4. The Taguchi method was employed to automatically determine the best strategy of combining various techniques that could improve the performance of the YOLOv4 in the modeling process.

2.1. Description of the System Hardware

In the present study, several expired patents (Kitamura et al. [2]; Tokoyama [3,4]) have been referred to in order to build the system hardware, as shown in Figure 4. This system hardware includes a powder delivery device, a sieving device, a circular platform, an image capture device, and a recycling device. The functions of these devices are described as follows:

• Powder delivery device: The powders in the sampling bottle are sent into the cyclone with compressed air. The air flows out from the upper outlet of the cyclone and powders are separated onto the sieving device.
• Sieving device: A stepping motor is employed to produce the periodic motion of a connecting rod, such that the screen moves up and down through the vibration of the built-in spring in order to spread the powder evenly on the circular platform.
• Circular platform: The platform is driven by a stepping motor and rotates counterclockwise with constant speed to transport powders to the image capture device and recycling device. The platform is white so that the PVC powders blend with the background. In addition, the platform is covered with glass, preventing the powders from adhering due to static electricity.
• Image capture device: The image capture device is the integration of an optical module with variable focal length lens and front-end illuminated-ring white light source and is utilized to capture images of PVC powder and foreign matter.
• Recycling device: Two vacuum pipes are installed for recycling foreign matter and powders. The foreign matter vacuum pipe is placed under the lens of the image capture device. As foreign matter is identified by the algorithm, the control unit will send a signal to the solenoid to activate the vacuum to collect the foreign matter. The vacuum pipe collects all the inspected PVC powders.

Figure 4. Side view of the system hardware.

Figure 4. Side view of the system hardware.

The procedure of the automatic foreign matter detection system in Figure 4 is described in the following. As the system is turned on, compressed air flows through the powder delivery device carrying powders to the cyclone. Powders are distributed evenly on the circular platform after passing through the oscillating sieving device. The circular platform conveys powders at a constant speed. Once foreign matter is detected by the image capture device, it will be drawn by the vacuum pipe. The rest of the powders are conveyed continuously until they are collected by the recycling device.

2.2. Description of Detection Algorithm

Based on the above image captured by the system hardware, the foreign matter detection algorithm will be described in this section.

2.2.1. Automatic-Annotation

The foreign matter detection system, shown in Figure 4, is able to capture images quickly for normal powders and foreign matter. However, foreign matter samples must be annotated as the ground truth before training the model. Usually, the operators do not have enough time to perform time-consuming annotation tasks. Because there are different types of foreign matter, size variations, or similar characteristics between categories, the quality of annotation varies from one to another. In order to reduce the workload of foreign matter annotation and maintain the consistency of annotation standards, an auto-annotation algorithm has been developed in the present study.

Initially, only normal powders were evenly distributed on the circular platform and characterized in the HSV color space ranging from [S−, S+]. Later, a few samples of foreign matter of the first type (iron particle) were planted among the normal powders. The images of foreign matter were captured, as shown in Figure 5. The pixels of these images with S value between [S−, S+] were set to 0, or else they were set to 1. Therefore, the foreign matter could be annotated with the connected component labeling (CCL) algorithm, scanning the image from left to right and from top to bottom. If the grayscale values between adjacent pixels were found to be similar during scanning, they were classified as a connected component and given the same index. As foreign matter is regarded as a connected component, the coordinates for the smallest bounding box and class of the foreign matter could be obtained effectively and quickly, and these data were automatically placed into an XML file. After the identification process for iron particles was completed, the process was repeated for the non-magnetic slags after cleaning the existing foreign matter. It should be noted that the non-magnetic slags and iron particles in the PVC powder can be separated with magnetism. Therefore, the procedure mentioned above for separated distribution and auto-annotation can be operated smoothly.

2.2.2. Foreign Matter Detection Model

YOLO is a model that treats object detection as a regression problem. YOLO is applied to the whole image with a neural network and predicts the bounding boxes and confidences of the objects, achieving end-to-end model optimization. This model has evolved from YOLOv1 to YOLOv4. It has the advantages of high detection speed, small object detection, compact and dense object detection, etc. Therefore, YOLO is suitable for detecting small samples of foreign matter in a fast production environment. In the present study, there are several important components in YOLOv4; the structure of YOLOv4 is presented in Figure 6. The descriptions of the components are as follows:

• CSPDarkNet53 is used as the backbone for feature extraction. There is no pooling, nor a fully connected layer in this feature extractor. In the forward propagation process, the output feature map is reduced to 1/32 times the input image through stride = (2, 2) of the five convolution kernels.
• Spatial pyramid pooling (SPP) and a path aggregation network (PAN) were used as the neck to expand the reception field and establish a multi-level bounding box prediction mechanism. Multi-scale feature sampling detection is utilized to detect foreign matter of different sizes with the help of PAN’s concept of combining up-sampling and down-sampling.
• Yoloblock is used as the head for bounding box prediction. The finer the grid unit, the finer the objects that can be detected. In the prediction, three 3D-tensors of different sizes in each image are considered as dependent variables, and the sizes of the three tensors are 19, 38, and 76, respectively. There are a total number of ((19 × 19) + (38 × 38) + (76 × 76)) × 3 = 22,743 outputs, because it is necessary to output the prediction frame on each grid unit of each scale. The depths of these three tensors are all B × (5 + C), where B represents the number of anchor boxes in each scale. The B value in the present study is 3 because the total number of anchor boxes is 9, and the default size of the anchor box is determined by the default value or through k-means; 5 represents the parameters centroid coordination, width, height, and confidence of the predicted box, and C is the number of predicted classes. C is 2 in this study.

  Figure 6. Structure of YOLOv4 in the present study.

  Figure 6. Structure of YOLOv4 in the present study.
• The definition of the loss function. The goal is to balance the center coordinate, width, height, confidence, and classification. The first term in Equation (1) represents the location loss; the second and third terms in Equation (1) represent the confidence loss, while the fourth term in Equation (1) represents the classification loss. Comprehensive IoU (CIoU) is used to evaluate the closeness of the predicting boxes to the ground truth boxes. CIoU considers the scale of overlap, center distance, and aspect ratio of the frame on the basis of IoU, which is a more comprehensive measurement. CIoU is adopted for the evaluation of location loss, while binary cross-entropy is employed for the evaluation of confidence loss and classification loss. Basically, the larger the CIoU and negative binary cross-entropy values, the closer the predicted value to the actual value.

$$\begin{array}{c}{\displaystyle {\displaystyle \sum }_{i=0}^{S^2}}{\displaystyle {\displaystyle \sum }_{j=0}^B}{\mathbf{1}}_{ij}^{obj}\times {\lambda }_{ij}\times \left[1-CIOU\left(\left[x_{ij},y_{ij},w_{ij},h_{ij}\right],\left[{\widehat{x}}_{ij},{\widehat{y}}_{ij},{\widehat{w}}_{ij},{\widehat{h}}_{ij}\right]\right)\right]\hfill \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+{\displaystyle {\displaystyle \sum }_{i=0}^{S^2}}{\displaystyle {\displaystyle \sum }_{j=0}^B}{\mathbf{1}}_{ij}^{obj}\times -\left[C_{ij}log{\widehat{C}}_{ij}+\left(1-C_{ij}\right)log\left(1-{\widehat{C}}_{ij}\right)\right]\hfill \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+{\displaystyle {\displaystyle \sum }_{i=0}^{S^2}}{\displaystyle {\displaystyle \sum }_{j=0}^B}{\mathbf{1}}_{ij}^{nonobj}\times -\left[C_{ij}log{\widehat{C}}_{ij}+\left(1-C_{ij}\right)log\left(1-{\widehat{C}}_{ij}\right)\right]\hfill \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+{\displaystyle {\displaystyle \sum }_{i=0}^{S^2}}{\mathbf{1}}_{ij}^{obj}\times -{\displaystyle {\displaystyle \sum }_{c\in classes}}\left[p_{ij}\left(c\right)log{\widehat{p}}_{ij}\left(c\right)+\left(1-p_{ij}\left(c\right)\right)log\left(1-p_{ij}\left(c\right)\right)\right]\hfill \end{array}$$

(1)

where ${\mathbf{1}}_{ij}^{obj}$ denotes that the jth predicting box in cell i is responsible for that prediction. It is equal to 1 if there is foreign matter in cell i, and the confidence of the jth predicting box of this cell is the highest among all the predicting boxes of this cell. ${\mathbf{1}}_{ij}^{nonobj}$ is almost the same except its value is 1 when there is no foreign matter in cell i. λ_ij = 2−w_ij × h_ij ϵ [1, 2] which relates to width and height of jth predicting box in cell i. λ_ij can increase the loss of the small predicting boxes.

2.2.3. Efficiency Improvement of Hyper-Parameter Search with the Taguchi Method

In the process of the training model, YOLOv4 has several important hyper-parameters that will affect the accuracy of prediction, such as mosaic image augmentation on the generated image, class label smoothing when processing benchmark truth, determination of the default size of the anchor box with k-means and load pre-trained weight as the model is initialized, and options to run the warm-up stage and cosine annealing scheduler in the settings of the optimization method. The definitions and uses of the various mechanisms mentioned above are described as follows:

• Mosaic image augmentation: There are four images stitched together with corresponding bounding boxes for each image. After the four images are stitched together, a new image and annotation file are obtained. This is equivalent to delivering four images for learning at the same time. In this way, the model can learn to identify objects at a smaller scale. In addition, the necessary mini-batch size can be reduced significantly in training.
• Class label smoothing: This is a technology that changes the labels that are assigned. Generally, the correct classification for a bounding box is represented as a one-hot vector. However, when a model becomes more arbitrary, with a prediction close to 1.0, the prediction is often wrong, overfit, and overlooks the complexities of other predictions in some way. It is more reasonable to encode the class label representation to evaluate the uncertainty to some degree; therefore, 0.9 was chosen to represent the correct classification. In this way, the real label becomes less extreme, leading to a larger error tolerance in labeling and the improvement in the generalization ability of the model.
• Determination default size of anchor boxes: The default size of the anchor boxes of YOLOv4 may not be suitable for customized image sets. If the size difference between the anchor boxes and the ground truth boxes are large, the effectiveness of model detection will be affected. If the anchor box is replaced by k-mean clustering, the preset size can be used to cluster the bounding box of the training set to automatically generate a set of nine anchors that are more suitable for the image set, which may be helpful for foreign matter detection.
• Loading pre-trained weights: The pre-trained weights, products of the VOC image library, can be loaded when training YOLOv4; therefore, the ability to extract features should have a certain degree of generalization. However, these image libraries are quite different from the foreign matter detection in this study. The weights trained with the customized image collection from the powders would perhaps be helpful for the detection mode.
• Performing warm-up stage: The warm-up learning rate is performed when training YOLOv4 to freeze the weights of the first few layers. Only the weights of the last few layers are adjusted through the Adam algorithm. The learning rate of the Adam algorithm is a small value of 0.001. After a stable loss value is calculated, a usable model is obtained, then the training procedure with the fine-tune stage can be performed. Meanwhile, all the weights of the layers are unfrozen, and the learning rate of the Adam algorithm is adjusted to a smaller value of 0.00001 to continue training to achieve the purpose of fine-tuning the weights.
• Cosine annealing scheduler: The loss function may be in multipeak mode, so there could be multiple regional optimal solutions in addition to the global optimal solution. When YOLOv4 is training, it is easy for the gradient descent method to fall to the local minimum. Therefore, a suddenly increased learning rate could be utilized to jump out of the local minimum and obtain the path to the global minimum.

In the process of training YOLOv4, these strategies may be employed to improve the efficiency of detection. However, no combination of these strategies has been proven to be superior. Ultimately, the efficiency of detection depends on the conditions of the image set. Therefore, the analysis needs to be performed while training the model.

As long as the above strategies are regarded as factors, different levels are introduced with each strategy, and the validation mean average precision (mAP) is utilized as the dependent variable; a design of experiment (DOE) can be established to determine the best combination of factors and levels systematically. Due to the lengthy training time required for testing various hyper-parameter combinations one by one, the Taguchi method is utilized to reduce the number of experiments. Although this method is not as good as the full factor experiment, it can find the best hyper-parameter combination and the approximate optimal solution with the least experimental combination. In addition, the mAP is chosen to be the response of Taguchi’s method in the present study. After foreign matter is detected, the precision-recall (PR) curve can be obtained according to the difference between the predicting boxes and the ground truth boxes of the validation image set. The mAP is the average number of APs of different samples of foreign matter classes and is defined as the average of the precision of the 11 selective recall rates to approximate the area under the PR curve (i.e., AP). AP is calculated as in Equation (2):

$$AP=\frac{1}{11}{\displaystyle \sum }_{r\in \left\{0,0.1,\dots ,1.0\right\}}\underset{Recall\ge r}{\max }Precision$$

(2)

3. Results and Discussions

In this section, the capture image sets are introduced before the process of optimizing the prediction accuracy of YOLOv4. The results of foreign matter detection are then presented, and the prediction results of the optimized YOLOv4 are compared with other existing methods.

3.1. Description of Capture Image Sets

PVC powder is the observed object in the present study. These powders were collected from the product bag and stored in a sampling bottle. The particle size is about 100 μm, its shape is slightly oval, and its color is white. The iron particles were picked up with a magnetic rod, and slag particles with a dark red color were collected through visual inspection. These particles were mixed with PVC powder and placed in the foreign matter detection system for PVC powder to capture the images of iron and slag particles. Those pictures are presented in Figure 7. A total of 6964 images with foreign matter were obtained: 71.5% with iron particle and 28.5% with slag particle. Those images were split at a ratio of 6:1:3 to perform stratified sampling. Therefore, 4178, 696, and 2090 images were used for training, validation, and testing, respectively. It can be clearly seen in Figure 7 that the PVC powders were scattered and oriented randomly in the picture. It is evident that the iron and slag particles are black and dark red in color.

3.2. Combinations of Model Training Strategies with the Taguchi Method

Through the analysis of the validation mAP of YOLOv4 combined with various model training strategies, there is an opportunity to find the key combination of model training strategies that affect the mAP of foreign matter detection, which can then be used as the basis for improving the prediction ability of the model. However, due to the lengthy training time required to review various combinations of model training strategies one by one, the Taguchi method was employed to improve the efficiency of the deep learning model and reduce the number of experiments. Although the Taguchi method is not as good as the full factor experiment, it can be used to find the approximate optimal solution with the least number of experiments.

As mentioned in Section 2.2.3, there are six experiments, including mosaic data augmentation, label smoothing, default size of anchor boxes, pre-trained weights, warm-up stage, and cosine annealing scheduler, in the model training strategy. There are two levels, closed and open, for each experiment, as indicated in

Table 1. Experimental factors and level tables.

Factors	Mosaic Data Augmentation	Label Smoothing	k-Means for Size of Anchor Boxes	Pre-Trained Weights	Warm-Up Stage	Cosine Annealing Scheduler
Level 1	off	0.0	off	VOC	0	off
Level 2	on	0.1	on	random	50	on

. Therefore, L8 (26) orthogonal tables, a total of eight combinations of model training strategy (also named treatment), were performed with the Taguchi method.

As the Taguchi method is performed, the input, model, and output of YOLOv4 were set according to each treatment. After training, the validation mAP corresponding to each treatment was recorded and then converted into the average of the validation mAPs of each treatment at different levels. Figure 8 depicts the average value main effect plot for each treatment. It can be observed in Figure 8 that the validation mAPs for k-mean and random initial weights present a positive slope and a larger difference. Therefore, k-mean and random initial weight model training strategies were selected in the present study due to larger validation mAP and positive slope.

The k-mean training strategy was used because the predicted size of the anchor box and the size of the foreign matter to be detected in this study are too different in the image. Figure 9a was learned from the VOC data. It can be seen that more than half of the anchor frame sizes are relatively large and are restricted by the size of the foreign object itself. When these large anchor boxes are applied to the dataset of this research, it is difficult to exert their effect and they appear redundant. Compared with Figure 9a, Figure 9b shows a more suitable anchor box size, because the size of the foreign matter in the dataset of this study is between 120 and 150 μm. In addition, the random initial weight training strategy is adopted because the pre-trained network learned through the VOC dataset cannot show good generalization ability for our customized images. This may be due to the fact that the training samples prepared by this study can be used to fully learn the features of foreign matter for YOLOv4, and the anchor size is also adjusted for the size of foreign matter by k-means.

3.3. Demonstration and Accuracy Comparison of Foreign Matter Detection Results

Because the combination of the model training strategy recommended by the Taguchi method is not included in the eight treatments, YOLOv4 was trained again following the recommendation of the Taguchi method that only the model training strategy of the k-means anchor box and random initial weights were included to obtain the optimized YOLOv4. There are 500 images utilized for testing in the inference. The experimental results show that the optimized YOLOv4 has a test mAP of 0.9385, compared to 0.8653 and 0.7999 for naïve YOLOv4 and Faster RCNN, respectively. The comparisons of these values are displayed in

Table 2. Comparison of mAP for faster RCNN, naïve YOLOv4, and optimized YOLOv4.

Model	Faster RCNN	Naïve YOLOv4	Optimized YOLOv4
Testing mAP	0.7999	0.8653	0.9385

. Additionally, there is no false alarm to the images without foreign matter with the optimized YOLOv4.

In addition to mAP, this research also uses optimal YOLOv4, Faster RCNN, and naïve YOLOv4 for demonstrating the results of foreign matter detection, as shown in Figure 10. Figure 10(a1–a5) are images taken by the machine in Figure 4. Figure 10(b1–c5) are the detection results using the Faster RCNN and naïve YOLOv4. It can be seen that Faster RCNN and naïve YOLOv4 easily miss the detection of small foreign matter, and the levels of confidence are very unsatisfactory. This might be because Faster RCNN and naïve YOLOv4 are not suitable for detecting small particles of foreign matter. From Figure 10(d1–d5), the optimized YOLOv4 exhibits excellent detection capabilities, and the foreign matter detection capabilities of optimized YOLOv4 take another step forward.

4. Conclusions

In industry, the main process of quality inspection for PVC powder utilizes magnetic rods and visual inspection with the human eye. However, the disadvantages of this approach are that it is expensive, subjective, and inconsistent. Because the sizes of foreign objects are extremely small, leading to them often being overlooked, the inspection process may harm the vision of the operator. In the present study, a foreign matter detection system based on machine vision and deep learning has been constructed to perform automatic foreign matter detection for PVC powder. This system can also perform an auto-annotation process to save time in feature engineering. Therefore, when using this system, the operator does not need to annotate foreign matter, nor need to be experts in designing the features of foreign matter. In the future, this system could be employed to inspect other powders, as long as the powder and the foreign matter is collected in advance. Then, images of foreign matter could be automatically annotated and identified, and automatic model optimization training and foreign matter detection could be carried out. In addition, the function of the system could be expanded to have the ability of powder particle size invariance and powder color invariance. In the future, other approaches may also be applied to the identification of foreign matter, such as machine learning [31] and Hirshfeld surface analysis [32].