Detection of Impurity Rate of Machine-Picked Cotton Based on Improved Canny Operator

Abstract

Aiming at the real-time detection of the impurity rate in machine-picked cotton processing, a detection method for the impurity rate in machine-picked cotton was proposed based on an improved Canny operator. According to the characteristics of different saturations between cotton and impurities, the impurities were separated by extracting the image S channel. Due to problems existing in the traditional Canny operator’s edge detection, the Gaussian filter was replaced by employing mean filtering and nonlocal mean denoising, which could effectively remove the noise in the image. A YOLO V5 neural network was used to classify and identify the impurities after segmentation, and the densities of various impurities were measured. The volume–weight (V–W) model was established to solve the impurity rate based on mass. Compared with a single thread, the data processing time was shortened by 43.65%, and the frame rate was effectively improved by using multithread technology. By solving the average value of the impurity rate, the anti-interference performance of the algorithm was enhanced, and has the characteristics of real-time detection and stability. This method solved the problems of low speed, poor real-time detection, and ease of interference, and can be used to guide the cotton production process.

1. Introduction

China is a major cotton producer, and Xinjiang is the most important cotton-producing area in China. In northern Xinjiang, the mechanization level of cotton planting and harvesting has reached 97%. The mechanization level of southern Xinjiang, according to data from 2020, has also reached 40%, with the mechanization level increased year by year, but with cotton mixed with impurities also increasing, including machine-picked seed cotton in the presence of bell shells, stiff petals, cotton branches, cotton leaves, dust, and other natural impurities. At the same time, the machine-picked cotton processing technology has gradually developed, and machine-picked cotton processing equipment, the complex process, and the working state of the equipment have a great impact on the cleaning of cotton, such as the machine-picked cotton processing equipment’s rotating speed being too low to achieve a cleaning effect in the cotton, while a high speed will cause damage to the cotton fiber. Therefore, real-time monitoring of impurity content in seed cotton can provide a basis for adjusting the speed of machine cotton-picking processing equipment, and is of great significance for improving the quality of cotton.

In the process of impurity rate detection based on image information, the primary task is to segment impurities from the cotton image. In terms of cotton impurity detection, Lu Xufeng et al. [1] and Zhang Yun et al. [2] detected and identified heterotopic fibers in cotton based on machine vision. Dong Chaoqun et al. [3] analyzed the advantages and disadvantages of various optical imaging technologies in the detection of heterotopic cotton fibers. Shi Hongyu et al. [4] detected heterotopic fibers in cotton based on the method of image multiresolution difference. Cao Zhenjun et al. [5] used raspberry PI to detect fabric defects based on deep learning. Ding Xiaokang et al. [6] used the Canny operator to detect colored foreign fibers. Zhao et al. [7] and Zhao Xuehua et al. [8] selected the characteristics of heterotopic fibers in cotton by improving particle swarm optimization and ant colony algorithms to classify and identify heterotopic fibers. Wang et al. [9] proposed a segmentation algorithm for lint heterotopic fibers. He Xiaoyun et al. [10] used deep learning to detect heterotopic fibers in cotton. Pavani et al. [11] used pattern-recognition technology to classify impurities in raw cotton. Zhang Chengliang et al. [12,13] applied the regional color method and genetic algorithm to optimize the a support vector machine (SVM) parameter model to segment impurities in images of machine-picked cotton. Wang Haopeng et al. [14], Wang Xin et al. [15], and Wei Ping et al. [16] proposed a machine-vision-based method for cotton fiber recognition. In the process of image segmentation, most of the segmentation objects need to have their edges found, and the traditional image edge-detection methods mostly extract high-frequency signals as the main means; for example, the Roberts operator, Sobel operator, Prewitt operator, or LOG operator. Compared with these differential operators, the Canny operator based on an optimization algorithm is widely used due to its advantages of a large SNR and high detection accuracy. Many scholars at home and abroad have studied the optimization of the Canny operator [17,18,19,20,21,22,23]. Wang Zuocheng et al. [24] improved the traditional Canny operator by introducing the Otsu threshold method to select threshold values according to the characteristics of the image itself so that the detected edges were more continuous, and the existence of false edges was reduced. Jin Yanhong et al. [25] improved the precision of edge location by improving the calculation of gradient amplitude and the interpolation direction of the nonextreme suppression process. Zhang Zhishun et al. [26] used a genetic algorithm to automatically optimize the threshold function based on generalized cross-validation (GCV) criteria, and used the evaluation function to adaptively set the high and low threshold value of the Otsu method, which reduced the noise in the image and improved the detection accuracy. Wang Rong et al. [27] and Bai Tingting et al. [28] proposed an edge-detection method that combined the Canny method and a wavelet transform. The edge images obtained by the two methods were fused to ensure the integrity of the edge, but the interference information also increased, and the running time of the algorithm was not considered, which was not conducive to real-time detection. However, because the traditional Canny operator is sensitive to noise and requires an artificial setting of Gaussian filter parameters, it lacks an adaptability to different images, so these methods have achieved experimental results to a certain extent. However, there are still limitations, such as the processing time being too long to carry out real-time detection. In addition, the detection result is also affected by the environment, and the anti-interference is poor.

Domestic and foreign scholars have done a lot of work to solve the problem of noise influence. For example, A Ouahabi [29] used wavelet denoising technology to denoise medical ultrasound and magnetic resonance images. Many questions could be answered by a prospective study using an undecimated discrete wavelet transform as a contourlet transform to facilitate the processing of edge features, and thus achieve a more efficient noise reduction. However, when using wavelet multiresolution analysis, there is a trade-off between noise reduction and preservation of important details. Because wavelet denoising transforms the time domain information to the frequency domain for processing, the number of calculations will greatly increase; this can be applied in static image processing, but cannot be applied in the real-time detection of machine-picked seed cotton.

In this study, based on image-processing technology, the impurity rate in machine-picked cotton was examined, and the impurity content in cotton processing was fed back in real time to guide the cotton processing. In this paper, an improved Canny operator segmentation, a traditional Canny operator segmentation, SVM segmentation, K-means clustering, and other algorithms were used to find impurities in the seed cotton image-segmentation machine, and a YOLO V5 neural network was used to classify and identify the impurities after segmentation and measure the density of various impurities, establish the V–W model, and solve the impurity rate based on quality. In addition, as a comparative experiment, impurities were manually separated and weighed by electronic balance to obtain the impurity inclusion rate of a mass method, which was compared with the impurity inclusion rate based on image information, in order to compare the merits and demerits of various algorithms. Raspberry PI was used to build a detection platform for the impurity rate of machine-picked seed cotton, and the collected images were transmitted to the server for processing to realize the real-time detection of the impurity rate.

2. Materials and Methods

2.1. Real-Time Detection System for Machine-Picked Seed Cotton

The real-time detection system of machine seed-cotton picking was divided into two parts; namely, the lower computer and the upper computer, as shown in Figure 1. The lower computer collected the image in real time through the camera, encoded the image information, and transmitted it to the server of the upper computer through the Raspberry PI. The server decoded the image information first, segmented the image for impurities, classified and identified the impurities in the segmented image using YOLO V5, and processed the data based on the V–W model. The real-time impurity rate and average impurity rate based on quality were obtained, and the results were finally output together with the segmented image.

The test material was seed cotton collected by the Xinluzao No. 59 machine of the 149 Regiment of the Xinjiang Eighth Agricultural Division, and the impurities included bell shells, cotton twigs, weeds, and leaf scraps. The PI model was a Raspberry4B (8 GB). A Ylr-sn85 camera with a resolution of 3840 pixels × 2160 pixels, a CMOS size of 1/2.7 inch, a frame rate of 30 frames/s, and a USB power supply was used. The lens had a maximum aperture of 1.4, a focal length of 5–50 mm, and 3 million pixels. The lighting source was an AFT-WL21244-22W strip LED diffuse light source, and the light source controller was an AFT-ALP2430-02. The processor of the server was an Intel(R) Core(TM) I5-9400F [email protected], the number of cores was 96, the GPU was an NVIDIA GeForce GTX 1660, the memory was 16 GB, the operating system was 64-bit Windows 10, and the algorithm-running environment was PyCharm 2021.1. The programming language was Python, using the open source OpenCV visual library.

The light source was installed on an adjustable bracket, which was convenient to adjust the irradiation angle of the light source. At the same time, the light source bracket was fixed on the bench profile by t-nuts and bolts, which could be used to adjust the distance between the light source and the cotton, providing high brightness and balanced lighting for the system. The camera was attached to the cotton and light source via a cantilever bracket, and was connected to a computer via USB. When the system was operating, the cotton was placed at the bottom of the platform manually, the distance of light source and the angle of irradiation were adjusted, and the real-time collection of machine-picked seed cotton images was carried out by the HD camera. In order to collect the same sample many times and calculate the mean of the impurity rate, the cotton sample needed to be constantly turned. The real-time images were encoded and transmitted to the server, which decoded the image information to detect the impurity rate, and output the detection results.

A photo of the image-acquisition system of the lower computer is shown in Figure 2, and the real-time processing results of the upper computer are shown in Figure 3.

2.2. Detection of Impurity Rate of Machine-Picked Cotton

2.2.1. Detection Process

A detection flow chart of the impurity rate of the machine-picked seed cotton is shown in Figure 4. First, by the image real-time acquisition machine adopted in the seed cotton impurity camera, the collected image color space transformation, the HSV color space S (saturation) channel extracted from image in, and truncation in the S channel of the image segmentation, the improved Canny operator was used for edge detection, and a morphological closing operation was carried out on the edge detection after the image formed the closed area. In addition, fills formed a closed area of the holes due to the existence of impurities associated with image edge images, and the use of the noise image with hole filling after a grey value addition operation for the image, as determined by gray histogram threshold segmentation. Then, for the gray image binarization operation, the processed image was analyzed by the YOLO v5 network model to carry out the classification identification of the impurities. Then it was output to another thread, using the V–W model to solve the real-time and average clutter-containing rate based on quality; and finally, to output the real-time clutter-containing rate, average clutter-containing rate, and segmentation image.

2.2.2. Traditional Canny Operator Edge Detection

In the process of impurity detection based on image information, the primary task was to segment impurities in the cotton images. Due to the large number and variety of cotton impurities in machine-picked seed cotton, the detection of the impurity rate of machine-picked seed cotton encountered great difficulties. Canny operator edge detection could better detect the contour information of impurities.

• Smoothing images

The Canny algorithm selected an appropriate one-dimensional Gaussian function to smooth and denoise the image $F \left(x, y\right)$ according to rows and columns, respectively, which was equivalent to convolution of the image signals. The selected Gaussian function [30] was:

$$G\left(x\right)=\frac{1}{\sqrt{2\mathsf{\pi }}\sigma }{e}^{\frac{-x^2}{2{\sigma }^2}}$$

where σ is the standard deviation of the Gaussian curve, which controlled the smoothness.

• Calculate the magnitude and direction of the gradient

The gradient amplitude $M\left(x, y\right)$ and gradient direction$H\left(x, y\right)$ of smoothed image $I\left(x, y\right)$ were calculated by the finite difference of the first-order partial derivatives of 2 × 2 neighborhood; i.e. [30]:

$$M\left(x,y\right)=\sqrt{K_x^2\left(x,y\right)+K_y^2\left(x,y\right)}$$

$$H\left(x,y\right)=\arctan (\frac{K_x\left(x,y\right)}{K_y\left(x,y\right)})$$

$$f_x=\left[\begin{array}{cc}-0.5& 0.5\\ -0.5& 0.5\end{array}\right],f_y=\left[\begin{array}{cc}0.5& 0.5\\ -0.5& -0.5\end{array}\right]$$

2.2.3. Dark Impurity Segmentation

• Nonmaximum suppression of gradient amplitude was performed

The purpose of maximum inhibition was to find the $I \left(x, y\right)$ in all of the possible edge points. The basic idea was to use 3 × 3 neighborhood effects on a gradient amplitude array$M \left(x, y\right)$ of all points, if the neighborhood center gradient amplitude$M \left(x, y\right)$ of the two adjacent points along the gradient direction amplitude was large, it would be the current center of the neighborhood discriminant for possible edge points, Otherwise, $M\left(x, y\right)$ was assigned as zero and identified as a nonedge point.

• Double threshold method to detect and connect edges

A dual-threshold algorithm was used to segment the nonmaximum-suppressed images with high and low thresholds to obtain two threshold edge images $T1\left[i, j\right]$ and $T2\left[i, J\right]$, respectively. Since the image $T1\left[i, j\right]$ had a high threshold value, it contained almost no false edges, but $T1\left[i, j\right]$ may have had a break in the contour. Therefore, the double-threshold algorithm connected the edges into contours in T1[i, j]. When the contour endpoint was reached, the algorithm looked for the edges that could be connected to the contour in the eight-neighborhood position of the edge image $T2\left[i, j\right]$ of the low-threshold value. Thus, the recursive tracking algorithm continuously collected edges in $T2\left[i, j\right]$ until all gaps in $T1\left[i, j\right]$ were joined.

• Defect analysis of traditional Canny operator image edge-detection algorithm

In the image edge detection, image denoising was the first step, and the traditional Canny operator used a Gaussian filter to remove noise. The parameter σ, which controlled the degree of smoothness, was manually set. The larger the parameter σ, the narrower the frequency band became; the high frequency signal is well suppressed, and the occurrence of false edge points may be avoided. Because the signal frequency at the edge was also high, some details would be lost, resulting in unclear image edges. The smaller the σ setting, the wider the frequency band became, preserving edge details but not achieving the desired noise reduction [31]. In practice, it is difficult to find an accurate value for σ, so it presents some limitations in image edge detection.

2.2.4. Improved Canny Operator Edge Detection

Because cotton fibers are thin and intertwined with impurities, in the process of image collection of machine-picked seed cotton, even though impurities could be clearly distinguished through color-space conversion, it was still easy to produce a lot of noise, as shown in Figure 5. Due to the interference of noise points, the traditional Canny operator had poor edge detection effect and could not achieve effective segmentation. The processing effect is shown in Figure 6.

Aiming at the edge-detection effect of the Canny operator in machine-picked seed cotton images, this study used mean filtering and nonlocal average denoising to replace Gaussian filtering. This could reduce noise and smooth the impurity area, so as to better realize impurity edge detection.

• Mean filtering

Mean filtering is also called linear filtering, and the main method adopted was neighborhood averaging. The basic principle of linear filtering is an average instead of the original image for each pixel value, and involves the current pixel I (x,y) and choosing a template; this template was composed of the neighbor number of pixels, strove for the average of all pixels in the template, and then gave the average of the current pixel I (x,y) as a grayscale image at that point in g (x,y); namely:

$$g\left(x,y\right)=\frac{{{\displaystyle \sum }}^{ }f\left(x,y\right)}{m}$$

where m is the total number of pixels, including the current pixel in the template.

Because the noise in the machine-picked seed cotton image was a fog noise with a gray value between the background and impurities, the mean filter could be used to distinguish noise, impurities, and background, and the edge details would not be lost due to the high gray value of impurities in the image. An image processed by mean filtering is shown in Figure 7.

• Nonlocal mean denoising

Buades et al. [32] proposed the nonlocal means (NLM) algorithm by taking advantage of the fact that natural images contain many similar structures. The basic idea of the algorithm is to search the corresponding similar blocks in the global scope for the image blocks where the denoised pixels are located, and to find a weighted average for the similar structures to reduce the random noise, so as to achieve the denoising. The NLM algorithm is superior to many common denoising algorithms.

With noise image $f=\left\{f\left(i\right)\right|i\in I\}$, any of the gray levels of pixels to the I place by filtering $\widehat{f}\left(i\right)$ were determined as follows:

$$\begin{gathered} \widehat{f}\left(i\right)={\displaystyle \sum }_{j\in I}w\left(i,j\right)f\left(i\right) \\ w\left(i,j\right)=\frac{1}{Z\left(i\right)}{e}^{-{\left|\left|f\left(N_i\right)-f\left(N_j\right)\right|\right|}_{2,\propto }^2/h^2} \end{gathered}$$

where the weight $w\left(i,j\right)$ satisfies $0\le w\left(i,j\right)\le 1$ and ${\displaystyle \sum }_{j}w\left(i,j\right)=1$; ${\left||·|\right|}_{2,\propto }^2$ to measure pixels i and j of the similar degree of Gaussian-weighted Euclidean distance; a is the standard deviation of the Gaussian kernel, a >0; h is the parameter controlling the attenuation degree; and $Nk$ is the square neighborhood whose center is located at pixel K. The regularization constant $Z\left(i\right)$ is:

$$Z\left(i\right)={\displaystyle \sum }_{j\in I}{e}^{-{\left|\left|f\left(N_i\right)-f\left(N_j\right)\right|\right|}_{2,\propto }^2/h^2}$$

To avoid overweighting, when $i=j$, the weight $w\left(i,j\right)$ is:

$$w\left(i,j\right)=max\left(w\left(i,j\right)\right),i\ne j$$

The NLM algorithm was used to denoise the machine-picked seed cotton images after mean filtering, and as shown in Figure 8, the impurity area became smooth. Using a Canny operator for edge detection could effectively identify the edge information for impurities, as shown in Figure 9. Compared with the detection results of a traditional Canny operator (as shown in Figure 6), it can be seen that the contour was more obvious, and many interference factors were reduced.

• Improved gradient amplitude calculation

The original Canny operator used a 2 × 2 domain first-order finite-difference operation to calculate the gradient amplitude of the images. Only the gradient values in the horizontal and vertical directions were considered, so it was susceptible to interference from external noise. Therefore, this study added 45° and 135° directions to calculate the gradient amplitude based on an isotropic Sobel operator. The gradient template for the four directions in the 3 × 3 field is shown in the formulas below:

$$H_x=\left[\begin{array}{ccc}-1& -\sqrt{2}& -1\\ 0& 0& 0\\ 1& \sqrt{2}& 1\end{array}\right] H_y=\left[\begin{array}{ccc}-1& 0& 1\\ -\sqrt{2}& 0& \sqrt{2}\\ -1& 0& 1\end{array}\right]$$

$$H_{{45}^{°}}=\left[\begin{array}{ccc}-\sqrt{3}& -1& 0\\ -1& 0& 1\\ 0& 1& \sqrt{3}\end{array}\right] H_{{135}^{°}}=\left[\begin{array}{ccc}0& 1& \sqrt{3}\\ 1& 0& 1\\ \sqrt{3}& 1& 0\end{array}\right]$$

The convolution operation was performed on the gradient template in the horizontal direction, vertical direction, 45° direction, and 135° direction using the filtered image to obtain the gradient components $M_x$, $M_y$, $M_{{45}^{°}}$, and $M_{{135}^{°}}$, respectively, in the four directions. Finally, the gradient amplitude $M$ was obtained as follows:

$$M=\sqrt{M_x^2+M_y^2+M_{{45}^{°}}^2+M_{{135}^{°}}^2}$$

The total gradient components in the four directions were $M_x^{’}$ and $M_y^{’}$. in the horizontal and vertical directions, respectively, as shown below:

$$M_x^{’}=M_x+\frac{\sqrt{2}}{2}{M}_{{45}^{°}}-\frac{\sqrt{2}}{2}{M}_{{135}^{°}}$$

$$M_y^{’}=M_y+\frac{\sqrt{2}}{2}{M}_{{45}^{°}}-\frac{\sqrt{2}}{2}{M}_{{135}^{°}}$$

The gradient direction $\delta$ was obtained from $M_x^{’}$ and $M_y^{’}$, as shown in the following formula:

$$\delta =arctan(\frac{M_y^{’}{}^2}{M_x^{’}{}^2})$$

2.2.5. Determination Algorithm of Intersection between Impurities and Image Edge

In the process of edge detection, all that was detected were the edges of impurities. Therefore, the impurity region could be obtained by filling the area enclosed by the edges. However, some edges were still unclosed after edge detection, so it was necessary to perform a morphological closure to form the closed region, as shown in Figure 10. It can be seen in Figure 11 that there were still some unfilled areas in the filled image. An analysis showed that the reason was that the impurity region intersected with the image edge and the impurity edge line was not closed, so the hole filling could not be carried out.

In order to realize the automation of the algorithm, the determination algorithm for the intersection between impurities and the image edge was set as follows:

$$K=\left\{\left(0,:\right),\left(maxc,:\right),\left(:,0\right),\left(:,maxl\right)\right\}$$

$$f\left(k\right)=\{\begin{array}{c}yes, if g\left(k\right)=255\\ no, otherwise\end{array} k\in K$$

where K represents the set of coordinates of the outermost pixels of the image; maxc represents the maximum number of lines of the image; maxl represents the maximum number of columns of the image; The ‘:’ represents all values from 0 to the maxi-mum value of the domain, and g(k) represents the gray value of pixel point K.

First, the gray value of the outermost pixel point of the image after filling the hole was traversed. If there was a point with a gray value of 255, it was judged that the impurity intersected with the image edge.

To determine the impurity and the image-edge images of the intersection, after using the nonlocal average noise of images and hole filling after the image-addition operation, as shown in Figure 12, it can be seen that the unpopulated area had high grey value by looking at the gray histogram of the image; for the segmentation threshold, which was finally in the range of (45, 80), we selected 60. The processed image (as shown in Figure 13) was output to the data-processing thread to find the impurity rate.

2.2.6. Impurity Classification and Recognition Based on YOLO V5

In the classification and recognition of impurities in cotton, earlier studies were mostly based on manually selected features combined with a machine-learning classifier model for training, and the trained classifier was then used to classify impurities according to features. On the one hand, the selected features were one-sided, and it was difficult to find all the features that could distinguish each type of impurities. Therefore, the selection of features directly affected the classification effect of the model. On the other hand, the learning characteristics of early machine-learning models were relatively shallow, and the models’ expression abilities were not adequate. In addition, improvements in their classification performances were less affected by the number of data sets.

A deep convolutional neural network can automatically extract and learn more essential features of images from training data. Applying a deep convolutional neural network to machine-picked seed cotton impurity classification and recognition technology can significantly improve the classification effect. YOLO V5, which integrates the advantages of faster-CNN, SSD, and ResNet models, is the target detection network with the most balanced speed and accuracy so far. It will achieve remarkable results when applied to the classification and identification of machine-picked seed cotton impurities.

In this study, the photos of the machine-picked seed cotton containing impurities were manually marked to construct a training data set. All training and testing took place on a deep-learning workstation with a main configuration of an Intel(R) Xeon Platinum 8277L [email protected], a 24 GB NVIDIA GeForce RTX 3090, and 96 GB of running memory. The program was written in the Python language was run using CUDA, Cudnn, and OpenCV on a 64-bit Windows 10 system. In model training, pretraining weight files on the official website were used to initialize network weight parameters and accelerate model convergence. The training parameters were set according to experience; the training parameter values are shown in

Table 1. Training parameter values.

Parameter Name	Parameter Value
batch	16
subdivisions	8
iterations	5000
steps	4000 times and 4500 times
img-size	640 × 640
depth_multiple	0.67
width_multiple	0.75
nms	0.25

.

The appropriate prior frame size in YOLO V5 directly affected the detection speed of impurities in the machine-picked seed cotton and the positioning accuracy of the target frame. In order to obtain the optimal prior frame size, YOLO V5 automatically learned the prior frame size again. After the image in the model was scaled to 640 × 640, the optimal prior frame sizes were: (12,16), (19,36), (40,28), (36,75), (76,55), (72,146), (142,110), (192,243), (459,401), and these prior frame sizes were used for detection experiments.

• Model training

The final model was trained 5000 times, which took 9 h, and 1017 images were used in the training. Three enhancement methods; i.e., scaling, color space adjustment, and Mosaic enhancement, were used to increase the training data and enhance the model recognition ability. In the training process, the dynamic training was observed by drawing the model loss curve; the corresponding loss value change curves are shown in Figure 14.

It can be seen in Figure 14 that the loss value of the model decreased rapidly in the early iterations, and the model was fitted quickly. After 50 iterations, the loss value decreased slowly. When the iteration reached 200 times, the loss curve was almost flat, and the accuracy reached 99.8%. In order to prevent overfitting and reduce training costs, the training was immediately ended, and the best weight was saved. The detection picture used in the training process is shown in Figure 15.

It can be seen in Figure 15 that impurities in the image could be detected through image enhancement, and the trained model could be used to classify and identify impurities in the machine-picked seed cotton.

• Identify the impurities of machine-picked cotton after segmentation

A total of 200 groups of impurities after image segmentation were collected and input into the YOLO V5 model for classification and recognition, and the accuracy was counted. The results are shown in

Table 2. Classification and recognition accuracy of various impurities.

Type of Impurity	Sample Size	Number of Wrong/Missing Samples	Recognition Accuracy/%
Bell housing	186	3	98.388
Cotton branch	174	2	98.851
Weeds	221	6	97.285
Blackjack	946	25	97.357

.

As can be seen in Table 2, the lowest recognition accuracy was 97.285%, which could realize the classification and identification of impurities in the machine-picked cotton.

2.2.7. Data-Processing Threads

In this paper, the use of multithreading technology; that is, the image-processing work and data-processing work (calculation of the impurity rate) respectively used the CPU’s two threads for calculation. The CPU model was an Intel(R) Core(TM) I5-9400F [email protected], and the number of cores was 96.The two threads did not interfere with each other, avoiding the congestion problem of one-thread operation. This could speed up and facilitate real-time detection. The process for the data-processing thread is outlined below.

According to the national standard, the quality method is the main method to detect the impurity rate. The so-called quality method refers to the percentage of impurities in the sample to the total mass of cotton containing miscellaneous seeds as the impurity rate. Using to the cotton images collected, the proportion of the impurity area was calculated as the impurity rate of the cotton in the acquisition layer, which reflected the impurity content of the cotton to a certain extent. In order to establish a connection between the image information and quality, based on the area of the image information, the impurity rate was extended to the three-dimensional space, as shown in Figure 16, because in a large pile machine used in the process of picking cotton, the cotton impurity rate value was stable, based on the detection layer impurity rate as a benchmark, and the impurities in the process of a continuous processing rate were same as the rate of the impurity detection layer, Therefore, an impurity rate based on volume was obtained.

The proportion of the impurity area obtained from the image information was used as the proportion of impurity volume, as follows:

$$P_s=\frac{s}{S}=\frac{v}{V}$$

where $P_s$ is the impurity rate based on area, s is the impurity area in the image, S is the overall area of the image, v is the volume of impurities in seed cotton, and V is the volume of seed cotton.

In addition, by establishing the volume–weight (V–W) model, the impurity rate based on mass could be calculated as:

$$P_v={\displaystyle \sum }_{i\in Z}{P}_{si}\frac{{\rho }_i}{{\rho }_{sc}}=\frac{{{\displaystyle \sum }}^{ }{M}_{im}}{M_{sc}}$$

where $P_v$ is the impurity rate based on the V–W model; Z is the impurity type, including bell shells, cotton branches, weeds, and leaf crumbs; impurity Psi is the ith impurity rate based on area; ρ_i is the first-class i impurity density; ρ_sc is the natural fleeciness state of the average density of the seed cotton; $M_{im}$ is the quality of the impurities; and $M_{sc}$ is the machine adopting the quality of seed cotton, measured, and machine mining seed cotton in the density of the material, as shown in

Table 3. Density of each substance in machine-picked seed cotton.

Material Type	Density (g/cm3)
Bell shell	0.461
Cotton branch	0.152
Weeds	0.803
Leaf crumbs	0.642
Machine-picked seed cotton in a natural fluffy state	0.481

.

The image processed by the image-processing thread was input into the data-processing thread, and the image was traversed by pixel points, the number of points with a gray value of 255 in various impurity areas was recorded, and the real-time impurity rate and average impurity rate based on quality were calculated. Using multithreading technology, the data-processing process was arranged in another thread for the detection of the impurity rate. As a comparison, the single thread was used for the detection of the impurity rate, and the processing time of each image was counted. A comparison of the processing times for each type of thread is shown in

Table 4. Comparison of single- and multithread processing times.

Single-Thread Processing Time (s)	Multithreaded Processing Time (s)
0.501	0.285
0.531	0.293
0.522	0.301
0.512	0.299
0.522	0.297
0.523	0.281
0.531	0.296

.

When using a single thread to detect the impurity rate, the average time for detecting an image was 0.520 s; when using multithread technology to detect an image, it was 0.293 s, a reduction in processing time of 43.65%. It can be seen that multithreading technology could improve the real-time detection frame rate and feedback of the processing status of cotton in real time.

3. Results

In order to verify the effectiveness of the proposed method, the proposed algorithm was compared with the detection results of the traditional Canny operator detection algorithm, SVM algorithm, and K-means clustering algorithm. The detection results are shown in Figure 17.

As can be seen in Figure 17, the improved Canny operator had a more accurate segmentation effect than the traditional canny operator, avoiding the influence of the cotton background and reducing the part of mis-segmentation. However, the traditional algorithm had a poor segmentation effect on the image in this acquisition environment; large-area impurities were prone to produce defects, and small-area impurities had incomplete segmentation. K-means clustering segmentation was affected by the images, and had poor robustness. It was easy to mis-segment large cotton areas. SVM segmentation required manual selection of the foreground and background, so the segmentation effect was affected by human factors, and large impurities were prone to a lack of holes. Among the above algorithms, the average processing times for a single image using the improved Canny operator and the traditional Canny operator were 0.293 s and 0.506 s, respectively, which were adequate for real-time detection. In addition, the average processing times of the K-means clustering and SVM segmentation for a single image were 1.632 s and 16 s, respectively, which were not suitable for real-time detection.

By preparing 10 samples, the improved canny operator and the traditional Canny operator were used to collect 3 s images of the samples and calculate the average impurity rate. Then, K-means clustering and SVM segmentation were used to detect the impurity rate based on quality. Finally, these were compared with the impurity rate obtained by the mass method. The experimental results are shown in

Table 5. Detection results of impurity rate of different algorithms (%).

Detection Method	Improved Canny Operator	Traditional Canny Operator	K-Means Clustering	SVM Segmentation	Quality Standard
Sample 1	6.26377	7.63346	7.72711	4.17013	5.37385
Sample 2	6.76638	6.6714	6.72257	4.56455	6.99251
Sample 3	6.71422	7.30694	10.0976	5.2115	6.84337
Sample 4	6.12833	6.35443	6.48337	5.11429	5.12907
Sample 5	6.71352	6.89394	7.8915	4.38055	6.55075
Sample 6	6.349	7.54174	6.30183	5.94764	6.1273
Sample 7	6.79262	6.40658	13.16772	4.71404	6.70949
Sample 8	6.03854	6.66527	7.15422	6.7967	6.01452
Sample 9	6.33458	7.19902	7.19294	5.49516	6.86348
Sample 10	6.18918	6.47001	11.00082	5.42304	6.98239

.

It can be seen in Table 5 that the impurity rate of the detection method with the improved Canny operator was closer to that of the mass method. The traditional Canny operator was prone to mis-segmentation and had poor applicability to machine-picked cotton images. The detection result fluctuated greatly compared with the improved Canny operator. The detection results of the K-means clustering were affected by the images and fluctuated greatly. The detection results of the SVM segmentation were close to the improved Canny operator. Considering the methods comprehensively, the detection method of the improved Canny operator was selected as the core processing algorithm of the real-time detection system.

4. Discussion

In this study, the impurity content of mechanically harvested seed cotton was analyzed based on an image-processing method, and was closer to the real impurity content, which provided a basis for adjusting the parameters of cotton-processing machinery, which will be helpful in improving cotton-processing quality and increasing economic benefits.

5. Conclusions

(1)

Aiming at the real-time detection of the impurity rate in machine-picked cotton processing, this paper proposed a detection method for the impurity rate in machine-picked cotton based on an improved Canny operator. According to the characteristics of different saturations between cotton and impurities, the impurities were separated by extracting the image S channel.

(2)

In view of the problems existing in traditional Canny operator edge detection, mean filtering and nonlocal mean denoising were used to replace Gaussian filtering; this could effectively remove the noise in the image.

(3)

A YOLO V5 neural network was used to classify and identify the impurities after segmentation, and the density of various impurities was measured. The V–W model was established to solve the impurity rate based on mass.

(4)

Using multithread technology, the processing time was shortened by 43.65% compared with that of a single thread, and the processing frame rate was effectively improved. By finding the average value of the impurity rate, the anti-interference performance of the algorithm was enhanced, and had the characteristics of real-time detection and stability.

This method countered the problems such as a slow solving speed, low detection accuracy, poor real-time performance, and easy interference, and then guided the cotton production and processing processes.