A Machine Vision-Based Method of Impurity Detection for Rapeseed Harvesters

Abstract

The impurity rate is one of the core indicators for evaluating the quality of rapeseed combine harvesters. It directly affects the economic value of rapeseed. At present, the impurity rate of rapeseed combine harvesters mainly relies on manual detection during shutdown, which cannot be monitored in real time. Due to the lack of accurate real-time impurity rate data, the operation parameters of rapeseed harvesters mainly depend on the driver’s experience, which results in large fluctuations in field harvest quality. In this research, a machine vision-based method of impurity detection for rapeseed harvesters, including an image acquisition device and impurity detection algorithm, was developed. The image acquisition device is equipped with a direct-current light source, a conveyor belt, and an industrial camera for taking real-time images of rapeseed samples. Based on the color and shape characteristics of impurity and rapeseed, the detection of rapeseed and impurity was achieved. A quantitative model for the rapeseed impurity rate was constructed to calculate the real-time impurity rate of machine-harvested rapeseed accurately. The field experiment showed that the average accuracy of the detection system for the impurity rate in rapeseed was 86.36% compared with the manual detection data. The impurity detection system proposed in this paper can swiftly and effectively identify rapeseed and impurity and accurately calculate the impurity rate, which can be applied to rapeseed harvesters to provide data support for the adjustment of operating parameters.

1. Introduction

At present, the mechanization of rapeseed harvesting has become an important component of agricultural mechanization [1]. In the actual operation process of the rapeseed combine harvester, the selection of machine operation parameters will directly affect the purity of the harvested rapeseed. Specifically, when the fan speed is too slow and the screen plate angle is not appropriate, the impurity in rapeseed will be greatly increased, which can lead to a decrease in harvest quality and bring direct economic losses to farmers [2,3]. At present, the impurity rate of rapeseed combine harvesters mainly relies on manual detection during shutdown, which is a complex process that entails low efficiency and poor real-time performance. The key operational parameters, such as the impurity rate in rapeseed combine harvesters, are difficult to obtain in real time, and the control of key component operational parameters mainly relies on driver experience, lacking a quantitative basis, resulting in unstable operational quality [4,5]. The rapeseed impurity detection system can monitor the impurity rate in the grain tank in real time during the operation of the combine harvester, transmit the monitoring results to the driver’s cab for display, help the driver adjust the working parameters of the machine promptly, improve the quality of the rapeseed combine harvester operation, and reduce economic losses. However, there is currently no accurate and reliable device for rapeseed impurity detection available on the market. Therefore, further research into rapeseed impurity monitoring technology to achieve visual monitoring and alerting of the harvesting quality of rapeseed harvesters holds significant research value.

With the improvement of computer operation speed and image processing levels, machine vision technology has been widely used in the field of grain quality detection in modern agricultural production because of its advantages of fast response, low damage, and high precision [6,7]. Guan et al. developed an impurity-detection system for tracked rice combine harvesters based on DEM and Mask R-CNN. The results of a bench test showed that the detection accuracy of the designed system for five varieties was 91.15~97.26%. The relative error between the impurity detection system and the manual method was in the range of 5.71~11.72% under field conditions [8]. Chen et al. presented a method of image acquisition, soybean component identification, and quality monitoring for mechanized harvesting based on machine vision. The results showed that the detection accuracy of intact soybean, broken soybean, and impurity components were 87.26%, 86.45%, and 85.19%, respectively [9]. Chen et al. proposed an improved U-Net model for rice impurity segmentation based on machine vision. The test results showed that the comprehensive evaluation indexes of broken grains, straw impurities, and stem impurities reached 92.92%, 90.65%, and 90.52%, respectively [10]. Zhang et al. proposed a lightweight detection method for corn kernel broken rates and impurity rates suitable for small and large detection targets based on YOLOv8n. The bench test showed that the proposed detection method could accurately detect the corn grain breakage and impurity content, and the detection accuracy was as high as 95.33% and 96.15% [11]. Geng et al. proposed a device and method for detecting corn kernel damage based on deep learning, which used a single-layer device to continuously obtain high-quality image data of corn kernels and used a two-stage model of deep learning segmentation and classification to detect damaged corn kernels. The results showed that the classification accuracy of the BCK-CNN model for intact and damaged corn kernels reached 96.5% and 94.2%, respectively [12]. Li et al. designed a sampling device that can be fixed in the grain box of a corn kernel harvester and can achieve a single-layer distribution of grains. In order to better detect small-sized damaged corn kernels, a small object detection layer was added to YOLOv8n for training the corn kernel dataset [13]. Liu et al. constructed an online detection system of soybean crushed rates and impurity rates based on the DeepLabV3+model. The bench test results showed that the relative error between the average value of the online detection method for the soybean crushing rate and the average value of manual detection is 0.36% [14]. Chen et al. proposed an online detection method of the wheat machine harvesting impurity rate based on the improved U-Net model combined with attention. In the bench test and field test, the average online detection of impurity content in the device was 1.69% and 1.48%, respectively, with errors of 0.26% and 0.13% compared to manual detection [15]. Liu et al. proposed a rice grain detection method based on the YOLOv7 fusing of GhostNetV2 to detect the impurity in rice grain. The accuracy and recall rate after three improvements reached more than 90% and met the requirements of the detection index [16]. However, research on the detection of impurity content in rapeseed is still challenging, and there is a lack of reliable detection equipment for rapeseed impurity content in the market [17,18,19]. It is necessary to carry out further research on the online detection system of impurity content in the mechanized harvesting environment of rapeseed.

In recent years, the online detection of the impurity rate of combine harvesters mainly adopts the technical route of combining “material sampling” with “visual recognition”, which identifies impurity through the samples collected by camera [20,21,22]. However, machine vision can only detect impurities on the surface of rapeseed, and the mutual obstruction between rapeseed and impurities has a significant impact on the detection results [23]. The closed sampling device needs to provide a special artificial light source, and the lighting effect has a great impact on the subsequent image processing [24]. These factors restrict the development and application of online detection technology for the impurity rate of rapeseed combine harvesters.

In this research, an impurity detection device for rapeseed harvester was built to solve the missing detection problem of impurity caused by material obstruction and uneven illumination. The shielding of impurity was reduced by adjusting the inclination angle of the conveyor belt and limiting the material layer thickness. The image quality was improved by optimizing the light source form and brightness equalization algorithm. The impurity segmentation algorithm of rapeseed based on color and shape features was studied, and the pixel-mass calibration model of rapeseed and impurity was constructed and verified by experiments, to realize the online detection of impurity rate of rapeseed.

2. Materials and Methods

This study introduces a machine vision-based method of impurity detection for rape-seed harvesters. Initially, an image sampling device suitable for rapeseed harvesters was designed to collect the images of rapeseed harvested in real time. Simultaneously, pixel-mass calibration was performed on images of rapeseed and impurity. Then, the quality of the image was improved by using image preprocessing algorithms. By extracting the color and shape characteristics of impurity and rapeseed, the detection of rapeseed and impurity was achieved separately. Finally, based on the pixel-mass calibration relationship between rapeseed and impurity, the impurity rate of rapeseed was derived from the pixel proportion of impurity in the image.

2.1. Design of Real-Time Detection System for Impurity Rate of Rapeseed Harvesters

The online detection system for the impurity rate of rapeseed harvesters was composed of the rapeseed image acquisition device, an industrial computer, a power supply, a motor, and other working components. Its structure is shown in Figure 1a. The rapeseed image acquisition device mainly included a rapeseed conveyor belt, an industrial camera, a light source, a transparent partition, a scraper, and a metal case, etc. Its structure is shown in Figure 1b. The industrial computer processor, graphics card, and system memory were Intel Core i7-4790S (Intel, CA, USA), Intel high-definition graphics 4600 (Intel, CA, USA), and 8GB RAM (random access memory) (AMD, CA, USA), respectively. The industrial camera was a MER-132-43U3C (IMAVISION, Beijing, China) camera equipped with a VS-LDA4 zoom lens (Zhirui Vision, Chefoo, China), with a resolution of 1292 × 964, a pixel size of 3.75 µm × 3.75 µm, and a frame rate of 43fps. A direct-current power supply was used to power the conveyor belt motor, the light source, and the industrial computer.

The rapeseed image acquisition device was installed under the grain outlet of the rapeseed combine harvester. Rapeseeds, impurities, and other materials entered the image acquisition device from the sampling port. The conveyor belt was driven by a direct-current motor at a stepless adjustable speed. Under the joint action of the conveyor belt and the baffle, the rapeseeds moved from the sampling port to the unloading port. During this period, the rapeseeds were imaged by a camera and then left the conveying device to fall into the grain tank. By adjusting the installation angle of the detection device and the speed of the conveyor belt, the thickness of the material layer was adjusted to reduce the coverage and obstruction of impurities. The direct-current light source provided stable illumination for the detection system, and the acrylic transparent partition separated the image acquisition device into two enclosed spaces. The industrial camera and light source were installed on the completely enclosed upper side, and the rapeseed and conveyor belt were located below the transparent partition to avoid dust damage to the lens during field harvesting. The image acquisition device uses dual direct-current light sources to ensure uniform brightness of the image. In the field harvest experiment, the exposure time of the industrial camera was set to 2000 µs, which can clearly capture the picture of rapeseed material flow.

The industrial camera captured rapeseed images on conveyor belt through transparent partitions and transmitted the pictures to the industrial computer. The industrial computer processed the images in real time to calculate the impurity rate of rapeseed, achieving the online detection of rapeseed impurity.

2.2. Design of Algorithm for Detecting Impurity Rate in Rapeseed

2.2.1. Image Preprocessing Algorithm

At present, the commonly used image enhancement algorithms mainly include histogram equalization and the improved image enhancement algorithm based on Retinex theory and the partition tone mapping algorithm, which has a certain effect on image contrast equalization [25,26]. However, the above methods are also prone to negative effects such as halo and edge blur, which increase the difficulty of subsequent image processing. In this study, we used a contrast limited adaptive histogram equalization (CLAHE) method to balance the brightness of the image dynamically [27]. The CLAHE algorithm improved the global contrast and local detail information of the image by enhancing the traditional histogram equalization (HE) method. The theoretical flowchart of the CLAHE algorithm is shown in Figure 2.

Firstly, the image was divided into several smaller, equally sized tiles, which can be overlapping or nonoverlapping. The size of these tiles (e.g., 8 × 8 pixels) was typically predefined. For each tile, a histogram was computed to represent the distribution of pixel intensities within that tile. Then, histogram equalization was applied to each small tile to resolve the issue of uneven brightness within each small block, thereby improving the contrast of the image. The histogram equalization formula calculated independently within each small tile was as follows:

$$s=(L-1){\int }_0^r{p}_r\left(w\right)dw$$

(1)

where $s$ was the output pixel value, $r$ was the input pixel value, $p_r\left(w\right)$ was the probability density function of the input pixel value, and $(L-1)$ was the maximum possible output pixel value (e.g., 255 in an 8-bit image).

To avoid noise amplification and detail loss caused by excessive enhancement, a contrast limiting factor was set to limit the pixel values after equalization. This factor is usually a value between 0 and 1, which is used to control the contrast enhancement. Specifically, the gray level mapping relationship was obtained according to the histogram of each tile, that is, the mapping from the original gray level to the new gray level. In the process of mapping, if the pixel number of a gray level exceeds the contrast limit factor, the number of pixels of the gray level is trimmed or reassigned to limit the contrast of the gray level. Due to the possibility of image discontinuity between adjacent tiles caused by block processing, the interpolation method (e.g., bilinear interpolation) was used to smoothly transition the gray levels of adjacent blocks to eliminate this discontinuity. Bilinear interpolation was a method of interpolation in two-dimensional space that estimated the pixel value of a given position by considering the four nearest pixels. Finally, all the processed small tiles were reassembled into a complete image to obtain the result of image brightness equalization processing. The results of preprocessing the collected rapeseed images using the CLAHE algorithm in this study are shown in Figure 3.

2.2.2. Analysis of Image Features of Rapeseed

To detect impurity in materials using machine vision, it was necessary to analyze the differences in color space between rapeseed and impurity in the image. The rapeseed images captured by industrial cameras were based on the RGB color model, which differed significantly from the human eye’s perception of color. In contrast, the HSV color model was more suitable for the color representation of machine vision because it was closer to the perception of color by human eyes [28]. Therefore, the HSV color model was selected to process rapeseed image in this study.

The color moment method has been widely used in image detection and classification because of its simple calculation and insensitivity to image size and rotation. As the color information of the image was mainly distributed in the low-order moments, this study used the first-order moments (mean) and the second-order moments (variance) to express the color distribution characteristics of the images. The first moment described the color mean of image pixels, which reflected the overall brightness or hue of the image color. The second moment described the standard deviation of image pixels, which reflected the contrast or dispersion of the image color. The mathematical definition of these color moment features was as follows:

$${\mu }_i={\displaystyle \frac{1}{N}}\sum _{j=1}^N{p}_{ij}$$

(2)

$${\sigma }_i={\displaystyle \frac{1}{N}}\sum _{j=1}^N{\left(p_{ij}-{\mu }_i\right)}^2$$

(3)

where $p_{ij}$ was the pixel value of the image’s j-th pixel in the i-th color channel; $N$ was the total number of pixels in the image; ${\mu }_i$ was the average value of all pixels in the i-th color channel; and ${\sigma }_i$ was the standard deviation of all pixels in the i-th color channel.

In addition, appearance was the key feature to distinguish impurity from rapeseed. The geometric features directly reflect the appearance characteristics of rapeseed and impurity, which is of great significance for machine recognition and the classification of rapeseed and impurity. The Hu first-order invariant moment, the Hu second-order invariant moment, the aspect ratio, the boundary pixels, and the circularity used to characterize the geometric property of the images.

The Hu invariant moments were seven moment eigenvectors constructed from 2nd and 3rd order normalized central moments in the Cartesian coordinate system, proposed by Ming-Kuei Hu et al., with the characteristic of value invariance during image scaling and rotation. When using moments as a description of grayscale images, the geometric moments $M_{pq}$ and the central moments ${\mu }_{pq}$ in the $p+q$ order of a digital image with a size of $W\times H$ pixels can be represented as follows:

$$M_{pq}=\sum _{u=1}^W\sum _{v=1}^H{v}^p{u}^{q}f(v,u)$$

(4)

$${\mu }_{pq}=\sum _{u=1}^W\sum _{v=1}^H{(v-\overline{v})}^p{(u-\overline{u})}^{q}f(v,u)$$

(5)

where $f(v,u)$ was the grayscale value of the image at coordinate point $(v,u)$; $(\overline{v},\overline{u})$ was the centroid of the image, which can be calculated through geometric moments; and $\overline{v}=M_{10}/M_{00}$, $\overline{u}=M_{01}/M_{00}$.

Then, the grayscale changes caused by scaling in the image was removed by normalizing the central moments. The normalized central moment was obtained by dividing by a certain normalization factor.

$${\eta }_{pq}={\displaystyle \frac{{\mu }_{\mathrm{p}\mathrm{q}}}{{{(\mu }_{00})}^r}}$$

(6)

where ${\mu }_{\mathrm{p}\mathrm{q}}$ was the central moment; ${\mu }_{00}$ was the zero-order central moment; and $r=1+(p+q)/2$.

Due to the more prominent descriptive effect of Hu invariant moments in low dimensions, the first-order feature and second-order feature were used here, and their expressions were as follows:

$${\mathsf{{\rm I}}}_1={\eta }_{20}+{\eta }_{02}$$

(7)

$${\mathsf{{\rm I}}}_2={{(\eta }_{20}-{\eta }_{02})}^2+4{{\eta }_{11}}^2$$

(8)

Considering the large data range of the processing results of Hu moments, logarithmic processing was applied to the data for comparison.

$${\mathsf{\Phi }}_i=\mathrm{a}\mathrm{b}\mathrm{s}\left(\log {\mathsf{{\rm I}}}_i\right)$$

(9)

where ${\mathsf{\Phi }}_i$ was the final result of the Hu invariant moment.

The aspect ratio was defined as the ratio of the length to the width of the smallest bounding rectangle of an object in an image. In image recognition, the aspect ratio was used to distinguish objects of different shapes. The aspect ratio of a circular object was close to 1, while the aspect ratio of a slender object was much greater than 1. By calculating and analyzing the aspect ratio of the contour, objects in the image can be better understood and classified.

Circularity was a parameter used to describe the proximity of an object to a standard circle. The mathematical definition of the circularity was as follows:

$$C=4\pi S/\left(L^2\right)$$

(10)

where $S$ was the area of the object in the image; and $L$ was the circumference of the object in the image.

To extract image features of rapeseed and impurities, 50 images were randomly selected from the 100 rapeseed images collected in the field as image samples. Firstly, it is necessary to ensure that 100 images are captured clearly. Secondly, the distribution of impurities and rapeseed in the image is representative, including various forms of rapeseed and impurities. Afterwards, to reduce computational complexity, 50 images were randomly selected as samples. Impurity components and rapeseed components in the sample image were manually intercepted. The feature statistical results of rapeseed and impurity images are shown in

Table 1. Feature statistical results of rapeseed and impurity images.

Component	Impurity	Rapeseed
First-order moment of hue	0.0130~0.0897	0.1046~0.3140
Second-order moment of hue	0.0343~0.1289	0.1746~0.3680
Hu first-order invariant moment	0.4283~3.4806	0.1593~0.3972
Hu second-order invariant moment	0.1495~5.7469	0.0001~0.1038
Aspect ratio	1.2621~13.6539	1.0000~3.4625
Boundary pixels	313~1859	79~540
Circularity	0.0444~0.2913	0.3212~ 0.9152

.

According to the data distribution range of the characteristic parameters in Table 1, the distinction between the first-order moment of hue, the first-order Hu invariant moment, and the distribution range of circularity between impurity and rapeseed was most obvious. Therefore, the combination of the first-order moment of hue, the first-order Hu invariant moment, and the circularity was used to distinguish between rapeseed and impurity.

2.2.3. Segmentation Algorithm for Rapeseed and Impurity

The architectural scheme of segmentation algorithms for rapeseed and impurity is shown in Figure 4. To avoid the impact of image size on algorithm complexity, oversized images need to be resized to a specified size before image processing. Firstly, based on the color characteristics of rapeseed and impurity, the rapeseed and impurity in the image were preliminarily extracted separately. Secondly, the preliminary extracted image was binarized to segment the foreground and background in the image. Thirdly, hole filling and noise pixel deletion operations were performed on the binary image. Finally, the rapeseed and impurity were accurately detected by detecting the shape feature of each connected domain in the binary image.

The original image containing rapeseed and impurity is shown in Figure 5. The segmentation results of rapeseed sample images using the color threshold segmentation method, watershed algorithm, and the image segmentation algorithm used in this paper are shown in Figure 6. Based on the combined features set in Table 1, the rapeseed and impurity in the image were extracted, as shown in Figure 6a. The results of image segmentation based on the color threshold are shown in Figure 6b. The results of image segmentation based on the watershed are shown in Figure 6c. The detection results showed that the detection results based on the color threshold segmentation method and watershed algorithm both have cases of missing impurities or the misidentification of impurities. The image segmentation results based on combined features have the best performance, of which impurity and rapeseed in the image were well detected, and the shape, position, and size of rapeseed and impurity remained unchanged.

To measure the accuracy of the algorithm in detecting impurities and rapeseeds in this article, precision ($P$), recall ($R$), and the comprehensive evaluation indicator $F_1$ were used to quantitatively evaluate the identification results of impurities and rapeseed, respectively. Precision refers to the accuracy rate, which is the percentage of correct parts in the detection result to the entire detection result. Recall refers to the recall rate, which is the percentage of correct parts in the detection results to the actual total correct parts. The comprehensive evaluation index $F_1$ reflects the overall index comprehensively. The calculation formulas are as follows:

$$P={\displaystyle \frac{T_P}{\left(T_P+F_P\right)}}\times 100\%$$

(11)

$$R={\displaystyle \frac{T_P}{\left(T_P+F_N\right)}}\times 100\%$$

(12)

$$F_1={\displaystyle \frac{2PR}{\left(P+R\right)}}\times 100\%$$

(13)

where $T_P$ was the number of pixels for correct detection, $F_P$ was the number of pixels for error detection, and $F_N$ was the number of pixels for missed detection.

2.2.4. Calculation Method of Impurity Rate in Rapeseed

In addition to obtaining the pixel proportion of impurity components through the image processing method, we also calibrated the unit pixel mass ratio of rapeseeds and impurity components, so as to obtain the true proportion of impurity components in rapeseeds. By taking pictures of rapeseeds and impurities at the same height and weighing the corresponding mass and pixel number, the fitting relationship of the unit pixel mass of rapeseeds and impurities was calculated, as shown in Figure 7. Sample images of rapeseeds and impurities (with a resolution of 1600 × 1200) were obtained separately, with 20 images for each category.

Images of rapeseed and impurity were taken at the same height. For each image, the mass of rapeseed and impurity was measured separately, and the number of pixels was counted separately. Finally, the fitting relationship between the unit pixel mass of rapeseed and impurity was calculated. The relationship between the actual quality and the pixel number of the rapeseed was linearly fitted as follows:

$$f\left(x_i\right)=k\times x_i$$

(14)

where x_i was the number of pixels occupied by rapeseeds in the image, f(x_i) was the actual quality of rapeseeds, and k was the fitting coefficient between the quality of rapeseeds and the number of pixels.

The relationship between the actual quality of impurity components and the number of pixels in the image was linearly fitted as follows:

$$g\left(x_j\right)=u\times x_j$$

(15)

where $x_j$ was the number of pixels occupied by impurity components in the image, $g\left(x_j\right)$ was the actual quality of impurity components, and $u$ was the fitting coefficient between the quality of impurity components and the number of pixels.

Therefore, the formula for calculating the actual impurity content of rapeseed grain was as follows:

$$p={\displaystyle \frac{g\left(x_j\right)}{f\left(x_i\right)+g\left(x_j\right)}}\times 100\%$$

(16)

where $p$ was the actual impurity rate in rapeseed.

2.2.5. Software Design of Impurity Detection System

The software of impurity detection for rapeseed harvester was written in Python 3.8. Firstly, based on the software development toolkit of industrial cameras, we set appropriate camera parameters, including exposure time and white balance parameters, to capture the original image of rapeseed. Secondly, the CLAHE method was used for image preprocessing to improve image quality. Thirdly, based on the color and shape features of rapeseed and impurity, the rapeseed and impurity in the image were detected separately. Then, based on the pixel-mass calibration relationship between rapeseed and impurity, the impurity rate of rapeseed was derived from the pixel proportion of impurity in the image. Finally, the test results were displayed in the display window. The software workflow diagram is shown in Figure 8. As shown in Figure 9, the pixel proportion and real impurity content of impurity components are displayed in the upper left corner of the display window.

2.3. Field Test

To verify the accuracy of the online impurity detection system for rapeseed harvesters, a field experiment of rapeseed mechanical harvesting with a rapeseed pickup machine was carried out in Gaoyou Farm, Yangzhou City (119.459° E, 32.782° N), Jiangsu Province, May 2024. The rapeseed in this plot was planted by mechanical direct seeding. The test machine was the Kubota 4LZ-4J(PRO988Q-Q) PLUS (Kubota, Shanghai, China) type full-feed tracked combine harvester, as shown in Figure 10. The forward speed of the harvester was 0.95 m/s. The harvested rapeseed variety was the “Ningza 1818”. Its basic properties were as follows: an average plant height of 1400 mm, an average row spacing of 320 mm, a yield of 3600 kg/hm², rapeseed moisture content of 21.47%, and a one-thousand seed mass of 3.96 g. Rapeseed harvesting experiments were carried out on the harvester for 6 trips, and the harvesting distance of each group of experiments was 5 m.

After each group of harvest tests, the rapeseed sample from the grain tank of the rape harvester was and weighed as $M_1$. After removing the impurity, the net mass of rapeseed was weighed as $M_2$. The impurity rate of manual detection was calculated as follows:

$$P_m={\displaystyle \frac{M_1-M_2}{M_1}}\times 100\%$$

(17)

where $P_m$ was the impurity rate of manual detection.

3. Result and Discussion

3.1. Pixel Density Calibration

In this study, the data of the mass and the pixel number of the rapeseed and impurity were fitted with a straight line based on the principle of least squares. By setting the minimum sum of squares of the vertical distances (i.e., errors) from all data points to the line, the optimal function matching of the data was achieved. The relationships between the pixel number and the mass of rapeseed and impurity are shown in Figure 11.

The relationship between the actual quality and the pixel number of the rapeseed was linearly fitted as follows:

$$F\left(x_i\right)=5.0198\times {10}^{-5}{x}_i$$

(18)

where $x_i$ was the number of pixels occupied by rapeseed in the image, and $F\left(x_i\right)$ was the actual quality of rapeseed.

The relationship between the actual quality of impurity components and the number of pixels in the image was linearly fitted as follows:

$$G\left(x_j\right)=0.5993\times {10}^{-5}{x}_j$$

(19)

where $x_j$ was the number of pixels occupied by the impurity component in the image, and $G\left(x_j\right)$ was the actual quality of impurity components.

The R-square (coefficient of determination) and RMSE (root mean squared error) were used to evaluate the fitting results. The R-square and RMSE of regression Equation (18) were 0.9912 and 0.2667, and regression Equation (19) were 0.8231 and 0.1090, respectively. The accuracy for the rapeseed pixel density is higher than that of the impurity. It is because the moisture content of impurity is not as stable as that of rapeseed.

Therefore, the actual mass ratio of rapeseed per unit pixel to impurity was simplified to 8.3761:1 in this study. The impurity rate of rapeseed can be directly calculated based on the detected impurity pixels and the number of rapeseed pixels in the image. The calculation formula was as follows:

$$p={\displaystyle \frac{x_j}{{{8.3761x}_i+x}_j}}\times 100\%$$

(20)

where $p$ was the impurity rate of rapeseed, $x_i$ was the number of pixels occupied by rapeseeds in the image, and $x_j$ was the number of pixels occupied by impurity in the image.

3.2. Result of Impurity Detection

To verify the accuracy of the image segmentation algorithm, this study conducted statistical analysis on the detection results of impurity and rapeseed in 100 images. The rapeseed and impurities detected by the segmentation algorithm were compared with the manually annotated images. The results are shown in

Table 2. Quantitative evaluation of detection results.

Detect Category	Precision (%)	Recall (%)	${\mathit{F}}_1$ (%)
Impurity	91.58	89.47	90.51
Rapeseed	85.62	84.31	84.96

.

The value of the comprehensive evaluation index $F_1$ of impurity was 90.51%. Most of the impurities in the image were accurately identified with well-preserved shapes and sizes. The value of the comprehensive evaluation index $F_1$ of rapeseed was 84.96%. The reason for the false detection of rapeseed is that some rapeseed is adhered to impurity, and their colors are similar, leading to the identification error. In future research, the detection accuracy will be improved by analyzing the color, shape, and texture characteristics of rapeseed and impurities and further subdividing the types of detection objects.

3.3. Field Test

During the experiment, the image acquisition device operated smoothly without any material congestion. The impurity detection software worked stably and accurately detected the impurity in mixed materials. The field test results of machine detection are shown in Figure 12 and Figure 13. The original image of rapeseed collected by the detection system is shown in Figure 12a, which was trimmed to 800 × 800 pixels to reduce the amount of data for image processing. The binary diagram of impurity detected by the system is shown in Figure 12b. The binary diagram of rapeseed detected by the system is shown in Figure 12c. It can be seen from Figure 12 that the detection results of impurity were relatively accurate. Because of the similar color of rapeseed and the conveyor belt, there were a small number of false identifications in the test results. In future work, this problem will be improved in both hardware and software aspects. Specifically, in terms of hardware, by adjusting the design of the device, the color difference among the conveyor belt, rapeseed, and impurity will be increased, thereby reducing the difficulty of detecting rapeseed and impurities. In contrast, on the software front, the detection accuracy of rapeseed and impurities will be improved by extracting the color, texture, and other features of the conveyor belt, rapeseed, and impurities separately, and then distinguishing them based on the features with the greatest difference. In addition, the segmentation of impurities and rapeseed will be improved by using more advanced algorithms, such as deep learning, to deal with variations in color, shape, and texture.

The results of impurity rates detected by the rapeseed impurity detection system are shown in Figure 13. The average value of the impurity rate result detected by the machine was 3.80%. The results of the impurity rate detected by manual sampling are shown in

Table 3. Results of rapeseed impurity rate detected by manual sampling.

Manual Detection	Gross Mass (g)	Net Mass (g)	Impurity Rate (%)
Sample 1	147.0	140.8	4.22
Sample 2	202.2	192.9	4.60
Sample 3	254.9	243.6	4.43
Sample 4	266.2	251.9	5.37
Sample 5	338.5	325.8	3.75
Sample 6	448.2	430.1	4.04

. Six groups of rapeseed samples were randomly selected manually to detect the impurity rate of rapeseed. The average value of the six groups of manual detecting results was 4.40%. The field test showed that the average accuracy of the detection system for rapeseed impurity rate was 86.36%.

3.4. Discussion of Segmentation Algorithm

By configuring an image acquisition device, a stable lighting environment and image background were provided for the impurity detection system, laying a good foundation for using classical image processing methods in this study. After extracting various features such as the color and shape of rapeseed and impurity, we proposed an impurity detection algorithm based on combined features to achieve the accurate detection of rapeseed and impurity.

The deep learning algorithms and classical image segmentation methods have their own advantages [29,30,31]. The reason for adopting classical methods in this study lies in their simplicity and comprehensibility, rapid calculation speed, suitability for real-time applications, and minimal data requirements [32,33]. The limitations of classical methods lie in their weak generalization ability, dependence on manually designed parameters, and poor adaptability to complex scenarios. In contrast, deep learning algorithms, trained on large-scale datasets, can automatically learn effective feature representations, boasting strong generalization capabilities and the ability to handle complex scenarios [34,35,36,37,38]. However, deep learning models are relatively complex, requiring a large amount of labeled data for training and posing high hardware requirements.

Therefore, the next step of this project involves selecting a lightweight deep learning model with a small training sample to train the recognition of rapeseed and impurities, thereby enhancing the accuracy of impurity detection.

4. Conclusions

In this research, a rapeseed image acquisition device and impurity recognition algorithm based on machine vision were developed to detect the impurity rate of rapeseed harvesters in real time. The rapeseed material was laid in a single layer to reduce occlusion by using an image acquisition device. Impurities in rapeseed images were identified through image processing algorithms, and the real-time impurity rate of rapeseed was calculated based on the fitting relationship between the pixels and the quality of rapeseeds and impurities. The specific work is summarized as follows:

(1)

In this research, a rapeseed image acquisition device was designed to avoid the interference of dust in the grain bin on the camera and restrict the rapeseed material to flow uniformly in a single-layer layout, thereby reducing the obstruction between materials. The device was equipped with a light source to solve the problem of large changes in natural lighting and an image processing module to process real-time image data and calculate the rapeseed impurity rate.

(2)

Rapeseed images were binarized by setting the color and shape features of rapeseed and impurity in the image in HSV color space. The impurity components in the image were distinguished from the background using morphological processing methods. By manually calibrating the pixel count and the mass of rapeseed and impurity components, the fitting relationship between the mass and the pixel number of rapeseed and impurity was calculated separately.

(3)

The hardware and software of the monitoring system were integrated into the rapeseed harvester and tested in the field environment. Experimental results have shown that the impurity detection system designed in this study has an accuracy of 86.36% in detecting the impurity rate of rapeseed compared to manual detection, which can be used as an effective means for evaluating the performance of rapeseed harvesters.

The accuracy of the rape impurity detection method based on machine vision is mainly affected by two factors: the accuracy of image detection and the fitting relationship between pixels and quality. In future research, the accuracy of impurity detection will be improved by analyzing the color and texture characteristics of rapeseed, impurities, and the conveyor belt and further subdividing the types of detection objects. By increasing the image of rapeseed and impurity and the number of samples for pixel-mass calibration, a more accurate fitting relationship between them will be obtained. In addition, the use of convolutional neural networks to improve the accuracy of impurity detection and the integration of impurity detection systems with the control system of harvesters are also the future research tasks of this project.