Lemon Grading and Sorting Using Computer Vision ^†

Abstract

This paper presents an automatic system for sorting and grading lemons using computer vision. It eliminates human errors in sorting processes. Lemons are sorted into three categories; ripe, semi-ripe, and a combined class of defective and unripe. A camera is used to capture an image of the lemon, and image analysis is done using Raspberry Pi. A conveyor belt system and a mechanical pusher put the lemon into its respective class.

1. Introduction

Image processing and computer vision that use object recognition and edge detection to avoid obstacles have been used extensively, for example, in robots and self-driving cars. Face recognition systems use computer vision for security. In agriculture, computer vision has been applied for several tasks such as grading, counting, and sorting for about two decades. Sorting and grading using computer vision have been improving over time. It allows farmers to categorize products accurately and provides them with better control over their products, enabling farmers to make better decisions about the target market.

The sorting and grading of fruits and vegetables have an important role in the post-harvesting process. Manually sorting and grading the products is a very tiring job and requires large amounts of time and workers to complete the task. Computer vision techniques can help the farmer categorize fruits and vegetables correctly. Computer vision can be applied to count the fruits in images and can provide a good estimate about the number of fruits even before harvesting. Automated sorting and grading of vegetables and fruits using computer vision are accomplished using a digital photograph of the products. The automated sorting and grading use non-destructive visual features to classify the products, meaning the product can be classified quite accurately without damaging it.

The automated system is not guided by humane interventions and is not subject to biological limits such as tiredness and diversion of attention. Therefore, it can provide consistent results.

2. Literature Review

Mangoes can be sorted based on their color and shape. The geometric features such as shape can be compared with a reference shape. Shape analysis is a good feature for a variety of mangoes. For grading purposes, the pixel value is another good feature. A pixel value greater than 100 means the skin is good and pure. This method has 83.3% accuracy [1].

To evaluate the quality of fruit, a new method was proposed using the HIS color model. A digital image of fruit, taken using a CCD camera and captured in RGB color space, was transferred into HIS color space. The color intensity histogram of only the hue H channel was calculated. The histogram was provided to the backpropagation neural network as input. The output of the network was the description of the quality of the fruit [2].

Fruits can be sorted based on their maturity. A camera is used to acquire a digital image, followed by noise removal using a pseudo-median filter. The image is then converted to binary for edge detection. The method is 90% accurate overall [3].

A date fruit sorting and grading system was proposed. The hardware section included a conveyor belt system with a camera integrated into it. A computer loaded with software was used to analyze the digital image of dates and classify them. The overall accuracy was found to be 80%. Problems associated with detecting the flabbiness of the fruit were observed [4].

A robot was designed to identify and pick fruits automatically using computer vision. A physical system was designed that could be mounted to a tractor. A camera was used to capture the images. The image was further processed to detect defective apples. A vacuum grabber was used to pick the apples [5].

A lemon grading embedded system based only on color and volume was designed. No defect-based classification was done. Greenish lemons with smaller sizes were considered as grade B, while larger yellowish lemons were considered as grade A. Two cameras captured the image to cover the maximum lemon area [6].

3. Methodology

Figure 1 shows the block diagram of the developed system. The system consists of four main stages.

3.1. Stage 1: Image Acquisition

An image of the top view of the lemon is captured using a camera inside an image-capturing chamber. The chamber is lit using a distributed light to minimize the glare due to the spherical shape of the lemon. It is a 3-channel RGB image.

3.2. Stage 2: Pre-Processing

The image needs to be enhanced and pre-processed to extract useful features. The following pre-processing steps are taken:

• Cropping the fruit area.
• Background removal; the black background is used to make this operation easy.
• Noise removal; a 2D Gaussian filter is used in this operation.

The pre-processed image is shown in Figure 2.

3.3. Stage 3: Feature Extraction

Useful features are extracted in this stage. Color is an important feature for ripeness detection. Mean values of red, green, and blue channels are used for the purpose. The second most important feature is the size of the lemon, which is determined by counting the number of pixels under the area of fruit. Defective fruit has irregularities and spots on the skin. The following features are important to measure surface impurity:

• Global standard deviation is a measure of the smoothness of the overall lemon skin.
• Local standard deviation determines the variation of image intensity locally in 64 × 64 patches.
• Center surround is a spatial filter independent of the global context.

3.4. Stage 4: Prediction

Extracted features are normalized and used as input to a backpropagation neural network. The neural network was trained using 99 samples of lemons. The trained neural network was then used to predict the quality of lemons.

4. Results and Discussion

The neural network’s error rate for training, testing, and validation data sets is shown in Figure 3. It can be seen that the training, testing, and validation errors decrease with the number of training iterations, and an optimum solution was found after 21 iterations.

A cross-entropy plot is the measure of the quality of neural network predictions rather than just the classification error. The training error after 21 iterations was reduced to 4.3%, whereas the testing error was about 6%. A confusion matrix shows the overall classification accuracy of the neural network in Figure 3. It shows an overall accuracy of about 94%. Two out of twenty-seven samples of good quality lemons were classified as average quality, and one was classified as defective or unripe since the third category combines both defective and green lemons. There is no misclassification for average lemons; therefore, an accuracy of 100% was obtained, shown in column four. Three of the defective or unripe lemons were misclassified; one was classified as good, and two were classified as average quality.

The fourth row of the confusion matrix shows the true positive and false-positive rates. The figure shows that a total of twenty-five lemons were classified as good quality, and only one lemon in this class was false positive, compared to the average class where out of twenty-two lemons, four were false positive. Fifty-two lemons were classified as defective or unripe, out of which only one was a false positive. No lemon from the average class was misclassified to other classes, and this class also had the highest number of false positives.

5. Conclusions

The objective of the project was to eliminate human intervention in decision-making. A physical system was designed to make the whole sorting process automatic. Two major steps are pre-processing and feature extraction. The neural network was trained, and overall accuracy of 94% was achieved.