You are currently viewing a new version of our website. To view the old version click .
MDPIMDPI
Search all articles
Agronomy
Download PDF
  • Review
  • Open Access

16 October 2024

From Outside to Inside: The Subtle Probing of Globular Fruits and Solanaceous Vegetables Using Machine Vision and Near-Infrared Methods

,
,
,
,
,
,
and
1
School of Mechanical Engineering, Xihua University, Chengdu 611743, China
2
School of Mechatronics, Chengdu Agricultural University, Chengdu 611130, China
3
Institute of Urban Agriculture, Chinese Academy of Agricultural Sciences, Chengdu 610213, China
*
Author to whom correspondence should be addressed.
Agronomy2024, 14(10), 2395;https://doi.org/10.3390/agronomy14102395
This article belongs to the Special Issue Intelligent Detection and Classification of External Traits in Crop Plants, Fruits, and Vegetables
Version Notes
Order Reprints

Abstract

Machine vision and near-infrared light technology are widely used in fruits and vegetable grading, as an important means of agricultural non-destructive testing. The characteristics of fruits and vegetables can easily be automatically distinguished by these two technologies, such as appearance, shape, color and texture. Nondestructive testing is reasonably used for image processing and pattern recognition, and can meet the identification and grading of single features and fusion features in production. Through the summary and analysis of the fruits and vegetable grading technology in the past five years, the results show that the accuracy of machine vision for fruits and vegetable size grading is 70–99.8%, the accuracy of external defect grading is 88–95%, and the accuracy of NIR and hyperspectral internal detection grading is 80.56–100%. Comprehensive research on multi-feature fusion technology in the future can provide comprehensive guidance for the construction of automatic integrated grading of fruits and vegetables, which is the main research direction of fruits and vegetable grading in the future.

1. Introduction

According to the statistics of the Forward-looking Industry Research Institute, the global fruit planting area used to be 64.86 million hectares, and the output used to be 88.7 million tons in 2020. China, India and the United States were among the top fruit producers in the world [1,2]. The global vegetable output used to be 1.15 billion tons in 2020. China, India and Brazil were among the top vegetable producers in the world. With the development of global agriculture and information technology [3] and the rise of related industries, it is very important to test and grade the quality of fruit and vegetables. The augmented production of fruit [4] and vegetables [5] has resulted in a progressive increase in the demand for labor worldwide. However, owing to the scarcity of the labor force globally, the traditional fruit and vegetable grading is readily influenced by human psychological factors [6], characterized by high intensity [7,8], low efficiency [9] and low precision. Hence, the application of novel detection and grading technology is urgently required [10]. The research of machine vision technology, near-infrared spectroscopy technology and hyperspectral technology has effectively solved the above problems of traditional grading.
Nowadays, machine vision technology, near-infrared light technology and hyperspectral technology are rapidly maturing and being applied to fruit and vegetables grading. Machine vision can improve the efficiency of the fruit and vegetables sorting process, reduce labor costs and pave the way for automation in agriculture by introducing faster, cost-effective and non-destructive methods. By analyzing the spectral characteristics of fruits and vegetables, near-infrared light technology and hyperspectral technology can quickly detect the maturity [11], quality and intrinsic attributes of fruits and vegetables such as moisture, sugar and nutrition. Varghese, Renju Rachel et al. [12] proposed a real-time fruit and vegetables grading system based on machine vision for the shelf life of fruits and vegetables to help users choose ideal vegetables and fruits for consumption. Tang Yunchao et al. [13] used machine vision technology to identify and locate fruits in complex environments, and reviewed two main methods for fruit recognition and location, including digital image processing technology and algorithms based on deep learning. Ismail, Nazrul et al. [14] used a variety of models to grade apples and bananas, and found that the Efficient Net model had the highest accuracy of apple and banana grading, 96.7% and 93.8%, which made a contribution to the future automatic-grading accuracy. Fan Shuxiang et al. [15] independently developed a rapid-response portable Vis/NIR device prototype. The experimental results showed that the ratio of the predicted determination coefficient (R-p (2)), the predicted root mean square error (RMSEP) and the standard deviation of the reference destructive SSC to RMSEP (RPD) was 0.690, 0.604% and 1.794, respectively, proof that the device can detect Apple internal features. Yuan Yuhui et al. [16] used three object detection algorithms, Faster R-CNN, Yolov3-Tiny and Yolov5s, to extract the bruised area of apples for damage detection to grade the bruises of apples. The results showed that the accuracy of the three algorithms for early bruised and non-bruised apples was more than 99%. The accuracy was above 96% for apples with no bruising, mild bruising, and severe bruising, and the shortest detection speed for a single image was 6.8 ms. When Unal, Zeynep. et al. [17] used NIR data for training, AlexNet, InceptionV3, and VGG16 models had high accuracy in bruise detection, with 99.33%,100%, and 100%.
In view of this, in order to achieve accurate grading of the exterior and interior of fruits and vegetables, researchers have carried out extensive research in machine vision technology, near-infrared light technology, and hyperspectral technology. In this paper, the advantages and disadvantages of different experiments are summarized and analyzed. This paper reviewed the research progress of the three technologies, discussed how to effectively improve the grading efficiency, and analyzed the limitations and challenges of the existing technologies, as well as the development trend of fruits and vegetable grading. It is pointed out that automatic grading in a multi-source environment, improving the recognition accuracy of fruits and vegetables, and automatic integrated grading are important directions for future research.

2. External Inspection of Fruits and Vegetables

Machine vision is one kind of comprehensive technology that emulates the human eye using computers and can identify specific images through training. It mainly includes five steps: image acquisition, image preprocessing, feature extraction, size grading and hierarchical output. It is widely applied in various fields of social production. Due to its advantages of high precision, fast speed, and user-friendly operation, products from international industry leaders such as Cognex (Boston, MA, USA), Basler (Arnsberg, Germany), Keyence (Japanese large version), and Omron (Kyoto, Japan) have found extensive use in external inspection and grading of fruit and vegetable products [18]. Autoline’s fruit grading equipment from the United States holds a leading position globally [8]. The process of fruit and vegetable grading based on machine vision technology mainly includes the following core steps: image acquisition, image preprocessing, feature extraction, size grading, and graded output. This section presents a comprehensive study on the application of machine vision in fruits and vegetables grading, as shown in Table 1, and elaborates the methods, conclusions, advantages and disadvantages of each study. A detailed review is made for the follow-up research.
Table 1. Research and comparison of machine vision technology in fruit and vegetable grading.

2.1. Fruit and Vegetable External-Size Detection

In terms of citrus color and grade detection [33,34], Zou Wei [19] transformed RGB images into HSV images and processed them with image noise reduction and binarization. The Canny algorithm was used for edge detection and the Find Contours function was used to identify the contours of citrus fruits. The area of the contour is calculated and graded by the counter area function (Figure 1). Experimental results show that the proposed method can accurately distinguish citrus fruits of different sizes and qualities. It has made a contribution to the target area recognition, but this method is only suitable for static, not for dynamic situations. Li Lang et al. [20] developed an online citrus detection system using machine vision. (Figure 2) In this system, each frame image is segmented, sorted, denoised, and two-dimensional coloring ratio is extracted to reduce the influence of coloring rate in the image. Experimental results show that the maximum error tolerance is 5%, and the overall grading accuracy is 90.54%. Compared with Zou Wei [19], the accuracy of this experiment is higher, and it is suitable for grading in dynamic environments.
Figure 1. Machine-vision technology features.
Figure 2. Schematic diagram of the whole machine structure [19]. 1. Hoister. 2. Sorting mechanism. 3. Chain delivery mechanism. 4. Image acquisition system. 5. Screening unit. 6. Electric cabinet. 7. Cup. 8. Belt flipping module. 9. PT-A. 10. Orange. 11. PT-B. 12. LED. 13. Ken. 14. Industrial camera.
For the apple image recognition and grading process (Figure 3), Shi Ruiyao [21] used the Support Vector Machine (SVM) to segment and fill the apple image, calculated the threshold according to the OTSU algorithm, and reconstructed the image (Figure 4). In the experiment, the penalty factor C = 0.01 and the number of iterations 1000 have the best effect, reaching 95% accuracy. The experiment carried out a comprehensive detection, and can identify the defects by extracting the image, but there will be errors in the stem or calyx of the apple, which will be improved in the future. Peng Yankun et al. [22] used machine vision and a robotic arm to detect and grade apples [35,36]; the grading accuracy reached 95%, and the grading time was about 5.2 s. Using the PP-YOLO target-detection algorithm and the control software developed in Python to train and test images has high reliability and practicability. However, the grading speed will be affected by the manipulator. Liu Jiahao et al. [23] used the improved algorithm to fit and extract the edge image of apple, and then converted it into HIS Canny edge-detection image to calculate its roundness. Compared with the traditional manual identification of the apple diameter, the error is within ±1.5 mm, which meets the actual grading requirements. This method effectively solves the problem of high false detection rate and low detection efficiency of spherical fruit.
Figure 3. System hardware [21].
Figure 4. Core methods [21].
Rao Jian et al. [24] employed an online detection system for the identification and grading of spherical fruits (Figure 5). The proposed method utilizes the minimum external matrix to determine the fruit shape index, while also analyzing fruit surface defects through morphological region-filling techniques. In this experimental study, a total of 120 test sets were utilized for grading detection, resulting in the detection of 18 special-grade fruits, 39 first-grade fruits, 39 s-grade fruits, and 24 externally identical fruits. The experimental results demonstrate an average recognition rate of 94.4%. This research possesses the advantages of extracting surface-defect features accurately and achieving high grading accuracy. However, it has limitations in terms of not being able to detect the entire fruit surface and potential impact from fruit surface-defect characteristics on the detection results.
Figure 5. Overall dimension detection system of spheroid fruit.
Zhang Yuhua et al. [25] employed multi-sensor information fusion technology for the detection of various fruits. The findings demonstrate that fruit grading should be approached from multiple perspectives, rather than a singular standpoint. This study enables accurate fruit grading, although it is limited by single-point detection and predominantly static images collected in the university laboratory, which do not fully capture the dynamic nature of real-world scenarios. Su Boni et al. [26] identified and graded strawberries [37,38]. The images are collected by SLR camera, and the images are processed by median filter for denoising, gray enhancement and binarization. The experimental results demonstrate that a white background enables clear segmentation of strawberry fruit, while the median-filtering algorithm effectively removes salt-and-pepper noise present in the image collection process. The five typical algorithms successfully segment the image contour. This experiment offers online, lossless, and real-time advantages; however, it does not consider factors such as strawberry rot or breakage, nor does it address complex background scenarios when completing strawberry fruit segmentation and maturity recognition. The machine vision technology was employed by Li Jianhang et al. [27] to accurately identify and classify blueberries (Figure 6). The image segmentation was performed using the maximum between-cluster variance method, followed by morphology-based removal of connected regions and fitting using the least squares method [39]. Experimental results demonstrate that the average accuracy for calculating blueberry fruit area is 98.93%, while the average accuracy for calculating blueberry fruit circumference is 87.74%. Comparing to manual segmentation as the standard calculation accuracy, area grading proves to be more accurate than perimeter grading. Blueberry fruits are classified based on their area and perimeter calculations, offering a low-complexity and cost-effective approach. However, it should be noted that accurate segmentation of fruit stems remains challenging, particularly in cases where images are unimodal.
Figure 6. Schematic diagram of picture acquisition. 1. PC. 2. Blueberry. 3. Light source. 4. Industrial camera. 5. Camera support.
The potato peel images (Figure 7) were preprocessed using MATLAB R2016a in Tang Zhensan’s [28] study on potato grading technology [40]. Additionally, support vector machine and BP neural network models were developed to effectively grade and classify the roughness of the potato peel. In this particular experiment, a total of 79 potato samples were employed, with 55 samples allocated as the training set and 24 samples designated as the test set. The experimental findings demonstrate that SVM achieves an average accuracy rate of 87.5%, surpassing that achieved by the BP neural network in terms of potato peel grading. Consequently, a highly effective grading model was established utilizing support vector machines alongside feature comparison using GLCM, enabling recognition and grading of potato peels. However, it is worth noting that one limitation is the relatively lower recognition accuracy.
Figure 7. Technical roadmap of potato surface recognition [19].
Zhang Yanjun [29] optimized the yolov6 neural network to accurately predict the Huanghua joint node, and added the heat map branch to enhance the non-maximum suppression algorithm. The experimental results show that the recognition accuracy is 91.02%, and the positioning accuracy is 99.8%. The Agaric recognition grading system (Figure 8) was developed by Wang Mengxin et al. [30], using image processing techniques of the DMV-VGR software and PLC data monitoring. The system is able to perform initial grading and provide efficient and accurate grading solutions. The disadvantages are low grading accuracy and lack of omnidirectional features.
Figure 8. Overall design scheme of the system.
Bhargava, A. et al. [31] used Gaussian filtering for denoising, fuzzy C-means clustering and grab-cut for image segmentation in their experiments on detection and grading of various vegetables and fruits, respectively. Subsequently, they compared the accuracy with SVM and four other decision methods. The experiments were conducted on five vegetables and four fruits. Experimental results show that the accuracy of SVM decision detection reaches 97.63%, and the accuracy of grading reaches 96.59%. The SVM with the highest accuracy is selected for validation. Renju Rachel Varghese et al. [12] used SVM, K-NN and the Anna algorithm to design a system that can identify and grade a variety of fruits and vegetables without a network, and the overall accuracy of the experiment reached 70%. Anuja Bhargava and Atul Bansal [41] developed a fruit grading system using K-NN, SVM, SRC, ANN, and other methods. The system shows superior accuracy compared to existing techniques, being able to analyze and identify fruits based on color, geometry, statistics, and texture.

2.2. External-Defect Detection of Fruits and Vegetables

A set of image acquisition devices was developed by Chen Yingxue et al. [32] (Figure 9) for lossless feature recognition and grading, using mangoes. In the experiment, Sobel operator and Canny operator are used to detect the edge, which solves the problem of unclear and discontinuous edge in the previous defect recognition. The results indicate that the accuracy of first-grade fruits is 93.3%, while second-grade fruits exhibit an accuracy of 95%, and third-grade fruits demonstrate a similar accuracy level of 95%. This research successfully addresses issues related to unclear and discontinuous edges, while also mitigating noise interference in the images captured. However, it should be noted that variations in illumination during experimentation can impact image quality.
Figure 9. Image acquisition device. 1. Objective table. 2. Mango. 3. Light source. 4. Light source. 5. Industrial camera. 6. Date line. 7. Computer.
Due to the good grading performance of the VGG network, Zhou HaiYan [42] used the Random Weighted Average (SWA) optimizer and w-softmax loss function to improve the VGG network to identify Qingmei defects. The accuracy of defect grading and recognition is increased by 9.8% and 16.6%, and the test speed is increased by 1.87 ms and 6.21 ms, respectively. This experiment can well identify the external defects of green plum, but cannot identify the stem of green plum and the whole green plum. This experiment is not suitable for dynamic identification. For dynamic defect identification, Chen Yaohui [43] developed the SORT algorithm to track and identify citrus defects, as shown in Figure 10, which made up for the shortcomings of static identification. The overall recall, precision, and F1 score are 0.87, 0.88, and 0.871, respectively. The accuracy of defect-grading judgment in dynamic recognition is low, and high accuracy is needed to locate the position of the orange.
Figure 10. Comparison of mobile-citrus network results and development trace results. The tracking series of five identified defective mandarins are shown in figure. Green box stands that this mandarin is identified as normal case, while red box stands that this mandarin is identified as defect [43].
Non-destructive identification of defects in fruits and vegetables is still a difficult problem in the field. Zhang Xinxing [44] uses the optimized YOLO-V4 model for detection and the Efficient Net model for grading. The accuracy and F1 score are 0.890 and 0.872, respectively. The YOLO-V4 model before and after optimization identifies defects well, and after optimization it can be classified more accurately (Figure 11), but it will lose confidence, location, and grading. Improving the model accuracy and simplifying the model are the main research directions in the future.
Figure 11. Recognition and grading before and after optimization [44].
The YOLOv5 network uniformly changes the size of the input image to 640 × 640, which will improve the speed of inference and detection. The YOLOv5 model was optimized by Hu Wenxin [45], and the training comparison with the YOLOv5x model is shown in Figure 3. The average precision, precision and recall reach 95.5%, 94.0% and 95.1%, respectively, which are 5.8%, 3.6% and 7.6% higher. At the same time, the image detection speed is increased by 22.1 ms. Experiments show that the improved network can achieve good performance in citrus skin defect detection, but there is a problem, in that the defect data set is difficult to collect.
We examined recent studies on fruit defect detection, which are organized into tables as shown in Table 2. In recent years, most works on fruit defect detection are based on convolutional neural networks. The datasets used in the literature mainly consist of an image with a single object in a laboratory environment as the dataset and a dataset with multiple objects in an orchard environment as the background. However, the datasets used in the above studies are different, so it is not possible to directly compare the performance of the detection networks used. For fair evaluation, the image content and background, network and detection performance of the dataset were selected for comparative study. As can be seen from Table 2, the detection effect of the optimized detection network is better than that of the original network, and the detection difficulty of the data set collected in the orchard environment is usually greater than that in the laboratory environment. This indicates that it is effective and necessary to improve the network when the image contains multiple target objects and a complex background.
Table 2. Some research studies on fruit defect detection in recent years.
The accuracy of machine vision in grading the external dimensions and defects of fruits and vegetables is 70–99.8% and 88–95%, and it has good grading ability under specific lighting conditions. However, there are still some shortcomings, such as high hardware and lighting requirements, a relatively single environment, complex image processing, and being unable to detect multiple defects at the same time. In the future, deep-learning and computer-vision technologies should be combined to develop more efficient image processing algorithms to improve the recognition ability of surface defects in fruits and vegetables. Using the AI Vision cloud platform improves the simultaneous detection of multiple defects through online annotation and learning, reducing the complexity and cost of custom development.

3. Internal Inspection of Fruits and Vegetables

The near-infrared spectrum refers to the range of wavelengths between the red edge of the visible spectrum and the infrared spectrum, spanning from 780 to 2526 nm [46]. Detection technology utilizes electromagnetic waves within the wavelength range of 700 to 1100 nm [47]. In recent years, the Perkin Elmer factory and Thermos factory in the United States, and the Bruker company in Germany have ventured into research and development of near-infrared spectrometers. Infrared spectroscopy is widely used in many fields, such as food detection, chemical analysis, and biological research. For fruits and vegetables specifically, near-infrared light technology enables non-destructive detection of various indicators such as sugar content and moisture levels. Its advantages can be harnessed for damage-free fruits and vegetable grading, with significant implications. The United States conducted the initial study on imaging spectrometers in 1983. AVIRIS and DAIS in the United States, FLI and CASI in Canada, ROSIS in Germany and HyMap in Australia have successively studied fruit and vegetable grading in the hyperspectral field. The “atlas and spectrum integration” technology of hyperspectral analysis utilizes a wide range of continuous-wavelength spectral data to extract object characteristics. This technology captures information within the visible-light-to-infrared spectrum, enabling change detection and target tracking [48]. It provides more detailed and comprehensive data compared to human eyes or conventional photography. Hyperspectral “map and spectrum integration” technology has found extensive applications in diverse fields such as agriculture, geological exploration, environmental monitoring, and medical diagnosis. Table 3 shows the application of machine vision and NIR spectroscopy and hyperspectral technology in fruit and vegetable grading in recent years.
Table 3. Comparative study of near-infrared light technology in fruit and vegetable grading.
The main process of fruits and vegetables grading by near infrared spectroscopy technology includes the selection of a calibration sample set and prediction sample set, spectrum collection, chemical value detection and, finally, the establishment and testing of the mathematical model. The process of fruit and vegetable grading based on hyperspectral technology mainly includes the following core steps: data acquisition and preprocessing, data dimensionality reduction, model construction and grading. The application of NIR spectroscopy and hyperspectral technology in fruit and vegetable grading is reviewed in detail below.
Liao Zhiqiang [49] used six preprocessing methods (Figure 12) for internal detection and grading of apples, and used the ridge model for preprocessing; the accuracy of grading ranged from 88.38% to 90.84%. Experimental results show that the ridge regression model has better stability and higher grading accuracy. Peng Yankun et al. [22] developed a robot hand system with lossless perception and grading (Figure 13). The PP-YOLO target-detection algorithm is used to improve the detection speed, the normalized spectral ratio method is used for brix modeling, and NSR + CARS is used as the robot-hand spectrum model. The accuracy of the method is 0.9655, the recall rate is 1, the accuracy rate is 0.998 4, and the model detection speed is 38 frames/s, which further improves the grading accuracy. It has high reliability. However, there is a lack of sufficient internal indicators, and the dynamic acquisition of the spectrum may have an impact on the model. The application of hyperspectral imaging technology solves the above problems. Wang Jifang [50] used hyperspectral imaging and the BP neural network to detect three indicators of apple sugar content, pH value and hardness. The R values of the experiment are 0.85, 0.46, and 0.36, and the RMSE values are 0.69, 0.76, and 0.86, respectively, which has a strong predictive ability and is suitable for the grading of apples according to their sugar content. It should be noted that the accuracy of using single-channel spectra needs to be further optimized. For the application of multichannel spectroscopy, Huang Yuping et al. [51] used multi-channel hyperspectral imaging technology to detect and classify apples, and the use of spectral combination improved the recognition rate of the SR spectrum, reaching a grading accuracy of 0.994. However, the spectral combination cannot improve the results of the best single SR spectra in the visible region, which is a difficult point to be solved in the future. Çetin, N. et al. [62] predicted apple hardness and soluble solid content and pointed out that the potential use of ANN and DT methods for hyperspectral imaging was more effective for hardness, while DT and MLR were more effective for SSC. These methods are very feasible for industrial applications.
Figure 12. Six kinds of spectral preprocessing methods [49].
Figure 13. The diagram illustrates the structure of a robotic hand system.
Wang Yali et al. [52] designed a set of maize vitality detection and grading devices (Figure 14), and the partial least squares method can be used to better analyze the vitality. Experimental results show that SG convolutional smoothing shows superior performance, the calibration accuracy reaches 98.7%, the prediction set accuracy reaches 96%, and the overall vitality prediction accuracy reaches 97%. This experiment effectively solves the problems of low efficiency, low modeling accuracy and instability associated with single granulation. In the actual agricultural application, there are some problems such as the mismatch between the sliding speed and the identification rate, and the changeable environment. In research by Chia et al. [53], a low-cost artificial neural network based on k-fold cross validation was developed to classify the sugar content of white radish. Experimental results show that the accuracy of this method is 80.56%. Since the internal mass variation is small, pre-dispersive reflectance spectrum acquisition is used to improve the accuracy of the method. However, the grading accuracy of this method for white radish is relatively low, and it is more susceptible to other factors, such as illumination, in the actual grading process.
Figure 14. Schematic diagram of seed-vigor detection and grading equipment. 1. Computer. 2. Near-infrared spectrometer. 3. Optical fiber probe. 4. Light source. 5. Seed box (energetic). 6. Seed box (not energetic). 7. Sorting pipeline. 8. Running pipeline. 9. Single granulation device. 10. Fiber optic sensor.
In terms of hyperspectral imaging technology, Li Bin et al. [54] used random forest, least squares support vector machine, and other four methods to establish the spectral characteristics of loquat to classify loquat contusions (Figure 15). The hybrid image model has superior prediction performance, and the least squares support vector machine method can achieve 100% prediction accuracy. It provides a high precision scheme for the future damage identification. The fruit stem and pulp are relatively difficult to distinguish in grading. Lu Wei et al. [55] used hyperspectral ensemble learning (Figure 16) to grade anthocyanin content in black Lycium barbarum (Figure 17). Stacking ensemble learning is used to improve the grading accuracy from 0.9417 to 0.9833. The spectral extraction and grading of fruit stems and pulp are carried out, which verifies the feasibility of hyperspectral ensemble training for rapid grading of Lycium ruthenicum.
Figure 15. Schematic diagram of hyperspectral imaging system.
Figure 16. Hierarchical flow chart of hyperspectral ensemble learning of Lycium ruthenicum [55].
Figure 17. Hyperspectral imaging system. 1. CCD-camera. 2. Spectrograph. 3. Lens. 4. Lamps. 5. Black goji berry. 6. Translation platform. 7. Dark box. 8. Motor controller. 9. Computer.
The degree of granulation of fruit affects its taste. For fruits with thick skin, it can be challenging to predict internal fruit mass using imaging techniques [63]. Ref. [64] developed a method to estimate the ripening time of avocado using hyperspectral imaging combined with deep learning, with an average error of 1.17 days per fruit in the test set. When Liu Yande et al. [56] graded the granulation degree (Figure 18), the UVE-LS-SVM model based on RBF-Kernel was used for detection, and the misjudgment rate was only 0.78%. The established model effectively reduces the false positive rate and spectral space dimension, which provides a solid foundation for navel orange detection and grading. The model is only suitable for the granulation grading of thin-skinned fruits. When grading thick-skinned fruits, Sun Xiaopeng et al. [57] used the continuous projection-k-nearest neighbor algorithm to predict the accuracy, sensitivity and specificity of the model to reach more than 0.97, 0.9231 and 0.9784, respectively (Figure 19). It is suitable for grades 1–3, and the recognition accuracy of grades 4 and 5 is low. Miguel Vega-Castellote et al. [58] utilized near-infrared spectroscopy technology to detect the maturity and soluble solid content (SSC) of watermelon based on its penetration characteristics. In this experiment, NIR spectral data were obtained from two squares of one watermelon, while spectra were acquired from five different measurement points of 19 other watermelons. The results revealed that the penetration depth for both intact watermelon and pulp at four measurement points was 11 mm. Moreover, the optimal performance in terms of SSC percentage exhibited R2cv = 0.73, RMSECV = 0.39%, R2p = 0.81, and RMSEP = 0.30%. This study holds potential for watermelon grading; however, it is worth noting that the accuracy of the model heavily relies on optical geometry measurements, necessitating further optimization of the instrument. Sharma, S. et al. [65] used the GA-PLSR model and SPA-PLSR model to measure DM, TSS and FC of durian pulp, providing a quality inspection and grading system for durian packaging companies.
Figure 18. Equipment imaging using hyperspectral technology. (a) Schematic diagram, (b) picture [56].
Figure 19. Acquisition system of image information.
Lin Zhang et al. [59] used VISSA-GWO-SVM model to detect and classify saccharin jujube, and achieved a grading accuracy of 91.67%. Three different algorithms are used to tune the Support Vector Machine (SVM). The experimental method effectively overcomes the limitations of traditional manual detection methods, and has the advantages of being fast, accurate and lossless. It has made a contribution to food safety. Ci Jiangtao et al. [60] used hyperspectral technology [61], the CARS-MLR model, to detect and classify the external features of eggplant, and the accuracy of the prediction set reached 96.82%. The grading accuracy is improved, and the eggplant damage can be effectively graded and evaluated. However, the CARS-MLR model has strong dependence on data, and feature selection will affect its stability. The content of potato can be accurately determined using hyperspectral technology, as demonstrated by Jing Zhang et al. [66]. They developed the OSC-CARS–PLSR model based on partial least squares regression and orthogonal signal correction techniques. The obtained R2 values were 0.9606 and 0.8925, with corresponding RMSE values of 0.070% and 0.1385%. The OSC-CARS-PLSR model improves the prediction accuracy and stability, but it increases the computational complexity and needs to use more computing resources. Qi Hengnian et al. [67] found in the non-destructive determination of soluble-solid content of crown pear that the combination of Vis/NIR spectroscopy and the MLP-CNN-TCN method can quickly and non-destructively detect the SSC of crown pear, providing a new regression option for predicting the SSC of fruit.

5. Conclusions

In order to solve the problems existing in fruit and vegetable grading, the research status of fruit and vegetable grading was reviewed. At present, in the field of unmanned grading, the research on automatic grading systems and machines is not sufficient. However, current scoring techniques are rapidly evolving, providing a wide range of algorithms and techniques that can be used to enhance the capabilities of automated scoring systems and machines. This is the main contribution to fully automatic grading of fruits and vegetables.
This paper reviews the application of machine vision technology, near-infrared light technology and hyperspectral technology in fruit and vegetable grading in the past five years, and expounds the advantages and disadvantages of found in each research study. The results show that the accuracy of machine vision technology in fruit and vegetable size grading is 70% to 99.8%. The accuracy with respect to external defects is 88–95%. The accuracy of near-infrared light technology and hyperspectral technology in the detection and grading of internal content of fruits and vegetables is 80.56% to 100%, which is a high accuracy.
The machine vision technology, near-infrared light technology and hyperspectral technology are applied to the grading of fruits and vegetables, which realizes accurate and efficient sorting, and solves the problem of inconsistent quality. Further research on hyperspectral-based grading devices can facilitate online detection and grading of fruits and vegetables, compensating for the limitations of manual inspection and grading processes. Ultimately, this will lead to automated, unmanned, and mechanized operations. In the future, extensive efforts should be dedicated to studying non-destructive testing methods and theories for fruits and vegetables, as well as designing more advanced algorithms and software solutions. Integration of various beneficial advanced technologies is crucial in developing grading equipment suitable for diverse environmental conditions. This will ensure automation, safety, environmental friendliness, and advanced portability capabilities, among others advantages, while enhancing the quality, not only within fruit and vegetable grading, but also across other industries. Reliable support, along with technological advancements, are essential for fostering a healthy development within the fruit and vegetable industry chain.

Author Contributions

Conceptualization: W.M. and M.Z.; writing—original draft preparation: J.L. and M.Z.; article ideas: W.M. and Y.H.; methodology: J.L. and Y.Y. and Z.T.; article search: J.L. and M.Z.; article collation: J.C. and H.L.; visualization: J.L.; supervision: W.M. and Y.H.; project administration: W.M. All authors have read and agreed to the published version of the manuscript.

Funding

The work was funded by a number of projects, including the Chengdu Agricultural Science and Technology Center Project (NASC2022KR08) and (NASC2021KR07), and the Chengdu Agricultural College Youth Project (22ZR112).

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Tan, J.J.; Yang, S.D.; Zhou, L.Q.; Tan, H.W. High-quality citrus industry development strategies of Guangxi based on China-ASEAN fruit trade. Nanning J. South. Agric. 2020, 51, 470–476. [Google Scholar]
  2. Weng, M. The status quo and characteristics of China’s import and export of agricultural products. Int. Trade 2013, 6, 30–35. [Google Scholar]
  3. Tian, Y.; Yang, G.; Wang, Z.; Wang, H.; Li, E.; Liang, Z. Apple detection during different growth stages in orchards using the improved YOLO-V3 model. Comput. Electron. Agric. 2019, 157, 417–426. [Google Scholar] [CrossRef]
  4. Wu, Z.Y.; Li, Y.R.; Dong, Z.D. Research on the Development Status and Countermeasure of Chinese fruit market. Chin. Fruits Veg. 2023, 43, 79–83+87. [Google Scholar]
  5. Wang, H.; He, J.; Aziz, N.; Wang, Y. Spatial Distribution and Driving Forces of the Vegetable Industry in China. Land 2022, 11, 981. [Google Scholar] [CrossRef]
  6. Wu, L.; Zhou, Y.Q. Research on imperfect grain detection technology of wheat based on improved YOLOX. J. Henan Univ. Technol. Nat. Sci. Ed. 2023, 44, 109–117. [Google Scholar]
  7. Gill, H.S.; Murugesan, G.; Mehbodniya, A.; Sajja, G.S.; Gupta, G.; Bhatt, A. Fruit type grading using deep learning and feature fusion. Comput. Electron. Agric. 2023, 211, 107990. [Google Scholar] [CrossRef]
  8. Bai, Z.W.; Yan, F.W.; Yuan, P.H.; Zhang, H.M.; Yang, R.S. Design and experiment of fruit sorting robot based on embedded machine vision. J. Intell. Agric. Mech. 2023, 4, 61–70. [Google Scholar]
  9. Qin, Y.C.; Li, T.; Li, X.; Wang, M.L.; Tan, Z.Y. Design of apple damage automatic detection system based on machine vision. Food Mach. 2024, 40, 138–142. [Google Scholar]
  10. Wen, J.; He, J. Agricultural development driven by the digital economy: Improved EfficientNet vegetable quality grading. Front. Sustain. Food Syst. 2024, 8, 1310042. [Google Scholar] [CrossRef]
  11. Liu, C.H.; Li, X.; Zhu, Z.J.; Bao, Z.J.; Xie, X.N.; Yang, Z.Z.; Wang, Y.; Liu, F. Progress of Non-destructive Testing Technology in Mango Quality. Sci. Technol. Food Ind. 2021, 42, 413–422. [Google Scholar] [CrossRef]
  12. Varghese, R.R.; Jacob, P.M.; Sooraj, S.; Ranjan, D.M.; Varughese, J.C.; Raju, H. Detection and Grading of Multiple Fruits and Vegetables Using Machine Vision. In Proceedings of the 2021 8th International Conference on Smart Computing and Communications (ICSCC), Kochi, Kerala, India, 1–3 July 2021; pp. 85–89. [Google Scholar]
  13. Tang, Y.; Chen, M.; Wang, C.; Luo, L.; Li, J.; Lian, G.; Zou, X. Recognition and Localization Methods for Vision-Based Fruit Picking Robots: A Review. Front. Plant Sci. 2020, 11, 510. [Google Scholar] [CrossRef] [PubMed]
  14. Ismail, N.; Malik, O.A. Real-time visual inspection system for grading fruits using computer vision and deep learning techniques. Inf. Process. Agric. 2022, 9, 24–37. [Google Scholar] [CrossRef]
  15. Fan, S.X.; Wang, Q.Y.; Tian, X.; Yang, G.Y.; Xia, Y.; Li, J.B.; Huang, W.Q. Non-destructive evaluation of soluble solids content of apples using a developed portable Vis/NIR device. Biosyst. Eng. 2020, 193, 138–148. [Google Scholar] [CrossRef]
  16. Yuan, Y.H.; Yang, Z.R.; Liu, H.B.; Wang, H.B.; Li, J.H.; Zhao, L.L. Detection of early bruise in apple using near-infrared camera imaging technology combined with deep learning. Infrared Phys. Technol. 2022, 127, 104442. [Google Scholar] [CrossRef]
  17. Ünal, Z.; Kizildeniz, T.; Ozden, M.; Aktas, H.; Karagoz, O. Detection of bruises on red ap-ples using deep learning models. Sci. Hortic. 2024, 329, 113021. [Google Scholar] [CrossRef]
  18. Chen, B.Q.; Wu, Z.H.; Li, H.Y.; Wang, J. Research of machine vision technology in agricultural application: Today and the future. Sci. Technol. Rev. 2018, 36, 54–65. [Google Scholar]
  19. Zou, W. Research on citrus fruit size grading based on machine vision technology. Agric. Technol. 2023, 43, 48–51. [Google Scholar]
  20. Li, L.; Wen, T.; Dai, X.Y.; Wang, Z.Y. Online detection and grading system of citrus whole surface color. Food Mach. 2022, 38, 121–126. [Google Scholar]
  21. Shi, R.Y.; Tian, Y.W.; Lai, X.T.; Gu, W.J. Development of apple intelligent on-line inspection and grading system based on machine vision. China Agric. Sci. Technol. Rev. 2018, 20, 80–86. [Google Scholar]
  22. Peng, Y.K.; Sun, C.; Zhao, M. Dynamic non-destructive perception and grading robotic hand system for apple quality. Trans. CSAE 2022, 38, 293–303. [Google Scholar]
  23. Liu, J.H.; Gao, J.W.; Zhang, B.X.; Wang, J.C. Design of fruit grading system based on machine vision. Food Mach. 2023, 39, 112–118. [Google Scholar]
  24. Rao, J.; Lu, Z.Y. Research on external quality grading method of spherical fruit based on machine vision. Technol. Innov. 2022, 63–65. [Google Scholar]
  25. Zhang, Y.H.; Meng, Y.; Zhang, M.G.; Jiang, P.H.; Chen, D.J.; Guo, F.J. Fruit comprehensive quality detection based on Near infrared, machine vision and Information Fusion. Food Ind. 2018, 39, 247–250. [Google Scholar]
  26. Su, B.N.; Hua, X.Y. Research on strawberry image processing based on machine vision. J. Cap. Norm. Univ. (Nat. Sci. Ed.) 2018, 39, 42–45. [Google Scholar]
  27. Li, J.H.; Yi, K.C.; Liu, L.; Zhang, X.W.; Sun, Y.R. Research on blueberry fruit grading based on machine vision. Wuhan J. Jianghan Univ. 2021, 49, 64–69. [Google Scholar]
  28. Tang, Z.S.; Yuan, J.L.; Kang, L.H.; Cheng, L.X.; Lu, T.; Yang, C.; Zhang, F. Potato tuber skin roughness grading analysis based on image characteristics recognition. Sci. Agric. Sin. 2023, 56, 4428–4440. [Google Scholar]
  29. Zhang, Y.J.; Zhao, J.X. Research on daylily joint detection algorithm based on multiple neural networks. J. Chin. Agric. Mech. 2024, 45, 228–234. [Google Scholar]
  30. Wang, M.X.; Meng, Q.X.; Wang, M.J. Design of grading Control system of black Fungus Based on machine Vision. China Sci. Technol. Inf. 2023, 21, 91–94. [Google Scholar]
  31. Bhargava, A.; Bansal, A.; Goyal, V. Machine Learning-Based Detection and Sorting of Multiple Vegetables and Fruits. Food Anal. Methods 2022, 15, 228–242. [Google Scholar] [CrossRef]
  32. Chen, Y.X.; Yang, Y. Research on appearance quality of mango based on machine vision. Electron. Manuf. 2023, 31, 68–71. [Google Scholar]
  33. Wu, Y.L.; Chen, Y.Y.; Lian, X.Q.; Liao, Y.; Gao, C.; Guan, H.N.; Yu, C.C. Study on the Identification Method of Citrus Leaves Based on Hyperspectral Imaging Technique. Spectrosc. Spectr. Anal. 2021, 41, 3837–3843. [Google Scholar]
  34. Zhang, X.X.; Li, P.; Yu, M.; Jiang, L.W.; Liu, X.; Shan, Y. Progress in non-destructive citrus quality detection using near-infrared spectroscopy. Beijing Shipin Kexue/Food Sci. 2022, 43, 260–268. [Google Scholar]
  35. Zhang, Z.J.; Fu, X.Y.; Chen, K.M.; Zhao, Z.L.; Zhang, Z.X.; Zhao, J. Research on non-destructive detection method of moldy apple core by fusing density and spectral features. Beijing Food Ferment. Ind. 2022, 48, 281–287. [Google Scholar]
  36. Chandel, A.K.; Khot, L.R.; Sallato, B.C. Apple powdery mildew infestation detection and mapping using high-resolution visible and multispectral aerial imaging technique. Sci. Hortic. 2021, 287, 110228. [Google Scholar] [CrossRef]
  37. Xiao, J.-R.; Chung, P.-C.; Wu, H.-Y.; Phan, Q.-H.; Yeh, J.-L.A.; Hou, M.T.-K. Detection of Strawberry Diseases Using a Convolutional Neural Network. Plants 2021, 10, 31. [Google Scholar] [CrossRef]
  38. Liu, X.G.; Fan, C.; Li, J.N.; Gao, Y.L.; Zhang, Y.Y.; Yang, Q.L. Identification method of strawberry based on convolutional neural network. Trans. Chin. Soc. Agric. Mach. 2020, 51, 237–244. [Google Scholar]
  39. Wang, W.; Fang, Y.J.; Liu, J.Q.; Wang, Z.Q. Blueberries grading detection algorithm based on machine vision. Mod. Electron. Technol. 2018, 41, 38–42. [Google Scholar]
  40. Xue, Y.J. Research status and prospect of potato grading technology. Xinjiang Agric. Mech. 2024, 54–57. [Google Scholar]
  41. Bhargava, A.; Bansal, A. Automatic Detection and Grading of Multiple Fruits by Machine Learning. Food Anal. Methods 2020, 13, 751–761. [Google Scholar] [CrossRef]
  42. Zhou, H.; Zhuang, Z.; Liu, Y.; Liu, Y.; Zhang, X. Defect Classification of Green Plums Based on Deep Learning. Sensors 2020, 20, 6993. [Google Scholar] [CrossRef] [PubMed]
  43. Chen, Y.H.; An, X.S.; Gao, S.M.; Li, S.J.; Kang, H.W. A Deep Learning-Based Vision System Combining Detection and Tracking for Fast On-Line Citrus Sorting. Front. Plant Sci. 2021, 12, 622062. [Google Scholar] [CrossRef]
  44. Zhang, X.; Xun, Y.; Chen, Y. Automated identification of citrus diseases in orchards using deep learning. Biosyst. Eng. 2022, 223, 249–258. [Google Scholar] [CrossRef]
  45. Hu, W.X.; Xiong, J.T.; Liang, J.H.; Xie, Z.M.; Liu, Z.Y.; Huang, Q.Y.; Yang, Z.A. A method of citrus epidermis defects detection based on an improved YOLOv5. Biosyst. Eng. 2023, 227, 19–35. [Google Scholar] [CrossRef]
  46. Manley, M. Near-infrared spectroscopy and hyperspectral imaging: Non-destructive analysis of biological materials. Chem. Soc. Rev. 2014, 43, 8200–8214. [Google Scholar] [CrossRef]
  47. Ma, J.J.; Wang, K.Q. Research Progress of Optical Nondestructive Testing Technology for Fruit Quality. Sci. Technol. Food Ind. 2021, 42, 427–437. [Google Scholar]
  48. Ding, C.; Chen, J.Y.; Zheng, M.M.; Zhang, L.; Wei, W.; Zhang, Y.N. Research progress of 529 hyperspectral image change detection technology. Chin. J. Image Graph. 2024, 530, 1714–1729. [Google Scholar] [CrossRef]
  49. Liao, Z.Q.; He, C.X. Research on prediction and grading of sugar content of apple based on Near-infrared spectroscopy. Inf. Technol. Informatiz. 2023, 93–98. [Google Scholar]
  50. Wang, J.F.; Du, C.L.; Zou, X.Q.; Lin, C.Y.; Hao, Y.B. Research on non-destructive testing technology of apple based on hyperspectral and BP neural network. J. Jiamusi Univ. (Nat. Sci. Ed.) 2023, 41, 98–101+105. [Google Scholar]
  51. Huang, Y.P.; Yang, Y.T.; Sun, Y.; Zhou, H.Y.; Chen, K.J. Identification of Apple Varieties Using a Multichannel Hyperspectral Imaging System. Sensors 2020, 20, 5120. [Google Scholar] [CrossRef]
  52. Wang, Y.L.; Peng, Y.K.; Zhao, X.L.; Shen, L.Y. Design and experiment of Non-destructive testing and grading device for corn seed vigor. Trans. Chin. Soc. Agric. Mach. 2020, 51, 350–356. [Google Scholar]
  53. Chia, K.S.; Jam, M.N.H.; Gan, Z.; Ismail, N. Pre-dispersive near-infrared light sensing in non-destructively classifying the brix of intact pineapples. J. Food Sci. Technol. 2020, 57, 4533–4540. [Google Scholar] [CrossRef]
  54. Li, B.; Han, Z.Y.; Wang, Q.; Sun, Z.X.; Liu, Y.D. Research on scratch grade detection of loquat based on hyperspectral imaging technology. Spectrosc. Spectr. Anal. 2023, 43, 1792–1799. [Google Scholar]
  55. Lu, W.; Cai, M.M.; Zhang, Q.; Li, S. Hyperspectral and ensemble learning for fast grading of Lycium. Spectrosc. Spectr. Anal. 2021, 41, 2196–2204. [Google Scholar]
  56. Liu, Y.D.; Li, M.P.; Hu, J.; Xu, Z.; Cui, H.Z. Study on detection of navel orange granulation by near infrared hyperspectral spectroscopy. Spectrosc. Spectr. Anal. 2022, 42, 1366–1371. [Google Scholar]
  57. Sun, X.P.; Liu, C.C.; Lu, H.Z.; Xu, S. Detection of honey pomelo in different granulation levels based on near-infrared transmittance spectroscopy combined with machine vision. J. Food Sci. Technol. 2021, 39, 37–45. [Google Scholar]
  58. Miguel, V.-C.; María-Teresa, S.; Petter, W.J.; Kristian, A.N.; Dolores, P.-M. Near infrared light penetration in watermelon related to internal quality evaluation. Postharvest Biol. Technol. 2023, 204, 112477. [Google Scholar]
  59. Zhang, L.; Sun, J.; Zhou, X.; Nirere, A.; Wu, X.; Dai, R. Grading detection of saccharin jujube based on hyperspectral imaging technology. J. Food Process. Preserv. 2020, 44, e14591. [Google Scholar] [CrossRef]
  60. Chi, J.T.; Zhang, S.J.; Ren, R.; Lian, M.R.; Sun, S.S.; Mu, B.Y. Detection of eggplant external defects using hyperspectral technology. Mod. Food Sci. Technol. 2021, 37, 279–284+178. [Google Scholar]
  61. Zhang, J.W.; Zhi, J.H.; Guan, X.M.; Zhang, S.; Su, T.; Lin, S.Y.; Yu, P.L.; Li, M.B. Identification Method of Understory Crops Quality Grade Using Hyperspectral Image Technology: A Case of Astragalus membranaceus. J. Northeast. For. Univ. 2024, 52, 79–84. [Google Scholar]
  62. Çetin, N.; Karaman, K.; Kavuncuoglu, E.; Yildirim, B.; Jahanbakhshi, A. Using hyperspectral imaging technology and machine learning algorithms for assessing internal quality parameters of apple fruits. Chemom. Intell. Lab. Syst. 2022, 230, 104650. [Google Scholar] [CrossRef]
  63. Magwaza, L.S.; Tesfay, S.Z. A review of destructive and non-destructive methods for determining avocado fruit maturity. Food Bioproc. Technol. 2015, 8, 1995–2011. [Google Scholar] [CrossRef]
  64. Davur, Y.J.; Kämper, W.; Khoshelham, K.; Trueman, S.J.; Bai, S.H. Estimating the Ripeness of Hass Avocado Fruit Using Deep Learning with Hyperspectral Imaging. Horticulturae 2023, 9, 599. [Google Scholar] [CrossRef]
  65. Sharma, S.; Sirisomboon, P.; Sumesh, K.C.; Terdwongworakul, A.; Phetpan, K.; Kshetri, T.B.; Sangwanangkul, P. Near-infrared hyperspectral imaging combined with machine learning for physicochemical-based quality evaluation of durian pulp. Postharvest Biol. Technol. 2023, 200, 112334. [Google Scholar] [CrossRef]
  66. Zhang, J.; Guo, Z.; Ren, Z.; Wang, S.; Yin, X.; Zhang, D.; Wang, C.; Zheng, H.; Du, J.; Ma, C. A non-destructive determination of protein content in potato flour noodles using near-infrared hyperspectral imaging technology. Infrared Phys. Technol. 2023, 130, 104595. [Google Scholar] [CrossRef]
  67. Qi, H.N.; Shen, C.; Chen, G.; Zhang, J.Y.; Chen, F.N.; Li, H.Y.; Zhang, C. Rapid and non-destructive determination of soluble solid content of crown pear by visible/near-infrared spectroscopy with deep learning regression. J. Food Compos. Anal. 2023, 123, 105585. [Google Scholar] [CrossRef]
  68. Zhang, B.; Gu, B.; Tian, G.; Zhou, J.; Huang, J.; Xiong, Y. Challenges and solutions of optical-based nondestructive quality inspection for robotic Fruits and vegetables grading systems: A technical review. Trends Food Sci. Technol. 2018, 81, 213–231. [Google Scholar] [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Journal Statistics
Multiple requests from the same IP address are counted as one view.
Download PDF
Crucial Factors Influencing the Efficiency of Androgenesis in Oat (Avena sativa L.) Through Anther and Microspore Cultures
Previous
Effect of Biostimulants and Glyphosate on Morphophysiological Parameters of Zea mays (L.) Seedlings under Controlled Conditions
Next