Next Article in Journal
Prediction of Landslide Deformation Region Based on the Improved S-Growth Curve Model
Previous Article in Journal
Real-Time Behaviour of Dredged Slurry Treated by Air-Booster Vacuum Consolidation
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Sorting of Fresh Tea Leaf Using Deep Learning and Air Blowing

1
School of Tea and Food Science & Technology, Anhui Agricultural University, Hefei 230036, China
2
State Key Laboratory of Tea Plant Biology and Utilization, Hefei 230036, China
3
School of Engineering, Anhui Agricultural University, Hefei 230036, China
*
Author to whom correspondence should be addressed.
Appl. Sci. 2023, 13(6), 3551; https://doi.org/10.3390/app13063551
Submission received: 10 January 2023 / Revised: 8 March 2023 / Accepted: 8 March 2023 / Published: 10 March 2023

Abstract

:
The sorting of machine-picked fresh tea leaves after mechanized harvesting remains a challenge because of the complex morphological characteristics and physicochemical properties of fresh tea leaves. First, the recognition results of four types of models, namely, YOLOv5, YOLOv3, Fast RCNN, and SSD, were compared. It was found that YOLOv5, with guaranteed recognition accuracy, had a recognition speed of 4.7 ms/frame (about four times that of the second ranked YOLOv3). Therefore, this study presents a novel fresh tea leaf sorting system that provides rapid and high-precision multi-channel sorting for four grades of tea leaves using a tea leaf recognition model based on the You Only Look Once (YOLOv5) deep learning model. Subsequently, a raw dataset, consisting of 6400 target images of different grades and different moisture contents, was used to evaluate three different optimization methods. Among these, the Stochastic Gradient Descent (SGD) optimization method was found to provide the best model training results with an average recognition accuracy of 98.2%. In addition, the recognition efficacy of the recognition model was found to be positively correlated with the gradient coverage of tea’s moisture content in the training set. Theoretical analysis was then conducted, along with the experimental investigation of the air-blowing force on the fresh tea leaves in the sorting process, with 30° determined to be the optimal air-blowing angle. Finally, the overall results showed that the construction of the full moisture content training set enabled a model recognition accuracy of up to 88.8%, a recall of 88.4%, a recognition speed of 4.7 ms/frame, and an overall sorting accuracy of 85.4%. This result is promising for multi-channel sorting of fresh tea leaf grades in complex situations, and as such provides a strong basis for the application of tea leaf sorting equipment.

1. Introduction

Tea is an important economic crop in China, where the tea cultivation area ranks first in the world and the total value of production is more than CNY 600 billion (USD 86.6 billion). To improve the utilization of tea resources, the machine harvesting of fresh tea leaves has begun in several important tea growing areas. The fresh leaves of machine-picked tea have different specifications, including single leaf (SL), one bud and one leaf (OBOL), one bud and two leaves (OBTL), and one bud and three leaves (OBRL), as well as low uniformity [1]. These four types of tea fresh leaves not only differ in the number of leaves they contain in appearance, but they also differ in the content of internal substances such as L-theanine and catechins. Furthermore, the production of Chinese famous tea has very high requirements for fresh leaf raw materials, for example, Liu’an Gua Pian made only from SL as raw material.
Therefore, it is impossible to use machine harvesting in the processing of famous tea products, resulting in a significant decline in economic benefits. There are two main ways to solve the above problem: One is to develop an intelligent harvesting machine for famous tea [2] to realize the intelligent identification of the fresh tea leaf grade of the tea. The other, proposed in this study, is applying the idea of picking and fusion to develop an intelligent sorting system for fresh tea leaf sorting according to the requirements of different types of famous tea. The fresh tea leaves are categorized into SL, OBOL, OBTL, and OBRL, as shown in Figure 1, and correspond to the grading system used in the processing of famous tea.
Owing to the complexity and variability of the physical properties of fresh tea leaves, especially the fact that the fresh leaves will continue to lose moisture for some time after harvesting, which leads to significant changes in the physical surface properties of the fresh leaves after harvesting up until the sorting process [3], it is challenging to develop a fast and highly accurate identification system suitable for the fresh tea leaf grades. Computer vision is nondestructive and highly automated, and it has been used in strawberry grading systems [4], fermentation degree monitoring of black tea [5,6], apple sorting systems [7], and in other fields. In recent years, conventional machine learning has been applied to the development of fresh tea leaf grade recognition systems and has achieved some success [8,9]. Gan et al. investigated feature classification methods using an improved genetic algorithm to screen features with the best combination of three different classifiers with the best number of dimensions, and finally a support vector machine (SVM) achieved a recognition accuracy of 97% in the 28-dimensional feature set [10]. Chen et al. extracted the features of tea leaves by also determining the topology of the tea leaf and used SVM to identify the fresh tea leaves based on the sorting; consequently, the recognition accuracy reached 94% [11]. However, it is difficult for machine learning to obtain effective feature values for some complex problems, and there are differences in recognition results when different sorting scenarios are present. Recently, deep learning algorithms have been proven effective in tea pest detection [12,13,14], famous tea picker recognition [15], and fresh tea leaf sorting [16]. They have also been successfully applied in product sorting research [17] for date fruit classification [18], grading and sorting of garlics [19], and seed sorting [20,21]. However, given the large number of materials in the fresh tea leaf grading link and the problems of fresh leaf collusion and entanglement, its detection speed and accuracy do not achieve the desired effect. The proposed You Only Look Once (YOLO) model provides unlimited opportunities to solve this problem and has been applied to the development of various agricultural grading systems, such as lotus seed sorting [22], fruit classification [23], and pear fruit sorting [24]. For example, Gui et al. used YOLOv5-based lightweight tea bud detection, the average precision of which was 9.66% higher than YOLOv5 [25]. Bao et al. utilized a combination of the multiscale RFB module and YOLOv5-based dual-dimensional mixed attention (DDMA) to reduce missed detection and false detection caused by dense blade distribution, thus resulting in a 6.5% recall increase [14].
In addition, it is not easy to develop high-precision, fast, and intelligent sorting equipment suitable for the actual production of fresh tea leaf grades. Currently, most common tea sorting devices use mechanized classification, mainly tumbling sieve sorting, round sieve sorting, and wind sorting [26,27,28]. These methods can categorize the machine-picked fresh leaf raw materials by size to a certain extent and effectively separate normal buds and leaves from broken buds or coarse stems and other things, but there is still a certain discrepancy between the grading effect and the requirements of famous fresh tea leaves. Zhen-Yu Gao et al. developed a fresh tea leaf intelligent sorting system and built a convolutional neural network recognition model based on a seven-layer structure to recognize and sort fresh tea leaves automatically with a recognition rate of 90% [29]. Chen et al. developed an automatic sorting machine based on the recognition method of vision and applied SVM to the recognition and sorting of four grades of fresh tea leaves using nozzles, with a success rate of 80% and an efficiency of 15 kg/h [11]. Although the aforementioned studies have achieved some success in the intelligent classification of fresh tea leaves, there is still room for improvement in terms of sorting accuracy and speed, especially given the large number of fresh tea leaf sorting scenarios.
To perform the sorting of tea leaves quickly and with high precision, a database of fresh tea leaf samples with different staking states was first created in this study. The YOLOv5 model was trained using databases at different spreading degrees, and the optimal training database composition was determined (contains as many fresh leaves as possible at different spreading stages), resulting in a fresh tea leaf grade recognition system with high generalization ability for the variation in physical features on the fresh tea leaf surface. We equipped the identification system on a three-stage conveyor belt with a multi-channel air-blowing device with high-speed execution capability and determined the optimal air-blowing angle and material collection position of the fresh tea leaf sorting equipment by positioning and analyzing the whole process from identification to the sorting link. Finally, a fresh tea leaf intelligent sorting system with a fast sorting speed and high success rate was developed for sorting the four grades—SL, OBOL, OBTL, and OBRL.
The following points summarize the contributions of this study:
(1)
A deep learning recognition model with high generalization ability for fresh tea leaf samples is developed; it provides a better solution to the degradation of recognition accuracy caused by the presence of mixed fresh tea leaf grades; a mixture of old, mature, and tender leaves; and collusion and entanglement among individual fresh leaves during sorting.
(2)
The potential relationship between the recognition accuracy of recognition models and water content of fresh tea leaves is investigated.
(3)
Based on the theory and experiment, the whole process of positioning fresh tea leaves, from identification to sorting, is achieved.
The deep learning recognition model is mounted on a sorting platform with a high-speed air-blowing device to achieve high-precision and fast sorting of fresh tea leaf grades.

2. Materials and Methods

The development process of the fresh tea leaf sorting equipment is based on a combination of theoretical calculations, simulations, and design trials.

2.1. Program Design

Figure 2 shows the structure of the fresh tea leaf sorting platform, which includes three levels. The first level is sorting platform SL, the second level is sorting platform OBOL, the third level is sorting platform OBRL, and OBTL is the last platform level for the free fall of material, which finally realizes the four levels of fresh tea leaf sorting. The structure of each level of the sorting platform mainly includes a vibrating screen feeding system, a transfer device, an image acquisition system, a central control system, and a high-speed air-blowing sorting execution system.
The vibrating screen feeding system makes the fresh tea leaves fall evenly on the conveyor belt by adjusting the vibration frequency of the V-shaped vibrating screen (50 Hz), and the conveyor belt serves as a transfer device to transfer the fresh tea leaves to the image acquisition system. The image acquisition system consists of a Basler industrial camera (Basler Inc., Ahrensburg, Germany), two LED strips (14.2 mA, 12.2 W, 20 Klux), and a light shield. The industrial camera is attached to the top light shield to capture continuous images of the material and transmit them to the PC via a USB data cable. The PC acts as a central control system to process the image patterns transmitted from the camera and sends the processing results and position information to the high-speed air-blow sorting execution system through the serial port. The high-speed air-blow sorting actuator system includes a 32-bit STM32F407VET6 microcontroller based on the ARM® Cortex® core, an air pump, and a high-speed solenoid valve.
To achieve fast and high-precision sorting, the material is first shaken by the V-shaped vibrating screen and falls evenly onto the conveyor belt, which transports the material to the image acquisition system. In the image acquisition system, the Balser industrial camera captures the images and transmits them to the computer, which imports the images into the trained fresh tea leaf grade recognition model for recognition processing and outputs the label results with coordinate positioning. If the label result corresponds to the target material label of the sorting platform at this level, the PC sends the extracted fresh tea leaf coordinates to the STM32, which executes the air blow command to blow the target material into the collection frame after receiving the command. If the label result does not correspond to the target material label of the sorting platform at this level, the air blow command will not be executed, and the material will automatically enter the conveyor belt of the next level to wait for sorting again, as shown in Figure 3.

2.2. Image Acquisition and Dataset Creation

In this study, fresh tea leaves, picked from the tea tree variety C. sinensis cv. Nongkangzao were used as the test material. The grades of fresh tea leaves were classified into four categories, namely, SL, OBOL, OBTL, and OBRL, and the corresponding labels were 0, 1, 2, and 3, as shown in Figure 1. In this study, to solve the problem of significant differences in the surface characteristics of fresh tea leaves with different moisture content values and according to the degree of spreading, the tea fresh leaves were categorized into four groups (a, b, c, and d). For each group were collected 400 images each of SL, OBOL, OBTL, and OBRL.
Then the actual sorting scenario was simulated to ensure that the fresh tea leaves were vibrated down from the vibrating screen onto the conveyor belt. Then, they were passed through the image acquisition area at a speed of 4 cm/s, where the camera captured the different spreading patterns. The fresh leaf samples were captured at a rate of 0.8 s per frame. A total of 6400 raw images in jpg format with a pixel resolution of 1430 × 1030 was obtained using the method described above (see Table 1).
In this study, the Labellmg annotation tool was used to annotate the original images. To increase the size of the dataset and the diversity of the samples and mitigate overfitting during the model training, the dataset was expanded five-fold by mapping, brightening, changing contrast Normalization, and sharpening of the samples to randomly divide the dataset into a training set and a test set at a ratio of 8:1:1 for model training, optimization, and testing, respectively.

2.3. Model Selection

Advanced deep learning models are increasingly being proposed with the advancement in technology. Popular deep learning methods include YOLO, Fast RCNN, SSD, and Alex net; each has its advantages. Therefore, it is extremely important to select a model suitable for a particular dataset for the development of recognition systems.
In this study, we used all 6400 datasets to train and compare models (YOLOv5, YOLOv3, Fast RCNN, SSD). The comparison results are shown in Table 2.
From the table, it can be observed that the recognition accuracy and speed of the Fast RCNN and SSD models are lower compared to the YOLO series models. It can be established that although the recognition accuracy of the YOLOv5s recognition model is about 0.8% lower than that of YOLOv3, its image recognition speed is about 4 times higher. In addition to this, YOLOv5 is far ahead of the other three models in terms of weights size. Therefore, considering all aspects, we chose the YOLOv5s model to apply the development of the tea fresh leaf recognition system.

2.4. Model Construction

YOLOv5 is one of the most advanced models in the field of deep learning. It features high detection accuracy and fast detection speed, and it meets the requirements of real-time monitoring. YOLOv5 is divided into four versions: s, m, l, and x. Considering the computer performance, training time, and other factors, the recognition model YOLOv5s was built in this experiment.
The YOLOv5s network model architecture built in this study is summarized in Figure 4. It mainly consists of a Backbone network, Neck network, and Prediction network. Image samples are input from the input side, feature extraction is performed in the Backbone network, and then features are combined in the Neck layer and passed to the Prediction layer for the final prediction output.

2.5. Identification of Indicators to Evaluate Model Test Results

The accuracy (Precision, P), recall (Recall, R), mAP_0.5:0.95 (mean Average Precision), and F1 Score were used as evaluation metrics to objectively evaluate the criteria for evaluating the overall recognition performance of the trained recognition models, and the calculation formulas are as follows:
Pr ecision = TP TP + FP ,
Recall = TP TP + FN ,
mAP = 1 n i = 1 n A P i ,
F 1 = 2 1 Precision + 1 Recall ,
where TP is the number of correctly matched prediction frames with labeled frames; FP is the number of incorrectly predicted prediction frames; and FN is the number of mislabeled frames.

2.6. Localization of Fresh Tea Leaves and Tracking Modeling

To achieve continuous sorting of fresh tea leaves and improve sorting efficiency, it is necessary to determine the exact position of fresh tea leaves at different links and moments. Since the quality distribution and shape of tea leaves are irregular, and there are differences between different grades and even between different leaves of the same grade, the uncertainties in leaving the conveyor belt and making a throwing motion to reach the actual sorting point have a certain influence on the actual air-blowing effect. In the process of air blowing, the closer the air-blowing point is to the center of the fresh tea leaves, the better the sorting effect. If the air-blowing point is at the edge of the leaf, it is easy for the trajectory to shift or the air-blowing process to fail and so forth. Therefore, in this study, the locations of the morphological centroids of the fresh tea leaves (as shown in Figure 5) were considered as the coordinates of the mass of the fresh tea leaves by determining them.
The target fresh leaves move in the direction of the conveyor belt with speed V. The image captured by the camera on the conveyor belt has a size of 640 × 640 pixels, which is used as the reference plane for defining the coordinate system for real-time positioning of fresh tea leaves. As shown in Figure 6, point A is the image acquisition point of the fresh tea leaves. The camera captures the target material, and the image is transmitted to the PC. The PC identifies the material based on the smallest rectangular box and determines the coordinates of the four corners of the rectangular box ( x 1 , y 1 ), ( x 2 , y 2 ), ( x 3 , y 3 ), and ( x 4 , y 4 ), and finally obtains the morphology of the fresh tea leaves by calculating the coordinates of the center point A. Point O is the middle point of the coordinate system, whose coordinates are known to be (320, 320). The distance from point O to the end of the conveyor belt at point C is Q . The conveyor belt runs at a speed V 0 Point B is the location of the target after the fresh tea leaves are collected by the camera and processed at t 1 time, and the coordinates of the corresponding fresh tea leaves are ( x b , y b ); then the fresh tea leaves are transported to the end of the conveyor belt at point C, and the transport time is set as t 2 ; then the fresh tea leaves make a flat throwing motion to reach the actual sorting point D, and the time of this process is set as t 3 .
The fresh tea fresh leaf picture after the processor calculation process shows tea leaf coordinate positions when moving from point A to point B. Let the PC send to the air-blow sorting execution system through the serial port after the waiting time T , for the execution of the air blow command. Then,
T = t 2 + t 3 .
Here,
t 2 = Q - ( x b - 320 ) V 0 ,
t 3 = 2 h g ,
and   h = H · cos α · sin α
yield
T = 2 · H · cos α · sin α g + Q - ( x b - 320 ) V 0 ,
where H is the height of the blowing nozzle from the conveyor belt, and h is the height of the actual air-blowing point from the conveyor belt.
In this process, the fresh tea leaves make a parabolic movement after they leave the conveyor belt. In this process, the fresh tea leaves rotate by themselves, which leads to the uncertainty of the contact area between the airflow from the air-blowing mouth and the fresh tea leaves, which has a certain influence on the sorting success rate. Therefore, in this study, the force area of the fresh tea leaves was analyzed when the airflow was blown to the morphological center of the tea leaves at each angle, using the plane of air-blowing nozzle as the first viewpoint, and the optimal air-blowing angle was determined based on the trajectory analysis.
As shown in Figure 7, fresh tea leaves undergo rotational motion due to body movement. Set and airflow are directly opposite the deflection angle β; α is the angle between the gas blowing mouth and the ground; the gas blowing mouth has a diameter of 2R; the gas blowing contact surface long axis is a ; the short axis is b ; and the area is S . The height of the air-blowing sorting point from the ground is L , and the height of the material collecting port is I . Fresh tea leaves continue to undergo parabolic motion due to the airflow generated by the force F ; thus,
a = 2 R cos α ,   ( 0 ° α < 90 ° )
b = 2 R sin β , ( 2 R < b < 6 R )
S = π · R 2 cos α · sin β ,
F = P · S .
From the momentum theorem, it follows that:
V = F · Δ t m
where P = 1 MPa, Δ t is the action time of the force F on the fresh tea leaves, which is set as 0.01 s, and m is the mass of fresh tea leaves, which is set to 0.1 g.
Fresh tea leaves perform a parabolic motion after being subjected to air-blowing force, and it can be seen from the above figure that the fresh tea leaves have different trajectories when the angle of the air-blowing mouth is different. In this section, to develop the optimal device for collecting fresh tea leaves considering the relevant factors, the motion trajectory under different air-blowing angles is analyzed, and the time t of the air-blowing force to the movement to the highest point can be expressed as follows:
t = P · π · R 2 · Δ t · sin α m · g · cos 2 α
The displacement X of the motion in the vertical direction can be expressed as follows:
X = P 2 · π 2 · R 4 · Δ t · sin α m 2 · g · cos 3 α .
The displacement Y of the motion in the vertical direction can be expressed as follows:
Y = P 2 · π 2 · R 4 · Δ t 2 · sin 2 α 2 · m 2 · g · cos 4 α 1 2 · g · t 2 .
Finally, we have
Y = 1 2 · X · tan α g · X 2 · sin 2 β · m 2 2 · P 2 · π 2 · R 4 · Δ t ,
where t is the movement time of the fresh tea leaves in the air after receiving the air-blowing force.
According to the analysis, under the strong air-blowing action at an air-blowing pressure of 1 MPa, the influence of different air-blowing action areas caused by the self-rotation of fresh tea leaves in free fall can be ignored. Once the correlation between the air-blowing trajectory and the air-blowing angle is known, theoretical analysis of the air-blowing angle and the size and position of the material collection port can be performed. In this study, the material collection port is first set at a position with a horizontal distance of 0.8 m from the air-blowing port. When the air-blowing port is horizontal, that is, α is equal to 0°, the material dropping point at X = 0.8 m is below the air-blowing nozzle. However, considering that the trajectory deviation is caused by the position of the blade edge blown by the air flow in the actual process, it is necessary to extend the height of the collection port downward to increase the collection range. However, in the structure of the machine, the downward extension of the collection port easily coincides with the V-shaped vibrating screen, which leads to the failure of material collection. When the angle between the air-blowing angle and the horizontal plane is 30°, the ideal height of the material collection point is 0.46 m above the air-blowing port, and the downward extension range of the collection port height is large at this height, which theoretically completely improves the collection. When the air-blowing angle continues to increase, both the collection height and the movement time of fresh tea leaves in the air increase, in addition to the uncertainty factors, which affect the success rate of sorting. Therefore, the air-blowing angle was set at 30° in this test.

3. Results and Discussion

3.1. Model Optimization

In this study, preliminary experiments were conducted on different types of fresh tea leaf samples before starting the formal training of the model.
In the training process of the YOLOv5 model, the spreading of parameters such as the number of iteration steps and batch size also had a great influence on the recognition accuracy of the model. Since the processing of small batches can lead to poor batchnorm statistics, this study set the number of iteration steps to 300 and the batch size to the maximum that the computer memory could handle, which was 48.
Gradient descent is the most commonly used classification model optimization algorithm, and the YOLOv5 model provides three different forms, namely, SGD (Stochastic Gradient Descent), Adam (Adaptive Moment Estimation), and AdamW (Adam Weight Decay). The SGD optimization algorithm has a faster training speed but is not always able to reach the optimal point; the Adam optimization algorithm can solve the problem of excessive oscillation and accelerate the convergence of the function; and AdamW, i.e., Adam + weight decay, has the same effect as Adam + L2 regularization. However, in AdamW, the gradient of the regular term is directly added to the backpropagation formula, eliminating the need to manually insert the regular term into the loss, which makes the calculation more efficient. The results are shown in Figure 8. It can be seen that SGD decreased faster in terms of training loss value and required fewer iterative steps for model convergence than did AdamW and Adam. In terms of validation accuracy, the average training accuracy of the four sample models trained with SGD was 98.2%, which is 4.2% better than the average training accuracy of the four second-ranked sample models trained with the AdamW optimizer. In summary, this study tested SGD as an algorithm for optimizing classification models.

3.2. Fresh Tea Leaf Identification Classification Results and Analysis

To investigate the influence of the change in surface characteristics of fresh tea leaves on the recognition model accuracy at different time points, from harvesting to the sorting process proposed in the previous study, and to provide guidance for subsequent research and staking time before the tea sorting process in actual production, the optimal model parameters investigated in Section 3.1 were used. Several water content databases such as the four water content gradient databases abcd and double water content gradient databases ab, ac, ad, bc, bd, and cd were constructed. It was ensured that the same number of image samples were used in each group of databases through the independent and combined use of four types of amortized green samples: a, b, c, and d. Then, each group of sample data was separately used to train the model, and the results are shown in Table 3.
As can be seen from the data in Table 2, the four evaluation indexes in the model training sample all showed a significant positive correlation between the coverage rate of water content and the model testing effect, with the highest accuracy of 81.6% for the predictive model trained with water content alone. This was 2.4% lower than the average accuracy of 84.0% for the six prediction models in the double water content gradient group, while the prediction accuracy of the prediction model with a four water content gradient reached 88.8%, which is 7.2% higher than the highest accuracy of the single moisture content gradient prediction model. Overall, the recognition accuracy of 88.8% is relatively high for the rapid recognition of multiple fresh leaves in complex situations, but there is still room for improvement.
First, it can also be concluded from the confusion matrix be shown in Figure 9. that the recognition effect was positively correlated with the gradient coverage of the moisture content of the fresh tea leaves in the training set. Secondly, by comparison, it was found that the main misclassification type of each prediction model was the confusion of OBTL with OBRL, and the misidentification rate of OBTL as OBRL was higher, followed by the confusion degree of OBTL with OBOL. The main reason for the confusion is that the identification of tea leaf grades is mainly determined by the presence of tea buds and the number of leaves, but in different perspectives, especially for grades with more leaves such as OBTL and OBRL, the obscuring of tea buds by leaves is inevitable. Therefore, in our future work, we will investigate whether a mechanical structure can be designed or a method can be used to solve this problem.
The recognition model proposed in this study is based on the recognition of the fresh tea leaf grades in the actual sorting scenario, and the example recognition results are shown in Figure 10. As can be seen from the figure, the proposed recognition model could complete the rapid recognition of multiple quantities of tea leaves with high accuracy and could also complete the recognition of multiple fresh tea leaves that were entangled and colluded quite well.

3.3. Air-Blowing Mouth Angle Experiment

As described in Section 2.5, α was set to 30°. In this section, the team responsible for the above theoretical basis for the validation test design conducted a test with 0° and 30° air-blowing angles of tea fresh leaf samples, which was repeated 100 times with air blowing, each time the tea fresh leaves in the air-blowing point were recorded 0.8 m away from the drop location. From Figure 11 it can be seen that some errors will be caused by the air resistance and the differences in the quality and shape of the fresh leaves; however, under the action of strong air-blowing pressure, the maximum blanking height of each fresh tea leaf at X = 0.8 m was essentially consistent with the theoretical height determined in Section 2.5. Therefore, when the material collecting port is manufactured, the collecting port only needs to be extended downwards by 0.4 m from the theoretical maximum (i.e., I = 0.4), and a collection rate of about 97% can be achieved.

3.4. Fresh Tea Leaf Sorting Results and Analysis

The sorting equipment of fresh tea leaf we developed is shown in Figure 12. The whole machine sorting experiment was conducted in three groups of fresh tea leaf grade sorting, and each group of test samples was mixed without spreading, with 2 h spreading, 4 h spreading, and 6 h spreading, each spreading degree contains 40 fresh leaves, and each 40 fresh leaves contained four grades of fresh leaf samples of 10 each; the experimental results are shown in Table 4.
The machine sorting success rates in the three parallel sets of experiments were 86.9%, 83.1% and 86.3%, respectively, with an average accuracy of 85.4%. OBTL had the highest sorting success rate in all three groups. We believe this is related to the fact that leaves do not go through air blowing but direct drop collection, reducing interference factors. In addition, the sorting equipment can sort about 200 fresh leaf samples per minute with an efficiency of about 3 kg/h. According to the conversion standard of Chen et al. [11], the efficiency can even reach 30 kg/h.
We believe that there are three main reasons for the failure of sorting:
  • Collusion and entanglement of fresh leaves cannot be avoided, and such a situation will affect the identification accuracy rate. Secondly, even if there is successful identification, in the air-blowing link there will be a blowing away of more than one fresh tea leaf. This is the main reason for the low sorting success rate.
  • In the sorting process, part of the material mass is located in the middle of the two air-blowing nozzles to ensure that there are no sorting actuators to perform the sorting.
  • In (1), we analyzed that the reason for the low success rate of single-leaf sorting is that the number of fresh tea leaves on the primary conveyor belt is large and the problem of collusion and entanglement of fresh leaves is relatively pronounced, but this is obviously contradictory to the result of the low success rate of one-bud-three-leaf sorting on the tertiary conveyor belt. Observing the experimental process, we found that the fresh leaves of OBRL have a larger shape and heavier quality, and the air-blowing effect is not ideal for the existing air-blowing pressure and air flow contact area. Therefore, we propose to increase the air-blowing pressure and the size of the air-blowing nozzle on the third sorting platform to improve the sorting success rate.
Solving the above problems will be an important task in our subsequent research.

4. Conclusions

A YOLOv5-based tea fresh leaf intelligent fresh tea leaf sorting system was developed to provide high precision and fast online sorting for fresh tea leaf grade identification.
In this study, tea leaf samples with different gradients of moisture content were collected and combined into a dataset with different gradients of moisture content to investigate the effect of the moisture content of tea leaves on the recognition accuracy of the model. The results show that the recognition effect of the tea recognition model on fresh leaf samples mixed with different staking times is positively correlated with the coverage rate and moisture content gradient of the fresh leaves in the model training data set, with the recognition model accuracy of the four moisture content gradient groups as high as 88.8%. At the same time, online grade recognition with the YOLOv5 model cannot only realize the fast and high-precision recognition of 4.5 ms/frame but also cope well with the recognition problem of a small amount of tea leaf collusion and entanglement caused by uneven vibration in the actual sorting process.
After determining the optimal structural parameters of the system in terms of sorting success rate through a combination of theory and experiment, the system performance was statically evaluated, and the sorting success rate reached 85.4%, and the sorting efficiency was 3 kg/h, which can cope well with the fast and high-precision sorting of fresh tea leaf grades. However, there is room for improvement. For future work, we intend to start with the modification of the model architecture and the optimization of the sorting equipment structure to further improve the accuracy and speed of sorting. This should increase the possibility of using machine-picked tea for the processing of fine tea.

Author Contributions

J.C.: Formal analysis, Data curation, Investigation, Methodology, Software, Validation, Visualization, Writing—original draft, Writing—review and editing. Z.W.: Conceptualization, Data curation, Funding acquisition, Methodology, Validation, Writing—review and editing. X.Z.: Formal analysis, Investigation, Software, Validation. K.L.: Investigation, Writing—review and editing. B.Z.: Validation, Writing—review and editing. C.S.: Conceptualization, Supervision. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by National Key Research and Development Program of China (2021YFD1601102), the National Natural Science Foundation of China (Grant No.52105239), and Anhui Provincial Education Department Key Projects (KJ2020A0133).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Consent to Participate Informed consent was obtained from all individual participants included in the study.

Data Availability Statement

The datasets generated and/or analyzed during the current study are available from the corresponding author on reasonable request.

Conflicts of Interest

We declare that we have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References

  1. Tang, X.; Li, W.; Fan, Q. Research progress of classification technology and related equipment for machine-picked tea fresh leave. China Tea Process. 2015, 2, 5–8. [Google Scholar]
  2. Luo, K.; Wu, Z.; Cao, C.; Qin, K.; Zhang, X.; An, M. Biomechanical characterization of bionic mechanical harvesting of tea buds. Agriculture 2022, 12, 1361. [Google Scholar] [CrossRef]
  3. Zhang, X.; Jia, G.; Zhao, Z.; Zhong, J.; Qiao, X. Research on the evaluation model of tea stall green based on multispectral image parameters. J. Zhejiang Univ. Technol. 2017, 45, 125–129. [Google Scholar]
  4. Xu, L.; Zhao, Y. Automated strawberry sorting system based on image processing. Comput. Electron. Agric. 2010, 71, S32–S39. [Google Scholar]
  5. Zhu, H.; Liu, F.; Ye, Y.; Chen, L.; Li, J.; Gu, A.; Zhang, J.; Dong, C. Application of machine learning algorithms in quality assurance of fermentation process of black tea- based on electrical properties. J. Food Eng. 2019, 263, 165–172. [Google Scholar] [CrossRef]
  6. Dong, C.; Liang, G.; Hu, B.; Yuan, H.; Jiang, Y.; Zhu, H.; Qi, J. Prediction of Congou black tea fermentation quality indices from color features using non-linear regression methods. Sci. Rep. 2018, 8, 3081–3088. [Google Scholar] [CrossRef] [Green Version]
  7. Sofu, M.M.; Er, O.; Kayacan, M.C.; Cetişli, B. Design of an automatic apple sorting system using machine vision. Comput. Electron. Agric. 2016, 127, 395–405. [Google Scholar] [CrossRef]
  8. Yan, G.; Zhu, F. Intelligent grading technology for machine-picked fresh tea leaves based on machine vision. Integr. Circ. Appl. 2022, 39, 176–177. [Google Scholar] [CrossRef]
  9. Wu, Z. Research on Tea Grading Algorithm and Design and Experiment of Sorting Equipment. Ph.D. Thesis, Anhui Agricultural University, Hefei, China, 2020. [Google Scholar] [CrossRef]
  10. Gan, N.; Sun, M.; Lu, C.; Li, M.; Wang, Y.; Song, Y.; Ning, J.; Zhang, Z. High-speed identification system for fresh tea leaves based on phenotypic characteristics utilizing an improved genetic algorithm. J. Sci. Food Agric. 2022, 102, 6858–6867. [Google Scholar] [CrossRef] [PubMed]
  11. Chen, Z.; He, L.; Ye, Y.; Chen, J.; Sun, L.; Wu, C.; Chen, L.; Wang, R. Automatic sorting of fresh tea leaves using vision-based recognition method. J. Food Process. Eng. 2022, 43, e13474. [Google Scholar] [CrossRef]
  12. Hu, G.; Wang, H.; Zhang, Y.; Wan, M. Detection and severity analysis of tea leaf blight based on deep learning. Comput. Electron. Eng. 2021, 90, 107023. [Google Scholar] [CrossRef]
  13. Hu, G.; Yang, X.; Zhang, Y.; Wan, M. Identification of tea leaf diseases by using an improved deep convolutional neural network. Sustain. Comput.-Inf. Syst. 2019, 24, 100353. [Google Scholar] [CrossRef]
  14. Bao, W.; Zhu, Z.; Hu, G.; Zhou, X.; Zhang, D.; Yang, X. UAV remote sensing detection of tea leaf blight based on DDMA-YOLO. Comput. Electron. Agric. 2023, 205, 107637. [Google Scholar] [CrossRef]
  15. Chen, Y.; Chen, S. Localizing plucking points of tea leaves using deep convolutional neural networks. Comput. Electron Agric. 2020, 171, 105298. [Google Scholar] [CrossRef]
  16. Huang, H.; Chen, X.; Han, Z.; Fan, Q.; Zhu, Y.; Hu, P. A tea shoot grading method based on multi-scale attention mechanism and knowledge distillation. J. Agric. Mach. 2022, 53, 399–407+458. [Google Scholar]
  17. Gulzar, Y. Fruit Image Classification Model Based on MobileNetV2 with Deep Transfer Learning Technique. Sustainability 2023, 15, 1906. [Google Scholar] [CrossRef]
  18. Albarrak, K.; Gulzar, Y.; Hamid, Y.; Mehmood, A.; Soomro, A.B. A Deep Learning-Based Model for Date Fruit Classification. Sustainability 2022, 14, 6339. [Google Scholar] [CrossRef]
  19. Thuyet, D.Q.; Kobayashi, Y.; Matsuo, M. A robot system equipped with deep convolutional neural network for autonomous sorting and sorting of root-trimmed garlics. Comput. Electron. Agric. 2020, 178, 105727. [Google Scholar] [CrossRef]
  20. Gulzar, Y.; Hamid, Y.; Soomro, A.B.; Alwan, A.A.; Journaux, L. A Convolution Neural Network-Based Seed Classification System. Symmetry 2020, 12, 2018. [Google Scholar] [CrossRef]
  21. Loddo, A.; Loddo, M.; Di Ruberto, C. A novel deep learning based approach for seed image classification and retrieval. Comput. Electron. Agric. 2021, 187, 106269. [Google Scholar] [CrossRef]
  22. Lu, A.; Guo, R.; Ma, Q.; Ma, L.; Cao, Y.; Liu, J. Online sorting of drilled lotus seeds using deep learning. Biosyst. Eng. 2022, 221, 118–137. [Google Scholar] [CrossRef]
  23. Mamat, N.; Othman, M.F.; Abdulghafor, R.; Alwan, A.A.; Gulzar, Y. Enhancing Image Annotation Technique of Fruit Classification Using a Deep Learning Approach. Sustainability 2023, 15, 901. [Google Scholar] [CrossRef]
  24. Parico, A.I.B.; Ahamed, T. Real time pear fruit detection and counting using YOLOv4 models and deep SORT. Sensors 2021, 21, 4803. [Google Scholar] [CrossRef]
  25. Gui, Z.; Chen, J.; Li, Y.; Chen, Z.; Wu, C.; Dong, C. A lightweight tea bud detection model based on YOLOv5. Comput. Electron. Agric. 2023, 205, 107636. [Google Scholar] [CrossRef]
  26. Yang, J.; Li, Z.; Yuan, L.; Liu, S.; Wu, X.; Sheng, Z. Preliminary study on fresh leaf sorting technology of machine-picked tea. China Tea Process. 2011, 2, 41–45. [Google Scholar]
  27. Yuan, H.; Lu, C.; Mao, Z.; Yin, J.; Lu, R.; Xu, Y.; Lu, D. Preliminary study on the sorting effect of new fresh leaf sieving machine for famous green tea. China Tea 2011, 33, 19–21. [Google Scholar]
  28. Zhang, L.; Dong, Z.; Tang, M.; Cai, W.; Wen, Z.; Luo, Y. Research on the sorting technology of fresh leaves of famous tea machine picking. J. Zhejiang Univ.-Agric. Life Sci. 2012, 38, 593–598. [Google Scholar] [CrossRef]
  29. Gao, Z.; Wang, A.; Liu, Y.; Zhang, L.; Xia, Y. Intelligent fresh-tea-leaves sorting system research based on convolution neural network. Trans. Chin. Soc. Agric. Mach. 2017, 48, 53–58. [Google Scholar]
Figure 1. Fresh tea leaf grade diagram.
Figure 1. Fresh tea leaf grade diagram.
Applsci 13 03551 g001
Figure 2. Detailed structure of the grade sorting system.
Figure 2. Detailed structure of the grade sorting system.
Applsci 13 03551 g002
Figure 3. Sorting flowchart.
Figure 3. Sorting flowchart.
Applsci 13 03551 g003
Figure 4. Structure of the YOLOv5s model.
Figure 4. Structure of the YOLOv5s model.
Applsci 13 03551 g004
Figure 5. Distribution of morphological centroids of fresh leaves.
Figure 5. Distribution of morphological centroids of fresh leaves.
Applsci 13 03551 g005
Figure 6. Detailed diagram of the sorting process.
Figure 6. Detailed diagram of the sorting process.
Applsci 13 03551 g006
Figure 7. Schematic diagram of air-blowing trajectory.
Figure 7. Schematic diagram of air-blowing trajectory.
Applsci 13 03551 g007
Figure 8. Training results of the different optimizers. (A) Training loss diagram of different optimizers; (B) Training precision of different optimizers.
Figure 8. Training results of the different optimizers. (A) Training loss diagram of different optimizers; (B) Training precision of different optimizers.
Applsci 13 03551 g008
Figure 9. Confusion matrix.
Figure 9. Confusion matrix.
Applsci 13 03551 g009
Figure 10. Recognition effect diagram.
Figure 10. Recognition effect diagram.
Applsci 13 03551 g010
Figure 11. Air-blowing track effect.
Figure 11. Air-blowing track effect.
Applsci 13 03551 g011
Figure 12. (a) Effect of the whole machine. (b) Effect of the image acquisition system. (c) Enlarged view of air-blowing nozzle.
Figure 12. (a) Effect of the whole machine. (b) Effect of the image acquisition system. (c) Enlarged view of air-blowing nozzle.
Applsci 13 03551 g012
Table 1. Sample data statistics.
Table 1. Sample data statistics.
Sample NumberSample
Type
Original ImageAugmented Image
SLOBOLOBTLOBTLSLOBOLOBTLOBTL
awithout spreading4004004004002000200020002000
bspreading 2 h4004004004002000200020002000
cspreading 4 h4004004004002000200020002000
dspreading 6 h4004004004002000200020002000
Table 2. Model comparison data.
Table 2. Model comparison data.
PR[email protected]Time (ms)Weights (MB)
YOLOv5s0.8880.8840.9104.713.7
YOLOv30.8960.9130.93318.7117
Fast RCNN0.7630.9740.975138108
SSD0.7250.7060.75911092.1
Table 3. Model evaluation results.
Table 3. Model evaluation results.
ModelPRF1Map
abcd0.8880.8840.910.748
a0.8160.7940.8040.629
b0.7790.7450.7440.581
c0.7960.8040.8060.631
d0.7620.7720.7640.575
ab0.8440.8480.8590.692
ac0.8570.8580.8790.709
ad0.8420.8420.8590.689
cd0.8480.850.8640.684
bc0.8050.8190.8390.643
bd0.8420.8330.8460.677
Table 4. Statistical analysis results.
Table 4. Statistical analysis results.
GroupCategoryGrade
SLOBOLOBTLOBTLAll
Group 1Box133310
Box203510
Box372376
Box400134
success rate82.5%87.5%92.5%85%86.9%
Group 2Box130311
Box233401
Box373365
Box400333
success rate75%85%90%82.5%83.1%
Group 3Box133302
Box233521
Box332363
Box410234
success rate82.5%87.5%90%85%86.3%
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Cao, J.; Wu, Z.; Zhang, X.; Luo, K.; Zhao, B.; Sun, C. Sorting of Fresh Tea Leaf Using Deep Learning and Air Blowing. Appl. Sci. 2023, 13, 3551. https://doi.org/10.3390/app13063551

AMA Style

Cao J, Wu Z, Zhang X, Luo K, Zhao B, Sun C. Sorting of Fresh Tea Leaf Using Deep Learning and Air Blowing. Applied Sciences. 2023; 13(6):3551. https://doi.org/10.3390/app13063551

Chicago/Turabian Style

Cao, Jie, Zhengmin Wu, Xuechen Zhang, Kun Luo, Bo Zhao, and Changying Sun. 2023. "Sorting of Fresh Tea Leaf Using Deep Learning and Air Blowing" Applied Sciences 13, no. 6: 3551. https://doi.org/10.3390/app13063551

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop