Tomato Sorting System Based on Machine Vision

Abstract

In the fresh tomato market, it is crucial to sort and sell tomatoes based on their quality. This is important to enhance the competitiveness and profitability of the market. However, the manual sorting process is subjective and inefficient. To address this issue, we have developed an automatic tomato sorting system that uses the Raspberry PI 4B as the control platform for the robot arm. This system has been integrated with a human–computer interaction interface sorting system. Our experimental results indicate that this sorting method has an accuracy rate of 99.1% and an efficiency of 1350 tomatoes per hour. This development is in line with modern agricultural mechanization and intelligence needs.

1. Introduction

The tomato is an annual herb belonging to the Solanaceae genus. It is originally from South America, and today, countries such as the United States, Russia, Italy, and China are the primary producers of tomatoes. It is a good source of carotenoids (namely, lycopene, β-carotene, and lutein), flavonoids, phenols, minerals, vitamin C, vitamin E, dietary fiber, and other essential nutrients and phytochemicals [1,2,3]. Regarded as the most crucial vegetable [4], it is one of the most widely consumed vegetable crops across the globe [5,6]. People love it for both daily consumption and food processing purposes.

In the tomato market, it is crucial to test the quality of tomatoes and classify them appropriately for sale. The main factors that determine the quality of a tomato are its color and appearance [7]. Traditionally, fruit farmers use manual sorting to identify the quality of tomatoes by visually inspecting them [8]. However, this process is inefficient, labor-intensive, expensive, and subjective. Therefore, we developed an automatic tomato sorting system. This system can quickly sort a large number of tomatoes, improving efficiency and reducing the workload for fruit farmers. Additionally, since different markets have varying demands for tomato quality, our sorting system can detect three key attributes to obtain more accurate data than manual sorting. It ensures that each tomato meets the quality requirements of different markets, ultimately improving the competitiveness of our tomatoes in the market.

We utilized the technology of machine vision to recognize and process images of tomatoes. Machine vision offers several advantages, such as being non-destructive, accurate, and consistent. This technology is capable of identifying the internal and external characteristics of agricultural products, including external size, maturity, defect area, internal moisture, and nutrient composition [9]. By using machine vision technology for automatic sorting, we can reduce costs, improve efficiency, and increase the accuracy of sorting. Zhang et al. developed an automatic cherry tomato grading system based on machine vision. The system collected images of each cherry tomato from three different angles and extracted nine feature points from each image. The Principal Component Analysis (PCA) and the Linear Discriminant Analysis (LDA) were used for feature analysis and classification. The system classified cherry tomatoes into three levels: mature, semi-mature, and immature. The accuracy of the system was tested on a total of 414 cherry tomatoes and achieved an overall accuracy rate of 94.9% [10]. M. Haggag et al. proposed an intelligent hybrid experimental deep-learning algorithm based on a tomato sorting controller. Five deep learning training algorithms were used as variables to analyze the three groups of tomato images, identify the maturity and defect degree of tomatoes, and obtain the optimal algorithm through comparison to improve the reliability and classification accuracy of automatic tomato sorting [11]. Harvey Jake G. Opena et al. used the artificial neural network trained by ABC to automatically grade tomato maturity [12]. M. Fojlaley et al. studied tomato classification and sorting in machine vision based on SVM, MLP, and LVQ [13].

We utilized machine vision analysis technology along with a deep learning algorithm to capture tomato image features, including defect area, size, and maturity. These three indicators were accurately analyzed simultaneously, resulting in more refined sorting and making it more convenient to sell. We categorized tomatoes into three grades based on three characteristic indices: premium grade, first grade, and second grade.

In terms of hardware, O. O. Arjenaki et al. used computers to process CCD camera signals and image capture [14]. Our hardware mainly used the Raspberry PI, which is an efficient and powerful small computer [15]. Raspberry Pi offers several advantages over traditional computers, including its low cost, compact size, and user-friendly interface with 40 GPIO (universal input/output) pins, allowing for easy connection of multiple external hardware devices. Moreover, it supports almost all of the peripherals of Arduino. Raspberry Pi also integrates machine learning and deep learning models [16], and it has a faster processor than Arduino and other microcontroller modules. The latest Raspberry Pi 4B version integrates a 1.6 GHz processor, while the Arduino board integrates the controller. As a result, this board has a faster response time compared to other modules. At this stage, the Raspberry PI embedded system has become an important scientific and engineering application [17], which helps to improve the objectivity and simplicity of microcomputer and camera fruit physical and chemical properties and quality parameter measurement systems [18]. The Raspberry PI in this system is equipped with image processing libraries OpenCV and YOLOv8 for image recognition and processing, and the robot arm is controlled by the Arduino CNC shield V3.0 driving board for sorting.

The rest of the paper is arranged as follows: The second part introduces the overall process of the tomato sorting system and grading standard. The third part presents the materials and methods used in the system implementation. The fourth part describes the implementation of the interactive interface of the software. The fifth part explains the experimental method and shows the experimental data. Finally, the sixth part summarizes the conclusion.

2. Overall Scheme of the System

2.1. System Process

This system is based on a Raspberry PI 4B, a USB camera, a robotic arm, and conveyor belt hardware. It classifies tomatoes based on their maturity, size, and defect degree and sorts them into three levels: premium, first, and second. To check if all system components are functioning normally, click the start button on the user interface. Once the detection is successful, the conveyor belt will begin transporting the tomato to the location where the camera takes photos. When the camera detects the object, the conveyor belt will stop moving and send a signal to the Raspberry Pi to take photos of the tomatoes. Upon receiving the signal, the Raspberry Pi will control the camera to capture images of the tomatoes and store them on a specific path. Subsequently, the Raspberry Pi will analyze and categorize the images to determine the grade of the tomatoes. Then, based on the grade, the Raspberry Pi will send instructions to the robot arm, which will proceed to place the tomatoes in the corresponding frame. The general flow chart of the system is shown in Figure 1.

2.2. Tomato Grading Standards

• The tomato grading standard of this system refers to the “NY/T 940-2006 Tomato Grade Specification” [19], which divides tomatoes into special grade, first grade, and second grade according to their size, maturity, and defect degree.
• Special grade: the fruit is large, round, and fully ripe, with a uniform and bright color and smooth skin without any defects;
• First grade: the fruit is also full but slightly deformed, with bright and uniform color and slightly defective skin;
• Second grade: the fruit is medium-sized, slightly underripe, and has some blemishes on the skin.

Other fruits are not recommended for sale due to poor quality.

Table 1. Tomato grade parameters.

Size	Maturity	Defect Area	Grade
≥70 mm	≥90%	0 cm2	Premium Grade
50–70 mm	80–90%	0–1 cm2	Grade I
≤50 mm	≤80%	0–1 cm2	Grade II

shows the specific parameters of tomato size, maturity, and defect degree.

3. Materials and Methods

3.1. Materials

The system’s physical components are depicted in Figure 2. The robot arm is equipped with a USB camera that captures images of tomatoes. Subsequently, the robot arm picks up the tomatoes and positions them within the red frame. The Raspberry Pi is employed for tomato image recognition and classification, as well as for controlling the robot arm. The Arduino CNC shield V 3.0 is utilized to operate the motor on the robot arm, while the PC is used for debugging and display purposes. The tomato sorting system consists of four main parts, including image recognition, communication between software and hardware, software design, and hardware workflow. The composition and workflow of each part and the relationships between them are shown in Figure 3. In the following sections, we will provide an introduction to each component and describe how they interact with one another.

3.1.1. PC

By using the existing personal computer to achieve remote login to the Raspberry PI (Seeed Studio, Shenzhen, China), this method can realize the graphical interface operation of the Raspberry PI without an external display and save the cost of the Raspberry PI display. The remote login here can use the remote login function of the Windows platform. We only used the laptop to analyze, develop, deploy, test, and configure the tomato sorting system based on the Raspberry PI. It is not part of the final system.

3.1.2. USB Camera

The camera model used in this system is the NVIDIA Jeston driverless series USB camera, which supports a variety of UVC protocols on the Raspberry PI, has adjustable visual focus, and can realize AI face recognition. The Raspberry PI is connected to a USB camera through its built-in USB port, and the USB camera is called to obtain tomato images for further image processing by the Raspberry Party. To minimize inaccuracies and post-processing difficulties caused by uneven lighting and shadows, it is important to use a light source with uniform lighting when taking pictures of tomatoes. LED lights should be positioned around the camera to provide consistent illumination from top to bottom, effectively reducing the impact of lighting on tomato picture quality.

3.1.3. Raspberry PI

The Raspberry PI is a low-cost single-board computer with desktop computer functions. In this system, the primary task of the Raspberry Pi is to regulate the hardware and process tomato detection images. It also plays a vital role in designing the driver, communication, control flow, and human–computer interaction interface.

3.1.4. Arduino CNC Shield V3.0

Arduino CNC Shield V3.0 can carry stepper motor driver modules such as A4988 and DRV8825, all manufactured by Lexin in Shenzhen, China. In this system, the Raspberry PI is connected to the Arduino CNC shield V3.0 through a PWM GPIO port to realize communication. The Raspberry PI sends the tomato grading results to Arduino CNC shield V3.0 through instructions and then controls the mechanical arm to sort tomatoes.

3.1.5. Mechanical Arm

This system uses a 3D-printed robotic arm with a base, a movable arm segment, and a gripper. The robotic arm is powered by three 42-stepper motors, a reducer, and a driver combination. The gripper consists of a servo and claws that rotate to open, close, and grasp the tomato. The 42-stepper motor provides the necessary power, the driver controls the speed and torque, and the reducer ensures smooth action. The Arduino CNC shield V3.0 receives instructions from the Raspberry Pi and drives the stepper motor of the mechanical arm to sort tomatoes as a whole.

3.1.6. Conveyor Belt

We have selected a PVC material conveyor belt with a size of 530 mm × 70 mm × 80 mm, a 3730 deceleration motor (AC380V, output speed of 100 mm/s), rolling bearings, and rollers to create a tomato conveying device. The conveyor belt has a fixed tomato slot, and each slot is 50 cm apart. The conveyor belt rotates once every 5 s, and the distance of 50 cm is rotated each time to transfer each tomato to a set position where a robotic arm can easily grab it. This ensures that the tomato is accurately conveyed to the robotic arm for further processing.

3.2. Method

3.2.1. Detect Defects in Tomatoes

For the defects in tomatoes, we graded them according to the defect area. The specific standards for grading tomato defects are shown in

Table 2. Standard table of defect degree.

	No Defect	Minor Defect	Serious Defect
Defect area	0 (cm2)	0–1 (cm2)	>1 (cm2)

. To accurately detect the degree of defect in tomatoes, we improved the YOLOv8 model to achieve the defect degree of tomatoes.

(1)

YOLOv8

YOLOv8 (You Only Look Once version 8, Ultralytics, Los Angeles, CA, USA) is a cutting-edge object detection technology that builds on the success of previous YOLO versions on object detection tasks, further improving performance and flexibility. The main innovations compared to the previous Yolo series include a new backbone network, a new Ancher-Free detection head, and a new loss function. Its main network architecture is shown in Figure 4.

(2)

Dataset preparation and training

We obtained tomato images by both taking photos and downloading them from the internet. The target borders and categories in each image were annotated using the annotation tool in Roboflow Annotate. To ensure good training results, we collected and made a large number of datasets, with 470 images in total, including 370 images in the training set, 50 in the test set, and 50 in the validation set. Since the Convolutional Neural Network (CNN) requires the input image to have a fixed size, we fixed the image size to 640 × 640 pixels. In order to accelerate the model’s convergence, we rescaled the pixel values of the image from [0, 255] to [0, 1]. Figure 5 shows a comparison of the loss function before and after image scaling. We observed that when the input feature values have a large range, the model’s convergence rate is slow, and numerical instability is encountered during the training process. Additionally, scaling the image pixel values helps to reduce the model’s sensitivity to input data noise, thereby improving its generalization ability.

Some images and annotations are shown in Figure 6 below. To detect the defect degree of tomatoes in different growth cycles under various complex environmental backgrounds and improve the robustness of the model, we used an adaptive anchor box calculation method to automatically generate the optimal anchor box size configuration by analyzing the size distribution of tomatoes, which improves the accuracy of boundary box localization. In the selection of the loss function, the cross-entropy loss and CIoU loss were combined. The former is responsible for optimizing the classification accuracy, while the latter further improves the prediction ability of the model for the tomato bounding box.

(3)

Analysis of training results

In the process of model training, in order to continuously improve the detection accuracy of the model, we constantly adjust the number of training rounds and continuously optimize the parameters of the model. During the model training process, we made necessary adjustments to the number of training rounds and optimized the model parameters to improve its detection accuracy.

The selection of the learning rate is crucial for the performance and training stability of the model. We implemented different strategies to adjust the learning rate while keeping other model parameters constant. Initially, we employed the linear attenuation approach and observed a consistent decrease in the learning rate throughout the training process. This strategy is simple and intuitive and can balance training speed and generalization ability to a certain extent. However, the learning rate becomes too low in the late training period, hindering further optimization of the model. The cosine attenuation strategy maintains a small learning rate in the late training period, which helps avoid shocks and improves the generalization ability of the model. Consequently, we decided to adjust the learning rate using the cosine attenuation strategy to achieve better overall performance.

The following table shows the detection accuracy of the validation set under different training times with the optimal model parameters set. In

Table 3. Training results with different training times.

Epochs	Class	Image	Labels	p	R	map@0.5	map@0.5:0.95
30	All	50	50	0.981	0.963	0.981	0.790
60	All	50	50	0.938	0.964	0.983	0.856
100	All	50	50	0.974	0.933	0.986	0.877
200	All	50	50	0.958	0.963	0.982	0.881
300	All	50	50	0.967	0.947	0.991	0.873
400	All	50	50	0.965	0.967	0.98	0.873

, we observe that the model performance on the training set steadily improves as the number of training epochs increases. However, we also notice a trend: as the number of epochs increases, the model starts to overfit on the validation set.

In order to select the models with the best average performance, we focused on the mAP@0.5 metric. This metric measures the mean Average Precision (mAP) of a model at an IoU (Intersection over Union) threshold of 0.5. mAP@0.5 is an important parameter that not only reflects the model’s performance at a particular IoU threshold but also evaluates the overall performance of a model across a range of IoU thresholds. By comparing the performance under different training cycles, we found that the model achieved the best average performance after training 300 cycles. In deep learning practice, model training is often monitored by the decreasing trend of the loss function. Figure 7 shows the performance of the model after training 300 epochs, which provides us with an intuitive understanding of the learning dynamics of the model.

Figure 7 is divided into two parts, with the top half displaying the results obtained from the test set and the bottom half showing the results obtained from the validation set. Localization loss (box_loss) is used to calculate the error between the predicted and actual position of an object. The smaller the localization loss, the more accurate the localization. Objectness is used to determine whether a region may contain an object in object detection. A higher probability of objectness indicates a higher likelihood of an object being present in that region. Classification loss calculates whether the anchor and corresponding labeled classification are correct. The smaller the classification loss, the more accurate the classification. Precision is the proportion of the predicted class that is positive, while recall is the proportion of all samples that are actually positive and are correctly predicted to be positive. mAP@0.5 and mAP@0.5:0.95 stand for mean Average Precision, which is a common metric to evaluate the performance of object detection models. mAP@0.5 represents the average precision at an IoU (Intersection over Union) threshold of 0.5. mAP@0.5:0.95 indicates the average of the average precision at IoU thresholds ranging from 0.5 to 0.95. The relationship between precision and recall is often described as a precision–recall (PR) curve, as shown in Figure 8. We can learn that the average value of mAP@0.5 of the trained model for the detection of three types of targets is as high as 99.0%, which has a very high recognition rate for identifying the degree of defects in tomatoes.

3.2.2. Detect the Size of Tomatoes

In order to determine the size of a tomato, we need to extract a feature quantity that reflects its size. The feature quantities that describe the size of an object include radius, perimeter, area, and so on. Radius is the most direct reflection of the size of the object and can be obtained by calculating the perimeter or area of the object. It can also be obtained by calculating the radius of its circumcircle, which can be conducted by cutting a rectangle around the object and then measuring its radius. The perimeter is proportional to the radius. Therefore, it can also reflect the size of the regular object to some extent.

(1)

The calculation of the perimeter

The basic geometric characteristics of tomatoes play a very important role in the study of tomato grading. There are three definitions of the perimeter in digital images. First, the pixels of an image are regarded as small squares of unit area, so the image is composed of target area squares and background squares. This understanding is more suitable for calculating the area of the target region. The second representation is the area of the boundary, which is the sum of the points of the boundary. This method is relatively simple for the perimeter, but the accuracy is not high. The third definition considers a pixel as a point, and the perimeter is the sum of the distances between all points. We used the 8-connected method to calculate the perimeter, as shown in Figure 9. This method includes the up, down, left, and right connections, as well as the four diagonal connections to neighboring pixels. The distances between all adjacent boundary pixels are added, taking into account the diagonally oriented neighbors, which may be 1 (horizontal and vertical neighbors) or $\sqrt{2}$ (diagonal neighbors).

(2)

Specific steps of perimeter calculation

• The 8-connected edge image of the tomato is obtained by the bottom processing of the image;
• The edge of the image is detected to extract the edge pixels in the image, and the total number of edge pixels (E) is calculated;
• The number of 4-connected points (X) in horizontal direction and vertical direction is calculated, respectively (whether the difference between horizontal coordinate and vertical coordinate is 1 is judged);
• The number of points (Y) in diagonal connections is obtained by subtracting the number of 4-connected points from the total pixel value:

  $$Y=E-X$$

  (1)
• Finally, we obtain

$$P=X+\sqrt{2}Y$$

(2)

Add all the distance values, and the result is the perimeter (P) of the target object.

(3)

Tomato size grading experiment

It is noticeable that the size of a circle’s circumference can indicate the size of its radius. The circumference and radius have a proportional relationship. Therefore, by measuring the circumference of a tomato image, one can estimate the diameter of the tomato fruit. The standard NY/T 940-2006 [19] provides parameters for classifying tomato diameters into large, medium, and small categories, as shown in

Table 4. Classification standard for tomato fruit diameter.

	Large	Middle	Small
Diameter	>7 (cm)	5 to 7 (cm)	<5 (cm)
Pixels	>250.34	200.52–250.34	<200.52

.

It is evident from the table provided that the maximum cross-sectional diameter of different grades of tomatoes varies by 20 mm, which is a significant difference. The digital image shows that the perimeter of tomatoes of different grades also varies considerably. This tolerance range makes it possible to quickly classify the size of the tomatoes. By calculating the length and width of a unit pixel as 0.27962 mm, we can classify large fruit as having a diameter greater than 250.34 pixels, middle fruit as having a diameter between 200.52 and 250.34 pixels, and small fruit as having a diameter less than 200.52 pixels. Therefore, we can classify the size of the tomato by setting a threshold.

3.2.3. Detect the Maturity of the Tomatoes

In order to accurately detect the ripeness of tomatoes, the first step is to convert the captured RGB image into an HSV image. This will enable us to perform tomato color threshold segmentation later. Color thresholding is a commonly used technique in the field of image processing to separate regions with a specific color from an image. HSV (Hue, Saturation, Value) is a color space created based on hue (H), saturation (S), and lightness (V), also known as the Hexcone Model. The HSV color space is widely used in color thresholding due to its similarity to human visual perception. This color model is particularly useful in detecting tomato maturity. A diagram of the tomato ripening stages is as follows Figure 10:

First, we loaded the image using the OpenCV library and converted it from RGB to HSV space to accurately utilize the color information, as shown in Figure 11. We set color thresholds for green and red in HSV space beforehand and used the cv2.inRange function to extract the corresponding color regions, resulting in a mask image. These masks were then merged, and we applied erosion and dilation operations to remove noise and smooth edges. Finally, we used the cv2.findContours function to extract the contours in the mask and judged the maturity of the target object based on the size of the contour area. The smaller contours were marked as immature, and the larger contours were marked as mature. The specific classification criteria for tomato maturity are shown in

Table 5. Classification criteria for tomato maturity.

	Ripe	Medium	Unripe
Red area	90% or higher	80–90%	80% or less

.

The detection results obtained according to the above method are illustrated in Figure 11. The experimental results show that this method can accurately identify the maturity of tomatoes, and the accuracy rate can reach 98.37%.

3.2.4. The Working Method of the Robotic Arm

When the conveyor belt transmits the tomato to the robot arm, the USB camera obtains the tomato picture and sends it to the Raspberry PI, which grades the tomato and obtains the tomato coordinates. The robot arm accurately grabs the tomato and puts it into the corresponding position according to the tomato grading results and coordinates. The working principle of the robot arm uses reverse kinematics, solves the joint angle of the robot arm through the position and orientation of the end of the robot arm, calculates the motion parameters of each joint of the robot arm, and converts them into the pulse times of the motor rotation to realize accurate control.

According to the forward kinematics of the manipulator arm, the transformation matrix of the tool coordinate system relative to the base coordinate system can be solved. The meaning of the transformation matrix can be obtained.

$${\mathsf{\theta }}_2=\mathrm{A}\tan 2({\mathrm{s}}_1,{\mathrm{c}}_1)$$

(3)

$${\mathsf{\theta }}_1=\mathrm{A}\tan 2\left(\mathrm{y},\mathrm{x}\right)-\mathrm{A}\tan 2({\mathrm{k}}_2,{\mathrm{k}}_1)$$

(4)

$${{\mathsf{\theta }}_1+{\mathsf{\theta }}_2+\mathsf{\theta }}_3=\mathrm{A}\tan 2\left({\mathrm{s}}_{\mathrm{\varnothing }},{\mathrm{c}}_{\mathrm{\varnothing }}\right)=\mathrm{\varnothing }$$

(5)

Among them, the meaning of the variables used in the formula is as follows:

• ${\mathsf{\theta }}_1$, ${\mathsf{\theta }}_2$, ${\mathsf{\theta }}_3$: The angles of the 1st, 2nd, and 3rd joints of the robotic arm.
• x, y: The position coordinate of the tool coordinate system with respect to the base coordinate system.
• ${\mathrm{k}}_1,{\mathrm{k}}_2$: Intermediate variables calculated by ${\mathrm{l}}_1,{\mathrm{l}}_2$, and joint Angle ${\mathsf{\theta }}_2$.
• ${\mathrm{s}}_{\mathrm{\varnothing }},{\mathrm{c}}_{\mathrm{\varnothing }}$: The sine and cosine values of $\mathrm{\varnothing }$.
• ${\mathrm{c}}_1,{\mathrm{s}}_1$: Cosine and sine of theta 1.
• ${\mathrm{l}}_1,{\mathrm{l}}_2$: The length of the two connecting rods of the robot arm.
• $\mathrm{\varnothing }$: The rotation Angle of the tool coordinate system with respect to the base coordinate system.
• $\mathrm{Atan}2$(): The inverse tangent function is used to find the Angle.

At the same time, the robot arm was simulated using LabVIEW 2024 (National Instruments, Austin, TX, USA) based on the three degrees above, as illustrated in Figure 12. By processing the corresponding relationship and initial position of the parts, writing the control program in the while loop, connecting the hardware of the manipulator, and using the appropriate interface or communication protocol for data exchange, the control of the manipulator can be realized. Through the accurate calculation and control of inverse kinematics, the manipulator can achieve a high degree of automation and intelligence in the fruit sorting process.

4. Implementation of Software Interactive Interface

The tomato grading software was designed with the Raspberry PI driver, communication, control process, and human–computer interaction interface. The development of the Raspberry PI, communication functions, and control software was carried out using RPI.GPIO0.7.1, Socket3.11, and PyQt5.15. It completes the overall software design of fruit sorting equipment. Apart from the automatic control process, the manual control function is also essential. Manual control enables real-time control of hardware such as conveyor belts, mechanical arms, grading mechanisms, and cameras. Additionally, manual control includes operations such as selecting regions of interest in the image and selecting image processing methods.

4.1. Communication Implementation

The Raspberry PI comes with a WiFi Bluetooth module, which can be set as an AP hotspot by modifying the relevant system files. The specific process is shown in Figure 13.

When we disable the WLAN0 interface, the Raspberry PI may not be able to connect to other hotspots. In addition, the WLAN0 port used as the AP hotspot should have a fixed IP address. To set up the hotspot, we downloaded the Hostap program and modified the hotspot configuration, including the network interface, hotspot name, encryption method, and password. To ensure complete network communication, we allocated an IP address to the new device (PC). The Raspberry PI 4B comes with a WiFi Bluetooth module, which can use the AP hotspot to achieve TCP/IP communication based on Socket mode. The Socket provides an interface for the implementation of the application layer and acts as an intermediate software abstraction layer between the application layer and the transport layer of the TCP/IP protocol family. The communication equipment is divided into client and server, where the PC is the client and the Raspberry PI is the server.

4.2. Setting of User Interaction Interface

The function modules of the control software of the grading system are shown in Figure 14. The user interface is the embodiment of the integration of the device functions. This software includes the user information module, the device connection module, the image operation and classification control module, the hardware control module, the information display module, and the file operation and database module.

The application form module of the software is designed using PyQT5, an integrated library that is the Python interface of the graphics programming framework Qt5. The application form of the software is designed by the Qt Designer tool, and the controls are arranged in the edit box of the form by dragging. After setting the relevant parameters, such as position, size, and inclusion relationship of the controls, it is saved as a UI file. Pyqt5 is used to compile the UI file into a Python file to form the GUI module of the form. The Pyqt5 signal and slot programming mechanism were used to write the transaction logic of the form application. The main interface of the software is shown in Figure 15.

5. Experimental Data and Discussion

In this study, the three indicators of defect degree, size, and maturity of tomatoes are used as the basis for evaluating the quality of tomatoes. According to the method mentioned above, the defect degree, size, and maturity of tomatoes are simultaneously detected, and the tomatoes are graded according to the tomato grading standards specified by us. The following table is part of our actual test results, as shown in

Table 6. Datasheet of tomato sorting part.

Tomato Picture	Detection Effect	Tomato Size	Tomato Ripeness	Tomato Defect Detection	Tomato Grade Determination
		7.52 cm	ripe	No defect	Premium Grade
		5.98 cm	ripe	No defect	Primary fruit
		4.79 cm	ripe	No defect	Secondary fruit
		4.91 cm	medium	No defect	Secondary fruit
		5.96 cm	medium	Minor defect	Not recommended for sale
		6.54 cm	ripe	Minor defect	Not recommended for sale
		3.64 cm	unripe	No defect	Not recommended for sale
		4.23 cm	ripe	Serious defect	Not recommended for sale
		4.67 cm	unripe	Minor defect	Not recommended for sale
		3.94 cm	unripe	Serious defect	Not recommended for sale

.

We used 90 tomatoes to do experiments, and the accuracy of size was 100%, the accuracy of ripening was 98.37%, and the accuracy of defect area was 99%, so the accuracy of our entire system could be 99.1% on average of these three indicators. We have conducted tests on the tomato sorting system and gathered data from ten tests, as detailed in

Table 7. Efficiency test data of the tomato sorting system (counts/min).

	Group 1	Group 2	Group 3	Group 4	Group 5
The number of tomatoes	22	23	22	22	21
	Group 6	Group 7	Group 8	Group 9	Group 10
The number of tomatoes	22	23	24	23	23

. Based on our findings, we estimate that this tomato sorting system can process 1350 tomatoes per hour, significantly surpassing the manual sorting capacity of 726 tomatoes per hour [20,21]. In addition, the efficiency of manual sorting decreases over time, leading to slower sorting and reduced accuracy.

The tomato sorting system studied by Joice A. et al. has an efficiency of 1800 tomatoes per hour, but it only sorts the size and maturity of tomatoes. The accuracy percentage for sorting full-ripened tomatoes is 98% [22]. In comparison, our system can identify and classify three characteristics of tomatoes: size, ripeness, and defect area, and the accuracy percentage of sorting full-ripened tomatoes is 98.37%. Haggag, M. et al.’s theoretical classification accuracy of tomato ripeness and defect (overripe and rotten) was 90% [11]. Rajkumar P et al.’s tomato color sorter only recognizes the maturity of tomatoes, with an overall recognition accuracy of 94.6%. Additionally, their system operates at an efficiency of 40 kg of tomatoes per hour, equivalent to an estimated 254 tomatoes per hour based on the average weight of tomatoes [23]. Most of the current tomato sorting systems rely on computers to identify and classify tomatoes [10,11,14,23]. However, the Raspberry Pi, as a convenient and cost-effective microcomputer, offers a price advantage over using traditional computers for tomato identification and classification.

6. Conclusions

This paper presents a solution to the problem of tomato sorting by designing and developing a tomato sorting system based on machine vision. The system utilizes the Raspberry Pi 4B as the core control platform and a mechanical arm as the sorting device. It detects the size, maturity, and defect area of tomatoes and integrates the judgment results of these three characteristics to obtain the final grade of the tomatoes. The experimental results demonstrate that the system achieves a total sorting accuracy of over 99.1%, with 100% in size, 98.37% in maturity, and 99% in detecting defect areas. Additionally, the sorting efficiency of the system can reach 1350 tomatoes per hour. Overall, this system has the potential to enhance the competitiveness and profitability of the fresh tomato market, and it holds significant economic value, making it worthy of promotion and widespread application.

The limitations of this study are as follows: First, the system fails to take into account the differences between individual tomato data of different varieties under different environmental conditions, and this system is designed specifically for tomatoes, which has limited adaptability to the sorting of other crops and will decrease the accuracy rate. Meanwhile, the initial setting and calibration of the human–computer interaction interface will become an obstacle for users to use. Therefore, our future research will comprehensively collect and train the data of various varieties of tomatoes under different environmental conditions so that the system has no blind spots in the tomato market. Meanwhile, datasets from other crops will be added and trained to improve the universality of the system. More sensitive and easier-to-adjust sorting equipment will be used on the hardware to adapt to the shapes of different crops: height and size. We will also optimize the human–computer interaction interface, reduce the complexity of user use without affecting the function, make the interface clearer, and add a user operation guide and online question module to help users quickly grasp the basic operations.