Machine Learning and Queuing Algorithm Integration for Real-Time Citrus Size Classification on an Industrial Sorting Machine

Abstract

The classification of lemons by size is a crucial industrial process that ensures specific quality standards. Lemon sorting is typically performed by hand or often using expensive, outdated machines. In this article, we develop Machine Learning and Queuing algorithms, program them on low-cost hardware—specifically, a microcontroller and a single-board computer—and integrate them with an existing fruit-sorting machine, which classifies lemons by size. We acquired a dataset of 3127 lemon images in six industry-standardized sizes. We developed algorithms to extract geometric features, including one based on the peduncle location, which is estimated using a pre-trained Faster Objects, More Objects (FOMO) model. We used these features to train and evaluate five machine learning models, with the best-performing model achieving 87.22% accuracy over a set of lemons acquired under controlled conditions. We tested the proposed system in a real industrial environment, proving its feasibility by sorting 1558 lemons and obtaining an accuracy of 78.00%, despite the industrial-standard sizes having considerable overlap.

1. Introduction

Lemon is a citrus fruit whose origin is estimated to be in India. It is a typical fruit of tropical regions with multiple uses in the food and pharmaceutical industries. Mexico produces three varieties of lemons. Colima, Michoacán, and Guerrero states produce the most common variety, the Mexican lemon (also named Colima lemon) variety (Citrus aurantifolia). Veracruz, Quintana Roo, Oaxaca, Yucatán, and Tabasco produce the second most common lemon variety, the Persian lemon (Citrus latifolia). Finally, Tamaulipas and San Luis Potosí produce the Italian (also named Yellow) lemon variety (Citrus limon).

According to statistics from the United States Department of Agriculture (USDA), in 2023, Mexico was the second-largest producer of lemons and limes [1]. In addition, Mexico’s Ministry of Agriculture and Rural Development (SADER) confirms Tamaulipas as one of the top lemon-producing states nationwide. Furthermore, based on statistics from the Tamaulipas State Government, in 2016 this state was the largest exporter of Italian lemons, estimating some 22 thousand tons, with the main client being the United States [2]. Italian lemons are mainly used for sale as fresh fruit, although they are also of interest to industries dedicated to juice production.

In the harvesting, handling, and distributing of fruits and vegetables, the grading process is essential to maintain current quality standards and regulations that companies must comply with. The environmental impact of lemon harvesting and distribution is high. In [3], the authors estimate the carbon, water, and energy footprints of the production and process of the Persian lemon in Mexico. Therefore, developing an efficient and compact lemon sorting software with minimum power requirements is indispensable. The developed software is the system’s brain, coupled to a conveyor band to separate the fruit by activating the appropriate actuators.

A sorting machine allows the user to sort fruits by some characteristic (size, ripeness level) faster than manual sorting. Companies adapt fruit-sorting machines to requirements and quality standards goals. Some artificial vision systems allow for discarding fruits in lousy condition or affected by a pest and classify them by size or maturity level, ensuring a quality product and value in the market [4].

In the state-of-the-art, we found several articles reporting work on the classification of lemons. Most of them report classification results under ideal conditions, from images captured with good illumination and sensors that handle high resolutions, generally without motion-generated noise, which favors classification algorithms in obtaining high classification results. The other class includes works that report algorithms trained and/or tested on functional separating machines, which introduce certain complications to the classification stage, such as variations in illumination, motion noise, and environmental variations, even in a controlled environment. Concerning the grading algorithms, there is a great diversity of approaches to classifying lemons. Some use classical approaches (extracting features that feed a classifier), while others use methods based on deep learning (where feature extraction is not required). Both approaches are valid, and the authors of the related research highlight their advantages and disadvantages.

Recent studies have explored various computer vision and deep learning approaches for grading and defect detection in lemons and citrus fruits. Early works [5,6,7], focused on traditional image processing and regression-based systems for volume and size estimation, achieving accuracies above 98%. More recent research has shifted toward deep learning architectures, including YOLO variants, ResNet, VGG, and ANNs, for the automated classification of lemons based on quality, ripeness, size, and defects. For instance, ref. [8,9,10] obtained accuracies of 92%, 93.14% and 96.6%, respectively, while [11] reported a 100% accuracy in defect detection using CNNs. Systems such as YOLOv4 [12] and YOLOv5s [13] have improved object detection and grading performance by integrating real-time processing and feature fusion techniques.

Regarding citrus quality [14] and defect detection [15], VGG16, VGG19, and ResNet50 models are used to detect citrus defects, achieving up to 96% accuracy with ResNet50 and Softmax classifiers. Other researchers have proposed embedded and robotic implementations—for example, ref. [16] reports that lightweight CNN-based classifiers achieve 94% accuracy, while [17,18] integrate robotic arms and LSTM-based predictors for automated sorting. Recent hybrid architectures, such as Stacked Auto-Encoders with CNNs [19], have reached nearly 99% accuracy. Overall, the literature demonstrates a clear trend toward combining deep learning, real-time vision, and robotic systems to achieve high-accuracy, automated, and scalable solutions for lemon grading. Related to the implementation of citrus classification on SBC, there are CNN-based approaches [20] and a lightweight CNN-based approach called Sortnet [21].

In this project, we designed an embedded vision system adapted to work with an operating (old) mechanical lemon sorter. The problem with the current sorter is that it is not possible to obtain the necessary spare parts to keep its original vision system (which has been in operation for more than 20 years) because they have been discontinued or are not in stock, which causes an increase in operating costs due to the downtime of the sorter. Still, a PC running Windows 2000 and the manufacturer’s citrus sorting software controls the machine. We implemented a lightweight machine-learning-based vision system on an SBC. To train this machine learning algorithm, we developed image processing algorithms to extract the required classification features, including a small model to locate the lemon’s peduncles. We also implemented a queuing algorithm on a microcontroller to determine the timing required to extract the lemon based on the classification.

The main contributions of this article are:

• A lightweight lemon classification model (LCM) by size implemented on an SBC in real time.
• A peduncle detection model (PDM) whose results generate a feature that is useful for the lemon size classification algorithm. These features, in combination with other geometric features, are used to train the LCM. The inference time for the lemon peduncle detection is low enough to be executed in real time as part of the feature extraction process.
• An image processing algorithm that extracts features from lemon images, which a trained model evaluates.
• The queuing algorithm is executable on an 8-bit microcontroller, which manages the activation times of the ejector solenoids that send the lemons to the appropriate chutes.
• The integration of the above modules into a lemon classification system in an existing industrial lemon sorting system. Due to its modularity, this system is easy to maintain and cost-effective.

This paper is structured as follows. In Section 2, we describe the material and methods required to sort lemons by size. In Section 3, we present and discuss the results obtained and analyze the advantages of the proposed system. Finally, in Section 4, we conclude and provide directions for future work.

2. Materials and Methods

2.1. Analysis of Existing Lemon Classifier Machine

After visiting the sorting machine for which we will propose an improvement, we analyzed its components to carry out a modification proposal. The original sorter has eight roller that transport the lemons at high speed; on the roller, there is a camera that is in charge of acquiring the image; underneath the image, at different distances from each other, there are the people in charge of transferring the lemons to the next roller. The solenoids transfer the lemons to the next roller based on the classification assigned by that vision system. Figure 1 shows the main components of the industrial sorting machine. Figure 1a–f describe each component.

The photoelectric sensor coordinates the components since each pulse sent when detecting a citrus fruit sends a pulse to the camera to capture a citrus image, keeping a count to know the activation order and offset of the solenoids. A PC coordinates the latter since it requires both processing and memory in conjunction with the control cards, which define the solenoid activation in the proper order. A power management stage is needed between the machine components since the solenoid works with 24 V and the photoelectric sensor works with 10 V. The machine allows the operator to adjust the parameters through a computer interface, which ensures that all the belts sort the citrus similarly. The operator changes the sorter parameters to adapt to the lemon shapes, which can be either more elongated or rounded.

2.2. Hardware and Components

The hardware components are:

• Single Board Computer (Raspberry Pi 4 model B+, Raspberry Pi Holdings plc, South Wales, United Kingdom).
• Microcontroller (Arduino Mega 2560, Arduino AG, Monza, Italy).
• Camera (Model: HT-SUA134GC-T1V Huateng USB 3.0, Huateng Vision, Shenzhen, China).
• Mosfet-based solenoid driver board.
• Opto-isolation and attenuation module.
• Led lighting ring.

Figure 2 shows the block diagram of the new vision system coupled to the industrial citrus sorting machine. The SBC and the microcontroller receive, through an opto-isolation and attenuation module, the pulses generated by a photoelectric sensor coupled to the conveyor chain. At each pulse, the SBC requests a frame from the USB camera, and when the frame is ready, the SBC processes the received frame to obtain the class to which the observed lemon belongs. The SBC sends the lemon class and the corresponding number of pulses to wait to the microcontroller via an SPI interface through a logic level converter. The microcontroller receives the information from the SBC and queues the pulses to wait, depending on the class to which the lemon observed by the camera corresponds. Once the counter of pulses to wait reaches zero, the microcontroller sends activation pulses to the solenoids via the solenoid driver, which activates the ejectors located at each chute of the sorting machine.

A roller conveyor transports the lemons, which pass under the camera where the ring light illuminates the fruit. A photoelectric sensor produces pulses synchronized with the position of the pockets on which the lemons travel. With each pulse, the vision system captures an image of the lemon, analyzes it, determines the class to which it belongs, and sends the information to the microcontroller to activate the solenoids.

The software components are:

• The SBC hosts the software that captures the images of lemons from the camera, processes those images, and determines the size to which the citrus fruit belongs. Based on the generated size, it sends commands to the microcontroller to activate the solenoids responsible for separating the lemons. Another feature of this software is that it receives configuration parameters to perform various classifications of lemons according to the user’s requirements.
• The microcontroller hosts the software routines that receive input signals from the OBC and send signals to the solenoids via a signal amplifier board to separate the lemon fruits.

2.3. Lemon Images Dataset

Figure 3 shows a mechanical lemon-size gauge, which is a caliper specifically designed to measure lemon size according to citrus industry standards. The device has adjustable jaws that measure the minor axis diameter of the lemon. This diameter determines the fruit size code (“count”), which indicates how many lemons fit into a standard 40-pound box. We can highlight that, for some lemon sizes, the available measurement ranges overlap, which can lead to classification errors, and that this is often subject to user interpretation.

To generate the lemon dataset, we captured the images using the camera connected to the SBC. We turned on the sorting machine and passed a set of previously graded lemon fruits to simulate the on-field operating conditions of the sorting machine. Once we obtained the lemon image dataset, we applied threshold- and contour-based image processing to detect the features used for training the classifiers. We generated a dataset of 3127 lemon images that includes six variants in lemon size. We adhered to the lemon industry standards, which specify a numbering system to indicate lemon size, ensuring uniformity, efficiency, and transparency throughout the supply chain. Each size number corresponds to the approximate number of lemons that fit into a standard 40-pound (18-kg) box. The lemon sizes included in the generated dataset are 75, 95, 115, 140, 165, and 200.

Table 1. Lemon image dataset.

Lemon Size Class Label	Number of Images
75	545
95	535
115	361
140	624
165	519
200	543
Total	3127

provides information on the number of images for each variant as well as the total number of lemon images in the dataset. Figure 4 shows selected images from the lemon dataset generated in this article, which correspond to the previously mentioned industry-standard sizes.

2.4. FOMO-Based Peduncle Position Estimation

Based on the images of lemons in the dataset, we generated a second dataset that labeled the Peduncle region in each lemon image. We do the peduncles labeling in the EdgeImpulse platform, which streamlines the development, deployment, and scaling of machine learning models for edge and embedded (TinyML) systems [22]. Figure 5 shows selected input images with the labeled peduncle. The Edge Impulse platform provides a comprehensive workflow that encompasses data acquisition, preprocessing, training/optimization, evaluation, and deployment. It includes several object detection models that users can generate and customize using their provided images. The FOMO model is best suited for applications that require real-time object detection on low-power or resource-limited devices, such as microcontrollers or embedded systems, where models like MobileNet or YOLO are too computationally intensive to run efficiently. Unlike YOLO, which provides full bounding boxes and high accuracy but demands significant processing power, or MobileNet, which is mainly for classification tasks, FOMO offers a middle ground by delivering lightweight, grid-based object localization with minimal memory and energy use, making ideal for edge AI tasks where speed, efficiency, and deployability on tiny hardware are more critical than achieving the highest precision. Based on the results, we choose the FOMO model, which is an object detection algorithm that divides the image into a grid of smaller cells and predicts a class label (or no object) for each cell [23]. The FOMO model enables real-time detection of multiple objects while using less memory and processing power than other models. It uses a compact CNN backbone (such as MobileNetV2 or EfficientNet-Lite) to produce a feature map.

Once we set the peduncle labels for the input lemon images and the model type in the EdgeImpulse platform, we generate a float32-precision TensorFlow Lite (.tflite) model that can be used on any PC to estimate peduncle localization. We use this model to estimate the previously described peduncle/radius ratio feature.

2.5. Feature Extraction for Size-Based Lemon Classification

Figure 6 illustrates a block diagram outlining the operation of the image processing routines for extracting features from lemon images. The roller conveyor starts operating and acquires a lemon image at each pulse. To obtain the features for the classification algorithm, we apply the following image processing algorithm to each RGB captured image. As the first step, we convert the image from the RGB color space to the CIELab color space. We observed that in the chromaticity channels, a and b, of this color space, the citrus color information appeared well segmented from the background. We also observed that in channel a, the pixels corresponding to green lemons were better segmented, and that in channel b, the pixels corresponding to the fruit for yellow lemons were better segmented than in channel a. Based on these observations, we decided to process both channels, a and b, to produce the binary mask from which we extracted the classification features. After converting the RGB image to CIELab space, we applied a median filter to smooth the lemon surface in both channels. Next, we thresholded the filtered images using the Otsu algorithm and detected contours in the resulting binary images. We then filter the contours by area, keeping the one with the largest area and discarding the others. The latter operation isolates the binary mask corresponding to the lemon, and we then use a bitwise OR operator to produce the final binary image. Using the previously computed lemon contour, we extract the following features:

• Contour area (in pixels).
• Contour perimeter (in pixels).
• Length of major axis LMaA (in pixels).
• Length of minor axis LMiA (in pixels).
• LMaA/LMiA ratio.
• Peduncle over major axis ratio. To estimate this ratio, the software detects the lemon peduncle and computes the distance to the centroid of the binary images. Then, the software computes the ratio of this distance and one half of the major axis.

The selected features allows us to characterize the actual fruit body and grade the lemon based on industry-standard size categories (e.g., 75, 95, 115 mm).

2.6. Queuing Algorithm to Activate the Ejector Solenoids

We propose a queuing algorithm that enables us to parallelize solenoid activation on a sequential device. We implemented this algorithm on a 8-bit microcontroller, which performs the operations described next. The queuing algorithm begins by initializing the global pulse counter and the pulse output variable to 0. Next, we configure the pulse output port to generate parallel output. In the next step, we initialize the matrix storing the queue data for each chute to the maximum value for the type of variable used to store each element of the matrix (in this case, uint8). We defined the matrix size as the number of chutes, d, and the number of pockets, p, along the roller conveyor line between the camera position and the last chute. In our case, we had 10 chutes and 78 pockets. The next step in the algorithm is to initialize a vector of length d, which contains the values to write to the output port when the backwards pulse counting reaches 0. We then configure the hardware interrupts, including the pin number and the interrupt service routine (ISR) to execute for each event. Before entering an infinite loop, we define a variable to store the duration of the positive pulse generated by the outputs and initialize a flag that will be set to true when the interrupt pin receives an external pulse. Inside the infinite loop, we poll the SBC for SPI data that contains the chute number and the number of pulses to wait before writing a high state to the output. If we receive new data, we push the number of pulses to wait before activating the outputs into the queue index corresponding to the chute number. We also register the current number of elements in the queue in a vector, using the chute number as the index. The algorithm verifies two conditions: the elapsed time since the last output activation is greater than or equal to the pulse width, and it also sets the output activation flag to true. If the algorithm finds both conditions true, we write 0 to the whole output port and set the output activation flag to false. Otherwise, the algorithm returns to the beginning of the infinite loop.

On the other hand, the microcontroller executes an ISR when receiving an external pulse. This routine begins by subtracting one from every element of the queue matrix. Next, the algorithm enters a for loop executed for each chute index. If the current component of the first row of the queue matrix is 0, we shift the elements of the column corresponding to the current chute index up one row in the queue matrix. We then update the number of elements in the queue for the current chute index, set the pulse output variable to high for that chute index, and write it to the output ports. Finally, we update the pulse output flag to true and return to the beginning of the for loop. The output flag controls the width of the high pulse generated by the microcontroller. Figure 7 illustrates the flow diagram for the complete queuing algorithm previously described.

Figure 8 illustrates how the queuing algorithm works for a hypothetical eight-pulse case in which the classification system detects lemons at each pulse. We can highlight that each chute is assigned a specific number of pulses, which increases proportionally to the chute’s distance from the camera. Therefore, the number of pulses to queue will always be greater than the number of pulses already queued. In this example, we have assigned chutes $C_1$, $C_2$ …$C_c$ values 1, 2, …, c, respectively, which represent the number of pulses to wait before solenoid activation. In the initial state of the queue matrix (Figure 8a), the algorithm initializes all its elements to the maximum value the variable can store (255 in this case, as its type is uint8). The dimension of the matrix is $p\times c$, where p is the number of pockets (space between adjacent rollers of the conveyor), and c is the number of chutes (outputs to the belt conveyors) used in the sorting machine. Also, the vector S contains the state of the variable to output through the microcontroller port. The initial state of each vector element is 0. Figure 8b shows that at the first external pulse, the SBC indicates to the microcontroller that the fruit it has processed must exit through chute $C_3$, thus three pulses are queued in the matrix in the first available cell corresponding to the referenced chute. The queued numbers will decrease by one at each pulse (if they are greater than 0), emulating a countdown before activating the corresponding solenoid. Figure 8c shows that at the second external pulse, the SBC indicates to the microcontroller that the processed fruit has to exit through chute $C_2$. Once the microcontroller queues the number of pulses to wait in the corresponding chute column, the already-queued pulses are decremented by 1, so the number of pulses to wait in chute $C_3$ changes to 2. In the third external pulse (Figure 8d), the SBC signals the microcontroller to send the processed fruit through chute $C_3$. Since a previous element was in the queue, the microcontroller places the new number of pulses to wait in the second row of the corresponding chute column. In the fourth pulse (Figure 8e), the countdown of the elements in the first row for the chutes $C_2$ and $C_3$ reaches 0, hence the state of the output vector, S, changes to 1 for the second and third elements, activating in that moment the corresponding solenoids. This state remains high for a specified period of time, then the microcontroller resets the state to 0. After activating the solenoids, the microcontroller shifts the queued elements up by one. In the fifth pulse (Figure 8f), the SBC indicates the microcontroller to queue one pulse for chute $C_1$, and the already queued elements are decreased by one. Figure 8g illustrates that at the sixth pulse, the SBC indicates to the microcontroller to queue four pulses for the chute $C_4$. At the same time, the countdown for the first three chutes reaches 0, causing the microcontroller to set the corresponding cells of vector S to 1 and activate the attached solenoids by setting the corresponding output pins high. Before the end of the external pulse, the microcontroller sets the output vector elements to 0. In the seventh pulse (Figure 8h), the SBC indicates the microcontroller to queue one pulse for chute $C_1$, while decreasing the queued element in the fourth column by one. In the eighth pulse (Figure 8i), the SBC sends the microcontroller a command to queue one pulse for chute $C_1$. The microcontroller also decrements the queued elements by 1, thus setting them to 2 for chute $C_4$ and 0 for chute $C_1$, activating the corresponding output pin via vector S.

3. Results

Table 2. Classification results for different percentages of samples (0, 20, and 50) added to the minority class by the SMOTE algorithm. We obtained accuracy results for random forest (RF), K-nearest neighbors (KNN), support vector machines (SVM), gradient boosting (GB), decision trees (DT), and multilayer perceptron (MLP) classifiers.

Classifier	SMOTE %	Accuracy Without Penduncle over Major Axis Ratio	Accuracy with Penduncle over Major Axis Ratio
RF	0	0.8642	0.8706
	20	0.8610	0.8722
	50	0.8610	0.8722
KNN	0	0.8402	0.8382
	20	0.8498	0.8530
	50	0.8498	0.8530
SVM	0	0.8562	0.8530
	20	0.8610	0.8530
	50	0.8610	0.8530
DT	0	0.8514	0.8466
	20	0.8563	0.8544
	50	0.8563	0.8544
GB	0	0.8562	0.8594
	20	0.8646	0.8695
	50	0.8646	0.8695
MLP	0	0.8594	0.8546
	20	0.8691	0.8699
	50	0.8691	0.8699

summarizes the results of the addition of minority samples by the SMOTE algorithm for the selected classifiers. We performed a comparative analysis demonstrating that SMOTE augmentation improves baseline performance across most architectures. Still, we also observe that increasing synthetic samples from 20% to 50% yields no additional gain in accuracy. The inclusion of the peduncle-over-major-axis-ratio feature enhances ensemble-based methods, such as the Random Forest classifier, which achieved an accuracy of 87.22%. In contrast, it introduces noise that degrades the performance of SVM and Decision Trees. Consequently, the combination of Random Forest with the additional morphological feature and moderate data balancing constitutes the most robust modeling strategy for this dataset.

3.1. Color Space Selection

We selected the CIELab color space because Otsu’s thresholding algorithm produced binary images that required less post-processing to extract the region of interest corresponding to a citrus fruit. In addition, we found that the yellow and green color information characteristic of lemons appeared more complete in the A and B color channels of this color space than in other color spaces (HSV, YUV, and XYZ). Figure 9 shows the thresholding of the channels of the HSV, YUZ, XYZ, and CIELab color spaces, where we can observe that the thresholding of the A and B channels of the CIELab color space yields cleaner binary images.

3.2. Peduncle Detection

To generate a model that enables us to detect the peduncle location, we trained four models on the Edge Impulse platform. We choose the FOMO model, which has proven its reliability in segmentation tasks.

Table 3. Comparison of FOMO models to detect lemon peduncles.

Model	Img_ Width	Img_ Height	Resize_ Mode	Col_dep	32-Bit Accuracies		8-Bit Accuracies
					Validation	Test	Validation	Test
1	224	224	fit-long	Grayscale	88.6%	61.8%	87.2%	63.2%
2	96	96	squash	Grayscale	89.1%	78.8%	86.9%	78.8%
3	96	96	fit-short	RGB	87.8%	75.0%	86.7%	75.0%
4	224	224	squash	RGB	89.7%	73.8%	88.0%	70.0%

presents the parameters of the generated models and the accuracy results obtained. We integrated the best-performing model into the feature-detection algorithm to estimate the peduncle/radius ratio, thereby enabling more accurate classification of lemons by size.

Regarding the trained models, we carried out tests with two image sizes (96 × 96 and 224 × 224). For each image size, we use the grayscale or the RGB version. The table reports accuracies for the 32-bit (float-32) and 8-bit (Int8) trained models. We also test three image-resizing methods: fit-long, fit-short, and squash (interpolation). Among the trained models, the best was the one that used an input image with dimensions 224 × 224 in RGB format, achieving the highest accuracy in both vectorized and non-vectorized versions, regardless of the data type (float32 or int8).

3.3. Dataset Balancing Strategy

We can infer from the class counter shown in Table 1 that the dataset is imbalanced. To mitigate class imbalance, we used the SMOTE (Synthetic Minority Over-sampling Technique) algorithm, which addresses class imbalance in machine learning by generating synthetic samples by interpolating existing minority examples and their nearest neighbors [24]. This process effectively increases the representation of the minority class, reducing bias toward the majority class in classifiers and improving model generalization on imbalanced datasets.

Regarding the key parameters of the SMOTE algorithm, for the sampling_strategy parameter, we use the “minority” option to balance all minority classes to the number of samples in the majority class. For the random_state parameter, we set it randomly (42) to ensure consistent results across multiple runs when generating synthetic samples. We did not specify the remaining parameters of the SMOTE algorithm.

Empirically, we conducted three tests to add a percentage of the minority class. We used 0%, 20%, and 50% as the percentages, and evaluated the accuracy of the classifiers we used. The tests confirm that the best-performing classifiers are those where we added 50% of the minority class samples.

3.4. Evaluation of Classifiers

To train the selected classifiers, we divided the dataset into 80% for training and 20% for testing. From the test set, we performed an analysis of the dataset to balance the data using the majority class count criterion, testing without adding synthetic samples (0%) and adding 20% and 50% of the minority class sample count, respectively, as synthetic samples to integrate the new dataset, using the SMOTE algorithm.

We run the GridSearchCV algorithm to find the optimal parameters for the lemon image classifiers that maximize accuracy, thereby achieving the best possible performance. The Grid Search algorithm is an optimization method that performs an exhaustive parameter search [25]. It defines a grid (set) of possible values for each hyper-parameter, then trains and evaluates the model on every possible combination of those values. For each combination, the cross-validation scheme evaluates the model’s performance using a combination that yields the best performance metric.

Table 4. Used classifiers and the best parameters found by the GridSearchCV algorithm.

Classifier	Best Parameters	SMOTE %	Improves with Penduncle-Based Attribute	CV Score	Accuracy
RF	‘max_depth’: 10, ‘max_features’: ‘sqrt’, ‘min_samples_leaf’: 2, ‘n_estimators’: 200	50%	YES	0.8778	0.8722
KNN	n_neighbors: 9, p: 1, weights: ‘distance’	50%	YES	0.8498	0.8530
SVM	‘C’: 5.0, ‘gamma’: ‘auto’, ‘kernel’: ‘rbf’	50%	NO	0.8646	0.8610
DT	‘criterion’: ‘gini’, ‘max_depth’: None, ‘min_samples_leaf’: 16	50%	NO	0.8544	0.8563
GB	‘learning_rate’: 0.1, ‘max_depth’: 5, ‘n_estimators’: 100, ‘subsample’: 0.8	50%	NO	0.8711	0.8646
MLP	‘activation’: ‘relu’, ‘alpha’: 0.01, ‘hidden_layer_sizes’: (50, 50, 50, 50, 50, 50, 50), ‘learning_rate’: ‘constant’, ‘solver’: ‘adam’	50%	YES	0.8684	0.8699

displays the classifiers used, along with the parameters obtained by the GridSearchCV algorithm after its execution.

Table 5. Detailed classification results for the trained classifiers, measuring per-class and micro, macro, and weighted average, precision, recall, and F1-score.

Classifier	Class	Precision	Recall	F1-Score
KNN	75	0.933962	0.908257	0.920930
	95	0.864078	0.831776	0.847619
	115	0.738636	0.902778	0.812500
	140	0.875000	0.840000	0.857143
	165	0.845238	0.682692	0.755319
	200	0.824000	0.944954	0.880342
	macro avg	0.846819	0.851743	0.845642
	weighted avg	0.853891	0.849840	0.848610
	micro avg	0.849840	0.849840	0.849840
SVM	75	0.944444	0.935780	0.940092
	95	0.916667	0.822430	0.866995
	115	0.733333	0.916667	0.814815
	140	0.884298	0.856000	0.869919
	165	0.823529	0.673077	0.740741
	200	0.801587	0.926606	0.859574
	macro avg	0.850643	0.855093	0.848689
	weighted avg	0.858443	0.853035	0.852038
	micro avg	0.853035	0.853035	0.853035
Decision Tree	75	0.933333	0.899083	0.915888
	95	0.855769	0.831776	0.843602
	115	0.752941	0.888889	0.815287
	140	0.866142	0.880000	0.873016
	165	0.857143	0.634615	0.729282
	200	0.804688	0.944954	0.869198
	macro avg	0.845003	0.846553	0.841045
	weighted avg	0.850853	0.846645	0.844270
	micro avg	0.846645	0.846645	0.846645
Gradient Boosting	75	0.952381	0.917431	0.934579
	95	0.865385	0.841121	0.853081
	115	0.732558	0.875000	0.797468
	140	0.876033	0.848000	0.861789
	165	0.860759	0.653846	0.743169
	200	0.801527	0.963303	0.875000
	macro avg	0.848107	0.849784	0.844181
	weighted avg	0.855494	0.849840	0.848170
	micro avg	0.849840	0.849840	0.849840
MLP	75	0.968750	0.853211	0.907317
	95	0.836364	0.859813	0.847926
	115	0.720430	0.930556	0.812121
	140	0.875000	0.840000	0.857143
	165	0.835443	0.634615	0.721311
	200	0.796875	0.935780	0.860759
	macro avg	0.838810	0.842329	0.834430
	weighted avg	0.846767	0.838658	0.837189
	micro avg	0.838658	0.838658	0.838658
Random Forest	75	0.944444	0.935780	0.940092
	95	0.909091	0.841121	0.873786
	115	0.767442	0.916667	0.835443
	140	0.899160	0.856000	0.877049
	165	0.857143	0.692308	0.765957
	200	0.807692	0.963303	0.878661
	macro avg	0.864162	0.867530	0.861832
	weighted avg	0.870686	0.865815	0.864508
	micro avg	0.865814	0.865814	0.865814

compares six classifiers (KNN, SVM, Decision Tree, Gradient Boosting, MLP, and Random Forest), and the metrics exhibit consistent patterns that highlight their strengths and limitations in multiclass classification. Random Forest shows the highest micro-average (0.8658), indicating the best overall performance when weighting all samples equally, and it also achieves the strongest performance across most individual classes, particularly classes 3 and 5, where recall is exceptionally high. SVM and Gradient Boosting follow closely with micro averages of 0.8530 and 0.8498, respectively; both maintain balanced precision–recall trade-offs across classes, though SVM struggles slightly on class 4 and Boosting on classes 2 and 4. KNN and Decision Tree show similar mid-level performance (micro-averages around 0.8498 and 0.8466), with KNN slightly better for classes 0 and 5. At the same time, the Decision Tree shows greater inconsistency, with drops in class 4 recall. MLP exhibits the lowest micro-average (0.8387), mainly due to weaker performance on class 4 and somewhat unstable precision patterns, despite strong recall for classes 2 and 5. Overall, Random Forest consistently delivers the most reliable and stable classification across all six classes.

Across the six classes, performance varies notably, with some classes being easier to classify than others. Class 0 and class 3 show consistently high precision, recall, and F1-scores across all models, with SVM, Gradient Boosting, and especially Random Forest achieving the best balance. Classes 1 and 2 exhibit moderate difficulty: Random Forest and SVM lead due to their strong recall and balanced metrics, while other models show greater variability. Class 4 is clearly the most challenging, with all classifiers achieving low recall, though Random Forest remains the top performer. In contrast, class 5 is the easiest to identify, with uniformly high recall across all models and the best F1 scores achieved by Random Forest and Gradient Boosting. Overall, Random Forest consistently provides the strongest per-class performance, followed by SVM and Gradient Boosting, while KNN, Decision Tree, and MLP show more inconsistent behavior across classes.

Although all classifiers exhibit reasonably strong performance across the six classes, Random Forest emerges as the most effective model, achieving the highest micro-average and balanced precision–recall scores across nearly every class. SVM and Gradient Boosting also perform competitively, offering strong generalization with only minor weaknesses in specific classes. KNN and Decision Tree produce acceptable results but are more variable, showing sensitivity to class imbalance and class-specific complexity. MLP, while generally competent, yields the lowest macro- and micro-average scores due to inconsistent class-level behavior. Overall, ensemble-based methods—primarily Random Forest—provide the most robust and consistent performance in this multiclass scenario.

To show the performance of the size classifiers on the lemon images, we show a confusion matrix for each classifier. The confusion matrices illustrate the model performance across six classes labeled 75, 95, 115, 140, 165, and 200, with the diagonal values representing correct predictions. Figure 10 shows the confusion matrices for the selected classifiers.

For the KNN classifier (Figure 10a), most samples were correctly classified, with the highest accuracy observed for classes 75 (100 correct), 140 (106 correct), and 200 (102 correct). Some confusion occurs between neighboring classes, specifically between class 115 (4 misclassified cases), class 165 (32 misclassified cases), and class 200 (7 misclassified cases).

The SVM classifier predicted correctly 102 samples of class 75, 89 of class 95, 68 of class 115, 105 of class 140, 69 of class 165, and 101 of class 200. However, there are minor misclassifications: class 95 (18 cases), class 115 (4 cases), class 140 (20 cases), class 165 (35 cases) and 200 (8 cases) (Figure 10b).

For the decision tree classifier (Figure 10c), the classes with the highest error are for classes 200 (25 samples classified as 165) and 115 (15 samples classified as 95 and 11 samples classified as 140). In the case of classes 75 and 165, the classification error is lower, with only 17 samples classified as 95. Additionally, in the case of class 165, nine samples were classified as 140 and five as 200.

The Gradient Boost classifier (Figure 10d) performs well for most classes, especially with classes 75 (99 correct out of 106), 95 (92/102), 140 (107/122), and 200 (103/130), showing strong accuracy and consistent predictions. However, there is moderate confusion between classes 115 and 140. For class 115, 8 were classified as 95 and 10 were classified as 140. for class 140, 5 were classified as 115, and 10 were classified as 165.

For the MLP classifier (Figure 10e), the classes with the highest error are for classes 165 (8 samples classified as 140, and 26 classified as 200) and 95 (11 samples classified as 115 and 9 samples classified as 75). In the case of class 75, the classification error is lower, with only five samples classified as 95. Additionally, in the case of class 200, eight samples were classified as 165.

The RF classifier (Figure 10f) correctly classified most samples, with the highest accuracy observed for classes 75 (102 correct), 115 (66 correct), and 200 (104 correct). Some confusion occurs between neighboring classes, specifically class 95 (five samples classified as class 75, and eight samples classified as class 115), class 115 (one sample classified as class 95 and five samples classified as class 140), class 165 (eight samples classified as class 140 and 24 samples classified as class 200).

All classifiers agree on the most accurate classes, which are 75, 95, and 200. In addition, the highest number of misclassified cases is between classes 115 (KNN) and 165 (SVM, DT, GB, RF). We can also observe that the mislabeled samples belong to contiguous classes in the feature space, so it is possible to confuse a lemon from case 75 with one from 95, but not with one from 115 or 140. Overall, the matrices reflect strong classification accuracy, with moderate confusion among adjacent or visually identical categories, which is typical for the tested classifiers, as there is slight overlap in class boundaries in feature space.

Regarding the CV score obtained by the GridSearchCV algorithm to find the classifier’s optimal parameters, the RF classifier achieved the best score (0.8778). The CV score measures a model’s generalization performance by averaging the scores across all cross-validation folds for a given hyper-parameter combination. Regarding the classifiers’ accuracy, the best was the RF classifier, which achieved an accuracy of 0.8722. The accuracy obtained by the best classifier (RF) does not exceed 90%. This accuracy is due to the similarity between the dimensions of the geometric characteristics extracted for lemons in adjacent categories, as shown by the calibrator in Figure 3.

3.5. Execution Times and Memory Usage Estimation

To estimate the inference time of the selected classification model, we conducted a test in which the classification machine continuously classified 11,660 unlabeled lemons. During this test, we recorded the average inference time (13.69 ms) and the minimum and maximum times (6.60 ms and 87.09 ms, respectively). On the other hand, we measured the time between the photoelectric sensor pulse and the microcontroller’s solenoid activation signal. For the single sample shown in Figure 11, we measured a 72 ms delay. It is worth noting that for this test, we let the microcontroller shift the pulses in the queuing matrix, but forced the write to the output port so that the microcontroller generated the solenoid activation pulse in the current cycle. This time is short enough to process citrus fruits in real time, achieving an average of 14 citrus fruits per second, which is less than the time required to process lemons at the machine’s current acquisition speed (9 lemons per second). Regarding memory usage, we estimated memory consumption during operation of the proposed classification system on the SBC using Python’s 3.12.2 MemoryProfiler library (version 0.61.0). The results show an average memory usage of 153 MB.

3.6. On-Field Test of the Vision System

Figure 12 shows the location of the developed machine vision module during the on-field test and lemon samples used for the classification test. We performed the test in two stages. In a first stage, we used a group of lemons previously sorted by the legacy machine to avoid manual measurement, since this is very time-consuming, and we had limited access time to the industrial sorter. For this test, we used 1558 lemons distributed in five classes, as follows: 175 for class 95, 241 for class 115, 308 for class 140, 390 for class 165, and 444 for class 200. Figure 13 shows the confusion matrix of this live run test. We observe that the accuracy is lower (78% accuracy) than those obtained during training (87% accuracy). The reason for this increase in error is that, in our model, we are not accounting for the classification error introduced by the legacy sorting machine.

The results show that class 95 is the most stable, with a precision and recall of 84.0%. The model confused 26 actual class 95 samples, treating them as 115, and accepted 27 class 115 samples as class 95. Thus, the model distinguishes this class well from the rest (140, 165, 200), but the boundary with class 115 is not clear. The model achieved 82.1% precision and 79.7% recall for class 115. This class confuses samples between classes 95 and 140. After the 95-class, this is the class with the fewest samples (241), which could affect the model’s ability to define more precise limits. Class 140 is where the model produces the most false negatives. It has a precision of 80.6%, but a recall of 67.5%, losing 32.5% of the actual data for this class. Out of 308 cases, the model wrongly classified 83 samples as class 165. The latter could be because features from class 140 are similar to those of class 165, and the model tends to predict the higher class when in doubt. Class 165 has the lowest precision (69.6%) and recall (68.7%), resulting in significant false positives. The low precision could be due to errors propagated from class 140 and to the model sending 92 cases to the next class (200). The class 200 has a precision of 80.4% and the highest recall (91.7%). The precision for this class drops slightly because it is receiving false predictions from class 165 (92 cases). Finally, we observe that the model shows a systematic bias towards predicting a higher class than the actual one, causing the high false negatives in classes 115 and 140.

In a second stage, let the system run for 2 h over non-labeled lemons. In this test, the vision system sorted 11,756 fruits with no downtime or operator intervention. In this test, we observed that the sorting machine failed to sort 96 lemons, sending them to the return conveyor. In 58 of these unsorted lemons, the images contained stacked lemons, producing deformed binary images. In the remaining cases, we observed a lack of ejectors in the pouches transporting the fruits. We will conduct the comparison with the legacy vision system in follow-up work.

4. Conclusions and Future Work

In this work, we proposed an easy-to-maintain, cost-effective machine vision system and the hardware modules for its integration into an existing industrial lemon sorting machine to perform real-time lemon classification. The vision module of this system is based on a lightweight LCM by size implemented on an SBC. We trained the LCM using standard morphological features and a feature based on the PDM-detected peduncle position, which we also trained. Based on these algorithms, the SBC classifies the fruits by size and sends the classification results to an 8-bit microcontroller, where we implemented a queuing algorithm that manages the activation times of the ejector solenoids that send the lemons to the appropriate chutes. The system developed is easily scalable due to the technology used and its low cost, which may be an advantage for its adoption in the regional citrus industry.

To generate the lemon dataset, we captured images of lemons using the camera connected to the SBC. We turned on the sorting machine and fed a set of previously graded lemons to simulate on-field operating conditions. We generated a dataset of 3127 lemon images with six variants in lemon size (75, 95, 115, 140, 165, and 200).

For the PDM, we used the FOMO model available in the Edge Impulse platform. We trained four models and selected the best, which we integrated into the feature-detection algorithm to estimate the peduncle/radius ratio. We found that using a 224 × 224 RGB image resolution yielded an accuracy of 89.7%, using the squash (interpolation) resize mode.

For the LCM, we run GridSearchCV to find the optimal parameters for the lemon image classifiers that maximize accuracy. We chose the following classifiers: K-Nearest Neighbors, Support Vector Machine, Decision Tree, Gradient Boosting, MLP Classifier, and Random Forest. The last classifier achieved the highest CVScore and accuracy (0.8775 and 0.8658, respectively). Based on these results, we extracted the morphological features (including the one based on the peduncle position) from the lemon image in the dataset. We used them to train an RF classifier and obtain the LCM.

We proposed a queuing algorithm adaptable to different configurations of industrial fruit-sorting machines, given the matrix-based design we used. This algorithm enabled us to parallelize solenoid activation on a sequential device.

Among the improvements we can make to this system is incorporating algorithms to classify fruits not only by size but also by ripeness level or defects, creating a more robust solution. The latter would require the acquisition of a new image dataset captured using the proposed system. We can improve the PDM and LCM models by retraining them on an extended lemon dataset, thereby enhancing lemon feature detection. One potential improvement is using a more powerful LED lamp, which would allow the camera to capture images with greater detail than the current configuration. We can explore using cameras across different spectral bands (infrared and ultraviolet) to detect defects with greater certainty. We will also replicate the manufactured module so it can operate on at least four more lines. Finally, it should be possible to add a communication module via wired communication to change the parameters of the fruits, and a remote software interface to facilitate the sorting machine configuration. The comparison with the legacy vision system will be conducted in follow-up work.

For a cost comparison, we provide an estimate of the materials cost for the proposed vision system, which is around 1000 dollars (600 dollars without the case). However, we do not have an estimate of the cost of a machine vision system similar to the legacy system. We do have a quotation for a simple camera-processor system from Keyence, which was 4200.00 dollars in 2020, not including the electronics to interact with the actuators or a case. Furthermore, the latter kind of systems is closed, and adapting them to specific needs is not possible. After asking Google Gemini for information on the cost of industrial vision systems, we found that entry-level systems cost around $ 15,000 per lane.