GL-SeqNet: Global–Local Fusion for Intra-Cluster Tomato Harvesting Sequence Optimization

Abstract

Tomato harvesting in protected horticulture is a critical task that faces challenges due to labor shortages, high environmental stress in greenhouses, and the complex nature of clustered fruit arrangements. This study proposes the global–local sequence ranking network (GL-SeqNet), a novel model designed to optimize intra-cluster tomato harvesting sequences by integrating global and local features using a deep learning approach. GL-SeqNet fuses the global structural information of tomato clusters with local fruit attributes to dynamically update the harvesting sequence, ensuring the optimal target is selected at each step. The model features a dual-stream architecture, comprising separate global and local backbones, and utilizes a ranking head for prioritizing intra-cluster targets. An AI-based image object-removal tool was used to simulate the dynamic structural changes within a cluster during harvesting, facilitating the creation of a state evolution dataset. The experimental results showed that the best overall performance was achieved with a global resolution of 112 × 112 and a local resolution of 56 × 56, yielding a Top-1 accuracy of 0.950, a position match rate (PMR) of 0.970, and an inference time of only 22.6 ms, along with faster convergence. The results underscore the potential of global and local fusion strategies and ranking-based learning for effective harvesting sequence optimization.

1. Introduction

Tomatoes are a primary crop in protected horticulture, serving both the fresh market and the processing industry, and are widely cultivated worldwide [1,2,3]. However, the continuous decline in labor supply, coupled with accelerated population aging, has steadily increased the cost of manual harvesting [4]. In addition, the high temperature and humidity typical of greenhouse environments markedly intensify the difficulty of harvesting. Under the dual pressures of rising costs and challenging environmental conditions, harvesting practices are rapidly shifting from manual to mechanized and intelligent modes. Meanwhile, the large-scale and structured development of protected horticulture has provided a solid foundation for the transition of harvesting robots from research to practical application [5].

Mechanized harvesting not only helps alleviate labor shortages and ensure operational efficiency and consistent quality but also provides conditions for data-driven and standardized production management [6]. However, tomatoes are mostly distributed in clusters, where severe occlusion among fruits and limited operating space pose significant challenges for grasping and path planning [7,8]. One of the primary challenges lies in selecting intra-cluster harvesting targets and optimizing harvesting sequences. If the harvesting sequence is improperly designed, the manipulator is easily disturbed by neighboring fruits, leading to failure. Therefore, it is urgent to establish a method for stepwise optimal target selection and dynamic sequence updating, enabling the system to determine the best target at each harvesting step.

In recent years, convolutional neural networks (CNNs) have significantly enhanced the visual capabilities, covering tasks such as detection [9,10,11,12,13], pose estimation [14,15,16,17,18], and keypoint localization [19,20,21,22], especially in the field of harvesting robots. Zhao et al. [23] augmented YOLOv7 with a keypoint head and a mask head, and proposed the YOLOv7-hv model for cucumber pose keypoint detection and instance segmentation. Experimental results showed a keypoint detection OKS of 0.882, a segmentation mIoU of 93.8%, and an overall network speed of 43 FPS. Liu et al. [24] proposed SIAB and the Simpleformer, which were integrated with CNNs into CS-Net, achieving higher segmentation accuracy and faster inference than SOTA methods. Zhao et al. [25] proposed a lightweight end-to-end model, YOLO-GP, using Ghost Bottleneck and adding keypoint prediction to achieve joint detection of grape bunches and picking points. Experiments on the Grape-PP dataset showed 93.27% mAP for cluster detection and a picking-point distance error of less than 40 pixels. These studies demonstrate that CNNs are highly effective in extracting features from fruits and vegetables. However, their applications remain largely limited to the perception level, such as object pose estimation. They still lack sequence decision-making mechanisms specifically designed for harvesting tasks.

Accordingly, harvesting sequence optimization has been studied in two main directions: path or time-optimal planning, which focuses on global travel efficiency [26,27], and intra-cluster sequence planning, which focuses on local geometric constraints. For example, Lin et al. [28] employed YOLOX-S for tea bud detection and modeled the harvesting sequence as a traveling salesman problem (TSP), which was solved using a modified pointer network. Experiments demonstrated that the algorithm achieved a runtime of 1.69 ms for sequence planning of up to 100 tea buds, satisfying real-time requirements. Wang Xiaorong et al. [29] proposed a safflower filament harvesting sequence planning method based on 3D TSP and Ptr-Nets-AC, which demonstrated higher training efficiency on multi-scale datasets and shorter, faster paths during testing. Dai et al. [30] optimized tomato cluster harvesting sequences by combining density clustering with the shortest motion path, and selected the tomato closest to the end-effector within the cluster as the harvesting target. Greenhouse experiments achieved a collision-free success rate of 70.9% with an average operation cycle of 8.6 s. Although a considerable body of research has focused on path or time-optimal models, these approaches are not directly applicable to intra-cluster harvesting sequence optimization of clustered fruits. Moreover, existing intra-cluster sequence planning methods for clustered fruits often rely on heuristic or empirically defined rules, which suffer from limited robustness and adaptability.

This study focuses on optimizing the intra-cluster harvesting sequence for clustered fruits, emphasizing the integration of both cluster-level global structural information and fruit-specific local features in the decision-making process. In recent years, multimodal fusion methods based on convolutional neural networks have provided a feasible approach for unified modeling of such information. Huang et al. [31] proposed a multimodal fusion network, CornMFN, which employed ConvNeXt to extract image features and an MLP to represent meteorological variables, while cross-attention was used to fuse heterogeneous information. This approach achieved an accuracy of 91.19% for maize phenology classification. Liu et al. [32] developed a multimodal instance segmentation network, YOLACTFusion, which used dual ResNet-50 backbones to extract RGB and NIR features, and integrated them through parallel attention for multi-scale cross-modal fusion. On the tomato stem segmentation task, YOLACTFusion improved the mAP by 7.09% compared with YOLACT.

Based on the above, this paper proposes the GL-SeqNet model, a novel method for optimizing tomato harvesting sequences within clusters. The model leverages dual-channel features of cluster structure and individual fruit geometry, and employs deep learning to achieve stepwise optimal target selection with dynamic sequence updating after each grasp. After each simulated harvesting action, the remaining cluster structure is updated dynamically, allowing the model to continuously adjust the subsequent picking order. The main contributions of this study are as follows:

(1)

A novel GL-SeqNet model is proposed that integrates feature information from the entire tomato cluster and individual tomatoes to realize stepwise optimal target selection.

(2)

An AI-based image object-removal tool is introduced to simulate intra-cluster structural changes in tomatoes after each harvesting action, enabling the construction of a sequence state evolution dataset that better aligns training with real-world conditions.

(3)

A systematic comparison is conducted among RankNet, ListNet, and mean squared error (MSE) loss functions, as well as different input resolutions, to analyze their effects on ranking quality and system real-time performance.

2. Materials and Methods

2.1. Image Acquisition

Data collected under different regions and greenhouse conditions have a significant impact on the generalization performance of the model. To improve its applicability and robustness, sampling was conducted at two locations: the Yangdu Base in Jiaxing, Zhejiang Province (collected in 2022), and the Wuwangnong Farm in Hangzhou, Zhejiang Province (collected in 2024). The data acquisition devices included a Redmi K30 smartphone (Xiaomi Corporation, Beijing, China) and an Intel RealSense L515 camera (Intel Corporation, Santa Clara, CA, USA), with the original image resolutions being 1080 × 1920 and 1280 × 720, respectively. All images were stored in JPEG (Joint Photographic Experts Group) format. Figure 1a,b show representative tomato plant samples collected from the two sites.

2.2. Dataset Construction

In this study, a tomato cluster is defined as a group of adjacent fruits within an image. The dataset construction process is illustrated in Figure 2. First, the original images were manually segmented from the tomato clusters to obtain cluster images with complete geometric relationships preserved, while samples with severe blur, overexposure, or unclear boundaries were removed. Subsequently, the mature fruits within each cluster were also manually segmented. The annotations were specifically handled by a researcher with extensive practical experience in tomato-harvesting robots. Since the model input consists of one global image and a set of local images, the number of local images must remain consistent. In contrast, the number of mature fruits within a cluster varies. To address this, a black-image padding strategy was adopted, unifying the number of local images to five by adding pure black images. The choice of five images was based on the observation that the number of mature fruits in a single cluster rarely exceeded five. The annotation strategy was designed to simulate human decision-making preferences and to serve as the learning objective of the model. Sequence labels started from 1, while the padded black images were uniformly labeled as 6. To further approximate the state evolution in real harvesting processes, an AI-based image object-removal tool (FliFlik KleanOut for Photo v6.2.0) was used to remove the selected optimal harvesting target and fill the masked region with background content, thereby generating post-harvesting images. In this new state, the remaining mature fruits were re-annotated for sequence labeling, and this process was iterated until all mature fruits within the cluster were processed. This ensured that the training of GL-SeqNet better matched real harvesting scenarios.

A total of 200 tomato clusters were obtained and divided into training and validation sets with a ratio of 4:1, yielding 160 clusters for training and 40 clusters for validation.

2.3. GL-SeqNet Model

Currently, numerous algorithms have been developed for object classification [33,34], detection [35,36], segmentation [37,38], and keypoint regression [39,40], making significant contributions to visual perception. However, visual perception alone is insufficient for accurate decision-making in mechanized tomato harvesting. The lack of decision modeling often causes repeated obstructions of the manipulator in clustered fruit scenarios, thereby reducing harvesting success rates. Existing deep learning models remain limited in performing optimal target selection and dynamic sequence updating. To address these challenges, this study proposes GL-SeqNet, which enables intra-cluster tomato sequential harvesting planning through the fusion of global and local information and learning-to-rank.

2.3.1. GL-SeqNet Model Architecture

In selecting the optimal harvesting target within a tomato cluster, both global and local information are indispensable. Global information characterizes cluster topology and the relationships with the main-stem and fruit pedicels, thereby providing macroscopic guidance for harvesting sequence decisions. In contrast, local information captures fruit attributes such as maturity, surface appearance, occlusion intensity, and potential collision risks, offering cues for target prioritization. Based on this, a novel GL-SeqNet model is proposed, with its architecture illustrated in Figure 3. The model comprises four main components: a global backbone, a local backbone, a global and local feature fusion module, and a ranking head. The design concept is as follows: the global backbone captures cluster structural information, while the local backbone extracts detailed fruit features. These are integrated through cross-fusion to achieve complementary information, and the ranking head subsequently outputs harvesting priorities, thereby generating a rational intra-cluster harvesting sequence.

(1)

Input and dual-stream encoding: The cluster-cropped image is used as the input to the global branch, while five mature fruit local images are provided as inputs to the local branch. Features are extracted by lightweight modules, the global backbone and the local backbone, producing a global feature $g\_feat\in R^{B\times D}$ and a local feature $l\_feat\in R^{B\times N\times D}$. The activations use Leaky ReLU with a dropout rate of 0.2. Both the global backbone and the local backbone extract features through four cascaded Residual Blocks. Each Residual Block comprises two 3 × 3 convolutional layers (Conv), each followed by batch normalization (BN) and a Leaky ReLU activation, with skip connections configured according to the input–output dimensions and stride. The channel width is progressively expanded to 16, 32, and 64, ensuring sufficient representational capacity while alleviating gradient vanishing through residual connections.

(2)

Global and local feature fusion: After feature extraction by the global and local branches, feature concatenation (cat) is applied for fusion. Specifically, the global feature vector $g\_feat$ is broadcast along the candidate dimension to [B, N, D], and concatenated with each local feature $l\_feat$ along the channel dimension, resulting in a fused representation of [B, N, 2D].

(3)

Ranking head: Following the above extraction and fusion, a fully connected layer outputs a priority score $scores\in R^{B\times N}$ for each candidate fruit, where a larger score indicates a higher harvesting priority. During training, the scores can be optimized with RankNet [41], ListNet [42,43], or MSE [44] losses. During inference, the scores are sorted in descending order to generate the intra-cluster harvesting sequence.

2.3.2. Model Loss Function

The loss function measures the deviation between the model output and the supervisory target, and serves as the driving signal for backpropagation in training. In this study, three types of loss functions are employed: RankNet, ListNet, and MSE. Among them, RankNet loss is a pairwise ranking loss based on neural networks, which optimizes the model by minimizing the relative ranking errors of sample pairs with precedence relationships. Specifically, RankNet maps the score difference in candidate pairs into a logistic surrogate loss, thereby approximating discrete ranking criteria in a continuous space. The pairwise labels are defined as follows:

$$y_{ij}=sign(g_i^{\ast }-g_j^{\ast })\in \{-1,0,+1\},\hspace{1em}i<j$$

(1)

where $i$ and $j$ represent the indices of two different candidates within the same cluster; $y_{ij}$ represents $(i,j)$ paired labels; +1 represents $i$ precedes $j$; −1 represents $j$ precedes $i$; 0 represents a tie (in which case the pair is skipped); and $g_i^{\mathrm{*}}$ represents the gain of the i-th candidate obtained by mapping the annotation rank.

Based on these definitions, the RankNet loss is given as follows:

$$\left\{\begin{array}{l}P = \{(i,j) | 1\le i<j\le L, g_i\ne g_j \}\\ Los{s}_{RankNet} = {\displaystyle \frac{1}{\left|P\right|}}{\displaystyle \sum _{(i,j)\in P}\log }(1+\exp [- y_{ij}(s_i-s_j)])\end{array}\right.$$

(2)

where $P$ represents the set of effective pairs; $L$ represents the number of candidates within the cluster; $s_i$ represents the predicted score of the i-th candidate.

ListNet is a listwise ranking loss function. Its underlying idea is to normalize the candidate score vector into a probability distribution and minimize the cross-entropy between the target distribution and the predicted distribution, thereby approximating the correct order at the list level.

The ListNet loss formula is given as follows:

$$\left\{\begin{array}{l}{q}_i = {\displaystyle \frac{\exp (g_i^{\ast })}{{\displaystyle \sum _{k=1}^L\exp (g_k^{\ast })}}} ,\hspace{1em}\hspace{1em}i=1,\dots ,L\\ {\widehat{p}}_i = {\displaystyle \frac{\exp (s_i)}{{\displaystyle \sum _{k=1}^L\exp }(s_k)}} ,\hspace{1em}\hspace{1em}i=1,\dots ,L\\ Los{s}_{\mathrm{ListNet}} = -{\displaystyle \sum _{i=1}^L{q}_i} \log {\widehat{p}}_i\end{array}\right.$$

(3)

where $i$ represents the index of the i-th candidate within the same cluster; $L$ represents the number of candidates within the cluster; $s_i$ represents the predicted score of the i-th candidate; $g_i^{\mathrm{*}}$ represents the gain of the i-th candidate obtained by mapping the annotation rank; $q_i$ represents the target distribution obtained by applying softmax to the gain vector; ${\widehat{p}}_i$ represents the predicted distribution obtained by applying softmax to the score vector.

MSE follows a pointwise supervision paradigm, in which the model output scores are directly aligned with the annotated gains. The learning objective is to minimize the element squared error in order to capture the overall scale. The formulation is given as follows:

$$Los{s}_{MSE} ={\displaystyle \frac{1}{L}}{\displaystyle \sum _{i=1}^L{(s_i-g_i^{\ast })}^2}$$

(4)

where $i$ represents the index of the i-th candidate within the same cluster; $L$ represents the number of candidates within the cluster; $s_i$ represents the predicted score of the i-th candidate; $g_i^{\mathrm{*}}$ represents the gain of the i-th candidate obtained by mapping the annotation rank.

RankNet minimizes relative order errors at the pairwise level and directly approximates the optimal ranking objective. ListNet aligns the target and predicted probability distributions at the listwise level, providing a closer approximation of overall ranking quality. MSE performs pointwise regression from scores to gains, which offers stability and ease of training but imposes weaker direct constraints on relative order.

2.4. Evaluation Metrics

Tomato harvesting robots operate on a cluster-by-cluster basis, where the mature fruits within each cluster must be prioritized to identify the optimal harvesting target. In this study, the accuracy of the optimal harvesting target is regarded as the primary evaluation metric, while the correctness of the entire sequence is considered a secondary metric. Top-1 accuracy [45] is used to assess the accuracy of the optimal harvesting target, and Position Match Rate (PMR) is employed to evaluate the overall ranking quality.

Top-1 accuracy is a widely used metric in both classification and ranking tasks. In this study, the Top-1 accuracy of the optimal harvesting target is defined as follows:

$$\left\{\begin{array}{l}{\widehat{y}}_n = \arg \underset{i}{\max } s_{n,i}\\ \text{Top-1}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{n=1}^{N}1}\left({\widehat{y}}_n=y_n\right)\end{array}\right.$$

(5)

where $y_n$ represents the index of the annotated optimal harvesting target within the n-th cluster; ${\widehat{y}}_n$ represents the index of the predicted optimal harvesting target for the n-th cluster; n represents the cluster index; $N$ represents the total number of clusters in the validation set; $1\left(·\right)$ represents the indicator function, which equals 1 if the condition is true and 0 otherwise; $\text{Top-1}$ represents the accuracy of the optimal harvesting target.

PMR was first used as an evaluation metric in learning-to-rank and sequence prediction tasks. This metric measures the consistency ratio between the predicted sequence and the ground-truth sequence at each position. A larger PMR indicates a higher correctness of the predicted sequence. Its formula is defined as follows:

$$\left\{\begin{array}{l}m\left(y_i^j,{\widehat{y}}_i^j\right)=\left\{\begin{array}{ll}1,\hfill & y_i^j={\widehat{y}}_i^j\hfill \\ 0,\hfill & y_i^j\ne {\widehat{y}}_i^j\hfill \end{array}\right.\\ PMR={\displaystyle \frac{1}{N}}{\displaystyle {\displaystyle \sum _{i=1}^N}\frac{1}{L}}{\displaystyle {\displaystyle \sum _{j=1}^L}m\left(y_i^j,{\widehat{y}}_i^j\right)}\end{array}\right.$$

(6)

where $N$ represents the total number of clusters in the validation set; $L$ represents the number of tomato local images within each cluster; $y_i^j$ represents the element at position j of the ground-truth sequence for the i-th sample; ${\widehat{y}}_i^j$ represents the element at position j of the predicted sequence for the i-th sample.

2.5. Experimental Setups

To evaluate the performance of GL-SeqNet, experiments were conducted on the dataset. All experiments were conducted on a computer running Ubuntu 20.04 LTS, equipped with an NVIDIA GeForce RTX 2060 SUPER GPU, an Intel Core i5-12490F CPU, and 16 GB of RAM. The program was implemented in Python 3.10, using PyTorch 1.12.1, CUDA 11.3, Torchvision 0.13.1, NumPy 1.23.0, and OpenCV 4.12.

To ensure reproducibility, the random seed was fixed at 1024, and the random number generators of Python, NumPy, and PyTorch were controlled. The cluster was regarded as the minimal unit of data samples. The batch size was set to 1, with each batch consisting of one global image and five corresponding local images. Training was performed for 175 epochs. Except for the compared factors, all other hyperparameters were kept constant. Experiments were conducted to compare the effects of different input resolutions and different loss functions.

3. Results

3.1. Experiments on Different Resolutions and Loss Functions

This study first compared the effects of different input resolutions under various loss functions on the constructed dataset using the GL-SeqNet model described in Section 2.3, and the results are summarized in

Table 1. Experimental results of GL-SeqNet under different input resolutions and loss functions.

Global Resolution	Local Resolution	Loss Function	Top-1	PMR	Time/ms
224 × 224	224 × 224	RankNet	0.925	0.950	22.8
112 × 112	112 × 112	RankNet	0.925	0.960	22.6
56 × 56	56 × 56	RankNet	0.950	0.970	22.3
224 × 224	224 × 224	ListNet	0.925	0.950	22.9
112 × 112	112 × 112	ListNet	0.925	0.950	22.7
56 × 56	56 × 56	ListNet	0.925	0.950	22.2
224 × 224	224 × 224	MSE	0.850	0.920	23.0
112 × 112	112 × 112	MSE	0.850	0.925	22.8
56 × 56	56 × 56	MSE	0.900	0.945	22.3

. From the perspective of resolution, when RankNet loss was applied, reducing the resolution from 224 × 224 to 56 × 56 improved the Top-1 accuracy from 0.925 to 0.950 and the PMR from 0.950 to 0.970, while the inference time decreased from 22.8 ms to 22.3 ms, demonstrating a clear performance gain. In contrast, when ListNet loss was employed, changes in resolution had little effect on the evaluation metrics. Under the MSE loss, reducing the resolution from 224 × 224 to 56 × 56 increased the Top-1 accuracy from 0.850 to 0.900 and the PMR from 0.920 to 0.945, while the inference time decreased from 23.0 ms to 22.3 ms. Overall, with the exception of ListNet, all models achieved their best performance at the resolution of 56 × 56 under the same loss function.

From the perspective of loss functions, both RankNet and ListNet achieved high and stable performance, with Top-1 accuracy and PMR remaining in the ranges of 0.925 to 0.950 and 0.950 to 0.970, respectively. This indicates that learning-to-rank loss functions are more suitable for the task of optimizing intra-cluster harvesting sequences. However, the overall performance of the MSE loss was significantly lower, particularly at a resolution of 224 × 224, with a Top-1 accuracy of 0.850 and a PMR of 0.920. This result indicates that the traditional point-to-point regression loss is difficult to effectively model the relative priority relationship between fruits within a tomato cluster, and therefore has limitations in the sorting task.

Overall, when the global and local resolutions were kept the same, GL-SeqNet achieved its best comprehensive performance at a resolution of 56 × 56 with the RankNet loss function, reaching a Top-1 accuracy of 0.950, a PMR of 0.970, and an inference time of only 22.3 ms. These results confirm that the proposed global and local fusion ranking approach maintains high accuracy even under low-resolution inputs, and the ranking-based loss functions more effectively drive the model to capture intra-cluster fruit harvesting priorities. Therefore, the proposed method not only demonstrates superior accuracy but also ensures efficiency, highlighting its potential for deployment in practical greenhouse harvesting scenarios.

As shown in Figure 4, the loss curves of the nine experimental configurations during training are presented. Overall, the training losses of all nine configurations exhibit a stable downward trend, indicating that the models were well fitted without obvious underfitting or overfitting.

As shown in Figure 5, the performance curves of the nine experimental configurations during training are presented. Overall, for the same loss function, different resolutions exhibited similar trends in performance metrics. However, the best results were achieved when both global and local images were set to 56 × 56 with the RankNet loss function.

When the RankNet loss function was applied, the ranking visualization results at resolutions of 224 × 224, 112 × 112, and 56 × 56 are shown in Figure 6a, Figure 6b, and Figure 6c, respectively. In each subfigure, the left panel displays the global image and the right panel displays the local images. The red label “P” indicates the predicted harvesting sequence, while the green label “G” denotes the ground-truth sequence. As illustrated, the model fully predicted the correct harvesting sequence at a resolution of 56 × 56. In contrast, at resolutions of 224 × 224 and 112 × 112, the model incorrectly predicted the first and second harvesting targets, failing to effectively capture the subtle occlusion relationship between them. This visualization further validated the results reported in Table 1.

At a resolution of 56 × 56, the ranking visualization results using RankNet, ListNet, and MSE loss functions are shown in Figure 7a, Figure 7b, and Figure 7c, respectively. The model correctly predicted the entire harvesting sequence with RankNet loss, whereas with ListNet and MSE losses, it incorrectly predicted the second and third harvesting targets, failing to effectively capture the influence of green unripe tomatoes on harvesting priority. This visualization further validated the results reported in Table 1.

3.2. Comparison of Different Global and Local Resolution Combinations

Given that the resolution of the original global image is usually higher than that of the local images, preliminary experiments have demonstrated that the best performance was achieved when both global and local images were set to 56 × 56. To further evaluate the effect of global image resolution under the condition that local images were fixed at 56 × 56, the global image resolution was set to 112 × 112 and 224 × 224 for comparative experiments, with the results summarized in

Table 2. Experimental results of GL-SeqNet performance under different global and local resolution combinations.

Global Resolution	Local Resolution	Loss Function	Top-1	PMR	Time/ms	FLOPs/G
224 × 224	56 × 56	RankNet	0.950	0.970	22.7	2.567
112 × 112	56 × 56	RankNet	0.950	0.970	22.6	1.100
56 × 56	56 × 56	RankNet	0.950	0.970	22.3	0.734

. The results showed that increasing the global resolution from 56 × 56 to 224 × 224 or 112 × 112 had little impact on the ranking performance, with Top-1 accuracy and PMR remaining unchanged. However, the computational cost increased substantially with higher global resolutions. Specifically, FLOPs increased from 0.734 G at 56 × 56 to 1.100 G at 112 × 112 and further to 2.567 G at 224 × 224, representing increases of 1.499 times and 3.498 times, respectively.

Figure 8 shows the evolution of the validation set performance metrics during training. As shown, the performance curves under the three global resolutions exhibit highly consistent overall trends. However, at a resolution of 112 × 112, the model reached its optimal performance more quickly and maintained higher stability. Therefore, the combination of a global resolution of 112 × 112 and a local resolution of 56 × 56 can be considered the most effective setting.

4. Discussion

In this study, a novel GL-SeqNet model was proposed for optimizing the intra-cluster tomato harvesting sequence, enabling the effective prediction of optimal harvesting targets. Systematic comparative experiments were conducted to evaluate its performance under different resolutions and loss functions.

Regarding the impact of resolution on performance, the results showed that when the global and local image resolutions were identical, the model achieved the best balance between accuracy and efficiency at a resolution of 56 × 56. In particular, under RankNet and MSE losses, reducing the resolution from 224 × 224 to 56 × 56 significantly improved the Top-1 accuracy and PMR, while further shortening the inference time. This indicates that excessively high resolutions do not provide additional information gain for intra-cluster fruit ranking tasks; instead, they may introduce redundant details and noise, thereby interfering with the model’s ability to learn key topological relationships. By contrast, low-resolution inputs preserve the essential geometric and relative positional information while improving generalization ability and computational efficiency, thereby confirming the accuracy of the global and local fusion framework under lightweight input conditions.

In terms of loss function selection, RankNet and ListNet consistently maintained high Top-1 accuracy and PMR, clearly outperforming MSE. Ranking-based losses are better suited to modeling the relative priority relationships among fruits within a cluster, which aligns more closely with the logic of harvesting decisions. In contrast, MSE, as a pointwise regression loss, only measures numerical deviations between predictions and labels without modeling relative order relationships, leading to inherent limitations in performance.

With the local resolution fixed at 56 × 56, further comparisons were conducted using different global image resolutions. The results showed that increasing the global resolution from 56 × 56 to 112 × 112 or 224 × 224 did not yield significant differences in Top-1 accuracy or PMR, which remained stable at 0.950 and 0.970, respectively. The only observed differences are in inference time and fitting speed, where a 112 × 112 resolution achieved faster convergence, and a 56 × 56 resolution provided slightly faster speed. Overall, the combination of a global resolution of 112 × 112 with a local resolution of 56 × 56 provided a more balanced trade-off in terms of accuracy, convergence, and stability, making it the most favorable input configuration.

Overall, GL-SeqNet exhibited high accuracy and efficiency, underscoring its potential for deployment in greenhouse harvesting robots. Nevertheless, certain limitations remain.

(1)

This study focused on static image-based ranking. It did not systematically assess robustness under dynamic conditions such as wind disturbances and foliage motion, which are common in practical greenhouse environments.

(2)

Although GL-SeqNet effectively models intra-cluster harvesting sequence optimization, it does not incorporate joint optimization with related tasks such as manipulator grasp pose estimation.

(3)

The current dataset scale remains relatively limited and was collected under specific greenhouse conditions. In addition, part of the dataset was constructed using AI-based object removal to simulate post-harvesting cluster states, which may still differ from real harvesting scenarios. Therefore, further validation using multi-greenhouse, multi-device, and real sequential harvesting datasets is still required to comprehensively evaluate the robustness and generalization capability of the proposed method.

(4)

The current framework focuses on image-level intra-cluster harvesting sequence prediction and does not yet integrate key robotic components such as grasp pose estimation, motion planning, and collision avoidance. As a result, the proposed method should be regarded as a decision-level ranking module rather than a fully deployed robotic harvesting system. This also limits the direct evaluation of end-to-end robotic performance in real-world harvesting tasks.

Future research will focus on improving dynamic environment adaptability, expanding large-scale multi-scene datasets, incorporating real sequential harvesting image validation, and establishing a more complete robotic harvesting pipeline by integrating sequence prediction with grasp pose estimation and motion planning modules. In addition, multi-task collaborative optimization will be further investigated to enhance the robustness, efficiency, and practical deployment capability of tomato harvesting robots.

5. Conclusions

This study proposed GL-SeqNet, a sequence ranking model that integrates global and local information to address the challenge of intra-cluster tomato harvesting sequence optimization. By introducing stepwise optimal target selection and dynamic sequence updating, together with a sequence state evolution dataset constructed using an AI-based image removal tool, the model can effectively simulate the actual harvesting process. The experimental results demonstrate that:

(1)

With respect to resolution, low-resolution inputs can ensure high prediction accuracy while significantly improving inference efficiency.

(2)

With respect to loss functions, the ranking-based RankNet and ListNet substantially outperform MSE, better capturing the priority relationships among fruits within a cluster.

(3)

The best overall performance was achieved with a global resolution of 112 × 112 and a local resolution of 56 × 56, yielding a Top-1 accuracy of 0.950, a PMR of 0.970, and an inference time of only 22.6 ms, along with faster convergence.

In summary, GL-SeqNet achieved excellent performance and provided valuable decision support for greenhouse harvesting robots.