Tomato Pedicel Picking-Point Localization via Improved YOLOv8n-EED-Seg and RGB-D Fusion

Abstract

Accurate and rapid localization of tomato pedicel picking points presents a significant challenge for automated harvesting, due to factors such as occlusion by dense foliage, overlapping fruits, variable lighting conditions, and the slender morphology of pedicels. To address these, we propose an integrated picking decision system combining enhanced instance segmentation with RGB-D fusion. In this study, a lightweight detection model named YOLOv8n-EED-seg is introduced. An optimized EfficientRep backbone is integrated to enhance computational efficiency, while the EMAttention mechanism and a refined DynamicHead module strengthen multi-scale feature representation for slender pedicels. The model further incorporates the Zhang–Suen algorithm for skeleton extraction and a large-neighborhood mean method for depth restoration, enabling precise 3D localization. Experiments are conducted on a dataset of 3310 images collected in a greenhouse environment. Compared with the baseline YOLOv8n-seg, our model improves precision, recall, F1 score, and mAP₅₀ by 5.09%, 2.78%, 3.63%, and 4.31%, respectively. The system achieves an inference speed of 4.8 ms per frame, enabling real-time performance, while attaining a 93.88% success rate in 3D picking-point localization. Furthermore, the proposed model demonstrates superior robustness in complex environments compared with common segmentation models, effectively balancing accuracy, speed, and model complexity. This study provides a reliable technical pathway for high-precision, vision-based tomato-harvesting robots.

1. Introduction

As a globally cultivated high-value crop, tomato production remains heavily dependent on manual labor [1]. While automated harvesting systems exist, they typically rely on one-time harvesting. The selective batch-picking approach based on fruit ripeness has yet to be fully automated. Labor scarcity and rising costs underscore the urgency for tomato-picking robots to ensure sustainable production. Central to such robots is a visual system capable of real-time fruit recognition and precise 3D localization [2]. However, field conditions such as leaf occlusion, variable environmental lighting, and fruit overlap pose significant challenges to reliable perception [3]. Traditional machine vision methods (e.g., color, texture, or shape-based thresholding) show limited robustness and generalization in unstructured environments, where pedicels often adhere to foliage or structures [4]. Dynamic illumination conditions significantly compromise the accuracy of RGB-based segmentation in greenhouse settings, with recent studies reporting performance degradation of up to 7% under varying light intensities [5], while single-feature approaches like Hu moments achieved only 65% accuracy in cross-cultivar tasks [6]. Moreover, cost-effective depth sensors often suffer from data loss; specifically, the largest missing regions average approximately 2.23% of the total image area [7], which can compromise positioning precision.

In information-based agriculture, deep learning-driven object detection is now the mainstream approach in fruit recognition, surpassing early limited-feature methods [8,9]. The emergence of CNNs (YOLO, R-CNN series) has greatly enhanced both accuracy and speed. Yan et al. [10] adapted YOLOv5s for real-time apple detection, achieving a mAP of 86.75% on apple targets. Gai et al. [11] introduced TL-YOLOv8 for blueberries, integrating attention and reparameterization to accelerate training and enrich features, achieving 84.6% precision and 94.1% mAP. However, these methods target fruit-level detection, without addressing the finer-grained challenge of pedicel localization, a task requiring sub-centimeter precision rather than fruit-level detection. Recent research has shifted toward instance segmentation for precise robotic picking. Wang et al. [12] used Mask R-CNN for overlapping apple segmentation, achieving 96.5% precision and 97.4% recall, yet its two-stage design requires 270 ms per image, unsuitable for real-time harvesting. Yuan et al. [13] developed a lightweight SSD variant for cherry tomato segmentation, balancing speed and accuracy, yet its single-shot design struggles with small, slender targets. Liu et al. [14] proposed YOLACTFusion, an attention-guided RGB-NIR fusion method for tomato stem detection, improving mAP from 39.20% to 46.29%; however, it relies on multimodal input not always available in greenhouses. Attention mechanisms in YOLO models further improved robustness under occlusion and varying light [15,16], exemplified by Song et al. [17], who integrated SegNext-attention into YOLOv8-seg for tomato segmentation and maturity classification, achieving 86.9% precision and 84.8% mAP. Nevertheless, these attention mechanisms focus on backbone enhancement rather than multi-scale fusion across detection heads, leaving room for improvement in pedicel segmentation.

Although instance segmentation provides richer 2D shape information, precise 3D spatial positioning remains a key challenge for automated picking. Yoshida et al. [18] used an RGB-D camera to obtain 3D tomato point clouds, applied region growing for clustering, and determined optimal picking points by integrating voxel connectivity, Mahalanobis distance, and pedicel geometry, achieving 90% success in 15 s, which limits real-time application. Zhang et al. [19] proposed Completion-BiPy-Disp, fusing bilateral filtering with pyramid models to restore missing depth in disparity maps. Yet over 15% of regions lacked reliable depth, with RMSE reaching 3–5 pixels (4–7 mm) on texture-sparse surfaces. Zheng et al. [20] combined RAFT-Stereo with improved YOLOv5 to segment and crop point clouds via masks, then fitted spheres to compute centroids and radii, yielding a mean absolute radius error of 2.4 mm. However, the systematic depth deviated up to 3.7 mm due to point cloud holes and noise. Rong et al. [21] fused RGB-D data for dynamic tomato cluster tracking using YOLOv5 and ByteTrack (mAP 94.5%), yet failed to identify pedicel cutting or grasping points.

Skeletonization of segmentation masks enables precise extraction of picking points by converting instance masks into one-pixel-wide topological skeletons that preserve the morphological characteristics of pedicels. This approach has been effectively applied to branch-like structure analysis in agricultural and forestry contexts, including tree branch reconstruction [22,23], root system phenotyping [24], and plant phenotyping [25], demonstrating improved localization accuracy over centroid-based methods. For depth recovery, we adopt a neighborhood-based compensation strategy to reconstruct missing depth data caused by sensor limitations by interpolating invalid regions using valid depth values from surrounding pixels. Learning-based depth completion methods typically require extensive training data, which limits their practicality for rapid deployment [26]. In contrast, conventional interpolation and filtering approaches are computationally efficient but often struggle with complex occlusions and texture-sparse surfaces, producing over-smoothed results that blur fine-grained pedicel structures [27]. To balance these trade-offs, neighborhood-based depth compensation strategies offer a lightweight alternative suitable for real-time harvesting applications while maintaining local depth consistency [28].

Table 1. Summary of critical analysis of existing methods and corresponding improvements in this study.

Reference/Method	Identified Limitation	Supporting Evidence	Proposed Improvement
Yan et al. [10]	Constrained to fruit-level detection, pedicel localization remains unaddressed	Achieves mAP of 86.75% on apple fruit detection using YOLOv5s	Enables pedicel-level segmentation achieving sub-centimeter localization accuracy
Gai et al. [11]	Limited to blueberry fruit detection, lacks a dedicated module for pedicel recognition	Attains 84.6% precision and 94.1% mAP on blueberry fruit detection	Introduces a morphology-specific module tailored for pedicel feature extraction
Wang et al. [12]	Two-stage architecture imposes real-time constraints	Achieves 96.5% precision and 97.4% recall, yet requires 270 ms per image	Employs a one-stage lightweight design achieving 4.8 ms inference time
Yuan et al. [13]	Single-shot design struggles with small and slender targets	Demonstrates lightweight efficiency but yields suboptimal performance on slender structures	Implements enhanced feature extraction specifically designed for slender pedicel morphology
Liu et al. [14]	Relies on RGB-NIR multimodal input, limiting practical deployment	Improves mAP from 39.20% to 46.29% using YOLACTFusion	Adopts single-modal RGB input with attention-based feature fusion
Song et al. [17]	Attention mechanism confined to backbone level	Achieves 86.9% precision and 84.8% mAP with FGS-YOLOv8s-seg incorporating SegNext-attention	Facilitates multi-scale feature fusion across detection heads
Yoshida et al. [18]	Excessive processing latency limits real-time viability	Attains 90% picking success within 15 s	Delivers real-time performance with 4.8 ms inference time per frame
Zhang et al. [19]	Elevated RMSE on texture-sparse surfaces	Reports RMSE of 3–5 pixels (approximately 4–7 mm) on texture-sparse surfaces	Implements adaptive depth compensation reducing RMSE to 3.2 mm
Zheng et al. [20]	Systematic depth deviation induced by point cloud holes and noise	Achieves mean radius error of 2.4 mm with systematic depth deviation up to 3.7 mm	Employs neighborhood-based compensation ensuring local depth consistency
Rong et al. [21]	Fails to identify pedicel cutting points despite cluster-level accuracy	Attains 94.5% mAP for cluster detection without pedicel point identification	Integrates pedicel segmentation with picking point detection
Lin et al. [22]	Designed for point cloud structures; requires complete data	Achieves skeletal representations from point clouds but assumes complete structures	Applies iterative refinement for pedicel skeletons under occlusion
Qiu et al. [23]	Assumes complete point cloud input	Achieves point cloud completion and skeletonization but fails under occlusion	Combines skeletonization with depth completion to handle occlusion
Xu et al. [24]	Designed for root system reconstruction, not applicable to above-ground pedicel structures	Achieves skeleton-based 3D reconstruction from multi-view images but targets root phenotyping	Adapts skeletonization approach for pedicel morphology in above-ground tomato plants
Wang et al. [25]	Focuses on main stem and leaves segmentation	Extracts stem and leaf phenotypes from tomato plants, not pedicel-specific	Introduces pedicel-level skeletonization tailored for picking-point localization
Zhang et al. [26]	Surveys learning-based depth estimation without real-time consideration	Reviews monocular depth estimation methods highlighting data requirements	Adopts training-free neighborhood-based depth compensation
Hu et al. [27]	Surveys learning-based depth completion requiring extensive data	Comprehensive review of deep depth completion methods emphasizing data dependency	Implements training-free neighborhood-based depth compensation
Calderero & Henríquez [28]	Fixed kernel size induces over-smoothing	Morphological filtering exhibits inherent limitations with fixed kernels	Adopts adaptive kernel size based on local depth variance

summarizes the critical analysis of existing methods and the corresponding improvements proposed in this study.

Building on these insights, current research has advanced fruit recognition and positioning, but core challenges remain in incomplete 3D perception and limited accuracy. Most methods rely on idealized point cloud fitting, which is sensitive to depth loss and noise, or only perform cluster-level detection without providing precise operation points. Despite the progress reviewed above, three core scientific gaps remain unaddressed: lack of sub-centimeter pedicel localization, insufficient cross-scale feature fusion across detection heads, and poor depth completion robustness on weakly textured, elongated pedicel surfaces. To address these gaps, this work is built upon two design considerations: (1) the incorporation of multiple attention mechanisms along with an efficient backbone enhances pedicel segmentation accuracy without compromising real-time inference speed; (2) skeletonization of segmentation masks enables precise extraction of picking points, while a neighborhood-based depth compensation strategy effectively reconstructs missing depth data caused by sensor limitations. Guided by these considerations, we propose a tomato picking-point localization method that integrates a modified YOLOv8n-seg model with RGB-D fusion. Based on YOLOv8n-seg, we incorporate an optimized EfficientRep backbone, the EMAttention mechanism, and a refined DynamicHead module, tailored to pedicel morphology and model efficiency [29,30,31], yielding the YOLOv8n-EED-seg model. The pedicel mask is skeletonized to extract its main structure and derive picking-point coordinates [32]. The core contribution of this study lies in a systematic redesign of the detection pipeline for pedicel perception, rather than a simple assembly of existing components. In addition, specific improvements have been introduced to the existing EfficientRep and DynamicHead modules to better suit the characteristics of slender pedicel targets.

Specifically, the 8-direction shift convolution in the EfficientRep backbone enlarges the receptive field without adding parameters. The enriched features then feed into the EMAttention module for cross-scale fusion across detection heads. Finally, the refined DynamicHead decouples classification, regression, and segmentation tasks, avoiding gradient interference.

Beyond segmentation, we further address the challenge of missing depth data caused by sensor limitations on weakly textured, elongated pedicel surfaces. A large-neighborhood mean method is introduced to compensate for invalid depth values, enabling accurate 3D localization through RGB-D fusion. This depth compensation strategy, together with the cascaded feature refinement architecture, forms a complete perception pipeline from 2D segmentation to 3D localization. This problem-driven module reorganization has been validated through greenhouse harvesting experiments, bridging the gap between laboratory research and field applications. The main contributions of this work are as follows:

• This study integrates the EMAttention mechanism into the YOLOv8n-seg model to enhance the recognition and segmentation of small pedicels via cross-dimensional interaction and multi-scale feature calibration.
• To address the trade-off between inference speed and feature representation, this work introduces an improved EfficientRep lightweight backbone network. Furthermore, an improved DynamicHead module is employed to replace the original detection head. These modifications are expected to enhance feature representation and detection performance, thereby making the model more suitable for embedded deployment.
• A 3D positioning system integrating image segmentation, skeletonization analysis, and depth restoration algorithms is designed. This system achieves stable and high-precision localization of tomato picking points, providing a reliable visual perception solution for picking robots.

2. Materials and Methods

2.1. Image Dataset

Image data were collected at the Tianhe Smart Agriculture Park (Guangzhou, China) between 10 December 2024 and 26 March 2025, daily from 10:00 to 12:00. In their controlled glass greenhouse, two cherry tomato cultivars: ‘Israel Red Cluster’ (red-fruit) and ‘Yuekeda’ (yellow-fruit) are cultivated by a trellis system. During data collection, to simulate real harvesting conditions, images were captured by a vivo S19 mobile phone holding by a human operator walking along the greenhouse pathway. The main rear camera features a 1/1.55-inch CMOS sensor with 50 MP resolution. In total, 3310 tomato cluster images at a resolution of $3468\times 4624$ pixels were collected. The operator held the phone at chest height (1.2–1.5 m) with shooting angles of $30°$–$60°$ relative to the horizontal plane, oriented to capture either front or side views of the pedicel axis. Working distances ranged from 10 to 70 cm. Based on the EXIF information, the ISO was set to 200, shutter speed to 1/80 s, and white balance to 5000 K to maintain natural color reproduction under greenhouse illumination. No additional lighting or filters were used, ensuring the images reflect genuine greenhouse conditions.

To enhance model robustness, the dataset encompasses diverse illumination scenarios: direct sunlight, diffuse light, and backlighting. Following Lalonde et al. [33], illumination is classified by contrast and shadow patterns. We adapted these criteria to greenhouse conditions, where visible sky and reliable vertical shadows are absent. Illumination categories are quantified via statistical image analysis: direct sunlight is characterized by high contrast (standard deviation $>60$) and high mean brightness (>120); diffuse light by low contrast (standard deviation $<50$) and uniform illumination; backlighting by a center-to-periphery brightness ratio $<0.77$, calculated as the mean brightness of the central region ($25\%$ to $75\%$ of image dimensions) divided by that of the peripheral region. This classification is performed via grayscale histogram analysis, as illustrated in Figure 1e. Detailed statistical validation of these thresholds (including mean brightness, standard deviation, confidence intervals, and p-values) is provided in Appendix A (

Table A1. Statistical validation of illumination classification thresholds.

Illumination Category	Metric	Value (Mean ± Std)	95% CI
Direct sunlight	Mean brightness	$142.3\pm 18.7$	$[138.6,146.0]$
Direct sunlight	Standard deviation	$68.4\pm 12.3$	–
Diffuse light	Mean brightness	$98.5\pm 15.2$	$[95.4,101.6]$
Diffuse light	Standard deviation	$42.1\pm 8.5$	–
Backlighting	Center-to-periphery ratio	$0.51\pm 0.09$	$[0.49,0.53]$

).

Viewing angles are categorized based on the orientation of the primary tomato pedicel relative to the camera: front view (pedicel axis faces the camera) and side view (pedicel axis perpendicular to the camera). As summarized in

Table 2. Dataset composition by illumination condition and viewing angle.

Category	Subcategory	Images
Illumination Intensity	Direct sunlight	1432
	Diffuse light	1091
	Backlighting	787
Viewing Angle	Front View	1745
	Side View	1565

, the dataset includes 1432 direct sunlight, 1091 diffuse light, and 787 backlighting images, comprising 1745 front views and 1565 side views.

Prior to manual annotation, a color-based pre-filtering step in the HSV color space was applied to identify images containing mature fruit [34]. Fruit regions were segmented using saturation and value thresholds ($S>60$, $V>60$) [34], while green leaves (hue $60°$–$80°$) [35] and white background ($S<30$, $V>200$) were removed. Based on statistical analysis of randomly selected samples (103 red-fruit, 101 yellow-fruit images), red fruit is defined by hue in the range $0°$–$25°$, and yellow fruit by hue in the range $33°$–$45°$. Detailed statistical results, including means, standard deviations, confidence intervals, percentiles, and adopted ranges, are provided in Appendix A (Figure A1,

Table A2. Statistical analysis of fruit region hue peaks with confidence intervals.

Cultivar	Sample Size (n)	Mean ($\mathit{\mu }$)	Std ($\mathit{\sigma }$)	95% Confidence Interval	5th Percentile	95th Percentile	Calculated Range ($\mathit{\mu }\mathbf{\pm }\mathbf{2}\mathit{\sigma }$)	Adopted Range
Israel Red Cluster	103	$3.11°$	$4.95°$	$[2.14°,4.08°]$	$1.00°$	$19.30°$	$-6.8°$–$13.0°$	$0°$–$25°$
Yuekeda	101	$38.89°$	$2.92°$	$[38.31°,39.47°]$	$35.00°$	$42.00°$	$33.0°$–$44.7°$	$33°$–$45°$

). Images with a mature fruit area ratio below 0.5% were excluded, as they lack harvestable fruit. Following pre-filtering, 3230 valid images were retained for annotation. No images were discarded based on this filtering; instead, they were retained to preserve dataset diversity and to enable the model to learn the ability to identify images without harvestable pedicels.

Pedicels were annotated at the instance level using ISAT v2 software. A pickable pedicel is defined as a pedicel connected to at least one mature fruit with a sufficiently visible structure to enable human annotation (i.e., the pedicel can be identified and its midline traced) under partial occlusion. Fully occluded or completely invisible pedicels were not annotated. After annotation, the RGB images were randomly split into training, validation, and test sets in an 8:1:1 ratio with a fixed random seed (42) to ensure reproducibility. Dataset statistics are summarized in

Table 3. Strategy of dataset division.

Dataset	Number of Images	Number of Pickable Pedicels
Training set	2584	3065
Validation set	322	383
Test set	324	389

. Representative annotation examples are provided in Appendix A (Figure A3).

2.2. The Proposed YOLOv8n-EED-Seg Model

To address the challenges in segmenting slender tomato pedicels, we propose YOLOv8n-EED-seg, an improved instance segmentation model based on YOLOv8n-seg. Three enhancements are introduced: an improved EfficientRep backbone for better hardware efficiency, an EMAttention mechanism for adaptive multi-scale feature fusion, and an improved DynamicHead module for detection accuracy. The overall model architecture is shown in Figure 2a. The C2f, Bottleneck, and CBS modules are illustrated in Figure 2b,c; detailed descriptions are provided in the figure caption. Appendix A (

Table A4. Layer-by-layer architectural comparison between baseline YOLOv8n-seg and proposed YOLOv8n-EED-seg.

Stage	Baseline YOLOv8n-Seg	Proposed YOLOv8n-EED-Seg	Description of Modification
Backbone	Standard C2f modules (stages 1–4)	Improved EfficientRep with 8-direction shift convolution	Expands the receptive field without increasing parameters.
Neck	Concatenation + C2f	EMAttention after P3, P4, P5	Enhances cross-scale feature fusion under occlusion.
Detection Head	Original decoupled head (FC in ${\pi }_C$)	Improved DynamicHead (ECA replaces FC)	Decouples tasks while reducing head parameters by 98.8%.
Post-processing	Mask decoding only	Zhang–Suen skeletonization + large-neighborhood depth completion	Enables 3D picking-point localization.

) provides a layer-by-layer architectural comparison between the baseline YOLOv8n-seg and the proposed YOLOv8n-EED-seg model.

2.3. Improved EfficientRep Network Architecture

The standard stacked convolutional backbone of YOLOv8n-seg suffers from restricted receptive field, limited multi-scale fusion, and high computational cost, which is problematic for segmenting slender tomato pedicels with low pixel proportion and sparse features.

To address these limitations, we propose an improved EfficientRep architecture. Specifically, we replace RepConv with 8-direction shift convolution [30], which achieves directional feature extraction by directly shifting pixels of the input feature map in memory space across eight spatial directions (Figure 3a) to develop the S-RepConv module, where the multi-branch structure of the original RepConv is substituted with an 8-direction shift convolution-based multi-branch reparameterization structure [8]. During training, parallel multi-branch reparameterization is adopted—comprising 8-direction shift convolution branches with diverse shift scales and a 1 × 1 convolution branch, with branch outputs aggregated for feature fusion. SiLU is used as the activation function due to its smoother gradient and improved performance. During inference, all branches are fused into a single equivalent 8-direction shift convolution structure via the reparameterization strategy (Figure 3b), eliminating the need for 9-weight multiplications per pixel (3 × 3 convolution) and maintaining the lightweight inference property.The SPPF module (replacing SimSPPF), as shown in Figure 3c, consists of three sequential max-pooling layers with kernel sizes of 5 × 5, 9 × 9, and 13 × 13. Each pooling layer preserves the spatial dimensions through appropriate padding, and the outputs from each stage are concatenated along the channel dimension to achieve efficient multi-scale fusion, enhancing slender-target perception while lowering computational cost. The improved EfficientRep network architecture is illustrated in Figure 3d. Each block uses a Conv-BN-SiLU structure to strengthen feature robustness and generalization.

2.4. EMAttention Mechanism

Tomato pedicels, typically slender in images, are often occluded by foliage and affected by lighting variations. Robust detection therefore requires models capable of capturing both local details and global contextual information across multiple scales. Accordingly, this study integrates the lightweight cross-spatial attention module EMAttention [29] into the feature fusion layer, as illustrated in Figure 3e. The EMA module captures both long- and short-range dependencies through multi-scale and cross-spatial learning without dimensionality reduction, thereby enhancing feature representation while preserving computational efficiency (Detailed mathematical derivations are provided in Appendix B).

2.5. Improved DynamicHead Module

In tomato pedicel segmentation, multi-scale and small-scale targets pose a challenge. The YOLOv8-seg detection head uses a decoupled architecture with parallel branches for box and mask prediction, processing three-scale FPN features independently. This design lacks adaptive multi-scale interaction, limiting its ability to capture fine-grained details of slender pedicels (typically smaller than $64\times 64$ pixels) and fuse cross-scale context, leading to suboptimal segmentation for small or occluded targets.To address this, an improved DynamicHead (DyHead) module (Figure 4b) replaces the original detection head. DyHead includes three core attention modules: scale-aware (${\pi }_L$), spatial-aware (${\pi }_S$), and task-aware (${\pi }_C$). ${\pi }_L$ dynamically fuses multi-scale features based on semantic relevance, while ${\pi }_S$ enhances attention sparsity via deformable convolution. The core optimization focuses on ${\pi }_C$: traditional fully connected (FC) layers are replaced with an Efficient Channel Attention (ECA) module (Figure 4a). The ECA module operates on the principle of local cross-channel interaction without dimensionality reduction. It first performs global average pooling (GAP) on the input feature map to compress spatial information into channel-wise statistics, producing a $1\times 1\times C$ vector where C represents the number of input channels ($C=256$ in this study). Subsequently, a 1D convolution kernel of size K is applied to capture local cross-channel interactions. The kernel size K is adaptively determined by the channel dimension according to the equation:

$$K={⌊{\displaystyle \frac{{log}_2\left(C\right)}{2}}+{\displaystyle \frac{1}{2}}⌋}_{\mathrm{odd}}$$

(1)

where C is the number of input channels and the subscript “odd” indicates rounding to the nearest odd number. For the 256-channel feature maps used in tomato pedicel segmentation, this calculation yields K = 3.

Compared with the original ${\pi }_C$ module using two fully connected layers ($256\times 256$ each, totaling 65,536 parameters), the proposed ECA-based replacement reduces the number of parameters to 768 ($K\times C/2$), achieving a 98.8% reduction. This is accomplished by replacing dense FC layers with local 1D convolution and removing bias terms, capturing local dependencies via a sliding window. The original hard sigmoid in DyHead is replaced with standard sigmoid, as hard sigmoid’s piecewise linear approximation causes fine-grained channel weight loss, impairing weak feature capture for small-scale pedicels. Standard sigmoid’s smooth nonlinearity enables more precise adaptive channel weight learning, preserving these fragile features. Furthermore, a batch normalization layer is added after the ECA block to stabilize training. A residual fusion mechanism with learnable initial weights ($[1,0,0,0]$) ensures stable initial output, prevents gradient vanishing, and enables adaptive weight allocation across classification, box regression, and segmentation tasks.

2.6. Skeletonization Processing

In the visual system of tomato-picking robots, the picking point is defined as a pixel on the pedicel midline. To accurately extract this key coordinate from the segmentation mask, our system employs the Zhang–Suen algorithm—characterized by simple implementation and high computational efficiency [32]. This parallel thinning method iteratively removes boundary points meeting specific criteria via two alternating sub-iterations until convergence, yielding a single-pixel-width pedicel midline skeleton.

As illustrated in Figure 5, the algorithm evaluates each foreground pixel $P_1(i,j)$ (marked in red) and its eight surrounding neighborhood pixels $P_2\sim P_9$. In each iteration step, a pixel can only be deleted if it meets the following conditions simultaneously: the number of foreground pixels in the 8-neighborhood satisfies $2\le B\le 6$; the number of $0\to 1$ transitions is one during sequential traversal of the neighborhood; there is at least one background pixel in the specific neighborhood combination.

Among them, the conditions for the first sub-iteration are:

$$\left\{\begin{array}{c}{P}_2\times P_4\times P_6=0\hfill \\ P_4\times P_6\times P_8=0\hfill \end{array}\right.$$

(2)

The conditions for the second sub-iteration are as follows:

$$\left\{\begin{array}{c}{P}_2\times P_4\times P_8=0\hfill \\ P_2\times P_6\times P_8=0\hfill \end{array}\right.$$

(3)

Equation (2) is used to remove objects on the lower and right boundaries, while Equation (3) is used to remove objects on the upper and left boundaries.

2.7. Fusion of Depth Information for Picking-Point Localization

Depth information extraction is critical for acquiring 3D coordinates of picking points. Due to the slender morphology of tomato pedicels and the limited resolution of depth cameras, direct depth sampling from raw depth maps frequently leads to missing or invalid values. Conventional hole-filling strategies, such as region-growing algorithms, can only repair small, localized missing regions, failing to handle large-area depth loss caused by uneven illumination or specular reflection, as visualized in Figure 6. To overcome such drawbacks, this study adopts a large-neighborhood mean filtering strategy for depth completion, with the complete procedure detailed in Algorithm A1. In contrast to complex inpainting networks and variational optimization methods that demand high computational costs and extensive training data, the large-neighborhood mean method robustly estimates missing values using valid neighboring depth information. The depth completion process proceeds as follows. First, the binary mask obtained from pedicel segmentation is used to extract the depth set of the pedicel region via element-wise multiplication with the depth map. The mean depth $z_1$ of the pedicel region is then calculated. Let z be the original depth value at the picking point coordinates. The final depth $z_P$ is determined by:

$$z_P=\left\{\begin{array}{cc}{z}_1,\hfill & \mathrm{if}z=0\hfill \\ z,\hfill & \mathrm{if}z\ne 0\mathrm{and}|z_1-z|\le k\hfill \\ z_1,\hfill & \mathrm{if}z\ne 0\mathrm{and}|z_1-z|>k\hfill \end{array}\right.$$

(4)

where k is a reference threshold. The optimal value of k was determined through preliminary experiments under the specific harvesting environment, as detailed in

Table A3. Depth completion performance with different k values.

Threshold k (cm)	Abnormal Depth Rejection Rate (%)	Average Localization Error (cm)
1.5	94.3	1.82
2.0	92.4	1.35
2.5	88.6	1.05
3.0	86.7	1.48
3.5	83.0	1.75

in Appendix A.

After depth completion, the RGB and depth images must be aligned to establish pixel-wise correspondence. Alignment is performed by warping the depth image into the RGB coordinate frame using the SDK’s align module, which relies on factory-calibrated intrinsic and extrinsic parameters. For an aligned pixel $(u,v)$ with depth value d (mm), the 3D coordinate $(X,Y,Z)$ (mm) is given by the pinhole model:

$$X={\displaystyle \frac{(u-c_x)·d}{f_x}},Y={\displaystyle \frac{(v-c_y)·d}{f_y}},Z=d$$

where $f_x$ and $f_y$ are the focal lengths, and $(c_x,c_y)$ is the principal point. Detailed calibration parameters are listed in Appendix B.

2.8. Experiment Environment and Model Evaluation

The experiments were performed on a Windows 10 system using the PyTorch 2.0+cu118 framework and Python 3.8. Hardware included a 24 GB NVIDIA GeForce RTX 3090 GPU and an Intel Xeon Platinum 8362 CPU with 60 GB RAM. Training was conducted for 300 epochs (determined empirically as the point where validation loss and mAP₅₀ stabilized beyond 250 epochs in preliminary experiments), requiring approximately 5 h. Detailed hyperparameters are listed in

Table 4. Training parameters.

Parameter	Configuration
Epoch	300
Batch size	16
Optimizer	AdamW
Weight decay	0.0005
Initial learning rate	0.001
Cosine learning rate scheduling	True

. All model training and comparisons were conducted under identical conditions. The training data was augmented with a consistent pipeline: Mosaic (multi-scale context enrichment), MixUp (linear interpolation with $\lambda \sim \mathrm{Beta}(1.5,1.5)$), and random affine transformations (rotation $\pm 15°$, scaling $0.8$ to $1.2$, translation $\pm 10\%$, shearing $\pm 10°$) to enhance spatial invariance. Model performance—including Precision, Recall, F1-score, and mAP—was evaluated using the best checkpoint after 300 epochs of training. For each model, three independent training runs were conducted with different random seeds (42, 123, 456).

In this study, model performances were evaluated by four key metrics: mean average precision at 50% IoU threshold (mAP₅₀), Precision, Recall, and F1-score. Here, mAP₅₀ specifically denotes the mAP value computed with an Intersection over Union (IoU) threshold of 0.5.

$$\begin{array}{cc}\hfill \mathrm{Precision}& ={\displaystyle \frac{TP}{TP+FP}},\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill \mathrm{Recall}& ={\displaystyle \frac{TP}{TP+FN}},\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill F_1\text{-}\mathrm{score}& =2\times {\displaystyle \frac{\mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}},\hfill \end{array}$$

(7)

where $TP$, $FP$, and $FN$ denote true positives, false positives, and false negatives, respectively; C represents the number of classes;

3. Results and Analysis

To comprehensively assess the proposed tomato-picking decision system and validate its core improved modules, systematic experiments were conducted on self-constructed tomato pedicel RGB and RGB-D test datasets under real agricultural field conditions. The analysis quantifies each module’s contribution to pedicel recognition and picking-point localization, and elaborates on the integrated system’s operational mechanisms, focusing on the synergistic effects of its key techniques on slender pedicel detection and precise 3D localization in complex unstructured environments.

3.1. Ablation Experiments

This section evaluates the impacts of the improved EfficientRep backbone, the EMAttention module, and the improved Dyhead module through ablation experiments. All reported results are presented as mean values from three independent runs (random seeds: 42, 123, and 456). The stability and reproducibility of the experimental results are validated in Appendix C (

Table A6. Performance variability of different model variants across three runs (seeds: 42, 123, 456). Results are reported as mean ± std. Best performance per metric within each variant is in bold.

Model	Metric	Random Seed			Mean ± Std
		42	123	456
YOLOv8n-EED-seg	Precision (%)	91.98	92.15	92.11	92.08 ± 0.09
	Recall (%)	81.92	82.11	82.26	82.10 ± 0.17
	${\mathrm{F}}_1$ Score (%)	86.40	86.82	86.71	86.64 ± 0.21
	${\mathrm{mAP}}_{50}$ (%)	86.92	87.10	87.01	87.01 ± 0.09
YOLOv8n-ERD-seg	Precision (%)	89.69	89.61	89.52	89.61 ± 0.09
	Recall (%)	81.64	81.02	81.86	81.51 ± 0.44
	${\mathrm{F}}_1$ Score (%)	83.83	83.42	83.45	83.57 ± 0.23
	${\mathrm{mAP}}_{50}$ (%)	84.70	84.80	84.72	84.74 ± 0.05
YOLOv8n-EMAD-seg	Precision (%)	89.70	90.06	90.03	89.93 ± 0.20
	Recall (%)	82.40	82.73	82.13	82.42 ± 0.30
	${\mathrm{F}}_1$ Score (%)	85.30	85.45	85.62	85.46 ± 0.16
	${\mathrm{mAP}}_{50}$ (%)	86.36	86.61	86.77	86.58 ± 0.21
YOLOv8n-Dyhead-seg	Precision (%)	87.76	88.12	88.06	87.98 ± 0.19
	Recall (%)	80.12	80.15	80.36	80.21 ± 0.13
	${\mathrm{F}}_1$ Score (%)	83.72	83.41	83.25	83.46 ± 0.24
	${\mathrm{mAP}}_{50}$ (%)	84.21	84.10	84.50	84.27 ± 0.20
YOLOv8n-seg	Precision (%)	86.91	87.04	87.01	86.99 ± 0.07
	Recall (%)	79.43	79.21	79.33	79.32 ± 0.11
	${\mathrm{F}}_1$ Score (%)	82.95	83.06	83.01	83.01 ± 0.06
	${\mathrm{mAP}}_{50}$ (%)	82.61	82.80	82.70	82.70 ± 0.10

), where all metrics are presented as mean ± standard deviation.

Table 5. Ablation analyses of different modules’ effects on recognition task.

Model	Precision/%	Recall/%	F1 Score/%	mAP50/%	Size (MB)	FLOPs (G)	Processing Time per Photo (ms)
YOLOv8n-EED-seg	92.08 ± 0.09	82.10 ± 0.17	86.64 ± 0.21	87.01 ± 0.09	7.5	9.1	4.8
YOLOv8n-ERD-seg	89.61 ± 0.09	81.51 ± 0.44	83.57 ± 0.23	84.74 ± 0.05	7.0	8.5	4.7
YOLOv8n-EMAD-seg	89.93 ± 0.20	82.42 ± 0.30	85.46 ± 0.16	86.58 ± 0.21	7.8	9.7	5.0
YOLOv8n-Dyhead-seg	87.98 ± 0.19	80.21 ± 0.13	83.46 ± 0.24	84.27 ± 0.20	7.1	8.9	4.8
YOLOv8n-seg	86.99 ± 0.07	79.32 ± 0.11	83.01 ± 0.06	82.70 ± 0.10	6.6	8.2	4.5

summarizes results of ablation experiments on the proposed framework. The improved Dyhead alone (Row 4) enables adaptive fusion and task decoupling, which increases mAP₅₀ from 82.70% to 84.27% (+1.57%) with minimal computational overhead (model size: 6.6 MB to 7.1 MB; inference: 4.5 ms to 4.8 ms). EMAttention (Row 3) strengthens cross-scale feature aggregation under occlusion, which further improves mAP₅₀ to 86.58% (+3.88%) and F₁-score to 85.46, but increases model size to 7.8 MB and inference time to 5.0 ms (model size +18.2%, inference +11.1%). The improved EfficientRep backbone with Dyhead (Row 2) enlarges the receptive field without adding parameters, which reduces model size to 7.0 MB and inference time to 4.7 ms. Simultaneously, a 84.74% mAP₅₀ is achieved, enabling better feature extraction for slender pedicels.

The full YOLOv8n-EED-seg model (Row 1) integrates all three enhancements and delivers the best performance: mAP₅₀ reaches 87.01% (+4.31% over baseline), Precision 92.08%, Recall 82.10%, and F₁-score 86.64%, with competitive FLOPs of 9.1 G and inference time of 4.8 ms. Relative to Dyhead-seg (Row 4), the full model achieves a 2.74% higher mAP₅₀ with negligible increases in model size (0.4 MB) and FLOPs (0.2 G). Relative to EMAD-seg (Row 3), it achieves higher accuracy accompanied by a smaller model size of 7.5 MB (compared to 7.8 MB) and faster inference of 4.8 ms (compared to 5.0 ms). Relative to ERD-seg (Row 2), it delivers substantial accuracy gains (mAP₅₀ +2.27%, precision +2.47%, ${\mathrm{F}}_1$-score +3.07%) with only modest increases in model size (0.5 MB) and FLOPs (0.6 G). These results confirm that the three modules work synergistically: EfficientRep enlarges the receptive field without adding parameters, EMAttention strengthens cross-scale feature fusion under occlusion, and DynamicHead decouples tasks for fine-grained localization.

Figure 7 presents the qualitative ablation results across five model variants. In the first column (Israel Red Cluster pedicel segmentation), the baseline model achieves a confidence score of 0.78, which improves to 0.85 with DyHead, to 0.88 with EMAttention, and the full EED-seg model reaches the highest confidence score of 0.89. In the second column (apical pedicel segmentation), both EED-seg and EMAD-seg achieve 0.92, while ERD-seg, Dyhead-seg, and the baseline incorrectly split the pedicel into two instances due to branch interference, with confidence scores of 0.89, 0.85, and 0.85, respectively. In the third column (two harvestable pedicels), EED-seg achieves the best performance among all compared models. In the fourth column (Yuekeda cultivar with three pickable pedicels), EED-seg achieves the highest confidence scores across all instances (0.93, 0.87, 0.91), outperforming EMAD-seg (0.92, 0.81, 0.90), ERD-seg (0.92, 0.87, 0.82), Dyhead-seg (0.92, 0.85, 0.83), and the baseline (0.89, 0.85, 0.82). These results demonstrate that the full EED-seg model attains the highest overall recognition confidence among all evaluated approaches, exhibiting balanced and robust performance in segmenting both fine-grained and regular pedicels.

3.2. Performance Comparisons of Different Models on Target Detection Tasks

To evaluate the proposed YOLOv8n-EED-seg, we compared it with YOLOv9-seg [36], YOLOv11-seg [37], YOLACT [38], and Seg-rtdetr [39] under identical conditions. All reported results are presented as mean values from three independent runs with different random seeds (42, 123, and 456). The stability and reproducibility of the experimental results are further validated in Appendix C (

Table A7. Performance variability of different models across three runs (seeds: 42, 123, 456). Results are reported as mean ± std. Best overall performance per metric across all models is in bold.

Model	Metric	Random Seed			Mean ± Std
		42	123	456
YOLOv8n-EED-seg	Precision (%)	91.98	92.15	92.11	92.08 ± 0.09
	Recall (%)	81.92	82.11	82.26	82.10 ± 0.17
	${\mathrm{F}}_1$ Score (%)	86.40	86.82	86.71	86.64 ± 0.21
	${\mathrm{mAP}}_{50}$ (%)	86.92	87.10	87.01	87.01 ± 0.09
YOLOv9-seg	Precision (%)	89.12	89.01	89.12	89.08 ± 0.06
	Recall (%)	79.64	79.22	79.98	79.61 ± 0.38
	${\mathrm{F}}_1$ Score (%)	82.33	82.11	81.98	82.14 ± 0.18
	${\mathrm{mAP}}_{50}$ (%)	82.12	82.30	81.87	82.10 ± 0.22
YOLOv11-seg	Precision (%)	86.11	86.97	86.55	86.54 ± 0.43
	Recall (%)	80.22	80.78	80.47	80.49 ± 0.28
	${\mathrm{F}}_1$ Score (%)	83.52	83.12	83.74	83.46 ± 0.31
	${\mathrm{mAP}}_{50}$ (%)	83.14	83.26	83.33	83.24 ± 0.10
YOLACT	Precision (%)	87.16	87.05	87.21	87.14 ± 0.08
	Recall (%)	80.17	80.25	80.32	80.25 ± 0.08
	${\mathrm{F}}_1$ Score (%)	83.14	83.11	83.15	83.13 ± 0.02
	${\mathrm{mAP}}_{50}$ (%)	82.27	82.74	82.80	82.60 ± 0.29
SEG-RTDETR	Precision (%)	86.51	86.74	86.13	86.46 ± 0.31
	Recall (%)	80.44	80.09	79.33	79.95 ± 0.57
	${\mathrm{F}}_1$ Score (%)	82.91	83.76	83.11	83.26 ± 0.45
	${\mathrm{mAP}}_{50}$ (%)	83.41	84.04	83.74	83.73 ± 0.32

), where all metrics are presented as mean ± standard deviation, with the best overall performance per metric across all models highlighted in bold. As summarized in

Table 6. Comparison of different models on the test dataset.

Model	Precision/%	Recall/%	F1 Score/%	mAP50/%	Size (MB)	FLOPs (G)	Processing Time per Photo (ms)
YOLOv8n-EED-seg	92.08 ± 0.09	82.10 ± 0.17	86.64 ± 0.21	87.01 ± 0.09	7.5	9.1	4.8
YOLOv9-seg	89.08 ± 0.06	79.61 ± 0.38	82.14 ± 0.18	82.10 ± 0.22	6.4	8.7	4.9
YOLOv11-seg	86.54 ± 0.43	80.49 ± 0.28	83.46 ± 0.31	83.24 ± 0.10	6.3	8.5	4.7
YOLACT	87.14 ± 0.08	80.25 ± 0.08	83.13 ± 0.02	82.60 ± 0.29	46.5	32.4	20.0
Seg-rtdetr	86.46 ± 0.31	79.95 ± 0.57	83.26 ± 0.45	83.73 ± 0.32	11.3	12.8	5.1

, the proposed model (Row 1) achieves the best performance across all accuracy metrics: ${\mathrm{mAP}}_{50}$ of 87.1%, precision of 92.08%, and ${\mathrm{F}}_1$-score of 86.82%. It outperforms YOLOv9-seg by 4.8% in ${\mathrm{mAP}}_{50}$, 6.03% in precision, and 4.49% in ${\mathrm{F}}_1$-score; outperforms YOLOv11-seg by 3.77%, 5.18%, and 3.08%; outperforms YOLACT by 4.3%, 4.94%, and 3.68%; and outperforms Seg-rtdetr by 3.06%, 5.41%, and 3.06%, respectively.

In terms of computational efficiency, the proposed model requires 9.1 G FLOPs (Column 7) and 7.5 MB of parameters (Column 6), with an inference speed of 4.8 ms per frame (Column 8). Compared to YOLOv9-seg (8.7 G, 6.4 MB, 4.9 ms) and YOLOv11-seg (8.5 G, 6.3 MB, 4.7 ms), it has modestly higher computational cost but achieves substantially better accuracy. Compared to YOLACT (32.4 G, 46.5 MB, 20 ms) and Seg-rtdetr (12.8 G, 11.3 MB, 5.1 ms), it is significantly more efficient.

Among the compared models, YOLOv9-seg and YOLOv11-seg are lightweight successors in the YOLO series, designed for efficient deployment but with limited accuracy for fine-grained pedicel segmentation. YOLACT is a real-time instance segmentation model that generates prototype masks, but its large model size (46.5 MB) and high computational cost (32.4 G FLOPs, 20 ms per frame) make it unsuitable for real-time harvesting applications. Seg-rtdetr adopts a Transformer-based architecture with multi-head self-attention and a hybrid encoder, achieving competitive accuracy (84.04% ${\mathrm{mAP}}_{50}$) but at the cost of larger model size (11.3 MB) and slower inference (5.1 ms) compared to lightweight YOLO variants.

Although YOLOv11-seg offers the fastest inference (4.7 ms) and smallest model size (6.3 MB), its ${\mathrm{mAP}}_{50}$ is only 83.24%, approximately 3.77% lower than that of the proposed model (87.01%). This trade-off between lightweight deployment and detection accuracy is effectively balanced by the proposed YOLOv8n-EED-seg, which achieves superior accuracy while maintaining competitive efficiency.

Figure 8 qualitatively compares the inference results of different models in real greenhouse scenarios. Rows (a–e) correspond to YOLOv8n-EED-seg (proposed), YOLOv9-seg, YOLOv11-seg, YOLACT, and SEG-RTDETR, respectively. In Column 1 (Israel Red Cluster with two harvestable pedicels), the proposed model achieves the highest confidence scores (0.90, 0.87). In Column 2 (slender pedicel), only EED-seg and YOLACT succeed, with EED-seg achieving superior confidence scores (0.82, 0.85) versus YOLACT (0.72, 0.86), while EED-seg has a much smaller model size. In Column 3 (standard pedicel), EED-seg achieves the highest confidence score (0.86), matching YOLOv11-seg and outperforming others by 0.04–0.05. In Column 4 (Yuekeda with 60% occlusion), EED-seg achieves the best confidence scores for both the upper pedicel (surpassing others by 0.01–0.05) and the lower pedicel (0.93). These results demonstrate that EED-seg outperforms competing models across diverse challenging scenarios while maintaining a compact size (7.5 MB) and achieving an optimal balance between detection accuracy and computational efficiency.

Furthermore, the YOLOv8n-EED-seg model exhibits robust generalization across varying illumination conditions, with detailed results provided in Appendix D (

Table A8. Model performance under different lighting conditions.

Illumination Condition	Precision (%)	Recall (%)	${\mathbf{mAP}}_{\mathbf{50}}$ (%)
Direct Sunlight	92.1	81.5	86.7
Diffuse Light	92.5	82.8	87.4
Backlighting	91.9	81.3	86.1

).

3.3. Results on Picking-Point Localization

Figure 9 illustrates the complete recognition pipeline, including skeletonization and picking-point localization, based on a mobile-acquired test dataset.The process consists of five stages. In the first stage (data acquisition), RGB images $I_{\mathrm{rgb}}$ and depth images $I_{\mathrm{depth}}$ are synchronously captured using an Intel RealSense D455 RGB-D camera, where the RGB image provides texture and color information of the pedicel while the depth image directly provides the depth value (in mm) for each pixel. In the second stage (semantic segmentation), the RGB image is fed into the trained YOLOv8n-EED-seg model to generate a binary pedicel mask $M_{\mathrm{stem}}$, with $M_{\mathrm{stem}}(u,v)=1$ indicating that pixel $(u,v)$ belongs to the pedicel region and $M_{\mathrm{stem}}(u,v)=0$ indicating background. The third stage is depth extraction, in which the depth information of the pedicel region is extracted by pixel-wise multiplication: $I_{\mathrm{depth}\_\mathrm{stem}}(u,v)=M_{\mathrm{stem}}(u,v)·I_{\mathrm{depth}}(u,v)$. In the fourth stage (depth completion), the large-neighborhood mean method is applied to compensate for missing depth values, resulting in the completed depth map $I_{\mathrm{depth}\_\mathrm{completed}}$. In the fifth and final stage (coordinate transformation), for each pixel $(u,v)$ with completed depth value $d=I_{\mathrm{depth}\_\mathrm{completed}}(u,v)$, the RealSense SDK function $\mathtt{rs}\mathtt{2}\_\mathtt{deproject}\_\mathtt{pixel}\_\mathtt{to}\_\mathtt{point}\left(\right)$ is used to directly convert the pixel coordinates and depth value into 3D world coordinates, obviating the need for manual manipulation of intrinsic and extrinsic matrices.

The pipeline demonstrates robust performance across diverse scenarios. In Column 1, the pedicel, fragmented by branch breakage, produces two independent skeletons; the picking point is defined as the midpoint between their endpoints, treating the disconnected parts as a single structure. Columns 2 and 3 exhibit accurate localization for the ‘Yuekeda’ cultivar despite morphological challenges—forward curvature (Column 2) and limited visibility (Column 3). Column 4 highlights the method’s superiority under favorable conditions. Collectively, these results validate the robustness and adaptability of the proposed approach across challenging greenhouse scenarios.

As has been emphasized in Section 2 of Methods, depth information extraction is critical for acquiring 3D coordinates of picking points. This paper introduces the large-neighborhood mean method to robustly estimate missing values using valid neighboring depth information. To determine the optimal threshold k for the large-neighborhood mean method, a comparative experiment is conducted. The evaluation is based on two metrics: first, the abnormal depth rejection rate, defined as the proportion of outliers correctly identified and replaced by the mean depth $z_1$; second, the picking localization error, measured as the distance deviation between the picking point computed from the restored depth and its actual position.

In the tomato greenhouse, the camera mounted on the robotic arm is typically positioned 20–30 cm from the target pedicel to balance depth measurement accuracy and operational safety. The fixed distance of 25 cm for 3D evaluation falls within this optimal range, ensuring the experimental setup is representative of actual harvesting conditions. This specific distance was determined empirically: the depth camera was fixed perpendicular to the greenhouse rail with an initial offset of 23 cm from the cultivation pot, then incrementally adjusted using real-time depth feedback until the final working distance of 25 cm was set. A total of 53 samples are used in the experiment; this sample size is determined based on the availability of representative pedicel instances with complete depth annotations across varying occlusion levels and lighting conditions. The 53 samples ensure statistical validity while maintaining manual measurement feasibility, covering diverse scenarios including slender pedicels (the bounding boxes of these pedicels are typically smaller than <32 × 32 pixels, following the MS COCO definition of small objects), and different cultivars (Israel Red Cluster and Yuekeda), thereby providing a robust basis for threshold optimization.

Table A3 summarizes the depth completion performance across different threshold values of k, from which $k=2.5$ cm is identified as the optimal threshold, balancing a relatively high abnormal depth rejection rate (88.6%) and the lowest average localization error (1.05 cm). Taking the best threshold $k=2.5$ cm, the corresponding localization results of picking-point depth information are presented in detail in Figure 10. The workflow begins with acquiring the original RGB image for 2D picking-point localization to accurately locate the target pedicel and its picking position. The original scene depth image is captured synchronously with the RGB image to record the initial depth data for 3D coordinate calculation. Finally, the large-neighborhood mean method is applied to compensate for missing depth values, yielding the precise 3D depth localization result of the picking point. In this randomly selected example, the obtained depth value is approximately 24.175 cm.

To quantify the localization accuracy of the proposed approach, a systematic evaluation was performed on a depth camera-captured test dataset comprising 324 images, which collectively contain 343 manually annotated pickable pedicels. During image acquisition, the distance between the depth camera and prominent primary pickable pedicels was deliberately fixed at 25 cm to establish uniform experimental conditions. A localization was considered successful if the Euclidean distance between the estimated and ground-truth positions was ≤15 mm, a tolerance determined based on the pedicel diameter (3–5 mm) and the end-effector grasping tolerance (∼10 mm). All test images were processed through the picking-point localization pipeline detailed in the preceding sections, including 2D image target detection for pedicel region identification, depth completion via the large-neighborhood mean method for missing depth data compensation, skeleton extraction for picking point derivation, and coordinate transformation for 3D positioning. Experimental results reveal that 322 of the 343 picking points are accurately localized within the 15 mm tolerance, yielding an overall success rate of 93.88%, with depth errors bounded to approximately $\pm 1.2$ cm. Detailed uncertainty analysis, including localization variability, depth sensor accuracy, and confidence estimation, is provided in Appendix D.2.

To further validate the effectiveness of the depth completion component, we conducted quantitative comparisons of the proposed large-neighborhood mean method with alternative approaches, including conventional interpolation methods (bilinear and bicubic interpolation) and a learning-based method (BP-Net). As shown in

Table 7. Quantitative comparison of different depth completion methods.

Method	RMSE (cm)	MAE (cm)	Inference Time (ms)	Training Required
Bilinear Interpolation	1.34	1.27	3.2	No
Bicubic Interpolation	1.28	1.12	3.5	No
BP-Net (learning-based)	0.92	0.72	23.0	Yes
Ours (large-neighborhood mean)	1.05	0.81	3.9	No

, our method achieves a localization RMSE of 1.05 cm and MAE of 0.81 cm at a working distance of 25 cm, with an inference time of 3.9 ms, while BP-Net achieves slightly better accuracy (RMSE of 0.92 cm, MAE of 0.72 cm), it requires extensive training data and has a much higher inference time of 23.0 ms, making it unsuitable for real-time harvesting applications. In contrast, our method requires no training and achieves a fast inference time of 3.9 ms, significantly outperforming BP-Net in efficiency while maintaining competitive accuracy. Compared to bilinear interpolation (RMSE 1.34 cm, MAE 1.27 cm, 3.2 ms) and bicubic interpolation (RMSE 1.28 cm, MAE 1.12 cm, 3.5 ms), our method achieves substantially better accuracy with a modest increase in inference time.

These results demonstrate that our method achieves a favorable balance between accuracy and efficiency, making it particularly suitable for real-time greenhouse harvesting applications where computational resources are limited. The high localization success rate (93.88%), coupled with the stringent depth error tolerance and efficient depth completion, thoroughly validates the robustness and reliability of the proposed method, confirming its suitability for real-world agricultural automation scenarios.

3.4. Real-Time Performance Evaluation on Edge Devices

To validate real-world deployability, we evaluated the proposed model on a Jetson Orin NX edge device (100 TOPS). As shown in

Table 8. Real-time performance of different pipeline components on edge devices (Jetson Orin NX, 100 TOPS).

Model/Pipeline	Components	Inference Time (ms)	FPS
YOLOv8n (baseline)	Baseline detection only	7.6	132
YOLOv8n-EED-seg	Detection + segmentation	9.3	108
YOLOv8n-EED-seg + Post-processing	Detection + segmentation + post-processing	16.2	62

, the baseline YOLOv8n achieves 7.6 ms (132 FPS), while our YOLOv8n-EED-seg achieves 9.3 ms (108 FPS). With full post-processing (mask extraction, skeletonization, and depth completion), the total time increases to 16.2 ms (62 FPS). Since 30 FPS is the standard real-time benchmark for embedded systems, both 108 FPS and 62 FPS far exceed this requirement, validating the feasibility of real-time picking-point localization in greenhouse environments. However, this level of performance requires at least 100 TOPS of computational power, which is provided by the Jetson Orin NX. Future work will focus on model compression techniques to enable deployment on lower-power devices such as the Jetson Nano.

4. Discussion

This study proposed a tomato pedicel picking-point localization method based on an improved YOLOv8n-EED-seg model and RGB-D fusion. The main contributions of this work are as follows: (1) The proposed YOLOv8n-EED-seg achieves 92.08% precision and 87.01% ${\mathrm{mAP}}_{50}$ for pedicel segmentation. (2) The picking-point localization success rate reaches 93.88% with depth error bounded to $\pm 1.2$ cm. (3) The inference time is 4.8 ms per image, enabling real-time operation. (4) The model maintains robust performance under partial occlusion (success rate 0.89 with partial occlusion). These results support the effectiveness of our idea: (I) Combining backbone enhancement (EfficientRep), attention-guided fusion (EMAttention), and task decoupling (improved DynamicHead) yields synergistic improvements for pedicel-level perception. (II) Skeletonization of segmentation masks enables precise picking point extraction and neighborhood-based depth compensation effectively reconstructs missing depth data. To contextualize the proposed method, we compare it with existing approaches in terms of segmentation accuracy, real-time inference, and localization precision, as detailed below.

4.1. Comparison with Previous Work

Segmentation accuracy: The proposed YOLOv8n-EED-seg model achieves 87.1% mAP₅₀ and 92.08% precision for pedicel segmentation. Direct comparisons are confined to methods that specifically target pedicel or stem segmentation. In the domain of tomato stem detection, Liu et al. [14] reported 46.29% mAP₅₀ using YOLACTFusion, while for tomato main stem and lateral branch segmentation, Ji et al. [39] achieved 79.3% mAP₅₀ with EDI YOLO. The proposed method substantially outperforms both approaches, yielding advantages of 40.81 and 7.8 percentage points, respectively. For broader context, we also compare with fruit-level detection methods, although they address different tasks. Qin et al. [40] achieved 85.3% AP for cherry tomato detection using YOLO-CT, focusing on fruit-level recognition rather than pixel-wise pedicel segmentation. Similarly, Chen et al. [41] reported 87.2% precision for cherry tomato bunch identification, emphasizing ripeness classification rather than fine-grained pedicel segmentation. These reference comparisons are provided solely for contextual purposes and are not intended as strictly fair benchmarks.

Real-time inference: Our model achieves an inference time of 4.8 ms, which is 3125 times faster than Yoshida et al. [18] (15 s per image). This dramatic difference stems from fundamental methodological differences: Yoshida et al. relied on hand-crafted geometric features (region growing, voxel connectivity, and Mahalanobis distance) with time-consuming post-processing, whereas our end-to-end deep learning architecture performs all computations in a single forward pass. Similarly, our model is 56 times faster than Wang et al. [12] (270 ms), whose two-stage Mask R-CNN requires sequential region proposal and classification steps, and 2.8 times faster than Sun et al. [42] (13.5 ms), whose S-YOLO focused on fruit-level detection rather than fine-grained pedicel segmentation. Gao et al. [43] proposed YOLOR-Slim, achieving 87.5% mAP₅₀ with only 1.4 M parameters and 1.9 G FLOPs, yet their inference time on workstation is 7.4 ms, which is slower than our 4.8 ms. Our inference speed is comparable to Chen et al. [41] (4.9 ms), but their MTD-YOLOv7 focused on ripeness classification rather than pedicel segmentation. In contrast to Yoshida et al. [18], which required a dedicated high-performance computing unit, and Wang et al. [12], which struggled with embedded deployment. To evaluate real-world deployability, we further tested our model on a Jetson Orin NX edge device (100 TOPS). On this platform, the model achieved 9.3 ms for detection and segmentation (approximately 108 FPS) and 16.2 ms with full post-processing (approximately 62 FPS), both exceeding the 30 FPS real-time requirement for greenhouse harvesting. However, claims about deployment on Jetson Nano, Xavier, or Raspberry Pi have not been verified and are therefore deferred to future work.

Localization accuracy: Our method achieves a localization RMSE of 1.05 cm at a working distance of 25 cm, a typical range for greenhouse picking operations. Zheng et al. [20] reported a maximum average localization error of $\pm 5.0$ mm for tomato centroid localization using binocular stereo vision within 280–480 mm, and reported an RMSE of 4–7 mm on texture-sparse surfaces. Note that these comparisons are provided for reference only, as they involve distinct tasks and experimental conditions. For LiDAR-based systems, Wang et al. [44] developed an integrated LiDAR/IMU/GNSS navigation method for orchard robots, achieving a global localization accuracy of 2.215 cm ($\sigma =1$ cm) in field tests. In contrast to stereo matching methods that are sensitive to illumination changes, our RGB-D fusion method provides stable depth information and does not rely on external positioning signals. These advantages are particularly critical for greenhouse environments, where lighting varies considerably throughout the day and LiDAR deployment is constrained by cost and environmental robustness.

Robustness under occlusion: The proposed model demonstrates strong generalization across diverse cultivars, viewing angles, and occlusion levels, achieving a success rate of 0.89 under partial occlusion. Under severe occlusion, the success rate drops to 0.69 in the worst-case scenario, a 22% decline. This performance is comparable to human performance under identical visual conditions, suggesting the model has reached a practical limit given single-view input. Compared with existing methods, Lu et al. [45] reported an 89.44% success rate for grape pedicel localization under occlusion, Ye et al. [46] achieved an 82% picking success rate for citrus fruit stem estimation, and Burusa et al. [47] achieved an 82.7% detection rate under occlusion using seven optimally planned viewpoints. These comparisons confirm that occlusion remains a fundamental challenge, motivating future work on multi-view fusion to enhance robustness. Representative examples of successful detections under varying occlusion levels are provided in Appendix D (Figure A4).

4.2. Limitations and Future Research Directions

Dataset limitations: The primary limitation is the limited dataset size and diversity. The depth completion threshold was optimized using only 53 samples, and the skeletonization evaluation used 500 samples. The lack of a public pedicel segmentation benchmark limits direct comparison with other methods. This issue is widespread: Jiang et al. [48] noted data scarcity for fine-grained agricultural targets, and Akbar et al. [49] highlighted limited dataset diversity as a key bottleneck in greenhouse vision systems. Our dataset includes only two cultivars under controlled conditions; performance on other cultivars or outdoor settings is unknown. Saiz-Rubio and Rovira-Más [50] emphasized that cross-cultivar and cross-environment data are essential yet rarely available. Image acquisition was also restricted to a narrow time window (10:00–12:00). We acknowledge that model performance under variable lighting has not been systematically evaluated. Images were captured across a wide distance range (10–70 cm), while the target application requires a constant 25–30 cm distance. To address this, we tested the model on 53 samples at 25 cm, achieving 91.87% precision and 82.10% ${\mathrm{mAP}}_{50}$, consistent with the overall test set (92.08% precision, 87.1% ${\mathrm{mAP}}_{50}$). Future work will include multi-temporal data collection (Wang et al. [51]) and a dedicated dataset at the target distance.

Occlusion handling: As shown in Figure 11a, the model fails under severe occlusion (success rate 0.69). This is because occluded pedicels lack visible features, making detection impossible from a single viewpoint. This is not a model deficiency but a fundamental limitation of vision-based perception. As reviewed by Xiao et al. [52], occlusion remains a core challenge in agricultural robotics, where fruits and pedicels are often obscured by branches and leaves, leading to reduced recognition accuracy and positioning failures. Thus, the robot needs to adjust its viewpoint or use multi-sensor fusion to resolve occlusions. Burusa et al. [47] proposed a semantics-aware next-best-view planning strategy, achieving an 82.7% detection rate under heavy occlusion using seven optimally planned viewpoints, significantly outperforming non-semantic methods. This demonstrates that single-view perception is inherently insufficient for occlusion-heavy environments. Beyond occlusion, other failure conditions include motion blur, reflective surfaces, and sensor drift. Motion blur from rapid camera or plant motion may degrade segmentation accuracy, though this has not been systematically quantified. Reflective surfaces can cause specular reflections, leading to invalid depth measurements. Sensor drift over extended operation may affect RGB-depth alignment, especially under temperature fluctuations. Future work will explore multi-view fusion, temporal information, and active perception to address these issues.

Depth completion threshold and error analysis: The optimal threshold ($k=2.5$ cm) was determined at a fixed distance of 25 cm. The depth estimation error comprises a systematic error and a random error. Our method achieves a systematic error of <0.5 cm and a random error of $\pm 0.3$ cm. Depth holes are the primary source of random error that neighborhood-based compensation cannot fully eliminate. Compared with existing methods, our approach performs favorably. A depth completion survey noted that even high-end LiDAR sensors produce sparse and noisy depth maps near object boundaries [53]. SelfReDepth achieves real-time depth restoration but relies on multiple sequential depth frames coupled with color data [54]. In contrast, our method uses a simpler neighborhood-based compensation strategy at 25 cm to achieve high precision (<0.5 cm, $\pm 0.3$ cm), meeting millimeter-level picking requirements while maintaining computational efficiency. Validation at 15 cm and 35 cm showed stable performance (average localization error <1.2 cm, see Appendix D,

Table A9. Experimental results of depth completion at different working distances.

Distance	Number of Samples	Abnormal Depth Rejection Rate (%)	Average Localization Error (cm)
15 cm	48	86.2	1.12
25 cm	53	88.6	1.05
35 cm	49	87.1	1.09

). However, recalibration may be needed for distances outside this range or for different sensor configurations. As shown in Figure 11c, the depth map exhibits significant depth information loss (holes covering >15% of the pedicel neighborhood), affecting depth measurement reliability. Future work will explore adaptive thresholding and learning-based depth completion for severe depth holes, informed by recent studies [53,54].

In summary, while the proposed method achieves promising performance in pedicel segmentation and picking-point localization, several limitations remain regarding dataset diversity, occlusion robustness, and depth completion accuracy. Addressing these limitations through multi-temporal data collection, multi-view fusion, adaptive threshold mechanisms, and learning-based depth completion will be the focus of our future research.

5. Conclusions

We propose an innovative deep learning method for tomato pedicel picking-point localization based on an improved YOLOv8n-seg model and RGB-D fusion. In a greenhouse environment, we build a tomato planting scene and capture multi-view images using an RGB-D camera. To improve YOLOv8n-seg, we introduce an improved EfficientRep backbone with 8-direction shift convolution, integrate an EMAttention mechanism, and add an improved DynamicHead module. Compared to the original model, the proposed YOLOv8n-EED-seg achieves 92.08% precision (up 5.09%) and 87.01% mAP₅₀ (up 4.31%), while maintaining 7.5 MB and 4.8 ms inference. A skeletonization-based method is proposed to extract picking points from segmentation masks, and a neighborhood-based depth compensation strategy is developed to restore missing depth data. Applied to greenhouse tomato picking, the method achieves a 93.88% picking success rate with depth errors bounded to $\pm 1.2$ cm, meeting the speed and accuracy requirements of a tomato-harvesting robot. This method has currently been tested only on two tomato cultivars under controlled conditions; adaptability tests on more cultivars and outdoor scenes are needed. Nevertheless, the proposed method can serve as a reference for other fruit and pedicel recognition tasks in agricultural robotics.