A Fruit Gripping Evaluation System Based on Tactile Fusion Analysis

Abstract

To address the evaluation requirements for agricultural robotic harvesting, this work presented a fruit-grasping assessment system based on tactile fusion analysis. Four piezoresistive pressure sensors were symmetrically integrated into the inner surfaces of a flexible gripper. A signal-conditioning circuit and a data acquisition module transmitted tactile signals to a Transformer–Mamba fusion network for feature extraction and target classification. After being trained on a dataset comprising 300 samples, the model extracted deep tactile features to distinguish among three target categories: citrus fruits, branches, and leaves. Classification outputs generated control commands for a robotic manipulator, enabling obstacle-avoidance retraction and precise harvesting operations. Experimental evaluations, conducted in both indoor and outdoor environments, demonstrated a target recognition accuracy of 93.12%. The manipulator response time was below 0.5 s, and the operational success rate was 90%. The proposed sensing system and algorithmic framework showed strong adaptability and supported quantitative assessment of grasping performance. The fruit detachment, compression damage, and plant-collision risks were effectively reduced while operational stability and harvesting efficiency improved.

1. Introduction

Harvesting is a critical part of fruit and vegetable production, since its efficiency and quality directly affect product quality and the industry’s economic benefits. Agricultural harvesting robots can effectively replace manual labor in fruit and vegetable harvesting, promoting agricultural automation and reducing labor costs. Currently, extensive research on fruit and vegetable harvesting robots has been conducted worldwide, with a primary focus on visual recognition, target localization, and harvesting path planning. By leveraging deep learning, stereo vision, and related technologies, high-precision detection and classification of various fruit and vegetable targets have been achieved [1,2], laying a solid technical foundation for the practical deployment of harvesting robots.

For instance, a sweet pepper harvesting robot developed at Wageningen University employed an RGB-D camera combined with a morphological threshold detection algorithm for fruit recognition and localization, relying solely on vision for harvesting decisions. Experimental results demonstrated a success rate of 61–73% under minimal occlusion, which dropped to 18% in complex natural environments, indicating that the performance is highly sensitive to branches, leaves, and adjacent fruits [3]. Subsequent studies introduced a hand–eye visual servoing framework that improved target localization accuracy through multi-view scanning and visual calibration [4]; however, limitations remained in state perception and process evaluation capabilities. In North America, Chu et al. developed an apple-harvesting robot based on Mask R-CNN with a suppression mechanism, achieving a recognition success rate of 90% across different varieties and lighting conditions and positioning accuracy that met operational requirements [5]. Nevertheless, the integrated harvesting success rate was 64.06%, with fruit detachment and separation difficulties affecting grasp stability significantly [6]. Hayashi et al. developed a strawberry harvesting robot for elevated substrate cultivation, combining binocular vision with an HSI-based maturity evaluation algorithm, achieving accuracies of 95.6% and 82.1% for two target types. However, environmental factors affected harvesting performance, resulting in success rates of 41.3% and 34.9% for suction and non-suction picking, respectively [7]. Zou et al. proposed a hand-eye calibration optimization method based on the Sylvester equation for a visual litchi-harvesting robot, achieving a positioning accuracy of 92.3% and an overall harvesting success rate of 76.5%, with errors primarily attributed to field disturbances and occlusion-induced fruit displacement [8]. In addition, mechanical safety and fruit damage remain critical concerns during continuous operation. Li et al. combined trunk skeleton extraction with an improved RRT-Connect path planning algorithm, achieving an average obstacle-avoidance success rate of 89% [9]. A single-finger flexible gripper was designed to reduce strawberry damage, and its feasibility was validated using pressure sensors, although the grasping failure rate remained high [10]. In general, prior studies shown strong visual recognition performance but limited grasping effectiveness, often evaluating system performance solely by quantity and operation speed, without analyzing grasp stability or fruit damage during harvesting.

To address the limitations of purely visual techniques, tactile perception has emerged as an important research direction in agricultural harvesting robots. Gao et al. evaluated different picking modes to optimize end-effector design, emphasizing the role of tactile feedback in determining optimal grasp points and reducing fruit damage to 5.2% [11]. Han and the team developed a parallel gripper integrated with tactile sensors and quantitatively assessed peach maturity through force–displacement responses during grasping [12], thereby supplementing visual detection and directly capturing mechanical parameters indicative of fruit quality. Zhou et al. were the first to integrate deep learning with tactile sensors in a multi-DOF soft robotic gripper, achieving 89.4% accuracy across multiple grasping scenarios [13]. Robotic hands equipped with tactile sensors can measure thermal properties and contact loads, enabling accurate object recognition [14]. Wang et al. designed a multimodal vision-tactile sensor system. They validated comprehensive object perception from macro to micro scales across three types of fruit, thereby enhancing robotic operation and optimizing tactile technology [15]. More broadly, Cao et al. pointed out that integrating tactile perception with vision and language models has become an inevitable trend [16]. Tactile sensing addresses the limitations of visual detection, especially for assessing fruit damage. However, existing research primarily focuses on force perception, maturity assessment, and the optimization of grasp parameters using tactile data. The application of tactile signals to differentiate among grasped objects and detect branch–leaf interference remains underexplored. This gap restricts the prevention of mechanical collisions, the reduction in fruit compression damage, and the achievement of consistent grasping performance. Additionally, tactile information has not been systematically integrated as an evaluation criterion across the entire harvesting process. Consequently, a noticeable gap still exists between visual recognition accuracy and actual harvesting success. To resolve this problem, this work integrated tactile perception with visual localization to assess grasping quality throughout the harvesting process.

Accurate harvesting control requires tight integration of visual perception and tactile feedback. Therefore, to tackle practical challenges such as fruit damage caused by mis-grasping, insufficient grasp stability, and mismatches between harvesting actions and operating conditions, a tactile-fusion-based fruit grasping assessment method and an associated device were introduced in this work. Focusing on citrus as the target, four tactile sensor modules were integrated into key regions of a four-finger flexible gripper to establish a tactile perception network. The system was designed to recognize branch–leaf interference and classify contact objects. Since most existing studies rely on single temporal networks such as CNN and LSTM for tactile feature extraction, a dual-branch Transformer–Mamba fusion algorithm was developed. The Transformer extracted spatial correlations among multi-channel tactile signals, while Mamba modeled temporal pressure dynamics to improve feature representation. The proposed method enabled accurate classification of contact objects and established a comprehensive evaluation framework for the entire grasping process. Field trials across multiple conditions demonstrated successful discrimination among branches, leaves, and fruits, thereby reducing mispicking and enhancing the performance of intelligent harvesting systems.

2. Signal Processing and Algorithm Design

To enable effective information processing, a Transformer–Mamba fusion model was proposed to extract object-specific features and responses from multi-channel sensor temporal-spatial data. The model adopted a dual-branch architecture, in which the Transformer branch captured global spatial dependencies via self-attention, while the Mamba-based state-space model (SSM) modeled temporal dynamics. A feature fusion module integrated these complementary representations, sharpening the model’s overall capability to capture global spatial dependencies and temporal dynamics jointly. The overall structure was shown in Figure 1, and each module was detailed below.

Figure 1. Transformer–Mamba fusion model architecture. All sub-branches within the Transformer branch adopted identical structures.

2.1. Input Embedding and Initial Projection

The model input consisted of raw sensor signal sequences X ∈ R^B×L×4, where B denoted the batch size, L as the variable sequence length, and 4 as the number of sensor channels. Due to low dimensionality and differing physical meanings across channels, directly feeding the data into deep temporal modules would limit feature representation. Therefore, a learnable linear projection mapped the 4-dimensional features at each time step to a higher model-internal dimension d_model = 64, producing embeddings H(0) ∈ R^B×L×64. This projection enhanced feature representation and provided a unified space for subsequent dual-branch processing. A dropout layer (0.2) was applied to reduce overfitting [17].

2.2. Transformer Branch: Global Dependency Modeling

The Transformer encoder relied on a multi-head self-attention mechanism, which effectively captured long-range dependencies across sequence positions and was well-suited for extracting global contextual information from action segments [18]. Applied to four-channel pressure sensor data, it captured force magnitude, balance, and spatial distribution, computing weights across time steps and channels to extract spatial grasp features.

Internally, the network was configured with a unified 64-dimensional feature space and 8 attention heads, and the data format was adapted for batch temporal sequences. The embedding sequence H⁽⁰⁾ was processed by a stack of N_layers = 3 Transformer encoder blocks. Shallow layers extracted fundamental force features from individual sensor channels, while deeper layers integrated multi-channel data to abstract global spatial distribution features. Each encoder layer included multi-head self-attention with 8 heads, Pre-LayerNorm, feed-forward networks (FFN), and residual connections, with ReLU activation and Dropout(0.2) applied for regularization. After encoding, only the first time-step output h_cls ∈ R^B×64 was retained. Following the BERT [CLS] token design, this output aggregates high-level featured across the entire sequence, providing a compact representation of global spatial information. Once fed into the Transformer, multi-head attention computed correlations across sensor channels and time steps, generating adaptive weights to highlight informative grasp patterns. The attention-weighted features were then passed through a feed-forward network for nonlinear transformation, improving feature expressiveness and enabling discrimination of different grasp states. Finally, the first-time-step output aggregated global spatial information across the entire sequence into a fixed-dimensional representation, completing spatial feature extraction.

2.3. Mamba Branch: Temporal Evolution Modeling

Although the Transformer effectively captured long-range dependencies, its quadratic computational complexity limited deployment in edge computing scenarios and reduces efficiency in modeling continuous state transitions. Mamba, a structured state-space sequence model, enabled efficient modeling of continuous dynamic processes with linear complexity [19]. The module was designed for sensor time-series processing. It extracted local temporal variations using convolutional operations, followed by gated units suppressing noise while retaining informative features. Temporal dependencies were preserved via hidden states, which were recursively updated to model the evolution of continuous dynamic signals. The architecture imposed no-explicit constraint on sequence length, enabling direct processing of variable-length sensor time-series data.

The module was initialized with predefined feature dimensions, state dimensions and convolution kernel configurations. A depth-wise convolution layer was employed to capture local temporal patterns, such as pressure fluctuations and abrupt force variations, while the same padding was adopted to preserve sequence length. For the embedding sequence H⁽⁰⁾, a depth-wise separable convolution with kernel size d_conv = 4 and the number of groups equal to d_model was applied to extract short-term features within local neighborhoods without altering the input length. The convolutional features were subsequently fused with the original residual through a gated linear unit (GLU). Two gating values, g₁ and g₂, were computed and activated by the SiLU function:

SiLU(g₁) ⊙ H⁽⁰⁾ + SiLU(g₂) ⊙ H_conv,

(1)

where ⊙ denoted element-wise (Hadamard) multiplication. This gating mechanism adaptively balanced the contributions of raw and convolutional features, facilitating the extraction of informative temporal patterns while suppressing sensor noise and transient disturbances.

The module relied on the initialized hidden states to traverse the signal along the temporal dimension. State variables were updated recursively using historical memory and real-time sensor data, enabling long-term temporal feature extraction. Residual connections were incorporated to integrate complete temporal information. The outputs captured features that represented dynamic force variations. The core state-space updates followed a discrete-time SSM formulation. For a learnable state transition matrix A ∈ R^{dstate×dmodel} (d_state = 16), linear mappings B: R^dmodel → R^dstate, C: R^dstate → R^dmodel and a skip-connection coefficient D ∈ R^dmodel, A governed the temporal evolution of hidden states, B projected input features into the state space, C mapped hidden states to output features and D established a direct input–output pathway. Together, these parameters enabled iterative state updates and feature extraction, efficiently modeling continuous dynamic changes in sensor signals. For each time step t, the hidden state h_t was updated as:

h_t = h_t−1 ⊙ exp(A) + B(x_t),

(2)

⊙ denoted element-wise (Hadamard) multiplication, and the elements of A were scaled by 1/d_state. The corresponding output at the current step was computed as:

y_t = C(h_t) +D ⊙ x_t,

(3)

The computation iterated along the temporal dimension, naturally supporting sequences of arbitrary length. Outputs from all-time steps were concatenated with the input residual to form the complete Mamba output sequence H_mamba ∈ R^B×L×64. Temporal average pooling was applied to generate a fixed-dimensional global feature f_mamba ∈ R^B×64, effectively preserving overall sequence statistics and providing robustness to temporal shifts.

2.4. Feature Fusion and Object Classification

The 64-dimensional global spatial feature h_cls extracted by the Transformer branch and the 64-dimensional temporal feature f_mamba generated by the Mamba branch encoded complementary aspects of the input sequence. To combine their advantages, the feature vectors were concatenated along the feature dimension, forming a 128-dimensional fusion vector that jointly represented spatial force distribution and temporal pressure variation. The fusion vector was then fed into a two-layer fully connected network. The first layer reduced the feature dimension from 128 to 64, followed by ReLU activation and Dropout (0.2). The second layer maintained the 64-dimensional representation. Through dimensionality reduction and nonlinear transformation, the network produced the final joint representation F_fusion. The fusion network learned the interactions and relative contributions of the two feature branches, generating a unified representation that captures both global spatial distribution and temporal dynamics.

The fused feature F_fusion was subsequently fed into an object classification head for contact target recognition. A single forward passed outputs classification logits z_big ∈ R^B×3, corresponding to three categories, including fruit, branch, and leaf. During training, cross-entropy loss was employed for optimization:

L_total = L_CE (z_big, y_big),

(4)

where L_total denoted the total training loss, L_CE represented the cross-entropy loss, z_big was the predicted output, and y_big was the ground-truth label. By minimizing the discrepancy between predictions and labels during training, the model achieved accurate contact-object classification, providing reliable support for harvesting-operation evaluation.

2.5. Model–Sensor Compatibility Analysis

The compatibility of the proposed model with the four-channel resistive pressure sensors symmetrically arranged on the inner side of the four-finger flexible gripper was evaluated from input specifications, feature logic and signal characteristics. The initial model input dimension was set to 4, matching the number of sensor channels. Each input dimension corresponded to the pressure value measured by a single sensor, allowing raw signals to be fed directly into the network without additional dimensionality transformations. The symmetrical hardware layout aligned with the Transformer’s spatial feature-extraction logic, enabling accurate recognition of both centered and eccentric grasping states. The Mamba accommodated variable-length sequences, consistent with the continuous time-series data collected by the sensors, while filtering interference generated under complex field conditions to recover true force states.

During actual grasping operations, the four sensors reflected both static spatial force distributions and dynamic pressure variation. The Transformer branch analyzed static grasp spatial states, while the Mamba branch predicted the dynamic evolution of forces. The dual-branch collaboration reconstructed the complete interaction between the gripper and the target (fruit, branches, and leaves), effectively supporting grasp evaluation in real orchard scenarios.

Furthermore, sensor data streams often varied in length during operation, with sample counts ranging from 1150 to 1220. Thus, the support for variable-length sequences was essential for real-world deployment. Both branches were designed to handle such inputs: Transformer accepted variable-length sequences via its self-attention mechanism with masked padding. In contrast, the Mamba branch preserved sequence consistency through symmetric padding in convolutional operations and sequence-length-independent SSM updates. An explicit length-truncation mechanism was also incorporated to prevent dimensional inconsistencies arising from numerical boundary errors. These features enabled the model to be deployed directly for real-time processing of streaming sensor signals.

3. Experimental Materials and Methods

3.1. Monitoring and Decision-Making System Design

The system was designed as a real-time monitoring and decision-making framework based on four-channel pressure sensing and embedded data acquisition. High-precision pressure sensors captured force information throughout the harvesting and grasping process. The overall architecture was shown in Figure 2, consisting of three layers: the sensor layer, the data acquisition and transmission layer, and the PC-based processing and control layer.

Figure 2. Overall system architecture.

The sensor layer comprised four flexible thin-film resistive pressure sensors, which detected the pressure distribution generated during contact between the gripper and fruits, leaves, or branches. The data acquisition and transmission layer were centered on a main control chip that used four ADC channels and a DMA transfer module to acquire analog voltages from the four sensors simultaneously. The signals were then converted to USB via a CH340 chip and transmitted to the PC through a USART serial interface. The PC-based processing and control layer ran on a standard PC and was responsible for receiving and storing the serial data stream, constructing training samples, and training the model. During actual grasping, a pre-trained classification model was loaded to perform online inference on real-time pressure data and determine the current grasping state. Control commands were generated according to state-machine logic and sent to the robotic arm via communication protocols to execute the corresponding actions. Figure 3 elaborately illustrated the signal processing workflow of the system.

Figure 3. System Workflow. The arrows indicated the execution flow direction of the flowchart.

The overall system workflow was shown in Figure 4. Upon MCU startup, the ADC and DMA were initialized, and the four pressure sensors’ voltage values were continuously acquired via a cyclic scan. After each acquisition cycle, the DMA transferred the data to the memory buffer. The MCU consolidated the four-channel raw ADC values and transmitted them to the PC over the serial port at 115,200 bps. The PC-side data acquisition and control software supported two modes: sample collection and real-time recognition control. The former was used to construct the training dataset, while the latter was used for online inference and control during actual harvesting operations.

Figure 4. State Transition. The arrows indicated the execution flow direction of the flowchart.

The proposed system implemented the control logic using a finite-state machine (FSM) to drive the robotic arm and gripper to adjust position, thereby achieving stable target recognition and fruit harvesting. The state transition diagram was shown in Figure 4, and the functions of each state were described as follows.

S0: Initial Grasping. The robotic arm moved to the target region. After the gripper closed till a predefined initial pressure threshold was reached, it reopened and read the real-time classification result.

S1: Leaf Detected. The contact target was classified as a leaf. The gripper opened, the robotic arm returned to its initial position, and a new grasping path was replanned before returning to S0.

S2: Branch Detected. The contact target was classified as a branch. The gripper opened, the robotic arm returned to its initial position, and a new grasping path was replanned before returning to S0.

S3: Fruit Detected. The contact target was classified as a fruit. A harvesting command was issued, after which the system entered the terminal state.

The FSM operated using a polling-based scheduling mechanism. After each state prediction was completed, the PC determined the next state based on the current state and predefined transition conditions and transmitted the corresponding command to the robotic arm controller via IP communication. Upon completion of the commanded action, the controller returned a completion flag, allowing the FSM to transition to the next state.

The experimental platform was built on a tracked autonomous mobile chassis, as shown in Figure 5, which provided mobility for field operations. The harvesting manipulator consisted of an X-arm6 six-degree-of-freedom industrial robotic arm (UFACTORY Co., Ltd., Shenzhen, China). In the current implementation, spatial pose planning was executed under an open-loop control scheme to simplify experimental validation. The experiments were performed jointly using the robot arm supporting software UFACTORY Studio V1.9.0 and PyCharm Community Edition 2022.1. A custom-designed parallel four-finger flexible gripper (WHEELTEC Intelligent Technology Co., Ltd., Dongguan, China) was mounted at the end effector, as shown in Figure 6, to perform object grasping and deformation buffering. The gripper reduced compression and tearing damage to fruits and plants while providing a stable platform for the tactile sensing unit.

Figure 5. Experimental platform and harvesting manipulator. (a) Tracked autonomous mobile chassis; (b) X-arm6 six-DOF robotic arm with flexible gripper.

Figure 6. Flexible four-finger gripper. (a) physical prototype; (b) CAD model.

3.2. Tactile Sensing System

The tactile sensing system served as the core unit for grasp-state recognition, contact-object classification, and quantitative evaluation of grasping actions. It primarily consisted of resistive pressure sensors, signal conditioning modules, and a main control unit. Four SF-15-54 flexible resistive pressure sensors, each with an independent signal-conditioning module, were selected (Shenzhen Jinke Electronic Technology Co., Ltd., Shenzhen, China).

Each sensor was thinner than 0.5 mm, making it suitable for installation on the gripper’s inner soft surface. Figure 7 shown the sensor shape and detailed dimensions. The sensors had fast dynamic response and excellent sensitivity with the response time of less than 1 ms and the recovery time less than15 ms. They were rated for over one million press cycles, allowing long-term use in repeated orchard-grasping operations. The sensors were non-polarized components, with a voltage-divider acquisition circuit that converted changes in resistance under applied force into an analog voltage output. The output voltage increased with applied pressure, with the maximum determined by the sensor supply voltage Vcc, providing an effective range of 0–Vcc [15,20].

Figure 7. Dimensions of the tactile sensor.

As illustrated in Figure 8, the characteristic curve of film resistance versus applied pressure indicates significant resistance variation in the 0.5–5 kg range, with good force-to-voltage linearity and high sensitivity. Table 1 listed the detailed data of resistance versus pressure corresponding to Figure 8. Practical testing confirmed that the pressure generated when grasping the three types of targets falls entirely within this range. Therefore, the selected flexible pressure sensors met the system’s measurement requirements. Additionally, when the sensor was subjected to a constant force and maintained under pressure, a slight drift in the output resistance was observed during testing, with a variation of up to 5% over time. The error was reduced through periodic calibration and adjustment of the potentiometer in the signal amplification module, ensuring that the AD value variations in the four sensor channels remained largely consistent. However, because the sensor’s operating conditions may vary across different application scenarios, recalibration and additional testing were required for other tasks.

Figure 8. Tactile Sensor Pressure–Resistance Characteristic Curve.

Table 1. Pressure–Resistance Test Data. Note: All data were measured under standard static pressure conditions.

Parameter	Measured Values
Pressure /kg	0.5	1	1.5	2	2.5	3	3.5	4	4.5	5
Resistance /kΩ	15.12	13.69	13.09	12.66	12.25	11.82	11.43	11.09	10.81	10.6

The main control board was based on the STM32F103C8T6 minimum system (STMicroelectronics, Geneva, Switzerland), which featured multiple independent ADC channels to support high-frequency synchronous acquisition of four tactile analog signals [21]. The development software for microcontroller programming was Keil. The system connected the outputs of the four pressure sensors to ADC1 channels PA0–PA3, respectively. ADC1 was configured in independent, continuous and scan-conversion modes. Using DMA1 Channel 1 in a cyclic manner, all channel conversion results were automatically transferred to the global array AD_Value [4] without CPU intervention. In the main loop, this array was periodically traversed and each ADC value was formatted as a four-digit decimal string (padded with leading zeros) via a custom function Serial_SendNumber (AD_Value[i], 4) and transmitted. The ADC values ranged from 0 to 4095 (12-bit ADC). All hardware nodes in the sensing system were connected via shielded cables to mitigate electromagnetic interference and high-frequency noise from the robotic arm motors. The control system circuit was shown in Figure 9.

Figure 9. Circuit Connections of the Data Acquisition System.

At the current stage of system development, the tactile acquisition system was integrated using a dedicated enclosure to centrally organize the main controller and signal-conditioning modules. Further optimization of the wiring layout and the incorporation of shielded cables will be considered in subsequent system refinement to reduce electromagnetic interference from motor components and improve the stability of long-distance tactile signal transmission.

3.3. Sensor Installation Layout and Calibration

Four resistive pressure sensors were symmetrically attached to the front-central region of the inner gripping surfaces of the four-finger gripper, as shown in Figure 10. To avoid damaging the sensor substrate, insulating electrical tape was used to secure the sensor edges. During installation, a 1.5 cm clearance was maintained between the sensor tip and the gripper fingertip edge, and the tape was applied outside the central sensing area to ensure unrestricted deformation under load.

Figure 10. Sensor Layout and Installation. (a) Sensor layout diagram; (b) Installation process.

For sensor calibration, standard weights were applied to generate controlled preload conditions. The potentiometers of the four signal-conditioning modules were then adjusted to ensure consistent output sensitivity across all channels under both initial and identical loading conditions.

3.4. Experimental Objects

Naturally matured citrus fruits, native branches and clustered leaf groups were selected as the physical interaction objects in the experiments. Figure 11 presented the appearance of the citrus fruit and the corresponding measurement scheme. A total of 100 citrus fruits with intact peels were selected as the primary targets. Their transverse diameters ranged from 48 to 78 mm (average 61.5 mm) while longitudinal diameters from 39 to 80 mm (average 58.6 mm). The average fruit volume was approximately 900 cm³ and the mean weight was about 120 g.

Figure 11. Fruit Sample and Measurement Method.

Branch samples were collected from native citrus trees, with diameters ranging from 5 to 20 mm and a length of approximately 300 mm per sample. Since individual leaves generate only weak deformation reaction forces, the gripper tended to close completely during grasping, making it difficult to capture effective tactile features. Therefore, naturally clustered leaf groups were adopted as the grasping objects to reproduce the realistic mechanical feedback encountered when the gripper mistakenly grasps dense foliage in orchard environments.

3.5. Experimental Methods

3.5.1. Indoor Experiment

Indoor experiments were conducted in a controlled warehouse environment with stable lighting and no wind disturbances, as shown in Figure 12. This setup was primarily used for tactile signal calibration, model training, and control parameter tuning. For non-fruit samples, operators held freshly harvested citrus branches with diameters of 5–20 mm and clusters of leaves, simulating different orientations and intrusion angles into the sensing area of the four-finger gripper to collect diverse branch-and-leaf contact mechanics. For fruit samples, citrus fruits were suspended from a simulated support beam using flexible ropes, preserving their natural growth posture and allowing slight swinging, ensuring that the collected temporal data accurately reflected mechanical transitions during grasping.

Figure 12. Indoor environment setup and close-up.

3.5.2. Outdoor Experiment

Outdoor experiments were conducted in a natural citrus orchard to provide realistic conditions. The test area had slightly uneven soil to assess the stability of the mobile platform and robotic arm on non-flat surfaces, as shown in Figure 13. Experiments were performed outside the citrus harvesting season, with trees devoid of natural fruit. Harvested fruits were fixed to the original branches using ropes to simulate natural fruit-bearing conditions. During operational evaluation, the system executed targeted stops and picking commands via the mobile platform. This setup focused on assessing the tactile perception system under realistic pressure feedback, including branch-and-leaf occlusions, signal interference, and gripper misalignment caused by visual localization errors.

Figure 13. Outdoor environment setup and grasping close-up.

3.5.3. Experimental Procedure

The harvesting-operation evaluation system developed in this work consists of an execution layer and a signal acquisition layer. The execution terminal was a parallel four-finger gripper mounted on the end effector of an Xarm-6 six-degree-of-freedom industrial robotic arm. To simplify data collection, the gripper operated under an open-loop control scheme and opening or closing actions could be triggered through predefined command inputs. The flexible pressure sensors deployed on the inner surfaces of the four gripper fingers followed a resistive sensing mechanism. When the gripper closed and applied force, the sensing layer’s resistance decreased while the output voltage increased. After signal conditioning, the analog voltage signals were sampled in real time through four ADC channels of the microcontroller and synchronously transmitted to the host computer via serial communication. To ensure measurement consistency, the potentiometers of the amplification modules were calibrated before the experiments, ensuring that all sensors exhibited similar response characteristics under identical initial conditions and providing a stable data stream for subsequent temporal feature extraction.

The dataset used for model training included three categories of targets: fruits, branches, and leaves. During data acquisition, the target category was preselected via the host computer interface. To increase grasping diversity, multiple grasping poses were defined for fruit samples according to the relative position between the grasping point and the gripper center. For each sample, a standardized acquisition cycle was performed. After the trigger button was pressed, the gripper executed a 2 s closing action to capture dynamic contact-force characteristics, followed by a 2 s opening action for release. Each acquisition generated a temporal sequence containing approximately 1200 data points (1150–1220 samples). Sufficient sampling points can preserve the force evolution throughout the entire grasping process and the correlations among multi-channel pressure signals, providing comprehensive raw information for the model to capture long-range temporal dependencies and distinguish subtle deformation differences between thin branches and leaves.

The experiments were conducted sequentially in a controlled indoor environment and a natural outdoor orchard scenario to evaluate the robustness of the proposed system under varying illumination conditions and environmental background interference. Indoor data acquisition yielded stable tactile signals with low temporal noise and clear patterns of variation, facilitating high-quality model training. In contrast, the complex outdoor environment enhanced the model’s resistance to interference and improved its reliability under practical conditions.

After data collection, preprocessing was performed, during which a large number of irregularly collected samples and samples with insufficiently distinctive features were manually removed. This process inevitably resulted in a substantial reduction in dataset size. Given that the primary objectives of this work were to validate the feasibility of the tactile sensing-based grasping recognition scheme and to investigate the adaptability of the proposed temporal fusion architecture, a small-scale but high-quality dataset was used for model training and evaluation. The retained temporal samples underwent strict quality control, providing sufficient information density while effectively reducing the negative impact of low-quality noisy samples on model learning. Future studies will establish larger, more diverse datasets for broader tactile applications, such as fruit pose estimation, thereby enabling data sharing, reducing repeated field data collection, and improving sample utilization efficiency.

The initial dataset consisted of 100 fruit samples collected under different grasping poses, together with 50 branch samples and 50 leaf samples. To balance the dataset, a temporal data augmentation method was applied to expand the non-target categories to 100 samples each. The resulting dataset comprised 300 samples, including fruits (100), branches (100), and leaves (100), and was used for classification model training. The dataset was divided into training, validation and test sets in an 8:1:1 ratio [22]. The model was constructed and trained based on the PyTorch deep learning framework under the Python 3.9 environment, and the training and inference computation was accelerated by an NVIDIA GeForce RTX 4070 Laptop GPU. The AdamW optimizer was adopted for model training, with the initial learning rate set to 1 × 10⁻⁴. A StepLR scheduler was employed to reduce the learning rate by 50% every 15 epochs. The maximum number of training epochs was set to 80, with a batch size of 4 and two-step gradient accumulation, resulting in an effective batch size of 8. The global random seed was fixed at 42. Due to the limited scale of the high-quality dataset, the model’s generalization capability may be affected by insufficient coverage of diverse grasping conditions, susceptibility to environmental interference, and potential overfitting. Therefore, two lossless augmentation strategies, Gaussian noise injection and random temporal shifting, were applied to tactile sequences during training. A class-weighted cross-entropy loss was introduced to alleviate recognition bias among easily confused categories, particularly branches and leaves. To mitigate overfitting, L2 weight decay regularization and an early stopping strategy were adopted. Training was terminated when the validation accuracy improvement remained below 0.1% for 20 consecutive epochs, thereby reducing generalization limitations due to limited samples and improving recognition stability.

During real-time harvesting evaluation, the trained Transformer–Mamba fusion model performed contact-object classification. When a branch or a leaf cluster was identified, the system immediately regarded the grasp as invalid. It commanded the robotic arm to return to a predefined position for subsequent path replanning, thereby reducing plant-surface damage and mechanical wear caused by ineffective grasping. When a fruit was identified, the robotic arm remained stationary while the gripper maintained its closed position to prepare for harvesting. This process enabled tactile-based target recognition, mis-grasp correction and intelligent harvesting operations.

3.6. Evaluation Metrics

This work established a comprehensive evaluation framework from offline algorithm verification to physical execution feedback to systematically assess perception accuracy and decision-making effectiveness in fruit harvesting operations. For the Transformer–Mamba fusion model’s performance in object classification, accuracy and recall were adopted as the core statistical metrics. They were defined as follows:

1. Object Classification Accuracy (Acc_obj):

$${\mathit{Acc}}_{\mathit{obj}}={\displaystyle \frac{N_{\mathit{correct}\_\mathit{obj}}}{N_{\mathit{total}}}} \times 100\%,$$

(5)

where N_total represented the total number of samples, N_{correct_obj} symbolized the number of correctly predicted object categories (fruit, branch, or leaf).

2. Fruit Capture Recall (R_fruit):

To quantify the completeness of the system in recognizing targets under complex backgrounds and to prevent fruits from being misclassified as branches or leaves, recall was defined as:

$$R_{\mathit{fruit} }= {\displaystyle \frac{T{P}_{\mathit{fruit}}}{T{P}_{\mathit{fruit}} + F{N}_{\mathit{fruit}}}} \times 100\%,$$

(6)

where TP_fruit was the number of samples correctly identified as fruit, and FN_fruit was the number of fruit samples incorrectly classified as branches or leaves.

The physical standard for prediction accuracy served as the primary basis for evaluating the system’s grasp judgment and control capability. A prediction was considered correct if the predicted label matched the actual target grasped. If, during fruit grasping, environmental complexity caused the sensors to mistakenly contact large or stacked branches and triggering an incorrect retract action, the prediction was counted as a failure. The work also specifically considered analysis and evaluation under special interference conditions during harvesting. Such conditions primarily included large initial gripper position deviations caused by coarse visual localization errors that prevented effective fruit coverage, dynamic slip-induced misjudgment due to smooth fruit surfaces and insufficient contact friction, and significant fluctuations in recognition signals when fruits and branches were simultaneously contacted. Data collected and used for model training and testing focused exclusively on the target’s normal grasping states. Data under these special mixed conditions were uniformly excluded.

The current work aims to establish a fundamental tactile perception framework and validate the feasibility of tactile information transmission and the adaptability of the fusion model architecture. Therefore, following a progressive strategy from basic perception to operational optimization, this work primarily focuses on the accurate recognition and state classification of three target categories under standard grasping conditions: fruits, branches, and leaves. This design ensured that the model can stably learn the spatial pressure distributions and temporal evolution patterns associated with pure target contact, enabling basic harvesting object classification and preliminary evaluation.

Future research will further expand the evaluation dimensions by incorporating grasp stability, fruit damage rate, and harvesting efficiency as performance indicators. Dedicated algorithm optimization and model refinement will be conducted in complex mixed-contact scenarios to improve the system’s generalization capability and operational robustness under severe field disturbances.

4. Results and Discussion

4.1. Signal Acquisition and Discussion

During real-time grasping, the pressure sensors sensed the forces generated by gripper closure and object contact, converting tactile stimuli into weak electrical signals. As described previously, the system, in conjunction with the host computer, enabled real-time data collection. To reduce data transmission redundancy and memory usage while improving communication efficiency [23], a grasp-state-triggered mechanism was employed, activating the serial data stream only when effective contact was detected, i.e., upon entering the grasping detection phase, thereby significantly compressing idle communication periods. Regarding the choice of transmission baud rate, a conservative rate of 9600 was initially tested, but data acquisition during grasping yielded fewer than 20 samples per trial, indicating extremely low transmission efficiency. The baud rate was subsequently increased to 115,200, resulting in approximately 1180–1220 samples per acquisition cycle. Across multiple consecutive trials, the signal acquisition and communication system remained stable. The host computer received continuous, complete data streams with no packet loss, garbled characters, delays, or interruptions. Statistical analysis indicated that the effective data acquisition rate exceeded 95% under both indoor and outdoor environments, demonstrating high reliability and real-time performance. These results confirmed that the system could support precise tactile signal transmission, rapid prediction by deep learning models, and coordinated control of the robotic arm [24].

During the system’s real-time acquisition sequence, consisting of a 2 s closure followed by a 2 s opening, the pressure signals exhibited distinct temporal features across different targets [25]. Figure 14 illustrated the tactile information of the three target objects across spatial and temporal dimensions.

Figure 14. Temporal–Spatial ADC Characteristics of Three Object Categories. (a) Fruit; (b) Branch; (c) Leaf.

During fruit grasping, the measured channel pressure curves started from an initial AD value of approximately 100 and gradually increased with gripper closure, reaching a range of 600–2000 before stabilizing. Upon opening the gripper, the pressure values gradually returned to baseline, fully reflecting the contact, deformation, and release processes of the flexible citrus fruit.

During branch grasping, the pressure rise began later than for fruits due to size differences but increased at a faster rate, reflecting the higher hardness and smaller deformation of branches. The pressure values quickly reached a high and stable level. Meanwhile, the elongated branch structure caused slight bending and elastic recovery during grasping, resulting in fluctuations in the signal across the rising, steady, and falling phases. For clustered leaves, the signals showed minimal variation. Gripper closure and opening caused only minor fluctuations in the pressure curves, with amplitude changes far lower than those observed for fruits and branches.

Based on the spatial distribution characteristics of the four pressure sensors, the gripper fingers were divided into upper-left, lower-left, lower-right, and upper-right positions, corresponding to sensors 1–4, respectively. Therefore, the distinct response patterns of different object categories provided an additional basis for classification.

For fruit samples, three or more sensors typically exhibited significant pressure responses simultaneously when the gripper achieved a centered grasp. For branch samples, pressure responses were generally observed in only one or two sensors, although slight branch bending occasionally induced fluctuations in other channels. For leaf-cluster samples, only weak responses from a single sensor were typically observed, and in some cases, no obvious signal variation was observed. This phenomenon was consistent with the soft and highly deformable nature of clustered leaves. Tactile sensing can complement the missing contact-mechanical temporal features in visual perception by providing physical information such as grasping deformation and spatial force distribution. This enables multimodal fusion between visual and tactile information, overcoming the limitations of purely visual perception under field conditions, including occlusion and illumination variations. Meanwhile, the tactile signals collected in this work were well-suited to the processing characteristics of the Transformer–Mamba model, providing experimental evidence for deep learning-based feature extraction and category optimization [26].

4.2. Model Comparison Experiments

To verify the effectiveness of the proposed Transformer–Mamba fusion model for tactile time-series classification, comparative experiments were conducted on the same dataset using the same training–testing partition strategy. K-nearest neighbors (KNN), support vector machine (SVM), temporal convolutional network (TCN), and long short-term memory network (LSTM) were selected for comparative experiments, with prediction accuracy on the test set used as the evaluation metric. Since SVM and KNN cannot directly process continuous sensor sequences, the original tactile signals must be compressed into fixed-dimensional handcrafted features, thereby losing pressure dynamics and inter-channel force correlations. LSTM relied on a unidirectional recurrent gating structure, enabling only one-way temporal dependency modeling and limiting its ability to capture global long-range temporal correlations and multi-channel feature interactions simultaneously. TCN was constructed using stacked causal one-dimensional convolutions, which primarily captured short-range local temporal variations via limited receptive fields and lack comprehensive global dynamic modeling capabilities. Both methods primarily extracted local or unidirectional temporal features, resulting in inherent limitations. In contrast, the proposed fusion model directly accepted variable-length raw sequences, preserving complete spatiotemporal information for classification [27].

As shown in Table 2, SVM, KNN, TCN and LSTM achieved overall recognition accuracies of 0.76, 0.80, 0.77 and 0.83, respectively. The proposed Transformer–Mamba fusion model achieved the highest test accuracy of 0.87. It also obtained an MCC of 0.82 and a weighted F1-score of 0.87. All global evaluation metrics outperformed those of the comparison models. These results indicated that the fusion model exhibited better compatibility with the tactile sensing system and a higher potential for feature representation.

Table 2. Comparison of Training Performance Among Different Algorithms.

Model	Overall Accuracy	Matthews Correlation Coefficient (MCC)	Weighted F1-Score	Specificity
Transformer-mamba	0.87	0.82	0.87	0.93
LSTM	0.83	0.76	0.83	0.92
TCN	0.77	0.67	0.74	0.88
SVM	0.76	0.70	0.72	0.88
KNN	0.80	0.74	0.77	0.90

As shown in Figure 15, a detailed analysis of the test set confusion matrix was conducted. Fruit samples achieved high recognition accuracy across all five models, with almost all samples correctly classified, demonstrating the effectiveness of the tactile sensing system in capturing object information during grasping. Branch samples remained the main recognition challenge for the comparison models. SVM and KNN achieved branch recall rates of only 30% and 40%, respectively, while LSTM and TCN reached 60% and 40%, respectively. In contrast, the proposed Transformer–Mamba fusion model achieved a branch recall rate close to 100%. For leaf samples, the proposed model achieved a test accuracy of 70%. The slightly lower performance was mainly attributed to complex contact conditions, overlapping leaf postures, and increased tactile signal noise. Further optimization will be conducted to improve the recognition of subtle force variations.

Figure 15. Confusion Matrices of Different Recognition Models. (a) Transformer-Mamba Fusion Model; (b) SVM; (c) KNN; (d) TCN; (e) LSTM.

Overall, the proposed Transformer–Mamba architecture effectively exploited both spatial and temporal characteristics of tactile signals while matching the intrinsic properties of the sensor data. Its feature-selection mechanism further suppressed noise and enhanced feature extraction, resulting in consistently strong performance across all three object categories. In comparison, the LSTM model also achieved competitive performance across multiple metrics and can serve as an alternative architecture for the current task. However, the proposed fusion model exhibited better overall performance and greater potential, providing advantages for further applications such as target position discrimination. The superior predictive performance demonstrated the feasibility and generalization capability of the proposed fusion model, trained on a moderate-scale yet high-quality dataset, for tactile recognition and evaluation of grasping in fruit-harvesting robots.

To investigate the contributions of the Transformer and Mamba branches to the classification task and to validate the superiority of the fusion model, two ablation models were constructed by retaining only the Transformer or Mamba branch. All experiments were conducted using the same dataset partition and training configurations to ensure that the branch structure was the only variable. The recognition performance before and after branch removal was compared using the same test set.

According to Table 3, the fusion model achieved better overall performance than the single-branch ablation models. The Transformer-only model showed the largest performance degradation, primarily because it focused on global feature extraction and was less effective at capturing subtle transient force variations during branch and leaf grasping. The Mamba-only model outperformed the Transformer-only model but remained inferior to the complete fusion model due to its limited ability to capture global spatial correlations.

Table 3. Ablation study results of the proposed model.

Model	Overall Accuracy	Matthews Correlation Coefficient (MCC)	Weighted F1-Score	Specificity
Transformer-mamba	0.87	0.82	0.87	0.93
Only Transformer	0.70	0.56	0.70	0.85
Only Mamba	0.83	0.77	0.84	0.92

The results indicated that Transformer and Mamba provide complementary features, with the former extracting global spatial contact information and the latter capturing local temporal deformation patterns. Removing either branch reduced accuracy, MCC, and specificity, confirming the effectiveness of the proposed fusion architecture.

4.3. Real-World Grasping Classification Results

To further evaluate the object classification capability of the proposed tactile perception model, real grasping experiments were conducted on citrus fruits, branches, and leaves in both indoor and outdoor environments using samples independent of the training dataset. Twenty valid branch samples and twenty leaf samples were collected under natural conditions to assess the model’s robustness against environmental disturbances and variations in plant morphology. During branch testing, the gripper occasionally contacted surrounding leaves due to the dense canopy. However, branch-induced tactile responses were generally more pronounced than leaf signals, and the superimposed leaf responses rarely affected branch recognition. Therefore, only branch interference was carefully avoided during leaf-sample collection. A total of 120 fruit samples, representing typical harvesting scenarios, were selected to evaluate the stability of classification across different grasping states. Half of the validation experiments were conducted in a controlled indoor environment, and the other half in a natural orchard setting, allowing assessment of both detection accuracy and environmental robustness [28].

Figure 16 presented the classification results for different targets. Among the 20 branch samples, 17 were correctly identified, resulting in an accuracy of 85.00%. Among the 20 leaf samples, 19 were correctly classified, yielding an accuracy of 95.00%. During leaf grasping, the pressure signals exhibited only slight fluctuations throughout the grasping cycle and rarely formed a stable force plateau. This response pattern differed significantly from those of fruits and branches, resulting in highly distinguishable tactile features and superior recognition performance. Some branch misclassifications occurred because thin branches exhibited greater flexibility and deformation under loading, producing tactile patterns partially similar to those of leaves. In addition, leaves were naturally attached to branches, making it difficult to distinguish between the two categories during grasping.

Figure 16. Statistics of Tactile Classification Results for Different Grasped Targets.

For fruit samples, 113 of the 120 trials were correctly identified, while only seven samples were misclassified. The high recognition rate verified the effectiveness of the four-channel tactile sensing scheme in capturing force distributions from multiple perspectives. It also showed that the complementary advantages of the Transformer and Mamba branches helped extract discriminative features and suppress abnormal signal interference, thereby contributing to stable classification performance. Despite environmental disturbances and sample variability, the proposed model reliably distinguished fruits from branches and leaves.

Overall, the system achieved an object classification accuracy of 93.12% and a fruit recall rate of 94.16% on 160 test samples, indicating that the proposed tactile perception framework effectively met the requirements for harvesting object identification. The remaining errors were mainly attributed to grasp-position deviations, variations in sample placement and the similarity of tactile characteristics among thin branches, withered leaves and small fruits, which occasionally resulted in category confusion. The 95% confidence interval for the overall classification accuracy was calculated using the Wilson score interval for binomial proportions, yielding [0.8802, 0.9618]. For the 30-sample offline test set, the overall classification accuracy was 0.8667, with a 95% confidence interval of [0.7036, 0.9471]. A chi-square test of independence was further performed using a 2 × 2 contingency table to compare the offline test results with real-world validation results [29]. The test yielded χ² = 0.842, d_f = 1, and p = 0.3587 (>0.05), indicating no statistically significant difference between the two scenarios. Representative grasping scenarios were shown in Figure 17.

Figure 17. Grasping Experiments on Different Object Categories in Indoor and Outdoor Environments. (a) Indoor fruit grasping; (b) Indoor branch grasping; (c) Indoor leaf grasping; (d) Outdoor no-load grasping; (e) Outdoor fruit grasping; (f) Outdoor branch grasping; (g) Outdoor leaf grasping.

4.4. Robotic Arm Cooperative Control Performance

Following object identification, the classification results were used as control inputs to drive the robotic arm to perform corresponding actions. For harvesting purposes, branch and leaf targets were assigned to the same control category. Once either was detected, the robotic arm immediately retreated to a predefined safe Cartesian position:

SAFE_BACK_POS = [391.34, 43.86, −114.57, 178.01, −17.13, 11.15],

the first three values represented Cartesian coordinates (mm) and the last three denoted joint angles (°). When a citrus fruit was identified, the system considered the grasp valid and used tactile feedback to support adaptive posture adjustments. If a standard grasping state was recognized, the gripper closed smoothly to initiate the harvesting operation [30].

Experimental results showed that the robotic arm responded within approximately 0.5 s after the model generated a prediction, achieving a control execution rate of 90%. The system reliably executed commands generated by the classification results and successfully performed precise fruit grasping and obstacle avoidance. The few control failures were mainly attributed to unstable electrical signal transmission and occasional stalling of robotic joints. Representative control execution results after target recognition were shown in Figure 18 and Figure 19. At the current research stage, deployment tests were conducted to verify the compatibility between the proposed system and robotic platforms. The model weight file occupied only 3.3 MB and could run on an embedded robotic device. The average inference time per sample was approximately 152.4 ms, with a peak GPU memory usage of 93.38 MB, indicating that the system response speed satisfied the requirements of basic recognition tasks. However, systematic engineering optimization and comprehensive performance evaluation for edge deployment scenarios have not yet been conducted and will be investigated in future work. The real-time recognition process was successfully executed on a conventional PC and lightweight embedded platforms, such as Raspberry Pi-based development boards. The obtained model size was related to the scale of the training dataset. With further dataset expansion and large-scale training, the model weight file may increase slightly but would be expected to remain compatible with the existing hardware architecture. Therefore, the proposed system was unlikely to be constrained in practical deployment under conventional orchard environments.

Figure 18. Collaborative Control Response to Non-Fruit Identification. (a) Closed grasping of leaves; (b) Closed grasping of branches; (c) Opening and data analysis; (d) Command issuing and returning to preset position.

Figure 19. Collaborative Control Response to Fruit Identification. (a) Closed grasping of fruit; (b) Opening and data analysis; (c) Command issuing and stable closing.

The proposed system utilized tactile time-series signals as the primary feedback source for grasp-state evaluation, designating the grasping point as the coordinate origin. Serial robotic manipulators with varying degrees of freedom, including six-axis, five-axis, and three-axis configurations, can operate within a unified spatial reference framework and generate motion commands based on their respective kinematic structures. Consequently, the tactile evaluation strategy was not restricted to a specific robotic platform, thereby enhancing system adaptability. Additionally, the tactile acquisition circuit was designed as an independent external module, operating separately from the robot motion-control system to reduce potential interference and improve operational stability.

This flexibility enabled the proposed framework to be adapted for a wide range of agricultural applications. In this work, citrus was selected as a representative fruit to validate the tactile perception and evaluation strategy. The gripper and sensing configurations can be modified to accommodate various target characteristics and grasping requirements. For spherical fruits such as apples and pears, the current multi-finger gripper and tactile sensing configuration could serve as a reference. For smaller fruits, including strawberries and kiwifruits, three-finger or cross-shaped grippers with corresponding tactile channels would be adopted. For elongated fruits, additional sensing units may be integrated, ranging from single-finger dual-sensor structures to six-channel tactile configurations, to obtain more comprehensive contact information. Beyond harvesting robots, the tactile sensing module, with additional protection and adaptation, could be incorporated into agricultural machinery for related physical-interaction tasks such as weeding and pruning. When combined with visual perception, the framework could be extended to multimodal applications, including seedling transplantation and post-harvest sorting. Consistent performance in both indoor and outdoor conditions highlighted the proposed framework’s potential across diverse agricultural environments. Figure 20 illustrated the schematic diagram of tactile signals corresponding to different objects during the grasping process for better comprehension.

Figure 20. Schematic Illustration of Tactile Characteristics for Fruit, Branch, and Leaf Grasping.

5. Conclusions

This work designed and developed a fruit-gripping-action evaluation system based on tactile fusion analysis. Four flexible film pressure sensors were symmetrically integrated into a four-finger flexible gripper. A signal-conditioning circuit and an STM32-based data acquisition unit were developed to collect and transmit tactile information. The system captured temporal pressure signals and spatial force distributions during contact with citrus fruits, branches, and leaves. A Transformer–Mamba fusion algorithm extracted features from multi-channel tactile time-series data. These features were used to establish an object-classification framework to distinguish harvesting targets from branch–leaf obstacles and identify fruits.

Experimental results showed that the system maintained stable signal transmission, with effective data acquisition rates exceeding 95% in both indoor and outdoor environments. The overall object recognition accuracy reached 93.12%. Based on the model outputs, the robotic arm successfully performed stable fruit grasping and obstacle-avoidance retreat actions, achieving an average control response time of approximately 0.5 s and an operational success rate of 90%. These results demonstrated that the proposed system could effectively reduce the risks of mechanical collision and fruit compression damage during harvesting while improving grasping stability. Misclassifications mainly resulted from the limited discriminative ability between fine branches and withered leaves, as well as the close attachment of branches and leaves under certain operating conditions. In addition, further expansion of the dataset and validation under more complex orchard environments were still required. Future work would focus on expanding multi-condition datasets, improving feature extraction capabilities, and enhancing recognition robustness to severe branch–leaf interference.

As an effective complement to vision-based perception, the proposed tactile sensing scheme enabled the evaluation of harvesting quality throughout the gripping process. The system effectively mitigated the effects of environmental factors, such as illumination variation and branch–leaf occlusion, while reducing fruit damage from mechanical interactions. The findings provided a reference for tactile sensor deployment, feature recognition algorithms and cooperative control of robotic arms in automated harvesting robots, as well as practical parameters on developing intelligent harvesting equipment for spherical fruits and vegetables.