Design of a Chili Pepper Harvesting Device for Hilly Chili Fields

Abstract

To address issues such as leaf occlusion, misalignment of the harvesting robotic arm, and limited harvesting range in hillside chili fields, this paper designs an intelligent harvesting system based on 3D point cloud reconstruction and multi-mechanism collaborative leveling. The system integrates real-time data from a LiDAR and IMU inertial navigation system to reconstruct the chili point cloud occluded by leaves from multiple perspectives. To address issues such as misalignment of the robotic arm caused by terrain undulations, the system integrates an adaptive leveling platform and an H-shaped planar slide, combined with a gyroscope to dynamically adjust the arm’s posture in real time, ensuring arm stability while expanding its workspace. In addition, to ensure harvesting efficiency and pepper integrity, an integrated cutting–gripping flexible end effector is designed to achieve synchronized cutting and collection operations. The experiment shows that the system achieves recognition accuracy of 81.95% for occluded chili peppers and 89.04% for non-occluded chili peppers. The harvesting success rate is 86.33%, with a single harvesting operation taking 13.17 s. During prolonged operation, the harvesting success rate can be maintained at approximately 85.1%. In summary, the intelligent harvesting system based on 3D point cloud reconstruction and multi-mechanism collaborative leveling provides a feasible solution for automated pepper harvesting.

1. Introduction

Since the 21st century, the chili pepper industry has developed rapidly, and the chili pepper production in China has continued to grow [1,2,3,4,5]. In the decade since 2015, the annual planting area of chili peppers remained stable at over 2.1333 million hectares [6]. In addition, the chili pepper industry contributes 1.14% to the income of Chinese farmers, with an annual output value reaching 70 billion yuan [7]. As a result, chili pepper cultivation has become an important source of income for farmers in China. However, chili pepper harvesting still primarily relies on manual labor [8]. With the increase in production and labor costs, the production costs of chili peppers have continued to rise [9]. Currently, there is no mature intelligent harvesting platform for chili peppers. Therefore, exploring the importance of intelligent harvesting platforms in chili pepper harvesting is of significant practical importance and application potential.

In this context, with the continuous advancement of technology, the application of artificial intelligence in the field of smart agriculture has become increasingly widespread, providing solid technical support for the realization of intelligent chili pepper harvesting platforms [10,11]. In the context of fruit and vegetable harvesting, a 3D radar-based environmental perception system can construct high-density point cloud datasets in real time [12], and by extracting surface features of different objects from the point cloud data and performing object detection, it facilitates further environmental analysis, thus enabling precise environmental construction and providing reliable spatial information support for accurate robotic arm harvesting [13,14]. Compared to traditional vision-based methods, 3D radar demonstrates significant advantages in complex lighting conditions, ensuring the stability and reliability of the perception system [15,16]. The experimental area is primarily characterized by hilly terrain, and the chili pepper planting environment exhibits significant topographical complexity, presenting two major technical challenges for automated harvesting. The first involves the accurate localization of chili peppers, while the second concerns the stable and precise harvesting based on their detection [17]. To achieve high-precision detection and localization of target objects, significant progress has been made in the application of LiDAR (Light Detection and Ranging) technology in the academic community. Liu et al. obtained point cloud data of fruit trees using 3D LiDAR [18], achieving accurate recognition of the trees. Wang et al. innovatively applied the SDF point cloud surface reconstruction algorithm to achieve high-precision geometric reconstruction [19]. Liu et al. extracted and fused features at different scales using feature embedding layers and Transformer layers [20], effectively addressing the issue of point cloud data detail completion and optimization. In addition, Yang et al. trained a model on the U2-net network and then applied the MVS algorithm for fine-scale 3D reconstruction of plants [21]. In harvesting operations, although point cloud data obtained from 3D LiDAR, combined with various algorithms, can effectively handle noise and achieve accurate recognition and localization of target objects, achieving efficient and precise harvesting operations remains one of the core challenges in current research [22,23]. For operations in hilly terrain, robotic arms integrated with adaptive leveling systems can effectively address the misalignment issues caused by terrain undulations, thereby improving the stability of harvesting operations. Yang et al. designed a hydraulic leveling mechanism based on a planar linkage bidirectional active adjustment system [24], which simultaneously controls the tilt angles of the platform in both the longitudinal and lateral directions, ensuring rapid response and stability during the leveling process. Peng et al. designed a dual hydraulic cylinder synchronous lifting system, which is equipped with a connection device featuring lifting functionality [25]. The citrus harvesting platform designed by Bao et al. is equipped with an adaptive leveling platform [26], utilizing data obtained from a three-axis sensor to control the balance by adjusting the extension of two electric cylinders.

However, despite the proposal of various harvesting solutions by several scholars, there are still limitations in their application in hilly chili pepper fields. Cao et al. used the YOLOv5n model to identify crop fruits [27], but this method is not only susceptible to lighting and occlusion effects but also requires improvements in detection speed. The hilly citrus harvesting system designed by Wang et al. constructed a robotic arm and depth camera [28], but due to the absence of an integrated leveling platform, the positioning accuracy of the robotic arm’s end effector is limited when the system operates in hilly terrain. The citrus harvesting platform designed by Bao et al. is equipped with an adaptive leveling platform [26], which controls balance by adjusting the extension of two electric cylinders. However, there is still room for optimization in controlling the accuracy of the balance adjustment. The dual robotic arm collaborative working mode harvesting platform designed by Gong et al. faces the issue of high hardware costs [29]. Building on previous research, this paper preprocesses the obtained 3D LiDAR point cloud data using an improved DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm and then employs EKF (Extended Kalman Filtering) to fuse the 3D point cloud data with IMU (Inertial Measurement Unit) data for precise pose estimation, enabling point cloud registration and reconstruction. Additionally, an adaptive leveling platform is incorporated, which dynamically adjusts the robotic arm’s posture based on real-time gyroscope data [30], further ensuring the stability of the robotic arm in a hilly environment [31,32]. Furthermore, an H-shaped planar slide is designed to expand the robotic arm’s workspace without the need to increase the number of robotic arms. Finally, to ensure both harvesting efficiency and the integrity of the chili peppers, the end effector of the robotic arm is designed as an integrated cutting–gripping flexible end effector, enabling the simultaneous cutting and collection of chili peppers.

The main contributions of the proposed intelligent chili pepper harvesting device are as follows:

• Using 3D LiDAR-generated point cloud data, combined with point cloud preprocessing, to enable the reconstruction of chili pepper point cloud data.
• Applying morphological analysis to identify the chili pepper stem location and precisely determine the cutting position.
• Achieving dynamic balancing of the robotic arm through an adaptive leveling platform and positioning the arm at the optimal harvesting location using the H-shaped planar slide, accurately completing the cutting, gripping, and collection processes.

2. Materials and Methods

2.1. Overall Design of the Chili Pepper Harvesting System

According to the ‘Technical Regulations for Land Use Investigation’, chili fields are classified based on ground slope into flat land (≤2°, 2°~6°), gentle slopes (6°~15°, 15°~25°), and steep land (>25°) [33,34]. Based on this data, the experimental site for the device developed in this study is located in Fogang County, Qingyuan City (23°76′ N, 113°32′ E), in a chili field using the mulching method, as shown in Figure 1. The ground slope typically ranges from 5° to 15°, classifying it as a gentle slope. To address the issues of robotic arm origin displacement and reduced working space caused by the gentle slope, this paper designs an adaptive leveling platform and an H-shaped planar slide, which autonomously levels the robotic arm and expands the overall workspace of the system. Furthermore, to ensure the integrity of the chili peppers, a torque-controllable integrated cutting–gripping flexible end effector is designed.

2.1.1. Design of the Harvesting Platform

When the robotic arm is in a tilted position, also known as a singular configuration, the probability of the chili peppers appearing at the boundary of its workspace increases, making it difficult to achieve accurate harvesting. To ensure that the robotic arm remains balanced during operation and reduce the probability of reaching a singular configuration, this paper designs an adaptive leveling platform with a range of [−15°, 15°]. The adaptive leveling platform is designed to accommodate the undulating terrain between chili pepper field ridges, ensuring that the robotic arm remains level with the ground throughout the harvesting process. Additionally, due to the limited harvesting space of a conventional robotic arm, an H-shaped planar slide is designed to support the robotic arm, further expanding its workspace, as shown in Figure 2.

This paper expands the workspace of the robotic arm through the use of an H-shaped planar slide. The actual working range of the H-shaped planar slide is 350 mm × 1000 mm, consisting of two synchronous belt slides with a stroke of 1000 mm and one synchronous belt slide with a stroke of 350 mm. All slides are from the XINRX RXP45 series (Ruixin Technology, Chengdu, China), with a horizontal maximum load capacity of 15 kg. The selected drive motor is a 57-stepper motor with a holding torque of 2.8 N·m. The two 1000 mm stroke synchronous belt slides are placed at the bottom, while the 350 mm stroke synchronous belt slide, equipped with the adaptive platform, is vertically mounted above them. The motors drive the corresponding synchronous belts to achieve free movement in their respective directions. The synchronous belt slides work together to allow the adaptive platform to move freely within its active range.

This paper achieves the balancing task by designing a miniature Stewart parallel six-degree-of-freedom (DOF) swinging platform. Traditional Stewart parallel six-DOF platforms typically use hydraulic telescopic cylinders for balance control, which face issues such as high cost, slow response, and large size. In contrast, the miniature Stewart parallel six-DOF swinging platform uses a combination of servo motors and universal joints to replace the telescopic function of hydraulic cylinders. This design offers faster response, smaller size, and enables precise position and posture control in three-dimensional space, better meeting the operational requirements of the robotic arm.

The miniature Stewart parallel six-degree-of-freedom swinging platform consists of a fixed base platform, a movable upper platform, fisheye universal bearing linkages, servo motor arms, and a gyroscope inertial navigation sensor. The servo motor arms and fisheye bearings, six of each, are paired and installed at the six vertices of the regular hexagonal base platform and the movable upper platform to ensure the stability of the movable upper platform. The gyroscope is mounted at the center of the upper platform, and by controlling the rotation angles of the six servo motor arms, it lifts the fisheye universal bearing linkages, thereby controlling the tilt angle of the movable upper platform for balance.

The structure of the H-shaped planar slide and the adaptive leveling platform is shown in Figure 3.

This paper uses a motion model to calculate the initial pose of the platform, which corresponds to the default position of the platform when it is in a state of no rotation and no translation. Using the kinematic model and polar coordinate formulas, the following can be derived:

$${\mathrm{R}}_{\mathrm{P}}=\mathrm{R}\text{\_}\mathrm{P}\times \left[\begin{array}{cccccc}\cos ({\displaystyle \frac{\mathrm{\pi }}{3}}-{\mathrm{\alpha }}_{\mathrm{P}})& \cos ({\displaystyle \frac{\mathrm{\pi }}{3}}-{\mathrm{\alpha }}_{\mathrm{P}})& \cos ({\displaystyle \frac{\mathrm{\pi }}{3}}-{\mathrm{\alpha }}_{\mathrm{P}})& -\cos ({\mathrm{\alpha }}_{\mathrm{P}})& -\cos ({\mathrm{\alpha }}_{\mathrm{P}})& \cos ({\displaystyle \frac{\mathrm{\pi }}{3}}+{\mathrm{\alpha }}_{\mathrm{P}})\\ -\sin ({\displaystyle \frac{\mathrm{\pi }}{3}}-{\mathrm{\alpha }}_{\mathrm{P}})& \sin ({\displaystyle \frac{\mathrm{\pi }}{3}}-{\mathrm{\alpha }}_{\mathrm{P}})& \sin ({\displaystyle \frac{\mathrm{\pi }}{3}}-{\mathrm{\alpha }}_{\mathrm{P}})& \sin ({\mathrm{\alpha }}_{\mathrm{P}})& -\sin ({\mathrm{\alpha }}_{\mathrm{P}})& -\sin ({\displaystyle \frac{\mathrm{\pi }}{3}}+{\mathrm{\alpha }}_{\mathrm{P}})\\ 0& 0& 0& 0& 0& 0\end{array}\right]$$

(1)

$${\mathrm{R}}_{\mathrm{B}}=\mathrm{R}\text{\_}\mathrm{B}\times \left[\begin{array}{cccccc}\cos ({\mathrm{\alpha }}_{\mathrm{B}})& \cos ({\mathrm{\alpha }}_{\mathrm{B}})& -\cos ({\displaystyle \frac{\mathrm{\pi }}{3}}+{\mathrm{\alpha }}_{\mathrm{B}})& -\cos ({\displaystyle \frac{\mathrm{\pi }}{3}}-{\mathrm{\alpha }}_{\mathrm{B}})& -\cos ({\displaystyle \frac{\mathrm{\pi }}{3}}-{\mathrm{\alpha }}_{\mathrm{B}})& -\cos ({\displaystyle \frac{\mathrm{\pi }}{3}}+{\mathrm{\alpha }}_{\mathrm{B}})\\ -\sin ({\mathrm{\alpha }}_{\mathrm{B}})& \sin ({\mathrm{\alpha }}_{\mathrm{B}})& \sin ({\displaystyle \frac{\mathrm{\pi }}{3}}+{\mathrm{\alpha }}_{\mathrm{B}})& \sin ({\displaystyle \frac{\mathrm{\pi }}{3}}-{\mathrm{\alpha }}_{\mathrm{B}})& -\sin ({\displaystyle \frac{\mathrm{\pi }}{3}}-{\mathrm{\alpha }}_{\mathrm{B}})& -\sin ({\displaystyle \frac{\mathrm{\pi }}{3}}+{\mathrm{\alpha }}_{\mathrm{B}})\\ 0& 0& 0& 0& 0& 0\end{array}\right]$$

(2)

R_P represents the 3D coordinates of the connection points between the six servos and linkages of the platform, and R_B represents the 3D coordinates of the connection points between the base and the linkages. R_P and R_B are the radii of the platform and the base, respectively, while α_P and α_B are the angles of the platform and base planes.

By calculating the vertical distance between the platform linkage connection points and the servo connection points, the height h of each linkage is determined. For each pair of connection points, the horizontal distance difference between the platform connection point and the base connection point, along with the heights h_i of all six linkages, can be expressed as follows:

$$\Delta x_i=x_{pi}-x_{bi}, \Delta y_i=y_{pi}-y_{bi}$$

(3)

$$h_i=\sqrt{rod\text{\_}lengt{h}^2+servo\text{\_}arm\text{\_}lengt{h}^2-\Delta {x_i}^2-\Delta {y_i}^2}-z_{pi}$$

(4)

rod_length represents the length of the linkage, and servo_arm_length represents the length of the servo arm. Let the coordinates of the platform and base vertices be denoted as P_i(x_pi, y_pi, z_pi) and B_i(x_pi, y_pi, z_pi), respectively.

The heights h of all six linkages are stored in a vector. Let the initial position of the platform be denoted as h_pos. When the platform undergoes rotation, the corresponding rotation matrix is T_BP.

Each leg vector is as follows:

$$\left[\mathrm{L}\mathrm{e}{\mathrm{g}}_{\mathrm{i}}\right]=\mathrm{t}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{s}+\mathrm{h}\text{\_}\mathrm{p}\mathrm{o}\mathrm{s}+{\mathrm{T}}_{\mathrm{B}\mathrm{P}}\ast {\mathrm{R}}_{\mathrm{P}}-{\mathrm{R}}_{\mathrm{B}}$$

(5)

Trans represents the translation matrix of the platform.

The length of each leg, leg_i_length, is calculated from the leg vector leg_i, and the coordinates P_i of each point on the platform after movement are as follows:

$${\mathrm{P}}_{\mathrm{i}}={\mathrm{l}\mathrm{e}\mathrm{g}}_{\mathrm{i}}+{\mathrm{B}}_{\mathrm{i}}$$

(6)

The actual rotation angle angle_i that the servo should rotate is derived from the geometric relationship between servo_arm_length, rod_length, and Leg_length as follows:

$${\mathrm{L}}_{\mathrm{i}}=\mathrm{L}\mathrm{e}\mathrm{g}\text{\_}\mathrm{l}\mathrm{e}\mathrm{n}\mathrm{g}\mathrm{t}{{\mathrm{h}}_{\mathrm{i}}}^2-\mathrm{r}\mathrm{o}\mathrm{d}\text{\_}\mathrm{l}\mathrm{e}\mathrm{n}\mathrm{g}\mathrm{t}{{\mathrm{h}}_{\mathrm{i}}}^2+\mathrm{s}\mathrm{e}\mathrm{r}\mathrm{v}\mathrm{o}\text{\_}\mathrm{a}\mathrm{r}\mathrm{m}\text{\_}\mathrm{l}\mathrm{e}\mathrm{n}\mathrm{g}\mathrm{t}{{\mathrm{h}}_{\mathrm{i}}}^2$$

(7)

$${\mathrm{M}}_{\mathrm{i}}=2\times \mathrm{s}\mathrm{e}\mathrm{r}\mathrm{v}\mathrm{o}\text{\_}\mathrm{a}\mathrm{r}\mathrm{m}\text{\_}\mathrm{l}\mathrm{e}\mathrm{n}\mathrm{g}\mathrm{t}{\mathrm{h}}_{\mathrm{i}}\times ({\mathrm{z}}_{\mathrm{p}\mathrm{i}}-{\mathrm{z}}_{\mathrm{b}\mathrm{i}})$$

(8)

$$N_i=2\times servo\text{\_}arm\text{\_}length\times (\mathit{cos}({\beta }_i)\times (x_{pi}-x_{bi})+\mathit{sin}({\beta }_i)\times (y_{pi}-y_{bi}))$$

(9)

$$\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{l}{\mathrm{e}}_{\mathrm{i}}={\sin }^{-1}({\displaystyle \frac{{\mathrm{L}}_{\mathrm{i}}}{\sqrt{{{\mathrm{M}}_{\mathrm{i}}}^2+{{\mathrm{N}}_{\mathrm{i}}}^2}}})-{\tan }^{-1}({\displaystyle \frac{{\mathrm{N}}_{\mathrm{i}}}{{\mathrm{M}}_{\mathrm{i}}}})$$

(10)

β_i represents the polar coordinate angle value of each point on the base.

To meet performance requirements, the main control chip selected is the STM32F407VET6, which is capable of efficiently handling the matrix computations and signal processing tasks required by the platform. The servo used is the Hiwonder HTD-45H serial bus servo, with a maximum torque of 45 kg∙cm, and features position, temperature, and voltage feedback, enabling precise closed-loop control. The power supply uses a DC Bus voltage step-down system, and the serial power supply method helps prevent excessive currents during motor stalling, with additional software-based current protection to enhance the platform’s durability and safety. The platform receives data from the HWT606 gyroscope, performs matrix calculations using a DSP library, and computes the platform’s attitude in real time, as well as the required servo rotation angles. The servos then receive the angle information and adjust their positions to achieve real-time dynamic adjustments of the platform.

To ensure the long-term stable operation of the platform, a hierarchical calibration and multiple error suppression system has been established. During the platform initialization phase, the initial pose is precisely located through mechanical limit contacts, completing the automatic alignment of the servo electrical zero position and the mechanical reference position, providing a high-precision reference foundation for subsequent control. To address potential issues such as mechanical wear and sensor drift during operation, the platform introduces a real-time fusion mechanism based on gyroscopic attitude data and servo angle feedback. Combined with a PID closed-loop control algorithm, this mechanism dynamically corrects deviations in the calculation of the effective linkage length. Additionally, the platform utilizes the multi-directional buffering characteristics of universal joints to achieve stress distribution, further reducing the impact of localized wear on overall accuracy. To address the attitude accumulation errors that may arise from prolonged operation, an attitude error integration compensator is embedded in the control algorithm, employing a progressive fine-tuning strategy to achieve adaptive error correction. All key calibration parameters are securely stored in the Flash memory of the main control chip, ensuring reliable data retention after power loss. During daily use, only one manual calibration is required before the first startup, ensuring the platform’s high-precision and stable operation.

2.1.2. Design of the End Effector

The traditional chili pepper harvesting method is manual picking, which inevitably causes some damage to the chili fruits. To address this issue, this paper designs a flexible harvesting end effector with an integrated cutting–gripping structure, which uses a cutting and gripping method to detach the chili pepper from the stem during harvesting. Compared to the pulling separation method used in manual harvesting, the cutting–gripping separation method employed by this end effector can effectively reduce skin damage to the chili fruits during the collection process, ensuring the integrity of the chili peppers.

The end effector in this paper consists of three parts: the cutting module, the gripping module, and the power output module. The power output module controls the opening and closing of the gripping and cutting modules through a lead screw mechanism. The overall structure of the mechanism is shown in Figure 4.

In its natural state, the blades are in an open position. During the harvesting operation, the power output module generates torque to drive the motor in the forward direction, which rotates the lead screw and closes the flexible gripper to grasp the chili pepper. The cutting module then closes to clamp and cut the chili pepper’s stem.

The function of the gripping module is to pre-compress and position the chili pepper fruit, facilitating the cutting by the blade. After the chili fruit stem is cut, the gripper provides sufficient clamping force to hold the chili pepper, preventing it from falling and ensuring stable gripping. Both left and right grippers are made of TPU 95A material, which is flexible, and are fixed below the cutting blades.

The end effector employs FOC (Field-Oriented Control) to achieve high-precision force control and cutting control. The FOC control strategy decouples the three-phase current of the motor into two independent components, enabling precise control and improving harvesting efficiency and stability. The FOC control process is shown in Figure 5. The FOC strategy in this paper decomposes the three-phase alternating current into a DC component to simplify the control.

The specific process is as follows:

Initially, the Clarke transformation is applied to convert the three-phase currents (i_a, i_b, i_c) into a two-phase stationary coordinate system (i_α, i_β), reducing the complexity of the control system. The formula is as follows:

$${\mathrm{i}}_{\mathrm{\alpha }}={\displaystyle \frac{2}{3}}{\mathrm{i}}_{\mathrm{a}}-{\displaystyle \frac{1}{3}}{\mathrm{i}}_{\mathrm{b}}-{\displaystyle \frac{1}{3}}{\mathrm{i}}_{\mathrm{c}}$$

(11)

$${\mathrm{i}}_{\mathrm{\beta }}={\displaystyle \frac{\sqrt{3}}{3}}{\mathrm{i}}_{\mathrm{b}}-{\displaystyle \frac{\sqrt{3}}{3}}{\mathrm{i}}_{\mathrm{c}}$$

(12)

Next, the stationary coordinate system (i_α, i_β) is further transformed into the synchronous rotating coordinate system (i_d, i_q) through the Park transformation. The calculation formula is as follows:

$${\mathrm{i}}_{\mathrm{d}}={\mathrm{i}}_{\mathrm{\alpha }}\cos \mathrm{\theta }+{\mathrm{i}}_{\mathrm{\beta }}\sin \mathrm{\theta }$$

(13)

$${\mathrm{i}}_{\mathrm{q}}=-{\mathrm{i}}_{\mathrm{\alpha }}\sin \mathrm{\theta }+{\mathrm{i}}_{\mathrm{\beta }}\cos \mathrm{\theta }$$

(14)

Here, i_d is responsible for controlling the motor’s magnetic field, while i_q is responsible for controlling the motor’s torque.

In terms of hardware, the power supply section, after filtering through a capacitor, connects to the DRV8303 (DigiKey, Dallas, TX, USA). The internal buck circuit of the DRV8303 steps down the voltage to 12 V for use by other parts of the power system. Additionally, another 5 V power supply is stepped down to 3.3 V, which is used separately for the analog power AVCC and the digital power VCC to ensure the proper operation of the STM32 microcontroller (Fanke Technology, Nanning, China) and the AS5047P encoder (ams OSRAM, Premstaetten/Graz, Austria). The STM32 controls the DRV8303, which in turn controls the switching of the full-bridge NMOS (N—Channel Metal-Oxide-Semiconductor), ultimately achieving motor control.

The DRV8303 features two current-sensing pins. After connecting sampling resistors of 22 Ω in series, the three-phase motor currents can be derived using Kirchhoff’s current law. These currents are then used to calculate and control i_q through the Clarke and Park transformations.

$$\sum _{\mathrm{k}=1}^{\mathrm{n}}{\mathrm{i}}_{\mathrm{k}}=0$$

(15)

By appropriately controlling the PWM duty cycle, the target vector voltage is made to rotate along a near-circular trajectory in a spatial position. For the k-th sector, the desired synthesized voltage space vector U_ref, which is composed of the U_α and U_β components, has its application time calculated using the following formulas:

$${\mathrm{T}}_{\mathrm{x}}={\displaystyle \frac{\sqrt{3}{\mathrm{U}}_{\mathrm{\beta }}}{{\mathrm{U}}_{\mathrm{d}\mathrm{c}}}}{\mathrm{T}}_{\mathrm{S}}$$

(16)

$${\mathrm{T}}_{\mathrm{y}}={\displaystyle \frac{\sqrt{3}{\mathrm{T}}_{\mathrm{S}}}{{\mathrm{U}}_{\mathrm{d}\mathrm{c}}}}({\displaystyle \frac{\sqrt{3}}{2}}{\mathrm{U}}_{\mathrm{\alpha }}-{\displaystyle \frac{1}{2}}{\mathrm{U}}_{\mathrm{\beta }})$$

(17)

$${\mathrm{T}}_0={\mathrm{T}}_{\mathrm{s}}-({\mathrm{T}}_{\mathrm{x}}+{\mathrm{T}}_{\mathrm{y}})$$

(18)

T_x and T_y represent the duration times of the basic vectors, T₀ is the duration time of the zero vector, T_s is the PWM carrier period, and U_dc is the DC-link voltage.

SVPWM control not only improves the utilization of the bus voltage but also further reduces the motor’s torque ripple, enhancing operational smoothness. Additionally, it reduces switching losses, thereby improving system efficiency.

Based on the force data from multiple cutting–gripping experiments, this paper analyzes the variation in gripping and cutting forces during the harvesting process and performs curve fitting. The fitted force variation curve for the entire chili pepper harvesting process is shown in Figure 6. When the gripper receives the grasping command, the drive motor gradually applies gripping force to ensure the chili pepper is securely held. The data shows that the gripping force gradually increases between 2.2 s and 2.6 s, ultimately reaching a peak pressure of 6.2 N, at which point the chili pepper remains stable without slipping or damage. In the stable gripping state, the cutting mechanism is activated at 2.8 s to complete the separation of the chili pepper from the plant. After the cutting process, the gripping force is reduced to 5.3 N, minimizing mechanical damage to the chili pepper. Subsequently, the robotic arm moves the gripped chili pepper to the collection area. Finally, when the target position is reached, the gripper receives a release command and slowly releases the chili pepper until it is fully detached.

2.2. Chili Pepper Harvesting Point Localization

2.2.1. Point Cloud Preprocessing

The use of 3D LiDAR point cloud data with significant noise can directly lead to inaccurate chili pepper point cloud construction, which is one of the key reasons for the difficulty in realizing intelligent chili pepper harvesting. This paper employs an optimized DBSCAN algorithm to process the 3D LiDAR-generated point cloud data in real time, filtering out the majority of the noise while accurately clustering the positions of the chili peppers.

DBSCAN is a density-based spatial clustering algorithm. The algorithm divides areas with sufficient density into clusters and discovers arbitrarily shaped clusters in spatial databases with noise, where a cluster is defined as the largest set of density-connected points. DBSCAN does not require the number of clusters to be specified in advance. It identifies clusters in the data based on the neighborhood radius ϵ and the minimum number of data points, MinPts, and is suitable for handling non-spherical datasets. ϵ(epsilon) is the neighborhood radius of a data point, defining the area around the point. MinPts is the minimum number of data points required in each cluster. If a data point’s neighborhood (within the specified radius ϵ) contains at least MinPts points, the point is considered a core point. If a point’s neighborhood contains fewer than MinPts points but is within the neighborhood of a core point, it is considered a border point. If a point is neither a core point nor within the neighborhood of any core point, it is considered a noise point.

When clustering in the chili pepper field, the process starts from any unvisited point. If the current point is a core point, it is used to expand the cluster by finding all points within its neighborhood, and these points are recursively processed in the same way. If the current point is a border point, it is added to an existing cluster. If the current point is a noise point, it is marked as noise, and no further operations are performed. Finally, the clustering is expanded, and noise points are handled. For each core point, all points within its neighborhood are checked. If any point within the neighborhood is also a core point, the expansion continues until no more points can be added to the current cluster.

Neighborhood definition:

$${\mathrm{N}}_{\mathsf{ϵ}}\left(\mathrm{p}\right)=\left\{\mathrm{q}\in \mathrm{D}\left|\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{t}\left(\mathrm{p},\mathrm{q}\right)\le \mathsf{ϵ}\right.\right\}$$

(19)

dist(p, q) represents the distance between point p and point q.

$$\mathrm{M}\mathrm{i}\mathrm{n}\mathrm{P}\mathrm{t}\mathrm{s}\ge \mathrm{d}+1$$

(20)

d is the dimension of the data. Point p is a core point if the following formula is satisfied:

$$\left|{\mathrm{N}}_{\mathsf{ϵ}}\left(\mathrm{p}\right)\left|\ge \right.\mathrm{M}\mathrm{i}\mathrm{n}\mathrm{P}\mathrm{t}\mathrm{s}\right.$$

(21)

N_ϵ(P) represents the number of points within the neighborhood of point p.

In this paper, the improved DBSCAN algorithm is first used for filtering and smoothing, while also separating the ground point cloud to better analyze the chili pepper field environment. During DBSCAN clustering, a k-nearest neighbor search is employed to find the k nearest neighbors of each point, and the distance to the k-th nearest neighbor is selected as ϵ. The initial value for MinPts is set to 5. These parameter settings allow the DBSCAN clustering to better adapt to the complex environment of the chili pepper field, especially when processing point clouds like chili peppers, which have different spatial distribution characteristics.

To further optimize the clustering process, this paper introduces spatial constraints when calculating the Euclidean distance between point clouds. By considering the relative positions and distribution densities of the points in the point cloud, the distance metric between points is adjusted, enhancing the adaptability to different regions. Additionally, normal direction and curvature information are used as geometric constraints to further improve clustering accuracy. In calculating the similarity between point clouds, normal direction and surface normal are incorporated into the distance calculation to constrain the local surface morphology and variations of the point cloud. The normal direction in the point cloud is determined by calculating the normal vector of each point, which is used to further assess the geometric consistency of the point cloud in its neighborhood. Meanwhile, curvature information is calculated using the local fitting method PCA (Principal Component Analysis), which reflects the curvature of the point cloud surface and provides key geometric features for distinguishing surface regions with different shapes. By incorporating geometric information into the distance metric, the clustering process can better identify and separate different point cloud clusters, thereby improving the accuracy and robustness of the chili pepper field environment analysis.

According to field survey results, some chili peppers in the experimental field are obscured by leaves, as shown in Figure 7.

The overall curvature of the chili pepper is low, and its surface normals change gradually between adjacent points. When partial occlusion of the chili pepper is detected, the point cloud exhibits inconsistencies in the normal direction or abrupt changes in curvature, which are then marked as occluded point clouds. By incorporating these geometric features into the clustering analysis, it becomes possible to more accurately identify the chili peppers and distinguish between their occluded and non-occluded parts.

2.2.2. Point Cloud Reconstruction and Localization

In an occluded state, relying solely on the partially missing point cloud data cannot accurately localize the chili peppers. To address this issue, many scholars have proposed various methods. Among them, 3D reconstruction methods have certain advantages when dealing with large areas of occlusion and large amounts of data, making them suitable for the chaotic point clouds of chili pepper fields. Traditional 3D point cloud reconstruction relies on the synchronization of camera and 3D LiDAR data, but when the lighting dims, this method amplifies the reconstruction errors of the chili pepper point cloud, affecting the accuracy of point cloud reconstruction. This paper uses EKF for data fusion between 3D LiDAR and IMU inertial navigation systems to obtain more accurate pose information. By combining point cloud data from various positions, the ICP (Iterative Closest Point) algorithm is used for chili pepper point cloud reconstruction, and the DBSCAN algorithm is further improved for precise localization of the chili peppers without the need to consider lighting conditions. The specific process is shown in Figure 8.

This paper utilizes data obtained from various sensors, applying a state prediction model and a point cloud observation model for forecasting. IMU data is used to update the system’s state, and the observation information provided by the point cloud data is used to correct the state estimation. Then, Kalman gain is introduced for data updating, ultimately resulting in accurate pose data.

First, for the occluded point clouds, multiple frames are sampled over time, and the sampled point clouds are matched with the nearest points based on Euclidean distance in real time. Then, by minimizing the squared distance between matching points from the source point cloud and the target point cloud, the rigid transformation of the source point cloud is calculated, enabling the registration between point clouds from different frames. The rigid transformation model gives the following:

$${\mathrm{P}}_{\mathrm{i}}=\mathrm{R}\left(\mathrm{t}\right)\cdot {\mathrm{P}}_{\mathrm{j}}+\mathrm{t}\left(\mathrm{t}\right)$$

(22)

P_i is the point cloud at time t_i; P_j is the point cloud at time t_j; R(t) is the rotation matrix from t_j to t_i; t(t) is the translation vector, representing the displacement from t_j to t_i.

Subsequently, multiple iterations are performed to update and gradually optimize the source point cloud until the result converges, yielding the repaired point cloud. The repair effect is shown in Figure 9, where the red section represents the fully recognized or 3D point cloud reconstructed chili pepper point cloud, and the blue section represents the detected occluded chili pepper point cloud.

2.3. Experimental Design

In this section, we calibrate and validate the designed intelligent harvesting system. Through experimental testing, the overall recognition accuracy of the intelligent harvesting system for chili peppers is evaluated, along with the precision of chili pepper harvesting.

The main test platform in this study is shown in Figure 10. The control system of the intelligent harvesting platform is deployed on a Republic of Gamers laptop (G634, ASUS, Maricopa County, AZ, USA), equipped with an RTX 4080 dedicated graphics card. The system is equipped with a MID-360 hybrid solid-state 3D LiDAR (Livox, Shenzhen, China), which has a vertical field of view ranging from −7° to 52° and a horizontal field of view of 360°, with an angular random error of less than 0.15°. Additionally, the robot is equipped with a WHEELTEC N200 nine-axis IMU inertial navigation module (WHEELTEC, Dongguan, China) to provide precise IMU data. The platform also features an adaptive leveling platform constructed from fiberglass materials, a mechanical arm 3D printed with PLA materials, and flexible grippers printed with TPU flexible material. The chassis of the intelligent harvesting platform is powered by a 48 V power supply, capable of supporting operation for over 10 h, with an operational range of approximately 20 km. The control program of the intelligent harvesting system is implemented based on the ROS (Robot Operating System) framework running on the Linux Ubuntu 20.04 operating system.

This study set up multiple experimental indicators. The harvesting success rate is defined as the ratio of the number of successfully harvested chili peppers to the total number of harvest attempts. Harvesting time consumption is defined as the total time taken from the beginning of chili pepper recognition to the completion of harvesting that chili pepper. The recognition success rate is defined as the ratio of the number of successfully recognized chili peppers to the total number of chili peppers in each ridge. Harvesting damage rate is defined as the ratio of the number of chili peppers damaged during harvesting to the total number of harvested chili peppers.

2.3.1. Comparative Experiment

The experimental area is set in a typical chili pepper field, with a distance of 60 cm between two ridges, a ridge height of 30 cm, and a ridge width of 50 cm, as shown in Figure 11.

To better highlight the practical efficiency and necessity of the intelligent harvesting platform, four sets of schemes are compared in this paper, as shown in

Table 1. The configuration of the four experimental schemes.

	Adaptive Leveling Platform	MID-360	IMU Inertial Navigation System	Visual System
Scheme 1		√	√
Scheme 2		√		√
Scheme 3	√	√	√
Scheme 4	√	√		√

The symbol ‘√’ in the table indicates the method used in the corresponding scheme.

.

During the experiment, each set of schemes harvests chili peppers located on the left and right ridges of the harvesting system platform. At the start of the experiment, the harvesting system platform follows the path between the ridges to perform the harvesting. The following metrics are recorded and calculated: harvesting time consumption, harvesting success rate, and the recognition success rates for both occluded and non-occluded chili peppers.

2.3.2. Continuous Experiment

To investigate whether there is any systematic accumulation error in the chili pepper harvesting system developed in this study, a continuous experiment was designed to verify the stability of the chili pepper harvesting system. The experimental scenario was set in a chili pepper field with a total length of 56 m and a total width of 60 m, with a slope not exceeding 15°, containing 27 complete ridges. The harvesting system carrier moved along the ground between the ridges, following the planned route for harvesting. The experimental harvesting area covered all the ridges within the experimental chili pepper field. During the experiment, the harvesting success rate was recorded at every moment.

3. Results

The experiment was conducted in the test chili field under sunny conditions. The overall experimental process is shown in Figure 12.

3.1. Comparative Experimental Results

The data from the four experimental schemes are collected, organized, and analyzed, with the results shown in

Table 2. The experimental data of the four experimental schemes.

Scheme	Harvesting Success Rate	Harvesting Time Consumption/s	Occluded Pepper Recognition Success Rate	Unoccluded Pepper Recognition Success Rate	Pepper Damage Rate
Scheme 1	63.67%	15.72	72.27%	78.94%	1.84%
Scheme 2	64.33%	15.37	73.09%	78.63%	1.76%
Scheme 3	86.33%	13.17	81.95%	89.04%	0.80%
Scheme 4	85.96%	12.93	82.16%	88.60%	0.82%

.

As shown in the data from Table 2, without the adaptive leveling platform, the robotic arm’s workspace cannot fully cover the chili peppers on the ridges, resulting in significant missed harvesting, with harvesting success rates of only 63.67% and 64.33% for Scheme 1 and Scheme 2, respectively. Additionally, the robotic arm’s pose cannot be adjusted in a timely manner, and more collision factors need to be considered during inverse kinematics, leading to increased harvesting time consumption. The harvesting time for Scheme 1 and Scheme 2 is 15.72 s and 15.37 s, respectively. In contrast, with the adaptive platform, the harvesting success rates for Scheme 3 and Scheme 4 increased by 22.66% and 21.63%, respectively, and the harvesting time was also reduced, making the system more efficient and accurate for automated chili pepper harvesting in hilly chili fields.

In terms of chili pepper recognition, without the adaptive leveling platform, the point cloud origin deviates from the horizontal plane, resulting in distorted point clouds. The data shows that the recognition system’s accuracy in this case is significantly lower than when the adaptive leveling platform is installed. Additionally, a comparison is made with traditional visual point cloud reconstruction, where the recognition success rate for non-occluded chili peppers is slightly higher in the proposed method than in visual recognition. However, for occluded chili peppers, visual recognition generally performs slightly better than the proposed method. Nonetheless, the recognition accuracy of both strategies is very close, with the difference being less than 1%.

In terms of harvesting damage rate, the lack of an adaptive leveling platform leads to inaccurate robotic arm positioning, causing the end effector to fail to align with the chili pepper plants. This misalignment increases the likelihood of damage during the cutting process.

3.2. Continuous Experimental Results

Based on the data obtained from the experiment, a real-time curve was plotted with the harvesting success rate as the vertical axis and time as the horizontal axis, as shown in Figure 13.

As shown in Figure 12, it can be observed that the harvesting success rate of the chili pepper intelligent harvesting device exhibits a slow decline as time increases. After 20 min, the harvesting success rate stabilizes around 85.1%. In the first 10 minutes of the harvesting operation, the harvesting success rate remains above 92.5%, maintaining a high level with very few cases of missed or incorrect harvesting. Subsequently, due to the accumulation of system errors, the harvesting success rate begins to decrease, eventually stabilizing around 85.1%. At this point, a small number of missed harvests may occur, but the overall harvesting success rate remains at a relatively high level.

4. Discussion

This paper proposes an intelligent harvesting system based on 3D point cloud reconstruction and multi-mechanism collaborative leveling. The system integrates data from LiDAR and IMU for 3D point cloud reconstruction and incorporates an adaptive leveling platform, an H-shaped planar slide, and an integrated cutting–gripping flexible end effector to achieve automated chili pepper harvesting.

The citrus harvesting system proposed by Wang et al. [28] achieved a harvesting success rate of 80.0% and an average harvesting time of 15.2 s in laboratory conditions. However, the absence of an integrated leveling platform means that its harvesting success rate will likely decrease, and harvesting time will increase in real-world environments. In contrast, the intelligent harvesting system used in this study achieves a harvesting success rate of up to 86.33% in the more complex chili pepper field environment, maintaining a harvesting success rate of around 85.1% even with the accumulation of system errors. Chen et al. and Bao et al. both employed visual methods for crop recognition [26,35], achieving high recognition success rates. However, the physical characteristics and occlusion of chili peppers make these methods less effective in chili pepper harvesting. Additionally, both Chen et al. and Bao et al.’s approaches are highly sensitive to lighting conditions [26,35,36], which is disadvantageous for all-weather operation. The pepper recognition and localization method proposed by Liu et al. achieved an overall recognition accuracy of 86% [37], which is slightly higher than the 81.95% attained by the current system. However, its performance under backlit conditions declined sharply by 14 percentage points, dropping well below 81.05%. This indicates limited robustness to illumination variations, which compromises its suitability for all-weather operation and results in considerable limitations in real-world production scenarios. In contrast, the chili pepper harvesting system proposed in this study not only achieves high recognition and harvesting success rates, but is also unaffected by lighting intensity, allowing for all-weather operation, thereby improving efficiency in real-world agricultural production and meeting the needs for automated chili pepper harvesting.

However, this study does have certain limitations. The intelligent harvesting system based on 3D point cloud reconstruction and multi-mechanism collaborative leveling is more suitable for autonomous harvesting in the hilly chili pepper fields of southern China. In other harvesting scenarios with distinct target features, the proposed recognition method may significantly lag behind other methods in terms of efficiency. The system integrates an adaptive leveling platform and an H-shaped planar slide, with a workspace focused on crop harvesting on both sides of the robotic arm and limited height, which presents certain limitations in the context of agricultural harvesting. Due to the complexity of the hilly chili pepper field environment, point clouds of chili peppers are generally sparse, making recognition and harvesting failures unavoidable in actual production. Additionally, system error accumulation is also difficult to avoid. The proposed system needs further improvements. Improving the adaptability and operational efficiency of intelligent harvesting systems constitutes a key challenge to be addressed in future studies. As different fruits and vegetables may require different forces and positional requirements, adaptation can be achieved by re-evaluating the required cutting and gripping forces and replacing the shear interface with a suitably positioned mechanism. To increase the system’s universality, future work should focus on the modular redesign of hardware components, such as interchangeable grippers and reconfigurable motion platforms, and on enhancing the software framework through adaptive recognition algorithms and robust sensor fusion strategies. In addition, considering future application scenarios, the shape of the flexible grippers could be adjusted for different fruit and vegetable sizes and shapes. At the same time, multiple gripping force experiments should be conducted to ensure the minimization of damage to various fruits and vegetables. Looking ahead, the applicability and harvesting efficiency of intelligent picking systems based on 3D point cloud reconstruction and multi-mechanism collaborative leveling could also be refined to enhance their practical value and economic effectiveness.

5. Conclusions

This study designs an intelligent chili pepper harvesting system based on 3D point cloud reconstruction and multi-mechanism collaborative leveling. The system integrates an adaptive leveling platform, an H-shaped plane slide table, and a flexible end effector. It also utilizes Extended Kalman Filter (EKF) fusion to combine real-time data from a 3D LiDAR and IMU inertial navigation system, enabling the restoration of chili pepper point clouds occluded by leaves from multiple perspectives. This approach enhances both the precision and integrity of the harvesting process. Experimental data collected in real chili pepper field environments demonstrate the effectiveness of the system. The results show that the system achieves a harvesting success rate of 86.33%, with a harvesting time of 13.17 s. The success rates for identifying occluded and non-occluded chili peppers are 81.95% and 89.04%, respectively. Notably, the system significantly reduces the chili pepper damage rate, which is only 0.80%. The system also demonstrates good stability when facing the accumulation of system errors, maintaining an overall harvesting success rate of approximately 85.1%. The chili pepper intelligent harvesting system, based on 3D point cloud reconstruction and multi-mechanism collaborative leveling, provides a foundation for the subsequent development of unmanned chili pepper cultivation in hilly areas. In future studies, the system demonstrates promising potential for broader crop adaptability through the re-evaluation of cutting and gripping forces, as well as the replacement or repositioning of the shear mechanism to accommodate different agricultural products.