Research on Continuous Obstacle Avoidance Picking Planning Based on Multi-Objective Clustered Crabapples

Abstract

In view of the low efficiency and slow development of fruit and vegetable picking in China, the picking sequence and obstacle avoidance of clustered crabapples were studied with them as the picking target. The multi-objective picking sequence of crabapples was planned, and the adaptive pheromone factor, heuristic function, and volatile factor were used to improve the ant colony (ACO) algorithm, so as to improve the convergence speed, adaptability, and global search ability of the algorithm. In order to avoid the collision between the robotic arm and the branches of the fruit tree, the three-dimensional reconstruction of the fruit tree was carried out, the shape and position information of the obstacle branch was determined, the artificial potential field was fused with the RRT, the search orientation of the RRT algorithm was enhanced, the inflection point was reduced, and the convergence speed was improved. The results showed that the average success rate of picking was 89.58%, and the robotic arm did not collide with the branches according to the planned picking sequence during the picking process, so as to achieve the picking purpose and picking effect.

1. Introduction

China is a large agricultural country, especially in recent years, and with the increase in market demand for fruits, the planting area of fruits has increased annually [1]. In particular, crabapples are widely planted in Northeast China because of their rich vitamins and unique taste [2,3]. Currently, there are few studies on the automatic picking of crabapples in China, and manual picking remains the primary method. To achieve automatic picking, research is conducted on crabapple-picking robots. However, the fruit growth distribution on the tree is relatively random and is blocked by obstacles such as fruit tree branches, leading to issues such as low picking efficiency and difficulty in picking [4]. Therefore, in order to improve the picking efficiency of the picking robot on the tree and avoid the collision between the robot and the fruit tree branches during the picking process, the study of the multi-objective crabapple picking sequence planning and the obstacle avoidance problem of the picking robot has been conducted.

In recent years, scholars both domestically and internationally have conducted extensive research and experiments on the planning of fruit-picking sequences and obstacle avoidance for picking machines. These studies have yielded promising results that have been effectively applied in practical production settings [5,6]. Kurtser et al. [7] used a target sequencing method to plan the picking order of densely planted bell peppers. Experimental verification under greenhouse conditions showed that the harvest cycle time is shortened by 12% compared with disordered random picking. Li et al. [8] used a genetic algorithm to plan the order of multi-target fruit-picking tasks on trees, and compared with the disordered random method, the picking time was significantly shortened and the picking efficiency was improved. Ye et al. [9] used the target gravity and adaptive coefficient adjustment method to improve the Bi-RRT algorithm to achieve collision-free picking of lychees on the tree, and the planning speed of the improved Bi-RRT algorithm was faster than that before the improvement. Cao et al. [10] introduced target gravity into the RRT algorithm and used GA to optimize the path generated by the RRT algorithm, so as to plan the obstacle avoidance path of the lychee picking manipulator and improve the path quality and planning efficiency.

In order to solve the problem of continuous picking of multi-objective clustered crabapples on crabapple trees and avoid a collision between the robotic arm and the tree branches during the picking process, this paper proposes a method for planning the picking sequence and the obstacle avoidance path for picking, implemented through improved ACO and improved RRT algorithms. The adaptive pheromone factor and heuristic function factor are used to improve transfer probability, and the adaptive volatility factor is introduced to optimize the pheromone update to improve the ACO algorithm, thereby increasing the convergence speed of the algorithm in order to achieve the planning of the picking order of multi-objective crabapples on the tree. By combining the APF algorithm with the RRT algorithm to introduce the gravitational field, the guidance of the RRT algorithm path search is enhanced, the unnecessary inflection point is reduced, the convergence speed is improved, and the superiority of the improved algorithm is verified by simulation analysis compared with the previous RRT algorithm. Finally, a test platform was constructed to conduct a picking test analysis to verify the effectiveness and feasibility of the improved algorithm proposed in this paper.

The algorithm proposed in this paper can be applied to the obstacle avoidance of sand fruit picking. It can also be applied to other scenarios. For example, Sun Siya et al. [11] proposed an improved BI-RRT trajectory planning algorithm containing an artificial potential field to solve the problems of low support efficiency and low labor intensity of mechanized drilling and anchoring equipment in coal mines. Compared with the traditional RRT algorithm, the algorithm increased the optimization speed by 69.8% and shortened the trajectory length by 46.6%. Aiming at the problems of autonomy, safety, and efficiency of tea picking robots in tea gardens, Li Xin et al. [12] proposed an adaptive stepping RRT* (as-RRT *) path planning algorithm for tea picking robot arms. The experimental results demonstrate that the AS-RRT* algorithm reduces the path length by 14.18%, and the path planning time is less than 1 s. The proposed algorithm and the two improved algorithms are all improved versions of the RRT algorithm. The proposed algorithm can be applied to many scenarios if modified according to the actual operation requirements of the robot arm and has strong applicability.

The obstacle avoidance planning algorithm proposed in this paper is suitable for multi-objective picking of clustered sand fruit. Compared with traditional algorithms, it shows that the algorithm can optimize both picking efficiency and obstacle avoidance path. While improving the shortcomings of the ACO algorithm, such as long operation time, slow convergence, and a tendency to fall into local optimal solutions, the strong robustness and global searching ability of the ACO algorithm are retained. The algorithm provides a new method for fruit tree picking automation equipment, which helps to promote the intelligence and efficiency of modern agriculture.

In the second chapter, the Realsense D415 depth camera was used to obtain the cloud image of fruit tree points, and the cylindrical box method was used to envelop the robot arm and fruit tree branches to construct the environment. In Section 3, the ACO algorithm is improved by improving the transition probability and optimizing the updating of pheromone concentration, and the simulation experiment is compared with the traditional ACO. In Section 4, the artificial potential field method is combined with the RRT algorithm, and the simulation experiment is compared with the traditional RRT algorithm. In Section 5, the improved YOLOv5 algorithm and depth camera were used to obtain the three-dimensional spatial coordinates of the picking points, and the self-made pneumatic coupling manipulator and AUBO-E robot arm were assembled to build an experimental platform. Combined with the obstacle avoidance algorithm proposed in this paper, the multi-cluster sand fruit picking experiment was carried out, and the experimental results met the actual picking requirements. Finally, in Section 6 and Section 7, the experimental results are discussed, and the full text is summarized.

2. Related Work

In the process of modern agricultural automation, the fruit and vegetable picking robotic arm has become an important field of intelligent agriculture development. Many studies, based on their predecessors and aiming to improve equipment or algorithms, have achieved very good results. However, in many orchards, the fruit trees grow in a complex environment, and the fruits mostly grow in clusters, and obstacles such as leaves and dry leaves will be encountered during the picking process. Therefore, how to achieve efficient multi-objective, continuous obstacle avoidance and intelligent planning in the working process of the picking robot arm has become a new challenge. This chapter will introduce the relevant information about the robot arm and compare the key parameters of the existing research with those of this research.

2.1. Information About Robotic Arms

The robotic arm used in this paper is the AUBO-E5 6-DOF cooperative robot produced by El (Beijing, China) Intelligent Technology Co., LTD., as shown in Figure 1a. The robot has a wide range of applications and rich configurations, and it can achieve a ±360° rotation. Its working space is shown in Figure 1b. The D-H(Denavit-Hartenberg) modeling method is adopted to establish each joint, as shown in Figure 2. The D-H parameters of the AUBO-E5 robot are shown in

Table 1. D-H parameters of the AUBO-E5 robot.

Joint i	${\mathsf{\alpha }}_{\mathbf{i}-1}$/mm	${\mathsf{\alpha }}_{\mathbf{i}-1}$(rad)	${\mathbf{d}}_{\mathbf{i}}$/mm	${\mathsf{\theta }}_{\mathbf{i}}$(rad)	Joint Range (°)
1	0	0	121.5	${\mathsf{\theta }}_1$(0)	$\pm 175$
2	0	$-$pi/2	121.5	${\mathsf{\theta }}_2$(pi/2)	$\pm 175$
3	408	0	0	${\mathsf{\theta }}_3$(0)	$\pm 175$
4	376	0	0	${\mathsf{\theta }}_4$($-$pi/2)	$\pm 175$
5	0	$-$pi/2	102.5	${\mathsf{\theta }}_5$(0)	$\pm 175$
6	0	pi/2	94	${\mathsf{\theta }}_6$(0)	$\pm 175$

.

The working space range of AUBO-E5 can be solved according to the working range and forward kinematics of each joint given by the AUBO-E5 6-DOF robotic arm. However, it is also necessary to consider the growth posture of the sand fruit tree and the position height of the sand fruit. The working space range of AUBO-E5 can be solved according to these factors. The highest and lowest results were 2.91 m and 2.91 m from the ground, 0.42 m. The main trunk of the sand fruit tree was set 0.4 m away from the base coordinate of the mechanical arm, and the highest working point 1 position (0.4, 0.5, 2.91) and the lowest working point 2 position (0.4, 0.5, 0.42) were established. According to the D-H parameters and forward kinematics equations of the robot, the working space of the end of the robot arm is drawn using MATLAB, as shown in Figure 3, where the red dot represents the highest picking working point 1, the green dot represents the lowest picking working point 2, and the blue dot represents the working range of the end of the robot arm.

As can be seen from Figure 3, the working range of the end of the robotic arm includes the highest and lowest picking working points. Therefore, the AUBO-E5 six-degree-of-freedom robotic arm adopted in this paper can realize the picking of the whole sand fruit tree and meet the requirements of the picking work.

2.2. Research Comparison

In recent years, the rapid development of agricultural intelligence has promoted research on picking robotic arms, and many researchers have designed a lot of valuable equipment. This section will compare the differences between existing research and this study.

Wei Wang [13] proposed a method that combines the advantages of RRT and RRT-connect algorithms, introduces a greedy algorithm to enhance the efficiency of the algorithm, and integrates quintic B-spline interpolation to smooth and optimize the path, thereby reducing the time and length of the planned path.

To address the obstacle avoidance and path planning issues of robotic arms under complex working conditions, Jingkai Fu [14] proposed an improved RRT algorithm that presets piecewise interpolation through dimensionality reduction and pre-experimentation.

Ding Li [15] proposed an improved ${\mathrm{R}\mathrm{R}\mathrm{T}}^{*}$ algorithm based on a goal-oriented strategy and combined with bidirectional expansion.

As shown in Figure 4, first, the obstacles in each paper are modeled. Then, the algorithm program of this paper is added, and its path planning time is compared with other algorithms. To avoid the randomness of a single experiment, a program for averaging the results of multiple experiments is added to this algorithm. The specific time data of path planning are shown in

Table 2. Comparison of key parameters.

Reference	Path Planning Time of This Algorithm (ms)	Path Planning Time of the Reference Algorithm (ms)
[13]	735.4	387.8
[14]	16,366	1720.4
[15]	6160	288.4

.

According to the comparative data in Table 2, it can be seen that the algorithm proposed in this paper significantly shortens the search time compared to traditional methods and shows significant advantages in efficiency improvement. This result validates the practicality and innovation of the algorithm, indicating its significant research value and application potential in optimizing search performance.

2.3. Preliminaries

The following will introduce several software programs used in this article.

AUBO Studio (AUBO Robot Studio 5.1.0): The software that comes with the AUBO-E5 six-degree-of-freedom collaborative robot is AUBO Studio (or AUBO Robot Studio), which is an integrated control platform developed by AUBO Robotics specifically for its collaborative robots. This software provides drag-and-drop teaching, script programming (such as Python), motion trajectory planning, simulation, and other functions, supporting users in quickly deploying robot tasks, suitable for industrial automation, scientific research experiments, and other scenarios. Its intuitive graphical interface and rich API interfaces make it highly flexible in robot control and secondary development. This software is used in Section 2.1 of this article.

Intel RealSense Viewer v2.56.3: Intel RealSense Viewer is the official companion software for the RealSense D415 depth camera(Intel Corp., Santa Clara, CA, USA), mainly used for real-time viewing, adjustment, and testing of the camera’s depth, RGB, and infrared data streams. Users can adjust depth parameters (such as resolution and frame rate), calibrate sensors, record and replay data, and visually observe the depth point cloud effect through this software. This tool is commonly used for camera performance evaluation and initial configuration in applications such as robot vision, 3D scanning, and SLAM. This software is used in Section 3.1 and Section 6.1 of this article.

Depth Quality Tool for Intel RealSense v2.56.3: The Depth Quality Tool is another professional tool provided by Intel, specifically designed to evaluate and optimize the data quality of RealSense depth cameras. It can measure key indicators such as depth accuracy, noise level, and fill rate, and support the generation of depth error heat maps to help users optimize camera settings (such as laser power and exposure time) to adapt to different lighting environments. This tool is particularly important in applications that require high-precision depth data, such as robot navigation and 3D reconstruction, to ensure optimal camera performance. This software is used in Section 3.1 and Section 6.1 of this article.

3. Environment Set Up

3.1. Sugarcane Tree 3D Reconstruction

During the process of picking clusters of crabapples from the trees, there may be obstacles such as tree branches on the picking path, leading to collisions between the robotic arm or end effector and the tree branches. This can result in damage to the robotic arm, end effector, or the crabapple trees and fruits, preventing the continuation of the picking task. Therefore, it is essential to first identify the location of the obstacles and conduct a three-dimensional reconstruction of the crabapple trees within the robotic arm’s picking movement space. Given that the crabapple stalks, leaves, and fine branches (with diameters less than 3 mm) are soft and will not cause damage to the picking movement of the robotic arm and end effector, this paper considers crabapple tree branches (with diameters greater than 3 mm) obstacles.

The Realsense D415 depth camera was used to obtain the cloud point map of the crabapple tree. The cloud point map of the tree is filtered by straight pass filtering, statistical filtering, and color filtering to remove the interference of the environmental background, leaves, and thin branches (radius less than 3 mm) to obtain the point cloud map of the branches and trunks, as shown in Figure 5b. The filtered point cloud map of the branches of the crabapple tree is matched, so as to realize the three-dimensional point cloud reconstruction of the branches of the crabapple tree as shown in Figure 5c. The information on each branch is shown in

Table 3. Parameters of each branch of the crabapple tree.

Branch	Starting Point (m)	Endpoint (m)	Radius (mm)
1	(0.00, 0.540, 0.610)	(0.00, −0.210, 0.610)	29.5
2	(0.00, 0.180, 0.610)	(0.00, −0.167, 0.490)	12.0
3	(−0.026, 0.107, 0.661)	(−0.183, −0.101, 0.818)	13.5
4	(0.027, 0.159, 0.625)	(0.230, −0.037, 0.778)	15.0

.

According to the cloud map parameters of each branch point of the crabapple tree in Table 3, the complete space of the tree branches is constructed, which provides branch information for the collision detection between the next robotic arm and the tree. By constructing the crabapple tree branches and picking a robotic arm model, a more appropriate collision detection model is selected, and the branch information of the tree is provided, which provides obstacle position information for the obstacle avoidance path planning of the arm in the later stage, so as to facilitate the obstacle avoidance path planning of the robotic arm picking.

3.2. Robotic Arm Collision Detection Method

In order to avoid collision during the picking process, the collision detection among the arm, end effector, and branches of the crabapple tree was studied in this paper. According to its shape, a simple and reasonable cylindrical bounding box was selected to envelop it, which can simplify the collision detection process and improve the detection efficiency [16]. Because the leaves and twigs (radius less than 3 mm) of the tree will not affect the picking path and collision picking robotic arm, it can be ignored. The thicker branches of the tree were mainly enveloped by a cylindrical bounding box so that the collision detection problem of picking arm, end effector, and the tree branches could be simplified as the collision detection problem between cylinders, that is, it was converted into the shortest distance between cylinder and cylinder axis. By judging the relationship between the shortest distance between two axes and the sum of the radii of two cylinders, whether the picking robot collides with the branches can be judged. The positional relationship between the two enveloping cylinders is shown in Figure 6. Among them, $C_i$ means that the radius of the envelope cylinder between the joint $i$ of the robotic arm and the joint $i+1$ is $r_i$, $D_i$ means that the radius of the envelope cylinder of the branches of the tree is $r_j$, and $L_{ij}$ means the distance between the axis of the envelope cylinder of the robotic arm and the branches of the tree, by comparing the relationship between $r_j+r_i$ and $L_{ij}$, it is determined whether the robotic arm collides with the tree branch.

If the position coordinates of joint $i$ and joint $i+1$ of the robotic arm are ($x_i$, $y_i$, $z_i$) and ($x_{i+1}$, $y_{i+1}$, $z_{i+1}$), respectively, then the $C_i$-axis equation of the cylindrical envelope of the connecting rod is as follows:

$${\displaystyle \frac{x-x_i}{x_{i+1}-x_i}}={\displaystyle \frac{y-y_i}{y_{i+1}-y_i}}={\displaystyle \frac{z-z_i}{z_{i+1}-z_i}}=t\hspace{1em}t\in (0,1)$$

(1)

In the formula: $t$—scale factor; $i$—a positive integer from 0 to 7.

Simplify the above equation to the following:

$$\left\{\begin{array}{c}{x}_i(t)=(x_{i+1}-x_i)t+x_i\\ y_i(t)=(y_{i+1}-y_i)t+y_i\\ z_i(t)=(z_{i+1}-z_i)t+z_i\end{array}\right.\hspace{1em}t\in (0,1)$$

(2)

The branch $D_j$ of the sand fruit tree is enveloped by a cylinder, and the position coordinates of the two ends of the branch are ($x_j$, $y_j$, $z_j$) and ($x_{j+1}$, $y_{j+1}$, $z_{j+1}$). The linear equation of the axis of the cylinder $D_j$ enclosed by the two ends of the branch is as follows:

$${\displaystyle \frac{x-x_j}{x_{j+1}-x_j}}={\displaystyle \frac{y-y_j}{y_{j+1}-y_j}}={\displaystyle \frac{z-z_j}{z_{j+1}-z_j}}=q\hspace{1em}q\in (0,1)$$

(3)

Similarly, the above equation can be simplified as follows:

$$\left\{\begin{array}{c}{x}_j(q)=(x_{j+1}-x_j)t+x_j\\ y_j(q)=(y_{j+1}-y_j)t+y_j\\ z_j(q)=(z_{j+1}-z_j)t+z_j\end{array}\right.\hspace{1em}q\in (0,1)$$

(4)

By using the cylindrical bounding box collision detection method, the linkage of the robotic arm, the end effector, and the branches of the fruit tree are enveloped in a cylinder to simplify the collision detection process and improve detection efficiency. When a collision is detected between the robot and the branches of the sand fruit tree, further research on autonomous obstacle avoidance of the picking robot is needed to plan a reasonable obstacle avoidance path, so that the picking robot will not collide with the branches of the sand fruit tree during the picking process.

4. ACO Algorithm Path Planning and Improvement

For the problem of sequential path planning for multiple crabapple picking in three-dimensional space, the complexity of path planning increases, requiring the algorithm to have strong robustness and global search capabilities. The Ant Colony Optimization (ACO) algorithm has shown good optimization results, strong robustness, and global search capabilities in solving the Three-Dimensional Traveling Salesman Problem (TSP), which is more suitable for solving spatial multi-objective planning problems. However, the algorithm still has some drawbacks, such as long computation time, slow convergence speed, and the tendency to become stuck in local optimal solutions. Therefore, the ACO algorithm was improved and optimized in this paper, and it was used as a spatial algorithm for the multi-objective crabapple picking sequence planning algorithm.

4.1. Improved Transfer Probability

When the distribution of multiple fruit positions on a fruit tree is relatively random, if the picking robot picks randomly and in no order based on the spatial location information of the fruit obtained, a lot of picking time may be wasted, and the picking efficiency may be reduced. Therefore, the spatial locations of multiple fruits on the identified tree will be analyzed and studied, and the picking sequence of multiple clustered crabapples will be planned to save picking time and improve picking efficiency. In this paper, the improved ACO algorithm was used to plan the picking sequence of multiple targets on the crabapple tree.

The analysis of the traditional ACO algorithm reveals that the pheromone factor $\alpha$ and the heuristic function factor $\beta$ in its transition probability function are constants [17]. However, research has shown that when $\alpha$ and $\beta$ are set to be too large or too small, the ACO algorithm may exhibit a decrease in search efficiency, leading to the generation of local optimal solutions or an increased search randomness that hinders the discovery of the optimal solution. Therefore, this paper adopts a dynamic heuristic function factor $\beta$ and pheromone factor $\alpha$, allowing them to change with the number of iterations (the dynamic heuristic function factor $\beta$ is a key parameter used to regulate the degree of influence of heuristic information on ant decision-making. Unlike traditional fixed $\beta$ values, dynamic $\beta$ can adaptively adjust based on algorithm runtime, search status, or environmental changes, thus balancing the exploration and development capabilities of the algorithm). After analyzing the ACO algorithm, it is found that during the initial phase of the iteration loop, $\beta$ plays a significant role in path search, and thus, $\beta$ should be set to a larger value while $\alpha$ should be set to a smaller value. As the number of iterations increases, the concentration of pheromones on relatively shorter paths in global path planning is higher. At this point, the concentration of pheromones takes the lead in the path search, and to better find the optimal path and accelerate convergence, the value of $\alpha$ should be increased while the value of $\beta$ should be decreased. However, as the iteration progresses toward the end, the concentration of pheromones on relatively shorter paths is significantly higher than on other paths. To avoid premature convergence, the impact of pheromone concentration on path selection should be reduced, which means decreasing the value of $\alpha$ and increasing the value of $\beta$. This adjustment will facilitate the search for the global optimal route. The improved $\alpha$ and $\beta$ are as follows:

$$\alpha \left(N\right)=A\sin ({\displaystyle \frac{N\pi }{N_{\max }}})+C$$

(5)

$$\beta \left(N\right)=B\cos ({\displaystyle \frac{2N\pi }{N_{\max }}})+D$$

(6)

In the formula, $N$ is the current iteration number, $N_{\max }$ is the maximum iteration number, and A, B, C, and D are constants.

The improved transfer probability is the following:

$$p_{ij}^k\left(t\right)=\left\{\begin{array}{cc}{\displaystyle \frac{{\left[{\tau }_{ij}\left(t\right)\right]}^{\alpha \left(N\right)}{\left[{\eta }_{ij}\left(t\right)\right]}^{\beta \left(N\right)}}{{\displaystyle \sum _{s\in we{d}_k}\left[{\tau }_{is}^{\alpha \left(N\right)}\left(t\right)\right]\left[{\eta }_{is}^{\beta \left(N\right)}\left(t\right)\right]}}},& j\in we{d}_k\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} 0\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} ,& otherwise\end{array}\right.$$

(7)

In the formula

${\tau }_{ij}(t)$ represents the pheromone concentration at the persimmon picking point $i$ to picking point $j$, ${\eta }_{ij}(t)$ is the heuristic function,

${\eta }_{ij}=1/D_{ij}$ is the desired degree of Ant $k$ from the crabapple picking point $i$ to the picking point $j$,

$D_{ij}$ is the Euclidean distance from picking point $i$ to picking point $j$, and ${wed}_k$ is an ant $k$ that has not been collected at the picking point.

4.2. Optimized Pheromone Concentration Updates

In the process of path planning, the update of pheromones is a crucial task that affects the global search ability of the path, in which the volatile factor $\rho$ is one of the key factors affecting the concentration of pheromones on the path [18]; when $\rho$ is too small, the ethereal volatilization of pheromones on the path is slow, resulting in the accumulation of pheromones on the path, causing the algorithm to converge prematurely and fall into the local optimum. When $\rho$ is too large, the pheromone volatilization on the path is faster, and the search randomness and global search are enhanced, but the convergence time is longer. Therefore, in this paper, the adaptive volatilization factor $\rho$ is introduced to optimize the update of pheromones, and, at the beginning of the algorithm iteration, $\rho$ takes a larger value to expand the randomness and globality of the search. In the later stage of algorithm iteration, $\rho$ takes a smaller value to improve the convergence speed of the algorithm. The improved $\rho$ can be calculated as follows:

$$\rho \left(N\right)=\cos \left({\displaystyle \frac{N}{N_{\max }}}\right)$$

(8)

$\rho (N)$ decreases with the increase in the number of iterations, and the pheromone concentration disappears faster in the previous stage of path planning, and the global search is strong. With the increase in the number of iterations, the path planning tends to be stable, and the convergence speed accelerates. The optimized pheromone update rules are as follows:

$$\left\{\begin{array}{c}{\tau }_{ij}\left(t+1\right)=\left(1-\rho \left(N\right)\right){\tau }_{ij}\left(t\right)+\Delta {\tau }_{ij}\left(t\right)\\ \hspace{1em}\Delta {\tau }_{ij}\left(t\right)={\displaystyle \sum _k^m\Delta {\tau }_{ij}^k\left(t\right)}\end{array}\right.$$

(9)

In the formula, $\Delta {\tau }_{ij}^k(t)$ represents the increment of pheromone from apricot picking point $i$ to picking point $j$ by ant $k$.

$$△{\tau }_{ij}^k(t)=\left\{\begin{array}{c}{\displaystyle \frac{Q}{L_k}},(Ant k from point i to point j)\\ 0,\hspace{1em}otherwise\end{array}\right.$$

(10)

In the formula, Q is a constant, $L_k$ represents the path taken by ant $k$.

The specific algorithm flow of the improved ACO algorithm is as follows:

(1) Initialize the parameters, such as the number of picking points n and the number of ants m. The initial pheromone concentration is ${\tau }_{ij}(0)=c$ (c is a constant), the sum of the pheromones left by the ants on the path from picking point i to j at the initial moment is $\Delta {\tau }_{ij}(0)=0$, the initial number of iterations is $N=0$, and the maximum number of iterations is $N_{\max }$.

(2) Place m ants randomly at n picking points and add the ant placement points to the taboo table ${ban}_k$.

(3) The improved transition probability principle is used for ant $k$( $k$= 1 to m), ant k moves to the next picking point, and the moved picking point is added to the ${ban}_k$ until the n picking points are traversed.

(4) Calculate the path length of each ant ${}^{k}l_{ij}$, (${}^{k}l_{ij}$ represents the path of the kth ant from picking point $i$ to picking point $j$, $k$ = 1,2,3, …, m).

(5) Calculate the path length $L_{\min }={\displaystyle \sum _{z=1}^n{}^{k}l_{ij}^z}$ traveled by ant $k$ after completing an iterative cycle and update the global shortest path.

(6) Update the pheromone concentration on the path according to Formulas (4), (5), and (6).

(7) Prepare to continue the looping again while $N=N+1$, if $N>N_{\max }$, the loop ends and outputs the optimal solution. Otherwise, return to Step (2).

The improved ACO algorithm flowchart is shown in Figure 7, and the pseudocode can be found in Supplementary S1.

4.3. Comparative Analysis of Parameter Influence

Set up two control groups, one is the traditional ACO and the other is the improved ACO. In the traditional ACO, the three parameters $\alpha$, $\beta$, and $\rho$ are static values, while in the improved ACO, the three parameters $\alpha$, $\beta$, and $\rho$ are dynamic values, as shown in

Table 4. Parameter setting.

Parameter	Traditional ACO	Improve ACO
$\alpha$	$\alpha =1.0$	$\alpha (N)=A\sin (N\pi /N_{\max })+C$
$\beta$	$\beta =2.0$	$\beta (N)=B\sin (2N\pi /N_{\max })+D$
$\rho$	$\rho =0.1$	$\rho (N)=\cos (N/N_{\max })$

.

The experimental test questions are eil51 (51 cities) and att48 (48 cities) in TSPLIB. The parameters are 50 ants, 200 iterations, and an average of 10 runs. The results are shown in

Table 5. Important parameter test results.

Algorithm	Path Length (eil51)	Convergent Algebra	Computing Time (s)	Path Length (att48)	Convergent Algebra
Traditional ACO	$436.2\pm 5.7$	$142$	$12.4$	$335.1\pm 4.2$	$128$
Only dynamic $\alpha$	$429.5\pm 4.9$	$118$	$12.1$	$330.8\pm 3.8$	$105$
Only dynamic $\beta$	$427.8\pm 6.2$	$156$	$12.7$	$328.4\pm 4.5$	$140$
Only dynamic $\rho$	$431.3\pm 5.1$	$135$	$12.3$	$332.6\pm 3.9$	$122$
Improve ACO	$423.6\pm 3.8$	$95$	$12.5$	$324.2\pm 2.7$	$88$

.

The results show that in terms of path length. Improved ACO optimizes 2.9% and 3.3% on eil51 and att48, respectively, compared to traditional ACO. Improving ACO convergence speed reduces convergence algebra by 33%, and the additional computational overhead introduced by dynamic parameters can be ignored in terms of computational efficiency (time difference < 2%). It can be proven that dynamic parameters perform consistently better than static parameters on multiple TSP.

4.4. Comparison of Ant Colony Algorithm Before and After Improvement

The improved ACO algorithm was compared with the traditional ACO algorithm, and the simulation experiments were conducted to verify the feasibility and superiority of the improved ACO algorithm. The simulation results of the picking sequence planning for multiple target picking points in three-dimensional space are shown in Figure 8, where the red solid indicates the starting point of the end effector of the robotic arm. Figure 8a indicates the picking sequence planned by the traditional ACO algorithm, Figure 8b indicates the optimal path length of each iteration of the traditional ACO algorithm, Figure 8c shows the picking order of the improved ACO algorithm, and Figure 8d shows the optimal path length of each iteration of the improved ACO algorithm. The test results are shown in

Table 6. Comparison of data before and after improvement.

Algorithm	Path Length/mm	Number of Iterations/Times	Convergence Time/s
Traditional ACO	1314.82	48	0.98
Improved ACO	1279.95	30	0.54

.

The improved ACO algorithm and its predecessor were planned in a 100 × 100 × 1200 three-dimensional space, with the starting coordinates at (0, 50, 600) (a red solid point), to map out the sequence of multiple crabapple picking paths for the end-effector of a robotic arm, starting and ending at the same point. As shown in Figure 8a,c, both the planning algorithms can successfully plan the sequence of picking multiple points without traversing the same picking point multiple times, thus validating the effectiveness and feasibility of the proposed algorithm. By comparing Figure 8b,d and Table 4, it can be seen that the improved ACO algorithm has shorter paths, fewer iterations, and faster convergence speed. Therefore, the improved ACO algorithm in this paper outperforms the traditional ACO algorithm in planning the sequence of picking multiple points.

5. Improve Obstacle Avoidance Path Planning for RRT

5.1. Based on RRT and APF Algorithms

The rapidly exploring random tree (RRT) algorithm is a type of random sampling method in which the starting point is used as the root node and the target point is used as the end node, and the intermediate node is added to expand a random tree by randomly selecting the sampling point in sampling space [19]. The algorithm has the advantage of probabilistic completeness, which can search for the picking path without solving all the spatial points, and quickly and effectively plan the picking path in the 3D sampling space. The RRT algorithm has a strong search ability and is suitable for high-dimensional spaces. However, it has some disadvantages, such as high randomness of sampling points, which may result in a large number of offset target points, leading to long operation time and uneven paths planned by the algorithm [20].

Artificial potential field (APF) is a widely used obstacle avoidance path planning algorithm, which assumes that there is repulsion on the obstacle and gravity at the target point, and the combined force of gravity and repulsion is used as the direction of motion of the moving object, and the target point has an attraction to the moving object, inducing the moving object to move towards the target point, and the obstacle has a repulsive force on the object, so as to effectively avoid the collision between the moving object and the obstacle [21]. Its advantage is that it does not need to accurately obtain the spatial location of obstacles and can effectively avoid obstacles in space. However, when the moving object is subjected to the repulsive force and the force of attraction are in the same straight line and in opposite directions, the moving object falls into the local optimal solution and cannot reach the target position, or when the distance between multiple obstacles is close, the robotic arm is prone to unstable states such as wandering and shaking, resulting in vibration and deadlock phenomenon [22]. Therefore, the APF algorithm is not suitable for obstacle avoidance path planning of the robotic arm alone.

The artificial potential field contains two types of force fields: the gravitational field formed by the position of the moving target and the repulsive field formed by obstacles.

$$U(q)=U_{att}(q)+U_{rep}(q)$$

(11)

Among them, $U_{att}$ is the gravitational field that guides the robot to move towards the target position; $U_{rep}$ is a repulsive field that guides robots to avoid obstacles.

The gravitational field function is as follows:

$$U_{att}(q)=\left\{\begin{array}{cc}{\displaystyle \frac{1}{2}}\zeta d^2(q,q_{goal})& d(q,q_{goal})\le d_{goal}^{\ast }\\ d_{goal}^{\ast }\zeta d(q,q_{goal})-{\displaystyle \frac{1}{2}}\zeta {(d_{goal}^{\ast })}^2& d(q,q_{goal})>d_{goal}^{\ast }\end{array}\right.$$

(12)

Its gradient is as follows:

$$\nabla U_{att}(q)=\left\{\begin{array}{cc}\zeta (q-q_{goal})& d(q,q_{goal})\le d_{goal}^{\ast }\\ {\displaystyle \frac{d_{goal}^{\ast }\zeta (q-q_{goal})}{d(q,q_{goal})}}& d(q,q_{goal})>d_{goal}^{\ast }\end{array}\right.$$

(13)

The gravitational field function is a piecewise function, and when $d<d_{goal}^{\ast }$, the magnitude of gravitational potential energy is proportional to the square of the distance from the current position to the target position; when $d>d_{goal}^{\ast }$, reduce the value of the gravity calculation function to avoid the problem of excessive gravity when moving away from the target position.

The commonly used functions for repulsive fields are as follows:

$$U_{rep}(q)=\left\{\begin{array}{c}{\displaystyle \frac{1}{2}}\eta {({\displaystyle \frac{1}{D(q)}}-{\displaystyle \frac{1}{Q^{\ast }}})}^2, D(q)\le Q^{\ast }\\ 0\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}, D(q)>Q^{\ast } \end{array}\right.$$

(14)

Its gradient is as follows:

$$\nabla U_{rep}(q)=\left\{\begin{array}{c}\eta ({\displaystyle \frac{1}{Q^{\ast }}}-{\displaystyle \frac{1}{D(q)}}){\displaystyle \frac{1}{D^2(q)}}\nabla D(q), D(q)\le Q^{\ast }\\ 0\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}, D(q)>Q^{\ast } \end{array}\right.$$

(15)

Among them, $D(q)$ is the distance to the nearest obstacle; $\eta$ is the repulsive gain constant; and $Q^{\ast }$ is the threshold range of the obstacle’s action. Within this threshold range, the obstacle will generate a repulsive force, and beyond this range, no repulsive force will be generated. The superposition of gravitational and repulsive fields forms an artificial force field.

According to the respective advantages and disadvantages of common path planning algorithms, the RRT algorithm and APF algorithm were combined to plan the collision-free picking path of the picking points on the crabapple tree in this paper.

5.2. Improvements to RRT

The APF algorithm is a good obstacle avoidance algorithm widely used in obstacle avoidance paths. By assuming that there are attractive and repulsive forces between the target point and obstacles in the workspace, the moving object is subjected to the combined force of the two and continuously moves towards the target point. This algorithm guides the moving object to move towards the target point while avoiding collision with obstacles during the movement. The movement direction of the APF path planning algorithm is shown in Figure 9.

The RRT algorithm and the APF algorithm were combined to enhance the search towards the picking point. Integrating the gravitational field of the APF algorithm into the RRT algorithm, the target point had a gravitational field guiding the nodes of the random tree to deviate towards the target point, which reduced unnecessary redundant nodes and shortened computation time, thus accelerating the convergence speed. The specific algorithm process was as follows:

(1)

Set $q_{start}$ as the starting point of exploration, $q_{goal}$ as the final destination point, where both $q_{start}$ and $q_{goal}$ are points in the workspace $C_{free}$;

(2)

Set $q_{start}$ as the root node of random tree $L$;

(3)

Sampling points $q_{rand}$ are obtained by random sampling in the working interval $C_{free}$ for each iteration;

(4)

Search for the node closest to $q_{rand}$ in the random tree $L$, denote it as $q_{near}$;

(5)

Starting from $q_{near}$, generate a new node $p_1$ with a specific step size in the direction from $q_{near}$ to $q_{rand}$, and generate a new node $p_2$ under the gravitational field of the target point $q_{goal}$ with a certain step size from $q_{near}$, where both new nodes $p_1$ and $p_2$ are in the working space $C_{free}$;

(6)

Generate a new node $q_{new}$ in the direction of the diagonals formed by the parallelogram from $q_{near}$ to $p_1$ and from $q_{rand}$ to $p_2$; then, denote the path from node $q_{rand}$ to the new node $q_{new}$ as $Ei$;

(7)

Judge whether $Ei$ meets the obstacle avoidance conditions within the working area $C_{free}$. If it does, set $q_{new}$ as the new $q_{rand}$ node; if not, return to step (3) to re-find a randomly sampled point $q_{rand}$;

(8)

If the obstacle avoidance conditions are met, repeat steps (3) to step (7) in the next iteration cycle until node $q_{goal}$ is reached, and $q_{start}$ to $q_{goal}$ paths are output. The improved RRT algorithm flowchart is shown in Figure 10, and the pseudocode can be found in Supplementary S2. The comparison chart of RRT before and after improvement is shown in Figure 11.

In hybrid algorithms, the guiding role of artificial potential fields is mainly reflected in three key aspects:

(1)

In the node sampling stage, although the basic characteristics of random sampling are still maintained, the continuous guidance of the gravitational field makes the sampling points naturally biased towards the direction of the target point, significantly improving the target directionality.

(2)

During the tree expansion process, whenever a new node is generated, APF calculates the attraction and repulsion in real-time based on the current environment. Attraction pulls the node towards the target point, while repulsion pushes the node away from obstacles. The balance of these forces allows the new node to automatically avoid obstacles while maintaining the expansion direction.

(3)

At the level of overall path optimization, the continuous effect of APF enables the tree structure grown to naturally form smooth detours in areas with dense obstacles, while maintaining the characteristic of approaching the target in a straight line in open areas. This not only avoids the redundant exploration that may occur in traditional RRT but also makes the final path both safe and optimal through dynamic adjustment of the force field.

5.3. Comparison of RRT Before and After Improvement

The improved RRT algorithm after adding the gravitational field is compared with the original RRT algorithm, and simulation experiments were carried out to analyze and verify the validity and superiority of the improved RRT algorithm, with red representing the starting point (0, 0, 0) and blue representing the endpoint (700, 800, 1000), using three types of models, orange cylinders, pink spheres, and violet cuboids, as obstacles. The simulation test results are shown in Figure 12, where Figure 12a depicts the planning path of the RRT algorithm before improvement, and Figure 12b depicts the planning path of the improved RRT algorithm.

The improved front and rear RRT algorithms are compared in a three-dimensional space of 1000 × 1000 × 1000, with a starting point of (0, 0, 0) and an ending point of (700, 800, 1000). The collision-free path from the starting point to the endpoint was planned in a three-dimensional space with obstacles. It can be seen from Figure 12 that the path planning of the RRT algorithm with gravitational field extends rapidly towards the endpoint to avoid random diffusion of random trees. At the same time, the path planning search time of the RRT algorithm with gravitational field from the start to the endpoint is 2.17 s, while the path planning search time of the RRT algorithm without improvement is 7.34 s. Therefore, it can be concluded that the search orientation of the improved RRT algorithm can shorten the operation time and improve the search efficiency.

The overall algorithm flow chart is shown in Figure 13.

6. Picking Test and Result Analysis

6.1. Identification and Positioning

The YOLOv5 algorithm, improved by incorporating CBAM and BiFPN structures, was adopted, which utilizes the Realsense D415 depth camera for the identification, localization, and detection of crabapples from different angles, fruit quantities, and cluster quantities on trees. The three-dimensional spatial coordinates of crabapple picking points were outputted and saved, as illustrated in Figure 14b for the random placement of five clustered crabapples on a tree, and Figure 14d for the identification and localization of three clustered crabapples at random positions on a tree. The corresponding output coordinates are listed in Table 5. Furthermore, the actual distance from the camera to the fruit was measured using a measuring tool for error analysis.

Comparing the spatial coordinates of crabapples obtained by the D415 camera with the actual measured positions, it was found that there was a certain error between the coordinates obtained by the D415 camera and the actual measurement results, as shown in

Table 7. Three-dimensional coordinates of crabapple space.

Number of Clustered Crabapples	Picking Point	D415 Measurements (X, Y, Z)/m	Actual Measurements (X, Y, Z)/m	Absolute Positioning Error (X, Y, Z)/m
5	A	(0.203, 0.105, 0.595)	(0.206, 0.110, 0.600)	(0.003, 0.005, 0.005)
	B	(0.132, −0.064, 0.512)	(0.137, −0.068, 0.514)	(0.005, 0.004, 0.002)
	C	(−0.019, −0.072, 0.472)	(−0.023, −0.075, 0.475)	(0.004, 0.003, 0.003)
	D	(−0.137, 0.044, 0.651)	(−0.140, 0.040, 0.653)	(0.003, 0.004, 0.002)
	E	(−0.189, −0.034, 0.487)	(−0.192, −0.038, 0.490)	(0.003, 0.004, 0.003)
3	A	(0.176, −0.043, 0.548)	(0.180, −0.045, 0.550)	(0.004, 0.002, 0.002)
	B	(−0.023, −0.068, 0.538)	(−0.020, −0.070, 0.542)	(0.003, 0.002, 0.004)
	C	(−0.146, 0.029, 0.550)	(−0.145, 0.032, 0.552)	(0.001, 0.003, 0.002)

. Specifically, the average error in the X-axis direction is 3.25 mm, 3.375 mm in the Y-axis direction, and 2.875 mm in the Z-axis direction. By calculating the relative error for each picking point, it was found that the relative error for each group did not exceed 2%. This indicated that using the D415 camera for crabapples positioning was relatively accurate and met the positioning requirements, thus proving the effectiveness and feasibility of the proposed method of using the Realsense D415 depth camera for clustered crabapples recognition and localization.

The error of the camera has a potential influence on practical application and should be paid attention to. For example, when applied to robot navigation, if the error is large, the robot may misjudge the distance of obstacles, resulting in collision or inaccurate path planning; real-time positioning and mapping will affect the quality of 3D point clouds, causing the map to distort or drift. When applied to 3D scanning and modeling, the error will cause noise on the surface of the scanned object, resulting in distortion of the 3D reconstructed model.

6.2. Multi-Objective Picking Sequence Planning on Crabapple Trees

The problem of picking the sequence of multiple crabapples on the tree was simplified to the TSP in three-dimensional space. Based on the spatial coordinate position information identified and located in the previous text, the improved ACO algorithm was used to pick the sequence of three clustered crabapples and five clustered crabapples on the tree, respectively. In order to better reflect the picking sequence, the spatial coordinate unit of the clustered crabapples was converted from m to mm, the initial positions of the end effector of the robotic arm were represented by solid red dots, and the picking points of clustered crabapples were represented by hollow blue dots. The planning results of the picking sequence of five clustered crabapples on a three-dimensional fruit tree are shown in Figure 15a. The corresponding picking sequence on the tree was represented by red sequence numbers and red arrows, as shown in Figure 15b. Similarly, the planning results of the picking sequence of three clustered crabapples and the corresponding picking sequences on the tree are shown in Figure 16.

6.3. Simulation Analysis of Crabapple Picking Robotic Arm

Based on the three-dimensional reconstruction of the branches of crabapple trees, the improved RRT algorithm was used to analyze and study the simulation experiment of obstacle avoidance paths for robotic arm picking. In this experiment, Matlab 2020a was used as the simulation platform for obstacle avoidance and the picking path planning of crabapples. The AUBO-E5 six-degree-of-freedom robot was employed to pick crabapples, and the D-H model of the AUBO-E robot was created. The robot toolbox in Matlab was used to solve the motion of the robotic arm. The initial position of the robotic arm was [0.57, 0.41, 0.1], and the crabapples were located on the slender branches between the two main branches (with a radius less than 3 mm) and on the main branches. The tree was reconstructed in three dimensions, and the improved RRT algorithm was used to plan the path for picking crabapples. The analysis was focused on whether the joints and links of the robotic arm collided with the tree during the picking process. If no collision occurred, the end of the robotic arm would start from its initial position for crabapple picking. If a collision occurred, the path would be replanned. The simulation of picking and obstacle avoidance for single and multiple crabapples on the tree is shown in Figure 17.

Continuous picking experiments were conducted, respectively, on a single crabapple and multiple crabapples from thin branches (with a radius less than 3 mm) between two thick branches of the tree. The experiment results showed that there was no collusion between the robotic arm and the branches of the tree during the picking process. It verified that the improved algorithm presented in this article could successfully plan the obstacle avoidance path for the robotic arm, ensuring that the robotic arm could avoid collision with the thick branches of the tree during picking operations, and complete the predetermined work tasks.

6.4. Multi-Target Clustered Crabapples Picking Experiment

6.4.1. Rigid-Flexible Pneumatic Coupling Picking Manipulator

In order to avoid damage to the crabapples by the end effector and ensure the strength and stiffness of the robotic arm during the picking process, the rigid-flexible pneumatic coupling picking manipulator, self-made by the research group, was used as the end effector for these picking experiments, as shown in Figure 18.

The rigid-flexible pneumatic coupling picking manipulator consists of six pneumatic soft fingers, a rigid skeleton, a rubber pad, and two connecting parts. Its working principle is as follows. The 0.08 MPa air pressure is used to drive the soft fingers to bend, envelope the clustered crabapples, and cooperate with the rigid skeleton to complete the task of picking the clustered crabapples on the tree. The soft fingers are made of silicone material to avoid damage to the crabapples caused by the rigid structures.

6.4.2. Construction of the Picking Test Platform

In this test, the rigid-flexible pneumatic coupling picking manipulator was installed on the AUBO-E5 robotic arm, and the D415 camera was fixed on the picking platform at a relatively stationary position relative to the base of the robotic arm. The crabapples were identified and positioned, and their identification and positioning information were transmitted to the controller of the robotic arm. The robotic arm and the rigid-flexible pneumatic coupling picking manipulator were controlled to cooperate to complete the picking task by the controller. The picking test platform is shown in Figure 19.

6.4.3. Crabapples Picking Test

In order to verify the feasibility of the picking test platform for crabapple picking, validation tests of the overall picking operation effect were conducted after completing the construction of the test platform. The artificial simulation crabapple tree was used for the picking tests, and the crabapple picking test progress is shown in Figure 20.

From Figure 20, it can be seen that the end effector moves to the first picking point from the initial position, the soft fingers are used to envelop the clustered crabapples, and then the clustered crabapples are picked through the end effector movement. As the fingers are released, the picked crabapples will be transported to the collection basket through the conveyor pipe. By the improved RRT obstacle avoidance path planning, the end effector is moved to the second picking point to realize the picking of the second clustered crabapples. Then, the picking of other clustered crabapples on the tree is completed sequentially, and finally, the end effector is returned to the initial position to wait for a new picking task.

To further verify the feasibility and accuracy of the clustered crabapple picking test, multiple groups of different clusters (ranging from 1 to 6) were placed at random positions. The success rate of recognizing, positioning, and picking clustered crabapples, as well as the number of collisions between the robotic arm and the tree, were used as evaluation indicators for the crabapples-picking robot. The experimental results are shown in

Table 8. Results of clustered crabapple picking tests.

Group	Number of Picking /Clusters	Number of Successful Recognizing/Clusters	Number of Successful Positioning /Clusters	Number of Successful Picking /Clusters	Recognizing Success Rate/%	Positioning Success Rate/%	Picking Success Rate/%	Number of Collisions/Times
1	3	3	3	3	100	100	100	0
2	4	4	4	4	100	100	100	0
3	5	5	5	5	100	100	100	0
4	6	6	6	5	100	100	83.33	0
5	7	6	6	6	85.71	85.71	85.71	0
6	7	7	7	6	100	100	85.71	0
7	8	7	7	7	87.50	87.50	87.50	0
8	8	8	7	7	100	87.50	87.50	0
sum	48	46	45	43	95.83	93.75	89.58	0

.

From the experimental data in Table 8, it can be seen that the average recognition success rate of the robot for clustered crabapples (1 to 6) is 95.83%, the average positioning success rate is 93.75%, and the average picking success rate is 89.58%. Moreover, there are no collisions between the robotic arm and the main branches of the tree during the picking process, which meets the actual picking needs. Therefore, it can be concluded that the algorithm proposed in this article for recognizing, positioning, and planning the robotic arm’s picking path of clustered crabapples is effective and feasible.

7. Discussion of Experimental Results

(1)

In terms of the rationality of the experimental design, the experiment simulated the growing environment of sand fruit based on the real picking scene. Five and three sand fruit types were set at different locations on the sand fruit tree, and the number of sand fruit was different for each cluster, covering diverse scenes such as different cluster distributions, fruit density, and branches and leaves occlusion degree. Compared with the traditional AOC algorithm and the traditional RRT algorithm, the path planning and path length are set up in the experiment. Through the comparison of simulation data, the superiority of the algorithm is demonstrated.

(2)

As for the scientific evaluation of the results, the experimental results show the success rate of identification, positioning, picking, and collision with the robotic arm of different groups, which are all important evaluation indicators to measure the obstacle avoidance algorithm, and the experimental results are scientific, rigorous, and convincing.

(3)

The experimental results also have an impact on the actual picking operation. The experimental results show that the algorithm can improve the picking efficiency of the picking robot arm in the actual scene, which reflects the adaptability to the clustered sand fruit environment and meets the requirements of continuous picking.

(4)

In terms of improvement and applicability, the algorithm planning effect can be further improved by combining deep learning, reinforcement learning, and other methods. The algorithm can be applied not only to sand fruit trees, but also to grapes, cherries, and other clustered fruits.

8. Conclusions and Future Work Plan

8.1. Conclusions

In order to achieve a continuous obstacle-avoidant picking of multi-target clustered crabapples on the tree, and avoid collision with the tree branches during the picking process, the picking sequence and obstacle avoidance plan for picking clustered crabapples on trees were studied to improve the efficiency of picking and save picking time.

(1)

The problem of picking the sequence of multiple crabapples on the tree was simplified to the TSP in three-dimensional space. The ACO algorithm was improved by improving the transition probability and optimizing the pheromone concentration update. Compared with the traditional ACO algorithm, it can be concluded that the improved ACO algorithm has shorter picking sequence paths, fewer iterations, faster convergence speed, and can effectively plan the picking sequence of multiple crabapples on the tree.

(2)

To avoid collisions between the robotic arm and the tree branches during picking progress, which may cause damage to the robotic arm and the trees, the APF algorithm and the RRT algorithm were combined, and a gravitational field was introduced to enhance the directional guidance of the RRT algorithm’s path search. Compared to the original RRT algorithm, it can be concluded that the improved RRT algorithm has fewer turning points and faster convergence speed and can achieve an obstacle-avoidant picking path planning for the robotic arm.

(3)

To verify the effectiveness and feasibility of the algorithm proposed in this article, a picking test platform was constructed to conduct multiple groups of picking tests on clustered crabapples (1 to 6 clusters) on an artificial simulation crabapple tree. The test results show that the average recognition success rate is 95.83%, the average positioning success rate is 93.75%, and the average picking success rate is 89.58%. Moreover, there are no collisions between the robotic arm and the main branches of the tree during the picking process, and the robotic arm can pick the crabapples according to the planned picking sequence with no collision, meeting the actual picking requirements and achieving the goal of picking.

8.2. Future Work Plan

(1)

The rigid-flexible pneumatic coupling picking manipulator needs to be further optimized. Although some fruits can be successfully identified and positioned, the picking manipulator cannot fully envelop and pick fruits, thus reducing the success rate of picking.

(2)

With the continuous change and development of computer technology, target detection technology is also constantly changing. This study adopts the YOLOv5 algorithm as the basis for research, and other novel detection algorithms can be explored in the following research.

(3)

In this study, the AUBO-E5 robotic arm is used, which has high input and maintenance costs. Next, a robotic arm that meets the requirements of picking will be studied, and collection boxes and walking mechanisms will be assembled for field picking.

(4)

In the later stage, an integrated deep learning model will be added on the basis of this algorithm to improve the detection accuracy and real-time algorithm optimization. The complex orchard picking situation will be deeply studied, and the algorithms of other studies will be copied and compared with this algorithm to improve the obstacle avoidance and retrieval ability of this algorithm.