An Intelligent Operation Area Allocation and Automatic Sequential Grasping Algorithm for Dual-Arm Horticultural Smart Harvesting Robot

Abstract

Aiming to solve the problem that most existing apple-picking robots operate with a single arm and that the overall efficiency of the machine needs to be further improved, a prototype of a dual-arm picking robot was built, and its picking operation planning method was studied. Firstly, based on the configuration and motion mode of the AUBO-i5 robotic arm, the overlapping dual-arm layout of the workspace was determined. Then, a prototype of a dual-arm apple-picking robot was built, and, based on the designed dual-arm spatial layout, a dual-arm picking operation zoning planning method was proposed. The experimental results showed that in the four simulation experiments, the highest value of the maximum parallel operation proportion of the dual arms was 83%, and the lowest value was 50.6%. The highest value of the maximum operation length of the single arm was 7323 mm, and the lowest value was 5654 mm. The total length of the dual-arm operation path was 12,705 mm, and the lowest value was 8770 mm. Furthermore, a fruit-picking sequence planning method based on dual robotic arm operation was proposed. Fruit traversal simulation verification experiments were conducted. The results showed that there was no conflict between the left and right arms during the motion of the dual robotic arms. Finally, the proposed dual-arm robot operation zoning and picking sequence planning method was validated in the apple experimental station. The results showed that the proportion of dual-arm parallel operations was the lowest at 50.7% and the highest at 72.4%. The total length of the dual-arm operation path was the highest at 8604 mm and the lowest at 6511 mm.

1. Introduction

Against the backdrop of accelerating global agricultural modernization, the fruit industry, as an important component of agriculture, is facing a sharp contradiction between increasing production demand and labor shortages [1]. As one of the widely cultivated and economically valuable fruits in the world, apples play a crucial role in the entire production process during the picking process [2,3,4]. The traditional manual picking method not only has high labor intensity and low efficiency but also has increasingly highlighted limitations with the worsening factors of population aging and the continuous rise of labor costs [5,6,7,8,9,10]. Therefore, developing efficient harvesting robots that can replace manual labor has become an inevitable choice to solve the problem of apple harvesting and promote the sustainable development of the apple industry [11].

A single-arm apple-picking robot was built based on a three-finger grasping end effector using a ZED binocular camera. The robot is equipped with a 5-degree-of-freedom robotic arm and the apple branch separation method is grasping separation. The results of the robot’s picking performance test in an apple orchard show that the visual system recognizes one frame of image in 0.026 s, the picking time for a single fruit is 12.53 s, and the picking success rate is 81.6% [12]. Abundant Robotics [13] developed and produced an air suction apple-picking robot, which takes only 1.5 s to pick a single fruit. However, the robot is designed for the “V-shaped” fruit tree wall planting mode of apple trees in foreign countries, while the spindle-shaped apple trees widely planted in China have complex crown spaces (including the occlusion of apples by branches). Therefore, its direct application to the automated fruit picking of spindle-shaped apple trees in modern orchards in China needs further verification. In addition, the air suction apple branch separation method frequently pulls off the apple stem directly, which is not conducive to the storage and sale of fresh apples after harvesting.

The dual-arm apple-picking robot has shown great potential in improving picking efficiency with its dual-arm collaborative operation capability. Compared with single-arm picking robots, dual-arm picking robots can more efficiently complete a series of actions, such as grabbing [14,15,16], picking [17,18], and placing apples. However, in order to fully leverage the advantages of dual-arm apple-picking robots and achieve efficient and precise picking operations, the planning of dual-arm picking operation zones and fruit-picking sequences are two core issues that urgently need to be addressed.

The zoning planning of dual-arm apple-picking operation is the foundation for the efficient operation of dual-arm apple-picking robots. In the actual apple orchard environment, if the picking area is not reasonably divided, the dual-arm robot may encounter problems, such as collision between the two robotic arms during operation, which directly leads to picking failure. Reasonable dual-arm operation zoning can divide the entire fruit-picking area into different zones based on the layout of the dual robotic arms on the robot, allowing the robot to perform dual-arm picking operations according to predetermined zones, ensuring the safety of dual-arm operations. On the other hand, dual-arm operation zoning can also provide a basic framework for subsequent fruit-picking sequence planning, making the development of picking sequence planning methods more reasonable.

The planning of fruit-picking sequence is a key factor determining the operational efficiency of dual-arm apple-picking robots. If the picking sequence of fruits is not properly planned within the work zone responsible for each robotic arm, it may lead to frequent adjustments of the picking posture, increased movement distance, and angle changes in the robotic arm, thereby reducing the picking speed and accuracy. On the other hand, an unreasonable picking sequence may also cause damage to the fruit and fruit trees, affecting the subsequent growth and quality of the fruit. Therefore, it is necessary to propose a scientifically effective dual-arm robot fruit-picking sequence planning method, which comprehensively considers the distance factors between each fruit to be picked and the robotic arm and plans and formulates the optimal fruit-picking sequence, so that the dual-arm apple-picking robot can safely and reliably complete the picking task.

At present, although scholars at home and abroad have conducted some research on agricultural robot operation planning [19,20,21,22,23,24], there is relatively little research on the operation zoning and picking sequence planning methods [25,26,27,28,29] for dual-arm apple-picking robots. Most existing research focuses on the picking planning of single-arm robots, and there is a lack of in-depth research on the partitioning strategy of dual-arm picking robots in complex orchard environments and the fruit-picking sequence planning method that integrates multiple factors.

To address the efficiency limitations of single-arm harvesting robots, this study developed a kind of prototype layout of a dual-arm harvesting robot and proposed a corresponding operational zoning planning method. The primary goal was to establish an operational zoning pattern of a dual-arm harvesting robot for enhancing subsequent harvesting efficiency. Based on prior knowledge of apple harvesting, a fruit-harvesting sequence planning method based on dual-arm operation was proposed. Through this study, this planning method is expected to improve the efficiency of intelligent apple harvesting operations, reduce harvesting costs, provide theoretical support and technical support for the promotion and application of dual-arm apple harvesting robots, and enhance the intelligence level of the apple-harvesting industry.

2. Space Layout of Dual Robotic Arms for Harvesting Robots

The AUBO-i5 [30,31,32,33,34] robotic arm has high-precision positioning and operation capabilities, as well as high motion control accuracy. Its flexible six-degree-of-freedom joint structure enables it to adapt to fruit-picking needs at different heights and angles. On the other hand, the robotic arm can stably carry a load of 5 kg, fully meeting the needs of apple picking and can be used for grasping various varieties of apples. In addition, the robotic arm has multiple safety mechanisms, such as collision detection, torque monitoring, and emergency stop functions, which can avoid harm to operators or fruit trees, meeting the requirements of human–machine cooperation scenarios. AUBO-i5 adopts a modular design, supporting the quick disassembly and replacement of end effectors, and can be adapted to different picking tools (such as flexible grippers, vacuum suction cups, and other intelligent apple-picking hand end effectors). Therefore, this study designs and builds a dual-arm picking robot based on the AUBO-i5 robotic arm. The spindle-shaped apple tree and key structural dimensions of AUBO-i5 are shown in Figure 1 and Figure 2.

The spatial layout design of the dual robotic arms of the picking robot is directly related to the picking space range of the robot and whether there will be physical collisions between the arms during the picking process. Taking into account the continuity of the picking operation area of the dual-arm robot, the collision avoidance safety of parallel operation of the dual robotic arms, and the operational stability of the dual-arm assembly on the robot chassis, the “dual arm working space overlapping type” was determined as the spatial layout of the dual robotic arms of the picking robot.

The front view schematic diagram of the layout of the overlapping dual robotic arms in the workspace is shown in Figure 3, where O₁ and O₂ are the center points of the left and right robotic arm bases, and O is the midpoint of O₁O₂. OO₁ and OO₂ are of equal length, both 550 mm (taking into account the installation space margin of the robotic arm picking end effector), and the power supply sockets of the left and right robotic arms are facing the outside of the paper. In the dual-arm posture shown in the figure below, the rotation angle between joint 2 and joint 3 of the left robotic arm is 90°, the rotation angle between joint 2 and joint 3 of the right robotic arm is −90°, and the rotation angles of the other four joints are all 0°, respectively. Joints 2, 3, and 4 of the robotic arm are represented by J2, J3, and J4, respectively, and the connecting rods of the robotic arm are indicated in the schematic diagram.

Vrep (Virtual Robot Experiment Platform, Version 3.6.2) [35,36] is a software that can achieve the kinematic simulation of robotic arms, and it is mainly used in the field of robot simulation modeling. It can use embedded scripts, remote API clients, etc. to achieve distributed control structures and is an ideal tool for robot simulation modeling. The controller can be implemented in languages, such as C/C++, Python, Java, Lua, Matlab, Octave, or Urbi, to control the execution of relevant actions by the robotic arm in the simulator for visual analysis of the model and output simulation data parameters. The schematic diagram of the spatial overlapping dual-arm layout simulation based on the Vrep software is shown in Figure 4, which provides a more intuitive understanding of the dual-arm spatial layout design.

Due to the nature of the dual-arm picking robot, the left arm is responsible for picking apples in the left cutting area and the right arm is responsible for picking fruits in the right cutting area during operation. Therefore, when using visual constraints to servo-pick apples, there will be no collision between the ends of the arms. On the other hand, based on the configuration of the AUBO-i5 robotic arm, due to the spatial overlapping dual-arm layout, the distance between the left and right arm bases on the far side (2 × 605 mm, as shown in Figure 4) is greater than twice the length of the connecting rod between joint 2 and joint 3 of the AUBO-i5 robotic arm (2 × 520 mm, as shown in Figure 1), so there will be no collision between joint 3 of the two arms and the connecting rod between joint 2 and joint 3. Although there are overlapping areas in the picking workspace, the double-arm layout with an overlapping workspace can achieve collision avoidance by using visual constraints to pick fruits in the left and right areas separately. In order to analyze the workspace distribution of the left and right robotic arms under the dual-arm layout more intuitively, a schematic diagram of the overlapping dual-arm workspace is drawn based on the Python programming language (Version 3.7.10), as shown in Figure 5.

Among them, O₁ and O₂ are the center points of the left and right robotic arm bases, respectively. The workspace is formed by partially overlapping and merging two single-arm hemispherical workspace intervals, with a hemispherical workspace radius of 886.5 mm. Based on this layout, the image of the dual robotic arm built on the robot chassis is shown in Figure 6.

3. Partition Planning Method for Double-Arm Picking Operation

In order to avoid collisions during the operation of the dual-arm picking robot, the working space of both arms is directly divided, so that each arm can perform its own duties and the working areas do not interfere with each other. This method effectively avoids the problem of conflicts between different robotic arms and can achieve parallel work of both arms. Based on the Python programming language, constraints are imposed on the picking operation area of the robot. The implementation method is based on the division of the operation area perceived by the robot vision system to plan and guide the robotic arms to work in their respective picking areas.

The specific core components of each component of the picking robot are as follows: dual robotic arms, robotic arms, a visual perception system, a robot tracked mobile chassis, and a robot control terminal. Within the spatial range faced by the D435 depth camera (as indicated in Figure 6), the spatial area is divided into two equal parts based on its horizontal axis, and the picking space is divided into left and right parts, corresponding to the working areas of the left and right robotic arms, respectively. The visual system allocates the picking area to the apple target based on its horizontal coordinate value in the camera coordinate system. The fruit on the left side of the picking area boundary is divided into the working area of the left robotic arm, and the fruit on the right side of the picking area boundary is divided into the working area of the right arm. During the robot picking operation, after opening the visual terminal for fruit target recognition and positioning, two robotic arm control terminals are opened separately to control the parallel motion of both arms.

The steps of the proposed dual-arm picking operation zoning planning method are as follows:

(1)

Activate the picking robot vision system: Start the apple target recognition and positioning program on the robot control terminal, enable real-time fruit target detection, and determine whether there are apple targets waiting to be picked in the double-arm picking space.

(2)

Partition of apple target to be picked: Based on the x-coordinate value of the centroid position of the apple target obtained by the visual system, if x ≤ 0, the apple is assigned to the left area and picked by the left arm; if x > 0, the apple will be assigned to the right area and picked by the right arm;

(3)

Fruit-picking operation: Based on the partition allocation of the apples to be picked in step (2), both arms perform the picking operation on the fruits they are responsible for picking.

(4)

Determine whether the planned fruit has been harvested: the perception of apple targets is based on the visual system of apple-picking robots and object detection algorithms. Therefore, when the apple target is picked, the object detection algorithm will detect that the apple target at the original position no longer exists, and, therefore, feedback the result to the picking robot terminal, which determines whether the apple has been successfully picked. This process is an automated process. If the planned fruit has not been harvested, continue with the apple-picking operation; if all the planned fruits have been harvested, the visual system will monitor whether there are still fruits within the harvestable range. If there are still fruits, the loop starts from step (2); if there are no fruits to be picked, end the picking operation in the current area.

The process of partitioning the double-arm picking operation is shown in Figure 7.

The schematic diagram of the zoning plan for the dual-arm picking operation is shown in Figure 8. The red dashed line is the boundary between the left and right zones, and the x-coordinate value of this line in the camera coordinate system is 0. If the x-coordinate value of the apple centroid is greater than 0, it is divided into the right robotic arm picking area (marked as ② on the fruit), and if the x-coordinate value ≤ 0, it is divided into the left robotic arm picking area (marked as ① on the fruit).

In order to intuitively analyze the spatial distribution of the left and right arms in the picking operation partition planning method, a schematic diagram of the dual-arm picking three-dimensional space from different perspectives was drawn using the Python programming language, as shown in Figure 9. The green solid ball and black solid ball represent the center points of the left and right robotic arm bases, respectively. The sky blue solid ball represents the spatial position of the D435 depth camera, and the red plane (indicated by a dark blue dashed line in the front and top views) is the equal partition plane with an x-coordinate of 0. The green hemisphere represents the spatial range that the left robotic arm can reach, and the black hemisphere represents the operating range of the right arm. After dividing the double-arm picking space based on the double-arm partitioning method, if the centroid position x coordinate of the apple target is ≤0, it is divided into the left area and picked by the left arm, as shown by the red small sphere in the following figure; if the centroid position x coordinate of the apple target is greater than 0, it will be divided into the right area and picked by the right arm, as shown by the blue small sphere in the following figure. The orange small sphere represents fruits that are not within the range of picking with both arms.

To verify the effectiveness and efficiency of the proposed dual-arm picking partition planning algorithm in dual-arm operation, the partition planning algorithm was implemented using Python programming language and simulated for fruit traversal in dual-arm operation partitions. Before planning the simulation experiment, it was necessary to set the three-dimensional spatial position of the key hardware of the robot in the simulation experiment based on the actual physical spatial relationship between the components in the physical prototype of the dual-arm picking robot (Figure 6), including the center of the left and right picking robotic arm bases, the position of the D435 depth camera, and the positions of the left and right robotic arm end effectors. The key components of the robot have been marked with different colored spheres, and the simulation diagram is shown in Figure 10 below (red spheres represent apples located in the left area, blue spheres represent apples located in the right area, and orange spheres represent fruits outside the range of double-arm picking).

Regarding the determination of the position of the end effectors of the left and right robotic arms, due to the tendency of the robotic arm to collide with obstacles, such as tree branches, during the process of directly moving from the fruit placement position to the apple target position, resulting in picking failure, it is necessary to apply necessary constraints to the posture of the robotic arm picking apples to ensure the success rate of picking. It is necessary to preset the preparatory posture before picking apples, which is the preset transition posture that the robotic arm needs to move to before preparing to pick the next fruit after fruit placement. Set the x and y values of the end spatial position of the robotic arm in this posture to be the same as the x and y values of the base center coordinates of the robotic arm, and set the z value to half the length of the AUBO-i5 robotic arm (985 mm) as 492.5 mm. The simulation of the preparation posture for picking apples with the left and right arms is shown in Figure 11.

According to the actual picking situation, the robotic arm starts the apple-picking operation from the picking preparation position every time, and, after releasing the fruit, it also needs to return to the preparation position before picking the next fruit. Therefore, the picking stroke of the robotic arm can be equivalent to the linear distance between the fruit to be picked and the end effector of the robotic arm in the picking preparation posture. For the left and right robotic arms, the sum of the straight-line distances between the apples in their respective zones and the end effector in their respective picking preparation postures is calculated to obtain the total path length required for the left and right robotic arms to operate.

Based on the determination of the key component positions of the robotic arm and the implementation of a dual-arm picking partition planning method through autonomous programming, four sets of simulation partition planning experiments were conducted to analyze the effectiveness and efficiency of the proposed planning algorithm. Each simulation experiment includes one scenario where the sum of distances from the left area is greater than or equal to the right area, and one scenario where the sum of distances from the right area is greater than the left area. After partitioning the simulated apple targets based on the dual-arm picking partition planning method, the fruits were connected to the end of the corresponding partition robotic arm in a straight line to visually display the distance between them and the end of the robotic arm. The three-dimensional schematic diagrams of the simulation experiments for the zoning planning of four sets of dual-arm picking are shown in Figure 12, Figure 13, Figure 14, and Figure 15, respectively. Among them, the red small sphere represents the apples divided into the left area, the blue small sphere represents the apples divided into the right area, and the orange small sphere represents the fruits outside the range of picking of both arms.

The simulation results of the corresponding dual-arm picking zoning planning are shown in

Table 1. Experimental results of the zoning planning method for double-arm picking operation.

Test Group	Comparison of the Sum of Distances Between Left and Right Zones/Distribution of Fruit Quantity in Left and Right Zones		Total Length of Dual-Arm Operation Path (mm)	Left-Arm Operation Path length (mm)	Right-Arm Running Path Length (mm)	Maximum Proportion of Parallel Tasks for Both Arms (%)	Maximum Single-Arm Operation Length (mm)
1	left zone ≥ right zone	left zone: 13 right zone: 9	12,705	7323	5382	73.5	7323
	left zone < right zone	left zone: 8 right zone: 13	11,851	4716	7135	66.1	7135
2	left zone ≥ right zone	left zone: 10 right zone: 5	8770	5823	2947	50.6	5823
	left zone < right zone	left zone: 8 right zone: 11	10,997	4966	6031	82.3	6031
3	left zone ≥ right zone	left zone: 10 right zone: 6	9191	5654	3537	62.6	5654
	left zone < right zone	left zone: 7 right zone: 10	10,081	4267	5814	73.3	5814
4	left zone ≥ right zone	left zone: 13 right zone: 7	11,894	7322	4572	62.4	7322
	left zone < right zone	left zone: 9 right zone: 11	10,963	4972	5991	83.0	5991

. The distribution of apple quantities in the left and right zones, the length of the left- and right-arm operation paths, the total length of the dual-arm operation paths, the maximum single-arm operation length, and the maximum proportion of parallel operations in both arms are calculated for each experimental group under two different fruit distributions.

Among them, the maximum single-arm operation length index is the larger value of the running path length of the left and right robotic arms, which represents the longest path of single-arm operation. Due to the fact that the picking time of the longest single-arm operation path is the total time required for the current dual-arm apple picking task in the region, it also reflects the overall efficiency of the dual-arm robot operation. The larger this value, the longer the time required for the picking task. The calculation formula for the maximum proportion of parallel tasks in both arms is as follows:

$$Maximum proportion of parallel tasks for both arms={\displaystyle \frac{Minimum single arm operation length}{Maximum single arm operation length}}\times 100\%$$

(1)

where the minimum single-arm operation length is the smaller of the left and right arm running path lengths. For dual-arm picking robots facing the same picking area, the efficiency of parallel operation of dual robotic arms is higher than that of asynchronous operation of dual robotic arms. The maximum proportion of parallel tasks for both arms can represent the proportion of time spent on parallel tasks for the left and right arms of a robot to the total task time, reflecting the efficiency of parallel tasks for the robot. The higher the proportion of parallel tasks, the higher the overall picking efficiency of the robot, that is, the higher the overall utilization rate of the machine.

From Table 1, it can be seen that in the four simulation experiments, the highest value of the maximum parallel operation ratio of the two arms is 83%, and the lowest value is 50.6%. The highest value of the maximum operation length of a single-arm is 7323 mm, and the lowest value is 5654 mm. The total length of the dual-arm running path is 12,705 mm, and the lowest value is 8770 mm. Since the picking partition allocation is based on the x-coordinate of the fruit center of mass, the maximum parallel operation ratio of the two arms depends on the spatial distribution of the current apples, that is, the closer the distance between the left and right apples and the left and right arm ends, the higher the proportion of parallel operation of the two arms. The larger the distance and gap between the left and right zones, the lower the proportion of maximum parallel tasks for both arms.

According to comprehensive analysis, the advantage of the proposed dual-arm picking operation zoning method lies in the “low energy consumption” of picking, that is, each apple to be picked is assigned to the closest mechanical arm to the fruit for picking, and the total stroke of the left and right mechanical arms to complete the apple picking task is the shortest. The overall distance required for dual-arm picking is also the smallest, ensuring the lowest energy consumption for the overall apple picking motion of the robot. On the other hand, from the perspective of picking efficiency, this task partitioning method is not optimal because it does not try to synchronize the two arms in parallel for picking operations as much as possible.

4. Planning Method for the Fruit-Picking Sequence of Dual-Arm Robots

Based on prior knowledge of apple picking, it is generally recommended to first pick fruits that are closer in terms of distance. This prior knowledge is also reasonable for harvesting apples using the AUBO-i5 six-joint robotic arm. Due to the fact that this research is aimed at apple trees under the spindle-shaped cultivation mode in modern standard orchards, if the fruit closest to the end of the robotic arm is not picked first, there may be a phenomenon where the fruit closer to the end of the robotic arm is hit during the picking motion, resulting in apple damage, falling, or damage to the end effector of the robotic arm, leading to picking failure. Therefore, the fruit-picking sequence planning method for dual-arm operation zones follows the principle of “prioritizing picking the fruits closest to the end of the robotic arm in a straight line”. For the fruits in the left and right zones, they are sorted from small to large according to the distance from the apples in the zone to the end of the robotic arm, and the fruit-picking sequence for the left/right zone is determined. The calculation formula for the straight-line distance L_apple between the apple target and the end of the robotic arm is as follows:

$$L_{apple}=\sqrt{{(x_{apple}-x_{end})}^2+{(y_{apple}-y_{end})}^2+{(z_{apple}-z_{end})}^2}$$

(2)

where x_apple, y_apple, and z_apple, respectively, represent the three-dimensional x, y, and z coordinates of the apple target in the base coordinate system of the partitioned robotic arm. x_end, y_end, and z_end, respectively, represent the three-dimensional x, y, and z coordinates of the end effector in the base coordinate system of the corresponding robotic arm within the partition.

To verify the effectiveness of the proposed fruit-picking sequence planning method for dual-arm picking partition in dual-arm operation, a fruit traversal simulation experiment was conducted based on the principle of “prioritizing picking the fruit closest to the end of the robotic arm in a straight line distance” after implementing the dual-arm operation partition planning using Python programming language. Three sets of simulation picking sequence planning experiments were conducted based on randomly generated apple target spatial points. The simulation schematic diagram is shown in Figure 16 (the key components of the robot have been marked with different colored spheres, with red spheres indicating apples located in the left area, blue spheres indicating apples located in the right area, and orange spheres indicating fruits outside the range of double-arm picking, as shown in Figure 10). Among them, the picking sequence numbers of apple targets in different zones are automatically marked above the fruits, and the apples are connected in a straight line according to their picking sequence to visually display the path of the robotic arm picking the fruits.

To verify the safety of the proposed fruit-picking sequence planning method for dual-arm picking partition in dual-arm operation, the x-coordinate values of the ends of both arms were monitored and recorded over time during the movement of the left and right robotic arms in the three simulation experiments mentioned above, in order to analyze whether collisions occurred during the movement of both arms. Due to the fact that the six axis-picking robotic arm needs to perform fruit release operations after picking each apple before picking the next fruit, in the simulation experiment, the dual robotic arm starts running from the apple picking preparation posture. After completing the spatial position traversal of each fruit, the robotic arm will return to the picking preparation posture and then run to the next fruit location. The origin of the coordinate system of the dual robotic arm system is located at the midpoint of the line connecting the centers of the two arm bases, as shown by point O in Figure 17. The positive directions of the X–Y–Z axes are indicated in Figure 17.

The changes in the x-coordinate values of the left and right robotic arm cease over time in the three sets of simulation experiments shown in Figure 18a, 18b, and 18c, respectively. It can be seen that during the movement of the left and right arms, the x-coordinate values at the ends are distributed on both sides of the x = 0 boundary line, and there is no crossing of the x-value change curves at the ends. Therefore, there was no collision between the left and right arms during the movement of the robotic arm, indicating that the proposed fruit-picking sequence planning method for dual-arm picking zones is safe and reliable.

5. Operation Verification Experiment of a Dual-Arm Robot in an Orchard

A dual-arm robot operation experiment based on the proposed picking zoning planning method was conducted at the modern apple orchard in Baishui County, Weinan City, Shaanxi Province (109° E, 35° N, Baishui County, Weinan City, Shaanxi Province) in the spindle-shaped Red Fuji apple tree planting area. The seasonal period in which the experiment was conducted was the mature harvest season of Red Fuji apples, and the harvesting efficiency experiment was conducted in the planting area of low rootstock tightly planted spindle-shaped Red Fuji apple trees in the modern apple orchard of the experimental station. The fruit growth status was good, and all were in a natural growth state.

The general apple-picking operation is carried out under non-rainy environmental conditions; therefore, the orchard experiment was conducted in a weather environment without rain. In order to control a single variable, the fruit on the apple tree was not subjected to fruit stem-branch separation operation, and only the fruits included in the planning range on the fruit tree were subjected to spatial position traversal operation to verify and analyze the operational performance of the dual-arm robot embedded in the picking partition planning algorithm proposed in this paper. The verification test scenario of the dual-arm robot orchard is shown in Figure 19.

Four sets of robot operation efficiency orchard verification experiments were conducted in a standard orchard, and dual-arm robot fruit traversal and access operations were carried out. The order of apple traversal adopted the fruit-picking sequence planning method proposed in this article. On the other hand, in the experiment, the dual robotic arms started running from the apple picking preparation posture. After completing the traversal of the spatial position of a fruit each time, the robotic arms would pivot to the picking preparation posture and then run to the next fruit spatial position. The results of the four orchard experiments are shown in

Table 2. Orchard validation test results of the dual-arm robot operation zoning and picking sequence planning method.

Test Group	Distribution of Fruit Quantity in Left and Right Zones	Total Length of Dual-Arm Traversal Path (mm)	Left-Arm Traversal Path Length (mm)	Right-Arm Traversal Path Length (mm)	Maximum Proportion of Parallel Tasks for Both Arms (%)	Total Traversal Time of Robot (s)
1	left zone: 6 right zone: 8	7518	3156	4362	72.4	22.8
2	left zone: 8 right zone: 4	6511	4321	2190	50.7	22.9
3	left zone: 6 right zone: 10	8604	3155	5449	57.9	26.2
4	left zone: 6 right zone: 9	7139	2737	4402	62.2	24.8

.

In addition, the indicators mentioned in the study, such as “traversal length”, “running path length”, “task length”, and ”traversal path length”, all refer to the path length (mm) traveled by the end effector (picking manipulator) of the AUBO-i5 robotic arm.

From the four sets of orchard experiment data in the table above, it can be seen that the higher the proportion of parallel operations, the higher the overall picking efficiency of the robot, that is, the higher the overall utilization rate of the machine. Among them, the proportion of dual-arm parallel jobs using the partition planning method proposed in this article is the lowest at 50.7% and the highest at 72.4%.

Continuing the analysis based on the data in Table 2, on the other hand, the total traversal time indicator of the robot represents the time required for the robot to traverse all the fruits planned in the current partition, which can also reflect the operational efficiency of the dual-arm robot. From the perspective of this indicator, the shortest total traversal time for robots using the partition planning method proposed in this article is 22.8 s, while the longest is 26.2 s. Analyzing the traversal path length of the left and right robotic arms, the maximum difference in traversal path length between the left and right arms using the partition planning method proposed in this article is 2294 mm, and the minimum is 1206 mm. The maximum total length of the dual-arm operation path is 8604 mm, and the minimum value is 6511 mm. In addition, the distribution difference in fruit quantity between the left and right zones using the partition planning method proposed in this article is 2, while the maximum is 4.

Through comprehensive analysis, it can be concluded that the advantage of using the dual-arm robot operation zoning planning method proposed in this article lies in the “low energy consumption” of picking, that is, for apples in the same scene, the total distance required for the left and right robotic arms to complete the apple traversal task is the shortest, and the overall distance and minimum motion required for the dual-arm traversal are minimized. This minimizes the energy consumption of the entire robot’s movement process.

6. Results and Discussion

Research objective: In order to achieve parallel apple picking with dual robotic arms and improve the operational efficiency of the picking robot, the picking operation planning method of the dual-arm picking robot was studied.

(1)

Proposed method: A dual-arm picking operation partition planning method was proposed, and the proposed dual-arm partition planning method was implemented based on Python autonomous programming, and simulation verification experiments were conducted separately.

Simulation test: The simulation test results show that for the proposed dual-arm robot task partition planning method, the highest value of the maximum parallel task proportion of the dual arms is 83%, and the lowest value is 50.6%. The total length of the dual-arm running path is 12,705 mm, and the lowest value is 8770 mm.

(2)

Proposed method: A fruit-picking sequence planning method based on dual-arm operation was proposed based on prior knowledge of apple picking.

Simulation test: Based on the Vrep robotic arm motion simulation software, the fruit traversal simulation verification of the dual-arm fruit-picking sequence planning method was carried out. Based on the monitoring of the motion pose of the robotic arm, it was proved that there was no conflict between the left and right arms during the motion of the dual robotic arm, indicating that the proposed fruit-picking sequence planning method is safe and reliable.

(3)

Orchard validation experiment: Orchard experiments were conducted at the apple experimental station in Baishui County, Weinan City, Shaanxi Province to evaluate the planning performance of the proposed dual-arm operation zoning method. The experimental results show that the proposed method has the lowest proportion of dual-arm parallel tasks at 50.7% and the highest at 72.4%. The shortest total traversal time of the robot is 22.8 s, and the longest is 26.2 s. The maximum total length of the dual-arm travel path is 8604 mm, and the minimum is 6511 mm. The maximum difference in traversal path length between the left and right arms of the robot is 2294 mm, and the minimum is 1206 mm.

(4)

Optimization directions for future research

Future hardware optimization direction: Although this study has achieved certain results in the zoning and picking sequence planning of dual-arm apple-picking robots, there are still areas that are worth further exploration and optimization. In terms of hardware for robots, efforts can be made in the future to develop robotic arm structures that are lighter, more precise, and have stronger environmental adaptability, in order to enhance the flexibility and stability of robots in complex orchard environments, reduce energy consumption, and extend battery life. At the same time, it is important to optimize the sensor system of the robot, integrate more types and higher precision sensors, such as more advanced visual sensors, tactile sensors, etc., to more accurately perceive the fruit status, surrounding environmental information, and the posture of the robotic arm itself, providing richer data support for picking operations.

Future software optimization direction: In terms of software and algorithms for robots, for the dual-arm picking sequence planning of fruits, more factors can be considered to be introduced, such as differences in actual maturity and difficulty evaluation of picking. A more comprehensive decision model can be constructed to achieve more efficient and scientific picking sequence planning. On the other hand, strengthening the integration and interaction between picking robots and orchard information management systems, achieving real-time uploading and analysis of picking data in order to provide technical support for precise management of orchards, will be important.