Design and Testing of a Four-Arm Multi-Joint Apple Harvesting Robot Based on Singularity Analysis

Abstract

The use of multi-joint arms in a high-spindle environment can solve complex problems, but the singularity problem of the manipulator related to the structure of the serial manipulator is prominent. Therefore, based on the general mathematical model of fruit spatial distribution in high-spindle apple orchards, this study proposes two harvesting system architecture schemes that can meet the constraints of fruit spatial distribution and reduce the singularity of harvesting robot operation, which are four-arm dual-module independent moving scheme (Scheme A) and four-arm single-module parallel moving scheme (Scheme B). Based on the link-joint method, the analytical expression of the singular configuration of the redundant degree of freedom arm group system under the two schemes is obtained. Then, the inverse kinematics solution method of the redundant arm group and the singularity avoidance picking trajectory planning strategy are proposed to realize the judgment and solution of the singular configuration in the complex working environment of the high-spindle. The singularity rate of Scheme A in the simulation environment is 17.098%, and the singularity rate of Scheme B is only 6.74%. In the field experiment, the singularity rate of Scheme A is 26.18%, while the singularity rate of Scheme B is 13.22%. The success rate of Schemes A and B are 80.49% and 72.33%, respectively. Through experimental comparison and analysis, Scheme B is more prominent in solving singular problems but still needs to improve the success rate in future research. This paper can provide a reference for solving the singular problems in the complex working environment of high spindles.

1. Introduction

Global production of apples is expected to increase to 83.1 million tons by 2024 [1]. As the world’s largest producer and consumer of apples, China’s apple planting area and production account for more than 40% of the world’s total and show an increasing trend year by year [2,3]. As a labor-intensive industry, apple harvesting accounts for about 30% of the total production process and 35% of the total production cost, but it is still mainly manual harvesting [4]. With the increasing hollowing out of rural areas and the aging of fruit farmers, the shortage of human resources has become the biggest problem in the production process, and high-speed mechanized operation has become an urgent need [5].

In response to the urgent needs of the apple harvesting industry, countries around the world are accelerating the development of corresponding machinery. At present, harvesting robots have become a research hotspot, and with high speed and high efficiency as the core, it is gradually developing in the direction of large-scale, multi-arm, and unmanned [6]. Among them, the rational use of harvesting robots with multiple manipulators working at the same time can significantly improve the harvesting efficiency of robots [7,8,9,10,11], which has become the main trend in the development of harvesting robots. At the same time, with the increasing demand for apple production year by year and the continuous improvement in agronomic technology, apple planting has changed from traditional unstructured orchard structure to dwarf and dense orchard, such as Tall-Spindle Structure, V-Trellis Structure, Vertical Fruiting Wall [12,13]. Compared with the traditional unstructured apple orchard, the dwarfed apple trees have early fruiting, are suitable for close planting, and have a high yield. At the same time, its canopy is small, and the fruit distribution is relatively concentrated, which provides the possibility for mechanized harvesting operations [14]. However, the actual management and maintenance of dwarf and close planting apple orchards in China cannot meet the agronomic technical standards, and the individual differences in fruit, canopy, and inter-row environment still exist. The complex internal environment of the apple tree canopy and the wide and uneven distribution of fruits lead to many problems in robot harvesting in this environment.

For the spatially distributed fruits and complex canopy, the rational design of the robot’s structural scheme has become the primary prerequisite for high-speed harvesting performance and successful harvesting. In the aspect of mechanical arm selection, a multi-joint arm can realize the dexterous operation similar to a human arm [15,16]. Because of its advantages of precision, flexibility, and good flexibility, it has been applied from the industrial field to the agricultural field and has become a research hotspot of agricultural harvesting robot selection and harvesting technology. Marsela et al. proposed a pepper harvesting robot system for a structured greenhouse environment [17]. This system adopts a 6-DOF flexible multi-joint manipulator, which can effectively solve the influence of branch and leaf occlusion on the accessibility of harvesting robots in complex working environments. Yoshida et al. designed a dual-arm fruit harvesting robot for the joint V-shaped trellis [18]. The 6-DOF manipulator can achieve selective fruit harvesting by bypassing obstacles through motion planning. Aiming at the problem of automatic harvesting of eggplant in an unstructured environment, Delia SepúLveda et al. used a flexible 6-DOF manipulator and proposed a decision strategy of a dual-arm robot system, which improved the success rate of harvesting [19]. In the scheme design of the harvesting system, the existing research puts forward the harvesting requirements of full coverage of fruit growth area. Through the simulation analysis of the spatial size relationship between the robot and the canopy or fruit growth area of fruit trees, the arm size that can meet the requirements of coverage harvesting is obtained [20,21], and the robot configuration scheme and key components that can meet the conditions of full coverage harvesting are optimized [22,23,24,25,26]. Some studies have analyzed the operable space or accessibility of robots [27,28] and used robot kinematics modeling and trajectory planning to maximize the robot harvest rate [29,30,31].

However, the use of multi-joint arms in a high-spindle environment can solve complex problems, but the singularity problem of the manipulator related to the structure of the serial manipulator is prominent. The singular configuration is a special structural configuration inside the mechanism. When the manipulator is in a singular configuration, it will cause the manipulator’s operable degree of freedom to be missing, the flexibility to be reduced, and the speed to change abruptly. As a result, the manipulator system loses its precise control of its velocity matrix, which makes the stability of the multi-joint manipulator worse and even stops acting [32]. Due to the decrease in the degree of freedom and flexibility, there are holes in the operable space of the robot, such that some fruits within the range of the manipulator’s arm can be theoretically reached but cannot be actually reached, which reduces the net rate and the success rate of harvesting.

The size of the robot’s singular workspace is closely related to the robot architecture scheme, which directly affects the success rate and efficiency of the operation. However, most of the existing design schemes are based on the idealization of the working environment. There are few studies on the analysis of the influence of singularity on the working performance of the manipulator in complex environments, and there is insufficient discussion on the coupling relationship between the singularity of the manipulator and the singularity space of the robot. Therefore, indicators such as harvesting success rate and efficiency are more idealized and do not conform to the actual situation in the field [33,34,35].

Therefore, this paper aims to address the problem of the singularity of the multi-joint arm in the high-spindle environment, which leads to the interruption of the operation and the reduction in the robot’s operable space. By generalizing the spatial distribution characteristics of the fruit in the high-spindle apple orchard, two harvesting system architecture schemes that can meet the spatial distribution constraints of the fruit and reduce the singularity of the harvesting robot’s operation are proposed, which are the four-arm dual-module independent moving scheme and the four-arm single-module parallel moving scheme. Then, according to the operation scheme of the harvesting robot, the inverse kinematics solution method of the redundant degree of freedom arm group system and the picking trajectory planning strategy for singularity avoidance is proposed. Finally, the singularity judgment and solution in the complex working environment of the high-spindle are finally realized.

This paper is structured as follows: The singularity problem faced by the harvesting robot in the high-spindle environment is analyzed in Section 2.1. In Section 2.2, two design schemes of harvesting robots under complex constraints of high-spindle are proposed. The singular configuration of the harvesting robot is analyzed, and the singularity avoidance planning strategy is proposed in Section 2.3. In Section 3.1 and Section 3.2, the singularity rate analysis and harvesting success rate analysis of the two design schemes are carried out, respectively. Finally, the research is summarized, and the prospect of future research is presented in Section 4.

2. Materials and Methods

2.1. The Singularity Problem of Harvesting Robot in High-Spindle Environment

2.1.1. Complexity of High-Spindle Cultivation Mode

The high-spindle cultivation mode has the advantages of high yield, good fruit quality, convenient management, and suitable mechanized operation, and it has become the main form of dwarf and dense planting orchard promotion in China. Take the high-spindle apple orchard in Sweet Orchard in Suqian City, Jiangsu Province, as an example. The planting row spacing is 350 cm, the passable width is 160 cm, the tree height is between 350 and 400 cm, the fruit height interval is between 30 and 300 cm, the average crown width of fruit trees is between 100 and 150 cm, and the crown depth can reach 80~110 cm. There is a trapezoidal ridge surface with a height of 20–30 cm under the fruit tree. The overall structural characteristics are shown in Figure 1.

According to the investigation results, the cultivation mode of high-spindle apple trees in Sweet Orchard has tended to be standardized, but the management level is low. The main performance is that there is a gap between the pruning, flower thinning, fruit thinning, and other links of fruit trees and the relevant technical standards, resulting in obvious differences and complexity in different tree specifications and fruit distribution. (

Table 1. The challenges and specific performance of harvesting robots in high-spindle apple orchards.

Challenges	Specific Performance
The fruit distribution is uneven, and the coverage is wide
	Projection of fruit distribution density on the xy plane	Projection of fruit distribution density on the xz plane	Projection of fruit distribution density on the yz plane
The canopy structure is complex
	Canopy specifications were significantly different	Canopy occlusion crisscross	Uneven canopy depth

).

The projection distribution results of fruits on xy, xz, and yz planes in Table 1 showed that the fruits in Sweet Orchard were approximately symmetrically distributed in the depth direction and the width direction with the canopy center as the symmetrical axis, and the distribution intervals were [−40, 40] cm and [−60, 80] cm, respectively. The height direction distribution range can reach 50~300 cm, and the fruit is concentrated between 50 and 200 cm. Therefore, the fruit distribution of a high-spindle apple orchard is characterized by uneven height and wide coverage.

At the same time, due to the limitation of the orchard management level, there are obvious differences in canopy specifications between different trees, and the canopy occlusion between adjacent trees is serious. The crown depth of fruit trees is characterized by a narrow upper and wide lower, and the overall depth is obviously uneven. This large-scale distribution of fruits and highly unstructured operating conditions not only have a greater demand for the robot’s operating space but also pose a challenge to the robot’s adaptability and flexible action in a differentiated environment.

2.1.2. The Singularity Problem Under the Influence of Multiple Factors

The singular configuration is a special configuration of the multi-joint manipulator, which is affected by the structure of the manipulator and the task. When the rotation of some joints is in the limit position, or the mechanism enters the dead point due to the interference of the connecting rod during the movement, the manipulator enters the singular state. The multi-joint arm used in the existing agricultural scene is mostly an industrial cooperative arm. In the actual operation process, the maximum arm span of the manipulator is often greater than the linear distance to the fruit. However, when the picking action is performed in a certain position, the degree of freedom of the joint is lost or even triggers a sudden stop, resulting in unreachable actual operation. At this time, the manipulator is in a singular configuration state, and it usually needs manual intervention to start working again, which seriously affects the picking continuity and operation efficiency.

Affected by the management level of high-spindle apple orchards, the complexity and uncertainty of robot-picking tasks have multiplied. Therefore, the multi-joint arm makes it easy to enter the singular state in the face of the operation task in this environment. The main reasons for the singularity include the following:

(1)

In the high-spindle operating environment, the robot is required to meet the requirements of large-scale and non-uniform distribution of fruits and to adapt to the highly unstructured canopy. Therefore, the robot’s operable space formed by the canopy-body manipulator does not match the environmental constraint space. When the manipulator is located at the edge of the workspace, it will enter the boundary singularity due to the joint reaching the limit state, as shown in Figure 2a; when picking the proximal end, the joints are more likely to pin each other down, thus entering the internal singularity, as shown in Figure 2b;

(2)

In order to ensure successful harvesting, the end-effector needs to approach and pick the fruit in a specific posture. Therefore, when the target fruit is located in the interlaced area of the branches of the fruit tree or the outer area of the large canopy at the bottom, the changeable posture of the end may force some joints of the manipulator to enter the singularity, increasing the risk of singularity in the joint space, thus reducing the effective working space of the robot.

(3)

The existing industrial cooperative arm mostly uses the built-in controller for IK solution and path planning. Therefore, the singularity problem of the manipulator is not paid attention to when performing the harvesting action of the target point, which leads to the singularity of the manipulator during the movement from the initial picking point to the target picking point.

The successful harvesting in complex agricultural scenes does not depend entirely on the manipulator itself. The robot configuration scheme with a multi-joint arm determines the interaction mode between the manipulator and the working environment, which in turn affects the singular spatial distribution of the manipulator. In view of the above trigger reasons, based on the analysis of high-spindle scene constraints, it is important to explore the mapping relationship between robot operation requirements and its own singularity and to propose configuration schemes and motion planning methods that can reduce the singularity of robots.

2.2. Design of Harvesting Robot Under Complex Constraints of High-Spindle

2.2.1. Establishment of Fruit Spatial Distribution Model

The fruit distribution of high-spindle apple orchards is characterized by non-uniformity and significant differences among different trees, which aggravates the complexity of picking tasks. Therefore, the establishment of mathematical models to express the spatial distribution characteristics of high-spindle apple orchards has become the primary prerequisite for clarifying the operating space of harvesting robots.

A random sampling method was used to randomly select 50 5-year-old fruit trees in the orchard. The local base coordinate system $O_{origin}$ was established at the intersection of the apple tree trunk and the ridge surface, and the three-dimensional coordinates of the fruit center point were measured. The x (Depth), y (Width), and z (Height) coordinates of the fruit in the collected data were extracted. After the data were standardized, the distribution test and nonlinear curve fitting were performed, respectively. The results showed that the z coordinate satisfied the normal distribution $N\left({\mu }_z,{\sigma }_z^2\right)$. Here, the mean value of the z coordinate ${\mu }_z$ was 149.7239 cm, the standard deviation ${\sigma }_z$ was 65.61215, and the 95% confidence interval of the mean value was [143.5121, 155.9357] cm (Figure 3a). The x and y coordinates do not satisfy the normal distribution and show multi-peak characteristics, and there are multiple aggregation areas (Figure 3b).

The Gaussian Mixed Model can solve the problem of the data containing multiple different distributions and can be used to establish a mathematical model under the multi-peak distribution characteristics of the x and y coordinates of the high-spindle fruit. From a geometric point of view, GMM is defined as follows [36]:

$$P\left(x\right)={\displaystyle {\sum }_{i=1}^n{\alpha }_i}N\left(x\left|{\mu }_i,{\delta }_i\right.\right)$$

(1)

Here, ${\alpha }_i$ is the weight, and ${\displaystyle {\sum }_{i=1}^n{\alpha }_i}$ = 1, ${\mu }_i,{\delta }_i$ are the mean and variance of the i th Gaussian mixture distribution.

In this paper, the Bayesian information criterion (BIC) is used to determine the optimal number of Gaussian distributions in the data set. Based on the GMM clustering idea, the expectation maximization (EM) algorithm is used to solve the posterior probability distribution of the hidden variable zi corresponding to the x and y coordinates of the fruit after the t-cycle as follows:

$${\gamma }_t{\left(z_i^{\left(n\right)}\right)}_x={\displaystyle \frac{{\alpha }_i^{\left(t\right)}P\left(x\left|{\mu }_i^{\left(t\right)},{\delta }_i^{\left(t\right)}\right.\right)}{{\displaystyle {\sum }_{j=1}^k{\alpha }_j^{\left(t\right)}P\left(x\left|{\mu }_j^{\left(t\right)},{\delta }_j^{\left(t\right)}\right.\right)}}},{\gamma }_t{\left(z_i^{\left(n\right)}\right)}_y={\displaystyle \frac{{\alpha }_i^{\left(t\right)}P\left(y\left|{\mu }_i^{\left(t\right)},{\delta }_i^{\left(t\right)}\right.\right)}{{\displaystyle {\sum }_{j=1}^k{\alpha }_j^{\left(t\right)}P\left(y\left|{\mu }_j^{\left(t\right)},{\delta }_j^{\left(t\right)}\right.\right)}}}$$

(2)

Here, $P\left(x\left|\mu ,\delta \right.\right)$ and $P\left(y\left|\mu ,\delta \right.\right)$ are the normal distribution probability density functions of fruit x and y coordinates, respectively.

The weight ${\alpha }_i^{t+1}$, mean ${\mu }_i^{\left(t+1\right)}$, and covariance ${\displaystyle {\sum }_i^{\left(t+1\right)}}$ corresponding to the x and y coordinates of the fruit in the t + 1th cycle are further solved until convergence. The fitting convergence results are shown in Figure 4. The mean value ${\mu }_x$ of the final x coordinate converges to 1.03765 cm, and the standard deviation ${\delta }_x$ converges to 15.85759 cm. The mean value ${\mu }_y$ of the y coordinate converges to 6.058 cm, and the standard deviation ${\delta }_y$ converges to 24.43194 cm.

Considering the extreme situation of fruit growth in the orchard, according to the 2σ principle in statistics, the growth space U of high-spindle apple is described as follows:

$$U=\left\{\left(x_a,y_a,z_a\right),x_a\in \left({\mu }_x-2{\sigma }_x,{\mu }_x+2{\sigma }_x\right),y_a\in \left({\mu }_y-2{\sigma }_y,{\mu }_y+2{\sigma }_y\right),z_a\in \left({\mu }_z-2{\sigma }_z,{\mu }_z+2{\sigma }_z\right)\right\}$$

(3)

Here, the x, y, and z coordinates of the fruit are 95.44% likely to fall into the above interval.

2.2.2. Four-Arm Synchronous Harvesting System Architecture Scheme

Under the high-spindle cultivation mode, the fruit trees are symmetrically distributed on both sides of the tree row, and the parameters such as plant spacing and row spacing are relatively uniform. Therefore, based on the standardized structural parameters of the high-spindle orchard and the mathematical model of fruit spatial distribution (Section 2.2.1), two kinds of system architecture schemes of four-arm synchronous harvesting are proposed, as shown in Figure 5. It is composed of an autonomous navigation chassis system (1-1, 1-2), a four-arm two-dimensional moving harvesting system (2-1, 2-2, 2-3, 2-4, 2-5), a fruit recovery system (3-1, 3-2) and a control system (4). In Figure 5a, 6-DOF cooperative arms are symmetrically arranged on both sides of the two-level bidirectional moving module. In Figure 5b, four 6-DOF cooperative arms are arranged in pairs on both sides of a horizontal two-way moving module. In both configuration schemes, the harvest of fruits inside the canopy is completed through the horizontal two-way moving module sliding table feed and the cooperative division of labor between the vertical ipsilateral two-arm system.

In terms of theory, the reachability of the robot’s hand-arm depends on the depth distance of the fruit in the canopy and the extension limit of the hand-arm. Therefore, the longer the arm’s length, the easier it is to approach the fruit near the center of the canopy. However, the hand-arm growth will directly increase the probability of collision with the crown element and its own singularity, reducing the flexibility of the manipulator [28]. To this end, we define a composite system composed of a swallowing end effector, a 6-DOF cooperative arm, and a horizontal bidirectional moving module sliding table as an arm group system, as shown in Figure 6. The arm group system can realize the global workspace expansion of the robot through the lateral movement of the horizontal bidirectional moving module to reduce the collision and singular configuration problems caused by the arm length when the installation position of the manipulator is fixed.

2.2.3. Four-Arm Layout and Workspace Allocation

In order to meet the basic needs of the harvesting robot working space to cover the fruit growth space, the robot adopts the straight walking method and the walking–stop–harvesting operation method between the tree rows. With the trunk center as the symmetry axis, the robot walks straight along the tree row while completing the fruit harvesting in the red dotted line area inside the tree row. After the whole row harvesting is completed, the robot turns at the head of the row and starts the next line of operation.

Due to the non-uniform distribution of apples in the height direction under the high-spindle cultivation mode, the four-arm workspace should be divided according to the task volume. For a bilateral four-arm dual-module independent moving scheme (Hereinafter referred to as Scheme A), the operation area of the upper side arm group system is defined as W_u, and the operation area of the lower side arm group system is W_d (Figure 7a). The movement amount of the upper side arm group system in the z-axis direction is h_a1, and the movement amount of the lower side arm group system in the z-axis direction is h_a2. The two operation areas meet the following conditions:

$${\displaystyle {\sum }_{i=1}^n{\rho }_i}={\displaystyle {\sum }_{i=n}^m{\rho }_i}={\displaystyle \frac{1}{2}}{\displaystyle {\sum }_{i=1}^m{\rho }_i}$$

(4)

Here, n is the number of fruits obtained by the visual system in W_u, ${\rho }_i$ is the Euclidean distance from the fruit in the {O_world} coordinate system to the origin of the corresponding arm group system base, m is the number of fruits obtained by the visual system in W.

For the double-sided four-arm single-module parallel moving scheme (Hereinafter referred to as Scheme B), because the upper and lower arms have a parallel relationship in the y-axis direction, W_u and W_d are the common working areas of the upper and lower arm group system. At this time, W_u and W_d are equally divided into sub-operation areas W_u1, W_u2, W_d1, and W_d2 (Figure 7b) according to the principle of the same tasks. At this time, the movement of the arm group system in the z-axis direction is h_a, and the sub-operation area meets the following conditions:

$${\displaystyle {\sum }_{i=1}^{n_1}{\rho }_i}={\displaystyle {\sum }_{i=n_1}^{n_2}{\rho }_i}={\displaystyle {\sum }_{i=n_2}^{n_3}{\rho }_i}{\displaystyle {\sum }_{i=n_3}^m{\rho }_i}={\displaystyle \frac{1}{4}}{\displaystyle {\sum }_{i=1}^m{\rho }_i}$$

(5)

Here, n₁ is the number of fruits obtained by the visual system in W_u1, n₂ is the number of fruits obtained by the visual system in W_u2, and n₃ is the number of fruits obtained by the visual system in W_d1.

The working space and working process of the harvesting robot are shown in Figure 8. Every time the chassis stops, the working space on one side of the harvesting robot is represented by the red dotted line area W_i, and its expression is as follows:

$$W_i=\left\{\left(x_a,y_a,z_a\right),x_a\in \left(0,{\mu }_x+2{\sigma }_x\right),y_a\in \left({\mu }_y-2{\sigma }_y,{\mu }_y+2{\sigma }_y\right),z_a\in \left({\mu }_z-2{\sigma }_z,{\mu }_z+2{\sigma }_z\right)\right\}$$

(6)

The schematic diagram of the working process of the four-arm apple harvesting robot. In order to meet the reachable demand of the target fruit in the W_i area, the working radius of the arm is set to R_m, and the relationship should be satisfied as follows:

$$\left\{\begin{array}{l}{H}_d+H_a+H_c-H_r-R_m\le H_{\min }\\ H_u+H_d+H_a+H_c-H_r+R_m\ge H_{\max }\end{array}\right.$$

(7)

$$\left\{\begin{array}{l}{R}_m\le \sqrt{{\left({\displaystyle \frac{D_l-L}{2}}\right)}^2+{\left(H_d+H_a+H_c-H_r-H_{\min }\right)}^2}\\ R_m\ge {\displaystyle \frac{{\mu }_x+2{\sigma }_x}{2}}\end{array}\right.$$

(8)

Here, H_r is the height of the ridge surface, H_a is the height of the control box, H_c is the height of the chassis, and H_d is the distance between the lower arm group and the chassis. R_m is the maximum working radius of the hand-arm, H_min is the minimum growth height of the fruit in the robot workspace, and H_max is the maximum growth height of the fruit in the robot workspace. D_l is the planting spacing, and L is the maximum installation spacing of the horizontal ipsilateral arm group system.

Combined with the working environment parameters of the high-spindle apple orchard, the selection results, and the above relationship derivation, the key parameters of the four-arm harvesting robot are obtained, as shown in

Table 2. Key parameters of four-arm apple harvesting robot.

Parameters	Value/cm
Hand-arm length Rm	95
Horizontal moving maximum distance L	180
The lowest height of the arm group from the ridge surface Hd	130
The highest height of the arm group from the ridge surface Hu	260
Control box height Ha	30
Chassis height Hc	60
Vertical moving distance of the arm group	[0, 200]
The level moving distance of the arm group	[0, 60]

.

2.3. Singularity Analysis and Non-Singularity Planning of Harvesting Robot

2.3.1. Singular Configuration Description of Harvesting Robot

For the two system architecture schemes proposed in this research, the arm groups are parallel operation relationships; thus, there is no kinematic coupling constraint between the end-effectors. Therefore, the arm group system still belongs to the series manipulator group, and the singularity analysis of the single arm group system can be carried out according to the independent system.

In this paper, a simple method based on link-joint is used to construct the Jacobian matrix of the arm group system to avoid the complex matrix operation of the traditional analytical method [37,38] and judge the singularity condition of the arm group system according to the Jacobian matrix.

Firstly, the forward kinematics model of the arm group system needs to be established. For Scheme A, the arm group system includes two moving pairs and six rotating pairs at the base part, and its degree of freedom is 8. The swallowing end-effector is directly fixed on the end flange of the manipulator; thus, it can be regarded as a connecting rod with a length of d₇. There is no relative rotation relationship between the sixth joint and the end of the manipulator. Aiming at Scheme B proposed in this study, the arm group system includes two moving pairs at the base part and six revolute pairs provided by the upper and lower arms, respectively, with a degree of freedom of 14.

The standard D-H parameter method is used to establish the connecting rod coordinate system of the arm group system of the two schemes, as shown in Figure 9. The corresponding D-H parameters of the 6-DOF cooperative arm are shown in

Table 3. D-H parameters of the 6-DOF cooperative arm.

Link i	θi/°	di/mm	ai/mm	αi/°
1	θ1	d1	0	0
2	θ2	d2	0	−90
3	θ3	0	a3	180
4	θ4	0	a4	180
5	θ5	d5	0	−90
6	θ6	d6	0	90

. Here, θ_i is joint angle, d_i is link offset, a_i is link length, and α_i is link twist.

For Scheme A, in the {O_world} coordinate system, the base of the arm group moves in the Y₀ and Z₀ directions. Assuming that the distance of the base in the Y₀ direction is d_a and the distance in the Z₀ direction is h_a, then the pose transformation matrix between the arm group base and the world coordinate system is as follows:

$${}_0{}^{world}T=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& d_a\\ 0& 0& 1& h_a\\ 0& 0& 0& 1\end{array}\right]$$

(9)

The pose transformation relationship between joint 6 of the 6-DOF manipulator and the base of the arm group can be expressed by the following homogeneous transformation matrix of the adjacent connecting rod:

$${}_i{}^{i-1}T=\left[\begin{array}{cccc}{c}_i& -s_{i}c{\alpha }_i& s_{i}s{\alpha }_i& a_i{c}_i\\ s_i& c_{i}c{\alpha }_i& -c_{i}s{\alpha }_i& a_i{s}_i\\ 0& s{\alpha }_i& c{\alpha }_i& d_i\\ 0& 0& 0& 1\end{array}\right]$$

(10)

Here, $c_i$ is $\cos {\theta }_i$, $s_i$ is $\sin {\theta }_i$, $c{\alpha }_i$ is $\cos {\alpha }_i$, $s{\alpha }_i$ is $\sin {\alpha }_i$.

In the {O_tool} coordinate system, the end-effector offsets d₇ in the Z₆ direction. Thus, the pose transformation matrix of the swallowing end-effector and the sixth joint of the manipulator is as follows:

$${}_{tool}{}^{6}T=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& d_7\\ 0& 0& 0& 1\end{array}\right]$$

(11)

The forward kinematics equation of the arm group system can be expressed as follows:

$${}_{tool}{}^{world}T={}_0{}^{world}T{}_1{}^{0}T{}_2{}^{1}T\cdots {}_{tool}{}^{6}T=\left[\begin{array}{cccc}{n}_x& o_x& a_x& p_x\\ n_y& o_y& a_y& p_y\\ n_z& o_z& a_z& p_z\\ 0& 0& 0& 1\end{array}\right]$$

(12)

Here, ${\left[\begin{array}{ccc}{n}_x& n_y& n_z\end{array}\right]}^T$ is the x-axis unit vector of the {O_tool} coordinate system, ${\left[\begin{array}{ccc}{o}_x& o_y& o_z\end{array}\right]}^T$ is the y-axis unit vector of the {O_tool} coordinate system, ${\left[\begin{array}{ccc}{a}_x& a_y& a_z\end{array}\right]}^T$ is the z-axis unit vector of the {O_tool} coordinate system, ${\left[\begin{array}{ccc}{p}_x& p_y& p_z\end{array}\right]}^T$ is the end position vector.

For the rotating joint i in the arm group system, the Jacobian matrix can be directly constructed by using the direction vector $z_{i-1}$ of each joint axis and the position vector ${\mathit{p}}_{i-1}$ of the connecting rod. The generalized Jacobian matrix under the composite system is composed of the manipulator, and the module is a 6 × 8 matrix. The expression is as follows:

$$\mathit{J}\left({\mathit{\theta }}_i,d_a,h_a\right)=\left[\begin{array}{ccc}{\mathit{J}}_{Pi}& {\mathit{J}}_{d_a}& {\mathit{J}}_{h_a}\\ {\mathit{J}}_{Oi}& 0& 0\end{array}\right]=\left[\begin{array}{ccc}{z}_{i-1}\times \left({\mathit{p}}_e-{\mathit{p}}_{i-1}\right)& {\mathit{J}}_{d_a}& {\mathit{J}}_{h_a}\\ z_{i-1}& 0& 0\end{array}\right]$$

(13)

Here, $\left[\begin{array}{c}{\mathit{J}}_{Pi}\\ {\mathit{J}}_{Oi}\end{array}\right]$ is the Jacobian matrix J(θ_i) of the rotary joint, $\left[\begin{array}{cc}{\mathit{J}}_{d_a}& {\mathit{J}}_{h_a}\\ 0& 0\end{array}\right]$ is the Jacobian matrix J(d_a,h_a) of the base translation pair. ${\mathit{J}}_{Pi}$ and ${\mathit{J}}_{Oi}$ are the linear velocity part and the angular velocity part of the Jacobian matrix J(θ_i) of the rotary joint, respectively, $z_{i-1}$ is the axis direction vector of the joint i−1, ${\mathit{p}}_e$ is the position vector of the end reference point, and ${\mathit{p}}_{i-1}$ is the position vector of the manipulator link i−1.

For the base moving pair in the arm group system, because it only affects the movement in the Y₀ and Z₀ directions in the coordinate system {O_world}, the following is established:

$${\mathit{J}}_{d_a}={\left[\begin{array}{ccc}0& d_a& 0\end{array}\right]}^T,{\mathit{J}}_{h_a}={\left[\begin{array}{ccc}0& 0& h_a\end{array}\right]}^T$$

(14)

For Scheme B, the generalized Jacobian matrix under the composite system of the manipulator and the module is a 6 × 14 matrix, and the block expression is as follows:

$$\mathit{J}\left({\mathit{\theta }}_i,{\mathit{\theta }}_j,d_a,h_a\right)=\left[\begin{array}{cccc}{\mathit{J}}_{Pi}& {\mathit{J}}_{Pj}& {\mathit{J}}_{d_a}& {\mathit{J}}_{h_a}\\ {\mathit{J}}_{Oi}& {\mathit{J}}_{Oj}& {{\mathit{J}}_{d_a}}^{\prime }& {{\mathit{J}}_{h_a}}^{\prime }\end{array}\right]=\left[\begin{array}{cccc}{z}_{i-1}\times \left({\mathit{p}}_e-{\mathit{p}}_{i-1}\right)& z_{j-1}\times \left({{\mathit{p}}_e}^{\prime }-{\mathit{p}}_{j-1}\right)& {\mathit{J}}_{d_a}& {\mathit{J}}_{h_a}\\ z_{i-1}& z_{j-1}& {{\mathit{J}}_{d_a}}^{\prime }& {{\mathit{J}}_{h_a}}^{\prime }\end{array}\right]$$

(15)

Here, $\left[\begin{array}{c}{\mathit{J}}_{Pi}\\ {\mathit{J}}_{Oi}\end{array}\right]$ is the Jacobian matrix J(θ_i) of the upper arm rotary joint, $\left[\begin{array}{c}{\mathit{J}}_{Pj}\\ {\mathit{J}}_{Oj}\end{array}\right]$ is the Jacobian matrix J(θ_i) of the lower arm rotation joint, and $\left[\begin{array}{cc}{\mathit{J}}_{d_a}& {\mathit{J}}_{h_a}\\ {{\mathit{J}}_{d_a}}^{\prime }& {{\mathit{J}}_{h_a}}^{\prime }\end{array}\right]$ is the Jacobian matrix J(d_a,h_a) of the base translation pair.

For the base moving pair of the arm group system in this scheme, it only affects the movement in the Y₀ and Z₀ directions in the {O_world} coordinate system. Therefore, the following is established:

$${{\mathit{J}}_{d_a}}^{\prime }={\left[\begin{array}{ccc}0& d_a& 0\end{array}\right]}^T,{{\mathit{J}}_{h_a}}^{\prime }={\left[\begin{array}{ccc}0& 0& h_a\end{array}\right]}^T$$

(16)

2.3.2. Non-Singular Inverse Kinematics Solution

In the two schemes proposed in this paper, because the arm group system contains redundant degrees of freedom, the IK solution for singular configuration avoidance in the robot workspace becomes a difficult problem [39]. Therefore, the decomposition method is used to describe the redundancy of the degree of freedom of the arm group system, and the IK solution method for the singularity avoidance of the redundant degree of freedom of the arm group system is proposed by considering the singularity constraint.

In Scheme A, the redundant degrees of freedom of the arm group can be decomposed into two translational degrees of freedom and six rotational degrees of freedom (Figure 10a). Since the axes of the adjacent joints 2, 3, and 4 are parallel to each other and satisfy the Pieper criterion, the IK solution task of the desired end pose can be divided into the following two steps:

(1)

Position solution: Firstly, the translation variable h_a is determined according to the operation space allocation result of the harvesting robot. Then, by converting the expected position ${\mathit{P}}_e={[\begin{array}{ccc}{x}_e& y_e& z_e\end{array}]}^T$ of the end-effector to the position of joint 3 of the manipulator, the translation variable d_a and the joint rotation θ₁, θ_2, and θ₃ can be solved.

(2)

Attitude solution: After obtaining the values of θ₁, θ_2, and θ₃, the rotation amount of the last three joints, θ₄, θ_5, and θ₆, is solved by using the rotation matrix ${}_3{}^{0}R$ and the end attitude matrix ${\mathit{R}}_e=\left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right]$.

According to Figure 10a, the position vector of joint 3 in the {O_world} coordinate system is obtained from the expected position P_e of the end-effector as follows:

$${\mathit{P}}_3={\mathit{P}}_e-\left(d_6+d_7\right){\mathit{R}}_e{\left[\begin{array}{ccc}0& 0& 1\end{array}\right]}^T$$

(17)

Let the position of joint 1 in the {O_world} coordinate system be P₁ = ${\left[\begin{array}{ccc}{x}_1& d_a& d_1+h_a\end{array}\right]}^T$, then the position vector of joint 3 pointing to joint 1 in the {O_Base} coordinate system is as follows:

$${}_3{}^1\mathit{v}={\mathit{R}}_z\left({\mathit{P}}_3-{\mathit{P}}_1\right)$$

(18)

Here, ${\mathit{R}}_z$ is the transformation matrix from {O_world} coordinate system to {O_Base} coordinate system.

By projecting ${}_3{}^1\mathit{v}$ to the x₁z₁ plane of joint 1 and eliminating the vector in the y₁ direction, we can obtain the following:

$${\theta }_1=\mathrm{atan}2\left[y_e-r_{23}\left(d_6+d_7\right)-d_a,x_e-r_{13}\left(d_6+d_7\right)-x_1\right]$$

(19)

$${\theta }_2=\mathrm{atan}2\left(s,r\right)-\mathrm{atan}2\left(a_3\sin {\theta }_3,a_3+a_4\cos {\theta }_3\right)$$

(20)

$${\theta }_3=\mathrm{atan}2\left[\pm \sqrt{1-{\left({\displaystyle \frac{r^2+s^2-a_3^2-a_4^2}{2a_3{a}_4}}\right)}^2},{\displaystyle \frac{r^2+s^2-a_3^2-a_4^2}{2a_3{a}_4}}\right]$$

(21)

$$d_a=y_e-r_{23}\left(d_6+d_7\right)-r\sin {\theta }_1$$

(22)

Here, ${}_3{}^6\mathit{R}_{ij}\left(i=1,2,3;j=1,2,3\right)$ is the element of row i and column j in the matrix ${}_3{}^6\mathit{R}$.

When the multiple sets of IK of the target pose do not satisfy the non-singularity condition, that is, to avoid the singular IK solution results of the redundant arm group, the non-singular condition is as follows:

$$\omega =\sqrt{\det \left(\mathit{J}\left({\mathit{\theta }}_i,d_a,h_a\right){\mathit{J}}^T\left({\mathit{\theta }}_i,d_a,h_a\right)\right)}>0$$

(23)

Here, $\omega$ is the manipulability, and $\mathit{J}\left(\mathit{\theta },d_0,h_0\right){\mathit{J}}^T\left(\mathit{\theta },d_0,h_0\right)$ is a 6 × 6 manipulability matrix.

In Scheme B, the redundant degrees of freedom of the arm group can be decomposed into two translational degrees of freedom, six rotational degrees of freedom of the upper arm, and 6-DOF of the lower arm (Figure 10b).

For the expected position ${\mathit{P}}_e={[\begin{array}{ccc}{x}_e& y_e& z_e\end{array}]}^T$ of the end-effector, it is first necessary to determine the operating area according to the y_e. When $y_e>h_a$, it is located within the operating range of the upper arm group system. On the contrary, it is located in the working range of the lower arm group system. The solution method of the arm group translation d_a and the joint rotation θ_i (i = 1, …, 6) is the same as that of Scheme A.

The non-singular condition is as follows:

$$\omega =\sqrt{\det \left(\mathit{J}\left({\mathit{\theta }}_i,{\mathit{\theta }}_j,d_a,h_a\right){\mathit{J}}^T\left({\mathit{\theta }}_i,{\mathit{\theta }}_j,d_a,h_a\right)\right)}>0$$

(24)

Here, $\omega$ is the manipulability, and $\mathit{J}\left({\mathit{\theta }}_i,{\mathit{\theta }}_j,d_a,h_a\right){\mathit{J}}^T\left({\mathit{\theta }}_i,{\mathit{\theta }}_j,d_a,h_a\right)$ is a 6 × 6 manipulability matrix.

2.3.3. Picking Trajectory Planning Strategy for Singularity Avoidance

In Section 2.3.2, the IK problem of the redundant degree-of-freedom arm group system is solved by the decomposition method. The non-singular IK solution can be directly judged and screened by the singular condition to achieve the purpose of singularity avoidance in the joint space. However, avoiding a singular configuration also requires considering the end trajectory. Since the industrial cooperative arm mostly uses the built-in controller to carry out path planning directly, the path planning algorithm only considers the optimal path and does not pay attention to the singularity problem. Therefore, the joint velocity will still change abruptly or fluctuate violently during the movement of the manipulator. At this time, the trajectory of the end effector will enter or close to the singular region.

Aiming at the singularity problem in the harvesting trajectory, this paper proposes a singularity avoidance planning strategy combining Cartesian space planning and joint space planning and combines the built-in controller of the 6-DOF cooperative arm to complete the singularity avoidance control. The planning and control process is shown in Figure 11.

Firstly, the IK solution of the arm group system needs to be calculated according to the target pose ${\mathit{v}}_e=[R_e,P_e]$ corresponding to the fruit coordinates. When the target pose does not have a non-singular IK solution, the point coordinates are eliminated, and a non-singular pose set {v_i} is generated. Then, the picking action of each target point in {v_i} is decomposed into three steps: preview, swallow, and avoidance (Figure 12). The picking path can be divided into the following three path points (v_a, v_b, and v_c):

$${\mathit{v}}_a=\left[\begin{array}{cccc}{R}_{11}& R_{12}& R_{13}& p_{x_1}\\ R_{21}& R_{22}& R_{23}& p_{y_1}\\ R_{31}& R_{32}& R_{33}& p_{z_1}\\ 0& 0& 0& 1\end{array}\right],{\mathit{v}}_b=\left[\begin{array}{cccc}{R}_{11}& R_{12}& R_{13}& p_{x_1}\\ R_{21}& R_{22}& R_{23}& p_{y_1}+\Delta y_1\\ R_{31}& R_{32}& R_{33}& p_{z_1}\\ 0& 0& 0& 1\end{array}\right],{\mathit{v}}_c=\left[\begin{array}{cccc}{R}_{11}& R_{12}& R_{13}& p_{x_1}\\ R_{21}& R_{22}& R_{23}& p_{y_1}+\Delta y_1-\Delta y_2\\ R_{31}& R_{32}& R_{33}& p_{z_1}\\ 0& 0& 0& 1\end{array}\right]$$

(25)

Here, $\Delta y_1$ is the amount of feed, and $\Delta y_2$ is the safety value of avoidance.

In each path point, when the target velocity of the end-effector is x, the joint velocity q can be obtained by using the generalized Jacobian matrix J as follows:

$$\mathit{q}={\mathit{J}}^T{\left(\mathit{J}{\mathit{J}}^T+{\lambda }^{2}I\right)}^{-1}\mathit{x}$$

(26)

Here, λ is the damping coefficient, and I is the unit matrix.

The damping coefficient λ is determined by the non-singular condition (Section 2.3.2) as follows:

$$\lambda =\left\{\begin{array}{cc}0& if\omega =0\\ {\lambda }_0+{\displaystyle \frac{{\lambda }_c}{\omega +1}}& if\omega >0\end{array}\right.$$

(27)

Here, ${\lambda }_0$ is the damping coefficient of foundation, and ${\lambda }_c$ is the strange suppression gain.

At time $t\in [0,T]$, when the starting joint angle of the manipulator is θ₀ and the end joint angle is θ_T, for each adjacent path point, the quintic polynomial is used for interpolation as follows:

$$\theta \left(t\right)=a_0+a_{1}t+a_2{t}^2+a_3{t}^3+a_4{t}^4+a_5{t}^5$$

(28)

The constraint condition is as follows:

$$\left\{\begin{array}{l}{\theta }_0=a_0,{\theta }_T=a_0+a_{1}T+a_2{T}^2+a_3{T}^3+a_4{T}^4+a_5{T}^5\\ {\dot{\theta }}_0=a_1,{\dot{\theta }}_T=a_1+2a_{2}T+3a_3{T}^2+4a_4{T}^3+5a_5{T}^4\\ {\ddot{\theta }}_0=2a_2,{\ddot{\theta }}_T=2a_{2}T+6a_{3}T+12a_4{T}^2+20a_5{T}^3\end{array}\right.$$

(29)

3. Results and Discussion

3.1. Simulation Experiment

In order to verify whether the design scheme proposed in this paper is helpful in solving the singularity problem in the complex environment of high-spindle, the simulation test is carried out in MATLAB R2019 b software.

3.1.1. Singularity Distribution

Based on the description of the singular configuration in Section 2.3.1, the singular space of the arm group under the two design schemes is solved by using the MATLAB Robotics Toolbox. The singular rate is used to represent the proportion of singular space, and the singular rate is defined as follows:

$${\delta }_s={\displaystyle \frac{n_s}{n_c}}$$

Here, n_s is the number of singular points in the operable space, and n_c is the number of sampling points in the operable space.

For Scheme A, the upper side arm group is taken as an example to calculate its singular space. The scheme sets the upper arm group movement constraint f₁(d₁, h₁) with d₁ being 80 cm, h₁ being 120 cm, and the arm group being located at the outermost position of the horizontal two-way moving module. The projection of the singular space S_f1 on the xy, xz, and yz planes is in Figure 13 when the number of sampling points N = 50,000 and the generalized Jacobian matrix $\mathit{J}\left({\mathit{\theta }}_i,d_0,h_0\right)$ of the arm group satisfies the singularity determination condition.

The red spherical region is the singular space of the arm group, and the red scatter is the singular point in the operable space. The coordinates of singular points are extracted, and 2% extreme values are removed. The relative frequency curves of the number of singular points with the change in x, y, and z values are drawn, as shown in Figure 14. The number of singular points is 8549, and the singularity rate is 17.098%.

The change in the relative frequency of the singular point distribution of the arm group with the values of x, y, and z is an approximate even function of the values of x, y, and z. In the cylindrical region composed of x ∈ [−28.2, 36.2] cm, y ∈ [35.7, 129.7] cm, z ∈ [−20.85, 62.85] cm, it is a singular high-incidence area, which is consistent with the trend reflected in Figure 13. For Scheme B, Figure 15a–c is the projection of the singular space of the arm group when the number of sampling points is N = 50,000, d₁ = 80 cm, h₁ = 120 cm. Among them, the red spherical region is the singular space S_up of the upper arm, and the blue spherical region is the singular space S_down of the lower arm. The union of the two is the singular space S_f2 of the right arm group. The total number of singular points is 3370, and the singularity rate is 6.74%.

The coordinates of singular points are extracted, and 2% extreme values are removed. The relative frequency curves of the number of singular points with x, y, and z values are drawn as shown in Figure 16a–c. It can be seen from the above results that the relative frequency of the singular point distribution of the arm group is also an approximate even function of x, y, and z values, and the columnar region composed of x ∈ [−36.2, 19.0] cm, y ∈ [33.7, 127.7] cm, z ∈ [−21.9, 71.1] cm is a singular high-incidence area, which is basically similar to the singular spatial distribution of the arm group in the bilateral Scheme A.

Through comparative analysis, the difference between the working range of the lower arm group of the two schemes is small. However, when the two arm groups carry out picking operations in the same working space, the singularity rate of Scheme B is significantly lower than that of Scheme A. The singular spatial distribution of the two arm groups contains the following characteristics: the z-axis is the cylindrical axis, and the cylindrical area near the base of the arm group is the singular high-incidence area. Therefore, in the two system design schemes proposed in this paper, the bilateral arm groups of the harvesting robot can solve the singularity problem in the y direction through horizontal bidirectional moving.

3.1.2. Dynamic Characteristics

According to the environmental parameters of the orchard and the structural parameters of the harvesting robot, it is assumed that the y-axis of the {O_world} coordinate system coincides with the x-axis of the {O_tree} coordinate system when the harvesting robot is working and the working space of the harvesting robot in the {O_tree} coordinate system is converted to the {O_world} coordinate system, the workspace of four-arm harvesting robot is $W_1=\left\{\left(x_a,y_a,z_a\right),x_a\in \left(-42.81,54.92\right),y_a\in \left(-142.25,175\right),z_a\in \left(-11.5,250.95\right)\right\}$.

In order to verify the effectiveness of the trajectory planning strategy for singularity avoidance, two target poses are selected from W₁. For Schemes A and B, with P₀ = (−25, −18, −49) as the initial position, P₁ = (16, 36, 52) as the target position, and R = (94.35, −2.24, −169.97) as the harvesting pose. The angle change, angular velocity change, and angular acceleration change of each joint of the manipulator during the movement of the end-effector from P₀ to P₁ under the two schemes are recorded as shown in Figure 17 and Figure 18. The kinematic parameters have obvious mutations and perturbations from 5-time steps to 35-time steps.

There is a non-singular optimal IK for the target position P₁ in both schemes. However, during the movement from the initial position P₀ to the target position P₁, the kinematic parameters of the manipulator fluctuate, indicating that the end of the movement passes near the singularity.

According to the method of picking the trajectory planning strategy proposed in Section 2.3.3, the motion parameters in the motion process are optimized by setting ${\lambda }_0$ = 0.05. The results are shown in Figure 19 and Figure 20. After optimization, the changes in angle, angular velocity, and angular acceleration of each joint tend to be smooth when the time steps are 5, 20, and 25, respectively (Figure 19). At the same time, the mutation of angle, angular velocity, and angular acceleration of each joint tend to be gentle and continuous when the time steps are 5, 15, 20, 25, 30, and 35, respectively (Figure 20). This proves that this method can effectively reduce the fluctuation in dynamic parameters when the multi-joint manipulator is close to the singular region.

3.2. Field Experiment

3.2.1. Experiment Condition

In order to verify the actual operation effect of the harvesting robot in the high-spindle apple orchard, the prototype is developed, as shown in Figure 21. In October 2023 and 2024, field performance tests were carried out in Sweet Orchard, Suqian City, Jiangsu Province. The depth camera model used in the four-arm apple harvesting robot is Intel Relsense D435. The mechanical arm adopts the AUBO i5 cooperative arm. The end effector is the second-generation swallowing end-effector developed by the research group. The industrial computer adopts Advantech IPC-610 L, and the processor model is Core i7-9700. The main control system of the whole machine is developed based on the Windows 10 system.

3.2.2. Multi-Round Picking Singularity Rate Test

The harvesting process of the four-arm apple harvesting robot is shown in Figure 22, which mainly includes five steps: (1) Aiming, (2) Approaching, (3) Swallowing, (4) Closing, and (5) Avoiding. When picking the target fruit, we define the ratio of the number of fruits that cause the robot to have a singular configuration to the total number of harvested fruits as the singularity rate when repeating these five steps. The results are shown in

Table 4. Statistical results of singular occurrence during the harvesting process.

Scheme	Number	Quantity of Fruits	Aiming	Approaching	Swallowing	Closing	Avoiding	Singularity Rate (%)
A	1	33	2	2	4	0	1	27.27
	2	35	1	2	3	0	2	22.86
	3	29	2	1	1	0	0	13.79
	4	32	1	1	2	0	3	21.88
	5	39	3	3	4	0	3	33.33
	6	41	2	3	6	0	4	36.59
	7	30	1	1	4	0	2	26.67
	8	32	2	2	3	0	3	31.25
	9	34	1	1	3	0	2	20.59
	10	35	3	3	1	0	1	22.86
Average	-	34	1.8	1.9	3.1	0	2.1	26.18
B	1	32	0	1	1	0	1	9.38
	2	36	1	1	1	0	1	11.11
	3	44	1	2	2	0	2	15.91
	4	35	1	0	3	0	1	14.29
	5	40	2	2	2	0	2	20.00
	6	36	1	1	1	0	1	11.11
	7	33	0	2	1	0	2	15.15
	8	31	1	1	0	0	0	6.45
	9	39	1	1	2	0	1	12.82
	10	37	1	1	2	0	1	13.51
Average	-	36.3	0.9	1.2	1.5	0	1.2	13.22

.

According to the statistical results of Table 4, it can be found that the singularity rate of Scheme A is about 26.18%, while the singularity rate of Scheme B is about 13.22%, which is about 12.96% lower than that of Scheme A. In addition, in the five steps of the harvesting process, both of the two schemes showed that the most singular configurations were generated when the end was swallowed, followed by concession and proximity, and no singularity occurred during the end closure process. This shows that whether the manipulator produces a singular configuration in addition to its own structural influence, its picking steps will also affect the singularity rate.

3.2.3. Multi-Round Picking Success Rate Test

The success rate of harvesting is a comprehensive index to measure the performance of the harvesting robot. In this paper, the success rate of harvesting is defined as the ratio of the number of fruits successfully picked within the scope of robot operation to the number of fruits that should be picked. When the harvesting robot has finished picking, and singularity occurs during the avoiding step (Figure 22), we consider that the fruit has been successfully harvested.

Because the operating conditions have a great influence on the success rate of harvesting, this paper chooses other harvesting robots developed for high-spindle apple orchards as the comparison objects, and there are some differences in canopy conditions and fruit density. The comparison of field test results is shown in

Table 5. Performance comparison of different apple harvesting robots.

Author	Canopy Conditions	Fruit Density	Quantity of Fruits	Success	Success Rate (%)
Jiang et al. [40]	Complex	Middle	57	23	40.35
Zhang et al. [12]	Complex	High	155	101	65.16
Huang et al. [41]	General	Middle	304	234	76.97
Scheme A	General	Middle	328	264	80.49
Scheme B	Complex	High	477	345	72.33

.

By comparison, the success rate of Scheme A is 80.49%, which has the best performance in terms of picking success rate. For Scheme B, the success rate is 72.33%. The reason is that there are large differences in canopy conditions and fruit distribution among different tree rows in the experimental orchard, and the experimental conditions of Scheme B are more complex than those of Scheme A, resulting in a lower actual harvest success rate than Scheme A.

The results of

Table 6. Comparison of the causes of harvesting failure between Scheme A and Scheme B.

Scheme	Quantity of Failures	Position Error (%)	Obstacle (%)	Singularity (%)		Attachment Error (%)
				Elimination	Stop
A	64	10.94	14.06	48.43	21.88	4.69
B	132	15.91	21.21	37.12	20.45	5.31

show the reasons for the failure of harvesting in Scheme A and Scheme B, including position error, obstacle, singularity, and attachment failure. Meanwhile, the harvesting failure caused by singularity includes two cases: the fruit that does not meet the non-singular condition is removed, and the manipulator’s action is stopped. Although affected by the working environment, Scheme B has more failures in positioning errors and obstacles, which also leads to a significant reduction in the success rate of harvesting. However, compared with Scheme A, Scheme B performs better in solving singular problems.

Therefore, in view of the high-spindle environmental constraints and the singularity of the multi-joint arm, the two harvesting robot configuration design schemes proposed in this study have a good overall performance. Among them, Scheme B is more prominent in solving singular problems, but the success rate of harvesting is greatly affected by the operating environment, which needs to be improved in subsequent research.

4. Conclusions

Aiming at the problem of singular configuration caused by the use of multi-joint arms in a high-spindle environment, two harvesting system architecture schemes that can meet the constraints of fruit spatial distribution and reduce the singularity of harvesting robot operation were proposed in this paper. The conclusions are summarized as follows:

(1)

Firstly, in order to make the harvesting robot meet the requirements of complex canopy and non-uniform fruit distribution, a generalized mathematical model was established according to the spatial distribution characteristics of high-spindle orchard fruits, which was used as the design constraint.

(2)

Then, two design constraints-based schemes were proposed. Based on the link-joint method, the singular configuration conditions of the harvesting robot in the two schemes were obtained.

(3)

Finally, according to the harvesting robot operation scheme, the IK solution and trajectory planning strategy for singular avoidance were proposed to realize the singular configuration judgment and solution in the high-spindle orchard.

(4)

The simulation results showed that the singularity rate of Scheme A was 17.098%, while Scheme B was 6.74%. Meanwhile, the picking trajectory planning strategy can effectively solve the problem of velocity fluctuation when the joints pass through the singular region.

(5)

In the field experiment, the singularity rate of Scheme A was 26.18%, while Scheme B was 13.22%. Affected by the complexity of the testing environment, the success rate of Scheme B was 72.33%, which was lower than Scheme A (80.49%). However, Scheme B performs better in solving singular problems, which is crucial in this study. Therefore, Scheme B is better than Scheme A.

In the future, we will focus on how to improve the success rate of Scheme B. The increase in the complexity of the environment leads to the increase in the probability of position error and obstacle of Scheme B; hence, it will be solved by improving the performance of positioning and path planning algorithms under the complex canopy. At the same time, we find that there is a correlation between the singularity configuration of the harvesting robot and the setting of the picking steps. It is necessary to explore the rationality of the picking steps to optimize the singularity avoidance performance of the multi-joint four-arm harvesting robot.