Real-Time Kinematically Synchronous Planning for Cooperative Manipulation of Multi-Arms Robot Using the Self-Organizing Competitive Neural Network

Abstract

This paper presents a real-time kinematically synchronous planning method for the collaborative manipulation of a multi-arms robot with physical coupling based on the self-organizing competitive neural network. This method defines the sub-bases for the configuration of multi-arms to obtain the Jacobian matrix of common degrees of freedom so that the sub-base motion converges along the direction for the total pose error of the end-effectors (EEs). Such a consideration ensures the uniformity of the EE motion before the error converges completely and contributes to the collaborative manipulation of multi-arms. An unsupervised competitive neural network model is raised to adaptively increase the convergence ratio of multi-arms via the online learning of the rules of the inner star. Then, combining with the defined sub-bases, the synchronous planning method is established to achieve the synchronous movement of multi-arms robot rapidly for collaborative manipulation. Theory analysis proves the stability of the multi-arms system via the Lyapunov theory. Various simulations and experiments demonstrate that the proposed kinematically synchronous planning method is feasible and applicable to different symmetric and asymmetric cooperative manipulation tasks for a multi-arms system.

1. Introduction

The development of artificial intelligence technology has facilitated the research on the autonomous manipulation of robot manipulators. Meanwhile, the increase in requirement of using robots to replace human hands’ manipulation has made the cooperative motion of the robot important in the autonomous operations (e.g., manipulating the rudder, using pliers or a wrench, carrying large objects, or other similar manual tasks in daily life) [1]. Various multi-arm robots, such as Baxter [2], YUMI [3], Justin [4], and Robonaut [5], have been proposed to satisfy the requirement because of their outstanding capability of cooperative manipulation in replacing humans.

Multi-arm robots can not only complete the manipulation task of single-arm robot (or robot manipulator) but also accomplish more complex cooperative manipulation task, which is attributed to their large workspace, more degrees of freedom (DOFs), and greater flexibility. The cooperative manipulation of robots has been studied a lot. The methods for the cooperative manipulation could be divided into force-based [6,7,8,9] and kinematics-based strategies [10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29].

The force-based strategies include two types: One relies on dynamics control, but the highly complicated nonlinear dynamic model makes it difficult to apply in robot system. The other one depends on position compensation control that adjusts the position of the end-effector (EE) to maintain a certain interaction force and that has been studied extensively. The hybrid control of force and position [6,7] and the impedance control [8,9] are the typical method using position compensation control. The hybrid control of force and position needs to accurately decompose the coordinate space of the EE into the position space and force space. The impedance control changes the contact force and the EE position according to the impedance value, but the impedance parameters must be adjusted adaptively to ensure the compliance of interaction in the cooperative manipulation. Although the variable impedance is proposed to improve the flexibility of interaction, the real impedance parameters are difficult to obtain. The intelligent control methods are proposed to simplify the control schemes and reduce the difficulty of modeling the robot control system caused by the strong coupling, time-varying, and uncertainty, such as a neural network [10]. In addition, the force-based strategies rely on force/torque sensors and the corresponding control algorithms and are mainly used in the non-redundant robot system [11].

The kinematics-based strategies are simple and easy to combine with an intelligent algorithm to realize robot autonomous manipulation, thereby resulting in the kinematics-based strategies becoming a research focus of the cooperative manipulation. Thus, the kinematics-based cooperative manipulation is mainly considered in this paper. Usually, the higher requirement for the motion synchronization among the arms needs to be satisfied in the kinematics-based cooperative manipulation. The existing kinematics-based strategies for the cooperative manipulation rely on motion planning, including leader–follower [12,13], cooperative-task space (CTS) [14,15,16,17,18,19], task-oriented [20,21], and intelligent approaches [11,22,23].

The leader–follower approach defines the leader arm and the follower arm for the robot system, and the follower carries out motion planning according to the movement of the leader [12,13]. In the CTS approach, robot arms without a leader-follower relationship are shared equally to achieve symmetrical cooperative manipulation tasks by defining the relative and absolute motions. The extended CTS approach was further proposed to accomplish asymmetric behavior and uncoordinated tasks [18,19]. The task-equation-based approach uses the general formula of the defined cooperative task to transform the coordinate system kinematics among arms to plan the motion of single arm [20,21]. The intelligent approach considers the collaborative task of arms as the constrained quadratic programming problems and utilizes the advanced neural network as the solver of the problems to control the arms motion [11,22]. The intelligent approaches could simplify the solving process of inverse kinematics for dual arms, but the adjustment of the EE attitudes is neglected in cooperative tasks. In Ref. [23], the dual-arm path-planning problem was transformed into a multi-objective optimization problem, and a co-evolutionary method with shared workspace was proposed to serve as a solver. A motion planner based on the kinematic model of a dual-arm robot system in [24] was designed to ensure grasping stability and dexterity. The movement under the relative motion frame of EEs was studied in [25] for the problem of two-hand assembly. An asymmetric task-planning method based on the Lyapunov theory was proposed in [26] to solve the problems of designing the control law of absolute motion tasks and updating the distribution of relative tasks among arms. Fractional-order derivative and the uncertain fractional-order differential equations were utilized to predict and correct motion trends, and the rationality of the method is verified by different cases [27]. A state feedback robust controller based on local information was designed to ensure that the states of multiple robots converge to a common motion state [28]. An extended Kalman filter collaborative algorithm based on the error compensation was proposed to reduce the state estimation error of delayed filtering in multi robot systems [29].

The key focus of the kinematics-based strategies planning is ensuring the motion synchronization in collaborative tasks. The motion of one arm is always taken as a reference to plan the motion of other arms in the existing literature. Moreover, the application of the existing approaches is more suitable for the multi-arms robot than those coupled with the common and fixed base. Even if there is an application in the robot with a dynamic coupling base, the dynamic coupling base is set to be stationary to ensure the cooperation between arms [24], and this is caused by the uncoordinated movement between the EEs of redundant arms. Few studies have been devoted to the kinematics-based planning of the redundant multi-arms with a dynamic coupling base to ensure the cooperation between the arms.

Therefore, on this basis of the previous studies [24,30,31], this paper proposed a novel kinematics-based synchronous planning for collaborative manipulation of the redundant multi-arms with dynamic coupling, which involves the inverse kinematics based on the sub-base method and the self-organizing competitive neural network. The following aspects differ from those in the existing literature.

A class of cooperative manipulation tasks of multi-arm robot described by generalized coordinate transformation matrix are summarized, such as carrying, manipulating the rudder, using a wrench, manipulating pliers, multi-station manipulation, and other similar cooperative manipulation. The configuration branch division of the multi-arm robot based on the sub-base method is proposed to identify each branch of the robot, and the inverse kinematics is calculated based on the damped least square method. Then, the multi-arms robot system can synchronously converge along the reducing direction of the total error. The self-organizing competitive neural network is proposed to promote motion synchronization between multiple arms, and it regards the cooperative movement between arms as the competitive relationship of neurons instead of relying on a defined arm motion as a reference in existing research. The inner star learning rule is used to change the neuron weight, and all neuron weight values are updated to adjust the motion of multi-arms in every instance of competitive learning. Thus, the multi-arm robot motion planning method is formed and realizes the synchronization of the arms’ motion state. The stability of the motion-planning algorithm is analyzed by using the Lyapunov theory and the inner star learning rule principle. The feasibility of the proposed method, the synchronization of motion state, and its applicability are demonstrated by dual-arm and three-arm robots with a dynamic base in different symmetric and asymmetric cooperative manipulations.

The remainder of this paper is organized as follows. Section 2 presents a type of cooperative manipulation, the sub-base description, and the Jacobian matrix definition for the multi-arms with physical coupling. Section 3 discusses the real-time kinematically synchronous planning method of collaborative manipulation based on the self-organizing competitive neural network and the stability. Section 4 and Section 5 provide the simulation and experimental results in different cases, respectively. Section 6 presents the conclusion.

2. Cooperative Manipulation of Multi-Arms

2.1. A Type of Kinematically Cooperative Manipulation

In Figure 1, a type of cooperative manipulation for multi-arms is considered for this paper, such as carrying, manipulating rudder, using pliers and multi-station manipulation, etc. Figure 2 presents the common features for the kinematically cooperative manipulations that can be concluded as follows:

(1) The coordinate system {T_i} in or out of the specified object is referenced for the object pose, t_i, of each arm EE in real time. The object pose, t_i, is defined as follows:

$$\{\begin{array}{c}{Q}_i=Q_{{\mathrm{T}}_i}-{}_{t_i}{}^{{\mathrm{T}}_i}R{D}_i \\ R_{t_i}\left({\widehat{f}}_{t_i},{\psi }_{t_i}\right)={}_{{\mathrm{T}}_i}{}^{{\mathrm{O}}_0}R{}_{t_i}{}^{{\mathrm{T}}_i}R\\ t_i={\left(Q_i,{\widehat{f}}_{t_i}·{\psi }_{t_i}\right)}^{\mathrm{T}}\end{array},$$

(1)

where $Q_i$ is the desired position vector of the i-th EE in coordinate system {O₀}. $Q_{{\mathrm{T}}_i}$ is the vector from the coordinate system {O₀} to the i-th coordinate system {T_i}. $D_i$ is the position vector in coordinate system {O₀} for t_i. ${}_{t_i}{}^{{\mathrm{T}}_i}R$ denotes the rotation matrix from the coordinate system for t_i to {T_i}. ${}_{{\mathrm{T}}_i}{}^{{\mathrm{O}}_0}R$ refers to the rotation matrix from {T_i} to {O₀}. $R_{t_i}\left({\widehat{f}}_{t_i},{\psi }_{t_i}\right)$ defines the rotational operator about the axis direction ${\widehat{f}}_{t_i}$ by ${\psi }_{t_i}$ radians. t_i$\in {\mathrm{R}}^{\mathrm{b}\times 1}$. b = 3 for planar robot. b = 6 for spatial robot. i = 1, 2, …, N. N is the number of arms (or EEs).

(2) When the arms perform collaborative operations, the arms form a closed loop, and there is a certain motion constraint relationship between the EEs. Thus, the motion states of EEs from “1, 2, …, i” to “1’, 2’, …, i’” are kinematically consistent and synchronous. The descriptions for movement states are mutual during the execution of these tasks, like the motion error and the motion rate for each EE. Such common motion states of cooperative manipulations are expressed as the following problem in Equation (2), and Equation (2) is used as the judgment criteria for coordinated synchronous motion and is proved in Section 3.2.

$$\{\begin{array}{c}\underset{t\to \infty }{\lim }{e}_i=t_i-s_i=0 \left(\mathrm{a}\right)\\ \underset{t\to \infty }{\lim }\left(\Vert v_i-{\dot{t}}_i\Vert \right)=0 \left(\mathrm{b}\right)\end{array},$$

(2)

where $e_i$ denotes the pose error of the i-th EE, $v_i$ defines the i-th EE velocity, $s_i$ is the pose of the i-th EE, and t is the time. Equation (2a) refers to the pose errors of EEs along the direction of error convergence, which not only can make the multi-arms reach the execution position of the manipulated object at the same time but also can ensure the minimization of the movement error between the arms in the process of cooperative manipulation. Equation (2b) guarantees that the movement speeds of the EEs are synchronous and that the manipulation speed of the EE is equal/close to the set or constraint value in the manipulation. Finally, the synchronization of arms during cooperative manipulation can be achieved.

The i-th EE pose, velocity, and object pose are $s_i$, $v_i$ and $t_i$, respectively. $s_i$, $v_i$, $t_i\in {\mathrm{R}}^{\mathrm{b}\times 1}$. The i-th EE pose error and velocity can be calculated as follows:

$$e_i\left(\mathrm{T}\right)=t_i\left(\mathrm{T}\right)-s_i\left(\mathrm{T}\right) ,$$

(3)

$$v_i\left(\mathrm{T}\right)={\dot{s}}_i=\frac{s_i\left(\mathrm{T}\right)-s_i\left(\mathrm{T}-\Delta \mathrm{T}\right)}{\Delta \mathrm{T}},$$

(4)

where $\mathrm{T}$ signifies the current time, $\left(\mathrm{T}-\Delta \mathrm{T}\right)$ is the last sampling time, $\Delta \mathrm{T}$ denotes the sampling period, $s$ = (s₁, s₂, …, s_k)^T, and t = (t₁, t₂, …, t_k)^T.

2.2. Multi-Arms Robot with Physical Coupling

This paper considers the general configuration of the multi-arm robot with physical coupling to achieve the cooperative manipulation, as shown in Figure 3. Suppose that there are r joints of the multi-arms and each value of θ_j is the joint angle. The completely joint configuration of the multi-arms is defined as Θ = (θ₁, …, θ_r)^T, $\Theta \in R^{r\times 1}$. The pose mapping of the multi-arm robot from joint space to Cartesian space can be expressed as follows:

$$s=f\left(\Theta \right),$$

(5)

and $s_i=f_i\left(\Theta \right)$ for the EE.

The corresponding inverse mapping is as follows:

$$\Theta =f^{-1}\left(s\right).$$

(6)

where f is a highly nonlinear operator and difficult to solve. The iterative method via Jacobian matrix is used to approach the good solution of the mapping problem.

The traditional Jacobian matrix, J, is the partial derivative matrix of the whole chain system relative to the EE $s$. The Jacobian matrix is obtained via linear approximations of inverse kinematic problems. They linearly simulate the motion of the EE with respect to the instantaneous system changes of the link translation and joint angle. The traditional Jacobian matrix, J, is a function of the joint angle, $\Theta$, defined as follows:

$$J{\left(\Theta \right)}_{ij}={\left(\frac{\partial s_i}{\partial {\theta }_j}\right)}_{ij},$$

(7)

where I = 1, …, N. j = 1, …, r. $J{\left(\Theta \right)}_{ij}\in R^{\mathrm{b}\times 1}$.

2.3. Definition of Sub-Bases

The traditional Jacobian matrix can ensure that each EE converges along its own error reduction direction. Unlike the traditional Jacobian matrix, $J$, the sub-bases are defined to make the EEs converge along the reducing direction of the system’s total error and guarantee that the EEs converge at the same time.

This paper defines that the nodes with multiple branches as the sub-bases for the multi-arms configuration, as shown in Figure 3. The Jacobian matrix is modified as follows:

$$J\left(\Theta \right)=\mathrm{diag}\left(J_{1,1},J_{n,1},\dots ,J_{n,k},J_1,\dots ,J_{\mathrm{N}}\right).$$

(8)

For the 1-th sub-base pose, $P_{1,1}$, the corresponding element of the Jacobian matrix is as follows:

$$J_{1,1}{\left(\Theta \right)}_j={\displaystyle \sum }_{i=1}^{\mathrm{N}}{\left(\frac{\partial s_i}{\partial {\theta }_j}\right)}_j,$$

(9)

where $J_{1,1}{\left(\Theta \right)}_j\in R^{\mathrm{b}\times 1}$. $J_{1,1}\left(\Theta \right)\in R^{\mathrm{b}\times {\mathrm{M}}_{0,1}}$. ${\theta }_j$ belongs to the chain $P_0—P_{1,1}$ with M_0,1 DoFs.

For the n,k-th sub-base pose, $P_{n,k}$, the corresponding element of the Jacobian matrix is as follows:

$$J_{n,k}{\left(\Theta \right)}_j={\left[{\displaystyle \sum }_{i=\mathrm{N}-{\mathrm{N}}_k+1}^{\mathrm{N}}\left(\frac{\partial s_i}{\partial {\theta }_j}+\dots +\frac{\partial s_{\mathrm{N}}}{\partial {\theta }_j}\right)\right]}_j,$$

(10)

where $J_{n,k}{\left(\Theta \right)}_j\in R^{\mathrm{b}\times 1}$. $J_{n,k}\left(\Theta \right)\in R^{\mathrm{b}\times {\mathrm{M}}_{n,k}}$. ${\theta }_j$ belongs to the chain $P_{1,1}—P_{n,k}$ with M_n,k DoFs. By analogy, the Jacobian matrix corresponding to other sub-bases can be obtained.

For $J_1,\dots ,J_N$ without common degrees of freedom, the corresponding elements can be obtained according to (6).

$$\mathrm{N}={\mathrm{N}}_1+{\mathrm{N}}_2+\dots +{\mathrm{N}}_k,$$

(11)

$$r={\mathrm{M}}_{0,1}+{\mathrm{M}}_{n,1}+\dots +{\mathrm{M}}_{n,k}+{\mathrm{M}}_1+\dots +{\mathrm{M}}_N.$$

(12)

The velocities of 1-th sub-base and n,k-th sub-base are calculated by the following:

$${\dot{P}}_{1,1}={\eta }_1·\frac{1}{\mathrm{N}}·{\displaystyle \sum }_{i=1}^{\mathrm{N}}{k}_p·\frac{d\left(t_i-s_i\right)}{dt},$$

(13)

$${\dot{P}}_{n,k}={\eta }_{n,k}·\frac{1}{{\mathrm{N}}_k}·{\displaystyle \sum }_{i=\mathrm{N}-{\mathrm{N}}_k+1}^{\mathrm{N}}{k}_p·\frac{d\left(t_i-s_i\right)}{dt},$$

(14)

where ${\eta }_1$ and ${\eta }_{n,k}$ are the gain coefficient. N and N_k are fixed value and related to the configuration of multi-arms.

The inverse kinematics is as follows:

$$\dot{\Theta }=J^{*}\left(\Theta \right)\dot{S}=J^{*}\left(\Theta \right){\left({\dot{P}}_{1,1} {\dot{P}}_{n,1} \cdots {\dot{P}}_{n,k} \dot{s}\right)}^{\mathrm{T}},$$

(15)

where J^* denotes the pseudo-inverse of Jacobian matrix, J(Θ), based on the damped least squares method, and J^*= J^T(JJ^T + λI)⁻¹. $J\left(\Theta \right)\in R^{b\left(k+N+1\right)\times r}$. $J^{*}\in R^{r\times \mathrm{b}\left(k+\mathrm{N}+1\right)}$. λ (λ > 0) represents the damping factor that can handle the ill-conditioned J in the neighborhood of singular configurations for redundant manipulators and guarantee the EEs with the minimum possible deviation at all configurations. I is a unit matrix with the dimension $\mathrm{b}(\mathrm{k}+\mathrm{N}+1) \times \mathrm{b}(\mathrm{k}+\mathrm{N}+1)$. In accordance with the traditional fixed proportion-based method [31] for the real-time tracking of a given object pose, $\dot{t}$, the EE velocities are planned as follows:

$$\dot{s}=\dot{t}+k_p·\left(t-s\right),$$

(16)

where $k_p$ is the gain coefficient.

2.4. Iteration for Multi-Arms Robot Motion

The iterative method is utilized to achieve the real-time movement of multi-arms via updating the joint angles, Θ, according to (17).

$$\Theta \left(\mathrm{T}\right)=\Theta \left(\mathrm{T}-\Delta \mathrm{T}\right)+\Delta \Theta .$$

(17)

where $\Delta \Theta$ deduced from (15) becomes

$$\begin{gathered} \Delta \Theta \approx J^{*}\left(\Theta \right)\Delta S=J^{*}\left(\Theta \right){\left(\Delta P_{1,1} \Delta P_{n,1} \cdots \Delta P_{n,k} \Delta s\right)}^{\mathrm{T}} \\ =J^{*}\left(\Theta \right)\left[\begin{array}{c}{\eta }_1·\frac{1}{\mathrm{N}}·{\displaystyle {\displaystyle \sum }_{i=1}^{\mathrm{N}}}\mu ·\left(t_i-s_i\right)\\ {\eta }_{n,1}·\frac{1}{{\mathrm{N}}_1}·{\displaystyle {\displaystyle \sum }_{i=1}^{{\mathsf{N}}_1}}\mu ·\left(t_i-s_i\right)\\ ⋮\\ {\eta }_{n,k}·\frac{1}{{\mathrm{N}}_k}·{\displaystyle {\displaystyle \sum }_{i=\mathrm{N}-{\mathrm{N}}_k+1}^{\mathrm{N}}}\mu ·\left(t_i-s_i\right)\\ \Delta t+\mu ·\left(t-s\right)\end{array}\right], \end{gathered}$$

(18)

where μ = k_p·ΔT, and μ < 1. Moreover, $\Delta t$ represents the changing pose of the object at a sampling time interval, ΔT.

Then, according to the sub-base method, the movement of the multi-arm robot can be achieved. The sub-base motion facilitates the synchronous convergence, and the synchronous performance is more obvious when the common DoFs are enough. The corresponding verifications are presented in Section 4 and Section 5.2.

3. Kinematically Synchronous Planning

3.1. Synchronous Planning Using Self-Organizing Competitive Neural Network

The DoFs of the sub-base are not always enough to ensure the system convergence along the reducing error direction for the pose of the EEs, thereby resulting in the asynchronous EE motion. Thus, the self-organizing competitive neural network based on the rule of inner star model is proposed to adjust the synchronism of multi-arms movement. Equation (2) indicates that the error $e_i$ and the $v_i$ will tend to a stable value. In accordance with the principle of the self-organizing competitive neural network, the kinematically synchronous planning for the multi-arm robot is designed as shown in Figure 4.

The learning rule of the inner star model defines the weight updating as follows:

$$\Delta w_i=\eta \left(P_i-w_i\right)Y_i,$$

(19)

where $\eta$ denotes the learning rate; $P_i$ is the i-th input element of the neuron and the minimum pose velocity error norm, min($\Vert v_i\left(\mathrm{T}\right)-{\dot{t}}_i\Vert$); $w_i$ is the weight value; i = 1, 2, …, N; N is the number of arms (or EE); and $Y_i$ is the value of output neuron and is defined as

$$Y_i=\{\begin{array}{c}1, \mathrm{if} P_i^{\prime }{P}_i>\epsilon \\ 0, \mathrm{Otherwise} \end{array},$$

(20)

where

$$\epsilon =\frac{PP}{\mathrm{N}}.$$

(21)

The input element, $P_i$, and the weight value, $w_i$, are defined as

$$P_i=\Vert \tilde{v}\Vert =\min \left(\Vert v_1\left(\mathrm{T}\right)-{\dot{t}}_1\Vert ,\Vert v_2\left(\mathrm{T}\right)-{\dot{t}}_2\Vert ,\dots ,\Vert v_{\mathrm{N}}\left(\mathrm{T}\right)-{\dot{t}}_{\mathrm{N}}\Vert \right),$$

(22)

$$w_i=\Vert v_i\left(\mathrm{T}\right)-{\dot{t}}_i\Vert .$$

(23)

The input vector $P$ is constituted by $P_i$ and defined as

$$P={\left(P_1, P_2,\dots ,P_{\mathrm{N}}\right)}^{\mathrm{T}}, P\in R^{\mathrm{N}\times 1}.$$

(24)

Since the minimum of $\Vert \tilde{v}\Vert$ is used as an input and each EE may become the one with the minimum of $\Vert \tilde{v}\Vert$, the proposed method makes the planner no longer use a manipulator as a reference as in the existing literature. In each period, the new input vectors, $P^{\prime }$, in neural networks are defined as

$$P^{\prime }={\left(P_1^{\prime },P_2^{\prime },\dots ,P_{\mathrm{N}}^{\prime }\right)}^{\mathrm{T}}.$$

(25)

where

$$P_i^{\prime }=\Vert v_i\left(\mathrm{T}\right)-{\dot{t}}_i\Vert .$$

(26)

In order to make the system quickly reach the state of synchronization, the winning weight value will get more rewards in a cycle. The new weight value, $\Delta {\widehat{w}}_i$, is designed as

$$\Delta {\widehat{w}}_i=\Delta w_i·\tanh \left(\frac{c_i\Vert v_i\left(\mathrm{T}\right)-{\dot{t}}_i\Vert }{{{\displaystyle \sum }}_{i=1}^N\Vert v_i\left(\mathrm{T}\right)-{\dot{t}}_i\Vert +{\sigma }_i}\right),$$

(27)

where ${\sigma }_i$ is a small positive real number; and $c_i$ denotes a positive real number to adjust the updating slope of neuron weight and contributes to the synchronization of the EE motion. The response time of converges is shorter when the parameter $c_i$ becomes larger.

The i-th EE’s planned related velocity is deduced according to the iterative method as follows:

$${\widehat{v}}_i\left(\mathrm{T}\right)=\mu ·\left(t_i-s_i\right)·\left[1-\frac{\Delta {\widehat{w}}_i}{\Vert v_i\left(\mathrm{T}\right)-{\dot{t}}_i\Vert +{\delta }_i}\right].$$

(28)

Then, the planned velocity for EEs is

$${\overline{v}}_i\left(\mathrm{T}\right)={\widehat{v}}_i\left(\mathrm{T}\right)+\Delta t_i.$$

(29)

Thus, (18) becomes

$$\Delta \Theta \approx J^{*}\left(\Theta \right)\Delta S=J^{*}\left(\Theta \right)\left[\begin{array}{c}{\eta }_1·\frac{1}{\mathrm{N}}·{\displaystyle {\displaystyle \sum }_{i=1}^{\mathrm{N}}}{\widehat{v}}_i\left(\mathrm{T}\right)\\ {\eta }_{n,1}·\frac{1}{{\mathrm{N}}_1}·{\displaystyle {\displaystyle \sum }_{i=1}^{{\mathrm{N}}_1}}{\widehat{v}}_i\left(\mathrm{T}\right)\\ ⋮\\ {\eta }_{n,k}·\frac{1}{{\mathrm{N}}_k}·{\displaystyle {\displaystyle \sum }_{i=\mathrm{N}-{\mathrm{N}}_k+1}^{\mathrm{N}}}{\widehat{v}}_i\left(\mathrm{T}\right)\\ \overline{v}\left(\mathrm{T}\right)\end{array}\right],$$

(30)

where $\overline{v}\left(\mathrm{T}\right)={\left[{\overline{v}}_1\left(\mathrm{T}\right),{\overline{v}}_2\left(\mathrm{T}\right),\dots ,{\overline{v}}_{\mathrm{N}}\left(\mathrm{T}\right)\right]}^{\mathrm{T}}$. The learning rate η, η₁ and η_n,k are related to the step size for each iteration. The response time of converges is shorter when the learning rate becomes larger.

3.2. Stability Analysis

Supposing that the learning rate, i.e., $\eta$, ${\eta }_1$, ${\eta }_{n,1}$, …, ${\eta }_{n,k}$, is small, based on the rule of inner star model, the weight value with the minimum EE pose error, $e_{\min }$, is never updated, and the minimum pose error changes in a cycle is $\Delta e_{\min }\left(\mathrm{T}\right)$. Thus, for the EE with the minimum pose error, the Lyapunov function is defined as follows:

$$V\left(\mathrm{T}\right)=\frac{1}{2}{e}_{\min }^2\left(\mathrm{T}\right),$$

(31)

$$V\left(\mathrm{T}+\Delta \mathrm{T}\right)=\frac{1}{2}{e}_{\min }^2\left(\mathrm{T}+\Delta \mathrm{T}\right),$$

(32)

The change of the pose error is

$$e_{\min }\left(\mathrm{T}+\Delta \mathrm{T}\right)=e_{\min }\left(\mathrm{T}\right)-\Delta e_{\min }\left(\mathrm{T}\right),$$

(33)

For the EE with the minimum pose error, $e_{\min }\left(\mathrm{T}\right)$, the planned iterative step based on the fixed proportion method [31] is $\mu e_{\min }\left(\mathrm{T}\right)$ in each control cycle, as shown in Figure 5. However, the actual motion of EE will have a small deviation, $\phi$, as follows:

$$\Delta e_{\min }\left(\mathrm{T}\right)=\mu e_{\min }\left(\mathrm{T}\right)+\mathsf{\phi },$$

(34)

where 0 < μ < 1.

Due to the high-precision motion of the advanced manipulator and the small learning rate in the proposed method, the deviation, $\mathsf{\phi }$, is usually in a very small range. The norm of the deviation $\Vert \mathsf{\phi }\Vert$ is less than $\Vert \mu e_{\min }\left(\mathrm{T}\right)\Vert$ to ensure that the EE can move along the planned path, i.e., $\Vert \mathsf{\phi }\Vert <\Vert \mu e_{\min }\left(\mathrm{T}\right)\Vert$. Therefore,

$$\begin{gathered} \Delta V\left(\mathrm{T}\right)=\frac{1}{2}{e}_{\min }^2\left(\mathrm{T}+\Delta \mathrm{T}\right)-\frac{1}{2}{e}_{\min }^2\left(\mathrm{T}\right) \\ =\frac{1}{2}\left\{{\left[e_{\min }\left(\mathrm{T}\right)-\mu e_{\min }\left(\mathrm{T}\right)-\mathsf{\phi }\right]}^2-e_{\min }^2\left(\mathrm{T}\right)\right\} \\ =\frac{1}{2}\left\{{\left[\left(1-\mu \right)e_{min}\left(T\right)-\mathsf{\phi }\right]}^2-e_{min}^2\left(T\right)\right\} \\ \le \frac{1}{2}\left\{{\left[\left(1-\mu \right)e_{min}\left(T\right)+\Vert \mathsf{\phi }\Vert \right]}^2-e_{min}^2\left(T\right)\right\} \\ <\frac{1}{2}\left\{{\left[\left(1-\mu \right)e_{min}\left(T\right)+\mu e_{min}\left(T\right)\right]}^2-e_{min}^2\left(T\right)\right\} \\ =\frac{1}{2}\left\{{\left[\left(1-\mu \right)e_{min}\left(T\right)+\mu e_{min}\left(T\right)\right]}^2-e_{min}{\left(T\right)}^2\right\} \\ =0 \end{gathered}$$

(35)

Thus,

$$\underset{\mathrm{T}\to \infty }{\lim }{e}_{\min }\left(\mathrm{T}\right)=0,$$

(36)

and the minimum pose error, $e_{\min }$, is convergent.

Based on Equations (3) and (4), the derivation of ${\dot{e}}_{\min }\left(\mathrm{T}\right)$ is

$$\begin{gathered} {\dot{e}}_{\min }\left(\mathrm{T}\right)={\dot{t}}_{\min }\left(\mathrm{T}\right)-v_{\min }\left(\mathrm{T}\right)={\dot{t}}_{\min }\left(\mathrm{T}\right)-{\dot{s}}_{\min }\left(\mathrm{T}\right) \\ =\frac{t_{\min }\left(\mathrm{T}\right)-t_{\min }\left(\mathrm{T}-\Delta \mathrm{T}\right)}{\Delta \mathrm{T}}-\frac{s_{\min }\left(\mathrm{T}\right)-s_{\min }\left(\mathrm{T}-\Delta \mathrm{T}\right)}{\Delta \mathrm{T}} \\ =\frac{t_{\min }\left(\mathrm{T}\right)-s_{\min }\left(\mathrm{T}\right)}{\Delta \mathrm{T}}-\frac{t_{\min }\left(\mathrm{T}-\Delta \mathrm{T}\right)-s_{\min }\left(\mathrm{T}-\Delta \mathrm{T}\right)}{\Delta \mathrm{T}} \\ =\frac{1}{\Delta \mathrm{T}}\left[e_{\min }\left(\mathrm{T}\right)-e_{\min }\left(\mathrm{T}-\Delta \mathrm{T}\right)\right] \\ =\frac{1}{\Delta \mathrm{T}}\left[e_{\min }\left(\mathrm{T}-\Delta \mathrm{T}\right)\right] \\ =\frac{1}{\Delta \mathrm{T}}\left[\mu e_{\min }\left(\mathrm{T}-\Delta \mathrm{T}\right)+\mathsf{\phi }\right] \end{gathered}$$

Since $\Vert \phi \Vert <\Vert \mu e_{\min }\left(\mathrm{T}\right)\Vert$ and $\underset{\mathrm{T}\to \infty }{\lim }{e}_{\min }\left(\mathrm{T}\right)=0$,

$$\underset{\mathrm{T}\to \infty }{\lim }{\dot{e}}_{\min }\left(\mathrm{T}\right)=0,$$

(37)

According to the rule of inner star model, the input and the weight ultimately become equal, as follows:

$$P_i=w_i.$$

(38)

This is

$$\Vert \tilde{v}\Vert =\Vert {\dot{e}}_{\min }\left(\mathrm{T}\right)\Vert =\Vert v_i\left(\mathrm{T}\right)-{\dot{t}}_i\Vert =0,$$

(39)

$$\Delta {\widehat{w}}_i=\Delta w_i=0.$$

(40)

Then,

$$\underset{\mathrm{T}\to \infty }{\lim }\Vert e_i\left(\mathrm{T}\right)\Vert =\underset{\mathrm{T}\to \infty }{\lim }\Vert e_{\min }\left(\mathrm{T}\right)\Vert =0.$$

(41)

$$\underset{\mathrm{T}\to \infty }{\lim }\Vert {\dot{e}}_i\left(\mathrm{T}\right)\Vert =\underset{\mathrm{T}\to \infty }{\lim }\Vert {\dot{e}}_{\min }\left(\mathrm{T}\right)\Vert =0.$$

(42)

Therefore, the proposed planning method based on the self-organizing competitive neural network is convergent, and Equation (2) is proved to be valid to achieve motion synchronization.

4. Simulation

The three-arm robot with 15-DoFs is utilized to provide the contrast simulations for inverse kinematics by using inverse kinematics based on the sub-bases and the traditional method [31], as shown in Figure 6. The common DoFs are 3 that are enough for all EEs to make the pose error converge along the decreasing direction of the total error simultaneously. The synchronization performances of EE movements are compared by tracking the static-designated location on the specified-carried object. The robot configuration parameters, initial parameters, and kinematics parameters are presented in

Table 1. Parameters of three-arm robot.

li	1	2	3	4	5
Length (m)	1.18	0.88	0.88	0.88	0.88
li	6	7	8	9	10
Length (m)	0.88	57.85	0.88	0.88	0.88
li	11	12	13	14	15
Length (m)	57.85	0.88	0.88	0.88	57.85

,

Table 2. Initial joint parameters of three-arm robot.

${\theta }_i$	1	2	3	4	5
Initial angle (°)	−5.0	5.0	5.0	30.0	20.0
${\theta }_i$	6	7	8	9	10
Initial angle (°)	20.0	20.0	−70.0	20.0	20.0
${\theta }_i$	11	12	13	14	15
Initial angle (°)	20.0	10.0	10.0	20.0	20.0

and

Table 3. Kinematic parameters.

Parameters	μ	${\eta }_{1,1}$	N	b	$k$	R
Value	0.08	1/6	3	3	1	15
Parameters	M0,1	M1	M2	M3	λ	$\Delta \mathrm{T}$
Value	3	4	4	4	0.01	0.05

. The poses of the static object are t₁ = (2.49 m, 2.79 m, 1.536 rad)^T, t₂ = (5.0 m, −0.70 m, 1.536 rad)^T, and t₃ = (4.49 m, 1.50 m, 1.536 rad)^T. Moreover, t = (t₁, t₂, t₃)^T, and $\dot{t}$ = 0. The simulation results are presented in Figure 7 and Figure 8.

Both methods can make the EEs arrive at the designated location of the specified object, as illustrated in Figure 7a and Figure 8a. The joint angles are shown in Figure 7b and Figure 8b. Due to $\dot{t}=0$, $\Delta e_i\left(\mathrm{T}\right)$ is equal to $v_i\left(\mathrm{T}\right)\Delta \mathrm{T}$, and the velocity curves of the EE pose are similar with those of the EE pose error, as shown in Figure 7c–f and Figure 8c–f. Because of the sub-base motion, the convergence of the EE pose error is much faster than that based on the traditional method. Furthermore, the EE pose velocities form uniform motion states before complete convergence in Figure 8c,d. Thus, the proposed sub-base method is conducive to the synchronization from the initial motion state to the cooperative motion state and improves the efficiency for the carrying of the multi-arm robot.

5. Experimental Verification

5.1. Experimental Setup

The two-arm robot with 13-DoFs is used to compare the proposed synchronous planning for the collaborative manipulation, and 1-DoF is common in the base joint, as shown in Figure 9. The D-H parameters of the two-arm robot are presented in

Table 4. D-H parameters of two-arm robot *.

	i	${\mathit{d}}_{\mathit{i}}$	${\mathit{a}}_{\mathit{i}}$ (mm)	${\mathit{\alpha }}_{\mathit{i}}$ (rad)	${\mathit{\beta }}_{\mathit{i}}$		i	${\mathit{d}}_{\mathit{i}}$	${\mathit{a}}_{\mathit{i}}$	${\mathit{\alpha }}_{\mathit{i}}$ (mm)	${\mathit{\beta }}_{\mathit{i}}$ (rad)
Left Arm	1	0	$h_1$ = 42	0	0	Right Arm	1	0	$h_1$ = 42	π	0
	2	0	h2 = 84	π/2	0		2	0	$h_2$ = 84	π/2	0
	3	0	$h_3$ = 84	−π/2	0		3	0	$h_3$ = 84	−π/2	0
	4	0	$h_4$ = 84	π/2	0		4	0	$h_4$ = 84	π/2	0
	5	0	$h_5$ = 78	−π/2	0		5	0	$h_5$ = 78	−π/2	0
	6	0	h7 = 71	π/2	0		6	0	$h_7$ = 71	π/2	0
	7	0	$h_7$ = 71	0	0		7	0	$h_7$ = 71	0	0

* The base (i.e., ${\theta }_1$) is the common joint of two arms.

. In Figure 10, the principle of the experimental setup can be briefly described as follows.

(1)

The global depth camera transfers the observed frame of depth and color images to the computer by using a universal serial bus (USB) in real time. The robot operating system (ROS) node runs in the computer and extracts the position information of the object from each frame image. The vision-processing procedures in the computer are developed based on the morphology, using the Open Source Computer Vision Library and the camera Software Development Kit.

(2)

The joint feedback data, Θ, of the robot are transmitted to the TMS320F28335 controller through the Controller Area Network (CAN) bus. At the same time, the controller transfers the received joint data, Θ, to the ROS nodes through the Serial Communication Interface (SCI) bus in real time.

(3)

The ROS nodes receive the data (x_obj, Θ) and execute the forward and inverse kinematics calculation and motion-planning algorithm. The real-time control command data, Θ_d and ${\dot{\Theta }}_d$, are sent to the robot joint actuator through the RS485 bus to control the motion. The baud rate of the CAN bus, USB serial bus, SCI bus, and RS485 bus is set to 1 MHz. The control period of the whole system containing the proposed kinematically synchronous planning method is less than 10 ms.

5.2. Synchronous Planning Experiments

(1)

Collaboration Carrying

The collaboration carrying an object was provided to verify the feasibility of the proposed planning method, as illustrated in Figure 11. The parameters for carrying are shown in

Table 5. Kinematic parameters of two-arm robot.

Parameters	μ	${\eta }_{1,1}$	N	b	$k$	r
Value	0.005	5	2	2	1	13
Parameters	M0,1	M1	M2	λ	$\Delta \mathrm{T}$	—
Value	1	6	6	0.01	0.05	—

and

Table 6. The parameters of self-organizing competitive neural network.

Parameters	${\delta }_i$	${\sigma }_i$	$c_i$	$\eta$
Value	1 × 10−5	1 × 10−5	2.0	0.03

, respectively. The carried object has translational and rotational motion. The rotation center position is (−1.5 mm, −342.7 mm, −0.85 mm)^T. The EEs of left and right arms reach the initial locations, as shown in Figure 11a. The tracked initial locations on the carried object are $t_1^{\mathrm{init}}$ = (115.841, −342.7, −0.85, 1.2092, −1.2092, −1.2092)^T, $t_2^{\mathrm{init}}$ = (−118.841, −342.7, −0.85, 1.2092, −1.2092, −1.2092)^T. At 6.75 s, two arms begin to carry the object, as shown in Figure 11b. The rotation angle relative to the rotation center is 0.0005 rad, and the translational motion of the rotation center is 0.15 mm in a cycle period, $\mathsf{\Delta }\mathrm{T}$. During the collaboration process, the position of the tracked locations on the carried object moves back and forth in a 5 s cycle, and the tracked pose trajectory in the initial 2.25 s (from 6.75 s to 9 s) is as follows:

$$t_1=t_1^{\mathrm{init}}-\left(\begin{array}{c}117.341·\left[\sin \left(0.0005\left(\mathrm{T}-6.75\right)+0.0005\Delta \mathrm{T}\right)-\sin \left(0.0005\left(\mathrm{T}-6.75\right)\right)\right]+0.15\left(\mathrm{T}-6.75\right)\\ 117.341·\left[\cos \left(0.0005\left(\mathrm{T}-6.75\right)+0.0005\Delta \mathrm{T}\right)-\cos \left(0.0005\left(\mathrm{T}-6.75\right)\right)\right]\\ -0.15\left(\mathrm{T}-6.75\right)\\ {\widehat{f}}_1·{\psi }_1\end{array}\right)$$

(43)

$$t_2=t_2^{\mathrm{init}}+\left(\begin{array}{c}117.341·\left[\sin \left(0.0005\left(\mathrm{T}-6.75\right)+0.0005\Delta \mathrm{T}\right)-\sin \left(0.0005\left(\mathrm{T}-6.75\right)\right)\right]+0.15\left(\mathrm{T}-6.75\right)\\ 117.341·\left[\cos \left(0.0005\left(\mathrm{T}-6.75\right)+0.0005\Delta \mathrm{T}\right)-\cos \left(0.0005\left(\mathrm{T}-6.75\right)\right)\right]\\ 0.15\left(\mathrm{T}-6.75\right)\\ {\widehat{f}}_2·{\psi }_2\end{array}\right)$$

(44)

where ${\widehat{f}}_1$, φ₁, ${\widehat{f}}_2$, and φ₂ can be obtained according to the following equations:

$$\begin{gathered} {\mathbf{R}}_1({\widehat{f}}_1,{\phi }_1)={\mathbf{R}}_2({\widehat{f}}_2,{\phi }_2) \\ =\left[\begin{array}{ccc}\cos \left(-\frac{\pi }{2}-0.0005\left(T-6.75\right)\right)& -\sin \left(-\frac{\pi }{2}-0.0005\left(T-6.75\right)\right)& 0\\ \sin \left(-\frac{\pi }{2}-0.0005\left(T-6.75\right)\right)& \cos \left(-\frac{\pi }{2}-0.0005\left(T-6.75\right)\right)& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ 0& 0& 1\\ 0& -1& 0\end{array}\right] \end{gathered}$$

(45)

All the parameters in Table 5 and Table 6, the rotation center position, $t_1^{\mathrm{init}}$, and $t_1^{\mathrm{init}}$, were used in Figure 4. The inverse kinematics is based on Equation (30) to obtain the joint angle, Θ. The change of the new weight value, $\Delta {\widehat{w}}_i$, of the self-organizing competitive neural network is obtained according to Equation (27). The learning rate, i.e., $\eta$, ${\eta }_1$, ${\eta }_{n,1}$, …, ${\eta }_{n,k}$, is used to calculate the carrying step size of EE motion in Cartesian space. With the continuous learning and competition of the proposed planning method, the position velocities of EEs gradually form a consistent movement. Figure 12a,b show the motion paths of EEs and joint trajectories, respectively. Figure 12c,d show the periodic change of pose in Cartesian space where the Y-axis position, the Z-axis position, and the attitude of the EEs remain the same. The position velocity and the attitude velocity reach synchronous motion states when the position velocity error and the attitude velocity error decrease to 0, as illustrated in Figure 12e–h.

(2)

Manipulating the pliers

Figure 13 shows that the two arms manipulated a pair of pliers. The parameters are illustrated in Table 5 and Table 6. The rotation center position is (−50.0 mm, −347.55 mm, and 0.85 mm)^T. The length of the pliers’ handle is 135 mm. The tracked initial locations on the carried object for the EEs of left and right arms are $t_1^{\mathrm{init}}$ = (86.841, −212.7, 0.85, 1.2092, −1.2092, −1.2092)^T and $t_2^{\mathrm{init}}$ = (−186.841, −212.7, 0.85, 1.2092, −1.2092, −1.2092)^T. At 6.25 s, two arms begin to manipulate the pliers, as shown in Figure 14b. During the collaboration process, the position of the tracked locations on the pliers moves back and forth in a 3 s cycle, and the tracked pose trajectory in the initial 1.5 s (from 6.25 s to 7.75 s) is as follows:

$$t_1=t_1^{\mathrm{init}}+\left(\begin{array}{c}135·\left[\sin \left(0.0005\left(\mathrm{T}-6.25\right)+0.0005\Delta \mathrm{T}\right)-\sin \left(0.0005\left(\mathrm{T}-6.25\right)\right)\right]\\ 135·\left[\cos \left(0.0005\left(\mathrm{T}-6.25\right)+0.0005\Delta \mathrm{T}\right)-\cos \left(0.0005\left(\mathrm{T}-6.25\right)\right)\right]\\ 0\\ {\widehat{f}}_1·{\psi }_1\end{array}\right)$$

(46)

$$t_2=t_2^{\mathrm{init}}+\left(\begin{array}{c}135·\left[\sin \left(-0.0005\left(\mathrm{T}-6.25\right)-0.0005\Delta \mathrm{T}\right)-\sin \left(-0.0005\left(\mathrm{T}-6.25\right)\right)\right]\\ 135·\left[\cos \left(0.0005\left(\mathrm{T}-6.25\right)+0.0005\Delta \mathrm{T}\right)-\cos \left(0.0005\left(\mathrm{T}-6.25\right)\right)\right]\\ 0\\ {\widehat{f}}_2·{\psi }_2\end{array}\right)$$

(47)

where ${\widehat{f}}_1$, ${\mathsf{\psi }}_1$, ${\widehat{f}}_2$, and ${\mathsf{\psi }}_2$ can be obtained according to the following equations:

$${\mathbf{R}}_1({\widehat{f}}_1,{\phi }_1)=\left[\begin{array}{ccc}\cos \left(-\frac{\pi }{2}-0.0005\left(\mathrm{T}-6.25\right)\right)& -\sin \left(-\frac{\pi }{2}-0.0005\left(\mathrm{T}-6.25\right)\right)& 0\\ \sin \left(-\frac{\pi }{2}-0.0005\left(\mathrm{T}-6.25\right)\right)& \cos \left(-\frac{\pi }{2}-0.0005\left(\mathrm{T}-6.25\right)\right)& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ 0& 0& 1\\ 0& -1& 0\end{array}\right]$$

(48)

$${\mathbf{R}}_2({\widehat{f}}_2,{\phi }_2)=\left[\begin{array}{ccc}\cos \left(-\frac{\pi }{2}+0.0005\left(\mathrm{T}-6.25\right)\right)& -\sin \left(-\frac{\pi }{2}+0.0005\left(\mathrm{T}-6.25\right)\right)& 0\\ \sin \left(-\frac{\pi }{2}+0.0005\left(\mathrm{T}-6.25\right)\right)& \cos \left(-\frac{\pi }{2}+0.0005\left(\mathrm{T}-6.25\right)\right)& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ 0& 0& 1\\ 0& -1& 0\end{array}\right]$$

(49)

The inverse kinematics is based on Equation (30) to obtain the joint angle, Θ. The change of the new weight value, $·{\widehat{w}}_i$, is obtained according to Equation (27) and used to adjust the EE motions according to Equation (28). The learning rate, i.e., $\eta$, ${\eta }_1$, ${\eta }_{n,1}$, …, ${\eta }_{n,k}$, is used to calculate the step size of manipulating the pilers in the Cartesian space. The proposed planning method makes the position velocities of EEs gradually form a consistent movement. Figure 14a,b show the motion paths of EEs and joint trajectories, respectively. Figure 14c,d show the periodic change of pose in Cartesian space, where the Y-axis position, the Z-axis position, and the attitude of the EEs remain the same. The position velocity and the attitude velocity of the EEs are almost the same and showed a small error when synchronous motion states were reached, as illustrated in Figure 14e–h.

(3)

Manipulating a rudder

Manipulating a rudder is illustrated in Figure 15. The parameters are shown in Table 5 and Table 6. The rotation center position is (0.0 mm, −342.7 mm, −0.85 mm)^T. The diameter of the rudder is 136.84 mm. The tracked initial locations on the carried object for the EEs of left and right arms are $t_1^{\mathrm{init}}$ = (136.84, −342.7, −0.85, 1.2092, −1.2092, −1.2092)^T and $t_2^{\mathrm{init}}$ = (−136.84, −342.7, 0.85, 1.2092, −1.2092, −1.2092)^T. At 6.75 s, two arms begin to manipulate the rudder, as shown in Figure 16b. During the collaboration process, the position of the tracked locations on the rudder moves back and forth in a 2.5 s cycle, and the tracked pose trajectory in the initial 1.25 s (from 6.75 s to 8 s) is as follows:

$$t_1=t_1^{\mathrm{init}}-\left(\begin{array}{c}136.841·\left[\sin \left(0.0005\left(\mathrm{T}-6.75\right)+0.0005\Delta \mathrm{T}\right)-\sin \left(0.0005\left(\mathrm{T}-6.75\right)\right)\right]\\ 136.841·\left[\cos \left(0.0005\left(\mathrm{T}-6.75\right)+0.0005\Delta \mathrm{T}\right)-\cos \left(0.0005\left(\mathrm{T}-6.75\right)\right)\right]\\ 0\\ {\widehat{f}}_1·{\psi }_1\end{array}\right)$$

(50)

$$t_2=t_2^{\mathrm{init}}+\left(\begin{array}{c}136.841·\left[\sin \left(0.0005\left(\mathrm{T}-6.75\right)+0.0005\Delta \mathrm{T}\right)-\sin \left(0.0005\left(\mathrm{T}-6.75\right)\right)\right]\\ 136.841·\left[\cos \left(0.0005\left(\mathrm{T}-6.75\right)+0.0005\Delta \mathrm{T}\right)-\cos \left(0.0005\left(\mathrm{T}-6.75\right)\right)\right]\\ 0\\ {\widehat{f}}_2·{\psi }_2\end{array}\right)$$

(51)

where ${\widehat{f}}_1$, ${\psi }_1$, ${\widehat{f}}_2$, and ${\psi }_2$ can be obtained according to the following equations:

$$\begin{gathered} {\mathbf{R}}_1({\widehat{f}}_1,{\phi }_1)={\mathbf{R}}_2({\widehat{f}}_2,{\phi }_2) \\ =\left[\begin{array}{ccc}\cos \left(-\frac{\pi }{2}-0.0005\left(\mathrm{T}-6.75\right)\right)& -\sin \left(-\frac{\pi }{2}-0.0005\left(\mathrm{T}-6.75\right)\right)& 0\\ \sin \left(-\frac{\pi }{2}-0.0005\left(\mathrm{T}-6.75\right)\right)& \cos \left(-\frac{\pi }{2}-0.0005\left(\mathrm{T}-6.75\right)\right)& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ 0& 0& 1\\ 0& -1& 0\end{array}\right] \end{gathered}$$

(52)

Similar to the previous experiments, the inverse kinematics is based on Equation (30) to obtain the joint angle, Θ. The change of the new weight value, $\Delta {\widehat{w}}_i$, is obtained according to Equation (27) and used to adjust the EE motions according to Equation (28). The learning rate, i.e., $\eta$, ${\eta }_1$, ${\eta }_{n,1}$, …, ${\eta }_{n,k}$, is used to calculate the step size of manipulating the rudder in Cartesian space. Since the shape of the rudder is symmetrical at the center, the tracked locations are also symmetrical, and the EE attitude changes are the same. In Figure 16c,d, the two arms begin to manipulate the rudder at 6.75 s. In Figure 16e–h, the EE pose errors can converge to 0, and the corresponding EE motions own almost the same states ultimately by using the proposed planning method with the learning of the self-organizing competitive neural network.

Considering (1)–(3) in Section 5.2 comprehensively, the pose speed and pose error converge to 0 for the successful execution of coordination manipulation and correspond to Equation (2), and no arm motion is set as the reference for the other arms. Hence, the proposed planning method based on the self-organizing competitive neural network owns the feasibility, synchronism, and effectiveness in achieving the collaboration manipulations for the arms with physical coupling and has the contributions to the practical application.

6. Conclusions

This paper presents a real-time kinematically synchronous planning method for collaborative manipulation through the self-organizing competitive neural network. This method considers a type of collaborative manipulation known as the synchronization of EE motion. The sub-bases are defined for the configuration of multi-arms to obtain the Jacobian matrix of common DoFs and ensure the pose errors converging along the reducing direction of the EE total pose errors. The simulations of multi-arms with common DoFs display the consistency before the pose errors converge completely and make contributions to the collaborative manipulation of multi-arms. On this basis, an unsupervised competitive neural network is raised to regard the EE synchronous motion as the competition of neurons and adaptively increase the convergence ratio of multi-arms through the mutual learning and competition of neurons by using the inner star rules. The stability of multi-arms system is analyzed through the Lyapunov theory. Various simulations and experiments confirm that the proposed synchronous planning method is feasible, synchronous, and has the application potentiality in different cooperative manipulation tasks.