Buffer Compliance Control of Space Robots Capturing a Non-Cooperative Spacecraft Based on Reinforcement Learning

Abstract

Aiming at addressing the problem that the joints are easily destroyed by the impact torque during the process of space robot on-orbit capturing a non-cooperative spacecraft, a reinforcement learning control algorithm combined with a compliant mechanism is proposed to achieve buffer compliance control. The compliant mechanism can not only absorb the impact energy through the deformation of its internal spring, but also limit the impact torque to a safe range by combining with the compliance control strategy. First of all, the dynamic models of the space robot and the target spacecraft before capture are obtained by using the Lagrange approach and Newton-Euler method. After that, based on the law of conservation of momentum, the constraints of kinematics and velocity, the integrated dynamic model of the post-capture hybrid system is derived. Considering the unstable hybrid system, a buffer compliance control based on reinforcement learning is proposed for the stable control. The associative search network is employed to approximate unknown nonlinear functions, an adaptive critic network is utilized to construct reinforcement signal to tune the associative search network. The numerical simulation shows that the proposed control scheme can reduce the impact torque acting on joints by 76.6% at the maximum and 58.7% at the minimum in the capturing operation phase. And in the stable control phase, the impact torque acting on the joints were limited within the safety threshold, which can avoid overload and damage of the joint actuators.

1. Introduction

With the development of space technology, the number of spacecraft launched every year is increasing, thereby generating a series of high-intensity and high-risk space missions, such as on-orbit fuel refueling, on-orbit maintenance, recovery of failed spacecraft, space debris removal [1,2,3,4], etc. As outer space is a harsh environment with high pressure, extreme temperatures, high vacuum and strong electromagnetic radiation, it is very hazardous for astronauts to out of the module to carry out the above mission operations. Giordano et al. [5] proposed a dynamics decomposition that decouples the end-effector task from the base force actuator and reduces the use of thrusters. Qin et al. [6] proposed a fuzzy adaptive robust control (FARC) strategy which is adaptive to these model variations for trajectory tracking control of space robots. Virgili-Llop et al. [7] presented an optimization-based guidance algorithm for onboard implementation and real-time use suitable for space robots. Ai and Chen [8] considered the process of capturing spacecraft by dual-arm clamping and the force/position control of its post-stabilization movement and proposed a fuzzy control scheme based on the passivity theory. Therefore, it is a better choice to use space robots to replace astronauts to complete on-orbit services (OOS) missions. Because the capture and operation ability of space robots is the basic and key technology to realize OOS missions. Liu et al. [9] studied the on-orbit services space robot considering joint friction, based on the Jourdain’s velocity variation principle and the single direction recursive construction method derived the dynamic equation of the system. Lim and Chung [10] analyzed the dynamic behavior of a tethered satellite system for space debris capture, by using the absolute nodal coordinate formulation established the equations of motion of the tethered satellite system. Shah et al. [11] presented strategies for point-to-point reactionless manipulation of a satellite- mounted dual-arm robotic system for capturing tumbling orbiting objects. Uyama et al. [12] studied an impedance-based contact control of a free-flying space robot with respect to the coefficient of restitution Therefore, the research on capturing operation technology of space robots has become a hot topic in the aerospace field in recent years.

The operation process of space robot capturing a non-cooperative spacecraft can be divided into four phases: (1) the observation phase, in this phase, the position and attitude of the target spacecraft are observed; (2) the approaching phase, through trajectory planning and motion control of space robot to reaching the capture area; (3) the capturing operation phase, the space robot uses the end-effector to capture the target spacecraft; (4) stable control phase, considering the post-capture unstable motion which is caused by the collision and impact of the capturing operation phase, design the stability control strategy of the hybrid system formed by the space robot and the target spacecraft. Considering that the space robot will inevitably experience a violent contact and collision with the target spacecraft during phase 3 of the capturing operation process, in this process, the joint of the manipulator arm will be subjected to a great impact moment [13]. If the impact torque affecting the joint is too large, it could cause the impact damage to the joints and lead to the failure of the space mission. At present, there is no effective way to solve the problem except using the minimum relative approaching velocity. Although this method is feasible for cooperative spacecraft, it is basically not applicable to non-cooperative spacecraft. Therefore, in the phases 3 and 4 of the capture of a non- cooperative spacecraft, it is of great exploratory value and significance to take certain measures to avoid the damage to joint actuators caused by such impact and collision.

Recently, the dynamics and control of space robots capturing a spacecraft have become the focus of aerospace technicians, and some research results have emerged. It is worth mentioning that the studies mainly focus on pre-capture motion planning and post-capture attitude control. For the motion planning and trajectory tracking control of space robots, Jiang et al. [14] investigated the finite-time control problem associated with attitude stabilization of a rigid spacecraft subject to external disturbances, actuator faults, and input saturation, and proposed an adaptive fixed-time-based finite-time attitude controller designed to guarantee finite-time reachability of the attitude orientation in a small neighborhood of the equilibrium point. Liu et al. [15] studied the effect of payload collisions on the dynamics and control of a flexible dual-arm space robot capturing an object, proposed a method for the determination of initial conditions for post-impact dynamic simulation of the system and proposed a PD controller to maintain stabilization of the robot system after the capture of the object. Walker et al. [16] presented an adaptive control method that achieves globally stable trajectory tracking in the presence of uncertainties in the inertial parameters of the system. Yi and Ge [17] studied an indirect Legendre pseudospectral method for attitude motion tracking control of an asymmetric underactuated rigid spacecraft equipped with only two pairs of jet thrusters. Sands [18] proposed a novel optimization whiplash compensation method to realize automatic control of flexible space robotics. Stolfi [19] focused on the issue of maintaining a stable first contact between the arms end-effectors and a target satellite before the grasp is performed, investigates the application of the Impedance + PD control approach to a two-arm space manipulator used to capture a non-cooperative target. Zhang and Zhu [20] presented the notion that the planning task does not need to solve the inverse kinematics, investigating a novel motion planning algorithm based on rapidly-exploring random trees (RRTs) for an free-floating space robots from an initial configuration to a goal end-effector pose. Cocuzza [21] aimed at locally minimizing the dynamic disturbances transferred to the spacecraft during trajectory tracking maneuvers, based on a constrained least-squares approach, proposed a novel solution for the inverse kinematics of redundant space manipulators. Du et al. [22] based on the continuous finite-time control technique, studied the attitude stabilization of spacecraft, a finite-time attitude tracking control law has been designed for a single spacecraft and a distributed finite-time attitude synchronization algorithm has also been developed for a group spacecraft. Aghili [23] presented a combined prediction and motion-planning scheme for robotic capturing of a drifting and tumbling object with unknown dynamics using visual feedback, and used the estimated states, parameters, and predicted motion trajectories to plan the trajectory of the robot’s end-effector to intercept a grapple fixture on the object with zero relative velocity in an optimal way. In order to realize the attitude stabilization and joint tracking control of the space robot with flexible links and elastic base, Yu [24] proposed a terminal sliding mode controller based on desired trajectory to control the free-flying space manipulator when parametric uncertainties and modeling errors exist.

For the post-capture attitude stable control of space robots, Cheng [25] studied the attitude management of space robots after capturing a satellite, the control of the auxiliary docking operation and presented an adaptive control scheme based on extreme learning machine to achieve the coordinated control of the target. Wang et al. [26] considered identifying the mass properties and eliminating the unknown angular momentum of space robotic systems after capturing a non-cooperative target, designing an integrated control framework which includes a detumbling strategy, coordination control and parameter identification, and proposed a coordination control scheme for stabilizing both the base and end-effector based on impedance control implemented considering the target’s parameter uncertainty. Zhang et al. [27] proposed a modified adaptive sliding mode control algorithm to reduce the momentum, which can reduce the unknown angular momentum of a target, and uses a new signum function and time-delay estimation to assure fast convergence and achieve good performance with a small chattering effect. Wu et al. [28] developed a generic frictional contact model which can represent the contact forces between the robot’s end-effector and the target object and designed a resolved motion admittance control method based on the frictional contact model. Rekleitis [29] developed a planning and control methodology for manipulating passive objects by cooperating orbital free-flying servicers in zero gravity. Although the above control algorithms focus on the dynamics and control of space robots capturing a spacecraft, the protection of the joint actuators of the space robot under the impact torque is not considered. Since a space robot’s joints are easily destroyed by the impact torque during the process of space robot on-orbit capturing a non-cooperative spacecraft, therefore, the studies on compliance control of space robots during the capturing process need to be improved.

For the series elastic actuator (SEA) in the ground robot, Gu et al. [30] presented a modularized series elastic actuator aimed to improve the compliance of the robotic arm. Calanca and Fiorini [31] refined and improved the stability analysis of the environment-adaptive force control of SEAs. Wang et al. [32] presented a practical control approach for series elastic actuators which can work well even in the presence of unknown payload parameters and external disturbances. Considering that SEA devices play a key role in protecting the robot’s joints from impact damage when the ground robot collides with the outside environment, therefore, this paper designs a rotary series elastic actuator (RSEA) device suitable for space robots, and at the same time, designs an active controller strategy which can timely control the opening and closing of joint actuators to achieve buffer compliance control. The RSEA also leads to joint flexibility due to the presence of a buffer spring inside the system. Since the system meets the law of conservation of linear momentum and law of conservation of angular momentum, its orbital dynamics and base attitude are coupled, which make its links’ locomotion leads to the base’s reactions, and consequently a variation of the end-effector position. At the same time, momentum, momentum moment and energy transfer change also exist in the pre-contact and post-capture phase of systems consisting of a space robot and spacecraft. In addition, due to the high velocity and rotation characteristics of the non-cooperative target spacecraft, the dynamic parameters of the post-capture hybrid system are difficult to obtain accurately. The above multiple complex situations make research on the dynamic modeling and control of the on-orbit capturing process of space robots equipped with RSEA devices very complicated.

In an effort to address the various aforementioned drawbacks, this work investigates the dynamic modeling, buffer compliance control and vibration suppression of a space robot capturing a non-cooperative spacecraft. First of all, dynamic models of the space robot and the target spacecraft before capture are obtained by using the Lagrange approach and Newton-Euler method. Second, based on singular perturbation theory, the post-capture hybrid system was transformed into two subsystems, a slow rigid motion subsystem, and a fast flexible-joint subsystem. For the fast subsystem, the velocity difference feedback controller is used to actively suppress the elastic vibration of the joints’ flexibility. For the slow subsystem, a buffer compliance control scheme based on reinforcement learning (RL) is proposed. The proposed reinforcement learning consists of two modules: associative search network (ASN) and adaptive critic network (ACN). ASN is used to approximate unknown nonlinear terms of mixed systems; the ACN adopts the online learning method. The learning strategy of RL obtains the original error evaluation signal through the performance evaluation unit, this error evaluation signal is coupled with ACN to generate the reinforcement signal. Then, the updated result is used as the learning rule of the neural network to train the neural network weight adaptive law of ASN and ACN, which can adjust and optimize the control strategy in real time. For the reinforcement learning strategy, Liu et al. [33] obtained the system dynamics model of space robota by reinforcement learning, by comparison with the traditional PD control method, that shows the self-learning ability of the reinforcement learning strategy. Sands [34] proposed deterministic artificial intelligence that can applied to both unmanned underwater vehicles and space robotics. Tang and Liu [35] studied the control and stability issues of a trajectory tracking of an n-link rigid robot manipulator, and obtained an optimal control signal by a reinforcement learning strategy. Cui et al. [36] proposed a reinforcement learning strategy to investigate the trajectory tracking problem for a fully actuated autonomous underwater vehicle with external disturbances, control input nonlinearities and model uncertainties. On this basis, the proposed control scheme can absorb the impact energy generated in the collision process through the stretching and compression of the built-in spring in the collision capture phase. In the stable control phase, the control strategy based on reinforcement learning is used to actively turn on and off the joints’ actuators to ensure that the joints’ actuators will not be overloaded and damaged. In addition, the reinforcement learning strategy has the advantage of not needing the precise dynamics model of the hybrid system and can effectively improve the intelligence and reliability of the on-orbit acquisition operation of the space robot. The numerical simulation shows that the proposed control scheme can not only effectively absorb the impact energy generated by the on-orbit capture, but also open and close the joint actuators in a timely way when the impact energy is too large, which can avoid overload and damage to the joint actuators.

The paper is organized as follows: in Section 2, the compliant mechanism and buffer compliance strategy are introduced. In Section 3, the dynamic model of the space robot capturing a non-cooperative target spacecraft is established. In the same section, the impact effect during the capturing operation phase is discussed. In Section 4, a reinforcement learning control algorithm combined with a compliant mechanism is proposed to achieve buffer compliance control and its stability is verified by introducing the suitable Lyapunov function. In Section 5, numerical simulations are carried out to validate the proposed buffer compliance control strategy. Finally, the conclusions are given in Section 6.

2. Buffer Compliance Strategy

The RSEA consists of five modules: input disk, sweeping arm, support axis, springs, block. The RSEA device of the space robot system is installed between the actuators and the manipulator and is connected to the actuators through its input disk. The block is firmly connected to the input disk. The hollow shaft of the sweeping arm is connected with the support axis fixed on the input disk through a bearing. When the motor rotates it drives the input disk to rotate. Through the block compression spring, the spring transfers the force to the sweeping arm. The hollow shaft of the sweeping arm is directly connected with the manipulator, so as to complete the smooth transfer of motion and force. The general structure diagram of the space manipulator is shown in Figure 1, and the structure of the designed RSEA device is shown in Figure 2. In Figure 2, R is the effective radius of sweeping arm and r is the radius of spring.

In the capture phase, the end-effector of the manipulator contacts and collides with the spacecraft, whereupon the joint of the manipulator will be subjected to a huge impact torque. The impact torque acts on the output sweeping arm of the RSEA device first, and then is transferred to the spring group. The impact energy generated by the collision is stored in the spring through the deformation of the spring group, so as to realize the protection of the joint. In the stable control phase, the joints are also affected by the impact torque due to the impact of the capture collision. If the torque exceeds the limit that the joint actuators can withstand and the actuators do not turn off, the actuators will be damaged. Therefore, it is necessary to set a shutdown torque threshold to turn off the actuators according to the torque limit that the joint can withstand. When the impact torque on the joints is detected to exceed the shutdown torque threshold value, all actuators turn off. In this time, the internal spring assembly of the RSEA device provides an elastic force to reduce the impact torque on the joints. In addition, in practical operation, if only the shutdown torque threshold is set, the actuators will be switched on and off frequently, thus affecting the actuators performance. On this basis, the control strategy proposed also sets a startup torque threshold value of actuators, when the joint torque exceeds the shutdown torque threshold, the actuators turn off, and when the joint torque is below the startup torque threshold, the actuators turn on again.

3. Dynamics Modeling and Impact Effect Analysis

The structure of a space robot with RSEA and target spacecraft systems is shown in Figure 3. The space robot consists of a rigid base B₀, rigid links B_i (i = 1,2), and rigid target spacecraft B₃. We build the inertial coordinate system XOY, while at the same time, the local coordinate system x_iO_iy_i (i = 0,1,2) of each component B_i (i = 1,2) is established; O₀ is the rotation center of the base, O_i is the rotation center of B_i (i = 1,2); m₀ is the mass of the base, m_s is the mass of the non-cooperative spacecraft, m_i is the mass of B_i (i = 1,2). I₀ is inertial moment of the base with respect to its mass center, I_s is the inertial moment of the non-cooperative spacecraft with respect to its mass center, I_i (i = 1,2) is the inertial moment of B_i (i = 1,2) with respect to their mass center. I₀ represents the distance from point O₀ to O₁, l_i (i = 1,2) represents length of B_i along the x_i axis. d_i (i = 1,2) is the distance from the mass center of B_i to O_i. I_im (i = 1,2) is inertial moment of the i-h actuator. k_im (i = 1,2) is the spring stiffness of the RSEA device. r_c is the position vector of the mass center of the entire system in inertial coordinate system (XOY). r_i (i = 1,2) is position vector of the mass center of B_i in the inertial coordinate system (XOY).

Regarding the target spacecraft as a homogeneous rigid body, its dynamic equation can be obtained by the Newton-Euler method:

$$D_{\mathrm{s}}{\ddot{q}}_{\mathrm{s}}=J_{\mathrm{s}}^{\mathrm{T}}{F}^{\prime }$$

(1)

where $q_{\mathrm{s}}={[x_{\mathrm{s}},y_{\mathrm{s}},{\theta }_{\mathrm{s}}]}^{\mathrm{T}}$ the generalized coordinates of the target spacecraft system; $x_{\mathrm{s}}$, $y_{\mathrm{s}}$ are the position vectors of the mass center of $B_3$, ${\theta }_{\mathrm{s}}$ is the attitude angle of the spacecraft system. $D(q)\in {\mathbf{R}}^{3\times 3}$ are the inertia positive definite matrices, $J_{\mathrm{s}}\in {\mathbf{R}}^{3\times 3}$ is its impact contact point corresponding to the motion Jacobi matrix. $F^{\prime }\in {\mathbf{R}}^{3\times 1}$ is the force acting on the spacecraft.

According to the position vector relation in Figure 2, the position vectors of the mass center of $B_i(i=0,1,2)$ in the pre-contact phase are:

$$\left\{\begin{array}{c}{r}_0={[x_a,y_a]}^{\mathrm{T}}\\ r_1=r_0+l_0{e}_0+d_1{e}_1\\ r_2=r_0+l_0{e}_0+l_1{e}_1+d_2{e}_2\end{array}\right.$$

(2)

where $x_a$, $y_a$ are the position vector of the mass center of base $B_0$, $e_i(i=0,1,2)$ is the unit vector along the x_i axis in the $x_i{O}_i{y}_i$ frame.

Differentiating Equation (2) with respect to time t, then the total kinetic energy of the space robot with RSEA is:

$$T={\displaystyle \sum _{i=0}^2(\frac{1}{2}}{m}_i{\dot{r}}_i^{\mathrm{T}}{\dot{r}}_i+\frac{1}{2}{I}_i{\omega }_i^{\mathrm{T}}{\omega }_i)+{\displaystyle \sum _{j=1}^2\frac{1}{2}{I}_{jm}{\omega }_{jm}^{\mathrm{T}}{\omega }_{jm}}$$

(3)

where ${\omega }_i(i=0,1,2)$ is the angular velocity of the rotation center $O_i$, ${\omega }_{jm}(j=1,2)$ is the angular velocity of the actuator.

Neglecting the micro-gravity in space, the potential energy of the system only comes from the RSEA device, so the total potential energy of the system is:

$$U={\displaystyle \sum _{i=1}^2\left\{\frac{3}{2}{k}_{im}\left[{(\Delta x_{iL})}^2+{(\Delta x_{iR})}^2\right]\right\}}$$

(4)

where $\Delta x_{iL}=x({\alpha }_i)$, $\Delta x_{iR}=-x({\alpha }_i)$, $x({\alpha }_i)=R\sin ({\alpha }_i)$. $x({\alpha }_i)$ is the deformation of the spring on the block of the RSEA device, ${\alpha }_i$ is the angular difference between the sweeping arm and the input disk.

Based on Equations (3) and (4), and combing with the Lagrange equations, the dynamic equations of the space robot of pre-capture phase are as follows

$$\left\{\begin{array}{c}D(q)\ddot{q}+C(q,\dot{q})\dot{q}={\tau }_c+J^{\mathrm{T}}F\\ I_m{\ddot{\theta }}_m+K({\theta }_m-\theta )={\tau }_m\\ K({\theta }_m-\theta )={\tau }_{\theta }\end{array}\right.$$

(5)

where $q={[x_a,y_a,{\theta }_0,{\theta }_1,{\theta }_2]}^{\mathrm{T}}$ are the generalized coordinates of the system, ${\theta }_0$ is the attitude angle displacement of the base, ${\theta }_i(i=1,2)$ is the attitude angle displacement of the i-th link, ${\theta }_{im}(i=1,2)$ is the attitude angle displacement of the i-th actuator. $D(q)\in {\mathbf{R}}^{5\times 5}$ are the inertia positive definite matrices, $C(q,\dot{q})\dot{q}\in {\mathbf{R}}^{5\times 1}$ is the Coriolis/centrifugal matrix. ${\theta }_m={[{\theta }_{1m},{\theta }_{2m}]}^{\mathrm{T}}$, $\theta ={\left[{\theta }_1,{\theta }_2\right]}^{\mathrm{T}}$. ${\tau }_c={[{\tau }_a^{\mathrm{T}},{\tau }_0,{\tau }_{\theta }^{\mathrm{T}}]}^{\mathrm{T}}$, ${\tau }_a={[0,0]}^{\mathrm{T}}$ is the position control torque of the base, ${\tau }_0$ is the attitude control torque of base. ${\tau }_m={[{\tau }_{1m},{\tau }_{2m}]}^{\mathrm{T}}$ is the joint torque/force delivered by actuators. $I_m=\mathrm{diag}(I_{1m},I_{2m})$, $K=\mathrm{diag}(k_1,k_2)$ is the equivalent stiffness of joints, and its calculation formula is given in Equation (46). $J\in {\mathbf{R}}^{3\times 5}$ is its end-effector impact contact point corresponding motion Jacobi matrix, $F\in {\mathbf{R}}^{3\times 1}$ is the force acting on the end-effector.

In the capturing operation phase, the space robot contacts and collides with the target spacecraft, and the interaction force at the end is satisfied:

$$F=-F^{\prime }$$

(6)

Based on Equation (6), and combining it with Equations (1) and (5), we can obtain that:

$$D(q)\ddot{q}+C(q,\dot{q})\dot{q}={\tau }_c-J^{\mathrm{T}}{(J_{\mathrm{s}}^{\mathrm{T}})}^{-1}{D}_{\mathrm{s}}{\ddot{q}}_{\mathrm{s}}$$

(7)

The actuators will be turned off during the capture phase, which is ${\tau }_c=0_{5\times 1}$. Integrating Equation (7) over the momentary period of collision [13]:

$$D(q)(\dot{q}(t_0+\Delta t)-\dot{q}(t_0))+J^{\mathrm{T}}{(J_{\mathrm{s}}^{\mathrm{T}})}^{-1}{D}_{\mathrm{s}}({\dot{q}}_{\mathrm{s}}(t_0+\Delta t)-q_{\mathrm{s}}(t_0))=0$$

(8)

The space robot and spacecraft satisfy the velocity constraint in the post-capture phase. Based on this, the following generalized velocity of the post-capture hybrid system can be obtained:

$$\dot{q}(t_0+\Delta t)=N^{-1}[D(q)\dot{q}(t_0)+J^{\mathrm{T}}{(J_{\mathrm{s}}^{\mathrm{T}})}^{-1}{D}_{\mathrm{s}}\dot{q}(t_0)]$$

(9)

where $N=D(q)+J^{\mathrm{T}}{(J_{\mathrm{s}}^{\mathrm{T}})}^{-1}{D}_{\mathrm{s}}{J}_{\mathrm{s}}^{-1}J$.

Integrating first item of Equation (5), we have:

$$D(q)(\dot{q}(t_0+\Delta t)-\dot{q}(t_0))=J^{\mathrm{T}}P$$

(10)

where $P={\displaystyle {\int }_{t_0}^{t_0+\Delta t}F}\mathrm{d}t$ is the impact impulse during the capture phase. Invoking Equations (9), and (10), we can obtain that:

$$P={(J^{\mathrm{T}})}^{+1}D(q)[N^{-1}(D(q)\dot{q}(t_0)+J^{\mathrm{T}}{(J_{\mathrm{s}}^{\mathrm{T}})}^{-1}{D}_{\mathrm{s}}\dot{q}(t_0))-\dot{q}(t_0)]$$

(11)

where ${(J^{\mathrm{T}})}^{+1}$ is the Moore-Penrose pseudo-inverse of $J^{\mathrm{T}}$. The period of contact is transient: $\Delta t\to 0$, then the collision force can be approximated as:

$$F=\frac{P}{\Delta t}$$

(12)

After the space robot capturing the target spacecraft, a hybrid system is formed. Consider the velocity constraint relationship of arm and target, we can obtain that:

$$J\dot{q}=J_{\mathrm{s}}{\dot{q}}_{\mathrm{s}}$$

(13)

Differentiating Equation (13), we have:

$${\ddot{q}}_{\mathrm{s}}=J_{\mathrm{s}}^{-1}[J\ddot{q}+(\dot{J}-{\dot{J}}_{\mathrm{s}}{J}_{\mathrm{s}}^{-1}J)\dot{q}]$$

(14)

Invoking Equations (1), (5) and (14), we can obtain that:

$$\left\{\begin{array}{c}{D}_{\mathrm{A}}(q)\ddot{q}+C_{\mathrm{A}}(q,\dot{q})\dot{q}={\tau }_c\\ I_m{\ddot{\theta }}_m+K({\theta }_m-\theta )={\tau }_m\\ K({\theta }_m-\theta )={\tau }_{\theta }\end{array}\right.$$

(15)

where $D_{\mathrm{A}}(q)=D(q)+J^{\mathrm{T}}{(J_{\mathrm{s}}^{\mathrm{T}})}^{-1}{D}_{\mathrm{s}}{J}_{\mathrm{s}}^{-1}J$; $C_{\mathrm{A}}(q,\dot{q})=C(q,\dot{q})+J^{\mathrm{T}}{(J_{\mathrm{s}}^{\mathrm{T}})}^{-1}{D}_{\mathrm{s}}{J}_{\mathrm{s}}^{-1}(\dot{J}-{\dot{J}}_{\mathrm{s}}{J}_{\mathrm{s}}^{-1}J)$.

In order to facilitate the design of subsequent control strategies, the first item of Equation (15) of the hybrid system can be expressed in the form of block matrices as follows, so as to obtain the fully controllable formal dynamics model:

$$\left[\begin{array}{cc}{D}_{\mathrm{A}11}& D_{\mathrm{A}12}\\ D_{\mathrm{A}21}& D_{\mathrm{A}22}\end{array}\right]\left[\begin{array}{c}{\ddot{q}}_a\\ {\ddot{q}}_{\theta }\end{array}\right]+\left[\begin{array}{cc}{C}_{\mathrm{A}11}& C_{\mathrm{A}12}\\ C_{\mathrm{A}21}& C_{\mathrm{A}22}\end{array}\right]\left[\begin{array}{c}{\dot{q}}_a\\ {\dot{q}}_{\theta }\end{array}\right]=\left[\begin{array}{c}{\tau }_a\\ {\tau }_b\end{array}\right]$$

(16)

where $q_a={[x_a,y_a]}^{\mathrm{T}}$, $q_{\theta }={[{\theta }_0,{\theta }_1,{\theta }_2]}^{\mathrm{T}}$, ${\tau }_b={[{\tau }_0,{\tau }_{\theta }^{\mathrm{T}}]}^{\mathrm{T}}$. $D_{\mathrm{A}11}\in {\mathbf{R}}^{2\times 2}$, $D_{\mathrm{A}12}\in {\mathbf{R}}^{2\times 3}$, $D_{\mathrm{A}21}\in {\mathbf{R}}^{3\times 2}$, $D_{\mathrm{A}22}\in {\mathbf{R}}^{3\times 3}$ the submatrices of $D_{\mathrm{A}}$, $C_{\mathrm{A}11}\in {\mathbf{R}}^{2\times 2}$, $C_{\mathrm{A}12}\in {\mathbf{R}}^{2\times 3}$, $C_{\mathrm{A}21}\in {\mathbf{R}}^{3\times 2}$, $C_{\mathrm{A}22}\in {\mathbf{R}}^{3\times 3}$ the submatrices of $C_{\mathrm{A}}$, and $C_{\mathrm{A}11}$, $C_{\mathrm{A}21}$ are zero matrix. Equation (16) can be decomposed into:

$$D_{\mathrm{A}11}{\ddot{q}}_a+D_{\mathrm{A}12}{\ddot{q}}_{\theta }+C_{\mathrm{A}11}{\dot{q}}_a+C_{\mathrm{A}12}{\dot{q}}_{\theta }={\left[\begin{array}{cc}0& 0\end{array}\right]}^{\mathrm{T}}$$

(17)

$$D_{\mathrm{A}21}{\ddot{q}}_a+D_{\mathrm{A}22}{\ddot{q}}_{\theta }+C_{\mathrm{A}21}{\dot{q}}_a+C_{\mathrm{A}22}{\dot{q}}_{\theta }={\tau }_b$$

(18)

From Equation (17), we have:

$${\ddot{q}}_a=-D_{\mathrm{A}11}^{-1}(D_{\mathrm{A}12}{\ddot{q}}_{\theta }+C_{\mathrm{A}11}{\dot{q}}_a+C_{\mathrm{A}12}{\dot{q}}_{\theta })$$

(19)

Invoking Equations (18) and (19), we can obtain that:

$$\left\{\begin{array}{c}{D}_x{\ddot{q}}_{\theta }+C_x{\dot{q}}_{\theta }={\tau }_b\\ I_m{\ddot{\theta }}_m+K({\theta }_m-\theta )={\tau }_m\\ K({\theta }_m-\theta )={\tau }_{\theta }\end{array}\right.$$

(20)

where $D_x=D_{\mathrm{A}22}-D_{\mathrm{A}21}{D}_{\mathrm{A}11}^{-1}{D}_{\mathrm{A}12}$, $C_x=C_{\mathrm{A}22}-D_{\mathrm{A}21}{D}_{\mathrm{A}11}^{-1}{C}_{\mathrm{A}12}$. And ${\dot{D}}_x-2C_x$ is an antisymmetric matrix.

4. Two-Time Scale Control

4.1. Fast Subsystem and the Corresponding Controller

In order to actively suppress the flexible vibration of the joint caused by the RSEA device, based on singular perturbation theory, the post-capture hybrid system was transformed into two subsystems, a slow rigid motion subsystem, and a fast flexible-joint subsystem. This controller consists of a slow sub-controller and a fast flexible-joint sub-controller:

$${\tau }_m={\tau }_{\mathrm{f}}+{\tau }_{\mathrm{s}}$$

(21)

where ${\tau }_{\mathrm{f}}\in {\mathbf{R}}^{2\times 1}$ is the fast flexible-joint sub-controller, ${\tau }_{\mathrm{s}}\in {\mathbf{R}}^{2\times 1}$ is the slow sub-controller.

Defining the positive proportional factor $\epsilon$ and the positive definite diagonal matrix $K_1$, it satisfies:

$$K=\frac{K_1}{{\epsilon }^2}$$

(22)

Invoking Equation (22), the flexible-joint fast subsystem is:

$${\epsilon }^2{\ddot{\tau }}_{\theta }=I_m^{-1}{K}_1({\tau }_m-I_m\ddot{\theta }-{\dot{\tau }}_{\theta })$$

(23)

In order to suppress the elastic vibration of the system, the following speed difference feedback controller is designed to control the fast subsystem:

$${\tau }_{\mathrm{f}}=-K_{\mathrm{f}}({\dot{\theta }}_m-\dot{\theta })$$

(24)

where $K_{\mathrm{f}}=K_2/\epsilon$, $K_2\in {\mathbf{R}}^{2\times 2}$ is a positive definite diagonal matrix.

Substituting Equations (21), (24) into Equation (23), we have:

$${\epsilon }^2{I}_m{\ddot{\tau }}_{\theta }=K_1({\tau }_{\mathrm{s}}-I_m\ddot{\theta }-{\tau }_{\theta })-\epsilon K_2{\dot{\tau }}_{\theta }$$

(25)

It can be shown that while $\epsilon \to 0$, the equivalent stiffness of joints $K\to \infty$. At this point, the hybrid system is equivalent to a rigid model. Then the dynamic equation of the slow subsystem can be obtained from first item of Equations (20) and (21)

$$D_{x\theta }{\ddot{q}}_{\theta }+C_{x\theta }{\dot{q}}_{\theta }={\tau }_{x\theta }$$

(26)

where $D_{x\theta }=D_x+I_x$, $I_x=\mathrm{diag}(0,I_{1m},I_{2m})$. $C_{x\theta }$ is the corresponding matrix of $C_x$ when $\dot{\theta }={\dot{\theta }}_m$, ${\tau }_{x\theta }={[{\tau }_0,{\tau }_{\mathrm{s}}^{\mathrm{T}}]}^{\mathrm{T}}$.

4.2. Slow Subsystem and the Corresponding Controller

The buffer compliance control based on reinforcement learning is shown in Figure 4, Where ASN is used to approximate the unknown nonlinear term of the system, ACN is used to construct reinforcement signals to optimize ASN.

Define the trajectory tracking error as:

$$e=q_{\theta d}-q_{\theta }$$

(27)

where $q_{\theta d}\in {\mathbf{R}}^{3\times 1}$ are desired trajectories of the hybrid system.

At the same time, the error evaluation signal is defined as:

$$z=\dot{e}+\Lambda e$$

(28)

where $\Lambda \in {\mathbf{R}}^{3\times 3}$ is a positive definite diagonal matrix.

Invoking Equations (27) and (28), the dynamic equation of the slow subsystem can be written as:

$$D_{x\theta }\dot{z}=-C_{x\theta }z+d-{\tau }_{x\theta }$$

(29)

where $d=D_{x\theta }({\ddot{q}}_{\theta d}+\Lambda \dot{e})+C_{x\theta }({\dot{q}}_{\theta d}+\Lambda e)$ is the unknown nonlinear term of the system. Considering that it cannot be obtained directly, it can be approximated by the ASN:

$$d=W_a^{\mathrm{T}}\Phi (x)+\varsigma (x)$$

(30)

where $W_a\in {\mathbf{R}}^{n\times 3}$ is the ideal weight matrix of a radial basis function neural network (RBFNN), $\varsigma (x)$ is the optimal approximation error. The radial basis kernel functions $\Phi (x)={[{\Phi }_1,{\Phi }_2,\cdots {\Phi }_n]}^{\mathrm{T}}$ are represented by a Gaussian radial basis function (GRBF) as:

$$\Phi (x)=\exp (\frac{{‖x-c‖}^2}{2{\sigma }^2})$$

(31)

where $x={[q_{\theta }^{\mathrm{T}},{\dot{q}}_{\theta }^{\mathrm{T}},{\dot{q}}_{\theta d}^{\mathrm{T}},{\ddot{q}}_{\theta d}^{\mathrm{T}}]}^{\mathrm{T}}$, $c$ and $\sigma$ are the variance and the centre vector of the GRBF.

On this basis, the slow rigid motion subsystem control law is given as:

$${\tau }_{x\theta }={\widehat{W}}_a^{\mathrm{T}}\Phi (x)+K_{z}z+{\tau }_a$$

(32)

where ${\widehat{W}}_a$ is the estimate of the ideal weight $W_a$. Defining the estimation error ${\tilde{W}}_a=W_a-{\widehat{W}}_a$, it satisfies ${\dot{\tilde{W}}}_a=-{\dot{\widehat{W}}}_a$. $K_z\in {\mathbf{R}}^{3\times 3}$ is a positive definite diagonal matrix. ${\tau }_a$ is a robust control law, which is defined as:

$${\tau }_a=K_{a}z/‖z‖$$

(33)

where $K_a\in {\mathbf{R}}^{3\times 3}$ is a positive definite diagonal matrix.

Substituting Equations (32) and (33) into Equation (29), we have:

$$D_{x\theta }\dot{z}=-(C_{x\theta }+K_z)z-K_{a}z/‖z‖+{\tilde{W}}_a\Phi +\varsigma (x)$$

(34)

In order to optimize ASN, reinforcement learning signals are defined by CAN:

$$r=z+‖z‖{\widehat{W}}_c^{\mathrm{T}}\Phi (x)$$

(35)

where ${\widehat{W}}_c\in {\mathbf{R}}^{m\times 3}$ is the estimate of the ideal weight $W_c$.

Assumption 1.

The ideal weights $W_a$, $W_c$are bounded and satisfy:

$$‖W_a‖\le W_{aM}$$

$$‖W_c‖\le W_{cM}$$

where$W_{aM}$, $W_{cM}$is an unknown positive constant.

Assumption 2.

The optimal approximation error $\varsigma (x)$is bounded and satisfies:

$$‖\varsigma (x)‖\le {\varsigma }_M$$

where${\varsigma }_M$is an unknown positive constant.

Next, the weight adaptive law of neural network can be further designed as:

$${\dot{\widehat{W}}}_a=K_b\Phi (x)r^{\mathrm{T}}-\eta K_b‖z‖{\widehat{W}}_a$$

(36)

$${\dot{\widehat{W}}}_c=-K_c‖z‖\Phi (x){({\widehat{W}}_a^{\mathrm{T}}\Phi (x))}^{\mathrm{T}}-\eta K_c‖z‖{\widehat{W}}_c$$

(37)

where $K_b$, $K_c$ are positive definite diagonal matrix. $\eta$ is a positive constant. Defining estimation error as ${\tilde{W}}_c=W_c-{\widehat{W}}_c$, and it satisfied ${\dot{\tilde{W}}}_c=-{\dot{\widehat{W}}}_c$.

Theorem 1.

For the dynamic equation of the slow subsystem (26) with unknown nonlinear terms, supposing that Assumptions 1 and 2 hold and adopting the weight adaptive law (36) and (37), the control law (32) based on reinforcement learning signals (35) can ensure that the trajectory tracking error$e$converges to zero asymptotically.

Proof of Theorem 1.

Introducing the Lyapunov function:

$$V=\frac{1}{2}{z}^{\mathrm{T}}{D}_{x\theta }z+\frac{1}{2}tr\left\{{\tilde{W}}_a^{\mathrm{T}}{K}_b^{-1}{\tilde{W}}_a\right\}+\frac{1}{2}tr\left\{{\tilde{W}}_c^{\mathrm{T}}{K}_c^{-1}{\tilde{W}}_c\right\}$$

(38)

Differentiating Equation (13), we have:

$$\dot{V}=\frac{1}{2}{z}^{\mathrm{T}}{\dot{D}}_{x\theta }z+z^{\mathrm{T}}{D}_{x\theta }\dot{z}-tr\left\{{\tilde{W}}_a^{\mathrm{T}}{K}_b^{-1}{\dot{\widehat{W}}}_a\right\}-tr\left\{{\tilde{W}}_c^{\mathrm{T}}{K}_c^{-1}{\dot{\widehat{W}}}_c\right\}$$

(39)

Substituting Equations (34)–(37) into Equation (39) yields:

$$\begin{gathered} \dot{V}=-z^{\mathrm{T}}{K}_{z}z+z^{\mathrm{T}}\varsigma +‖z‖tr\{-{\tilde{W}}_a^{\mathrm{T}}\Phi {({\widehat{W}}_c^{\mathrm{T}}\Phi )}^{\mathrm{T}}+\eta {\tilde{W}}_a^{\mathrm{T}}{\widehat{W}}_a+{\tilde{W}}_c^{\mathrm{T}}\Phi {({\widehat{W}}_a^{\mathrm{T}}\Phi )}^{\mathrm{T}} \\ +\eta {\tilde{W}}_c^{\mathrm{T}}{\widehat{W}}_c\}-z^{\mathrm{T}}{K}_{a}z/‖z‖ \\ \le -z^{\mathrm{T}}{K}_{z}z+z^{\mathrm{T}}\varsigma +‖z‖tr\{-{\tilde{W}}_a^{\mathrm{T}}\Phi {({\widehat{W}}_c^{\mathrm{T}}\Phi )}^{\mathrm{T}}+\eta {\tilde{W}}_a^{\mathrm{T}}{\widehat{W}}_a \\ +{\tilde{W}}_c^{\mathrm{T}}\Phi {({\widehat{W}}_a^{\mathrm{T}}\Phi )}^{\mathrm{T}}+\eta {\tilde{W}}_c^{\mathrm{T}}{\widehat{W}}_c\} \end{gathered}$$

(40)

Combining Assumption 1, we have:

$${\tilde{W}}_a^{\mathrm{T}}{\widehat{W}}_a\le ‖{\tilde{W}}_a^{}‖W_{aM}-{‖{\tilde{W}}_a^{}‖}^2$$

$${\tilde{W}}_c^{\mathrm{T}}{\widehat{W}}_c\le ‖{\tilde{W}}_c^{}‖W_{cM}-{‖{\tilde{W}}_c^{}‖}^2$$

and combing Assumption 2, Equation (40) is rewritten as:

$$\begin{gathered} \dot{V}\le -z^{\mathrm{T}}{K}_{z}z+‖z‖{\varsigma }_M+‖z‖\{(W_{aM}+W_{cM}{‖\Phi ‖}^2)‖{\tilde{W}}_a‖ \\ +(W_{cM}+W_{aM}{‖\Phi ‖}^2)‖{\tilde{W}}_c‖-\eta {‖{\tilde{W}}_a‖}^2-\eta {‖{\tilde{W}}_c‖}^2\} \end{gathered}$$

(41)

Considering the regression vector $\Phi$ is bounded, it can be set ${‖\Phi ‖}^2\le \stackrel{-}{\Phi }$. $K_{z\mathrm{m}}$ is minimum eigenvalue of $K_z$. Equation (41) satisfies:

$$\dot{V}\le -K_{z\mathrm{m}}{‖z‖}^2-\eta ‖z‖\{{(‖{\tilde{W}}_a‖-a_1)}^2+{(‖{\tilde{W}}_c‖-a_2)}^2-[a_1^2+a_2^2+\frac{{\varsigma }_M}{\eta }]\}$$

(42)

where $a_1=\frac{W_{aM}+W_{cM}\stackrel{-}{\mathsf{\Phi }}}{2\eta }$, $a_2=\frac{W_{cM}+W_{aM}\stackrel{-}{\mathsf{\Phi }}}{2\eta }$. To assure $\dot{V}\le 0$, we only require one of the following conditions:

$$‖z‖>\eta [a_1^2+a_2^2+\frac{{\varsigma }_M}{\eta }]/K_{z\mathrm{m}}$$

(43)

$$‖{\tilde{W}}_a‖>a_1+\sqrt{a_1^2+a_2^2+\frac{{\varsigma }_M}{\eta }}$$

(44)

$$‖{\tilde{W}}_c‖>a_2+\sqrt{a_1^2+a_2^2+\frac{{\varsigma }_M}{\eta }}$$

(45)

Based on the analysis results of the above steps, and combing with the Lyapunov stability theorem, which implies that the whole closed-loop system is stable the trajectory tracking error $e$ converges to zero asymptotically from the stability analysis in Theorem 1. The proof is thus completed. □

5. Simulation Results

5.1. Impact Resistance Performance Simulation in the Capture Phase

To show the performance of the proposed controller, simulations are carried out on a planar space robot with the RSEA and target spacecraft systems shown in Figure 3. The actual parameters of the system are as follows: $m_0=80\mathrm{kg}$, $m_1=5\mathrm{kg}$, $m_2=5\mathrm{kg}$, $m_{\mathrm{s}}=30\mathrm{kg}$; $I_0=30\mathrm{kg}·{\mathrm{m}}^2$, $I_1=3\mathrm{kg}\cdot {\mathrm{m}}^2$, $I_2=3\mathrm{kg}\cdot {\mathrm{m}}^2$, $I_{\mathrm{s}}=15\mathrm{kg}\cdot {\mathrm{m}}^2$, $I_{1m}=0.05\mathrm{kg}\cdot {\mathrm{m}}^2$, $I_{2m}=0.05\mathrm{kg}\cdot {\mathrm{m}}^2$; $k_{1m}=k_{2m}=1000\mathrm{N}/\mathrm{m}$; $l_0=1\mathrm{m}$, $l_1=l_2=2\mathrm{m}$, $d_1=d_2=1\mathrm{m}$.

The equivalent stiffness of joints [30] is as follows:

$$K=2K_m(3R^2+r^2)(2{\cos }^2\phi -1)$$

(46)

where $K_m=\mathrm{diag}(k_{1m},k_{2m})$, $R=0.1\mathrm{m}$, $r=0.01\mathrm{m}$. $\phi$ is the angle of sweeping arm when the force $F={[20\mathrm{N}·\mathrm{m},20\mathrm{N}·\mathrm{m},0]}^{\mathrm{T}}$ acting on the end of the space manipulator, select $\phi =\mathrm{diag}(3^{\circ },2^{\circ })$.

In order to verify impact resistance performance simulation in the capture phase, the space robot system with/without RSEA device was used to carry out acquisition simulation tests on spacecraft with different velocity. The simulation results are shown in

Table 1. RSEA impact resistance at different initial velocities of spacecraft.

Initial Velocity of Satellite/ (m/s, m/s, rad/s)	Impact Torquein Joint 1/ (N·m, N·m)	Impact Torquein Joint 2/ (N·m, N·m)	Maximum Percentage Reduction
[0.45, 0.5, 0]T	[413.6, 102.8]T	[91.2, 46.7]T	75.1%
[0, 0.5, 0.5]T	[208.2, 86.0]T	[68.2, 46.5]T	58.7%
[0.45, 0.5, 0.5]T	[472.9, 110.5]T	[91.8, 48.2]T	76.6%

.

In Table 1, the first column of velocity terms, the first two are linear velocities, and the third is angular velocities. In the second and third columns, the preceding and the following items are the impact torques of joints without and with RSEA devices respectively. The fourth column has the maximum percentage reduction in joint impact torque with the RSEA device. As can be seen from Table 1, for the capture phase of spacecraft at different initial velocities, the configuration of RSEA device can effectively reduce the impact torque acting on joints, and effectively realize the protection of the joint.

5.2. Buffer Compliance Control Performance Simulation in Stable Control Phase

To show the buffer compliance control performance of the proposed controller, simulations are carried out for stable control phase. The actual parameters of the system are as follows: K₂ = diag(5,5), Λ = diag(5,5,5), K_z = diag(400,400,400), ε = 0.5, K_a = diag(20,20,20), K_b = diag(50,50,50), η = 1, K_c = diag(10,10,10). In pre-impact phase $q_{\theta }={[{90}^{\circ },{45}^{\circ },{45}^{\circ }]}^{\mathrm{T}}$, assuming that the space robot system capturing a non-cooperative spacecraft at $t_0=0\mathrm{s}$. At this time, the velocity of the spacecraft is $v_t={[0.45\mathrm{m}/\mathrm{s},0.45\mathrm{m}/\mathrm{s},0.5\mathrm{rad}/\mathrm{s}]}^{\mathrm{T}}$, the desired trajectory of post-capture hybrid system is $q_{\theta d}={[{100}^{\circ },{30}^{\circ },{60}^{\circ }]}^{\mathrm{T}}$. Assume that when the joint actuators running, the limit of the impact torque it can bear is $90\mathrm{N}·\mathrm{m}$. In order to protect the joint actuators, the buffer compliance control strategy of active opening and closing actuators (named switching strategy) is adopted. The shutdown torque threshold is $60\mathrm{N}·\mathrm{m}$, and the startup torque threshold is $6\mathrm{N}·\mathrm{m}$. The simulation results are shown in Figure 5, Figure 6, Figure 7, Figure 8, Figure 9 and Figure 10.

Figure 5 shows the impact torque acting on the joints when not adopting the switching strategy, where it can be found that the impact torque still exceeds the safety threshold of the joint at this time. Figure 6 shows the impact torque acting on the joints when adopting a switching strategy. By comparing Figure 5 and Figure 6, it can be seen that the impact torque acting on the joints can be limited within a safe range by combining with the buffer compliance control, which ensures the protection of the joint motor during the stable control phase. Figure 7 shows the evaluation factor signal. It can be found that the ACN is optimized through the interaction with the environment and the reward signal is obtained, and finally reaches the stable state.

To show the effectiveness of the defined reinforcement signal, the tracking accuracy is quantitatively analyzed by comparison of the trajectory tracking error of the proposed RL control scheme, RL with robust controller off and neural network control strategy without reinforcement signal (turn off RL). The mean absolute error MAE = $\frac{1}{n}{\displaystyle \sum _{i=1}^n\left|e_i\right|}$ was used to evaluate the tracking accuracy, the simulation results are shown in

Table 2. The mean absolute error of trajectory tracking.

The Control Scheme	${\mathit{\theta }}_0$ (°)	${\mathit{\theta }}_1$ (°)	${\mathit{\theta }}_2$ (°)
The proposed RL	0.0015	0.0020	0.0022
Turn off robust controller	0.0476	0.2866	0.2737
Turn off RL	0.0184	0.0993	0.0949

. It can be seen in Table 2 the mean absolute error of proposed RL is smaller than the other control strategies, which shown that the proposed control method has high tracking accuracy and good tracking performance.

Figure 8, Figure 9 and Figure 10 shows the stabilization of the hybrid system when the proposed buffer compliance controller is adopted. The solid line is the trajectory tracking curve of the system when the control algorithm based on reinforcement learning is adopted, the dotted line is the trajectory tracking curve when turn off the robust term ${\tau }_a$, the double line is the trajectory tracking curve when turn off RL. By comparing them, it can be found that the unstable hybrid system finally reaches the stable and expected state, and the proposed RL control scheme has faster convergence speed and higher tracking accuracy.

If the fast subsystem controller of the system is turned off, the system trajectory tracking curve shown in Figure 11, Figure 12 and Figure 13 can be obtained. By comparing Figure 8, Figure 9 and Figure 10 with Figure 11, Figure 12 and Figure 13, it can be seen that if the fast subsystem is turned off, the elastic vibration of the unstable hybrid system will continue to increase and eventually lead to the divergence of the system. Therefore, the proposed velocity difference feedback controller can actively suppress the elastic vibration of the system joint, and then achieve stable track of the trajectory.

6. Conclusions

In this paper a space robot with a RSEA device to protect the joint of the robot under impact torque during the satellite capture process is designed. The timely opening and closing of joint actuators was proposed to achieve buffer compliance control. The dynamic model of the post-capture hybrid system was derived from the Lagrange equations, law of conservation of momentum and the constraints of kinematics and velocity. Then, based on singular perturbation theory, the hybrid system was decomposed into a slow subsystem and a fast subsystem. A buffer compliance control based on a reinforcement learning algorithm was applied to control the slow subsystem with unknown unknown nonlinear disturbances term. The fast control was designed with speed difference feedback controller. The simulation results show that the proposed strategy can reduce the impact torque by 76.6% at the maximum and 58.7% at the minimum during the capture phase, which reflects a good anti-impact performance. In the stable control phase, the impact torque acting on the joint is guaranteed to be limited within the safety threshold, so as to avoid the overload and damage of the joint actuators. In addition, the proposed reinforcement learning strategy has strong online adaptability and autonomous learning ability under complex conditions and can be continuously optimized through real-time interaction with the complex space environment, so as to ensure the accuracy and stability of the system stabilization motion.

Note that this paper only considers that space manipulators mounted on their spacecraft are rigid. For future research, the buffer compliance control of space robot with a flexible-link capturing a non-cooperative spacecraft control problem will be studied, and the control scheme extended to practical applications.