# Buffer Compliance Control of Space Robots Capturing a Non-Cooperative Spacecraft Based on Reinforcement Learning

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Buffer Compliance Strategy

## 3. Dynamics Modeling and Impact Effect Analysis

_{0}, rigid links B

_{i}(i = 1,2), and rigid target spacecraft B

_{3}. We build the inertial coordinate system XOY, while at the same time, the local coordinate system x

_{i}O

_{i}y

_{i}(i = 0,1,2) of each component B

_{i}(i = 1,2) is established; O

_{0}is the rotation center of the base, O

_{i}is the rotation center of B

_{i}(i = 1,2); m

_{0}is the mass of the base, m

_{s}is the mass of the non-cooperative spacecraft, m

_{i}is the mass of B

_{i}(i = 1,2). I

_{0}is inertial moment of the base with respect to its mass center, I

_{s}is the inertial moment of the non-cooperative spacecraft with respect to its mass center, I

_{i}(i = 1,2) is the inertial moment of B

_{i}(i = 1,2) with respect to their mass center. I

_{0}represents the distance from point O

_{0}to O

_{1}, l

_{i}(i = 1,2) represents length of B

_{i}along the x

_{i}axis. d

_{i}(i = 1,2) is the distance from the mass center of B

_{i}to O

_{i}. I

_{im}(i = 1,2) is inertial moment of the i-h actuator. k

_{im}(i = 1,2) is the spring stiffness of the RSEA device.

**r**

_{c}is the position vector of the mass center of the entire system in inertial coordinate system (XOY).

**r**

_{i}(i = 1,2) is position vector of the mass center of B

_{i}in the inertial coordinate system (XOY).

_{i}axis in the ${x}_{i}{O}_{i}{y}_{i}$ frame.

## 4. Two-Time Scale Control

#### 4.1. Fast Subsystem and the Corresponding Controller

#### 4.2. Slow Subsystem and the Corresponding Controller

**Assumption**

**1.**

**Assumption**

**2.**

**Theorem**

**1.**

**Proof**

**of**

**Theorem**

**1.**

## 5. Simulation Results

#### 5.1. Impact Resistance Performance Simulation in the Capture Phase

#### 5.2. Buffer Compliance Control Performance Simulation in Stable Control Phase

**K**

_{2}= diag(5,5),

**Λ**= diag(5,5,5),

**K**

_{z}= diag(400,400,400), ε = 0.5,

**K**

_{a}= diag(20,20,20),

**K**

_{b}= diag(50,50,50), η = 1,

**K**

_{c}= diag(10,10,10). In pre-impact phase ${q}_{\theta}={[{90}^{\circ},{45}^{\circ},{45}^{\circ}]}^{\mathrm{T}}$, assuming that the space robot system capturing a non-cooperative spacecraft at ${t}_{0}=0\text{}\mathrm{s}$. At this time, the velocity of the spacecraft is ${v}_{t}={[0.45\text{}\mathrm{m}/\mathrm{s},0.45\text{}\mathrm{m}/\mathrm{s},0.5\text{}\mathrm{rad}/\mathrm{s}]}^{\mathrm{T}}$, the desired trajectory of post-capture hybrid system is ${q}_{\theta d}={[{100}^{\circ},{30}^{\circ},{60}^{\circ}]}^{\mathrm{T}}$. Assume that when the joint actuators running, the limit of the impact torque it can bear is $90\text{}\mathrm{N}\xb7\mathrm{m}$. In order to protect the joint actuators, the buffer compliance control strategy of active opening and closing actuators (named switching strategy) is adopted. The shutdown torque threshold is $60\text{}\mathrm{N}\xb7\mathrm{m}$, and the startup torque threshold is $6\text{}\mathrm{N}\xb7\mathrm{m}$. The simulation results are shown in Figure 5, Figure 6, Figure 7, Figure 8, Figure 9 and Figure 10.

## 6. Conclusions

## Author Contributions

## Funding

## Informed Consent Statement

## Conflicts of Interest

## References

- Flores-Abad, A.; Ma, O.; Pham, K.; Ulrich, S. A review of space robotics technologies for on-orbit servicing. Prog. Aerosp. Sci.
**2014**, 68, 1–26. [Google Scholar] [CrossRef] [Green Version] - Yu, X.; Chen, L. Observer-based two-time scale robust control of free-flying flexible-joint space manipulators with external disturbances. Robotica
**2017**, 35, 2201–2217. [Google Scholar] [CrossRef] - Meng, Q.L.; Liang, J.X.; Ma, O. Identification of all the inertial parameters of a non-cooperative object in orbit. Aerosp. Sci. Technol.
**2019**, 91, 571–582. [Google Scholar] [CrossRef] - Boning, P.; Dubosky, S. A kinematic approach to determining the optimal actuator sensor architecture for space robots. Int. J. Robot. Res.
**2011**, 30, 1194–1204. [Google Scholar] [CrossRef] [Green Version] - Giordano, A.M.; Ott, C.; Albu, A. Coordinated control of spacecraft’s attitude and end-effector for space robots. IEEE Robot. Autom. Lett.
**2019**, 4, 2108–2115. [Google Scholar] [CrossRef] - Qin, L.; Liu, F.C.; Liang, L.H.; Gao, J.F. Fuzzy adaptive robust control for space robot considering the effect of the gravity. Chin. J. Aeronaut.
**2014**, 27, 1562–1570. [Google Scholar] [CrossRef] [Green Version] - Virgili-Llop, J.; Zagaris, C.; Zappulla, I.R.; Bradstreet, A.; Romano, M. A convex-programming-based guidance algorithm to capture a tumbling object on orbit using a spacecraft equipped with a robotic manipulator. Int. J. Robot. Res.
**2019**, 38, 40–72. [Google Scholar] [CrossRef] - Ai, H.P.; Chen, L. Force/position fuzzy control of space robot capturing spacecraft by dual-arm clamping. J. Harbin Eng. Univ.
**2020**, 41, 1847–1853. [Google Scholar] - Liu, X.; Li, H.; Wang, J.; Cai, G. Dynamics analysis of flexible space robot with joint friction. Aerosp. Sci. Technol.
**2015**, 47, 164–176. [Google Scholar] [CrossRef] - Lim, J.; Chung, J. Dynamic analysis of a tethered satellite system for space debris capture. Nonlinear Dyn.
**2018**, 94, 1–18. [Google Scholar] [CrossRef] - Shah, S.V.; Sharf, I.; Misra, A. Reactionless path planning strategies for capture of tumbling objects in space using a dual-arm robotic system. In Proceedings of the AIAA Guidance, Navigation, and Control Conference, Boston, MA, USA, 15 August 2013; p. 4521. [Google Scholar]
- Uyama, N.; Hirano, D.; Nakanishi, H.; Nagaoka, K.; Yoshida, K. Impedance-based contact control of a free-flying space robot with respect to coefficient of restitution. In Proceedings of the IEEE/SICE International Symposium on System Integration, Kyoto, Japan, 9 February 2012; pp. 1196–1201. [Google Scholar]
- Jing, C.; Li, C. Mechanical analysis and calm control of dual-arm space robot for capturing a satellite. Chin. J. Theor. Appl. Mech.
**2016**, 48, 832–842. (In Chinese) [Google Scholar] - Jiang, B.; Hu, Q.; Friswell, M.I. Fixed-time attitude control for rigid spacecraft with actuator saturation and faults. IEEE Trans. Control Syst. Technol.
**2016**, 24, 1892–1898. [Google Scholar] [CrossRef] - Liu, S.P.; Wu, L.C.; Lu, Z. Impact dynamics and control of a flexible dual-arm space robot capturing an object. Appl. Math. Comput.
**2007**, 185, 1149–1159. [Google Scholar] [CrossRef] - Walker, M.W.; Wee, L.-B. Adaptive control of space-based robot manipulators. IEEE Trans. Robot. Autom.
**1991**, 7, 828–835. [Google Scholar] [CrossRef] - Yi, Z.G.; Ge, X.S. Attitude motion trajectory tracking for underactuated spacecraft based on indirect legendre pesudospectral method. J. Astronaut.
**2018**, 39, 648–655. (In Chinese) [Google Scholar] - Sands, T. Optimization Provenance of Whiplash Compensation for Flexible Space Robotics. Aerospace
**2019**, 6, 93. [Google Scholar] [CrossRef] [Green Version] - Stolfi, A.; Gasbarri, P.; Sabatini, M. A combined impedance-PD approach for controlling a dual-arm space manipulator in the capture of a non-cooperative target. Acta Astronaut.
**2017**, 139, 243–253. [Google Scholar] [CrossRef] - Zhang, H.W.; Zhu, Z.X. Sampling-Based Motion Planning for Free-Floating Space Robot without Inverse Kinematics. Appl. Sci.
**2020**, 10, 9137. [Google Scholar] [CrossRef] - Cocuzza, S.; Pretto, I.; Debei, S. Least-Squares-Based Reaction Control of Space Manipulators. J. Guid. Control Dyn.
**2012**, 35, 976–986. [Google Scholar] [CrossRef] - Du, H.; Li, S.; Qian, C. Finite-Time Attitude Tracking Control of Spacecraft with Application to Attitude Synchronization. IEEE Trans. Autom. Control
**2011**, 56, 2711–2717. [Google Scholar] [CrossRef] - Aghili, F. A prediction and motion-planning scheme for visually guided robotic capturing of free-floating tumbling objects with uncertain dynamics. IEEE Trans. Robot.
**2012**, 28, 634–649. [Google Scholar] [CrossRef] - Yu, X. Hybrid-Trajectory Based Terminal Sliding Mode Control of a Flexible Space Manipulator with an Elastic Base. Robotica
**2019**, 38, 550–563. [Google Scholar] [CrossRef] - Cheng, J.; Chen, L. Elm neural network control of attitude management and auxiliary docking maneuver after dual-arm space robot capturing spacecraft. Robotica
**2017**, 39, 724–732. (In Chinese) [Google Scholar] - Wang, M.; Luo, J.; Yuan, J.; Walter, U. An integrated control scheme for space robot after capturing non-cooperative target. Acta Astronaut.
**2018**, 147, 350–363. [Google Scholar] [CrossRef] - Zhang, B.; Liang, B.; Wang, Z.; Mi, Y.; Zhang, Y.; Chen, Z. Coordinated stabilization for space robot after capturing a noncooperative target with large inertia. Acta Astronaut.
**2017**, 134, 75–84. [Google Scholar] [CrossRef] - Wu, S.; Mou, F.; Liu, Q.; Cheng, J. Contact dynamics and control of a space robot capturing a tumbling object. Acta Astronaut.
**2018**, 151, 532–542. [Google Scholar] [CrossRef] - Rekleitis, G.; Papadopoulos, E. On-orbit cooperating space robotic servicers handling a passive object. IEEE Trans. Aerosp. Electron. Syst.
**2015**, 51, 802–814. [Google Scholar] [CrossRef] - Gu, X.; Wang, K.; Cheng, T.; Zhang, X. Mechanical design of a 3-DOF humanoid soft arm based on modularized series elastic actuator. In Proceedings of the IEEE International Conference on Mechatronics and Automation, Beijing, China, 3 September 2015; pp. 1127–1131. [Google Scholar]
- Calanca, A.; Fiorini, P. Understanding environment-adaptive force control of series elastic actuators. IEEE ASME Trans. Mechatron.
**2018**, 23, 413–423. [Google Scholar] [CrossRef] - Wang, M.; Sun, L.; Yin, W.; Dong, S.; Liu, J. Nonlinear disturbance observer based torque control for series elastic actuator. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Daejeon, Korea, 1 December 2016; pp. 286–291. [Google Scholar]
- Liu, S.; Wu, S.; Liu, Y.; Wu, Z.G.; Mao, Z.M. Autonomous reinforcement learning control for space robot to capture non-cooperative targets. Sci. Sin. Phys. Mech. Astron.
**2019**, 49, 113–122. [Google Scholar] [CrossRef] - Sands, T. Development of Deterministic Artificial Intelligence for Unmanned Underwater Vehicles (UUV). J. Mar. Sci. Eng.
**2020**, 8, 578. [Google Scholar] [CrossRef] - Tang, L.; Liu, Y.J. Adaptive neural network control of robot manipulator using reinforcement learning. J. Vib. Control
**2014**, 20, 2162–2171. [Google Scholar] [CrossRef] - Cui, R.; Yang, C.; Li, Y.; Sharma, S. Adaptive Neural Network Control of AUVs With Control Input Nonlinearities Using Reinforcement Learning. IEEE Trans. Syst. Man Cybern. Syst.
**2017**, 47, 1019–1029. [Google Scholar] [CrossRef] [Green Version]

**Figure 2.**Structure of the proposed rotary series elastic actuator. (

**a**) Planar model. (

**b**) Graphic model.

Initial Velocity of Satellite/ (m/s, m/s, rad/s) | Impact Torquein Joint 1/ (N·m, N·m) | Impact Torquein Joint 2/ (N·m, N·m) | Maximum Percentage Reduction |
---|---|---|---|

[0.45, 0.5, 0]^{T} | [413.6, 102.8]^{T} | [91.2, 46.7]^{T} | 75.1% |

[0, 0.5, 0.5]^{T} | [208.2, 86.0]^{T} | [68.2, 46.5]^{T} | 58.7% |

[0.45, 0.5, 0.5]^{T} | [472.9, 110.5]^{T} | [91.8, 48.2]^{T} | 76.6% |

The Control Scheme | ${\mathit{\theta}}_{0}$ (°) | ${\mathit{\theta}}_{1}$ (°) | ${\mathit{\theta}}_{2}$ (°) |
---|---|---|---|

The proposed RL | 0.0015 | 0.0020 | 0.0022 |

Turn off robust controller | 0.0476 | 0.2866 | 0.2737 |

Turn off RL | 0.0184 | 0.0993 | 0.0949 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Ai, H.; Zhu, A.; Wang, J.; Yu, X.; Chen, L.
Buffer Compliance Control of Space Robots Capturing a Non-Cooperative Spacecraft Based on Reinforcement Learning. *Appl. Sci.* **2021**, *11*, 5783.
https://doi.org/10.3390/app11135783

**AMA Style**

Ai H, Zhu A, Wang J, Yu X, Chen L.
Buffer Compliance Control of Space Robots Capturing a Non-Cooperative Spacecraft Based on Reinforcement Learning. *Applied Sciences*. 2021; 11(13):5783.
https://doi.org/10.3390/app11135783

**Chicago/Turabian Style**

Ai, Haiping, An Zhu, Jiajia Wang, Xiaoyan Yu, and Li Chen.
2021. "Buffer Compliance Control of Space Robots Capturing a Non-Cooperative Spacecraft Based on Reinforcement Learning" *Applied Sciences* 11, no. 13: 5783.
https://doi.org/10.3390/app11135783