# Stereo Vision-Based Relative Position and Attitude Estimation of Non-Cooperative Spacecraft

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Status Model

#### 2.2. Measurement Model

#### 2.3. Feature Points Extraction, Tracking, and Matching

#### 2.4. Feature Point Management

## 3. Closed-Loop Simulation Experiment

#### 3.1. Simulation in Matlab

#### 3.2. Simulation in Unity

#### 3.3. Calculation in NCT Autonomous Navigation Algorithm

## 4. Experimental Results and Discussion

#### 4.1. Filter Initiation

#### 4.2. Filter Navigation Performance

## 5. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Li, W.J.; Cheng, D.Y.; Liu, X.G.; Wang, Y.B.; Shi, W.H.; Tang, Z.X.; Gao, F.; Zeng, F.M.; Chai, H.Y.; Luo, W.B.; et al. On-orbit service (OOS) of spacecraft: A review of engineering developments. Prog. Aerosp. Sci.
**2019**, 108, 32–120. [Google Scholar] [CrossRef] - Flores-Abad, A.; Ma, O.; Pham, K.; Ulrich, S. A review of space robotics technologies for on-orbit servicing. Prog. Aerosp. Sci.
**2014**, 68, 1–26. [Google Scholar] [CrossRef] [Green Version] - Wang, D.; Hou, B.; Wang, J.; Ge, D.; Li, M.; Xu, C.; Zhou, H. State estimation method for spacecraft autonomous navigation: Review. Hangkong Xuebao Acta Aeronaut. Astronaut. Sin.
**2021**, 42. [Google Scholar] - Cassinis, L.P.; Fonod, R.; Gill, E. Review of the robustness and applicability of monocular pose estimation systems for relative navigation with an uncooperative spacecraft. Prog. Aerosp. Sci.
**2019**, 110, 100548. [Google Scholar] [CrossRef] - Bordeneuve-Guibé, J.; Drouin, A.; Roos, C. Advances in Aerospace Guidance, Navigation and Control; Springer: Berlin/Heidelberg, Germany, 2015. [Google Scholar]
- Xu, W.; Liang, B.; Li, C.; Liu, Y.; Wang, X. A modelling and simulation system of space robot for capturing non-cooperative target. Math. Comput. Model. Dyn. Syst.
**2009**, 15, 371–393. [Google Scholar] [CrossRef] - Leinz, M.R.; Chen, C.-T.; Beaven, M.W.; Weismuller, T.P.; Caballero, D.L.; Gaumer, W.B.; Sabasteanski, P.W.; Scott, P.A.; Lundgren, M.A. Orbital Express Autonomous Rendezvous and Capture Sensor System (ARCSS) flight test results. In Proceedings of the Sensors and Systems for Space Applications II, Orlando, FL, USA, 17–18 April 2008; Volume 6958. [Google Scholar]
- Bodin, P.; Larsson, R.; Nilsson, F.; Chasset, C.; Noteborn, R.; Nylund, M. PRISMA: An in-orbit test bed for guidance, navigation, and control experiments. J. Spacecr. Rocket.
**2009**, 46, 615–623. [Google Scholar] [CrossRef] - Woods, J.O.; Christian, J.A. Lidar-based relative navigation with respect to non-cooperative objects. Acta Astronaut.
**2016**, 126, 298–311. [Google Scholar] [CrossRef] - Qiu, Y.; Guo, B.; Li, C.; Liang, B. Study on Federal Filter of Relative Navigation for Non-cooperative Spacecraft. J. Astronaut.
**2009**, 30, 2206–2212. [Google Scholar] - Aghili, F.; Su, C.-Y. Robust Relative Navigation by Integration of ICP and Adaptive Kalman Filter Using Laser Scanner and IMU. IEEE/ASME Trans. Mechatron.
**2016**, 21, 2015–2026. [Google Scholar] [CrossRef] - Augenstein, S.; Rock, S.M. Improved frame-to-frame pose tracking during vision-only SLAM/SFM with a tumbling target. In Proceedings of the IEEE International Conference on Robotics and Automation, Shanghai, China, 9–13 May 2011; pp. 3131–3138. [Google Scholar]
- Ruel, S.; Luu, T.; Berube, A. Space Shuttle Testing of the TriDAR 3D Rendezvous and Docking Sensor. J. Field Robot.
**2012**, 29, 535–553. [Google Scholar] [CrossRef] - Lichter, M.D. Shape, Motion, and Inertial Parameter Estimation of Space Objects using Teams of Cooperative Vision Sensors; Massachusetts Institute of Technology: Cambridge, MA, USA, 2005. [Google Scholar]
- Setterfield, T.P.; Miller, D.W.; Leonard, J.J.; Saenz-Otero, A. Mapping and determining the center of mass of a rotating object using a moving observer. Int. J. Robot. Res.
**2018**, 37, 83–103. [Google Scholar] [CrossRef] - Tweddle, B.E.; Setterfield, T.P.; Saenz-Otero, A.; Miller, D.W.; Leonard, J.J. Experimental Evaluation of On-board, Visual Mapping of an Object Spinning in Micro-Gravity aboard the International Space Station. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Chicago, IL, USA, 14–18 September 2014; pp. 2333–2340. [Google Scholar]
- Lyzhoft, J.; Wie, B. New image processing algorithm for terminal guidance of multiple kinetic-energy impactors for disrupting hazardous asteroids. Astrodynamics
**2019**, 3, 45–59. [Google Scholar] [CrossRef] - Gong, B.; Li, W.; Li, S.; Ma, W.; Zheng, L. Angles-only initial relative orbit determination algorithm for non-cooperative spacecraft proximity operations. Astrodynamics
**2018**, 2, 217–231. [Google Scholar] [CrossRef] - Determination, A.; Target, C. A New Method of Relative Position and Attitude Determination for Non-Cooperative Target. J. Astronaut.
**2011**, 32, 516–521. [Google Scholar] - Segal, S.; Carmi, A.; Gurfil, P. Stereovision-based estimation of relative dynamics between noncooperative satellites: Theory and experiments. IEEE Trans. Control. Syst. Technol.
**2014**, 22, 568–584. [Google Scholar] [CrossRef] - Pesce, V.; Lavagna, M.; Bevilacqua, R. Stereovision-based pose and inertia estimation of unknown and uncooperative space objects. Adv. Sp. Res.
**2017**, 59, 236–251. [Google Scholar] [CrossRef] - Feng, Q.; Zhu, Z.H.; Pan, Q.; Hou, X. Relative State and Inertia Estimation of Unknown Tumbling Spacecraft by Stereo Vision. IEEE Access
**2018**, 6, 54126–54138. [Google Scholar] [CrossRef] - K, S.-G.; Crassidis, J.L.; Cheng, Y.; Fosbury, A.M.; Junkins, J.L. Kalman Filtering for Relative Spacecraft Attitude and Position Estimation. J. Guid. Control Dyn.
**2011**, 32, 516–521. [Google Scholar] - Watanabe, Y.; Johnson, E.N.; Calise, A.J. Optimal 3-D guidance from a 2-D vision sensor. In Proceedings of the Collection of Technical Papers—AIAA Guidance, Navigation, and Control Conference, Providence, RI, USA, 16–19 August 2004; Volume 1. [Google Scholar]
- Julier, S.J.; Uhlmann, J.K. New Extension of the Kalman Filter to Nonlinear Systems. In Proceedings of the Signal Processing, Sensor Fusion, and Target Recognition VI, Orlando, FL, USA, 21–24 April 1997; Volume 3068. [Google Scholar]
- Feng, Q.; Liu, Y.; Zhu, Z.H.; Hu, Y.; Pan, Q.; Lyu, Y. Vision-Based Relative State Estimation for A Non-Cooperative Target. In Proceedings of the 2018 AIAA Guidance, Navigation, and Control Conference, Kissimmee, FL, USA, 8–12 January 2018. [Google Scholar]
- Wahba, G. A least squares estimate of satellite attitude. SIAM Rev.
**1965**, 7, 409. [Google Scholar] [CrossRef] - Markley, F.L. Fast quaternion attitude estimation from two vector measurements. J. Guid. Control Dyn.
**2002**, 25, 411–414. [Google Scholar] [CrossRef] [Green Version] - Viswanathan, D. Features from Accelerated Segment Test (FAST). In Proceedings of the 10th workshops on Image Analysis for Multimedia Interactive Services, London, UK, 6–9 May 2009. [Google Scholar]
- Song, X.; Yang, L.; Wu, Y.; Liu, Z. A New Depth Measuring Method for Stereo Camera Based on Converted Relative Extrinsic Parameters. In Proceedings of the International Symposium on Photoelectronic Detection and Imaging 2013: Imaging Sensors and Applications, Beijing, China, 25–27 June 2013; Volume 8908. [Google Scholar]
- Karthika Pragadeeswari, C.; Yamuna, G.; Yasmin Beham, G. A robust algorithm for real time tracking with optical flow. Int. J. Innov. Technol. Explor. Eng.
**2019**, 9, 887–892. [Google Scholar] [CrossRef] - Turner, A.J. An Open-Source Extensible Spacecraft Simulation and Modeling Environment Framework. Master’s Thesis, Virginia Polytechnic Institute and State University, Blacksburg, VA, USA, 2003. [Google Scholar]

**Figure 9.**Configuration of hardware-in-the-loop (HIL) simulation platform. NCT autonomous navigation algorithm is deployed on a pose calculation computer, i.e., NVIDIA JetsonTX2 (NVIDIA Corporation, Inc., Santa Clara, CA, USA) module.

**Figure 13.**Convergence binocular images simulated from Unity (

**left**: from the

**left**camera,

**right**: from the

**right**camera).

**Figure 15.**Feature management. The green points are feature points that have been detected in history and can be tracked in this frame. The red points are newly detected feature points and added to the feature point sequence.

Parameter | Left Camera | Right Camera |
---|---|---|

Internal parameters (The first two rows take the unit of pixel, and the third row has no units) | $\left[\begin{array}{ccc}2.0298\times {10}^{3}& 0.0000& 6.4000\times {10}^{2}\\ 0.0000& 2.0298\times {10}^{3}& 6.4000\times {10}^{2}\\ 0.0000& 0.0000& 1.0000\end{array}\right]$ | $\left[\begin{array}{ccc}2.0298\times {10}^{3}& 0.0000& 6.4000\times {10}^{2}\\ 0.0000& 2.0298\times {10}^{3}& 6.4000\times {10}^{2}\\ 0.0000& 0.0000& 1.0000\end{array}\right]$ |

Translation vector (mm) | [0.0000 0.0000 0.0000] | [920.00 0.0000 0.0000] |

State | Statistic | Result |
---|---|---|

Attitude (deg) | Mean | [0.0238 8.2274 × 10^{−4} 0.0094] |

Standard deviation | [0.4482 0.5480 0.5298] | |

Angular velocity (deg/s) | Mean Standard deviation | [0.0222 0.0353 0.0030] [0.5400 0.6336 0.2833] |

Centroid position (m) | Mean Standard deviation | [0.0033 0.0032 0.0090] [0.1146 0.1157 0.1161] |

Centroid position velocity (m/s) | Mean | [0.5471 × 10^{−4} 9.6595 × 10^{−4} 4.2910 × 10^{−4}] |

Standard deviation | [0.0298 0.0308 0.0309] |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Chang, L.; Liu, J.; Chen, Z.; Bai, J.; Shu, L.
Stereo Vision-Based Relative Position and Attitude Estimation of Non-Cooperative Spacecraft. *Aerospace* **2021**, *8*, 230.
https://doi.org/10.3390/aerospace8080230

**AMA Style**

Chang L, Liu J, Chen Z, Bai J, Shu L.
Stereo Vision-Based Relative Position and Attitude Estimation of Non-Cooperative Spacecraft. *Aerospace*. 2021; 8(8):230.
https://doi.org/10.3390/aerospace8080230

**Chicago/Turabian Style**

Chang, Liang, Jixiu Liu, Zui Chen, Jie Bai, and Leizheng Shu.
2021. "Stereo Vision-Based Relative Position and Attitude Estimation of Non-Cooperative Spacecraft" *Aerospace* 8, no. 8: 230.
https://doi.org/10.3390/aerospace8080230