Motion and Inertia Estimation for Non-Cooperative Space Objects During Long-Term Occlusion Based on UKF-GP

Abstract

This study addresses the motion and inertia parameter estimation problem of a torque-free, tumbling, non-cooperative space object (target) under long-term occlusions. To solve this problem, we employ a data-driven Gaussian process (GP) to simulate sensor measurements. In particular, we implement the multi-output GP to predict the projection measurements of a stereo-camera system onboard a chaser spacecraft. A product kernel, consisting of two periodic kernels, is used in the GP models to capture the periodic trends from non-periodic projection data. The initial guesses for the periodicity hyper-parameters of the GP models are intelligently derived from fast Fourier transform (FFT) analysis of the projection data. Additionally, we propose an unscented Kalman filter–Gaussian process (UKF-GP) fusion algorithm for target motion and inertia parameter estimation. The predicted projections from the GP models and their derivatives are used as the pseudo-measurements for UKF-GP during long-term occlusion. Results from Monte Carlo (MC) simulations demonstrate that, for varying tumbling frequencies, the UKF-GP can accurately estimate the target’s motion variables over hundreds of seconds, a capability the conventional UKF algorithm lacks.

1. Introduction

In recent years, developing autonomous strategies for proximity operations between two spacecraft, such as docking, refueling, repairing, or robotic capture of space debris, has gained tremendous interest due to economic incentives, societal needs, and technological advancement. Such operations typically require physical interactions between two spacecraft. Consequently, it is important to have good knowledge of the motion parameters of the spacecraft to create a safe policy for the approach and interaction segments of the operation. Proximity operations between two cooperative spacecraft can be executed relatively easily due to both spacecraft being able to relay information regarding their motions. However, in proximity operations like space debris removal, the non-cooperative target space object cannot relay any information. It is essential to understand the target’s motion parameters to design safe policies for approach and interaction. Additionally, the target’s physical properties may be unknown and require determination.

Sensors such as the laser camera system (LCS) and mono and stereo-vision camera systems can be utilized to collect data regarding the target. The LCS sensor is less prone to sensor occlusion and has a higher measurement sampling frequency [1]. However, vision-based camera systems are more energy-efficient. This study is conducted with the assumption that the chaser carries a stereo-vision camera system. The stereo images captured by the cameras can be processed through computer vision algorithms, which return information regarding the features of the target. In general, features can either be regions [2], curves or lines [3], or points [4], which have higher visibility than their surrounding areas and, thus, are easily distinguishable in the captured images. In the context of the considered satellite proximity operations, the features indicate salient points located on the surface of the target, such as corners of solar panels and endpoints of the poles on antennas.

To obtain reliable estimates of the motion and physical parameters of a target space object from the sensor measurements, researchers have primarily relied on physics-based methods, which utilize the underlying dynamics equations associated with the target’s motion. Two of the widely utilized algorithms are the extended Kalman filter (EKF) and the unscented Kalman filter (UKF), where the latter does not require the linearization of the non-linear dynamics and observation models. Segal et al. conducted one of the earliest investigations on the estimation of the relative rotational and translational motion of a target object using the iterated EKF (IEKF) using the measurements of a fixed number of point features from a stereo-camera system [5,6]. Multiple IEKF models are run in parallel with a different choice for the inertia parameters, and the best inertia tensor was chosen from a maximum a posteriori (MAP) estimation. Later, Pesce et al. proposed an IEKF-based algorithm where the inertia parameter ratios are estimated without the MAP estimation [7]. They also utilized the optical flow or the rate of projection as measurements, which reduces the number of point features to five for the IEKF model to converge. A similar approach is investigated by Chang et al. in [8], except they considered a varying number of point features. Ge et al. showed in [9] that it is possible to achieve convergence of an EFK model to estimate the target motion from the stereo-camera measurements of only three-point features. Feng et al. developed an EFK model to estimate target motion where two dependent variables parameterize the inertia ratios that fully encode the physical constraints between the inertia parameters [10]. Li et al. proposed an interacting multiple model unscented Kalman filter (IMM-UKF) to estimate the relative motion of a target to deal with uncertainties caused by unknown noise [11]. Mazzucato et al. validated an EKF model for estimating the motion of a target both numerically and experimentally in [12], whereas De Jongh et al. experimentally validated the estimation capability of a SLAM-based EKF algorithm tasked with estimating the relative motion, shape, and size of a target object [13]. Jiang et al. developed a constrained EFK algorithm for estimating the relative motion of a target that uses physical constraints among point features of the target to ensure that the system is observable [14]. A similar geometric constraint among point features was considered by Jixiu et al. for an EKF algorithm to estimate a target object’s relative motion [15]. In [16], Maestrini et al. proposed an algorithm called the coarse model-based relative navigation (CoMBiNa) to create a 3D model and estimate the motion as well as inertia parameters of an unknown and non-cooperative target object. A coarse point-cloud model of the object using ORB-SLAM from stereo-camera measurements was later used as a reference for the UKF model to estimate the rotational and translational motion and the inertia parameters of the target. A similar problem was also addressed in [17] with the consideration of the unscented Kalman filter-based technique and the stereo-camera sensor. Apart from Kalman filter-related techniques, several other physics model-based approaches have been studied for estimating the target motion and inertia parameters, which include the non-linear least square technique [18], the smoothing and mapping factor-graph-based method [19], the constrained convex optimization method [20], etc.

Physics model-based approaches provide a way to estimate motion and physical parameters that are consistent with the underlying physics models. However, these approaches come with some major drawbacks. One of the key issues is the inability of these algorithms to provide reliable estimates of the target states if the sensor measurements are not available for a long time (i.e., hundreds or thousands of seconds). In space, long-term occlusions can happen for numerous reasons, especially for visual sensors. Examples include when the target is between the sun and the chaser or when the target is in the Earth’s shadow while orbiting the Earth. In both of these scenarios, the chaser is unable to observe the target using a visual camera sensor.

With the advancements in machine learning (ML) techniques and computational capabilities, researchers have started investigating the capability of model-free ML techniques. In recent work, Yu et al. implemented the sparse pseudo-input Gaussian process (SPGP) technique to predict the inertial position of a point of interest and a body frame attitude quaternion of a tumbling space object for ten seconds [21]. This method uses a simple RBF kernel, which is not capable of capturing the important trends of the training datasets and, consequently, is not appropriate for long-term occlusion. Researchers have also investigated combining ML and physics model-based approaches to improve parameter estimation performance. An example is the development of a dual-vector quaternion-based mixed artificial neural network (DVQ-MANN) estimation algorithm to assist a dual-vector quaternion-based extended Kalman filter (DVQ-EKF) during sensor failure [22]. The authors demonstrated that this approach helped prevent the associated error of the state variables of the DVQ-EKF from diverging. However, the presented results do not demonstrate if the algorithm is capable of further reducing the estimation errors during occlusions. Additionally, Barbier combined three Gaussian process (GP) models with a two-stage cascading EKF system to estimate the relative motion and create a map of a target object in [23]. After executing the first EKF model, which estimated the relative motion of the chaser, GP models were generated to predict the angular velocity of the target to propagate the second EKF model. However, this study did not consider any occlusion and, hence, did not demonstrate how the estimation algorithm may perform in the absence of sensor measurement.

One of the goals of this study is to predict the projection measurements of a stereo-camera system during occlusion with the help of an ML algorithm, and we select the Gaussian process (GP) for this task. The GP is a non-parametric algorithm defined by a mean function and a covariance or kernel function. For this study, we are interested in an advanced GP regression algorithm called the multi-output GP. The traditional single-output GP model can only learn the correlations among the inputs, but the multi-output GP can capture the correlations among the inputs and the correlations among the outputs. Due to this capability, the multi-output GP model can provide more accurate predictions of a vector-valued function with correlated components from smaller training data sets than single-output GP models. However, the multi-output GP models require longer training time due to the additional task of finding the covariances among the outputs. The projections of the point features on the stereo-camera image planes are functions of the correlated components of the feature positions relative to the cameras, which makes the projection components correlated as well. Therefore, the multi-output GP model is chosen as the data-driven method to predict the projections in this study. On the other hand, GP has been shown to be more accurate at providing predictions compared to neural networks (NNs) if trained from a training dataset of the same size [24], which is another reason for favoring GP in our work.

The primary contribution of this study is the development of a UKF-GP fusion algorithm to estimate the rotational motion variables and the inertia parameters of a non-cooperative, tumbling target space object under long-term occlusion. The following contributions are made in the development of the UKF-GP algorithm:

• We utilize multi-output GP models to predict projection measurements from a stereo-camera system so that the UKF-GP model can be updated during occlusion. GP models are trained using the projection measurements provided by stereo-camera systems. A product kernel, consisting of two periodic kernels, is designed to capture periodic trends in the non-periodic and noisy training data.
• The initial guesses for the periodicity hyper-parameters are intelligently obtained from the FFT analysis of the training data, enhancing the hyper-parameter training procedure.
• The projection predictions from the GP models and their derivatives obtained from the finite difference method (FDM) are used as pseudo-measurements in the UKF-GP fusion algorithm during long-term occlusion.
• Monte Carlo simulations across multiple tumbling frequencies demonstrate that GP models accurately predict projections for thousands of seconds under occlusion. Furthermore, the UKF-GP algorithm outperforms the conventional UFK in estimating motion and inertia parameters.

Note that an algorithm called the GP-UKF is proposed in [25] that also combines UKF and GP to improve the estimation accuracy. In that work, the GP models are used to replace the propagation and observation models of the UKF model. Differently, we use GPs to predict the stereo-camera measurements in this work. Additionally, we remark that the predictions from the GP models can be used as pseudo-measurements for any suitable physics-based estimation algorithm.

The paper is structured as follows: in Section 2, we provide descriptions of the problem scenario, dynamics and observation models, a summary of the UKF algorithm, multi-output GP models, overall UKF-GP frameworks, and the simulation workflow. In Section 3, we present the results from MC simulations to demonstrate the effectiveness of GP models and the UKF-GP algorithm. Finally, in Section 4, we provide our concluding remarks.

2. Methodology

In this section, we provide the proposed UKF-GP framework in detail. First, we present the problem scenario, followed by descriptions of the dynamics and observation models for the UKF algorithm. Then, we present a procedure to create GP models that are used for predicting projection measurements. Finally, we discuss the overall UKF-GP framework and the considered simulation workflow for this study.

2.1. Problem Scenario

In this paper, we aim to estimate the relative rotational motion of a target space object with respect to a chaser spacecraft. Hence, we assume that both the chaser and the target center of mass (CM) are inertially fixed, and the relative position of the target CM with respect to the chaser CM is exactly known. Also, the chaser is presumed to be rotationally fixed. The overall scenario is illustrated in Figure 1. An inertial frame $\mathcal{I}$ is defined with the origin being located at the chaser CM. Additionally, the $\mathcal{B}$ frame is the principal axe’s body frame of the target, which has the origin at the target CM. The attitude quaternion of $\mathcal{B}$ with respect to $\mathcal{I}$ is $\mathbf{q}={[q_s,q_x,q_y,q_z]}^T$. The corresponding rotation matrix of $\mathbf{q}$ is defined as $\mathbf{C}\left(\mathbf{q}\right)={\mathbf{C}}_{\mathcal{I}}^{\mathcal{B}}$, such that ${}^{\mathcal{B}}\mathit{\nu }={\mathbf{C}}_{\mathcal{I}}^{\mathcal{B}}·{}^{\mathcal{I}}\mathit{\nu }$, where ${}^{\mathcal{I}}\mathit{\nu }$ and ${}^{\mathcal{B}}\mathit{\nu }$ indicate an arbitrary vector expressed in $\mathcal{I}$ and $\mathcal{B}$. The angular velocity associated with $\dot{\mathbf{q}}$ is denoted by ${}^{\mathcal{I}}\mathit{\omega }$ and ${}^{\mathcal{B}}\mathit{\omega }$ in $\mathcal{I}$ and $\mathcal{B}$, respectively. The relative position of the target CM with respect to the chaser CM in $\mathcal{I}$ is ${}^{\mathcal{I}}\mathit{\rho }_0={[x_0,y_0,z_0]}^T$, which is known, as mentioned earlier. The position of the i-th point feature with respect to the target CM in $\mathcal{B}$ and the chaser CM in $\mathcal{I}$ are defined as ${}^{\mathcal{B}}\mathbf{p}_i$ and ${}^{\mathcal{I}}\mathit{\rho }_i$. The relationship between ${}^{\mathcal{B}}\mathbf{p}_i$ and ${}^{\mathcal{I}}\mathit{\rho }_i$ is given as follows:

$${}^{\mathcal{I}}\mathit{\rho }_i={}^{\mathcal{I}}\mathit{\rho }_0+{\left[{\mathbf{C}}_{\mathcal{I}}^{\mathcal{B}}\right]}^T{·}^{\mathcal{B}}{\mathbf{p}}_i.$$

(1)

2.2. Unscented Kalman Filter (UKF)

Since we do not aim to modify the conventional UKF algorithm in this paper, we present the dynamics and observation models of the UKF model in this section while skipping the overall UKF algorithm. There exist numerous books and papers that discuss the theoretical derivation and the practical implementation of the UKF algorithm, such as [26,27], which the reader can consult to get an in-depth understanding of the UKF algorithm.

2.2.1. Dynamics Model

This study focuses on estimating the relative rotational motion and inertia parameters of the target space object; hence, the state variable $\mathbf{X}$ is defined as follows:

$$\mathbf{X}={[{{}^{\mathcal{B}}\mathit{\omega }}^T,{\mathbf{q}}^T,k_1,k_2,{\left\{{}^{\mathcal{B}}\mathbf{p}_i^T\right\}}_{i=1}^{n_f}]}^T,$$

(2)

where $n_f$ is a constant that indicates the number of features. Also, $k_1$ and $k_2$ are the inertia parameter ratios, as defined in [19]. Tweddle [19] showed that it is not possible to observe all three principal axes of inertia parameters of a tumbling object without applying external torque, and only two out of three degrees of freedom are observable. Hence, they introduced two inertia parameter ratios, which are defined as follows: given that $I_1$, $I_2$, and $I_3$ are the principal axes inertia parameters and the inertia matrix $\mathbf{I}$ in $\mathcal{B}$ is

$$\mathbf{I}=\left[\begin{array}{ccc}{I}_1& 0& 0\\ 0& I_2& 0\\ 0& 0& I_3\end{array}\right],$$

(3)

then

$$k_1=log\left({\displaystyle \frac{I_1}{I_2}}\right),k_2=log\left({\displaystyle \frac{I_2}{I_3}}\right).$$

(4)

Accordingly, $\mathbf{I}$ can be rewritten in terms of $k_1$ and $k_2$ as follows:

$$\mathbf{I}=\left[\begin{array}{ccc}{I}_1/I_2& 0& 0\\ 0& 1& 0\\ 0& 0& I_3/I_2\end{array}\right]=\left[\begin{array}{ccc}exp\left(k_1\right)& 0& 0\\ 0& 1& 0\\ 0& 0& exp(-k_2)\end{array}\right].$$

(5)

The torque-free rotational dynamics equations describing the rate of change of ${}^{\mathcal{B}}\mathit{\omega }$ are given as follows:

$${}^{\mathcal{B}}\dot{\mathit{\omega }}={\mathbf{I}}^{-1}([{}^{\mathcal{B}}\mathit{\omega }\otimes ]\mathbf{I}·{}^{\mathcal{B}}\mathit{\omega })=\left[\begin{array}{c}exp(-k_1)(1-exp(-k_2)){}^{\mathcal{B}}\omega _y{}^{\mathcal{B}}\omega _z\\ (exp(-k_2)-exp\left(k_1\right)){}^{\mathcal{B}}\omega _x{}^{\mathcal{B}}\omega _z\\ exp\left(k_2\right)(exp(-k_1)-1){}^{\mathcal{B}}\omega _x{}^{\mathcal{B}}\omega _y\end{array}\right],$$

(6)

where

$$[{}^{\mathcal{B}}\mathit{\omega }\otimes ]=\left[\begin{array}{ccc}0& -{}^{\mathcal{B}}\omega _z& {}^{\mathcal{B}}\omega _y\\ {}^{\mathcal{B}}\omega _z& 0& -{}^{\mathcal{B}}\omega _x\\ -{}^{\mathcal{B}}\omega _y& {}^{\mathcal{B}}\omega _x& 0\end{array}\right],$$

(7)

with ${}^{\mathcal{B}}\omega _x$, ${}^{\mathcal{B}}\omega _y$, and ${}^{\mathcal{B}}\omega _z$ being the components of ${}^{\mathcal{B}}\mathit{\omega }$. Note that this work does not consider perturbations such as the gravity of the Earth or atmospheric drag.

The rotational kinematics equation describes the rate of change in $\mathbf{q}$, which is mathematically written as follows:

$$\dot{\mathbf{q}}={\displaystyle \frac{1}{2}}\mathbf{\Omega }\mathbf{q},$$

(8)

where

$$\mathbf{\Omega }=\left[\begin{array}{cccc}0& -{}^{\mathcal{B}}\omega _x& -{}^{\mathcal{B}}\omega _y& -{}^{\mathcal{B}}\omega _z\\ {}^{\mathcal{B}}\omega _x& 0& {}^{\mathcal{B}}\omega _z& -{}^{\mathcal{B}}\omega _y\\ {}^{\mathcal{B}}\omega _y& -{}^{\mathcal{B}}\omega _z& 0& {}^{\mathcal{B}}\omega _x\\ {}^{\mathcal{B}}\omega _z& {}^{\mathcal{B}}\omega _y& -{}^{\mathcal{B}}\omega _x& 0\end{array}\right].$$

(9)

We assume that the target object is a rigid body; therefore, we can ignore the movements of the internal components. Under this assumption, the rates of change in the inertia parameter ratios and the feature positions in $\mathcal{B}$ are given as follows:

$$\begin{array}{cc}\hfill {\dot{k}}_1& =0,\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill {\dot{k}}_2& =0,\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill {}^{\mathcal{B}}\dot{\mathbf{p}}_i& ={\mathbf{0}}_{3\times 1}.\hfill \end{array}$$

(12)

For the UKF model, we consider a discrete-time dynamics model for propagating state variables. Mathematically, the state variable $\mathbf{X}(k+1)$ at time-step $k+1$ is written as follows:

$$\mathbf{X}(k+1)=\mathbf{X}\left(k\right)+{\int }_{t=t\left(k\right)}^{t=t\left(k\right)+\Delta t}\mathbf{f}\left(\mathbf{X}\left(k\right)\right)dt+\mathbf{w}\left(k\right),$$

(13)

where $\Delta t$ is the discrete time-interval, and $t\left(k\right)$ and $\mathbf{w}\left(k\right)$ are the time instance and the zero mean Gaussian white noise denoting the process noise at time step k, respectively. Additionally, $\mathbf{f}\left(\mathbf{X}\right)$ is obtained from Equations (6)–(12) and written as follows:

$$\mathbf{f}\left(\mathbf{X}\right)=\left[\begin{array}{c}{\mathbf{I}}^{-1}([{}^{\mathcal{B}}\mathit{\omega }\otimes ]\mathbf{I}·{}^{\mathcal{B}}\mathit{\omega })\\ {\displaystyle \frac{1}{2}}\mathbf{\Omega }\mathbf{q}\\ {\mathbf{0}}_{(3n_f+2)\times 1}\end{array}\right].$$

(14)

The process noise covariance matrix $\mathbf{Q}$ is defined as a constant diagonal matrix and written as follows:

$$\mathbf{Q}=\mathbb{E}\left[\mathbf{w}{\mathbf{w}}^T\right]=diag\left([{\mathbf{\Sigma }}_{\omega }^Q,{\mathbf{\Sigma }}_q^Q,{\mathrm{\Sigma }}_{k_1}^Q,{\mathrm{\Sigma }}_{k_2}^Q,{\mathbf{\Sigma }}_{p_1}^Q,\dots ,{\mathbf{\Sigma }}_{p_{n_f}}^Q]\right),$$

(15)

where ${\mathbf{\Sigma }}_{\omega }^Q\in {\mathbb{R}}^3$, ${\mathbf{\Sigma }}_q^Q\in {\mathbb{R}}^4$, ${\mathrm{\Sigma }}_{k_1}^Q\in \mathbb{R}$, ${\mathrm{\Sigma }}_{k_2}^Q\in \mathbb{R}$, and ${\{{\mathbf{\Sigma }}_{p_i}^Q\in {\mathbb{R}}^3\}}_{i=1}^{n_f}$ contain the process noise variances of ${}^{\mathcal{B}}\mathit{\omega }$, $\mathbf{q}$, $k_1$, $k_2$, and ${\left\{{}^{\mathcal{B}}\mathbf{p}_i\right\}}_{i=1}^{n_f}$. Note that the initial state error covariance matrix ${\mathbf{P}}_0$ has the same size as $\mathbf{Q}$, and the vectors containing the corresponding error variances of ${\mathbf{P}}_0$ are denoted by the same symbols as those used for $\mathbf{Q}$, except the superscript ‘Q’ is replaced by the superscript ‘P’.

2.2.2. Observation Model

In this study, we assume that the chaser is equipped with a stereo-vision camera system to collect information regarding the chaser. The stereo images captured by the stereo-camera system can be processed by computer vision algorithms to detect point features and perform feature matching. There exist a number of methods for feature detection, such as SIFT [28], SURF, AKAZE, ORB [29], etc. In this work, we do not implement feature detection methods and assume that the noisy projection data of the features are available from image processing algorithms.

The detected feature positions are obtained as projections on camera planes. The schematic showing the onboard stereo-camera system is illustrated in Figure 2. The working mechanism of the cameras is based on the pinhole camera model. The center of projection (COP) of the right camera is assumed to be located at the chaser CM or the origin of $\mathcal{I}$. Additionally, we assume that the camera frame $\mathcal{C}$ is aligned with $\mathcal{I}$ at any time since the chaser does not perform any rotation. The COP of the left camera is located at ${[-b,0,0]}^T$, expressed in $\mathcal{I}$. The image planes are located perpendicular to the Y axis with a focal length f distance from the corresponding COPs. Let the i-th feature located at ${}^{\mathcal{B}}\mathbf{p}_i$ in $\mathcal{B}$ have the position vector ${}^{\mathcal{I}}\mathit{\rho }_i={[x_i,y_i,z_i]}^T$ at time-step k. Then, the projection of the feature on the right (left) camera plane is defined as the intersecting point of ${}^{\mathcal{I}}\mathit{\rho }_i$ (${}^{\mathcal{I}}\mathit{\rho }_i+{[b,0,0]}^T$) and the right (left) camera plane. The 2D projection vectors of the feature on the image planes are denoted by ${\mathbf{w}}_{R,i}={[u_{R,i},v_{R,i}]}^T$ and ${\mathbf{w}}_{L,i}={[u_{L,i},v_{L,i}]}^T$ for the right and the left cameras, respectively, where $u_{(·)}$ and $v_{(·)}$ indicate the projection components in the horizontal and vertical directions on the image planes. The expressions of $u_{R,i}$, $v_{R,i}$, $u_{L,i}$, and $v_{L,i}$ are given as follows:

$$\begin{array}{cc}\hfill u_{R,i}& =f{\displaystyle \frac{x_i}{y_i}},v_{R,i}=f{\displaystyle \frac{z_i}{y_i}},\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill u_{L,i}& =f{\displaystyle \frac{x_i+b}{y_i}},v_{L,i}=f{\displaystyle \frac{z_i}{y_i}}.\hfill \end{array}$$

(17)

In practice, one can numerically calculate the optical flows or the rates of change in the projections, ${\dot{\mathbf{w}}}_{R,i}$ and ${\dot{\mathbf{w}}}_{L,i}$ from ${\mathbf{w}}_{R,i}$ and ${\mathbf{w}}_{L,i}$ (e.g., using the finite difference method). Therefore, the optical flows ${\dot{\mathbf{w}}}_{R,i}$ and ${\dot{\mathbf{w}}}_{L,i}$ are included in the observation model in this study similar to [7,30]. The expressions of ${\dot{\mathbf{w}}}_{R,i}$ and ${\dot{\mathbf{w}}}_{L,i}$ are derived in a similar way to that shown in [31], which are given below:

$$\begin{array}{cc}\hfill {\dot{\mathbf{w}}}_{R,i}& =\left[\begin{array}{c}{\dot{u}}_{R,i}\\ {\dot{v}}_{R,i}\end{array}\right]={\mathbf{A}}_{R,i}·{}^{\mathcal{I}}\mathit{\omega },\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill {\dot{\mathbf{w}}}_{L,i}& =\left[\begin{array}{c}{\dot{u}}_{L,i}\\ {\dot{v}}_{L,i}\end{array}\right]={\mathbf{A}}_{L,i}·{}^{\mathcal{I}}\mathit{\omega },\hfill \end{array}$$

(19)

where

$$\begin{array}{cc}\hfill {\mathbf{A}}_{R,i}& =\left[\begin{array}{ccc}{\displaystyle \frac{x_i(z_i-z_0)}{y_i^2}}& {\displaystyle \frac{(z_i-z_0)}{y_i}}& -\left({\displaystyle \frac{y_i-y_0}{y_i}}+{\displaystyle \frac{x_i(x_i-x_0)}{y_i^2}}\right)\\ \left({\displaystyle \frac{y_i-y_0}{y_i}}+{\displaystyle \frac{z_i(z_i-z_0)}{y_i^2}}\right)& -{\displaystyle \frac{(x_i-x_0)}{y_i}}& -{\displaystyle \frac{z_i(x_i-x_0)}{y_i^2}}\end{array}\right],\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill {\mathbf{A}}_{L,i}& =\left[\begin{array}{ccc}{\displaystyle \frac{(x_i+b)(z_i-z_0)}{y_i^2}}& {\displaystyle \frac{(z_i-z_0)}{y_i}}& -\left({\displaystyle \frac{y_i-y_0}{y_i}}+{\displaystyle \frac{(x_i+b)(x_i-x_0)}{y_i^2}}\right)\\ \left({\displaystyle \frac{y_i-y_0}{y_i}}+{\displaystyle \frac{z_i(z_i-z_0)}{y_i^2}}\right)& -{\displaystyle \frac{(x_i-x_0)}{y_i}}& -{\displaystyle \frac{z_i(x_i-x_0)}{y_i^2}}\end{array}\right].\hfill \end{array}$$

(21)

The complete derivation process of Equations (18)–(21) is provided in Appendix A.

Together, Equations (16)–(19) for $n_f$ number of features define the observation model $\mathbf{h}\left(\mathbf{X}\right)$:

$$\mathbf{h}\left(\mathbf{X}\left(k\right)\right)={[{\mathbf{h}}_1^T,{\mathbf{h}}_2^T,\dots ,{\mathbf{h}}_{n_f}^T]}^T,$$

(22)

where, for any $i=1,2,\dots ,n_f$,

$${\mathbf{h}}_i={[{\mathbf{w}}_{R,i}^T,{\mathbf{w}}_{L,i}^T,(u_{L,i}-u_{R,i}),{\dot{\mathbf{w}}}_{R,i}^T,{\dot{\mathbf{w}}}_{L,i}^T]}^T.$$

(23)

The measurement model is defined as follows:

$$\mathbf{z}\left(k\right)=\mathbf{h}\left(\mathbf{X}\right(k\left)\right)+\mathbf{v}\left(k\right),$$

(24)

where $\mathbf{v}\left(k\right)\in {\mathbb{R}}^{9n_f}$ is the measurement noise with components being sampled from Gaussian distributions with zero mean and standard deviations ${\sigma }_p$, ${\sigma }_d$, and ${\sigma }_o$ for projections, disparity, and optical flows. The measurement noise covariance matrix is written as follows:

$$\mathbf{R}=E\left[\mathbf{v}{\mathbf{v}}^T\right],$$

(25)

where $\mathbf{R}$ is a diagonal matrix, which is expressed in terms of ${\sigma }_p$, ${\sigma }_d$, and ${\sigma }_o$.

2.3. The Gaussian Process (GP)

The GP is a Bayesian regression technique that is traditionally implemented to generate a predictive distribution of a scalar function over the single or multi-dimensional input domain. In other words, a GP model returns a scalar output’s mean and uncertainty at a given input. In achieving this, a covariance function or kernel is required to determine the correlations among the inputs in the training data. In a broader sense, a kernel helps capture useful features such as smoothness, periodicity, etc., regarding the underlying output function from the discrete training data. Hence, selecting an appropriate kernel to create a GP model with good prediction ability is crucial.

Multiple single-output GP models can be generated to predict a vector-valued function where each GP model is tasked with estimating one function component. This approach is ineffective if the components of the function are correlated due to the single-output GP models’ inability to capture the correlation among the output domains, thus requiring relatively bigger training data sets. To create prediction models of a vector-valued function with correlated components, it is more data-efficient to implement an advanced version of GP called the multi-output GP, which depends on an additional kernel that detects the correlations among the outputs. However, training a multi-output GP model takes longer due to the additional covariance determination task for the output dimensions.

In this work, we aim to predict the right and left camera projections of the features, ${\left\{{\mathbf{w}}_{R,i}\right\}}_{i=1}^{n_f}$ and ${\left\{{\mathbf{w}}_{L,i}\right\}}_{i=1}^{n_f}$. Intuitively, the components of ${\mathbf{w}}_{R,i}$ and ${\mathbf{w}}_{L,i}$ are correlated since they are functions of ${}^{\mathcal{I}}\mathbf{p}_i$ whose components are correlated as well. Therefore, it is advantageous to generate a multi-output GP model ${\mathcal{G}}_i$ for the right and left projections of each feature, which can find the correlations among the components of the projection measurements. For simplification, let us define a vector ${\mathbf{w}}_i={[u_{R,i},v_{R,i},u_{L,i}]}^T$ for the i-th feature so that all projection components are included in one vector. We do not include $v_{L,i}$ in ${\mathbf{w}}_i$ because $v_{L,i}=v_{R,i}$ by definition.

To train the GP model ${\mathcal{G}}_i$ for the i-th feature, a dataset ${\mathcal{D}}_i$ is required, which contains the projection measurements ${\tilde{\mathbf{w}}}_i$. The dataset ${\mathcal{D}}_i$ is four-dimensional with time as the input and three projection components of ${\tilde{\mathbf{w}}}_i$ as the output. The format of the dataset ${\mathcal{D}}_i$ is shown as an example in

Table 1. Format of the training dataset ${\mathcal{D}}_i$.

Input	Output
Time, t	Projection Components, ${\tilde{\mathbf{w}}}_i$ [rad]
[sec]	${\mathit{u}}_{\mathit{R}}$ Component	${\mathit{v}}_{\mathit{R}}$ Component	${\mathit{u}}_{\mathit{L}}$ Component
$t\left(1\right)$	${\tilde{u}}_{R,i}\left(1\right)$	${\tilde{v}}_{R,i}\left(1\right)$	${\tilde{u}}_{L,i}\left(1\right)$
$t\left(2\right)$	${\tilde{u}}_{R,i}\left(2\right)$	${\tilde{v}}_{R,i}\left(2\right)$	${\tilde{u}}_{L,i}\left(2\right)$
⋮	⋮	⋮	⋮
$t\left(k\right)$	${\tilde{u}}_{R,i}\left(k\right)$	${\tilde{v}}_{R,i}\left(k\right)$	${\tilde{u}}_{L,i}\left(k\right)$

.

We use the intrinsic coregionalization model (ICM) kernel [32] to determine the covariance among output domains. Also, we utilize a product kernel as the covariance function for the input domain for each of the GP models. The kernel is formulated as the product of two periodic kernels. The product kernel $K(t_i,t_j)$ for ${\mathcal{G}}_i$ is given as follows:

$$K(t_i,t_j)={\sigma }_1^{2}exp\left(-{\displaystyle \frac{2}{l_1^2}}{sin}^2\left({\displaystyle \frac{\pi \delta t}{P_1}}\right)\right)·{\sigma }_2^{2}exp\left(-{\displaystyle \frac{2}{l_2^2}}{sin}^2\left({\displaystyle \frac{\pi \delta t}{P_2}}\right)\right),$$

(26)

where $\delta t=|t_i-t_j|$. Also, $l_1$ and $l_2$ are the length-scale hyper-parameters, ${\sigma }_1$ and ${\sigma }_2$ are the smoothness hyper-parameters, and $P_1$ and $P_2$ are the periodicity hyper-parameters. The hyper-parameters are obtained from the maximum likelihood optimization operation performed on the training dataset [33].

Note that, for a torque-free tumbling object, the components of the position of an arbitrary point located on the object expressed in an inertial frame are not periodic functions [34]. Hence, the components of ${}^{\mathcal{I}}\mathit{\rho }_i$ are non-periodic. However, the underlying reason behind the variations of ${}^{\mathcal{I}}\mathit{\rho }_i$ components is the angular velocity of the target, which is periodic if expressed in $\mathcal{B}$, with the periodicity being the polhode period $T_p$. Hence, we speculate that even though the components of ${}^{\mathcal{I}}\mathit{\rho }_i$ are non-periodic, they should have some intrinsic periodic properties, which we want to capture using the periodic kernels. The logic behind using two periodic kernels in the product kernel is that one periodic kernel should capture the effect of the tumbling rate or the magnitude of the angular velocity while the other periodic kernel should learn the effect of the polhode period on the variations of ${\mathbf{w}}_i$.

Although the periodicity hyper-parameters $P_1$ and $P_2$ in Equation (26) can be determined by hyper-parameters training from the given training datasets, it significantly improves the performance of the trained GP models if we provide good initial guesses of the periodicity hyper-parameters. This reduces the chance of the hyper-parameter training optimization algorithm getting stuck at local solutions. Hence, we utilize a fast Fourier transform-based strategy to determine the initial guesses of $P_1$ and $P_2$. The formal approach to obtain the initial guesses of $P_1$ and $P_2$ for the product kernel K is provided in Algorithm 1.

Algorithm 1 Periodicity hyper-parameters initial guess determination algorithm for ${}^{\mathcal{I}}\mathit{\rho }_i$.

1: for  $j=1\mathrm{to}3$do   2:       Perform the fast Fourier transform (FFT) operation on the j-th component of ${\tilde{w}}_i={[{\tilde{u}}_{R,i},{\tilde{v}}_{R,i},{\tilde{u}}_{L,i}]}^T$   3:       Determine the frequencies $f_{j1}$ and $f_{j2}$ with the highest and second highest magnitudes from the FFT operation   4:       Calculate the corresponding periods $P_{j,1}$ and $P_{j,2}$ of $f_{j,1}$ and $f_{j,2}$ as $P_{j,(·)}=1/f_{j,(·)}$   5: end for   6: Obtain the unique periods from $[{\left\{P_{j1}\right\}}_{j=1}^3,{\left\{P_{j2}\right\}}_{j=1}^3]$   7: Create all possible combinations of a pair of periods from the list of unique periods.   8: Calculate the absolute difference between the periods and the sum of magnitudes provided by FFT for each pair of unique periods.   9: Multiply the absolute time difference and the sum of the magnitudes 10: Select the pair of unique periods with the highest product as the initial guesses of the periodicity hyper-parameters.

2.4. UKF-GP Algorithm

The flowchart of the proposed UKF-GP algorithm is presented in Figure 3. In the first step, the noisy projection data ${\left\{{\tilde{\mathbf{w}}}_i\left(k\right)\right\}}_{i=1}^{n_f}$ are collected from time step $k=1$ (equivalently, $t=0$) to $k=T_{train}/\Delta t$ or $t=T_{train}$ with the measurement frequency $f_s=1/\Delta t$. Then, the collected noisy projection measurements are stored in the dataset ${\left\{{\mathcal{D}}_i\right\}}_{i=1}^{n_f}$. Next, the hyper-parameters of the GP models ${\left\{{\mathcal{G}}_i\right\}}_{i=1}^{n_f}$ are trained from the datasets ${\left\{{\mathcal{D}}_i\right\}}_{i=1}^{n_f}$ and the kernel functions. In the next step, the predictions of the projections ${\left\{{\widehat{\mathbf{w}}}_i\left(k\right)\right\}}_{i=1}^{n_f}$ are obtained from ${\left\{{\mathcal{G}}_i\right\}}_{i=1}^{n_f}$. The predictions of the optical flows ${\left\{{\widehat{\dot{\mathbf{w}}}}_i\left(k\right)\right\}}_{i=1}^{n_f}$ are then calculated from ${\left\{{\widehat{\mathbf{w}}}_i\left(k\right)\right\}}_{i=1}^{n_f}$ using the finite difference method (FDM). We implement the first-order forward and backward differences as well as the second and fourth-order central differences [35] to compute the optical flows, and the corresponding FDM equations are given as follows:

$${\widehat{\dot{\mathbf{w}}}}_i\left(k\right)=\left\{\begin{array}{cc}{\displaystyle \frac{{\widehat{\mathbf{w}}}_i(k+1)-{\widehat{\mathbf{w}}}_i\left(k\right)}{\Delta t}},\hfill & \mathrm{if}k=1\hfill \\ {\displaystyle \frac{{\widehat{\mathbf{w}}}_i(k+1)-{\widehat{\mathbf{w}}}_i(k-1)}{2\Delta t}},\hfill & \mathrm{if}k=2\mathrm{or}k=T_{final}/\Delta t-1\hfill \\ {\displaystyle \frac{{\widehat{\mathbf{w}}}_i\left(k\right)-{\widehat{\mathbf{w}}}_i(k-1)}{\Delta t}},\hfill & \mathrm{if}k=T_{final}/\Delta t\hfill \\ {\displaystyle \frac{-{\widehat{\mathbf{w}}}_i(k+2)+8{\widehat{\mathbf{w}}}_i(k+1)-8{\widehat{\mathbf{w}}}_i(k-1)+{\widehat{\mathbf{w}}}_i(k-2)}{12\Delta t}},\hfill & \mathrm{otherwise}\hfill \end{array}\right..$$

(27)

In the final step, the projection and optical flow predictions, ${\left\{{\widehat{\mathbf{w}}}_i\left(k\right)\right\}}_{i=1}^{n_f}$ and ${\left\{{\widehat{\dot{\mathbf{w}}}}_i\left(k\right)\right\}}_{i=1}^{n_f}$, are utilized as the pseudo-measurements for UKF-GP during the long-term occlusion.

2.5. Simulation Workflow

The formal simulation workflow according to which we have verified the UKF-GP algorithm is provided in Algorithm 2. In the beginning, $n_{train}=T_{train}/\Delta t$ sensor measurements are collected from $t=0$ s till $t=T_{train}$ with the measurement frequency $f_s$, and we assume that occlusion happens from $t=T_{train}$ until the end of the simulation ($t=T_{total}$). The projection measurements obtained from $t=0$ until $t=T_{train}$ are stored in the datasets ${\left\{{\mathcal{D}}_i\right\}}_{i=1}^{n_f}$. Next, the multi-output GP models ${\left\{{\mathcal{G}}_i\right\}}_{i=1}^{n_f}$ are trained using the datasets, the product kernel, and the hyper-parameter initial guesses. Afterward, ${\left\{{\widehat{\mathbf{w}}}_i\right\}}_{i=1}^{n_f}$ are generated for the whole duration of the simulation, from $t=0$ s to $t=T_{total}$ ($T_{total}>T_{train}$) sec with the prediction sampling frequency $f_{GP}$. Next, ${\left\{{\widehat{\dot{\mathbf{w}}}}_i\left(k\right)\right\}}_{i=1}^{n_f}$ are calculated from ${\left\{{\widehat{\mathbf{w}}}_i\right\}}_{i=1}^{n_f}$ using FDM provided in Equation (27). Then, the UKF model is initiated with the discrete-time interval $\Delta t_{GP}=1/f_{GP}$, total simulation time $T_{total}$, the dynamics model $\mathbf{f}\left(\mathbf{X}\right)$, the observation model $\mathbf{h}\left(\mathbf{X}\right)$, the initial guesses of the state variable ${\widehat{\mathbf{X}}}_0$, the initial error covariance matrix ${\mathbf{P}}_0$, the process noise covariance matrix $\mathbf{Q}$, and the measurement noise covariance matrix $\mathbf{R}$. Once the model is initiated, the state variables are updated using ${\left\{{\widehat{\mathbf{w}}}_i\left(k\right)\right\}}_{i=1}^{n_f}$ and ${\left\{{\widehat{\dot{\mathbf{w}}}}_i\left(k\right)\right\}}_{i=1}^{n_f}$ as the pseudo-measurements at every time-step until the end of the simulation.

Algorithm 2 Simulation workflow.

1: From $t=0$ to $t=T_{train}$, collect $n_{train}$ number of sensor-provided feature projection measurements ${\left\{{\widehat{\mathbf{w}}}_i\left(k\right)\right\}}_{i=1}^{n_f}$ with the measurement frequency $f_s$.   2: Store the projection measurement data in the ${\left\{{\mathcal{D}}_i\right\}}_{i=1}^{n_f}$ and the kernel functions.   3: Train $n_f$ number of multi-output GP models ${\left\{{\mathcal{G}}_i\right\}}_{i=1}^{n_f}$ from data sets ${\left\{{\mathcal{D}}_i\right\}}_{i=1}^{n_f}$.   4: From ${\left\{{\mathcal{G}}_i\right\}}_{i=1}^{n_f}$, sample projection predictions ${\{{\widehat{\mathbf{w}}}_R={[{\widehat{u}}_{R,i},{\widehat{v}}_{R,i},{\widehat{u}}_{L,i}]}^T\}}_{i=1}^{n_f}$ from $t=0$ to $t=T_{total}$ with the sampling frequency $f_{GP}$.   5: Calculate the optical flows ${\left\{{\widehat{\dot{\mathbf{w}}}}_i\left(k\right)\right\}}_{i=1}^{n_f}$ from ${\left\{{\widehat{\mathbf{w}}}_i\left(k\right)\right\}}_{i=1}^{n_f}$ according to Equation (27).   6: Initiate the UKF-GP model with $\Delta t_{GP}$, ${\widehat{\mathbf{X}}}_0$, ${\mathbf{P}}_0$, $\mathbf{Q}$, $\mathbf{R}$, $\mathbf{f}\left(\mathbf{X}\right)$, $\mathbf{h}\left(\mathbf{X}\right)$, $T_{total}$.   7: for$k=1$ to $T_{total}/\Delta t_{GP}$ do   8:       Predict the state variables and the state error covariance matrix using the discrete dynamics model and the sigma points.   9:       Update the estimated state variables using the observation model with the GP-provided predicted projections ${\left\{{\widehat{\mathbf{w}}}_i\left(k\right)\right\}}_{i=1}^{n_f}$ and the optical flows ${\{{\widehat{\dot{\mathbf{w}}}}_i\left(k\right)\}}_{i=1}^{n_f}$. 10:     $k\leftarrow k+1$ 11: end for

3. Results and Discussion

In order to demonstrate the effectiveness of the proposed UKF-GP algorithm over the conventional UKF model to estimate the rotational motion and inertia parameters of a non-cooperative and tumbling target space object, we conducted Monte Carlo (MC) simulations for different tumbling frequencies, and the results are presented in this section. First, we present the MC simulation results regarding the GP models to portray the prediction accuracy of the GP models, which are followed by the MC simulation results regarding the UKF-GP and conventional UKF algorithms.

The common simulation parameters for the MC simulations of GP and UKF-GP are provided below:

• Discrete-time intervals: $\Delta t=\Delta t_{gp}=1$ s;
• Sensor measurement sampling rate: $f_s=1$ Hz;
• Principal axes inertia parameters of the target: $I_x=90{\mathrm{kgm}}^2$, $I_x=100{\mathrm{kgm}}^2$, $I_z=112{\mathrm{kgm}}^2$;
• Tumbling frequencies: $f_T=0.025$, $0.075$, $0.125$, and $0.175$ Hz (the corresponding polhode periods $T_p$ = 441.045 s, 147.015 s, 88.209 s, and 63.006 s, respectively);
• Initial angular velocities in $\mathcal{B}$: ${}^{\mathcal{B}}\mathit{\omega }_0=2\pi f_T{[0.67,-0.33,0.67]}^T$ rad/s;
• Initial attitude quaternion: ${\mathbf{q}}_0={[1,0,0,0]}^T$;
• Standard deviation of the projection measurement: ${\sigma }_p={10}^{-5}$ rad.

The sensor measurement frequency $f_s$ is assumed to be 1 Hz because visual camera sensors have a slower sampling frequency [1], and this value was considered by similar works that were previously conducted [6,7]. Both the tumbling frequency $f_T$ and the sensor measurement frequency $f_s$ influence the prediction performance of the GP models. We considered $f_s$ to be at least five times $f_T$, which we determined by trial and error using the following reasons. This criterion is set to ensure that the fluctuations of the projection measurements are properly captured in the training dataset and that the GP models can provide good predictions. Since $f_s=1$ Hz is assumed to be fixed in our work, $f_T$ should be below 0.20 Hz, and we considered the $f_T$ values in this study accordingly. Note that this is a drawback that is caused by the selected sensor with low measurement frequency, not by the GP algorithm itself. Additionally, the considered standard deviation of the projection measurements ${\sigma }_p$ is also assumed by previous similar studies [6,7]. Moreover, the necessary mathematical equations to calculate the polhode period $T_p$ given the principal inertia parameters and the initial angular velocity are provided in Appendix B and adapted from [34].

All results presented in this section were obtained by conducting simulations on a desktop computer with 12th Gen ${\mathrm{Intel}}^{\circledR }{\mathrm{Core}}^{\mathrm{TM}}$ i7-12700 (3.60 GHz) and 32 GB of RAM. Additionally, we used a Python-based open-source Gaussian process framework, ‘GPy’ [36], for generating GP models. The hyper-parameter optimization was performed by implementing the ‘L-BFGS’ optimizer.

3.1. Prediction Performance of the GP

Before demonstrating the estimation performance of the proposed UKF-GP algorithm, we present the prediction accuracy of the GP models in varying scenarios. For this task, we conducted MC simulations for four tumbling frequencies and nine different training data time spans. The specific simulation parameters for these MC simulations are given below:

• Number of features: $n_f=1$;
• Position of the feature in $\mathcal{B}$: ${}^{\mathcal{B}}\mathbf{p}={[1,1,1]}^T$ m;
• Training data time-span: $T_{train}=75$, 100, 200, 500, 1000, 1500, 2000, 2500, and 3000 s;
• Duration of prediction: $T_{predict}=T_{total}-T_{train}=2000$ s;
• Number of runs: $n_{run}=20$.

To compare the prediction performances among the GP models in different scenarios, we calculated the root mean square error (RMSE) of the predicted projection vector ${\left\{{\widehat{\mathbf{w}}}_i\right\}}_{i=1}^{n_f}$ from $k=T_{train}/\Delta t$ to $k=(T_{train}+T_{predict})/\Delta t$, in other words, for 2000 time steps starting from $k=T_{train}/\Delta t$. The box plots of the RMSE distributions and the GP model train time for all MC runs are provided in Figure 4.

In Figure 4a, the RMSE vs. the training data time span $T_{train}$ box plots are provided, where we can observe the gradual decrease in the RMSEs for all $f_T$ as $T_{train}$ increases. For $f_T=0.025$ Hz and the first three values of $T_{train}$, significantly high RMSEs were observed, indicating that GP models are incapable of generating highly accurate projection predictions in these scenarios due to the lack of sufficient training data. However, for $T_{train}\ge 1000\mathrm{s}\approx 2T_p$, GP models can predict projections with RMSEs on the scale of ${10}^{-6}$ rad, which shows the GP models’ prediction accuracy with longer $T_{train}$.

For $f_T=0.075$ Hz, a similar trend of the RMSE over $T_{train}$ was observed, but the drop of the RMSEs occured at a smaller value of $T_{train}$. For $T_{train}=75$ and 100 s, the RMSEs were very high but suddenly dropped below ${10}^{-5}$ rad at $T_{train}=200\mathrm{s}\approx 3.4T_p$. For higher $T_{train}$, the RMSEs remained below ${10}^{-5}$ rad, which illustrated the highly accurate prediction ability of the GP models in these scenarios.

For $f_T=0.125$ and $T_{train}=75$ and 100 s, the RMSEs were quite high due to insufficient training data. However, starting from $T_{train}=200\mathrm{s}\approx 2.26T_p$, the RMSE distribution was mostly on the scale of ${10}^{-5}$ rad and below ${10}^{-5}$ rad for higher $T_{train}$.

The box plots for $f_T=0.175$ Hz at smaller values of $T_{train}$ show a more interesting trend. While for all other $f_T$ at $T_{train}=75$ and 100 s, the spans of the RMSE box plots were very narrow, the spans for $f_T=0.175$ Hz at these $T_{train}$ were quite long. These long spans of the box plots indicate that even at these smaller $T_{train}$ and smaller-sized datasets, the GP models in some of the MC runs managed to be trained properly; therefore, in those runs, the prediction accuracy was acceptable. In particular, at $T_{train}=100$ s, in more than 50% of the runs, the GP models generated good projection predictions as the median line was observed to be in the scale ${10}^{-5}$ rad. For $T_{train}=200$ s $\approx 3.17T_p$, the RMSE distribution was mostly on the scale of ${10}^{-5}$ rad, which was below ${10}^{-5}$ for $T_{train}\ge 500$ s.

From the above analysis, it is evident that the minimum duration of the training data time span $T_{train}$ to create GP models capable of generating quality predicted projections is about $3T_p$. However, this empirical relationship between $T_{train}$ and $T_p$ is only applicable to the considered $f_T$. A general expression for the minimum $T_{train}$ to create capable GP models requires a more exhaustive investigation.

In Figure 4a, several anomalies can be observed for all tumbling frequencies except $f_T=0.125$ Hz. These anomalous GP models were not properly trained because, occasionally, the optimization algorithm got stuck at some local minima during the hyper-parameter training process; thus, non-optimal values were selected as the model hyper-parameters. The occurrence of these anomalies may have been avoided if a global optimizer was used, which is a more computationally expensive choice.

The training time box plots provided in Figure 4b exhibit that the training time spanning from a few seconds to a few hours has a rather logarithmic relationship with the training data time span regardless of the choice of $f_T$. In combination, Figure 4a,b helps to conclude that $T_{train}=1000$ s is the optimal choice of $T_{train}$ to train good GP models, which are capable of providing highly accurate predictions for thousands of seconds while the model training time requires only a few minutes.

3.2. Prediction Performance of UKF-GP

The MC simulations to demonstrate the effectiveness of the UKF-GP algorithm were conducted for four $f_T$ values and one $T_{train}$ value. The details of the MC simulations of the UKF-GP models are provided below:

• Total simulation duration: $T_{total}=3000$ s.
• Duration of the sensor data availability: $T_{train}=1500$ s.
• Duration of the occlusion: $T_{predict}=1500$ s.
• Number of features: $n_f=5$.
• User-defined constant parameters: $\alpha =0.1$, $\beta =2$, $\kappa =0$ (from [26]).
• Position of the features in $\mathcal{B}$:

  –

  Feature 1: ${}^{\mathcal{B}}\mathbf{p}_1={[0.1302,-0.6648,-0.2264]}^T$ m;

  –

  Feature 2: ${}^{\mathcal{B}}\mathbf{p}_2={[1.0343,-1.4858,-1.1352]}^T$ m;

  –

  Feature 3: ${}^{\mathcal{B}}\mathbf{p}_3={[0.5122,0.9775,-1.0898]}^T$ m;

  –

  Feature 4: ${}^{\mathcal{B}}\mathbf{p}_4={[0.2252,1.1739,-0.8723]}^T$ m;

  –

  Feature 5: ${}^{\mathcal{B}}\mathbf{p}_5={[-0.9440,-1.1748,-0.8409]}^T$ m.

• Initial guesses of the state variables:

  –

  ${}^{\mathcal{B}}\widehat{\mathit{\omega }}_0=1.75{}^{\mathcal{B}}\mathit{\omega }\left(0\right)$ rad/s;

  –

  ${\widehat{\mathbf{q}}}_0={[0.9707,0.1386,0.13860,0.1386]}^T$;

  –

  ${\widehat{k}}_1={\widehat{k}}_2=0$;

  –

  ${}^{\mathcal{B}}\widehat{\mathbf{p}}_i={}^{\mathcal{B}}\mathbf{p}_i+{[0.7,0.7,0.7]}^T$ m.

• Standard deviation of the measurement noise of the optical flow:

  –

  For $f_T=0.025$ Hz, ${\sigma }_w={10}^{-5}$ rad/s;

  –

  For $f_T=0.075$ Hz, ${\sigma }_w=5\times {10}^{-4}$ rad/s;

  –

  For $f_T=0.125$ Hz, ${\sigma }_w=5\times {10}^{-4}$ rad/s;

  –

  For $f_T=0.175$ Hz, ${\sigma }_w={10}^{-3}$ rad/s.

• Standard deviation of the measurement noise of the disparity: ${\sigma }_d={\sigma }_p={10}^{-5}$ rad.
• Variances for the initial state error covariance matrix:

  –

  Variance of ${}^{\mathcal{B}}\mathit{\omega }$: ${\mathrm{\Sigma }}_{\mathit{\omega }}^P=0.1225[1,1,1]{\mathrm{rad}}^2{\mathrm{/s}}^2$;

  –

  Variance of $\mathbf{q}$: ${\mathrm{\Sigma }}_q^P=0.04[1,1,1,1]$;

  –

  Variance of $k_1$ and $k_2$: ${\mathrm{\Sigma }}_{k_1}^P={\mathrm{\Sigma }}_{k_2}^P=0.01$;

  –

  Variance of ${}^{\mathcal{B}}\mathbf{p}_i$: ${\mathrm{\Sigma }}_{p_i}^P=2.25[1,1,1]{\mathrm{m}}^2$.

• Variances in the process noise covariance matrix:

  –

  Variance in ${}^{\mathcal{B}}\mathit{\omega }$:

  ∗

  For $f_T=0.025$ Hz, ${\mathrm{\Sigma }}_{\mathit{\omega }}^Q=2.5\times {10}^{-9}[1,1,1]{\mathrm{rad}}^2{\mathrm{/s}}^2$;

  ∗

  For $f_T=0.075$ Hz, ${\mathrm{\Sigma }}_{\mathit{\omega }}^Q=2.5\times {10}^{-7}[1,1,1]{\mathrm{rad}}^2{\mathrm{/s}}^2$;

  ∗

  For $f_T=0.125$ Hz, ${\mathrm{\Sigma }}_{\mathit{\omega }}^Q=2.5\times {10}^{-5}[1,1,1]{\mathrm{rad}}^2{\mathrm{/s}}^2$;

  ∗

  For $f_T=0.175$ Hz, ${\mathrm{\Sigma }}_{\mathit{\omega }}^Q=2.5\times {10}^{-5}[1,1,1]{\mathrm{rad}}^2{\mathrm{/s}}^2$.

  –

  Variance in $\mathbf{q}$:

  ∗

  For $f_T=0.025$ Hz, ${\mathrm{\Sigma }}_q^Q=4\times {10}^{-10}[1,1,1,1]$;

  ∗

  For $f_T=0.075$ Hz, ${\mathrm{\Sigma }}_q^Q=4\times {10}^{-8}[1,1,1,1]$;

  ∗

  For $f_T=0.125$ Hz, ${\mathrm{\Sigma }}_q^Q=4\times {10}^{-6}[1,1,1,1]$;

  ∗

  For $f_T=0.175$ Hz, ${\mathrm{\Sigma }}_q^Q=4\times {10}^{-6}[1,1,1,1]$.

  –

  Variance in $k_1$ and $k_2$: ${\mathrm{\Sigma }}_{k_1}^Q={\mathrm{\Sigma }}_{k_2}^Q=3.6\times {10}^{-5}$.

  –

  Variance in ${}^{\mathcal{B}}\mathbf{p}_i$: ${\mathrm{\Sigma }}_{p_i}^Q={10}^{-6}[1,1,1]{\mathrm{m}}^2$.

In this study, the selection of the number of features $n_f$ was adopted from a previous study [7]. Also, the standard deviations of the optical flow measurement noise were obtained from FDM. Additionally, the diagonal variance components of ${\mathbf{P}}_0$ and $\mathbf{Q}$ matrices were chosen manually by trial and error.

To illustrate the superiority of the UKF-GP model over the conventional UKF model, we conducted MC simulations of UKF under long-term occlusion with the same setup as that of UKF-GP. Moreover, MC simulations of the UKF model without long-term occlusion were also performed, and the results are presented along with UKF and UKF-GP as a benchmark so that the reader can easily compare the superior performance of UKF-GP to that of UKF. In the subsequent figures, the results of UKF with and without long-term occlusion are denoted by the ’UKF’ and ’Benchmark’, respectively.

Before presenting the MC simulation results, we provide the estimation error performances of UKF, Benchmark, and UKF-GP for a single run so that the reader can easily observe the performance differences among the considered approaches. The log-scale plots of the absolute estimation errors of ${}^{\mathcal{B}}\widehat{\mathit{\omega }}$, $\widehat{\mathbf{q}}$, ${\widehat{k}}_1$, ${\widehat{k}}_2$, and ${}^{\mathcal{B}}\widehat{\mathbf{p}}_1$ obtained from UKF, Benchmark, and UKF-GP are provided in Figure 5, Figure 6, Figure 7 and Figure 8 for $f_T=0.025$, $0.075$, $0.125$, and $0.175$ Hz, respectively.

Figure 5 shows that for $f_T=0.025$ Hz and UKF, the absolute estimation errors of the ${}^{\mathcal{B}}\widehat{\mathit{\omega }}$ components rapidly decrease at the beginning and mostly stay below ${10}^{-3}$ rad/s from $t=300$ s until the beginning of occlusion at $t=1500$ s, which is the same for Benchmark. However, once the occlusion starts at $t=1500$ s, the estimation errors rapidly diverge, and the upper limit of the errors reaches the scale of ${10}^{-1}$ rad/s. The divergence of error occurs because the UKF model cannot be updated during occlusion due to the lack of stereo-camera observations; hence, only the propagation of the ${}^{\mathcal{B}}\widehat{\mathit{\omega }}$ according to the dynamics model cannot provide reliable estimates. Similar behavior was observed for the estimation error of $\widehat{\mathbf{q}}$, where the errors quickly converge at the beginning and the errors remain below ${10}^{-2}$, but during occlusion, the diverging errors exceed the value of 1. The state variables $k_1$, $k_2$, and ${}^{\mathcal{B}}\mathbf{p}_1$ are constant parameters which do not change with time. Hence, their estimates ${\widehat{k}}_1$, ${\widehat{k}}_2$, and ${}^{\mathcal{B}}\widehat{\mathbf{p}}_1$ keep converging before occlusion because of the availability of the camera measurements, but during occlusion, they retain their values after the last update. Since the estimation errors of these parameters already reach smaller values ($\approx {10}^{-3}$) by the time occlusion is initiated, they remain the same throughout the entire span of occlusion.

From Figure 5, we observe that the performance of UKF-GP is very similar to that of UKF and Benchmark before occlusion, as the estimation errors of the state variables rapidly converge at the beginning and remain small before occlusion. However, the main contrast between UKF-GP and UKF is clearly visible during occlusion. Unlike UKF, UKF-GP manages to stop the divergence of the rotational motion variable estimates ${}^{\mathcal{B}}\widehat{\mathit{\omega }}$ and $\widehat{\mathbf{q}}$ during occlusion and the errors are comparable with that of Benchmark. Also, the errors are confined by the $3-\sigma$ uncertainty boundaries for UKF-GP during occlusion, which is not observed for UKF. These improvements result from the highly accurate projection estimates ${\left\{{\widehat{\mathbf{w}}}_i\right\}}_{i=1}^{n_f}$ from the GP models and the optical flow estimates ${\left\{{\widehat{\dot{\mathbf{w}}}}_i\right\}}_{i=1}^{n_f}$, which are used as the pseudo-measurements for UKF-GP during occlusion. For ${\widehat{k}}_1$, ${\widehat{k}}_2$, and ${}^{\mathcal{B}}\widehat{\mathbf{p}}_1$, the estimation errors of UKF-GP also mimic that of Benchmark.

For $f_T=0.075$ Hz, the state estimation errors from UKF and UKF-GP behave similarly to the previous scenario, as seen in Figure 6. The estimation errors of ${}^{\mathcal{B}}\widehat{\mathit{\omega }}$ and $\widehat{\mathbf{q}}$ from UKF converge at the beginning and remain small before $t=1500$ s for UKF but quickly diverge to considerably high values during occlusion. Contrary to that, the state estimation errors of UKF-GP remain significantly small throughout the entire duration except for the initial convergence, which is similar to Benchmark.

The most notable observation from the estimation errors plots for $f_T=0.125$ Hz in Figure 7 is that ${}^{\mathcal{B}}\widehat{\mathit{\omega }}$ errors diverge exponentially from $t=2000$ s to $t=2438$ s for UKF during occlusion. At the end, the order of magnitude of ${}^{\mathcal{B}}\widehat{\mathit{\omega }}$ errors reaches 6. The simulation is forced to stop here because the covariances of ${}^{\mathcal{B}}\widehat{\mathit{\omega }}$ reaches virtually infinity at this point. During occlusion, the errors of $\widehat{\mathbf{q}}$ for UKF also become significantly high, whereas the other state variable errors remain constant and small. On the other hand, UKF-GP continues to provide reliable estimates of all state variables as before throughout the entire simulation. Similar observations about the state estimations errors from UKF and UKF-GP are made for $f_T=0.175$ Hz, except the UKF simulation stops at $t=1993$ s due to the extremely high covariances associated with ${}^{\mathcal{B}}\widehat{\mathit{\omega }}$.

The MC simulation results of UKF, Benchmark, and UKF-GP for four $f_T$ values are provided in Figure 9, where we present the box plots of the RMSEs of the estimations of the state variables during occlusion from $t=1500$ s until $t=3000$ s. First, we will point our focus towards the box plots corresponding to the RMSEs of ${}^{\mathcal{B}}\widehat{\mathit{\omega }}$ shown in Figure 9a. The RMSE distributions of UKF are located at the higher orders of magnitudes ($\approx -1$ for 0.025 Hz and 0.075 Hz and 4 to 5 for 0.125 Hz and 0.175 Hz) compared to the RMSE distributions of Benchmark. This discrepancy indicates that the UKF-estimated ${}^{\mathcal{B}}\widehat{\mathit{\omega }}$ diverges during occlusion for all runs and all tumbling frequencies. The significantly high magnitudes of the RMSE of UKF at the higher tumbling frequencies also indicate that at the end of the simulations, ${}^{\mathcal{B}}\widehat{\mathit{\omega }}$ diverges exponentially, which we have observed in Figure 7 and Figure 8. However, for UKF-GP, the RMSE distributions are either below (0.075 Hz) or at the same level as the distributions of Benchmark (0.125 Hz and 0.175 Hz) at all $f_T$ values except 0.025 Hz.

The RSME distribution of UKF-GP at 0.025 Hz spans from the scale of ${10}^{-4}$ rad/s to ${10}^{-1}$ rad/s, and the median is on the scale of ${10}^{-3}$ rad/s. This long span is due to the fact that among 20 runs, UKF-GP provides reliable ${}^{\mathcal{B}}\widehat{\mathit{\omega }}$ in 10 runs, and in the other 10 runs, UKF-GP is not so accurate at the estimation task. It turns out that among a total of 100 GP models (five GP models for five features per run), 15 GP models in 10 runs could not be trained properly, which affects the estimation performance of UKF-GP in those runs. UKF-GP is largely affected by this issue, particularly at $f_T=0.025$ Hz because the corresponding $T_p=441.045$ s, which means that $T_{train}=1500$ s is only 3.4 times $T_p$, whereas for the other frequencies with shorter $T_p$, $T_{train}$ is 10.2, 17.005, 23.807 times $T_p$. With a relatively small number of training data, the GP models get less chance to learn about the underlying periodic trends from the training data for $f_T=0.025$ Hz compared to the other $f_T$ values.

Similar trends are observed for the RMSE box plots of $\widehat{\mathbf{q}}$ in Figure 9b. The RMSEs of UKF in all runs have significantly higher values (on a scale of 1) compared to that of Benchmark (on a scale of ${10}^{-3}$). Except for 0.025 Hz, UKF-GP’s RMSE boxes are either significantly below (0.075 Hz) or close to the boxes of Benchmark (0.125 Hz and 0.175 Hz). The box of UKF-GP at 0.025 Hz has a very long span due to some improperly trained GP models, as we have explained above.

The box plots for ${\widehat{k}}_1$ and ${\widehat{k}}_2$ in Figure 9c,d show that the RMSEs of UKF are in smaller scales (${10}^{-3}$ or less), unlike the rotational motion variables. This happens for two reasons: first, the $k_1$ and $k_2$ are inertia parameters that do not change with time and during occlusion, their estimations cannot be updated anymore, resulting in constant errors. Second, by the time when the occlusion starts ($t=1500$ s), the ${\widehat{k}}_1$ and ${\widehat{k}}_2$ have already reached close to the truth, and the errors have become quite small. Therefore, the errors remain constant and small during the occlusion, and as a result, the order of magnitude of the corresponding RMSEs is small. The RMSEs of UKF-GP also remain small and are comparable to Benchmark for all tumbling frequencies except for 0.025 Hz.

Finally, the box plots for ${\left\{{}^{\mathcal{B}}\widehat{\mathbf{p}}_i\right\}}_{i=1}^5$ are provided in Figure 9e–i. For UKF, the errors do not change during the occlusion because of the reason explained above, and as a result, the corresponding RMSEs are quite small (${10}^{-2}$ m or smaller). Similarly, the RMSEs of UKF-GP also remain small (${10}^{-2}$ m or smaller) except 0.025 Hz.

For UKF-GP and all $f_T$ except 0.025 Hz, we can observe a small number of anomalies, which are indicated by the ‘+’ symbol in Figure 9. These anomalies are attributed to a few GP models that are not trained properly. These GP models could not provide highly accurate estimates of the projections of the corresponding features, which affected the estimation accuracy of UKF-GP.

4. Conclusions

This study proposes an unscented Kalman filter–Gaussian process (UKF-GP) fusion algorithm, which provides reliable estimates of the rotational motion variables and the inertia parameters of a non-cooperative tumbling target space object under long-term occlusions (unavailability of sensor measurements for hundreds or thousands of seconds). We assumed that a stereo-camera system onboard a chaser spacecraft provides projection information of multiple point features of the target. The collected projection data were used as training data for multi-output GP models, and we utilized periodic kernels to capture the periodic trends from non-periodic and noisy projection measurements. The initial guesses of the periodicity hyper-parameters were obtained from the fast Fourier transform (FFT) analysis of the projection data as a way to improve the hyper-parameter training procedure. Once trained, the GP models provided the predicted projection information regarding the features. Then, optical flows were obtained from the finite difference method (FDM) and predicted projections. Finally, predicted projections and optical flows were used as the pseudo-measurements for the UKF-GP algorithm during long-term occlusion.

To validate the capabilities of the GP models and the proposed UKF-GP algorithm, Monte Carlo (MC) simulations were performed for four tumbling frequencies: 0.025, 0.075, 0.125, and 0.175 Hz. The MC simulations of the GP models demonstrated that the GP models can provide highly accurate projection prediction (RMSEs in the scale of ${10}^{-6}$ rad) for 2000 s and for all considered tumbling frequencies if the time span of the training data is approximately three times the polhode period or longer. The MC simulations conducted for UKF-GP show that throughout the entire duration of the simulations and for all tumbling frequencies except 0.025 Hz, UKF-GP can provide reliable estimates of the rotational motion variables for 1500 s long occlusion, which are as good as the estimates from Benchmark (conventional UKF without the long term occlusion). On the contrary, the estimates from the conventional UKF algorithm diverged once occlusion begins. It was also observed that both the UKF and UKF-GP algorithms can provide good estimates of the inertia parameters and the feature positions in the body frame during occlusion, except for UKF-GP at 0.025 Hz. UKF-GP cannot perform well in 50% of the runs for the tumbling frequency of 0.025 Hz because of the lack of sufficient training data. Overall, it is evident from the results that the UKF-GP algorithm outperforms the conventional UKF algorithm by a large margin in estimating the rotational motion variables and the inertia parameters of a non-cooperative tumbling target object under long-term occlusions.

In the future, we plan to enhance the UKF-GP algorithm so that both the relative rotational and translational motion variables of a target space object can be estimated during long-term occlusions. Also, we aim to make the UKF-GP algorithm compatible with a varying number of features instead of a fixed number of features, as assumed in this study. Finally, we will validate the effectiveness of the UKF-GP algorithm in an experimental setup.