Online Estimation of Allan Variance Coefficients Based on a Neural-Extended Kalman Filter

As a noise analysis method for inertial sensors, the traditional Allan variance method requires the storage of a large amount of data and manual analysis for an Allan variance graph. Although the existing online estimation methods avoid the storage of data and the painful procedure of drawing slope lines for estimation, they require complex transformations and even cause errors during the modeling of dynamic Allan variance. To solve these problems, first, a new state-space model that directly models the stochastic errors to obtain a nonlinear state-space model was established for inertial sensors. Then, a neural-extended Kalman filter algorithm was used to estimate the Allan variance coefficients. The real noises of an ADIS16405 IMU and fiber optic gyro-sensors were analyzed by the proposed method and traditional methods. The experimental results show that the proposed method is more suitable to estimate the Allan variance coefficients than the traditional methods. Moreover, the proposed method effectively avoids the storage of data and can be easily implemented using an online processor.


Introduction
Inertial sensors are valuable sensors for navigation of aircraft systems, vehicles and strategic weapons [1]. However, stochastic errors, inherently present in inertial sensor outputs, significantly affect the performance of inertial sensors. To eliminate the effect of stochastic errors, they should be modeled and identified to properly compensate or filter them before integrating into the navigation system [2][3][4]. In general, the noise analysis methods used for inertial sensors include online and offline estimation methods. In the offline methods, frequency and time-domain approaches have been used to model the stochastic errors of inertial sensors. As a frequency-domain method, power spectral density (PSD) is commonly used to investigate the stochastic errors of inertial sensors. Although, the PSD-based method is straightforward to estimate the transfer functions of stochastic errors, it is difficult for non-system analysts to understand [5][6][7]. As a time-domain analysis technique, the Allan variance is a simple and useful method in determining the characteristics of the underlying random processes causing the data noise. It has been widely used for identifying stochastic processes such as quantization noise, white noise, correlated noise, sinusoidal noise, random walk, and flicker noise in inertial sensors [8][9][10]. Recently, modified Allan variance methods such as sliding average Allan variance [11] and fully and not fully overlapping Allan variance [12] have been developed. However, these methods are time-consuming, offline, and error-prone [13].
Compared to offline methods, online methods have rarely been studied. One online method is reported in [14,15], and another is reported in [16,17]. In [14,15], an equivalent ARMA was used to model the MEMS IMU stochastic errors to obtain a linear Gaussian state-space model, and then a recursive EM algorithm proposed by Elliot and Krishnamurthy [18] was used. The method proposed in [14,15] does not require the storage of any data and can be implemented using an online processor, however, it is only valid for the stochastic errors driven by white noise. Therefore, the quantization noise cannot be estimated directly [19]. Moreover, transformations during the modeling are complex, particularly for the estimation of four parameters. The main advantage of the method proposed in [16,17] is that it models the dynamic expression of Allan variance using exponentially weighted moving average algorithm to solve the complex transformations of a linear state-space model reported in [15]. The method also can estimate any stochastic errors present in raw data. However, it still has two major drawbacks: (I) The additional error can be introduced in the measurement equation by the exponentially weighted moving average algorithm; and (II) the Sage-Husa adaptive Kalman filter used to estimate Allan variance coefficients may cause filter divergence while estimating process noise covariance matrix Qk and measurement noise covariance matrix Rk simultaneously [20]. Finally, it must be pointed out that the method reported in [16,17] is out of the scope of this paper, because this study focuses on static gyros rather than onboard gyros.
To reduce the complexity of modeling and estimate all five Allan variance coefficients in real time, a new online estimation method different from the above methods is proposed in this paper. In the proposed method, the recursive expressions of Allan variance were modeled directly instead of using exponentially weighted moving average algorithm to obtain an accurate nonlinear state-space model and implemented by a neural-extended Kalman filter (NEKF) algorithm to estimate the Allan variance coefficients. Because the state equation is directly modeled by random errors and zero-mean Gauss white noise, the measurement equation is a recursive expression of Allan variance, and a NEKF algorithm is used to estimate the Allan variance coefficients. The proposed method avoids the limitations of the existing online methods. The experimental results show that the proposed method has a simple modeling and can estimate five Allan variance coefficients online. This paper is organized as follows: In Section 2, the sources of the stochastic errors of inertial sensors are described. Section 3 reviews the existing online estimation methods based on a linear state-space model. The proposed method comprising the new nonlinear state-space model of Allan variance coefficients and NEKF is described in detail in Section 4. Section 5 discusses the experimental results in two subsections. In the first subsection, an ADIS16405 MEMS sensor was used as the stochastic error source to evaluate the performance of the Allan variance method, existing online estimation methods, and proposed method. In the second subsection, another experiment was conducted to estimate Fiber Optic Gyro (FOG) stochastic errors to verify that the proposed method can estimate the coefficients of five basic stochastic errors simultaneously without any limitation. Finally, the conclusion and future work are summarized in Section 6.

Stochastic Error Sources in Inertial Sensors
The aim of this section is to discuss the five basic noise terms and give their corresponding differential equations. The stochastic model of inertial sensors is shown in Figure 1. The five basic noise terms in Figure 1 are quantization noise, angle random walk (ARW), bias instability, rate random walk (RRW), and rate ramp. The definitions and their detailed derivations are given in [21]. Quantization noise: This is one of the errors introduced into an analog signal by encoding it in a digital form. It represents the minimum resolution level of the sensor. The Allan variance of quantization noise can be expressed as follows: where Q is the quantization noise coefficient, and τ is the sample interval. ARW: This is a high-frequency noise and characterized by a white-noise rational spectrum on gyro rate output voltages. The differential equation and Allan variance of ARW can be expressed as follows: where N is the ARW coefficient, and v 1 (t) is the unit white noise. Bias instability: This noise is originated from the electronics or other components that are susceptible to random flickering. It is also known as flicker noise and approximated by the first-order Gauss-Markov process as follows: where D is the differential operator, β is the reciprocal correlation time and can be determined by autocorrelation, B is the flicker noise parameter, and v 2 (t) is the unit white noise. The Allan variance of bias instability can be expressed as follows: RRW: This is a random process of uncertain origin, possibly a limiting case of an exponentially correlated noise with a very long correlation time. It is associated with the PSD rate. The differential equation and Allan variance of RRW can be expressed as follows: where K is the RRW coefficient, and v 3 (t) is the unit white noise. Rate ramp: The error terms considered so far have random character. This is also probably because of a very small acceleration of the platform in the same direction and persisting over a long period of time. Because this noise has nonrational spectra, the second-order Gauss-Markov process is used as the approximation: where R is the ramp noise parameter, ω 0 is the undamped natural frequency of this second-order system and needs to be determined, and v 4 (t) is the unit white noise. The Allan variance of ramp noise can be expressed as follows: The Allan variance is a method of representing root-mean-square (RMS) random drift error as a function of averaging times [22]. As shown in [5], the above stochastic errors can be identified from the standard Allan variance plot. The Allan variance coefficients and the slope of the corresponding curves are shown in Table 1.

Online Estimation Methods Based on Linear State-Space Model: A Review
An online estimation method based on a linear state-space model was first introduced by Ford and Evans in [14]. Only two random errors, ARW and RRW, were estimated using this method. Based on this method, another online estimation method was developed to estimate three random errors, ARW, RRW, and bias instability, by Saini and Rana [15]. Although there are differences between these methods, in some sense, the method proposed in [15] is an extended version of that proposed in [14]. For convenience, the method proposed in [15] was used as the existing method in the rest of this paper. In this paper, the proposed online estimation method was compared to the existing method. To compare the complexity of the modeling process and select between the existing and proposed methods in this paper, the existing method was reviewed first. The existing method mainly contains two steps: The first step is to model the stochastic errors to obtain a linear state-space model, and the second step is to use the finite-dimensional filters to estimate the coefficients of stochastic errors. Herein, the two steps are introduced in Subsections 3.1 and 3.2. The detailed derivations of this method are reported in [15].

Linear State-Space Model
The main steps of modeling the stochastic errors in a discrete linear state-space model can be described as follows: Step 1. Model an equivalent ARMA model [23] driven with a single white noise as follows: where Y(t) is used to represent the mixture of yrrw(t), yf(t), and yrr(t).
Step 2. Substituting Equations (4), (6) and (8) into Equation (10), we obtain The left hand side of Equation (11) is an equivalent AR model, and the right hand side of Equation (11) is an equivalent MA model. Moreover, the Equation (11) can be rewritten as: where the superscript of Y(t) and r represent the differential of Y(t) and r. c1, c2, c3, c4 and θ0, θ1, θ2, θ3 are coefficients of equivalent ARMA model, the r is the white noise.
Referring to [15] to solve Equation (12), here we only give the results as follows: Step 3. The Equation (12) can be written as a linear state space mode as follow: where X(t) is continuous-time state vector, and the   where k = 1, 2, …, Xk is state vector, and Yk is the measurement. The Φk is state-transition matrix, and Θk is the measurement matrix. The Gk and Ψk are the process and measurement noise matrix, respectively. Here ξk and υk are discrete-time noise, and they are white, zero-mean, uncorrelated.

New Finite-Dimensional Filters
The new finite-dimensional filters were proposed by Elliott and Krishnamurthy in 1999 [18]. In a linear dynamic system, they were used with expectation maximization (EM) algorithm to yield the maximum likelihood estimation. Compared to the standard KF-EM algorithm, the new finite-dimensional filters have two main advantages. The first advantage is that the memory requirements are significantly reduced, and the second advantage is that it can be easily used in a multiprocessor system. The main process of this recursive filter is shown below, and the detailed derivations are reported in [18]. The linear state-space model that the states xk are observed indirectly via the observations yk can be written as [15,18]: where Ak is state-transition matrix, and Ck is the measurement matrix. The Jk and Vk are the process noise and measurement noise matrix, respectively. Note that k Â and k Ĉ are the estimate of Ak and Ck, respectively. Both ξk and υk are noise that have been defined below Equation (15). The Kalman filter can be expressed as [18]: where  k P represents the a priori state covariance, and  k P represents the a posteriori state covariance. The  k x represents the a posteriori state estimate, and the superscript T is a symbol that represents a transposed matrix. The , M = 0, 1, 2 as follows: where i, j ϵ {1, 2, …, m}, n ϵ {1, 2, …, n}, ei, ej, en denote the unit column vectors in the ith, jth, and nth columns, respectively. Here Tr [] denotes the trace of matrix, and  , denotes the inner product.
The initializations of the above equations are defined as follows:  can be expressed as follows: As shown in [15], the major drawback of this method is that the quantization noise cannot be directly incorporated into the error model because the equivalent ARMA model used here is driven with white noise [23].

New Method
As shown in Subsection 3.1, the modeling of stochastic errors in the existing method is indirect. This is because an equivalent ARMA theory was used to model the stochastic errors to obtain a linear state-space model. Moreover, the estimation process of the existing online estimation method is complex, particularly for the estimation of four parameters. In this situation, it is difficult to obtain the solution of Equation (12) and tedious to calculate Equations (23)-(33) described in Subsection 3.2. Focusing on the disadvantages and inspired by the expression of δ 2 total (τ), which is nonlinear in nature, a simple and direct online estimation method based on a nonlinear state-space model and NEKF is proposed in the following two subsections.

New Nonlinear State-Space Model of Allan Variance Coefficients
To estimate the Allan variance coefficients directly in real time, δ 2 total (τ) should be a dynamic expression at discrete-time k. The recursive algorithm is the best choice for online estimation because the computation can be carried out as soon as a new sample arrives. The detailed derivation of the recursive formulation for Allan variance is shown below: Assume  is the total number of data points. According to [5,6], the two steps of Allan variance computation can be expressed as follows: where l is the length of the data cluster, and l L /   is the number of data clusters. Recall that, the τ is sample time that has been defined in Section 2, and   l m  is the correlation time. Equation (41) shows the average of each cluster, and each of them can be rewritten in the recursive form as follows: where k ≥ 2 and   When l = 1, the Allan variance of each data point can be calculated using Equation (44). Assuming that the Allan variance coefficients are composed of true values and the zero-mean Gauss white noise. Therefore, they can be expressed as follows: where

Neural-Extended Kalman Filter
NEKF is known as an online adaptive estimation system developed by incorporating the training into the state estimate [24,25]. In fact, NEKF is an extension of the neural Kalman filter (NKF), which has been used to overcome the two major limitations related to the utilization of KF for INS/GPS integrated system: (I) Accurate stochastic model for each of the sensor errors has to be accurately predefined; (II) Prior information about the covariance values of both inertial and GPS data as well as the statistical properties of each sensor system has to be accurately known [26,27]. Although the nonlinear state-model composed of Equations (45) and (47) look like a model that a EKF algorithm could be applied, the EKF cannot directly used in this nonlinear state-model. Because Equation (47) is nonlinear with Q, N, B, K and R, and there is not any statistical measurement noise in it. The high-order items in Taylor expansion can be seen as virtual measurement noise to compensate linearization error, however, the time-variable statistic of this noise is still unknown. The NEKF that trains neural network online can be used in nonlinear system with random noise [28], therefore, it was used to estimate the Allan variance coefficients in this study.
In a NEKF, an extended Kalman filter (EKF) is used as an estimator and a training paradigm. As an estimator, EKF estimates the states and the input and output weights of neural networks. As a training paradigm, it is driven by the same residuals as the state estimator and ensures that the residuals are as small as possible; it approximates the difference between the prior model used in the prediction steps of the estimator and the actual model dynamics [29]. In this application, the augmented state vector of NEKF contains both the state estimates and the weights (input and output) of the neural network. NEKF can improve the performance by estimating the weights of neural network, which in turn is used to modify the state estimate predictions of the filter as the measurements are processed. Based on this point, the NEKF algorithm is presented to online estimation Allan variance coefficients is presented. The main flow of NEKF are shown below: The nonlinear system can be modeled as [28]: where xk is the state vector, f(xk) is state function, zk is measurement vector, and h(xk) is the output function. The k and ζk represent the random noise, respectively.
Step 1. The state xk is augmented as follows: where Wk is the weights of neural network. It is composed of input weights ηk and output weights λk. Note that the neural network used in this study comprises only one hidden layer. Suppose that the number of xk is q, the number of hidden node [30] is p, and the number of neural network output is u. Therefore, the dimension of ηk is (q × p) × 1, the dimension of λk is (p × u) × 1, and the dimension of Wk is (q × p + p × u) × 1.
The NN(xk, ηk, λk) function used in this study is: Note that the states xk are considered the input to the neural network, whereas the weights Wk are the "parameters" of function approximator [28] that can be used to model something in the noise. More informance about the NN(xk, ηk, λk) can be found in [24][25][26][27][28][29].
Step 2. The general equations of the NEKF are defined as [26]: where Fk is state Jacobian, and Hk is measurement Jacobian. The 0 represents zero vector, k K is the Kalman Gain,  k P is the a priori state covariance, and  k P is the a posteriori state covariance. The  k x is the a priori state estimate,  k x is the a posteriori state estimate, k R is the measurement noise covariance, and k Q is the covariance of the process noise. The I represents the unit matrix.

Experimental Results
To evaluate the performance of the proposed method, the static data of an ADIS16405 MEMS sensor and low-precision FOG were collected. The reason this MEMS sensor was tested in the first experiment is that this sensor exhibits ARW, RRW, and bias instability [6,15]. Moreover, the existing method is also valid to estimate ARW, RRW, and bias instability; therefore, it can be used to compare the performance of the proposed method in this experiment. FOG, as a typical inertial sensor with five basic stochastic errors, was also used to verify the effect of the proposed method. As shown in [19], the existing method is invalid to estimate quantization noise. If the proposed method can accurately estimate the Allan variance coefficients of FOG, then it can be shown that the proposed method is more suitable to estimate the Allan variance coefficients of inertial sensors than the existing method. In the first experiment, we will use the proposed, existing, and traditional Allan variance methods to estimate the stochastic errors of the ADIS16405. However, because of quantization noise considered in the second experiment, we used the proposed and traditional Allan variance methods to analyze the stochastic errors of FOG. In each experiment, the main steps of the proposed method can be implemented as follows: Step 1. Model the nonlinear state-space model based on the main stochastic errors.
Step 2. Make a preliminary estimate of the coefficients of the main stochastic errors.
Step 3. Use NEKF to estimate the Allan variance coefficients in real time. Figure 2 shows the actual MEMS sensor ADIS16405 which was used in our experiment. The 5-h static data were collected from the ADIS16405 at room temperature at 100 Hz. The raw data of gyro X, gyro Y, gyro Z and their corresponding Allan standard deviation plots are shown in Figures 3-8.        According to the Allan variance plot analysis, three main stochastic errors should be considered for the gyro in this test, namely, ARW, bias instability, and angular RRW with the mean-square values of N, B and K, respectively. Therefore, the linear state-space model of gyro can be deduced as follows:

Estimation for MEMS Experimental Data
The unified ARMA model Equation (10) consisting of ARW and bias instability can be written as follows: Substituting Equations (4) and (6) into Equation (58), the unified ARMA model can be written as follows: According to Subsection 3.1, the Equation (59) also can be rewritten as follows: where σ1, σ2 and σ3 are coefficients of equivalent ARMA model. According to [15,19], the differential equation Equation (60) can be solved as follows: The corresponding state-space form of the above differential equation can be written as follows: The discrete form of Equation (62) is: where τ is the sampling interval that has been defined in Section 2. Note that Equation (63) is the linear state-space model that can be implemented by new finite-dimensional filters.
Compared to the complex process of modeling a linear state-space model, the state equation of nonlinear state-space model for Gyro can be directly written as follows: is to use traditional methods to analyze the stochastic errors of sampling gyro, and the second step is to select the initial values based on the results of the first step to estimate the same type of gyros. Based on the prior analysis of ADIS 16405, the initialization of N, B and K were taken as 1.5 (°/h 1/2 ), 20 (°/h) and 50 (°/h 3/2 ), respectively, in this experiment. To show the initial change clearly in the estimation process, only 20,000 samples for the estimation of N, B and K are shown in Figures 9-17.         As shown in Figures 9-17, the curves of the existing method (red curve) and proposed method (blue curve) are convergent. Moreover, the convergent values of the two online estimation methods are close to those estimated by the Allan variance method. Therefore, both the existing and proposed online estimation methods can accurately estimate the coefficients of ARW, Bias Instability, and RRW in same setting conditions.
To evaluate which method (algorithm) is better, two aspects were usually considered: (1) speed and (2) accuracy. In this paper, the proposed and existing methods are online estimation methods, and they estimate the Allan variance coefficients in real time, indicating that the computation can be carried out as soon as a new sample arrives. Therefore, the speed was not compared directly here. The stability of filter used in those methods was used to evaluate which of them is better in this study. Figures 9-17 show that although the trends of the two online estimation methods are basically identical, the estimation curves of existing method fluctuated significantly in the estimation process of B and K. Moreover, the standard deviation sizes of proposed method shown in Table 2 are slightly smaller than those of existing method. Therefore, the experiment suggest the NEKF in proposed method might have advantages over the new finite-dimensional filters in existing method. As to the accuracy of the methods, Table 2 summarizes the parameter estimation results for the Allan variance, existing, and proposed methods. For a better visualization of the performance comparison, the performance indicator (|method A − method B|/method A) × 100% was defined to demonstrate the percentage of difference between methods B and A. Although the results of the Allan variance method can be affected by many factors, it is the IEEE Std [5] that is used to analyze the stochastic errors of inertial sensors. Moreover, in the online estimation of Allan variance coefficients, the results of the Allan variance method have been used as the reference values and compared to the online estimation methods proposed in [14][15][16]. Herein, the Allan variance method was also used as the reference method (method A), providing a baseline for comparison. According to Table 2, the performance degradation values of the proposed method over the Allan variance method are smaller than those of the existing method against the Allan variance method for N, B, and K, respectively.
It can also be seen that the linear state-space model was modeled using Equation (58) to Equation (63), while the nonlinear state-space model can be directly modeled in Equations (64) and (65). Therefore, in some sense, the complexity of the modeling of the proposed method is lower than that of the existing method.

Estimation for Fiber Optic Gyro (FOG) Experimental Data
To verify that the proposed method can estimate all the five Allan variance coefficients of inertial sensors simultaneously without any limitations, the stochastic errors of the actual low-precision FOG were analyzed in this experiment. The actual low-precision FOG used in our experiment is shown in Figure 18. The 3-h static data were collected from the #1 FOG and #2 FOG at room temperature with 100 Hz. The raw data of #1 FOG and #2 FOG, and their corresponding Allan variance plots are shown in Figures 19-22.         Table 3. Figures 23-32 show that the estimation curves of proposed method (blue curves) are all convergent curves. Moreover, Figures 23-32 also show that the coefficients of five basic stochastic errors converge to the results of Allan variance method.
From Table 3, it can be seen that the mean values of five basic stochastic errors estimated by the proposed method are close to their corresponding values analyzed by Allan variance method. Therefore, this experiment prove that the proposed method can not only estimate ARW, bias instability, RRW and drift rate ramp but also valid to estimate quantization noise. According to the above two experiments, the proposed method has a simple modeling process, and it can be used to estimate all the five basic stochastic errors. Moreover, NEKF was used in the proposed method, resulting in a better estimation results.

Conclusions and Future Work
In this paper, a new online estimation method based on a nonlinear state-space model and NEKF is proposed: the model was used instead of the traditional linear state-space model and complex finite-dimensional filter algorithm. In examination of ADIS 16405 gyro data, the proposed method performed favourably compared to the existing online method, relative to the baseline estimates obtained from the Allan variance method. In the actual FOG gyro data, the proposed method could estimate all the five basic stochastic errors simultaneously. Moreover, unlike the offline methods, the proposed method avoids the storage of a large amount of data and analyzes the Allan variance graph manually.
The success of the proposed method shows an encouraging direction in accurately estimating the Allan variance parameters for inertial sensors with recursive online analysis. However, although the proposed method performs well for static data, the onboard performance is not known. Theoretically, the new method can be used for the autonomous estimation of the Allan variance coefficients for onboard inertial sensors. However, in practice, it should be used carefully because the online estimation method can be affected by the initial values of the parameters, noise mean, and variance. The onboard performance analysis and improvement of the new online method will be studied in the future.