An Application of the Kalman Filter Recursive Algorithm to Estimate the Gaussian Errors by Minimizing the Symmetric Loss Function

: Kalman ﬁltering is a linear quadratic estimation (LQE) algorithm that uses a time series of observed data to produce estimations of unknown variables. The Kalman ﬁlter (KF) concept is widely used in applied mathematics and signal processing. In this study, we developed a methodology for estimating Gaussian errors by minimizing the symmetric loss function. Relevant applications of the kinetic models are described at the end of the manuscript.


Introduction
Linear quadratic estimation (LQE) is a statistical method with many technological applications. The most common applications of LQE are for navigation, guidance, and the control of vehicles [1]. Other applications [2,3] are based on the lag between the issuing signals and receiving a response, in which the Kalman filter (KF) makes assessments of the condition of the framework and issues directions.
In general, a KF assumes that the noise is normally (Gaussian) distributed and the observation models are linear [4]. Furthermore, the filter gives the posterior conditional probability estimate in the case when the errors follow a Gaussian distribution.
Over the past few decades, generalizations of and extensions to the KF method have been established, such as the unscented Kalman filter (UKF) and the extended Kalman filter (EKF), which both work on nonlinear systems [5,6]. EKF is probably the most widely used estimation algorithm for nonlinear systems [7].
The KF uses a dynamic model, known control inputs, and multiple consecutive measurements to make an estimate of its varying quantities, which leads to a better estimation than if they were obtained by using one measurement alone [8][9][10].
The Kalman filter is a productive recursive filter that evaluates the inward condition of a direct powerful framework from a progression of noisy estimations. It is utilized in a wide scope of econometric and engineering applications ranging from radar and computer vision to estimations of basic macroeconomic models [11,12], and is an important tool in control system engineering and control framework design. Together with the linear quadratic regulator (LQR), the KF determines the solutions of a direct quadratic Gaussian control issue.
Other related works in the mathematics field have been carried out as well [13][14][15]. The most representative works were [16][17][18]. Some mathematical researchers [19,20] proposed the use of a KF for the elimination of noise. An interesting approach [21] uses the extended KF for linearizing the nonlinear systems while improving the applicability of these methods. Anderson et al. [22] presented the theory and applications of Kalman filtering in a unified and organized fashion, which encompassed major contributions by a substantial number of researchers of this topic. Jazwinski [23] took the Bayesian point of view regarding filtering, while other economists [24,25] set a procedure for the frequency domain design of statically optimal filters. Starting from what is already known in this area, the novelty of this study consists of using a recursive algorithm of the Kalman filter, the semigroup of the Hamiltonian matrices, and the contraction property to estimate the Gaussian errors. The main difference between our work and classical approaches, such as EKF and LQE, is that it handles the time-varying measurement uncertainty. For this, the covariance matching approach is used to update the observation covariance matrix.
The rest of this paper is organized as follows. The methodology and the formulation of the system equations are described in Section 2. Section 2 also describes the KF model, the general case of the KF, and the Gaussian case. The subsequent section, Section 3, describes the semigroup of the Hamiltonian matrices and the contraction property. The results, including the need for the KF-based techniques under different conditions for the numerical and experimental study, are presented in Section 3. Based on the observed results, some conclusions and areas for future work, as well as possible applications of the proposed methodology, are identified in the last section of the paper.

The Kalman Filter Model
The KF is based on the following mathematical theory. Consider B m = 1, 2, 3, . . . data observed from p × 1 vectors. B m is a sequence that is presumed to have the following distribution: In this regard, the dynamic representation is as follows: D m = Z m A m , m = 1, 2, 3 . . . . whose factors are defined below. Next, we continue with an observed data model: Equation (1) is related to the p × 1 vector of B m data points, with an r × 1 A m vector. In this case, Z m is a p × r matrix, while ϕ m is a p × 1 vector with E[ϕ m ] = 0 and Var[ϕ m ] = M m .
The vector A m is not straightforwardly observable, but we could assume that it follows a Markov chain process of order 1, i.e., A m = X m A m−1 + θ m , m = 1, 2, 3, . . .
The states' behavior is described by Equation (2). In this case, X m is an r × r matrix and θ m is an r × 1 error vector, which is not correlated with θ m since E[θ m ] = 0 and Var[θ m ] = P m .
We also assumed that the matrices X m , M m , P m , and Z m , as defined before, are not random matrices. The above two statements (1) and (2) reveal that A m could be constructed as a linear combination of θ d and ϕ d , for d ≤ m − 1 and A 0 for any m.

The General Case of the Kalman Filter
From the least-squares class of estimators, the KF is a recursive estimator. The matrices X m , M m , P m , and Z m are presumed to be known for any m. Consider a sequence Γ m , which includes all the data observations recorded until moment m, inclusively.
Consider now an optimal estimator x m|m of A m based on the data observations of Γ m . The r × r matrix of the mean of the squared errors (EPM) of x m|m is: where " ' " stands for the transpose of a matrix. Considering x m−1|m−1 and Q m−1|m−1 , the estimators x m|m−1 and Q m|m−1 of A m and Q m|m , which are based on the data points of Γ m−1 , can be calculated from the prediction formulas: and i.e., Q m|m−1 = X m Q m−1|m−1 X' m + P m , for m = 1, 2, 3, . . .
By (4) and (5), the average of the connected terms of the multiplication is zero. Since A m−1 is a linear combination of the terms θ d and ϕ d , d ≤ m − 1, and the A 0 term is also not correlated with ϕ m .
The equations (7) and (8) above represent the KF. Given A 0 and Q 0 , the KF gives the optimal estimator of the state when new data points are accessible.

The Gaussian Case
Now, we consider the following system: where • X m and Z m are deterministic r × r and p × r matrices, respectively, and • A m ∈R r , θ m ∈R q , ϕ m , and B m ∈R p are random variables. Now, we assume that the pair of vectors (θ m ,ϕ m ), m ≥ 1, are independent and have a Gaussian distribution with an average of 0 and a covariance matrix equal to J, where J is the unit matrix and A 0 has a Gaussian distribution with an average of a 0 and a covariance matrix Q 0 . We then assume that the determinant of X m is not zero, m ≥ 1. For any m ≥ 1, let an N m σ-algebra be generated by B 1 , . . . , B m [26] and If N m is known,Â m is the best estimator of A m and Q m is the conditional variancecovariance matrix of the errors. Now, we consider Q, respectively Q 0 , which is the set of symmetric matrices that are non-negatively defined (respectively positively defined) and of r × r dimensions. For any Q ∈ Q and m ≥ 1, we then define: where S m = Z' m Z m and J is the unit matrix of order r. It follows that ξ m takes P 0 into itself. The basic result shown above is related to the fact that the applications ξ m , m ≥ 1, are contractions in Q 0 if we enhance Q 0 with the Riemann measurement δ [27], which is invariant to conjugacy. Furthermore, the contractions could be strict or uniform, depending on the controllability and observability properties of the above system (9).
To demonstrate the results in this view, we considered the symplectic group of matrices. The matrices are invariant to P 0 . In this view, they could be considered a generalization of the Perron-Frobenius theorem. In some sense, the study of Bougerol [28] is elementary.
The symplectic matrices utilize some of the already existing ideas [29]. The semigroup Σ was induced by Wojtkowsky [30] in a similar context. Finally, we note that the Riccati transformation [31] has a contractibility property, which is a generalization of the positive matrices under a Hilbert space metric [32].

A Recursive KF Algorithm Application
We considered an object moving along a straight line. Its movements are captured by a video camera. The object is moving along a straight line, with random acceleration a t , and ∆ t is the time interval. Its location is described by the x-coordinate. We also assumed that the acceleration has a Gaussian distribution with zero mean and constant variance (σ 2 a ). The position of the object, denoted as y t , is also noisy (Gaussian with zero mean and constant variance). We denote the variance as σ 2 y . The state vector of this motion is given by v t = (x t ,v t ), where the speed is the first derivative of the distance, i.e., v t = .
x t . We also assumed that there was no process control and get: where • F t is a known state transition matrix applied to the t − 1 state x t−1 , • Q t is the variance matrix, and • w t is a process noise vector, which has the joint distribution as a multivariate Gaussian with a variance matrix Q t and µ i,t = E[w i,t ] = 0 for i = 1, n.
Then, we get: This means that F t , Q t , and G t are independent of t. Hence, we will call them just F, Q, and G. To measure the object's position at time t, we get: where • v t is the observation noise and • H t is an observation matrix.
Hence, the variance matrix is: We denotex t|t as the estimate of state x t at time t and P t|t as the variance matrix of the error x t −x t|t . Our goal was to minimize P t|t .
The initial position of the object isx 0|0 = (0, 0) T . If this position vector is known accurately, then the variance matrix is zero, i.e., otherwise, we will have that: where a is real and suitably large, a > 0. When t = 1, we first predictedx 1|0 and computed its variance matrix P 1|0 by using the prediction equations:x where B t is the control matrix, which is applied to a control vector u t . We then calculated the auxiliary data S 1 and z 1 from the equations: where R t is the variance matrix of a multivariate Gaussian distribution vector v t . Then, we could compute the optimal Kalman gain K 1 and the position vectorx 1|1 using the equations:x Finally, we could compute P 1|1 to prepare for time t = 2 by using the following equation: which minimizes the variance matrix P t|t .

EKF Recursive Algorithm Application
We extended our previous example of an object moving along a straight line by adding the orientation of the vehicle θ to the position vector (x,y). Then, we estimated the state vector p, i.e., p = (x,y,θ). The kinematic model is given by the following system: where ∆ t is the time interval, while v and w are the speed and the yaw rate of the vehicle. We also denotev andŵ as the speed and yaw rate measurements. We assumed that their errors followed the Gaussian distribution as ∆v t ∼ N 0, σ 2 v t and ∆w t ∼ N 0, σ 2 w t . Now, suppose that the vehicle is equipped with a GPS, and the GPS measurements are denoted by vector z with a measurement model given by: where γ ∼ N 0, σ 2 γ is the measurement error that follows a Gaussian distribution with zero mean and variance σ 2 γ . This is an EKF model, which could be locally linearized as follows: where u t = (v t , w t ).
As we can see, the matrices A p t−1 , u t and B p t−1 , u t of the state evolution function are Jacobian with respect to p t−1 and u t , respectively.
Hence, we have a locally linearized EKF model. The prediction of the state vector is:
The yaw rate measurements and the speed measurements were synthesized according toŵ t ∼ N w t , σ 2 w andv t ∼ N v t , σ 2 v . The GPS measurements were synthesized according toẑ t ∼ N p t , σ 2 γ . Then, the KF was applied to the synthetic data and estimates of the state of the vehicle were obtained. The results can be seen in Figure 1, in which:

•
The red line represents the estimated vehicle trajectory. The GPS measurements were synthesized according to ̂~( , 2 ). Then, the KF was applied to the synthetic data and estimates of the state of the vehicle were obtained. The results can be seen in Figure 1, in which:


The red line represents the estimated vehicle trajectory.  The black line stands for the ground truth of the vehicle trajectory.  The blue crosses are the GPS measurements of the position of the vehicle. As we can see from the above figure, the estimated trajectory of the vehicle was smoother than the jumping measurements. Figure 2 shows the results of 100 Monte Carlo trials. As we can observe, the measurement errors were larger than the estimated errors, just like in the previous example.  As we can see from the above figure, the estimated trajectory of the vehicle was smoother than the jumping measurements. Figure 2 shows the results of 100 Monte Carlo trials. As we can observe, the measurement errors were larger than the estimated errors, just like in the previous example.
We also calculated the orientation estimate errors and the results of the same 100 Monte Carlo simulations are shown in Figure 3. As we can observe, these errors varied around 0.05 radians.
Hence, we concluded that, without using estimations in this application, we could not have any estimate of the vehicle's orientation except a random guess, which could be as large as π/2 radians.
This last example shows another important utility of the estimation, i.e., a proper estimation of the Gaussian errors could reveal the state part, which is not directly measurable. As we can see from the above figure, the estimated trajectory of the vehicle was smoother than the jumping measurements. Figure 2 shows the results of 100 Monte Carlo trials. As we can observe, the measurement errors were larger than the estimated errors, just like in the previous example. We also calculated the orientation estimate errors and the results of the same 100 Monte Carlo simulations are shown in Figure 3. As we can observe, these errors varied around 0.05 radians.  Hence, we concluded that, without using estimations in this application, we could not have any estimate of the vehicle's orientation except a random guess, which could be as large as π/2 radians.
This last example shows another important utility of the estimation, i.e., a proper estimation of the Gaussian errors could reveal the state part, which is not directly measurable.

Discussion and Conclusions
The Kalman filter is an adequate recursive filter that evaluates the internal capacity of a linear dynamic system (LDS) using a series of error estimates. Possible applications of the KF are in economics, biology, and the environmental sciences, and it is an important topic in control systems engineering and control theory. Together with the LQR, the KF solves the linear quadratic Gaussian (LQG) control problem. The linear quadratic Gaussian controller, linear quadratic regulator, and Kalman filter are solutions to some of the most important problems in control theory.
In many applications, the few "detectable" measured parameters are smaller than the internal state. In this situation, the Kalman filter could estimate the entire internal state by combining a series of measurements.

Discussion and Conclusions
The Kalman filter is an adequate recursive filter that evaluates the internal capacity of a linear dynamic system (LDS) using a series of error estimates. Possible applications of the KF are in economics, biology, and the environmental sciences, and it is an important topic in control systems engineering and control theory. Together with the LQR, the KF solves the linear quadratic Gaussian (LQG) control problem. The linear quadratic Gaussian controller, linear quadratic regulator, and Kalman filter are solutions to some of the most important problems in control theory.
In many applications, the few "detectable" measured parameters are smaller than the internal state. In this situation, the Kalman filter could estimate the entire internal state by combining a series of measurements.
The Kalman filter could be adapted to a wide variety of economic applications. It is mainly an ordinal least-square (OLS) procedure that gives the minimum mean square estimators (MMSE) under the Gaussian assumption. Even when the normality assumption is waived, the Kalman filter minimizes any symmetric loss function. The KF is not only used directly in economic problems but it could also be used as a background to other estimation techniques, such as the estimation of Markov switching models or the quasi-maximum likelihood estimation procedure.
In the environmental sciences, the Kalman filter has wide applications in the areas of the pollution of aquatic systems, measuring atmospheric pollution, environmental policy management, and forecasting hurricanes [33,34]. The KF could also be used to estimate the temporal variations in associations between the radial growth of black oak and physiologically based climate indices [35,36]. In biology, biological samples and bacteriological infections could be detected using a multi-process Kalman filter to provide an online estimate of the inflection point of changes in the growth of an organism [37].
A wide collection of Kalman filters have been developed in the past few decades from the original formulation of the Kalman filter. The information filter, Schmidt's extended filter, or the Kalman-Bucy filter are a wide variety of filters that were developed by Thornton, Bierman, and many others. Probably the most used type of Kalman filter is the phase-locked loop (PLL), which is now omnipresent in television sets, radios, satellite communications receivers, and almost any other communications equipment. In this paper, we presented the KF model, the general case of the KF, and the Gaussian case. The semigroup of the Hamiltonian matrices and contraction property were used in the KF algorithm to estimate the Gaussian errors.
The main result of this study was the use of a recursive algorithm of the Kalman filter to estimate the Gaussian errors.
Several possible variations of our approach exist. Model herring inflow as a state variable and model herring inflow as pure noise with a non-zero mean or model noise as pure white (non-geometric) noise assume perfect observations of the control variables (catch and inflow). These variations were all tested and led to only small changes in the recursive algorithm fit and small variations in the parameter estimates.
The main limitation of this study came from the fact that there were not many applications of the algorithms described in the previous sections. Future work, as well as possible applications of the methodology, could be applied in mathematics, physics, and engineering research areas. Moreover, a future research topic is how to treat more general noise terms, for example, terms including unknown parameters.