- freely available
Sensors 2012, 12(7), 8877-8894; doi:10.3390/s120708877
Published: 27 June 2012
Abstract: A matrix Kalman filter (MKF) has been implemented for an integrated navigation system using visual/inertial/magnetic sensors. The MKF rearranges the original nonlinear process model in a pseudo-linear process model. We employ the observability rank criterion based on Lie derivatives to verify the conditions under which the nonlinear system is observable. It has been proved that such observability conditions are: (a) at least one degree of rotational freedom is excited, and (b) at least two linearly independent horizontal lines and one vertical line are observed. Experimental results have validated the correctness of these observability conditions.
Inertial navigation systems (INS) have been widely used in many systems, such as ground vehicles, airplanes, helicopters, robotic systems, etc. However, INS drift will lead to exponential growth of errors in the navigation solutions due to the double integration of acceleration signals within inertial navigation computations. Therefore, to overcome such a problem, it is a very common practice to integrate INS with other sensors, which can calibrate the inertial sensor errors. In most outdoor applications, a Kalman filter estimator can be used for optimally combining both IMU and GPS measurements . However, an indoor navigation system cannot use GPS since its signals are not available.
An alternative approach to calibrate INS errors is via the use of other sensors, such as cameras and magnetic sensors. Combining these two sensors to form a vision-aided inertial navigation system (V-INS) has recently become a popular topic of research . By sensing the Earth's magnetic field a magnetic sensor can provide a drift-free heading estimate. Accurate 3-D orientation estimates of a rigid body by inertial/magnetic sensing were exploited in , where the aiding sensors (accelerometer and magnetic sensor) helped mitigate low-frequency gyro drift errors, while, in turn, the signals from the aiding sensors, which are prone to relatively high-frequency errors, are smoothed using gyro data. They are all based on the concept of vector matching, which requires, in principle, the measurements of constant reference vectors (e.g., gravity and the Earth's magnetic field) .
In this paper, we present a matrix Kalman filter (MKF) in which the estimate of the state matrix is expressed in terms of the matrix parameters of the original plant. The MKF has the statistical properties of the ordinary EKF, while retaining the advantages of a compact matrix notation by expressing the estimated matrix in terms of the original plant parameters .
The major contribution of this paper is to elucidate under which conditions a MKF-based nonlinear system for indoor navigation using visual/inertial/magnetic sensors is observable; in other words, the conditions when sufficient information is available for estimating a state matrix that contains, in the present case, the body attitude matrix, the gyro bias vector, relative velocity vector, the dual part of landmark and the magnetic variation superimposed to the magnetic reference vector. For the purpose of orientation determination, an accurately known homogeneous magnetic field in the environment is needed. Magnetic homogeneity is difficult to achieve, especially indoors, due to the presence of iron construction materials in floors, walls and ceilings, or to interferences from various types of equipment. In order to compensate for magnetic variations, a first-order Gauss-Markov vector random process is chosen to model the magnetic variation. To the best of our knowledge, there has been no such observability analysis so far for the integrated navigation systems in 3-D. We have extended the current work for the observability analysis for an orientation system described in , to the 3-D navigation systems based on inertial/visual/magnetic sensors.
2. Sensor Modeling
2.1. Inertial Sensor
Without loss of generality, the navigation reference frame is selected as the local-level frame (North-East-Down). Denote the navigation frame by n, the INS body frame by b, the camera frame by c and the inertial frame by i. Using gyros/accelerometers outputs, the relative velocity vn and the body attitude matrix satisfy the kinematic equations as [1,7]:where ξn is an arbitrary point on the observed line, rb is the lever arm from the IMU to the camera, as shown in Figure 1. ba and bg are 3 × 1 vectors that describe the biases affecting the accelerometer and gyro measurements, respectively. ba can be compensated on time scales up to few hours, using the procedure described in . bg are modeled as random walk processes, driven by the white Gaussian noise vectors nωg. is the skew symmetric matrix of . In the context of MEMS sensors, the component in the gyro output due to the Earth's rotation can be neglected as compared to the sensor errors.
2.2. Visual Sensor
The line point is taken as line representation, which is defined as the intersection of a line feature with a line passing through the image origin that is perpendicular to the line feature. The line point is unique for all lines except the lines passing through the origin. The line point is calculated by Goddard as :
Line features are represented by quaternion. Î = l + εm, where l is the unit direction vector of the observed line and m is related to the position by m = p × 1. In the n-frame:while in the c-frame: where rb can be ignored when the IMU and the camera are mounted closely together.
If relative position ξn is orthogonal to ln, that is, ξn·ln = 0, then we can obtain:
That is to say, the norm of mn is the minimum distance from the vehicle to the observed line. Using Equation (1), we obtain:
For simplification, only vertical lines and horizontal lines are chosen as landmarks. According to Equation (5), we obtain:
A monocular camera is not enough to calculate mz due to its inherent limitation of depth information deficiency. In order to solve this problem, we can use stereo cameras or a monocular camera with height information to obtain mz.
2.3. Magnetic Sensor
A first-order Gauss-Markov vector random process with statistically independent components is chosen to model magnetic variations as follows :where α is a positive constant, and nωh is white Gaussian noise.
3. MKF Algorithm
3.1. Process Model
The evolving state is described as follows:where mn is the relative distance of the vehicle with respect to the observed line, vn is the relative velocity.
The relative position of the vehicle with respect to the observed line, ξn, can be calculated as follows:
Calculate the point of intersection of the observed lines. According to Equation (6), we obtain:
Calculate the translation along the observed line direction from the intersection to the vertical point. According to Equation (8), we obtain:
Because the unique of perpendicular point, the translation along the observed line direction, ρ, is also uniqueness. From Equation (14), we obtain:
Calculate the relative position:
Furthermore, wk can be written as:where h.o.t denotes the terms of the second order Δt2 and higher. Note that the process noise matrix, wk, is state dependent, and that the first-order term is linear in the components of the white Gaussian noise vectors ng.ij denotes a 8 × 8 matrix with 1 at position (ij) and 0 elsewhere; and the noise matrix, Wk, is defined as:
The process noise covariance matrix is:where, as defined by :
L is the 9 × 3 matrix defined as:with e1 = [1 0 0]T, e2 = [0 1 0]T, and e3 = [0 0 1]T.
The process Equation (24) is not linear because in the matrix , r = 2, 3, 4, are the functions of elements of . One way of overcoming this difficulty is to substitute for in the process equation. This substitution yields a pseudo-linear process equation.
3.2. Measurement Model
The matrix measurement equation:where the measurement matrices are: and the measurement noise covariance matrix is given as:
Putting m = 3, n = 8, p = 3, q = 2, μ= 8, v = 2 into Equations described in , we can obtain time update and measurement update equations. However, due to the existence of the input matrix, the time update equation below is not the same as that given in :
Measurement update equations:where ⊗ denotes the Kronecker product , is a 3 × 3 submatrix of the 24 × 6 matrix Kk+1 defined by: and Elj is a 2 × 8 matrix with 1 at position (ij) and 0 elsewhere:
The Kalman filter is not designed to preserve any relationship among the components of the estimated state matrix . It would not preserve the orthogonality. Iterative brute-force orthogonalization is presented for its low computation load . The iterative brute-force procedure is as follows :
Although this algorithm is suboptimal, it is much simpler than the optimal algorithm given by Equation (45). When applied recursively, this algorithm produces a sequence of estimates that converge to the optimal solution of Equation (45) . Moreover, as evidenced by simulations, orthogonality can be usually reached in all practical applications after just one or two iterations of Equation (44):
4. Observability Analysis
A system is observable if its states at a certain time instant can be uniquely determined given a finite sequence of its outputs . Intuitively, observability means that the measurements of an observable system can provide sufficient geometric information for estimating its states. In contrast, the state vector of an unobservable system cannot be recovered regardless of the duration of the estimation process.
In this paper, we study the observability of the nonlinear system describing the visual/inertial/magnetic sensor based navigation process. The system state vector is chosen as follows:where c is defined as:
Equation (3) can be rewritten as:
For arbitrary 3 × 1 vector p, we define:b are the gyro and accelerometer measurements.
Also, note that f0 is a 24 × 1 vector, while f̠1 and f̠2 are both compact representations of three vectors of dimension 24 × 1, i.e.,where, for i = 1,2,3, fli denotes the i th column vector comprising f̠1 and ωx, ωy, and ωz are the scalar components of the rotational velocity vector.
The measurement functions are as follows:where:
Furthermore, we enforce the orthogonal constraints by employing the following additional measurement equations:
It suffices to show that a subset of the rows of the observability matrix O (cf. Equation (75)) are linearly independent. In the remainder of this section, we will prove that the system described by Equations (50)–(57) is observable by computing among the candidate the zeroth- and first- order Lie derivatives of , , , and , the ones whose gradients ensure that the observability matrix O is full rank.
The zeroth-Order Lie Derivatives : By definition, the zeroth-order Lie derivative of a function is the function itself, i.e.,
Therefore, the gradients of the zeroth-order Lie derivatives are exactly the same as the Jacobians of the corresponding measurement functions:where:
The first-Order Lie Derivatives : The first-order Lie derivatives of and with respect to f0 are computed as:
In these expressions, X1 and X2 are both 3 × 9 matrices and regardless of their values, they will be eliminated from the following derivations; hence, they need not be computed explicitly.
The next first-order Lie derivative of interest is that of with respect to f̠1, i.e., . At this point, it is important to note that f̠1 as defined in Equation (50) is a compact representation of three column vectors. Similarly, we can also write the resultant Lie derivative in a compact form (i.e., a 3 × 3 matrix):
Stacking the gradients of the three columns together gives:where the matrices: of dimensions 9 × 9 and 9 × 3, have the block-row elements (for i = 1,2,3) as follows:
Stacking together all the previously computed gradients of the Lie derivatives, we form the observability matrix O as:
In order to prove that the system described by Equations (50)–(57) is observable, we show that the matrix O is full rank (i.e., the state space of the system is spanned by the gradients of the Lie derivatives of the measurement functions [16,17]). Before presenting the main result of this section, we first present the following two lemmas whose proofs are detailed in Appendixs B and C, respectively.
Lemma 1: The 15 × 12 matrix:is rank 9 if at least one degree of rotational freedom is excited.
Lemma 2: The 6 × 6 matrix:is full rank if two linearly independent horizontal lines are observed.
The observability matrix O (cf. Equation (75)) is full rank when: (a) at least one degree of rotational freedom is excited; (b) at least one vertical line and two linearly independent horizontal lines are observed. The proofs are given in Appendix D.
5. Experimental Results
5.1. Hardware Description
In order to demonstrate the validity of the proposed algorithm in realistic situations, we conducted indoor experiments using a testbed that consists of an ISIS IMU, a Firewire camera, a magnetometer, and a computer for data acquisition. The IMU, the camera, and the magnetometer were rigidly mounted on the chassis and their relative pose did not change during the experiment. The magnetometer was placed far from current wires, computer, and ferromagnetic materials. The raw data were delivered through an USB interface to the computer. The intrinsic parameters of the camera and transformation among the IMU, the camera and the magnetometer were calibrated prior to the experiment and were treated as constants. A monocular camera cannot determine the depth information. In order to solve the problem, the height of the camera from the ground plane was measured, and the lines which are on the ground and perpendicular to the ground were chosen as landmarks. We used basic trigonometry to determine the depth information. mc can be calculated as measurements. Accuracies of the sensors are listed in Table 1. The IMU, camera and magnetometer were electronically time-synchronized.
5.2. Experiment Profile
A short distance experiment was carried out in the hallway. The testbed was rigidly mounted on the chassis of a pushcart. The estimated 3-D trajectory of the pushcart can be seen in Figure 2. The initial position of the pushcart is denoted by ‘*’.
The line features were extracted using the Hough transform. The line-points can be easily calculated with the detection values (ρ,θ) from the Hough transform. It was assumed that the camera measurements were corrupted by additive white Gaussian noise with a standard deviation of 2 pixels. The actual pixel noise is less than 2 pixels. However, in order to compensate the existence of the unmodeled nonlinearities and imperfect camera calibration, the noise standard deviation was increased to 2 pixels.
5.3. Algorithm Performance
The trajectory of the pushcart included two loops. The final position estimate, expressed with respect to the starting pose, is [0.81,0.59,0.06]T m. From the initial and final parking spots of the pushcart, it is known that the true final position expressed with respect to the initial pose is approximately [0,0,0]T m. Thus, the final position error is approximately 1 m at the end of a trajectory of 100 m, i.e., an error of 1% of the travelled distance.
It is noteworthy that the camera motion was almost in parallel to the optical axis, a condition which is particularly adverse for the image-based motion estimation algorithms . In Figure 3, the 3σ bounds for the errors in the attitude matrix and the velocities along the three axes are shown. The plotted values are 3-times the square roots of the corresponding diagonal elements of the state covariance matrix. Position errors were mainly caused by the precision of visual measurement and attitude estimate, especially attitude estimate. Because the steel framed buildings affect the magnetic field, the precision of attitude estimate by inertial/magnetic sensors is not high.
In this paper, we have studied observability of a MKF-based algorithm for indoor navigation using visual/inertial/magnetic sensors. The estimation algorithm uses a compact matrix notation to produce the matrix estimate and the estimation error covariance matrix. The MKF is a natural and straightforward extension of the ordinary KF. Compared with the ordinary KF, the MKF model consists of both pseudo-linear process model and nonlinear measurement model.
The observability of the nonlinear system describing the integrated navigation with visual/inertial/magnetic sensors was investigated by employing the observability rank conditions defined with the Lie derivatives. The pseudo-linear process help simplifying the gradient operator in the process of observability analysis. This paper has for the first time proved that the observability matrix is full rank when: (a) at least one degree of rotational freedom is excited; (b) at least two linearly independent horizontal lines and one vertical line are observed. When these conditions are satisfied, the state matrix that contains the body attitude matrix, the gyro bias vector, relative velocity vector, the dual part of landmark and the magnetic variation superimposed to the magnetic reference vector are observable. The indoor experimental results have demonstrated the correctness of observability conditions.
This work was supported in part by Research Fund for the Doctoral Program of Higher Education of China (20069998009), and the New Century Excellent Talents in University of China (NCET-07-0225).
For arbitrary two vector p and q
We compute Γ and γ:where h + bh = [h1 h2 h3]T. The variables, shown on the right side and next to the row numbers indicate the control input that is involved in using the corresponding row numbers: when a row is retained in the observability matrix (e.g., the second row), the underlying assumption is that the corresponding control input (i.e., ωx) is nonzero.
If none of the components in h + bh is zero, we can obtain:
It is worth-mentioning that each selection of the rows is based on two nonzero components of . For example, we can use either Equation (B3) which assumes that ωy and ωz are nonzero, or Equation (B4) which assumes that ωx and ωz are nonzero, or Equation (B5) which assumes that ωx and ωy are nonzero.
Only if the rotational axis does not coincide with b coordinate axis, will one degree of rotation have two nonzero components of . The rank of A is 9 if at last one degree of rotational freedom is excited.
Note that is full rank.where: det(B) = 0 when: (a); mh1 = mh2= 0 (b) and are linearly dependent. Only (a) and (b) are both satisfied, B is not full rank. To avoid the situation, two linearly independent horizontal lines are observed or the line in the plane (zn = 0) is not chosen. In practice, the former is more feasible. Even one of horizontal line is linearly dependent with , the another is linearly independent with . Then, (b) is not satisfied. So B is full rank if two linearly independent horizontal lines are observed.
Based on Lemma 1, the rank of A is 9. We diagonalize it with the Gaussian elimination:
Then we can easily eliminate all other elements of the first columns:
We can eliminate all other elements of the fourth and fifth columns using , respectively:
Based on Lemma 2, B is full rank, thus we diagonalize it with the Gaussian elimination:
Note that and are both full rank. It is easy to see the matrix is full rank, indicating that the matrix O is also full rank.
- Titterton, D.H.; Weston, J.L. Strapdown Inertial Navigation Technology, 2nd ed. ed.; Peter Peregrinus Ltd. on behalf of the Institute of Electrical Engineers: London, UK, 2004.
- Dias, J.; Vinzce, M.; Corke, P.; Lobo, J. Special issue: 2nd workshop on integration of vision and inertial sensors. Int. J. Robot. Res. 2007, 26, 519–535, doi:10.1177/0278364907079279.
- Sabatini, A.M. Estimating three-dimensional orientation of human body parts by inertial/magnetic sensing. Sensors 2011, 11, 1489–1525, doi:10.3390/s110201489. 22319365
- Wahha, G. A least-square estimate of spacecraft attitude. SLAM Rev. 1965, 7, 409.
- Choukroun, D.; Weiss, H.; Bar-Itzhack, I.Y.; Oshman, Y. Kalman filtering for matrix estimation. IEEE Trans. Aerosp. Elect. Syst. 2006, 42, 147–159, doi:10.1109/TAES.2006.1603411.
- Sabatini, A.M. Kalman-filter-based orientation determination using inertial/magnetic sensors: Observability analysis and performance evaluation. Sensors 2011, 11, 9182–9206, doi:10.3390/s111009182. 22163689
- Groves, P.D. Principles of GNSS, Inertial, and Multi-Sensor Integrated Navigation Systems; Artech House: Norwood, MA, USA, 2008.
- Ferraris, F.; Grimaldi, U.; Parvis, M. Produre for effortless in-field calibration of three-axis rate gyros and accelerometers. Sens. Mater. 1995, 7, 311–330.
- Goddard, J.S. Pose and motion estimation from vision using dual quaternion-based extended Kalman filtering. Proc. SPIE 1997, 3313, 189–200.
- Roetenberg, D.; Luinge, H.J.; Baten, C.T.M.; Veltink, P.H. Compensation of magnetic disturbances improves inertial and magnetic sensing of human body segment orientation. IEEE Trans. Neur. Syst. Rehabil. Eng. 2005, 12, 395–405.
- Choukroun, D.; Weiss, H.; Bar-Itzhack, I.Y.; Oshman, Y. Direction Cosine Matrix Estimation from Vector Observations Using a Matrix Kalman Filter. Proceedings of AIAA Guidance, Navigation, and Control Conference and Exhibit, Austin, TX, USA, 11– 14 August 2003.
- Bellman, R. Introduction to Matrix Analysis; McGraw-Hill: New York, NY, USA, 1960.
- Bar-Itzhack, I.Y.; Reiner, J. Recursive attitude determination from vector observations: DCM identification. J. Guid. Contr. Dynam. 1984, 7, 51–56, doi:10.2514/3.56362.
- Bar-Itzhack, I.Y.; Meyer, J. On the convergence of iterative orthogonalization processes. IEEE Trans. Aerosp. Electron. Syst. 1976, 12, 146–151, doi:10.1109/TAES.1976.308289.
- Brogan, W.L. Modern Control Theory; Prentice-Hall: Upper Saddle River, NJ, USA, 1990.
- Hermann, R.; Krener, A. Nonlinear controlability and observability. IEEE Trans. Autom. Control 1977, 22, 728–740, doi:10.1109/TAC.1977.1101601.
- Sastry, S. Nonlinear Systems: Analysis, Stability, and Control; Springer-Verlag: New York, NY, USA, 1999.
- Oliensis, J. A new structure-from-motion ambiguity. IEEE Trans. Pattern Anal. Mach. Intell. 2000, 22, 685–700, doi:10.1109/34.865186.
|Table 1. Accuracies of the Sensors.|
|IMU||1||gyro drift: <0.01°/s (short term)||100 Hz|
|accelerometer bias: ±2 mg (short term)|
|Camera||1||Resolution: 656 × 490||10 Hz|
|focus length: 25 mm|
|field of view: 60°|
|Magnetometer||1||σbh:10 u.a./s(×10−3)||100 Hz|
© 2012 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).