# Deep Kalman Filter: Simultaneous Multi-Sensor Integration and Modelling; A GNSS/IMU Case Study

## Abstract

**:**

## 1. Problem Statement

#### 1.1. Literature Review

#### 1.2. Kalman Filter

_{t}, is the noise of the observation model. The state and observation models are assumed to be linear in the Kalman filter. Therefore, these functions can be replaced by F and G matrices, respectively. The system model can be rewritten as:

#### Shortcomings of the Kalman Filter

## 2. Methodology

#### 2.1. Expectation Maximization

#### 2.2. Recurrent Neural Network

#### 2.3. Long Short-Term Memory

## 3. Implementation

^{−3}learning rate decay. We tried several sequence lengths and studied the impact of sequence length on accurate error modeling.

## 4. Experiment

## 5. Results

## 6. Conclusions

## Conflicts of Interest

## Appendix A. System and Observation Models of Kalman Filter

## References

- Bochkati, M.; Schön, S.; Schlippert, D.; Schubert, C.; Rasel, E. Could cold atom interferometry sensors be the future inertial sensors?—First simulation results. In Proceedings of the 2017 DGON Inertial Sensors and Systems (ISS), Karlsruhe, Germany, 19–20 September 2017; pp. 1–20. [Google Scholar]
- Jekeli, C. Inertial Navigation Systems with Geodetic Applications; Walter de Gruyter: Berlin, Germany, 2001. [Google Scholar]
- Noureldin, A.; Karamat, T.B.; Georgy, J. Fundamentals of Inertial Navigation, Satellite-Based Positioning and Their Integration; Springer: Berlin/Heidelberg, Germany, 2013. [Google Scholar]
- Shin, E.-H.; El-Sheimy, N. A new calibration method for strap down inertial navigation systems. Z. Vermess
**2002**, 127, 1–10. [Google Scholar] - Wang, S.; Deng, Z.; Yin, G. An Accurate GPS-IMU/DR Data Fusion Method for Driverless Car Based on a Set of Predictive Models and Grid Constraints. Sensors
**2016**, 16, 280. [Google Scholar] [CrossRef] [PubMed] - Chiang, K.W.; Noureldin, A.; El-Sheimy, N. Constructive Neural-Networks-Based MEMS/GPS Integration Scheme. IEEE Trans. Aerosp. Electron. Syst.
**2008**, 44, 582–594. [Google Scholar] [CrossRef] - Noureldin, A.; El-Shafie, A.; Bayoumi, M. GPS/INS Integration Utilizing Dynamic Neural Networks for Vehicular Navigation. Inf. Fusion
**2011**, 12, 48–57. [Google Scholar] [CrossRef] - Abdel-Hamid, W.; Noureldin, A.; El-Sheimy, N. Adaptive Fuzzy Prediction of Low-Cost Inertial-Based Positioning Errors. IEEE Trans. Fuzzy Syst.
**2007**, 15, 519–529. [Google Scholar] [CrossRef] - Zhang, L.; Liu, J.; Lai, J.; Xiong, Z. Performance Analysis of Adaptive Neuro Fuzzy Inference System Control for MEMS Navigation System. Math. Probl. Eng.
**2014**, 2014, 961067. [Google Scholar] [CrossRef] - Toth, C.; Grejner-Brzezinska, D.A.; Moafipoor, S. Pedestrian Tracking and Navigation Using Neural Networks and Fuzzy Logic. In Proceedings of the 2007 IEEE International Symposium on Intelligent Signal Processing, Alcala de Henares, Spain, 3–5 October 2007; pp. 1–6. [Google Scholar]
- Navidi, N.; Landry, R., Jr.; Cheng, J.; Gingras, D. A New Technique for Integrating MEMS-Based Low-Cost IMU and GPS in Vehicular Navigation. J. Sens.
**2016**, 2016, 5365983. [Google Scholar] [CrossRef] - Xing, H.; Hou, B.; Lin, Z.; Guo, M. Modeling and Compensation of Random Drift of MEMS Gyroscopes Based on Least Squares Support Vector Machine Optimized by Chaotic Particle Swarm Optimization. Sensors
**2017**, 17, 2335. [Google Scholar] [CrossRef] [PubMed] - Minsky, M.L.; Papert, S.A. Perceptrons: Expanded Edition; MIT Press: Cambridge, MA, USA, 1988. [Google Scholar]
- Mirowski, P.; Lecun, Y. Dynamic Factor Graphs for Time Series Modelling. In Proceedings of the Machine Learning and Knowledge Discovery in Databases—European Conference, Bled, Slovenia, 7–11 September 2009; Volume 5782, pp. 128–143. [Google Scholar]
- Gu, J.; Yang, X.; De Mello, S.; Kautz, J. Dynamic Facial Analysis: From Bayesian Filtering to Recurrent Neural Network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 1531–1540. [Google Scholar]
- Krishnan, R.G.; Shalit, U.; Sontag, D. Structured Inference Networks for Nonlinear State Space Models. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, CA, USA, 4–9 February 2017. [Google Scholar]
- Ghahramani, Z.; Roweis, S.T. Learning nonlinear dynamical systems using an EM algorithm. In Advances in Neural Information Processing Systems 11; MIT Press: Cambridge, MA, USA, 1999; pp. 431–437. [Google Scholar]
- Goodfellow, I.; Bengio, I.; Courville, A. Deep Learning; MIT Press: Cambridge, MA, USA, 2016. [Google Scholar]
- Hochreiter, S.; Schmidhuber, J. Long Short-Term Memory. Neural Comput.
**1997**, 9, 1735–1780. [Google Scholar] [CrossRef] [PubMed] - Geiger, A.; Lenz, P.; Stiller, C.; Urtasun, R. Vision Meets Robotics: The KITTI Dataset. Int. J. Robot. Res.
**2013**, 32, 1231–1237. [Google Scholar] [CrossRef] - Hosseinyalamdary, S.; Balazadegan, Y. Error Modeling of Reduced IMU using Recurrent Neural Network. In Proceedings of the Indoor Positioning and Indoor Navigation (IPIN), Sapporo, Japan, 18–21 September 2017. [Google Scholar]
- Zhang, S.; Wu, Y.; Che, T.; Lin, Z.; Memisevic, R.; Salakhutdinov, R.; Bengio, Y. Architectural complexity measures of recurrent neural networks. In Advances in Neural Information Processing Systems 29; Curran Associates Inc.: Red Hook, NY, USA, 2016; pp. 1830–1838. [Google Scholar]

**Figure 1.**The Kalman filter procedure, which consists of prediction (

**left-up box**) and update steps (

**right-up box**).

**Figure 2.**The probabilistic graphical model of the Kalman filter (

**a**) and deep Kalman filter (

**b**); x, z, and h are the state vector, observation vector, and latent vector, respectively. The matrices F and G are the system model and observation model of the Kalman filter and W is the coefficient matrix of our proposed IMU model. The two upper layers of the deep Kalman filter are similar to the Kalman filter and the added two lower layers enable the deep Kalman filter to estimate our system model.

**Figure 3.**The deep Kalman filter procedure. The IMU modelling step (

**right-bottom**) has been added to the Kalman filter. The modelling is accomplished in two steps: in the first step, the current latent vector is estimated based on previous latent and state vectors; and in the second step, the current state vector is estimated based on the current latent vector.

**Figure 4.**IMU error modelling reformulated as a time series prediction. The posterior estimation of the state vector is utilized in the target node (

**top layer**), the output of our model is considered as the output node (

**second layer**), the previous state vectors are in the input layer (

**bottom layer**) and the latent vectors are in the hidden layer (

**third layer**).

**Figure 5.**The trajectory of the KITTI dataset, #34. It is the longest KITTI dataset where the vehicle travels more than 1.7 km.

**Figure 6.**The RMSE of the deep extended Kalman filter and extended Kalman filter [21]. We used different sequence lengths of a simple recurrent neural network (RNN) for the IMU modelling of the deep extended Kalman filter. (

**a**) Simple RNN with a sequence length of 10; (

**b**) simple RNN with a sequence length of 20; (

**c**) simple RNN with a sequence length of 50.

**Figure 7.**RMSE of the deep extended Kalman filter and extended Kalman filter. The deep extended Kalman filter IMU modelling was based on LSTM with a sequence length of 10.

© 2018 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Hosseinyalamdary, S.
Deep Kalman Filter: Simultaneous Multi-Sensor Integration and Modelling; A GNSS/IMU Case Study. *Sensors* **2018**, *18*, 1316.
https://doi.org/10.3390/s18051316

**AMA Style**

Hosseinyalamdary S.
Deep Kalman Filter: Simultaneous Multi-Sensor Integration and Modelling; A GNSS/IMU Case Study. *Sensors*. 2018; 18(5):1316.
https://doi.org/10.3390/s18051316

**Chicago/Turabian Style**

Hosseinyalamdary, Siavash.
2018. "Deep Kalman Filter: Simultaneous Multi-Sensor Integration and Modelling; A GNSS/IMU Case Study" *Sensors* 18, no. 5: 1316.
https://doi.org/10.3390/s18051316