# A Mixed Deep Recurrent Neural Network for MEMS Gyroscope Noise Suppressing

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

- (1)
- It was the first time a mixed LSTM and GRU method has been applied to MEMS gyroscope noise modeling, which might be an inspiration for applying DL in MEMS IMU de-noising.
- (2)
- It was a bright idea to develop a mixed multi-layer RNN; detailed analysis of the multi-layer LSTM, multi-layer GRU, LSTM–GRU, and GRU–LSTM were presented and compared, which could provide valid reference while selecting proper methods for MEMS gyroscope noise modeling.

## 2. Methods

#### 2.1. Long Short Term Memory (LSTM)

#### 2.2. Gated Recurrent Unit (GRU)

#### 2.3. Mixed LSTM and GRU

## 3. Results

#### 3.1. Input Data and Training

#### 3.2. Comparison of LSTM–GRU and GRU–LSTM

#### 3.3. Comparison of LSTM–GRU, Two-Layer LSTM, and Two-Layer GRU

- (1)
- There was an obvious improvement in the attitude errors for all the three deep neural networks. The two-layer LSTM performed 64.4%, 49.3%, and 53.3% improvements in attitude errors, the two-layer GRU performed 56.3%, 54.5%, and 47.9% decreases in attitude errors, and the attitude errors of LSTM–GRU decreased by 72.2%, 69.3%, and 58.4%.
- (2)
- Specifically, for the x axis gyroscope data, LSTM–GRU had a large training loss, but the LSTM–GRU still showed 7.8% and 15.9% improvements compared with the two-layer LSTM and two-layer GRU. The minor difference of the standard deviation of the de-noised signals may account for this.

## 4. Discussion

## 5. Conclusions

- (1)
- Two-layer LSTM, two-layer GRU, LSTM–GRU, and GRU–LSTM were effective for this application. The two-layer LSTM performed a 64.4%, 49.3%, and 53.3% improvement in attitude errors, the two-layer GRU performed a 56.3%, 49.3%, and 47.9% decrease in attitude errors, and the attitude errors of LSTM–GRU decreased by 72.2%, 69.3%, and 58.4%;
- (2)
- With a limited training dataset, LSTM–GRU outperformed GRU–LSTM; LSTM–GRU had a large training loss, but the LSTM–GRU still showed an improvement compared with the two-layer LSTM and two-layer GRU.

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Dow, J.M.; Ruth, E.N.; Chris, R. The international GNSS service in a changing landscape of global navigation satellite systems. J. Geod.
**2009**, 83, 191–198. [Google Scholar] [CrossRef] - Montenbruck, O.; Steigenberger, P.; Khachikyan, R.; Weber, G.; Langley, R.B.; Mervart, L.; Hugentobler, U. IGS-MGEX: Preparing the ground for multi-constellation GNSS science. Inside GNSS
**2014**, 9, 42–49. [Google Scholar] - Hewitson, S.; Wang, J. GNSS receiver autonomous integrity monitoring (RAIM) performance analysis. GPS Solut.
**2006**, 10, 155–170. [Google Scholar] [CrossRef] - Lashley, M.; Bevly, D.M.; Hung, J.Y. Performance analysis of vector tracking algorithms for weak GPS signals in high dynamics. IEEE J. Sel. Top. Sign. Process.
**2009**, 3, 661–673. [Google Scholar] [CrossRef] - Jiang, C.; Chen, S.; Chen, Y.; Bo, Y.; Wang, C.; Tao, W. Performance analysis of GNSS Vector tracking loop based GNSS/CSAC integrated navigation system. J. Aeronaut. Astronaut. Aviat.
**2017**, 49, 289–297. [Google Scholar] - Chen, R.; Chen, Y.; Pei, L.; Chen, W.; Liu, J.; Kuusniemi, H.; Takala, J. A DSP-based multi-sensor multi-network positioning platform. In Proceedings of the 22nd International Technical Meeting of The Satellite Division of the Institute of Navigation, Savannah, GA, USA, 22–25 September 2009; pp. 615–621. [Google Scholar]
- Chen, Y.; Tang, J.; Jiang, C.; Zhu, L.; Lehtomäki, M.; Kaartinen, H.; Zhou, H. The accuracy comparison of three simultaneous localization and mapping (SLAM)-based indoor mapping technologies. Sensors
**2018**, 18, 3228. [Google Scholar] [CrossRef] [PubMed] - Jiang, C.; Chen, S.; Chen, Y.; Zhang, B.; Feng, Z.; Zhou, H.; Bo, Y. A MEMS IMU de-noising method using long short term memory recurrent neural networks (LSTM-RNN). Sensors
**2018**, 18, 3470. [Google Scholar] [CrossRef] - Chiang, K.W.; Huang, Y.W. An intelligent navigator for seamless INS/GPS integrated land vehicle navigation applications. Appl. Soft Comput.
**2008**, 8, 722–733. [Google Scholar] [CrossRef] - Chiang, K.W.; Duong, T.T.; Liao, J.K. The performance analysis of a real-time integrated INS/GPS vehicle navigation system with abnormal GPS measurement elimination. Sensors
**2013**, 13, 10599–10622. [Google Scholar] [CrossRef] - Syed, Z.F.; Aggarwal, P.; Goodall, C.; Niu, X.; El-Sheimy, N. A new multi-position calibration method for MEMS inertial navigation systems. Meas. Sci. Technol.
**2007**, 18, 1897. [Google Scholar] [CrossRef] - Cho, S.Y.; Park, C.G. MEMS based pedestrian navigation system. J. Navig.
**2006**, 59, 135–153. [Google Scholar] [CrossRef] - Brown, A.K. GPS/INS uses low-cost MEMS IMU. IEEE Aerosp. Electron. Syst. Mag.
**2005**, 20, 3–10. [Google Scholar] [CrossRef] - Jiang, C.H.; Chen, S.; Chen, Y.Y.; Bo, Y.M. Research on chip scale atomic clock driven GNSS/SINS deeply coupled navigation system for augmented performance. IET Radar. Sonar Navig.
**2018**. [Google Scholar] [CrossRef] - Ning, Y.; Wang, J.; Han, H.; Tan, X.; Liu, T. an optimal radial basis function neural network enhanced adaptive robust Kalman filter for GNSS/INS integrated systems in complex urban areas. Sensors
**2018**, 18, 3091. [Google Scholar] [CrossRef] [PubMed] - Niu, X.; Nassar, S.; El-Sheimy, N. An accurate land-vehicle MEMS IMU/GPS navigation system using 3D auxiliary velocity updates. Navigation
**2007**, 54, 177–188. [Google Scholar] [CrossRef] - Li, W.; Wang, J. Effective adaptive Kalman filter for MEMS-IMU/magnetometers integrated attitude and heading reference systems. J. Navig.
**2013**, 66, 99–113. [Google Scholar] [CrossRef] - Bhatt, D.; Aggarwal, P.; Devabhaktuni, V.; Bhattacharya, P. A novel hybrid fusion algorithm to bridge the period of GPS outages using low-cost INS. Expert Syst. Appl.
**2014**, 41, 2166–2173. [Google Scholar] [CrossRef] - El-Sheimy, N.; Hou, H.Y.; Niu, X.J. Analysis and modeling of inertial sensors using Allan variance. IEEE Trans. Instrum. Meas.
**2008**, 57, 140–149. [Google Scholar] [CrossRef] - Allan, D.W. Historicity, strengths, and weaknesses of Allan variances and their general applications. Gyroscopy Navig.
**2016**, 7, 1–17. [Google Scholar] [CrossRef] - Radi, A.; Nassar, S.; El-Sheimy, N. Stochastic error modeling of smartphone inertial sensors for navigation in varying dynamic conditions. Gyroscopy Navig.
**2018**, 9, 76–95. [Google Scholar] [CrossRef] - Aggarwal, P.; Syed, Z.; Niu, X.X.; EI-Sheimy, N. A standard testing and calibration procedure for low cost MEMS inertial sensors and units. J. Navig.
**2008**, 61, 323–336. [Google Scholar] [CrossRef] - Wang, D.; Dong, Y.; Li, Q.; Li, Z.; Wu, J. Using Allan variance to improve stochastic modeling for accurate GNSS/INS integrated navigation. GPS Solut.
**2018**, 22, 53. [Google Scholar] [CrossRef] - Zhang, Q.; Wang, X.; Wang, S.; Pei, C. Application of improved fast dynamic Allan variance for the characterization of MEMS gyroscope on UAV. J. Sens.
**2018**. [Google Scholar] [CrossRef] - Wang, L.; Zhang, C.; Gao, S.; Wang, T.; Lin, T.; Li, X. Application of fast dynamic Allan variance for the characterization of FOGs-Based measurement while drilling. Sensors
**2016**, 16, 2078. [Google Scholar] [CrossRef] [PubMed] - Su, W.P.; Hao, Y.S.; Li, Q.C. Arma-akf model of mems gyro rotation data random drift compensation. Appl. Mech. Mater.
**2013**, 321–324, 549–552. [Google Scholar] [CrossRef] - Huang, L. Auto regressive moving average (ARMA) modeling method for Gyro random noise using a robust Kalman filter. Sensors
**2015**, 15, 25277–25286. [Google Scholar] [CrossRef] [PubMed] - Khashei, M.; Bijari, M. A novel hybridization of artificial neural networks and ARIMA models for time series forecasting. Appl. Soft Comput.
**2011**, 11, 2664–2675. [Google Scholar] [CrossRef] - Waegli, A.; Skaloud, J.; Guerrier, S.; Parés, M.E.; Colomina, I. Noise reduction and estimation in multiple micro-electro-mechanical inertial systems. Meas. Sci. Technol.
**2010**, 21, 156–158. [Google Scholar] [CrossRef] - Bhatt, D.; Priyanka, A.; Prabir, B.; Vijay, D. An enhanced mems error modeling approach based on nu-support vector regression. Sensors
**2012**, 12, 9448–9466. [Google Scholar] [CrossRef] - Xing, H.F.; Hou, Bo.; Lin, Z.H.; Guo, M.F. Modeling and compensation of random drift of MEMS gyroscopes based on least squares support vector machine optimized by chaotic particle swarm optimization. Sensors
**2017**, 17, 2335. [Google Scholar] [CrossRef] - Jiang, C.; Chen, S.; Chen, Y.; Bo, Y.; Han, L.; Guo, J.; Feng, Z.; Zhou, H. Performance Analysis of a Deep Simple Recurrent Unit Recurrent Neural Network (SRU-RNN) in MEMS Gyroscope De-Noising. Sensors
**2018**, 18, 4471. [Google Scholar] [CrossRef] - Understanding LSTM Networks. Available online: https://colah.github.io/posts/20150–8-Understanding-LSTMs/ (accessed on 27 August 2015).
- Hosseinyalamdary, S. Deep Kalman filter: Simultaneous multi-sensor integration and modelling; A GNSS/IMU case study. Sensors
**2018**, 18, 1316. [Google Scholar] [CrossRef] [PubMed] - Chung, J.; Gulcehre, C.; Cho, K.; Bengio, Y. Gated feedback recurrent neural networks. In Proceedings of the International Conference on Machine Learning, Lille, France, 6–11 July 2015; pp. 2067–2075. [Google Scholar]
- Gers, F.A.; Schraudolph, N.N.; Schmidhuber, J. Learning precise timing with LSTM recurrent networks. J. Mach. Learn. Res.
**2002**, 3, 115–143. [Google Scholar] - Ordóñez, F.J.; Roggen, D. Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors
**2016**, 16, 115. [Google Scholar] [CrossRef] [PubMed] - MSI3200. Available online: http://www.mtmems.com/product_view.asp?id=28 (accessed on 5 August 2017).

**Figure 12.**Zooming out of Figure 11 from the 5th to 25th epoch.

MEMS IMU | Gyroscope | range | $\pm $300 °/s |

Bias stability (1 $\sigma $) | $\le $10 °/h | ||

Bias stability (Allan) | $\le $2 °/h | ||

Angle random walk | $\le $10 °/$\sqrt{h}$ | ||

Accelerometer | range | $\pm $15 g | |

Bias stability (1 $\sigma $) | 0.5 mg | ||

Bias stability (Allan) | 0.5 mg | ||

Power consumption | 1.5 W | ||

Weight | 250 g | ||

Size | $70\text{}\mathrm{mm}\times \text{}54\mathrm{mm}\times 39\text{}\mathrm{mm}$ | ||

Sampling rate | 400 Hz |

X (degree/s) | Y (degree/s) | Z (degree/s) | |
---|---|---|---|

Training loss | 0.00132 | 0.00534 | 0.00139 |

LSTM-RNN | 0.060 | 0.037 | 0.025 |

Original signals | 0.069 | 0.083 | 0.047 |

X (degree/s) | Y (degree/s) | Z (degree/s) | |
---|---|---|---|

Training loss | 0.00136 | 0.0055 | 0.00142 |

LSTM-RNN | 0.059 | 0.034 | 0.026 |

Original signals | 0.069 | 0.083 | 0.047 |

X (degree/s) | Y (degree/s) | Z (degree/s) | |
---|---|---|---|

Training loss | 0.00127 | 0.00469 | 0.00134 |

LSTM-RNN | 0.060 | 0.035 | 0.0246 |

Original signals | 0.069 | 0.083 | 0.047 |

X (degree) | Y (degree) | Z (degree) | |
---|---|---|---|

two-layer LSTM | 0.136 | 0.240 | 0.184 |

two-layer GRU | 0.167 | 0.215 | 0.205 |

LSTM–GRU | 0.104 | 0.145 | 0.164 |

Original signals | 0.382 | 0.473 | 0.394 |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Jiang, C.; Chen, Y.; Chen, S.; Bo, Y.; Li, W.; Tian, W.; Guo, J.
A Mixed Deep Recurrent Neural Network for MEMS Gyroscope Noise Suppressing. *Electronics* **2019**, *8*, 181.
https://doi.org/10.3390/electronics8020181

**AMA Style**

Jiang C, Chen Y, Chen S, Bo Y, Li W, Tian W, Guo J.
A Mixed Deep Recurrent Neural Network for MEMS Gyroscope Noise Suppressing. *Electronics*. 2019; 8(2):181.
https://doi.org/10.3390/electronics8020181

**Chicago/Turabian Style**

Jiang, Changhui, Yuwei Chen, Shuai Chen, Yuming Bo, Wei Li, Wenxin Tian, and Jun Guo.
2019. "A Mixed Deep Recurrent Neural Network for MEMS Gyroscope Noise Suppressing" *Electronics* 8, no. 2: 181.
https://doi.org/10.3390/electronics8020181