# RIANN—A Robust Neural Network Outperforms Attitude Estimation Filters

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

#### 1.1. The Challenge of Generalizability in Inertial Attitude Estimation

#### 1.2. The Potential of Neural Networks in Inertial Attitude Estimation

#### 1.3. Contributions

- We propose three domain-specific advances for neural networks in the context of inertial attitude estimation.
- We identify two methods that enable neural networks to handle different sampling rates in system identification tasks.
- We present the attitude estimation neural network RIANN, which results from these advances, and make it publicly available at [28].
- We combine six different publicly available datasets for a comprehensive evaluation of the robustness of attitude estimation methods.
- We compare RIANN with commonly used state-of-the-art attitude estimation filters in three evaluation scenarios with different degrees of practical relevance.
- We show that RIANN consistently outperforms commonly used state-of-the-art attitude estimation filters across different applications, motion characteristics, sampling rates, and sensor hardware.

## 2. Problem Statement

## 3. Neural Network Structure and Implementation

#### 3.1. Choice of the Neural Network Structure

#### 3.2. Neural Network Implementation with General Best Practices

#### 3.3. Loss Function

#### 3.4. Generalization across Sampling Rates

#### 3.5. Data Augmentation

#### 3.6. Grouped Input Channels

## 4. Neural Network Optimization

#### 4.1. Ablation Study

#### 4.2. Sampling Rates Study

#### 4.3. Network Size Analysis

## 5. Performance Evaluation

- restrictive scenario: It is assumed that the sequence starts with a period of perfect rest, during which the attitude estimation can converge to an accurate estimate before the actual motion starts. Moreover, it is assumed that the turn-on bias of the gyroscopes has been removed in a preprocessing step, which requires a sufficiently long rest phase.
- partially restrictive scenario: We still assume a rest phase prior to the motion onset, but no turn-on bias correction has been conducted. We emulate this scenario by adding a random constant bias, which is drawn from a zero-mean normal distribution with a standard deviation of $0.5\xb0/\mathrm{s}$, to the bias-free test sequences of the restrictive scenario.
- realistic scenario: The sensor already moves when it is turned on and the attitude estimation is started. The test sequences have the same gyroscope bias as in the partially restrictive scenario, but the initial rest periods are removed.

## 6. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## References

- Seel, T.; Kok, M.; McGinnis, R.S. Inertial Sensors—Applications and Challenges in a Nutshell. Sensors
**2020**, 20, 6221. [Google Scholar] [CrossRef] - Euston, M.; Coote, P.; Mahony, R.; Kim, J.; Hamel, T. A complementary filter for attitude estimation of a fixed-wing UAV. In Proceedings of the 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, Nice, France, 22–26 September 2008; pp. 340–345. [Google Scholar] [CrossRef]
- Ding, W.; Xu, M.; Ma, Y.; Shi, G. Tricycle Attitude Estimation and Turn Control Based on MEMS Sensing Technology. In Proceedings of the 2018 IEEE 1st International Conference on Micro/Nano Sensors for AI, Healthcare, and Robotics (NSENS), Shenzhen, China, 5–7 December 2018; pp. 30–34. [Google Scholar] [CrossRef]
- Valarezo Añazco, E.; Han, S.J.; Kim, K.; Lopez, P.R.; Kim, T.S.; Lee, S. Hand Gesture Recognition Using Single Patchable Six-Axis Inertial Measurement Unit via Recurrent Neural Networks. Sensors
**2021**, 21, 1404. [Google Scholar] [CrossRef] - Marco, V.R.; Kalkkuhl, J.; Seel, T. Nonlinear observer with observability-based parameter adaptation for vehicle motion estimation. In Proceedings of the 18th IFAC Symposium on System Identification, (SYSID), Stockholm, Sweden, 9–11 July 2018; pp. 60–65. [Google Scholar] [CrossRef]
- Woodman, O.J. An Introduction to Inertial Navigation; Technical report; University of Cambridge, Computer Laboratory: Cambridge, UK, 2007. [Google Scholar]
- Nazarahari, M.; Rouhani, H. 40 years of sensor fusion for orientation tracking via magnetic and inertial measurement units: Methods, lessons learned, and future challenges. Inf. Fusion
**2021**, 68, 67–84. [Google Scholar] [CrossRef] - De Vries, W.; Veeger, H.; Baten, C.; Van Der Helm, F. Magnetic distortion in motion labs, implications for validating inertial magnetic sensors. Gait Posture
**2009**, 29, 535–541. [Google Scholar] [CrossRef] - Kok, M.; Hol, J.D.; Schön, T.B. An optimization-based approach to human body motion capture using inertial sensors. IFAC Proc. Vol.
**2014**, 47, 79–85. [Google Scholar] [CrossRef][Green Version] - Teufl, W.; Miezal, M.; Taetz, B.; Fröhlich, M.; Bleser, G. Validity, test-retest reliability and long-term stability of magnetometer free inertial sensor based 3D joint kinematics. Sensors
**2018**, 18, 1980. [Google Scholar] [CrossRef] [PubMed][Green Version] - Lorenz, M.; Taetz, B.; Bleser, G. An Approach to Magnetometer-free On-body Inertial Sensors Network Alignment. In Proceedings of the IFAC World Congress, Berlin, Germany, 12–17 July 2020. [Google Scholar]
- Eckhoff, K.; Kok, M.; Lucia, S.; Seel, T. Sparse Magnetometer-free Inertial Motion Tracking—A Condition for Observability in Double Hinge Joint Systems. In Proceedings of the 21st IFAC World Congress, Berlin, Germany, 12–17 July 2020; pp. 1–8. [Google Scholar]
- Grapentin, A.; Lehmann, D.; Zhupa, A.; Seel, T. Sparse Magnetometer-Free Real-Time Inertial Hand Motion Tracking. In Proceedings of the IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), Karlsruhe, Germany, 14–16 September 2020. [Google Scholar] [CrossRef]
- Lehmann, D.; Laidig, D.; Deimel, R.; Seel, T. Magnetometer-free Inertial Motion Tracking of Arbitrary Joints with Range-of-motion Constraints. In Proceedings of the 21st IFAC World Congress, Berlin, Germany, 12–17 July 2020; pp. 1–8. [Google Scholar]
- Caruso, M.; Sabatini, A.M.; Laidig, D.; Seel, T.; Knaflitz, M.; Della Croce, U.; Cereatti, A. Analysis of the Accuracy of Ten Algorithms for Orientation Estimation Using Inertial and Magnetic Sensing under Optimal Conditions: One Size Does Not Fit All. Sensors
**2021**, 21, 2543. [Google Scholar] [CrossRef] - Laidig, D.; Caruso, M.; Cereatti, A.; Seel, T. BROAD—A Benchmark for Robust Inertial Orientation Estimation. Data
**2021**, 6, 72. [Google Scholar] [CrossRef] - Rich, S. The Bitter Lesson. Available online: http://www.incompleteideas.net/IncIdeas/BitterLesson.html (accessed on 9 March 2021).
- Andersson, C.; Ribeiro, A.H.; Tiels, K.; Wahlström, N.; Schön, T.B. Deep Convolutional Networks in System Identification. arXiv
**2019**, arXiv:1909.01730. [Google Scholar] - Oord, A.V.D.; Dieleman, S.; Zen, H.; Simonyan, K.; Vinyals, O.; Graves, A.; Kalchbrenner, N.; Senior, A.; Kavukcuoglu, K. WaveNet: A Generative Model for Raw Audio. arXiv
**2016**, arXiv:1609.03499. [Google Scholar] - Brossard, M.; Barrau, A.; Bonnabel, S. RINS-W: Robust Inertial Navigation System on Wheels. arXiv
**2020**, arXiv:1903.02210. [Google Scholar] - Brossard, M.; Bonnabel, S.; Barrau, A. Denoising IMU Gyroscopes with Deep Learning for Open-Loop Attitude Estimation. arXiv
**2020**, arXiv:2002.10718. [Google Scholar] [CrossRef] - Chiang, K.W.; Chang, H.W.; Li, C.Y.; Huang, Y.W. An Artificial Neural Network Embedded Position and Orientation Determination Algorithm for Low Cost MEMS INS/GPS Integrated Sensors. Sensors
**2009**, 9, 2586–2610. [Google Scholar] [CrossRef][Green Version] - Dhahbane, D.; Nemra, A.; Sakhi, S. Neural Network-Based Attitude Estimation. In Artificial Intelligence and Renewables Towards an Energy Transition; Lecture Notes in Networks and Systems; Hatti, M., Ed.; Springer International Publishing: Cham, Switzerland, 2021; pp. 500–511. [Google Scholar]
- Al-Sharman, M.K.; Zweiri, Y.; Jaradat, M.A.K.; Al-Husari, R.; Gan, D.; Seneviratne, L.D. Deep-Learning-Based Neural Network Training for State Estimation Enhancement: Application to Attitude Estimation. IEEE Trans. Instrum. Meas.
**2020**, 69, 24–34. [Google Scholar] [CrossRef][Green Version] - Esfahani, M.A.; Wang, H.; Wu, K.; Yuan, S. AbolDeepIO: A Novel Deep Inertial Odometry Network for Autonomous Vehicles. IEEE Trans. Intell. Transp. Syst.
**2019**, 1–10. [Google Scholar] [CrossRef] - Esfahani, M.A.; Wang, H.; Wu, K.; Yuan, S. OriNet: Robust 3-D Orientation Estimation With a Single Particular IMU. IEEE Robot. Autom. Lett.
**2020**, 5, 399–406. [Google Scholar] [CrossRef] - Weber, D.; Gühmann, C.; Seel, T. Neural Networks Versus Conventional Filters for Inertial-Sensor-based Attitude Estimation. arXiv
**2020**, arXiv:2005.06897. [Google Scholar] - Weber, D. RIANN (Robust IMU-Based Attitude Neural Network). Available online: https://github.com/daniel-om-weber/riann (accessed on 15 April 2021).
- Beuchert, J.; Solowjow, F.; Trimpe, S.; Seel, T. Overcoming Bandwidth Limitations in Wireless Sensor Networks by Exploitation of Cyclic Signal Patterns: An Event-triggered Learning Approach. Sensors
**2020**, 20, 260. [Google Scholar] [CrossRef][Green Version] - Wolf, T.; Debut, L.; Sanh, V.; Chaumond, J.; Delangue, C.; Moi, A.; Cistac, P.; Rault, T.; Louf, R.; Funtowicz, M.; et al. HuggingFace’s Transformers: State-of-the-art Natural Language Processing. arXiv
**2020**, arXiv:1910.03771. [Google Scholar] - Gonzalez, J.; Yu, W. Non-linear system modeling using LSTM neural networks. IFAC-PapersOnLine
**2018**, 51, 485–489. [Google Scholar] [CrossRef] - Cho, K.; van Merrienboer, B.; Bahdanau, D.; Bengio, Y. On the Properties of Neural Machine Translation: Encoder-Decoder Approaches. arXiv
**2014**, arXiv:1409.1259. [Google Scholar] - Tallec, C.; Ollivier, Y. Unbiasing Truncated Backpropagation Through Time. arXiv
**2017**, arXiv:1705.08209. [Google Scholar] - Ioffe, S.; Szegedy, C. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. arXiv
**2015**, arXiv:1502.03167. [Google Scholar] - Liu, L.; Jiang, H.; He, P.; Chen, W.; Liu, X.; Gao, J.; Han, J. On the Variance of the Adaptive Learning Rate and Beyond. arXiv
**2019**, arXiv:1908.03265. [Google Scholar] - Zhang, M.R.; Lucas, J.; Hinton, G.; Ba, J. Lookahead Optimizer: K steps forward, 1 step back. arXiv
**2019**, arXiv:1907.08610. [Google Scholar] - Howard, J.; Gugger, S. Fastai: A Layered API for Deep Learning. Information
**2020**, 11, 108. [Google Scholar] [CrossRef][Green Version] - Smith, L.N. Cyclical Learning Rates for Training Neural Networks. arXiv
**2017**, arXiv:1506.01186. [Google Scholar] - Loshchilov, I.; Hutter, F. SGDR: Stochastic Gradient Descent with Warm Restarts. arXiv
**2017**, arXiv:1608.03983. [Google Scholar] - Jaderberg, M.; Dalibard, V.; Osindero, S.; Czarnecki, W.M.; Donahue, J.; Razavi, A.; Vinyals, O.; Green, T.; Dunning, I.; Simonyan, K.; et al. Population Based Training of Neural Networks. arXiv
**2017**, arXiv:1711.09846. [Google Scholar] - Li, L.; Jamieson, K.; Rostamizadeh, A.; Gonina, E.; Hardt, M.; Recht, B.; Talwalkar, A. A System for Massively Parallel Hyperparameter Tuning. arXiv
**2020**, arXiv:1810.05934. [Google Scholar] - Virtanen, P.; Gommers, R.; Oliphant, T.E.; Haberland, M.; Reddy, T.; Cournapeau, D.; Burovski, E.; Peterson, P.; Weckesser, W.; Bright, J.; et al. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nat. Methods
**2020**, 17, 261–272. [Google Scholar] [CrossRef][Green Version] - Shoemake, K. Animating rotation with quaternion curves. In Proceedings of the 12th Annual Conference on Computer Graphics and Interactive Techniques; SIGGRAPH ’85; Association for Computing Machinery: New York, NY, USA, 1985; pp. 245–254. [Google Scholar] [CrossRef][Green Version]
- Perez, L.; Wang, J. The Effectiveness of Data Augmentation in Image Classification using Deep Learning. arXiv
**2017**, arXiv:1712.04621. [Google Scholar] - Cui, X.; Goel, V.; Kingsbury, B. Data Augmentation for Deep Neural Network Acoustic Modeling. IEEE/ACM Trans. Audio Speech Lang. Process.
**2015**, 23, 1469–1477. [Google Scholar] [CrossRef] - Zhang, Q.; Niu, X.; Shi, C. Impact Assessment of Various IMU Error Sources on the Relative Accuracy of the GNSS/INS Systems. IEEE Sen. J.
**2020**, 20, 5026–5038. [Google Scholar] [CrossRef] - Zheng, Y.; Liu, Q.; Chen, E.; Ge, Y.; Zhao, J.L. Time Series Classification Using Multi-Channels Deep Convolutional Neural Networks. In Web-Age Information Management; Springer International Publishing: Cham, Switzerland, 2014; Volume 8485, pp. 298–310. [Google Scholar]
- Schubert, D.; Goll, T.; Demmel, N.; Usenko, V.; Stückler, J.; Cremers, D. The TUM VI Benchmark for Evaluating Visual-Inertial Odometry. In Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, 1–5 October 2018; pp. 1680–1687. [Google Scholar] [CrossRef][Green Version]
- Burri, M.; Nikolic, J.; Gohl, P.; Schneider, T.; Rehder, J.; Omari, S.; Achtelik, M.W.; Siegwart, R. The EuRoC micro aerial vehicle datasets. Int. J. Robot. Res.
**2016**, 35, 1157–1163. [Google Scholar] [CrossRef] - Caruso, M.; Cereatti, A.; Croce, U.D. Mimu_Optical_Sassari_Dataset; type: Dataset; IEEE: New York, NY, USA, 2020. [Google Scholar] [CrossRef]
- Chen, C.; Zhao, P.; Lu, C.X.; Wang, W.; Markham, A.; Trigoni, N. OxIOD: The Dataset for Deep Inertial Odometry. arXiv
**2018**, arXiv:1809.07491. [Google Scholar] - Szczesna, A.; Skurowski, P.; Pruszowski, P.; Peszor, D.; Paszkuta, M.; Wojciechowski, K. Reference Data Set for Accuracy Evaluation of Orientation Estimation Algorithms for Inertial Motion Capture Systems. In Computer Vision and Graphics; Lecture Notes in Computer Science; Chmielewski, L.J., Datta, A., Kozera, R., Wojciechowski, K., Eds.; Springer International Publishing: Cham, Switzerland, 2016; pp. 509–520. [Google Scholar]
- Jetson Nano Developer Kit. Available online: https://developer.nvidia.com/embedded/jetson-nano-developer-kit (accessed on 7 January 2021).
- Open Source IMU and AHRS Algorithms—x-io Technologies. Available online: https://x-io.co.uk/open-source-imu-and-ahrs-algorithms/ (accessed on 28 May 2020).
- Garcia, M. Mayitzin/ahrs. Available online: https://github.com/Mayitzin/ahrs (accessed on 7 January 2021).
- Mahony, R.; Hamel, T.; Pflimlin, J.M. Nonlinear Complementary Filters on the Special Orthogonal Group. IEEE Trans. Autom. Control
**2008**, 53, 1203–1217. [Google Scholar] [CrossRef][Green Version] - ONNX Runtime: Cross-Platform, High Performance ML Inferencing and Training Accelerator. Available online: https://github.com/microsoft/onnxruntime (accessed on 10 February 2021).
- Madgwick, S. An Efficient Orientation Filter for Inertial and Inertial/Magnetic Sensor Arrays; Report x-io and University of Bristol: Bristol, UK, 2010; pp. 113–118. [Google Scholar]

**Figure 1.**IMUs are used in various applications to measure an object’s attitude with respect to the vertical axis. A robust attitude estimator, unlike conventional filters, performs well across the different sensor hardware, motion characteristics, environmental conditions, and sampling rates without application- or trial-specific parameter tuning. Graphic based on [27,29].

**Figure 2.**The structure of the neural network (

**a**); the grouped-input neural network (

**b**) with separate layers for accelerometer and gyroscope; and the time-aware neural network (

**c**) with the time-difference as an additional input for attitude estimation.

**Figure 3.**The dataset collection is composed of six publicly available datasets, which are split into training, validation, and test data. While the validation data is used to find the best performing network configuration and hyperparameters, the test data is reserved for the final performance evaluation in Section 5.

**Figure 4.**Raw signal magnitudes over time for exemplary sequences from all six datasets. Rotational and translational characteristics vary largely.

**Figure 5.**Comparison of motion characteristics of the different datasets. Differences between datasets are considerable. The datasets BROAD, Sassari, and RepoIMU contain motions with a relatively large variety of characteristics.

**Figure 6.**Ablation study of domain-specific advances, which are added successively (bottom to top). The RMSE distributions over 12 validation sequences are compared. Adaptations of the loss function and data augmentation have the largest impact on accuracy. Bold font indicates the chosen final configuration.

**Figure 7.**Comparison of time-aware networks with training data that was resampled to frequencies drawn equidistantly from sampling time ${t}_{s}$, sampling rate ${f}_{s}$, or both combined. At least 100 different frequencies are required to achieve high accuracy. The ${f}_{s}$ space models are the most accurate across the entire frequency range.

**Figure 8.**Comparison of the mean RMSE of NN-JITR and NN-TA over an extended sampling frequency range. NN-TA performs slightly better within the trained frequency range but fails outside. Due to just-in-time resampling, NN-JITR shows stable performance over the entire extended frequency range. Bold font indicates the chosen final configuration.

**Figure 9.**Performance of NN-TA for varying network size. Performance is quantified by box plots and the mean of the RMSE over all validation sequences. Increasing the network size leads to continuous performance improvements at the cost of an exponentially growing number of parameters.

**Figure 10.**Inference latency and mean RMSE of different neural network sizes, compared to the latency of a C and native Python implementation of popular conventional filters. On both CPU and GPU, the network RIANN with 200 neurons per layer is slightly faster than the native Python implementation.

**Figure 11.**Attitude error of Filter-A and RIANN in the first 20 s of a sequence starting with rest (

**a**) or without rest (

**b**) with an unknown attitude. Filters exhibit a trade-off between taking a long time converging to a low error and having a larger error over the complete sequence. In sequence (

**a**), which starts with rest, only one filter configuration provides good results. In sequence (

**b**), which starts without rest, there is no good performing filter configuration, whereas RIANN performs similarly well as in sequence (

**a**).

**Figure 12.**Comparison of RIANN with Filter-A and Filter-B, which are optimized either on the entire training data or on the specific test dataset. Across all scenarios and datasets, RIANN performs at least as good as the conventional filters and often outperforms them, especially in the realistic scenario and partially restrictive scenario.

**Figure 13.**RIANN’s attitude RMSE distribution (over all test sequences from all datasets) plotted over different sampling rates to which all test data is resampled. The most challenging, the realistic scenario, is considered. Performance is consistent in the filters and in RIANN, with the latter achieving consistently smaller errors.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Weber, D.; Gühmann, C.; Seel, T.
RIANN—A Robust Neural Network Outperforms Attitude Estimation Filters. *AI* **2021**, *2*, 444-463.
https://doi.org/10.3390/ai2030028

**AMA Style**

Weber D, Gühmann C, Seel T.
RIANN—A Robust Neural Network Outperforms Attitude Estimation Filters. *AI*. 2021; 2(3):444-463.
https://doi.org/10.3390/ai2030028

**Chicago/Turabian Style**

Weber, Daniel, Clemens Gühmann, and Thomas Seel.
2021. "RIANN—A Robust Neural Network Outperforms Attitude Estimation Filters" *AI* 2, no. 3: 444-463.
https://doi.org/10.3390/ai2030028