Next Article in Journal
Performance Study of Distance-Weighting Approach with Loopy Sum-Product Algorithm for Multi-Object Tracking in Clutter
Next Article in Special Issue
Neuromuscular Control before and after Independent Walking Onset in Children with Cerebral Palsy
Previous Article in Journal
Credibility Assessment Method of Sensor Data Based on Multi-Source Heterogeneous Information Fusion
Previous Article in Special Issue
Development and Evaluation of a Low-Drift Inertial Sensor-Based System for Analysis of Alpine Skiing Performance
 
 
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Analysis of the Accuracy of Ten Algorithms for Orientation Estimation Using Inertial and Magnetic Sensing under Optimal Conditions: One Size Does Not Fit All

1
PolitoBIOMed Lab—Biomedical Engineering Lab and Department of Electronics and Telecommunications, Politecnico di Torino, 10129 Torino, Italy
2
Department of Excellence in Robotics & AI, The BioRobotics Institute, Scuola Superiore Sant’Anna, 56127 Pisa, Italy
3
Control Systems Group, Technische Universität Berlin, 10623 Berlin, Germany
4
Department of Biomedical Sciences, University of Sassari, 07100 Sassari, Italy
*
Author to whom correspondence should be addressed.
Sensors 2021, 21(7), 2543; https://doi.org/10.3390/s21072543
Received: 31 January 2021 / Revised: 19 March 2021 / Accepted: 24 March 2021 / Published: 5 April 2021
(This article belongs to the Collection Sensors for Gait, Human Movement Analysis, and Health Monitoring)

Abstract

:
The orientation of a magneto and inertial measurement unit (MIMU) is estimated by means of sensor fusion algorithms (SFAs) thus enabling human motion tracking. However, despite several SFAs implementations proposed over the last decades, there is still a lack of consensus about the best performing SFAs and their accuracy. As suggested by recent literature, the filter parameters play a central role in determining the orientation errors. The aim of this work is to analyze the accuracy of ten SFAs while running under the best possible conditions (i.e., their parameter values are set using the orientation reference) in nine experimental scenarios including three rotation rates and three commercial products. The main finding is that parameter values must be specific for each SFA according to the experimental scenario to avoid errors comparable to those obtained when the default parameter values are used. Overall, when optimally tuned, no statistically significant differences are observed among the different SFAs in all tested experimental scenarios and the absolute errors are included between 3.8 deg and 7.1 deg. Increasing the rotation rate generally leads to a significant performance worsening. Errors are also influenced by the MIMU commercial model. SFA MATLAB implementations have been made available online.

1. Introduction

The accurate estimation of the orientation of a rigid body from the recordings of miniaturized low-cost magneto-inertial measurement units (MIMUs) is still an open challenge for the human movement analysis community. Errors affecting the orientation estimates have a direct negative impact on the quality of estimated quantities in both angular and linear kinematics, therefore limiting the full exploitation of inertial sensing in monitoring daily-life physical activities, as well as in clinical and sports applications [1].
In its full configuration, a MIMU embeds a triaxial accelerometer which measures the specific force (i.e., the vector difference between the coordinate and the gravity accelerations), a triaxial gyroscope which measures the angular rate, and a triaxial magnetometer which senses the local magnetic field (i.e., the vector sum between the Earth’s magnetic field and the external magnetic fields created by ferromagnetic disturbances). The sensor fusion approach aims at estimating the absolute orientation of the MIMU with respect to a global coordinate system (GCS), usually defined to have a vertical axis aligned with the gravity direction and one horizontal axis direction aligned with the Earth’s magnetic north, by exploiting the complementary characteristics of the signals recorded by the MIMU. The first step consists in integrating the kinematics equation which links the angular rate with the orientation change over time to obtain a first approximation of the orientation estimate. The initial conditions for the integration can be obtained by an absolute orientation estimate by using only the accelerometer and the magnetometer measurements in absence of motion [2]. However, the orientation estimated this way is prone to drift due to the integration of the slow-varying bias affecting the gyroscope measurements [3]. To cope with this problem the accelerometer and the magnetometer measurements are employed to correct the drift in both roll and pitch angles (also jointly known as inclination) and yaw angle (also known as declination or heading). Nonetheless, such correction shows some limitations. In fact, the inclination estimated by the accelerometer is highly reliable only during static conditions (i.e., when the coordinate acceleration is negligible and only the gravity direction is sensed). Moreover, the heading resulting from the magnetometer measurements needs to be dealt with care due to the presence of external magnetic fields. Researchers from different fields such as navigation and biomechanics have proposed several sensor fusion implementations over the years, including machine and deep learning approaches, to provide accurate orientation estimates using MIMUs [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23]. The large majority of the published sensor fusion algorithms (SFAs) can be grouped in two main classes: Kalman filters (KF) [24] and complementary filters (CF). In the last decades, several formulations of both classes have been proposed including different mathematical orientation representations (e.g., quaternion, rotation matrix, Euler angles etc.), different Kalman filter formulations (direct or indirect, linear, extended, unscented, etc.), and different strategies to fuse the signal information (algebraic or optimization) [15]. Despite the large number of studies aimed at comparatively evaluating different sensor fusion algorithms [2,7,8,9,14,15,16,17,18,22,25,26,27,28,29] and type of sensors (SFA inter-consistency) [30], contradictory results have been observed and the literature is still inconclusive about the expected level of accuracy associated to the MIMU orientation estimation.
Based on the existing body of literature, it is difficult to draw conclusions regarding the “best” algorithm and filtering approach (e.g., CF or KF). Furthermore, errors appear to be highly variable depending on the experimental scenario, commercial device, and algorithm, thus making the generalization of the results impossible [18]. A way of looking at the body of literature is to group studies proposing novel algorithms (“original algorithm studies”) ([2,7,8,9,14,15,16,17,18,22]) and studies focusing on the comparison of existing algorithms/software packages without proposing new ones (“comparative studies”) ([25,26,27,28]). A summary of the results in “comparative studies” and “original algorithm studies” are reported in the Appendix A (Table A1 and Table A2). As a general observation, the magnitude of the errors reported in “comparative studies” are usually higher than those reported in the “original algorithm studies”, up to one order of magnitude. For example, the errors for Madgwick’s filter reported in the original study [9] amounted to about 1 deg, while in [25,28], which involved more challenging experimental conditions, errors were greater than 13 deg. Furthermore, the filter proposed by Guo et al., in [17] was compared with the KF by Valenti et al. [13] for which the errors were twice as large than those reported in the original article. These differences may be due to a number of reasons. First, in the “comparative studies” the experimental conditions under which SFAs are tested are often different from those employed in the original paper in terms of hardware, sensor noise, rotation rates, accelerations magnitude, ferromagnetic disturbances, type of motion, etc. In addition, when a new SFA is presented and its performance evaluated, the proponents often know how to optimally tune the SFA for the specific operating conditions based either on ground-truth knowledge or following a trial-and-error approach [31]. However, when non-experts apply SFAs to different experimental datasets, or in specific human movement applications, the SFA optimization can be difficult resulting in a performance deterioration. It has often been observed that feeding the proper parameter values to any SFA is crucial [22,27,32]. Several intrinsic and extrinsic factors affect the choice of the parameter values, among them the most influencing are amplitude of motion, sensors noise specification, time required by the algorithm to reach convergence, and the amount of ferromagnetic disturbances [31,33].
Based on the considerations above, it is evident that carrying out any “fair”, meaningful and generalizable comparative evaluation among SFAs requires the SFAs parameters to be properly tuned and used under identical experimental conditions (i.e., using the same dataset).
The primary aim of this work is to perform a thorough comparative evaluation of the accuracy of ten of the most popular SFAs proposed in the literature by considering experimental data recorded by three commercial products (Xsens-MTx (Xsens, Enschede, The Netherlands), APDM-Opal (APDM INC., Portland, OR, U.S.A.), and Shimmer-Shimmer3 (Shimmer Sensing, Dublin, Ireland)) and for three rotation rates (slow, medium, and fast motions) using the orientation provided by a multi-camera stereo-photogrammetric system (SP) as ground truth. For the sake of analysis generalizability, the SFAs performance is assessed under optimal and default parameters tuning. In the optimal tuning configuration, parameters are determined by minimizing the absolute orientation error with respect to the gold standard for each experimental scenario, allowing for the assessment of the filter performance under its best possible conditions. In addition, the errors obtained using the default parameter values as defined by the SFA proponents were also computed to highlight the impact of using non-tuned and generic parameter values for different experimental scenarios. Computation time of the different SFAs was also evaluated.
To the best of our knowledge, this study is the most comprehensive study evaluating a considerable number of SFAs under optimal filter parameter tuning condition and under various experimental scenarios.

2. Materials and Methods

2.1. Optimal Working Conditions

To work properly, each SFA requires the tuning of a variable number of parameters [32]. In the present context, optimal working conditions refer to the parameter values providing the lowest absolute average orientation error for a given experimental data recording (i.e., they are specialized for each dataset) and hence the best achievable performance (best case scenario). In other words, each parameter value of each SFA was optimally tuned on each of the nine experimental scenarios (three rotation rates for three commercial products). The selection of the optimal parameter values is performed relying on the gold standard orientation. This strategy is implemented exclusively for comparative purposes and may be replicated only with the aid of a reference orientation (e.g., SP system with sub-millimeter accuracy).

2.2. Selected Algorithms

A total of ten SFAs, including five complementary filters and five Kalman filters were selected among the most popular and performing ones: Mahony et al. 2008 [6] (MAH), Madgiwck et al., 2011 [9] (MAD), Sabatini 2011 [34] (SAB), Valenti et al., 2015 [2] (VAC), Ligorio and Sabatini 2015 [12] (LIG), Valenti et al., 2016 [13] (VAK), Seel et al., 2017 [16] (SEL), Guo et al., 2017 [17] (GUO), MATLAB complementary filter R2020a (MCF) which is the MathWorks implementation of VAC but with only two parameters, MATLAB Kalman filter R2020a (MKF), which is the MathWorks implementation of the filter by Roetenberg et al., 2005 [5,35]. The details for each SFA are reported in Table 1 including the total number of parameters exposed.
As stated in the introduction, all the SFAs are based on the angular velocity integration to obtain a first approximation of the orientation estimate. The differences are related to how the accelerometer and magnetometer measurements are used to compensate for the drift caused by the time integration of the angular velocity and to additional strategies designed to deal with the linear acceleration and the ferromagnetic disturbances. In the following, a short description of each filter is given.
MAD is a CF in which the accelerometer and the magnetometer measurements are fused by means of a gradient descent algorithm. For the magnetic readings, only the horizontal projection is used to correct the orientation. The fusion process is governed by a unique parameter. A low value of it gives more weight to the gyroscope measurements. MAH is a CF which considers the discrepancy between the measured Earth’s fixed vector (gravity and magnetic field) and their estimates obtained using the previous orientation. This discrepancy (called error) is then weighted by a parameter and subtracted from the gyroscope signal before its integration. As opposite to MAD the magnetic readings influence both the attitude and heading. In neither filter is it possible to weigh differently the accelerometer and the magnetometer contributions and no strategy is implemented to compensate for the linear acceleration or the magnetic disturbances. SEL is a CF with independent accelerometer-based inclination correction and magnetometer-based heading correction. The latter is purely horizontal, which ensures that magnetic disturbances cannot affect the inclination. The algorithm is parameterized via two correction constants for the inclination and heading disagreements, one optional bias estimation parameter and an adaptation factor that reduces the weight of the accelerometer readings during dynamic motions. VAC is a CF which employs the accelerometer readings to correct the inclination by comparing the actual and the observed gravity direction. The magnetometer readings are then projected onto the horizontal plane and the angle between the observed magnetic North and the estimated one is used to correct the heading. The two correction processes are governed by two independent gains. VAC implements a linear two-thresholds method to progressively reject the measurements whereas their magnitude exceeds the expected value (i.e., 9.81 m/s2 and the local magnetic norm, respectively). MCF is the implementation of VAC by MathWorks from Sensor Fusion and Tracking Toolbox.
VAK, LIG GUO, SAB, and MCF belong to the class of the KFs. As a general rule, the weight given to the information provided by each of the three sensors is governed by dedicated parameters. In particular, the higher is the value of these parameters the less the information provided is trusted. For this reason, they are called “inverse weight”. A typical feature of the KF is the possibility to track the disturbances with the “state-vector augmentation technique”. While on one hand it represents an advantage, on the other hand each quantity tracked in the state vector must be weighted with dedicated parameters to account for the uncertainty in their modelling and, above all, a large state vector dimension may result in observability problem (i.e., the information contained in the output variables is no longer sufficient to completely describe the system behavior).
VAK employs the same algebraic approach of VAC to correct the orientation, but as opposite to VAC no thresholds are used to reject linear accelerations or ferromagnetic disturbances which are instead employed in SAB. In addition, SAB allows the modelling of the ferromagnetic disturbances, seen as a time-variant bias superimposed to the magnetometer readings. LIG consists of two KFs which separately estimate the inclination and the heading, using the information provided by the gyroscope/accelerometer and gyroscope/magnetometer, independently. The two pieces of information are then merged by using an algebraic method. Linear accelerations and ferromagnetic disturbances are modelled as a first order Gauss-Markov model. GUO is a KF explicitly designed to perform fast. To this end, an algebraic approach which fuses the accelerometer and magnetometer measurements for the orientation correction is adopted and no additional strategy to filter out the linear accelerations and ferromagnetic disturbances is implemented. MKF is the MathWorks implementation (Sensor Fusion and Tracking Toolbox) of the filter originally proposed by Luinge et al. [4] and extended by Roetenberg et al. [5] which is also embedded in the Xsens software. Differently from the other four KFs described, MKF is an indirect KF, which means that it minimizes the uncertainty of the orientation error rather than of the absolute orientation (direct formulation). In this filter, the inclination and heading errors are separately computed by comparing the actual and the estimated directions of the gravity and global magnetic field using the information provided by gyroscope/accelerometer and gyroscope/magnetometer, respectively. These two orientation errors are then included in the state vector to be minimized. MKF augments its state vector with the gyroscope bias, acceleration errors (seen as the linear acceleration component in the accelerometer output) and the ferromagnetic disturbances.
For each SFA, the optimal tuning involved a heuristic space search of the two most important parameters (namely p 1 and p 2 ) when relevant. The decision to tune at most two parameters was a compromise between the search space dimension and the related computational time. As a general rule, considering that the gyroscope is the main source of information in a sensor fusion framework, the parameter related to the weight given to it was always tuned, when exposed by the SFA. All the remaining parameter values are set to default. In fact, it has to be said that the accelerometer and magnetometer related parameters should not be set based on the sensor noise only (i.e., electrical noise) because in the strict sense it is not representative of real problems affecting the two sensors: the linear accelerations and the ferromagnetic disturbances.
All the SFA codes have been made available on GitHub website (the link is reported in “Data Availability Statement” section). All the implementations are written in MATLAB code. All the details for each implementation are given in the code headers.

2.3. Experimental Setup

Considering that the performance of the SFAs could greatly vary due to the hardware characteristics, in this study, three pairs of commercial MIMUs were considered: Xsens-MTx, APDM-Opal, Shimmer-Shimmer3. The specifications for each model are reported in Table A3, Table A4 and Table A5 of Appendix B. A wooden board was used and the three MIMU pairs were aligned as depicted in Figure 1. A T-square was employed to draw the lines which ensured the accurate positioning of the MIMUs and the markers. The orthogonal tolerance of the instrument is reflected in an alignment error lower than 0.2 deg. The MIMUs were positioned at a relative distance of 50 mm. A total of eight reflective spherical markers (diameter equal to 14 mm, minimum inter-distance of 85 mm) were located on the board to provide the orientation reference whose trajectories were acquired by 12 infrared SP cameras (Vicon T20, VICON, Yarnton, UK; software: Nexus 2.7). The three central markers were used to define the SP Local Coordinate System (LCS) aligned with the MIMUs LCS. The additional five markers were exploited to strengthen the orientation estimation by using the singular value decomposition (SVD) technique [36]. The board was placed over an aluminum tripod (to limit the entity of the ferromagnetic disturbances) above a force platform integrated with the SP and used to synchronize the two systems by means of mechanical shocks.

2.4. Experimental Protocol

Before starting the experiments, a ten-minute instrument warm-up was performed to limit the temperature effects on the sensor readings. A first measure consisted of one-minute static acquisition with the board lying horizontally on the tripod above the force platform. This recording was used to compute the gyroscope bias which was subtracted from the angular velocities’ readings. Afterwards, a dynamic recording was executed to manually orient the board by covering the three rotational degrees of freedom. The operator held the board at both ends and performed both single-axis and multiaxial rotations (the acquisitions have been video-recorded and made available). At the end, the board was placed back on the tripod for another minute. To allow the identification of the synchronization points, two knocks were given to the board at the beginning and at the end of the recording. The described protocol was repeated at three different angular rate conditions. The RMS of the angular velocity of each recording was assessed during the post processing and amounted at 120 deg/s for a total of 70 s (slow), 260 deg/s for a total of 45 s (medium), and to 380 deg/s for a total of 30 s (fast). The temperature was kept constant at about 20 °C.
All the acquisitions were conducted in a volume of approximately 1 m3 at a distance greater than 1 m from the floor. For this reason, the ferromagnetic disturbances could be neglected as also observed in the post processing by observing the almost constant magnetometer norm (the maximum difference was limited to 1 µT).
The sampling frequency of the MIMU systems amounted to 100 Hz for Xsens (MT Manager Version 1.7) and Shimmer (Consensys v.1.5.0), and to 128 Hz for OPAL (Motion Studio Version 1.0.0.201712300). The calibrated data provided by each software were used in this work. Optical data were recorded at 100 Hz. All the MIMU synchronized data and the SP orientation (computed as described in Section 2.5.1) together with the videos recorded during the experiment have been uploaded as stated in the Data Availability Statement section at the end of the manuscript [37].

2.5. Data Processing

2.5.1. SP Data Pre-Processing and Synchronization with MIMUs Signals

Data processing was entirely carried out in MATLAB R2020a (The MathWorks Inc., Natick, MA, USA), except for the optical data which were first processed in Nexus 2.7 following the suggestion by Bergamini et al. [25]. The synchronization was performed in two steps: firstly, all data were delimited by means of the two force and acceleration peaks recorded by the vertical axes of the SP force plate and of the MIMU accelerometer, respectively. Afterward, a resampling of all the signals at 100 Hz was executed. To refine the synchronization, all the MIMU data were aligned with those of SP by cross-correlating the angular velocity recorded by each gyroscope and that estimated by the three central markers as described in [38]. The LCS orientation was estimated with respect the GCS of SP by means of SVD-based technique. The resulting gold standard orientation ( q S P G ) was expressed using quaternion. After trigonometry considerations (taking into account the cluster size of the three central markers and that the marker position errors amounts to about 0.1 mm [31,39]), it can be assumed that the errors affecting the gold-standard orientation are limited to 0.5 deg.

2.5.2. Orientation Estimation and Error Computation under Optimal Conditions

The procedure used for the orientation estimation for each SFA is detailed below and reported in the pseudocode (Algorithm 1). In the following, the procedure used to obtain the set of orientations for a given SFA for each of the nine experimental scenarios is briefly described. Quantities highlighted in bold are intended to be vectors or matrices.
An algebraic quaternion, obtained with only the accelerometer and magnetometer measurements [2], was used to initialize the orientation of each MIMU to reduce the convergence time. For each MIMU (A and B) the absolute orientation ( q A G and q B G ) was computed separately for every combination of the values of the two parameters (stored in p 1 v e c and p 2 v e c , respectively) from 0 to u p p e r 1 and from 0 to u p p e r 2 , respectively. The only exception is represented by ath2 of VAC whose lower limit was set to the value of the first accelerometer threshold (a lower value would be meaningless). The values for u p p e r 1 and u p p e r 2 were chosen to be large enough to ensure the exploration of all the relevant search space. In other words, errors obtained for values of p 1 and p 2 set to u p p e r 1 and u p p e r 2 , respectively, are large. Figures in Appendix C display the values chosen for u p p e r 1 and u p p e r 2 for each SFA. The number of points for each parameter interval (i.e., the length of the p 1 v e c and p 2 v e c vectors) was different for each algorithm and it is a trade-off between the search space dimension and the computational cost; on average about 360 solutions (i.e., length ( p 1 v e c ) × length ( p 2 v e c )) were explored for each SFA. Since the GCSs of the MIMU and SP were not aligned on the horizontal plane, to enable a meaningful comparison between the orientation obtained for the two systems, it was necessary to refer the latter to a common GCS. To this end, it was possible to benefit from the accurate alignment of the LCS of each system. Therefore, q A G , q B G , and q S P G were separately referred to their initial frame to obtain q A , q B , and q S P , respectively, as follows (the and * operators represent the product and complex conjugate operator in the quaternion algebra, respectively):
q A =   q A G ( 1 ) *   q A G , q B =   q B G ( 1 )   *   q B G , q SP =   q S P G ( 1 )   *   q S P G .
The absolute orientation errors Δ q a b s   A and Δ q a b s   B were computed in the quaternion form as follows:
Δ q a b s   A =   q A *   q S P , Δ q a b s   B =   q B *   q S P .
To obtain a compact representation of the errors and the relative difference, the absolute rotation angle was computed from the scalar component of each quaternion Δ q a b s   A and Δ q a b s   B to obtain Δ θ a b s   A and Δ θ a b s   B , respectively. Then, the two absolute error values Δ θ a b s   A and Δ θ a b s   B were averaged to obtain Δ θ a b s . Lastly, the RMS value of Δ θ a b s was computed only during the dynamic portions of the recording to obtain e p 1 ,   p 2 . This procedure was repeated for each combination of p 1 v e c and p 2 v e c to populate the e matrix which contains the absolute errors which were rounded to an accuracy of 0.1 deg.
Algorithm 1. Pseudocode to detail the orientation estimation process for each SFA.
for each pair of MIMUs (Xsens, APDM, and Shimmer)
  for each angular rate condition (slow, medium, fast)
remove the static bias for each gyroscope
compute the starting orientation for each MIMU
initialize the matrix e (#rows = length( p 1 ), #columns = length( p 2 ))
for each value p 2 belonging to p 2 v e c between [ 0 , u p p e r 2 ]
for each value p 1 belonging to p 1 v e c between [ 0 , u p p e r 1 ]
compute the absolute orientation of each MIMU separately with the SFA under analysis to obtain q A G and q B G
refer q A G and q B G to the starting orientation to obtain q A and q B , as done in (1)
compute the absolute orientation error of q A and q B separately using the gold standard q S P to obtain Δ q a b s   A and Δ q a b s   B , as done in (2)
convert Δ q a b s   A and Δ q a b s   B into angular rotation errors to obtain Δ θ a b s   A and Δ θ a b s   B
compute the average value between the two absolute errors to obtain Δ θ a b s
compute the RMS of Δ θ a b s considering only the dynamic parts of the recording to obtain e p 1 ,   p 2
add e p 1 ,   p 2 to the matrix e
end
end
find the optimal region of ( p 1 v e c , p 2 v e c ) which correspond to the range of e which includes its minimum ( e o p t ) + 0.5 deg to obtain p o p t _ 1 and p o p t _ 2
find the value of e which correspond to the default parameter values to obtain e d e f
end
end

2.6. Data Analysis

The evaluation of the algorithms’ performance followed the steps described in the following. The first step consisted in identifying the optimal regions and the corresponding errors for each SFA and for each experimental scenario (i.e., 90 errors in total, 10 SFAs × 3 rotations rates × 3 commercial products). The absolute orientation errors were also computed for each SFA and for each scenario using the default parameter values listed in Table 1 (i.e., 90 errors) which were provided by the authors in their papers or in the original implementations of their SFAs. Then, the influence of the following factors on the absolute orientation accuracy was analyzed:
  • SFA analytical formulation
  • rotation rate magnitude
  • different commercial products.
To this extent, a statistical analysis was performed by aggregating data according to the influencing factor under inspection.
Finally, an analysis of the computation time of each SFA was performed by measuring the amount of time needed by each SFA to perform a single orientation update iteration (i.e., a single time step).

2.6.1. Identification of the Optimal Regions and the Corresponding Errors

The optimal region, for each scenario, is defined as the combination of p 1 and p 2 which corresponds to the minimum of absolute orientation error.
For each scenario, the following quantities were determined:
  • Minimum absolute orientation error which corresponds to the selection of the optimal parameter values: e o p t = min ( e ( p 1 , p 2 ) ) , where e is the matrix of the average errors between the two MIMUs of dimensions equal to [length( p 1 ), length( p 2 )].
  • Optimal parameter region is defined as the range of parameter values for which the relevant orientation errors are equal to the minimum error plus 0.5 deg (i.e., the SP uncertainty band, as stated in Section 2.5.1). These regions are defined as: { p o p t 1 , p o p t 2 } = { ( p 1 , p 2 )   |   e   e o p t + 0.5   d e g } . An example of optimal region is illustrated in Figure 2 for the VAK filter. When only one parameter was tuned (MAD, VAC, GUO, MKF) e was a vector and the optimal region degenerated into a 1D interval.

2.6.2. Identification of the Default Errors

The absolute error corresponding to the default values of ( p 1 D E F , p 2 D E F ) was obtained as: e D E F =   e ( p 1 _ D E F , p 2 _ D E F ) .

2.6.3. Statistical Analysis to Evaluate the Influence of the SFAs and of the Experimental Factors

To evaluate the influence of the different factors on the 90 values of e o p t were aggregated as explained in the following Table 2:
For each influencing factor, the normality of each distribution was tested with a Shapiro-Wilk’s test suitable for small sample size. Since it turned out that the distributions were not normal, the nonparametric Friedman’s test was applied to assess whether significant differences existed among the error distributions. When the null hypothesis was rejected, post-hoc multiple comparison tests were applied to perform pairwise comparisons. In particular, to test the SFA influence, the Tukey’s honest significant difference criterion was used since it is less strict than Bonferroni having 10 distributions [40]. Bonferroni’s correction was instead used to perform pairwise comparisons when testing the rotation rate and commercial product effects.

2.6.4. Computation Time of the Different SFAs

The average execution time for a single iteration of each SFA is measured for an Intel® Core™ i7-10510U CPU @ 1.80 GHz (Intel ©, Santa Clara, CA, USA)—Microsoft™ Windows 10 (Microsoft ©, Redmond, WA, USA) when processing a dataset of 25386 samples without executing any other programs.

3. Results

3.1. Optimal and Default Errors

The minimum absolute error ( e o p t ) and the errors corresponding to default parameter values ( e d e f ) are reported in Table 3 for each SFA and for each experimental scenario.

3.2. Optimal Regions

The optimal regions for each experimental scenario for each SFA are reported in the figures in Appendix C for sake of completeness. Figure 2 provides an example of the optimal regions determined for each of the nine experimental scenarios in the case of VAK. In more detail, the optimal region identified by p o p t _ 1 and p o p t _ 2 is represented with a different color for each scenario.

3.3. Influence of the Experimental Factors on the Absolute Accuracy

3.3.1. Influence of the Specific SFA (3 Rotation Rates × 3 Commercial Products)

Mean ± STD of the eopt values obtained by each SFA are listed in ascending order in Table 4.
As evident from Table 3, in many conditions the use of the default parameters (listed in Table 1) does not guarantee for low values of e d e f for each SFA. For this reason, the analyses performed in the following sections to investigate the influence of the rotation rate and the commercial product were limited to e o p t distributions (it would be meaningless for those of e d e f ).
The Shapiro-Wilk test, (α = 0.05) revealed that not all the e o p t distributions were normal (p < 0.05). The small p-value (0.0035) resulting from the Friedman’s test cast doubts on the validity of the null hypothesis. A multiple comparison test with Tukey’s correction (α = 0.05) revealed that no statistically significant differences existed among the 10 SFAs under optimal working conditions.

3.3.2. Influence of the Rotation Rate (10 SFA × 3 Commercial Products)

Optimal distributions are represented in Figure 3 for slow, medium, and fast rotation rates. In addition, the mean ± STD errors for each rotation rate scenario are reported in Table 5.
The Shapiro-Wilk test, (α = 0.05) revealed that not all distributions were normal (p < 0.05). The small p-value (<1 × 10−9) resulting from the Friedman’s test cast doubts on the validity of the null hypothesis. A multiple comparison test with Bonferroni’s correction (α = 0.05) revealed a statistically significant difference among the three distributions (Table 6).

3.3.3. Influence of the Commercial Product (10 SFA × 3 Rotation Rates)

Noise description for each sensor of each MIMU and gyroscope bias were reported in Table A4 and Table A5, respectively (Appendix B) [41].
Optimal distributions are represented in Figure 4 for Xsens, APDM, and Shimmer commercial products. In addition, the mean ± STD errors for each commercial product scenario are reported in Table 7.
The Shapiro-Wilk test, (α = 0.05) revealed that not all the distributions were normal (p < 0.01). The small p-value (<1 × 10−7) resulting from the Friedman’s test cast doubts on the validity of the null hypothesis. A multiple comparison test with Bonferroni’s correction (α = 0.05) revealed a statistically significant difference among the three distributions (Table 8).

3.4. Computation Time of the Different SFAs

In Figure 5, the average execution time of each SFA is reported for a single iteration.

4. Discussion

4.1. The Importance of Properly Tuning Each SFA

Results from the present study have confirmed that the selection of appropriate parameter values plays a fundamental role in determining the level of orientation accuracy and that parameters must be tuned differently based on the experimental scenario [15,27,31], thus enabling the best possible performance.
We found that each SFA exhibits the optimal performance only for a limited interval of parameter values. If the parameter values are optimized for a specific experimental scenario, the same values can lead to large errors when varying the experimental conditions; the only exception is represented by the MKF with a common intersection between σ g y r 2 = [0.0125, 0.0275] (rad/s)2. This evidence can be also graphically observed in the figures in Appendix C, in fact for all algorithms but MKF there is not a common intersection among the optimal regions when varying the experimental scenario. The above-mentioned figures also prove that for some SFAs (e.g., SAB), the specific tuning for each experimental scenario is particularly critical since the overlapping among the optimal regions is very limited. At the same time the errors obtained using the default parameter values ( e d e f ) highlight the inadequacy to estimate the absolute orientation with the same parameter values for a given SFA under different experimental scenarios. These findings provide a further justification of the different level of accuracy reported for the different SFAs in previous studies which entailed the comparison among filters optimally tuned and filters fed with the default or non-optimal parameters (e.g., [14,16]). It is clear that any comparison carried out without a common strategy to tune the SFA parameter values would be lacking generality. In fact, for some algorithms e d e f can greater than 100 deg (see Table 3). It is worth pointing out that the optimal errors reported in Table 3 can be considered the lower bound for those SFAs under similar experimental conditions. Lower errors can be only obtained under less challenging scenarios and/or using higher performing MIMUs.
As already mentioned, the optimal parameter tuning requires the orientation reference to be available (e.g., a SP system). Since the MIMUs are conceived to be used also outside the laboratory, the proposed approach is not always feasible and there is the need to search for alternative strategies for the selection of reasonable parameter values without using any orientation reference. To the best of our knowledge, the procedure described in [31] is the only one which meets this requirement by exploiting the fact that two MIMUs aligned on a rigid body must have a null orientation difference during any movement.

4.2. Influence of the SFA and of the Experimental Factors on the Absolute Accuracy

The influence of the specific SFA and the filtering class (CF or KF) on the orientation accuracy has been widely investigated by several authors (e.g., [7,25]). We found, over SFAs, errors ranging from 3.8 deg to a maximum of 7.1 deg, but no statistically significant differences. This suggests that, based on the present study, it is not possible to identify the best performing SFA and that a proper fine tuning of the parameter values can be the key point to obtain a reasonable absolute accuracy, regardless of the filtering class or the total number of parameters exposed by the SFAs. In fact, it is possible to observe from Table 1 and Table 3 that a larger number of parameters does not necessarily guarantee for a better accuracy with respect to a SFA with only one parameter to be set. This can be explained by the fact that, ideally, the several sources of errors can be better modelled by filters with a large number of parameters. On the other hand, their tuning is more difficult since the final orientation estimate is strongly influence by the mutual influence of the parameters. Among the ten tested SFAs, LIG exhibited the lowest average errors while VAC and MCF the highest being the average differences under 0.5 deg (this is expected since they are the implementations of the same filter described in [2]). It has to be said that this experimental design was not conceived to enhance small differences across the performance of ten algorithms due to the weak statistical power. If of interest, several repetitions for each experimental scenario would need to be collected.
Results showed that even under optimal parameter tuning orientation accuracy is strongly influenced by the experimental factors considered.
The influence of the rotation rate magnitude has been recognized by previous works ([26,27,33,42]) in which accuracy worsening was observed when the rotation rate increased, however few SFAs were tested. In general, higher values of rotation rate are associated with higher linear accelerations (except when the MIMU is coincident with the center of rotation), which are directly reflected on the specific force recorded by the accelerometer. Since the accelerometer aids the sensor fusion process by providing the gravity direction information to compensate for the inclination drift, when the gravity recording is corrupted by high values of linear acceleration then the accelerometer contribution becomes detrimental. Many algorithms cope with this problem by rejecting the accelerometer information when the magnitude of its measures overcome a certain threshold. However, as highlighted by Fan et al., in [18], despite the simplicity of this strategy that can be adopted by both CF and KF classes, the main drawback is the choice of the threshold value and the resulting orientation instability for accelerometer values close to the threshold. Observing the results in Table 5, the effect of the rotation rate is confirmed also for all the SFAs analyzed: errors obtained at the fast rotation rate are worse than those at slow rotation rate and the three distributions statistically differ across all of them (Figure 3). In particular, the performance worsens of 3.8 ± 2.1 deg on average. The minimum worsening amounts to 0.2 deg for SAB-Xsens and the maximum to 9.7 deg for GUO-Shimmer.
Finally, it is known that the hardware components embedded in the commercial products, although sharing similar specifications (measurement ranges, sensitivity, resolution, etc.) exhibit different noise levels both in terms of STD and offset and bias instability, which in turns influence the estimation accuracy [26,33,42]. In particular, the most critical factor when estimating the orientation is the slow-varying bias affecting the gyroscopes. Table A3 and Table A4 reported in the Appendix B show the measured noise levels for different commercial products and the difference between the gyroscope bias computed at the beginning and at the end of the recordings. The differences between the noise levels of two MIMUs of the same commercial product are evident. Obviously, when considering different commercial products, the differences are even higher. Moreover, the bias of the gyroscopes changed during the same recordings of a few minutes. The problem of a slow varying bias is one of the major problems to address when estimating orientation since the angular velocity is the main source of information in a sensor fusion process. This may partially explain the significant difference found in Table 8 and shown in Figure 4 between Xsens and APDM for which the bias changes are up to two orders of magnitude higher. However, the exact mechanism with which the noise level is reflected on the absolute accuracy is still not completely clear due to the high number of variables involved in the sensor fusion process. It is worth highlighting, as a limitation, that the different locations of the MIMUs on the board, which lead to different acceleration magnitudes, can have a minor and limited influence on the results. In fact, as shown in Figure 4, the medians of the APDM and Shimmer distributions are extremely similar. Overall, Xsens showed the smallest average errors of 3.5 deg, while Shimmer the highest of 6.5 deg.

4.3. Computation Time of the SFAs

Depending on the specific application, the computation time required by each SFA to compute the orientation may be crucial. In those applications providing feedback to the patient such as the tele-rehabilitation and neuroprosthesis systems, the “near” real-time estimation is a fundamental requirement [43].
It is observed that most tested CFs are faster than the KFs. The two exceptions are GUO (a KF explicitly designed by the authors to perform fast), which resulted to be the second fastest and MCF (a CF), which is the MATLAB implementation of Valenti 2015 (VAC). MCF is more than 32 times slower than VAC, despite the mathematical formulation being the same. This aspect has been analyzed through the MATLAB profiler and it emerged that the quaternion library used by MATLAB (more specifically the creation of a quaternion object) is the limiting factor. The same applies to MKF (the MathWorks implementation of [5] embedded in the “Sensor Fusion and Tracking Toolbox”) which is the slowest KF. The higher computation time of Kalman filters can be due to the matrix operations involved in the mathematical implementation including multiplications and inversions. Finally, a monotonic relationship between the dimension of the state vector and the time required to complete an iteration can be observed. This is justified by the fact that an increase in the state vector dimension involves matrices of higher dimensions (e.g., state transition, process and measurement covariance matrices, etc.) with a consequent increase of the computational burden. It has to be said that not all the MATLAB implementations were optimized by the authors to perform fast. The same implementations written in another programming languages such as C/C++ could be executed much faster.

5. Conclusions

In this study the two most influencing parameters were identified and optimally tuned for each SFA, thus obtaining the corresponding errors, which are indicative of the best possible performance under each tested experimental scenario. Moreover, as suggested by recent literature [15,27,31,44,45], this study confirms that the selection of the value of each parameter is crucial to obtain a satisfying performance of each SFA despite its filtering class or mathematical formulation. The use of fixed parameter values may be not suitable for every scenario since, in general, the optimal value regions do not intersect. The authors stress the importance of comparing the performance of the different SFAs only when all of them are run under the same working conditions and suggest all future works to follow this recommendation. This study also analyzes how accuracy is influenced by the SFA, the rotation rate, and the commercial product.
An important finding is that all methods exhibited errors within a range of 3.3 deg (from 3.8 deg to 7.1 deg). Therefore, it is difficult to rank the ten implemented SFAs, and it is not possible to identify the best performing SFA since no statistically significant differences were found.
Errors at high rotation rate are statistically different from those obtained at low rotation rate. On average, the lowest errors associated with a slow movement are expected to be about 3.5 deg and they could increase up to 7.3 deg when the rotation rate reaches an RMS value of about 380 deg/s. The errors are also influenced by the model of the commercial product. Xsens provided the lowest average errors (3.5 deg) and statistically different with respect to APDM (6.0 deg), and Shimmer (6.5 deg).
Finally, complementary filters were found to be faster than Kalman filters. This can be substantiated by the mathematical implementation of the filters. Two exceptions were observed: GUO (a KF explicitly designed by the authors to perform fast) and MCF (the MATLAB CF). In addition, an increase of the state vector dimension is monotonically reflected on the time required for each iteration to estimate the orientation.
In conclusion, this study shows the importance of equally tuning the parameter values for each SFA in order to enable a meaningful comparison among the different algorithms. In addition, performances are strongly influenced by the experimental conditions. It has to be pointed out that while this contribution, for the best of our knowledge, is the most extensive so far (10 SFAs × 3 motion intensities × 3 commercial products) there are many relevant aspects that are also worth considering when evaluating the accuracy of SFAs, such as the effect of translations, long uninterrupted motion phases, and the influence of magnetic disturbances. The proposal of a complete and standard benchmark to test the SFAs under different experimental scenarios would be beneficial for any future comparison. As pointed out by Nazarahari and Rouhani [29] there is still the lack of a shared protocol and the movement analysis community should move in this direction.

Author Contributions

Conceptualization, M.C. and A.C.; methodology, M.C. and A.C.; software, M.C., A.M.S., D.L. and T.S.; validation, M.C., A.C.; formal analysis, M.C.; resources, U.D.C.; data curation, M.C.; writing—original draft preparation, M.C.; writing—review and editing, M.C., A.C., A.M.S., D.L., T.S., M.K. and U.D.C.; supervision, A.C., M.K.; project administration, A.C.; funding acquisition, A.C. All authors have read and agreed to the published version of the manuscript.

Funding

This research was partially funded by DoMoMEA grant, Sardegna Ricerche POR FESR 2014/2020 and by MOBILISE-D grant, Innovative Medicines Initiative 2 Joint Undertaking under grant agreement No 820820. This Joint Undertaking receives support from the European Union’s Horizon 2020 research and innovation programme and EFPIA www.imi.europa.eu.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The sensor fusion algorithm together with the optimization codes are available as MATLAB functions and scripts at https://github.com/marcocaruso/sensor_fusion_algorithm_codes (accessed on 27 March 2021). The complete dataset is available on both IEEE DataPort at http://dx.doi.org/10.21227/b23b-rx94 (last updated on 9 July 2020) [37] and also at https://github.com/marcocaruso/mimu_optical_dataset_caruso_sassari (accessed on 27 March 2021).

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Glossary

The main acronyms and definitions are summarized below for reader convenience:
CFcomplementary filter
GCSglobal coordinate system
KFKalman filter
LCSlocal coordinate system
MIMUmagneto-inertial measurement unit
RMSroot mean square
SFAsensor fusion algorithm
SPstereo-photogrammetric system
STDstandard deviation
Absolute orientationthe orientation of the local coordinate system (LCS) of a system with respect to its GCS
Absolute orientation errorthe difference between the orientation of the LCS of a magneto-inertial measurement unit (MIMU) computed by a sensor fusion algorithm (SFA) and its actual orientation computed by the optical reference (SP) and expressed by the angle given by the axis-angle convention
e d e f absolute orientation error corresponding to the selection of the default parameter values
e o p t minimum absolute orientation error which corresponds to the selection of the optimal parameter values
Optimal parameter regionthe range of parameter values for which the orientation errors are equal to e o p t plus 0.5 deg

Appendix A

In this Appendix are shown the results of the “original algorithm studies” (Table A1) and the “comparative studies” (Table A2).
Table A1. Results of the Original Algorithm Studies. All the Error Units Are in Degrees.
Table A1. Results of the Original Algorithm Studies. All the Error Units Are in Degrees.
First AuthorSFA(s)
Employed
MIMU(s)
Employed
Experimental ProtocolStandardDeclared Errors
Picerno 2011 [30]Xsens filter (KF)9 Xsens MTwMIMUs were aligned on a rigid body which was oriented in 12 different poses. Only the static orientation was considered.noneDifferences were up to 11.4 for yaw angle (in an ideal case they should be null).
Bergamini 2014 [25]Madgwick [9] (CF)
Sabatini [34] (KF)
1 APDM-OpalManual tasks (slow velocities, short time, and small capture volume).
Locomotion task (larger capture volume, three minutes, no static phases).
SPRMS errors were similar for CF and KF: from 5.5 (manual) to 21 (locomotion) tasks.
Lebel 2015 [26]Xsens filter (KF)
APDM filter (KF)
Inertial Labs filter (CF)
4 Xsens-MTx
4 APDM-Opal
4 Inertial Labs-Osv3
MIMUs attached to a gimbal.
Rotation of the gimbal axes to obtain both planar (2D) and 3D motions at quasi-constant low and high rotation rates (90 dps and 180 dps) for 120 s.
SPMean errors increased up to 7 when the rotation rate increased, although this was less evident for Xsens filter.
Ricci 2016 [27]APDM filter (KF)
Tian [46] (CF)
6 APDM-OpalMIMUs attached to a robot arm Static (different orientations)
Dynamic (sinusoidal rotations around MIMU axis, the RMS of the angular velocity ranged from 2.1 dps to 150 dps).
Robot anglesIn static the maximum errors amounted to 1 for CF and 1.6 for KF. In the dynamic trials, the KF exhibited the best performance. Errors increased when the velocity increased.
Ludwig 2018 [28]Madgwick [9] (CF)
Mahony [6] (CF)
Marins [47] (KF)
1 embedded on a quadcopterThe quadrotor flew to perform both loop and random sequences within the volume capture (1m × 1m × 1m).SPRMS errors amounted to 11, 13, and 13.3 for Mahony, Madgwick and Marius, respectively.
Table A2. Results of the “Comparative Studies”. All the Error Units Are in Degrees.
Table A2. Results of the “Comparative Studies”. All the Error Units Are in Degrees.
First AuthorSFA(s)
Employed
MIMU(s)
Employed
Experimental ProtocolStandardDeclared Errors
Young 2009 [7]Proposed CF
Yun [48] (KF)
1 OrientGently movements by hand (20 s).
Walking with MIMU on the lower leg (five trials, 30 s each).
SPRMS inclination (and yaw) errors. Gently movements: 3.2 (9.6) for CF and 5.4 deg (10.1) for KF. Walking: 4 deg (10.9) for CF and 11.9 (31.8) for KF.
Fourati 2014 [8]Proposed CF
Xsens filter (KF)
1 Xsens-Mti
1 Xsens-MTi-G
Manual (3 straight translations along each MIMU axis and a free 3 D motion).noneAverage inclination (and yaw) difference between the estimates of the CF and KF ranged between [1,2,3] ([2,3,4,5]).
Madgwick [9]Proposed CF
Xsens filter (KF)
1 Xsens-MTxStatic.
Dynamic (manual motions).
SPRMS errors in dynamic amounted to 1.1 for the proposed and 1.3 for KF.
Valenti 2015 [2]Proposed CF
Sabatini [34] (KF)
Madgwick [9] (CF)
PhidgetSpatial 3/3/3 embedded in a quadrotorThe vehicle flew in a volume of 10 m × 10 m × 10 m to perform loop trajectories.SPRMS inclination (and yaw) errors amounted to 1.7 (16.6), 2.5 (20.3), and 3 (76.2) for the proposed CF, the KF, and the CF by Madgwick.
Marantos 2016 [14]Proposed CF
St iNEMO filter (KF)
Mahony [6] (CF)
DesignedMIMU mounted on a gimbal which was subjected to a combined motion in all axes including strong acceleration and ferromagnetic disturbances.Gimbal encodersMean inclination (and yaw) errors amounted to 1.0 (2.5), 5.5 (10.5), and 4.9 (11.5) for the proposed CF, the CF from Mahony and KF.
Olivares 2016 [15]Two proposed KFs (optimal approach vs. algebraic solution)1 Wagyromag (designed)MIMU mounted on a device moved at 3 speeds including high accelerations and magnetic disturbances.Potentiometer—mounted on the hinge.Mean RMS errors: 1.5 for KF with optimal approach and 2.1 for KF with algebraic solution.
Seel 2017 [16]Proposed CF
Madgwick [9] (CF)
noneSimulated magnetic field environment.Synthetic ground-truth.Inclination errors for Madgwick CF were higher than 4.
Guo 2017 [17]Proposed KF
Fourati 2014 [8] (CF)
Marantos 2016 [14] (CF)
Valenti 2016 [2] (KF)
1 MicroStrain 3DM-
GX3-25
Random movements carried out by hand.Provided by the proprietary algorithm (not properly a gold standard).Mean inclination (and yaw) errors amounted to 0.1 (0.1), 0.2 (0.7), 1.3
(0.6), and 2.2 (3.7) for the proposed KF, the CF by Fourati, the CF by Marantos, and the KF by Valenti.
Fan 2018 [18]Basic proposed CF
Finite-state proposed CF
1 Xsens-MTwFast MIMU movement up and down (range of 60 cm) close to a big ferromagnetic box.SPRMS inclination (and yaw) errors 6 (7) and 1
(1) for basic CF and finite-state CF during large acceleration. RMS Yaw errors 7 and 1 for basic CF and finite-state CF during magnetic disturbances.
Weber 2020 [22]Proposed neural-network
See1 2017 [16] (CF)
1 Myon AG-aktos-tRoto-translation movements at different speeds including pauses.SPRMS errors equal to 1.4 for the neural network and to 2.8 for the CF.

Appendix B

This appendix provides a list of the characteristics of the MIMUs considered in the paper. Table A4 reports the noise standard deviation of the sensors of each unit computed over one minute of static acquisition.
Table A3. Sensor STD Computed during One Minute of Static Acquisition.
Table A3. Sensor STD Computed during One Minute of Static Acquisition.
STDAccelerometer (mg)Gyroscope (deg/s)Magnetometer (µT)
xyzxyzxyz
Xsens-MTx #10.860.800.850.380.390.370.060.040.04
Xsens-MTX #20.820.860.800.440.400.400.050.060.06
APDM-OPAL #10.380.330.380.160.230.110.260.230.20
APDM-OPAL #20.340.320.350.160.270.190.260.250.20
Shimmer-Shimmer 3 #11.060.971.260.090.080.090.840.840.69
Shimmer-Shimmer 3 #21.121.091.290.060.060.060.970.970.58
#1 and #2 denote the two units of the same commercial product.
Table A3 highlights the differences among the measured noise levels for the sensors embedded in the commercial products under analysis. As expected, the differences are higher [49] when considering sensors from different commercial products. However, by observing the noise values of two units of the same model it is worth noting that they are not exactly the same.
The values for each gyroscope bias computed in the static parts at the beginning and at the end of each recording are listed in Table A4 to appreciate their changes. In particular, for each MIMU and for each recording, the values computed at the beginning (before) and their difference (diff) with respect to the end are reported.
Table A4. Sensor Gyroscope Biases during the First 60 s of Static Acquisition before the Movements (before) and Difference (diff) with the Bias Record at the End of Experiments.
Table A4. Sensor Gyroscope Biases during the First 60 s of Static Acquisition before the Movements (before) and Difference (diff) with the Bias Record at the End of Experiments.
Gyroscope Bias (deg/s)
SlowMediumFast
xyzxyzxyz
Xsens-MTx #1before−0.24−1.70−0.32−0.26−1.70−0.33−0.25−1.69−0.33
diff0.00−0.050.00−0.010.00−0.02−0.01−0.01−0.02
Xsens-MTX #2before−0.260.760.42−0.260.760.43−0.280.740.41
diff0.010.010.020.030.000.03−0.01−0.02−0.02
APDM-OPAL #1before0.78−0.570.340.59−0.800.360.74−0.780.37
diff0.080.04−0.02−0.12−0.020.00−0.020.12−0.01
APDM-OPAL #2before−1.10−0.06−0.71−1.20−0.05−0.48−1.110.17−0.48
diff0.070.01−0.03−0.17−0.17−0.05−0.09−0.03−0.10
Shimmer-Shimmer 3 #1before−0.03−0.06−0.01−0.02−0.050.01−0.03−0.070.02
diff−0.01−0.030.00−0.010.000.000.00−0.010.00
Shimmer-Shimmer 3 #2before−0.06−0.030.09−0.06−0.030.08−0.06−0.050.10
diff−0.01−0.030.01−0.01−0.030.01−0.01−0.030.03
#1 and #2 denote the two units of the same commercial product.
By observing Table A4, the first conclusion that can be drawn is that the biases are not constant within the same recording even after less than one minute and the maximum change arose to 0.17 deg/s.
It is worth highlighting that, despite a proper characterization of the bias instability can be only obtained by the analysis of the Allan variance over a long time recording [50,51], the values reported in Table A4 give an overview of the severity of the gyroscope bias variation during the same logging. As evident from the literature, several efforts have been done to try to estimate the bias online (e.g., [6,9,16,35]). In fact, the problem of a slow-varying bias is one the major when estimating the orientation since its variations are very difficult to be predicted and modelled.
Finally, Table A5 lists the measurement characteristics of each sensor embedded in the employed MIMUs.
Table A5. Sensor Specifications. Reprinted with permission from ref. [31]. Copyright 2020 IEEE.
Table A5. Sensor Specifications. Reprinted with permission from ref. [31]. Copyright 2020 IEEE.
RangeA/D ResolutionAlignment Error
Xsens-MTx
Accelerometer±50 m/s216 bits0.1 deg
Gyroscope±1200 deg/s16 bits0.1 deg
Magnetometer±75 µT16 bits0.1 deg
APDM-OPAL
Accelerometer±16 m/s214 bits
Gyroscope±2000 deg/s16 bits
Magnetometer±800 µT12 bits
Shimmer-Shimmer3
Accelerometer±16 m/s216 bits
Gyroscope±2000 deg/s16 bits
Magnetometer±400 µT16 bits

Appendix C

In this appendix the optimal regions are reported for each SFA. When only one parameter was tuned (MAD, MCF, GUO, MKF) the bidimensional regions degenerated into a mono-dimensional interval.
Figure A1. Optimal regions for each SFA and for each experimental condition. (a) MAH, (b) SAB, (c) MAD, (d) LIG, (e) VAC, (f) VAK, (g) SEL, (h) GUO, (i) MCF, (j) MKF.
Figure A1. Optimal regions for each SFA and for each experimental condition. (a) MAH, (b) SAB, (c) MAD, (d) LIG, (e) VAC, (f) VAK, (g) SEL, (h) GUO, (i) MCF, (j) MKF.
Sensors 21 02543 g0a1aSensors 21 02543 g0a1bSensors 21 02543 g0a1c

References

  1. Cereatti, A.; Trojaniello, D.; Della Croce, U. Accurately measuring human movement using magneto-inertial sensors: Techniques and challenges. In Proceedings of the 2015 IEEE International Symposium on Inertial Sensors and Systems (ISISS) Proceedings, Hapuna Beach, HI, USA, 23–26 March 2015; pp. 1–4. [Google Scholar] [CrossRef]
  2. Valenti, R.G.; Dryanovski, I.; Xiao, J. Keeping a good attitude: A quaternion-based orientation filter for IMUs and MARGs. Sensors 2015, 15, 19302–19330. [Google Scholar] [CrossRef][Green Version]
  3. Sabatini, A.M. Quaternion-Based extended Kalman filter for determining orientation by inertial and magnetic sensing. IEEE Trans. Biomed. Eng. 2006, 53, 1346–1356. [Google Scholar] [CrossRef]
  4. Luinge, H.J.; Veltink, P.H. Inclination measurement of human movement using a 3-D accelerometer with autocalibration. IEEE Trans. Neural Syst. Rehabil. Eng. 2004, 12, 112–121. [Google Scholar] [CrossRef]
  5. Roetenberg, D.; Baten, C.T.M.; Veltink, P.H. Estimating body segment orientation by applying inertial and magnetic sensing near ferromagnetic materials. IEEE Trans. Neural Syst. Rehabil. Eng. 2007, 15, 469–471. [Google Scholar] [CrossRef] [PubMed][Green Version]
  6. Mahony, R.; Hamel, T.; Pflimlin, J.M. Nonlinear complementary filters on the special orthogonal group. IEEE Trans. Autom. Control 2008, 53, 1203–1218. [Google Scholar] [CrossRef][Green Version]
  7. Young, A.D. Comparison of orientation filter algorithms for realtime wireless inertial posture tracking. In Proceedings of the 2009 Sixth International Workshop on Wearable and Implantable Body Sensor Networks, Berkeley, CA, USA, 3–5 June 2009; pp. 59–64. [Google Scholar] [CrossRef]
  8. Fourati, H.; Manamanni, N.; Afilal, L.; Handrich, Y. Complementary observer for body segments motion capturing by inertial and magnetic sensors. IEEE ASME Trans. Mechatron. 2014, 19, 149–157. [Google Scholar] [CrossRef][Green Version]
  9. Madgwick, S.O.H.; Harrison, A.J.L.; Vaidyanathan, R. Estimation of IMU and MARG orientation using a gradient descent algorithm. In Proceedings of the 2011 IEEE International Conference on Rehabilitation Robotics, Zurich, Switzerland, 29 June–1 July 2011; pp. 1–7. [Google Scholar] [CrossRef]
  10. Calusdian, J.; Yun, X.; Bachmann, E. Adaptive-Gain complementary filter of inertial and magnetic data for orientation estimation. In Proceedings of the IEEE International Conference on Robotics and Automation, Shanghai, China, 9–13 May 2011; pp. 1916–1922. [Google Scholar] [CrossRef][Green Version]
  11. Mazzà, C.; Donati, M.; McCamley, J.; Picerno, P.; Cappozzo, A. An optimized Kalman filter for the estimate of trunk orientation from inertial sensors data during treadmill walking. Gait Posture 2012, 35, 138–142. [Google Scholar] [CrossRef] [PubMed]
  12. Ligorio, G.; Sabatini, A.M. A novel Kalman filter for human motion tracking with an inertial-based dynamic inclinometer. IEEE Trans. Biomed. Eng. 2015, 62, 2033–2043. [Google Scholar] [CrossRef]
  13. Valenti, R.G.; Dryanovski, I.; Xiao, J. A linear Kalman filter for MARG orientation estimation using the algebraic quaternion algorithm. IEEE Trans. Instrum. Meas. 2016, 65, 467–481. [Google Scholar] [CrossRef]
  14. Marantos, P.; Koveos, Y.; Kyriakopoulos, K.J. UAV State Estimation Using Adaptive Complementary Filters. IEEE Trans. Control Syst. Technol. 2016, 24, 1214–1226. [Google Scholar] [CrossRef]
  15. Olivares, A.; Górriz, J.M.; Ramírez, J.; Olivares, G. Using frequency analysis to improve the precision of human body posture algorithms based on Kalman filters. Comput. Biol. Med. 2016, 72, 229–238. [Google Scholar] [CrossRef]
  16. Seel, T.; Ruppin, S. Eliminating the effect of magnetic disturbances on the inclination estimates of inertial sensors. IFAC PapersOnLine 2017, 50, 8798–8803. [Google Scholar] [CrossRef]
  17. Guo, S.; Wu, J.; Wang, Z.; Qian, J. Novel MARG-sensor orientation estimation algorithm using fast Kalman filter. J. Sens. 2017, 2017, 8542153. [Google Scholar] [CrossRef][Green Version]
  18. Fan, B.; Li, Q.; Liu, T. Improving the accuracy of wearable sensor orientation using a two-step complementary filter with state machine-based adaptive strategy. Meas. Sci. Technol. 2018, 29, 115104. [Google Scholar] [CrossRef]
  19. Del Rosario, M.B.; Khamis, H.; Ngo, P.; Lovell, N.H.; Redmond, S.J. Computationally efficient adaptive error-state Kalman filter for attitude estimation. IEEE Sens. J. 2018, 18, 9332–9342. [Google Scholar] [CrossRef]
  20. Majumder, S.; Deen, M.J. A robust orientation filter for wearable sensing applications. IEEE Sens. J. 2020, 20, 14228–14236. [Google Scholar] [CrossRef]
  21. Esfahani, M.A.; Wang, H.; Wu, K.; Yuan, S. OriNet: Robust 3-D orientation estimation with a single particular IMU. IEEE Robot. Autom. Lett. 2020, 5, 399–406. [Google Scholar] [CrossRef]
  22. Weber, D.; Gühmann, C.; Seel, T. Neural networks versus conventional filters for inertial-sensor-based attitude estimation. arXiv 2020, arXiv:2005.06897. [Google Scholar]
  23. Deibe, Á.; Augusto, J.; Nacimiento, A.; Peña, F.L. A Kalman Filter for nonlinear attitude estimation using time variable matrices and quaternions. Sensors 2020, 20, 6731. [Google Scholar] [CrossRef]
  24. Kalman, R.E. A new approach to linear filtering and prediction problems. J. Fluids Eng. Trans. ASME 1960, 82, 35–45. [Google Scholar] [CrossRef][Green Version]
  25. Bergamini, E.; Ligorio, G.; Summa, A.; Vannozzi, G.; Cappozzo, A.; Sabatini, A.M. Estimating orientation using magnetic and inertial sensors and different sensor fusion approaches: Accuracy assessment in manual and locomotion tasks. Sensors 2014, 14, 18625–18649. [Google Scholar] [CrossRef] [PubMed][Green Version]
  26. Lebel, K.; Boissy, P.; Hamel, M.; Duval, C. Inertial measures of motion for clinical biomechanics: Comparative assessment of accuracy under controlled conditions—Changes in accuracy over time. PLoS ONE 2015, 10, e79945. [Google Scholar] [CrossRef]
  27. Ricci, L.; Taffoni, F.; Formica, D. On the orientation error of IMU: Investigating static and dynamic accuracy targeting human motion. PLoS ONE 2016, 11, e0161940. [Google Scholar] [CrossRef][Green Version]
  28. Ludwig, S.A.; Burnham, K.D. Comparison of Euler Estimate using extended Kalman filter, Madgwick and Mahony on quadcopter flight data. In Proceedings of the 2018 International Conference on Unmanned Aircraft Systems, Dallas, TX, USA, 12–15 June 2018; pp. 1236–1241. [Google Scholar] [CrossRef]
  29. Nazarahari, M.; Rouhani, H. 40 Years of sensor fusion for orientation tracking via magnetic and inertial measurement units: Methods, lessons learned, and future challenges. Inf. Fusion 2020, 68, 67–84. [Google Scholar] [CrossRef]
  30. Picerno, P.; Cereatti, A.; Cappozzo, A. A spot check for assessing static orientation consistency of inertial and magnetic sensing units. Gait Posture 2011, 33, 373–378. [Google Scholar] [CrossRef]
  31. Caruso, M.; Sabatini, A.M.; Knaflitz, M.; Gazzoni, M.; Della Croce, U.; Cereatti, A. Orientation estimation through magneto-inertial sensor fusion: A Heuristic approach for suboptimal parameters tuning. IEEE Sens. J. 2020, 21, 3408–3419. [Google Scholar] [CrossRef]
  32. Cavallo, A.; Cirillo, A.; Cirillo, P.; De Maria, G.; Falco, P.; Natale, C.; Pirozzi, S. Experimental comparison of sensor fusion algorithms for attitude estimation. IFAC Proc. Vol. 2014, 47, 7585–7591. [Google Scholar] [CrossRef][Green Version]
  33. Caruso, M.; Sabatini, A.M.; Knaflitz, M.; Gazzoni, M.; Della Croce, U.; Cereatti, A. Accuracy of the orientation estimate obtained using four sensor fusion filters applied to recordings of magneto-inertial sensors moving at three rotation rates. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Berlin, Germany, 23–27 July 2019; pp. 2053–2058. [Google Scholar] [CrossRef]
  34. Sabatini, A.M. Estimating three-dimensional orientation of human body parts by inertial/magnetic sensing. Sensors 2011, 11, 1489–1525. [Google Scholar] [CrossRef][Green Version]
  35. Roetenberg, D.; Luinge, H.J.; Baten, C.T.M.; Veltink, P.H. Compensation of magnetic disturbances improves inertial and magnetic sensing of human body segment orientation. IEEE Trans. Neural Syst. Rehabil. Eng. 2005, 13, 395–405. [Google Scholar] [CrossRef][Green Version]
  36. Cappozzo, A.; Cappello, A.; Della Croce, U.; Pensalfini, F. Surface-Marker cluster design criteria for 3-D bone movement reconstruction. IEEE Trans. Biomed. Eng. 1997, 44, 1165–1174. [Google Scholar] [CrossRef]
  37. Caruso, M.; Cereatti, A.; Della Croce, U. MIMU_OPTICAL_SASSARI_DATASET. Available online: https://ieee-dataport.org/documents/mimuopticalsassaridataset (accessed on 29 March 2021). [CrossRef]
  38. Chardonnens, J.; Favre, J.; Aminian, K. An effortless procedure to align the local frame of an inertial measurement unit to the local frame of another motion capture system. J. Biomech. 2012, 45, 2297–2300. [Google Scholar] [CrossRef]
  39. Chiari, L.; Della Croce, U.; Leardini, A.; Cappozzo, A. Human movement analysis using stereophotogrammetry. Part 2: Instrumental errors. Gait Posture 2005, 21, 197–211. [Google Scholar] [CrossRef]
  40. Lee, S.; Lee, D.K. What is the proper way to apply the multiple comparison test? Korean J. Anesthesiol. 2018, 71, 353–360. [Google Scholar] [CrossRef][Green Version]
  41. IEEE Electron Devices Society; Microelectromechanical Systems Standards Development Committee; Institute of Electrical and Electronics Engineers; IEEE-SA Standards Board. IEEE Standard for Sensor Performance Parameter Definitions; IEEE: New York, NY, USA, 2014. [Google Scholar]
  42. Lebel, K.; Boissy, P.; Hamel, M.; Duval, C. Inertial measures of motion for clinical biomechanics: Comparative assessment of accuracy under controlled conditions—Effect of velocity. PLoS ONE 2013, e79945. [Google Scholar] [CrossRef] [PubMed]
  43. Zedda, A.; Gusai, E.; Caruso, M.; Bertuletti, S.; Baldazzi, G.; Spanu, S.; Riboni, D.; Pibiri, A.; Monticone, M.; Cereatti, A.; et al. DoMoMEA: A Home-Based telerehabilitation system for stroke patients. In Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, QC, Canada, 20–24 July 2020; pp. 5773–5776. [Google Scholar] [CrossRef]
  44. Ludwig, S.A.; Jiménez, A.R. Optimization of gyroscope and accelerometer/magnetometer portion of basic attitude and heading reference system. In Proceedings of the 2018 IEEE International Symposium on Inertial Sensors and Systems (INERTIAL), Lake Como, Italy, 26–29 March 2018; pp. 1–4. [Google Scholar] [CrossRef]
  45. Cardarelli, S.; Verdini, F.; Mengarelli, A.; Strazza, A.; Di Nardo, F.; Burattini, L.; Fiorettiet, S. Position Estimation of an IMU Placed on Pelvis Through Meta-heuristically Optimised WFLC. In Proceedings of the World Congress on Medical Physics and Biomedical Engineering 2018, Prague, Czech Republic, 3–8 June 2018; pp. 659–664. [Google Scholar]
  46. Tian, Y.; Wei, H.; Tan, J. An Adaptive-Gain complementary filter for real-time human motion tracking with MARG sensors in free-living environments. IEEE Trans. Neural Syst. Rehabil. Eng. 2013, 21, 254–264. [Google Scholar] [CrossRef] [PubMed]
  47. Marins, J.L.; Yun, X.; Bachmann, E.R.; McGhee, R.B.; Zyda, M.J. An extended Kalman filter for quaternion-based orientation estimation using MARG sensors. In Proceedings of the Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180), Maui, HI, USA, 29 October–3 November 2001; pp. 2003–2011. [Google Scholar] [CrossRef]
  48. Yun, X.; Bachmann, E.R. Design, implementation, and experimental results of a quaternion-based Kalman filter for human body motion tracking. IEEE Trans. Robot. 2006, 22, 1216–1227. [Google Scholar] [CrossRef][Green Version]
  49. Sabatini, A.M. Awavelet-Based bootstrap method applied to inertial sensor stochastic error modelling using the allan variance. Meas. Sci. Technol. 2006, 17, 2980–2988. [Google Scholar] [CrossRef]
  50. El-Sheimy, N.; Hou, H.; Niu, X. Analysis and modeling of inertial sensors using allan variance. IEEE Trans. Instrum. Meas. 2008, 57, 140–149. [Google Scholar] [CrossRef]
  51. Hussen, A.A.; Jleta, I.N. Low-Cost inertial sensors modeling using allan variance. Int. Sch. Sci. Res. Innov. 2015, 9, 1069–1074. [Google Scholar]
Figure 1. Board equipped with six magneto-inertial measurement units (MIMUs) (relevant local coordinate system (LCS) in blue) and the eight reflective markers. The three central markers were used to define the stereo-photogrammetric (SP) system LCS (in green). Board axes (in red) are coincident with MIMUs and SP system LCSs. Reprinted with permission from ref. [31]. Copyright 2020 IEEE.
Figure 1. Board equipped with six magneto-inertial measurement units (MIMUs) (relevant local coordinate system (LCS) in blue) and the eight reflective markers. The three central markers were used to define the stereo-photogrammetric (SP) system LCS (in green). Board axes (in red) are coincident with MIMUs and SP system LCSs. Reprinted with permission from ref. [31]. Copyright 2020 IEEE.
Sensors 21 02543 g001
Figure 2. Optimal regions (one for each experimental scenario) for Valenti et al., 2016 (VAK). σacc values are exponentially spaced.
Figure 2. Optimal regions (one for each experimental scenario) for Valenti et al., 2016 (VAK). σacc values are exponentially spaced.
Sensors 21 02543 g002
Figure 3. Rotation rate effect: slow, medium, and fast e o p t distributions (10 SFAs × 3 commercial products). It is possible to assess that errors obtained at the fast rotation rate are worse than those at slow rotation rate and the three distributions statistically differ across all of them.
Figure 3. Rotation rate effect: slow, medium, and fast e o p t distributions (10 SFAs × 3 commercial products). It is possible to assess that errors obtained at the fast rotation rate are worse than those at slow rotation rate and the three distributions statistically differ across all of them.
Sensors 21 02543 g003
Figure 4. Commercial product effect: Xsens, APDM, and Shimmer e o p t distributions (10 SFAs × 3 rotation rates). It is possible to assess that APDM and Shimmer distributions statistically differ from that of Xsens.
Figure 4. Commercial product effect: Xsens, APDM, and Shimmer e o p t distributions (10 SFAs × 3 rotation rates). It is possible to assess that APDM and Shimmer distributions statistically differ from that of Xsens.
Sensors 21 02543 g004
Figure 5. Execution time needed for a single update step by each SFA.
Figure 5. Execution time needed for a single update step by each SFA.
Sensors 21 02543 g005
Table 1. Details of each Sensor Fusion Algorithm (SFA) considered. The # Params column reports the total number of parameters of each SFA. The p1 and p2 report the description of the parameter tuned to detect the optimal values. a.u. = arbitrary units.
Table 1. Details of each Sensor Fusion Algorithm (SFA) considered. The # Params column reports the total number of parameters of each SFA. The p1 and p2 report the description of the parameter tuned to detect the optimal values. a.u. = arbitrary units.
CF# Params p 1 Default p 2 Default
MAH2kp—inverse gyroscope weight1rad/ski—weight for online bias estimation0.3rad/s
MAD1β—inverse gyroscope weight0.1rad/s//
VAC9gmag—magnetometer weight0.01a.u.ath2—threshold for accelerometer vector selection0.2a.u.
SEL4τacc—accelerometer time constant1sτmag—magnetometer time constant3s
MCF2gmag—magnetometer weight0.01a.u.//
KF# Params p 1 Default p 2 Default
SAB6σgyr—inverse gyroscope weight0.007rad/sath—threshold for accelerometer vector selection40mg
LIG6σgyr—inverse gyroscope weight1rad/scb—Gauss-Markov parameter of the prediction model to set the variance of external acceleration and ferromagnetic disturbances1a.u.
VAK3σgyr—inverse gyroscope weight0.004rad/sσacc—inverse accelerometer weight0.014 m/s2
GUO3σgyr—inverse gyroscope weight0.001rad/s//
MKF8σ2gyr—inverse gyroscope weight9.14 × 10−5 (rad/s)2//
Table 2. Statistical analysis plan to evaluate the influence of SFAs, rotation rate, and commercial product on the errors.
Table 2. Statistical analysis plan to evaluate the influence of SFAs, rotation rate, and commercial product on the errors.
Influencing FactorNumber of DistributionsNumber of Values for Each Distribution
SFA10 (one for each SFA)9 (=3 rotation rates × 3 commercial products)
Rotation rate3 (one for each rotation rate)30 (=10 SFAs × 3 commercial products)
Commercial product3 (one for each commercial product)30 (=10 SFAs × 3 rotation rates)
Table 3. The Optimal Errors Are Reported with the Absolute Errors Obtained Using the Default Parameter Values.
Table 3. The Optimal Errors Are Reported with the Absolute Errors Obtained Using the Default Parameter Values.
CF e o p t e d e f KF e o p t e d e f
XsensSlowMAH2.54.2SAB2.267.9
Medium2.411.92.196.6
Fast4.013.02.453.9
APDMSlow3.83.95.077.5
Medium4.817.75.762.6
Fast8.212.38.39.9
ShimmerSlow3.45.94.571.1
Medium4.638.24.914.5
Fast7.617.08.530.0
XsensSlowMAD2.74.7LIG1.93.7
Medium2.55.22.03.9
Fast4.06.82.94.8
APDMSlow3.84.13.63.6
Medium4.64.64.95.0
Fast8.18.24.64.6
ShimmerSlow3.94.34.44.4
Medium4.95.24.04.2
Fast8.88.96.36.5
XsensSlowVAC4.04.1VAK1.222.3
Medium5.05.91.621.4
Fast7.210.02.572.8
APDMSlow3.53.63.629.6
Medium6.111.86.030.4
Fast8.315.19.281.9
ShimmerSlow3.83.84.032.6
Medium10.219.24.448.8
Fast11.523.68.2100.1
XsensSlowSEL3.14.0GUO2.33.7
Medium2.54.62.34.9
Fast5.16.75.710.6
APDMSlow3.73.84.24.5
Medium7.17.35.15.3
Fast8.08.89.412.0
ShimmerSlow3.43.54.04.0
Medium5.08.45.15.7
Fast9.411.813.716.7
XsensSlowMCF3.34.5MKF4.24.9
Medium6.16.24.88.7
Fast6.67.86.710.9
APDMSlow3.84.23.64.8
Medium12.312.35.314.3
Fast7.99.37.210.7
ShimmerSlow5.05.23.95.8
Medium10.010.18.445.2
Fast8.612.09.919.0
All the unit are in degrees.
Table 4. The Mean ± STD of the Errors for Each SFA for Optimal Parameter Values Selection.
Table 4. The Mean ± STD of the Errors for Each SFA for Optimal Parameter Values Selection.
LIGVAKMAHMADSABSELGUOMKFVACMCF
e o p t 3.8 ± 1.44.5 ± 2.84.6 ± 2.14.8 ± 2.24.8 ± 2.45.3 ± 2.45.8 ± 3.76.0 ± 2.26.6 ± 2.97.1 ± 2.9
All units are in degrees.
Table 5. Mean ± STD Errors for Each Rotation Rate Scenario.
Table 5. Mean ± STD Errors for Each Rotation Rate Scenario.
(deg)SlowMediumFast
e o p t 3.5 ± 0.95.2 ± 2.57.3 ± 2.6
Table 6. Results of Friedman’s Test with Bonferroni’s Correction to Investigate the Differences among the Three Rotation Rate Conditions.
Table 6. Results of Friedman’s Test with Bonferroni’s Correction to Investigate the Differences among the Three Rotation Rate Conditions.
ScenarioOptimal Conditions
Slow vs. fastSignificantly different (p < 1× 10−4)
Slow vs. mediumSignificantly different (p < 1× 10−3)
Fast vs. mediumSignificantly different (p = 0.013)
Table 7. Mean ± STD Errors for Each Commercial Product Scenario.
Table 7. Mean ± STD Errors for Each Commercial Product Scenario.
(deg)Xsens-MTxAPDM-OpalShimmer-Shimmer 3
e o p t 3.5 ± 1.76.0 ± 2.36.5 ± 2.8
Table 8. Results of Friedman’s Test with Bonferroni’s Correction to Investigate the Differences among the Three Commercial Product Conditions.
Table 8. Results of Friedman’s Test with Bonferroni’s Correction to Investigate the Differences among the Three Commercial Product Conditions.
ScenarioOptimal Conditions
Xsens vs. APDMSignificantly different (p < 1× 10−5)
Xsens vs. ShimmerSignificantly different (p < 1× 10−6)
APDM vs. ShimmerNot significantly different (p = 1)
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Caruso, M.; Sabatini, A.M.; Laidig, D.; Seel, T.; Knaflitz, M.; Della Croce, U.; Cereatti, A. Analysis of the Accuracy of Ten Algorithms for Orientation Estimation Using Inertial and Magnetic Sensing under Optimal Conditions: One Size Does Not Fit All. Sensors 2021, 21, 2543. https://doi.org/10.3390/s21072543

AMA Style

Caruso M, Sabatini AM, Laidig D, Seel T, Knaflitz M, Della Croce U, Cereatti A. Analysis of the Accuracy of Ten Algorithms for Orientation Estimation Using Inertial and Magnetic Sensing under Optimal Conditions: One Size Does Not Fit All. Sensors. 2021; 21(7):2543. https://doi.org/10.3390/s21072543

Chicago/Turabian Style

Caruso, Marco, Angelo Maria Sabatini, Daniel Laidig, Thomas Seel, Marco Knaflitz, Ugo Della Croce, and Andrea Cereatti. 2021. "Analysis of the Accuracy of Ten Algorithms for Orientation Estimation Using Inertial and Magnetic Sensing under Optimal Conditions: One Size Does Not Fit All" Sensors 21, no. 7: 2543. https://doi.org/10.3390/s21072543

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop