^{1}

^{*}

^{2}

^{2}

^{1}

^{1}

This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

Ellipsoid fitting algorithms are widely used to calibrate Magnetic Angular Rate and Gravity (MARG) sensors. These algorithms are based on the minimization of an error function that optimizes the parameters of a mathematical sensor model that is subsequently applied to calibrate the raw data. The convergence of this kind of algorithms to a correct solution is very sensitive to input data. Input calibration datasets must be properly distributed in space so data can be accurately fitted to the theoretical ellipsoid model. Gathering a well distributed set is not an easy task as it is difficult for the operator carrying out the maneuvers to keep a visual record of all the positions that have already been covered, as well as the remaining ones. It would be then desirable to have a system that gives feedback to the operator when the dataset is ready, or to enable the calibration process in auto-calibrated systems. In this work, we propose two different algorithms that analyze the goodness of the distributions by computing four different indicators. The first approach is based on a thresholding algorithm that uses only one indicator as its input and the second one is based on a Fuzzy Logic System (FLS) that estimates the calibration error for a given calibration set using a weighted combination of two indicators. Very accurate classification between valid and invalid datasets is achieved with average Area Under Curve (AUC) of up to 0.98.

The use of Microelectromechanical (MEMS) Magnetic, Angular Rate and Gravity (MARG) sensors has exponentially increased during recent years, leading to an estimated $3, 500M business in 2013 and a predicted $9, 000M business by 2017 [

MEMS MARG sensors need to be calibrated in order to transform measured raw data into meaningful physical units and/or to compensate for a series of undesired effects. Calibration is, thus, a very important step to improve the accuracy of the measurements and, consequently, to increase the quality and efficiency of the system in which the sensors are embedded. A large amount of works regarding accelerometer and magnetometer calibration algorithms has been published over the last decade. Some representative examples can be found in [

Most calibration algorithms are based on 3-D ellipsoid fitting methods, which minimize a cost function to estimate the value of a set of calibration parameters that define the sensors model [

Covering all the positions to define a perfectly populated ellipsoid is not a straightforward task as it is complicated to keep a visual record of all the positions that have already been swept. As a consequence, it is usual to gather sets of points that are ill-distributed and lack important parts of the ellipsoid.

On the other hand, some navigation systems recalibrate their sensors periodically. If the vehicle in which the navigation system is installed has not covered enough positions relative to a reference frame, then the data that are continuously being gathered can lead to an erroneous calibration. Lotters

Ill-distributed calibration sets result in a less defined ellipsoid which, in turn, results in a suboptimal set of calibration parameters estimated by the ellipsoid fitting algorithm [

Camps

This is a simple and valid solution if there is a base station to which the sensor sends the calibration data in real time. However, what happens if we want to gather in-field calibration data and we do not have any equipment to act as a base station? What happens if IMU, or the development board containing the sensors, stores the data locally and does not have the ability to transfer data in real time? In these two cases, the feedback visualization tool would be of no help.

A possible solution is to develop an automatic algorithm that is able to determine the validity of the calibration dataset by analyzing the spatial distribution of the data while it is being gathered. To do so, we will need to identify a series of indicators that can link the spatial distribution of input calibration data to the resultant calibration error that would be obtained if such a dataset was provided to the calibration algorithm. The calibration error can be defined in many ways, e.g., the error of the estimated calibration parameters with respect to the optimal ones, or the error of the norm of the calibrated measured physical magnitude with respect to the norm of a known reference. In this work, we opted to use the latter definition as it will be further detailed in the experiments section.

The main objectives of this work are listed below:

Find a series of indicators that contain information about the spatial distribution of the input calibration data of MARG sensors.

Develop an automatic system that uses one or a combination of various indicators to validate the calibration dataset and ensure a proper calibration process.

The paper is structured as follows. Section 2 includes the basics of ellipsoid-fitting calibration algorithms as well as models used to describe the output of MARG sensors. Section 3 presents the selection of indicators and the proposal of two algorithms, one using a single input and based on a simple threshold comparison, and another one using a multiple input and based on a fuzzy system. Section 4 includes all the experiments that were carried out to test the proposed approaches and a selection of the most significant results. Section 5 contains a discussion of the obtained results and the validity of the developed system. Finally, conclusions are drawn in Section 6.

Ellipsoid fitting algorithms are a recurring solution [_{i}_{i}_{i}_{i}

This model can be complicated by adding other parameters that model undesired effects such as the orthogonality error between sensor axes,

The following model proposed by [_{k} transforms the vector expressed in the orthogonal sensor reference frame _{k}_{c}_{c}_{c}_{p}_{p}_{p}

_{k}, _{k} and _{k} are the rotation angles around X', Y' and Z' axes respectively.

Now, knowing that the norm of the actual measurement u is constant under stable magnetic conditions (in the case of the magnetometer), quasi-static conditions (in the case of the accelerometer), constant turning rate (in the case of the gyroscope), we obtain,
_{x}_{y}_{z}

The goal of ellipsoid-fitting algorithms is to estimate the set of parameters defined in the sensor model in order to compute the value of the calibrated measurement in _{x}_{y}_{z}_{x}_{y}_{z}

Magnetometer:

Accelerometer:

Gyroscope:

The minimization process can be carried out using linear or non-linear optimization algorithms. Another way to estimate the parameters in

Now that we have briefly revised the basic principles of MARG sensor modeling and ellipsoid-fitting calibration algorithms, we can proceed to present the core of our work.

As aforementioned, the main goal of our work is to develop an algorithm that links the spatial distribution of input calibration data to the error obtained after the calibration process. Therefore, we need to analyze the calibration input data to obtain a reliable verdict of the suitability of their spatial distribution. To do so, we need to select a set of indicators that contain relevant information about the spatial distribution of data. We have selected four parameters that we originally thought would provide useful information. The chosen indicators are:

Indicator A: Sum of the difference of the third quartile (Q3) and the first quartile (Q1) of spherical coordinates

Indicator B: Sum of the number of empty bins in the histograms of spherical coordinates _{θi}_{ϕi}_{θ}_{ϕ}

Indicator C: Sum of the Euclidean distance between the maximum and minimum values of each of the gathered data components, _{k}

Indicator D: Volume of the convex hull defined by the dataset. The convex hull of a set of G points in the Euclidean space is defined as the smallest convex set that contains G. Therefore, we will be computing the volume contained in the smallest quadric surface defined by the points in the dataset. Indicator D is defined as,

The first approach that comes to mind is to individually use each one of the selected indicators to check their relation to the calibration error. This approach consists of computing the indicator that is best linked to the calibration error and apply a thresholding algorithm. The thresholding algorithm compares the value of the selected indicator with an empirically predefined threshold and will only validate the calibration dataset if it exceeds it (in the case of indicators A, C and D) or it is under it (indicator B).

_{i}_{i}

A series of experiments are needed to check the strength of the relation of each one of the proposed indicators with the calibration error and their suitability as validators of the goodness of the calibration dataset. In addition, such experiments will also reveal the optimal value of the predefined thresholds. Experiments carried out with this purpose are later explained in Section 4.

In general, indicators extracted from each set of calibration points are not directly related to the error value,

Rules like

More precisely, for the calibration set ^{k}^{k}^{k}_{m}_{1}_{2}…_{M}

Defining such a fuzzy system would require the following: choosing the number of inputs (M); choosing the number of Membership Functions (MFs) as well as their type and parameters; and selecting the scalar consequents for each rule. Although it is possible to perform all this design manually choosing the value of each parameter, it would be very time expensive, suboptimal and strongly influenced by the designer's subjective criterion. Thus, to avoid these issues, an automatic approach has been selected in this work in order to create an adaptive and nearly optimum FLS.

First of all, we have decided to use triangular MFs comprising a complete partition in the input space [

The remaining design decisions are those related to the MFs parameters, _{train}, defined in the following way,
_{calib} is the actual calibration error that we wish to estimate, which is known during the training process, and

In general terms, the adaptive strategy adopted in this work is the same as in [

More precisely, the employed error function is as follows,
_{m}

If we consider the quantity ^{k}^{k}

Considering that,

However, in this work, the approximated signal is the same as the signal used to compute the error, so we used C = 1.

In the case of the MFs' centers, a similar correction strategy is adopted, this time based on a modified gradient descent algorithm,

This algorithm prevents the centers from changing their order, which in turn prevents the loss of their linguistic meaning. This updating method uses a saturation function based on the gradient magnitude, so the centers can move, at most half the distance from their immediate neighbors.

Finally, it is important to highlight three important issues: first, because of the cumbersomeness of the maneuvers to gather each set of calibration points, the total number of input-output pairs available is not very high (around 50). Such a limited number of points would lead to a very poor fuzzy training. Thus, to avoid this from happening, these 50 points have been used up to 10 several times to provide the fuzzy system with at least 500 hundred points to learn the fuzzy relation. Despite re-usage of these points will not add new information, an artificially larger dataset provides the algorithm with more iterations to improve its convergence.

The second issue to be highlighted is that this training phase would likely need to be computed offline (depending on the power of the processor embedded in the MIMU), presumably in a computer. However, the evaluation function found after the training process can be easily included in the embedded firmware and executed online as it is not computationally expensive.

The third issue to highlight is that although it would have been possible to use optimal consequents adaptation (as in [

We have carried out two sets of experiments, one for each of the proposed approaches. We will start by describing the datasets gathered to train and test the validators. We have gathered two groups of calibration distributions; the first one is composed of 27 data distributions and we will refer to it as “dataset 1”; the second group is composed of 55 distributions and we will name it “dataset 2”. Both datasets contain ill-distributed and well distributed input calibration data. _{i}

Once the data to be used are ready, we apply the following pre-processing steps for each one of the calibration distributions:

Compute all four indicators (A, B, C and D).

Execute the ellipsoid-fitting calibration algorithm presented in [

Apply the computed calibration parameters to calibrate the input calibration data.

Compute the norm of the calibrated data and calculate the error with respect to the value of the norm of Earth's magnetic field in our location (0.432 Gauss [

Build a vector containing the calibration errors for each one of the distributions of both datasets.

Select a maximum error tolerance (_{max}

Build a validation binary marker associated to both datasets, which is set to 1 if the distribution is valid and to 0 if not.

Now, once all the auxiliary parameters are ready, the experiments carried out to test the single-indicator thresholding validator are very straightforward. As we explained in Section 3, we compare the value of each one of the indicators with a threshold. Imagine that indicator C has a value of 500 empty bins for distribution

The value of the predefined threshold has to be set empirically and, in order to find the value resulting in the optimal ROC, we have carried out a grid search. This can be considered as the training process and it has the following steps:

Build a vector containing the initial threshold, the step and the final threshold values for each one of the four indicators.

Apply an iterative process in which every value of the threshold is applied to the indicator in order to build and estimated validation marker and compute the ROC values.

Find the threshold values resulting in the best ROC values

The selection of the values of the threshold vectors depend on the characteristics of the employed sensor. In our case, we employed the triaxial magnetometer included in a WAGYROMAG IMU [

Indicator A ((_{θ}− Q_{θ}_{ϕ}_{ϕ}

Indicator B (Total number of empty bins in

Indicator C (Sum of the Euclidean distance between the maximum and minimum gathered components): Ideally, the radius of the sphere will be in the [0, 512] range. Therefore, the sum of the three ellipsoid axes will be in [0, 3072] range and so will be the threshold vector.

Indicator D (Volume of the convex hull): given that the volume of a reference well distributed and highly populated raw calibration set is around 5 × 10^{8}, while the volume of a poorly distributed set is in the order of 10^{7}, then the threshold vector will be set to [1 × 10^{6}, 6 × 10^{8}].

The selection of a grid-search parameter optimization strategy over any other error minimization algorithm is well justified in this case since the number of possible values the parameters can take is known and limited. Moreover, the grid-search ensures finding the optimal parameters and presents no problems of convergence to local minima as the Gauss-Newton and Levenberg-Mardquart algorithms.

The maximum tolerated calibration error _{max}_{max}_{1} = 0.0273 Gauss and _{max}_{2} = 0.0262 Gauss for datasets 1 and 2 respectively.

After the training process, we need to check the validity of the computed parameters using dataset 1 by applying them over dataset 2 and viceversa.

The second part of the experiments is conducted to test the performance of the Multiple-indicator Fuzzy Validator.

We trained the FLS with all the possible input combinations of two indicators and then tested it over the dataset that was not used for the training. The output of the FLS is an estimation of the calibration error values that would be obtained for each one of the input calibration distributions. The estimated calibration error also needs to be compared with a predefined error tolerance, which should ideally be equal to _{max}

_{max}

When analyzing the results obtained by the Single-indicator Thresholding Validator, we see that for dataset 1 it was possible to find a threshold for all indicators, which ensured perfect classification,

Indicator A: It shows the poorest results of all indicators. The sum of the difference between Q3 and Q1 does not consider the possible outliers (extreme values, see

Indicator B: Counting the number of empty bins in the histograms offers an acceptable precision but it can be an ambiguous indicator. Having, for example, two distributions containing a similar number of points, the number of empty bins can be very similar or even identical. However, if one of the distributions covers the complete surface of the ellipsoid with a low density of points, while the other one covers only a limited region of the surface with a high density of points, the calibration error will be very different. The less populated distribution will achieve a lower error since it gives better information to the fitting algorithm about the shape of the ellipsoid.

This indicator is only trustable when we set a very strict threshold, i.e., when the number of permitted empty bins is very low (∼10% out of the total). By setting a very strict threshold, we can guarantee that the calibration process will yield good results. However, this restriction eliminates valid distributions that have a much lower number of points but offer good performance. In other words, a restrictive threshold causes no false positives (ill-distributed calibration sets being validated) but it may generate a large number of false negatives (well-distributed calibration sets being discarded). If, on the other hand, we set a less restrictive threshold, the number of false negatives will decrease but the number of false positives will increase (which is a less desirable situation).

Indicator C: The sum of the Euclidean distance between the maximum and minimum values gathered in each axis of the sensor has shown to be the second best indicator among the tested ones. This indicator shows if axis antipodal areas are covered. Then, if it has a high value, it indicates that we have at least covered the three ellipses around the three axes. However, this indicator does not provide further information about the number and density of points in other areas located outside the ellipsoid main axes. The fitting calibration algorithm may have trouble converging to optimal parameters if only points in the axes are provided. In an extreme case in which we only have points contained in the three ellipses around the axes, the indicator will present a high value and tag the distribution as valid while it would actually result in the estimation of erroneous calibration parameters. However, since this is a very extreme and rare case, we can conclude that indicator C is a good indicator of the validity of the input calibration set.

Indicator D: The volume contained by the smallest surface wrapping the data points (the convex hull) has revealed to be the best of the tested indicators as it contains trustful information about the proper distribution of the calibration sets. It achieved perfect classification for both datasets in the training process (

_{well} = 3113, _{ill} = 2954, _{well} = 4.9196 × 10^{8} and D_{iill} = 3.9166 × 10^{8}. Notice how the difference between _{well} and _{ill} is more accentuated as the difference between _{well} and _{ill}, hence, the better discrimination ability of indicator D.

The Multiple-indicator fuzzy approach was developed to test the performance of a decision system using combinations of weighted indicators as input. The FLS showed its power to obtain an estimation of the calibration error using only the indicators as the input during the training phase. The success of the testing phase varied depending on the employed combination of indicators.

The combination of indicators A and B achieved the poorest performance with an average AUC of 0.8963 ± 0.0105 (

The FLS approach is also easier to tune, as the output is an estimate of the calibration error, so the threshold can be directly set according to the desired degree of calibration error tolerance. On the other hand, it is harder to regulate (manually set) the value of the indicators' threshold when using the Single-indicator approach since the relationship between the value of the indicator and the calibration error is not functional and therefore, not evident.

In this work, we have presented the problems derived from the use of ill-distributed datasets as the input of ellipsoid fitting algorithms used to calibrate MARG sensors. We have presented two different systems that monitor and assess the goodness of the spatial distribution of data being gathered during the calibration maneuvers and, therefore, validate them only when a low calibration error is ensured.

Both approaches are based on the computation and analysis of four distribution indicators, namely the sum of the difference of the third quartile and the first quartile of spherical coordinates

The first approach, referred to as the “Single-indicator Thresholding Approach” is based on comparing the value of a single indicator with a predefined threshold from which it is considered that the error will be low. We carried out a grid-search training process that revealed the optimal value of the threshold of each indicator. For indicator D, we found a threshold value that ensured perfect classification,

The second approach was designed to use weighted combinations of two indicators as the input. We designed a TSK Fuzzy Logic System with 10 membership functions that was used to infer the relationship between the indicators and the calibration error.

Results showed that the estimation of the error was very accurate and an average AUC of 0.9861 ± 0.0197 was obtained using parameters C and D as the input.

Results have been obtained for a triaxial magnetometer since the data gathering process is faster than for the accelerometer. Accelerometers need to be placed in quasi-static positions to avoid the linear acceleration from distorting the gravity, which is used as the reference. This fact considerably increases the duration of the calibration maneuvers to obtain the same number of points as with the magnetometer. Results are generalizable since when using ellipsoid fitting algorithms the only difference comes from the physical magnitude used as a reference (norm of Earth's magnetic field, norm of Earth's gravitational field or a constant known angular rate).

It has been shown that the calibration error of ellipsoid fitting algorithms can be predicted by analyzing the spatial distribution of the input datasets. The system we propose could be used to improve the performance of automatic in-use calibration algorithms that periodically recalibrate MARG sensors, as well as to ease the calibration maneuvers carried out by in-field operators. In addition, in case that the IMU being calibrated had local processing capabilities, our algorithm could be included in the firmware and send an alert to the operator (through an LED, playing a sound or with any other kind of actuator) whenever the calibration dataset is well distributed, to inform the operator that no further maneuvers are required.

Future work will be oriented to implement an algorithm to detect and reject outliers in data distributions caused by magnetic distortions to improve the performance of our system when applied to magnetometers. Measurements of magnetometers can be disrupted by external magnetic fields other than the Earth's. If the MIMU is used in a very unstable magnetic environment, the magnetometer's output will be distorted and the indicators may get erroneous values, which may fool the detection system. We will also evaluate other alternatives to the FLS such as neural networks and multifactor regression to reduce the computational load of the training process.

This work was partly supported by the MICINN under the TEC2012-34306 project, the MEEC under the TIN2012-32039 (hpMooN) and the Consejería de Innovación, Ciencia y Empresa (Junta de Andalucía, Spain) under the Excellence Projects P09-TIC-4530 and P11-TIC-7103.

The authors declare no conflict of interest.

Center N.G.D. NGDC Geomagnetic Calculators. NOAA's National Geophysical Data Center and the collocated World Data Centers, Boulder, operated by NOAA/NESDIS/NGDC, archive and make available geomagnetic data and information relating to Earth's magnetic field and Earth-Sun environment, including current declination, geomagnetic field models and magnetic indices, geomagnetic observatory data, and geomagnetic surveys.

Gathered input magnetometer calibration datasets: well distributed (

Representation of a non-orthogonal sensor triplet (

Boxplot of gathered magnetic field components (_{x}_{y}_{z}

Dataset 1: Indicators

Dataset 2: Indicators

FLS training using dataset 1 (

Well distributed dataset: Euclidean distances (

Results of the training process of the Single-indicator Thresholding Validator.

_{max} |
|||||||
---|---|---|---|---|---|---|---|

1 | A | 0.0273 | 59.0000 | 1.0000 | 1.0000 | 1.0000 | 0.9648 ± 0.0497 |

2 | A | 0.0262 | 109.0000 | 0.9231 | 0.9310 | 0.9297 | |

1 | B | 0.0273 | 1.3530E + 3 | 1.0000 | 1.0000 | 1.0000 | 0.9947 ± 0.0075 |

2 | B | 0.0262 | 1.3040E + 3 | 0.9615 | 1.0000 | 0.9894 | |

1 | C | 0.0273 | 2.3810E + 3 | 1.0000 | 1.0000 | 1.0000 | 0.9987 ± 0.0019 |

2 | C | 0.0262 | 2.2440E + 3 | 0.9615 | 1.0000 | 0.9973 | |

1 | D | 0.0273 | 1.1030 + E8 | 1.0000 | 1.0000 | 1.0000 | 1.0000 ± 0.0000 |

2 | D | 0.0262 | 9.8000 + E7 | 1.0000 | 1.0000 | 1.0000 |

Results of the test procedure of the Single-indicator Thresholding Validator.

1 | A | 59.0000 | 1.0000 | 0.7241 | 0.8620 ± 0.1951 |

2 | A | 109.0000 | 0.6667 | 1.0000 | 0.8333 ± 0.2357 |

1 | B | 1.3530E + 03 | 0.9615 | 0.8966 | 0.9290 ± 0.0459 |

2 | B | 1.3040E + 03 | 0.8333 | 1.000 | 0.9166 ± 0.1179 |

1 | C | 2.3810E + 03 | 0.9231 | 1.0000 | 0.9616 ± 0.0544 |

2 | C | 1.2440E + 03 | 1.0000 | 0.7778 | 0.8889 ± 0.1571 |

1 | D | 1.1030E + 08 | 1.0000 | 1.0000 | 1.0000 ± 0.0000 |

2 | D | 9.8000E + 07 | 1.0000 | 0.7778 | 0.8889 ± 0.1571 |

Results of the test procedure of the Multiple-indicator Fuzzy Validator. Datasets 1 and 2.

_{max} |
|||||||
---|---|---|---|---|---|---|---|

1 | A&B | 0.0262 | 0.0250 | 1.0000 | 0.8276 | 0.9038 | 0.8963 ± 0.0105 |

2 | A&B | 0.0273 | 0.0220 | 0.7778 | 1.0000 | 0.8889 | |

1 | A&C | 0.0262 | 0.0230 | 0.9615 | 0.8966 | 0.8893 | 0.9168 ± 0.0390 |

2 | A&C | 0.0273 | 0.0230 | 0.8889 | 1.0000 | 0.9444 | |

1 | A&D | 0.0262 | 0.0310 | 0.9615 | 0.9310 | 0.9198 | 0.9460 ± 0.0371 |

2 | A&D | 0.0273 | 0.0250 | 0.9444 | 1.0000 | 0.9722 | |

1 | B&C | 0.0262 | 0.0360 | 0.9615 | 0.8621 | 0.8548 | 0.9274 ± 0.1027 |

2 | B&C | 0.0273 | 0.0490 | 1.0000 | 1.0000 | 1.0000 | |

1 | B&D | 0.0262 | 0.0260 | 0.9615 | 0.7931 | 0.7858 | 0.8790 ± 0.1318 |

2 | B&D | 0.0273 | 0.0240 | 0.9444 | 1.0000 | 0.9722 | |

1 | C&D | 0.0262 | 0.0250 | 1.0000 | 1.0000 | 1.0000 | 0.9861 ± 0.0197 |

2 | C&D | 0.0273 | 0.0220 | 0.9444 | 1.0000 | 0.9722 |