An Evaluation of MEMS-IMU Performance on the Absolute Trajectory Error of Visual-Inertial Navigation System

Nowadays, accurate and robust localization is preliminary for achieving a high autonomy for robots and emerging applications. More and more, sensors are fused to guarantee these requirements. A lot of related work has been developed, such as visual-inertial odometry (VIO). In this research, benefiting from the complementary sensing capabilities of IMU and cameras, many problems have been solved. However, few of them pay attention to the impact of different performance IMU on the accuracy of sensor fusion. When faced with actual scenarios, especially in the case of massive hardware deployment, there is the question of how to choose an IMU appropriately? In this paper, we chose six representative IMUs with different performances from consumer-grade to tactical grade for exploring. According to the final performance of VIO based on different IMUs in different scenarios, we analyzed the absolute trajectory error of Visual-Inertial Systems (VINS_Fusion). The assistance of IMU can improve the accuracy of multi-sensor fusion, but the improvement of fusion accuracy with different grade MEMS-IMU is not very significant in the eight experimental scenarios; the consumer-grade IMU can also have an excellent result. In addition, the IMU with low noise is more versatile and stable in various scenarios. The results build the route for the development of Inertial Navigation System (INS) fusion with visual odometry and at the same time, provide a guideline for the selection of IMU.


Introduction
In recent years, with the rapid growth in the fields of autonomous driving [1], augmented reality [2], virtual reality [3], and other emerging applications, the question of how to accurately obtain their localization information has become a crucial premise and foundation. To fulfill the requirements of these applications, a lot of exploration and novel work has been carried out by researchers, such as the work which fusions WiFi and IMU with floorplan [4]. Among many localization and navigation methods, inertial navigation is one of the mainstream methods at present. Inertial Measurement Unit (IMU) is the prerequisite of inertial navigation, as it plays a considerable role in indoor, urban high buildings, planetary exploration, and other GPS denial scenes. In terms of the indoor positioning technologies which heavily rely on IMU and other modal sensors, there are two major categories: building independent and dependent. Building independent draws support from image-based technologies and dead reckoning [5]. Building-dependent localization is realized by multi-modal sensors, such as Wi-Fi, Bluetooth, Ultra-Wide Band, Visible Light Communication, etc. With the development of the diversity of positioning technology, the hybrid patterns of indoor positioning based on smartphone cameras and IMU are also emerging [6]. IMU is significant for positioning and orientation applications. IMU is a device composed of a triaxial accelerometer that senses the linear acceleration coupled with   The rest of the paper is structured as follows: In Section 2, the related work abou evaluation and analysis experiments and IMU selection are discussed. In Section 3, th framework flow of the overall experiment which is divided into four parts: hardware pla forms (A), sensor setup (B), sensors parameter configuration (C), and evaluation (D) i   The rest of the paper is structured as follows: In Section 2, the related work abou evaluation and analysis experiments and IMU selection are discussed. In Section 3, th framework flow of the overall experiment which is divided into four parts: hardware plat

Related Work
In order to make the positioning technologies more suitable for deployment and wide use, a lot of related work has been proposed. In the subsection Evaluation and Analysis Work, they paid more attention to the performance of many open-sourced algorithms on CPU/GPU. In subsection Multi-IMUs Work, the work on multiple IMUs is summarized, but their IMU type is single and the grade span is not wide. They mainly use redundant sensor groups to enhance the stability of the system. In addition, in partial work, MEMS-IMU was evaluated for user convenience.

Evaluation and Analysis Work
Six monocular visual-inertial algorithms were compared on several computing-constrained, single-board computers, and the accuracy of each algorithm, the time consumed per frame, and the utilization of CPU and memory were evaluated [22]. Similarly, Jinwoo et al. [23] compared the performance of nine visual(-inertial) odometry algorithms on different hardware platforms (Jetson TX2, Xavier NX, and AGX Xavier), evaluated the CPU utilization, and corresponding pose accuracy of each algorithm on each platform. Giubilato et al. [24] evaluated the performance of the visual odometry algorithm on Jetson TX2 and revealed the robustness, CPU/GPU utilization, frame rate, and so on. Alwin et al. [25] comprehensively evaluated and analyzed the performance of five sensor fusion algorithms of heading estimation using smartphone sensors, such as LKF, EKF, UKF, PF, and CF. These works are important for the selection of embedded platforms, fusion performance, and software deployment. However, they are more aimed at comparing the computing hardware and algorithms without considering the performance of the algorithms with different IMU. The impact of IMU with different performances on the accuracy of VIO is still blank.

Multi-IMUs Work
Three different IMU sensors were considered in [26,27], but they were mainly used to improve the robustness of the state estimation. Kevin et al. [27] utilized the information from multiple inertial measurement units in order to resile when one of them has sensor failures. Without considering the impact of different levels of IMU on the system, the authors in [28] exploited four IMUs and magnetometers with different positions to calculate the angular velocity and velocity information. However, there is no more consideration for the performance of IMU. In [29], Chao conducted a comparative investigation and evaluation on some low-cost IMUs, but focused on the sensor packages and available software solutions, and listed their specifications. The authors in [30] evaluated the accuracy of the IMU in the two smartphones and the inertial sensor in XIMU at a yaw angle. The IMU in the smartphone is MPU6500 and the IMU module is produced by BOSCH. XIMU refers to the combination of IMU and AHRS. The specific model is not indicated. The two IMUs used for comparison in this work are consumer IMUs, and the IMU with different performance is not considered. In [31], the authors evaluated four orientation algorithms (Madgwick, Mahony, EKF, TKF-Q) in the simulation environment. Nevertheless, this work was carried out in the simulation environment, considering the noise and bias instability, the veritable IMU will be affected by many factors, which cannot be represented in the simulation environment, and these IMUs were not evaluated in the actual environment. In [32], the authors carried out the evaluation of ZUPT, ZARU, and HDR algorithms which focus on heading drift reduction in pedestrian indoor navigation with consumption level IMU. In this work, the author focused on the error of the algorithm and did not analyze the impact of different performance IMUs. In [33], the authors evaluated seven commercial IMUs for persistent healthcare-related use cases. Through the analysis of the operating environment, cost performance, battery life, memory size, and other specifications, the selection criteria of IMU are obtained. This work considers more the convenience of these modules in use, and the performance span of different IMUs is relatively small. They are all assessed with consumer IMUs. Compared with the above-related work, they did not pay attention to the grade, diversity of MEMS-IMU, and the impact of IMU on VIO fusion algorithms. In this paper, we have chosen six representative IMUs with different performances from consumer-grade to tactical grade for exploration and analysis.

Experiments and Methods
The specific framework of the experiment is shown in Figure 2. The hardware platform was built as shown in Figure 2 first, including the mobile platform and multi-IMU camera suite. Then, the sensors parameter configuration was conducted, including Allan's analysis of variance, temporal-spatial calibration between IMU and camera, etc. Next, the algorithm was executed in the specified scene by controlling the hardware platform while recording the RTK (Real-Time Kinematic) data as the ground truth at the same time. The localization accuracy of RTK is ±(8 + 1 × 10 −6 D) mm.
In order to explore the influence of IMU on sensor fusion accuracy, five kinds of motion states were designed: slow, normal, fast, varying velocity, and spin move. In these scenes, IMU has different levels of excitation to identify the IMU's ability. For example, in uniform velocity scenes, IMU's incentive is small. However, IMU will be more motivated in varying velocity and spin move scenes. At the same time, experiments in the strong light environment with different motion states and weak texture special scenes which to analyze the auxiliary effect of IMU were also carried out. In addition, the performance of the fusion system with different IMUs under long-term running conditions that last for 30 min was also explored. The specific scenes are shown in Table 1. The tightlycoupled algorithm about Visual-Inertial Systems (VINS_Fusion [10,11]) was executed as the evaluation framework. Firstly, feature detection [34] and tracking were carried out by leveraging visual information. By preintegrating [35,36] the IMU information, the motion constraints were built with regard to the timestamps of adjacent image frames. Then, the initialization procedure was invoked by using feature information and preintegration information, in order to maintain a constant amount of computing, the optimization algorithm based on the sliding window was adopted. Finally, because the two types of trajectories did not belong to the same coordinate, the alignment was carried out by an iterative closest point algorithm (ICP) [37] and evaluated the absolute trajectory error (ATE) of the corresponding experiment. The ATE can be calculated by comparing the ground-truth value with the visualinertial odometry results. p rtk i represents the true position value of timestamp i, p odo i expresses the position output of VIO, and E i represents the ATE. As shown in the section Appendix A, the RMSE, median, and mean statistical errors are calculated with ATE. Through the rigid body transformation S composed of R and t, we can align the estimated results to the real value. To solve S, we aligned the estimated trajectory with the RTK trajectory using ICP [37].

of 23
There is little drift for a period of time after the system starts executing, and the initial trajectory segment is used for alignment. If the whole trajectory is used for alignment, the error after a long time will be adjusted due to the transformation matrix S, giving rise to inconsistency error. Here, our trajectory segment selects the first 70 data (k = 70) of the whole trajectory, that is, data of the first 15 s. In the experiment, because the vehicle ran on an approximate two-dimensional plane, we concentrated on the localization accuracy in x, y directions. In a section of Appendix A, we summarized the comparison table of algorithm errors in different scenarios.

System States
The vector states of the system include n + 1 pose state x in the sliding window and m + 1 landmarks, where n represents the size of a sliding window.
The pose state x k includes position, velocity, orientation, accelerometer bias, and gyroscope bias.
(·) wb k represents the state of body frame b with respect to the world frame w at time k. b a and b g represent accelerometer bias and gyroscope bias respectively.

Visual Constraints
Using the extracted feature points, we can construct visual constraints. According to the reprojection process of the camera, we can construct the error cost function, which is called the reprojection error, and the error is expressed as the estimated value minus the measured value: The x c j y c j z c j T represents the estimated value of the feature points of frame ith projected to the jth frame camera coordinate system according to the transformation matrix of T, the specific projection formula is shown below. The feature points of ith frame are first transformed into the world coordinate system by the pose matrix of the ith frame, and then the estimated value of the reprojection of these feature points under the jth frame is obtained by the pose matrix of the jth frame, where 1 λ denotes the depth information. The u c j v c j T represents the measurement value of the feature point in the jth frame in the camera coordinate system.

Pre-integration of IMU
In the VIO algorithm, since the output frequencies of the camera and IMU are different, the vision is generally about 30 Hz, and the IMU is generally about 200 Hz. To match each other between image and IMU measurements, we need to preintegrate the information of IMU. The IMU pre-integration changes the reference coordinate system to the body coordinate system of the previous frame rather than the world frame. This information is regarded as the motion constraint provided by the IMU.
Through the pre-integration [35,36], the following motion constraints can be constructed: represents the pre-integration measurement and [q] im represents the imaginary part of a quaternion q. Through this step, we obtain a constraint on the IMU preintegration information to constrain the state between two moments. For example, p wb j and p wb i denote the position of moment ith and moment jth respectively. The position state is one of the system states.

Nolinearity Optimization
When the respective cost functions are constructed, we use the nonlinear optimization algorithm to jointly optimize the objective function (14). This objective function contains three residual terms, namely, the prior constraint with marginalization information, the IMU pre-integration measurement constraint, and the visual reprojection constraint. Figure 2 shows the mobile platform used in the experiment. It is equipped with five modules.

Sensor Setup
The MEMS-based IMU is becoming more and more precise, reliable, and rugged, indicating a great future potential as the MEMS technology continues to be developed. In addition, it has a smaller size, weight, lower cost, and power and is an ideal choice for UAVs, unmanned vehicles, wearable devices, and many other applications. Considering that most fusion scenarios require lightweight hardware systems, the MEMS IMU has gradually become the mainstream. The six different IMUs we selected in the paper are all based on MEMS.
In terms of performance and usage scenarios, IMU is divided into four categories [29,33]. The first is the navigation grade, which is mainly used in spacecraft, aircraft, ships, missiles, and other rugged demand occasions. The second is the tactical grade, which is mainly used for UAV navigation and localization, smart munitions, etc. It is the most diverse and has smaller footprints and lower cost than the navigation grade. The third is the industrial grade, mainly used in industrial equipment, industrial robots, and other fields. The last is consumer-grade. IMU of this grade is a common occurrence, which is mainly used in mobile phones, wearable devices, motion-sensing games, and so on.
In the experiment, we developed six IMUs with different performances, and all were rigidly mounted on the circuit board as shown in Figure 3. Two of them are classified into consumer-grade IMU ( MPU6050, HI219) and four of them are classified into tactical grade IMU ( NV-MG-201, ADIS16488, ADIS16490, MSCRG). The module of MPU6050 is very prevailing and easy to access in the community. The nominal performance is the worst, and the price is only $1. The module of HI219, which costs $20, has been processed by the manufacturer. Many internal specifications and parameters are unknown because of the internal processing. NV-MG-201 is a tactical IMU and costs $500. The next is the two tactical products of the ADI manufacturer. The accuracy of ADIS16488 is slightly lower than ADIS16490. ADIS16488 costs $2500 and ADIS16490 costs $3000. The last IMU module named MSCRG is composed of a gyroscope from Japan and an accelerometer from Switzerland. MSCRG IMU offers high immunity to vibration and shock because of the unique resonating cos2θ ring structure for the gyroscope and is the best in class capacitive bulk MEMS accelerometer, and costs $3500. Table 2 shows the nominal specification parameters provided by the manufacturer within the six IMUs. Apart from these IMU modules, the binocular camera with the type of RealSense D435i was used to obtain the image data.
The next is the two tactical products of the ADI manufacturer. The accuracy of ADIS16488 is slightly lower than ADIS16490. ADIS16488 costs $2500 and ADIS16490 costs $3000. The last IMU module named MSCRG is composed of a gyroscope from Japan and an accelerometer from Switzerland. MSCRG IMU offers high immunity to vibration and shock because of the unique resonating cos2θ ring structure for the gyroscope and is the best in class capacitive bulk MEMS accelerometer, and costs $3500. Table 2 shows the nominal specification parameters provided by the manufacturer within the six IMUs. Apart from these IMU modules, the binocular camera with the type of ➆ RealSense D435i was used to obtain the image data.

Calibration of MEMS-IMUs
Allan variance is widely applied to evaluate the noise parameters of IMU [38,39]. We used the open-sourced tool kalibr_allan [40] to analyze Allan's deviation of each IMU. The Allan curve was plotted in Figure 4, and the Allan result was summarized in Table 3. If bias stability is taken as the evaluation standard, their accelerometer performance ranking from low to high is roughly: ADIS16488, HI219, MPU6050, NV-MG-201, MSCRG, ADIS16490. For gyroscopes, the ranking is ADIS488, MSCRG, MPU6050, ADIS16490, NV-MG-201, and HI219. In addition, there is no strong correlation with price. Although there is no strict standard, the tactical IMU has lower bias stability. Surprisingly, as a consumergrade HI219 gyroscope, it has the lowest bias stability.

Calibration of MEMS-IMUs
Allan variance is widely applied to evaluate the noise parameters of IMU [38,39]. We used the open-sourced tool kalibr_allan [40] to analyze Allan's deviation of each IMU. The Allan curve was plotted in Figure 4, and the Allan result was summarized in Table 3. If bias stability is taken as the evaluation standard, their accelerometer performance ranking from low to high is roughly: ADIS16488, HI219, MPU6050, NV-MG-201, MSCRG, ADIS16490. For gyroscopes, the ranking is ADIS488, MSCRG, MPU6050, ADIS16490, NV-MG-201, and HI219. In addition, there is no strong correlation with price. Although there is no strict standard, the tactical IMU has lower bias stability. Surprisingly, as a consumer-grade HI219 gyroscope, it has the lowest bias stability.
In-run bias stability, often called the bias instability, is an indication of how the bias will drift during a period of time at a certain temperature and is a considerable characterization parameter. For bias repeatability, it represents the dispersion of the sensor's bias at each powerup. How similar is the bias at each powerup of IMU? Because the thermal, physical, electrical, etc., will not be exactly the same during each powerup, there will be fluctuations in the bias. If the bias repeatability is greater, the bias consistency is worse and will affect the accuracy of the system. The inertial navigation system can estimate the bias after each powerup. The noise represents the measurement noise of the sensor. Wiener process is usually used to model the process of bias changing continuously with time, which is called bias random walk. The noise and bias random walk constitute the diagonal covariance matrix of the sensor noise term.  In-run bias stability, often called the bias instability, is an indication of how the bias will drift during a period of time at a certain temperature and is a considerable characterization parameter. For bias repeatability, it represents the dispersion of the sensor's bias at each powerup. How similar is the bias at each powerup of IMU? Because the thermal, physical, electrical, etc., will not be exactly the same during each powerup, there will be fluctuations in the bias. If the bias repeatability is greater, the bias consistency is worse and will affect the accuracy of the system. The inertial navigation system can estimate the bias after each powerup. The noise represents the measurement noise of the sensor. Wiener process is usually used to model the process of bias changing continuously with time, which is called bias random walk. The noise and bias random walk constitute the diagonal covariance matrix of the sensor noise term.
Due to the incompleteness of the IMU measurement model, the parameters above cannot be directly used in the configuration parameters of VINS-Fusion after discretization, otherwise, the trajectory will drift. We appropriately enlarge and adjust the discretized parameters to obtain their configuration parameters. To control the variables in the experiment and consider only the different performances of IMUs, we average the configuration parameters and obtained a common configuration parameter. In this way, only the impact of IMU on the fusion algorithm is considered. The common configuration parameters are _ (0.170), _ (0.014), _ (0.008), _ (0.00042). The off-line calibration of cameras and IMU is a widely studied problem. In order to effectively fuse visual information with IMU information, we need to unify the camera and IMU into a certain coordinate system, that is, the spatial calibration of the camera and IMU. Spatial calibration bridges the gap between the data in different coordinate systems.  Due to the incompleteness of the IMU measurement model, the parameters above cannot be directly used in the configuration parameters of VINS-Fusion after discretization, otherwise, the trajectory will drift. We appropriately enlarge and adjust the discretized parameters to obtain their configuration parameters. To control the variables in the experiment and consider only the different performances of IMUs, we average the configuration parameters and obtained a common configuration parameter. In this way, only the impact of IMU on the fusion algorithm is considered. The common configuration parameters are acc_ n(0.170), gyr_ n(0.014), acc_ w(0.008), gyr_ w(0.00042).

Camera-IMU Temporal-Spatial Calibration
The off-line calibration of cameras and IMU is a widely studied problem. In order to effectively fuse visual information with IMU information, we need to unify the camera and IMU into a certain coordinate system, that is, the spatial calibration of the camera and IMU. Spatial calibration bridges the gap between the data in different coordinate systems. In addition, since the camera and IMU are triggered separately under different clock sources, and there are problems such as transmission delay and CPU overload, it is necessary to correct the time offset between the camera and IMU. We exploited the open-sourced calibration tool Kalibr [41,42] to obtain the spatial transformation. As to the time offset error between IMU and cameras, we enabled the VINS_ Fusion's online calibration algorithm [43].

Sensors Frequency
For the sake of experimental consistency, the output frequency of the six IMUs was uniformly set to 200 Hz and the frequency of the camera was 30 Hz. In addition, the resolution of the left and right images was 640 × 480.

Loop Closure
In the experiment, the goal was to explore the fusion of vision and inertial independently. In consideration of the visual loop would calibrate the accumulated error, loop-closure detection was not enabled here.

Weak Texture in Corridor
We chose the corridor as the test scenario, walked straight for a distance, and returned the same way. There was a weak texture area as shown in Figure 5 at the corner, which lasted for 0.5 s. From Figure 6, it can be found that the IMU's assistance makes the track coincide, and only the camera will directly give the wrong track.
In addition, since the camera and IMU are triggered separately under different clock sources, and there are problems such as transmission delay and CPU overload, it is necessary to correct the time offset between the camera and IMU. We exploited the opensourced calibration tool Kalibr [41,42] to obtain the spatial transformation. As to the time offset error between IMU and cameras, we enabled the VINS_ Fusion's online calibration algorithm [43].

Sensors Frequency
For the sake of experimental consistency, the output frequency of the six IMUs was uniformly set to 200 Hz and the frequency of the camera was 30 Hz. In addition, the resolution of the left and right images was 640 × 480.

Loop Closure
In the experiment, the goal was to explore the fusion of vision and inertial independently. In consideration of the visual loop would calibrate the accumulated error, loop-closure detection was not enabled here.

Weak Texture in Corridor
We chose the corridor as the test scenario, walked straight for a distance, and returned the same way. There was a weak texture area as shown in Figure 5 at the corner, which lasted for 0.5 s. From Figure 6, it can be found that the IMU's assistance makes the track coincide, and only the camera will directly give the wrong track.   sources, and there are problems such as transmission delay and CPU overload, it is necessary to correct the time offset between the camera and IMU. We exploited the opensourced calibration tool Kalibr [41,42] to obtain the spatial transformation. As to the time offset error between IMU and cameras, we enabled the VINS_ Fusion's online calibration algorithm [43].

Sensors Frequency
For the sake of experimental consistency, the output frequency of the six IMUs was uniformly set to 200 Hz and the frequency of the camera was 30 Hz. In addition, the resolution of the left and right images was 640 × 480.

Loop Closure
In the experiment, the goal was to explore the fusion of vision and inertial independently. In consideration of the visual loop would calibrate the accumulated error, loop-closure detection was not enabled here.

Weak Texture in Corridor
We chose the corridor as the test scenario, walked straight for a distance, and returned the same way. There was a weak texture area as shown in Figure 5 at the corner, which lasted for 0.5 s. From Figure 6, it can be found that the IMU's assistance makes the track coincide, and only the camera will directly give the wrong track.

Uniform Velocity Motion State
In these experiments, due to the high-precision RTK ground truth can be provided outdoors, we selected the environment as shown in Figure 7 for evaluation. There were three uniform velocity motion modes: slow, normal, and fast, and they moved for 300, 200, and 120 s, respectively. To maintain experimental consistency, each motion state was evaluated five times. One of the trajectory diagrams and ATE diagrams are drawn as shown in Figure 8.

Uniform Velocity Motion State
In these experiments, due to the high-precision RTK ground truth can be provided outdoors, we selected the environment as shown in Figure 7 for evaluation. There were three uniform velocity motion modes: slow, normal, and fast, and they moved for 300, 200, and 120 s, respectively. To maintain experimental consistency, each motion state was evaluated five times. One of the trajectory diagrams and ATE diagrams are drawn as shown in Figure 8.   outdoors, we selected the environment as shown in Figure 7 for evaluation. There w three uniform velocity motion modes: slow, normal, and fast, and they moved for 200, and 120 s, respectively. To maintain experimental consistency, each motion state evaluated five times. One of the trajectory diagrams and ATE diagrams are draw shown in Figure 8.

Alternating Acceleration and Deceleration Motion State
In this scenario, the wheeled robot ceaselessly kept accelerating and decelerating in order to motivate IMU. Figure 9 plots the trajectory and the ATE error. Similarly, to maintain experimental consistency, this motion state was evaluated five times. It can be clearly reported that MPU6050 has a poor performance and larger drift.
Micromachines 2022, 13, x FOR PEER REVIEW 12 of 23 in the experimental environment). The right represents the ATE results for VIO. (The x−axis and y−axis represent the running timestamp and error, respectively.) (Other similar result graphs also follow this rule).

Alternating Acceleration and Deceleration Motion State
In this scenario, the wheeled robot ceaselessly kept accelerating and decelerating in order to motivate IMU. Figure 9 plots the trajectory and the ATE error. Similarly, to maintain experimental consistency, this motion state was evaluated five times. It can be clearly reported that MPU6050 has a poor performance and larger drift.

Spin Move Forward Motion State
In this case, the wheeled robot moved forward with frequent rotation in order to motivate IMU. The trajectory shown in Figure 10 can reflect this movement. This motion was evaluated eleven times due to the complexity of the motion state. In this case, MPU6050 is easy to crash due to rapid rotation.

Strong Sun Light Scene
In this scene, the situation in a strong illumination environment was evaluated. As shown in Figure 11, there was obviously strong light in the environment. A cross-over

Spin Move Forward Motion State
In this case, the wheeled robot moved forward with frequent rotation in order to motivate IMU. The trajectory shown in Figure 10 can reflect this movement. This motion was evaluated eleven times due to the complexity of the motion state. In this case, MPU6050 is easy to crash due to rapid rotation.
in the experimental environment). The right represents the ATE results for VIO. (The x−axis and y−axis represent the running timestamp and error, respectively.) (Other similar result graphs also follow this rule).

Alternating Acceleration and Deceleration Motion State
In this scenario, the wheeled robot ceaselessly kept accelerating and decelerating in order to motivate IMU. Figure 9 plots the trajectory and the ATE error. Similarly, to maintain experimental consistency, this motion state was evaluated five times. It can be clearly reported that MPU6050 has a poor performance and larger drift.

Spin Move Forward Motion State
In this case, the wheeled robot moved forward with frequent rotation in order to motivate IMU. The trajectory shown in Figure 10 can reflect this movement. This motion was evaluated eleven times due to the complexity of the motion state. In this case, MPU6050 is easy to crash due to rapid rotation.

Strong Sun Light Scene
In this scene, the situation in a strong illumination environment was evaluated. As shown in Figure 11, there was obviously strong light in the environment. A cross-over

Strong Sun Light Scene
In this scene, the situation in a strong illumination environment was evaluated. As shown in Figure 11, there was obviously strong light in the environment. A cross-over study to evaluate the performance of different motion states under strong light was conducted. The trajectory under the variable speed scene is chosen as shown in Figure 12.
Micromachines 2022, 13, x FOR PEER REVIEW 13 of 23 study to evaluate the performance of different motion states under strong light was conducted. The trajectory under the variable speed scene is chosen as shown in Figure 12.

Long Term Scene
In long-term test experiment, which included three forms of motion: constant speed, variable speed, and spin move, was performed for 30 min. The trajectory is shown in Figure 13. From Table 4, the HI219 IMU improved the localization accuracy of the fusion system, while other IMUs deteriorated the accuracy of the system.   study to evaluate the performance of different motion states under strong light was conducted. The trajectory under the variable speed scene is chosen as shown in Figure 12.

Long Term Scene
In long-term test experiment, which included three forms of motion: constant speed, variable speed, and spin move, was performed for 30 min. The trajectory is shown in Figure 13. From Table 4, the HI219 IMU improved the localization accuracy of the fusion system, while other IMUs deteriorated the accuracy of the system.

Long Term Scene
In long-term test experiment, which included three forms of motion: constant speed, variable speed, and spin move, was performed for 30 min. The trajectory is shown in Figure 13. From Table 4, the HI219 IMU improved the localization accuracy of the fusion system, while other IMUs deteriorated the accuracy of the system. study to evaluate the performance of different motion states under strong light was conducted. The trajectory under the variable speed scene is chosen as shown in Figure 12.

Long Term Scene
In long-term test experiment, which included three forms of motion: constant speed, variable speed, and spin move, was performed for 30 min. The trajectory is shown in Figure 13. From Table 4, the HI219 IMU improved the localization accuracy of the fusion system, while other IMUs deteriorated the accuracy of the system.

Results and Analysis
Through the preceding experiments and according to Tables A1-A6 in Section of Appendix A, we summarize the votes of accuracy improvement of each IMU as shown in Table 5. If an IMU improved the accuracy the most in an evaluation scenario, we would vote for this IMU and count the votes. For example, among the fifteen evaluations in uniform velocity scenes, the ADIS16488 performed best only once in the six IMUs. As for ADIS16490, the best performance was three times. Some results are revealed based on the evaluation: (1) In the weak texture scenario over short time intervals, the localization posture is significantly improved with the aid of IMU, and only leveraging the camera will be defeated because of the absence of visual constraints. (2) The IMU's incentive is relatively small in the nearly constant speed scenario, there is no salient difference between IMUs with different performances, consumer IMUs will also perform well, and better IMUs do not show more visible advantages. (3) In the case of varying speed scenarios, the assistance of an accelerometer is needed. The IMU with the excellent specification of bias stability has better performance, such as ADIS16490 and MSCRG. (4) In the spin move situation, the gyroscope is needed. At this time, the HI219 and MSCRG have better results. (5) These performance trends are maintained in the cross experiments facing a strong sunlight environment. (6) In the 30 min long-term test, only the HI219 improves the odometry accuracy of the fusion system. Among these results, it is surprising that HI219, as a consumer-grade also has a good performance.

Turntable Test
In order to make the results clearer, the professional multiaxial turntable as shown in Figure 14 was used to uniformly measure the angular velocity of the six IMUs. As can be seen from Figure 15a, the output of HI219 is always maintained at zero when the angular velocity is less than 0.83 • /s, there is a threshold to perceive the rotational motion, while other sensors output their perceived angular velocity although the value is amiss. For example, when the robot is stationary, orientation will not drift due to the zero-bias of the gyroscope. This phenomenon makes HI219 bring less drift and results in the lowest bias stability of the gyroscope as shown in Figure 4b. In addition, when the angular velocity is greater than 16 • /s, except mpu6050, the rest IMUs' error has been reduced to less than 9%, and their error discrimination is very small as shown in Figure 15b, resulting in the algorithm error between them not being very remarkable.
It should be noted that the triaxial gyroscope sensor is actually composed of triaxial identical gyroscopes placed in orthogonal directions, hence the rotation of the gyroscope around the Z-axis was measured, and there are more cases around the gravity direction in-plane motion.

Quantitative Analysis
The measurement model of the accelerometer and gyroscope is given follows: ̃ and ̃ represent the measurements of the gyroscope and accelerometer, respectively. and represent the ideal value of the gyroscope and accelerometer, respectively. Due to various factors, measurements are affected by gyroscope bias , acceleration bias , and noise . In addition, we assume that the noise in gyroscope and acceleration measurements are Gaussian, that is, ~(0, 2 ), ~(0, 2 ). As for the bias, gyroscope bias and acceleration bias are modeled as a random walk, whose derivatives are gaussian, ~(0, 2 ), ~(0, 2 ), so we can obtain: ̇= (10) Figure 14. The multiaxial turntable with multi-IMUs camera platform is rigidly mounted. It should be noted that the triaxial gyroscope sensor is actually composed of triaxial identical gyroscopes placed in orthogonal directions, hence the rotation of the gyroscope around the Z-axis was measured, and there are more cases around the gravity direction in-plane motion.

Quantitative Analysis
The measurement model of the accelerometer and gyroscope is given follows: ̃ and ̃ represent the measurements of the gyroscope and accelerometer, respectively. and represent the ideal value of the gyroscope and accelerometer, respectively. Due to various factors, measurements are affected by gyroscope bias , acceleration bias , and noise . In addition, we assume that the noise in gyroscope and acceleration measurements are Gaussian, that is, ~(0, 2 ), ~(0, 2 ). As for the bias, gyroscope bias and acceleration bias are modeled as a random walk, whose derivatives are gaussian, ~(0, 2 ), ~(0, 2 ), so we can obtain: ̇= (10) It should be noted that the triaxial gyroscope sensor is actually composed of triaxial identical gyroscopes placed in orthogonal directions, hence the rotation of the gyroscope around the Z-axis was measured, and there are more cases around the gravity direction in-plane motion.

Quantitative Analysis
The measurement model of the accelerometer and gyroscope is given follows: w b and a b represent the measurements of the gyroscope and accelerometer, respectively. w b and a w represent the ideal value of the gyroscope and accelerometer, respectively. Due to various factors, measurements are affected by gyroscope bias b g , acceleration bias b a , and noise n. In addition, we assume that the noise in gyroscope and acceleration measurements are Gaussian, that is, n g ∼ N 0, σ 2 g , n a ∼ N 0, σ 2 a . As for the bias, gyroscope bias and acceleration bias are modeled as a random walk, whose derivatives are gaussian, n b g ∼ N 0, σ 2 b g , n b a ∼ N 0, σ 2 b a , so we can obtain: .
We can obtain the above four noise coefficients (n g , n a , n b g , n b a ) through the Allan analysis in Section 3.4.1. These configuration parameters constitute the noise covariance matrix Q: We can update the covariance matrix of pre-integration by the transition matrix F, V, and noise matrix Q, see Appendix B for specific elements of F and V: Similarly, there is a covariance matrix for visual information. These covariance matrices represent the noise of IMU information and visual information. We can optimize the system state X through the following cost function: r obj = min r p − J p X 2 + r I MU Z I MU , X 2 p I MU + r visual Z visual , X 2 P visual (14) It should be noted that through the covariance matrix P I MU and P visual , the Euclidean distance is converted into Mahalanobis distance. It fixes the problem of inconsistent and related dimensions in European distance. Although the information is very different between the IMU and visual, they can be optimized in the same cost function (14) through Mahalanobis distance. In this way, the residual information of IMU and visual is statistically equivalent to each other. For visual information, it is consistent throughout the evaluation experiment, so only IMU information affects the accuracy of visual-inertial odometry.
Through the algorithm evaluation results in Tables A1-A6 in the section of Appendix A, we summarized the accuracy of the localization improved by adding IMU and obtained the improvement under the following four main motion scenes just as shown in Table 6. Table 6. Improvement of average localization accuracy after adding IMU in different scenarios. Unit: m (we considered the situation of improvement of accuracy. Each item represents accuracy improvement in the situation of IMU auxiliary relative to the situation of only visual localization. For example, the auxiliary of MPU6050 improves the accuracy in uniform scenes by 0.1031 m). Combined with quantitative results, in the uniform velocity situation, the lifting amplitude of each IMU is in the range of 0.1 m. In the varying velocity situation, the IMU produced by ADI has increased the most, which is better than MPU6050 in this scenario.

Scenes
In spin move situations, because of the special processing of HI219 and the superiority of the MSCRG structure, the accuracy is greatly improved by their assistance. In the strong illumination situation, there is little difference between them. Except for NV-MG-201, the lifting effect of different IMUs is 0.15m. The most expensive IMU is 2.5 cm higher in accuracy than the cheapest one.
In summary, the improvement of fusion accuracy with different grade MEMS-IMU is limited in these experimental scenarios, the consumer-grade IMU can also have an excellent result. The improvement of accuracy depends more on the algorithm.
As shown in formula (14), if the measurement noise of IMU is smaller, the theoretical accuracy will be higher. However, when the difference in measurements between these IMU is small, the difference in residual information between them is smaller. Due to the weighting effect in formula (14), this results in limited differences in accuracy.

Conclusions
In general, there are many internal and external factors that affect the IMU. We can obtain the representative parameters through the Allan variance method to quantitatively analyze the performance of IMU. However, the method cannot represent all performance in actual applications. In this paper, many scenario experiments were conducted, and the professional turntable was employed to analyze the error of six IMUs. The following conclusions are reached: The assistance of IMU can improve the accuracy of multi-sensor fusion, and is more notable in weak texture scenes. In the constant speed scene, there is no obvious difference between IMUs with different performances. Under the excitation of rotation, acceleration, and deceleration, IMUs with excellent performance will have higher accuracy and are more stable. Owing to the lower bias stability and noise, making their performance more robust. The improvement of fusion accuracy is not directly proportional to the price with regard to the expensive ADIS16490 IMU, even so, it is more versatile in various scenarios. For HI219, a consumer IMU, there is a threshold for sensing rotation motion, performing well in rotation scenes, which may provide a reference for the processing of the algorithm. At the same time, according to the MSCRG IMU results, IMU with resistance to vibration and impact is more needed in the situation of frequently strenuous movement.

Acknowledgments:
The authors would like to thank Kunfeng Wang for his efforts with MEMS analysis and PengCheng Zheng, and Haifeng Zhang for assistance with the experimental operation and analysis. Thanks to everyone who helped us in our work.

Conflicts of Interest:
The authors declare no conflict of interest.

Appendix A
This appendix records the absolute trajectory error (ATE) results evaluated under each scenario. Each column represents different IMUs, and each row represents the algorithm accuracy including mean, median, and RMSE about ATE. Each cell represents the error results with IMU assistance and without IMU assistance (0.9124/1.1250 ↑). The bold represents that the accuracy has been improved with the aid of IMU. The symbol ↑ represents the IMU with the greatest accuracy improvement in this evaluation. We vote for the IMU through the symbol and make statistics according to the number of votes.