Multi-Modal Tightly Coupled Robust Pose Estimation for Mobile Robots in Complex Degraded Scenarios
Abstract
1. Introduction
2. Related Work
2.1. Evolution of Tightly Coupled Multi-Modal Odometry Architectures
2.2. Kinematic Constraints and Strategies for Coping with Extreme Degeneration
2.3. Manifold Filtering and Non-Gaussian Robust Control
3. System Overview and Kinematics Modeling
3.1. Frames of Reference and Notational Conventions
3.2. IMU Kinematics and Pre-Integration Model
3.3. Multimodal Observation Model
4. Adaptive Robust Multi-Source Manifold Iterative Filtering Algorithm
4.1. Pre-Integration-Driven iESKF State Propagation Framework
4.1.1. Error-State Parameterization on Lie Group Manifolds
4.1.2. Preintegration-Driven High-Frequency Prior State Propagation
4.1.3. Manifold Propagation of the State Covariance Matrix
4.2. Multi-Source Manifold Residual Mapping and Adaptive Robust Update Mechanism
4.2.1. Unified Lie Algebra Mapping of Heterogeneous Multi-Modal Residuals
4.2.2. Mahalanobis Chi-Square Anomaly Detection and IRLS Covariance Adaptive Soft-Isolation
| Kernel Function | Weight Formula | Switching Trigger Condition |
|---|---|---|
| Huber Kernel | Default Mode: Applied when the multi-source health metrics (defined in Table 3) of the corresponding sensor remain within nominal bounds. | |
| Cauchy Kernel | Degraded Mode: Triggered strictly when a specific modal health metric breaches safety boundaries (e.g., tracked features or chassis kinematic violation). |
| Sensor | Health Metric | Degradation Trigger Condition | Adaptive Compensation |
|---|---|---|---|
| LiDAR | Minimum eigenvalue of the information matrix | (Vanishing of constraints in critical directions) | Activate non-holonomic constraints (NHC) and consistency check for point cloud distribution entropy |
| Visual Odometry | Effective tracking of feature point count | Number of tracked points | Increase IMU preintegration weight; lock visual scale bias |
| Wheel Odometry | Chassis side-slip and vertical motion residuals | Violation of kinematic consistency | Trigger skidding discrimination logic; exponentially inflate covariance |
4.3. Smooth Degradation and State Reconstruction Under Composite Degraded Conditions
4.3.1. Multi-Source Perceptual Health Quantification and Dynamic Weight Allocation
4.3.2. Fault-Tolerant Strategy of State Machine Under Sensing Failure and Slipping Conditions
- (1)
- When the performance of vision or laser systems degrades without slippage occurring, NHC is adopted to suppress the offset of the Z-axis and lateral directions.
- (2)
- If wheel slip is detected, increase the measurement noise covariance of the wheel speed sensor and NHC. At this time, only IMU preintegration and other normal external sensors need to be used.
4.4. Algorithm Process Architecture and Computational Efficiency Verification
4.4.1. Comprehensive Workflow of the Adaptive Robust Filtering Algorithm
- (1)
- Weight Reconstruction (Soft Isolation): For abnormal measurements with non-Gaussian heavy-tailed characteristics, invoke the Huber or Cauchy kernel function according to the magnitude of residuals, recalculate the diagonal weight penalty matrix, and realize the adaptive expansion of the observation noise covariance.
- (2)
- Multi-Round Iterative Solving: The weighted and adaptively inflated observation covariance is incorporated into the manifold optimization equation. Non-linear solving for the minimal error-state increment is executed cyclically for 2 to 4 iterations until the norm of the increment converges within a predefined threshold .
4.4.2. Time Complexity and Efficiency Analysis on Edge Computing Platforms
5. Experimental Results
5.1. Experimental Platform and System Parameter Configuration
5.2. Accuracy Comparison Under Structured Benchmark Scenarios
5.3. Robustness Verification Under Composite Degraded Conditions
5.3.1. Closed-Loop Scenario Design and Objective Evaluation Benchmarks
5.3.2. Divergence Mechanism Analysis and White-Box Validation of the State Machine
- (1)
- At t = 40 s, upon entering the corridor blind zone, the information matrix eigenvalue quantification system detects that the system observability is below the safety threshold. The state machine suspends the all-source tightly coupled state and activates the chassis non-holonomic constraints (NHC), preventing tail-wagging by introducing a zero lateral velocity expectation.
- (2)
- At t = 46 s, during the wheel slippage phase, the Mahalanobis distance of the measurement residuals breaches the chi-squared defense line, and the state machine discards all contaminated observations. At this point, the system shunts the processing based on underlying dynamic constraints: since the IMU does not integrate a noticeable displacement at the instant of skidding, the state machine automatically downgrades to the Level 3 zero-velocity correction mode shown in Figure 5, avoiding divergence in the Z-axis direction.
- (3)
- When the robot exits the blind zone and environmental features recover at t > 60 s, the state machine upgrades its dimension and resumes the all-source solution.
5.3.3. Ablation Study: Validating the Contribution of the Adaptive Robust Mechanism
5.4. Robustness Analysis Against Non-Gaussian Perturbations and Wheel Skidding
5.5. Generalizability Validation in Challenging Industrial Scenarios with Non-Gaussian Disturbances
5.6. Edge Computational Overhead and Real-Time Performance Evaluation
5.6.1. Experimental Design and Evaluation Benchmarks
5.6.2. Experimental Results and Analysis
5.6.3. Computational Efficiency Discussion
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Cadena, C.; Carlone, L.; Carrillo, H.; Latif, Y.; Scaramuzza, D.; Neira, J.; Reid, I.; Leonard, J.J. Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age. IEEE Trans. Robot. 2016, 32, 1309–1332. [Google Scholar] [CrossRef]
- Ebadi, K.; Bernreiter, L.; Biggie, H.; Catt, G.; Chang, Y.; Chatterjee, A.; Denniston, C.E.; Deschênes, S.-P.; Harlow, K.; Khattak, S.; et al. Present and Future of SLAM in Extreme Underground Environments. arXiv 2022, arXiv:2208.01787. [Google Scholar] [CrossRef]
- Wu, K.J.; Guo, C.X.; Georgiou, G.; Roumeliotis, S.I. VINS on wheels. In Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore, 29 May–3 June 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 5155–5162. [Google Scholar] [CrossRef]
- Huang, G. Visual-inertial navigation: A concise review. In Proceedings of the 2019 IEEE International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, 20–24 May 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 9572–9582. [Google Scholar] [CrossRef]
- Forster, C.; Carlone, L.; Dellaert, F.; Scaramuzza, D. On-Manifold Preintegration for Real-Time Visual--Inertial Odometry. IEEE Trans. Robot. 2017, 33, 1–21. [Google Scholar] [CrossRef]
- Yang, Y.; Geneva, P.; Zuo, X.; Huang, G. Online Self-Calibration for Visual-Inertial Navigation: Models, Analysis, and Degeneracy. IEEE Trans. Robot. 2023, 39, 3479–3498. [Google Scholar] [CrossRef]
- Xu, W.; Cai, Y.; He, D.; Lin, J.; Zhang, F. FAST-LIO2: Fast Direct LiDAR-Inertial Odometry. IEEE Trans. Robot. 2022, 38, 2053–2073. [Google Scholar] [CrossRef]
- Zhang, J.; Singh, S. Low-drift and real-time lidar odometry and mapping. Auton. Robots 2017, 41, 401–416. [Google Scholar] [CrossRef]
- Wisth, D.; Camurri, M.; Fallon, M. VILENS: Visual, Inertial, Lidar, and Leg Odometry for All-Terrain Legged Robots. IEEE Trans. Robot. 2023, 39, 309–326. [Google Scholar] [CrossRef]
- Brossard, M.; Barrau, A.; Bonnabel, S. RINS-W: Robust inertial navigation system on wheels. In Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China, 3–8 November 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 2068–2075. [Google Scholar] [CrossRef]
- Brossard, M.; Barrau, A.; Bonnabel, S. AI-IMU dead-reckoning. IEEE Trans. Intell. Transp. Syst. 2020, 5, 585–595. [Google Scholar] [CrossRef]
- Barrau, A.; Bonnabel, S. The invariant extended Kalman filter as a stable observer. IEEE Trans. Autom. Control 2017, 62, 1797–1812. [Google Scholar] [CrossRef]
- Sola, J. Quaternion kinematics for the error-state Kalman filter. arXiv 2017, arXiv:1711.02508. [Google Scholar] [CrossRef]
- Qin, T.; Li, P.; Shen, S. VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator. IEEE Trans. Robot. 2018, 34, 1004–1020. [Google Scholar] [CrossRef]
- Shan, T.; Englot, B.; Meyers, D.; Wang, W.; Ratti, C.; Rus, D. LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping. In Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA, 24 October 2020–24 January 2021; IEEE: Piscataway, NJ, USA, 2020; pp. 5135–5142. [Google Scholar] [CrossRef]
- Bai, C.; Xiao, T.; Chen, Y.; Wang, H.; Zhang, F.; Gao, X. Faster-LIO: Lightweight Tightly Coupled Lidar-Inertial Odometry Using Parallel Sparse Incremental Voxels. IEEE Robot. Autom. Lett. 2022, 7, 4861–4868. [Google Scholar] [CrossRef]
- Campos, C.; Elvira, R.; Rodríguez, J.J.G.; Montiel, J.M.M.; Tardós, J.D. ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM. IEEE Trans. Robot. 2021, 37, 1874–1890. [Google Scholar] [CrossRef]
- Geneva, P.; Eckenhoff, K.; Lee, W.; Yang, Y.; Huang, G. OpenVINS: A Research Platform for Visual-Inertial Estimation. In Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France, 31 May–31 August 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 4666–4672. [Google Scholar] [CrossRef]
- He, D.; Xu, W.; Chen, N.; Kong, F.; Yuan, C.; Zhang, F. Point-LIO: Robust High-Bandwidth Light Detection and Ranging Inertial Odometry. Adv. Intell. Syst. 2023, 5, 2200459. [Google Scholar] [CrossRef]
- Ye, H.; Chen, Y.; Liu, M. Tightly Coupled 3D Lidar Inertial Odometry and Mapping. In Proceedings of the 2019 IEEE International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, 20–24 May 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 3144–3150. [Google Scholar] [CrossRef]
- Shan, T.; Englot, B.; Ratti, C.; Rus, D. LVI-SAM: Tightly-coupled Lidar-Visual-Inertial Odometry via Smoothing and Mapping. In Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi’an, China, 30 May–5 June 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 5692–5698. [Google Scholar] [CrossRef]
- Lin, J.; Zhang, F. R3LIVE: A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package. In Proceedings of the 2022 IEEE International Conference on Robotics and Automation (ICRA), Virtual, 23–27 May 2022; IEEE: Piscataway, NJ, USA, 2022; pp. 10672–10678. [Google Scholar] [CrossRef]
- Zheng, C.; Zhu, Q.; Xu, W.; Liu, X.; Guo, Q.; Zhang, F. FAST-LIVO: Fast and Tightly-coupled Sparse-Direct LiDAR-Inertial-Visual Odometry. In Proceedings of the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan, 23–27 October 2022; IEEE: Piscataway, NJ, USA, 2022; pp. 433–440. [Google Scholar] [CrossRef]
- Zuo, X.; Geneva, P.; Lee, W.; Liu, Y.; Huang, G. LIC-Fusion: LiDAR-Inertial-Camera Odometry. In Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China, 3–8 November 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 5848–5854. [Google Scholar] [CrossRef]
- Zhao, S.; Zhang, H.; Wang, P.; Nogueira, L.; Scherer, S. Super Odometry: IMU-centric LiDAR-Visual-Inertial Estimator for Challenging Environments. In Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi’an, China, 30 May–5 June 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 8329–8336. [Google Scholar] [CrossRef]
- Hartley, R.; Ghaffari, M.; Eustice, R.M.; Grizzle, J.W. Contact-aided invariant extended Kalman filtering for robot state estimation. Int. J. Rob. Res. 2020, 39, 402–430. [Google Scholar] [CrossRef]
- Chen, B.; Liu, X.; Zhao, H.; Principe, J.C. Maximum correntropy Kalman filter. Automatica 2017, 76, 70–77. [Google Scholar] [CrossRef]
- Lin, J.; Zheng, C.; Xu, W.; Zhang, F. R2LIVE: A Robust, Real-Time, LiDAR-Inertial-Visual Tightly-Coupled State Estimator and Mapping. IEEE Robot. Autom. Lett. 2021, 6, 7469–7476. [Google Scholar] [CrossRef]
- Yang, Y.X.; Ren, X.; Yan, X. Main progress of adaptively robust filter with applications in navigation. J. Navig. Position. 2013, 1, 9–15. [Google Scholar]
- Zhang, J.; Kaess, M.; Singh, S. On degeneracy of optimization-based state estimation problems. In Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, 16–21 May 2016; IEEE: Piscataway, NJ, USA, 2016; pp. 809–816. [Google Scholar] [CrossRef]
- Skog, I.; Handel, P.; Nilsson, J.O.; Rantakokko, J. Zero-velocity detection—An algorithm evaluation. IEEE Trans. Biomed. Eng. 2010, 57, 2657–2666. [Google Scholar] [CrossRef] [PubMed]
- Zhu, J.; Li, H.; Wang, Z.; Wang, S.; Zhang, T. i-Octree: A Fast, Lightweight, and Dynamic Octree for Proximity Search. In Proceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA), Yokohama, Japan, 13–17 May 2024; IEEE: Piscataway, NJ, USA, 2024; pp. 12290–12296. [Google Scholar] [CrossRef]










| State Component | Notation Definition | Manifold Space | Generalized Addition Operation () |
|---|---|---|---|
| Rotation matrix | |||
| Position | |||
| Velocity | |||
| Accelerometer bias | |||
| Gyroscope bias |
| Algorithmic Architecture | Mathematical Mechanism | Computational Complexity of State Estimation | Computational Complexity of Query | Features and Advantages |
|---|---|---|---|---|
| Standard EKF | First-order linear propagation | Extremely lightweight, prone to divergence | ||
| FGO (Factor Graph) | Sliding window optimization | High accuracy, heavy computational overhead | ||
| FAST-LIO2 | iESKF + ikd-Tree | Efficiency balance, weak capability in handling skidding | ||
| Proposed | Adaptive iESKF + i-Octree | Balances accuracy and robustness |
| Algorithmic Module | Average Execution Time (ms) | Percentage of Total Runtime |
|---|---|---|
| IMU Pre-integration & Prior State Propagation | 0.9 | 7.0% |
| Spatial Map Query (i-Octree Search) & Feature Association | 4.1 | 32.0% |
| Jacobian Construction & Point-wise Mahalanobis Check | 2.8 | 21.9% |
| Adaptive M-Estimation Weighting (Robust Module) | 0.6 | 4.7% |
| iESKF Iterative State Update (State-space matrix inversion) | 3.2 | 25.0% |
| Map Incremental Maintenance (i-Octree Update) | 1.2 | 9.4% |
| Total Frame Processing Time | 12.8 | 100% |
| Algorithm | Translation RMSE (m) | Rotation RMSE (°) |
|---|---|---|
| Ours (AR-iESKF) | 0.12 | 0.85 |
| FAST-LIO2 | 0.15 | 0.92 |
| FAST-LIVO | 0.13 | 0.88 |
| LVI-SAM | 0.11 | 0.79 |
| Algorithm | Closed-Loop Translation Error (m) | Maximum Z-Axis Drift (m) |
|---|---|---|
| Ours (Full AR-iESKF) | 1.24 | 0.32 |
| Naïve Wheel-LVI (Ablation Baseline) | 2.84 | 1.45 |
| FAST-LIO2 | 2.78 | 1.87 |
| LVI-SAM (FGO) | 11.70 | 21.45 |
| Algorithm | End-to-End Drift (m) | Max Z-Axis Jitter (m) |
|---|---|---|
| Ours (AR-iESKF) | 0.35 | 0.12 |
| FAST-LIO2 | 0.76 | 0.26 |
| LVI-SAM (FGO) | 1.12 | 0.41 |
| Algorithm | Average Time Consumption (ms) | Median Time Consumption (ms) | 99th Percentile Latency (ms) | Real-Time Level |
|---|---|---|---|---|
| Ours | 12.8 | 12.2 | 17.9 | Near 100 Hz |
| FAST-LIO2 | 16.5 | 15.9 | 24.6 | Meets 50 Hz |
| FAST-LIVO | 23.1 | 21.8 | 32.5 | Meets 50 Hz |
| LVI-SAM | 41.2 | 37.5 | 82.7 | Meets 20 Hz |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
Share and Cite
Tian, H.; Li, T. Multi-Modal Tightly Coupled Robust Pose Estimation for Mobile Robots in Complex Degraded Scenarios. Sensors 2026, 26, 4485. https://doi.org/10.3390/s26144485
Tian H, Li T. Multi-Modal Tightly Coupled Robust Pose Estimation for Mobile Robots in Complex Degraded Scenarios. Sensors. 2026; 26(14):4485. https://doi.org/10.3390/s26144485
Chicago/Turabian StyleTian, Huating, and Tao Li. 2026. "Multi-Modal Tightly Coupled Robust Pose Estimation for Mobile Robots in Complex Degraded Scenarios" Sensors 26, no. 14: 4485. https://doi.org/10.3390/s26144485
APA StyleTian, H., & Li, T. (2026). Multi-Modal Tightly Coupled Robust Pose Estimation for Mobile Robots in Complex Degraded Scenarios. Sensors, 26(14), 4485. https://doi.org/10.3390/s26144485

