#
Continuous Fusion of Motion Data Using an Axis-Angle Rotation Representation with Uniform B-Spline^{ †}

^{1}

^{2}

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

- can fuse asynchronous motion measurements with different types (including poses, velocities and accelerations) and frequencies directly without loss of information;
- is processed in the time forward and backward directions to refine fusion results;
- provides pose, velocity and acceleration fusion results continuously in time.

**Figure 1.**An application example of applying our Uniform B-spline-based continuous motion fusion approach. Our approach can apply pose, velocity and acceleration measurements with different frequencies to optimize two uniform B-splines ${\mathbf{S}}_{\mathrm{t}}\phantom{\rule{4.pt}{0ex}}\mathrm{and}\phantom{\rule{4.pt}{0ex}}{\mathbf{S}}_{\mathrm{r}}$ (translation and rotation). Afterwards, the fused pose, velocity and acceleration results can be estimated continuously in time.

## 2. Related Work

## 3. Algorithm Overview

#### 3.1. Uniform B-Spline

#### 3.2. Rotation Parametrization

#### 3.3. Time Derivative of a Rotation

#### 3.4. Motion Model

#### 3.5. Uniform B-Spline-Based Fusion System Concept

## 4. Experimental Evaluation

#### 4.1. Sensor Setup

#### 4.2. Data Set

#### 4.3. IMU Extrinsic Calibration

#### 4.4. Uniform B-Spline vs. Pose Graph Optimization

#### 4.5. Runtime Analysis

#### 4.6. Pose, Velocity and Acceleration of the Fused Results

#### 4.7. Point Cloud Accumulating

## 5. Conclusions and Future Work

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Yun, S.; Lee, Y.J.; Sung, S. IMU/Vision/Lidar integrated navigation system in GNSS denied environments. In Proceedings of the 2013 IEEE Aerospace Conference, Big Sky, MT, USA, 2–9 March 2013; pp. 1–10. [Google Scholar]
- Alatise, M.; Hancke, G. Pose Estimation of a Mobile Robot Based on Fusion of IMU Data and Vision Data Using an Extended Kalman Filter. Sensors
**2017**, 17, 2164. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Samadzadegan, F.; Abdi, G. Autonomous navigation of Unmanned Aerial Vehicles based on multi-sensor data fusion. In Proceedings of the 20th Iranian Conference on Electrical Engineering (ICEE2012), Tehran, Iran, 15–17 May 2012; pp. 868–873. [Google Scholar]
- Ren, Y.; Ke, X. Particle Filter Data Fusion Enhancements for MEMS-IMU/GPS. Intell. Inf. Manag.
**2010**, 2, 417–421. [Google Scholar] [CrossRef] [Green Version] - Kim, H.; Liu, B.; Goh, C.Y.; Lee, S.; Myung, H. Robust Vehicle Localization Using Entropy-Weighted Particle Filter-based Data Fusion of Vertical and Road Intensity Information for a Large Scale Urban Area. IEEE Robot. Autom. Lett.
**2017**, 2, 1518–1524. [Google Scholar] [CrossRef] - Murtra, A.C.; Mirats Tur, J.M. IMU and Cable Encoder Data Fusion for In-Pipe Mobile Robot Localization. In Proceedings of the 2013 IEEE Conference on Technologies for Practical Robot Applications (TePRA), Woburn, MA, USA, 22–23 April 2013; pp. 1–6. [Google Scholar]
- Merfels, C.; Stachniss, C. Pose Fusion with Chain Pose Graphs for Automated Driving. In Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea, 9–14 October 2013; pp. 3116–3123. [Google Scholar]
- Mascaro, R.; Teixeira, L.; Hinzmann, T.; Siegwart, R.; Chli, M. GOMSF: Graph-Optimization Based Multi-Sensor Fusion for robust UAV Pose estimation. In Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Daejeon, Korea, 9–14 October 2018; pp. 1421–1428. [Google Scholar]
- An Experimental Study on Relative and Absolute Pose Graph Fusion for Vehicle Localization. In Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China, 26–30 June 2018; pp. 630–635.
- Chang, H.H.; Yan, H. Vectorization of Hand-Draw Image Using Piecewise Cubic Bezier Curves Fitting. Pattern Recognit.
**1998**, 31, 1747–1755. [Google Scholar] [CrossRef] - de Boor, C. On Calculating with B-splines. J. Approx. Theory
**1972**, 6, 50–62. [Google Scholar] [CrossRef] [Green Version] - Cox, M.G. The Numerical Evaluation of B-Splines. IMA J. Appl. Math.
**1972**, 10, 134–149. [Google Scholar] [CrossRef] - Cohen, E.; Lyche, T.; Schumaker, L.L. Degree Raising for Splines. J. Approx. Theory
**1986**, 46, 170–181. [Google Scholar] [CrossRef] [Green Version] - Wang, G.; Chen, Q.; Zhou, M. NUAT B-spline Curves. Comput. Geom. Des.
**2004**, 21, 193–205. [Google Scholar] [CrossRef] - Murray, R.M.; Sastry, S.S.; Zexiang, L. A Mathematical Introduction to Robotic Manipulation, 1st ed.; CRC Press, Inc.: Boca Raton, FL, USA, 1994. [Google Scholar]
- Ritto-Corrêa, M.; Camotim, D. On the Differentiation of the Rodrigues Formula and Its Significance for the Vector-like Parameterization of Reissner–Simo Beam Theory. Int. J. Numer. Methods Eng.
**2002**, 55, 1005–1032. [Google Scholar] [CrossRef] - Hartley, R.; Zisserman, A. Multiple View Geometry in Computer Vision, 2nd ed.; Cambridge University Press: Cambridge, UK, 2004. [Google Scholar] [CrossRef]
- Helgason, S. Differential Geometry and Symmetric Spaces; Pure and Applied Mathematics; Elsevier Science: Amsterdam, The Netherlands, 1962. [Google Scholar]
- Hall, B. Lie Groups, Lie Algebras and Representations; Graduate Text in Mathematics; Springer: Berlin/Heidelberg, Germany, 2004. [Google Scholar]
- Gallego, G.; Yezzi, A. A Compact Formula for the Derivative of a 3-D Rotation in Exponential Coordinates. J. Math. Imaging Vis.
**2015**, 51, 378–384. [Google Scholar] [CrossRef] [Green Version] - TaS, Ö.S.; Salscheider, N.O.; Poggenhans, F.; Wirges, S.; Bandera, C.; Zofka, M.R.; Strauss, T.; Zöllner, J.M.; Stiller, C. Making Bertha Cooperate–Team AnnieWAY’s Entry to the 2016 Grand Cooperative Driving Challenge. IEEE Trans. Intell. Transp. Syst.
**2018**, 19, 1262–1276. [Google Scholar] [CrossRef] - Irie, K. A Graph Optimization Approach for Motion Estimation using Inertial Measurement Unit Data. Robomech J.
**2018**, 5. [Google Scholar] [CrossRef] - Strauß, T.; Ziegler, J.; Beck, J. Calibrating Multiple Cameras with Non-overlapping Views Using Coded Checkerboard Targets. In Proceedings of the 17th International IEEE Conference on Intelligent Transportation Systems (ITSC), Qingdao, China, 8–11 October 2014; pp. 2623–2628. [Google Scholar]
- Kümmerle, J.; Kühner, T. Fast and Precise Visual Rear Axle Calibration. In Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand, 27–30 October 2019; pp. 3942–3947. [Google Scholar]
- Kümmerle, J.; Kühner, T.; Lauer, M. Automatic Calibration of Multiple Cameras and Depth Sensors with a Spherical Target. In Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, 1–5 October 2018. [Google Scholar]
- Sons, M.; Lauer, M.; Keller, C.G.; Stiller, C. Mapping and Localization using Surround View. In Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV), Los Angeles, CA, USA, 11–14 June 2017; pp. 1158–1163. [Google Scholar] [CrossRef]

**Figure 2.**The vehicle driving track used for evaluation shown in Bing aerial imagery using the GNSS measurements for visualization.

**Figure 3.**Plotting of the visual localization poses (highlighted with blue color), motion fusion results using pose-graph-based approach and IMU pre-integration (highlighted with red color), and using our uniform B-spline-based approach (highlighted with green color) for comparison. (

**a**) The 3D position plotting. (

**b**) The 3D orientation plotting with roll, pitch, and yaw angles. (

**c**) The 2D position plotting with detailed zooming of one critical part. Here, we can see that the green curve is closer to the blue curve than the red curve, which means that our approach can estimate the motion fusion results more accurately.

**Figure 4.**Pose signals and pose difference signal details of the visual localization results (highlighted with blue color) and the fused results by applying our uniform B-spline-based motion fusion approach (highlighted with red color). (

**a**) The pose signals. (

**b**) The pose difference signals. From the plotting, by fusing the IMU measurements, the fusion results are smoother and more realistic.

**Figure 5.**Point cloud accumulation results. (

**a**,

**b**) are two example scenarios. These two cropped areas are shown in detail in the lower part of images using the green ellipse; (

**c**,

**d**) present the accumulated point cloud using linear interpolation between two visual localization system measurements; (

**e**,

**f**) show the accumulated point cloud using the fusion results from the pose-graph-based approach; (

**g**,

**h**) visualize the accumulated point cloud using our fusion results.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Hu, H.; Beck, J.; Lauer, M.; Stiller, C.
Continuous Fusion of Motion Data Using an Axis-Angle Rotation Representation with Uniform B-Spline. *Sensors* **2021**, *21*, 5004.
https://doi.org/10.3390/s21155004

**AMA Style**

Hu H, Beck J, Lauer M, Stiller C.
Continuous Fusion of Motion Data Using an Axis-Angle Rotation Representation with Uniform B-Spline. *Sensors*. 2021; 21(15):5004.
https://doi.org/10.3390/s21155004

**Chicago/Turabian Style**

Hu, Haohao, Johannes Beck, Martin Lauer, and Christoph Stiller.
2021. "Continuous Fusion of Motion Data Using an Axis-Angle Rotation Representation with Uniform B-Spline" *Sensors* 21, no. 15: 5004.
https://doi.org/10.3390/s21155004