Virtual Worlds

The advent of mixed reality (MR) systems has revolutionized human–computer interactions by seamlessly integrating virtual elements with the real world. Devices like the HoloLens 2 (HL2) enable intuitive, hands-free interactions through advanced hand-tracking technology, making them valuable in fields such as education, healthcare, engineering, and training simulations. However, despite the growing adoption of MR, there is a noticeable lack of comprehensive comparisons between the hand-tracking accuracy of the HL2 and high-precision benchmarks like motion capture systems. Such evaluations are essential to assess the reliability of MR interactions, identify potential tracking limitations, and improve the overall precision of hand-based input in immersive applications. This study aims to assess the accuracy of HL2 in tracking hand position and measuring kinematic hand parameters, including joint angles and lateral pinch span (distance between thumb and index fingertips), using its tracking data. To achieve this, the Vicon motion capture system (VM) was used as a gold-standard reference. Three tasks were designed: (1) finger tracing of a 2D pattern in 3D space, (2) grasping various common objects, and (3) lateral pinching of objects with varying sizes. Task 1 tests fingertip tracking, Task 2 evaluates joint angle accuracy, and Task 3 examines the accuracy of pinch span measurement. In all tasks, HL2 and VM simultaneously recorded hand positions and movements. The data captured in Task 1 were analyzed to evaluate HL2’s hand-tracking capabilities against VM. Finger rotation angles from Task 2 and lateral pinch span from Task 3 were then used to assess HL2’s accuracy compared to VM. The results indicate that the HL2 exhibits millimeter-level errors compared to Vicon’s tracking system in Task 1, spanning in a range from 2 mm to 4 mm, suggesting that HL2’s hand-tracking system demonstrates good accuracy. Additionally, the reconstructed grasping positions in Task 2 from both systems show a strong correlation and an average error of 5°, while in Task 3, the accuracy of the HL2 is comparable to that of VM, improving performance as the object thickness increases. Full article