Next Article in Journal
A Lightweight 1-D Convolution Augmented Transformer with Metric Learning for Hyperspectral Image Classification
Previous Article in Journal
Association between Oxygen Consumption and Surface Electromyographic Amplitude and Its Variation within Individual Calf Muscles during Walking at Various Speeds
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Comparison of the Performance of the Leap Motion ControllerTM with a Standard Marker-Based Motion Capture System

1
Optimization, Robotics and Biomechanics, Institute of Computer Engineering, Heidelberg University, 69120 Heidelberg, Germany
2
Canada Excellence Chair in Human-Centred Robotics and Machine Intelligence, University of Waterloo, Waterloo, ON N2L 3G1, Canada
*
Author to whom correspondence should be addressed.
Sensors 2021, 21(5), 1750; https://doi.org/10.3390/s21051750
Submission received: 30 December 2020 / Revised: 12 February 2021 / Accepted: 25 February 2021 / Published: 3 March 2021
(This article belongs to the Section Biomedical Sensors)

Abstract

:
Over the last few years, the Leap Motion Controller™ (LMC) has been increasingly used in clinical environments to track hand, wrist and forearm positions as an alternative to the gold-standard motion capture systems. Since the LMC is marker-less, portable, easy-to-use and low-cost, it is rapidly being adopted in healthcare services. This paper demonstrates the comparison of finger kinematic data between the LMC and a gold-standard marker-based motion capture system, Qualisys Track Manager (QTM). Both systems were time synchronised, and the participants performed abduction/adduction of the thumb and flexion/extension movements of all fingers. The LMC and QTM were compared in both static measuring finger segment lengths and dynamic flexion movements of all fingers. A Bland–Altman plot was used to demonstrate the performance of the LMC versus QTM with Pearson’s correlation (r) to demonstrate trends in the data. Only the proximal interphalangeal joint (PIP) joint of the middle and ring finger during flexion/extension demonstrated acceptable agreement (r = 0.9062; r = 0.8978), but with a high mean bias. In conclusion, the study shows that currently, the LMC is not suitable to replace gold-standard motion capture systems in clinical settings. Further studies should be conducted to validate the performance of the LMC as it is updated and upgraded.

1. Introduction

The Leap Motion Controller™ (LMC) device is a portable, low-cost marker-less motion capture system and has been progressively used to replace gold-standard marker-based motion capture systems, especially in clinical environments. The LMC is designed to track elbow, wrist and finger joint positions. It is used in a myriad of different scenarios, for example human-robot interaction [1,2,3], surgery training [4] and gesture recognition [5,6]. The LMC has also found application in aiding hand rehabilitation; for example, it has been used as a development and evaluation system for post-stroke patients [7] and used for video-game-based therapy in the upper extremities [8,9,10]. In another study [11], the LMC was used to investigate hand tremor, a common movement disorder, often found in patients with dystonia, Parkinson’s disease, and cerebellar ataxia. While the LMC has uses in various hand rehabilitation therapies, it is also used as a measurement device. The validation of such marker-less technology is therefore warranted because it has found application in such diverse areas. Over the years, a number of validation studies have been conducted, but with mixed results.
Initially, the LMC was validated for point or target-based implementations, as the device primarily was designed for gesture-based control. In this regard, endpoint tracking, via a reference pen, was used to validate the LMC with an accuracy of 0.2 mm. The study argued that an accuracy of <2.5 mm is possible with an average of 1.2 ± 0.7 mm [12]. One study performed a Fitts’ tapping task and reported high error rates, where the percentage of out-of-target selections, for the LMC, was 7.2% versus a standard mouse device with 2.3%. They also showed that the movement time was significantly longer with the LMC compared with the mouse movements (945 versus 565 ms). However, the study did not synchronise or simultaneously record the trials with the mouse and the LMC [13].
There were only a few studies that attempted to validate the LMC against standard marker-based motion capture systems. In one of the first studies, they restricted themselves to the wrist and the forearm movements [14]. The results showed that the LMC was unreliable for forearm pronation/supination movements. However, for wrist flexion/extension and deviation, it had acceptable performance, suitable within a 10% threshold region, as reported in studies with electronic goniometers [14,15,16]. In a separate study, the temporal characteristics were compared, where the LMC was deemed reliable for movement time and deceleration interval parameters. However, spatial measurement parameters such as endpoint accuracy, precision and maximum grip aperture were unreliable [17]. In another study, LMC when compared against the Optotrak motion capture system had a high correlation, 0.995 on the horizontal axis and 0.945 on the vertical axis, with the Optotrak. The finger accuracy of the LMC was 17.3 ± 9.56 mm versus the Optotrak [18]. None of these motion capture investigations focused on finger kinematics in any way.
To our knowledge, there have been only two validation studies in which the LMC was compared against goniometer measurements for finger kinematics. One study compared finger abduction and flexion of the Metacarpophalangeal (MCP) joints using both LMC and manual goniometers [19]. It was found that abduction angles between the index and middle finger were in good agreement; however, ring and the little finger abduction angles were less reliable. In the case of MCP flexion angles, only the middle finger showed acceptable agreement [19]. A more thorough study, with a larger sample size, was conducted, where it was demonstrated that the measurements between the LMC and the goniometers were not agreeable [20].
It is evident that the LMC is being used in a variety of applications; however, there is a lack of validation and replication studies with regards to clinical acceptance of this low-cost marker-less device. Specifically, there is a gap in the evaluation of the performance of the LMC with respect to finger kinematics, as well as a lack of synchronised validation of the LMC with a gold-standard motion capture system. To the best of our knowledge, this is the first validation study to synchronise a gold-standard marker-based motion capture system with the LMC with respect to finger kinematic assessment. In this paper, we perform both the static measurement of finger segment lengths and dynamic conditions measuring the joint angles, for all five fingers for the flexion and extension movements. The experiment was designed in a manner that can be reproduced in a clinical setting.

2. Materials and Methods

The subjects who volunteered for the this study were recruited from the university campus. They were deemed healthy and free from injury or neurological disorders. All subjects agreed and signed a written informed consent prior to any experiments. The experimental protocol and practices were performed with the complete approval of the Faculty of Behavioural and Cultural Studies, University of Heidelberg Ethics Committee.

2.1. Subject Information

Ten right-handed subjects (6 male and 4 female) of a mean age of 26.8 ± 4.56 years were recruited for this study. The experimental protocol was completed without incident, and all subjects successfully performed the assigned tasks.

2.2. Marker-Based Motion Capture Technology

Ten Oqus 500 motion capture cameras (Qualisys Track Manager (QTM); Qualisys, Gothenburg, Sweden), placed at appropriate locations, were used at a frame rate of 150 Hz. Each subject wore 24 retro-reflective markers of 4mm in size on their right hand (dominant hand), as shown in Figure 1a. These markers were carefully placed on the bony landmarks of the hand. The use of this marker set was previously validated in [21] with a mean repeatable accuracy of 5.1 . The motion capture system tracks these markers in the given XYZ space. The standard calibration practice, as detailed in the instruction manual, was used. A calibration was acceptable if the average residuals were 0.6mm, and data collection commenced only after a successful calibration.

2.3. Leap Motion Controller™

The LMC (Ultraleap, Bristol, United Kingdom) renders a skeletal model of the hand where the joint centres are similar to the markers placed for the QTM system, as depicted in Figure 1b. It is a marker-less, motion capture device, as shown in Figure 1c, which incorporates infrared and stereo vision technologies. The device weighs 32 g and is rectangular in shape with dimensions of 30 mm × 11.30 mm × 80 mm. It has three light emitting diodes (LEDs) spaced in between the two stereo cameras. The cameras are spaced 40 mm apart with a resolution of 640 × 240 pixels. It typically operates at 120 Hz. The data are streamed via a USB2.0/3.0 hybrid cable. The device works on Windows, Macintosh and Linux platforms. The controller tracks hands within a 3D interaction area that extends up to 60–80 cm from the device in a typical field of view of 140 × 120 degrees. The LMC generally is pre-calibrated; however, it was calibrated again with a calibration score of 95. A higher calibration score yields better stable measurements [22]. The LMC does not provide a depth scene of any kind or emit any form of structured light. A proprietary algorithm performs the calculation to obtain the position of the joints/objects from stereo-vision images. The LMC skeletal model is generated from a proprietary algorithm, which cannot be changed by a third party. This algorithm estimates the joint centre positions of the hand.

2.4. Data Collection

The two data acquisition systems were time synchronised because each system streams data with a different sampling frequency. So far, studies [12,14,20] have recorded data simultaneously with other measuring systems or have measured data independently. The LMC module has a constantly evolving Software Developing Kit (SDK) which currently provides support for Java, C and Python. Python was used as the programming language of choice. The data acquisition code along with the software synchronization of the LMC and QTM can be found via the link given the Data Availability Statement. The associated animations also capture the exact nature of the flexion movements.
All participants performed a static trial and a series of flexion/extension tasks. Instructions were prompted on the screen, and all tasks were randomised. Two trials per subject were conducted. In each trial, the subjects performed flexion/extension movements three times. Overall, there were 10 successful static trials, and out of 120 dynamic trials, 103 were successful. The unsuccessful trials were attributed to LMC failure to synchronize with the QTM system or the system failure of either the LMC or QTM.

2.5. Data Processing

The LMC is known for streaming data with a variable sampling frequency [14,17]. In order to overcome this problem, a software program was written in Python to synchronise the data acquisition of QTM and the LMC. However, upon initial review of the data, it was revealed that the LMC data streaming was variable, but the time taken to complete the trials closely matched the QTM system. The temporal alignment of the LMC will not influence the real-world angular accuracy, and analyses of the temporal precision are not the objective of this study.
Since frame per second data vary, a new time vector was calculated. For i samples, the time would be 1 / f i , and f i was estimated as the difference between samples i 1 and i. The new time vector would be the aggregation of the time intervals between 0 and i samples. The LMC data are interpolated over this new time vector, and the data are re-sampled at 150 Hz, matching the QTM data acquisition sampling rate.

2.6. Data Analysis

Figure 2 shows the geometrical diagram of a finger during a flexion/extension movement. The joint angles from the LMC and QTM were calculated from their recorded joint centre positions using the following set of equations:
θ M = cos 1 W M · M P W M · M P
θ P = cos 1 M P · P D M P · P D
θ D = cos 1 P D · D T P D · D T
where the operator . is the magnitude of the designated vector.
Calculation of the segment length was performed by taking the Euclidean distance of the joint centres of each phalange.
The Bland-Altman (BA) plot is a good method to determine the effectiveness of a new method/measurement to substitute an established or gold-standard method/measurement. In this paper, the LMC, a low-cost, portable hand tracking module, is evaluated against a more expensive, but gold-standard motion capture system, QTM. There have been other studies that have used a difference or the BA plot to show their results pertaining to the LMC versus the motion capture system [14,19,20].

3. Results

Initial Analysis

The study was divided into static and dynamic analysis. In the static case, the segment lengths of each subject were calculated and compared. The calculation method is described in Section 2.5. Note that the LMC estimates the joint centre position through a skeletal model that cannot be changed by a third party as access is not granted to make any changes to the algorithm. The QTM on the other hand tracks the markers placed on the bony landmarks of the hand. Therefore, the LMC measured joint positions are based on the accuracy of the inherent skeletal model of the device and not the camera of the system. This is in contrast to the QTM system, where each marker is visualised by at least three cameras. This makes the tracking in the captured volume more accurate relative to the laboratory origin.
In the dynamic case, the range of motion (ROM) of every movement per finger was analysed. A standard BA plot was used to compare the LMC measurements against the gold-standard QTM measurements. A trend of the results for each joint is shown along with the correlation coefficient. The bias, as well as the 95% confidence interval (CI) were also calculated to render a complete observation of the results.
Figure 3 shows the calculated segment lengths of ten subjects. The LMC data provide metacarpal base and height positions; however, for QTM measurements, the metacarpals were not considered apart from the thumb. The thumb has only three bones, metacarpal, proximal and distal phalanges, whereas the rest of the fingers have four. It is observed that each measuring system on its own showed little difference. However, the overall mean and standard deviation of the study are given in Table 1. The table shows that for the thumb, the LMC recordings were higher than QTM for all segments (distal, proximal and metacarpal). However, the QTM recordings were higher for the proximal and intermediate phalanges of the index, middle, ring and pinky fingers, respectively.
Figure 4 shows the flexion angles’ trajectories computed for the LMC and QTM systems for the MCP, Proximal interphalangeal joint (PIP) and Distal interphalangeal joint (DIP) joints of the index finger, respectively. For the MCP joint, in the case of QTM, the maximum angle measured was 78.01 and the minimum 9.62 , whereas, in the case of the LMC, 80.85 was the maximum and 9.69 the minimum flexion angle measured. For the PIP joint, QTM measured a maximum flexion angle of 73.80 and a minimum angle of 2.33 , whereas the LMC measured 83.40 as the maximum and 2.62 as the minimum flexion angle. For the DIP joint, the QTM measured a maximum flexion angle of 17.88 and a minimum of 1.63 , whereas the LMC measured 70.12 as the maximum and 1.68 as the minimum flexion angle. While for the MCP and PIP joints, the amplitudes of the trajectories from the two systems were similar, the LMC measured angle for the DIP joint was grossly over-estimated. The two systems where time synchronised by starting and ending the trials at the same time programmatically, the time difference between the two systems for this trial was 1.13 s.
Table 2 compares the range of motion calculated from the LMC and QTM measuring systems. It is seen that the difference in the calculated ROM for the interphalangeal joint (IP) joint and MCP during thumb flexion is high, 28.29 and 19.31 , respectively. The difference in the ROM of the PIP joint during flexion of the middle finger is the highest at 61.51 , which shows that the LMC grossly underestimated the flexion angles during this movement. The table shows that for thumb abduction and index finger flexion, the difference between the LMC and QTM was relatively low. However, in other cases, the ROM of the DIP joint for the ring finger and the PIP joint of the ring and pinky, the differences were relatively high.
The results shown in Figure 4 show the following:
(1)
The time recorded for the trials by the two systems varied. This shows the need to compare dynamic trials using techniques that will handle such time dependent data. In this paper, we used time normalisation to compare flexion trials between the LMC and QTM.
(2)
Table 2 shows that the ROM calculation for each joint during the flexion trials was quite varied, and this information is not enough to conclusively say that LMC performed better or worse than the QTM system. Therefore, subsequent results compared the LMC and QTM with the help of BA plots.
Figure 5A shows a BA plot for CMC joint with a negative bias of 28.14 with r = 0.1385 , P = 0.6088 . This shows that there is a weak correlation and an increase in the difference with the magnitude of the measurement. Figure 5B shows a positive bias of 7.72 with r = 0.0777 , P = 0.7747 of the MCP joint. Lastly, Figure 5C shows a negative bias of 4.94 with r = 0.1094 , P = 0.6865 of the IP joint. During the thumb abduction movement, none of the three thumb joints demonstrated a high correlation significance between the QTM and LMC measuring systems. It seems that for thumb abduction/adduction movement, there is no agreement for the joint angles of the MCP and IP joints.
Figure 6A shows a BA plot for the CMC joint with a negative bias of 31.31 with r = 0.0084 , P = 0.9735 . Figure 6B shows a positive bias of 9.19 with r = 0.6381 , P = 0.0043 for the MCP joint. Lastly, Figure 6C shows a positive bias of 2.01 with r = 0.3219 , P = 0.1926 for the IP joint. During the thumb flexion movement, only the flexion of the MP joint demonstrated the highest correlation significance between the QTM and LMC measuring systems. However, the CMC joint showed the least correlation.
Figure 7A shows a BA plot for the MCP joint with a negative bias of 1.25 with r = 0.3963 , P = 0.1152 . Figure 7B shows a negative bias of 1.1 with r = 0.1062 , P = 0.6894 for the PIP joint. Lastly, Figure 7C shows a positive bias of 0.1 with r = 0.0384 , P = 0.8835 for the DIP joint. During the index finger flexion movement, none of the joints demonstrated a positive correlation between the two measurement systems.
Figure 8A shows a BA plot for the MCP joint with a positive bias of 10.67 with r = 0.0116 , P = 0.9646 . Figure 8B shows a positive bias of 14.92 with r = 0.9062 , P < 0.001 fir the PIP joint. Lastly, Figure 8C shows a negative bias of 3.51 with r = 0.4352 , P = 0.0815 for the DIP joint. During the middle finger flexion movement, the PIP joint demonstrated the highest positive correlation between the two systems.
Figure 9A shows a BA plot for the MCP joint with a positive bias of 4.83 with r = 0.3679 , P = 0.1608 . Figure 9B shows a positive bias of 18.76 with r = 0.8978 , P < 0.001 for the PIP joint. Lastly, Figure 9C shows a positive bias of 2.67 with r = 0.4137 , P = 0.1111 for the DIP joint. During the ring finger flexion movement, all joints showed positive correlations, as well as positive bias; however, the highest correlation was observed for the PIP joint.
Figure 10A shows a BA plot for the MCP joint with a positive bias of 13.64 with r = 0.2378 , P = 0.3749 . Figure 10B shows a positive bias of 10.97 with r = 0.6676 , P = 0.0047 for the PIP joint. Lastly, Figure 10C shows a positive bias of 4.75 with r = 0.5736 , P = 0.0201 for the DIP joint. During the little finger flexion movement, all joints showed positive correlations, as well as positive bias; however, the maximum correlation was observed for the PIP joint.

4. Discussion

In earlier studies, regarding the accuracy of the LMC for static and dynamic tracking, a mannequin arm was used, where, for the static case, the standard deviation was less than 0.5 mm. However, with dynamic tracking, the LMC was inconsistent in performance, with the accuracy reduced by 5 mm [23]. The static results reported in this paper (Figure 3) are consistent with [23]. The time synchronisation revealed more stable data collection during the static case; however, there was still some variability for the dynamic trials.
There has not been any study so far that has validated the LMC against marker-based motion capture with respect to finger kinematics; however, there have been other measuring devices such as the goniometer [16,20]. The LMC was not in agreement with the goniometric readings for most of the joints with the exception of the wrist and the MCP of the index finger [20]. Overall, it was concluded that the LMC was not yet ready as a clinical measurement tool, and this conclusion matches with the overall findings of this paper.

4.1. Why Is a Trend Observed between Two Measurement Techniques?

In such comparative studies, it is possible to observe a clear trend where the difference increases with the increase in the magnitude of the parameter being measured, in this case the ROM. This consequently shows that the variances would be different, which was evident with the different biases reported for every joint with respect to their fingers. The results show that the standard deviation of the mean difference between the LMC and QTM is high, because the LMC measurement is lower than QTM for low values, as seen in the distal joints, and greater than QTM for high values, as seen in the MCP joints. This is the primary reason why a non-zero correlation between the difference and the mean was observed. However, the thumb MCP joint (Figure 5B), the thumb CMC joint (Figure 6A), the index finger DIP joint (Figure 7C) and the middle finger MCP joint (Figure 8A) showed a negligible correlation of little significance.

4.2. Repeatability

Repeatability is an important criterion in a study that involves method comparison, such as validation studies between the LMC and gold-standard motion capture technology. This is because the repeatability of two different methods of measurement has a probability to limit the amount of their agreement. If one of the methods has a poor repeatability, significant variation is expected in the same subject. To mitigate such a scenario in this study, the subjects were allowed to flex/extend three times to achieve the maximum functional ROM of their fingers. However, the LMC showed a considerable difference during these attempts, and there were instances when the trial failed because of synchronisation failure. Therefore, further trials with a larger participant cohort are necessary to take into account the repeatability criteria in order to conclusively validate the performance of the LMC with respect to standard motion capture technology.

4.3. Clinical Recommendation

In the static case, where the hand is poised over the LMC device with negligible movement, the LMC is able to measure the joint centre positions on each finger segment efficiently. The subsequent calculation of the segment length showed that the LMC is reliable with a low standard deviation, as shown in Figure 4. However, in the case of the dynamic condition, it was evident from the results that the LMC was not reliable enough to effectively track the movements of the fingers. The ring finger and the little finger showed weak correlations with QTM at all joint levels. Overall, the results for the thumb, index and middle finger were not promising either. Currently, the LMC cannot be recommended to monitor finger ROM in a clinical setting. However, from previous studies, it has been demonstrated that the LMC can be used to monitor wrist ROM, demonstrating acceptable accuracy within a 10% threshold, reported by goniometric measurements [14,15,16,20].
Before recommending the LMC, as an alternative to gold-standard motion capture or goniometers, further research is required to evaluate the performance of the device to effectively capture finger kinematic data in an unsupervised clinical environment.

4.4. Study Limitations

Our attempt to synchronise the start and end time of data acquisition was found to be suitable only for the static conditions, where the LMC performed well. However, for the dynamic conditions, we encountered various inherent problems that prevented a more reliable data capture. One of the main limiting factors was the variable frequency of the LMC during measurements as opposed to the constant frequency for QTM. The variable frequency was found to be mainly in dynamic trials because the total duration of the trial was longer than static trials. This affected the calculation of the flexion/extension and abduction angles for the finger joints. The LMC algorithm does not directly measure joint positions, but estimates them; however, it has been shown that the endpoint positions seem more reliable than joint centre positions [24]. It has been previously reported that the measurement of finger joint angles is susceptible to large variability [25]. Therefore, an acceptable clinical error needs to be established for joints of the fingers, as has been done for the wrist and forearm [15,16], and to determine an effective range.
The synchronisation program for the LMC and QTM was dependent on whether the two programs were run on two different machines. However, in this case, a loop-back interface was used to minimise the delay in the signal between the two data loggers. If the signal was weak or there was a momentary break, this would affect the data collection; hence, some of the trials were classed as unsuccessful because of synchronisation failure. The presence of retro-reflective markers on the subjects may have contributed to the unreliable LMC data even though the marker size was as small as possible to mitigate such a scenario. These markers are crucial for any standard motion capture system as they reflect infrared light. The LMC also emits infrared light; hence, this may cause disruption during the recordings.
In this study, we attempted to record the flexion/extension movements of our participants; however, the study protocol was restrictive. The participants were seated in a standard seat with an appropriate height and distance from a table. The LMC was fixed at an optimal reaching distance for each participant. Although the tasks were randomised, each participant followed a cue on a screen; however, the speed of the each movement was not fixed, and the height of the hand from the LMC varied within each trial. These variations were incorporated based on recommendations by a previous validation study, which focused on wrist and forearm movements [14].

4.5. Further Research

The purpose of this study was to validate the LMC data specifically for finger kinematics as this device is mainly used for gesture-based applications. Our experimental protocol included only healthy subjects, and therefore, before issuing specific clinical guidelines, further research is required to replicate this study in different pathological populations and conditions. This study is part of a larger investigation, where, grasping tasks are assessed using the low-cost LMC with clinical feasibility as the focus. We are currently in the process of evaluating the performance of the LMC in a clinical environment among tetraplegic subjects.
Finally, the LMC development team is constantly improving and producing new versions of their software. These developments contain a more robust computer vision software and tracking algorithms paired with virtual reality and haptics. Future studies from our group will focus on larger data sets with a specific clinical application study design, which will give more detailed information with respect to the use of the LMC as a low-cost replacement for clinical evaluation tools.

5. Conclusions

The LMC device has the potential to replace maker-based motion capture technology; however, it requires significant programming knowledge to extract data from the device for offline analysis. Along with this paper, we provide the software for the data extraction, which minimises the need for significant programming knowledge. The major problem with the LMC is the variable sampling frequency, which is yet to be resolved. Methods such as Kalman filtering and dynamic time warping may be useful to circumvent this drawback. Finally, we suggest that the LMC provides agreeable data in static conditions; however, for dynamic cases, further validation studies should be conducted before it can be recommended as a viable alternative for marker-based motion capture in clinical environments.

Author Contributions

Conceptualization, A.G. and K.M.; methodology, A.G.; software, A.G. and G.R.; formal analysis, A.G.; investigation, A.G.; resources, A.G. and G.R.; data curation, A.G. and G.R.; writing—original draft preparation, A.G.; writing—review and editing, A.G., G.R. and K.M.; visualization, A.G.; supervision, K.M.; validation, A.G. and G.R.; project administration, A.G. and K.M.; funding acquisition, K.M. All authors read and agreed to the published version of the manuscript.

Funding

This research study was funded by EIT Health under grant id 19340.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Ethics Committee of UNIVERSITY OF HEIDELBERG (AZ Mom 2016 1/2-A1, 2016 with amendment 2019).

Informed Consent Statement

Written informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Synchronisation code and raw data are available online at this repository https://github.com/Amartya32/Leap_and_Motion_Capture (accessed on 9 November 2020).

Acknowledgments

We would like to take this opportunity to thank all participants who volunteered for this study, as well as Kevin Stein, Elif Öztürk and Klaus Breuer for their valuable support during the experimental setup.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study, collection, analyses or interpretation of data; in the writing of the manuscript; nor in the decision to publish the results.

Abbreviations

The following abbreviations are used in this manuscript:
LMCLeap Motion Controller™
QTMQualisys Track Manager
BABland–Altman
MCPMetacarpophalangeal
PIPProximal interphalangeal joint
DIPDistal interphalangeal joint
CMCCarpometacarpal phalangeal joint
IPInterphalangeal joint
WMidpoint of the wrist joint
MMidpoint of MCP joint
PMidpoint PIP joint
DMidpoint of DIP joint
TTip of the finger
ROMRange of motion
CIConfidence interval
LOALimits of agreement

References

  1. Cheng, L.L.; Liu, H.B. Examples of Quadrocopter control on ROS. In Proceedings of the 2015 IEEE 9th International Conference on Anti-counterfeiting, Security, and Identification (ASID), Xiamen, China, 25–27 September 2015; pp. 92–96. [Google Scholar]
  2. Sarkar, A.; Patel, K.A.; Ram, R.G.; Capoor, G.K. Gesture control of drone using a motion controller. In Proceedings of the 2016 International Conference on Industrial Informatics and Computer Systems (CIICS), Dubai, United Arab Emirates, 13–15 March 2016; pp. 1–5. [Google Scholar]
  3. Fernandez, R.A.; Sanchez-Lopez, J.L.; Sampedro, C.; Bavle, H.; Molina, M.; Campoy, P. Natural user interfaces for human-drone multi-modal interaction. In Proceedings of the 2016 International Conference on Unmanned Aircraft Systems (ICUAS), Arlington, VA, USA, 7–10 June 2016; pp. 1013–1022. [Google Scholar]
  4. Partridge, R.W.; Brown, F.S.; Brennan, P.M.; Hennessey, I.A.; Hughes, M.A. The LEAPTM gesture interface device and take-home laparoscopic simulators: A study of construct and concurrent validity. Surg. Innov. 2016, 23, 70–77. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Carrieri, M.; Petracca, A.; Lancia, S.; Basso Moro, S.; Brigadoi, S.; Spezialetti, M.; Ferrari, M.; Placidi, G.; Quaresima, V. Prefrontal cortex activation upon a demanding virtual hand-controlled task: A new frontier for neuroergonomics. Front. Hum. Neurosci. 2016, 10, 53. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Naidu, C.; Ghotkar, A. Hand gesture recognition using leap motion controller. Int. J. Sci. Res. (IJSR) ISSN (Online) 2016, 5, 2319–7064. [Google Scholar]
  7. Kim, S.; Park, S.; Lee, O. Development of a Diagnosis and Evaluation System for Hemiplegic Patients Post-Stroke Based on Motion Recognition Tracking and Analysis of Wrist Joint Kinematics. Sensors 2020, 20, 4548. [Google Scholar] [CrossRef]
  8. Fernández-González, P.; Carratalá-Tejada, M.; Monge-Pereira, E.; Collado-Vázquez, S.; Baeza, P.S.H.; Cuesta-Gómez, A.; Cano-de la Cuerda, R. Leap motion controlled video game-based therapy for upper limb rehabilitation in patients with Parkinson’s disease: A feasibility study. J. Neuroeng. Rehabil. 2019, 16, 1–10. [Google Scholar] [CrossRef] [PubMed]
  9. Bostanci, H.; Emir, A.; Tarakci, D.; Tarakci, E. Video game-based therapy for the non-dominant hand improves manual skills and grip strength. Hand Surg. Rehabil. 2019, 39, 265–269. [Google Scholar] [CrossRef] [PubMed]
  10. Tarakci, E.; Arman, N.; Tarakci, D.; Kasapcopur, O. Leap Motion Controller–based training for upper extremity rehabilitation in children and adolescents with physical disabilities: A randomized controlled trial. J. Hand Ther. 2020, 33, 220–228. [Google Scholar] [CrossRef] [PubMed]
  11. Kincaid, C.J.; Vaterlaus, A.C.; Stanford, N.R.; Charles, S.K. Frequency response of the leap motion controller and its suitability for measuring tremor. Med. Eng. Phys. 2019, 63, 72–78. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  12. Weichert, F.; Bachmann, D.; Rudak, B.; Fisseler, D. Analysis of the accuracy and robustness of the leap motion controller. Sensors 2013, 13, 6380–6393. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Bachmann, D.; Weichert, F.; Rinkenauer, G. Evaluation of the leap motion controller as a new contact-free pointing device. Sensors 2015, 15, 214–233. [Google Scholar] [CrossRef] [Green Version]
  14. Smeragliuolo, A.H.; Hill, N.J.; Disla, L.; Putrino, D. Validation of the Leap Motion Controller using markered motion capture technology. J. Biomech. 2016, 49, 1742–1750. [Google Scholar] [CrossRef] [PubMed]
  15. Jonsson, P.; Johnson, P.W. Comparison of measurement accuracy between two types of wrist goniometer systems. Appl. Ergon. 2001, 32, 599–607. [Google Scholar] [CrossRef]
  16. Carter, T.I.; Pansy, B.; Wolff, A.L.; Hillstrom, H.J.; Backus, S.I.; Lenhoff, M.; Wolfe, S.W. Accuracy and reliability of three different techniques for manual goniometry for wrist motion: A cadaveric study. J. Hand Surg. 2009, 34, 1422–1428. [Google Scholar] [CrossRef] [PubMed]
  17. Niechwiej-Szwedo, E.; Gonzalez, D.; Nouredanesh, M.; Tung, J. Evaluation of the Leap Motion Controller during the performance of visually-guided upper limb movements. PLoS ONE 2018, 13, e0193639. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Tung, J.Y.; Lulic, T.; Gonzalez, D.A.; Tran, J.; Dickerson, C.R.; Roy, E.A. Evaluation of a portable markerless finger position capture device: Accuracy of the Leap Motion controller in healthy adults. Physiol. Meas. 2015, 36, 1025. [Google Scholar] [CrossRef] [PubMed]
  19. Trejo, R.L.; Vázquez, J.P.; Ramirez, M.L.; Corral, L.E.; Marquez, I.R. Hand goniometric measurements using leap motion. In Proceedings of the 2017 14th IEEE Annual Consumer Communications and Networking Conference (CCNC), Las Vegas, NV, USA, 8–11 January 2017; pp. 137–141. [Google Scholar]
  20. Nizamis, K.; Rijken, N.H.; Mendes, A.; Janssen, M.M.; Bergsma, A.; Koopman, B.F. A Novel Setup and Protocol to Measure the Range of Motion of the Wrist and the Hand. Sensors 2018, 18, 3230. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  21. Metcalf, C.D.; Notley, S.V.; Chappell, P.H.; Burridge, J.H.; Yule, V.T. Validation and application of a computational model for wrist and hand movements using surface markers. IEEE Trans. Biomed. Eng. 2008, 55, 1199–1210. [Google Scholar] [CrossRef]
  22. Leap Motion Controller Data Sheet. Available online: https://www.ultraleap.com/product/leap-motion-controller (accessed on 9 November 2020).
  23. Guzsvinecz, T.; Szucs, V.; Sik-Lanyi, C. Suitability of the Kinect sensor and Leap Motion controller—A literature review. Sensors 2019, 19, 1072. [Google Scholar] [CrossRef] [Green Version]
  24. Guna, J.; Jakus, G.; Pogačnik, M.; Tomažič, S.; Sodnik, J. An analysis of the precision and reliability of the leap motion sensor and its suitability for static and dynamic tracking. Sensors 2014, 14, 3702–3720. [Google Scholar] [CrossRef] [Green Version]
  25. Cook, J.R.; Baker, N.A.; Cham, R.; Hale, E.; Redfern, M.S. Measurements of wrist and finger postures: A comparison of goniometric and motion capture techniques. J. Appl. Biomech. 2007, 23, 70–78. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Figure 1. This figure shows the representation of the components used for the experiment. (a) Placement of the markers on the hand; (b) shows the skeletal hand model from the LMC; and (c) shows the LMC device with the axis orientation, which is the same for QTM.
Figure 1. This figure shows the representation of the components used for the experiment. (a) Placement of the markers on the hand; (b) shows the skeletal hand model from the LMC; and (c) shows the LMC device with the axis orientation, which is the same for QTM.
Sensors 21 01750 g001
Figure 2. Geometrical sketch of a finger from the wrist joint depicting joint angles within the XYZ coordinate frame. The +X axis is from the ulna to radial styloid, the +Z axis from the midpoint of the wrist joint W to the metacarpal joint M and the +Y axis is the common line perpendicular to the X and Z axes.
Figure 2. Geometrical sketch of a finger from the wrist joint depicting joint angles within the XYZ coordinate frame. The +X axis is from the ulna to radial styloid, the +Z axis from the midpoint of the wrist joint W to the metacarpal joint M and the +Y axis is the common line perpendicular to the X and Z axes.
Sensors 21 01750 g002
Figure 3. Segment lengths of the fingers of all ten subjects.
Figure 3. Segment lengths of the fingers of all ten subjects.
Sensors 21 01750 g003
Figure 4. An example comparing the flexion trajectories of the LMC and QTM. This example is of index flexion motion, showing the flexion angles of the MCP, PIP and DIP joints.
Figure 4. An example comparing the flexion trajectories of the LMC and QTM. This example is of index flexion motion, showing the flexion angles of the MCP, PIP and DIP joints.
Sensors 21 01750 g004
Figure 5. A BA plot of thumb abduction. (A) CMC joint, (B) MCP joint and (C) IP joint.
Figure 5. A BA plot of thumb abduction. (A) CMC joint, (B) MCP joint and (C) IP joint.
Sensors 21 01750 g005
Figure 6. A BA plot of thumb flexion. (A) CMC joint, (B) MCP joint and (C) IP joint.
Figure 6. A BA plot of thumb flexion. (A) CMC joint, (B) MCP joint and (C) IP joint.
Sensors 21 01750 g006
Figure 7. A BA plot of index finger flexion. (A) MCP joint, (B) PIP joint and (C) DIP joint.
Figure 7. A BA plot of index finger flexion. (A) MCP joint, (B) PIP joint and (C) DIP joint.
Sensors 21 01750 g007
Figure 8. A BA plot of middle finger flexion. (A) MCP joint comparison, (B) PIP joint and (C) DIP joint.
Figure 8. A BA plot of middle finger flexion. (A) MCP joint comparison, (B) PIP joint and (C) DIP joint.
Sensors 21 01750 g008
Figure 9. A BA plot of ring finger flexion. (A) MCP joint comparison, (B) PIP joint and (C) DIP joint.
Figure 9. A BA plot of ring finger flexion. (A) MCP joint comparison, (B) PIP joint and (C) DIP joint.
Sensors 21 01750 g009
Figure 10. A BA plot of pinky finger flexion. (A) MCP joint comparison, (B) PIP joint and (C) DIP joint.
Figure 10. A BA plot of pinky finger flexion. (A) MCP joint comparison, (B) PIP joint and (C) DIP joint.
Sensors 21 01750 g010
Table 1. Overall mean ± the s.d. of the segment lengths of ten subjects.
Table 1. Overall mean ± the s.d. of the segment lengths of ten subjects.
FingersSegmentLMC (mm)QTM (mm)
ThumbDistal22.68 ± 10.6119.27 ± 10.52
Proximal33.06 ± 10.7032.32 ± 11.65
Metacarpal48.40 ± 12.4138.12 ± 11.15
IndexDistal18.99 ± 10.2517.08 ± 9.72
Intermediate23.42 ± 10.6925.73 ± 12.14
Proximal39.17 ± 12.8644.89 ± 13.19
MiddleDistal18.21 ± 12.0817.71 ± 11.57
Intermediate27.58 ± 11.5130.21 ± 12.07
Proximal46.46 ± 13.2450.59 ± 13.07
RingDistal18.10 ± 10.8417.40 ± 10.75
Intermediate26.87 ± 10.5628.95 ± 10.94
Proximal43.33 ± 11.5944.90 ± 10.90
PinkyDistal16.67 ± 8.5016.69 ± 8.65
Intermediate18.96 ± 8.4119.58 ± 7.96
Proximal34.28 ± 2.7636.31 ± 3.58
Table 2. Functional range of motion of the fingers at every joint for all trials.
Table 2. Functional range of motion of the fingers at every joint for all trials.
FingersJointLMC ( )QTM ( )QTM-LMC ( )
Thumb AbductionIP31.2637.756.49
MCP41.3849.658.27
CMC54.1042.44–11.66
Thumb FlexionIP43.8672.1528.29
MCP43.1662.4719.31
CMC53.6645.73–7.93
IndexDIP68.7868.31–0.47
PIP81.0981.340.25
MCP107.8686.66–21.2
MiddleDIP33.8021.38–12.42
PIP43.59105.1061.51
MCP72.3692.6420.28
RingDIP47.4919.86–27.63
PIP69.55103.9434.39
MCP78.8983.394.5
PinkyDIP41.1650.329.16
PIP57.3594.6837.33
MCP78.0895.6117.53
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Ganguly, A.; Rashidi, G.; Mombaur, K. Comparison of the Performance of the Leap Motion ControllerTM with a Standard Marker-Based Motion Capture System. Sensors 2021, 21, 1750. https://doi.org/10.3390/s21051750

AMA Style

Ganguly A, Rashidi G, Mombaur K. Comparison of the Performance of the Leap Motion ControllerTM with a Standard Marker-Based Motion Capture System. Sensors. 2021; 21(5):1750. https://doi.org/10.3390/s21051750

Chicago/Turabian Style

Ganguly, Amartya, Gabriel Rashidi, and Katja Mombaur. 2021. "Comparison of the Performance of the Leap Motion ControllerTM with a Standard Marker-Based Motion Capture System" Sensors 21, no. 5: 1750. https://doi.org/10.3390/s21051750

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop