A distinction is made between the joint angles determined with the Nimble VR software only (square markers) and joint angles determined with the Kalman filter (asterisk markers). The actual joint angles of the wooden hand model are represented by horizontal lines (MCP: dashed line; PIP: dash-dotted line). Measurements lying closer to these lines are by definition more accurate. In each plot, at the top left corner, two means and two standard deviations are shown per joint. The first mean and standard deviation are that of the Nimble VR joint angle measurements taken over the entire range, and the second mean and standard deviation are that of the Kalman filtered data. A mean that lies closer to the actual joint angle indicates an overall improvement in joint angle approximation accuracy, and a lower standard deviation indicates an improvement in precision. Lastly, a solid triangle marker on the horizontal axes was used to indicate that the difference between the mean joint angle of the PIP joint obtained with Nimble VR and the Kalman filter is not statistically significant. This same display-method was not used for the MCP joint, because for this joint the measured angles were the same in approximately 50% of the cases, thereby cluttering the graphs if we were to display this.

#### 4.1. Wooden Hand Model Orientation Measurements

The measured hand orientation angles are shown as a function of actual hand angle in

Figure 2a–c. There are several orientation ranges of the hand at which the measurements of the hand angles are imprecise. For the pitch orientation (

Figure 2a), all measured angles below −35 deg show very large standard deviations, that is, when the hand was pitched far downwards. The yaw angles (

Figure 2b) show high precision for the range [−50, 50] deg. At angles below −50 deg, when the thumb was angled away from the screen, the measurements show large standard deviations, whereas above 50 deg the standard deviation increases slightly. Lastly, for roll angles (

Figure 2c) in the range [−110, −60] deg, the measured angles have slightly larger standard deviations, which is because of visual occlusion of the fingers. At −90 deg, the hand is vertically aligned with the thumb on top. In this condition, the observed surface area of the hand is small, and only the thumb and index fingers can be distinguished by the Nimble VR software. As a result, in this range the orientation measurement becomes somewhat less reliable.

For the yaw measurements, a constant mean difference of about 9 deg is observed between the measured and actual yaw angle. Moreover, for the roll measurements a misalignment is seen at −30°, which is on account of the roll orientation having been measured in two separate sessions. A slight drift from actual the actual roll angle can be seen in the first range, where at −30 deg the measurement was stopped, the hand rotated 90 deg and reoriented, and the measurements (as well as the software) reinitialized. The re-measured roll angle of the hand is then free from drift and closer to the actual angle.

#### 4.2. Wooden Hand Model Finger Joint Measurements—Index and Thumb Fingers

At hand orientations yielding a low precision (SD > 5 deg, see

Figure 2, graphs a to c), a similar effect on precision can be seen for most of the finger joint angle estimates of the Nimble VR. The consequence of imprecise hand orientation measurements is either a decrease in finger joint angle estimation precision (

i.e., SD > 10 deg) or an unrealistically high precision (

i.e., SD < 1 deg) combined with a poor accuracy (> 30 deg shift from the true angle). This high precision is the result of visual self-occlusion of the finger, and the Nimble VR software accordingly making an estimation of the joint angles based on the estimated posture. This can for example clearly be seen in

Figure 2, graph f2, at angles −60 to −15 deg, where for the PIP joint a 50 deg difference is observed between measured and true angle (thus having a low accuracy), while the observed precision is around 1 deg. Due to the orientation independent standard deviation of the Data Glove, the Kalman filter output has a low standard deviation, even when the standard deviation of the Nimble VR data is high. Furthermore, because the Data Glove output is independent of the hand orientation, the Glove contributes to improved accuracy of the Kalman filter output over all the hand orientation ranges, in particular for the PIP joint. In order to assess both the accuracy and the precision before and after implementation of the filter,

Figure 2, subfigures d2-f4, should be referred to, as accuracy and precision are dependent on hand pose and assessed orientation range. The accuracy for a given joint, hand pose, or orientation range is equal to the mean joint angle (provided in the top left of every graph) minus the true joint angle. The precision is given by the standard deviations provided in the top left of every plot.

**Figure 2.**
(**a–c**) Three plots providing measured orientation of the wooden hand model for varying actual hand orientations (i.e., pitch, yaw, and roll), indicated by red dashed unity lines; (**d1–i1**) Photos of the measured hand orientations; (**d2–i4**) Plots showing the measured metacarpophalangeal joint (MCP, blue) and proximal interphalangeal (PIP, green) joint angles, as determined with the Nimble VR system (square markers, □) and after fusion of the Data Glove data through the application of the Kalman filter (asterisk markers, *). Data is presented with error bars ranging from mean – 1·SD to mean + 1·SD. The actual angles at which the fingers were placed are indicated with the red dotted and dash-dotted lines, and are illustrated in the photos provided on the left. All plots show data collected on the index finger, unless otherwise specified below the photo on the left. Indicated in the top left of every graph are the mean and standard deviation (format: mean ± SD | mean ± SD) calculated over the entire hand orientation range before (left) and after implementation of the Kalman filter (right), for both joints. Note that these mean and standard deviations are calculated as the mean of the mean, and the mean of the standard deviations, calculated per 5 deg step. The triangle markers on the horizontal axes indicate whether the difference in mean PIP joint angle between Nimble NR and the Kalman filter is not statistically significant (note: MCP joint is not visualised in this way).

**Figure 2.**
(**a–c**) Three plots providing measured orientation of the wooden hand model for varying actual hand orientations (i.e., pitch, yaw, and roll), indicated by red dashed unity lines; (**d1–i1**) Photos of the measured hand orientations; (**d2–i4**) Plots showing the measured metacarpophalangeal joint (MCP, blue) and proximal interphalangeal (PIP, green) joint angles, as determined with the Nimble VR system (square markers, □) and after fusion of the Data Glove data through the application of the Kalman filter (asterisk markers, *). Data is presented with error bars ranging from mean – 1·SD to mean + 1·SD. The actual angles at which the fingers were placed are indicated with the red dotted and dash-dotted lines, and are illustrated in the photos provided on the left. All plots show data collected on the index finger, unless otherwise specified below the photo on the left. Indicated in the top left of every graph are the mean and standard deviation (format: mean ± SD | mean ± SD) calculated over the entire hand orientation range before (left) and after implementation of the Kalman filter (right), for both joints. Note that these mean and standard deviations are calculated as the mean of the mean, and the mean of the standard deviations, calculated per 5 deg step. The triangle markers on the horizontal axes indicate whether the difference in mean PIP joint angle between Nimble NR and the Kalman filter is not statistically significant (note: MCP joint is not visualised in this way).

Looking at the index MCP joint, the two mean joint angles lie close together, and although an improvement in precision can be seen, only 49% of the time a significant difference between the measurement systems was observed, mostly at MCP joint differences larger than about 10 deg. The PIP joint is more affected by the filter, and in a substantial portion of cases (83%) a significant improvement was observed. As indicated by the triangular markers on the horizontal axes in (

Figure 2, graphs d2 to i4), no significant difference is present when the Nimble VR output overlap with the filter output, which occurs when the Nimble VR measurements already approach the true PIP joint angle. Moreover, at high standard deviations of Nimble VR data, statistically significant differences with the filter data are not always obtained.

Following, the separate hand postures will be discussed.

In the Flat Hand posture (

Figure 2, graphs d

_{1} to d

_{4}), with both MCP and PIP joint angles being 0 deg, the Nimble VR PIP joint estimate shows the poorest accuracy, especially at low hand pitch angles (graph d

_{2}). The Kalman filter output adjusts this and keeps both the MCP and PIP joint estimates around 0 deg, even in the ranges where hand orientation measurements are imprecise. This is most clearly shown by the decrease in standard deviation for both joints at graphs d

_{2} to d

_{4}.

At Pure PIP Flexion (

Figure 2, graphs e

_{1} to e

_{4}), where the MCP joint angles are kept at 0 deg and the PIP joint at 90 deg, one can see large fluctuations of accuracy in the PIP flexion angle estimate. For the pitch range (graph e

_{2}), in the region below −50 deg, the PIP angle is grossly underestimated, but for the remainder of the range, it is close to the actual angle. The Kalman filter output decreases the large variations over this range, keeping the joint estimate relatively accurate with some fluctuations around the actual PIP angle. For both the yaw (graph e

_{3}) and roll (graph e

_{4}) orientation ranges an improvement in precision and less variation in the accuracy can be seen. The MCP joint estimate deviates from the actual angle at high pitch angles (graph e

_{2}), but is relatively accurate for the other orientations (graphs e

_{3} and e

_{4}).

At Pure MCP Flexion, (

Figure 2, graphs f

_{1} to f

_{4}), with the PIP joint angles kept at 0 deg, a bias is seen during pitch (graph f

_{2}). As with pure PIP flexion, the PIP joint angle is wrongly estimated by the Nimble VR up until −10 deg, after which it correctly approaches the actual angle. The Kalman filter output adequately corrects for this bias, and keeps the estimated PIP joint angle around 0 deg at all angles. This comes, however, at the expense of the accuracy with which the MCP joint angle is estimated, which slightly worsens due to the Kalman filter. This is exemplified by the mean and standard deviations of the MCP joint angles taken over the entire range (see top left of graph f

_{2}), showing a slight increase in angle underestimation (

i.e., a lower accuracy) and an increase in standard deviation (

i.e., a lower precision). However, the reverse is thus true for the PIP joint. This same effect is seen to a lesser extent for the yaw and roll orientations (graphs f

_{3} and f

_{4}).

For Combined MCP and PIP Flexion

, (

Figure 2 graphs g

_{1} to g

_{4}), which is a more natural hand closure posture than the pure flexion of either the MCP or PIP joints, the advantage of using the Kalman filter is most pronounced in the PIP joint estimate. Where the Nimble VR measurements for this joint greatly vary for all orientations and are grossly overestimated, the Kalman filter yields a reliable and more accurate PIP joint angle estimate.

Lastly, for the Pinch Grip posture (

Figure 2, graphs h and i), we show both the index (graphs h

_{1} to h

_{4}) and thumb fingers (graphs i

_{1} to i

_{4}). Again, the Kalman filter increases the precision of the PIP joint output of the index finger (graphs h

_{2} to h

_{4}). However, there is a significant overestimation of the joint angle for all orientations. For the thumb (graphs i

_{2} to i

_{4}), the Kalman filter slightly increases precision for both joints estimates and slightly improves the MCP joint accuracy. However, the filter’s effect is less pronounced here as compared to the index finger.

#### 4.3. Wooden Hand Model Finger Joint Measurements—All Fingers

In order to assess the improvements gained for all fingers, in

Figure 3 the difference between the true joint angles and the mean joint angles taken over the range of all assessed hand orientation ranges are given. These differences are equal to the mean joint angle in the top left of every graph in

Figure 2 minus the true joint angle. The accompanying standard deviation is provided in

Figure 3 as well. Hence,

Figure 3 shows the mean accuracy and precision measures for all fingers, joints and poses, for the respective orientation range. It can be seen that for all fingers, the filter increases precision for the PIP joints and to a lesser extent for the MCP joints, regardless of hand posture. Accuracy improvements are seen for the PIP joints for the flat hand, pure PIP flexion, pure MCP flexion, and combined MCP and PIP flexion postures, but not for the pinch grip posture.

Lastly, we calculated the overall mean accuracy and precision improvements per joint gained by implementation of the filter. The overall accuracy and overall precision estimates were calculated across 2050 means and 2050 SDs, respectively (82 angles (19 pitch angles + 25 yaw angles + 38 roll angles) * 5 postures * 5 fingers). The results show that accuracy of the MCP slightly worsens by 6% from 12.7 deg (SD = 11.5 deg) to 13.5 deg (SD = 12.9 deg). This is offset by an accuracy improvement for the PIP joint of 31%, from 24.4 deg (SD = 17.4 deg) to 16.8 deg (SD = 15.7 deg). The precision of the MCP joint assessment improves with 5%, from 2.3 deg (SD = 2.5 deg) to 2.2 deg (SD = 2.2 deg), whereas the precision of the PIP joint improves with 79%, from 4.5 deg (SD = 4.1 deg) to 0.9 deg (SD = 1.1 deg). Overall, the filter thus marginally affects the MCP joint estimation, but strongly improves PIP joint estimation.

**Figure 3.**
Precision and accuracy for all fingers of the hand model and for all orientation ranges. (

**a**) Pitch range; (

**b**) Yaw range; (

**c**) Roll range. For each graph, the absolute difference is given between the mean calculated joint angle (

i.e., mean of the mean joint angles) and the true joint angle. The accompanying standard deviation is given as well (

i.e., the mean of the standard deviations at all angles), shown as ± 2·SD. Note that the values provided here are equal to the values given in the top left of the graphs in

Figure 2 minus the true joint angles of the assessed hand pose. A distinction is made between the Nimble VR data (left) and the Kalman filtered date (right), as well as between the metacarpophalangeal (MCP, blue) and proximal interphalangeal (PIP, green) joint. From left to right the fingers are presented; t = thumb, i = index, m = middle, r = ring, p = pinky.

**Figure 3.**
Precision and accuracy for all fingers of the hand model and for all orientation ranges. (

**a**) Pitch range; (

**b**) Yaw range; (

**c**) Roll range. For each graph, the absolute difference is given between the mean calculated joint angle (

i.e., mean of the mean joint angles) and the true joint angle. The accompanying standard deviation is given as well (

i.e., the mean of the standard deviations at all angles), shown as ± 2·SD. Note that the values provided here are equal to the values given in the top left of the graphs in

Figure 2 minus the true joint angles of the assessed hand pose. A distinction is made between the Nimble VR data (left) and the Kalman filtered date (right), as well as between the metacarpophalangeal (MCP, blue) and proximal interphalangeal (PIP, green) joint. From left to right the fingers are presented; t = thumb, i = index, m = middle, r = ring, p = pinky.

#### 4.4. Human Hand Active Finger Flexion Measurements

Dynamic flexing of the fingers while performing Marker Tracking of the joint angles and measuring the Nimble VR, Data Glove, and Kalman filter output yields the results shown in

Figure 4 and

Figure 5. In

Figure 4, one of five sessions are shown, for the three differently sized hands. Additionally, all fifty full finger flexions per hand as measured with the Nimble VR system and obtained through the Kalman filter are plotted

vs. the marker tracked angles. In

Figure 5 the MCP and PIP joints are shown separately as well in combination for 10 flexions performed by the medium sized hand (Hs = 0.84). Note that the Marker Tracking results are free from any time delay.

**Figure 4.**
Finger joint estimates at dynamic finger flexions. (**a1–a4**) small hand size; (**b1–b4**) medium hand size; (**c1–c4**) and large hand size. Five sessions with 10 full finger flexion repetitions each were performed. All graphs show the Nimble VR system (red dash dotted line), Kalman filter output (green dashed line), and Marker Tracking measurements (blue continuous line). Left three graphs show the combined metacarpophalangeal (MCP) and proximal interphalangeal (PIP) joint angles of the third performed measurement session. The remaining nine plots show the Nimble VR and Kalman filter output plotted vs. the Marker Tracked angles of all fifty finger flexions performed per hand. The black line presents the true angle, and the plots are given for separate and combined joints: (a2–c2) MCP; (a3–c3) PIP; (a4–c4) combined MCP plus PIP joints. In the top right, the Pearson correlation coefficient between the measured angle and marker tracked angle are given (red = Nimble VR, green = Kalman filter output).

**Figure 4.**
Finger joint estimates at dynamic finger flexions. (**a1–a4**) small hand size; (**b1–b4**) medium hand size; (**c1–c4**) and large hand size. Five sessions with 10 full finger flexion repetitions each were performed. All graphs show the Nimble VR system (red dash dotted line), Kalman filter output (green dashed line), and Marker Tracking measurements (blue continuous line). Left three graphs show the combined metacarpophalangeal (MCP) and proximal interphalangeal (PIP) joint angles of the third performed measurement session. The remaining nine plots show the Nimble VR and Kalman filter output plotted vs. the Marker Tracked angles of all fifty finger flexions performed per hand. The black line presents the true angle, and the plots are given for separate and combined joints: (a2–c2) MCP; (a3–c3) PIP; (a4–c4) combined MCP plus PIP joints. In the top right, the Pearson correlation coefficient between the measured angle and marker tracked angle are given (red = Nimble VR, green = Kalman filter output).

In

Figure 4, at the peaks of the graphs (

i.e., the points of maximum finger flexion), a noticeable effect can be seen of the hand size on the degree of under- or overestimation of the joint angles as compared to the Marker Tracking angles. The following presented under- or overestimation values have been calculated over all 50 flexions combined for each hand. At the small hand (graphs a1-a4) one can see that both the Nimble VR and the filter output underestimate the MCP joint angle considerably by 51 deg. However, this is compensated by an overestimation for the PIP joint (Nimble VR: 13 deg, SD = 17 deg; Kalman filter: 14 deg, SD = 11 deg), resulting in an overall underestimation of the full finger flexion by 38 deg (SD = 12 deg) with the Nimble VR system and 36 deg (SD = 8 deg) for the Kalman filter output. This underestimation of the MCP joint angle is less prominent at the medium sized hand (graphs b1-b4), where the Nimble VR underestimates the MCP joint with 15 deg (SD = 15 deg), overestimates the PIP joint with 28 deg (SD = 8 deg), leading to an overall overestimation of 13 deg (SD = 16 deg). For the medium sized hand, the output from the Kalman filter underestimates the MCP joint with 16 deg (SD = 15 deg) and overestimates the PIP joint with: 26 deg (SD =10 deg), adding up to a combined overestimation of 9 deg (SD = 9 deg). Lastly, for the large hand (graphs c1-c4), an underestimation is again seen for the MCP joint (filter: 23 deg, SD = 11 deg), but the filter output overestimates the PIP joint (filter: 27 deg, SD = 9 deg), leading to an overall small overestimation of 4 deg (SD = 12 deg) (as compared to the Nimble VR output providing an underestimation of 29 deg, SD = 20 deg).

Summarizing, the Kalman filter system underestimates the MCP joint angle (small, medium, large hand underestimation: 61%, 20%, and 50%, respectively) while the PIP joint is overestimated (small, medium, large hand overestimation: 18%, 34% and 28%, respectively). The combined finger flexion approximation is underestimated at the small hand (small hand: 22%), but marginally overestimated at medium and large hands (medium hand: 6%; large hand: 3%). The overall contribution of the Kalman filter compared to the Nimble VR data is relatively limited for the small and medium sized hands, providing a 1% and 3% reduction in under- and overestimation respectively. However, at the large hand Nimble VR data an underestimation of 20% is present, which after implementation of the filter changes to a small overestimation of 3%.

**Figure 5.**
Active index finger flexion comparison between Marker Tracking, Nimble VR, and Kalman filtered joint angles. Shown data are from the medium sized hand, first measurement session. (**a**) sum of the metacarpophalangeal (MCP) and proximal interphalangeal (PIP) joint angles plotted versus time; (**b**) MCP joint angles versus time; (**c**) PIP joint angles versus time.

**Figure 5.**
Active index finger flexion comparison between Marker Tracking, Nimble VR, and Kalman filtered joint angles. Shown data are from the medium sized hand, first measurement session. (**a**) sum of the metacarpophalangeal (MCP) and proximal interphalangeal (PIP) joint angles plotted versus time; (**b**) MCP joint angles versus time; (**c**) PIP joint angles versus time.

The correlation coefficients provided in the top right of the graphs in

Figure 4 indicate the degree of linearity in the datasets. At all hands and finger joints, a higher correlation coefficient was found in the Kalman filter output than for the Nimble VR. The correlation for the Kalman filter output is relatively low at the MCP joint of the small hand (

r = 0.64), and strongest at the PIP joint of the large hand (

r = 0.97).

The root-mean-square errors (RMSE) calculated of the Nimble VR and Kalman filter output with respect to the marker tracked angles for the small hand were 43 deg and 33 deg, for the medium hand 38 deg and 20 deg, and the large hand 39 deg and 16 deg, respectively. A substantial improvement is thus visible when using the Kalman filter. Important to note is that these RMSE values were calculated over the entire 50 hand flexures, where the maximum absolute error at a particular point in time was 167 deg for the Nimble VR system, and 158 deg for the Kalman filter.

Lastly, in

Figure 5, the discrepancy in time between the Nimble VR angle measurement and the Marker Tracking output is shown. A delay is present at both joints. Although for the Kalman filter output at the MCP joint the delay of 0.4 s persists (SD = 0.2 s), calculated over the 50 finger flexions of the medium sized hand, this effect is less pronounced at the PIP joint. For the PIP joint, the delay before and after implementation of the filter are 0.17 s (SD = 0.07 s) and 0.07 s (SD = 0.04 s), respectively. The resulting combined finger flexion estimation has a delay of 0.23 s (SD = 0.07 s) before implementation of the filter, and 0.12 s (SD = 0.03 s) after. Lastly, looking at the Nimble VR data at the PIP joint one can see that this joint is at times measured unsteadily, and that some erratic fluctuations occur, which are smoothed in the filter output.