A Pilot Study on Mixed-Reality Approaches for Detecting Upper-Limb Dysfunction in Multiple Sclerosis: Insights on Cerebellar Tremor

Abstract

The assessment and rehabilitation of upper-limb functionality are crucial for addressing motor disorders in individuals with multiple sclerosis (PwMS). Traditional methods often lack the sensitivity to quantify subtle motor impairments, with cerebellar tremor diagnosis typically based on subjective visual inspections by clinicians. This study explored the feasibility of using Microsoft HoloLens2 for motion capture to assess upper-limb function in PwMS. Using the ROCKapp application, kinematic metrics such as movement quality and oculomotor coordination were recorded during pick-and-place tasks. Data from twelve healthy individuals served as benchmarks, while nine PwMS, including three with cerebellar tremor and one with ataxia, were tested to evaluate the tool’s diagnostic potential. Clustering algorithms applied to the kinematic data classified participants into distinct groups, showing that PwMS without cerebellar symptoms sometimes displayed behavior similar to healthy controls. However, those with cerebellar conditions, like tremor and ataxia, were more easily differentiated. While the HoloLens2 shows promise in detecting motor impairments, further refinement is required to improve sensitivity for those without overt cerebellar symptoms. Despite these challenges, this approach offers potential for personalized rehabilitation, providing detailed feedback that could improve interventions and enhance quality of life for PwMS. In conclusion, these findings highlight the potential of mixed-reality tools to refine diagnostic accuracy, suggesting future studies to validate their integration in clinical rehabilitation programs.

1. Introduction

Multiple sclerosis (MS) is a chronic autoimmune disease primarily affecting the central nervous system, including the brain and spinal cord [1]. It is commonly diagnosed in young adults aged 20 to 50, affecting around 2.8 million people globally [2], including 130,000 in Italy [3,4,5]. Symptoms like optic neuritis, muscle weakness, coordination issues, and cognitive impairments significantly impact quality of life [6,7,8]. Severe fatigue and motor dysfunctions such as spasticity and sensory disturbances, including numbness and pain, can make daily tasks difficult [1,9,10]. Visual issues like double vision also affect hand–eye coordination, further complicating daily activities [11]. Overall, the disease impacts physical, cognitive, sensory, and emotional aspects of daily life, emphasizing the need for interventions to improve independence and quality of life [12,13].

PwMS may exhibit cerebellar symptoms, including tremor and ataxia, with intention tremor, truncal ataxia, and nystagmus often indicating cerebellar dysfunction [14,15,16,17]. Cerebellar tremor, characterized by involuntary limb oscillations, affects 25% to 58% of PwMS and is distinct from essential and Parkinsonian tremors [18,19,20,21,22]. A prevalent subtype is intentional tremor, which worsens during targeted movements, linked to cerebellar lesions and impaired coordination [23,24,25]. Ataxia may manifest as dysmetria, dysdiadochokinesia, and nystagmus, often leading to an unstable gait [14,26]. Eye–hand coordination is particularly compromised, impacting motor tasks [27,28,29,30]. Cerebellar tremor and ataxia in PwMS highlight the need for comprehensive management approaches to address the significant disabilities they cause.

Impairments in upper-limb function affect daily activities, with occupational therapy focusing on functional recovery. Customized rehabilitation regimens are aimed at improving motor coordination and preserving functional autonomy in individuals with multiple sclerosis. The diagnosis of cerebellar conditions is currently operator-dependent as it is carried out through visual inspections [31,32,33,34].

In this context, cutting-edge technologies like motion capture systems and virtual reality platforms offer new ways to assess and rehabilitate motor abilities in objective ways. These tools offer immediate feedback and adaptive environments, potentially improving motor learning and compensatory techniques [35].

The integration of immersive technologies, such as virtual reality and mixed reality, enhances rehabilitation by creating stimulating environments, improving motivation and treatment outcomes [36,37,38,39,40]. Personalized feedback—visual, auditory, or tactile—is key to maintaining patient engagement, a critical factor in rehabilitation success [41,42,43]. More details are as follows:

• Virtual reality (VR) offers immersive, real-life scenarios for safe motor skills practice [44,45,46].
• Augmented reality (AR) overlays digital information onto real-world scenes, providing real-time guidance during exercises [47,48].
• Mixed reality (MR) combines AR with the physical world, allowing interaction with virtual objects for more engaging rehabilitation [49,50,51,52].

Another advantage of such technologies is their possibility to exploit cameras, sensors, and additional instruments for biometric measurements for the evaluation of both motor and hand–eye coordination abilities in people affected by a neurological disorder. Motion capture techniques have undergone substantial evolution, enabling the precise and detailed measurement of limb movements in various applications [53]. Recent advancements in AR and MR technologies have introduced head-mounted visors as a state-of-the-art tool for motion capture [54]. Devices like Microsoft HoloLens2 seamlessly integrate MR capabilities with advanced motion tracking functionalities, presenting a range of significant advantages [55,56]:

• Markerless Tracking: head-mounted visors utilize internal sensors and cameras to monitor upper-limb movements without the requirement for reflective markers, enhancing user comfort and operational ease.
• Enhanced Interaction: MR environments overlay digital data onto the real-world environment, facilitating interactive and immersive rehabilitation exercises and training scenarios. This feature enhances patient engagement and allows for dynamic therapy sessions.
• Portability and Convenience: these devices are compact and simple to deploy, making them adaptable for diverse settings, including clinical facilities and patients’ homes.

However, challenges like reduced tracking accuracy, a limited field of view, and battery life need to be addressed to maximize their impact in rehabilitation. In [55], Hololens2 and ROCKapp’s limitations are discussed extensively.

VR-based interventions show promising results specifically applied to PwMS’s rehabilitation, altering their sense of presence [40]. Saladino and colleagues [57] demonstrated advantages in boosting daily functioning and satisfaction, with respect to traditional therapy procedures. The use of serious games and gamification in VR-based rehabilitation helps patients engage in repetitive training with greater motivation, improving clinical adherence [58]. VR exergames are particularly effective in MS rehabilitation, especially for enhancing upper-limb movements [59,60,61]. Although XR technologies show less employment compared to VR solutions, they present promising results in term of engagement and rehabilitation outcomes [62]. Even fewer studies in the literature cover the usage of XR technologies as assessment and diagnostic tools for the quality of movement of upper-limb activity in PwMS.

Despite the growing body of research on rehabilitation methods for individuals with neurological disorders, current techniques often fail to provide precise feedback on motor and coordination deficits experienced by patients. MS is particularly challenging due to its complex symptoms affecting motor skills and hand–eye coordination. Traditional methods for assessing these impairments can be labor-intensive and lack the nuanced data needed for highly personalized rehabilitation strategies. Thus, integrating advanced motion capture systems and immersive technologies like AR and MR into rehabilitation could fill this gap by offering detailed and adaptive assessments. These technologies can provide clinicians with objective metrics that are essential for tailoring more effective interventions.

The primary objective of this research was to enable clinicians to make a detailed categorization between healthy individuals, PwMS, those with cerebellar tremor, and those with other motor impairment, considering the quality of upper-limb movement, thereby improving the understanding of upper-limb impairments and facilitating tailored interventions. Using the ROCKapp application for HoloLens2, previously introduced in [55], this study integrated holographic elements with physical objects to assess pick-and-place tasks.

Data from twelve healthy subjects were collected to calculate clinically relevant kinematic metrics—such as movement quality and coordination—which served as benchmarks for assessing upper-limb impairments in individuals with MS. Subsequently, nine PwMS (three with cerebellar tremor and one with ataxia) were recruited to evaluate the effectiveness of these metrics as both an assessment and diagnostic tool. A clustering algorithm was implemented to determine whether these metrics could accurately identify the severity of impairment among the participants, demonstrating the potential of this approach in clinical settings.

2. Materials and Methods

2.1. Participants

In this study, a population of healthy subjects (control group) was recruited to perform the kinematic acquisition of both hand and gaze displacements while performing a pick-and-place task with HoloLens2 (4 males, 8 females, average age of $42.15\pm 8.13$). Nine PwMS were recruited, including four males and five females, with an average age of $43.22\pm 13.71$ years. The PwMS group was characterized by different levels of disability both in the upper and lower limb. In particular, three individuals had been diagnosed with cerebellar tremor while one had been diagnosed with ataxia (

Table 1. Demographic and clinical characteristics of the study participants. Data are expressed as the study ID, sex, age, side (handedness), EDSS (Expanded Disability Status Scale), 9-HPT (nine-hole peg test), BBT (box-and-block test), and clinical cerebellar condition, if present. N/A is reported if the cerebellar condition is not present.

ID	Sex	Age	Side	EDSS	9-HPT	BBT	Cerebellar Condition
S1	M	29	R	1	20.43	66	N/A
S2	F	28	R	1.5	15.53	71	N/A
S3	F	49	R	4.5	42.28	31	Cerebellar tremor
S4	M	49	R	2	25.30	49	N/A
S5	F	38	L	6	24.70	42	Ataxia
S6	M	34	R	3.5	35.50	52	N/A
S7	M	35	R	1	17.60	59	N/A
S8	F	64	L	6.5	N/A	12	Cerebellar tremor (severe)
S9	F	63	L	6.5	66.34	33	Cerebellar tremor

).

All subjects were recruited according to Declaration of Helsinki and under the ENACT01 protocol (229/2022) approved by the Ethical Committee of Liguria Region (Italy) on 14 November 2022. The participants signed a written informed consent form after being introduced to the objectives of this study.

2.2. ROCKapp and Experimental Set-Up

In this study, ROCKapp (v1.0) [55]—an app developed for HoloLens2 from Microsoft (Redmond, Washington, USA)—was employed to provide a novel approach to functional MS assessment. This application integrates holographic elements with physical objects and markers to create interactive environments where PwMS can perform specific upper-limb tasks, such as pick-and-place exercises. The mixed-reality environment was constructed using Unity 2021.2.16f1, with the integration of both the Mixed Reality Toolkit 3 (MRTK3) and PTC Vuforia extensions. The MRTK package offers essential components for spatial interactions and user interface elements. This framework equips developers with the necessary APIs to utilize the user’s hands as interactive tools. It can accurately compute the position and orientation of each hand joint, including the fingers (with each phalanx), knuckles, palm, and wrist. PTC Vuforia, an augmented reality software, enables holographic interaction by exploiting image-based recognition algorithms. Upon the detection of specific image targets by the HoloLens2 camera, the program anchors holograms to these targets, enhancing the user experience.

For ROCKapp, a cylindrical image target representing a rocket was placed on top of a 500 mL plastic bottle (Figure 1). When the target was recognized by the HoloLens2 camera, the user could visualize the digital version of the same image on top of the physical marker. The bottle was chosen to be transparent to allow the optical hand-tracking system to see the hand even when the finger joints were occluded by the object during the grabbing phase.

Each participant was expected to move the object based on four different cues positioned as cardinal points: in front and close (S), in front and distant (N), on the left (W), and on the right (E). The virtual target N was placed in front of the user at a distance of the maximum arm extension. S, W, and E coincided with the vertices of a square with a side length of 28 cm and were automatically generated according to the chosen north coordinates. Position N served as a reference upon which the bottle had to be placed back on once moved to either S, W, or E. Please refer to

Table 2. The nomenclature of discrete movements to be performed within the game.

Movement	Label
North–South	NS
South–North	SN
North–East	NE
East–North	EN
North–West	NW
West–North	WN

for the nomenclature adopted in this manuscript.

The order of appearance of the cues S, E, and W was randomized (Figure 1). Placing the object on an activated target triggered the launch of the holographic rocket. This was possible by employing Unity Engine’s colliders. For additional details, please refer to the work of Bucchieri and colleagues [55]. The ROCKapp task was conducted in a seated position to accommodate individuals with significant lower-limb impairments.

Participants were instructed to repeat the task 30 times, distributed across 5 trials with 6 movements each. During runtime, hand- and eye-tracking data were recorded through HoloLens2 at a frequency of 50 Hz.

2.3. Experimental Protocol

Each participant of both the control and PwMS groups signed an informed consent form before proceeding with the experimentation. The PwMS previously underwent clinical evaluation to assess their level of motor and cognitive disability. The experimenter then started the application installed on HoloLens2 and, by using a custom menu, selected the side (right or left) on which the participant would carry out the pick-and-place tasks. The environment was manually placed by the experimenter in front of the participant so that the target N was placed at the subject’s arm length. The participants then donned the HoloLens2 device and performed ocular calibration. Ocular calibration is a built-in feature of Microsoft HoloLens2 and was conducted only once for each participant. This step was essential to enhance both the accuracy and reliability of clue placement and the assessment of eye–hand coordination.

Upon starting ROCKapp, the participants were asked to execute movements in a natural manner to simulate an activity of daily living. Moreover, they were required to keep their hand on the table, grasp the bottle, place it on a target (N, S, E, or W according to the movement to be performed), and place their hand back on the table. The participants were given the option to take breaks between repetitions to prevent the introduction of bias in movement execution caused by fatigue.

Following the completion of the session, the participants were requested to complete the NASA Task Load Index (NASA-TLX) [63], a widely used subjective questionnaire that measures perceived workload across six dimensions—mental demand, physical demand, temporal demand, performance, effort, and frustration—during the task performance (Figure 2).

Each experimental session lasted approximately 30 min, with a minimum 1 h interval between subjects for recharging the HoloLens2 battery.

2.4. Data Processing

Hand- and eye-tracking data were processed using Matlab 2021b. Kinematic data from HoloLens2, recorded relative to a world reference frame set up between the subjects’ eyes, were initially not directly comparable across subjects. To standardize the data, the following steps were taken:

• Coordinate System Adjustment: data were first roto-translated from HoloLens2’s left-handed coordinate system to a standard right-handed system.
• Centroid Calculation: the centroids of the target areas N and S were computed from the hand dataset by averaging the x- and y-coordinates.
• Reference System Alignment: the centroid of the target area N was translated to the origin (0, 0), and all other points were adjusted accordingly.
• Angle Correction: the inclination angle of the line between the centroids N and S relative to the y-axis was calculated, and a rigid rotation around the z-axis was applied to correct this inclination.

The absolute positions and velocities of the hand were computed for each participant. Before derivation, the positions were filtered with a fourth-order low-pass Butterworth filter at a cut-off frequency of 6 Hz, according to other upper-limb kinematics analysis found in the literature [64,65].

Outlines were exploited to isolate the pick-and-place movements (NS, SN, NE, EN, NW, WN) from the reach-to-grasp and back-to-resting-position phases. After a rough manual cut of the outlines, the starting and ending points of the interested movements were defined by setting a threshold of 5% of the maximum velocity in the velocity profile.

Some trajectories experienced data loss while recording with HoloLens2. For each subject, we calculated the percentage of loss of data points as

$$\mathrm{Data}\mathrm{loss}\%={\displaystyle \frac{\mathrm{Number}\mathrm{of}\mathrm{data}\mathrm{points}\mathrm{lost}}{\mathrm{Total}\mathrm{number}\mathrm{of}\mathrm{data}\mathrm{points}}}·\%$$

As Subject 2 (PwMS) experienced a percentage exceeding 80%, the participant was excluded from further analyses. For the same reason, Subject 5 (control group) was excluded.

Moreover, signals containing more than 25% points undetected by HoloLens2 were excluded to ensure the dataset accurately represented the performed movements, without significant distortions or artifacts introduced by interpolation. Not considering S2 (PwMS) and S14 (control group), 3.77% of the available dataset was discarded. Hand positions presenting lower than 25% data loss were interpolated with a 5th-order polynomial following the minimum jerk theory [54].

2.5. Kinematic Metrics

Five kinematic metrics and one proprioceptive were calculated to quantify the quality and efficiency of movement execution, providing valuable insights into the motor performance of PwMS during the rehabilitation exercises. Specifically, the metrics related to hand and eye displacements and velocities, focusing on smoothness, efficiency, planning, and hand–eye coordination. The kinematic metrics were carefully selected from a previous study in which their clinical relevance was investigated [64].

2.5.1. Smoothness

• Spatial Arc Length (SPARC): SPARC is defined as the arc length of the frequency spectrum derived from the Fourier transform of the velocity profile [65,66,67,68,69,70].
• Number of Velocity Peaks (NVP): The NVP denotes the number of submovements required to complete an action. Hand motion patterns showing multiple peaks in the velocity curve signify impair smoothness, while a bell-shaped velocity profile is characteristic of normal, healthy movement [68,69,70,71,72,73,74].

2.5.2. Efficiency

• Movement Time (MT): MT is defined as the duration of the movement from the moment the object is picked up to when it is placed on the target [69,70,74,75,76].

2.5.3. Planning

• Symmetry: Symmetry is defined as the ratio between the duration of the acceleration phase and the duration of the deceleration phase while performing a kinematic movement [77]. Clinically, high symmetry in movements indicates effective motor control and suggests the proper functioning of the patient’s neuromuscular system [78]. In contrast, movement asymmetry can signify motor impairments or compensatory strategies, potentially necessitating further therapeutic intervention.
• Kurtosis: Kurtosis offers insights into the distribution of velocity throughout a movement [77]. High kurtosis indicates that the movement features more frequent extreme values (peaks), while low kurtosis suggests a more consistent distribution of velocities. Clinically, kurtosis can aid in evaluating the smoothness and control of a patient’s movements. Higher kurtosis values may point to abrupt or jerky motions, which are often associated with motor control issues or neurological disorders. Conversely, lower kurtosis values suggest smoother and more controlled movements.

2.5.4. Hand–Eye Coordination

• Gaze Accuracy—Number of Zero Crossing Points (N0C): This a novel metric introduced in this manuscript to evaluate the hand–eye coordination ability of a person. The N0C, calculated as the distance between the eye path and the hand path instantaneously, refers to the measurement of how closely the gaze trajectory follows the hand trajectory at each moment in time. Particularly, the Number of Zero Crossing Points of the first derivative of the distance between the eye path and the hand path (N0C) is calculated. This approach provides a quantitative measure of how often the gaze trajectory intersects or deviates from the hand trajectory over time. By analyzing the derivative’s zero crossings, we can capture changes in gaze accuracy dynamically throughout the task or experimental session. Ideally, a low N0C would indicate healthy hand–eye coordination, suggesting that the person can smoothly execute the movement without frequent shifts in their gaze. Conversely, a high number may suggest uncertainty in movement execution, with the person needing to frequently shift their gaze between the hand and the task. Since this metric is task-dependent, the baseline was first computed using the control group and then compared to the recruited PwMS.

2.6. Metrics-Based Clustering: PwMS and Control Group

In this study, the K-means clustering algorithm was used to evaluate whether the proposed kinematic metrics could be utilized to classify PwMS according to the severity of their impairment. The K-means clustering algorithm is a widely utilized tool for classification tasks in research focused on motor-related studies. Its simplicity, efficiency, and ability to group data based on similarity make it a preferred choice for researchers aiming to identify distinct patterns or categories within complex datasets [79].

The analysis involved calculating indicators for both PwMS and the control group. Two different classification approaches were applied based on statistical analysis of the dataset and based on clinicians’ directives:

• Four-Class Classification Based on Clinical Evaluation: this approach identified four clusters, corresponding to the control group, PwMS, PwMS with intention tremor, and PwMS with ataxia.
• Six-Class Classification Based on the Silhouette Method tested on the complete dataset prior to classification: this method determined the optimal number of clusters to be six.

The kinematic metrics were pooled together regardless of the movement direction.

To ensure consistency across the dataset, each metric was normalized relative to the average metric calculated from the control group across all movements. This normalization step was essential for standardizing the dataset.

2.7. Statistical Analysis

A statistical analysis was performed to evaluate differences in kinematic behavior between the control group and PwMS without cerebellar symptoms (S1, S4, S6, and S7, as shown in Table 1). The hypothesis tested was that PwMS and the control group would exhibit statistically significant differences in kinematic behavior, specifically in terms of smoothness, efficiency, planning, and hand–eye coordination.

To test this hypothesis, the non-parametric Friedman test was employed with a significance level set at 0.05. If the test indicated significant differences, pairwise comparisons were conducted using the Mann–Whitney test to contrast the behavior of each PwMS with that of the control group. For these comparisons, the significance level was adjusted using the Bonferroni correction (α = 0.05/4 = 0.0125) [80].

An additional statistical test was performed to evaluate, if present, differences in the perceived workload between PwMS and the control group, based on the results of the NASA-TLX questionnaire. As the requested task was a pick-and-place one with limited cognitive and physical effort required, the hypothesis tested was that PwMS and the control group would not exhibit statistically significant differences in terms of mental demand, physical demand, temporal demand, performance, effort, and frustration. A pairwise Mann–Whitney test was employed to test this hypothesis, with a significance level, α, of 0.05.

3. Results

3.1. Clustering with Four Classes Based on Clinical Evaluation

The classification of the subjects’ conditions established by the test realized by clinicians consisted of four classes:

• PwMS affected by ataxia;
• PwMS affected by cerebellar tremor;
• PwMS with no cerebellar tremor;
• The control group.

The results of our analysis, based on a preliminary sample of subjects, showed that PwMS with no cerebellar conditions (S1, S4, S6, S7) were grouped together with the control group (green bars in Figure 3).

Subjects S3, S8, and S9, all affected by intention tremor, were expected to cluster together. This was true for S3 and S9, while S8 was classified separately (red bar). Finally, S5, who was affected by ataxia, was correctly classified in their own group.

Since some PwMS were grouped with the control group, a statistical analysis was conducted to assess the differences between these individuals (S1, S4, S6, S7) and the healthy group across various kinematic metrics. The Friedman test demonstrated statistically significant differences among all groups across each indicator, with p-values less than 0.001 for SPARC, the NVP, MT, kurtosis, and the N0C and a p-value of 0.011 for symmetry (

Table 3. Non-parametric Friedman tests (significance level: 0.05) and pairwise Mann–Whitney tests (significance level: 0.05/4 = 0.0125) performed only between the control group and each PwMS classified as in the control group from the K-means clustering (k = 4). * indicates statistical significance.

Metric	Friedman Test	S1	S4	S6	S7
SPARC	<0.001 *	0.03	0.63	0.002 *	0.06
NVP	<0.001 *	<0.001 *	0.057	0.91	<0.001 *
MT	<0.001 *	<0.001 *	<0.001 *	0.62	<0.001 *
SYMMETRY	0.011 *	0.008 *	0.11	0.318	0.034
KURTOSIS	<0.001 *	0.044	<0.001 *	0.33	0.06
N0C	<0.001 *	0.002 *	0.40	0.13	<0.001 *

).

These results suggest that the hypothesis that PwMS and the control group would exhibit different kinematic behavior in terms of movement smoothness, planning, and efficiency cannot be rejected.

The pairwise Mann–Whitney test (significance level: 0.0125) further highlighted significant differences between the control group and each PwMS (Table 3, Figure 4):

• S1 vs. Control: significant differences in four out of six metrics (NVP, MT, Symmetry, and N0C).
• S4 vs. Control: significant in two out of six metrics (MT and Kurtosis).
• S6 vs. Control: significant in one out of six metrics (SPARC).
• S7 vs. Control: significant in three out of six metrics (NVP, MT, and N0C).

Figure 4. Boxcharts of patients’ distributions classified with healthy subjects for the clusters number k = 4 and the control group distributions. Each graph represents one of the metrics considered for the classifier (SPARC, N0C, MT, SIMMETRY, KURTOSIS, NVP). Statistical significance is reported with a horizontal line and * symbol above.

Figure 4. Boxcharts of patients’ distributions classified with healthy subjects for the clusters number k = 4 and the control group distributions. Each graph represents one of the metrics considered for the classifier (SPARC, N0C, MT, SIMMETRY, KURTOSIS, NVP). Statistical significance is reported with a horizontal line and * symbol above.

3.2. Clustering with Six Classes Based on Silhouette Method

The K-means algorithm, applied to six classes, revealed that S4 and S6 were grouped with the control group, while S1 and S7 formed their own cluster. S5 and S8 remained in individual classifications, while S3 and S9 were assigned to separate clusters, indicating distinct kinematic behaviors between the two subjects affected by intention tremor (Figure 5).

3.3. Movement-Based Kinematic Evaluation of PwMS

In the previous section, metrics for each movement were grouped without distinguishing between trajectory directions. However, it may be useful to differentiate the kinematic motor abilities of PwMS.

For example, Figure 6 shows the average Spectral Arc Length metric (SPARC) for each PwMS (in red), superimposed onto the average for the control group (in blue). The x-axis shows different movement directions (NS, SN, NE, EN, NW, WN), while the y-axis shows the SPARC values. Similar patterns to that observed for SPARC were also present for the other kinematic metrics, which are omitted for brevity.

It is observed that for S1, S4, S6, and S7, who did not have tremor, the SPARC values are generally comparable to those of the control group for all movements. In contrast, S5, affected by ataxia, exhibited patterns similar to PwMS with intention tremor (S3, S9), who showed smooth movements in one direction (WN) but impairments in others. S8, with severe intention tremor, demonstrated significantly abnormal SPARC values around −4, indicating pronounced motor difficulties.

Figure 7 shows the value of the Number of Zero Crossings (N0C), a novel metric that highlights differences in hand–eye coordination between PwMS and control groups. While the controls showed low, stable N0C values across movement directions, PwMS exhibited significantly higher values, indicating greater coordination deviations, especially in participants like S5 and S8. S8 showed extreme difficulty, with N0C values exceeding 60. PwMS without cerebellar issues, such as S4 and S6, aligned more closely with the control group, unlike S1 and S7, who showed greater deviation. This metric provides clinicians with valuable insights to target specific movements and coordination challenges in therapy.

3.4. NASA-TLX Questionnaire

Figure 8 presents boxplots comparing subjective workload scores between PwMS and the control group across six dimensions of the NASA Task Load Index (NASA-TLX) questionnaire: mental demand, physical demand, temporal demand, performance, effort, and frustration.

In terms of mental demand , PwMS showed a slightly higher median compared to the control group, with both distributions being similar (p = 0.55). For physical demand, both groups exhibited low scores, though the control group showed a slightly higher median, with an outlier present in the PwMS group (p = 0.69). A significant difference was observed in temporal demand, where the Mann–Whitney test resulted in p = 0.0117 between the two groups. In performance, PwMS reported lower scores compared to the control group, although their distributions overlapped (p = 0.079). PwMS reported expending more effort than the control group, as reflected by a wider distribution and higher median. However, the two distributions did not present statistically significant differences (p = 0.63). Lastly, frustration levels were similar between the two groups, with PwMS showing a marginally higher median and fewer outliers in comparison to the control group (p = 0.81).

Considering these findings, the hypothesis that PwMS and the control group would show similar workload demand for ROCKapp is not rejected for all categories apart from temporal demand.

4. Discussion and Conclusions

This study introduces an innovative use of the Microsoft HoloLens2 device and the immersive application ROCKApp for the motion capture of upper-limb kinematics in people with multiple sclerosis (PwMS). Our method offers an objective and data-driven method for diagnosing motor impairments. This contrasts with current clinical approaches, which rely heavily on subjective assessments by clinicians [31]. The primary objective of this study was to evaluate the feasibility of using Microsoft HoloLens2 for calculating kinematic features that support the assessment and diagnosis of PwMS. Considering the varied severity of the disease among the PwMS recruited (Table 1), the K-means algorithm was initially tested with four classes, aiming to differentiate between PwMS, healthy individuals (control group), PwMS with intention tremor, and PwMS with ataxia. The clustering algorithm was applied based on six performance metrics related to movement smoothness (Spectral Arc Length, Number of Velocity Peaks), motor planning (kurtosis, symmetry), efficiency (movement time), and oculomotor coordination (Number of Zero Crossings). However, the clustering results revealed that the PwMS S1, S4, S6, and S7 were grouped with the control group, contradicting the expectation that PwMS without cerebellar conditions would display kinematic behavior distinct from healthy individuals (Figure 3). The Friedman test, applied to S1, S4, S6, S7, and the control group, indicated significant differences across multiple metrics (p < 0.001 for all but symmetry, where p = 0.011; Table 3), suggesting that some aspects of motor performance differed markedly between PwMS and healthy controls. Pairwise comparisons revealed that only in certain instances did subjects show significantly different kinematic behaviors compared to the control group (p < 0.0125), with S1 and S7 exhibiting at least three statistically significant differences out of six metrics. These findings suggest that HoloLens2, in combination with ROCKapp specifically, may not be suitable for distinguishing PwMS without cerebellar conditions from healthy individuals. PwMS with no cerebellar impairments may retain certain motor functions that are not detected by this tool. Clinical tests such as the box-and-block test (BBT) [81] and the nine-hole peg test (9-HPT) [82] may be more appropriate for identifying subtle motor impairments, likely due to the temporal constraints inherent in these tasks, which make kinematic deficiencies more evident. Regarding PwMS affected by cerebellar conditions, S3 and S9—both affected by intention tremor—were clustered together, affirming the sensitivity of the kinematic metrics in identifying motor deficiencies in individuals with this condition. S8, who was expected to cluster with the other tremor-affected subjects, was classified separately, which may point to the unique severity or manifestation of the tremor in this individual. The correct classification of S5 (ataxia) highlights the model’s ability to differentiate the nature of motor impairments.

In Figure 6 and Figure 7, we present a clear depiction of the average behavior of each PwMS (in red) compared to the control group (in blue) across various movement sets (NS, SN, NE, EN, NW, WN). This graphical representation allows clinicians to easily identify the following:

• The overall differences in motor abilities compared to the control group.
• The specific movements for which the subject may require focused rehabilitation.

For example, S1 and S7 showed deviations in the N0C compared to healthy participants, while S4 and S6 exhibited behavior more similar to the control group, with fewer gaze–hand coordination deviations.

The silhouette method identified six optimal clusters, with K-means clustering (k = 6) distinguishing S1 and S7 from S4 and S6, which resembled the control group. S5 and S8 formed separate clusters, while S3 and S9, both affected by intention tremor, were also classified separately. Despite similar box-and-block test (BBT) scores, S3 and S9 differed significantly in the nine-hole peg test (9-HPT), reflecting variations in dexterity and hand–eye coordination. Additionally, S3 exhibited cognitive impairments not present in S9.

In conclusion, this pilot study explored the potential of clustering algorithms to assess motor performance in PwMS and highlighted the utility of advanced kinematic metrics and mixed-reality assessments. The comparison between clustering with four and six classes underscored the importance of refining analytical approaches for evaluating motor impairments. Specifically, the initial K-means clustering with four classes demonstrated promise as a diagnostic tool for cerebellar conditions in PwMS by successfully differentiating individuals with intention tremor from those with ataxia and severe tremor. However, it failed to distinguish PwMS without cerebellar conditions from healthy individuals, as evidenced by the grouping of S1, S4, S6, and S7 with the control group.

The application of the silhouette method revealed that the six-class model more effectively captured the nuances in kinematic behavior, providing a more accurate representation of motor skill variability among PwMS. This refinement was particularly notable in distinguishing S1 and S7 from S4 and S6, as well as S3 and S9, who, despite similar box-and-block test (BBT) scores, exhibited significant differences in fine motor skills as measured by the nine-hole peg test (9-HPT). These findings align with prior studies—such as those by Carpinella et al. [83]—which have utilized inertial sensor signals to classify motor impairments and successfully differentiate PwMS with intention tremor from healthy individuals during tasks like finger-to-nose ones.

Nevertheless, the pilot nature of this study must be emphasized. While the findings suggest that advanced clustering techniques can enhance diagnostic capabilities, the results remain preliminary and require validation in larger cohorts. For example, S5, affected by ataxia, was successfully classified into a distinct group, indicating the tool’s potential to differentiate ataxia from intention tremor. However, this distinction remains speculative without further investigation and a larger sample size to confirm its diagnostic utility. Another critical consideration is the inherent challenge in distinguishing between overlapping motor conditions such as ataxia and intention tremor, which can lead to potential misdiagnoses [21,22]. Addressing this challenge would benefit from integrating evaluations based on activities of daily living (ADLs), offering a more comprehensive understanding of motor performance in real-world settings. Traditional clinical tests like the 9-HPT and BBT provide valuable insights, but advanced kinematic metrics add depth to the assessment of motor function, particularly for PwMS with cerebellar impairments. From a cognitive perspective, the use of ROCKapp and a mixed-reality solution was well received by both PwMS and the control group, with low perceived workload overall.

Interestingly, pairwise comparisons of workload metrics revealed no statistically significant differences, except for temporal demand, where the control group reported higher perceived pressure. This unexpected result suggests that the control group may have experienced a greater sense of urgency to complete tasks quickly, possibly due to competitiveness feelings, potentially influencing their perceived workload. In contrast, individuals with PwMS might have approached the test with a clinical mindset, prioritizing careful task execution over speed and exerting extra effort to perform the tasks thoroughly. Nevertheless, PwMS reported low cognitive demand, reinforcing the feasibility of employing HoloLens2 in rehabilitative settings.

Overall, MR, compared to VR, has proven to be beneficial in the rehabilitation setting as it provides more effective hand–eye coordination experience compared to typical applications with fully immersive or two-dimensional displays [84,85,86,87]. While rehabilitative applications of Microsoft HoloLens have been explored for recording kinematic data in upper-limb assessments (e.g., kinematic hand-tracking and hand–eye coordination) [88,89], these studies have not thoroughly investigated the potential of this technology for both the assessment and diagnosis of individuals with neurological disorders.

In summary, this exploratory study provides preliminary evidence supporting the utility of mixed-reality-based assessments and advanced clustering techniques for monitoring PwMS with diverse motor impairments. While the results are promising, further research with larger, more diverse cohorts is essential to validate these findings and to fully realize the potential of these tools in improving diagnosis, monitoring, and rehabilitation for PwMS.

This study highlights the potential of using Microsoft HoloLens2 and the ROCKapp application to assess upper-limb functionality in PwMS, particularly those with cerebellar conditions like tremor and ataxia. Kinematic metrics related to movement quality, coordination, and accuracy showed promise for identifying motor impairments in PwMS, although further refinement is necessary to improve sensitivity for those without overt cerebellar symptoms. The ability to differentiate between PwMS with cerebellar impairments and healthy controls supports the potential of this approach as a valuable tool for both assessment and diagnosis.

Despite its potential, the current study has limitations. The small sample size does not fully represent the variability in motor impairments experienced by PwMS. Future work should involve testing a larger population of both PwMS and healthy individuals to validate the generalizability of the results. The recruited sample size should encompass a diverse population, considering both cultural and environmental factors, to ensure representation of a broadly applicable, general population. Additionally, developing an automatic algorithm to identify pick-and-place tasks from other movement phases will streamline data analysis and improve real-time diagnostics. This could be useful to create a desktop application that allows clinicians to directly visualize kinematic data from HoloLens2, enhancing the practical utility of this tool in clinical settings.

By offering a low-workload, objective assessment tool, the proposed framework holds great promise for real-time clinical use, potentially revolutionizing how motor impairments are diagnosed and treated in PwMS.

Future developments should prioritize advancing MR-based applications to explore techniques for reducing intention tremor in affected individuals, with the proposed framework serving as a valuable tool for evaluating the effectiveness of these therapies. It would be valuable to explore additional tasks related to ADL (e.g., eating, cooking, self-hygiene) using our classification approach to further evaluate their respective effectiveness in highlighting differences in impairment levels. Such advancements have the potential to greatly enhance the quality of care and independence for people with multiple sclerosis.