1. Introduction
Stroke is the leading cause of disability in the world, with a very high social impact. Recovery is partial in 85% of stroke survivors [
1], 35% of which have a persisting serious disability. Rehabilitation programs are mainly focused on walking recovery, with insufficient attention being paid to upper limb recovery. Thirty percent to 60% of patients treated with conventional therapy still exhibit functional deficits of the paretic arm, resulting in a reduction of autonomy for daily-life activities, productivity and the ability to socially reintegrate [
2,
3]. Robot-mediated therapy for the recovery of the upper limb is gaining an increasing attention from clinicians and researchers, providing promising results [
4,
5,
6]. Recent studies suggest that robotics can facilitate recovery after stroke, promoting the mechanisms of brain plasticity and connectivity re-modulation [
7], according to the baseline cortical excitability [
8]. Several types of robotic and electromechanical systems, such as the exoskeleton or the end-effector [
9,
10], have been developed. Almost all scientific papers in the literature have focused on the effects of the use of one or, at most two, robotic devices. The anatomy, the kinematics and the motor function of the upper limb, especially the hand, are extremely complex; however, almost all commercial devices act on a limited number of joints and limit the workspace on a plane. Moreover, these commercial devices are often used in the research field rather than in clinical practice. For this reason, it is crucial to identify a set of robots and electromechanical systems, each one acting on a different joint and/or on a different plane, for a comprehensive upper limb rehabilitation (in all segments, including the hand), and verify the feasibility of their use in clinical practice.
In the light of the above, the aims of the study were: (1) to describe an approach to be used by a multidisciplinary team with the purpose of identifying a set of technological and robotic devices to treat the upper limb globally, and (2) to evaluate the feasibility of the identified set in clinical practice, with a pilot study on stroke patients.
2. Materials and Methods
The study has been conducted in two steps (
Figure 1): (1) the robotic device set’s identification and (2) the feasibility pilot study.
2.1. The Robotic Device Set’s Identification
In the first step of the study, a multidisciplinary team worked to identify a set of technological and robotic devices to treat the upper limb globally. The team was created in April 2015, at our institution (Fondazione Don Carlo Gnocchi, FDG) consisting of the medical director of FDG, 3 physiatrists, 2 neurologists, 3 physiotherapists, and 4 bioengineers, coordinated by the head of the Innovation and Health Technology Assessment department. The team, created with the intent of developing a strategy for the implementation of robotic rehabilitation within the FDG centers, performed an analysis of robotic solutions commercially available, to generate a prioritized list of solutions of potential interest for acquisition by the FDG. The team also had the objective of following the deployment of the robotic solutions in the various centers of our institution and promoting the use of such solutions. The group, involving five different centers of the FDG throughout Italy, assessed different devices. A consensus building process was used to create a specific form aimed at standardizing the description of the devices and allowing for an easier comparison between them (see the supporting document: FDG Robotic System Characteristics Form). The analyzed features included general information (commercial name, manufacturer, distributor, name of the compiler, and confidence level of the compiler), system characteristics (type of system, treated body segment, stage of development, type of movement, portability, type of assistance provided by the system, main control inputs, configurability, normative values, and outcome measures), the possibility for patients to use it with a wheelchair, safety issues, literature data, costs of purchase and maintenance, an indication of the purchasing priority (by motivations), and efficiency parameters (autonomous use by the patient, preparation time, possibility of using the solution in group therapies, and number of clinicians involved during treatment). These features were selected according to the clinical and rehabilitative needs; organizational aspects as well.
The team evaluated 10 different devices by filling out 40 forms (the same solution was evaluated by several people in the team). The evaluated solutions included two exoskeletons, four end-effector systems, two unweighting systems for the upper limb (one electromechanical and one spring based) and two sensorized technological systems (
Table 1). All the data were collected and the “qualitative” texting scales of the forms were transformed into “quantitative” ones (
Table 2). Then, the last three parameters were combined to evaluate the efficiency level (Equation (1)):
The efficiency level is an index representing the potential resource optimization, in terms of cost reduction, that could be obtained by using the robotic solution. In Equation (1) the efficiency level increases when the robotic solution can be used by the patient autonomously and if it is suitable for group therapies, while it becomes lower if it requires constant control by a physiotherapist. Similarly, the number of clinicians involved in the treatment also influence the efficiency level: if more than 1 clinician is needed to let the patient use robotic solution, the efficiency level decreases.
To rank the robotic solutions in terms of purchasing priority, the team defined an algorithm based on the weighted sum of the scores of the items in the evaluation form (Equation (2)):
where the parameters and corresponding weights are depicted in
Table 3. The score given to a robotic solution through Equation (2) takes into account the main characteristics of the device with a special focus on usability and sustainability in the framework of the FDG rehabilitation processes. According to the formula in Equation (1), the ideal robotic solution is a device at a high technology readiness level (TRL) that (i) is able to provide outcome measures that can be compared with reference normative values; (ii) has no safety issues and few contraindications; (iii) has strong scientific evidence supporting the efficacy; (iv) can be used even with patients with a high level of impairment; (v) gives the possibility of customizing the exercises; and (vi) can improve the efficiency of rehabilitation processes by optimizing the use of resources. The overall score given to a robotic solution is obtained by a weighted sum of the items listed above.
The choice of the parameters and weights was based on the priorities given by the case mix of FDG patients, sustainability of the robotic rehabilitation process and the need for innovative, technology-enabled solutions that could measure the outcome of the rehabilitation objectively. To take into account the knowledge of the evaluators, the average of the scores obtained by the same solution from different compilers was weighted according to their confidence level. Finally, the ranked device list obtained from this algorithm was integrated with the information of the cost of each device. The set of devices that was eventually defined was the one that had the highest average score among all the combinations giving the possibility of treating the upper limb globally (as defined by the physicians of the multidisciplinary team) and not exceeding a fixed budget. Such a budget was calculated considering the long-term sustainability of the rehabilitation services to be provided with the robots. It must be noted that the purpose of the method herein described was not to evaluate the quality of the robotic solutions per se, but rather to define a set of devices capable of responding to the clinical, organizational, and sustainability needs, as well as to the strategic objectives of our Institution. The set identified includes the following devices: Diego, Amadeo and Pablo (Tyromotion GmBH, Graz, Austria), and Motore (Humanware srl, Pisa, Italy). After the results of the set identification step, the identified robotic devices were installed in one center to run the feasibility pilot study.
2.2. Feasibility Pilot Study
In the second step of the study, 30 stroke patients were enrolled to compare the conventional therapeutic approach with a robotic approach, using the set of four devices identified, in the rehabilitation of upper limb.
2.2.1. Sample
We enrolled 30 consecutive patients after ischemic or hemorrhagic stroke, aged between 47 and 82 years (mean time since onset: 120 ± 46 days). Patients were recruited at the two centers of Don Gnocchi Foundation Onlus of Rome: (i) Santa Maria della Provvidenza, where the robotic/technological devices were installed (Robotic Center), and (ii) Santa Maria della Pace (Conventional Center). Inclusion criteria were: sub-acute patients (time latency since stroke ranging from two weeks to six months) after only one ischemic or hemorrhagic stroke, verified by MRI or CT; between 40 and 85 years old; time latency since the stroke between two weeks to six months; and the ability to understand simple instructions. Exclusion criteria included: fixed contraction deformity in the affected limb that would interfere with active therapy (ankylosis, Modified Ashworth Scale = 4) and severe deficits in visual acuity. Patients from the Robotic Center were treated by means of robotic devices for the upper limb (robotic group, RG, n = 16), while patients from the Conventional Center were treated according to conventional rehabilitation protocols for the upper limb (conventional group, CG, n = 14).
2.2.2. Clinical Evaluation and Instrumental Assessment
Patients in both the RG and CG were evaluated twice, at baseline (T0) and at the end of the rehabilitation program (T1). The clinical evaluations included scales for the upper limb function (the Fugl–Meyer [
11] and the Motricity Index [
12]), spasticity (the Modified Ashworth Scale for shoulder, elbow and wrist [
13]), lower limb performance (the Deambulation Index [
14]), and activities of daily living (the modified Barthel Index [
15]). The instrumental evaluations included the evaluation of the muscle strength (handgrip dynamometer) and the finger pinch (pinch gauge). In addition, the Visual Analogue Scale (VAS) for Satisfaction (a self-assessment scale from a 10-cm horizontal axis where 0 means “no satisfaction” and 10 “extreme satisfaction”) was used by patients to rate their satisfaction by making a vertical mark on the 10-cm line [
16]. The measurement in centimeters was converted to a number ranging from 0 to 10. The exact question was, “Are you satisfied with the robotic rehabilitation?”
2.2.3. Usability of the Set
To assess the subjective experiences of physiotherapists about the usability of the set, we used the System Usability Scale (SUS) [
17]. The SUS is based on 10 questions and has a 5-point scale (1 = strongly disagree and 5 = strongly agree), with a total score of the SUS ranging from 0 to 100. A score below 50 indicates usability difficulties and is, therefore, not acceptable; a score between 50 and 70 indicates marginal acceptability; a score above 70 indicates a good probability of acceptance; a score above 85 indicates excellent usability; and finally, a score above 90 indicates the best imaginable usability [
18].
2.2.4. Rehabilitation Treatments
Rehabilitation treatment, whether conventional or robotic, was performed daily for 45 min, for 5 days per week. A total of 30 sessions were performed. In the RG, during the rehabilitation session, both the distal and the proximal parts of the patient’s upper arm were treated by means of robotic and technological devices. A ratio of one therapist to three or four patients were used, depending on the patient’s severity, according to physician’s and physiotherapists’ opinions. During each session, the physiotherapist was able to use one or two systems for each patient, to minimize the time required to move the patients from one system to another. The rehabilitation program started with the robotic device for the shoulder and elbow joints, followed by the robotic device for the hand, the sensor-based device for the shoulder, elbow, and wrist, and finally, the electro-mechanical system for the shoulder. Therefore, during the 30-session treatment, the patient used all the identified devices. The adopted protocol followed general indications, in order to ensure the homogeneity of treatment; however, the physiotherapist selected and adapted the exercise to the patient’s residual ability. In the CG, a traditional approach was used. More detail on the rehabilitation treatments (both conventional and robotics) are reported elsewhere [
19]. All patients underwent a conventional treatment focused on balance, walking, and lower limb recovery with a ratio of one therapist to one patient.
2.3. Statistical Analysis
Data related to the pilot study are expressed as medians (range). Considering the ordinal nature of the variables and the small sample size, non-parametric tests were used. Specifically, to test for baseline difference, the two groups were compared by means of the Mann–Whitney U test. To assess the effects of the rehabilitation approaches separately (within-group analysis), data obtained at T0 and T1 in the two groups were compared by means of Wilcoxon signed rank tests. Finally, to compare the effects obtained in the two groups (between-group analysis), for each variable we computed the change score (T1–T0); then, we compared the change score obtained in the two groups by means of the Mann–Whitney U tests. For all the statistical analysis, a p-value of 0.05 was deemed significant. Statistical analysis was performed with SPSS (IBM, version 25).
4. Discussion
In recent years, there has been a significant increase in the number of available robotic systems for rehabilitation of the upper limb. Until 2014, 120 upper limb robotic systems were identified in a review [
20]. However, while numerous studies on the efficacy of these systems are available [
6], there is a lack of scientific data on the usability [
21], efficiency, and applicability in clinical practice [
22,
23]. Indeed, even when an intervention has good evidence of benefits, the application of evidence-based practice for stroke can still raise challenges [
24].
This problem is of particular importance when dealing with rehabilitation treatments based on technology and robotics. Therefore, clinicians specialized in the rehabilitation field should have tools to identify the appropriate type of technological system, according to the specific rehabilitation aims.
To the best of our knowledge, a standardized and detailed model to evaluate and compare these technological devices is not described in the scientific literature. Therefore, moving from the above-mentioned considerations, in this paper we suggest a methodological approach to choose a set of devices, both electromechanical and robotic, based on the rehabilitation aims, followed by a pilot study to evaluate the feasibility of the identified set of devices in clinical practice.
A multidisciplinary team from our institution discussed the clinical needs of upper limb rehabilitation; namely, the need to perform an intensive and a comprehensive treatment of the arm (including shoulder, elbow, wrist, and hand). Then, it analyzed some robotic solutions available commercially, in order to generate a prioritized list of devices of potential interest to be acquired by our institution (considering the rehabilitation aims discussed herein). A specific form was created to standardize the description of the robotic devices, and therefore, to allow an easier comparison between them. The analyzed features included general information, the system characteristics, the accessibility for wheelchair users, safety issues, literature data, costs of purchase and maintenance, an indication of purchasing priority by physicians and/or physiotherapists, and efficiency parameters. Ultimately, four devices were selected for intensive and comprehensive upper limb rehabilitation.
Our main goal was to implement, in a sustainable way, the robotic rehabilitation in our rehabilitation department, and therefore, we gave priority to the efficiency level (weight = 4). In this way, we were able to select a set of robots that: had a reduced set-up time, were accessible for wheelchair users, were easy to be autonomously used by the patient, were suitable for group therapy, and required a low number of physiotherapists to be involved in the treatment (i.e., more than one patient can be treated with the supervision of one physical therapist). Moreover, we had privileged robots able to provide outcome measures, in order to easily quantify the rehabilitation path of the patient and to modify it, accordingly. Finally, we selected a set of devices able to treat patients with different levels of upper limb impairment, from severe to mild. In the second phase, to support our institution in planning the most convenient investment strategy aimed to improve the clinical quality and management efficiency in seven of the 29 centers of our institution, a feasibility pilot study was performed. The purpose of this pilot clinical study was to verify the usability and applicability of these systems, according to organizational model of our institution, and to investigate the effects of the robotic rehabilitation (using, precisely, a set of different devices), before extending the set in other centers of our Institution.
The pilot study compared patients recruited in two FDG centers (one equipped with the set of devices and the other not equipped with it) and showed interesting results. Specifically, after rehabilitation, only the robotic group showed significant improvements in clinical functions of the upper limb (mainly in the elbow and wrist–hand, measured by the Fugl–Meyer and the Motricity Index) and instrumental outcome of the hand (obtained by dynamometer for the strength of the handgrip and with a pinch gauge for the fingered pinch). These results are consistent with those available in literature (see, for example, the recent meta-analysis by Mehrholz et al. [
6] or the review by Bertani et al. [
25]). We observed a significant reduction of the upper limb spasticity after treatment only in the control group. These data can be explained because of the different distribution of spasticity between the two groups (higher in the CG) at baseline. The pilot study allowed us to test the effects of the rehabilitation using a set of robotic systems, and therefore, the ability of our methodology to identify an effective set from a clinical perspective and rehabilitation aims. Note that the comparison between robotic and conventional treatment in this pilot study aimed to obtain preliminary clinical information before the introduction of the set in other centers of the Foundation. In addition, the pilot study showed that the treatment was well accepted by the patients; moreover, according to the practitioners (i.e., the physiotherapists involved in the robotic rehabilitation), the use of the identified set did not demonstrate major usability issues, and therefore, its degree of acceptance was high.
The choice to identify a set, rather than a single device, allowed us to treat patients with a high level of technology and with a global approach (with different devices acting on different joints), optimizing the available resources (one therapist treating three patients at the same time) [
26,
27,
28]. Due to the high cost of devices, it is important to select devices that are economically viable, to be used in clinical practice in a sustainable way. It is also worth noting that the selected devices are not noisy, and they did not require specific room features to be installed, although these characteristics were not explicitly evaluated by the form. We will consider both aspects in a future updated version of the form, together with the overall dimensions of the device. In fact, these features can be crucial, especially when the aim is to select a set of devices to be installed in a single room, with one physiotherapist supervising more than one patient.
The feasibility study, more than to evaluate the (expected) clinical effects of robotics, was aimed to investigate the possibility of using the set in clinical practice, an indispensable prerequisite for the implementation of robotics in routine care (applicability) [
24]. In that regard, a limitation is the lack of a longitudinal analysis of the cost-effectiveness of the implemented solution [
29]. A further limitation of this study is that it was carried out in only one center of our Institution (the coordination center of the robotic rehabilitation group); another is the low statistical power of the pilot study. However, it is worth noting that this pilot study has been followed by the introduction of the set in seven other centers of our Institution; and by a multicenter RCT, aimed to evaluate the efficacy of the upper limb robotic rehabilitation, compared to conventional treatment [
19], which confirms the feasibility and the usability of the selected set. Today, the robotic set is routinely used in these centers in clinical practice.
In our opinion, the methodology described above can be used as a model to select robotic devices suitable for different rehabilitation settings and aims (according to physician’s priority), using the same form but modifying the weight of each parameter. Moreover, the proposed approach, with very limited changes to the form, could be used to evaluate and identify devices; e.g., for walking or balance rehabilitation. Future studies should be performed to confirm these hypotheses.