Next Article in Journal
Reviewing Breakthroughs and Limitations of Implantable and External Medical Device Treatments for Spinal Cord Injury
Previous Article in Journal
Extreme Wind Speed Prediction Based on a Typhoon Straight-Line Path Model and the Monte Carlo Simulation Method: A Case for Guangzhou
Previous Article in Special Issue
The Effect of Sensory-Based Priming Using Repetitive Peripheral Magnetic Stimulation on Motor Skill Performance in Individuals with Stroke
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Applied Sciences
Volume 15
Issue 15
10.3390/app15158487
applsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Systematic Review

Innovative but Difficult to Replicate: A Systematic Review of the Reporting Quality of Robotic and Conventional Upper-Limb Interventions in Stroke Rehabilitation Randomized Controlled Trials Using the TIDieR-Rehab Checklist

by
Emeline Gomes
,
Gemma Alder
,
Kate Boardsworth
,
Kate L. Anderson
,
Sharon Olsen
and
Nada Signal
*
Faculty of Health and Environmental Sciences, Auckland University of Technology, Auckland 0627, New Zealand
*
Author to whom correspondence should be addressed.
Appl. Sci. 2025, 15(15), 8487; https://doi.org/10.3390/app15158487 (registering DOI)
Submission received: 29 May 2025 / Revised: 24 July 2025 / Accepted: 28 July 2025 / Published: 31 July 2025
(This article belongs to the Special Issue Current Advances in Rehabilitation Technology)
Browse Figures
Review Reports Versions Notes

Abstract

Background: Upper-limb impairment is a major cause of post-stroke disability, limiting participation in meaningful activities. Robotic rehabilitation may address this by delivering high-dosage, task-oriented training while reducing clinician workload. However, limited clinical translation of robotic interventions may be partly due to poor reporting in the literature. This systematic review evaluated the intervention-reporting quality (completeness and consistency) of randomized controlled trials (RCTs) comparing robotic and conventional upper-limb stroke rehabilitation. Methods: Four databases were searched for RCTs investigating robotic upper-limb interventions compared with dose-matched conventional interventions for people with stroke. Intervention reporting was assessed using the TIDieR-Rehab checklist. Trained reviewers independently extracted and evaluated data, resolving discrepancies through consensus. Completeness and consistency were analyzed descriptively. Results: Among 25 RCTs, the overall reporting completeness was low (43%). Robotic interventions were better described (50%) than conventional interventions (36%). While timing and total dose were commonly reported, critical details on provider expertise, active dose, progressive challenge, personalization, and harms were often omitted. Reporting consistency was moderate (68%), with key information dispersed across article sections. Conclusions: Inadequate reporting limits the transparency, replication, and implementation of robotic upper-limb interventions. Adopting structured reporting frameworks like TIDieR-Rehab is essential for advancing the field.

1. Introduction

Globally, an estimated 101 million people live with the effects of stroke, with approximately 80% of individuals experiencing upper-limb disability that profoundly impacts their participation in everyday activities [1,2]. Both clinicians and people with stroke recognize upper-limb rehabilitation as essential to promoting neuroplasticity and regaining independence [3,4]. However, healthcare systems often prioritize lower-limb rehabilitation to enhance the person’s mobility and expedite hospital discharge, deprioritizing or even neglecting the impact of upper-limb disability [3]. This disregard is concerning, as upper-limb rehabilitation is inherently complex, requiring specialized expertise and interdisciplinary collaboration to address the intricate sensorimotor, multi-joint, and bilateral arm functions involved in meaningful, everyday activities [3]. Even when upper-limb rehabilitation is prioritized, limited staffing and resource constraints often result in therapy dosages that fall short of the intensity needed to achieve clinically meaningful improvements [5,6]. To address these challenges, emerging technologies, such as robotic rehabilitation, are gaining attention as a potential solution to enhancing the accessibility and efficacy of upper-limb rehabilitation [7].
Robotic rehabilitation is advocated for its ability to personalize upper-limb rehabilitation tasks, promote active engagement, and deliver high dosages of task-specific training with reduced clinician involvement [8,9,10,11]. Meta-analyses suggest that robotic rehabilitation is potentially as effective, if not more effective, than conventional rehabilitation, for improving upper-limb capacity [10,11,12]. These effects may be attributed to specific robotic device features, such as the joints being mobilized, assistance and feedback type, portability, and laterality of the device [11]. Additionally, program parameters, including rehabilitation setting, clinician and peer involvement, number of devices used, type of training, and amount of practice, may also influence outcomes [10,11]. However, despite improvements in upper-limb capacity, robotic rehabilitation has not consistently translated to improvements in the performance of activities of daily living [8,10,11,12]. This raises questions about the mechanisms by which robotic rehabilitation contributes to improved upper-limb function and which device features and program parameters most effectively promote recovery [8,10,11,12].
Efforts to isolate the true effects of robotic upper-limb interventions and their conventional comparators are complicated by poor intervention reporting [8,11,12]. Key details regarding device features (e.g., assistance type) and program parameters (e.g., training amount, supervision level) are often poorly described or omitted from clinical trials [8,11,12]. Given the rapid evolution of this field, poor reporting is particularly problematic, as it perpetuates research waste [13]. Without complete and transparent reporting, person- and intervention-specific features that influence outcomes cannot be identified, compared, or optimized [13,14,15]. Moreover, clinicians cannot advocate for nor replicate robotic interventions if research does not provide sufficient guidance on their use [13,14,15]. These issues are exacerbated by the inconsistency of intervention reporting location, where key intervention details may be obscurely dispersed across various article sections (e.g., Introduction, Methods, Results, Figures, Appendix, Supplementary Files) and are not readily accessible [16]. Poor intervention reporting quality may significantly deter the optimization and clinical uptake of robotic upper-limb interventions in stroke rehabilitation [13,14,15,17,18].
Given the evidence supporting the efficacy of upper-limb robotics, its potential to augment or offer advantages over conventional rehabilitation, and the extensive time and cost being invested into optimizing their device and intervention design, it is crucial that the advancement of robotics is supported by comprehensive intervention reporting [8,11,12]. Addressing reporting challenges requires a systematic assessment of intervention reporting quality. The TIDieR-Rehab checklist, an extension of the original Template for Intervention Description and Replication (TIDieR) [16], was developed to evaluate the reporting of complex rehabilitation interventions and is well-suited to evaluating robotic interventions [15]. Therefore, this study used the TIDieR-Rehab checklist to evaluate the quality of intervention reporting in randomized controlled trials (RCTs) comparing robotic upper-limb interventions with conventional upper-limb interventions in stroke rehabilitation. Unlike previous reviews that focus on treatment effects [11,17,18,19], this study uniquely evaluates the completeness and consistency of intervention reporting to elucidate specific reporting gaps and potentially offer practical guidance to improve intervention reporting quality, and in turn, facilitate the advancement of future research and design, and promote clinical translation of robotic upper-limb interventions in stroke rehabilitation.

2. Methods

2.1. Study Design

This systematic review of intervention reporting was guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [20], is reported in accordance with the PRISMA-S checklist (refer to Supplementary File S1) [21], and was pre-registered in Open Science Framework (osf.io/ynm3r). It is informed by an earlier systematic review and meta-analysis by Boardsworth and colleagues [11] which, similar to other systematic reviews in the field [17,18], examined the efficacy of robotic upper-limb interventions in stroke rehabilitation but was significantly limited in its ability to analyze the influence of device and program features on outcomes due to poor intervention reporting in the included articles. To examine this further, the current study used the TIDieR-Rehab checklist to systematically evaluate the intervention reporting quality of robotic upper-limb stroke rehabilitation.

2.2. Data Selection

In accordance with the data selection process of Boardsworth et al. [11], this systematic review included RCTs that investigated the efficacy of robotic upper-limb interventions compared with dose-matched conventional upper-limb interventions, on upper-limb capacity or activities of daily living (ADLs) following stroke. To align the dataset with current expectations of intervention reporting in rehabilitation RCTs, two additional exclusion criteria were applied. First, the dataset excluded pilot or feasibility studies, which were defined as those with fewer than 30 participants and/or lacked power justification through sample size calculations [22]. While such studies offer valuable insights for the emerging field of robotic rehabilitation, they often have different reporting expectations and are not typically designed for intervention replication. Therefore, their inclusion could have skewed the results [23]. Second, articles published prior to 2015 were excluded as key intervention reporting guidelines, such as the original TIDieR checklist [16], were not yet available for adoption. These additional criteria ensured that included articles were fully powered RCTs published during a period reflecting contemporary standards for the transparency and replicability of intervention reporting.
Refer to Table 1 for the full eligibility criteria. Boardsworth and colleagues [11] conducted their initial search on 31 March 2023 with no start date limit. For the current study, all articles included in their systematic review and additional articles were identified by updating the search on 9 July 2024 (see Supplementary File S2 for the search strategy). One reviewer (EG) deduplicated and screened articles by title and abstract using Endnote, then manually conducted full-text screening to identify eligible articles. Two additional authors (GA and NS) reviewed and confirmed the eligibility of the final dataset.

2.3. Data Extraction and Evaluation

The TIDieR-Rehab checklist [15] is an extension of the original TIDieR tool specifically developed for the planning, reporting, synthesis, and critique of rehabilitation intervention studies. The TIDieR-Rehab checklist [15] was used to evaluate intervention reporting quality according to 22 items across the following sections: ‘Brief name’, ‘Why’, ‘Who’, ‘When’, ‘What’, ‘Who provided’, ‘How’, ‘Where’, ‘How much’, ‘How challenging’, ‘Regression/Progression’, ‘Personalisation’, ‘Protocol deviations’, ‘How well’, and ‘Harms’. Two reviewers (EG and KA) undertook data extraction and evaluation. One reviewer (EG), a physiotherapist and early career academic, had comprehensive knowledge of the TIDieR-Rehab checklist from their contribution to its development. The other reviewer (KA) was a senior neurological physiotherapist and clinical educator with extensive experience in stroke rehabilitation but had no previous experience with the TIDieR-Rehab checklist. Neither reviewer had been involved in the earlier systematic review and meta-analysis [11].
Prior to data extraction, the two reviewers completed structured training and piloting to build foundational knowledge, refine the methods, and establish standards of reporting specific to robotic and conventional upper-limb stroke rehabilitation interventions. In this training process, they familiarized with seven articles that had been excluded [24,25,26,27,28,29,30], mapping intervention descriptions against the TIDieR-Rehab checklist to clarify reporting expectations, identify common reporting gaps, and develop a table of reference (refer to Supplementary File S3) containing intervention-specific guidance and examples to support evaluation. Then, they independently piloted data extraction and evaluation using the table of reference on these seven excluded articles [24,25,26,27,28,29,30] and a further three articles that were included in the review [31,32,33], resolving discrepancies and refining the table of reference iteratively with another author (GA). This process ensured a shared understanding of reporting standards and readiness for systematic data extraction and evaluation.
Using the TIDieR-Rehab checklist and table of reference, the two reviewers independently evaluated the reporting quality of robotic and conventional upper-limb interventions within the included articles and their associated materials (e.g., Appendix, Supplementary Files, referenced protocols). Intervention information was extracted into a Microsoft Excel sheet, aligned with the relevant TIDieR-Rehab item, and labeled with the reporting location of each excerpt. Information pertaining to each TIDieR-Rehab item was assessed for two aspects of reporting quality: (i) completeness, and (ii) key location of reporting. Completeness was evaluated according to whether the intervention description pertaining to a TIDieR-Rehab item was fully described to a level that would allow for replication and was categorized as complete, incomplete, or absent. Key reporting location was evaluated by determining the article location (e.g., Introduction, Methods, Results, Figure, Appendix) where the most key intervention information for replication was reported; this was used to inform the analysis of consistency. Evaluations of completeness and consistency of intervention reporting are further defined in Table 2. Following independent evaluations, the two reviewers met to resolve disagreements. Any outstanding discrepancies were mediated by a third reviewer (GA) until consensus was reached.

2.4. Data Analysis

Inter-rater agreement was analyzed by percentage agreement and Cohen’s Kappa. An a priori guide was set to interpret the percentage of inter-rater agreement as acceptable (75% to 80%) or high (>80%) and Cohen’s Kappa agreement as poor (<0), slight (0–0.20), fair (0.21–0.40), moderate (0.41–0.60), substantial (0.61–0.80), or excellent (0.81–1.00) [34,35]. A sensitivity analysis was conducted to compare the inter-rater agreement with and without the piloting phase.
Completeness and consistency of reporting location were analyzed descriptively by percentages. For completeness, the percentage of complete, incomplete, and absent reporting was calculated separately for each TIDieR-Rehab checklist item and across all items. For consistency of reporting location, first, the most common key reporting location for each TIDieR-Rehab item across all interventions was identified. Then, the percentage of intervention groups that used this key location to report on the respective item was calculated. All calculations were performed separately for robotic interventions, conventional interventions, and all interventions combined, both for each individual article and across all articles. An a priori guide was set to separately interpret completeness and consistency of location as poor (<50%), moderate (50–79%), or excellent (≥80%) [36]. All data extraction, evaluation, comparisons, and analysis were conducted in Microsoft Excel.

3. Results

3.1. Article Selection and Characteristics

The article selection process is summarized in Figure 1. Updating Boardsworth and colleagues’ [11] search yielded 335 new results. After initial screening against their eligibility criteria, 11 new articles were added to their original 54 [11], yielding 65 potential articles for review. Application of our refined eligibility criteria relating to RCTs’ sample size and year of publication finally resulted in 25 articles being included in the current systematic review: 8 from the updated search and 17 from their original dataset.

3.2. Article Characteristics

Key intervention and participant characteristics of the included articles are provided in Supplementary File S4. The 25 included RCTs comprised 55 intervention groups, including four multi-arm trials [32,33,37,38]. In accordance with the study’s aim to evaluate the intervention reporting quality within and across interventions, which includes capturing differences between arms, each intervention arm was evaluated independently. Common reasons for exclusion were ineligible intervention types (e.g., socially assistive robots), outcomes limited to impairment-based measures, and ineligible study design and publication timeframe. Studies were conducted across a range of countries, including Italy, Korea, China, Taiwan, Japan, Singapore, India, Switzerland, France, the United States, and the United Kingdom and took place within a variety of settings, including inpatient and outpatient hospitals, rehabilitation centers, and research facilities. In total, 2218 participants with stroke were enrolled to receive robotic or conventional upper-limb rehabilitation. Both robotic and conventional interventions encompassed a range of rehabilitation approaches, including soft-tissue mobilization, active-assisted movement, repetitive practice, stretching and strengthening exercises, sensorimotor stimulation, and task-oriented and ADLs training.

3.3. Inter-Rater Agreement of TIDieR-Rehab Ratings

Across all completeness and key location ratings, the overall inter-rater agreement was acceptable at 77%. For completeness, the inter-rater agreement across the three categories (complete, incomplete, or absent) was 76% with a Cohen’s Kappa of 0.64, indicating acceptable and substantial agreement, respectively. When collapsed into binary categories (comparing complete versus incomplete/absent ratings and complete/incomplete versus absent ratings), a high percentage of agreement was achieved (86% and 87%) and substantial statistical agreement (0.70 and 0.73) was maintained. For consistency in key location ratings, 93% inter-rater agreement and 0.57 statistical agreement were achieved, indicating high and substantial agreement, respectively. Sensitivity analysis excluding the ratings from the three pilot articles maintained almost identical agreement for completeness (75%, 0.62) and was comparable for consistency of location ratings (93%, 0.65), indicating stability of inter-rater agreement across the pilot and review phases.

3.4. Quality of Intervention Reporting

Percentage of completeness and consistency of reporting location across both robotic and conventional interventions are presented in Supplementary File S5.

Completeness

Table 3 displays completeness of reporting for each of the 22 TIDieR-Rehab items by intervention group and article. On average, the completeness across robotic and conventional upper-limb interventions was poor (43%), with most articles (17/25) providing less than 50% complete reporting, and the remaining eight articles achieving moderate completeness (range of 50–59%). No articles reached excellent completeness (≥80%). On average, robotic interventions were reported more completely than conventional interventions, with robotic interventions meeting the threshold for moderate completeness (50%), whereas conventional interventions were poorly reported (36%). The only TIDieR-Rehab item for which the reporting completeness was higher in conventional interventions compared to robotic interventions was ‘How’ (48% vs. 32%).
Across both robotic and conventional upper-limb interventions, items with excellent completeness (≥80% of articles complete) included ‘Who’ (85%) and ‘When’ (100%). Most items pertaining to the intervention dose within the section ‘How much’ also displayed excellent completeness of reporting (‘Session duration’ 98%, ‘Frequency’ 87%, and ‘Intervention length’ 91%). However, ‘Essential elements amount’ was poorly reported (25%), with few studies providing metrics such as the number of repetitions or total active time participants engaged with the intervention. While more than half of the TIDieR-Rehab items (14/22) were poorly reported, eight items were particularly poor with <25% completeness. These items included: ‘Who provided’ (20%), ‘How challenging’ (9%), ‘Regression/Progression’ (9%), ‘Personalisation’ (‘Needs’ (7%) and ‘Preferences’ (0%)), ‘Protocol deviations’ (13%), ‘How well–Actual’ (22%), and ‘Harms–Plan’ (16%). Reporting on two of these items, ‘Who provided’ and ‘How challenging’, were most frequently rated incomplete (53% and 51%, respectively). The remaining six items were most frequently rated absent and together accounted for 56% of all absent ratings. Figure 2 displays completeness of reporting by TIDieR-Rehab item in robotic upper-limb interventions only.
Consistency of Reporting Location
Across all TIDieR-Rehab items and intervention groups, the Methods section was the most common location for reporting intervention features, accounting for 68% of all key reporting locations and achieving moderate consistency overall. By item, most TIDieR-Rehab items (18/22), including ‘Brief name’, ‘Who’, ‘When’, ‘Who provided’, ‘How much—Session duration’, ‘How much—Frequency’, ‘Personalisation—Needs’, and ‘How well—Plan’, were reported in the Methods ≥80% of the time, achieving excellent consistency across the robotic and conventional upper-limb rehabilitation groups. However, some items were primarily reported elsewhere. For example, the item ‘Why’ was commonly reported in the Introduction section (45%), ‘Protocol deviations’ in Referenced Articles (33%) such as protocol papers and ‘How well—Actual’ and ‘Harms—Actual’ in the Results (44% and 65% respectively). When combined, ‘Other’ reporting locations, including the Abstract, Figures, Appendices, Supplementary Files, and Referenced Articles, accounted for 21% of all key reporting locations. In fact, by item, ‘Where’, ‘Regression/Progression’, and ‘Harms—Plan’ were frequently reported in these ‘Other’ locations (41%, 38%, and 45%, respectively). Figure 3 illustrates the range of key reporting locations identified by each TIDieR-Rehab item for robotic interventions only.

4. Discussion

This is the first systematic review to evaluate the reporting quality of robotic upper-limb interventions in stroke rehabilitation research. Although the evidence supporting the efficacy of upper-limb robotics is rapidly advancing, with increasing calls for their implementation in stroke rehabilitation practice [8,10,11,58], our findings reveal substantial inadequacies in the quality of their description in RCTs. The consistency of reporting location was moderate, with intervention descriptions commonly dispersed across article sections and ‘Other’ locations. Completeness of reporting across robotic and conventional upper-limb interventions was poor, with gaps in critical areas, including intervention supervision (‘Who provided’), planned and actual dosage (‘How much—Essential elements amount’, ‘How challenging’, ‘Regression/Progression’, ‘How well’), personalization to promote person-centered care (‘Personalisation’), and safety (‘Protocol deviations’, ‘Harms’). Given that robotics are promoted for their utility in reducing clinician workload, increasing the amount of practice, and tailoring the intervention to the individual [8,10,11,58], these shortcomings have significant implications for the replicability, optimization, and clinical advancement of upper-limb robotics in stroke rehabilitation.
A notable limitation in the reporting of robotic upper-limb interventions was the inadequate description of intervention providers and the delivery mode (e.g., face to face, in a group), with only 21% and 32% of articles specifying ‘Who provided’ and ‘How’, respectively. While conventional interventions alluded to clinician-led delivery, robotic interventions rarely described the expertise or training of intervention providers and failed to distinguish the level of supervision or input provided during robotic rehabilitation. This is particularly concerning given the critical role that clinicians may play in selecting robotic tasks, defining dosage parameters, adapting challenge, providing personalized feedback, ensuring safety, and building a therapeutic relationship with the person throughout rehabilitation [59,60,61,62]. Alternatively, some sophisticated robotic systems are capable of partially or fully automating these functions based on user performance [12,61,63]. Such advancements have the potential to streamline clinician workloads, reduce reliance on 1:1 supervision from highly skilled clinicians, and enable more time- and cost-effective models of intervention delivery [12,61,63]. Indeed, a core justification for robotic rehabilitation is the potential to reduce staffing demands by facilitating broader support from a diverse range of supervisors, including trained therapy assistants, family members and caregivers, and the facilitation of self-directed practice [57,64]. Poor reporting of ‘Who provided’ and ‘How’ makes it unclear how this supervision may occur, and impossible to evaluate how effectively automated robotic systems perform these functions compared to skilled clinicians or how different supervision models influence rehabilitation outcomes. This lack of transparency obscures the nuanced interplay between technological and clinical oversight [8,11,12]. In an era where health systems are looking toward technology to alleviate resource constraints [60,61], the comprehensive reporting of intervention providers is essential to leveraging upper-limb robotics to enhance the capacity and efficacy of stroke rehabilitation.
Underpinning neurorehabilitation and the rationale for upper-limb robotics is the recognition that dose or amount of practice is a pivotal driver of motor learning and recovery [65]. Robotics are widely promoted for their ability to deliver high repetitions, reduce inactive time, and sustain patient engagement through mechanisms like assisted practice, gamification, and real-time feedback [59,63,66]. Although this review found that planned ‘Session duration’, ‘Frequency’, and ‘Intervention length’ were well reported (100%, 89%, and 93%, respectively) in robotic interventions, this may be attributed to our study’s inclusion criteria, which are based on the earlier systematic review and meta-analysis [11] and required adequate reporting to determine if the interventions were dose-matched. In contrast, the plan for how much time and/or repetitions would be spent actively engaging with core components of the robotic intervention (‘How much—Essential elements amount’, 32%) was largely absent. Robotic systems are capable of monitoring and quantifying the amount of practice within a session, offering a unique opportunity to measure the actual time spent engaging in the intervention, detect differences between planned and actual dose, and compare between robotic and conventional interventions [67]. Yet, across both interventions, the plan to capture this detail (‘How well—Plan’, 27%) and the actual amount of rehabilitation undertaken (‘How well—Actual’, 22%) were also poorly documented. Given that the efficacy of these interventions is likely contingent on maximizing active practice time, the ability to accurately quantify the dose and separate this from inactive time (e.g., setting up the device, giving instructions) is critical to investigating and optimizing rehabilitation dose–response relationships [11,14,67]. This underscores the importance of comprehensive documentation and reporting on both the planned and actual dose, including active and inactive time, to enhance intervention replication, evaluation, and outcomes.
Another critical consideration in upper-limb stroke rehabilitation is the ability to determine and systematically progress challenge for the individual [63,65]. Maintaining an optimal level of challenge over the course of rehabilitation is vital to ensuring engaging, rewarding, and effective rehabilitation [63,67]. Yet, the methods for setting and monitoring challenge (‘How challenging’) was 51% incomplete, and the progression of dosage (‘Regression/Progression’) was 62% absent across robotic and conventional interventions. This could indicate either a limited understanding of the use of challenge in rehabilitation or an assumption that it is self-evident within practice [62]. In conventional rehabilitation, task challenge was often described as being set and progressed based on ‘therapist judgement’ [38,48,50,51,52,54,55]. While this may be an attempt to simplify the multi-factorial complexities of optimizing challenge to the individual [14,62], such descriptions offer little insight into how this critical parameter is operationalized. In robotic rehabilitation, systems such as adaptive assist-as-needed technologies can monitor, respond to, and provide real-time feedback on performance, enabling the manipulation of challenge in a structured and measurable way [63,67]. But, without precise reporting of how challenge is monitored, set, progressed, and implemented, robotic interventions cannot be replicated, nor can research data be meta-analyzed to investigate the effect of different challenge levels [11,68]. This confusion is exacerbated by inconsistent terminology, where terms like ‘active’, ‘active-assisted’, and ‘assistive’ are frequently used interchangeably to describe the level of robotic assistance, without any further definition [11]. Transparent, detailed reporting on challenge and progression is, therefore, essential to advancing the field and fully leveraging the therapeutic potential of upper-limb robotics in stroke rehabilitation.
Despite being central to intervention acceptability, aspects related to ‘Personalisation’, ‘Harms’, and ‘Protocol deviations’ were the most poorly and sporadically reported. Personalization, a cornerstone of person-centered rehabilitation, is critical when introducing novel health technologies, as it shapes both clinicians’ and patients’ perceptions of an intervention’s legitimacy [59,60,69]. Addressing the diverse needs of people with stroke, including responding to spasticity, cognitive changes, fatigue, pain, technology literacy, and accessibility are essential to enabling participation in robotic upper-limb rehabilitation [70,71,72,73]. However, personalization to individual needs was only completely reported in three robotic intervention groups (‘Personalisation—Needs’, 11%). Similarly, personally meaningful rehabilitation is essential to the individual’s enjoyment and satisfaction from engaging with robotics [59,69,72,74]. Yet, personalization to preferences was entirely absent (‘Personalisation—Preferences’, 0%), with no interventions reporting the ways in which individuals’ interests, goals, life roles, choice of game or object, inclination toward specific motivational cues, or other preferences were considered. Poor reporting of personalization may perpetuate the use of generic, non-functional tasks that diminish the perceived value of robotic rehabilitation [72]. Alongside personalization, the adoption of complex, novel technologies hinges upon their safe use for a range of individuals [71,74]. As safety data for robotic interventions is still developing, the lack of planning to monitor their adverse consequences (‘Harms—Plan’, 18% complete) and the inconsistent reporting of actual adversity (‘Harms—Actual’, 65% consistency) poses a significant limitation, as it risks exacerbating unfavorable attitudes toward health technology and may restrict clinical uptake [75]. Without complete reporting, it is difficult to determine who the intervention is suitable for or how to manage safety risks, making it unsuitable for widespread use [75]. Furthermore, beyond what is ‘planned’, responsiveness to personalization and harm often occurs through real-time adjustment. Yet, the poor and scattered reporting of robotic intervention ‘Protocol deviations’ (11% complete, 25% consistency), impedes visibility over how such adjustments occur to ensure person-centered and safe implementation. Overall, these gaps not only hinder the acceptability and efficacy of upper-limb robotics but also impedes the generation of high-quality evidence needed to advance its use for clinical uptake in stroke rehabilitation.

4.1. Future Research

Future research should prioritize complete and consistently located intervention descriptions to ensure robotic upper-limb interventions can be optimized, advocated for, and translated to diverse clinical settings. Better reporting will also enable investigation of targeted research questions to advance robotic intervention design, such as those regarding ‘Who provided’ the intervention and ’How’ (e.g., what are the minimum provided expertise or qualifications for safe, unsupervised robotic therapy?), ‘How much’ and ‘How well’ (e.g., how do differences in real-time, active amount influence outcome variability?), ‘How challenging’ and ‘Regression/Progression’ (e.g., how do adaptive challenge algorithms compare with therapist-determined progression in terms of patient experiences and outcomes?), and ‘Personalisation’ (e.g., what strategies best support personalisation of robotic interventions to individual needs and preferences, and how does this impact engagement, adherence, and outcomes?). Addressing such critical questions around feasibility, efficacy, and acceptability require robust study designs, such as factorial and sequential multiple assignment randomized trials, which depend on clear and replicable intervention reporting. Additionally, future research should explore potential relationships between intervention reporting quality and other aspects of research design. For example, 75% of the articles from Boardsworth and colleagues review [11] included in this study that were rated as having a high risk of bias also achieved less than 50% total reporting completeness, suggesting a possible association between poor internal validity and inadequate intervention reporting that warrants further investigation.

4.2. Limitations

The data selection process for this study presents potential limitations. Prioritizing adequately powered RCTs led to the exclusion of many pilot, feasibility, and uncontrolled studies, which may limit the breadth of interpretation. However, the reporting quality issues identified here are likely to be even more pronounced across the wider literature when smaller or methodologically less rigorous studies are included. Furthermore, our inclusion criteria of only dose-matched studies may have artificially elevated the intervention reporting quality of dose parameters. Similarly, as consistency was based on the location of intervention reporting, items with ‘absent’ reporting were excluded from this calculation, potentially inflating the apparent consistency. Nevertheless, potential underestimations of the extent of poor reporting only reinforces the concerns raised by this review’s findings and underscores the need for better intervention reporting practices within the field. Similarly, although unlikely to have impacted the findings given the small number of studies, evaluating intervention reporting quality independently for each group within the four multi-arm trials may have led to an over-representation of these studies in the overall results.

5. Conclusions

This systematic review marks a call to action to ensure the advancement of upper-limb robotics research in stroke rehabilitation. Currently, the potential advantages of robotic rehabilitation for streamlining clinician workload and promoting the efficacy of and engagement with high-dose, progressive, and task-specific training [8,9,10] is severely undermined by poor intervention reporting. This systematic review exposed the missing and incomplete reporting of critical details, such as intervention providers, active dose, progressive challenge, personalization, safety, and real-time protocol changes. This issue was perpetuated by only moderate consistency of reporting location, where details were often buried or dispersed across various article sections and associated materials. Not only does this limit the evaluation and replication of current robotic interventions, it also hinders the identification and optimization of device and program parameters that influence outcomes, risking untargeted and wasteful research [13,15,16]. Furthermore, poor reporting impedes our ability to robustly compare the efficacy and feasibility of robotics with conventional upper-limb rehabilitation [13,15]. For upper-limb robotics to be widely adopted in clinical practice, clinicians and healthcare organizations require evidence that they promote equal, if not superior, outcomes or provide significant cost savings compared to conventional rehabilitation [69]. Even if organizational support is gained, incomplete intervention descriptions may threaten clinician’s knowledge and confidence to integrate novel robotic technologies into practice [69]. Indeed, poor intervention reporting quality risks portraying robotics as inaccessible and suboptimal, exacerbating the underutilization of a promising rehabilitation tool [13,70]. Comprehensive and consistent intervention reporting is urgently needed to advance robotic rehabilitation research and clinical translation.
To address these reporting gaps, researchers and journal editors should consider adopting reporting frameworks such as the CONSORT [76], TIDieR-Rehab [15], and our associated table of reference (Supplementary File S3) to facilitate comprehensive and structured intervention planning and reporting. Furthermore, incomplete reporting, in part, is likely attributed to a widespread emphasis on the reporting of research design and outcomes in the field, leaving limited space for researchers to fully articulate intervention protocols [15,16]. Therefore, journals and researchers may encourage and explore alternative publication formats that enable the depth of intervention reporting required for replication, including trial registrations and study protocol papers. Researchers should also maximize their use of supplementary resources, such as Tables, Figures, online websites, Appendices, and Supplementary Files for comprehensive reporting. Furthermore, to enhance accessibility for researchers and clinicians, we recommend that authors consistently and explicitly articulate all planned intervention procedures, protocol deviations, and references to supplementary resources describing the intervention within the Methods section of their article, while actual fidelity and adverse consequences should be reported in the Results. In conclusion, enhancing the reporting quality of both robotic and conventional upper-limb intervention trials is essential to harnessing the therapeutic potential of upper-limb robotics, paving the way for more robust evaluation, greater personalization, optimized implementation, and improved outcomes in stroke rehabilitation.

Supplementary Materials

The following supporting information can be downloaded at https://www.mdpi.com/article/10.3390/app15158487/s1: File S1: Completed PRISMA-S checklist [21]; File S2: Search Strategy [11]; File S3: Table of reference for evaluating the reporting quality of robotic and conventional upper limb interventions in stroke rehabilitation according to the TIDieR-Rehab checklist [11,15,24,25,26,27,28,30,31,32,33]; File S4: Intervention group and participant characteristics of included articles [31,32,33,37,38,48,50,51,52,54,55,57,58,59,60,61,62,63,64,65,66,67,68,69]; File S5: Completeness and consistency across robotic and conventional robotic and conventional upper limb interventions.

Author Contributions

N.S., G.A., K.B. and E.G. conceptualized the study. E.G., G.A., N.S., K.B. and S.O. developed the methodology. E.G. and K.L.A. undertook training, data extraction, and consensus processes with the supervision of G.A. and N.S. E.G. conducted data analysis, which was interpreted by all authors. E.G. and N.S. wrote the first draft. S.O. reviewed and edited the manuscript. N.S. acquired the funding for this research. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by a Health Research Council New Zealand Emerging Researcher First Grant (Signal-19/624).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

No new data were created or analyzed in this study.

Conflicts of Interest

The authors declare no conflicts of interest.

Abbreviations

The following abbreviations are used in this manuscript:
ADLsActivities of Daily Living
AUTAuckland University of Technology
CONSORTConsolidated Standards of Reporting Trials
OSFOpen Science Framework
PRISMAPreferred Reporting Items for Systematic Reviews and Meta-Analyses
PRISMA-SPreferred Reporting Items for Systematic Reviews and Meta-Analyses literature search extension
RCTRandomized Controlled Trial
TIDieRTemplate for Intervention Description and Replication
TIDieR-RehabTemplate for Intervention Description and Replication—Rehabilitation Extension

References

  1. Feigin, V.L.; Brainin, M.; Norrving, B.; Martins, S.; Sacco, R.L.; Hacke, W.; Fisher, M.; Pandian, J.; Lindsay, P. World Stroke Organization (WSO): Global stroke fact sheet 2022. Int. J. Stroke 2022, 17, 18–29. [Google Scholar] [CrossRef]
  2. Ingram, L.A.; Butler, A.A.; Brodie, M.A.; Lord, S.R.; Gandevia, S.C. Quantifying upper limb motor impairment in chronic stroke: A physiological profiling approach. J. Appl. Physiol. 2021, 131, 949–965. [Google Scholar] [CrossRef] [PubMed]
  3. Purton, J.; Sim, J.; Hunter, S.M. Stroke survivors’ views on their priorities for upper-limb recovery and the availability of therapy services after stroke: A longitudinal, phenomenological study. Disabil. Rehabil. 2023, 45, 3059–3069. [Google Scholar] [CrossRef] [PubMed]
  4. Kwakkel, G.; Kollen, B.J. Predicting activities after stroke: What is clinically relevant? Int. J. Stroke 2013, 8, 25–32. [Google Scholar] [CrossRef]
  5. Newton, S.P.; Dalton, E.J.; Ang, J.Y.; Klaic, M.; Thijs, V.; Hayward, K.S. Dose, content, and context of usual care in stroke upper limb motor interventions: A systematic review. Clin. Rehabil. 2023, 37, 1437–1450. [Google Scholar] [CrossRef] [PubMed]
  6. Huang, J.; Ji, J.R.; Liang, C.; Zhang, Y.-Z.; Sun, H.-C.; Yan, Y.-H.; Xing, X.-B. Effects of physical therapy-based rehabilitation on recovery of upper limb motor function after stroke in adults: A systematic review and meta-analysis of randomized controlled trials. Ann. Palliat. Med. 2022, 11, 521–531. [Google Scholar] [CrossRef]
  7. Loureiro, R.C.; Harwin, W.S.; Nagai, K.; Johnson, M. Advances in upper limb stroke rehabilitation: A technology push. Med. Biol. Eng. Comput. 2011, 49, 1103–1118. [Google Scholar] [CrossRef]
  8. Chien, W.T.; Chong, Y.Y.; Tse, M.K.; Chien, C.W.; Cheng, H.Y. Robot-assisted therapy for upper-limb rehabilitation in subacute stroke patients: A systematic review and meta-analysis. Brain Behav. 2020, 10, e01742. [Google Scholar] [CrossRef]
  9. Wu, X.; Zhang, Q.; Qiao, J.; Chen, N.; Wu, X. Calligraphy-based rehabilitation exercise for improving the upper limb function of stroke patients: Protocol for an evaluator-blinded randomised controlled trial. BMJ Open 2022, 12, e052046. [Google Scholar] [CrossRef] [PubMed]
  10. Norouzi-Gheidari, N. Effects of robot-assisted therapy on stroke rehabilitation in upper limbs: Systematic review and meta-analysis of the literature. J. Rehabil. Res. Dev. 2012, 49, 479–496. [Google Scholar] [CrossRef] [PubMed]
  11. Boardsworth, K.; Rashid, U.; Olsen, S.; Rodriguez-Ramirez, E.; Browne, W.; Alder, G.; Signal, N. Upper limb robotic rehabilitation following stroke: A systematic review and meta-analysis investigating efficacy and the influence of device features and program parameters. J. NeuroEng. Rehabil. 2025, 22, 164. [Google Scholar] [CrossRef]
  12. Veerbeek, J.M.; Langbroek-Amersfoort, A.C.; van Wegen, E.E.; Meskers, C.G.; Kwakkel, G. Effects of robot-assisted therapy for the upper limb after stroke. Neurorehabil. Neural Repair 2017, 31, 107–121. [Google Scholar] [CrossRef]
  13. Lohse, K.R.; Pathania, A.; Wegman, R.; Boyd, L.A.; Lang, C.E. On the reporting of experimental and control therapies in stroke rehabilitation trials: A systematic review. Arch. Phys. Med. Rehabil. 2018, 99, 1424–1432. [Google Scholar] [CrossRef]
  14. Hayward, K.S.; Churilov, L.; Dalton, E.J.; Brodtmann, A.; Campbell, B.C.; Copland, D.; Dancause, N.; Godecke, E.; Hoffmann, T.C.; Lannin, N.A.; et al. Advancing stroke recovery through improved articulation of nonpharmacological intervention dose. Stroke 2021, 52, 761–769. [Google Scholar] [CrossRef]
  15. Signal, N.; Gomes, E.; Olsen, S.; Gemma, A. Enhancing the reporting quality of rehabilitation interventions through an extension of the Template for Intervention Description and Replication (TIDieR): The TIDieR-Rehab checklist and supplementary manual. BMJ Open 2024, 14, e084320. [Google Scholar] [CrossRef]
  16. Hoffmann, T.C.; Glasziou, P.P.; Boutron, I.; Milne, R.; Perera, R.; Moher, D.; Altman, D.G.; Barbour, V.; Macdonald, H.; Johnston, M.; et al. Better reporting of interventions: Template for Intervention Description and Replication (TIDieR) checklist and guide. BMJ 2014, 348, g1687. [Google Scholar] [CrossRef]
  17. Mehrholz, J.; Pollock, A.; Pohl, M.; Kugler, J.; Elsner, B. Systematic review with network meta-analysis of randomized controlled trials of robotic-assisted arm training for improving activities of daily living and upper limb function after stroke. J. NeuroEng. Rehabil. 2020, 17, 83. [Google Scholar] [CrossRef]
  18. Johansen, T.; Sørensen, L.; Kolskår, K.K.; Strøm, V.; Wouda, M.F. Effectiveness of robot-assisted arm exercise on arm and hand function in stroke survivors—A systematic review and meta-analysis. J. Rehabil. Assist. Technol. Eng. 2023, 10, 20556683231183639. [Google Scholar] [CrossRef]
  19. Lee, B.O.; Saragih, I.D.; Batubara, S.O. Robotic arm use for upper limb rehabilitation after stroke: A systematic review and meta-analysis. Kaohsiung J. Med. Sci. 2023, 39, 435–445. [Google Scholar] [CrossRef]
  20. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021, 372, n71. [Google Scholar] [CrossRef]
  21. Rethlefsen, M.L.; Kirtley, S.; Waffenschmidt, S.; Ayala, A.P.; Moher, D.; Page, M.J.; Koffel, J.B. PRISMA-S: An extension to the PRISMA statement for reporting literature searches in systematic reviews. J. Med. Libr. Assoc. 2021, 109, 174–200. [Google Scholar] [CrossRef]
  22. Whitehead, A.L.; Julious, S.A.; Cooper, C.L.; Campbell, M.J. Estimating the sample size for a pilot randomised trial to minimise the overall trial sample size for the external pilot and main trial for a continuous outcome variable. Stat. Methods Med. Res. 2016, 25, 1057–1073. [Google Scholar] [CrossRef]
  23. Lancaster, G.A.; Thabane, L. Guidelines for reporting non-randomised pilot and feasibility studies. Pilot Feasibility Stud. 2019, 5, 114. [Google Scholar] [CrossRef]
  24. Burgar, C.G.; Lum, P.S.; Scremin, A.M.E.; Garber, S.L.; Van der Loos, H.F.M.; Kenney, D.; Shor, P. Robot-assisted upper-limb therapy in acute rehabilitation setting following stroke: Department of Veterans Affairs multisite clinical trial. J. Rehabil. Res. Dev. 2011, 48, 445–458. [Google Scholar] [CrossRef]
  25. Kim, J.A.; Chun, M.H.; Lee, A.; Ji, Y.; Jang, H.; Han, C. The effect of training using an upper limb rehabilitation robot (HEXO-UR30A) in chronic stroke patients: A randomized controlled trial. Medicine 2023, 102, e33246. [Google Scholar] [CrossRef]
  26. Hesse, S.; Heß, A.; Werner, C.C.; Kabbert, N.; Buschfort, R. Effect on arm function and cost of robot-assisted group therapy in subacute patients with stroke and a moderately to severely affected arm: A randomized controlled trial. Clin. Rehabil. 2014, 28, 637–647. [Google Scholar] [CrossRef]
  27. Kahn, L.E.; Zygman, M.L.; Rymer, W.Z.; Reinkensmeyer, D.J. Robot-assisted reaching exercise promotes arm movement recovery in chronic hemiparetic stroke: A randomized controlled pilot study. J. NeuroEng. Rehabil. 2006, 3, 12. [Google Scholar] [CrossRef]
  28. Kutner, N.G.; Zhang, R.; Butler, A.J.; Wolf, S.L.; Alberts, J.L. Quality-of-life change associated with robotic-assisted therapy to improve hand motor function in patients with subacute stroke: A randomized clinical trial. Phys. Ther. 2010, 90, 493–504. [Google Scholar] [CrossRef]
  29. Tomić, T.J.; Savić, A.M.; Vidaković, A.S.; Rodić, S.Z.; Isaković, M.S.; Rodríguez-De-Pablo, C.; Keller, T.; Konstantinović, L.M. ArmAssist robotic system versus matched conventional therapy for poststroke upper limb rehabilitation: A randomized clinical trial. BioMed Res. Int. 2017, 2017, 7659893. [Google Scholar] [CrossRef]
  30. Hsieh, Y.W.; Lin, K.C.; Wu, C.Y.; Shih, T.Y.; Li, M.W.; Chen, C.L. Comparison of proximal versus distal upper-limb robotic rehabilitation on motor performance after stroke: A cluster controlled trial. Sci. Rep. 2018, 8, 2091. [Google Scholar] [CrossRef]
  31. Aprile, I.; Germanotta, M.; Cruciani, A.; Loreti, S.; Pecchioli, C.; Cecchi, F.; Montesano, A.; Galeri, S.; Diverio, M.; Falsini, C.; et al. Upper limb robotic rehabilitation after stroke: A multicenter, randomized clinical trial. J. Neurol. Phys. Ther. 2020, 44, 3–14. [Google Scholar] [CrossRef]
  32. Crema, A.; Bassolino, M.; Guanziroli, E.; Colombo, M.; Blanke, O.; Serino, A.; Micera, S.; Molteni, F. Neuromuscular electrical stimulation restores upper limb sensory-motor functions and body representations in chronic stroke survivors. Med 2022, 3, 58–74.e10. [Google Scholar] [CrossRef]
  33. Rodgers, H.; Bosomworth, H.; Krebs, H.I.; van Wijck, F.; Howel, D.; Wilson, N.; Aird, L.; Alvarado, N.; Andole, S.; Cohen, D.L.; et al. Robot assisted training for the upper limb after stroke (RATULS): A multicentre randomised controlled trial. Lancet 2019, 394, 51–62. [Google Scholar] [CrossRef]
  34. McGoldrick, A.; Byrne, H.; Cadogan, C. An assessment of the reporting of tapering methods in antidepressant discontinuation trials using the TIDieR checklist. Int. J. Clin. Pharm. 2023, 45, 1074–1087. [Google Scholar] [CrossRef]
  35. Slade, S.C.; Finnegan, S.; Dionne, C.E.; Underwood, M.; Buchbinder, R. The Consensus on Exercise Reporting Template (CERT) applied to exercise interventions in musculoskeletal trials demonstrated good rater agreement and incomplete reporting. J. Clin. Epidemiol. 2018, 103, 120–130. [Google Scholar] [CrossRef]
  36. Hansford, H.J.; Wewege, M.A.; Cashin, A.G.; Hagstrom, A.D.; Clifford, B.K.; McAuley, J.H.; Jones, M.D. If exercise is medicine, why don’t we know the dose? An overview of systematic reviews assessing reporting quality of exercise interventions in health and disease. Br. J. Sports Med. 2022, 56, 692–700. [Google Scholar] [CrossRef]
  37. Takebayashi, T.; Takahashi, K.; Amano, S.; Gosho, M.; Sakai, M.; Hashimoto, K.; Hachisuka, K.; Uchiyama, Y.; Domen, K. Robot-assisted training as self-training for upper-limb hemiplegia in chronic stroke: A randomized controlled trial. Stroke 2022, 53, 2182–2191. [Google Scholar] [CrossRef]
  38. Yuan, R.; Qiao, X.; Tang, C.; Zhou, T.; Chen, W.; Song, R.; Jiang, Y.; Reinhardt, J.D.; Wang, H. Effects of uni- vs. bilateral upper limb robot-assisted rehabilitation on motor function, activities of daily living, and electromyography in hemiplegic stroke: A single-blinded three-arm randomized controlled trial. J. Clin. Med. 2023, 12, 2950. [Google Scholar] [CrossRef]
  39. Bhattacharjee, S.; Barman, A.; Patel, S.; Sahoo, J. The Combined Effect of Robot-assisted Therapy and Activities of Daily Living Training on Upper Limb Recovery in Persons With Subacute Stroke: A Randomized Controlled Trial. Arch. Phys. Med. Rehabil. 2024, 105, 1041–1049. [Google Scholar] [CrossRef]
  40. Budhota, A.; Chua, K.S.; Hussain, A.; Kager, S.; Cherpin, A.; Contu, S.; Vishwanath, D.; Kuah, C.W.K.; Ng, C.Y.; Yam, L.H.L.; et al. Robotic Assisted Upper Limb Training Post Stroke: A Randomized Control Trial Using Combinatory Approach Toward Reducing Workforce Demands. Front. Neurol. 2021, 12, 622014. [Google Scholar] [CrossRef]
  41. Calabro, R.S.; Accorinti, M.; Porcari, B.; Carioti, L.; Ciatto, L.; Billeri, L.; Andronaco, V.A.; Galletti, F.; Filoni, S.; Naro, A. Does hand robotic rehabilitation improve motor function by rebalancing interhemispheric connectivity after chronic stroke? Encouraging data from a randomised-clinical-trial. Clin. Neurophysiol. 2019, 130, 767–780. [Google Scholar] [CrossRef]
  42. Carpinella, I.; Lencioni, T.; Bowman, T.; Bertoni, R.; Turolla, A.; Ferrarin, M.; Jonsdottir, J. Effects of robot therapy on upper body kinematics and arm function in persons post stroke: A pilot randomized controlled trial. J. NeuroEng. Rehabil. 2020, 17, 10. [Google Scholar] [CrossRef]
  43. Chang, J.Y.; Chun, M.H.; Lee, A.; Lee, A.; Lee, C.M. Effects of training with a rehabilitation device (Rebless®) on upper limb function in patients with chronic stroke: A randomized controlled trial. Medicine 2024, 103, e38753. [Google Scholar] [CrossRef]
  44. Chen, Z.J.; He, C.; Xu, J.; Zheng, C.-J.; Wu, J.; Xia, N.; Hua, Q.; Xia, W.-G.; Xiong, C.-H.; Huang, X.-L. Exoskeleton-assisted anthropomorphic movement training for the upper limb after stroke: The EAMT randomized trial. Stroke 2023, 54, 1464–1473. [Google Scholar] [CrossRef]
  45. Choi, J.B.; Cho, K.I. Effects of virtual reality-based robot therapy combined with task-oriented therapy on upper limb function and cerebral cortex activation in patients with stroke. Medicine 2024, 103, e38723. [Google Scholar] [CrossRef]
  46. Hsu, H.Y.; Chiu, H.Y.; Kuan, T.S.; Tsai, C.L.; Su, F.C.; Kuo, L.C. Robotic-assisted therapy with bilateral practice improves task and motor performance in the upper extremities of chronic stroke patients: A randomised controlled trial. Aust. Occup. Ther. J. 2019, 66, 637–647. [Google Scholar] [CrossRef]
  47. Hsu, H.Y.; Yang, K.C.; Yeh, C.H.; Lin, Y.C.; Lin, K.R.; Su, F.C.; Kuo, L.C. A Tenodesis-Induced-Grip exoskeleton robot (TIGER) for assisting upper extremity functions in stroke patients: A randomized control study. Disabil. Rehabil. 2022, 44, 7078–7086. [Google Scholar] [CrossRef]
  48. Kim, J.H.; Ko, M.H.; Park, J.W.; Lee, H.J.; Nam, K.Y.; Nam, Y.-G.; Oh, C.-H.; Kwon, B.S.; Park, J.H. Efficacy of electromechanically-assisted rehabilitation of upper limb function in post-stroke patients: A randomized controlled study. J. Rehabil. Med. Clin. Commun. 2021, 4, 1000074. [Google Scholar] [CrossRef]
  49. Lee, M.J.; Lee, J.H.; Lee, S.M. Effects of robot-assisted therapy on upper extremity function and activities of daily living in hemiplegic patients: A single-blinded, randomized, controlled trial. Technol. Health Care 2018, 26, 659–666. [Google Scholar] [CrossRef]
  50. Li, Y.; Lian, Y.; Chen, X.; Zhang, H.; Xu, G.; Duan, H.; Xie, X.; Li, Z. Effect of task-oriented training assisted by force feedback hand rehabilitation robot on finger grasping function in stroke patients with hemiplegia: A randomised controlled trial. J. Neuroeng. Rehabil. 2024, 21, 77. [Google Scholar] [CrossRef]
  51. Lin, Y.; Li, Q.Y.; Qu, Q.; Ding, L.; Chen, Z.; Huang, F.; Hu, S.; Deng, W.; Guo, F.; Wang, C.; et al. Comparative Effectiveness of Robot-Assisted Training Versus Enhanced Upper Extremity Therapy on Upper and Lower Extremity for Stroke Survivors: A Multicentre Randomized Controlled Trial. J. Rehabil. Med. 2022, 54, jrm00314. [Google Scholar] [CrossRef]
  52. Ranzani, R.; Lambercy, O.; Metzger, J.C.; Califfi, A.; Regazzi, S.; Dinacci, D.; Petrillo, C.; Rossi, P.; Conti, F.M.; Gassert, R. Neurocognitive robot-assisted rehabilitation of hand function: A randomized control trial on motor recovery in subacute stroke. J. NeuroEng. Rehabil. 2020, 17, 115. [Google Scholar] [CrossRef]
  53. Şenocak, E.; Korkut, E.; Aktürk, A.; Ozer, A.Y. Is the robotic rehabilitation that is added to intensive body rehabilitation effective for maximization of upper extremity motor recovery following a stroke? A randomized controlled study. Neurol. Sci. 2023, 44, 2835–2843. [Google Scholar] [CrossRef]
  54. Takahashi, K.; Domen, K.; Sakamoto, T.; Toshima, M.; Otaka, Y.; Seto, M.; Irie, K.; Haga, B.; Takebayashi, T.; Hachisuka, K. Efficacy of upper extremity robotic therapy in subacute poststroke hemiplegia: An exploratory randomized trial. Stroke 2016, 47, 1385–1388. [Google Scholar] [CrossRef]
  55. Térémetz, M.; Hamdoun, S.; Colle, F.; Gerardin, E.; Desvilles, C.; Carment, L.; Charron, S.; Cuenca, M.; Calvet, D.; Baron, J.-C.; et al. Efficacy of interactive manual dexterity training after stroke: a pilot single-blinded randomized controlled trial. J. Neuroeng. Rehabil. 2023, 20, 93. [Google Scholar] [CrossRef]
  56. Villafañe, J.H.; Taveggia, G.; Galeri, S.; Bissolotti, L.; Mullè, C.; Imperio, G.; Valdes, K.; Borboni, A.; Negrini, S. Efficacy of Short-Term Robot-Assisted Rehabilitation in Patients With Hand Paralysis After Stroke: A Randomized Clinical Trial. Hand 2018, 13, 95–102. [Google Scholar] [CrossRef]
  57. Wolf, S.L.; Sahu, K.; Bay, R.C.; Buchanan, S.; Reiss, A.; Linder, S.; Rosenfeldt, A.; Alberts, J. The HAAPI (Home Arm Assistance Progression Initiative) trial: A novel robotics delivery approach in stroke rehabilitation. Neurorehabil. Neural Repair 2015, 29, 958–968. [Google Scholar] [CrossRef]
  58. Wu, J.; Cheng, H.; Zhang, J.; Yang, S.; Cai, S. Robot-assisted therapy for upper extremity motor impairment after stroke: A systematic review and meta-analysis. Phys. Ther. 2021, 101, pzab010. [Google Scholar] [CrossRef]
  59. Alder, G.; Taylor, D.; Rashid, U.; Olsen, S.; Brooks, T.; Terry, G.; Niazi, I.K.; Signal, N. A brain computer interface neuromodulatory device for stroke rehabilitation: Iterative user-centered design approach. JMIR Rehabil. Assist. Technol. 2023, 10, e49702. [Google Scholar] [CrossRef]
  60. Stephenson, A.; Stephens, J. An exploration of physiotherapists’ experiences of robotic therapy in upper limb rehabilitation within a stroke rehabilitation centre. Disabil. Rehabil. Assist. Technol. 2018, 13, 245–252. [Google Scholar] [CrossRef]
  61. Nik Ramli, N.N.; Asokan, A.; Mayakrishnan, D.; Annamalai, H. Exploring stroke rehabilitation in malaysia: Are robots better than humans for stroke recuperation? Malays. J. Med. Sci. 2021, 28, 14–23. [Google Scholar] [CrossRef]
  62. Gomes, E.; Alder, G.; Bright, F.A.S.; Signal, N. Understanding task “challenge” in stroke rehabilitation: An interdisciplinary concept analysis. Disabil. Rehabil. 2025, 47, 560–570. [Google Scholar] [CrossRef]
  63. Metzger, J.-C.; Lambercy, O.; Califfi, A.; Dinacci, D.; Petrillo, C.; Rossi, P.; Conti, F.M.; Gassert, R. Assessment-driven selection and adaptation of exercise difficulty in robot-assisted therapy: A pilot study with a hand rehabilitation robot. J. NeuroEng. Rehabil. 2014, 11, 154. [Google Scholar] [CrossRef]
  64. Li, L.; Tyson, S.; Weightman, A. Professionals’ views and experiences of using rehabilitation robotics with stroke survivors: A mixed methods survey. Front. Med. Technol. 2021, 3, 780090. [Google Scholar] [CrossRef]
  65. Maier, M.; Ballester, B.R.; Verschure, P. Principles of neurorehabilitation after stroke based on motor learning and brain plasticity mechanisms. Front. Syst. Neurosci. 2019, 13, 74. [Google Scholar] [CrossRef]
  66. Mubin, O.; Alnajjar, F.; Jishtu, N.; Alsinglawi, B.; Al Mahmud, A. Exoskeletons with virtual reality, augmented reality, and gamification for stroke patients’ rehabilitation: Systematic review. JMIR Rehabil. Assist. Technol. 2019, 6, e12010. [Google Scholar] [CrossRef]
  67. Jamin, P.; Duret, C.; Hutin, E.; Bayle, N.; Koeppel, T.; Gracies, J.-M.; Pila, O. Using robot-based variables during upper limb robot-assisted training in subacute stroke patients to quantify treatment dose. Sensors 2022, 22, 2989. [Google Scholar] [CrossRef]
  68. Olsen, S.; Alder, G.; Rashid, U.; Gomes, E.; Aislabie, M.; Chee, F.; Smith, C.; Kean, B.; Towersey, N.; Signal, N. Challenge level contributes to the efficacy of treadmill interventions after stroke: A systematic review and meta-analysis. Brain Sci. 2023, 13, 1729. [Google Scholar] [CrossRef]
  69. Olsen, S.; Lim, X.M.L.; Alder, G.; Stavric, V.; Signal, N. Factors influencing adoption and sustained use of rehabilitation technologies: A scoping review and qualitative analysis. Disabil. Rehabil. Assist. Technol. 2024, 20, 804–822. [Google Scholar] [CrossRef]
  70. Lo, K.; Stephenson, M.; Lockwood, C. Adoption of robotic stroke rehabilitation into clinical settings: A qualitative descriptive analysis. JBI Evid. Implement. 2020, 18, 376–390. [Google Scholar] [CrossRef]
  71. Proulx, C.E.; Higgins, J.; Gagnon, D.H. Occupational therapists’ evaluation of the perceived usability and utility of wearable soft robotic exoskeleton gloves for hand function rehabilitation following a stroke. Disabil. Rehabil. Assist. Technol. 2023, 18, 953–962. [Google Scholar] [CrossRef]
  72. Charlesworth, K.; Kumar, E.; Olsen, S.; Collis, J.; Rodriguez-Ramirez, E.; Browne, W.; Signal, N. Stroke rehabilitation clinicians’ perspectives of device and programme requirements for effective upperlimb robotic rehabilitation following stroke. In Proceedings of the Smart Strokes 2024, Gold Coast, Australia, 29–30 August 2024. [Google Scholar]
  73. Banyai, A.D.; Brișan, C. Robotics in physical rehabilitation: Systematic review. Healthcare 2024, 12, 1720. [Google Scholar] [CrossRef] [PubMed]
  74. van Ommeren, A.L.; Smulders, L.C.; Prange-Lasonder, G.B.; Buurke, J.H.; Veltink, P.H.; Rietman, J.S. Assistive technology for the upper extremities after stroke: Systematic review of users’ needs. JMIR Rehabil. Assist. Technol. 2018, 5, e10510. [Google Scholar] [CrossRef]
  75. Akalin, N.; Kristoffersson, A.; Loutfi, A. Do you feel safe with your robot? Factors influencing perceived safety in human-robot interaction based on subjective and objective measures. Int. J. Hum.-Comput. Stud. 2022, 158, 102744. [Google Scholar] [CrossRef]
  76. Boutron, I.; Altman, D.G.; Moher, D.; Schulz, K.F.; Ravaud, P. CONSORT statement for randomized trials of nonpharmacologic treatments: A 2017 update and a CONSORT extension for nonpharmacologic trial abstracts. Ann. Intern. Med. 2017, 167, 40–47. [Google Scholar] [CrossRef] [PubMed]
Applsci 15 08487 g001
Figure 1. Article selection process [11].
Figure 1. Article selection process [11].
Applsci 15 08487 g001
Applsci 15 08487 g002
Figure 2. Completeness of reporting in robotic upper-limb interventions only.
Figure 2. Completeness of reporting in robotic upper-limb interventions only.
Applsci 15 08487 g002
Applsci 15 08487 g003
Figure 3. Consistency of key location in robotic upper-limb interventions only.
Figure 3. Consistency of key location in robotic upper-limb interventions only.
Applsci 15 08487 g003
Table 1. Article eligibility criteria.
Table 1. Article eligibility criteria.
InclusionExclusion
ParticipantsAdults over the age of 18 with stroke resulting in reduced upper-limb function.Cerebellar or brainstem stroke.
Experimental interventionRobotic exoskeleton or end-effector rehabilitation targeting the upper limb for one or more sessions.Robotic rehabilitation combined with another non-conventional, exploratory intervention, such as transcranial direct current stimulation or brain–computer interfaces.
Control interventionConventional occupational therapy or physiotherapy rehabilitation targeting the upper limb, such as task-specific training, strength training, repetitive practice, constraint-induced movement therapy, or a combination.
Dose-matched with the experimental robotic intervention in terms of total training time.		Conventional rehabilitation that also uses robotics, unless the robotic component was very brief (<10 min).
Conventional rehabilitation combined with another non-conventional, exploratory intervention such as transcranial direct current stimulation or brain computer interfaces.
OutcomesEvaluation of ‘activity’ level outcomes including (a) upper-limb capacity or (b) activities of daily living as classified by the International Classification of Functioning Disability and Health model.‘Impairment’ level outcomes only, such as muscle strength, passive range of motion, or muscle tone.
Study designRandomized controlled trials with a parallel-group trial design.Randomized crossover trials.
Randomized controlled trials with less than 30 participants and/or without a sample size calculation *.
PublicationFull-text peer-reviewed journal articles published in English.Articles published prior to 2015 *.
Note. Eligibility criteria adapted from Boardsworth and colleagues systematic review [11]. Abbreviation: *, additional exclusions applied for the current systematic review.
Table 2. Definitions and evaluation of intervention reporting quality.
Table 2. Definitions and evaluation of intervention reporting quality.
Aspect of Reporting QualityDefinitionEvaluation
Completeness of reporting The completeness of reported intervention information according to the TIDieR-Rehab item and table of reference.Complete = Full and clear description, to an extent which would allow it to be replicated.
Incomplete = Partial or ambiguous description.
Absent = No description.
Consistency of reporting location The consistency of article location where key intervention information for replication was reported.After all relevant descriptions pertaining to a TIDieR-Rehab item were extracted, the key location containing the most pertinent information for replication was identified (e.g., Methods). Intervention groups that had ‘absent’ reporting of an item were not included in the evaluation of consistency for that item.
Note. Completeness of reporting and consistency of reporting location were evaluated within and across intervention groups and TIDieR-Rehab checklist items.
Table 3. Completeness of reporting by article and intervention group.
Table 3. Completeness of reporting by article and intervention group.
AuthorsInterventionTIDieR-Rehab Item
Brief nameWhyWhoWhenWhat—MaterialsWhat—ProceduresWho providedHowWhereHow much—Session(s) durationHow much—Essential element(s)How much—FrequencyHow much—Intervention lengthHow challengingRegression/
Progression 		Personalisation—NeedsPersonalisation—PreferencesProtocol deviations How well—PlanHow well—ActualHarms—PlanHarms—ActualIntervention group
% complete 		Article total
% complete
Aprile et al. (2020) [31]Experimental 64%54%
Control 45%
Bhattacharjee et al. (2024) [39]Experimental 55%45%
Control 36%
Budhota et al. (2021) [40]Experimental 55%45%
Control 36%
Calabrò et al. (2019) [41]Experimental 59%45%
Control 32%
Carpinella et al.
(2020) [42]		Experimental 68%50%
Control 32%
Chang et al. (2024) [43]Experimental 59%48%
Control 36%
Chen et al. (2023) [44]Experimental 64%52%
Control 41%
Choi et al. (2024) [45]Experimental 41%41%
Control 41%
Crema et al. (2022) [32]Experiment 1 36%35%
Experiment 2 36%
Control 1 41%
Control 2 27%
Hsu et al. (2019) [46]Experimental 59%57%
Control 55%
Hsu et al. (2022) [47]Experimental 59%50%
Control 41%
Kim et al. (2021) [48]Experimental 50%43%
Control 36%
Kim et al. (2023) [25]Experimental 45%43%
Control 27%
Lee et al. (2018) [49]Experimental 32%27%
Control 23%
Li et al. (2024) [50]Experimental 45%48%
Control 50%
Lin et al. (2022) [51]Experimental 41%36%
Control 32%
Ranzani et al. (2020) [52]Experimental 64%59%
Control 55%
Rodgers et al. (2019) [33]Experimental 73%52%
Control 1 68%
Control 2 14%
Şenocak et al. (2023) [53]Experimental 45%34%
Control 23%
Takahashi et al. (2016) [54]Experimental 55%48%
Control 41%
Takebayashi et al. (2022) [37]Experiment 1 45%41%
Experiment 2 41%
Control 36%
Térémetz et al. (2023) [55]Experimental 36%27%
Control 18%
Villafañe et al. (2018) [56]Experimental 27%25%
Control 23%
Wolf et al. (2015) [57] Experimental 59%52%
Control 45%
Yuan et al. (2023) [38] Experiment 1 36%33%
Experiment 2 36%
Control 27%
% complete by item 64%47% *85%100%55%36% *20% *40% *33% *98%25% *87%91%9% *9% *7% *0% *13% *27% *22% *16% *62%
Note. Applsci 15 08487 i001 Complete reporting; Applsci 15 08487 i002 Incomplete reporting; Applsci 15 08487 i003 Absent reporting; *, poor completeness of reporting (<50%).
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Gomes, E.; Alder, G.; Boardsworth, K.; Anderson, K.L.; Olsen, S.; Signal, N. Innovative but Difficult to Replicate: A Systematic Review of the Reporting Quality of Robotic and Conventional Upper-Limb Interventions in Stroke Rehabilitation Randomized Controlled Trials Using the TIDieR-Rehab Checklist. Appl. Sci. 2025, 15, 8487. https://doi.org/10.3390/app15158487

AMA Style

Gomes E, Alder G, Boardsworth K, Anderson KL, Olsen S, Signal N. Innovative but Difficult to Replicate: A Systematic Review of the Reporting Quality of Robotic and Conventional Upper-Limb Interventions in Stroke Rehabilitation Randomized Controlled Trials Using the TIDieR-Rehab Checklist. Applied Sciences. 2025; 15(15):8487. https://doi.org/10.3390/app15158487

Chicago/Turabian Style

Gomes, Emeline, Gemma Alder, Kate Boardsworth, Kate L. Anderson, Sharon Olsen, and Nada Signal. 2025. "Innovative but Difficult to Replicate: A Systematic Review of the Reporting Quality of Robotic and Conventional Upper-Limb Interventions in Stroke Rehabilitation Randomized Controlled Trials Using the TIDieR-Rehab Checklist" Applied Sciences 15, no. 15: 8487. https://doi.org/10.3390/app15158487

APA Style

Gomes, E., Alder, G., Boardsworth, K., Anderson, K. L., Olsen, S., & Signal, N. (2025). Innovative but Difficult to Replicate: A Systematic Review of the Reporting Quality of Robotic and Conventional Upper-Limb Interventions in Stroke Rehabilitation Randomized Controlled Trials Using the TIDieR-Rehab Checklist. Applied Sciences, 15(15), 8487. https://doi.org/10.3390/app15158487

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop