The Feasibility and User Experience of a Program of Progressive Cued Activity to Promote Functional Upper Limb Activity in the Inpatient Rehabilitation Setting with Follow-Up at Home

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This is a study on the feasibility and user experience using a wearable system to promote upper limb physical activity in stroke survivors. It is very well written and complete study. 

specific comments: 
– with the process number of the institution approval and the dates of recruitment period (in subsection 2.1. Participant Recruitment);

– authors also need to add to the Limitations section that the sample had very little sensory impairments (have already discussed about low motor impairment  in the las paragraph of the Discussion section (lines 665–677) and further discuss how people that cannot sense well the vibrotactile stimulus could beneficiate or not from the Souvenir system and what can be the alternatives for those with strong loss of somatosensory input.

Since author's overall sample has very little sensory impairments (the median is the maximum score on the FMA sensory), I’m curious about how those with lower scores (reported a minimum of 5 points) complied with the vibration stimulus? Could they sense it well enough to engage in upper limb physical activity?

Figures and tables look alright.

Author Response

Each of reviewer 1's (R1) comment's (.#) are quoted directly and highlighted in red. Black text describes how we have addressed the concern. Blue text quotes directly from the modified manuscript with changed text underlined.

R1.1: "with the process number of the institution approval and the dates of recruitment period (in subsection 2.1. Participant Recruitment);"

We added the institutional approval numbers to subsection 2.1 of the manuscript as requested - approval dates can be found under Institutional Review Board Statement. We also included the years in which the study was conducted.

The study was conducted at the inpatient rehabilitation facility (IRF) Froedtert Bluemound Rehabilitation Hospital, Milwaukee, Wisconsin USA from 2022-2024. All procedures received institutional approval from Medical College of Wisconsin (PRO-42316) and Marquette University (Reliance 21-082, HR-4044) in accordance with the Declaration of Helsinki.   

R1.2: authors also need to add to the Limitations section that the sample had very little sensory impairments (have already discussed about low motor impairment  in the las paragraph of the Discussion section (lines 665–677) and further discuss how people that cannot sense well the vibrotactile stimulus could beneficiate or not from the Souvenir system and what can be the alternatives for those with strong loss of somatosensory input.

We have revised the limitations section (4.3) to specifically address the high percentage of participants with minimal or no sensory impairments. The inclusion criteria for this study required participants to have had a unilateral stroke, ensuring that all individuals could perceive the vibrotactile cues on their intact side, as intended by the system design, to prompt the activities. The demonstration portion of the protocol also ensured that users could successfully identify and respond to vibrotactile cues when prompted.

Finally, our sample included a higher percentage of participants with relatively high motor and sensory function (reflected by FMA-UE scores) and normal cognition (reflected by MoCA scores), which may limit the generalizability of our results.

Future research should include larger sample sizes and a wider range of functional stratification (such as patients with severe motor and/or sensory impairments). Doing so could provide more robust evidence that user experience is comparable across cognitive, sensory, and motor function levels.

R1.3: Since author's overall sample has very little sensory impairments (the median is the maximum score on the FMA sensory), I’m curious about how those with lower scores (reported a minimum of 5 points) complied with the vibration stimulus? Could they sense it well enough to engage in upper limb physical activity?

Thank you for your thoughtful comment. The cue responses from the participants are provided in the supplementary materials (as well as FMA sensory scores). For the two participants with lower sensory function (FMA sensory scores of 5 and 6) we observed a difference in their pre-cue and post-cue response rates. However, due to the small number of participants in this group and the potential for confounding factors – such as cognitive differences – we did not conduct statistical analysis on these responses.

Additionally, the vibrotactile cues were delivered to both wrists to enable those with sensory impairments to feel the cues on the less involved side. All participants were able to feel the cues and successfully engage in the upper limb physical activity during the demonstration of devices. We clarified with a description of the device demonstration in section 2.3.1:

After clinical testing, the study staff demonstrated the system and its nominal use. During this demonstration, we verified that all participants were able to feel the cues on at least one of the two arms.

Furthermore, we acknowledge that users may benefit from cues other than vibrotactile, such as auditory or visual cues. We have expanded the following section in 4.3 Limitations to the following:

Alternative methods of cue delivery may also expand the accessibility of our system to individuals with greater sensory impairment such as severe sensory loss or neglect. However, further research is needed, as only two participants in our study had sensory deficits per FMA-UE sensory scores, limiting our ability to draw statistical conclusions about this group.

R1.4: Figures and tables look alright.

Thank you for the positive feedback.  

Reviewer 2 Report

Comments and Suggestions for Authors

The Authors report the results of a cued upper limb activity program. The program is delivered via wearable vibroatactile devices that stimulate therapeutic activity during the idle time, thus minimizing caregiver burden. Statistical analysis related to rehabilitation of 30 participants in an inpatient rehabilitation facility post-stroke is performed. The paper is interesting and worth publishing in this special issue.
The main question addressed by the research is:
(a) Description of a cued upper limb activity program delivered via wearable vibroatactile devices that stimulate therapeutic activity during idle time, thus minimizing caregiver burden.
(b) Statistical analysis related to rehabilitation of 30 participants in an inpatient rehabilitation facility post-stroke.
The topic is considered original and relevant in the field, and it addresses a specific gap. The study is the first to assess the feasibility of delivering high dosage of vibrotactile cues to motivate a program of cued therapeutic activity of the paretic upper limb during idle time in an inpatient rehabilitation facility. The subject matter falls within the scope of the special issue.
It adds a description and statistical analysis of the new approach compared with other published material.
The conclusions are consistent with the evidence and arguments presented, and they address the main question posed.
The references are appropriate.
I have no additional comments on the tables and figures.
As for the question about What specific improvements should the authors consider regarding the methodology? What further controls should be considered? 
I think is More extensive statistical research.

Author Response

Each of reviewer 2's (R1) comment's (.#) are quoted directly and highlighted in red. Black text describes how we have addressed the concern. Blue text quotes directly from the modified manuscript with changed text underlined.

R2.1: What specific improvements should the authors consider regarding the methodology? What further controls should be considered?: More extensive statistical research.

We thank the reviewer for their consideration. As this is a feasibility study, our primary focus was on testing specific hypotheses related to the system’s feasibility and functionality. We have included relevant post-hoc analyses in the appendix of the manuscript, which we believe aid in interpreting our primary hypotheses, though we acknowledge that these tests are not extensive. We feel that the statistical analyses provided are appropriate for the study’s objectives and offer valuable insights within the scope of this investigation.

Reviewer 3 Report

Comments and Suggestions for Authors

This study investigated the feasibility and user experience of promoting upper limb activities of post-stroke patients during rehabilitation through a cued activity program with wearable devices and progressive challenges, and demonstrated the potential of wearable technology in the field of post-stroke upper limb rehabilitation. Generally, this manuscript is well organized and presented, but there are still some issues that need to be improved.

Main Comments

1. As mentioned in this manuscript, there were cases of data missing in the study. Although the overall data capture rate was high, the reasons for data loss in the home environment were not very clear. It is recommended to elaborate on the measures taken to reduce data loss in the home environment, as well as the methods for handling missing data to ensure that data loss does not have a significant impact on the reliability of the research results.
2. On page 9,the processing of acceleration data (such as the filtering parameters of 0.25Hz - 2.5Hz) and the verification of device accuracy were not fully explained. It is recommended to supplement technical verification data or provide algorithm details in the appendix.
3. As shown in Table 1, the upper limb function (FMA-UE average 45.6) and cognitive level (MoCA average 25.9) of the patients in the sample are relatively high, which may limit the applicability of the results to patients with severe functional impairments. It is recommended to clarify the sample limitations in the discussion and suggest that future research include a wider range of functional stratification (such as patients with severe motor impairments).

Minor Comments

1. Although the article mentions that some patients failed to wear the device as planned or complete the study (such as technical problems, aversion to vibration cues, etc.), there is little in depth analysis and specific solutions to these problems. It is recommended that the authors further explore the potential reasons for the low participant compliance, such as psychological burden, insufficient device design, etc., in order to provide a basis for improving the device.
2. In section 3.2.1, the follow-up time seems relatively short (average 124.2 ± 33.1 days), and only the feasibility of wearable devices in the home environment is evaluated, without involving the long-term effects of the intervention on upper limb function recovery. It is recommended that the authors mention this shortcoming in the discussion and plan to increase the follow-up time in future research to explore the clinical effects of the intervention in long-term rehabilitation.
3. When calculating compliance, the current method does not consider the situation where the user takes off the wristband but the mobile phone is nearby. Although there are relevant calculation methods, they have not been verified in the stroke population. It is recommended to provide more reasonable and clear explanations for the accuracy, limitations, and applicability of the research results.

Overall, it is recommended that the authors supplement the relevant content in the revised manuscript to further improve the integrity and scientific nature of the research.

Author Response

Each of reviewer 3's (R1) comment's (.#) are quoted directly and highlighted in red. Black text describes how we have addressed the concern. Blue text quotes directly from the modified manuscript with changed text underlined.

R3.Main.1: As mentioned in this manuscript, there were cases of data missing in the study. Although the overall data capture rate was high, the reasons for data loss in the home environment were not very clear. It is recommended to elaborate on the measures taken to reduce data loss in the home environment, as well as the methods for handling missing data to ensure that data loss does not have a significant impact on the reliability of the research results.

We address in the limitations (4.3) that we had more difficulty diagnosing technological errors contributing to data loss in home settings. We clarified a potentially confusing statement in the results section to reflect our suspicion that technical failures contributed to data loss in the home environment even if they were not reported by users: 

No other technical failures were reported by users, although we suspect such issues were probable based on IRF findings.

We provided additional considerations for solutions to such issues:

Enhancing the smartphone app to include error logging and diagnostics could also help identify technical issues, which would be particularly useful for device use in less supervised home settings.

Lastly, we elaborated on measures taken to reduce data loss in the home environment, but emphasize that solutions should largely be directed to the technology rather than place further burden on the user.

Participants received written instructions for using the device at home, including a section on basic troubleshooting to reduce the risk of data loss. We believe the most effective approach to addressing data loss is through technological improvements, rather than placing the burden on the user by adding to these instructions.

For the purposes of the current study, missing accelerometer data should not have impacted the reliability of the user experience survey results collected in the home setting.

R3.Main.2: On page 9,the processing of acceleration data (such as the filtering parameters of 0.25Hz - 2.5Hz) and the verification of device accuracy were not fully explained. It is recommended to supplement technical verification data or provide algorithm details in the appendix.

We thank the reviewer for this comment and provided additional information about the filtering approach in section 2.3.5:

The resulting accelerometry values along each axis were filtered using a bandpass window of 0.25 Hz to 2.5 Hz (MATLAB function bandpass); this window was chosen to be similar to windows used by Bailey et al. 2014 to filter out the static effect of gravity as well as other sources of extraneous motion or noise [44].

We also clarified details about the accelerometer in section 2.2 which we obtained from the hardware data sheet.

For this study of system feasibility, we sampled the 3-axis accelerometers (16-bit resolution: 1.2 × 10−4 g; non-linearity: 0.5% full scale range) at a modest rate of 12.5 samples per second to minimize energy consumption and to allow for rapid upload of files containing the 8 hours of time-stamped data.

Because the MetaMotionR+ devices are off-the-shelf components, we relied on their hardware specifications and did not perform independent technical verification.

R3.Main3: As shown in Table 1, the upper limb function (FMA-UE average 45.6) and cognitive level (MoCA average 25.9) of the patients in the sample are relatively high, which may limit the applicability of the results to patients with severe functional impairments. It is recommended to clarify the sample limitations in the discussion and suggest that future research include a wider range of functional stratification (such as patients with severe motor impairments).     

We agree with your comment on this important detail and have revised our limitation section (4.3) with the following statements:

Future research should include larger sample sizes and a wider range of functional stratification (such as patients with severe motor impairments). Doing so could provide more robust evidence that user experience is comparable across cognitive, sensory, and motor function levels.

R3.Minor.1: Although the article mentions that some patients failed to wear the device as planned or complete the study (such as technical problems, aversion to vibration cues, etc.), there is little in depth analysis and specific solutions to these problems. It is recommended that the authors further explore the potential reasons for the low participant compliance, such as psychological burden, insufficient device design, etc., in order to provide a basis for improving the device.

We believe the revisions to the manuscript in response to comment R1.3 and R3.1, which address potential technical solutions and alternative methods of cue delivery, enhance the previous content. As noted in earlier comments, these additions provide a comprehensive basis for further improving the device.

R3.Minor.2: In section 3.2.1, the follow-up time seems relatively short (average 124.2 ± 33.1 days), and only the feasibility of wearable devices in the home environment is evaluated, without involving the long-term effects of the intervention on upper limb function recovery. It is recommended that the authors mention this shortcoming in the discussion and plan to increase the follow-up time in future research to explore the clinical effects of the intervention in long-term rehabilitation.

As stated in section 2.3.4, the follow-up time was conducted to assess the feasibility of re-engaging with the participants following discharge from IRF and to assess compliance with the device wear in unstructured home settings in order to inform future efficacy studies requiring a longitudinal protocol spanning recovery in both the IRF and at home. We agree that this is an insufficient amount of time to assess the long-term effects of the intervention on upper limb function, and this question would be best suited for a preliminary efficacy study that includes a control group. We further clarify this point for our readers with the following addition to Conclusions (5):

Follow-up involving system use in a home environment was feasible for this cohort, with 76.7% of participants completing the follow-up protocol. We plan to use these findings to inform the design of a future efficacy study of a progressive-challenged cued activity program to improve clinical outcomes after stroke.

Our average follow-up period of 124 ± 33 falls between the timepoints of 3- to 6-months post discharge, which are commonly used in longitudinal studies assessing the efficacy of interventions. We recognize that some studies may extend follow-up to 12 months or longer to evaluate long-term outcomes, but we believe 3 to 6 months is an appropriate and accepted timeframe for assessing initial efficacy. 

R3.Minor.3: When calculating compliance, the current method does not consider the situation where the user takes off the wristband but the mobile phone is nearby. Although there are relevant calculation methods, they have not been verified in the stroke population. It is recommended to provide more reasonable and clear explanations for the accuracy, limitations, and applicability of the research results.

We agree that our interpretation of results could be clarified in regard to compliance calculations. We expanded on this topic with the following in section 4.3:

Lastly, our calculations of compliance did not include estimation of non-wear compliance in which the user doffs the wristbands while keeping them within proximity of the phone. While computations for this type of non-wear exist [67], they have not yet been validated in the stroke population. Excluding non-wear time from our compliance calculation is unlikely to significantly affect the overall trends observed between pre-cue activity and post-cue response rates because choosing to doff the devices equally impacts pre- and post-cue activity.