Preliminary User-Centred Evaluation of a Bio-Cooperative Robotic Platform for Cognitive Rehabilitation in Parkinson’s Disease and Mild Cognitive Impairment: Insights from a Focus Group and Living Lab in the OPERA Project
Abstract
1. Introduction
2. Study 1: Focus Group
2.1. Rationale
2.2. Methods
2.2.1. Participants
2.2.2. Procedure
2.2.3. Materials
2.2.4. Statistical Analysis
2.3. Results
2.3.1. Clinical–Functional Aspects
2.3.2. Technical Expectations and Preferences
2.3.3. General Health and Mobility
2.3.4. Robot Interaction and Expectations
2.3.5. Suggestions and Open Feedback
3. Study 2: Living Lab
3.1. Rationale
3.2. Methods
3.2.1. Participants
3.2.2. Procedure
3.2.3. Materials
Assessment Tools
3.3. Statistical Analysis
3.4. Results
3.4.1. Usability and User Experience
3.4.2. Experience and Engagement
3.4.3. Technology Acceptance
3.4.4. Acceptance and Confidence in the System
3.4.5. Workload and Effort
3.4.6. Correlation Analysis
4. General Discussion
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
GSR | Galvanic Skin Response |
HUBBI | eHealth UsaBility Benchmarking Instrument |
I-UTAUT | The Italian version of the Unified Theory of Acceptance and Use of Technology |
MCI | Mild Cognitive Impairment |
NASA-TLX | NASA Task Load Index |
PD | Parkinson’s Disease |
PRoBio | Bio-Cooperative Robotic Platform |
RGB-D | Red Green Blue-Depth camera |
SUS | System Usability Scale |
TAM | Technology Acceptance Model scales |
UCBM | Campus Bio-Medico University of Rome |
UEQ-S | Short Version of the User Experience Questionnaire |
VR | Virtual Reality |
VRRS | Virtual Reality Rehabilitation System |
References
- Postuma, R.B.; Berg, D.; Stern, M.; Poewe, W.; Olanow, C.W.; Oertel, W.; Obeso, J.; Marek, K.; Litvan, I.; Lang, A.E.; et al. MDS clinical diagnostic criteria for Parkinson’s disease. Mov. Disord. 2015, 30, 1591–1601. [Google Scholar] [CrossRef]
- Litvan, I.; Goldman, J.G.; Tröster, A.I.; Schmand, B.A.; Weintraub, D.; Petersen, R.C.; Mollenhauer, B.; Adler, C.H.; Marder, K.; Williams-Gray, C.H.; et al. Diagnostic criteria for mild cognitive impairment in Parkinson’s disease: Movement Disorder Society Task Force guidelines. Mov. Disord. 2012, 27, 349–356. [Google Scholar] [CrossRef]
- Aarsland, D.; Batzu, L.; Halliday, G.M.; Geurtsen, G.J.; Ballard, C.; Chaudhuri, K.R.; Weintraub, D. Parkinson disease–associated cognitive impairment. Nat. Rev. Dis. Primers 2021, 7, 47. [Google Scholar] [CrossRef]
- Baiano, C.; Barone, P.; Trojano, L.; Santangelo, G. Prevalence and clinical aspects of mild cognitive impairment in Parkinson’s disease: A meta-analysis. Mov. Disord. 2020, 35, 45–54. [Google Scholar] [CrossRef]
- Jellinger, K.A. Mild cognitive impairment in Parkinson’s disease: Current view. Front. Cogn. 2024, 3, 1369538. [Google Scholar] [CrossRef]
- Emre, M.; Aarsland, D.; Brown, R.; Burn, D.J.; Duyckaerts, C.; Mizuno, Y.; Broe, G.A.; Cummings, J.; Dickson, D.W.; Gauthier, S.; et al. Clinical diagnostic criteria for dementia associated with Parkinson’s disease. Mov. Disord. 2007, 22, 1689–1707. [Google Scholar] [CrossRef]
- Papagno, C.; Trojano, L. Cognitive and behavioral disorders in Parkinson’s disease: An update. I: Cognitive impairments. Neurol. Sci. 2018, 39, 215–223. [Google Scholar] [CrossRef]
- Orgeta, V.; McDonald, K.R.; Poliakoff, E.; Hindle, J.V.; Clare, L.; Leroi, I. Cognitive training interventions for dementia and mild cognitive impairment in Parkinson’s disease. Cochrane Database Syst. Rev. 2020, 2, CD011961. [Google Scholar] [CrossRef]
- Kwon, S.H.; Park, J.K.; Koh, Y.H. A systematic review and meta-analysis on the effect of virtual reality-based rehabilitation for people with Parkinson’s disease. J. Neuroeng. Rehabil. 2023, 20, 94. [Google Scholar] [CrossRef]
- Cipresso, P.; Giglioli, I.A.C.; Raya, M.A.; Riva, G. The past, present, and future of virtual and augmented reality research: A network and cluster analysis of the literature. Front. Psychol. 2018, 9, 2086. [Google Scholar] [CrossRef]
- Canning, C.G.; Allen, N.E.; Nackaerts, E.; Paul, S.S.; Nieuwboer, A.; Gilat, M. Virtual reality in research and rehabilitation of gait and balance in Parkinson disease. Nat. Rev. Neurol. 2020, 16, 409–425. [Google Scholar] [CrossRef]
- Tamantini, C.; di Luzio, F.S.; Hromei, C.D.; Cristofori, L.; Croce, D.; Cammisa, M.; Cristofaro, A.; Marabello, M.V.; Basili, R.; Zollo, L. Integrating physical and cognitive interaction capabilities in a robot-aided rehabilitation platform. IEEE Syst. J. 2023, 17, 6516–6527. [Google Scholar] [CrossRef]
- Pandita, S.; Won, A.S. Clinical applications of virtual reality in patient-centered care. In Technology and Health; Academic Press: Cambridge, MA, USA, 2020; pp. 129–148. [Google Scholar]
- Krueger, R.A. Focus Groups: A Practical Guide for Applied Research; Sage Publications: Thousand Oaks, CA, USA, 2014. [Google Scholar]
- Penna, M.F.; Trigili, E.; Zollo, L.; Cipriani, C.; Cappello, L.; Controzzi, M.; Dalise, S.; Chisari, C.; Gruppioni, E.; Crea, S.; et al. Design and administration of a questionnaire for the user-centered design of a novel upper-limb assistive device for brachial plexus injury and post-stroke subjects. In Proceedings of the International Conference on Computers Helping People with Special Needs, Lecco, Italy, 11–15 July 2022; pp. 420–427. [Google Scholar]
- The Jamovi Project. Jamovi [Computer Software]. 2024. Available online: https://www.jamovi.org (accessed on 15 November 2024).
- Leminen, S.; Westerlund, M. Living labs: From scattered initiatives to a global movement. Creat. Innov. Manag. 2019, 28, 250–264. [Google Scholar]
- Ballon, P.; Schuurman, D. Living labs: Concepts, tools and cases. Info 2015, 17. [Google Scholar] [CrossRef]
- Dell’Era, C.; Landoni, P. Living Lab: A methodology between user-centred design and participatory design. Creat. Innov. Manag. 2014, 23, 137–154. [Google Scholar] [CrossRef]
- Pallot, M.; Trousse, B.; Senach, B.; Scapin, D. Living lab research landscape: From user centred design and user experience towards user cocreation. In Proceedings of the First European Summer School “Living Labs”, Paris, France, 25–27 August 2010. [Google Scholar]
- Brooke, J. SUS-A quick and dirty usability scale. Usability Eval. Ind. 1996, 189, 4–7. [Google Scholar]
- Lewis, J.R. The system usability scale: Past, present, and future. Int. J. Hum.-Comput. Interact. 2018, 34, 577–590. [Google Scholar] [CrossRef]
- Broekhuis, M.; van Velsen, L.; Bartali, V. Validation of the Hubbi: A Usability Benchmarking Tool for Ehealth Services. 2023. Available online: https://ssrn.com/abstract=4618554 (accessed on 15 November 2024).
- Schrepp, M.; Hinderks, A. Design and evaluation of a short version of the user experience questionnaire (UEQ-S). Int. J. Interact. Multimed. Artif. Intell. 2017, 4, 103–108. [Google Scholar] [CrossRef]
- Davis, F.D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989, 13, 319–340. [Google Scholar] [CrossRef]
- Venkatesh, V.; Bala, H. Technology acceptance model 3 and a research agenda on interventions. Decis. Sci. 2008, 39, 273–315. [Google Scholar] [CrossRef]
- D’Iorio, A.; Garramone, F.; Rossi, S.; Baiano, C.; Santangelo, G. The Italian version of the unified theory of acceptance and use of technology questionnaire: A pilot validation study. Front. Robot. AI 2025, 12, 1371583. [Google Scholar] [CrossRef]
- Hart, S.G.; Staveland, L.E. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Adv. Psychol. 1988, 52, 139–183. [Google Scholar]
- Devos, H.; Gustafson, K.; Ahmadnezhad, P.; Liao, K.; Mahnken, J.D.; Brooks, W.M.; Burns, J.M. Psychometric properties of NASA-TLX and index of cognitive activity as measures of cognitive workload in older adults. Brain Sci. 2020, 10, 994. [Google Scholar] [CrossRef]
- Broadbent, E.; Stafford, R.; MacDonald, B. Acceptance of healthcare robots for the older population: Review and future directions. Int. J. Soc. Robot. 2009, 1, 319–330. [Google Scholar] [CrossRef]
- Picard, R.W. Affective computing: Challenges. Int. J. Hum.-Comput. Stud. 2003, 59, 55–64. [Google Scholar] [CrossRef]
- Greenhalgh, M.; Duvall, J.; Candiotti, J.; Chung, C.S.; Waters, K.J.; Smolinksi, G.; Peterson, S.; Schneider, U.; Cooper, R.; Cooper, R.A. Emerging Technologies in Neuroengineering to Advance Rehabilitation, Improve Quality of Care Delivery, and Encourage Independent Living. In Handbook of Neuroengineering; Springer Nature: Singapore, 2022; pp. 1–31. [Google Scholar]
- Zipfel, N.; Horreh, B.; Hulshof, C.T.; de Boer, A.G.; van der Burg-Vermeulen, S.J. The relationship between the living lab approach and successful implementation of healthcare innovations: An integrative review. BMJ Open 2022, 12, e058630. [Google Scholar] [CrossRef] [PubMed]
- Hossain, M.; Leminen, S.; Westerlund, M. A systematic review of living lab literature. J. Clean. Prod. 2019, 213, 976–988. [Google Scholar] [CrossRef]
- Schuurman, D.; De Marez, L.; Ballon, P. The impact of living lab methodology on open innovation contributions and outcomes. Technol. Innov. Manag. Rev. 2016, 6, 7–16. [Google Scholar] [CrossRef]
- Figueiredo, T.; Midão, L.; Carrilho, J.; Videira Henriques, D.; Alves, S.; Duarte, N.; Bessa, M.J.; Fidalgo, J.M.; García, M.; Facal, D.; et al. A comprehensive analysis of digital health-focused Living Labs: Innovative approaches to dementia. Front. Med. 2024, 11, 1418612. [Google Scholar] [CrossRef]
Section | Question | Response from Patients (% of Respondents) | Response from Caregiver (% of Respondents) | Response from Healthcare Professionals (% of Respondents) |
---|---|---|---|---|
Section 2—Clinical-Functional Aspects | Daily difficulties * | -Following a long or complex conversation (50.0%) -Get oriented (30.0%) -Remembering meetings (10.0%) -Other (10.0%) | -Following a long or complex conversation (40.0%) -Get oriented (40.0%) -Attention-demanding tasks (20.0%) | -Following a long or complex conversation (37.5%) -Attention-demanding tasks (37.5%) -Remembering meeting (12.5%) -Other (12.5%) |
Frequency of cognitive activities | -Every day (60.0%) -Some days (10.0%) -Rarely (30.0%) | -Every day (20.0%) -Some days (60.0%) -Rarely (20.0%) | - | |
Importance of cognitive abilities | -Very important (90.0%) -Important (10.0%) | -Very important (60.0%) -Important (20.0%) -Slightly important (20.0%) | - | |
Section 3—Technical Expectations and Preferences | Key aspects of system (robot/VR) ** | -Ease of use (60.0%) -Personalization (30.0%) -Engagement (50.0%) -Progress monitoring (10.0%) | -Ease of use (100.0%) -Safety (40.0%) -Personalization (40.0%) -Engagement (80.0%) -Progress monitoring (60.0%) | -Ease of use (50.0%) -Personalization (75.5%) -Progress monitoring (62.5%) |
Preferred activities/exercises (robot/VR) ** | -Memory games (50.0%) -Virtual simulations (20.0%) -Cognitive challenges (40.0%) | -Memory games (80.0%) -Virtual simulations (40.0%) -Cognitive challenges (60.0%) | -Memory games (12.5%) -Virtual simulations (75.0%) -Cognitive challenges (37.5%) | |
Importance of immediate feedback | -Very important (60.0%) -Important (40.0%) | -Very important (100.0%) | -Very important (62.5%) -Important (37.5%) | |
Main goals of cognitive rehabilitation * | -Autonomy in daily activities (50.0%) -Reduce anxiety and stress (30.0%) -Improve memory (10.0%) -Increase concentration (10.0%) | -Autonomy in daily activities (40.0%) -Reduce anxiety and stress (40.0%) -Improve memory (20.0%) | -Autonomy in daily activities (75.0%) -Reduce anxiety and stress (12.5%) -Improve memory (12.5%) | |
Section 4—General Health and Mobility | Motor difficulties (impact tech use) | -Yes (20.0%) | -Yes (40.0%) | -Yes (50.0%) |
Fine motor difficulties | -Yes (60.0%) | -Yes (80.0%) | -Yes (87.5%) | |
Autonomous use of technology devices | -Yes (70.0%) | -Yes (100.0%) | -Yes (87.5%) | |
Physical needs/ limitations for design | -Yes (20.0%) | -Yes (40.0%) | -Yes (25%) | |
Section 5—Robotic Interaction and Expectations | Useful support by robotic arm * | -Cognitive activities (60.0%) -Physical movements (40.0%) | -Cognitive activities (80.0%) -Physical movements (20.0%) | -Cognitive activities (62.5%) -Physical movements (37.5%) |
Preferred activities with robotic arm ** | -Manipulating objects (20.0%) -Simulating daily activities (20.0.7%) -Motor Rehabilitation (40.0%) | -Manipulate objects (20.0%) -Complete Puzzles (40.0%) -Simulate daily activities (40.0%) -Motor Rehabilitation (40.0%) | -Manipulate objects (25.0%) -Complete puzzles (37.5%) -Simulate daily activities (87.5%) -Motor Rehabilitation (62.5%) | |
Desired technical features (robotic arm) * | -Adjusting arm gestures (30.0%) -Soft and precise movements (30.0%) -Safety in use (20.0%) -Other (20.0%) | -Adjusting arm gestures (20.0%) -Soft and precise movements (20.0%) -Safety in use (20.0%) -Other (40.0%) | -Adjusting arm gestures (62.5%) -Soft and precise movements (12.5%) -Safety in use (25.0%) | |
Importance of robot detecting emotional/ fatigue states * | -Very important (50.0%) -Important (50.0%) | -Very important (80.0%) -Important (20.0%) | -Very important (87.5%) -Important (12.5%) |
Questionnaire/Subscale | Healthcare Professionals (Mean; SD) | Healthy Volunteers (Mean; SD) | p-Value |
---|---|---|---|
SUS (Total) | 70.1 (12.3) | 83.0 (10.5) | 0.23 |
HUBBI—Basic System Performance | 3.5 (0.8) | 4.2 (0.7) | 0.04 |
HUBBI—Task-Technology Fit | 3.8 (0.7) | 4.0 (0.6) | 0.13 |
HUBBI—Design and Presentation | 3.9 (0.9) | 4.2 (0.6) | 0.26 |
HUBBI—Navigation and Structure | 4.0 (0.7) | 4.1 (0.5) | 0.66 |
HUBBI—Information and Terminology | 3.7 (0.8) | 3.8 (0.5) | 0.65 |
HUBBI—Guidance and Support | 3.6 (0.6) | 3.8 (0.4) | 0.31 |
HUBBI—Satisfaction | 4.1 (0.7) | 4.2 (0.5) | 0.58 |
UEQ-S—Pragmatic Quality | 1.1 (0.4) | 1.1 (0.5) | 0.94 |
UEQ-S—Hedonic Quality | 0.9 (0.5) | 1.2 (0.4) | 0.61 |
UEQ-S (Total) | 1.0 (0.4) | 1.2 (0.4) | 0.79 |
TAM—Perceived Usefulness (PU) | 5.2 (1.1) | 5.4 (0.9) | 0.63 |
TAM—Perceived Ease of Use (PEOU) | 5.0 (1.2) | 5.2 (0.8) | 0.48 |
I-UTAUT—Performance Expectancy | 3.8 (0.9) | 3.9 (0.8) | 0.96 |
I-UTAUT—Social Influence | 3.6 (0.8) | 3.7 (0.7) | 0.90 |
I-UTAUT—Facilitating Conditions | 3.9 (0.9) | 4.0 (0.8) | 0.84 |
I-UTAUT—Self-Efficacy | 3.7 (0.7) | 3.8 (0.6) | 0.91 |
I-UTAUT—Anxiety | 2.8 (0.6) | 2.9 (0.5) | 0.48 |
I-UTAUT (Total) | 3.7 (0.8) | 3.8 (0.7) | 0.90 |
NASA-TLX—Mental Demand | 30.2 (8.5) | 28.5 (7.9) | 0.65 |
NASA-TLX—Physical Demand | 25.1 (6.7) | 24.3 (5.9) | 0.56 |
NASA-TLX—Temporal Demand | 28.3 (7.4) | 29.1 (7.1) | 0.27 |
NASA-TLX—Performance | 75.0 (9.2) | 77.3 (8.5) | 0.41 |
NASA-TLX—Effort | 35.1 (8.1) | 34.3 (7.5) | 0.72 |
NASA-TLX—Frustration | 20.0 (7.3) | 21.1 (6.9) | 0.84 |
NASA-TLX (Total) | 35.1 (6.8) | 35.6 (6.5) | 0.90 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Crocetto, Y.; Abagnale, S.; Martinelli, G.; Della Bella, S.; Pavan, E.; Rondoni, C.; Voscarelli, A.; Pirini, M.; Scotto di Luzio, F.; Zollo, L.; et al. Preliminary User-Centred Evaluation of a Bio-Cooperative Robotic Platform for Cognitive Rehabilitation in Parkinson’s Disease and Mild Cognitive Impairment: Insights from a Focus Group and Living Lab in the OPERA Project. J. Clin. Med. 2025, 14, 7042. https://doi.org/10.3390/jcm14197042
Crocetto Y, Abagnale S, Martinelli G, Della Bella S, Pavan E, Rondoni C, Voscarelli A, Pirini M, Scotto di Luzio F, Zollo L, et al. Preliminary User-Centred Evaluation of a Bio-Cooperative Robotic Platform for Cognitive Rehabilitation in Parkinson’s Disease and Mild Cognitive Impairment: Insights from a Focus Group and Living Lab in the OPERA Project. Journal of Clinical Medicine. 2025; 14(19):7042. https://doi.org/10.3390/jcm14197042
Chicago/Turabian StyleCrocetto, Ylenia, Simona Abagnale, Giulia Martinelli, Sara Della Bella, Eleonora Pavan, Cristiana Rondoni, Alfonso Voscarelli, Marco Pirini, Francesco Scotto di Luzio, Loredana Zollo, and et al. 2025. "Preliminary User-Centred Evaluation of a Bio-Cooperative Robotic Platform for Cognitive Rehabilitation in Parkinson’s Disease and Mild Cognitive Impairment: Insights from a Focus Group and Living Lab in the OPERA Project" Journal of Clinical Medicine 14, no. 19: 7042. https://doi.org/10.3390/jcm14197042
APA StyleCrocetto, Y., Abagnale, S., Martinelli, G., Della Bella, S., Pavan, E., Rondoni, C., Voscarelli, A., Pirini, M., Scotto di Luzio, F., Zollo, L., Cicarelli, G., Polito, C., & Estraneo, A. (2025). Preliminary User-Centred Evaluation of a Bio-Cooperative Robotic Platform for Cognitive Rehabilitation in Parkinson’s Disease and Mild Cognitive Impairment: Insights from a Focus Group and Living Lab in the OPERA Project. Journal of Clinical Medicine, 14(19), 7042. https://doi.org/10.3390/jcm14197042