Upper Limb Physical Rehabilitation Using Serious Videogames and Motion Capture Systems: A Systematic Review
Abstract
:1. Introduction
2. Materials and Methods
2.1. Eligibility Criteria
2.2. Search Strategy
2.3. Description of the Selection Process of the Study
3. Results
3.1. Selection of the Study
3.2. General Characteristics of the Study
3.2.1. Motion Capture Systems Reported in the Studies
3.2.2. Diagnosis or Clinical Condition on Which the Technology Described in the Works Was Focused
3.2.3. Population Involved in the Validation of the Results
3.2.4. Affordability of the Technology Used
3.3. Technologies as Support in the Physical Rehabilitation of the Upper Limb
3.3.1. Use of Motion Capture Systems in Upper Limb Physical Rehabilitation
Optical Systems Used
Non-Optical Systems Used
3.3.2. Use of Videogames in Upper Limb Physical Rehabilitation
3.3.3. Diagnosis and Treatments Supported by Technology
Technological Support in Post-Stroke Motor Recovery
Technological Support in the Recovery from Other Diagnoses
4. Discussion
5. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- PNUD Objetivos de Desarrollo Sostenible | PNUD. Available online: https://www.undp.org/content/undp/es/home/sustainable-development-goals.html (accessed on 8 November 2019).
- World Health Organization; The World Bank. World Report on Disability; WHO Press: Geneva, Switzerland, 2011. [Google Scholar]
- International Disability Alliance | International Disability Alliance. Available online: http://www.internationaldisabilityalliance.org/ (accessed on 3 February 2020).
- Turolla, A.; Rossettini, G.; Viceconti, A.; Palese, A.; Geri, T. Musculoskeletal Physical Therapy During the COVID-19 Pandemic: Is Telerehabilitation the Answer? Phys. Ther. 2020, 100, 1260–1264. [Google Scholar] [CrossRef]
- Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; Altman, D.; Antes, G.; Atkins, D.; Barbour, V.; Barrowman, N.; Berlin, J.A.; et al. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. 2009, 6. [Google Scholar] [CrossRef] [Green Version]
- Bionics, B.T.O. ISPO 17th World Congress Abstract Book. Prosthet. Orthot. Int. 2019, 43, 1–600. [Google Scholar] [CrossRef] [Green Version]
- MacIntosh, A.; Desailly, E.; Vignais, N.; Vigneron, V.; Biddiss, E. A biofeedback-enhanced therapeutic exercise video game intervention for young people with cerebral palsy: A randomized single-case experimental design feasibility study. PLoS ONE 2020, 15, e0234767. [Google Scholar] [CrossRef] [PubMed]
- Condino, S.; Turini, G.; Viglialoro, R.; Gesi, M.; Ferrari, V. Wearable Augmented Reality Application for Shoulder Rehabilitation. Electronics 2019, 8, 1178. [Google Scholar] [CrossRef] [Green Version]
- Agyeman, M.O.; Al-Mahmood, A.; Hoxha, I. A Home Rehabilitation System Motivating Stroke Patients with Upper and/or Lower Limb Disability. ACM Int. Conf. Proceeding Ser. 2019. [Google Scholar] [CrossRef]
- Fuertes, G.; Mollineda, R.; Gallardo, J.; Pla, F. A RGBD-Based Interactive System for Gaming-Driven Rehabilitation of Upper Limbs. Sensors 2019, 19, 3478. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Lui, J.; Menon, C. Would a thermal sensor improve arm motion classification accuracy of a single wrist—Mounted inertial device? Biomed. Eng. Online 2019, 18, 1–17. [Google Scholar] [CrossRef] [Green Version]
- Wittmann, F.; Lambercy, O.; Gassert, R. Magnetometer-Based Drift Correction During Rest in IMU Arm Motion Tracking. Sensors 2019, 19, 1312. [Google Scholar] [CrossRef] [Green Version]
- Fernandes, F.G.; Cardoso, A.; Lopes, R.D.A. Games applied to children with motor impairment using the myo wearable device. An. Acad. Bras. Cienc. 2020, 92, 1–14. [Google Scholar] [CrossRef] [Green Version]
- Ma, M.; Proffitt, R.; Skubic, M. Validation of a Kinect V2 based rehabilitation game. PLoS ONE 2018, 13, 1–15. [Google Scholar] [CrossRef] [PubMed]
- David, V.; Forjan, M.; Paštěka, R.; Scherer, M.; Hofstätter, O. Development of a multi-purpose easy-to-use set of tools for home based rehabilitation: Use cases and applications developed during the rehabitation project. ACM Int. Conf. Proc. Ser. 2018, 323–330. [Google Scholar] [CrossRef] [Green Version]
- Yang, Z.; Rafiei, M.H.; Hall, A.; Thomas, C.; Midtlien, H.A.; Hasselbach, A.; Adeli, H.; Gauthier, L.V. A Novel Methodology for Extracting and Evaluating Therapeutic Movements in Game-Based Motion Capture Rehabilitation Systems. J. Med. Syst. 2018, 42. [Google Scholar] [CrossRef] [PubMed]
- Da, J.S.; Neto, C.; Filho, P.P.R.; Ferreira, G.P.; Silva, D.A.; Da, N.B.; Olegario, C.; Duarte, B.F.; Albuquerque, V.H.C.D.E.; Ao, J.O. Dynamic Evaluation and Treatment of the Movement Amplitude Using Kinect Sensor. IEEE Access 2018, 6, 17292–17305. [Google Scholar] [CrossRef]
- Bonnechère, B.; Jansen, B.; Haack, I.; Omelina, L.; Feipel, V.; Jan, S.V.S.; Pandolfo, M. Automated functional upper limb evaluation of patients with Friedreich ataxia using serious games rehabilitation exercises. J. Neuroeng. Rehabil. 2018, 15, 1–9. [Google Scholar] [CrossRef]
- Goršič, M.; Cikajlo, I.; Novak, D. Competitive and cooperative arm rehabilitation games played by a patient and unimpaired person: Effects on motivation and exercise intensity. J. Neuroeng. Rehabil. 2017, 14, 1–18. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Wittmann, F.; Held, J.P.; Lambercy, O.; Starkey, M.L.; Curt, A.; Höver, R.; Gassert, R.; Luft, A.R.; Gonzenbach, R.R. Self-directed arm therapy at home after stroke with a sensor-based virtual reality training system. J. Neuroeng. Rehabil. 2016, 13, 1–10. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Bird, M.L.; Cannell, J.; Callisaya, M.L.; Moles, E.; Rathjen, A.; Lane, K.; Tyson, A.; Smith, S. “FIND Technology”: Investigating the feasibility, efficacy and safety of controller- free interactive digital rehabilitation technology in an inpatient stroke population: Study protocol for a randomized controlled trial. Trials 2016, 17, 1–6. [Google Scholar] [CrossRef] [Green Version]
- Mobini, A.; Behzadipour, S.; Saadat, M. Test–retest reliability of Kinect ’ s measurements for the evaluation of upper body recovery of stroke patients. Biomed. Eng. OnLine 2015, 14, 1–13. [Google Scholar] [CrossRef] [Green Version]
- Nathan, D.; Huynh, D.Q.; Rubenson, J.; Rosenberg, M. Estimating Physical Activity Energy Expenditure with the Kinect Sensor in an Exergaming Environment. PLoS ONE 2015, 10, 1–22. [Google Scholar] [CrossRef] [Green Version]
- Callejas-cuervo, M.; Díaz, G.M.; Ruíz-olaya, A.F. Integration of emerging motion capture technologies and videogames for human upper-limb telerehabilitation: A systematic review. Rev. DYNA 2015, 82, 68–75. [Google Scholar] [CrossRef]
- Basteris, A.; Nijenhuis, S.M.; Buurke, J.H.; Prange, G.B. Lag—Lead based assessment and adaptation of exercise speed for stroke survivors. Rob. Auton. Syst. 2015, 73, 144–154. [Google Scholar] [CrossRef] [Green Version]
- OMS Atención Médica y Rehabilitación. Available online: https://www.who.int/disabilities/care/es/ (accessed on 8 November 2019).
- Guzik-Kopyto, A.; Michnik, R.; Wodarski, P.; Chuchnowska, I. Determination of Loads in the Joints of the Upper Limb during Activities of Daily Living. In Proceedings of the Advances in Intelligent Systems and Computing; Piętka, E., Badura, P., Kawa, J., Wieclauek, W., Eds.; Springer: Kamień Śląski, Poland, 2016; Volume 472, pp. 99–108. [Google Scholar]
- Tobón, R. The Mocap Book: A Practical Guide to the Art of Motion Capture; Foris Force: Orlando, FL, USA, 2010. [Google Scholar]
- Srinivasan, S.R.; Sridhar, S.; Balasubramanian, G.; Vasu, K. Complex Animal Movement Capture and Live Transmission (CAMCALT). In Proceedings of the TENCON 2018—2018 IEEE Region 10 Conference, Jeju, Korea, 28–31 October 2018; pp. 0978–0981. [Google Scholar]
- Lammatha, K.A.; Chinnusamy, K.; Malgireddy, N.R.C. Marker less motion capture (Mocap) information compressor & facts mining IRC monocular object. J. Int. Pharm. Res. 2019, 46, 117–124. [Google Scholar] [CrossRef]
- Liao, Y.; Vakanski, A.; Xian, M.; Paul, D.; Baker, R. A review of computational approaches for evaluation of rehabilitation exercises. Comput. Biol. Med. 2020, 119, 103687. [Google Scholar] [CrossRef] [Green Version]
- Mandery, C.; Terlemez, Ö.; Do, M.; Vahrenkamp, N.; Asfour, T. Unifying Representations and Large-Scale Whole-Body Motion Databases for Studying Human Motion. IEEE Trans. Robot. 2016, 32, 796–809. [Google Scholar] [CrossRef]
- Callejas-Cuervo, M.; Ruiz, O.A.; Gutiérrez, R. Métodos de captura de movimiento biomecánico enfocados en telefisioterapia. Pan Am. Health Care Exch. PAHCE 2013, 1–6. [Google Scholar] [CrossRef]
- Solberg, R.T.; Jensenius, A.R. Optical or inertial? Evaluation of two motion capture systems for studies of dancing to electronic dance music. In Proceedings of the SMC 2016—13th Sound Music Computing Conference Proceedings, Hamburg, Germany, 31 August–3 September 2016; pp. 469–474. [Google Scholar]
- Topley, M.; Richards, J.G. A comparison of currently available optoelectronic motion capture systems. J. Biomech. 2020, 106, 109820. [Google Scholar] [CrossRef]
- Wang, X.; Ge, W.; Li, J. Hand tracking and fingertip detection based on Kinect. In Proceedings of the IOP Conference Series: Materials Science and Engineering; IOP Publishing: Xi’an, China, 2020; Volume 740. [Google Scholar]
- Jintronix Sense Your Progress—Jintronix. Available online: http://www.jintronix.com/ (accessed on 19 February 2020).
- 5DT Technologies Home—5DT. Available online: https://5dt.com/ (accessed on 29 July 2020).
- Golomb, M.R.; McDonald, B.C.; Warden, S.J.; Yonkman, J.; Saykin, A.J.; Shirley, B.; Huber, M.; Rabin, B.; AbdelBaky, M.; Nwosu, M.E.; et al. In-Home Virtual Reality Videogame Telerehabilitation in Adolescents With Hemiplegic Cerebral Palsy. Arch. Phys. Med. Rehabil. 2010, 91, 1–8. [Google Scholar] [CrossRef] [PubMed]
- Pollind, M.; Soangra, R. Development and Validation of Wearable Inertial Sensor System for Postural Sway Analysis. Meas. J. Int. Meas. Confed. 2020, 165, 108101. [Google Scholar] [CrossRef]
- Sers, R.; Forrester, S.; Moss, E.; Ward, S.; Ma, J.; Zecca, M. Validity of the Perception Neuron inertial motion capture system for upper body motion analysis. Meas. J. Int. Meas. Confed. 2020, 149, 107024. [Google Scholar] [CrossRef]
- Feuvrier, F.; Sijobert, B.; Azevedo, C.; Griffiths, K.; Alonso, S.; Dupeyron, A.; Laffont, I.; Froger, J. Inertial measurement unit compared to an optical motion capturing system in post-stroke individuals with foot-drop syndrome. Ann. Phys. Rehabil. Med. 2019, 63, 195–201. [Google Scholar] [CrossRef] [PubMed]
- Khobkhun, F.; Hollands, M.A.; Richards, J.; Ajjimaporn, A. Can we accurately measure axial segment coordination during turning using inertial measurement units (IMUs)? Sensors 2020, 20, 2518. [Google Scholar] [CrossRef] [PubMed]
- Beange, K.H.E.; Chan, A.D.C.; Beaudette, S.M.; Graham, R.B. Concurrent validity of a wearable IMU for objective assessments of functional movement quality and control of the lumbar spine. J. Biomech. 2019, 97, 109356. [Google Scholar] [CrossRef] [PubMed]
- Hafer, J.F.; Provenzano, S.G.; Kern, K.L.; Agresta, C.E.; Grant, J.A.; Zernicke, R.F. Measuring markers of aging and knee osteoarthritis gait using inertial measurement units. J. Biomech. 2020, 99, 109567. [Google Scholar] [CrossRef]
- Kinestica Bimeo PRO Neurological Rehabilitation/Stroke—Kinestica—Motivating Neurological Rehabilitation. Available online: http://www.kinestica.com/bimeo-pro.html (accessed on 19 February 2020).
- Hocoma Armeo®Senso—Hocoma. Available online: https://www.hocoma.com/solutions/armeo-senso/ (accessed on 19 February 2020).
- Wittmann, F.; Lambercy, O.; Gonzenbach, R.R.; Van Raai, M.A.; Hover, R.; Held, J.; Starkey, M.L.; Curt, A.; Luft, A.; Gassert, R. Assessment-driven arm therapy at home using an IMU-based virtual reality system. IEEE Int. Conf. Rehabil. Robot. 2015, 707–712. [Google Scholar] [CrossRef]
- Boyali, A.; Hashimoto, N.; Matsumoto, O. Hand posture and gesture recognition using MYO armband and spectral collaborative representation based classification. In Proceedings of the 2015 IEEE 4th Global Conference on Consumer Electronics (GCCE), Osaka, Japan, 27–30 October 2015; pp. 200–201. [Google Scholar]
- Fan, Y.; Yang, C.; Wu, X. Improved teleoperation of an industrial robot arm system using leap motion and MYO armband. In Proceedings of the 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO), Dali, China, 6–8 December 2019. [Google Scholar]
- Popovici, I. Experimental results on the accuracy of the Myo Armband for short-range pointing tasks. In Proceedings of the 15th International Conference on Development and Application Systems, DAS 2020—Proceedings, Suceava, Romania, 21–23 May 2020; pp. 185–188. [Google Scholar]
- Valve Corporation Dashy Square en Steam. Available online: https://store.steampowered.com/app/461230/Dashy_Square/ (accessed on 10 August 2020).
- Unity Technologies Unity. Available online: https://unity.com (accessed on 12 August 2020).
- World Health Organization Cardiovascular Diseases. Available online: https://www.who.int/health-topics/cardiovascular-diseases/#tab=tab_1 (accessed on 2 July 2020).
- Ramlee, M.H.; Gan, K.B. Function and Biomechanics of Upper Limb in Post-Stroke Patients—A Systematic review. J. Mech. Med. Biol. 2017, 17, 1–19. [Google Scholar] [CrossRef]
- Martínez de Zabarte Fernández, J.M.; Ros Arnal, I.; Peña Segura, J.L.; García Romero, R.; Rodríguez Martínez, G. Bone health impairment in patients with cerebral palsy. Arch. Osteoporos. 2020, 15, 3–9. [Google Scholar] [CrossRef]
- Koeppen, A.H.; Mazurkiewicz, J.E. Friedreich ataxia: Neuropathology revised. J. Neuropathol. Exp. Neurol. 2013, 72, 78–90. [Google Scholar] [CrossRef] [Green Version]
- Li, S.; Xue, J.J.; Hong, P.; Song, C.; He, Z.H. Comparison of energy expenditure and substrate metabolism during overground and motorized treadmill running in Chinese middle-aged women. Sci. Rep. 2020, 10, 1–7. [Google Scholar] [CrossRef]
- Naeemabadi, M.R.; Dinesen, B.; Andersen, O.K.; Najafi, S.; Hansen, J. Evaluating accuracy and usability of microsoft kinect sensors and wearable sensor for tele knee rehabilitation after knee operation. In Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies, Madeira, Portugal, 19–21 January 2018; pp. 128–135. [Google Scholar]
- Schlagenhauf, F.; Sreeram, S.; Singhose, W. Comparison of Kinect and Vicon Motion Capture of Upper-Body Joint Angle Tracking. In Proceedings of the 2018 IEEE 14th International Conference on Control and Automation (ICCA), Anchorage, AK, USA, 12–15 June 2018; pp. 674–679. [Google Scholar]
- Napoli, A.; Glass, S.; Ward, C.; Tucker, C.; Obeid, I. Performance analysis of a generalized motion capture system using microsoft kinect 2.0. Biomed. Signal Process. Control 2017, 38, 265–280. [Google Scholar] [CrossRef]
- Callejas-cuervo, M.; Gutierrez, R.M.; Hernandez, A.I. Joint amplitude MEMS based measurement platform for low cost and high accessibility telerehabilitation: Elbow case study. J. Bodyw. Mov. Ther. 2016, 21, 574–581. [Google Scholar] [CrossRef] [PubMed]
- Cao, Z.; Hidalgo Martinez, G.; Simon, T.; Wei, S.-E.; Sheikh, Y.A. OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. IEEE Trans. Pattern Anal. Mach. Intell. 2019, 1–14. [Google Scholar] [CrossRef] [Green Version]
Database | Search Parameters |
---|---|
Scopus | TITLE-ABS-KEY (((rehabilitation OR health OR “physical therapy” OR “musculoskeletal”) AND (videogames OR “video games” OR “video-games” OR “serious videogames” OR “serious games” OR “serious video games” OR “exergames” OR “exergaming” OR “active videogames”) AND (“upper limb” OR “elbow” OR “shoulder” OR “arm” OR “wrist” OR “humerus”) AND (“inertial sensor” OR “motion capture” OR “motion capture system” OR mocap OR wearable))) AND (LIMIT-TO (PUBYEAR, 2020) OR LIMIT-TO (PUBYEAR, 2019) OR LIMIT-TO (PUBYEAR, 2018) OR LIMIT-TO (PUBYEAR, 2017) OR LIMIT-TO (PUBYEAR, 2016) OR LIMIT-TO (PUBYEAR, 2015)) |
PubMed | ((rehabilitation OR health OR “physical therapy” OR “musculoskeletal”) AND (videogames OR “video games” OR “video-games” OR “serious videogames” OR “serious games” OR “serious video games” OR “exergames” OR “exergaming” OR “active videogames”) AND (“upper limb” OR “elbow” OR “shoulder” OR “arm” OR “wrist” OR “humerus”) AND (“inertial sensor” OR “motion capture” OR “motion capture system” OR mocap OR wearable)) |
IEEE Xplore and Web of Science | ((rehabilitation OR health OR “physical AND therapy” OR musculoskeletal) AND (videogames OR “video AND games” OR video-games OR “serious AND videogames” OR “serious AND games” OR “serious AND video AND games” OR exergames OR exergaming OR “active AND videogames”) AND (“upper AND limb” OR “elbow” OR “shoulder” OR “arm” OR “wrist” OR “humerus”) AND (“inertial AND sensor” OR “motion AND capture” OR “mocap” OR “motion AND capture AND system” OR wearable)) |
No. | Mocap System | Clinical Condition | Population (Sample) * | Technology Used ** | Part of the Body Rehabilitated | Reference |
---|---|---|---|---|---|---|
1 | IMU | Cerebral palsy | 19 P | Mixed: Myo bracelet, adapted commercial videogame (Dashy Square and personalized software development) | Hand and wrist | [7] |
2 | MS HoloLens | ROM | 25 H | Mixed: MS HoloLens and developed videogame | Shoulder | [8] |
3 | IMU | Stroke | 8 H | Proposed system: an environment of games and software for the therapist | Upper and lower limbs | [9] |
4 | MS Kinect | Upper limb lesions | 10 P | Mixed: MS Kinect V2, videogame development, and web application | Arm | [10] |
5 | IMU | N/A | 11 H | Proposed system | Arm | [11] |
6 | IMU | N/A | N/A | Commercial: ArmeoSenso | N/A | [12] |
7 | IMU | Upper limb lesions | 10 P | Mixed: Myo bracelet and a developed videogame | Arm | [13] |
8 | MS Kinect | Stroke | 30 H | Commercial: MS Kinect V2 and Mystic Isle (videogame integrated to Kinect) | Upper part of the human body | [14] |
9 | MS Kinect | Stroke | 11 P | Mixed: MS Kinect and a developed videogame | Arm | [15] |
10 | MS Kinect | Stroke | 24 P | Mixed: MS Kinect and Recovery Rapids ™ (personalized videogame) | Arm | [16] |
11 | MS Kinect | ROM | 10 H | Mixed: MS Kinect and development of a personalized system | Arm | [17] |
12 | MS Kinect | Friedreich’s ataxia | 27 P, 43 H | Mixed: MS Kinect and development of a videogame. | Arm | [18] |
13 | IMU | Stroke | 29 P | Commercial: Bimeo | Arm | [19] |
14 | IMU | Stroke | 11 P | Commercial: ArmeoSenso. | Arm | [20] |
15 | MS Kinect | Stroke | 74 P | Commercial: JRS Wave | Human body | [21] |
16 | MS Kinect | Stroke | 18 P, 12 H | Proposed system | Upper part of the human body | [22] |
17 | MS Kinect | Energy expenditure | 19 H | Mixed: MS Kinect and development of a system | Human body | [23] |
18 | Other optical systems | Lesions due to brain injury | N/A | Mixed | Hand | [24] |
19 | Orthosis with IMU | Stroke | 7 P | Proposed system | Wrist and hand | [25] |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Alarcón-Aldana, A.C.; Callejas-Cuervo, M.; Bo, A.P.L. Upper Limb Physical Rehabilitation Using Serious Videogames and Motion Capture Systems: A Systematic Review. Sensors 2020, 20, 5989. https://doi.org/10.3390/s20215989
Alarcón-Aldana AC, Callejas-Cuervo M, Bo APL. Upper Limb Physical Rehabilitation Using Serious Videogames and Motion Capture Systems: A Systematic Review. Sensors. 2020; 20(21):5989. https://doi.org/10.3390/s20215989
Chicago/Turabian StyleAlarcón-Aldana, Andrea Catherine, Mauro Callejas-Cuervo, and Antonio Padilha Lanari Bo. 2020. "Upper Limb Physical Rehabilitation Using Serious Videogames and Motion Capture Systems: A Systematic Review" Sensors 20, no. 21: 5989. https://doi.org/10.3390/s20215989