Training with Agency-Inspired Feedback from an Instrumented Glove to Improve Functional Grasp Performance
Abstract
:1. Introduction
2. Materials and Methods
2.1. Subjects
2.2. Equipment for Operating the Instrumented Glove
2.3. Additional Equipment for Running Functional Tasks
2.4. Experiment Protocol to Train Glove on Secure Grasp
2.5. Experiment Protocol for Functional Task
2.6. Data and Statistical Analysis
3. Results
3.1. ANN Training Results
3.2. Machine-Learning Detection of Secure Grasp
3.3. Glove Feedback Effects on Grasp Performance
4. Discussion
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Dobkin, B.H.; Dorsch, A. The promise of mHealth: Daily activity monitoring and outcome assessments by wearable sensors. Neurorehabilit. Neural Repair 2011, 25, 788–798. [Google Scholar] [CrossRef]
- Sigrist, R.; Rauter, G.; Riener, R.; Wolf, P. Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review. Psychon Bull. Rev. 2013, 20, 21–53. [Google Scholar] [CrossRef] [Green Version]
- Clark, D.; Schumann, F.; Mostofsky, S.H. Mindful movement and skilled attention. Front. Hum. Neurosci. 2015, 9. [Google Scholar] [CrossRef] [Green Version]
- Gómara-Toldrà, N.; Sliwinski, M.; Dijkers, M.P. Physical therapy after spinal cord injury: A systematic review of treatments focused on participation. J. Spinal Cord Med. 2014, 37, 371–379. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- McCulloch, K. Attention and dual-task conditions: Physical therapy implications for individuals with acquired brain injury. J. Neurol. Phys. Therapy 2007, 31, 104–118. [Google Scholar] [CrossRef] [PubMed]
- Dollar, A.M. Classifying Human Hand Use and the Activities of Daily Living. In The Human Hand as an Inspiration for Robot Hand Development; Balasubramanian, R., Santos, V.J., Eds.; Springer International Publishing: Cham, Switzerland, 2014; Volume 95, pp. 201–216. [Google Scholar]
- Dobkin, B.H. Behavioral self-management strategies for practice and exercise should be included in neurologic rehabilitation trials and care. Curr. Opin. Neurol. 2016, 29, 693. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Merians, A.S.; Poizner, H.; Boian, R.; Burdea, G.; Adamovich, S. Sensorimotor Training in a Virtual Reality Environment: Does It Improve Functional Recovery Poststroke? Neurorehabilit. Neural Repair 2006, 20, 252–267. [Google Scholar] [CrossRef]
- Cappello, L.; Meyer, J.T.; Galloway, K.C.; Peisner, J.D.; Granberry, R.; Wagner, D.A.; Engelhardt, S.; Paganoni, S.; Walsh, C.J. Assisting hand function after spinal cord injury with a fabric-based soft robotic glove. J. Neuroeng. Neurorehabilit. 2018, 15, 59. [Google Scholar] [CrossRef] [PubMed]
- Heo, P.; Gu, G.M.; Lee, S.-j.; Rhee, K.; Kim, J. Current hand exoskeleton technologies for rehabilitation and assistive engineering. Int. J. Precis. Eng. Manuf. 2012, 13, 807–824. [Google Scholar] [CrossRef]
- Nassour, J.; Amirabadi, H.G.; Weheabby, S.; Al Ali, A.; Lang, H.; Hamker, F. A Robust Data-Driven Soft Sensory Glove for Human Hand Motions Identification and Replication. IEEE Sens. J. 2020, 20, 12972–12979. [Google Scholar] [CrossRef]
- Weber, P.; Rueckert, E.; Calandra, R.; Peters, J.; Beckerle, P. A low-cost sensor glove with vibrotactile feedback and multiple finger joint and hand motion sensing for human-robot interaction. In Proceedings of the 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), New York, NY, USA, 26–31 August 2016; pp. 99–104. [Google Scholar]
- Roa, M.A.; Suárez, R. Grasp quality measures: Review and performance. Auton. Robot. 2015, 38, 65–88. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Ghazaei, G.; Alameer, A.; Degenaar, P.; Morgan, G.; Nazarpour, K. Deep learning-based artificial vision for grasp classification in myoelectric hands. J. Neural Eng. 2017, 14, 036025. [Google Scholar] [CrossRef]
- Moore, J.W. What Is the Sense of Agency and Why Does it Matter? Front. Psychol. 2016, 7, 1272. [Google Scholar] [CrossRef]
- Nataraj, R.; Sanford, S.; Shah, A.; Liu, M. Agency and Performance of Reach-to-Grasp with Modified Control of a Virtual Hand: Implications for Rehabilitation. Front. Hum. Neurosci. 2020, 14. [Google Scholar] [CrossRef]
- Haggard, P.; Clark, S.; Kalogeras, J. Voluntary action and conscious awareness. Nat. Neurosci. 2002, 5, 382–385. [Google Scholar] [CrossRef]
- Alon, G.; Levitt, A.F.; McCarthy, P.A. Functional electrical stimulation enhancement of upper extremity functional recovery during stroke rehabilitation: A pilot study. Neurorehabilit. Neural Repair 2007, 21, 207–215. [Google Scholar] [CrossRef]
- David, F.J.; Baranek, G.T.; Giuliani, C.A.; Mercer, V.S.; Poe, M.D.; Thorpe, D.E. A pilot study: Coordination of precision grip in children and adolescents with high functioning autism. Pediatric Phys. Ther. 2009, 21, 205–211. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Lin, K.-c.; Chen, Y.-a.; Chen, C.-l.; Wu, C.-y.; Chang, Y.-f. The effects of bilateral arm training on motor control and functional performance in chronic stroke: A randomized controlled study. Neurorehabilit. Neural Repair 2010, 24, 42–51. [Google Scholar] [CrossRef] [PubMed]
- U.S. Food & Drug. Information Sheet Guidance for Institutional Review Boards (IRBs), Clinical Investigators, and Sponsors. Available online: https://www.fda.gov/science-research/guidance-documents-including-information-sheets-and-notices/information-sheet-guidance-institutional-review-boards-irbs-clinical-investigators-and-sponsors (accessed on 27 January 2021).
- Dixon, J.R. The international conference on harmonization good clinical practice guideline. Quality Assurance 1999, 6, 65–74. [Google Scholar] [CrossRef]
- Corbetta, D.; Sirtori, V.; Castellini, G.; Moja, L.; Gatti, R. Constraint-induced movement therapy for upper extremities in people with stroke. Cochrane Database Syst. Rev. 2015. [Google Scholar] [CrossRef] [Green Version]
- Nataraj, R.; Li, Z.-M. Integration of marker and force data to compute three-dimensional joint moments of the thumb and index finger digits during pinch. Comput. Methods Biomech. Biomed. Eng. 2015, 18, 592–606. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Wulf, G.; Shea, C.; Lewthwaite, R. Motor skill learning and performance: A review of influential factors. Med. Educ. 2010, 44, 75–84. [Google Scholar] [CrossRef] [PubMed]
- Dewey, J.A.; Knoblich, G. Do implicit and explicit measures of the sense of agency measure the same thing? PLoS ONE 2014, 9, e110118. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Enders, L.R.; Hur, P.; Johnson, M.J.; Seo, N.J. Remote vibrotactile noise improves light touch sensation in stroke survivors’ fingertips via stochastic resonance. J. Neuroeng. Rehabil. 2013, 8. [Google Scholar] [CrossRef] [PubMed] [Green Version]
ANN Data Set | ANOVA | Tukey post hoc | ||||||
---|---|---|---|---|---|---|---|---|
METRIC | Training (Tr) | Validation (Va) | Testing (Te) | p-value | F-stat | Tr vs Va | Tr vs Te | Va vs Te |
Cross-Entropy | 1.48 ± 0.65 | 3.73 ± 1.89 | 3.73 ± 1.88 | 8.7 × 10−5 | 11.4 | 4.0 × 10−4 | 4.0 × 10−4 | 1 |
Error (%) | 7.22 ± 5.67 | 7.22 ± 5.96 | 7.38 ± 5.73 | 0.99 | 0.0042 | 1 | 0.99 | 0.99 |
METRIC | Standard MSE | Rounded MSE | T-test p-value | T-statistic | ||||
Mean Squared Error (MSE, unitless) | 0.062 ± 0.057 | 0.083 ± 0.088 | 3.6 × 10−3 | 3.41 |
Pinch ANN (1) | Pinch Analytical (2) | Tri-pod ANN (3) | Whole-Hand ANN (4) | ANOVA F-stat | ANOVA p-value |
---|---|---|---|---|---|
87.3 ± 1.0% | 63.3 ± 2.4% | 91.3 ± 5.8% | 89.8 ± 5.5% | 29.4 | 1.1 × 10−4 |
Post hoc 1 vs 2 | Post hoc 1 vs 3 | Post hoc 1 vs 4 | Post hoc 2 vs 3 | Post hoc 2 vs 4 | Post hoc 3 vs 4 |
5.3 × 10−4 | 0.67 | 0.88 | 1.8 × 10−4 | 2.6 × 10−4 | 0.97 |
Performance metric: COMPLETION TIME (sec)→ ANOVA p-value = 4.9× 10−4, F-stat = 5.51 | |||||
Feedback Mode | Time 1 | Time 2 | Difference | Std Err | Post hoc p-value |
NF | 1 | 2 | −0.063 | 0.043 | 0.32 |
NF | 1 | 3 | −0.016 | 0.030 | 0.86 |
NF | 2 | 3 | 0.048 | 0.040 | 0.47 |
IF | 1 | 2 | 0.12 | 0.043 | 0.021 |
IF | 1 | 3 | 0.083 | 0.030 | 0.022 |
IF | 2 | 3 | −0.037 | 0.040 | 0.63 |
IBF | 1 | 2 | 0.18 | 0.043 | 0.00039 |
IBF | 1 | 3 | 0.09 | 0.030 | 0.012 |
IBF | 2 | 3 | −0.09 | 0.040 | 0.077 |
Performance metric: PATHLENGTH (m)→ ANOVA p-value = 0.019, F-stat = 3.10 | |||||
Feedback Mode | Time 1 | Time 2 | Difference | Std Err | Post hoc p-value |
NF | 1 | 2 | −0.0052 | 0.0022 | 0.057 |
NF | 1 | 3 | −0.0023 | 0.0023 | 0.59 |
NF | 2 | 3 | 0.0029 | 0.0020 | 0.31 |
IF | 1 | 2 | 0.00032 | 0.0022 | 0.98 |
IF | 1 | 3 | 0.0017 | 0.0023 | 0.75 |
IF | 2 | 3 | 0.0013 | 0.0020 | 0.78 |
IBF | 1 | 2 | 0.0045 | 0.0022 | 0.11 |
IBF | 1 | 3 | 0.0065 | 0.0023 | 0.018 |
IBF | 2 | 3 | 0.0020 | 0.0020 | 0.57 |
Performance metric: PLACEMENT ERROR (m)→ ANOVA p-value = 0.99, F-stat = 0.033 | |||||
Feedback Mode | Time 1 | Time 2 | Difference | Std Err | Post hoc p-value |
NF | 1 | 2 | −0.0075 | 0.0037 | N/A |
NF | 1 | 3 | −0.00086 | 0.00044 | N/A |
NF | 2 | 3 | 0.0066 | 0.0037 | N/A |
IF | 1 | 2 | −0.0061 | 0.0037 | N/A |
IF | 1 | 3 | −0.00018 | 0.00044 | N/A |
IF | 2 | 3 | 0.0059 | 0.0037 | N/A |
IBF | 1 | 2 | −0.0062 | 0.0037 | N/A |
IBF | 1 | 3 | −0.00007 | 0.00044 | N/A |
IBF | 2 | 3 | 0.0061 | 0.0037 | N/A |
Intra-Training Rate/Slope for COMPLETION TIME (sec per trial) | |||||||
Feedback Mode | ANOVA | Tukey post hoc | |||||
NF | IF | IBF | p-value | F-stat | NF vs IF | NF vs IBF | IF vs IBF |
0.0067 ± 0.006 | −0.0023 ± 0.0064 | −0.0048 ± 0.0061 | 3.6 × 10−6 | 16.5 | 2.6 × 10−4 | 5.0 × 10−6 | 0.47 |
Intra-Training Rate/Slope for PATHLENGTH (m per trial) | |||||||
Feedback Mode | ANOVA | Tukey post hoc | |||||
NF | IF | IBF | p-value | F-stat | NF vs IF | NF vs IBF | IF vs IBF |
2.64 × 10−4 ± 3.02 × 10−4 | −1.15 × 10−4 ± 4.44 × 10−4 | −2.58 × 10−4 ± 3.12 × 10−4 | 4.84 × 10−4 | 8.9 | 0.076 | 3 × 10−4 | 0.12 |
Intra-Training Rate/Slope for PLACEMENT ERROR (m per trial) | |||||||
Feedback Mode | ANOVA | Tukey post hoc | |||||
NF | IF | IBF | p-value | F-stat | NF vs IF | NF vs IBF | IF vs IBF |
2.40 × 10−5 ± 1.65 × 10−4 | −1.32 × 10−5 ± 1.03 × 10−4 | −8.63 × 10−6 ± 9.67 × 10−5 | 0.6434 | 0.45 | N/A | N/A | N/A |
Post-Training Effect (Difference After Training from Before) for COMPLETION TIME (sec) | |||||||
Feedback Mode | ANOVA | Tukey post hoc | |||||
NF | IF | IBF | p-value | F-stat | NF vs IF | NF vs IBF | IF vs IBF |
0.0156 ± 0.1312 | −0.0831 ± 0.105 | −0.090 ± 0.135 | 0.028 | 3.8 | 0.064 | 0.043 | 0.98 |
Post-Training Effect (Difference After Training from Before) for PATHLENGTH (m) | |||||||
Feedback Mode | ANOVA | Tukey post hoc | |||||
NF | IF | IBF | p-value | F-stat | NF vs IF | NF vs IBF | IF vs IBF |
0.0022 ± 0.0072 | −0.0017 ± 0.0123 | −0.0065 ± 0.0081 | 0.034 | 3.6 | 0.46 | 0.026 | 0.30 |
Post-Training Effect (Difference After Training from Before) for PLACEMENT ERROR (m) | |||||||
Feedback Mode | ANOVA | Tukey post hoc | |||||
NF | IF | IBF | p-value | F-stat | NF vs IF | NF vs IBF | IF vs IBF |
8.65 × 10−4 ± 0.0024 | 1.81 × 10−4 ± 0.0014 | 7.06 × 10−5 ± 0.0016 | 0.40 | 0.94 | N/A | N/A | N/A |
Intra-Training Rate/Slope for COMPLETION TIME (sec) | |||||
NF | IF | IBF | |||
p-value | T-stat | p-value | T-stat | p-value | T-stat |
0.0003 | 4.65 | 0.15 | −1.50 | 0.0053 | −3.22 |
Intra-Training Rate/Slope for PATHLENGTH (m per trial) | |||||
NF | IF | IBF | |||
p-value | T-stat | p-value | T-stat | p-value | T-stat |
0.0024 | 3.59 | 0.92 | −0.10 | 0.0036 | −3.41 |
Intra-Training Rate/Slope for PLACEMENT ERROR (m per trial) | |||||
NF | IF | IBF | |||
p-value | T-stat | p-value | T-stat | p-value | T-stat |
0.56 | 0.60 | 0.60 | −0.53 | 0.72 | −0.368 |
Post-Training Effect (Difference After Training from Before) for COMPLETION TIME (sec) | |||||
NF | IF | IBF | |||
p-value | T-stat | p-value | T-stat | p-value | T-stat |
0.63 | 0.49 | 0.0049 | −3.27 | 0.0139 | −2.76 |
Post-Training Effect (Difference After Training from Before) for PATHLENGTH (m) | |||||
NF | IF | IBF | |||
p-value | T-stat | p-value | T-stat | p-value | T-stat |
0.22 | 1.28 | 0.59 | −0.55 | 0.0044 | −3.32 |
Post-Training Effect (Difference After Training from Before) for PLACEMENT ERROR (m) | |||||
NF | IF | IBF | |||
p-value | T-stat | p-value | T-stat | p-value | T-stat |
0.15 | 1.51 | 0.59 | 0.55 | 0.86 | 0.18 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, M.; Wilder, S.; Sanford, S.; Saleh, S.; Harel, N.Y.; Nataraj, R. Training with Agency-Inspired Feedback from an Instrumented Glove to Improve Functional Grasp Performance. Sensors 2021, 21, 1173. https://doi.org/10.3390/s21041173
Liu M, Wilder S, Sanford S, Saleh S, Harel NY, Nataraj R. Training with Agency-Inspired Feedback from an Instrumented Glove to Improve Functional Grasp Performance. Sensors. 2021; 21(4):1173. https://doi.org/10.3390/s21041173
Chicago/Turabian StyleLiu, Mingxiao, Samuel Wilder, Sean Sanford, Soha Saleh, Noam Y. Harel, and Raviraj Nataraj. 2021. "Training with Agency-Inspired Feedback from an Instrumented Glove to Improve Functional Grasp Performance" Sensors 21, no. 4: 1173. https://doi.org/10.3390/s21041173
APA StyleLiu, M., Wilder, S., Sanford, S., Saleh, S., Harel, N. Y., & Nataraj, R. (2021). Training with Agency-Inspired Feedback from an Instrumented Glove to Improve Functional Grasp Performance. Sensors, 21(4), 1173. https://doi.org/10.3390/s21041173