Inferring Static Hand Poses from a Low-Cost Non-Intrusive sEMG Sensor
Abstract
:1. Introduction
- The creation of a public dataset containing 35 subjects with six dissimilar hand gestures.
- A method for discrete hand pose classification with sEMG signals. The proposed method is based on deep learning and it obtains a high recognition accuracy.
2. Background and Related Work
3. sEMG Dataset Recording and Processing
3.1. EMG Sensor Type Discussion
3.2. Recording Hardware
3.3. Recording and Labeling Data
4. System Description
Gated Recurrent Unit Network
5. Experiments and Results
6. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Cook, A.M.; Polgar, J.M. Essentials of Assistive Technologies; ELSEVIER Mosby: Amsterdam, The Netherlands, 2012. [Google Scholar]
- Costa, A.; Martinez-Martin, E.; Cazorla, M.; Julian, V. PHAROS—PHysical Assistant RObot System. Sensors 2018, 18, 2633. [Google Scholar] [CrossRef]
- LeBlanc, M. The LN-4 Prosthetic Hand. In Give Hope—Give a Hand; The Ellen Meadows Prosthetic Hand Foundation: San Francisco, CA, USA, 2008. [Google Scholar]
- Momen, K.; Krishnan, S.; Chau, T. Real-time classification of forearm electromyographic signals corresponding to user-selected intentional movements for multifunction prosthesis control. IEEE Trans. Neural Syst. Rehabil. Eng. 2007, 15, 535–542. [Google Scholar] [CrossRef]
- Amsuss, S.; Goebel, P.M.; Jiang, N.; Graimann, B.; Paredes, L.; Farina, D. Self-correcting pattern recognition system of surface EMG signals for upper limb prosthesis control. IEEE Trans. Biomed. Eng. 2014, 61, 1167–1176. [Google Scholar] [CrossRef]
- Boostani, R.; Moradi, M.H. Evaluation of the forearm EMG signal features for the control of a prosthetic hand. Physiol. Meas. 2003, 24, 309. [Google Scholar] [CrossRef]
- Gijsberts, A.; Atzori, M.; Castellini, C.; Müller, H.; Caputo, B. Movement Error Rate for Evaluation of Machine Learning Methods for sEMG-Based Hand Movement Classification. IEEE Trans. Neural Syst. Rehabil. Eng. 2014, 22, 735–744. [Google Scholar] [CrossRef]
- Atzori, M.; Gijsberts, A.; Castellini, C.; Caputo, B.; Hager, A.G.M.; Elsig, S.; Giatsidis, G.; Bassetto, F.; Müller, H. Electromyography data for non-invasive naturally-controlled robotic hand prostheses. Sci. Data 2014, 1, 140053. [Google Scholar] [CrossRef] [Green Version]
- Asimov, I. The Three Laws of Robotics; Gnome Press: New York, NY, USA, 1941. [Google Scholar]
- HRI ’11: Proceedings of the 6th International Conference on Human-robot Interaction, Lausanne, Switzerland, 6–9 March 2011; ACM: New York, USA, 2011; p. 609114.
- Scholtz, J. Human Robot Interactions: Creating Synergistic Cyber Forces; AAAI Technical Report FS-02-03; Springer: Dordrecht, The Netherlands, 2002. [Google Scholar]
- Yang, C.; Chang, S.; Liang, P.; Li, Z.; Su, C.Y. Teleoperated robot writing using EMG signals. In Proceedings of the 2015 IEEE International Conference on Information and Automation, Lijiang, China, 8–10 August 2015; pp. 2264–2269. [Google Scholar]
- Reddivari, H.; Yang, C.; Ju, Z.; Liang, P.; Li, Z.; Xu, B. Teleoperation control of Baxter robot using body motion tracking. In Proceedings of the 2014 International Conference on Multisensor Fusion and Information Integration for Intelligent Systems (MFI), Beijing, China, 28–29 September 2014; pp. 1–6. [Google Scholar] [CrossRef]
- Xu, Y.; Yang, C.; Liang, P.; Zhao, L.; Li, Z. Development of a hybrid motion capture method using MYO armband with application to teleoperation. In Proceedings of the 2016 IEEE International Conference on Mechatronics and Automation, Harbin, China, 7–10 August 2016; pp. 1179–1184. [Google Scholar] [CrossRef]
- Bisi, S.; De Luca, L.; Shrestha, B.; Yang, Z.; Gandhi, V. Development of an EMG-Controlled Mobile Robot. Robotics 2018, 7, 36. [Google Scholar] [CrossRef]
- Allard, U.C.; Nougarou, F.; Fall, C.L.; Giguère, P.; Gosselin, C.; Laviolette, F.; Gosselin, B. A convolutional neural network for robotic arm guidance using semg based frequency-features. In Proceedings of the Intelligent Robots and Systems (IROS), Daejeon, Korea, 9–14 October 2016; pp. 2464–2470. [Google Scholar]
- Ahsan, M.D.; Ibrahimy, M.I.; Khalifa, O.O. Advances in Electromyogram Signal Classification to Improve the Quality of Life for the Disabled and Aged People. J. Comput. Sci. 2010, 6, 706–715. [Google Scholar] [CrossRef]
- Brose, S.W.; Weber, D.J.; Salatin, B.A.; Grindle, G.G.; Wang, H.; Vazquez, J.J.; Cooper, R.A. The role of assistive robotics in the lives of persons with disability. Am. J. Phys. Med. Rehabil. 2010, 89, 509–521. [Google Scholar] [CrossRef]
- Blok, J.H.; Van Dijk, J.P.; Drost, G.; Zwarts, M.J.; Stegeman, D.F. A high-density multichannel surface electromyography system for the characterization of single motor units. Rev. Sci. Instrum. 2002, 73, 1887–1897. [Google Scholar] [CrossRef]
- Drost, G.; Stegeman, D.F.; van Engelen, B.G.; Zwarts, M.J. Clinical applications of high-density surface EMG: A systematic review. J. Electromyogr Kinesiol. 2006, 16, 586–602. [Google Scholar] [CrossRef]
- Saponas, T.S.; Tan, D.S.; Morris, D.; Balakrishnan, R. Demonstrating the Feasibility of Using Forearm Electromyography for Muscle-computer Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Florence, Italy, 5–10 April 2008; ACM: New York, NY, USA, 2008; pp. 515–524. [Google Scholar] [CrossRef]
- Rojas-Martínez, M.; Mañanas, M.A.; Alonso, J.F.; Merletti, R. Identification of isometric contractions based on High Density EMG maps. J. Electromyogr. Kinesiol. 2013, 23, 33–42. [Google Scholar] [CrossRef]
- Zhang, X.; Zhou, P. High-density myoelectric pattern recognition toward improved stroke rehabilitation. IEEE Trans. Biomed. Eng. 2012, 59, 1649–1657. [Google Scholar] [CrossRef]
- Amma, C.; Krings, T.; Böer, J.; Schultz, T. Advancing Muscle-Computer Interfaces with High-Density Electromyography. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, Korea, 18–23 April 2015; ACM: New York, NY, USA, 2015; pp. 929–938. [Google Scholar] [CrossRef]
- Du, Y.; Wong, Y.; Jin, W.; Wei, W.; Hu, Y.; Kankanhalli, M.; Geng, W. Semi-Supervised Learning for Surface EMG-based Gesture Recognition. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, Melbourne, Australia, 19–25 August 2017; pp. 1624–1630. [Google Scholar] [CrossRef]
- Staudenmann, D.; Kingma, I.; Stegeman, D.; Van Dieen, J. Towards optimal multi-channel EMG electrode configurations in muscle force estimation: A high density EMG study. J. Electromyogr. Kinesiol. 2005, 15, 1–11. [Google Scholar] [CrossRef]
- Atzori, M.; Gijsberts, A.; Heynen, S.; Mittaz Hager, A.G.; Deriaz, O.; Van Der Smagt, P.; Castellini, C.; Caputo, B.; Müller, H. Resource for the Biorobotics Community (Ninapro). In Proceedings of the IEEE International Conference on Biomedical Robotics and Biomechatronics, Rome, Italy, 24–27 June 2012. [Google Scholar]
- Geng, W.; Du, Y.; Jin, W.; Wei, W.; Hu, Y.; Li, J. Gesture recognition by instantaneous surface EMG images. Sci. Rep. 2016, 6, 36571. [Google Scholar] [CrossRef] [Green Version]
- Rojas-Martínez, M.; Mañanas, M.A.; Alonso, J.F. High-density surface EMG maps from upper-arm and forearm muscles. Neuroeng. Rehabil. 2012, 9, 85. [Google Scholar] [CrossRef] [Green Version]
- Pizzolato, S.; Tagliapietra, L.; Cognolato, M.; Reggiani, M.; Müller, H.; Atzori, M. Comparison of six electromyography acquisition setups on hand movement classification tasks. PLoS ONE 2017, 12, e0186132. [Google Scholar] [CrossRef]
- Oskoei, M.A.; Hu, H. Myoelectric control systems—A survey. Biomed. Signal Process. Control 2007, 2, 275–294. [Google Scholar] [CrossRef]
- Atzori, M.; Cognolato, M.; Müller, H. Deep Learning with Convolutional Neural Networks Applied to Electromyography Data: A Resource for the Classification of Movements for Prosthetic Hands. Front. Neurorobot. 2016, 10, 9. [Google Scholar] [CrossRef]
- Wang, K.J.; Tung, H.W.; Huang, Z.; Thakur, P.; Mao, Z.H.; You, M.X. EXGbuds: Universal Wearable Assistive Device for Disabled People to Interact with the Environment Seamlessly. In Proceedings of the HRI `18 Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, Chicago, IL, USA, 5–8 March 2018; pp. 369–370. [Google Scholar]
- Bahdanau, D.; Cho, K.; Bengio, Y. Neural machine translation by jointly learning to align and translate. arXiv, 2014; arXiv:1409.0473. [Google Scholar]
- Mikolov, T.; Karafiát, M.; Burget, L.; Černocký, J.; Khudanpur, S. Recurrent neural network based language model. In Proceedings of the 11th Annual Conference of the International Speech Communication Association, Makuhari, Chiba, Japan, 26–30 September 2010. [Google Scholar]
- Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural Comput. 1997, 9, 1735–1780. [Google Scholar] [CrossRef]
- Sutskever, I.; Vinyals, O.; Le, Q.V. Sequence to Sequence Learning with Neural Networks. In Advances in Neural Information Processing Systems; Neural Information Processing Systems Foundation, Inc.: Denver, CO, USA, 2014. [Google Scholar]
- Cho, K.; Van Merriënboer, B.; Gulcehre, C.; Bahdanau, D.; Bougares, F.; Schwenk, H.; Bengio, Y. Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation. In Proceedings of the Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, Doha, Qatar, 25 October 2014. [Google Scholar]
- Yin, W.; Kann, K.; Yu, M.; Schütze, H. Comparative Study of CNN and RNN for Natural Language Processing. CoRR 2017, arXiv:1702.01923. [Google Scholar]
- Chung, J.; Gülçehre, Ç.; Cho, K.; Bengio, Y. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. CoRR 2014, arXiv:1412.3555. [Google Scholar]
- Van Dijk Johannes, P. High-density Surface EMG: Techniques and Applications at a Motor Unit Level. Biocybern. Biomed. Eng. 2012, 32, 3–27. [Google Scholar]
- Guger, C.; Krausz, G.; Allison, B.; Edlinger, G. Comparison of Dry and Gel Based Electrodes for P300 Brain–Computer Interfaces. Front. Neurosci. 2012, 6, 60. [Google Scholar] [CrossRef]
- Gonzalo Pomboza-Junez, J.H.T. Hand Gesture Recognition Based on sEMG Signals Using Support Vector Machines; Consumer Electronics: Berlin, Germany, 2016. [Google Scholar]
- Allard, U.C.; Fall, C.L.; Drouin, A.; Campeau-Lecours, A.; Gosselin, C.; Glette, K.; Laviolette, F.; Gosselin, B. Deep Learning for Electromyographic Hand Gesture Signal Classification by Leveraging Transfer Learning. CoRR 2018, arXiv:1801.07756. [Google Scholar]
- Côté-Allard, U.; Fall, C.L.; Campeau-Lecours, A.; Gosselin, C.; Laviolette, F.; Gosselin, B. Transfer Learning for sEMG Hand Gestures Recognition Using Convolutional Neural Networks. In Proceedings of the IEEE International Conference on Systems, Banff, AB, Canada, 5–8 October 2017. [Google Scholar]
- Adafruit. Myo Armband Teardown. 2016. Available online: https://learn.adafruit.com/myo-armband-teardown/inside-myo (accessed on 14 September 2018).
- Farina, D.; Cescon, C.; Merletti, R. Influence of anatomical, physical, and detection-system parameters on surface EMG. Biol. Cybern. 2002, 86, 445–456. [Google Scholar] [CrossRef]
- Kuiken, T.A.; Lowery, M.M.; Stoykov, N.S. The effect of subcutaneous fat on myoelectric signal amplitude and cross-talk. Prosthet. Orthot. Int. 2003, 27, 48–54. [Google Scholar] [CrossRef]
- Ruder, S. An overview of gradient descent optimization algorithms. CoRR 2016, arXiv:1609.04747. [Google Scholar]
- Johnson, N.E.; Sowden, J.; Dilek, N.; Eichinger, K.; Burns, J.; Mcdermott, M.P.; Shy, M.E.; Herrmann, D.N. Prospective Study of Muscle Cramps in Charcot Marie Tooth Disease. Muscle Nerve 2015, 51, 485–488. [Google Scholar] [CrossRef]
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Nasri, N.; Orts-Escolano, S.; Gomez-Donoso, F.; Cazorla, M. Inferring Static Hand Poses from a Low-Cost Non-Intrusive sEMG Sensor. Sensors 2019, 19, 371. https://doi.org/10.3390/s19020371
Nasri N, Orts-Escolano S, Gomez-Donoso F, Cazorla M. Inferring Static Hand Poses from a Low-Cost Non-Intrusive sEMG Sensor. Sensors. 2019; 19(2):371. https://doi.org/10.3390/s19020371
Chicago/Turabian StyleNasri, Nadia, Sergio Orts-Escolano, Francisco Gomez-Donoso, and Miguel Cazorla. 2019. "Inferring Static Hand Poses from a Low-Cost Non-Intrusive sEMG Sensor" Sensors 19, no. 2: 371. https://doi.org/10.3390/s19020371