High-Density Surface EMG-Based Gesture Recognition Using a 3D Convolutional Neural Network
Abstract
:1. Introduction
2. Materials and Methods
2.1. Data and Pre-Processing
2.2. Construction of 3D Convolutional Neural Network
2.2.1. 3D Convolution
2.2.2. 3D CNN Architecture
2.2.3. Network Training
2.3. Majority Voting
2.4. Experiments
3. Results and Discussion
4. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Hakonen, M.; Piitulainen, H.; Visala, A. Current state of digital signal processing in myoelectric interfaces and related applications. Biomed. Signal Process. Control 2015, 18, 334–359. [Google Scholar] [CrossRef] [Green Version]
- Xing, K.; Yang, P.; Huang, J.; Wang, Y.; Zhu, Q. A real-time EMG pattern recognition method for virtual myoelectric hand control. Neurocomputing 2014, 136, 345–355. [Google Scholar] [CrossRef]
- Mccool, P.; Petropoulakis, L.; Soraghan, J.J.; Chatlani, N. Improved pattern recognition classification accuracy for surface myoelectric signals using spectral enhancement. Biomed. Signal Process. Control 2015, 18, 61–68. [Google Scholar] [CrossRef] [Green Version]
- Duan, F.; Dai, L.; Chang, W.; Chen, Z.; Zhu, C.; Li, W. sEMG-based identification of hand motion commands using wavelet neural network combined with discrete wavelet transform. IEEE Trans. Ind. Electron. 2016, 63, 1923–1934. [Google Scholar] [CrossRef]
- Atzori, M.; Gijsberts, A.; Castellini, C.; Caputo, B.; Mittaz Hager, A.-G.; Elsig, S.; Giatsidis, G.; Bassetto, F.; Müller, H. Electromyography data for non-invasive naturally-controlled robotic hand prostheses. Sci. Data 2014, 1, 140053. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Shim, H.M.; Lee, S. Multi-channel electromyography pattern classification using deep belief networks for enhanced user experience. J. Cent. South Univ. 2015, 22, 1801–1808. [Google Scholar] [CrossRef]
- Monica, R.M.; Miguel, A.M.; Joan, F.A. High-density surface EMG maps from upper-arm and forearm muscles. J. Neuroeng. Rehabil. 2012, 9, 85. [Google Scholar]
- Atzori, M.; Cognolato, M.; Müller, H. Deep learning with convolutional neural networks applied to electromyography data: A resource for the classification of movements for prosthetic hands. Front. Neurorobot. 2016, 10, 9. [Google Scholar] [CrossRef] [Green Version]
- Huebner, A.; Faenger, B.; Schenk, P.; Scholle, H.C.; Anders, C. Alteration of Surface EMG amplitude levels of five major trunk muscles by defined electrode location displacement. J. Electromyogr. Kinesiol. 2015, 25, 214–223. [Google Scholar] [CrossRef]
- Rojas-Martnez, M.; Mañanas, M.A.; Alonso, J.F.; Merletti, R. Identification of isometric contractions based on high density EMG maps. J. Electromyogr. Kinesiol. 2013, 23, 33–42. [Google Scholar] [CrossRef]
- Geng, W.; Du, Y.; Jin, W.; Wei, W.; Hu, Y.; Li, J. Gesture recognition by instantaneous surface EMG images. Sci. Rep. 2016, 6, 36571. [Google Scholar] [CrossRef] [PubMed]
- Amma, C.; Krings, T.; Böer, J.; Schultz, T. Advancing muscle-computer interfaces with high-density electromyography. In Proceedings of the 33rd annual ACM Conference on Human Factors in Computing Systems, Seoul, Korea, 18–23 April 2015; pp. 929–938. [Google Scholar]
- Ghaderi, P.; Marateb, H.R. Muscle activity map reconstruction from high density surface EMG signals with missing channels using image inpainting and surface reconstruction methods. IEEE Trans. Biomed. Eng. 2017, 64, 1513–1523. [Google Scholar] [CrossRef] [PubMed]
- Angkoon, P.; Pornchai, P.; Chusak, L. Feature reduction and selection for EMG signal classification. Expert Syst. Appl. 2012, 39, 7420–7431. [Google Scholar]
- Angkoon, P.; Franck, Q.; Sylvie, C.; Christine, S.; Franck, T.-B.; Yann, L. EMG feature evaluation for improving myoelectric pattern recognition robustness. Expert Syst. Appl. 2013, 40, 4832–4840. [Google Scholar]
- Tkach, D.; Huang, H.; Kuiken, T.A. Study of stability of time-domain features for electromyographic pattern recognition. J. Neuroeng. Rehabil. 2010, 7, 21. [Google Scholar] [CrossRef] [Green Version]
- Chan, A.D.C.; Englehart, K.B. Continuous myoelectric control for powered prostheses using hidden markov models. IEEE Trans. Biomed. Eng. 2004, 52, 121–124. [Google Scholar] [CrossRef]
- Oskoei, M.A.; Hu, H. Support vector machine-based classification scheme for myoelectric control applied to upper limb. IEEE Trans. Biomed. Eng. 2008, 55, 1956–1965. [Google Scholar] [CrossRef]
- Khezri, M.; Jahed, M. A neuro-fuzzy inference system for sEMG-based identification of hand motion commands. IEEE Trans. Ind. Electron. 2011, 58, 1952–1960. [Google Scholar] [CrossRef]
- LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444. [Google Scholar] [CrossRef]
- Shrestha, A.; Mahmood, A. Review of deep learning algorithms and architectures. IEEE Access 2019, 7, 53040–53065. [Google Scholar] [CrossRef]
- Angkoon, P.; Erik, S. EMG pattern recognition in the era of big data and deep learning. Big Data Cogn. Comput. 2018, 2, 21. [Google Scholar]
- Mukhopadhyay, A.K.; Samui, S. An experimental study on upper limb position invariant EMG signal classification based on deep neural network. Biomed. Signal Process. Control 2020, 55, 101669. [Google Scholar] [CrossRef]
- Zhang, X.; Yang, Z.; Chen, T.; Chen, D.; Huang, M.-C. Cooperative sensing and wearable computing for sequential hand gesture recognition. IEEE Sens. J. 2019, 19, 5775–5783. [Google Scholar] [CrossRef]
- Allard, U.C.; Nougarou, F.; Fall, C.L.; Giguère, P.; Gosselin, C.; Laviolette, F.; Gosselin, B. A convolutional neural network for robotic arm guidance using sEMG based frequency-features. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea, 9–14 October 2016; pp. 2464–2470. [Google Scholar]
- Zhai, X.; Jelfs, B.; Chan, R.H.M.; Tin, C. Self-recalibrating surface EMG pattern recognition for neuroprosthesis control based on convolutional neural network. Front. Neurosci. 2017, 11, 279. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Hu, Y.; Wong, Y.; Wei, W.; Du, Y.; Kankanhalli, M.; Geng, W. A novel attention-based hybrid CNN-RNN architecture for sEMG-based gesture recognition. PLoS ONE 2018, 13, e0206049. [Google Scholar] [CrossRef] [PubMed]
- Wei, W.; Wong, Y.; Du, Y.; Hu, Y.; Kankanhalli, M.; Geng, W. A multi-stream convolutional neural network for sEMG-based gesture recognition in muscle-computer interface. Pattern Recognit. Lett. 2019, 119, 131–138. [Google Scholar] [CrossRef]
- Ji, S.; Xu, W.; Yang, M.; Yu, K. 3D convolutional neural networks for human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2013, 35, 221–231. [Google Scholar] [CrossRef] [Green Version]
- Al-Hammadi, M.; Muhammad, G.; Abdul, W.; Alsulaiman, M.; Hossain, M.S. Hand gesture recognition using 3D-CNN model. IEEE Consum. Electron. Mag. 2020, 9, 95–101. [Google Scholar] [CrossRef]
- Chen, J.; Zhang, X.; Cheng, Y.; Xi, N. Surface EMG based continuous estimation of human lower limb joint angles by using deep belief networks. Biomed. Signal Process. Control 2018, 40, 335–342. [Google Scholar] [CrossRef]
- Shan, H.; Zhang, Y.; Yang, Q.; Kruger, U.; Kalra, M.K.; Sun, L.; Cong, W.; Wang, G. 3-D convolutional encoder-decoder network for low-dose CT via transfer learning from a 2-D trained network. IEEE Trans. Med. Imaging 2018, 37, 1522–1534. [Google Scholar] [CrossRef]
- Tran, D.; Bourdev, L.; Fergus, R.; Torresani, L.; Paluri, M. Learning spatiotemporal features with 3D convolutional networks. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 7–13 December 2015; pp. 4489–4497. [Google Scholar]
- Ioffe, S.; Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on Machine Learning, Lille, France, 6–11 July 2015; pp. 448–456. [Google Scholar]
- Srivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; Salakhutdinov, R. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 2014, 15, 1929–1958. [Google Scholar]
- Jiao, Z.; Gao, X.; Wang, Y.; Li, J.; Xu, H. Deep convolutional neural networks for mental load classification based on EEG data. Pattern Recognit. 2018, 76, 582–595. [Google Scholar] [CrossRef]
10 | 20 | 30 | 40 | 60 | 80 | 100 | 120 | 150 | |
---|---|---|---|---|---|---|---|---|---|
Number1 | 3.08 | 8.71 | 17.11 | 22.75 | 36.79 | 50.80 | 64.86 | 78.87 | 101.29 |
Number2 | 4.05 | 7.88 | 15.36 | 19.20 | 26.88 | 38.18 | 49.50 | 57.18 | 72.31 |
40 | 50 | 60 | 70 | 80 | 90 | 100 | 110 | 120 | 130 | 140 | 150 | |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2D CNN + MV (%) 1 | 94.5 | 95.3 | 96.0 | 96.2 | 96.4 | 96.4 | 99.6 | 96.7 | 96.7 | 96.8 | 96.8 | 97.1 |
3D CNN + MV (%) 1 | 95.5 | 96.7 | 97.2 | 97.8 | 97.9 | 98.0 | 98.3 | 98.3 | 98.4 | 98.4 | 98.6 | 98.6 |
2D CNN + MV (%) 2 | 61.7 | 62.7 | 64.1 | 64.9 | 65.8 | 67.1 | 67.7 | 67.8 | 69.6 | 69.6 | 71.0 | 72.1 |
3D CNN + MV (%) 2 | 77.0 | 79.1 | 81.1 | 83.4 | 84.6 | 84.7 | 86.5 | 86.9 | 87.9 | 89.5 | 90.6 | 90.7 |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Chen, J.; Bi, S.; Zhang, G.; Cao, G. High-Density Surface EMG-Based Gesture Recognition Using a 3D Convolutional Neural Network. Sensors 2020, 20, 1201. https://doi.org/10.3390/s20041201
Chen J, Bi S, Zhang G, Cao G. High-Density Surface EMG-Based Gesture Recognition Using a 3D Convolutional Neural Network. Sensors. 2020; 20(4):1201. https://doi.org/10.3390/s20041201
Chicago/Turabian StyleChen, Jiangcheng, Sheng Bi, George Zhang, and Guangzhong Cao. 2020. "High-Density Surface EMG-Based Gesture Recognition Using a 3D Convolutional Neural Network" Sensors 20, no. 4: 1201. https://doi.org/10.3390/s20041201
APA StyleChen, J., Bi, S., Zhang, G., & Cao, G. (2020). High-Density Surface EMG-Based Gesture Recognition Using a 3D Convolutional Neural Network. Sensors, 20(4), 1201. https://doi.org/10.3390/s20041201