Experimental Study of Real-Time Classification of 17 Voluntary Movements for Multi-Degree Myoelectric Prosthetic Hand
Abstract
:Featured Application
Abstract
1. Introduction
2. Related Works
- How well can the state-of-the-art classification scheme classify 17 types of movements?
- Can a consumer grade EMG sensor serve as a modality in myoelectric prosthetic control?
- How long can the user control the prosthetic hands until the accuracy drops significantly?
3. Data Acquisition and Preprocessing of EMG Signals
4. Signal Processing Algorithm for Classification of 17 Motions
4.1. Feature Extraction Based on Spectral Analysis
4.2. Training of Artificial Neural Network Classifier
5. Online Classification Experiment
- (1)
- Classification accuracy of 17 motions
- (2)
- Classification accuracy without re-training the classifier
- (3)
- Classification accuracy of 9 motions (motions that are required to complete most daily tasks).
5.1. Experimental Conditions
5.1.1. Participants
5.1.2. Experiment Protocol
- (1)
- Classification of 17 voluntary movements: This classification scenario aims to evaluate the classification performance of the proposed system. The result of this experiment will give us a hint of the extent of using state-of-the-art pattern recognition scheme to classify 17 types of motions. It will also help demonstrate the validity of low-cost hardware as a modality for myoelectric control. In this experiment, ANN is trained using all five training data sets and the accuracy is evaluated using all 10 testing data sets.
- (2)
- Classification of 17 movements using testing data obtain on a different day: This classification scenario aims to determine how well the proposed method can perform without having to re-train the classifier. To do so, the ANN parameters from day1 are used to classify the testing data obtained in day 2, and vice-versa.
- (3)
- Classification of 9 movements: This scenario is intended to see how much the accuracy improves when the number of classes are reduced to nine. The 9 motions involved in this classification scenario are pronation, supination, hand grasping, hand opening, flexion and extension of thumb and index finger, and a rest state. These motions were selected because they are the basic hand/arm movements needed to carry out most daily tasks, e.g., turning a door knob, grabbing and relocating an object, picking up a small object. The same EMG data involved in the previous classifications are used in this scenario. Since both training and testing data sets contains EMG signals for 17 motions, the epoch of the non-target motions along with subsequent rest period was cut off and the remaining epoch were merged together to form a new time series data set lasting 100 s.
- (4)
- Classification of 9 movements using testing data obtain on a different day: Similar to the scenario 2, this scenario hopes to test the functionality of the proposed algorithm without having to re-train the classifier.
5.2. Results and Discussions
5.2.1. Classification of 17 Voluntary Movements
- (1)
- How well can the state-of-the-art classification scheme classify 17 types of movements?
- (2)
- Can a consumer grade EMG sensor serve as a modality in myoelectric prosthetic control?
- (3)
- How long can the user control the prosthetic hands until the accuracy drops significantly?
5.2.2. Classification of 17 Movements Using Testing Data Obtain on a Different Day
5.2.3. Classification of 9 Movements
5.2.4. Classification of 9 Movements Using Testing Data Obtain on a Different Day
5.2.5. Erroneous Classification
6. Conclusions
Author Contributions
Conflicts of Interest
References
- Chao, E.Y.S.; An, K.-N.; Cooney, W.P.; Linscheid, R. Biomechanics of the Hand: A Basic Research Study; World Scientific Publishing: Teaneck, NJ, USA, 1989; pp. 5–75. ISBN 978-9971-5-0103-7. [Google Scholar]
- Pillet, J.; Didierjean-Pillet, A. Aesthetic hand prosthesis: Gadget or therapy? Presentation of a new classification. J. Hand Surg. Br. Eur. Vol. 2001, 6, 523–528. [Google Scholar] [CrossRef]
- Ministry of Health, Labour and Welfare, Social Welfare and War Victims’ Relief Bureau, Department of Health and Welfare for Persons with Disabilities Policy Planning Division. Report on Survey on Persons with Physical Disability 2006; Ministry of Health, Labour and Welfare: Tokyo, Japan, 2006; pp. 3–4. (In Japanese)
- Pezzin, L.E.; Dillingham, T.R.; Mackenzie, E.J.; Ephraim, P.; Rossbach, P. Use and satisfaction with prosthetic limb devices and related services. Arch. Phys. Med. Rehabil. 2004, 85, 723–729. [Google Scholar] [CrossRef]
- Biddiss, E.A.; Chau, T.T. Upper limb prosthesis use and abandonment: A survey of the last 25 years. Prosthet. Orthot. Int. 2007, 31, 236–257. [Google Scholar] [CrossRef]
- Bionic Hand with 14 Grip Patterns That Makes Common Tasks Easy—Bebionic. Available online: http://bebionic.com/the_hand/grip_patterns (accessed on 18 July 2016).
- I-Limb Ultra|Touch Bionics. Available online: http://www.touchbionics.com/products/active-prostheses/i-limb-ultra (accessed on 18 July 2016).
- Wiste, T.E.; Dalley, S.A.; Withrow, T.J.; Goldfarb, M. Design of a multifunctional anthropomorphic prosthetic hand with extrinsic actuation. IEEE/ASME Trans. Mechatron. 2009, 14, 699–706. [Google Scholar] [CrossRef]
- Losier, Y.; Clawson, A.; Wilson, A.; Scheme, E.; Englehart, K.; Kyberd, P.; Hudgins, B. An overview of the UNB hand system. In Proceedings of the 2011 MyoElectric Controls/Powered Prosthetics Symposium, Fredericton, NB, Canada, 14–19 August 2011. [Google Scholar]
- Belter, J.T.; Dollar, A.M. Novel differential mechanism enabling two DOF from a single actuator: Application to a prosthetic hand. In Proceedings of the IEEE International Conference on Rehabilitation Robotics (ICORR), Seattle, DC, USA, 24–26 June 2013; pp. 1–6. [Google Scholar] [CrossRef]
- Cipriani, C.; Controzzi, M.; Carrozza, M.C. The SmartHand transradial prosthesis. J. NeuroEng. Rehabil. 2011, 8, 1–13. [Google Scholar] [CrossRef]
- Liu, H.; Wu, K.; Meusel, P.; Seitz, N.; Hirzinger, G.; Jin, M.H.; Liu, Y.W.; Fan, S.W.; Lan, T.; Chen, Z.P. Multisensory five-fingered dexterous hand: The DLR/HIT Hand II. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots Systems (IROS), Nice, France, 22–26 September 2008; pp. 3692–3697. [Google Scholar] [CrossRef]
- Kamikawa, Y.; Maeno, T. Underactuated five-finger prosthetic hand inspired by grasping force distribution of humans. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots Systems (IROS), Nice, France, 22–26 September 2008; pp. 717–722. [Google Scholar] [CrossRef]
- Krausz, N.E.; Rorrer, R.A.L.; Weir, R.F.F.F. Design and Fabrication of a Six Degree-of-Freedom Open Source Hand. IEEE Trans. Neural Syst. Rehabil. Eng. 2016, 24, 562–572. [Google Scholar] [CrossRef]
- Jahan, M.; Manas, M.; Sharma, B.B.; Gogoi, B.B. Feature extraction and pattern recognition of EMG-based signal for hand movements. In Proceedings of the 2015 International Symposium on Advanced Computing and Communication (ISACC), Silchar, India, 14–15 September 2015. [Google Scholar] [CrossRef]
- Zhang, H.; Zhao, Y.; Yao, F.; Xu, L.; Shang, P.; Li, G. An adaptation strategy of using LDA classifier for EMG pattern recognition. In Proceedings of the 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka, Japan, 3–7 July 2013; pp. 4267–4270. [Google Scholar] [CrossRef]
- Ahsan, M.R.; Ibrahimy, M.I.; Khalifa, O.O. Electromygraphy (EMG) signal based hand gesture recognition using artificial neural network (ANN). In Proceedings of the 2011 4th International Conference on Mechatronics (ICOM), Kuala Lumpur, Malaysia, 17–19 May 2011. [Google Scholar] [CrossRef]
- Oskoei, M.A.; Hu, H. Support Vector Machine-Based Classification Scheme for Myoelectric Control Applied to Upper Limb. IEEE Trans. Biomed. Eng. 2008, 55, 1956–1965. [Google Scholar] [CrossRef]
- Benatti, S.; Casamassima, F.; Milosevic, B.; Farella, E.; Schönle, P.; Fateh, S.; Burger, T.; Huang, Q.; Benini, L. A Versatile Embedded Platform for EMG Acquisition and Gesture Recognition. IEEE Trans. Biomed. Circuits Syst. 2015, 9, 620–630. [Google Scholar] [CrossRef]
- Riillo, F.; Quitadamo, L.R.; Cavrini, F.; Gruppioni, E.; Pinto, C.A.; Cosimo Pastò, N.; Sbernini, L.; Albero, L.; Saggio, G. Optimization of EMG-based hand gesture recognition: Supervised vs. unsupervised data preprocessing on healthy subjects and transradial amputees. Biomed. Signal Process. Control 2014, 14, 117–125. [Google Scholar] [CrossRef]
- Tenore, F.V.G.; Ramos, A.; Fahmy, A.; Acharya, S.; Etienne-Cummings, R.; Thakor, N.V. Decoding of Individuated Finger Movements Using Surface Electromyography. IEEE Trans. Biomed. Eng. 2009, 56, 1427–1434. [Google Scholar] [CrossRef]
- Huang, H.; Zhou, P.; Li, G.; Kuiken, T. Spatial Filtering Improves EMG Classification Accuracy Following Targeted Muscle Reinnervation. Ann. Biomed. Eng. 2009, 37, 1849–1857. [Google Scholar] [CrossRef]
- Jiang, M.W.; Wang, R.C.; Wang, J.Z.; Jin, D.W. A Method of Recognizing Finger Motion Using Wavelet Transform of Surface EMG Signal. In Proceedings of the 27th Annual International Conference of the Engineering in Medicine and Biology Society, Shanghai, China, 1–4 September 2005; pp. 2672–2674. [Google Scholar] [CrossRef]
- Englehart, K.; Hudgins, B.; Parker, P.A.; Stevenson, M. Classification of the myoelectric signal using time-frequency based representations. Med. Eng. Phys. 1999, 21, 431–438. [Google Scholar] [CrossRef]
- Nazmi, N.; Rahman, M.A.A.; Yamamoto, S.; Ahmad, S.A.; Zamzuri, H.; Mazlan, S.A. A Review of Classification Techniques of EMG Signals during Isotonic and Isometric Contractions. Sensors 2016, 16, 1304. [Google Scholar] [CrossRef]
- Huang, Y.; Englehart, K.B.; Hudgins, B.; Chan, A.D.C. A Gaussian mixture model based classification scheme for myoelectric control of powered upper limb prostheses. IEEE Trans. Biomed. Eng. 2005, 52, 1801–1811. [Google Scholar] [CrossRef]
- Pylatiuk, C.; Schulz, S.; Doderlein, L. Results of an Internet survey of myoelectric prosthetic hand users. Prosthet. Orthot. Int. 2007, 31, 362–370. [Google Scholar] [CrossRef]
- Drake, R.; Vogl, A.W.; Mitchell, A. Gray’s Anatomy for Students, 2nd ed.; Churchill Livingstone: Amsterdam, The Netherlands, 2004; ISBN 9781455755417. [Google Scholar]
- Tsujimura, T.; Yamamoto, S.; Izumi, K. Hand Sign Classification Employing Myoelectric Signals of Forearm. In Computational Intelligence in Electromyography Analysis—A Perspective on Current Applications and Future Challenges; Naik, G.R., Ed.; InTech: Houston, TX, USA, 2012. [Google Scholar] [CrossRef]
- Fukuda, O.; Bu, N.; Tsuji, T. Control of an Externally Powered Prosthetic Forearm Using Raw-EMG Signals. Trans. Soc. Instrum. Control Eng. 2004, 40, 1124–1131. (In Japanese) [Google Scholar]
- Liu, L.; Liu, P.; Clancy, E.A.; Scheme, E.; Englehart, K.B. Electromyogram Whitening for Improved Classification Accuracy in Upper Limb Prosthesis Control. IEEE Trans. Neural Syst. Rehabil. Eng. 2013, 21, 767–774. [Google Scholar] [CrossRef]
- Adewuyi, A.A.; Hargrove, L.J.; Kuiken, T.A. An Analysis of Intrinsic and Extrinsic Hand Muscle EMG for Improved Pattern Recognition Control. IEEE Trans. Neural Syst. Rehabil. Eng. 2016, 24, 485–494. [Google Scholar] [CrossRef]
- Kanitz, G.R.; Antfolk, C.; Cipriani, C.; Sebelius, F.; Carrozza, M.C. Decoding of Individuated Finger Movements Using Surface EMG and Input Optimization Applying a Genetic Algorithm. In Proceedings of the 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Boston, MA, USA, 30 August–3 September 2011; pp. 1608–1611. [Google Scholar] [CrossRef]
- Hargrove, L.J.; Englehart, K.; Hudgins, B. A comparison of surface and intramuscular myoelectric signal classification. IEEE Trans. Biomed. Eng. 2007, 54, 847–853. [Google Scholar] [CrossRef]
- Kambayashi, A.; Kuniyasu, K.; Jiralerspong, T.; Ishikawa, J. Identification of voluntary movements of five fingers for myoelectric prosthetic hand. In Proceedings of the 32nd Annual Conference of the Robotics Society of Japan, Tokyo, Japan, 4–6 September 2014; p. 32. (In Japanese). [Google Scholar]
- Merletti, R. Standards for Reporting EMG Data. J. Electromyogr. Kinesiol. 2017, 35, 1–2. [Google Scholar] [CrossRef]
- Ebecken, N.F.F. An Overview on the Use of Neural Networks for Data Mining Tasks. J. Br. Neural Netw. Soc. 2011, 9, 202–212. [Google Scholar] [CrossRef]
- Kodaka, T. Hjimete no Kikai Gakushu; Ohmsha: Tokyo, Japan, 2011; pp. 172–219. ISBN 978-4-274-06846-1. (In Japanese) [Google Scholar]
- Shim, H.; An, H.; Lee, S.; Lee, E.H.; Min, H.; Lee, S. EMG Pattern Classification by Split and Merge Deep Belief Network. Symmetry 2016, 8, 148. [Google Scholar] [CrossRef]
Channel Number | Muscle | Function |
---|---|---|
1 | Flexor carpi radialis muscle | Flexion of the hand at the wrist, pronation of the forearm (works along with other muscles which pronate the forearm) |
2 | Flexor carpi ulnaris muscle | Flexion of the hand at the wrist |
3 | Flexor digitorum profundus muscle | Flexion of the fingers (excluding the thumb) |
4 | Flexor pollicis longus muscle | Flexion of the thumb |
5 | Extensor carpi radialis longus muscle | Extension of the wrist, assists movements of the digits |
6 | Extensor pollicis longus muscle | Extension of the thumb |
Classification Scenario | Number of Motions | Training Data | Testing Data | Participant | Number of Neurons in the Input Layer/Hidden Layer/Output Layer | Learning Coefficient |
---|---|---|---|---|---|---|
1 | 17 | Day 1 | Day 1 | A | 36/20/17 | 1.2 |
Day 2 | Day 2 | A | 36/20/17 | |||
Day 1 | Day 1 | B | 36/20/17 | |||
Day 2 | Day 2 | B | 36/20/17 | |||
Day 1 | Day 1 | C | 36/24/17 | |||
Day 1 | Day 1 | D | 36/20/17 | |||
Day 1 | Day 1 | E | 36/24/17 | |||
Day 2 | Day 2 | E | 36/20/17 | |||
Day 1 | Day 1 | F | 36/16/17 | |||
Day 1 | Day 1 | G | 36/20/17 | |||
Day 1 | Day 1 | H | 36/18/17 | |||
Day 2 | Day 2 | H | 36/18/17 | |||
Day 1 | Day 1 | I | 36/20/17 | |||
Day 1 | Day 1 | J | 36/22/17 | |||
Day 1 | Day 1 | K | 36/24/17 | |||
Day 1 | Day 1 | L | 36/20/17 | |||
2 | 17 | Day 1 | Day 2 | A | 36/20/17 | 1.2 |
Day 2 | Day 1 | A | 36/20/17 | |||
Day 1 | Day 2 | B | 36/20/17 | |||
Day 2 | Day 1 | B | 36/20/17 | |||
Day 1 | Day 2 | E | 36/24/17 | |||
Day 2 | Day 1 | E | 36/20/17 | |||
Day 1 | Day 2 | H | 36/18/17 | |||
Day 2 | Day 1 | H | 36/18/17 | |||
3 | 9 | Day 1 | Day 1 | A | 36/14/9 | 1.2 |
Day 2 | Day 2 | A | 36/12/9 | |||
Day 1 | Day 1 | B | 36/14/9 | |||
Day 2 | Day 2 | B | 36/16/9 | |||
Day 1 | Day 1 | C | 36/20/9 | |||
Day 1 | Day 1 | D | 36/14/9 | |||
Day 1 | Day 1 | E | 36/16/9 | |||
Day 2 | Day 2 | E | 36/14/9 | |||
Day 1 | Day 1 | F | 36/12/9 | |||
Day 1 | Day 1 | G | 36/18/9 | |||
Day 1 | Day 1 | H | 36/20/9 | |||
Day 2 | Day 2 | H | 36/16/9 | |||
Day 1 | Day 1 | I | 36/20/9 | |||
Day 1 | Day 1 | J | 36/16/9 | |||
Day 1 | Day 1 | K | 36/14/9 | |||
Day 1 | Day 1 | L | 36/18/9 | |||
4 | 9 | Day 1 | Day 2 | A | 36/14/9 | 1.2 |
Day 2 | Day 1 | A | 36/12/9 | |||
Day 1 | Day 2 | B | 36/14/9 | |||
Day 2 | Day 1 | B | 36/16/9 | |||
Day 1 | Day 2 | E | 36/16/9 | |||
Day 2 | Day 1 | E | 36/14/9 | |||
Day 1 | Day 2 | H | 36/20/9 | |||
Day 2 | Day 1 | H | 36/16/9 |
Author/s | Year | No. of Electrodes | No. of Movements | Sampling Frequency | Accuracy | Number of Subjects | |
---|---|---|---|---|---|---|---|
Fukuda et al. [30] | 2004 | 6 | 6 W | 200 Hz | 1 | 92.1% | 3 H and 2 A |
Jiang et al. [23] | 2005 | 4 | 6 IF | 2000 Hz | 1.5 | 80% | 10 H |
Oskoei & Hu [18] | 2008 | 4 | 6 (5 W + R) | 1000 Hz | 1.5 | 95 % | 11 H |
Tenore et al. [21] | 2009 | 32 | 12 (10 IF & 2 CF) | 2000 Hz | 0.38 | 90% | 5 H |
19 | 0.63 | 83% | 1 A | ||||
Ahsan et al. [17] | 2011 | 2 | 4 W | 1000 Hz | 2 | 88.4% | 3 H |
Cipriani et al. [11] | 2011 | 8 | 7 (4 IF + 2CF + HG) | 10,000 Hz | 0.88 | 89% | 5 H |
79% | 5 A | ||||||
Kanitz et al. [33] | 2011 | 16 | 13 (12 IF + R) | 16,000 Hz | 0.81 | 80% | 5 H and 1 A |
Tsujimura et al. [29] | 2012 | 3 | 3 CF | 10,000 Hz | 1 | 97% | 1 H |
Benatti et al. [19] | 2015 | 8 | 7 (5 W + IF + R) | 1000 Hz | 0.88 | 90% | 4 H |
Adewuyi et al. [32] | 2016 | 19 | 19 (12 IF + 7 HG) | 1000 Hz | 1 | 96% | 9 H |
This work | - | 6 | 17 (10 IF + 6 W + R) | 200 Hz | 2.83 | 63.8% | 12 H |
9 (4 IF + 4 W + R) | 1.5 | 72.9% |
© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Jiralerspong, T.; Nakanishi, E.; Liu, C.; Ishikawa, J. Experimental Study of Real-Time Classification of 17 Voluntary Movements for Multi-Degree Myoelectric Prosthetic Hand. Appl. Sci. 2017, 7, 1163. https://doi.org/10.3390/app7111163
Jiralerspong T, Nakanishi E, Liu C, Ishikawa J. Experimental Study of Real-Time Classification of 17 Voluntary Movements for Multi-Degree Myoelectric Prosthetic Hand. Applied Sciences. 2017; 7(11):1163. https://doi.org/10.3390/app7111163
Chicago/Turabian StyleJiralerspong, Trongmun, Emi Nakanishi, Chao Liu, and Jun Ishikawa. 2017. "Experimental Study of Real-Time Classification of 17 Voluntary Movements for Multi-Degree Myoelectric Prosthetic Hand" Applied Sciences 7, no. 11: 1163. https://doi.org/10.3390/app7111163
APA StyleJiralerspong, T., Nakanishi, E., Liu, C., & Ishikawa, J. (2017). Experimental Study of Real-Time Classification of 17 Voluntary Movements for Multi-Degree Myoelectric Prosthetic Hand. Applied Sciences, 7(11), 1163. https://doi.org/10.3390/app7111163