Next Article in Journal
Analysis of Asymmetry in Active Split-Ring Resonators to Design Circulating-Current Eigenmode: Demonstration of Beamsteering and Focal-Length Control toward Reconfigurable Intelligent Surface
Previous Article in Journal
Wide-Band Interference Mitigation in GNSS Receivers Using Sub-Band Automatic Gain Control
Article

Enhanced Recognition of Amputated Wrist and Hand Movements by Deep Learning Method Using Multimodal Fusion of Electromyography and Electroencephalography

by 1,†, 2,†, 3, 4, 2,5,6,* and 1,7,8,*
1
Department of Convergence IT Engineering, Pohang University of Science and Technology, Pohang 37673, Korea
2
Department of Rehabilitation Medicine, College of Medicine, Dankook University, Cheonan 31116, Korea
3
Department of Medical Device Management and Research, SAIHST, Sungkyunkwan University, Seoul 03063, Korea
4
Department of Rehabilitation Medicine, Daejeon Hospital, Daejeon 34383, Korea
5
Department of Nanobiomedical Science & BK21 NBM Global Research Center for Regenerative Medicine, Dankook University, Cheonan 31116, Korea
6
Institute of Tissue Regeneration Engineering (ITREN), Dankook University, Cheonan 31116, Korea
7
Department of Electrical Engineering, Pohang University of Science and Technology, Pohang 37673, Korea
8
Department of Mechanical Engineering, Pohang University of Science and Technology, Pohang 37673, Korea
*
Authors to whom correspondence should be addressed.
These authors contributed equally to this work.
Academic Editor: Steve Ling
Sensors 2022, 22(2), 680; https://doi.org/10.3390/s22020680
Received: 8 December 2021 / Revised: 9 January 2022 / Accepted: 14 January 2022 / Published: 16 January 2022
(This article belongs to the Section Intelligent Sensors)
Motion classification can be performed using biometric signals recorded by electroencephalography (EEG) or electromyography (EMG) with noninvasive surface electrodes for the control of prosthetic arms. However, current single-modal EEG and EMG based motion classification techniques are limited owing to the complexity and noise of EEG signals, and the electrode placement bias, and low-resolution of EMG signals. We herein propose a novel system of two-dimensional (2D) input image feature multimodal fusion based on an EEG/EMG-signal transfer learning (TL) paradigm for detection of hand movements in transforearm amputees. A feature extraction method in the frequency domain of the EEG and EMG signals was adopted to establish a 2D image. The input images were used for training on a model based on the convolutional neural network algorithm and TL, which requires 2D images as input data. For the purpose of data acquisition, five transforearm amputees and nine healthy controls were recruited. Compared with the conventional single-modal EEG signal trained models, the proposed multimodal fusion method significantly improved classification accuracy in both the control and patient groups. When the two signals were combined and used in the pretrained model for EEG TL, the classification accuracy increased by 4.18–4.35% in the control group, and by 2.51–3.00% in the patient group. View Full-Text
Keywords: brain–computer interface (BCI); convolutional neural network (CNN); electroencephalography (EEG); electromyography (EMG); transforearm amputees; transfer learning (TL) brain–computer interface (BCI); convolutional neural network (CNN); electroencephalography (EEG); electromyography (EMG); transforearm amputees; transfer learning (TL)
Show Figures

Figure 1

MDPI and ACS Style

Kim, S.; Shin, D.Y.; Kim, T.; Lee, S.; Hyun, J.K.; Park, S.-M. Enhanced Recognition of Amputated Wrist and Hand Movements by Deep Learning Method Using Multimodal Fusion of Electromyography and Electroencephalography. Sensors 2022, 22, 680. https://doi.org/10.3390/s22020680

AMA Style

Kim S, Shin DY, Kim T, Lee S, Hyun JK, Park S-M. Enhanced Recognition of Amputated Wrist and Hand Movements by Deep Learning Method Using Multimodal Fusion of Electromyography and Electroencephalography. Sensors. 2022; 22(2):680. https://doi.org/10.3390/s22020680

Chicago/Turabian Style

Kim, Sehyeon, Dae Y. Shin, Taekyung Kim, Sangsook Lee, Jung K. Hyun, and Sung-Min Park. 2022. "Enhanced Recognition of Amputated Wrist and Hand Movements by Deep Learning Method Using Multimodal Fusion of Electromyography and Electroencephalography" Sensors 22, no. 2: 680. https://doi.org/10.3390/s22020680

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop