Motor Imagery Multi-Tasks Classification for BCIs Using the NVIDIA Jetson TX2 Board and the EEGNet Network
Abstract
:1. Introduction
- Results comparison of channel selection between the ARbC method and CMIbA.
- Reliable accuracy results of the tongue, passive, left and right hands, and left and right legs MI tasks classification.
- Processing time reduction using the NJT2 platform resources.
- Convergence acceleration of the learning process implementing the Cyclic Learning Rate (CLR) algorithm.
2. Related Works
3. Materials and Methods
3.1. Overall Flowchart
3.2. Referred Public Dataset
3.3. NVIDIA Jetson TX2 Embedded Board
3.4. The EEGNet Network Architecture
3.5. Data Processing
- If and are independents,Therefore,
- If = ,
4. Numerical Results
4.1. Channel Selection Results
4.2. Results Processing Discriminant Channel Signals
5. Discussions
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
BCI | Brain–Computer Interface |
EEG | Electroencephalogram |
BCV | Brain–Controlled Vehicle |
EOG | Electrooculogram |
EMG | Electromyogram |
SSVEP | Steady-State Evoked Potentials |
MI | Motor Imagery |
MI–EEG | Motor Imagery EEG |
FPGA | Field-Programmable Gate Arrays |
SVM | Support Vector Machines |
EEGNet | Compact convolutional neural network for EEG-based BCI |
NJT2 | NVIDIA Jetson TX2 |
STFT | Short-Time Fourier Transform |
VMD | Variational Mode Decomposition |
GAN | Generative Adversarial Network |
ARbC | Accuracy Rating-based Classifier |
CMIbA | Channels Mutual Information-based Approach |
CLR | Cyclic Learning Rate |
EBCI | Embedded Brain–Computer Interface |
ICA | Independent Component Analysis |
PC | Portable Computer |
CNN | Convolutional Neural Network |
eGUI | Graphical User Interface |
ASCII | American Standard Code for Information Interchange |
CPU | Central Processing Unit |
GPU | Graphics Processor Unit |
SDK | Software Development Kit |
LPDDR4 | Low Power Double Data Rate |
eMMC | Embedded Multi-Media Card |
TFLOPS | Trillion Floating-Point Operations Per Second |
WLAN | Wireless Local Area Network |
ELU | Exponential Linear Unit |
KLD | Kullback–Leibler Divergence |
DCS | Discriminant Channel Subset |
t-SNE | t-distributed Stochastic Neighborhood Embedding |
LUT | Look-Up-Table |
SPS | Samples Per Second |
CLR | Cyclical Learning Rate |
MFLOPS | Million Floating-Point Operations Per Second |
References
- He, B.; Yuan, H.; Meng, J.; Gao, S. Brain–computer interfaces. In Neural Engineering; Springer: Berlin/Heidelberg, Germany, 2020; pp. 131–183. [Google Scholar]
- Herbet, G.; Duffau, H. Revisiting the functional anatomy of the human brain: Toward a meta-networking theory of cerebral functions. Physiol. Rev. 2020, 100, 1181–1228. [Google Scholar] [CrossRef]
- Gao, X.; Wang, Y.; Chen, X.; Gao, S. Interface, interaction, and intelligence in generalized brain-computer interfaces. Trends Cogn. Sci. 2021, 25, 671–684. [Google Scholar] [CrossRef]
- Fraiwan, M.; Alafeef, M.; Almomani, F. Gauging human visual interest using multiscale entropy analysis of EEG signals. J. Ambient. Intell. Humaniz. Comput. 2021, 12, 2435–2447. [Google Scholar] [CrossRef]
- Alafeef, M.; Fraiwan, M. On the diagnosis of idiopathic Parkinson’s disease using continuous wavelet transform complex plot. J. Ambient. Intell. Humaniz. Comput. 2019, 10, 2805–2815. [Google Scholar] [CrossRef]
- Papanastasiou, G.; Drigas, A.; Skianis, C.; Lytras, M. Brain computer interface based applications for training and rehabilitation of students with neurodevelopmental disorders. A literature review. Heliyon 2020, 6, e04250. [Google Scholar] [CrossRef]
- Hekmatmanesh, A.; Nardelli, P.H.; Handroos, H. Review of the state-of-the-art of brain-controlled vehicles. IEEE Access 2021, 9, 110173–110193. [Google Scholar] [CrossRef]
- Zhang, Y.; Xie, S.Q.; Wang, H.; Zhang, Z. Data analytics in steady-state visual evoked potential-based brain-computer interface: A review. IEEE Sens. J. 2020, 21, 1124–1138. [Google Scholar] [CrossRef]
- Shajil, N.; Mohan, S.; Srinivasan, P.; Arivudaiyanambi, J.; Arasappan Murrugesan, A. Multiclass classification of spatially filtered motor imagery EEG signals using convolutional neural network for BCI based applications. J. Med. Biol. Eng. 2020, 40, 663–672. [Google Scholar] [CrossRef]
- Yu, X.; Aziz, M.Z.; Sadiq, M.T.; Fan, Z.; Xiao, G. A New Framework for Automatic Detection of Motor and Mental Imagery EEG Signals for Robust BCI Systems. IEEE Trans. Instrum. Meas. 2021, 70, 1006612. [Google Scholar] [CrossRef]
- Al-Qazzaz, N.K.; Alyasseri, Z.A.A.; Abdulkareem, K.H.; Ali, N.S.; Al-Mhiqani, M.N.; Guger, C. EEG feature fusion for motor imagery: A new robust framework towards stroke patients rehabilitation. Comput. Biol. Med. 2021, 137, 104799. [Google Scholar] [CrossRef]
- Cuomo, G.; Maglianella, V.; Ghanbari Ghooshchy, S.; Zoccolotti, P.; Martelli, M.; Paolucci, S.; Morone, G.; Iosa, M. Motor imagery and gait control in Parkinson’s disease: Techniques and new perspectives in neurorehabilitation. Expert Rev. Neurother. 2022, 22, 43–51. [Google Scholar] [CrossRef]
- Abenna, S.; Nahid, M.; Bouyghf, H.; Ouacha, B. EEG-based BCI: A novel improvement for EEG signals classification based on real-time preprocessing. Comput. Biol. Med. 2022, 148, 105931. [Google Scholar] [CrossRef] [PubMed]
- Jiao, Y.; Zhou, T.; Yao, L.; Zhou, G.; Wang, X.; Zhang, Y. Multi-View Multi-Scale Optimization of Feature Representation for EEG Classification Improvement. IEEE Trans. Neural Syst. Rehabil. Eng. 2020, 28, 2589–2597. [Google Scholar] [CrossRef] [PubMed]
- Khan, M.A.; Das, R.; Iversen, H.K.; Puthusserypady, S. Review on motor imagery based BCI systems for upper limb post-stroke neurorehabilitation: From designing to application. Comput. Biol. Med. 2020, 123, 103843. [Google Scholar] [CrossRef]
- Huang, Q.; Zhang, Z.; He, S.; Li, Y. An EEG-/EOG-Based Hybrid Brain-Computer Interface: Application on Controlling an Integrated Wheelchair Robotic Arm System. Front. Neurosci. 2019, 13, 1243. [Google Scholar] [CrossRef] [PubMed]
- Al-Nuaimi, F.A.; Al-Nuaimi, R.J.; Al-Dhaheri, S.S.; Ouhbi, S.; Belkacem, A.N. Mind Drone Chasing Using EEG-based Brain-Computer Interface. In Proceedings of the 2020 16th International Conference on Intelligent Environments (IE), Madrid, Spain, 20–23 July 2020; pp. 74–79. [Google Scholar] [CrossRef]
- Moctezuma, L.; Abe, T.; Molinas, M. Two-dimensional CNN-based distinction of human emotions from EEG channels selected by Multi-Objective evolutionary algorithm. Sci. Rep. 2022, 12, 3523. [Google Scholar] [CrossRef] [PubMed]
- Mwata-Velu, T.; Ruiz-Pinales, J.; Rostro-Gonzalez, H.; Ibarra-Manzano, M.A.; Cruz-Duarte, J.M.; Avina-Cervantes, J.G. Motor Imagery Classification Based on a Recurrent-Convolutional Architecture to Control a Hexapod Robot. Mathematics 2021, 9, 606. [Google Scholar] [CrossRef]
- Smith, L.N. Cyclical Learning Rates for Training Neural Networks. In Proceedings of the 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), Santa Rosa, CA, USA, 24–31 March 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 464–472. [Google Scholar] [CrossRef]
- Ramzan, M.; Dawn, S. A survey of brainwaves using electroencephalography EEG to develop robust brain-computer interfaces (BCIs): Processing techniques and algorithms. In Proceedings of the 2019 9th International Conference on Cloud Computing, Data Science & Engineering (Confluence), Noida, India, 10–11 January 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 642–647. [Google Scholar]
- Al-Saegh, A.; Dawwd, S.A.; Abdul-Jabbar, J.M. Deep learning for motor imagery EEG-based classification: A review. Biomed. Signal Process. Control 2021, 63, 102172. [Google Scholar] [CrossRef]
- Belwafi, K.; Gannouni, S.; Aboalsamh, H. Embedded Brain Computer Interface: State-of-the-Art in Research. Sensors 2021, 21, 4293. [Google Scholar] [CrossRef]
- Majoros, T.; Oniga, S. Overview of the EEG-Based Classification of Motor Imagery Activities Using Machine Learning Methods and Inference Acceleration with FPGA-Based Cards. Electronics 2022, 11, 2293. [Google Scholar] [CrossRef]
- Dabas, D.; Lakhani, M.; Sharma, B. Classification of EEG signals for hand gripping motor imagery and hardware representation of neural states using Arduino-based LED sensors. In Proceedings of the International Conference on Artificial Intelligence and Applications, Crete, Greece, 17–20 June 2021; Springer: Berlin/Heidelberg, Germany, 2021; pp. 213–224. [Google Scholar]
- Lawhern, V.J.; Solon, A.J.; Waytowich, N.R.; Gordon, S.M.; Hung, C.P.; Lance, B.J. EEGNet: A compact convolutional neural network for EEG-based brain–computer interfaces. J. Neural Eng. 2018, 15, 056013. [Google Scholar] [CrossRef] [PubMed]
- Zhu, Y.; Li, Y.; Lu, J.; Li, P. EEGNet with ensemble learning to improve the cross-session classification of SSVEP-based BCI from Ear-EEG. IEEE Access 2021, 9, 15295–15303. [Google Scholar] [CrossRef]
- Feng, L.; Yang, L.; Liu, S.; Han, C.; Zhang, Y.; Zhu, Z. An efficient EEGNet processor design for portable EEG-Based BCIs. Microelectron. J. 2022, 120, 105356. [Google Scholar] [CrossRef]
- Tsukahara, A.; Anzai, Y.; Tanaka, K.; Uchikawa, Y. A design of EEGNet-based inference processor for pattern recognition of EEG using FPGA. Electron. Commun. Jpn. 2020, 104, 53–64. [Google Scholar] [CrossRef]
- Ak, A.; Topuz, V.; Midi, I. Motor imagery EEG signal classification using image processing technique over GoogLeNet deep learning algorithm for controlling the robot manipulator. Biomed. Signal Process. Control 2022, 72, 103295. [Google Scholar] [CrossRef]
- Ingolfsson, T.M.; Hersche, M.; Wang, X.; Kobayashi, N.; Cavigelli, L.; Benini, L. EEG-TCNet: An accurate temporal convolutional network for embedded motor-imagery brain-machine interfaces. In Proceedings of the 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Toronto, ON, Canada, 11–14 October 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 2958–2965. [Google Scholar]
- Ma, X.; Zheng, W.; Peng, Z.; Yang, J. Fpga-based rapid electroencephalography signal classification system. In Proceedings of the 2019 IEEE 11th International Conference on Advanced Infocomm Technology (ICAIT), Jinan, China, 18–20 October 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 223–227. [Google Scholar]
- Manjunath, N.K.; Paneliya, H.; Hosseini, M.; Hairston, W.D.; Mohsenin, T. A low-power lstm processor for multi-channel brain eeg artifact detection. In Proceedings of the 2020 21st International Symposium on Quality Electronic Design (ISQED), Santa Clara, CA, USA, 25–26 March 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 105–110. [Google Scholar]
- Khatwani, M.; Hosseini, M.; Paneliya, H.; Mohsenin, T.; Hairston, W.D.; Waytowich, N. Energy Efficient Convolutional Neural Networks for EEG Artifact Detection. In Proceedings of the 2018 IEEE Biomedical Circuits and Systems Conference (BioCAS), Cleveland, OH, USA, 17–19 October 2018; pp. 1–4. [Google Scholar] [CrossRef]
- Maiti, S.; Mandal, A.S.; Chaudhury, S. Classification of Motor Imagery EEG Signal for Navigation of Brain Controlled Drones. In Proceedings of the Intelligent Human Computer Interaction: 11th International Conference, IHCI 2019, Allahabad, India, 12–14 December 2019; Proceedings 11. Tiwary, U.S., Chaudhury, S., Eds.; Springer: Cham, Switzerland, 2020; pp. 3–12. [Google Scholar]
- Ascari, L.; Marchenkova, A.; Bellotti, A.; Lai, S.; Moro, L.; Koshmak, K.; Mantoan, A.; Barsotti, M.; Brondi, R.; Avveduto, G.; et al. Validation of a Novel Wearable Multistream Data Acquisition and Analysis System for Ergonomic Studies. Sensors 2021, 21, 8167. [Google Scholar] [CrossRef] [PubMed]
- Lucan Orășan, I.; Seiculescu, C.; Căleanu, C.D. A Brief Review of Deep Neural Network Implementations for ARM Cortex-M Processor. Electronics 2022, 11, 2545. [Google Scholar] [CrossRef]
- Hernandez-Ruiz, A.C.; Enériz, D.; Medrano, N.; Calvo, B. Motor-Imagery EEGNet-Based Processing on a Low-Spec SoC Hardware. In Proceedings of the 2021 IEEE Sensors, Sydney, Australia, 31 October–3 November 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 1–4. [Google Scholar]
- Enériz, D.; Medrano, N.; Calvo, B.; Hernández-Ruiz, A.C.; Antolín, D. Real-Time EEG Acquisition System for FPGA-based BCI. In Proceedings of the 2022 37th Conference on Design of Circuits and Integrated Circuits (DCIS), Pamplona, Spain, 16–18 November 2022; IEEE: Piscataway, NJ, USA, 2022; pp. 1–5. [Google Scholar]
- Kaya, M.; Binli, M.; Ozbay, E.; Yanar, H.; Mishchenko, Y. A large electroencephalographic motor imagery dataset for electroencephalographic brain-computer interfaces. Sci. Data 2018, 5, 180211. [Google Scholar] [CrossRef] [PubMed]
- Yan, Z.; Yang, X.; Jin, Y. Considerate motion imagination classification method using deep learning. PLoS ONE 2022, 17, e0276526. [Google Scholar] [CrossRef] [PubMed]
- Keerthi Krishnan, K.; Soman, K. CNN-based classification of motor imaginary using variational mode decomposed EEG-spectrum image. Biomed. Eng. Lett. 2021, 11, 235–247. [Google Scholar] [CrossRef]
- An, Y.; Lam, H.K.; Ling, S.H. Auto-Denoising for EEG Signals Using Generative Adversarial Network. Sensors 2022, 22, 1750. [Google Scholar] [CrossRef] [PubMed]
- Pérez-Velasco, S.; Santamaria-Vazquez, E.; Martinez-Cagigal, V.; Marcos-Martinez, D.; Hornero, R. EEGSym: Overcoming Inter-Subject Variability in Motor Imagery Based BCIs With Deep Learning. IEEE Trans. Neural Syst. Rehabil. Eng. 2022, 30, 1766–1775. [Google Scholar] [CrossRef] [PubMed]
- NVIDIA SDK Manager. Available online: https://developer.nvidia.com/nvidia-sdk-manager (accessed on 15 October 2022).
- JETPACK SDK 4.6.2. Available online: https://developer.nvidia.com/embedded/jetpack-sdk-462 (accessed on 15 October 2022).
- Mwata-Velu, T.; Avina-Cervantes, J.G.; Ruiz-Pinales, J.; Garcia-Calva, T.A.; González-Barbosa, E.A.; Hurtado-Ramos, J.B.; González-Barbosa, J.J. Improving Motor Imagery EEG Classification Based on Channel Selection Using a Deep Learning Architecture. Mathematics 2022, 10, 2302. [Google Scholar] [CrossRef]
- Shoji, T.; Yoshida, N.; Tanaka, T. Automated detection of abnormalities from an EEG recording of epilepsy patients with a compact convolutional neural network. Biomed. Signal Process. Control 2021, 70, 103013. [Google Scholar] [CrossRef]
- Waytowich, N.; Lawhern, V.J.; Garcia, J.O.; Cummings, J.; Faller, J.; Sajda, P.; Vettel, J.M. Compact convolutional neural networks for classification of asynchronous steady-state visual evoked potentials. J. Neural Eng. 2018, 15, 066031. [Google Scholar] [CrossRef]
- Bertrand, A. Utility Metrics for Assessment and Subset Selection of Input Variables for Linear Estimation [Tips & Tricks]. IEEE Signal Process. Mag. 2018, 35, 93–99. [Google Scholar] [CrossRef]
- Narayanan, A.M.; Bertrand, A. Analysis of miniaturization effects and channel selection strategies for EEG sensor networks with application to auditory attention detection. IEEE Trans. Biomed. Eng. 2019, 67, 234–244. [Google Scholar] [CrossRef]
- Van der Maaten, L.; Hinton, G. Visualizing data using t-SNE. J. Mach. Learn. Res. 2008, 9, 2579–2605. [Google Scholar]
Works | Platform | Dataset | Ch | Latency |
---|---|---|---|---|
per Task | ||||
Khatwani et al. [34] | NJT2 | Own | 64 | ≤84.1 ms |
Maiti et al. [35] | NJT2 | BCI competition IV | 3 | 9–10 ms |
Ascari et al. [36] | NJT2 | Own | 2 | 0 ms |
No. | Subject | Classes | Sessions | Samples |
---|---|---|---|---|
1 | A | 6 | 3 | 2877 |
2 | B | 6 | 3 | 2869 |
3 | C | 6 | 2 | 1916 |
4 | E | 6 | 3 | 2855 |
5 | F | 6 | 3 | 2879 |
6 | G | 6 | 3 | 2867 |
7 | H | 6 | 2 | 1912 |
8 | I | 6 | 2 | 1836 |
9 | J | 6 | 1 | 946 |
10 | K | 6 | 2 | 1914 |
11 | L | 6 | 2 | 1904 |
12 | M | 6 | 3 | 2866 |
13 | All | 6 | 29 | 27,641 |
Relaxation | ||||||
---|---|---|---|---|---|---|
Left hand | Right hand | Passive | Left leg | Tongue | Right leg |
Label | Characteristics |
---|---|
NJT2 board | Serial 0320218091017, model 699-82597-0000-501 C |
GPU | 256-core NVIDIA PascalTM GPU architecture with 256 NVIDIA CUDA cores |
CPU | Dual-Core NVIDIA Denver 2 64-Bit CPU Quad-Core ARM® Cortex®-A57 MPCore |
Memory | 8 GB 128-bit LPDDR4 Memory 1866 MHx—59.7 GB/s |
Storage | 32 GB eMMC 5.1 |
Computing capacity | 1.33 TFLOPs |
Power consumption | 7.5 W/15 W |
Mechanical | 69.6 mm × 45 mm, 260-pin edge Connector |
Networking | 10/100/1000 BASE-T, 802.11ac WLAN, Bluetooth |
Parameters | Descriptions |
---|---|
nb_classes | Number of classes to classify |
Chans | Number of channels |
Samples | Number of EEG data time points |
DropourRate | Dropout fraction |
kerneLength | Length of temporal convolution in the first layer (Conv2D). |
F1, F2 | Numbers of temporal filters (F1) and pointwise filters (F2) to learn. |
D | Number of spatial filters to learn within each kerneLength |
dropoutType | Either SpatialDropout2D or Dropout |
Layer (Type) | Output Shape | Parameters |
---|---|---|
Input Layer | (None, k, 170, 1) | 0 |
Conv2D | (None, k, 170, 4) | 16 |
Batch_normalization_1 | (None, k, 170, 4) | 16 |
Depthwise_conv2D | (None, 1, 170, 16) | 96/128 |
Batch_normalization_2 | (None, 1, 170, 16) | 64 |
Activation_1 | (None, 1, 170, 16) | 0 |
Average_pooling2D_1 | (None, 1, 42, 16) | 0 |
Dropout_1 | (None, 1, 42, 16) | 0 |
Separable_conv2D | (None, 1, 42, 16) | 512 |
Batch_normalization_3 | (None, 1, 42, 64) | 64 |
Activation_2 | (None, 1, 42, 16) | 0 |
Average_pooling2D_2 | (None, 1, 5, 16) | 0 |
Dropout_2 | (None, 1, 5, 16) | 0 |
Flatten | (None, 80) | 0 |
Dense | (None, 6) | 486 |
Softmax | (None, 6) | 0 |
Ref. | Channel | Brain Area | Accuracy (%) |
---|---|---|---|
1 | Fp1 | Frontal (attention) | 39.5 |
2 | Fp2 | Frontal (Judgment restrains impulses) | 39.1 |
3 | F7 | Frontal (Verbal expression) | 38.4 |
4 | F3 | Frontal (Motor planning of left-upper extremity) | 36.4 |
5 | Fz | Frontal central (Motor planning (midline)) | 36.4 |
6 | F4 | Frontal (Motor planning of left-upper extremity) | 35.1 |
7 | F8 | Frontal (Emotional expression) | 39.2 |
8 | T3 | Temporal (Verbal memory) | 34.7 |
9 | C3 | Central (sensorimotor integration (right)) | 36.5 |
10 | Cz | Central (sensorimotor integration (midline)) | 37.0 |
11 | C4 | Central (sensorimotor integration (left)) | 35.9 |
12 | T4 | Temporal (Emotional memory) | 35.9 |
13 | T5 | Temporal (Verbal understanding) | 36.3 |
14 | P3 | Parietal (cognitive processing special temporal) | 37.4 |
15 | Pz | Parietal (cognitive processing) | 35.7 |
16 | P4 | Parietal (“Math word problems”, “Non-verbal reasoning”) | 36.7 |
17 | T6 | Temporal (Emotional understanding and motivation) | 36.4 |
18 | O1 | Occipital (visual processing) | 37.0 |
19 | O2 | Occipital (visual processing) | 36.7 |
Subject | Channel | Average Accuracies (%) Depending on the Number of Channels | |||
---|---|---|---|---|---|
Selection | 6 | Accuracy | 8 | Accuracy | |
A | ARbC | {Fp1,F8,Fp2,F7,P3,Cz} | 80.6 | {Fp1,F8,Fp2,F7,P3,Cz,O1,P4} | 86.8 |
CMIbA | {P4,T6,T3,P3,F4,O2} | 86.5 | {P4,T6,T3,P3,F4,O2,Fp2,Fz} | 89.0 | |
B | ARbC | {Fp1,F8,Fp2,F7,P3,Cz} | 63.9 | {Fp1,F8,Fp2,F7,P3,Cz,O1,P4} | 68.0 |
CMIbA | {P4,T6,T3,P3,F4,O2} | 68.7 | {P4,T6,T3,P3,F4,O2,Fp2,Fz} | 76.3 | |
C | ARbC | {Fp1,F8,Fp2,F7,P3,Cz} | 89.1 | {Fp1,F8,Fp2,F7,P3,Cz,O1,P4} | 90.9 |
CMIbA | {P4,T6,T3,P3,F4,O2} | 83.0 | {P4,T6,T3,P3,F4,O2,Fp2,Fz} | 92.2 | |
E | ARbC | {Fp1,F8,Fp2,F7,P3,Cz} | 76.6 | {Fp1,F8,Fp2,F7,P3,Cz,O1,P4} | 78.3 |
CMIbA | {P4,T6,T3,P3,F4,O2} | 70.8 | {P4,T6,T3,P3,F4,O2,Fp2,Fz} | 82.5 | |
F | ARbC | {Fp1,F8,Fp2,F7,P3,Cz} | 71.6 | {Fp1,F8,Fp2,F7,P3,Cz,O1,P4} | 79.2 |
CMIbA | {P4,T6,T3,P3,F4,O2} | 72.4 | {P4,T6,T3,P3,F4,O2,Fp2,Fz} | 80.4 | |
G | ARbC | {Fp1,F8,Fp2,F7,P3,Cz} | 84.0 | {Fp1,F8,Fp2,F7,P3,Cz,O1,P4} | 86.0 |
CMIbA | {P4,T6,T3,P3,F4,O2} | 81.9 | {P4,T6,T3,P3,F4,O2,Fp2,Fz} | 87.3 | |
H | ARbC | {Fp1,F8,Fp2,F7,P3,Cz} | 57.0 | {Fp1,F8,Fp2,F7,P3,Cz,O1,P4} | 57.8 |
CMIbA | {P4,T6,T3,P3,F4,O2} | 56.2 | {P4,T6,T3,P3,F4,O2,Fp2,Fz} | 65.5 | |
I | ARbC | {Fp1,F8,Fp2,F7,P3,Cz} | 56.4 | {Fp1,F8,Fp2,F7,P3,Cz,O1,P4} | 57.6 |
CMIbA | {P4,T6,T3,P3,F4,O2} | 53.7 | {P4,T6,T3,P3,F4,O2,Fp2,Fz} | 67.9 | |
J | ARbC | {Fp1,F8,Fp2,F7,P3,Cz} | 99.6 | {Fp1,F8,Fp2,F7,P3,Cz,O1,P4} | 99.5 |
CMIbA | {P4,T6,T3,P3,F4,O2} | 98.8 | {P4,T6,T3,P3,F4,O2,Fp2,Fz} | 99.7 | |
K | ARbC | {Fp1,F8,Fp2,F7,P3,Cz} | 83.0 | {Fp1,F8,Fp2,F7,P3,Cz,O1,P4} | 79.4 |
CMIbA | {P4,T6,T3,P3,F4,O2} | 76.8 | {P4,T6,T3,P3,F4,O2,Fp2,Fz} | 79.3 | |
L | ARbC | {Fp1,F8,Fp2,F7,P3,Cz} | 85.7 | {Fp1,F8,Fp2,F7,P3,Cz,O1,P4} | 93.9 |
CMIbA | {P4,T6,T3,P3,F4,O2} | 90.4 | {P4,T6,T3,P3,F4,O2,Fp2,Fz} | 98.0 | |
M | ARbC | {Fp1,F8,Fp2,F7,P3,Cz} | 78.7 | {Fp1,F8,Fp2,F7,P3,Cz,O1,P4} | 83.7 |
CMIbA | {P4,T6,T3,P3,F4,O2} | 79.5 | {P4,T6,T3,P3,F4,O2,Fp2,Fz} | 81.9 | |
{A,B, …, M} | ARbC | {Fp1,F8,Fp2,F7,P3,Cz} | 55.5 | {Fp1,F8,Fp2,F7,P3,Cz,O1,P4} | 59.3 |
CMIbA | {P4,T6,T3,P3,F4,O2} | 52.2 | {P4,T6,T3,P3,F4,O2,Fp2,Fz} | 55.2 |
Subject | Average Accuracies (%) per MI Task | |||||
---|---|---|---|---|---|---|
Left Hand | Right Hand | Passive | Left Leg | Tongue | Right Leg | |
A | 80 | 80 | 80 | 80 | 80 | 80 |
B | 75 | 75 | 75 | 75 | 75 | 75 |
C | 90 | 90 | 90 | 90 | 90 | 90 |
E | 75 | 75 | 75 | 75 | 75 | 75 |
F | 77 | 77 | 77 | 77 | 77 | 77 |
G | 85 | 85 | 85 | 85 | 85 | 85 |
H | 67 | 67 | 67 | 67 | 67 | 67 |
I | 67 | 67 | 67 | 67 | 67 | 67 |
J | 100 | 100 | 100 | 100 | 100 | 100 |
K | 80 | 80 | 80 | 80 | 80 | 80 |
L | 100 | 100 | 100 | 100 | 100 | 100 |
M | 80 | 80 | 80 | 80 | 80 | 80 |
Average | 81.3 | 81.3 | 81.3 | 81.3 | 81.3 | 81.3 |
{A, B, …, M} | 57 | 57 | 57 | 57 | 57 | 57 |
Subject | Works | |||||
---|---|---|---|---|---|---|
Keerthi et al. [42] | Yan et al. [41] | Proposed Method | ||||
VMD + STFT + EEGNet | EEGNet | CbA/CbMI + EEGNet | ||||
Sel.Ch. | Acc.(%) | Sel.Ch. | Acc.(%) | Sel.Ch. | Acc.(%) | |
A | 3 | 86.74 | 19 | 87.40 | 8 | 89.0 |
B | 3 | 97.42 | 19 | 67.22 | 8 | 76.3 |
C | 3 | 82.93 | 19 | 82.36 | 8 | 92.2 |
E | 3 | 91.84 | 19 | 76.94 | 8 | 82.5 |
F | 3 | 94.27 | 19 | 70.32 | 8 | 80.4 |
G | 3 | 89.02 | 19 | 89.33 | 8 | 87.3 |
H | 3 | 87.25 | 19 | 43.46 | 8 | 65.5 |
I | 3 | 90.18 | 19 | 44.25 | 8 | 67.9 |
J | 3 | 88.55 | 19 | 98.84 | 8 | 99.7 |
K | 3 | 85.76 | 19 | 81.03 | 6 | 83.0 |
L | 3 | 92.49 | 19 | 95.35 | 8 | 98.0 |
M | 3 | 96.01 | 19 | 84.93 | 8 | 83.7 |
– | 90.20 | – | 76.79 | – | 83.7 |
Subject | Average Latency (ms) per MI Task | |||||
---|---|---|---|---|---|---|
Left Hand | Right Hand | Passive | Left Leg | Tongue | Right Leg | |
A | 56.1 | 56.1 | 56.1 | 56.1 | 56.1 | 56.1 |
B | 55.7 | 55.7 | 55.7 | 55.7 | 55.7 | 55.7 |
C | 42.7 | 42.7 | 42.7 | 42.7 | 42.7 | 42.7 |
E | 55.2 | 55.2 | 55.2 | 55.2 | 55.2 | 55.2 |
F | 57.2 | 57.2 | 57.2 | 57.2 | 57.2 | 57.2 |
G | 55.8 | 55.8 | 55.8 | 55.8 | 55.8 | 55.8 |
H | 42.3 | 42.3 | 42.3 | 42.3 | 42.3 | 42.3 |
I | 43.8 | 43.8 | 43.8 | 43.8 | 43.8 | 43.8 |
J | 36.7 | 36.7 | 36.7 | 36.7 | 36.7 | 36.7 |
K | 42.1 | 42.1 | 42.1 | 42.1 | 42.1 | 42.1 |
L | 41.8 | 41.8 | 41.8 | 41.8 | 41.8 | 41.8 |
M | 56.0 | 56.0 | 56.0 | 56.0 | 56.0 | 56.0 |
Average | 48.7 | 48.7 | 48.7 | 48.7 | 48.7 | 48.7 |
{A,B, …, M} | 135 | 135 | 135 | 135 | 135 | 135 |
Methods | Platform | Dataset | Number of | Latency |
---|---|---|---|---|
Channels | per Task | |||
Khatwani et al. [34] | NJT2 | Own | 64 | ≤84.1 ms |
Maiti et al. [35] | NJT2 | BCI competition IV | 3 | 9–10 ms |
Ascari et al. [36] | NJT2 | Own | 2 | 0 ± 0 ms |
Proposed method | NJT2 | HaLT [40] | 6, 8 | 48.7 ms |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Mwata-Velu, T.; Niyonsaba-Sebigunda, E.; Avina-Cervantes, J.G.; Ruiz-Pinales, J.; Velu-A-Gulenga, N.; Alonso-Ramírez, A.A. Motor Imagery Multi-Tasks Classification for BCIs Using the NVIDIA Jetson TX2 Board and the EEGNet Network. Sensors 2023, 23, 4164. https://doi.org/10.3390/s23084164
Mwata-Velu T, Niyonsaba-Sebigunda E, Avina-Cervantes JG, Ruiz-Pinales J, Velu-A-Gulenga N, Alonso-Ramírez AA. Motor Imagery Multi-Tasks Classification for BCIs Using the NVIDIA Jetson TX2 Board and the EEGNet Network. Sensors. 2023; 23(8):4164. https://doi.org/10.3390/s23084164
Chicago/Turabian StyleMwata-Velu, Tat’y, Edson Niyonsaba-Sebigunda, Juan Gabriel Avina-Cervantes, Jose Ruiz-Pinales, Narcisse Velu-A-Gulenga, and Adán Antonio Alonso-Ramírez. 2023. "Motor Imagery Multi-Tasks Classification for BCIs Using the NVIDIA Jetson TX2 Board and the EEGNet Network" Sensors 23, no. 8: 4164. https://doi.org/10.3390/s23084164
APA StyleMwata-Velu, T., Niyonsaba-Sebigunda, E., Avina-Cervantes, J. G., Ruiz-Pinales, J., Velu-A-Gulenga, N., & Alonso-Ramírez, A. A. (2023). Motor Imagery Multi-Tasks Classification for BCIs Using the NVIDIA Jetson TX2 Board and the EEGNet Network. Sensors, 23(8), 4164. https://doi.org/10.3390/s23084164