Next Article in Journal
Behavioral and Oxidative Stress Changes in Mice Subjected to Combinations of Multiple Stressors Relevant to Irritable Bowel Syndrome
Previous Article in Journal
Sleep and Cognition in Children with Fetal Alcohol Spectrum Disorders (FASD) and Children with Autism Spectrum Disorders (ASD)
Open AccessArticle

A BCI System Based on Motor Imagery for Assisting People with Motor Deficiencies in the Limbs

Department of Electronics and Communications Engineering, College of Engineering and Technology, Arab Academy for Science, Technology and Maritime Transport, Alexandria P.O. Box 1029, Egypt
*
Author to whom correspondence should be addressed.
Brain Sci. 2020, 10(11), 864; https://doi.org/10.3390/brainsci10110864
Received: 1 October 2020 / Revised: 27 October 2020 / Accepted: 6 November 2020 / Published: 17 November 2020
(This article belongs to the Section Neural Engineering, Neuroergonomics and Neurorobotics)

Abstract

Motor deficiencies constitute a significant problem affecting millions of people worldwide. Such people suffer from a debility in daily functioning, which may lead to decreased and incoherence in daily routines and deteriorate their quality of life (QoL). Thus, there is an essential need for assistive systems to help those people achieve their daily actions and enhance their overall QoL. This study proposes a novel brain–computer interface (BCI) system for assisting people with limb motor disabilities in performing their daily life activities by using their brain signals to control assistive devices. The extraction of useful features is vital for an efficient BCI system. Therefore, the proposed system consists of a hybrid feature set that feeds into three machine-learning (ML) classifiers to classify motor Imagery (MI) tasks. This hybrid feature selection (FS) system is practical, real-time, and an efficient BCI with low computation cost. We investigate different combinations of channels to select the combination that has the highest impact on performance. The results indicate that the highest achieved accuracies using a support vector machine (SVM) classifier are 93.46% and 86.0% for the BCI competition III–IVa dataset and the autocalibration and recurrent adaptation dataset, respectively. These datasets are used to test the performance of the proposed BCI. Also, we verify the effectiveness of the proposed BCI by comparing its performance with recent studies. We show that the proposed system is accurate and efficient. Future work can apply the proposed system to individuals with limb motor disabilities to assist them and test their capability to improve their QoL. Moreover, the forthcoming work can examine the system’s performance in controlling assistive devices such as wheelchairs or artificial limbs.
Keywords: brain–computer interface (BCI); motor Imagery (MI); electroencephalography (EEG); wavelet packet decomposition (WPD); feature selection (FS) brain–computer interface (BCI); motor Imagery (MI); electroencephalography (EEG); wavelet packet decomposition (WPD); feature selection (FS)

1. Introduction

Motor deficits are a barrier for many paralyzed patients and people suffering from difficulty in moving their limbs. There are several causes of motor impairments such as spinal cord injury, a stroke of the motor cortex, neurodegeneration in the basal ganglia and cerebellum, or damage to the associational motor cortex. Spinal cord injury is mutilation to the spinal cord that leads to short-term or permanent alterations in its functions. Such alterations are developed deliberately in the sense of continuing neuroplasticity that reveals the damage of afferent response from the separated limbs [1]. Stroke is the foremost reason for long-term adult debility. It disrupts the brain’s blood supply, leading to an injury in brain tasks, specifically motor function. After a stroke, motor tasks’ impulsive response is related to activity feedback in the primary motor cortex [2].
On the other hand, Basal ganglia disease is a collection of mental complications that arise when the nuclei in the brain known as the basal ganglia defeat to restrain undesirable movements correctly or appropriately inform upper motor neurons circuits to recruit motor function [3]. Examples of Basal ganglia disease are Parkinson and Dystonia diseases. The cerebellar illness leads to incoordination, ineptness [4,5], and tremor as the cerebellum controls flattening and taming voluntary movements. It does not lead to the loss of any sole task but, conversely, produces an overall motor reaction [6]. Injury in various areas of the association cortex can generate higher-level syndromes of behavior. Apraxia is a condition of motor control that can happen after injury in the parietal association cortex, premotor cortex, or supplementary motor cortex. There is no disability of patient muscles or limbs in apraxia, and muscle power may not be reduced [7]. Motor deficiencies are affecting millions of people worldwide [8]. People with such disabilities experience an encumbrance in performing their daily functions. This dysfunction will decrease, and cease disable people’s daily routines and affect their quality of life (QoL) [9]. The QoL for those people can benefit from intelligent assistive technology that can enhance communication, house-environment arrangement, and motion, according to the individual’s enduring motor capabilities [10,11]. Thus, intelligent assistive systems such as brain–computer interface (BCI) are essential for helping those people achieve their daily actions and enhance their overall QoL [9]. Such a BCI system can use these people’s brain signals to control assistive devices to enable them to do their everyday tasks regardless of their motor disability.
BCI systems which are also known as the brain-machine interface (BMI) is a method used for controlling or communicating between a patient brain’s electrical activity and an external device such as a computer or robotic arm [11,12]. There are two ways to measure brain signals, namely invasive or non-invasive methods. Although the invasive method is very accurate, it requires a surgical operation to place the skull’s electrodes. On the other hand, the non-invasive approach is more comfortable in implementation and does not require any surgical intervention as only electrodes are attached to the scalp [13]. Among the non-invasive methods, electroencephalography (EEG) is a well-known technique that captures brain activity. Its popularity is due to the simplicity of use, mobility, lower cost, and high temporal resolution [14,15]. To produce an efficient and accurate BCI system capable of performing an intended task such as moving a hand, a foot, or a control wheelchair, significant features should be extracted from preprocessed EEG signals and then processed and analyzed. In BCI systems, extracted features are usually of high dimension. Therefore, the feature extraction process [16] is vital to pool out important information from EEG signals. Moreover, the features selection procedure is essential to reduce computational time and select a reduced number of features. Moreover, channel reduction is critical to lower the computational cost [17] further.
Motor imagery (MI) is when an individual internally imagine a part of his body is moving or performing an action. [18,19,20,21]. It can also be described as locomotion’s intellectual practice without any activation of the muscles [22]. This imagination of the motion produces a significant variation in the μ and β frequency bands’ amplitude in an individual’s EEG, recognized by the BCI system. This BCI system is beneficial for paralyzed people, as they are incapable of truly moving their limbs. However, they have kept the capacity to conceive the motion of paralyzed limbs. In this form, the BCI system can be used to control assistive devices such as prosthetic limbs [23]. Another sort of MI-based BCI system is spinal cord injury rehabilitation. Such a BCI system is connected to powered exoskeletons located on the paralyzed limbs. These exoskeletons use the EEG signal acquired during the MI task, which activates brain regions to enable individuals to move [23,24]. Several studies indicated the positive impact of BCI systems on the rehabilitation process [25,26]. EEG signals acquired during MI tasks are commonly used to constructed assistive BCI systems. A BCI system based on MI aims to get distinct EEG signals by imagining specific motor activities, which are then converted into actions [27,28]. Several BCI schemes based on MI have been proposed to help people with different motor disabilities attain a normal lifestyle [29,30]. Among these schemes, the studies [31,32] indicated that MI-based-EEG signals operated in BCI systems effectively treat patients with spinal cord injury. Also, the articles [22,33,34] verified that MI has a constructive influence on motor rehabilitation after stroke. Since the execution of MI is interior to the patient, and therefore not directly noticeable, BCI can ease the MI-based stroke rehabilitation by providing a straight and instant response on the MI performance [22]. Moreover, the authors in [35] proved that MI could activate neuroplasticity in ipsilesional motor cortical areas despite acute paralysis after stroke.
The authors in [28] introduced a system that allowed paralyzed patients to control wheelchairs by imagining the right and left hands and feet’ movement. The authors first used surface Laplacian derivation (SLD) for spatial filtering. Then they used the Mahalanobis linear distance (MLD) classifier for classification. The authors in [36] proposed a BCI system to help the paralyzed patients control artificial limbs. In [37], the authors designed a BCI system to control a real-time wheelchair. The authors in [38] built a BCI system that successfully integrates a wheelchair with artificial limbs. They used the common spatial patterns (CSP) method to extract features and an SVM classifier. Although the system proposed in [38] was built to perform an everyday task, no feature selection was used to reduce the execution time. Edla et al. [39] proposed a system to control wheelchairs based on wavelet packet decomposition (WPD). Another approach was suggested by Roy et al. [40] to control artificial limbs. The authors in [41] proposed a system depending on spatial-frequency temporal domains to extract new informative features. The classification process was done using a sparse representation classifier (SRC). The BCI competition III–IVa dataset [42] was used later by Singh et al. [43] who proposed a BCI system that is not patient dependent. The authors of this study applied a spatial filter to reduce the data dimension.
Then using known channels, they extracted information features using the symmetric positive definite (SPD) method. Selim et al. [44] extracted features using CSP. Moreover, the authors proposed a novel feature selection model that reduces the number of extracted features. Also, they applied segmentation to remove epochs during resting time. In [45], Yuan et al. used frequency domain features by applying the Fourier transform (FT) for MI-based BCI. The authors in [46] extracted features using a time domain, including the root mean square (RMS) method. The authors compared the performance of the proposed system with different channel sets. A summary of recent related BCI systems is shown in Table 1.
This study proposes a novel efficient BCI system based on MI to differentiate between different limb motor tasks. Another goal is the make the system easier to set up. To achieve these goals, we proposed a novel system to investigate a different combination of channel sets previously used in the literature instead of using the whole number of channels available in the dataset. These channels sets were used separately in different research articles and were verified to produce efficient results. However, we examined which set has the highest performance using the hybrid feature set proposed in the paper. Second, to further reduce the number of channels, we proposed a hybrid feature selection approach that selects the most significant features that impact the BCI performance. These reduced selected features will consequently reduce the number of channels. The performance of the proposed BCI to classify MI tasks with high accuracy is tested. The possibility of reducing the number of features and channels while improving classification accuracy is also examined. The system may be considered an initial step for achieving a complete real-life--based framework to assist people with motor disability. The proposed BCI uses such people’s EEG signals to control assistive devices such as wheelchairs or wheelchairs regardless of their motor deficiency. To overcome the challenges from previous studies that used either time, frequency, or time-frequency feature extraction methods, the proposed system introduces a hybrid feature extraction used to classify MI tasks. Also, to reduce the computational cost of the BCI system, which was overlooked in previous studies, we present a feature selection procedure to determine the most significant features that are capable of differentiating between right hand and foot motor imagery tasks based on the μ frequency range (8–13 Hz) and β frequency range (13–30 Hz) of EEG signals [47]. Also, different combinations of channels are investigated to select the combination, which has the highest impact on the proposed BCI system’s performance. Finally, three machine-learning classifiers are used to translate human brain EEG signals to commands or actions. To test the proposed BCI system’s efficacy, two public datasets are used, specifically, the BCI competition III–IVa dataset and the autocalibration and recurrent adaptation dataset.
Table 1. Literature review of current BCI systems.
Table 1. Literature review of current BCI systems.
ArticleApplicationFeature ExtractionClassifierDisadvantagesAccuracy (%)
[27]Limb Motor TaskSpatial-frequency-temporal patternsSRCHigh computational cost75.46
[46]Limb Motor TaskRMSLDATime-domain features not suitable for analysis EEG datasets78.77
[38]Integrate a wheelchair and an artificial limbCSPSVMCSP suffers from degradation in performance in case of non-Gaussian distributions80
[44]Limb Motor TaskCSPSVMRequired high input channels 85.01
[29]Limb Motor TaskCSPMDRMRequired high input channels86.13
[40]Control an artificial limbDWT 1LDA QDA KNNDWT gives lower frequency resolution than WPD86.9
[36]Control artificial limb by paralyzed patientsHOS 2 + DWTLDADWT gives lower frequency resolution than WPD89.5
[39]control wheelchairsWPDSVMHigh computational cost90.698
[45]Limb Motor TaskFTMNFDFT not suitable for analysis of EEG data90.89
[28]Limb Motor TaskCSP/CSP + AR 3LDAPatient dependent and cannot be generalized on all patients in any dataset
[23]Real-time wheelchair controlFFT 4FNNFFT not suitable for analysis EEG dataset92.0
[48]Limb Motor TaskDWT + WPDKNNKNN suffers from the curse of dimensionality92.8
[28]2-D virtual wheelchair controlSLDMLDSpatial filters such as SLD may destroy features‘ information98.4
1 DWT is the discrete wavelet transform, 2 HOS is higher-order statistics, 3 AR is autoregression, 4 FFT is the fast Fourier transform.
This paper’s remainder is organized as follows: In Section 2, we describe the two datasets used and provide details for the proposed BCI model. In Section 3, we present the performance evaluation metrics, and we demonstrate the results in Section 4. In Section 5, we discuss the significance of the proposed FS on the proposed BCI’s computational cost and compare the proposed BCI system’s performance with previous studies. Finally, we provide some conclusive remarks for this study in Section 6.

2. Materials and Methods

2.1. Datasets

We used two publicly available datasets to test the effectiveness of the proposed BCI system. These datasets contain EEG signals collected during MI tasks. The two datasets are the BCI competition III–IVa dataset [42] and the autocalibration and recurrent adaptation dataset [49]. Details for these datasets are included in Appendix A and Appendix B, respectively.

2.2. The Proposed BCI System

This study proposes a novel BCI system based on MI to classify different limb motor tasks. The proposed system’s key objective is to construct a portable and easy to set up EEG-BCI based on motor-imagery training neuro-feedback system to classify motor-imagery tasks in real time with high accuracy. The proposed system can be supposed as a primary stage for attaining an entire real-life-based framework to assist individuals with motor debility. The proposed BCI system consists of the following steps: signal preprocessing, feature extraction, feature selection, and finally, classifying features. These steps are presented in Figure 1.

2.2.1. Preprocessing

Noise removal is necessary for EEG. Preprocessing is used to enhance noisy signals and remove artifacts. Sources of noise and artifacts may include power-line interference, electrocardiography (ECG), electromyography (EMG), electrooculogram (EOG), and individuals’ movements [14,50]. In this study, the preprocessing phase is divided into four steps. In the first step, a bandpass filter is used. The two EEG frequency sub-bands (μ and β rhythms) are extracted from the EEG signals by cutting the frequency from 10 Hz to 30 Hz and removing all the remaining range. The bandpass filter is implemented using a 5th order Butterworth finite impulse response (FIR) filter. Afterwards, the input signal is normalized to remove the absolute amplitude and keep the relative amplitudes. Normalization significantly reduces the channel dispersion in comparison with non-normalized recordings. Both datasets are normalized using Equation (1).
z i = w i w ¯ δ
where z i is the normalized EEG signal, w i is the original EEG signal, w ¯ and δ are the mean and standard deviation, respectively.
In the third step, a notch filter is applied to eject the power-line interference at 60 Hz from the BCI competition III–IVa dataset’s EEG signals. Please note that a notch filter was already applied to the autocalibration and recurrent adaptation dataset according to that dataset’s owners. Lastly, in the fourth step, EEG signals are segmented to extract the epochs with windows from [0.5 to 3.5] s duration for the BCI competition III–IVa dataset and from [3 to 8] s duration for the autocalibration and recurrent adaptation dataset.

2.2.2. Feature Extraction

One of the challenges in any BCI system is to extract the most representative features capable of distinguishing between different MI tasks from the processed EEG signals. Motor activities are then translated into real actions. Useful features lead to an accurate BCI system; therefore, we present two feature sets in this study. The former is extracted in the time domain, whereas. The second is extracted in the time-frequency domain. Afterwards, the two feature sets are combined to form a hybrid feature set to test feature fusion’s influence on the proposed BCI [51]. The details of feature extraction methods are presented in this section.
(1)
Feature extraction in the time domain (Feature Set 1)
Time-domain features are directly extracted from the processed signal. In this study, six features are extracted in the time domain, including RMS, Renyi entropy, and Hjorth parameters, including the following parameters: (activity, mobility, complexity), wavelength, and mean absolute value.
i.
Root Mean Square
RMS is the square root of the average of the signal’s squared value in the time domain and can be represented with the following equation [46].
RMS = 1 M j = 1 M y j 2
ii.
Renyi Entropy
Renyi uses brain activity to identify the complexity of time series. Then, the disorder in the BCI system, as illustrated in the following equation [46]:
Renyi = log j = 1 M y j q 1 q
iii.
The Hjorth parameter [52]
The Hjorth parameters include the complexity and mobility parameters.
mobility = v a r ( y j ) v a r ( y j )
Complexity = mobility ( y j ) mobility ( y j )
iv.
The waveform length [53]
Waveform length (WL) measures the signal complexity
WL = j M | y j y j 1 |
v.
The mean absolute value [48]
μ = 1 M j = 1 M | y j |
where M is the number of samples in each sub-band, y j {yj = 1, yj = 2, …, yj = M} are the samples in the time domain, and the ‘q’ parameter controls the shape of the probability distribution, v a r ( y j ) is the variance of the first derivative of the signal sample yj, v a r ( y j ) is the variance of the signal sample yj, y j   and   y j 1 are the current and previous samples in the time domain, mobility ( y j ) is the mobility of the first derivative of the signal sample yj, mobility ( y j ) is the mobility the signal sample yj, | y j | is the absolute value of the sample yj.
(2)
Feature extraction in the time-frequency domain (Feature Set 2)
FT and linear models have been commonly used to analyze the EEG signals; however, the analysis using time-frequency models achieved remarkable success over FT [54,55,56]. Among the various wavelet transform (WT) methods, the discrete wavelet transform technique (DWT) decomposes the signal in the time-frequency domain to approximation coefficients, representing the low pass filter and the detail coefficients, which represent the high pass filter. For multi-stage decomposition, only the approximate components are further decomposed, followed by down-sampling by 2. On the other hand, the WPD is an improvement or extension to the DWT. For multi-level decomposition, WPD decomposes both the approximation (lowpass) and detail (highpass) [57]. WPD produces 2 j sets of wavelet coefficients and gives a better time-frequency resolution for the decomposed signal more than the DWT [58].
In this study, four levels of WPD are used to decompose signals and then four different features are extracted from the decomposed signals. In addition to two HOS features [48].
i.
The absolute mean of coefficients in each sub-band (μ)
μ = 1 N i = 1 N | x i |
ii.
The average power of coefficients in each sub-band (Pav)
P a v = 1 N i = 1 N x i 2
iii.
The standard deviation of the coefficients in each sub-band ( σ )
σ = 1 N i = 1 N ( x i μ ) 2
iv.
The ratio of the absolute mean values of coefficients of adjacent sub-bands ( γ )
γ = 1 N i = 1 N | x i | 1 N i = 1 N | z i |
v.
The Skewness of the coefficients (signal) in each sub-band (S)
S = 1 N i = 1 N ( x i μ ) 3 σ 3
vi.
The Kurtosis of the coefficients (signal) in each sub-band (K)
K = 1 N i = 1 N ( x i μ ) 4 σ 4
where N is the cue’s length in each sub-band, and X{x1, x2, … xN} and Z{z1, z2, …, zN} are two adjacent sub-bands after WPD.
Finally, the two feature sets are combined and used to test the impact of the feature fusion process on the proposed BCI system’s performance.

2.2.3. Feature Selection

The hybrid feature set extracted in the feature extraction stage is of large dimension. Such a large dimension increases the BCI system’s classification step’s complexity and usually reduces its performance. Therefore, feature selection (FS) is essential. FS is commonly used in medical systems to decrease the feature set’s size and omit excessive and irrelevant features [59]. Feature selection is divided into three main categories which are: filter, wrapper, embedded, and hybrid. The filter FS method is the easiest and quickest method. Filter FS employs a metric for selecting features. The main drawback of such a technique is the independence of the classification process.
On the other hand, the wrapper FS is dependent on the classification procedure, but it is more complicated and slower than the previous filter method. In the embedded method, the FS process is inserted within the classifier structure. The embedded FS technique comprises the interface within the classification process. It requires less execution time than the wrapper FS method. Finally, the hybrid method fuses two or more of the previous FS methods, which usually enhances the BCI system’s performance.
For this reason, we use a hybrid feature selection method in this study, where filters and wrappers methods are combined. Most significant features are selected at the beginning of the available feature set via the computationally efficient Correlation-based Feature Selection (CFS) filter method [60]. For more refinement, the classifier subset evaluator wrapper method [51], which includes SVM, LDA, and KNN classifiers, is used to select significant features. We applied a method where the filters and wrappers are merged to select the most notable features among the ranked features.
Correlation-based Feature Selection (CFS) is a well-known filter FS method which measures the similarity between two features. If two features are correlated, the correlations coefficients value will be between (−1 to 1), and if the two features are not correlated, they will have a correlation coefficient near 0. Classifier Subset Evaluation (CSE) is one of the most commonly known wrapper feature selection methods. It uses a classifier to estimate the ‘merit’ of a set of attributes. This feature subset is used to train a classifier, and the accuracy of that classifier determines its effectiveness. The classifier subset evaluator avoids overfitting by using cross-validation measures of predictive accuracy. In this study, a forward stepwise searching strategy method is used where the model starts with no features, then iteratively adds features that improve the model performance. Once additional features no longer improve the classification accuracy, the CSE method will not add any more features to the subset [61].

2.2.4. Classification

All feature combinations on the classification accuracy are tested on three different classifiers, specifically, SVM, LDA, and KNN. Although every subject in the dataset contains separate train and test sets, they are combined into one dataset due to the low number of trials. A 10-fold cross-validation (CV) approach is employed to validate the results where the dataset is randomly split into ten different sets with equal sizes. The ten groups consist of nine training sets and one testing set. The model is trained using the nine different training sets and tested using the testing set every round. The classification accuracy (CA) of the testing set is then calculated. This process is repeated ten times, and the average accuracy of all rounds are calculated. In this study, a linear kernel was used for the SVM classifier. For the k-NN, the Euclidean distance was used. The three classifiers used in this study were implemented in MATLAB using the X, Y, Z libraries/packages versions X, Y, Z [50,62,63].

3. Performance Evaluation

Numerous metrics are used to assess the performance of the proposed BCI system. These metrics are the CA, sensitivity, specificity, F1-score, precision, and receiver operating characteristic curve (ROC) [14]. Equations (14)–(18) are used to calculate these metrics.
Accuracy is an evaluation metric used to determine how the entire data classifier has correctly classified many motor tasks. Therefore, it specifies the capability of the classifier to execute well.
Accuracy =   TP + TN TP + TN + FP + FN
Sensitivity = TP TP + FN
Specificity = TN TN + FP
where TP, FN, TN, and FP represent the true positive, false negative, true negative, and false-positive rates, respectively.
Precision is calculated as the ratio of correctly predicted positive examples divided by the summation of true positive and false-positive predictions.
Precision = TP TP + FP
The F1-score is calculated to evaluate the system performance. Classification accuracy is commonly used as it is a single measure used to summarize model accomplishment. F-Measure provides a way to combine both precisions and recall into a single measure and captures both properties.
F 1 Score = ( 2 × Precision × Recall ) Precision + Recall
The area under the receiver operating characteristic curve (AUC) is a method for evaluating models based on each point’s average on the ROC curve. The ROC curve is a plot of the true positive rate against the false-positive rate. The AUC is the area under this curve, and its value is always between 0 and 1. For a given classifier, a higher AUC value indicates a better classifier performance.

4. Experimental Results

This study aims to construct an effective BCI system based on MI to distinguish amid several limb motor tasks. Moreover, develop a portable and affordable EEG-BCI based on MI training neuro-feedback system to classify motor-imagery tasks in real time with high accuracy. The presented BCI system could be deemed a former phase for accomplishing a comprehensive real-life--based framework to help people with motor deficiencies. To achieve this goal, we present a new BCI system that consists of four experiments. Reducing the number of EEG channels used in MI tasks classification would make the BCI system more mobile and easier to set up, and maintain a real-time EEG-based BCI system. Therefore, the four experiments for the proposed BCI system are carried out on different combinations of channel sets to determine the significance of channel reduction and select the channel set, which has a higher impact on the BCI system’s performance. For the BCI competition III–IVa dataset, the full set of channels is 118 channels. Three combinations of channel sets are used. The first channel set includes the C3, Cz, and C4 channels suggested by S. Selim et al. in [35] and is known as “Channel set 1 BCI III”. The second channel set contains 18 electrodes around the sensorimotor cortex in the β frequency range, including the channels “C5, C3, C1, C2, C4, C6, CP5, CP3, CP1, CP2, CP4, CP6, P5, P3, P1, P2, P4, and P6” and it was suggested by Wang et al. [64] and known as “Channel set 2 BCI III”. In addition, finally, the third channel set further, which includes the 25 channels on the parietal lobe that was suggested in [35] and named “Channel set 3 BCI III”. On the other hand, for the autocalibration and recurrent adaptation dataset, two-channel sets were suggested. The first channel set consists of 13 channels, including C3 (FC3, C5, CP3, and C1), Cz (FCz, C1, CPz, and C2) and C4 (FC4, C2, CP4, and C6) and is named “Channel set 1 Auto”. The second channel set consists of 3 channels, C3, Cz and C4 and are called “Channel set 2 Auto”.
Figure 2 illustrates the four experiments conducted in this study that are constructed with different channel sets. The four experiments are summarized as follows:
  • Experiment 1—six time-domain features are extracted and used for classifying MI tasks.
  • Experiment 2—six time-frequency features are extracted and used for classifying MI tasks.
  • Experiment 3— a hybrid feature set is used for classifying MI tasks.
  • Experiment 4—a hybrid FS is employed to select the most relevant features and classify MI tasks.

4.1. Experiment 1—Time-Domain Features (Feature Set 1)

The results of Experiment 1 are discussed in this section for BCI competition III–IVa and autocalibration and recurrent adaptation datasets.

4.1.1. BCI Competition III–IVa Dataset Results

This section covers the effect of using only time-domain features on the classification of mental motor-imagery tasks using the BCI competition III–IVa dataset. Figure 3 shows the mean classification accuracies for the five subjects of the dataset using SVM, LDA, and KNN classifiers constructed with the three-channel sets. Figure 3 can clearly show the superiority of the channel set 3 BCI (25 channels) set over the two other electrodes sets. This is because the SVM classifiers achieved mean accuracies of 79.78% and 84.22% using the channel set 1 BCI (3 channels) and channel set 2 BCI (18 channels), respectively, which are lower than the mean CA of 86.3% achieved by the SVM classifier constructed using the channel set 3 BCI. The LDA and KNN classifiers constructed with channel set 1 BCI attained 77.54% and 78.64%, respectively. However, when using the SVM classifier, a mean accuracy of 79.78% is reached. Mean accuracy of 81.4% and 83.92% is obtained using the LDA and KNN classifiers constructed with channel set 2 BCI, whereas using the SVM classifier, an accuracy of 86.3% is reached. On the other hand, the KNN classifier constructed with channel set 3 BCI (25 channels) reached a mean CA of 84.46%, which is lower than the SVM (86.3%) constructed with the same channel set. The results of Experiment 1 show that the highest mean accuracy is achieved with the SVM classifier (86.3%) built with channel set 3 BCI (25 channels).

4.1.2. Autocalibration and Recurrent Adaptation Dataset Results

The results of Experiment 1 for the autocalibration and recurrent adaptation dataset are shown in Figure 4. This figure shows a comparison between SVM, LDA, and KNN classifiers using the channel set 2 Auto (3 channels) and channel set 1 Auto (13 channels). It is evident from Figure 4 that the SVM classifier shows a remarkable success over the other two classifiers. It is clear that the mean CA (82.23%) for all subjects for SVM constructed using the channel set 1 Auto (13 channels) is higher than that of the SVM (80.93%) constructed using the channel set 2 Auto (13 channels).

4.2. Experiment 2—Time-Frequency Domain Features (Feature Set 2)

4.2.1. BCI Competition III–IVa Dataset Results

This section discusses the results of Experiment 2 on the BCI competition III–IVa dataset. Figure 5 shows the CA results for SVM, LDA, and KNN classifiers constructed using the three different channel sets’ time-frequency features. Figure 5 shows that the SVM classifier achieved the highest mean CA among classifiers. The mean CA using SVM constructed with channel set 1 BCI (3 channels) is 81.86%. However, channel set 2 (18 channels) achieved a mean CA of 90.34% 12, which is higher than the mean CA (89.28%) performed using the channel set 3 BCI (25 channels) and with 13 channel set 1 BCI (3 channels). It is clear from Figure 5 that the time-frequency features have increased the CA of the three classifiers for the three combinations of channel sets compared to Experiment 1 results (see Figure 3). Although the performance of the proposed BCI system constructed with time features has shown the highest CA of 86.3% (Experiment 1) when applying the channel set 3 BCI (25 channels set), the time-frequency features (Experiment 2) enhanced the CA. The highest CA of 90.4% is achieved with channel set 2 BCI (18 channels), both using an SVM classifier.

4.2.2. Autocalibration and Recurrent Adaptation Dataset Results

The results of Experiment 2 for the autocalibration and recurrent adaptation dataset are shown in Figure 6. This figure shows a comparison between SVM, LDA, and KNN classifiers using the channel set 2 Auto (3 channels) and channel set 1 Auto (13 channels). SVM classifier shows the highest mean CA for all subjects. SVM classifier achieved a mean CA of (83.56%) for all subjects using the channel set 1 Auto (13 channels), which is higher than that of the SVM (82.06%) constructed using the channel set 2 Auto (3 channels).

4.3. Experiment 3—Hybrid Features

4.3.1. BCI Competition III–IVa Dataset Results

The results of the hybrid feature set for the BCI competition III–IVa dataset are shown in Figure 7. The mean CA for channel set 2 BCI (18 channels) outperforms the other channels’ accuracy for the three classifiers. Channel set 1 BCI (3 channels) showed the lowest performance with mean CA range (80.3–83.22%), whereas channel set 2 BCI (18 channels) showed the highest mean CA range (84.5–91.72%). The SVM classifier achieved the most elevated mean CA for the three-channel sets configurations. In contrast, the LDA classifier yielded the worst classification accuracy results among the three classifiers. The highest mean CA of 91.72% is achieved using an SVM classifier constructed with channel 2 BCI (18 channels).
Once a model is constructed, it is crucial to decide whether it is adequate to make robust predictions. CA alone is typically not enough metric to make this decision. Therefore, sensitivity, specificity, precision, F 1-score, and area under ROC curve (AUC) have been calculated for all subjects to give a clear and accurate evaluation of the proposed system and are shown in Table 2. All the mentioned parameters have been measured for SVM classifiers constructed with hybrid features of channel set 2 BCI (18 channels) as it showed the highest performance in Table 2. Subject “al” achieved the most heightened sensitivity, specificity, and precision. The highest F1-score and AUC of 0.989 and 1 were performed by subject “al,” as well.

4.3.2. Autocalibration and Recurrent Adaptation Dataset Results

In this section, the results are presented and discussed for the autocalibration and recurrent adaptation dataset. Figure 8 shows a comparison between CAs of SVM, LDA, and KNN for the two-channel sets. Channel set 1 Auto includes 13 channels, whereas channel set 2 consists of 3 channels. Each subject performed two or three runs. Figure 8 indicates that the SVM classifier has superior performance with CA ranging from 82.76%–85%, which is higher than that of the K-NN (81.79%–84.57%), as well as the LDA (81.48%–83.96%).
The sensitivity, specificity, precision, F1-score, and area under the ROC curve (AUC) metrics are calculated for all subjects to give a clear and accurate evaluation of the proposed system (Table 3). These metrics have been measured for SVM classifiers constructed using hybrid features of channel set 1 Auto (13 channels) since it showed the highest performance (Figure 8). It is evident that the first subject’s first-run “S01A” yielded the most heightened sensitivity, specificity, precision, and F1-score. The AUC for all subjects’ runs was equal.

4.4. Experiment 4—Feature Selection for Hybrid Features

4.4.1. BCI Competition III–IVa Dataset Results

This section covers using the introduced FS on the proposed BCI system’s performance using the BCI competition III–IVa dataset. Table 4 shows the CA and the mean CA for the five subjects of the dataset using SVM, LDA, and KNN classifiers constructed using the channel set 2 BCI (18 channels) since this channel set achieved the highest performance in Experiment 3.
In Table 4, we compare the performances of the three classifiers after applying the presented FS technique. We show that FS improves the CA from 91.72% (the highest CA achieved in Experiment 3) to 93.46% using Linear-SVM, as shown in Table 4 (an improvement of 1.74%). Also, for LDA and K-NN classifiers, the CAs have improved by 2.23% and 3.12%, respectively.
Figure 9 shows the CA for Experiment 4 using the SVM classifier constructed with hybrid features using the channel set 2 BCI (18 channels), which showed its superiority in Experiment 3. In this figure, we compare the subject CA before and after applying FS. The average CA increased from 91.72% to 93.56% using the channel set 2 BCI (18 channels).
In Table 5, we show the performance metrics for the SVM classifier constructed with the selected features of the hybrid feature set using the channel set 2 BCI (18 channels) for the BCI competition III–IVa dataset. The proposed FS has enhanced sensitivity, specificity, precision, F1-score, and AUC. For example, sensitivity for subject “aa” has increased from 0.907 to 0.914, and the specificity has increased from 0.893 to 0.907 after feature selection. On the other hand, the precision and F1-score for subject “av” have risen from 0.705 and 0.720 to 0.780 and 0.783, respectively.

4.4.2. Autocalibration and Recurrent Adaptation Dataset Results

This section describes the effect of using the introduced FS on the proposed BCI system’s performance using the autocalibration and recurrent adaptation dataset. Table 6 shows the CA and the mean CA for the 12 subjects of the dataset using SVM, LDA, and KNN classifiers constructed with the channel set 1 Auto (13 channels). This channel set achieved the highest performance in Experiment 3. Table 6 shows that the mean CA for all subject using SVM classifier reached 86.41% instead of 85.0% without FS (Experiment 3). The bold values in Table 6 represent the highest CA achieved using the SVM classifier for each subject individually and all subjects. The SVM classifier showed better performance for all subjects except for subject S01. S01 had an average CA of 99.5% when using K-NN and 99.25% for the Linear-SVM.
In Figure 10, we compare using the hybrid features extracted from the channel set 1 Auto before and after applying feature selection techniques using the SVM classifier. The results emphasize the importance of applying feature selection on each subject. After applying feature selection, CA has increased for all subjects. The mean CA has increased from 85 to 86.41% using the channel set 1 Auto.
Table 7 shows the SVM classifier’s performance metrics constructed with selected features of hybrid feature set using the channel set 1 Auto (13 channels) for autocalibration and recurrent adaptation dataset. Applying the proposed FS technique has enhanced sensitivity, specificity, precision, F1-score, and AUC. For example, sensitivity for subject “S01A” has increased from 0.936 to 0.957, and the specificity has increased from 0.929 to 0.950 as well, after feature selection. Also, the precision and F-Measure for subject “S02A” have increased from (0.705 and 0.720) to (0.780 and 0.783), respectively.

5. Discussion

This study proposes a novel efficient BCI system based on EEG data collected while performing MI tasks. The proposed system’s primary goal is to construct a portable and low-cost BCI based on MI training neuro-feedback system to classify limb MI tasks in real time with high accuracy. The proposed system is an initial step for developing a comprehensive real-life--based framework to help people with motor deficiencies to perform their daily activities. The evaluation of the proposed BCI system consists of three components. Lowering the number of EEG channels employed in the BCI system construction can lead to a more portable and easier system to set up. It also maintains a real-time EEG-based BCI system. Thus, the system’s three evaluation metrics are based on different combinations of channels set to determine the significance of channel reduction and then select the channel set with the strongest influence on the BCI system’s performance. We first compare two feature sets. The first feature set consists of statistical time features, whereas the second one consists of time-frequency features. Second, we examine the influence of fusing time and time-frequency features (hybrid feature set). The results suggest that the combination of time and frequency features increases the proposed BCI system’s performance. To create an efficient BCI system, the computational system cost should be lowered, and this could be done by selecting a reduced number of significant features. Lastly, we used an FS approach to reduce the feature set, which impacts the proposed BCI system’s accuracy.
To further evaluate the performance of the proposed FS (Experiment 4), two more parameters were considered: the number of features selected and the execution time before and after FS, as shown in Table 8 for BCI competition III–IVa and autocalibration and recurrent adaptation datasets. The results in Table 8 indicate that FS significantly decreases the BCI competition III–IVa dataset’s execution time. Moreover, the number of features was reduced. For channel set 2 (the 18 channels configuration), the average number of features was reduced from 1836 features per run to 6 features. Moreover, the average execution time decreased from 13.1964 s to 1.2694 s. FS has significantly reduced the execution time for the autocalibration and recurrent adaptation dataset as well. Also, the number of features was reduced, and the average number of features decreased from 1326 features per run to 9.679 features per run. The average execution time was also reduced from 12.6015 s to 1.8571 s.
To verify our proposed system’s effectiveness, the results were compared with the classification accuracy of recent related studies (Tables S7 and S8). In Table S7, we compare the CA of the proposed BCI system and recent studies based on the BCI competition III–IVa dataset. Although Wang et al. method [64] obtained an accuracy of 94.2%. This CA is patient dependent and is not generalizable to all patients. This is because the authors used three different feature extraction methods that are dependent on the patient. For ‘al,’ ‘aw,’ and ‘ay,’ they used the CSP algorithm on Event-Related Desynchronization (ERD). The remaining subjects used the hybrid feature set (CSP and Autoregressive (AR)) extracted from 18 channels. Singh et al. [43] designed a spatial filter which reduces the dimension of Sample Covariance Matrices. The authors achieved an average CA of 86.13%, which is lower than the proposed method. On the other hand, “Spatially Sparse CSP” filters had been implemented by Arvaneh et al. [22]. SSCSP filters have emphasized that they have heavy weights within the area of the motor cortex. They reached a CA of 73.5%, which is 19.96% lower than our proposed system. In [65], the CSP approach was used for the training trials before assigning a score to each channel based on L1 norm scores. The authors in [46] extracted the RMS feature from the time domain only. An LDA classifier was built with 18 channels as well. They reached an average CA of 78.77%, which is lower than the proposed BCI system. Miao et al. [41] achieved a CA of 86.38% after extracting features depending on spatial-frequency-temporal patterns. The results suggest that R-CSP-A considerably outperforms the other methods concerning overall CA. Selim et al. reduced the number of CSP features and used the same 18 channels employed in [44] and introduced a hybrid feature selection model. J. Kervin et al. [48] achieved a CA of 92.8% using WPD and KNN classifier. In [66], the authors implemented a regularized-CSP with aggregation (R-CSP-A), in which a few R-CSPs are aggregated, providing an ensemble-based solution. They reached an average accuracy of 83.9%. The CSP approach was applied for feature extraction in [67]. A mutual information-based frequency band selection approach was proposed and got a mean CA of 91.68%. The results in Table S7 verify the competence performance of the proposed BCI system compared to other recent studies.
In Table S8, we compare the CA of the proposed BCI system and recent studies based on the autocalibration and recurrent adaptation dataset. The authors of [49] implemented an optimized system for rapid setup and fast co-adaptive training and reached CA 76.0%, which is about 10% lower than the result achieved by our proposed system. According to the data presented in Tables S7 and S8, the proposed BCI shows promising performance compared to recent studies. The performance of the proposed BCI to classify MI tasks with high accuracy is tested. The possibility of reducing the number of features and channels while improving classification accuracy is also examined. The proposed BCI results verified that the system has successfully reduced the number of channels and features while achieving a higher accuracy, which is greater than other recent related work, as shown in Tables S7 and S8 of the Supplementary Materials. Also, the system has avoided the limitations that existed in other related work. Thus, the proposed system may be considered an initial step for achieving a complete real-life--based framework to assist people with motor disability. Future work will apply the proposed system to individuals with limb motor disabilities to test their capability to improve their QoL.

6. Conclusions

This study proposes a novel efficient BCI system based on EEG data collected while performing MI tasks. The proposed system’s core objective was to develop a portable and low-cost BCI based on MI training neuro-feedback system to classify limb MI tasks in real time with high accuracy. This system is a preliminary stage for constructing a comprehensive real-life-based framework to aid people with motor deficiencies to make their daily activities. The proposed BCI system extracts features in the time-frequency domain for different sets of electrodes. It then fuses these features to form a hybrid feature set used to train SVM, LDA, and KNN classifiers. The results showed the proposed hybrid feature had increased the accuracy of the system. A hybrid feature extraction approach was also presented. This proposed FS lowered the computation cost and enhanced the BCI system’s accuracy for both the BCI competition III–IVa and the autocalibration and recurrent adaptation datasets. For the former dataset, the hybrid fused features using 18 channels after feature selection reached a mean CA of 93.56%, which is higher than most recent studies. For autocalibration and recurrent adaptation dataset, the hybrid fused features using 13 channels after feature selection yielded a mean CA of 86.41%, which is also higher than existing BCI systems. Our proposed systems’ competitive performance encourages the efficient usage of this system in future experiments to attain a comprehensive framework to assist people with motor disabilities in performing their daily functions. Future work will focus on testing the proposed BCI system’s capacity to improve individuals’ overall QoL with motor deficiencies. Also, upcoming research will investigate the system’s ability to successfully aiding people with wheelchairs or artificial limbs.
Further future work will investigate the use of the BCI system for rehabilitation procedures. Nevertheless, different experiments are still necessary to evaluate real-world rehabilitation treatments’ performance to test its performance on enhancing the rehabilitation treatments. Additional forthcoming work will focus on using the proposed BCI system in other applications such as driving, controlling a robot, communications, etc. New multiclass datasets containing more MI tasks should be used to test the proposed system’s performance.

Supplementary Materials

The following are available online at https://www.mdpi.com/2076-3425/10/11/864/s1. Table S1: Experiment 1—Time domain features—Classification Accuracies Results for every subject of BCI competition III–IVa dataset Table S2: Experiment 1—Time domain features—Classification Accuracies (%) for every subject of the autocalibration and recurrent adaptation dataset. Table S3: Experiment 2–Time-Frequency domain—Classification Accuracies for every subject of the BCI competition III–IVa dataset. Table S4: Experiment 2—Time-Frequency domain–Classification Accuracies Results for every subject of the autocalibration and recurrent adaptation dataset. Table S5: Experiment 3—Hybrid features—Classification Accuracies for every subject of the BCI competition III–IVa dataset. Table S6: Experiment 3—Hybrid features—Classification Accuracies for every subject of the autocalibration and recurrent adaptation dataset. Table S7: A comparison between the CA of the proposed BCI system and recent related studies based on the BCI competition III–IVa dataset. Table S8: A comparison between the CA of the proposed BCI system and recent associated studies based on autocalibration and recurrent adaptation dataset.

Author Contributions

Conceptualization, O.A., M.T., and A.A.N.; Data curation, J.A.; Formal analysis, J.A. and O.A.; Investigation, J.A., O.A., M.T., and A.A.N.; Methodology, J.A., and O.A.; Resources, J.A.; Software, J.A. and O.A.; Supervision, O.A., M.T., and A.A.N.; Validation, O.A.; Visualization, J.A.; Writing—original draft, J.A. and O.A.; Writing—review & editing, O.A., M.T., and A.A.N. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

The BCI competition III–IVa dataset was collected from five healthy subjects (aa, al, av, aw, and ay). They were first seated in a comfortable chair. Then, EEG signals were acquired using 118 channels for the five subjects. Subjects were asked to perform MI tasks during EEG recording. MI tasks consisted of right-hand movement (RH) and foot movement (F). Visual signs indicated the type of MI task that each subject was to perform for 3.5 s. The number of such signs was 280. Short breaks of around 2 s were given to subjects between each successive visual sign. Electrodes were attached according to the international 10–20 system to describe the location of scalp electrodes. A bandpass filter was then applied to EEG signals from 0.05 Hz to 200 Hz. Signals were then digitized with a sampling rate of 1000 Hz then down-sampled to 100 Hz. 100 Hz signals were used in this study. The trials were unevenly split into training and evaluation trials for each subject, as shown in Table A1.
Table A1. Training and test trials for each person.
Table A1. Training and test trials for each person.
PersonTraining TrialsEvaluation of Trials
“aa”168112
“al”22456
“av”84196
“aw”56224
“ay”28252

Appendix B

The autocalibration and recurrent adaptation dataset were recorded for 12 healthy participants. These participants included seven males and five females (ages 24.8 ± 3 years). Participants sat in a comfortable chair and asked to relax. Then, EEG signals were acquired for all participants while performing a MI task using 13 electrodes, including C3 (FC3, C5, CP3, and C1), Cz (FCz, C1, CPz, and C2) and C4 (FC4, C2, CP4, and C6). These MI tasks consisted of a right-hand movement (RH) and a foot movement (F). Electrodes were attached according to the international 10–20 system to describe the location of scalp electrodes. Signals were digitized at a sampling frequency of 512 Hz. Afterwards, artifacts were removed using a bandpass filter from 0.5 Hz to 100 Hz. For this dataset, a notch filter was applied to eject the 50 Hz power-line noise. Visual signs indicated the type of MI task that each participant was to perform for 5 s. The number of such signs was 200. Short breaks of around 3s were given to the subjects between each successive visual sign presentation. The trial structure is shown in Figure A1. Two electrode sets were chosen; the 13 channels (full set) available as in27 and the 3 channels set C3, Cz, and C4.
Figure A1. Experiment Paradigm [49].
Figure A1. Experiment Paradigm [49].
Brainsci 10 00864 g0a1

References

  1. Nardone, R.; Höller, Y.; Brigo, F.; Seidl, M.; Christova, M.; Bergmann, J.; Golaszewski, S.; Trinka, E. Functional brain reorganization after spinal cord injury: Systematic review of animal and human studies. Brain Res. 2013, 1504, 58–73. [Google Scholar] [CrossRef]
  2. Yu, Z.; Prado, R.; Quinlan, E.B.; Cramer, S.C.; Ombao, H. Understanding the impact of stroke on brain motor function: A hierarchical Bayesian approach. J. Am. Stat. Assoc. 2016, 111, 549–563. [Google Scholar] [CrossRef]
  3. Purves, D.; Augustine, G.J.; Fitzpatrick, D.; Hall, W.C.; LaMantia, A.-S.; McNamara, J.O.; White, L.E. Neuroscience, 4th ed.; Sinauer Associates: Sunderland, MA, USA, 2008; Volume 857, p. 944. [Google Scholar]
  4. Serrao, M.; Chini, G.; Bergantino, M.; Sarnari, D.; Casali, C.; Conte, C.; Ranavolo, A.; Marcotulli, C.; Rinaldi, M.; Coppola, G. Identification of specific gait patterns in patients with cerebellar ataxia, spastic paraplegia, and Parkinson’s disease: A non-hierarchical cluster analysis. Hum. Mov. Sci. 2018, 57, 267–279. [Google Scholar] [CrossRef]
  5. Serrao, M.; Chini, G.; Casali, C.; Conte, C.; Rinaldi, M.; Ranavolo, A.; Marcotulli, C.; Leonardi, L.; Fragiotta, G.; Bini, F. Progression of gait ataxia in patients with degenerative cerebellar disorders: A 4-year follow-up study. Cerebellum 2017, 16, 629–637. [Google Scholar] [CrossRef]
  6. Griggs, R.C.; Józefowicz, R.F.; Aminoff, M.J. Approach to the patient with neurologic disease. In Goldman Goldman’s Cecil Medicine, 24th ed.; Schafer AI, L., Ed.; Elsevier Saunders: Philadelphia, PA, USA, 2011; pp. 2228–2235. [Google Scholar]
  7. Stamenova, V.; Black, S.E.; Roy, E.A. An update on the conceptual–production systems model of apraxia: Evidence from stroke. Brain Cogn. 2012, 80, 53–63. [Google Scholar] [CrossRef]
  8. Quiles, E.; Suay, F.; Candela, G.; Chio, N.; Jiménez, M.; Álvarez-Kurogi, L. Low-Cost Robotic Guide Based on a Motor Imagery Brain–Computer Interface for Arm Assisted Rehabilitation. Int. J. Environ. Res. Public. Health 2020, 17, 699. [Google Scholar] [CrossRef]
  9. Rafael-Palou, X.; Vargiu, E.; Dauwalder, S.; Miralles, F. Monitoring and supporting people that need assistance: The backhome experience. In Information Filtering and Retrieval; Springer: Berlin/Heidelberg, Germany, 2017; pp. 79–96. [Google Scholar]
  10. Anitha, T.; Shanthi, N.; Sathiyasheelan, R.; Emayavaramban, G.; Rajendran, T. Brain-computer interface for persons with motor disabilities-A review. Open Biomed. Eng. J. 2019, 13, 127–133. [Google Scholar] [CrossRef]
  11. Cincotti, F.; Mattia, D.; Aloise, F.; Bufalari, S.; Schalk, G.; Oriolo, G.; Cherubini, A.; Marciani, M.G.; Babiloni, F. Non-invasive brain–computer interface system: Towards its application as assistive technology. Brain Res. Bull. 2008, 75, 796–803. [Google Scholar] [CrossRef]
  12. Yaacoub, C.; Mhanna, G.; Rihana, S. A genetic-based feature selection approach in the identification of left/right hand motor imagery for a brain-computer interface. Brain Sci. 2017, 7, 12. [Google Scholar] [CrossRef]
  13. Angrisani, L.; Arpaia, P.; Casinelli, D. Instrumentation and measurements for non-invasive EEG-based brain-computer interface. In Proceedings of the 2017 IEEE International Workshop on Measurement and Networking (M&N), Naples, Italy, 27–29 September 2017; pp. 1–5. [Google Scholar]
  14. Attallah, O. An Effective Mental Stress State Detection and Evaluation System Using Minimum Number of Frontal Brain Electrodes. Diagnostics 2020, 10, 292. [Google Scholar] [CrossRef] [PubMed]
  15. Attallah, O. Multi-tasks Biometric System for Personal Identification. In Proceedings of the 2019 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC), New York, NY, USA, 1–3 August 2019; pp. 110–114. [Google Scholar]
  16. Attallah, O.; Sharkas, M.A.; Gadelkarim, H. Fetal Brain Abnormality Classification from MRI Images of Different Gestational Age. Brain Sci. 2019, 9, 231. [Google Scholar] [CrossRef] [PubMed]
  17. Hassan, M.A.; Ali, A.F.; Eladawy, M.I. Classification of the imagination of the left and right hand movements using eeg. In Proceedings of the 2008 Cairo International Biomedical Engineering Conference, Cairo, Egypt, 18–20 December 2008; pp. 1–5. [Google Scholar]
  18. Caligiore, D.; Mustile, M.; Spalletta, G.; Baldassarre, G. Action Observation and Motor Imagery for Rehabilitation in Parkinson’s Disease: A Systematic Review and an Integrative Hypothesis. Neurosci. Biobehav. Rev. 2017, 72, 210–222. [Google Scholar] [CrossRef] [PubMed]
  19. Tong, Y.; Pendy, J.T., Jr.; Li, W.A.; Du, H.; Zhang, T.; Geng, X.; Ding, Y. Motor imagery-based rehabilitation: Potential neural correlates and clinical application for functional recovery of motor deficits after stroke. Aging Dis. 2017, 8, 364–371. [Google Scholar] [CrossRef] [PubMed]
  20. Vourvopoulos, A.T.; Jorge, C.; Abreu, R.; Figueiredo, P.; Fernandes, J.-C.; Bermúdez i Badia, S. Efficacy and brain imaging correlates of an immersive motor imagery BCI-driven VR system for upper limb motor rehabilitation: A clinical case report. Front. Hum. Neurosci. 2019, 13, 244–260. [Google Scholar] [CrossRef]
  21. Nicholson, V.; Watts, N.; Chani, Y.; Keogh, J.W. Keogh Motor imagery training improves balance and mobility outcomes in older adults: A systematic review. J. Physiother. 2019, 65, 200–207. [Google Scholar] [CrossRef]
  22. Arvaneh, M.; Guan, C.; Ang, K.K.; Ward, T.E.; Chua, K.S.; Kuah, C.W.K.; Joseph, G.J.E.; Phua, K.S.; Wang, C. Facilitating motor imagery-based brain–computer interface for stroke patients using passive movement. Neural Comput. Appl. 2017, 28, 3259–3272. [Google Scholar] [CrossRef]
  23. Merante, A.; Zhang, Y.; Kumar, S.; Nam, C.S. Brain–Computer Interfaces for Spinal Cord Injury Rehabilitation. In Neuroergonomics; Springer: Berlin/Heidelberg, Germany, 2020; pp. 315–328. [Google Scholar]
  24. Louie, D.R.; Eng, J.J.; Lam, T. Gait speed using powered robotic exoskeletons after spinal cord injury: A systematic review and correlational study. J. Neuroeng. Rehabil. 2015, 12, 82. [Google Scholar] [CrossRef]
  25. Donati, A.R.; Shokur, S.; Morya, E.; Campos, D.S.; Moioli, R.C.; Gitti, C.M.; Augusto, P.B.; Tripodi, S.; Pires, C.G.; Pereira, G.A. Long-term training with a brain-machine interface-based gait protocol induces partial neurological recovery in paraplegic patients. Sci. Rep. 2016, 6, 30383. [Google Scholar] [CrossRef]
  26. Thomschewski, A.; Ströhlein, A.; Langthaler, P.B.; Schmid, E.; Potthoff, J.; Höller, P.; Leis, S.; Trinka, E.; Höller, Y. Imagine there is no plegia. mental motor imagery difficulties in patients with traumatic spinal cord injury. Front. Neurosci. 2017, 11, 689. [Google Scholar] [CrossRef]
  27. Wang, H.; Li, Y.; Long, J.; Yu, T.; Gu, Z. An asynchronous wheelchair control by hybrid EEG–EOG brain–computer interface. Cogn. Neurodyn. 2014, 8, 399–409. [Google Scholar] [CrossRef] [PubMed]
  28. Huang, D.; Qian, K.; Fei, D.-Y.; Jia, W.; Chen, X.; Bai, O. Electroencephalography (EEG)-based brain–computer interface (BCI): A 2-D virtual wheelchair control based on event-related desynchronization/synchronization and state control. IEEE Trans. Neural Syst. Rehabil. Eng. 2012, 20, 379–388. [Google Scholar] [CrossRef] [PubMed]
  29. Padfield, N.; Zabalza, J.; Zhao, H.; Masero, V.; Ren, J. EEG-based brain-computer interfaces using motor-imagery: Techniques and challenges. Sensors 2019, 19, 1423. [Google Scholar] [CrossRef] [PubMed]
  30. Rashid, M.; Sulaiman, N.; PP Abdul Majeed, A.; Musa, R.M.; Ab Nasir, A.F.; Bari, B.S.; Khatun, S. Current Status, Challenges, and Possible Solutions of EEG-Based Brain-Computer Interface: A Comprehensive Review. Front. Neurorobot. 2020, 14, 1–35. [Google Scholar] [CrossRef] [PubMed]
  31. Höller, Y.; Thomschewski, A.; Uhl, A.; Bathke, A.C.; Nardone, R.; Leis, S.; Trinka, E.; Höller, P. HD-EEG based classification of motor-imagery related activity in patients with spinal cord injury. Front. Neurol. 2018, 9, 955. [Google Scholar] [CrossRef]
  32. Müller-Putz, G.R.; Daly, I.; Kaiser, V. Motor imagery-induced EEG patterns in individuals with spinal cord injury and their impact on brain–computer interface accuracy. J. Neural Eng. 2014, 11, 035011. [Google Scholar] [CrossRef]
  33. Butler, A.J.; Page, S.J. Mental practice with motor imagery: Evidence for motor recovery and cortical reorganization after stroke. Arch. Phys. Med. Rehabil. 2006, 87, 2–11. [Google Scholar] [CrossRef]
  34. Sharma, N.; Pomeroy, V.M.; Baron, J.-C. Motor imagery: A backdoor to the motor system after stroke? Stroke 2006, 37, 1941–1952. [Google Scholar] [CrossRef]
  35. Frolov, A.A.; Mokienko, O.; Lyukmanov, R.; Biryukova, E.; Kotov, S.; Turbina, L.; Nadareyshvily, G.; Bushkova, Y. Post-stroke rehabilitation training with a motor-imagery-based brain-computer interface (BCI)-controlled hand exoskeleton: A randomized controlled multicenter trial. Front. Neurosci. 2017, 11, 400. [Google Scholar] [CrossRef]
  36. Kołodziej, M.; Majkowski, A.; Rak, R.J. A new method of EEG classification for BCI with feature extraction based on higher order statistics of wavelet components and selection with genetic algorithms. In Proceedings of the International Conference on Adaptive and Natural Computing Algorithms, Ljubljana, Slovenia, 14–16 April 2011; Springer: Berlin/Heidelberg, Germany; pp. 280–289. [Google Scholar]
  37. Abiyev, R.H.; Akkaya, N.; Aytac, E.; Günsel, I.; Çağman, A. Brain-computer interface for control of wheelchair using fuzzy neural networks. BioMed Res. Int. 2016, 2016, 1–9. [Google Scholar] [CrossRef]
  38. Li, Y.; Huang, Q.; Zhang, Z.; Yu, T.; He, S. An EEG-/EOG-Based Hybrid Brain-Computer Interface: Application on Controlling an Integrated Wheelchair Robotic Arm System. Front. Neurosci. 2019, 13, 1243–1251. [Google Scholar]
  39. Edla, D.R.; Ansari, M.F.; Chaudhary, N.; Dodia, S. Classification of facial expressions from eeg signals using wavelet packet transform and svm for wheelchair control operations. Procedia Comput. Sci. 2018, 132, 1467–1476. [Google Scholar] [CrossRef]
  40. Roy, R.; Konar, A.; Tibarewala, D.N. CONTROL OF ARTIFICIAL LIMB USING EEG & EMG-A REVIEW. In Proceedings of the AICTE Sponsored National Conference, West Bengal, India, 29–30 July 2011; pp. 1–6. [Google Scholar]
  41. Miao, M.; Wang, A.; Liu, F. A spatial-frequency-temporal optimized feature sparse representation-based classification method for motor imagery EEG pattern recognition. Med. Biol. Eng. Comput. 2017, 55, 1589–1603. [Google Scholar] [CrossRef] [PubMed]
  42. Fraunhofer FIRST; Intelligent Data Analysis Group. Campus Benjamin Franklin of the Charité Data set IVa ‹motor imagery, small training sets›. Available online: http://www.bbci.de/competition/iii/desc_IVa.html (accessed on 16 June 2020).
  43. Singh, A.; Lal, S.; Guesgen, H.W. Reduce calibration time in motor imagery using spatially regularized symmetric positives-definite matrices based classification. Sensors 2019, 19, 379. [Google Scholar] [CrossRef] [PubMed]
  44. Selim, S.; Tantawi, M.M.; Shedeed, H.A.; Badr, A. A CSP$\backslash$AM-BA-SVM Approach for Motor Imagery BCI System. IEEE Access 2018, 6, 49192–49208. [Google Scholar] [CrossRef]
  45. Yuan, H.; Doud, A.; Gururajan, A.; He, B. Cortical imaging of event-related (de) synchronization during online control of brain-computer interface using minimum-norm estimates in frequency domain. IEEE Trans. Neural Syst. Rehabil. Eng. 2008, 16, 425–431. [Google Scholar] [CrossRef] [PubMed]
  46. Selim, S.; Tantawi, M.; Shedeed, H.; Badr, A. Reducing execution time for real-time motor imagery based BCI systems. In Proceedings of the International Conference on Advanced Intelligent Systems and Informatics, Cairo, Egypt, 24–26 October 2016; pp. 555–565. [Google Scholar]
  47. Jusas, V.; Samuvel, S.G. Classification of Motor Imagery Using a Combination of User-Specific Band and Subject-Specific Band for Brain-Computer Interface. Appl. Sci. 2019, 9, 4990. [Google Scholar] [CrossRef]
  48. Kevric, J.; Subasi, A. Comparison of signal decomposition methods in classification of EEG signals for motor-imagery BCI system. Biomed. Signal. Process. Control. 2017, 31, 398–406. [Google Scholar] [CrossRef]
  49. Faller, J.; Vidaurre, C.; Solis-Escalante, T.; Neuper, C.; Scherer, R. Autocalibration and recurrent adaptation: Towards a plug and play online ERD-BCI. IEEE Trans. Neural Syst. Rehabil. Eng. 2012, 20, 313–319. [Google Scholar] [CrossRef]
  50. Baraka, A.; Shaban, H.; Abou El-Nasr, M.; Attallah, O. Wearable Accelerometer and sEMG-Based Upper Limb BSN for Tele-Rehabilitation. Appl. Sci. 2019, 9, 2795. [Google Scholar] [CrossRef]
  51. Abougharbia, J.; Attallah, O.; Tamazin, M.; Nasser, A.A.A. A Novel BCI System Based on Hybrid Features for Classifying Motor Imagery Tasks. In Proceedings of the 2019 Ninth International Conference on Image Processing Theory, Tools and Applications (IPTA), Istanbul, Turkey, 6–9 November 2019; pp. 1–6. [Google Scholar]
  52. Cecchin, T.; Ranta, R.; Koessler, L.; Caspary, O.; Vespignani, H.; Maillard, L. Seizure lateralization in scalp EEG using Hjorth parameters. Clin. Neurophysiol. 2010, 121, 290–300. [Google Scholar] [CrossRef] [PubMed]
  53. Lotte, F. A new feature and associated optimal spatial filter for EEG signal classification: Waveform Length. In Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012), Tsukuba, Japan, 11–15 November 2012; pp. 1302–1305. [Google Scholar]
  54. Herman, P.; Prasad, G.; McGinnity, T.M.; Coyle, D. Comparative analysis of spectral approaches to feature extraction for EEG-based motor imagery classification. IEEE Trans. Neural Syst. Rehabil. Eng. 2008, 16, 317–326. [Google Scholar] [CrossRef] [PubMed]
  55. Akin, M. Comparison of wavelet transform and FFT methods in the analysis of EEG signals. J. Med. Syst. 2002, 26, 241–247. [Google Scholar] [CrossRef] [PubMed]
  56. Magosso, E.; Ursino, M.; Zaniboni, A.; Gardella, E. A wavelet-based energetic approach for the analysis of biomedical signals: Application to the electroencephalogram and electro-oculogram. Appl. Math. Comput. 2009, 207, 42–62. [Google Scholar] [CrossRef]
  57. Zhang, Y.; Liu, B.; Ji, X.; Huang, D. Classification of EEG signals based on autoregressive model and wavelet packet decomposition. Neural Process. Lett. 2017, 45, 365–378. [Google Scholar] [CrossRef]
  58. Subasi, A.; Jukic, S.; Kevric, J. Comparison of EMD, DWT and WPD for the localization of epileptogenic foci using Random Forest classifier. Measurement 2019, 146, 846–855. [Google Scholar] [CrossRef]
  59. Attallah, O.; Karthikesalingam, A.; Holt, P.J.; Thompson, M.M.; Sayers, R.; Bown, M.J.; Choke, E.C.; Ma, X. Feature selection through validation and un-censoring of endovascular repair survival data for predicting the risk of re-intervention. BMC Med. Inform. Decis. Mak. 2017, 17, 115–133. [Google Scholar] [CrossRef]
  60. Karegowda, A.G.; Manjunath, A.S.; Jayaram, M.A. Comparative study of attribute selection using gain ratio and correlation based feature selection. Int. J. Inf. Technol. Knowl. Manag. 2010, 2, 271–277. [Google Scholar]
  61. Rejer, I.; Lorenz, K. Genetic algorithm and forward method for feature selection in EEG feature space. J. Theor. Appl. Comput. Sci. 2013, 7, 72–82. [Google Scholar]
  62. Ragab, D.A.; Sharkas, M.; Attallah, O. Breast cancer diagnosis using an efficient CAD system based on multiple classifiers. Diagnostics 2019, 9, 165. [Google Scholar] [CrossRef]
  63. Attallah, O.; Sharkas, M.A.; Gadelkarim, H. Deep Learning Techniques for Automatic Detection of Embryonic Neurodevelopmental Disorders. Diagnostics 2020, 10, 27. [Google Scholar] [CrossRef] [PubMed]
  64. Wang, S.; James, C.J. Extracting rhythmic brain activity for brain-computer interfacing through constrained independent component analysis. Comput. Intell. Neurosci. 2007, 2007, 041468. [Google Scholar] [CrossRef] [PubMed]
  65. Meng, J.; Liu, G.; Huang, G.; Zhu, X. Automated selecting subset of channels based on CSP in motor imagery brain-computer interface system. In Proceedings of the 2009 IEEE International conference on robotics and biomimetics (ROBIO), Guilin, China, 19–23 December 2009; pp. 2290–2294. [Google Scholar]
  66. Lu, H.; Eng, H.-L.; Guan, C.; Plataniotis, K.N.; Venetsanopoulos, A.N. Regularized common spatial pattern with aggregation for EEG classification in small-sample setting. IEEE Trans. Biomed. Eng. 2010, 57, 2936–2946. [Google Scholar] [PubMed]
  67. Kumar, S.; Sharma, A.; Tsunoda, T. An improved discriminative filter bank selection approach for motor imagery EEG signal classification using mutual information. BMC Bioinform. 2017, 18, 126–137. [Google Scholar] [CrossRef] [PubMed]
Figure 1. The proposed BCI system.
Figure 1. The proposed BCI system.
Brainsci 10 00864 g001
Figure 2. The four experiments of the proposed BCI system.
Figure 2. The four experiments of the proposed BCI system.
Brainsci 10 00864 g002
Figure 3. Experiment 1—Time-domain features—Classification Accuracies of BCI competition III–IVa dataset.
Figure 3. Experiment 1—Time-domain features—Classification Accuracies of BCI competition III–IVa dataset.
Brainsci 10 00864 g003
Figure 4. Experiment 1—Time-domain features—Classification Accuracies (%) autocalibration and recurrent adaptation dataset.
Figure 4. Experiment 1—Time-domain features—Classification Accuracies (%) autocalibration and recurrent adaptation dataset.
Brainsci 10 00864 g004
Figure 5. Experiment 2—Time-Frequency domain—Classification Accuracies—BCI competition III–IVa dataset.
Figure 5. Experiment 2—Time-Frequency domain—Classification Accuracies—BCI competition III–IVa dataset.
Brainsci 10 00864 g005
Figure 6. Experiment 2—The Time-Frequency domain features’ classification accuracies for the autocalibration and recurrent adaptation dataset.
Figure 6. Experiment 2—The Time-Frequency domain features’ classification accuracies for the autocalibration and recurrent adaptation dataset.
Brainsci 10 00864 g006
Figure 7. Experiment 3—The Hybrid features’ classification accuracies for BCI competition III–IVa dataset.
Figure 7. Experiment 3—The Hybrid features’ classification accuracies for BCI competition III–IVa dataset.
Brainsci 10 00864 g007
Figure 8. Experiment 3—Hybrid features—Classification Accuracies for the autocalibration and recurrent adaptation dataset.
Figure 8. Experiment 3—Hybrid features—Classification Accuracies for the autocalibration and recurrent adaptation dataset.
Brainsci 10 00864 g008
Figure 9. Experiment 4—The Hybrid Feature Selection using the SVM classifier constructed using the channel set 2 BCI. for the BCI competition III–IVa dataset.
Figure 9. Experiment 4—The Hybrid Feature Selection using the SVM classifier constructed using the channel set 2 BCI. for the BCI competition III–IVa dataset.
Brainsci 10 00864 g009
Figure 10. Experiment 4—Hybrid Feature Selection using SVM for the autocalibration and recurrent adaptation dataset–Channel Set 1 Auto.
Figure 10. Experiment 4—Hybrid Feature Selection using SVM for the autocalibration and recurrent adaptation dataset–Channel Set 1 Auto.
Brainsci 10 00864 g010
Table 2. Experiment 3—Hybrid features—Performance metrics for SVM classifier with channel set 2 (18 channels) for BCI competition III–IVa dataset.
Table 2. Experiment 3—Hybrid features—Performance metrics for SVM classifier with channel set 2 (18 channels) for BCI competition III–IVa dataset.
Subjects“aa”“al”“av”“aw”“ay”Mean (SD)
Sensitivity0.9071.000.7360.9640.9210.9056 (0.091)
Specificity0.8930.9790.6930.9640.9140.8886 (0.097)
Precision0.8940.9790.7050.9640.9150.8914(0.098)
F1-score0.9010.9890.7200.9640.9180.8984(0.095)
AUC0.9301.000.8301.000.9800.948(0.064)
Table 3. Experiment 3—Hybrid features—Performance metrics for SVM classifier with channel set 1 Auto (13 channels) for the autocalibration and recurrent adaptation dataset.
Table 3. Experiment 3—Hybrid features—Performance metrics for SVM classifier with channel set 1 Auto (13 channels) for the autocalibration and recurrent adaptation dataset.
Subjects“S01”“S02”“S03”Mean (SD)
RunsS01AS01BS02AS02BS03AS03B
Sensitivity0.9360.9140.8790.8790.8710.8710.892 (0.025)
Specificity0.9290.9000.8640.8640.8640.8710.882 (0.024)
Precision0.9290.9010.8660.8660.8650.8710.883 (0.024)
F1-Score0.9180.9000.8640.8640.8520.8710.878 (0.023)
ROC Area0.990.990.990.990.990.990.99 (0)
Table 4. Experiment 4—The Hybrid Feature Selection using the channel set 2 BCI (18 channels) for the BCI competition III–IVa dataset.
Table 4. Experiment 4—The Hybrid Feature Selection using the channel set 2 BCI (18 channels) for the BCI competition III–IVa dataset.
SubjectsaaalavawayCASD
Linear-SVM92.299.479.998.997.093.468.1
LDA83.9397.1571.9393.2287.486.739.72
K-NN82.198.674.396.492.188.710.24
Table 5. Experiment 4—The Hybrid Feature Selection’ performance metrics of the SVM classifier constructed with the channel set 2 BCI (18 channels) for the BCI competition III–IVa dataset.
Table 5. Experiment 4—The Hybrid Feature Selection’ performance metrics of the SVM classifier constructed with the channel set 2 BCI (18 channels) for the BCI competition III–IVa dataset.
Subjects“aa”“al”“av”“aw”“ay”Mean (SD)
Sensitivity0.9141.000.7860.9930.9500.9286 (0.0778)
Specificity0.9070.9860.7790.9930.9350.92 (0.0774)
Precision0.9010.9860.7800.9930.9370.9194 (0.0774)
F-Measure0.9310.9930.7830.9930.9370.9274(0.0769)
ROC Area0.9330.9930.7820.9930.9430.9288(0.7747)
Table 6. Experiment 4—The Hybrid Feature Selection CA using the Channel Set 1 Auto (13 channels) for the autocalibration and recurrent adaptation dataset.
Table 6. Experiment 4—The Hybrid Feature Selection CA using the Channel Set 1 Auto (13 channels) for the autocalibration and recurrent adaptation dataset.
Classifier RunsS01S02S03S04S05S06S07S08S09S10S11S12CA (SD)
SVM1st99.097.097.090.094.577.089.066.594.081.092.064.5
2nd99.597.596.092.087.583.095.075.088.070.075.066.5
3rd---------------81.096.071.581.5--
Mean for each subject99.2597.2596.591.091.080.092.074.1792.774.782.8365.586.41 (10.31)
LDA1st99.096.596.588.094.076.588.565.591.580.590.063.5
2nd99.596.595.093.085.081.594.075.087.069.073.566.0
3rd--------------81.097.070.579.5--
Mean for each subject99.2596.595.7590.589.579.091.2573.8391.8373.3381.064.7585.54 (10.44)
KNN1st99.5 96.596.588.594.077.088.566.093.080.091.064.0
2nd99.597.096.093.086.082.094.575.087.570.074.065.0
3rd---------------81.596.571.081.0--
Mean for each subject99.597.2596.2590.7590.079.591.574.1792.3373.6782.064.585.95 (10.54)
Table 7. Experiment 4—Hybrid Feature Selection—Performance Metrics of SVM classifier constructed with Channel Set 1 Auto (13 channels) for autocalibration and recurrent adaptation dataset.
Table 7. Experiment 4—Hybrid Feature Selection—Performance Metrics of SVM classifier constructed with Channel Set 1 Auto (13 channels) for autocalibration and recurrent adaptation dataset.
Subjects“S01”“S02”“S03”Mean (SD)
RunsS01AS01BS02AS02BS03AS03B
Sensitivity0.9570.9140.8930.8860.8790.8790.9058 (0.0279)
Specificity0.9500.9070.9000.8710.8790.8790.9014 (0.0277)
Precision0.9500.9070.8990.8730.8790.8790.9016 (0.0269)
F1-Score0.9500.9070.8960.8690.8790.8790.9 (0.0271)
ROC Area1.001.001.001.000.990.990.998 (0.0049)
Table 8. The number of features and execution time after the proposed Hybrid feature selection for the BCI competition III–IVa and the autocalibration and recurrent adaptation datasets.
Table 8. The number of features and execution time after the proposed Hybrid feature selection for the BCI competition III–IVa and the autocalibration and recurrent adaptation datasets.
DatasetFeatures of the Full SetFeatures after FSEx. Time before FS (s)Ex. Time after FS (s)
BCI competition III–IVa dataset1836613.19641.2694
Autocalibration and recurrent adaptation13269.67912.60151.8571
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Back to TopTop