A Decoding Prediction Model of Flexion and Extension of Left and Right Feet from Electroencephalogram

Detection of limb motor functions utilizing brain signals is a significant technique in the brain signal gain model (BSM) that can be effectively employed in various biomedical applications. Our research presents a novel technique for prediction of feet motor functions by applying a deep learning model with cascading transfer learning technique to use the electroencephalogram (EEG) in the training stage. Our research deduces the electroencephalogram data (EEG) of stroke incidence to propose functioning high-tech interfaces for predicting left and right foot motor functions. This paper presents a transfer learning with several source input domains to serve a target domain with small input size. Transfer learning can reduce the learning curve effectively. The correctness of the presented model is evaluated by the abilities of motor functions in the detection of left and right feet. Extensive experiments were performed and proved that a higher accuracy was reached by the introduced BSM-EEG neural network with transfer learning. The prediction of the model accomplished 97.5% with less CPU time. These accurate results confirm that the BSM-EEG neural model has the ability to predict motor functions for brain-injured stroke therapy.


Introduction
Most patients with stroke incidence have motor function deficiency in both left and right feet [1], causing a substantial loss of motor occupation and daily activities [1][2][3]. Stroke therapy tends to stimulate UE motor recovery and restore motor function of both feet. A main rehabilitation process is the understanding of the EEG signals to supply a non-invasive solution for the brain signal gain model (BSM), employed in all EEG signal models. BSM systems include the following steps: EEG reading, image processing, and controller [1][2][3][4]. Motor function of left and right feet target the objects in the surroundings [4,5].
The brain signal gain model (BSM) is a learning model that can capture EEG signals and convert them into motion function. BSMs are extensively found in brain-injured therapy cases. The brain signals lead to a non-intrusive answer for the BSM. BSM models the steady state of the EEG signals for motor functions feedback systems [5,6]. the images contain disparities of muscles motivated by the brain signals [7][8][9][10][11]. In our research, EEGs are captured from stroke patient cases with motor function disabilities for stroke patients. BSM systems can stimulate the motor function-lacking body part to regain the nerves of the injured parts (left and right feet in our case). Deep learning models are usually applied for BSM schemes, spatial feature selection, classification, and recognition models [10][11][12][13][14]. The researchers in [15] presented a support vector machine to classify motor signals from images. The researchers in [16] introduced the score prediction technique, and conquered accurate classification [17]. EEG signals are investigated and used in a deep learning prediction model. This model outperformed previous models especially on large datasets. Deep learning models can label properties without geometrical engineering. This defines Behav. Sci. 2022, 12, 285 2 of 14 the neural structure systems as feature selection for EEG brain signals using BSM. Current models operated deep learning systems to capture deep features. The researchers in [15] introduced a neural network with an auto-encoder with higher classification precision than prior models on the BSM-2b sets. Researchers in [16] presented a belief deep learning prediction model using the Boltzmann model. Researchers in [17] presented the envelope map of EEG signals by employing the Hilbert technique and constructed a motor imagerybased BSM prediction deep model. They employed the model to the BSM EEG-2 dataset and exhausted the most progressive prediction accuracy stated. Researchers in [16] utilized a deep learning model depiction of multiple channel EEG signal to enhance the accuracy. The researchers in [17] built 3D feature vectors of the EEG data with a parallel CNN model. The model in [18] attained high accuracy. Deep learning techniques use EEG feature mining and achieve higher precision [18][19][20][21]. However, feature mining becomes difficult due to the medical state of stroke incident cases especially for the EEG, since capturing is hard with an effect on large-size databases. The usage of these systems for motor function spatial studies in stroke cases is limited. Our model incorporates transfer learning methodologies to efficiently reduce the size of the required training set [22][23][24][25]. Features used by transfer learning utilize incident similarities and by sub-parameter inheritance [25][26][27][28]. These parameters can be reused in a reduced dataset and can increase the effectiveness of the EEGs feature learning models [28][29][30][31][32][33]. Our research contributions are summarized as depicted below: Designing a deep learning neural system with a number of additional modules and cascading transfer learning stages. Improving the precision of the BSM system for the prediction of motor functions for stroke patients from their EEG signals. Proposing an extension to the Dense-Net using parameter tuning and transfer learning (BSM-EEG). Confirming the accuracy of the proposed model by performing a comparison to similar published models.
The remainder of this paper is organized as follows. The dataset description is presented in Section 2. The proposed model with transfer learning is presented in Section 3. The experiments and performance comparison are depicted in Section 4. Section 5 concludes the work.

Data Description
The dataset contains the EEG data of 100 cases (an average of 20 different motor function for each case). The EEG signal per case continues for 3.5 s as depicted in Figure 1. The public dataset has records of the EEG signals of the patient while he is doing different motor functions of his left and right feet. Then, we let him relax for two seconds. The public dataset can be accessed by registration from https://www.bbci.de/competition/iv/ #dataset2a accessed on 12 May 2022 and https://www.bbci.de/competition/iii/#data_set_ iiia accessed on 15 May 2022.
Behav. Sci. 2022, 12, x 2 of 15 accurate classification [17]. EEG signals are investigated and used in a deep learning prediction model. This model outperformed previous models especially on large datasets. Deep learning models can label properties without geometrical engineering. This defines the neural structure systems as feature selection for EEG brain signals using BSM. Current models operated deep learning systems to capture deep features. The researchers in [15] introduced a neural network with an auto-encoder with higher classification precision than prior models on the BSM-2b sets. Researchers in [16] presented a belief deep learning prediction model using the Boltzmann model. Researchers in [17] presented the envelope map of EEG signals by employing the Hilbert technique and constructed a motor imagerybased BSM prediction deep model. They employed the model to the BSM EEG-2 dataset and exhausted the most progressive prediction accuracy stated. Researchers in [16] utilized a deep learning model depiction of multiple channel EEG signal to enhance the accuracy. The researchers in [17] built 3D feature vectors of the EEG data with a parallel CNN model. The model in [18] attained high accuracy. Deep learning techniques use EEG feature mining and achieve higher precision [18][19][20][21]. However, feature mining becomes difficult due to the medical state of stroke incident cases especially for the EEG, since capturing is hard with an effect on large-size databases. The usage of these systems for motor function spatial studies in stroke cases is limited. Our model incorporates transfer learning methodologies to efficiently reduce the size of the required training set [22][23][24][25]. Features used by transfer learning utilize incident similarities and by sub-parameter inheritance [25][26][27][28]. These parameters can be reused in a reduced dataset and can increase the effectiveness of the EEGs feature learning models [28][29][30][31][32][33]. Our research contributions are summarized as depicted below: o Designing a deep learning neural system with a number of additional modules and cascading transfer learning stages. o Improving the precision of the BSM system for the prediction of motor functions for stroke patients from their EEG signals. o Proposing an extension to the Dense-Net using parameter tuning and transfer learning (BSM-EEG). o Confirming the accuracy of the proposed model by performing a comparison to similar published models.
The remainder of this paper is organized as follows. The dataset description is presented in Section 2. The proposed model with transfer learning is presented in Section 3. The experiments and performance comparison are depicted in Section 4. Section 5 concludes the work.

Data Description
The dataset contains the EEG data of 100 cases (an average of 20 different motor function for each case). The EEG signal per case continues for 3.5 s as depicted in Figure 1. The public dataset has records of the EEG signals of the patient while he is doing different motor functions of his left and right feet. Then, we let him relax for two seconds. The public dataset can be accessed by registration from https://www.bbci.de/competition/iv/#dataset2a accessed on 12 May 2022 and https://www.bbci.de/competition/iii/#data_set_iiia accessed on 15 May 2022 . These data items are recorded and labeled in a public dataset that we utilized for our experiments [15]. The motor functions of the foot are depicted in Figure 2.  Figure 2c displays the pronation and supination of the foot in the ranges of 0-30 and 0-60 respectively The statistics of the motor function of left and right feet data are shown in Table 1. These data are extracted from the public dataset in [15]. The recorded data include the foot with all the reflexes. The dataset statistics are depicted in Tables 1 and 2.
These data items are recorded and labeled in a public dataset that we utilized for o experiments [15]. The motor functions of the foot are depicted in Figure 2.  Table  These data are extracted from the public dataset in [15]. The recorded data include t foot with all the reflexes. The dataset statistics are depicted in Tables 1 and 2.    The four preprocessing steps were as follows: 1.
Removal of noisy channels, we erased the channel AFz as it is impacted by eye blinks.

2.
Removal of static outliers using ICA using EEG signal with frequency 0.5-60 Hz to capture the outliers. We erased static outliers by applying the zero-phase band-pass filter using independent component analysis. We concentrated the EEG channels with principal component analysis and kept only components that capture 98% of the variations of the data. 3.
Detection of attempts with transitory artefacts (EEG signal from 0.5-60 Hz). We distinguished transitory artefacts using EEGProc and signaled attempts for denial with values more than −90 µV or less than 90 µV.
A total of 120 attempts were recorded for each patient as depicted from the data. We used K-fold validation dividing the data into 70%, 15%, and 15% for training, testing, and validation respectively.

Deep Learning Phase: The Proposed BSM-EEG Model
BSM-EEG is a deep learning model with cascading transfer learning model for handling EEG signals through training on EEG signals of healthy cases and the motor functions associated with them. The prediction phase is to predict the motor function from the EEG of the brain-injured cases.

Methodology
Our methodology aims to achieve a learning transfer model from other deep learning models that are trained on other motor functions for brain-injured cases, namely as upper limb movements (source domain 1) [7] and knee movements (source domain 2) [5]. Each source domain contains an average of 30,000 different labeled motor function EEG. To do so, we employed several input domains D s to get the suitable learning transfer models. Figure 3 displays the phases to accomplish this objective. We can have several input domains. For each domain D si , an optimal deep neural network was attained via Bayesian procedure. The optimization module output the parts of D si which was utilized to train the final deep learning model. The training data of the transfer learning model were chosen due to its prediction accuracy over the labeled target domain. The flow diagram of the proposed model is depicted in Figure 3.
The presented model comprises four stages: (1) Transfer training in input domain D s1 utilizing upper limb labeled, motor function labeled EEG signals. A deep neural network was trained to learn the EEG signals for upper limb motor functions. The structure of this deep learning network was optimized to realize higher accuracy. (2) Unsupervised training phase on the same dataset D s1 utilizing non-labeled data items from D s1 and from other data items not included in D s1 . We adjusted the pre-trained deep learning model from first phase by utilizing the same neural weights. (3) Fine-tuning in the target input domain D T using 271 labeled EEGs with their desired lower limb motor functions.

Architecture
To choose the suitable deep learning model with the correct weights is the Bayesian selection process [21]. Random Bayesian selection of the convolutional weight space leads to higher accuracy. In a Bayesian optimization model, the parameters of the deep learning model are computed as the optimization of an objective function. The objective function's goal is to optimize the loss function of the deep learning model by adjusting the selection space.  The presented model comprises four stages: 1) Transfer training in input domain utilizing upper limb labeled, motor function labeled EEG signals. A deep neural network was trained to learn the EEG signals for upper limb motor functions. The structure of this deep learning network was optimized to realize higher accuracy. In this paper, we present several Bayesian procedures to obtain the deep learning model that achieves better performance on the source input domains. The training process phases is depicted as follows.
The first phase was to train an initial model DL i with arbitrary preliminary parameters and optimize a loss function L i for the source input domains D si in each source domain.
For each source D s , an optimized deep learning (DL) neural network was achieved via Bayesian optimizer. During this process, the source input domain was divided into a training subset and validation subset. The DL model was verified based on the transfer learning performance using the target input domains. This Bayesian optimizer was applied on all source domain data. The model with the highest performance was chosen by computing the performance metrics on transfer learning functions using a function.
The last step was to validate the usefulness of prior models for transfer learning optimization by utilizing all target datasets by optimizing the loss function (L Bays ) of the Bayesian optimizer.
where ⇒ is a transfer learning operator. DL i is a deep learning model and is trained with only a single source input domain and its weights are transferred to the other source domains by fine-tuning the fully connected layers in DL j . The loss function Loss is calculated as a weighted (w) accuracy (acc) average and the average loss in both the learning and validation process and is computed as follows: The final step of this stage was a set of DL models equal to the count of source domains. The architecture of transfer learning training and prediction from actual labeled clinical data is depicted in Figure 4.

Training
The proposed model training was done on a Sun station CPU X6-3320 V2@ 3.60 GHz* 16 with 64 bits Linux operating system as depicted in Table 3. The deep learning model was implemented in Python 3.6.0. The method of the training was to modify the filter weights to ensure that the classified result is near to the labeled class. The utilized dataset was partitioned into three partitions. The first partition was the training subset and it included 70% of the dataset. The second partition was the validation subset and it included 15% of the dataset. The third partition was the validation subset and it included 15% of the dataset for testing the efficiency of the model. Adam optimizer was employed for fine-tuning the neural weights to minimize the loss. Table 4 depicts the hyperparameters utilized for training.

Experiment Setting
The experimental setting included determining the number of hidden layers of the DL model and the number of neurons in each layer, number of epochs, and learning rate. To define the construction of the neural structure, hidden layers and the neurons in the different layers had to be defined. The results of various hidden layer numbers and neuron counts are depicted in Table 5 and displayed in Figure 5. The count of iterations was 1900.   The various learning rate also impacts the accuracy of the neural network. We test learning rate between 0.05 and 0.15, with step of 0.02. The results of several learning ra are depicted in Table 6 and displayed in Figure 6. The results prove that the propos model had the highest performance with learning rate equals to 0.07.  The various learning rate also impacts the accuracy of the neural network. We tested learning rate between 0.05 and 0.15, with step of 0.02. The results of several learning rates are depicted in Table 6 and displayed in Figure 6. The results prove that the proposed model had the highest performance with learning rate equals to 0.07.  Figure 6. The impact of learning rate on performance.

Performance Metrics
To analyze the performance of the proposed model, several performance metrics were utilized, which proved the efficiency of the model in predicting foot movement from the EEG. The evaluation metrics were recall, f1-score, precision, and accuracy (they are defined in the following equations).

=
(3) = + (4) where is the number of true positive predictions, is the number of true negative predictions, is the number of false positive predictions, and V is th number of false negative predictions.
The classification accuracy, recall, and F1-score of our model are depicted in Table 7. The mentioned table compares between the performance metrics of our model and transfer learning with one and two source domain.

Performance Metrics
To analyze the performance of the proposed model, several performance metrics were utilized, which proved the efficiency of the model in predicting foot movement from the EEG. The evaluation metrics were recall, f1-score, precision, and accuracy (they are defined in the following equations).
where TP is the number of true positive predictions, TN is the number of true negative predictions, FP is the number of false positive predictions, and V is th FN number of false negative predictions. The classification accuracy, recall, and F1-score of our model are depicted in Table 7. The mentioned table compares between the performance metrics of our model and transfer learning with one and two source domain.

Confusion Matrix
The confusion matrices of predicting foot movement from the EEG is depicted in Tables 8-10, which display the true label (ground truth) at the y-axis and the predicted foot movement at the x-axis. The confusion matrices are for the proposed model without transfer learning (Table 8), the proposed model with transfer learning from one source domain (Table 9), and the proposed model with transfer learning from two source domains (Table 10).

Time Complexity Versus Accuracy
In this research, it was essential to compute the time complexity for the deep learning model and how transfer learning could affect the training time complexity. Moreover, it was important to see the tradeoff between the deep learning model alone and the trade off when we incorporated the transfer learning for one or more sourced domains. The results are presented in Tables 11 and 12.

Performance Comparison of Different Models
The experiments have a big role in determining the hidden layers and the optimized count of neuron with learning rate in accordance. The selected parameters were applied to our proposed deep learning model. We comparatively evaluated our models with other DL models with transfer learning with the same parameter settings. The compared models were BP neural [13], TransferN [19], DLN [21], CNN [27], and STL [31]. The parameter settings were the same for all the compared models. Since transfer learning models need relatively lengthy training times, the training time and prediction time of different models are shown in Table 13.

Conclusions
The goal of this research was to decode the left and right foot motor functions from EEG signals. The proposed deep learning model realized high prediction precision which can lead to a better a brain signal gain model (BSM) which can be employed in several limb assistive devices. The proposed research attained high accuracy by applying transfer learning from other source domains such as from elbow and knees source input domains. Our method realized higher accuracy of 97.4% by training through EEG signals of healthy cases performing motor feet functions. The presented classifier can be deployed in several classes of BSM as control signals for operative foot neuro pros. The research also concluded that the proposed BSM-EEG model with cascading transfer learning with deep learning can be competently employed on a small size input.
This research indicates that the presented model can transfer learning for the same pattern. The experimental results depict that transfer learning should be incorporated in the paradigm of EEG processing. The BSM-EEG outperformed other state-of-the-art neural deep learning models in motor imagery detection. The experiments showed that we can utilize a small-sized dataset for training by incorporating feature extraction through other source domains. The mechanism of this study can be generalized by using n source domains instead of only two source domains.