Motor Imagery Classification Improvement of Two-Class Data with Covariance Decentering Eigenface Analysis for Brain–Computer Interface Systems

Abstract

This study is intended to improve the motor imagery classification performance of two-class data points using newly developed covariance decentering eigenface analysis (CDC-EFA). When extracting the classification for the given data points, it is necessary to precisely distinguish the classes because the left and right features are difficult to differentiate. However, when centering is performed, the unique average data of each feature are lost, making them difficult to distinguish. CDC-EFA reverses the centering method to enhance data characteristics, making it possible to assign weights to data with a high correlation with other data. In experiments with the BCI dataset, the proposed CDC-EFA method was used after preprocessing by filtering and selecting the electroencephalogram data. The decentering process was then performed on the covariance matrix calculated when acquiring the unique face. Subsequently, we verified the classification improvement performance via simulations using several BCI competition datasets. Several signal processing methods were applied to compare the accuracy results of the motor imagery classification. The proposed CDC-EFA method yielded an average accuracy result of 98.89%. Thus, it showed improved accuracy compared with the other methods and stable performance with a low standard deviation.

1. Introduction

The brain is composed of quite a lot of neurons that control emotions, memories, thoughts, and activities [1,2]. Brain activity is typically produced by the electrical activity of human neurons [3]. Thus, we can measure electrical signals, which we call electroencephalograms (EEGs), or magnetic signals, which we call magnetoencephalograms (MEGs) [4]. Functional magnetic resonance, acoustic, spectroscopy, computed tomography, and positron emission tomography equipment measure brain signals using magnetic, optical, and gamma rays, respectively [5,6,7,8,9]. A brain–computer interface (BCI) is a kind of artificial system manipulating objects with signals generated [10]. Motor imagery is a type of virtual interface that can analyze neuronal activity to control external devices without using the hands or feet [11,12]. Current BCI systems can be utilized to control artificial bodies, such as electrical hand prostheses or artificial legs [13]. Therefore, it is crucial to analyze and classify brain signals with high accuracy.

A BCI system includes several parts, including signal acquirement, signal processing, and application interfaces [14]. Preprocessing, feature extraction, and classification are the three major steps in signal processing [15]. A directly received brain signal is called a raw EEG. It is necessary to select the necessary brain signals and filter out unwanted noise signals coming from the measurement instruments and electrodes during preprocessing because the raw EEG is very weak, and the sensitivity is also low [16]. In addition, the EEG signal measurement is processed during a certain period with a determined measured paradigm from multiple electrodes; therefore, it is impossible to use raw data without signal processing [17]. Consequently, a feature extraction process extracts the appropriate features and classification processes that classify the processed signals after preprocessing [18]. Therefore, the recognition of raw EEG signal processing in motor imagery in BCI systems is a very important research topic.

In the classification process, linear discriminant analysis (LDA) or a support vector machine (SVM) is generally used for the motor imagery of BCI systems [19,20]. LDA is used to reduce a dimensional that maximizes the class variances followed by a Gaussian distribution and is efficient for low computation capabilities [21]. SVM is a classification technique based on fundamental statistical learning theory; therefore, it is relatively suitable for complex classification problems, even in small datasets [22]. In the SVM method, the hyperplane is not only the boundary that divides the datasets but also the farthest from the nearest training samples to have as much space as possible between the datasets [23]. Because of this characteristic, SVM is sometimes called large-margin classification.

In the feature extraction processes, the fast Fourier transform (FFT), principal component analysis (PCA) or common spatial pattern (CSP), independent component (ICA), and eigenface analysis (EFA) methods have been widely used to provide training data and extract features from test data [24]. FFT or wavelet transform is also used to extract different features based on power spectral density [25,26]. A short-time Fourier transform is a method that divides several frames from a time-varying signal into a window frame [27]. The Morlet wavelet transform is used to detect and analyze time-varying signals to obtain power spectrum data [2]. The Hilbert–Huang transform (HHT), which is a type of Fourier transform method, utilizes arbitrary and analytic signals obtained from data and calculates the coefficient of the FFT [27]. Fundamental PCA research started as a geometric optimization problem to find the most suitable straight line or plane for data scattering in a complex multi-dimensional space and was utilized to find the principal component that maximizes the variance of the variable [28]. CSP is an algorithm that extracts features by creating a spatial filter that maximizes the dispersion difference between each signal in the brain areas according to the body part [29]. The ICA method is used to separate multiple various and complex signals into independent signals without any predetermined conditions [30]. This method is useful for generating independent signals with less noise but requires a large computation period. EFA is a feature extraction method that emphasizes data discrimination using the calculated eigenface coefficient and reducing complex dimensions [31].

We propose a newly developed covariance decentering eigenface analysis (CDC-EFA) method to improve the BCI system accuracy, specifically for two-class data points. Our proposed method, which is newly developed extended version of the results [23] is used after preprocessing through filtering of raw EEG data and data selection. Decentering is also performed on the covariance matrix calculated when acquiring a unique face. Section 2 describes the motivation for this research of the CDC-EFA method. Section 3 presents the simulated results obtained using the CDC-EFA method and currently developed methods. Section 4 is the conclusions in this research.

2. Materials and Methods

We assumed that each EEG data channel is independent of the others because brain activation patterns differ according to the motor imagery of the BCI system [32]. Based on this assumption, in the case of left and right motor imagery classification, the brain activation pattern differs according to each imagery direction; therefore, each channel can be better distinguished, and the accuracy of motor imagery classification can be increased.

Figure 1 presents a flowchart describing the fundamental process of the CDC-EFA algorithm. An explanation of each step is provided. Raw EEG data need to be preprocessed to improve the signal identity. The preprocessed data points are used in the whitening technique to differentiate the channel data based on Gram–Schmidt orthogonalization theory [31]. Therefore, the channel whitening method is helpful for channel independence. The obtained EEG data points are changed into artificial image data because the EEG data are processed with differentiated direction, so this process further helps data discrimination. After the covariance decentering process is performed, an eigenface is constructed and then, feature extraction is processed.

Centering is a method for reducing the variable correlation. The centering for random variable X is calculated using Equation (1):

$$\mathrm{Centering}=\overline{\mathrm{X}}=\mathrm{X}-\mathrm{mean}\left(\mathrm{X}\right)$$

(1)

When extracting the classification features of motor imagery, it is necessary to precisely distinguish the classes because the left and right features are difficult to differentiate. However, when centering is performed, the unique average data of each feature are lost, making them more difficult to differentiate.

The proposed decentering method reverses the centering method to enhance data characteristics. The covariance matrix calculated in the process of extracting unique facial features is a value that indicates the correlation between each data unit. The proposed covariance decentering method considers the calculated covariance matrix as a new random variable and decenters the covariance matrix by adding the average of each column to the covariance matrix. Therefore, the decentering method for an arbitrary covariance matrix is calculated using Equation (2).

$${\mathrm{C}}^{\prime }= \mathrm{C}+\mathrm{mean}\left(\mathrm{C}\right)$$

(2)

where C and C^′ are the input and output.

After adding the mean or expectation of the covariance matrix, we apply the decentering method to the data, making it possible to assign weights to data with high correlation with other data. In addition, EFA calculates a basis vector using a covariance matrix calculated from image vectors and uses the basis vector to increase the eigenvalues; thus, the data characteristics can be further emphasized. Although the original covariance matrix is symmetric, the symmetry of the covariance matrix disappears during the decentering process. Because the basis vectors are not orthogonal to each other for a non-symmetric matrix, it is necessary to find orthogonal basis vectors to determine a direction that represents the data distribution well. Therefore, in the proposed CDC-EFA method, the eigenface is constructed after the Gram–Schmidt orthogonalization-supported whitening method is performed on the eigenvector calculated from the decentered covariance matrix. Feature extraction is also performed after covariance decentralization. The application of the covariance decentering technique to the feature extraction process is calculated using Equation (3).

$$\mathrm{C}= \mathrm{cov}\left(\mathsf{\Phi }\right),{\mathrm{C}}^{\prime }=\mathrm{CDC}\left(\mathrm{C}\right), \mathsf{\Omega }={\mathrm{C}}^{\prime }\times Г$$

(3)

where the input is Φ and Г, and the output is Ω.

In the experiments, we evaluated the performance of CDC-EFA in the BCI system. The feature distribution and accuracy changes were analyzed to determine performance. In addition, the performances of the previously developed and proposed CDC-EFA methods were compared using motor image classification. The experiments were conducted using MATLAB (Ver. 2023b or later). To compare the effects of each accuracy performance improvement, we used five methods.

Table 1. Method of classification.

	Whitening	Eigenface After Decentering
EFA	X	X
Ver. 1	X	O
Ver. 2	ICA	X
Ver. 3	O	X
Ver. 4	ICA	O
Ver. 5	O	O

lists each method, with Ver. 5 indicating the results of the proposed CDC-EFA method.

The 10–20 system for EEGs is a well-known technique to show the electrode locations for BCI study [33,34]. Rather than using such symbols, we use the numbers on the electrode positions to preprocess the data in the programming.

These five methods depend on the aforementioned classifications. The following methods are divided according to whether whitening or ICA and decentering are applied. Ver. 1 uses only the decentering method, Ver. 2 uses the ICA method, Ver. 3 uses the whitening method, Ver. 4 uses the ICA and decentering methods, and Ver. 5 uses both the whitening and decentering methods. Therefore, Ver. 1, Ver. 2, Ver. 3, Ver. 4, and Ver. 5 represent the decentering, ICA, whitening, ICA combined with decentering, and proposed method (CDC-EFA), respectively.

For data extraction comparison, the EFA, ICA, and CDC-EFA methods used the publicly available BCI Competition III dataset IIa (C3D3a_2C) and BCI Competition IV dataset IIa (C4D2a_2C) data [35,36,37,38,39]. When selecting the time range to use for the EEG data, the time taken for the subject to begin imagining after confirming the motor imaginary instruction displayed on the screen was considered. While subjects were in relaxed conditions, the data for a total of 2 sec were used from 0.5 sec to 2.5 sec after the indication signal was generated. In the preprocessing stage, the BCI Competition III dataset IIIa data and BCI Competition IV dataset IIa data used in the experiment were filtered at about 7–30 Hz, including mu and beta waveforms related to motor imagery, using a fifth-order Butterworth filter. In addition, nine channels [F3_a, F_z, F4_a, C3, C_z, C4, P3_a, P_z, and P4_a] were used to represent the left/right motion image while reducing the amount of computation. Figure 2 shows the electrode locations used in the simulation.

The preprocessing of the BCI data was also filtered at approximately 7–30 Hz using a Butterworth filter, and 12 channels were used. The time was the same as in this experiment, and data for two seconds from the time point 0.5 sec after the indication point were also used. Figure 3 shows the positions of the electrodes used in the experiments.

The two classes in the feature extraction were classified; therefore, two characteristics were considered for extracting data features. For an eigenface, two fundamental vectors contain the largest eigenvalues of the fundamental vectors for space reduction and noise elimination. The currently developed methods were used to estimate the motor imagery performance results.

The LDA method was used for classification and accuracy comparison with the data.

Table 2. Classification comparison for class 1 left and class 2 right hands.

	Label
	Class 1, Left Hand	Class 2, Right Hand
Class 1, Left Hand	A, True	B, False
Class 2, Right Hand	C, False	D, True

lists the criteria for true and false answers for the left and right hands. The LDA method, the applied classification scheme that we have adapted, is not a deep learning method but a method of data regression. The LDA needs training data to obtain a fitting line or classification plan to classify the testing data. Based on the BCI competition specification [40], the BCI competition dataset is composed of two parts, training data and testing or evaluation data. Under these circumstances, we divided the BCI competition data into two parts, i.e., the training and testing datasets. “A, True” is the classification anticipated by the actual left hand for the actual left hand data. “B, False” is the classification anticipated by the incorrect left hand for the actual right hand. “C, False” is the classification anticipated by the incorrect right hand for the actual left hand. “D, True” is the classification anticipated by the actual right hand for the actual right hand data. The accuracy is the ratio of the total # of classifications to the # of true classifications, as shown in Equation (4):

$${\mathrm{A}}_{\mathrm{acc}}={\displaystyle \frac{\mathrm{A},\mathrm{true}+\mathrm{D},\mathrm{true}}{\mathrm{A},\mathrm{true}+\mathrm{B},\mathrm{false}+\mathrm{C},\mathrm{false}+\mathrm{D},\mathrm{true}}}$$

(4)

3. Results and Discussion

Table 3. EEG analysis results when using several methods.

		Subjects
		A1	A2	A3	Average
Accuracy (%)	EFA	52.22	46.67	63.33	54.07
	Ver. 1	55.56	81.67	68.33	68.52
	Ver. 2	58.62	49.10	50.00	52.57
	Ver. 3	57.78	55.00	61.67	58.15
	Ver. 4	87.36	90.91	69.64	82.64
	Ver. 5	100	98.33	98.33	98.89

shows the two-class data classification results of the motor imagery of the BCI Competition III dataset IIIa using the currently developed and proposed methods. Whitening, ICA, and covariance decentering methods have been found to improve performance. As indicated by the accuracy measurement results for Ver. 1 to Ver. 3, the accuracy increased; however, they did not reach meaningful data levels when applying these algorithms for practical patient applications.

Among the three methods, Ver. 1, which performed covariance decentering in the unique face construction stage, recorded accuracies of 55.56%, 81.67%, and 68.33%, with an average accuracy of 68.52% for the three subjects. Compared to the results obtained using Ver. 2 and Ver. 3, the performance was further improved. In particular, the results using only the whitening or ICA methods were 58.62%, 49.10%, and 50.00%, with mean accuracies of 52.57%, 57.78%, 55.00%, and 61.67%, respectively, with a mean accuracy of 58.15% recorded for three subjects. Therefore, these results indicate a significant performance improvement compared with the EFA method (54.07%).

Ver. 4 and Ver. 5 showed a significant performance improvement. Ver. 5, in which the proposed method was used in the unique face construction stage, showed better performance with an accuracy of 100%, 98.33%, and 98.33%, and an average accuracy of 98.86% for the three subjects. This method showed significantly improved results compared to the results with the ICA method and other methods, with uniformly high accuracy for all three subjects. When feature selection was compared with whitening alone, these results did not allow the automatic selection of excellent independent components, owing to the permutation problem that occurred during ICA. As many independent components as the number of input channels could not be found after performing ICA.

The whitening technique was used to improve the variance performance in the accuracy level based on Gram–Schmidt orthogonalization theory [31]. The quick-response eigenface analysis method used a three-dimensional direction so the images were converted to three-dimensional eigenface data [41]. These two techniques were also verified using the BCI competition three and four datasets together.

Figure 4a shows the results of the covariance matrix when the whitening method is not used. Figure 4b presents the results of the covariance matrix when the whitening method is applied. The left and right sides in Figure 4 show the results of the covariance matrix before and after the covariance decentering process, respectively. Figure 4c is an enlarged view of the upper two Figure 4b. After applying the whitening method, we can easily observe and compare the covariance matrix variance owing to covariance decentering from the side. In Figure 4, the values or features are obtained after the CSP filtering process. The features obtained from the CSP transformation are unitless. This is because the CSP involves a linear transformation that projects the original sensor signals into the directions associated with the maximum and minimum eigenvalues of the covariance matrices. In Figure 4, the x and y axes are the number of trials in the experiments and the z axis (height) is the value of covariance matrixes. The number of trials is different from the BCI competition dataset. In fact, in the MATLAB program, three-dimensional colored surfaces or plots intensify the height or the value from the lowest brightness to the highest brightness by a colormap, where brighter colors signify higher values, and darker colors denote lower values.

The decentering method can be used only for other data correlations so that trial data can be more accurately weighted by the decentering method. In Figure 4c, the covariance matrix of the trials with a high correlation with other data becomes larger, and the covariance trials with a low correlation become smaller when the decentering method of the covariance matrix is utilized from the side view.

Figure 5 shows the results of each method for the motor imagery classification listed in Table 3. This graph shows the average accuracy and the highest value among the classification results of motor imagery for the three subjects. The red and blue dots denote the highest and average accuracies, respectively, among the three subjects when using each method. The area from the beginning to the end of the gray area shows the results obtained using the EFA method. The area from the beginning of the green area to the area before the beginning of the pink area is the result when using only the CDC method, and the whitening method from the beginning of the pink area. The pink area shows the results obtained when using the whitening or ICA methods.

As shown in Figure 5, Ver. 1 to Ver. 3 using the EFA method, showed that the maximum accuracy was low. The difference between the average and maximum accuracy indicates that a stable performance was not achieved for all three subjects. As shown in the results for Ver. 2 and Ver. 3. When the whitening and ICA methods were applied, they did not independently improve the performance. However, when using the CDC methods together, they results were interpreted as boosting the performance.

Table 4. Accuracy comparison results when using CSP, EFA, and CDC-EFA methods.

	BCI Competition III
	Dataset IIIa			Dataset IVa
Sub	A1	A2	A3	B1		B2	B3	B4	B5
CSP	95.56	61.67	93.33	66.07		96.43	47.45	71.88	49.6
EFA	53.33	48.33	63.33	98.21		78.57	86.94	62.5	75
CDC-EFA	100	98.33	98.33	90.18		96.43	94.05	92.86	100
	BCI Competition IV
	Dataset IIa
Sub	C1	C2	C3	C4	C5	C6	C7	C8	C9
CSP	88.89	51.39	96.53	70.14	54.86	71.53	81.25	93.75	93.75
EFA	52.78	52.78	54.56	60.42	57.64	50.69	54.17	56.94	53.47
CDC-EFA	100	100	98.61	99.31	99.31	97.22	49.31	97.92	94.44

shows the accuracy comparison for two-class data when using the CSP, EFA, and proposed CDC-EFA methods for various BCI competition data. Table 4 shows the means, medians, and standard deviations calculated from

Table 5. Statistical comparison results when using CSP, EFA, and CDC-EFA methods.

	Overall
	Mean	Median	Standard Deviation
CSP	75.53	71.88	18.17
EFA	62.33	56.94	14.08
CDC-EFA	94.49	98.33	11.99

.

As shown in Table 4 and Table 5, the CDC-EFA method showed improved accuracy compared with the CSP and EFA methods. We can observe that the CDC-EFA method not only improves the average accuracy, but also obtains a stable high accuracy with a low standard deviation.

Table 6. Accuracy results when using the CDC-EFA methods with nine subjects.

	Accuracy
Subjects	EFA	CDC-EFA
1	48.61	100
2	54.86	100
3	57.64	100
4	60.42	100
5	55.56	100
6	56.25	100
7	57.64	45.14
8	57.64	100
9	54.17	100

indicates the feature classification outcomes for the EEG data for the BCI Competition IV dataset IIa. The CDC-EFA method was recorded with 100% accuracy for the BCI Competition IV dataset IIa, except for the seventh subject (45.14%). Therefore, the proposed CDC-EFA method showed superior and more stable performance compared to the existing method.

4. Conclusions

Brain–computer interfaces (BCIs) have come a long way since the first EEG measurements were made of the human brain. In the past, the main research purpose of BCI systems was for medical purposes to replace injured body parts, such as arms and legs. Technologies closely related to human life, such as the Internet of Things and wearable devices, are emerging with BCI technology devices. Accordingly, research on signal processing methods that distinguish various EEG signals more accurately and quickly, which is the core of BCI technology, is being actively conducted.

Existing research is used to improve accuracy with EEG EFA methods, considering EEG signals as images. In movement imagery problems, which are studies of EEG data accuracy, the EFA considers signals as images, unlike the CSP method mainly used in existing studies; therefore, it is possible to classify more than two classes. However, we can show that different characteristics depend on the direction because EEG signal data points are three-dimensional data. When analyzing each attempt at the implementation of BCI for 2-class data points, accuracies of 52.22%, 46.67%, and 63.33% were recorded for the three subjects in the BCI Competition III dataset IIIa, showing a low and unstable pattern.

To solve this problem, in the feature extraction process, a whitening technique was applied to the source signal, and the change in classification performance when the feature was extracted was observed. We confirmed that the accurate and stable classification of EEG data is possible using the CDC-EFA method. EEG data for the motor imagery classification problem were obtained from three subjects in the BCI Competition III dataset IIIa, which was utilized in previous studies. In addition, the performance of CDC-EFA was verified using data from nine subjects from the BCI Competition IV dataset IIa. We can confirm that the CDC-EFA method not only improves the average accuracy but also obtains a stable high accuracy with a low standard deviation. Therefore, the proposed CDC-EFA method exhibits a superior and stable performance.

The limitation of the proposed method requires several steps (channel whitening and data image conversion) before feature extraction. Therefore, low-speed computers might be undesirable to obtain a prompt response. In the future, our proposed method will be applied to three- and four-class datasets to be generalized before actual product fabrication.