Simultaneously Spatiospectral Pattern Learning and Contaminated Trial Pruning for Electroencephalography-Based Brain Computer Interface

Abstract

Electroencephalography (EEG)-based brain computer interfaces (BCIs) translate motor imagery commands into the movements of an external device (e.g., a robotic arm). The automatic design of spectral and spatial filters is a challenging task, as the frequency bands of the spectral filters must be predefined by previously published studies and given that they may be affected during trials by artifacts and improper motor imagery (MI). This study aimed to eliminate the contaminated trials automatically during classifier training, and to simultaneously learn the spectral and spatial patterns without the need for predefined frequency bands. Compared with previous studies that measured the discriminative power of a frequency band based on mutual information, this study determined the difference of the class conditional probability density function between two MI classes. This information was further shared to measure the contamination level of the trial that simplified the computation. A particle-based approximation technique iteratively constructed a filter bank that extracted discriminative features, and simultaneously removed potentially contaminated trials. The particle weight was estimated by an analysis of variance F-test instead of mutual information as commonly used in previous studies. The experimental results of a publicly available dataset revealed that the proposed method outperformed the other BCI in terms of the classification accuracy. Asymmetrical spatial patterns were found on left- versus right-hand MI classifications. The learnt spectral and spatial patterns were consistent with prior neurophysiological knowledge.

1. Introduction

A brain computer interface (BCI) provides direct communication between the brain and an external device that translates neuronal signals into user commands without the use of the peripheral nerve and muscle pathways. Many research groups have created electroencephalography (EEG)-based BCIs that offer sufficient time resolution, usability, and are associated with lower brain surgery risks [1,2,3,4] for medical and industrial applications. The analysis of EEG signals induced by motor imagery (MI), i.e., the imagery of body-part movements, has received considerable attention because of its widespread application in motor tasks, such as wheelchairs and mouse cursor controls [1]. Although many data-driven approaches have been developed to automatically design BCIs, low signal-to-noise ratios of EEG signals and the highly complex connectivity of the human brain make the decoding of the user’s intention difficult.

The EEG signal possesses task-specific features in the spectral and spatial domains during the MI task [5]. The spectral features are extracted by a spectral filter with a specific frequency band. The spectral-filtered signals are spatially transformed by a spatial filter. The spatially transformed signals are expected to be maximally discriminative between the two MI classes. Therefore, the identification of spectral and spatial filters constitutes the main BCI design issue.

In the spectral domain, task-related changes in the EEG band power induced by MI tasks, commonly referred to as event-related desynchronization (ERD) or event-related synchronization (ERS), have attracted attention in the analysis of EEG signals. A power decrease in the µ-rhythm (8–14 Hz), and a power increase in the β-rhythm (14–30 Hz) of the motor and somatosensory cortices, respectively, indicate ERD and ERS during the execution of MI tasks. Therefore, the selection of frequency bands that possess discriminative power among different MI tasks is the key issue of the BCI design. However, ERD/ERS patterns vary greatly across subjects and even across trials [6]. This makes the selection of frequency bands difficult. The spectral filter is often either manually selected based on visual inspection or is usually designed for broadband frequency operations, including both µ- and β-rhythms [6,7,8,9]. Several research studies have attempted to determine the optimal frequency bands and spatial patterns related to ERD/ERS using the techniques of signal processing and pattern recognition [7,10]. However, these approaches cannot find the optimal frequency bands in a general analytic manner.

For spatial filter optimization, machine learning has been considered as the main tool because of the increasing development of computational hardware and statistical learning theory [7]. A common spatial pattern (CSP) algorithm, a data-driven method, has been extensively used in the design of spatial filters for two-class MI problems [6,11]. This CSP identifies the spatial filter and an optimal projection direction that maximizes the variance of the data conditioned on one class. Simultaneously, the CSP minimizes the variance of the data conditioned on the other class. Raw EEG signals are then projected to this direction which makes MI classification easier. However, the CSP is highly sensitive to the spectral filter and the EEG signal. Improper spectral filters or misleading EEG signals may lead to degraded BCI performances [7,10,12]. Therefore, simultaneously optimizing spectral and spatial filters is desirable in most EEG-based BCIs.

A CSP augments spatial signals with time-delayed signals, thus resulting in double channels [13]. However, the first-order, finite impulse response, filter optimization approach, restricts the adaptability of this method. Accordingly, the time-delay parameter must be tuned by experience. The adaptability of this work was extended by introducing arbitrary order finite impulse response filters that allow the extraction of spectral features with a higher degree for the CSP [14]. However, the filter coefficients were strongly correlated with the initial parameters.

Numerous iterative procedures have been proposed to jointly optimize both the spectral and spatial filters. One approach uses a convex optimization to automatically learn feature extraction, selection, and combine them in the spectral and spatial domains [15]. An iterative spatiospectral pattern learning approach was proposed to learn spatial filters based on the optimized spectral filters and the classifier in the preceding iteration [16].

Numerous research groups developed a bank of bandpass filters to extract spectral features in various frequency bands. Filter-bank CSP (FBCSP) has been considered as the most computationally efficient algorithm that divides a broadband frequency range into small nonoverlapping frequency bands with fixed bandwidths. FBCSP then independently employs an individual CSP module to each individual frequency band to extract spatial features from EEG signals. A feature set possessing discriminative power was selected by a maximal mutual information criterion [17]. Conversely, a coefficient decimation method was developed to select a subject-specific CSP discriminative filter bank [18]. However, these methods determine optimal filters for unimodal multivariate normal distributions only. An optimal spatiospectral filter network further optimizes the spatiospectral filters by estimating the mutual information between the features and the class labels, and can thus be applied to complex data structures [19].

The frequency bands of the filter bank usually cover the µ- and β-rhythms, and the spatial filter was optimized in each frequency band which led to improved classification accuracy [17,20]. However, these methods need to predefine the frequency bands according to the neurophysiological knowledge [18,19]. A Bayesian spatiospectral filter optimization (BSSFO) was proposed to simultaneously design spectral and spatial filters without predefined frequency bands [21]. Nevertheless, the features extracted from EEG signals contaminated by noise and artifacts may not be related to the motor imagery tasks, and may thus lead to inaccurate learning of spectral and spatial patterns.

The EEG signals are easily contaminated by background noise and eye-blink artifacts which lead to low signal-to-noise ratios. Conversely, the EEG signals may be contaminated by improper motor imagery wherein the subjects do not perform the imagery tasks properly, or performed the wrong mental tasks. Data-driven BCI classifiers were sensitive to these noisy and atypical observations (contaminated trials). The performance of BCI was degraded when the contaminated trials participated in classifier training. Therefore, eliminating the contaminated trials is desired during classifier training to achieve reliable MI classification. The contaminated trials induced by muscle and eye-blinks can either be removed by independent component analysis (ICA), or rejected by threshold methods [22]. However, ICA requires a visual inspection to select the artificial components that make the implementation of an automatic BCI system nearly impossible. In addition, both the ICA and threshold methods cannot detect the noise caused by improper imagery.

A relevant dimensionality estimation-based method was proposed [23] to detect contaminated trials caused by improper motor imagery. Wang et al. proposed a trial pruning method that used a Gaussian mixture model and a genetic algorithm to eliminate the trials contaminated by artifacts and improper imagery [12]. These approaches extracted features with predefined frequency bands along with CSP. Therefore, the trial pruning method may be improved further when the spectral and spatial patterns can be automatically learnt without predefined frequency bands.

This study aims to tackle the challenges associated with the design of filter banks without predefined frequency bands and with contaminated trials for the EEG-based BCIs. This study proposes Parzen Windows-based spatiospectral patterns with trial pruning (PWSPTP) to achieve the research goal. The main contributions of the present study are summarized as follows:

• A particle-based approximation technique was developed to iteratively construct a filter band and detect potential contamination trials. The spectral and spatial features were learnt as the contaminated trials were eliminated during classifier training that led to improved MI classification accuracy;
• The discriminative power of a feature and the contamination level of a trial were simultaneously estimated by the difference of the class conditional probability density function (pdf) instead of using mutual information. The class conditional pdf was estimated by the Parzen window density estimator;
• The importance weight (particle weight) of each feature was estimated by analysis of variance (ANOVA) F-tests with the use of the class conditional pdf that can be simply implemented.

The remaining parts of this study are organized as follows: Section 2 illustrates the proposed PWSPTP and provides a better understanding of the relationship between PWSPTP and other methods, including similarities and differences. Section 3 demonstrates the performance of the PWSPTP for MI classification. Finally, Section 4 summarizes the present study.

2. Parzen Windows-Based Spatiospectral Patterns with Trial Pruning

This study aimed to design an EEG-based BCI that could automatically eliminate the contaminated trials during classifier training, and to simultaneously learn the spectral and spatial patterns without predefined frequency bands and negative effects of the contaminated trials. Inspired by a particle-based approximation technique, this study constructs a filter bank with overlapping frequency bands. The discriminative power of each frequency band is estimated by the Parzen window density estimator. Compared with BSSFO, the present study measured the discriminative power of the frequency bands based on the difference of the class conditional pdf of two MI classes instead of using the mutual information. Furthermore, the particle weight was estimated by analysis of variance (ANOVA) F-tests instead of mutual information. Conversely, inspired by the trial pruning method proposed by [12], the present study eliminated the contaminated trials during classifier training. The trials potentially contaminated by artifacts and improper imagery are detected by the class conditional pdf. This study combines the merits of the particle-based approximation technique and trial pruning method to simultaneously measure the discriminative power of the frequency band and eliminate the contaminated trials. This innovation overcame the limitation of the trial pruning method that adopted fixed nonoverlapping frequency bands and overcame the negative effects of the contaminated trials when learning the spectral and spatial patterns.

PWSPTP mainly consisted of three parts: (a) estimation of the discriminative power of the feature, (b) estimation of the contamination level of a trial, and (c) composition of a filter bank without contaminated trials. Figure 1 shows the plot of the PWSPTP. In the training phase, a set of EEG signals corresponding to two MI classes were processed by a set of spectral filters (filter bank) and spatial filters (CSP). The discriminative power of spatiospectral features and contamination level of a trial were quantified by a Parzen window density estimator. A filter bank was constructed by the particle filter whose particle weight was estimated by an ANOVA F-test. A spectrally weighted classifier was trained according to the learnt spectral and spatial features without contaminated trials. In the testing phase, the classifier made an MI prediction based on the features extracted by the learnt spatiospectral filters.

2.1. Spatiospectral Filter Optimization

Consider a single-EEG trial $x\in {ℝ}^{\mathsf{\tau }}$ of a MI task, where τ denotes the number of electrode channels. This study considered a binary classification problem whose class label ω was positive (+) or negative (−), e.g., left- or right-hand motor imagery. The features of x are usually extracted by the following three steps:

(1)

spectral filtering: $z=h\otimes x$;

(2)

spatial filtering: $y=W^{T}z$;

(3)

feature extraction: $f=\log \left(\mathrm{var}\left(y\right)\right)$;

where h represents the spectral filter, $W$ represents the spatial filter, $z$ represents the spectrally filtered EEG, $y$ represents the spatially filtered signal, and $\mathrm{var}$ represents the variance function. Each spectral filter is specified by the start $b^s$ and the end $b^e$ of the frequency band, whereby $b^s<b^e$. The goal was to find a set of spectral filters whose features could be correctly discriminated between two MI tasks. The function $p\left(\mathit{b}\right)$ of the random variable $\mathit{b}={\left[\begin{array}{cc}{b}^s& b^e\end{array}\right]}^T$ defines the probability of the spatially filtered signal and its correct classification between the two MI tasks. Given K trials of EEGs $\mathit{X}={\left\{{\mathit{x}}_k\right\}}_{k=1}^K$ and their corresponding class labels $\Omega ={\{{\omega }_k\}}_{k=1}^K$, $\omega \in \left\{+,-\right\}$, a posterior pdf was defined as $p\left(\mathit{b}|\mathit{X},\Omega \right)$. As the spectrally filtered EEG $z$, spatially filtered signal $y$, and the feature $f$ were computed deterministically, $p\left(\mathit{b}|\mathit{X},\Omega \right)$ could be directly obtained by a posterior pdf $p\left(\mathit{f}|\mathit{X},\Omega \right)$.

Inspired by the particle-based approximation [21], a weighted particle-set ${\mathfrak{B}}^{\mathrm{old}}={\{{\mathit{b}}_n^{old},{\pi }_n,{\mathit{M}}_n\}}_{n=1}^N$ was generated to approximate $p\left(\mathit{f}|\mathit{X},\Omega \right)$, where $\pi$ denotes the importance weight of the particle, N denotes the number of particles, ${\mathit{b}}_n$ denotes a particle (a single-frequency band), and ${\mathit{M}}_n={\{m_n^k\}}_{k=1}^K$ denotes a set of noisy trial marks with $m_n^k\in \left\{1,0\right\}$. Each particle represents a frequency band of a spectral filter and its corresponding noisy trial mark. An individual CSP module was independently performed based on the features that correspond to one spectral filter to obtain ${\widehat{\mathit{W}}}_n$ [8]. The features ${\{{\mathit{f}}_n\}}_{n=1}^N$ were extracted by these spectral filters (particles) and their corresponding spatial filters. The importance weight of each particle ${\pi }_n$ was computed by a class conditional pdf (detailed in Section 2.2). The noisy trial mark $m_n^k$ was updated by the class conditional pdf as well (detailed in Section 2.3). For the next sampling iteration, a new set of particles was generated by an effective prior ${\mathfrak{B}}^{\mathrm{old}}$, as defined in the previous iteration. A particle ${\mathit{b}}_j^{old}$ was randomly chosen from ${\mathfrak{B}}^{\mathrm{old}}$ by roulette wheel selection with the use of ${\pi }_j$. In other words, the chance for the selection of ${\mathit{b}}_j^{old}$ was proportional to ${\pi }_j$. A detailed description of roulette wheel selection could be found in [24]. The chosen ${\mathit{b}}_j^{old}$ was copied as the new particle ${\mathit{b}}_n^{new}$ for the next iteration. If ${\mathit{b}}_j^{old}$ has been chosen multiple times, a diffusion method was applied to avoid identical copies of the particle and was implemented by adding noise to the particle as follows,

$${\mathit{b}}_n^{new}={\mathit{b}}_j^{old}+\mathbf{\sigma }$$

(1)

where $\mathbf{\sigma }$ is a Gaussian noise with a zero mean and a unit variance. This ensured that the particle with a large discriminative power could be reserved and was prevented from converging to a local optimum [25]. Once a set of new particles ${\{{\mathit{b}}_n^{new}\}}_{n=1}^N$ was obtained, the new particles replaced old particles ${\{{\mathit{b}}_n^{old}\}}_{n=1}^N$. Iterative sampling of the particle around the current particle with the importance weight helped the particles converge to $p\left(\mathit{f}|\mathit{X},\Omega \right)$. The resulted particles constructed a weighted filter bank whose spectral filter bandwidths may be different from those of other particles and may overlap each other. This overcame the limitation of the predefinition of the frequency bands according to accumulated neurophysiological knowledge [17,19].

2.2. Importance Weight Update Rule for Particles

The importance weight of each particle was updated based on the evaluation of the discriminative power of the features. Compared with the BSSFO [21], the discriminative power of a feature was estimated based on the difference of the class conditional pdf instead of mutual information. Furthermore, the class conditional pdf was also adopted to measure the contamination level of a trial.

Inspired from the F-score for feature selection [26], ANOVA [27] in conjunction with class conditional pdf was used to measure the discriminative power of a feature (particle). It has been known that the trials with low signal-to-noise ratios and the trials contaminated by improper motor imagery may reduce the discriminative power of a feature. The trials with low signal-to-noise ratios possessed low class conditional probabilities. The trials contaminated by improper motor imagery possessed higher class conditional probability for the improper class than the class assigned in the MI task. Therefore, a difference of the class conditional pdf for the trial of the class $+$ was defined as

$$e_n^{k+}=\frac{p({\mathit{f}}_n^k|+)-p({\mathit{f}}_n^k|-)}{p({\mathit{f}}_n^k|+)+p({\mathit{f}}_n^k|-)}$$

(2)

while that for the trial of the class $-$ was defined as

$$e_n^{k-}=\frac{p({\mathit{f}}_n^k|-)-p({\mathit{f}}_n^k|+)}{p({\mathit{f}}_n^k|+)+p({\mathit{f}}_n^k|-)}$$

(3)

The discriminative power of a feature was large when $e_n^k$ was large, and vice versa. Furthermore, $e_n^k$ was small for the trial contaminated by noise and artifacts. Thus, this value could be adopted to detect contaminated trials as well (detailed in Section 2.2). The class conditional pdf $p({\mathit{f}}_n^k|\omega =c)$ was determined using a Parzen window density estimator [28,29] as follows:

$$p({\mathit{f}}_n^k|\omega =c)=\frac{1}{K_c}{\displaystyle \sum }_{\left(j\mathrm{s}.\mathrm{t}.{\omega }_j=c\right)}\psi \left({\mathit{f}}_n^k-{\mathit{f}}_n^j,\lambda \right)$$

(4)

where

$$\psi \left({\mathit{f}}_n^k-{\mathit{f}}_n^j,\lambda \right)=\frac{1}{{\left(2\pi \right)}^{r/2}{\lambda }^r{\left|\Sigma \right|}^{1/2}}\exp \left[-\frac{{\left({\mathit{f}}_n^k-{\mathit{f}}_n^j\right)}^T{\Sigma }^{-1}\left({\mathit{f}}_n^k-{\mathit{f}}_n^j\right)}{2{\lambda }^2}\right]$$

(5)

where $c\in \left\{+,-\right\}$, $K_c$ denotes the number of trials belonging to class c, $\lambda$ denotes the width of the window function, $\Sigma$ denotes a covariance matrix, and r denotes the dimension of the feature. Note that a multivariate Gaussian window function $\psi (·)$ was adopted.

The importance weight of each particle ${\pi }_n$ was defined as follows,

$${\pi }_n=\frac{\mathrm{variance}\mathrm{between}\mathrm{groups}{\left\{e_n^{k+}\right\}}_{k\in J}\mathrm{and}{\left\{e_n^{k-}\right\}}_{k\in J}}{\mathrm{variance}\mathrm{within}\mathrm{groups}{\left\{e_n^{k+}\right\}}_{k\in J}\mathrm{and}{\left\{e_n^{k-}\right\}}_{k\in J}}$$

(6)

where $J=\underset{k}{{\displaystyle \cup }}\left(m_n^k=1\right),k\in \left\{1,2,\dots ,K\right\}$ represents a set of trials that are not marked as noisy trials. In other words, the importance weight was calculated exclusively for noisy trials to ensure the discriminative power of a feature (particle) could be carefully measured without the effect of noisy trials. The numerator denotes the discrimination between two MI classes, while the denominator denotes the discrimination within each MI class based on the use of the difference of the class conditional pdf.

2.3. Trial Pruning Process

Trial pruning aimed to prune trials contaminated by noise, artifacts, and improper motor imagery in the training data. The contamination level of the trial was measured by the class conditional pdf that was also adopted in the importance weight-update rule of the particles. The trial contaminated by noise and artifacts were identified as a type-1 noisy trial and had low $p({\mathit{f}}_n^k|\omega =c)$ for either the $+$ or $-$ classes, and led to small sums $p({\mathit{f}}_n^k|+)+p({\mathit{f}}_n^k|-)$. Therefore, the sum of the class conditional pdf was used as a type-1 noisy indicator and was defined as follows,

$$q_n^k=p({\mathit{f}}_n^k|+)+p({\mathit{f}}_n^k|-)$$

(7)

Conversely, the trials contaminated by improper motor imagery were identified as type-2 noisy trials. These trials had lower $p({\mathit{f}}_n^k|+)$ values than $p({\mathit{f}}_n^k|-)$ when the assigned class label was $+$, and vice versa. This led to negative $e_n^{k+}$ or $e_n^{k-}$ values. Therefore, the difference of the class conditional pdf, $e_n^k$, was used as a type-2 noisy indicator, while it was also used to update the importance weight of each particle.

Once the contamination level of each trial has been measured, $q_n^k$ was sorted in order as follows,

$$q_n^{u_{+}\left(1\right)}\le q_n^{u_{+}\left(2\right)}\le q_n^{u_{+}\left(3\right)}\le \cdots \le q_n^{u_{+}(K_{+})}$$

(8)

$$q_n^{u_{-}\left(1\right)}\le q_n^{u_{-}\left(2\right)}\le q_n^{u_{-}\left(3\right)}\le \cdots \le q_n^{u_{-}(K_{-})}$$

(9)

where $u_{+}(·)$ and $u_{-}(·)$ denote the sorted indices of the trials in classes $+$ and $-$, respectively, and $K_{+}$ and $K_{-}$ denote the numbers of trials in classes $+$ and $-$, respectively. The first $\lfloor {\epsilon }_1\ast K_{+}\rfloor$ and $\lfloor {\epsilon }_1\ast K_{-}\rfloor$ noisy trial marks of $m_n^k$ were set to zero for classes $+$ and $-$, where ${\epsilon }_1$ denote a predefined percentage of the type 1 noisy trials and $\lfloor ·\rfloor$ represent the “flooring” operation. In other words, type-1 noisy trials are detected as follows,

$$m_n^k=0,k\in \left\{u_{+}\left(1\right),u_{+}\left(2\right),\dots ,u_{+}\left(\lfloor {\epsilon }_1\ast K_{+}\rfloor \right),u_{-}\left(1\right),u_{-}\left(2\right),\dots ,u_{-}\left(\lfloor {\epsilon }_1\ast K_{-}\rfloor \right)\right\}$$

(10)

For type-2 noisy trials, $e_n^{k+}$ and $e_n^{k-}$ were sorted according to the following order,

$$e_n^{v_{+}\left(1\right)}\le e_n^{v_{+}\left(2\right)}\le \cdots \le e_n^{v_{+}\left(K_{+}^0\right)}\le 0\le \cdots \le e_n^{v_{+}(K_{+})}$$

(11)

$$e_n^{v_{-}\left(1\right)}\le e_n^{v_{-}\left(2\right)}\le \cdots \le e_n^{v_{-}\left(K_{-}^0\right)}\le 0\le \cdots \le e_n^{v_{-}(K_{-})}$$

(12)

where $v_{+}(·)$ and $v_{-}(·)$ denote the sorted indices of the trials in classes $+$ and $-$, respectively. The first $\lfloor {\epsilon }_2\ast K_{+}\rfloor$ and $\lfloor {\epsilon }_2\ast K_{-}\rfloor$ noisy trial marks of $m_n^k$ were set to zero for classes $+$ and $-$, respectively, where ${\epsilon }_2$ denotes a predefined percentage of the type-2 noisy trials. In other words, type-2 noisy trials were detected as follows,

$$m_n^k=0,k\in \left\{v_{+}\left(1\right),v_{+}\left(2\right),\dots ,v_{+}\left(\lfloor {\epsilon }_2\ast K_{+}^0\rfloor \right),v_{-}\left(1\right),v_{-}\left(2\right),\dots ,v_{-}\left(\lfloor {\epsilon }_2\ast K_{-}^0\rfloor \right)\right\}$$

(13)

Note that the spatiospectral filter (corresponding to one particle) may filter out the noise that leads to a high $q_n^k$ value. Therefore, the noisy trials in one particle may be different from those in another particle. These contaminated trials were iteratively pruned when the CSP was executed at the instant at which the importance weight of the particle was updated. The impact of the contaminated trials could then be reduced during the design of the filters and classifier in the training phase. As a result, a set of spectral and spatial filters could be iteratively constructed by simultaneously estimating the discriminative power of the features and contamination levels of the trials. The overall algorithm of the proposed PWSPTP is summarized in Figure 2.

2.4. Mixture of Expert Classifiers

It has been determined that a mixture of expert classifiers could improve the classification accuracy [30]. The present study linearly combined multiple classifiers to predict the MI class. Each classifier $s_n$ played the role of one expert and was trained by the features extracted from one spectral filter with bandwidth ${\mathit{b}}_n$ and one spatial filter ${\widehat{\mathit{W}}}_n$. The mixing coefficient of $s_n$ was the importance weight ${\pi }_n$. A support vector machine (SVM) that possessed a strong classification performance [31] was used as the classifier. Note that these classifiers were trained after the spectral and spatial filters were optimized by the PWSPTP and the contaminated trials were removed during the training process. The output of the mixture of expert models was

$$\widehat{\omega }=\{\begin{array}{cc}+,& 0\le {\displaystyle {\displaystyle \sum }_{n=1}^N}{\pi }_n·s_n\left({\mathit{f}}_n\right)\\ -,& 0>{\displaystyle {\displaystyle \sum }_{n=1}^N}{\pi }_n·s_n\left({\mathit{f}}_n\right)\end{array}$$

(14)

2.5. Performance Evaluation

To demonstrate the efficiency of discriminative feature extraction of the PWSPTP, this study considered the evaluation protocol of paired binary classification, i.e., left- versus right hand, left hand versus foot, left hand versus tongue, right hand versus foot, right hand versus tongue, and foot versus tongue. This protocol evaluated the discriminative power of the features extracted by the learnt frequency bands when the contaminated trials were eliminated during the training process.

The classification accuracy of the evaluation protocol was the percentage of correctly classified trials. All experiments were conducted with MATLAB (Version R2019a, MathWorks, Natick, MA, USA), with an Intel (R) Core (TM) i7-6900K central processing unit (CPU) @3.2 GHz, a 64 GB random access memory (RAM), and the Microsoft Windows 10 operating system.

3. Experimental Results and Analysis

This section introduced the dataset used in the experiment and then presented the setting of the proposed PWSPTP. The classification results of the PWSPTP were presented when FBCSP and BSSFO are compared. Finally, a discussion is provided to analyze the experimental results.

3.1. Dataset

BCI Competition IV Datasets 2a provided by the Institute for Knowledge Discovery were used for the evaluation of the proposed PWSPTP [32]. Nine subjects performed four MI tasks (left hand, right hand, both feet, and tongue). One training session and one testing session were recorded with 22 EEG channels and were sampled with a frequency of 250 Hz for each subject. Each session consisted of 288 trials with 72 trials for each MI task. The EEG signals in the dataset were originally bandpass filtered (0.5 Hz–100 Hz) and further processed by a 50 Hz notch filter to suppress line noise. This study bandpass filtered (4 Hz–40 Hz) the EEG signals to cover the µ (8–14 Hz) and β (14–30 Hz) rhythms. Although prior neurophysiological knowledge was adopted, there were no predefined frequency bands in the filter bank. A small Laplacian derivation [33] was applied to each electrode using anterior, posterior and both lateral electrodes to increase the signal-to-noise ratio. Following the same protocol as that adopted by [34], the time segment with the length of 2 s between 0.5 s and 2.5 s after visual cue onset in each trial was adopted for MI classification in both training and testing sessions. Detailed descriptions of the BCI Competition IV Datasets 2a could be found in [32].

3.2. Setting of PWSPTP

The initial prior distribution which was adopted to generate a set of particles was set as a mixture of Gaussian functions according to prior neurophysiology knowledge

$$p\left(\mathit{b}\right)=\frac{1}{2}\mathcal{N}\left(\mu ,\mathsf{\Sigma }\right)+\frac{1}{2}\mathcal{N}\left(\beta ,\mathsf{\Sigma }\right)$$

(15)

where µ and β represent the mean frequencies of the µ- (8–14 Hz) and β-rhythms (14–30 Hz), respectively, and the covariance $\mathsf{\Sigma }$ was set to be an identical matrix. Therefore, the start $b^s$ and the end $b^e$ of the frequency band (particle) were initially sampled from the mixture of Gaussians.

The half number of the spatial patterns, d, was set to one. A Gaussian kernel-based SVM was used in the present study owing to its superior classification performance of BCI studies [7,21,35].

3.3. Effects of Numbers of Particles

The number of particles (N) was the hyperparameter of the PWSPTP. To investigate the effects of the hyperparameter to the MI classification accuracy, PWSPTP used various N values were implemented. Figure 3 shows the experimental results with various N values on the left- and right-hand MI classification accuracy for subject 2. The classification accuracy was improved as N increased, but was not monotonically improved when N was larger than 35. Therefore, the PWSPTP was used for the construction of the filter bank with 35 frequency bands.

3.4. Effects of Pruning Percentages

The pruning percentages (${\epsilon }_1$ and ${\epsilon }_2$) determined the numbers of type-1 and -2 noisy trials and were designed empirically. The previous study revealed that the trial pruning method improved the classification accuracies of the subjects that were associated with low-MI classification accuracies [12]. The left-hand versus right-hand classification of subject 2 has been known as a strongly confused MI binary classification [12,21,36]. The left- and right-hand MI classifications of subject 2 were conducted to analyze the effects of various ${\epsilon }_1$ and ${\epsilon }_2$, as presented in Figure 4.

Increasing the values of ${\epsilon }_1$ and ${\epsilon }_2$ improved the classification accuracy when ${\epsilon }_1$ and ${\epsilon }_2$ were smaller than 3% and 7%, respectively. The classification accuracy decreased when ${\epsilon }_1$ and ${\epsilon }_2$ were larger than 3% and 7%, respectively. Therefore, ${\epsilon }_1$ and ${\epsilon }_2$ were empirically set to 3% and 7%, respectively, to remove two types of noisy trials during the training phase.

3.5. Visualization of Spatial Patterns and Distribution of Discriminative Frequency Band

The most significant spatial patterns learnt along with the spectral filter (particle) with maximum importance weight value of subject 1 are shown in Figure 5. These spatial patterns are distinguishable among six MI pairs, and were consistent with the neurophysiological plausibility [5]. The learnt spatial patterns identified the attenuations and increases in the localized neural rhythmic activities and helped improve the MI classification.

The distribution of the discriminative frequency band learnt by the proposed PWSPTP is presented in Figure 6. The probability was computed within an interval of 0.5 Hz. Most optimized particles distributed around the µ-rhythm. This indicates that these discriminative frequency bands matched the neurophysiological knowledge.

3.6. Classification Performance

The classification performance of the PWSPTP was averaged over ten independent runs owing to its stochastic properties. The experimental results of six paired binary MI classifications among nine subjects are presented in

Table 1. Classification performance of the PWSPTP for six paired binary MI classifications among nine subjects. In this case, L, R, F, and G denote left hand, right hand, foot, and tongue respectively, and μ and σ denote mean and standard deviation, respectively. Bold values indicate the highest classification accuracy among the six paired binary MI classifications.

Class	A1	A2	A3	A4	A5	A6	A7	A8	A9	μ ± σ
LvsR	83.33	56.25	93.06	65.97	61.81	69.44	65.28	87.50	83.33	74.00 ± 12.96
LvsF	95.14	85.42	97.92	73.61	65.50	64.58	93.06	82.64	88.19	82.90 ± 12.44
LvsG	94.44	62.50	95.14	86.11	74.31	64.58	81.94	91.67	90.28	82.33 ± 12.49
RvsF	97.22	91.67	84.03	77.78	52.78	63.89	98.61	50.56	69.44	76.22 ± 18.20
RvsG	98.61	65.97	95.14	80.56	68.75	58.33	94.44	73.61	82.64	79.78 ± 14.25
FvsG	79.17	91.67	81.94	66.67	53.47	69.44	68.06	76.39	75.69	73.61 ± 10.84
μ ± σ	91.32 ± 8.05	75.58 ± 15.82	91.21 ± 6.59	75.12 ± 7.94	62.77 ± 8.53	65.04 ± 4.13	83.57 ± 14.23	77.06 ± 14.62	81.60 ± 7.82

. The classification accuracies varied from subject to subject. Specifically, subjects 1 and 3 obtained higher accuracies, while subject 5 and 6 obtained lower accuracies. For the paired MI classification, the PWSPTP achieved better performance in the MI of the left hand vs. foot, left hand vs. tongue, and right hand vs. tongue (in this order).

The PWSPTP was compared with the CSP outcomes [6] to evaluate the effects of the spatiospectral filter optimization integrated in the trial pruning method. The experimental results in

Table 2. Performance comparison with CSP where L, R, F, and G denote left hand, right hand, foot, and tongue respectively, and μ and σ denote the mean and standard deviation, respectively. Bold values indicate the highest classification accuracy among the CSP and PWSPTP.

	L vs. R	L vs. F	L vs. G	R vs. F	R vs. G	F vs. G
CSP	73.46 ± 16.93	72.76 ± 17.93	74.07 ± 16.57	73.07 ± 15.82	72.38 ± 17.83	67.28 ± 11.40
PWSPTP	74.00 ± 12.96	82.90 ± 12.44	82.33 ± 12.49	76.22 ± 18.20	79.78 ± 14.25	73.61 ± 10.84

revealed that the proposed PWSPTP outperformed the CSP. The trial pruning improved the classification accuracy based on the removal of the contaminated trials during the optimization of the spatiospectral filter.

4. Discussion

4.1. Effect of Number of Particles on the Classification Accuracy

When the number of particles increased, the classification accuracy was not monotonically improved. As the number of particles was small, only few spectral and spatial patterns could be extracted by these particles (frequency bands) that may not possess sufficient discriminative power for the MI tasks. Conversely, as the number of particles was too large, some particles may be resampled from the same particle with high discriminative power. These particles may possess high overlapping portions of frequency bands compared with the other particles. Although these particles possess high discriminative power, these frequency bands may result in highly similar spectral and spatial patterns that do not provide diverse features for training the BCI. Furthermore, the BCI may overfit the training data that possess large dimensional features. Therefore, an appropriate number of particles could not only extract a sufficient number of spectral and spatial patterns, but prevent too many particles being almost identical.

4.2. Effect of Pruning Percentages on the Training Phase

Continuous increases of the ${\epsilon }_1$ and ${\epsilon }_2$ values may prune more trials that may not be contaminated trials. When too many trials are eliminated, the PWSPTP could not learn accurately the spatiospectral filters owing to the small amount of trials available for the training phase. Insufficient data amounts may lead to overfitting issues in the training of the classifier. These results suggested that approximately 3% and 7% of the trials were respectively contaminated by background noise/artifacts and improper MI for subject 2.

4.3. Asymmetrical Spatial Patterns on Left- versus Right-Hand MI Classification

The learnt spatial patterns of the left-hand versus right-hand MI classification addressed neurophysiologically plausible areas. The spatial patterns of the left- and right-hand MIs focus on the right and left motor cortices, respectively. These asymmetrical spatial patterns on left- versus right-hand MI classifications are consistent with the neurophysiological plausibility [5]. This suggested that the PWSPTP could automatically learn MI-related spatial patterns which match neurophysiological evidence. From the distribution of the discriminative frequency band, the PWSPTP could automatically learn the discriminative frequency bands related to the $\mu$-rhythm without predefining them so that the discriminative spectral features associated with MI could be effectively extracted for classification.

4.4. Effect of Trial Pruning on Paired Binary MI Classification Performance

It has been revealed that the paired binary MI classification of the left-hand vs. foot was better than the left hand vs. right hand among nine subjects. When the subject performed four MI tasks, the likelihood of performing improper left- /right-hand MI may be higher than that of left-hand/foot MI. Therefore, the left- /right-hand MI may result in more type-2 noisy trials than the left-hand/foot MI. Although PWSPTP could prune the contaminated trials, a few type-2 noisy trials may not be eliminated owing to the use of fixed pruning percentages. Thus, the classification accuracy of the left-hand/foot MI was higher than that of the left-hand/right-hand MI.

The CSP is a data-driven method that is sensitive to contaminated trials. The contaminated trials may increase the likelihood of the MI classifier learning spectral–spatial features irrelevant to the MI task, and could thus hamper the MI classification accuracy. The PWSPTP not only integrated trial pruning in the spatiospectral filter optimization, but shared the information of the discriminative power of spectral features to measure the contamination levels of the trials. Comparing with the CSP, the PWSPTP extracted the spatiospectral features without any noted effects on the outcomes from trials conducted in the presence of noise, and then improved the classification accuracy.

5. Conclusions and Future Work

The present study proposed a PWSPTP that constructed a filter bank without predefined frequency bands and without contaminated trials during the training phase. A Parzen window density estimator was applied to measure both the discriminative power of the spatiospectral features and contamination levels of the trials. A set of discriminative spectral filters was learnt by a particle-based approximation technique. Meanwhile, the use of ANOVA F-tests makes the computation of an importance weight of each spectral filter simpler. A set of potentially contaminated trials were removed during the training of classifiers without the effects of artifacts and improper motor imagery. Six paired binary MI classifications among nine subjects were conducted to evaluate the effectiveness of the learnt filter bank and trial pruning. The PWSPTP outperformed the CSP in terms of classification accuracy. The learnt spatial patterns and discriminative frequency bands were approximately consistent with the neurophysiological plausibility. The number of spectral filters and the number of percentages of pruned trials were hyperparameters and should be defined before running the PWSPTP. Future work will be devoted to the adjustment of the number of spectral filters and the number of percentages of pruned trials in the training phase.