Simultaneous Decoding of Eccentricity and Direction 2 Information for a Single-Flicker SSVEP BCI 3

: The feasibility of a steady-state visual evoked potential (SSVEP) brain - computer interface 15 (BCI) with a single flicker stimulus for multiple-target decoding has been demonstrated in a number


Introduction
Steady-state visual evoked potential (SSVEP), as one of the most widely used responses in electroencephalogram (EEG) -based brain-computer interfaces (BCIs), has received sustained attention [1][2][3][4][5][6][7].When participants attend a periodic visual stimulus, SSVEPs are elicited at the stimulation frequency and its harmonics [8].Correspondingly, by encoding different targets with distinct frequencies, BCI systems can be realized via real-time frequency recognition of the recorded SSVEPs [3,9].To date, the frequency-coding SSVEP BCIs have achieved significant progress, featured by the relatively large number of simultaneously decodable targets and the high communication speed [5,6], hereby promising for real-life applications such as letter typing.
When flicker stimuli were presented at different spatial locations in the visual field, distinct SSVEP responses would be elicited [10].The phenomenon, known as the retinotopic mapping [11,12], has gained increasing interest in recent BCI studies.Based on the retinotopic mapping of SSVEP, while pilot BCI studies have mainly focused on designing visual spatial patterns to increase possible BCI target numbers [13] or enhance the signal-to-noise ratio (SNR) of SSVEP [14] Peer-reviewed version available at Electronics 2019, 8, 1554; doi:10.3390/electronics8121554[15,16].Unlike the traditional frequency-coded SSVEP BCI paradigm in which SSVEP responses were modulated by targets with different frequencies [3,9], it is feasible to design a spatially-coded SSVEP BCI by encoding responses by targets with different spatial locations.Indeed, previous studies have demonstrated that overtly attending to targets at distinct spatial directions relative to a centrallydisplayed flicker stimulus could evoke separable SSVEP responses [15,16].Moreover, the differences in responses are sufficient to support the decoding of directions at a single-trial level to achieve a dial [15] and spatial navigation task [16], suggesting the feasibility of a single-stimulus, multi-targets SSVEP BCI.Compared with the frequency-coded BCIs in which multiple stimuli are required to encode multiple targets, this single-stimulus design can considerably simplify the stimulation setup and the user interface of BCIs [17,18].In addition, given the fact that the stimulus always appears in the peripheral visual field, this single-flicker SSVEP BCI paradigm is expected to reduce the visual burden at the same time [16], indicating its potential to be a good candidate for practical applications.
However, the previous spatially-coded SSVEP studies only utilized spatial directions to encode targets, the resulting nine-or four-command designs have limited the potential applications of spatially-coded BCIs when compared with the conventional frequency-coded SSVEP BCIs.For example, in a drone control task, while previous designs are only sufficient to control the moving directions, it is possible to send more commands such as speeding, stopping, climbing etc., if more command channels could be achieved.One way to extend the feasible application scenarios is to include the visual eccentricity information for increasing the number of targets.Indeed, SSVEP responses have been observed to reduce along with the increase of the eccentricity of stimuli from the fixation spot [19], providing neurophysiological evidence in support of the eccentricity decoding in SSVEP responses.Joint decoding of eccentricity and direction information is expected to substantially increase the number of targets, by making a better use of the visual spatial information.
Nvertheless, the eccentricity information could contribute to extending the encoding dimension only when the spatial patterns remain separable even with a large eccentricity.Specifically, the weaker SSVEP responses along with increasing eccentricities may lead to a reduced accuracy for the direction classification at the same time, thus influencing the BCI performance in a complex way.Although there are previous studies suggesting a relatively stable spatial patterns of visual motion-onset responses with increasing eccentricities [17,18], efforts are still needed to evaluate how visual eccentricity information modulates the SSVEP responses and whether this modulation could contribute to decoding visual spatial information at a single-trial level.
In the present study, the feasibility of a spatially-coded BCI to encode targets with both the eccentricity and direction information simultaneously was evaluated.Eight directions (left, left-up, up, right-up, right, right-down, down, and left-down) and two eccentricities (2.5° and 5°) relative to one flicker stimulus were employed to encode 16 targets.During the experiment, participants were instructed to direct their overt attention to one of the targets with EEG recorded.Then, SSVEP responses modulated by different visual directions and eccentricities were analyzed, and the 16target classification performances were evaluated in an offline manner.Our results suggest the feasibility of the simultaneous decoding of visual eccentricity and direction information based on SSVEP.

Participants
Twelve participants (five females, aged from 23 to 28 years, mean 24.8 years) with normal or corrected-to-normal vision participated in the experiment.All participants were given informed consent before experiments and received financial compensation for their participation.The study was approved by the local Ethics Committee at the Department of Psychology, Tsinghua University.

Visual Stimulation
The visual stimulation in the experiment is illustrated at the top panel of Figure.50 cm) was used to present the stimulation.A white disk (radius = 2.5°) was centrally displayed on the screen (indicated as the gray disk in the top panel of Fig. 1).During the experiment, the disk flickered at 12 Hz with a sampled sinusoidal stimulation method [20], forming a flicker stimulus to elicit SSVEPs.
The stimulus lasted 4000 ms in total.One small red square (0.25°×0.25°) would appear on the screen to indicate where the participants should direct their overt attention during the experiment.There are 16 possible targets arranged surrounding the central circle at eight directions (left, left-up, up, right-up, right, right-down, down, and left-down) and two eccentricities (2.5° and 5°).Since a previous study observed a rapid drop of SSVEP responses when the stimulus presented beyond 5° away from the central fixation spot [19], 2.5° and 5° were chosen conservatively to evaluate the feasibility of eccentricity decoding in the present study.Eccentricities larger than 5° will be explored in further studies.

Experimental Procedure
The experiment included ten blocks in total.The duration of the inter-block intervals was controlled by participants themselves with a lower limit of 30 seconds set in the experimental program.In each block, 16 trials corresponding to each attention target were presented with a random order.As demonstrated at the bottom panel of Figure .1, for each trial, one red square was displayed to cue the to-be-attended target for 1000 ms at the beginning, then following by a 4000-ms flicker stimulus.The red square existed for the whole flickering duration for the participants to attend.The inter-trial interval varied from 1000 to 1500 ms, during which participants could blink or swallow.
The Psychophysics Toolbox [20,21] based on MATLAB (The Mathworks, Natick, MA, USA) was employed to present the stimulation.

EEG Recordings
EEG was recorded continuously at a sampling rate of 1000 Hz with a SynAmps2 amplifier (Compumedics NeuroScan, USA).Sixty-four electrodes were recorded according to the international 10-20 system with a reference at the vertex and a forehead ground at AFz. Electrode impedances were kept below 10 kΩ during the experiment.The experiment was carried out in an electromagnetically shielded room.

Data preprocessing
Continuous EEG data were first band-pass filtered to 1.5-80 Hz, and a 50-Hz notch filter was used to remove the line noise.Next, EEG data were segmented into 4000-ms trials after the onset of the stimulus, resulting in 10 trials for each of the 16 attentional targets.Then, a set of 9 electrodes covering the parietal-occipital (PO5/6/7/8, O1/2, Pz, POz, and Oz), where the SSVEPs typically show maximal responses, was chosen for further analysis.

SNR evaluation
In order to describe the SSVEP response strength when attending to targets at different directions and eccentricities in a quantitively way, a newly-proposed method [22], which could evaluate the SSVEP SNR of the multi-channel EEG data response while considering multiple harmonics, was employed in this present study.Here, the stimulus frequency, as well as its second and third harmonics, were included in SNR calculation and the following-up BCI classification.
First, for each subject, the segmented EEG data were averaged for each attentional target.Then, the SSVEP signal was defined as the projection of the averaged EEG data in the subspace of the stimulus frequency and its harmonics, while noise was defined as the residual after the projection.SNR, which was defined as the ratio between signal and noise, was calculated as the index of the responses for each attentional target with the formula (1).Details about the mathematical derivation could be found in [22].
Here, T is the 9-channel averaged EEG data, and ϕ is the reference signal.
Finally, a two-way repeated measure analysis of variance (RMANOVA) with two within-subject factors, i.e., direction (left, left-up, up, right-up, right, right-down, down and left-down) and eccentricity (2.5° and 5°), was conducted to determine their possible effects on the SNR of SSVEP statistically.P values smaller than 0.05 were considered statistically significant after Greenhouse-Geisser correction.Statistical analyses were performed with SPSS (22.0.0,IBM, Armonk, NewYork, USA).

BCI Classification
In the offline performance evaluation, the single-trial 4000-ms EEG data were used for BCI classification without any artifact rejection.A canonical correlation analysis (CCA) based classification algorithm [23] was employed to capture the distinct SSVEP patterns, as reported in [15,16].Note that all the offline classifications were evaluated with a 10-fold cross-validation procedure.
First of all, in order to evaluate how directions and eccentricities contribute to the classification performances, 8-directions classification at each eccentricity and the 2-eccentricity classification in each direction were conducted.
In the training phase, K-trial EEG data when the participant was attending to the target location c were concatenated as X c .Then, the reference signal Y was obtained by replicating the ϕ (see formula (2)) K times: Here, N is the target number.For 8-directions classification, N = 8, and for the 2-eccentricity classification, N = 2.The M is the number of canonical correlation coefficients and is set as 6, the same as reported in [15][16].
Then, for each trial in the training set, a 1×(N*M) feature vector was composed by calculating the canonical correlations   for all N targets and concatenating them as [r 1 r 2 … r N ], which was used to train a support vector machine (SVM) classifier using the LIBSVM toolbox [24].
In the testing phase, the EEG trial to be classified is filtered with Wx c , and the correlation coefficients   with the corresponding reference signals Wy c ϕ are computed, (c = 1, 2, …, N).The concatenated correlation coefficients [r 1 r 2 … r N ] constituted the feature vector for the testing trial, which then was used to recognize the target by the classifier.
After decoding the directions and eccentricities separately, a 16-target classification which decoded the visual eccentricity and direction information simultaneously was conducted with the above-mentioned CCA method.Here, N = 16.
Finally, in order to evaluate how the visual eccentricity information influences the joint classification of directions and eccentricities, three conditions: individual filter, 2.5° filter, and 5° filter were compared.The individual filter means the spatial filters Wx c and Wy c (c = 1, 2, …, 16) were trained with data from their respective eccentricities, corresponding to the results in Table I.The 2.5° filter, however, indicates the classification accuracies were calculated all by using spatial filters trained with data with an eccentricity of 2.5°, even for those with an eccentricity of 5°.The 5° filter could be explained similarly.

Results
As illustrated in Figure .2, a typical SSVEP response over occipital and parietal areas could be found across conditions.When attending to targets at different directions and eccentricities, distinct SNR topographies for SSVEP were elicited with a shift of the response over the parietal-occipital areas.Specifically, when participants attended to the target at the right side, the flicker stimulus appeared in their left visual field, leading to a right-dominant response, and the opposite relation held for the target at the left side, suggesting a contralateral response.In addition, the SSVEP spatial patterns remained similar, along with the increasing eccentricities of the flicker stimulus.The accuracies were 75.5±14.9%,and 59.4 ±15.0% at 2.5° and 5°, respectively.As observed, the classification accuracy is reduced for the targets with the larger eccentricity.As reflected in Figure 5, the 2-eccentricity classification achieved an accuracy of 89.6±15.0%,91.7 ±10.7%, 89.6±13.3%,84.2 ±11.0%, 91.7±13.0%,93.8±9.38%,87.9±16.0%,and 90.4 ±9.00% for left, left-up, up, right-up, right, right-down, down and left-down, respectively.There was no significant difference between any pair of directions after a paired t-test with Bonferroni correction.
The results so far demonstrated the feasibility of decoding directions and eccentricities separately.Then, the 16-target classification results, which decoded directions and eccentricities at the same time, are summarized in Table I.When using a 4-s data, the mean accuracy across participants is 66.8 ±16.4%, well above chance level for the 16-target classification problem (i.e., 6.25%).Note that an individual difference could be found in classification accuracies, ranging from 38.8% to 90.0%.±15.7% for using data with a length at 2s, 3s, and 4s, respectively.Although a decreasing trend could be observed, the 2-s data still provided accuracies well above chance level.This present study also took a closer look at how the visual eccentricity information contributed to this spatially-coding paradigm.First of all, when attending to targets at increased eccentricities, the corresponding reduced SNRs and decreased 8-direction classification accuracies suggested a weaker response along with the larger eccentricity.Furthermore, this decrease of SSVEP responses could be a contributing feature for the eccentricity decoding, supported by the 2-eccentricity classification accuracies ranged from 84.2% to 93.8% in 8 directions.Then, the 8-direction classification accuracies at 5° were found to achieve an accuracy of 59.4 ±15.0%, much higher than chance level.More importantly, compared with those classifications using spatial filters trained from their corresponding eccentricities, the 16-target classification accuracies, though significantly decreased, still remained comparable when using spatial filters from data with an eccentricity of 5°.
, efforts have been devoted to decoding the spatial information embedded in SSVEP responses directly recent years Preprints (www.preprints.org)| NOT PEER-REVIEWED | Posted: 15 October 2019 © 2019 by the author(s).Distributed under a Creative Commons CC BY license.

PreprintsFigure 2 .
Figure 2. The topographies of the SNR of SSVEP from a representative participant (sub 2).The inner circle represents the eccentricity of 2.5°, while the outer circle represents the eccentricity of 5°.All SNRs were normalized into z-values so that the positive and negative values indicate SNRs above and below the mean level across electrodes, respectively, in z units.

Figure 5 .
Figure 5.The boxplot of the 2-eccentricity classification accuracy at each of the eight directions.The black dashed line indicated the chance level of classification.

Figure 6 .
Figure 6.The boxplot of accuracies influenced by spatial filters.The individual filter means the spatial filters were trained with data from their respective eccentricities.The 2.5° filter label indicates the classification accuracies were calculated all by using spatial filters trained with data with an eccentricity of 2.5°.The 5° filter label could be explained in a similar way.The black dashed line indicated the chance level of classification.

Figure 7 .
Figure 7. Confusion matrix for 16-target classifications.L, LU, U, RU, R, RD, D, LD are short for left, left-up, up, right-up, right, right-down, down, and left-down.The rows show true labels and the columns show predicted labels.The 2.5° filter label means the confusion matrix was calculated by all using spatial filters trained with data at an eccentricity of 2.5°.The 5° filter label could be explained in a similar way.The individual filter means the spatial filters were trained with data from their respective eccentricities.

PreprintsFigure 8 .
Figure 8. Classification accuracies as the function of data length.Error bars indicate standard error.The black dashed line indicated the chance level of classification.
These classification accuracies provide evidence in support of the weaker yet stable spatial patterns across eccentricities.Taken all, our results suggested that the decreased SSVEP responses and relatively stable spatial patterns provided the neural basis of the joint decoding of visual eccentricity and direction information, supporting the feasibility of the visual eccentricity information as an encoding dimension in spatially-coded BCIs.Besides, it should also be noted that the feasibility of transferring spatial filters across eccentricities indicated the potentials to reduce training time, if it is Preprints (www.preprints.org)| NOT PEER-REVIEWED | Posted: 15 October 2019 Peer-reviewed version available at Electronics 2019, 8, 1554; doi:10.3390/electronics8121554

) Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 15 October 2019
Peer-reviewed version available at Electronics 2019, 8, 1554; doi:10.3390/electronics8121554HereK is 9 for each target as 90% of the EEG data were used as the training set.CCA was employed to find spatial filters Wx c and Wy c (c = 1, 2, …, N) to maximize the canonical correlation   = [ 1 …   ] between X and reference signal Y:

Table 1 .
The summary of 16-target classification accuracy when using 4-s SSVEP data.Then, accuracies obtained by using different spatial filters were shown in Figure.6.The accuracies were 66.8 ±15.7%, 62.3±15.5%, and 61.0 ±16.4% for individual filter, 2.5° filter, and 5° filter respectively.A decreasing trend can be observed in three conditions.A paired t-test was used to conduct a comparison with Bonferroni correction.No significant difference between accuracies was obtained in the individual filter condition and the 2.5° filter condition (t(11)= 2.20, p = 0.956).Furthermore, although accuracies obtained in both two conditions were higher than those from the 5° filter condition significantly (individual filter > 5° filter, t(11)= 2.20, p <0.001); 2.5° filter> 5° filter, t(11)= 2.20, p <0.001), it should be noted that the absolute numbers of the accuracies are comparable.