Next Article in Journal
Automatic Indoor Reconstruction from Point Clouds in Multi-room Environments with Curved Walls
Previous Article in Journal
Point-Plane SLAM Using Supposed Planes for Indoor Environments
 
 
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A BCI Gaze Sensing Method Using Low Jitter Code Modulated VEP

Neurosensory Engineering Laboratory, Department of Biomedical Engineering, University of Miami, Coral Gables, FL 33146, USA
*
Author to whom correspondence should be addressed.
Sensors 2019, 19(17), 3797; https://doi.org/10.3390/s19173797
Received: 7 July 2019 / Revised: 28 August 2019 / Accepted: 29 August 2019 / Published: 2 September 2019
(This article belongs to the Section Biomedical Sensors)

Abstract

:
Visual evoked potentials (VEPs) are used in clinical applications in ophthalmology, neurology, and extensively in brain–computer interface (BCI) research. Many BCI implementations utilize steady-state VEP (SSVEP) and/or code modulated VEP (c-VEP) as inputs, in tandem with sophisticated methods to improve information transfer rates (ITR). There is a gap in knowledge regarding the adaptation dynamics and physiological generation mechanisms of the VEP response, and the relation of these factors with BCI performance. A simple, dual pattern display setup was used to evoke VEPs and to test signatures elicited by non-isochronic, non-singular, low jitter stimuli at the rates of 10, 32, 50, and 70 reversals per second (rps). Non-isochronic, low-jitter stimulation elicits quasi-steady-state VEPs (QSS-VEPs) that are utilized for the simultaneous generation of transient VEP and QSS-VEP. QSS-VEP is a special case of c-VEPs, and it is assumed that it shares similar generators of the SSVEPs. Eight subjects were recorded, and the performance of the overall system was analyzed using receiver operating characteristic (ROC) curves, accuracy plots, and ITRs. In summary, QSS-VEPs performed better than transient VEPs (TR-VEP). It was found that in general, 32 rps stimulation had the highest ROC area, accuracy, and ITRs. Moreover, QSS-VEPs were found to lead to higher accuracy by template matching compared to SSVEPs at 32 rps. To investigate the reasons behind this, adaptation dynamics of transient VEPs and QSS-VEPs at all four rates were analyzed and speculated.

1. Introduction

Recently brain–computer interface (BCI) research has shown remarkable progress as witnessed by an increasing number of publications. The application areas of BCIs cover neural-rehabilitation or neuroprosthesis, locomotion, control, communication, entertainment, and gaming [1,2,3,4]. BCI provides a direct communication and control channel between the human brain and output devices to achieve a desired output function by using the control signals derived from the human brain [5]. Most of the BCI research is focused on increasing the information transfer rate (ITR), essentially increasing the throughput of the bridge between brain and device. Depending on the signal acquisition methods, Electroencephalography (EEG) based BCIs are gaining increased attention due to non-invasiveness and practical EEG headsets available on the market [6]. Research in the field of wearable sensors for EEG measurement systems for BCIs is critical for the expansion of BCIs. In [6], researchers introduced two multichannel EEG headsets in which all electrodes easily reach the scalp, and no preparation is required. In EEG based BCI applications, there are four main signal classes, namely slow cortical potentials (SCP), sensorimotor rhythms (SMR), P300 evoked potentials and visual evoked potentials (VEP) [1,2,3,5,7]. VEP based BCIs paradigms are common due to several advantages. These are mainly high ITR, simple setup, and little if any user training [8].
VEPs are generally named after stimulation types, such as flash VEP, pattern reversal VEP, pattern-onset-offset VEP [9]. In clinical testing, pattern-reversal VEP is the most preferred one due to having the least variability in the waveform and latencies [10]. VEP stimulus can be modulated by space, time, luminance, contrast, color, pattern, and depth [10]. VEPs can be grouped into two, depending on the stimulation rate. At low stimulation rates (<3–4 reversals per second (rps)) electrical activity on the occipital scalp elicits abrupt changes by the contrast reversing stimulus and then settles until the presentation of the next stimulus. This type of VEP is called ‘transient VEP’ (TR-VEPs) [11,12]. When the visual stimulus is presented at a higher rate, a periodic response containing frequency components with constant amplitude and phase, which is called ‘steady-state VEP’ or SSVEP [13,14], is generated.
In VEP based BCIs, one question arises: independent or dependent? VEP is an exogenous signal generated by external visual stimulation, and it depends on the effector organs, such as the activity of the ocular muscles. For this reason, VEP BCIs are dependent BCIs. At the later stages of ALS disease, ocular muscle controls are lost and subjects loose gaze control. Thus, BCI must allow a mode of communication for these patients. A covert spatial attention mechanism was proposed for an independent VEP BCI [15]. Eye-tracking systems can be another option for the control signal. Eye trackers have been efficient for communication purpose so far. On the other hand, it is shown that EEG based (P300) 3 × 3 matrix BCI performed better than an eye-tracker and when it is combined with an eye-tracker, it achieved a higher performance than either [16]. Nezamfar et al. revealed that not only for the subjects with visual impairments but also for healthy individuals, VEP based gaze detection can be preferred instead of eye-tracking [17].
Depending on VEP modulation, three main VEP BCI types exist: time modulated VEP (t-VEP) BCI, frequency-modulated VEP (f-VEP) BCI, and pseudorandom code modulated VEP (c-VEP) BCI [18]. Both f-VEP and c-VEP BCIs produce high ITRs. Steady-state VEP (SSVEP) (f-VEP) based BCIs have the advantages of easy system configuration, excellent signal to noise ratio (SNR), immunity to interferences, minimal user training, easy quantification, and high ITR [5,18,19,20,21,22,23]. However, possible target frequencies are limited due to computer display refresh rate limitations, and the SSVEP has low amplitudes at high stimulation rates [5,19,23].
Pseudo random binary sequences (PRBS) based the c-VEP BCI paradigm offers promising results in terms of the ITR, accuracy, and use of many targets in applications [1,18,24]. Even though PRBS are mostly used in system identification, they are also used in physiological applications, such as deconvolution and BCI. One particular PRBS, the maximum length sequences or m-sequences were used by Sutter in a BCI application [25]. Although m-sequences have a-priori remarkable noise attenuation due to their sharp autocorrelation properties, they do not provide the expected SNR improvement in physiological recordings due to adaptation [26]. Working on BCI applications, Bin et al., found that when stimuli with sharp autocorrelation are used, the physiological system generates responses with lower autocorrelation sharpness, significantly reducing the a-priori effectivity of that BCI system [24]. Therefore, the knowledge of the adaptation characteristics of the visual system should be integrated into the design of the whole BCI system to achieve optimal performance.
Low-jitter PRBSs have been successfully used to unveil the unitary responses of the visual system through pattern electroretinograms (PERG) and VEPs [27,28]. Low jitter stimuli elicit c-VEPs with a narrow spectrum, similar to the SSVEP and are named as Quasi SSVEP (QSS-VEP); they have unique features, such as the capability for deconvolution of transient VEPs (TR-VEPs), immunity to adaptation, and shared brain generators of the SSVEP.
The use of a physiologically adapted low jitter PRBS stimuli in a BCI gaze sensor may capture advantages of both f-VEPs and c-VEPs paradigms, reducing their shortcomings. For this reason, in this study, we developed a dual-display BCI gaze sensor based on specially designed low-jitter PRBS codes eliciting QSS-VEP and adopted the template matching as the detection method.

2. Materials and Methods

A two target right and left gaze sensing systems was developed using two independent visual fields (right field RF and left field LF) on a fast Visual Display Unit (VDU). Four stimulation rates (10, 32, 50, and 70 rps) were explored in a normal subject population. For each rate, the detection was evaluated using QSS-VEP and TR-VEPs. For the rates of 10 and 32 rps, the SSVEPs were also evaluated. Fifty and 70 rps SSVEPs were excluded due to low amplitudes [29]. Using raw EEG recordings, the system performance, in all conditions, was quantified off-line by using receiver operating characteristic (ROC) curves, areas under the curves (AUC), accuracies, and ITRs.

2.1. Subjects

A total of 8 subjects (5M, 3F) with no history of neurological impairment participated in this study. The subject ages ranged from 18 to 31. All the subjects had either normal or corrected-to-normal vision and were right eye dominant. They signed informed consent forms approved by the University of Miami Institutional Review Board (IRB) before the participation.

2.2. Stimuli Design and Stimulator Display

The display consisted of two fast switchings on-demand LED-based regions for left (Left Field, LF) and right targets (Right Field, RF) (Figure 1). Subjects were placed 1 meter away from the stimulator display to ensure both targets lie in the foveal field of view. The stimulator provided two target stimuli, each covering a visual field of 5.7° × 5.7°. The luminance of the display was 580 Cd/m2. The stimulus pattern of the targets consisted of 12 horizontal black and white bars with a spatial frequency of 1.03 cycles/degree. The display was built with the same LED display technology that was described in [27] and [30]. The ON/OFF rising and falling times were less than 105 µs, and it was used in their experiment to obtain 78 rps responses [27]. Odd and even bars of the display were driven by the controllers of the digital output simultaneously to make the pattern reversal synchronous. Right and left fields were driven by two stimuli of slightly different length and jitter but at a similar rate (Table 1). The sequences were designed to be orthogonal to obtain individual responses to each display field. Low jittered, low noise amplification sequences similar to [27,28,31] were used. These sequences have low jitter and thus, resemble the steady-state iso-chronic stimulation sequences. Moreover, they are non-singular in the frequency domain and allow deconvolution to extract transient VEPs (See [32] for deconvolution). A cue light blue for Left, red for Right, green for Center was lit for an indication of gaze direction to the subjects.

2.3. Experimental Setup and EEG Recording

Although many electrodes can be used to obtain better accuracies by spatial filtering techniques, we aimed to keep it simple to see only the effects rate and stimuli dependent physiological phenomena. In [23], they found that all the subjects had the maximum signal at the Oz electrode location. From the previous recordings we have done so far in this laboratory, we obtained good results with the Oz–Fz configuration, and we continued on this bipolar derivation.
Three passive Ag/AgCl electrodes were used in the experiments for three electrode locations (Oz (+), Fz (-), and forehead (gnd)). Electrode impedances were checked and kept below 7 kΩ. EEG was recorded with an Intelligent Hearing System (IHS, Miami, FL, USA) data acquisition module (16 bits) at a sampling rate of 2 kHz. The analog signal was bandpass filtered with 1 to 300 Hz (6 dB/octave) filters. Raw EEG data along with stimulation triggers and synchronization cues, were stored for off-line analysis.
In this study, we collected between 4 and 6 trials for all the conditions. Some of the trials were used for template generation and the remaining for testing. We aimed to achieve a similar sweep number for the templates. Within the QSS comparison, we used 124 to 128 sweeps in template generation. For 10 rps SS/QSS comparison, we selected 117 to 127 sweeps; for 32 rps QSS/SS comparison, 93 to 96 sweeps were used. The acquisition order was 10 rps, 32 rps, 50 rps, and 70 rps, consecutively. Template acquisition was done by averaging EEG sweeps, for number of sweeps, refer to Table 1. During the recordings, the operator monitored the experiment to ensure signal quality, no powerline interference, subject’s cooperation, subject’s gaze. In the event of poor cooperation by subject, sleepiness or excessive eye blinks, the recording was halted and either repeated or postponed for another time. Between recordings, subjects had 30 s rest intervals.

2.4. EEG Offline Processing

Template VEPs were computed by conventional ensemble averaging of the epochs (sweeps) of the same condition; artifacts and noisy sweeps with peak to peak amplitudes >80 µV were rejected. The residual noise was estimated using the +/- averaging method as the average of the odd sweeps minus the even sweeps [33,34]. SNR was computed as the ratio of the signal power to noise power in dB.
For one cycle/trial of stimulation when the subject gazed at one stimulation region, the response to the other stimulation region is almost zero due to orthogonality which can be seen from y R L ( t ) (red) and y L R ( t ) (blue) plots in Figure 1.
Left and right display regions were driven by h L ( t ) and h R ( t ) stimulation sequences, respectively, as shown in Figure 1. The 10 rps QSS-VEPs for left and right regions were obtained by averaging the EEG data with window lengths of 1024 and 1016 points or 512 ms and 508 ms windows. For the other reversal rates, the window sizes or sweep durations are given in Table 1. The QSS-VEP responses are hypothesized to result from the convolution of the stimulation sequences h L ( t ) or h R ( t ) and ideal transient response at that particular adaptation rate. This can be seen from Equations (1) and (2).
y L L ( t ) = h L ( t ) r ( t ) Y L L ( f ) = H L ( f ) R ( f ) ,
y R R ( t ) = h R ( t ) r ( t ) Y R R ( f ) = H R ( f ) R ( f ) .
Using frequency domain deconvolution [26], the transient responses for the right field ( r R R ( t ) ) and left field stimulation and left gazes ( r L L ( t ) ) were obtained, as shown in Equations (3) and (4).
R R R ( f ) = Y R R ( f ) H R ( f ) r R R ( t ) = I F F T ( Y R R ( f ) H R ( f ) ) ,
R L L ( f ) = Y L L ( f ) H L ( f ) r L L ( t ) = I F F T ( Y L L ( f ) H L ( f ) ) .
The stimulation sequences were designed to have similar jitter characteristics; this led to very similar transient responses of left and right fields.
By averaging the multiple EEG data segments of left or right sequence lengths, it was possible to extract QSS-VEP responses to right and left targets (see Figure 1) for different data sizes. Since stimulation sequences were nonsingular, the deconvolution method in the frequency domain was applied to obtain TR-VEP responses. QSS-VEP and TR-VEP templates were computed by averaging the two EEG files for 10 rps and 50 rps, similarly three EEG files for 32 rps and 70 rps for each gaze direction.

2.5. Gaze Detection and ROC Estimation

The gaze detection process involved cross-correlation between the reference templates and the signal. We utilized Pearson’s product-moment correlation coefficient between the subject’s template VEP and the subject’s actual test data. The correlation coefficient was compared to a threshold for decision; if the correlation coefficient was higher than the threshold, the signal matched the template’s gaze direction. For this purpose, templates for each gaze and rate condition were extracted following the training sessions. Proper calculation and selection of threshold values from cross-correlation coefficients were critical in the gaze detection. The ROC is a curve representing the performance of the classifier; by the change of operation threshold value, the ROC determines the probability of error and accuracy (true positive ratio) in the system. After the files were recorded and the templates were ready to be used in the comparison, ROC curves were calculated to evaluate the performance of the classifier. To compute ROC curves, two classes of files (left and right gaze files) were fed into an algorithm where left or right templates were used as references. Since file gaze conditions were known, true positives, false positives, true negatives, false negatives were available while the threshold was varied from −1 to 1 by steps of 0.05 for the correlation coefficient. ROC curves were computed for 0.5 s, 1 s, 1.5 s, 2 s, 2.5 s, 3 s, 3.5 s, 4 s of data length.

3. Results

Overall system performance was evaluated by offline analysis. This covered ROC curves and areas, accuracies, ITR computations, plots, and comparisons. Generally, t-test paired two samples for means were used to examine the effects of various conditions on the performance, particularly on accuracy and ITR. The effect of gazing left or right stimulus was tested by computing accuracies. It was found that statistically there was no significant difference in the mean accuracies by left or right gaze using single sweep (0.5 s) QSS-VEP signature at all rates. Therefore, for the same conditions left and right gazed performances were averaged to generate a single parameter to represent the performance at that particular condition. This reduced the number of comparisons and hence, complexity in the analysis.
There were many signatures to test the system performance with. For 10 rps stimulation, these were mainly the combinations of right and left, QSS-VEP, and TR-VEP responses. It followed the same signatures for 32 rps, 50 rps, and 70 rps stimulation. By averaging left and right QSS-VEP performances in terms of accuracy or ROC area, an average accuracy or ROC area was computed for each rate of QSS (10, 32, 50, 70 rps) and steady-state (SS) conditions (10 and 32 rps).

3.1. VEP Morphology by Rate Adaptation

The recording electrode sites might reveal different signals depending on the adaptation characteristics of dominant neural populations over particular sites [35]. Four rates of 10 rps, 32 rps, 50 rps, and 70 rps QSS-VEPs are shown in Figure 2A. It can be observed that 70 rps QSS-VEP has smaller peaks compared to others. In Figure 2B, transient TR-VEP waveforms are shown. While 10 rps TR-VEP had characteristic early peaks and negativities, 32 rps, 50 rps, and 70 rps had totally different waveforms. There a positive peak replaced the negativity around 120 to 150 ms. When we compare the TR-VEP amplitudes, 32 rps had the largest amplitude 12 µV, 50 rps had 10.9 µV, 10 rps had 10.5 µV, and 70 rps had the minimum 5.2 µV amplitude.

3.2. Offline Analysis and Performance

Template matching was adopted as the method for target recognition. Threshold values for right and left target detection were pulled from ROC curves as the threshold index value corresponding to max ROC AUC. Once thresholds were set and template signals were uploaded, files with known status were tested with the system, and the results were plotted.
There are many metrics to evaluate the performance of a BCI system. Thompson et al., compared these metrics of accuracy or error rate, Cohen’s Kappa coefficient, confusion matrix, mutual information, and ITR in terms of throughput, categorical output, unbiased/biased, practicality [36]. They suggested ITR as the optimum metric for level-1 BCI system. BCIs consisting of only the control module, difficult to work at high speed, accuracy, and ease of use, which remains mainly used in research, medical, and clinical context, are referred to as level-1. Our system was tested offline with QSS-VEP and TR-VEP signatures and for all stimulation rates. ITR and accuracies were used in this evaluation. ITR can be calculated by the following Equation (5):
I T R = [ l o g 2 ( N ) + P   l o g 2 ( P ) + ( 1 P )   l o g 2 ( 1 P N 1 ) ] 60 T .
P is the probability of correct selection, N is the number of choices, and T (seconds/selection) time required to select a choice/target [36]. Its unit is given in bits per minute (bpm). The ITR simply follows the accuracy of the classification, the higher the accuracy, the bigger is the ITR. There is a limit of the ITR by selection time and maximum accuracy of 1. For a two target system, at maximum accuracy and for 0.5 s selection time, max ITR is120 bpm.
In the experiments subject 6 achieved a remarkable accuracy of 0.98 and a corresponding ITR of 100 bpm at 32 rps 0.5 s data and QSS-VEP right gazing BCI switch condition. QSS-VEP and TR-VEP ROC curves for 50 rps and left gaze conditions were compared. For data durations 0.5 s and 1.5 s, 50 rps mean ROC areas were found as 0.86 and 0.92, respectively, for the left QSS signal. Similarly, for left TR-VEP, 0.5 s and 1.5 s data durations, accuracies were 0.73 and 0.80, respectively. It should be noted that the ROC area increases by the increase in data size. Inclusion of extra sweeps in the average increases the SNR and better ROC area and accuracy are achieved.
When ROC curves for QSS-VEP and TR-VEPs were compared, for single sweep (0.5 s), QSS-VEPs were found to perform better than TR-VEPs for 32 rps stimulation (p < 0.05) (See Figure 3). Classifier performances for TR-VEPs using 0.5 to1 s sweep data were compared in Figure 3b. It can be observed from Figure 3 that among S3 ROC curves for different rates, 32 rps had the largest ROC area for both left QSS-VEP and left TR-VEP. Ten reversals per second and 50 rps had similar ROC areas for QSS-VEP, 70 rps had the minimum ROC area for both QSS-VEP and TR-VEP which summarizes the system performance at these rates. The QSS-VEP paradigm boosts the ROC area performances obtained by TR-VEPs by 12.4% for 32 rps for S3.
Table 2 presents the single sweep (0.5 s) QSS-VEP accuracy, ITR, mean testing signal SNR, and root mean square (RMS) values. Single sweep SNRs were computed using the subject individual VEP references for each condition as the signal estimate, and the noise was computed as the difference between single sweep and the reference. These noise and signal energies were averaged to get the mean SNR for 0.5 s data. It can be observed from Table 2 that when the SNR was high within the subject values, it corresponded to high accuracy and ITR. On the other hand, if one subject had higher SNR than the other one, accuracy and ITR were higher as well. The 32 rps QSS-VEP had larger average single sweep SNR (p < 0.05) and thus, higher accuracy and ITR. The single sweep average SSVEP SNRs were calculated similarly and found to be 0.31 dB for 10 rps, and 0.01 dB for 32 rps which were much smaller compared to QSS-VEP counterparts.
As mentioned previously, the left and right QSS-VEP performances were combined to create a single parameter for that condition, which revealed the rate-dependent characteristics easily in one figure. See Figure 4 for the rate effect on QSS-VEP and TR-VEP signals. It was found that for QSS-VEP 32 rps had the biggest accuracy and ITR among all rates for 0.5 s data. QSS-VEP ITRs were bigger than TR-VEP ITRs. On the other hand, TR-VEP ITR performances can be seen from the bottom right panel in Figure 4 which shows that, except for 10 rps, the remaining rates had ITR values around 20 to 23 bpm for 0.5 s data duration. Ten reversals per second had low TR-VEP accuracy performance and low ITR around 14 bpm with single sweep data. The lower accuracy of 10 rps can be attributed to the alpha band activity interference, which was reported in [29] and [37].
When 10 rps QSS and steady-state (SS) performances were compared, QSS-VEP 10 rps had a similar performance with 10 rps SS performance as seen in Figure 5. On the other hand, when 32 rps QSS was compared to SS, SS performed much lower than 32 rps QSS (p < 0.05) for all data durations. QSS-VEP 32 rps paradigm had on the average 23.5% higher accuracy than the SSVEP paradigm for 0.5 s data. This shows the improvement in high rate SSVEP accuracies by QSS paradigm.

4. Discussion

Although SSVEP based BCIs demonstrated their excellence in ITRs, performances of pattern-reversal c-VEP and transient-VEP based BCIs can be improved. Since there is a limited number of studies with pattern-reversal VEP, we proposed a research into different reversal rates and the template matching performance of the pattern reversal c-VEP or particularly low jittered QSS-VEPs. For TR-VEP ROC comparisons, 10 rps performed low compared to other rates. This might be attributed to the interference with the alpha band at 10 rps. There existed a large intra-subject variability in the accuracies and ITRs. It was found that user variation could be reduced by selecting subject-specific electrode location, stimulus frequency, and amount of EEG data used in the signature computation [8]. On the other hand, in addition to Oz, TP9 and TP10 electrode areas behind the ears also gave strong responses to the color and luminance stimuli [38]. The advantage of this method is that it provides more comfortable electrode montages due to no hair on these areas. It is very important for BCI systems to be mounted on the subject easily.
Another source of variation could be attention and cooperation during the particular tasks; if the user lacked attention during target gazing, the resultant target VEP characteristics would be affected. Hence, accuracy and ITRs would be influenced adversely.
For the fatigue problem, on the other hand, it is known that SSVEPs cause fatigue and epileptic seizures [39]. However, with high rate SSVEP stimulation (above 30 Hz), fatigue and seizures can be reduced [40]. In addition to the high rate, Xie et al., demonstrated that visual fatigue can be decreased by using random visual noise around the visual targets [41].
However, on the overall performance, 32 rps QSS-VEP paradigm led to the highest accuracy values and ITRs in the offline analysis compared to 10 rps, 50 rps, and 70 rps. One of the reasons behind this is that the number of visible peaks with 32 rps stimulation in the resultant QSS-VEP was higher than the number of visible peaks with other QSS stimulation rates. However at 50 rps and 70 rps, due to convolution and superposition of transient VEPs, the resulting QSS-VEPs did not elicit that many peaks contributing to template matching. Another aspect of the research was a comparison of SSVEP to QSS-VEP in template matching accuracy. As mentioned previously, SSVEPs had smaller peaks at high rates [29]. QSS-VEPs, on the other hand, had multiple peaks and higher amplitudes compared to SSVEPs.
QSS stimulation improved the SNR performance due to jitter in the paradigm, and this jitter canceled out adverse effects of adaptation on signal peaks. QSS-VEP generated larger peaks at higher rates where steady-state responses were diminished by adaptation. The low jitter design of QSS stimulation allowed the extraction of base/unitary TR-VEPs responses. If TR-VEPs had bigger amplitudes at particular rates, it was due to the boosting effect of special QSS stimulation. As compared QSS-VEPs at 32 rps performed significantly (p < 0.05) better than 32 rps SS counterparts for 1, 2, or 3 sweeps averages.
The performance of the gaze sensors is not only affected by the SNR but also by the saliency and the number of the peaks in the templates. Among all the investigated rates and stimulation characteristics, QSS-VEP at 32 rps performed better. This paradigm provides a good compromise between SNR and richness of the template. The rich 32 rps signature (Figure 2) was generated by the sharp characteristics of the transient response convolved with the stimulation sequence. To optimize the gaze sensor performance, both the underlying physiological knowledge and the sequence characteristics should be used as guidelines in the design.

5. Conclusions

This work aimed to present a better design of stimulation sequences and use of the QSS stimulation paradigm to demonstrate the importance of designing the stimulations sequences in line with the transient VEPs obtained at the rate of stimulation. Offline analysis was given for this purpose, and arguments were supported by these results. However, for the final evaluation of the proposed method, online accuracy, and ITR analysis can be carried out. It is projected to work on online testing of the system in future works.
This research revealed that QSS-VEP paradigm offers a feasible alternative to the methods applied to c-VEP or SSVEP based BCIs. The advantages of the QSS-VEP method are that it increases the accuracy of the SSVEP template matching at the same reversal rates, and it enables extraction of clinically significant TR-VEP from low jittered QSS-VEPs. By special design of low jitter QSS sequences, it was possible to enhance accuracy at higher rates compared to rates interfering with the alpha band. Thirty-two reversals per second achieved the maximum boosting effect by the QSS-VEP, which led to higher accuracy performances in template matching based dual-target BCI switch. The results can also be improved by using more electrodes and spatial filtering methods. However, the main issue addressed in this study is to use a simple technique to investigate the underlying physiologic phenomena and its relation to BCI performance. Another aspect to be considered is visual fatigue; the fatigue resulting from the QSS paradigm can be compared with the SSVEP fatigue at the same stimulation rate in future studies.

Author Contributions

Conceptualization, I.K., J.B., Ö.Ö.; methodology, I.K., J.B., Ö.Ö.; software, I.K.; validation, I.K.; formal analysis, I.K.; investigation, I.K.; resources, J.B.; data curation, I.K.; writing—original draft preparation, I.K.; writing—review and editing, I.K., J.B., Ö.Ö.; visualization, I.K.; supervision, J.B., Ö.Ö.; project administration, Ö.Ö.

Funding

This research is partially funded by Turkish Ministry of Education and Neurosensory Engineering Laboratory of University of Miami.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

ALSAmyotrophic Lateral SclerosisQSS-VEPQuasi-Steady-State VEP
AUCArea Under the CurveRFRight Field
BCIBrain Computer InterfaceRLRight seq. averaging of Left gaze
bpmBits per minuteRMSRoot mean square
c-VEPCode modulated VEPROCReceiver Operating Characteristics
EEGElectroencephalographyrpsreversal per second
FPRFalse Positive RateRRRight seq. averaging of Right gaze
f-VEPFrequency-modulated VEP r ( t ) R ( f ) Unitary transient response
h ( t ) H ( f ) Stimulus sequenceSCPSlow Cortical Potential
IFFTInverse Fast Fourier TransformSMRSensorimotor Rhythm
ITRInformation Transfer RateSNRSignal to Noise Ratio
LFLeft FieldSSVEPSteady-state VEP
LLLeft seq. averaging of Left gazeTR-VEPTransient Visual Evoked Potential
LRLeft seq. averaging of Right gazeTPRTrue Positive Rate
m-sequenceMaximum length sequencet-VEPTime modulated VEP
PERGPattern ElectroretinogramVEPVisual Evoked Potential
PRBSPseudo Random Binary Sequence y ( t ) Y ( f ) VEP response

References

  1. Nicolas-Alonso, L.F.; Gomez-Gil, J. Brain computer interfaces, a review. Sensors 2012, 12, 1211–1279. [Google Scholar] [CrossRef] [PubMed]
  2. Gant, K.; Guerra, S.; Zimmerman, L.; Parks, B.A.; Prins, N.W.; Prasad, A. EEG-controlled functional electrical stimulation for hand opening and closing in chronic complete cervical spinal cord injury. Biomed. Phys. Eng. Express 2018, 4, 065005. [Google Scholar] [CrossRef]
  3. Mora-Cortes, A.; Manyakov, N.; Chumerin, N.; Van Hulle, M. Language model applications to spelling with brain-computer interfaces. Sensors 2014, 14, 5967–5993. [Google Scholar] [CrossRef] [PubMed]
  4. Martišius, I.; Damaševičius, R. A prototype SSVEP based real time BCI gaming system. Comput. Intell. Neurosci. 2016, 2016, 18. [Google Scholar] [CrossRef] [PubMed]
  5. Wang, Y.; Gao, X.; Hong, B.; Jia, C.; Gao, S. Brain-computer interfaces based on visual evoked potentials. IEEE Eng. Med. Biol. Mag. 2008, 27, 64–71. [Google Scholar] [CrossRef] [PubMed]
  6. Kim, J.; Lee, J.; Han, C.; Park, K. An Instant Donning Multi-Channel EEG Headset (with Comb-Shaped Dry Electrodes) and BCI Applications. Sensors 2019, 19, 1537. [Google Scholar] [CrossRef] [PubMed]
  7. Padfield, N.; Zabalza, J.; Zhao, H.; Masero, V.; Ren, J. EEG-based brain-computer interfaces using motor-imagery: Techniques and challenges. Sensors 2019, 19, 1423. [Google Scholar] [CrossRef] [PubMed]
  8. Wang, Y.; Wang, R.; Gao, X.; Hong, B.; Gao, S. A practical VEP-based brain-computer interface. IEEE Trans. Neural Syst. Rehabil. Eng. 2006, 14, 234–240. [Google Scholar] [CrossRef] [PubMed]
  9. Odom, J.V.; Bach, M.; Brigell, M.; Holder, G.E.; McCulloch, D.L.; Mizota, A.; Tormene, A.P. ISCEV standard for clinical visual evoked potentials-(2016 update). Doc. Ophthalmol. 2016, 133, 1–9. [Google Scholar] [CrossRef]
  10. Fahle, M.; Bach, M. Origin of the visual evoked potentials. In Principles and Practice of Clinical Electrophysiology of Vision; Heckenlively, J.R., Arden, G.B., Eds.; The MIT Press: Cambridge, MA, USA, 2006; pp. 207–234. [Google Scholar]
  11. Sokol, S. Visually evoked potentials: Theory, techniques and clinical applications. Surv. Ophthalmol. 1976, 21, 18–44. [Google Scholar] [CrossRef]
  12. Zemon, V.M.; Gordon, J. Quantification and statistical analysis of the transient visual evoked potential to a contrast-reversing pattern: A frequency-domain approach. Eur. J. Neurosci. 2018, 48, 1765–1788. [Google Scholar] [CrossRef] [PubMed]
  13. Capilla, A.; Pazo-Alvarez, P.; Darriba, A.; Campo, P.; Gross, J. Steady-state visual evoked potentials can be explained by temporal superposition of transient event-related responses. PLoS ONE 2011, 6, e14543. [Google Scholar] [CrossRef] [PubMed]
  14. Regan, D. Steady-state evoked potentials. J. Opt. Soc. Am. 1977, 67, 1475–1489. [Google Scholar] [CrossRef] [PubMed]
  15. Kelly, S.P.; Lalor, E.C.; Finucane, C.; McDarby, G.; Reilly, R.B. Visual spatial attention control in an independent brain-computer interface. IEEE Trans. Biomed. Eng. 2005, 52, 1588–1596. [Google Scholar] [CrossRef] [PubMed]
  16. Choi, J.S.; Bang, J.; Park, K.; Whang, M. Enhanced perception of user intention by combining EEG and gaze-tracking for brain-computer interfaces (BCIs). Sensors 2013, 13, 3454–3472. [Google Scholar] [CrossRef] [PubMed]
  17. Nezamfar, H.; Mohseni Salehi, S.; Higger, M.; Erdogmus, D. Code-VEP vs. eye tracking: A comparison study. Brain Sci. 2018, 8, 130. [Google Scholar] [CrossRef]
  18. Bin, G.; Gao, X.; Wang, Y.; Hong, B.; Gao, S. VEP-based brain-computer interfaces: Time, frequency, and code modulations [Research Frontier]. IEEE Comput. Intell. Mag. 2009, 4, 22–26. [Google Scholar] [CrossRef]
  19. Chen, X.; Chen, Z.; Gao, S.; Gao, X. Brain–computer interface based on intermodulation frequency. J. Neural Eng. 2013, 10, 066009. [Google Scholar] [CrossRef]
  20. Cheng, M.; Gao, X.; Gao, S. Design and implementation of a brain–computer interface with high transfer rates. IEEE Trans. Biomed. Eng. 2002, 49, 1181–1186. [Google Scholar] [CrossRef]
  21. Gao, X.; Xu, D.; Cheng, M.; Gao, S. A BCI-based environmental controller for the motion-disabled. IEEE Trans. Neural Syst. Rehabil. Eng. 2003, 11, 137–140. [Google Scholar] [CrossRef]
  22. Li, Y.; Bin, G.; Gao, X.; Hong, B.; Gao, S. Analysis of phase coding SSVEP based on canonical correlation analysis (CCA). In Proceedings of the 5th International IEEE/EMBS Conference on Neural Engineering, Cancun, Mexico, 27 April–1 May 2011; pp. 368–371. [Google Scholar] [CrossRef]
  23. Jia, C.; Gao, X.; Hong, B.; Gao, S. Frequency and phase mixed coding in SSVEP-based brain–computer interface. IEEE Trans. Biomed. Eng. 2011, 58, 200–206. [Google Scholar] [CrossRef] [PubMed]
  24. Bin, G.; Gao, X.; Wang, Y.; Li, Y.; Hong, B.; Gao, S. A high-speed BCI based on code modulation VEP. J. Neural Eng. 2011, 8, 025015. [Google Scholar] [CrossRef] [PubMed]
  25. Sutter, E.E. The brain response interface: Communication through visually-induced electrical brain responses. J. Microcomput. Appl. 1992, 15, 31–45. [Google Scholar] [CrossRef]
  26. Bohórquez, J.; Özdamar, Ö. Signal to noise ratio analysis of maximum length sequence deconvolution of overlapping evoked potentials. JASA 2006, 119, 2881–2888. [Google Scholar] [CrossRef] [PubMed]
  27. Toft-Nielsen, J.; Bohórquez, J.; Özdamar, Ö. Unwrapping of transient responses from high rate overlapping pattern electroretinograms by deconvolution. Clin. Neurophysiol. 2014, 125, 2079–2089. [Google Scholar] [CrossRef]
  28. Bohórquez, J.; Lozano, S.; Kao, A.; Toft-Nielsen, J.; Özdamar, Ö. Deconvolution and modeling of overlapping visual evoked potentials. In Proceedings of the 29th IEEE Southern Biomedical Engineering Conference, Miami, FL, USA, 3–5 May 2013; pp. 31–32. [Google Scholar] [CrossRef]
  29. Diez, P.F.; Müller, S.M.T.; Mut, V.A.; Laciar, E.; Avila, E.; Bastos-Filho, T.F.; Sarcinelli-Filho, M. Commanding a robotic wheelchair with a high-frequency steady-state visual evoked potential based brain–computer interface. Med. Eng. Phys. 2013, 35, 1155–1164. [Google Scholar] [CrossRef]
  30. Toft-Nielsen, J.; Bohorquez, J.; Ozdamar, O. Innovative pattern reversal displays for visual electrophysiological studies. In Proceedings of the 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Boston, MA, USA, 30 August–3 September 2019. [Google Scholar]
  31. Kaya, I.; Bohorquez, J.E.; Özdamar, Ö. Brain Computer Interface Switch Based on Quasi-Steady-State Visual Evoked Potentials. In Proceedings of the IEEE NER 9th International IEEE/EMBS Conference on Neural Engineering, San Francisco, CA, USA, 20–23 March 2019; pp. 1175–1178. [Google Scholar] [CrossRef]
  32. Özdamar, Ö.; Bohórquez, J. Signal-to-noise ratio and frequency analysis of continuous loop averaging deconvolution (CLAD) of overlapping evoked potentials. JASA 2006, 119, 429–438. [Google Scholar] [CrossRef]
  33. Turetsky, B.I.; Raz, J.; Fein, G. Noise and signal power and their effects on evoked potential estimation. Electroencephalogr. Clin. Neurophysiol. Evoked Potentials Sect. 1988, 71, 310–318. [Google Scholar] [CrossRef]
  34. Özdamar, Ö.; Delgado, R.E. Measurement of signal and noise characteristics in ongoing auditory brainstem response averaging. Ann. Biomed. Eng. 1996, 24, 702–715. [Google Scholar] [CrossRef]
  35. Heinrich, S.P.; Bach, M. Adaptation dynamics in pattern-reversal visual evoked potentials. Doc. Ophthalmol. 2001, 102, 141–156. [Google Scholar] [CrossRef]
  36. Thompson, D.E.; Stefanie, B.M.; Jane, E.H. Performance assessment in brain-computer interface-based augmentative and alternative communication. Biomed. Eng. Online 2013, 12, 43. [Google Scholar] [CrossRef] [PubMed]
  37. Vialatte, F.B.; Maurice, M.; Dauwels, J.; Cichocki, A. Steady-state visually evoked potentials: Focus on essential paradigms and future perspectives. Prog. Neurobiol. 2010, 90, 418–438. [Google Scholar] [CrossRef] [PubMed]
  38. Floriano, A.; Diez, P.F.; Freire Bastos-Filho, T. Evaluating the influence of chromatic and luminance stimuli on SSVEPs from behind-the-ears and occipital areas. Sensors 2018, 18, 615. [Google Scholar] [CrossRef] [PubMed]
  39. Cao, T.; Wan, F.; Wong, C.M.; da Cruz, J.N.; Hu, Y. Objective evaluation of fatigue by EEG spectral analysis in steady-state visual evoked potential-based brain-computer interfaces. Biomed. Eng. Online 2014, 13, 28. [Google Scholar] [CrossRef] [PubMed]
  40. Volosyak, I.; Valbuena, D.; Luth, T.; Malechka, T.; Graser, A. BCI demographics II: How many (and what kinds of) people can use a high-frequency SSVEP BCI? IEEE Trans. Neural Syst. Rehabil. Eng. 2011, 19, 232–239. [Google Scholar] [CrossRef] [PubMed]
  41. Xie, J.; Xu, G.; Luo, A.; Li, M.; Zhang, S.; Han, C.; Yan, W. The role of visual noise in influencing mental load and fatigue in a steady-state motion visual evoked potential-based brain-computer interface. Sensors 2017, 17, 1873. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Overall system setup and the experimental procedure are shown for a 10 rps stimulation. The stimulator is shown at the top left, driving sequences for the left field (LF) and right field (RF) are shown at the bottom left, the quasi-steady-state (QSS) and transient responses from subject 5 are shown in the bottom right figures. Interference signals after averaging 127/128 sweeps in one file for left/right, y R L ( t ) and y L R ( t ) , are also displayed to highlight the orthogonality of the sequences. Refer to Section 2.4 and abbreviation list at the end of the paper for the abbreviations used in the figure.
Figure 1. Overall system setup and the experimental procedure are shown for a 10 rps stimulation. The stimulator is shown at the top left, driving sequences for the left field (LF) and right field (RF) are shown at the bottom left, the quasi-steady-state (QSS) and transient responses from subject 5 are shown in the bottom right figures. Interference signals after averaging 127/128 sweeps in one file for left/right, y R L ( t ) and y L R ( t ) , are also displayed to highlight the orthogonality of the sequences. Refer to Section 2.4 and abbreviation list at the end of the paper for the abbreviations used in the figure.
Sensors 19 03797 g001
Figure 2. Population average left field quasi-steady-state-visual evoked potentials (QSS-VEP) (A), and transient-visual evoked potentials (TR-VEP) (B) plots are shown. (A) 10 rps, 32 rps, 50 rps, and 70 rps QSS-VEPs are at the left. (B) TR-VEPs are at the right. Note the morphology change in the TR-VEPs by rate from 10 rps to 32 rps.
Figure 2. Population average left field quasi-steady-state-visual evoked potentials (QSS-VEP) (A), and transient-visual evoked potentials (TR-VEP) (B) plots are shown. (A) 10 rps, 32 rps, 50 rps, and 70 rps QSS-VEPs are at the left. (B) TR-VEPs are at the right. Note the morphology change in the TR-VEPs by rate from 10 rps to 32 rps.
Sensors 19 03797 g002
Figure 3. Subject 3 (S3) single sweep 0.5-second condition. (a) Left QSS-VEP receiver operating characteristic (ROC) curves and areas for all the rates are shown at the left. (b) Left TR-VEP ROC curves and areas for all the four rates are shown at the right.
Figure 3. Subject 3 (S3) single sweep 0.5-second condition. (a) Left QSS-VEP receiver operating characteristic (ROC) curves and areas for all the rates are shown at the left. (b) Left TR-VEP ROC curves and areas for all the four rates are shown at the right.
Sensors 19 03797 g003
Figure 4. QSS-VEP performances for left and right gazes were averaged as single QSS-VEP performance parameter at each rate. For single sweep 0.5 s, 1 s, and 1.5 s, 32 rps had the highest accuracy and ITR performance (p < 0.05) with QSS-VEP signature while 70 rps had the lowest performance (p < 0.05). For the eight subject population, average ITR values for QSS-VEP (top-right) and TR-VEPs (bottom-right) were plotted for each rate and data durations.
Figure 4. QSS-VEP performances for left and right gazes were averaged as single QSS-VEP performance parameter at each rate. For single sweep 0.5 s, 1 s, and 1.5 s, 32 rps had the highest accuracy and ITR performance (p < 0.05) with QSS-VEP signature while 70 rps had the lowest performance (p < 0.05). For the eight subject population, average ITR values for QSS-VEP (top-right) and TR-VEPs (bottom-right) were plotted for each rate and data durations.
Sensors 19 03797 g004
Figure 5. QSS-VEP and steady-state visual evoked potentials (SSVEP) performances are compared. QSS-VEPs performed better than SSVEPs for short data durations.
Figure 5. QSS-VEP and steady-state visual evoked potentials (SSVEP) performances are compared. QSS-VEPs performed better than SSVEPs for short data durations.
Sensors 19 03797 g005
Table 1. Stimulation properties for quasi-steady-state (QSS) and steady-state (SS) stimulation paradigms and sweep number used for template generation.
Table 1. Stimulation properties for quasi-steady-state (QSS) and steady-state (SS) stimulation paradigms and sweep number used for template generation.
SignalNominal Rate (rps)Mean Rate (rps)Field
Left-LF
Right-RF
Sweep Duration (ms)Trial Sweep CountTemplate Acquisition Sweep Number
QSS109.76LF512127127
9.84RF508128128
QSS3231.25LF51231124
30.25RF49632128
QSS5049.6LF50462124
52.4RF49663126
QSS7070.5LF49632128
70.3RF51231124
QSS109.76LF512127127
9.84RF508128128
SS1010LF50039117
10.5RF487.540120
QSS3231.25LF5123193
30.25RF4963296
SS3231LF5123193
32RF4963296
Table 2. Single sweep (0.5 s) quasi-steady-state-visual evoked potentials (QSS-VEP) accuracy, information transfer rates (ITR), mean signal to noise ratio (SNR), and root mean square (RMS) overall average values for all rates and eight subjects (S1–S8).
Table 2. Single sweep (0.5 s) quasi-steady-state-visual evoked potentials (QSS-VEP) accuracy, information transfer rates (ITR), mean signal to noise ratio (SNR), and root mean square (RMS) overall average values for all rates and eight subjects (S1–S8).
SubjectAccuracy
10 rps32 rps50 rps70 rps
LRLRLRLR
S10.820.770.840.790.890.780.890.73
S20.900.920.920.930.850.830.890.88
S30.730.710.820.780.730.660.580.67
S40.830.780.840.790.780.840.700.74
S50.750.710.790.870.760.790.710.67
S60.880.940.950.980.820.870.720.86
S70.700.610.720.720.650.710.610.60
S80.620.710.830.860.840.860.840.79
Mean
(STD)
0.77
(0.10)
0.84
(0.07)
0.79
(0.07)
0.74
(0.10)
SubjectITR (bpm)
10 rps32 rps50 rps70 rps
LRLRLRLR
S138.3927.2744.1730.1158.5828.7860.3718.68
S264.1072.5970.4875.6446.5241.0861.1055.45
S318.8515.9138.6529.2219.548.912.4010.09
S441.3529.6644.7531.9529.4443.3114.5420.79
S522.0816.3831.0254.4424.6031.2515.759.84
S657.5279.7685.1299.7637.6153.4417.8448.65
S714.104.3017.6717.358.0215.453.923.35
S85.0314.9941.6349.5842.7448.9542.4631.48
Mean26.6443.8831.0220.79
SubjectSNR (dB)
10 rps32 rps50 rps70 rps
LRLRLRLR
S10.770.670.420.080.920.740.580.10
S21.080.971.561.420.660.770.950.70
S30.290.110.190.160.230.06−0.110.07
S40.570.440.660.350.450.230.120.15
S50.240.220.350.290.300.240.13−0.02
S60.550.640.821.050.240.310.090.28
S70.13−0.160.090.100.04−0.13−0.04−0.13
S8−0.01−0.240.280.430.310.760.230.16
Mean0.390.520.380.20
SubjectAverage QSS-VEP RMS (µV)
10 rps32 rps50 rps70 rps
LRLRLRLR
S14.984.484.103.264.754.384.302.89
S23.853.824.454.602.862.843.363.38
S33.253.614.073.443.343.051.952.85
S42.262.122.281.952.172.201.201.38
S52.512.562.713.022.222.632.091.61
S64.674.904.805.153.193.172.372.77
S72.222.202.332.231.771.651.531.41
S81.811.912.683.043.083.252.022.10

Share and Cite

MDPI and ACS Style

Kaya, I.; Bohórquez, J.; Özdamar, Ö. A BCI Gaze Sensing Method Using Low Jitter Code Modulated VEP. Sensors 2019, 19, 3797. https://doi.org/10.3390/s19173797

AMA Style

Kaya I, Bohórquez J, Özdamar Ö. A BCI Gaze Sensing Method Using Low Jitter Code Modulated VEP. Sensors. 2019; 19(17):3797. https://doi.org/10.3390/s19173797

Chicago/Turabian Style

Kaya, Ibrahim, Jorge Bohórquez, and Özcan Özdamar. 2019. "A BCI Gaze Sensing Method Using Low Jitter Code Modulated VEP" Sensors 19, no. 17: 3797. https://doi.org/10.3390/s19173797

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop