The Effect of Jittered Stimulus Onset Interval on Electrophysiological Markers of Attention in a Brain–Computer Interface Rapid Serial Visual Presentation Paradigm

Abstract

Brain responses to discrete stimuli are modulated when multiple stimuli are presented in sequence. These alterations are especially pronounced when the time course of an evoked response overlaps with responses to subsequent stimuli, such as in a rapid serial visual presentation (RSVP) paradigm used to control a brain–computer interface (BCI). The present study explored whether the measurement or classification of select brain responses during RSVP would improve through application of an established technique for dealing with overlapping stimulus presentations, known as irregular or “jittered” stimulus onset interval (SOI). EEG data were collected from 24 healthy adult participants across multiple rounds of RSVP calibration and copy phrase tasks with varying degrees of SOI jitter. Analyses measured three separate brain signals sensitive to attention: N200, P300, and occipitoparietal alpha attenuation. Presentation jitter visibly reduced intrusion of the SSVEP, but in general, it did not positively or negatively affect attention effects, classification, or system performance. Though it remains unclear whether stimulus overlap is detrimental to BCI performance overall, the present study demonstrates that single-trial classification approaches may be resilient to rhythmic intrusions like SSVEP that appear in the averaged EEG.

1. Introduction

1.1. Rapid Serial Visual Presentation (RSVP)

Brain–computer interface (BCI) systems make use of neurophysiological signals to accomplish myriad goals [1]. Consequently, there are many varieties of BCI, such as implantable systems to restore spinal function [2], serve as a speech neuroprosthesis [3], or act as a digital switch [4]. Complementary to implantable systems, scalp-recorded EEG-based BCIs remain popular and accessible options for numerous applications [5]. Among their possible uses, EEG-based BCIs can assist with state monitoring [6,7] or augmentative and alternative communication (AAC-BCI) for individuals with disabilities [8,9]. One established AAC-BCI paradigm is rapid serial visual presentation (RSVP) [10,11,12,13], which presents a user with a rapid sequence of images (e.g., letters) and seeks to differentiate user intent to select one of the presented stimuli by capturing the so-called P300 response. Though RSVP relies on a user’s ability to process visual stimuli, it has the benefit of being gaze independent, meaning that it is accessible to individuals who have limited ocular motility [14]. This advantage is in contrast to the conventional matrix speller that benefits from functional eye movement [15].

1.2. Problems with Overlapping Adjacent Brain Responses

A possible limitation of RSVP is that rapidly presented visual stimuli have the potential to interfere with one another. Indeed, it is well established that brain responses to isolated stimuli can be modulated when preceded or followed by other stimuli, as a result of either order or pace [16,17,18]. In one highly-relevant study [17], the author detailed the extent to which event-related potential (ERP) responses can distort one another when stimuli are presented in rapid succession. That is, when there is a short delay between the onset of the first stimulus and the onset of the second (stimulus onset latency; SOI), then the first ERP response has not completed before the second is evoked. This proximity results in overlap between the responses and makes it difficult to differentiate or measure isolated responses. To this point, recent research has described differences in attentional N200 and P300 ERP amplitudes between different presentation rates during RSVP, as well as a drastic reduction in an attention-related alpha attenuation effect during faster presentations (i.e., shorter SOIs) [19].

1.3. Stimulus Onset Interval “Jitter”

Despite a general understanding that short SOIs might pose a problem for the measurement of ERPs, there has been little to no effort to control for such interference in the context of RSVP BCIs. To be sure, it is altogether unclear whether overlapping evoked responses are necessarily detrimental to the function of such systems, since a classifier only needs to differentiate a target from non-targets, not quantify the true amplitude or latency of a specific ERP component. In fact, the rhythmic presentation of visual stimuli will often generate a steady-state visual-evoked potential (SSVEP), which can itself be used to guide BCI function [20]. However, assuming some relationship between fidelity of evoked responses and system performance, one classic approach to the problem of measuring overlapping ERPs has been to randomize or “jitter” SOI within a pre-determined range, such as in the adjacent response (ADJAR) technique [17,21] (not to be confused with P300 “latency jitter” [22,23,24,25]). Put simply, randomizing the amount of overlap between responses by jittering SOIs should minimize constructive and destructive interference between repeated stimulus presentations, reduce measurable presence of an SSVEP, and result in more accurate estimates of target-related responses.

1.4. Aims and Hypotheses

The objective of the current study was to determine whether the application of jittered SOIs would meaningfully affect electrophysiological measures of attention observed during an RSVP paradigm, where attentional responses to discrete stimuli typically overlap with one another, and whether these changes would improve performance of the BCI system. In the context of this investigation, “attention” is defined as selective visual attention to the presentation of a target letter stimulus. Likewise, SOI refers to the time elapsed between the onset of two sequential stimuli, which some groups refer to instead as stimulus onset asynchrony (SOA) [11,26]. Our study analyzed three EEG-based signals known to be evident during RSVP: N200, P300, and alpha attenuation [11,19]. The N200 and P300 are endogenous ERPs known to reflect contextual deviation and the recognition of target stimuli, respectively [27]. Alpha rhythms are neural oscillations in the approximate range of 8–13 Hz that are thought to result from thalamocortical synchronization [28]. Posterior alpha attenuation is known to correspond with attentional allocation and shifts in visuospatial attention [29,30]. All three of these signals are known to show increased response amplitudes to target stimuli, compared to non-target stimuli in RSVP. We hypothesized that N200 and P300 attentional responses would show larger amplitudes in the jittered conditions, compared to non-jittered stimulus presentations. Similarly, we predicted that attentional alpha attenuation responses would be larger for jittered stimulus presentations than for non-jittered presentations. We predicted that machine learning classification of these three attentional markers would improve with the application of jittered SOIs that decrease the SSVEP component in averaged ERPs, and that participants would have higher accuracy rates during on-line copy phrase tasks in the jittered conditions.

Secondary aims of this study included exploratory examinations of individual differences in the alpha attenuation signal, and also measurement of variance in the ERPs. A slowed-down version of the RSVP task was presented at the end of each visit to collect non-overlapping electrophysiological markers, but this task existed as a non-experimental addition only. Results of the slowed-down task are discussed intermittently; however, there is significant difficulty in measuring an experimental baseline effect of stimulus overlap without qualitatively changing the RSVP task (see Section 4.3). To supplement the primary alpha attenuation analysis, we also performed a comparison of “responder” and “non-responder” groups (see Section 3.7.2). The motivation for expanding our scrutiny of this measure was two-fold: firstly, compared to the literature on the N200 and P300 responses, alpha attenuation in RSVP is less understood. As such, the present investigation is an opportunity to document important characteristics of this measure. Secondly, differences between slow and fast letter presentations in RSVP were much more dramatic for alpha attenuation than for N200 or P300 in a previous report [19], and individual differences have been previously identified as a potential contributing factor to similar alpha measures [19,31].

2. Materials and Methods

To address the objectives of the present study, a sample of generally healthy adult research participants were recruited to participate in a single 2.5 h experimental visit at Oregon Health & Science University (OHSU) in Portland, OR, USA. All study activities were reviewed and approved by the OHSU Institutional Review Board (IRB; protocol #24803). A waiver of authorization was obtained to conduct screening procedures. All study participants provided written informed consent prior to engaging in the primary testing procedure. Individuals were compensated USD 40 in cash for completing the study.

2.1. Participants: Recruitment and Screening

Twenty-four participants enrolled in the study following recruitment via physical flyers posted at OHSU, advertising on local research opportunities email listservs, and also word of mouth. Demographic information for the study cohort is presented in

Table 1. Demographics. A summary of participant characteristics for the present study. The sample represented a wide range of ages and was balanced with regard to gender. However, participants were predominantly white, non-Hispanic, and highly educated.

	Participants (n = 24)
Age: mean years ± SD (range)	38.71 ± 17.82 (18–76)
Gender
Female	12
Male	11
Non-Binary	1
Race
American Indian or Alaska Native	1
Asian or Asian American	1
White	19
Other/Multiple	3
Ethnicity
Hispanic/Latino	3
Not Hispanic/Latino	21
Education
Some college but no degree	2
Associate degree	1
Bachelor’s degree	10
Postgraduate degree	11

. A majority of participants were white and highly educated. Enrollment was balanced with regard to gender.

All individuals who expressed interest in the study were required to complete a telephone screening with a study coordinator prior to enrollment. Eligibility criteria stipulated that participants must (1) be age 18–80 years old; (2) agree to abstain from alcohol and drug use on the same calendar day as the study visit; (3) be fluent in English; (4) self-identify as generally healthy; and (5) have normal or corrected-to-normal vision. Additionally, an individual was ineligible to participate in the study if they (1) had a significant medical or neurological disease that would interfere with the study outcomes (as discussed with the senior author, a trained neurologist); (2) were regularly taking medications or substances known to compromise the study outcomes or EEG (e.g., benzodiazepines); (3) reported regular insomnia and poor sleep behaviors; or (4) scored < 31 on the modified Telephone Interview for Cognitive Status (TICS-m) [32] or < 3 on the judgement subtest of the Neurobehavioral Cognitive Status Exam (NCSE) [33].

2.2. Procedure

Eligible participants who completed a phone screening scheduled a single 2.5 h experimental visit to OHSU. Upon arrival, participants provided written informed consent and completed a limited set of intake questionnaires, including a demographics questionnaire. Participants reported the number of hours they had slept the night before the visit and completed a near-field Snellen visual acuity test (binocular) to confirm that they had at least 20/30 visual acuity. All individuals confirmed that they had abstained from alcohol, marijuana, and other recreational drug use on the day of the visit. After intake, research staff helped participants to don an EEG cap before proceeding with the primary visit procedure.

The visit procedure consisted of three repetitions of the RSVP calibration task (Figure 1), each of which was followed by two attempts at an RSVP copy phrase task (see Section 2.3). That is, participants would complete a calibration, then attempt to spell two different words in copy phrase using a model trained on the calibration data, before finally moving on to the next calibration. Each of the three repetitions corresponded with a test condition: (1) letters presented at a steady rate of 5 Hz, with no jitter applied to the SOI; (2) letters presented at 5 Hz, with a “small” uniform distribution jitter of ±50 ms; and (3) letters presented at 5 Hz, with a “large” uniform distribution jitter of ±100 ms. In order to control for order effects due to fatigue and/or practice, the sequence of these three conditions was pseudo-randomly balanced across the 24 participants, such that each of the six possible permutations of condition order occurred exactly four times. Participants were not informed of the condition types or order.

Before and after each round of calibration and copy phrase, participants provided self-report estimates of their sleepiness, according to the Stanford Sleepiness Scale (SSS) [34,35]. Sleepiness is one aspect of vigilance (or fatigue) that is known to have an impact on BCI performance [6]. Participants were also asked at these same intervals to rate their levels of both head (or headache) pain and general pain. After completing the third assigned test condition, participants responded to a brief series of user experience questions. Specifically, participants were asked (1) if they perceived a difference in various rounds of calibration and copy phrase tasks; (2) if so, what that difference was; (3) if they had a preference for or against any particular round of calibration or copy phrase; and (4) to rank the conditions from best to worst, even if they perceived no differences or had no strong feelings.

After completing the user experience questions, participants completed one final round of calibration with the letters presented at a significantly slower rate of one per second, with no jitter. This “slow” instance of the calibration task was recorded in order to elicit non-overlapping brain responses for the sake of visual and secondary comparisons. However, since this version of calibration generally seemed to be far more fatiguing than the test conditions, it was always presented at the end of the visit, so as not to interfere with the other presentations. Likewise, this “slow” version of the task was not accompanied by any copy phrase tasks, since its primary purpose was to collect non-overlapping profiles of the ERP and alpha attenuation responses for the individual participants.

2.3. RSVP Task

2.3.1. RSVP Task: Practice

All participants viewed instructional videos for the RSVP calibration and copy phrase tasks immediately preceding the first instance of each task. The instructional videos lasted approximately three minutes apiece. In addition to offering an explanation of the tasks, the RSVP calibration video also included a scripted series of four practice sequences, similar to those used in the test version of the task. Participants were asked to verbally indicate whether a specific target letter was present or absent from the given sequences. The purpose of this practice was to confirm that individuals understood the objective of RSVP and were able to complete the task with letters presented at a pace comparable to that of the test calibrations. Participants were required to achieve 75% accuracy before proceeding to the first test calibration. All participants passed the practice presentation after a single attempt: 22 participants achieved 100% accuracy and the remaining 2 participants achieved three out of four correct responses.

2.3.2. RSVP Task: Calibration

RSVP calibration tasks each consisted of 100 inquiry sequences, where a single inquiry sequence comprised a target letter prompt onscreen for 1 s, followed by a fixation cross for 500 ms, and then a series of ten letter stimuli presented one after another at a pace determined by the test condition (no jitter; “small” jitter; or “large” jitter). Participants were instructed to take note of the target prompt, watch for that same character in the subsequent letter stream, and mentally react to the target character when they saw it appear onscreen. The experimenter asked participants to do their best to refrain from physically reacting to the letter presentations and to wait until between sequences to blink, yawn, cough, etc.

Letter stimuli included all 26 characters of the English alphabet, as well as the character “_” to indicate a space. All letter stimuli appeared at the center of the screen, while a counter at the upper-left corner of the screen kept track of the number of inquiries completed out of the total 100. In 90% of inquiry sequences, the ten flashing letters comprised one target to match the most recent prompt, as well as nine non-target stimuli. In the remaining 10% of inquiries, however, only non-target characters were presented. Target and non-target characters were determined and sequenced randomly for each inquiry. There was a 2 s blank delay period between the disappearance of the final stimulus of one inquiry sequence and the onset of the target prompt for the next inquiry.

In the “no-jitter” condition (Figure 1B), the ten letter stimuli following the fixation cross were presented at a steady rate of 5 Hz (i.e., a fixed SOI of 200 ms), with no substantial blank interval between the offset and onset of adjacent characters. In the “small” jitter condition, these same letters were presented using a uniform random range of 150–250 ms (200 ± 50 ms). For the “large” jitter condition, this uniform range was increased to 100–300 ms (200 ± 100 ms) per character. Jitter ranges of 100 and 200 ms were selected because they, respectively, constituted 50% and 100% of the fixed SOI duration in the no jitter condition. Lastly, in the supplementary “slow” calibration at the end of the visit, letter stimuli were presented at a pace of 1 per second, with no jitter. Additionally, the number of inquiry sequences for the slow calibration was reduced from 100 down to 50, in order to help with participant fatigue.

2.3.3. RSVP Task: Copy Phrase

With the exception of the final slow calibration, each instance of RSVP calibration was immediately followed by two attempts at RSVP copy phrase, each with matching condition presentation parameters (e.g., the small jitter calibration was followed by two copy phrase sessions with the same SOI ranges). The paradigm was highly similar to calibration, with the primary exception that there were no target prompts before fixation. Instead, participants used a model trained from the most recent calibration (see Section 2.7.1) and attempted to use their mental strategy to select characters from the flashing letter streams. Participants had the objective to copy the words “HELLO” and “WORLD” to complete the phrase “HELLO_WORLD”. The purpose of separating these two words into separate sessions was to reduce participant frustration in cases where errors were made on the first word, especially since there was no opportunity to backspace or correct erroneous selections in this particular version of the task. A reference phrase at the top of the screen showed participants both their target phrase and also their progress as they moved through the task.

2.4. Stimuli

All BCI tasks were presented on a 16″ laptop monitor cycling at 165 Hz and placed at an approximate viewing distance of 70 cm, though head position was not constrained. The presentation computer was a Lenovo Legion 5 Pro with an 11th Gen Intel i7-11800H CPU @ 2.30 GHz and 16 GB of RAM running Windows 11 Home (64 bit). Application processes utilized an NVIDIA GeForce RTX 3050 laptop GPU to ensure presentation fidelity. All letter stimuli were rendered in Overpass-Mono font at the center of the screen. Letter stimuli were drawn in white against a solid black background, with the exceptions that target prompts in calibrations were yellow, and all fixation crosses were drawn in red. Assuming a screen distance of 70 cm, the monospace font constrained letter stimuli to appear within an imaginary box that subtended 2.05° by 3.27° visual angle, or approximately 2.5 cm wide by 4.0 cm tall on the screen. Characters had a stroke width of approximately 0.4 cm, or roughly 0.33°.

2.5. Electrophysiological Recordings

EEG data were recorded at 10/20 sites F7, FCz, Pz, P4, Oz, PO7, and PO8 from a DSI-VR300 dry electrode cap (Wearable Sensing, San Diego, CA, USA) with linked ear reference (A1 and A2) and ground at A1. The recording sites were selected primarily to ensure good measurement of the P300 response [36], but custom modification of our system moved the standard DSI-VR300 P3 site to F7 in order to capture limited EOG and eye blink activity. Data were collected at a sampling rate of 300 Hz, 16-bit A/D conversion, and signal quality was adjusted prior to recording to be within the ranges recommended by the manufacturer. Signal quality was assessed in the Wearable Sensing DSI-Streamer software (v.1.08.44), but all experimental recordings were collected using the acquisition client in BciPy using version 2.0.1rc2 [37].

2.6. Electrophysiological Processing

The majority of offline analyses were conducted using BrainVision Analyzer Professional Edition (v. 2.1.0.327; BrainVision LLC, Morrisville, NC, USA). To mirror past work which examined highly similar ERP and alpha attenuation outcomes [19], our across-participant analyses focused on electrode site Pz and also a “pooled” occipitoparietal signal, which was the average of sites Pz, P4, Oz, PO7, and PO8. Within-participant examinations of alpha attenuation retained data from each of the 5 recording occipitoparietal recording electrodes. Offline analyses were conducted on calibration recordings only; EEG data recorded during copy phrase were only used online for real-time target classification and letter selection. One participant had consistently poor contact at site Oz, so that site was removed from the pooled signal and all other analyses for that individual.

2.6.1. ERP Analyses

In accordance with established recommendations for attentional N200 and P300 studies [38], EEG recordings were filtered 0.1–45 Hz with a 60 Hz notch (Butterworth zero-phase infinite impulse response filter; 48 dB/octave). Filtered recordings were downsampled from 300 Hz to 150 Hz and segmented into 1 s test epochs ranging from −200 to +800 ms, relative to the onset of each letter stimulus (t = 0 ms). Test epochs were then baseline corrected using the 200 ms prior to stimulus onset (−200 to 0 ms), separated according to target class, and averaged within each condition. In order to estimate ERP amplitudes, a semi-automatic peak detection tool labelled weighted local maxima (0.50 weight strength) for N200 (200–350 ms) and P300 (300–550 ms) in the averaged target letter epochs. Labels were visually inspected and adjusted as necessary to correct cases of high-frequency peak capture or other general inaccuracies. Finally, N200 and P300 amplitudes for target and non-target averages were estimated as the mean signed voltage (in microvolts; µV) ± 4 sampled points from the labelled N200 and P300 peak latencies of the respective target responses. These estimates allowed us to analyze not only the ERP responses to target and non-target letter stimuli, but also the target attention effect, which we defined as the difference between measured target and non-target amplitudes.

To approximate signal variance, we measured the standard deviation of each participant’s grand-averaged test epochs. As well, we generated a second variance metric, which took an average of the standard deviation measured across all available epochs at each sampled point within the window of 200–600 ms post-stimulus onset. The logic behind using these two measures was that the standard deviation of the grand average would offer a general sense of noise, while the average standard deviation of the smaller window would offer a picture of variability within the window of the attentional ERPs.

2.6.2. Time–Frequency Analyses

In order to measure target-related alpha attenuation, EEG data were reprocessed to generate time–frequency estimates of alpha activity. EEGs were filtered and downsampled, similar to the ERP analyses, with a single change that the high-pass filter was raised from 0.1 Hz to 1.0 Hz. Filtered data were segmented into 2.5 s test epochs ranging from −1250 to +1250 ms, relative to the onset of each letter stimulus (t = 0 ms), and then separated according to target class. Individual epochs were transformed into scaleograms via continuous wavelet transform (CWT) using a complex Morlet mother wavelet (c = 5), with frequencies ranging 4–16 Hz in 48 linear steps. CWT output was normalized to uniform scale power (unit energy normalization), such that different frequency layers all possessed an energy value of 1, and complex-valued output was transformed to real-valued voltage. Lastly, mean and standard deviation information were extracted from a 500 ms baseline window of −600 to −100 ms and used to transform all time–frequency samples within each of the test epochs into Z-scores.

To more pointedly examine the target-related alpha attenuation response derived from time–frequency decomposition, we defined “alpha activity” as the average of all Z-scored samples within the response window of 300–800 ms post-stimulus onset [19]. In turn, the alpha attenuation response was defined as the difference in alpha activity responses following target versus non-target stimuli. Rather than look at the entire alpha band (~8–13 Hz), values were taken on a per-participant basis from the wavelet layer nearest to that participant’s individual alpha frequency (IAF) [39]. To identify each participant’s IAF, a fast Fourier transformation (FFT) was performed on all available 2.5 s inquiry epochs for the pooled occipitoparietal signal and averaged within each of the calibration recordings. The FFT was parameterized in Brain Vision to utilize a 20% Hanning window, periodic variance correction, and produce non-complex voltage output normalized relative to a frequency range of 4–20 Hz. Local amplitude maxima were identified 7–13 Hz and manually reviewed to correct cases of obvious peak capture by noise or a harmonic of the SSVEP signal. There was no apparent IAF in 19/96 calibration recordings, so in these cases a default IAF value of 10 Hz was established to broadly approximate alpha activity. Importantly, the resolution of the FFT was limited computationally to 0.293 Hz, so estimates of IAF were interpreted accordingly as approximations. The result of this IAF process was that each participant had four estimates of IAF: one per each of the calibration recordings.

2.6.3. Artifact Rejection

Although artifact minimization is ostensibly an important part of EEG and ERP preprocessing, there are surprisingly few BCI systems that use online tools to deal with transient artifacts such as EOG and EMG [40,41]. Nevertheless, to ensure that our results in this study were not contaminated by or attributable to gross artifact (e.g., blinking, EOG, EMG, poor electrode contact), offline across- and within-participant analyses made use of artifact rejection in EEG preprocessing. The only exceptions to this rule were that classifiers made use of the entire data set, in order to better match the current iteration of BciPy software, and align more realistically with real use cases, where artifact rejection is often absent. As applied, artifact rejection occurred after epoch segmentation for both the ERP and time–frequency analyses, but prior to separation of the stimuli into target and non-target classes. We applied semi-automatic review, triggered by any violations of pre-defined criteria: voltage slope > 50 µV/ms; change in voltage > 125 µV within a span of 50 ms; absolute voltage > 75 µV; and lack of activity, defined as absolute voltage < 0.5 µV sustained for at least 100 ms. Flagged violations were reviewed by experienced research staff and removed from the analysis if there was any apparent artifact (EOG, EMG, blinking, movement artifact, or other non-descript EEG noise) coincident with the ERP and alpha attenuation effect windows.

Overall rejection rates were similar across the test conditions for both ERP and alpha analyses, according to two-tailed paired-samples t-tests (all p values ≥ 0.40). Because of unequal target class representation in the RSVP paradigm, we also looked more specifically at target rejection rates. Again, there were no significant differences in target rejection rates between the test conditions for the ERP or time–frequency analyses (all p values ≥ 0.077). The majority of participants retained at least two-thirds of their target stimuli across the various calibration recordings in both the ERP and alpha analyses following artifact rejection. We included all participants in the primary analysis, regardless of target rejection rates. Further discussion is included in Section 3.7.1.

2.7. BCI Classifiers

Machine learning classification of the ERP data was performed offline to provide classification models for copy phrase tasks, as well as to generate offline estimates for statistical comparisons. Alpha data were classified offline as well, but not used to drive the copy phrase task. For offline comparisons, we generated estimates of balanced accuracy in order to accommodate the approximately 9-to-1 non-target-to-target unbalanced class ratios:

acc_bal = (acc_target + acc_non-target)/2

(1)

Statistical tests utilized mean balanced accuracies following 10-fold cross validation. Python code for the BCI classifiers used in this project can be found online [42,43].

2.7.1. BCI Classifiers: ERP Data

Online classification methods for ERP data in BciPy have been outlined in previous work [6,37]. Time series inquiry data collected during calibration were filtered 1–20 Hz with a 60 Hz notch and divided into 500 ms epochs ranging 0–500 ms, relative to each discrete stimulus onset. Processed data from channels Pz, P4, Oz, PO7, and PO8 were then subject to principal component analysis (PCA), regularized discriminant analysis (RDA), and kernel density estimation (KDE). Resultant models were used within condition to drive classification in copy phrase tasks.

2.7.2. BCI Classifiers: Alpha Data

In order to generate offline classification estimates for alpha, we utilized PyWavelets version 1.4.1 [44] to perform pre-processing steps similar to those outlined in Section 2.6.2 and Section 2.7.1. Data from channels Pz, P4, Oz, PO7, and PO8 were filtered 1–20 Hz and segmented before CWT with a Morlet mother wavelet scaled to each individual participant’s IAF (scaled wavelets were not normalized in Python). Data were Z-scored with the same temporal windows as those outlined in Section 2.6.2. These time series representations of alpha activity were then fed into a logistic regression classifier with L2 regularization, which we selected because it was the most performant option from a previous report [19]. Similar to the ERP classifier, we output mean balanced test accuracies to use for statistical comparisons. Lastly, in an attempt to maximize classifier performance, we re-ran logistic regression models and allowed the start of Z-scoring baseline and effect windows to shift on an individual-by-individual basis between −1050 and −600 ms and between +150 and +550 ms, respectively (this approach is also outlined in [19]). Compared to the default windows, these “tuned” windows were shifted iteratively by the system to maximize accuracy estimates.

2.8. Statistical Analyses

Data were scrutinized using SPSS Statistics, v.27 (IBM Corporation, Armonk, NY, USA). All across- and within-participant electrophysiological measures were tested for normality with Shapiro–Wilk tests and visually screened for outliers using box plots. The across-participant ERP measures were mixed: roughly one-third of N200 measures were either non-normal or contained outliers, and approximately one-eighth of P300 measures were either non-normal or contained outliers. On the other hand, only one-third of the mean alpha activity measures were normally distributed and without outliers. An overwhelming majority of the within-participant alpha amplitude measures contained outliers as well, and were not normally distributed, according to both Shapiro–Wilk and Kolmogorov–Smirnov tests.

Because distributions for the electrophysiological measures contained ubiquitous outliers and were not consistently normal, we decided to utilize non-parametric tests to compare differences in median values between our measures. This approach is similar to previous work examining comparable measures [19]. Friedman tests were used to determine the presence of condition effects for the EEG measures, followed by pairwise comparisons with Bonferroni correction in cases where there were significant differences. We used two-tailed related samples Wilcoxon signed-rank tests to compare across-participant measures, and Mann–Whitney U-tests to compare differences in median alpha amplitude between target and non-target responses within individual RSVP recordings. To satisfy assumptions of Mann–Whitney U, measures were visually inspected to confirm a general similarity in shape across distributions. Relationships between alpha and ERP responses were quantified with Spearman’s rank–order correlations. Similarly, Spearman’s rank–order correlations were used to relate electrophysiological measures and self-report data.

3. Results

All participants completed the planned calibration and copy phrase tasks. All participants reported sleeping at least six hours the night prior to the visit, with a single exception of one individual who reported four hours of sleep. Neither age nor hours slept prior to the study visit correlated with the ERP target effects or alpha attenuation measures in the experimental conditions (all absolute r_s(22) values ≤ 0.371; all p values ≥ 0.075). Generally, changes in self-reported sleepiness (SSS), headache pain, and general pain did not predict any of the electrophysiological attention effects. There was only a single significant exception to this pattern, such that decreases in general pain related to decreases in the alpha attenuation effect in the pooled signal for large jitter calibrations (r_s(22) = −0.417, p = 0.043), though notably this was an inversion of the expected relationship. There were no significant effects of jitter condition on any of the self-report measures of sleepiness, headache pain, or general pain assessed before or after the 5 Hz test calibration sessions, or for changes in those measures pre-/post-calibration (all χ²(2) values ≤ 3.391; all p values ≥ 0.183).

3.1. ERP Analyses

3.1.1. ERP Analyses: N200

Averaged ERP waveforms are presented in Figure 2; illustrative scalp topographies are shown in Figure 3. Wilcoxon signed-rank tests indicated that there were no significant N200 target effects at site Pz within any of the experimental conditions or supplementary slow calibration (all p values ≥ 0.092). In the pooled signal, however, we observed significant N200 target effects within every calibration (all p values ≤ 0.002). A related samples Friedman test indicated that there were no significant differences in the N200 target effect distributions at Pz among the no jitter (Mdn = 0.186), small (Mdn = −0.580), or large jitter (Mdn = −0.796) conditions, χ²(2) = 1.583, p = 0.453. In the pooled signal as well, there were no meaningful changes in the N200 target effect across the no jitter (Mdn = −1.643), small (Mdn = −1.949), or large jitter (Mdn = −2.064) conditions, χ²(2) = 2.333, p = 0.311.

3.1.2. ERP Analyses: P300

Significant P300 target effects were measured in all experimental conditions and the slow calibration, both at Pz and in the pooled signal (all p values < 0.001). A Friedman test indicated that there were no significant differences in the P300 target effect distributions at Pz across the no jitter (Mdn = 8.482), small (Mdn = 7.041), or large jitter (Mdn = 8.763) conditions, χ²(2) = 3.250, p = 0.197. Looking at the pooled signal, there again were no meaningful changes in the P300 target effect across the no jitter (Mdn = 5.382), small (Mdn = 4.270), or large jitter (Mdn = 5.564) conditions, χ²(2) = 3.083, p = 0.214.

3.1.3. ERP Analyses: Signal Variance

There were significant effects of condition on standard deviation of the grand averaged epochs for both targets and non-targets, at both Pz and in the pooled signal (all χ²(2) values ≥ 8.083; all p values ≤ 0.018). In the pooled signal, small jitter target averages (Mdn = 2.30) demonstrated smaller standard deviations than no jitter targets (Mdn = 2.64), Z = 3.031, adjusted p = 0.007. Similarly for pooled signal non-targets, small jitter variance (Mdn = 0.88) was smaller than no jitter (Mdn = 1.15), Z = 3.175, adjusted p = 0.004, and the same was true for the large jitter condition (Mdn = 0.79), Z = 5.052, adjusted p < 0.001. At Pz, standard deviations of small jitter target averages (Mdn = 3.05) were smaller than either no jitter targets (Mdn = 3.78), Z = 2.454, adjusted p = 0.042, or large jitter targets (Mdn = 3.41), Z = −2.742, adjusted p = 0.018. For non-targets measured at Pz, standard deviation estimates were significantly lower in the large jitter condition (Mdn = 0.88) than in the no jitter condition (Mdn = 1.03), Z = 2.742, adjusted p = 0.018.

Unlike the grand averages, there were no effects of condition on the point-by-point averages of standard deviations within the window of 200–600 ms post-stimulus onset. The only item of note was a trend in the pooled signal toward smaller average ERP non-target standard deviations for the small (Mdn = 8.97) and large jitter (Mdn = 8.99) conditions, compared to no jitter (Mdn = 9.37), χ²(2) = 5.250, p = 0.072.

3.2. Time–Frequency Analyses

Peak alpha estimates (IAFs) ranged 7.32–12.01 Hz across participants, but remained relatively stable (within 0.5 Hz) across conditions for 16/24 participants. Of the eight participants who demonstrated fluctuating IAF values, only four evinced a range >1 Hz, and two of those four were the result of a default estimate due to a missing peak. We performed preliminary tests of the IAF estimates, but excluded data points where the lack of a clear IAF was manually set to 10.00 Hz, so as not to artificially affect the distributions. There were no differences among IAF estimates in the no jitter (Mdn = 9.96), small (Mdn = 9.81), or large jitter (Mdn = 10.25) recordings, χ²(2) = 1.830, p = 0.401. Spearman’s rank–order correlations indicated a significant relationship between IAF and the alpha attenuation effect only in the pooled signal for the no jitter condition (r_s(17) = 0.503, p = 0.028). Correlations between age and IAF estimates did not reach statistical significance, though there was a trend toward significance in both the no jitter (r_s(17) = −0.451, p = 0.053) and small jitter conditions (r_s(16) = −0.417, p = 0.085), such that older participants tended toward lower-frequency IAF estimates.

3.2.1. Alpha Effects: Across Participants

Average alpha activity waveforms are presented in Figure 4; illustrative FFT plots are shown in Figure 5. The alpha attenuation effect was significant in the slow calibration for both signal sources (p values ≤ 0.003). Among the experimental conditions, however, the alpha attenuation effect never quite reached statistical significance. At most, the difference between target (Mdn = 0.190) and non-target (Mdn = 0.230) alpha activity at Pz in the small jitter condition was trending toward significance, Z = −1.857, p = 0.063. The same trend was also evident between target (Mdn = 0.029) and non-target (Mdn = 0.187) stimuli at Pz in the large condition, Z = −1.771, p = 0.076. Friedman tests revealed no significant changes in the alpha attenuation distributions among the no jitter (Mdn = −0.042), small (Mdn = −0.120), or large jitter (Mdn = −0.130) conditions at Pz, χ²(2) = 0.750, p = 0.687, and no changes in alpha attenuation distributions across the no jitter (Mdn = −0.051), small (Mdn = −0.085), or large jitter (Mdn = −0.100) conditions for the pooled signal, χ²(2) = 0.333, p = 0.846.

3.2.2. Alpha Effects: Within Participants

We used Mann–Whitney U-tests to examine differences between target and non-target alpha activity responses within individual calibration sessions. In this way, we were able to scrutinize alpha attenuation responses at the level of the individual in all three test conditions, as well as the slow calibration. The incidence of significant alpha attenuation effects was maximal in the slow calibration, both at Pz (8/24) and also in the pooled signal (8/24), with an additional two trending significant effects (p < 0.10) in both signal sources. One participant, however, demonstrated significant increases in alpha following target stimuli in both signal sources for the slow calibration. Across the test condition calibrations, the number of significant alpha attenuation effects was reduced. At site Pz, there were 3/24 significant attenuation effects in the no jitter condition, 2/24 in the small jitter condition, and 3/24 in the large jitter condition (Figure 6). Similarly, the pooled signal demonstrated 6/24 significant target-related attenuation effects in the no jitter condition, 1/24 in the small jitter condition, and 2/24 in the large jitter condition. With the exception of site Pz in the small jitter condition, among the test conditions, there was always a single additional instance of a significant alpha activity difference in the opposite direction, and even two instances of such at Pz during the no jitter calibration.

3.3. Correlations between Across-Participant ERP Target Effects and Alpha Attenuation

Regardless of signal source, neither N200 nor P300 attention effects were significantly predictive of alpha attenuation in any of the three experimental conditions or the slow calibration (all absolute r_s(22) values ≤ 0.375; all p values ≥ 0.071). Similarly, N200 and P300 target effects did not significantly relate to one another in any of the test conditions (all absolute r_s(22) values ≤ 0.349; all p values ≥ 0.095). However, in the slow calibration, N200 and P300 target effects were significantly correlated in the pooled signal (r_s(22) = 0.522, p = 0.009).

3.4. Classification

Machine learning classification estimates of ERP and alpha-oriented time–frequency data were generated using all available EEG data from calibration recordings. These estimates did not make use of artifact rejection procedures, so as to better emulate real-world use cases.

3.4.1. Classification: ERPs

A Friedman test showed no statistically significant differences in the mean balanced test accuracy distributions among the no jitter (Mdn = 0.756), small (Mdn = 0.747), or large jitter (Mdn = 0.781) calibrations, χ²(2) = 1.750, p = 0.417. The slow calibration had lower model accuracies (Mdn = 0.669) than any of the three experimental 5 Hz conditions (all p values < 0.001). PCA/RDA/KDE classification estimates of the ERP signals (i.e., mean balanced accuracies) showed significant positive associations with the N200 attention effects measured in the pooled signal in the no jitter (r_s(22) = −0.634, p < 0.001), small jitter (r_s(22) = −0.535, p = 0.007), and large jitter (r_s(22) = −0.509, p = 0.011) calibrations. At site Pz, however, there was only a marginal relationship between N200 target effects and classifier balanced accuracies in the no jitter calibrations (r_s(22) = −0.371, p = 0.074). Classification accuracies were significantly positively correlated with P300 target effects in all experimental conditions, both at Pz and in the pooled signal (all r_s(22) values ≥ 0.479; all p values ≤ 0.018). Lastly, classifier-balanced accuracies were strongly positively correlated with accuracy during the copy phrase tasks in all test conditions: no jitter (r_s(22) = 0.795, p < 0.001); small jitter (r_s(22) = 0.541, p = 0.006); and large jitter (r_s(22) = 0.547, p = 0.006).

3.4.2. Classification: Alpha

We observed no significant differences among no jitter (Mdn = 0.557), small (Mdn = 0.554), or large jitter (Mdn = 0.539) mean balanced accuracy estimates from calibrations, χ²(2) = 4.750, p = 0.093. With window parameter tuning, there again were no notable differences among balanced accuracy distributions in the no jitter (Mdn = 0.548), small (Mdn = 0.557), or large jitter (Mdn = 0.560) calibrations, χ²(2) = 0.583, p = 0.747. Regardless of whether we used default or tuned windows, balanced accuracy estimates from the alpha classifier did not correlate with alpha attenuation effects measured offline in Brain Vision (all absolute r_s(22) values ≤ 0.373; all p values ≥ 0.073).

3.5. Copy Phrase Performance

No statistically significant differences were observed among overall accuracy rates in the no jitter (Mdn = 0.90), small (Mdn = 0.90), or large jitter (Mdn = 0.80) copy phrase tasks, χ²(2) = 1.853, p = 0.396. P300 target effects measured from both signal sources during calibration significantly predicted copy phrase accuracy rates in all experimental conditions (all r_s(22) values ≥ 0.453; all p values ≤ 0.026), with the single exception that the effect did not quite reach significance in the pooled signal during the small jitter condition (r_s(22) = 0.386, p = 0.062). N200 target effects measured during calibrations did not predict copy phrase performance, with the single exception that N200 target minus non-target differences in the pooled signal predicted accuracy rates during the no jitter condition (r_s(22) = −0.532, p = 0.007).

3.6. User Experience Questionnaire

In total, 13 out of 24 participants reported some perceived difference between the experimental conditions, and 12 of these individuals cited at least some element of speed or pace as a difference. However, only three participants articulated any perceptions of unevenness in the timing of stimulus presentations (e.g., “wobble”, as one individual described it). Out of the 24 participants, 14 responded that they had a preference for one of the conditions, though sometimes respondents cited internal factors (e.g., boredom or fatigue) as their reasons for picking one condition over another. There was not a statistically significant difference in participant rankings of the different conditions (Mdn = 2.00 for all conditions), χ²(2) = 1.583, p = 0.453.

3.7. Supplementary Analyses

3.7.1. Supplementary Analyses: Artifact Rejection

There is a known trade-off between increasing signal quality and loss of statistical power associated with the removal of samples—so much so that some advocate foregoing artifact removal altogether [45]. In the ERP analysis, 6/96 calibration recordings lost more than one-third of target epochs, and an additional 5/96 lost more than one-half of target epochs. In the alpha analysis, these counts increased to 12/96 and 7/96, respectively. Due to high rates of artifact rejection for a subset of participants, we removed individual recordings with >50% target stimuli rejection rates and re-ran key tests of condition effects on the electrophysiological measures. No more than two recordings were ever removed per condition. Friedman tests indicated that there were no changes to our previous results, and that there were no significant effects of condition on N200 or P300 target effect distributions, or on the alpha attenuation distributions.

3.7.2. Supplementary Analyses: Alpha “Responder” and “Non-Responder” Groups

Previous investigations have suggested that the alpha and other time–frequency measures are subject to individual differences [31,46]. As such, it is possible that the attenuation effect might only be visible in a subset of individuals, so-called “responders”. To ensure that our across-participant results were not unduly blunted, we partitioned our sample according to “responders” and “non-responders”, where a “responder” was defined as a participant who showed a significant target-related alpha attenuation effect in at least 2/5 occipitoparietal recording electrodes during the 1 Hz slow calibration, where the alpha attenuation effect is most visible [19]. With this criterion, we identified 9 responders and 15 non-responders in our sample (Figure 7). Two non-responders showed significant differences in target versus non-target alpha activity, though in the opposite direction. Across the recording sites in the slow calibration, the alpha attenuation effect was most often significant at electrode P4, followed closely by Pz. Because of the prominence of the attenuation effect at P4, we decided to use that site for this particular supplementary analysis, in an attempt to maximize group differences.

All responders demonstrated significant target-related alpha attenuation effects at P4 in the slow calibration. However, these effects dissipated in the test conditions, where only 3/9, 1/9, and 2/9 responders demonstrated significant effects in the no jitter, small, and large jitter conditions. One additional responder demonstrated a significant target-related increase in alpha activity during the no jitter calibration. For non-responders, 2/15 individuals showed significant alpha effects during the slow calibration, but both were in the wrong direction. The only significant instances of target-related alpha attenuation observed in the non-responder group were in the no jitter (1/15) and large jitter condition calibrations (2/15). Even so, there were actually more numerous cases of significant inverse effects in both the no jitter (2/15) and large jitter (3/15) conditions. Non-responders showed no significant effects in the small jitter condition.

Using the responder and non-responder designations, we re-ran key across-participant condition comparisons for alpha attenuation measures in the separate groups. As expected, non-responders showed no significant target effects in any of the three test conditions (all p values ≥ 0.256), and no significant effect of condition on target versus non-target alpha differences in either the pooled signal or Pz (both χ²(2) values ≤ 2.533; both p values ≥ 0.282). For responders, there again were no significant differences between target and non-target alpha responses in the no jitter and large jitter conditions (all p values ≥ 0.110). However, in the small jitter condition, target effects were trending in both signal sources (both p values ≤ 0.066). Responders demonstrated no significant effect of condition on alpha attenuation distributions in either signal source (both χ²(2) values ≤ 1.556; both p values ≥ 0.459).

4. Discussion

This study compared the performance of a BCI RSVP system across three different ranges of jittered SOI. We measured changes in three electrophysiological markers of attention: N200, P300, and alpha attenuation. We also quantified differences in signal classification and typing performance between the test conditions. Supplementary analyses scrutinized variance of the ERPs and individual differences in the alpha attenuation effect.

4.1. Summary of Findings

Null hypotheses were retained (see Section 1.4). We observed expected significant target effects for ERPs N200 and P300 in all test conditions. The N200 was localized more posterior than the parietal maximal P300, and P300 target effect was more consistently related to classifier performance than the N200. However, we found no effect of SOI jitter on the size of these target effects. The overall variability of ERP averages was decreased in the jittered conditions, but there was no effect of condition on the average variance of the attentional ERPs. Alpha attenuation effects were not quite significant in the test conditions and were unaffected by jitter. Machine learning classification was more performant for ERPs than for the alpha data, but neither showed obvious improvement with increasing jitter range. Intrusion of SSVEP artifact in the electrophysiological measures was visibly reduced in the jittered conditions. However, typing performance was similar between the test conditions and, subjectively, participants were not consistently aware of or concerned with SOI jitter.

4.2. Adjacent ERP Overlap, Jittered SOI, and SSVEP

Our research hypotheses were formed under the assumption that systematic distortion of averaged ERPs—which we observed as increased signal variance in the non-jitter condition—would translate to changes in performance of a BCI RSVP system. We also suspected that this rhythmic distortion, which we conceptualized as unwanted SSVEP artifact, contributed to changes in an alpha attenuation effect which were observed previously between slow and fast RSVP paradigms [19]. However, because our measures were generally unchanged by jittered SOIs, one should consider possible problems with the logic underpinning our assumptions. Specifically: (1) jittered SOI is not a one-size-fits-all solution to adjacent ERP overlap; (2) it is unclear whether SSVEP artifact has deleterious effects on event-related responses during RSVP; and (3) there are notable differences between the current design and previous investigations which may help to explain our results.

With regard to the first of these points, high-pass filtering can attenuate unwanted long-latency potentials, though in the case of P300 measurement, this solution perhaps is not practical [38]. Additionally, in cases of random stimulus presentation, target versus non-target differences are understood to be somewhat resilient to distortions caused by adjacent ERP overlap, though not necessarily all types of order effects [17]. Critically, this scenario accurately describes the current RSVP design, where the classifier is focused on class differences. Some have even concluded that detection of a relevant target response can occur without precise knowledge of the timing of stimulus onset [24].

As for SSVEP artifact, we acknowledge that many ERP-based RSVP systems continue to function with acceptable accuracy [13], despite a general lack of concern over overlapping adjacent ERPs. ERP- and SSVEP-based BCIs are generally characterized as separate mechanisms, to be sure, and have even been fused in hybrid designs [47]. Of importance to the question of jittered SOI, code-modulated visual-evoked potentials (c-VEP) also use irregular pseudo-randomized stimulus presentation durations, but there is no consensus that c-VEP is functionally beneficial relative to routine SSVEP [48]. Dramatic changes in alpha attenuation between slow and fast RSVP presentations [19], on the other hand, may very well be the result of qualitative endogenous changes in the task (e.g., cognitive load), rather than exogenous factors, such as interference from the SSVEP or an associated harmonic.

Lastly, we would like to consider our results as they relate to previous relevant investigations. To our knowledge, no previous work has explored jittered SOI in the context of the RSVP BCI paradigm, so we have no ideal candidates for comparison. The visual changes observed in our ERP waveforms with increased jitter latencies were ostensibly similar to those illustrated previously [17]. As noted above, however, stimulus randomization can be an effective tool for dealing with overlap when looking at target versus non-target differences. Additionally, compared to relevant discussion in [17] which assumed equal target and non-target likelihoods, the present RSVP design possessed unequal classes. As well, while previous reports described changes to ERP attention effects with increasing SOI durations, the present study used dramatically shorter latencies overall than [18], and another comparison of different SOA values on ERPs and VEPs did so using a block design [26]. Increasing N200 and P300 amplitudes observed in that study and a lack of such effects in the present study might be explained by our use of uniform random distributions, which result in an unchanging mean SOI value across conditions.

4.3. Limitations and Future Directions

Multiple factors limit the generalizability of the present report. First, the findings of this study are limited by a small sample size, and participants in our sample were generally healthy adults. Future investigations would benefit from increased sample sizes and the inclusion of persons from clinical populations, such as individuals with neurodegenerative conditions like amyotrophic lateral sclerosis (ALS), especially when investigating AAC-BCI paradigms. Second, though we sought to address problematic adjacent ERP overlap with jittered SOI, it is difficult to estimate baseline effects of overlap in RSVP without fundamentally altering the paradigm. Slowing down presentation rates (as we did in our non-experimental 1 Hz slow calibration) often results in increased fatigue, and the cognitive demands of the task may be qualitatively different from those at faster presentation rates (e.g., sustained attention, vigilance, or stimulus discrimination). Future testing would benefit from exploration of alternative designs that address overlap, but do not alter presentation rate, such as presenting RSVP stimuli at different spatial locations [49], or something akin to a modified matrix speller. Lastly, there were limitations in our analyses. Our online ERP classifier used for copy phrase was restricted to a window of 0–500 ms, which did not wholly match our offline measurements of N200 and P300, although these different approaches did correlate significantly. Our use of a regularized discriminant in the RSVP paradigm, generally speaking, might not match results from similar manipulations of other visual presentation paradigms, like a matrix speller, or results from machine learning approaches other than the regularized discriminant. As well, the logistic regression approach used to classify our alpha attenuation effect was clearly ineffective. Logistic regression is a relatively simple approach to class prediction, and it is likely that more sophisticated methods would demonstrate better performance. Likewise, it is possible that the temporal windowing constraints used for offline alpha analyses were not as well suited to our classifier. Future work should seek to identify an optimal approach to classification for the alpha attenuation measure.

5. Conclusions

Prior research suggests that temporal overlap of brain responses to sequential stimuli can adversely affect measurement of those brain responses. However, data from the present study show that performance during a BCI RSVP paradigm using a regularized discriminant classifier was almost completely unaffected by one known remedy for temporal overlap: SOI jitter. These findings indicate that SOI jitter does not significantly improve classification of ERPs or event-related alpha attenuation during RSVP. Likewise, the application of jitter did not significantly increase electrophysiological responses to target stimuli during RSVP in the current context, though these results also suggest that jitter would not be detrimental in other similar use cases. It remains unclear if temporal overlap of stimuli is detrimental to BCI RSVP performance.