Next Article in Journal
Detection of Crack Locations in Aluminum Alloy Structures Using FBG Sensors
Next Article in Special Issue
A Multi-Parametric Wearable System to Monitor Neck Movements and Respiratory Frequency of Computer Workers
Previous Article in Journal
A New Cycle Slip Detection and Repair Method Using a Single Receiver’s Single Station B1 and L1 Frequencies in Ground-Based Positioning Systems
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sensors
Volume 20
Issue 2
10.3390/s20020348
sensors-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Development of a Brain–Computer Interface Toggle Switch with Low False-Positive Rate Using Respiration-Modulated Photoplethysmography

by
Chang-Hee Han
,
Euijin Kim
and
Chang-Hwan Im
*
Department of Biomedical Engineering, Hanyang University, Seoul 04763, Korea
*
Author to whom correspondence should be addressed.
Sensors 2020, 20(2), 348; https://doi.org/10.3390/s20020348
Submission received: 28 November 2019 / Revised: 1 January 2020 / Accepted: 7 January 2020 / Published: 8 January 2020
(This article belongs to the Collection Respiratory Monitoring for Healthcare, Sport and Physical Activity: Sensors, Techniques and Applications)
Browse Figures
Versions Notes

Abstract

:
Asynchronous brain–computer interfaces (BCIs) based on electroencephalography (EEG) generally suffer from poor performance in terms of classification accuracy and false-positive rate (FPR). Thus, BCI toggle switches based on electrooculogram (EOG) signals were developed to toggle on/off synchronous BCI systems. The conventional BCI toggle switches exhibit fast responses with high accuracy; however, they have a high FPR or cannot be applied to patients with oculomotor impairments. To circumvent these issues, we developed a novel BCI toggle switch that users can employ to toggle on or off synchronous BCIs by holding their breath for a few seconds. Two states—normal breath and breath holding—were classified using a linear discriminant analysis with features extracted from the respiration-modulated photoplethysmography (PPG) signals. A real-time BCI toggle switch was implemented with calibration data trained with only 1-min PPG data. We evaluated the performance of our PPG switch by combining it with a steady-state visual evoked potential-based BCI system that was designed to control four external devices, with regard to the true-positive rate and FPR. The parameters of the PPG switch were optimized through an offline experiment with five subjects, and the performance of the switch system was evaluated in an online experiment with seven subjects. All the participants successfully turned on the BCI by holding their breath for approximately 10 s (100% accuracy), and the switch system exhibited a very low FPR of 0.02 false operations per minute, which is the lowest FPR reported thus far. All participants could successfully control external devices in the synchronous BCI mode. Our results demonstrated that the proposed PPG-based BCI toggle switch can be used to implement practical BCIs.

1. Introduction

The brain–computer interface (BCI) is an emerging technology for providing patients that have neurodegenerative diseases with alternative communication channels by decoding neural signals voluntarily modulated to control external devices or generate messages [1]. BCIs have been developed using a variety of neuroimaging modalities, including electroencephalography (EEG) [2,3,4,5,6], magnetoencephalography [7,8], functional near-infrared spectroscopy [9], and functional magnetic resonance imaging [10,11,12]. Among these, EEG is most widely used because of its advantages over the other modalities, e.g., its cost-effectiveness, high temporal resolution, and portability [13,14]. Over the past decades, neuroscientists have developed various BCI paradigms based on specific EEG signal patterns, such as the steady-state visual evoked potential (SSVEP) [15,16,17,18,19], auditory steady-state response [20,21], event-related potential [5,22,23,24], slow cortical potential [25], and event-related synchronization/desynchronization [26,27]. These BCI paradigms can allow patients in a locked-in state to communicate.
EEG-based BCIs can be realized in either the synchronous or asynchronous mode [28]. The synchronous BCI is a traditional design in which information regarding the exact timing for the classification is provided. The users of synchronous BCIs can operate the BCI system only within a limited time period designated by operators. They cannot freely change the BCI operation mode, i.e., from the no control (NC) state to the intentional control (IC) state or vice versa. In contrast, the asynchronous BCI can detect the intention of the user without changing the operation mode and thus, is considered a better approach. Ideally, the users of asynchronous BCIs can use the BCI system whenever they wish, without time constraints.
Although there have been numerous attempts to realize practical asynchronous BCIs using various EEG patterns [28,29,30,31], most of them exhibited a high false-positive rate (FPR) in the asynchronous mode. One of the solutions for this problem is a two-step approach, in which the BCI system is first turned on using a “brain toggle switch” and then the intention of the user is identified in a synchronous BCI mode. EEG-based brain-switch systems have been introduced [32,33,34,35]; however, their FPR was too high to be used in practical scenarios. Please note that hereafter turning on a BCI system is defined as positive. Reducing the FPR is important because unwanted operation of BCI systems might sometimes cause dangerous situations, such as the malfunction of wheelchairs and robotic arms.
Physiological signals other than brain signals can also be used to implement a BCI toggle switch. Electrooculogram (EOG) signal patterns elicited by successive eye blinks have been widely used to toggle on and off BCI systems [36,37]. A recent study by Li et al. [36] reported an EOG-based BCI toggle switch system with an average accuracy of 99.5%, an average response time of 1.3 s, and an average FPR of 0.10 false positives per minute (FPs/min), which was tested with healthy subjects. Wang et al. [37] also developed a hybrid BCI utilizing EOG. Their hybrid BCI system combined motor imagery, P300, and eye blinking to implement forward, backward, and stop control of a wheelchair. Their EOG-switch based on eye blinking showed an average accuracy of 92%, an average response time of 2.0 s, and an average FPR of 0.30 FPs/min. The performances of these two EOG-based BCI toggle switches may look promising; however, the system might not be applicable to patients with severe amyotrophic lateral sclerosis (ALS), who are regarded as major target patients of EEG-based BCI technologies, because oculomotor function (including eye blinking) is generally impaired at the later stage of ALS [38]. Indeed, many ALS patients with impaired oculomotor function have difficulty in controlling their eyelid, but most of them can still use SSVEP-based BCIs [39]. Moreover, the FPR of 0.10/min is still high (six false operations per hour).
Herein, we propose a new BCI toggle switch that patients can use to toggle on or off the BCI by holding their breath for a few seconds. Inspired by the fact that respiration pattern is reflected in photoplethysmography (PPG) signals [40], features associated with breath holding (BH) were extracted from the respiration-modulated PPG signals. We used a PPG sensor instead of other respiration sensors (e.g., a respiration belt, temperature sensor, or CO2 sensor) because PPG sensors are significantly less expensive and easier to use. Moreover, there are many wearable PPG sensors available in the market; thus, the switch system can be readily incorporated with any type of BCI system. In this study, the implemented PPG switch was combined with an SSVEP-based BCI designed to control four external devices. When a user wanted to toggle on (or off) the BCI system, he/she simply needed to hold his/her breath for approximately 10 s. Then, two states—normal breath (NB) and BH—were classified using a linear discriminant analysis (LDA) in real time. The parameters of the PPG switch were optimized through an offline experiment with five subjects, and the performance of the switch system was evaluated in an online experiment with seven subjects.

2. Materials and Methods

In this section, we describe the experiments performed in the present study. Furthermore, we explain the overall procedure of the offline and online data analyses.

2.1. Participants

Five (all males, age: 25.80 ± 2.48 years) and seven (four males and three females, age: 23.86 ± 1.64 years) healthy subjects participated in our offline and online experiments, respectively. A comprehensive summary of the experimental procedure was provided to each subject before the experiments. The participants provided written informed consent and were reimbursed for their participation in the experiment. This study was reviewed and approved by the Institutional Review Board Committee of Hanyang University Hospital (HYUH 2015-11-031-001) and conformed to the tenets of the Declaration of Helsinki.

2.2. Offline Experiment

An offline experiment was designed to develop the respiration-modulated PPG switch. The following three issues needed to be addressed for developing the respiration-modulated PPG switch: (1) Can the PPG signals be used to differentiate the NB and BH states? (2) What is the best PPG feature for classifying NB and BH? (3) What is the optimal window size for classifying NB and BH using PPG signals?
To address these issues, PPG and respiration data were simultaneously recorded using a multi-channel biosignal acquisition system (ActiveTwo; BioSemi, Amsterdam, the Netherlands) at a sampling rate of 2048 Hz. The PPG signals were recorded using a finger-type PPG sensor (MLT1020FC, ADInstruments, Australia) attached to the left index finger of each participant, and the respiration data were measured using respiration belts fastened around the chests of the participants (SleepSense 1387-kit). Note that the respiration data were used as ground-truth data just to check whether participants performed the BH task well.
Figure 1a presents a schematic of the offline experiment. At the beginning of the experiment, a brief instruction was presented for 5 s to provide each participant with preparation time. Then, all the participants alternately conducted NB and BH tasks for 30 and 15 s, respectively. They were asked to rest during the NB periods and to hold their breath during the BH periods. During the entire experiment, their body movements were strictly limited to avoid potential movement artifacts. A set of NB and BH periods was repeated 20 times.

2.3. Offline Data Analysis

The PPG and respiration data were preprocessed using a series of signal-processing algorithms to remove unwanted artifacts. First, the PPG and respiration signals were bandpass-filtered at 0.2- and 0.4-Hz cutoff frequencies using a third-order Butterworth zero-phase filter. The frequency band of 0.2–0.4 Hz was selected, considering that the typical resting respiratory rate of a healthy adult is 12–18 breaths per minute (0.2–0.3 Hz) [41]. First-order differentiation was used to determine the slope values in the filtered PPG data. Finally, epochs corresponding to NB and BH periods were separately extracted (0–15 s for BH and 10–25 s for NB). The initial NB period (0–10 s) was not included in the epoch, because this period might have included deep breaths after the BH task.
To determine whether the PPG signals could be used to differentiate NB and BH, we compared the preprocessed respiration and PPG signals recorded while the participants performed NB and BH tasks.
To investigate the optimal feature for the classification of NB and BH states, a statistical analysis was performed using offline experimental data. Although the average respiration rate is approximately 0.3 Hz, individual variability may exist owing to several factors, such as age and health [42]. To select the optimal frequency range for differentiating NB and BH, the 0–1-Hz frequency interval was evenly divided into 0.0625-Hz-sized bins, and the average power spectral density (PSD) values were calculated for both the NB and BH states. The PSDs were obtained by an equation below. The PSD of a random time signals x(t) can be calculated using
S x ( f ) = lim E T → ∞ { 1 2 T | ∫ − T T x ( t ) e − j 2 π ft dt | 2 }
where E represents the expected value. Then, a paired t-test was performed to confirm the statistical significance between the two conditions.
Finally, pattern classification was performed to determine the optimal size of PPG data for classifying NB and BH. PSD values for each task were calculated in the optimal frequency range selected via the statistical analysis described in the previous paragraph and were used as a feature vector for the classification of NB and BH. Note that the number of features was just one. The LDA classifier was used to calculate the classification accuracy [43]. Then, the average offline classification accuracies were evaluated for different PPG data sizes using the leave-one-out cross-validation method, considering the relatively small number of task trials (20 trials for each task). Consequently, for each iteration, 38 and 2 trials were used for training and testing, respectively.

2.4. Online Experiment

The main objectives of our online experiments were to develop an online respiration-modulated PPG switch and to validate its performance with regard to the accuracy and FPR. PPG and EEG signals were measured using the same multi-channel biosignal acquisition device that was employed in the offline experiment. A Velcro-type PPG sensor (MLT1020PPG, ADInstruments, Dunedin, New Zealand) was attached to the left forearm of each participant using an adhesive disk. In the online experiments, we changed the location of the PPG sensor from the left index finger to the left forearm because it was reported that the forearm is better than a finger for measuring respiration-related PPG signal changes [44]. Three active electrodes (Oz, O1, and O2) were used to record EEG signals originating from the primary visual cortex, and the ground electrode was replaced with two electrodes—a common mode sense active electrode and a driven right leg passive electrode—both of which were attached in the central region (near CP1 and CP2, respectively). The offset voltage of all the EEG electrodes between the A/D box and the body was maintained between 25 and 50 mV, as recommended by the EEG device manufacturer. Both the PPG and EEG data were sampled at 2048 Hz.
The online experiment comprised an offline training session and an online test session. An LDA classifier for the PPG switch was trained using the PPG data recorded during the training session, and the performance of the PPG switch was evaluated in the online test session. Figure 1b shows a schematic of the training session. Before the training session, an instruction was provided for 5 s. The participants prepared for the training session during this period. After the instruction, each subject performed the BH task for 10 s, followed by 45 s of rest. We asked them to restore their NB during this rest period. Finally, each subject performed NB for 45 s while watching short video clips. This overall procedure was repeated three times. Figure 1c describes the test session. A start instruction for 45 s was provided to all subjects before starting the test session. At the end of the instruction, a beep sound was presented, and each subject performed the BH task until the PPG switch turned on the SSVEP-BCI system. If the SSVEP-BCI system was turned on, four-class SSVEP stimuli were presented for 10 s. We asked all subjects to gaze at a target stimulus during this period. Instruction for the target SSVEP stimulus was provided with a number during the start instruction period and the NB (video) period (see Figure 1c). The numbers were presented in the upper left corner of the monitor. After the SSVEP task, the NC state was provided for 190 s, and each subject watched a short video clip while performing NB. The time period of 190 s was introduced to monitor false operations of the system. These procedures were repeated five times.
In the four-class SSVEP-BCI paradigm, four visual stimuli were presented on a liquid-crystal display monitor. The distance between the monitor and the subjects was approximately 60 cm, and the refresh rate of the monitor was 60 Hz. Considering the refresh rate, four stimulation frequencies (6, 6.66, 7.5, and 10 Hz) were selected to elicit SSVEPs, and they were assigned to four different visual stimuli. A conventional black and white three-by-three checkerboard pattern was used as a visual stimulus and was reversed at different frequencies.
Four different environment devices (an electric fan, a heater, a lamp, and an emergency alarm) were used in the online experiment. When the subjects gazed at a target SSVEP stimuli, an environment device corresponding to the SSVEP-BCI result turned on or off. We controlled the environment devices by using Internet of Things-based smart plugs (SPW010P, Silo, South Korea).

2.5. Online Data Analysis

The raw PPG data were preprocessed using the same signal-processing methods that were employed in the offline analysis. Three 10-s epochs were extracted from the preprocessed PPG data for each of the NB and BH tasks. Because it was reported that the main frequency band associated with respiration does not depend on the PPG recording sites [44], we employed the frequency range of 0.2–0.4, which was determined in the offline data analysis. The PSD values in the frequency range of 0.2–0.4 Hz were calculated for each epoch, and this feature set was then used to train an LDA classifier.
While the respiration-based PPG switch was operating, the PSD feature was seamlessly calculated every second using the past 10 s of data. Simultaneously, the LDA classifier determined whether the participant was in the IC or NC state according to the classification outputs. The classification outputs were denoted by 1 or 2, representing NB and BH, respectively. Six consecutive classification outputs constituted a pattern, e.g., 111,221, 112,211, and 221,212, and the pattern was updated every second. The respiration-based PPG switch was turned on or off when the output pattern matched a predetermined template pattern (111,222 in this study). When a subject starts the BH task, the first classification is made using the 10-s PPG data recorded during the previous NB period, resulting in a classification output of ‘1’. After at least 5 s from the starting time of the BH task, the 10-s PPG data would have a chance to be classified as ‘2’. Therefore, at least 8 s are needed to make the target pattern of ‘111,222’.
When a participant turned on the BCI system by holding his/her breath, raw EEG data were recorded from three occipital electrodes (Oz, O1, and O2) for 10 s. We selected these electrodes because they have been widely employed in many previous SSVEP-based BCI studies [4,16,32]. The spectral powers at the four stimulation frequencies (6, 6.66, 7.5, and 10 Hz) for each electrode were calculated using a fast Fourier transform, with the window size set to be 10 s. In this procedure, EEG data from 0 to 1 s were rejected to minimize the potential motion artifact due to the deep breathing right after the breath-holding period. For the identification of the SSVEP stimulus that a participant was staring at, the spectral powers at four stimulation frequencies (6, 6.66, 7.5, and 10 Hz) and those at their second/third harmonics were summed over all three electrodes, and the frequency exhibiting the highest power value was selected.

3. Results

3.1. PPG Signal Modulated by Respiration

Figure 2 presents examples of respiration and PPG signals recorded while a participant alternately performed NB and BH tasks. The respiration and PPG signals exhibited similar patterns. When the participant performed the NB task, clear sinusoidal wave patterns were observed in the PPG signals as well as in the respiration signals. However, the sinusoidal nature of the waveform was diminished when the participant held his/her breath. This example indicates the possibility of using the respiration-modulated PPG signal as a toggle switch to turn on/off a BCI system.

3.2. PPG Feature for Classifying NB and BH

To identify PPG features for accurate classification of NB and BH, a frequency band having significantly different PSD values between BH and NB was selected via statistical analysis. Figure 3 presents the statistical analysis results. As shown, a significant difference between NB and BH was observed in the frequency band of 0.2–0.4 Hz (averaged p-value < 0.05). This frequency band was used to evaluate the PSD of the PPG signal, which was then used as a feature to classify NB and BH.

3.3. Optimal Time-Window Size

Figure 4 shows the classification accuracies with respect to size of the PPG data. The red dotted horizontal line represents the chance level. The appropriate level of chance for binary classification should be set as 70% when the number of trials is 20 and the confidence level is 99% [45]. PPG data from 0 to 1 s were excluded, considering the fluctuation of the PPG data immediately after BH. As shown in Figure 4, a larger window used for the classification yielded a higher classification accuracy. The offline classification accuracy was almost saturated when 10 s of PPG data were used for the classification of NB and BH (the classification accuracy reached 88.5%). Considering the tradeoff between the communication speed and classification accuracy, we used a 10-s window for the detection of BH in the subsequent online experiments.

3.4. Performance of PPG Switch

Figure 5 presents an example of the online experimental results. The vertical axis indicates the classification outputs, where 1 and 2 correspond to NB and BH, respectively. As shown in the figure, only two FPs were observed for five repeated trials.
Table 1 presents the overall online experimental results. All subjects succeeded in turning on the BCI system by holding their breath. The average elapsed time for turning on the BCI system was 10.57 ± 2.38 s. The true-positive rates (TPRs) were 100%. Furthermore, the FPRs were as low as 0.02 FPs/min. Five out of the seven participants did not exhibit any FPs. These results indicate that the proposed respiration-modulated PPG switch can be used as a reliable and robust BCI switch with an extremely low FPR and low inter-individual variability.

3.5. Online Control of External Devices

Table 2 presents the online performance of the four-class SSVEP-based BCI, which was turned on by using the PPG-based switch. The target object was randomly presented to the participants, who were asked to operate one of the four external devices by staring at one of the pattern-reversal visual stimuli, which corresponded to an emergency alarm, a light, a heater, and a fan, respectively. After five repeated trials, the average accuracy was reported as 88.57%. Three out of the seven subjects achieved a classification accuracy of 100%. When the first 1 s of EEG data were included in the analysis epoch, the classification accuracy was dropped to 77.14%, suggesting that the first 1 s period might include some artifacts affecting the overall classification accuracy. Figure 6 presents a series of snapshots obtained during the online experiments when one of the participants successfully controlled one of the environmental devices. A video clip of the online experiment can be found in YouTube (see the following URL for more information: https://youtu.be/9KEcCh-evAA).

4. Discussion

We developed a respiration-modulated PPG switch with a low FPR and evaluated its performance by combining it with an SSVEP-based BCI. Users could turn on the BCI system using the PPG switch and could control four environment devices using the four-class SSVEP-based BCI. Our offline and online experimental results indicated that the proposed respiration-modulated PPG switch can be used for implementing asynchronous BCIs. To the best of our knowledge, this was the first study in which PPG signals have been used for implementing a BCI toggle switch.
Our respiration-modulated PPG switch exhibited a very low FPR (as low as 0.02 FPs/min), suggesting that users can operate our switch system for approximately 50 min without any false operations. The FPR of 0.02 FPs/min is the lowest FPR reported thus far for biosignal-based BCI toggle switches. EEG-based brain-switch systems [32,33,34,35,46] have the advantage that they do not require an additional signal-acquisition system other than the EEG; however, their FPRs are too high for application in practical scenarios. For example, a brain switch based on P300 exhibited a high FPR of 1.00 FPs/min [46]. SSVEP-based BCI toggle switches also exhibited high FPRs of 0.38 FPs/min [32] and 0.98 FPs/min [33], and motor-imagery-based BCI toggle switches exhibited a high FPR of 0.67 FPs/min [35] and 3.00 FPs/min [47]. In addition, hybrid BCI toggle switches, which simultaneously use two different BCI paradigms, showed 0.49 FPs/min [48] and 0.15 FPs/min [49]. EOG signals elicited by successive eye blinks can also be used for implementing a BCI toggle switch. A recently introduced EOG-based BCI toggle switch exhibited a high accuracy of 99.5% and a short response time of 1.3 s [36]. However, its FPR (0.10 FPs/min) was high compared with that of our respiration-modulated PPG switch. More importantly, this system might not be applicable to late-stage ALS patients, who generally have impaired oculomotor function. Our BCI toggle switch system not only has a very low FPR of 0.02 FPs/min but also is based on a straightforward respiration task. The users of our toggle switch are asked to simply hold their breath for a few seconds. In general, patients whose respiratory function is not impaired can easily hold their breath for a certain period of time; however, patients with ALS—who are among the major target users of EEG-based BCIs—gradually lose the ability to breathe voluntarily in the late stage of the disease. Artificial ventilation is generally used for patients with late-stage ALS [50]. Therefore, it should be confirmed whether patients with ALS can hold their breath for >10 s. To confirm this, we visited 10 ALS patients and measured their BH time using a stopwatch. The average R-ALSFRS [51] score of the ALS patients was 25.00 ± 14.42 out of 48. Two of the 10 patients had late-stage ALS, and their R-ALSFRS scores were just 1 and 4. The average BH time was 18.00 ± 4.11 s, and the two patients with severe ALS could also hold their breath for 10 and 20 s, respectively. The average BH time of approximately 18 s was long enough to use our respiration-modulated PPG switch, because the average time needed to turn on our BCI toggle switch was approximately 10 s. In the current system, some ALS patients might sometimes fail to turn on the BCI system; however, please note that they can still try the BH task once again after having a short rest. In addition, the average BH time is expected to be further reduced in future studies. The simplest way to reduce the time to turn on the BCI system is to shorten the length of the target pattern; however, this should increase the FPR. Therefore, new methods for BH detection, such as threshold-free methods [52] and dynamic-stopping approaches [53], should be considered in future studies.
Respiration signals can also be measured using a respiration belt as shown in Figure 2, where the average amplitude of respiration signals recorded from a respiration belt was larger than that from a PPG sensor. Although we might expect higher classification accuracy by employing a respiration belt instead of a PPG sensor, we employed the PPG sensor in this study because the PPG sensor is more convenient to use and easier to wear than the respiration belt. Indeed, it was difficult and often risky to put on the respiration belt to the ALS patients who were bed-ridden with mechanical ventilation. In addition, since most patients with ALS have severe respiratory muscle weakness, it is possible that the breathing belt can press the patient’s chest and disturb their normal respiration.

5. Conclusions

We developed a respiration-modulated PPG switch to implement an asynchronous BCI with a low FPR. Offline and online experimental results confirmed that the proposed PPG-based BCI toggle switch has a very low FPR of 0.02 FPs/min, indicating that a false operation occurs every 50 min. Additionally, all the participants in the experiments could successfully control four different environment devices by using the SSVEP-BCI incorporated with the PPG switch. It is expected that the proposed PPG-based BCI toggle switch can be used to implement asynchronous BCIs that can be employed in practical scenarios.

Author Contributions

C.-H.H. performed the experiments, analyzed the data, and wrote the paper; E.K. conducted the experiments; C.-H.I. supervised the overall study and revised the paper. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported in part by an Institute of Information & Communications Technology Planning & Evaluation grant funded by the Korean government (MSIT) (2017-0-00432, Development of non-invasive integrated BCI SW platform to control home appliances and external devices by user’s thought via AR/VR interface) and in part by the National Research Foundation of Korea (NRF) grants funded by the Ministry of Science and ICT (NRF-2019R1A2C2086593).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Wolpaw, J.R.; Birbaumer, N.; McFarland, D.J.; Pfurtscheller, G.; Vaughan, T.M. Brain–computer interfaces for communication and control. Clin. Neurophysiol. 2002, 113, 767–791. [Google Scholar] [CrossRef]
  2. Wolpaw, J.R.; McFarland, D.J.; Neat, G.W.; Forneris, C.A. An EEG-based brain-computer interface for cursor control. Electroencephalogr. Clin. Neurophysiol. 1991, 78, 252–259. [Google Scholar] [CrossRef]
  3. Guger, C.; Daban, S.; Sellers, E.; Holzner, C.; Krausz, G.; Carabalona, R.; Gramatica, F.; Edlinger, G. How many people are able to control a P300-based brain–computer interface (BCI)? Neurosci. Lett. 2009, 462, 94–98. [Google Scholar] [CrossRef] [PubMed]
  4. Hwang, H.J.; Lim, J.H.; Jung, Y.J.; Choi, H.; Lee, S.W.; Im, C.H. Development of an SSVEP-based BCI spelling system adopting a QWERTY-style LED keyboard. J. Neurosci. Methods 2012, 208, 59–65. [Google Scholar] [CrossRef] [PubMed]
  5. Hong, B.; Guo, F.; Liu, T.; Gao, X.; Gao, S. N200-speller using motion-onset visual response. Clin. Neurophysiol. 2009, 120, 1658–1666. [Google Scholar] [CrossRef] [PubMed]
  6. Han, C.H.; Kim, Y.W.; Kim, S.H.; Nenadic, Z.; Im, C.H. Electroencephalography-based endogenous brain–computer interface for online communication with a completely locked-in patient. J. Neuroeng. Rehabil. 2019, 16, 18. [Google Scholar] [CrossRef] [PubMed]
  7. Mellinger, J.; Schalk, G.; Braun, C.; Preissl, H.; Rosenstiel, W.; Birbaumer, N.; Kübler, A. An MEG-based brain–computer interface (BCI). NeuroImage 2007, 36, 581–593. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  8. Hramov, A.E.; Pitsik, E.N.; Chholak, P.; Maksimenko, V.A.; Frolov, N.S.; Kurkin, S.A.; Pisarchik, A.N. A MEG Study of Different Motor Imagery Modes in Untrained Subjects for BCI Applications. In Proceedings of the 16th International Conference on Informatics in Control, Automation and Robotics, Prague, Czech Republic, 29–31 July 2019. [Google Scholar]
  9. Naseer, N.; Hong, K.S. fNIRS-based brain-computer interfaces: A review. Front. Hum. Neurosci. 2015, 9, 3. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  10. Weiskopf, N.; Mathiak, K.; Bock, S.W.; Scharnowski, F.; Veit, R.; Grodd, W.; Goebel, R.; Birbaumer, N. Principles of a brain-computer interface (BCI) based on real-time functional magnetic resonance imaging (fMRI). IEEE Trans. Biomed. Eng. 2004, 51, 966–970. [Google Scholar] [CrossRef]
  11. Ramsey, N.F.; van de Heuvel, M.P.; Kho, K.H.; Leijten, F.S. Towards human BCI applications based on cognitive brain systems: An investigation of neural signals recorded from the dorsolateral prefrontal cortex. IEEE Trans. Neural Syst. Rehabil. Eng. 2006, 14, 214–217. [Google Scholar] [CrossRef]
  12. Valente, G.; Kaas, A.L.; Formisano, E.; Goebel, R. Optimizing fMRI experimental design for MVPA-based BCI control: Combining the strengths of block and event-related designs. NeuroImage 2019, 186, 369–381. [Google Scholar] [CrossRef] [PubMed]
  13. Min, B.K.; Marzelli, M.J.; Yoo, S.S. Neuroimaging-based approaches in the brain–computer interface. Trends Biotechnol. 2010, 28, 552–560. [Google Scholar] [CrossRef] [PubMed]
  14. Casson, A.J. Wearable EEG and beyond. Biomed. Eng. Lett. 2019, 9, 53–71. [Google Scholar] [CrossRef] [PubMed]
  15. Allison, B.Z.; McFarland, D.J.; Schalk, G.; Zheng, S.D.; Jackson, M.M.; Wolpaw, J.R. Towards an independent brain–computer interface using steady state visual evoked potentials. Clin. Neurophysiol. 2008, 119, 399–408. [Google Scholar] [CrossRef] [Green Version]
  16. Hwang, H.J.; Kim, D.H.; Han, C.H.; Im, C.H. A new dual-frequency stimulation method to increase the number of visual stimuli for multi-class SSVEP-based brain–computer interface (BCI). Brain Res. 2013, 1515, 66–77. [Google Scholar] [CrossRef]
  17. Bin, G.; Gao, X.; Yan, Z.; Hong, B.; Gao, S. An online multi-channel SSVEP-based brain–computer interface using a canonical correlation analysis method. J. Neural Eng. 2009, 6, 046002. [Google Scholar] [CrossRef]
  18. Jia, C.; Gao, X.; Hong, B.; Gao, S. Frequency and phase mixed coding in SSVEP-based brain--computer interface. IEEE Trans. Biomed. Eng. 2011, 58, 200–206. [Google Scholar]
  19. Choi, G.Y.; Han, C.H.; Jung, Y.J.; Hwang, H.J. A multi-day and multi-band dataset for a steady-state visual-evoked potential–based brain-computer interface. GigaScience 2019, 8, giz133. [Google Scholar] [CrossRef]
  20. Kim, D.W.; Hwang, H.J.; Lim, J.H.; Lee, Y.H.; Jung, K.Y.; Im, C.H. Classification of selective attention to auditory stimuli: Toward vision-free brain–computer interfacing. J. Neurosci. Methods 2011, 197, 180–185. [Google Scholar] [CrossRef]
  21. Hill, N.; Schölkopf, B. An online brain–computer interface based on shifting attention to concurrent streams of auditory stimuli. J. Neural Eng. 2012, 9, 026011. [Google Scholar] [CrossRef] [Green Version]
  22. Sellers, E.W.; Krusienski, D.J.; McFarland, D.J.; Vaughan, T.M.; Wolpaw, J.R. A P300 event-related potential brain–computer interface (BCI): The effects of matrix size and inter stimulus interval on performance. Biol. Psychol. 2006, 73, 242–252. [Google Scholar] [CrossRef] [PubMed]
  23. Kübler, A.; Furdea, A.; Halder, S.; Hammer, E.M.; Nijboer, F.; Kotchoubey, B. A brain–computer interface controlled auditory event-related potential (P300) spelling system for locked-in patients. Ann. N. Y. Acad. Sci. 2009, 1157, 90–100. [Google Scholar] [CrossRef] [PubMed]
  24. Lian, J.; Bi, L.; Fei, W. A Novel Event-Related Potential-Based Brain–Computer Interface for Continuously Controlling Dynamic Systems. IEEE Access 2019, 7, 38721–38729. [Google Scholar] [CrossRef]
  25. Kübler, A.; Neumann, N.; Kaiser, J.; Kotchoubey, B.; Hinterberger, T.; Birbaumer, N.P. Brain-computer communication: Self-regulation of slow cortical potentials for verbal communication. Arch. Phys. Med. Rehabil. 2001, 82, 1533–1539. [Google Scholar] [CrossRef]
  26. Pfurtscheller, G.; Neuper, C. Motor imagery and direct brain-computer communication. Proc. IEEE 2001, 89, 1123–1134. [Google Scholar] [CrossRef]
  27. Sannelli, C.; Vidaurre, C.; Müller, K.R.; Blankertz, B. A large scale screening study with a SMR-based BCI: Categorization of BCI users and differences in their SMR activity. PLoS ONE 2019, 14, e0207351. [Google Scholar] [CrossRef] [Green Version]
  28. Aloise, F.; Schettini, F.; Aricò, P.; Leotta, F.; Salinari, S.; Mattia, D.; Babiloni, F.; Cincotti, F. P300-based brain–computer interface for environmental control: An asynchronous approach. J. Neural Eng. 2011, 8, 025025. [Google Scholar] [CrossRef]
  29. Martínez-Cagigal, V.; Gomez-Pilar, J.; Álvarez, D.; Hornero, R. An Asynchronous P300-Based Brain-Computer Interface Web Browser for Severely Disabled People. IEEE Trans. Neural Syst. Rehabil. Eng. 2017, 25, 1332–1342. [Google Scholar] [CrossRef]
  30. Ortner, R.; Allison, B.Z.; Korisek, G.; Gaggl, H.; Pfurtscheller, G. An SSVEP BCI to control a hand orthosis for persons with tetraplegia. IEEE Trans. Neural Syst. Rehabil. Eng. 2011, 19, 1–5. [Google Scholar] [CrossRef]
  31. Townsend, G.; Graimann, B.; Pfurtscheller, G. Continuous EEG classification during motor imagery-simulation of an asynchronous BCI. IEEE Trans. Neural Syst. Rehabil. Eng. 2004, 12, 258–265. [Google Scholar] [CrossRef]
  32. Lim, J.H.; Kim, Y.W.; Lee, J.H.; An, K.O.; Hwang, H.J.; Cha, H.S.; Han, C.H.; Im, C.H. An emergency call system for patients in locked-in state using an SSVEP-based brain switch. Psychophysiology 2017, 54, 1632–1643. [Google Scholar] [CrossRef] [PubMed]
  33. Pan, J.; Li, Y.; Zhang, R.; Gu, Z.; Li, F. Discrimination between control and idle states in asynchronous SSVEP-based brain switches: A pseudo-key-based approach. IEEE Trans. Neural Syst. Rehabil. Eng. 2013, 21, 435–443. [Google Scholar] [PubMed] [Green Version]
  34. Borisoff, J.F.; Mason, S.G.; Bashashati, A.; Birch, G.E. Brain-computer interface design for asynchronous control applications: Improvements to the LF-ASD asynchronous brain switch. IEEE Trans. Biomed. Eng. 2004, 51, 985–992. [Google Scholar] [CrossRef] [PubMed]
  35. Müller-Putz, G.R.; Kaiser, V.; Solis-Escalante, T.; Pfurtscheller, G. Fast set-up asynchronous brain-switch based on detection of foot motor imagery in 1-channel EEG. Med. Biol. Eng. Comput. 2010, 48, 229–233. [Google Scholar] [CrossRef] [PubMed]
  36. Li, Y.; He, S.; Huang, Q.; Gu, Z.; Yu, Z.L. A EOG-based switch and its application for start/stop control of a wheelchair. Neurocomputing 2018, 275, 1350–1357. [Google Scholar] [CrossRef]
  37. Wang, H.; Li, Y.; Long, J.; Yu, T.; Gu, Z. An asynchronous wheelchair control by hybrid EEG–EOG brain–computer interface. Cogn. Neurodyn. 2014, 8, 399–409. [Google Scholar] [CrossRef]
  38. Lim, J.H.; Hwang, H.J.; Han, C.H.; Jung, K.Y.; Im, C.H. Classification of binary intentions for individuals with impaired oculomotor function: ‘Eyes-closed’SSVEP-based brain–computer interface (BCI). J. Neural Eng. 2013, 10, 026021. [Google Scholar] [CrossRef]
  39. Hwang, H.J.; Han, C.H.; Lim, J.H.; Kim, Y.W.; Choi, S.I.; An, K.O.; Lee, J.H.; Cha, H.S.; Kim, S.H.; Im, C.H. Clinical feasibility of brain-computer interface based on steady-state visual evoked potential in patients with locked-in syndrome: Case studies. Psychophysiology 2017, 54, 444–451. [Google Scholar] [CrossRef]
  40. Kim, H.; Kim, J.Y.; Im, C.H. Fast and robust real-time estimation of respiratory rate from photoplethysmography. Sensors 2016, 16, 1494. [Google Scholar] [CrossRef] [Green Version]
  41. Barrett, K.E. Ganong’s Review of Medical Physiology; In McGraw-Hill Medical: New York, NY, USA, 2010. [Google Scholar]
  42. Lindh, W.Q.; Pooler, M.; Tamparo, C.D.; Dahl, B.M.; Morris, J. Delmar’s Comprehensive Medical Assisting: Administrative and Clinical Competencies; Cengage Learning: Boston, MA, USA, 2013. [Google Scholar]
  43. Lotte, F.; Congedo, M.; Lécuyer, A.; Lamarche, F.; Arnaldi, B. A review of classification algorithms for EEG-based brain–computer interfaces. J. Neural Eng. 2007, 4, R1. [Google Scholar] [CrossRef]
  44. Nilsson, L.; Goscinski, T.; Kalman, S.; Lindberg, L.G.; Johansson, A. Combined photoplethysmographic monitoring of respiration rate and pulse: A comparison between different measurement sites in spontaneously breathing subjects. Acta Anaesthesiol. Scand. 2007, 51, 1250–1257. [Google Scholar] [CrossRef] [PubMed]
  45. Müller-Putz, G.; Scherer, R.; Brunner, C.; Leeb, R.; Pfurtscheller, G. Better than random: A closer look on BCI results. Int. J. Bioelectromagn. 2008, 10, 52–55. [Google Scholar]
  46. Zhang, H.; Guan, C.; Wang, C. Asynchronous P300-based brain-computer interfaces: A computational approach with statistical models. IEEE Trans. Biomed. Eng. 2008, 55, 1754–1763. [Google Scholar] [CrossRef]
  47. Xu, R.; Jiang, N.; Dosen, S.; Lin, C.; Mrachacz-Kersting, N.; Dremstrup, K.; Farina, D. Endogenous sensory discrimination and selection by a fast brain switch for a high transfer rate brain-computer interface. IEEE Trans. Neural Syst. Rehabil. Eng. 2016, 24, 901–910. [Google Scholar] [CrossRef]
  48. Li, Y.; Pan, J.; Wang, F.; Yu, Z. A hybrid BCI system combining P300 and SSVEP and its application to wheelchair control. IEEE Trans. Biomed. Eng. 2013, 60, 3156–3166. [Google Scholar]
  49. Peng, N.; Zhang, R.; Zeng, H.; Wang, F.; Li, K.; Li, Y.; Zhuang, X. Control of a nursing bed based on a hybrid brain-computer interface. In Proceedings of the 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA, 16–20 August 2016; pp. 1556–1559. [Google Scholar]
  50. Radunovic, A.; Annane, D.; Jewitt, K.; Mustfa, N. Mechanical ventilation for amyotrophic lateral sclerosis/motor neuron disease. Cochrane Database Syst. Rev. 2009. [Google Scholar] [CrossRef] [Green Version]
  51. Cedarbaum, J.M.; Stambler, N.; Malta, E.; Fuller, C.; Hilt, D.; Thurmond, B.; Nakanishi, A.; Bdnf Als Study Group. The ALSFRS-R: A revised ALS functional rating scale that incorporates assessments of respiratory function. J. Neurol Sci. 1999, 169, 13–21. [Google Scholar] [CrossRef]
  52. He, S.; Zhang, R.; Wang, Q.; Chen, Y.; Yang, T.; Feng, Z.; Zhang, Y.; Shao, M.; Li, Y. A P300-based threshold-free brain switch and its application in wheelchair control. IEEE Trans. Neural Syst. Rehabil. Eng. 2016, 25, 715–725. [Google Scholar] [CrossRef]
  53. Schreuder, M.; Höhne, J.; Blankertz, B.; Haufe, S.; Dickhaus, T.; Tangermann, M. Optimizing event-related potential based brain–computer interfaces: A systematic evaluation of dynamic stopping methods. J. Neural Eng. 2013, 10, 036025. [Google Scholar] [CrossRef] [Green Version]
Sensors 20 00348 g001 550
Figure 1. Schematics of the offline and online experiments: (a) offline experiment; (b) training session in the online experiment; (c) test session in the online experiment. The red double-sided arrows represent a single trial.
Figure 1. Schematics of the offline and online experiments: (a) offline experiment; (b) training session in the online experiment; (c) test session in the online experiment. The red double-sided arrows represent a single trial.
Sensors 20 00348 g001
Sensors 20 00348 g002 550
Figure 2. Examples of signals recorded while a participant alternately performed NB and BH tasks: (a) Respiration signals (denoted by RESP); (b) PPG signals. The red and green vertical lines indicate the onset times of BH and NB, respectively.
Figure 2. Examples of signals recorded while a participant alternately performed NB and BH tasks: (a) Respiration signals (denoted by RESP); (b) PPG signals. The red and green vertical lines indicate the onset times of BH and NB, respectively.
Sensors 20 00348 g002
Sensors 20 00348 g003 550
Figure 3. Results of the paired t-test in the offline data analysis. The red dotted horizontal line indicates a p-value of 0.05. The vertical axis indicates the p-value averaged across subjects in the offline experiment.
Figure 3. Results of the paired t-test in the offline data analysis. The red dotted horizontal line indicates a p-value of 0.05. The vertical axis indicates the p-value averaged across subjects in the offline experiment.
Sensors 20 00348 g003
Sensors 20 00348 g004 550
Figure 4. Average offline classification accuracy with respect to the size of the PPG data. The red horizontal line indicates the level of chance.
Figure 4. Average offline classification accuracy with respect to the size of the PPG data. The red horizontal line indicates the level of chance.
Sensors 20 00348 g004
Sensors 20 00348 g005 550
Figure 5. Example of online experimental results for a participant. The vertical axis corresponds to the classification outputs, which are indicated by the blue vertical lines. The red and green vertical lines indicate the onset times for BH and NB, respectively.
Figure 5. Example of online experimental results for a participant. The vertical axis corresponds to the classification outputs, which are indicated by the blue vertical lines. The red and green vertical lines indicate the onset times for BH and NB, respectively.
Sensors 20 00348 g005
Sensors 20 00348 g006 550
Figure 6. Series of snapshots taken during the online experiment with a healthy participant. (a) Participant held her breath until the PPG-based BCI toggle switch was turned on. (b) Once the PPG switch was turned on, auditory feedback (exclamation of the word “on”) was provided to her. (c) Immediately thereafter, four checkerboard stimuli with different reversing frequencies were presented for 10 s. During this period, she gazed at one of these visual stimuli. (d) After the classification, auditory feedback corresponding to the classification result was provided to her, and simultaneously, one of the environment devices was turned on.
Figure 6. Series of snapshots taken during the online experiment with a healthy participant. (a) Participant held her breath until the PPG-based BCI toggle switch was turned on. (b) Once the PPG switch was turned on, auditory feedback (exclamation of the word “on”) was provided to her. (c) Immediately thereafter, four checkerboard stimuli with different reversing frequencies were presented for 10 s. During this period, she gazed at one of these visual stimuli. (d) After the classification, auditory feedback corresponding to the classification result was provided to her, and simultaneously, one of the environment devices was turned on.
Sensors 20 00348 g006
Table
Table 1. Online performance of the respiration-based PPG switch.
Table 1. Online performance of the respiration-based PPG switch.
SubTime Elapsed for Turning Switch on (s)TPR (%)FPR (FPs/min)Classification Accuracy (%)
18.03 ± 1.031000.00099.68 ± 0.47
210.17 ± 1.0601000.06099.63 ± 0.82
39.91 ± 0.911000.00099.09 ± 0.79
411.69 ± 4.2101000.00100.00 ± 0.00
510.41 ± 2.0201000.00099.77 ± 0.52
612.29 ± 3.5201000.00100.00 ± 0.00
711.49 ± 3.9301000.06099.61 ± 0.87
AVG10.57 ± 2.3801000.02099.68 ± 0.50
Table
Table 2. Online experimental results for the SSVEP-based BCI.
Table 2. Online experimental results for the SSVEP-based BCI.
SubjectOrder of TargetsClassification ResultsAccuracy (%)
113,12413,124100
213,12413,14480
321,32421,32280
421,32421,34480
521,32421,324100
631,24331,24480
731,24331,243100
AVG--88.57

Share and Cite

MDPI and ACS Style

Han, C.-H.; Kim, E.; Im, C.-H. Development of a Brain–Computer Interface Toggle Switch with Low False-Positive Rate Using Respiration-Modulated Photoplethysmography. Sensors 2020, 20, 348. https://doi.org/10.3390/s20020348

AMA Style

Han C-H, Kim E, Im C-H. Development of a Brain–Computer Interface Toggle Switch with Low False-Positive Rate Using Respiration-Modulated Photoplethysmography. Sensors. 2020; 20(2):348. https://doi.org/10.3390/s20020348

Chicago/Turabian Style

Han, Chang-Hee, Euijin Kim, and Chang-Hwan Im. 2020. "Development of a Brain–Computer Interface Toggle Switch with Low False-Positive Rate Using Respiration-Modulated Photoplethysmography" Sensors 20, no. 2: 348. https://doi.org/10.3390/s20020348

APA Style

Han, C. -H., Kim, E., & Im, C. -H. (2020). Development of a Brain–Computer Interface Toggle Switch with Low False-Positive Rate Using Respiration-Modulated Photoplethysmography. Sensors, 20(2), 348. https://doi.org/10.3390/s20020348

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop