Functional Mapping of the Brain for Brain–Computer Interfacing: A Review

Abstract

Brain–computer interfacing has been applied in a range of domains including rehabilitation, neuro-prosthetics, and neurofeedback. Neuroimaging techniques provide insight into the structural and functional aspects of the brain. There is a need to identify, map and understand the various structural areas of the brain together with their functionally active roles for the accurate and efficient design of a brain–computer interface. In this review, the functionally active areas of the brain are reviewed by analyzing the research available in the literature on brain–computer interfacing in conjunction with neuroimaging experiments. This review first provides an overview of various approaches of brain–computer interfacing and basic components in the BCI system and then discuss active functional areas of the brain being utilized in non-invasive brain–computer interfacing performed with hemodynamic signals and electrophysiological recording-based signals. This paper also discusses various challenges and limitations in BCI becoming accessible to a novice user, including security issues in the BCI system, effective ways to overcome those issues, and design implementations.

1. Introduction

In modern brain–computer interface (BCI) design, the BCI is a direct link between the brain of a person and a computer [1]. More precisely, BCI is a system that measures and processes the nervous system activity used by a computer to restore and/or replace the natural output of the brain [2]. For example, BCI can be used to analyze the behavior of the damaged part of the brain, or it can be used to drive and analyze the brain’s outputs that normally are not available to the computer. These could be the person’s moods, voluntary motor actions, and other similar applications. Probably the first BCI was the polygraph (invented in 1921 and popularly known as the lie detector) in which measurements such as blood pressure, pulse respiration, skin conductivity, pupil dilation, respiration, et cetera, are taken by different systems from all over the body [3].

The billions of neurons engaged in different cognitive and motor functions in the brain consume the oxygen in the brain, leading to different electromagnetic signatures for different brain activities. This brain activity can be measured using sensors based on different technologies measuring different quantities in the brain. For example, electroencephalography (EEG) [4] measures the electrical potentials at the scalp and produces readings in the form of irregular traces obtained from multiple sensors. Mere visual inspection of these data is insufficient for conclusions, and therefore computational methods are needed to analyze these data. This is where the interdisciplinary fields of neuroscience and computer science unify to give rise to the BCI.

Categorization of the BCI system can be done based on two broad themes: invasiveness and agency. In terms of invasiveness, BCIs are categorized as invasive, partially invasive, and non-invasive, depending on the placement of the electrodes. In invasive BCI, electrodes are directly placed into the grey matter; therefore, implanted electrodes produce signals with the highest quality. However, these electrodes are prone to scar tissue formation that weakens the signals [5]. Further, the invasive subdural electrodes generally start failing over time, so the underlying patient may need to be operated on again. Another problem with invasive BCI is that in some medical conditions the body may not accept the electrodes or sensors. Invasive BCI is quite useful in the case of paralyzed patients. Since the electrodes/sensors can be implanted directly into the gray matter, with an appropriate BCI system the patient can control a prosthetic arm or operate a wheelchair. In partially invasive BCI, electrodes are implanted on the cortical surface but not in the cortical tissue. Signals produced by the partially invasive BCIs have a comparatively lower resolution than invasive BCI [6]. Partially invasive BCI is offered when one needs less complication, less clinical risk, and good stability. Examples include electrocorticography (ECoG), where we place a strip or grid of electrodes on the surface of the brain [7], and intracortical microelectrodes [8], which can be implanted a few millimeters into the brain where they may serve as either sensors or stimulators. In non-invasive BCIs, the brain’s electrical activity is measured by electrodes placed on the scalp. Non-invasive BCIs have found wide application because of their ease of use, with EEG being one of the most popularly used modalities.

In terms of agency, BCIs are categorized as active, reactive, and passive. In active BCIs, the user is asked to perform a mental task that has a known brain activity signature. The BCI system detects this signature and uses this to perform further processing. Examples of active BCIs include motor imagery for body part movement [9], commanding a robotic arm for assistance while drinking [10], driving a wheelchair [11], et cetera. In reactive BCI systems, the user’s brain activity is altered by an external stimulus that is provided by the BCI system. This alteration in brain activity is used to decode the user’s response and perform actions on their behalf. The most popular example is the P300 paradigm, where the user is shown letters or symbols on a screen in quick succession, and the user directs their attention to the symbol they want to select. The P300 paradigm is useful for controlling a TV [12], making art [13], and for use as tactile sensors for motion control [14]. Passive BCIs simply measure the brain activity of the user without any goal or task. This has applications in monitoring the attention of the users to avoid dangerous situations in workplaces [15].

Recent advances in neuroimaging technologies have also led to the better functional anatomical mapping of brain regions [16]. This has helped for understand of the brain anatomical regions that should be taken into consideration in different types of BCI systems. While historical invasive brain studies facilitated understanding of the coarse-level functions of anatomical brain regions, noninvasive neuroimaging techniques can help map functions to brain regions more precisely at an individual subject level [17]. This can be useful for the placement of electrode sensors in both invasive and non-invasive systems. Neuroimaging techniques can especially be useful for mapping different functions to anatomical regions in the eloquent cortex at the subject level.

2. Materials and Methods

The ultimate aim of this review is to explore the active areas of the brain in BCI while using fMRI, fNIRS, EEG, and MEG systems. For the literature search, we searched three databases, namely, Scopus, PubMed, and Google Scholar. We started exploring the literature related to basic components in a typical BCI system. Then we searched the title and abstract with different combinations of active areas/ targeted areas keywords with fMRI, fNIRS, EEG, and MEG. In total, by searching keywords in the title and abstract, we found 2439 records. Out of these, 1203 duplicate records were excluded from the study. For the remaining 1236 records, we studied the abstracts and excluded 989 records, since they were unrelated. After thoroughly studying all articles, 989 records were excluded because of full-text unavailability or unclear explanation. The remaining 247 full-text articles were accessed for eligibility, and a further 112 of the 247 articles whose results were either similar to other articles or had unreliable results were also removed. Finally, 135 articles that completely followed the PRISMA guidelines and met all the criteria were reviewed. The details about the inclusion and exclusion criteria according to Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines are shown in Figure 1.

3. Basic Components in a Typical BCI System

A BCI connects the brain signals of the organism to a digital system that is further connected to external devices. Thereafter, the BCI can be used to control external devices using brain signals. Since the BCI works in almost real-time, it can be used for repairing, enhancing, or replacing cognitive or sensory–motor functions [19]. It can also be used as a feedback signal so that a person can change his/her brain activation. For example, moving a hand to lift an object in which the neurological signals are read out directly from the brain converts these signals into a command, using some interpretation and decrypting, and transmits these commands through the spinal cord to move the prosthetic arm at the desired location [20]. Therefore, if the spinal cord somehow is injured, it could be replaced by a BCI [21]. We can tap electrical signals from the brain using electrodes and after some pre-processing and mathematical algorithms, deliver those conditioned signals to the robotic arm to do a specific task (Figure 2).

Since a BCI is multidisciplinary in nature, the interfacing of neural activity to computer systems requires knowledge of signal processing, computer science, machine learning, and mechanical and electrical engineering (Figure 3). It takes input signals from the targeted areas of brain tissues using different types of sensors. These sensors can be invasive (i.e., direct brain implants), partially invasive (ECoG), or non-invasive (fMRI, EEG, MEG, fNIRS, etc.) [22]. The raw data collected with these sensors are noisy, include stochastic neural activity, and require several signal processing steps such as filtering and amplification before further processing. This involves understanding the sources and nature of artifacts in the brain signals so that they can be appropriately identified and removed. Feature extraction and selection are performed on the processed data before further computation. Thereafter, techniques from machine learning, computational intelligence, statistical optimization, and information theory are applied to the extracted feature set [23,24]. BCIs can use machine learning algorithms to find specific pattern representations. In the case of EEG signals, we analyze small chunks for learning specific patterns and label them based on the mental state of the subject. In the following sections, we will discuss these steps in detail with current progress in the literature.

3.1. Data Acquisition and Pre-Processing

Data acquired using different modalities, viz., fMRI, fNIRS, EEG, and MEG, contain several noise artifacts in their raw form. The most obvious noise added to the raw signal are from physiological factors associated with cardiac signals, breathing, ocular signals, muscle contractions, et cetera. Windowing techniques where frequency filtering is performed to remove frequency components related to physiological noise can be used. Principal component analysis (PCA) is an effective method for filtering physiological noise by creating a new dimension of the raw data and identifying dimensions that are responsible for the most variations. Therefore, it is easy to discard the variability due to those dimensions.

Additionally, raw data also contain non-physiological contributions, which are generally derived from factors during data acquisition. Non-physiological contributions include motion artifacts and measurement noise (i.e., random noise from imperfect measurement, or white noise). Although non-physiological noise can be reduced at the time of BCI data acquisition, it cannot be eliminated [25,26]. During data acquisition, these contaminations can be avoided by making proper acquisition arrangements [27,28]. During the pre-processing, we can use frequency filtering to remove any machine drift. We can use PCA, spline, wavelet filtering, or correlation-based signal improvement (CBSI) to remove motion artifacts.

3.2. Feature Extraction

After processing raw signals, a typical BCI system extracts features from the processed signals. This creates a low-dimensional latent representation of the original data. Usually, features included for further processing involve temporal, spatial, or both components. Temporal features are generally extracted after band-pass/low-pass filtering. Commonly used temporal features include spike detection [29,30,31], wavelet transforms [32,33,34,35,36], bandpass filtering techniques [37,38,39,40,41], Kalman filtering [42,43,44,45,46], Fourier transform [47,48,49,50], and template matching [51,52]. A time series signal can be a good choice of features with correct temporal resolution. The magnitude of the Fourier transform can be a good choice of features with perfect spectral resolution. However, the temporal resolution is lost with the Fourier transform magnitude-based features, and this can be overcome by using the wavelet transform. Since EEG signals are non-stationary, the wavelet transform may be a good choice in the time–frequency domain as a spectral approximation. The disadvantage associated with wavelets is the need for selecting the right principal wavelet.

3.3. Classification

Early research on classifying brain states from extracted brain signals used traditional machine learning techniques such as logistic regression [53,54], support vector machines, clustering (unsupervised), feed-forward networks [55,56], decision trees, and so on [57]. Unsupervised machine learning techniques were used to find hidden structures in the given dataset to gain a deeper understanding and draw inferences from the given dataset.

The neural network is one of the most preferred topics within this research area when it comes to the classification task. Over the years, there has been a lot of progress in the field of neural networks, and today we see a lot of neural networks in the military, medical, imaging, time-series analysis, and video processing fields, and even in the field of BCI. The word “neuro” in the neural network is derived from the biological neuron, and the network represents the connectivity of these neurons. An artificial neural network is a modest attempt to mimic the human brain [58]. A neural network may comprise two or more layers, i.e., the input layer, one or more hidden layers (in the case of multi-layer neural networks), and the output layers. The NN takes the input feature vector, $X^T$, at the input layer (e.g., raw brain signal in BCI system), and optimizes certain parameters, $\theta$, to map the desired output, $y$, corresponding to the inferred command, e.g., moving the hand, moving the wheelchair in a specified direction, etc. The mapping between input $X^T$ and the desired output $y$ is given as $y=f\left(X^T,\theta \right)$, where $X^T={ℝ}^{C\times N}$, $C$ is the number of channels, and $N$ is the number of samples extracted from the electrodes (after desired pre-processing). The output $y$ then goes to the device, which needs to be controlled, and the BCI may be fed back in a loop if the given/targeted device needs to be in control recursively.

Although the artificial Neural Network is a modest attempt to mimic the human brain, with advancements in machine learning techniques, however, such as convolutional neural networks (CNNs) [59] and recurrent neural networks (RNNs), more advanced filters for capturing spatiotemporal features have been proposed and applied [57]. With the advancements in GPU-based high-performance computing systems, deep neural networks are extensively being researched for a BCI system. Broadly, the deep learning algorithms can be classified into four different categories as follows [60]: (1) Discriminative deep learning models: These models optimize the learning parameters using a probabilistic approach and non-linear transformation for the predefined label. The discriminative models may include deep learning architectures such as multilayer perceptron [61], CNN, and RNN. (2) Representative deep learning models: The learning parameters are optimized from the pure and representative input features. These architectures have only feature engineering ability and often fail in the classification task. Some examples of these algorithms are autoencoder and deep belief networks. (3) Generative deep learning models: These types of models have powerful methods to learn any kind of input features in an unsupervised manner. The two most powerful are variational encoders and generative adversarial networks. (4) Hybrid deep learning models: these models simply combine two or more deep learning models [62]. Multiple deep learning architectures can be used as ensemble learning, or different models can be used for different tasks, such as a representative model for feature engineering and a discriminative model for classification purposes.

4. Active Areas of the Brain as a Target for BCI

Multiple neuroimaging studies have been conducted to understand the functional and anatomical mapping of the brain. Although the initial efforts in this direction were invasive, recent efforts leveraged advances in neuroimaging techniques to understand the functional association of different brain regions. For example, with functional MRI, subjects are either asked to remain at rest in the scanner or are asked to perform predefined tasks. Thereafter, the activations/deactivations in the brain are studied to correlate them with rest or task performance [63].

The brain is broadly divided into four main parts, i.e., the cortex, the subcortex, the brainstem, and the cerebellum [64,65]. The cerebral cortex consists of white matter and grey matter and is responsible for the integration and processing of sensory inputs. The cerebrum is made up of a complex network of neurons and is commonly subdivided into the frontal lobe, parietal lobe, occipital lobe, and temporal lobe. The frontal lobe is known for its cognitive and executive decision-making. The frontal lobe also encodes experiential memory and is one of the most functionally variable regions across subjects. The parietal lobe processes and reacts to sensory inputs such as touch, temperature, body position (proprioception), and pain (nociception) [66,67]. The occipital lobe is known for visual and spatial perception. The temporal lobe is associated with functions related to auditory processing, language recognition, and speech [67,68].

Most of the brain tissues are connected to the rest of the body via a distal brain structure called the brainstem. It contains three structures, i.e., the midbrain, the pons, and the medulla oblongata. The brainstem facilitates communication between the cerebrum, cerebellum, and spinal cord. This makes the brainstem an important part of the brain, as communication between the sensory–motor, sensory cortex, and the peripheral nervous system occurs through it. The brainstem is also key for the regulation of cardiac and respiration functions, consciousness, and the sleep cycle.

The cerebellum, or the popularly called “little brain”, is located in the posterior inferior region of the brain adjoining the occipital and temporal lobes. Although it accounts for approximately 10% of the brain’s volume, it contains over 50% of the total number of neurons in the brain. The cerebellum is involved in the maintenance of balance and posture, the coordination of voluntary movements, and the adapting and fine-tuning of motor movements over time.

The subcortical structures comprise a group of diverse neural formations deep within the brain that are located between the cerebral cortex and the brainstem. Subcortical structures include the diencephalon, pituitary gland, limbic structures, and basal ganglia. The diencephalon is known for relaying the auditory and optic signals (thalamus) between the brain stem and the cerebrum, maintaining sleep cycles (pineal gland) and endocrine and autonomic functions (hypothalamus). The pituitary gland is involved in the secretion of the thyroid-stimulating hormone and the storage and release of oxytocin and vasopressin. The limbic system controls olfaction, memory, emotions, and body homeostasis. The basal ganglia perform functions related to voluntary and involuntary motor movement.

From the above overview of the functional specialization of different brain anatomical areas, it can be inferred that the motor sensory pathway is not limited to a particular brain area but spans the different regions of the body. There exist three main clinically relevant tracts for motor (corticospinal tracts) and somatosensory (spinothalamic tracts and posterior column pathways) functions for the body, spanning the brain, brainstem, and spinal cord.

Motor imagery refers to a dynamic state of body movements during which the individual imagines body part movement but no actual physical activity [69]. By measuring the value of $\mu$ and $\beta$ rhythms of EEG signals during the motor imagery task being performed, motor imagery can be broadly classified into two categories, namely, (1) event-related desynchronization (ERD), where values of $\mu$ and $\beta$ rhythms reduce considerably, and (2) event-related synchronization (ERS), where the values of $\mu$ and $\beta$ rhythms are higher than particular threshold values [70]. By observing the ERD/ERS signals associated with spontaneous EEG signals, one can analyze the intention of the subject under investigation. The signals related to motor imagery tasks usually develop at the sensory–motor cortex, which specifically includes $C_3$, $C_4$, and $C_z$ electrodes. Whenever the subject/user imagines movement of any body part or activity/task, signals regularly involved in these electrodes are quite different from each other. For example, the ERS of $C_3$, $C_4$, and $C_z$ electrodes are more obvious in the case of tongue movements; the ERS of $C_3$ and $C_4$ and ERD of $C_z$ are most obvious for foot movement; the ERS of $C_3$ and ERD of $C_4$ are most obvious for left-hand movement; and the ERD of $C_3$ and ERS of $C_4$ are the most obvious for right-hand movements [70].

Audrey et al. [71] proposed an intelligent strategy to control a virtual helicopter in 3D space using a 64-channel EEG scalp recording method. For motor imagery feature extraction, electrodes were limited to the sensorimotor cortex, and the autoregressive amplitude was measured from $C_4,C_6$, and $C{P}_4$ for the right movement and $C_3,C_1$, $C{P}_5$ for left movements for all the points in 3D physical space. LaFleur et al. [72] modulated sensorimotor rhythms of five human subjects to control an AR drone in 3D space. Divided electric signals for hand movements using motor imagery were generated from a 64-channel EEG cap. The author showed that the individual subject was able to acquire the target with about 90% accuracy while traveling at a speed of 0.69 m per second in a straight direction. The success rate of motor imagery signals is often limited, which can be improved by certain recent feature extraction techniques [73].

Steady-state visually evoked potential (SSVEP) BCI systems can be set up with minimum training time, high performance, and comparatively lower bit error rate (5.32 bits/s) [74]. SSVEP is a periodic evoked potential that is induced when the human visual system is stimulated by repetitive flickering at a specific frequency rate (typically >6 Hz). The brain reacts to these activities at a similar frequency in the brain cortex region. When the subject under the trial observes multiple flicking at different frequencies, it becomes possible to locate the target by generating multiple mental commands. These commands are being used to steer the drone and make it possible to create an SSVEP-evoked BCI system for drone control [75]. However, such brain activity signals are much more prone to various internal and external noises; therefore, proper signal preprocessing is required before sending the commands to control the drone. In addition, the complete setup for SSVEP-evoked BCI requires a bulky setup (LED/CRT screen for displaying the stimulus). Over the last few years, BCI-based drone control has become a great area of research, where researchers are actively involved in developing such technologies.

The SSVEP BCI system is frequently used in augmented and virtual reality. For example, in [18], the authors evoked SSVEP by flickering the body parts, and the used experimental procedure is shown in Figure 4. In [76], Wang et al. proposed SSVEP-based BCI for navigating a quadcopter in 3D physical space. EEG signals were extracted using 14 electrodes located at $P{O}_3,P{O}_4{O}_1$, and $O_2$. In [77], Chen et al. presented a real-time SSVEP BCI for controlling the air swimmer drone. SSVEP features were extracted from the occipital region using the CCA method by visual stimulation using LCD with a 60 Hz refresh rate. The state-of-the-art BCI suffers from several serious issues such as low transfer bit rate, low accuracy, and low user acceptability. In the context of eliminating these issues, researchers are also moving towards multimodal BCIs. In [78], the authors combined FFDEP and added motor imagery features for controlling the quadcopter in space. Motor imagery features were extracted using the complete information common spatial (CICP) patterns and were used to fly the quadcopter in the left and right direction. SSVEP features were extracted using the canonical correlation analysis (CCA) technique and were used to move the quadcopter in the up and down directions.

5. Functional Map of Brain for BCI with Different Neuroimaging Modalities

5.1. Hemodynamic Methods Based BCI (fMRI and fNIRS)

5.1.1. fMRI Based BCI

fMRI-BCI allows us to measure the neuron activity across the entire brain by measuring BOLD signals with a relatively high spatial resolution (in the range of millimeters) and moderated temporal resolution (in the range of seconds) [68,79]. The big advantage of fMRI-BCI is that it helps us to measure the activity of the neurons in very specific parts of brain regions. Even though the BOLD signals are a relatively indirect measure, and not individually quantitative, there is strong evidence indicating a strong correlation between BOLD signals and electrical brain activity.

Due to its critical role in emotional regulation and inspired by the previous experimental studies, the amygdala was used as a neuro-feedback target for fMRI for targeting the abnormalities associated with major depressive disorder (MDD) [80]. Experimental results suggest an increase in left amygdala activity in healthy participants during the positive autobiographical memory recall, while the participants with MDD showed a decrease in the left amygdala during the negative autobiographical memory recall. Sarkheil et al. [81] set up real-time fMRI neurofeedback for the modulation of voluntary emotion regulation. Cognitive reassessment strategies were used to control the negative emotions of participants for 13.5 s. Participants were asked to see some images in view-regulated blocks during backward counting. Their findings suggest the amygdala is a key factor in the emotion-processing network. A recent study by Koush et al. [82] showed that individuals can learn to increase top-down connectivity from the dorsal medial frontal cortex over the amygdala. Results showed several new findings to use connectivity-based neurofeedback for increasing emotion regulation functions. It was shown in [83] that dysregulation of the amygdala may result in post-traumatic stress disorder.

Chronic pain is a frequent disease in clinical practice. The brain regions such as the ACC, prefrontal cortex, and subcortical areas are involved in pain perception [84]. A recent study suggested ACC regulation using fMRI-BCI that can significantly reduce the intensity of pain [79]. Tinnitus is a disease where a person feels a sound even though there is no presence of any physical source of the sound. There have been several attempts by animals and humans to explore the affective brain regions in tinnitus. A recent biofeedback study [85] suggested that steady feedback may be more effective in the long term for targeting the auditory cortex in tinnitus patients, while intermittent feedback may be better in short-term experiments.

Several successful experiments with fMRI-BCI have been conducted to prove that a variety of behavioral and mental functions can be used to regulate the BOLD signals in different cortical brain regions, including sensory, e.g., anterior cingulate [86,87], inferior frontal [68,88], and (pre) motor [68,89]. As an extension, translational studies also find some possibilities of fMRI-BCI in removing the pathological brain symptoms associated with various neurological and psychiatric disorders, including obesity [90,91], Parkinson’s disease [92], and major depressive disorder [93,94].

A pilot study explored the possibility of fMRI-BCI to down-regulate the BOLD signals in the anterior insula in obsessive–compulsive disorder (OCD) patients [84]. Pre- and post-training experiments were conducted on two female candidates by targeting the left anterior insula and right anterior insula. Active and neutral images were consistently provided to patients, and anxiety ratings were recorded during fMRI neuro-feedback training. Results indicated the possibility of down-regulation of the insula by decreasing the BOLD signals in fMRI neurofeedback in OCD patients. The sleeping problem can be seen in chronic insomnia disorder patients, and it is a major health problem worldwide. In [95], a real-time fMRI was used to regulate the amygdala activities in 28 patients. The voxel-based degree centrality was analyzed from the gyrus, rolandic operculum, and insula amygdala for sleep improvement in chronic insomnia disorder patients by altering intrinsic functional hubs.

5.1.2. fNIRS-Based BCI

Functional near-infrared spectroscopy (fNIRS) is a non-invasive, optical brain monitoring technique that uses near-infrared spectroscopy to measure brain activity by estimating cortical hemodynamic activity that occurs in response to neural activity. Although fNIRS, like EEG, is also portable and uses blood oxyhemoglobin and de-oxyhemoglobin concentrations, it can only measure cortical signals and fails to access deeper brain regions such as the cingulate cortex and the subcortical regions. However, fNIRS is particularly suited for BCI because the targeted regions that are related to the dorsolateral prefrontal cortex, medial prefrontal cortex, motor cortex, and occipital cortex are easily accessible to fNIRS. BCI research targeting these regions has been performed using fNIRS [96,97,98,99].

The motor cortex is responsible for all voluntary body movements and generating signals for imagery and the execution of motor tasks [100,101]. There have been several fNIRS BCI tasks using motor execution proprioceptive feedback from contracting muscles from different parts of the body such as hand lifting, finger tapping, arms lifting, et cetera. In [18], an experimental setup was designed using fNIRS for recording the hemodynamic neuronal response in the motor cortex. A systematic illustration of the experimental procedure is shown in Figure 5. Cui et al. [102] presented noise reduction methods in head motion during a blocked-design finger-tapping task on ten healthy participants. In this study, the authors investigated the correlation of oxy/deoxy-Hb in real-time situations under varying noise contaminated by head motion. In a recent study, Zhang et al. [103] experimented on fifteen participants to explore the feasibility of motor imagery and motor execution in the same motion process. The PFC and motor cortex were analyzed for the changes in oxy/deoxy-Hb during a hand extension and finger-tapping task. Using the SVM classifier, they achieved around 87% accuracy in a four-class classification task. In [104], hemodynamic responses in motor cortex regions generated by knee extension and arm lifting were analyzed using a custom eight-channel fNIRS. The hemodynamic responses from eighty healthy participants were classified using naïve Bayes classifiers. Here, PCA was used for selecting the most relevant features from hemodynamic responses and achieved 95% binary class classification accuracy.

Other exciting applications of fNIRS BCI can be found in translating thought processes from motor imagery tasks. Several studies have shown that brain activations and corresponding neural activities during motor imagery training are similar to physical motor training [98,101,105]. Motor imagery can be an add-on to physical therapy to aid the restoration of motor activity after a stroke. Chiarelli et al. [105] proposed multimodal EEG-fNIRS BCI in guided left and right-hand motor imagery tasks. Fifteen right-handed healthy subjects with a mean age of 32 years and no signs of neurological or psychiatric disease were recruited. Participants were asked to sit on the chair and then were asked to squeeze left or right-hand imagery. The motor imagery task sequence was recorded for 5 s and 10 s for rest. A deep neural network was used to classify the collected data. In another study, Erdogan et al. [106] used 48-channel fNIRS BCI using motor imagery and motor execution tasks during rest. Ten healthy subjects were recruited for the experiment. Visual clues were shown to the participants on the computer screen. Hemodynamic signals from motor imagery and motor execution tasks were recorded for various hand movements for each visual cue. Hemodynamic signals were then passed through band pass filtering followed by feature extraction from block time series. Finally, data were classified using SVM, random forest, and an artificial neural network.

Motor imagery neuro-feedback protocols are generally generated in younger healthy individuals [107]; however, stroke occurs in mostly older populations. Accordingly, several studies suggest that cognitive and functional age-related gains made in younger subjects are not transferable to older subjects [107,108]. In addition, following recent behavior-related studies, older subjects are significantly less accurate in estimating temporal and spatial features of motor imagery [109,110]. The signals from the frontal, motor, and visual areas were extracted, and then motor arithmetic and motor imagery classification was performed using the fNIRS-guided attention network [95]. In this work, the author obtained a joint 3D EEG and fNIRS feature representation from the spatially aligned 1D EEG and fNIRS signals. A maximum of 86% accuracy was achieved for the MA task.

5.2. Electrophysiological-Based BCI (EEG and MEG)

5.2.1. EEG-Based BCI

Another very common method is electroencephalography (EEG). EEG measures the ionic current fluctuations produced by the brain neurons that propagate to the scalp [111,112,113]. Therefore, by placing EEG electrodes on the scalp, electrodes pick up fluctuations in voltage that occur due to neuronal processing in the brain.

EEG electrodes are usually placed on the scalp according to the 10 to 20 international placement system. The 10 and 20 refer to the placement of electrodes at fixed distances or 10% and 20% from anatomical landmarks. This method was developed to ensure a consistent protocol between studies to allow for comparison between subjects. Electrodes are labeled with both a letter (for example F for frontal, C for central, P for parietal, T for temporal, et cetera) and a number (even for right and vice versa) according to the site at which they record the brain activity. Although EEG signals have poor spatial resolution and suffer from noise and artifacts due to their nonlinear nature [114], they have a high temporal resolution and are portable, cost-effective, and easy to use. EEG signals are also noise based on the mood swings and posture of the participants during acquisition [115].

Similar to the fMRI, EEG signals can also be categorized as evoked or exogenous and spontaneous or endogenous signals [116]. The evoked signals are dependent on the external visual, sensory, or auditory stimuli, while spontaneous signals are intrinsic and are related to the internal continuous mental activity of the subject [117]. Evoked EEG signals can be further divided into two categories: visually evoked potential (VEP) signals that are related to the visual stimulus, and event-related potential (ERP) signals that are related to the response from cognitive or sensory events. SSVEP signals are the natural responses from visual stimulation at specific frequencies (3.5–75 Hz) and are widely researched signal categories of VEP signals due to fast command input, less training time, and high accuracy. A recent study suggests that the SSVEP signal can be an alternative in the EEG BCI paradigm to conventional flicker stimulation techniques due to its flicker-free nature [118]. For ERP signals, the P300, known to be the largest EEG signal (can be as high as 20 microvolts in magnitude), which generally occurs 250 ms to 950 ms after a stimulus, is used to measure phenomena related to external stimuli.

Most BCI investigations use brain waves. Several important recent advancements have been investigated on publicly available databased [119]. Rashid et al. [120] investigated the motor imagery signals using ensembled learning. The first EEG and ECoG data were collected from the right motor cortex area, the second from EEG motor imagery, the third were related to the right hand and right foot, and the fourth were related to different MI tasks. All four datasets are publicly available. Further, the data were classified using the k-NN classifier and 99.2%, 99.3%, 99.3%, and 90.3% accuracy for datasets 1, 2, 3, and 4, respectively. In [121], two large publicly available databased, namely, MBT-42 (targeting left/right motor imagery) and Med-62 databases (motor imagery) were analyzed. Several state-of-the-art deep learning models (ShallowNet, DeepNet, ParaNet) were compared with EEGNet. It was shown that EEGNet significantly outperforms all the other models ($p<0.0001$). In [122], BCI competition IV (motor imagery, eye/wrist movements) and HGD data were used for motor imagery classification tasks. The author compared their proposed attention-based CNN with EEGNet, deep CNN, and Shallow CNN. They showed maximum values of 86.8% and 96.2% accuracy for BCI competition IV and HGD datasets, respectively.

5.2.2. MEG-Based BCI

Magnetoencephalography (MEG) is a non-invasive medical imaging technology that measures the magnetic fields produced by the brain’s electrical currents, which allows it to map the brain’s function. This is done by virtue of the flow of electric charges associated with the firing of neurons in the brain. Since the magnetic field created by the neurons are very tiny and faint, MEG uses a highly sensitive magnetism detector called a superconducting quantum interference device (SQUID). To reduce noise interference, MEG is conducted in a shielded room built with a thick layer of metal that blocks magnetic fields from the outside [123,124]. Another problem associated with using SQUIDs for MEG is that ultra-low temperature (−270 °C) is required, which is overcome with the use of liquid helium. Besides neurofeedback, MEG is used for conducting basic research into perceptual and cognitive brain processes, mapping the brain function to different parts, and localizing regions affected by pathology before surgical removal. Mellinger et al. [125] suggested the feasibility and effectiveness of a MEG-based BCI system for communication using brain activities rather than muscles. The voluntary AM of sensorimotor mu and beta rhythms was investigated. The S/N ratio was enhanced using spatial filtering. Using the feedback training on six participants, the AM signals were localized in the motor cortex area.

MEG has been used for brain–computer interfaces. In one study, a computer was set up to move a cursor on a screen when a particular kind of brain activity was detected by MEG. Stroke victims who could not move their hands learned to produce that kind of brain activity [126]. Stroke victims were able to move the cursor with their minds. When the computer was connected to a mechanical hand, the stroke victims could open and close the hand with their brain activity. Similar studies have been performed with monkeys controlling robotic arms. With this kind of technology, we can potentially restore movement to people who cannot move and communicate with people who cannot speak. We can even work out what people are seeing and thinking by measuring their brain activity. Much of this kind of research involves implants in the brain rather than MEG, but it is great when MEG can be used because it is very easy and non-invasive. However, the point is that technologies such as MEG are taking us closer and closer to Luke Skywalker’s robot hands and mind reading. In summary, magnetoencephalography works by using super-sensitive magnetism detectors called SQUIDs to measure and locate the electrical activity of the brain. It must be carried out in a shielded room with special cooling, but it is relatively easy to do, and it has great spatial and temporal resolution.

The decoding of intentions directly from the brain using BCI is also possible, and these decoded intentions can be useful in controlling external movement and can help in case of any functional loss in the brain. In an interesting work, Wittevrongel et al. [127] presented MEG-BCI-based neural interfacing. The author used an optical pumping mechanism in two visual BCI systems and claimed that the proposed scheme is highly efficient compared to the traditional scalp EEG. Ovchinnikova et al. [128] presented a MEG-BCI framework for eye fixation. The MEG signals from 25 healthy participants were collected and the data were classified using the linear finite impulse response (LFIR-CNN) and vector autoregressive (VAR-CNN) models. The AUC was achieved 0.66 and 0.67 for LFIR-CNN and VAR-CNN, respectively. In [129], the author classified motor arithmetic and motor imagery classification tasks. The features were extracted from the frontal, motor, and visual areas of the brain. The extracted features were preprocessed and then were classified using the proposed fNIRS-guided attention network and obtained a mean classification accuracy of 91.96% and 78.59% for motor arithmetic and motor imagery tasks, respectively. The performance comparisons for the most important trends and the recent works based on the experimental studies are shown with the statistics in

Table 1. The performance comparisons for the most important trends and the recent works based on the experimental studies.

Reference	Functionality	Objective	Active Brain Areas	Methods	Outcome
[81]	fMRI-BCI	Emotion regulation	Lateral prefrontal cortex	Deficits in emotion regulation	The brain activity can be modified by fMRI feedback during the given task
[81]	-do-	Learning control over emotion network	Dorsal medial frontal cortex	Connectivity-based neurofeedback approach	Controlling specific behavioral changes
[84]	-do-	Self-regulation of the anterior insula	Anterior insula	Amygdala-based rtfMRI-neurofeedback	Achieved insula down-regulation and symptom alleviation in OCD disorder
[95]	-do-	Sleep improvement during Chronic insomnia disorder	Gyrus, rolandic operculum, insula	Amygdala-based rtfMRI-neurofeedback, voxel-based degree centrality	Sleep improvement in chronic insomnia-disoriented patients by altering intrinsic functional hubs
[100]	fNIRS-BCI	Classification of fNIRS signals for motor cortex signals	Motor cortex	Support vector machine on statistical features from the oxygenated hemoglobin	Constructed GA-optimized SVM and trained on statistical features. The classification was done on motor imagery vs. rest.
[18]	-do-	Classification of motor-related brain activities	Motor cortex	Motor execution proprioceptive feedback	The spatial dynamics in the motor cortex were classified with an accuracy of 100% for real movements and an accuracy of 90% for motor imagery
[105]	-do-	Motor imagery classification	Sensorimotor regions (C3 and C4)	Brain hemodynamics activities were recorded and classified using DNN, LDA, and SVM.	A combined EEG-fNIRS DNN framework was proposed and showed 83.28% accuracy on motor imagery classification
[129]	-do-	Motor arithmetic and motor imagery classification	Frontal, motor, and visual areas	fNIRS-guided attention network	The classification was done on motor arithmetic and motor imagery tasks with a mean classification accuracy of 91.96% and 78.59%, respectively
[120]	EEG-BCI	Classification of motor imagery response	Motor cortex	Training KNN on common spatial pattern features	Binary and multi-class motor imagery classification was performed and claimed 93.8% accuracy and compared with the state-of-the-art methods
[121]	-do-	Classification of motor imagery tasks	Mixed brain regions	Deep neural networks	Comparing state-of-the-art deep learning models for motor imagery tasks. The highest accuracy for EEGNet was reported, i.e., 75.5%.
[122]	-do-	Motor imagery tasks of body moves	Mixed brain regions	Attention-based CNN	BCI-based intelligent health care system for assisting disabled patients with an accuracy of 86.8% on the BCIC IV 2a dataset from the TCACNet and 96.2% accuracy on the HGD dataset
[125]	MEG-BCI	Source localization of the amplitude-modulated signal to the motor cortex	Motor cortex	Voluntary amplitude modulation of sensorimotor Mu rhythms	Analysis of communicating intentions by brain activities without involving muscles
[127]	-do-	Investigating the neural responses	Event-related potentials (ERPs)	OPM-MEG	Real-time MEG-BCI for “mind spelling” using optically pumped magnetometers
[128]	-do-	Single-trial eye fixation classification	306 sensors at 102 positions around the head	LF-CNN and VR-CNN on MEG segments	Voluntary eye fixations ROC AUC of 0.66 for LF-CNN, and 0.67 for VAR-CNN (M ± SD)

.

6. Limitations and Future Scope

The research in BCI is highly interdisciplinary, encompassing areas in neuroscience, computer science, signal processing, physics, and robotics. A successful BCI system addresses problems in a very similar manner as other pattern recognition systems, whereby it tries to find patterns in the data acquired from the brain. Although many advancements have been made in the domain in recent years, several challenges need to be addressed before BCI becomes accessible to a novice user. We list and discuss some of the challenges associated with BCI systems here.

6.1. Hardware/Software

The first problem associated with BCI is the signal acquisition capability of the hardware. For example, taking the case of EEG, which is highly portable, it has a high temporal resolution and is non-invasive. However, the signals generated in the EEG are weak and have to travel from the neuron to the EEG machine. During this journey, signals from neuronal activity are contaminated with noise and other artifacts. This not only makes it important to design reliable electrodes that are immune to external noise from devices placed close to the patient during acquisition but also to develop noise reduction techniques to remove additive noise. If we consider invasive BCI, the signal strength from electrodes is high. However, invasive BCI has hardware constraints because the hardware needs to function for an extended period and should last long. Ideally, the acquisition infrastructure used for BCI should be non-invasive, portable, have a high temporal and spatial resolution, and be cost-effective. However, no single imaging modality currently used possesses all these features.

EEG processing software needs to process high throughput, noisy, and multi-dimensional data requiring inter-process control or device input emulation. These requirements make the development of software for this purpose extremely challenging. However, there is a number of both desktop-based stand-alone and web-based software that cater to BCI data processing. BCI2000 is the most popular and standardized research platform for BCI development [130]; OpenViBE is another platform that has been developed to support real-time BCI research, offering a graphical programming language for signal processing and visualization [131]; BCI2000Web and WebFM are web-based frameworks that work alongside BCI2000 [132]. However, the adoption of newer web-based software in labs is a challenge primarily because many of these platforms are recent and need dedicated effort from the community to expand and become mainstream.

6.2. Knowledge of Biological Signals and Their Variability

It is difficult to decode the captured brain signal due to the inter-session and inter-subject variability involved. The reason behind the inter-subject variability is the different folding of the cortex across human brains. This leads to a difference in functional maps or active locations of excitation during task performance between subjects. The placement of sensors also varies slightly across sessions, which leads to inter-session variability. Therefore, the importance maps or weights obtained from two different people doing the same task over multiple sessions may look quite different.

There is a need to account for this inter-subject and inter-session variability in BCI. This inherent variability of signal features raises the requirement for an adaptive BCI procedure. Although there exist some widely accepted basic guidelines for the selection and adjustment of different brain signals captured via BCI, more research needs to be conducted to fully explore why some BCI features are effective with some individuals, while others are not.

6.3. Signal Processing

The sensors used in BCI have very low signal-to-noise ratios; hence, sensitive measurements are difficult to obtain. The signals obtained by EEG sensors are mathematically complex, and all these sensors record the same signal, i.e., the superposition of all brain activity. For an EEG, we can use a large number of sensors, but it has been observed that all of these sensors give a similar output. There is a very slight difference among the output of all sensors; therefore, these signals need to be computationally disentangled for an understanding of BCI. This requires sophisticated signal processing. There is a lot of scope for further research in developing external noise cancellation techniques that can suppress external noise. To this end, recently developed unsupervised adaptive filtration techniques can be a good choice.

6.4. Limitations of Specific Task Measurement

The brain performs a diverse set of functions, and neurons are therefore involved in multiple processes. While functional specialization captures one aspect of neuronal function, multiple regions in the brain are known for their multimodal, integrative roles in brain function. Such multimodal regions make it difficult to target a specific range of neurons for a specific task with BCI, since it makes it difficult to decode the relationships between the signal and the task under consideration. For example, while measuring brain activity during the motor imagery experiment, the sensors may pick up sensory experiences from the same limb.

6.5. Need for Calibration

In modern BCI systems, there is often a lengthy calibration procedure involved. Since there are multiple unknown variables in BCI, there is a need to properly calibrate BCI. The calibration of BCI should involve multiple data samples over multiple parameters such as gender, age, task, prior knowledge, etc. The need for large datasets becomes further important when machine learning algorithms are involved. Additionally, the calibration time for the BCI device is also a matter of concern and should be as small as possible. Recent developments in machine learning algorithms suggest reusing calibrated subject data from previous BCI sessions or calibrated data collected from other users.

6.6. Security Issues

From a security point of view, any BCI system must follow the concept of security, i.e., integrity, confidentiality, availability, and safety [133]. Over the years, as BCCI has progressed, BCCI has made significant improvements in the quality of life of patients [134], such as restoring damaged hearing, improving movements and communication abilities in paralysis patients, helping people with disabilities, etc. Following these promising applications, the current focus of researchers is to further develop BCI technical capabilities in the context of brain-to-brain and brain-to-internet. The BCI aspects related to security issues have not yet matured enough and also provide opportunities for a sophisticated attacker. There is an urgent need to standardize and unify the BCI system. With the advancement of ML algorithms and their applications to BCI systems, the risk of adverse attacks has increased further. Sensors are key components in any BCI system, and deceptive stimulation attacks are also occurring frequently these days. In the medical field, the security of patients and their data is highly sensitive and should be confidential in most cases. The authors in [135] provide a detailed discussion of important aspects related to security.

7. Discussion

The brain is a massive network of neurons with multiprocessing capabilities that upon receiving stimuli from the environment, process them and then produce a suitable response. In this article, we discussed the different parts of the brain that are crucial for the functioning of the BCI system. Functional mapping of a human brain with various available neuroimaging techniques provides insight into the roles played by various regions in the brain. These roles are important in BCI experiments, as the signal identification and classification approaches are dependent upon the signals obtained from these active areas.

Functional brain mapping is a technique of finding different functional areas in the brain that control some critical functions such as vision, taste, eye movements, decision-making, etc. These areas are often known as eloquent areas. Once the location of the targeted area and its functioning are identified, the custom map of an individual’s brain can be created, and the information can be transferred from that part to other parts of the body or some artificial means with the help of the BCI system. Different active areas were identified through the study of experiments based on functional neuroimaging and BOLD response-based BCI (performed with fMRI and fNIRS), which helped to identify various structural areas of the brain, including the amygdala, anterior cingulate cortex, prefrontal cortex, anterior insula, and subcortical areas with fMRI. The BCI experiments with fNIRS showed active roles in motor execution and motor imagery areas. Similarly, the neurophysiological recording-based BCI experiments (performed with EEG and MEG) showed the involvement of various active areas that are functional and actively target by BCI systems. Continuous research with advanced data processing and classification approaches is needed for identifying and understanding the roles of various regions of the brain that play active roles during cognition and can be targeted for the efficient design of BCI systems. This review provided insight into various experiments performed with BCI together with neuroimaging modalities.

We found that studies on non-invasive signals dominate BCI research. Among the few works that investigated invasive BCI, most of them worked on ECoG (partially invasive) instead of the invasive intracortical signals. This is expected, since the invasive BCI has higher hardware and human resources involved with them. Even ECoG requires a volunteer patient and a surgeon who will perform a craniotomy. This also leads to a paucity of data (few participants) and public datasets. We also observed that among the noninvasive paradigms, EEG has emerged to be the most popular BCI paradigm. The advancement of deep learning paradigms has made it possible to extract complex information in the data from brain sensors and decode it. However, there are both fundamental challenges related to SNR, the temporal and spatial resolution of signal acquisition, and emerging challenges that are related to the security of BCI systems that have come to the fore because of their new and more advanced applications.

The BCI research community has found that researchers working in the field of BCI techniques often face difficulties when they try to compare the results found in the literature. Hence, the BCI community emphasizes the need for an objective method to compare BCI technologies. In Figure 3, we show a basic functional model with a minimal set of components to define a BCI system. A good BCI system should be designed in such a way that it does not hinder future designs and can be easily adopted without any hindrance. As we discussed in Section 3, the BCI system should be flexible so that it can adopt a range of system configurations, including multimodal design, and support any type of device if needed. Due to its multidisciplinary nature, the BCI system requires neural activity, and mechanical, electrical, computer science, and artificial intelligence. Therefore, a well-defined BCI system may be represented by nine components, namely, the user, electrodes, amplifier, feature extractor, feature translator, control interface, device controller, devices, and operating environment. Not all of these components are always required and may be redundant from application to application. For example, several BCI systems map the electrical signals from the electrode into a discrete control signal. Now if a user needs to make a decision based on the input signal, the featured translator (also called the feature classifier or simply classifier) will be required, and if there is no need for any knowledge-based decision, this component may be skipped. The electrodes are the most important components of any BCI system design, which collect functional information in the form of electrical signals from the targeted brain region. The performance of any effective BCI design highly depends upon the sensitivity and the quality of these electrodes.

8. Conclusions

Functional mapping of the human brain with various available neuroimaging techniques provides insight into the roles played by various regions in the brain. These roles are important in BCI experiments, as the signal identification and classification approaches are dependent upon the signals obtained from these active areas. This review provided insight into various experiments performed with BCI together with neuroimaging modalities. The paper discussed the concept of BCI and basic components in a typical BCI system. Further, various active areas were identified through the study of experiments based on functional neuroimaging and BOLD response-based BCI (performed with fMRI and fNIRS), which helped to identify various structural areas of the brain, including the amygdala, anterior cingulate cortex, prefrontal cortex, anterior insula, and subcortical areas with fMRI. The BCI experiments with fNIRS showed active roles of motor execution and motor imagery areas. Similarly, the neurophysiological recording-based BCI experiments (performed with EEG and MEG) showed the involvement of various active areas that are functional and actively targeted by BCI systems. Continuous research with advanced data processing and classification approaches are needed for identifying and understanding the role of various regions of the brain, which plays an active role during cognition and can be targeted for the efficient design of BCI systems.