A study of neurophysiological brain activity attracts significant interest of researchers from various fields of science due to its interdisciplinary nature. The considerable progress in the development of experimental approaches for neuroimaging and mathematical methods for big data analysis provides great opportunities for vast and detailed studies of specific phenomena in the brain neural network and the creation of sensor systems for monitoring brain states. Recent progress in this field has been achieved at the junction of mathematics, physics, engineering and neuroscience [1
]. This is confirmed by an increasing number of papers related to brain research and published in multidisciplinary journals (see, e.g., [2
From the viewpoint of nonlinear dynamics, brain is a very complex dynamical system, containing around 86 billion neurons [6
]. The neurons are connected by synapses, thus forming a complex network with nodes and links represented respectively by neurons and synapses. Specific features of time-spatial activity of the neural network in the brain cortex and cooperative dynamics of different brain areas (e.g., event-related synchronization/desynchronization) on a sensor level provide important information about the current state of the nervous system and cognitive brain ability [7
]. Particular brain states are associated with motor brain activity during either real or imaginary movement.
Revealing specific features of spatial brain cortex activity related to real motions and motor imagery of different limbs can be essential not only for basic research in neuroscience, but also for applications in medicine to improve the quality of life of post-traumatic and post-stroke patients using brain-computer interfaces (BCI) for rehabilitation [11
] or to control prostheses and exoskeletons [14
]. One of the important BCI functions is online detection of specific features of electromagnetic brain activity using electroencephalography (EEG) [15
] or magnetoencephalography (MEG) [16
], and transformation of certain patterns into control commands to perform specific actions in the environment without the need of “classical” methods of human–machine interaction [17
Apart from EEG and MEG, other methods are also used to acquire information about brain states. In particular, functional near-infrared spectroscopy (fNIRS) [18
] is a powerful tool of noninvasive optical imaging successfully used in BCI for registration of brain activity and control command formation [20
]. Control commands for this kind of BCI should not be affected by any muscular activity [23
]. Therefore, a study of brain states related to motor imagery is very important for designing such BCI [16
]. Motor imagery is a mental process by which a person rehearses or simulates a given action with no real motor activity. Some researchers treat motor imagery as a conscious application of unconscious preparation for real motor activity [25
]. A number of studies have highlighted common features for real and imaginary motor activity [26
]. One of the common features, important for the BCI development, is that the cortical layout in the primary motor cortex M1 is quite similar between motor execution and motor imagery.
The most popular technique for studying brain activity during motor executions is EEG, that implies the location of special sensors on the scalp (or directly into the brain) and recording EEG signals as electric currents generated by a group of neurons [29
]. The EEG signal or electrical response of the neural network is characterized by a complex time-frequency structure containing specific frequency ranges, oscillatory patterns, stochastic components, artifacts, etc. [30
]. It is well known that there is a strong correlation between EEG rhythmic activity and functional states of the organism [31
] that can be used for revealing specific features related to real and imagery motor activity [34
]. The problem of detection and classification of different types of motor execution requires the application of various methods for time-frequency and spatio-temporal analyses [37
], including artificial intelligence methods [4
], recurrence measures of signal complexity [38
], as well as event-related synchronization (ERS) and event-related desynchronization (ERD) [39
In this work, we use fNIRS, an efficient noninvasive technique for brain activity estimation [40
], that employs near-infrared light to detect changes in oxygenated (HbO) and deoxygenated (HbR) hemoglobin levels due to hemodynamic brain activity and the rapid delivery of oxygenated blood to active cortical areas through neurovascular coupling [41
]. A high efficiency of fNIRS is achieved due to the use of laser lights with two different wavelengths which penetrate most tissues in the head, but are highly absorbed by oxyhemoglobin (HbO) and deoxyhemoglobin (HbR). According to changes in the absorption of these two wavelengths, corresponding changes in HbO and HbR concentrations can be calculated and hence the oxygenation of brain tissues can be assessed.
It should be noted that fNIRS has a common physiological basis with functional magnetic resonance imaging (fMRI), so that their signals are correlated [42
]. Meanwhile, fNIRS has many advantages over fMRI, namely, resistance to motor artifacts, high mobility due to significantly smaller sizes, and higher temporal resolution for the determination of oxyhemoglobin and deoxyhemoglobin. At the same time, fNIRS has some drawbacks, in particular, fNIRS has lower spatial resolution than fMRI. Despite relatively low temporal resolution and the existence of time delay in the hemodynamic response compared to EEG signals, fNIRS is a powerful tool of functional neuroimaging and can compete with other imaging techniques such as fMRI and EEG. The fNIRS imaging is superior to other techniques in studying neural activity in the primary motor cortex M1 because this area lies on the outer cortices which is within the blood oxygenation level deprivation (BOLD) scanning range for fNIRS [43
]. A significant advantage of fNIRS over MEG is its portability and more easy mounting of sensors on the patient’s head. Moreover, the fNIRS equipment is cheaper as compared to fMRI and MEG. Thus, fNIRS holds a special place among modern methods of neuroimaging due to a good combination of temporal and spatial resolutions, overall mobility and simplicity of use.
In this paper, we analyse fNIRS data acquired during human real and imaginary motor activity associated with motor cortex hemodynamics. In particular, we extract specific features of the fNIRS signals related to different types of motor activity, which can be used in BCIs. We also propose an universal method to classify fNIRS trials obtained during motor imagery and develop a sensor of motor activity to be used for neurorehabilitation systems after various brain injuries, including stroke.
The classification of brain activity trials associated with motor imagery using different methods of neuroimaging is a modern research topic widely explored by many researchers from different fields of science [13
]. The diverse literature indicates that different brain regions can be involved in motor imaginary [59
], including primary motor cortex (M1) [60
], supplementary motor area (SMA) [61
], posterior parietal cortex (PPC) [62
], prefrontal cortex [63
], etc. Moreover, there are at least two types of motor imagination (kinesthetic and visual) characterised by the involvement of various areas of the brain cortex. In particular, a well-pronounced suppression of alpha activity was observed in the occipital region in subjects exhibiting the visual type of motor imagery. In contrast, kinesthetic subjects displayed a pronounced suppression of mu activity in the motor and somatosensory cortexes [16
]. In the referenced paper, 65%–80% classification accuracy was reported when choosing optimal MEG channels for both kinesthetic and visual untrained subjects. Notably, such accuracy for untrained subjects is similar to classification accuracy obtained in EEG studies with trained subjects [64
]. Based on the effect of suppressing mu/alpha and beta activity during motor imagery, it is possible to obtain up to 70%–85% accuracy using various classifiers, including machine learning methods, SVM, ICA, etc. [65
]. The main obstacle for achieving higher classification accuracy of single trials is the significant variability of temporal-spatial brain activity during motor imagery, which does not allow revealing universal patterns associated with a particular type of motor imagination.
In addition, the use of the hemodynamic response for the classification of motor imagery (e.g., for neurorehabilitation) also imposes a number of limitations. Firstly, the hemodynamic response is quite inertial, so that the registration of mild changes in oxygenation and deoxygenation of blood is required in the area of interest. Secondly, due to a relatively high cost and a large size, the MRI equipment is usually used in clinics for diagnosis, but not for regular training of motor imagery to restore motor activity after a stroke, as in the case of using EEG. In this context, it is of considerable interest to use the fNIRS device as a sensor of brain activity with an appropriate classification algorithm for the allocation of motor-related brain activity on the sensor level. As we have shown, the hemodynamic response generated by motor activity is strong enough to distinguish the corresponding region of the motor cortex responsible for the motor pattern formation. Thanks to the hemodynamic analysis, which is not sensitive to EEG background activity, it is possible to increase classification accuracy due to better signal repeatability in the analysis of single events. As a result, the classification accuracy of imaginary activity was increased up to more than 90%, which is quite sufficient for medical practice in neurorehabilitation procedures. It is important to emphasize that the fNIRS-based sensor is able to classify not only the kinesthetic mode of imagination, which is characterized by a pattern similar to real movements but also the visual type of imagination. However, we have to assume that in some cases, the classifier cannot recognize motor activity (the “uncertanity” algorithm branch in Figure 4
In conclusion, it should be noted that imaginary movements are often used to form control commands in active brain-computer interfaces (BCI) [66
]. Typically, such systems are based on the registration of electric brain activity (EEG) due to the fact that EEG gives a very fast response to changes in cognitive states. Moreover, EEG sensors are cheap and portable and do not impede the subject movement. Although the proposed fNIRS-based sensor system is also portable that allows the installation of sources and detectors faster and more convenient than EEG (or the same in the case of dry EEG electrodes [68
]), the fNIRS device is more expensive than EEG. At the same time, the number of needed fNIRS channels for classifying activity is not high that can reduce the system cost. On the other hand, due to slow hemodynamic response, fNIRS exhibits a relatively large delay in detecting brain activity as compared to EEG, which restrict its application in BCIs. Nevertheless, the proposed fNIRS classifier can be used as a motor imagery sensor in neurorehabilitation systems with patients for which a 10–15 s latency is not significant. However, if such a BCI is used in game interfaces [71
] or control interfaces of external devices (wheelchair, manipulator, etc.) [73
], such delay time is critical and negate the advantages of our system in classification accuracy of brain states. One of the possible solutions to this problem could be the use of hybrid fNIRS-EEG BCIs [76
]. The consideration of this type of classifier is out of the scope of this paper and might be the matter of future research.
In this paper, we have carried out the analysis of fNIRS data acquired during real and imaginary movements. Distinct spatial dynamics in the motor cortex when performing motor actions (real or imaginary) with the left or right hand exhibits pronounced laterality between two hemispheres. This allowed us to reveal hemodynamic biomarkers for classification of the type of movement. The proposed fNIRS-based sensor provides close to 100% recognition accuracy in the detection of real movements, while the classification accuracy of motor imagery is a little smaller and reach 90%.
The important advantage of the proposed method is the possibility to efficiently classify different types of movement, both real and imaginary, without recalculation of the system parameters. This essential feature of the developed sensor results from pronounced laterality of the hemodynamic brain response to motor activity.
The knowledge of the hemodynamic behavior in the motor cortex during real and imaginary motor activity along with approaches for its detection can be helpful not only for fundamental studies on human motor-related tasks but also for the development of fNIRS-based BCIs.