Brain Computer Interface-Based Action Observation Game Enhances Mu Suppression in Patients with Stroke

: Action observation (AO), based on the mirror neuron theory, is a promising strategy to promote motor cortical activation in neurorehabilitation. Brain computer interface (BCI) can detect a user’s intention and provide them with brain state-dependent feedback to assist with patient rehabilitation. We investigated the effects of a combined BCI-AO game on power of mu band attenuation in stroke patients. Nineteen patients with subacute stroke were recruited. A BCI-AO game provided real-time feedback to participants regarding their attention to a flickering action video using steady-state visual-evoked potentials. All participants watched a video of repetitive grasping actions under two conditions: (1) BCI-AO game and (2) conventional AO, in random order. In the BCI-AO game, feedback on participants’ observation scores and observation time was provided. In conventional AO, a non-flickering video and no feedback were provided. The magnitude of mu suppression in the central motor, temporal, parietal, and occipital areas was significantly higher in the BCI-AO game than in the conventional AO. The magnitude of mu suppression was significantly higher in the BCI-AO game than in the conventional AO both in the affected and unaffected hemispheres. These results support the facilitatory effects of the BCI-AO game on mu suppression over conventional AO.


Introduction
Stroke is a leading cause of long-term disability and is often associated with persistent impairment of the upper extremities [1]. Approximately 70-80% of patients with stroke experience upper extremity impairments, and 5 years after stroke onset, approximately 56% of patients continue to report pronounced hemiparesis [2,3]. This loss of functionality can lead to difficulties in activities of daily living (ADLs) and participation in the community [1]. Therefore, providing effective and efficient training to restore upper limb function is important in patients with stroke to enable them to perform ADLs and participate in social environments [4].
Recently, brain plasticity has served as a therapeutic basis for the neurorehabilitation of patients with stroke. Brain plasticity refers to the ability of the brain to modify and reorganize its structure and function, not only in response to injury, but also as a result of external stimuli and experiences [5]. Brain reorganization can be induced by continuous stimulation of the motor cortex including repetitive and active training [6].
Action observation (AO) is a promising technique used to enhance rehabilitation outcome by promoting motor cortical activation in patients with stroke [7]. Recent studies have reported the effectiveness of AO for motor performance improvement and motor memory formation in healthy people and patients with stroke [8][9][10][11][12]. Researchers have suggested that the modulation of the motor system in AO is mediated by activating the mirror neuron system (MNS). The MNS is a cortical neural network that is activated during performance of a particular action as well as observation of a similar action being performed by others [13]. The MNS supports various neurocognitive functions, such as social cognition, language, emotion recognition, motor learning, action imitation, in particular, understanding the meaning of actions and meditating the process of action execution [14][15][16]. The neural networks forming MNS include the inferior frontal gyrus, the inferior parietal lobule, the lower part of the precentral gyrus, and also the temporal, occipital, and parietal visual areas [15]. Examination of the power of mu rhythm (8)(9)(10)(11)(12)(13) oscillations in electroencephalography (EEG) is convenient and a noninvasive method to study human MNS function [17][18][19][20]. Mu rhythm is typically reduced during motor execution, AO, or motor imagery.
In patients with stroke, conventional AO has several limitations. First, it is difficult for patients with stroke to pay attention for long periods of time. Further, it is challenging to ascertain whether a patient is actively participating in AO. Patients have difficulty concentrating, maintaining interest, and actively participating during treatment because they passively observe the same series of repeated movements without any feedback [21]. This may limit the effects of AO on neural plasticity. Therefore, better methods are needed to address these issues and increase AO effectiveness.
Brain computer interface (BCI) systems can detect a user's intention and provide them with brain state-dependent feedback through certain feedback mechanisms using brain signals to assist with patient rehabilitation [22,23]. Recently, steady-state visual-evoked potential (SSVEP) technology has been introduced in BCI systems [24][25][26]. SSVEPs are natural responses to visual stimuli at specific frequencies, with the strongest response at frequencies approximately 15 Hz or 20 Hz [27]. This technique involves a blinking stimulus presented at a specific frequency that results in brain waves of the same frequency synchronizing in the occipital lobe, which can be used to control various devices and develop BCI applications [25,[27][28][29]. In a preliminary study, we demonstrated that mu suppression and SSVEP expression were simultaneously observed during AO training with a flickering action video and it was possible to confirm whether the user was actually watching the flickering action video [30]. Further, we found that mu suppression was not affected by flickering itself in a previous study of healthy subjects [31].
We hypothesized that BCI feedback would induce greater mu suppression than conventional AO, by increasing active participation in patients with stroke. Therefore, the aim of this study was to investigate the effects of a combined BCI-AO game, relative to conventional AO, on mu suppression in patients with stroke.

Study Population
Participants in the subacute stage after stroke were recruited for this experimental study. The inclusion criteria were that participants needed to (1) be older than 20 years of age, (2) have a unilateral primary ischemic or hemorrhagic stroke diagnosed by magnetic resonance imaging (MRI) or computed tomography scans, (3) have paresis of the affected extremity (upper extremity Fugl-Meyer assessment (FMA); 0-66) [32], (4) be between 7 days and 3 months after stroke onset, and (5) have sufficient cognitive ability to understand the experiments. Participants were excluded if they had cognitive deficits (<24 on the Mini Mental State Examination) [33], visual neglect (indicated by the star cancellation test), visual field defects (restricting the access to visual stimuli), or aphasia (unable to comprehend instructions). All participants provided written informed consent, and the study was approved by the Medical Ethics Committee (IRB number: 2018-03-011).

Video for AO
We followed the same experimental protocol and electroencephalography (EEG) analysis method used by Lim et al [31]. A video showing grasping of a ball with the right and left hands was used as the action video. The action of grasping the ball was repeated three times in each action video (one task). The playback time of each task was 10 seconds. The only difference between the action videos for the BCI-AO game and conventional AO was that during the BCI-AO game, the background of the video flickered. In the flickering action video, background pixels flickered white and black while the action video was playing (Figure 1a). The interval of the frames was 1/60 seconds and six frames were included in one period, resulting in a flickering frequency of 15 Hz. The action videos were presented at the center of a 20-inch monitor located 50 cm from the patient using Unity3D software (Unity Technologies, San Francisco, CA, USA). The resolution was 1920 × 1080 pixels, with a vertical refresh rate of 60 Hz.

Experimental Procedure
The experiment was conducted under four conditions. The rest condition and training conditions preceded the BCI-AO game and conventional AO conditions. After the rest and training conditions, the BCI-AO game and conventional AO condition were randomly presented. As a result, the order of the four conditions was as follows: (a) rest, (b) training, (c) BCI-AO game, and d) conventional AO or (a) rest, (b) training, (d) conventional AO, and (c) BCI-AO game. The action videos of the right and left hands were displayed in a pseudorandomized order. In the BCI-AO game and conventional AO conditions, the task was repeated 15 times with a 10 s rest period between each task. Both conditions lasted for 300 s in total.
(a) Rest: Before the experiment, we recorded EEG data during the resting state in each patient for 3 min.
(b) Training: This condition was performed to classify whether the patient was watching the action videos. A flickering action video was presented with a black background. The action videos were shown 10 times.
(c) BCI-AO Game: The BCI-AO game determined whether patients watched the action video and compensated for their attentiveness accordingly (Figure 1b). Prior to playing the game, an instructive video describing how to play the game was shown to the patients. This game provided feedback of patients' observation scores (a score for one trial) and observation times (time for observation for a trial; real-time feedback). Once the SSVEP and mu feature identified that the patient watched the action video, the observation time was increased and observation score for one trial presented at the end of each action video for a single observation task, i.e., every 10 s. The observation score was calculated as the time that the patient watched the action video relative to the amount of time the video was presented; zero points indicated no observation (observation time = 0 s), and a perfect score of 100 points indicated observation of the entire video for one task (observation time = 10 s). Real-time feedback on observation performance was provided using two methods during the game. First, the observation progress bar surrounding the action video was increased continuously at the time of observation during each task for 10 s; it was reset when the next task started. Second, the observation progress bar located at the top of the screen was activated at the time of observation during the entire game play for 300 s; it ran continuously from the beginning to the end of the game.
(d) Conventional AO: This condition involved a conventional AO training program where the patients simply watched the action videos ( Figure 1c). It used a non-flickering action video where the background pixels of the action video showing the hand grasping the ball were white, and the background image outside the action video was the same as that of the BCI-AO game. No feedback was provided based on the patient's attentiveness.
The OpenVIBE software platform (ver. 1.2.0) (INRIA, Rennes, France) was used for classification, and individual classifiers were created to distinguish between observation and resting states to confirm attentiveness. We created common spatial pattern filters (CSPF) and classifiers using data obtained from the training condition. First, to design the CSPFs, EEG data were classified into two states: observation and rest. The data obtained during the first second from the beginning of presentation were excluded to eliminate the possibility of attentional delay. Data were then filtered for SSVEP (14.5-15.5 Hz) and mu (8)(9)(10)(11)(12)(13). Thereafter, the two CSPFs were designed using filtered signal types. After design of CSPFs, classifier was designed with applying the CSPFs. In the step of designing classifier, training EEG data were separated into two states and went through temporal filter for each feature (SSVEP and mu) like the CSP filter training step. The filtered data went through each CSPFs, then time-based epoching was conducted with 1 sec window size for every 0.1 sec. In this epoch, SSVEP and mu power were extracted, subsequently two features were aggregated into one feature vector. Using these feature vectors, the classifier was designed so that the it could distinguish the two states. The designed CSPF and classifier were applied during the BCI-AO game condition ( Figure 2).

Measured Parameters
EEG data was analyzed using MATLAB (MathWorks Inc, Natick, MA, USA). Mu range (8-13 Hz) power was analyzed for each condition at each electrode site separately to evaluate the effect of AO conditions on mu suppression. The power of mu range in each condition (BCI-AO game, conventional AO) was extracted from the short-time Fourier transform (STFT) with a 1 s window size and 90% overlap, and then averaged. The mean power of the mu range in the rest condition was used as the reference power. Mu suppression index was calculated as the log ratio of the power during observation relative to the reference power. Accordingly, a log-ratio of negative value would indicate mu suppression, while zero would indicate no mu suppression.
To evaluate the time course of mu suppression, a STFT was performed on data from C3 and C4 during the entire period of observation using the same method described above. After extracting and averaging the power of the mu range, a mu suppression index was calculated as the log-ratio, and the values were averaged to calculate a group mean. A 60 s point moving average filter was used to compare the trend by smoothing the time course analysis.

Statistical Analysis
To evaluate the effectiveness of the BCI-AO game on mu suppression, the mu suppression indices were compared between the BCI-AO game and conventional AO condition with a paired ttest. We also carried out analyses of variance (ANOVAs) to see the overall pattern of the results for significant areas. To assess mu suppression according to lesion side, the mu suppression indices were compared between the affected and unaffected hemispheres in each condition with two-way ANOVAs. The time course of mu suppression at C3 and C4 was compared between the BCI-AO game and conventional AO condition. For this, paired t-tests were performed with mu suppression values of BCI-AO game and conventional AO for every time point. To analyze mu suppression according to the location of brain lesions, the mu suppression indices were compared between the supratentorial and infratentorial lesions for each condition with independent t-tests. Correlation analysis was used to determine the relationship between mu suppression during the BCI-AO game and motor impairment severity. We compared mu suppression between healthy subjects in the previous study and patients with stroke in the present study in all channels except for O1 and O2. Since the mean age was significantly different between two subject groups, the ANOVAs for repeated measures with age covariate [group (healthy subjects and stroke patients) × condition (BCI-AO and conventional AO)] was conducted. All statistical analyses were performed using SPSS Statistics 20 (IBM Corp., Armonk, NY, USA). Values of p < 0.05 were considered statistically significant.

Characteristics of the Study Population
Twenty-three participants were eligible for inclusion. Data from 19 of the 23 patients were analyzed; the rest were excluded due to discontinuation of the experiment (n = 1), poor quality of data (n = 2), and inability to acquire data (n = 1). Table 1 shows the baseline characteristics of the participants. The mean age of the participants was 69.00 ± 11.70 years. The mean time interval between stroke onset and the experiment was 25.89 days ( Table 1).  Figure 3a shows the difference in mu suppression at central areas (C3 and C4) during AO under the two conditions. C3 and C4 showed significant mu suppression during the BCI-AO game compared to conventional AO (C3: t = 2.676, p = 0.015; C4: t = 2.452, p = 0.025). Topographic images indicated that overall mu suppression was more strongly induced by the BCI-AO game than by conventional AO (Figure 3b); this greater mu suppression was significant not only at C3 and C4, but also at most other regions such as P3 (t = 2.846, p = 0.

Mu Suppression According to Brain-Lesion Location
When analyzed by supratentorial and infratentorial lesion, mu suppression at central areas (C3 and C4) in the unaffected hemisphere for the BCI-AO game condition was significantly greater in the supratentorial lesion than in the infratentorial lesion (t = 2.180, p = 0.044) ( Figure 5). There were no significant differences at other areas in both conditions.  Figure 6 shows the time course of mu rhythm over 300 s in C3 and C4. Both the BCI-AO game and conventional AO conditions showed time-dependent mu suppression. However, greater mu suppression was observed under the BCI-AO game condition compared to conventional AO condition in both C3 and C4 during the training. The differences were statistically significant for majority of the time during training (p < 0.05). Significant differences between the two conditions were observed at the end of training in C4 but not in C3. Figure 6. Comparison of mu suppression over time between brain computer interface-based action observation (BCI-AO) game and conventional AO conditions. The red bar under the time course graph indicates a significant difference between the two conditions (paired t-test, p < 0.05).

The Effects of Motor-Impairment Severity on Mu Suppression in the BCI-AO Game
When analyzed according to the severity of motor impairment for the BCI-AO game condition, significant correlations between FMA and amount of mu suppression were observed at C3/C4 (r = −0.493, p = 0.032) and T3/T4 (r = −0.518, p = 0.023) in the unaffected hemisphere. There were no significant correlations between FMA and the amount of mu suppression in other areas in the unaffected hemisphere or in any areas in the affected hemisphere during the BCI-AO game condition, or those in conventional AO. Thus, patients with lower FMA had stronger mu suppression than those with higher FMA in the unaffected central areas, particularly in the BCI-AO game. Conversely, no mu suppression occurred in the conventional AO regardless of the severity of motor impairment.

Behavioral Results in the BCI-AO Game
The feedback score was 58.63 ± 4.63 in average for the BCI-AO game condition. There was no feedback score in conventional AO condition.

Discussion
We investigated the effects of a new rehabilitation program combining a BCI game and AO in patients with stroke. We demonstrated that compared to conventional AO, the BCI-AO game induced greater mu suppression in both affected and unaffected hemispheres. As the mu suppression is closely related to the MNS activity, this finding has important implications for neurorehabilitation.
The strong cortical activations of brain areas belonging to the MNS are associated with improvement in motor performance, by facilitating the reorganization of cortical motor loops that leads to the recruitment of stored motor programs [34][35][36]. Since neural substrates that play a role in learning new motion patterns include the MNS, the role of the MNS in motor learning is an important part of rehabilitation therapy for patients with stroke [37]. Therefore, these results suggest that BCI-AO game can engage patients over conventional AO and has potential for new rehabilitation tool in stroke patients.
To our knowledge, this is the first study to investigate the effects of a BCI-AO game providing real-time feedback based on observation degree on mu suppression in patients with stroke. In line with previous work where we studied the effects of a BCI-AO game on healthy subjects [31], the BCI-AO game showed greater mu suppression than conventional AO in patients with stroke. In the comparison of mu rhythm between stroke patients and healthy subjects, the mu suppression in stroke patients was reduced compared to healthy subjects in several areas. As participants were not agematched controls, one could argue that the difference between the two study groups is due to age factor. However, we included age as covariate factor. Moreover, it has been reported that the MNS activation during AO is not age-dependent [38], We could speculate that brain damage may have contributed to the reduction of mu suppression in patients with stroke compared to healthy subjects.
In this study, the BCI-AO game induced greater mu suppression in the parietal, temporal, occipital, and central motor areas than conventional AO. This is consistent with other cortical activity studies during AO with EEG [7,39]. In a study by Tani et al., mu suppression during AO had a more widely distributed involvement of the prefrontal and parietal areas in addition to the sensorimotor cortex, compared to motor imagery in patients with stroke [7]. In a study by Kim and colleagues, a low level of alpha power in the frontal, central, parietal, and occipital areas during AO and a relatively lower level of alpha range compared to motor imagery were observed [39]. In a metaanalysis that analyzed the cortical areas involved in AO, the AO network was found to include bilateral frontal premotor, parietal, and temporo-occipital areas [40]. Extended cortical activation during AO in our study is also consistent with previous functional MRI (fMRI) studies reporting that AO involves many brain regions associated with MNS activation that are functionally connected [41][42][43][44]. In addition, the extent of activation by AO correlates with lesion volume, suggesting adaptive plasticity [45]. Thus, if BCI-AO game is applied to other modalities such as motor imagery or mirror therapy, it can be possible to increase the effectiveness of rehabilitation in motor learning and lead to functional recovery by activating the extended motor networks [15,35,46].
As limiting factors in our work, we acknowledge that mu rhythm in the parieto-occipital cortex has a possibility to be the alpha rhythm. The suppression of alpha rhythm in parieto-occipital cortex is associated with basic visual processing. However, it is meaningful in this brain area, even if it was alpha rhythm not mu rhythm. The parieto-occipital cortex has the role in spatial attention, sensorimotor integration, and movement planning [35]. Since it is the strongest cortical EEG rhythm, which is profoundly modulated by attention, previous studies demonstrated alpha rhythms in parieto-occipital cortex as one of the mechanisms of attention-related neural synchronization [47,48]. Further, SSVEP-based real-time feedback could elevate participant' arousal level, leading to arousallinked alpha desynchronization. Thus, our results show anyway the possible benefit of BCI-AO that recruit brain areas involved in motor execution, spatial attention, and arousal.
In the BCI-AO game and conventional AO, mu suppression was observed in both the affected and unaffected hemispheres. This is associated with the existence of a bi-hemispheral human MNS. Previous EEG and fMRI studies reported that AO induces bi-hemispheral activation in healthy participants and patients with stroke [18,45,49]. We observed that the difference in mu suppression between the affected and unaffected hemispheres was not significant, which is inconsistent with previous findings [7,17]. According to a study investigating the relationship between stroke lesion and mu suppression, the magnitude of mu suppression during AO was significantly reduced in the affected hemisphere when compared to the unaffected hemisphere, with greater damage in the inferior parietal lobule being associated with less mu suppression [17]. A possible reason for these discrepancies is the high proportion of patients with mild impairment and lesions that involved subcortical regions in our study, as mu suppression is reduced over hemispheres affected by cortical rather than subcortical damage [50,51].
Mu suppression at central areas in the unaffected hemisphere was significantly less in infratentorial lesions than in supratentorial lesions; this result aligns with the findings from the study by Park et al., which reported different topography between supratentorial and infratentorial lesions [52]. In that study, EEG activation pattern of infratentorial lesions more closely resembled that of healthy subjects, compared with that of supratentorial lesions containing motor cortex during the active motor and motor imagery tasks. These different patterns based on the depth of lesion location are associated with interhemispheric inhibition. Interhemispheric inhibition between primary motor cortices affects infratentorial lesions less than supratentorial lesions [53]. Therefore, patients with infratentorial lesions may experience fewer inhibitory effects in the unaffected hemisphere when compared to supratentorial lesions. We inferred that this was the reason underlying the significant difference in mu suppression between supratentorial and infratentorial lesions in central areas. Due to the small sample size and lack of studies examining neural mechanisms specific to supratentorial and infratentorial lesions, the exact mechanisms underpinning these effects are unclear and should be elucidated in the future.
Although AO has less mu suppression effect in the affected hemisphere [7,17], the BCI-AO game implemented in this study induced greater mu suppression both in the affected and unaffected hemispheres than conventional AO. Activation of the MNS in the affected hemisphere is important in the recovery of patients with stroke in the subacute phase [54]. In a study investigating temporal changes in event-related desynchronization (ERD) after stroke, there was an increase in ERD in the affected hemisphere related to functional recovery over time in patients with good recovery; conversely, no change was observed in the ERD of the affected hemisphere in patients with poor functional recovery [55]. In addition, as activation in contralesional motor cortex correlates with favorable motor recovery in several studies [56,57], mu suppression observed in the unaffected hemisphere is also a beneficial effect of the BCI-AO game. This BCI-AO game has the potential to more effectively induce the neural plasticity by stronger MNS activation both in the affected and unaffected hemispheres. The direct effects of the BCI-AO game on neural plasticity should be clearly revealed by future study including sham conditions.
The major novelty of our AO program was the augmentation of mu suppression by combining AO with a BCI game providing feedback based on the degree of observation in patients with stroke. As flickering itself did not affect mu suppression and a gaming display without interaction had no critical impact on attention [31], therefore, the greater mu suppression under the BCI-AO game condition in the present study may be due to compensatory mechanism through BCI-AO paradigm. By providing real-time feedback using SSVEP, this BCI-AO game paradigm not only provides information on the degree of participation of the patient, but also enables greater participation in observation based on feedback scores, thereby enhancing the effects of mu suppression. The active participation in rehabilitation is of notable importance, as it allows the patient to perform AO more precisely and intensely [35]. Recent BCI systems have been used to detect primary motor cortex activation and provide matched sensory stimulation, allowing patients to modulate their signals through learning based on afferent feedback [58,59]. Feedback latency is important to provide effective feedback to the BCI system. By providing a short time interval between observation and feedback, which is the important factor in Hebbian theory, the BCI-AO game may increase synaptic efficacy which can lead to neuroplasticity for motor learning [60,61].
The greater mu suppression in the BCI-AO game over the majority of the training period despite a time-dependent reduction suggests that the patients participated in AO for longer when compared to conventional AO. The progression bar, which is activated when patients observe action videos, can arouse interest in the patients by confirming their participation in AO in real time. In addition, with the feedback scores provided after the end of each AO task, patients can confirm how much they concentrated on the action videos and are afforded learning opportunities on how to engage in more observation and consequently increase their game score. This can lead to greater participation in observation training. In addition, this therapy may also be advantageous in older patients, patients with cognitive dysfunction who may have difficulty with motor imagery, or patients who are unable to participate in physical therapy due to severe paralysis; as patients only need to observe the motion without internally simulating themselves or learning difficult techniques, it may be easier for these individuals to partake in this form of rehabilitation.

Limitations
This study has a few limitations. First, we were unable to determine the long-term effects of the BCI-AO game on MNS activation and functional improvements in stroke patients. In future studies, it is necessary to confirm the long-term effects of the BCI-AO game through multiple training sessions and follow up after completion of training. Second, we did not compare mu suppression in the BCI-AO condition with conditions in a sham paradigm (flickering alone or gaming display with sham feedback). Although there was no impact of flickering or gaming display without feedback on mu suppression in healthy subjects [31], there is a possibility that the results differ from those in stroke patients. Third, if mu suppression in the motor area can be provided for real-time feedback in our BCI game, it may be more useful than providing only SSVEP data as feedback. However, this game is meaningful because it can provide feedback even in the case of undetectable or indecipherable mu signals in the motor area of patients with stroke. Technological advancements utilizing multifaceted systems are needed to provide simultaneous feedback through both mu signals and SSVEP. Forth, due to the small number of patients recruited and various locations of brain lesions, the effects of the BCI-AO game segregated by location of the brain lesion could not be analyzed. Further, the number of patients who had moderate to severe impairment was small, limiting the ability to analyze the precise correlation of mu suppression associated with functional impairment severity in each condition. Nevertheless, in the BCI-AO game condition, significant correlations between FMA and mu suppression were observed at C3/C4 and T3/T4 in the unaffected hemisphere, which suggests that the game may induce neural plasticity even in severely affected patients with stroke using the unaffected hemisphere during the recovery period. Finally, as the location and degree of brain activation varies depending on the type of action, a game showing various actions is needed. In future studies, different types of actions with different complexities should be employed; for example, a variety of objects of different sizes or a variety of upper extremity motions in ADLs could be presented.

Conclusions
The BCI-AO game is superior to conventional AO for facilitating Mu suppression in both affected and unaffected hemispheres of patients with stroke. Therefore, we suggest that the BCI-AO game paradigm may be an effective tool for the rehabilitation of stroke patients. Through multiple training sessions, the long-term effects of the BCI-AO game on functional improvements in patients with stroke should be examined in the future.