Effects of Integrated Virtual Reality and Galvanic Vestibular Stimulation on Standing Balance

Abstract

Background/Objectives: Galvanic vestibular stimulation (GVS) integrated into virtual reality (VR) environments enhances immersion and mitigates cybersickness. It is well known that GVS can affect standing balance. Most studies have investigated the effects of GVS in VR in seated conditions. The purpose of this study was to evaluate the impact of joint GVS and VR with moving visual stimulus on standing balance. Methods: Using a repeated measures counter-balanced design, motion sickness, postural sway, and velocity utilizing the center of pressure (COP) along the mediolateral (ML) and anteroposterior (AP) axes were obtained in 18 subjects during optokinetic (OPK) stimulus (black and white vertical bars moving from left to the right) in VR across three interventions: GVS in the same direction of visual stimulus—left to right ear (Positive GVS), GVS in the opposite direction of visual stimulus—right to left ear (Negative GVS), and without GVS (Null GVS). Motion sickness symptom scoring was obtained using the Pensacola Diagnostic Index. Results: The PDI score was increased significantly in the Negative GVS. The root mean square and sway range of COP along ML was greater during the Positive GVS and Negative GVS than the Null GVS, while, along AP, it was only greater during Negative GVS. During Positive GVS, mean positive and negative peak velocities, only in ML, were increased and decreased, respectively. During Negative GVS, only negative peak velocities in both ML and AP directions were decreased. Conclusions: This research highlights the importance of testing combined VR and GVS to assess standing balance while mitigating cybersickness.

1. Introduction

Virtual reality (VR) technology places the user in a three-dimensional virtual environment using a visual display to create a highly interactive experience [1]. As immersive as the VR environment can be due to technological advancements, especially in the case of head-mounted displays (HMDs), it still primarily interacts with the visual system. This inevitably leads to significant motion sickness during VR exposure due to the realism without accompanying physical motion [2]. Motion sickness experienced in VR, termed “cybersickness,” is pervasive, with studies reporting that between 60 and 95% of all individuals who are exposed to VR through HMDs will be symptomatic to some degree [3,4,5]. It is widely believed that the sensory conflict between the visual system and a lack of vestibular and proprioceptive inputs is the cause of cybersickness [6]. An effect called oculo-vestibular decoupling is responsible, in which the lack of vestibular sensations of motion during a VR simulation does not match the ocular inputs [7]. This is a well-documented and persistent issue for immersive simulation, especially in aviation simulations [1,2,8,9,10]. Thus, integrating feelings of vestibular motion into VR environments is crucial for user comfort in VR.

Directional galvanic vestibular stimulation (GVS) is a method of stimulating the vestibular system into providing sensations of motion in the absence of actual motion. While there are multiple types of GVS, defined by how electrical current flows and affects the vestibular system, the current technology makes use of bipolar DC-like electrical stimulation applied to the head to alter the vestibular system’s afferent signals, affecting the efferent responses [11]. GVS has been used to ameliorate simulator sickness in non-immersive environments [7,12], but, more recently, it has been successfully used in VR environments to generate disorienting flight illusions [13] and to perform oculo-vestibular recoupling [10].

There have been recent advancements in VR and augmented reality (AR) environments that have users standing, reaching, and even ambulating. It is also well known that GVS use alone affects standing sway [11,14], but there is a lack of studies that have investigated the impact of integrated GVS and VR on standing balance. Understanding how standing balance is affected is a critical safety measure in the pathway to integrating motion sensation into the virtual reality environment. In this study, we are investigating the effects of combined directional GVS and vection-inducing VR on standing balance, while also assessing motion sickness severity.

This is an exploratory study meant to investigate the joint effects of motion-inducing visual and vestibular stimuli in VR on standing balance. Hence, we used a clinically proven optokinetic (OPK) visual stimulus to inhibit visual fixation and promote visual stimulation [15,16] in the VR headset.

2. Materials and Methods

2.1. Subjects

Eighteen participants (9 male, 9 female) enrolled in this study, which was approved by the Mayo Clinic Institutional Review Board (IRB). All participants were between 18 and 55 years of age with no history of balance disorders, migraines, severe motion sensitivity, or eye movement disorders. A negative urine pregnancy test was required for female participants. Informed consent was obtained from all participants prior to study enrollment, in accordance with Mayo Clinic’s IRB regulations. Participant demographics had the mean values (+ standard deviation (SD)) of age (32 + 8 years), height (68 + 4 in), and weight (166 + 37 lbs). Each participant also completed a clinical questionnaire prior to taking part in the research study that assessed motion sickness susceptibility; on a scale of 0–4 (0 = not susceptible, 4 = highly susceptible), the average score was 0.5. VR experience was not assessed prior to this research appointment, as the OPK stimulus they were exposed to was abstract in nature with low presence and had little similarity to realistic VR environments used in entertainment and gaming.

2.2. Equipment

The galvanic vestibular stimulator developed for this study consisted of two bipolar electrodes on the mastoid processes (i.e., behind each ear) to provide directional galvanic vestibular stimulation (GVS) in right and left yaw directions [7]. A 3-degrees-of-freedom (DOF) Logitech Freedom 2.4 GHz joystick (Logitech, San Jose, CA, USA) held by the investigator was used to control the onset and intensity of the GVS stimulation delivered. The angular displacement of the joystick along the yaw axis was inputted to the GVS system to generate real-time vestibular stimulation delivered to the participant. The maximum amplitude of GVS was set to 2 mA. The entire range of joystick movement from the resting position to the farthest position, in the left or right direction, was proportionally matched to the range of electric current from 0 to 2 mA in the corresponding direction. For example, joystick movement from resting to the farthest right position delivered the 2 mA GVS current from left ear to right ear to provide right yaw self-motion perception. Conversely, the joystick movement from resting to the farthest left position delivered the 2 mA GVS current from right to left ear to provide left yaw self-motion perception.

A SteamVR virtual reality headset (HTC Corporation, Bellevue, WA, USA) had the OPK stimulus projected onto the visual field. The stimulus consisted of black and white vertical bars that moved from left to right at a consistent speed and created a circular vection effect [15,16]. The Bertec Portable Essential’s dual-balance platform (Bertec, Columbus, OH, USA) was used for measuring standing balance parameters during GVS and VR exposure. The platform consisted of force plates that measure the forces exerted by a participant’s feet. The force plate recorded the center of pressure (COP) sway kinetic data and other balance-related metrics at a sampling frequency of 1000 Hz. The side-to-side sway along the x-axis indicated mediolateral (ML) movements on the platform, while the back-and-forth sway along y-axis indicated anteroposterior (AP) movements.

2.3. Procedures

The experiment occurred in a quiet climate-controlled room in the Aerospace Medicine and Vestibular Research Laboratory (AMVRL) at Mayo Clinic, Arizona. Two electrodes were placed on the two mastoid processes (left and right) to deliver the electric currents through the galvanic stimulator. The VR headset was put on participants in the seated position. Before putting on the VR headset, participants were familiarized with the force plate placed one step away from their resting feet while sitting on the chair. For all the tests, the participant stood up and stepped onto a Bertec Portable Essential dual-balance force plate system to capture their standing sway and posture metrics while wearing the headset. The experimenters were near the participants to guide them as they stepped on the force plate platform and helped them align their feet correctly.

The OPK immersive video was presented in a VR headset while the participant stood on the Bertec dual-balance force plate that continuously measured standing sway for the 10 s duration of the moving stimulus video. All participants were tested in three separate conditions in a single experimental session. In the Null GVS (control) condition, OPK moving visual stimulus in the form of black and white vertical bars from left to right was presented with no accompanying GVS. In the Positive GVS condition, GVS 2 mA current was applied in the same direction (left to right ear) as the OPK moving visual stimulus from left to right. In the Negative GVS condition, GVS 2 mA current was applied in the opposite direction (right to left ear) to the OPK moving visual stimulus from left to right. The GVS was turned on 1–2 s after the onset of the OPK visual stimulus and turned off after the OPK stimulus was over and the force plate stopped recording the balance data.

After each condition, the participant was guided back to a seated position in the chair and had a sufficient wash-out period between each condition to minimize the carryover of visual or vestibular effects. During this wash-out period, motion sickness symptom scoring was obtained using the Pensacola Diagnostic Index (PDI). The PDI provides an acute score derived using diagnostic criteria introduced by Graybiel et al. [17] by obtaining the subjective intensity of eight different modalities of symptoms (nausea, skin pallor, sweating, salivation, drowsiness, headache, and dizziness/vertigo) and signs reported on a “slight/moderate/severe” basis used to derive a weighted “malaise index”. The order of conditions was counterbalanced across participants to control for training effects. In the Null GVS condition, electrodes were placed in the same positions behind each ear, but the GVS remained off for the duration of the condition. During the experiment, participants were told that if they experienced significant motion sickness or discomfort, they could close their eyes and notify the experimenters, and the trial would be halted. No participant reported closing their eyes during any of the conditions.

2.4. Data Analysis

After every condition (Null GVS, Positive GVS, and Negative GVS), PDI scores were recorded. The purpose of this study is to evaluate body balance by quantifying postural sway with the participant in the standing position of a force platform. This involved measuring excursions of the center of pressure (COP) and mean velocities, both along the mediolateral (ML—x-axis) and anteroposterior (AP—y-axis) axes. The raw kinetic COP data recorded at the sampling frequency of 1000 Hz using the calibrated Bertec force plate were centered to remove the variability of foot positioning on the force platform. The centered COP data were filtered using a standard fourth-order low-pass Butterworth filter with a 20 Hz cutoff frequency [18]. Then, the filtered data were resampled at 25 Hz to extract displacement and velocity-related COP features such as root mean square, sway range, mean speed, mean peak positive velocity, and mean peak negative velocity along ML and AP axes for each of the three conditions. The centering, filtering, resampling, and extraction of features were performed using MATLAB (version 2023b, MathWorks Inc., Natick, MA, USA). A statistical analysis of extracted features was conducted using SPSS Statistics (version 28, IBM Corp., Armonk, NY, USA). Shapiro–Wilk tests for normality were performed for each feature. For features that violated the assumption of normality, non-parametric comparison testing was conducted using the Friedman test followed by the pairwise comparisons of the post hoc analysis using the Wilcoxon Signed Rank test. For features that did not violate the assumption of normality, comparison testing was conducted using the one-way repeated measure ANOVA followed by the pairwise comparisons of the post hoc analysis using the t-test. An alpha error level of 0.05 was used as our test criterion. For multiple comparisons, post hoc analysis was conducted using the Bonferroni correction.

3. Results

All 18 participants in the study completed the experiment. There were no adverse effects or dermal discomfort due to GVS stimulation for currents up to 2 mA reported in this study.

3.1. Motion Sickness Symptom Reporting

After testing each condition in VR with OPK stimulus with and without GVS, the overall severity of sickness was assessed by a Pensacola Diagnostic Index (PDI) single score derived from the eight different modalities of symptoms and signs discussed previously. There was a statistically significant difference in PDI symptom level depending on which type of GVS (or no GVS) was applied in VR with OPK moving visual stimulus, χ2(2) = 7.9, p = 0.019. There were no significant differences in PDI symptom level between the Null GVS and Positive GVS (Z = −1.46, p = 0.143) or between the Positive GVS and Negative GVS (Z = −1.069, p = 0.285) despite an overall increment in PDI score during Negative GVS vs Positive GVS. However, there was a statistically significant increase in PDI score in the Negative GVS vs. Null GVS (Z = −2.636, p = 0.008).

3.2. COP Sway and Displacement

The integration of GVS in an immersive VR environment enhances self-motion perception and affects sway and balance, especially in standing positions. The principal sway direction [19] metric represents the relative contribution of the ML and AP components to the excursions of the COP. The major sway direction during a given duration is computed using the principal component analysis (PCA) technique of the COP bi-dimensional signal (x, y) in the ML-AP plane. Figure 1 shows the sample scatter plot of COP signals in the ML-AP plane for a participant standing on the force platform for 10 s with VR and OPK visual stimulus going in the left to right direction and simultaneously with Positive/Negative GVS or without GVS. Also, as seen in Figure 1a, the principal sway direction is the angle (θ) between the AP axis and the direction of the main eigenvector computed by the PCA, which indicates the direction of the highest variance or maximum dispersion of the COP.

Figure 1b shows the overall average response of principal sway direction across 18 participants for all three conditions. OPK in VR without GVS (green segment) did not show any significant sway along the ML axis and is close to the vertical AP axis. However, Positive GVS (blue segment) that provides yaw right self-motion perception in the same direction of the OPK visual stimulus caused a significantly larger sway towards the right direction as compared to Null (green segment) and Negative GVS (red segment). Interestingly, Negative GVS that provides yaw left self-motion perception in the opposite direction of the OPK visual stimulus did not cause sway towards the left direction with that amount of magnitude as compared to its counterpart.

The root mean square (RMS) of the centered COP data corresponds to the standard deviation along the trajectory of the ML and AP axes. There was a statistically significant group difference in ML RMS values depending on which type of GVS or no GVS was applied in VR with OPK moving visual stimulus, χ2(2) = 16.35, p < 0.001 (Figure 2). Median (IQR) ML RMS for the Null GVS, Positive GVS, and Negative GVS were 0.13 (0.07 to 0.18) cm, 0.22 (0.19 to 0.44) cm, and 0.18 (0.13 to 0.26), respectively. There were no significant differences between the Positive GVS and Negative GVS (Z = 1.4, p = 0.163), despite an overall reduction in ML RMS in the Negative GVS compared to the Positive GVS condition. There was a statistically significant increment in ML RMS during the Positive GVS vs. Null GVS condition (Z = 3.57, p < 0.001) and during the Negative GVS vs. Null GVS condition (Z = 2.63, p = 0.009). The RMS values along the AP axis (Figure 2) differed statistically significantly between conditions (F = 4.95, p = 0.013, η2p = 0.24). Post hoc analysis with a Bonferroni adjustment revealed that AP RMS was statistically significantly increased from Negative GVS to Null GVS (0.15 (95% CI, 0.04 to 0.27) cm, p = 0.008), but not from Positive GVS to Null GVS (0.09 (95% CI, −0.06 to 0.25) cm, p = 0.348). The AP RMS during Negative GVS was greater compared to Positive GVS, but was not statistically significant (0.06 (95% CI, −0.07 to 0.18) cm, p = 0.673).

There was a statistically significant group difference in the ML sway range (χ2(2) = 18.88, p < 0.001) and AP sway range (χ2(2) = 7.44, p = 0.024) across the three conditions (Figure 3). Along the ML axis, the sway range during Positive GVS (Z = 3.26, p = 0.001) and Negative GVS (Z = 3.1, p = 0.002) was statistically significantly larger as compared to the Null GVS, while, along the AP axis, only Negative GVS showed a statistically significant increment in the sway range (Z = 2.76, p = 0.006) compared to the Null GVS.

3.3. COP Sway Velocity

A repeated measures ANOVA without violating the assumption of sphericity determined that the mean sway speed in the ML direction differed statistically significantly (Figure 4) between the three conditions (F = 6.84, p = 0.004, η2p = 0.31). Post hoc analysis with a Bonferroni adjustment revealed that the ML mean sway speed was statistically significantly increased from Positive GVS to Null GVS (0.26 (95% CI, 0.023 to 0.49) cm/s, p = 0.029), and also from Negative GVS to Null GVS (0.3 (95% CI, 0.031 to 0.574) cm, p = 0.027). The ML mean speed during Negative GVS was greater compared to Positive GVS, but was not statistically significant (0.05 (95% CI, −0.15 to 0.25) cm, p = 0.8). The mean sway speed in the AP direction showed weakly significant group differences across conditions (F = 3.46, p = 0.04, η2p = 0.17), but there were no significant pairwise differences between Null GVS, Positive GVS, and Negative GVS after Bonferroni correction (Figure 4).

Along the ML direction, both mean positive peak velocity (F = 5.95, p = 0.006, η2p = 0.27) and mean negative peak velocity (F = 8.79, p < 0.001, η2p = 0.36) differed statistically significantly between the three conditions (Figure 5a,b). Post hoc analysis with a Bonferroni adjustment revealed that both ML mean positive and negative peak velocities were statistically significantly increased and decreased, respectively, from Positive GVS to Null GVS (0.27 (95% CI, 0.062 to 0.475) cm/s, p = 0.009 and −0.34 (95% CI, −0.55 to −0.13) cm/s, p = 0.002). However, only the mean negative peak velocity was statistically significantly decreased from Negative GVS to Null GVS (−0.24 (95% CI, −0.47 to −0.006) cm, p = 0.04). Along the AP direction, only the mean negative peak velocity differed statistically significantly between the three conditions (F = 5.2, p = 0.01, η2p = 0.24). Post hoc analysis with a Bonferroni adjustment revealed that the AP mean negative peak velocity was statistically significantly decreased from Negative GVS to Null GVS (−0.43 (95% CI, −0.79 to −0.06) cm/s, p = 0.02).

4. Discussion

Visual, vestibular, proprioceptive, and cognitive inputs must converge at the level of the brainstem and cortex to successfully maintain standing balance in healthy individuals. The main contributors are linear and angular vestibular inputs from the semicircular canals and otolithic organs, proprioceptive inputs (especially from the ankle angle for on-feet balance), and visual inputs. In a healthy system, all inputs are synthesized to maintain quiet stance [20]. Posturography is the process of quantifiably analyzing postural control variables to assess how a patient’s balance is compared to normal references and how a person’s activities of daily living could be affected by their balance [21]. Persistent issues meaningfully describing sway parameters emanate from the sheer complexity of the quantification process and the high number of variables used to describe balance and center of pressure (COP). Quijoux and colleagues [22] completed a systematic review in which they proposed a compendium of definitions of COP variables that are most used in the literature surrounding balance and falls. In a series of tables, each feature, in mediolateral (X) and anteroposterior (Y) planes, is defined and its formula and units are included with the feature.

This study assessing standing balance in healthy individuals with moving visual stimulus in VR and GVS conditions produced a few significant findings. There was a statistically significant increase in the PDI score in the Negative GVS condition when compared to the Null GVS condition (Z = −2.636, p = 0.008) as compared to no significant increase in Positive GVS from the Null GVS condition (Z = −1.46, p = 0.143). This shows that the Negative GVS, which is in the opposite direction of the OPK visual stimulus, may have caused more disorientation, leading to an increase in the overall motion sickness severity. This result is consistent with the previous work, where mismatched GVS to the flight simulation scenarios in VR was used to simulate disorienting flight illusions [13]. On the other hand, Negative GVS, which is supposed to provide yaw left sway/self-motion perception, did not cause sway towards the left direction with the same degree of magnitude when compared to the Positive GVS condition. This reiterates the importance of re-coupling GVS with the visual input to increase the fidelity and immersion in VR environments that cause consistent self-motion perception following the visual field and have the potential to mitigate cybersickness [7,10,12]. Since both VR and GVS technologies are portable and inexpensive, these findings are applicable and beneficial to develop low-cost military training simulations, where military personnel can receive refresher training by standing in a remote location within an evolving mission. With wireless VR headsets, GVS integration can make gaming and video entertainment enhancements feasible to be experienced by standing or even by moving around in the space. To extend the current findings of this work, the next step is to test the integration of GVS in a sophisticated VR environment where participants are required to interact or perform tasks by standing or physically moving within the environment.

Compared with the controlled condition of Null GVS, the RMS, sway range, and mean negative peak velocity were significantly larger during Negative GVS along the AP axes, but not during Positive GVS. This indicates that the GVS in the opposite direction of the moving visual stimulus (both along the x-axis) causes more disruption in orthogonal axes (y-axis) than the GVS moving in the same direction of the visual stimulus. This might lead to disorientation, causing higher motion sickness PDI scores in Negative GVS from the Null GVS condition.

One limitation of this study was not including a GVS-only control condition to compare the effects of GVS without OPK stimulation to the Positive GVS and Negative GVS conditions. Multiple previous studies, however, using bilateral mastoid GVS montages alone, have demonstrated ML tilt [11,14,23,24]. When an individual is exposed to GVS with a bilateral mastoid montage, the spontaneous firing patterns of the vestibular afferents are modified, as a higher frequency is stimulated on the cathodal side and a lower frequency on the anodal side [23,24]. Thus, a person can feel that they are rolling to the right or left in the ML plane (x-axis). Studies on the effects of yaw GVS alone on standing balance and posture show that, when the user’s feet are planted together with the head facing forward, the body will lean to the side of the anode on the ML plane after approximately two seconds of stimulation [14]. When the GVS stimulation is lifted, the body will return to a fully upright position [11,24]. This compensatory response is to stabilize the body in response to the vestibular inputs being received from GVS. In the current Negative GVS paradigm, where the moving visual stimulus was sent in the opposite direction of the GVS, the resulting disorientation is shown in the orthogonal plane with negative AP (y-axis) values representing backwards tilt. In this experiment, all stimulation was sent along the ML plane, and there was no force inducing participants to sway along the AP axis. Only in the Negative GVS condition was there significant AP change, which was not mirrored in the Positive GVS condition. This warrants future studies on GVS-only stimuli that stimulate self-motion perception in different axes to investigate the relationship between vestibular and standing balance to advance the integration of GVS technology in the future VR industry. Further, for the effectiveness of the future widespread adoption of these findings, large-scale studies need to be designed to broaden the participant pool, including individuals with varying VR experience levels and wider demographic diversity.

5. Conclusions

To the best of our knowledge, this is one of the first studies to elucidate the effects of GVS with moving visual stimulus to assess standing balance in VR. There are technological and clinical implications for this study’s findings. First, it is inevitable that GVS will be integrated with VR in a consumer environment, whether it be in a military, aerospace, medical, or entertainment sphere. Augmented reality (AR), where the objects and elements in a person’s real setting are integrated into a VR environment, is increasing in popularity, and users are encouraged to stand and move to interact with the technology. The issue of cybersickness is still prevalent in these conditions, encouraging the integration of GVS with VR to promote oculo-vestibular recoupling; this study lays the groundwork, however, for further investigations into the proprioceptive and postural effects of these joint stimulations to avoid injury to participants. Clinically, VR is emerging as a tool that can be used in the treatment and rehabilitation of balance disorders, including compensation for bilateral and unilateral vestibular losses [25,26,27]. Patients with vestibular weaknesses can complete interactive training in VR to expose themselves to complex visual inputs and overcome difficulties in synthesizing inputs to maintain balance. Such immersive training can place patients in varying complex visual conditions while under the supervision of their vestibular rehabilitators to encourage vestibular compensation. This study emphasizes the importance of understanding the disorienting effects that joint GVS and VR can have on standing balance and, while the clinical utility is immense, safety precautions must be taken. As both the market and technology grow for this kind of tool, the current research is increasingly valuable to inform providers and patients alike about the unexpected effects on balance that disorienting visual and vestibular conditions can have. Future research will be completed with more realistic moving VR environments, like flight simulation, to provide a more ecologically valid user experience.

6. Patents

Gaurav N. Pradhan, Michael J. Cevette, and Jan Stepanek, system and method for integrating three-dimensional video and galvanic vestibular stimulation, US Patent 11,904,165 B2, filed 19 December 2016, and issued 17 May 2022.