The Influence of Brain Activity on the Interactive Process through Biofeedback Mechanisms in Virtual Reality Environments ^†

Abstract

This work focuses on the development of a software link interface tool between the Looxid Link Device coupled to the HTC Vive Pro VR HeadSets and the Unity platform, to generate real-time interactivity in virtual reality applications. The software incorporates a dynamic and parameterizable algorithm to be used as a core-engine in the real-time Biofeedback process, recognizing the values of the biological signals registered in each of the EEG channels of the Looxid Link device. The values of EEG frequencies detected in real time can be used to generate elements of interactivity, with different frequencies and intensities.

1. Introduction

Exposure to immersive environments creates different sensations with different intensities in participants. The participant’s responses to the stimuli sent by the virtual environment reflect the effect it causes. During exposure, the participant’s brain activity is stimulated and changes in this activity can be measured by electroencephalography. The analysis of the results of the registered signals reflects the effect provoked on the participant [1,2].

Since the changes in brain activity are a participant’s response, usually involuntary, to the stimuli sent by the system, the interest in using them as an element of interaction becomes relevant. If the system can detect and interpret these values, it can use them to become more interactive, adapting its behavior to the participant’s reactions. Stimuli trigger emotions and emotions generate reactions. The inclusion of interactivity during the experience by real-time Biofeedback mechanisms increases the feeling of presence in the virtual environment [3].

The real-time data obtained on the user’s physiological aspects allows for determining how the stimuli affect him. On the other hand, when the user receives information in real time about a certain aspect of his physiology, he can determine how his mental changes can influence his state [4].

The introduction of biofeedback systems in the design of a simple immersive environment transform it into an Emotionally Adaptive Immersive Environment, where the user experience can be optimized through the continuously adaptation of stimuli to the user emotional state. The quantity and intensity of the stimuli are determined through an adaptive affective algorithm, which collect, interpret, and convert the user’s physiological data [5].

The main objective of this project is to develop a software tool that incorporates a dynamic and parameterizable algorithm to be used as a core engine in the real-time Bio-feedback process, this software tool is the element responsible for making the connection between the Looxid Link device coupled to VR HeadSet and the unity platform that manages the immersive environment.

2. Materials and Methods

2.1. Materials

Table 1. Hardware and software used to develop and implement the project.

Equipment	Specifications
Computer	CPU: Intel® CoreTM i7-9700K (3.60 GHz–4.90 GHz)
	Graphic card: NVIDIA® GeForce® RTX 2080 Ti
	Memory: 64 GB RAM
VR Headset HTC Vive ProTM	High resolution Dual AMOLED 3.5” diagonal screens
	1440 × 1600 pixels per eye (2880 × 1600 pixels combined)
	Refresh rate: 90 Hz; Field of view: 110 degrees
	Integrated microphones with 3D Spatial Audio
	Four SteamVR Base Station 2.0: 10 m × 10 m
	VIVE Wireless Adapter
Looxid LinkTM Mask for VIVE	EEG sensors; Looxid Link Hub
	6 channels: AF3, AF4, AF7, AF8, Fp1, Fp2
	1 reference: FPz at extended 10–10 system
	Dry electrodes on flexible PCB; Sampling rate: 500 Hz
	Resolution: 24 bits per channel (with 1 LSB = 0.27 $\mathsf{\mu }$V)
	Filtering: digital notch filters at 50 Hz and 60 Hz, 1–50 Hz
	Digital bandpass; Real-time data access
	Raw EEG data: 500 Hz (with/without filter options)
	Feature indexes (alpha, beta, gamma, theta, delta): 10 Hz
	Mind indexes (attention, relaxation, balance): 10 Hz
Software	Unity Personal 2020.3 LTS
	C# programming language

describes the hardware and software used to develop the project. For a non-evasive electroencephalogram device (Looxid Link^TM Mask for VIVE) coupled with VR Headsets HTC Vive Pro^TM, the software was developed using the C# programming language.

2.2. Methods

Unity3D software was used as a development and interconnection platform between the Looxid Link device coupled to VR HeadSet and the VR application. Using the C# programming language, an algorithm was developed to create a real-time Bio-feedback core-engine generator.

The algorithm recognizes the values of the biological signals registered in each of the EEG channels of the Looxid Link device; these values reflect the participant’s states of relaxation and attention, allowing the definition of value intervals, with the attribution to each one of interactive variables that can be used in the Biofeedback process.

The development of the algorithm included five aspects. Access the values of the EEG signals detected by the Looxid Link device through sensors AF3, AF4, AF7 AF8, Fp1, and Fp2, and Fpz. Group the values detected by each of the sensors into frequency ranges aggregating these values into two indicators, relaxation and attention. Assign an interaction variable to each frequency range. Assign the desired functions to each interaction variable.

Parameterize the period between readings of the values of the interaction variables allows for determining the desired degree of interactivity.

3. Results

The software incorporates a dynamic and parameterizable algorithm to be used as a core-engine in the real-time biofeedback process. The algorithm recognizes the values of the biological signals registered in each of the EEG channels of the Looxid Link device, identifying the participant’s states of relaxation and attention and allowing the definition of value intervals, with an interaction variable attributed to each one that can be used in the Biofeedback process by VR applications.

4. Discussion and Conclusions

One of the applications of the developed core-engine can focus on the participant’s states of relaxation and attention. The Looxid Link device determines these two states in real time during exposure, allowing them to be used as an interactive element. The algorithm uses these values and, according to predefined intervals, readjusts the intensity of the stimuli generated by the system or triggers new stimuli.

This feature has two important advantages. It increases the level of interactivity generated by the application, as this results from the real-time reading of the participant’s brain activity that was generated by the emitted stimuli, with the new stimuli generated adapting to the participant’s real reactions. The other advantage is the interoperability of the algorithm, which can be adapted to different virtual reality applications, simply by parameterizing the intervals according to the desired amount and sensitivity of the system. After this parameterization, it is possible to assign which stimuli are intended for each interaction variable.