A Pilot Study on Emotional Equivalence Between VR and Real Spaces Using EEG and Heart Rate Variability
Abstract
1. Introduction
1.1. Background
1.2. Challenges
2. Proposal
2.1. Objective and Methodology
2.2. The Pair of Identically Designed VR and Real Spaces
2.3. Evaluation Method
2.3.1. Cognitive Evaluation: Semantic Differential (SD)
2.3.2. Electroencephalography (EEG)
2.3.3. Heart Rate Variability (HRV)
2.4. Estimation and Visualization of Emotion Using EEG and HRV
3. Experiment Method
3.1. Overview
3.2. Equipment and Spaces Used
3.3. Experiment Place
3.4. Experimental Procedure
- (1)
- The participant wears the ECG sensor and, while blindfolded to avoid seeing the space, moves to the measurement chair at the designated location.
- (2)
- After sitting in a chair surrounded by partitions to prevent visibility of the real space, the participant is fitted with the EEG device. If the VR condition is presented, the HMD is also worn.
- (3)
- A two-minute resting period is conducted.
- (4)
- The participant views the stimulus space for two minutes.
- (5)
- If the HMD was worn, it is removed, and the participant completes the questionnaire.
- (6)
- If the VR space was presented in steps (3) and (4), the same procedure is repeated using the real space, and vice versa.
- (7)
- The participant is blindfolded and moved to a different measurement location.
- (8)
- Steps (2) to (6) are repeated at the new location.
3.5. Participants
4. Experimental Analysis and Result
4.1. Statistical Analysis and Evaluation Overview
- Subjective Evaluation of Emotional Responses: Subjective emotional responses were assessed using the Semantic Differential (SD) method, a widely used cognitive impression technique. The results of this analysis are detailed in Section 4.2.
- Physiological Evaluation—Brainwaves: In Section 4.3, we evaluated physiological responses not consciously perceived by participants but elicited by the spatial environments. First, we conducted a basic assessment of alpha and beta brainwave activity. Alpha waves are typically associated with relaxation, whereas beta waves are linked to attention and arousal. To quantify the balance between these states, we computed the β/α ratio, comparing values across the VR and real-world conditions to determine the dominant emotional state.
- Memory-Related Brain Activity: As discussed in Section 2.3.2, theta waves are implicated in memory encoding, retrieval, and working memory maintenance [34,49,50]. We also analyzed gamma wave activity—subdivided into low gamma (30–50 Hz) and high gamma (50–70 Hz)—to investigate sensory processing and cognitive load [34,36]. The upper bound for high gamma was limited to 70 Hz to minimize artifact contamination.
- Autonomic Response—Heart Rate Variability (HRV): Following the method described in Section 2.4, we analyzed pNN50, an HRV index strongly linked to parasympathetic nervous activity. Higher pNN50 values reflect relaxed states, whereas lower values indicate heightened psychological stress. Accordingly, temporal variations in pNN50 were examined as physiological indicators of emotional change during spatial experiences. We compare the results using this index.
- Emotion Mapping in Affective Space: To enhance interpretability, we constructed a two-dimensional Emotion Map, using pNN50 to represent valence and the β/α ratio to represent arousal. This approach allowed for the visual mapping of emotional states elicited by each spatial condition (see Section 2.4). Additionally, gender-based comparisons were conducted to explore possible differences in emotional responses.
- Temporal Analysis and Signal Processing: To observe time-series changes in autonomic nervous activity between rest and spatial exposure, we generated time-series plots of HRV indicators.
4.2. Subjective Evaluation of Emotional Responses
4.3. Physiological Evaluation—Brainwaves
4.4. Memory-Related Brain Activity
4.5. Emotion Mapping in Affective Space
Autonomic Response—Heart Rate Variability (HRV)
4.6. Emotion Mapping in Affective Space
- (1)
- Based on the average evaluation
- (2)
- Focus on the gender-based evaluation
4.7. Temporal Analysis and Signal Processing
5. Discussion
5.1. Contributions
5.2. Limitations and Future Work
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Lanier, J. Virtual Reality: The Promise of the Future. Interact. Learn. Int. 1992, 8, 275–279. [Google Scholar]
- Furness, T.A. The Super Cockpit and Its Human Factors Challenges. Proc. Hum. Factors Soc. Annu. Meet. 1986, 30, 48–52. [Google Scholar] [CrossRef]
- Marín-Morales, J.; Higuera-Trujillo, J.L.; Greco, A.; Guixeres, J.; Llinares, C.; Gentili, C.; Scilingo, E.P.; Alcañiz, M.; Valenza, G. Real vs. immersive-virtual emotional experience: Analysis of psycho-physiological patterns in a free exploration of an art museum. PLoS ONE 2019, 14, e0223881. [Google Scholar] [CrossRef] [PubMed]
- Menck, J.H.D.; Lechte, H.; Riedel, M.; Jaja, J.H.C.; Tümler, J. Realism and Experiments: Investigating Virtual Reality Experiments. Rising like a Phoenix: Emerging from the Pandemic and Reshaping Human Endeavors with Digital Technologies. In Proceedings of the International Conference on Information Systems, ICIS 2023, Hyderabad, India, 10–13 December 2023. [Google Scholar]
- Chen, B.; Wang, Y.; Wang, L. The Effects of Virtual Reality-Assisted Language Learning: A Meta-Analysis. Sustainability 2022, 14, 3147. [Google Scholar] [CrossRef]
- Kahlon, S.; Lindner, P.; Nordgreen, T. Virtual Reality Exposure Therapy for Adolescents with Fear of Public Speaking: A Non-Randomized Feasibility and Pilot Study. Child Adolesc. Psychiatry Ment. Health 2019, 13, 47. [Google Scholar] [CrossRef] [PubMed]
- Mateer, J. Directing for Cinematic Virtual Reality: How the Traditional Film Director’s Craft Applies to Immersive Environments and Notions of Presence. J. Media Pract. 2017, 18, 14–25. [Google Scholar] [CrossRef]
- International Interior Design Association. Available online: https://iida.org/about/what-is-interior-design (accessed on 26 February 2025).
- International Federation of Interior Architects/Designers. Available online: https://ifiworld.org/wp-content/uploads/2020/09/IFI-Interiors-Declaration_ENGLISH_rev-seal-2020.pdf (accessed on 26 February 2025).
- Ching, F.D.K. Interior Design Illustrated; Wiley: New York, NY, USA, 2014. [Google Scholar]
- Pile, J. A History of Interior Design; Wiley: New York, NY, USA, 2007. [Google Scholar]
- Norman, D. The Design of Everyday Things; Basic Books: New York, NY, USA, 2013. [Google Scholar]
- Kopec, D. Environmental Psychology for Design; Fairchild Books: New York, NY, USA, 2012. [Google Scholar]
- McLennan, J.F. The Philosophy of Sustainable Design; Ecotone: Kansas City, MO, USA, 2004. [Google Scholar]
- Kilmer, R.; Kilmer, W.O. Construction Drawings and Details for Interiors; Wiley: New York, NY, USA, 2014. [Google Scholar]
- Bouchlaghem, D. Virtual Reality and Augmented Reality in Architecture and Design; Springer: Berlin/Heidelberg, Germany, 2012. [Google Scholar]
- Strand, I. Virtual Reality in Design Processes: A Literature Review of Benefits, Challenges, and Potentials. FormAkademisk 2020, 13, 1–19. [Google Scholar] [CrossRef]
- Xiang, J.; Mansor, N.; Azmi, A. Effects of VR-Based Interactive Interior Design on the Emotional and Mental Health of Office Workers. J. Asian Archit. Build. Eng. 2024, 1–14. [Google Scholar] [CrossRef]
- Wu, S.; Han, S. System Evaluation of Artificial Intelligence and Virtual Reality Technology in the Interactive Design of Interior Decoration. Appl. Sci. 2023, 13, 6272. [Google Scholar] [CrossRef]
- Kurmann, D.; Elte, N.; Engeli, M. Real-time modeling with architectural space. In CAAD Futures 1997, Proceedings of the 7th International Conference on Computer Aided Architectural Design Futures, Munich, Germany, 4–6 August 1997; Kluwer Academic Publishers: Norwell, MA, USA, 1997; pp. 809–819. [Google Scholar]
- Fukuda, T.; Nagahama, R.; Nomura, J. Networked VR system: Kitchen layout design for customers. In Proceedings of the 2nd Symposium on Virtual Reality Modeling Language VRML 97, Monterey, CA, USA, 24–26 February 1997; pp. 93–100. [Google Scholar]
- Dorta, T.; Lalande, P. The Impact of Virtual Reality on the Design Process. In Proceedings of the Digital Design Studios: Do Computers Make a Difference? Acadia ’98, Québec City, QC, Canada, 22–25 October 1998; pp. 138–161. [Google Scholar]
- Levenson, R.W. Stress and illness: A role for specific emotions. Psychosom. Med. 2019, 81, 720–730. [Google Scholar] [CrossRef]
- Llinares, C.; Higuera-Trujillo, J.L.; Montañana, A. A Comparative Study of Real and Virtual Environment via Psychological and Physiological Responses. Appl. Sci. 2024, 14, 232. [Google Scholar] [CrossRef]
- Marín-Morales, J.; Higuera-Trujillo, J.L.; Guixeres, J.; Llinares, C.; Alcañiz, M.; Valenza, G. Heart rate variability analysis for the assessment of immersive emotional arousal using virtual reality: Comparing real and virtual scenarios. PLoS ONE 2021, 16, e0254098. [Google Scholar] [CrossRef]
- Li, M.; Pan, J.; Li, Y.; Gao, Y.; Qin, H.; Shen, Y. Multimodal Physiological Analysis of Impact of Emotion on Cognitive Control in VR. IEEE Trans. Vis. Comput. Graph. 2024, 30, 2044–2054. [Google Scholar] [CrossRef]
- Kuhne, C.; Kecelioglu, E.D.; Maltby, S.; Hood, R.J.; Knott, B.; Ditton, E.; Walker, F.R.; Kluge, M.G. Direct comparison of virtual reality and 2D delivery on sense of presence, emotional and physiological outcome measures. Front. Virtual Real. 2023, 4, 1211001. [Google Scholar] [CrossRef]
- Osgood, C.E.; Suci, G.J.; Tannenbaum, P.H. The Measurement of Meaning; University of Illinois Press: Urbana, IL, USA, 1957. [Google Scholar]
- Hariyady, H.; Ibrahim, A.A.; Teo, J.; Suharso, W.; Barlaman, M.B.F.; Bitaqwa, M.A.; Ahmad, A.; Yassin, F.M.; Salimun, C.; Weng, N.G. Virtual reality and emotional responses: A comprehensive literature review on theories, frameworks, and research gaps. ITM Web Conf. 2024, 63, 01022. [Google Scholar] [CrossRef]
- Mauss, I.B.; Robinson, M.D. Measures of emotion: A review. Cogn. Emot. 2009, 23, 209–237. [Google Scholar] [CrossRef]
- Shu, L.; Xie, J.; Yang, M.; Li, Z.; Li, Z.; Liao, D.; Xu, X.; Yang, X. A review of emotion recognition using physiological signals. Sensors 2018, 18, 2074. [Google Scholar] [CrossRef]
- Bradley, M.M.; Lang, P.J. Measuring emotion: The self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 1994, 25, 49–59. [Google Scholar] [CrossRef]
- Mahakwe, G.; Johnson, E.; Karlsson, K.; Nilsson, S. A systematic review of self-report instruments for the measurement of anxiety in hospitalized children with cancer. Int. J. Environ. Res. Public Health 2021, 18, 1911. [Google Scholar] [CrossRef]
- Burke, J.F.; Zaghloul, K.A.; Jacobs, J.; Williams, R.B.; Sperling, M.R.; Sharan, A.D.; Kahana, M.J. Synchronous and Asynchronous Theta and Gamma Activity during Episodic Memory Formation. J. Neurosci. 2013, 33, 292–304. [Google Scholar] [CrossRef]
- Keizer, A.W.; Verment, R.S.; Hommel, B. Enhancing cognitive control through neurofeedback: A role of gamma-band activity in managing episodic retrieval. NeuroImage 2010, 49, 3404–3413. [Google Scholar] [CrossRef] [PubMed]
- Ossandón, T.; Vidal, J.R.; Ciumas, C.; Jerbi, K.; Hamamé, C.M.; Dalal, S.S.; Bertrand, O.; Minotti, L.; Kahane, P.; Lachaux, J.-P. Efficient “Pop-Out” Visual Search Elicits Sustained Broadband Gamma Activity in the Dorsal Attention Network. J. Neurosci. 2012, 32, 3414–3421. [Google Scholar] [CrossRef]
- Yang, K.; Tong, L.; Shu, J.; Zhuang, N.; Yan, B.; Zeng, Y. High Gamma Band EEG Closely Related to Emotion: Evidence from Functional Network. Front. Hum. Neurosci. 2020, 14, 89. [Google Scholar] [CrossRef]
- Zhao, J.; Nagai, Y.; Gao, W.; Shen, T.; Fan, Y. The effects of interior materials on the restorativeness of home environments. Int. J. Environ. Res. Public Health 2023, 20, 6364. [Google Scholar] [CrossRef]
- Nikookar, N.; Sawyer, A.O.; Goel, M.; Rockcastle, S. Investigating the impact of combined daylight and electric light on human perception of indoor spaces. Sustainability 2024, 16, 3691. [Google Scholar] [CrossRef]
- Kobayashi, T.; Jadram, N.; Sugaya, M. Evaluation of the effect of three-dimensional shape in VR space on emotion using physiological indexes. In Virtual, Augmented and Mixed Reality—16th International Conference, VAMR 2024, Held as Part of the 26th HCI International Conference, HCII 2024; Proceedings; Chen, J.Y.C., Fragomeni, G., Eds.; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2024; Volume 14706, pp. 213–223. [Google Scholar]
- Wang, M.-F.; Hsiao, S.-W. A Kansei-oriented method for interior design. In Proceedings of the 22nd International Conference on Computer Supported Cooperative Work in Design (CSCWD 2018), Nanjing, China, 9–11 May 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 485–490. [Google Scholar]
- Tian, F.; Wang, X.; Cheng, W.; Lee, M.; Jin, Y. A comparative study on the temporal effects of 2D and VR emotional arousal. Sensors 2022, 22, 8491. [Google Scholar] [CrossRef]
- Cahill, L.; McGaugh, J.L. A Novel Demonstration of Enhanced Memory Associated with Emotional Arousal. Conscious. Cogn. 1995, 4, 410–421. [Google Scholar] [CrossRef] [PubMed]
- Verhoeven, J.W.M.; Van Rompay, T.J.L.; Pruyn, A.T.H. Let Your Workspace Speak for Itself: The Impact of Material Objects on Impression Formation and Service Quality Perception. Adv. Consum. Res. 2007, 34, 669–674. [Google Scholar]
- Rolland, J.P.; Hua, H. Enabling Focus Cues in Head-Mounted Displays. Proc. IEEE 2016, 105, 805–824. [Google Scholar]
- Kim, S.; Dowgwillo, E.A.; Kratz, A.L. Examining the Dynamic Relationship between Positive and Negative Emotions as a Function of Pain, Fatigue, and Stress in Individuals with and without Fibromyalgia. J. Pain 2022, 23, 56–57. [Google Scholar] [CrossRef]
- Reali, P.; Cosentini, C.; de Carvalho, P.; Traver, V.; Bianchi, A.M. Towards the development of physiological models for emotions evaluation. In Proceedings of the 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, HI, USA, 18–21 July 2018; pp. 110–113. [Google Scholar]
- Fourcade, A.; Klotzsche, F.; Hofmann, S.M.; Mariola, A.; Nikulin, V.V.; Villringer, A.; Gaebler, M. Linking brain–heart interactions to emotional arousal in immersive virtual reality. Psychophysiology 2024, 61, e14696. [Google Scholar] [CrossRef]
- Enriquez-Geppert, S.; Huster, R.J.; Figge, C.; Herrmann, C.S. Self-regulation of frontal-midline theta facilitates memory updating and mental set shifting. Front. Behav. Neurosci. 2014, 8, 420. [Google Scholar] [CrossRef] [PubMed]
- Werkle-Bergner, M.; Müller, V.; Li, S.-C.; Lindenberger, U. Cortical EEG correlates of successful memory encoding: Implications for lifespan comparisons. Neurosci. Biobehav. Rev. 2006, 30, 839–854. [Google Scholar] [CrossRef]
- IEEE Standard 10186304; IEEE Recommended Practice for Electroencephalography (EEG) Neurofeedback Systems, 2010–2023. IEEE: Piscataway, NJ, USA, 2023. Available online: https://ieeexplore.ieee.org/document/10186304 (accessed on 26 June 2025).
- Shaffer, F.; Ginsberg, J.P. An overview of heart rate variability metrics and norms. Front. Public Health 2017, 5, 258. [Google Scholar] [CrossRef] [PubMed]
- Bong, S.Z.; Murugappan, M.; Yaacob, S. Analysis of electrocardiogram (ECG) signals for human emotional stress classification. In Communications in Computer and Information Science, Proceedings of the International Conference on Intelligent Robotics, Automation and Manufacturing (IRAM 2012), Kuala Lumpur, Malaysia, 28–30 November 2012; Ponnambalam, S.G., Ed.; Springer: Berlin/Heidelberg, Germany, 2012; Volume 330, pp. 198–205. [Google Scholar]
- Ikeda, Y.; Horie, R.; Sugaya, M. Estimating emotion with biological information for robot interaction. Procedia Comput. Sci. 2017, 112, 1589–1600. [Google Scholar] [CrossRef]
- Posner, J.; Russell, J.A.; Peterson, B.S. The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology. Dev. Psychopathol. 2005, 17, 715–734. [Google Scholar] [CrossRef] [PubMed]
- Barrett, L.F.; Gross, J.J.; Christensen, T.C.; Benvenuto, M. Individual differences in the experience of emotion. Annu. Rev. Psychol. 2001, 52, 371–399. [Google Scholar]
- Rankin, C.H.; Abrams, T.; Barry, R.J.; Bhatnagar, S.; Clayton, D.F.; Colombo, J.; Coppola, G.; Geyer, M.A.; Glanzman, D.L.; Marsland, S.; et al. Habituation revisited: An updated and revised description of the behavioral characteristics of habituation. Neurobiol. Learn. Mem. 2009, 92, 135–138. [Google Scholar] [CrossRef]
- Meta Platforms, Inc. Meta Quest 2 Virtual Reality Headset; Meta Platforms, Inc.: Menlo Park, CA, USA, 2020. [Google Scholar]
- OpenBCI Inc. Ganglion Board (EEG Acquisition Device); OpenBCI Inc.: Brooklyn, NY, USA, 2016. [Google Scholar]
- Union Tool Co. MyBeat Heart Rate Sensor; Union Tool Co.: Tokyo, Japan, 2011. [Google Scholar]
- Gramfort, A.; Luessi, M.; Larson, E.; Engemann, D.A.; Strohmeier, D.; Brodbeck, C.; Goj, R.; Jas, M.; Brooks, T.; Parkkonen, L.; et al. MEG and EEG data analysis with MNE-Python. Front. Neurosci. 2013, 7, 267. [Google Scholar] [CrossRef]
- Pion-Tonachini, L.; Kreutz-Delgado, K.; Makeig, S. ICLabel: An automated electroencephalographic independent component classifier, dataset, and website. Neuroimage 2019, 198, 181–197. [Google Scholar] [CrossRef]
- Li, A.; Feitelberg, J.; Saini, A.P.; Höchenberger, R.; Scheltienne, M. MNE-ICALabel: Automatically annotating ICA components with ICLabel in Python. J. Open Source Softw. 2022, 7, 4484. [Google Scholar] [CrossRef]
- Weijs, M.L.; Jonauskaite, D.; Reutimann, R.; Mohr, C.; Lenggenhager, B. Effects of environmental colours in virtual reality (VR) on perceived spaciousness and emotion. J. Environ. Psychol. 2022, 82, 101866. [Google Scholar]
Adjective Pair | |
---|---|
Liked | Disliked |
Calm | Unsettled |
Beautiful | Dirty |
Showy | Plain |
Tense | Relaxed |
Comfortable | Uncomfortable |
Vulgar | Elegant |
Cool | Warm |
Inspiring | Ordinary |
Narrow | Spacious |
Wave | Frequency Band (Hz) | Example |
---|---|---|
Theta | 4–7 Hz | Memory encoding and retrieval; working memory |
Alpha | 8–12 Hz | Relaxation; restful states |
Beta | 13–30 Hz | Attention; cognition; stress; arousal |
Low Gamma | 31–50 Hz | Sensory integration; attention; perceptual load; local neural synchronization |
High Gamma | 51–70 Hz | Higher-order cognitive processes; memory encoding and retrieval; visual information integration; episodic memory |
pNN50 | Male | Female | β/α | Male | Female |
---|---|---|---|---|---|
VR1 | 0.021356 | 0.03918 | VR1 | 0.562203 | 1.12972 |
REAL1 | 0.01538 | 0.007253 | REAL1 | 0.461346 | 0.73335 |
VR2 | 0.015968 | 0.02675 | VR2 | 0.822684 | 1.11505 |
REAL2 | 0.017831 | 0.01891 | REAL2 | 0.56983 | 0.439284 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kobayashi, T.; Jadram, N.; Ninomiya, S.; Suzuki, K.; Sugaya, M. A Pilot Study on Emotional Equivalence Between VR and Real Spaces Using EEG and Heart Rate Variability. Sensors 2025, 25, 4097. https://doi.org/10.3390/s25134097
Kobayashi T, Jadram N, Ninomiya S, Suzuki K, Sugaya M. A Pilot Study on Emotional Equivalence Between VR and Real Spaces Using EEG and Heart Rate Variability. Sensors. 2025; 25(13):4097. https://doi.org/10.3390/s25134097
Chicago/Turabian StyleKobayashi, Takato, Narumon Jadram, Shukuka Ninomiya, Kazuhiro Suzuki, and Midori Sugaya. 2025. "A Pilot Study on Emotional Equivalence Between VR and Real Spaces Using EEG and Heart Rate Variability" Sensors 25, no. 13: 4097. https://doi.org/10.3390/s25134097
APA StyleKobayashi, T., Jadram, N., Ninomiya, S., Suzuki, K., & Sugaya, M. (2025). A Pilot Study on Emotional Equivalence Between VR and Real Spaces Using EEG and Heart Rate Variability. Sensors, 25(13), 4097. https://doi.org/10.3390/s25134097