Virtual Reality with 360-Video Storytelling in Cultural Heritage: Study of Presence, Engagement, and Immersion
Abstract
:1. Introduction
2. Background
2.1. Interactive Digital Storytelling
2.2. Underwater Virtual Reality
3. Archaeological VR Application
3.1. System Architecture
3.2. Apparatus
3.3. User Interaction with the Application
3.4. VR Storytelling
4. Materials and Methods
4.1. Participants
4.2. Procedure
4.3. EEG Data Collection
4.4. Questionnaires
4.5. Data Analysis
5. Results
5.1. Questionnaire Results
5.2. EEG Results
5.2.1. Frontal EEG
5.2.2. Parietal EEG
5.3. Correlations
6. Discussion
Limitations
7. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Slater, M. Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philos. Trans. R. Soc. B Biol. Sci. 2009, 364, 3549–3557. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Riva, G.; Davide, F.; IJsselsteijn, W.A. Being there: The experience of presence in mediated environments. Comput. Sci. 2003, 5, 3–16. [Google Scholar]
- Slater, M.; Sanchez-Vives, M.V. Enhancing Our Lives with Immersive Virtual Reality. Front. Rob. AI 2016, 3. [Google Scholar] [CrossRef][Green Version]
- Abulrub, A.H.G.; Attridge, A.N.; Williams, M.A. Virtual reality in engineering education: The future of creative learning. In Proceedings of the 2011 IEEE Global Engineering Education Conference (EDUCON), Amman, Jordan, 4–6 April 2011. [Google Scholar] [CrossRef]
- Freina, L.; Ott, M. A literature review on immersive virtual reality in education: State of the art and perspectives. In Proceedings of the International Scientific Conference eLearning and Software for Education, Islamabad, Pakistan, 23 April 2015. [Google Scholar]
- Skarlatos, D.; Agrafiotis, P.; Balogh, T.; Bruno, F.; Castro, F.; Petriaggi, B.D.; Demesticha, S.; Doulamis, A.D.; Drap, P.; Georgopoulos, A.; et al. Project iMARECULTURE: Advanced VR, iMmersive Serious Games and Augmented REality as Tools to Raise Awareness and Access to European Underwater CULTURal heritagE. In Proceedings of the 6th International Conference, Nicosia, Cyprus, 31 October–5 November 5 2016. [Google Scholar] [CrossRef]
- Bruno, F.; Lagudi, A.; Gallo, A.; Muzzupappa, M.; Davidde Petriaggi, B.; Passaro, S. 3d Documentation of Archeological Remains in the Underwater Park of Baiae. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 2015, XL-5/W5, 41–46. [Google Scholar] [CrossRef][Green Version]
- Bruno, F.; Lagudi, A.; Barbieri, L.; Muzzupappa, M.; Ritacco, G.; Cozza, A.; Cozza, M.; Peluso, R.; Lupia, M.; Cario, G. Virtual and augmented reality tools to improve the exploitation of underwater archaeological sites by diver and non-diver tourists. In Proceedings of the Euro-Mediterranean Conference, Nicosia, Cyprus, 31 October–5 November 5 2016. [Google Scholar]
- Bruno, F.; Barbieri, L.; Lagudi, A.; Cozza, M.; Cozza, A.; Peluso, R.; Muzzupappa, M. Virtual dives into the underwater archaeological treasures of South Italy. Virtual Real. 2018, 22, 91–102. [Google Scholar] [CrossRef]
- Bruno, F.; Lagudi, A.; Barbieri, L.; Cozza, M.; Cozza, A.; Peluso, R.; Davidde Petriaggi, B.; Petriaggi, R.; Rizvic, S.; Skarlatos, D. Virtual tour in the sunken “Villa con ingresso a protiro” within the underwater archaeological park of Baiae. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 2019, XLII-2/W10, 45–51. [Google Scholar] [CrossRef][Green Version]
- Tcha-Tokey, K.; Christmann, O.; Loup-Escande, E.; Richir, S. Proposition and Validation of a Questionnaire to Measure the User Experience in Immersive Virtual Environments. Sci. Arts Metiers 2016, 16, 33–48. [Google Scholar] [CrossRef]
- Csikszentmihalyi, M.; Abuhamdeh, S.; Nakamura, J. Flow. In Flow and the Foundations of Positive Psychology: The Collected Works of Mihaly Csikszentmihalyi; Csikszentmihalyi, M., Ed.; Springer: Dordrecht, The Netherlands, 2014; pp. 227–238. [Google Scholar] [CrossRef]
- Hettinger, L.J.; Riccio, G.E. Visually Induced Motion Sickness in Virtual Environments. Presence Teleoperators Virtual Environ. 1992, 1, 306–310. [Google Scholar] [CrossRef]
- Sanei, S.; Chambers, J. EEG Signal Processing; John Wiley & Sons: Chichester, UK; Hoboken, NJ, USA, 2007. [Google Scholar]
- Ray, W.J.; Cole, H.W. EEG alpha activity reflects attentional demands, and beta activity reflects emotional and cognitive processes. Science 1985, 228, 750–752. [Google Scholar] [CrossRef]
- Laufs, H.; Krakow, K.; Sterzer, P.; Eger, E.; Beyerle, A.; Salek-Haddadi, A.; Kleinschmidt, A. Electroencephalographic signatures of attentional and cognitive default modes in spontaneous brain activity fluctuations at rest. Proc. Natl. Acad. Sci. USA 2003, 100, 11053–11058. [Google Scholar] [CrossRef][Green Version]
- Laufs, H.; Kleinschmidt, A.; Beyerle, A.; Eger, E.; Salek-Haddadi, A.; Preibisch, C.; Krakow, K. EEG-correlated fMRI of human alpha activity. NeuroImage 2003, 19, 1463–1476. [Google Scholar] [CrossRef]
- Klimesch, W. EEG alpha and theta oscillations reflect cognitive and memory performance: A review and analysis. Brain Res. Rev. 1999, 29, 169–195. [Google Scholar] [CrossRef]
- Miller, C.H. Digital Storytelling 4e: A Creator’s Guide to Interactive Entertainment; Taylor and Francis: Oxford, UK, 2004. [Google Scholar]
- Schoenau-Fog, H. Adaptive storyworlds: Utilizing the spacetime continuum in interactive digital storytelling. In Interactive Storytelling; Springer: Berlin/Heidelberg, Germany, 2015; pp. 58–65. [Google Scholar]
- Aylett, R. Emergent narrative, social immersion and storification. In Proceedings of the 1st International Workshop on Narrative and Interactive Learning Environments, Leeds, UK, 20 August 2000–19 August 2001. [Google Scholar]
- Aylett, R. Narrative in virtual environments-towards emergent narrative. In Proceedings of the AAAI Fall Symposium on Narrative Intelligence, North Falmouth, MA, USA, 5–7 November 1999. [Google Scholar]
- Louchart, S.; Aylett, R. Solving the narrative paradox in VEs – lessons from rpgs. In Intelligent Virtual Agents; Springer: Berlin/Heidelberg, Germany, 2003; pp. 244–248. [Google Scholar]
- Tempt, B.F.; Schoenau-Fog, H. Coffee tables and cryo chambers: A comparison of user experience and diegetic time between traditional and virtual environment-based roleplaying game scenarios. In Interactive Storytelling; Springer: Berlin/Heidelberg, Germany, 2012; pp. 102–113. [Google Scholar]
- Rizvic, S.; Boskovic, D.; Okanovic, V.; Sljivo, S. Kyrenia - Hyper Storytelling Pilot Application. In Proceedings of the Eurographics Workshop on Graphics and Cultural Heritage, Graz, Austria, 27–29 September 2017. [Google Scholar] [CrossRef]
- Rizvic, S.; Djapo, N.; Alispahic, F.; Hadzihalilovic, B.; Cengic, F.F.; Imamovic, A.; Okanovic, V.; Boskovic, D. Guidelines for interactive digital storytelling presentations of cultural heritage. In Proceedings of the 2017 9th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games), Athens, Greece, 6–8 September 2017. [Google Scholar]
- Argyriou, L.; Economou, D.; Bouki, V. 360-degree interactive video application for cultural heritage education. In Proceedings of the Third Immersive Learning Research Network Conference, Coimbra, Portugal, 29 June–26 July 2017. [Google Scholar] [CrossRef]
- Sylaiou, S.; Liarokapis, F.; Kotsakis, K.; Patias, P. Virtual museums, a survey and some issues for consideration. J. Cult. Herit. 2009, 10, 520–528. [Google Scholar] [CrossRef]
- Bruno, F.; Bruno, S.; Sensi, G.D.; Luchi, M.L.; Mancuso, S.; Muzzupappa, M. From 3D reconstruction to virtual reality: A complete methodology for digital archaeological exhibition. J. Cult. Herit. 2010, 11, 42–49. [Google Scholar] [CrossRef]
- Anderson, E.F.; McLoughlin, L.; Liarokapis, F.; Peters, C.; Petridis, P.; de Freitas, S. Developing Serious Games for Cultural Heritage: A State-of-the-art Review. Virtual Real. 2010, 14, 255–275. [Google Scholar] [CrossRef][Green Version]
- Carrozzino, M.; Bergamasco, M. Beyond virtual museums: Experiencing immersive virtual reality in real museums. J. Cult. Herit. 2010, 11, 452–458. [Google Scholar] [CrossRef]
- Liarokapis, F.; Petridis, P.; Andrews, D.; de Freitas, S. Multimodal Serious Games Technologies for Cultural Heritage. In Mixed Reality and Gamification for Cultural Heritage; Springer: Berlin/Heidelberg, Germany, 2017; pp. 371–392. [Google Scholar] [CrossRef]
- Bekele, M.K.; Pierdicca, R.; Frontoni, E.; Malinverni, E.S.; Gain, J. A Survey of Augmented, Virtual, and Mixed Reality for Cultural Heritage. J. Comput. Cult. Herit. 2018, 11. [Google Scholar] [CrossRef]
- González Vargas, J.C.; Fabregat, R.; Carrillo-Ramos, A.; Jové, T. Survey: Using Augmented Reality to Improve Learning Motivation in Cultural Heritage Studies. Appl. Sci. 2020, 10, 897. [Google Scholar] [CrossRef][Green Version]
- Watts, G.P.; Knoerl, T.K. Entering the Virtual World of Underwater Archaeology. In Out of the Blue: Public Interpretation of Maritime Cultural Resources; Jameson, J.H., Scott-Ireton, D.A., Eds.; Springer: Boston, MA, USA, 2007; pp. 223–239. [Google Scholar] [CrossRef]
- Varinlioğlu, G. Data Collection for a Virtual Museum on the Underwater Survey at Kaş, Turkey. Int. J. Naut. Archaeol. 2011, 40, 182–188. [Google Scholar] [CrossRef][Green Version]
- Katsouri, I.; Tzanavari, A.; Herakleous, K.; Poullis, C. Visualizing and Assessing Hypotheses for Marine Archaeology in a VR CAVE Environment. Comput. Cult. Herit. 2015, 8, 1–18. [Google Scholar] [CrossRef]
- Bruno, F.; Barbieri, L.; Mangeruga, M.; Cozza, M.; Lagudi, A.; Čejka, J.; Liarokapis, F.; Skarlatos, D. Underwater augmented reality for improving the diving experience in submerged archaeological sites. Ocean. Eng. 2019, 190, 106487. [Google Scholar] [CrossRef]
- Jain, D.; Sra, M.; Guo, J.; Marques, R.; Wu, R.; Chiu, J.; Schmandt, C. Immersive Terrestrial Scuba Diving Using Virtual Reality. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, San Jose, CA, USA, May 2016. [Google Scholar] [CrossRef]
- Chapman, P.; Conte, G.; Drap, P.; Gambogi, P.; Gauch, F.; Hanke, K.; Long, L.; Loureiro, V.; Papini, O.; Pascoal, A.; et al. VENUS, Virtual ExploratioN of Underwater Sites. In Proceedings of the 7th International Symposium on Virtual Reality, Archaeology and Cultural Heritage (VAST 2006), Nicosia, Cyprus, 30 October–4 November 2006. [Google Scholar]
- Haydar, M.; Maidi, M.; Roussel, D.; Mallem, M.; Drap, P.; Bale, K.; Chapman, P. Virtual Exploration of Underwater Archaeological Sites: Visualization and Interaction in Mixed Reality Environments. In Proceedings of the 9th International Conference on Virtual Reality, Archaeology and Cultural HeritageVAST (2008), Braga, Portugal, 2–5 December 2008. [Google Scholar] [CrossRef]
- Haydar, M.; Roussel, D.; Maidi, M.; Otmane, S.; Mallem, M. Virtual and Augmented Reality for Cultural Computing and Heritage: A Case Study of Virtual Exploration of Underwater Archaeological Sites (Preprint). Virtual Real. 2011, 15, 311–327. [Google Scholar] [CrossRef]
- Stone, R. Virtual heritage: The willing suspension of disbelief for the moment. World Herit. Rev. 1999, b, 18–27. [Google Scholar]
- Liarokapis, F.; Kouril, P.; Agrafiotis, P.; Demesticha, S.; Chmelík, J.; Skarlatos, D. 3D modelling and mapping for virtual exploration of underwater archaeology assets. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 2017, 42, 425–431. [Google Scholar] [CrossRef][Green Version]
- Bruno, F.; Lagudi, A.; Ritacco, G.; Agrafiotis, P.; Skarlatos, D.; Čejka, J.; Kouřil, P.; Liarokapis, F.; Philpin-Briscoe, O.; Poullis, C.; et al. Development and integration of digital technologies addressed to raise awareness and access to European underwater cultural heritage. An overview of the H2020 i-MARECULTURE project. In Proceedings of the OCEANS 2017-Aberdeen, Aberdeen, UK, 19–22 June 2017. [Google Scholar] [CrossRef]
- Kouřil, P.; Liarokapis, F. Simulation of Underwater Excavation Using Dredging Procedures. IEEE Comput. Graphics Appl. 2018, 38, 103–111. [Google Scholar] [CrossRef]
- Doležal, M.; Vlachos, M.; Secci, M.; Demesticha, S.; Skarlatos, D.; Liarokapis, F. Understanding Underwater Photogrammetry For Maritime Archaeology Through Immersive Virtual Reality. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 2019, XLII-2/W10, 85–91. [Google Scholar] [CrossRef][Green Version]
- Woods, A.; Oliver, N.; Bourke, P.; Green, J.; Paterson, A. Beacon Virtua: A Virtual Reality Simulation Detailing the Recent and Shipwreck History of Beacon Island, Western Australia. In 3D Recording and Interpretation for Maritime Archaeology; McCarthy, J.K., Benjamin, J., Winton, T., van Duivenvoorde, W., Eds.; Springer: Berlin/Heidelberg, Germany, 2019; pp. 197–210. [Google Scholar] [CrossRef][Green Version]
- Petriaggi, B.D.; Petriaggi, R.; Bruno, F.; Lagudi, A.; Peluso, R.; Passaro, S. A digital reconstruction of the sunken “Villa con ingresso a protiro” in the underwater archaeological site of Baiae. IOP Conf. Ser. Mater. Sci. Eng. 2018, 364, 012013. [Google Scholar] [CrossRef][Green Version]
- Rizvic, S.; Boskovic, D.; Bruno, F.; Petriaggi, D.B.; Sljivo, S.; Cozza, M. Actors in VR Storytelling. In Proceedings of the 11th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games), Vienna, Austria, 4–6 September 2019. [Google Scholar]
- Homan, R.W.; Herman, J.; Purdy, P. Cerebral location of international 10–20 system electrode placement. Electroencephalogr. Clin. Neurophysiol. 1987, 66, 376–382. [Google Scholar] [CrossRef]
- Hart, S.G.; Staveland, L.E. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Advances in Psychology; Hancock, P.A., Meshkati, N., Eds.; Human Mental Workload: North-Holland, The Netherlands, 1988; Volume 52, pp. 139–183. [Google Scholar] [CrossRef]
- Delorme, A.; Makeig, S. EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 2004, 134, 9–21. [Google Scholar] [CrossRef][Green Version]
- Chang, C.; Hsu, S.; Pion-Tonachini, L.; Jung, T. Evaluation of Artifact Subspace Reconstruction for Automatic EEG Artifact Removal. In Proceedings of the 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, HI, USA, 18–21 July 2018. [Google Scholar] [CrossRef]
- Winkler, I.; Brandl, S.; Horn, F.; Waldburger, E.; Allefeld, C.; Tangermann, M. Robust artifactual independent component classification for BCI practitioners. J. Neural Eng. 2014, 11, 035013. [Google Scholar] [CrossRef][Green Version]
- Jensen, O.; Tesche, C.D. Frontal theta activity in humans increases with memory load in a working memory task. Eur. J. Neurosci. 2002, 15, 1395–1399. [Google Scholar] [CrossRef]
- Kim, J.Y.; Son, J.B.; Leem, H.S.; Lee, S.H. Psychophysiological Alteration After Virtual Reality Experiences Using Smartphone-Assisted Head Mount Displays: An EEG-Based Source Localization Study. Appl. Sci. 2019, 9, 2501. [Google Scholar] [CrossRef][Green Version]
- Clemente, M.; Rodríguez, A.; Rey, B.; Alcañiz, M. Assessment of the influence of navigation control and screen size on the sense of presence in virtual reality using EEG. Expert Syst. Appl. 2014, 41, 1584–1592. [Google Scholar] [CrossRef]
- Slobounov, S.M.; Ray, W.; Johnson, B.; Slobounov, E.; Newell, K.M. Modulation of cortical activity in 2D versus 3D virtual reality environments: An EEG study. Int. J. Psychophysiol. 2015, 95, 254–260. [Google Scholar] [CrossRef] [PubMed]
- Song, J.; Davey, C.; Poulsen, C.; Luu, P.; Turovets, S.; Anderson, E.; Li, K.; Tucker, D. EEG source localization: Sensor density and head surface coverage. J. Neurosci. Methods 2015, 256, 9–21. [Google Scholar] [CrossRef] [PubMed][Green Version]
EEG Band | Baseline | Initial | Final |
---|---|---|---|
Frontal Theta | 1.112 (0.845) | 1.887 (1.383) | 1.383 (0.594) |
Frontal Alpha | 0.502 (0.278) | 0.602 (0.323) | 0.597 (0.261) |
Frontal High beta | 0.197 (0.096) | 0.232 (0.132) | 0.247 (0.123) |
Parietal Theta | 0.917 (0.416) | 1.273 (0.465) | 1.526 (0.748) |
Parietal Alpha | 0.580 (0.480) | 0.695 (0.275) | 0.705 (0.273) |
Parietal High beta | 0.293 (0.175) | 0.301 (0.172) | 0.303 (0.145) |
Scale | Presence | Engagement | Immersion | Flow |
---|---|---|---|---|
Engagement | 0.796 (0.000) | - | - | - |
Immersion | 0.540 (0.038) | 0.651 (0.009) | - | - |
Flow | - | - | 0.767 (0.001) | - |
Emotion | 0.562 (0.029) | 0.613 (0.015) | 0.621 (0.013) | - |
Techn. adoption | - | 0.589 (0.021) | 0.780 (0.001) | 0.606 (0.017) |
Judgement | 0.596 (0.019) | 0.760 (0.001) | 0.800 (0.000) | 0.567 (0.027) |
Judgement | and emotion | 0.653 (0.008) | ||
and tech. ad. | 0.576 (0.025) |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Škola, F.; Rizvić, S.; Cozza, M.; Barbieri, L.; Bruno, F.; Skarlatos, D.; Liarokapis, F. Virtual Reality with 360-Video Storytelling in Cultural Heritage: Study of Presence, Engagement, and Immersion. Sensors 2020, 20, 5851. https://doi.org/10.3390/s20205851
Škola F, Rizvić S, Cozza M, Barbieri L, Bruno F, Skarlatos D, Liarokapis F. Virtual Reality with 360-Video Storytelling in Cultural Heritage: Study of Presence, Engagement, and Immersion. Sensors. 2020; 20(20):5851. https://doi.org/10.3390/s20205851
Chicago/Turabian StyleŠkola, Filip, Selma Rizvić, Marco Cozza, Loris Barbieri, Fabio Bruno, Dimitrios Skarlatos, and Fotis Liarokapis. 2020. "Virtual Reality with 360-Video Storytelling in Cultural Heritage: Study of Presence, Engagement, and Immersion" Sensors 20, no. 20: 5851. https://doi.org/10.3390/s20205851