Cultural Heritage in Fully Immersive Virtual Reality

Abstract

Fully immersive virtual reality (VR) applications have modified the way people access cultural heritage—from the visiting of virtual museums containing large collections of paintings to the visiting of ancient buildings. In this paper, we propose to review the software that are currently available that deal with cultural heritage in fully immersive virtual reality. It goes beyond technologies that were available prior to virtual reality headsets, at a time where virtual was simply the synonym of the application of digital technologies to cultural heritage. We propose to group these applications depending on their content—from generic art galleries and museums to applications that focus on a single artwork or single artist. Furthermore, we review different ways to assess the performance of such applications with workload, usability, flow, and potential VR symptoms surveys. This paper highlights the progress in the implementation of applications that provide immersive learning experiences related to cultural heritage, from 360 images to photogrammetry and 3D models. The paper shows the discrepancy between available software to the general audience on various VR headsets and scholarship activities dealing with cultural heritage in VR.

1. Introduction

The word “virtual” and the expression “virtual reality” can be ambiguous and refer to different types of applications. A virtual world is a computer-simulated environment, and virtual reality (VR) can be defined as a simulated experience. In this paper, we consider fully immersive VR as the use of virtual reality with a head-mounted display (HMD). It is different than approaches using non-immersive VR that happen with regular video games on a computer screen, and semi-immersive VR happening with the use of large projectors [1] in a CAVE (Cave Automatic Virtual Environment)-like system [2]. Fully immersive virtual reality has left the stage of prototype [3], and a large number of applications have been developed since the mid-2010s. The simultaneous development of consumer-grade VR headsets, 3D game engine, mobile devices, and the increase of computational power for processing multimedia content has allowed the emergence of consumer-grade VR technologies in various domains such as education and entertainment. The literature combining works at the frontier between education, computer science, and engineering has developed substantially to propose a wide range of studies and applications related to immersive learning and cultural heritage. Bekele et al. proposed a broad survey of Augmented, Virtual, and Mixed Reality (i.e., extended reality (XR)) for cultural heritage highlighting applications in education, exhibition enhancement, exploration, reconstruction, and virtual museums [4]. In the present paper, we focus only on fully immersive virtual reality, as it is currently the closest representation of what reality can be, with the field being mature enough to discuss.

Virtual heritage is one of the computer-based interactive technologies in virtual reality where it creates a visual representation of monuments, artifacts, buildings and culture to deliver openly to global audiences. In education, virtual heritage can enhance the learning process of students; it can facilitate the analysis of key events and historical elements for the use of students and researchers. Eight requirements have been specified for the development of virtual heritage application: high geometric accuracy, capture of all details, photorealism, high automation level, low cost, portability, application flexibility, and model size efficiency [5].

The goal of this paper is to provide a review of virtual worlds in cultural heritage using fully immersive virtual reality, stressing the challenges related to the inequality of access to cultural artifacts and its wide impact on society. This paper reviews different types of measurements that can be used for assessing the performance of software presenting virtual worlds related to cultural heritage. We identify different technical constraints and recommendations for the creation of virtual worlds that render cultural artifacts and, more importantly, for the acquisition of content that can later be used in virtual worlds.

Cultural heritage is defined by UNESCO as “the legacy of physical artifacts and intangible attributes of a group or society that are inherited from past generations, maintained in the present and bestowed for the benefit of future generations” [6]. The first article of the UNESCO definition considers monuments (e.g., architectural works, works of monumental sculpture and painting), groups of buildings (e.g., groups of separate or connected buildings with outstanding universal value from the point of view of history, art or science), and sites (e.g., archaeological sites). Cultural heritage includes the heritage of tangible and intangible heritage assets of a group or society, which is inherited from past generations. Cultural heritage is part of the heritage that is selected by society as what should be protected. This cultural heritage is a product of numerous selections performed by groups or society in such a way that not all heritages of ancient generations are classified as cultural heritage. These selected assets have historical, aesthetic, archaeological, scientific, ethnological, or anthropological value. On one hand, tangible cultural heritage includes books, buildings, landscapes, monuments, and any tangible artifacts; on the other hand, intangible cultural heritage includes folklore, knowledge, language, and traditions. Another type of cultural heritage is natural heritage, e.g., culturally significant landscapes, such as national parks and biodiversity. Cultural heritage is an expression of the ways of living developed by a community and passed on from generation to generation, including customs, practices, places, objects, artistic expressions and values. The main goal of virtual heritage is to represent cultural heritage in realistic virtual environments where the public can immerse and/or interact with these artifacts. Virtual heritage involves directly computer graphics and/or multimedia content, i.e., 3D objects, 2D images, sounds, music.

The remainder of the paper is organized as follows: In Section 2, we provide the rationale of how fully immersive VR applications can become solutions for the access of cultural heritage. In Section 3, we provide an overview of the state-of-the-art VR applications related to cultural heritage. Then, several types of evaluation are described in Section 5 for assessing VR applications performance. Finally, we discuss recommendations for the implementation of cultural heritage in VR. It includes the challenges related to the interdisciplinary nature of the field, the challenges in education for the deployment of VR applications in the classroom, how serious gaming may increase the audience, and how the social experience represents an essential part of cultural heritage.

2. Space and Time

The representation of cultural heritage in virtual reality transcends space and time: (1) users can visit remote places that are not accessible to regular users, or places that are too far; (2) users can visit places the way they were at a different time in history through the digital reconstruction of a historical building [7]. Virtual worlds can represent cultural landmarks in their natural environment or within a museum. For instance, a painting can be displayed in a virtual environment representing a museum, or it can be displayed in a location where it was originally thought to be displayed (e.g., church, castle, mansion, bedroom). It may even be displayed in a fantasy world. Virtual worlds allow users to access environments such as buildings and landscapes that are too far away from the user or inaccessible such as unreachable underwater places [8]. While monuments can be visited by tourists, famous monuments require visitors to travel long distances. For many people, traveling abroad represents some substantial financial cost, and VR can become the only option to visit places. A green argument can be brought to support the development of virtual worlds in fully immersive virtual reality where users can appreciate a location with high details without traveling and therefore without any travel carbon footprint. Furthermore, visiting such places in the real world can damage the real environment when the number of visitors is too high, when the human factors can lower the experience of the visit, e.g., when a museum is too crowded, and when there is a long cue to access a specific location. The Lascaux cave is an example of how visitors to a cultural heritage site can damage it and alternative solutions for the visitors must be found. The Lascaux cave complex was first opened to the public in 1948. By 1955, the effects of the daily 1200 visitors had a substantial impact on the paintings present in the cave. These visitors produced carbon dioxide, heat, humidity, and other contaminants. Due to the deterioration of air, the walls hosting the paintings became infected by fungi and lichen. As a result, the cave was closed to the public in 1963. In order to still allow the public to appreciate the paintings in the cave, different replicas were created. These replicas show it is not necessary to see the exact original content in order to appreciate it. Another argument related to the time dimension is the problem of climate change. The effects of climate change with severe weather conditions provide a direct threat to both cultural heritage sites and institutions in many places over the world [9]. Digital preservation and access using fully immersive VR would enable next generations to experience heritage that can be lost or damaged. Finally, while some ancient places provided access for individuals with disabilities [10], many cultural heritages places (e.g., castles, towers, churches) were not built to consider accessibility issues, such as the possibility to use wheelchairs to navigate within narrow places (e.g., spiral narrow staircases, ladders). Adding accessibility elements such as wheelchair ramps, accessible lifts, and disability parking spaces while maintaining the requirement of restoration and conservation is often impossible [11]. VR can provide a direct solution for people with physical disabilities when irreconcilable differences occur between conservation planning authorities and physical access improvement.

3. VR Applications

Multiple venues welcome publications related to cultural heritage using fully immersive virtual reality (e.g., the International Conference of the Immersive Learning Research Network (iLRN), the Journal of Virtual World Research, Virtual Worlds, PRESENCE: Virtual and Augmented Reality, etc). However, software is a pillar of modern research and a stepping stone for productive academic research. Software is now integral to scholarly research, and it should be recognized as an output of research that is equivalent to an academic paper [12]. In this section, we are covering the main applications that are available on the Steam platform and/or the Oculus Store that are in fully immersive virtual reality. The search for these applications included keywords such as “art”, “museum”, and “education”. The search was completed in June 2022. On the Steam platform, the search can be performed by selecting VR-supported and/or VR-only applications. The search in the Oculus Store can be performed for the Rift, Quest, Go VR headsets. This survey provides an overview of the different applications that are available and how they have evolved over time in relation to the availability of toolboxes for better managing VR applications. The applications include only those that are directly related to cultural heritage. Some applications aim at reproducing ancient buildings but by simply using assets that could be used in any heroic fantasy video games. As such, the selected applications represent existing visual art and/or architecture. Many VR applications related to cultural heritage are available for free on Steam, and they can be easily used in educational settings. Furthermore, the quality is typically higher than what can be found from student’s projects trying to solve a research problem or to showcase a technological feature. Therefore, the applications that are described thereafter have all their place as scholarly works. The software can have different constraints in relation to the type of headset that can be used. Some applications accept only one type of VR headset, while others can be used by different VR headsets. They can be used in different modes: sitting position, standing position, or with room-scale where users can move within the VR environment.

A scholarly activity can be defined as a creative work that is peer reviewed and publicly disseminated [13]. VR applications that are released on Steam can be reviewed by users: it includes a positive or negative score followed by a comment justifying the positive or negative comment. These comments can be used by the developers to modify the program and fix bugs. Discussions can also be used where users provide richer comments and feedback or list potential issues. As such, these comments have the same role as the mechanisms that are used by some publishers to improve the peer-review process. Scholarly activity is a creative work that is peer reviewed and publicly disseminated. There exist multiple forms of scholarship: the discovery of new knowledge; development of new technologies, methods, materials, or uses; and integration of knowledge leading to new understanding. When dealing with cultural heritage with new technologies, developers provide tools that can be utilized for different purposes such as outreach and education.

We can group VR applications for cultural heritage into different categories based on their content: (1) software with a broad content: it includes art galleries from multiple paintings, (2) software with a specific content: it includes applications that are dedicated to a single artwork, e.g., Mona Lisa, a single artist, or a specific set of works from a given artist. VR applications can also be split in relation to their degree of interactivity: if it is a VR experience in which the user is passive, if the user is active with events to interact and manipulate with the environment. Finally, the VR applications may contain gaming components; most of them do not include game mechanics.

3.1. Generic Art Collections

The virtual museum is defined as the communicative projection of the real museum [14]. Many aspects of the discussions related to the definition are related to the virtual museum as an experience on a regular computer and not within a VR environment. Seeing a painting or an object on the screen of a computer does not create the same perceptual impression that one has when viewing the object on the site. The museum is clearly more than what meets the eyes. There is no ranking in the senses that are used for discovering a museum. The vision is not the primary sense to be used in a museum but only one of them. Visitors move through rooms, around and in front of the objects. It is critical to have a sense of position in space for obtaining a perception of size, texture, and volume.

Virtual museums and art galleries represent one of the most common applications in VR related to cultural heritage. It allows us to appreciate paintings and sculptures in their real size within realistic environments, aiming at reproducing the museum experience. Screenshots of VR museums available on the Steam platform are given in Figure 1, while applications only available on the Oculus store are given in Figure 2. For the developers, these applications do not require collaborations with cultural heritage sites or institutions, as many artifacts are directly available, free of access, and online. High-resolution images can be obtained from various websites (e.g., Wikipedia) where famous painters and their main paintings are available. The number of paintings depends on the selected language in a Wikipedia page. In addition, the information that can be found online can contain errors: the year of the painting can be incorrect, the height and width of a painting are often swapped, or they have the wrong units. Images from the Google Art project or from museums that offer high-resolution images can be used. The Metropolitan Museum of Art provides open access images and metadata of about 300 paintings. The Getty Museum provides 860 high-resolution paintings through its Getty Search Gateway (https://search.getty.edu/gateway/landing (accessed on 9 September 2022)). The resolution of the paintings varies substantially across museums and across applications. The low quality of the images can decrease the level of immersion and prevent their addition to a VR museum. For instructional materials for art history, images are typically in the public domain. Images of paintings from ancient painters are in the public domain, as the work is in the public domain in its country of origin and other countries and areas where the copyright term is the author’s life plus 100 years or fewer; the work is in the public domain in the United States because it was published (or registered with the U.S. Copyright Office) before 1 January 1926.

The VR Museum of Fine Art from Finn Sinclair (90% of positive reviews among 288—free) includes 15 high-fidelity sculptures and XXX paintings [15]. The artworks are arranged into four exhibits, each with its own theme. At the educational level, informative holographic plaques are placed next to each piece of art. More importantly, the museum has multiple floors and includes elements of architecture from real museums. There is a place for tickets, which are free, and drinks. While the content is limited, this application from 2016 paved the way to more complete virtual museums.

The Kremer Collection VR Museum (80% positive review among 10—not free) includes pieces by Rembrandt, Aelbert Cuyp, Frans Hals and other Old Masters from the Dutch Golden Age [16]. The frames of the paintings have a high level of detail, and the museum architecture has received lots of attention. While the collection is limited to a particular type of artist, the Kremer Collection VR Museum provides a genuine experience in realistic settings.

The Infinite Art Museum (90% positive reviews out of 31—free) includes a growing collection of over 800 original paintings, illustrations, digital art and photographs from 71 artists around the world [17]. The art galleries are set on six different levels. The artists are not world-famous artists from art history but rather present-day artists who are sharing their work in the Infinite Art Museum.

Mocove Arts VR (four positive reviews out of seven—not free) is a VR museum with a large collection: more than 1000 paintings and more than 50 sculptures in a single seamless virtual space [18]. The paintings are objects that are placed in some realistic art galleries. The museum includes an interactive art guide providing information about the paintings: title, painter, date, media, location, original size.

The Smithsonian American Art Museum “Beyond The Walls” (71% positive reviews out of 46—free) takes place in a realistic museum where exhibits include three unique pieces for users to engage with, leading to a broader, interactive experience of the original art pieces [19]. The users can move freely around the museum’s east wing from inside their headsets and view a dozen paintings, four works of sculpture, and one video art installation in high fidelity. Upon approaching Adams Memorial, users are transported out of the museum and into a 3D volumetric recreation in Rock Creek Park Cemetery, can experience a 360-degree, 6K video of an actual aurora borealis, and are greeted by a volumetric hologram of the artist Alex Prager explaining the inspiration for the 2013 film, Face in the Crowd.

Great Paintings VR (95% positive reviews out of 75—free) allow users to browse among more than 1000 famous paintings from the early Renaissance to the beginning of the twentieth century [20]. Most of the paintings have a high resolution, users can access paintings one by one or through art galleries dedicated to a specific artist or an art movement. The images are taken from Wikipedia with many images coming from the Google Art project, in which museums provide content in high resolution for free. Some images were upsampled using artificial intelligence techniques, i.e., deep learning, to increase the quality of the images. A description of the work and the distribution of the available paintings has been published [29].

The Museum of ThroughView (only one unique positive review—not free) showcases 2D to 3D conversions of famous paintings and old photographs [21]. When looking at these paintings in VR, the paintings themselves can also be experienced in 3D. The application includes 54 paintings and photographs on display by artists such as Rembrandt, Bierstadt, and Monet. It has also an exhibition of photographs taken in New York before 1900 by Jacob Riis. The main originality of this program comes from the representation of the paintings that are both in 2D and 3D where the different layers of a painting are decomposed through different depths (i.e., foreground, background having different depths). This approach brings 3D effects that can only be achieved through virtual reality.

The OmniGallery (100% of positive reviews among 14—free) is another virtual museum that provides ancient paintings into various unusual museum locations (a hill where paintings are floating on top of the grass) [22]. The Classic Hill contains some famous paintings from the Western world (paintings by Da Vinci, Van Gogh and Klimt) with famous 3D models of marble sculptures. The different artistic artworks are displayed in different galleries.

Art Plunge (97% positive reviews out of 101) is a VR experience containing an art gallery with multiple paintings that are placed in a specific environment so the user can feel he/she is inside the painting [23]. As such, this application combines the existing artwork and a VR interpretation of the artwork within the environment. It includes five famous paintings: Mona Lisa, Starry Night, The Birth of Venus, The Creation of Adam, and Girl Reading a Letter at an Open Window.

The Finnish Virtual Art Gallery (three positive reviews out of three—free) is a VR museum focusing on nine Finnish painters (e.g., Fredrik Ahlstedt, Fanny Churberg, Helene Schjerfbeck) [24]. Strangely, the art gallery is located on a space station. It includes the basic gallery, a warehouse, and an extra gallery for Hugo Simberg’s paintings. The content is limited and mixes different types of graphical assets, changing radically the typical museum experience.

Boulevard (193 ratings, 2/5 stars) is a VR museum, for Rift, Rift S VR headsets, that can be found in the Oculus store [25]. The paintings are particularly well inserted within virtual environments, in rooms that describe well the typical museum experience in real life. The software is heavily criticized due to its poor user interface and controls.

Virtual Museum De Fornaris (nine ratings, 3/5 stars) is a VR art gallery for Rift, Rift S VR headsets, presenting 70 masterpieces from the De Fornaris collection (e.g., Hayez, Morbelli, Pellizza da Volpedo) [26]. This virtual museum showcases, highlights, and allows users to admire Fondazione’s most outstanding works of art.

VR Museum: Art Through Time (two ratings, 4.5/5 stars—not free) is another VR museum for Rift, Rift S VR headsets that contains about 60 paintings from the 1500s to 1800s [27]. The same developers have created “VR Museum: Blacksmithing Through Time” (two ratings, 5/5 stars—not free) which is about historical blacksmiths [27].

Städel Time Machine (560 ratings, 4/5 stars) is for Gear VR and Oculus Go. It is a museum dedicated to Frankfurt’s Städel Museum and how it looked in 1878. Oculus Go works on its own while Gear VR is completely powered by the phone [28].

3.2. Single Painting or Series

Different VR applications focus on a single painting or a single artist. These applications typically provide rich and deep explanations of the cultural materials as well as the life of the artist (see Figure 3). Furthermore, the raw material that is presented is often transformed to give a new vision going beyond what can be achieved in the real world. These experiences are often short and close to a technical demonstration in terms of content.

Mona Lisa: Beyond the Glass (98% of positive reviews out of 67—free) reveals the latest scientific research on Leonardo da Vinci’s artistic innovation and his painting techniques and processes through visualization in VR [30]. It was the first VR experience presented to the public by the Louvre Museum. It was conceived as part of the Museum’s seminal Leonardo da Vinci exhibition commemorating the 500th anniversary of the artist’s death in France.

Eye of the Owl—Bosch VR (92% of positive reviews out of 75—free) is a VR experience dedicated to Hieronymus Bosch, who was one of the most important representatives of the Early Netherlandish painters [31]. In addition to the presentation of his paintings, the application allows the user to visit the artist’s studio and explore his masterpiece in high resolution: a triptych oil painting entitled The Garden of Earthly Delights. It provides to the user a magnifying glass to explore the painting in detail while the artist describes the bizarre and mystifying scenes.

Claude Monet—The Water Lily obsession (85% of positive reviews out of 27—not free) is a contemplative VR experience [32]. It is not a traditional VR museum; it is a passive experience that invites the user on a sensory journey starting off in Claude Monet’s garden, stopping along the way at the workshop of the artist and ending in the exhibition rooms of the Orangerie Museum containing large paintings. The user is given the opportunity to relive a perpetual renewal of nature, to explore time and space while immersed in the Water lilies paintings. This application focuses only on the water lilies part of the work of Claude Monet with close to no interactive content.

The Scream (114 positive reviews out of 133) is a VR experience focusing on The Scream by Norwegian artist Edvard Munch [33]. This famous painting with its terror-twisted face has made a profound impact on art and in popular culture (e.g., the Scream series of horror movies). It is worth noting that it is even used for an emoji representing anxiety. This VR experience combines the original artwork and some special effects in VR. The demons and ghosts emerge from the canvas to plunge the player into Munch’s sources of inspiration.

The Night Cafe: A VR Tribute to Vincent van Gogh (96% of positive reviews out of 251—free) is a VR experience allowing the user to explore the world of Vincent van Gogh [34]. It uses the 2D paintings of van Gogh and transforms them into virtual environments. Instead of looking at the painting in its original size, the VR environment allows one to dive within the painting where the user can enjoy the sunflowers painting in 3D or walk around the chair he painted in his bedroom to see it from different vantage points.

Maria Blanchard Virtual Gallery (two positive reviews out of two—free) is a virtual art gallery dedicated to the painter Maria Blanchard (1881–1932) [35]. She was a Franco-Spanish painter and a key figure in the pictorial revival that took place in Europe during the First World War. The galleries include 65 artworks displayed in several rooms marked by key dates in her history accompanied by a virtual character who explains his biography in an entertaining and didactic way. The art works are presented in realistic rooms with paintings including a plaque with the title of the painting in English, French, and Spanish. It requires the Oculus Rift virtual reality headset.

4. Historical Places

Multiple VR experiences exist, placing the user in different times and locations of human history. It includes prehistory, ancient Egypt, and ancient Rome. Figure 4 gives representative examples of screenshots from the following examples. The technology being used varies from 360 images to photogrammetry, which is the science and technology of obtaining reliable information about physical objects and the environment through the process of recording, measuring and interpreting photographic images and patterns of electromagnetic radiant imagery and other phenomena. Compared to art museums representing paintings, the accurate representation of buildings represents a substantial endeavor.

The Dawn of Art (94% of positive reviews out of 125—free) is a VR experience that gives users a unique opportunity to visit the Chauvet-Pont-d’Arc Cave in the southeastern part of France [36]. This historical and cultural landmark is a Unesco World Heritage Site, and it contains some of the best-preserved figurative cave paintings in the world with multiple evidence of Upper Paleolithic life. The VR experience includes an immersive film and a virtual visit of the cave, inviting the user to explore prehistoric times.

Nefertari: Journey to Eternity (93% positive reviews out of 199—free) is a 15-min educational adventure through the tomb of one of the most prominent queens of ancient Egypt [37]. The player can explore the dark tomb to look the way it would have over 3000 years ago, with two light-wielding controllers. The player can take a richly detailed and close-up look at painted hieroglyphics, he/she can interact with the hieroglyphics to receive voiced information. In this application, the player can learn about Nefertari and the most famous Egyptian gods and goddesses. The high-resolution texture and the work for modeling the environment and the lights are key assets of this application.

Roma VR—Domus (three positive reviews out of five—not free) is an experience that allows the user to visit a house in ancient Pompeii [38]. The user can take a voice guided tour or explore on his/her own in this glimpse of life in ancient Roman times. This application is about the famous ruins of Pompeii and its inhabitants who lay still for the world to see.

Rome Reborn: Roman Forum (six positive reviews out of nine—not-free) is a VR experience that allows users to visit the Roman Forum in Ancient Rome [39]. The application has mixed reviews, with complaints about the 3D representation of the elements in the application. Instead of 3D models, 2D 360 photos are mainly used.

Baalbek Reborn: Temples (79% of positive reviews out of 39—free) is a VR experience taking the user on a virtual tour of the Roman temples in Baalbek, which is a UNESCO World Heritage Site in Lebanon [40]. It is possible to observe the ruins of the archaeological park and the rebuilt building as it existed in the third century of our era. The tour includes about 40 stops with explanations. Unfortunately, this experience is mainly based on 360 photos, without any 3D models, such as what can be achieved with photogrammetry. Hence, the overall experience is poor due to visuals.

In IL DIVINO: Michelangelo’s Sistine Ceiling in VR (96% positive reviews among 147—free), as the name suggested, is a fully immersive VR experience where the user can walk through and learn about the Sistine Chapel ceiling [41]. The developers mention that it has been developed with the Unreal Engine. This VR experience was originally created exclusively for the SIGGRAPH 2019 Immersive Pavilion by the team that has created the previous SIGGRAPH 2017 VR piece: Il Gigante: Michelangelo’s David in VR. Il Divino gives a genuine experience of the highest fidelity with high-resolution textures. This application allows the user to step onto Michelangelo’s own scaffold and learn about how he painted the ceiling or enter a Vatican conservator’s mobile aerial platform to see the ceiling up close and learn about the controversial cleaning. Beyond the high-resolution textures, the program provides educational content with more than 100 clickable items about Michelangelo’s work in the Sistine Chapel.

Church Art Of Sweden (nine positive reviews out of 13—free) is a museum presentation of church artifacts and full size scanned medieval churches [42].

Mannerheim Virtual Experience (five positive reviews out of five—free) is a VR experience that focuses on a very specific location and time in history during the second World War [43]. It gives a virtual tour of the war-time office of Carl Gustaf Emil Mannerheim, the Field Marshal of Finland. The developers used photogrammetry for faithful representation of the environment. The headquarters of the Finnish army operated out of and around the city of Mikkeli during the wars in 1939–1940 and 1941–1944. The user can learn historical facts and witness a 3D representation of Mannerheim reading a report from the front lines while visiting the office located in the Mikkeli Central Elementary School.

Hernández et al. describe a system allowing users to physically walk while exploring the virtual world [44]. In this interactive experience of the virtual reconstruction of a housing unit on a pre-Roman settlement, fully immersive VR strongly enhances the experience of perception of space in architecture. Haydar et al. investigated the visualization and interaction with reconstructed underwater archaeological sites [45]. They used an NVisor HMD, which was highly novel in 2011, before the Oculus and HTC Vive VR headsets. They stress how both archaeologists and the general public can obtain new insights on the reconstructed archaeological sites. On one hand, it allows archaeologists to analyze directly from within the virtual site, and on the other hand, a broad audience can explore a realistic reconstruction of the sites. In [46], Barsanti et al. describe the implementation of a VR experience that aims at enhancing the visibility of the Egyptian funeral objects presented in the Sforza Castle in Milan through an interactive scenario. This work combined the Oculus Rift DK2 and the Leap Motion controller. In [47], they propose a fully immersive VR experience for users to visit the Salla World War II graveyard, which is located in an inaccessible border zone between Finland and Russia. The aim of this virtual graveyard was to create an accurate visual representation of the Salla graveyard, including its atmosphere with sound. The user evaluation suggests the need to have multiple modalities for locomotion. In [48], a video game entitled “Discover Hoyo Negro: The Cave of America’s Human History” is described. It is a game that gives children the opportunity to learn more about digital archaeology and paleontology by visiting the Negro cave system with precise data using structure-from-motion photogrammetric techniques. In [49], Selmanović et al. investigate the extent to which 360 VR videos contribute to user immersion in the preservation of intangible cultural heritage. They propose a case study based on the Mostar bridge (or Stari Most) diving project. While the application is focused on a single building, the Old Bridge in Mostar, Bosnia and Herzegovina, it goes beyond the presentation of the building as it allows users to virtually jump off the bridge. Finally, this application is structured around multiple 360 videos that lead to a quiz and then a final 360 video about cliff diving.

4.1. Single Buildings

As for paintings, a VR application can be dedicated to a single building. The following examples show how historical buildings, such as castles, can be implemented in VR applications. A screenshot is given for each application in Figure 5. VR scenes that are not directly parts of applications exist as well. For instance, the Tower Bridge of London and an English church are stand-alone environments for the SteamVR platform [50,51].

Guildford Castle VR (98% of positive reviews out of 54—free) is a free roaming, narrative VR experience, presenting one of England’s first castles that has stood for over 900 years [52]. This recent application provides a state-of-the-art implementation with an engaging narrative audio tour bringing the castle’s history to life. Users can discover English medieval history through various audio-visual hot spots placed in the environment.

VersaillesVR (72% positive reviews out of 110) is an interactive VR experience allowing users to visit the palace built by Louis XIV [53]. It is possible to visit the main rooms and to be fully immersed in the Palace of Versailles. Key locations of the palace include: the King and Queen’s State Apartments, the Royal Opera House, the high altar in the Royal Chapel, and the Hall of Mirrors. Tapestries, paintings, ceilings, sculptures and furniture are particularly well rendered.

Parliament—Parlement (100% positive review out of 16—free) is a historical VR experience with no interaction [54]. It takes the user deep into the heart of Parliament, a Canadian landmark and architectural masterpiece, whose doors are now closed. The user can travel through the building with a realistic representation with photorealism of the different rooms. Animals spring from stone walls. The application highlights figures who helped shape Canada for over 100 years. This VR experience was created by the Library of Parliament and the National Film Board of Canada.

Visite virtuelle de l’Assemblée nationale (25 positive reviews out of 34—free) is an interactive VR experience (still in Early Access) that allows users to visit the Palais Bourbon, which is the meeting place of the French National Assembly [55]. The visit includes key parts of the building such as the hemicycle, the library, or the large salons. This educational VR application gives a guided tour and a way to interact with the statues and paintings that are in the building.

4.2. Other Types

Collins et al. have proposed GaeltechVR, a VR experience designed for the HTC VIVE Pro VR headset to improve situated identity in Irish learners focusing on their attitudes, motivation, and confidence as Irish language learners [56]. This study shows that cultural heritage goes beyond virtual museums and applications showing architectural works by improving the Irish language identity, attitudes, and motivations of adult Irish language learners.

5. Evaluation

The evaluation of fully immersive VR applications can be assessed at different levels. In this section, we propose to review the main types of questionnaires that can be provided for assessing such applications. The number of questions and therefore the duration needed to acquire the answers are parameters to consider when assessing multiple conditions or comparing different modalities (e.g., 2D screen vs. VR).

5.1. Workload

The evaluation of mental workload is an important criterion in the research and development of human–machine interfaces, targeting higher efficiency, levels of comfort, satisfaction, and safety. It is particularly relevant when assessing the performance of a fully immersive VR application in which users must reach particular items, i.e., visit certain elements. For a task associated with work, or for the ability to perform different tasks in a virtual world, a key objective is to control task demands over time so that they neither underload nor overload a user, who could be a player or a student. Overload can be detrimental to the user experience as there are too many things to do; the user is flooded with information. Moreover, there exists the stress of underload and boredom of the user in a virtual environment. Boredom and underload are key problems in video games and applications related to education that require some engagement by the audience. In comments associated with applications presenting art content, it is often mentioned that it will appeal to art enthusiasts, but it can be difficult to access for people who do not have prior knowledge about the content. Many tools exist for the evaluation and prediction of mental workload. They can be grouped into three categories based on the type of measurements that are gathered: (1) performance-based measures, (2) subjective measures, and (3) physiological measures [57]. The NASA Task Load Index (TLX), the Subjective Workload Assessment Technique (SWAT), and the Workload Profile (WP) are some examples. The evaluation of mental workload is a key element in the research and development of human–machine interfaces including novel approaches such as VR [58].

The NASA Task Load Index (TLX) [59] uses six dimensions to assess mental workload: mental demand, physical demand, temporal demand, performance, effort, and frustration (see

Table 1. NASA-TLX evaluation.

	Criteria	Question
1	Mental Demand	How mentally demanding was the task?
2	Physical Demand	How physically demanding was the task?
3	Temporal Demand	How hurried or rushed was the pace of the task?
4	Effort	How successful were you in accomplishing what you were asked to do?
5	Performance	How hard did you have to work to accomplish your level of performance?
6	Frustration	How insecure, discouraged, irritated, stressed, and annoyed were you?

). Twenty-step (21 marks) bipolar scales are used to obtain ratings for each dimension. A score from 0 to 100 (assigned to the nearest point 5) is obtained on each scale. A weighting procedure is used to combine the six individual scale ratings into a global score. The weights are determined by paired comparison tasks to be performed prior to the workload assessments. For each of the 15 different pairs (e.g., mental demand vs. physical demand), the operator has to select the dimension that is more relevant to workload. The number of times a dimension is chosen as more relevant is the weighting of that dimension scale for a given task. Finally, a workload score (0–100) is obtained for each rated task by combining the weight by the individual dimension scale score.

The Subjective Workload Assessment Technique (SWAT) [60] is a subjective rating technique using three dimensions (time load, mental effort load, and psychological stress load) and three levels (low, medium, and high) for each dimension to assess workload. It uses conjoint measurement and scaling techniques to develop a single, global rating scale with interval properties.

5.2. Usability

The System Usability Scale (SUS) is a reliable tool for measuring usability. It is based on a series of 10 questions (see

Table 2. System Usability evaluation.

	Questions
1	I think that I would like to use this feature frequently.
2	I found the feature unnecessarily complex.
4	I think that I would need the support of a technical person to be able to use this feature.
5	I thought the various functions in this feature were well integrated.
6	I thought there was too much inconsistency in this feature.
7	I would imagine that most people would learn to use this feature very quickly.
8	I found the feature very cumbersome to use.
9	I felt very confident using the feature.
10	I needed to learn a lot of things before I could get going with this feature.

), with five response options (from Strongly agree ($R_i=5$) to Strongly disagree ($R_i=1$), $1\le i\le 10$). The SUS score (0–100) is defined as follows:

$$\begin{array}{ccc}\hfill SUS& =& 2.5·\sum _{i=1}^{10}\left\{\begin{array}{cc}\hfill cc5-R_i& \mathrm{if} mod(i,2)==0\\ \hfill R_i-1& \mathrm{otherwise}\end{array}\right.\hfill \end{array}$$

(1)

The SUS score is not a percentage; it should be considered only in terms of their percentile ranking.

5.3. Flow

When wearing a virtual reality headset, the user may feel isolated. It is a double-edged sword: on one hand, the user is cut off from the outside world and from the people around him/her; on the other hand, the user can be focused on a task without any distraction from the surrounding areas. In an immersive application, users should be absorbed in what they are doing, with their thoughts and actions flowing freely. Therefore, assessing the flow is a relevant measurement for assessing the quality of a fully immersive VR application related to cultural heritage. The flow can be defined as the positive experience of complete absorption in an activity, which is both spontaneous and effortless [61]. The flow experience can be described following six components [62]: (1) Merging of action and awareness, (2) Centering of attention on a limited stimulus field, (3) Loss of self-consciousness, (4) The feeling of control of one’s action and the feeling of control over the demands of the environment, (5) Coherent, non-contradictory demands for action and clear, unambiguous feedback, and (6) Autotelic nature (no need for external goals or rewards). There are multiple questionnaires that have been created to assess the flow. The Flow State Scale (FSS) is a measure of flow in sport and physical activity settings. The nine FSS scales of the 36-item instrument represent the dimensions of flow, and each scale is measured by four items [63,64]. An example from Engeser et al. [65] of a flow questionnaire is presented in

Table 3. Flow Short Scale (FSS) with item allocation (ABA = Absorption by activity factor, FP = Fluency of performance factor): Scale of the answers: 1–7, from not at all (1) to very much (7).

	Questions
1	I feel just the right amount of challenge (ABA)
2	My thoughts/activities run fluidly and smoothly (FP)
3	I do not notice time passing (ABA)
4	I have no difficulty concentrating (FP)
5	My mind is completely clear (FP)
6	I am totally absorbed in what I am doing (ABA)
7	The right thoughts/movements occur of their own accord (FP)
8	I know what I have to do each step of the way (FP)
9	I feel that I have everything under control (FP)
10	I am completely lost in thought (ABA)

. It also includes three questions (scale 1–9): (1) “Compared to all other activities which I partake in, this one is …” (easy to difficult); (2) “I think that my competence in this area is …” (low to high); (3) “For me personally, the current demands are …” (too low to too high). This questionnaire was used in the Greek cultural context [66].

5.4. VR Symptoms

There are different versions of the Virtual Reality Sickness Questionnaire (VRSQ). VRSQ${}_1$ contains 13 questions grouped into two categories: general body symptoms (General discomfort, Fatigue, Boredom, Drowsiness, Headache, Dizziness, Difficulty concentrating, Nausea) and eye-related symptoms (Tired eyes, Sore/aching eyes, Eyestrain, Blurred vision, and Difficulty focusing) [67]. For its evaluation, for each potential symptom, participants score 0 for no effect, 1 to 2 for a slight effect, 3 to 4 for a moderate effect, and finally 5 to 6 for severe effects. The Simulator Sickness Questionnaire (SSQ) includes 16 symptoms, which can be split into three overlapping categories: nausea, oculomotor, and disorientation: (1) Nausea only (Increased salivation, Sweating, Stomach awareness, and Burping); (2) Oculomotor only (Fatigue, Headache, and Eyestrain); (3) Disorientation (Fullness of head, Dizzy (eyes open), Dizzy (eyes closed), and Vertigo); (4) Nausea and Oculomotor (General discomfort and Difficulty concentrating); (5) Nausea and Disorientation (Nausea); (6) Oculomotor and Disorientation (Difficulty focusing and Blurred vision). The participants report the extent to which they experience each of the different symptoms. They score the symptoms as 0 for no effect, 1 for slight effect, 2 for moderate effect, and 3 for severe effect. The SSQ is then adapted to another VRSQ, VRSQ${}_2$, with only nine items grouped into two categories: Oculomotor (General discomfort, Fatigue, Eyestrain, and Difficulty focusing) and Disorientation (Headache, Fullness of head, Blurred vision, Dizzy (eyes closed), and Vertigo) [68].

6. Discussion

6.1. Recommendations

For researchers and developers to create virtual worlds that are directly geared toward cultural heritage, it is necessary to access the raw materials in formats that can be easily accessible and with high details. It includes textures and 3D models in high resolutions, with all the meta information related to the raw materials: author, date, type of materials, origin, …The historical and/or artistic content should be presented with high details as the VR environment cannot allow any approximations: low-resolution textures and approximate 3D models can deteriorate the immersion, the user experience, and the overall quality of the application. Large collections should be easily accessible in formats that allow the extraction of data and metadata in an easy way.

The size of the screen and their resolutions has substantially increased in the last decades, the development of fully immersive virtual reality technology increases the requirements in terms of quality for the 3D models and textures that can be used in a VR environment. Images that were considered as high resolution for a computer are now not at a sufficient resolution if they need to be displayed in a virtual environment. Such is the case for large paintings. While 360-degree images or panorama images were used in the first VR applications, the resolution and degree of immersion is too limited. Applications that are similar to Google Street View with a sequence of 360-degree images do not provide a realistic VR experience despite having real views of the real world. For posh galleries, exhibition areas, and auditoriums, the preferred size is generally 24${}^{\prime \prime }$× 36${}^{\prime \prime }$ and 30${}^{\prime \prime }$× 40${}^{\prime \prime }$, which represents images of size up to 4500 × 6000 pixels. The problem with VR and the possibility to define large rooms is the temptation to display a complete series of paintings, to show everything from an artist (e.g., all the variations of a painting, all the portraits, …). This quest for completion and complete series cannot hold the attention of the ordinary visitor. After seeing the third or fourth example of a portrait, the user will lose interest and move onto a different activity. This element shows the importance of a museum curator, even within a VR environment, the same way that video games need game designers.

VR applications should be available for different types of headsets and account for the specific constraints related to the controllers, the position of the user (sitting or standing) to accommodate different places and the users. For instance, people with physical disabilities may not be able to stand and turn to access different options; they may not be able to use a VR controller, and alternative input modalities are required (i.e., laser pointer and head pointer). VR locomotion should include multiple modalities for locomotion (i.e., to let the user move through the environment normally, to teleport (long jump/instantaneous)).

6.2. Interdisciplinary Challenges

While it is possible to search in databases of scientific papers (e.g., Google Scholar, Scopus, IEEE Xplore) to find academic papers, it is more difficult to track the availability of VR software related to cultural heritage. The field is inherently interdisciplinary with connections between engineering, science, and the humanities; many applications fall in the category of digital humanities. Research groups require collaboration with archaeologists, historians, visual artists, writers, and film professionals to design and develop virtual cultural heritage software that have both an educational value and are engaging and entertaining for a wide audience. There is a tight relationship between the research process and software, going beyond the field of computer science and software engineering. Software must be used to run experimental paradigms, to analyze data, to analyze results …In short, software runs through the entire research process. However, software is often seen as just a tool to test a research hypothesis, to solve a research problem; the developers of these software do not receive the same recognition and prestige as the people running the experiments. When dealing with research in immersive learning with applications in cultural heritage, software can have a large impact going beyond the number of citations of a research paper: these software can be experienced by a large audience; they can be run in educational settings (i.e., in the classroom), during outreach events, or simply at home thanks to the development of consumer-grade VR headsets. The number of downloads over time, the distribution of downloads across countries, the average time spent during the game, and the number and quality of reviews, which can be directly accessed through the Steam platform or independently through analytics providers, are measurements that can assess the success of a research output. The application then uses an interface of the 3D engine (e.g., Unity, Unreal), and analytics providers offer a backing implementation. Such data analytics provide ways to assess the impact of a VR application, going beyond the number of citations or downloads of a paper.

6.3. Challenges in Education

Despite the availability of applications that could be used in the classroom, fully immersive VR is far from being a technology that is implemented in the classroom in history or art history courses. Beyond the cost associated with the hardware, which can be scaled in relation to the number of students, learning activities must be created to integrate these software. As a limited number of VR devices are often shared among students, there are two types of VR learners: performers who obtain virtual learning experience through direct engagement with the VR applications, and observers who obtain such an experience through observation [69]. It has been shown that the observed VR learning experience can be an adequate alternative to direct VR engagement; these participants demonstrated similar learning patterns in reflection, emotion, engagement, and social interaction [69]. Given the typical size of the classroom and the number of available VR headsets, asymmetric learning activities (one student is using the system, other students are interacting with the student who is using the system, or they are doing something else) must be created in which instructors can align their learning outcomes with the available tools. Driving the learning activities based on the VR software that are available, and not the opposite way, may prevent instructors from fitting a software to the learning outcomes that must be conveyed to students. These potential issues suggest the need to better involve instructors into the development cycles of VR applications related to cultural heritage that can be used in education. Another solution is to give options in the application to tune the content in relation to a specific need, such as a level editor in video games. Furthermore, these applications should include active learning activities and assessments (formative or summative) within the VR experiences to measure the extent to which the learning objectives have been met.

6.4. Gaming Aspects

Many applications related to cultural heritage are experiences with a short duration, with limited interactions, no story, and/or no gaming component. When deploying an application on the Steam platform or any platform that is mainly geared toward a large audience, it can be expected that the content will provide several hours of visits and enjoyment. Cultural heritage applications should not be labeled as walking simulators. This derogatory term represents games focused on gradual exploration and discovery through observation, with limited interactions with the environment. Indeed, most of the VR applications do not possess game mechanics. There is still the need to develop substantially VR applications related to cultural heritage to add gaming components, to turn VR experiences into serious games, enhancing the overall experience, and reaching a wider audience, going beyond people who are interested in art and/or history [70]. It is important that after using such applications, the learner should have acquired new knowledge, skills, and/or attitudes. A gameplay that is based on hidden objects or an object search is well suited for such applications, as it allows the players to search and discover the environment. Another direction that can enrich cultural heritage applications is the development of storytelling to enhance the level of edutainment. Rizvic et al. introduce the notion of hyper storytelling where the information is divided in a set of short stories, emphasizing the motivation factor to deal with the narrative paradox problem [71]. It aims at attracting users through the quality of storytelling and efficient information distribution, enabling users to visit the interactive VR environment of the selected cultural heritage object/site.

6.5. Social Experience

Nowadays, cultural heritage should have as a focus inclusion, diversity, equity, access, and success (IDEAS). Cultural heritage should fully encompass IDEAS by discussing, understanding, and building consensus around its practices and goals. Fully immersive VR provides an efficient tool to improve inclusion, diversity, and equity by bringing cultural heritage to a wider audience. Most museums are in large cities, and students that live far from these museums cannot access their content and have a representation, albeit virtual, of the artworks.

With the increasing number of VR applications dedicated to cultural heritage, VR is becoming an increasingly important tool for the research, communication, and popularization of cultural heritage. Many 3D interactive reconstructions of artifacts, monuments and complete sites have been implemented and proposed to a wide audience, reaching users that would not have visited and experienced these sites otherwise, due to time, financial, or other reasons. Most of these reconstructions are static, and they often miss the human presence. Most of the VR experiences and applications are tailored for single users and lack social experience. As such, virtual museums and art galleries do not allow users to meet other people as in VRChat or ChilloutVR. The lack of social interactions prevents virtual museums from fully representing the museum experience. Indeed, being in a museum includes conversations and social practice sets of actions and cognitive processes, which are enacted in response to, and within, specific socio-cultural contexts and/or within specific social relationships. Visitor research provides a way to explain how visitors take advantage of the opportunities to communicate with other people (e.g., museum staff, visitors). Such research studies about the behaviors of the audience give key insights about the practices happening within the museum [72].

While the social experience in VR is absent for synchronous activities between users, it is possible for users to use digital tools to share experiences with others. On the Steam platform, it can occur using discussions, reviews, and community content where users can post screenshots and comments of an application. Discussions and reviews are an essential part of the life cycle of an application where the developers can implement suggestions and welcome feedback from users. Users can share their experiences, not only to list bugs to fix, but to share information about the content, and give remarks. In this case, the social experience is different: it goes beyond what can be completed in the real world with the constraints of space and time to interact with people, by proposing interactions with a wider audience online and at different times, asynchronously.

Another trend is to include virtual humans in cultural-related VR applications. Such virtual humans can substantially enhance immersion and improve the notion of scale and depth with the environment [73]. A human being in motion can become a personal virtual assistant in digital heritage, helping the user in his/her visit.

7. Conclusions

In this paper, we have presented an overview of the most recent works related to the combination of virtual worlds with cultural heritage using fully immersive virtual reality. A substantial number of applications are not published in peer-reviewed papers and yet propose valuable insights for both research and development. Research papers and software do not have the same life cycles. Software maintenance is a key phase of the software development life cycle; as long as there are users, the software should be maintained. Hence, the content in VR applications can be updated and improved over time, making their evaluation difficult. Since the mid 2010s, consumer grade virtual reality headsets have changed the way people, including students and researchers, can visit cultural heritage sites through fully immersive virtual reality. The COVID-19 global pandemic has played the role of a catalyst for increasing the needs of VR applications dedicated to cultural heritage for a wide audience. The diversity of hardware (different headsets, different levels of performance) has created a large gap between applications and what users can expect. All these applications must be rigorously assessed at different levels as any new software and human–computer interaction experience.