The Virtual Experience for Cultural Heritage: Methods and Tools Comparison for Geguti Palace in Kutaisi, Georgia ^†

Abstract

The virtual experience of Geguti Palace will be presented in this paper in order to demonstrate how it’s possible to offer a deep understanding of a space from abroad by absorbing reality into a virtual environment. Starting from the digital documentation and 3D survey of this medieval royal palace, developed by the Department of Architecture of the University of Ferrara and the Tbilisi State Academy of Arts, in collaboration with the National Agency for Cultural Heritage Preservation of Georgia, the paper will stress the VR issue focusing on several hardware devices and software platforms for digital content management. This case study will contribute to the analysis of further exploitations of Virtual Experiences for the Cultural Heritage field.

1. Introduction and Approach

Over last years, methods and processes for data collection have faced a fast and effective improvement, allowing the possibility of a growing number of interdisciplinary applications thanks to a lot of valuable research project on this topic. Recently, the most common technologies on the market for 3D data capturing are based on images (e.g., SFM, photogrammetry) and range data (e.g., active sensors like laser scanners). Both approaches have advantages and disadvantages and, generally, the case-by-case choice is done accordingly to several criteria as the budget, the project size, required details and objectives. Image-based methods are widely used for the 3D reconstruction of architectural objects and for the precise modelling of terrain and cities or monuments and statues [1].

In particular, 3D photo-based scanning allows achieving simplified and lightweight models in a very fast and efficient way, which is essential for AR and VR applications, especially for the texturing quality provided. The resulting models are useful for a wide variety of use cases, such as visualization, archiving, tourism, simulations, guiding interventions, and much more [2].

By absorbing reality into a virtual environment, a much deeper understanding of the space in a very holistic way could be gained [3]. They can even be an effective means of archiving physical spaces so they can be digitally preserved, such as in a museum setting, enabling remote viewing of those spaces that are difficult or impractical to access physically or do not exist anymore. From researchers and scholars to visitors and tourists, new digital technologies allow developing user-oriented narratives and to access the site from abroad, preserving its memory and disseminating its value all over the world. By this, an inclusive and flexible valorization strategy can be arranged. The access to the digital model could be also provided directly on site, in order to enrich the monument enjoyment. Moreover, the promotion and valorization phase should take the opportunity to exploit the captured technical data for the digital cultural heritage accessibility, making contents and resources available for as many people as possible by using ICT functionalities and applications.

As demonstration case of this theoretical approach, the digital documentation and 3D survey of Geguti Palace will be presented in this paper. It is a work-in-progress joint research, developed by the Department of Architecture of the University of Ferrara and the Tbilisi State Academy of Arts, in collaboration with the National Agency for Cultural Heritage Preservation of Georgia.

In particular, this paper will stress the VR issue focusing [Figure 1] on several hardware devices and software platforms for digital content management. This case study will contribute to the analysis of further exploitations of Virtual Experiences for the Cultural Heritage field.

State of the Art: A Brief History of Virtual Reality Devices and Software Platforms

We can define Virtual Reality (VR) as a set of techniques that allow simulating an experience of reality, even if partial, by the use of images, sounds, smells or the perception of movements. Virtual reality artificially creates immersive sensory experiences of physical presence in places in the real world or imagined worlds and lets the user interact with that world.

VR’s first experiences in the 1960s and 1970s represented a fascinating project of the future. The first use of a VR presentation in a heritage application was in 1994 when a museum visitor interpretation provided an interactive “walk-through” of a 3D reconstruction of Dudley Castle in England as it was in 1550. This consisted of a computer controlled laserdisc-based system designed by British-based engineer Colin Johnson [4].

Since then, the huge technological advancement and several experiences in this field have improved the state of the art [5]. Nowadays, virtual reality is becoming more and more an instrument applicable to many areas of science and visual communication. Furthermore, the experience can be also accessed from different places and at different times even when the original sites are inaccessible to the public or no longer existing [6]. In this framework, the case study of the valorization of Geguti Palace allows addressing different themes regarding instruments and techniques for VR’s use in cultural heritage.

The wearable devices used for the immersive VR, known as HMD (head-mounted display) or today commonly called headset, allow experiencing a simulated reality once they are worn. They can be subdivided into 2 macro categories:

• Non-portable systems—devices connected to an external graphics processing unit (such as Oculus Rift or PlayStation VR)
• Portable systems—devices in which the graphics processing unit is embedded into the headset (such as Samsung VR Gear)

From the point of view of the immersive visual experience, the software techniques (referring to the computational graphic field) are raster or vector-based with CGI-specific techniques through the use of real-time rendering. Advantages and disadvantages are related to the choice of used hardware and software configurations to create immersive content in VR.

Non-portable HMDs feature low latency high resolution (up to 4k) displays with a wide viewing range (more than 120°) that are connected by wire (such as a common monitor) to a graphics workstation where real-time graphical information is processed. The movement is given by the gyro sensors inside the HMD. Graphical quality can be extremely high and directly proportional to the hardware features, enabling a clear scalability of the system. On these systems, due to the available computational power, such as video card SLI techniques, vector 3D graphics with real-time render is mostly used. Differently, the portable HMDs arise from a Google idea with the Cardboard project [7], starting from the assumption that a smartphone already contains all the necessary hardware, such as multi-core CPUs, dedicated GPUs, high-resolution displays, compass, gyroscope and the ability to enjoy 360° graphic content. Google Cardboard is a low-cost development platform for Virtual Reality through the use of a smartphone, with a specific mobile application package and a cheap head mount for smartphones.

In parallel, hardware and software solutions have been developed during years, bridging the gap between the two systems with common elements in shapes and functionalities. Oculus VR, acquired in 2014 by Facebook, was the company that has driven the development of the industry both in reference to non-portable HMDs with Oculus Rift and portable HMDs through the partnership with Samsung since 2015, within which the Samsung VR head mount for Galaxy Smartphone Series has been developed.

Portable HMDs are an interesting solution both in terms of cost and ability to be used in different contexts. By contrast, the computational power compared to the above-mentioned graphic workstations is tens of times lower, which, as in the present case, implies the development of immersive solution more oriented to the use of raster than 3D vector graphics. Not least, the mobile HMDs were the first low-cost devices to offer the possibility of Immersive Augmented Reality.

Regarding the creation of contents, the navigation experience and the enjoyment of VR immersive environments [8], the gaming industry is driving, since 2008, the development and optimization of solutions to increase the VR experience, offering a full immersion. It is worth noting how some manufacturers do not point to the performance improvement of the devices, but on the comfort of use to allow the continuous use for long sessions. The need to release video games with low production costs (online games, mobile devices, portable consoles, etc.) has brought some software houses to create multiplatform authoring tools for creating 2D/3D video games or other interactive multimedia content such as static visualizations or 3D animations in real time. Nowadays, the most popular platform in the VR/AR sector is UNITY3D. Also, the LUMBERYARD platform (which is integrated with other Amazon Webservice IT services) has been released, even if still under beta version.

2. Materials and Methods

The joint research activities on Geguti Palace have been performed by the direct funding of the two universities and the National Agency for Cultural Heritage Preservation of Georgia. While the survey project has been developed by the Department of Architecture of the University of Ferrara, the fieldwork has been performed, in summer 2016, by an interdisciplinary team that consists of researchers by both institutions, in order to ensure a valuable understanding of the site.

Furthermore, thanks to the institution involvement of the National Agency for Cultural Heritage Preservation of Georgia into the stakeholder panel of the INCEPTION project, coordinated by the Department of Architecture of the University of Ferrara, was possible to directly exploit preliminary results and tools by the EU-funded research. The data processing was addressed at the creation of a comprehensive database for research purposes and future interventions, as well as knowledge dissemination. Thus, the immersive virtual environment was developed trying to reach the best compatibility with a headset experience by the user. Furthermore, the related narrative can be displayed directly into a 3D space, connected to specific objects, or it can be listened during a free virtual tour. The application has been already tested during exhibitions, in order collect feedbacks to further improve the user experience, getting to a final version that could be integrated into permanent collections. As well, a specific version could be available for download by users around the world, promoting the real site visiting and its surroundings.

2.1. The Survey of Geguti Palace

The earliest structure of Geguti Palace was built as a summer residence for Georgian Kings and dates back in XII-XIV centuries. The Palace is frequently mentioned as a beloved place of the Georgian royalty [9]. This medieval royal palace, located near Kutaisi (Georgia) on the Rioni River, at the border of European territory, is a great opportunity to better understand the relationship between our countries and cultures, and our common roots. Nowadays, the palace is in ruins and archaeological excavations are still ongoing. But, anyway, it is the only Georgian royal palace preserved till today and, nevertheless, is categorized under National Importance by the Georgian state [10].

The complex includes the palace itself and the church. Also, there are ongoing archaeological excavations on the territory, which proves the existence of previous structures. The Palace spreads on about 2000 sqm and it is composed by the huge central hall and small additional rooms around it. Bricks are the main building material but, depending on the different building periods, river stones, lime stones and different kind of bricks are used. The palace is distinguished with its history and architecture.

The survey campaign aimed at achieving a comprehensive documentation of the site [11]. The multi-layered nature of the structure asked for an integrated data capturing and an aware usage of instruments [12]. The following ones have been used in order to include metric information as well as the state of conservation:

• Geomax Zoom 300, a time-of-flight laser scanner provided for free by Geomax AG, for metrical survey;
• NCTech iStar 360, a spherical camera by NCTech, for 360° virtual tours [Figure 2];
• Samsung WB690 compact camera for photo-modeling source and photographic documentation.

In the processing phase, the 3D model from the laser-scanner point cloud has been used to connect several three-dimensional SFM textured models and 360° photos allowing a non-technical user to easily interact and understand the site. During a one-week fieldwork, the following results have been achieved:

• creation of the high-density 3D point cloud composed by the 200 million points taken from 60 stations. The dataset was registered with ICP method, average error—3.8 cm, using X-PAD Office FUSION by Geomax;
• capturing of 30 HDR 360° spherical photos in order to develop an exhaustive visualization for VR (Virtual Reality) viewers;
• creation of 3D models for 17 archaeological excavation areas by the photo SFM modeling in order to create and easily updatable documentation of excavation processes.

2.2. VR Contents and Tools Choice

The development of contents for portable HMDs has been carried out looking for the achievement of flexibility and small equipment cost, which often means more easy content dissemination.

All kind of data captured during the survey campaign (three-dimensional point clouds and 360° panoramic images) were structurally useful at the three-dimensional accessing and understanding of the site, allowing a variety of ways to build immersive experiences, but technically speaking, the development and implementation choice for VR has been oriented to the use of 360° images and video in relation to the following reasons:

• the photographic data is the simplest formal coding of the space shared by age groups (especially in relation to childhood and the elderly), allowing an easier and immediate immersive experience;
• the raster content requires less computational resources, making them more suitable for mobile devices;
• simplicity and cost-effectiveness can be achieved in the authoring process;
• the immersive content can be used by mobile devices even without the need for HMDs;
• the produced content is easy to port to other and more commonly used digital platforms for the communication project, such as enjoying the Virtual Tour directly from the website of the complex;
• point-to-point 3D models suffer from obvious flickering issues (which heavily penalize fruition) on use in portable HMDs due to the frequency of refresh in stereography.

We have gone through the analysis of the types of usable HMD mobile devices and their software solutions, focusing on three different product types, identifying the technology and the price range [

Table 1. Devices comparison. The chart summarizes main features of each device.

	Meizu M1 Note	LG Nexus 5	Samsung Galaxy S6 Edge	Wimius 3D VR
OS	Android 5.0	Android 5.1.1	Android 6.0.1	Andriod 5.1 + Nibiru
Chipset	MediaTek MT6752	Qualcomm MSM8974	SAMSUNG Exynos 7420	ND
CPU	Cortex-A53	Snapdragon 800	Cortex-A53 Quad-core 1.5 GHz	Cortex-A53
	Octa-core 1.7 GHz	Quad Core 2.3 GHz	Cortex-A57 Quad-core 2.1 GHz	Quad-core 1.7 GHz
GPU	ARM Mali-T760 MP2	Adreno 330	ARM Mali-T760	ARM Mali-450 MP6
RAM	2 GB	2 GB	3 GB	2 GB
HD	32 GB	32 GB	32 GB	8 GB
Display	5.5′′	4.95′′	5.1′′	5.5′′
Resolution	1080 × 1920 pixel	1080 × 1920 pixel	1440 × 2560 pixel	1080 × 1920 pixel
ppi	401 ppi	445 ppi	577 ppi	400 ppi
Battery	3140 mah	2300 mah	2600 mah	3000 mah
Release date	14 October	13 May	15 February	16 November
Tested HMD	TecnaXX VR TX-77	TecnaXX VR TX-77	Samsung Gear VR	Wimius 3D VR

].

The first is a typologically “cardboard” system. It has been tested with good quality smartphones, even if not last-generation ones (MEIZU M1 and LG Nexus5), in combination with a cheap head mount (TecnaXX VR Glasses TX-77) that, anyway, could be worn enough comfortably.

The second system could be actually considered the standard in the world of portable HMDs: the Samsung Gear VR By Oculus with the Galaxy S6 Edge smartphone, a high-end product.

The third system is a low-cost product explicitly developed for VR immersive experience. A true portable HMD that integrates all hardware and software internally (Wimius 3D VR all in one).

3. Results

The content production focused on the requirements by the European Commission’s invitation to participate in the Europe Day 2017 celebrations in Brussels, as one of the demonstration cases brought by the INCEPTION project [13]. Indeed, the event has been used as test and verification field for the immersion capabilities of the systems presented above, due to the availability of large and differentiated audience of families with young children.

Immersive virtual tour of the Geguti Palace has been produced avoiding interactive dynamic environments since they would require a specific training phase in order to offer the real enjoyment of the contents. It was definitely important that the tour was structured to guide you through Geguti Palace, adding information from the historical, cultural and geographic point of view.

This was ensured by the articulation of the didactic content, fitting the most appropriate ones on the specific 360° image. Thus, we produced a 360° video where textual and graphic information appears on a spherical photographic description of the reality and guide you through the archaeological site of Geguti Palace, telling its story and the history of Georgian Dynasties. Furthermore, information from the excavation phases that allowed reconstructive hypothesis on the architectural configuration has been included as well.

The main strategy carried out with the 360° video was to immerse the user into a virtual space and let him follow the narration without any instruction on using VR devices.

The Immersive Experience

In order to achieve this, we have been using the Affinity software that allows us to quickly edit the two-dimensional planar development of spherical images, enabling the insertion of text and graphic information with the right projection in the space. The 360° raster images produced have been then processed with the most appropriate timing in order to create the video. The high-resolution exported video (8000 × 4000 px) has been enriched with specific metadata to make it readable by players and 360° navigable [Figure 3]. Different videos were produced for the Brussels event:

• a 15-min long versions with more detailed historical information;
• a 2-min short version with a selection of the main content.

The experience is completely immersive since the very first few seconds in which only the spherical environment is displayed. Then, after 15 s, the first textual information and graphical elements start to appear and guide the user through the tour.

For the long version, twenty-two nodes (360° photos) has been used. Only four for the short one. The average duration per node is about 40 s. The timing has been identified to make the visit as much entertaining as possible, building the narration on the complex thanks to the alternation of texts and images. Particular attention was put in the choice of the font and its size to allow the user to read it effortlessly [14]. Also, the vocabulary was to 6000 terms in order to make the text accessible and comprehensible even to young children.

4. Discussion

In addition to testing and verifying the appeal of VR in relation to the different age groups, the Brussels test has also made possible to check the usability of the devices. Indeed, the three devices behaved in an extremely different way.

The Samsung Gear VR by Oculus in combination with the Galaxy S6 Edge smartphone proved to be the most mature product considering its hybrid nature (smartphone and gear VR). The navigation experience is externally fluid and addictive. Samsung app works perfectly and allows you to enjoy contents through a simple and dedicated GUI, specifically designed with a visual point-and-select input system, integrated with physical buttons on the head mount. VR mode is automatically activated when the smartphone is put on place inside the head mount. The single focus wheel is simple and precise.

The only main downside is related to the display management. Even if it is a 3 K (2560 × 1440), every content is resampled at a much lower resolution (HD—1280 × 720), in order to reduce the load on CPU and GPU and allow a very fluid experience in movement and rotation as well as reduce energy consumption. Furthermore, the Samsung Gear has a special sensor to detect if the head mount is worn or not and to pause the device accordingly.

The “cardboard” system shows all the limitations of this technology on low-cost devices. Since the head mount is compatible with all smartphones with a 3.5′′ to 6′′ display, the focusing procedure needs to be performed every time differently for each device and user, through uncomfortable scroll sliders that have to adjust the interpupillary distance too. This leads to a great difficulty in approaching the technology and often insufficient fine regulation, hence causing a non-enjoyment of immersive content. Must be noticed that older people and children under the age of 7 have failed to use this type of device, confirming the inadequacy of low-cost systems for the purpose.

Furthermore, low-specs devices often do not have enough resources to make the VR experience fluid and comfortable. The overheating of the devices is significant and led to stop the device after about 50 min of continuous use. The battery allows no more than 2 h without charging it.

For what regards the usability, applications must be started before putting the device inside the head mount and the interaction with the device is limited and based on physical buttons or optional Bluetooth remote.

The third tested system (Wimius 3D VR all-in-one) proved to be a very well balanced product. This is a new category of VR devices that, as we have already said, integrate all hardware components into a single device at an extremely competitive price. The navigation experience is fluid and engaging with an outstanding definition for a static use but a slightly worse quality for dynamic one. Anyway, the experience is still a lot different from the Samsung system in which everything is more homogeneous, smooth and natural.

This device is based on Nibiru VR AIO operating system, a platform for mobile devices. Even in this case, there is a dedicated GUI, specifically designed with a visual point-and-select input system, integrated with physical buttons on the head mount. The platform is extremely interesting since also the SDK is freely available. As well, the system is available as an android app, to be used use with cardboard.

Because the system does not have removable parts, it doesn’t need for any focusing system, greatly simplifying the interaction experience, especially for elderly and children. Anyway, it has no active system for preserving battery life. No overheating problems with intense use have been found.

In conclusion, the Samsung product looks like the most mature, even in terms of comfort (use for hours), but surely the Wimius all-in-one product comes out as a great solution in terms of price and development environment and, for this reason, it can be a good solution for large-scale usage like schools, museums, etc.

5. Conclusions

The paper focused on portable HMD immersive systems in relation to the enhancement of the Geguti Palace site. The data collected during the survey campaign was the starting point for building a VR experience. From the state of the art analysis and the testing clearly emerged that cardboard devices are the easiest access to VR’s world, even if the immersive experience is often difficult to use and not so engaging. On the other side, specific devices (i.e., Samsung or Wimius) allow extremely easy, efficient and immersive use.

One of the critical issues of portable HMD is the poor scalability of the system (better in the Samsung system), partially offset by the low cost of non-portable HMD. We expect in the near future the development of hybrid systems, able to operate both independently (portable HMD) or connected to a workstation or console (non-portable HMD).

Currently, available solutions on the market allow VR experiences with low production costs by the use of medium-range of hardware and software. Most of those tools are mature enough to be considered useful for educational purpose and cultural dissemination.

During the Europe Day 2017 initiative, the presented VR systems have been tested on over 200 people of different ages [Figure 4], enabling us to test both the immersive VR technology and the ease of use of the devices. Most of the time, users were able to easily enjoy the VR experience. From the very first seconds, users were already moving freely into the virtual space, even looking for an interaction with this. It was deeply impressing the immersive experience of pre-school children, who often needed compare the reality with the virtual environment.

The Brussels experience has shown how VR technology can be very successful both in the edutainment sector as well as in the teaching one. A wide amount of already existing digital contents can be easily and quickly adapted to new VR environments.

Surely the next hardware development of mobile devices will also allow enhancing the use 3D graphics for realizing immersive and more interactive experiences even with portable HMDs.

Different realities, simulated realities, dreamed realities, invisible reality, future or past realities represent a wide scenario where VR can find a huge field of application. A next future already became reality.