Using Different Ways of 3D Reconstruction of Historical Cities for Gaming Purposes: The Case Study of Nafplio

Abstract

Virtual reality (VR) technologies offer a unique opportunity in “3D reconstruction” of cultural monuments and historical cities that do not longer exist or have been modified (partially or totally). Even though technological capabilities are growing fast, emphasis is mainly given on developing virtual museums and exhibitions, while archaeological places and excavations are following. The present paper presents two distinct approaches for 3D reconstruction of the historical city of Nafplio (Greece), by developing 3D models for buildings and monuments the way they used to be in the 19th century. The authors have used their professional and academic experience deriving from their participation in a European Commission co-funded project in the framework of the Partnership Agreement 2007–2013, which was implemented by the “V. Papantoniou” Peloponnesian Folklore Foundation (the project’s beneficiary) in Nafplio. The authors aim (a) to present the importance of reconstructing past cities and (b) to implement gaming technologies in cultural organizations as a means of achieve specific educational goals. Results point out the growing need for close cooperation between the cultural and academic worlds with 3D creators, and, moreover, reveal obstacles and opportunities in reconstructing historical cities and monuments.

1. Introduction

During the last five years, already existing technologies such as Virtual Reality (VR) have rapidly emerged, creating a dynamic environment with great opportunities for “3D reconstruction” of cultural monuments and historical cities that do not longer exist or have been modified (partially or totally). Digital technologies have developed enough sophisticated tools to create realistic objects and environments [1] in order to offer a much richer user experience, while the idea of visualizing three-dimensional (3D) context is gaining pace from both a technological and a cost-efficient perspective.

Exploring opportunities for 3D reconstruction has gained researchers’ interest for more than 15 years. The European Union has funded a series of research projects whose complexity and interaction with mobile devices (such as mobile phones and tablets) have gradually increased. These projects have mainly focused on three distinct areas of interest:

• Archaeological excavations where the public access is limited and there is limited physical content to be actually viewed,
• Historical places that do not longer exist or have been modified,
• Monuments, sculptures, and artifacts with limited access or that no longer exist.

One of the pioneer projects was the SHAPE [2], aiming to educate users on archaeological artifacts and their history. A more complex project was 3DMURALE, aiming at developing a series of 3D multimedia tools applied to various aspects of archaeological ruins (such as terrain geometry, buildings and their parts, statues, and textures). The proposed tools offered solutions in encoding, reconstructing, and visualizing the ruins, offering a rich user experience [3].

Another case was the Ename 974 project, which used IBM’s 3D computer technology (called TimeFrame) to develop an interpretation system providing knowledge on the archaeological site and the history of Ename, leading to the reconstruction and communication of archaeological and historic remains with multimedia and virtual reality technologies [4]. The ARCHEOGUIDE project (Augmented Reality-based Cultural Heritage On-site GUIDE) was one of the first augmented reality implementations [5] to deliver to users a multi-modal interaction user interface and AR reconstructions of ancient cultural sites [6].

Following the same pattern, LIFEPLUS explored users’ experience on 3D simulations of virtual humans, animals, and plants in real-time [7]. Finally, the PURE FORM project concentrated on 3D reconstruction of art pieces, creating two separate systems; one for museums and another one for gallery-like environments [8]. Both systems relied on the idea of creating virtual environments where the user could interact with digital representations.

Apart from the first research attempts, there are some more recent projects worth mentioning. A 3D model of Prague was developed based on the scale model of Antonin Langweil, which was created between 1826 and 1837 [9]. The “Romereborn 2.0” project presented the urban development of ancient Rome (from 1000 BC to 500 AD), with emphasis on well documented buildings (Circus Maximus, Colosseum, etc.) and implementing credible but not necessarily true buildings for the rest of the city [10]. Similar projects have been developed for Hamburg [11], Marsal, Aire sur la Lys, and Saint-Omer [12].

Even though there is ongoing research on 3D reconstruction, this research reveals several limitations. The first limitation is related to the reasons why 3D models are developed. A recent research by Münster and Koehler [13] revealed that reconstructions are most commonly used for presentation. This fact is related with the general trend of creating 3D models for existing buildings/artifacts, while the no-longer existing structures are not preferred. This leads to the second limitation, related to the limited systematic research on how 3D modeling is affected by disciplinary boundaries related to historic topics [13]. Model building is one aspect, but historical documentation is an equally significant task, requiring cooperation between researchers coming from different disciplines (e.g., technological, historical/archaeological, and social sciences). A third limitation comes from what Vayanou et al. [14] considered as social context, which shapes visiting experience. Including “personal, sociocultural, and physical context” in a 3D reconstruction is crucial for developing the framework needed to organize the complexity of meaning-making applications.

Most of the projects described in the above publications have dealt with data acquisition and model building for existing artifacts. Only a small number of projects have focused on structures that no longer existed physically. What are the current trends regarding 3D reconstruction of cultural heritage artifacts? There are many individual projects using 3D technologies to reconstruct historic items. Research shows that such reconstructions are most commonly used for presentation, research purposes, and sometimes for digital conservation of large buildings or city models and are often realized by interdisciplinary workgroups.

Taking into account these limitations, this article’s scope is to portray and present various 3D reconstructions of the historical city of Nafplio, including monuments and buildings that either do not exist anymore or have been modified. We aim (a) to present the importance of reconstructing past cities and (b) to implement gaming technologies in cultural organizations as a means to achieve specific educational goals. Results point out the growing need for close cooperation between the cultural and academic worlds with 3D creators, and, moreover, reveal obstacles and opportunities in reconstructing historical cities and monuments.

2. Nafplio in the 19th Century as Case Study—Methodology

Authors used their professional experience coming from both (a) the City of Nafplio 3D depiction, as was shaped in the 19th century, and (b) the educational applications’ development by using gaming technologies. The aforementioned experience came from their participation in a European Commission co-funded project in the framework of PA 2007–2013 which was implemented by the “V. Papantoniou” Peloponnesian Folklore Foundation (project’s beneficiary) in Nafplio.

The historical city of Nafplio, as was shaped in the 19th century (Figure 1), incorporates various cultural characteristics as a result of the impact that different conquerors left behind (Frankish, Venetian, and Turkish rules). Moreover, the city took active part in the Greek Revolution of 1821 and was the first Capital of the new-born Greek state. New monuments and buildings were added, while older were ruined or changed their usage, adding new elements to their architecture which were closely related to the first steps of the Greek state.

Within the project, a 3D reconstruction of Nafplio was developed for educational purposes, based on original architectural drawings of the buildings and the city in the 19th century. The reconstruction, as well as the initial character and the objectives of the educational applications, where defined by the project’s beneficiary. The foundation’s educational purposes included offering online visitors a historical game based on 3D reconstructions linked with a multiple choice quiz. By using gaming techniques, it aimed at reaching a larger audience of primary and secondary school students all over Greece, supporting their historical background knowledge of Nafplio, the first capital city of Greece. The gaming approach has been selected from the foundation as the most appropriate for increasing historical knowledge and linking students with specific historical monuments and buildings of Nafplio. The game is still available to play for free at the following link: http://www.pli.gr/en/content/19th-century-nafplion (proposed browser for web playing: Internet Explorer).

Once the project was completed, the authors developed a whole new 3D application, which was based on the existing 3D models from “Nafplio in the 19th Century” game (Figure 2). Both games are presented in the next section of the current paper.

The project team faced a series of challenges coming mainly from the city’s history, including aspects such as:

• Monuments that no longer exist/buildings that are only presented in photographs or gravures;
• Existing monuments/buildings with interventions coming from more recent years.

In order to understand the extent of this research and its difficulty, a list of 3D reconstructions and the status of each building/monument is presented (

Table 1. List of buildings/monuments reconstructed via 3D.

A/A	Name of Building/Monument	Status
1	Church of Agios Spyridon	Existing
2	Venetian Palace	Existing
3	First Cadet School of Greece	Existing
4	Joseph Ludwig von Armansperg Residence	Interventions
5	Fragkoklissia Church	Existing
6	Greek High School	Existing
7	Turkish Hammam	Interventions
8	Statue of Ioannis Capodistrias	Existing
9	Government House	Non-Existing
10	Monument to Ypsilantis	Existing
11	Georg Ludwig von Mauer Residence	Interventions
12	Manto Mavrogenous Dwelling	Interventions
13	Calliope Papalexopoulou Residence	Existing
14	Land Gate	Non-Existing
15	The Trianon (Old Mosque)	Interventions
16	“The Vouleftiko” (Old Parliament Building)	Interventions
17	Saint George	Existing
18	First official pharmacy	Interventions
19	Peloponnesian Folklore Foundation	Existing

). The main difficulty arises from major interventions that buildings have undergone as a result of changes of different authorities (Venetian, Ottoman, Greek), while many buildings have also undergone changes in their primary use (from mosque to prison, from church to mosque, etc.). For all these cases, historical research was conducted and evidences were used to represent the building’s view as it was in the 19th century.

Following existing literature for diachronic reconstruction [15,16], a three-dimensional survey was conducted including (a) current state (if applicable), (b) recognition of interventions from different ages, and (c) historical/philological research. Bibliographic research and historical documentation not only provide description of the monuments/buildings’ state but, moreover, reveal interventions. Supplementary material regarding conditions of living and the city’s general perspectives (e.g., a wealthy state or after-war ruins) can help researchers understand what a visitor would “see by himself” in order for researchers to make 3D reconstructions more realistic. Collections of maps and images are rather helpful to recognize major interventions in a city’s urban structure. Drawings and paintings can reveal important details on non-existing monuments/buildings.

Another important element of analysis are photographs from the past, but, moreover, from the current state. Photographs of the past are important because in some cases the under-research monument/building is revealed as it was in the 19th century (even though the photograph is coming from the first years of the 20th century). Moreover, photographs from different angles inside the urban area and from different periods of the city’s history lead to a successful 3D reconstruction (Figure 3). Current photographs are also important because they represent the current view and facilitate recognition of external interventions. For buildings that have small (or no) historical significance, representations of elements, with the same typology and age, were developed.

For the purposes of this project, a more gamified approach was chosen in order to showcase Nafplio’s cultural heritage. As a result, the workflow and tools chosen to complete the project share a lot of similarities with the game production pipeline of a small game. The whole process was separated into four smaller phases which are represented in Figure 4:

• Pre-production: Including game design, concept art, and mechanics;
• Prototyping;
• Production: Including 2D art and 3D models as well as putting together a level;
• Polishing.

Pre-production: During the initial phase, the whole material was evaluated starting from instructions and assets to concepts and ideas. A quick pen & paper prototyping procedure followed in order to figure out which ideas work and which do not. The main focus lay around setting the tone for the game. During this phase, only Microsoft Office products where used (Word & PowerPoint) to quickly share and bounce off ideas from one another.

Prototyping: During this phase different aspects of game development were combined in order to test the ideas proposed in the previous phase. These ideas include art, music, design, and code. No attention was given to detail since everything would be soon thrown away, while the main objective was to test if the core ideas of each prototype worked. During this phase, various tools were used such as Photoshop for concept arts and initial designs, 3DS Max to create all necessary assets, Visual Studio for scripting, and, finally, Unity3D to bring everything together.

Production: After deciding on which prototype to base the game on, the team started to work from scratch. This is done in order to follow the correct design/code/art principles and secure the longevity and future development of the project.

Polishing: During this phase the project’s initial output was shared with a relatively small user base and adjustments were made based on their feedback in order to get the best possible result.

As far as tools are concerned, a large variety was used. For designing, idea sharing, and presenting, as mentioned before, Microsoft Office & PowerPoint were used, because it involved sharing documents with each other and everyone was already familiar with the software. For scripting, Visual Studio was used since, by that time, it had recently added support for Unity which was the main game development engine of choice. Additionally, Visual Studio was already quite popular among most developers. For making 2D art as well as guidelines for 3D Artists, Adobe Photoshop was used as it is compatible with both 3DS Max and Unity as well as being the industry’s standard. For 3D Art, we used 3DS Max, since most of the models were structures and not characters.

Last but not least, in order to bring everything together and create the actual game, the Unity3D engine was the preferable choice. Being quick with prototyping, having compatibility with all other previously mentioned tools, and being able to export the game in a variety of different formats made Unity the ideal choice.

The methodology used (Figure 5) that stands out in this project, was the recreation in digital format of cultural heritage structures, specifically because they do not exist anymore. There are five steps to that process: (a) Gather reference photos and information regarding the structure, (b) have the concept artist draw a picture of it, (c) have experts review the picture and decide if it is an accurate or plausible representation, (d) based on the concept art, draw a 2D architectural representation of the structure for the 3D artist to use, (e) create the 3D model of the structure.

Figure 2, Figure 3 and Figure 6 present some photographic evidence which was used for 3D reconstruction of existing historical buildings in Nafplio. Most of the used material and evidence of Nafplio as it was in the 19th century came from “V. Papantoniou” Peloponnesian Folklore Foundation. Established in Nafplio, the foundation assumed a rich collection of folklore artifacts and historical archives for the city throughout the centuries. Holding an educational role, the foundation intended its material to be exploited in order to build educational, digital games.

This kind of approach is widely accepted and well recognized from both the academic and business community. Gamifying cultural heritage (e.g., serious games) or even creating commercial games is a successful methodology of re-using digitized content from galleries, libraries, archives, and museums (GLAM). This trend was facilitated by the following factors:

• The great and large-scale efforts of many cultural institutions to digitize tangible cultural heritage [17];
• The on-going procedure of digitization of intangible heritage (e.g., knowledge, skills, traditions) [18];
• The creation of widely accepted standards for metadata descriptions [19];
• The technological advantages in areas such as Virtual Reality and Augmented Reality [20].

Two major types of games using cultural content are developed: (a) Entertainment and (b) serious games. The former mainly deliver the so-called documentary games [21] (e.g., battle games relying on historic events), which can be used for non-leisure purposes such as cultural content presentation [22].

Even though most users target personal entertainment, these kind of games serve educational purposes at the same time because of their historical accuracy and realism [23]. The latter, also known as applied games, use virtual reality and augmented reality technologies [24,25] to embody users in realistic and historical scenarios, through interactive simulations. Types of such games are:

• Quizzes–tasks–explorations (interactive, virtual games): Games commonly found in museums, such as “Walk through Ancient Olympia” [22] and “ThIATRO” [26];
• Demonstrators (3D reconstructions): Games modeling historical cities, such as “Pompei: The Legend of Vesuvius” [27] and “Roma Nova” [28];
• Skills and language training games (3D simulations): Games aiming at reaching cultural knowledge and understanding intangible cultural heritage, such as the Tactical Language and Culture Training System (TLCTS) and the “Virtual Environment Cultural Training for Operational Readiness” [29];
• Knowledge games: Encouraging players to bring accurate metadata about cultural artifacts, such as “One-Up” which is a mobile game with metadata tagging orientation [30].

In the following section, the two games are presented while a short technological analysis is conducted.

3. Three Dimension (3D) Reconstruction Games for Cultural Heritage of Nafplio

“Nafplion in the 19th Century” is a web and mobile game (available free for online visitors at: http://www.pli.gr/en/content/19th-century-nafplion) which focuses on the history and the important buildings of the city of Nafplio as it was back in the 19th century before any interventions. The game is developed on the game engine Unity 3D and the models are designed on Blender software. The gameplay is divided into three levels of difficulty targeting different ages from early school years to adults. The main goal is to answer several questions regarding the history of the city (5 questions for the easy level, 10 questions for the medium level, and 15 questions for the difficult level) and the game can be played by a single player or two players acting as opponents.

In the game, there are 19 buildings that were implemented as 3D models based on historic evidence (drawings, prints, gravures) in order to represent the actual buildings of the 19th century. The game is staged on a 3D terrain where the 3D models are placed (Figure 7), while the camera is initially located on the sky and is “zooming in” on the appropriate building each time a question is selected. Once the answer is submitted, the camera is “zooming out” and placed in the initial status. This approach was chosen from the project’s beneficiary to achieve greater emphasis on the educational goal of deriving knowledge about historical buildings, rather than exploring the whole city (a more entertaining approach). Apart from the actual game section, there is a presentation section in the application which presents the 3D models of the buildings, one by one, in order for the player to get a first impression of them and be prepared for the following questions.

One of the most challenging aspects was to place the buildings/monuments that no longer exist on valid positions. Such an example was the “Kiverneio” (translated as Governorate, Figure 8) which was totally destroyed in 1929. Difficulties arise in both: (a) Developing a historically accurate 3D model and (b) placing it on an architecturally correct place. Cooperation between developers and researchers was essential.

The second game is more of an entertainment game with educational aspects. The already available 3D reconstructions and historical documentations were used to create an environment where the user can be navigated inside the city of Nafplio. The game’s purpose is to give the user historical information, and during playing mode, the user has to reach a random choice of buildings/monuments according to their age, from the most historic to the most recently built (Figure 9).

A point system was developed in order to reward faster players (a global leaderboard for each level of difficulty was implemented), while there is a time limit in order to finish each level. The game has three levels of difficulty, while each user has to “age-order” 5 to 10 buildings (the range has to do with the difficulty level) out of a set of 18 historical buildings/monuments, existing and non-existing nowadays (Figure 10). The game leads users to gain two distinct skills: (a) Active knowledge of the architecture of Nafplio in 19th century in order to be capable of finding the requested buildings/monuments and (b) historical knowledge about when these historic spots where built.

As far as the first skill is concerned, someone who has already visited Nafplio will surprisingly discover that the city has not changed dramatically, in terms of the city’s architectural structures, from the 19th century to nowadays. A game player that will visit Nafplio in the future will find out that the game has helped him to orient himself within the city.

The historical information about the age of the buildings/monuments was chosen because even though all the presented places existed in the 19th century, many of them were created in previous centuries. This can reveal the Frankish, Venetian, and Turkish influences in the city, and the user is gradually led to discover the multi-cultural environment of the city via a 3D game.

4. Conclusions

Taking into account that back in 2000, 3D visualization of cultural content was just a tool to digitally replace physical (e.g., damaged or missing) artifacts [31], it is notable that a wide range of usages is now established. From research and educational orientation to entertainment and business purposes [32], 3D models can visualize historic artifacts but, moreover, can “time-travel” users to historical places/cities/buildings. More significant is that 3D reconstructions and Virtual Reality technologies can transfer users to a specific historical period delivering a feeling of how daily life was in various aspects, from walking to the city or visiting a temple, to more complex tasks, such as politics and war.

Such a procedure demands cooperation between researchers and professionals from different fields of knowledge. From archaeologists/historians and architects to computer scientists and game developers, an approach with a high degree of interdisciplinarity is needed. The motives for establishing such a cooperation vary according to researcher/professional points of view, from merely business oriented (e.g., creating a successful, commercial, historical game) to purely academically oriented. 3D reconstructions are helpful for conservation tasks in historic cities/buildings and can facilitate cultural artifacts management. Moreover, a virtual reconstruction of no-longer existing artifacts/buildings/cities is a more challenging task because using descriptions of historical sources, widely accepted typologies, and “interpreting” how a 3D reconstruction would look like, might be difficult.

An even closer cooperation is needed on “how” the final result will be delivered to the public. Gaming technologies and gamification techniques are valuable tools that reach large audiences, specifically youths which tend to be pioneers in using digital applications. Serious games, like the educational/quiz version of “Nafplio in the 19th Century”, fit better in delivering richer and more accurate historical information. Moreover, these kind of games offer a large range of freedom related to: (a) Free choice of learning about a place, (b) flexible time-management, (c) autonomy in the learning process, and (d) degrees of freedom on how intense the cooperation or the competition with other users will be [33]. Skills development is another important aspect of serious games giving emphasis on problem-solving techniques, knowledge sharing tools, systematic thinking, and methodologies for achieving knowledge tasks. On the other hand, serious games cannot reach as many audiences as entertainment games, because a large degree of “fun” is sacrificed in favor of educational goals.

Both types of games, serious games and entertainment games, following their scope and the chosen gameplay can “educate” users in areas [34] such as: Social behavior to other players, team management and engagement, emotional handling, following rules, competing with rules, developing creativity in problem-solving, accepting failure as part of the learning/living procedure, self-confidence about abilities and knowledge, etc. These benefits should not be underestimated even when talking about entertainment games, while the historical or cultural angle is somehow less important for users, the “fun” experience is a primary goal. From a cultural perspective, even in these cases, it is valuable to “educate” users in appreciating historical accuracy and to popularize cultural heritage.

Recognizing these aspects can help the cultural sector to successfully face the need of implementing Virtual Reality and Augmented Reality technologies in order to [20]:

• Reuse its exclusive digital content;
• Exploit its content for mass markets on a global level, such as gaming;
• Further enrich user experience.

The core idea lies on the ongoing procedure of digitalization of cultural content. This digitalization could lead to the creation of historical, virtual worlds (3D reconstructions such as the one presented in the current paper) or to enrich virtual worlds (both historical and fantastic) with cultural artifacts/buildings/cities. This procedure gives the opportunity to cultural organizations to use their digitized and documented cultural assets multiple times and for various tasks (e.g., research, educational, or gaming) to gain popularity, and to reach large audiences, especially between the ages of 12–25 years who are the majority of online gamers.

Virtual worlds and 3D reconstructions can be constantly enriched with more cultural content, with documentation on buildings/artifacts/events, and more depth. From a tour to a virtual city, users could gradually be led to investigate the interior of historical buildings or monuments. In a second level, users could experience traveling back in time, realizing how the cultural and natural environment has changed over the centuries (or years) by creating what Micoli et al. [16] determined as diachronic 3D reconstructions (4D virtual cities, where the fourth dimension is “time”). However, more revolutionary would be the implementation of intangible cultural heritage in virtual cities. The “soul” of a city consists of legends, citizens’ stories, traditional rituals, and folklore events [35]. Creating interactive interfaces to buildings and interiors, as well as the development of dynamic visual media, would lead to further enrich user experience, while this could be a motive for revising already used 3D reconstructions in order to see “what is new”.

Following current trends in the cultural sector and future technological perspectives, authors claim that 3D reconstructions of historical places can be an important element of a cultural repository where digital content from historical and cultural sources will be stored (by cultural, educational, and research institutions), evaluated for their historical accuracy and cultural quality (from professionals coming from creative industries), used (for their initial purpose) and re-used (for supplementary purposes) until reaching large audiences in various forms.