Learning History Using Virtual and Augmented Reality

Abstract

Master lectures of history are usually quite boring for the students, and to keep their attention requires a great effort from teachers. Virtual and Augmented Reality have a clear potential in education and can solve this problem. Serious games that use immersive technologies allow students to visit and interact with environments dated in different ages. Taking this in mind, this article presents a playful virtual reality experience set in Ancient Rome that allows the user to learn concepts from that age. The virtual experience reproduces as accurately as possible the different buildings and civil constructions of the time, making it possible for the player to create Roman cities in a simple way. Once built, the user can visit them, accessing the buildings and being able to interact with the objects and characters that appear. Moreover, in order to learn more information about every building, users can visualize them using Augmented Reality using marker-based techniques. Different information has been included related to every building, such as their main uses, characteristics, or even some images that represent them. In order to evaluate the effectiveness of the developed experience, several experiments have been carried out, taking as sample Secondary School students. Initially, the game’s quality and playability has been evaluated and, subsequently, the motivation of the virtual learning experience in history. The results obtained support on the one hand its gameplay and attractiveness, and on the other, the student’s increased interest in studying history, as well as the greater fixation of different concepts treated in a playful experience.

1. Introduction

Motivation to study is an issue of great concern at the moment [1,2]. In this context, the use of games, as well as gamification, has been investigated as a motivational strategy in the school environment [3]. Works, such as those presented in [4,5,6], demonstrate that using games improves the level of motivation and commitment of the students, since they can combine entertainment with their own training activities. Serious games are defined as video games whose purpose is not only the user’s entertainment, although this does not imply that these kinds of games have to be boring [7]. Moreover, gamification has become a popular method of enriching information systems [8].

Due to the great popularity of this type of entertainment among the new generations, different fields other than leisure have taken advantage of this circumstance to apply this technology to their field and make their tasks more attractive. A very highlighted area has been education, and within this, the study of history and the cultural heritage recreation [9]. Educational video games are games developed specifically to help players understand certain concepts, develop new problem-solving skills, or learn new knowledge through various methods. However, many of the serious games developed in this field achieve their purpose: to raise awareness of different cultures and heritage, but they forget the recreational component, which is what drives most users to use these technologies.

Closely linked to the world of video games are visualization technologies, such as Virtual Reality (VR) and Augmented Reality (AR). Both of them aim to increase the acceptance and engagement of the gamified application by the end user.

Virtual Reality is a technology that makes it possible to visualize an imaginary environment, making the user feel as if he or she is in that environment [10]. VR systems are characterized by interaction and immersion [11]. The interaction between the person and the VR system is achieved through many sensory channels (sight, hearing, touch, and smell) [12] and the immersion is the degree to which the person feels enveloped within the system. VR provides a different experience of interaction and immersion from traditional computer monitors. Shelstad et al. demonstrated in [13] that VR enhances overall game user satisfaction, enjoyment, engrossment, creativity, sound, and graphics quality.

Augmented Reality combines real and virtual objects with a three-dimensional registry, making it possible in some cases to interact in real time [14]. This technique has presented itself as a good tool to increase engagement to the use of the developed applications [15]. Moreover, AR converts learning situations into contextual situations, which usually improves educational praxis [16]. The concept of “presence” in these games makes that users feel that the game is being developed in the place they are physically located, so the sense of ‘being there’ in both real and virtual worlds while playing the game has been shown to improve the learning environment [17]. Video games that include AR in the field of education have reached areas as different such as English, environmental sciences, and of course, history.

Digital devices, and especially mobile ones, are very popular with young people and most of the recent research have used them as platforms for game-based learning [18]. According to Virvou and Alepis [19], mobile learning includes the concept of anytime/ anywhere [20] and allows leisure moments to be used as an opportunity for learning. In addition, mobile or wireless devices are currently available to almost everyone, because of the great popularity of the smartphones in the society. This fact avoids increasing the costs of the experiments if they are used in any research. This has enabled the use of these devices in an educational environment through AR.

Regarding the application of these disruptive technologies in the education field, Kangas et al. [21] presented the “playful learning environment(s)” (PLE) concept. It denotes an innovative, technology-enriched play and learning environment whose components are located indoors as well as outdoors. In addition, this digital environment has neither rules nor prizes when users play, unlike a video game. Considering this concept, this article presents a VR playful experience aimed at deepening the knowledge of cultural heritage and thus make learning about this subject more enjoyable. It is set in the time of Ancient Rome. To feel immersed in that time period, users will be able to design and visit a typical city of that time. This will be visualized by using a Head-Mounted Display (HDM) that makes the experience more immersive. The virtual experience has been completed with the opportunity of viewing the buildings and characters of Ancient Rome using AR marker-based technology by means of a digital device. A series of markers will enable the visualization of the objects and some basic information related to each one of them will be shown. The students can access the necessary knowledge related to these assets of the game using their own mobile devices to visualize them in the same place they are located, mixing the virtual and the real environment.

Once the students have studied the information related to every building by using the AR application, the VR playful experience will offer the possibility to practice the acquired knowledge and thus reinforce the learning process. Specifically, the experience is based on the construction of cities in Ancient Rome. Different types of buildings have been modeled to accurately reproduce the buildings that existed at that time so that the user can choose the type of buildings to include in the city. The player builds the city from an aerial view by choosing the building to be added and placing it on the ground. Once the city has been designed, the video game also offers the possibility to walk around the city, switching to a first-person view mode. In this mode, the player can visit each building by entering them and appreciating every detail that makes them up, as well as interacting with other avatars walking around the city. To complete the immersion offered by the game, information panels have been included in the city that show an explanatory text about the use of these buildings in Ancient Rome. Both the VR playful experience and the AR application have been designed following the indications of a Human Centered Design (ISO 9241-210: 2019) [22], so different meetings and studies have been carried out to adapt the design to the characteristics of the final users.

In order to be able to evaluate both the cultural interest and the addiction level of the playful experience, user tests have been carried out. The designed tests were extracted from scientific articles that analyze these topics [23,24]. In terms of education, and with the aim of evaluating whether the use of the created playful virtual experience is an effective tool in the learning of Ancient Roman concepts, two different ways of teaching these concepts were compared, applying each of them to an independent sample of students. In one of the samples, the video game was used as a learning tool, while in the other, the content transmission process was carried out by the teacher, using traditional learning. In order to assess the effectiveness of the playful experience in motivating students to learn, a concept evaluation questionnaire was administered to the students. The analysis of the results obtained corroborates that students learn more quickly because they feel motivated.

The rest of the work is organized as follows. Section 2 briefly reviews important serious games and VR and AR applications on cultural heritage and education. Section 3 analyzes the design and the main contributions of the work. In Section 4, the art developed for the game is shown. The AR application will be described in Section 5. The functions and the playability of the VR video game is analyzed in Section 6. The educational component of the game is presented in Section 7. Next, the experiments carried out and the results obtained are discussed in Section 8. Finally, conclusions and future work are presented in Section 9.

2. Virtual and Augmented Reality Related to Cultural Heritage

The popularity of video games among young people make them an excellent tool to be used for educational purposes [25]. Current video game technologies offer the possibility of visualizing realistic scenarios in real time [26], an aspect of particular relevance when the aim is to reconstruct cultural heritage scenarios with historical accuracy. Mixing history and video games is something that has been done for many years [27,28,29,30]. This paper presents a playful experience quite similar to a serious game. This is because rules have been implemented about the construction of the city, which requires knowledge of some restrictions about the positioning of the building in the city area. So, some available video games that deal with this issue have been reviewed in this section.

2.1. Video Games Related to Ancient Rome

Analyzing video games where the main objective is to have fun, one of the most popular video game genres is the one based on a real-time medieval conquest strategy. The first release of Total War saga [31] is a representative video game of this genre, set in that historical period. It is a real-time strategy video game developed by Creative Assembly, which allows for a very realistic representation of that stage. Another video game series about Ancient Rome is the Imperivm saga Real Time Conquest [32] developed by the company Haemimont Games, focused on military conquest, where users have to protect their city and conquer their opponent’s city.

City-building video games are another popular genre [33,34,35]. The mentioned video game series Imperivm also included the Civitas Saga, composed of three video games where users have to build, govern, protect their city, satisfy the needs of the citizens, and so on. The first of the games, Imperium Civitas, was published in 2006 [36], a city-building and strategy game whose main goal is to build a large city in an orderly, non-random, and functional way. To do this, a very simple and intuitive menu system is used, consisting of a circumference formed by circles containing the available buildings.

The video game series Caesar [37] simulates the management and urban planning of Roman cities with extensive trade networks, which is protected by the possibility of creating an army. Moving forward in history, Anno 1404 [38] recreates that year. It is a real-time strategy and economic simulation city builder which allows the representation of cities and ports.

2.2. Video Games in History Education

There are numerous experiences in which video games have been used in classrooms, demonstrating its effectiveness in learning the subject of history [39,40,41]. With the educational component in mind, Wainwright [42] conducted a student-focused study based on the content and historical focus of popular video games in 2019. The author argues that games can represent history and increase the engagement of the players in these tasks. Zhang [43] analyzed the impact of using virtual simulation in teaching history. According to their experiences, student engagement and performance also increased after viewing a virtual simulation related to the historical period being studied. These virtual simulation can have different levels of immersion. The highest level is achieved if the Head Mounted Display (HDM) are used to visualize the 3D environments [43].

A project that took use of this visualization device is “History Maker VR”, which is focused on schools [44]. It is an immersive content creation tool that helps students learn American history in an entertaining and engaging way. Focusing the analysis on Ancient Rome, a project that also has used immersive visualization is “VR Travelling in the Roman Empire” [45]. It shows representative ancient Roman cultural architecture. The virtual reconstruction of ancient civilizations or emblematic buildings from different periods of history is the target of the project “Rome Reborn” [46]. It offers the possibility of taking a tour through ancient Rome with a richly detailed, accurate recreation of the urban development of Ancient Rome (from 1000 BC to 500 AD), with emphasis on well documented buildings and implementing credible but not necessarily true buildings for the rest of the city.

Regarding AR, it has been successfully applied as a tool to helps users learn about different types of content. Noh et al. [47] present a survey where different AR works and applications are analyzed, focusing on the virtual reconstruction of historical heritage. One example is the MAGIC and TROC system [48]. This is a mobile game in which the players discover archaeological objects while moving, which has been widely used in history education. Cardoso et al. [49] studied AR frameworks applied in the context of outdoor Roman mosaic ruins with the final aim of developing a multi-platform mobile AR application. Another research regarding this topic is the ARCHEOGUIDE project [50]. It is an AR-based system for personalized tours of cultural heritage sites. It was developed for learning about a cultural heritage site for the visitors.

Combining both visualization technologies, EON-XR [51] is presented. It allows user to navigate Ancient Rome in AR and VR in a fully immersive learning experience. In this experience, students can investigate the Colosseum, as well as others public and private buildings typical of Roman cities, to better understand the social and cultural contexts of these buildings.

3. The Playful Experience

Some educational material used in traditional lectures about Ancient Rome has been reviewed and it has been found that students have to study it by analyzing images of the buildings or the citizens, memorizing the texts that inform about life in that period. The main idea of this work was that students could analyze, explore, get information, and zoom in on every detail of the buildings and civil structures that were in the cities of Ancient Rome. By analyzing the existing video games reviewed in the previous section, it was found that none of them fit these requirements. The games take place at that time, but the gameplay did not support the objectives pursued. So, in order to improve the learning experience, a playful environment that uses Mixed Reality was initially raised. Different video game genres were analyzed to design the playing experience, and some opinions from students and teachers were also collected. The ones that were proposed were conquest strategy, endless runner, city building, shooter, and survival. The questions were answered by the same amount of boys and girls (15) that study these topics in their courses, and by the same numbers of teachers (15). All of them answered the questions about which genre they considered most appropriate for learning about life in Rome, and which genre they felt could provide the most information about the buildings and citizens. The city-building genre option was selected by 50% of them, followed by conquest strategy (30%) and survival (10%). The remaining 10% were distributed among the rest of the options. Following the recommendations of some relevant papers related to the technique “learn by doing” [52,53], this article presents a playful learning environment where two well-differentiated parts have been included: one that offers AR visualization by using mobile devices and another one that uses HMD to visualize a 3D environment.

On the one hand, students will be able to analyze, learn some information, and visit the most representative buildings of that period using AR based on AR-markers. Using a mobile device, a building or a citizen will be visualized when the camera detects the marker associated with it. Some information will also be displayed or heard when the object appears. On the other hand, a city-building application has been developed based on Ancient Rome. This educational application will make students feel like an architect of the cities of that period. They will be able to build a city following some rules previously informed, and later, visit the designed city by conducting a tour around the buildings or even by stepping inside them.

This playful experience has been specially designed to learn about the cities in that period of time, so it differs from the rest of video games previously analyzed, because it combines the possibilities of learning by checking the objects, building a city, and finally walking around it. Next, the developed art related to Ancient Rome is presented and then, more details about the AR and the VR applications are provided.

As for the hypothesis, it is considered that the use of these disruptive visualization technologies will motivate the students to learn how life was in that era and will fix the knowledge acquired.

4. Artistic Components of the Experience

The art is a very important part of the developed educational playful experience, because it is essential that all the details and characteristics of the models created are respected in order to be faithful to the period to be represented. Buildings, civil constructions, and even typical characters of the era have been modeled, with the aim of being as accurate as possible.

The emblematic buildings of the period have been modeled, faithfully reproducing every detail. These buildings have been grouped into four categories: houses, public buildings, basic infrastructures, and leisure buildings. These categories are visible to the user on the HUD (Head-Up Display) and all of them are accessible using the user-devices associated to the HMD used. As for the characters, a man and a woman have been modeled, to which clothing has been added to represent three types of avatars: legionaries, patricians, and citizens.

All designed 3D objects, both constructions and characters, have been modeled with the 3DS Max modeling tool. [54]. Next, more details about every modeled object is offered.

4.1. Buildings

The houses that appear in the serious game are the following:

• Domus: the houses of the families of the highest social class in the city, such as the rich merchants. Figure 1a shows a reproduction of these buildings, in which the vestibule and the atrium, opened in the center, can be distinguished;
• Villa: houses that belonged to the families of the patricians. Figure 1b depicts a typical construction of this type;
• Insulae: the dwellings of the lower classes, known as the citizens. Figure 1c illustrates such a construction.

The following modeled public buildings were included:

• Foro: the central area of the city, consisting of a rectangular square where the institutions of government, commerce, and religion were located. In this project, the forum was designed following the basic structure of the period (Figure 2a,b);
• Triumphal Arch: a monument to commemorate a military victory. Figure 2c shows the modeling made of this type, inspired by the arch of Constantine.

The leisure buildings that appear in the application are:

• Theatre: in this building, the actors of the time performed plays to entertain the people (see Figure 3a);
• Amphitheatre: where the great entertainment events of the time took place, including executions and gladiator fights, etc. The amphitheatre of this project (see Figure 3b) is inspired by the famous Colosseum in Rome;
• Roman Circus: leisure events, in particular chariot or horse races, also took place in this type of building. Figure 3c shows the modeling of this type of building.

Regarding the public infrastructures, the ones that have been represented are:

• Aqueducts: one of the most important constructions in the city, because they provided access to the water supply at many points within the city. In this case, Figure 4a shows the basic structure of the aqueduct. The user of the game, simply by dragging with the mouse, can repeat it, generating aqueducts of greater length by simply concatenating this structure, as shown in Figure 5a;
• Doors and walls: these constructions functioned as the city’s defense. The gates were attached to two large towers (see Figure 4b).As for the walls, like the aqueducts, the basic structure has simply been modeled, see Figure 4c, but the user can repeat it as many times as necessary to add walls surrounding the city, as shown in Figure 5b.

4.2. Characters

To represent a city from Ancient Rome, it is also necessary to depict its inhabitants. In this case, different kinds of inhabitants have been modeled to give the player the appearance of a living city as they stroll through the created city. Three profiles of the inhabitants who settled in the city have been chosen. These are detailed below.

• Legionnaires: members of the Roman legion, heavy infantrymen, consisting of men over the age of 15 (see Figure 6a);
• Patricians: the citizens of the highest class of the Roman Empire—the nobility. In this case, both male and female characters have been created (Figure 6b), each dressed in a tunic adapted to the way men and women of this social class dressed at the time;
• Citizens: this term was used to refer to the inhabitants of the middle and lower social classes, see Figure 6c.

5. The Augmented Reality Application

Augmented Reality allows users to visualize virtual worlds combined with real environments. This is usually used to increase the engagement of the users to the applications, so that the user can feel surrounded by the digital objects. Students will use the AR application to show and obtain information about the buildings (Figure 7a) and characters of Ancient Rome (Figure 7b) that were typical of that time. Some AR markers have been associated to the visualization of these objects, so when the students visualize them with the camera of the mobile device, the objects are shown on the AR marker. Every object will be visualized next to an optional text that explain some details about it, so the student can obtain this characteristic information that improves the learning process. This activity will increase the educational value of the game, as by informing the user about the details of life in Ancient Rome, the immersion in this era is intensified. In addition, taking into account the efficiency of AR in the field of education, this application promotes the motivation and engagement of students in obtaining this knowledge.

The application has been created for mobile devices with Android operating systems, developed with UNITY, version 2018.3.8f1, managing the AR with Vuforia plug-in, which includes camera management [55]. It has been developed following the design principles of learning games with AR for mobile aimed at children [56].

Other uses of this application have been addressed for the assessment of the acquired knowledge. Teachers proposed in the co-design process to give some random AR-markers to the students and ask them about the information that they had previously learned, in order to evaluate the learned concepts.

6. The Virtual Reality Experience

The virtual playful experience is composed of two clearly differentiated phases: the construction of the city and its subsequent visit. The application starts with an interface showing the different game possibilities (Figure 8):

• New city: this option allows the player to start designing a new city on empty terrain;
• Load a pre-designed city: this offers to load a city that already exists in the game. The player could directly switch to first person mode and start exploring the existing buildings and observe the different characters walking among them;
• Exit: this allows the player to exit the game.

The game programming was carried out with version 2018.3.8f1 of the Unity 3D game engine [57], scripting using the C Sharp language and the XR Interaction Toolkit (Virtual Reality SDKs). This first version of the game has been developed to run on a PC using the Oculus Rift VR headset connected to it, so that it can be played in a computer classroom.

6.1. City Design Stage

In the initial stage, the series of emblematic buildings and constructions of the period are offered to the player to build the city. This makes it possible to design a customized version of the city. The process is done from an aerial point of view, which makes it possible to get an overview of the city that is being designed. In addition to the buildings, the possibility to add walls and access gates is also offered, so that the player can surround the created city and form an enclosed area. The user can distribute buildings such as aqueducts, circuses, amphitheatres, and the forum, as well as buildings that the inhabitants of that historical period lived in.

Once the player starts the game, they are shown an isometric view of the world that only shows the terrain on which to place the various elements available in a head-up display. During the city-design process, the player can access a toolbar, where the different designed buildings can be selected, grouped into categories for a better selection. Figure 9a shows an example of the user interface for when the group of buildings that are intended for leisure is selected.

Once the building to be located has been selected from this interface, the position in which it is going to be placed will be indicated on the terrain with the motion touch controller. This position is restricted in order to maintain a certain distance from the existing buildings in the world, which have already been previously positioned. To this end, the surrounding ground has been associated with each one of the buildings created, which delimits that required for their placement (Figure 9b). This elements can be rotated to get the right orientation within the city.

Some constructions can be instanced in order to extend it, such as the aqueduct and the walls. Once one of this type of constructions is selected, the placement of a first instance is done by clicking on a button on the joystick. Then, its orientation is determined, and by simply moving the pointer in that direction, this instance is repeated until the button joystick is pressed again. This allows the user to create walls surrounding the city, or to create aqueducts of a given length, having only modeled a basic structure. Different historical restrictions have also been taken into account when building the city, for example, in a city there can only be one amphitheatre, one circus, etc. On the other hand, the inhabitants can have as many dwellings as the user wishes.

Once the city is built, the VR experience of viewing the city from above, available through the HDM, provides the user an aerial perspective of the city, so the player can see how individual buildings and monuments fit into a larger pattern of urban organization

6.2. City Visit Stage

Once the city has been created, the player can switch the game mode to a first-person mode. In this case, the camera changes its position and is placed at human height (170 cm from the ground), to allow the player to simulate a walk through the created city, so that it is possible to visit the created buildings and observe the environment, as shown in Figure 10. The interaction and movement through the created scene has been developed by programming the buttons on the joystick, which makes movement in the four main directions possible, as shown in Figure 11.

In this second phase, characters appear, walking around the city and dressed according to some of the social classes that existed in Ancient Rome, as shown in Figure 12a. Finally, in order to increase the historical potential of the game, scrolls with historical information can be accessed in each of the buildings and constructions. Modeled lecterns inserted in the buildings allow the user to interact with them, and after this action, the scrolls are displayed on the objects, the text informing the user of life in Roman times, as shown in Figure 12b.

6.3. Artificial Intelligence

The characters walking around the city give the appearance of a living city. They are controlled by Artificial Intelligence (AI), following a Path-Finding algorithm. For this reason, different routes have been programmed for the avatars that appear in the playful experience. The routes are determined by a sequence of points positioned next to the public buildings that make up the path to be followed, so that a cyclic graph will be created, which roams over the city. Figure 13 shows the graph generated to create random paths around the Forum.

The programmed sequence of the path finding algorithm is detailed next. Characters initially go out from the dwellings and walk in the first instance to the public buildings. Once the user has added a building of the dwelling type (domus, villa or insulae) to the city, if public buildings were previously located on the terrain, four avatars appear from each of these dwellings and randomly go to the existing public buildings. Once the avatar is close enough to access the network, it will first go to the closest point on the trajectory to the building it is in. Once it reaches this point, the next point where it will go is chosen at random, and so on. By introducing randomness into the decision to choose the next point, it ensures that each character chooses a different path as they move through the city. Regarding the type of character that appears from each house, this depends on the type of dwelling it is. From the Domus and Villas, patricians appear, and from the Insulae, citizens appear. Legionaries only appear when the walls and gates of the city are set up.

The path finding algorithm has been improved to ensure that when a battalion of legionnaires moving in five rows have to pass through a narrow street, the battalion is reorganized into two or three, so that their movement through the narrow streets is done properly.

7. Educational Component of the Game

The aim of the game is to apply the “learning by doing” concept to the study of history. According to [53], this is the process whereby people make sense of their experiences, especially those in which they actively engage in making things and exploring the world. This playful experience has been designed to allow the students the possibility of interacting with different buildings, objects, and even citizens of the daily life in Ancient Rome. So, two approaches are offered to them.

On one hand, an AR application is used to inform and prepare the student about the different types of buildings and characters that will appear in the construction and visit of the city. This activity allows the user to learn the details of life in Ancient Rome by analyzing every building, construction, or citizen individually.

On the other hand, a VR experience is offered to the user, allowing the player to design a city composed of all the buildings and constructions that he/she considers. This experience is addressed to the methodology of learning by doing. Once the city has been built, the student is immersed in an interactive environment in which he/she can move around as well as interact by collecting historical information.

8. Experiments and Results

In order to evaluate the proposed serious game, two types of tests have been carried out: one to assess the playability of the game and the other to assess the effectivity in the learning process.

8.1. Game Assessment

The first experiment was addressed to evaluate the playability of the playful experience. Twenty-five first-year Secondary School students aged approximately 12 years old took part in the experiment. They were called the technological group. Regarding gender, 12 were girls and the rest were boys. All of them confirmed that they have had previous experience with video games, although they were not regular players. Previously, a consent form was sent to the parents, who gave their approval for their children to participate in the experiment.

Before starting the experience, the students received a 60-minute-introduction lecture to the history and life in Ancient Rome. In this class, in addition to receiving a theoretical explanation of the historical moment in which the game takes place and how the cities of that time were like, the students used the AR app to become familiar with the different elements of the game. Next, the students watched a video of how to manage the playful experience presented. After this, the students played the game for about 30 min and then answered questions about certain concepts of the game.

When designing the test that evaluates the serious game, several existing works in the literature dealing with this topic were reviewed [58]. In their article, Pavapootanont and Prompoon [23] present metrics to evaluate the quality of a video game based on the concepts of comprehension, usability, operability, and attractiveness. Suryapranata et al. [24] also present in their work metrics based on [23] although oriented towards serious games. Their work is focused on assessing concept learning and motivation.

Following the methodology presented in [23,24],

Table 1. Questions related to assess the quality of the serious game presented.

Targets	Attributes	Questions
Content	Comprehension	Is the aim of the game easy to understand?
	Learning	Are the rules of the game easy to understand?
	Operability	Does the thread of history provide an insight into life at the period?
	Appealing	Is the gameplay appealing?
	Frustration	Is it easy to move around the world and access the storytelling?
Gameplay	Comprehension	Do you know how to play easily by watching the video tutorial?
	Learning	Is the information shown easy to understand?
	Operability	Are the rules clear?
	Appealing	Does the gameplay appeal to you?
	Frustration	Is the obtained result what you were looking for?

and

Table 2. Metrics related to assess the quality of the presented serious game.

Targets	Attributes	Metric
Content	Comprehension	Game understanding
	Learning	Clarity of rules
	Operability	Consistency
	Appealing	Attractiveness of interaction
	Frustration	Satisfaction with the game
Gameplay	Comprehension	Validity of previous demonstration
	Learning	Clarity of rules
	Operability	Clarity of messages
	Appealing	Attractiveness of interaction
	Frustration	Satisfaction with usability

show a series of questions that analyze the content and gameplay concepts of the video game. Each of these objectives will be assessed by analyzing the concepts of:

• Understanding: the ability of the software to enable users to understand how the game works;
• Learning: the game capacity designed for users to learn how to manage gameplay;
• Operability: the ability of the game to allow users to easily operate and control the game;
• Appealing: the attractiveness of the game to users;
• Frustration: the ability to give users a sense of whether or not they have achieved their goal.

All these questions were answered by the students in the sample group, each of them rated on a scale between 1 and 5 following the Likert scale, where 1 indicates the option “Strongly disagree” and 5 corresponds to “Strongly agree”. The results obtained are shown in the

Table 3. Test results obtained by the technological group in assessing the playability experiment.

Targets	Questions	Mean	Standard Deviation
Content	C1	4.36	0.81
	C2	4.72	0.45
	C3	4.38	0.84
	C4	4.32	0.80
	C5	4.28	0.73
Gameplay	J1	4.48	0.77
	J2	4.40	0.70
	J3	4.68	0.47
	J4	4.48	0.71
	J5	4.08	0.81

and analyzed graphically in the Figure 14.

8.2. Assessment of Learning Improvement

The second experiment was designed to assess if the application helps to consolidate knowledge. In order to test the hypothesis that the use of the created video game facilitates the learning of concepts about Ancient Rome, two independent samples of first-year Secondary School students were compared. The first, consisting of the 25 students mentioned above (the technological group), acquired these concepts through the video game, and the second, consisting of another 25 students (the traditional group), 11 girls and 14 boys, received their lessons in the traditional way, i.e., the entire weight of the content transmission process fell on the teacher. This traditional group attended a 60-minute lecture to match the same amount of time the technology group had in its formation. Then, they had the possibility of studying this topic by their own for 30 min, using some books and images that illustrate Ancient Rome. The parents of this second sample of students were also given a consent form for their children to participate in the experiment, giving their approval.

At the end of the learning process, the aim was to find out whether there are significant differences in the knowledge acquired by the students in the two groups. Although the students had learned in different ways, it was necessary for them to be assessed by the same evaluation method in order to obtain the objective results that could corroborate the hypothesis put forward. To this end, a questionnaire was drawn up with 20 questions, in which 3 possible answers were proposed to each of them, and only 1 was correct. The questions were related to four main topics, trying to evaluate the knowledge about the different topics that had been studied. These are shown below:

• Buildings and monuments: identify buildings or monuments;
• Functionality of buildings and monuments: find out what a building or monument was used for;
• Architectural elements: identify a given architectural element;
• Characters: identify a character;
• Character-related activity: find out what a character did for living.

Once the students had answered the questionnaire, we proceeded to the evaluation phase of the questionnaire. The results obtained are reflected in the graph shown in Figure 15.

8.3. Discussion of Results

Regarding the first objective of establishing the quality and functionality of the game presented, the results show that the students in the technological group have been enthusiastic about the novelty of learning with a video game, as the results show that most of the questions have an average score very close to 5. Analyzing the results in terms of content, the game is easy to understand, with clear rules, with a story that makes it easy to learn about the history of Ancient Rome, and a gameplay that engages them. With regard to frustration, although the data tends to be close to 5, the average score is around 4. This indicates that some of the children found it a little difficult to move around in the world that was created.

As for the gameplay analysis, the data are quite similar, with the question regarding frustration also standing out in this case. The data on this question were more dispersed, although none of them scored below 3. Although it can be considered that they liked the result obtained, this lower average than the rest makes us consider an improvement for future work in relation to this aspect.

As for the second test that assessed the acquired knowledge, it is considered proven that the students in the technological group who obtained their knowledge through the video game obtained clearly better results than those who used traditional teaching. This can be seen by analysing the data in the graph in the Figure 15, since 10 of the 25 students in the technological group answered 20 questions correctly, which represents a value of 100% correct answers, and another 2 students obtained 19 correct answers, a value close to 100% correct answers. This would mean that almost half of the students in this group have assimilated most of the concepts they were taught. In the case of the traditional group, no student achieved this success rate. In this group, the maximum number of correct answers was 16, and only 3 of the 25 students achieved this. On the other hand, the minimum number of correct answers obtained by the students in the technological group was 7, while in the traditional group, one student fell below this value, only achieving 5 out of the 20 questions proposed.

9. Conclusions and Future Work

This article presents the use of a playful experience in order to learn history by using both AR and VR technology. Following the “learning by doing” principles, this technological experience makes the student manage and experiment with the objects and situations they are studying. Regarding our hypotheses, it was supposed that this application will consolidate the acquired knowledge. Initially, AR was used as a support tool for acquiring the necessary knowledge that would allow the students to learn and examine the different elements of the environment with which they were to interact. In addition, a VR environment was presented, visualized by the means of a HMD, aimed at deepening the knowledge of cultural heritage in an enjoyable way, offering the possibility of building a city of Ancient Rome, which could be visited once it had been built.

The game was evaluated with students aged from 12 to 13 years. Two experiments were developed. One measured the playability of the playful experience and the other one assessed if the concepts were more consolidated than if they were acquired in a masterclass. The first experiment resulted in good student acceptance levels. The results of the data collected during the experience test were very satisfactory. As expected, the game allowed the children to learn historical concepts of the cities of Ancient Rome and have fun. From the different analyses carried out, a series of conclusions were obtained regarding the use of the video game created as an educational resource in the classroom. The results indicate the students’ satisfaction with the use of the video game as a work tool. They found the experience extremely interesting, simple, and attractive. In short, the game and its playability were highly valued by the students.

In the case of the learning assessment, the educational experience carried out through the video game was the clear winner. It is possible that the students, motivated by the novelty of using a video game, paid more attention to the information they were learning, and were able to retain more knowledge, as their learning was active. On the other hand, the group that received traditional teaching was more demotivated as their learning was passive, and their performance was worse, and therefore the concepts learned were less.

As for future work, we propose to continue adding, with historical rigor, new constructions and interiors that will allow the contents offered by the game to be expanded. We also want to add new ways of learning, not only by using the information offered on the information stands, but also by including challenges that allow or not allow to advance in the game. In this way, the game itself will be evaluated, including gamification techniques that motivate the students to continue playing.

Another line of future work is to extend the game to allow interaction with other users, and to make some competitions possible. Finally, we are planning to carry out an experimental study involving more participants to collect more data and perform more investigations.