Educational Seismology through an Immersive Virtual Reality Game: Design, Development and Pilot Evaluation of User Experience

Abstract

Virtual Reality (VR), especially in its immersive form, is a promising technology employed to support education and training in various fields. VR offers unique opportunities to experience situations and environments that are otherwise inaccessible or risky. Educational Seismology aims to inform and educate the public on earthquakes; to this end, the use of VR is investigated as an attractive solution. VRQuake is an immersive VR application designed and developed for Educational Seismology purposes. VRQuake is structured in five consecutive scenes and is organized as a game. It allows users to interact with virtual objects in real time and apply learned rules and good practices in reaction to an earthquake, thus providing a dynamic learning environment. A pilot evaluation of VRQuake is performed by volunteer university students who play the game and then answer a questionnaire with closed- and open-type questions referring mostly to the user experience. Analysis of the answers has shown positive results regarding usability, clarity and acceptance of the application. Answers are also encouraging as to the educational potential of VRQuake. Furthermore, qualitative analysis of open-type questions has contributed user suggestions and demands that point to interesting new directions for further improvement of user experience and learning outcomes.

1. Introduction

Educational environments rely on modern technologies to attract, engage, motivate and educate learners. Virtual Reality (VR) is a technology that has early received considerable attention in education research and innovation, thanks to certain features that lend themselves nicely to education and learning [1,2,3,4]. In particular, VR allows the learner to ‘experience’ situations or setups that would otherwise be inaccessible due to security, cost, size and other restrictions. Matter at micro- and nano-scales, outer space, as well as the earth underground constitute such examples [5,6,7]. Moreover, VR allows for the interaction of the learner with items or objects or individuals in the virtual world—a safe and low-cost yet realistic alternative (or otherwise, a smooth introduction) to the live experimentation necessary in order to get ‘hands-on experience’. Despite certain well-known shortcomings, such as the requirement to have and bear/hold special equipment (glasses, controllers or head-mounted displays), to install special software and to invest considerable resources and effort in the design and development phases, VR applications and especially those of the immersive type are gaining ground in education and professional training [8]—among other fields [9,10,11].

Earthquakes are natural phenomena studied by the sciences of geology and seismology [12]. They originate in the earth underground, in the lithosphere, while the energy released produces effects (disasters) on the surface, either on the ground or in the sea. Educational Seismology informs and educates the general public on earthquakes, aiming to raise awareness (knowledge on the natural phenomenon and its major mechanisms and effects), preparedness (measures and practices) and protection (correct actions or reactions) during and after the phenomenon—both at the individual and at the social scale [13,14]. Seismological research along with the education and training of the public and the prompt response from state authorities have considerable effectiveness in increasing resilience and in reducing the effects produced and the harm made by earthquakes. It is generally agreed that since seismic risk is beyond human control, scientific, social and state effort should be concentrated in rendering the ‘society plus built structures system’ resilient. Scientists therefore contribute their experience and expertise in registering, processing and assessing seismic activity and parameters to raise awareness and sensitivity of the public towards earthquakes. Among other lines of actions, scientists popularize seismological knowledge to make it accessible to a broader public. As earthquakes constitute a hazard and risk for all, relevant knowledge and protection measures are addressed to everyone. Educational Seismology turns its attention to educational institutions or training programs in workplaces as advantageous focal points for its activities. Educated and trained students and employees can serve as ‘multipliers’ to spread relevant knowledge, awareness and good practices learned at school or at work to families, friends, neighbors or other social groups. Beyond the individual level, human daily functions take place within narrow or broad social environments; individual decisions and actions have effects at the social level, consequently. This is why Educational Seismology has a strong social character directly relating to Citizen Seismology. The motivation behind the present study is to employ attractive technologies such as VR to serve the aims of Educational Seismology at the individual and the social level.

VR constitutes a promising technology to attain these goals, given the specific features of the earthquake phenomenon: earthquakes cannot be predicted and are not taking place ‘on demand’, while the true, physical experience of an earthquake may be hazardous or even fatal. Education and training of the public are therefore necessary before and not during or after the true phenomenon is experienced. This is why technologies that fall under the class of ‘simulation’ in the broad sense, like VR, are obvious candidates for the task.

Depending on the target devices, applications can be classified as (i) non-immersive, desktop or smartphone VR, where the user interacts through a 2D screen, (ii) immersive VR, where the user bears a head mounted display (HMD), or (iii) those suitable for both the above types of devices. The virtual spaces depicted may be internal [15,16,17,18,19,20] or external [21] or mixed: users initially find themselves in an internal space, but they have to exit to an external space following the earthquake [17].

Avatars are already being used in the design of applications [16,17,18], either as a virtual body model of the user whose movements are tracked [16] or as non-player characters (NPCs) used according to the scenario in order to represent other building occupants inside the VR environment [17,21], or to represent classmates and the class teacher [21].

A realistic user experience is sought through special equipment such as a chair-shaking system that provides the physical shaking of the user sitting on it in time alignment with the earthquake experienced by the user in the virtual environment [17].

Along these lines, the present research proposes the design and development of an immersive VR application, namely VRQuake, that can be used in the framework of Educational Seismology to deliver education and training in an attractive and engaging way. In comparison to existing research, the current approach is fully immersive in order to offer the user the desired sense of presence and immersion. It rolls out in an internal space (juvenile room) and does not use technical aids such as a shaking chair for the user. Novel elements of VRQuake are (a) the introduction of a game embedded in the virtual educational environment, (b) the support for two different ways for the user to move around in the VR environment, either by physically moving in the real world or by tele-porting, (c) the functionalities that allow users to interact with the virtual objects in real time and thus dynamically reshape the environment and (d) the opportunity for the user to practice learned rules and behaviors in a safe environment. Interactivity is established as a key feature with regard to the effectiveness of VR applications. Existing research, such as [22], finds that interactivity influences immersion and presence and eventually relates to user satisfaction and therefore suggests the development of interactive against passive VR environments. VR games constitute an attractive type of interactive VR applications. The advantages of introducing digital games in education have long been established, e.g., [2] among others. Research on the use of games in VR environments, however, assesses entertainment and educational advantages against certain disadvantages such as user discomfort, dizziness, etc. [23].

The following research questions are addressed by this research:

• Is VRQuake self-explicatory, understandable and clear?
• Is VRQuake entertaining?
• Do users see a potential for education in VRQuake?

In order to answer these research questions, a quasi-experiment is set up with volunteers (university students) who used VRQuake and took part in a pilot evaluation by completing a questionnaire on user experience, entertainment as well as on the education potential they see in VRQuake. Results are seen as satisfactory, while analysis of answers to the open questions calling for user suggestions for changes or improvements reveals interesting paths for further development in the future.

2. Literature Review

VR has been successfully employed for education and/or training in relation to various fields, Seismology being one of them. A spectrum of relevant VR applications has been designed and developed within recent research works, each supporting different aims and addressing different research questions.

Shu et al. [15] compare HMD versus desktop VR in terms of the user sense of presence, immersion and task-oriented self-efficacy regarding earthquake preparedness. Both HMD and desktop VR technologies are used by Zhang et al. [16] to evaluate the effectiveness of earthquake safety actions in indoors environments. Ooi et al. [21] develop a desktop VR-based drill for students to train in safe evacuation following a disaster (in that case, an earthquake in Japan). Feng et al. [17] focus on the internal human processes of decision-making during emergency situations; they employ immersive VR and verbal protocol analysis to investigate how building inhabitants make decisions during earthquakes and post-earthquake evacuation in a hospital setting. Decision-making is the aim of Rajabi et al. [20] who use standard Anxiety and Stress questionnaires (DASS and BAI tests) to evaluate the effect of education and preparedness on the (in)correct decision-making of residents under the stress of an earthquake. It is interesting that VR applications may be used either independently or jointly with a training/exercise program in the physical, real-world environment, such as a school classroom [20].

A broader view is taken by Caballero et al. in [18], where various disaster situations rather than only earthquakes are addressed; an American Sign Language scenario is implemented in desktop VR to reduce disaster risk for people with hearing loss. The same broad spectrum of disaster situations is addressed by Carrozzino et al. in [24], where the VR application developed recreates a range of operation conditions (day/night, presence of people, dangerous locations, etc.) in four different settings to train personnel to properly respond to them. A triple disaster scenario of fire, earthquake and typhoon is simulated by Balahadia and Savaman [25], who develop a VR application for training young children on how to respond to disasters. The VR spaces include several settings in a generic house building and employ non-player characters (NPC) in the scenarios implemented.

Disasters caused by earthquakes dominate the scenarios employed in certain other studies. Doungutha and Theppituck [26] use VR to raise preparedness for a tsunami caused by an earthquake; they also investigate user perceptions towards VR. An adventure game is employed for their purposes. A virtual flood not related to an earthquake is the situation addressed in Mol et al. [27], who use VR to train users against flood results. Lu et al. [19] address the emergency situation of post-earthquake fires and develop a highly realistic immersive VR environment and scenario, including indoor smoke-spreading simulation, to provide a reliable training for evacuation and rescue, again in a hospital setting. Park et al. [28] also address a fire rather than an earthquake scenario—in the subway, in that case. VR is used to train older people in correctly responding to the fire. The fire scenario is simulated in the study by Zhang et al. [29], who use VR to replicate fire evacuation and investigate the decision-changing behavior in crowd evacuation in a specific shopping mall floor. Six avatars are employed to support the scenario.

Games or game elements are introduced in the design and the scenarios employed in certain research works. Apart from [26], this approach is taken by De Fino et al. [30], who design a VR-based serious game to train users in correct responsive behaviors under a heat wave and an earthquake scenario. The settings are outdoor urban spaces where users are required to communicate and interact; an immersive VR and a desktop VR version are developed. Lovreglio et al. [31] also design and use VR-based serious games to investigate user behavior and to train users (building occupants) for earthquake emergency situations. The setting employed is again a hospital, while a vibrating platform is used for realism.

Another type of research is represented by He et al. [32], who employ VR to develop and evaluate the effectiveness of an Early Warning Message system for earthquakes in China. They employ immersive VR and use an indoor setting.

In comparison to these categories of existing research works, the present study addresses earthquakes rather than other disaster scenarios and employs purely immersive VR rather than desktop or mixed technologies. It has an education-oriented character and an age-independent design (it can be used by younger and older ages alike). The target group is the general population, in the sense that VRQuake does not refer to a specific target group (e.g., to people with hearing loss). Finally, it falls into the class of game-based VR as it employs gamification and point collection under two different learning scenarios. In contrast to existing research works, VRQuake trains users to prepare before the event, by emphasizing good practices in indoors spaces. Emergency situations such as decision-making during urgent evacuation are not included in the present design, because a calm setting is considered educationally advantageous—at least as an initial phase. Decision-making under stress would be better suited to a subsequent assessment phase of a complete earthquake education and training plan.

3. Materials and Methods

3.1. Design

VRQuake is designed on the basis of a scenario that implements safety rules and good practices on how to prepare and how to correctly respond to an earthquake. A juvenile room is selected as the indoors setting. It consists of a sequence of five (5) scenes:

• Kick-off scene,
• Acquaintance with the room setting and walk about,
• Experience a virtual earthquake while in the room,
• Repositioning room objects or furniture to safer places,
• Selection of items to prepack in a backpack, ready to take away when leaving home in emergency.

The first two scenes aim at smoothly introducing the user to (i) a 360° immersive VR world, (ii) the use of equipment (VR headset and handheld controllers) and (iii) the teleport mechanism for virtual transportation of the user. The third scene exposes the user to a virtual earthquake. The last two scenes are designed as digital games where users compete by collecting award points. To win these games, the user should apply the learned protection rules and good practices against an earthquake: reposition room objects or furniture to safer places or prepare an emergency backpack.

VRQuake is designed using the Unity 3D game engine [https://unity.com/] (accessed on 12 October 2023), to be used with the HTC Vive Pro headset and controllers [https://www.vive.com/us/product/vive-pro-full-kit/] (accessed on 12 October 2023). For safety purposes, during the first three scenes the user is physically seated on an armless wheel chair that may rotate 360° in order to avoid getting entangled with the VR cables. Teleport is used to change user position in the virtual space. In the last two scenes, the user is allowed to stand up and freely walk around the physical space while interacting with the virtual one. The empty physical space necessary for the user to freely walk around without bumping into obstacles is 3.5 m by 3.5 m minimum.

3.2. Development

3.2.1. Scene 1: Kick-off Scene

In Scene 1, the user finds him/herself in an unrestricted space. On either side are visible the logos of the project and project partners. A tall table with a conspicuous red cube on top is located in the center of the space. A teleport spot is located right in front of the table, denoted by a blue circle (Figure 1). The user is instructed to use the teleport mechanism to position him/herself in front of the table; then, by pressing and holding down the red cube, the user starts the application. The user teleports using the handheld controller and presses down the red cube using the virtual hand. The time duration of the scene is not limited; it terminates only when the red cube is pressed and held down for at least 1 s.

3.2.2. Scene 2: Acquaintance with the Room Setting and Walk about

Scene 2 aims at familiarizing the user with the indoor virtual setting. It is designed as a bedroom on the ground floor equipped with a French door, a door leading to the garden, a bed, a chest of drawers, a coat stand, a desk with a chair, a wall-mounted shelf bearing the TV set and a standalone bookcase (Figure 2). The user is physically seated on an armless, wheeled chair that rotates 360° and uses teleport to freely inspect the virtual room and garden. Four teleport locations are located in the room and one in the garden to allow the user to view different aspects of the room and closely observe furniture and objects. The time duration of Scene 2 is 60 s. The next scene is automatically loaded upon expiration. A countdown timer in seconds is shown in a pink frame on the room wall.

3.2.3. Scene 3: Experience of a Virtual Earthquake while in the Room

Scene 3 gives the user the experience of a virtual earthquake whose onset is programmed after a few seconds of calm. Furniture shakes, objects fall to the ground and certain objects break. Physics is exploited to allow lighter objects break while heavier objects shake less. At the onset of the earthquake, the coat stand is programmed to bump into the French door glass, break the glass and then fall on the ground (Figure 3). The broken glass is shown on the ground while falling is accompanied by the recognizable sound of breaking glass. During the earthquake, the soundscape is dominated by the sound of an earthquake, an alarm going off and the sound of objects falling down. Randomness is introduced in the way objects shake and move; in that sense, Scene 3 is unique in its every replay. Furthermore, to allow more breathing space for the user, the bed is deliberately omitted from the room setup in Scene 3. Otherwise, Scene 3 is an experience rather than an interactive scene. The user is physically seated and teleport is deactivated. The scene lasts 20 s, after which the next scene is automatically loaded. A time countdown in seconds is shown in a pink frame on the room wall.

3.2.4. Scene 4: Repositioning of Room Objects/Furniture to Safer Places (Game)

Scene 4 is a game the user plays to collect award points. The user is required to (a) locate objects/furniture placed so that they represent a hazard in case of an earthquake and (b) reposition each of them to a safer place. Both these goals give award points, 30 points each, 60 points in total. Collected award points are shown in a frame on the room wall, next to the time countdown frame.

The objects/furniture designed to be in the ‘wrong’ places in the room are (a) a decorative glass vase placed on a wall-mounted shelf over the bed; (b) an old-fashioned, heavy TV set placed on another high shelf opposite the bed; (c) a coat stand standing close to the glass door (Figure 4).

(a)

When the user locates an object in a wrong (unsafe) place, he/she should touch it with the virtual hand. A hit (correct object selection) is signaled by a yellow frame showing up around the object, a voice cheering ‘OK!’ and +10 award points in the points frame. Neither of these happens at a miss (wrong object selection).

(b)

To move the touched (yellow-framed) object to a safer place, the user ‘locks’ it to the virtual hand by pressing and holding the trigger on the back of the controller. An object thus locked may be freely moved around in the virtual room. The user selects the new, safer place and repositions the object by releasing the trigger on the controller. A hit (correct new place, as in Figure 5) is signaled by an ‘OK!’ voice message and +10 award points in the points frame. Neither happens at a miss (wrong new place); in that case, the user may lock the object and try another place.

Scene 4 lasts for 240 s, after this, the next scene is automatically loaded. The remaining time countdown is shown in a pink time frame on the room wall.

3.2.5. Scene 5: Packing an Emergency Backpack (Game)

Scene 5 is another game the user plays to collect award points. The goal is to select all necessary objects from among those available on a desk and pack a backpack in case the user has to leave home in an emergency, i.e., after an earthquake. Ten candidate items are laid on a desk as in Figure 6, namely a toilet paper roll, whistle, bottle of water, canned food, torch, notebook, ruler, teddy-bear, toy crocodile and glass vase; only five of them are indeed necessary and should be packed in case of emergency. A hit (correct, necessary item) gives 10 points, while a miss (unnecessary item) gives no points; 50 points is the maximum score for all five necessary items selected and packed. The score is shown in the points frame on the room wall.

The user must move freely in the physical space or teleport in order to get in front of the virtual desk. Then he/she should select one-by-one the necessary items and pack each item in the backpack. To place an item in the backpack, the user should touch it with the virtual hand (a yellow frame shows up around the touched item), ‘lock’ it using the control trigger as in Scene 4, move it into the backpack and release it. A hit (correct, necessary item) is signaled by a voice cheering ‘OK!’. The selected item disappears into the backpack and +10 award points are given in the points frame on the wall. Neither of these happen at a miss (unnecessary item); the selected item is tossed out of the backpack and falls randomly somewhere in the room, accompanied by a suitable ‘failure’ sound. Scene 5 lasts for 60 s and afterwards the VRQuake automatically concludes and exits.

4. Results

VRQuake has undergone a pilot evaluation by volunteer university students in the premises of one of the partners, namely, the Department of Electrical and Electronics Engineering, University of West Attica, Athens-Egaleo, Greece. Forty-two students who responded to the relevant call tried the VRQuake application in laboratory conditions, in a controlled and safe environment, after a briefing by the first author. Then, they completed an evaluation questionnaire of 15 questions, including closed-type (12) and open-type (3) questions. Questions mainly referred to the quality of the user experience with VRQuake, to the entertainment and education potential they see in it and to the improvements they may propose.

4.1. Quantitative Analysis

The 12 closed-type questions along with the results obtained from the answers provided are given in

Table 1. Answers to the closed-type questions of the VRQuake evaluation questionnaire.

Questions on the Overall User Experience with VRQuake (*)
1. My previous experience with VR is:	null	19 (45.2%)
	limited	13 (30.9%)
	medium	6 (14.2%)
	extended	4 (9.5%)
2. I play videogames/computer games:	never	6 (14.2%)
	scarcely	9 (21.4%)
	often	14 (33.3%)
	daily	13 (30.9%)
3. This particular VR application:	left me tired and/or dizzy	0 (0.0%)
	was marginally tolerable	2 (4.7%)
	did not leave me tired or dizzy at all	40 (95.2%)
4. Manipulations I had to carry out in the virtual environment:	gave me a lot of trouble	0 (0.0%)
	gave me some trouble	1 (2.3%)
	did not particularly trouble me	6 (14.2%)
	did not trouble me at all	35 (83.3%)
Questions on VRQuake Scene 3 (virtual earthquake) (*)
5. The virtual earthquake experience scene was self-explanatory and clear:	very much	30 (71.5%)
	moderately	12 (28.5%)
	not at all	0 (0%)
Questions on VRQuake Scene 4 (game of object repositioning to safer places) (*)
6. The game of object repositioning to safer places(please check up to 2 answers):	was fun	42 (100.0%)
	was educative; I did learn something new	31 (73.8%)
	was dull	0 (0.0%)
	I did not learn anything new	9 (21.4%)
7. Before this game, I did not have the chance to apply my knowledge on correct furniture/object location for the case of an earthquake:	Yes	37 (88.0%)
	No	6 (14.2%)
8. This game made me think that I should move certain furniture/objects in my home to safer places:	Yes	36 (85.7%)
	No	6 (14.2%)
Questions on VRQuake Scene 5 (game of packing an emergency backpack) (*)
9. The game of packing an emergency backpack(please check up to 2 answers):	was fun	40 (95.2%)
	was educative; I did learn something new	31 (73.8%)
	was dull	2 (4.7%)
	I did not learn anything new	11 (26.1%)
10. Before this game, I did not have the chance to apply my knowledge on packing an emergency backpack for the case of an earthquake:	Yes	38 (90.4%)
	No	4 (9.6%)
11. This game made me think that I should keep an emergency backpack at home, ready at all times:	Yes	37 (88.1%)
	No	5 (11.9%)
Question on the educational potential of VR (*)
12. For education and training in situations that are not accessible/safe/realistic in the real world, I consider VR technology as:	ideal	32 (76.1%)
	acceptable	10 (23.8%)
	questionable	0 (0.0%)
	unacceptable	0 (0.0%)

* Participants were instructed to mark one answer per question, except where more answers were explicitly allowed.

, in absolute numbers and percentages.

4.2. Qualitative Analysis

Open-type questions included in the evaluation questionnaire refer to Scene 3 (virtual earthquake), Scene 4 (move room objects into safer locations) and Scene 5 (preparation of an earthquake kit-backpack to take away). Answers are analyzed and the results are summarized below.

4.2.1. “For Scene 3 (Virtual Earthquake) to Become More Immersive, I Would Suggest…”

Most users found the virtual earthquake scene self-evident, realistic and immersive enough. In fact, 9 out of 42 users, or 21.5%, found it quite satisfactory and asked for no improvements. A few users reported that they sensed an urge to take cover under a desk or a bed during the earthquake, while three users proposed to change the tasks of the scene and include the option that the player takes cover under the desk when the earthquake occurs. They also asked for louder sound effects from (more) falling or breaking objects inside the room, such as shelves or even walls, or louder alarms sounding from outside the room. More pronounced effects were also proposed by a few users, such as the room lights to go on and off or debris to fall from the ceiling. Other users proposed to use a vibrating chair instead of a regular one (4 out of 42 or 9.5%) and/or to add vibration functionality to the hand-held controllers (6 out of 42 or 14%). A vibrating user view inside the HMD was also proposed, at the risk of user dizziness. Contradicting views were also encountered: one user asked for more vigorous object shaking while another user noted that shaking is too strong and should be attenuated.

4.2.2. “For Scene 4 (Object/Furniture Repositioning to Safer Places) to Become more Entertaining and/or Educative, I Would Suggest…”

Most users found this scene both educational and entertaining, while all but five users had interesting things to propose along these lines. Specifically, 13 out of 42 users, or 31%, asked for more objects/furniture pieces available to move to safer locations, while a few users asked for increased interaction between the user and room objects/furniture, e.g., provision that the user may sit on the bed or in a chair, and for more detailed physics in the movement of objects, e.g., any objects the users drop should break. A total of 8 out of 42 users, or 19%, asked for more detailed directions or explanations or tooltips to be offered either before or during the scene. One user proposed to allow more time for the initial familiarization of the user with the room, while another user proposed to add recapitulation screens after the completion of the scene, for educational purposes. A total of 5 out of 42 users, or 12%, asked for a virtual earthquake or aftershock to occur at the end of the scene. A few users proposed more commercial gaming mechanics, e.g., sound effects, breaking walls, scary atmosphere, score keeping for competing players or more verbose time countdown of the last few seconds.

4.2.3. “For Scene 5 (Emergency Backpack Preparation) to Become more Entertaining and/or Educative, I Would Suggest…”

Most users found this scene both educational and entertaining, while all but three users had interesting things to propose. Specifically, 9 out of 42 users, or 21.5%, suggested increasing the quantity and variety of the useful and non-useful items, in order to render the game more demanding. Four users asked for a waste basket to reject the non-useful items, while five users suggested that useful items should be spread around the room instead of being placed on the table, so that the user should move or teleport to search and retrieve them. More detailed graphics and improved physics for object representation and movement were also mentioned by 2–3 users. One user proposes to introduce more sound effects to signal hits (successful object selections) or game over, while another user proposed a more complex scenario that will allow users to fully realize the need for an earthquake kit.

5. Discussion

The results of the pilot evaluation of VRQuake in its current form are encouraging as to the potential of this application to serve the purposes of Educational Seismology. Despite the fact that the majority of participants (undergraduate university students) are computer game/videogame players (Question 2), their previous experience with VR is limited: 45% has had no previous VR experience, while only 10% state that they have extensively used VR before (Question 1). They felt no discomfort from the use of the HMD during the duration of VRQuake (Question 3) and stated that they found the use of VR equipment easy and straightforward (Question 4). They found that the virtual earthquake experience scene was very much (70%) or moderately (30%) self-explanatory and clear (Question 5). In the respective qualitative question (Section 4.2.1), all users stated they found the virtual earthquake sufficiently realistic; however, the majority had interesting suggestions for its improvement. The frequent reference to shaking chairs and similar advanced technology equipment reveals the levels of technological literacy and acceptance by the students. It is clear that more pronounced effects will render the user experience more realistic and increase his/her immersion and sense of presence.

The object/furniture repositioning game (Scene 4) was unanimously (100%) enjoyed by the participants, while over 70% stated that they learned something new (Question 6). For 88% of them, this was their first chance to apply knowledge relevant to the safe arrangement of room furniture/objects (Question 7) and 85.7% of them were led by the game to consider rearranging objects/furniture in their homes (Question 8).

The preparation of an emergency backpack game (Scene 5) was also enjoyed by 95% of the participants and only 2 of the 42 found it dull. Over 70% of the participants state that they learned something new, while 11 participants feel they did not (Question 9). For 90% of them, this was their first chance to apply relevant knowledge (Question 10) and 88% of them were led by the game to think about keeping an emergency backpack at home ready at all times (Question 11).

The above findings constitute a strong indication as to the effectiveness and suitability of VR for the purposes of Educational Seismology. Indeed, in their answers to Question 12, participants verify this overall picture obtained by the first 11 questions: 76% consider VR as an ideal tool and 24% as an acceptable tool for education and training. These results are in line with existing research such as [4], where VR is seen as a technology that advances learning, [2] where VR-based instruction is found to be quite effective, [20] where VR is characterized as a successful educational tool, [18] where desktop VR is considered as a valid alternative tool for the development of competencies and [15] where VR is characterized as a “suitable method for practical training courses of hazard and disaster education” while immersive VR “can provide a real, low-cost, highly mobile, and risk-free disaster prevention and preparedness.” On the other hand, a formal evaluation of the learning outcomes achieved through VRQuake is necessary before its educational merits are established.

In the respective open-type question for Scenes 4 and 5 (Section 4.2.2 and Section 4.2.3), users proposed several improvements such as more richly furnished and decorated environments, more pronounced/louder effects as well as physical vibration/shaking. They even proposed more elaborate and comprehensive educational scenarios, in order for the user-learners to fully grasp the meaning of their choices and live the consequences of their decisions. Participants propose, for example, that the game in Scene 4 continues with a second earthquake that occurs in the rearranged, safe room interior, and the damages incurred help the user fully realize the quality of his/her choices. Another suggestion about Scene 5 proposed to extend the current scenario to a situation where the user does leave home urgently and does come to need the items packed in the backpack, again in order to fully realize the quality of his/her choices.

Taken together into consideration, these user suggestions imply the following:

(a)

Along the line of experience: the need for more realistic virtual worlds that would increase immersion and the sense of presence. This need for increased fidelity of the virtual world to the physical one as well as for increased interactivity is reported in existing analogous studies such as [17,19]. Furthermore, the connection between immersion, the sense of presence and VR-facilitated learning is not superficial. Existing research has indicated that immersion, among other factors, is a strong predictor for presence, while presence is a key factor for immersive VR-facilitated learning [33]. (Tele-)presence is conversely indicated as a predictor for immersion elsewhere, [22]. It is clear that these two concepts are closely related. Finally, as concluded in [15], the increased level of immersion offered by HMD-facilitated VR makes it easier to mentally engage the participants.

(b)

Along the line of education: the need for more elaborate, comprehensive educational scenarios that provide feedback to the users not only in the form of game scores but also in the form of having the user face the consequences of his/her choices. This is an interesting finding of the qualitative analysis performed in the present research that needs more attention.

Limitations

The limitations that are inevitably present in the current work come from conceptual as well as technical considerations. Under the former category fall the design decisions on the basic scenarios included, especially the game Scenes 4 and 5, which are deliberately located before the earthquake and placed in a calm indoor setting that allows users to concentrate on practicing the learned content. Stress is introduced at the minimum, in Scene 3, through a mild virtual earthquake whose effects are conspicuous but not scary (e.g., no collapsing walls or buildings, etc.) A different design would be needed to simulate a stronger, more disastrous earthquake.

Regarding the educational aspect, VRQuake offers a space where users apply knowledge obtained previously/elsewhere; Scene 3 offers experiential learning and learning by observation, while Scenes 4 and 5 are essentially drill-and-practice spaces. In general, immersive VR applications are designed for a limited time duration, to prevent user discomfort. VRQuake ‘invests’ this limited time in experience, drill and practice rather than lecture or presentation of rules and good practices that would confine the user to a passive role.

The choice for immersive rather than desktop VR introduces a major technical limitation: it is best suited to individual rather than group viewing, while the need for specialized equipment (HMD and controllers) is another practical issue that counterbalances the desired sense of immersion. A desktop companion application might therefore be meaningful for larger audiences like school classes or workplaces. Another option for educational intervention organizers, e.g., class teachers, is to have an individual use the application with HMD and the rest of the team attend through computer screens in desktop mode. Team competition may also be organized in the gaming parts (Scenes 4 and 5) if large groups are organized into teams.

Another aspect of the same issue is the increased sense of realism that could have been created by the use of real-world photos of spaces incorporated in the design of the virtual spaces, instead of using computer graphics. The later choice is made in VRQuake, however, and the Unity 3D game engine is employed as a consequence, because the level of realism offered by computer graphics is deemed adequate for an educational application that does not aim for a fully realistic experience. The development of artistic/cultural experience virtual environments where realism is required at the highest level are indeed feasible through current modeling, animation, simulation and rendering tools, such as the Unreal engine and 4-D cinema, yet at a prohibitive computational cost, as verified in [24], among others.

6. Conclusions–Further Research

The design, development and pilot evaluation of a novel immersive VR application for Educational Seismology is presented in this paper. Earthquakes belong to a spectrum of natural disasters that call for the education and training of the public. Scientists, states and institutions already exploit modern and attractive technologies such as VR to raise public awareness, to teach good practices and protection measures and to train the public to exhibit the proper behavior during and after disastrous events or phenomena. The suitability and acceptance of immersive VR to these ends, reported in existing similar studies [15,17], are verified in the present research through an initial pilot use and evaluation of user experience. Certainly, a more comprehensive evaluation remains to be organized and carried out, both as to the quality of the user experience and as to the educational potential of this application, in order to obtain reliable results on its merits and to guide next versions of VRQuake development.

An interesting finding of the present research, however, is that in addition to the need for an increased sense of presence and immersion, the pilot evaluation has revealed the need for more elaborate and more comprehensive educational scenarios that will offer meaningful feedback other than game scores, and thus allow the users to fully benefit from the educational aspects of such applications. The implication of this finding for future research design is the need to incorporate education and pedagogy specialists in the already interdisciplinary teams that design and develop disaster-resilience oriented applications like VRQuake. A similar suggestion is made in [4], where the need to ‘include the instructor’ in the VR application development team is stressed. Indeed, given the complex character of relevant projects, the educational and pedagogical aspect is often overlooked in favor of scientific accuracy and technical soundness; and yet, it keeps emerging as a critical factor for the message conveyed by these applications to hit its target. In [1], authors have early and pertinently called for a ‘sound instructional design and pedagogy’ that should prevail over the mere novelty of the Technology. Along with this line of thought, our future research design aims to improve VRQuake both scientifically and technically, to enhance user experience, as well as educationally and pedagogically, to enhance its learning aspects. A cross-ages design is the challenge posed by both these aims.