Next Article in Journal
The Potential for Future Shifts in Tree Species Distribution Provided by Dispersal and Ecological Niches: A Comparison between Beech and Oak in Europe
Next Article in Special Issue
Exploring Hong Kong Youth Culture via a Virtual Reality Tour
Previous Article in Journal
Spatial Differentiation of Non-Grain Production on Cultivated Land and Its Driving Factors in Coastal China
Previous Article in Special Issue
AR Learning Environment Integrated with EIA Inquiry Model: Enhancing Scientific Literacy and Reducing Cognitive Load of Students
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Design, Implementation, and Evaluation of an Immersive Virtual Reality-Based Educational Game for Learning Topology Relations at Schools: A Case Study

by
Jalal Safari Bazargani
1,†,
Abolghasem Sadeghi-Niaraki
1,2,† and
Soo-Mi Choi
2,*
1
Geoinformation Technology Center of Excellence, Faculty of Geodesy & Geomatics Engineering, K.N. Toosi University of Technology, Tehran 19697, Iran
2
Department of Computer Science and Engineering and Convergence Engineering for Intelligent Drone, Sejong University, Seoul 143-747, Korea
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Sustainability 2021, 13(23), 13066; https://doi.org/10.3390/su132313066
Submission received: 12 October 2021 / Revised: 19 November 2021 / Accepted: 23 November 2021 / Published: 25 November 2021
(This article belongs to the Special Issue Augmented Reality and Virtual Reality-supported Sustainable Education)

Abstract

:
Education has always been modified by employing different technologies to enhance the knowledge acquisition and performance of students. Virtual Reality (VR) along with the Game Industry is among those evolving technologies for educational applications. This study aimed to design, implement, and evaluate an immersive VR-based Educational Game (IVREG) for learning topology relations. Topology relations are one of the fundamental topics which exist in Geospatial Information Science (GIS); due to the great capabilities offered by GIS, learning these basic topis is of great importance. A total of thirty-seven male middle-school students participated in this study. A total of four questionnaires were designed to evaluate the suitability of the proposed learning environment and its components at schools, particularly for learning geospatial topics. In conclusion, students found the IVREG useful and effective in classrooms. Additionally, the results showed that the components, namely the integrated pedagogical approach, gamification, and VR technology, were all suitable for being used at schools. On a final note, however, the study indicated that the immersion aspect of such learning environments should be enhanced in future studies.

1. Introduction

Emerging technologies have the potential to aid educational reform, leading to improvements in teaching and learning [1]. For centuries, the educational system has been changing. It has always evolved to the available technology and altered to meet the demands of the pupils [2]. Virtual reality (VR), one of the emerging technologies, is currently being used widely in a variety of education-related applications such as science [3] and art [4]. VR technology offers the user a safe 3D educational environment in which they may build their knowledge through trial and error approaches that mimic real-life circumstances and occurrences [5]. In addition to VR, digital games are said to be intrinsically motivating because they keep players interested while requiring them to devote time and energy to gameplay for no other reason than to play the game [6]. With the help of input interactions and feedback, digital games provide highly engaging learning environments due to their potentialities to create amazing stories through challenging and immersive environments.
Students are frequently ignorant of geospatial-related technologies, such as the capabilities of the Geospatial Information System (GIS), and the ways in which these technologies affect their lives. From navigating an unfamiliar neighborhood to tracking down the world’s most wanted terrorist, geospatial technology touches nearly every area of life [7]. Therefore, the significance of geospatial topics is by no means negligible. Besides, with the capabilities offered by educational games and VR, the idea of implementing an IVREG for learning such topics, especially for K-12 students, can be investigated. A case study of Iranian middle-school students is discussed in this paper since teaching geospatial topics specifically with the help of technologies has not been noticed among Iranian schools. This paper aims to 1) design and implement an environment for learning topology relations by integrating gamification, VR, Deep Learning, and pedagogical approaches, and 2) evaluate the performance of the proposed learning environment and its elements. To be specific, the usability and effectiveness of the presented solution for learning topology relations among middle-school students are investigated. Moreover, students’ perspectives on the usability of i) pedagogical approaches in VRLEs, and ii) gamification and VR at schools, were obtained to assess the impact of such elements on K-12 education, in particular, the geospatial-related lessons.
The rest of the paper is organized as follows: Section 2 covers the literature review. The methodology is provided in Section 3. In Section 4, the results of the pre and post-tests are being investigated and a discussion is provided. Finally, Section 5 presents a conclusion for the study.

2. Literature Review

2.1. VR Learning Environment (VRLE)

VR can be considered as a new learning platform that is catching the interest of academics and practitioners alike. It is created to make it easier for individuals to cope with information, and it has been effectively used to improve learning and task performance. In general, VR educational applications can be divided into two main categories based on visualization and interaction devices, namely: non-immersive, in which the user’s vision of the world is provided by a flat-screen computer acting as a "window," and immersive, in which the user is completely immersed in a virtual world by wearing glasses with two small lenses or a head-mounted headset [8,9,10]. The authors of [11,12] highlight that students’ interest and involvement in learning are increased when immersive VR is used. The VR version of learning environments can be obtained with the help of Computer Graphics technology in generating and showing a wide range of digital images. To be specific, VRLE is defined as a computer-simulated, three-dimensional environment in which a student may interact with the environment and experience telepresence [13]. The VR learning environment is considered intuitive and immersive, because it is a shared information context with unique interactions that can be customized for different pedagogical approaches such as inquiry-based learning [14] and collaboration [15].

2.2. VR Game-Based LE

Young people are becoming increasingly interested in digital media, which, among those, video games could be prioritized [16,17,18]. Apart from entertainment, many industries such as marketing are employing digital games directly or indirectly. In terms of education, digital gaming environments provide a suitable context for developing different skills in learners and educating them about a wide range of topics. As it is asserted in [19,20], this type of learning environment has produced noticeable impacts on learning procedures. According to the authors of [21,22], game-based learning environments provide a pedagogical benefit by encouraging motivation, interaction, and difficulty.
On the other hand, VR offers immersive gaming experiences, which are more enjoyable than traditional alternatives. In addition, when players play digital games in VR settings, they feel a high level of immersion and presence, which has been linked to memories that are "difficult to forget" [23]. Therefore, using this type of gamification in a learning environment could lead to offering more advantages, including its ability to highly engage users [24]. The combination of a VRLE and a VR game results in a game-based VRLE. In comparison to VRLE, game-based VRLE demonstrates its benefits by offering challenges and fun, both of which clearly increase learning motivation [25,26]. Learners can witness both realistic and fanciful events being played out on the screen in these environments. This, in turn, results in providing the opportunity to acquire patterns and rules for navigating the digital world successfully [27]. Exploring and problem-solving are among the skills that must be targeted to be strengthened, so as to navigate games and complete the tasks inside them. Because VR and digital game-based learning have gained appeal as educational methods for promoting knowledge and skill acquisition, it is a great opportunity to integrate gamification into VRLE to boost knowledge acquisition.

2.3. Current State of the Research Field

The authors of [28,29] indicate that VR is an effective technique for developing spatial abilities. Furthermore, the authors of [30] state that geography is prioritized in the K-12 curriculum in most parts of the world, which is a testimony to the significance of learning these topics from early school ages. Therefore, one of the topics that can be educated through VR and digital games can be geospatial topics. Apart from considering geospatial-related topics solely as learning materials, learning these topics can help students with benefiting from the highly valuable decision-making toolkits offered by GISs. The author of [31] highlights that 7th to 12th-grade students who learn to use geospatial technology will have an advantage in pursuing professions in this rapidly growing sector. A widely used topic in GIS, especially for data management, integrity, and analyzing spatial relationships, is topology. It deals with spatial and structural properties of geometric objects, regardless of their extension, type, or geometric form. Simply put, by introducing topological primitives such as border, interior, and exterior, topology avoids dealing with geometry [32]. This, in turn, results in simplifying analysis functions. Topology relations give us a sense of the overall structure and relationships between spatial objects.
Although various literature has studied the adoption of different technologies such as VR in education, few papers have aimed at teaching geospatial subjects. The authors of [33,34] tried to explore the potentialities of VR in geo-education. The authors of [33] used Google Earth VR during a physical geography course to investigate geographical objects in 3D form; their findings confirmed that VR is highly adaptable of achieving teaching purposes. The authors of [35] evaluated the spatial skills of students by using a VR application that provided mental rotation tests. According to this paper, the usage of VRLE can aid in the development and improvement of students’ spatial skills. To summarize, papers in this research field have focused on general topics or employed pure VR. Simply put, topics such as gamification, Artificial Intelligence, and pedagogical approaches have not been recognized among papers in this research field. Additionally, specific geospatial subjects such as that in this study have not been noticed either.

3. Methodology

The whole study procedure was reviewed by the school officials. They were informed about the benefits of the study. In addition, they used the IVREG in order to make sure of its suitability for use by the students. Finally, all students signed the consent form before attending the study.

3.1. Purpose of the Study

The main learning objective of the game is to learn to identify different types of topology relations. Therefore, this paper aims to evaluate the implemented IVREG in terms of different criteria in learning topology relations. Different questionnaires were performed to assess the suitability of the IVREG. The detailed goals being targeted in this study are as follows:
  • Design and implement an IVREG benefiting from Deep Learning for learning geospatial topics;
  • Examine the performance of integrating inquiry-based learning and instruction through observation in IVREG;
  • Evaluate the usability of VR and Gamification in learning environments compared to traditional learning environments;
  • Examine the implemented IVREG;
  • Assess the IVREG potential as a useful tool for teaching geospatial topics in K–12 grades.
In summary, this paper aims to assess if this IVREG has a positive impact on students’ learning about topology.

3.2. Participants and Equipment

Participants. A total of Thirty-seven male students participated in this study. The students were aged 13 and from a middle-school located in Gilan Province, Iran. This group of students used the IVREG for learning topology relations and also took the pre-test and post-test questionnaires. At a first glance, the tasks involved in the game might seem easy for students at this age; however, it was found in our initial testing sessions that students younger than this age had noticeable difficulties understanding topology relations and using the VRLE. That could stem from the absence of sufficient geospatial courses in elementary schools, and fewer experiences with VR among children at that age, at least in the province where the study took place. Additionally, by studying middle-school students’ characteristics and motivations [36,37], the proposed solution was found suitable enough not only to enhance students’ motivation at this age but also to result in reliable findings.
Equipment. The game was implemented using a desktop workstation (CPU Intel i7 10700, RAM 16GB, GPU ASUS STRIX RTX 2060). The VR system used in this study was a headset from HTC (HTC VIVE Pro, 110° horizontal field of view, resolution 1440 × 1600 pixels per eye, refresh rate 90 Hz, integrated microphone, controller specs, and adjustable headphones) along with two base stations for tracking. In addition to the mentioned specifications, this device comes with a rubber guard to prevent the light from entering through the user’s nose and the headset. Moreover, there is a sizing dial at the back of the strap to adjust the headset on the user’s head. In conclusion, it appears to be a very comfortable device to wear, especially for extended sessions.

3.3. Materials

The materials used in this study can be categorized into three parts. The former consisted of two video clips and one image. These were supposed to provide learning materials about topology relations. It is worth mentioning that, due to a lack of enough teaching materials for topology relations, one of the videos was captured by authors explaining the topics in slide presentation formats.
The second group of materials consisted of questionnaires to evaluate the proposed learning environment. All the responses were designed based on a 5-point Likert Scale. The questionnaires included Learning Perspective Questionnaire (LPQ), Virtual Reality Perspective Questionnaire (VRPQ), Educational Game Perspective Questionnaire (EGPQ), and Playability Questionnaire (PlQ). The former one aimed to monitor how the learning perspective of students changed after using the system. Specifically, this questionnaire was designed to firstly monitor students’ motivation and secondly their perspective on inquiry-based learning and instruction through presentation. The second questionnaire, the VRPQ, aimed at assessing VR-based systems in terms of being effective in education. Similarly, EGPQ was designed to check students’ perspectives on the effectiveness of educational games. The essential features of VRLE have been the subject of several studies. In short, the most often acknowledged properties of VRLE in knowledge representation are natural interaction and simulated presence. Meanwhile, usability and playability, as well as related concepts such as likeability and enjoyment, have been discovered to have a major role in user experience. Learning in VRLE has also been found to be aided by motivation and other psychological variables. As a result, the PQ was produced to evaluate user experience and students’ satisfaction, see the Appendix A. This questionnaire was designed to check the playability of the IVREG with respect to four dimensions, including engagement, gameplay, learning motivation, and VR experience.
Finally, the software used in this study includes Unity3d and Unity XR toolkit for VR scenes. For interactions in VR, the XR Interaction Toolkit package was used, which is a high-level, component-based interaction system. Simply put, it provides a framework that makes 3D and UI interactions available from Unity input events. With the help of Synty Studios and Unity free assets, the learning environment was designed. A one-room house along with a front and a back yard located in the middle of mountains were designed to serve as the game environment, see Figure 1.
In order to evaluate the student’s answer, a Convolutional Neural Network has been developed for image classification using TensorFlow. To be specific, this model is responsible for classifying the student’s drawing in terms of topology relations. A dataset of 1141 images was created for this purpose, see Figure 2.
The image size of (224, 244) with three color channels for Red, Green, and Blue (RGB) was also defined. 80% of the images were considered for training, and 20% for validation. The RGB channel values are between 0 and 255. As this is not the best situation for a neural network, the RGB values were rescaled to a value in [0,1]. To avoid overfitting, some data augmentation approaches were used, such as random horizontal flip, random rotation, and random zoom to generate more additional training images. Dropout, which is one type of regularization, is another approach for reducing overfitting in the network being considered in this network. When Dropout is applied to a layer, it randomly removes several output units from the layer throughout the training phase (by setting the activation to zero). The model is made up of three convolution blocks, each having a max pool layer. An RELU activation function activates a fully connected layer with 128 units on top of it. Figure 3 indicates the training and validation accuracy, and loss over 15 epochs.
The trained model was then converted to an ONNX model in order to be used in Unity. The ONNX model was used with the help of the Barracuda Package in Unity. This package is a lightweight cross-platform neural network inference library for Unity.

3.4. Design and Implementation

According to the authors of [10], some essential considerations should be explored before commencing the design and development of a VR educational application. Of these considerations, one example would be the usage of such a VR resource that must benefit the teaching–learning process (by enhancing motivational activities that lead to more effective learning). Therefore, the design process of the present study can be categorized into two parts; namely, gaming design and educational design. These two types of design were employed in order to enhance the motivation of students in using such systems for educational purposes.
Gaming design. The author of [38] states that four basic components must be taken into account in order to develop a game. Those elements are story, mechanics, technology, and aesthetics. Although having a story in the game makes it more exciting, the proposed game did not need a specific story, such as being invaded by aliens. Having said that, the mission to escape the room can be somehow considered as the story. As the story component includes some events to form the story of the game, such as being trapped into a room and trying to escape from it, these events can be considered as forming the story section of the game. In addition, as the author of [39] suggests, the escape room is among those media games that can enhance the logic of thinking. Lastly, the interactions in VR can also lead to excitement. The game mechanics component is discussed in Section 3.5. The term technology not only refers to state-of-the-art technology, but also to any material or tool that can make the game work. The author of [40] states that interactivity is believed to be the most essential component of effective learning. As a result, in this game, VR technology is used to satisfy the required interaction and immersion in the learning environment. Finally, for the last component, which is aesthetics, different approaches have been taken into account. Of these, one is the colorization of the scene, which makes it more alive and suitable for targeting learning goals, see Figure 1. The way our brain operates and employs color to build pattern recognition, memory, and even absorbing new information, has an impact on your learning. Besides, color can help children discover and compare information more quickly by providing visual guidance. Therefore, the role of color in educational games is by no means negligible. Another component of aesthetics is audio. By adding different audio sources such as fireplace sound, pouring water sound, drawing sound, and triggering the remote button sound, etc., the aesthetic component of the game has been enhanced. Lastly, the Graphical User Interface, along with some pictorial information, has been added to the game for different purposes, see Figure 4. Mostly they are used as hints to guide the user through the game. Furthermore, some of the GUIs contain educational information to serve the purpose of instruction through presentation, see Figure 5.
Educational design. Entertainment and learning should be balanced in VR-based educational games; otherwise, VR might contribute to issues such as game addiction [41]. In order to create this balance, the authors tried to add daily activities such as painting to the proposed system; because incorporating such activities is one way to balance entertainment and learning aspects of an educational game [42]. Moving forward with pedagogical approaches, educators and researchers are increasingly realizing the importance of inquiry learning and problem-based learning (PBL) in presenting geographic facts, concepts, and skills to students of all ages [31]. In inquiry-based learning, the process of learning is guided by questions, problems, or challenges, which students are supposed to deal with in order to complete the learning task. According to the author of [30], inquiry-based learning, and especially PBL, has the ability to “support or encourage the development of geographic knowledge skills, and practices,”. Therefore, inquiry-based learning is one of the approaches that was employed in the game. All the challenges and problems presented while escaping the room can be categorized into this type of learning and are discussed in detail in the following section. In addition to inquiry-based learning, we wanted to educate students even after they escape the room. As per [43], people are able to attend to the learning process only by observation and not by performing tasks. As a result, the method called instruction through presentation was chosen to be employed in the IVREG. Knowledge acquisition at the remembering and understanding levels through educational material presentation has been studied and approved by different papers [44,45,46]. Therefore, different instances of topology relations have been displayed in the front yard, so as to ensure that the user remembers and understands the knowledge about different topology relations, see Figure 5.

3.5. Procedure (and the Gameplay Mechanic)

The overall procedure of the study is shown in Figure 6. In addition, the game mechanic component mentioned in Section 3.4 is also discussed here, see Figure 7.
Firstly, students were introduced to the terms being used in this study namely: inquiry-based learning, instruction through presentation, escape room, VR, and gamification. A short definition of each term was presented to the students to make them familiar not only with the study but also the terms in the questionnaires. In the second stage, students were asked to complete the pre-tests. These pre-tests consisted of three questionnaires, namely, LPQ, VRPQ, and GPQ. After that, with help of an operator, students put on the headset and started to play the game. At the beginning of the game, the student is introduced to the basic interactions provided by the game, such as grabbing an object, painting using a brush, and different types of locomotion systems, see Figure 8. Then, although the student can explore his surroundings in a limited way, there is no way to the front yard but through the house. As soon as the student enters the house, he is somehow trapped in the house. In this phase, the escape room gamification starts. He is supposed to explore the room in order to find different materials to be educated about topology relations. As mentioned earlier, two video clips can be played on the television and the tablet in the room. The last educating material to find, which is an image of all topology relations, is situated behind an art frame.
Next, the student can use the question desk, located in one of the corners of the room, to answer questions and find a way out of the room. There, the student is asked to draw shapes illustrating the topologic relation mentioned in a randomly generated question, see Figure 9a. Using a brush, the student can draw objects indicating the required relation, see Figure 9b. After submitting the answer, the drawn image is sent to a pre-trained convolutional neural network model to predict the relation. If the answer is correct, he will receive a hint for escaping the room. Otherwise, he must continue answering questions correctly in order to see the hint. The game does not end here. The provided hint does not reveal the location of the key to the door explicitly. In other words, the hint itself is indicating a topology relation, see Figure 9c. Therefore, the student is supposed to find objects indicating that specific relation, in order to find the key to open the door and escape the room. After this stage, the student can play and interact with objects in the front yard. However, the learning process continues. By using instruction through presentation, the topology relation of some objects in the front yard is explained for the student in order to somehow remind him of different types of relations. The factor for primarily evaluating the likeability of the IVREG is the length of time of playing. As a result, the student is not asked to exit the game immediately after he enters the front yard. Finally, after a 10-minute rest time, the student is asked to answer the three pre-tests along with another questionnaire named the Playability Questionnaire (PQ). With the developments of technologies, children are exposed to technology from early ages [47]. They normally find technology and games fascinating, and their immediate reaction towards using new technologies and devices might be considered as delight effects. This, in turn, might result in biased responses. In order to avoid that, students were asked to review their responses and feel free to make any changes one week after their first encounter with the game.

4. Results and Discussion

4.1. Learning Perspective

Table 1 indicates the questions available in LPQ. In addition, the results are shown in Figure 10 in the form of box plots, so that the interpretation of pre-test and post-test would be way easier. As shown in Figure 10, and the box plot for Q1, there was a rise in students’ interests in learning topology relations. In addition, students’ perspectives on the necessity and easiness of learning geospatial topics changed after playing the IVREG. Noticeably, the median for the Q3 box plot moved considerably, illustrating the swift change in students’ mindset about the easiness of learning such topics. To study students’ perspectives on whether they will notice the relationships between objects in their surroundings, question four was designed. The only changed element between the pre-test and post-test box plot for Q4 was the median. It indicates that they start to pay attention to their surrounding from now on, which was one of the ultimate goals of geospatial topics, in particular topology relations. Finally, in order to evaluate the pedagogical approaches being employed in the IVREG, questions five and six were taken into consideration. As it is illustrated in Figure 10, the box plots changed dramatically. The first and third quartiles along with the medians were moved to higher scores. Fortunately, the effectiveness of the employed pedagogical strategies in VR games gained high attention, indicating that students found this type of integration useful in educational applications.

4.2. VR Perspective

To evaluate VR effectiveness in classrooms, and particularly for geospatial topics, questions one and three were put in the questionnaire, see Table 2. By observing the Q1 and Q3 box plots in Figure 11, a change in all elements of a box plot such as quartiles, median, and interquartile range can be seen. Most of the students were on the same page in considering VR as being useful in schools, and, in this case, for teaching topology relations. For the Q2 box plots, assessing the effect of VR on students’ creativity, although the minimum and the first quartile did not change, the median moved dramatically to higher scores, which indicates that there was a unanimous rise in considering VR as an improving tool for students’ creativity. With regard to Q5’s corresponding box plot, students had the idea after the play that the learning part of the IVREG is also noticeable. That is to say, when students notice the learning part of the LE as well as its entertainment part, the idea asserting that education can be brought into VR games can be supported. However, according to Q4 boxplot, the presence of a teacher along with such IVREG is by no means negligible, meaning that still the presence of someone to guide students and even maybe prohibit them from ignoring the learning part of the game is needed. After playing the IVREG, students changed their perspective on the fact that playing VR games might be somehow embarrassing in front of other students. Similarly to LPQ, here we also checked the effectiveness of integrating VR and the mentioned pedagogical approaches in Q7. As expected, students assert these kinds of educating combinations. Finally, with all the positive attitudes towards using VR in educational domains, students had almost similar predictions on having more VR in the near future for educational purposes.

4.3. Educational Game Perspective

Table 3 indicates the questions included in the EGPQ, and Figure 12 displayed the reported results for the pre-test and post-test. By looking at Q1, Q2, and Q3 box plots, the effectiveness of educational games can be concluded. Students disagreed on the fact that educational games might have negative effects on their creativity. Q5 box plot can be a testimony to the conclusion made just above. Simply put, when students are motivated by educational games, we have a better chance to educate them by means of this type of game.

4.4. Playability

Table 4 shows the mean and standard deviation values of each dimension being aimed in PlQ. Learning motivation took the lead while VR experience had the least mean value. A mean value above four for learning motivation illustrates that the educational aspect of the IVREG successfully influenced the students’ idea towards the fact that the IVREG is effective in educational applications. However, 3.5 as the mean value for VR experience indicates that there is still a lot of work to be conducted in order to provide the students with the best VR experience. Moving forward with the other two dimensions, the gameplay met the students’ expectations to a high level and was able to make them satisfied. Finally, although a value just above 3.5 for engagement is considered a good score, similarly to VR experience, the engagement part of such systems is still not perfect.
Before discussing the whole results together, it should be considered that this study was a gender-specific one. Therefore, the interpreted results are true for male students. As mentioned earlier, children at this age show great interest in different technologies. Apart from that, different papers—mentioned in Section 2—have already found great potential in using VR and games in education that can lead to engaging students significantly. Hence, i) finding the proposed game fascinating and engaging by students, and ii) the positive impacts of employing such technologies in schools, could be predicted, and the results of this study can be considered as another testimony to it. Having said that, gamified VR learning environments, along with integrating pedagogical approaches for learning geospatial topics, particularly topology relations, are considered as the contribution of the present study. It was shown in the results that this learning environment was efficient in learning topology relations. This finding of the study adds to the significance of employing new technologies for teaching and learning geospatial subjects. As mentioned in the introduction, such subjects are not being taught pervasively all around the world at schools. With the developments of technologies, it would be a good idea to use technologies and add such subjects to school curriculums; the findings of this paper support such insights.

5. Conclusions

In today’s world, everyone must make personal decisions, such as where to reside, that involve geospatial-related consideration. One can hardly deny the significant influence of such choices on one’s life. While these decisions may appear minor when examined individually, they have massive cultural, economic, and environmental consequences for other people and places when taken together. The presented paper employs the integration of educational games and VR technology in order to design and implement an IVREG for learning topology relations for middle-school students. In additions, pedagogical approaches, namely inquiry-based learning and instruction through presentation, have been used in the proposed learning environment to check their integration compatibility with VR and educational games, as well as their suitability for teaching geospatial topics, in particular topology relations. The results show that the proposed IVREG performed quite well. According to the pre- and post-tests, firstly, students asserted the effectiveness of inquiry-based learning along with instruction through presentation not only in VR technology but also for learning topology relations. The students’ perspectives on VR and educational games changed after experiencing the implemented IVREG. In other words, the questions regarding the effectiveness of educational games and VR technology in classrooms and especially for learning topology relations gained way higher scores in the post-test compared to the pre-test. This, in turn, indicates the high usability and effectiveness of these technologies for educational purposes.

5.1. Implications

Because solving a problem in a VR environment increases one’s sense of presence and is linked to less distraction and higher engagement [48], we tried to enhance the immersion of the game by integrating two pedagogical approaches. In addition, as learners remember knowledge for a considerably longer time when game characteristics are integrated into a VRLE compared to when game characteristics are used in traditional techniques, we used educational games in the VRLE for topology relations. This is due to the high level of immersion and presence that learners experience when they are in a VR environment. As GIS-related topics are not being taught pervasively at schools, the proposed solution can aid policymakers in including such lessons in school curriculums. In addition, introducing a new course might not normally be fascinating enough for students. Therefore, this paper’s solution that benefits from the integration of emerging technologies would make teaching new courses fascinating enough at schools. The findings of this paper revealed the benefits of employing technologies in educational purposes. As a result, researchers can use the provided valuable information to work on their research further.

5.2. Limitations

As for the limitations of this study, the work that is needed to be enacted in enhancing the immersion of such learning environments can be considered as one of the challenges for future studies. It is concluded from the PlQ results that VR experience gained an almost good score (3.5) but compared to other dimensions of playability considered in this study, that score was the lowest compared with the others. Although students asserted that the proposed IVREG is much better than looking at just 2D presentations in controversial classrooms, we expected that the PlQ scores would be higher. Another challenge would be to produce enough, appropriate teaching materials for geospatial topics, especially topology relations. Different pedagogical approaches can be investigated as well. Finally, future studies can evaluate the skills and knowledge gained from educational games for geospatial topics, as this study evaluated the knowledge gained based on students’ responses, not their performance in later real exams.

Author Contributions

Conceptualization, J.S.B. and A.S.-N.; methodology, J.S.B.; software, J.S.B.; validation A.S.-N. and S.-M.C.; formal analysis, J.S.B.; investigation, J.S.B., A.S.-N. and S.-M.C.; resources, A.S.-N.; data curation, J.S.B.; writing—original draft preparation, J.S.B.; writing—review and editing, A.S.-N. and S.-M.C.; visualization, J.S.B.; supervision, A.S.-N. and S.-M.C.; funding acquisition, S.-M.C. All authors have read and agreed to the published version of the manuscript.

Funding

This research was financially supported by the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2021-2016-0-00312) supervised by the IITP (Institute for Information & communications Technology Planning & Evaluation) and the Ministry of Trade, Industry and Energy (MOTIE) and Korea Institute for Advancement of Technology (KIAT) through the International Cooperative R&D program (Project No. P0016038).

Institutional Review Board Statement

Ethics approval is not required for this type of study, according to the local legislation.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A. Playability Questionnaire (PQ)

Time passes faster than when I am in traditional classrooms.
  • I can play the game for a long time.
  • If someone calls my name, I do not notice.
  • I do not comprehend what people are talking about around me.
  • The tasks in the game are meaningful.
  • Exploring the scene is fascinating.
  • Escaping the room is exciting.
  • I play well with the help of the tutorial of the game.
  • I feel motivated to use VR educational games more often.
  • The game is appropriate enough to be used in schools.
  • I better understand the topology relations by using this game.
  • The learning part of the game is noticeable.
  • Being in a virtual environment is not scary.
  • I lose track of my position in the classroom.
  • I do not feel motion sickness during the game.
  • Playing VR is not tiring for the eyes.
  • The game feels immersive.
  • Wearing the headset does not make me uncomfortable.

References

  1. Elstad, E. Digital Expectations and Experiences in Education; Springer: Rotterdam, The Netherlands, 2016. [Google Scholar]
  2. Kamińska, D.; Sapiński, T.; Wiak, S.; Tikk, T.; Haamer, R.E.; Avots, E.; Helmi, A.; Ozcinar, C.; Anbarjafari, G. Virtual Reality and Its Applications in Education: Survey. Information 2019, 10, 318. [Google Scholar] [CrossRef] [Green Version]
  3. Liou, W.-K.; Chang, C.-Y. Virtual reality classroom applied to science education. In Proceedings of the 2018 23rd International Scientific-Professional Conference on Information Technology (IT), Zabljak, Montenegro, 19–24 February 2018; pp. 1–4. [Google Scholar]
  4. Chen, X.; Gao, Y. Application and Innovation of Using Virtual Reality in Art Education. In Proceedings of the 9th International Conference on Education and Management (ICEM 2019), Tsuru, Japan, 25–27 October 2019. [Google Scholar]
  5. Fällman, D.; Backman, A.; Holmlund, K. VR in education: An introduction to multisensory constructivist learning en-vironments. In Proceedings of the Conference on University Pedagogy, Umea University, Umea, Sweden, 18–19 February 1999. [Google Scholar]
  6. Garris, R.; Ahlers, R.; Driskell, J.E. Games, Motivation, and Learning: A Research and Practice Model. Simul. Gaming 2002, 33, 441–467. [Google Scholar] [CrossRef]
  7. Cimons, M. Geospatial Technology as a Core Tool: Impacts Everything from Navigating to Law Enforcement. Published in US News and World Report. 2011. Available online: https://www.yumpu.com/en/document/view/37151814/geospatial-technology-as-a-core-tool-us-news-and-world-report (accessed on 1 September 2021).
  8. Lee, V.W.; Hodgson, P.; Chan, C.-S.; Fong, A.; Cheung, W.L. Optimising the learning process with immersive virtual reality and non-immersive virtual reality in an educational environment. Int. J. Mob. Learn. Organ. 2020, 14, 21–35. [Google Scholar] [CrossRef]
  9. Slater, M. Implicit Learning Through Embodiment in Immersive Virtual Reality; Springer: Singapore, 2017; pp. 19–33. [Google Scholar]
  10. Vergara, D.; Rubio, M.P.; Lorenzo, M. On the Design of Virtual Reality Learning Environments in Engineering. Multimodal. Technol. Interact. 2017, 1, 11. [Google Scholar] [CrossRef] [Green Version]
  11. Lin, J.-W.; Duh, H.; Parker, D.; Abi-Rached, H.; Furness, T. Effects of field of view on presence, enjoyment, memory, and simulator sickness in a virtual environment. In Proceedings of the IEEE Virtual Reality 2002, Orlando, FL, USA, 24–28 March 2002; IEEE: New York, NY, USA, 2003; pp. 164–171. [Google Scholar]
  12. Cavazza, M.; Charles, F.; Mead, S.J. Developing Re-Usable Interactive Storytelling Technologies; Springer: Singapore, 2008; Volume 156, pp. 39–44. [Google Scholar]
  13. Steuer, J. Defining Virtual Reality: Dimensions Determining Telepresence. J. Commun. 1992, 42, 73–93. [Google Scholar] [CrossRef]
  14. Tsivitanidou, O.E.; Georgiou, Y.; Ioannou, A. A Learning Experience in Inquiry-Based Physics with Immersive Virtual Reality: Student Perceptions and an Interaction Effect Between Conceptual Gains and Attitudinal Profiles. J. Sci. Educ. Technol. 2021, 30, 841–861. [Google Scholar] [CrossRef]
  15. Bouta, H.; Retalis, S. Enhancing primary school children collaborative learning experiences in maths via a 3D virtual environment. Educ. Inf. Technol. 2012, 18, 571–596. [Google Scholar] [CrossRef]
  16. Karaoğlan Yılmaz, F.G.; Yılmaz, R.; Kılıç, A.E. Examination of digital game habits of high school students. In Proceedings of the International Child and Information Safety Congress, Ankara, Turkey, 11—13 April 2018. [Google Scholar]
  17. Chassiakos, Y.R.; Radesky, J.; Christakis, D.; Moreno, M.A.; Cross, C. Children and Adolescents and Digital Media. Am. Acad. Pediatrics 2016, 138, e20162593. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Liu, M.; Rosenblum, J.A.; Horton, L.; Kang, J. Designing Science Learning with Game-Based Approaches. Comput. Sch. 2014, 31, 84–102. [Google Scholar] [CrossRef]
  19. Shin, N.; Sutherland, L.M.; Norris, C.A.; Soloway, E. Effects of game technology on elementary student learning in mathematics. Br. J. Educ. Technol. 2011, 43, 540–560. [Google Scholar] [CrossRef] [Green Version]
  20. Virvou, M.; Katsionis, G.; Manos, K. Combining software games with education: Evaluation of its educational effectiveness. J. Educ. Technol.Soc. 2005, 8, 54–65. [Google Scholar]
  21. Giannakos, M. Enjoy and learn with educational games: Examining factors affecting learning performance. Comput. Educ. 2013, 68, 429–439. [Google Scholar] [CrossRef]
  22. Bourgonjon, J.; Valcke, M.; Soetaert, R.; Schellens, T. Students’ perceptions about the use of video games in the classroom. Comput. Educ. 2010, 54, 1145–1156. [Google Scholar] [CrossRef]
  23. Chittaro, L.; Buttussi, F. Assessing Knowledge Retention of an Immersive Serious Game vs. a Traditional Education Method in Aviation Safety. IEEE Trans. Vis. Comput. Graph. 2015, 21, 529–538. [Google Scholar] [CrossRef]
  24. Boyle, E.; Connolly, T.M.; Hainey, T. The role of psychology in understanding the impact of computer games. Entertain. Comput. 2011, 2, 69–74. [Google Scholar] [CrossRef]
  25. Maitem, J. Math World: A Game-Based 3D Virtual Learning Environment (3D VLE) for Second Graders. Int. J. Multimedia Its Appl. 2012, 4, 1–13. [Google Scholar] [CrossRef]
  26. Udeozor, C.; Toyoda, R.; Abegão, F.R.; Glassey, J. Perceptions of the use of virtual reality games for chemical engineering education and professional training. High. Educ. Pedagog. 2021, 6, 175–194. [Google Scholar] [CrossRef]
  27. Ricker, A.A.; Richert, R.A. Digital gaming and metacognition in middle childhood. Comput. Hum. Behav. 2021, 115, 106593. [Google Scholar] [CrossRef]
  28. Torner, J.; Alpiste, F.; Brigos, M. Virtual Reality Application to Improve Spatial Ability of Engineering Students. 2016. Available online: https://upcommons.upc.edu/bitstream/handle/2117/97244/vr_spatial.pdf (accessed on 1 September 2021).
  29. Molina-Carmona, R.; Pertegal-Felices, M.; Jimeno-Morenilla, A.; Mora-Mora, H. Assessing the impact of virtual reality on engineering students’ spatial ability. In The Future of Innovation and Technology in Education: Policies and Practices for Teaching and Learning Excellence; Jimeno-Morenilla, A., Ed.; Emerald Publishing Limited: Bingley, UK, 2018. [Google Scholar]
  30. Bednarz, S.; Heffron, S.; Huynh, N. A road map for 21st century geography education. Geography Education Research (A Report from the Geography Education Research Committee of the Road Map for 21st Century Geography Education Project). Available online: http://gisgeo.org/assets/files/NGS_RoadMapConcept_GERC_07.pdf (accessed on 1 September 2021).
  31. Schlemper, M.B.; Athreya, B.; Czajkowski, K.; Stewart, V.C.; Shetty, S. Teaching Spatial Thinking and Geospatial Technologies Through Citizen Mapping and Problem-Based Inquiry in Grades 7-12. J. Geogr. 2018, 118, 21–34. [Google Scholar] [CrossRef]
  32. Zlatanova, S. Topological Relationships and Their Use, in Encyclopedia of GIS; Springer International Publishing: Cham, Switzerlandm, 2015; pp. 1–21. [Google Scholar]
  33. Shakirova, N.D.; Al Said, N.; Konyushenko, S.M. The Use of Virtual Reality in Geo-Education. Int. J. Emerg. Technol. Learn. (iJET) 2020, 15, 59–70. [Google Scholar] [CrossRef]
  34. Hagge, P. Student Perceptions of Semester-Long In-Class Virtual Reality: Effectively Using “Google Earth VR” in a Higher Education Classroom. J. Geogr. High. Educ. 2020, 45, 1–19. [Google Scholar] [CrossRef]
  35. Guzsvinecz, T.; Orbán-Mihálykó, É.; Perge, E.; Sik-Lányi, C. Analyzing the Spatial Skills of University Students with a Virtual Reality Application using a Desktop Display and the Gear VR. Acta Polytech. Hung. 2020, 17, 35–56. [Google Scholar] [CrossRef]
  36. Anderman, L.H.; Midgley, C. Motivation and middle school students. In What Current Research Says to the Middle Level Practitioner; Irvin, J.L., Ed.; National Middle School Association: Columbus, OH, USA, 1997; Volume 26, pp. 325–346. [Google Scholar]
  37. Wigfield, A.; Eccles, J.S. Students’ Motivation During the Middle School Years. In Improving Academic Achievement; Elsevier: Amsterdam, The Netherlands, 2002; pp. 159–184. [Google Scholar]
  38. Schell, J. The Art of Game Design: A Book of Lenses; CRC Press: Boca Raton, FL, USA, 2008. [Google Scholar]
  39. David, D.; Edwin; Arman, E.; Hikari; Chandra, N.; Nadia, N. Development of Escape Room Game using VR Technology. Procedia Comput. Sci. 2019, 157, 646–652. [Google Scholar] [CrossRef]
  40. Martirosov, S.; Kopecek, P. Virtual Reality and its Influence on Training and Education—Literature Review. In Proceedings of the DAAAM Proceedings; DAAAM International: Vienna, Austria, 2017; pp. 0708–0717. [Google Scholar]
  41. Griffiths, M.D.; Kuss, J.D.; King, D.L. Video game addiction: Past, present and future. Curr. Psychiatry Rev. 2012, 8, 308–318. [Google Scholar] [CrossRef] [Green Version]
  42. Shi, A.; Wang, Y.; Ding, N. The Effect of Game–Based Immersive Virtual Reality Learning Environment on LearningOut-Comes: Designing an Intrinsic Integrated Educational Game for Pre–Class Learning. Interactive Learning Environments. 2019. Available online: https://www.tandfonline.com/doi/citedby/10.1080/10494820.2019.1681467?scroll=top&needAccess=true (accessed on 1 September 2021). [CrossRef]
  43. Bandura, A.; Bryant, J.; Zillmann, D. Social cognitive theory of mass communication. In Media Effects: Advances in Theory and Research; Routledge: London, UK, 2002; pp. 94–124. [Google Scholar]
  44. Maiti, S.K.; Ghosh, D. Plant–soil interactions as a restoration tool. In Climate Change and Soil Interactions; Prasad, M.N.V., Pie-trzykowski, M., Eds.; Elsevier: Amsterdam, The Netherlands, 2020; pp. 689–730. [Google Scholar]
  45. Chiang, T.H.; Yang, S.J.; Hwang, G.-J. Students’ online interactive patterns in augmented reality-based inquiry activities. Comput. Educ. 2014, 78, 97–108. [Google Scholar] [CrossRef]
  46. Ibáñez, M.-B.; Delgado-Kloos, C. Augmented reality for STEM learning: A systematic review. Comput. Educ. 2018, 123, 109–123. [Google Scholar] [CrossRef]
  47. Kaye, L. Young Children in a Digital Age: Supporting Learning and Development with Technology in Early Years. Routledge: London, UK, 2016. [Google Scholar]
  48. Feng, Z.; González, V.A.; Amor, R.; Lovreglio, R.; Cabrera-Guerrero, G. Immersive virtual reality serious games for evacuation training and research: A systematic literature review. Comput.Educ. 2018, 127, 252–266. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Different views of the designed learning environment.
Figure 1. Different views of the designed learning environment.
Sustainability 13 13066 g001
Figure 2. A sample of images for training.
Figure 2. A sample of images for training.
Sustainability 13 13066 g002
Figure 3. (a) Training and validation Accuracy plot; (b) Training and validation loss plot.
Figure 3. (a) Training and validation Accuracy plot; (b) Training and validation loss plot.
Sustainability 13 13066 g003
Figure 4. (Left) UI with pictorial information at the beginning of the game; (Right) UI in the tutorial part of the game.
Figure 4. (Left) UI with pictorial information at the beginning of the game; (Right) UI in the tutorial part of the game.
Sustainability 13 13066 g004
Figure 5. (Left) Instruction through presentation for the “Meet” relation; (Right) Instruction through presentation for the “Disjoint” relation.
Figure 5. (Left) Instruction through presentation for the “Meet” relation; (Right) Instruction through presentation for the “Disjoint” relation.
Sustainability 13 13066 g005
Figure 6. The overall procedure of the study.
Figure 6. The overall procedure of the study.
Sustainability 13 13066 g006
Figure 7. The game mechanic component of the gaming design.
Figure 7. The game mechanic component of the gaming design.
Sustainability 13 13066 g007
Figure 8. A student playing the IVREG.
Figure 8. A student playing the IVREG.
Sustainability 13 13066 g008
Figure 9. (a) A randomly generated question, (b) the user is drawing objects indicating the required topology relation, (c) the message shown when the answer is correct.
Figure 9. (a) A randomly generated question, (b) the user is drawing objects indicating the required topology relation, (c) the message shown when the answer is correct.
Sustainability 13 13066 g009
Figure 10. The pre-test and post-test results of LPQ in box plot form.
Figure 10. The pre-test and post-test results of LPQ in box plot form.
Sustainability 13 13066 g010
Figure 11. The pre-test and post-test results of VRPQ in box plot form.
Figure 11. The pre-test and post-test results of VRPQ in box plot form.
Sustainability 13 13066 g011
Figure 12. The pre-test and post-test results of EGPQ in box plot form.
Figure 12. The pre-test and post-test results of EGPQ in box plot form.
Sustainability 13 13066 g012
Table 1. Learning Perspective Questionnaire (LPQ).
Table 1. Learning Perspective Questionnaire (LPQ).
IdItem
Q1I am interested in learning topology relations.
Q2It is necessary to learn geospatial topics.
Q3Learning topology relations is easy at the moment.
Q4I easily notice the relationships between objects in my surroundings.
Q5Instruction through observation is effective.
Q6The integration of inquiry-based learning and VR games is effective.
Table 2. Virtual Reality Perspective Questionnaire (VRPQ).
Table 2. Virtual Reality Perspective Questionnaire (VRPQ).
IdItem
Q1Virtual Reality (VR) is useful in classrooms.
Q2VR develops students’ creativity.
Q3VR is useful in teaching geospatial topics.
Q4VR is solely instructive enough and the presence of teachers is not required.
Q5Students using VR pay attention only to the entertainment parts.
Q6Students feel unconfident to use VR in classrooms.
Q7VR is more effective compared to looking at 2D presentations.
Q8VR is likely to be used in school frequently in the near future.
Table 3. Educational Game Perspective Questionnaire (EGPQ).
Table 3. Educational Game Perspective Questionnaire (EGPQ).
IdItem
Q1I would like to play educational games at school.
Q2Educational games enhance the process of passing down knowledge.
Q3Educational games stifle students’ creativity.
Q4Educational games are easy to understand and use.
Q5Educational games increase student motivation.
Table 4. Playability Questionnaire (PlQ).
Table 4. Playability Questionnaire (PlQ).
DimensionsMeanStd
PlQEngagement3.5675680.901666
Gameplay3.7905410.773317
Learning motivation4.0135140.73509
VR experience3.50.826095
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Safari Bazargani, J.; Sadeghi-Niaraki, A.; Choi, S.-M. Design, Implementation, and Evaluation of an Immersive Virtual Reality-Based Educational Game for Learning Topology Relations at Schools: A Case Study. Sustainability 2021, 13, 13066. https://doi.org/10.3390/su132313066

AMA Style

Safari Bazargani J, Sadeghi-Niaraki A, Choi S-M. Design, Implementation, and Evaluation of an Immersive Virtual Reality-Based Educational Game for Learning Topology Relations at Schools: A Case Study. Sustainability. 2021; 13(23):13066. https://doi.org/10.3390/su132313066

Chicago/Turabian Style

Safari Bazargani, Jalal, Abolghasem Sadeghi-Niaraki, and Soo-Mi Choi. 2021. "Design, Implementation, and Evaluation of an Immersive Virtual Reality-Based Educational Game for Learning Topology Relations at Schools: A Case Study" Sustainability 13, no. 23: 13066. https://doi.org/10.3390/su132313066

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop