A Systematic Review of the Design of Serious Games for Innovative Learning: Augmented Reality, Virtual Reality, or Mixed Reality?

Abstract

The recent integration of educational technologies and emerging learning approaches into education systems has been driven largely by the pandemic. This paper conducts a systematic review and delves into the new wave of research on serious games designed for innovative learning using augmented reality (AR), virtual reality (VR), and mixed reality (MR). The review was referenced to the review protocol, PRISMA 2020. Using the Scopus Database with a time filter from 2007 to 2023 (27 July), we searched 329 articles and shortlisted 273 relevant studies. Notably, European countries contributed the most (62.9%) to this research area. Among the most frequent keywords, VR (90.9%) was commonly used in AR/VR/MR, while e-learning (95.3%) was among the popular innovative learning approaches. Further research studies are needed to employ AR and MR technologies, as well as other innovative learning approaches, to enable performance evaluation and comparison of various educational technologies and learning approaches. We conducted an in-depth analysis of the relevant studies and their basic characteristics. Additionally, we introduced 15 essential and recently published AR/VR/MR standards to ensure better reliability, quality, and safety of architectures, systems, products, services, and processes. To facilitate performance evaluation and analysis, we surveyed 15 recently published benchmark education datasets. This review suggested four future research directions, including multisensory experiences, generative artificial intelligence, personalization and customization, and real-time interaction.

1. Introduction

Education is a vital element of the sustainable development of individuals, nations, and the world [1]. Many worldwide alliances and agendas have been established to promote education. One of the most prominent is the 2030 Agenda for Sustainable Development [2], which was agreed upon and adopted by 193 countries in the United Nations General Assembly in September 2015. The United Nations Educational, Scientific and Cultural Organization (UNESCO) coordinates communities across the globe to achieve Sustainable Development Goal 4 (SDG 4), namely “Quality Education”, through different means such as partnerships and policy guidance [3]. SDG 4 is one of the 17 SDGs outlined in the agenda and aims to “ensure inclusive and equitable quality education and promote life-long learning opportunities for all” [2]. Another initiative is UNESCO’s Global Education Coalition [4], which aims to support countries in scaling up their best distance learning practices and reaching children and youth who are most at risk.

Innovative learning. According to the SDG Report 2022 [5] compiled by the United Nations Statistics Division, the COVID-19 pandemic has led to 147 million children missing over half of the in-person instruction in 2020–2021, and remote learning is offered to 3 million Ukrainian children in the chaos of war as of April 2022. All of these crises have accelerated the transition of traditional education toward innovative learning, such as e-learning, agile learning, and blended learning. E-learning has become one of the mainstream ways of learning new skills and knowledge; for example, a common form of e-learning is online courses such as those on MIT’s Open Courseware (ocw.mit.edu) (accessed on 1 July 2023) and Coursera (www.coursera.org) (accessed on 1 July 2023). E-learning allows students to access learning materials and learn anywhere and anytime on any smart device, and its learning effectiveness has also been supported by a lot of research studies [6]. On the other hand, agile learning refers to the ability to adapt learning strategies quickly in response to change and bridge the gap between various identified needs of stakeholders and the learning process being deployed [7], and it is a concept transferred from agile methods in software engineering to the learning design process [8]. Noguera et al. [9] showed that agile strategies for teamwork organization can improve students’ team regulation and project management in collaborative project-based learning. Another example of innovative learning is blended learning, which refers to the combination of face-to-face and computer-mediated instructions, such as online learning and self-paced study [10]. Blended learning is an effective approach to creating a more engaging and effective learning experience for students [11,12]. During the pandemic, it has become a common solution to blend online learning and remote face-to-face learning to keep the students’ study progress on track and to ensure effective learning [11].

Serious games. Serious games are “digital games created with the intention to entertain and to achieve at least one additional goal (e.g., learning or health)” [13]. They have been applied in different educational contexts because of their effectiveness in promoting engagement and learning in different modes of learning, see, e.g., ref. [14] for e-learning, ref. [15] for agile learning, and [16] for blended learning. Two closely related concepts on serious games in education are gamification and game-based learning (GBL). Gamification is the process of combining game theory and design, game elements, game aesthetics, and game mechanics into a learning experience, in which serious games are created as products, while GBL is a pedagogical approach to teaching in which students can explore relevant aspects of games in a learning context designed by the teachers [17].

VR, AR, and MR in education. Virtual Reality (VR) is a computer technology that creates a 3D virtual environment and simulates the user’s physical presence in a virtual world [18]. Augmented Reality (AR) also generates 3D virtual environments for users, but AR integrates the virtual world with reality by anchoring 3D models to specific locations or objects in the physical environment, such as those captured on a camera screen. Mixed Reality (MR) combines elements of both VR and AR to create an interactive and immersive experience that blends the digital and physical worlds, in which digital objects and information are also superimposed onto the real world like AR, yet MR creates a spatially aware environment that allows users using a see-through display or a headset, e.g., Microsoft’s HoloLens 2 device, to interact with virtual objects in a 3D environment in a way different to AR [19]. Over the past decade, the affordability and user-friendliness of the hardware and software supporting VR and AR (referred to collectively as immersive reality) have improved, leading to their growing use in education (e.g., refs. [20,21,22,23,24]; refer to surveys [25,26] for more examples). The utilization of VR and AR in education enables learners to practice skills and gain knowledge in a safe and controlled virtual environment. It also provides them with access to experiences and resources that may not be available in traditional classrooms [26]. VR and AR serious games are particularly well-suited for skill-based education and attract students’ interest in learning [27].

Existing reviews on innovative learning. Numerous reviews have been conducted on innovative learning across diverse subject domains, with each review emphasizing a specific technology within its respective field of study. In the medical domain, Gaballa et al. [28] conducted a study to explore the potential of augmented reality (AR), virtual reality (VR), and mixed reality (MR) systems in enhancing upper-limb prosthesis rehabilitation. Kleinsorgen et al. [29] performed a descriptive analysis of publication activities related to digital teaching and learning in the GMS Journal for Medical Education from 1984 to 2020. Shahid et al. [30] compared various information technology applications for the treatment of autism. On the other hand, the education domain also benefits from the integration of innovative learning technologies. For instance, Ongoro and Fanjiang [31] conducted a systematic review that examined the use of digital game-based learning technology for English language instruction in preschool and elementary schools. Baxter et al. [32] developed a framework through a comprehensive preliminary discursive review of the literature on innovative education and computer technology, specifically in the context of teaching history. Ullah et al. [33] conducted a systematic literature review focused on serious games for science education, highlighting research trends and patterns from 2011 to 2021.

In addition to industrial training, innovative learning technologies have also been examined in other domains, such as personal skills development. For example, Vigoroso et al. [34] conducted a synthesis of available works on game-based safety training across various occupational domains. Williams-Bell et al. [35] reviewed several serious games and virtual simulations that aid in fire service training. Wahyudin and Hasegawa [36] focused on the role of serious games in disaster and safety education. Feng et al. [37] explored the use of immersive virtual reality serious games for building evacuation training and research, specifically for indoor emergencies like fire and earthquakes.

Some reviews cover multiple domains and investigate specific technologies or methodologies. For instance, Hare and Tang [38] categorized player modeling and game adaptation methods in serious games for higher education. Chavez and Bayona [39] selected articles published between 1999 and 2017 across various fields of study to identify the characteristics of virtual reality technology and its impact on the learning process. Checa and Bustillo [27] reviewed 135 proposals for immersive virtual reality serious games, examining their standards and differences between training and learning applications. Yomeldi et al. [40] conducted a systematic literature review of serious games for mobile learning, highlighting the potential influence of visualization, enjoyment, immersion, and interactivity on learning outcomes.

However, there is currently a lack of research specifically focused on the design of serious games for innovative learning.

1.1. Research Contributions

To address the above significant aspects and gaps, the objective of this systematic review using a review protocol (PRISMA 2020) [41] is to provide an in-depth analysis of the relevant studies on serious games designed for innovative learning using augmented reality (AR), virtual reality (VR), and mixed reality (MR):

• We provide an in-depth analysis of the 273 relevant studies and their basic characteristics.
• Our work introduces 15 essential AR/VR/MR standards to ensure better reliability, quality, and safety of architectures, systems, products, services, and processes.
• Lastly, to facilitate performance evaluation and analysis, we survey 15 benchmark education datasets.

1.2. Organization of the Article

The rest of the article is organized as follows. Section 2 presents the methodology of article search and selection processes to shortlist relevant articles in AR/VR/MR research areas for innovative learning using serious games. It is followed by the discussion of results in Section 3, comprising basic characteristics of shortlisted articles and an in-depth analysis of journal articles. To support environments with good quality, repeatability, safety, interoperability, and compatibility, AR, VR, and MR standards (published and under development) will be summarized in Section 4. Lastly, a conclusion is drawn in Section 5. Figure 1 summarizes the structure of this article.

2. Methodology

As a review article, this section starts with an illustration of the search process, followed by a selection process to exclude and shortlist articles for an in-depth literature review in the following sections.

2.1. Search Process

Advanced document search of Scopus Database was used to search for relevant articles, which is the largest database in this domain. The search was based on article title, abstract, and keywords. The query string comprised: (“serious game”) AND (“blended learning” OR “hybrid learning” OR “e-learning” OR “agile learning”) AND (“virtual reality” OR “VR” OR “augmented reality” OR “AR” OR “mixed reality” OR “MR”). The learning approaches covered the latest categories of innovative learning. The shortlisted articles must be serious games-related and fall into at least one of the learning and reality approaches. Three authors (L.-K.L., X.W., and K.T.C.) checked the query and confirmed the results. The initial search responses were 329 articles with a time filter from 2007 to 2023 (27 July, the first date on which the team prepared this article).

2.2. Articles Selection Process

The inclusion criteria for the review were that the articles must be related to education purposes and that details must be presented to describe the game setting and approach of innovative learning. Three authors independently and randomly reviewed all papers’ titles and papers. Each author read about 74 papers. Discussions were made when there was any inconsistency. To shortlist appropriate articles, the following exclusion criteria were adopted.

• To ensure technical content is discussed, only non-review articles under the document type are included. Therefore, 30 articles are excluded, including conference review (n = 21), article review (n = 6), short survey (n = 1), note (n = 1), and book (n = 1);
• To ensure the best communication in academia, only articles presented in English are included. Thus, four articles are excluded, including Spanish (n = 1), French (n = 1), German (n = 1), and Hungarian (n = 1);
• The authors carefully read the titles and abstracts of the articles and further excluded 22 articles, including conference review (n = 11), article review (n = 10), and book chapter review (n = 1).

After applying the criteria, 273 articles were shortlisted. Among the shortlisted articles, an in-depth analysis will be conducted on journal articles (n = 44). To summarize the article selection process, a workflow is presented in Figure 2.

3. Results

The shortlisted 273 articles are analyzed in various aspects: (i) basic characteristics include the number of publications, distribution of the document type, geographical distribution of country affiliations, distribution of subject areas, publication sources, keywords analysis, and Word cloud; and (ii) literature review of shortlisted journal articles.

3.1. Basic Characteristics

The number of articles published between 2007 and 2023 (up to 27 July 2023) is summarized in Figure 3. It can be seen from the trend that the research areas have generally received increasing intentions (although it is not monotonically increasing) from 2007 (two articles) to 2022 (40 articles). Notably, more articles were published during the COVID-19 pandemic (2020–2022) because the face-to-face learning mode was usually not feasible [42,43]. To further study the distribution of the document type of these articles, Figure 4 presents a pie chart showing 223 conference papers (82%), 44 articles (16%), and six book chapters (2%). The distribution indicates that many research studies published as conference papers are initial ideas, preliminary analyses, and case studies. It reflects that there is room for future research for innovative learning using VR and AR.

Table 1. Geographical distribution of the country affiliations of all the authors.

Countries/Territories	Number of Articles	Countries/Territories	Number of Articles
Germany	38	Austria	2
United States	34	Croatia	2
United Kingdom	23	Ecuador	2
Italy	17	Indonesia	2
France	16	Iran	2
Portugal	15	Malaysia	2
Greece	14	Mexico	2
Spain	13	Poland	2
Australia	11	Slovakia	2
Brazil	10	Switzerland	2
China	10	Taiwan	2
Netherlands	9	Thailand	2
Canada	8	Algeria	1
Norway	8	Cyprus	1
New Zealand	7	Egypt	1
Denmark	6	Hungary	1
South Korea	6	Iceland	1
Sweden	6	Ireland	1
Colombia	5	Lithuania	1
Japan	5	Pakistan	1
Belgium	4	Philippines	1
Chile	4	Qatar	1
Finland	4	Saudi Arabia	1
Morocco	4	Serbia	1
Romania	4	Sri Lanka	1
Singapore	4	Tunisia	1
Bosnia and Herzegovina	3	Turkey	1
Bulgaria	3	Ukraine	1
India	3

summarizes the geographical distribution of the authors’ country affiliations (n = 160). Of the 273 articles, 57 countries/territories contributed to the research. The continents’ contributions are ranked in descending order: Europe (62.9%), Asia (13.5%), North America (13.2%), South America (6.29%), Africa (2.10%), and Oceania (2.10%). The research studies of VR and AR for innovative learning are dominated by European institutions, which is attributable to the unique characteristics of their educational systems [44,45]. However, this does not mean other continents far less value innovative learning, where they devote effort to other learning tools (less on AR/VR/MR). During a pandemic, the research and adoption of innovative learning have been accelerated because the changing mode of learning drives the mitigation (at least some trials) from face-to-face learning to any innovative learning.

Figure 5 shows the distribution of the articles based on 22 subject areas, where 7 out of 22 (31.8%) subject areas comprise at least ten articles encompassing AR/VR/MR for innovative learning. The top three dominant subject areas are computer science (82.1%), engineering (43.4%), and social sciences (28.8%). The first two subject areas can be explained by the fact that AR/VR/MR and educational technologies usually cover discussion in the domain of computer science and engineering. To further analyze the key contributions, the annual publications in the top three subject areas are shown in Figure 6. The contributions started to be more obvious in 2017. Key observations are illustrated in each area: (i) The number of annual publications in computer science remained steady every two years (2017–2018, 2019–2020, and 2021–2022) and stepped up to the next level (about five articles increment); (ii) The number of annual publications in engineering fluctuated (up and down) four times from 2017 to 2021; and (iii) The number of annual publications in social sciences remained steady (about 10 on average).

Table 2. Selected AR/VR/MR publication sources for innovative learning (from 2007 to 27 July 2023).

Publication Sources	Number of Articles	Publication Sources	Number of Articles
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics	26	10th International Conference on Information Intelligence Systems and Applications Iisa 2019	2
ACM International Conference Proceeding Series	9	2019 IEEE 7th International Conference on Serious Games and Applications for Health Segah 2019	2
Studies in Health Technology and Informatics	7	2022 IEEE Games Entertainment Media Conference Gem 2022	2
2016 8th International Conference on Games and Virtual Worlds for Serious Applications vs. Games 2016	5	2022 International Conference on Interactive Media Smart Systems and Emerging Technologies Imet 2022 Proceedings	2
Advances in Intelligent Systems and Computing	5	ACM SIGGRAPH 2019 Posters SIGGRAPH 2019	2
Ceur Workshop Proceedings	5	ASEE Annual Conference and Exposition Conference Proceedings	2
Proceedings of The European Conference on Games Based Learning	5	Conference on Human Factors in Computing Systems Proceedings	2
2017 9th International Conference on Virtual Worlds and Games for Serious Applications vs. Games 2017 Proceedings	4	IEEE Access	2
2017 IEEE 5th International Conference on Serious Games and Applications for Health Segah 2017	4	IEEE Conference on Computational Intelligence and Games Cig	2
Communications in Computer and Information Science	4	Iadis International Conference on Cognition and Exploratory Learning in Digital Age Celda 2013	2
IEEE Global Engineering Education Conference Educon	4	Information Switzerland	2
Virtual Reality	4	International Journal of Emerging Technologies in Learning	2
2018 IEEE 6th International Conference on Serious Games and Applications for Health Segah 2018	3	Proceedings 2022 IEEE Conference on Virtual Reality And 3D User Interfaces Abstracts and Workshops Vrw 2022	2
Advanced Engineering Informatics	3	Proceedings Frontiers in Education Conference Fie	2
Computers and Education	3	Proceedings IEEE Virtual Reality	2
International Archives of The Photogrammetry Remote Sensing and Spatial Information Sciences ISPRS Archives	3	Proceedings of 2022 8th International Conference of The Immersive Learning Research Network Ilrn 2022	2
Proceedings of The European Conference On E-Learning Ecel	3	Proceedings of The International Conferences on E Society 2022 And Mobile Learning 2022	2
Segah 2021 2021 IEEE 9th International Conference on Serious Games and Applications for Health	3	Tale 2021 IEEE International Conference on Engineering Technology and Education Proceedings	2

summarizes the selected list of AR/VR/MR publication sources for innovative learning, with at least two articles published between 2007 and 2023 (up to 27 July 2023). In total, there are 160 sources, 36 of which (22.5%) are listed in Table 2. The leading publication source Lecture Notes in Computer Science, Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, is mainly supported by collecting various conference proceedings since 2012.

Table 3. Selected AR/VR/MR keywords for innovative learning (from 2007 to 27 July 2023).

Keywords	Number of Articles	Keywords	Number of Articles
E-learning	261	Learning Environments	15
Serious Games	254	Human	14
Virtual Reality	249	Three-Dimensional Computer Graphics	14
Students	61	Training	14
Serious Game	52	User Interfaces	14
Education	46	Helmet Mounted Displays	13
Augmented Reality	37	Learning	13
Interactive Computer Graphics	35	Virtual Learning Environments	13
Personnel Training	35	Immersive	12
Computer Aided Instruction	29	Motivation	11
Engineering Education	28	Surveys	11
Virtual Worlds	28	User Experience	11
Gamification	27	Article	10
Human-Computer Interaction	27	Head-Mounted Displays	10
Immersive Virtual Reality	26	Health	10
Learning Systems	26	Humans	10
Teaching	25	Mixed Reality	10
Game-based Learning	22	Simulation	10
Education Computing	20	Virtual Reality Technology	10
Educational Game	17	Virtual Reality Environment	10

summarizes the top 40 AR/VR/MR keywords for innovative learning. Figure 7 provides the trends of the top three keywords. Some key observations are concluded: (i) The number of annual publications of all three keywords dropped in 2018; (ii) The number of annual publications in the top keyword, e-learning, increased monotonically from 2017 to 2021; (iii) The number of annual publications in the second top keyword, serious games fluctuated (up and down) from 2017 to 2021; and (iv) The number of annual publication in the third top keyword, virtual reality increased monotonically from 2017 to 2022. As a full picture, Figure 8 shows a word cloud of all keywords.

3.2. Literature Review of Shortlisted Journal Articles

To balance a concise and in-depth discussion of AR/VR/MR for innovative education, only shortlisted journal articles (n = 44) are fully investigated in this subsection.

Table 4. Summary of journal articles in AR/VR/MR for innovative learning (from 2007 to 27 July 2023).

Work	AR/VR/MR	Innovative Learning	Gaming	Digital Tools	Application	Subjects	Duration of Study	Results
[46]	AR	Immersive learning	Serious games	Hardware: Software: Unity, Adobe Photoshop CC, and Blender	Enhance the understanding of prism (a Mathematics topic)	20 students ages nine years old	Played the game once and answered pre- and post-test questions in 30 min	Enhanced the mean test result from 12.6 to 14.6 (out of 30)
[47]	AR	E-learning	Serious games	Hardware: Smartphone and computer; Software: Dental Simulator v1.13	Enhance dental skills for local anesthesia	19 students studying for a dental degree	10 days	About 90% of students agreed on the benefit of games in the learning process
[48]	AR	E-learning and blended learning	Serious games	Hardware: Computer; Software: pharmacokinetics simulator and pharmacodynamics simulator	Harmonize and improve clinical therapeutics and pharmacology education	99 clinical therapeutics and pharmacology teachers	Not applicable using questionnaires	70% of respondents used innovative learning
[49]	VR	Digital experiential learning	Serious role-playing game	Hardware: Computer and mobile devices; Software: Reallusion and Unity	Enhance healthcare skills of the healthcare professionals in various aspects, such as the awareness of the enduring impact of the social determinants of health and the importance of cultural humility	50 volunteers, with ages of 21 to 70	Played the game once and answered pre- and post-survey questions	Enhanced the mean survey result from 4.19 to 4.4 (out of 5)
[50]	VR	Immersive learning	Serious games	Hardware: Neuromender rehabilitation system, VR headset, and computer; Software: Neuromender	Enhance upper-limb rehabilitation post-stroke	Four participants, ages 62 to 82	Three game settings of 2, 3, and 4 min	Average score of 3.6 (out of 4.0) in 15 engagement levels
[51]	VR	Immersive learning	Serious games	Hardware: HTC Vive; Software: Unity	Enhance the learning and skills for motor tasks	57 participants	A game of 3 min	Average usability score of 81.7
[52]	VR	Immersive learning	Serious games	Hardware: Head-mounted display and wearable haptic devices; Software: Unity	Rehabilitation training for motor-impaired children	20 participants ages 7 to 32	12 to 20 min in the kinematic assessment	Lowered the mean missed contacts from 4.16 to 3.29
[53]	VR	Immersive learning	Problem-based serious games	Hardware: Immersive VR headset and computer; Software: Unity	Earthquake emergency training for the enhancement of knowledge and self-efficacy	99 university staff and students, ages 18 to 53	Played the game once and answered pre- and post-test questions	Enhanced knowledge from 7.52 to 8.7 and self-efficacy from 3.21 to 10.24
[54]	VR	Immersive learning	Serious games	Hardware: Computer; Software: Unity	Enhance hazard detection in and knowledge of underground coal mines	30 participants with a mining or gaming development background	One week (at least 30 min per day)	Increased test score from 7.22 to 8.76
[55]	VR	Immersive learning	Serious games	Hardware: Oculus Rift/Quest and computer; Software: Unity	Enhance road safety education and awareness	30 participants	Experienced one virtual accident	Average rating of playing and learning experience of 4.8 (out of 5)
[56]	VR	Immersive learning	Serious games	Hardware: HTC Vive; Software: Mammoth XR	Enhance the basic operation steps for the fire extinguisher	93 participants, ages 21 to 61	Played the game once and answered pre-, post, and retention test questions	Increased average test score from 1.7 to 4.4 (post-test) and 3.7 (retention test)
[57]	VR	Immersive learning	Serious games	Hardware: Oculus Rift; Software: Unity	Train the earthquake response and post-earthquake evacuation	191 participants, ages 11 to 53	Played the game once	Average ease of score of 3.64 (out of 4) in children and 2.45 (out of 3)
[58]	VR	Immersive learning	Serious games	Hardware: HTC Vive; Software: Unity	Evaluate risk aversion and safety awareness of the miners in underground mines	20 participants, ages 20 to 40	Played the game for 12 min and answered post- and retention test questions	Average points of 77.6 (post-test) and 66 (retention test)
[59]	VR	Immersive learning	Serious games	Hardware: Head-mounted display, Xbox 360, and computer; Software: Unity	Training of aviation safety procedures	96 participants, ages 18 to 36	Played the game once and answered pre-, post, and retention test questions	Average score of 2.91 (pre-test), 4.08 (post-test), 4.39 (retention test)
[60]	VR	Immersive learning	Serious games	Hardware: Oculus Rift S and computer; Software: Unreal Engine v4.26 and PhysX5	Enhance the understanding of projectile kinematics (a Physics concept)	133 students	45–60 min	Average immersion score of 4.52
[61]	VR	Immersive learning	Serious games	Hardware: HTC Vive; Software: Cognitive Optical Brain Imaging Software v1.3.0.19	Understand the DNA replication process	100 participants	Not applicable	Enhanced hemodynamic response compared with lecture
[62]	VR	Immersive learning	Serious games	Hardware: HTC Vive Pro and computer; Software: Unity, Photon Unity Networking 2, Ready Player Me, CC3 Base+, MetaHumans, and MakeHuman	The design of an Avatar	Six participants	Played the game once	Average score of 4.78 out of 7
[63]	VR	Immersive learning	Serious games	Hardware: Z800 3DVisor and computer; Software: Unity, Rhinoceros, and 3ds Max	Training of collaborative manufacturing tasks between humans and robots	30 participants	Played the game once	Average satisfaction score of 4.02 (out of 5)
[64]	VR	Non-immersive learning	Serious games	Hardware: Computer; Software: Adobe Fuse CC, Autodesk Revit, 3ds max, and Unity3D	Train the skills for healthcare fire safety	78 staff members in the hospital, ages 24 to 64	Played the game once and answered pre-, and post-test questions	Increased average score from 4.55 to 7.23
[65]	VR	E-learning	Serious games	Hardware: Computer; Software: 3ds Max, V-Ray, Blender, ZBrush, and Unity	Educate the learning approach to coma	50 medical students	Played the game once	Enhanced mean score of learning outcome from 12 to 14.1
[66]	VR	E-learning	Serious games	Hardware: Computer; Software: Unity and MedStar Digital Simulation Platform	Enhance the mannequin-based training of nurses and physicians	36 participants with a mean age of 32.8	3 months	Average post training perception score of 3.41 (out of 5)
[67]	VR	E-learning	Serious games	Hardware: Head-mounted display and smartphone; Software: mStikk and wStikk	Enhance teaching and learning for biomedical laboratory science education	Nine participants	60 to 80 min	Only qualitative analysis
[68]	VR	E-learning	Serious games	Unspecified	Enhance the teaching and learning of medical knowledge, skills, and experience	143 participants	Not applicable	72% of respondents agreed the helpfulness of VR
[69]	VR	E-learning	Serious games	Unspecified	Understand the expectations and stressors of teaching and learning	372 students with a mean age of 23.9	Not applicable	Only 17% of respondents recognized serious games and VR
[70]	VR	E-learning	Serious games	Hardware: Apple Macintosh, Truevision3D, and Surgical Theater Surgical Planner; Software: VMWare Fusion, Adobe Flash, and Autodesk 3D Studio Max	Visualize anatomical Spaces for Pterygopalatine Fossa	Unspecified	Not applicable	Only qualitative analysis
[71]	VR	Project-based learning	Serious games	Hardware: HTC Vive; Software: Unreal Engine 4	Enhance the interactions in urban planning, construction, and architecture education	133 participants with a mean age of 28.6	Played the game once	Average bipolar laddering assessment score of 8.65
[72]	VR	Collaborative learning	Serious games	Hardware: VR headset; Software: Unreal Engine	Training of the advanced cardiac life support	148 clinicians	28 min	Increased correctness of tasks from 39.4% to 58.3%
[73]	VR	E-learning and immersive learning	Serious games	Hardware: Oculus Quest2 and computer; Software: 3DMax	Enhance rescue skills in a confined space	22 participants with a mean age of 27.2	Played the game once	Reduced the mistake rate from 36% to 13.1%
[74]	VR	Immersive learning and active learning	Serious games	Hardware: HTC Vive, LiDAR, and computer; Software: Unity	Train construction safety	14 participants	Average of 66 s per VR scene	VR system achieved similar performance as sensor system
[75]	VR	Immersive learning and active learning	Serious games	Hardware: smartphones; Software: Unity	Increase the engagement of the student learning process	Unspecified number of students	Played the game once	Integrating educational content into different games will further enhance the performance
[76]	VR	Immersive learning and blended learning	Serious games	Hardware: Computer Software: Unity	Enhance the learning outcomes and cognitive benefits of STEM education	41 teachers ages 18 or above	Within one hour	Only 61% of participants completed the game
[77]	VR	Immersive learning and blended learning	Serious games	Hardware: VR glasses and computers; Software: CAVE automatic virtual environment	Enhance the security and safety education	418 high school students age 17 to 23	Played the game once	Average assessment score of 3.29 (out of 5)
[78]	VR	Immersive learning, short-term learning, and long-term learning	Serious games	Hardware: Samsung Gear VR, smartphone, and computer; Software: Unity	Study of the influence of eyestrain and apparatuses on the quality of experience and learning performance	42 participants, ages 18 to 39	Average of 34.3 min	Average improvement of game score by 6.40%
[79]	VR	Immersive learning, autonomous learning, and online learning	Serious games	Hardware: Oculus Touch and computer; Software: Blender	Increase students’ learning and satisfaction	77 students with a mean age of 18.6	Played the game once	Reduced error from 28% to 21%
[80]	MR	Smart learning	Serious games	Unspecified	Provide interactive and user-driven learning experiences for visitors at cultural heritage sites	Unspecified	Not applicable	Only qualitative analysis
[81]	VR and MR	Immersive learning	Serious games	Hardware: HTC Vive and computer; Software: Unity	Enhance the teaching of battery concepts	More than 500 participants	Not applicable	Only qualitative analysis
[82]	AR and VR	Active learning	Serious games	Hardware: HTC Vive; Software: Unity	Enhance construction safety hazard awareness	30 participants	Average of 7.18 min	Average success rate of 33%
[83]	AR and VR	E-learning	Serious games	Hardware: Head-mounted device; Software: Unity	Train cadets	113 participants	Two years	Enhanced average grade from 3.78 to 4.13 (first year) and 4.22 (second year)
[84]	AR and VR	E-learning and blended learning	Serious games	Hardware: Computer; Software: STOP Disaster online video game	Enhance skills to manage disaster events	52 students	Played the game once	Enhanced correctness of questions from 61% to 82.8%
[85]	Virtual environments	E-learning	Serious games	Hardware: Learning management system Software: Arctic Economy	Evaluate and quantify the parameters related to learning success and motivation of serious game-based learning	97 students ages 19 to 39	Played the game once and answered pre-, post-, and retention test questions	Enhanced average score from 12 to 26.3 (post-test) and 25.7 (retention test)
[86]	Virtual environment	E-learning	Serious games	Hardware: Computer; Software: Unity	Training of cardiopulmonary resuscitation protocol for nursing education	109 nursing students	Played the game once	Enhanced average test score from 35.7 to 47.5
[87]	Virtual environments	E-learning and blended learning	Serious games	Hardware: Learning management system; Software: SCORM and Reload Editor	Enhance the monitoring of the learning management system	Unspecified	Not applicable	Only qualitative analysis
[88]	Virtualization	Hybrid learning and online learning	Serious games	Hardware: Computer; Software: LegoCAD	Enhance the lean construction education	Unspecified number of students	Not applicable	Ontology helped virtualization of games
[89]	Virtual worlds	E-learning	Serious games	Hardware: Computer; Software: Virtual Singapura	Experience in a virtual environment for science education	28 pre-service teachers	Not applicable	Only qualitative analysis

summarizes the crucial elements of the existing studies [46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89]. The publication years of the journal articles ranged from 2011 to 2023. Among 44 articles, the majority of studies utilized VR (n = 36, 81.8%), followed by AR (n = 6, 13.6%), unspecified type of reality (n = 5, 11.4%), and MR (2, 4.55%). Regarding innovative learning, various approaches were considered, such as immersive learning (n = 23, 52.3%), e-learning (n = 15, 34.1%), blended learning (n = 5, 11.4%), active learning (n = 2, 4.55%), online learning (n = 2, 4.55%), digital, experiential learning (n = 1, 2.27%), short-term learning (n = 1, 2.27%), long-term learning (n = 1, 2.27%) hybrid learning (n = 1, 2.27%), non-immersive learning (n = 1, 2.27%), project-based learning (n = 1, 2.27%), autonomous learning (n = 1, 2.27%), smart learning (n = 1, 2.27%), and collaborative learning (n = 1, 2.27%). Please note that the virtual experience in emergency-, safety-, or security-related applications is common and effective (n = 16, 36.4%). In the literature, only a few works utilized MR [80] and mixed approaches [81,82,83,84] that reflect room for future research studies. In addition, some works [85,86,87,88,89] did not specify the specific approaches among AR/VR/MR.

4. Standards of AR/VR/MR

Standardization helps to enhance the reliability, quality, and safety of architecture, systems, products, services, and processes [90]. It is often linked to compatibility and standards. An illustrative explanation is that compatibility provides a mechanism for different parties to work to accomplish standards jointly, whereas standards guarantee compatibility [91].

Table 5. Standards of VR and AR.

Name of Standards	Publication Year	Descriptions of Standards
IEEE P2048.1-IEEE P2048.12 [92]	Under development	Standards for VR and AR (i) device taxonomy and definitions; (ii) immersive video taxonomy and quality metrics; (iii) immersive video file and stream formats; (iv) person identity; (v) environment safety; (vi) immersive user interface; (vii) map for virtual objects in the real world; (viii) interoperability between virtual objects and the real world; (ix) immersive audio taxonomy and quality metrics; (x) immersive audio file and stream formats; (xi) in-vehicle augmented reality; (xii) content ratings and descriptors
IEEE P7030 [93]	Under development	A standard specifies the overview, definitions, classifications, and ethics of extended reality.
IEEE 1857.9-2021 [94]	Published in 2022	A data compression standard for effective compressions, decompressions, and reconstructions of VR and AR contents.
ISO/IEC 23488:2022 [95]	Published in 2022	A standard defines the representation of target environments/objects using (3D) images for mixed reality and AR.
ISO/TS 9241-430:2021 [96]	Published in 2021	The guidance specifies gesture sets’ design, selection, and optimization processes for VR and AR.
ISO/IEC TS 23884:2021 [97]	Published in 2021	A standard supplements the scenes and models for VR and AR of various standards, e.g., ISO/IEC 3721-1 [107] and ISO/IEC 19775 [108]
ANSI/CAN/UL 8400 [98]	Published in 2021	A standard specifies the safety regulation of VR and AR.
IEEE 1589-2020 [99]	Published in 2020	An interoperability standard for AR-related learning systems. It helps to establish online marketplaces and experience repositories.
ISO/IEC 18038:2020 [100]	Published in 2020	A standard specifies the architecture, information reference model, system functions and integration, and exchange format between mixed reality applications.
ISO/IEC TR 23843:2020 [101]	Published in 2020	A standard defines a catalog model for efficiently searching VR and AR in education.
ISO/IEC TR 23842-1:2020 [102]	Published in 2020	A standard considers the training, education, and learning of the VR content.
ISO/IEC 18039:2019 [103]	Published in 2019	A standard provides mixed reality and AR definitions, terms, concepts, and reference models.
ISO/IEC 18040:2019 [104]	Published in 2019	A standard for representing and controlling live actors and entities in mixed reality and AR scenes. The concepts, functions, frameworks, models, and system integration are also defined.
ISO/IEC 18520:2019 [105]	Published in 2019	A standard provides the benchmarking criteria and processes of vision-based spatial registration and tracking approaches for mixed reality and AR.
ISO/IEC 23000-13:2017 [106]	Published in 2017	A standard describes the scene elements for AR content.

summarizes the 15 latest standards (two of them are under development) of AR/VR/MR, where standards beyond the range can be referred to in previous literature reviews [92,93,94,95,96,97,98,99,100,101,102,103,104,105,106]. Many standards organizations actively establish AR/VR/MR standards. Examples include the Institute of Electrical and Electronics Engineers (IEEE), the International Organization for Standardization (ISO), the International Electrotechnical Commission (IEC), the American National Standards Institute (ANSI), and UL Solutions (UL).

Data collection has played an important role (usually one of the very first steps) in conducting research studies. Whenever it involves the recruitment or participation of volunteers, cost and time concerns limit the continuity of data collection. Therefore, researchers usually desire benchmark datasets ready to conduct research studies.

Table 6. VR and AR Datasets in Education.

Work	Publication Year	Accessibility	Nature of Datasets	Applications
[109]	2023	Open access	447 samples of various hand gestures using VR	Learning and interpreting sign languages
[110]	2023	A reasonable request to the corresponding author	30 students participated in AR-based classroom learning activities	Engagement detection of STEM education
[111]	2022	A reasonable request to the corresponding author	177,238 education-related tweets using AR; 299,917 tweets education-related tweets using VR	polarity analysis; emotional analysis
[112]	2022	A reasonable request to the corresponding author	4032 videos include 672 actions using AR	Hand action detection for piano training
[113]	2022	A reasonable request to the corresponding author	Popularity, comfort rating, user rating, and price of 4687 educational VR apps	Analysis of educational VR apps
[114]	2022	A reasonable request to the corresponding author	3700 students learned martial arts Taijiquan using VR	VR martial arts education
[115]	2022	A request by returning a completed form called an end user license agreement	22 students read books with AR devices	Classifying the emotional states of students
[116]	2022	A reasonable request to the corresponding author	13 students adopted VR-reconstructed operation scenarios to study interaction designs	Contextual substitution for fieldwork in design education
[117]	2022	A reasonable request to the corresponding author	35 students learned Mathematics and Nature and Society lessons in AR-based environments	Student engagement analysis
[118]	2022	A reasonable request to the corresponding author	143 students learned environmental awareness topics via AR-based websites.	Enhancement of student environmental awareness
[119]	2021	Open access	116 educational mobile AR apps and 1752 user reviews	User feedback analysis
[120]	2021	Open access	Logs of 20 users’ behaviors while using AR apps	User behavior analysis
[121]	2020	Open access	118,000 images inside a room using AR and mixed reality	Enhancement of the simulation of indoor 3D scenes
[122]	2018	Open access	19 drivers used an AR-based driving simulator	Driving training to improve vehicle-pedestrian interactions
[123]	2018	Open access	169 users contributed 1980 gazes and head trajectories using VR	automatic alignment of VR video cuts

summarizes 15 benchmark datasets in descending order in publication year [109,110,111,112,113,114,115,116,117,118,119,120,121,122,123], comprising VR [109,111,113,114,116,123] or AR [110,112,115,117,118,119,120,121,122] data for educational research.

For datasets [120,121,123], although the applications are indirectly related to education, the datasets can contribute to user behavior analysis [120], enhancement of the simulation environment [121], and alignment of VR video cuts [123]. There are three types of accessibility for the benchmark datasets, namely open access [109,119,120,121,122,123], a reasonable request to the corresponding author [110,111,112,113,114,116,117,118], and a request by returning a completed form called an end user license agreement [115]. The first type best maximizes the download rate, whereas the other types facilitate better interactions between researchers (data owners and downloaders).

5. Conclusions

This paper explores the integration of serious games into education to enhance the quality of learning. The post-pandemic era has facilitated the adoption of innovative approaches to teaching and learning, including the use of gaming elements such as AR, VR, and MR. While VR dominates (81.8% among three reality approaches) in the literature review of 273 articles on developing serious games for innovative learning between 2007 and 27 July 2023, AR and MR possess unique characteristics that support different educational research. AR often provides extra information to users in real-world environments, whereas MR considers the interactions between digital and physical elements. Interactions between AR and MR enable the joint environment between computers, physical elements, and humans [124]. However, this increases the computing load for realistic environment simulation and deployment [125]. Our paper also provides an overview of typical guidelines and settings in software and hardware development for 15 AR/VR/MR standards and 15 benchmark datasets (six open-access datasets and nine datasets with reasonable requests to the corresponding authors) for conducting research studies. Although all authors devoted total effort to preparing this article, we observe limitations of the review processes, such as the search being restricted to the Scopus Database, only English articles being included, and journal articles being thoroughly analyzed in Table 4. The presented article is believed to provide a good systematic review of the latest developments in the design of serious games for innovative learning using AR/VR/MR.

Last but not least, the authors suggest several future research directions: (i) Apart from vision and hearing, consider the multisensory experiences (e.g., taste, smell, and touch) of the virtual environments, which enhances participant engagement. A review article of 105 articles revealed that 85% of them show a positive impact of multisensory VR experiences [126]. Another work [127] compared the sense of presence between audiovisual- and multisensory-based VR. The results found that multisensory experiences enhanced the sense of presence (from 4.37 to 4.44), in a participant size of 80. In the multisensory environment, the system complexity will be increased, and a larger space is required; (ii) employ generative artificial intelligence (e.g., data generation algorithms) to synthesize data and AR/VR/MR environments. This extends the data distribution from only ground truth information. The generative adversarial network has been a leading approach for data generation [128]. An augmented flow network was generated to support the AR- and VR-based ceramic art exhibitions to enhance visitors’ experience [129]. It took advantage of updating the design easily and frequently. It is a common issue of huge computational power requirements to render AR and MR information. A deep learning-based 3D point cloud generation was proposed to reduce the requirements [130]; (iii) enhance the learning experience with personalization to customize the learning environments and systems for individual users. One can consider balancing various factors, such as comfortability, excitability, and durability. An immersive VR neuro-learning platform was proposed to customize learning scenarios for medical education [131]. Both qualitative and quantitative revealed the effectiveness. In one paper [132], customized VR-based exercises (based on patients’ performance) were designed to enhance gait rehabilitation. The ease of the system was confirmed by three experienced physiotherapists (15–33 years of experience); and (iv) real-time interaction, engagement, and feedback between teachers and students to support multi-user using the same AR/VR/MR environment. In one paper [133], researchers studied the real-time interaction between teachers and students in construction teaching using VR from a teaching perspective. Another work [134] also focused on teaching perspective to design VR-based lectures to enhance teacher-student interaction and after-class activities. Further work is required to analyze the interaction from a student perspective.