Next Article in Journal
Strategies for Flipped Learning in the Health Professions Education in South Korea and Their Effects: A Systematic Review
Previous Article in Journal
Outdoor Adventure Education: Trends and New Directions—Introduction to a Special Collection of Research
Previous Article in Special Issue
Cloud-Based ICME Software Training
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Use of Augmented and Virtual Reality in Remote Higher Education: A Systematic Umbrella Review

Cyber-Physical Systems Laboratory, Institute of Electronics and Computer Science, LV-1006 Riga, Latvia
*
Author to whom correspondence should be addressed.
Educ. Sci. 2021, 11(1), 8; https://doi.org/10.3390/educsci11010008
Submission received: 26 November 2020 / Revised: 22 December 2020 / Accepted: 24 December 2020 / Published: 31 December 2020
(This article belongs to the Special Issue Digital Learning in Open and Flexible Environments)

Abstract

:
In this systematic umbrella review we aggregate the current knowledge of how virtual and augmented reality technologies are applicable to and impact remote learning in higher education; specifically, how they impact such learning outcomes as performance and engagement in all stages of higher education from course preparation to student evaluation and grading. This review was done as part of a state wide research effort of Latvia, to mitigate the impact of COVID-19 and specifically to provide a framework for a technological transformation of education in this context. In this work we search the Scopus and Web of Science databases for articles describing the use of virtual and/or augmented reality technologies in remote learning for higher education and their impact on learning outcomes. We identified 68 articles from which, after multiple screening and eligibility phases, nine review articles were left for extraction phase in which 30 structural elements with corresponding interventions and measured effects were extracted. Of these, 24 interventions had a measured effect on student performance (11 positive, seven negative, six no impact) and six interventions had a measured effect on student engagement (all six positive).

1. Introduction

The COVID-19 pandemic has highlighted the need for transformation of remote learning to not only survive a wave of crisis, but to potentially fit the new normal. A trend among governments across the world has been emerging to emphasize the potential for new technologies such as artificial intelligence and virtual/augmented reality to mitigate the problems remote learning has compared to on-site learning, such as academic dishonesty, decreased social aspects of studying, lack of practical kinesthetic interactions, problems keeping students’ attention, practice of technological boundaries, etc. As these are complex and expensive technologies, a decision for their use must be based not on technological hype but scientifically validated outcomes.
When it became clear that remote learning will have to be extended after the first wave, the government of The Republic of Latvia initiated a research programme in technological transformation of remote education. This is a 6 months long research programme providing a 500,000 EUR grant to an interdisciplinary team of researchers from multiple research institutions to evaluate how the Latvian society dealt with the coronavirus crisis and to provide recommendations for societal resilience in the future.This project has several work packages including study of societal dynamics in Latvia during this crisis, evaluation of labour market and employment structures, psychological effects of COVID-19 on individuals and families, evaluation of media and health communication, strategic communication and governance and finally education transformation. This work is part of this project, specifically the last work package and is aimed at finding evidence of the impact of virtual reality (VR) and augmented reality (AR) technologies on remote learning in higher education-specifically impact on performance and engagement. This is done through a systematic umbrella literature review-a review of literature reviews. This review conforms to PRISMA guidelines. Our research question is defined as "Which interventions using virtual and/or augmented reality technologies for students in higher education in remote learning have measured impact on student performance and engagement" and this question is defined according to PICOS approach in Table 1.

2. Methodology

In this section we describe the methodology of this systematic umbrella review (see flow diagram in Figure 1 for an overview).
This research conforms to PRISMA guidelines with the exception that the review protocol was not registered beforehand due to the time sensitive nature of the funding project related to the pandemic situation.

2.1. Identification

To identify the articles for inclusion in the review, a search was conducted in September, 2020 in two databases indexing peer-reviewed articles: Scopus and Web of Science. The scope was defined as “Use of virtual and/or augmented reality technologies in remote learning of higher education and their impact on learning outcomes”. For query results see Table 2.

2.2. Screening

After removing duplicates 66 articles were screened by reading their titles and abstracts. The screening criteria was:
  • only review articles are included;
  • only articles about higher education are included;
  • only articles using AR/VR technologies are included;
  • only articles about remote learning are included
During the screening process 36 articles were discarded leaving 30 articles for the Eligibility phase.
Out of these 36 excluded articles 16 articles were not review articles, 13 articles were not about AR/VR technologies, four articles were not available, two articles were not in English and one article was not about higher education.

2.3. Eligibility

During the eligibility phase, the articles were randomly distributed among the authors for a full text analysis. The eligibility criteria was as follows:
  • The article full text is available in English;
  • The article contains a review of multiple articles;
  • The article is about higher education;
  • The article is about remote learning;
  • The article is about AR/VR technologies.
In this stage 12 articles were excluded as ineligible and 18 articles were deemed eligible for inclusion and data extraction.
Out of these 12 excluded articles 10 were not review articles, one article was not available, and one article was not about higher education.

2.4. Included

Finally, the eligible articles were processed to extract all interventions that affect one of two variables: student performance or student engagement.
In this way 30 interventions were extracted. These interventions were then divided by the stage of the remote education process. The following stages were considered:
  • Course design, content planning;
  • Development of digital learning materials;
  • Cognitive load and time management;
  • Remote lecturing and content delivery;
  • Feedback and interactivity;
  • Social involvement, interaction;
  • Remote practice, labs, kinesthetic learning;
  • Remote evaluation.

3. Results

In total 68 articles were first identified, of which 66 were left after removing duplicates, 30 were left after the screening, 18 were left after eligibility checks and finally, 9 review articles were included in the final data extraction.
From those articles 30 structural elements and related interventions were extracted that contained AR or VR intervention in remote learning in higher education and had a measured impact on either performance or engagement.
The interventions related to augmented reality (AR) in these results are defined as interactive technology that allows to combine/complement/enhance real-world objects by computer-generated perceptual information on some sort of smart device, whule interventions related to virtual reality (VR) are defined as technology that allows to simulate real-world objects, events and interactions in digital computer generated domain/world/environment. Usually, VR term is used to describe virtual reality experience that could be obtained using VR-Headset, but in this paper VR term is used in a broader sense. The VR term also includes virtual laboratories on mobile devices or PC for experiments demonstration and performance.
In these interventions the impact on performance refers to observed change or no change to either the efficiency of accomplishing assignments, cumulative grade obtained during course or a metric that represents student ability to accomplish given task, using previously obtained knowledge. The impact on engagement refers to change or no change to the tendency of students to participate in study process, student enjoyment, satisfaction and feel of meaningfulness of ongoing process, willingness of students to attend classes, participate in in-class/after-class, course related, activities and interest inobtainment of additional materials.
Specifically 24 interventions described measured impact on performance (of those 11 had a positive impact, 7 had a negative impact and 6 had no effect) and 6 interventions described measured impact on engagement with all 6 reporting positive impact.
The education stage with the most identified interventions was “Remote practice, labs, kinesthetic learning”. There was one stage without any matching interventions-“Development of digital learning materials”. In 6 out of 9 reviewed papers VR/AR technologies were used in the field of medicine. Other fields where VR/AR technology was used were engineering, physics, chemistry. In two papers the field wasn’t specified and it was used for study purposes. The following percentages of review articles and original articles contained use of AR/VR technologies for the specific stages of remote educational process: Course design, content planning 11/2%, Development of digital learning materials 0/0%, Cognitive load and time management 11/8%, Remote lecturing and content delivery 11/6%, Feedback and interactivity 11/4%, Social involvement, interaction 11/8%, Remote practice, labs, kinesthetic learning 89/63% and Remote evaluation 11/8% respectively.
The identified interventions together with the intervention stage, intervention value, intervention effect, review article in which the intervention was identified and list of original articles supporting the intervention results can all be seen in the results in Table 3.

4. Discussion

In order to determine if AR/VR technologies might be beneficial to technological transformation of remote learning, in this article we describe an umbrella review of related literature.
The main limitations of the review are the inability to access 5 of the identified articles and inability to properly analyze 2 identified articles which were not in English.
The results show that most of the current experiments pertain to organizing laboratory or practical exercises within virtual or augmented reality in cases when physical presence is not feasible. This overall seems to provide positive results, except for a few cases [28,32,44]. In cases where practical, spatial or kinesthetic skills are required the results were very encouraging, especially in medicine related education [23,30,52].
In addition to the specific results extracted, the literature also suggests that virtual/augmented reality is not capable of completely replacing on site studies, because whenever it was tried, the student grades suffered [32,44].
As can be seen in Table 3 in multiple studies the mere fact of VR/AR usage already created an impact on performance or engagement. This could be explained by multiple mechanisms, the three more plausible ones are (a) either the AR/VR technologies actually impact the learning process directly, or (b) they impact the outcomes indirectly e.g., these technologies might improve social contact, which in turn improves overall outcomes or (c) the result might be due to a novelty and thus diminish in time as well as stop functioning if new novelty technique is introduced. The latter can only be distinguished if the same group of students is followed through several semesters.
The fact that in all interventions where engagement was measured, the engagement increased, leads us to speculate that novelty of technology used has a direct positive impact on engagement. If this is the case, it means that novelty itself is a potential intervention, and any newly hyped technology could provide similar results. If this is true then another question should be researched—whether there exists a cumulative novelty resistance and whether it accumulates for a person in general with any novelty, or just a subset. Does “acumulative novelty resistance”-the effect when introducing next new technology to study process with purpose of increasing the engagement and/or performance of students-have any effect due to satiation.
The possibility of such novelty requirements could lead to future experiments to determine the best way to keep the engagement and performance of students until the end of the study year.
In every study that showed increase of performance or engagement, the course was well designed and teachers had good qualification to use benefits of AR/VR for learning purposes, however, AR/VR is not a panacea. In cases when students or teachers were not familiar with AR/VR technologies or when courses were not adapted well for AR/VR usage or when teacher of the course was not prepared enough to work with AR/VR, a noteable decrease in performance was noted in the articles explored [2,3,4,5,6,28,32,44]. This leads to a highly vital conclusion-an unprepared teacher can’t prepare a student well.
The potential solution is:
  • create courses for teachers and lecturers on how to prepare/adapt courses for AR/VR;
  • create a framework that would allow teachers easily prepare/adopt their material for AR/VR;
  • Do not overload students with need to get familiar with AR/VR in a short time. there should be a possibility to use classical methods to get through the course;
At the same time AR/VR proved that it could help to understand abstract and complex content more easily due to good visualisation capabilities and interactivity. In multiple of the reviewed articles it was shown that kinesthetic learning, when instead of a classic lecture, students are working in 3D world, performing experiments alone or together with a teacher, is much more efficient than, previously mentioned, classic method [10,11,16,17,18,20,24,30,31].
The creation of AR/VR adopted courses could have a great effect on knowledge availability. An opinion in the educational community and society at large that has been reinfoced by the 2020 lockdown, is that online learning currently could be the future of education. If this is the case, then based on the fact that multiple papers show that AR/VR labs are of similar benefits as traditional “offline” labs with real equipment [24,26], it could be argued that properly adopted AR/VR based courses could, potentially, rise good, qualified specialists all around the globe, not only in local regions, democratizing education in hands on skills.
The performance is not the only factor that we need to take into account, emotional wellness is at least as important, as performance in terms of grades. Scientific groups that were researching Virtual Worlds as substitution for university environment showed that students feel much better if they could see their avatar in some virtual world, they could associate with, walk around virtual campus and explore it, like it would be real university [12,13].
It also must be noted that VR is still relatively complex and expensive technology and even though the prices are going down, still outfitting each student with VR/AR systems for remote learning is a complex and expensive task, which suggests that some of the future remote learning could happen from semi-centralized labs outfitted with VR/AR technologies, where students could arrive to work, but educators would connect remotely.

Author Contributions

Conceptualization, K.N. and V.A.; methodology, V.A.; validation, J.O. and A.M.; investigation, K.N., V.A., J.O. and A.M.; data curation, K.N.; writing—original draft preparation, K.N.; writing—review and editing, J.O. and A.M.; visualization, V.A.; supervision, K.N.; project administration, K.N.; funding acquisition, K.N. All authors have read and agreed to the published version of the manuscript.

Funding

This research has been supported by Latvian State Research Programme project No. VPP-COVID-2020/1-0013.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

All data generated or analysed during this study are included in this published article.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. Alzahrani, N. Augmented reality: A systematic review of its benefits and challenges in e-learning contexts. Appl. Sci. 2020, 10, 5660. [Google Scholar] [CrossRef]
  2. Alkhattabi, M. Augmented reality as e-learning tool in primary schools’ education: Barriers to teachers’ adoption. Int. J. Emerg. Technol. Learn. 2017, 12, 91–100. [Google Scholar] [CrossRef] [Green Version]
  3. Dunleavy, M.; Dede, C.; Mitchell, R. Affordances and limitations of immersive participatory augmented reality simulations for teaching and learning. J. Sci. Educ. Technol. 2009, 18, 7–22. [Google Scholar] [CrossRef]
  4. Chang, Y.J.; Chen, C.H.; Huang, W.T.; Huang, W.S. Investigating students’ perceived satisfaction, behavioral intention, and effectiveness of English learning using augmented reality. In Proceedings of the 2011 IEEE International Conference on Multimedia and Expo, Barcelona, Spain, 11–15 July 2011; pp. 1–6. [Google Scholar]
  5. Wu, H.K.; Lee, S.W.Y.; Chang, H.Y.; Liang, J.C. Current status, opportunities and challenges of augmented reality in education. Comput. Educ. 2013, 62, 41–49. [Google Scholar] [CrossRef]
  6. Wu, P.H.; Hwang, G.J.; Yang, M.L.; Chen, C.H. Impacts of integrating the repertory grid into an augmented reality-based learning design on students’ learning achievements, cognitive load and degree of satisfaction. Interact. Learn. Environ. 2018, 26, 221–234. [Google Scholar] [CrossRef]
  7. Bos, A.S.; Herpich, F.; Kuhn, I.; Guarese, R.L.; Tarouco, L.M.; Zaro, M.A.; Pizzato, M.; Wives, L. Educational Technology and Its Contributions in Students’ Focus and Attention Regarding Augmented Reality Environments and the Use of Sensors. J. Educ. Comput. Res. 2019, 57, 1832–1848. [Google Scholar] [CrossRef]
  8. Diegmann, P.; Schmidt-Kraepelin, M.; Eynden, S.; Basten, D. Benefits of augmented reality in educational environments-a systematic literature review. Benefits 2015, 3, 1542–1556. [Google Scholar]
  9. Niu, B.; Liu, C.; Liu, J.; Deng, Y.; Wan, Q.; Ma, N. Impacts of Different Types of Scaffolding on Academic Performance, Cognitive Load and Satisfaction in Scientific Inquiry Activities Based on Augmented Reality. In Proceedings of the 2019 Eighth International Conference on Educational Innovation through Technology (EITT), Biloxi, MS, USA, 27–31 October 2019; pp. 239–244. [Google Scholar]
  10. Quintero, E.; Salinas, P.; González-Mendívil, E.; Ramírez, H. Augmented reality app for calculus: A proposal for the development of spatial visualization. Procedia Comput. Sci. 2015, 75, 301–305. [Google Scholar] [CrossRef] [Green Version]
  11. Virata, R.O.; Castro, J.D.L. Augmented reality in science classroom: Perceived effects in education, visualization and information processing. In Proceedings of the 10th International Conference on E-Education, E-Business, E-Management and E-Learning, Tokyo, Japan, 10–13 January 2019; pp. 85–92. [Google Scholar]
  12. Kesim, M.; Ozarslan, Y. Augmented reality in education: Current technologies and the potential for education. Procedia Soc. Behav. Sci. 2012, 47, 297–302. [Google Scholar] [CrossRef] [Green Version]
  13. Giraudeau, P.; Olry, A.; Roo, J.S.; Fleck, S.; Bertolo, D.; Vivian, R.; Hachet, M. CARDS: A Mixed-Reality System for Collaborative Learning at School. In Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces, Daejeon, Korea, 10–13 November 2019; pp. 55–64. [Google Scholar]
  14. Chu, H.C.; Chen, J.M.; Hwang, G.J.; Chen, T.W. Effects of formative assessment in an augmented reality approach to conducting ubiquitous learning activities for architecture courses. In Universal Access in the Information Society; Springer: Berlin/Heidelberg, Germany, 2019; pp. 1–10. [Google Scholar]
  15. Pellas, N.; Fotaris, P.; Kazanidis, I.; Wells, D. Augmenting the learning experience in primary and secondary school education: A systematic review of recent trends in augmented reality game-based learning. Virtual Real. 2019, 23, 329–346. [Google Scholar] [CrossRef]
  16. Chien, C.H.; Chen, C.H.; Jeng, T.S. An interactive augmented reality system for learning anatomy structure. In Proceedings of the International Multiconference of Engineers and Computer Scientists. International Association of Engineers, Hong Kong, China, 17–19 March 2010; Volume 1, pp. 17–19. [Google Scholar]
  17. Santos, O.C.; Corbi, A. Can Aikido Help With the Comprehension of Physics? A First Step Towards the Design of Intelligent Psychomotor Systems for STEAM Kinesthetic Learning Scenarios. IEEE Access 2019, 7, 176458–176469. [Google Scholar] [CrossRef]
  18. Iqbal, M.Z.; Mangina, E.; Campbell, A.G. Exploring the use of Augmented Reality in a Kinesthetic Learning Application Integrated with an Intelligent Virtual Embodied Agent. In Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Beijing, China, 10–18 October 2019; pp. 12–16. [Google Scholar]
  19. Vosinakis, S.; Koutsabasis, P. Interaction design studio learning in virtual worlds. Virtual Real. 2013, 17, 59–75. [Google Scholar] [CrossRef] [Green Version]
  20. Dalgarno, B.; Lee, M.J. What are the learning affordances of 3-D virtual environments? Br. J. Educ. Technol. 2010, 41, 10–32. [Google Scholar] [CrossRef]
  21. Hansen, M. Versatile, immersive, creative and dynamic virtual 3-D healthcare learning environments: A review of the literature. J. Med. Internet Res. 2008, 10. [Google Scholar] [CrossRef]
  22. Byrne, C.M. Water on Tap: The Use of Virtual Reality as an Educational Tool. Ph.D. Thesis, University of Washington, Seattle, DC, USA, 1996. [Google Scholar]
  23. Dev, P.; Youngblood, P.; Heinrichs, W.; Kusumoto, L. Virtual Worlds and Team Training. Anesthesiol. Clin. 2007, 25, 321–336. [Google Scholar] [CrossRef]
  24. Youngblood, P.; Srivastava, S.; Harter, P.; Dev, P.; Wallen, C.; Tsai, L.; Heinrichs, W. Validation of a web-based VR simulation for training trauma teams. In Proceedings of the MMVR, Long Beach, CA, USA, 24–29 January 2005. [Google Scholar]
  25. De Jong, T.; Linn, M.; Zacharia, Z. Physical and virtual laboratories in science and engineering education. Science 2013, 340, 305–308. [Google Scholar] [CrossRef] [Green Version]
  26. Wiesner, T.F.; Lan, W. Comparison of student learning in physical and simulated unit operations experiments. J. Eng. Educ. 2004, 93, 195–204. [Google Scholar] [CrossRef]
  27. Kyaw, B.; Posadzki, P.; Paddock, S.; Car, J.; Campbell, J.; Tudor Car, L. Effectiveness of digital education on communication skills among medical students: Systematic review and meta-analysis by the digital health education collaboration. J. Med. Internet Res. 2019, 21. [Google Scholar] [CrossRef] [Green Version]
  28. Deladisma, A.M.; Cohen, M.; Stevens, A.; Wagner, P.; Lok, B.; Bernard, T.; Oxendine, C.; Schumacher, L.; Johnsen, K.; Dickerson, R.; et al. Do medical students respond empathetically to a virtual patient? Am. J. Surg. 2007, 193, 756–760. [Google Scholar] [CrossRef]
  29. Kaltman, S.; Talisman, N.; Pennestri, S.; Syverson, E.; Arthur, P.; Vovides, Y. Using technology to enhance teaching of patient-centered interviewing for early medical students. Simul. Healthc. 2018, 13, 188–194. [Google Scholar] [CrossRef]
  30. Mahmood, T.; Scaffidi, M.; Khan, R.; Grover, S. Virtual reality simulation in endoscopy training: Current evidence and future directions. World J. Gastroenterol. 2018, 24, 5439–5445. [Google Scholar] [CrossRef] [PubMed]
  31. Tuggy, M.L. Virtual reality flexible sigmoidoscopy simulator training: Impact on resident performance. J. Am. Board Fam. Pract. 1998, 11, 426–433. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  32. Gerson, L.; Van Dam, J. A prospective randomized trial comparing a virtual reality simulator to bedside teaching for training in sigmoidoscopy. Endoscopy 2003, 35, 569–575. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  33. Sedlack, R.E.; Kolars, J.C.; Alexander, J.A. Computer simulation training enhances patient comfort during endoscopy. Clin. Gastroenterol. Hepatol. 2004, 2, 348–352. [Google Scholar] [CrossRef]
  34. Sedlack, R.E.; Kolars, J.C. Computer simulator training enhances the competency of gastroenterology fellows at colonoscopy: Results of a pilot study. Am. J. Gastroenterol. 2004, 99, 33–37. [Google Scholar] [CrossRef]
  35. Ahlberg, G.; Hultcrantz, R.; Jaramillo, E.; Lindblom, A.; Arvidsson, D. Virtual reality colonoscopy simulation: A compulsory practice for the future colonoscopist? Endoscopy 2005, 37, 1198–1204. [Google Scholar] [CrossRef]
  36. Cohen, J.; Cohen, S.A.; Vora, K.C.; Xue, X.; Burdick, J.S.; Bank, S.; Bini, E.J.; Bodenheimer, H.; Cerulli, M.; Gerdes, H.; et al. Multicenter, randomized, controlled trial of virtual-reality simulator training in acquisition of competency in colonoscopy. Gastrointest. Endosc. 2006, 64, 361–368. [Google Scholar] [CrossRef]
  37. Park, J.; MacRae, H.; Musselman, L.J.; Rossos, P.; Hamstra, S.J.; Wolman, S.; Reznick, R.K. Randomized controlled trial of virtual reality simulator training: Transfer to live patients. Am. J. Surg. 2007, 194, 205–211. [Google Scholar] [CrossRef]
  38. Yi, S.Y.; Ryu, K.H.; Na, Y.J.; Woo, H.S.; Ahn, W.; Kim, W.S.; Lee, D.Y. Improvement of colonoscopy skills through simulation-based training. Stud. Health Technol. Inf. 2008, 132, 565. [Google Scholar]
  39. Haycock, A.; Koch, A.D.; Familiari, P.; van Delft, F.; Dekker, E.; Petruzziello, L.; Haringsma, J.; Thomas-Gibson, S. Training and transfer of colonoscopy skills: A multinational, randomized, blinded, controlled trial of simulator versus bedside training. Gastrointest. Endosc. 2010, 71, 298–307. [Google Scholar] [CrossRef]
  40. McIntosh, K.S.; Gregor, J.C.; Khanna, N.V. Computer-based virtual reality colonoscopy simulation improves patient-based colonoscopy performance. Can. J. Gastroenterol. Hepatol. 2014, 28, AB364. [Google Scholar] [CrossRef] [PubMed]
  41. Gomez, P.P.; Willis, R.E.; Van Sickle, K. Evaluation of two flexible colonoscopy simulators and transfer of skills into clinical practice. J. Surg. Educ. 2015, 72, 220–227. [Google Scholar] [CrossRef] [PubMed]
  42. Grover, S.C.; Garg, A.; Scaffidi, M.A.; Jeffrey, J.Y.; Plener, I.S.; Yong, E.; Cino, M.; Grantcharov, T.P.; Walsh, C.M. Impact of a simulation training curriculum on technical and nontechnical skills in colonoscopy: A randomized trial. Gastrointest. Endosc. 2015, 82, 1072–1079. [Google Scholar] [CrossRef] [PubMed]
  43. Grover, S.C.; Scaffidi, M.A.; Khan, R.; Garg, A.; Al-Mazroui, A.; Alomani, T.; Jeffrey, J.Y.; Plener, I.S.; Al-Awamy, M.; Yong, E.L.; et al. Progressive learning in endoscopy simulation training improves clinical performance: A blinded randomized trial. Gastrointest. Endosc. 2017, 86, 881–889. [Google Scholar] [CrossRef] [PubMed]
  44. Sedlack, R.E. Validation of computer simulation training for esophagogastroduodenoscopy: Pilot study. J. Gastroenterol. Hepatol. 2007, 22, 1214–1219. [Google Scholar] [CrossRef] [PubMed]
  45. Di Giulio, E.; Fregonese, D.; Casetti, T.; Cestari, R.; Chilovi, F.; D’Ambra, G.; Di Matteo, G.; Ficano, L.; Delle Fave, G. Training with a computer-based simulator achieves basic manual skills required for upper endoscopy: A randomized controlled trial. Gastrointest. Endosc. 2004, 60, 196–200. [Google Scholar] [CrossRef]
  46. Shirai, Y.; Yoshida, T.; Shiraishi, R.; Okamoto, T.; Nakamura, H.; Harada, T.; Nishikawa, J.; Sakaida, I. Prospective randomized study on the use of a computer-based endoscopic simulator for training in esophagogastroduodenoscopy. J. Gastroenterol. Hepatol. 2008, 23, 1046–1050. [Google Scholar] [CrossRef]
  47. Ferlitsch, A.; Schoefl, R.; Puespoek, A.; Miehsler, W.; Schoeniger-Hekele, M.; Hofer, H.; Gangl, A.; Homoncik, M. Effect of virtual endoscopy simulator training on performance of upper gastrointestinal endoscopy in patients: A randomized controlled trial. Endoscopy 2010, 42, 1049–1056. [Google Scholar] [CrossRef] [Green Version]
  48. Ende, A.; Zopf, Y.; Konturek, P.; Naegel, A.; Hahn, E.G.; Matthes, K.; Maiss, J. Strategies for training in diagnostic upper endoscopy: A prospective, randomized trial. Gastrointest. Endosc. 2012, 75, 254–260. [Google Scholar] [CrossRef]
  49. Lim, B.S.; Leung, J.W.; Lee, J.; Yen, D.; Beckett, L.; Tancredi, D.; Leung, F.W. Effect of ERCP mechanical simulator (EMS) practice on trainees’ ERCP performance in the early learning period: US multicenter randomized controlled trial. Am. J. Gastroenterol. 2011, 106, 300–306. [Google Scholar] [CrossRef]
  50. Liao, W.C.; Leung, J.W.; Wang, H.P.; Chang, W.H.; Chu, C.H.; Lin, J.T.; Wilson, R.E.; Lim, B.S.; Leung, F.W. Coached practice using ERCP mechanical simulator improves trainees’ ERCP performance: A randomized controlled trial. Endoscopy 2013, 45, 799–805. [Google Scholar] [CrossRef] [PubMed]
  51. Meng, W.; Leung, J.W.; Yue, P.; Wang, Z.; Wang, X.; Wang, H.; Zhu, K.; Zhang, L.; Zhou, W.; Li, X. Sa1238 Practice With ERCP Mechanical Simulator (EMS) Improves Basic ERCP Skills of Novice Surgical Trainees. Gastrointest. Endosc. 2016, 83, AB267–AB268. [Google Scholar] [CrossRef]
  52. De Souza-Junior, V.; Mendes, I.; Marchi-Alves, L.; Jackman, D.; Wilson-Keates, B.; De Godoy, S. Peripheral Venipuncture Education Strategies for Nursing Students: An Integrative Literature Review. J. Infus. Nurs. 2020, 43, 24–32. [Google Scholar] [CrossRef] [PubMed]
  53. Loukas, C.; Nikiteas, N.; Kanakis, M.; Georgiou, E. Evaluating the effectiveness of virtual reality simulation training in intravenous cannulation. Simul. Healthc. 2011, 6, 213–217. [Google Scholar] [CrossRef]
  54. Ryall, T.; Judd, B.; Gordon, C. Simulation-based assessments in health professional education: A systematic review. J. Multidiscip. Healthc. 2016, 9, 69–82. [Google Scholar] [CrossRef] [Green Version]
  55. Bick, J.S.; DeMaria, S., Jr.; Kennedy, J.D.; Schwartz, A.D.; Weiner, M.M.; Levine, A.I.; Shi, Y.; Schildcrout, J.S.; Wagner, C.E. Comparison of expert and novice performance of a simulated transesophageal echocardiography examination. Simul. Healthc. 2013, 8, 329–334. [Google Scholar] [CrossRef]
  56. Lipner, R.S.; Messenger, J.C.; Kangilaski, R.; Baim, D.S.; Holmes, D.R., Jr.; Williams, D.O.; King, S.B., III. A technical and cognitive skills evaluation of performance in interventional cardiology procedures using medical simulation. Simul. Healthc. 2010, 5, 65–74. [Google Scholar] [CrossRef]
  57. Grantcharov, T.P.; Carstensen, L.; Schulze, S. Objective assessment of gastrointestinal endoscopy skills using a virtual reality simulator. JSLS J. Soc. Laparoendosc. Surg. 2005, 9, 130. [Google Scholar]
  58. Hawkins, R.; MacKrell Gaglione, M.; LaDuca, T.; Leung, C.; Sample, L.; Gliva-McConvey, G.; Liston, W.; De Champlain, A.; Ciccone, A. Assessment of patient management skills and clinical skills of practising doctors using computer-based case simulations and standardised patients. Med. Educ. 2004, 38, 958–968. [Google Scholar] [CrossRef]
Figure 1. Systematic review flow chart.
Figure 1. Systematic review flow chart.
Education 11 00008 g001
Table 1. Research question of our umbrella review defined according to PICOS approach.
Table 1. Research question of our umbrella review defined according to PICOS approach.
Ppatientsstudents in higher education
Iinterventionuse of virtual and/or augmented reality technologies in remote learning
Ccomparisonnone
Ooutcomeimpact on student performance and engagement
Sstudy designsystematic reviews
Table 2. Identification search queries and results.
Table 2. Identification search queries and results.
DatabaseQueryResults
ScopusTITLE-ABS-KEY ((“Virtual reality” OR “Augmented reality”) AND (((online OR distance OR remote) AND (study OR education OR learning)) OR e-learning) AND students) AND PUBYEAR > 1999 AND (LIMIT-TO (DOCTYPE, “re”))66
Web of ScienceTOPIC: ((“Virtual reality” OR “Augmented reality”)) AND TOPIC: (students) AND TOPIC: ((“e-learning” OR ((“online” OR “distance” OR “remote”) AND (“study” OR “education” OR “learning”)))) AND YEAR PUBLISHED: (>1999)2
Table 3. Extracted structural elements with corresponding interventions and measured effect.
Table 3. Extracted structural elements with corresponding interventions and measured effect.
No.StageInterventionVariableEffectReview ArticleOriginal Articles
1Course design, content planningUse of AR technologies with insufficient pedagogue  trainingPerformanceDecrease[1][2]
Development of digital learning materials-----
2Cognitive load and time managementUsage of complex AR simulations for students, who are not familiar with this complex technology, leading to confusion and astoundmentPerformanceDecrease[1][3]
3Cognitive load and time managementUse of AR with insufficient support that can confuse learners and delay the learning processPerformanceDecrease[1][4]
4Cognitive load and time managementUnassisted AR experience with high load/complex course leading to cognitive overloadPerformanceDecrease[1][3,5,6]
5Remote lecturing and content deliveryUse of AR in lecturing and content delivery improves focus, attention levels, study process becomes more enjoyable, fun and satisfyingEngagementIncrease[1][7,8,9]
6Feedback and interactivityUse of AR for calculus and abstract concept visualisation promotes mathematical and cognitive skills in engineering studentsPerformanceIncrease[1][10,11]
7Social involvement, interactionUse of AR for better face-to-face and remote interactions and collaborationsEngagementIncrease[1][12]
8Social involvement, interactionUse of AR enables interactions and collaborations which are more similar to natural face-to-face collaboration than screen-based interactionEngagementIncrease[1][13]
9Social involvement, interactionUse of AR in academic settings improces learners’ motivation and engagement, especially when game-based approaches are utilizedEngagementIncrease[1][14,15]
10Remote practice, labs, kinesthetic learningUse of AR had positive influence on learning rate and memorization process of medical studentsPerformanceIncrease[1][16]
11Remote practice, labs, kinesthetic learningUse of AR increases motivation, engagement, interest and knowledge retentionEngagementIncrease[1][16,17,18]
12Remote practice, labs, kinesthetic learningUse of virtual worlds promotes student motivation and engagmentEngagementIncrease[19][20]
13Remote practice, labs, kinesthetic learningUse of virtual worlds promotes spatial knowledge and capability to transfer the knowledge to real world skillsPerformanceIncrease[19][20]
14Remote practice, labs, kinesthetic learningUse of VR for interactive presentation and visualization of complex physical experiments has positive effect on learning processPerformanceIncrease[21][22]
15Remote practice, labs, kinesthetic learningWirtual worlds are as effective for learning as the more traditional Human Patient SimulatorPerformanceIncrease[23][24]
16Remote practice, labs, kinesthetic learningUse of virtual labs in physics and chmeistry is as efficient as traditional labsPerformanceNo change[25][26]
17Remote practice, labs, kinesthetic learningUse of virtual partner simulation for medical students reduced performancePerformanceDecrease[27][28]
18Remote practice, labs, kinesthetic learningUse of virtual partner simulation for medical students didn’t change the performancePerformanceNo change[27][29]
19Remote practice, labs, kinesthetic learningUse of VR for medical students leads to faster mean completion time, lower directional error in Flexible SigmoidoscopyPerformanceIncrease[30][31]
20Remote practice, labs, kinesthetic learningUse of VR simulation for medical students reduces the mean score and the number of individually completed retroflexion casesPerformanceDecrease[30][32]
21Remote practice, labs, kinesthetic learningUse of VR simulation for medical students did not change the average task time and patient satisfactionPerformanceNo change[30][32]
22Remote practice, labs, kinesthetic learninguse of VR simulation for medicine students in flexible sigmoidoscopy increases patient comfort levelPerformanceIncrease[30][33]
23Remote practice, labs, kinesthetic learningUse of VR simulation for medical students in flexible sigmoidoscopy did not change procedural skills such as independence, identifying pathology, landmarks, performing biopsies, adequate visualizationPerformanceNo effect[30][33]
24Remote practice, labs, kinesthetic learningUse of VR simulation for medical students improves colonoscopy capabilitiesPerformanceIncrease[30][34,35,36,37,38,39,40,41,42,43]
25Remote practice, labs, kinesthetic learningUse of VR simulation for medical students made student Esophagogastroduodenoscopy capability worsePerformanceDecrease[30][44]
26Remote practice, labs, kinesthetic learningUse of VR simulation for medical students improved student Esophagogastroduodenoscopy capabilityPerformanceIncrease[30][45,46,47,48]
27Remote practice, labs, kinesthetic learningUse of VR simulation for medical students improved endoscopic retrograde cholangiopancreatography (ERCP) capabilitiesPerformanceIncrease[30][49,50,51]
28Remote practice, labs, kinesthetic learningUse of VR simulation for students - nurses shows that after investment in training new intermediary students and a group of experts had equivalent performancePerformanceIncrease[52][53]
29Remote evaluationUse of VR in remote evaluation is able to discriminate between expert and novice performersPerformanceNo change[54][55,56,57]
30Remote evaluationComputer based simulations and virtual standard patient examinations were unable to distinguish between different exeprience levelsPerformanceNo change[54][58]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Nesenbergs, K.; Abolins, V.; Ormanis, J.; Mednis, A. Use of Augmented and Virtual Reality in Remote Higher Education: A Systematic Umbrella Review. Educ. Sci. 2021, 11, 8. https://doi.org/10.3390/educsci11010008

AMA Style

Nesenbergs K, Abolins V, Ormanis J, Mednis A. Use of Augmented and Virtual Reality in Remote Higher Education: A Systematic Umbrella Review. Education Sciences. 2021; 11(1):8. https://doi.org/10.3390/educsci11010008

Chicago/Turabian Style

Nesenbergs, Krisjanis, Valters Abolins, Juris Ormanis, and Artis Mednis. 2021. "Use of Augmented and Virtual Reality in Remote Higher Education: A Systematic Umbrella Review" Education Sciences 11, no. 1: 8. https://doi.org/10.3390/educsci11010008

APA Style

Nesenbergs, K., Abolins, V., Ormanis, J., & Mednis, A. (2021). Use of Augmented and Virtual Reality in Remote Higher Education: A Systematic Umbrella Review. Education Sciences, 11(1), 8. https://doi.org/10.3390/educsci11010008

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop