Next Article in Journal
Physics-Based Watercraft Simulator in Virtual Reality
Previous Article in Journal
Integration of Immersive Approaches for Collaborative Processes with Building Information Modeling (BIM) Methodology for the AEC Industry: An Analysis of the Current State and Future Challenges
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Virtual Worlds
Volume 2
Issue 4
10.3390/virtualworlds2040023
virtualworlds-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

360-Degree Virtual Reality Utilising Head-Mounted Devices in Undergraduate Nursing and Midwifery Education: A Scoping Review

by
Maram A. Alammary
1,2,
Lesley Halliday
1 and
Stathis Th. Konstantinidis
1,*
1
School of Health Science, University of Nottingham, Nottingham NG7 2HA, UK
2
Maternity and Child Health Nursing Department, College of Nursing, Princess Nourah bint Abdulrahman University, Riyadh 13412, Saudi Arabia
*
Author to whom correspondence should be addressed.
Virtual Worlds 2023, 2(4), 396-421; https://doi.org/10.3390/virtualworlds2040023
Submission received: 24 July 2023 / Revised: 4 November 2023 / Accepted: 4 December 2023 / Published: 7 December 2023
Browse Figures
Versions Notes

Abstract

:
Immersive Virtual Reality (IVR) is a promising tool for improving the teaching and learning of nursing and midwifery students. However, the preexisting literature does not comprehensively examine scenario development, theoretical underpinnings, duration, and debriefing techniques. The aim of this review was to assess the available evidence of how 360-degree Virtual Reality (VR) utilising head-mounted devices has been used in undergraduate nursing and midwifery education programmes and to explore the potential pedagogical value based on Kirkpatrick’s evaluation model. This review followed the Joanna Briggs Institute (JBI) methodology. A comprehensive electronic search was conducted across five databases. All studies published in English between 2007–2022 were included, regardless of design, if the focus was undergraduate nursing and midwifery programmes and utilised fully immersive 360-degree VR scenarios. Out of an initial pool of 1700 articles, 26 were selected for final inclusion. The findings indicated a limited diversity in scenario design, with only one study employing a participatory approach. Within the Kirkpatrick model, the most measurable outcomes were found at level 2. The main drawback observed in interventional studies was the absence of a theoretical framework and debriefing. The review concludes that the increased use of fully IVR in nursing education has improved student learning outcomes; however, published literature on midwifery education is scarce.

1. Introduction

The changing needs of modern learners, as well as an increasing number of learning styles and preferences, have prompted the need for new methods and approaches to education [1]. Nursing education involving the use of simulation is relatively new and presents an active learning method, allowing students to develop their clinical skills and experience in augmented real-life scenarios [2]. Various forms of simulation teaching methods are currently available, such as peer-to-peer learning, computer simulations, and Virtual Reality (VR), including VR Simulation (VRS), which is one form of simulation that can be used as a novel method of teaching complex skills in a safe environment [3].
Virtual Reality falls under the umbrella term “Extended Reality” (XR). XR is a comprehensive phrase that refers to technologies that combine aspects from both the real and virtual worlds [4]. Currently, VR, AR (augmented reality), and MR (mixed reality) are well-known technologies in this domain. MR is defined as the combination of the real and virtual worlds, whereas AR is adding digital data into real-world scenarios [4].

1.1. Immersive Virtual Reality Simulation

The Society for Simulation in Healthcare [5] defines VR as “the use of computer technology to create an interactive 3D world in which the objects have a sense of spatial presence” (p. 55). However, educators are frequently confused about what VR is and how it differs from three-dimensional (3D) virtual worlds. A true VR medium is distinguished by utilising a Head-Mounted Display (HMD). This technology can provide users with the sensation of being “there” [4] by creating a user-centred 3D immersive environment [6]. This device is crucial to the creation of an immersive experience, as it transports the user’s visual and auditory senses into a virtual environment [6]. HMDs enable users to view different sections of the VR sphere by simply moving their head.
The use of a HMD is a distinctive feature of authentic, immersive VR, providing a high level of immersion and engagement that distinguishes it from other available technologies [7]. If an HMD is not used, this is termed “non-immersive VR”, as scenarios are typically viewed on a computer or television screen. [7]. This paper is focused on the use of fully Immersive Virtual Reality (IVR). The hallmarks of VR technology are immersion, presence, and interactivity. These features have greater potential in VR than in other forms of information technology [4] and can enable learners to become more actively involved in their learning. This can facilitate faster, more accurate, and easier cognitive engagement, which subsequently translates into improved learning for clinical practice [7].
Fully IVR is a promising tool for improving the teaching and learning experiences of Nursing and Midwifery Students (NMSs). When using IVR, students can fully engage through all their senses, improving clinical judgement and critical thinking [8]. In medical fields, IVR use is increasing due to its ability to provide a variety of simulation scenarios that can be used by students without compromising patient safety [9].

1.2. 360-Degree Video

Videos are one of the most utilised media types in education [10]. However, despite the numerous advantages of videos, such as improved understanding and enhanced collaboration [11], they lack certain features that can enhance effectiveness and interest. For example, videos are only shown from one angle, and viewers are usually passive observers, whereas 360-degree videos can be shown from various angles and perspectives [12] and can be viewed on various devices such as smartphones, tablets, and computers (in non-immersive formats), or as part of an IVR experience via an HMD. HMD IVR packages are becoming increasingly popular in educational applications, as they enable curricula and content to be developed and edited for specific educational purposes [13]. In medical education, the use of novel VR modalities, such as mobile VR, is limited. According to a recent review, most studies (60.5%) have used commercially available surgical VR simulators, whereas only 3.5% of studies used mobile VR.
Although immersive 360-degree videos are often referred to under the umbrella term “VR”, the VR per se is created entirely by software, whereas 360-degree videos usually deploy real-world footage [14].
A recent scoping review explored the used of IVR in tertiary Nursing and Midwifery Education (N and ME) [15] and revealed only two articles, one containing empirical data and the other a concept development article. This was the only related scoping review identified, although two recent systematic reviews were identified that measured the effectiveness and outcomes of IVR on nursing students’ learning [16,17]. However, these reviews were limited to specific types of study designs such as RCTs or quasi-experimental studies. Saying that, both supported the quality of IVR in nursing education and the need to further explore the educational appropriateness of VRS programmes [18].
Due to the novel nature of 360-degree VR video, the evidence base regarding their use in educational settings has been relatively limited [7,19]. Similarly, little is known about the scope and implementation of IVR in N and ME [15]. The importance of simulation in nursing and midwifery education arises from the need for students to have the opportunity to “practice” complex clinical skills in a safe environment. In these disciplines, where high-stakes scenarios and complex interactions are prevalent, immersive learning experiences can have a significant impact [16]. Further research is required to establish the appropriate standards for IVR use in this context [14,19]. Given the dearth of research focused on immersive 360-degree video, the aim of this review was to map the available evidence in undergraduate NMSs in nursing education programmes as well as to explore the pedagogical value based on Kirkpatrick’s evaluation model. This review will provide a valuable insight of IVR and may guide further research on this topic.

2. Review Question

How is immersive 360-degree VR used in undergraduate nursing and midwifery education?

3. Inclusion Criteria

Studies were considered eligible for inclusion if they involved undergraduate NMSs, using fully immersive 360-degree VR scenarios projected by HMD with or without haptics in any educational setting. Both nursing and midwifery students were included, as many universities use integrated curriculums for these groups [20]. All included studies were published after 2006 when the use of virtual simulation in nursing became an accessible pedagogical approach [21]. All studies had been peer reviewed; the full texts were available and included all research designs.

4. Methods

This review utilised the Joanna Briggs Institute (JBI) framework, which recommends clarification of the main concepts in the research question using the “Population, Concept, and Context” (PCC)—a nine-step framework for scoping reviews [22]. A review protocol was not published.

4.1. Search Strategy

An electronic search of MEDLINE (Ovid), Embase (Ovid), CINAHL (EBSCO), Eric, and Scopus was completed between 6 July 2022 and 18 September 2022. A systematic search technique was created using all identified keywords and index terms in the titles and abstracts. Additionally, the reference lists of all included articles were screened to identify further studies. A research librarian was consulted for feedback on the search strategy. The search strategy for MEDLINE can be found in Table S1 (Supplement File S1).

4.2. Selection of Sources

All identified citations were uploaded into the reference manager EndNote X9.3.3 (Clarivate Analytics, Philadelphia, PA, USA), and duplicates were removed. Titles and abstracts were screened for compliance with the inclusion criteria. There were no limitations on research design because the scoping review focused on the broad use of VR in N and ME. Full texts of relevant studies were subsequently retrieved and reviewed.

4.3. Data Extraction

A data extraction table was developed based on the categories outlined in the JBI guidance [22]: author(s), country, population, research design, educational intervention and time, learning theories, debriefing, and main results.

4.4. Analysis and Presentation of Results

The data were presented in tabular format followed by a descriptive analysis. Narrative analysis was applied to discuss and integrate study results and to explore the implementation of VR to N and ME.

5. Results

5.1. Search Results

The search process resulted in 1700 articles. After 314 duplicate articles were removed, 1386 articles remained. Titles and abstracts were screened for relevancy, yielding 153 full-text articles for eligibility assessment. Following the application of exclusion criteria, 127 articles were excluded with reasons, and 26 eligible articles were considered directly related to the research question. The full texts of these articles were then reviewed as outlined in the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) flow diagram [23] (Figure 1).

5.2. Characteristics of Included Studies

The 26 studies included in this review were conducted in nine countries: the US (n = 8), Republic of Korea (n = 6), Taiwan (n = 4), Ireland (n = 2), and others (n = 6) [6,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49] (Table 1). None of the included studies were conducted in developing countries or the Middle East. The studies were published between 2018 to 2022, with most studies (n = 9) conducted in 2022. Eleven out of 26 studies used quasi-experimental designs, and the remainder used Mixed Methods Research (MMR; n = 3), qualitative descriptive (n = 3), feasibility MMR (n = 2), RCTs (n = 2), descriptive study (n = 2), crossover (n = 1), cross-sectional (n = 1), and action research (n = 1) designs. The population used in all studies were undergraduate nursing and midwifery students (n = 1205) and were recruited using convenience sampling Recruitment approaches included announcements through web pages and social networking media. Several studies discussed user experiences with the intervention. One study [35] stated that participants were included if they had no prior experience of education related to the intervention topic. Other studies required clinical training experience as an inclusion criterion [28,30,47]. However, most studies provided theoretical lectures before implementing the VR intervention [29,31,32,34,35,42,43,44,46,47,49].
Researchers who were concerned about the influence of past VR experiences on study results excluded participants who had previously experienced VR [26,36,47]. In addition, participants who were feeling ill, had pre-existing binocular vision abnormalities, or a history of severe motion sickness were also excluded (to prevent motion sickness) [40,42]. Table 2 displays the article counts for the characteristics of the included studies.

5.3. Characteristics of Interventions

5.3.1. Scenario Development

Ten studies identified the intervention development process of the scenario content (Table 3). Some studies were based on a review of relevant literature, whereas others used subject matter experts to develop the scenario. Only one study indicated that they applied a participatory design (co-creation) and engaged students in developing the scenario [35].
Studies explored effectiveness in nurse skill acquisition and training competence in several areas, including disaster skills of decontamination [43,44], Nasogastric Tube (NGT) feeding [30], catheter insertion [26], airway management [41], and wound dressing [31]. Additionally, two papers explored students’ experience in anatomy and physiology of the human body [41] and empathy [38]. In the final papers, only five IVR scenarios were related to midwifery and included neonatal resuscitation [47], a non-stress test [29], childbirth [27], high-risk neonatal infection control [48], and the baby’s life inside the mother’s womb [34].

5.3.2. Interactivity

Nineteen scenarios were interactive, requiring students to perform a specific skill with the controllers, such as NGT care [28] or infection control [49]. In other research, participants were asked to select the correct response from a series of options presented onscreen [37,45]. In contrast, the remaining designs employed were passive, requiring students to observe the scenario without interaction [6,27,32,33,34,36,38]. Table 4 displays the various forms of interactivity within the studies.

5.3.3. Duration

There was limited consistency among the IVR applications in relation to duration and frequency. The shortest IVR experience lasted for 8 min, whereas the longest lasted for an hour (Table 5). Only 20 of the included studies provided data on average duration (Mean = 22.5; SD = 17.4) (Table 1).

5.3.4. Types of HMD

The Included papers used fully immersive VR through an Oculus Rift (n = 7), HTC VIVE (n = 3), Oculus Go (n = 2), Oculus Quest (n = 1), and not specified (n = 5). Seven studies used phone VR headsets with 360-degree video simulations [6,27,33,34,36,37,39]. A single study [30] used AR simulation with VIVEPAPER HTC to provide the sensation of 360-degree video.

5.3.5. Comparators

A control group was used in an RCT [30], a quasi-experimental study [27], and an MMR [38]. Six quasi-experimental designs [6,26,33,43,48,49] applied routine practice, such as lab training or hospital rotation. The other RCT [39] used traditional workshops, and the other quasi-experimental designs [35,36,47] provided theoretical approaches. Furthermore, MMR used written instructions for the control group [32,44].

5.3.6. Theoretical Framework

Several theoretical frameworks and models were employed, with the NLN/Jeffries Simulation framework [3] (n = 5) being the most prevalent. Other studies (n = 4) utilised Kolb’s [50] experiential learning theory or the theory’s extension model. In addition, Bandura’s various theories, including the social learning theory [51] and self-efficacy theory [52], were utilised. However, eight studies did not have an underpinning theoretical framework [25,31,36,37,38,39,40,41,45] (Table 6).

5.3.7. Debriefing

The duration of debriefing was specified as 20 min in three studies [35,48,49], with others failing to include details of duration or the type of debriefing techniques employed. Thus, participants in a large number of research studies (n = 16) cannot be assumed to have received a debriefing or feedback after their experience.

5.3.8. Retention, Acceptability, and Effectiveness

Only one RCT reported drop-out rates [39], stating that the number of participants who withdrew from the study were similar in the intervention (n = 3, 1.5%) and control (n = 3, 1.5%) groups. Furthermore, fully implemented IVR interventions appeared to demonstrate significant support for acceptability and/or effectiveness. Nevertheless, two studies [33,43] concluded that there were no statistically significant differences between the intervention and control groups, as both exhibited improved knowledge over time, suggesting that IVR is just as beneficial as High-Fidelity Simulation (HFS).

5.3.9. Adverse Effects

Six articles found that participants who experienced fully IVR felt motion sickness. Nonetheless, participants in one study were permitted to remain seated to prevent negative consequences [40].

5.4. Characteristics of Outcome Measures

The most-used validated outcome measure relating to VRS use was the System Usability Scale (SUS). The SUS is a simple ten-item scale that is considered highly reliable (a = 0.91). Choi [31] utilised the Simulator Usability Questionnaire (SUQ) [53] and User Acceptance Questionnaire (UAQ). SUQ evaluates usability, ease of use, and ease of learning. UAQ covers aspects of the Technology Acceptance Model (TAM) [54]. The validity of both instruments was not reported.
Thompson [46] used the Perceived Engagement Questionnaire (PEQ) to compare participants’ engagement with VR and current teaching modality. Furthermore, the VR Learning Environments (VRLE) tool was used to evaluate students’ attitudes towards IVR and had internal consistency of 0.95 [34]. Several studies used a variety of instruments to evaluate the users’ sense of presence, including the Presence Questionnaire (PQ) [41], Spatial Presence Experience Scale (SPES) [38], and the Independent Television Company Sense of Presence Inventory (ITC-SOPI) [6]. With regard to instrument reliability, it was reported that the PQ value was 0.88, and the ITC-SOPI value was 0.962. The SPES value was not reported.
Only one study assessed cybersickness effects through the VR Sickness Questionnaire (VRSQ). The resulting instrument demonstrated reliability greater than 0.85 [41].

5.5. Effectiveness of IVR in Nursing Students

As outcome measures, the pedagogical benefits of fully IVR include experience and perception, student satisfaction, clinical knowledge, skill performance, confidence, attitudes and motivation, critical thinking, and clinical reasoning. This review’s outcomes were categorised and assessed using Kirkpatrick’s four-level evaluation framework [55]. The framework comprises four levels: reaction, learning, behaviour, and results. Most of the included studies assessed more than one outcome that could be categorised within level one and/or level two of the Kirkpatrick evaluation framework. However, none of the studies gathered data on behaviour or results. More details on the category of outcomes are given in Table S2 (Supplement File S2). Figure 2 shows the characteristics of interventions and outcomes used in included studies.

6. Discussion

A nine-step scoping review was conducted in accordance with the JBI-described framework, and several significant results were identified. A total of 26 studies were included in this review, published between 2018 and 2022, conducted in developed countries, and mostly adopting quasi-experimental designs. Seven papers (including mixed-methods studies) collected qualitative data, whereas three employed qualitative research designs [28,32,40]. It is acknowledged that qualitative research enables a deeper, in-depth understanding of an experience [33]; however, combining elements of both quantitative and qualitative data provides more in-depth findings related to previous research undertaken in this area [25].
The aim of this review was to map the research field of fully IVR simulation in nursing and midwifery education. All of the participants were undergraduate nursing students, with only one study including nursing and midwifery students [34]. A paucity of studies in the area of midwifery indicates the need for more studies. Research findings have consistently demonstrated that students experience significantly improved learning gains, heightened motivation, and increased satisfaction when VR simulation methods are used. Furthermore, students enjoy learning experiences based on VR, expressed ease of use and application, and acknowledge a positive impact on learning [16,28].
Experts and educators were occasionally consulted during the scenario design and development process. However, despite the importance and feasibility of including students in participatory approaches [56], only a single study used this strategy [35]. Participatory design (PD) considers users to be at the centre of design processes and seeks to assure the usability, simplicity, and intelligibility of the product for its intended end-users. It would be beneficial to include participatory design as a framework for the co-creation of VR scenarios for nursing and midwifery students, as it would allow designer–user cooperation [57,58]. Pears [59] indicated that when videos are co-created with students, their engagement is enhanced, and the videos present a realistic and credible perspective.
The review has also indicated that there is a paucity of research on interactive and immersive 360-degree VR video. For example, most IVR simulation (n = 22) research was limited to exploring nurse education through 3D modelling. Hence, facial expressions, emotions, and other nonverbal cues of real patients cannot be completely simulated [37], suggesting that the incorporation of a real actor would enhance realism and give an enhanced sense of presence [33,60].
Additionally, most university teachers emphasised the importance of passive scenarios as the primary method of learning and tended to be antipathetic to the concept of acquiring knowledge through active learning, as they believed that this could lead to a loss of time and prevent them from achieving their academic goals [61]. From a student perspective, moving from a passive role to being the main protagonist of their learning was a significant undertaking [62]. Future generations of students are more likely to be influenced by instant gratification linked to their consumption of experiences [63]. In a study involving nursing students, researchers analysed the educational benefits of using immersive 360-degree VR video to learn about childbirth and found that students who used immersive technology had higher learning achievement than those who used a conventional video [27].
In terms of actual study design, the duration of the IVR session was considered a critical aspect. However, in the reviewed research, no guidelines regarding optimal IVR session duration were provided, but the length of IVR simulation varied from eight minutes to one hour. This mirrored a similar lack of IVR assessment duration found in a scoping review of healthcare patient education [64].
Sharples [65] indicated that prolonged use of IVR has been linked to an increase in the possibility of developing adverse effects. Therefore, researchers in the literature on educational video use concur that videos should be presented in short portions to optimise learner concentration [66].
Motion sickness, also known as VR sickness, is a key barrier to the effective use of IVR. To lessen potential negative outcomes, researchers have instructed participants to engage in IVR while seated [40] or excluded participants with a history of motion sickness or mental disorders [40,42]. Researchers should also consider that IVR sickness can be reduced by greater immersion, higher graphics and sound quality, and clearer explanations [67]. When designing successful IVR applications, it is essential to provide high-quality sound to enhance the sense of immersion and realism for the user and to reduce potential negative VR side effects [37,67].
The lack of a clear theoretical framework was the primary limitation in the reviewed papers. This had also been noted in a scoping review of IVR in undergraduate medical education, where only 2.6% of the studies incorporated theoretical frameworks in their design [68]. The inclusion of a theoretical framework allows researchers to discuss their findings in terms of existing theories and would assist in enhancing the quality, clarity, and repeatability of research [68,69].
In studies using a theoretical framework, the NLN/Jeffries Simulation Framework [3] was the most frequently used. Yu [48] indicated that the effectiveness of learning is dependent on the interactions between students and teachers. However, in IVR simulations, students do not directly interact with instructors, making it challenging to apply this concept in IVR programmes. There are alternative frameworks that can be used, including the Technology Acceptance Model (TAM) used to design digital information systems and to study the acceptability of technology [12]. A review of TAM involving over 400 articles published between 1999–2017 revealed that the model was widely used in fields related to education [70]. Yang [71] used 360-degree video to teach nursing students physical examination techniques and the results indicated that ease of use enhanced students’ interest, engagement, and motivation to learn. Fokides [12] agreed and suggested examining the influence of 360-degrees videos on learning through the TAM to determine if students’ willingness to use is associated with enhanced achievement of stated learning outcomes.
The ADDIE model was used for instructional design to develop teaching materials in most studies [27,35] and was successful for developing a cybersecurity in healthcare workshop, resulting in significant increases in participants’ confidence levels [72].
Debriefing is a key element of health professional education and is based on cognitive reasoning processes, involving reconstructing experiences to identify behaviours and actions that can be modified or improved [18]. Although debriefing is regarded as a vital component of educational simulation, it was not mentioned in over 60% (61.5%) of studies in this the review. Researchers who did use debriefing did not specify which type they employed. Similarly, a meta-analysis for assessing the efficacy of neurosurgical simulation reported that few researchers reported the debriefing process [73]. There are two types of debriefing that can be embedded in simulation education: structured and unstructured. Structured debriefing is commonly used in nursing simulation, and it has been shown that it can improve the learning outcomes [74,75]. In contrast, unstructured debriefing can be useful in revealing various details of a learners’ experience [18]. An RCT comparing virtual environments with and without debriefing revealed that computer debriefing improved midwifery students’ nontechnical skills, self-efficacy, and knowledge [76]. As a result, when using IVR simulation, time allocated is needed for debriefing so that students can share their insights and further develop their knowledge [48].
Outcome measures were classified according to Kirkpatrick’s evaluation framework. Level 2 of the model corresponds to the most measurable outcomes: knowledge and confidence. Knowledge was assessed using MCQs [16] or true/false questions [30], with results demonstrating significant improvements in knowledge [27,35]. However, reviews of virtual simulation in undergraduate medical education have identified a lack of evidence on IVR effectiveness on levels 3 and 4 of Kirkpatrick’s model that focus on behaviour and results [77]. Similarly, the lack of research based on longitudinal designs to assess VR education’s long-term effects found in this review may be due to the infancy of the underlying technology [78].
A further essential aspect of using fully IVR as a learning tool is how to minimise costs, which is a major consideration for decision makers considering the adoption of IVR solutions (e.g., curriculum or university course designers). The costs associated with IVR serve as a barrier to its use in healthcare education and delivery [17]. Immersive VR technology is constantly evolving, can be costly, and requires regular maintenance and updating [35]. There are, however, low-cost IVR systems available by combining the capabilities of smart devices and 360-degree video. These systems have been found to achieve effectiveness in a variety of health situations, including labour [27] and mental health [37]. Consequently, affordable IVR technologies have the potential to reduce technology access disparities and increase future utilisation in healthcare education [40].
A methodological strength of this review involves an extensive and in-depth analysis of completed studies in the IVR field. Nonetheless, this review is limited to studies that used HMDs to introduce VR to undergraduate NMSs. Moreover, it is possible that, despite the researcher teams best efforts, some articles might have been missed.

Implications for Future Research and Practice

The insights from this review can inform future research into the feasibility and efficacy of IVR in undergraduate nursing and midwifery education. Further research is required, particularly a qualitative exploration, to better understand and address the challenges affecting educational intervention and the student experience. To aid in the design and implementation of IVR systems in a range of academic contexts, more research is needed to determine comparisons between different HMDs and their effects on user presence and cost-effectiveness. Research is also needed to determine the impact an intervention has on certain learning outcomes, with an emphasis on the Kirkpatrick Model’s levels 3 and 4—behaviour and results. This would allow for more clarity and in-depth understanding of the specific training effects, from initial reactions and knowledge acquisition (Levels 1 and 2) to long-term changes in behaviour and tangible results in relation to clinical practice and patient safety.

7. Conclusions

There has been an increased use of full IVR in nursing education, which has demonstrated improvements in student learning outcomes. However, the published literature on the use of IVR in midwifery education is scarce. The limited availability implies that there is a significant opportunity for more investigation and analysis to gain a thorough understanding of the potential benefits, difficulties, and efficacy of incorporating IVR into midwifery education practices. The impact and effectiveness of IVR depends on its design, content, and implementation. Therefore, the value of including key stakeholders in the co-design process should be further incorporated into the development process. There is a notable absence of innovative IVR modalities like 360-degree video and smartphone VR in the literature. The demonstrated cost-effectiveness of this design type highlights its potential significance. Further research is needed, particularly at levels 3 and 4 of the Kirkpatrick framework, as longitudinal studies are currently lacking, and insights into longer term benefits are lacking. It is crucial that future research includes theoretical frameworks in the creation of IVR applications. Additionally, the implementation of debriefing sessions and guidelines for VR session durations should be carefully considered. These considerations are critical for optimizing the efficacy and impact of IVR-based education.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/virtualworlds2040023/s1, Table S1: MEDLINE Search Strategy. Table S2: Outcome Measures Using Kirkpatrick’s Evaluation Framework. References [6,25,26,27,29,30,31,33,34,35,36,37,39,41,42,43,44,45,46,47,49] are cited in the Supplementary Materials.

Author Contributions

Conceptualization, M.A.A., L.H. and S.T.K.; methodology, M.A.A.; software, M.A.A.; validation, M.A.A., L.H. and S.T.K.; formal analysis, M.A.A.; investigation, M.A.A.; resources, M.A.A.; data curation, M.A.A.; writing—original draft preparation, M.A.A.; writing—review and editing, M.A.A., L.H. and S.T.K.; supervision, L.H. and S.T.K.; project administration, M.A.A., L.H. and S.T.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study did not require ethical approval.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data are provided in the Supplementary Materials.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. DiLullo, C.; McGee, P.; Kriebel, R.M. Demystifying the millennial student: A reassessment in measures of character and engagement in professional education. Anat. Sci. Educ. 2011, 4, 214–226. [Google Scholar] [CrossRef]
  2. Olaussen, C.; Heggdal, K.; Tvedt, C.R. Elements in scenario-based simulation associated with nursing students’ self-confidence and satisfaction: A cross-sectional study. Nurs. Open 2020, 7, 170–179. [Google Scholar] [CrossRef]
  3. Jeffries, P.R.; Rodgers, B.; Adamson, K. NLN Jeffries simulation theory: Brief narrative description. Nurs. Educ. Perspect. 2015, 36, 292–293. [Google Scholar]
  4. Georgieva, D.; Koleva, G.; Hristova, I. Virtual Technologies in the Medical Professions-Creation of 360-Degree Environments for Health Care Training. TEM J. 2021, 10, 1314–1318. [Google Scholar] [CrossRef]
  5. Terminology and Concepts Working Group. Healthcare Simulation Dictionary, 2nd ed.; Lioce, L., Loprieato, J., Downing, D., Chang, T.P., Robertson, J.M., Anderson, M., Diaz, D.A., Spain, A.E., Eds.; Agency for Healthcare Research and Quality: Rockville, MD, USA, 2020. [Google Scholar]
  6. Dang, B.K.; Johnson, B.K.; Butzlaff, A.; Gilbert, G.E.; Schleicher, M.; Palicte, J.; Wood, A.M.; Kardong-Edgren, S. A Multi-Site Evaluation of Observer Roles and Telepresence Technology in Simulation with Prelicensure Nursing Students. Clin. Simul. Nurs. 2021, 55, 86–94. [Google Scholar] [CrossRef]
  7. Evens, M.; Empsen, M.; Hustinx, W. A literature review on 360-degree video as an educational tool: Towards design guidelines. J. Comput. Educ. 2023, 10, 325–375. [Google Scholar] [CrossRef]
  8. Green, J.; Wyllie, A.; Jackson, D. Electronic portfolios in nursing education: A review of the literature. Nurse Educ. Pract. 2014, 14, 4–8. [Google Scholar] [CrossRef] [PubMed]
  9. Clarke, E. Virtual reality simulation—The future of orthopaedic training? A systematic review and narrative analysis. Adv. Simul. 2021, 6, 2. [Google Scholar] [CrossRef]
  10. Spring, D. Gaming history: Computer and video games as historical scholarship. Rethink. Hist. 2015, 19, 207–221. [Google Scholar] [CrossRef]
  11. Yousef, A.M.; Chatti, M.A.; Schroeder, U. The state of video-based learning: A review and future perspectives. Int. J. Adv. Life Sci. 2014, 6, 122–135. [Google Scholar]
  12. Fokides, E.; Arvaniti, P.A. Evaluating the effectiveness of 360 videos when teaching primary school subjects related to environmental education. J. Pedagog. Res. 2020, 4, 203–222. [Google Scholar] [CrossRef]
  13. Jensen, L.; Konradsen, F. A review of the use of virtual reality head-mounted displays in education and training. Educ. Inf. Technol. 2018, 23, 1515–1529. [Google Scholar] [CrossRef]
  14. Snelson, C.; Hsu, Y.C. Educational 360-degree videos in virtual reality: A scoping review of the emerging research. TechTrends 2020, 64, 404–412. [Google Scholar] [CrossRef]
  15. Fealy, S.; Jones, D.; Hutton, A.; Graham, K.; McNeill, L.; Sweet, L.; Hazelton, M. The integration of immersive virtual reality in tertiary nursing and midwifery education: A scoping review. Nurse Educ. Today 2019, 79, 14–19. [Google Scholar] [CrossRef] [PubMed]
  16. Choi, J.; Thompson, C.E.; Choi, J.; Waddill, C.B.; Choi, S. Effectiveness of immersive virtual reality in nursing education: Systematic review. Nurse Educ. 2022, 47, E57–E61. [Google Scholar] [CrossRef] [PubMed]
  17. Shorey, S.; Ng, E.D. The use of virtual reality simulation among nursing students and registered nurses: A systematic review. Nurse Educ. Today 2021, 98, 104662. [Google Scholar] [CrossRef] [PubMed]
  18. Shin, H.; Rim, D.; Kim, H.; Park, S.; Shon, S. Educational characteristics of virtual simulation in nursing: An integrative review. Clin. Simul. Nurs. 2019, 37, 18–28. [Google Scholar] [CrossRef]
  19. Blair, C.; Walsh, C.; Best, P. Immersive 360 videos in health and social care education: A scoping review. BMC Med. Educ. 2021, 21, 590. [Google Scholar] [CrossRef]
  20. World Health Organization. Four-Year, Integrated Nursing and Midwifery Competency-Based Curriculum, Prototype Curriculum for the African Region. Available online: https://apps.who.int/iris/bitstream/handle/10665/331471/9789290232612-eng.pdf (accessed on 16 August 2022).
  21. Nehring, W.M.; Lashley, F.R. Nursing simulation: A review of the past 40 years. Simul. Gaming 2009, 40, 528–552. [Google Scholar] [CrossRef]
  22. Peters, M.D.J.; Godfrey, C.; McInerney, P.; Munn, Z.; Tricco, A.C.; Khalil, H. Chapter 11: Scoping reviews (2020 version). JBI manual for evidence synthesis. JBI 2020, 406–451. [Google Scholar] [CrossRef]
  23. Tricco, A.C.; Lillie, E.; Zarin, W.; O’Brien, K.K.; Colquhoun, H.; Levac, D.; Moher, D.; Peters, M.D.; Horsley, T.; Weeks, L.; et al. PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Ann. Intern. Med. 2018, 169, 467–473. [Google Scholar] [CrossRef] [PubMed]
  24. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; PRISMA Group*. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Ann. Intern. Med. 2009, 151, 264–269. [Google Scholar] [CrossRef] [PubMed]
  25. Adhikari, R.; Kydonaki, C.; Lawrie, J.; O’Reilly, M.; Ballantyne, B.; Whitehorn, J.; Paterson, R. A mixed-methods feasibility study to assess the acceptability and applicability of immersive virtual reality sepsis game as an adjunct to nursing education. Nurse Educ. Today 2021, 103, 104944. [Google Scholar] [CrossRef] [PubMed]
  26. Butt, A.L.; Kardong-Edgren, S.; Ellertson, A. Using Game-Based Virtual Reality with Haptics for Skill Acquisition. Clin. Simul. Nurs. 2018, 16, 25–32. [Google Scholar] [CrossRef]
  27. Chang, C.Y.; Sung, H.Y.; Guo, J.L.; Chang, B.Y.; Kuo, F.R. Effects of spherical video-based virtual reality on nursing students’ learning performance in childbirth education training. Interact. Learn. Environ. 2022, 30, 400–416. [Google Scholar] [CrossRef]
  28. Chang, Y.M.; Lai, C.L. Exploring the experiences of nursing students in using immersive virtual reality to learn nursing skills. Nurse Educ. Today 2021, 97, 104670. [Google Scholar] [CrossRef] [PubMed]
  29. Chao, L.F.; Huang, T.Y.; Moser, D.K.; Chung, F.F.; Lau, Y.T.; Xiao, X. Development and Pilot Testing of a Non-stress Test Virtual Reality Simulator. CIN 2022, 40, 357–361. [Google Scholar] [CrossRef]
  30. Chao, Y.C.; Hu, S.H.; Chiu, H.Y.; Huang, P.H.; Tsai, H.T.; Chuang, Y.H. The effects of an immersive 3d interactive video program on improving student nurses’ nursing skill competence: A randomized controlled trial study. Nurse Educ. Today 2021, 103, 104979. [Google Scholar] [CrossRef]
  31. Choi, K.S. Virtual reality simulation for learning wound dressing: Acceptance and usability. Clin. Simul. Nurs. 2022, 68, 49–57. [Google Scholar] [CrossRef]
  32. Farra, S.L.; Smith, S.J.; Ulrich, D.L. The Student Experience With Varying Immersion Levels of Virtual Reality Simulation. Nurs. Educ. Perspect. 2018, 39, 99–101. [Google Scholar] [CrossRef]
  33. Giordano, N.A.; Whitney, C.E.; Axson, S.A.; Cassidy, K.; Rosado, E.; Hoyt-Brennan, A.M. A pilot study to compare virtual reality to hybrid simulation for opioid-related overdose and naloxone training. Nurse Educ. Today 2020, 88, 104365. [Google Scholar] [CrossRef] [PubMed]
  34. Hardie, P.; Darley, A.; Carroll, L.; Redmond, C.; Campbell, A.; Jarvis, S. Nursing & Midwifery students’ experience of immersive virtual reality storytelling: An evaluative study. BMC Nurs. 2020, 19, 78. [Google Scholar] [CrossRef]
  35. Jeong, Y.; Lee, H.; Han, J.-W. Development and evaluation of virtual reality simulation education based on coronavirus disease 2019 scenario for nursing students: A pilot study. Nurs. Open 2022, 9, 1066–1076. [Google Scholar] [CrossRef] [PubMed]
  36. Jung, A.-R.; Park, E.-A. The Effectiveness of Learning to Use HMD-Based VR Technologies on Nursing Students: Chemoport Insertion Surgery. Int. J. Environ. Res. Public Health 2022, 19, 4823. [Google Scholar] [CrossRef] [PubMed]
  37. Lee, Y.; Kim, S.K.; Eom, M.-R. Usability of mental illness simulation involving scenarios with patients with schizophrenia via immersive virtual reality: A mixed methods study. PLoS ONE 2020, 15, e0238437. [Google Scholar] [CrossRef] [PubMed]
  38. Ma, Z.; Huang, K.-T.; Yao, L. Feasibility of a Computer Role-Playing Game to Promote Empathy in Nursing Students: The Role of Immersiveness and Perspective. Cyberpsychol. Behav. Soc. Netw. 2021, 24, 750–755. [Google Scholar] [CrossRef]
  39. Silva, L.I.M.; de la Calle, R.C.; Cuevas-Budhart, M.A.; Martin, J.O.M.; Rodriguez, J.M.B.; Madrid, M.G. Development of Communication Skills Through Virtual Reality on Nursing School Students: Clinical Trial. CIN 2022, 41, 24–30. [Google Scholar] [CrossRef]
  40. Saab, M.M.; Landers, M.; Murphy, D.; O’Mahony, B.; Cooke, E.; O’Driscoll, M.; Hegarty, J. Nursing students’ views of using virtual reality in healthcare: A qualitative study. J. Clin. Nurs. 2022, 31, 1228–1242. [Google Scholar] [CrossRef]
  41. Samosorn, A.B.; Gilbert, G.E.; Bauman, E.B.; Khine, J.; McGonigle, D. Teaching Airway Insertion Skills to Nursing Faculty and Students Using Virtual Reality: A Pilot Study. Clin. Simul. Nurs. 2020, 39, 18–26. [Google Scholar] [CrossRef]
  42. Siah, R.C.; Xu, P.; Teh, C.L.; Kow, A.W. Evaluation of nursing students’ efficacy, attitude, and confidence level in a perioperative setting using virtual-reality simulation. Nurs. Forum 2022, 57, 1249–1257. [Google Scholar] [CrossRef]
  43. Smith, S.; Farra, S.L.; Hodgson, E. Evaluation of two simulation methods for teaching a disaster skill. BMJ Simul. Technol. Enhanc. Learn. 2021, 7, 92–96. [Google Scholar] [CrossRef]
  44. Smith, S.J.; Farra, S.L.; Ulrich, D.L.; Hodgson, E.; Nicely, S.; Mickle, A. Effectiveness of Two Varying Levels of Virtual Reality Simulation. Nurs. Educ. Perspect. 2018, 39, E10–E15. [Google Scholar] [CrossRef] [PubMed]
  45. Taçgin, Z. The Perceived Effectiveness Regarding Immersive Virtual Reality Learning Environments Changes by the Prior Knowledge of Learners. Educ. Inf. Technol. 2020, 25, 2791–2809. [Google Scholar] [CrossRef]
  46. Thompson, D.S.; Thompson, A.P.; McConnell, K. Nursing students’ engagement and experiences with virtual reality in an undergraduate bioscience course. Int. J. Nurs. Educ. Scholarsh. 2020, 17, 20190081. [Google Scholar] [CrossRef] [PubMed]
  47. Yang, S.-Y.; Oh, Y.-H. The effects of neonatal resuscitation gamification program using immersive virtual reality: A quasi-experimental study. Nurse Educ. Today 2022, 117, 105464. [Google Scholar] [CrossRef] [PubMed]
  48. Yu, M.; Yang, M.; Ku, B.; Mann, J.S. Effects of Virtual Reality Simulation Program Regarding High-risk Neonatal Infection Control on Nursing Students. Asian Nurs. Res. 2021, 15, 189–196. [Google Scholar] [CrossRef] [PubMed]
  49. Yu, M.; Yang, M.R. Effectiveness and Utility of Virtual Reality Infection Control Simulation for Children with COVID-19: Quasi-Experimental Study. JMIR Serious Games 2022, 10, e36707. [Google Scholar] [CrossRef] [PubMed]
  50. Kolb, D.A. Experiential Learning: Experience as the Source of Learning and Development; FT Press: Upper Saddle River, NJ, USA, 2014. [Google Scholar]
  51. Bandura, A.; Walters, R.H. Social Learning Theory; Prentice Hall: Englewood Cliffs, NJ, USA, 1977. [Google Scholar]
  52. Bandura, A. Self-Efficacy: The Exercise of Control; W.H. Freeman: New York, NY, USA, 1997. [Google Scholar]
  53. Lewis, J.R. IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. Int. J. Hum. Comput. Interact. 1995, 7, 57–78. [Google Scholar] [CrossRef]
  54. Davis, F.D.; Bagozzi, R.P.; Warshaw, P.R. User acceptance of computer technology: A comparison of two theoretical models. Manag. Sci. 1989, 35, 982–1003. [Google Scholar] [CrossRef]
  55. Kirkpatrick, D.L.; Kirkpatrick, J.D. Implementing the Four Levels; Berrett-Koehler Publishers: Oakland, CA, USA, 2007. [Google Scholar]
  56. Wharrad, H.; Windle, R.; Taylor, M. Designing digital education and training for health. In Digital Innovations in Healthcare Education and Training; Academic Press: Cambridge, MA, USA, 2021; pp. 31–45. [Google Scholar] [CrossRef]
  57. Taylor, M.; Wharrad, H.; Konstantinidis, S. Immerse Yourself in ASPIRE-Adding Persuasive Technology Methodology to the ASPIRE Framework. In International Conference on Interactive Collaborative Learning; Springer International Publishing: Cham, Switzerland, 2021; pp. 1106–1117. [Google Scholar] [CrossRef]
  58. Pears, M.; Henderson, J.; Konstantinidis, S. Repurposing a Reusable Learning Object on Effective Communication with Adolescents to an Interactive 360° Immersive Environment by Adapting the ASPIRE Framework. In International Conference on Interactive Collaborative Learning; Springer International Publishing: Cham, Switzerland, 2021; pp. 1096–1105. [Google Scholar] [CrossRef]
  59. Pears, M.; Henderson, J.; Antoniou, P.E.; Ntakakis, G.; Nikolaidou, M.; Bamidis, P.D.; Schiza, E.; Pattichis, C.S.; Frangoudes, F.; Gkougkoudi, E.; et al. Feasibility and Acceptance of Virtual Reality Reusable e-Resources Embedded in Healthcare Curricula. In Proceedings of the 2022 International Conference on Interactive Media, Smart Systems and Emerging Technologies (IMET), Limassol, Cyprus, 4–7 October 2022; pp. 1–8. [Google Scholar] [CrossRef]
  60. Strojny, P.M.; Dużmańska-Misiarczyk, N.; Lipp, N.; Strojny, A. Moderators of social facilitation effect in virtual reality: Co-presence and realism of virtual agents. Front. Psychol. 2020, 11, 1252. [Google Scholar] [CrossRef]
  61. Murillo-Zamorano, L.R.; López Sánchez, J.Á.; Godoy-Caballero, A.L.; Bueno Muñoz, C. Gamification and active learning in higher education: Is it possible to match digital society, academia and students’ interests? Int. J. Educ. Technol. High. Educ. 2021, 18, 15. [Google Scholar] [CrossRef]
  62. Mayrose, J. Active learning through the use of virtual environments. AJEE 2012, 3, 13–18. [Google Scholar] [CrossRef]
  63. Sackin, S. What gen Z wants in a career (and how to give it to them). Forbes Agency Counc. 2018, 1–5. Available online: https://www.forbes.com/sites/forbesagencycouncil/2018/08/28/what-gen-z-wants-in-a-career-and-how-to-give-it-to-them/?sh=27ca21507a4a (accessed on 13 June 2023).
  64. Van der Kruk, S.R.; Zielinski, R.; MacDougall, H.; Hughes-Barton, D.; Gunn, K.M. Virtual reality as a patient education tool in healthcare: A scoping review. Patient Educ. Couns. 2022, 105, 1928–1942. [Google Scholar] [CrossRef] [PubMed]
  65. Sharples, S.; Cobb, S.; Moody, A.; Wilson, J.R. Virtual reality induced symptoms and effects (VRISE): Comparison of head mounted display (HMD), desktop and projection display systems. Displays 2008, 29, 58–69. [Google Scholar] [CrossRef]
  66. Shephard, K. Questioning, promoting and evaluating the use of streaming video to support student learning. Br. J. Educ. Technol. 2003, 34, 295–308. [Google Scholar] [CrossRef]
  67. Kourtesis, P.; Collina, S.; Doumas, L.A.; MacPherson, S.E. Validation of the virtual reality neuroscience questionnaire: Maximum duration of immersive virtual reality sessions without the presence of pertinent adverse symptomatology. Front. Hum. Neurosci. 2019, 13, 417. [Google Scholar] [CrossRef]
  68. Haowen, J.; Vimalesvaran, S.; King Wang, J.; Boon, L.K.; Mogali, S.R.; Tudor Car, L. Virtual reality in medical students’ education: A scoping review. JMIR Med. Educ. 2022, 8, e34860. [Google Scholar] [CrossRef]
  69. Ravitch, S.M.; Riggan, M. Reason & Rigor: How Conceptual Frameworks Guide Research; Sage Publications: Thousand Oaks, CA, USA, 2016. [Google Scholar]
  70. Rahimi, B.; Nadri, H.; Afshar, H.L.; Timpka, T. A systematic review of the technology acceptance model in health informatics. Appl. Clin. Inform. 2018, 9, 604–634. [Google Scholar] [CrossRef]
  71. Yang, T.Y.; Huang, C.H.; An, C.; Weng, L.C. Construction and evaluation of a 360 degrees panoramic video on the physical examination of nursing students. Nurse Educ. Pract. 2022, 63, 103372. [Google Scholar] [CrossRef]
  72. Pears, M.; Konstantinidis, S.T. Cybersecurity training in the healthcare workforce–utilization of the ADDIE model. In Proceedings of the 2021 IEEE Global Engineering Education Conference (EDUCON), Vienna, Austria, 21–23 April 2021; pp. 1674–1681. [Google Scholar] [CrossRef]
  73. Davids, J.; Manivannan, S.; Darzi, A.; Giannarou, S.; Ashrafian, H.; Marcus, H.J. Simulation for skills training in neurosurgery: A systematic review, meta-analysis, and analysis of progressive scholarly acceptance. Neurosurg. Rev. 2021, 44, 1853–1867. [Google Scholar] [CrossRef] [PubMed]
  74. Endacott, R.; Gale, T.; O’Connor, A.; Dix, S. Frameworks and quality measures used for debriefing in team-based simulation: A systematic review. BMJ Simul. Technol. Enhanc. Learn. 2019, 5, 61. [Google Scholar] [CrossRef] [PubMed]
  75. Foronda, C.; Gattamorta, K.; Snowden, K.; Bauman, E.B. Use of virtual clinical simulation to improve communication skills of baccalaureate nursing students: A pilot study. Nurse Educ. Today 2014, 34, e53–e57. [Google Scholar] [CrossRef] [PubMed]
  76. Michelet, D.; Barre, J.; Truchot, J.; Piot, M.A.; Cabon, P.; Tesniere, A. Effect of computer debriefing on acquisition and retention of learning after screen-based simulation of neonatal resuscitation: Randomized controlled trial. JMIR Serious Games 2020, 8, e18633. [Google Scholar] [CrossRef]
  77. Wu, Q.; Wang, Y.; Lu, L.; Chen, Y.; Long, H.; Wang, J. Virtual simulation in undergraduate medical education: A scoping review of recent practice. Front. Med. 2022, 9, 855403. [Google Scholar] [CrossRef]
  78. Tang, H. Applied research of VR technology in physical education. Int. J. Electr. Eng. Educ. 2021, 60, 40–48. [Google Scholar] [CrossRef]
Virtualworlds 02 00023 g001
Figure 1. Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) statement [24].
Figure 1. Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) statement [24].
Virtualworlds 02 00023 g001
Virtualworlds 02 00023 g002
Figure 2. Characteristics of interventions and outcomes used in studies.
Figure 2. Characteristics of interventions and outcomes used in studies.
Virtualworlds 02 00023 g002
Table 1. Data extraction table.
Table 1. Data extraction table.
Year and Authors/
Country		AimResearch Design/
Sample Size		Theoretical ModelsVR Modality and TimeOutcome Measures Using Kirkpatrick’s Evaluation FrameworkDebriefingFindings
2022
Taiwan [27]		Development of childbirth-learning contextsQED
I = 32
C = 32 (watched a video)		SLT + ADDIE model.Google Cardboard and 360-degree childbirth videos1 and 2NoVR group had significant effects on learning achievement, motivation, and satisfaction and no significant effects on critical thinking and attitude.
2022
Republic of Korea [49]		Development of COVID-19 paediatric simulation about donning and doffing of PPEQED
I = 25
C = 25 (routine practice)		Self-efficacy theory15 min
HMD and controller		1 and 2YesVR group had significant improvements in PPE knowledge (z = −3.28, p < 0.001), infection control performance (t48 = 4.89, p < 0.001), and self-efficacy (t36.2 = 4.93, p < 0.001).
2021
Republic of Korea [48]		Examine the effects of a VRS on nursing students learning outcomesQED
I = 25
C = 25 (routine practice)		NLN/Jeffries Simulation Framework40 min total of 3 scenarios
HTC VIVETM, controller		1 and 2YesVR group had significant improvements in self-efficacy (t = −2.16, p = 0.018) and satisfaction (t = 5.59, p < 0.001).
2020
Canada [46]		Describe VR experience in a first-year nursing course on health assessmentCross-sectional design
I = 46		ELT1 h VR sessions Oculus Rift and hand controllers1NoStudents perceived their engagement to be higher in VR and found that it was easy to use and helped their learning, and they recommended it.
2021
USA [43]		Examine the differences in learning outcomes for the disaster skill of decontamination.QED
I = 61
C = 60 (HFS)		NLN/Jeffries Simulation Framework10 min
Oculus Rift and hand controllers		1 and 2NoNo statistically significant results were noted for any of the study outcomes. Both groups felt equally confident.
2022
Singapore [42]		Evaluate learning outcomes among nursing students as scrub nursesDescriptive study design
I = 207		Flow theory and ELT15 min
HMD and touch controllers		1 and 2YesMore than 90% of the participants indicated “neutral” and above in terms of efficacy, attitude, and confidence level.
2022
Republic of Korea [35]		Evaluate the effectiveness of a VR program using a COVID-19 scenarioQED
I = 32
C = 33 (lecture only)		ADDIE model15 min
Oculus Quest 2
Character hands		1 and 2YesVR group had a significantly higher learning satisfaction (t = 3.01, p = 004).
Both groups presented significant differences in knowledge, self-efficacy, and clinical reasoning, and there were no differences between the groups.
2020
USA [33]		Pilot the use of a VR to identify signs and symptoms of an opioid-related overdoseQED
I = 19
C = 31 (hybrid simulation)		NLN/Jeffries Simulation Framework20 min
Cardboard headset		2YesAll participants’ attitudes scores decreased from baseline to follow up by <2 points.
Knowledge retention neither decreased nor improved over time for both groups.
2018
USA [32]		Examine varying levels of immersion using disaster-based VRQDR
N = 100
I = 36 (VR)
I = 36 (computer monitor)
C = 28 (written instructions)		NLN Jeffries Simulation Framework10 min
Oculus Rift		1NoFour themes: simulation learning experience; simulation design; participant outcomes; and participant simulation experience.
2018
USA [44]		Evaluate the impact of two different decontamination VR interfacesQED
N = 172
I = 59 (VR)
I = 58 (computer monitor)
C = 55 (written instructions)		NLN Jeffries Simulation Framework10 min
Oculus Rift and controller
character built in Autodesk Maya.		2NoCognitive and performance scores were significantly higher after treatment. Six months later, they were significantly lower in cognitive scores at the retention measurement, and performance time completion was found to be significant and faster only between computer group and written instruction group.
2021
USA [6]		Explore differences in perceived levels of presence among four common roles in simulationQED
N = 160
I = 29 (active participants)
I = 51 (VR)
C = 20 (bedside observers)
C = 60 (Audio Visual observers)		Observational Experiential Learning15 min
HMD with mobile
Syminar©
live stream the simulation from a 360-camera		2YesVR learners had significantly higher presence scores than
AV group and similar scores to active participant roles.
There were no significant differences in knowledge outcomes.
SUS aggregate score was 69.46.
2020
Ireland [34]		To explore students’ perception of a VR storytelling experienceMMR consisting of a cross-sectional survey and an observational study.
N = 132
I = 94
Nursing and midwifery students
Response rate of 71.2% (n = 94)		ELTVR One Plus
Wonderful You developed by BHD.		1 and 2NoParticipants reported positively to engaging with and learning from an VR storytelling.
Three themes emerged: memorable learning experience; negative elements to VR; and applications of VR in nursing and midwifery education.
2022
Republic of Korea [47]		To examine the effects of VR neonatal resuscitation gameQED
N = 88
I = 29 (VR)
I = 28 (HFS)
C = 26 (online lectures)		Keller’s ARCS model50 min
Oculus Rift		2YesNeonatal resuscitation knowledge and learning motivation were significantly higher in the VR and simulation groups than in the control group, whereas problem-solving ability and self-confidence were significantly higher in the VR group than in the simulation and control groups. Anxiety was significantly lower in the simulation group than in the VR and control groups.
2018
USA [26]		To explore the usability of a game-based VR system for urinary catheterisationMMR
I = 10
C = 10 (traditional practice)		Deliberate practice theory60 min (10–15 min orientation)
Oculus Rift and controller		1NoSubjects using the VR system spent more time practicing (p = 0.001) and completed more procedures in 1 h than students who practiced traditionally (p < 0.001).
Follow-up skill demonstration pass rates between groups were identical at 2 weeks
2021
Taiwan [28]		To understand the experience of nursing students in using VR skill-learning processQDR
I = 60		ELTHTC VIVE HMD1NoFive themes revealed: convenient to practice but requires adaptation; fast skill learning process; stress-free learning environment; environmentally friendly; and lacks a sense of reality.
2021
Taiwan [30]		Examined the effects of an immersive 3D interactive video program on improving nursing students’ NGT feeding skillRCTs
I = 22 (AR)
C = 23 (watched a video)		Teaching skills in nursing framework10–20 min
VIVEPAPER HTC, and touch a piece of paper		1 and 2NoParticipants knowledge and confidence were significantly improved after the intervention in both groups, and there was no significant difference.
Statistically significant differences in satisfaction between groups (p = 0.026).
2022
Taiwan [29]		To develop a basic prototype of the non-stress test VR and to describe students’ and educators’ reactionsAction research design
I = 56		Information processing theory50 min with debriefing
HMD
Speech recognition, which requires participant to engage verbally.		1YesParticipants were very receptive, with a satisfaction rate of 98.9%.
2021
USA [38]		Investigate the feasibility and effectiveness of computer role-playing games on students’ empathy with a focus on immersiveness and perspectiveFeasibility MMR
N = 69
I = 18 (VR family)
I = 19 (VR health care provider)
C = 16 (laptops family)
C = 16 (laptops health care provider)		N/A10 min
Oculus Go		N/ANoVR participants had a significant effect on empathy (p = 0.032), leading to greater spatial presence but not on perceived possible action.
Participants in the health care provider’s condition reported greater empathy than those in the patient’s family’s condition.
2021
Ireland [40]		Explored nursing students’ views of using VR in healthcareQDR
I = 26		N/A6–10 min
HMD with controllerDesigned using Unreal Engine 4.12 (UE4)		1NoFour themes were identified: positive experiences of VR; challenges to using VR; settings where VR can be implemented; and blue-sky and future applications of VR.
Older adults reported minor technical difficulties.
2021
UK [25]		To investigate the impact of VR sepsis game and to explore students’ perceptions of the acceptability and applicabilityFeasibility MMR
N = 282
I = 19
Response rate = 100%		N/A20–30 min
HMD		1 and 2YesPre-post-test scores revealed a significant increase in self-confidence (p < 0.001) and a significant decrease in anxiety (p < 0.001)
Four themes emerged: acceptability, applicability, areas of improvement for a VR sepsis game, and limitations of a VR game.
2022
Hong Kong [31]		To investigate the acceptance and usability of the desktop simulator (DTS) and an immersive simulator (IMS) wound-dressing simulation system.Crossover design
I = 30		N/AHTC VIVE and controllers1 and 2NoThe mean knowledge scores were similar for both groups.
The mean usability scores for IMS were relatively higher.
IMS favoured the experience for higher level of realism and DTS for being more convenient to use.
2020
Republic of Korea [37]		Evaluate the ease of use and usefulness of the VR simulation for mental health nursing educationMMR
I = 60		N/A13 min (mean of total videos)
Oculus Go and one controller.		1NoAll participants had positive experiences and reported that it was realistic (n = 39, 65%) and an interesting experience (n = 15, 25%).
Some suggested to further refine the picture and sound quality.
2022
Spain [39]		Develop a VR to improve communication skillsRCTs
I = 50
C = 50 (case-based theoretical workshop)		N/A360-degree VR goggles, and headphones2NoStatistically significant difference found between the VR method (p < 0.001) and a better-developed communication skill.
VR-based intervention showed better results in older students.
2022
Republic of Korea [36]		Develop and evaluate the effect of a mobile HMD-based VR for a Chemo port insertionQED
I = 30
C = 30 (self-study)		N/AMobile HMD
using Autodesk 3DS Max
Storyboard		1 and 2YesVR group significantly improved post-intervention knowledge (p = 0.001), learning attitude (p = 0.002), and satisfaction (p = 0.017).
Sub-domains of motivation, attention (p < 0.05), and relevance (p < 0.05) were significantly different between both groups post-intervention.
2020
Turkey [45]		Investigate the features of a VR learning environment and evaluate its perceived simulation effectivenessDescriptive quantitative design
I = 14 3rd year
I = 52 1st year		N/A10–32 min
Single HMD		2NoSignificant difference was only for the confidence sub-dimension (p < 0.05).
No significant differences among first (p = 0.070), second (p = 0.255), third (p = 0.408), and fourth (p = 0.244) applications.
The fourth practice shows that the learning environment is a more effective tool after adapting to use it (p = 0.048).
Highly significant relationship among attitudes, learning, and confidence in the first-year participants.
2020
USA [41]		Examine if intervention with a pilot contemporary IVRS builds knowledge and is feasibleQED
I = 21		Bauman’s layered-learning model (BLLM)20 min
Oculus Rift and controller		1 and 2NoStudents felt a high degree of presence and immersion, experienced little to no cybersickness and significantly improving knowledge of airway management (p < 0.0001).
AR = Augmented reality; C = Control; ELT = Experiential learning theory; HFS = High-fidelity simulation; I = Intervention; MMR = Mixed-methods research; N/A = Not available; NLN = National league for nursing; QDR = Qualitative descriptive research; QED = Quasi-experimental design; SLT = Social-learning theory.
Table 2. Number of articles for characteristics of included studies.
Table 2. Number of articles for characteristics of included studies.
20182019202020212022Have Research QuestionNo Research QuestionPre-VR Clinical TrainingPre-VR Theory Lectures
30689620311
Table 3. Scenario development.
Table 3. Scenario development.
Scenario Development[48][42][35][33][47][30][25][31][37][36]
Literature Review√ √ √√ √√√
Experts√√√√√ √√√√
Students √
Table 4. Types of interactivities used in the studies.
Table 4. Types of interactivities used in the studies.
StudyType of HMDTask Specified
[49]Not specified with controllerDonning of PPE and provide respiratory care
[48]HTC VIVETM, controllerNo
[46]Oculus Rift and hand controllersParticipants can zoom in and out
[43]Oculus Rift and hand controllersMCQ throughout tasks to assess their knowledge
[42]Not specified and touch controllersQuizzes at the end in the form
[35]Oculus Quest 2Character hands to put on PPE and perform nursing assessment to patient
[44]Oculus Rift and controllerSelect tools required for the task
[47]Oculus RiftNo
[26]Oculus Rift and controllerNo
[28]HTC VIVENo
[30]VIVEPAPER HTCTouch a piece of paper through answering and reading
[29]Not specifiedSpeech recognition, which requires participant to engage verbally
[38]Oculus Go and ControllerNavigate and pick avatar characters
[40]Not specified with controllerSelf-representation using hand avatars
[25]Not specifiedNo
[30]HTC VIVE and controllersNo
[37]Oculus Go and one controllerParticipants’ select answers to test their knowledge
[45]Not specifiedParticipants interact with the menu components via natural hand movements
[41]Oculus Rift and controllerNo
Table 5. Duration frequency of IVR scenario.
Table 5. Duration frequency of IVR scenario.
Value in MinutesFrequency
81 (5%)
105 (25%)
132 (10%)
154 (20%)
202 (10%)
211 (5%)
251 (5%)
502 (10%)
602 (10%)
Table 6. Theoretical frameworks of included studies.
Table 6. Theoretical frameworks of included studies.
Theory/FrameworkStudyObjective/sFeatures of the Framework
NLN/Jeffries Simulation Framework[48]Examine the effects of VR on students’ knowledge, self-efficacy, and satisfactionThe central proposition of this model is that student outcomes are influenced by the incorporation of best education practices into the design and implementation of the simulation experience. Furthermore, the model posits that learning depends on the teacher and student interactions, expectations, and roles.
[43]Examine the differences in learning outcomes satisfaction and self-confidence based on type of simulation VR or HFS
[33]Evaluate the use of VR simulation compared to hybrid simulation as a training modality
[32]Provide new evidence in how varying levels of immersion are perceived by nursing students using VRS
[44]Evaluate the use of VRS by examining two varying levels of VS on learning outcomes and retention and gain an understanding of the student experience
Experiential Learning Theory (ELT)[46]Describe students’ experiences with VR on anatomy and health assessment.
Applied the exploratory learning model (ELM) Kolb extension		Stage 1: participation in a VR experience.
Stage 2: observation and reflection.
Stage 3: learning outcomes from VR experience.
Stage 4: testing new situations (e.g., planning or applying to future situations).
[28]Understand students’ feelings toward self-learning using the VR skill education system
[34]Examine students’ perceptions of levels of immersion, interaction, imagination, motivation, and enhanced problem-solving capacity with IVR storytelling
[42]Evaluate the efficacy, attitude, and level of confidence among nursing students as scrub nurses in a perioperative environment via VRS
Information Processing Theory[29]Developed and implemented a non-stress test VR simulator (NST-VRS) to examine students’ and instructors’ receptionDerived from Cognitive Learning Theory (CLT); argues that sensory content can establish itself within long-term memory through rehearsal.
Social Learning Theory (SLT)[27]Applying SLT combined with smart mobile devices to increase the learning performance of nursing studentsIndividuals can recognise learning content, and their behaviours are affected through SLT embedded in ICT. Application learners’ SLT to ICT could support collaborative learning and lead to positive interactions and learning motivations.
ADDIE ModelAnalysis, Design, Development, Implementation and Evaluation[27]Investigate the development of childbirth learning contextsA: acquire basic data necessary to develop VR; D: set based on the results from the analysis step, research design, and programme operation method; D: develop prerequisite learning; I: total time; E: evaluation measurements.
[35]Develop VR simulation programme on the COVID-19 scenario and assess effectiveness
Self-Efficacy Theory[49]Used pre-briefing based on the self-efficacy theory and designed the programme to enhance self-efficacy“Individuals who have high self-efficacy will exert effort that, if well-executed, leads to successful performance and outcomes, such as clinical competency and simulation performance” (p. 2).
Observational Experiential Learning[6]Determine whether learner role and perceived level of presence in a simulation impacted knowledgeIncorporates ELT, SLT, and social cognitive theories.
A form of active learning that captures attention and engagement.
ARCS Model[47]Encourage learning motivation by using strategies to secure perceptual and inquiry arousal and variability in programme developmentDeveloped VR instructional design. A: attention, which induces and retains learners’ attention; R: relevance, which enables the recognition that the content learned is beneficial for achieving learners’ goals; C: confidence that learning can be well achieved; and S: satisfaction, which provides rewards that meet learners’ expectations.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Alammary, M.A.; Halliday, L.; Konstantinidis, S.T. 360-Degree Virtual Reality Utilising Head-Mounted Devices in Undergraduate Nursing and Midwifery Education: A Scoping Review. Virtual Worlds 2023, 2, 396-421. https://doi.org/10.3390/virtualworlds2040023

AMA Style

Alammary MA, Halliday L, Konstantinidis ST. 360-Degree Virtual Reality Utilising Head-Mounted Devices in Undergraduate Nursing and Midwifery Education: A Scoping Review. Virtual Worlds. 2023; 2(4):396-421. https://doi.org/10.3390/virtualworlds2040023

Chicago/Turabian Style

Alammary, Maram A., Lesley Halliday, and Stathis Th. Konstantinidis. 2023. "360-Degree Virtual Reality Utilising Head-Mounted Devices in Undergraduate Nursing and Midwifery Education: A Scoping Review" Virtual Worlds 2, no. 4: 396-421. https://doi.org/10.3390/virtualworlds2040023

APA Style

Alammary, M. A., Halliday, L., & Konstantinidis, S. T. (2023). 360-Degree Virtual Reality Utilising Head-Mounted Devices in Undergraduate Nursing and Midwifery Education: A Scoping Review. Virtual Worlds, 2(4), 396-421. https://doi.org/10.3390/virtualworlds2040023

Article Metrics

Back to TopTop