Efficacy of Virtual Reality Simulation in Teaching Basic Life Support and Its Retention at 6 Months

Educational efficiency is the predetermining factor for increasing the survival rate of patients with cardiac arrest. Virtual reality (VR) simulation could help to improve the skills of those undergoing basic life support–automated external defibrillation (BLS–AED) training. Our purpose was to evaluate whether BLS–AED with virtual reality improves the skills and satisfaction of students enrolled in in-person training after completing the course and their retention of those skills 6 months later. This was an experimental study of first-year university students from a school of health sciences. We compared traditional training (control group—CG) with virtual reality simulation (experimental group—EG). The students were evaluated using a simulated case with three validated instruments after the completion of training and at 6 months. A total of 241 students participated in the study. After the training period, there were no statistically significant differences in knowledge evaluation or in practical skills when assessed using a feedback mannequin. Statistically significant results on defibrillation were poorer in the EG evaluated by the instructor. Retention at 6 months decreased significantly in both groups. The results of the teaching methodology using VR were similar to those obtained through traditional methodology: there was an increase in skills after training, and their retention decreased over time. Defibrillation results were better after traditional learning.


Introduction
Data from 27 European countries collected in 2016 showed that the incidence of cardiac arrest (CA) ranged between 19 and 104 cases per 100,000 inhabitants per year. Although 25% of these patients arrived at the hospital with a pulse, only 10.3% survived 30 days after hospital discharge. CA continues to be a large public health problem [1].
Although the population is becoming increasingly aware of and trained in basic life support (BLS) and the use of an automated external defibrillator (AED), fewer than half of cardiac arrests are attended by eyewitnesses and only a minority of CA cases receive adequate application of these techniques from first responders. Cheng et al. [2] suggested a deepening of educational efficiency and its implementation in the general population, especially in students and basic education centers, relatives of those at risk of CA, etc. This is one of the predetermining factors for increasing patient survival rates. In the same sense, a group of experts of the American Heart Association (AHA) concluded that one of the limitations of the 2015 guidelines [3] was the scarce optimization of educational strategies, and this gap continues in 2021 [4].
From the current educational strategies in teaching BLS-AED to traditional training with an instructor as a model, recently, alternative options (mixed or online) have emerged [5], with controversial results due to the progressive decline of knowledge and skills [6] resulting in suboptimal clinical care and poor survival outcomes [2]. It is important to develop new teaching techniques to update and maintain knowledge and skills in BLS-AED [7]. Virtual reality (VR) is one such option, with numerous publications since 2014 related to the training in BLS-AED [8], most of which are focused on student satisfaction [9], as well as instructors [10,11] or observational studies [12].
The evaluation of new educational techniques must also include the economic costs, the simplification of equipment, student facilities, especially those related to updating and maintaining knowledge, and finally the improvement in clinical practice with a reduction in morbimortality.
For VR to be introduced and recommended as a teaching methodology, its effectiveness must be validated. The main objective of our study was to evaluate whether training in BLS-AED with VR improves the skills and satisfaction of health sciences students with respect to traditional training immediately after the course and their retention at 6 months.

Materials and Methods
We conducted an experimental study of teaching innovation in BLS-AED that compared the results between a control group, CG (traditional training), and an experimental group, VG (training with VR). The study was approved by the Ethics Committee of the University.
The population consisted of first-year university students from the School of Health Sciences (Medicine, Nursing and Psychology) in the 2020-2021 academic year.
A representative sample presented with "non-inferiority analysis" and was obtained from the study conducted by the same group of researchers to compare 2 teaching methodologies in BLS-AED [13], where a minimum of 40 students per group was sufficient to obtain conclusive results after the training and at 6 months.
Students who had attended a BLS course in the last 3 years or did not sign the informed consent form were excluded. The students were randomized with QuickCalcs software (https://www.graphpad.com/quickcalcs/ accessed on 6 September 2020).
The experimental and traditional training randomized groups received official training from the European Resuscitation Council (ERC) based on a 4-step methodology [14] of 4 teaching hours modified by the training recommendations of the COVID-19 pandemic (avoid forehead-chin maneuver and approach to assess breathing, and minimize the risk of infection by placing a cloth or towel over the mouth of the mannequin before performing chest compressions and defibrillation) [15].
The classrooms were large, well ventilated and adequately disinfected before and after the training. The instructor-student ratio was 1:6. Each student worked with a Little Anne Laerdal ® (Laerdal, Madrid, Spain) mannequin with feedback.
The outline of the course is shown in Table 1. For the integrated simulations, 6 cases were used that were completely solved on an individual basis in a 15 min period. The rest of the students observed the performance of their classmates or practiced on their own mannequins. The two groups received the same training, with the only difference being that the integrated simulations were performed in the virtual group with the help of VR. The SVB-AED training tool in VR was developed by the company LUDUS ® under scientific supervision. Designed and implemented for use in VR, the application allows interaction with the 3D virtual environment with six degrees of freedom (6DoF). The interaction with the tangible elements of the virtual environment is performed with "leap motion", technology that monitors the movement of the hands. The software is compatible with HTC Vive PRO and Cosmos; Oculus Rift; and HP Reverb. The instructor began by explaining the use of the platform, the placement of the glasses, the permitted movements and the environment. The software contains six scenarios (one per student, identical to the CG), and once explained by the instructor, all students could perform BLS maneuvers on their mannequin or watch their classmate using VR complete the maneuver on a big screen. The simulation ended with a structured debriefing [16,17]. Each simulation lasted 15 min.
Prior to the course, students completed a sociodemographic questionnaire (age, sex, faculty, etc.), estimating their knowledge and skills in BLS-AED prior to training, selfassessed with a Likert scale of 0-10 and their impression of the training ability of the course. To assess the degree of satisfaction with the course, a 10-item questionnaire with a Likert scale (0-10) was created, including an open-ended question to determine students' opinions of the new training methodology.
The students were evaluated through a simulated case using 3 instruments after the training and after 6 months: 1.
For the evaluation of theoretical knowledge, a multiple-choice question (MCQ) was used by the local scientific society which provides accreditation to students participating in BLS courses. This MCQ is used in different published articles [13,18].

2.
For the evaluation of practical skills, which was conducted by the instructor, a validated grid was used in the process of publication with acceptable psychometric properties (Cronbach's alpha of 0.72 and a KMO statistic value of 0.719) to score the practical skills using the BLS-AED algorithm. This grid was adapted to the pandemic situation, replacing the assessment of "see, hear and feel breathing" with the following variables: Put the hand on the patient's thorax, turn the victim's head and cover their mouth with a handkerchief (the maximum score is 16 points and the minimum is 0 points).

3.
For the evaluation of technical skills, the data monitored by the intelligent QCPR mannequin from Laerdal ® on quality cardiopulmonary resuscitation (CPR) were used (the software was configured to evaluate only chest compressions for 2 min).
The qualitative variables are presented as absolute frequencies (n) and percentages (%), and the quantitative variables are presented as the mean and standard deviation (SD) or median and quartile interval (QI). For the statistical comparison between groups, Student's t-test (for independent data) or Mann-Whitney U tests were used according to the normality of quantitative variables, whereas the chi-squared test was used for qualitative variables. For the comparison of means between the values obtained at the end of the course and at 6 months, Student's t-test was used for paired data. The data were converted to a scale from 0 to 10 to facilitate their interpretation. Statistically significant differences were p-values equal to or less than 0.05. For the statistical analysis, the software SPSS for Windows version 18 was used.

Results
A total of 241 students participated in the study. The sociodemographic, academic and perception variables were homogeneously distributed throughout the two groups (Table 2). After the training, there were no statistically significant differences in the evaluation of knowledge or in the practical skills evaluated by the mannequin (Table 3). When breaking down these data, this was observed (Table 4). Competency retention at six months decreased significantly in each of the evaluations performed (Tables 3 and 4) by values of around 1.5 points out of 10 in theoretical knowledge and practical skills and by more than 10 out of 100 in the overall score calculated by the intelligent dummy. At 6 months, global knowledge and skill scores were similar between the two groups. The only statistically significant value between the two groups was found in the correct position of the hands, evaluated by the smart mannequin, which was higher in the CG ( Table 3).
The qualitative satisfaction measurement yields a similar score, with an average of 9.56 points for the 10 questions between the two groups, without significant differences between them (p = 0.99). One hundred percent of the VG commented that VR was a very good training tool, and the open comments organized and grouped: emotional realism (30%), scenic realism (34%) and improvement in teaching methodology (found in 31% of the answers).

Discussion
Since the 1990s, an explosion in the application of virtual environments and related technologies has occurred throughout the healthcare field. Applications of these technologies are being implemented in the following areas: surgical procedures (remote surgery or telepresence, planning and simulation of procedures before surgery), medical therapy, preventive medicine and patient education and medical education and training. Such applications have improved the quality of healthcare, and in the future, they will result in substantial cost savings [4].
Virtual worlds enable the inclusion and practice of activities for experiential learning and the simulation and modeling of complex scenarios, providing opportunities for collaboration and co-creation that cannot easily be experienced through other platforms [12].
Responding to an emergency situation such as a cardiac arrest requires widespread training in BLS-AED for the entire population, especially first responders, relatives of atrisk patients, police officers, firefighters, etc. VR makes scenario creation and collaboration easier and, in the short term, will reduce costs.
Although international guides continue to support traditional face-to-face training in BLS-AED [4], they are beginning to recommend alternative training methodologies such as virtual learning [13], self-training [19], peer tutoring [20], gaming [21] and, over the last decade, VR, the use of which increased due to the recent pandemic [5]. Since the first publications of Sameraro et al. [11], different European organizations have used VR in training and have published their results [22][23][24], although most of the published studies are heterogeneous, with observational designs [12,13].
Our study has the strength of working with a homogeneous and representative sample throughout the study (maintaining a minimum of 40 participants per group at six months), with validated data collection instruments and with a follow-up over time.
In addition, it integrates VR within the traditional classroom methodology (for the VG), using the teaching methodology endorsed by the ERC [14] in order to demonstrate the effectiveness of VR. The articles found are heterogeneous in terms of the teaching methodology used, comparing 20 min of VR training with the same amount of face-toface time [24,25] and another 35 min of VR [26], making it difficult to compare teaching methodologies. Others interpret VR exclusively as online training [5] without comparison between groups. Regardless of the optimal duration, in our opinion, the most important thing is that the system manages to motivate the student, capture their attention and improve their learning [27].
The objective evaluation of skills acquired during training is also heterogeneous in the literature: some use MCQ [11,28], whereas others use mannequin data [23,24], and still others use Likert-type scales to evaluate satisfaction [12]. However, none of them use all the current tools (objective and validated) to ensure the competence of the students in BLS-AED.
After training, knowledge from VR was similar to that obtained from traditional training [8.21 (1.41) vs. 8.44 (1.65); p = 0.24], with an improvement after training in both groups similar to other "blended" methodologies [13] used in the literature. We can confirm that VR has no influence on the acquisition of knowledge in BLS-AED.
In the skills evaluated by the mannequin, no significant differences were observed between the two groups [67.86 (24.99) vs. 64.54 (28.85); p = 0.34]. However, we obtained depth compression results 0.5 mm lower than those obtained in the study by Beom et al. [29] in both groups. Their study was aimed at professional ambulance technicians, which could explain this difference. On the other hand, skills evaluated by the instructor during the simulated case were statistically better after training in the CG [9.10 (1.2) vs. 8.61 (1.48); p = 0.05], mainly influenced by the item "Safe Discharge". We believe that these isolated data have little value both in this study and in the clinic.
The satisfaction obtained in this study is similar to that found in the literature but grouped into three large blocks: the scenic realism that it offers [9] with statements such as, "I think that VR brings that sensation of feeling that it is happening to you" and "It brings an experience, since it is as if you lived the case without imagining it"; realism or emotional security [28], with statements such as, "It provides the assurance that you will know how to react in a similar situation"; or how to improve the teaching methodology [30] with statements such as, "More effective learning" and "You feel much more effective, I think that it is the part in which you learn the most". We believe that VR is capable of taking a qualitative leap in the training of BLS-AED. We believe that simulation is a good tool which allows students to acquire and practice skills without risk of injury or harm to the patient [31]. However, we question whether it is capable of generating the stress and emotional burden that VR produces in these cases [28]. Although we did not conduct a survey on instructors' satisfaction and opinion with regard to VR, all of them believe that any initiative to improve the satisfaction and motivation of the students should be introduced, to the greatest extent possible, within the training in BLS-AED.
Like Smith and Hamilton [32], who support the use of VR simulation as a supplemental tool for teaching students, we consider that VR simulation is a BLS-AED teaching method with similar results compared to traditional face-to-face training. Regardless, Cerezo et al. [27] found that the results are discreetly better. We believe, however, that it provides emotional, situational and reality qualities that no other methodology provides and should be introduced for future generations, in training aimed at teenagers and the general population.
It should be noted that VR training in BLS-AED is still in need of improvement; therefore, it is necessary to dedicate time to the students to explain the operation of the actions that must be carried out for the student to become used to the format. The adaptation to glasses was not well tolerated by all students (in our study, four students became dizzy when using VR), so we think that it is necessary that an instructor trained in VR be dedicated exclusively to students who perform this training and that another instructor is needed to guide the rest of the students. The dynamism of the class could be increased by providing more than one set of VR glasses. Both solutions increase the costs of the courses.
Much more development of technology is needed, not only in terms of cabling, hardware, etc., but also to allow more space for student movement, ensuring that the environment is dynamic (in our case, the AED always appeared behind the virtual element that interacted to save the victim, generating distortion. We believe that this is the reason why there is a difference from the control group) and that the people who appear in action interact with each other and the resuscitator to achieve greater realism and participation [23].
VR must be able to immediately evaluate the student without an instructor's input. The software itself must have a training mode and an evaluation mode.
In addition to improvements in educational performance, it is necessary to demonstrate that these new teaching initiatives are effective in the clinical field. This could be a future research direction.
Six months after the course, the three overall evaluations decreased substantially with respect to the evaluation performed after the training. We have not found articles that measure retention with the use of VR, although this trend is similar to the published literature comparing other methodologies [13]. The introduction of VR did not improve the retention of skills in BLS-AED over time, and hand position was significantly different in the CG [96.38 (16.2) vs. 91.7 (25.3): p = 0.02] when we further examined the evaluation performed by the mannequin. These data were excellent in both groups (greater than 90%).
We must continue to search for a method that obtains more sustained retention over time in this discipline, with VR being a good tool for refresher courses. New technologies, which are accessible to the majority of the population, have great potential to provide society with fast, easy and accessible CPR training that can be performed at home at a low cost [24]. As Semeraro et al. [23] argue, we believe that VR in BLS-AED is a valid and acceptable tool for training programs aimed at general populations, schoolchildren and health professionals, with a gaming approach that is very useful for refreshing knowledge.

Conclusions
The results of the teaching methodology with VR are similar to those of the traditional methodology: there is an increase in competency after training, and its retention decreases over time.
VR has similar results to the integrated simulations within a standard BLS-AED course.
VR technology has room for improvement, and we hope that it will be part of the training in BLS-AED as another tool in the immediate future. Informed Consent Statement: Informed consent was obtained from all the subjects involved in the study.