The Effect of the Degree of Anxiety of Learners during the Use of VR on the Flow and Learning Effect

Abstract

Virtual reality (VR) learning content that provides negative experiences makes learners anxious. Thus, experimental research was conducted to determine how anxiety felt by learners using VR impacts learning. To measure the learning effects, flow, a leading element of learning effects, was measured. Flow has a positive effect on learning as a scale of how immersed an individual is in the work he or she is currently performing. The evaluation method used the empirical recognition scale by Kwon (2020) and the six-item short-form State-Trait Anxiety Inventory (STAI) from Marteau and Becker (1992), which were used in the preceding study. The difference in flow between high- and low-anxiety groups was explored by measuring the degree the study participants felt using an Fire Safety Education Game based on VR that allows learners to feel the heat and wind of the fire site with their skin. As a result of the experiment, no difference in flow was found between the high- and low-anxiety groups that played the same VR game with cutaneous sensation. However, the high-anxiety group who played the VR game with cutaneous sensation showed a higher flow than the group that played the basic fire safety education VR game. Based on these results, the following conclusions were drawn: the closer to reality the VR learning and training system for negative situations is reproduced, the more realistically the learner feels the anxiety. In other words, the closer to reality the virtual environment is reproduced, the more realistically the learner feels the feelings in the virtual space. In turn, through this realistic experience, the learner becomes immersed in the flow more deeply. In addition, considering that flow is a prerequisite for the learning effect, the anxiety that learners feel in the virtual environment will also have a positive effect on the learning effect. As a result, it can be assumed that the more realistically VR is reproduced, the more effective experiential learning using VR can be.

1. Introduction

As virtual reality (VR) technology continues to develop, studies on verifying the effectiveness of learning and training using it have been steadily conducted in various fields. In particular, since the early days of VR research, studies have been continuously conducted to verify the effectiveness of applying immersive VR to science classes because it is possible to experience learning content that is not easy to actually practice and experience through VR.

In the initial stage of the field’s development, Bricken and Byrne (1993) performed an experimental study that explored the education applications of VR at Summer Studies in Virtual Reality so as to assess the potentiality of VR as a tool for learning. The experiment was conducted, in which participants created seven virtual reality spaces and made an expedition into them through seven sessions over seven weeks. The seven VR spaces are the Planetscapes of a futuristic world, Virtual Valley, Cloud Lands, Moon Colony made from the imagination of future Moon, Neighborhood, and Medieval Space Station where participants can experience the past and the future together. Mr. Mountain is a personalized world. The result was that a creative teaching–learning environment was created through VR, and the students quickly learned complex concepts and techniques and understood the conception of presence in VR [1].

In a recent study, Parong and Mayer (2018) compared the results learned through Head-mounted display (HMD) and desktop slideshow in a science class for college students regarding how cells in the human bloodstream work. Although the learning effect of the use of the desktop slideshow was higher than that of the use of HMD, the learners’ interest and motivation were higher in the latter. Based on these results, the possibility of learning using VR was lauded, and it was stressed that while learning using VR does not guarantee improvement in the learning effect at this stage, it showed an improvement of the learners’ interest and motivation, which have a critical impact on the learning effect [2].

Kwon (2019) used HMD-based immersive VR technologies in elementary school science classes to see if it is possible to have experiential learning similar to real life in a virtual space. He developed lunar exploration content that students could not carry out in real life, and conducted comparative experiments. The result was that the better the VR technology’s vividness and interactivity were, the greater the learning effect was, as the learners could have virtual experiences close to the real experiences. Based on the results of such experiment, the findings of such experiment showed that experiential learning with VR is possible and, along with the continuous evolution of technology, experiential learning that offers experience identical to reality in the near future [3].

Furthermore, many researchers conducted studies on the application of VR to science classes, believing that VR-based education provides students with engaging experiences, makes them display more positive attitudes towards a learning topic, and helps them concentrate on the learning content [1,4,5,6].

Aside from science classes, VR is used in various other fields: military education [7,8], emergency responder training [9], fire safety education [10], construction hazard education [11], psychotherapy [12], phobia treatment and rehabilitation [13,14,15], and sports training [16,17]. Similar to science classes, in most of these other fields, VR is applied to experiences that are hard to reproduce, and in areas where actual training is important. In most of the studies on applying VR in these other fields, it was concluded that education and training using VR could positively impact the learning effect in the long term by improving the learners’ interest, motivation, immersion, presence, and flow.

There was a study, however, that raised the question of whether an overly realistic VR experience positively or negatively impacts the learning effect [10]. Having employed the empirical scale shown in

Table 1. Definition of terminology from the empirical recognition scale by Kwon (2020) [10]. Copyright © 2020, Springer-Verlag London Ltd.

Terminology	Definition
Vividness	reproducible richness of mediated environment [18],
Tactile interactivity	The degree to which users can directly interact with a virtual object in virtual space. [3]
Locomotive interactivity	The degree to which users can move inside virtual space through actual walking [3].
Presence	The feeling of a “sense of being there [19]”. The most basic characteristic of VR [19,20,21], which is the criterion for determining whether the experience of VR can be felt like a direct experience [3].
Flow	Named by Csikszentmihalyi (1990) and the scale of how immersed an individual is in the work he or she is currently performing [25]. The prerequisite of learning having a positive effect on the learning effect [26,27,28].
Experientiality	The scale of whether a user’s experience in virtual space is close to indirect or direct experience, derived from a study in Gibbons and Hopkins (1980) [3,24]

, Kwon (2020) conducted a study to verify if the extension of the sensations felt by the learners in a virtual space improves the learners’ vividness, interactivity, presence, experientiality, and flow. Vividness refers to the reproducible abundance of the mediated environment [18], the tactile interactivity corresponds to the degree of direct interaction with virtual objects in the virtual space, and the locomotive interactivity is representative of the extent to which learners can move in the virtual space through physical movements [3]. Presence means the feeling of “sense of being there” [19]. It is the most basic characteristic of VR [19,20,21] and is a criterion for determining if the VR experience can approximate a direct experience that can be achieved in realty, which is different from indirect experience through a medium such as book or TV, and comes from the theory of experiential learning by Dewey (1959; 2004) [22,23]. Experientiality, the scale is drawn from Gibbons and Hopkins (1980) [24], which is to measure whether the user’s experience in a virtual space is close to indirect or direct experience. Lastly, flow is the criterion for measuring how immersed an individual is in the current work [25], and is the criterion for evaluating the improvement of the learning effect because it is the prerequisite for the learning effect [26,27,28].

An experiment was conducted to examine and record the stated variables when users participate in an educational VR game for fire safety. The game provides the users with cutaneous sensations, including temperature and wind. These same variables are also then recorded in the use of VR game that used simple tactile and locomotive interactivity. The experiments showed that if the learners feel the temperature and wind through an actual skin sensation in a virtual environment, the vividness and presence significantly improve while the flow does not [10]. It was concluded that these results were due to the influence of the study participants’ anxiety level on the flow. In the interviews of the study participants where they were asked how they felt while playing the VR game, many users of the fire safety education VR game with cutaneous sensation, to which a device was added to allow the users to feel the temperature and wind, said that they felt scared or insecure because the VR experience felt too realistic. In particular, there were many learners who said that their fear or surprise peaked when their skin actually felt hot in front of a virtual flame. As a result of these interviews, it can be said that when learners virtually experience being in a fire site, the fire site feels so realistic through the hot temperature and wind that it is likely to affect the flow by creating the same feelings of anxiety and fear that can be felt at an actual fire site.

This means that people’s experiences that make them feel anxious, such as that of being at a fire or disaster site or in a battlefield, can have a significant effect on the flow if the experience in the virtual space is extremely realistic, and as the improvement of the flow is a prerequisite for the improvement of the learning effect [26,27,28], it is necessary to verify whether the degree of anxiety that occurs at this time positively or negatively impacts the flow. Therefore, in this study, how the degree of anxiety felt by the learners in a virtual space affects the flow was determined.

2. Literature Review

Anxiety is an emotion characterized by an unpleasant state and the feeling of uneasiness, worry, tension, nervousness, and stress [29,30]. Anxiety is defined as “unpleasant psychological state in reaction to perceived stress concerning the performance of a task under pressure” [31]. In the real world, it is common for people to feel anxiety, and many previous studies addressed the relationship between the anxiety that people feel in the real world and their performance [32].

According to the previous relevant studies, anxiety generally degrades people’s performance in the real world [33], and negatively affects the storage and processing capacity of their working memory [34]. When it comes to studies on anxiety associated with the virtual environment, however, there has been a study on the treatment of anxiety disorders using VR [35,36,37] but there has been no study on how the anxiety that occurs within VR simulations for educational purposes impacts the learners and the learning effect.

To analyze the relationship between anxiety and other variables, the degree of anxiety should be measured. This can be performed through the following methods. First, the degree of anxiety can be measured based on the various behavioral symptoms shown by anxious people, such as fidgeting, shaking one’s legs, biting one’s fingers, and yawning frequently. If these symptoms appear to be significantly different from a person’s normal behaviors, they can be regarded as signs of anxiety [38]. Another way of measuring the degree of anxiety is physiological measurement. When there is anxiety, physical signs appear, such as stiffness of the muscles, increased heart rate, decreased attention, increased blood pressure, increased fatigue, and especially more sweat secretion on the palms, compared to normal times. These are the indications of anxiety [38]. A third way of measuring the degree of anxiety is to measure the chemical changes that occur in the body when a person becomes anxious. For example, the epinephrine in the hormones increases two to three times when a person is nervous compared to when he or she is not [38]. Such indicators, however, do not commonly occur in everyone, and each person has different anxiety symptoms. Moreover, they are not easy to measure quantitatively, so they are impractical to use for measuring the degree of anxiety of a person.

Therefore, the most commonly used method for measuring the degree of anxiety in experimental studies is to make the study participants complete self-reporting questionnaires. This is a method in which the experiment participants evaluate their own anxiety level based on certain questions. It is widely used as a method of measuring the degree of anxiety because it has the advantage of quantified measurement and convenient use in the field. Studies on anxiety measurement scales began to increase rapidly with the development of the Manifest Anxiety Scale (MAS) by Taylor (1956) [39], and have often used the State-Trait Anxiety Inventory (STAI) developed by Spielberger et al., (1983) [30]. STAI is a scale designed to measure people’s state and trait anxiety levels and consists of 40 questions scored based on a four-point Likert scale. It consists of 20 items for measuring the degree of anxiety that a person feels in the current situation (state anxiety) and 20 items for measuring the degree of anxiety that a person usually feels (trait anxiety). As STAI, developed by Spielberger et al., (1983) has too many items, however, Marteau and Bekker (1992) developed a short form of STAI for use in cases where the full form is inappropriate because it has too many items. The short form of STAI reduced the state anxiety items, or those pertaining to the degree of anxiety that a person feels in the current situation, to only six items [40].

Considering the total number of items in the questionnaire, this study employed the short (six-item) form of STAI developed by Marteau and Becker (1992) to measure the degree of anxiety felt by learners in the virtual space, and analyzed the effect of VR experience on the flow when it made the learners anxious because it was extremely realistic.

3. VR Learning Content and Systems Overview

People are very likely to feel fear or anxiety in dangerous environments like fire sites [41]. Even if a fire site is only virtually displayed, the learner is more likely to feel anxiety when the virtual experience feels too real. Therefore, the fire safety education game that was used in a previous study [10] was used for the VR content and systems in this study. The gameplay video can be found in Video S1 (see the Supplementary Materials).

As shown in Figure 1, the fire safety education game is a VR education game that reproduces a fire situation in a school laboratory. The game provides users with training in how to respond to the outbreak of a fire. The techniques covered include the use of a fire extinguisher, how to reach ground level, how to escape using an air mattress, and how to reach a rescue helicopter. While providing positive instruction it also warns users of what not to do in certain situations, such as using an elevator.

For the VR systems, a basic fire safety education VR game based on tactile and locomotive interactivity (see Figure 2a) and a fire safety education VR game with cutaneous sensation that allows the users to feel the heat and wind on their skin (see Figure 2b) were used. In the preceding study [10], a considerable number of participants said that they became anxious when they felt the heat and wind while playing the fire safety education VR game with cutaneous sensation. Therefore, through the experiments using the two aforementioned systems, it was expected that a difference in flow would be found between the high- or low-anxiety groups while they were experiencing a fire simulation through the fire safety education VR game with cutaneous sensation. At the same time, a difference in flow would be found between the study participants who played the basic fire safety education VR game, based on tactile and locomotive interactivity.

• High-anxiety group: the group that feels the virtual fire experience in the virtual space as closer to reality and thus feels a strong sense of anxiety while playing the fire safety education VR game
• Low-anxiety group: the group that perceives the virtual fire experience in the virtual space as unrealistic and thus do not feel anxiety as strongly while playing the fire safety education VR game

4. User Study

An experiment was conducted to ascertain the level of anxiety felt by learners taking part in the VR simulation as this, in the real world, affects flow. The experiment was carried out in two parts.

4.1. Research Design

First, as shown in

Table 2. User profile breakdown from the previous study [10]. Copyright © 2020, Springer-Verlag London Ltd.

Description	Number of Users	Age	Gender		Percentage (%)
	47	20 to 25 Years Old	Male	Female	100
VR with Cutaneous Sensation	24	20 to 25 years old	18	6	51.1
VR	23	20 to 25 years old	19	4	48.9

, 47 college students, aged from 20 to 25, participated in the compared experiment with VR with cutaneous sensation and basic VR learning effect [10]. The result of the interview showed that some participants who had experienced VR with cutaneous sensation felt the anxiety by experiencing hot temperature realistically.

As such, in the present study, the participants were made to play the fire safety education VR game with cutaneous sensation, which was expected to make many learners feel very anxious, and then their anxiety levels were measured to see if there was a difference in flow between the high- and low-anxiety groups.

In the previous experiment, however, it was uncertain if the flow improved in the high- or low-anxiety group. Therefore, the present study established a two-tailed hypothesis rather than a one-tailed one.

In addition, it is not possible to assume that there will be differences in the variables of vividness, tactile interactivity, locomotive interactivity, presence, and experientiality because the high- and low-anxiety groups were to use the same VR game with cutaneous sensation. Therefore, the following first hypothesis was established:

Hypothesis 1 (H1).

When experiencing VR under the same conditions, the high- and low-anxiety groups will show a difference in flow.

Second, prior research [10] demonstrates that the flow of those who experienced cutaneous sensations during the VR game was higher than those who did it without. However, the differences were not significant. This may be attributed to either the lower flow of the high-anxiety group negatively affecting the overall flow, or the lower flow of the low-anxiety group also affecting the overall flow value. Therefore, the flows of the high-anxiety group (among the study participants who played the fire safety education VR game with cutaneous sensation) and the study participants who played the basic fire safety education VR game were compared, and the flows of the low-anxiety group and the study participants who played the basic fire safety education VR game were also compared. Based on these comparisons, the relationships between high- and low-anxiety and high and low flow were analyzed.

Hypothesis 2 (H2).

According to the degree of anxiety that occurred during the use of the VR game with cutaneous sensation, there will be a difference in flow between the users of the VR game with cutaneous sensation and the users of the basic VR game.

4.2. Experiment Procedure

To verify the hypothesis, an experiment and survey were carried out at a university in South Korea (Busan) from 20 January to 24 January, 2020. In performing the experiment, all the necessary safety measures were taken so as to reduce the possibility of users experiencing injury or simulator sickness. In this regard, the experiment’s plan was approved by the Institutional Review Board (IRB).

According to the experiment results from the previous study [3], simulator sickness is relieved when moving around with actual walk within virtual space, compared to moving around using a controller. Therefore, it is very unlikely that simulator sickness will occur in this experiment. Nevertheless, in case of simulator sickness, the experiment was conducted by ensuring and providing a space for participants to sit down or lie down for a while, and under conditions in which an emergency center could be called in case of a more urgent situation.

The experiment was conducted in three stages in total. First of all, the practice stage for training VR controllers and Treadmill was implemented before starting the fire safety education game in earnest. The Virtuix-Omni Treadmill used in the experiment works in a way that the slippery soles of the shoes slip over the Treadmill to guide walking. Because it is a VR mobile device that none of the participants in the experiment have used, it is difficult for them to walk on the Treadmill naturally from the beginning. Therefore, the participants had to practice the basic steps of walking. Although there were differences among them, most participants were generally able to walk naturally to a certain degree within at least three minutes.

After the completion of the practice stage, the fire safety education game started. The participants escape the virtual fire environment while learning how to deal with fires in real time according to the voice of the guide in VR. The learner can choose one of three situations: escape to the ground, evacuate to the rooftop, escape using an air mattress, and escape using a rescue helicopter. If he or she takes an elevator, which is a prohibited action in the event of a fire, he or she will die and have to start again on the corridor stage. It took five to ten minutes for each participant to play the learning game.

After completion of the experiment, participants fill out a questionnaire made by the empirical recognition scale of Kwon (2020) [10] and the short form of STAI of Marteau and Bekker (1992) [40]. Finishing the questionnaire took approximately ten to fifteen minutes.

4.3. Experiment Participants

The learning content of the experiment was centered on the correct actions and techniques in the event of a fire in a university laboratory. Thus, university students (20–25 years old) were selected as the subjects. A total of 39 participated in the experiment: 26 in the VR game with cutaneous sensations (22 males, 4 females) and 13 in the VR game that utilized tactile and locomotive interactivity (10 males, 3 females). In addition, most of the study participants (34) had experienced VR content before the study. As such, the two study groups had similar distributions in terms of gender and VR experience.

Table 3. User profile breakdown (For high- or low-anxiety groups, they are grouped by the mean of measuring anxiety).

Description	Number of Users		Age	Gender		Percentage (%)
	Anxiety	39	20 to 25 Years Old	Male	Female	100
VR with Cutaneous Sensation	High	13	20 to 25 years old	11	2	33.3
	Low	13	20 to 25 years old	11	2	33.3
VR		13	20 to 25 years old	10	3	33.3

shows the profiles of users participated in the experiment.

4.4. Measurements

The empirical recognition scale that was used in the study by Kwon (2020) was used in the present study for the comparative analysis of the conclusions drawn from the preceding study [10] and those drawn from the present study, and the degree of anxiety was additionally measured.

Kwon (2019) constructed the empirical recognition scale to measure the degree of direct recognition of experience in the study of verification and effectiveness of experiential learning using HMD-based impressive VR technologies [3]. The empirical recognition scale consists of vividness, tactile interactivity, locomotive interactivity, presence, flow, and experientiality [3]. Kwon (2019) established these items on the following grounds:

First of all, it was judged that presence measurement is important to determine whether learning experience through VR can be recognized as reality and thus achieve the effectiveness of experience learning. Furthermore, according to Steuer (1993), vividness and interactivity are factors that affect presence. Kwon (2019) broke interactivity into tactile interactivity and locomotive interactivity [3]. Bricken and Byrne (1993) emphasize tactile interface and mobility interface for location and directional sense in addition to visual and auditory interface to actively enter virtual environments created by computers [1]. Based on these distinctions, interactivity was fragmented into tactile interactivity and locomotive interactivity [3].

Based on a number of prior studies [42,43,44], flow was also judged to be a key characteristic of VR. Because flow is a leading factor that has a positive effect on the learning [42,43,44], it is considered to be an item to be measured for the verification of learning effectiveness.

Experientiality is a factor derived from the study of Gibbons and Hopkins (1980) due to the need to specifically measure phases of experience. Experientiality consists of three phases: feeling like watching a movie or TV as an item for the purpose of measuring the level at which a learner’s learning experience is at between indirect and direct experience; feeling like looking at a real environment and things; and feeling like actually walking around in space and touching things [3].

For presence, the scale from Schubert et al., (1999; 2003), which has been verified in many prior studies, is used [45,46,47]. Vividness, tactile interactivity, and locomotive interactivity were constructed by items drawn from factors contributing to presence in Witmer and Singer (1998) [48]. Flow was drawn from the scale from Jackson and Marsh (1996), which has been tested by many prior studies [49]. Experientiality was constructed and drawn from the study of Gibbons and Hopkins (1980) [24].

Presence has a subjective measurement method in the form of self-reporting to explore the feelings of reality that people have in a given environment, and an objective measurement method that focuses on observable objects or performance, such as pulse, breath, and surprised reactions [50]. However, because presence is subjective, subjective statements are typically accepted as the basic measurement method [51], and therefore, on this ground, the post-event survey questionnaire was employed to measure presence and the scale of Schubert et al. (1999, 2003) was used [45].

However, in the previous study, variables such as vividness, tactile interactivity, locomotive interactivity, presence, flow, experientiality are measured. If too many questions are configured, there could be a risk that the participants in the experiment will not be able to faithfully respond to the survey. Thus, the presence scale theoretically validated in the preceding study consisted of three questions, the minimum number to satisfy the reliability test [3,10].

Schubert et al., (1999, 2003) achieved four major components of spatial presence, involvement presence, experienced realness, and general presence through factor analysis with two experiments after constructing a survey questionnaire with a review of many prior studies. Spatial presence means not looking at a screen or manipulating a virtual object outside the screen, but rather a feeling of manipulating and moving things directly within a virtual space. In other words, it means a sense that does not recognize a medium such as HMD or TV and seems to exist physically in the virtual space and is deeply related to the concept of non-medium. Involvement presence relates to the obliviousness to reality and the perception of presence through paying attention to the virtual environment and experience in it. Experienced realness refers to the measurement of how the virtual environment is realistically perceived. Finally, general presence can be regarded as a result of integrating all of them, meaning the concept of “being there”: a feeling that a physical body exists somewhere else rather than where it really exists. Among these, however, experienced realness was excluded because its questions for measurement are composed of items of vividness. Three representative questions from the other three presence factors respectively are selected and taken as measurement. Following the survey on those three questions of the presence, a reliability analysis was conducted and the results satisfied the reliability, as shown in

Table 4. Reliability of the factors.

Factor	M	Cronbach’s α
Anxiety	2.11	0.63
Vividness	4.20	0.89
Tactile Interactivity	4.49	0.79
Locomotive Interactivity	4.09	0.84
Presence	4.05	0.70
Flow	4.27	0.64

.

For measuring the degree of anxiety, the short form of STAI that was developed by Marteau and Bekker (1992) [40] was used in the present study. As mentioned earlier, Marteau and Becker (1992) developed a scale that reduced the state anxiety items of Spielberger et al.’s full-form STAI, which measure the degree of anxiety that a person feels in the current situation, to only six items, for use when it is inappropriate to use the full form of STAI developed by Spielberger et al. (1983). The short-form scale developed by Marteau and Becker (1992) was used in the present study because the full-form STAI has too many items for measurement purposes. As the form was originally developed according to a 4-point scale, it was replicated here so as to provide continuity and secure the reliability of the results. The contents of the questionnaire can be found in Table S1 (See Supplementary Materials).

5. Results

The results of the analysis of the experiments and surveys are as follows. The reliability of the measurement variables was first analyzed, and the study hypothesis was then verified.

5.1. Reliability Analysis

The accuracy and precision of the variables were measured through a reliability analysis. As seen in Table 4, this demonstrated that the Cronbach α value for all six variables was higher than 0.6, thus demonstrating the reliability of the results.

5.2. Verification of the Hypotheses

To see if there is a difference in the flow between groups who feel high or feel low anxiety when experiencing VR under the same conditions, and whether there is a difference in the flow that users of VR with cutaneous sensation or users of basic VR feel, depending on the degree of anxiety that occurs when using VR with cutaneous sensation, Hypothesis 1 and Hypothesis 2 were tested.

5.2.1. Research Hypothesis 1 Verification Result

Based on the average anxiety degree value (M = 2.28) of the study participants who played the fire safety education VR game with cutaneous sensation, the said study participants were divided into the high- and low-anxiety groups. The flow between these two groups and the other experience recognition variables were analyzed, including the vividness, tactile interactivity, locomotive interactivity, presence, and experientiality.

The number of samples in each study group was less than 30. Moreover, the normality test revealed that the Kolmogorov–Smirnov and Shapiro–Wilk values for all variables were p < 0.05 and thus did not satisfy normality. Accordingly, the Mann–Whitney U test was adopted as it is a non-parametric method that analyzes differences between groups featuring low sample numbers and a lack of normality.

As shown in

Table 5. Results of the Mann–Whitney U test analysis for the high- and low-anxiety groups for all the variables while playing the fire safety education VR game with cutaneous sensation.

Variable	Anxiety	N	Mean Rank	Sum of Rank	Mann–Whitney U	Wilcoxon W	Z	p
Flow	High	13	15.88	206.50	53.50	144.50	−1.66	0.10
	Low	13	11.12	144.50
Vividness	High	13	16.04	208.50	51.50	142.50	−1.80	0.07
	Low	13	10.96	142.50
Tactile Interactivity	High	13	15.38	200.00	60.00	151.00	−1.36	0.18
	Low	13	11.62	151.00
Locomotive Interactivity	High	13	16.15	210.00	50.00	141.00	−1.81	0.07
	Low	13	10.85	141.00
Presence	High	13	15.69	204.00	56.00	147.00	−1.50	0.13
	Low	13	11.31	147.00
Experientiality	High	13	14.50	188.50	78.00	169.00	−0.43	0.67
	Low	13	12.50	162.50

, the analysis showed no difference in flow between the high- and low-anxiety groups among the study participants who played the fire safety education VR game with cutaneous sensation, contrary to the study hypothesis (Mann–Whitney U = 53.50; Wilcoxon = 144.50, p > 0.05). As can be seen in the boxplots in Figure 3, however, the median value of the high-anxiety group was higher than that of the low-anxiety group, and the distribution of the low-anxiety group was widely spread while that of the high-anxiety group was concentrated on the top side, resulting in a high flow in the high-anxiety group on the whole.

The other variables—vividness, tactile interactivity, locomotive interactivity, presence, and experientiality—demonstrated no significant difference. This reaffirms the suggestion that there is little, if any, difference in variables associated with experience recognition when learners experience the same sensations during a VR game.

5.2.2. Research Hypothesis 2 Verification Result

First, the flows of all the study participants who played the fire safety education VR game with cutaneous sensation and of those who played the basic fire safety education VR game were compared with the other results of the experience recognition survey.

As shown in

Table 6. Results of the Mann–Whitney U test analysis for the variables of the group of study participants who played the fire safety education VR game with cutaneous sensation and the group that played the basic fire safety education VR game.

Variable	Anxiety	N	Mean Rank	Sum of Rank	Mann– Whitney U	Wilcoxon W	Z	p
Flow	VR game with cutaneous sensation	26	22.21	577.50	111.50	202.50	−1.76	0.08
	Basic VR game	13	15.58	202.50
Vividness	VR game with cutaneous sensation	26	23.73	617.00	72.00	163.00	−2.98	0.00 **
	Basic VR game	13	12.54	163.00
Tactile interactivity	VR game with cutaneous sensation	26	19.98	519.50	168.50	519.50	−0.02	0.99
	Basic VR game	13	20.04	260.50
Locomotive interactivity	VR game with cutaneous sensation	26	21.52	559.50	129.50	220.50	−1.20	0.23
	Basic VR game	13	16.96	220.50
Presence	VR game with cutaneous sensation	26	22.50	585.00	104.00	195.00	−1.97	0.05 *
	Basic VR game	13	15.00	195.00
Experientiality	VR game with cutaneous sensation	26	21.65	563.00	123.00	217.00	−1.55	0.12
	Basic VR game	13	16.69	217.00

* p < 0.05; ** p < 0.01.

, there was no difference in flow between the two study groups (Mann–Whitney U = 111.50; Wilcoxon = 202.50; p > 0.05) while there were significant differences in vividness (Mann–Whitney U = 72.00; Wilcoxon = 163.00; p < 0.05) and presence (Mann–Whitney U = 104.00; Wilcoxon = 195.00; p < 0.05). These are consistent with the results of the previous study [10].

The boxplots in Figure 4 show, however, that the median value and interquartile range (IQR) of the group of study participants who played the fire safety education VR game with cutaneous sensation were higher than those of the group that played the basic fire safety education VR game in terms of the flow variable, along with the vividness and presence variables, resulting in the high flow of the group that played the fire safety education VR game with cutaneous sensation on the whole.

As shown in

Table 7. Results of the Mann–Whitney U test analysis for the variables of the high-anxiety group while playing the fire safety education VR game with cutaneous sensation and the group that played the basic fire safety education VR game.

Variable	Anxiety	N	Mean Rank	Sum of Rank	Mann– Whitney U	Wilcoxon W	Z	p
Flow	VR game with cutaneous sensation (high anxiety)	13	16.85	219.00	41.00	132.50	−2.31	0.02 *
	Basic VR game	13	10.15	132.00
Vividness	VR game with cutaneous sensation (high anxiety)	13	18.19	236.50	23.50	114.50	−3.22	0.00 **
	Basic VR game	13	8.81	114.50
Tactile interactivity	VR game with cutaneous sensation (high anxiety)	13	14.58	189.50	70.50	161.50	−0.76	0.45
	Basic VR game	13	12.42	161.50
Locomotive interactivity	VR game with cutaneous sensation (high anxiety)	13	16.23	211.00	49.00	140.00	−1.87	0.06
	Basic VR game	13	10.77	140.00
Presence	VR game with cutaneous sensation (high anxiety)	13	17.00	221.00	39.00	130.00	−2.36	0.02 *
	Basic VR game	13	10.00	130.00
Experientiality	VR game with cutaneous sensation (high anxiety)	13	15.35	199.50	60.50	151.50	−1.46	0.14
	Basic VR game	13	11.65	151.50

* p < 0.05; ** p < 0.01.

, however, in the analysis and comparison of the measurements from the high-anxiety group among the study participants who played the fire safety education VR game with cutaneous sensation and the group that played the basic fire safety education VR game, the variables of vividness (Mann–Whitney U = 23.50; Wilcoxon = 114.50; p < 0.05) and presence (Mann–Whitney U = 39.00; Wilcoxon = 130.00; p < 0.05) showed a significant difference, and the flow variable showed significant results (Mann–Whitney U = 41.00; Wilcoxon = 132.50; p < 0.05).

The boxplots of the flow variable in Figure 5 show that the median value and IQR of the group of study participants who played the fire safety education VR game with cutaneous sensation were significantly higher than those of the group that played the basic fire safety education VR game.

On the other hand, as shown in

Table 8. Results of the Mann–Whitney U test analysis for all the variables of the low-anxiety group while playing the fire safety education VR game with cutaneous sensation and the group that played the basic fire safety education VR game.

Variable	Anxiety	N	Mean Rank	Sum of Rank	Mann– Whitney U	Wilcoxon W	Z	p
Flow	VR game with cutaneous sensation (low anxiety)	13	14.58	189.50	70.50	161.50	−0.73	0.46
	Basic VR game	13	12.42	161.50
Vividness	VR game with cutaneous sensation (low anxiety)	13	16.27	211.50	48.50	139.50	−1.89	0.06
	Basic VR game	13	10.73	139.50
Tactile interactivity	VR game with cutaneous sensation (low anxiety)	13	12.38	161.00	70.00	161.00	−0.79	0.43
	Basic VR game	13	14.62	190.00
Locomotive interactivity	VR game with cutaneous sensation (low anxiety)	13	13.81	179.50	80.50	171.50	−0.21	0.84
	Basic VR game	13	13.19	171.50
Presence	VR game with cutaneous sensation (low anxiety)	13	15.00	195.00	65.00	156.00	−1.02	0.31
	Basic VR game	13	12.00	156.00
Experientiality	VR game with cutaneous sensation (low anxiety)	13	14.96	194.50	65.50	156.50	−1.13	0.26
	Basic VR game	13	12.04	156.50

, there were no significant differences in all the variables, including the flow (Mann–Whitney U = 70.50; Wilcoxon = 161.50; p > 0.05), that were measured from the low-anxiety group among the study participants who played the fire safety education VR game with cutaneous sensation and the group that played the basic fire safety education VR game.

The boxplots of the flow variable in Figure 6 show that the median values of the two study groups were identical, and the distribution ranges were very similar to the IQR.

The results of the experiments in this study allow for the following interpretations. As shown in Table 6, there was no significant difference in flow between the entire group of study participants that played the fire safety education VR game with cutaneous sensation and the group that played the basic fire safety education VR game. Figure 4, however, shows that the median value and IQR of the group of study participants that played the fire safety education VR game with cutaneous sensation were higher than those of the group that played the basic fire safety education VR game. Therefore, it is highly likely that the high- or low-anxiety group hindered the flow in the situations where overall significant results could have been produced.

The aforementioned assumption is explained in the analyses of Table 7 and Table 8. In the comparison of the high- and low-anxiety groups among those who played the fire safety education VR game with cutaneous sensation to the group that played the basic fire safety education VR game, the flow was found to be high in the high-anxiety group, as shown in Table 7, and low in the low-anxiety group, as shown in Table 8. These results show that VR environments that provide situations that can cause near-realistic anxiety, such as fire and disaster, do not interfere with the learners’ learning but rather make them focus on and become immersed in the situation at hand.

6. Discussion

The experiments and analyses that were performed in this study revealed no significant difference between the two study groups, contrary to the hypothesis that there would be a difference in flow between the high- and low-anxiety groups that played the fire safety education VR game under the same conditions. There was a significant difference in flow, however, between the high-anxiety group among the study participants who played the fire safety education VR game with cutaneous sensation and the group that played the basic fire safety education VR game. Given this, it can be said that learners can feel greater anxiety when using VR-based learning content that can make them feel much closer to reality, and in this case, the said learners are more deeply immersed in the virtual environment.

In other words, while there may be concerns that the anxiety generated by the reproduction of a virtual environment close to reality may hinder the learners’ flow and consequently the learning effect, it is judged that it will positively impact the learning effect based on the theory that the flow is a prerequisite [26,27,28] for the learning effect, by making the learners more immersed in the situation at hand.

The reason why anxiety does not interfere with the flow but rather strengthens it further can be found in the flow theory. The flow scale was derived from the flow theory by Mihaly Csikszentmihalyi, whose study shows that people can experience flow in a state of immersion in all situations of everyday life that we can experience, including composition, rock climbing, dancing, yachting, chess, art appreciation, listening to music, taste pleasure, and so on [25]. It is natural to experience flow in situations such as art appreciation and listening to music that are very stable and users do not feel anxiety. However, we feel the flow even if we also feel the anxiety because of risks such as rock climbing [25]. That is, whether or not users feel anxious in reality is irrelevant to the flow.

Whether the reproduced situation is a safe place or a place with a risk factor in a virtual situation implemented by VR is not a factor that affects the flow that users feel. Rather, whether or not the VR environment is elaborately reproduced is a key factor that makes users experience flow. Thus, the experimental results show that anxiety improves flow. This does not indicate simple causality. It proves that the more realistically reproduced an environment with risk factors is, such as a fire site, the more realistic the learner feels. Thus, he or she feels anxiety and flow like a feeling in the real world. Rather than directly affecting the flow, it can be said that the flow occurs when the anxiety itself faithfully reproduces the situation using VR in which the anxiety can occur.

After all, the anxiety that users feel in this study can be seen as a criterion for evaluating the fidelity of reproduction when reproducing dangerous environments such as fire sites into VR. Therefore, the higher the anxiety, the closer the virtual environment is to reality, and the flow felt by the learner can be improved to ensure an improvement in learning effectiveness. As a result, the study proves once again that the implementation of VR as an experience close to reality can improve learning effectiveness through effects such as experiential learning [3,10].

It is very important that when learning using VR, learners can experience the feeling that they may actually have in an environment where they have to use acquired knowledge beyond just acquiring knowledge. For example, even if people know how to deal with negative situations such as fire, disaster, and battlefields by learning, in real life, they may not be able to carry out what they have learned due to the feelings and the atmosphere of the scene. However, if the user simulates a situation as close to reality as possible so that the user can feel the sense in the virtual space, it is expected that the learner will be able to enjoy two effects of realistic training and improvement of learning effects by experiencing feelings in the virtual environment, which are similar to that of reality.

The possibility that the learning effect will be hampered, however, cannot be ruled out. If the learner’s sensation in a negative situation is excessive, it can constitute a reenactment of a sensation that can cause pain and fear, such as sensations that are beyond simple tactile, temperature, and wind sensations, the shock and pain of being shot in military training, or the breathing difficulties that can occur when one falls into the water. In such cases, it is believed that ethical considerations are necessary. Therefore, further research must be performed on how much sensory information should be provided to learners to create flows by providing them with an optimal experience [25] and improve the learning effect in reproducing extreme situations.

7. Conclusions

In this study, the effect of the anxiety felt by learners on the flow and learning effect was analyzed. Based on the results of the experiments and analyses that were performed in the study, it can be concluded that although the differences in the degree of anxiety felt by the learners while playing the same virtual reality (VR) game does not affect the flow, the closer to reality the VR learning and training system for negative situations is, the more the learners will feel the same anxiety that can be felt in reality, and the more the learners will be immersed in the virtual simulated situation, resulting in improved flow and thus a positive impact on the learning effect.

These results mean that the closer the virtual environment is reproduced to reality, the deeper the learner experiences the virtual space, the deeper he or she experiences flow, and as a result, the learning effect is improved. In other words, the more realistically VR is reproduced, the more effective the experiential learning using VR can be. In real life, especially as in the case of experiments in this study, when learning about disasters or fire sites where negative sense such as anxiety can occur and the effects of applying learning in real situations may be reduced, it would be much more efficient learning if the virtual environment is faithfully reproduced as close to reality as possible and these negative emotions are experienced in advance.

However, the empirical recognition scale used in this study [3] consists of multiple variables based on the findings from the preceding study, thus it has a limitation using reduced items on the presence scale used in the preceding study. Of course, the validity of the questions has been verified through reliability tests, but since presence is an important factor in VR experience, it will be necessary to derive more reliable results in future studies by measuring how much the learner feels being in virtual space using the overall scale of presence [45], which is widely verified and used. Therefore, further verification will be carried out in future studies using the overall scale of the presence [45] after obtaining more samples, and even more reliable verification results will be obtained by combining verification using extended measurement methods of brain waves, biometric signals, etc.

Moreover, to utilize VR in the future to carry out realistic experiential learning about negative situations more effectively, it is believed that the development of a negative-sensory-element technology that can actually be felt or perceived through the various senses in a virtual space, beyond a simple tactile, temperature, and wind level, will be necessary.