Predictors of Engagement in Virtual Reality Storytelling Environments about Migration

Abstract

Virtual reality (VR) environments provide a high level of immersion that expands the possibilities for perspective-taking so that people can be in the shoes of others. In that regard, VR storytelling environments are good for situating people in a real migration story. Previous research has investigated how users engage in narrative VR experiences. However, there is a lack of research on the predictors of engagement in VR storytelling environments. To fill this gap in the literature, this study aims to identify the predictors of engagement when VR is used as a medium to tell a migration story. A structural model based on hypotheses was validated using partial least squares structural equation modeling (PLS-SEM) with data from the interaction of 212 university students with a tailor-made VR experience developed in Unity to engage people in two migration stories. The results show that our model explains 55.2% of the variance in engagement because of the positive influence of immersion, presence, agency, usability, and user experience (UX).

1. Introduction

Engagement can be viewed from the behavioral, cognitive, and emotional dimensions [1], which refer to humans’ actions, thoughts, and feelings, respectively [2]. Engagement is a multifaceted and multidimensional concept. One of the well-known definitions of engagement is the one suggested by Fredricks, Blumenfeld, and Paris [3]: engagement is a multidimensional concept with three dimensions. The first one is the behavioral dimension, which refers to the involvement in academic and extracurricular activities and is linked to positive learning outcomes. The second dimension is the emotional one, which refers to the positive and negative affective reactions in the classroom, to the school, and to teachers. The third dimension is the cognitive dimension, which refers to making the best effort to succeed in learning activities by using appropriate self-regulated learning strategies. In the context of this study, we define engagement as the degree of involvement in an activity created by a combination of personal factors and technological affordances of the medium used for an interactive experience. Engagement has been investigated in different contexts, such as education [4,5,6], health [7,8,9], and consumer behavior [10,11], among others. In recent years, several studies have reported the use of virtual reality (VR) as a medium for analyzing the effects of immersive technologies on user engagement. Moreover, in the VR industry, engagement can be viewed as a key experience indicator [12]. In this article, we explore factors influencing engagement in VR, particularly in a VR storytelling environment involving human migration stories.

Human migration is a way in which people escape from pressures and problems to find new opportunities that cannot be encountered in the original contexts, and it represents biographical and contextual partitions in a human’s lifetime [13]. Telling migration stories helps empower people and make them aware of the realities that many migrants go through. In this context, storytelling has been used as a strategy for narrating vivid ideas, beliefs, experiences, and life lessons [14]. Its reflexive nature gives people the opportunity to bring back memories of their own or others’ lived experiences [15]. Moreover, creativity is a fundamental factor in constructing digital storytelling, and a story can be represented in multiple formats using different technologies. For instance, Karapakdee and Wannapiroon [16] introduce a game learning experience based on immersive digital storytelling addressing the topic of game developer competency. De Paolis et al. [17] developed a mobile augmented reality (AR) application that gives visitors information about the story and meaning behind the frescoes inside a basilica.

Despite the importance of the influence that engagement might have in VR scenarios, there remains a paucity of evidence on the factors influencing engagement in VR storytelling scenarios. There have been few studies that have assessed some constructs related to engagement in VR in other contexts. Nonetheless, to advance research in the field, in this study, we investigate how other factors associated with a VR experience might predict engagement in a migration storytelling environment. Thus, this study aims to identify the predictors of engagement during participants’ use of a VR storytelling environment. In that regard, this study provides insights into the factors that influence participants’ engagement. The main contribution of this study lies in the identification of how some aspects of the VR experience positively affect engagement. In this context, the main research question that drives this study is: What are the predictors of engagement in VR storytelling environments?

The rest of this paper is organized as follows: Section 2 describes the related work. Section 3 describes the hypotheses’ development. Section 4 describes the methodology, participants, research design, and details of the intervention. Section 5 describes the results of hypotheses testing, and in Section 6, the results are discussed. The limitations of this study are presented in Section 7. Finally, conclusions and future work are presented in Section 8.

2. Literature Review and Related Work

2.1. Virtual Reality and Engagement

The number of scientific investigations of VR and engagement has increased considerably over the past 8 years or so. To identify the publication trend in the field of VR and engagement, a specific search by the article title was conducted in the Scopus database by using the following query: TITLE (engagement AND “virtual reality”). This kind of scoping search allowed us to retrieve results that surely addressed both topics of engagement and VR. From this search, 131 documents were found. Figure 1 shows the publication trend. Since 2016, there has been a growing interest; 2019, 2020, 2021, and 2022 were the years when more documents were published, and 2023 seems to have the same tendency. In this context, it seems that studies related to engagement and VR are still in the research arena and are gaining momentum.

Figure 1. Engagement and VR—Documents by year—Scopus database.

In the same search, documents with more than 100 citations report different findings, such as a higher degree of engagement in VR scenarios [18]; similar engagement between 2D VR scenarios and the real world [19]; that engagement is related to the feel of being present, but the degree of presence is no always connected to the VR experience or the response to a specific treatment [20]; or that interactivity plays a crucial role in participants engagement when designing VR exercises for rehabilitation [9].

When talking about VR experiences, people might feel emotionally engaged if they like the content and are focused. This idea is supported in the study by Wagler and Hanus [21], who compared a real-life tourism experience with a VR 360° video. In that study, participants in the VR 360° video experience reported a higher level of spatial presence leading to a higher emotional engagement.

In other VR studies, some variables have been found to be related to engagement. For instance, Bueno-Vesga et al. [22] studied the effects of a desktop-based VR environment considering the cognitive load as a factor that affects student engagement. Dede [23] stated that immersive interfaces can promote engagement. Violante et al. [24] conducted a study in which 85 students were exposed to a desktop-based VR gallery of a digital media course. The authors evaluated factors influencing cognitive engagement in the 3D VR learning experience and found that learning outcomes, problem-solving, and increased engagement had a positive correlation. Škola et al. [25] measured presence, engagement, and immersion in a VR 360° storytelling environment involving cultural heritage. The application was tested by 16 participants, and user experience (UX) was one of the elements that positively influenced engagement. Rowe et al. [26] conducted a study in which 153 middle school students interacted with a narrative-centered learning microbiology environment. The authors found that engagement was associated with better learning gains. In the study by Healey et al. [27], the authors introduced a mixed-reality storytelling application in which children could describe how a story could end. The authors measured engagement and found that children preferred to spend more time with the application than using words to describe facts about the story.

In summary, current research on VR and engagement has focused on investigating to what extent participants engage with a VR environment. However, there is a lack of research on the predictors of engagement when participants use VR environments. This gap in the literature motivated the study presented in this paper.

2.2. Storytelling and Migration

Storytelling is the art of telling a story. The National Story Telling Network (2023) and authors like Lamarre [28] has described storytelling as an interactive art of using words to create narratives. Nowell [29] describes storytelling as a basic and ancient way to communicate things to each other, a way to transmit ideas through the use of stories while “including the signaling of an intention to share information”.

Another definition of storytelling includes a role related to our own recognition and personal understanding. Schachtner [30] introduces storytelling “as an act which contributes to producing the foundations of our lives as it helps us to comprehend the world” based on our own interpretation: “Individuals merge with the world through their narratives.” Those stories represent a series of actions that a creator/narrator uses “to reveal the elements and images of a story while encouraging the listener’s imagination” [31].

However, storytelling goes beyond our individual recognition through stories and plays an important role at a community level. Alterio and McDrury [32] emphasize that narrative approaches allow the readers to increase the emotional release and learn from that particular experience. This implies that we could use storytelling as a media to “promote discovery, exploration and re-imagination” that derives in “subsequent learning” [29]. Snow et al. [33] argue that storytelling is a pedagogical strategy to change perceptions of a topic by people “as they ‘make it real’, identify, and empathize with the characters and story on a personal level.”

Digital storytelling has been applied in research on migration as an alternative mechanism to tell the stories of migrants using different formats. For instance, Alexandra [34] did a workshop in which some migrants in Ireland took a central role in collaboratively presenting their stories through storytelling using images. Saltsman and Majidi [35] point out that storytelling can be a mechanism to put migrants at the center of discussion rather than using numbers to talk about migration. Storytelling can also contribute to changing the dominant narratives about migration by learning from the creation of stories. In the field of education, storytelling has been applied in migration education. Darvin and Norton [36] state that digital storytelling in the classroom is a good method so that migrants can be agents of their own representation and have the right to speak about themselves. Digital storytelling has also been used to promote the sexual health and well-being of migrants as reported in the literature review by Botfield et al. [37]. Nonetheless, according to the study by Bang et al. [38], digital storytelling as a learning activity about migration limits diversity in students’ knowledge and experiences about migration as a result of the introduction of some standardization drivers. These standardization drivers are related to some prejudices and political influence.

Based on the review of the literature, there is a scarcity of research on the predictors of engagement when participants use VR storytelling environments. In that regard, in this study, we fill part of the gap by investigating some of the predictors of engagement in VR storytelling environments.

2.3. Virtual Storytelling

The combination of VR technological affordances and storytelling creates new opportunities for the presentation of information in different fields. In particular, in VR storytelling, the user is not a passive viewer but an active agent. In that regard, the levels of engagement with the story can be higher than those created by other mediums, such as TV or films [39]. VR can be taken as a narrative potential means for storytelling [40]. Virtual storytelling formats allow users to become involved and feel engaged with space and time [39]. According to Bucher [41], in an interview with Jessica Billhart (the main filmmaker for VR at Google), engaging people, creating presence, involving characters, giving roles to viewers, stating a traditional three-act structure, varying experiences, and the experience/perception from users are some elements that drive decisions when designing VR storytelling. According to Yang and Zhang [42], immersive storytelling (i.e., VR storytelling), can improve the sense of presence, flow, and credibility, which, at the same time, has a positive effect on enjoyment. In that regard, VR storytelling can improve engagement with the story. Other affordances of VR storytelling include the opportunity to present abstract experiences, improve emotions, and convey intangible concepts, which jointly improve engagement and motivation [43]. Moreover, the creation of digital stories in VR can be used as a mechanism to engage students in reflective learning in the context of project-based learning activities [44]. In the study by Irshad [45], two VR environments with and without a narrative were compared, and the results showed that the narrative-based VR environment increased participants’ engagement; therefore, the interactive digital narratives are a key factor to increase the emotional response toward a topic. VR combined with digital narratives has also been used in the field of immersive journalism, and previous research has shown that this combination might facilitate empathy and understanding of social issues such as armed conflicts [46].

Overall, current research in the field of VR storytelling provides some insights into the affordances of VR storytelling to increase participants’ engagement. However, the factors that influence participants’ engagement in VR storytelling seem to remain underexplored. In that regard, the next section describes a model of hypotheses around the factors that influence student engagement.

3. Hypotheses Development

3.1. Immersion

Immersion has been defined in the literature from two perspectives: as an objective feature of a technical system or as a psychological state created by the perception of presence and interaction [47]. Immersion can be defined as “a phenomenon experienced by an individual when they are in a state of deep mental involvement in which their cognitive processes (with or without sensory stimulation) cause a shift in their attentional state such that one may experience disassociation from the awareness of the physical world” ([36], p. 6). Immersion occurs when the user focuses on what is being told, so good and well-told stories foster better immersion scenarios [48]. Digital storytelling (such as scenarios with VR and 360° storytelling) tries to maximize different elements of a storytelling process, including immersion [25]. In a study about the exposure to different levels of immersion to analyze motivation, engagement, performance, and spatial reasoning, it was found that moderate or higher levels of immersion might create a sense of novelty and, therefore, increase the level of engagement [49].

Research on immersion in VR scenarios has shown that a higher level of immersion can enhance engagement when compared with other types of media. Moreover, sensors and controls are elements that make users feel as if they were really part of the virtual world. Thus, VR scenarios can consider using virtual versions of the hand controllers inside to enhance the immersion [25].

Hypothesis 

(H1). The sense of immersion in the virtual reality storytelling environment has a positive and significant effect on engagement.

3.2. Presence

Presence can be seen as the sense of being there or being in one place [50]. In the literature on VR experiences, the concept of presence is also tackled as spatial presence, which can be defined as the “cognitive perception that a person’s body is occupying a space in which they feel in sync with the actions of the virtual experience” ([51], p. 2). Slater ([52], p. 432) defines presence as “the illusion of being there, notwithstanding that you know for sure that you are not. It is a perceptual but not a cognitive illusion”. Additionally, a high level of presence in VR might indicate that the virtual scenario is viewed as real and provide a sense that virtual things may happen in the real world [53].

Bueno-Vesga et al. [22] agree that presence is an essential element in VR scenarios. They found that presence positively influences cognitive engagement. In the same vein, Wagler and Hanus [21] found that spatial presence leads to a higher emotional engagement. Another study reported that the higher level of presence made users focus more on the VR interactions rather than on self-thoughts about their own performance, which alleviated the sense of frustration in a training scenario [54].

In contrast to those experiences, Pengnate et al. [55] identified that higher levels of realism or spatial presence can reduce user engagement. However, most of the studies support the idea that the boundaries and features of VR scenarios make presence a factor that positively influences engagement [51]. Moreover, the type of display used in the VR experience is important and affects the variables of presence and engagement [56].

Hypothesis 

(H2). The sense of presence in the virtual reality storytelling environment has a positive and significant effect on engagement.

3.3. Agency

Agency is defined as “the feeling of control over actions and their consequences” ([57], p. 1). The same author states that “the sense of agency refers to this feeling of being in the driving seat when it comes to our actions” ([57], p. 1). In the same vein, Jicol et al. ([58], p. 2) cited the concept of agency in VR, which “can be defined as the perceived ability to interact with elements of a virtual environment” and can be taken as “the degree to which they [users] can control their own actions and influence events in the virtual environment” ([58], p. 3).

Various studies have assessed the impact of the sense of agency in VR environments [58,59,60,61]. However, there is no evidence of studies that report how agency influences engagement in VR scenarios.

Hypothesis 

(H3). The sense of agency in the virtual reality storytelling environment has a positive and significant effect on engagement.

3.4. Usability

Some studies regarding VR scenarios have assessed the usability variable. For instance, Doolani et al. [62] tested the usability of a vocational training system based on a VR storytelling scenario. That study was conducted with 30 participants who were exposed to three different training methods (a VR environment, a 2D video, and a text manual). The authors found that usability in the VR environment was rated higher than in the other training methods. In another study, the usability was tested with 50 participants who interacted with a non-game application and a game-based VR environment [63]. As a result, it was identified that the VR environment presented more usability problems for novice gamers because of the complexity of the components used to make the VR game more attractive to users.

There are some studies that evaluate both concepts as independent constructs. For instance, Mondragón Bernal et al. [64] evaluated both usability and engagement, usability in terms of how easy it was to use the user interface of the immersive environment and engagement in terms of how immersed the users were while carrying out activities. They found that users had good immersion; however, the usability was marginally acceptable. Despite these results, authors from different studies recommend improving interactions between the user and the graphical interface and considering user-centered designs [64,65].

Hypothesis 

(H4). The usability in the virtual reality storytelling environment has a positive and significant effect on engagement.

3.5. User eXperience (UX)

In VR environments, UX is focused on making users feel good with the immersive experience and making the VR environment easy to understand and interact with [66]. In the context of the design of videogames, UX is considered to be one of the most important aspects affecting the engagement of users because the level of engagement is somehow linked to the different forms in which the interface elements are designed [67]. In the educational context, Mäkinen et al. ([68], p. 3211) pointed out that that “UXs affect users’ motivation and engagement, which may be the most important elements of learning experiences”.

Hypothesis 

(H5). The user experience in the virtual reality storytelling environment has a positive and significant effect on engagement.

3.6. Presence and Immersion

Presence is considered a way of feeling immersed in a VR environment. In this sense, high presence and high immersion can lead users to feel as if they were the protagonist in a VR storytelling environment [69]. In other words, immersion happens when a sense of presence is created [70] when people feel like they were in the real world or if they were an object/subject in the virtual scenario.

Hypothesis 

(H6). The immersion in the virtual reality storytelling environment has a positive and significant effect on presence.

3.7. Hypothesis Model

According to the above hypotheses defined, Figure 2 depicts the model of predictors of engagement in VR storytelling environments.

Figure 2. Hypotheses model.

4. Methodology and Research Design

This study aims to identify some of the predictors of engagement when exposing people to a VR storytelling scenario about migration. A description of that scenario is presented in Section 4.1. The following sections present a description of the participants (Section 4.2), the instruments used to collect data (Section 4.3), and the research design (Section 4.4). The intervention of this study was carried out at the Interactivity Lab of the Fundación Universitaria Konrad Lorenz and the VRLab of the Universidad EAN. The research procedure and instruments were approved by the Institutional Ethics Committee of the Fundación Universitaria Konrad Lorenz. Informed consent was obtained from all participants and the procedure was conducted according to the Declaration of Helsinki [71].

4.1. VR Storytelling Scenario

The VR storytelling scenario consists of a VR gallery where the user can explore two rooms with a different migration story in each one. The migration stories are about Venezuelan people who had to migrate to Colombia. The first story is about a woman who works as a waste picker, and the second one is about a singer who found a job as a singing teacher. Figure 3 depicts the two rooms. This scenario was developed using the Unity 3D tool. Stories are told using videos and images with representative objects from them. Users access the gallery using the Meta Quest 2 VR headsets. Videos are activated by using the hand controllers, which also allow the user to teleport from one point of the room to another.

Figure 3. VR storytelling gallery.

4.2. Participants

The R-squared method was used to define the participants’ sample size [72]. The maximum number of arrows pointing at engagement is five and the probability error is 90% to detect R² values of at least 0.25. With these conditions, the study required a minimum of 80 participants. Thus, to ensure the validity of the model, 212 participants were recruited. Participants were 212 university students—113 females and 99 males. Participants were chosen through intentional sampling, and participation was voluntary. In this study, we chose a sample of university students because previous research has shown that treatments of pro-social behaviors are more effective with people in the impressionable years (18–25 years old) [73]. Students who had experienced a personal or family migration process were excluded to reduce potential biases caused by previous experiences.

4.3. Instruments

A self-reported instrument was used to collect data. The instrument consisted of the following six scales for a total of 25 Likert scale items adapted for this study: five items from the User Engagement Scale (UES) [74] adapted by Barbot and Kaufman [75] for engagement; two items from the instrument by Barbot and Kaufman [75] for immersion; four items from the presence model introduced by Schubert et al. [76] for presence; two items from the Developing the Sense of Agency Rating Scale (SOARS) [77] for agency; seven and five items from the Virtual Reality Neuroscience Questionnaire (VRNQ) [78] for UX and five items adapted from the System Usability Scale for usability, accordingly. It is important to note that we did not find a validated instrument to measure engagement in the context of storytelling environments so we used the adaptation of the User Engagement Scale by Barbot and Kaufman [75].

4.4. Research Design

Figure 4 shows the research design for this study. The procedure was as follows: (1) participants were first recruited; (2) participants received and signed the informed consent; (3) participants were trained on how to interact with the VR environment; (4) participants were involved in the VR intervention; and (5) participants answered the instrument questions.

Figure 4. Experiment process.

5. Hypotheses Testing and Results

To validate the hypothesized model and identify the factors that influence engagement in a virtual reality storytelling environment involving migration, the partial least squares structural equation modeling (PLS-SEM) was applied using the SmartPLS software V4.0.9.3 [79]. The evaluation of the structural model was divided into three steps according to Hair et al. [72]: evaluation of the formative measurement model (see Section 5.1), the evaluation of the structural model (see Section 5.2), and the evaluation of the model’s predictive relevance (see Section 5.3).

5.1. Evaluation of the Formative Measurement Model

The formative measurement model consists of the following exogenous variables with their corresponding indicators (see Figure 3): immersion, presence, agency, usability, and UX (User eXperience). To evaluate this measurement model, redundancy analysis was applied to assess the convergent validity, as well as analysis of collinearity (VIF) and analysis of the outer loadings [72]. The results are shown in Table 1. In Table 1, the outer loadings are estimated by the PLS algorithm, using partial linear regressions where the indicators for each construct are independent variables and the formative construct acts as the dependent variable. The equations that are used by the PLS algorithm to estimate the outer loadings are presented in [80]. Moreover, convergent validity is “the extent to which a measure correlates positively with other measures of the same construct using different indicators” ([72], p. 140). According to Hair et al. [72], to estimate convergent validity, the idea is to test the correlation between a formatively measured construct and a reflective measure of the same construct. To do that, we applied the redundancy analysis method in which every variable with its constructs acts as an exogenous variable that is connected to an endogenous variable with one reflective indicator that corresponds to the average value of all indicators. Finally, after running the PLS algorithm, the strength of the path linking the two constructs indicates the validity of that construct. Values above 0.7 are indicative of good validity. Regarding collinearity, in PLS-SEM, the variance inflation factor (VIF) is analyzed to determine the correlation of each indicator with respect to the other indicators. VIF values between 5 and 10 or higher indicate multicollinearity.

Table 1. Outer loadings, convergent validity, composite reliability, and AVE of the formative measurement model.

Formative Constructs	Indicators for the Construct	Outer Loadings (>0.5)	Convergent Validity—Redundancy Analysis (>0.7)	Collinearity (VIF < 5)	Composite Reliability (>0.7)	AVE (>0.5)
Immersion					0.940	0.888
	IMM_1	0.954	1.000	2.507
	IMM_2	0.929		2.507
Presence					0.818	0.532
	PRE_1	0.675	1.000	1.345
	PRE_2	0.622		1.335
	PRE_3	0.850		1.654
	PRE_4	0.740		1.355
Agency					0.787	0.650
	AGE_1	0.877	1.000	1.101
	AGE_2	0.723		1.101
Usability					0.912	0.599
	USA_1	0.815	1.000	2.065
	USA_2	0.782		2.411
	USA_3	0.840		2.611
	USA_4	0.686		2.141
	USA_5	0.664		2.048
	USA_6	0.777		1.870
	USA_7	0.624		1.633
UX					0.838	0.509
	UX_1	0.835	1.000	1.273
	UX_2	0.639		1.292
	UX_3	0.723		1.661
	UX_4	0.584		1.605
	UX_5	0.620		1.581

In terms of reliability and to tackle the limitations of the Cronbach alpha coefficient, composite reliability is used for internal consistency. According to Hair et al. [72], composite reliability is calculated based on the outer loadings of the indicators. Values of composite reliability range from 0 to 1, with values above 0.7 indicating satisfactory reliability.

According to Hair et al. [72], the average variance extracted (AVE) is a measure to establish convergent validity. This measure is calculated as the mean value of the squared loadings of the indicators associated with each one of the constructs. Values higher than 0.5 in a construct indicate that (on average) the construct explains more than half of the variance in the respective construct. The AVE is calculated by the following equation where M is the number of indicators and l is the factor loading of each construct:

$$\mathrm{A}\mathrm{V}\mathrm{E}=\frac{{\sum }_{\mathrm{i}=1}^{\mathrm{M}}{l}_{\mathrm{i}}^2}{\mathrm{M}}$$

5.2. Evaluation of the Structural Model

To evaluate the structural model in SmartPLS, the recommended method is bootstrapping [72]. Table 2 shows the results of R² and R² adjusted for each endogenous latent construct (dependent variables) in the model as well as the total effects and indirect effects for exogenous constructs (independent variables). In Table 2, the R² and R² adjusted are the coefficients of determination, and this is a measure of the predictive power of the proposed model. It means the amount of explained variance. This measure is calculated as the squared correlation between each endogenous construct actual values with respect to the predicted values. When there is more than one exogenous construct (for instance, immersion, presence, agency, and usability), that predicts one endogenous construct (for instance, engagement), and the coefficient represents the combined amount of variance in the endogenous construct that is explained by the exogenous construct. Values for the R² coefficient range between 0 to 1, with values close to 1 indicating higher explained variance. The R² adjusted compensates for adding other nonsignificant constructs, which might increase the explained variance, but affects the reliability of the model. The R² adjusted can be used to compare models with different exogenous variables or datasets with different sample sizes [72].

Table 2. Results of R², R² adjusted, total effects, and indirect effects of the structural model.

Dependent Variable	Independent Variable	R2 (R2 Adjusted)	Total Effects
Engagement		0.552 (0.541)
	Immersion		0.212
	Presence		0.195
	Agency		0.291
	Usability		0.239
	UX		0.193
Presence		0.085 (0.081)
	Immersion		0.291

The results in Table 2 (above) show that the model explains 55.2% of the variance in engagement. All the factors have a positive effect on engagement, and the factor with the strongest effect is agency (0.291) followed by usability (0.239). Moreover, presence explains 8.5% of immersion because of its positive effect (0.291).

Table 3 shows the paths in the hypothesized model together with the path coefficients, effect size, t-value, p-value, and an indicator of whether the hypothesis is supported or not as a result of the PLS-SEM analysis. Table 3 shows that 6 out of the 6 hypotheses in the model are supported. In Table 3, the effect size (f²) indicates the impact on endogenous constructs when each exogenous construct is removed from the model. The effect size can be calculated with the following equation, where R²_included is the R² coefficient when the exogenous construct is included in the model while the R²_excluded is the R² coefficient when the exogenous construct is removed from the model [72]:

$${\mathrm{f}}^2=\frac{{\mathrm{R}}_{\mathrm{i}\mathrm{n}\mathrm{c}\mathrm{l}\mathrm{u}\mathrm{d}\mathrm{e}\mathrm{d}}^2-{\mathrm{R}}_{\mathrm{e}\mathrm{x}\mathrm{c}\mathrm{l}\mathrm{u}\mathrm{d}\mathrm{e}\mathrm{d}}^2}{1-{\mathrm{R}}_{\mathrm{i}\mathrm{n}\mathrm{c}\mathrm{l}\mathrm{u}\mathrm{d}\mathrm{e}\mathrm{d}}^2}$$

Table 3. Results of the hypotheses testing in the structural model.

Hypothesis	Path	Path Coefficient	Effect Size (f2)	t-Value	p-Value	Supported
H1	Immersion → Engagement	0.155	0.041	2.426	0.015	Yes
H2	Presence → Engagement	0.195	0.071	3.029	0.002	Yes
H3	Agency → Engagement	0.291	0.140	4.408	0.000	Yes
H4	Usability → Engagement	0.239	0.089	3.411	0.001	Yes
H5	UX → Engagement	0.193	0.048	2.534	0.011	Yes
H6	Immersion → Presence	0.291	0.093	4.400	0.000	Yes

Values of 0.02, 0.15, and 0.35 indicate small, medium, and large effect sizes, respectively. Regarding the t-value, these data correspond to Student’s t-test calculated with the following formula, where w is the weight in the original model estimation and se is the standard error:

$$\mathrm{t}=\frac{{\mathrm{w}}_1}{{\mathrm{s}\mathrm{e}}_{\mathrm{w}1}}$$

Finally, the p-value indicates the significance of each path in the model. The p-value is the probability that random sampling using bootstrapping yields a t-value of at least 1.96.

Figure 5 shows the validated model of predictors of engagement, including the path coefficients and outer loadings.

Figure 5. A validated model of predictors of engagement.

5.3. Predictive Relevance of the Structural Model

To evaluate the predictive relevance, the Stone–Geisser’s Q2 measure was used as recommended by Hair et al. [72]. The predictive relevance is useful in determining how well the model predicts the original values. This measure indicates how well the model predicts data not used in the original estimation of the mode. In other words, this measure indicates how well the model could predict engagement and presence. To calculate the Stone–Geisser’s Q2 measure in SmartPLS, the PLSpredict method was applied with an omission distance of 7 and using the cross-validated redundancy. The results (see Table 4) show that engagement exhibits high predictive power in our model.

Table 4. Stone–Geisser’s Q2 measure for predictive relevance of the structural model.

Endogenous Variables	Stone-Geisser’s(Q2)
Engagement	0.445
Presence	0.053

Moreover, the effect size (q²) was also computed to determine the relative predictive relevance of each exogenous construct with respect to the endogenous constructs. Following the recommendations of [72], the effect size was calculated from the Stone–Geisser’s Q2 measure. Table 5 shows the values of the q² effect sizes. The columns represent the endogenous variables, and the rows represent the predictors. Values of 0.02, 0.15, and 0.35 indicate small, medium, and large effect sizes, accordingly. The equation used to calculate the effect size (q²) is:

$${\mathrm{q}}^2=\frac{{\mathrm{Q}}_{\mathrm{i}\mathrm{n}\mathrm{c}\mathrm{l}\mathrm{u}\mathrm{d}\mathrm{e}\mathrm{d}}^2-{\mathrm{Q}}_{\mathrm{e}\mathrm{x}\mathrm{c}\mathrm{l}\mathrm{u}\mathrm{d}\mathrm{e}\mathrm{d}}^2}{1-{\mathrm{Q}}_{\mathrm{i}\mathrm{n}\mathrm{c}\mathrm{l}\mathrm{u}\mathrm{d}\mathrm{e}\mathrm{d}}^2}$$

Table 5. Values of the q² effect size.

	Engagement
Immersion	−0.0198
Presence	0.0036
Agency	0.1243
Usability	0.0378
UX	0.0559

6. Discussion

The aim of this study was to investigate the predictors of engagement in VR, particularly the influence of immersion, presence, agency, usability, and user experience on engagement when people use a VR migration storytelling scenario. The analysis was carried out by applying the PLS-SEM technique. The results show that the model explains 55.2% of the variance in engagement because immersion, presence, agency, usability, and UX positively affect engagement.

Immersion is one of the factors influencing engagement in VR migration storytelling scenarios. According to our model, this construct has a positive and significant effect on engagement (p = 0.015), thus supporting the H1 (the sense of immersion in the VR storytelling environment has a positive and significant effect on engagement). This result confirms the findings of previous studies that noted the positive effect of immersion on engagement [23,36]. An interpretation of this result might be that the level of immersion created by an immersive storytelling scenario is effective in increasing engagement. This result also has implications for developers of VR storytelling scenarios because the provision of more interactions, realistic objects, sounds, and visual effects that increase immersion are effective in increasing the level of engagement. However, immersion had a small and negative effect size (q² = −0.0198), which might indicate that the magnitude of the relationship between immersion and engagement can be reduced.

Another predictor of engagement in VR migration storytelling scenarios is presence. In this study, we have found that the sense of presence in the VR storytelling environment has a positive and significant effect on engagement (H2) with p = 0.002. In accordance with the present results, previous studies have demonstrated that the boundaries and features of VR scenarios make presence a factor that positively influences engagement [21,22,51,54]. The effect size of the relationship between presence and engagement was small (q² = 0.0036). A possible interpretation of a small effect size might be that the VR storytelling environment does not include embodiment [81], so the sense of presence can decrease. Although the effect size is small, the relationship between presence and engagement is significant; therefore, the sense of presence experienced by the participants is relevant to engaging them in a VR storytelling environment. From a theoretical point of view, this result might be explained by the levels of empathy developed around the VR storytelling experience. In that regard, Shin [12] found that cognitive processes that involve the sense of presence determine how people feel empathy and engagement with a story in VR. At this point of the discussion of results and by contrasting previous literature on the sense of presence, the effect of presence on engagement might be moderated or mediated by the levels of empathy created around the storytelling experience in VR. However, further research is needed to confirm this hypothesis.

In this study, we found that agency was also a predictor of engagement (H3—the sense of agency in the VR storytelling environment has a positive and significant effect on engagement) with p < 0.000. The effect size was medium (q² = 0.1243). To the best of our knowledge, our study is the first one that provides insights into the positive effect of agency on engagement in VR. This can be contrasted with previous research, suggesting that agency might be the fourth component of engagement [82]. The other three components of engagement are cognitive, behavioral, and emotional. In that regard, in this study, we empirically confirm that in VR experiences, agency can be another dimension of engagement. A theoretical implication of this result is that future theoretical models of engagement in VR environments might consider agency as another dimension. A practical implication of this result is that future developments of VR experiences— such as sensorimotor stimulus [83], visual feedback of actions [60], and tangible interaction [61] should include mechanisms to support agency.

We also found that the usability of the VR storytelling environment had a positive and significant effect on engagement (H4). Hypothesis H4 was supported with p = 0.001 and had a medium effect size (q² = 0.0378). An interpretation of this result might be that previous research has shown that the perceptions of the user interfaces and their usability affect user engagement [67]. A medium effect size indicates that the magnitude of the relationship is significant; therefore, usability is a good predictor of engagement. To the best of our knowledge, this is also the first study that sheds light on the influence of usability on engagement in VR experiences. A practical implication for developers is that a VR environment without usability issues might keep participants engaged with the VR experience. However, there are still contradictory findings because recent research has shown that the perceptions of the user interfaces do not influence student engagement [84]. Consequently, future research should confirm whether usability is a predictor of engagement.

The UX construct had a positive and significant effect on engagement in the VR storytelling scenario (H5), which was supported by p = 0.011 and a medium effect size (q² = 0.0559). This result corroborates previous research, suggesting that UX affects engagement [68] and contributes to shedding light on the effect of UX on engagement. However, to the best of our knowledge, the effect of UX on engagement in VR environments remains largely unexplored. In that regard, our result needs to be confirmed by future empirical research.

Finally, according to our model, immersion has a positive and significant effect on presence confirming the hypothesis that immersion in the VR storytelling environment had a positive and significant effect on presence (H6) with p < 0.000. This result is in line with a large body of research, suggesting that immersion is a prerequisite for increased presence in VR environments [85,86].

7. Limitations

Because this exploratory study reports on the experiences of users interacting in a VR migration storytelling scenario, the findings should be seen as indicative rather than as establishing an empirical evidence base. Moreover, the predictors of engagement identified in this study cannot be generalized to other domains apart from migration storytelling without further validation of this model in other fields. The model that we suggested and validated in this paper was created based on the literature around engagement and VR, so the model can be easily validated without any changes with the help of VR environments developed in other fields. In this sense, this model can be easily applied to other domains to identify predictors of engagement. For instance, the model can be validated in domains such as education (at any level), racism (such as 1000 Cut Journey [87]), and empathy development about social issues in which storytelling is used. In this study, we found some factors in the literature that influence engagement, but there could be more fundamental factors. The intervention with the VR application was cross-sectional, and therefore, the effects of some factors may decrease or increase over time, but the results show a positive effect for short interventions using VR.

8. Conclusions and Future Work

Research on VR and engagement has shown that VR experiences are effective in increasing engagement [18,23]. However, research on the specific predictors of engagement in a VR experience is still scarce. To fill this gap in the literature and as the main contribution to this study, we found some predictors of engagement when participants use a VR storytelling environment involving migration. Our model explains 55.2% of the variance of engagement as a result of the positive influence of immersion, presence, agency, usability, and user experience. The results of this study contribute to the body of research on engagement using VR storytelling environments and shed light on some of the predictors of engagement. Moreover, the results can be useful in informing the development of future VR storytelling applications that effectively engage participants.

Future research directions include further research on how empathy and its dimensions moderate or mediate the relationship between presence and engagement [88]. Moreover, future studies might consider longitudinal interventions to evaluate how the predictors affect engagement over time. The next phase in this research is to develop new VR scenarios in other topics different from migration to determine if our findings are still applicable and transferable to other contexts. Finally, our study can be replicated using VR experiences with a focus on other topics different from migration to identify if the model is still valid in other contexts.