Enabling Exploratory Yet Systematic Investigation of Presence Factors in Virtual Reality: Proposed Methodology, Research Tool Development, and Practical Application

Abstract

Presence is widely recognized as a key quality metric for immersive virtual reality (IVR) experiences. However, research on factors contributing to presence is impeded by the plethora of identified factors, contradictory results, and unclear interactions. Based on the analysis of the current literature, we propose a two-step research methodology combining exploratory and confirmatory paradigms to address these issues. As existing IVR study tools do not focus on presence and its determinants, we developed our own tool consisting of two components: an IVR app, based on Unreal Engine for designing and displaying IVR scenarios, and a browser-based experimenter interface using Vue.js, enabling precise control over presence factors and study procedure. The methodology and study tool underwent a formative expert evaluation (N = 6) and a first practical application within the AgeVR research project (N = 115). Their feasibility was confirmed by expert feedback, as well as data from 115 successfully completed exploratory studies with participants of various ages. The exploratory study procedure works for general presence, involvement, and plausibility illusion. Measures that will enable the seamless investigation of the remaining presence subcomponents are proposed. Our next step is to develop and test hypotheses in the confirmatory studies. The study tool was made publicly available as an open source project.

1. Introduction

Immersive virtual reality (IVR) has emerged as a profoundly relevant technology for a wide range of user groups, offering immersive experiences and versatile applications that cater to diverse interests and needs [1,2,3]. When designing virtual environments, immersion and the sense of presence emerge as key quality metrics from an HCI perspective [3,4,5,6], in addition to the established usability paradigm. Immersion is defined by Slater and Wilbur as “the objective degree to which a system is inclusive, extensive, surrounding, and vivid” [6]. Inclusive refers to the exclusion of real-world stimuli, extensive to the number of modalities considered, surrounding to the degree of panorama, and vivid to the resolution, fidelity, and variety of display technology [6]. In contrast, presence is a subjective experience, generally defined as the feeling of “being there” [7] in an immersive virtual environment (IVE). Throughout the last three decades of research, a plethora of definitions and operationalization of presence has been developed. However, its subcomponents can be broadly grouped into five categories: spatial [5,8,9], social [5,9], and self-presence [9], as well as involvement [8,10] and plausibility illusion [5,11]. When measuring presence, subjective methods have become the standard approach, offering several valid and reliable questionnaires (e.g., [8,10,12,13]), while objective methods such as physiological and behavioral measures are less established [5,14,15,16]. Presence enables the measurement of the effectiveness of IVE across different application contexts [17].

Motivation

Presence research in IVR is impeded by the abundance of factors that can influence IVR experiences. From a human-centered perspective, these presence determinants can be classified as either internal (dependent on the user) or external (dependent on the system) [18]. The substantial number of identified factors, contradictory results on their influence, and lack of clarity on their interaction [14,18,19] make the development of design guidelines for presence particularly difficult.

In this work, we develop a methodology of systematic investigation with the goal of making presence factors controllable. Internal factors can only be controlled through the purposeful recruitment of participants and rigorous documentation of user characteristics. However, this entails the risk of excluding certain user groups based on cognitive, perceptive, and motor abilities. As characteristics of the IVR system, external factors are the only ones that can be controlled by technological means. When investigating external factors, confirmatory studies (testing hypotheses) are the de facto standard. Typically, these studies compare different versions of an IVE [20,21,22], or entirely different IVR applications [23,24,25] addressing one or several specific presence factors. Studies that utilize exploratory approaches, which allow for a more open investigation of presence determinants, remain scarce (e.g., individual configurability [26,27,28]). As mentioned, confirmatory presence studies often lead to contradictory results (e.g., for factors such as display resolution [18], polygon count [29,30,31], emotion [32,33], VR scenario [34,35,36], and age [37,38,39]). In addition, it is hard to isolate the effect that singular factors may have on presence, since they often manipulate a conglomerate of factors within their experimental conditions. A common example is manipulating both display and interaction characteristics in one condition change [40,41,42,43,44,45]. This also complicates the identification of dependencies and interactions between factors.

We argue that a two-stage procedure that combines exploratory and confirmatory approaches is necessary for a more systematic investigation of presence determinants. The idea is to (1) establish empirically grounded hypotheses via exploratory investigation to then (2) verify/falsify these hypotheses by means of comparison. This process is described in detail in Section 4. Also, there is a lack of VR research tools allowing for the conduction of systematic investigations of presence factors as IVR applications are typically developed for one-time use in a single study (see Section 3). Thus, they have a clear context to be applied in and mostly do not allow for generalizable results. Furthermore, existing VR research toolboxes do not focus on presence and its determinants [46].

The contribution of this work is the establishment of a foundation for systematic studies on (external) presence factors by proposing a new research methodology (Section 4) and developing a two-component tool that implements this methodology (Section 5). The VR tool is designed to (1) enable experimental control over external presence factors, (2) guide the experimental process, and (3) facilitate data extraction. Our developed methodology and VR tool underwent a formative expert evaluation (N = 6) and a practical evaluation via exploratory user studies with N = 115 participants in the scope of a research project (Section 6).

2. Presence Factors in IVR: State of Research

The recent literature and meta-analyses that address determinants of presence aggregate findings on both immersive and non-immersive virtual reality settings [3,5,14,18]. Here, four general categories of presence factors can be identified. These are display fidelity and interaction fidelity [47], as well as engagement and personal characteristics [14]. To provide an overview of the state of presence research that focuses exclusively on immersive VR, we identified 118 studies (full table available at https://www.imis.uni-luebeck.de/sites/default/files/2021-07/index.html, accessed on 30 May 2025) as part of our own literature research. Inclusion criteria for studies were the use of immersive VR technology (e.g., CAVE, HMD) as well as some form of presence measurement (e.g., questionnaires, behavioral measures). These show that the IVR-focused state of research largely reflects the four categories of presence factors. IVR studies typically investigate the combined influence of several factors from the categories of display fidelity (N = 62), interaction fidelity (N = 35), engagement (N = 16), and user characteristics (N = 24). However, in IVR, additional factors arise that do not fit into any of the previous categories. Seventeen studies considered immersion/inclusiveness (see Section 1) by comparing immersive and non-immersive VR technologies (e.g., HMD vs. Desktop, [48]). Thirteen additional factors were classified as others and were examined mainly in individual studies. These factors encompass content variables like character anthropomorphism [49], nested virtual environments [50], realistic anchors [51], and transitional environments [52], as well as special variables like shared VR experience [53] and indoor/outdoor VR usage [54].

Of the 118 studies analyzed, 70 investigated exclusively external presence factors, 16 exclusively internal ones, and 28 investigated both. Research on external presence factors focuses primarily on visual representation [3,55], with 58 of 118 studies examining the influence of visual representations. Most visual characteristics of IVR belong to the category of display fidelity. To shed light on the limitations of the current state of presence research in IVR, namely the unsystematic character of investigations (multifactorial ambiguity) and contradictory nature of the results, we present display fidelity as an example for external presence factors. Subsequently, we describe user characteristics as an example for internal presence factors.

2.1. Display Fidelity

Display fidelity (also stimulus [5] or physical fidelity [3]) is “the objective degree of exactness with which real-world sensory stimuli are reproduced” by a display system [47]. In general, this definition includes all sensory impressions (esp. visual, auditory, haptic, olfactory, and gustatory). Previous VR studies have focused primarily on visual aspects, some studies on auditory aspects, and very few studies on other modalities [3,55]. A framework of visual display fidelity components was developed by Bowman and McMahan based on the state of research [55,56]. This includes (1) field of view, FOV; (2) field of regard, FOR; (3) screen size; (4) screen resolution; (5) stereoscopy; (6) head-based rendering; (7) illumination realism; (8) frame rate; and (9) update rate. The focus of this model lies on technical properties of the used hardware. Design-related characteristics of VEs are not considered (except for illumination realism). Other authors, however, emphasize (7) illumination realism [57,58], (10) texture realism [57], and (11) environmental details [57,58].

IVR studies examined these aspects in various combinations. Significant increases in presence were observed when using more realistic illumination [59,60], or more realistic textures and more environmental objects [61]. An increase in texture and geometric realism led to significant increases [29,30,62,63] or no significant differences [31] in presence. No significant difference could be found for increased illumination realism in combination with a higher texture resolution [64]. An increased texture realism alone did not lead to a significant difference in presence [65]. Still, texture realism contributed more to the sense of presence than the possibility of locomotion [66]. In a comparative evaluation of different VEs, the one with the highest values for illumination realism, texture realism, and environmental details led to the highest sense of presence [23]. So, in most cases, the enhancement of different factors within the category of (visual) display fidelity led to an increased sense of presence. However, most studies show a confounded study design, leading to a lack of factor isolation.

2.2. User Characteristics

User characteristics encompass all presence factors that can be classified as properties, abilities, or states of the user. This includes personal characteristics, such as Personality, Cognition, Imagination, Concentration, Knowledge, and VR/Game Experience [14]. The relation between various user characteristics and presence remains widely unclear. For instance, both a positive [38,67] and a negative correlation [39] between chronological age and presence were identified. However, other studies could not find any significant differences in presence between older and younger subjects [37,68,69,70]. Regarding personality traits, extraversion was either found to be positively [71] or negatively correlated with presence [72,73]. Again, others did not find such a correlation [74]. As an example for abilities, studies scarcely found spatial imagination to be significantly correlated with presence [72,73], whereas in the majority of experiments no influence could be observed [33,75,76,77]. The 3D gaming experience of users was found to have no association with presence [33,72]; in other instances, it was negatively associated [78]. Positive associations were found only in behavioral measures [79] or single questionnaire items [80].

Thus, when it comes to internal presence factors, these are often well isolated in the experimental design. However, studies often have conflicting results and generally are not sufficiently comparable to explain these.

3. VR Study Tools: State-of-the-Art

Existing VR research toolboxes do not focus on presence and its determinants [46]. In presence research, IVR applications are typically developed for one-time use in a single study (e.g., [62,81,82,83,84,85,86,87,88,89]), limiting their customizability and reusability. Here, experimental conditions are realized either by several separate builds (standalone software artifacts that can be run on a computer, e.g., exe-files) or scenes within the respective engine or alternatively by changing configurations directly in the engine’s editor. While the former might be resource intensive, the latter might introduce the risk of configuration errors by the experimenter. Game engines, such as the Unity Engine (e.g., [22,40,82,83,84,85,86,87,88,89,90,91,92]) and Unreal Engine (e.g., [23,62,81]) are the most widely used development software for creating such study environments [46]. Others utilize modified [93,94] or unmodified versions [25,33,95,96,97] of proprietary VR applications such as video games. In addition to the previously mentioned reusability and customizability issues, this approach might also reduce control over presence factors. Furthermore, the VR application is usually decoupled from the rest of the experimental process (e.g., randomization of experimental conditions, tutorials, questionnaires). Therefore, we propose an external interface for experimenters that covers the experimental process by streamlining its steps (see Section 5).

Focusing on tools for the general study of human factors in VR, Woelfel et al. [46] analyzed the current state of technology. The authors summarized the full range of feature sets that contemporary tools can provide. Study tools typically only satisfy a certain subset of features that is beneficial to their respective application context or research domain. (1) Setup and Control refers to the ability of a study tool to facilitate experiment planning, customization, and real-time intervention. A tool should provide process planning before and operator control during the experiment; sample scenes should be available and customizable. Furthermore, remote testing (outside the laboratory) and human- or agent-controlled social agents should be available. (2) Sensing Participants refers to the variety of sensors provided by VR hardware as a rich source of information (based on informed legal consent) enabling the development of a responsive virtual environment. This encompasses the recognition of controller-input, the tracking of head-orientation, body pose, eye gaze, facial features, brain activities and other biosignals (e.g., heart and respiration rate), audio and video recording, as well as the administration of in-VR questionnaires. (3) Representation of the virtual environment and its entities focuses primarily on visual realism and interaction possibilities. This comprises representing virtual entities (e.g., surrounding objects and agents), as well as real entities (e.g, participants, experimenter, physical objects) within the VE. (4) Data Handling refers to the reliability, validity, and transparency of data collection, processing, and management by the study tool. This includes the possibility to import, export, and stream data; the automation of analyses (e.g., behavior/emotion recognition); and the possibility to replay and annotate VR scenes and to share the experimental environment for replication purposes. (5) Integration addresses the question of how much of the tool’s source code is available open source and how much is proprietary code of the underlying engines (e.g., Unity, Unreal).

Our goal is to develop a VR tool specifically dedicated to investigating presence factors which, based on our literature research, does not exist to date. For this purpose, we developed a research methodology, which is presented in the following paragraph. The development of the modular VR tool based on this methodology is described in Section 5.

4. Research Methodology

As shown by the literature (Section 2), the current state of research does not allow for a well-founded formulation of hypotheses on the relationship between presence and the plethora of internal and external factors. As a direct deduction from this state and as a basis for our research methodology, we propose the following measures for the systematization of presence studies. (M1) Presence factors should be atomic in their operationalization, as far as possible (e.g., breaking down the factor illumination realism into illumination intensity, contrast ratio, lighting algorithm, etc.). (M2) The same presence factors must be investigated analogously in different IVR scenarios to improve generalizability and eliminate one possible source of contradictory results. When it comes to the control and manipulation of presence factors within an experimental setting, we build on the existing classifications by Felton et al. [18] and Souza et al. [14]. Internal factors depend on the user [18]; thus, we focus on all abilities, properties, and states of IVR users with potential relevance for presence. External factors must be divided into the ones that cannot be controlled by technological means (we call these environmental factors) and the ones that can (we call these technological factors). Environmental factors include ambient characteristics, namely real world stimuli and the physical surrounding. Technological factors include hardware and software characteristics, namely display fidelity, interaction fidelity, and content factors (see Figure 1). Our proposed measures in this regard are the following. (M3) external factors should be controlled by technological means as far as possible (in our case this refers to the previously defined “technological factors”). Thus, independent variables should be adjustable via the study tool used, and all other technological factors should be represented via reasonable default values (derived from the current state of research and capabilities of the technology). External factors relating to the environment should be regulated within a controlled laboratory setting. (M4) Internal factors, i.e., user characteristics, should be controlled through rigorous documentation. Purposeful recruitment by diversifying or narrowing the scope of participant acquisition can help to validate the subject sample within the study context.

Our methodology prioritizes technological presence factors since (A) they offer the highest degree of experimental controllability, and (B) they hold the greatest practical relevance for designers of virtual environments. In later stages, the focus of the methodology could shift toward human or environmental factors; for now, these are primarily well documented and controlled rather than manipulated. Thus, for our study design, we chose a two-stage procedure consisting of an initial exploratory study followed by a confirmatory study. The exploratory approach is essential to enable the derivation of hypotheses based on the data obtained. In the subsequent confirmatory study, these hypotheses are then tested via systematic manipulation of these presence factors, examining them in paired comparisons, enabling the derivation of design guidelines (see Figure 1). This two-step procedure is detailed in the following.

4.1. Exploratory Assessment of Preferred Configurations of Presence Factors

An exploratory study design is chosen as the basis for hypothesis formation. We define $T=\{t_1,\dots t_n\}$ as the set of technological factors and p as the number of independent variables (IVs). Thus, in the exploratory study, we investigate possible combinations $t_1\times t_2\times \dots t_p$ in order to find the optimal configurations to achieve the highest presence (see Figure 1). Based on the approach by Slater et al. [27], this optimum is to be determined by the subjects themselves. However, since this task is potentially complex, especially for subjects unfamiliar with IVR, the configurations are made by the experimenter in a guided process similar to subjective refraction (from optometry: process to determine the combination of lenses that provide the best corrected visual acuity [98]; the alternating adjustment and refinement of visual parameters inspired our experimental process). To identify an appropriate subset of T to be used as an IV in a study, the trade-off between comprehensiveness (considering as many factors as possible) and complexity must be taken into account. In our first practical application of the methodology, we chose $p=2$ as the minimal reproducible example and conducted two studies with different pairs of IVs (see Section 6.2.1). The general procedure of such an exploratory study is described below.

Before the IVR experiment, control over presence factors that are not considered as IVs must be ensured. The laboratory environment is set up in order to provide stable conditions in terms of real-world stimuli and physical surroundings. User characteristics with potential relevance for the IVs are assessed via suitable instruments (for an example of how to identify and document user characteristics, see Section 6.2.1). Technological factors that are not IVs are set to their respective default values. At the beginning of the IVR experiment, subjects are introduced to the concept of presence to ensure a correct understanding of this term. Then, subjects are informed that they will repeatedly experience two different versions of an IVE. Each time, they are asked to indicate in which of the two they feel more present. A short narrative introduction to the respective IVR scenario is given. Subjects put on the HMD and the IVR application is started. The experimenter increments one presence factor at a time, starting at the lowest possible value. The IV $t_1,\dots t_p$ are manipulated in a quasi-randomized order to compensate for sequence effects (all possible sequences are determined; these are then assigned consecutively to subjects based on their ID to ensure counterbalance). For instance, the experimenter might start by presenting the IVE with $t_1$ at level 0 vs. level 1, while $t_2,\dots t_p$ stay at their default values. If subjects state that they feel more present at level 1, the experimenter will increase $t_1$ presenting level 1 vs. level 2. This process is repeated until the lower value is preferred or subjects state that they do not recognize any differences between variations or if the maximum value is reached. After that, the same configuration is carried out for $t_2$ and all other IVs up to $t_p$. Subsequently, a refinement step is performed, starting again with $t_1$ and moving up to $t_p$, varying all IVs in a smaller range. This is intended to address a possible interdependence of presence factors that may not have become apparent from the default values. After this, the configuration is finished and all values are saved. With the resulting configuration, the subjects must then perform an IVR task involving locomotion in the IVE and interaction with virtual objects. As a control measure, all presence subcomponents are assessed after IVR exposure using a short scale (see Appendix B,

Table A1. The presence short scale developed for the proposed methodology based on established presence questionnaires (PQ, IPQ, SUS, MPS). It was designed to measure all categories of presence subcomponents, general presence, plausibility, involvement, physical/spatial presence, social presence, and self-presence with a respective item. Each item is to be rated on a 5-point Likert scale.

Item Name	Item Text
	In the computer-generated virtual environment …
general presence (GP)	… I had a sense of “being there”
plausibility illusion (PSI)	… I had the feeling, that what was apparently happening was really happening.
involvement (INV)	… my attention was completely captivated.
physical/spatial presence (PP)	… the objects and rooms felt like real (material) things.
social presence (SoP)	… the people and animals seemed to me like living beings (with consciousness and feelings).
self-presence (SeP)	… I felt as if my virtual body was my real body.

) based on established presence questionnaires (e.g., PQ [10], IPQ [8], SUS [12], MPS [13]).

To enable the derivation of hypotheses for subsequent studies, a cluster analysis of the resulting individual configurations $t_1\times t_2\times \dots t_p$ is conducted (clusters are groupings of data points around a specific value or location). In the scope of possible hypotheses, all possible cases must be covered. These are (H1) that clusters found relate to certain user characteristics, indicating a specific relationship between these user characteristics and the optimal configuration for presence; (H2) that clusters found are independent of user characteristics, representing different patterns of experiencing presence. It is also possible that no clusters are found at all, suggesting either (H3) a universal optimal configuration for presence or that (H4) optimal configurations can only be predicted for each user individually.

4.2. Confirmatory Study on the Combined Influence of Presence Factors

To test the hypotheses developed, a confirmatory study was carried out. This study formed the empirical basis for deriving presence-related design guidelines for IVR design. Measures to control environment factors and non-IV technological factors as well as the assessment of user characteristics were performed analogous to the previous study.

In this comparative study, presence factors $t_1,\dots t_p$ are systematically varied in several conditions, which are predefined depending on the hypothesis derived from the results of the previous study. Thus, the hypothesized optimal configurations $O_{cluster}(t_1,\dots t_p)$ for either (H1) each profile of user characteristics identified by the cluster analysis, or for (H2) each presence experience pattern identified by the cluster analyses, are tested against each other. Alternatively, the hypothesized (H3) universal optimal configuration $O_{uni}(t_1,\dots t_p)$ or (H4) optimal configuration for each individual user $O_{user}(t_1,\dots t_p)$ is tested against a certain number of hypothesized suboptimal configurations $S_i(t_1,\dots t_p)$ that derive strongly from it. In the IVE, the same IVR task as in the exploratory study is carried out once for each experimental condition (quasi-randomized order). After each condition, presence is recorded in detail via established presence questionnaires (e.g., PQ [10], IPQ [8], SUS [12], MPS [13]).

The data of this study were examined using multiple ANOVAs in the case of H3 or H4, having the experimental conditions (configurations of $t_1,\dots t_p$) as the within-subject factor. In the case of H1 or H2, multiple mixed ANOVAs were performed, having user characteristics or experience patterns as additional between-subject factors. The significance level was $\alpha =5\%$, applying Bonferroni correction where necessary. In the case of statistically significant differences, the effect sizes are calculated additionally. The analysis addresses the research question of how the examined presence factors differ regarding their effect on presence in IVEs. For this purpose, all five subcomponents of presence, namely spatial, social, and self-presence, involvement, and plausibility illusion (see Section 1) are analyzed via the subscales of PQ [10], IPQ [8], SUS [12], and MPS [13]. This analysis is used to derive design recommendations for maximizing presence in IVR, which are either aimed at specific user groups or provide universal solutions.

5. IVR Study Tool Development

To enable the conduction of the planned studies (see Section 4), our study tool requires the following central functionalities: (1) manipulation of IV (technological presence factors), (2) support of different IVR scenarios, (3) guidance of the experimental process, (4) IVR interaction techniques for certain tasks and respective tutorials, (5) data formatting and extraction. Furthermore, the system needs to be modular and expandable, especially in terms of manipulable IVs. Our tool was developed in the context of the AgeVR research project, which focused on the influence of age-related user characteristics and visual display fidelity on presence (https://www.imis.uni-luebeck.de/de/forschung/projekte/agevr, accessed on 30 May 2025). Accordingly, the four exemplary IVs that were implemented are in the category of visual display fidelity, namely illumination intensity (I), contrast ratio (C), objects density (O), and texture resolution (T). The proposed study process (see Section 4) and an analysis of the state-of-the-art features of industrially produced IVEs form the basis of our development process. For the industrial state-of-the-art analysis, ten contemporary IVR games (published 2018–2022) were investigated, focusing on visual fidelity and its configurability but also to identify industry-standard baselines for auditive and haptic fidelity (identified through journalistic articles on the topic of “Best IVR games”: Blade and Sorcery, Half-Life: Alyx, Hurbis (Demo), Kayak VR Mirage, Lone Echo, Meditation VR, Moss, Project Cars 2, Vader Immortal I-III, VR Walking Simulator). Most of the configurable visual fidelity aspects are covered by the four IVs (I, C, O, T); for the remaining aspects like anti-aliasing and color correction, common default values were chosen as baselines. The development process resulted in two main components: an IVR application and an experimenter interface (see Figure 2), which are described below. A formative evaluation with six field experts was conducted at the end of the first iteration (see Section 6.1). Based on this feedback, the study system was refined (see Section 4.1). Both components of the tool have been made freely available on github.com as an open source template that can be used to create custom IVR studies (https://github.com/RBW1999/VR-StudySystem-Template-VE/blob/main/README.md, accessed on 30 May 2025).

5.1. IVR Application

The IVR application is powered by Unreal Engine 5, which includes the virtualized geometry system Nanite (https://dev.epicgames.com/documentation/en-us/unreal-engine/nanite-virtualized-geometry-in-unreal-engine?application_version=5.3, accessed on 30 May 2025). It displays the IVR scenarios with which subjects are presented and can interact during the experiment. The IVR application listens for instructions provided by the experimenter through the web interface. These instructions include, e.g., manipulating the IVs and loading a new IVE. The subjects’ data (IV configurations, task completion scores, and times) are captured during and exported after the experiment by the IVR application. Unreal Engine 5 was chosen because it is one of the leading engines used for IVR research (next to Unity, see Section 3), enables high-fidelity graphics and physics, has a low-level entry point for novice developers (through constructs such as Blueprints), and has modularity and expandability.

5.1.1. Base System

To support the creation of adaptable scenarios, a series of systems were implemented using the Blueprint system provided by the Unreal Engine (https://dev.epicgames.com/documentation/en-us/unreal-engine/introduction-to-blueprints-visual-scripting-in-unreal-engine?application_version=5.0, accessed on 30 May 2025). The manipulation of IVs (which are referred to as Adaptation Dimensions (ADs) within the tool) is handled by the Adaptation Dimension Manager (ADManager). This is achieved through direct control of the player camera contrast (C), scaling the intensity of all light sources (I), toggling the visibility of objects (O), or the execution of console commands to manipulate the resolution of all textures (T) in the scene (https://developer.arm.com/documentation/102696/0100/Texture-mipmapping, accessed on 30 May 2025). The latter was implemented by extending the existing Unreal Engine construct of the Actor, creating the so-called Adaptation Dimension Actor (ADActor), which allows for the systematic manipulation of this Actor’s visibility. When an ADActor is placed in the scene, it is randomly determined at which degree of the AD (O) it becomes visible. At the maximum degree of O, all ADActors are visible. To address more complex object compositions, e.g., involving gravity (objects placed on top of each other), the ADComplexActor was created as an extension to the ADActor. It allows the developer to group multiple objects together and predetermine a specific order in which the individual objects appear (with the range of AD (O) degrees still being determined randomly). Figure 3 shows a bookshelf implemented as a ComplexADActor. In this example, at the AD degree of $O=5$ (left), there is only one book visible; with each degree more books are added, and these can lay or lean on previous books (middle). At the degree of 16, all books are visible (right).

Another basic functionality is the management of the IVR task. The collective of task-related objects (TaskObjects) is managed by the TaskManager Blueprint. In its current state, the IVR task consists of a simple search task in which the participants have to move through the IVE and search for and pick up certain objects. Thereby, the level of presence developed by subjects is based on an interactive experience of the IVE. The teleportation-based locomotion and the interaction with TaskObjects were implemented using native functions of the Unreal Engine. The completion of (sub)tasks (individual TaskObjects) is automatically assessed by the system. For complex tasks monitored by the experimenter, a checklist template is provided via the interface.

5.1.2. IVR Scenarios

We developed three types of IVE for the study tool: A sample IVE, a tutorial IVE, and several specific study IVEs. The Sample IVE serves as a template for the creation of a new experimental IVE. It contains sample versions of the ADActor, ComplexADActor, and TaskObject Blueprints, which can be copied and customized to fit the needs of the specific experiment and/or IVR scenario being depicted.

The Tutorial IVE was designed with the goal of communicating and practicing the study procedure and relevant IVR interactions. The IVR scene is kept intentionally abstract, having no corresponding real-world scenario. It consists of a walled zone with a few interactive objects (see Figure 4). The participant is required to perform three tasks: (1) practice room-scale walking in the IVE, (2) practice teleportation in the IVE, (3) practice object grabbing and repositioning, (4) learn how the configuration of IV works in the exploratory study process. When practicing the movement, the participant is asked to walk or teleport to certain positions marked on the floor. When practicing object interaction, the participant needs to sort different objects according to their type, by taking them and placing them in specific containers. Subjects learn how to configure the IV via the example of adjusting the color of an object in the scene. This way, the participant learns the process and necessary interaction with the experimenter on an abstract level without biasing later configurations during the actual experiment.

The four specific study IVEs were developed using the Qixel Megascans Libary, which provides high-quality 3D-scanned assets (https://quixel.com/megascans/home, accessed on 30 May 2025). To derive generalizable design guidelines for presence, these IVEs encompass a wide range of settings, ranging from artificial to natural spaces and from indoor to outdoor areas (see Figure 5). All IVR-Scenarios are based on real-life scenarios and are designed based on photo-references to enable an objective assessment of visual fidelity based on a real-world paradigm. The Forest IVE depicts a large forest with paths through a wide ravine. The task is to find three metal watering cans hidden near the paths (Figure 5, left). The Warehouse IVE portrays the inside of a warehouse with multiple aisles of shelves, storing cardboard boxes on wooden pallets. The task is to find and collect red paint cans (Figure 5, middle-left). The Antique Shop IVE presents a small, cluttered indoor shop filled with old objects of varying value. The windows offer a view of a small and quiet street. The task is to locate and collect hammers hidden throughout the shop (Figure 5, middle-right). The Urban IVE shows a large courtyard, surrounded by different buildings, containing parking lots, small green spaces, and a construction site. The task is to find and collect scuffed soccer balls (Figure 5, right).

5.2. Experimenter Interface

The experimenter interface is implemented as a web application using the JavaScript framework Vue.js (https://vuejs.org/, accessed on 30 May 2025). It is designed to navigate the experimenter through the study and thereby preventing possible operating errors. To ensure the double-blind principle [99], it limits the information the experimenter receives during the experiment to minimize biases in the interaction with the participant. Therefore, the view of the subject in the IVE is not shown to the experimenter during the experiment.

5.2.1. UI and Study Modes

At the beginning of each experiment, the interface allows for the selection of one of three study modes: (1) explorative study, (2) comparative study, and (3) debug mode (Appendix A, Figure A1). The latter is allowing for more operating freedom and insights regarding the experimental status (e.g., for pre-testing; Appendix A, Figure A2). The comparative study mode is added as the studies progress according to the methodology (see Section 4).

After entering the subject ID and choosing the explorative mode, the system determines quasi-randomly (see Section 4.1) in what order the IVs are configured and the order in which the IVEs are presented. All of this information is hidden for the experimenter. The next view contains the four configuration steps needed for the exploratory study (Appendix A, Figure A3). The experimenter increases, decreases, or continues to the next step according to the exploratory study procedure (see Section 4.1). After the fourth step is completed, the task can be started through the interface. During the task, the interface presents the subtask progression to the experimenter, but without revealing the status or type of specific subtasks. The configuration and IVR task get repeated for every IVE. Lastly, the experimenter has the option to add notes of unusual observations or occurrences during the experiment and to export all data recorded by the tool.

5.2.2. Communication Between Interface and IVR Application

The communication between the two components is by definition unidirectional, with the web-based experimenter interface sending requests to the IVR application. This communication is realized through http requests that are sent to a web server running in the Unreal Engine. This server is part of the remote control API (https://dev.epicgames.com/documentation/de-de/unreal-engine/remote-control-for-unreal-engine, accessed on 30 May 2025). It allows for the execution of functions in instantiated objects at the currently loaded level (IVE); for example, the manipulation of IVs (see Figure 6).

6. Evaluation and Practical Application

Our developed methodology (Section 4) and study tool (Section 5) underwent a formative expert evaluation to test the general usability and applicability for exploratory VR studies. To further evaluate our approach, we conducted exploratory user studies within the scope of the AgeVR research project with N = 115 participants. So far, our evaluation covers the initial half of our methodology, up to the stage of concluding the exploratory study (confirmatory study results are reported in a future publication).

6.1. Formative Expert Evaluation

6.1.1. Method

The evaluation was conducted with 6 experts (6 male, 22–32 years), with prior scientific or development experience in IVR. The exploratory study procedure (Section 4.1) was simulated by conducting a role play in pairs of two with one expert as experimenter and the other as participant. The participant’s role was to configure two IVs and complete a task within the IVE. The experimenter’s role was to control the experiment through the interface following written step-by-step instructions. Subsequently, the experimenter evaluated the usability of the interface through the ISONORM 9241/110-S usability questionnaire [100], while the subject role player’s presence was measured via the IGroup Presence Questionnaire (IPQ) [8] and Slater–Usoh–Steed presence questionnaires (SUSs) [12]. Afterwards, the experiment was simulated a second time, swapping the roles. Finally, a semi-structured interview with each individual expert was conducted. It focused on the adequacy of the study design and the tool’s implementation of the methodology as well as its feasibility for large and diverse subject samples. Due to its formative character, the evaluation was limited to the forest IVE (Section 5.1.2).

6.1.2. Results

Semi-Structured Interviews

Individual experts are identified via the pseudonyms E1-E6 in the following text.

Considering the suitability of the proposed methodology and the corresponding study design, it was stressed that it is essential for participants to have the correct mental model of the sense of presence in order to carry out the configuration task effectively (E2). Furthermore, unknown limits of the IVR hardware (e.g., maximum brightness of the HMD) could lead to confusion (E1), as well as certain IVs starting at a zero level, such as illumination intensity and contrast ratio (E3, E5). It was suggested that some of these IVs should rather be manipulated on a continuum rather than in discrete steps (E3). The importance of considering interference between IVs as well as auditory aspects in the development of design guidelines was emphasized (E2). Three experts questioned the double-blind principle, saying that it might hinder communication with participants (E1, E3, E4). However, it appeared to reduce biases, as four experts wanted more feedback on the status of the subjects (E1, E2, E4, E5), which was justified, e.g., by “being able to already think about result analysis during the experiment” (E1). Addressing the diversity of participants, experts stated that interaction with IVR technology poses different difficulties depending on the user groups (E4, E6). Users with an affinity for technology may expect more configuration options and wider ranges (E5), while too many options may lead to mental fatigue, not only in inexperienced users (E2). It was suggested to simplify the IVR task by adding optional task objects (E1). Being concerned with mobility requirements, especially for older users, experts suggested to reevaluate the time intervals that users were required to stand (E6) and how intensively they must bend down and pick up objects (E5). For the development of adequate IVR scenarios, E6 recommended focusing on the plausibility of IVE in terms of boundaries and lighting. Depending on the scenario, changes in IVs may not be recognizable (e.g., if there are few objects/texture details) (E5, E2). Additionally, it was stated that, when it comes to presence, there may be several different IV configurations that can be considered realistic (e.g., depending on the time of day or the underlying narrative) (E5). Furthermore, E4 stressed that IVs may have different influences in each scenario, which hints at the importance of our multi-scenario approach to achieve generalizability and counteract contradictory results.

Regarding how well the study tool implements the methodology and study procedure, two experts stated that they had not experienced any problems with the system during the experiment. In the following section, encountered problems and positive feedback by the experts are listed separately for the experimenter interface and the IVR application, respectively.

When asked about the experimenter interface, five of the six experts stated that they were adequately supported by the system during the procedure and that they could well imagine using it to conduct a large-scale exploratory study. The main reason given was the linear and stringent process guidance circumventing possible diversions (E2, E5, E6). Four experts described the UI as “tidy”, “uniformly designed”, and “easy to use” (E1, E2, E3, E4). The use of a web application was considered a technological advantage (E1). Several experts noted that error prevention and management could be improved. This included having clearer step descriptions and confirmation prompts for critical actions (E1), as well as checking subject IDs for duplicates when creating the study (E3). Another suggestion was the option to decrement IVs to enable the direct correction of configuration errors by the experimenter (E1, E2, E5, E6). The interface seemed to comply with the double-blind principle, as four experts stated that they did not know which specific IVs they were manipulating (E1, E3, E5, E6). While one expert described the labeling of the buttons as “unambiguous” (E3), another expert stated that the labels could easily be confused with each other (E1). It was also noted that progress bars were not directly recognized as such (E6). Additional features suggested include personalization, e.g., a dark mode (E1), keyboard shortcuts (E1), and the automatic loading of subject IDs into the system, e.g., via a text file (E3).

When asked about the IVR application, five experts described the forest IVE as very realistic due to its high level of fidelity, detail (E1, E2, E4, E5, E6), and large size (E1, E5). Two experts addressed navigability and wayfinding in the IVE, stating that the IVE and the teleport locomotion were suitable for carrying out the search task (E4), but especially for older users, this should be retested (E3). One expert confirmed that the task fulfilled its purpose of ensuring that users actively explore the IVE (E1); however, he found the distance of the teleport impractically short. Three experts wished for the IVE’s boundaries to be visualized more prominently (E2, E4, E5). Furthermore, an additional tutorial IVE was suggested to learn the teleport locomotion and picking up objects (E3). Regarding the task objects, marking already found objects was suggested to avoid confusion, as well as more realistic physics when picking them up (E3). Some experts mentioned performance problems, i.e., frame drops (E5, E6) and aliasing effects (E4, E6). It was suggested to perform additional performance tests to reduce the risk of cybersickness (E3).

Questionnaires

Seven-point Likert scales were used for ISONORM 9241/110-S, SUS, and IPQ. The SUS count sums up ratings of 6 or higher, up to a maximum of 6.

Results of the ISONORM 9241/110-S usability questionnaire ranged from moderate to high values $(Min=4.2,Max=5.95,M=5.43,SD=0.65)$. In line with the interview results, the subscales’ self-descriptiveness $(M=4.61,SD=0.92)$, error tolerance $(M=4.39,SD=0.59)$, and individualizability $(M=4.50,SD=0.29)$ were noticeably lower than those of task appropriateness $(M=5.95,SD=0.63)$, expectation conformity $(M=5.28,SD=1.59)$, learnability $(M=5.56,SD=0.59)$, and controllability $(M=5.39,SD=0.69)$. However, the variance between experts was higher for individualizability than for self-descriptiveness and error tolerance (see Figure 7).

Presence ratings obtained were divergent, with SUS scores ranging from 2.5 to 6.67, $(M=5.06,SD=1.62)$ and SUS counts ranging from 0 to 6 $(M=3.50,SD=2.59)$. The IPQ total score ranged from 3.64 to 5.43 $(M=4.68,SD=0.66)$; the subscales of General Presence $(M=5.83,SD=1.33)$ and Spatial Presence $(M=5.30,SD=0.59)$ were considerably lower than those of Involvement $(M=4.21,SD=0.73)$ and Realness $(M=4.22,SD=1.24)$. However, it is unclear which phase of the IVR-exposure the presence ratings refer to (during configuration or IVR-task).

6.2. Conduction of Exploratory IVR Studies in the Context of a Research Project

As a first practical application of the proposed methodology (Section 4), and the developed study tool (Section 5), we conducted an exploratory study with 115 participants in the context of the AgeVR (https://www.imis.uni-luebeck.de/de/forschung/projekte/agevr, accessed on 30 May 2025) research project. The study tool was previously adapted based on the results of the formative expert evaluation: duplicate checking for subject IDs was implemented, button labels were unified, and possibilities to correct IV configuration errors (decrement button) and to annotate experiment data (notes text field) were added. Furthermore, to prevent frame drops and aliasing effects, a high-performance VR-PC (GPU: GeForce RTX 4090, 24GB; CPU: AMD Ryzen 7 7700X, 8x 4.50 GHz; RAM:64 GB DDR5, 4800 MHz, CL40, 2x 32B Kit) was used in combination with a high-resolution HMD (https://varjo.com/products/varjo-xr-3/, accessed on 30 May 2025).

6.2.1. Method

This evaluation serves as an instantiation of the proposed methodology within the context of a concrete research project, which defines specific technological factors as IVs as well as relevant human and environment factors that need to be controlled (see Figure 8). The AgeVR research project aims to identify age-specific design guidelines for presence in IVR. The presence factors focused on in this project are those of visual display fidelity and associated age-related user characteristics. Technological factors serving as IVs for the experiment were the exemplary ones implemented in Section 5 (I, C, O, T). As a minimal reproducible example, only two IVs were analyzed at a time. Each of the two pairs (I, C and O, T) was configured in two different IVR scenarios (forest and warehouse, see Section 5.1.2) according to the exploratory study process (Section 4.1), leading to four IVR exposures for each participant. Age-related user characteristics were assessed prior to each study (visual short-term memory (block span), spatial imagination, visual system performance (visus, stereopsis, phorieder, color sense, visual field, contrast vision, glare sensitivity)). The main environment factors were the room characteristics stabilized by conducting all experiments in the same IVR laboratory, as well as a tracking/operating area of 4x4m. The developed presence short scale (see Appendix B, Table A1) was administered after each IVR exposure. An age-diverse sample was recruited (N = 115, 60 female, 55 male, age = 18–88, $mea{n}_{age}$ = 45.71, SD = 18.77).

6.2.2. Results

The following analysis is restricted to data that are essential for assessing the applicability of the methodology and the correct execution of the experiments. Data addressing the research questions are presented in a subsequent publication.

Frequently Encountered Problems

All 115 participants successfully completed the exploratory study process, despite 84% of participants having little or no IVR experience (55% were novice users). This means that all participants understood how to configure the IV in the IVR scenario and how to find and collect the task objects in the IVE. Then, 63 of the studies did not encounter any problems during the study procedure. The remaining 52 studies encountered problems that can by classified as technological errors, errors by the experimenter, and errors relating to the methodology (see Figure 9). In terms of technological errors, malfunctioning of the IVR software, i.e., the Unreal Engine application (see Section 5.1.1) occurred in 20 studies. This included unforeseen crashes of the application, physics glitches in task objects, and the teleport locomotion not working properly. Malfunctions in the IVR hardware occurred in 34 studies, specifically non-functioning controllers and blackscreens within the HMD, due to loose or damaged cable connections (as the material was heavily stressed by over 400 IVR exposures in total). Errors by the experimenter can be attributed to a lack of attention (25), such as forgetting to turn on the controllers before the experiment, or to plug the headphones into the HMD, or incorrect operations in the experimenter interface (2), such as incrementing/decrementing an IV by mistake and accidentally loading the next IVR scenario. In terms of methodology, communication errors between the experimenter and participant occurred in 13 studies, as task instructions or term definitions (i.e., presence) appeared to have been ambiguous since they were not correctly understood. Furthermore, the methodological requirements could not be met in 19 studies, as the experimenter sometimes had to give additional information for solving the search task or had to physically intervene (restart the controller, adjust the HMD/cable), or the experimenter’s voice led to a break in presence for some participants.

Configuration of IV for Optimal Presence

When analyzing the presence ratings, seven datasets had to be excluded due to having more than one blackscreen or data loss due to system crashes. In the IVR exposure, participants were tasked with finding their personal optimal setting for presence by configuring the two IVs (IxC or OxT, respectively). As a control measure, presence was assessed after performing the IVR search task with the set configuration. Figure 10 shows the distribution of post hoc ratings for all presence subcomponents (data were summarized for both IVR scenarios; thus, with N = 108 participants, there were 216 values for each subcomponent). There were only minor differences between the types of IV configured; IxC and OxT showed roughly the same distribution (${\Delta }_{mean}=2\%,{\Delta }_{max}=5\%$, for each level of the 5-point Likert scale). However, the subcomponents themselves differed substantially in how well participants were able to achieve their optimal configurations. Configuring the IV worked best for the subcomponents of general presence and involvement, with 94–96% of ratings being in the positive spectrum. A total of 60–63% of participants reached the maximum value for involvement, while for general presence, it was 47–49%. Regarding plausibility illusion, 75% were in the positive spectrum in both IV configurations (IxC and OxT), but only 27–29% at maximum value. Ratings for spatial/physical presence and self-presence ratings were less clear, both having 59% positive ratings after the OxT configuration and 62% after the IxC configuration and both having more negative (27–28% for spatial/physical presence, 25–29% for self-presence) than neutral ratings (11–14% for spatial/physical presence, 12–14% for self-presence). Regarding social presence, participants were least likely to achieve their optimal configuration as only 15–17% of ratings were positive. Here, most of the ratings were in the neutral spectrum (49–52%). All subscales violate the assumption of normal distribution.

7. Discussion

Research on presence factors is impeded by the substantial number of identified factors, contradictory results on their influence, and a lack of clarity regarding their interaction (Section 1). Based on the analysis of the current literature, we proposed a research methodology to address these problems using a two-step procedure, combining exploratory and confirmatory paradigms (Section 4). As current IVR study tools do not focus on presence and its determinants (Section 3), we developed our own tool consisting of two components: an IVR app based on the Unreal Engine for designing and displaying various IVR scenarios and a browser-based experimenter interface, enabling precise control of presence factors and study procedure (Section 5). A formative expert evaluation of both the methodology and the study tool was performed by role-playing experiments (Section 6.1). A first practical application in exploratory studies of the AgeVR research project enabled the first evaluation with study participants (Section 6.2).

Qualitative expert feedback suggests that the methodology can reduce biases (double blind principle) and improve the generalizability of results (multi-scenario approach). Experts had some concerns regarding understandability, physical strain, and cognitive load for certain user groups (unexperienced users, older adults). This was not confirmed in the exploratory studies, as all participants were able to complete the study process despite being an age-heterogeneous sample (18–88 years) and despite the majority being inexperienced or novice users. However, we did not account for extreme user groups (e.g., severe motion sickness or cognitive impairments), which may limit external validity and follow-up studies should include a wider range of users. Methodological problems that did occur were primarily communication errors between experimenter and participant as well as additional assistance that had to be given (additional information or physical intervention). Thus, task instructions and term definitions (i.e., presence) must be reformulated to make them unambiguous in further studies. Another issue of the proposed methodology was that the experimenter’s voice led to a break in presence for some participants. This could be addressed by giving the experimenter a virtual representation (avatar) within the IVE or by implementing a (virtual) communication device that is coherent to the IVR scenario (e.g., phone or walkie-talkie).

Considering the study tool, five of the six experts felt adequately supported during the study procedure and could well imagine conducting a large-scale IVR study using the experimenter interface. Expert usability ratings for the study tool reached moderate to high values. Self-descriptiveness, error tolerance, and individualizability were rated noticeably lower than the other scales. Since the tool is supposed to strictly guide the experimental process and minimize derivation, individualizability is not a desired property. Thus, only measures were taken to improve self-descriptiveness by unifying button labels and to increase error tolerance by implementing duplicate checking for subject ID and possibilities to correct IV configuration errors as suggested by the experts. Consequently, in the subsequent exploratory studies with participants, only two operating errors occurred in all 115 studies. Performance issues mentioned by the experts were addressed by upgrading the hardware used (VR-PC and HMD, see Section 6.2) and did not reoccur in the exploratory studies. However, software and hardware errors remained the most frequent issue within the exploratory studies, which could have distorted presence values. The former shows that additional debugging of our Unreal Engine-based IVR application is necessary to prevent unforeseen crashes, physics glitches, and other errors, while the web-based experimenter interface worked without difficulty. The latter demonstrates how high frequency use can put a strain on IVR equipment and material. We learned from over 400 IVR exposures that, e.g., cables simply reach their physical limits and must be replaced at some point.

To assess the combined effectiveness of the methodology and the study tool, the presence ratings obtained from the exploratory studies can be consulted. These show how well participants were able to achieve their optimal IV configuration for presence. While the ratings only differ marginally between the two pairs of IV (I,C and O,T), substantial differences could be identified between the different subcomponents of presence. Almost all participants were able to achieve a sense of general presence and involvement with their configurations, and two-thirds of participants had a plausibility illusion (Section 6.2.2). Only slightly more than half of the participants achieved a sense of self-presence. This was likely due to the chosen default value for the avatar, being a hands-only representation which may not have met the participants’ expectations for a realistic virtual body. Furthermore, the only IV that affected the avatar was the texture resolution variable, while others had a more indirect influence (illumination intensity, contrast ratio). Spatial/physical presence yielded ratings similar to social presence. Contrary to social presence, however, the IV configuration strongly influenced the visual appearance of virtual objects in the IVE. It is, therefore, not clear why about half of the participants did not manage to achieve spatial/physical presence with their IV configuration. One possible explanation might be that participants rated their spatial/physical presence for the whole IVR exposure (including the configuration task) and not only for the search task (as instructed). Thus, the great alternations in visual appearance might have led to conflicting perceptions regarding spatial/physical presence. Only one-sixth of participants achieved a sense of social presence with their IV configurations. The reason for that is probably the choice of IVR scenarios and their respective default values, as neither the forest nor the warehouse IVE included any form of social actor. Also, the IV configuration would not have influenced social actors, which shows the limited applicability of social presence in certain experimental settings. Thus, in terms of self-presence and social presence, results imply that (1) the IVs investigated (I, C, O, T) seem to have no influence. As the study tool will be extended with additional presence factors, their influences might be identified in future studies. (2) Default values play an important role within the IVR scenarios, as those chosen for self-presence and social presence (hands-only avatar and no social actors, respectively) seem to have inhibited the experience. Especially, if IVs with a greater impact on self-presence and social presence are to be investigated in future studies, more sophisticated body representations and the addition of agent or user-controlled social actors should be introduced to enable a more precise investigation of their influence.

8. Conclusions

In this paper, we systematically analyzed the state of research and derived measures for the systematization of presence studies (Section 4). Our proposed methodology addresses the fundamental problem that the isolation of presence factors is not always possible. The practical workaround is achieved by (1) always focusing on the combined influence of several presence factors, and (2) following a two-step procedure where an exploratory investigation serves as a basis for subsequent hypothesis formation and confirmatory investigations. Our exemplary case was to enable age-differentiated investigations of different presence factors by customizing IVEs to meet varying age-specific needs. The feasibility of our methodology and study tool was confirmed by expert feedback, as well as data from 115 successfully completed exploratory studies with participants of various ages. The exploratory study procedure works for general presence, involvement, and plausibility illusion. Measures that will enable the seamless investigation of the remaining presence subcomponents have been proposed (see Section 7). Our next step is to perform the cluster analysis prescribed by the methodology to develop and test hypotheses in the confirmatory studies. Beyond our own research agenda, the developed study tool was made publicly available as an open source project to enable other researchers to conduct custom experiments on presence factors in IVR (https://github.com/RBW1999/VR-StudySystem-Template-VE/blob/main/README.md, accessed on 30 May 2025). Currently, the tool supports the manipulation of four IVs (presence factors) in the category of visual display fidelity (I,C, O T). As our system was designed with expandability in mind, additional presence factors, IVR scenarios, and IVR tasks can easily be added.

9. Limitations

The evaluation of the proposed methodology was limited to $p=2$ independent variables; $p>2$ has yet to be validated. However, the complexity of configuration and interdependence of the presence factors will increase substantially as additional IVs are added. So far, our evaluation covers only the initial half of our methodology, up to the stage of concluding the exploratory study, because the AgeVR research project used for validation has not yet progressed beyond this step. From the experimental studies, only data relevant to the methodology’s applicability and experiment execution were analyzed; data addressing the project’s research questions will be published separately. Thus, our next step is to perform the cluster analysis, derive hypotheses, and conduct confirmatory studies, serving (1) the further evaluation of our methodology and (2) the explicit goals of the research project (age-specific design guidelines for presence).

The proposed methodology itself is limited in its applicability, as it currently accounts only for certain technological presence factors (visual display fidelity), while the remaining ones, as well as human and environmental factors, are kept in a controlled but undynamic state. To enable a holistic investigation, ways must be found to incorporate these presence factors as IVs into IVR experiments. Furthermore, we chose 17 discrete steps for the manipulation of presence factors whose perceptual distinctiveness could not yet be empirically validated. Moreover, for certain factors, a continuous approach might be more adequate (as suggested by experts). This could on the one hand enhance precision when detecting optimal configurations; on the other hand, it could confound optimal values due to the subject’s reaction times. Furthermore, the test–retest reliability of the IVs has not yet been formally assessed; thus, their current support remains indirect and interpretive.

The IVR task provided by the study tool is currently limited to simple teleport locomotion and object grasping, chosen as examples for standard IVR interactions. How well these results can be transferred to more intricate IVR interactions, such as hand gestures, passive or active haptic feedback, or even redirected walking techniques, remains unclear. Furthermore, the IVEs implemented so far focus on replicating real-life contexts (forest, warehouse, courtyard, shop). While this may serve as a good starting point for investigating presence factors, it does not exhaust its possibilities. We plan to add various other virtual scenarios (e.g., fantasy settings, abstract scenarios) in the future to fully leverage the design space for virtual environments.