Immersive Learning

Definition

Immersive learning conceptualizes education as a set of active phenomenological experiences that are based on presence. Immersive learning can be implemented using both physical and digital means, such as virtual reality and augmented reality.

1. History and Origin

For millennia, humans learned by doing. From the early days of hunters and food collectors to the first communities, humans were learning from nature and from each others through experience. Experiential learning is at the core of our existence and is fundamental in our physiological, behavioral and psychological development.

Learning in later stages of human history heavily relied on the oral proliferation of stories and tales from one generation to the other around the campfire or in public performances of theater and epic lyric poetry. Although this type of learning was separating the content of the learned experience from its context, it allowed societies to transfer knowledge cross-generationally and at a bigger scale. In a way, the lack of context was compensated by an increased engagement through narrative, imagination, staged performances, and often music. Humans were starting to be immersed, not only in the environments, conditions and situations, but in stories. The invention of writing and the authoring of text, manuscripts and eventually books unlocked learning at a substantial scale but further reduced the distance between the content and context of the learning experience. The reduction of learning from experience to reading text offered the tremendous potential for humans to increase their knowledge through someone else’s experience in exchange for personalizing that same experience. Learning occurs naturally and experientially in the real context, which occurs within the genuine three-dimensional physical environment. When this is impossible, too dangerous or unproductive, this condition can be approximated in artificial virtual spaces, which can emulate the authentic context and elicit immersion by their presence within the simulated system [1], by the diegetic concern with the contextual meaning and narrative [2] and/or by the psychological absorption with the challenges and tasks [2]. This type of educational experience is called immersive learning. The original context can be abstracted in many ways to represent various aspects of the intended learning using fantasy and stylized and illustrated worlds, similar to video games [3].

In the Socratic philosophical dialogs Republic, About Justice and Politics, Plato captured the allegory of the cave with respect to the nature of physical reality [4]. In this symbolic storified representation, humanity’s state resembles prisoners chained in a dark subterranean cave. Their only perception of reality is shadows of real objects’ images and other elements of the physical reality being projected onto the cave’s wall as they are held in front of a light-emitting source, a fire. The discussion concludes that the duty of a philosopher as an enlightened citizen is to break free of these restrictions, escape from the cave and experience that authentic light as well as the true forms of the physical objects. The cave allegory provides a foreshadowing of alternative reality representations while at the same time demonstrating the potentially paralyzing effect of legacy paradigms, systems and methods in education.

2. Theoretical Foundations of Immersion

Immersion is a fundamental state of human consciousness that stems from the willingness to engage with a certain attention-capturing and interest-inducing stimulus [3]. It is a psychological sensation that is metaphorically derived from the notion of diving into water and the experience of being submerged into a reality that surrounds and infiltrates the entire human being and the senses [5]. Immersion is a subjective phenomenological experience of partaking in an all-inclusive experience where self-consciousness and time awareness fade. It is possible to be both passively and actively immersed in external elements or internal motivations. This explains why humans can immerse themselves into a book or a theatrical play. A systematic review of immersion’s definition distinguishes, on the one hand, a system and perceptual immersion based on the properties of used technology; on the other hand, the authors distinguish the psychological immersion caused by components of story, action or engagement [2]. In other words, immersion can be the result of (a) affordances of a technological system that monopolizes users’ senses; (b) the psychological appeal of an imaginary or fictional world or narrative; and (c) challenges, tasks or actions to achieve an objective or overcome obstacles. In the context of extended reality (XR) and the Metaverse, immersion has also been used to point to objective, exocentric technological features [1].

Sensory system immersion can involve the voluntary suspension of disbelief and electing to view a synthetic world as a basis for self-reference [2]. Moreover, it can also induce a logic-defying involuntary suspension of disbelief as demonstrated by the virtual plank experiment; when users attempt to walk on a lean, virtual plank that appears to be on the top of a skyscraper, it triggers a fear of height despite their undeniable knowledge that they are actually located in an indoor laboratory room [6]. This phenomenon is called presence or telepresence, the subjective feeling derived from the failure to perceive or acknowledge the involvement of technology or human-made media in an experience [7], which is equivalent with the notion of being in a different place than where the actual person’s body resides (spatial presence/duality of presence). A meta-analysis on system immersion has revealed that functional fidelity is more important than the visual quality of the graphics towards spatial presence [8]. This is consistent with the perspective on immersion as arising from presence within the system, narrative meaning and challenges/tasks (i.e., agency). The argument for this consistency is that increases in the visual quality of graphics will tend to bring diminishing changes to meaning (narrative) or presence (notion of being within the simulated system), whereas functional fidelity will tend to expose novel areas of agency.

When immersion is combined with high levels of engagement with an activity under an optimal ratio between competency and task difficulty, a sense of flow can be facilitated [9]. Flow is a state of heightened concentration that ignores external stimuli while successfully executing a demanding activity [9]. Specifically, the challenge in focus must not be too easy or too difficult to undertake as the task will elicit emotions of boredom and indifference or anxiety and frustration, respectively. Narrative immersion arises from spatial, temporal and emotional narrative elements such as plot types, characters, relationships, events, and metaphors [10,11]. Users develop feelings about the story, characters or world that they are interested in and motivated to pursue [10,12].

3. Immersive Learning Methods and Technologies

Immersive learning can be implemented using both physical and digital means, methods and technologies.

3.1. Physical Immersive Learning Methods

Physical, analog immersive learning methods include simulations, role plays and games. Although these three methods share similarities and differences, they all belong to a continuum where overlaps are possible [13]. They all have in common the tacit notion that the aim of education should not be content delivery but behavioral change towards a desirable end goal through the self-regulated activation of learners.

Simulations provide a structured, hands-on, realistic representation of a real-world situation or event with the intention of familiarizing students with the procedures of professional practice [14]. Simulations are valuable for education and training because they activate cognitive, affective and psychomotor learning procedures, rendering them more effective than other passive instructional techniques [15]. Moot court competitions are academic-simulated appellate court procedures [16]. Learning in these mock legal cases happens as students argue against each other as attorneys who produce written and oral arguments in a court of law.

One form of organized role play is live-action role-playing (LARP). LARP constitutes a complex social experience that involves freeform roleplay. Educational LARP can be organized for experiential learning around improvisation, imagination and experimentation [17]. Educational LARP involves (i) an introductory session, where participants are informed about the context; (ii) the gameplay itself; and (iii) a reflective debrief [18]. Players assume a specific role in a fictional world with objectives that are within a set of agreed rules. The whole game can last from a few hours to until several days, which is usually controlled by one or more gamemaster. LARP has many similarities with digital massively multiplayer online role-playing games [19].

Games such as serious, epistemic escape rooms can be used to create immersive experiences with a pedagogical objective. These breakout games are organized around individual or team missions, usually aimed at finding a way out of confined spaces or solving a mystery within a limited time span [20]. The sense of immersion is created through the room’s narrative and characters being arranged within a theme. Themes such as World War II, science fiction or ancient Egypt are semiotic domains and fields of collective consciousness with unique attributes [21]. Escape rooms can be organized using both the physical space and with the help of digital technologies [22,23].

Other types of games with narratives that can facilitate psychological immersion are tabletop board games [24]. For example, the fantasy board game Dungeons and Dragons can be used as a context to orchestrate story-driven learning challenges for the practice of authentic skills [25].

3.2. Digital Immersive Technologies

A series of digital technologies can be utilized to facilitate immersive learning, namely virtual reality (VR), augmented reality (AR) and mixed reality (MR). These technologies are categorized as immersive technologies and can be summarized under the all-encompassing umbrella term extended reality or cross-reality (XR) [26]. XR, along with other technologies such as robotics, will be essential for the development of a new emerging computing paradigm: spatial computing [27]. Spatial computing technologies assist our transition from an existing flat or reduced dimension worldwide data network to an emerging immersive worldwide data ecosystem with spatial awareness and characteristics, giving physical objects virtual properties and vice-versa while augmenting our sensing and performing abilities. Developing relations between data and representations in extended physical dimensions requires a new, multimodal, advanced interaction model for the users, as well as a new infrastructure to support it. In an entirely new form of a virtual or hybrid synthetic human experience such as in spatial computing, the paradigm shifts content generation and consumption from representational to localized and often embodied, producing an effect of body awareness and self-presence within the content and the information that it carries.

VR refers to digital, computer-generated three-dimensional environments that are completely separated from physical reality [28]. The cutting edge of immersive VR technology uses head-mounted displays to create the sense of occlusion and separate the human senses from the physical environment that takes full control of them. Moreover, there are room-size VR applications based on 3D projection mapping, similar to the CAVE^TM technology from the 1990s, that can enable complex simulations with specialized real equipment and devices in a multi-person, high-resolution 3D video and audio environment [29]. AR operates in our physical surroundings by overlaying digital elements and objects that are visible through smart, wearable, see-through devices such as head mounted displays, glasses, mobile phones, glass windows, and car windshields [30]. MR has multiple definitions and interpretations, which mostly point to an advanced version of AR combining elements that track and interact dynamically in real-time with the physical environment [31]. An MR example would feature a digital non-player character guided by artificial intelligence that moves and hides behind the furniture of our living room.

XR technologies can operate separately and on a single-user mode; however, when the individual barriers are overcome and users meet in multiuser social spaces, they are entering the Metaverse, an open, interoperable network of virtual worlds, the Internet of place in the context of Web 3.0 [32]. Furthermore, the Metaverse is being built upon links between XR technologies, allowing for a seamless transition from the physical to the virtual, where users can meet, communicate, operate, work, and play on equal footing as their physical, digital (avatar) or projected self (hologram), regardless of their geographical location or preferred technological medium. For instance, meetings in the Metaverse will be organized in a phygital way, i.e., merging physical and digital by having a variety of participating modes: physical by joining a location; and remote–digital through an avatar or a holographic projection. Virtual participants will be visible to physical participants through wearable smart glasses, human-size large screens or other current and future immersive displays.

4. Immersive Learning Design

The theory of situated cognition posits that the essence of learning cannot happen outside of its authentic situation [33]. Knowledge resides within the activity, context and culture. Hence, deliberate immersive practice with and within realistic representations of challenging tasks having replicated versions of the original physical and social environment should be a priority in learning design. This notion is corroborated by a meta-review by Beck, Morgado and O’Shea [34] that illuminated augmented context, simulation and skill training as the most applied learning methods.

This notion is clearly illustrated in the four-dimensional framework [35]. The framework proposes four major components for immersive learning: learners characteristics, pedagogical methods, representational features, and context. The main premises of the framework were corroborated by the model for deep and meaningful learning in social VR environments [36]. The model suggests first identifying users’ needs, knowledge and skills so as to identify potential gaps; then, the model suggests making conscious pedagogical and andragogical theory-driven decisions regarding curriculum design and learning activities. It adds the dimension of technology in terms of usability, performance and robustness to properly execute the pedagogical vision of instructional designers.

Regarding representational features, learning design in immersive VR can include four categories of design attributes of varying intensity according to a systematic review by Won et al. [37]:

• Sensory, related to the representational fidelity of the virtual environment: (i) vision and graphics; (ii) sound; (iii) haptic feedback.
• Actional: (i) user control of the experience; (ii) virtual body movement.
• Narrative/diegesis/story related to content and learning activities: (i) roles; (ii) contextual stories; (iii) challenging tasks and achievements.
• Social: interaction with peers.

In light of the three aforementioned dimensions of immersion, these elements can be rearranged as follows: sensory features are linked to system immersion and story elements (i) and (ii) are relevant to narrative immersion, while actional features, social interaction and challenging tasks/achievements can be associated with challenge immersion.

A systematic review compiled a taxonomy of instructional methods supported by augmented reality applications, which is especially relevant for fields in science, technology, engineering, and mathematics [30]. This taxonomy features five stages starting with visualization, where educators provide 3D objects that students can observe and manipulate. The next step is integrating activities based on 3D content, while the third stage is adding the social element of cooperation or collaboration; students receive inquiry tasks and projects for completion through teamwork [32]. In the fourth level, immersive experiences are orchestrated through virtual simulations, experiential problem-solving and games. Experiential learning is the dominant learning theory used for education in VR [38]. The ultimate frontier is facilitating the transition of learners towards skill mastery through autonomous, self-regulated learning and production or artifacts.

Another systematic review identified five types of learning designs in immersive VR: passive observation, virtual object manipulation, active context participation, realistic practice, and repetitive practice based on smart and social personalized feedback [37]. These clusters in VR correspond to a very high degree to the aforementioned AR-based learning design taxonomy [30], a fact that supports their respective generalization potential for XR technologies in the Metaverse.

The Cognitive Affective Model of Immersive Learning (CAMIL) compiles research evidence into a comprehensive framework for immersive VR under the premise that learning outcomes depend both on the affordance of the medium and applied pedagogical methods as well as their level of appropriateness for the particular medium [39]; in the case of immersive VR, the fundamental affordances are presence and agency. Furthermore, instructional design should consider emotional, motivational and cognitive factors such as interest, motivation, self-efficacy, embodiment, cognitive load, and self-regulation [1,39].

5. Immersive Learning Impacts and Challenges

Why is immersive learning in the Metaverse important or relevant, and for which knowledge types is it important and relevant? The significance of immersive learning is illuminated by neuroscientific insights on human memory and, specifically, the distinction between semantic and episodic memory [40]. Semantic memory refers to general, encyclopedic and concept-related knowledge. However, abstract conceptual representations are not sufficient to epitomize the full spectrum of complex knowledge. Episodic memory is the human ability to store long-term complete personal experiences in their multimodal tempo-spatial context (vision, sound, place, smell, etc.) [41]. Immersive educational interventions triggering episodic memory increase retention and facilitate deep meaningful learning [36]. Several reviews have documented the benefits of immersive learning in the cognitive, affective–emotional and psychomotor domain, marking a paradigm shift for communication in comparison to the era of social media.

5.1. Communication in the Social Media and the Metaverse

The rise of social media has marked the transition from the read-only, static version of the World Wide Web (web 1.0) to its read–write, dynamic iteration (web 2.0). In other words, users can easily create, publish and share content with the world without the need for specialized computer science knowledge and programming skills [42]. Open social media platforms such as Twitter, Facebook and YouTube have facilitated human communication and networking on a global scale; their proliferation, combined with the ubiquitous presence of smart mobile phones, has created new virtual marketplaces, discourse habits, business avenues, and learning modes. Social networking platforms became new digital third places, avenues of private and social discourse where people socialize besides at work and at home [43]. More specifically, social networking platforms promoted and increased the use of flexible, asynchronous communication [44]. Users could send a private message, a public comment or publish a post, and recipients could respond affectively through icons (e.g., like button), sharing (e.g., retweet) or commenting with a time delay. These actions in mass-scale describe the undermined human communication from social to parasocial interaction [45]. Asynchronous communication through text is more cumbersome for expressing the full range of human emotions and thus, it is restricting or prone to miscommunication. A handful of few-line messages does not constitute a rich human communication. Similarly, a friend request is by no means an equivalent to the process of building trust and forging a true and honest friendship.

More importantly, social media platforms operate on a basis of user engagement through methods of distraction and instant user gratification [46]; however, this inhibits focused attention in all facets. Automatic notifications based on algorithmic optimization aim at triggering maximum and continuous user engagement, which can lead vulnerable persons to addictive, excessive online behaviors [46]. Immersive technologies in the context of social media can assist the rediscovery and establishment of true togetherness (despite eventual distance), focused attention and active listening as is experienced in physical spaces. Three-dimensional, avatar-based online spaces can be used as online third spaces [47]. The need for such places was clearly demonstrated during the COVID-19 pandemic, where people used any online communication medium to socialize. In the Metaverse, this shift will be achieved through the amplification of authentic, effective educational experiences in a synchronous mode.

5.2. Benefits in the Psychomotor, Affective and Cognitive Domain

Training in immersive VR for procedural and psychomotor “hard” skills was among the first and more frequent applications for experiential, simulation-based learning [38,48]. The operation of complicated equipment and vehicles (e.g., military airplanes) as well as critical health- and safety-related operations were the perfect testbeds for preparing students and practitioners to avoid costly mistakes with grave consequences in their fields [49]. This is achieved through repetitive, deliberate practice and mental rehearsal. Recent studies have shown that the training of surgeons in VR increased the speed of learning and practice performance in authentic operation tasks [50,51]. Immersive learning allows for the generation of affective and motivational outcomes through narrative and storyliving in education [52]. Moreover, perspective taking allows for experiential empathy building by seeing and experiencing the world through a different person’s eyes [53]. Affective gains include curiosity, interest, enjoyment, satisfaction, self-efficacy, intrinsic motivation, creativity, and career orientation [54]. Emotions deeply influence cognitive processes in learning [55]. As a result, positive emotions facilitate cognitive engagement and academic achievement [56] when immersive learning experiences are planned with activity-centered, challenge-based approaches [57]. Especially in STEM subjects, embodied cognition activated in immersive experiences with hands-on practice improve cognitive and metacognitive competencies [58,59].

Meta-analytic quantitative evidence of immersive learning in AR shows beneficial results in all three categories of studied learning outcomes: learner responses, knowledge/skills and performance [60]. Larger effect sizes were observed when AR methods were used to assist students in performing and applying what they learned in authentic contexts [60]. A meta-analysis of VR training programs emphasized the importance of the appropriateness of the chosen learning task and intervention for the technological medium [61]. Moreover, a meta-analysis has shown that technology-induced immersion has a medium-size effect on presence [8].

However, it is of paramount importance to stress that the efficacy of the assessment is linked with the design of the learning intervention, not the medium itself. It is the content that counts and eventual measurements that primarily reflect the applied design. For instance, it is therefore an oversimplification to coin VR or XR as “empathy machines”. Instead, it would be more accurate to state that immersive roleplay offers opportunities to experience different perspectives [62].

Regarding the application scale of immersive learning, VR and AR have supported learning prototypes and pilots for decades. Recent initiatives have successfully deployed tens of thousands of units in thousands of locations for millions of users in several fields. The US Airforce has deployed the Member Operations Training Analytics and Reports platform (MOTAR), a centralized learning management system (LMS), to funnel and share all immersive learning modules and content, combing VR, AR and artificial intelligence (AI) to achieve scale [63]. It can be described as a device-agnostic military “Netflix for learning” with a focus on procedural training. In the private sector, Walmart has globally organized over 2.4 million trainings to upskill more than 1.4 million employees using VR [64]. Large-scale implementations require interoperable content within the same architecture that can link with LMS for user tracking, multiple dashboards with data analytics and low/no code authoring tools for subject matter experts.

The systematic introduction of immersive technologies such as VR for education and training leads to dematerialization; costly and scarce physical materials are no longer required. Therefore, simulated activities support enhanced safety as learners operate in a safe environment. This adds the extra benefit of infinite repeatability for asynchronous, flexible practice. Another advantage is the utilization of metrics; in a synthetic environment, everything can be measured. Therefore, an assessment can be quantitatively planned, e.g., via biometric measurements. In contrast, in a physical environment, practitioners primarily rely on qualitative data collection methods such as participants’ observations, outcomes, time, self-reporting data, self-reflection, etc.

5.3. Challenges

Challenges related to immersive learning are related to three factors: technology, human physiology and effectiveness. More specifically, as immersive VR and AR have not matured from a technological standpoint, hardware needs to be frequently updated. Challenges associated with human factors include health, physiology and compatibility issues related with fatigue, cognitive load and health side effects [65]. Therefore, a priory assessment is recommended to determine if an XR application is the most effective solution. In this context, the ethical use of collected user data is paramount to ensure user adoption [66,67]. Finally, the ultimate aspect related to instructional design and evaluation is the optimum experience fidelity and design, which is used to achieve sustainable and in-depth competency transfer to the field. In this direction, there is incremental VR training, e.g., in emergency responses with noise, scents, screams, etc.

6. Conclusions and Prospects

The underlying technologies of the Metaverse, AR and VR can activate episodic memory. Episodic memory is the key towards high-quality education for lasting, deep and transformative knowledge; integrated transdisciplinary education; and competence and knowledge transfer into the workplace [68]. Immersive learning orchestrated in roleplaying, simulations, games, gamification, storytelling/storyliving, and play continuums can unearth immense creative potentials for effective prescriptive and emergent formal and informal learning in virtual worlds. Indeed, this conceptual area allows educators to organize and revolutionize learning in feasible and practical ways that are not possible in most cases in physical classrooms, which can be performed in addition to two-dimensional, web-based learning management systems. The ultimate potential of immersive learning is emotional: winning students’ hearts and inspiring their souls to inquire, discover, experiment, and carve their path in fields of their choice armed by tangible hard and intangible soft skills that are practiced in student-centered learning procedures. The role of teachers and educators in this endeavor is invaluable and requires a teaching practice paradigm shift from being a content transmitter to a learning experience designer [69]. At the macroscopic societal level, in the same way that writing and typography allowed us to educate on a massive scale by replacing context with engagement, immersive technologies in the Metaverse can bring back context and experiential learning at the same massive scale. The opportunities for learning across all domains are immense. Similar to the philosopher in Plato’s allegory of the cave, their onus will be to experience the value of immersive learning in their specific domain, return to their institutions to inform stakeholders and help them break out of the tyranny of suboptimal and ineffective education, aiming to discover the protopia of embodied immersive learning.