Neurocognitive Foundations of Memory Retention in AR and VR Cultural Heritage Experiences

Abstract

Immersive technologies such as augmented reality (AR) and virtual reality (VR) have emerged as powerful tools in cultural heritage education and preservation. Building on prior work that demonstrated the effectiveness of gamified XR applications in engaging users with heritage content and drawing on existing studies in neuroscience and cognitive psychology, this study explores how immersive experiences support multisensory integration, emotional engagement, and spatial presence—all of which contribute to the deeper encoding and recall of heritage narratives. Through a theoretical lens supported by the empirical literature, we argue that the interactive and embodied nature of AR/VR aligns with principles of cognitive load theory, dual coding theory, and affective neuroscience, supporting enhanced learning and memory consolidation. This paper aims to bridge the gap between technological innovation and cognitive understanding in cultural heritage dissemination, identifying concrete design principles for memory-driven digital heritage experiences. While promising, these approaches also raise important ethical considerations, including accessibility, cultural representation, and inclusivity—factors essential for equitable digital heritage dissemination.

1. Introduction

In an age of rapid technological transformation, cultural heritage is no longer confined to physical spaces and conventional educational tools. Immersive technologies such as augmented reality (AR) and virtual reality (VR) have opened new pathways for the interpretation, preservation, and dissemination of cultural narratives. As memory plays a central role in the educational value of heritage experiences, understanding the cognitive mechanisms behind memory formation and retention in immersive contexts becomes a crucial step toward designing impactful digital tools. Prior research has demonstrated that gamified XR environments have the potential to foster user engagement and emotional resonance, thereby creating favorable conditions for learning.

This paper builds upon the findings of [1], who explored gamified XR systems in the metaverse through the InHeritage application, highlighting increased emotional engagement and improved memory retention among young users. While the last study focused on usability, gamification, and content delivery, the current work takes a deeper dive into the neurocognitive mechanisms that may explain why such environments enhance learning outcomes. By bridging insights from cognitive neuroscience, multimedia learning theory, and empirical findings from prior user studies, this article aims to lay the groundwork for developing memory-driven digital heritage experiences rooted in scientific understanding. Therefore, this paper addresses the following research question: how do immersive AR/VR environments influence memory retention through neurocognitive mechanisms in cultural heritage education?

2. Background and Related Work

Immersive technologies have significantly reshaped educational strategies in the domain of cultural heritage. The convergence of AR/VR platforms with cognitive and affective engagement models has opened up opportunities to enhance both knowledge transfer and emotional resonance. Contemporary research highlights that beyond technological novelty, the effectiveness of XR environments lies in their capacity to support cognitive processes such as attention, perception, and memory encoding [2,3]. Notably, immersive interfaces facilitate a multisensory learning experience, aligning with the dual coding theory [4], which suggests that visual and verbal cues processed simultaneously can improve retention. Emotional engagement, which has long been linked with stronger memory consolidation [5], is especially relevant in cultural heritage contexts, where affective content plays a key role. This paper builds on earlier insights published in Applied Sciences, which explored gamified XR systems in cultural heritage education, further exploring how immersive gamified systems engage neurocognitive processes critical for learning and memory. Survey data from a study involving 50 participants aged 18–30 indicated that 46% of users reported improved memory retention in XR environments compared to traditional methods, supporting the affective encoding hypothesis. Unlike prior work that focused primarily on gamification and content usability, this study advances the field by proposing a comprehensive neurocognitive framework for XR-based memory enhancement in heritage education.

2.1. Memory and Learning in Immersive Environments

Memory is an essential cognitive function for meaningful learning, and immersive technologies offer unique affordances that influence how information is encoded and retrieved. According to the cognitive load theory [3], learning environments that reduce extraneous loads while enhancing germane cognitive processing foster better comprehension and retention. AR and VR, when appropriately designed, can help by simulating real-world contexts and presenting spatially distributed information that enhances cognitive mapping. Research by [6] shows that immersive learning environments can activate additional cognitive pathways, particularly when users experience a sense of spatial presence. Complementing this, studies in affective neuroscience suggest that emotionally charged content can modulate memory encoding through amygdala–hippocampus interactions [5]. The InHeritage project demonstrated that multisensory, interactive content fosters both emotional engagement and durable knowledge traces [1]. This combination of immersion and emotion supports a learning model where presence, interactivity, and affect function together to deepen memory consolidation in digital heritage education.

From Gamification to Embodied Cognition

Gamification, as an educational strategy, leverages motivation by integrating elements such as rewards, goals, and narrative progression into digital environments. Theoretical models such as the self-determination theory [7] emphasize autonomy, competence, and relatedness as key factors in sustained engagement, all of which can be facilitated through gamified design. In immersive systems, these principles are further augmented by embodied cognition—the idea that cognitive processes are grounded in bodily interaction with the environment [8]. XR environments provide fertile ground for such learning, as users engage through movement, gazes, and the manipulation of digital objects. In research [1], users indicated a preference for experiences that allowed them to actively explore content rather than passively consume it. Additionally, 76% of participants reported an emotional connection with the XR heritage content, highlighting the interplay between bodily immersion and affective engagement. This supports the notion that gamified XR systems, when designed with cognitive and emotional factors in mind, can promote deeper learning and memory retention through embodied interactions [9].

3. Cognitive and Neuroscientific Foundations

Immersive technologies interact deeply with human cognition, particularly through how users perceive, process, and retain information. While the visual and interactive affordances of AR and VR can significantly enrich educational content, their effectiveness largely depends on the underlying neurocognitive dynamics. This section examines three core theoretical frameworks—cognitive load theory, dual coding theory, and affective neuroscience—to understand how immersive experiences shape learning and memory. Recent findings from the InHeritage project, published with the MDPI [1], demonstrated measurable effects on users’ perceived memory performance and emotional engagement when interacting with gamified XR cultural content. These findings reflect broader insights in the literature, where scholars emphasize the importance of balancing sensory richness with cognitive clarity [2,3]. By exploring the intersection of cognition, emotion, and spatial perception, this section provides a foundation for designing AR/VR systems that enhance memory retention and educational impact in cultural heritage contexts.

3.1. Cognitive Load and Dual Coding in AR/VR

Cognitive load theory [3] posits that learning environments must carefully manage the mental effort required to process information. In AR and VR settings, learners are often exposed to high volumes of simultaneous stimuli—visual, auditory, and spatial—which can either facilitate or hinder memory formation. Properly designed XR experiences reduce extraneous load and promote germane load, which directly contributes to learning. Dual coding theory [4] further supports the notion that learners benefit from presenting information in both verbal and visual formats. In immersive environments, this dual presentation occurs naturally: spoken narration combined with 3D representations, spatial cues, and haptic feedback can significantly improve encoding. The InHeritage application, for instance, uses both textual and visual stimuli layered within a spatial environment, supporting users in constructing robust memory traces [1]. However, without a clear pedagogical structure, such richness may overwhelm cognitive resources, as cautioned by [2]. Thus, aligning XR content with cognitive load and dual coding principles is essential for effective memory retention.

Emotional Arousal and Memory Encoding

Emotion plays a critical role in memory formation, particularly in immersive learning. Affective neuroscience research demonstrates that emotional arousal can enhance memory consolidation by activating brain structures such as the amygdala and hippocampus [5]. In XR environments, this emotional activation is often elicited through narratives, interactivity, and presence, creating meaningful personal connections with the content. The InHeritage study reported that 76% of participants felt emotionally engaged while interacting with AR/VR cultural heritage content. This emotional engagement likely contributed to the reported improvement in memory recall by 46% of users, aligning with the affective encoding hypothesis. Unlike traditional learning methods, XR allows for immersive storytelling and first-person exploration, which heighten emotional involvement. Parong and Mayer [6] emphasize that combining affective stimuli with cognitive learning tasks strengthens long-term memory. Champion [10] emphasizes that narrative structures in virtual heritage systems can serve as a scaffold for memory, where interactivity and storytelling reinforce user retention and engagement. Therefore, designing heritage experiences that intentionally provoke emotional responses—through music, narrative tension, or user agency—can significantly enhance learning outcomes. Storytelling plays a crucial role in immersive memory retention, especially when narratives evoke personal relevance or cultural familiarity. Research in heritage studies shows that narrative-driven XR applications can anchor memory through emotionally resonant sequences and symbolic spatial landmarks [10]. By guiding users through a coherent story within a virtual setting, XR systems can simulate episodic memory encoding, where users recall content not just as data but as experienced events.

3.2. Spatial Presence and Mental Reconstruction

Spatial presence—the sensation of “being there” in a virtual environment—is a defining feature of AR/VR and has significant implications for memory. Research indicates that spatial immersion enhances users’ ability to mentally reconstruct environments and associate learned content with virtual locations [11]. This form of context-dependent memory can be especially powerful in cultural heritage, where historical or architectural information is closely tied to spatial cues. In the InHeritage system, users navigated reconstructed historical sites, where spatial layouts and object placements supported contextual learning. Spatial presence facilitates the creation of mental maps, which help encode information into episodic memory structures. When users later recall information, these spatial anchors serve as retrieval cues, aiding in the reconstruction of learned material. Studies by Radianti et al. (2020) [12] and Nguyen & Schneider [13] suggest that well-designed spatial environments in XR not only increase engagement but also deepen comprehension and recall. Thus, spatial structuring in AR/VR is not merely esthetic—it is a cognitive infrastructure for memory.

4. The Role of Immersion in Cultural Heritage Education

Immersion is a cornerstone of effective XR-based learning, particularly within the context of cultural heritage. It enables learners to transcend traditional passive observation and instead engage with heritage content in an active, sensorimotor, and emotionally resonant manner. Immersive technologies offer affordances for “learning by doing”—a key component of embodied cognition—which facilitates the formation of durable memory traces and conceptual understandings [2,8]. In educational heritage contexts, this means allowing users to explore virtual museums, reconstructed historical sites, or even interact with cultural artifacts through their motion, voice, or gaze. Prior studies, including the InHeritage application [1], demonstrate that increased interactivity and immersion lead to stronger user engagement and improved memory outcomes. These findings align with research in cognitive psychology and neuroscience suggesting that a physical interaction with learning content activates broader neural circuits, promoting deeper processing and more robust retention [5,6]. The following sections analyze how sensorimotor interactions and personalization strategies contribute to these effects.

4.1. Sensorimotor Engagement

Sensorimotor engagement refers to the bodily interactions users perform while navigating XR environments—such as moving through space, pointing, grabbing, or manipulating virtual elements. These actions stimulate motor-related brain regions, which in turn enhances cognitive processing and memory encoding [8]. In the context of the InHeritage application, users reported higher engagement when tasks involved physical interactions with virtual heritage elements (e.g., unlocking site-specific information by exploring reconstructed spaces), rather than passively viewing them. This reflects findings from Mayer [2], who argues that embodied activities reduce the cognitive distance between the learner and content, promoting a deeper understanding. Moreover, studies have shown that a sensorimotor interaction improves spatial memory and task recall due to the integration of kinaesthetic cues during learning [11]. When XR environments incorporate intuitive, bodily interactions—such as touch, gazes, or motion—they support embodied learning pathways that facilitate long-term memory retention, especially when aligned with educational goals and scaffolded feedback.

Personalization and Memory Traces

Personalization in immersive environments refers to the system’s ability to adapt content based on user behavior, preferences, and cognitive responses. This approach not only increases engagement but also enhances memory by tailoring stimuli to individual learning pathways. According to cognitive psychology, personalized content is more likely to be encoded into long-term memory because it aligns with pre-existing cognitive schemas [3,4]. In the survey conducted as part of the InHeritage study, participants emphasized a desire for personalized exploration routes, emotionally resonant narratives, and an adaptive difficulty. Such input supports the development of dynamic XR systems that respond to users in real time, optimizing memory retention through relevance and emotional salience. The combination of personalization and immersion creates a learning environment in which users build unique, contextually grounded memory traces—strengthened by spatial, emotional, and semantic dimensions. This aligns with recent work in affective computing and educational XR, which advocates for adaptive storytelling and interaction design as key to meaningful cultural heritage experiences [13,14]. In the context of museums, Mitchell et al. [15] demonstrate how personalized virtual learning environments can enhance user engagement and memory through customizable pathways and interactive feedback loops.

5. Case Reference: InHeritage Application (MDPI, 2024)

The InHeritage application, developed as part of a research initiative at the University of Zagreb and published in Applied Sciences [1], serves as a practical case study in exploring how immersive technologies support cultural heritage education. The project introduced a gamified XR mobile platform that combined AR and VR elements to deliver site-specific cultural content within a metaverse-like environment. The design aimed to enhance user motivation, emotional engagement, and memory retention through narrative interactivity, spatial navigation, and multisensory feedback. A user study involving 50 participants aged 18–30 gathered quantitative and qualitative data on perceived engagement, memory effects, and user preferences. The findings revealed positive correlations between interactivity, emotional involvement, and perceived memory retention. This section interprets those results through the lens of cognitive load theory, dual coding theory, and affective neuroscience, framing InHeritage as a model for neurocognitively informed design in cultural heritage dissemination. The InHeritage application featured site-specific cultural content drawn from Croatian historical and architectural heritage—such as virtual reconstructions of Roman ruins and medieval fortresses. The choice of participants aged 18–30 was intentional, reflecting the app’s target demographic for early adoption and feedback due to their digital literacy and familiarity with gamified environments. Conclusions were drawn based on a mixed-method evaluation: quantitative survey results (n = 50) and qualitative interviews, analyzed using thematic coding to assess emotional engagement and recall. The application was deployed in controlled settings during digital heritage workshops hosted in Split, Croatia.

6. Design Implications for Memory-Driven AR/VR Heritage Experiences

The integration of neurocognitive theory into the design of AR/VR cultural heritage experiences offers practical pathways for improving learning outcomes, as shown in Figure 1. As demonstrated through the InHeritage case and aligned with foundational theories in multimedia learning, memory encoding is most effective when experiences are emotionally engaging, cognitively structured, and sensorimotor-driven. Immersive heritage applications that support personalization, spatial navigation, and embodied interaction show an increased potential for producing durable memory traces. This section consolidates theoretical and empirical insights into actionable design strategies. Drawing from Mayer’s cognitive theory of multimedia learning [2], Sweller’s cognitive load theory [3], Paivio’s dual coding theory [4], and affective neuroscience research [5], we present design considerations aimed at balancing engagement and retention in XR environments. These implications are not only relevant for educators and developers but also for institutions aiming to preserve and disseminate cultural heritage in the digital age.

6.1. Key Takeaways from Cognitive Theories

As shown in Figure 2, Figure 3 and Figure 4 and theoretical frameworks provide essential guidance for XR heritage design. First, cognitive load theory advises minimizing extraneous cognitive processing by avoiding overly complex visual or auditory input. Designers should structure content into manageable segments, using scaffolding and layered navigation. Second, dual coding theory emphasizes pairing verbal explanations with spatialized visual representations—ideally synchronized—to aid in long-term retention. In the InHeritage system, this was reflected in overlays of textual labels onto 3D models and site reconstructions [1]. Third, affective neuroscience reveals that emotional salience enhances encoding; thus, incorporating music, ambient sounds, or narrative tension can support stronger memory formation. Lastly, embodied cognition supports the integration of sensorimotor tasks—such as spatial exploration or the manipulation of objects—which engage broader cortical areas. These theories converge on a critical point: memory-driven design is not about overwhelming users with features but about guiding them through emotionally rich, cognitively coherent, and physically interactive experiences [16]. Recent case studies in digital art also confirm these design principles; Gaia and Boiano [17] report how AR-based storytelling and interactive installations foster deeper cognitive and emotional immersion in cultural narratives. These design principles have been effectively applied in existing XR heritage projects. For instance, the “ReBlink” AR installation at the Art Gallery of Ontario allows visitors to view classical paintings through their smartphones, revealing layered contemporary reinterpretations. Similarly, the “The Kremer Collection VR Museum” offers fully immersive tours of Dutch Golden Age paintings, combining spatial navigation with expert narration. Both examples integrate spatial cues, emotional storytelling, and interactivity—demonstrating how theoretical frameworks translate into practice.

Recommendations for Future Design

Based on both theoretical models and empirical findings from the InHeritage project, several design recommendations can be proposed. First, use emotional “hooks” early in the experience—such as a personal story, historical anecdote, or immersive narrative opener—to establish affective engagement. Second, minimize cognitive noise by reducing unnecessary UI elements, background distractions, or unrelated animations that may interfere with learning. Third, embed spatial coordinates into content delivery; for example, allow users to unlock information by reaching specific virtual locations or interacting with heritage artifacts. This supports the spatial anchoring of memory. Fourth, ensure bidirectional interaction—offer users opportunities to make choices, respond to prompts, and influence the progression of the experience, reinforcing autonomy and competence [7]. Finally, include adaptive personalization mechanisms: allow users to control the pacing, difficulty, and focus areas. These approaches align with the cognitive principles discussed and lay the foundation for memory-optimized, inclusive, and engaging cultural heritage experiences in XR.

7. Discussion

While the integration of immersive technologies into cultural heritage education has shown promising results, particularly when guided by cognitive and neuroscientific principles, it is important to contextualize these findings within broader methodological and disciplinary frameworks. The InHeritage case study [1] offers initial empirical insights, but also opens critical questions regarding generalizability, equity, and scalability. This discussion examines both the limitations of the current evidence and the potential for future interdisciplinary collaboration to address these gaps. By reflecting on the empirical, technical, and ethical constraints of immersive cultural heritage systems, and by emphasizing the value of cross-sectoral dialog between designers, neuroscientists, educators, and curators, we outline a forward-looking vision for research and development in this domain.

7.1. Limitations of Current Evidence

Despite encouraging outcomes, the current evidence on the neurocognitive effects of AR/VR in cultural heritage remains limited in scope and scale. The InHeritage study, for example, relied on self-reported survey data from a relatively small and demographically narrow sample (n = 50), composed primarily of students aged 18–30. While the findings indicated positive trends in memory retention and emotional engagement, they lack biometric validation such as eye-tracking, EEG, or functional neuroimaging—which would offer deeper insights into underlying cognitive mechanisms [13]. Future research should involve collaboration with cognitive scientists to incorporate biometric validation tools such as EEG, eye-tracking, or galvanic skin responses, enabling a more precise assessment of cognitive and emotional engagement in XR environments. Additionally, data may be affected by user bias, novelty effects, and variable levels of digital literacy. Ethical concerns also emerge, particularly regarding accessibility: XR systems often require expensive hardware, potentially excluding marginalized populations. Cultural sensitivity in content design is another concern, as representations of heritage must be accurate, inclusive, and context-aware. Future research must address these gaps by incorporating more diverse samples, longitudinal methods, and objective neurocognitive measurements to substantiate current claims. Moreover, immersive and multisensory approaches hold potential for inclusivity. For example, haptic feedback and spatial audio can enhance accessibility for visually impaired users, while sign language avatars or captioned narration in XR can accommodate the hearing-impaired. Personalization also allows adaptations to users’ cognitive levels and learning styles, contributing to more equitable heritage education. Leading museums, such as the Louvre and the Smithsonian, have begun incorporating such features into their digital offerings, reflecting a shift toward inclusive design in cultural interpretation. This trend aligns with broader developments in digital heritage, where leading institutions explore online and virtual strategies to expand access and reinterpret historical narratives [18].

Opportunities for Interdisciplinary Convergence

The complex nature of memory, perception, and cultural representation in XR environments necessitates interdisciplinary collaboration. Neuroscience provides insights into the brain’s response to emotions, multisensory inputs, and spatial processing [5], while design research offers frameworks for structuring interactions and narrative flows [2]. Pedagogical theory contributes models of learning motivation, feedback, and engagement [7], and cultural studies add critical perspectives on representation, authenticity, and user contexts. The InHeritage project exemplifies the potential of such a convergence by aligning gamified content with cognitive theory and cultural relevance. Moving forward, partnerships between cognitive scientists, XR designers, museum curators, and educators can produce more robust, inclusive, and evidence-based heritage applications. Experimental designs combining qualitative and quantitative methods—including biometric tools—will be essential for validating impact. Ultimately, the field can benefit from a holistic research agenda that merges empirical rigor with creative innovation to shape the future of cultural learning in immersive spaces.

8. Conclusions

This paper has explored how immersive technologies such as augmented and virtual reality can support memory retention in the context of cultural heritage education. Drawing on established cognitive theories—including cognitive load theory [3], dual coding theory [4], and principles from affective neuroscience [5]—we have argued that immersive environments enhance learning when they balance sensory input, emotional engagement, and structured interactivity. Through the InHeritage case study [1], we demonstrated how gamified XR systems, when properly designed, can lead to increased user motivation, deeper emotional connection, and the improved recall of the heritage content. These findings highlight the need for interdisciplinary approaches that merge neuroscience, education, design, and cultural studies to inform the development of memory-driven digital heritage experiences. While current evidence remains preliminary, the integration of cognitive science into the design process offers a promising pathway for making cultural knowledge more accessible, engaging, and memorable. Future XR heritage applications should continue to embrace personalization, sensorimotor interactions, and emotional storytelling as key design strategies for fostering long-term cognitive and cultural impacts.