Designing the Space Archivists: A Metadata-Driven VR Game Concept for Children to Engage with Cultural Heritage

Abstract

Motivated to create a children’s VR game for the Netherlands Institute for Sound and Vision (NISV), this research asks how might we design an immersive game for children to meaningfully interact with media and metadata in cultural heritage contexts? First, during a ‘design salon,’ 13 data and heritage experts challenged children’s ability to interact with metadata. In response, we ran workshops with 19 children focused on understanding abstract media and data. We found that while (1) metadata has many challenges, (2) children understand abstract data when it is grounded in concrete experiences, are (3) motivated to interact with archival media through in immersive and collaborative contexts, and (4) are interested in exploring media diversity through categorisation games with high-level narrative goals. These findings inform our game concept and three core insights for designing immersive experiences for cultural heritage: Considering the Contextual Complexity of Data and Audience Needs, Connecting Data Abstractions to Embodied Narratives Through Categorisation Mechanics, and Supporting Abstract Meaning Making Using the Immersive Affordances of VR.

1. Introduction

Due to growing digitisation efforts, museums are exploring new ways of presenting media collections to children and diverse audiences that exploit data affordances and create new engagement opportunities. Digital museum twins [1,2], heritage site reconstructions [3,4], and recreations of individual artifacts [5,6] have become popular as exhibition mediums. These formats overcome existing cultural barriers by supporting remote or mobile-impaired visitors to virtually experience exhibitions [7], interact with fragile artifacts [8], and browse extensive collections [9]. Composed of multimedia (photos, videos, audio, text) and accompanying metadata (descriptive information about the context of a heritage object such as its creator, date, or location), digital collections have ‘big data’ [10] advantages; created on large scales [9], they are often connected across institutions using shared vocabularies [11]. This has led to new ‘generous interfaces’, enabling users to browse content using serendipitous exploration [12] rather than search, and interactive trend visualisations, which use temporal, geographic, or topic dimensions [13]. Notably, these approaches are primarily (1) browser-based and have not been explored within the context of VR-based heritage experiences, and (2) focus on professionals and researchers rather than lay audiences. We see this as a missed opportunity as immersive visualisations can create a sense of intimacy and immediacy [10], provoking emotional responses and making complex topics more approachable, particularly for young audiences, who learn well through embodied experiences [14].

We describe research informing the concept design of The Space Archivists, a VR metadata-driven game for children to engage with archival media and metadata at the Netherlands Institute for Sound and Vision (NISV) in the Netherlands. We focus on children aged 8–14, as these are the target audience of the museum. We situate our work in cultural heritage requirements and child Participatory Design (PD) practices and focus on VR as a framework for our work based on project proposal requirements. We describe three studies with cultural heritage professionals and children, and four emergent themes of the research that informed the game design. Finally, we present our game concept and three core lessons for designing metadata-driven immersive experiences for cultural heritage: (1) Considering the Contextual Complexity of Data and Audience Needs, (2) Connecting Data Abstractions to Embodied Narratives Through Categorisation Mechanics, and (3) Supporting Abstract Meaning Making Using the Immersive Affordances of VR.

2. Related Work

2.1. Designing Immersive Experiences for Cultural Heritage

We draw inspiration from large-scale digital heritage collections employing new data-driven interaction and storytelling opportunities. For instance, the Collection Space Navigator [5] and The Sensory Moving Image Archive [15] illustrate how heritage organisations can showcase a vast collection, allowing users to visually explore, filter, and zoom in on interesting data, such as examples that challenge hierarchical search-centred exploration [16]. Herbaria Heritage [17] additionally demonstrates how visualisations can help users understand colonial provenance, biases, and gaps in collection practices. Such interfaces contextualise individual experiences in a macroscopic view [18], allowing users to interactively engage with the composition, historical, and archival practices of a collection. However, these approaches are primarily browser-based, and focus on professionals. In a comprehensive review of XR technology for cultural heritage, Innocente (2023) et al. [19] suggest that data visualisations in XR can support learning about cultural heritage; however, little work has been carried out in this domain. Thus, there is a gap and an opportunity to create data-driven VR-based experiences that make archival topics more accessible to lay audiences, including children, for whom this domain is largely unexplored.

2.2. Children Engaging with VR

The focus of our research was children aged 8–14, as they are the target audience of our museum. The literature suggests that children are more interested in XR technologies than older populations [19]; however, limited work has been carried out with children in immersive domains. Research has considered VR as a tool in spatial learning experiences, such as for supporting autistic children developing awareness of social skills [20], preventing darkness phobias [21], and supporting empathy development in young children [22]. VR has likewise been used to help rehabilitate children with physical and visual disabilities (e.g., deaf children) by integrating perceptual learning with sensory and visual signals [23]. In cultural heritage, preliminary research has likewise considered how VR can support spatial learning. The Florida Natural History Museum developed an interactive VR game to showcase 3D objects in context, such as letting children swim with endangered underwater animals in the ocean [24], and Chu et al. (2021) [25] used VR to help Chinese children learn traditional culture and art. Overall, VR has focused on supporting child spatial learning goals, such as improving visual knowledge of historic buildings and their architecture [19], but it has not yet considered how children may engage with cultural heritage data.

2.3. Children Designing with Data in Cultural Heritage

In this research, we engaged children in PD practices to better understand how they conceptualise, interact, and tell stories with archival metadata. Most of the children who participated were between eight and twelve, an age when they begin to consider multiple perspectives and engage in flexible and logical thinking, but may lack the more sophisticated metacognition and abstract reasoning of their older peers [26,27]. However, with appropriate scaffolding (e.g., multiple representations [28,29,30], physical models [31], embodied experiences [14], ‘concreteness fading’ [32]) and opportunities to draw on personal experiences and concrete contextual information [14,29,31,33,34,35], children as young as six have been shown to identify patterns and generate abstract data visualisations [31]. Although less is known about how children engage with cultural heritage metadata, the logic governing children’s online searches can differ significantly from that of adults [36,37,38,39,40,41,42]. For example, children may pay attention to the physical attributes of items like size, colour, and shape [43,44], as well as their aesthetic relevance [45], by weighing subjective factors rarely captured through metadata, like ‘coolness’ and personal connection [39]. Moreover, children may express interest in open exploration and self-directed search in a manner that is not always addressed by child-facing search interfaces, suggesting the appeal of immersive sandbox environments and playful narrative frames [37,46].

In light of these differences, scholars recommend directly involving children in the design of immersive museum experiences [47,48,49,50,51,52,53,54,55]. Successful strategies employ crafting materials for prototyping [56,57] and museum artifacts and storytelling to support brainstorming [50]. Nevertheless, planning, facilitating activities, and extracting design insights from collaborations can be challenging due to children’s typically shorter attention spans [58], developing language skills [48,59], and variations in relevant content knowledge and experience with museums, design, and technology [48,49,59,60]. By working directly with child co-designers and our museum audience, we further address the gap in making archival topics more accessible for children, our primary demographic.

2.4. Research Question and Design Challenge

While existing digital collections have the potential to support the design of immersive interfaces, there is a gap in the literature about how to design engaging experiences for children in cultural heritage institution (CHI) contexts. In particular, the literature suggests that child museum visitors (1) are more comfortable interacting with individually curated artifacts over large volumes of digital media and metadata and may (2) have trouble understanding abstraction-based experiences that make connections between objects or trends [61]. Through three workshops with cultural heritage professionals and with children, we thus ask the following: how may we design a VR experience for children that allows them to meaningfully interact with cultural heritage media and metadata? We complement this research question with a design challenge to create such a game using our insights.

3. Method

The goal of our user-centred process was to inform the design of an immersive metadata-driven experience for children1. We situate this process in the Living Lab methodology [64,65], a goal of which is to involve all relevant stakeholders in the design process, which included adult data professionals and child audiences. First, we conducted a participatory ‘design salon’ with 13 professionals from cultural heritage institutions (CHIs) to obtain design recommendations; however, our participants expressed doubts regarding children’s ability to comprehend and interact with metadata. In response, we conducted (1) child PD workshops to understand how children engage with high-level data concepts (N = 9) and (2) design workshops with NISV child visitors to understand how they interpret media (N = 10). For all sessions, participant consent was collected prior to the research2, which included audio recording and photography. One researcher led the study, and up to three researchers took photos, videos, and observational notes.

3.1. Design Salon with Data Professionals

We conducted a design salon with 13 CHI professionals to understand how they would design metadata-driven immersive experiences for children. Participants had, on average, 10 years of experience in the cultural heritage, data, and storytelling sectors and included historians, researchers, data engineers, and training and development specialists.

During the three-hour salon, participants described their experiences with data-driven storytelling, and, in teams, created a low-fidelity immersive and interactive data-driven VR museum experience for children. Using a curated set of media and metadata about the Olympic Games, they envisioned (1) a narrative, (2) game mechanics, and (3) visual representations of metadata. Participants were first shown sample game mechanics and metadata visualisations and were asked to elaborate on them during a brainstorm, considering “what kinds of metadata patterns might you find in a dataset about the Olympic games?” and “what kind of stories could you tell with the individual narratives and metadata?” Participants then used low-fidelity materials to prototype their experience and present their designs (Figure 1).

3.2. Child Participatory Design Workshops on Data

To understand how children design with data, two (2) Cooperative Inquiry (CI) sessions were held with 9 child designers, aged 7–12, from Kidsteam, a kids research team [66], at the University of Maryland, who work weekly with adults as equitable design partners [57,66]. The participants included 5 girls and 4 boys. The sessions investigated (1) how children conceptualise data and (2) and how they interpret and interact with media data abstractions through patterns and stories.

Each CI session began with a discussion of the design topic and goals, after which small teams of 2–3 children and 1–2 adults began designing together. Then, the groups presented their ideas to the entire design team, while an adult partner performed a rapid thematic analysis and summarised ‘Big Ideas’ on a whiteboard [57]. After each session, adult partners discussed and refined the themes that arose. The first session set children up to think about data concepts; they interpreted abstract data patterns (from a collection of simple data visualisations [67]), then visualised their own behaviour as patterns (e.g., how often they wash their hands per day). Next, children were given a set of small physical artifacts (e.g., small dolls, a box, souvenirs) and clustered them by object characteristics or features (e.g., shape, size, or domain). The second session built on the first. First, children interpreted abstract data during group discussion. Then, using the ‘Bags of Stuff’ low-fidelity prototyping method [60], they designed and visually represented abstract 3D data experiences (Figure 2).

3.3. Child Participatory Design Workshops on Media

To investigate what media engages the target demographic (children aged 8–14), we conducted three sessions with 10 children who fit this demographic at NISV. The participants included 8 girls and 2 boys. The first two sessions (N = 7) explored archival topics and related stories and games that Dutch children would find interesting. As a follow-up, the third session (N = 3) explored how children organise museum media into patterns and categories.

In the first two sessions, children ordered images with media archive topics (e.g., nature, art, sports) from most to least interesting, explained what made the topics fun, and said what they would like to learn more about. After being introduced to VR, participants envisioned and described fun VR game scenarios with media archive topics and considered with whom they would want to complete the experience (e.g., alone, with a parent or friend). In the third session, children focused on categorising media. First, they were introduced to the concept of a museum archive and VR and were shown 20 images from the Europeana Sport dataset [68]. Then, children categorised images based on elements they found important. Finally, participants imagined narratives and categorisation games using metadata elements.

3.4. Data Analysis

Session data included notes, photographed design artifacts, audio-recorded debriefs, and video recordings of presentations. Data was transcribed and compiled into a manuscript, and a thematic analysis was performed using the 6-step process described by Braun and Clarke [69,70]. The lead author (primary researcher) is a VR expert working at NISV and used her background to inform the coding process, illustrated in Figure 3. First she read through the manuscript, generated initial open codes, and iterated on the codes by re-reading the manuscript several times. Next, she examined how the open codes fit into sub-themes and used these to generate design insights that informed the concept design (see Appendix B). Finally, over several discussions with a second coder (another researcher on the project), she clarified the relationships between themes and the design challenge, and generated high-level themes. Both the individual insights and themes influenced the concept design; the insights informed a set of game design requirements, while generating and reflecting on the themes helped reframe the high-level goals of the game.

4. Results

This section describes four emergent themes that informed The Space Archivists game design and our design insights. While theme 1 exposed several expert-described challenges of working with metadata and integrating it into a children’s game, theme 2 revealed that children can, in fact, understand abstract data when it is ground in concrete experiences and enjoy using it to create embodied narratives. Further, theme 3 showed that children were motivated to interact with archival media in immersive and collaborative contexts, and theme 4 illustrated that both adults and children were interested in categorisation games that served a larger narrative goal.

4.1. Theme 1: Expert-Described Metadata Challenges and Value for Storytelling

The design salon uncovered many challenges of working with metadata and integrating it into storytelling experiences for children. Professionals explained that metadata fractures, systemic changes in the way metadata is added to collections, created datasets that were “never a complete picture, [so] you cannot draw conclusions based on the archive as a whole” (P3, Design Salon). Furthermore, datasets often had “implicit meanings” understood only by researchers who “gathered and made [the] data” (P4, Design Salon). Moreover, data stories, visual narratives described through data patterns [71], were a key inspiration for our concept, but through the design salon, we found out that “journalistic examples [have] a ton of work behind them” and that they “need a lot of curation” (P8, Design Salon).

While professionals found metadata valuable for work, several participants (P1, P3, P4, P5, P12) did not think it should be included in a game. “It’s called metadata for a reason, right?” a participant explained; “it only makes sense as part of the material” (P1, Design Salon). Participants also did not think children would want to look at data visualisations: “There’s a story behind the data …[but] …no story in a data point” (P4, Design Salon). Instead, they reasoned that emotional connection should be at the forefront of a game experience; “you need to relate to it,” P7 and P13 explained.

4.2. Theme 2: Child Interpretations of Abstract Data

Data professionals doubted children’s ability to understand and interact with data; however, the design workshops revealed that children both understood data framed in concrete experiences and could create exploratory narratives from data abstractions. We found that children understood data abstractions when they were supported by “something to step into …something concrete” (F13, Kidsteam Session 2) and that recognising relationships between abstract and concrete concepts was key to understanding. For instance, while looking at a visualisation of drinks represented as bubbles, a visualisation key helped children realise they were “the same exact thing [was represented] in a different way” (F2, Kidsteam Session 2). This reflects Kolb’s model of experiential learning [27], in which knowledge is created first through a concrete experience, followed by reflection and conceptualisation of abstraction [72]. Notably, creating these scaffolds was challenging, so the “abstract representation has to be really simple” (F2, Kidsteam Session 2).

Relatedly, we observed that children could create embodied and exploratory narratives that zoomed in and out of abstract representations. One Kidsteam group bridged the gap between data and the play experience by creating a game that connected the trajectory of basketball player shots to their photos (F4, Kidsteam Session 2). Similarly, another created an abstract-to-concrete exploratory experience about US presidents: you “can climb up the graph …you can experience the president’s life …but you could also go in the blink caves …it represents boredom …then inside [is a] diary where you get to read [why they were] bored” (F1, Kidsteam Session 2).

4.3. Theme 3: Immersive and Collaborative Learning Contexts

We found that children were motivated by interactive and immersive learning and by collaborative contexts. NISV children found non-interactive experiences boring “because there’s no feedback” (P2, NISV Session 3) and wanted to “[show] you’ve understood [information by] interacting” (P1, NISV Session 3). They also wanted to learn through immersive experiences, such as walking through a painting where “you can really see …that beautiful lake …and hear about …how [the painting’s] made” (P1, P2, NISV Session 1). The Kidsteam designs likewise incorporated embodied interactions and immersive audiovisual elements, such as in the aforementioned data-driven basketball experience.

Children were also motivated by collaboration. An NISV child liked categorising media into groups because “we can talk to each other …because you [may] see something I don’t see” (P1, NISV Session 3), which is in line with cognitive developmental research, suggesting that collaborative discussion enables children to bring intuitive ideas to consciousness awareness [72]. Similarly, NISV children imagined complementary play experiences. One group imagined a game where “you have to split up …and then …discuss [a question] together in the end” (F1, NISV Session 3). Similarly, another envisioned a mystery puzzle game with separate journeys; at the end, “each of you has collected some kind of information, and together you solve that master mystery,” P1 (NISV Session 3) explained.

4.4. Theme 4: Exploring Media Diversity Through Categorisation Games

We found that children were interested in exploring media diversity through playful categorisation games. For instance, NISV children were interested in learning about the diversity of plants and animals, sports, and music and art (P1, P2, NISV Session 2). Remarkably, both children and professionals proposed categorisation as a game play mechanic. One Kidsteam participant suggested “a mini game. How fast can you categorise all these objects?” (F1, Kidsteam Session 1). Professionals proposed more sophisticated mechanics, allowing children to “contribute …to the richness of [a] dataset by adding …tags” using the mechanics of games like Where’s Wally? (P2, Design Salon). Notably, both adults and children envisioned categorisation insights furthering a high-level game mechanic or narrative (F1, Kidsteam Session 1), such as “foraging in a forest to find …herbs and stuff …[to build] a healing potion” (P11, Design Salon) or “a sorting game …that was …a gateway to other objects in the collection” (F1, Kidsteam Session 1). Insights could likewise contribute to a larger narrative, such as a scavenger hunt to solve a mystery (F1, Kidsteam Session 1) or a “treasure hunt …[where] you come across …stories that [then] play out together” (P3, NISV Session 3).

5. The Space Archivists: Helping Children Connect to Historical and Contextual Details

Complemented by the literature from experiential learning theory [14,26,27,28,29,30,31,32]), our findings were synthesised into a set of design insights (Appendix B), which informed the concept design of The Space Archivists (graphical abstract), a VR game for children to engage with archival media and metadata at NISV. The game combines elements of storytelling, teamwork, and metadata exploration in a social VR environment; each chamber is a categorisation game where two players organise media by associated metadata (e.g., image time period). Players are told a story about a damaged futuristic archive of human memory and are invited to fix the archive by categorising media, learning about the historical context of media items in the process, such as the biased historical ‘gaze’ of collections [73,74]. A detailed description of the narrative and game mechanics are presented in Appendix A.

While designing child experiences with metadata is complex (theme 1), the game scaffolds the sense-making process (theme 2) by prompting players to attend to concrete details in media and metadata, and through categorisation judgments (theme 4), build an internal framework of meaningful relationships and patterns [75]. Players see identical media in each chamber, iteratively looking for nuanced details as they make observations. This echoes dynamic learning models where individuals construct meaning in tandem with a learning environment, tools, and fellow learners [14,29,33,34]. Further, the collaborative game allows for shared interpretation and negotiation of meaning [72], supporting collaborative sense-making (theme 3) in an embodied narrative to save the archive (theme 4). This reflects museum visitors’ desire to connect to emotional and sensory experiences [76], with the chambers serving as intermediate goals in restoring the archive (theme 4) and building anticipation [72] for a final 3D experience. In playing, children thus create meaningful connections to historical events and shared cultural memories. Each time players win the game, they get access to different 3D ‘holodisc’ simulations, which creates a sense of ownership and motivation [72] and affords replayability.

6. Discussion

Asking how might we design a VR game for children to interact with metadata in CHI contexts, we describe research informing the concept design of The Space Archivists, a VR game for children to engage with archival media at NISV, a Dutch media museum. We present our design insights for cultural heritage institutions designing such immersive experiences: (1) considering the contextual complexity of data and audience needs, (2) connecting data abstractions to embodied narratives through categorisation mechanics, and (3) supporting abstract meaning making using the immersive affordances of VR.

6.1. Limitations

This work was limited by the scope of the research conducted with children. A total of 19 children participated in the study, who were recruited from the Kidsteam cooperative research group and who met the 8–14 age demographic at NISV. The Kidsteam group included culturally diverse children ages 8 to 12 from a middle-class suburb in North America, while the NISV children were primarily Caucasian (Dutch) children from upper-class neighbourhoods in the Netherlands. In both groups, the children came from relatively privileged upbringings and may not be representative of other children’s ability to interact with data. Further, there was a majority of girls in both groups, which may have likewise impacted our results.

Future work could enhance the generalisability of our findings by recruiting from a broader demographic that includes more boys and more culturally diverse children. Further, we had limited time to work with the groups, and the children at NISV did not have any design background. Future work may expand on our research by recruiting children with more experience in design research and conducting more iterative design sessions.

6.2. Design Insights

6.2.1. Considering the Contextual Complexity of Data and Audience Needs

We began with a ‘wicked’ [77] design question with several diverse factors (metadata, VR, child audiences), and our research shed light on their contextual complexity. While the design salon showcased the bias of working with metadata (theme 1), the child workshops showed that children understand data abstractions when they are scaffolded through concrete experiences (theme 2). These findings suggest that CHIs creating immersive experiences must consider both the context of their data (where it comes from, what it can say) and their audience (how they perceive information, what they will understand [26,27]).

6.2.2. Connecting Data Abstractions to Embodied Narratives Through Categorisation Mechanics

Both adults and children wanted to connect data to narrative experiences; professionals stressed that emotional connection should be at the forefront of a game (theme 1), whereas children created embodied narratives from abstract data (theme 2). In particular, both professionals and children considered data categorisation mechanics to further storytelling (theme 3 and 4), such as sorting media to build “a healing potion” (P11, Design Salon), open “a gateway to other objects in the collection” (F1, Kidsteam Session 1), or solve a mystery (P3, NISV Session 3). This suggests categorisation may be an accessible mechanism to connect data abstractions to embodied narratives.

6.2.3. Supporting Abstract Meaning Making Using the Immersive Affordances of VR

CHIs tend to use VR to re-create conventional narratives and interactions [1,2,3]; however, we found that the immersive affordances of VR can scaffold sense-making of more abstract experiences. As shown in theme 2, child designers wanted to “climb up [a] graph” to experience data related to a president’s life (F1, Kidsteam Session 2), and in theme 3, museum children wanted to immersively experience media, such as seeing “[a] beautiful lake” (P1, P2, NISV Session 1) in a painting. This suggests that the immersive affordances of VR can scaffold complex abstract cultural topics into meaningful experiences for children.

7. Conclusions and Future Work

This work considered how cultural heritage institutions might design a VR experience for children to meaningfully interact with media and metadata. To this end, we conducted a design salon with 13 data and heritage professionals and five workshops with 19 children focused on understanding abstract media and data. The output of this research informed The Space Archivists VR game concept and related design insights for CHIs.

While our eventual goal was to build the game, the primary focus of this paper was to understand how to help children connect to cultural heritage archival data. Our research helped us recognise that children can understand metadata concepts and that they enjoy engaging with metadata through categorisation tasks folded into high-level narratives. As shown in Appendix B, these insights helped us understand the different roles that metadata can have in a game experience (theme 1) and that it needs to be concretely represented to be meaningful to children (theme 2). Further, we found that collaborative rather than solo experiences motivate children to categorise metadata (theme 3) and learned that children categorise using different criteria than adults and want to use metadata to solve larger puzzles and narratives (theme 4). While these specific design insights informed our game concept, this research demonstrates the value of working with both professionals and children during the design process; had we only worked with professionals, we may have abandoned the goal of creating a metadata-driven game altogether.

Employing the Living Lab methodology [78], future work will build The Space Archivists VR game in collaboration with a VR game studio and iteratively test our concept with children at NISV. As a primary outcome of this work was creating a concept that supports meaning-making, we plan to test whether the game helps children understand how media is connected using metadata dimensions such as time. We plan to conduct several low-fidelity tests of the game concept and then finally measure engagement and learning in the VR game at the NISV museum with our target demographic.