Next Article in Journal
Semantic-Enhanced Underwater Videos Multi-Label Classification Network Based on Structural Graph Convolution
Previous Article in Journal
Presenting and Evaluating Smart2: A Bidirectional Real-Time Input Framework for Smartphone–Smartwatch Videogames
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Persistent Multi-User Virtual Reality Garden: Architecture, Traceability, and Technical Validation

by
Giovanni Giuliodori
*,
Erica Santaguida
,
Chiara Evangelista
and
Massimo Bergamasco
Institute of Mechanical Intelligence, Scuola Superiore Sant’Anna, 56127 Pisa, Italy
*
Author to whom correspondence should be addressed.
Multimodal Technol. Interact. 2026, 10(9), 87; https://doi.org/10.3390/mti10090087
Submission received: 9 July 2026 / Revised: 19 August 2026 / Accepted: 21 August 2026 / Published: 23 August 2026
(This article belongs to the Topic AI-Based Interactive and Immersive Systems)

Abstract

Virtual reality (VR) applications are often designed as episodic experiences, with limited support for persistence, longitudinal revisitation, and structured integration of interaction data across sessions. This paper presents a persistent multi-user VR garden architecture that combines snapshot-based state restoration, structured event, movement, and transcript records, an asymmetric owner–visitor workflow, cloud-mediated speech transcription, and deferred AI-supported synthesis. The architecture separates the current spatial configuration of the environment from the interaction traces through which it evolves, supporting repeated access, state restoration, historical consultation, and post-hoc processing. A controlled technical validation using synthetic or researcher-generated inputs was conducted through Unity Editor/backend tests and on Meta Quest 3 hardware. Persistence was evaluated at 20, 100, and 250 objects in the Editor and at 1, 50, and 100 objects on Quest, with two Quest replicas per load. Application-level visitor restrictions were verified across nine prohibited write operations. A frozen production speech-to-text corpus completed 40/40 requests with a micro-averaged word error rate of 9.50% and a median end-to-end latency of 1.619 s. The deferred AI pipeline was additionally verified as a functioning technical integration, while limitations in semantic-detail preservation were observed. The results support the implementation-level feasibility of the proposed persistent and traceable VR architecture. The present evaluation does not establish backend-level authorization guarantees, general AI or NPC-grounding performance, user outcomes, or clinical effectiveness.

1. Introduction

Virtual reality (VR) provides controlled, immersive, and configurable environments in which users can navigate, manipulate three-dimensional content, and interact with virtual or human-controlled entities. Immersive systems can elicit presence, embodied engagement, and realistic responses to virtual situations, making VR suitable for experiences involving spatial behavior, social interaction, and collaborative activity [1,2]. Beyond task-oriented simulations, recent developments in shared virtual environments, avatar-mediated interaction, collaborative mixed reality, virtual humans, and conversational agents have expanded the possibilities for communication, co-presence, and joint activity within immersive spaces [3,4,5,6,7,8].
Despite these developments, many VR applications remain primarily episodic. Users typically enter a predefined scenario, perform a bounded activity, and leave an environment that is reset or only partially preserved after the session. This model is appropriate for applications centered on discrete tasks or repeatable scenarios, but it provides limited support for open-ended construction, longitudinal revisitation, and the progressive accumulation of user-generated content. Moreover, the spatial state of an environment and the associated records of actions, movements, and verbal exchanges may be preserved separately, limiting their joint inspection across sessions.
This limitation becomes relevant when a virtual environment is intended to evolve through repeated use. In the present work, continuity involves both the restoration of the current environment and the preservation of complementary traces describing previous interactions. Work on long-term human–computer interaction has shown that continuity across repeated encounters can support evolving patterns of interaction and sustained engagement with interactive systems [9]. These considerations motivate VR infrastructures that treat a virtual space not only as a temporary scenario, but also as a durable artifact that can be constructed, revisited, compared, and discussed over time.
The present work is also informed by theories of distributed cognition, external cognition, embodied interaction, situated action, and the intelligent use of space [10,11,12,13,14,15]. From these perspectives, cognitive activity is supported not only by internal processes, but also by external artifacts, spatial arrangements, representational structures, and situated action. Within an immersive environment, placed objects, spatial configurations, and semantic annotations can function as external representations that users can manipulate and inspect. When these representations persist across sessions, the virtual environment can provide a spatial substrate for organizing and revisiting information.
Collaborative systems introduce an additional requirement: joint activity must be balanced with awareness, responsibility, and control over shared artifacts. Research in Computer-Supported Cooperative Work (CSCW) and collaborative learning has emphasized the importance of shared workspaces, awareness of others’ activities, and mechanisms for coordinating contributions [16,17,18,19]. In the architecture presented here, collaboration is defined in an asymmetric and bounded sense: participants may share presence, observation, navigation, and voice-based consultation, while authority over persistent modifications remains associated with the owner of the garden.
Motivated by these considerations, this paper presents a persistent multi-user VR garden architecture intended to support owner-led spatial construction, longitudinal revisitation, and collaborative consultation. The garden is conceived as an open and configurable immersive environment in which the owner can place, modify, remove, annotate, and revisit three-dimensional elements across multiple sessions. The system does not assign predefined meanings to objects or spatial configurations. Instead, it preserves user-generated spatial and semantic information so that the environment can be progressively constructed and later inspected.
Collaborative access is organized through owner and visitor roles. The owner is associated with the persistent garden and retains authority over modifications to its long-term state. A visitor can join the same immersive environment, observe its configuration, navigate, communicate with the owner, and participate in discussion without receiving persistent editing rights in the standard client workflow. This model separates co-presence and consultation from persistent control of the shared artifact.
The system integrates four architectural components. First, a snapshot-based persistence layer stores complete garden configurations for state restoration and read-only consultation of retained daily states. Second, event, movement, and transcript structures preserve complementary session-linked traces that support later inspection without constituting a complete replayable event history. Third, a multi-user networking layer supports avatar-based co-presence and voice communication within an asymmetric owner–visitor workflow. Fourth, a deferred artificial intelligence (AI) pipeline processes previously stored data and produces structured artifacts that can be supplied to an embodied conversational Non-Playable Character (NPC). The AI/NPC path is positioned downstream from the live interaction loop and is treated in this study as a technical integration layer rather than as the principal contribution.
A preliminary version of the virtual garden was previously presented as a conference prototype [20]. That prototype already described multi-session persistence, snapshots and event-oriented logging, Photon-based co-presence, owner and visitor roles, and asynchronous AI/NPC consultation. The present journal article therefore does not claim these individual components as newly introduced. Instead, it extends the earlier prototype through an implementation-level formalization of component and data responsibilities; explicit Snapshot, EventLog, MoveLog, and Transcript structures; a documented cloud-backed lifecycle and bounded recovery behavior; a clarified application-level owner–visitor model; and a reproducible quantitative technical-validation package. No participant data from the conference study are reused or analyzed in the present article.
Accordingly, the contributions of this paper are as follows:
  • The implementation-level formalization of a persistent multi-session VR architecture that separates authoritative garden snapshots from complementary event, movement, and transcript traces;
  • An explicit asymmetric collaboration model supporting owner-controlled persistent construction and visitor-based co-presence, observation, communication, and consultation;
  • A documented cloud-backed lifecycle integrating state restoration, structured logging, speech transcription, deferred synthesis, and later consultation through clearly separated architectural stages;
  • A reproducible internal technical validation covering Editor/backend and Meta Quest 3 persistence, structural consistency, owner–visitor restrictions, production speech transcription, and bounded end-to-end pipeline operation.
The reported evaluation uses synthetic or researcher-generated inputs and addresses implementation-level architectural feasibility and technical traceability. It does not evaluate usability, reflective value, sustained engagement, collaborative user outcomes, backend-level authorization guarantees, general AI or NPC-grounding performance, or clinical effectiveness.
The remainder of the paper is organized as follows. Section 2 positions the work in relation to technically comparable persistent, collaborative, trace-oriented, and longitudinal systems. Section 3 presents the materials and methods, including the design requirements, system architecture, data models, trust boundaries, persistence and logging mechanisms, deferred synthesis lifecycle, and technical-validation protocol. Section 4 reports the technical-validation results. Section 5 discusses the architectural implications, the illustrative application context, limitations, and future research directions. Section 6 presents the conclusions.

2. Related Work

2.1. Collaborative and Multi-User VR

Research on collaborative mixed reality and social VR has established the importance of co-presence, shared interaction, embodied communication, and awareness within immersive environments [3,4,6]. These studies provide the conceptual context for multi-user immersion, but do not by themselves define how user-authored environments should persist or how modification authority and interaction records should be managed across sessions.
More technically comparable platforms address distributed experimentation and multimodal interaction. Ubiq-Exp supports remote and distributed mixed-reality experiments through participant management, distributed logging, recording and replay, and experimenter control over data flows [21]. SIGVerse combines Unity, cloud services, and the Robot Operating System to support multimodal human–robot interaction and reproduction of previous experiences [22]. These platforms demonstrate reusable support for distributed immersive research. Their primary focus differs, however, from the daily continuity of a user-authored environment and the application-level owner–visitor workflow considered here. Table 1 summarizes this architectural comparison.

2.2. Persistent and Longitudinal VR

Research on long-term human–computer relationships motivates continuity across repeated encounters and the progressive use of information accumulated over time [9]. In conversational systems, LOCOMO provides a more explicit evaluation reference by using multi-session conversations to assess long-term memory through question answering, event summarization, and multimodal dialogue-generation tasks [23]. Such work shows that retaining previous interactions does not in itself establish accurate semantic memory.
The present architecture addresses a complementary problem: it preserves a restorable spatial configuration together with session-linked event, movement, and transcript records, from which a derived brief can be generated. LOCOMO is not a VR scene-persistence architecture and does not evaluate spatial traces or owner–visitor capabilities. Conversely, this study does not perform a LOCOMO-like semantic-memory benchmark. Its longitudinal claim is limited to implemented state continuity, structured record retention, and bounded deferred processing.

2.3. Traceability and Multimodal Logging

Logging in distributed immersive systems can support experiment management, reproduction, and later analysis. Ubiq-Exp provides distributed logging and session recording/replay, while SIGVerse stores multimodal information to support reproduction and analysis of human–robot interaction [21,22]. These capabilities are relevant comparators, but recording or replay is not equivalent to the application-specific separation of state and traces adopted here.
Snapshot, EventLog, MoveLog, and Transcript have distinct responsibilities. Snapshot represents the garden state used for restoration; EventLog records discrete application and interaction events; MoveLog provides coarse timestamped spatial traceability; and Transcript preserves session-linked utterances. These records support later inspection without constituting a complete replayable event history. The architecture is not presented as event sourcing, exactly-once delivery, transactional snapshot–log coupling, or ground-truth movement measurement.

2.4. Speech-to-Text and Conversational Agents

Automatic speech recognition and embodied conversational agents are established research areas. Whisper provides the speech-recognition basis used by the cloud-mediated transcription path [24], while prior work describes the representational, interaction, and trust-related implications of embodied and virtual conversational agents [7,8,25].
The contribution of the present work is neither a new speech-recognition model nor a general conversational-agent architecture. It concerns the integration and controlled technical evaluation of transcription within a session-linked VR data flow. Transcribed utterances are retained as structured records and supplied to deferred processing. The measured performance reported later is bounded to the tested production path and corpus and does not establish noise, overlapping-speech, population-level, or multilingual robustness.

2.5. AI-Supported Synthesis and Knowledge-Mediated Interaction

Domain-specific immersive systems have integrated interactive or perceived data with structured knowledge reasoning. Li et al. present “A Novel Agricultural Machinery Intelligent Design System Based on Integrating Image Processing and Knowledge Reasoning”, combining image-based component and feature processing with structured design knowledge and interactive virtual assembly [26]. This provides a cross-domain example of a scoped perception-to-knowledge workflow, but not evidence for longitudinal persistence, owner–visitor roles, STT, or clinical applications.
Tang et al., in “A Review: Research Progress in Bridge Structural Health Monitoring From the Perspective of AI Development”, describe the progression from knowledge-driven approaches toward data-driven and explainable AI in bridge structural-health monitoring, including interpretability, stability, and security concerns [27]. The work is not concerned with VR or conversational memory. It is used only as a cross-domain reminder that a functioning AI pipeline should not be equated with validated, interpretable, or trustworthy inference.
Accordingly, the deferred AI layer considered here is classified as technical integration feasibility evidence. The implemented pipeline generates and applies derived records, while a semantic-detail omission was observed. General semantic validation, NPC grounding, hallucination robustness, explainable-AI performance, and clinical effectiveness remain outside scope.

2.6. Reflection and Mental-Health-Oriented VR

VR research in mental-health contexts and work on personal sensing and digital phenotyping motivate reflection-oriented applications and careful treatment of longitudinal records [28,29,30,31]. These studies do not validate the present architecture or establish its suitability for clinical practice. The garden is therefore presented as a technical infrastructure for possible future research, not as an evaluated therapeutic system.
Table 1. Focused comparison with technically informative or reviewer-suggested literature. The boundaries identify differences in purpose and evaluation scope rather than inferring that unreported capabilities are absent.
Table 1. Focused comparison with technically informative or reviewer-suggested literature. The boundaries identify differences in purpose and evaluation scope rather than inferring that unreported capabilities are absent.
ReferenceTechnically Comparable DimensionPrincipal Boundary with the Present Work
Ubiq-Exp [21]Distributed mixed-reality experimentation, participant management, logging, and recording/replay.Experimental orchestration rather than daily user-authored state, owner–visitor modification authority, or transcript-derived synthesis.
SIGVerse [22]Cloud-mediated Unity platform, multimodal interaction, synchronization, and reproduction of previous experiences.Human–robot research rather than a persistent garden lifecycle with structured session records and asymmetric application roles.
LOCOMO [23]Multi-session conversational-memory corpus and explicit semantic-evaluation tasks.Not a VR scene-persistence architecture; the present study does not perform an equivalent semantic-memory benchmark.
Li et al. [26]Integration of perception, structured knowledge reasoning, and interactive virtual assembly.Different domain and task; it does not support claims concerning longitudinal records, owner–visitor permissions, STT, or clinical use.
Tang et al. [27]Cross-domain treatment of knowledge-driven, data-driven, and explainable approaches to AI-supported monitoring.Not VR or conversational memory; used only to distinguish technical integration from validated and interpretable monitoring.

2.7. Relation to the Previous Conference Prototype

A preliminary conference paper introduced the persistent garden concept and several components retained in the present architecture [20]. Table 2 distinguishes those inherited components from the implementation, documentation, and validation extensions reported in this journal article.
The journal novelty therefore does not reside in introducing persistence, multiplayer access, event logging, or AI/NPC consultation as isolated functions. It resides in their implementation-level formalization, the explicit multimodal trace structures and lifecycle boundaries, and a new quantitative technical-validation basis.

2.8. Concluding Positioning

Taken together, the literature provides established approaches to collaborative immersion, distributed logging, multimodal capture, speech recognition, conversational interaction, knowledge-mediated processing, and long-term conversational-memory evaluation. The contribution of this work is its bounded integration and technical validation within a persistent multi-session VR workflow: persistent state restoration; structured Snapshot, EventLog, MoveLog, and Transcript traces; an asymmetric owner–visitor workflow; a cloud-backed lifecycle; speech transcription; and deferred AI integration. The system is not presented as a general scene-versioning service, a trusted backend authorization mechanism, a complete replayable history, a semantically validated memory agent, or a clinically effective intervention.

3. Materials and Methods

3.1. Design Requirements

The system was designed around five architectural requirements derived from its intended use as a persistent, revisitable, and multi-user virtual environment. Persistence and traceability were treated as distinct but complementary concerns: the former restores the current garden configuration, whereas the latter preserves structured records of how users interacted with the environment across sessions.
Table 3 summarizes the resulting requirements and their implementation mechanisms.

3.2. System Architecture

The platform is organized into five cooperating layers: the Unity-based VR client; the Photon real-time networking layer; Unity Gaming Services for authentication, persistence, and server-side functions; external speech and AI services accessed through Cloud Code; and a downstream conversational-agent interface. Figure 1 provides an architectural overview.
Table 4 reports the principal software components and package versions verified directly in the project configuration and vendored plugin sources.
The Unity service identity and the application-level identity serve different purposes. Unity Authentication provisions an anonymous PlayerId used to access Unity Gaming Services. The application separately maintains a short logical username and an owner/visitor role used to select persistent records, Photon rooms, and permitted interactions. These identifiers must therefore not be interpreted as equivalent identity or authorization mechanisms.

3.3. Client-Side VR Application

The VR client was developed in Unity for Meta Quest 3. It provides direct interaction with persistent garden objects, including object creation, selection, movement, modification, and deletion where allowed by the active application role. Persistent objects expose serializable state that can be collected into a garden snapshot and restored in a later application session.
Interaction logic, serialization, role checks, cloud requests, live networking, and speech capture are coordinated by client-side controllers. The client also initiates autosave and logout finalization, requests previously stored records during login and reopening, and supplies the currently restored state to the local scene.
Speech input follows a push-to-talk workflow. Audio is captured at 16 kHz, divided into chunks of at most 20 s, encoded as mono PCM16 WAV, and forwarded through Cloud Code. The client permits at most two concurrent transcription requests. The transcription service is invoked through the WhisperSttModule/WhisperProxy Cloud Code endpoint using Italian as the requested language and gpt-4o-mini-transcribe as the configured model.
The returned transcript contains text rather than word-level timestamps. Speaker and role labels are supplied as application metadata and do not constitute acoustic speaker diarization. Raw microphone audio is handled in memory for transmission, and no intentional first-party path was identified that stores the raw recording in local application persistence or Unity Cloud Save.

3.4. Multi-User Networking Layer

Live multi-user operation is implemented through Photon PUN and Photon Voice. A Photon room is associated with the logical username of the garden owner. The owner creates or joins that room, whereas a visitor joins the same room using the owner’s identifier. The configured room capacity is two participants.
Photon PUN supports replicated avatars, networked scene objects, and RPC-mediated interaction during the live session. Photon Voice provides live voice communication. Photon is not the authoritative persistence layer: durable garden state and structured records are written separately to Unity Cloud Save.
The application adopts an asymmetric owner/visitor model. The owner can perform the write operations required to alter and persist the garden. The visitor participates in the shared room but is treated as read-only by the application workflow. Role checks are enforced by the client-side interaction and persistence paths before owner-only operations are executed. This mechanism defines the implemented interaction policy but is not presented as an independent backend authorization guarantee.

3.5. Persistence and Structured Traceability

3.5.1. Snapshot Persistence

Persistent garden state is represented through daily snapshot documents using the schema identifier garden-snapshot@1.0. Snapshot keys follow the pattern
  • snap_{userId}_{YYYY-MM-DD}
where userId is the logical application identifier and the date identifies the corresponding daily record. Multiple saves performed on the same day target the same daily key, so a later save updates the current daily representation rather than creating a separate snapshot for every interaction session.
The client first attempts to load the daily snapshot associated with the most recently completed session. If that key is unavailable, it searches for the latest available snapshot from a preceding date. After retrieval, the stored representation is deserialized and used to reconstruct the persistent objects in the Unity scene.
Autosave is configured at 300-s intervals. An explicit finalization sequence is also initiated during logout. Snapshot and trace documents are written independently and are not committed through a shared transaction; partial writes can therefore produce divergence between stored documents.

3.5.2. Snapshot Data Model

A snapshot contains the information required by the implementation to reconstruct the current persistent garden configuration. Table 5 summarizes its principal fields.
The snapshot is the authoritative artifact for reconstructing the current spatial configuration. It is not intended to preserve the complete sequence of actions that led to that configuration.

3.5.3. Event, Movement, and Transcript Records

Three additional daily document families complement the snapshot (Table 6). They retain session-linked traces for later inspection and deferred processing while serving different purposes from spatial-state restoration.
EventLog keys follow the daily pattern
  • event_{userId}_{YYYY-MM-DD}
The three trace families remain logically separate from the snapshot because they answer different questions. The snapshot records what the persistent garden currently contains; the EventLog records selected application events; the MoveLog records coarse movement transitions; and the Transcript records recognized speech and its application context. Together, these artifacts provide structured traceability without constituting a complete deterministic reconstruction of every application state transition.

3.6. Data Flow, Trust Boundaries, and Governance

Table 7 identifies the principal data flows, their persistence properties, and the relevant trust-boundary interpretation.
These boundaries describe the data paths and separations examined in the implementation. They do not constitute a complete security, privacy, or data-governance framework. In particular, the application-level owner/visitor role model must be distinguished from service-side authentication and authorization.

3.7. Asynchronous AI Synthesis and NPC Interface

The AI-supported component operates as a deferred processing layer rather than as part of the live persistence or multi-user control loop. Its input consists of previously stored structured records, and its outputs are additional versioned artifacts written through the cloud pipeline.
The processing sequence is as follows:
  • BuildSessionsOverview identifies the available daily and session-linked source records;
  • BuildSessionsData converts selected EventLog, MoveLog, and Transcript records into a garden-sessions-data@1.1 artifact;
  • BuildThemesData derives a garden-themes-data@1.0 artifact using the themes@v1 vocabulary and the configured synthesis service;
  • BuildNpcBrief produces a schema-constrained garden-npc-brief@1.1 artifact through the Responses API;
  • The Unity client retrieves the result through GetNpcBrief and supplies it to ConvaiNpcBridge.SetNpcBriefJson.
SessionsData is derived from structured event, movement, and transcript records rather than from the current garden snapshot. This preserves the architectural distinction between current spatial state and longitudinal interaction traces. ThemesData is derived separately from session-linked Transcript records using the themes@v1 vocabulary. Both SessionsData and ThemesData are supplied to BuildNpcBrief; neither is derived from the other. The resulting npcBrief is the bounded contextual artifact exposed to the conversational-agent layer.
The NPC integration does not receive direct access to the live Unity scene or the complete raw record archive. It receives the generated brief through the client-side bridge. Consequently, the conversational layer remains downstream of persistence, trace collection, and deferred synthesis and is not required for core snapshot restoration or multi-user interaction.
The present evaluation treats this component as a bounded technical integration and does not include a general semantic-validation benchmark. The principal request configuration used in the bounded experiment, including the prompt templates, output schema, principal generation parameters, configured model alias, and run identifier, is preserved in Supplementary File (AI_REQUEST_CONFIGURATION.md).

3.8. Session Lifecycle and Data Flow

The implemented lifecycle comprises login, restoration, interaction, periodic persistence, logout finalization, deferred processing, and later reopening.
Figure 2 summarizes two deterministic entry paths. Login restores the stored garden, whereas account creation initializes an empty garden. Both paths converge in the live VR session, followed by persistence at logout, later reopening, and downstream processing of completed-session records.
Figure 2 simplifies the restoration step for readability; operationally, the client attempts to load the latest available daily snapshot and falls back to the most recent earlier snapshot when needed.
At login, Unity Authentication establishes the anonymous Unity service identity, while the application resolves the logical username and owner/visitor role used by its workflow. The owner creates or joins the Photon room associated with the logical garden identifier. A visitor joins the same room under read-only application restrictions.
For restoration, the client first attempts to load the daily snapshot associated with the most recently completed session. If the expected daily key is absent, the latest available earlier snapshot is retrieved. The resulting document is deserialized and used to reconstruct the persistent garden state.
During the live session, Photon handles replicated multi-user interaction and voice communication. Owner-authorized garden operations update the active scene, while the EventLog, MoveLog, and Transcript collectors accumulate complementary structured records. Speech selected through the push-to-talk workflow is forwarded through Cloud Code and returned as recognized text.
Periodic autosave writes the current snapshot and the daily trace documents to Cloud Save. Logout initiates a final persistence sequence intended to store the latest available state and collected traces before the session closes.
Deferred processing can subsequently derive SessionsData and ThemesData separately from the indexed stored records. Both derived artifacts are then supplied to BuildNpcBrief, which produces the npcBrief.
The generated brief can then be retrieved and applied to the downstream NPC interface. This downstream path is separated from the live persistence and networking workflow, allowing the core garden state and trace records to remain usable independently of the conversational component.

3.9. Technical Validation Protocol

The technical validation examined persistence, structured-record consistency, application-level owner–visitor restrictions, production speech transcription, and the bounded deferred AI pipeline. The evaluation was conducted in the Unity Editor/backend and on Meta Quest 3 devices. Each run was retained only when its prescribed checkpoints were completed and its local artifacts were internally consistent. Preliminary or interrupted attempts were preserved but excluded from aggregate results. When an automated controller outcome conflicted with the underlying runtime, persistence, or network evidence, the discrepancy was reviewed through a documented adjudication rather than being silently relabeled.
Network conditions were not instrumented as an independent experimental factor. The Editor/backend persistence measurements were produced within one automated campaign, and the 40 speech-transcription requests were executed as one contiguous production run. The six Quest cycles used the same Quest 3 installation over Wi-Fi and were completed in two replication series. Photon was configured to the eu region; however, this configuration characterizes only the multiplayer path and does not identify the service regions used by Unity Cloud Save or speech transcription. Bandwidth, jitter, packet loss, client-to-service latency, client geographic location, and Cloud Save/STT service regions were not recorded. Therefore, fixed client and protocol configurations were maintained, but identical measured network conditions cannot be claimed across repetitions.
Snapshot persistence in the Unity Editor/backend was evaluated at scene loads of 20, 100, and 250 objects. Serialization and cloud round-trip conditions were repeated 20 times at each load. The recorded metrics included payload size and serialization, upload, download, and restoration latency. Behavioral conditions additionally covered same-day replacement of an earlier state by a later state, retrieval when the expected snapshot was absent, rejection of malformed JSON, and recovery from a failed write. The same-day replacement condition comprised 20 observations, whereas the missing-snapshot and malformed-JSON conditions comprised five observations each. Failed-write behavior was evaluated separately through five valid repetitions in which connectivity was interrupted before the later state was written; validity required the previously stored state to remain retrievable and the failed replacement not to appear remotely.
Device-side persistence was evaluated on Meta Quest 3 at loads of 1, 50, and 100 objects, with two valid repetitions per load. Each cycle comprised creation and autosave of state A, reopening and restoration of A, modification to state B, logout finalization, and a second reopening and restoration of B. Object counts, persistent identifiers, spatial values, and the expected final state were compared after restoration. Position and rotation comparisons used tolerances of 0.0001  m and 0.001 ° , respectively. Payload size and the elapsed times for autosave upload, restoration of A, logout upload, and restoration of B were recorded. Cleanup of test data was verified after each completed cycle. Complete run identifiers and artifact hashes were retained in the validation archive rather than reproduced in the manuscript.
Structured-record consistency was assessed through a ten-test suite covering Snapshot, EventLog, MoveLog, and Transcript artifacts. The checks examined schema conformance, required identifiers, session linkage, structural relationships, and consistency across a valid artifact bundle. Deliberately inconsistent fixtures were retained as negative controls and were required to trigger the corresponding validation findings. MoveLog was evaluated as a transition-based traceability structure; no positional-accuracy or continuous-trajectory benchmark was performed.
The owner–visitor workflow was evaluated by establishing an owner state and then attempting nine visitor-side operations that would modify local state, persistent Cloud Save data, or Photon-mediated shared state if permitted. A valid read-only outcome required the visitor operation to be denied without changing the visitor-side reconstruction, the owner’s stored state, or the relevant networked modification evidence. Reconnection was assessed only for continuity of the visitor role, read-only status, room membership, and unchanged Photon MasterClient. The protocol evaluates application-level restrictions and does not constitute a backend authorization or malicious-client security assessment.
Production speech-to-text behavior was evaluated using a frozen, controlled Italian corpus comprising 40 requests: 35 spoken utterances and five silence inputs. The spoken subset included short, longer, and pause-containing utterances recorded under the controlled acquisition conditions. Silence inputs were expected to produce a no-speech outcome rather than a fabricated transcription. Spoken outputs were evaluated using macro- and micro-averaged word error rate. End-to-end latency was measured from request initiation to the returned application result and summarized using the median, 95th percentile, and maximum. Request failures and retries were also recorded. This protocol does not evaluate multilingual performance, robustness to uncontrolled noise, speaker diversity, or model-only inference latency.
The deferred AI pipeline was evaluated as a bounded technical integration using records produced across the prescribed two-day lifecycle. The procedure checked restoration of the prior state, production of the SessionsData, ThemesData, and npcBrief artifacts, retrieval of the generated brief, and its application through the Unity–Convai bridge. The generated content was also inspected for preservation of two source-specific semantic details. No conversational NPC query was executed, and the procedure was not treated as a general semantic-validation, NPC-grounding, hallucination-robustness, or clinical-effectiveness benchmark.
For all validation components, raw and protected artifacts, machine-readable reports, adjudication records, manifests, and SHA-256 digests were retained locally. These records were used to distinguish direct runtime passes, passes supported by technical adjudication, excluded attempts, and limitations introduced through explicit scope reduction. Supplementary File S1 provides curated run identifiers, validation summaries, adjudication records, checksums, selected non-sensitive raw persistence measurements, final structured-record-consistency outputs, and sanitized STT tabular results; additional operational artifacts remain in the local validation archive.

4. Technical Validation Results

The validation results are reported by technical component. The observations support implementation-level feasibility within the evaluated configurations and should not be interpreted as evidence of usability, clinical effectiveness, complete security, or general AI performance.

4.1. Persistence in the Unity Editor and Backend

The Unity Editor/backend campaign produced 150 valid automated observations across serialization, cloud round-trip, same-day replacement, missing-snapshot, and malformed-JSON conditions, with no behavioral failures. Serialization and cloud round-trip measurements comprised 20 repetitions at each evaluated load of 20, 100, and 250 objects. Table 8 reports the corresponding payload and latency measurements.
All 20 same-day replacement observations restored the later state B rather than the earlier state A. The five missing-snapshot observations and five malformed-JSON observations produced the expected controlled outcomes without being misclassified as successful restorations.
Failed-write recovery was evaluated separately in five valid repetitions. In each case, network loss was introduced before state B could be written. The previously persisted state A remained available, state B was absent from remote storage, and no restoration mismatch was observed. The median return time for the failed state-B write was 23.551 ms. Eight preliminary attempts that did not satisfy the failed-write fixture were preserved but excluded from this result.
These results demonstrate the evaluated round-trip and recovery behavior of the implemented daily-snapshot workflow. They do not establish transactional commits across independently written snapshot and trace documents, or protection against all concurrent-writer conditions.

4.2. Persistence on Meta Quest 3

Six valid Meta Quest 3 persistence cycles were retained: two repetitions at each load of 1, 50, and 100 objects. Every cycle restored both state A and the subsequent state B with the expected object count and persistent identifiers. No position mismatch above 0.0001  m or rotation mismatch above 0.001 ° was detected. Cleanup was confirmed after each completed cycle. Table 9 reports the device-side payload and timing measurements.
One preliminary 50-object attempt from the second replication series did not complete the prescribed cycle and was excluded while remaining available in the archive. The autosave interval was temporarily reduced during that replication series and was restored to the production value of 300 s after the measurements.
The device results confirm snapshot serialization, cloud upload, reopening, and reconstruction for the tested loads and repetitions. They do not provide a general upper bound on scene capacity or network latency under other connectivity conditions.

4.3. Structured Record Consistency

The structured-artifact consistency suite completed 10/10 tests successfully. The valid bundle comprising Snapshot, EventLog, MoveLog, and Transcript documents produced no structural errors. The validator additionally reported two non-blocking warnings and fourteen informational findings. Six deliberately introduced negative-control violations were detected as expected.
The checks support the internal consistency of the implemented schema identifiers, required fields, session linkage, and cross-document relationships for the evaluated fixtures. They do not establish that the EventLog forms a complete replayable event history. MoveLog was verified only as a transition-based record of movement between spatial cells and zones. No claim is made regarding continuous-trajectory reconstruction or movement-measurement accuracy, and a known timing gap can omit stationary time accumulated after the last recorded transition.

4.4. Owner–Visitor Application-Level Restrictions

The visitor workflow denied all 9/9 evaluated operations that could otherwise modify the garden. Across these attempts, no visitor-originated change was found in the local persistent reconstruction, the owner’s Cloud Save state, or the examined Photon modification evidence. The owner’s previously established state remained preserved.
An initial controller comparison reported a possible owner-state change during the visitor sequence. Review of the persistence timing and underlying artifacts showed that the difference resulted from delayed persistence of the owner’s preceding action rather than from a visitor write. The observation was therefore retained and adjudicated as a controller-timing false positive rather than removed from the record.
After reconnection, the participant remained classified as a visitor and read-only, rejoined the same two-participant room, and left the original owner as Photon MasterClient. Because no additional prohibited write was attempted after reconnection, this evidence is limited to continuity of role, read-only state, room membership, and MasterClient assignment.
These findings support the implemented client-mediated owner–visitor workflow under the tested application path. They do not establish backend-enforced authorization or resistance to a modified or malicious client.

4.5. Production Speech-to-Text Validation

The frozen production speech-to-text corpus completed all 40/40 requests without retries or request failures. All 35 spoken inputs returned transcriptions, and all five silence inputs produced the expected no-speech classification. Aggregate accuracy and latency results are reported in Table 10.
The maximum latency was produced by a single retained outlier in one spoken-input repetition; the other requests determine the substantially lower median and 95th percentile. Because the measurements cover a controlled, single-speaker Italian corpus, they support only the evaluated production path. They do not establish robustness to uncontrolled noise, multilingual generalization, speaker-independent accuracy, or model-only inference latency.

4.6. Deferred AI/NPC Technical Integration

The two-day deferred-processing workflow restored the previously stored state and completed the production of SessionsData, ThemesData, and a schema-constrained npcBrief. The brief was subsequently retrieved by the Unity client and applied through the conversational-agent bridge. The bridge returned to its ready state without an invalid lifecycle condition.
Inspection of the generated brief showed that the pipeline preserved broad themes represented in the synthetic test input, including general well-being and organization of the virtual space. However, it omitted two specific facts present in the source transcript: that the user felt well and intended to care for the garden. No unsupported factual statement was identified in the inspected output.
No NPC question was submitted during this campaign. The result therefore demonstrates pipeline completion, brief generation, retrieval, and application, together with an observed semantic-detail omission. It is classified as technical-integration feasibility evidence and not as general semantic validation, NPC-grounding validation, hallucination-robustness validation, or evidence of clinical effectiveness. Run-level accounting is summarized in Table 11.
Taken together, the validation results support the implementation-level feasibility of snapshot-based persistence, complementary structured traces, asymmetric application-level collaboration, production speech transcription, and bounded deferred synthesis within the evaluated conditions. The evidence does not extend beyond the explicitly tested loads, fixtures, language conditions, roles, or integration scope.

5. Discussion

The technical validation supports the implementation-level feasibility of a VR environment designed as a persistent, traceable, and multi-user infrastructure rather than as an isolated single-session application. The contribution lies in the integration of spatial-state restoration, complementary interaction records, asymmetric multi-user access, production speech transcription, and bounded deferred synthesis. The results characterize the implemented technical paths under controlled conditions; they do not provide evidence of usability, collaborative user outcomes, reflective value, or clinical effectiveness.

5.1. Persistence and Structured Longitudinal Traceability

A central implication of the architecture is the distinction between persistence as state restoration and traceability as partial evidence of the process through which the stored state evolved. Across the evaluated Editor/backend and Meta Quest 3 conditions, the system serialized, stored, reopened, and restored garden states at the tested scene loads. This supports the feasibility of the implemented daily-snapshot lifecycle, including same-day replacement and bounded recovery behavior, but does not define a general capacity or latency envelope for other devices, network conditions, or scene complexities.
The separation between Snapshot, EventLog, MoveLog, and Transcript records is important to this interpretation. A snapshot preserves the current reconstructable garden configuration. Event, movement, and transcript documents instead provide complementary session-linked traces that can support later inspection. They do not form a complete, exactly-once, or deterministically replayable history of every state transition. Traceability should therefore be understood as structured and partial provenance rather than complete process reconstruction.
This separation remains relevant for longitudinal VR research because it allows the current spatial state to remain usable independently of later analytical questions. Subject to the completeness and interpretation limits of the retained records, future analyses could examine changes in persistent elements, semantic annotations, coarse spatial activity, dialogue, or collaborative access across sessions. The present validation establishes the technical availability and structural consistency of these artifacts, not the validity of any behavioral or psychological inference derived from them.

5.2. Asymmetric Collaboration and Illustrative Application Context

The owner–visitor model separates shared presence from persistent modification authority. The owner remains responsible for the long-term garden state, whereas the visitor can join the room, observe, communicate, and participate without using the tested application paths to alter that state. The validation supports this application-level distinction across the evaluated visitor operations and provides bounded evidence that the visitor role, read-only state, room membership, and MasterClient assignment remain coherent after reconnection.
This asymmetry preserves the distinction between participation and control, which may be useful whenever the virtual environment represents a personal or semi-personal evolving artifact. It should not, however, be interpreted as a general security guarantee: the tested restrictions are implemented within the application workflow and were not evaluated against a modified or malicious client.
A mental-health-oriented garden provides one illustrative application context for this architecture [28,29,30,31]. In a future appropriately governed study, a user might progressively organize and revisit personally relevant spatial content, while a qualified visitor could observe and discuss that content without modifying its persistent configuration. Historical state, interaction traces, and post-hoc summaries could support later consultation, but the system assigns no fixed psychological meaning to objects, movements, labels, or utterances. This scenario was not evaluated in the present campaign and provides no evidence of reflective, therapeutic, collaborative, diagnostic, or clinical effectiveness. Any such deployment would require a dedicated protocol, ethical approval, informed consent, appropriate data governance, and qualified human supervision.

5.3. Speech Transcription and Bounded Deferred AI Integration

The production speech-to-text results provide technical evidence for the evaluated cloud-mediated transcription path. The frozen corpus was completed without request failures or retries, and the reported word-error and end-to-end latency distributions characterize performance for the controlled Italian acquisition condition. The retained latency outlier is important because it shows that successful completion does not imply uniform response time. These observations support the implemented production path but cannot be generalized to other speakers, languages, acoustic environments, overlapping speech, or model-only inference performance.
The deferred AI layer is deliberately separated from live interaction, persistence, and multi-user control. This reduces coupling between the immersive client and computationally variable external processing while ensuring that synthesis operates on explicit stored artifacts. The final campaign demonstrated restoration of the preceding state, production of SessionsData and ThemesData, generation and retrieval of the npcBrief, and application of that brief through the Unity–Convai bridge.
The same execution also exposed an important distinction between technical pipeline completion and semantic preservation. Although the generated brief retained broad themes and introduced no unsupported factual statement in the inspected output, it omitted two specific facts present in the source transcript. The component should therefore be interpreted only as bounded technical-integration feasibility evidence. No NPC query was executed during the final campaign, and the results do not constitute a semantic-coverage benchmark, NPC-grounding validation, hallucination-robustness assessment, or evidence that the generated summaries are suitable for autonomous interpretation.
The conversational-agent layer remains conceptually positioned as a downstream interface to a bounded, precomputed artifact rather than as an authoritative interpreter of raw interaction data. This positioning is consistent with concerns regarding trust, intelligibility, and accountability in conversational and context-aware systems [25,32]. Any interpretation of the stored records or generated summaries remains a human responsibility grounded in the application context.

5.4. Limitations and Future Work

Persistence and consistency. Snapshot and trace documents are written independently and are not committed through a shared transaction. Concurrent updates follow the implemented last-writer-wins behavior, without ETag-based conditional updates, compare-and-swap semantics, monotonic revision enforcement, or stale-writer rejection. The failed-write experiment covered pre-write network loss and showed preservation of the previously stored state under that condition; it does not characterize failures occurring during remote acceptance, partial multi-document writes, acknowledgements, or concurrent updates. Future work should introduce explicit document revisions, conditional writes, idempotent operations, and broader fault-injection tests if stronger consistency guarantees are required.
Traceability. The EventLog records selected application events but does not guarantee complete or exactly-once capture, global ordering, or deterministic replay. MoveLog stores coarse transitions between spatial cells rather than a continuous trajectory, and its transition-based implementation has a known timing gap for stationary time accumulated after the final recorded transition. No trajectory-ground-truth or movement-accuracy validation was conducted. Future work could add stronger event identities, ordering and provenance links and, where required by a specific research question, compare MoveLog output against an external movement reference. Such validation would be necessary before deriving precise behavioral or biomechanical measures.
Owner–visitor enforcement and trust boundaries. The owner–visitor restrictions were validated through the tested application paths and should not be interpreted as backend-enforced authorization against a modified client. The documented trust boundaries identify the observed flows between the Quest client, Photon, Unity Gaming Services, external processing services, and the downstream conversational layer, but they do not constitute a complete security, privacy, or data-governance program. Service-side authorization, credential lifecycle, third-party retention, processing region, deletion, correction, and secondary-use policies were not comprehensively evaluated. Future deployments should enforce owner-only mutations through trusted backend controls and define auditable retention, deletion, access, and third-party-processing policies.
Speech transcription. The speech-to-text corpus was controlled, Italian, and recorded with a single speaker. It did not benchmark uncontrolled or moderate noise, overlapping speech, multilingual use, speaker diversity, or variation across headset microphones. The reported measurements are end-to-end application latencies rather than isolated model inference times, and the retained maximum-latency outlier shows that occasional long responses remain possible. A broader frozen corpus with multiple speakers, languages, devices, acoustic conditions, and repeated measurements would be required before making robustness or population-level accuracy claims.
Deferred AI and NPC integration. The Cloud Save listing path did not follow links.next, which can make source discovery incomplete when records extend beyond the first page. Reproducibility is further constrained by the evolving configuration of externally hosted AI services. Future technical work should add pagination handling and preserve immutable provider-side model/version identifiers, where available, together with the relevant request configuration.
The observed semantic-detail omission was inspected in a single bounded pipeline case. No controlled multi-session corpus was used to evaluate coverage, omissions, contradictions, or unsupported claims, and no NPC query or grounding benchmark was included in the final campaign. A controlled multi-session semantic benchmark would be necessary before making general synthesis claims. Likewise, an explicit NPC-grounding evaluation would be required if conversational retrieval or grounding claims were reintroduced in future work; neither activity is implied by the present technical evidence.
Evaluation scope. The study provides internal architectural and technical validation rather than a new participant study. It does not evaluate usability, acceptability, sustained longitudinal engagement, reflective outcomes, collaborative user outcomes, therapeutic benefit, or clinical effectiveness. Future user research should be undertaken only after the technical and governance limitations relevant to the intended deployment have been addressed. Any mental-health or clinical evaluation would additionally require an appropriate study design, ethical review, informed consent, data-protection measures, and qualified professional oversight.

6. Conclusions

This work presented a persistent multi-user VR garden architecture designed to support continuity across sessions while preserving structured evidence of user interaction. The architecture separates the persistent spatial state, represented through restorable snapshots, from complementary EventLog, MoveLog, and Transcript records. This distinction supports state restoration and later inspection without treating the recorded traces as a complete replayable history.
The system further formalizes asymmetric owner–visitor collaboration by separating shared presence and communication from persistent modification authority. Quantitative technical validation in the Unity Editor/backend and on Meta Quest 3, together with production-path speech-transcription measurements, supports the operation of the principal persistence, traceability, and interaction mechanisms under the evaluated conditions. Deferred AI synthesis and the downstream NPC interface remain bounded secondary integration layers, separate from the core persistence and multi-user workflow.
The reported evidence establishes implementation-level feasibility rather than usability, clinical effectiveness, collaborative user outcomes, or general AI performance. The resulting platform provides a technically validated foundation on which future longitudinal, user-facing, and domain-specific studies can be designed under appropriate methodological, ethical, and governance frameworks.

Supplementary Materials

The following supporting information can be downloaded at https://www.mdpi.com/article/10.3390/mti10090087/s1, Supplementary File S1: a curated technical-validation reproducibility package containing aggregate validation tables, machine-readable JSON and CSV summaries, selected non-sensitive raw persistence measurements, final structured-record-consistency outputs, sanitized STT tabular results, a run-identifier index, a validation manifest with SHA-256 digests, a summary of technical adjudications and scope decisions, and AI_REQUEST_CONFIGURATION.md, which preserves the principal request configuration used in the bounded deferred-AI integration experiment.

Author Contributions

G.G.: conceptualization, methodology, software, validation, formal analysis, investigation, data curation, system design, implementation, data model design, writing—original draft, visualization. E.S.: methodology, conceptualization, writing—review and editing. C.E.: supervision, writing—review and editing. M.B.: supervision, conceptualization, project administration, writing—review and editing. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the European Union-NextGenerationEU, under the Italian National Recovery and Resilience Plan (PNRR), Mission 4, Component 2, Investment 1.3, project RESTART (PE00000001, CUP J53C22003120001), through the PhD scholarship in Emerging Digital Technologies at Scuola Superiore Sant’Anna.

Institutional Review Board Statement

Not applicable. The technical validation reported in this study used synthetic or researcher-generated test inputs and did not involve recruited human participants, human-subject research data, or animals.

Informed Consent Statement

Not applicable. No human participants were enrolled in the technical validation reported in this study.

Data Availability Statement

Technical validation artifacts were generated and analyzed in this study, including JSON and CSV records, raw and aggregate validation reports, manifests, adjudication records, run identifiers, and SHA-256 digests. These materials contain synthetic or researcher-generated technical inputs and no participant datasets. Supplementary File S1 provides aggregate and machine-readable validation records together with selected non-sensitive raw measurements for Editor/backend persistence and failed-write recovery, final structured-record-consistency outputs, and sanitized STT tabular results. The two STT result tables were sanitized only by removing service request identifiers; the scientific values used to calculate WER and latency statistics were preserved unchanged. Other retained operational materials are not deposited because they intermix validation evidence with account-, service-, device-, and Cloud Save identifiers, controller/debug records, backups, executable or build material, and third-party artifacts.

Acknowledgments

The authors thank the collaborators and colleagues who provided feedback during the development and refinement of the prototype.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Slater, M. Place Illusion and Plausibility Can Lead to Realistic Behaviour in Immersive Virtual Environments. Philos. Trans. R. Soc. B 2009, 364, 3549–3557. [Google Scholar] [CrossRef] [Scilit]
  2. Slater, M.; Sanchez-Vives, M.V. Enhancing Our Lives with Immersive Virtual Reality. Front. Robot. AI 2016, 3, 74. [Google Scholar] [CrossRef] [Scilit]
  3. Kyrlitsias, C.; Michael-Grigoriou, D. Social Interaction with Agents and Avatars in Immersive Virtual Environments: A Survey. Front. Virtual Real. 2022, 2, 786665. [Google Scholar] [CrossRef] [Scilit]
  4. Pan, X.; Hamilton, A.F.d.C. Why and How to Use Virtual Reality to Study Human Social Interaction: The Challenges of Exploring a New Research Landscape. Br. J. Psychol. 2018, 109, 395–417. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  5. Nowak, K.L.; Fox, J. Avatars and Computer-Mediated Communication: A Review of the Definitions, Uses, and Effects of Digital Representations. Rev. Commun. Res. 2018, 6, 30–53. [Google Scholar] [CrossRef] [Scilit]
  6. Billinghurst, M.; Kato, H. Collaborative Mixed Reality. In Mixed Reality: Merging Real and Virtual Worlds; Ohta, Y., Tamura, H., Eds.; Springer: Berlin/Heidelberg, Germany, 1999; pp. 261–284. [Google Scholar]
  7. Cassell, J. Embodied Conversational Agents: Representation and Intelligence in User Interfaces. AI Mag. 2001, 22, 67–83. [Google Scholar]
  8. Yildirim, C. An Immersive Model of User Trust in Conversational Agents in Virtual Reality. In Proceedings of the 2021 Third International Conference on Transdisciplinary AI (TransAI); IEEE: Piscataway, NJ, USA, 2021; pp. 17–18. [Google Scholar] [CrossRef] [Scilit]
  9. Bickmore, T.; Picard, R. Establishing and Maintaining Long-Term Human-Computer Relationships. ACM Trans. Comput.-Hum. Interact. 2005, 12, 293–327. [Google Scholar] [CrossRef] [Scilit]
  10. Hutchins, E. Cognition in the Wild; MIT Press: Cambridge, MA, USA, 1995. [Google Scholar]
  11. Clark, A.; Chalmers, D. The Extended Mind. Analysis 1998, 58, 7–19. [Google Scholar] [CrossRef]
  12. Scaife, M.; Rogers, Y. External Cognition: How Do Graphical Representations Work? Int. J. Hum.-Comput. Stud. 1996, 45, 185–213. [Google Scholar] [CrossRef] [Scilit]
  13. Dourish, P. Where the Action Is: The Foundations of Embodied Interaction; MIT Press: Cambridge, MA, USA, 2001. [Google Scholar]
  14. Suchman, L. Human-Machine Reconfigurations: Plans and Situated Actions; Cambridge University Press: Cambridge, UK, 2007. [Google Scholar]
  15. Kirsh, D. The Intelligent Use of Space. Artif. Intell. 1995, 73, 31–68. [Google Scholar] [CrossRef] [Scilit]
  16. Schmidt, K.; Bannon, L. Taking CSCW Seriously: Supporting Articulation Work. Comput. Support. Coop. Work (CSCW) 1992, 1, 7–40. [Google Scholar] [CrossRef] [Scilit]
  17. Dourish, P.; Bellotti, V. Awareness and Coordination in Shared Workspaces. In Proceedings of the 1992 ACM Conference on Computer-Supported Cooperative Work; ACM: New York, NY, USA, 1992; pp. 107–114. [Google Scholar] [CrossRef] [Scilit]
  18. Dillenbourg, P. (Ed.) What Do You Mean by Collaborative Learning? In Collaborative Learning: Cognitive and Computational Approaches; Elsevier: Oxford, UK, 1999; pp. 1–19. [Google Scholar]
  19. Stahl, G. Group Cognition: Computer Support for Building Collaborative Knowledge; MIT Press: Cambridge, MA, USA, 2006. [Google Scholar]
  20. Giuliodori, G.; Santaguida, E.; Lanfranco, E.; Caponi, M.; Evangelista, C.; Carrozzino, M.; Bergamasco, M. A Virtual Garden as an Infrastructure for Collaborative Mental Health Exploration in VR. In Proceedings of the 2026 IEEE 5th International Conference on Intelligent Reality (ICIR); IEEE: Piscataway, NJ, USA, 2026; pp. 1–5. [Google Scholar] [CrossRef] [Scilit]
  21. Steed, A.; Izzouzi, L.; Brandstätter, K.; Friston, S.; Congdon, B.; Olkkonen, O.; Giunchi, D.; Numan, N.; Swapp, D. Ubiq-Exp: A Toolkit to Build and Run Remote and Distributed Mixed Reality Experiments. Front. Virtual Real. 2022, 3, 912078. [Google Scholar] [CrossRef] [Scilit]
  22. Inamura, T.; Mizuchi, Y. SIGVerse: A Cloud-Based VR Platform for Research on Multimodal Human–Robot Interaction. Front. Robot. AI 2021, 8, 549360. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  23. Maharana, A.; Lee, D.H.; Tulyakov, S.; Bansal, M.; Barbieri, F.; Fang, Y. Evaluating Very Long-Term Conversational Memory of LLM Agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Bangkok, Thailand; Association for Computational Linguistics: Stroudsburg, PA, USA, 2024; pp. 13851–13870. [Google Scholar] [CrossRef] [Scilit]
  24. Radford, A.; Kim, J.W.; Xu, T.; Brockman, G.; McLeavey, C.; Sutskever, I. Robust Speech Recognition via Large-Scale Weak Supervision. In Proceedings of the 40th International Conference on Machine Learning; Proceedings of Machine Learning Research (PMLR); Curran Associates, Inc.: Red Hook, NY, USA, 2023; Volume 202, pp. 28492–28518. [Google Scholar]
  25. Rheu, M.; Shin, J.Y.; Peng, W.; Huh-Yoo, J. Systematic Review: Trust-Building Factors and Implications for Conversational Agent Design. Int. J. Hum.-Interact. 2021, 37, 81–96. [Google Scholar] [CrossRef] [Scilit]
  26. Li, C.; Tang, Y.; Zou, X.; Zhang, P.; Lin, J.; Lian, G.; Pan, Y. A Novel Agricultural Machinery Intelligent Design System Based on Integrating Image Processing and Knowledge Reasoning. Appl. Sci. 2022, 12, 7900. [Google Scholar] [CrossRef] [Scilit]
  27. Tang, Y.; Wan, S.; Yang, Q.; Chen, Z.; Xu, Y. A Review: Research Progress in Bridge Structural Health Monitoring from the Perspective of AI Development. Struct. Control Health Monit. 2025, 2025, 8870840. [Google Scholar] [CrossRef] [Scilit]
  28. Freeman, D.; Reeve, S.; Robinson, A.; Ehlers, A.; Clark, D.; Spanlang, B.; Slater, M. Virtual Reality in the Assessment, Understanding, and Treatment of Mental Health Disorders. Psychol. Med. 2017, 47, 2393–2400. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  29. Rizzo, A.A.; Koenig, S.T. Is clinical virtual reality ready for primetime? Neuropsychology 2017, 31, 877–899. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  30. Mohr, D.C.; Zhang, M.; Schueller, S.M. Personal Sensing: Understanding Mental Health Using Ubiquitous Sensors and Machine Learning. Annu. Rev. Clin. Psychol. 2017, 13, 23–47. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  31. Onnela, J.P.; Rauch, S.L. Harnessing Smartphone-Based Digital Phenotyping to Enhance Behavioral and Mental Health. Neuropsychopharmacology 2016, 41, 1691–1696. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  32. Bellotti, V.; Edwards, K. Intelligibility and Accountability: Human Considerations in Context-Aware Systems. Hum.-Interact. 2001, 16, 193–212. [Google Scholar] [CrossRef] [Scilit] [PubMed]
Figure 1. High-level architecture of the persistent multi-user VR garden. Photon supports transient live interaction and voice, whereas Unity Gaming Services stores persistent state and structured Snapshot, EventLog, MoveLog, and Transcript records. Speech audio is transcribed through an external service before the recognized text is stored as a Transcript. Completed records are subsequently processed by deferred AI synthesis, and the resulting npcBrief provides context to the conversational agent. Dashed arrow: deferred npcBrief delivery.
Figure 1. High-level architecture of the persistent multi-user VR garden. Photon supports transient live interaction and voice, whereas Unity Gaming Services stores persistent state and structured Snapshot, EventLog, MoveLog, and Transcript records. Speech audio is transcribed through an external service before the recognized text is stored as a Transcript. Completed records are subsequently processed by deferred AI synthesis, and the resulting npcBrief provides context to the conversational agent. Dashed arrow: deferred npcBrief delivery.
Mti 10 00087 g001
Figure 2. Session lifecycle of the implemented system. After login, the application restores the latest available daily snapshot associated with the user’s garden; if the expected snapshot is unavailable, it falls back to the most recent earlier snapshot. New accounts initialize an empty garden. During the live VR session, the system supports interaction, co-presence, trace capture, and periodic autosave. At logout, the current session is finalized and archived, and deferred processing can generate post-hoc summaries for later access. Dashed arrows denote deferred processing.
Figure 2. Session lifecycle of the implemented system. After login, the application restores the latest available daily snapshot associated with the user’s garden; if the expected snapshot is unavailable, it falls back to the most recent earlier snapshot. New accounts initialize an empty garden. During the live VR session, the system supports interaction, co-presence, trace capture, and periodic autosave. At logout, the current session is finalized and archived, and deferred processing can generate post-hoc summaries for later access. Dashed arrows denote deferred processing.
Mti 10 00087 g002
Table 2. Verified component-level relationship between the previous conference prototype and the present journal contribution.
Table 2. Verified component-level relationship between the previous conference prototype and the present journal contribution.
ComponentConference PrototypeJournal ExtensionRelationship
PersistenceComplete snapshots, session-boundary or explicit saves, latest-state restoration, and historical consultation.Daily snapshot policy, autosave and logout behavior, latest-valid fallback, recovery cases, schemas, and quantitative persistence evidence.Inherited and substantially extended.
Interaction recordsEvent-oriented history with timestamped atomic interactions, including spatial and verbal activity.Explicit Snapshot, EventLog, MoveLog, and Transcript structures with differentiated responsibilities and structural-consistency evidence.Inherited concept; extended data model.
Multi-user and owner–visitorPhoton-based co-presence, voice communication, owner control, and visitor non-modification.Cross-layer role formalization, separation of transient networking from persistent authority, operation-level denial evidence, and bounded reconnect continuity.Inherited and technically verified.
Speech and transcriptVerbal interaction was mentioned, but no STT pipeline or separate transcript schema was defined.Cloud-mediated STT, role-linked Transcript records, and a quantitatively evaluated production corpus.New explicit component.
AI and NPCAsynchronous external-language-service synthesis and an embodied NPC for consultation were already present.Named intermediate structures, a bounded npcBrief path, technical integration evidence, and an explicit semantic-detail limitation.Inherited and integration-level extension.
Technical validationAn 11-participant, two-session pre-clinical descriptive evaluation using user-oriented measures.A separate internal engineering campaign covering persistence, record consistency, permissions, STT, and bounded pipeline operation; no conference participant data are reused.New validation basis.
Table 3. Design requirements and corresponding implementation mechanisms.
Table 3. Design requirements and corresponding implementation mechanisms.
RequirementPurposeImplementation Mechanism
Longitudinal PersistencePreserve and restore the garden across repeated sessions while supporting a recoverable daily lifecycleCloud-backed daily snapshots, periodic autosave, logout finalization, and fallback retrieval of the latest available prior snapshot
Interaction TraceabilityPreserve complementary evidence of user activity separately from the current spatial configurationDaily EventLog, MoveLog, and Transcript documents linked to the logical user and session lifecycle
Asymmetric Collaborative AgencySupport shared owner–visitor interaction while preventing visitor-side modifications through the standard application workflowPhoton rooms associated with the logical owner identifier, replicated multi-user interaction, and application-level read-only restrictions for visitors
Separation of Interaction and AnalysisKeep live interaction and persistence independent from deferred speech and AI-supported processingUnity and Photon support the live experience; Cloud Save stores state and traces; Cloud Code and external services perform transcription and deferred synthesis
Robustness and ExtensibilitySupport recovery, component evolution, and the addition of downstream processing without coupling it to core interactionVersioned document schemas, separation of persistent state from trace records, independently written artifacts, layered services, and deferred processing in which SessionsData and ThemesData are derived separately and both supplied to BuildNpcBrief
Table 4. Principal software and service configuration.
Table 4. Principal software and service configuration.
ComponentVersion or ConfigurationRole
Unity Editor6000.0.39f1Application development, scene management, serialization, and build production
XR Interaction Toolkit3.2.1XR input and interaction support
OpenXR Plugin1.15.1OpenXR-based Meta Quest runtime integration
Unity Authentication3.5.2Anonymous Unity service identity and PlayerId provisioning
Unity Cloud Code2.10.2Server-side persistence orchestration and external-service mediation
Unity Cloud Save3.2.2Storage of snapshots and structured trace documents
Photon PUN2.49Real-time rooms, object replication, avatars, and RPC-based coordination
Photon Voice2.59Real-time voice communication between room participants
OpenAI speech servicegpt-4o-mini-transcribeItalian speech-to-text processing through Cloud Code
OpenAI synthesis servicegpt-5-miniDeferred thematic extraction and schema-constrained npcBrief generation through Cloud Code
Convai Unity integrationVendored plugin implementationDownstream delivery of the generated npcBrief to the conversational-agent layer
Table 5. Principal fields of the garden-snapshot@1.0 data model.
Table 5. Principal fields of the garden-snapshot@1.0 data model.
FieldPurpose
schemaIdentifies the snapshot representation as garden-snapshot@1.0
userIdIdentifies the logical user and garden associated with the snapshot
sessionIdAssociates the saved state with the application session active when the snapshot was produced
savedAtRecords the snapshot save time as an ISO 8601 timestamp
elements[]Contains the serialized collection of persistent garden elements
idProvides the persistent identifier of an individual element
typeIdentifies the element type or corresponding prefab
position, rotationEulerStore the element’s world position and rotation at save time
parentId, localPosition, localRotationEulerRepresent parent–child relationships and the child’s transform relative to its parent
plantedAtRecords when the element was originally placed, without being updated by an ordinary resave
semanticTagStores the element’s semantic metadata, including its container, tag source, identifier or custom text, and cluster path
gardenZoneAssociates the element with its resolved garden area
Element scale is not serialized by the current snapshot schema.
Table 6. Structured trace documents used alongside the persistent snapshot.
Table 6. Structured trace documents used alongside the persistent snapshot.
DocumentSchemaRecorded Content and Purpose
EventLoggarden-eventlog@1.1Daily application events associated with user interactions and lifecycle operations; supports partial provenance and later inspection but is not treated as a replayable event history
MoveLoggarden-move-day@1.0Coarse movement transitions between 5-m grid cells, using an XZ origin of ( − 100 , − 100 ) ; represents spatial transitions rather than a continuous trajectory
Transcriptgarden-transcript@1.0 entries within a garden-transcript-day@1.0 documentRecognized speech text, application-supplied role or speaker metadata, and session-related information used for longitudinal consultation and deferred synthesis
Table 7. Principal data flows and trust boundaries.
Table 7. Principal data flows and trust boundaries.
FlowData and TimingArchitectural RoleTrust-Boundary Interpretation
Quest client–PhotonRoom identifiers, avatar and object state, RPC messages, and voice during the live session; transientReal-time co-presence, replication, and voicePhoton carries live interaction but is not the authoritative store for persistent garden state or Transcript records
Quest client–Unity Gaming ServicesAnonymous service identity, logical record identifiers, Snapshot, EventLog, MoveLog, and Transcript documents; persistentAuthentication, restoration, trace storage, and server-side orchestrationUnity service identity and application-level owner/visitor identity remain distinct; persistent documents are written independently
Quest client–Cloud Code–speech serviceCaptured speech audio and transcription parameters during push-to-talk; recognized text returns to the client and is then stored as a TranscriptSpeech-to-text conversion before persistent transcript storageThe external request is mediated by Cloud Code and credentials are obtained through Unity Secret Manager; the application does not claim persistent storage of raw audio
Unity Gaming Services–
Cloud Code–AI service
Completed structured session records and schema-constrained prompts; deferred, with derived outputs persistedSynthesis of higher-level artifacts and npcBriefProcessing remains outside the live interaction loop; general semantic validity is outside the present evaluation scope
Unity Gaming Services–Quest client–
conversational agent
Persisted npcBrief retrieved at later access and supplied as synthesized contextDownstream conversational contextualizationThe conversational agent receives the bounded brief rather than direct access to the complete raw archive or live scene state
Table 8. Unity Editor/backend persistence results. Latencies are reported as median and 95th percentile across 20 repetitions per load.
Table 8. Unity Editor/backend persistence results. Latencies are reported as median and 95th percentile across 20 repetitions per load.
ObjectsPayload (B)Serialization (ms)Upload (ms)Download (ms)Restore (ms)
2012,9050.647/0.916177.863/509.659258.667/302.079260.182/393.860
10063,9812.769/3.789370.840/438.322366.534/417.794374.774/421.409
250160,4086.953/11.224403.994/552.997477.024/550.940497.075/558.774
Values are median/95th percentile. Payload size is the serialized snapshot size for the corresponding scene load.
Table 9. Meta Quest 3 persistence measurements for the six valid cycles.
Table 9. Meta Quest 3 persistence measurements for the six valid cycles.
ObjectsRep.Payload A/B (B)Autosave A (ms)Restore A (ms)Logout B (ms)Restore B (ms)
11669/6851434.3652772.475834.3542997.345
50127,213/27,2281407.2993534.4241190.0102846.087
100154,327/54,3423962.8633591.7001560.0192621.108
12669/685367.8323106.321554.5041843.828
50227,213/27,228736.3483434.6361592.3943407.289
100254,327/54,3421101.5523647.4452230.3263319.807
Table 10. Frozen production speech-to-text validation results.
Table 10. Frozen production speech-to-text validation results.
MeasureObserved Result
Completed requests40/40
Spoken inputs completed35/35
Silence inputs classified as no speech5/5
Retries0
Request failures0
Macro-averaged WER9.31%
Micro-averaged WER9.50%
Median end-to-end latency1.619 s
95th-percentile end-to-end latency2.446 s
Maximum end-to-end latency17.638 s
Table 11. Run-level accounting for the reported technical-validation components.
Table 11. Run-level accounting for the reported technical-validation components.
ComponentAttemptedRetainedExcludedAdjudicated
Editor/backend persistence1100
Failed-write recovery2110
Quest persistence7610
Structured-record consistency1100
Owner–visitor restrictions1101
Production speech-to-text1100
Deferred AI integration1100
Total141221
Note: The total row is shown in bold. Repetitions, observations, operations, and individual service requests are not counted as separate runs. Adjudicated runs are already included among retained runs.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Giuliodori, G.; Santaguida, E.; Evangelista, C.; Bergamasco, M. A Persistent Multi-User Virtual Reality Garden: Architecture, Traceability, and Technical Validation. Multimodal Technol. Interact. 2026, 10, 87. https://doi.org/10.3390/mti10090087

AMA Style

Giuliodori G, Santaguida E, Evangelista C, Bergamasco M. A Persistent Multi-User Virtual Reality Garden: Architecture, Traceability, and Technical Validation. Multimodal Technologies and Interaction. 2026; 10(9):87. https://doi.org/10.3390/mti10090087

Chicago/Turabian Style

Giuliodori, Giovanni, Erica Santaguida, Chiara Evangelista, and Massimo Bergamasco. 2026. "A Persistent Multi-User Virtual Reality Garden: Architecture, Traceability, and Technical Validation" Multimodal Technologies and Interaction 10, no. 9: 87. https://doi.org/10.3390/mti10090087

APA Style

Giuliodori, G., Santaguida, E., Evangelista, C., & Bergamasco, M. (2026). A Persistent Multi-User Virtual Reality Garden: Architecture, Traceability, and Technical Validation. Multimodal Technologies and Interaction, 10(9), 87. https://doi.org/10.3390/mti10090087

Article Metrics

Back to TopTop