Development of an Immersive Digital Twin Environment Using Gaussian Splatting for Facility Visualization in Virtual Reality

Abstract

Digital twin technologies provide new opportunities for visualizing, analyzing, and managing complex physical systems; however, effective methods are needed to transform digital twin data into immersive environments that support human interaction and decision-making. This study presents the development of a Digital Twin Immersive Environment (DTIE) prototype for facility visualization and analysis at the U.S. Army Corps of Engineers Engineering Research and Development Center (ERDC) in Vicksburg, Mississippi. A photogrammetry-based workflow was used to capture the facility interior and generate a dense three-dimensional point cloud, while a separate Gaussian splat reconstruction was generated from the same image dataset. The Gaussian splat representation was spatially aligned with Revit-derived CAD geometry to provide complementary geometric and photorealistic representations within the immersive environment. A virtual reality (VR) environment was developed with a user interface (UI) supporting ray-cast interaction, speech-to-text data entry, multi-user collaboration, switching between CAD and Gaussian splat representations, in-scene web browsing, and occupant movement simulation. The resulting DTIE prototype demonstrates the technical integration of complementary CAD and Gaussian splat representations with immersive interaction capabilities for facility digital twins. The environment operated at approximately 71.8 FPS under the reported Meta Quest 2 configuration. This work establishes a prototype framework for future investigation of collaborative visualization, scenario analysis, and decision-support applications for complex infrastructure systems.

1. Introduction

Virtual reality (VR) technologies have rapidly expanded beyond entertainment into domains such as training, education, engineering, and simulation, creating increasing demand for intuitive and efficient interaction within immersive environments. A central component of these interactions is the user interface (UI), which enables users to access functions, configure settings, and navigate complex virtual spaces. Unlike traditional two-dimensional interfaces, VR environments operate within spatial, three-dimensional contexts where depth perception, spatial orientation, and controller-based interaction introduce new usability challenges. Consequently, interface designs that perform effectively in conventional desktop or mobile environments do not always translate successfully to immersive systems. As VR technologies continue to mature, they present new opportunities to support complex analysis, visualization, and decision-making processes through highly interactive digital environments.

One emerging capability that benefits significantly from immersive technologies is the development and application of digital twins. A digital twin has been defined as “a virtual representation of an object or system that spans its lifecycle, is updated from real-time data, and uses simulation, machine learning, and reasoning to assist decision-making.” Digital twins provide engineers and stakeholders with tools to monitor system performance, analyze operational conditions, and predict how systems may behave under changing circumstances. In addition to real-time monitoring and predictive analytics, digital twins function as powerful testbeds that allow experimentation and optimization without risk to physical systems. By replicating real-world infrastructure and operational scenarios in virtual environments, digital twins enable new approaches to research, design evaluation, and system management across a wide range of industries. Their ability to bridge physical and digital environments fundamentally changes how complex systems are understood, analyzed, and improved.

VR technologies can further enhance the capabilities of digital twins by providing immersive, collaborative environments in which stakeholders can interact with virtual representations of physical infrastructure. Installation commanders, facility managers, engineers, and design professionals can use immersive environments to explore design alternatives, analyze facility operations, and collaboratively evaluate potential solutions. However, multi-user collaboration in immersive environments presents several challenges, particularly when geographically distributed teams must interact with digital twin models throughout the lifecycle of a facility. Addressing these challenges requires the development of platforms that integrate digital twin data with immersive visualization and collaborative interaction tools.

This project presents the development of a Digital Twin Immersive Environment (DTIE) prototype designed specifically for facility digital twins. The DTIE platform aims to extend the functionality of earlier systems by integrating facility digital twin data with immersive visualization, better UIs, and interactive analysis capabilities. The environment is intended to support future incorporation of sensor data streams from building automation systems and infrastructure monitoring platforms, including electrical, water, gas, and occupancy systems. These data sources will be visualized within an immersive VR environment, allowing users to interact with and analyze operational information in real time. The environment will also support simulation of potential incidents and operational scenarios, enabling stakeholders to identify design limitations, evaluate system performance, and improve response strategies before issues occur in physical facilities. In the present prototype, the physical-to-digital relationship is established through facility data acquisition and reconstruction rather than continuous real-time synchronization. The architecture is designed to provide a foundation for future integration of sensor-driven state updates and operational data streams as the DTIE progresses toward a persistent facility digital twin.

The DTIE concept aligns with broader strategic goals related to modernization and resilience in infrastructure management. In particular, the platform supports objectives outlined in the U.S. Army Installations Strategy (December 2020), which emphasizes strengthening readiness and resilience through technological innovation and improved operational insight [1]. By enabling immersive interaction with facility digital twins, the DTIE contributes to the strategic objective of modernizing installation operations through advanced visualization, simulation, and collaborative decision-making tools.

The motivation for this work is also driven by practical challenges faced with large-scale infrastructure design and construction. The U.S. Army Corps of Engineers operates forty-three district locations and manages approximately 150 new design projects annually. Significant effort is invested in creating detailed digital models during the design process; however, these models are frequently reduced to two-dimensional representations when projects enter construction phases. As requirements evolve toward maintaining three-dimensional models throughout the full lifecycle of infrastructure projects, the demand for efficient methods to create, manage, and utilize immersive digital models has increased substantially. Digital twin technologies provide a pathway toward maintaining comprehensive three-dimensional representations that integrate structural models with operational and environmental data.

This combination of organizational scale and lifecycle information loss creates a broader infrastructure-management challenge. When detailed three-dimensional models are reduced to two-dimensional representations or remain separated from later operational information, geographically distributed stakeholders may lack a persistent, shared representation through which design intent, existing facility condition, and operational scenarios can be reviewed together. The DTIE is intended to address this gap by providing an immersive environment in which facility representations and collaborative interaction capabilities can remain accessible throughout successive phases of the infrastructure lifecycle. For USACE, such an approach has potential relevance not only to individual facility visualization but also to the coordination of distributed project teams and the continued use of digital models across a large annual portfolio of infrastructure projects.

Within a digital twin framework, infrastructure models can support three key operational capabilities: mirroring, monitoring, and simulation. The mirroring phase enables engineers, contractors, and researchers to visualize infrastructure models and interact with them through immersive technologies such as VR and augmented reality (AR). This capability allows stakeholders to review designs earlier in the development process and collaborate in real time to identify issues and refine system configurations. The monitoring phase integrates sensor data into the digital model, enabling continuous tracking of metrics such as occupancy levels, air quality, and acoustic conditions. Finally, the simulation phase allows users to explore “what-if” scenarios within the virtual environment, including testing emergency procedures, evaluating safety systems, and examining potential infrastructure modifications. Together, these capabilities support more informed decision-making and improve the overall quality and resilience of facility design and operation.

This work presents the development of the Digital Twin Immersive Environment (DTIE) prototype as an integrated framework for constructing and interacting with immersive facility-scale virtual environments. The primary contribution of this work is the integration of two complementary digital representations, structurally precise CAD geometry and photorealistic Gaussian splatting, within a single immersive VR environment. Rather than treating digital twin reconstruction and immersive interaction as separate development challenges, the DTIE combines the structural representation provided by Revit-derived CAD models with the visual fidelity of Gaussian splats and an interaction framework specifically designed for immersive facility review. The resulting environment further integrates a spatial panel-based UI, ray-cast interaction, speech-to-text input, multi-user functionality, in-scene web access, switching between CAD and Gaussian splat representations, and occupant movement simulation. Collectively, these capabilities extend the digital twin from a visual representation of a facility toward an interactive virtual environment in which users can explore multiple representations, access information, collaborate, and examine dynamic scenarios. The work therefore addresses research gaps at the intersection of immersive virtual environments, digital twin reconstruction, and human–computer interaction and provides a framework for investigating immersive facility digital twins as functional virtual worlds for engineering and infrastructure applications.

2. Literature Review

2.1. Growth of VR and the Role of User Interfaces

Recent research on virtual reality (VR) menu systems and usability reveals a notable gap in dedicated studies focused specifically on VR menu design. Much of the existing work emphasizes broader VR interface design principles, user experience frameworks, or application-specific implementations rather than systematic investigations of menu usability within immersive environments. Despite rapid advances in VR technology, research examining menu systems in VR remains relatively limited, presenting an important opportunity for further study in human–computer interaction and VR interface design. VR technologies are increasingly used across training, simulation, and industrial applications, where the effectiveness of these systems depends heavily on the design of the user interface (UI). A critical challenge for VR UI design is achieving an appropriate balance between immersion, usability, and ergonomic interaction. Recent comparative studies of VR menus and interfaces highlight recurring themes related to interaction design, usability evaluation, and emerging interface paradigms [2,3].

The rapid growth of VR technologies across diverse domains, including entertainment, education, healthcare, and industrial applications, has intensified interest in improving the usability of immersive systems [4]. As VR systems become more sophisticated and widely adopted, the design of user interfaces within these environments has emerged as a key factor influencing user experience and overall system effectiveness [5]. Within VR applications, menu systems serve as essential navigation and interaction mechanisms that allow users to access functions, configure settings, and control system behavior within three-dimensional virtual environments.

2.2. Challenges of Translating Traditional UI Design into VR

Traditional two-dimensional (2D) menu design principles developed for desktop and mobile computing environments do not directly translate to immersive 3D spaces. The transition from flat interfaces to volumetric environments introduces several new design challenges, including issues related to depth perception, spatial cognition, hand–eye coordination in three-dimensional space, and the absence of traditional input devices such as keyboards and mice [6]. These factors significantly influence how users perceive and interact with menus in VR systems. Consequently, the design of VR menus requires reconsideration of established interface design practices as well as the development of new interaction techniques and usability evaluation approaches [7].

2.3. Research Gap in VR Menu Usability

Despite the critical role that menu systems play in VR applications, the literature highlights a clear lack of focused research examining VR menu usability and design principles. While considerable research has explored general VR interface design and user experience, systematic investigations into menu-specific challenges, design methodologies, and usability evaluation frameworks remain limited [8]. This gap is particularly concerning because menu systems often represent the primary mechanism through which users interact with complex VR applications.

Many existing VR menu implementations exhibit usability issues such as spatial disorientation, difficulty selecting menu items, cognitive overload caused by complex hierarchical menu structures, and accessibility challenges for users with diverse abilities [9]. These challenges are further compounded by the absence of standardized design guidelines and evaluation methods specifically tailored to VR menu systems. Additionally, the rapid evolution of VR hardware platforms, including developments in hand tracking, eye tracking, and haptic feedback, introduces additional complexity when determining optimal menu design approaches for different interaction modalities.

2.4. Importance of Improving VR Menu Usability

Understanding and improving VR menu usability has implications that extend beyond academic research into practical system development and technology adoption. Poorly designed VR menu systems can lead to user frustration, reduced task efficiency, increased cognitive workload, and ultimately abandonment of VR technologies [10]. Conversely, well-designed menu systems can enhance user engagement, improve task performance, and support broader adoption of VR technologies across multiple domains.

Recent work has demonstrated the potential for significant improvements in VR interface usability through innovative design approaches. For example, Zhou et al. [11] reported that AI-driven adaptive user interface systems in VR environments achieved an 18.6% reduction in task completion time, a 47.8% reduction in errors, and a 34.9% increase in user satisfaction scores compared to static interface designs. These findings suggest substantial opportunities for improving VR menu usability through adaptive design approaches and emerging technologies.

2.5. Current Research Directions in VR Menu Systems

The existing literature on VR menu systems can generally be categorized into several areas: VR interface design principles, alternative interaction techniques, usability evaluation methods, and technical implementation frameworks. A comprehensive survey conducted by Chen et al. [8], which analyzed 438 publications on VR interaction interface design between 2011 and 2023, found that most studies focus on practical case studies within specific application contexts rather than systematic investigations of underlying design principles.

Researchers have also explored several alternative interaction modalities for VR menu systems. These include gaze-based navigation [12], gesture-based interfaces [13], and voice-controlled systems [14]. Such approaches suggest that multimodal interaction techniques may offer advantages over traditional controller-based menu navigation by improving naturalness, accessibility, and cognitive efficiency. However, comparative evaluations examining the relative effectiveness of these interaction methods across different menu types and user populations remain limited.

2.6. Emerging Adaptive Intelligent Interface Approaches

Another emerging area of research involves adaptive and intelligent interface systems. Studies investigating AI-driven adaptive interfaces demonstrate promising results for improving user performance and satisfaction in VR environments [11]. However, the application of artificial intelligence and machine learning techniques to VR menu design remains in its early stages, indicating substantial opportunities for future research.

2.7. Menu Archetypes and Layouts

Research examining VR menu archetypes and layouts has investigated several interface designs including radial, panel, and hybrid menu structures. Studies comparing radial and panel menus suggest that radial menus often outperform panel menus in terms of speed and error rates [15], although panel menus may offer advantages related to familiarity for users accustomed to traditional interfaces [2]. Similarly, research has shown that user preference for menu types can be highly context dependent. For example, world-fixed menus may improve immersion, while handheld menus provide improved accessibility and ease of use [16].

However, conflicting results have also been reported. Research conducted by Wall et al. [17] indicated that a laser-pointer-based menu interaction method outperformed radial menus in both speed and accuracy when evaluated with users who had little or no prior experience with VR systems. This illustrates that menu placement can play a significant role in usability. Handheld menus can improve accessibility for novice users [3], while world-anchored menus may enhance immersion but can also increase physical fatigue [18]. Additionally, factors such as icon size and grouping have been shown to influence usability outcomes. Larger icons and clustered arrangements can reduce user errors but may also contribute to visual clutter within the interface [19].

Ergonomic evaluations of VR menu systems emphasize the importance of placing menus at appropriate distances within the virtual environment. Studies recommend mid-depth menu placement combined with ray-casting interaction techniques to balance user comfort and selection accuracy [20]. Menus placed too close or too far from the user cause fatigue and degrade task efficiency.

2.8. Usability Evaluation of VR Menus

Studies examining VR menu usability frequently highlight a trade-off between two-dimensional and three-dimensional interface designs. While 2D menus tend to be easier for novice users to understand and navigate, 3D menu systems often provide greater immersion within virtual environments but may also increase error rates and cognitive load [19]. Radial menus have also been found to outperform panel menus in terms of selection efficiency and error reduction in several studies [2], although opposing results have been reported by Wall et al. [17].

Additional research examining hand-referenced menu systems indicates that such interfaces can improve efficiency during short tasks but may lead to interface clutter and user fatigue during extended interactions [21,22]. At a broader system level, immersive VR visualization has also been shown to improve situational awareness and reduce cognitive load in complex operational environments. For example, research examining VR interfaces for satellite operations demonstrated that immersive 3D visualization can significantly improve operator understanding compared to traditional 2D dashboards [23]. Standardized usability assessment tools such as the NASA-TLX workload index and the System Usability Scale (SUS) are commonly used to evaluate VR interaction systems and provide consistent measurement frameworks across studies [20].

Despite these advances, relatively few studies have examined issues related to accessibility and inclusivity in VR interface design. Review studies indicate that universal design principles remain underexplored in VR user interface research, particularly with regard to individuals with disabilities and culturally diverse user populations [24]. This gap underscores a need for broader usability studies beyond traditional user groups.

2.9. Emerging Trends in VR Menu Systems

The literature suggests that future VR menu systems are likely to become more adaptive and personalized, dynamically adjusting menu placement, interaction modalities, and visual layouts based on user behavior and contextual factors [24]. Emerging research also explores multimodal interaction techniques that integrate gestures, haptic feedback, eye-tracking, and voice commands to reduce reliance on any single interaction modality [16]. While VR menu research has historically focused on gaming applications, recent studies indicate that VR interfaces are increasingly being used in fields such as industrial engineering, aerospace operations, and education [23].

Collectively, these studies provide evidence-based guidelines for ergonomic menu placement, interaction design, and icon organization. The literature generally suggests that radial menus positioned at arm’s length with ray-casting interaction techniques can provide effective performance outcomes. However, conflicting findings regarding radial and panel menu usability indicate that optimal interface design may depend on user experience level and task context. Furthermore, while 2D menu systems may benefit novice users, spatial 3D menu systems can enhance immersion and engagement within virtual environments. Despite these advances, issues related to accessibility and cultural inclusivity remain insufficiently explored, representing an important direction for future research.

2.10. Impact of Immersive Review Processes for Digital Twins

Beyond interface design, immersive VR environments are increasingly being used to support the review and evaluation of digital twin systems. A systematic literature review analyzing approximately thirty studies that combine VR with digital twin technologies concluded that immersive environments significantly improve how users interpret complex system data by providing spatially intuitive visualizations [25]. These environments support enhanced collaboration, improved training capabilities, and remote monitoring of physical systems. However, challenges remain related to interaction design, hardware limitations, and network requirements needed to support real-time synchronization of digital twin data.

Additional research has examined immersive digital twin design processes within architectural contexts. One study proposed an immersive evaluation approach in which stakeholders, including patients, healthcare professionals, and architects, interacted with digital twin models of healthcare facilities in VR to assess environmental characteristics such as lighting and spatial layout [26]. Results indicated that immersive design reviews can provide valuable user-centered insights that traditional design review methods may fail to capture.

Other work has explored the integration of digital twins with immersive simulation environments. One study developed a co-simulation framework connecting digital twin systems with VR interfaces, enabling operators to monitor system behavior and interact with simulated industrial processes [27]. The results demonstrated that immersive digital twins improved situational awareness and operational decision-making in complex manufacturing environments.

Research examining embodied interaction with digital twins in VR has also highlighted the importance of natural interaction mechanisms. One study categorized digital twin interactions into tangible interaction, social gestures, and social touch, emphasizing the role of natural interaction design in improving collaboration within immersive environments [28].

In addition to industrial applications, immersive digital twin environments have also been explored for training and education. A systematic review examining digital twins within immersive learning environments found that these systems can improve problem-solving skills and learning outcomes. In some cases, students interacting with digital twin systems achieved up to 20% higher performance on assessments and reported increased confidence in technical concepts [29].

Across these studies, several consistent advantages of immersive digital twin review processes have been identified, including improved spatial understanding, enhanced collaboration among stakeholders, earlier validation of design decisions, safer experimentation within simulated environments, and more effective training platforms. However, the literature also identifies several challenges, including high development and hardware costs, difficulties integrating and synchronizing large data sources, interaction design challenges in immersive environments, scalability limitations for large infrastructure models, and the potential for cognitive overload when excessive information is presented within VR environments.

2.11. Gaussian Splatting for Digital Twin Reconstruction

Recent advances in neural rendering have introduced new approaches for reconstructing digital twin environments using splatting techniques. Gaussian splatting represents scenes using numerous small elliptical primitives that can stretch and blend together to approximate surfaces within a three-dimensional environment. Each Gaussian splat can contain unique attributes such as color and opacity, allowing large numbers of splats to collectively form highly detailed visual representations of real-world scenes.

Machine learning algorithms can generate millions of these splats to reconstruct photorealistic environments that can be rendered efficiently in real time. Gaussian splatting has emerged as an alternative representation for reconstructing visually detailed environments that can be rendered in real time. Prior research has reported computational and rendering advantages relative to some traditional mesh- and point-cloud-based approaches. However, computational requirements depend on factors such as scene complexity, splat count, representation encoding, hardware, and rendering configuration. Accordingly, performance characteristics reported for one implementation should not be assumed to generalize across Gaussian splat representations or deployment environments [30].

Several recent studies have applied Gaussian splatting techniques to digital twin development. Gao et al. used Gaussian splatting to generate building digital twins by integrating geographic information systems (GIS), mapping platforms, and large language model analysis into a unified pipeline for digital twin generation from geographic data [31]. Similarly, Do et al. and Choi et al. used UAV imagery to construct Gaussian splat models of buildings and campuses, demonstrating how high-quality spatial data can be captured and reconstructed into digital twin environments [32,33]. Other researchers have expanded these methods to more complex systems. Guo et al. combined geometry reconstruction with motion modeling to develop digital twins of articulated systems [34], while Wang et al. applied Gaussian splatting techniques to infrastructure inspection, enabling three-dimensional visualization of structural damage for monitoring applications [35].

2.12. Recent Advancements in Digital Twin Research

Recent advancements in digital twin research have increasingly extended beyond geometric representation toward secure data integration, predictive analytics, and intelligent system management. One emerging area concerns the secure exchange and aggregation of data between physical assets and their virtual counterparts. Xincheng et al. developed a homomorphic aggregation approach for digital twin environments that provides both integrity and confidentiality protection when data from multiple physical devices are aggregated. Their work highlights the growing importance of secure data handling as digital twin systems become increasingly dependent on distributed sensing and continuous information exchange [36].

Digital twins are also being combined with artificial intelligence and machine learning to support predictive maintenance and condition monitoring. Lu et al. developed a gearbox fault-prediction framework that integrated a dynamic digital twin with deep transfer learning. The digital twin was used to generate simulated fault data under operating conditions that are difficult to reproduce experimentally, while transfer learning supported application of the learned model to physical gearbox fault diagnosis. Their results demonstrated the potential for digital twins to move beyond visualization and monitoring toward predictive and data-driven operational support [37].

These recent developments demonstrate the continuing evolution of digital twins from primarily representational models toward integrated platforms capable of supporting secure data exchange, predictive analysis, simulation, and operational decision support. The DTIE presented in this study contributes to this broader progression from a complementary perspective by focusing on immersive visualization and interaction. Rather than emphasizing cybersecurity or machine-learning-based fault prediction, the present work integrates CAD geometry, Gaussian splatting, immersive VR interaction, collaboration, information access, and scenario simulation within a unified facility-scale environment. Together, these research directions illustrate the increasingly multidisciplinary nature of digital twin systems and the importance of integrating visualization, data, simulation, security, and intelligent analysis within future digital twin platforms.

2.13. Research Gaps and Contributions of the DTIE Framework

Although prior research demonstrates substantial progress in Gaussian splatting, immersive digital twins, and VR interaction design, the literature reviewed in this study indicates that these areas have largely developed as related but distinct research directions. Gaussian splatting research has demonstrated the ability to generate visually detailed representations of buildings, campuses, infrastructure, and articulated systems [31,32,33,35]. These studies establish the value of splatting techniques for high-fidelity digital twin reconstruction; however, their primary emphasis is on the generation, visualization, or analysis of the reconstructed representation rather than its integration into a comprehensive immersive facility-review environment. Similarly, research on triangular splatting has focused primarily on rendering performance, novel-view synthesis, and compatibility with mesh-based graphics pipelines [38,39,40]. Collectively, these studies demonstrate important advances in digital reconstruction but leave opportunities for investigating how emerging reconstruction methods can be incorporated into functional, interactive VR-based digital twin environments.

A parallel body of research has investigated VR menu systems, interaction techniques, and immersive digital twin applications. Previous studies have examined radial and panel menus, ray-casting, menu placement, gaze-, gesture-, and voice-based interaction, and multimodal interfaces [2,3,12,13,14,15,16,17,18,19,20,21,22,23,24]. Other studies have demonstrated the benefits and challenges of immersive digital twin review for spatial understanding, collaboration, training, monitoring, and scenario evaluation [25,26,27,28,29]. However, the literature reviewed here indicates a need to more closely connect high-fidelity digital twin reconstruction with the interaction mechanisms required for users to practically explore and use those representations within immersive environments. In particular, relatively limited attention has been given to end-to-end frameworks that combine multiple facility representations, ergonomic VR interaction, collaborative functionality, information access, and dynamic scenario visualization within a common immersive environment.

The DTIE framework addresses this gap by integrating the reconstruction, visualization, interaction, and simulation components of a facility digital twin into a unified VR platform. Rather than relying exclusively on either a traditional geometric model or a photorealistic reconstruction, DTIE incorporates both CAD-based geometry and Gaussian splat representations and allows users to transition between them within the immersive environment. This hybrid approach is complemented by a panel-based UI positioned approximately five feet from the user and operated through ray-cast interaction. The environment further integrates speech-to-text input, multi-user functionality, in-scene web access, and occupant movement simulation. Together, these capabilities establish an end-to-end workflow extending from physical facility capture and digital reconstruction to immersive visualization and user interaction.

Hybrid scene representations provide an important context for positioning the DTIE. Recent reconstruction and rendering research increasingly combines explicit geometric representations with learned or image-based scene representations to preserve geometric structure while improving visual realism. Such approaches demonstrate that geometric and neural or splat-based representations can function as complementary scene components rather than mutually exclusive alternatives [41,42,43]. However, the research gap addressed in the present study is not the mathematical development of a new neural-geometric fusion algorithm. Instead, the contribution lies in operationalizing complementary CAD and Gaussian splat representations within a facility-scale immersive digital twin architecture that additionally integrates user-controlled representation switching, collaborative VR interaction, multimodal information access, and scenario simulation. Thus, the novelty of the DTIE is primarily system-level and interaction-oriented rather than a new underlying rendering formulation.

To more systematically position the DTIE relative to representative prior work, Table 1 compares the principal capabilities reported across immersive digital twin and Gaussian-splat-based digital twin studies reviewed in this section. The comparison focuses on system-level characteristics relevant to the present contribution, including facility-scale representation, CAD or geometric integration, Gaussian splatting, immersive VR, interaction capabilities, collaborative functionality, scenario simulation, physical-data synchronization, and quantitative evaluation. The comparison illustrates that prior studies have individually advanced immersive digital twin interaction, reconstruction, synchronization, simulation, and Gaussian-splat-based representation; however, the combination of these capabilities within a common facility-scale immersive architecture remains comparatively underexplored.

Table 1. Comparison of representative prior digital-twin and immersive-system research with the DTIE framework (NR = not reported or not established from the reviewed description).

Study/System	Representation	Immersive VR	Interaction/Collaboration	Simulation/Sync.	Evaluation
Immersive DT review [25]	Varies	✓	✓/Varies	Varies	Varies
Kemkomnerd and Tirapas [26]	Facility/geometric	✓	Interactive review	NR	User evaluation
Havard et al. [27]	Industrial DT	✓	Interactive VR	Co-simulation/sync.	Quantitative
Gaussian-splat (GS) DTs [31,32,33,34,35]	GS/geometric	Limited/NR	Limited/NR	Application-specific	Reconstruction focused
DTIE (present study)	CAD + GS facility model	✓	Multimodal + multi-user	Rule-based scenarios; no physical sync.	Runtime characterization only

As shown in Table 1, the contribution of the DTIE does not arise from any individual technology in isolation. CAD-based facility modeling, Gaussian splatting, immersive VR interaction, collaborative functionality, and simulation have each been demonstrated in prior research. Rather, the DTIE addresses the intersection of these research directions by integrating Revit-derived facility geometry and a Gaussian splat representation with immersive interaction, multi-user functionality, in-environment information access, and rule-based scenario visualization within a common VR architecture. At the same time, the comparison highlights capabilities that remain outside the present prototype, particularly continuous physical-data synchronization and comprehensive quantitative validation. The DTIE should therefore be interpreted as an integrated prototype architecture that establishes technical feasibility and a foundation for subsequent development and evaluation rather than as a fully synchronized or comprehensively validated operational digital twin.

Based on the gaps identified in the literature, the primary contributions of this study are fourfold: (1) Hybrid facility digital twin representation through the integration of CAD-based architectural geometry with a Gaussian splat representation of observed interior conditions within a common immersive environment, allowing the complementary characteristics of both representations to be accessed within the DTIE, (2) End-to-end immersive digital twin workflow that extends from image-based facility capture and photogrammetric reconstruction through independent Gaussian splat generation, integration with Revit-derived CAD geometry, and deployment within an interactive VR environment, (3) Integrated ergonomic and multimodal VR interface that combines a spatial panel-based menu, ray-cast interaction, speech-to-text input, multi-user functionality, model-representation switching, and in-scene web access to provide a unified interaction framework for exploring facility digital twins, and (4) Scenario-oriented immersive visualization that incorporates rule-based occupant movement simulation, allowing users to visualize people entering and exiting the facility and establishing a foundation for scenario-based exploration within the reconstructed environment.

These contributions distinguish the DTIE from approaches centered primarily on reconstruction, rendering, or individual VR interaction techniques. The research question underlying the work is therefore how complementary digital representations and immersive interaction technologies can be integrated into a functional facility-scale virtual environment that supports visualization, interaction, collaboration, information access, and scenario exploration. Accordingly, the scientific contribution of the work is not the isolated use of CAD or Gaussian splatting, but the design and implementation of an immersive digital twin architecture in which complementary geometric and photorealistic representations are integrated with a common interaction, collaboration, information-access, and simulation layer.

3. Methods

This study involved the development of an immersive facility digital twin prototype for facilities located at the U.S. Army Corps of Engineers Engineering Research and Development Center (ERDC) in Vicksburg, Mississippi. Facility models were authored in Autodesk Revit and exported as FBX files for import into the Unity game engine, where they serve as the geometric foundation of the immersive digital twin environment. To complement the CAD-based geometry and improve photorealistic representation of building surfaces and surroundings, Gaussian splatting techniques were incorporated into the development pipeline. This hybrid approach combined the modeled architectural geometry of the Revit-derived CAD representation with a photorealistic representation of observed interior conditions, while remaining suitable for real-time deployment within the target VR environment.

Figure 1 presents the system architecture of the DTIE and distinguishes between components implemented in the present prototype and capabilities envisioned for future persistent digital-twin operation. The implemented workflow begins with facility information acquired from existing CAD resources and image-based capture of the physical environment. These data are processed into complementary digital representations, including Revit-derived CAD geometry, a photogrammetric point cloud, and a Gaussian splat representation of observed interior conditions. The CAD and Gaussian splat representations are spatially aligned within Unity, which serves as the integration layer for immersive visualization and interaction. Within this environment, users can access the facility representations through the VR interface, switch between representation types, access external information, interact with other users, and explore synthetic occupant-movement scenarios using the rule-based simulation described in Section 3.6.

Figure 1. System architecture and data flow of the Digital Twin Immersive Environment (DTIE). Solid connections represent components and data flows implemented in the present prototype, including physical-facility data acquisition, digital representation generation, Unity-based integration, scenario visualization, and VR interaction. Dashed connections represent future persistent digital-twin capabilities, including facility sensing, state updates, monitoring, synchronization, and feedback mechanisms that were not implemented or evaluated in the present study.

The current prototype does not implement continuous sensing, automated synchronization with the physical facility, or closed-loop control. Accordingly, the relationship between the physical facility and the DTIE in the present study is established through facility data acquisition and digital reconstruction rather than continuous real-time state updating. The architecture was designed to accommodate future data streams from building automation and infrastructure-monitoring systems, which could enable updated facility-state information to be incorporated into the immersive environment. These future pathways are shown as dashed connections in Figure 1 to distinguish them from the capabilities implemented and evaluated in the present study. Thus, the DTIE presented here should be interpreted as an extensible immersive digital-twin framework and prototype architecture rather than a fully synchronized, closed-loop operational digital twin.

Figure 2 shows the exterior and interior views of the model.

Figure 2. Revit-derived CAD representation used as the geometric foundation of the DTIE. The images illustrate exterior and interior views of the facility model after export from Autodesk Revit and import into Unity. The CAD representation provides the architectural geometry onto which the Gaussian splat representation is spatially aligned.

3.1. Data Collection

Image data for the Gaussian splat component of the digital twin were collected within the interior of the target facility at the U.S. Army Corps of Engineers Engineering Research and Development Center (ERDC). Data acquisition was performed using an iPhone 14 Pro (device identifier iPhone15.2 manufactured in Zhengzhou, China by Apple) running Pix4Dcatch version 2.13.0 in LiDAR-guided capture mode. The acquisition produced 2000 image captures at a resolution of 1920 × 1440 pixels, together with corresponding depth and confidence data generated through the LiDAR-guided capture process. All imagery used in the reconstruction represented interior facility spaces; no exterior imagery was included in the dataset used for Gaussian splat generation.

The focal length was maintained consistently throughout acquisition, with no digital zoom applied. Exposure and ISO were managed automatically by the Pix4Dcatch guided-capture pipeline, and these parameters were not manually adjusted during data collection. Images were acquired while systematically traversing the interior environment to obtain coverage from multiple positions and viewing angles.

A fixed numerical image-overlap threshold was not prescribed during acquisition. Instead, Pix4Dcatch’s LiDAR-guided capture mode provided live tracking and coverage guidance during the session, allowing the operator to identify areas requiring additional capture. This guided procedure was used to obtain sufficient spatial coverage of architectural features while reducing gaps associated with occluded or complex interior surfaces. Attention was also given to lighting consistency and shadowing to improve continuity across the captured image set.

The resulting imagery, depth information, and confidence data served as the source dataset for subsequent photogrammetric reconstruction and Gaussian splat generation. Because these source data depict a restricted government facility, the original imagery and associated spatial datasets cannot be publicly distributed.

3.2. Gaussian Splat Generation

The captured image dataset was processed through Pix4D Cloud using the TrustedLocationOrientation calibration pipeline and the Standard image-matching algorithm. Processing was performed using 20,000 keypoints per image at full image scale, with densification configured at point-density level 2. Mesh, digital surface model/digital terrain model (DSM/DTM), and orthomosaic processing stages used the standard Pix4Dcatch processing template available at the time of processing in April 2025.

The Pix4D photogrammetric workflow produced a dense point cloud containing approximately 13.2 million points. This point cloud represented the reconstructed spatial geometry derived from the interior image dataset. Gaussian splats were subsequently generated from the same captured image set using a separate splat-training process rather than directly through the Pix4D Cloud photogrammetric output. The resulting Gaussian splat representation consisted of 761,250 Gaussian primitives contained within a single unified interior asset.

The Gaussian splat representation occupied a world-space bounding volume of approximately 47.26 m × 12.11 m × 35.24 m, corresponding to approximately 20,160.9 m³ and an average density of approximately 37.8 splats/m³. For deployment in Unity, the compressed splat asset had an on-disk size of approximately 35.1 MB and used Norm11 position/scale encoding, Norm8 × 4 color encoding, and Norm6 spherical harmonics compression.

The Gaussian splat representation was integrated with the Revit-derived FBX geometry to provide a photorealistic representation of observed interior conditions alongside the modeled CAD geometry. In the present study, visual fidelity was assessed descriptively through inspection of the resulting immersive representation; quantitative image-quality metrics such as PSNR and SSIM and a formal measure of surface-artifact reduction were not evaluated. Users can control the visibility of the splat representation through the VR interface and compare it with the underlying CAD model. This approach enables the CAD representation to provide the modeled architectural geometry while the Gaussian splat representation captures the observed as-built visual condition of the interior environment, including features such as furniture and subsequent additions that may not be represented in the base CAD model.

Accordingly, the integration of splats was not governed by an automated surface-error threshold or artifact-detection rule. Instead, the CAD and Gaussian splat representations were spatially aligned and made available as alternative or overlaid visual representations that users can selectively display during immersive exploration and comparison. Figure 3 shows an example of the Gaussian splat representation within the building model.

Figure 3. Example of the Gaussian splat representation generated from the interior facility image dataset. The splat asset provides a photorealistic representation of observed interior conditions, including furniture and other visual features that may not be represented in the underlying CAD model. The splat representation can be displayed within the DTIE for comparison with the spatially aligned CAD geometry.

3.3. CAD Model Preparation

The geometric foundation of the digital twin environment was established using facility models authored in Autodesk Revit. The Revit models provide accurate architectural and structural representations of the ERDC buildings, including spatial layout, wall geometry, and interior features. Models were exported from Revit in FBX format, a widely supported interchange format compatible with real-time rendering engines. The exported FBX files were then imported into Unity, where they were prepared for immersive deployment, including material assignment, geometry optimization, and spatial alignment with the corresponding Gaussian splat data.

The two representations (CAD + Gaussian splats) therefore serve complementary purposes within the DTIE rather than functioning solely as alternative visualization styles. The CAD representation is most appropriate when users require access to the modeled architectural geometry and spatial organization of the facility, whereas the Gaussian splat representation provides a visually detailed record of conditions observed during image capture, including objects or modifications that may not be represented in the underlying CAD model. Displaying the representations individually or in spatial alignment provides a mechanism for visual comparison between modeled and observed conditions. In the present implementation, however, discrepancies are identified through user inspection rather than automated geometric comparison or change-detection algorithms. Accordingly, the DTIE enables visual comparison but does not quantify dimensional deviations or automatically classify differences between the CAD and Gaussian splat representations.

One potential use case for this complementary representation is facility-condition review. For example, a user could examine the CAD representation to understand the modeled facility configuration and then display or overlay the Gaussian splat representation to visually identify observed conditions that differ from or are absent in the base model, such as furniture, equipment, or subsequent facility additions. Such differences could identify locations warranting further inspection or model updating. This workflow represents a potential application of the DTIE architecture rather than a validated change-detection capability; the present study did not evaluate discrepancy-detection accuracy or the effect of representation comparison on engineering decisions.

3.4. VR Environment Implementation

The finalized digital twin environment, consisting of the Revit-derived FBX geometry and spatially aligned Gaussian splat representation, was implemented using Unity 6 and the Universal Render Pipeline (URP). The project was based on the Unity 6 VR Multiplayer Template and used OpenXR for extended-reality device integration. Development targeted the Meta Quest 2 head-mounted display, with testing performed through both Meta Quest Link and Air Link.

The application was configured for a 72 Hz headset display refresh rate. The measured per-eye render resolution was 2080 × 2096 pixels at a URP render scale of 1.0. The URP “Balanced” quality tier was used, with 4× multisample anti-aliasing (MSAA), a 15 m shadow distance, 2048-pixel shadow-map resolution, and high dynamic range (HDR) rendering disabled. Vertical synchronization (VSync) was disabled, and the Unity target frame rate was left uncapped.

Users interact with the environment through the Quest 2 hand controllers, which support six-degrees-of-freedom movement and ray-cast interaction with virtual interface elements. The resulting configuration was used to support immersive navigation, visualization of the combined CAD and Gaussian splat representations, multi-user functionality, and interaction with the DTIE interface. The runtime performance of this configuration was subsequently characterized to determine whether the integrated environment could maintain real-time operation on the target hardware.

3.5. Runtime Performance Characterization

Runtime performance of the implemented DTIE was measured using the configured Meta Quest 2 deployment environment. The application was operated at a headset refresh rate of 72 Hz with a per-eye render resolution of 2080 × 2096 pixels and a URP render scale of 1.0. The Unity URP “Balanced” quality configuration used 4× multisample anti-aliasing, a 15 m shadow distance, 2048-pixel shadow maps, HDR disabled, VSync disabled, and an uncapped target frame rate.

Under this configuration, the DTIE achieved an observed frame rate of approximately 71.8 FPS in XR mode, closely matching the configured 72 Hz Quest 2 display refresh rate. For comparison, the same application achieved approximately 386.3 FPS in desktop mode. These measurements demonstrate that the implemented CAD/Gaussian-splat environment could be operated in real time under the reported hardware and rendering configuration. However, the study did not include a controlled performance comparison between CAD-only, Gaussian-splat-only, and hybrid rendering conditions; therefore, the reported frame-rate measurements should be interpreted as implementation-performance characterization rather than evidence that one reconstruction method computationally outperforms another.

The reported Gaussian splat characteristics also provide implementation-scale context for interpreting the runtime results. The integrated representation contained 761,250 Gaussian primitives and was stored as a compressed Unity asset of approximately 35.1 MB using Norm11 position/scale encoding, Norm8 × 4 color encoding, and Norm6 spherical harmonics compression. The 35.1 MB value represents compressed on-disk asset size and should not be interpreted as runtime memory consumption. Runtime CPU/GPU memory utilization, asset loading time, and frame-time variability were not recorded during the present implementation. In addition, the study did not systematically vary splat count, compression level, or scene complexity. Consequently, the observed 71.8 FPS demonstrates real-time operation of the reported configuration but does not establish the relationship among splat complexity, compression, memory requirements, loading time, visual quality, and rendering performance. Controlled evaluation across multiple splat-complexity and compression conditions is required to characterize these trade-offs.

3.6. UI Development

The user interface (UI) for the virtual reality (VR) environment was designed to support interaction, data accessibility, and collaborative functionality within the Digital Twin Immersive Environment (DTIE). The UI was implemented as a spatially anchored, panel-based interface that appears as a flat, tablet-like surface within the virtual environment. When activated, the panel is positioned approximately five feet in front of the user, consistent with mid-depth menu placement approaches identified in prior VR interaction research.

Interaction with the UI is facilitated through a ray-casting input method, shown in Figure 4. Using the Meta Quest 2 hand controllers, users project a virtual ray onto the interface to select menu elements and access system functions while remaining immersed within the facility environment. The panel-based design was selected to retain visual and organizational characteristics familiar from conventional tablet-style interfaces while adapting them for three-dimensional interaction.

Figure 4. Ray-casting interaction with the spatially anchored DTIE panel interface. The panel appears approximately five feet in front of the user, and the Meta Quest 2 controller projects a virtual ray onto the interface for menu selection. This interaction method provides access to DTIE functions while allowing the user to remain immersed within the facility environment.

Several additional capabilities were incorporated into the UI to support multimodal interaction and information access. A speech-to-text feature provides an alternative to manual text entry and allows users to enter information while remaining within the immersive environment. The system also supports multi-user interaction, enabling multiple participants to simultaneously access and interact with the same digital twin environment and providing a shared context for collaborative review.

To support comparative visualization, the UI allows users to toggle between the computer-aided design (CAD) model and the Gaussian splat representation of the digital twin. This capability enables users to examine the modeled architectural geometry and the visually captured as-built condition within the same immersive environment. The two representations may also be viewed in combination to support comparison of current interior conditions, including furniture or subsequent additions, with the underlying CAD geometry.

An in-scene web browser was also integrated into the interface, allowing users to access external information, documentation, and other data sources without leaving the VR environment. Figure 5 illustrates the web-browser and speech-to-text capabilities incorporated into the DTIE.

Figure 5. Integrated spatial menu for accessing in-environment web-browser, comment system, and speech-to-text interface capabilities within the DTIE. These functions allow users to access external information and provide text input without leaving the immersive environment, illustrating the integration of information-access and multimodal interaction functions within the VR digital twin.

The UI also provides access to a dynamic occupant movement simulation for visualizing pedestrian activity within the facility. Occupant movement is implemented using a graph-based navigation structure composed of authored ingress and movement nodes distributed throughout the virtual environment. Node spacing is not defined by a fixed global parameter; instead, nodes are placed by the developer or user during scene authoring or at runtime according to the spatial configuration and desired movement paths within the facility.

Each occupant is assigned a walking speed at spawn, sampled from a uniform distribution ranging from 1.10 to 1.50 m/s, corresponding to a mean of approximately 1.30 m/s. Occupants transition sequentially between connected path nodes according to the authored navigation graph.

Occupancy at a node is defined according to the occupant’s simulation state. A node is considered occupied while an occupant is in the Dwelling state. Occupancy begins when the occupant physically reaches the node and transitions from Transiting to Dwelling. The occupant remains at the node until the sampled dwell duration expires, at which point the simulation transitions the occupant from Dwelling to Departing. If an alarm condition occurs during the dwell period, the normal dwell sequence can be interrupted and the occupant instead transitions from Dwelling to Evacuating.

Dwell duration is not fixed globally. For each visit, dwell time is sampled from a distribution defined for the individual node using its specified mean, standard deviation, minimum, and maximum bounds. Consequently, occupancy duration can vary both among nodes and across repeated visits to the same node.

At branching locations containing multiple candidate onward nodes, occupancy information influences route selection. Agents preferentially select available nodes that are not currently occupied by another dwelling occupant. When multiple viable onward nodes remain, the next destination is selected from the available alternatives, producing variation in movement patterns between simulation runs. Occupants reaching the terminal extent of an authored route may reverse direction and continue traversing the graph, supporting continuous bidirectional movement throughout the simulated environment.

The occupancy simulation was configured to represent approximately 650 simultaneous occupants, based on an assumed facility density of approximately one person per 15 m², with a hard population limit of 800 occupants. The simulation functions as a rule-based visualization layer within the DTIE and is intended to support exploration of hypothetical occupancy distributions, occupant movement patterns, and scenario behavior. It is not a validated pedestrian-flow prediction model, and the generated movement patterns were not calibrated or validated against observed pedestrian trajectories or facility occupancy data. Accordingly, the simulation outputs should be interpreted as synthetic, scenario-based occupant movement rather than predictions of actual facility behavior.

3.7. Reproducibility and Data-Access Constraints

The DTIE development workflow was documented to enable reproduction of its core non-sensitive technical components using alternative facility data. The image-acquisition device and resolution, Pix4Dcatch version and capture approach, photogrammetric processing configuration, Gaussian splat characteristics, Unity/XR configuration, rendering settings, and occupant simulation logic are reported in the preceding sections.

However, the facility imagery, LiDAR-associated capture data, CAD models, reconstructed point cloud, Gaussian splat asset, and facility-specific Unity environment used in this study cannot be publicly distributed because they are associated with a restricted U.S. government facility and are subject to International Traffic in Arms Regulations (ITAR), contractual restrictions, and project-specific data-access requirements. No public source-code repository is currently available for the DTIE implementation.

Accordingly, reproducibility of the study is supported through disclosure of the non-sensitive development workflow and implementation parameters rather than release of the underlying facility-specific data and software assets. Researchers may replicate the described workflow using unrestricted imagery, facility models, and equivalent commercial or open reconstruction and VR development tools.

4. Discussion

The development of the Digital Twin Immersive Environment (DTIE), including both the reconstruction pipeline and user interface (UI) design, aligns closely with trends and gaps identified in the existing literature on virtual reality (VR) systems and digital twin applications. As highlighted in prior work, VR interface design requires balancing immersion, usability, and ergonomic interaction, particularly in environments where users must interact with complex spatial data [2,3]. The DTIE was designed in response to these challenges through a hybrid reconstruction approach and a structured panel-based UI informed by prior VR interaction research.

From a reconstruction perspective, the integration of CAD-based geometry with Gaussian splatting reflects emerging approaches in digital twin generation. Prior research has reported computational advantages of Gaussian splatting for rendering visually detailed environments [30]. Within the DTIE implementation, the Gaussian splat asset contained 761,250 primitives with a compressed on-disk size of approximately 35.1 MB, and the combined CAD/Gaussian-splat environment operated at approximately 71.8 FPS in XR under the configured 72 Hz Meta Quest 2 environment. These measurements characterize the scale and real-time performance of the implemented configuration. However, runtime memory consumption and loading time were not measured, and splat count, compression level, and representation condition were not systematically varied. The present results therefore demonstrate compatibility with real-time immersive deployment under the reported configuration rather than computational superiority or an optimized trade-off between visual quality and computational cost.

This hybrid approach is consistent with recent studies applying Gaussian splatting to the representation of buildings, infrastructure, and complex systems [31,32,33,34,35]. By integrating a Gaussian splat representation with CAD-based geometry in a VR-ready environment, the DTIE extends this body of work toward immersive interaction with complementary facility representations. The present results demonstrate the technical feasibility of this integration but do not establish that the hybrid representation provides superior reconstruction fidelity or rendering performance relative to CAD-only, mesh-based, or Gaussian-splat-only alternatives.

The UI design implemented within the DTIE system also reflects key findings from the VR menu usability literature. The use of a panel-based interface positioned at a fixed distance in front of the user aligns with research indicating that mid-depth menu placement improves ergonomic comfort and reduces fatigue during extended interactions [20]. Additionally, the ray-casting interaction method used to select UI elements is consistent with established best practices for VR interaction, as it supports precise selection while maintaining spatial awareness [17,20]. These design decisions directly address challenges identified in prior studies, including difficulties with menu selection, spatial disorientation, and cognitive overload in immersive environments [9].

The choice of a flat, tablet-like panel interface further aligns with findings suggesting that familiar UI metaphors can improve usability, particularly for users with limited prior VR experience [2]. While radial and alternative menu systems have been shown to offer advantages in certain contexts [15], the literature also indicates that panel-based interfaces can provide improved learnability and user comfort due to their similarity to traditional computing interfaces [2]. By adopting a panel-based approach, the DTIE system prioritizes usability and accessibility while maintaining compatibility with immersive interaction techniques.

The integration of speech-to-text input, multimodal interaction, and in-environment data access reflects broader trends in VR interface development. Prior research has identified the potential of multimodal interaction systems, including voice, gesture, and gaze, to improve interaction naturalness and reduce cognitive load in VR environments [12,13,14]. Within the DTIE, speech-to-text functionality was implemented as an alternative to manual text entry, while the in-scene web browser provides access to external information without requiring users to leave the immersive environment. The present study demonstrates the implementation of these capabilities but does not evaluate their effects on interaction efficiency, cognitive load, or user performance.

The DTIE system also incorporates collaborative and scenario-based capabilities that align with the growing body of literature on immersive digital twin environments. Multi-user functionality enables multiple users to occupy and interact within a shared virtual environment, providing a technical foundation for future collaborative review applications [25]. Similarly, the ability to toggle between CAD and Gaussian splat representations allows users to examine complementary representations of the facility within the same environment. The rule-based occupant movement simulation provides a scenario-oriented mechanism for visualizing synthetic occupant activity within the reconstructed facility. Movement is generated according to the predefined graph structure, walking-speed distributions, dwell-time parameters, branching logic, and occupancy rules described in Section 3.6. The simulation is integrated into the DTIE as an interactive visualization layer for exploring hypothetical occupancy scenarios rather than as a predictive component of the digital twin. Because the movement model was not validated against observed pedestrian trajectories or facility occupancy data, the resulting movement patterns should not be interpreted as predictions of actual pedestrian behavior, flow, or facility operations. Future validation would be required before using the simulation to support operational planning or facility-management decisions.

The ability to display CAD and Gaussian splat representations individually or in spatial alignment provides a mechanism for visually comparing modeled architectural information with observed facility conditions. This complementary representation could support future workflows for identifying changes or conditions that warrant inspection or model updating. However, the present implementation relies on visual inspection and does not perform automated discrepancy detection, geometric deviation measurement, or validation of resulting engineering decisions.

The broader implications of this work are closely aligned with research examining the impact of immersive digital twin review processes. Prior studies have demonstrated that immersive VR environments can improve spatial understanding, collaboration, and training outcomes while enabling earlier validation of design decisions [25,26,27,28,29]. The DTIE system extends these capabilities by providing an integrated platform that combines real-time interaction, immersive visualization, and simulation within a single immersive environment. At the same time, the system reflects challenges identified in the literature, including the need to manage complexity, ensure usability, and balance visual detail with performance constraints.

An important contribution of the DTIE is therefore not any individual reconstruction or interaction technology in isolation, but the integration of these technologies into a unified immersive facility digital twin framework. Existing work reviewed in this study demonstrates Gaussian splatting for building, campus, infrastructure, and articulated-system digital twins [31,32,33,34,35], while separate bodies of research have examined VR menu design and immersive digital twin interaction [2,3,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29]. The DTIE bridges these areas by combining CAD-based structural geometry, Gaussian splat-based photorealistic representation, immersive VR interaction, collaborative functionality, information access, and dynamic occupant simulation within the same facility-scale virtual environment. Importantly, the ability to transition between CAD and Gaussian splat representations allows the environment to preserve access to both geometric and visually realistic views rather than requiring a single representation to satisfy all visualization needs. In this sense, the DTIE expands the role of immersive facility digital twins beyond static model viewing toward an interactive virtual-world infrastructure capable of supporting multiple representations, user interaction, collaboration, information access, and scenario visualization. This integration represents the principal contribution of the present work to immersive digital twin and virtual environment research.

Practical and Strategic Implications for Infrastructure Management

The practical significance of the DTIE extends beyond the demonstration of individual visualization and interaction functions. Within the U.S. Army Corps of Engineers (USACE), infrastructure design and management are distributed across many locations, with approximately 150 new design projects managed annually. At the same time, detailed three-dimensional models developed during design are frequently reduced to two-dimensional representations as projects transition into construction, creating a potential discontinuity in how digital information is retained, communicated, and used throughout the facility lifecycle. The DTIE framework addresses this broader challenge by maintaining facility information within an immersive three-dimensional environment that can support visualization, interaction, collaboration, and scenario exploration across project phases.

For geographically distributed teams, a shared immersive digital twin can provide a common spatial context in which engineers, facility managers, designers, contractors, and other stakeholders review the same facility representation rather than relying exclusively on separate drawings, models, and supporting information. The multi-user capability implemented in the DTIE allows multiple participants to access and interact with the same digital twin environment, while the in-scene browser and speech-to-text functions provide mechanisms for accessing and entering information without leaving the immersive workspace. The ability to switch between the CAD and Gaussian splat representations further enables users to compare the modeled architectural geometry with the visually captured as-built condition of the facility. Together, these capabilities establish a technical foundation for reducing fragmentation between design representations, observed facility conditions, and collaborative review activities. The present study does not quantitatively measure reductions in information silos or project-review time; rather, it demonstrates the integrated capabilities needed to investigate these outcomes in future operational deployments.

The framework also has implications for full-lifecycle facility management. The manuscript identifies mirroring, monitoring, and simulation as three important digital twin capabilities. Mirroring enables stakeholders to interact with a virtual representation of infrastructure; monitoring can connect operational data with the digital model; and simulation can provide a virtual environment for examining potential incidents, emergency procedures, safety systems, and infrastructure modifications. By maintaining an immersive three-dimensional environment across these activities, the DTIE provides a pathway for digital models created during design to remain useful beyond the initial design phase. For an organization managing a large annual portfolio of infrastructure projects, such continuity could support more consistent design review, facility familiarization, collaborative planning, operational assessment, and scenario-based analysis over the life of an asset.

These capabilities also align with the broader modernization and resilience objectives identified in the U.S. Army Installations Strategy. The manuscript notes that this strategy emphasizes readiness and resilience through technological innovation and improved operational insight, and identifies advanced visualization, simulation, and collaborative decision-making as mechanisms through which immersive digital twins can contribute to modernization of installation operations. In this context, the value of the DTIE is not limited to creating a more visually detailed virtual facility. Its broader role is to provide an extensible immersive infrastructure in which geometric models, visually captured conditions, operational information, collaboration tools, and scenario simulations can eventually be brought together to support a more persistent digital representation of the facility throughout its lifecycle.

Although the prototype was developed around a USACE facility, the underlying framework is not inherently limited to military infrastructure. The combination of CAD geometry and Gaussian splatting is particularly relevant to large, complex facilities in which an engineered geometric representation and the observed as-built environment both provide useful but different information. In principle, the same architecture could be transferred to other large public infrastructure environments, such as airport terminals, industrial campuses, or culturally significant facilities, by replacing the facility-specific CAD, imagery, and operational data while retaining the core reconstruction, visualization, interaction, collaboration, and simulation framework. In an airport terminal, for example, the approach could support comparison of engineered layouts with current interior conditions and scenario visualization of occupant movement. In a large industrial park, it could provide a common immersive environment for examining spatial relationships among facilities and supporting distributed review activities. For cultural heritage applications, the visually rich splat representation could complement structured geometric models by preserving the observable appearance of complex spaces for immersive review. These examples represent potential extensions of the framework rather than applications validated in the present study.

The broader research contribution is therefore the development of an immersive digital twin architecture that can bridge traditionally separate forms of infrastructure information and user interaction. By combining CAD-based structural representation, visually rich Gaussian splatting, multi-user VR interaction, information access, and scenario simulation within a single environment, the DTIE establishes a foundation for investigating how persistent immersive digital twins can support distributed collaboration and lifecycle-oriented infrastructure management at scales extending beyond the individual prototype demonstrated in this study.

Overall, the DTIE implementation demonstrates how digital twin reconstruction and VR interface technologies can be integrated within a common immersive facility environment. The interface design was informed by established VR interaction research, while its usability and effects on user performance remain to be evaluated empirically. Accordingly, this work provides a foundation for future research examining usability, adaptive interfaces, multimodal interaction, collaborative review, and scalable immersive digital twin systems.

5. Limitations

The present study focused on the development and technical implementation of the DTIE prototype rather than on controlled comparative validation of reconstruction quality or formal human-subject usability evaluation. Accordingly, several limitations should be considered when interpreting the findings.

First, although the Gaussian splat representation provided a visually detailed representation of the captured facility interior and was successfully integrated with the CAD model, the study did not perform a controlled comparison between the hybrid representation and a conventional mesh-only reconstruction. Objective image-quality measures such as peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM), as well as quantitative measures of surface-artifact reduction, were therefore not calculated. The contribution of the present work is consequently limited to demonstrating the implementation and integration of the hybrid representation rather than establishing its quantitative superiority over alternative reconstruction approaches.

Second, the panel-based VR interface was informed by previously published findings concerning VR menu placement, ray-casting, interface familiarity, and multimodal interaction, but the DTIE interface itself was not evaluated through a formal human-subject usability experiment. No System Usability Scale (SUS), NASA Task Load Index (NASA-TLX), task-completion time, error-rate, or user-satisfaction data were collected for the DTIE prototype. Statements regarding usability in this manuscript therefore refer to design intent and implementation characteristics, rather than experimentally demonstrated improvements in user performance or workload.

Future validation should address these limitations through two complementary experimental studies. Reconstruction performance should be evaluated using standardized image- and geometry-based metrics under CAD/mesh-only, Gaussian-splat-only, and hybrid conditions. In parallel, a structured VR usability study should recruit representative users, including facility managers and engineers with varying levels of prior VR experience, to complete standardized DTIE tasks. Such an evaluation could incorporate task-completion time, selection error rate, SUS, NASA-TLX, and subjective preference measures. These experiments would enable quantitative assessment of both the reconstruction approach and the human–computer interaction design beyond the system-development results presented here. Future computational benchmarking should additionally evaluate multiple splat-count and compression conditions while recording asset loading time, CPU/GPU memory utilization, frame-time variability, and rendering performance to characterize the trade-offs between representation complexity, storage requirements, visual quality, and runtime performance.

A further limitation concerns the scope of the validation environment. The current DTIE implementation was developed and evaluated using a single interior facility dataset captured at ERDC with an iPhone 14 Pro and deployed primarily on the Meta Quest 2. Consequently, the present study does not establish robustness across facilities of substantially different scale, architectural complexity, lighting conditions, capture hardware, or XR platforms. Similarly, although the DTIE supports multi-user interaction, formal stress testing under systematically varied numbers of concurrent users was not conducted. The results should therefore be interpreted as evidence of technical feasibility within the reported implementation environment rather than evidence of general performance across heterogeneous infrastructure and hardware conditions.

Future evaluation should adopt a multi-environment validation design incorporating facilities that differ in spatial scale, lighting, geometric complexity, and surface characteristics; multiple capture devices and XR hardware configurations; and controlled multi-user load conditions. Such testing would allow assessment of reconstruction robustness, rendering scalability, network performance, interaction responsiveness, and system stability under conditions representative of broader infrastructure applications.

6. Conclusions

This study presented the development of a Digital Twin Immersive Environment (DTIE) that integrates CAD-based geometry with Gaussian splatting reconstruction and an interactive virtual reality (VR) interface to support visualization, analysis, and collaboration within facility digital twins. By leveraging Autodesk Revit for architectural modeling, Pix4Dcatch and Pix4D Cloud for image acquisition and photogrammetric reconstruction, and a separate Gaussian splat generation process, the research team developed complementary geometric and photorealistic representations of the ERDC facility. These representations were spatially aligned within Unity and successfully deployed within the DTIE, which operated at approximately 71.8 FPS under the reported Meta Quest 2 configuration.

In addition to the reconstruction pipeline, this work emphasized the importance of user interface (UI) design in immersive environments. The DTIE system, developed in Unity Engine 6000.3.8f1 and deployed on the Meta Quest 2, implemented a panel-based UI combined with ray-casting interaction, speech-to-text input, and multi-user collaboration features designed to support interaction within the immersive environment. These design choices were informed by existing VR usability research, which highlights the importance of ergonomic menu placement, familiar interface structures, and multimodal interaction for improving user performance and reducing cognitive load [2,9,20]. The integration of additional capabilities, including toggling between CAD and Gaussian splat representations, in-scene web browsing, and a node-based occupant movement simulation with branching paths and occupancy-aware routing, demonstrates the technical integration of multiple visualization, information-access, and scenario-exploration functions within a common immersive environment.

The principal contribution of this work is the demonstration of an integrated approach to immersive facility digital twin development in which structurally precise CAD geometry, photorealistic Gaussian splatting, and immersive human–computer interaction are incorporated within a unified VR environment. The DTIE demonstrates that these components can function as complementary layers of an immersive facility representation rather than as independent visualization technologies. By incorporating multi-user interaction, multimodal data entry, in-environment information access, model-representation switching, and dynamic occupant simulation, the prototype establishes a framework for facility digital twins that extends beyond visualization toward interactive and collaborative virtual-world applications. The DTIE platform therefore provides a technical foundation for investigating whether immersive facility digital twins can improve spatial understanding, enhance collaboration, and support scenario-based analysis. These potential outcomes were not empirically evaluated in the present study and remain subjects for future user-centered validation [25,26,27,28,29]. At the same time, this work acknowledges ongoing challenges related to system complexity, data integration, and scalability that must be addressed to enable broader adoption of immersive digital twin technologies.

The present findings should be interpreted as a demonstration of technical feasibility and system integration; controlled reconstruction benchmarking, formal usability evaluation, and validation across additional facilities and hardware platforms remain important areas for future research.

Future research should explore the integration of adaptive and intelligent interface systems that dynamically respond to user behavior, as well as expanded multimodal interaction techniques incorporating gesture, eye tracking, and haptic feedback [11,12,13,14,24]. Additionally, further investigation is needed to evaluate the usability and effectiveness of immersive digital twin systems across diverse user populations and application domains, particularly with respect to accessibility and inclusivity.

Beyond the prototype implementation, the DTIE provides a foundation for persistent, lifecycle-oriented infrastructure digital twins. For organizations such as USACE that manage geographically distributed teams and large portfolios of infrastructure projects, the framework provides a common immersive environment through which modeled geometry, visually captured facility conditions, information resources, collaborative interaction, and scenario visualization can be brought together. This architecture has potential relevance to reducing information fragmentation across project phases and supporting continued use of three-dimensional facility information from design through operation. The same framework may also be adapted to other complex public infrastructure domains, including transportation facilities, industrial campuses, and culturally significant sites, where both engineered geometry and current as-built visual conditions are important to collaborative review and facility management. These broader applications require domain-specific validation but demonstrate the extensibility of the DTIE beyond the facility examined in this study.

Overall, the DTIE provides a foundational framework for investigating the integration of digital reconstruction, visualization, and interaction within immersive facility digital twins. The present study establishes the technical feasibility of the implemented architecture, while future user-centered evaluation is needed to determine its effectiveness for engineering analysis, facility management, collaborative review, and decision-support applications.