A Review of Collaborative Virtual Reality Systems for the Architecture, Engineering, and Construction Industry

Abstract

In this paper, we focus on interdisciplinary collaboration using intuitive virtual reality interfaces and building information models in the architecture, engineering, and construction industries. These systems have been a topic of research and development for the past ten years; however, there is still no widely open standard format, related software platform, or guidelines that are sufficiently mature; the complexity of such systems is very high. We review existing state-of-the-art interdisciplinary collaborative virtual reality systems, proposing solutions and standards. Thirteen state-of-the-art systems are reviewed and compared to illustrate emerging trends and insufficiencies. It is found that these systems differ significantly with respect to drawing capabilities, photorealism, construction simulation, and interdisciplinary communication. We discover trends in user interfaces that could be evolved to better standards, and provide future guidelines to developers. Combining the best aspects of existing systems, we provide a blueprint for an ideal system that combines the most advanced features for collaborative design.

1. Introduction

Advances in information and communication technology affect many aspects of the architecture, engineering, and construction industry (AEC). New tools and infrastructures improve practices and achieve better results. For example, building information modeling, often known as BIM, introduced by Van Nederveen and Tolman in 1992, drastically transformed the industry [1]. Recent innovations in virtual reality technology (VR) are paving the way for new “virtual” working environments and changing the way engineers work. For example, high-resolution headsets achieve photorealistic experiences [2], and advanced graphics engines and graphics cards offer optimized performance without sacrificing detail. However, VR has been slow to penetrate the AEC industry, as noted by [3]. Our goal is to collect scattered efforts, compare the features of each approach, and track barriers, so as to provide instructions for improving collaborative systems.

In Section 1 of this paper, we present the background processes currently followed in the AEC industry, as well as advantages and disadvantages regarding VR use as noted in the literature. In Section 2, we present existing VR solutions and perform a comparison. In Section 3, we describe the ideal VR system. Finally, in Section 4, conclusions are derived and the future of VR is discussed. In the end, we provide a table explaining acronyms for the reader’s convenience.

1.1. Background

To enhance collaboration among different disciplines and expedite information exchange and outsourcing procedures, the industry takes advantage of several technological advancements. Computer-aided design (CAD) constitutes an indicative example of such an advancement that is widely employed. CAD is a form of computer-assisted building modulation that architects, engineers, and contractors use to design and depict two- and three-dimensional models. A similar technology recently introduced in the AEC sector, revolutionizing the industry, is computer-aided engineering (CAE). This term encompasses finite element analysis (FEA), computational fluid dynamics (CFD), and multibody dynamics (MBD) for durability and optimization.

Although all aforementioned technological advancements played a critical role in the development of the industry, the need for further interdisciplinary collaboration led to the development of more efficient and sophisticated standards that oversee the process of creating and managing a digital model of a building. Therefore, BIM was created. BIM is an intelligent, model-based process that connects AEC professionals, allowing them to design, build, and operate buildings and infrastructure more efficiently [4]. Using BIM, designers enhance digital 3D models that include data associated with physical and functional characteristics. The data in a model define design elements and establish behaviors and relationships between model components.

There are a number of BIM “levels of maturity”, as shown in Figure 1. Level 0 describes unmanaged CAD, Level 1 describes managed CAD in 2D or 3D, and Level 2 involves developing building information in a collaborative 3D environment with data attached, but created in separate discipline models (often called closed BIM). Level 3 BIM, also known as iBIM or OpenBIM, refers to collaborative construction sequencing and cost and life-cycle management information [5]. OpenBIM extends the benefits of closed BIM by improving the accessibility, usability, management, and sustainability of digital data. At its core, OpenBIM is a collaborative process that is vendor-neutral. OpenBIM processes can be defined as sharable project information that supports seamless collaboration for all project participants. However, OpenBIM has several unresolved issues such as model ownership, design responsibility, lack of standardization, and liability in interoperation among partners [6]. The Industry Foundation Classes format (IFC) is a first step towards standardization; however, it does not encompass the entire collaboration process, as it is a standard for the final submission of a project which is not reversible and functional for each collaborator [4]. There is a need for more advanced formats and software platforms that allow seamless collaboration.

1.2. Advantages and Disadvantages of VR

VR has several advantages, but it also has many disadvantages, as noted in [3,7].

1.2.1. Advantages

Visualization and Drawing: The most prominent characteristics of VR in this category are:

(a)

Interactivity: users can interact with models;

(b)

Spatiality: models are represented in three spatial dimensions;

(c)

Immediacy: real-time feedback from actions is given without noticeable pause;

Telepresence: the virtual presence of collaborators in a VR environment eliminates the need for their physical presence. A system that relies on VR technology can enhance training speed and quality, be pandemic immune, and reduce travel costs for meetings;

Simulation: on-site task simulation can address a number of issues during construction, such as the feasibility of operations.

1.2.2. Disadvantages

BIM model preparation: It takes extra work and time to visualize a BIM model in VR; incompatibilities such as the lack of common 3D data structures and material systems across existing software are the main cause.

Adaptation to VR interfaces: VR controllers and hand gestures provide new interfaces that are often different across VR vendors and cause cognitive stress.

2. Existing VR Solutions for the AEC Industry

We reviewed mature commercial VR solutions in order to derive useful conclusions. The development of VR systems shifted to certain companies as it is not easy for academia to engage in such a demanding task; therefore, no open-source solution exists yet as a baseline. In Section 2.1, we present existing VR solutions, in Section 2.2 we outline interconnection solutions for VR with existing CAD/CAE/BIM systems, and in Section 2.3 we provide an analysis of the features of each VR and interconnection solution.

2.1. Existing VR Solutions

In

Table 1. Existing software solutions in the AEC industry.

#	Name	Discipline	Type	Interoperability—I/O Formats	Features
1	Arkio https://www.arkio.is (accessed on 7 June 2022)	Architects (VR design)	Standalone	Import and export to Revit and Rhino in real-time. Multi-user support.	Support of VR. Real-time editing. No textures support.
2	Enscape https://enscape3d.com (accessed on 7 June 2022)	Architects (Visualization)	Standalone	Revit input, 2D graphics export for concepts.	Messaging and photorealistic rendering.
3	Fuzor https://www.kalloctech.com/ (accessed on 7 June 2022)	All Disciplines (VR and interconnection software)	Standalone	Revit and Rhino interconnect in VR. Live sync.	Integrates all disciplines into a VR environment.
4	Gravity Sketch https://www.gravitysketch.com/ (accessed on 7 June 2022)	Architects (Design in VR)	Standalone	Import and export to Rhino in real-time.	Support of VR. Real-time editing. No textures support. Multi-user.
5	Holodeck Nvidia www.nvidia.com/en-us/design-visualization/technologies/holodeck/ (accessed on 7 June 2022)	Architects (Design in VR)	Standalone	Import: 3Ds Max or Maya, Export: plug-in to 3Ds Max or Maya. NVidia Iray is required.	Only for standard NVIDIA vMaterials. No connection with Omniverse yet.
6	IrisVR—The Wild https://irisvr.com/ https://thewild.com/ (accessed on 7 June 2022)	Architects (Design in VR)	Standalone	Connectors with Rhino, Revit, Navisworks, and Sketchup.	Multi-user support.
7	Mindesk https://mindeskvr.com/ (accessed on 7 June 2022)	Architects (Design in VR)	Plugin	Rhino, Revit, Solidworks, and Unreal connectors.	No web interface or database support.
8	TwinMotion https://www.unrealengine.com/en-US/twinmotion (accessed on 7 June 2022)	Architects	Standalone	Improves the realism of Revit and Rhino projects.	Support for VR. Exploit real-time tracing. Change materials and add decorations.

, we present several VR solutions. They are explained in great detail in the following paragraphs.

Arkio is a drawing VR software [8]. It covers a wide variety of drawing actions through its radial menu which is located on the left wrist (Figure 2a), but it can be anchored wherever in space (Figure 2b). A band on the wrist indicates the current selected drawing action. The radial tool allows object drawing (Figure 2c) as well as object coloring (Figure 2d), but it does not go further into highly realistic material properties and ray-tracing. Another innovative interface is the zooming, scaling, and rotating interface which is based on the distance of two hands, e.g., by holding the left and right grab buttons and lowering or increasing the distance, the zoom goes up or down (Figure 2e). A virtual camera can be used to take screenshots (Figure 2f) and a 3D slider can be used to change the time of day in order to see the shadows’ position (Figure 2g). It can import and export files for Rhino, Revit, Sketchup, and BIM360, but it is not live interconnected.

Enscape targets real-time realistic rendering. It also provides plugins for Revit, SketchUp, Rhino, Archicad, and Vectorworks for “Live Updates”. It aims to improve workflow among architects and clients by presenting photorealistic renderings of the design. One of its major disadvantages is that it does not support multi-user capability in real-time. It is based on an in-house graphics engine. Its advantages are that it can extend the materials library of Revit and also simulate daylight at any time point. The right controller can be used for teleporting, i.e., the instant change of view to a location without intermediate rendering of walking (Figure 3a). The left controller has an attached menu as shown in Figure 3b that can be used for various modes such as seated, room walking, walking, and flying. In Figure 3c, the presentation mode for teleporting to specific positions can be seen. In Figure 3d, a mini-map for teleporting is demonstrated. In Figure 3e, screenshot capturing and image presentation functionalities are shown. In Figure 3f, interconnectivity with Revit is demonstrated. Changing material into white clay, showing object outlines, and time-changing interfaces are demonstrated. The controllers can also be used for walking or flying in 3D, but this is not advised as it may cause VR sickness.

Fuzor is a multiuser VR software focusing on construction simulation. It can represent the construction site in VR together with vehicles, e.g., it allows for positioning building machines such as trucks and cranes. It can be interconnected with Revit, ArchiCAD, Rhino, and Navisworks through Live Link, which does not require file synchronization. The synchronization is bidirectional, i.e., changes can pass from one software to the other. It has multi-user presence capability. Another capability is to represent 4D scenes, i.e., the construction process through time. Its interfaces are shown in Figure 4. The main menu allows one to select between walk and fly modes (Figure 4a). Navigation is based on teleporting with the right controller (Figure 4b). The “Render” mode allows changing the daytime (Figure 4c). The “Pipes” menu allows drawing pipes by first selecting their diameter and then dragging the left controller (Figure 4d,e). The render modes menu also allows placing pre-built 3D items in space, as shown in Figure 4f. Finally, the elements property menu allows one to see BIM information for an element, as demonstrated in Figure 4g.

Gravity Sketch is a drawing tool for designing complex curves. When a drawing tool is selected, the right controller becomes transparent so as not to hide the drawing, and the drawing obtains several holders with round marks for further editing (Figure 5a,b). The left controller activates the menu (Figure 5c) which has a horizontal bar with design tools. A vertical bar has auxiliary functions such as drawing in a mirrored fashion (Figure 5d). The purple button on the left controller is a shortcut for drawing tools (Figure 5e). The gear icon on the menu can bring up files from the file system as a reference point (Figure 5f). Additional features supported include multiplayer capability, various devices (tablet, desktop) support, import and export models in various formats, and live interconnection with Rhino.

Holodeck allows architects to annotate and reposition objects collaboratively. Its user interface consists of a table, named “HoloTable”, and many hand attached widgets. HoloTable’s main menu contains the tabs “Table, View, Sun Position, and Clip”, as shown in Figure 6a. Various widgets on the left side of the menu allow the scaling of the model, and the positioning of the player inside the model. The “Table” tab contains widgets for changing the properties of the Holotable; namely, its translation, rotation, and vertical displacement. The “View” tab filters parts of the building with switches as shown in Figure 6b. The “Clip” tab contains sliders for the vertical and horizontal cross-sectioning of the 3D model. The hand menu contains widgets, as shown in Figure 6c,d, which contain multiplayer options, measurement tools, viewing angle change options, a materials library, and a screenshot-taking tool. An embodied web browser allows live interconnections with Google Hangouts (Figure 6e). The design tools allow for 2D and 3D sketching as shown in Figure 6f,g, respectively. The materials library contains several options and a palette for choosing existing materials, as shown in Figure 6h,i. Its plugins for 3DS Max and Maya can be used to import models inside VR.

“IrisVR” merged with “The Wild” in February 2021. IrisVR—The Wild is a VR software mainly targeting multi-discipline review and collaboration within VR. Its UI has several tools such as (a) the “Inspection” tool (Figure 7a), with the capability to view information fetched from Revit; (b) the “Viewpoints” tool (Figure 7b) which allows teleporting to certain points in space; (c) the “Layers” tool, which allows changing the visibility of Rhino layers (Figure 7c); (d) the “Markup” tool that allows annotating the model with 3D brushes (Figure 7d); (e) the “Screenshots” tool to take screenshots with a camera preview board (Figure 7e); and (f) the “Sun Settings” tab for changing the time of day and date to observe the lighting effect on the 3D model and the position of its shadow (Figure 7f). Iris VR has an in-house developed plugin named Prospect that can import models directly from Rhino, Revit, NavisWork, BIM360, and Sketchup.

The logic behind Mindesk is inversely set, as opposed to other VR solutions. Instead of having one standalone software interconnected with Rhino, Revit, and Unreal Engine, we have in this case three VR plugins, one for each platform, which can communicate with each other. The main plugin is the Rhino plugin which contains an extensive set of tools such as Grasshopper tools (Figure 8a) and several polysurfaces and NURBS tools (Figure 8b) able to exploit Rhino capabilities in advanced curves design. The Revit plugin has a basic interface where (as with much any other software) zooming is controlled by holding the grab buttons and approximating the controllers (Figure 8c). The main menu consists of five tools: “Selection”, “Screenshot”, “Inquire”, “Ruler”, and “Pen”. The “Inquire” button fetches BIM information from Revit as shown in Figure 8d, whereas “Pen” allows sketching in 3D as shown in Figure 8e. Multi-user capabilities are offered for users on the same local network (LAN). The Mindesk Unreal VR plugin offers affine transformations using live link interconnection to Rhino software. Grasshopper geometries can also be transferred to Unreal. The main advantage of Mindesk is its live interconnection feature, which reflects any change made in one plugin to all the others in real-time without the need of pressing sync buttons or exchanging files.

Twinmotion is a visualization software that allows architects and clients to construct photorealistic representations of buildings, virtual inhabitants, and the surrounding scenery [9]. Its menu is attached to the left controller; the right controller is used for clicking on items or teleporting (Figure 9a). The menu has five options, named “Settings”, “Floor level calibration”, “Weather condition change”, “Presenter”, and “Materials”. The settings option (Figure 9b) allows one to change graphics settings among four presets (Low, Medium, High, and Ultra) depending on the hardware available (Figure 9c). The “Weather” button allows one to change conditions such as hour, month, weather, and season (Figure 9d). The buttons slide up and down to regulate the selected value. The “Materials” button leads to five categories of materials, including glass, metal, concrete, wood, and stone (Figure 9e). When the in-scene model to be edited is clicked, a choice can be made from the available sub-materials, as shown in Figure 9f. The “Presenter” button leads to choices for teleporting (Figure 9g) that were predefined in the 2D interface. Twinmotion has export and import options for Unreal, Revit, Rhino, and other tools, but does not support live changes in real-time or multi-user functionalities.

2.2. Comparison

In this section, we compare VR solutions regarding their main characteristics, namely:

(1)

Scope: which phase of the project they target, and which discipline(s);

(2)

Broadness: how many disciplines they can target;

(3)

Features: the most important features available;

(4)

Multi-user capability: for real-time collaboration;

(5)

Design capability: to design at vertice level (not just simple affine transformations);

(6)

Texture support: support textures rather than only simple colors;

(7)

Interconnectivity: how they are interconnected to other software.

The results can be seen collectively in

Table 2. Capabilities of existing VR software in the AEC industry.

	Capability
Tool	1. Scope	2. Broadness	3. Features	4. Multiuser	5. Design Capability	6. Textures Support	7. Interconnectivity Type
Arkio	Design	Architecture	Design, Nurbs, Shadows change	🗸	🗸	✗	File Sync, Export/Import
Enscape	Visualization	Architecture	Shadows change, RTX	✗	✗	🗸	File Sync, Export/Import
Fuzor	BIM, Construction	Architecture, MEP, Construction	Issue tracking, 4D construction	🗸	✗	🗸	Real-time with Live Link
Gravity Sketch	Design	Architecture	Design, Nurbs, Texturing	🗸	🗸	🗸	File Sync, Export/Import
Holodeck NVidia	Visualization	Architecture	Issues tracking, Texturing	🗸	✗	🗸	File Sync, Export/Import, Omniverse
IrisVR—The Wild	BIM	Architecture, MEP	Issues tracking, Texturing	🗸	✗	🗸	File Sync, Export/Import
Mindesk	Design, BIM	Architecture, MEP	Design, Nurbs, Issues tracking	🗸	🗸	🗸	Real-time with Live Link
TwinMotion	Visualization	Architecture	RTX, Vegetation, Shadows	✗	✗	🗸	File Sync, Export/Import

, and are described in detail in the following paragraphs.

Arkio, Gravity Sketch, and Mindesk have advanced design features at the vertices level. Mindesk supports the full drawing capabilities of Rhino, such as polysurfaces and NURBS. Gravity sketch supports natively drawing with mirroring, NURBS, and splines. Arkio put effort into the radial menu, which very intuitively presents the tools selection options. Comparing these three, Arkio has the best UI experience; however, it lacks NURBS drawing. Mindesk has the most drawing features; however, it requires Rhino to achieve them. Gravity sketch has a quite good range of drawing capabilities and can be used as standalone software.

Fuzor, IrisVR, and Mindesk are collaboration and BIM information tools in VR. Their role is to allow the inspection of BIM information for problem-solving. The advantages of Fuzor are that it is highly realistic, provides a 4D simulation of the construction, and also allows pipe editing. IrisVR provides fewer features, but has a very intuitive user interface. Mindesk plugin for Revit is a rather new plugin with few features. Fuzor and Mindesk are live interconnected with Revit, whereas IrisVR should be synced with file exchange.

Twinmotion, Enscape, and Holodeck offer advanced visualizations that can be used for project presentations. They all support real-time ray tracing (RTX), textures, and daylight-changing interfaces. In particular, Twinmotion allows for AI avatars, vegetation growing, materials changing, and weather changing. A disadvantage is that Twinmotion’s VR interfaces are a subset of the desktop interfaces for avatars and vegetation configuration. Enscape offers basic features for rendering. Holodeck also provides multiuser support and 3D markup drawing. They all support affine transformations communication to Revit with file syncing.

2.3. Interconnection Solutions

In this section, we analyze state-of-the-art methodologies to transfer data from CAD/CAE/BIM software into VR and vice versa. Interconnection solutions can be divided into importing/exporting files, sending data to a central server, and using live link communication. Particularly interesting is central server communication, as data are permanently stored in a database where they can be retrieved, reviewed, and edited at any time. A common standard should be adopted to achieve interconnection. In

Table 3. Existing solutions for software interconnection.

#	Name	AEC Discipline	License	Software Supported	Features
1	3D Repo https://3drepo.com (accessed on 7 June 2022)	All disciplines	AGPL3	Revit, Navisworks, Autodesk, Civil3D, Autocad, Unity, Unreal	Clash detection, issues tracking, diffing, safety issues detection, and permissions allocation.
2	BHoM https://github.com/BHoM (accessed on 7 June 2022)	All disciplines	LGPL	Rhino, Revit, Ladybug, Excel, SAP2000, Unreal, Speckle	A communication standard across 3D software rather than a software solution.
3	Omniverse NVidia https://developer.nvidia.com/nvidia-omniverse-platform (accessed on 7 June 2022)	Architects (Design in VR)	Commercial	Rhino, Revit, Sketchup, Unreal, Unity, 3DS, Maya, Grasshopper	Holodeck provides VR capabilities in Omniverse. Support for many systems’ interconnection.
4	Re.AKT/AKT.Core	Structural Engineers (Interconnect CAD to CAE)	Own use	Interconnections between Rhino/Grasshopper, Unity, Revit, Sofistik, SAP2000, ETABS and Karamba	Produced by AKTII partner. Connected with Speckle.
5	Speckle http://speckle.systems (accessed on 7 June 2022)	All disciplines send and receive data into Speckle web database (Postgress)	GPL2	Async Interconnection with Rhino, Revit, Grasshopper, and Unreal with “Connectors”	Speckle Objects format for the interconnection of software, and a web interface.

, we summarized some open standards and software for that purpose. These are analyzed in the following paragraphs.

3DRepo is a full-fledged solution for remote collaboration based on web interfaces (Figure 10a). It supports interconnection through its plugins for Revit, Navisworks, Autodesk, Civil 3D, Autocad, Unity, and Unreal. It can be used for clash detection, issues tracking, diffing, safety issues detection, and permissions allocation (Figure 10b). It is based on MongoDB and RabbitMQ servers [10]. Although it is provided as open-source, the asset creation is based on the transformations of assets in a Unity format, which is closed source.

The Buildings and Habitats object Model (BHoM) is a universal object definition framework implemented as a set of open-source reference plugins for many platforms. It defines a common interface for software to work in parallel. It targets AEC organizations that develop their own software so that the final result follows a standard and can potentially be interconnected. It lacks central database implementation but can be used as a baseline standard for making such a database. It supports major brands such as Revit, Rhino, SAP2000, Unreal, and Unity.

Omniverse offers asynchronous collaboration through a central database named Nucleus, as shown in Figure 11. It is based on Pixar’s USD format for assets and scenes transfer. Omniverse Nucleus works via TCP/HTTP in a publish/subscribe model, where multiple clients are connected to it in such a manner that when one client submits a change (publishes it), every other client (subscriber) will immediately receive that change. One constellation of collaborators can have multiple Nucleus databases that can be synced (e.g., A and B, where A is used for architecture and B is used for mechanical, electrical, and public health (MEP) engineering). Several connectors (plugins) connect Rhino, Revit, Maya, Blender, Unreal and other software with Nucleus, as shown in Figure 11. NVidia provides a toolkit allowing one to develop their own plugin, as well as a physics simulation engine and an RTX toolkit for realistic representations.

Re.AKT software was developed in-house at AKTII over the past 10 years. Re.AKT interfaces directly with a range of industry-leading modeling, BIM, structural analysis, and drawing production software packages to form an interlinked and holistic design environment. This toolkit enables design teams to rapidly collaborate and refine their work, ensuring that all aspects of a project can be coordinated through a single ‘master model’. Re.AKT currently provides a complete connection between the following software and schemas: SAP2000 (v19, v20, and v22), Sofistik, ETABS, Karamba, Rhino, Grasshopper, the IFC format (therefore, Revit, Microstation, ArchiCAD, etc.), and many of AKT’s in-house developed tools (such as AKT Carbon and AKT Unity). Partial connectors also exist for Robot, Karamba, OpenSEES, Speckle, and native Revit elements. When software packages connect to Re.AKT, their data classes and elements are exported and converted into AKT’s underlying data classes: AKT.Core. Converting all elements to AKT.Core ensures that the same robust and low-level data formatting and compression processes can be applied to all data, irrespective of where they came from.

Speckle is the Git of 3D design for the AEC industry. It provides the infrastructure for pushing and pulling data to a joint repository in commits and branches, as shown in Figure 12. Speckle provides a common specification for data to be processed by popular AEC software and in parallel provides an open-source software used for synchronizing. Its API is based on GraphQL REST API and its database is based on PostgreSQL, which has the potential to be extended on a large scale. An instance of the server is also offered for free as a service (https://speckle.xyz (accessed on 7 June 2022)). An organization can also install the server technology on its own premises. It has a web back-end named “Dashboard” that can be used to inspect data in the cloud. Its current version supports Blender, Revit, Rhino, Grasshopper, Autocad, Excel, Dynamo, Tekla, Microstation, Unity, and Unreal. Its advantages include its openness, low cost, portability, and wide range of connectors. Its disadvantages are a poor permissions management system, as it has only three roles for an entire project (owner, contributor, reviewer), and the lack of a branch merging mechanism, as merging should be performed using Grasshopper software.

Overall, the aforementioned interconnection solutions rely on their own format, so they are not compatible with each other. Apart from BHoM, which is only a schema, all the other solutions offer software such as plugins and databases. Among them, only Speckle is completely open-source and free to use. Omniverse is a solution with a wide range of features, e.g., it supports digital twins and MEP-related applications.

3. Proposing a System for Multi-Disciplinary Collaboration in VR

In this section, we propose how the ideal VR system for cross-discipline collaboration could function from a technical perspective, namely by merging various technologies mentioned in Section 2. A VR collaborative system for AEC should ideally cover all different phases of AEC projects with proper functionalities and user interfaces. The VR system should communicate with existing software tools used in the AEC industry so that designs can be reciprocal in the traditional working chain and comply with standards (such as the IFC standard) as exported from Revit or Rhino software. In Section 3.1 we collected certain VR interfaces across solutions that were found useful and can serve as development guidelines for the ideal system. Next, in Section 3.2, we present an interconnection pipeline for synchronous and asynchronous collaboration which can be part of the ideal system.

3.1. UIs Guidelines

In

Table 4. Guidelines for VR interfaces.

	Interface Action	Implementation Pathway by the Majority of Vendors		Proposed Directive
1	Movement	Movement is allowed in two modes, namely Fly and Walk (Enscape, Fader, and IrisVR). Both modes are performed with teleporting as the most recommended way to avoid nausea [11].		VR software should support Fly and Walk mode in order to allow users to avoid VR sickness.
2	Zooming into model	Zooming functionality is achieved by holding both grab buttons in each VR controller and by approaching hands (Arkio, Mindesk, and IrisVR). This also allows the player to change the size of his or her avatar in order to fit into the model.		Holding the two grab buttons and approaching hands should make the user move closer to the table and make the user proportionally smaller.
3	Rotate Model	In Arkio and Mindesk the selected 3D model is rotated by holding the two grab buttons and steering hands either clock- or counter-clockwise.		Holding both grab buttons and rotating hands should rotate the selected model.
4	Toolbar	The toolbar is by default attached to the left VR controller in all VR solutions. The right controller serves as a selection and drawing tool. This is becasue most people are most skilled in selecting things with their right hand. Twinmotion has the option to change this setup for left-handed people.		The toolbar should be on the left VR controller and selections should be made with the right VR Controller (opposite for left-handedness).
5	Toolbar tweak 1	The toolbar is an essential tool that is often used. It is handy to attach the toolbar to a static position so that it can be easily accessed rather than opening it again (Arkio).		There should be an option to attach the toolbar to a static position for speeding up workflow.
6	Toolbar tweak 2	After a tool is selected and the toolbar closes, it is difficult for the user to remember which tool is active. Arkio has an armband for indicating the active tool. Gravity Sketch indicates which tool is active over the right controller buttons.		Virtual armbands and controllers’ buttons overlays should indicate which tool is active.
7	Toolbar organization	The organization of the toolbar in a rounded way (Arkio), or in a sequential way (IrisVR, Twinmotion, and Enscape) is easier to use than complicated vertical and horizontal logic (Gravity Sketch, Holodeck, and Fuzor).		Organize the tools in a rounded or a single logicsequential order. Avoid big grids and subpanels shown together.
8	Sliders widgets	In VR it is difficult to move a slider in the toolbar menu. Twinmotion implemented sliders as buttons that slide vertically to change time and weather.		For implementing sliders use big buttons that slide upwards or downwards.
9	Material change	Materials should be overlayed over big buttons and sequential menus (Twinmotion). The material is not easy to perceive while in 2D overview (Hololens and IrisVR). It is better to overview the material wrapped on a sphere (to see reflection properties).		Implement sequential selection menus with material representations in spheres. Do not use file browsers for selecting materials.
10	NURBS and polysurfaces	Drawing curves is essential. Gravity Sketch supports it natively; Mindesk supports it through Rhino.		Drawing curved lines should be supported natively, i.e., without the use of third party software.
11	Taking Screenshots	Screenshotting is efficiently performed when a preview screen is available and attached as a screen to the right hand (Arkio and IrisVR). The right trigger should enable photo shooting.		Have a preview screen on the right hand for taking screenshots.
12	Transparency in the menu	Transparent menus can cause cognitive stress (Gravity Sketch and Mindesk). Transparency should change automatically based on the contrast between the toolbar and the background.		Auto-contrast for the toolbar transparency reduces cognitive stress.
13	Store teleport positions	Teleporting to certain positions is a useful feature. These positions should be able to be saved from the VR menu (IrisVR), not only from the desktop menu (Twinmotion).		Enable the ability to save certain positions for fast teleporting.

, we summarize in the first column the most important features found in the review of the solutions in Section 2.1. How this feature was implemented by the majority of vendors is outlined in the second column. In the third column, we propose a directive that can be extracted from the findings described in column 2.

3.2. Optimum Interconnection and Collaboration Schema

The general scope is to enhance collaboration in immersive spaces by providing tools that enable a smooth workflow before, during, and after the construction of a building. The fragmented nature of the architecture, engineering, and construction industry is well recognized, and has led to several documented problems ostensibly relating to failures in communication and information processing [12]. In the Draft Safety Bill [13] currently under review, UK legislation is introducing the golden thread principle, requiring a detailed and traceable safety record of all the decisions made throughout the lifetime of the project, from early decisions during design through to legacy handovers as well as maintenance and repairs of the completed building. This highlights the need for a unified review and record-keeping platform that can be used to consolidate and keep track of the review and decision-making process during the entire lifecycle of a project. This can be achieved in an optimal way in VR; bringing the embodied experiential aspect to the simulated project facilitates the most informed and realistic encounter with the developing design. We envision VR as a review space; i.e., a communication, visualization, and project management tool that handles high-resolution immersive space as well as documentation and recordkeeping, allowing all parties involved in the creation and maintenance of a building to converge in a single experiential space.

There are several challenges that should be met in order for a single VR environment to cover the collaboration needs in the industry. A major challenge is supporting two types of collaboration: synchronous and asynchronous collaboration modes. In the synchronous mode, collaboration takes place in real-time; therefore, it can be used for discussing/examining issues and negotiating/resolving conflicts before solidifying them into released versions. At the implementation level, data are transferred with shared memory access for software programs that are connected through fast network topology, e.g., with the Live link technology of Unreal. VR tools that implement this type of interconnection are Fuzor and Mindesk. In the asynchronous mode, individual contributions have to be merged and saved in the master project by one person. It can be considered as the pull-merge-commit-push mechanism of the Git code versioning system. This is important due to the fact that AEC partners may reside in places that have completely different time zones, and need a way to update the design using a differential methodology, i.e., without storing a replica of the entire design just for a small change. Omniverse and Speckle are representative examples that offer software for a central database, REST TCP APIs, and connectors for many design software.

Another important challenge is the communication of the VR environment with existing tools such as Revit, Rhino-Grasshopper-Ladybug, SAP2000, Autocad, and Navisworks (to name just a few) which offer extra functionalities and provide the final release format. For example, Revit has a large database of building information modeling (BIM) objects [14], Rhino can design complicated 3D models based on mathematical functions [15], Grasshopper-Ladybug can perform annual daylight or thermal simulations [16], and SAP2000 can provide structural simulation 3D models [17]. There is no format that is acceptable by all vendors that can serve all the phases of a project from concept drafting and designing to integrating engineering solutions, to formulating construction details, and providing a maintenance plan. There have been some steps toward a joint format (e.g., using the IFC format) [18], but so far IFC is a lossy format across software vendors, and therefore used only for final project submission towards legal approval, usually via Revit.

3.3. Establishing a Theoretical Framework for Collaborative Design

We envision authoring an architectural project as an iterative process where the designers alternate between decision making (e.g., selecting/placing different materials and components) and previewing the result of their previous decisions so as to make new ones. Insufficient perception of the impact of their design choices may lead to badly-informed decisions and, in turn, to suboptimal design choices [19]. In addition, the design process of architectural projects is characterized by high complexity which stems from the problem of “parallel worlds”, a term used to describe the fact that several disciplines (architects and a variety of engineers) coexist in an architectural project, each with distinct requirements and roles. However, only “intersection”, by means of the close collaboration of these “parallel worlds”, can bring an architectural project to fruition. This “parallel world” aspect creates the necessity for an interdisciplinary tool capable of addressing the unique requirements of each discipline both individually and simultaneously, where all authors are able to work on the same architectural project and perceive it in their own, different way, as best suits their needs. This fact dictates the necessity for a “decomposition” of the architectural project into components that meet the needs of individual disciplines. Similarly, there is also a need for a “composition”, where individual designs, created separately, can be merged to form a unified architectural project.

Given the aforementioned needs, the ideal system for multi-discipline collaboration should implement the asynchronous collaboration mode, as mentioned in Section 3.2, which will be the backbone of the system. It should provide: (a) a central database that will always contain the latest version of the design for any discipline; (b) the presentation of the partial designs united in one common format; and (c) translation of this format back to the discipline dedicated format for on-screen editing. Next, the system should implement the synchronous collaboration mode which allows designers to co-exist in the same space and time, so as to solve design issues.

In Figure 13, we depict the concept of the proposed system in a diagram that merges these two collaboration modalities. The horizontal dimension represents asynchronous modality, allowing any changes on the AEC project to be stored persistently in a database; the vertical dimension represents synchronous modality, allowing real-time collaborative design. The scenario is as follows. On Date 1, an architect formulates the concept of the project in drafting software, such as Blender 3D or Maya, and pushes this design to the database using asynchronous mode. On Date 2, multiple architects pull the initial draft from the database using asynchronous mode connectors to Blender, Rhino, and VR, the latter made with Unreal or Unity Graphic Engines. They contribute to the initial draft design in real-time using synchronous mode connectors that allow for collaborative design. Then, the principal designer pushes the design back to the database using an asynchronous mode connector.

On Dates X1, X2, and X3, other disciplines can be involved. A MEP engineer can commit changes into the database from Revit by working synchronously with structural engineers in SAP2000 and an architect in VR. Additionally, in other cases all disciplines can work collaboratively inside VR to resolve conflicts, and one of them can push the final design into the database (e.g., on Dates Y1, Y2, and Y3,). It is possible to commit changes into the database from any type of software for each stage of the project, i.e., Blender 3D for drafting, Rhino 3D for architectural design, SAP2000 for structural simulations executions and proposing changes, Revit for committing engineering plans, and VR for collaborative design and conflict resolution.

3.4. Implementation Pathway

In the following, we propose an implementation blueprint for the theoretic system proposed in Section 3.3. It is represented by the diagram shown in Figure 14. On the left-hand side is onscreen software such as Rhino, Revit, SAP2000, and Blender, which represent what already exists in the industry. It is often called the AS-IS scenario by the IEEE830 requirements analysis standard [20]. These software were selected indicatively; however, other software can also be interconnected, as the system is modular. On the right-hand-side is the proposed VR software, which represents the TO-BE scenario [21]. In the center of the system we placed a Speckle system database, as it provides the benefits of a system that is open source and focused only on the AEC industry. It relies on Speckle connectors and Speckle object notation used to translate Revit, Rhino, and SAP2000 into a JSON text schema and save them within a PostgreSQL database through a commit mechanism. Communication with the Speckle database is achieved with REST HTTP GraphQL API (black lines). Alternatively, other closed source systems for interconnection can be used as described in Section 2.3.

The synchronous mode across VR and non-VR software is implemented with real-time data transfer achieved using Mindesk real-time transfer technologies such as memory access and Live-Link (blue lines). For example, the designs from Rhino can be transferred in real-time within VR; this is important, as Rhino has complex surface design tools that are difficult to implement in VR. Unreal Engine was selected as the overall graphics engine for VR as it offers photorealistic rendering of a higher quality level than other graphics engines.

The usage scenario is as follows. Architects 1 and 2 use Blender to draft the design concept collaboratively through the Speckle Blender connector (as shown on the left-hand mid-side of Figure 14). Next (left-hand side bottom), Architect 2, Architect 3, and other disciplines in Rhino can fetch the concept design, collaboratively make edits synchronously through the Mindesk connector, and commit the changes with the Speckle Rhino connector to the database. Structural engineers (upper left-hand side) can pull the design from the database and make simulations using either SAP2000 or Grasshopper-Karamba. Re.AKT connector can be used for communication between SAP2000 and the database. The Grasshopper Speckle connector can be used to push Grasshopper simulation geometries to the database. The Mindesk Grasshopper connector can be used for collaboration across Grasshopper users. MEP engineers (left-hand top side) can use Revit and Speckle connectors to fetch the design from the database, make MEP plans, work synchronously with other MEP engineers through the Mindesk connector, and submit changes to the Speckle Database through the Speckle connector.

VR technologies are found on the right-hand side. Architect 4 can be immersed in the VR environment and fetch the latest design through the Speckle Unreal connector. The Mindesk Unreal connector can be used to interconnect Unreal with Revit and Rhino. As the database contains a large amount of BIM information from Revit, VR design capabilities can be enhanced through a “semantic” module service. It can be offered using a knowledge base service used to interrogate the database for semantic information across commits (e.g., “fetch all users that modified this asset”). In addition, users can collaborate through Unreal Engine services such as Epic Online Services [20] for multiuser interaction and Vivox for audio communication [22].

4. Conclusions, Discussion, and Future Work

In our review, we discuss several issues regarding the collaboration of engineers within VR and between VR with on-screen software. Theoretically, VR collaboration systems are based on the concept of OpenBIM where information flows across disciplines. However, there are many technical issues to be solved before this happens in an efficient manner. We attempt to treat the technical side of the problem, and propose solutions that allow information to flow in a bidirectional manner from VR to on-screen software and vice versa. In our proposed ideal VR system, we attempt to treat collaboration with a centralized silo where all have access. Currently, much of the effort across VR vendors is consumed by maintaining the quality of the model when shifting to designing in VR and when transforming formats across graphic engines. This is due to the fact that each graphic engine vendor developed a closed source format, which can not be decrypted easily.

Apart from technical issues, a permissions and responsibilities system is missing from existing interconnection implementations. The current implementations stem from the field of software design for central databases (e.g., who has the authority to write, read or comment), which should be further researched and modified according to the needs of engineers. Such issues include model ownership, incorporation of BIM into the contractual relationship of the parties involved, design liability, reliance on data and the evolution and responsibility of the model, design responsibility, lack of standardization, litigation and protocol, working collaboratively, the role of the BIM coordinator, and sharing copyrighted data [6]. A theoretical approach to the treatment of legal issues can be found in the context of our project in [23]. However, it is beyond the scope of this study to extend the discussion in this direction. The permissions and responsibilities implementation methodology is a topic for future research and development.

As regards VR interfaces, there is a lack of standardization. For example, desktop 3D design software share the same interfaces for panning, scrolling, and zooming, or at least provide the option to change the interface to user preferences. This is not valid in VR software, as each vendor develops its own approach for each basic functionality. For example, it is not agreed if the user avatar should be scaled or the model should be scaled. When scaling down the avatar, the perception of space changes. In addition, the impression of other avatars at different scales causes disruption from tasks. All these issues, in conjunction with the derived guidelines in Section 3.1, are subjects for further investigation by informatics and psychology scientists. Usability studies should be performed using questionnaires and electronic methods (e.g., eye-tracking and analytics), to reduce the cognitive load in both perception and interaction.

In the future, VR collaboration systems could also exploit other technologies such as augmented reality, 5G networks, remote rendering, geospatial maps data, 3D scanning, mixed reality, and digital twins [24]. Today’s VR technologies are capable of intersecting the parallel worlds of the engineering disciplines in a common space that will promote designing and decision making that reshapes the collaboration process in architectural projects; however, great effort is required to increase usefulness and user friendliness so as to create an attractive and operational environment.