The result of this evaluation was an overview graphic of possible AR use cases for public authorities in the context of digital construction projects (see
Figure 2). The structure of the figure is based on the chronological process flow. The use cases are classified clockwise according to the five phases under consideration (zoning, planning, permission, construction, and operation). The use cases are also colour-coded in rings and assigned to the five main stakeholders. Here, too, the arrangement is chronological, according to their occurrence in the overall process, from the inside to the outside. The individual use cases are explained below in sections corresponding to the phases. Use cases 05 (digitisation of plan checking at the authority’s office with the help of AR), 06 (AR plan checking on-site), and 08 (AR as an on-site support for the building authority agent) have already been dealt with in detail by the authors in [
28], and are only briefly summarised here.
3.1. Zoning
The aims of “smart cities” place new demands on urban development. A central goal of a smart city is to strengthen social coexistence, e.g., through participatory opportunities for the citizens [
29,
30]. In Vienna, citizen participation has been implemented in an exemplary manner for a long time. It is to be further expanded in the future through the use of new technologies (e.g., digital participation platforms) [
31]. The integration of AR thus follows a central goal of current urban developments to promote cooperation between administration and citizens. Currently, the members of the public already have the opportunity to help shape the future appearance of the city by participating in spatial planning. Spatial planning in Vienna is regulated by the zoning and development plan. This plan governs the building possibilities of plots of land (above all, the building density and structure) using graphical representation and textual additions.
The zoning and development plan serves as a legal basis for architects in planning, as well as for the departments of the building authority for the examination of planned building projects. The process of deciding on the zoning for a new district can be divided into five steps [
32] (see
Figure 3): The process starts with the idea (Idea). Subsequently, the framework conditions are ascertained (Clarify), and concepts are developed on this basis (Planning). Before the spatial planning is decided (Zoning and Resolution), the opinions and objections of various stakeholders and the population are obtained and examined. Building on this process and existing possibilities for participation [
32], the authors developed new opportunities for participation using AR. The use cases are described below.
Before urban planning competitions are announced, public planning workshops can be held. In this format, ideas for urban concepts are developed cooperatively with citizens, local actors, initiators, and politicians. This phase aims to collect and analyse the stakeholders’ needs and summarise them in target definitions as a basis for planners. The existing process can be extended through AR and the gamification principle in this early development phase. Gamification describes the increase in intrinsic motivation to solve problems by using gaming elements in a new context [
33]. Finland has already tested this approach to designing public spaces using virtual reality [
34].
Augmented reality is mainly used for the visualisation of existing plans. However, in the development phase of spatial planning, BIM models do not yet exist. In this case, AR can be used to develop rough spatial structures. Building mass distribution considerations can be made using a modular system with various predefined elements, and effects such as shadowing can be simulated and visualised with AR. Positioning, orientation, and the height of buildings have a decisive influence on the lighting issue: the lighting situation of rooms depends on neighbouring buildings and the shading of public areas. The limitations of inner-city temperatures and the avoidance of urban heat islands (UHIs) will become more critical in the coming decades [
35,
36]. Intelligent spatial planning concepts can achieve a cooling effect similar to that of tree planting.
The placement of solids can indicate the division of residential and industrial areas or green spaces (see
Figure 4). Using parameterisation, information on the number of dwellings or workplaces created is directly displayed. The developed concepts then serve as input for urban planning competitions.
If urban planning concepts are already available, information exhibitions can be held. In Vienna, information exhibitions provide the public with information on the status of current district planning. To date, plans, scale models, or other visualisation concepts have been used with the disadvantages of high costs (scale models) and nonexpert citizens struggling to understand the technological aspects of the plans. By using AR, for example, a virtual spatial model of the district could be obtained via a QR code on an information poster and viewed on a tablet or one’s smartphone. If there are several design drafts, they can be cost-effectively and efficiently displayed and compared with each other by overlaying them. In terms of presentation, two variants are interesting: a combination of a haptic model with a virtual supplement or an utterly virtual presentation. In the first case, a cardboard or 3D-printed model of the spatial planning area (surroundings) is supplemented by a virtual overlay of the different planning drafts. This involves costs for the hardcopy model but provides citizens with easily understandable information. In the second variant, the entire representation is virtual. This can save costs, but investigations regarding suitability are recommended, especially for older people.
Augmented reality can support decision-makers in various phases, and the advantages of AR increase with the level of detail (planning progress). In the development phase of spatial planning, work is completed using designs with a low level of detail. In this context, AR can, for example, support juries of urban planning competitions or political decision-makers as a visualisation tool. With the implementation of an evaluation tool, designs could be assessed and feedback obtained for planners and decision-makers. The feedback from the citizens, collected during the information exhibition, can then be integrated into the decision-making process.
After a decision is made regarding the development plan, AR can also be used in architectural competitions for public buildings. Currently, building permissions in Vienna are exclusively established in analogue form. As an additional supplement to the planning documents, scaled models are often required but only used for architectural competitions. However, if produced only for this purpose, these models are very time-consuming and correspondingly expensive to produce. In the BRISE-Vienna research project [
37,
38], the building permission process is currently being developed on an openBIM basis for the City of Vienna. The development of the openBIM building permission process will make spatial digital building models available early in the planning process. With the help of AR, these could be used for digital visualisations instead of physical models and without additional costs for comparing different designs. The digital building models only need to be exported from the authoring software in an appropriate format (IFC or OBJ) and loaded into an AR viewer.
3.2. Planning
The use cases presented in the previous section are independent of the projects as part of the development of the zoning plan. In the next step, the focus is shifted from the superordinate level to the project-specific level. The building regulations in the zoning plan can be used as a basis for preliminary design planning. Due to the high requirements for dimensional accuracy during the objection process, the City of Vienna requires the preparation of a surveying plan. During this practice, the alignment lines are precisely located and subsequently checked by the city’s department.
In [
8], the authors described the current analogue permission process and developed a digital openBIM-supported process based on the prevailing disadvantages. Part of this process is the surveying plan. This contains alphanumeric information from the zoning plan (e.g., property number, zoning, or building class) and geometric information from the on-site survey in the form of lines (e.g., alignment lines) and areas (e.g., building site, existing buildings, or roads). The surveying plan serves as a planning basis for both the BIM model for architects and checking the BIM model for the authorities. At present, the surveyor measures prominent points, transfers them to CAD software, connects them to lines and surfaces, and finally provides them with alphanumeric information. A visual overlay and check of the surveying plan with the real building site is only possible with AR (overlay of virtual elements over a real environment). This could represent an additional module for quality assurance in the future. Based on these results, an AR use case was developed to create the surveying plan and check a so-called reference model (REM) within the future openBIM-based permission process.
Within the future openBIM-based permission process, the surveying plan is only an intermediate step. A reference model (REM) is automatically generated from the surveying plan and additional information is generated from the building design to verify the BIM model. The REM can be understood as a kind of 3D development plan of the construction site and the neighbouring buildings, which is required for the automated verification of a site’s buildability, correctness of position, and information details [
8]. The basis for the REM is the legal material and the surveying plan in IFC format, consisting of spatial lines and areas and including additional information in the form of attributes.
As a first step, AR could help to create the surveying plan and check the REM. In this case, AR in combination with an HMD can be seen as an alternative to computers and CAD software. After transferring the survey data (point cloud) to the HMD, the individual points could be connected to lines and surfaces using AR. When entering alphanumeric information (e.g., defining a line as a building line), the zoning plan is displayed as a base layer. A possible interface is shown in
Figure 5.
In a second step, AR could create a quality assurance tool for the REM. Once the surveying plan has been made, the REM can be generated based on this. This makes it possible to conduct a plausibility check of the REM with AR. Errors in the input of the surveying plan or the generation of the REM can be detected visually. AR thus represents a two-stage quality assurance tool for checking the survey results and the REM.
3.3. Permission
In the next phase in the building life cycle, the building authority reviews the permission documents. In the future, building permits in Vienna can be issued digitally based on a BIM model, once the planning has been completed based on the surveying plan. These BIM models enable the extension of the permit review to include AR applications [
28], especially in the area of plan checking. In Vienna, this plan checking not only includes checking compliance with all valid legal materials (zoning, building regulations, etc.) but also protecting the interests of neighbours [
28]. For this purpose, the authority examines the objections of neighbours as defined in the building code. In addition, neighbours can inspect project documents during the permission procedure to check for compliance with their objection options and, if necessary, raise an objection. At present, 2D plans are used for plan checking. The technical language of the plans requires spatial understanding and can lead to a feeling of a lack of transparency among nonexpert persons. Based on these problems, an AR app was designed to digitise the plan-checking process and is currently being tested. In addition to an adapted process for plan checking at the level of the authority, this concept also includes the possibility of plan checking on-site. Both use cases are described in detail in [
28].
An AR model consisting of two partial models is used for the citizens’ plan checking with regard to the authority. The submitted BIM model is reduced to the relevant information for the plan checking of the citizens and then inserted into a digital as-built model of the city (see
Figure 6). The objection rights of neighbours concern the building envelope. The reduction of the BIM model submitted to the building envelope thus corresponds to the legal situation. Additionally, this protects the client’s privacy and the planner’s intellectual property.
The data basis for the as-built model is provided by airborne laser scanning and terrestrial surveying [
39,
40]. The data are combined into a 3D geodata model with abstracted building bodies to capture the existing buildings. This data collection method is associated with inaccuracies of up to ±25 cm. These data are currently only collected at 2-year intervals; therefore, they are not guaranteed to be up to date at the time of submission. The data collection regarding the neighbouring buildings in the surveying plan enables comparison and, thus, ensures future data’s accuracy.
Subsequently, the two submodels are converted into a common model in an appropriate format for visualisation with AR (JSON). The spatial representation of the submitted building project, including neighbouring buildings, using AR helps improve the understanding of nonexpert citizens during the plan-checking process.
The same AR model can also be used for the hearing process. In Vienna, at the end of the standard permission process, the authority can hold a hearing with the parties involved. The hearing takes place at the authority’s office after the plan-checking phase. The basic idea is the same: instead of plans, the meeting is supported by AR visualisation. For this purpose, each participant receives their own tablet. With this tablet, everyone can view the AR model from their own perspective. In order to improve communication, the tablets can be synchronised. When synchronised, in the AR model, the building authorities official can select and highlight specific components which are displayed on all the tablets. This helps the participants to follow the explanations.
In contrast to the plan checking that occurs at the building authority level, the submitted BIM model is not embedded in the digital as-built model of the city but is displayed on-site, directly between the neighbouring buildings. The submitted building is located using a vector (defined by two points) and then visualised in the correct size and position [
28]. Thus, the use of AR also enables forms of participation for citizens.
At the same time, however, access for citizens imposes high requirements concerning authorisation questions. If not only participating persons but citizens in general are given the possibility of access, a new concept of information access is required. A possible starting point could be an online city map in combination with AR geotags.
The City of Vienna offers an unrestrictedly accessible city map based on the data of ViennaGIS [
41]. In this map, different kinds of information can be accessed via different layers (e.g., short-stay parking zones, markets, hiking trails, and drinking fountains). This website could be extended with layers for public building projects, e.g., AR models linked by geotags. Geotagging or georeferencing is the process of linking geographic metadata with other digital data (e.g., texts or images) [
42]. The result of the process (the link) is a geotag. In this case, the tags contain links to call up the corresponding AR models. If the link is called up, authentication takes place via a digital signature on the mobile device so that the user’s identity is established. This is followed by a comparison with the access authorisations of the building project in the city’s database. After successful authentication, the AR model is viewed.
This form of on-site representation can be used by both citizens and authorities. In Vienna, the permit process consists of many partial inspections, including statics, fire protection, required lighting, or the “cityscape-relevant assessment”. The latter focuses on checking the integration of the building project into the townscape from an architectural point of view. This often requires on-site plan checking. The virtual representation of the planned building project amid the actual neighbouring buildings offers considerable added value when preparing the expert opinion. The entire process can be carried out on-site using tablets.
3.4. Construction
In the future, the openBIM permission process will enable a different approach to the creation of a digital twin of the City of Vienna. Due to the increasing number of digital submissions, digital abstracted models of existing buildings based on measured data can be continuously replaced by accurate BIM models. The digital twin of the city, which is becoming more and more precise, can serve as a basis for various applications. These include building physics simulations (summer overheating of cities, energy efficiency, etc.), urban mining strategies, or the application of AR, such as for checking the building stock or as a support for the fire brigade [
43].
The authority aims to ensure a safe and proper construction process. Whereas new construction projects focus primarily on building consensus, safety questions are often clarified in existing buildings. This must be regularly checked by a building authority agent. AR can support building authority agents in several ways. Their tasks require on-site inspections on the construction site or in the existing building. AR glasses are particularly suitable for this purpose.
The tasks of the building authority agents cover the construction and operation phases. One task of a building authority agent is to check the building consensus regarding the approved building plans/models. These inspections can occur during or after the construction work’s completion. Due to the high cost of on-site plan checking for all construction sites (e.g., familiarisation with the project, orientation in the building, and dimensional checks through manual measurements), these are randomly carried out during construction. After the completion of construction, a notification of completion must be submitted to the authorities. In this notification, a civil engineer confirms compliance with the building permit. The inspections refer to the submitted documents focusing on building projects that have already shown conspicuous issues during construction.
Establishing an openBIM-based building permit in Vienna will enable future on-site plan checking using AR, which can occur both during construction and after completion. The basis for these checks is the BIM model submitted as part of the application for a building permit. The content of the assessment is the entire architectural model, which is superimposed on reality, and deviations are checked, documented, and reported. This type of review currently requires a great deal of familiarisation time. The orientation and review on the construction site are carried out based on the submitted plans and are often very time-consuming. If equipped with an HMD, the process could be easier for building authority agents. In the field of vision of the glasses, for example, a small floor plan, which marks the current position, can help with orientation in the building. Plan checking can be facilitated by overlaying the actual state of the building with a semitransparent representation of the BIM model. A measurement process is no longer necessary to determine whether elements (e.g., walls) are in the intended position. Current deviations in the location offer sufficient accuracy for this case [
24]. Differences can be quickly identified, quantified using a measurement feature, documented in the BIM model through screenshots, and referenced to the building component on-site. Future deviations could be detected and displayed in AR [
3]. At the authority level, the detected deviations are evaluated and summarised in reports. Depending on the severity, a request is made to the owner (e.g., adjustment of the digital model or adjustment of the real object required). The documentation can be exported as a BIM Collaboration Format (BCF) file for forwarding to the owner.
3.5. Operation
In Austria, the building authority has various tasks during the operational phase of a building. In this phase, multiple tasks can be simplified through AR support.
The random checks conducted by the building authorities could be used not only to verify the building consensus but also to check building safety based on the digital building register. In Vienna, operators must ensure safe building conditions according to legal regulations. For this purpose, the building regulations mean that a building register must be kept. This must contain the following pieces of information, among others [
44]:
The designation of the components that require regular inspection;
The date of the initial inspection and future intervals;
The results of the checks that are carried out.
National standards [
44] suggest relevant components, including inspection intervals in the form of checklists (e.g., an annual inspection of the roof truss and a monthly functional check of the ventilation of staircases). Although building logbooks are currently digitally created (in Excel or with special software), they are kept in paper form in the building for inspections (for the building authority). BIM models will enable the model-based documentation of these data in the future.
At the time of submission, this information is not yet available (LOI 300). Therefore, the review during construction was limited to the architectural model, as described in UC 08. Only in execution planning, awarding, and construction was the required information continuously entered into the BIM model. The resulting as-built model (LOI 500) was transmitted to the authorities after the completion of construction. This digital model was the basis for the AR use cases’ facility management (FM) and safety (firefighters).
An inspection of nonapproved construction measures (e.g., additions) or the control of the building book could be carried out in the future using HMD devices. During an inspection, the BIM model (LOI 500) overlaps the existing building. Structural deviations can thus be easily detected and documented. Information on the most recent inspections and required inspection intervals is displayed in the field of vision.
An as-built model, in combination with AR, can make processes more efficient for FM. With AR, for example, information can be transferred to the BIM model and kept up to date.
The operation phase is the most significant life cycle phase, in terms of both time and costs. This phase is also interesting for cities as operators of public buildings. In Vienna, the city manages 1613 buildings through the building management department/MA 34 authority [
45]. The operating phase accounts for up to 70% of the life cycle costs (LCCs) [
46]. The currently poor data transfer at the interface between construction and operation leads to cost factors, e.g., [
47]:
The maintenance of digital twins based on BIM models can help to reduce these costs. The focus here is on the transfer of digital data from construction to FM (BIM2FIM). Before linking information from the BIM model to CAFM software, the information must be stored in the BIM model. Augmented reality can support this process.
During progressive BIM-based planning, the information content of the digital model is constantly increasing. In building services, initial dimensioning and simulations are carried out, and components are optimised before the tender is issued. Product-independent performance requirements are finally put out to tender at the end of the planning phase. In the construction phase, the companies commissioned select suitable products based on the tender documents and install them. After the construction is finished, all important building parts for facility management are identified using a QR code or RFID. The LOI 500 required for many FM tasks requires transferring product-specific information into the digital model. This step could take place during the HVAC acceptance process. In the AR-AQ-Bau research project [
6], an AR-supported acceptance of building service components using the AR glasses Microsoft HoloLens 2 was developed for maintenance work during operation [
4]. The possibility of transmitting manufacturer information (e.g., performance and warranty data) using QR codes and AR glasses according to the BIM2QR principle was also explored. While recording defects, QR codes on installations could be scanned using AR glasses, and manufacturer information could be retrieved and stored in the BIM model using gestures. The data from the BIM model could then be linked to CAFM software. In addition, step-by-step instructions can be called up, or an expert can be consulted via a remote system.
Firefighters are exposed to many hazards during operations, including high temperatures, smoke poisoning, or falling parts, with the most significant danger being smoke. The reduction in risks for firefighters is the focus of the research project ProFiTex2 [
48] (see
Figure 7) and the system C-Thru by Qwake Technologies [
49]. In fires with heavy smoke development or darkness, the field of vision of the emergency services personnel can be almost entirely restricted.
Qwake Technologies has developed a two-component system consisting of AR glasses for the emergency forces and a remote coordination system. Using AR, the real environment can be overlaid with orientation information. The environment can be scanned via the infrared sensors of the AR glasses, and a spatial model can be generated. The recorded data of the point cloud are evaluated, and only edges are projected into the field of vision of the AR glasses. This digital information enables emergency forces to orientate themselves despite severe visual restrictions due to darkness or smoke. In addition, the system can be supplemented with thermal sensors to detect critical temperature ranges (see
Figure 7).
The second component is remote coordination. A bidirectional exchange occurs between the components; the field of vision of the emergency forces and the location are transmitted to the control centre and the control centre sends, e.g., navigation instructions to the emergency forces. At present, no building plans are available to the fire brigade in case of an emergency. The emergency forces have to orientate themselves in the building and correctly assess sources of danger. In the future, digital building models, which are available to the authority through the permission process, could support navigation within the facilities. The operations coordinator loads the submitted BIM model. This provides floor plans as well as material information regarding the building fabric. In combination with the live images of the emergency forces, the coordinator can, thus, better assess potential hazards and transmit information to the emergency forces, e.g., the remaining deployment time due to the fire resistance class of the building structure, the risk of falling components due to the component structures, priority rooms due to the rooms’ designations, or instructions for navigation in the building due to the floor plans. The transmission of the BIM model to the individual emergency forces for superimposition onto the actual situation using AR glasses is not reasonable in this scenario. The BIM model in the LOI 300 has too little information for pure orientation in limited-visibility conditions, as furnishings are not included.
Another task of building authority agents is the assessment of damage in existing buildings. For example, after a fire, the fire brigade notifies the building authority to check the structural situation and determine whether there is a danger of collapse. For this assessment, it is essential to understand the static concept of the corresponding building. The use of AR can also provide support in this case. The real building condition is overlaid with the digital BIM model. By selecting individual layers, it is possible to display only load-bearing components (feature: load-bearing = true). In this case, the real environment is overlaid with the statically effective spatial model via the AR glasses. Supported by this overlay, building authority agents can quickly recognise the load-bearing structure. AR could, for example, help identify which cladding levels need to be removed to assess the statically relevant component. This workflow means that inspections of the as-built plans on-site are no longer necessary.