Augmented reality (AR) is a research domain in ongoing transformation due to vast demand and developments in ICT. In recent years, the diffusion of AR has been mainly due to technological and social reasons. As far as the former is concerned, the extensive massive digitization of 2D and 3D data contributes to the definition of increasingly large repositories of digital information. Moreover, the leap forward in the hardware and software capacities devoted to this domain is crucial [1
]. Both conditions foster the application of AR, providing billions of users in several fields. In addition, this tool supports the growing desire to access and interact with digital information everywhere. The actual society is constantly engaged in multiple communication data flows that intersect and overlap. If managed properly, this information can significantly support and stimulate the understanding and interpretation of a complex reality. In this passage, the interaction between users, real data, and digital data becomes crucial, and AR has a central role to play.
In the past 20 years, AR research and training experiences have been carried out in several fields, including medical and biomedical, robotic, industrial, educational, and entertainment, to name but a few [2
]. This article explores the architecture domain, because it represents an experimental area with wide margins for development. Such potential is due to architecture’s pervasive diffusion, cultural content, social value, and the massive number of actors working in this realm. The European Union (EU) strongly supports these purposes. Starting from the Digital Agenda (2010), many efforts have been carried out to digitize cultural heritage (DCH), redefining new knowledge and communication paths about architectural content. Almost all the most recently funded projects on digital cultural heritage (INCEPTION; i-MareCulture; Time Machine; ViMM) concerned AR use. In the 2021–2024 Agenda, one of the pillars is devoted to empowering people with a new generation of technologies, contributing to the realization of the European “Digital Decade”. This will promote the built domain’s accessibility, interoperability between institutions and users, preservation of national identities, territorial belonging, and cultural awareness. The diffusion and multi-generational communicative capacity of AR will facilitate achieving these goals, suggesting an innovative architectural experience [5
] supporting design and space communication [6
The application of ICT for cultural content reading, understanding, and communication is already a consolidated practice in the cultural computing domain [7
]; it deepens digital applications in the arts and social sciences to represent, enhance, and extend knowledge, introducing new creative processes [8
]. The availability of digital data, the requirements of access to knowledge, and the process optimization purposes foster the built domain as an ideal field for AR experimentation. Architecture is defined by several passages, from design to building best practices, analysis, monitoring and interpretation activities, open access to knowledge, and promoting heritage. Moreover, architectural education has a central role in strengthening personal and collective culture, and the correct use of the built heritage. The observation of architectonic space introduces both the importance of the observer’s position and the spatial relationship between objects and the environment. Thus, AR can have a central role in cognitive terms, preparing the observer for a more conscious use, management, and design of built space.
Several aspects converge to form the AR built domain (Figure 1
), from workflow design (AR Project) to digital content creation (virtual content), from system creation (AR System) to platform definition (AR SDK) and the evaluation of the cultural/education experience (experience feedback). The transdisciplinary meaning is highlighted by the preferential relationship with some ERC fields according to the 2021/22 panel. The selection proposed in Figure 1
is a qualitative analysis based both on the contents and descriptions of the different ERC panels, and on the author’s experience. There is, therefore, the awareness that many of the fields not mentioned may play a role in this tree structure. However, the scheme intends to show the fragmentation of the disciplinary fields involved in AR for built architecture for skills and research content. The cultural, social, and content aspects are mainly supported by the social sciences and humanities (SSH). In contrast, hardware, software, and algorithmic aspects are analyzed more in the physical sciences and engineering (PE). In the SSH domain, there is a more significant sector fragmentation related to the topic. SH2 explores the issues of democratization of communication channels, and their impact on society. Studies devoted to the real environment, educational models, and transfer of skills are referred to as SH3. The content activities and construction related to the cultural studies and dissemination of past and existing tangible and intangible artifacts are referred to as SH5/SH6, up to local and geographical data management (SH7). Moreover, the PE6 sector is prevalent in the macro-sector PE, devoted to computational systems, guided and autonomous algorithmic methods, and user interface (HCI). It is emphasized that these connections and their different roles and weights within the methodology are strictly outlined in AR for architecture. This fragmentation (Figure 1
) involves numerous studies, which mainly deepen acquisition, tracking, visualization, and interaction aspects, suggesting a technological and algorithmic solution [10
]. Only a few are devoted to explaining the whole workflow in the built realm [7
], or focused on transferring knowledge [15
]. This lack may depend on the peculiarities and problem-solving approaches of the research in question, which can develop specific aspects without managing the subject as a whole. This specificity promotes the interest of experts in the domain, adopting a language and terminology consistent with the field, limiting the transdisciplinary dissemination of content, and promoting a watertight circuit that feeds itself.
This review aims to address an overview of AR in the architecture domain, encouraging a transverse and transdisciplinary reading of the entire pipeline and its specificity. The article follows a holistic approach, collecting the different experiences conducted in the field with the purpose of:
Defining AR framed in the path of architectural knowledge;
Organizing the AR pipeline, highlighting the main aspects of the chain;
Outlining the state of the art of AR in the architecture domain;
The in-depth study of these topics will be developed according to the following sections: Section 2
is dedicated to the definition of AR and its location within the reality–virtuality continuum, making some initial critical considerations. From Section 3
to Section 7
are discussed the main methodological steps. Specifically, in Section 3
are presented the main criteria to build an AR project, analyzing each inference with the architecture domain. Section 4
is devoted to digital content creation, explaining the levels of iconicity and data typology. Section 5
traces the primary HW and SW elements in an AR system, from the different devices to the tracking and registration systems, up to the HCI. Section 6
is dedicated to an in-depth analysis of the currently existing AR platforms related to the world of architecture, deepening the different types, purposes, and potentialities. Section 7
addresses the issue of feedback on the AR experience—a critical matter, especially in education, but necessary to improve the experience offered by AR. Finally, Section 8
is dedicated to the state of the art of AR research in Architecture, verifying the different topics related to the built domain. Some brief critical conclusions on the in-depth study close the review.
2. AR in the Real-Virtual Continuum
The augmented reality sphere relates to both the real environment and the virtual world. AR is placed within a well-known spatial interval (continuum) between real and virtual (Figure 2
), named X-reality (XR), which contains real environment (RE), augmented reality (AR), augmented virtuality (AV), and virtual reality (VR) [16
Mixed reality (MR) describes the domain between AR and VR. AR’s position in the continuum, close to physical reality [17
], introduces virtual information into an existing reality via devices. This integration can be additive, layering information that does not exist, or subtractive, covering or deleting parts of the real world. The first case is referred to as AR, while the second as mediated reality. The latter, like AR, can be placed close to the real environment in the continuum towards AV. Within architecture, the transformation of real or virtual data towards the AR environment is supported by well-established methodologies discussed in Section 4
. The connection between RE, AR, and VR (Figure 2
) is made through 3D surveying and modeling methodologies that allow for the digitization of reality, such as reality-based modeling (RBM) or image-based modeling (IBM). At the same time, parametric models are artificially built through a geometry-based approach [18
In the past decade, the definitions of AR highlighted and changed some specific elements or relations involved in the process. First, Milgram and Kishino in 1994 defined AR as: “any case in which an otherwise real environment is “augmented” by means of virtual (computer graphic) objects.
” This definition pointed out the secondary role of AR, as an integrative medium to better understand the real environment, or a virtual reality connected to it [17
]. This meaning was modified three years later in Azuma’s definition: “A system that combines real and virtual content, provides a real-time interactive environment, and registers in 3D
]. The new definition was focused on the real-time representation, interaction, and its location in three-dimensional space. The ”ethical” aspect of reality integration and multisensory approaches disappeared, stating the priority of the visual system over the other senses. In the following decades, this definition and the consequent taxonomy were expanded and integrated several times. AR is a system capable of improving the understanding of the world through the integration of virtual information, enlarging immersion and environmental knowledge [9
]. The main AR characteristics (Figure 3
), according to Azuma’s definition and subsequent studies [22
]. Outlined in the architecture domain, must present the following aspects:
It must merge the real architectural environment, or a portion of it, with virtual data in a common visualization platform on the device;
It must work in real time, reaching a synchronous update between the movements in the real space and the simulated virtual one;
It must allow a high level of engagement, not constrained to a single point of view, moving freely in the architectonic space or around it;
It must allow a variable level of interaction between the users, the real world, and the virtual contents.
The relationship between AR and the wider XR allows for the understanding of its interaction and immersion value [7
]. Virtual reality (VR) permits a complete immersion in a digital world, presenting an environment disconnected from the real one. However, it can represent an interpretive reproduction [3
], integrated with perceptual stimuli such as visual, auditory, and tactile stimuli to simulate the experience of a real environment [23
]. Interaction occurs only with virtual information through different devices. In a symmetrical position to AR, augmented virtuality (AV) enriches the virtual world with scenes from the real one, enhancing the sense of presence and interaction with an artificial environment [24
]. The domain between AR and VR is occupied by mixed reality (MR), in which real and virtual contents coexist, proposing an integrated, flexible, immersive, interactive platform for experience enhancement [25
]. Interaction, in this case, is mixed, as it works on three different levels: At the real level, it occurs with body movement in the real space. At the virtual level, it happens only with virtual elements visualized in devices. The third level connects the real and the virtual levels by manipulating elements such as markers. [27
]. All these environments (Figure 3
) are characterized by different levels of immersion (i.e., immersive, semi-immersive, or non-immersive) depending on the type of device and visualization. The reduced field of view and precise tracking requirements of devices for AR visualization lead to the consideration of AR as a low-immersive representation technique. In general, the integration between personal devices, sensors, and large projection screens justifies increasing the level of immersion towards VR, but losing the connection with reality, which is very useful in the architecture domain.
4. AR Virtual Content
Content creation (authoring) is central to the AR process, even if many articles refer to technical problems and development in hardware or software issues. The content must be coherent with the context (contextual awareness), following the three different AR aims (Figure 6
), and meeting the communicative/educational goal while respecting a specific type of user. Moreover, different levels of virtual content can be associated and defined by different construction methodologies and relationships with the real context. The text strings present a low level of complexity that requires content validation and placement in the real context with POIs [31
]. Even digital images, acquired by a camera or post-produced, show a spatial discrepancy between the 2D content and the 3D real space, using POIs unless the image is superimposed on reality according to a specific perspective. AR visualization projected on a plane display is suitable and consistent with 2D digital data representation.
Conversely, 3D models are different because they must define their state in the digital and real space, behavioral rules, relationships, and interactions [23
]. For this purpose, there are different types of models and modeling methods. The primary division consists of reality-based models (RB data) and virtual-based ones (VB data) [60
]. In 3D architectural modeling, the distinction between those models plays a key role, along with the model generation methodologies and issues addressed (Figure 8
4.1. Reality-Based 3D Models
AR in the architecture domain is strongly connected to the digitization process (Figure 9
). In the past 20 years, the development of 3D acquisition methodologies has allowed for an enormous increase in heritage digitization. The digitization activity is central in supporting the protection, maintenance, conservation, and promotion activities; It allows for the democratization and accessibility of much cultural heritage [61
], encouraging digital heritage and digital humanities to meet [62
In the early 2000s, range-based systems were very limited in terms of sampling volume and scanning time, limiting their application largely to sculptural elements inside survey projects set out with topographic or photogrammetric techniques [63
]. In 2005, the introduction of CW-AM systems led to a drastic reduction in the survey time, preserving a similar resolution, accuracy, and uncertainty [67
], and allowing the first complex 3D acquisition. In the past decade, the advancements in the scanning domain have been concentrated mainly on the reduction of equipment dimensions. Moreover, new methodological approaches have experimented with a multiresolution integrated pipeline [68
].and the generation of correct, reality-based models [69
In the passive system domain, the use of ever more advanced and cheap digital cameras has been accompanied by the evolution of software for image analysis, supporting photogrammetric processing, and the construction of 3D models [71
]. The introduction of these automatic or semi-automatic processes, such as Structure from Motion (SfM) in the computer vision field [72
], has gained these photogrammetric applications for the architectural survey and relative comparison works [74
]. This trend, amplified in recent years by the increasing use of RPASs equipped with digital cameras [77
], has supported the drive to build reality-based models referring to complex scenarios [78
Today, the massive integration of active and passive systems makes it possible to obtain ever more accurate and complete surveying campaigns, ensuring redundancy data for integration and validation activity, heading definitively towards a multiresolution approach. Triangular polygons characterize reality-based models. The surface normals are calculated during the meshing step, matching the relative RGB data with the vertices or the polygons. The final 3D model’s reliability is dependent on accuracy and uncertainty concerning the acquisition instrument and the reverse modeling process applied [79
]. In the end, the point clouds or polygonal models can be manually or automatically classified [80
], depending on the ontologies present in the architecture detected and in the analysis process. This subdivision can be highlighted and managed separately during visualization in AR.
4.2. Virtual-Based 3D Models
This category belongs to models created following bibliographic sources, iconographic sources, or 2D/3D survey information. These external data are translated into mathematical surfaces concerning the design choices or source interpretation. The interpretative effort and the control of relative tolerances are planned according to the aim(s) of the modeling process. Examples of architecture can be considered layered structures of geometrical, material, physical, and structural information necessary to read and fully understand an architectural organism. For this reason, virtual-based models produced by survey data define the basis of more consistent applications. Several BIM-based platforms offer the possibility of binding this information directly to the acquired data. The advantages of using mathematical modeling for AR are essentially threefold: On the one hand, the optimization of the management of 3D data simplifies the work of the graphics processor for real-time rendering. On the other hand, the flexibility in shape modification is helpful in the design process, and in the creation of diachronic built models. Finally, there is the possibility of better managing the stratification of information on different layers [59
]. The models (Figure 9
) are essentially divided into solid and surface models [81
]. The former is semantically coherent with the architectural modeling, using the volumes and material consistency of the artifact. The material stratification favors constructive validation, and all types of simulations, but presents several bottlenecks in free-form surface generation and model management. All BIM-based modeling is part of this domain [82
]. On the other hand, surface modeling proposes constructing building skins, improving computational management, and reproducing complex shapes. Moreover, it provides an abstraction of the real model, reducing it only to its outer shell. Mathematical and numerical models (meshes)—with some derivations, such as Sub-D modeling—belong to this domain. In general, the type of virtual-based modeling depends on the model’s precision and the reconstruction aim.
4.3. Extension of 3D Models to AR
Reality-based or virtual-based 3D models may require an additional step to be translated into data suitable for AR (Figure 9
). The more straightforward passage regards direct uploading to an AR application, in which the automatic process of model adaptation is carried out. This case happens when a 3D model is ready for AR visualization without requiring post-production. Moreover, some modeling platforms foresee specific modules/plugins supporting editing process, i.e., AR-media (3DS, Cinema4D); Kubity (Google SketchUp), Mindesk (Rhinoceros), Marui (Maya), and Revit Live (Revit). The third, and more complex, solution regards the necessity of managing geometric, material, illumination, or animation modifications. In this situation, the modeling activity diverges from AR data preparation, adopting game engine platforms. These programs support complex 3D model management, interoperability of file formats, rendering, animation, and data interaction. On the one hand, they are limited by the complexity of the platforms—a significant obstacle for inexperienced programmers. On the other hand, the passage between different platforms requires verification of the transition of the 3D model’s geometric and material characteristics. For example, in model generation, particular attention must be paid to the creation of shapes. The use of solid or mesh models, for different reasons, allows better directional control of the surface normals—a fundamental feature for the correct visualization of the model in real-time rendering.
In the mathematical surfaces, different directions of generative curves and main lines imply different distributions of normals. This aspect must necessarily be controlled a priori to reduce any subsequent post-processing work [18
]. The most popular game engines are Unity 3D (Unity Reflect in the architecture domain), OpenSceneGraph, Unreal Engine, and CryEngine. Unity 3D is perhaps the most accessible platform that supports fully cross-platform development, but it does not allow real-time modeling. OpenSceneGraph is widely used for visual simulation, modeling, games, and mobile applications [83
], but it is more oriented towards desktop and web-based applications. Unreal Engine 4 includes outstanding graphics features and is arguably the best tool for achieving accurate results. It enables the development and deployment of projects across all platforms and devices. CryEngine requires experienced developers.
6. AR Development Tools
Augmented reality software development kits (AR-SDKs) have a “bridge” function between the device and the virtual content through a platform devoted to data integration, management, and visualization. Each one offers a specific GUI (graphical user interface) to connect the user’s virtual content. These are intended as “interaction metaphors” between the input and output data, highlighting the communicative capacities and purposes of AR digital content in the real world. Several parameters define the use of these AR-SDKs, and their possible application in the architecture domain: cost, supported platform, image or target recognition, 3D and tracking, support by game engines, cloud or local databases, geolocation, and SLAM [143
]. Moreover, in this paragraph, a different clustering of AR-SDKs based on accessibility and availability is proposed, dividing them into open-source, free (proprietary), and paid (commercial) (Figure 13
). The other specific AR-SDK characteristics will be evaluated accordingly. The open-source systems guarantee their implementation through access to the source code. In contrast, the others differ only in terms of being free or paid, but do not allow a system modification.
Most of the leading AR-SDKs were developed between 2007 and 2012, but the app scenario has profoundly changed after 10 years. The number of platforms available on the market has increased significantly. Here, the main multipurpose platforms, or those devoted to the architecture domain, are discussed (Table 2
6.1. Open-Source SDKs
The main open-source and multipurpose SDKs are ArRToolkit, DroidAR, AR.js, and EasyAR (Table 2
). ARToolKit is a vision-based AR library that includes camera position/orientation tracking, camera calibration code, and cross-platform development. It uses video tracking capabilities that calculate camera position and orientation relative to square physical markers or natural feature markers in real time, solving both viewpoint tracking and virtual object interaction. Distributed with complete source code and initially used for PC applications [144
], it was the first mobile AR-SDK, seen running first on Symbian in 2005. It was tested in many applications for architecture and design purposes [24
], to support collaborative design decision meetings [145
DroidAR was developed in 2010 and was designed to create AR applications for the Android operating system, both for location and image recognition. It is the only open-source (GPLv3) AR SDK dedicated to Android applications, supporting build location-based and marker-based AR experiences. Evidence of this tool appears in [14
], and environmental documentation in [146
] and learning object creation for geometrical space analysis [148
EasyAR Sense, previously known as EasyAR SDK, is a standalone SDK that brings new algorithm components and platform support to increase AR capabilities. It supports sparse and dense spatial maps, as well as motion, surface, 3D object, and planar tracking.
6.2. Free/Proprietary SDKs
In early 2018, Google launched ARCore, an open-source (Apache 2.0 license) augmented reality SDK for bringing compelling AR experiences specifically to Android devices. ARCore employs three main capabilities: motion tracking, environmental understanding, and light estimation. ARCore’s motion tracking technology uses the phone’s camera to identify points of interest and tracks how these points move over time. In addition, ARCore can detect flat surfaces and estimate the average lighting in the surrounding area. The LayART project suggests the generation of 2D layouts from indoor images using SLAM and ARCore [149
], while Palma et al. used this application to superimpose real and reconstructed vault systems [150
As with the ARCore system, it is essential to cite ARKit, a system related only to the IOS platform, which integrates depth sensor information with depth API, location anchoring, and facial tracking. The merging of scene geometry calculation, person occlusion analysis, instant AR with LIDAR plane detection, and motion capture constitutes a powerful AR system. After almost a decade of using the free software Aurasma, today, a possible alternative with some implementation is offered by ARloopa—an augmented reality (AR) and virtual reality (VR) app and game development system providing advanced AR and VR services: cloud-based AR, custom-branded AR app and game development, virtual reality app and game development, and 2D and 3D content creation. Its simplicity of execution, while limiting functions, can help expand AR applications. An appealing and promising project concerning AR in the architecture domain is Archi-Lens, in which blockchain is combined with AR to store documents and AR models of the construction plans, as well as using AR to design and keep track of the state of the construction site [151
6.3. Commercial SDK
The most used and best known multipurpose commercial SDKs, not devoted to the architecture domain alone, are Layar, Wikitude, and Vuforia (Table 2
). Layar is the most widely used AR-SDK for service localization; it can store POIs in a remote database (DB) and retrieve the associated information based on the user’s position, using the Blippar augmented reality SDK to enable quick and easy mobile AR development. These features make this kind of application ideal for outdoor wayfinding applications and experiences in architecture and urban planning [5
] and create a framework for architecture and building engineering education [152
Wikitude is a platform released in 2008 that leverages both location-based and vision-based tracking. For the latter, it supports images and 3D model recognition. The generation of 3D-model-based targets is currently in beta version (SDK 7), but from 2017 it introduced SLAM technology and instant tracking, working with three-dimensional structures and objects. An interesting application for partially damaged or destroyed buildings to help visitors interact with the city’s monuments is offered by [49
], using this geo-localization to interact with archaeological sites [153
After removing Junaio (Metaio) from the market, Vuforia has become the most widely used toolkit for most developers. Its simple integration with Unity3D allows the visualization of refined 3D models, and a quick and easy cross-platform development. It supports various 2D and 3D target types, including image targets, 3D multitarget configurations, and a form of addressable fiducial marker known as VuMark. The author of [154
] presents a framework for visualizing 3D artifacts with Vuforia in archaeology, while [155
] explains the systematic structure, recognition of the target library, and the working process of the virtual exhibition hall system. At the urban/landscape scale, [156
] suggests an application, while [157
] proposes the visualization of interpretative 3D models, and [158
] proposes an AR BIM-based application.
Furthermore, some additional applications are implemented in the architecture domain. One is Maxst, a powerful AR-SDK system that combines virtually enhanced physical reality with AR (augmented reality), AI (artificial intelligence), and computer vision technology. MAXST provides various solutions, supported by VPS (visual positioning service), to define the user’s location by constructing a 3D spatial map for both in/outdoor AR content creation, instants, objects, QR codes, images, and marker trackers, VisualSLAM, and Cloud Recognizer. An application related to interior design and planning architecture furniture can be found in [159
]. Moreover, there are several AR-SDKs designed to support architecture design and visualization. One strictly connected to Autodesk is AkulAR, allowing the user to experience 3D digital models at real size, geo-located in the real world, using only a smartphone or tablet. IN2AR is a non-new ActionScript3 and Adobe Native Extension Library or Unity3D plugin that detects images and estimates their pose using standard webcams/cameras or mobile cameras. The pose information can place 3D objects and videos onto the image for AR applications. Augment supports the AEC domain with easy configuration; it is based on the Vuforia augmented reality SDK and OpenGL. Meanwhile ARki is a real-time AR visualization service for architectural models, which incorporates AR technology into architecture by providing 3D models overlaid onto existing 2D floor plans, with multiple levels of interactivity for both design and presentation purposes; it includes real-time shadow analysis and material selection. Fuzor is a real-time rendering app integrated with Revit, allowing users to move around, visualize, annotate, and inspect BIM information, while also syncing changes between the two tools; it supports various measurements, clash analysis, lighting analysis, color and visibility filters, cross-section and section cut rendering, and walkthrough video rendering with BIM information embedded. GammaAR is an application that uses AR technology to overlay 3D models using smartphones or tablets for the AEC industry; it helps the users to compare the real work with the planning information contained in the project. Moreover, Gamma AR connects the construction site and the office via a portal, enabling the user to access any annotation, photos, issues, or reports at any time. Dalux, developed in 2005 as a BIM viewer application, allows users to access the BIM model and sheets using smartphones or tablets; moreover, it developed a new technology tool called TwinBIM, which merges the BIM model with the physical world; it is compatible with Google ARCore and iOS ARKit, and works by scanning the physical environment and creating a 3D map of the surroundings, knowing exactly where to place the BIM model, aligning and anchoring it to the physical world. Fologram collects applications and plugins that run in mixed reality devices and integrate with Rhino and Grasshopper. In Fologram, geometry, layers, and grasshopper parameters can be modified in real time to create powerful tools for design, communication, collaboration, and fabrication using smart glasses or mobile devices. Finally, AR CHECK is an application that makes the use of 2D prints on construction sites unnecessary, and eliminates human error, preventing mistakes in the prefab construction industry.
7. AR User Experience
To conclude the AR development, its application must correspond to those design criteria planned in the design step, with feedback on the knowledge model. The AR application is a moment of augmented experiential deepening of context and content, affected by several factors (Figure 14
). In the architectural domain, the complexity of the case study analyzed, the point of scene observation, and the level of interaction play crucial roles. Moreover, the user experience achievement must be coherent with the project aims; this can be verified through a feedback system, optional in some applications and mandatory in others (Figure 6
). The intersection of context, content, and purpose may generate different punctual and localized experiences, or ones developed within precise knowledge pathways (Figure 15
The first point concerns the relationship between the model complexity and the capacity to transmit knowledge through AR. Architectural complexity is a vast topic, ranging from the artifact’s size to its geometric and material complexity, from the functional to the hierarchical relations of the individual components and their mutual inferences. Among these elements, the scale of visualization and information is crucial in the domain of AR. Architecture is characterized by a substantial variation of scale, from the territorial level to the individual construction detail. Each of these scales contains a different data level, which can be included and integrated into a multiscale 3D model. However, the experiential mode of AR, which usually provides fewer tools for visualization and management of digital data, reduces the layered model’s capacity, limiting the investigation and interaction with different levels of scale. Moreover, the complexity of information related to the individual scales poses both a problem of managing massive digital data and usefulness in experiential terms. Abstraction activity, through geometrical simplification, may lead to a better and faster understanding of phenomena or spatial relationships. Therefore, it may turn out to be more functional when using sequences of models to increase complexity concerning the knowledge and purposes of AR in the AR domain.
The experiential model also refers to the point of view (Figure 15
), central both for tracking and visualization activities. The dynamic relationship between the camera position and a 2D marker/image is vital to deepen the visualized element and its construction rules. This movement becomes necessary in 3D markers/artifacts to experience the real object and initiate many visualizations connected to different target areas. The introduction of the movement may involve both target and camera systems if the 2D marker is not bound to fixed elements; otherwise, it all depends on the camera reference system in the space. In both cases, the user’s freedom to discover additional information from the point of view starts an individual and collective educative experience. First, the user can develop his knowledge model, investigate the relevant aspects of the subject more deeply, and start an autonomous learning path framed in structured educational models. This experience is intertwined with a collective one. Comparing the different individual experiential models on the same subject may improve critical knowledge by comparing the different experiences. In the end, particular attention should be paid to a non-immersed visualization of interior architecture. The low immersive level of AR, unless using HHD systems, provides few solutions in this case. One such solution consists of making the external envelope transparent, allowing an internal visualization. Otherwise, it is appropriate to integrate the visualization with a predefined system that gives access to spherical images, alternating an AR and a VR vision.
Another issue is the interaction with the virtual model (Figure 15
). It is well known that the interaction of AR is much lower than that of MR and VR (Figure 3
). Nevertheless, it is possible to identify some levels of interaction with the subject. The first is the low-level one, based on the overlap between virtual and real data. In this experience, the personal interaction consists of choosing the points of view to investigate and deepen the digital data, starting a critical path based on observation. An intermediate level of interaction [154
] foresees some punctual functions on the display that start preloaded processes, such as variations in the visualization or animations. A third level concerns the real-time interaction of the user with the digital data, varying the visualization according to the user’s actions. This increasing interaction is related to the value of the personal knowledge path in the architecture and the specific purpose of the AR. Moreover, the more profound link between interaction, experience personalization, and content concerns new content creation. This last level refers to a possible research domain devoted to authoring new personalized digital models superimposed on reality through user–content interaction. Some variables, such as recorded reality data, tracking shapes, or GUI boundary conditions, managed by an AI system, may suggest new shapes in the space, introducing an absolute personalization of the cognitive path.
The last aspect refers to the experience feedback system for the validation of the process (Figure 15
), identifying the different improvement boundaries. Feedback occurs concerning the AR goals. First, feedback passes through the direct experience, and can be considered an added value compared to the available data in the actual scenario. In this case, feedback can happen via quick choice responses to better address the target group’s needs for the proposed cultural itinerary. The second level of feedback concerns an evaluation of the knowledge and understanding process:
How this AR visualization transmits content;
How this content is rooted in personal experience;
How AR experience is translated into a better understanding of the real phenomena.
It is necessary to plan a deeper system to evaluate feedback as a new and more effective channel for learning. According to the specific case analyzed, this process can occur through interviews or questions to test this learning stream. This planned feedback system, appropriately weighted concerning the type of user, allows the system to be implemented. In the educational domain, the group experience can also be evaluated. The authors of [160
] propose a mobile AR framework for peer assessment, in which students can submit their work and evaluate that of others. A final aspect explores the integration of augmented reality and data mining to record how a given application is used for a specific goal. The specific user–model interaction through a specific GUI can be recorded anonymously and processed by the system. Therefore, these usage data can be employed and implemented in a machine learning system to define new scenarios, optimize the system, and improve the recommendation abilities.
In the architecture domain, there are several experiences in more than 20 years of research on a global scale. In the final section, a synthetic and comparative overview of the possible AR applications in this domain is offered, supported by citations of some significant examples.
The subdivision of such a broad topic appears complex due to the multiple domain classifications. It was decided to group the application experiences into three macro-areas of research: AR for build deepening and enhancement, AR for the architectural design and building process, and AR for architectural training and education. The first area addresses historic and historicized architecture, which flows into cultural heritage, in which AR for architecture plays a fundamental role in knowledge, conservation, and visibility. The second area mainly focuses on AEC for new buildings, deepening AR in the interaction phase between designers and users (co-design) and managing the complex process of building construction. Finally, the last topic discusses knowledge transfer through didactic, educational, and training paths for cultural aims and professional training.
The selection of experiences refers to the following objective parameters, shared in the three areas, and not in order of importance: keyword, date publication, abstract feedback, and text analysis. The use of keywords makes it possible to narrow the scope of the search. Some of them have a cross-cutting value on the three domains (for example: architecture, built environment, augmented reality, state of the art), while others vary according to the domain in this way:
Area 1: Cultural heritage communication/valorization, build enhancement/management, buildings storytelling, cultural tourism, monument exploration;
Area 2: AEC, design process, project planning, co-design, monitoring construction, building process, BIM pipeline, collaborative process;
Area 3: Education, training activities, knowledge transmission, EdTech, professional skills, civil engineering study, assembling learning.
According to the order of keyword coherence, the publication date mainly —but not exclusively—targeted two periods. The first addressed the first or “founding” experiments in the field, highlighted by search engines as those with more citations, more visibility, and more rooted in the field of interest. However, this process has also highlighted significant articles in the past 20 years. A second search started in the year 2021 to present an overview of research updated to recent years. These search criteria allowed the definition of the first selection of articles. The selection process was then refined in the contents, going from the abstract to the paper, verifying data coherence and relevance.
8.1. AR for Build Deepening and Enhancement
There is a strict connection between understanding, knowledge, and reading of the existing environment with built enhancement. Indeed, this distinction is directly related to the different target groups to which the application of AR is addressed. Starting more broadly from the basis of the valorization of the built environment, outlined in the domains of both architecture and archaeology, there have been many applications and case studies analyzed in the last 20 years. The pioneering experiments in the early 2000s aimed to highlight some potential of AR in the reading of the archaeological heritage [161
]. It is worth mentioning the ARCHEOGUIDE [40
], ARICH, and ARCO projects [162
]. Ten years on from these experiences, a critical AR diffusion [9
] is highlighted, exploiting its relevance and adaptability, and showing the first MAR applications [43
Based on many experiments [50
], it is possible to state that there are three primary applications of AR experimentation and development for architecture and cultural heritage:
Applications to improve the visitor experience in contexts in which heritage is placed, distributed, or collected. Cultural tourism is an essential economic and social contributor worldwide. This area is more related to education [170
], valorization, and tourist routes in general, improving the accessibility of information in both existing and no-longer-existing urban places [171
] or architecture [150
]. The VALUE project [173
] highlights user location, multimodal interaction, user understanding, and gamification as the four main pillars of innovative built use. The authors of [167
] analyze the experiential aspect of in situ AR communication in cultural heritage and heritage management. The complexity of shapes and interaction with ambient light play a dominant role in AR visualization, increasing or decreasing the sense of presence [175
]. Some studies investigate the impact of AR technology on the visitor’s learning experience, deepening both cultural tourism and consumer psychology domains [176
]. Even if many studies focused on MAR in the cultural tourism domain, few of them would explore the adoption of augmented reality smart glasses (ARSGs). Han et al. [177
] contributed to highlighting the adoption of this technology in cultural tourism. The mobile capacity to merge locational instruments such as GPS and AR is helpful in guided tours and tourist storytelling, furthering the revivification of cultural heritage sites. Hincapié et al. [178
] highlight the differences in learning and user perception with or without mobile apps, underlining the improvement in the cultural experience. This aspect has also been deepened by Han et al. [4
], who studied the experiential value of AR on destination-related behavior. They investigated how the multidimensional components of AR’s experiential value support behavior through AR satisfaction and experiential authenticity. In general, AR contributes to cultural dissemination and perception of tangible remains, while offering virtual reconstructions and complementary information, sounds, images, etc. That capacity, well known and applied in the cultural heritage field, can be enlarged in other domains and territories, i.e., attracting people to rural areas, and improving their social and economic situation [179
To visualize the reconstructive interventions or the simulations of intervention on the artifacts. This second domain is undoubtedly addressed to experts or technicians in the field, who are called to read the built environment and manage it in the best possible way, planning any intervention operations. This activity can be conducted at the urban scale by introducing decision-making and participatory processes—as the SquareAR project has shown [180
]—or at the architectonic scale, showing the use of AR as a tool for reading the building to support the restoration process [181
To optimize and enrich the management and exploration of the monuments. This last strand is aimed at the managers or owners of the assets, using AR to manage the heritage and its main features better, increasing its value in terms of visibility and induction [182
]. Starting from the impact of the surrounding architectural heritage [183
], some articles investigate the value of AR in providing a practical guide to the existing structures, increasing the accessibility and value of the asset [184
]. Other research analyzes AR as a technology applied through devices and wearables for the promotion and marketing/advertising of tourist destinations [185
], replacing the traditional tourist channels of explanation of monuments and architectures in situ [186
8.2. AR for the Architectural Design and Building Process
The relationship between built architecture, design tools, and the use of AR technologies to simplify specific construction steps is a topic that began almost two decades ago and has found a very fertile research ground in architecture [187
]. Today, augmented reality (AR) plays a relevant role in Industry 4.0, showing great potential within the construction industry [188
]. One of the most important contributions in the architectural design domain is in representing a full-scale virtual model to visualize the project on-site [189
]. This supports all actors involved in the process, from architects to interior designers, from structural designers to clients, employers, and workers. AR may support the entire design chain of a building, simplifying the reading, interpretation, and implementation of that complex intersection of activities expected in the AEC domain, and increasingly systematized by the lean and BIM processes. Some current states of the art [28
] have highlighted the potential of AR in simplifying and optimizing that process and improving both the design and construction processes [191
], emphasizing the integration between AR and BIM in the workflow [192
]. Others have highlighted how AR can make BIM/CAD ubiquitous throughout the AR-BIM process, supporting and improving the management and budgeting of the artifact by analyzing critical consumption issues [193
]. Augmented reality, therefore, is proving to be a supportive tool for every aspect of the construction life cycle:
In project planning and co-design activities, the user can preview and share the building and urban context simulation. Urban/architecture planning requires a hierarchy of decision-making activities, enhanced, and optimized by collaborative interfaces. Co-design activities may involve both specialists and citizens, offering a decisive role in the urban and architecture pre-organization choices. Augmented reality can assume the role of a co-working platform. At the urban scale, AR is considered a practical design and collaboration tool within design teams. One of the first examples has been the ARUDesigner project, evaluating virtual urban projects in a real, natural, and familiar workspace [194
]. Urban areas represent a meeting point between stakeholders’ needs and public needs. For that reason, participatory processes are necessary, and AR can increase the range of participants and the intensity of participation, engaging citizens in collaborative urban planning processes [195
]. At the architectural scale, before construction, virtual walks around the future buildings may identify changes that must be applied without causing problems and delays in the building execution phase. In this way, communication delays can be minimized, optimizing project management processes [196
]. Some applications can overlay actual design alternatives onto drawings through MARs, allowing for flexible wall placement and application of interior and exterior wall materials, developing new, engaging architectural co-design processes [197
]. Multitouch user interfaces (MTUIs) on mobile devices can be effective tools for improving co-working practices by providing new ways for designers to interact on the same project, from the cognitive and representation points of view [198
In the construction monitoring process: the real-time overlap between the design and the as-built model allows for sudden validation of the progress of construction. It can also highlight the inconsistencies between the design documentation, 3D model, and the actual state [199
], assessing their effects and allowing for rapid response, reducing project risk. Structural elements can be monitored from the early construction stages to their maintenance [190
]. To monitor and control construction progress and perform comparisons between as-built and as-planned project data, quick access to easily understandable project information is necessary [200
], presenting a model that overlays progress and planned data on the user’s view of the physical environment. Moreover, in recent years RPAS platforms integrated with AR visualization are applied to the monitoring and inspection of buildings [202
In the reading and progress of AEC work, AR allows for supplemental or predictive information at the construction site—for example, by superimposing the structural layout with the architectural and plant systems in space, reducing errors during construction. An example is described by the D4AR project [203
]. This ability to incorporate information on multiple levels can help workers to process complex information in order to document and monitor the in-progress construction [201
]; it can also help workers to identify any on-site problems and avoid rework, quickly optimizing productivity;
AR technology supports some phases of reading, construction, and assembly on-site—for example, by facilitating the reading of plans, construction details, and positioning of structural elements, minimizing errors caused by misinterpretation. [204
]. Concerning distractions in on-site AR, emphasis should be placed on the careful use of this technology to avoid distractions caused by the screen and loss of awareness of the context in which one is immersed.
8.3. AR for Architectural Training and Education
This last section discusses AR as a tool for training different types of skills and users in architecture. What distinguishes the training aspect from the previous sections’ informational aspects regards the learning activity of a specific task—repetitive or not—or contextual knowledge, scalable to different examples. The increasing use of AR for learning over the past two decades [205
] comes to meet both the growing ability to use and manage digital data and the increasing need for dynamic content. The recent pandemic emergency has forced a sudden acceleration in the ”remote learning” and “EdTech” domains. AR communication methodologies can propose new scenarios of augmented education in a constructive framework [36
], verifying the learner’s acquisition process (metacognition) through the interaction between the virtual and real environments. Moving to the transmitted AR content, Arduini [208
] poses the need to modify it according to the new media. Consider, for example, the zSpace application, which imposes a reflection on the coherence between content and the way in which skills are transmitted. The implementation of MAR systems has undoubtedly facilitated AR in teaching, avoiding fixed technological infrastructure. Research on MAR in the education domain has increased significantly since 2013, showing the growing potential and performance of student learning. Dunleavy and Dede [209
] both addressed AR as a learning tool and explored its educational and pedagogical roles in the classroom. Radu [210
] compared AR and non-AR learning performance, listing advantages and disadvantages. In architecture, two main macro-areas of research are reported: education on the built heritage, and training for specific activities in the built domain. The former is required to understand the observed environment and artifact; the latter trains subjects in good construction, maintenance, and restoration practices on a given asset.
Heritage education involves all AR visualization of architectural models or, more generally, of cultural heritage, for educational purposes. It emphasizes the greater communicative and enhancement potential in the use of this form of visualization. This kind of in situ AR learning activity can be defined as discovery-based learning [211
]. Engaging in this sense is the state of the art in Europe in the use of apps for heritage education, as shown by Luna et al. [212
], highlighting that AR remains mostly applied for heritage education instead of modern/contemporary architecture. The state of the art proposed by Ibañez-Etxeberria [213
] also compares different projects related to promoting cultural context, highlighting the connections and users’ preferences with the cultural heritage analyzed, while Gonzàles Vargas et al. [214
] focused their attention on learning motivation. The AR for architecture promotion and understanding overlaps and differs for only one purpose: the dissemination and visibility of the artifacts, and their understanding. In this sense, the possibility of visualizing and efficiently managing a 3D architecture through a target exploring its main morphological features favors geometrical and functional learning [18
]. This can be extended to architecture that no longer exists, or to projects that have never been realized, where their representation helps further understanding of the logic of construction [215
On the other hand, education and training in the AEC field present a significant amount of literature related to AR learning in construction, engineering, architecture, and built management [216
]. Design education represents a step that develops the knowledge and skills necessary to start a critical operational path on new or existing architectures. The role of tangible AR (tangible user interface (TUI)) can support a remote interactive design by providing a link between tactile action and visualization according to collaborative learning [218
]. Morton [219
] suggests implementing AR systems in design studios to improve students’ design learning, quickly evaluating the multiple solutions developed in a design walkthrough. Design visualization also assumes a key role in the educational process. The introduction of the first luminous planning tables [220
], and their subsequent development through AR with SAR, contributes to supporting collaborative data visualization and simulation of boundary conditions to verify a project’s feasibility. In the context of project presentation, the students can enhance architecture representations by integrating 2D drawings and interactive shapes via SDAR [221
]; that condition can be extended to the interior design realm. The SARDE (spatial augmented reality design environment) project has shown how to use an SAR system to support design decisions, reducing the gap between the presentation and the project content [222
]. Finally, it is important to underline the impact of AR systems in the review of architectural designs based on the user’s perspectives, highlighting the capacity of AR to involve many people in the same activity, reinforcing a co-design approach [223
Finally, regarding the training activities for building construction, Shirazi and Behzadan Amir [224
] presented a collaborative, context-aware MAR tool to improve the knowledge paths in the civil engineering curriculum. Despite the apparent technological advancement in which AR plays a key role, its deployment in best practices for workforce management and project interpretation [225
] is still limited. On-site manual or printed drawings for project representation and communication are preponderant, leading to possible errors in interpretation. Moreover, on-site AR training can even involve just some parts of the entire design process, developing some optimized activities that reduce errors in the interpretation and commissioning of the project [227
]. Introducing such technology can provide awareness of 3D spatial interpretation of 2D drawings and assist overall project planning activities [228
]. Some training applications also include training activities to develop workers’ skills in assembling products and understanding on-site safety regulations and equipment operation [229
This article proposes an overview of augmented reality in the architecture domain, reporting the process of an AR project from the design step to the application, highlighting the pros and cons of each step. The goal is to suggest non-fragmented storytelling accessible to non-expert users in a specific research field. In retracing this process, each step has been outlined in the architecture domain, highlighting the main applications and outcomes. The architectonic domain is highly varied, proposing many transdisciplinary types of research not framed in a homogeneous structure. For this reason, cultural heritage and archaeological heritage at the architectural scale are often reported and discussed. The panorama that emerges from this state of the art is a highly active research field spanning more than 20 years, promising exciting future research opportunities. The temporal conjunction of some events makes this even more evident.
On the one hand, technological and algorithmic development brings AR closer to all users, investing in both MAR and SAR techniques; this latter is helpful in the architectural field, preserving a shared, co-design workflow. On the other hand, the increasingly pervasive adoption of geo-localized apps, supported by an extended network and massive digitization that involves both the built environment and processes, creates a fertile ground for development and experimentation. Finally, market interest in AR/MR/VR is increasingly evident, fostered by the vital intersection with data mining, deep learning, and machine learning. The next generation of MAR applications will add value to numerous application domains, including architecture, improving the quality of the user experience. The highly versatile demands of such applications should be supported by cloud-based, edge-based, localized, and hybrid architectures. The high bandwidth, ultra-low latency, and massive connectivity offered by future 5G systems herald a promising future for AR applications in the architecture domain, and will lead to an AR democratization process, projecting a new generation of hybrid representations and fostering knowledge of the built environment.