Mixed Reality in Building Construction Inspection and Monitoring: A Systematic Review

Abstract

Mixed reality (MR) technology has the potential to enhance building construction inspection and monitoring processes, improving efficiency, accuracy, and safety. This systematic review intends to investigate the present research status on MR in building construction inspection and monitoring. The review covers existing literature and practical case studies that scrutinize current technologies, their applications, challenges, and future trends in this rapidly evolving field. This article follows a methodology known as Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) to enhance the credibility and reliability of research. The study includes articles published between 2018 and 2023, identified through a comprehensive search of Scopus and Google Scholar databases. Findings indicate that MR technology has the potential to enhance visualization, communication, and collaboration between stakeholders, as well as increase efficiency and accuracy in inspection and monitoring tasks by providing real-time interactable data and quick decision-making among the project team members. The adoption of MR technology in the construction industry will not only boost its effectiveness but also improve its productivity. However, limitations such as high costs, technical issues, and user acceptance pose challenges to the widespread adoption of MR in building construction. Future research should address these limitations and investigate MR’s long-term impact on building construction inspection and monitoring.

1. Introduction

The construction industry is a critical sector of the economy and plays a significant role in building the infrastructure that supports nearly every aspect of modern life [1]. The productivity of construction industry is closely tied to the advancement of a nation’s economy. Choices related to infrastructure, standardization, and labor productivity have substantial implications for a nation’s economic future [2]. However, it is also an industry that faces significant challenges, such as high project complexity, low productivity, cost overruns, delays, and safety issues [3]. Other challenges in the construction industry include budget constraints, delays, inadequate project planning, and inconsistent quality control [4]. The most significant challenges in the construction industry are related to information and communication [5]. There is a limited adoption of automated systems and robotics in the construction industry [6]. With the expansion of the industry, the intricate and varied nature of construction sites and tasks has prompted the incorporation of automation and intelligent technologies to augment operational efficacy, minimize project expenditures, and safeguard the well-being of both construction personnel and infrastructure [7]. Over the years, technological innovation has played a crucial role in addressing these challenges. The construction industry continually seeks innovative technologies to improve efficiency, reduce errors, and enhance safety [8]. One such emerging technology is mixed reality (MR). MR, as shown in Figure 1, is a blend of virtual reality (VR) and augmented reality (AR), which overlays virtual objects in the real world, providing an immersive yet grounded experience [9]. This technology presents a significant opportunity for application in building construction inspection and monitoring, where it has the potential to revolutionize traditional methods [10]. Building construction inspection and monitoring are essential activities in the construction process [11]. Inspection and monitoring processes ensure compliance with safety regulations, quality standards, avoid delays, and cost overruns [12].

Traditional construction inspection practices rely mainly on human-based visual inspection, which can be labor-intensive and time-consuming [14,15,16]. The inspectors and decision-makers are not immediately provided with the information required to address the issues as they arise on-site [17]. The conventional inspection and site observation methodologies are manual human observations, representing an antiquated procedure requiring attention and remediation [18]. Current on-site inspection methodologies continue to rely on paper-based documentation, such as drawings and data collection forms, which have been proven several times to be inefficient for information dissemination and communication, often resulting in oversight of crucial matters and delays in on-site decisions [19]. However, these processes are still error-prone, costly, and time-consuming, leading to inefficiencies and safety risks [17]. The prevailing construction inspection and monitoring practices have inefficiencies that are part of cost overruns in 66% of construction projects and the schedule delays in 53%, affecting USD 1.4 trillion worth of construction in the United States [12]. There have been efforts to develop automatic inspection systems to minimize these issues and inefficiencies, and these systems have been further categorized into three groups: detection, visualization, and interpretation [20]. A substantial amount of research has been conducted on defect detection [21,22] and evaluation [23,24], but there is still a lack of work done on visualization [25]. The existing visualization systems lack an interactive visual user interface for inspection and monitoring [25]. In the case of segment displacement inspection during tunneling construction, AR requires several markers [26]. MR generally requires fewer markers than AR, where markers are used as reference points or visual cues to anchor virtual content onto the real-world environment. These markers act as triggers for the AR system to overlay virtual objects or information onto specific physical locations. In contrast, mixed reality seamlessly blends virtual and real-world elements, creating a unified and interactive environment. MR systems often utilize spatial mapping, depth sensing, and object recognition technologies to understand and integrate virtual objects into the real-world context without relying heavily on markers. This enables users to interact with virtual content more naturally without needing physical markers as reference points.

MR is a promising emerging technology [27]. This technology has the potential to overcome these challenges by providing immersive and interactive visualization, as well as enabling remote collaboration and real-time monitoring. Although AR is utilized these days for efficient visualization of inspection results [19], MR is a broader concept than AR and encompasses both real and virtual worlds, covering a continuum from AR to AV [28]. MR combines virtual and real-world elements, allowing users to interact with digital objects in a physical environment [28]. It is an advanced immersive technology that combines VR and AR elements. It is a combination of real-world and computer-generated data, where computer-generated graphical objects are visually mixed into the real environment and vice versa in real time [29]. Unlike VR, which creates an entirely virtual environment, or AR, which overlays virtual objects onto the real world, MR seamlessly integrates virtual and natural elements [30]. This creates an immersive experience where virtual objects can interact with the natural environment in real time. MR technology can be experienced using headsets like the Microsoft HoloLens or through applications on smartphones and tablets [31].

MR’s potential applications in building construction are vast [10], including to conduct building inspections effectively [32], to conduct construction defect management with the help of BIM [33], to facilitate pipe assembly [34], to improve construction safety [35], to enhance construction design collaboration [36], and to improve performance in building design projects [37]. Inspectors can visualize complex 3D models of buildings, inspect hidden elements like plumbing and electrical systems, identify discrepancies, and make decisions based on real-time data [38]. MR can also facilitate better communication and coordination among various stakeholders, from architects and engineers to contractors and clients [39]. MR can provide immersive visualization, enable remote collaboration, increase efficiency and accuracy, reduce inspection time, and increase the accuracy of measurements [40]. Despite these potential benefits, the adoption of MR in construction inspection and monitoring is still in its early stages, with various challenges to overcome.

This systematic review aims to investigate the current state of research on MR in building construction inspection and monitoring. It reviews the existing applications, challenges, and future trends and by doing so, it aims to provide a better understanding of the potential of this technology and its implications for the construction industry. Investigating its applications in the construction industry can help contribute to the advancement of its technology and its integration into the construction industry. This research highlights the significance of virtual technologies in enhancing the efficiency and accuracy of inspection and monitoring, along with increased communication and collaboration among the project stakeholders. The advent of such technologies has helped inspectors and other project team members with better and timely decision-making and enhanced problem-solving. This study is a seminal reference point for future research in the domain of MR relating to building construction inspection and monitoring.

2. Methodology

This research was conducted using the systematic literature review technique. To ensure the credibility and rigor of this systematic review, we adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [41]. The PRISMA methodology is a widely accepted framework that aids in the identification, screening, eligibility assessment, and inclusion of relevant studies [42]. This approach ensures high transparency and replicability in the review process. A comprehensive search of academic databases (e.g., Scopus, Google Scholar) was conducted using the following keywords: mixed reality, building construction, inspection, and monitoring. The search was limited to articles published between 2018 and 2023 and written in English. The search resulted in several articles screened based on title, abstract, and full-text eligibility criteria. The eligibility criteria were (1) the article must investigate the use of MR in building construction inspection and monitoring, (2) the article must report empirical findings, and (3) the article must be written in English. The final sample consisted of 22 articles that met the eligibility criteria.

2.1. Search Strategy

Our literature search spanned several academic databases that included Google Scholar and Scopus. The databases were selected for their extensive coverage of technology-, engineering-, and construction-related literature. The period for the search was limited to articles published between 2018 and 2023 to ensure the inclusion of the most recent and relevant research. There is limited research on the use of MR in building construction inspection and maintenance, as the number of research articles shortlisted from Google Scholar was 15 and from Scopus 8. The search terms were derived from the three main components of our study: mixed reality, building construction, and inspection and monitoring processes. The search used combinations of these terms, such as “mixed reality”, “building construction”, “inspection”, “monitoring”, and “construction inspection”, to maximize the chances of finding all relevant articles. Similarly, the search also used Boolean operators like “AND” and “OR” to refine the search further to combine or exclude specific terms. The exact search string was (TITLE-ABS-KEY (“mixed reality” AND “construction inspection” OR “infrastructure inspection” OR “AEC inspection” OR “building inspection” OR “inspection of building” OR “building monitoring” OR “monitoring of building”)).

2.2. Screening and Selection

The search results were initially screened by reviewing the titles and abstracts. This preliminary assessment helped to eliminate articles that were irrelevant to our research topic. The remaining articles underwent a full-text review to evaluate their relevance in greater detail. The inclusion criteria for the articles were as follows:

• The study must focus on using MR technology in building construction.
• It should provide insights into applying MR in inspection and monitoring processes.
• The study must be an original research article or a case study.
• In the case of multiple articles, duplicates were removed.
• Articles not written in English were also excluded.

Through this rigorous and systematic methodology, we aimed to provide a complete and unbiased overview of the current state of MR applications in building construction inspection and monitoring. Hence, we landed at a limited number of 22 research articles as shown in Figure 2. The following sections detail the findings of this systematic review.

3. Results

The data presented in Figure 3 depict a significant increase in the number of articles published annually on immersive technologies, particularly MR, from the year 2018 to 2019. However, an increase in the applications of MR was observed as this technology was gaining attention and evolving continuously until 2019. The year 2020 witnessed a decrease in interest, which was followed by a gradual increase towards 2022. The decrease in the research could be because of various reasons, such as the maturation of technology to a certain extent, the hype around new technology settling down, the availability of funding, and most importantly, COVID-19, as it had an impact on research activities across the industry. By the year 2023, the number of publications seemed to have stabilized. This consistent number of publications each year indicates the potential longevity of the technology while highlighting the ongoing efforts being made by professionals and researchers to improve it. Technology companies are dedicated to enhancing the user experience by continually releasing better products.

The 22 articles included in this review were classified according to their research focus: mixed reality, building information modeling, inspection, and monitoring, as shown in Figure 4.

Most of the articles focused on inspection visualization, highlighting the potential of MR to enhance spatial understanding and situational awareness among stakeholders. For instance, MR can provide immersive 3D models of buildings, allowing inspectors to detect errors and hazards more accurately and efficiently [43]. Communication and collaboration were also highlighted as key benefits of MR in building construction, as it enables remote collaboration and real-time feedback among stakeholders [44,45]. Efficiency and accuracy were also reported as benefits of MR, with several studies showing that MR can reduce inspection time and increase the accuracy of measurements and calculations [8].

Figure 4. Co-occurrence network using VOSviewer [46].

Figure 4. Co-occurrence network using VOSviewer [46].

The review revealed various applications of MR in construction inspection and monitoring. One prominent use is in the visualization of 3D building models and underlying infrastructures such as plumbing, electrical, and HVAC systems [47]. This allows inspectors to cross-verify the actual construction progress against the planned model, aiding in the early detection of discrepancies [48].

Mixed reality (MR) technology is finding numerous applications in the field of building construction inspection and monitoring. This section delves into three primary areas where MR has shown significant potential—visualization, integration with building information modeling (BIM), and inspections.

3.1. Visualization

MR technology allows for creating and visualizing complex 3D models in a real-world setting, as depicted in Figure 5. The graphics used for the illustration purpose were acquired from Icons8 [49]. This capability is particularly beneficial in building construction, providing a more intuitive and immersive understanding of the building design and its underlying systems, such as plumbing, electrical, and HVAC [50]. Inspectors can use MR devices to overlay these 3D models onto the actual construction site, allowing them to see the structure and its systems as they appear when completed. This real-time spatial visualization can aid in identifying discrepancies between the planned design and the actual construction, thereby reducing the likelihood of costly errors and reworks [28]. A study demonstrated the use of MR in visualizing 3D models of a multi-story building during construction. Using an MR headset, the inspectors were able to walk through the site and visually compare the actual construction progress with the planned 3D model. The study found that this approach significantly improved the detection of construction errors and enhanced the overall efficiency of the inspection process [51].

3.2. Integration with Building Information Modeling (BIM)

Another application is the integration of MR with BIM, which allows stakeholders to visualize BIM data on-site, improving coordination, communication, and decision-making as shown in Figure 6. The digital model serves as a shared knowledge resource for facility information, facilitating effective communication and collaboration among stakeholders involved in the design, construction, and operation of the building. BIM is increasingly adopted in the construction industry because it can improve project coordination, reduce errors and omissions, and streamline workflows [52]. MR technology can be integrated with BIM to bring this digital data into the physical world. Using MR devices, stakeholders can visualize BIM data on-site, significantly improving their understanding of the project and aiding decision-making [53]. An example is demonstrated in a study where they developed an MR application that could display BIM data in a real-world environment. The application allowed users to visualize architectural designs, structural elements, and system layouts as they appear in the real world. The study found that the MR–BIM integration enhanced communication among project teams, facilitated better project coordination, and improved the overall project outcomes [54].

3.3. Inspections

MR can enable inspectors to visualize potential hazards in the construction environment, as shown in Figure 7. For example, it can overlay safety information onto real-world objects, such as indicating areas where fall protection is required or areas that are at risk of being hit by moving equipment [55].

Safety is critical and one of the major concerns in the construction industry, with construction sites being among the most hazardous workplaces. MR technology can significantly enhance safety inspections and training [48]. An MR-based safety inspection system was developed that could identify and visualize potential hazards on a construction site. The system used a database of common construction hazards and their visual indicators, and when used with an MR headset, it could automatically detect and highlight these hazards in the user’s field of view. The study found that the system significantly improved the efficiency and effectiveness of safety inspections and served as a valuable training tool for educating workers about construction safety [56]. The use of MR in safety inspections is also promising, enabling the visualization of potential hazards and safety information overlaid in the natural environment [57]. In conclusion, MR technology is being increasingly utilized in building construction inspection and monitoring, with applications ranging from 3D model visualization and BIM integration to safety inspections.

These applications have shown significant potential to improve construction processes’ efficiency, accuracy, and safety. However, as with any emerging technology, some challenges and limitations need to be addressed, which we will discuss in the following section.

4. Challenges

Despite the evident potential, several challenges persist, as demonstrated in Figure 8. Technological limitations, such as the limited field of view, inadequate tracking, and graphical glitches, hamper the user experience. There are also significant adoption issues, including the high cost of MR devices, lack of technical skills among users, and resistance to change from traditional methods. Data privacy and security also pose significant concerns. As promising as the application of MR is in the realm of building construction inspection and monitoring, its implementation is not without challenges. Several issues span technological limitations, adoption barriers, and security concerns, which need addressing to achieve broader, more practical applications.

4.1. Technological Limitations

One of the primary challenges in implementing MR technology in construction inspection and monitoring is the current technological limitations. These include a limited field of view, inadequate tracking, and graphical glitches. The limited field of view of many MR devices often restricts users’ spatial understanding of virtual information overlaid in the real world, limiting the overall user experience. This can make it challenging considering safety to view large-scale 3D models or move around a construction site using an MR device [58,59]. Furthermore, tracking inaccuracies can cause misalignment between the virtual and physical worlds. This can be particularly problematic in outdoor construction environments where GPS signals may be unstable and tracking markers or beacons may be challenging to place [60]. Graphical glitches such as latency, low resolution, or poor rendering can also affect the user experience and the practical utility of MR in construction inspection and monitoring. One of the reported technological issues was the hardware often turning off [43].

4.2. Adoption Barriers

Even with advanced technology, adoption issues pose significant challenges. High costs associated with MR devices, particularly advanced headsets, may be prohibitive for many construction firms, particularly small to medium-sized enterprises [61]. Some argue that the weight of the technology should be addressed [62]. Additionally, there is often a lack of technical skills among construction professionals to use MR devices effectively. Users must be trained to become comfortable with MR interfaces and interpreting MR visualizations correctly [45]. Despite the potential benefits, many construction professionals may prefer standard processes and tools over new technologies that require learning new skills and adapting to new working methods.

4.3. Security and Privacy Concerns

Security and privacy are notable concerns in the application of MR in construction inspection and monitoring. The use of MR involves collecting, processing, and storing large amounts of data, some of which can be sensitive, such as detailed building designs and infrastructure details. Ensuring the security of these data is a significant challenge. Moreover, privacy issues arise when devices are used to capture images or videos on construction sites [63]. These devices may inadvertently capture individuals in their field of view, raising concerns about privacy and consent [44].

In conclusion, while MR offers significant potential for improving building construction inspection and monitoring, several challenges need to be addressed. Overcoming these will require technological advancements, training and education, policy and regulatory considerations, and robust security measures. The following section will discuss potential future trends and the ongoing improvements to address these challenges.

5. Future Trends

Based on the research and analysis, the authors assert that the integration of MR technology in construction inspection and monitoring holds significant promise for the future. As such, they put forth the recommendation for the implementation of MR shown in Figure 9. Advanced tracking technologies are expected to improve the accuracy and reliability of MR applications. Increasing affordability and user-friendliness of MR devices may also drive broader adoption. Further integration of MR with other emerging and developing technologies, such as robotics, the Internet of Things (IoT), artificial intelligence (AI), and 5G, could pave the way for more sophisticated applications [57].

The application and technology of MR are evolving rapidly, promising to address existing challenges and provide even more advanced capabilities for building construction inspection and monitoring. This section explores three critical future trends: improvements in MR technology, integration with other emerging technologies, and increasing standardization and regulation.

5.1. Improvements in MR Technology

Technological advancements are expected to address many of the current limitations of MR. Future MR devices will likely offer more realistic fields of view, better tracking accuracy, and improved graphical capabilities. This will enhance the user experience, provide more accurate overlays of virtual and real-world elements, and enable more sophisticated visualization of 3D models and building information modeling (BIM) data [64]. Moreover, advancements in hardware, such as lighter, more comfortable headsets, will make the technology more practical for use on construction sites. Similarly, improvements in battery life will enable longer use, which is crucial for comprehensive inspections and monitoring tasks [65,66].

5.2. Integration with Other Emerging Technologies

The integration of MR with other emerging technologies is another promising trend. For instance, the combination of MR with artificial intelligence (AI) and machine learning (ML) can enable innovative inspection and monitoring systems that automatically detect construction errors or safety hazards [67]. Similarly, the integration of MR with Internet of Things (IoT) devices can provide real-time data from various sensors installed on a construction site. This can allow inspectors to monitor aspects like structural integrity, temperature, humidity, or noise levels in real time, overlaying this information on their MR view of the site [57]. Drones equipped with MR capabilities can offer a bird’s-eye view of the construction site, allowing inspectors to access hard-to-reach areas and providing them with unique perspectives that can aid in their inspection tasks [68].

5.3. Increasing Standardization and Regulation

As MR becomes more prevalent in the construction industry, there will likely be an increase in standardization and regulation. This can include standards for MR data formats, interoperability, safety, privacy, and data security regulations. Standardization can facilitate the broader adoption of MR by ensuring that different systems and devices can work together seamlessly. It can also improve the quality and reliability of MR applications [69]. Regulations, on the other hand, can help address privacy and security concerns. By providing precise data collection, storage, and sharing guidelines, rules can ensure that MR is used responsibly and that sensitive information is adequately protected [70].

In conclusion, the future of MR in building construction inspection and monitoring looks promising. With advancements in technology, integration with other emerging technologies, and increasing standardization and regulation, MR has the potential to revolutionize this field. However, continuous research and development and collaboration among various stakeholders will be crucial to realize this potential fully. The following section provides a critical discussion on the status and future potential of MR in construction inspection and monitoring.

6. Discussion

Traditional building construction inspection and monitoring have been recognized as having certain shortcomings, necessitating improvements in the existing procedures. The integration of state-of-the-art technologies into construction projects has resulted in notable advancements. Virtual technologies, particularly in visualization tasks like inspection and monitoring, have shown promise despite being in the early stages of development. Various technology companies, including Hewlett Packard (HP), Microsoft, and others, have been actively involved in the development of new and advanced virtual headsets. MR, in particular, has emerged as a promising technology with the potential to enhance the efficiency, accuracy, and safety of construction inspection and monitoring, as shown in Figure 10.

However, there are existing concerns and challenges that must be addressed to fully realize its benefits. Research efforts should focus on developing more robust and user-friendly MR solutions, addressing data privacy and security concerns, and studying the organizational and human factors influencing MR adoption. Across various industries like manufacturing, healthcare, education, training, entertainment, gaming, and military, MR has demonstrated significant potential. The construction industry stands to benefit similarly, especially in the domain of inspection and monitoring, where MR’s ability to blend physical and virtual worlds facilitates visualization of complex 3D models, the integration of BIM, and safe inspections. Although MR applications in construction are still evolving, this systematic review indicates its potential to improve building construction inspection and monitoring processes, offering immersive visualization, enabling remote collaboration, increasing efficiency and accuracy, reducing inspection time, and improving measurement accuracy.

7. Conclusions

The systematic literature review conducted in this study provides valuable insights into the potential benefits of MR technology in the context of building construction inspection and monitoring. By integrating this virtual technology, construction projects can experience enhanced efficiency, accuracy, and safety. The main findings of this study underscore the significant role of MR in visualizing complex 3D models, facilitating remote collaboration, and increasing the accuracy of measurements during inspections. Moreover, MR’s ability to blend physical and virtual worlds presents a unique opportunity to improve decision-making processes in construction projects. The value this study adds to the body of knowledge lies in highlighting the potential of MR as a transformative technology for the construction industry. By leveraging MR’s capabilities, stakeholders can achieve better construction outcomes, leading to improved project delivery and resource utilization. However, it is essential to acknowledge the limitations of this study. The research in the field of MR in construction is still in its early stages, and while this systematic literature review provides valuable insights, it may not encompass all the latest developments and innovations in the field.

Considering this study’s findings, some specific and clear recommendations for future research can be made at both the macro and micro levels. At the macro level, researchers should focus on conducting large-scale empirical studies to assess the real-world impact of MR adoption on construction projects. Such studies can provide valuable data on the cost-effectiveness, safety improvements, and overall project performance related to MR implementation. At the micro level, research efforts should concentrate on addressing the current challenges and concerns associated with MR technology, such as data synchronization, latency, performance issues, etcetera. Developing more user-friendly interfaces and conducting usability studies with construction professionals will be essential to ensure seamless integration and successful adoption of MR solutions. Furthermore, exploring the organizational and human factors that influence the acceptance and implementation of MR in construction projects should be a priority for future research. Understanding the barriers to adoption and developing strategies to overcome them will be critical in maximizing the benefits of MR technology in the construction industry.

In conclusion, this systematic literature review highlights the significant potential of MR technology in improving building construction inspection and monitoring processes. By addressing the identified challenges and focusing on future research, the construction industry can embrace MR as a transformative tool to achieve better outcomes and enhance overall project performance. Mixed reality presents a significant opportunity for revolutionizing building construction inspection and monitoring. While technology is still nascent, with several challenges to overcome, its potential cannot be overstated. The construction industry should closely watch the development of this technology and actively participate in shaping its future.