A Synthetic Review of UAS-Based Facility Condition Monitoring

Abstract

Facility inspections are mainly carried out through manual visual inspections. However, it is difficult to determine the extent of damages to facilities, and it depends on the subjective opinion of the manager in charge of the monitoring. Additionally, when inspectors inspect facilities that cannot be safely accessed, such as high-rise buildings, there are high risks of fatal accidents. For this reason, the construction industry conducts research into unmanned aircraft system (UAS)-based facility inspections. These studies have been focusing on developing the technologies or processes for using UAS in facility condition monitoring, ranging from infrastructure systems to commercial buildings. This study conducted extensive and synthetic reviews of the recent studies in UAS-based facility monitoring using a preferred reporting items for systematic reviews and meta-analysis (PRISMA) method. A total of 32 papers were selected and classified through the types of facilities and the technologies addressed in the studies. This paper analyzes the trends of recent studies by synthesizing the selected papers and consolidates the further directions of UAS applications and studies in facility monitoring domains.

1. Introduction

The global outbreak of COVID-19 has slowed down the economics of the construction industry, resulting in reduced short-term construction demand and a negative impact on long-term investments [1]. McKinsey & Company recently emphasized the need to accelerate the digitalization of the construction industry to improve its [2]. With digitalization, the construction industry can expect a reconstruction of the supply chain that can improve elasticity in the short term, and increase investments in information technology integration, expand off-site construction, and improv environmental sustainability in the long term [3]. The construction industry is improving both productivity and safety by using fourth industrial revolution technologies, such as unmanned aircraft systems (UAS), artificial intelligence (AI), cloud computing, virtual reality (VR), or augmented reality (AR) [4]. Among the various fields that can use those technologies, construction is expected to be the one of the sectors that can have huge benefits from using them [5]. Facilities require regular and precise inspections, thus requiring the production and management of inspection data. For example, case studies by the Teal Group (2016) and EuroConsult (2020) found that the facility inspection and construction industry have high potentials for the use and commercialization of UAS in the next 10 years [6].

Facility inspections are currently conducted by visual inspections. Second inspections with testing devices might be required when the first inspection that visually conducted by the inspector is not sufficient to determine the condition of the facility [4,7,8]. Since the visual inspection is conducted from the subjective perspective of the inspector, there are limitations in obtaining objective visual data on damages to facilities [4]. Therefore, it is difficult to conduct objective analysis and make decisions based on the data. Moreover, when inspecting areas that are difficult for people to access, there is the risk of human casualties [4,7]. To address these challenges, UAS-based facility inspection has been introduced. For example, FLYABILITY uses UAS to conduct the inspection of the narrow space under bridges. This resulted in a significant drop in costs compared to past visual inspections while reducing inspection time by at least 25% [9]. Another example is the deployment of DRONEID to the site of the Russian cruise which crashed into the Gwangan Bridge in Busan on 29 February 2019 to investigate the damages and continue to monitor the crash site [10].

UAS can fly near facilities using various sensors to capture images as digital image data and can improve the efficiency and safety of facility maintenance status inspections. Therefore, many studies have been published on developing UAS-based facility condition monitoring by adopting deep learning [11,12] or image processing [13,14]. Likewise, there is a huge body of studies on UAS applications in the facility monitoring domains, which contribute to providing new applications as well as a better understanding of how those various UAS applications are used to collect and analyze the visual data. Additionally, the studies address the various types of facilities (e.g., bridge or building) to adopt the technologies However, it is necessary to conduct a secondary study (Level 2 shown in Figure 1) that can synthesize the context of the various UAS applications according to the type of technology used and facilities, as well as consolidate the further directions to address the current challenges. Therefore, the main goal of this study is to systematically synthesize and review the UAS-related first studies (Level 1 shown in Figure 1) in facility monitoring contexts to provide an overview of the future. This study adopted a preferred reporting items for systematic reviews and meta-analysis (PRISMA) method to examine the current state of UAS-based facility inspection applications and further research directions.

2. Facility Condition Monitoring

Currently, most facility condition inspections are conducted by an inspector who performs visual inspections [7]. This method requires a lot of time and expense, and it is challenging to access skyscrapers and underneath bridges. In addition to such problems, buildings are becoming higher and more significant compared to the past, and, with time, the number of old facilities requiring periodic condition monitoring has increased [8]. Therefore, it is necessary to find efficient and economic schemes in addition to the current visual inspections, and UAS-based facility condition monitoring has recently been introduced to solve these issues.

Research has explored UAS-integrated monitoring systems for various facilities, such as airports [13], bridges [7,15], and buildings [16,17]. Furthermore, advanced data processing techniques, such as photogrammetry, image processing [13,14], and deep learning [11,12] are used to identify damages to facilities. Among them, research was conducted to visually identify damages to bridges, such as cracks, efflorescence, detachment, etc., based on images captured using UAS [18]. The most common research focus was visually identifying damages to facilities using UAS, and surveys are also conducted on various facilities other than bridges [19,20]. Additionally, there has been research on identifying damages to facilities using point cloud data collected using UAS and conducting 3D modeling on it [21]. 3D modeling of facilities through point cloud data is being used more intensively as a part of UAS-based facility inspections. Deep learning-based study was also introduced to identify cracks automatically [22]. It can also be very effective for resolving problems that can occur while manual facility inspections are conducted. In addition to examining facility damages and inspection, various other studies are conducted, such as economic analysis, establishing optimal flight paths for autonomous UAS, and predicting risk factors during UAV inspection [16,22,23,24]. Likewise, there are numerous studies on developing UAS applications in facility monitoring to overcome the limitations of the manual labor-intensive inspection method. It has been proven that the UAS can be utilized as an efficient, accurate, and economic tool for this purpose.

3. Research Methodology

This study used a PRISMA method to identify recent research on UAS-based facility inspections. The method can systematically document the extensive knowledge presented and help to reduce the time and duplications of comprehensive reviews [25]. Following the PRISMA method, this study had three main steps: (1) selection of the studies; (2) synthesis of current knowledge; and (3) suggestion of further research directions.

To select the studies, the Google Scholar search engine was used for automated searches by using the Boolean operators and search strings identified. The keywords include abbreviations, such as ‘UAS’, ‘UAV’, and ‘drone’, which can be used synonymously for ‘unmanned aerial vehicle’. In addition, words used in facility inspection, such as ‘inspection’, ‘maintenance’, ‘building’, and ‘facility’, were selected as keywords. A total of 534 papers were searched, combining the aforementioned keywords. In this study, inclusion criteria and exclusion criteria were established to select the articles (as shown in

Table 1. Inclusion and exclusion criteria.

Criteria	Content
Inclusion Criteria	Papers drafted in the past 5 years
	Written in English
	Primary research paper
	Papers related to UAS-based facility inspections
Exclusion Criteria	Papers drafted prior to the past 5 years
	Conference, Book, Thesis, Report
	Secondary research paper
	Papers not related to facility inspections
	Papers on monitoring building energy and heat

).

The PRISMA-based article selection process is shown in Figure 2. First, to analyze the state of recent research, papers drafted in the past five years, from 2017 to 2021, were selected, and earlier papers were excluded. Second, among articles drafted in the past five years, published journals written in English were selected. Articles presented at academic conferences, books, diploma theses, research reports, etc., were excluded. Third, primary research articles, referring to scientifically described papers on technology development and its validation, were included, and secondary research articles were excluded. Finally, papers that were not UAS-based or are unrelated to the facility level study were excluded. Through the paper selection process, 32 papers were finally selected.

4. Synthesis of Research in UAS-Based Facility Monitoring

A total of 32 papers were selected in this study, and the type of facility and technology were categorized and coded for synthesis. The facility was categorized as ‘bridges (BR)’, ‘buildings (BU)’, and ‘other facilities (OT)’, such as transmission towers and airports, and the types of applied technologies were categorized as ‘visual identification (VI)’, ‘3D modeling (3D)’, and ‘automated identification (AI)’. VI was defined as a technology to identify damage to facilities using digital images captured by UAS, and AI was defined as technology to automatically detect damage to facilities using deep learning for captured images. 3D is a type of technology that enables 3D modeling, including the point cloud of buildings based on captured UAS images to identify damage to facilities. The papers described the technical issues (e.g., process, factor analysis) were coded as unclassified (UN).

Table 2. Classification of research by type of facility and technology.

Identifier (Reference)	Type of Facilities			Type of Technologies *
	Bridge (BR)	Building (BU)	Other Facility ** (OT)	Visual Identification (VI)	3D Modeling Technology (3D)	Automatic Identification (AI)	Unclassified *** (UN)
J01 (Liu et al., 2021) [7]	√				√
J02 (Sauti et al., 2018) [24]		√		√
J03 (Han et al., 2018) [17]	√				√
J04 (Lee et al., 2019) [11]	√				√
J05 (Grosso et al., 2020) [22]		√					√
J06 (Chen et al., 2021) [12]		√			√
J07 (Kung et al., 2021) [26]		√				√
J08 (Ayele et al., 2020) [27]	√					√
J09 (Kim et al., 2018) [8]	√						√
J10 (Cho et al., 2019) [28]		√		√
J11 (Freimuth et al., 2019) [29]		√			√
J12 (Jeong et al., 2019) [13]		√			√
J13 (Lee et al., 2019) [30]		√				√
J14 (Kim et al., 2021) [14]			√	√
J15 (Yang et al., 2021) [31]		√			√
J16 (Yusof et al., 2020) [6]		√		√
J17 (Jeong et al., 2020) [32]			√				√
J18 (Kim et al., 2019) [33]		√			√
J19 (Falorca et al., 2021) [34]		√		√
J20 (Akbar et al., 2018) [19]		√				√
J21 (Bolourian et al., 2020) [35]	√				√
J22 (Ruiz et al., 2021) [18]		√			√
J23 (Peng et al., 2021) [21]	√					√
J24 (Aliyari et al., 2021) [16]	√						√
J25 (Mattar et al., 2018) [20]			√			√
J26 (Bae et al., 2019) [36]		√				√
J27 (Aliyari et al., 2021) [23]	√						√
J28 (Falorca et al., 2021) [37]			√	√
J29 (Kaiwen et al., 2019) [38]		√					√
J30 (Seo et al., 2018) [39]	√			√
J31 (Siyuan et al., 2019) [4]	√				√
J32 (Besada et al., 2018) [15]			√				√

* The AI comes first if it overlapped with the 3D or VI when it was classified. ** OT includes airport, transmission tower, outdoor billboard, tank, and smokestack. *** UN studies include economic feasibility, impact or factor analysis, and process definition studies.

describes the result of the article selection through the PRISMA process.

Based on the type of facility, a total of 16 (50.0%) studies on BU inspection using UAS, 11 (34.4%) articles on BR inspection, and 5 (15.6%) articles on OT including airports and transmission towers were identified. Based on the type of technology adopted, a total of 7 studies (21.9%) on VI using aerial images, 11 (34.4%) on 3D technology, and 7 (21.9%) on AI technique were identified. In addition, 7 (21.9%) were not classified, since they mainly described economic analysis research, video processing, process proposal, autonomous flight UAS framework, and environmental impact analysis when using the UAS. Figure 3 describes the synthesis map composed of the selected articles for the secondary review in this study.

4.1. Synthesis of Contributions of Current UAS-Based Facility Monitoring Studies

Based on the review in this study, 3D technology tends to be the most common for inspecting facilities. A total of five studies out of 11 (45.5%) focusing on the BR adopted 3D techniques to use the point cloud model through the photogrammetry process. For the BU, a total of six out of 16 studies (37.5%) used 3D for detecting and analyzing damages to buildings. However, this technique was not used for OT facilities. The 3D point cloud data can be processed through the 2D UAS aerial images of bridges, and they can successfully help to inspect the damaged parts of the bridges [21,39]. Furthermore, this model can be integrated with the BIM for establishing the AR environment for rapid inspection [7]. Han et al., 2018, documented how camera angle changes at the corners of facilities might have significant effects on the accuracy of the 3D model due to the time lag during the change. Additionally, it is important to establish the well-organized and optimized flight path of UAS to collect 2D images for processing the 3D model and analysis [27,34]. Another contribution to the building inspection (BU) after disasters was also documented. A study proposed an emergency risk assessment system using the UAS to assess the condition of buildings after earthquakes [28]. In addition to the 3D model, a recent study proposed a 4D map based on the 3D point cloud model for providing information about the facility monitoring plan execution [31]. Through the review in this study, it was also revealed that more optimal path is very critical to avoid collisions with obstacles (e.g., buildings or bridges) as well as to generate accurate 3D models for detecting the damages to facilities [27,30]. In this study, it was also pointed out that there is no study on the use of 3D models for monitoring OT facilities, such as an airport runway, transmission tower, or billboard.

For the VI, the studies contributed to how UAS-based aerial images can be used efficiently for verifying various damages, such as bridge decks, leaks, stains, and detachment [6,32]. Another contribution was proving the effectiveness, safety, and economic benefits of UAS-based digital images for facility monitoring through case studies [20,23,32,36]. It was also revealed that UAS images can be modified into mosaic images for airport runway inspection. The mosaic photo can be generated through stitching together each aerial image with the geo-information [13]. Even it is hard to detect cracks or damages, it might be useful for recognizing the design compliance of the facility. The automated damage identification (AI) is also very efficient for identifying the various defects of facilities, such as cracks, detachment, and efflorescence by using deep learning-based object detection [11,29,33,39]. Deep learning-based AI includes image segmentation and transfer learning to train the recognition of cracks using the UAS images or 3D point cloud model [12,35]. This AI technique commonly called computer vision detects the object, and this can be used for identifying cracks and calculating their width with high accuracy: about 90% [34,39].

In addition to the technology development, it was revealed that researchers are also interested in the technical OT studies (a total of five studies were identified). Based on the economic feasibility studies, the UAS can reduce costs by 4-18% compared to the traditional way [18]. Furthermore, to enhance the usability of UAS in facility monitoring, factor analysis was conducted to deduce risk level to predict and manage risk factors (low temperature, malfunctions, or collision) on UAS flights for condition monitoring [21,22], and a decision-making process for inspectors was proposed [16].

Table 3. Synthesis of current state of knowledge.

Technology	Current State of Knowledge for Facility Monitoring
VI	It was commonly contributed that UAS-based facility condition monitoring can collect the image data of point-of-interest as well as help to identify various defects with high efficiency, safety, and economic benefits.
3D	A 3D point cloud-based crack detection and estimation framework for constructed-facility condition monitoring was commonly developed. One of the papers proposed an integrated system between UAS and BIM-based AR for facility condition monitoring.
AI	The selected articles developed a framework by using a deep learning-based object recognition model for detecting the defects of the facilities. They used commonly existing datasets (e.g., COCO dataset) for transfer learning and image segmentation.

describes the synthesized review consolidating the contribution to UAS-based facility monitoring according to the technologies adopted (VI, 3D, and AI).

4.2. Synthesis of Limitations of Current UAS-Based Facility Monitoring Studies

Based on the review, this study synthesized the main limitations of the studies by the type of technologies. The studies commonly indicated the low quality and high noise level of the processed 3D point cloud model based on the UAS aerial images. It is very challenging to identify the type and extent of the damage accurately. Also, it is very difficult to obtain 2D aerial images during flight because of accidents due to nearby buildings and surrounding facilities. This might result in reducing the accuracy of the 3D model. Therefore, it is necessary to manually collect 2D images that can help to generate the highly accurate 3D model of the facilities. For this reason, the 3D technique can only be only used to identify damages’ existence and their type. In addition, this method is not automated monitoring even though the UAS can fly over the facility autonomously. The depth of damages was too limited to be accurately pinpointed and had to be reexamined visually. Another limitation is that there are no studies exploring how the 3D model can be maintained and operated after using it for monitoring. The studies pointed out that it is very critical for continuous and sustainable monitoring of constructed facilities in the next decades.

For the deep learning-based AI technique, the studies pointed out that the deep-learning model is still limited in terms of updating and continuously enhancing the accuracy because it used an existing dataset. Furthermore, for training, it still requires a huge amount of data to acquire high accuracy of automated recognition. For the VI using the aerial images, it is useful to identify the location of the crack or damages at a glance rather than detecting the size or types of the defects. Since it is determined visually by looking at the captured images, the extent of damage cannot be accurately identified. One of the studies indicated that there may be large margins of error of about 10 mm. Also, it is difficult to take pictures when the distance is narrow, or there are obstacles in the flight path. Therefore, it is very limited for facility monitoring itself. The studies pointed out that VI should be combined with the 3D or AI techniques for more effectiveness and efficient uses. The UN papers are very limited, introducing the technological and economic benefits and process development without detailed verifications.

Table 4. Synthesis of current limitations.

Technology	BR and BU	OT
VI	It is possible to identify damages to facilities, but it cannot pinpoint damages such as cracks, efflorescence, and detachment, nor the level of such damages
3D	It was pointed out that the need for more accuracy of point cloud data collected through UAS makes it challenging to identify damages to facilities accurately.
	Insufficient accuracy of point cloud data collected through UAS. Current and previous studies focused on how to improve the accuracy of the 3D modeling data without identifying damages to facilities.	-
AI	Deep-learning model that can only identify single damage types of facilities. It required large amounts of data to generalize the deep-learning model.	-
	Deep-learning model that can only identify single damage types of facilities. 3D modeling research has yet to perform much. Therefore, it is hard to identify where the damage occurred in the facility.
UN	The process development or economic benefits are not conducted with actual UAS flight case studies.

describes the synthesis of current limitations.

4.3. Conslidation of Further Research Direction

Based on the review, this study consolidated further research directions. Studies that used VI and 3D methods are very limited to automatically identifying the damage at the facilities. Therefore, it is necessary to explore how the deep-learning model can be adopted for 3D point cloud modelling for automated object recognitions. While focusing only on improving the accuracy of the 3D model for identifying the defects efficiently, it should be established how the UAS can collect the aerial images with geo-information (x-, y-, and z-coordinates) for enhancing the 3D model’s accuracy through the photogrammetry process. This could be connected to the autonomous UAS flights and optical flight plan. Therefore, it is also necessary to propose a technical manual for generating the optical flight planning before the UAS flight.

In addition, it is necessary to develop a deep-learning model based on the specific training dataset for specific constructed facility by using the UAS. Additionally, it is very important to address the database management for continuous and sustainable uses of the deep-learning model for enhancing the accuracy. For AI, it is very critical to prepare a dataset that can learn all different types of damages to facilities. Lastly, the UN studies should have further case studies for validating the process proposed and autonomous UAS inspection system, as well as verifying the economic and environmental factor analysis through actual UAS uses in facility condition monitoring cases.

Table 5. Synthesis of further research direction.

Technology	BR and BU	OT
VI	It is necessary to develop technologies that can automatically identify damages to facilities.
3D	It is necessary to develop data processing technologies to improve data collection technologies and accuracy to acquire high-quality point clouds. It is necessary to develop technologies that can accurately identify damages to facilities through 3D modeling technologies.
AI	It is necessary to develop a deep-learning model that can simultaneously identify various damages to facilities. It is necessary to develop a system that can efficiently collect and manage large amounts of data. It is necessary to develop technologies that can improve the accuracy of the deep-learning model. It is necessary to develop technologies that visually identify locations where damages occurred linked with 3D modeling technologies. It is necessary to develop technologies that can improve the accuracy of the deep-learning model.
UN	It should validate and verify the process for developed autonomous flight systems through case studies.

describes the consolidation and synthesis of further research directions

5. Discussion and Conclusions

This section comprehensively summarizes the status of UAS-based facility inspection research based on the classification and analysis results of the studies carried out in Section 4 to present directions for future research. This study used the PRISMA scientific literature review methodology to identify research contributions to UAS-based facility inspections, limitations, and further research directions. A total of 32 studies are selected and categorized into the type of facilities (BR, BU, OT) and type of technology applied (VI, 3D, AI). The studies mainly focus on recent bridge and building inspections.

Based on the review, 3D is the most frequently used technology among the types of technology used for each facility studied. Additionally, technologies for automatic identification of damage to facilities require the highest-level technique (e.g., deep learning). These two techniques are very actively employed in the facility monitoring domain for detecting cracks or defects. Since manual facility inspections have many problems, such as costs, inspection periods, accuracy, and efficiency, much research focused on the AI/3D-based inspection, mainly on buildings and bridges. Meanwhile, seven papers were not categorized as they did not correspond to the three technical categories. These papers carried out research in various fields, such as economic analysis, proposal of facility inspection processes, analysis of optimal routes for autonomous UAS, and risk analysis of surrounding environment when using UAS. Different perspectives and continued follow-up research are needed beyond the three technology categories.

There were very few studies presenting research on other facilities (OT), but they were similar to trends in the types of technology used in bridges. Through this, the AI method was very low compared to the other facilities inspected. Finally, the limitations of the technology category are deduced, and the direction of future research is presented accordingly. In the case of studies on visual identification, the analysis results show a need for research to improve technologies for accurate identification, as facility damages are visually identified based on captured images. In the case of studies using 3D modeling, the accuracy of point cloud data collected by UAS was low, and it did not identify the damages to facilities based on 3D modeling. Therefore, it is necessary to develop technologies that can improve the accuracy of point cloud data and detect facility damages directly and accurately. Lastly, in the case of research on automatic identification, the deep-learning model had the limitations of identifying only one type of damage because it requires a high level of transfer learning and high number of datasets. Accordingly, follow-up research is needed to develop a deep-learning dataset composed of the UAS aerial images not present in the currently existing dataset. This study contributed to gathering information about the recent state of knowledge in UAS-based facility inspection, documenting the common challenges in the domain, and consolidating further research directions.