Next Article in Journal
Recycled Waste Powders for Alkali-Activated Paving Blocks for Urban Pavements: A Full Laboratory Characterization
Previous Article in Journal
Comparative Study of Seismic Design and Performance of OMRF Building Using Indian, British, and European Codes
Previous Article in Special Issue
An Integrated Uncertainty-Based Bridge Inspection Decision Framework with Application to Concrete Bridge Decks
Due to scheduled maintenance work on our core network, there may be short service disruptions on this website between 16:00 and 16:30 CEST on September 25th.
Article

A Practitioner’s Guide to Small Unmanned Aerial Systems for Bridge Inspection

1
Turner-Fairbank Highway Research Center, Federal Highway Administration, McLean, VA 22101, USA
2
Department of Civil and Environmental Engineering Clarkson University; Potsdam, NY 13699, USA
3
Department of Electrical and Computer Engineering, Utah State University; Logan, UT 84322, USA
4
Durham School of Architectural Engineering and Construction, University of Nebraska-Lincoln, NE 68182, USA
*
Author to whom correspondence should be addressed.
Infrastructures 2019, 4(4), 72; https://doi.org/10.3390/infrastructures4040072
Received: 15 October 2019 / Revised: 13 November 2019 / Accepted: 15 November 2019 / Published: 19 November 2019
(This article belongs to the Special Issue The Future of Infrastructure Inspection)

Abstract

Small unmanned aerial system(s) (sUAS) are rapidly emerging as a practical means of performing bridge inspections. Under the right condition, sUAS assisted inspections can be safer, faster, and less costly than manned inspections. Many Departments of Transportation in the United States are in the early stages of adopting this emerging technology. However, definitive guidelines for the selection of equipment for various types of bridge inspections or for the possible challenges during sUAS assisted inspections are absent. Given the large investments of time and capital associated with deploying a sUAS assisted bridge inspection program, a synthesis of authors experiences will be useful for technology transfer between academics and practitioners. In this paper, the authors list the challenges associated with sUAS assisted bridge inspection, discuss equipment and technology options suitable for mitigating these challenges, and present case studies for the application of sUAS to several specific bridge inspection scenarios. The authors provide information to sUAS designers and manufacturers who may be unaware of the specific challenges associated with sUAS assisted bridge inspection. As such, the information presented here may reveal the demands in the design of purpose-built sUAS inspection platforms.
Keywords: small unmanned aerial systems (sUAS); inspection; bridge; structure; practitioner; application small unmanned aerial systems (sUAS); inspection; bridge; structure; practitioner; application

1. Introduction

The Federal Highway Administration reports that the United States is home to more than six hundred thousand bridges. As of 2016, more than half are rated as being in poor or fair condition [1] and nearly ten percent are rated as structurally deficient [2]. Pursuant to federal regulations [3], bridges in the United States require periodic inspection—typically every 24 months—to assess condition and suitability for service. These inspections can be both time consuming and costly, with some authors estimating a total biannual bridge inspection cost exceeding $2.5B in the United States [4]. In current practice, nearly all bridge inspections are manned, meaning that a human inspector must access the substructure, deck, or superstructure to perform a manual inspection. Inspections, assisted with robots, have the potential to be safer, quicker, and more economical than manned inspections [4,5,6,7].
The recent literature includes many examples of autonomous or robotic bridge inspection, highlighting the need for advanced inspection technologies that reduce reliance on manned inspections [8,9]. The increasing availability and economy of small unmanned aerial system(s) (sUAS) make them another likely tool for bridge inspection. Dorafshan and Maguire (2018) summarized the history of sUAS applications for bridge inspections by the US Departments of Transportation (DOTs). Almost 30 state DOTs have deployed sUAS for inspection or other purposes, either in practice or research [7]. The applications of sUAS can go far beyond an assistive tool for the inspector with the integration of image processing or machine learning techniques, which can be used for autonomous detection of cracks in concrete [10,11,12] or fatigue cracks in steel [13]. The feasibility and application of using deep convolutional neural networks for concrete crack detection in sUAS assisted inspection can provide a similar accuracy to human inspections [14].
Despite fairly extensive research into the potential for sUAS assisted bridge inspections, practitioners have little guidance with respect to initiating sUAS assisted bridge inspection programs. While practitioners are well aware of the challenges associated with manned bridge inspections (e.g., traffic control, access to confined spaces, fall protection), they may not be aware of those associated with sUAS assisted inspections (e.g., aviation regulations, lighting, wind, and temperature). Practitioners require guidance with respect to the available technology and how it can be harnessed to overcome these challenges. At the same time, sUAS manufacturers require guidance with respect to the unique needs of the bridge inspection community so that they can design sUAS to better meet these needs. Thus, the objective of this paper is twofold. The authors seek to leverage their experiences with sUAS assisted bridge inspection to:
  • Provide information about existing sUAS technology and the challenges associated with sUAS assisted bridge inspection that will help practitioners successfully initiate sUAS assisted bridge inspection programs; and
  • Provide information about bridge inspection needs and challenges that will help sUAS designers better meet the needs of the bridge inspection community.

2. Bridge Inspections

The United States Code of Federal Regulations (CFR) lists eight general types of bridge inspections (Table 1). Bridge inspectors look for a variety of defects or targets, depending on the type of inspection and on the type of structure. In concrete bridges or decks, inspectors may look for structural cracks, delamination, spalls, efflorescence, or other indications of deterioration like corrosion staining or map cracking. In steel bridges, inspectors may look for corrosion, fatigue cracks, or problems with connection integrity (e.g., weld corrosion or missing fasteners). Inspectors may also use various technologies to construct a 3D model of a bridge for advanced condition assessment or asset management purposes.
Figure 1 shows several types of defects observable during sUAS assisted inspections. Figure 1a is a side view of a concrete bridge deck that shows surface cracking, efflorescence (white staining), and delamination. The surface cracks visible range in widths from 0.08 mm to 25 mm, are as long as 300 mm, and are fairly easy to detect with the naked eye. Other defects could include map cracking, which is indicative of alkali-silica reaction, or corrosion staining, which could indicate corrosion of embedded steel. Connections are a major area of concern for steel bridges. Figure 1b is a view of a connection in a steel superstructure with a concrete deck on top. The figure shows some corrosion in the upper flanges and plate connection, as well as spalling in the concrete deck. In rare cases, an inspector may find missing or loose fasteners in such a location. Distortion of plates may be visible. Significant corrosion is normally observable with little effort but quantifying the affected area and the net section loss may be difficult using a non-contact approach. The defects discussed above are all fairly large and certainly large enough to be observed with the naked eye. In most cases, they are also easy to access as they tend to occur on flat surfaces above deck or on the sides of girders. Fracture Critical Member (FCM) inspections are the most difficult and expensive bridge inspections to perform since hands-on inspections are required to detect possible fatigue cracks (within the arms-length of the region of interest). Fatigue cracks are very difficult to see and may have lengths shorter than 7 mm and widths narrower than 0.1 mm [13]. Fatigue cracks normally appear in the superstructure near large cross frames, welded stiffeners, or other complex geometries, making access difficult. Figure 1c shows a welded stiffener with weld and surface corrosion, as well as a fatigue crack through the bottom weld. The fatigue crack is terminated on both ends with drilled holes, which are intended to arrest crack growth. A big challenge for commercial (off-the-shelf) sUAS for bridge inspection is the need for repeatable measurements of defects. In addition to identifying defects, inspectors must quantify them. For example, an inspector may need to record the number, location, and length of fatigue cracks, the area of spalls, or the percent section loss in corroded steel members. Defect characteristics can be estimated by comparison with objects of known dimensions, but the accuracy of these estimates depends on the distance between the sUAS and the target, the photographic quality, and the field of view. As such, there is a need to develop real-time reference measurements for sUAS assisted bridge inspection applications.

3. sUAS Sensors

This section identifies sensors that can be deployed on sUAS to identify and quantify the inspection targets, e.g., defects, in the field. A variety of remote sensing devices (sensors) can be placed on sUAS, ranging from optical cameras in the visible or infrared spectra to light detecting and ranging (LIDAR) or other advanced navigation and ranging sensors. For structural defects, sensors in visible and infrared spectra are the most common and useful.

3.1. Visible Spectrum

Most bridge inspections rely on visual inspection of the physical structure. Thus, camera sensors are most useful for sUAS assisted bridge inspection. Manned inspections require a human inspector to access the structure for hands-on observation. A sUAS assisted inspection only requires a pilot to maneuver the sUAS into such a position that a camera can capture an image of the target. In this manner, the success of the inspection relies heavily on the ability to gather clear images of the inspection targets. Aside from sufficient illumination, the inspector needs to be able to change the camera setting for imaging, such as exposure and zoom. Higher sensor resolution, platform stability, and navigability in a challenging environment lead to a more successful inspection.
Entry-level sUAS platforms at the hobbyist level often include built-in cameras with relatively low resolution. These sensors may be sufficient for detection and/or measurement of macroscopic defects (e.g., surface cracks in concrete), but are not sufficient for observation of microscopic defects (e.g., fatigue cracks). More recent sUAS platforms include built-in high definition cameras with resolutions of 12 Mega-Pixel (MP) or higher, which are needed for inspections that require the detection of fine defects (e.g., FCM inspections). Custom-built sUAS—as well as some commercial models—may not include built-in sensors. Instead, these platforms include provisions for mounting a user-supplied sensor. This allows the user to select the sensor body, optics, and other features to suit their unique needs.
In general, sUAS-mounted camera sensors for bridge inspection should support remote viewing of images in real time so that the inspector can view inspection images and make decisions during the inspection flight—a discussion on real-time vs. post-flight inspections later in this paper provides more context to this statement. Among camera settings, exposure and zoom play a crucial task for UAS-assisted visual inspections. This allows the inspector to compensate for the various lighting conditions and standoff distances encountered during the inspection. Exposure is defined as the amount of light per unit area going through a visual sensor [15]. If the camera does not receive sufficient light, the image will be under-exposed, whereas, too much light can result in over-exposed images. Both over-exposed and under-exposed situations can prevent defect detection. Figure 2a shows a common scene of over-exposure on the sUAS-mounted camera, making it impossible to observe a poorly lit weld connection. In this case, increasing the camera exposure in the next image (Figure 2b) provided a better reference to rule out the presence of crack in the weld connection. Figure 2c shows another scene in which automatic exposure settings result in over-exposure of an inspection target (fatigue crack in a weld connection). It is impossible to detect any flaws in the over-exposed image, but the same image taken with manual control of exposure settings reveals a fatigue crack (Figure 2d).
Aside from the camera, inspectors must be careful with the camera gimbal on a sUAS before purchasing them. Fixed sensors are of little use for bridge inspection. Many targets require upward (e.g., bottoms of girder flanges) or downward (e.g., bridge decks) facing cameras, which is not always an option in commercial sUAS. Needless to say, gimbaled sensors are practically a requirement for bridge inspections. Some gimbals allow sensor rotation through the horizontal plane as well as the vertical plane. These are especially helpful in confined spaces or where navigation is difficult and it is easier to rotate the sensor than the sUAS itself. Some gimbals have a restricted range of motion and it may not be possible to point the camera directly upward. The authors have had success gathering acceptable inspection images of the bottoms of girder flanges using a gimbaled camera limited to 30 degrees of upward tilt. However, a gimbaled sensor with a full range of motion is best. The vibration caused by sUASs diminishes the image quality and many camera gimbals provide some level of vibration dampening. Many camera sensors also support vibration reduction through software or hardware options.
If desired, defect detection using visual images taken by sUAS can be done in an autonomous manner, using edge detectors [10] and deep learning convolutional networks [14] for surface crack detection; 3D model reconstruction [9]. A comprehensive investigation about using different image processing, machine learning, and computer vision algorithms for defect detection using visual images can be found in [7].

3.2. Infrared Spectrum

Thermal infrared (TIR) camera sensors are frequently placed on sUAS. TIR sensors have been used since at least 1978 to identify delamination in concrete bridge decks [16]. This type of sensor can be integrated on some commercial sUAS to allow thermographic inspection of concrete bridge decks to identify delamination and other surface or subsurface defects [17]. Figure 3 shows a TIR image of a mock bridge deck, in which a subsurface delamination and a surface crack are both visible. In this figure, the surface and sub-surface defects, i.e., delamination and cracks, are both manifested as cold regions, since the cause faster heat loss than the regions with sound concrete. Multispectral and hyperspectral sensors gather data in the visible, thermal, and other spectra in a single package. While incredibly versatile, these sensors are also the most costly—often prohibitively.

4. Aviation Regulations

In the United States, the Federal Aviation Administration (FAA) regulates sUAS under Code of Federal Regulation (CFR) Title 14, Part 107 (14 CFR 107). The contents of these regulations are extensive and too lengthy to include here in their entirety, but the following FAA regulations are of particular interest for sUAS assisted bridge inspections:
  • 14 CFR 107.12 requires commercial sUAS pilots to obtain an operating license;
  • 14 CFR 107.29 prohibits nighttime operation of sUAS;
  • 14 CFR 107.31 requires the pilot and visual observer (if applicable) to maintain line-of-sight (LOS) with the operating sUAS;
  • 14 CFR 107.39 prohibits flight over individuals not directly participating in the operation of the sUAS unless the individual is protected by a structure or stationary vehicle; and
  • 17 CFR 107.51 restricts sUAS altitude to 122 m; if the sUAS is flown within 122 m of a structure, its altitude must not exceed that of the tallest part of the structure by more than 122 m.
Under the right circumstances, some of these regulations are subjected to waivers. Practitioners interested in sUAS assisted bridge inspection must familiarize themselves with all relevant regulations in their jurisdiction and must keep up to date with changing regulations. With continual progression of technology and improvement of public perception of sUAS, the trend in the past decade has been to relax these restrictions [7]. Nevertheless, some of the above regulations restrict the capabilities of sUAS for bridge inspection. It may, in some cases, be possible to obtain waivers for some regulations.
One of the primary benefits of sUAS inspection technology is the ability to inspect targets that are difficult for human inspectors to access. For instance, Florida DOT reported using sUAS to inspect high-mast poles [18]. Similarly, Minnesota DOT demonstrated sUAS applications for structural assessment of bridges [19,20]. These types of inspections can be time consuming, expensive, and dangerous when human inspectors are used. These areas can be easily and safely accessed using sUAS. Unfortunately, many such targets suffer from limited visibility, and FAA regulations require a direct Line of Sight (LOS) between the pilot and platform. Thus, some of the most difficult areas to access—those which would benefit the most from sUAS assisted inspection—may still require manned inspection or an FAA waiver.
Manned deck inspections require traffic control and lane closures. Barring FAA regulations to the contrary, sUAS assisted deck inspections could realistically be completed without any traffic control. Similarly, inspections of the superstructure could be completed more safely with sUAS than with human inspectors. However, in both cases, the prohibition of sUAS flight over active traffic and pedestrians negates the potential benefit of sUAS assisted inspection. In most cases, state DOTs may go even further and prohibit flight near active traffic to prevent driver distraction. Many sUAS assisted inspection missions may therefore still require traffic control and/or lane closures.
Finally, the prohibition of nighttime flight limits the effectiveness of TIR inspections. TIR inspections are most informative when the inspector can compare two TIR images taken at opposite temperature extrema (i.e., thermal gradient or thermal inertia) [16]. For best results, the inspector should compare a TIR image taken during the coldest part of the night with another TIR image taken during the hottest part of the day. When nighttime flight is impermissible, the best course of action is to take one image in early dawn while the temperature and structure are still cool, and another during the hottest part of the day.
For a more descriptive limitations imposed on sUAS assisted bridge inspections, the readers can refer to current review papers on sUAS bridge inspection [7,21].

5. Challenges in the Field

This section summarizes the challenges faced by the authors during field bridge inspections using sUAS.

5.1. GPS-Denied Operations

Almost all sUAS rely on a Ground Positioning System (GPS) signal for navigation. Navigation in this case is the act of moving the sUAS from one spatial location to another and is separate from the inertial (stabilization) functions of a craft. When the GPS signal is blocked, as it will be when the sUAS is beneath the structure, alternate forms of navigation assistance are required. GPS-denied operation is still considered an open problem in designing commercial sUAS platforms. Other stabilization and obstacle avoidance schemes are available, including visual-based guidance and SONAR-based surface avoidance which set a limit on the sUAS distance to the surrounding objects. These systems would allow for better inspections by offering protection in close-up maneuvers, such as the inspection of fatigue cracks, which require a close standoff distance between the platform and target. The authors have achieved success using downward-facing stereo-vision positioning. The stereo-vision positioning system includes two cameras to find a fixed pattern, usually the ground, which would allow positioning with respect to that pattern when GPS signals are lost. However, when flying over running water (a common occurrence in sUAS assisted bridge inspections) the stereo-vision positioning system caused the sUAS to follow the current, making inspection nearly impossible. Stereo-vision positioning systems also struggled over uniform surfaces (e.g., wavy grass or fine gravel and sand). The authors have increased their previous success with such systems by placing discrete random targets below the bridge, which greatly improves the performance and stability of the craft by creating a distinct pattern for the downward cameras [6,22].

5.2. Platform Stablity

Assuming that the pilot can navigate the sUAS into position to inspect a selected target, the next step is to obtain an image of that target with sufficient clarity to assess its condition. Mechanical vibration within the sUAS platform during flight does not normally hinder flight stability but can limit image clarity. Wind can also make the pilot’s job exceedingly difficult. In a mock inspection of a bridge on Utah State University campus, the authors found that moderate wind speeds (10 m/s) increased the achievable standoff distance between the platform and target by a factor of three, from about 0.3 m without wind to about 0.9 m [5,6]. This is even more significant in the under-bridge environment, where turbulence and other aerodynamic phenomena make for unpredictable wind effects. In the aforementioned study, turbulence caused major instabilities even without discernible wind, often forcing the pilot to retreat to a safe distance and repeat the approach to the target. Smaller and lighter sUAS platforms provide longer flight times and allow access to more confined spaces. This may be desirable for inspections of the superstructure requiring access between girders or in the midst of other obstructions. However, there is an intrinsic trade-off between platform size and stability. Smaller and lighter platforms are more sensitive to wind. The provision of sensors with image stabilization can further compensate for unwanted platform vibration.

5.3. Access and Obstruction

Pilots and inspectors will encounter significant obstructions during sUAS assisted bridge inspections. Access to the under-bridge environment is particularly difficult, even without the LOS requirements discussed above. Observation of some defects will require the sUAS to fly above the target, which can be very difficult. Many defects will appear in corners or at connections, which limits the achievable standoff distance (Figure 4a). Cross-frames and bracing can further obstruct access (Figure 4b). The under-bridge environment is also particularly fraught with cobwebs, nests, feces, and other debris. A bird’s nest obstructs the inspection target in Fig 4b, and a wasp’s nest obstructs the target in Figure 4c. Dirt, corrosion, or graffiti may also obscure the target. In one case, the authors were unable to detect a fatigue crack because it was obscured by markings made during previous manned inspections. Where a human inspector can easily clear debris or other obstructions, this is not an option with sUAS assisted inspections yet. Girder flanges are a likely home for wildlife–protected birds and have been known to hinder under-bridge inspections. The adoption of sUAS assisted inspection technology does not mitigate these concerns, which, in many cases, may result in a failed or unsuccessful inspection.

5.4. Flight Time

Typical flight times for sUAS range from 10 to 30 min. Bridge inspection times vary greatly, from 15 min to several weeks. Flight times can be shortened due to wind/weather. A manned inspection team may be able to inspect a small bridge deck in its entirety in 15 min. Inspections in the congested under-bridge environment require more careful piloting and therefore, an exponentially longer time commitment. In a sUAS assisted FCM inspection of a bridge over the Fall River in Idaho, the authors were only able to inspect 3-4 targets per flight using an sUAS platform with a 15-min flight time [6,7]. These targets were the locations that were susceptible to fatigue cracking based on previous field inspections (connections between the longitudinal girders and transverse beams as seen in Figure 4a). A windy day or extreme cold weather can increase the amount of maneuvering by the pilot and drain the batteries quickly. It is therefore recommended to have auxiliary batteries and heating packs, self-heating batteries, and on-site charging to ensure a successful inspection.

6. Real-Time and Post-Flight Inspections

There are two general types of sUAS assisted inspections: real-time and post-flight. In real-time inspections, the inspector views images during flight through a remote link to the camera sensor and identifies defects during the inspection. In post-flight inspections, images (videos) are saved on a hard drive or a cloud and the inspector views them after the inspection flight is completed. In most cases, a combination of real-time and post-flight inspections will give the best results, although post-flight inspections increase the total required inspection time [13]. The authors performed a sUAS assisted inspection at the S-BRITE inspection training center at Purdue University with the intent of identifying fatigue cracks during real-time and post-flight inspections. The number of false positives during the real-time inspection was up to 50% higher than during post-flight inspections [22]. Knowing that they would be able to review images again during the post-flight inspection, the inspectors tended to err on the side of caution during the real-time inspection.

7. Case Study

FCM inspections are performed to identify and track the growth of fatigue cracks and section losses in steel members. These are the most time-consuming and expensive inspections [23], making them an attractive goal for sUAS assisted inspection. In this section, the results of a sUAS assisted inspection of a bridge in Ashton, Idaho are presented and compared to the manned inspection. The bridge is in Ashton, Idaho, and carries Ashton-Flagg Ranch road traffic over the Fall River (ITD Bridge Key 21105). As a case study, a bridge with FCMs was inspected using hands-on and sUAS assisted methods. Only the results of the inspections are provided in this section for brevity. For full discussion, readers are encouraged to read [13].
The bridge consists of two main longitudinal frames on the Northern and Southern sides. Hands-on inspection was carried out using an Under-Bridge Inspection Truck (UBIT) in four hours to inspect the whole bridge. The cost of the inspection was $391 USD per hour, including UBIT costs which led to a total cost of $1,564 for four hours (Table 2). Separately, a DJI Mavic Pro sUAS was used to inspect the bridge. The sUAS followed the water current without pilot control due to the downward stereovision positioning system; therefore, only a quarter of the fatigue prone locations were inspected using an sUAS. This included 12 susceptible connections in four floor beams, two girder splices, a girder web, a concrete barrier, and bottom flange two girders. The sUAS assisted inspection identified the presence of fatigue cracks in two floor beam connections. These cracks have previously been detected and marked through hands-on inspections. The images from these fatigue cracks show the marker lines, but not the actual cracks. In addition, the sUAS assisted inspection ruled out the presence of fatigue cracks in other inspected regions. Other defects such as concrete delamination, efflorescence, and steel rust were detected in the sUAS assisted inspection. The sUAS assisted inspection took 4.5 h with a net flight time of 1.5 h (90 min); the inspection cost in this case was $200 per hour (pilot charges included the equipment). Considering that a quarter of the bridge was inspected in 4.5 h, the inspection costs extrapolated to the whole bridge using the sUAS would be $3600. This case study shows the hourly cost of sUAS inspection is almost half of the hourly cost of UBIT inspection, which agrees with previous studies [19,20]. However, the extrapolated sUAS inspection time was longer than the actual UBIT assisted hands-on inspection. The additional time made sUAS assisted inspection 130% more expensive than the hands-on inspection. It should be noted that the time and cost associated with using sUAS are different for various situations as outlined in other places in this paper. In addition, using sUAS may still provide benefits such as occupational safety (for both inspectors and public), minimizing lane closures, and minimizing traffic distractions, which are not quantified by this analysis.
Some of the challenges mentioned above exhibited themselves in this case study. The size of fatigue cracks combined with the poorly lit, obstructed, and GPS-denied superstructure made for a challenging inspection. The DJI Mavic fit between the girders but it was sensitive to the wind speed and the pilot had to land the sUAS several times during the inspection due to its instability. In addition to more difficulty navigating, platform instability and vibration led to difficulties in obtaining clear inspection images.
A gimbaled camera is required and some targets may benefit from an upward facing camera (e.g., girder flanges). Auxiliary lighting is recommended, which can either be ground-based (e.g., handheld spotlight or reflective tarps) or platform-based but should be able to be toggled by the operators in case the light washes out the image. The possibility of detecting defects through visual inspection heavily relies on the lighting condition; this is even more important for under-bridge inspections due to the lack of or sudden changes in background illumination. LOS may be problematic for larger bridges, so the inspection team should carefully select the location of the ground stations to maximize inspection range. LOS requirements may also be satisfied when one or several “observer(s)” are present during the inspection.
A suitable platform for sUAS assisted FCM inspection must be capable of GPS-denied operation. Alternative navigation assistance systems will help the pilot comfortably navigate the sUAS near to the target and obstructions. Due to the previously discussed issues with stereo-vision positioning, a side mounted or upward facing positioning sensor may be preferred. The authors took photos of a fatigue crack specimen provided by the Idaho DOT to assess the effects of vibration and standoff distance on the ability to detect a fatigue crack in an sUAS inspection image. A fatigue crack is easily observable in the 3 o’clock position in Figure 5a, taken from a standoff distance of 0.2 m. When the standoff distance was increased to 0.9 m (Figure 5b), the crack is nearly undetectable. Similarly, Figure 5c,d show the effect of illumination on crack detectability. These pictures show that both lack and abundance of brightness can prevent the inspectors from detecting the crack.

8. Commercial sUAS for Bridge Inspections

In this section, available off-the-shelf sUAS platforms are presented with their suitability for different types of bridge inspections. The recommended sUAS in this section are based on the authors’ experience and do not represent the whole sUAS market. Due to lack of definitive guidelines to help with the selection of sUAS, sensors, and other equipment, it can be challenging for DOTs to successfully start a sUAS inspection program. Table 3 shows several sUAS along with their general specifications, price (as of April 2019), and the potential bridge inspection applications. The price of a sUAS for bridge inspection varies significantly, depending on the purpose of the inspection, quality and quantity of the integrated sensors, and computing capabilities. Integrating thermal cameras with the existing visual sensors can increase the price of the sUAS up to three times. If a requirement of inspection is 3D model reconstruction, the size and the price of the sUAS increases dramatically due to additional need for quality sensors. Neither of these options may be necessary to complete most types of bridge inspection. On the other hand, in the case of under-bridge inspections, the sUAS must have an auxiliary positioning system, vision system, to compensate with lack of GPS signals, in order to have a successful mission. The potential applications mentioned in this table are not without the limitations and challenges discussed throughout this paper; however, the content of this table guides the bridge owners and inspectors when purchasing an sUAS and provides a variety of commercial options. Furthermore, the table does not suggest that the entire bridge inspection can be performed using only the recommended sUAS. The possible challenges during each sUAS bridge inspection are expected to vary significantly since published inspection reports with sUAS are limited.
Even though the main market of sUAS remains recreational, some of the sUAS in this table are currently being marketed as inspection platforms for bridges, buildings, mechanical equipment, etc. There are also sUAS such as DJI Matrice that can be tailored for specific tasks in DOTs. In other words, these sUAS can have required sensors and payloads to satisfy a DOT mission.

9. Steps Before Inspections

A practitioner new to this technology may be concerned that they do not know what they do not know. This section seeks to define a framework that can help such an inspector identify a path to a successful inspection. The following steps are based on the authors’ experience in performing sUAS assisted bridge inspections in the field as well as sUAS assistant scientific remote sensing for construction, agricultural and water resources engineering:
  • Define the mission requirements. What type of bridge inspection is being performed? What have previous inspections revealed? What is the target? How large is the target? What is the structural geometry, girder spacing, etc.? What obstructions exist that may limit access? What is the project budget? Where will the base station be located? Can LOS be an issue? Is traffic control required?
  • Check the local and state UAS regulations before planning the inspection in addition to the Federal laws.
  • Find a pilot. A skilled pilot is required for GPS-denied navigation. If possible, the pilot should also be an experienced inspector, which helps with communication among the inspection team. Obtaining a flight certificate is relatively convenient and simple for skilled sUAS operators.
  • Choose an sUAS platform. This decision is mainly influenced by the mission requirements. Does the structural geometry limit the physical platform size? Is there a need for GPS-denied navigation? What is the budget? Once a platform is selected, it is important to note a few other criteria: What standoff distance is achievable? What is the flight time? How long does it take to recharge batteries?
  • Take environmental condition when selecting the inspection day (especially wind speed, temperature, lighting condition, and precipitation).
  • Choose/evaluate a sensor. What sensor type is appropriate? What is the required resolution given the target size and achievable standoff distance? sUAS platform and sensor selection may need to be optimized together because sensor options are fixed among sUAS platforms. A practical method is to inspect a defect, it can be artificial as long as it represents the size and texture of a real defect, in different clearances and lighting conditions (no flights) [6,7,13]. In order to make sure that the sensor has sufficient specifications, e.g., resolution, one can calculate the normalized minimum distance to the pixel size of the images where the defect was visible (more information in [13]).
  • Perform mock (practice) inspections. Performing mock inspections in GPS-denied environment improves the performance of the pilot significantly. How long does the mock inspection last? How long will the actual inspection last? Is there clear communication between the pilot and the inspector?
  • Calibrate the inspection results. Compare the results of mock sUAS assisted inspections to results from previous manned inspections to find the sUAS inspection strength and weaknesses for various inspection requirements. Is the sUAS assisted inspection mission likely to be successful for the situation? Are improvements necessary in pilot training, platform stability, sensor resolution, or logistics?
  • Perform the inspection. Identify and report defects. Use real-time and post-flight inspection techniques if necessary.
  • Debrief. Was the mission successful? What contributed to the mission success? What were barriers to success? Was the sensor adequate? Was the platform adequate? Was navigation problematic? Does pilot training need improvement? Was weather an issue?

10. Conclusions

As bridge inspectors begin to adopt sUAS technology for structural inspections, they will face countless decisions about which sUAS technologies are appropriate for their particular needs. This paper lists some of the challenges associated with sUAS assisted bridge inspections and discusses the technology and equipment that can be used to overcome those challenges. The goal of this discussion is technology transfer from research to practice. In general, sUAS assisted bridge inspection offers more benefits when manned inspection is more physically demanding or when access by human inspectors is limited (e.g., under-bridge inspection). Inspectors should be prepared for the numerous challenges that are involved in this type of inspection, including poor illumination, limited line-of-sight, obstructed access, and GPS-denied operation. Above-deck applications of sUAS assisted bridge inspection are limited due to FAA regulations prohibition flight over active traffic. Where traffic control is not an issue (e.g., for out-of-service bridges), sUAS are well suited to all types of above-deck inspections.

Author Contributions

The authors confirm contribution to the paper as follows: conceptualization, all authors; methodology, all authors; software, all authors; validation, all authors; formal analysis, all authors.; investigation, S.D.; resources, M.M.; data curation, all authors; writing—original draft preparation, S.D.; R.J.T.; M.M.; writing—S.D.; R.J.T.; M.M.; visualization, all authors; supervision, M.M.; project administration, M.M.; funding acquisition, M.M.

Funding

This research received no external funding.

Acknowledgments

Special thanks to the members of AggieAir at the Utah Water Research Laboratory for their assistance during sUAS assisted inspection missions, and to Hunter Buxton for piloting inspection missions with excellence.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. FHWA. Highway Bridge Condition by Highway System 2016; Federal Highway Administration: McLean, VA, USA. Available online: https://www.fhwa.dot.gov/bridge/britab.cfm (accessed on 2 October 2019).
  2. FHWA. Deficient Bridges by Highway System 2016; Federal Highway Administration: McLean, VA, USA. Available online: https://www.fhwa.dot.gov/bridge/deficient.cfm (accessed on 2 October 2019).
  3. CFR. United States Code of Federal Regulations: National Bridge Inspection Standards, 23 CFR 650C. Available online: https://www.gpo.gov/fdsys/granule/CFR-2011-title23-vol1/CFR-2011-title23-vol1-part650-subpartC (accessed on 2 October 2019).
  4. Zulifqar, A.; Cabieses, M.; Mikhail, A.; Khan, N. Design of A Bridge Inspection System (BIS) to Reduce Time and Cost; George Mason University: Farifax, VA, USA, 2014. [Google Scholar]
  5. Dorafshan, S.; Maguire, M.; Hoffer, N.V.; Coopmans, C. Challenges in bridge inspection using small unmanned aerial systems: Results and lessons learned. In Proceedings of the 2017 International Conference on Unmanned Aircraft Systems (ICUAS), Miami, FL, USA, 13–16 June2017; pp. 1722–1730. [Google Scholar]
  6. Dorafshan, S.; Maguire, M.; Hoffer, N.V.; Coopmans, C. Fatigue Crack Detection Using Unmanned Aerial Systems in Under-Bridge Inspection; RP 256 Idaho Transportation Department: Boise, ID, USA, 2017. Available online: http://apps.itd.idaho.gov/apps/research/Completed/RP256.pdf (accessed on 11 November 2019).
  7. Dorafshan, S.; Maguire, M. Bridge inspection: Human performance, unmanned aerial systems and automation. J. Civ. Struct. Health Monit. 2018, 8, 443–476. [Google Scholar] [CrossRef]
  8. Vemuri, S.H.; Atadero, R.A. Case study on rapid scanning techniques for concrete bridge decks with asphalt overlay: Ground-penetrating radar and infrared thermography. Pract. Period. Struct. Des. Constr. 2016, 22, 05016004. [Google Scholar] [CrossRef]
  9. Khaloo, A.; Lattanzi, D.; Cunningham, K.; Dell’Andrea, R.; Riley, M. Unmanned aerial vehicle inspection of the Placer River Trail Bridge through image-based 3D modelling. Struct. Infrastruct. Eng. 2018, 14, 124–136. [Google Scholar] [CrossRef]
  10. Dorafshan, S.; Thomas, R.J.; Maguire, M. Benchmarking Image Processing Algorithms for Unmanned Aerial System-Assisted Crack Detection in Concrete Structures. Infrastructures 2019, 4, 19. [Google Scholar] [CrossRef]
  11. Lattanzi, D.; Miller, G. Review of robotic infrastructure inspection systems. J. Infrastruct. Syst. 2017, 23, 04017004. [Google Scholar] [CrossRef]
  12. Hiasa, S.; Karaaslan, E. Infrared and High-definition Image-based Bridge Scanning Using UAVs without Traffic Control. In Proceedings of the 27th ASNT Research Symposium, Orlando, FI, USA, 26 March 2018; pp. 116–127. [Google Scholar]
  13. Dorafshan, S.; Thomas, R.J.; Maguire, M. Fatigue crack detection using unmanned aerial systems in fracture critical inspection of steel bridges. J. Bridge Eng. 2018, 23, 04018078. [Google Scholar] [CrossRef]
  14. Dorafshan, S.; Thomas, R.J.; Coopmans, C.; Maguire, M. Deep learning neural networks for sUAS-assisted structural inspections: Feasibility and application. In Proceedings of the 2018 International Conference on Unmanned Aircraft Systems (ICUAS), Dallas, TX, USA, 12–15 June 2018; pp. 874–882. [Google Scholar]
  15. Mutlu, M.; Melo, K.; Vespignani, M.; Bernardino, A.; Ijspeert, A.J. Where to place cameras on a snake robot: Focus on camera trajectory and motion blur. In Proceedings of the IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), West Lafayette, IN, USA, 18–20 October 2015; pp. 1–8. [Google Scholar]
  16. Clemena, G.G.; McKeel, W.T. Detection of delamination in bridge decks with infrared thermography. Transp. Res. Rec. 1987, 664, 180–182. [Google Scholar]
  17. Vaghefi, K.; Ahlborn, T.T.M.; Harris, D.K.; Brooks, C.N. Combined imaging technologies for concrete bridge deck condition assessment. J. Perform. Constr. Facil. 2013, 29, 04014102. [Google Scholar] [CrossRef]
  18. Otero, L.D. Proof of Concept for Using Unmanned Aerial Vehicles for High Mast Pole and Bridge Inspections; No. BDV28-977-02; Department of Transportation, Research Center: Tallahassee, FL, USA, 2015.
  19. Lovelace, B.; Zink, J. Unmanned Aerial Vehicle Bridge Inspection Demonstration Project; Research Project Final Report 40; Minnesota Department of Transportation: Saint Paul, MN, USA, 2015.
  20. Jennifer, W.; Lovelace, B.; Engineers, C. Unmanned Aircraft System Bridge Inspection Demonstration Project Phase II Final Report; No. MN/RC 2017-18; Minnesota Department of Transportation: Saint Paul, MN, USA; Research Services & Library: Beijing, China, 2017.
  21. Ham, Y.; Han, K.K.; Lin, J.J.; Golparvar-Fard, M. Visual monitoring of civil infrastructure systems via camera-equipped Unmanned Aerial Vehicles (UAVs): A review of related works. Vis. Eng. 2016, 4, 1. [Google Scholar] [CrossRef]
  22. Dorafshan, S.; Maguire, M.; Hoffer, N.V.; Coopmans, C.; Thomas, R.J. Unmanned Aerial Vehicle Augmented Bridge Inspection Feasibility Study; No. CAIT-UTC-NC31; Utah State University: Logan, UT, USA; Department of Electrical and Computer Engineering: Beijing, China, 2017. [Google Scholar]
  23. Connor, R.J.; Dexter, R.J.; Mahmoud, H. Inspection and Management of Bridges with Fracture-Critical Details; No. Project 20-5 (Topic 35–08); Transportation Research Board: Washington, DC, USA, 2005. [Google Scholar]
Figure 1. sUAS images of (a) efflorescence and delamination; (b) corrosion; (c) fatigue cracking.
Figure 1. sUAS images of (a) efflorescence and delamination; (b) corrosion; (c) fatigue cracking.
Infrastructures 04 00072 g001
Figure 2. Small unmanned aerial system (sUAS) visual image consideration: (a) underexposure using auto mode; (b) underexposure adjusted; (c) overexposure; (d) overexposure adjusted.
Figure 2. Small unmanned aerial system (sUAS) visual image consideration: (a) underexposure using auto mode; (b) underexposure adjusted; (c) overexposure; (d) overexposure adjusted.
Infrastructures 04 00072 g002
Figure 3. UAS thermal image showing surface and subsurface defects.
Figure 3. UAS thermal image showing surface and subsurface defects.
Infrastructures 04 00072 g003
Figure 4. sUAS access and obstruction due to: (a) geometry; (b) cross-bracing and bird nest; (c) wasp nest.
Figure 4. sUAS access and obstruction due to: (a) geometry; (b) cross-bracing and bird nest; (c) wasp nest.
Infrastructures 04 00072 g004
Figure 5. Images of the fatigue crack specimen captured by sUAS: (a) 0.2 m standoff distance; (b) 0.9 m standoff distance; (c) with 10 lx illumination; (d) with 6000 lx illumination.
Figure 5. Images of the fatigue crack specimen captured by sUAS: (a) 0.2 m standoff distance; (b) 0.9 m standoff distance; (c) with 10 lx illumination; (d) with 6000 lx illumination.
Infrastructures 04 00072 g005
Table 1. Bridge inspection types.
Table 1. Bridge inspection types.
Inspection TypeDescriptionMin. Interval
Damage inspectionUnscheduled inspection to assess condition, following potentially damaging event (e.g., vehicle impact or natural disaster)As needed
Fracture critical member (FCM) inspectionHands-on inspection of members or elements susceptible to fatigue cracking24 months
Hands-on inspectionArms-length inspection of structural components; can include visual and nondestructive evaluation; can be part of FCM, in-depth, special, or underwater inspectionVaries
In-depth inspectionHands-on inspection to determine deficiencies not detectable by routine inspection; may include inspection of entire structure (long interval) or specific members (short interval)Varies
Initial inspectionPreliminary inspection performed prior to entering service to determine baseline structural conditionNone
Routine inspectionInspection of physical and functional condition of structure to ensure that service requirements are satisfied24 months
Special inspectionScheduled inspection of known or suspected deficienciesDetermined by owner
Underwater inspectionInspection of underwater substructure or other elements requiring subsurface exploration60 months
Table 2. Cost of hands-on and sUAS assisted FCM inspections adopted from [7].
Table 2. Cost of hands-on and sUAS assisted FCM inspections adopted from [7].
Method of InspectionUBIT *Support Truck *UBIT Operator *Inspector *Pilot and UAS *Total *Full Bridge(total)
Hands-on$200$16$75$100-$391$1564
UAS---$100$100$200$3600
* Cost per hour.
Table 3. General specifications for sUAS assisted bridge inspections adopted from [7].
Table 3. General specifications for sUAS assisted bridge inspections adopted from [7].
sUASSensorsPositioningSize (cm)Flight Time (min)Price ($)Potential Bridge Inspection Applications
Parrot BEBOP 2VisualGPS32.8 by 38.2 25500–700Over-bridge, visual detection of macroscale surface cracks (thicker than 0.8 mm), routine inspection, checking the bridge structural integrity
3DR IrisVisualGPS63 by 38 20600–800
3DR SoloVisualGPS40 by 40 20800–1000
DJI Mavic AirVisualGPS, Vision System21.3 (diagonal)20800–900Over and under-bridge, visual detection of surface cracks (as thin as 0.04 mm), routine inspection, FCM inspection, checking the bridge structural integrity
DJI Mavic ProVisualGPS, Vision System33.5 (diagonal)271000–1200
DJI Phantom 4 ProVisualGPS and Vision System35 (diagonal)301800–2000
Visual and Thermal5500–8000Over and under-bridge, visual detection of surface cracks (as thin as 0.04 mm), subsurface defect detection (delamination), routine inspection, FCM inspection, checking the bridge structural integrity
DJI Mavic AirVisual and ThermalGPS, Vision System21.3 (diagonal204000–6000
DSLR Pros Law EnforcementVisual and ThermalGPS and Vision System64.3 (diagonal)1713,000–15,000
Altus LRXVisual, Thermal, LiDARGPS, LiDAR140 (diagonal)2040,000–50,000Over and under-bridge inspection, autonomous 3D model reconstruction, microscale defect detection (thinner than 0.02 mm), subsurface defect detection
Back to TopTop