Utilization and Verification of Imaging Technology in Smart Bridge Inspection System: An Application Study

Abstract

Image-based inspection technologies involving various sensors and unmanned aerial vehicles are widely used for facility inspections. The level of data analysis technology required to process the acquired data algorithmically (e.g., image processing and machine learning) is also increasing. However, compared with their development rate, the applicability of new inspection technologies to actual bridges is low. In addition, only individual technologies (for inspecting specific deteriorations) are being developed; integrated inspection systems have been neglected. In this study, the bottom-up method (which systematizes the applications of a specific technology) is avoided; instead, several technologies are summarized and a system of preliminary frameworks is established using a top-down method, and the applicability of each technology is verified in a testbed. To this end, the utility of the initially constructed technical system was assessed for two bridges; then, a strong utility technology was selected and applied to an offshore bridge under extreme conditions. The data obtained from the inspection were accumulated in a database, and a 3D-type external inspection map was produced and applied in the subsequent inspection via virtual and augmented reality equipment. Through the system, it was possible to obtain cost-effective and objective bridge inspection images in extreme environments, and the applicability of various technologies was verified.

1. Introduction

In Korea, it is predicted that the number of old social overhead capital (SOC) facilities constructed more than 30 years ago will nearly double by 2030. Hence, the maintenance of old facilities is becoming increasingly important [1]. However, most inspections conducted upon bridges (representative SOC facilities) still involve daily visual inspections in which the inspector carries basic measuring equipment to the bridge and assesses its deterioration. In the case of difficult-to-access bridge members, inspectors use an inspection rail installed on an abutment, pier or aerial ladder vehicle. Inspection systems that rely on visual inspection produce many errors and suffer from problems such as poor inspector safety and traffic flow disruption [2].

Therefore, it is necessary to apply new technologies to overcome these problems and build new inspection systems. Social infrastructure facilities such as bridges, tunnels, and dams are becoming larger; hence, manual inspection is gradually becoming more limited. To solve this problem, various inspection tools, unmanned utilities and inspection methodologies are being used, and customized equipment is being developed to suit the type and importance of the structure [3,4,5].

Drones, the most representative piece of unmanned equipment, can check the appearance of facilities based on images and videos; hardware and software developments are continuously being accomplished in this field [6,7,8]. For example, an inspection technology using a drone equipped with a thermal imaging camera and LiDAR module was presented and tested to facilitate the detailed, unmanned inspection of bridge members [4,5,9]. However, drones can cause crashes or can themselves crash because of vortexes or global positioning system (GPS) problems. Thus, for certain situations, it is difficult to make contact with or get close to the facility [10]. Moreover, many difficulties still arise when testing and applying practical solutions [11,12]. Hence, numerous studies are being conducted to improve the performances of new types of equipment, internal sensors and shooting modules [13,14]. Applicability tests are being conducted upon drone technologies across various fields and for structures besides bridges. Calculating the amount of earthwork at a construction site [15,16,17] and ensuring construction safety are typical examples of drone use in construction [18,19]. Drones are also being widely used in the inspection of dams and steel structures [4,20,21,22], and for exterior inspections of large power plant structures and interior equipment [23,24,25]; furthermore, drones are being used as a basic technology for structure mapping using point-cloud techniques [11,26]. Along with the development of smart phones, research on utilizing in-device sensors has also been conducted [27].

In terms of hardware, drones and various inspection tools customized to a facility’s characteristics are being developed. The equipment most commonly used in bridge facility inspections include wire cameras (to inspect the lower parts of a facility), rail cameras (to inspect the inside of box girders) and winch cameras (to inspect abutments and piers). In addition, bridge inspection drones and robots are being built and tested, and underwater inspection drones and robots [28,29] suitable for inspecting the cables of cable bridges are currently in use [30].

Numerous studies into real-time monitoring methodologies based upon sensors or closed-circuit television (CCTV) have been conducted in order to utilize inspection equipment and build smart inspection systems for bridges [31]. A typical integrity evaluation study involves installing a displacement sensor and accelerometer on a bridge to measure the acceleration under vehicle movements, perform finite element analysis, and predict the damaged area [32,33,34,35,36]. A robot-based technique to inspect the integrity of cables has been proposed for large cable bridges [30], and a study that monitors tension by installing a fiber-bragging grating sensor has been conducted. Efforts are being made to construct a monitoring system using CCTV video technology, and basic research has been conducted to derive scenarios for use in smart systems by combining CCTV with sensors [37,38,39].

In addition, a system that monitors bridges in real time using smart sensors, tracks the bridge displacement and collects various environmental factors in real time has been developed in the form of a location-based system adopting a spatial image map [40]. A technology that can link various types of information collected via sensors or equipment with the bridge management system in real time has been developed into a smartphone application; this is currently in use [41]. The inspection data obtained using this equipment can be applied in various ways via post-processing. Image processing research is actively being conducted to automatically extract deteriorations from inspection images. Deteriorations such as cracks can be detected in bridge images through binarization and conversion [42,43,44], and the sizes and widths of the crack can be obtained from inspection data [45,46]. In addition, a deep learning model that detects scaling and spalling (damage that can occur in concrete bridges) was developed using images of concrete structures, and its model performance was verified using images acquired from actual bridges [47]. In railways and tunnels, studies have been conducted to determine via machine learning the presence or absence of abnormalities, using images from railway line junction inspections [48,49]. In a more advanced study, the depth and shape of spalling in a subway tunnel were measured using an image-based machine learning model [50]. To systematize these various inspection results, an intuitive system and external inspection map are being designed to assist in facility managers’ decision-making (e.g., by converting previous 2D inspection results into 3D models). Lattanzi et al. [51] and Jahanshahi et al. [52] created a 3D model using videos and images taken from various points around a damaged facility, and they inspected the damage to the facility using this model. In addition, the data of bridges secured through new and various structural health monitoring technologies are becoming big data. The built big data can automatically extract deterioration items, measure quantities and locate them through artificial intelligence approaches, and these technologies are being combined with monitoring devices [53,54,55].

In the present study, by referring to the technology and equipment developed across previous works, we establish a system to comprehensively utilize inspection technologies that are both currently available and offer scalable potential. Then, we test the technology across various environments, verify the utility of the equipment, develop the system, and identify improvements and supplementary measures. In addition, from the inspection results obtained with the equipment, a 3D spatial external inspection map is developed for the convenience of inspectors and managers, and additional utilization plans using virtual reality (VR) equipment are derived. A 3D image database is built from the obtained data, and an automated system is constructed using image processing and machine learning. It is difficult to manage due to the gradually increasing maintenance cost of bridges. As a way to reduce this, we propose a new high-tech management system and try to build a more sustainable maintenance system.

2. Bridge Inspection

2.1. Bridge Inspection System

The basic principle of bridge maintenance is to identify the condition and performance of a bridge through regular on-site inspections, as well as to detect and measure at an early stage any deterioration that may adversely affect the safety and utility of the structure. By analyzing the data acquired during inspection, the changing status or performance can be identified, and an evidence-based maintenance plan can be established. The inspection level is determined by the age of the bridge. In Korea, in accordance with the “Special Act on Safety Management of Facilities,” enacted in 1995, the initial inspection, regular inspection, precise safety diagnosis, emergency inspection, and follow-up inspection are all performed separately.

Periodic inspection is necessary to ensure the conditions of use are satisfied by identifying physical or functional problems and reviewing any changes since the previous inspection. These inspections are carried out at least once every six months from the date of the facility’s construction or its approval for use. In detailed inspections, the bridge condition is evaluated according to the inspection result; the initial inspection is performed within three years of the facility completion date and is regularly repeated every 1–3 years, depending on the condition ratings. For regular and detailed inspections, measuring instruments such as crack rulers, tape measures and cameras are used alongside visual observation equipment. At this time, inspection passages or vehicles are employed such that even the parts that are difficult to access (e.g., the bottom of the deck) are inspected. The precise safety diagnosis is derived from visual inspections and equipment, material and load tests to identify any deterioration that is difficult to detect via simple visual inspection. If necessary, special types of equipment (e.g., bridge inspection carts, work scaffolding, and work boats) are used; nondestructive and material tests must be performed in parallel to determine the presence and extent of deterioration. The first inspection is carried out within one year of the ten-year anniversary of the structure’s completion, and the interval is set as 4–6 years depending on the condition ratings. The condition rating is obtained from detailed inspections and precise safety diagnoses; it takes into consideration defects, damage and deterioration and is expressed as a grade from A to E, according to detailed guidelines in

Table 1. Bridge inspection condition criteria in Korea.

Grade	Structure Performance
A (excellent)	Free from external defects, damage or collapse
B (good)	Defects have occurred in some bridge members, and it is necessary to determine whether or not to repair the defect by continuously observing the progression of the defect.
C (moderate)	Although defects have occurred in a wide range of bridge members, the overall safety of the facility is not affected. Hence, only a simple repair or reinforcement are required.
D (poor)	Requires urgent repair and reinforcement for serious defects; usage restrictions should be considered
E (bad)	Usage prohibited; requires reinforcement or remodeling because of a risk to the safety of the facility

[56].

During regular inspections, only major bridge members are assessed, whereas each member is evaluated during detailed inspections. However, an external inspection map is drawn up for the problem area, and the condition rating is graded for each member. Subsequently, in detailed safety diagnoses, the condition rating is calculated for the entire bridge. When evaluating the condition of each member, the girder and cross-beam are generally separated first. The evaluation subdivides items (including carbonation and chloride evaluation items) and identifies members that directly influence the structural integrity. The evaluation grades A–E are applied to the main members (e.g., girders, decks, substructures, and supports). In this study, the visual aspects of inspections are improved using image-based technologies, and previously obtained reports and inspection details are replaced with image- and video-based ones.

2.2. Overseas Inspection Systems

In the United States, when construction of the Interstate Highway began in 1956, numerous bridges were constructed; it is believed that these bridges are nearing the ends of their design lifetimes. Bridges vary by state; however, most of them are evaluated by considering the topographical characteristics of each state, according to the standards of the “Bridge Element Inspection Manual” published by The American Association of State Highway and Transportation Officials (AASHTO) [57]. The condition ratings for each status of the structure (the basis of the evaluation) are as shown in

Table 2. U.S. bridge inspection rating criteria.

Grade	Structure Status
N	Not applicable
9	Excellent condition
8	Very good condition
7	Good condition
6	Satisfactory condition
5	Fair condition
4	Poor condition
3	Serious condition
2	Critical condition
1	Imminent failure condition
0	Failed condition

[58].

In general, inspections are conducted within 24 months for each bridge, and the condition rating is recorded according to these condition standards. Inspection is visual, and the use of complex equipment or machinery (e.g., material testing and load capacity measurement equipment) is not required, except in special circumstances. After inspection, the rating data are entered into the database on the National Bridge Index (NBI). The NBI data are publicly distributed every year, and the basic information and condition rating data provided by the system are used as basic data for the development of degradation models.

In Japan, inspections are conducted by classifying them into five stages according to the daily, periodic, and special (inspections in cases of anomaly, follow-up inspections, and detailed inspections) inspection results. The basic daily inspection procedure depends on the inspector’s visual observations and the riding comfort of the road patrol. Inspections of the road surface and structure underside are performed daily and once a week, respectively; the aim is to detect large and significant portions of damage. Periodic inspections are performed using temporary scaffolding or inspection vehicles; the purpose of such inspections is to detect significant damage. According to bridge management standards, regular inspections are performed by an inspector once every five years, and an inspection vehicle or ship is used when a close visual inspection is necessary. In addition, during daily inspections, detailed inspections of difficult-to-access structural parts are performed, with special attention paid to the structural integrity. Special inspections are conducted for damage caused by unexpected disasters (e.g., earthquakes and traffic accidents); the results of the inspection are classified into the condition ratings shown in

Table 3. Japanese bridge inspection rating criteria.

Rating	Status
A	Severe damage requiring urgent repair
B	Damaged and in need of repair, but not urgent; in need of load-bearing capacity increase
C	Damage is present but kept as a record only
D	Minor damage (record not required); intact
Q	Inability to determine the degree of damage; condition requiring detailed inspection due to abnormal damage

[59].

The Japanese Road Agency considers the degree of damage to the deck on the basis of cracks. Therefore, in the Japan Expressway Corporation and Hanshin Expressway Corporation, the leaching or depositing of free lime or leaks at the bottom of the floor plate are included in the judgment criteria.

Aging bridge maintenance is required in Europe; hence, a research project was conducted by Bridge Management in Europe (BRIME) to understand the level and performance of the bridge structure management system. BRIME established a nationally integrated bridge maintenance and management system and accordingly organized and analyzed their results by conducting a survey to understand the status of the maintenance systems in each European country. Most existing maintenance systems in European countries evaluate soundness by inspecting the entire road structure using modeling, and their evaluation approaches tend to be similar. In addition, management systems are classified based on condition ratings. Each country builds its own system and selects classification criteria; Korea evaluates the condition of bridges using various grading criteria, including five of its own classification criteria alongside the United States’ ten criteria. Although the classification levels and degree of criteria may differ, most studies adopt similar methods. The criteria for classifying bridge conditions by country are as shown in

Table 4. Worldwide bridge condition rating standards.

Country	Bridge Condition Rating	#
USA	0–9 condition ratings	10
Korea	A–E condition ratings	5
Japan	A, B, C, E1, E2, M, S ratings	6
Germany	0 (good) to 4 (very poor) condition rating + three ratings for each bridge component	5
China	CS 1–CS 5 condition states	5
France	0–4 condition ratings	5
UK	1–5 severity rating + A–E extent ratings	5
Norway	1–4 severity rating + consequence code	4
Finland	0–4 condition ratings	5
Denmark	0–5 condition ratings	6
South Africa	Three categories: physical, functional, and economic conditions	3
Malaysia	0–5 condition ratings	5

[59,60,61,62,63].

Most are classified into 4–6 grades, except for South Africa’s three-grade classification standard. When classifying, the classification criteria vary; however, generally similar evaluations are obtained and applied to guide maintenance.

2.3. Inspection Methods

In Korea, bridges are inspected daily via visual inspections. Periodic inspection is performed visually, and the inspector brings only basic measuring equipment. To inspect bridge members, the inspector moves through the inspection railing installed on the abutment or pier; in an area where access is difficult, an articulated inspection vehicle or ladder truck is used to help the inspector. Bridge inspection in the US is also conducted visually, and complex equipment (e.g., for material and load capacity testing) is not required, except in special circumstances. After visual inspections, special equipment is applied as needed and the final bridge condition is evaluated not only by evaluating the structure but also by determining the bypass road problems, traffic restrictions, roadway width, bridge height and traffic volume affecting it. Subsequently, using the results, repair, reinforcement, and special measures are prepared. According to the US “Bridge Maintenance Reference Manual,” [57] bridge condition management monitors inspect once every 25 months, though additional inspections can be performed in the event of disasters (e.g., storms, earthquakes, rapid increases in traffic, and floods). In addition, the Collapse-Inducing Member Inspection Regulations (Michigan DOT manual) [64] stipulate close-up visual inspections as their principal method; these inspections are usually performed at least once within a 1–2 year period. [57] Corrosion and cracks are intensively inspected in fatigue-vulnerable areas, and non-destructive tests (NDTs) are conducted in areas suspected of harboring such cracks.

In Japan, maintenance efficiency has been increased by developing new inspection and repair technologies; these technologies have helped reduce the size of the overall maintenance budget and have helped overcome serious manpower shortages. In addition, for the application of technology, legal inspection and diagnosis regulations have been revised and information regarding the implementation status of inspections, maintenance reinforcements and measures has been collected. At present, the aim of Japan’s maintenance technology development plan is to match the technical requirements of the field with various technologies and to introduce these technologies rapidly to perform maintenance. It also intends to improve maintenance levels and reduce costs through prevention and structural preservation. In particular, the technical and industrial goals are to create a new maintenance market through new technologies involving robots, utilities, data, sensors, NDTs and longevity, as well as to promote data-based private technology developments [65].

In Europe, efforts to systematically secure technology through a specialized development roadmap are underway in facility maintenance. Since 2013, various infrastructure durability and life extension-, next-generation infrastructure-performance-evaluation-, NDT-, and maintenance-related technologies that have less of an impact on traffic flow have been developed through the Infrastructure Innovation Program (Infravation) Project. Currently, NDT is not an essential part of most European bridge inspection procedures [66].

2.4. Blind Spots in Bridge Inspection

Most bridge inspectors inspect bridge members using simple tools and instruments. These on-site inspections are problematic because certain areas are difficult to access, or taking equipment to them is difficult. In particular, inspection is often not performed properly in closed areas (e.g., the inside of box girders or the girders of large bridges) [67]. When these parts are not continuously and appropriately inspected, they can pose a significant risk to the bridge, and the accumulated deterioration leads to large repair and reinforcement costs. These weak inspection areas are defined as inspection blind spots; Figure 1 shows the representative damage types that occur in the inspection blind spots of bridges.

Accessible areas such as bridge pavements, handrails, curbs and expansion joints are easier to visually inspect. However, for tall bridges, it is necessary to use an existing inspection facility or special equipment to access the deck, girder, bridge, shoes and substructure. In other words, for accessible parts, an on-site inspection is performed but a poorly accessible blind spot occurs. For the safe maintenance of facilities, a meticulous inspection must be conducted, and smart inspection technology must be developed to this end.

3. Configuration of Smart Inspection Technology

Safety diagnoses and maintenance tasks are currently inefficient, owing to visual inspection problems, difficulties in data inquiry and sharing and insufficient maintenance data management. Defects may not be identified via visual inspection, and even if they are found, they may not be recorded on the external inspection map. In addition, the type and size of deterioration must depend on subjective judgment, and there is room for the structural safety level to be calculated differently; furthermore, a lot of time and money is lost in post-processing. Usually, after inspection, a photographic inspection map and visual network map are prepared and included in the final report. However, a lot of time is consumed in post-processing which connects and arranges flat data that are not arranged in two dimensions; furthermore, this process depends upon the technician, even after diagnosis. Hence, the costs of management, maintenance and reinforcement are increasing as facilities age.

For the inspection technologies being developed to solve these problems, a change to a preventive maintenance system is required to improve the current safety diagnosis system and prepare for aging. In particular, following the development of various robots and devices, inspection methods are being improved, and technologies for automatically determining defects and deteriorations and extracting the sizes thereof are being developed through image analysis, machine learning and artificial intelligence. Rail cams, wire cams and drones can be used for video- and image-based inspections, depending on the shape and size of the structure. Additionally, various types of monitoring and inspection technologies have been developed and adopted for use. The main image-based inspection technologies of the smart inspection system used here are as shown in Figure 2.

Smart inspection equipment is divided into inspection kits for inspectors, smart utilities to replace the existing visual inspection and shorten inspection times, real-time monitoring systems to assess the health of bridges and a smart office to oversee operations. This system makes it possible to create a database of work content accumulated from digital images and video inspection data. In addition, the database can be used for bridge inspections, maintenance and reinforcement.

For detailed technical configurations, rail cams, wire cams, winch cams and drones are commonly used. In addition, the inspector’s equipment consists of a panoramic head, smart glasses, 360° camera and neckband camera. CCTV and time-lapse cameras are used to continuously monitor bridge conditions. In the office (which oversees these inspection technologies), a technique is used to analyze images and videos acquired with various equipment and create a digital environment identical to that in the field; this proceeds by adopting 3D stitching techniques, establishing a database of secured inspection data for long-term operations and developing a management system.

When operating smart inspection technology, it is important to obtain and analyze the information required for structural inspections (e.g., the type, location, length, width, height, and obstacles of the target facility) before carrying out inspection tasks. Based on prior information, the inspection technology suitable for the facility is selected, and an initial inspection plan is established. In this step, the information required for structural inspection (e.g., the location where safety inspection is required) is combined with the design information of the structure. The inspection data obtained whilst operating the inspection system are transmitted to the post-processing system; thus, the analysis of the data proceeds quickly and an optimal operation plan can be established.

3.1. Smart Inspection Kit

When inspecting bridges, inspectors use only simple tools (e.g., cameras and tape measures) and mainly rely on visual inspection; when deterioration is observed, it is recorded as a photograph, and the location is marked on the external inspection map during post-processing. However, this traditional method results in inspection errors and produces uncertain results. Moreover, it is difficult to clearly define and classify deteriorations because non-experts perform the inspections. If a safety diagnosis or maintenance judgment is made based on inaccurate inspection results, excessive repair or reinforcement may occur, or areas requiring action may be overlooked.

In this research, a smart inspection kit for inspectors was used to solve the problems caused by errors in non-expert inspections and the secondary recording process. The smart inspection kit consists of three major pieces of equipment. First, smart glasses can share fieldwork videos and are used as key equipment to allow the manager to participate in the inspection alongside the field inspector. The glasses consist of a 360° camera for inspectors (to obtain 3D inspection image data for the overall bridge) and a neckband camera to verify the surrounding environment.

■

Smart glasses

Smart glasses are one of many smart wearable devices worn on the face; they offer safety, data transmission and computing functions during inspections. Their representative function is to project image data (e.g., basic information, drawings and structure subject inspection records) to on-screen inspections. The specifications of smart glasses are shown in

Table 5. Specification of smart glasses.

Element	Specifications
Model	Vernect remote
Core platform and functions	Android 8 (AOSP) + WearHF hands-free interface 2.0 GHz 8core Qualcomm Snapdragon 625 with Adreno 506 GPU-OpenGL
Connectivity and sensors	802.11a/b/g/n/ac-2.4 GHz and -5 GHz GPS, GLONASS, AGPS
Battery	3250 mAh Li-ion, rechargeable, and field swappable
Display	WVGA (854 × 480)
Multimedia	16MOP four-axis optical image stabilization, PDAF with LED
Weight	370 g
Camera module	1600 million px.
Price	$4000 (Solution include)

.

Smart glasses generally have five representative functions: (1) the search function, which allows users to search for the information required in the field using the equipment, without having to stop working; (2) the voice input function, which supports communication with the manager without buttons or mechanical actions; (3) the information display required for maintenance, which overlaps the design and maintenance data with field objects using GPS location information as a viewfinder; and (4) and (5) the video and camera functions, respectively, which take videos or pictures in the field, transmit them to the database and share them.

In this study, smart glasses were used to enable communication and information transfer between field inspectors and managers. The manager could check the real-time inspection video delivered through the smart glasses and instruct the main inspection location according to the inspection needs. In addition, in environments where voice connection is difficult because of ambient noise, inspection support can be realized in various ways using text or documents. The field applicability was verified through a test of the smart glasses, and the inspection system was established by using the equipment during actual inspections.

■

Neckband camera

Neckband cameras are worn similarly to necklaces; they are hands-free devices that facilitate 360° filming, voice calls and the free use of both hands. They are mainly used to review the risk of accidents to workers or the inspection environment, as well as to check that the bearer is wearing protective equipment and to send updates regarding work progress. Because this equipment is very effective in ensuring inspector safety, a neckband camera was selected as the key technology of our smart inspection kit, to be applied in combination with the smart glasses. The specifications of neckband camera are shown in

Table 6. Specifications of neckband camera.

Element	Specifications
Model	FITT 360
Core platform and functions	2.0 GHz 8core Qualcomm Snapdragon 625 with Adreno 506 GPU-OpenGL ES3.1 & OpenCL2.0
Connectivity and sensors	802.11a/b/g/n/ac-2.4 GHz and -5 GHz 9-DOF (three-axis accelerometer, magnetometer, and gyroscope),
Battery Display	3250 mAh Li-ion, rechargeable, and field swappable WVGA (854 × 480)
Multimedia Weight	16MOP four-axis optical image stabilization, PDAF with LED 370 g
Camera module	1600 million px.
Price	$400

.

When inspecting bridges (which are large infrastructures), accidents can occur in all areas, and unexpected situations can arise in the operator’s blind spots. When using a neckband camera, the manager can identify risk factors and thereby ensure the safety of field workers; these devices can also reduce the work time through facilitating more effective communications.

Smart glasses and neckband cameras have different roles. Smart glasses can record and transmit the inspection history from the inspector’s point of view and share it with the manager. However, a neckband camera is used to monitor an inspector’s surrounding environment to check risk factors and the inspection environment during an inspection. The data obtained through each device will be used to measure inspection routes and secure safety.

■

360° camera and panoramic head

In traditional inspections, an inspector is placed in the field with only a camera, and the result obtained after inspection is a flat picture of the damaged or abnormal component; that is, only the results of photographing in a plane are provided, and these only discontinuously. Discontinuously accumulated information is difficult for a new inspector to utilize in subsequent inspections of the bridge; furthermore, because it is not continuous, it is of too poor a quality to be used as maintenance data. In this study, to solve these problems and ensure the continuity of inspection data, a 360° camera and panoramic head for shooting were developed and used as equipment for a smart inspection kit. The 360° camera is a device developed to implement VR, and new products are being released as VR content evolves. In the smart inspection system, a field inspector uses the corresponding equipment to derive a 360° inspection image, which differs from the flat images currently in use. The generated 3D image provides a sense of space and realism, and the space in the image can be enlarged and reduced for observation. The development of 3D image-based inspection data for the target facility facilitates intuitive and continuous maintenance. The shape of the two equipment used in the experiment is shown in Figure 3.

3.2. Smart Utilities

Smart utilities are defined as devices used to minimize inspection blind spots (i.e., spots that are difficult for inspectors to access or bring devices to because of the environment in which the bridge is located). Representative smart inspection utilities that can be used on bridges include drones, rail cams, wire cams and inspection robots. Three smart inspection devices (drones, rail cams and wire cams) were used in this study, and the necessary equipment was selected and applied depending on the characteristics of the bridge.

Drones are the most representative smart inspection utility and are used not only for bridges but also for inspections of high-rise buildings, power plants, dams and dangerous infrastructures. In bridge inspection, drones can be used to inspect the appearance of bridges and access their substructures for photographing. In addition, rail and wire cams are used to inspect the lower part of the floor plate and the side of the girder, which are difficult for workers to access. A rail cam allows the rail cam robot to move along a pre-installed rail to capture images or photos of the inspection area, facilitating stable inspection. Meanwhile, the wire cam runs along a detachable wire that is installed in the inspection area to thereby obtain images or pictures. Rail cams can be used in enclosed spaces (e.g., inside box girders), and wire cams demonstrate high performances in the lower-plate inspections of small- and medium-sized bridges.

■

Drones

As the most representative device in smart inspection, drones can be applied to provide overall inspection results for bridges. They are the most effective equipment for exterior inspection; furthermore, in civil engineering, equipment suitable for each facility is applied to solve the maintenance difficulties attributable to facility enlargement. At present, drones are technologically superior to other utilities and are continuously being developed; however, their inspection capabilities are limited. Problems such as GPS reception and turbulence are typical; furthermore, drones do not receive signals well when inspecting the lower parts of bridges, and it is difficult to control them in turbulent airflows. In this study, rather than replacing the entire bridge inspection process, drones were applied to maximize efficiency by exploring suitable roles during inspection. During the application test, a 360° RGB camera module was installed on the drone to obtain various types of information. In addition, inspection data were obtained using a spherical drone to check the conditions between the girders and under the deck at the bottom of the bridge, where GPS problems occur. The specifications of the drone and thermal imaging sensor used at this time are shown in

Table 7. Drone specifications.

Element	Specifications
Model	MATRICE 600 PRO
Size	1668 mm × 1518 mm × 727 mm
Weight	9.5 kg (battery excluded)
P-GPS	Vertical: ±0.5 m; horizontal: ±1.5 m
Wind resistance	8 m/s
Max speed	65 km/h
Hovering hours	16–32 min
Max angular speed	Pitch: 300/S; Yaw: 150/S
Price	$6600

and

Table 8. Thermal imaging sensor specifications.

Element	Specifications
Model	Workswell WIRIS2
Resolution	640 × 512 px.
Temp. range	−25–150 °C/−40–550 °C
Temp. sensitivity	30 mk (0.03 °C)
Accuracy	±2% (±2 °C)
Zoom	1–14×

.

Subsequently, to identify various deteriorations, a thermal image was acquired via a thermal imaging module, and the deterioration revealed in the temperature difference was found. Deterioration data collection utilizes the principle that all objects with temperatures above absolute zero emit infrared energy at the speed of light; this energy is sensed and collected through the lens and detector of the infrared thermometer. Each material (e.g., concrete and steel) has a different emissivity; hence, a comparative analysis is made possible and deterioration items can be identified from temperature differences.

■

Rail cam

It is difficult for an inspector to enter the inside of a box girder (one of the main structural members of a bridge) because of its narrow space and long span. Rail camera equipment is mainly used for inspecting the insides of the box girders, exterior girders and inter-girder spaces. Semi-permanent rails are installed in areas that require inspection, and mobile robots are set to run on them. An inspector can remotely control the rail robot and diagnose the girder by moving forward and backward along the rail. The installed basic RGB module can be applied as a substitute for 360° cameras and thermal imaging modules.

Rails must be separately installed inside the box bridge and around the girder, and rail maintenance is required; however, a wide range of video inspections is made possible with only a small number of inspectors. Although an initial rail installation cost is incurred, it is cost-effective from a long-term perspective because the cost is lower than the cost of inspecting the inside of the box through visual inspection when using the equipment. In addition, the inspection efficiency can be increased by applying self-driving robots. The specifications of rail cam are shown in

Table 9. Rail cam specifications.

Element	Specifications
Model	Rail robot (Miraecity)
Material	Aluminum, carbon fiber
Motor	BLDC motor
Size	320 × 970 × 415 mm (W × L × H)
Weight	10 kg (+/−500 g)
Speed	0–40 km/h
Control	Radio control: 2.4 GHz SUMO & PWM
Gimbal	Three-axis (P/R/Y) encoder stabilizer
Camera module	4k HDR module/optical 12× zoom/clear image 24× zoom
Price	Self-production

.

■

Wire cam

A wire cam is a device that connects two fixed points via wires and moves between spans to perform inspection. The camera module mounted on the wire cam has the same compatibility as the rail cam described above and can conduct stable inspections via adjustment of the wire tension. This equipment is advantageous for inspecting the lower part of the bridge deck and between the girders; furthermore, because it is used in an open area, there is little risk of dangerous situations (e.g., radio wave interference between the equipment and inspector’s controller or transmission failure). This equipment has the advantage that the area that can be inspected is relatively wide compared with that of visual inspection; it can also solve the inspection blind spot problem, which is significant in large bridge maintenance. Moreover, non-experts can operate wire cams using a simple operation method, and an extensive image/video inspection is possible with even a small number of people. Unfortunately, using a wire cam requires a component to be installed at a specific location. To solve this problem, new technologies (e.g., fixed four-point autonomous driving wire cams and detachable hybrid wire cams) are being developed. The specifications of wire cam are shown in

Table 10. Wire cam specifications.

Element	Specifications
Model	Dual wire robot (Miraecity)
Material	Aluminum, carbon fiber
Motor	BLDC motor
Size	348 × 370 × 215 mm (W × L × H)
Weight	7.5 kg (+/−100 g)
Speed	0–20 km/h
Control	Radio control: 2.4 GHz SUMO & PWM
Gimbal	Three-axis (P/R/Y) encoder stabilizer
Camera module	4k HDR module/optical 12× zoom/clear image 24× zoom
Price	Self-production

.

The design images of rail cam and wire Cam are shown in Figure 4.

3.3. Monitoring System

Smart monitoring is a real-time technology used to identify bridge conditions and safety for daily bridge management. The system consists of a smart closed circuit television (CCTV) system that collects and analyzes traffic information and a time-lapse camera that continually inspects the displacement of the shoes to check the activity under temperature changes. Through the system, the response of the bridge under loads, abnormal signals and movements can be identified from images or videos. The CCTV used in the test and the monitoring screen on the CCTV are shown in Figure 5. The specifications of the device are shown in

Table 11. CCTV specifications.

Element	Specifications
Spec	4K UHD H.265 Max 30 fps@ 8 Mpx
Zoom	20× optical zoom/16× digital zoom
Visibility	Night vision: 200 m
Codec	H.265, H.264, MJPEG
Size (W × H)	Ø236.9 × 407.7 mm
Weight	7.1 kg
Function	De-fog function, day & night (ICR) auto tracking, IP66 (waterproof)
Storage (user)	6 TB (1y data basis)

.

The loads that normally act on a bridge during operation are temperature and live vehicle loads. The temperature-load-induced displacement depends on the characteristics of the material and is observed as the displacement of the bridge shoes. The vehicle load acts continually and can be confirmed from the bridge deflection. The smart inspection kit and utility technology described above can be used during on-site inspections; the smart monitoring system is a video-based inspection technology that detects real-time changes in the bridge’s structural integrity. The concept image and developed time-lapse camera are shown in Figure 6, and the specification of the camera module is shown in

Table 12. Time-lapse camera module specifications.

CMOS Size	Aperture	Focal Dist.	Angle of View
1/4 inch	F2.5	3.1 mm	160°

.

First, the time-lapse camera captures the temperature and displacement of shoes at regular time intervals and records the change in displacement over time, to review the change between temperature and displacement. This can be applied semi-permanently by installing additional solar panels and minimizing battery consumption through taking pictures at regular intervals. In addition, by miniaturizing the device, it can be more easily installed by the inspector. Smart CCTV was used to evaluate the condition of the bridge by measuring the deflection caused by the live load. Vehicle movement data obtained from videos were used for real-time traffic measurements and vehicle-type classification, through machine learning. This is because undesirable deflection can occur when traffic is irregular (owing to vehicle overloading and accidents); this can affect structural integrity. Hence, it is necessary to monitor, classify and define the live loads that produce deflection; thus, the type of vehicle used is a major factor in this study. An initial classification model was developed and classified vehicles into six types: cars, small trucks, medium trucks, heavy trucks and buses. Alongside the video collection function, the equipment uses an artificial-intelligence-based function to detect emergency situations (e.g., fires and accidents) and to send messages to institutions or managers. To store and manage images on CCTV, the server’s storage capacity and data processing method must be considered. About 120 MB is generated based on one 10 min video of one CCTV, and 6TB of data is secured on a yearly basis. It can be transferred to a server in the office as needed, and 6TB of offline storage has been built for sufficient space.

3.4. Smart Office

A smart office is a system that controls and comprehensively manages the various configuration technologies introduced above. It is developed as part of automated facility management and safety diagnosis and aims to ensure safety diagnosis efficiency, obtain prediction-based pre-emptive safety diagnoses, extend facility lifetimes and provide real-time safety services. The main functions of the currently developed smart inspection office are geographic-information-system-based facility management, image-based 3D external inspection maps, VR/AR-based inspector management training and inspection technology (time-lapse camera, smart inspection kit, smart CCTV) management. It is possible to expand smart inspection offices by continuously adding new and improved technologies.

4. Smart Inspection System

In this study, we verified the utility of various technologies in bridge inspection. The technologies were classified into four types, and a smart inspection system was built to combine them. Through application tests, currently available technologies can be improved and expanded according to the desired level of inspection.

The configuration of the system is shown in Figure 7.

First, the smart inspection kit that inspectors use consists of smart glasses, a neckband camera and a 360° camera. The equipment has video transmission, reception and communication functions; therefore, it can communicate with the office in real time during inspections. Next, smart utilities (as represented by drones, rail cams and wire cams) are included in the system. These utilities were deployed according to the inspection area and purpose. Drones are used as major equipment for the external inspection of bridges and girders; in particular, for higher bridges, they can be used to inspect the bridge underside. Basic RGB and thermal imaging cameras can be used to observe deterioration (e.g., leaks and efflorescence) via temperature differences. The rail cam is used to inspect the insides of box girders and can be semi-permanently adopted by pre-installing a rail. The inspector can remotely control the drone and inspect the bridge by attaching the desired modules. Depending on the lighting conditions, additional lighting equipment can be utilized. The wire cam can effectively inspect the lower parts of girders (the most difficult part of a bridge inspection), which are difficult to access owing to GPS problems and the requirement of additional equipment (e.g., a refraction vehicle). Therefore, wire cams are the most effective inspection technology; similarly to the rail cam, various modules can be used and automated inspection is possible. Smart monitoring systems can be classified into two types: smart CCTV and time-lapse cameras. Various sensor-based monitoring studies have been conducted; however, in this study, sensors were excluded, and we built a system using image and video data. Smart CCTV features accidental detection functions, and they can classify vehicles and analyze traffic volumes from traffic videos. Our method uses machine-learning-based image processing. In future, real-time vehicle-type classification will be improved and applied to evaluate structural integrity alongside sensor and expansion-joint-gap data. In addition, we developed a lightweight time-lapse camera to measure the movement of bearings; this could be powered by attaching a solar panel and battery as necessary. The periodic movement of the bearing is automatically analyzed via image processing; furthermore, the classification of overloaded vehicles and the evaluation of bearing integrity are performed in connection with the traffic volume.

The smart office, which oversees these inspection technologies, is a management system that includes real-time inspection statuses of bridges, the traffic volume and structural integrity analyses of the system. Bridge managers can obtain real-time soundness information using the system and intuitively view bridge inspection details via a 3D external inspection map. In addition, because real-time inspection scenes can be viewed and communicated via smart kits, collaboration between the office and field is made possible. The external inspection map produced with VR/AR content was implemented for the preliminary training of inspectors. Environmental information, influencing factors and inspection data obtained through inspections are stored in a database and can be used for facility deterioration models, life-cycle cost models and optimal maintenance strategy establishment processes.

5. Application Test

To verify the proposed smart inspection system, an application test was conducted in two stages. It was divided into the first application test on a test bridge (to examine the usability of the technology) and the second test on a large cable-stayed bridge located in an extreme environment. The processes and results of the two tests are as follows.

5.1. Technical Verification Test at Test Bridges

First, a test was conducted to confirm the applicability of the basic technologies at the Geumdang Bridge and Yeondae Bridge test roads, located in Yeoju, Korea. The bridges were constructed to test various roads and facilities and are used only for experimental purposes. Test applications of various inspection equipment were conducted on each of the two bridges, and a smart CCTV was installed so that the highway mainline located alongside the test road could also be used.

The Geumdang Bridge is a prestressed concrete (PSC) girder-type bridge, and the Yeondae Bridge is a steel box-type bridge. The Geumdang Bridge is 270 m long and 12.1 m wide, with nine spans. The superstructure is a PSC I-beam-type girder bridge, which was constructed in 2002 and has been in use for ~20 years. To inspect concrete cracks, scaling, spalling, corrosion and leaks (typical defects for concrete bridges), equipment suitable for the material was selected. The Yeondae Bridge, like the Geumdang Bridge, was constructed in 2002 and is a six-span bridge with a length of 180 m and a width of 12.1 m. In this bridge, representative deterioration items for steel bridges (e.g., peeling paint, corrosion and water leakage) are inspected. When inspecting the bridge, the model in Section 3 was used for the equipment. The inspection members and equipment utilized at each bridge are as shown in

Table 13. Smart inspection technology applied to each bridge member.

Inspected Members	Techniques
Pavement; expansion joint	Drones
Deck	Drones; wire cam
PSC I-type girder	Drones; wire cam
Steel box girder	Rail cam
Cross beam	Wire cam
Abutment; Pier	Drones; wire cam; winch cam
Bridge shoes	Time-lapse camera; spherical drone
Drainage	Thermal imaging module
Additional accessible area	Smart inspection kit

.

To minimize the inspector’s involvement, the aforementioned technology was fully implemented; the configuration diagram for the smart equipment is shown in Figure 8. When working, we inspected the bridge from top to side to bottom. For bridges where access to the lower components is difficult, the equipment can be operated safely on the surrounding flat ground to ensure the safety of the inspector.

We utilized two types of rotary-wing drone to inspect the overall structure and a spherical drone to inspect the bottom of the deck and inter-girder spaces. The modules installed during drone inspections use basic RGB devices and thermal imaging modules. The RGB module was used for basic exterior inspections and the same area was inspected again using a thermal imaging module. Photographs were taken at regular intervals in areas as close as possible to the places where vortexes did not occur. All drones were operated safely to prevent accidents due to falls and collisions.

The inspection scene obtained using the drone is shown in Figure 9.

The spherical drone was employed to solve the GPS reception problems at the bottom of the bridge and the drone-body control problem attributable to vortexes. When inspecting a bridge using a drone, the same inspection area was compared through RGB and thermal imaging modules. Using the RGB module, general damage such as cracks was confirmed, and efflorescence and water leakage could be clearly identified through the thermal image. The inspection results obtained using the thermal imaging module on the drone for the Geumdang Bridge and Yeondae Bridge tests are shown in Figure 10 and Figure 11, respectively.

In concrete bridges, concrete leaks and efflorescence can be clearly identified using the thermal imaging module. For steel structures, it is more effective to identify deterioration using temperature difference, and the scaling and spalling of girders can be easily confirmed. Observing deteriorations via temperature differences is the main principle; hence, deteriorations can be seen more clearly in the steel structure which is more sensitive to temperature changes. The thermal imaging module can identify additional degradation items that might be overlooked in an RGB image. A wire cam was used to inspect the bottom of the deck, the crossbeams and the stringers of the Geumdang Bridge (a PSC I-girder bridge). The equipment was operated by temporarily installing wires to cover inspection routes on abutments and piers; it was operated during the day when light conditions were sufficient. Figure 12 shows the operation of the wire cam under the girder.

Internal inspection of the steel box girder in the Yeondae Bridge was performed by installing a temporary rail and operating a rail cam. The rail cam was tested by installing temporary rails inside the steel box girder. The robot cam was moved remotely on the installed rail and could capture the entire 360° space from the desired location. The mounted gimbal was designed to rotate 360° (up and down, left and right), and the inspector could check the contents from the outside via the monitor. The rails needed to be installed separately and maintained. However, just a few inspectors can easily inspect the inside of a box girder. Figure 13 shows the operation of the rail cam on the box girder.

Wire cams and rail cams have similar communication and control technologies. The video signal (5.8 GHz band) and radio control (RC) signal (2.4 GHz band) were controlled from a long distance away (max. 1.5 km). To determine the inspection status in real time via remote control, we used the developed RC amplifier (2.0–2.4 GHz). The implementation of smart inspection utilities aims to eliminate blind spots during bridge inspection (e.g., areas that are difficult for site inspectors to access and where it is difficult to bring inspection devices). In addition to the test equipment, various inspection devices are available. A time-lapse camera developed for use as a monitoring device was installed near the bridge shoes and measured the amount of displacement of the shoe with respect to the temperature and time. This was undertaken by an inspector who set the power and inspected the images. The time-lapse camera captured pictures of the shoes at regular time intervals and recorded the displacement over time to examine changes in temperature and displacement. The goal of current camera module development is miniaturization to allow easy use by inspectors; this must be developed whilst considering communication and control systems. Using the developed equipment, inspections were performed for approximately one week, and the battery operated without requiring additional installation. Figure 14. shows the beraing inspection image through the time-lapse camera, and Figure 15. shows the displacement of the support over time analyzed by image processing.

For bridge monitoring, a smart CCTV was installed to detect events and observe responses to live loads and temperature loads. CCTV refers to a camera equipped with various video-analysis functions (in contrast to a general CCTV, which shows only a simple recorded scene). Using this equipment, it is possible to manage traffic volumes efficiently by detecting traffic problems or accidents in real time. In other words, during monitoring, the responses or abnormal movements of the bridge under loads can be confirmed by video or images and continually monitored. In this study, the built-in function of the smart CCTV and traffic image data for the region of interest (ROI) were utilized. The ultimate goal is to measure the amount of deflection attributable to the live load (the vehicle load). For this purpose, machine-learning-based vehicle type classification was performed as in a previous study. The ROI of the test bridge was designated, vehicles passing through the area were classified and the traffic volume was measured using the algorithm, because the live load generated in the upper part of the bridge causes deflection, and excessive deflection affects the safety and usability of the bridge. The main live load that caused deflection was the vehicle load, which relates to the size and weight of the vehicles. Therefore, it is necessary to classify and define the applied load before deflection measurement. Vehicle-type classification and traffic measurement technology for the ROI were empirically tested at the Geumdang Bridge in Yeoju, a testbed facility. A simplified flowchart of vehicle-type classification using CCTV is shown in Figure 16.

In the test, CCTV cameras installed on the Geumdang Bridge were connected, an ROI was set on the upper part of the bridge and vehicles were detected at intervals of 0.1 s. Depending on the domestic situation, by vehicle size/type, six types were classified into cars, buses and trucks, and the results were entered into a database and displayed on a video screen. The classification result by vehicle type can be used for the real-time prediction of the load applied to the bridge (by calculating the weight of each vehicle type) in future research. The model used the single-shot multibox detector (SSD) algorithm [68], which performs several classifications at once using a convolutional neural network. When creating the basic model, 137,743 images from the Miovision traffic camera dataset (MIO-TCD) [69] were used. To obtain data for model improvement, CCTV data from other regions were obtained and used. Example images are shown in Figure 17.

During initial learning, the hard negative mining method was applied more strongly than the existing training method to minimize the number of false positives in the background. For training, the batch size was set to 32, and 90,000 iterations were trained. After training, the evaluation result for the MIO-TCD was obtained with an accuracy of 78.9%, precision of 70% and recall of 71%. As a result of training the SSD algorithm, the deep learning model was composed of a neural network structure trained by the input dataset and weight values generated from each node of the neural network. This model received CCTV videos as the input signal and detected the desired object. If fine-tuning is required for the deep learning model, additional data for the desired vehicle model can be obtained in order to implement additional training and update the model to improve object detection. In addition, by fitting to the area in which the user wants to build the system, the accuracy of data accumulation can be improved.

5.2. Application Test Results

For the application tests, smart inspection kits, utilities and monitoring technologies were applied to the Geumdang and Yeondae Bridges. Generally, bridge inspection results are presented in the form of an external survey network that records the deterioration items of the bridge. The smart inspection results obtained in this study were presented as a 3D image-based external inspection map; this was created using smart utilities and monitoring equipment. The basic image is shown in Figure 18.

Subsequently, the captured inspection images were stitched together to create a 360° image. In this study, flat images taken from outside were created by direct stitching using Photoshop, based on various data by angle and location. When inspecting the interior space, only a part of the lighting was locally located, so a separate strobe was used during shooting. In the stitching process, the drone inspection image utilizes the image obtained from rotational shooting in a state where the nodal point is not aligned. Therefore, if the drone is too close to the subject, the image becomes misaligned. Considering this, the stitching software (Photoshop 2020) performs manual work when comparing the self-stitched and post-processed stitched images. A smart inspection system was constructed using 3D images and expressed in the form of a road view. The road view format can be rotated 360° on the screen, so that all members and the surrounding environment can be verified from the perspective of the actual inspector. This can be coupled with VR equipment to facilitate an intuitive grasp of the inspection location and items via high-resolution images. In addition, by providing a digital VR environment corresponding to the field, data can be shared with inspection technicians at all times within the smart office environment. The road view type of the external inspection map for the 3D-type Geumdang Bridge and Yeondae Bridge models (after post-processing) is shown in Figure 19.

The inspection indicates that the concrete girder of the Geumdang Bridge suffers from many surface defects and exhibits deterioration (e.g., that caused by water leakage from cracks or gaps, layer separation, and spalling). A considerable amount of deterioration was observed across the entire girder and slab, and many concrete cracks were also noted. Although fine unidirectional cracks were observed, a relatively large number of irregular crazings were also found. In the steel box bridge, certain defects (e.g., separation of the paint inside the box or water leakage) were found; however, no deterioration (e.g., poor weldings, loosened bolts or corrosion) was observed. In addition, rainwater leaked into the box, and traces of dampness on the box walls and of water penetration through fine cracks in the components connected to the top plate of the upper bridge were confirmed. The inspections conducted using the abovementioned equipment were faster than those relying on existing manpower; inspections used to take 2–4 weeks, but were shortened to one preliminary and two main inspections.

5.3. Seohae Bridge Application Test

The technologies that passed the basic test were further tested on the Seohae Bridge, a large bridge that is difficult to inspect. The Seohae Bridge is a cable-stayed bridge built in 2000; it is 7 km long and 182 m high. The application test was applied in the cable-stayed section of the bridge, which corresponds to the bridge center and is ~990 m long. Because the Seohae Bridge is an offshore cable-stayed bridge, regular visual inspections are costly and time-consuming. In this study, the inspection technology and system verified in the previous tests were applied in the locations most difficult to inspect (i.e., because of environmental conditions), using the new technology. Drones and unmanned equipment were operated from the lower part of the pier and approached minimally, avoiding danger to traffic during operation. A detailed inspection of the lower part of the bridge deck was conducted using an installed inspection rail. The results obtained using drones, rail cams and 360° cameras are as shown in Figure 20.

The outer parts of the girder, pylon, and substructure of the bridge were assessed using a drone. The side of the bridge and the areas surrounding the pylon were inspected from further away than other bridge members because of the risks associated with vortex flows. An inspector assessed the inside and outside of the pylon connection passage, primarily using the inspection kit and motorized panoramic head. Rails and wire cams were installed to investigate the bottom of the bridge. The crossbeam and slab were inspected separately using the wire cam. In addition, for the first time since construction, the inside of the pylon could be inspected using various cameras. The interior was inspected at appropriate intervals set according to the presence of obstacles; flash was used as a basis, and the white balance was adjusted to produce clear images. To inspect the elevator area (inside the pylon), a rotating panoramic head was installed by placing a separate mounting bracket at the top of the elevator. This equipment made it possible to inspect the inner surfaces of the large pylons. The basic image obtained through internal inspection and the image form used for 3D transformation are as shown in Figure 21.

In addition, a roadview-type external inspection map was created by converting the inspection image obtained during the external survey into a 3D model as shown in Figure 22.

To measure the displacement of the bridge shoes, a lightweight time-lapse camera was used and compared against the prototype model from the previous application test. The maximum movement of the bridge bearing, as analyzed using the image obtained by the time-lapse camera, is as shown in Figure 23:

For effective inspection, it is necessary to continuously observe change in movement with respect to the seasonal temperature; in the current application stage, the usability was monitored for ~1–2 weeks. The Seohae Bridge was inspected over a period of approximately three months, including additional supplementary inspections. A total of five inspections, one preliminary inspection and four main inspections were conducted by a group of three people in the aerial photography stage. Inspecting the Seohae Bridge via existing methods leads to many social and economic costs, including numerous personnel, equipment and traffic blockages. However, these problems can be solved through the application of the technology verified in this study, and the inspection time and cost can be drastically reduced.

No defects found on the inside or outside of the Seohae Bridge had increased since the bridge’s date of completion. Minor corrosion was observed on some of the bolted connections of the girder, and only a small portion of the paint had peeled off. In addition, efflorescence and cracks were detected inside the previously un-inspected pylon. It was confirmed that this 20-year-old bridge located on the seashore had been thoroughly managed.

6. Conclusions

In this study, the applicability of a new inspection technology design to replace the purely visual inspection of bridges was tested. The basic applicability of the technology was tested on Geumdang Bridge and Yeondae Bridge, test bridges located in Korea; an advanced test was conducted upon the Seohae Bridge (a large sea-based cable-stayed bridge) to assess the technology’s potential use in extreme situations. We developed a 3D external inspection map using image and video-based inspection technology as well as a smart inspection system based upon utilities, monitoring, and inspection kits. In addition, the test showed that it was possible to determine the status of current technology as well as the direction of future technology development. Based on these results, the following conclusions were drawn.

• Drones, rails and wire cams, which are representative utility equipment, were used to inspect the exterior and bottom of the bridge, to verify its usability. The camera module of the device could be changed between a high-resolution or thermal imaging module as needed. An expensive module must be used to achieve a resolution of 0.1 mm (the current crack inspection level); however, this remains an effective method in the long term because it can be used semi-permanently and reduces labor costs. In addition, the real-time video-sharing function between the office and inspector was confirmed through application tests of smart glasses, neckband cameras, and 360° cameras that inspectors can wear. The video- and image-based inspection data obtained from the equipment were developed into a 360° VR image instead of the existing external survey network. This is directly expressed in a 3D image; thus, both inspectors and managers can understand it intuitively. It is also suitable for training and preliminary inspections.
• A monitoring system was installed for continual bridge inspections. A smart CCTV system was installed on the side of the Geumdang Bridge to predict emergency situations and the load on the upper part of the structure; furthermore, vehicle classification and traffic volume measurement algorithms were developed from the obtained traffic video data. Using the basic and acquired data, a deep-learning six vehicle classification SSD algorithm was developed and applied to extract the vehicle types and hourly traffic volumes in the area of interest.
• A small time-lapse camera was developed for the real-time inspection of bridge integrity under the temperature-difference-induced loads, facilitating the image-based periodic inspection of bearing movements. The equipment additionally measures environmental factors (e.g., temperature and humidity) and transmits the information to a database in the office so that managers can assess the integrity in real time. For the current experiment, a battery was installed to provide sufficient energy to allow the device to capture images over approximately two weeks, though solar energy or other energy sources could be used as needed. The equipment was miniaturized and developed to be portable.
• The smart office connected the smart inspection technologies, and inspection kits and utilities facilitated video sharing and real-time communication with operators. In addition, the health and status of the facilities could be managed in real time via a monitoring system. We established a system that could digitalize monitoring and inspection information by dividing it into a certain time interval, systematically managing it and converting it into a database.
• The technologies were first verified on the Geumdang and Yeondae Bridges (which are test bridges); then, the technology was tested on the Seohae Bridge to develop a smart inspection system. The Seohae Bridge is Korea’s largest offshore cable-stayed bridge; its size and surrounding environment make on-site inspections and the application of new technologies very difficult. However, by applying the smart inspection kit alongside the inspection utilities currently in use, a 990 m length of the cable-stayed bridge could be inspected with only 2–3 people over ~3–4 weeks. Compared to the existing visual inspection methods, this system can conduct inspections in a short time with a very small number of personnel. It was possible to inspect the outside and inside of the pylon, which were previously difficult to access. Similar to the first application test, the inspection results could be converted to a 360° image-based VR-type exterior survey network.
• The generated 3D external inspection map was linked to the VR equipment. VR made the bridge appear realistic and reflected the inspection details; it can be used for pre-inspection training and site inspector safety training, and it allows managers and experts to analyze inspection results.
• The Seohae Bridge inspection costs more than $200,000 for regular inspection every year and $700,000~$800,000 for detailed inspection every 2–3 years. In this study, the cost was reduced by constructing and utilizing a smart inspection system to operate on a higher level than the existing method of inspection which was performed visually by on-site inspectors. Roughly calculated through the technology application test, the inspection cost was about $160,500, which was a 19% reduction compared with the previous method. In addition, the system configuration technology is gradually developing cost-effectively and the system can be expanded, so the range of alternatives is gradually expanding.
• This study has some disadvantages and limitations at the current stage. It is a matter of principle that expensive modules must be used for high-level inspection and that the results inspected by the device are not officially recognized. However, the performance of the module is gradually improving cost-effectively. As unmanned inspections are being used in many local governments as the number of cases for inspection increases, the need for standard changes is gradually being emphasized. In addition, it is necessary to combine various technologies for data analysis, so much development is needed in the future based on the basic system of this study.

7. Future Research

In this study, image- and video-based inspection equipment were selected as representative equipment offering both current applicability and high scalability. The proposed technology and system are in the early stages of smart inspection development. Automation (with few or no inspectors) is the final goal, and it is necessary to supplement and develop technologies to this end. Alongside the basic equipment, sensor systems such as accelerometers, displacement meters and tilt sensors linked to monitoring systems are being developed to improve monitoring and regular inspection functions. New utility equipment (e.g., winch cams and monorail cams) are also being developed. Equipment modules to satisfy inspection conditions (i.e., crack and surface damage width) for bridge deterioration items are also being developed. In terms of office technology, if edge computing is used, it is believed that more information can be efficiently provided to on-site inspectors.

Currently, the goal is to collect bridge inspection data, and there are plans to turn it into big data based on a video-based inspection system. Based on the data obtained, we will further combine technologies that detect and analyze deterioration items.