Potential Beneﬁts of Combining Anomaly Detection with View Planning for UAV Infrastructure Modeling

: This paper presents a novel method for UAV-based 3D modeling of large infrastructure objects, such as pipelines, canals and levees, that combines anomaly detection with automatic on-board 3D view planning. The study begins by assuming that anomaly detections are possible and focuses on quantifying the potential beneﬁts of the combined method and the view planning algorithm. A simulated canal environment is constructed, and several simulated anomalies are created and marked. The algorithm is used to plan inspection ﬂights for the anomaly locations, and simulated images from the ﬂights are rendered and processed to construct 3D models of the locations of interest. The new ﬂights are compared to traditional ﬂights in terms of ﬂight time, data collected and 3D model accuracy. When compared to a low speed, low elevation traditional ﬂight, the proposed method is shown in simulation to decrease total ﬂight time by up to 55%, while reducing the amount of image data to be processed by 89% and maintaining 3D model accuracy at areas of interest.


Introduction
The advent of small Unmanned Aerial Systems (sUAS) has given rise to a host of new applications for aerial imaging technology in many fields [1][2][3][4][5].Together with the increasing ease of obtaining imagery, advances in computer vision and computer processing power have led to a widespread increase in aerial mapping and 3D-reconstruction [6][7][8].In the field of infrastructure monitoring, the clear advantages of on-demand, high precision 3D modeling are driving companies and researchers to explore the possibilities of this technology (for an excellent overview, see [9]).However, current UAV and 3D reconstruction technology still has limitations.While excellent results can be obtained for single site projects as demonstrated by [10], UAV flight time, computational power, data storage and model processing time all constrain the scalability of this technology to large-scale infrastructure systems, such as pipelines, canals, levees, railroads, utility lines and other long linear features [11].Because of these constraints, creating a single detailed 3D model of a large infrastructure object is in many cases impractical.
In this paper, the authors attempt to address some of these limitations by introducing a concept in which a single UAV platform serves as a multi-scale monitoring system.The UAV first inspects the structure at a relatively high level, collecting imagery and searching for potential anomalies.A detection triggers a 3D flight planning algorithm that updates the UAV path to collect additional images of the detection area.This allows the area of interest to be reconstructed at a higher level of detail than would be possible in the baseline survey.This multi-scale monitoring concept focuses flight time and computational resources on areas of the infrastructure with high information content.
Although there is a large body of work relating to UAVs tracking moving objects and real-time detection of specific objects, real-time detection of unknown anomalies is a challenging problem, and this paper does not attempt to address it directly.See [12][13][14][15][16][17].Many possible approaches to detection are possible, including cascade classifiers, neural networks or change detection between monitoring flights and a known baseline dataset [18].The approach could also be expanded to other sensors such as multispectral or hyperspectral sensors, real-time stereo, LiDAR, chemical sniffers or any other sensor capable of detecting the desired anomalies.The body of the work proceeds on the assumption that accurate detections can be made, and the focus is placed on the potential benefits of a multi-scale approach in terms of model accuracy, flight time and data quantity, with the goal of motivating further work in real-time detection technology and algorithms.

Related Work in Linear Feature Monitoring
Although the use of UAV technology in civil applications is a relatively new field, a number of authors have explored various aspects of monitoring long linear infrastructure with UAVs.The work in [19] analyzes a wide variety of UAV platforms and sensors applicable to infrastructure inspection.They comment on what is currently available "off the shelf" to gather optical data of infrastructure using relatively low cost UAV systems.They also conduct a test case using a small quadrotor UAV to inspect a section of railroad tracks.The work in [20] considers two possible UAV systems for the inspection of natural gas pipelines.The first system, a low altitude, small UAV with visual sensors, is most similar to the UAV platform considered in the current work.Portions of the system are demonstrated, including automatic detection and marking of areas of concern.The authors conclude that although a lack of operation standards impedes immediate adoption of the technology, the system is technologically feasible in this application.
In another application, Zhang et al. [21] study the high speed inspection of power lines using UAVs.Their focus is primarily on automatic detection and tracking of the lines in video.The work in [22] details the implementation of an algorithm that utilizes optical imagery to guide a small UAV along long linear infrastructure, such as canals, roads and pipelines, without the use of GPS waypoints.Several experiments were conducted that demonstrated the effectiveness of the algorithm in navigating a small UAV over canals and roads to collect optical imagery.They conclude that small UAVs can gather a large amount of optical data of long linear sites in a relatively short period of time.
The work in [23] proposes a navigational framework to detect and track road networks.They were able to demonstrate via simulation that their framework could be used to navigate a UAV over long stretches of road for the purpose of visual inspection.The current work extends upon these studies by considering the case of a UAV platform that not only passively monitors infrastructure, but actively re-plans its flight path using optimized view planning when anomalies are detected.

Optimized View Planning
The work in [24] defines view planning as the strategic placement of sensor(s) to gather desired information about an object or scene.The origins of view planning are discussed by [25].View planning has its earliest roots in the "art gallery" problem, where one desires to place security guards in optimal locations to monitor an art gallery.The work in [26] examines how view planning has also been used in quality control, where ideal locations for sensors are desired to ensure product quality in manufacturing.
Modern examples of view planning often utilize remote optical sensors, and the placement is typically automated by robotic arms, gimbals and unmanned aerial vehicles.The work in [27] describes a step-by-step process for automating the surveillance of a construction site with a UAV, with an emphasis on the UAV's ability to easily survey areas that pose significant danger to human operators.A simple grid pattern or 'lawnmower' pathway is used at a set altitude, with the UAV constructing an orthophoto from images taken exclusively in the nadir position.The lawnmower pathway is commonly used in UAV applications due to its simplicity and even coverage.
The view planning problem is described by [28] as NP-complete, which is the most difficult subset of problems.The NP-complete class of problems is characterized by having an easy method to check if solutions are correct, but having no known algorithm for efficiently generating solutions.
Work by [29] shows the difficulty of the viewing problem.Saadat and Samadzadegan use a genetic algorithm to create a Pareto front where multiple optimized viewing networks are generated.However, an experienced operator is then needed to select which one is anticipated to result in a superior network.
The work in [30] emphasizes the need to generate view planning solutions for UAVs that give the best results given constraints, such as the flight time or battery life of the UAV.They address the situation of a UAV tasked with surveying more infrastructure objects than can be visited within a single battery charge.A weighted objective function is used that assigns rewards for exploring unknown areas, as well as inspecting certain points that are deemed more important.The authors use a simple heuristic approach to connect the waypoints, solving the traveling salesman problem with an approximation.Their overall method yielded good results in real time, even though sub-optimal heuristic approaches were used.
The work of [31] presents an algorithm for efficiently generating potential view points, then heuristically selecting the view points that best contribute to network strength.The flight planning method presented in the current work builds upon the algorithm developed by Hoppe et al.Their method is extended from known 3D structures to unknown objects located on potentially flat surfaces.The new algorithm presented is also configured to be called in real-time by the UAV during the flight to re-plan the mission based on updated information.

Simulation
Experiments for this paper are performed in both a terrain simulation environment and a software-in-the-loop flight simulator.This approach is relatively novel in this field, but it has been demonstrated in some capacity before [32].In one example of a similar approach, Piatti and Lerma [33] develop a simple 3D environment for use in photogrammetric simulations.They demonstrate a process for reconstructing perspective imagery from their scenes with user-specified camera parameters.While some mission planning capabilities are also shown, the scene geometry is limited to simple shapes, and the software package is developed primarily as a learning tool for students.The work in [34] also uses a simple 3D simulation, though in this case, the objective is to evaluate the effects of various UAV image configurations on the systematic error in the resulting DEM.
Although uncommon to this point in civilian applications, the simulation of UAV surveillance flights using synthetic imagery generated by military simulation packages has also been explored.In particular, the simulation environment Virtual Battle Space 3 has been used by [35] to perform experiments in the context of a UAV road monitoring and change detection problem.The authors concluded that although some artifacts were introduced by the rendering process, the resulting images were of sufficient fidelity to evaluate their computer vision algorithms.
Terragen 3, the environmental simulator used in the current work, has been used successfully by the authors in a previous paper, in which it was used to evaluate the performance of a genetic algorithm-based planner for UAV infrastructure modeling [36].This approach differs from related work in the field through an increased level of fidelity in the simulation environment, including highly detailed vegetation models, realistic atmospheric effects, cloud cover, haze and fine control of camera, lens and lighting conditions.This allows for simulations more closely approximating actual field conditions and increases the amount and quality of testing that can be performed prior to physical flight tests.

Novel Contributions and Paper Overview
This paper advances the state of the art in the following areas.

•
It is shown that when a reliable anomaly detection system becomes available, the proposed method will be capable of generating detailed 3D models of the areas of interest while avoiding the often unwieldy amounts of data produced by repeatedly creating 3D models of the entire structure in routine inspections.

•
The monitoring system incorporates on demand optimized view planning, taking advantage of the onboard processing capabilities of unmanned aircraft to maximize information gain through in-flight re-planning.

•
The potential benefits of this method are demonstrated and quantified in simulation, motivating further work on the supporting automatic detection technologies.
The paper begins with an overview of the simulation environment used to perform the experiments described in the paper.This is followed by an explanation of both the standard linear and the optimized flight paths used for generating synthetic UAV images in the simulation environment.The process of 3D model creation and accuracy testing is described, as is the configuration of the real-time simulated flight testing system.The results of the testing for 3D model accuracy, flight time and data quantity are then presented, and the implications of the results are summarized and discussed.

Simulated Test Scene
Synthetic images for this study are generated using the terrain simulation environment Terragen 3. The simulated scene is based on 0.5-m elevation data taken from a one-mile section of canal near Payson Utah.Satellite imagery at 12.5-cm resolution is overlaid on the elevation data, and high quality 3D vegetation and trees are added to approximate the actual environment at the site.Lighting conditions and the position of the Sun are chosen to match those for Payson, Utah, on 28 March 2016 at 12:00 p.m. Figure 1 shows a portion of the completed simulation site.In order to simulate a wide variety of potential infrastructure anomalies, additional 3D objects are introduced into the scene and placed at intervals along the canal.These included a set of power lines with structural damage, a colored disc placed on a road, a section of industrial piping with horizontal displacement and a short segment of railroad with a tie out of place.A slump in the side of the canal is also simulated by displacing a portion of the canal bank and filling the vacant area with water.Each anomaly in this study is a displacement on the order of 50 cm from a baseline case.A full description of the exact nature of the anomalies is not given here, as the purpose of the anomalies in this paper is only to illustrate potential locations where they could be found.A follow-up study will investigate the specific types of anomalies that can be detected and quantify the limitations of the method with regards to anomaly size.Closeups of four of the anomaly locations can be seen in Figure 2. Terragen 3 permits full control over camera positioning, orientation and parameters.Camera parameters for this study are chosen to simulate a Nikon D7100 DSLR camera and are detailed in Table 1.
While the simulation engine used is capable of generating photo-realistic images, it is not a perfect representation of reality.In particular the effects of wind on the scene and the rolling shutter effect due to camera movement are not accounted for.These effects are important in UAV photogrammetry, and including them in a simulation environment would be an interesting area of future work.

Standard Path Planning
To facilitate comparison with standard flights, a planner is created to generate flight paths along the canal.Given a vehicle speed, desired image overlap and camera frame rate, the planner generates a sequence of image locations at the required spacing and elevation.The planner is capable of generating single track paths, in which the UAV flies only one direction, as well as double track, in which the UAV flies down and back.This is illustrated in Figure 3.In the case of the double track path, the side overlap between the two tracks can also be specified.Image locations are generated for a range of flight speeds between 20 and 60 mph and image overlaps between 75% and 90%.Double track flight paths are also generated, with a 50% overlap between the tracks.These ranges are based on commonly-used corridor mapping guidelines [37].All paths are created using a fixed camera frame rate of 0.5 frames per second.
The required elevation and number of images for each path generated by the standard path planner are shown in Tables 2 and 3.Because the camera frame rate has been fixed at 0.5 frames per second in this study, some combinations of speed and image overlap produce a very high elevation requirement.These elevations are outside the typical operating range of a small UAV, but are included for completeness.

Optimized 3D Flight Path Planning
Calculations that result in the global optimum for view planning are difficult to formulate and lead to very expensive computational loads.The view planning algorithm developed in this paper therefore focuses on identifying a sub-optimal solution capable of producing desirable results with a computational load that permits near real-time calculations.The algorithm includes a high degree of adaptability to different UAV platforms and desired image resolutions by accepting several user-specified parameters that define the capabilities of the camera and the desired resolution.Given these parameters, the algorithm estimates the minimal set of camera locations required for 3D reconstruction, as well as the optimal order in which to visit them.For a typical site of interest (about 900 m 2 ), the algorithm finds acceptable image positions and a near optimal route through those points in about 0.65 s on an Intel I7 CPU with 16 GB of RAM.
As it is desirable for the algorithm to eventually run on-board a UAV, the algorithm is also tested on a Raspberry Pi 3. The Pi 3 is a small, single board computer with a 1.2-GHz 64-bit quad-core ARMv8 CPU and 1 GB of RAM.The board measures 85.60 mm × 56.5 mm and weighs 45 g, which makes it feasible for use on-board a UAV.Running on the Raspberry Pi 3, the algorithm took an average of 7.02 s to compute both the image positions and route for the five test areas in the study, placing it within range of in flight re-planning.
The view planning algorithm can be summarized as follows: 1. Receive detected anomaly location 2. Obtain a point-cloud 3. Add dome projection at the point of interest 4. Convert the point-cloud terrain into a triangle mesh 5. Place an aerial camera location on the normal line of every triangle 6. Remove underground and blocked images 7. Select the best images from the group based on a set of value heuristics 8. Find the shortest path through all of these points The following is a description of each of the above steps.Elevation data for the area to be surveyed is downloaded prior to the flight from public data sources, such as the USGS National Elevation Dataset.During an actual UAV flight, points of interest are flagged by anomaly detection; however, for the purposes of this paper, the points of interest are manually tagged before the simulated flight begins.When a point is detected, a geometric area of interest is constructed around the point to define the area to be inspected.
After identifying the area of interest, the full elevation dataset is trimmed down to an area only slightly larger than the area of interest.The slight excess of data is retained to ensure the UAV remains safe from potential obstacles or obstructions.Using only a subset of the original data also greatly helps to minimize computational costs when determining which points are within the view of each camera.
In many cases, the geometry of the detected anomaly is unknown and is not included in the available elevation data.This can lead to insufficient oblique imagery of the anomaly, especially if the shape of the anomaly differs significantly from the underlying terrain.To improve coverage, when an anomaly is detected, the algorithm automatically inserts a dome projection at the point of interest, as demonstrated in Figures 4 and 5.The authors have found that the addition of the dome into the dataset aids the algorithm in selecting sufficient oblique imagery to capture the sides of an unknown object, regardless of whether the object is on relatively flat ground or on a steep grade.The dome is created by translating the elevation of each data point upward using a spherical cap, which results in a dome shape that still maintains the underlying geometry of the surface.Without the dome, a flat surface would result in the camera set all pointing directly downwards, and the resulting 3D reconstruction could yield undesirable results.The authors have found that a dome height of 5-10 m and an incident angle with the horizontal of 15 • -30 • provide an optimal increase in coverage.If the approximate size of the anomaly is known, it is best that the dome is tall and wide enough to completely enclose the anomaly.In the case that additional information is available about the nature of the anomaly, that information should be used in the planning rather than the generic dome projection.To clarify, the dome is not inserted into the Terragen simulation, it is only used in the view planning algorithm to aid in selecting imaging locations.
These resulting elevation data are then converted to a triangle mesh, and the triangle size is normalized by recursive subdivision.This ensures even coverage of vertical features, such as cliffs, hills and canyons, even though the original telemetry data are spaced evenly for latitude and longitude.
After creating the triangle mesh and adjusting the terrain, tentative camera locations are generated at a fixed distance from each triangle midpoint along the normal from each surface.This typically results in several thousand tentative camera locations.Potential camera locations that are generated underground or are otherwise blocked from viewing their original triangle midpoints are removed.
The image selection criteria is based on minimizing the total number of images while maximizing the terrain coverage.A visibility matrix similar to the 3Mmatrix used by [38] is constructed, containing information on the visibility of each point in the terrain from each camera location and the angles (relative to each surface's normal) at which each point is viewed.The angles are divided into three ranges (see Figure 6), and the resulting range is recorded in the histogram.An iterative process then selects camera locations, which maximize the number of triangle midpoints viewed from previously unused angles.That is, once a point is viewed from an angle range, it is considered viewed, and it no longer contributes to the value of other camera locations, which view it from within the same angle range.This results in a sub-optimal heuristic algorithm, which, while not guaranteed to be optimal, can be completed very quickly with satisfactory results.As noted in [39], this type of heuristic algorithm provides a (1 + lnn) approximation, where n is the number of images in the optimal solution.Once the optimal images are chosen, a short route must be planned for the UAV to visit all of the points; this is an example of a 3D traveling salesman problem.To find a solution, the Christophides algorithm is employed.This algorithm is an approximation, but is guaranteed to find a solution within 1.5-times the length of the optimal solution [40].For this application, it was observed that the Christophides algorithm typically generates solutions that are within 10% of the optimal length as computed by integer linear programming, as shown in Equations ( 1 Here, each of the k waypoints is numbered from 1-k, and i and j represent the waypoints at either end of a selected pathway.D contains the distance measurements between each of the waypoints.The end solution x contains integers bounded between 0 and 1.The equality constraints are shown in Equation ( 2), and the inequality constraints are shown in Equation (3).
If a solution is achieved with subtours, Equation ( 3) is added as an inequality constraint, and the problem is solved again.The variable s represents a subtour that occurred in a previous solution.
A sample comparison between the Christophides solution and the optimal solution from linear programming can be found in Table 4.A completed solution for a generic anomaly, including selected camera locations and the shortest flight path, can be found in Figure 7.For comparison, Figure 7 also shows the optimal solution when the dome projection is not used.For comparison with Tables 2 and 3, the number of images taken by the optimized path planner at each anomaly location, as well as the average elevations of the images are shown in Table 5.The number of images required to model the section of industrial piping is significantly larger than the other anomaly areas.This can be explained by the fact that this anomaly location has the largest area of the five due to the fact that it contains the largest structure.

3D Modeling
The synthetic imagery from the terrain simulation environment is processed to created 3D point cloud models in the software package Agisoft Photoscan [41].No ground control points are used in processing the models, and the software is allowed to self-calibrate all camera parameters.An example of one of the 3D models generated is shown in Figure 8.The program settings used to generate the models are detailed in Table 6.

3D Accuracy Testing
For each flight case in Tables 2 and 3, the accuracy of the 3D model is measured at each of the five anomaly locations.Similar measurements are performed for the optimized case.The accuracy of the 3D models is evaluated by comparing against a ground truth point cloud created by exporting the original geometry of the scene from Terragen.This includes terrain, vegetation and 3D objects and preserves the scale of the scene.The geometry is exported as a 3D mesh and is then sampled at a high density to create a point cloud.
Each model is evaluated for accuracy against the ground truth model using the techniques described in [36].The models are aligned to the ground truth dataset using the open source software package CloudCompare [42].This alignment is done in 2 parts.The first is to roughly align the models using 4 pairs of corresponding points on each model where the chosen points correspond to distinct features like hard edges of canals or other structures.These points are used to scale and rotate the compared model relative to the ground truth data.Once the models are roughly aligned, the alignment is refined by using an iterative closest point (ICP) algorithm.The ICP algorithm minimizes the distance between two point clouds by using a mean squared error cost function to estimate the rotation, translation and scaling that most closely aligns the two point clouds.With the models aligned, a local quadratic fitting technique is used to determine the distance between each point of the compared model and the interpolated surface of the ground truth model.This technique is shown in Figure 9.The cloud to cloud differences are finally fit to a Gaussian distribution, and the mean error between the surfaces is computed (see Table 7).The standard deviation of the error is also computed.

Simulated System Implementation
Following the collection of simulated images, the physics of each flight is simulated using the ArduCopter Software-in-the-Loop (SITL) package.The main objective of these simulations is to compare the flight time required for each path.The optimized flight planner is implemented in Python and connected to the flight simulation using the 3DR DroneKit library.For these simulations, the desired waypoint navigation speed is set to 60 mph.It is assumed that the UAV platform used has the flight characteristics of a multi-rotor, though for longer endurance, it would be preferable to use a larger single-rotor craft or a UAV capable of transitioning between hovering and forward flight.
The base flight elevation is set to 90 m, which corresponds to a 10-cm Ground Sampling Distance (GSD) for an HD video camera.This is done to simulate a two-camera multi-scale setup in which most of the long linear feature is captured using lower resolution video, while points of interest are imaged using a higher resolution camera.
The UAV is set to fly a path along the one mile canal segment simulated in earlier sections.For this study, optimized flight planning is executed in-flight; however, anomaly detection is not performed in real time, and anomaly locations are marked in advance.When the vehicle reaches each of the pre-defined anomaly locations, the location of the anomaly is passed to the optimized flight planner, and an inspection path is generated in real time.The UAV then executes this path before continuing with its original flight route.The flight path taken by the UAV can be seen in Figures 10 and 11.The average planning time for each anomaly inspection is approximately 2.3 s.To create a baseline for comparison, two additional flights are performed with no anomalies present.The first flight is performed at 20 mph and provides an upper bound on flight time.The second, at 60 mph, provides a lower bound on flight time for the conditions tested.

3D Accuracy Testing Results
The results of the 3D accuracy testing described in Section 2.5 are shown in Tables 7 and 8.Note that because the flight in Case 1 is relatively close to the ground, the piping is outside the camera field of view and thus does not appear in the 3D model.As summarized in Figure 12, the average accuracy of the 3D models produced by the multi-scale approach is 11.0 cm.This is much better than the majority of the traditional flights, which average 29.8 cm, and is matched only by the low, slow flight of Case 1 with 11.5 cm.An unexpected result from this study is the decrease in accuracy between the single path and the double path, as the double path is expected to be more accurate.The authors believe that this result stems from the setup of the test scene, in which the canal lies directly between the two paths in the double path scenario.Terragen uses a realistic lighting model, which includes specular highlighting and glare on water.The authors believe that the moving highlights and glare on the canal during the flight make it difficult to match points between the forward and backward path during the 3D reconstruction and lead to decreased accuracy.Because the single path cases are flown directly above the canal, glare is reduced, and this problem is avoided.This is illustrated qualitatively in Figures 13 and 14. Figure 13 compares the original geometry of a model segment to the single path and double path reconstructions.Figure 14 shows a cross-section of the canal from one of the double path models.In both double path cases, it can be seen that the model is inaccurate and disjointed at the canal, supporting the author's conclusions.

Flight Time Results
The proposed monitoring system shows improvements over traditional methods in terms of flight time.Based on the simulated results, the average inspection time per anomaly including planning and flight time is 1.89 min.The flight time savings of this approach scale according to the number of anomalies detected during the flight.Figure 15 shows the case of a 100-mile flight, comparing traditional 20-mph and 60-mph monitoring flights to the multi-scale approach.Multi-scale flight times are calculated by multiplying the average anomaly inspection time by the number of anomalies found per mile and adding that time to the time required for a baseline 60-mph flight.As Figure 15 shows, flight time savings increase with decreasing anomalies, approaching the lower limit of the 60-mph flight.One anomaly every five miles produces a 55% savings in flight time versus the 20-mph case.The break even point or the point at which a 60-mph flight with inspection stops takes the same amount of time as a 20-mph flight without stops is 0.67 miles per anomaly.Combined with the above accuracy testing results, this means that for cases with anomalies spaced farther than 0.67 miles apart, the multi-scale approach produces 3D models at the same accuracy as a low slow flight with less flight time.

Data Quantity Results
The proposed methods also show improvement over traditional methods with regards to the amount of data collected.A major problem with current 3D reconstruction techniques is the large amount of data generated, which makes viewing, analysis and storage difficult on a standard workstation, requiring investments in more powerful computing resources.Reconstruction in particular scales poorly, with complexity estimates ranging from O(n 2 ) to O(n 4 ), where n is the number of images [43,44].
The new method alleviates this problem by focusing data collection and processing power on infrastructure areas with high information content.High resolution images for 3D reconstruction are captured only in problem areas, leaving the remainder of the infrastructure to be captured in lower resolution video for further review if needed.The benefits of this approach in terms of data savings are expressed in Figure 16.The values in Figure 16 were generated using Equation (4).
This equation describes the gigapixels of imagery collected over a flight using the two camera setup described above.Here, FPM (Frames Per Mile) is the number of HD (1920 × 1080) video frames needed per mile to achieve the desired percentage of FO (Frame Overlap) at the chosen flight speed.M1 is the number of megapixels per image captured by the HD video camera.MPA is the average number of Miles Per Anomaly detected.IPA (Images Per Anomaly) is the average number of high resolution images needed per anomaly.M2 is the number of megapixels per image captured by the full-resolution camera.For comparison, Figure 16 also includes the amount of data generated by capturing full resolution imagery of the entire length of infrastructure during 20-mph and 60-mph flights with 90% overlap.The parameters used to generate Figure 16 are given in Table 9.
Together, the results for 3D model accuracy, flight time and data quantity provide a number of interesting insights.If accurate anomaly detections are possible, the proposed multi-scale infrastructure monitoring approach has the potential to reduce flight times by up to 55% and the quantity of data generated by up to 89% while maintaining accuracy at areas of interest when compared to a low altitude, low speed flight.

Discussion
Much of the research done using UAVs for inspecting long linear infrastructure has focused on using vision to track and follow infrastructure features such as roads and pipelines [22,45,46].These tracking methods could potentially be combined with the results of this project to create an even more capable system.This project is more similar to the ideas presented in [47], where a network of sensors on an oil pipeline triggers a UAV to investigate an area of possible tampering.In contrast, the method in the current paper presents the idea of using sensors on-board the UAV itself to trigger flight planning for an area of interest.The view planning algorithm presented is similar to others in the literature [31,48], but as demonstrated provides value when combined with the ideas of anomaly detection and on-board flight planning.This is in contrast to most UAV re-planning literature, where in-flight re-planning is mainly used in exploring unknown environments [49,50].
The simulation environment used can be considered both a strength and a limitation of the study.It is a strength because it allows a variety of flight plans and imaging sequences to be constructed and evaluated, which can be difficult in physical tests.It also allows experiments to be conducted in custom environments that can be tailored to test a large combination of specific conditions.However, as no simulation is perfect, this adds the requirement that the results must eventually be validated in physical tests.The authors plan to confirm and extend the results of the study in future real-world flight tests.

Conclusions
This paper proposes a novel method for inspection and monitoring of long linear infrastructure features using UAVs equipped with real-time anomaly detection and in flight re-planning.A simulated test environment is constructed and used to collect synthetic photographs as if taken from UAV flights.A 3D flight planning algorithm suggests sub-optimal, but sufficiently accurate waypoints within acceptable CPU time constraints.The procedure and settings used for 3D model reconstruction are also detailed.Standard single and double linear flight paths are compared against the new method in terms of 3D model accuracy, flight time and the quantity of data collected.A basic demonstration of anomaly detection using a Haar classifier is shown and additional alternatives discussed.
It is shown that compared to a linear flight, the proposed method is able to maintain 3D accuracy at the areas of interest while reducing flight time by up to 55% and the amount of data generated by up to 89%.This reduces the time required in the field, the image storage required on-board the UAV and the computer power and storage required for data post processing.These savings reduce the time and cost associated with the monitoring of long linear infrastructure, such as pipelines, roads and levees.
This paper touches only briefly on the detection of unknown anomalies, instead focusing on quantifying the potential benefits given the assumption such detections are possible.The authors recommend that further work be performed in the area of real-time detection of unknown anomaly detection during UAV flights.Reliable, automatic detection of known objects of interest is valuable.Future work also includes improving the efficiency and effectiveness of the optimized view planning algorithm and extending it from multi-rotor and single-rotor aircraft to fixed wing platforms.While the simulated environment is useful for identifying general trends, field tests are also needed in the future to validate the simulated results and further explore the details of field implementation.As the simulations in this paper represent a physical location of interest, future field work planned by the authors at that site will be directly comparable to the presented results.

Figure 1 .
Figure 1.Simulation of the Highline Canal near Payson, Utah.

Figure 3 .
Figure 3. Illustration of single track (above) and double track (below) UAV flight paths.

Figure 4 .Figure 5 .
Figure 4. Dome projection inserted to account for unknown anomaly shape.(a) Terrain without dome projection; (b) terrain with dome projection.

Figure 6 .
Figure 6.Angle ranges for histogram, where α is the camera angle of view.

Figure 7 .
Figure 7. Example of an optimized camera view plan and flight path for a generic anomaly with and without the added dome projection.Yellow circles represent image locations; green triangles are corresponding image targets.(a) Optimized flight plan with dome projection; (b) optimized flight plan without dome projection; (c) top down view of (a); (d) top down view of (b).

Figure 8 .
Figure 8. Examples of a ground truth model and a reconstructed model for the piping anomaly location.(a) Ground truth model geometry exported from Terragen; (b) model reconstructed from images in Agisoft Photoscan.

Figure 9 .
Figure 9. Illustration of the quadratic fitting technique used to find the distance between the reference cloud (black) and the compared cloud (blue).

Figure 10 .
Figure 10.Flight path simulated along the Highline Canal section.

Figure 11 .
Figure 11.Top view of flight path simulated along the Highline Canal section.

Figure 12 .
Figure 12.Average accuracy of 3D models from test cases.

Figure 13 .
Figure 13.Qualitative comparison of canal models: (a) original geometry; (b) reconstruction from single path; (c) reconstruction from double path.Note the disjoint at the canal surface in the double path case.

Figure 14 .
Figure 14.Cross-section view of canal from a double path model.Again, the model is seen to be disconnected at the canal surface.

Figure 15 .
Figure 15.Flight time comparison for a 100-mile flight.

Figure 16 .
Figure 16.Data quantity comparison for a 100-mile flight.

Table 1 .
Camera settings for image generation.

Table 2 .
Standard path planner: single path results.

Table 3 .
Standard path planner: double path results.

Table 5 .
Optimized path planner results.

Table 6 .
Program settings for model reconstruction.