Next Article in Journal
Ultra-Short-Term Wind Power Forecasting Based on CGAN-CNN-LSTM Model Supported by Lidar
Previous Article in Journal
Microwave Devices for Wearable Sensors and IoT
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sensors
Volume 23
Issue 9
10.3390/s23094367
sensors-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Review of Artificial Intelligence Applications for Virtual Sensing of Underground Utilities

by
Kunle S. Oguntoye
1,
Simon Laflamme
1,*,
Roy Sturgill
1 and
David J. Eisenmann
2
1
Department of Civil, Construction, and Environmental Engineering, Iowa State University, Ames, IA 50011, USA
2
Department of Agricultural and Biosystem Engineering, Iowa State University, Ames, IA 50011, USA
*
Author to whom correspondence should be addressed.
Sensors 2023, 23(9), 4367; https://doi.org/10.3390/s23094367
Submission received: 22 March 2023 / Revised: 20 April 2023 / Accepted: 27 April 2023 / Published: 28 April 2023
(This article belongs to the Section Intelligent Sensors)
Browse Figures
Versions Notes

Abstract

:
Accurately identifying the location and depth of buried utility assets became a considerable challenge in the construction industry, for which accidental strikes can cause important economic losses and safety concerns. While the collection of as-built utility locations is becoming more accurate, there still exists an important need to be capable of accurately detecting buried utilities in order to eliminate risks associated with digging. Current practices typically involve the use of trained agents to survey and detect underground utilities at locations of interest, which is a costly and time-consuming process. With advances in artificial intelligence (AI), an opportunity arose in conducting virtual sensing of buried utilities by combining robotics (e.g., drones), knowledge, and logic. This paper reviewed methods that are based on AI in mapping underground infrastructure. In particular, the use of AI in aerial and terrestrial mapping of utility assets was reviewed, followed by a summary of AI techniques used in fusing multi-source data in creating underground infrastructure maps. Key observations from the consolidated literature were that (1) when leveraging computer vision methods, automatic mapping techniques vastly focus on manholes localized from aerial imagery; (2) when applied to non-intrusive sensing, AI methods vastly focus on empowering ground-penetrating radar (GPR)-produced data; and (3) data fusion techniques to produce utility maps should be extended to any utility assets/types. Based on these observations, a universal utility mapping model was proposed, one that could enable mapping of underground utilities using limited information available in the form of different sources of data and knowledge.

1. Introduction

Accidental utility strikes cost the United States approximately 30 billion dollars in damages in 2020 alone. These damages arise from direct (e.g., facility repairs), indirect (e.g., business closure, medical expenses), and societal (e.g., environmental pollution and traffic delays) costs. To promote safe excavation practices, the US utilizes the One-Call system [1], where excavators express intentions to dig and wait for concerned utility owners/operators to mark out the location of their facilities and indicate safe excavation areas. Yet, a recent study on the performance of the One-Call system in North Carolina highlighted challenges in the operation of the One-Call system to include prolonged wait times by operators, inaccurate locates, and low sensitization on excavation procedures among excavators [2]. The general inaccuracy and/or inefficiency of underground utility location practices is an important root cause of these accidental utility strikes [3].
A good share of investigation and locating practices employed are based on evidence of utilities, here termed evidence-based mapping, and often requires making inferences about the location of buried utility assets from the position of aboveground appurtenances. Evidence-based mapping also includes the use of any available information, such as as-planned and as-built maps. Nevertheless, while it is known that as-planned maps are often highly inaccurate, as-built maps may also be unreliable [4]. In particular, a recent survey reported that more than half of the utilities struck were missing from the available utility maps [5]. Another set of locating practices involves the use of nondestructive evaluation (NDE) techniques such as ground-penetrating radar (GPR) and acoustic emission (AE). However, these techniques can be costly and have equipment-specific challenges that limit mapping accuracy [6]. In particular, GPR is versatile in detecting metallic and non-metallic utilities, but performs poorly over highly conductive soils and the output data are complex to analyze; AE-based techniques performs well for non-metallic utilities, but is highly sensitive to noise [7].
Subsurface utility engineering (SUE) incorporates a classification approach for utility data quality levels to serve as a decision-support tool in the construction industry, ranging from quality-level A (high quality, low uncertainty) to quality-level D (lower quality, high uncertainty) [8]. Quality-level A data usually require sophisticated and more expensive mapping techniques [9]. Surveying visible (e.g., ground-level) appurtenances, akin to evidence-based mapping, is considered quality-level C data. On the other hand, mapping techniques that leverage NDE techniques typically output quality-level B data. These mapping practices are still labor intensive [10], thus contributing to prolonged mapping time, expenses, and inevitable human error, causing inconsistent data reliability.
A solution to improve mapping speed and accuracy is the automation of the mapping process through virtual sensing, where the location of utilities is determined by strategically combining limited information, including known utility positions and multi-source data. In the virtual sensing of underground utilities, the use of artificial intelligence (AI) tools becomes critical in creating inferences from available information in order to develop projected utility maps. The overarching goal of this paper is to review recent advances in the use of AI in automating the underground utility mapping process and recommend a pathway to implementing virtual sensing in future underground utility investigation approaches. Therefore, the main scientific contribution lies in the proposed universal underground utility mapping model developed to address the identified critical gaps in the consolidated literature.
The review of literature for this manuscript was conducted systematically by searching for all articles studying AI-aided mapping of underground utilities. The findings from this literature review were categorized within the areas of aerial mapping, terrestrial mapping, and data fusion of multiple data sources. Literature was also collected to support the proposed approach of using virtual sensing to investigate underground utilities. The categorized literature was then reviewed in deeper detail to prepare the narrative provided within this paper and the proposed method described herein.
This paper begins with a review of AI applications in underground utility mapping using single-source observations. As such, the next two sections review applications of aerial mapping, which includes the use of drones and satellite imagery, and applications of terrestrial mapping, which mostly concerns the use of NDE techniques. For clarity, the use of AI with photographic evidence is lumped in the aerial-based review. After, AI techniques leveraging multi-sourced data to create utility maps are reviewed. Finally, a path forward to conduct accurate virtual sensing of buried utilities is proposed, here termed universal utility mapping. The organization of the paper is illustrated in Figure 1.

2. Aerial Mapping

Aerial mapping (AM) of mapping underground utility typically consists of locating aboveground utility appurtenances to infer the likelihood of buried utility based on the utility flow of operations and industry rules. AM techniques are known to be cost-effective and non-intrusive [11]. AI in applications to AM is typically used as either (1) a computer vision tool to identify the assets or (2) a utility-mapping tool based on inference or logic rules to identify assets and predict the underground network. While it was argued by White et al. [12] that the concept of AM is more suitable for fluid-carrying utilities that depend on a gravity-flow rather than being pressurized, other works demonstrated the potential of AM to be successful for both fluid and non-fluid carrying utilities. For instance, Hassan et al. [13] generated an electrical utility map based on the location of transformers and poles using geographic information system (GIS) data.
One of the limitations of generating a utility map with the AM technique is its reliance on a knowledge database. While standards exist to harness such databases, such as those from the National Electric Safety Code and American Water Works Association, various uncertainties could lead to high variability in the accuracy of an automated process, including non-compliance [14] among utility operators and the presence of abandoned or out-of-service aboveground assets [15]. Additionally, the use of AM as an automated process was widely researched and demonstrated with manhole assets, but applications with other utility classes are still lacking.
A popular method to automate AM mapping of aboveground assets is through computer vision algorithms. This task consists of identifying key utility assets from pictures. Figure 2 illustrates an example through a street-level image, showing identifiable utility-related appurtenances and the position of underground utilities at that location.
Potential sources of image inputs for AM include satellite imagery, aerial photographs, mobile images, photogrammetry, and light detection and ranging (LiDAR) technology collection. In particular, the use of satellite and aerial imagery can yield improved object recognition performance due to their improvement in image resolution [16]. As an example, Shermeyer et al. [17] found that the improvement in satellite imagery resolution from 30 cm/pixel to 15 cm/pixel upgraded the mean average precision of their object detection algorithm from 13% to 36%. Related to the application of satellite and aerial imagery to mapping utilities, Mnih et al. [18] applied a neural network approach to learn and detect road patterns. The detection of road patterns can be critical in mapping utilities as these sets of infrastructure are typically co-located [19]. There are additional examples of utility mapping using imagery. Bartoli et al. [20] reconstructed urban drainage networks by detecting manhole covers using a circular detection filter on orthophotos and satellite images. Pasquet et al. [21] identified the presence of underground utilities by locating manhole covers and stormwater drains from high-resolution aerial images using a linear support vector machine (SVM) classifier and a histogram of oriented gradients (HOG) that performed similarly to the circular detection filter used in Bartoli et al. [20].
Other works investigated asset detection using cameras. In particular, Timofte et al. [22] used single and multi-view computer vision techniques combined with GPS data to detect manhole covers from pictures obtained from cameras mounted on vans. Santos et al. [23] identified storm drains and manholes from street-level color RGB images by integrating the deep-learning computer vision model. Moy de Vitry et al. [24] proposed a multi-view detection method to improve sewer-inlet location detection rate in images obtained from stitched and undistorted unmanned aerial vehicle (UAV) images. Al-Kaff et al. [25] reviewed literature incorporating computer vision algorithms in UAVs for autonomous inspection, surveillance, and mapping, and Liu et al. [26] proposed a deep convolution neural network for detecting small manhole covers also captured by UAV.
Utility asset detection from point cloud data was also conducted, for instance, through the use of LiDAR. For example, Yu et al. [27] proposed a framework that automatically detects manhole covers from mobile laser scanning data; Turkan et al. [28] proposed the use of a wavelet neural network to compress point cloud data and store asset inspection features; Wei et al. [29] experimented with improved HOG symmetry features combined with SVM to detect and identify manhole covers; Gouda et al. [30] developed a fully automated process for identifying electric light poles from LiDAR point cloud data.
Other works explored automatic mapping following asset identification. Early works by Osman et al. [31] established a semi-automated system for decision-making in routing new utilities to avoid clashes with other existing utilities, constraints from surrounding elements, and other physical constraints. They proposed a three-branch framework; ontology knowledge formalization, communication and knowledge sharing with GIS, and reasoners using AI-fuzzy logic to assist in making an informed decision in selecting the best utility route. Li et al. [14] proposed a framework to automate utility location verificaton checks, a crucial step in minimizing incidents such as the pipeline explosions. Textual data are extracted and formulated into a hierarchical order using the specification language model and a natural language processing (NLP) model. Spatial reasoning outputs computer-readable logic that automatically creates a GIS model used in evaluating the compliance of as-built utilities with regulatory rules [14]. Later works by Xu et al. [32] and Xu et al. [33] showed advances in using ontology and spatial reasoning in executing automated utility compliance checks. Chahinian et al. [34] mapped wastewater networks from manhole locations combined with logic rules. The utility map was constructed from the minimization of a cost function defined by the logic rules that included variables such as manhole edge proximity, the slope between manholes, and interactions between roads and buildings.
Overall, the use of AI in conjunction with AM processes includes a vast array of methods to conduct both asset identification and buried utility location. Yet, from the review, the vast majority of works focused on the detection of manholes and their use in utility mapping. This disproportionate attention precluded the applications of AM and AI techniques to other utilities, such as electricity, natural gas, and telecommunication. There is also a lack of frameworks that combine both the tasks of asset identification and utility location. To guide work in asset location and mapping, Table 1 lists several assets associated with utility types and their typical locations [35].
It is important to note from Table 1 that some assets belong to more than one utility class. While there exist physical methods to reduce uncertainties, such as investigating the type of utility by opening manholes, as carried out in Dou et al. [4] and Chen and Cohn [36], or accessing other visual information, such as identification icons or inscriptions, automated routing algorithms could help understand and cope with these uncertainties. This can be carried out, for example, through fuzzy logic [37]. Other commonly shared assets include valve covers, pedestals, and control boxes.

3. Terrestrial Mapping

Terrestrial mapping (TM), not including photography, is typically conducted to achieve higher levels of mapping accuracy than those attainable via AM techniques. These can include the use of geophysical investigations or intrusive measures to achieve SUE quality-level A mapping accuracy through techniques such as vacuum excavation [38]. This method is widely used in today’s utility locating practices [39,40]. However, intrusive exposures for mapping can be costly as it can potentially disrupt critical activities such as construction and traffic [41], especially where utilities are within transportation right of way. The invasive technique also requires close attention to details during excavation to prevent damage to buried assets. A solution is to adopt non-intrusive sensing systems and geophysical tools such as ground-penetrating radar (GPR), metal detectors, vibro-acoustic systems, radio-frequency locators, and geoelectrical-resistive sensing methods [42]. These sensing methods output SUE quality-level B data [8], which has the potential to be further refined using data fusion techniques [43,44].
However, non-intrusive sensing requires highly trained operators to assess and evaluate information from scanned data [45]. Data processing can also be challenging, even for experts [46]. AI techniques provide a promising solution for assisting the non-intrusive process by automatically and rapidly evaluating sensing data to support actionable decisions. This section of the paper reviews various AI algorithms used in automating the utility detection process from non-intrusive data collection methods. It should be noted that a significant number of works focused on using AI to assist GPR systems due to their popularity over other sensing devices and the complexity in GPR data interpretation.
GPR-generated data can be complex to analyze. Figure 3 shows a typical generated scan ouptut with a gray background and different textures of hyperbolic signatures [47] bounded by red boxes. Typically, the hyperbolic signatures form by joining the discrete travel time of the electromagnetic waves as they pulse back from the targets [48]. Information such as the object’s depth and horizontal position is estimated using the hyperbolic signature’s spread, position in radargram, and degree of resolution. Yet, the manual reading of radargrams is often challenging due to the intrinsic differences in material properties of targets and soil characteristics. Several AI tools were developed and applied to assist the interpretation of GRP-collected data, in particular for applications regarding underground utilities [49,50,51,52].
One popular tool is the Hough transform (HT), used to extract geometrical features in images [53]. It was used by Capineri et al. [54] in locating the position of buried objects represented by hyperbola in GPR radargrams. The HT was also used in Al-Nuaimy et al. [55], where the HT was combined with a backpropagating neural network to detect buried utilities and other solid objects. Al-Nuaimy et al. [56] proposed an adaptive non-accumulated Hough transform (ANHT) to reduce the memory demand relative to the conventional HT and demonstrated that the technique could be applied to detect linear structures such as pipes and cables. Modified HT variants were proposed. For instance, Simi et al. [57] developed the randomized HT that yielded underground utility detection accuracy greater than 85% over different ground conditions such as asphalt, grass, and concrete; Al-Nuaimy et al. [55] used a modified HT to detect targets in a GPR radargram with a significantly low signal-to-noise ratio; and Maas et al. [58] combined the Viola–Jones learning algorithm with the HT to separate regions of interest and extract vertices and locations of the hyperbola. Other works adopted the HT technique for recognizing, localizing, and evaluating hyperbolic indications, wave velocity, buried pipe diameters, and asset depths in radargram [59,60,61,62].
It was argued in Chen and Cohn [62] that some early methods enabling automatic detection in GPR hyperbolic signatures were not suitable for on-site applications due to their computation complexity and unsatisfactory detection rate resulting from insufficient training. The authors proposed a probabilistic hyperbolae mixture model based on a classification expectation-maximization algorithm and a Bayesian information criterion for model selection. Compared to other HT, their proposed model completed the utility identification task in 1.7 s, while the HT completed a similar task in 226.1 s. In Dou et al. [63], a column-connect clustering algorithm was developed, which first discriminates targeted utilities represented as hyperbolas from the radargram background, with the algorithm yielding a 23% improvement in detection rate than for the pattern recognition technique used for a similar task in [58]. A variant of the column-connect clustering algorithm was investigated by Rosin, P. [64] that resulted in a 6% improvement in the model accuracy (F1-score).
Support vector machine (SVM) gained popularity in GPR data classification tasks. Zhang et al. [65] used SVM for the automatic extraction and classification of buried landmines from a GPR radargram; El-Mahallawy et al. [66] adopted an SVM classifier to identify underground utilities and showed 100% and 78% recognition accuracies in an ideal and noise-degraded environment, respectively; Lu et al. [67] proposed to combine a discrete wavelet transform, fractional Fourier transform, and SVM for classifying subsurface objects material from GPR data; Terrasse et al. [68] detected buried gas pipes using an SVM classifier and reported a 96% detection rate; Kaur et al. [69] combined HOG with SVM to map deteriorated reinforcement bars; Noreen et al. [70] automated buried pipe detections using HOG features clustered with an SVM algorithm; and Ozkaya et al. [71] fed features extracted from GPR radargram images using convolutional neural networks into an SVM algorithm to detect and classify hyperbolic signatures, material, shape and soil type.
Genetic programming (GP) was employed in automating GPR data processing. Pasolli et al. [72] proposed an automatic detection process that uses an iterative genetic algorithm to detect and locate the position of underground utilities in GPR radargrams. Kobashigawa et al. [73] adopted GP to detect buried unexploded ordnances in GPR radargrams. The GP algorithm performance was compared against different neural networks of varying hidden layers and showed improved performance over complex radargrams. Additionally, Harkat et al. [74] proposed to utilize a radial basis network with a multi-objective genetic algorithm to improve region classifications.
There is also a vast amount of research that focused on the use of neural networks in processing GPR radargrams. For example, Birkenfeld, S. [75] presented a non-fully connected neural network model that identifies interfering or incomplete hyperbolas. Lei et al. [46] proposed the faster region-based convolution neural network (R-CNN) and data augmentation to detect regions of buried objects in GPR images. Liu et al. [76] proposed a deep learning model, termed a single shot multi-box detector, to detect hyperbolae. When compared with the faster R-CNN model adopted by Lei et al. [46], the technique showed a higher precision rate with lower computational time. Yamaguchi et al. [77] conducted the 3-D location of buried pipes in GPR radargrams using 3D-CNN. A 10% improvement in accuracy level was observed when compared to an SVM. The extracted positions, inclinations, and arrangement of pipes were visualized in a 3-D map using Kirchhoff migration. Liu et al. [78] identified limitations in using hyperbolic signatures as the primary representative feature for buried utilities. The 3D GPR scan images formed by stitching parallel scan data were used as input to a 3D CNN. The proposed method produced a detection accuracy of 82.7%, compared with 69% for CNN with 2D GPR input data and 61% to 73.5% for other methods that included the AlexNet, ResNet50, VGG16 and 19, and Triplanar algorithms.
While more limited, other works focused on non-intrusive techniques other than GPRs to detect buried utilities, such as metal detectors [79], vibro-acoustic sensing techniques [80,81], geoelectrical resistivity, and radio-frequency locators. For instance, Dutta et al. [81] automatically located the presence of underground utilities by using a novel pattern recognition and edge detection algorithm based on Canny filtering to analyze and identify hidden patterns from contour plots and sub-surface cross-sectional images in vibro-acoustic sensing data; Safatly et al. [79] reduced false-alarm rates during manual landmine investigation with metal detectors by developing a machine-learning model trained to discriminate between sounds produced when a landmine is present and not; and Vanijjirattikhan et al. [80] proposed a classification model trained on leakage sounds to improve leakage detection with acoustic techniques.
In summary, most of the reviewed literature in this section focused more on GPR because it remains the most used electromagnetic sensor for utility mapping. Other sensors demand AI applications to further improve data interpretation, such as vibro-acoustic and metal detectors that are used for non-metallic utility detection and rapid utility probing, respectively.

4. Data/Information Fusion

The previous two sections focused on the use of AI to interpret single-source data. In creating virtual sensors capable of mapping underground utilities based on limited information, one must consider the integration of multi-source data in order to improve the quality of the harnessed information and, thus, the accuracy of maps through a process known as data fusion [82,83,84,85]. For example, Dutta et al. [81] proposed to fuse GPR and vibro-acoustic data to combine sensing performance in reading through non-saturated soils (GPR) and saturated soils (vibro-acoustic), and Metje et al. [11] studied the potential of combining GPR, low-frequency quasi-static electromagnetic fields, and acoustic technologies to improve underground utility mapping accuracy. Generally, while the fusion of data is conducted through user-defined rules and metrics, the iterative process is automated for robustness over extensive knowledge and data space [86]. This section surveys different AI techniques used in fusing data in order to produce maps of underground utilities.
Fuzzy logic was used by Lanka et al. [37] as an approximate reasoning method to fuse data from different sources. The proposed model determines the probability of a utility to be located at any Cartesian point based on (1) proximity to manholes; (2) perpendicular distance from the point of interest to the nearest known utility line; (3) expert knowledge on a possible underground utility route; and (4) GPR data analysis. Another technique studied by Chen and Cohn [36] is based on Bayesian data fusion (BDF) to automatically produce refined utility maps by fusing GPR data, surveyed manholes, assumptions on pipe linearity, and utility records. In this research, manholes are investigated to determine the depths and orientations of the underground pipes. This study adopted the joint compatibility branch and bound algorithm developed by Neira and Tardos [87] to establish an initial map by matching utilities discovered from the manhole survey and GPR scan. Prior knowledge used in the BDF is deduced by matching connections in the as-built maps and GPR-detected points. With the Gaussian distribution assumed, the posterior probability of the BDF is estimated from prior and likelihood probabilities. The posterior map is automatically refined through an iterative process guided by a validation threshold.
Recent work by Zhou et al. [88] proposed a probabilistic pipeline mapping model that fuses records, pipe information from manhole covers, and remote sensing technologies such as GPR and electro-magnetic. The model maximizes the likelihood of fitting a pipeline from the set of detected points. By integrating the Bayesian information criterion, the proposed model avoids overfitting when classifying and fitting detected points into the most probable pipeline. The authors compared the performance of their algorithm with that proposed in Chen and Cohn [36] and demonstrated fewer mapping errors.
Additionally, Bilal et al. [89] mapped underground utilities with different probabilistic models. Figure 4 shows the sequence of workflow starting with maps, surveyed manholes, and sensor readings obtained from the mapping the underworld (MTU) initiative [11]. MTU is a United Kingdom initiative targeting the construction of highly accurate underground utility maps leveraging sensing data, such as GPR and vibroacoustic measurements, without excavation. The authors select candidate hypotheses from MTU sensor readings using a maximum likelihood probabilistic approach to recognize utility segments. Segment connections are completed using a Bayesian approach, and finally, the underground utility map is prepared for visualization.
Dou et al. [4] generated utility maps from the combination of information collected from manhole investigations and GPR data. The manhole investigation allows the utility to be characterized by their directional angle, depth, and material type. The probability of utilities routing along the observed directions is estimated with a Gaussian distribution. The extracted utility location hypotheses from GPR data reduces the degree of uncertainty of the different directions the utility can take beyond the observed range. The most probable connection between two utility ends (i.e., manholes) is derived by maximizing the product of different probabilities of utilities exiting both ends using integer linear programming (ILP). This model’s experimental result yielded a 100% accuracy (F1-score) on field data and an average of approximately 80% on synthetic data. The model’s dependency on investigating manholes as the starting or ending point renders it insufficient for broader applications.
Cai et al. [7,90] identified the limitations of the data-fusion techniques discussed in [36,81]. These limitations included overreliance on data sources, assumed distributions, lack of prior information, and difficulty in estimating prior probabilities. The authors further proposed a technique that fuses data from both sensor and non-sensor sources based on Dempster–Shafer’s theory. With GPR and as-built records as selected data sources, understudying the limitations and other uncertainties reveals the confidence level of every connection in each data source. Each connection is characterized using mass values assigned to the three keywords—existence, absence, and ignorance. The joint mass values from both data sources, thus, reveal every matched connection certainty and uncertainty level. While the models produced satisfying results when tested with an indoor experimental setup and simulated records, the assumption of pipe perpendicularity with the GPR survey pathway survey line is not always true for real-world data.
Dutta et al. [81] used dynamic Bayesian network to estimate the most probable 3D map of buried utilities using data from ground-excited vibroacoustic emissions, pipe-excited vibroacoustic emissions, and GPR systems. The authors adopted intelligent image-processing techniques to process individual sensor image data before fusing the extracted utility location and depth with the dynamic Bayesian network. The resulting model-generated map provided higher accuracy compared with utility records from Moravec, H.P. [83].
In Dou et al. [44], GPR, passive magnetic field, vibroacoustic, and magnetic gradiometer data were fused to improve a utility map’s accuracy. Data fusion was conducted using a novel marching cross-section algorithm and Kalman filter in marching scanned utility tracks. The bar plot in Figure 5 shows the progression and regression in detection rate and error, respectively, as the number of sensing modes increases.
The reviewed literatures combined different data sources, mostly GPR, utility records, manhole survey, and other sensors. One underlying common limitation to most work is the reliance on manhole surveys. Firstly, it restricts the applicability to manhole-related utilities, and secondly, it is burdensome to open up and determine the geospatial coordinates of every manhole in the area of interest at the same time. It should also be noted that techniques relying on as-built maps as prior knowledge can be challenging to apply when such information is unavailable or outdated.

5. Enabling Virtual Sensing of Underground Utilities

In the previous sections, we reviewed different applications of AI representing efforts to automatically map underground utilities, including those that employed sensing these utilities through the fusion of multi-source data. From the reviewed works, it is clear that the field made great progress at (1) detecting manhole locations and reconstructing utility maps using available knowledge from what manholes represent, whether it included physical investigation or not; (2) transforming complex GPR measurements into hypotheses on the location of utilities; and (3) fusing information obtained from manhole locations, GPR data, available as-built maps, and metadata such as those obtained through available records. Yet, it follows that important gaps exist in (1) mapping utilities from other assets, such as transformers and valves; (2) extending AI tools to extract useful information from other non-intrusive evaluation techniques; and (3) extending data fusion techniques to any utility assets/types (e.g., those listed in Table 1) and an array of different data sources.
Building on the surveyed literature, we see an opportunity in producing an integrated, universal utility mapping model that could enable mapping of underground utilities using limited information available in the form of different sources of data and knowledge. The proposed model is depicted in Figure 6. The process consists of, first, constructing an initial utility map based on a site survey. This initial map may include probabilistic information to improve the quality of initial guesses on utility locations. Afterwards, the map is refined as needed using non-intrusive evaluation techniques, such as GPR and vibroacoustic measurements. Importantly, a knowledge database would support the mapping process, which constitutes the central mechanism enabling virtual sensing. We foresee such a database to be open access and open sourced, with various stakeholders capable of modifying the inputs in order to improve the quality of the harnessed information and, thus, virtual sensing accuracy. The knowledge database is to include the following critical information.
Records and as-built maps: the readily available, recorded information on the location of utilities. These may include as-planned and as-built maps. Some of this information may be considered as “truth,” or at least assessed with a high probability of accuracy. As-built maps can be considered as a starting point in mapping utilities or used in guiding the non-intrusive process by comparing these maps with the initial reconstructed utility map [7,36].
Asset types and rules: a dictionary of possible asset types and asset-specific rules that can be used in mapping. These also include libraries of pictures that can be accessed for training computer vision algorithms. Rules associated with specific asset types, such as the depth and expected direction of utilities [35], will be used to create initial utility maps. These rules include fuzzy logic to determine the probability of utility routing.
Geographical and topographical information: a set of site-specific data relative to the geographical location and ground topography. This information includes local law pertaining to utility routing. Knowledge of the topography can be useful in guiding GPR inspections and establishing rules for gravitational systems (Chahinian et al. [34], for instance).
Expert knowledge: information input by field experts; for example, utility inspectors and managers. For example, the typical burial depth or possible deviations from assets, or known approximate placement and routing as a function of a given neighborhood based on prior investigations.
Semantic models: a set of models empowering the fusion of semantic data to improve the accuracy of the mapping process. These may include, for instance, the extraction and interpretation of utility burial rules/laws (Xu and Cai [32,33], for instance), and/or the direct interpretation of expert knowledge input.
The knowledge database cross-interacts both at the initial mapping level and non-intrusive mapping level through various AI rules, such as those reviewed in the previous sections. AI can be seen as a tool to extract useful information from the knowledge database and assist at creating utility maps. Its role spans from the computer vision algorithms used in detecting assets to data-fusion algorithms used in creating more accurate routing in the initial map. Once the refined map is created from the non-intrusive investigation campaign, new knowledge is created and fed back to the knowledge database. Ideally, this new information is used to update rules and/or available dictionaries of information.

6. Conclusions

This paper reviewed different techniques supporting virtual sensing of underground utilities leveraging advances in artificial intelligence (AI). In particular, the use of AI in aerial and terrestrial mapping of utility assets through visible appurtenances was reviewed, followed by a summary of AI techniques used in fusing multi-source data in creating underground infrastructure maps. Key findings from these sections identified the research gap in achieving accurate virtual sensing of utility assets. The biased attention of the majority of works to a single utility class or popular utility sensor, and limited data sources for the fusion process hindered the generalized application and high accuracy. Based on these findings, a universal utility mapping model was proposed. The model uses limited information available in the form of different sources of data and knowledge. It consists of first constructing an initial utility map based on a site survey, after which the map is refined as needed using non-intrusive evaluation techniques. Importantly, a knowledge database is used to support the mapping process, which constitutes the central mechanism enabling virtual sensing. Future work in the area could include the creation of an open-source knowledge database that could be populated, augmented, and altered by experts in order to refine the performance of AI systems at conducting virtual sensing of underground utilities.

Funding

This research was funded by the Iowa Energy Center under grant number #21-IEC-006.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data sharing not applicable.

Acknowledgments

The authors acknowledge the financial support of the Iowa Energy Center under Grant #21-IEC-006.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Chen, Q.; Davis, K.; Parker, C. Modeling Damage Prevention Effectiveness Based on Industry Practices and Regulatory Framework; American Society of Mechanical Engineers Digital Collection: New York, NY, USA, 2008; pp. 635–645. [Google Scholar]
  2. Al-Bayati, A.J.; Panzer, L.; Karakhan, A. Reducing Damages to Underground Infrastructure: Performance Evaluation of One-Call Notification Program. Pract. Period. Struct. Des. Constr. 2019, 24, 04019018. [Google Scholar] [CrossRef]
  3. Common Ground Alliance (CGA) Damage Information Reporting Tool 2020 Analysis and Recommendations. Available online: https://commongroundalliance.com/Publications-Media/DIRT-Report (accessed on 21 December 2022).
  4. Dou, Q.; Lin, Z.; Magee, D.R.; Cohn, A.G. 3D Mapping from Partial Observations: An Application to Utility Mapping. Autom. Constr. 2020, 117, 103229. [Google Scholar] [CrossRef]
  5. Metje, N.; Ahmad, B.; Crossland, S.M. Causes, Impacts and Costs of Strikes on Buried Utility Assets. Proc. Inst. Civ. Eng.-Munic. Eng. 2015, 168, 165–174. [Google Scholar] [CrossRef] [Green Version]
  6. Costello, S.B.; Chapman, D.N.; Rogers, C.D.F.; Metje, N. Underground Asset Location and Condition Assessment Technologies. Tunn. Undergr. Space Technol. 2007, 22, 524–542. [Google Scholar] [CrossRef]
  7. Cai, J.; Jeon, J.; Cai, H.; Li, S. Fusing Heterogeneous Information for Underground Utility Map Generation Based on Dempster-Shafer Theory. J. Comput. Civ. Eng. 2020, 34, 04020013. [Google Scholar] [CrossRef]
  8. ASCE. Standard Guidelines for the Collection and Depiction of Existing Subsurface Utility Data, 38th ed.; American Society of Civil Engineers: Reston, VA, USA, 2002; ISBN 978-0-7844-0645-8. [Google Scholar]
  9. Jung, Y.J. Utility Impact Rating with Subsurface Utility Engineering in Project Development. Can. J. Civ. Eng. 2009, 36, 1744–1754. [Google Scholar] [CrossRef]
  10. Huston, D.; Xia, T.; Zhang, Y.; Fan, T.; Orfeo, D.; Razinger, J. Urban Underground Infrastructure Mapping and Assessment. In Proceedings of the Sensors and Smart Structures Technologies for Civil, Mechanical, and Aerospace Systems 2017, Bellingham, WA, USA, 12 April 2017; Volume 10168, pp. 121–131. [Google Scholar]
  11. Metje, N.; Atkins, P.R.; Brennan, M.J.; Chapman, D.N.; Lim, H.M.; Machell, J.; Muggleton, J.M.; Pennock, S.; Ratcliffe, J.; Redfern, M.; et al. Mapping the Underworld—State-of-the-Art Review. Tunn. Undergr. Space Technol. 2007, 22, 568–586. [Google Scholar] [CrossRef]
  12. White, C.G.; Nielsen, M.G.; Hayward, R. Lessons Learned Mapping Critical Pressure Pipelines: City of Ottawa Case Studies. Pipelines 2017, 78–88. [Google Scholar] [CrossRef]
  13. Hassan, H.T.; Akhtar, M.F. Mapping of Power Distribution Network Using Geographical Information System (GIS). Int. J. Emerg. Technol. Adv. Eng. 2012, 2, 343–347. [Google Scholar]
  14. Li, S.; Cai, H.; Kamat, V.R. Integrating Natural Language Processing and Spatial Reasoning for Utility Compliance Checking. J. Constr. Eng. Manag. 2016, 142, 04016074. [Google Scholar] [CrossRef]
  15. Anspach, J.H.; Robison, A.G.; Clark, S.T.; Tomanek, M.J. The Need for Utility Data Governance Policies for Improving Lifeline Recovery. Lifelines 2022, 749–755. [Google Scholar] [CrossRef]
  16. Neupane, B.; Horanont, T.; Aryal, J. Deep Learning-Based Semantic Segmentation of Urban Features in Satellite Images: A Review and Meta-Analysis. Remote Sens. 2021, 13, 808. [Google Scholar] [CrossRef]
  17. Shermeyer, J.; Van Etten, A. The Effects of Super-Resolution on Object Detection Performance in Satellite Imagery. arXiv 2019, arXiv:1812.04098. [Google Scholar]
  18. Mnih, V.; Hinton, G.E. Learning to Detect Roads in High-Resolution Aerial Images. In Computer Vision, Proceedings of the ECCV 2010: 11th European Conference on Computer Vision, Heraklion, Crete, Greece, 5–11 September 2010; Daniilidis, K., Maragos, P., Paragios, N., Eds.; Springer: Berlin/Heidelberg, Germany, 2010; pp. 210–223. [Google Scholar]
  19. Analysing Utility Tunnels and Highway Networks Coordination Dilemma—ScienceDirect. Available online: https://www.sciencedirect.com/science/article/pii/S0886779808000771 (accessed on 21 December 2022).
  20. Bartoli, O.; Chahinian, N.; Allard, A.; Bailly, J.-S.; Chancibault, K.; Rodriguez, F.; Salles, C.; Tournoud, M.-G.; Delenne, C. Manhole Cover Detection Using a Geometrical Filter on Very High Resolution Aerial and Satellite Images. In Proceedings of the 2015 Joint Urban Remote Sensing Event (JURSE), Lausanne, Switzerland, 30 March–1 April 2015; pp. 1–4. [Google Scholar]
  21. Pasquet, J.; Desert, T.; Bartoli, O.; Chaumont, M.; Delenne, C.; Subsol, G.; Derras, M.; Chahinian, N. Detection of Manhole Covers in High-Resolution Aerial Images of Urban Areas by Combining Two Methods. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2016, 9, 1802–1807. [Google Scholar] [CrossRef] [Green Version]
  22. Timofte, R.; Van Gool, L. Multi-View Manhole Detection, Recognition, and 3D Localisation. In Proceedings of the 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), Online, 6–13 November 2011; pp. 188–195. [Google Scholar]
  23. Santos, A.; Marcato Junior, J.; de Andrade Silva, J.; Pereira, R.; Matos, D.; Menezes, G.; Higa, L.; Eltner, A.; Ramos, A.P.; Osco, L.; et al. Storm-Drain and Manhole Detection Using the RetinaNet Method. Sensors 2020, 20, 4450. [Google Scholar] [CrossRef] [PubMed]
  24. Moy de Vitry, M.; Schindler, K.; Rieckermann, J.; Leitão, J.P. Sewer Inlet Localization in UAV Image Clouds: Improving Performance with Multiview Detection. Remote Sens. 2018, 10, 706. [Google Scholar] [CrossRef] [Green Version]
  25. Al-Kaff, A.; Martín, D.; García, F.; de la Escalera, A.; María Armingol, J. Survey of Computer Vision Algorithms and Applications for Unmanned Aerial Vehicles. Expert Syst. Appl. 2018, 92, 447–463. [Google Scholar] [CrossRef]
  26. Liu, W.; Cheng, D.; Yin, P.; Yang, M.; Li, E.; Xie, M.; Zhang, L. Small Manhole Cover Detection in Remote Sensing Imagery with Deep Convolutional Neural Networks. ISPRS Int. J. Geo-Inf. 2019, 8, 49. [Google Scholar] [CrossRef] [Green Version]
  27. Yu, Y.; Guan, H.; Ji, Z. Automated Detection of Urban Road Manhole Covers Using Mobile Laser Scanning Data. IEEE Trans. Intell. Transp. Syst. 2015, 16, 3258–3269. [Google Scholar] [CrossRef]
  28. Turkan, Y.; Hong, J.; Laflamme, S.; Puri, N. Adaptive Wavelet Neural Network for Terrestrial Laser Scanner-Based Crack Detection. Autom. Constr. 2018, 94, 191–202. [Google Scholar] [CrossRef] [Green Version]
  29. Wei, Z.; Yang, M.; Wang, L.; Ma, H.; Chen, X.; Zhong, R. Customized Mobile LiDAR System for Manhole Cover Detection and Identification. Sensors 2019, 19, 2422. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  30. Gouda, M.; Shalkamy, A.; Li, X.; El-Basyouny, K. Fully Automated Algorithm for Light Pole Detection and Mapping in Rural Highway Environment Using Mobile Light Detection and Ranging Point Clouds. Transp. Res. Rec. 2022, 2676, 03611981221082531. [Google Scholar] [CrossRef]
  31. Osman, H.M.; El-Diraby, T.E. Knowledge-Enabled Decision Support System for Routing Urban Utilities. J. Constr. Eng. Manag. 2011, 137, 198–213. [Google Scholar] [CrossRef]
  32. Xu, X.; Cai, H. Semantic Approach to Compliance Checking of Underground Utilities. Autom. Constr. 2020, 109, 103006. [Google Scholar] [CrossRef]
  33. Xu, X.; Cai, H. Ontology and Rule-Based Natural Language Processing Approach for Interpreting Textual Regulations on Underground Utility Infrastructure. Adv. Eng. Inform. 2021, 48, 101288. [Google Scholar] [CrossRef]
  34. Chahinian, N.; Delenne, C.; Commandré, B.; Derras, M.; Deruelle, L.; Bailly, J.-S. Automatic Mapping of Urban Wastewater Networks Based on Manhole Cover Locations. Comput. Environ. Urban Syst. 2019, 78, 101370. [Google Scholar] [CrossRef]
  35. Highway/Utility Guide. Available online: https://www.fhwa.dot.gov/utilities/010604.pdf (accessed on 20 December 2022).
  36. Chen, H.; Cohn, A.G. Buried Utility Pipeline Mapping Based on Multiple Spatial Data Sources: A Bayesian Data Fusion Approach. In Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, Barcelona, Catalonia, Spain, 16–22 July 2011. [Google Scholar]
  37. Lanka, M.; Butler, A.; Sterling, R. Use of Approximate Reasoning Techniques for Locating Underground Utilities. Tunn. Undergr. Space Technol. 2001, 16, 13–31. [Google Scholar] [CrossRef]
  38. Sciulli, A.E. Surveys of Subsurface Utilities Can Reduce Risk in Plant Decommissioning. Nat. Gas Electr. 2015, 31, 9–14. [Google Scholar] [CrossRef]
  39. Uslu, B.; Jung, Y.J.; Sinha, S.K. Underground Utility Locating Technologies for Condition Assessment and Renewal Engineering of Water Pipeline Infrastructure Systems. J. Pipeline Syst. Eng. Pract. 2016, 7, 04016011. [Google Scholar] [CrossRef]
  40. Jeong, H.; Arboleda, C.; Abraham, D. Imaging and Locating Buried Utilities; JTRP Technical Reports; Joint Transportation Research Program, Indiana Department of Transportation and Purdue University: West Lafayette, IN, USA, 2003; pp. 57–96. [Google Scholar]
  41. Najafi, D.M.; Gokhale, D.S.; Calderón, D.R.; Ma, D.B. Overview and Comparison of Trenchless Technologies; McGraw-Hill Education: New York, NY, USA, 2021; ISBN 978-1-260-45873-2. [Google Scholar]
  42. Royal, A.C.D.; Atkins, P.R.; Brennan, M.J.; Chapman, D.N.; Chen, H.; Cohn, A.G.; Foo, K.Y.; Goddard, K.F.; Hayes, R.; Hao, T.; et al. Site Assessment of Multiple-Sensor Approaches for Buried Utility Detection. Int. J. Geophys. 2011, 2011, e496123. [Google Scholar] [CrossRef]
  43. Metwaly, M. Application of GPR Technique for Subsurface Utility Mapping: A Case Study from Urban Area of Holy Mecca, Saudi Arabia. Measurement 2015, 60, 139–145. [Google Scholar] [CrossRef]
  44. Dou, Q.; Wei, L.; Magee, D.R.; Atkins, P.R.; Chapman, D.N.; Curioni, G.; Goddard, K.F.; Hayati, F.; Jenks, H.; Metje, N.; et al. 3D Buried Utility Location Using A Marching-Cross-Section Algorithm for Multi-Sensor Data Fusion. Sensors 2016, 16, 1827. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  45. Ambrosanio, M.; Franceschini, S.; Autorino, M.M.; Pascazio, V. Convolutional Neural Networks for Tomographic MIMO Ground Penetrating Radar Imaging. In Proceedings of the 2021 7th Asia-Pacific Conference on Synthetic Aperture Radar (APSAR), Online, 1–3 November 2021; pp. 1–5. [Google Scholar]
  46. Lei, W.; Hou, F.; Xi, J.; Tan, Q.; Xu, M.; Jiang, X.; Liu, G.; Gu, Q. Automatic Hyperbola Detection and Fitting in GPR B-Scan Image. Autom. Constr. 2019, 106, 102839. [Google Scholar] [CrossRef]
  47. Manataki, M.; Vafidis, A.; Sarris, A. GPR Data Interpretation Approaches in Archaeological Prospection. Appl. Sci. 2021, 11, 7531. [Google Scholar] [CrossRef]
  48. Lee, K.L.; Mokji, M.M. Automatic Target Detection in GPR Images Using Histogram of Oriented Gradients (HOG). In Proceedings of the 2014 2nd International Conference on Electronic Design (ICED), Online, 19–21 August 2014; pp. 181–186. [Google Scholar]
  49. Slob, E.C.; Groenenboom, J.; Fokkema, J.T. Automated Acquisition and Processing of 3D GPR Data for Object Detection and Characterization. Subsurf. Sens. Technol. Appl. 2003, 4, 5–18. [Google Scholar] [CrossRef]
  50. Mazzucchelli, P.; Molteni, D.; di Buono, N.; Cottino, E. 3D GPR Real? Time Automated Detection of Buried Utilities. In Symposium on the Application of Geophysics to Engineering and Environmental Problems Proceedings 2012; Environment and Engineering Geophysical Society: Denver, CO, USA, 2012; pp. 309–315. [Google Scholar]
  51. Li, H.; Li, N.; Wu, R.; Wang, H.; Gui, Z.; Song, D. GPR-RCNN: An Algorithm of Subsurface Defect Detection for Airport Runway Based on GPR. IEEE Robot. Autom. Lett. 2021, 6, 3001–3008. [Google Scholar] [CrossRef]
  52. Singh, N.P.; Nene, M.J. Buried Object Detection and Analysis of GPR Images: Using Neural Network and Curve Fitting. In Proceedings of the 2013 Annual International Conference on Emerging Research Areas and 2013 International Conference on Microelectronics, Communications and Renewable Energy, Online, 4–6 June 2013; pp. 1–6. [Google Scholar]
  53. Illingworth, J.; Kittler, J. A Survey of the Hough Transform. Comput. Vis. Graph. Image Process. 1988, 44, 87–116. [Google Scholar] [CrossRef]
  54. Capineri, L.; Grande, P.; Temple, J.a.G. Advanced Image-Processing Technique for Real-Time Interpretation of Ground-Penetrating Radar Images. Int. J. Imaging Syst. Technol. 1998, 9, 51–59. [Google Scholar] [CrossRef]
  55. Al-Nuaimy, W.; Huang, Y.; Nakhkash, M.; Fang, M.T.C.; Nguyen, V.T.; Eriksen, A. Automatic Detection of Buried Utilities and Solid Objects with GPR Using Neural Networks and Pattern Recognition. J. Appl. Geophys. 2000, 43, 157–165. [Google Scholar] [CrossRef]
  56. Al-Nuaimy, W.; Lu, H.H.; Shihab, S.; Eriksen, A. Automatic Mapping of Linear Structures in 3-Dimensional Space from Ground-Penetrating Radar Data. In Proceedings of the IEEE/ISPRS Joint Workshop on Remote Sensing and Data Fusion over Urban Areas (Cat. No.01EX482), Rome, Italy, 8–9 November 2001; pp. 198–201. [Google Scholar]
  57. Simi, A.; Bracciali, S.; Manacorda, G. Hough Transform Based Automatic Pipe Detection for Array GPR: Algorithm Development and on-Site Tests. In Proceedings of the 2008 IEEE Radar Conference, Rome, Italy, 26–30 May 2008; pp. 1–6. [Google Scholar]
  58. Maas, C.; Schmalzl, J. Using Pattern Recognition to Automatically Localize Reflection Hyperbolas in Data from Ground Penetrating Radar. Comput. Geosci. 2013, 58, 116–125. [Google Scholar] [CrossRef]
  59. Chiang, P.; Tantum, S.L.; Collins, L.M. Signal Processing of Ground-Penetrating Radar Data for Subsurface Object Detection. In Proceedings of the Detection and Remediation Technologies for Mines and Minelike Targets VI, Bellingham, WA, USA, 18 October 2001; Volume 4394, pp. 470–475. [Google Scholar]
  60. Shihab, S.; Al-Nuaimy, W. Radius Estimation for Cylindrical Objects Detected by Ground Penetrating Radar. Subsurf. Sens. Technol. Appl. 2005, 6, 151–166. [Google Scholar] [CrossRef]
  61. Long, K.; Liatsis, P.; Davidson, N. Image Processing of Ground Penetrating Radar Data for Landmine Detection. In Proceedings of the Detection and Remediation Technologies for Mines and Minelike Targets XI, Bellingham, WA, USA, 18 May 2006; Volume 6217, pp. 837–848. [Google Scholar]
  62. Chen, H.; Cohn, A.G. Probabilistic Robust Hyperbola Mixture Model for Interpreting Ground Penetrating Radar Data. In Proceedings of the the 2010 International Joint Conference on Neural Networks (IJCNN), Barcelona, Spain, 18–23 July 2010; pp. 1–8. [Google Scholar]
  63. Dou, Q.; Wei, L.; Magee, D.R.; Cohn, A.G. Real-Time Hyperbola Recognition and Fitting in GPR Data. IEEE Trans. Geosci. Remote Sens. 2017, 55, 51–62. [Google Scholar] [CrossRef] [Green Version]
  64. Rosin, P.L. Unimodal Thresholding. Pattern Recognit. 2001, 34, 2083–2096. [Google Scholar] [CrossRef]
  65. Zhang, J.; Liu, Q.; Nath, B. Landmine Feature Extraction and Classification of GPR Data Based on SVM Method. In International Symposium on Neural Networks, Proceedings of the Advances in Neural Networks—ISNN 2004, Dalian, China, 19–21 August 2004; Yin, F.-L., Wang, J., Guo, C., Eds.; Springer: Berlin/Heidelberg, Germany, 2004; pp. 636–641. [Google Scholar]
  66. El-Mahallawy, M.S.; Hashim, M. Material Classification of Underground Utilities From GPR Images Using DCT-Based SVM Approach. IEEE Geosci. Remote Sens. Lett. 2013, 10, 1542–1546. [Google Scholar] [CrossRef]
  67. Lu, Q.; Pu, J.; Liu, Z. Feature Extraction and Automatic Material Classification of Underground Objects from Ground Penetrating Radar Data. J. Electr. Comput. Eng. 2014, 2014, 28. [Google Scholar] [CrossRef] [Green Version]
  68. Terrasse, G.; Nicolas, J.-M.; Trouvé, E.; Drouet, É. Automatic Localization of Gas Pipes from GPR Imagery. In Proceedings of the 2016 24th European Signal Processing Conference (EUSIPCO), Budapest, Hungary, 29 August–2 September 2016; pp. 2395–2399. [Google Scholar]
  69. Kaur, P.; Dana, K.J.; Romero, F.A.; Gucunski, N. Automated GPR Rebar Analysis for Robotic Bridge Deck Evaluation. IEEE Trans. Cybern. 2016, 46, 2265–2276. [Google Scholar] [CrossRef]
  70. Noreen, T.; Khan, U.S. Using Pattern Recognition with HOG to Automatically Detect Reflection Hyperbolas in Ground Penetrating Radar Data. In Proceedings of the 2017 International Conference on Electrical and Computing Technologies and Applications (ICECTA), Ras Al Khaimah, United Arab Emirates, 21–23 November 2017; pp. 1–6. [Google Scholar]
  71. Ozkaya, U.; Melgani, F.; Belete Bejiga, M.; Seyfi, L.; Donelli, M. GPR B Scan Image Analysis with Deep Learning Methods. Measurement 2020, 165, 107770. [Google Scholar] [CrossRef]
  72. Pasolli, E.; Melgani, F.; Donelli, M.; Attoui, R.; de Vos, M. Automatic Detection and Classification of Buried Objects in GPR Images Using Genetic Algorithms and Support Vector Machines. In Proceedings of the IGARSS 2008, 2008 IEEE International Geoscience and Remote Sensing Symposium, Boston, MA, USA, 6–11 July 2008; Volume 2, pp. II-525–II-528. [Google Scholar]
  73. Kobashigawa, J.S.; Youn, H.; Iskander, M.F.; Yun, Z. Classification of Buried Targets Using Ground Penetrating Radar: Comparison between Genetic Programming and Neural Networks. IEEE Antennas Wirel. Propag. Lett. 2011, 10, 971–974. [Google Scholar] [CrossRef]
  74. Harkat, H.; Ruano, A.E.; Ruano, M.G.; Bennani, S.D. GPR Target Detection Using a Neural Network Classifier Designed by a Multi-Objective Genetic Algorithm. Appl. Soft Comput. 2019, 79, 310–325. [Google Scholar] [CrossRef]
  75. Birkenfeld, S. Automatic Detection of Reflexion Hyperbolas in Gpr Data with Neural Networks. In Proceedings of the 2010 World Automation Congress, Kobe, Japan, 19–23 September 2010; pp. 1–6. [Google Scholar]
  76. Liu, H.; Lin, C.; Cui, J.; Fan, L.; Xie, X.; Spencer, B.F. Detection and Localization of Rebar in Concrete by Deep Learning Using Ground Penetrating Radar. Autom. Constr. 2020, 118, 103279. [Google Scholar] [CrossRef]
  77. Yamaguchi, T.; Mizutani, T.; Nagayama, T. Mapping Subsurface Utility Pipes by 3-D Convolutional Neural Network and Kirchhoff Migration Using GPR Images. IEEE Trans. Geosci. Remote Sens. 2021, 59, 6525–6536. [Google Scholar] [CrossRef]
  78. Liu, L.; Yu, H.; Xu, H.; Wang, B.; Li, J. Underground Object Classification Using Deep 3-D Convolutional Networks and Multiple Mirror Encoding for GPR Data. IEEE Geosci. Remote Sens. Lett. 2022, 19, 4021705. [Google Scholar] [CrossRef]
  79. Safatly, L.; Baydoun, M.; Alipour, M.; Al-Takach, A.; Atab, K.; Al-Husseini, M.; El-Hajj, A.; Ghaziri, H. Detection and Classification of Landmines Using Machine Learning Applied to Metal Detector Data. J. Exp. Theor. Artif. Intell. 2021, 33, 203–226. [Google Scholar] [CrossRef]
  80. Vanijjirattikhan, R.; Khomsay, S.; Kitbutrawat, N.; Khomsay, K.; Supakchukul, U.; Udomsuk, S.; Suwatthikul, J.; Oumtrakul, N.; Anusart, K. AI-Based Acoustic Leak Detection in Water Distribution Systems. Results Eng. 2022, 15, 100557. [Google Scholar] [CrossRef]
  81. Dutta, R.; Cohn, A.G.; Muggleton, J.M. 3D Mapping of Buried Underworld Infrastructure Using Dynamic Bayesian Network Based Multi-Sensory Image Data Fusion. J. Appl. Geophys. 2013, 92, 8–19. [Google Scholar] [CrossRef]
  82. Hall, D.L.; McMullen, S.A. Mathematical Techniques in Multisensor Data Fusion; Artech House: Norfolk County, MA, USA, 2004. [Google Scholar]
  83. Moravec, H.P. Sensor Fusion in Certainty Grids for Mobile Robots. In Sensor Devices and Systems for Robotics, Proceedings of the Special Issue on the IEEE 2008 International Geoscience and Remote Sensing Symposium (IGARSS 2008), Boston, MA, USA, 19 August 2008; Casals, A., Ed.; Springer: Berlin/Heidelberg, Germany, 1989; pp. 253–276. [Google Scholar]
  84. Dong, J.; Zhuang, D.; Huang, Y.; Fu, J. Advances in Multi-Sensor Data Fusion: Algorithms and Applications. Sensors 2009, 9, 7771–7784. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  85. Abdel-Aleem, M.; Chibelushi, C.C.; Moniri, M. Multisensor Data Fusion for the Simultaneous Location and Condition Assessment of Underground Water Pipelines. In Proceedings of the 2011 International Conference on Networking, Sensing and Control, Delft, The Netherlands, 11–13 April 2011; pp. 416–421. [Google Scholar]
  86. Brierley, N.; Tippetts, T.; Cawley, P. Data Fusion for Automated Non-Destructive Inspection. Proc. R. Soc. Math. Phys. Eng. Sci. 2014, 470, 20140167. [Google Scholar] [CrossRef] [PubMed]
  87. Neira, J.; Tardos, J.D. Data Association in Stochastic Mapping Using the Joint Compatibility Test. IEEE Trans. Robot. Autom. 2001, 17, 890–897. [Google Scholar] [CrossRef] [Green Version]
  88. Zhou, X.; Chen, Q.; Jiang, B.; Chen, H. An Underground Pipeline Mapping Method Based on Fusion of Multisource Data. IEEE Trans. Geosci. Remote Sens. 2022, 60, 4511711. [Google Scholar] [CrossRef]
  89. Bilal, M.; Khan, W.; Muggleton, J.; Rustighi, E.; Jenks, H.; Pennock, S.R.; Atkins, P.R.; Cohn, A. Inferring the Most Probable Maps of Underground Utilities Using Bayesian Mapping Model. J. Appl. Geophys. 2018, 150, 52–66. [Google Scholar] [CrossRef]
  90. Cai, J.; Li, S.; Cai, H. Accurate Mapping of Underground Utilities: An Information Fusion Approach Based on Dempster-Shafer Theory. In Proceedings of the Construction Research Congress 2018, New Orleans, LA, USA, 2–4 April 2018; pp. 712–721. [Google Scholar]
Sensors 23 04367 g001 550
Figure 1. Organization of the paper.
Figure 1. Organization of the paper.
Sensors 23 04367 g001
Sensors 23 04367 g002 550
Figure 2. (a) Street-level image showing positions of utility appurtenance in rectangular boxes and the underground utility asset location in continuous lines; (b) utility map at the given location.
Figure 2. (a) Street-level image showing positions of utility appurtenance in rectangular boxes and the underground utility asset location in continuous lines; (b) utility map at the given location.
Sensors 23 04367 g002
Sensors 23 04367 g003 550
Figure 3. GPR radargram informing likely positions of utilities with hyperbolic signatures shown within red rectangles (adapted from Illingworth et al. [53]).
Figure 3. GPR radargram informing likely positions of utilities with hyperbolic signatures shown within red rectangles (adapted from Illingworth et al. [53]).
Sensors 23 04367 g003
Sensors 23 04367 g004 550
Figure 4. Data fusion workflow from Bilal et al. [89].
Figure 4. Data fusion workflow from Bilal et al. [89].
Sensors 23 04367 g004
Sensors 23 04367 g005 550
Figure 5. Graph of error and rate of correctly located utility. PMF– passive magnetic field, MG– magnetic gradiometer, LFEM– low-frequency electromagnetic field, VA– vibro-acoustics (adapted from Dou et al. [44]).
Figure 5. Graph of error and rate of correctly located utility. PMF– passive magnetic field, MG– magnetic gradiometer, LFEM– low-frequency electromagnetic field, VA– vibro-acoustics (adapted from Dou et al. [44]).
Sensors 23 04367 g005
Sensors 23 04367 g006 550
Figure 6. Proposed integrated model enabling virtual sensing of underground utilities.
Figure 6. Proposed integrated model enabling virtual sensing of underground utilities.
Sensors 23 04367 g006
Table
Table 1. Aboveground features of different utility classes (adapted from [35]).
Table 1. Aboveground features of different utility classes (adapted from [35]).
UtilityAssetsLocations
Electricity1, 2, 3, 4, 5, 6, 15, 17, 18Sidewalk, property line, and toward buildings
Sewerage3, 7, 9, 17, 18Sidewalk, property line, and toward buildings
Stormwater3, 6, 8, 17, 18Along the road/right of way
Telecom.1, 2, 3, 5, 6, 10, 11, 17, 18Sidewalk, property line, along the road/right of way
Water3, 12, 13, 16, 17,18Sidewalk and toward buildings
Natural Gas6, 13, 14, 17, 18Sidewalk and toward buildings
1: Pedestal, 2: Control Box, 3: Manhole, 4: Transformer, 5: Electric Poles, 6: Trench Trail, 7: Cleanouts, 8: Storm Traps, 9: Culvert/Gate, 10: Antenna, 11: Closures, 12: Fire Hydrant, 13: Valve Cover, 14: Gas Meter, 15: Electric Meter, 16: Water meter, 17: Utility flags, 18: Field marks.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Oguntoye, K.S.; Laflamme, S.; Sturgill, R.; Eisenmann, D.J. Review of Artificial Intelligence Applications for Virtual Sensing of Underground Utilities. Sensors 2023, 23, 4367. https://doi.org/10.3390/s23094367

AMA Style

Oguntoye KS, Laflamme S, Sturgill R, Eisenmann DJ. Review of Artificial Intelligence Applications for Virtual Sensing of Underground Utilities. Sensors. 2023; 23(9):4367. https://doi.org/10.3390/s23094367

Chicago/Turabian Style

Oguntoye, Kunle S., Simon Laflamme, Roy Sturgill, and David J. Eisenmann. 2023. "Review of Artificial Intelligence Applications for Virtual Sensing of Underground Utilities" Sensors 23, no. 9: 4367. https://doi.org/10.3390/s23094367

APA Style

Oguntoye, K. S., Laflamme, S., Sturgill, R., & Eisenmann, D. J. (2023). Review of Artificial Intelligence Applications for Virtual Sensing of Underground Utilities. Sensors, 23(9), 4367. https://doi.org/10.3390/s23094367

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop