Wavelet Scattering Network-Based Machine Learning for Ground Penetrating Radar Imaging: Application in Pipeline Identification

Abstract

Automatic and efficient ground penetrating radar (GPR) data analysis remains a bottleneck, especially restricting applications in real-time monitoring systems. Deep learning approaches have good practice in automatic object identification, but their intensive data requirement has reduced their applicability. This paper developed a machine learning framework based on wavelet scattering networks to analyze GPR data for subsurface pipeline identification. Wavelet scattering network is functionally equivalent to convolutional neural networks, and its null-parameter property is intended for non-intensive datasets. A double-channel framework is designed with wavelet scattering networks followed by support vector machines to determine the existence of pipelines on vertical and horizontal traces separately. Classification accuracy rates arrive around 98% and 95% for datasets without and with noises, respectively, as well as 97% for considering surface roughness. Pipeline locations and diameters are convenient to determine from the reconstructed profiles of both simulated and practical GPR signals. However, the results of 5 cm pipelines are sensitive to noises. Nonetheless, the developed machine learning approach presents promising applicability in subsurface pipeline identification.

1. Introduction

Ground penetrating radar (GPR) is a well established non-destructive technology for the geophysical investigation of subterranean structures and substances by propagating electromagnetic waves. It remotely monitors the underground conditions as an echo listener and then produces highly-correlated signal profiles, e.g., for applications in infrastructure maintenance [1], archaeology surveys [2], and stratum investigation [3]. Detecting finite objects, namely, pipelines [4], land mines [5], and voids [6], from the noisy background is among common utilization; these targets generate distinct hyperbolas in the recorded profiles. Location accuracy is significant in object identification as errors reach around 30 cm in mapping underground utility infrastructure in real urban environments (summarized by Šarlah et al. [7]) and improving location accuracy has aroused research concerns recently. While absorbing new technologies these years for evolution, GPR has enhanced automaticity and efficiency in broad area measurement. By contrast, automatic and time-efficient data analysis remains a bottleneck especially restricting applications in online monitoring systems [8]. Deconvolution and filtering results depend absolutely on artificial processing experience and interpretation. Various developed inversion techniques based on Maxwell’s equations to migrate scattered signals back requires much integral computation [9,10], improving analysis accuracy but increasing computation burden. Therefore, automatic and desirable data processing methods have been subjected to meticulous investigations.

Artificial intelligence provides computational learning approaches for unmanned data processing. Different from static programs under explicit human instructions, machine learning algorithms acquire inner statistical properties according to sampled joint distribution [11] while ignoring the prior knowledge that is indispensable for classic GPR analysis, and then automatically extract targeted characteristics from radargrams. Various novel machine learning frameworks have contributed to the promotion of object identification performance, e.g., applying the Viola–Jones Algorithm in pattern recognition approach for locating reflection hyperbolas [12], utilizing one computer vision technique, namely, histogram of oriented gradients for landmine detection [13], and adapting region-based convolutional neural networks (CNNs) for subterranean objects recognition [14]. Deep learning including CNN is an increasingly glorious branch of machine learning, against extremely complicated problems but preserving promising learning accuracy. It constructs a considerable size of networks that have the versatile capability to learn GPR signal features. However, the favorable learning reliability relies on multitudinous training data that are difficult to collect all in GPR measurement. For instance, 84 original radargrams in [15] are insufficient for training the AlexNet CNN model containing 60 million parameters. Either extending the datasets over thousands by translation and scaling [15,16] or generating radargrams by numerical simulation [14,17,18] can complement data shortage, but these sources bring doubts on training excessive duplicates or time-consuming problems in data production, respectively. Therefore, a deep learning equivalent framework intended for non-intensive datasets is necessary to expand GPR-related learning researches.

Mallat [19] proposed a wavelet scattering convolution network with translation and rotation invariant operators based on the wavelet transform in 2012. Structurally similar to deep CNN, it decomposes input signals into multi-layer components, each layer consisting of linear and nonlinear operations. Specially, the convolution kernels are predefined by the chosen wavelet, similar to band-pass filters extracting characteristics with physical meanings, and activation functions are replaced by modulus operators that solve the covariant issue of the wavelet transform. As a result of such a model structure, the wavelet scattering network contains no parameters while keeping complicated, which indicates that data volume reduction and model reliability are available simultaneously. It has practically outperformed deep CNN in handwritten digits recognition and texture classification with an accuracy rate of up to 99.7% [20,21], and in synthetic aperture radar target recognition with an accuracy rate of up to 97.63% [22]. However, modulus operation in the wavelet scattering network eliminates location information, and, to the best of our knowledge, no existing researches apply it in automatic objects detection or GPR radargram interpretation.

In this paper, a machine learning framework consisting of wavelet scattering networks and support vector machine (SVM) is investigated for subterranean pipelines identification from GPR profiles. The 2D dataset is decomposed into 1D vertical signals and horizontal signals separately, intending to acquire pipeline coordinates by classification. Both numerically simulated and practical data are utilized to extend the applicability of the learning model. The remainder of this paper is organized as follows. Section 2 describes the data and methodology. Section 3 evaluates the applicability of the proposed learning framework in pipeline identification. Section 4 discusses the results and future improvement. Section 5 concludes the paper.

2. Materials and Methods

2.1. Data Description

To investigate the applicability of machine learning in pipeline identification, we use numerically simulated and practical datasets in this paper.

While considerable GPR data with similar subterranean attributes are required in the learning procedure to acquire internal statistical probability distribution but difficult to collect experimentally, numerical simulation provides an unlimited data production approach. An open source software, “gprMax” [23], is a favorable option to generate GPR profiles. It numerically solves Maxwell’s equations by the Finite-Difference Time-Domain method [24] and offers advanced subterranean modeling, succeeding in both academic and industrial applications [25,26,27]. In this research, we simulated a stochastic number (range: 0–16) of cylinder pipelines (random diameters ranging from 5 to 40 cm) buried randomly inside a 2 m × 1 m subsurface domain and then produced 40 such GPR profiles (a non-intensive dataset) with downsampled data resolution 400 × 448. The underground soil has certain attributes, with the relative permittivity of 8, the conductivity of 0.02 S/m, the relative permeability of 1 and the magnetic loss of 0. Pipelines are simulated as perfect electric conductors. The Ricker waveform with the central frequency 600 MHz is created to simulate the GPR antenna for subsurface detection. Example domains with pre-buried pipelines and corresponding GPR profiles are illustrated in Figure 1.

However, subterranean formation and measurement environment are generally idealized in numerical simulation, which overlooks the system noise, widens transmission angles of electromagnetic waves, and simplifies the material attributes. To extend the applicability of machine learning to a practical phase, three GPR profiles (one provided by the authors of [28]) from field measurements are adopted as the ultimate test set, seen in Figure 2. The first subsurface profile (Figure 2a) contains two separate pre-buried pipelines with central depths of 1 m, diameters of 0.32 m, and their interval is 2.5 m. The central frequency of the GPR antenna is 700 MHz. The second profile contains 5 × 2 double-layer concrete cylinders with central depths of 0.3 m & 0.45 m, diameters of 0.1 m and horizontal intervals of 0.4 m. The central frequency of the GPR antenna is 500 MHz. The third profile contains 3 pre-buried pipelines inside limestone areas (detailed information seen in [28]), and the detection frequency is 250 MHz.

2.2. Pre-Processing

For machine understanding better the targeted characteristics, two preliminary procedures, de-“wow” and amplitude gain, are considered to preprocess the original data.

The “wow” phenomenon refers to the unavoidable occurrence of low-frequency noise components in each GPR trace, predominantly arising from inductive coupling effects or electronic saturation along the ground–air interface [29,30]. This results in zero-offset signals and affects the subsequent procedure as the low-frequency component will obscure real signals after amplitude gain. A simple but practical approach to eliminate the “wow” effects is subtracting the mean signal amplitude.

$$x^{\star }\left(t\right)=x^{\prime }\left(t\right)-\frac{1}{n}\sum _{i=1}^n{x}^{\prime }\left(t_i\right)$$

(1)

where $x^{\prime }\left(t\right)$ is the original signal trace, $x^{\star }\left(t\right)$ represents the signal after de-“wow”, and $t_i$ ($i=1,2,\cdots ,n$) is the sampling time. Figure 3 shows an example of de-“wow” results.

In the two-way propagation of electromagnetic waves, the energy attenuates with time due to dielectric loss and geometrical spreading consumption (heavy scattering and refraction at interfaces) [31]. Amplitude gain is indispensable for enhancing the later arrived signals to a distinguishable degree. Considering the aforementioned attenuation, exponential and linear functions are combined for amplitude gain operation.

$$\begin{array}{cc}\hfill F\left(t\right)& =e^{2\alpha (t-t_0)}+2\beta (t-t_0)\hfill \\ \hfill x\left(t\right)& =F\left(t\right)x^{\star }\left(t\right)\hfill \end{array}$$

(2)

where $x\left(t\right)$ represents signals after preprocessing, $\alpha$ is the exponential attenuation coefficient corresponding to dielectric loss, $\beta$ is the linear attenuation coefficient related to geometrical spreading consumption, and $t_0$ is the corrected time at the ground–air interface.

The choices of $\alpha$ & $\beta$ are based on experience. In this paper, we determined $\alpha =0.0156\frac{N-N_0}{t-t_0}$ & $\beta =0.12\frac{N-N_0}{t-t_0}$ for simulated GPR signals, $\alpha =0.0112\frac{N-N_0}{t-t_0}$ & $\beta =0$ for the first two measurement profiles, and $\alpha =0.006\frac{N-N_0}{t-t_0}$ & $\beta =0.05\frac{N-N_0}{t-t_0}$ for the third measurement profile, where N represents the signal point number corresponding to the time t, and $N_0$ is the signal point number corresponding to the time $t_0$. Example data profiles after preprocessing are given in Figure 4.

2.3. Wavelet Scattering Network

Wavelet scattering is a null-parameter convolution network originally proposed by Mallat [19] for translation and rotation invariant characterization with specific wavelet approaches. Wavelet transform, which is both a mode recognition and decomposition approach, provides the basic theory for the scattering network to extract either evident or invisible data features. The physical meaning of the wavelet transform is to calculate the joint energy spectrum of signals in the frequency-time domain and thereby to identify both frequency and time information of the distinct modes [32]. The procedure of the wavelet decomposition is complete and reversible, which means no signal features would be lost. That is different from CNN, which is adjustable during training to preserve targeted signal features only. Wavelet scattering networks utilize the wavelet group and scaling functions as convolution kernels to filter signals in preset orientations and bandwidths, while CNN trains undetermined convolution kernels for filtering. This significant factor determines that wavelet scattering networks contain no parameters while performing functionally equivalent to CNN.

A wavelet function group is available by dilating and rotating the mother wavelet:

$${\psi }_{2^{j}r}\left(t\right)=2^{dj}\psi \left(2^j{r}^{-1}t\right)$$

(3)

where mother wavelet $\psi \in {\mathbf{L}}^2\left(R^d\right)$, $2^{-j}$ represents the dilation rate, and r is the rotation coefficient. Wavelet transform decomposes the original signal $x\left(t\right)$ through the band-pass filter ${\psi }_{\lambda }$ by convolution calculation.

$$W_{\lambda }x=x\otimes {\psi }_{\lambda }=\int x\left(\tau \right){\psi }_{\lambda }(t-\tau )d\tau$$

(4)

where $\lambda =2^{j}r$ to simplify notation, $W_{\lambda }$ is the wavelet transform operator, and ⊗ represents convolution calculation. At any transform scale $2^J$, signal components with frequency $2^j>2^{-J}$ are reserved. The low-frequency component not included is decomposed through the scaling functions within the space proportional to $2^J$,

$$A_{J}x=x\otimes {\varphi }_J$$

(5)

$${\varphi }_J\left(t\right)=2^{-dJ}\varphi \left(2^{-J}t\right)$$

(6)

where $A_J$ is the scaling operator, ${\varphi }_J$ is the scaling function at the scale $2^J$, while $\varphi$ is the scaling function at the original scale.

Although wavelet transform can map local signal features, the convolution calculation is translation covariant. It will distinguish similar characteristics at different locations into separate categories, thereby increasing the learning complexity. Translation invariance is significant for classification since the horizontal and vertical movement of the same objects should result in few classification mistakes [33]. Mallat [19] demonstrated that the integration of the modulus $\mid x\otimes {\psi }_{\lambda }\mid$ is translation invariant and introduced a path-sorted iteration operator on the modulus to create the scattering propagator.

$$U_{\lambda }x=\mid W_{\lambda }x\mid =\mid x\otimes {\psi }_{\lambda }\mid$$

(7)

$$U\left[p\right]=U_{{\lambda }_m}...U_{{\lambda }_2}{U}_{{\lambda }_1}$$

(8)

$$U\left[p\right]x=\mid \mid \mid x\otimes {\psi }_{{\lambda }_1}\mid \otimes {\psi }_{{\lambda }_2}\mid ...\otimes {\psi }_{{\lambda }_m}\mid$$

(9)

where $U_{{\lambda }_i}$$(i=1,2,\cdots ,m)$ is the modulus operator at the ith scattering stages, and $U\left[p\right]$ is the scattering propagator. Similar to the wavelet transform, a windowed scattering transform is introduced to extend the frequency scale.

$$S_J\left[p\right]x=U\left[p\right]x\otimes {\varphi }_J=\int U\left[p\right]x\left(\tau \right){\varphi }_J(t-\tau )d\tau$$

(10)

$$S_J\left[p\right]x=\mid \mid \mid x\otimes {\psi }_{{\lambda }_1}\mid \otimes {\psi }_{{\lambda }_2}\mid ...\otimes {\psi }_{{\lambda }_m}\mid \otimes {\varphi }_J$$

(11)

The operation of $U\left[p\right]x$ and $S_J\left[p\right]x$ is contractive and stable. Therefore, the wavelet scattering network is constructed by continuously calculating the convolution results of Equations (9) and (11), as shown in Figure 5.

The wavelet group and scaling functions compose the convolution kernels of the wavelet scattering network, and the modulus operator works as the activation function (seen in the network structure, Figure 5). Morlet wavelet assisted by the Gaussian window is utilized in this research for scattering propagation, as expressed in Equation (12) and shown in Figure 6.

$$\psi \left(t\right)=K_{{\sigma }_t}{e}^{-\frac{t^2}{2{\sigma }_t^2}}{e}^{2\pi ift}$$

(12)

where $K_{{\sigma }_t}$ is the normalization constant, ${\sigma }_t$ represents the wavelet duration, i is the imaginary unit, and f is proportional to the central frequency.

2.4. Support Vector Machine

Support vector machine (SVM) is a supervised classification approach rooted in the statistical learning theory, mathematically solving the dual optimization problem based on structural risk minimization [34,35]. In linear classification, where the classification model is a linear function of input parameters, data points are divided into different categories by a hyperplane. The optimal classification hyperplane to separate data points is achieved at point-plane distance maximization. For nonlinear classification, a hypercurved surface is required to separate the data points but difficult in the calculation. Kernel functions are adopted to map the input domain onto the high-dimensional Hilbert space, resulting in problem transformation into linear classification. Therefore, choosing an appropriate kernel function determines the applicability of the SVM classifier. The radial basis function kernel is the most common one with good applications in practical problems and is utilized in our research.

In binary classification, given input training set $T=\{(x_1,y_1),(x_2,y_2),...,(x_N,y_N)\}$ (where $x_i\in {\mathbf{R}}^n$, $y_i\in \{-1,1\}$, $i=1,2,...,N$), SVM classifier solves the following dual optimization problem.

$$\begin{array}{cc}\hfill & \underset{\gamma }{min}\frac{1}{2}{\gamma }^{T}Q\gamma -e^T\gamma \hfill \\ \hfill & s.t.y^T\gamma =0\hfill \\ \hfill & 0\le {\gamma }_i\le C,i=1,2,..,N\hfill \end{array}$$

(13)

where $\gamma$ is the classification hyperplane coefficient vector, Q is a $N\times N$ positive semidefinite matrix with $Q_{i,j}=y_i{y}_{j}K(x_i,x_j)$, $K(x_i,x_j)=h\left(x_i\right)h\left(x_j\right)$ is the kernel function with $h\left(x_i\right)$ mapping the input domain onto the high-dimensional Hilbert space, and C is the upper-boundary parameter of $\gamma$. Once achieving the optimal solution ${\gamma }^{\star }=({\gamma }_1^{\star },{\gamma }_2^{\star },...,{\gamma }_N^{\star })$, we can express the constant parameter of the hyperplane using any ${\gamma }_i^{\star }$.

$$b^{\star }=y_j-\sum _{i=1}^N{\gamma }_i^{\star }{y}_{i}K(x_i,x_j)$$

(14)

Therefore, the output decision function for any given input x is

$$f\left(x\right)=sgn(\sum _{i=1}^N{\gamma }_i^{\star }{y}_{i}K(x_i,x)+b^{\star })$$

(15)

where $sgn$ represents the sign function $sgn\left(x\right)=\frac{x}{\mid x\mid }$. In this research, the developed python module “scikit-learn” [36] is utilized for coding the nonlinear SVM classifier.

2.5. Learning Framework

As the general framework that directly classifies 2D GPR profiles overlooks the location and magnitude information [37,38], a specific architecture (seen in Figure 7) is designed for accurately identifying the positions of covered pipelines. The training set is decomposed into two datasets consisting of 1D vertical signals and horizontal signals respectively, followed by separate learning procedures where a multi-layer wavelet scattering network connected with SVM classifier is established. This double-channel framework works to determine the coordinates by recognizing the existence of pipelines on the single signal trace vertically and horizontally. After learning procedures, GPR profiles with visible pipeline locations are reconstructed utilizing data at output coordinates.

A brief scheme of profile reconstruction is provided by Figure 8, where we only illustrate two horizontal trace groups and two vertical trace groups. The blue traces are classified as “negative” traces (non-existence of pipelines) while the red traces are classified as “positive” traces (existence of pipelines) by the two SVM classifiers. The four trace groups form four intersections, and only the intersection D of the two “positive” groups $H2$ & $V2$ is identified as a “positive” section. Therefore, we reconstruct the ultimate profile by the signals inside all “positive” sections.

3. Results

3.1. Simulated Signal Results

First, our proposed learning framework is investigated by the 40 numerically produced GPR B-scan profiles, among which 32 and 8 profiles are randomly separated for the training set and validation set, respectively. Each signal profile is rescaled between 0 and 1 to ensure the learning model suitable for further practical applications. Although the input dataset is small, it indeed contains $40\times 400$ vertical traces and $40\times 448$ horizontal signals for the “wavelet scattering network +SVM” learning procedure, which is sufficient to achieve reasonable results.

A comparison between an independent SVM and our proposed framework at the classification stage is operated to visibly evaluate the learning results. The double-channel classifiers can determine whether pipelines exist on the vertical traces or the horizontal signals, with their learning accuracy shown in

Table 1. Accuracy and confusion results of vertical traces with or without wavelet scattering networks in the machine learning model.

Models	Training Accuracy		Validation
		Accuracy	False Positive	False Negative
WaveScat + SVM	99.73%	97.94%	0.91%	1.15%
SVM	78.59%	78.44%	16.87%	4.69%

and

Table 2. Accuracy and confusion results of horizontal signals with or without wavelet scattering networks in the machine learning model.

Models	Training Accuracy		Validation
		Accuracy	False Positive	False Negative
WaveScat + SVM	99.70%	98.41%	0.53%	1.06%
SVM	72.10%	72.38%	9.20%	18.42%

. The classifiers containing wavelet scattering networks remarkably outperform the independent SVM, improving validation accuracy to 97.94% and 98.41%. False rates drop below 2.1%, which means that average of 8 vertical traces or horizontal signals per profile are misidentified, and such insignificant errors confuse little of further pipeline recognition. The convincing correct classification rates indicate the feature extraction efficiency of wavelet scattering networks. By contrast, the independent SVM model fails to capture the signal characteristics arising from pipeline existence, as reflected by the unacceptable false rate of over 20%. The false positive rates of classifying both vertical and horizontal signals crumble into an eyesore, as 16.87% & 9.20% of signals not containing pipelines are misidentified. By this comparison, thus the visible results demonstrate the promising eligibility of the proposed framework in identifying pipeline presence.

The further procedure concentrates on automatically reconstructing GPR profiles to present distinguishable pipeline locations and sizes. Data points repeatedly labeled “positive” (represent pipeline existence) on both the vertical trace and the horizontal signal preserve their values while others are eliminated. The rescaled data less than 0.6 are also eliminated for better locating the upper pipeline interfaces. The results of single-pipeline profiles for validation are presented in Figure 9, where upper panels are input profiles (with pipelines manually marked by dash circles), middle panels are reconstructed by the proposed learning framework (“positive” areas in white and “negative” areas in gray) and lower panels are reconstructed after the single SVM. Although length coordinates of pipelines are convenient to determine from hyperbolic patterns (Figure 9a,b), reliable depth coordinates and diameters are unavailable. In contrast, the pipelines in Figure 9c,d are tangent to the “positive” rectangle with upper interface signals inside, thus determining coordinates by upper interfaces and diameters by minimum rectangle sides. The learning results of central coordinates and diameters are ((0.62 m, 0.636 m), 0.245 m) in Figure 9c and ((0.28 m, 0.669 m), 0.346 m) in Figure 9d, respectively, corresponding brilliantly with the actual sizes (seen in caption of Figure 9, errors within 1 cm). Two insignificant misclassified “positive” areas appear in Figure 9c but no upper interface occurs inside, affecting little on pipeline identification. Therefore, single-pipeline profiles are favorably reconstructed by the proposed learning framework. By comparison, although the upper interface of the pipeline is visible in Figure 9e, the identified coordinates and diameter are ((0.62 m, 0.64 m), 0.14 m) with a 44% diameter error. The pre-buried pipeline is identified as two small pipelines in Figure 9f, which means the independent SVM performs undesirable in reconstructing the profiles. The accuracy of classification determines the accuracy of profile reconstruction and location identification.

Interpreting reconstructed profiles of multiple pre-buried pipelines becomes complicated since “pseudo-positive” areas appear thus interrupting direct identification, e.g., two “positive” vertical traces and two “positive” horizontal signals can generate four “positive” intersections (seen in Figure 10c). Nonetheless, the intended approach is still to distinguish upper interfaces of pipelines according to their properties that they are approximately parallel to horizontal lines, and then determine the sizes by external “positive” sections. Figure 10a,c shows the learning results of two pipelines manually marked in dashed circles. Three visible hyperbolic patterns occur in the input profile, confusing the location determination of the two pipelines, and diameters are difficult to determine. Contrarily, although four “positive” sections except the two insignificant misclassified areas are reserved in learning results (Figure 10c), the reconstructed profile only contains two distinguishable upper interfaces, further determining coordinates and diameters as aforementioned. The output characterized values of two pipelines are ((1.365 m, 0.263 m), 0.145 m) & ((0.88 m, 0.831 m), 0.395 m), conforming promisingly to the pre-set sizes (seen in caption of Figure 10, errors within 1 cm), which indicates the learning approach is suitable for recognizing two pipelines. However, in the results after the independent SVM classifier (seen in Figure 10e), the upper pipeline is identified but the other pipeline is nearly invisible. The output coordinates and diameter are ((1.365 m, 0.243 m), 0.27 m) with an unacceptable diameter error of 93%. For multiple-pipeline profiles, the pipelines generate a large number of interference signals that obstruct the immediate recognition, as shown in Figure 10b,d. Numerous non-pipeline-related hyperbolic patterns appear in the input profile, hardly providing any magnitude information. In comparison, the six upper interfaces are convenient to identify inside the ‘positive’ sections since other non-interface signals are shattered and nonparallel to horizontal lines. The output characterized values of left five pipelines are ((0.125 m, 0.56 m), 0.095 m), ((0.51 m, 0.683 m), 0.095 m), ((0.7 m, 0.710 m), 0.195 m), ((1.105 m, 0.676 m), 0.045 m) & ((1.37 m, 0.464 m), 0.045 m), and height and width of the rightest pipeline are (0.377 m, 0.365 m) (the rightest pipeline has crossed two boundaries), which present brilliant correspondence with actual sizes. The horizontal and the diameter errors are within 1 cm while the vertical error is within 2.5 cm. The proposed machine learning approach achieves promising results in multiple pipeline identification. In comparison, all six pipelines are difficult to identify from the reconstructed profile after the independent SVM (seen in Figure 10f), which indicates the classification accuracy determines the further identification accuracy.

3.2. Noise Sensitivity

Second, the noise sensitivity of our learning framework is necessarily evaluated since diverse background noises and shattered subsurface formation affect actual signals from the objects. Random 2D Gaussian white noises with zero mean and standard deviation $\sigma$ = 0.1 have supplemented the rescaled GPR profiles as model inputs. A comparison between an independent SVM and our proposed framework at the classification stage is still under consideration to examine learning stability, with the learning accuracy shown in

Table 3. Accuracy and confusion results of vertical traces (noisy signals) with or without wavelet scattering networks in the machine learning model.

Models	Training Accuracy		Validation
		Accuracy	False Positive	False Negative
WaveScat + SVM	99.99%	94.81%	2.72%	2.47%
SVM	84.77%	76.25%	14.06%	9.69%

and

Table 4. Accuracy and confusion results of horizontal signals (noisy signals) with or without wavelet scattering networks in the machine learning model.

Models	Training Accuracy		Validation
		Accuracy	False Positive	False Negative
WaveScat + SVM	99.76%	94.98%	2.01%	3.01%
SVM	82.42%	67.64%	12.83%	19.53%

. The classification accuracy of “wavelet scattering network + SVM” has reached 94.81% and 94.98%, noticeably outperforming the independent SVM, and these promising results are adequate for further profile reconstruction. Affected by the complemented noises, the validation accuracy of our framework has decreased around 3%, which indicates noises are worth attention in GPR signal processing. Nonetheless, the accuracy rates maintain around 95%, demonstrating the stability of the learning procedures to the added noises.

The stability of identifying pipeline locations and diameters is further evaluated in reconstructed profiles. Figure 11 illustrates the results of single-pipeline profiles for validation. Although the hyperbolic patterns are still convenient to recognize, they are converted into speckled hyperbolas, increasing difficulty in acquiring reliable depth coordinates and diameters. There are still two insignificant misclassified “positive” areas with no identified upper interfaces, and they occur at different places from Figure 9c. Shattered noise speckles exist inside the pipeline areas, but they hardly affect the determination of pipeline locations and sizes. The learning results of central coordinates and diameters are ((0.6225 m, 0.631 m), 0.24 m) in Figure 11c and ((0.28 m, 0.667 m), 0.345 m) in Figure 11d respectively, corresponding brilliantly (errors within 1 cm) with the actual sizes and the aforementioned non-noise results. Therefore the proposed learning framework is stable in single pipeline identification. In contrast, since the learning accuracy of the independent SVM is below 85%, the corresponding reconstructed profiles fail to present the locations of the pipelines, as shown in Figure 11e,f. The added noises have increased difficulty for the simple learning procedure.

Situations of multiple pipeline identification become complicated since noises would ruin the hyperbolic patterns from weak GPR responses. Figure 12c shows the results of the reconstructed double-pipeline profile. Although insignificant, multiple misclassified areas appear and even interrupt the middle-lower “positive” section with “false-negative” identification. Nevertheless, two upper interfaces are distinguishable from the noises, further determining the pipeline characterized values, ((1.36 m, 0.263 m), 0.145 m) & ((0.88 m, 0.839 m), 0.395 m). The horizontal and the diameter errors are within 1 cm while the vertical error is within 2.5 cm. For multiple-pipeline profiles, the noises interrupt the hyperbolic patterns as shown in Figure 12b,d, which increases difficulty in immediate recognition. Although four upper pipeline interfaces are convenient to identify in Figure 12d, those of the two small pipelines (diameters of 0.05 m) are nearly ruined and hard to determine in the reconstructed profile. Therefore, our machine learning framework is sensitive to noises when overcoming the identification problem with multiple small pipelines (diameters of 0.05 m). Excepting this, the proposed approach presents promising eligibility in GPR profile reconstruction and pipeline identification. In contrast, the comparing results of the independent SVM remain blurring for pipeline recognition since the “positive” traces are shattered-distributed. Only one upper interface in each profile is visible (seen in Figure 12e,f), but the locations and diameters are difficult to determine. The low learning accuracy of the simple learning procedure has affected further profile reconstruction.

3.3. Effect of Surface Roughness

Third, the effect of surface roughness is analyzed since the rough surface can result in unstable signals [39,40] affecting further pipeline identification. The surface roughness is complemented to the 40 simulated radargrams by “gprMax”, with a maximum amplitude of 5 cm (5% of the depth, 12.5% of the maximum pipeline diameter). A comparison between an independent SVM and our proposed framework is considered to evaluate learning stability, with the learning accuracy shown in

Table 5. Accuracy and confusion results of vertical traces (considering surface roughness) with or without wavelet scattering networks in the machine learning model.

Models	Training Accuracy		Validation
		Accuracy	False Positive	False Negative
WaveScat + SVM	98.01%	96.72%	2.34%	0.94%
SVM	79.22%	77.81%	6.88%	15.31%

and

Table 6. Accuracy and confusion results of horizontal signals (considering surface roughness) with or without wavelet scattering networks in the machine learning model.

Models	Training Accuracy		Validation
		Accuracy	False Positive	False Negative
WaveScat + SVM	98.84%	97.09%	1.98%	0.93%
SVM	78.61%	76.10%	19.00%	4.90%

. The classification accuracy of “wavelet scattering network +SVM” has reached 96.72% & 97.09%, around 20% larger than that of the independent SVM. Affected by the rough surface, the validation accuracy of our framework has decreased around 1%, which indicates that surface roughness is a significant factor in GPR signal processing. Nonetheless, the high accuracy rates have demonstrated the stability of our learning procedure to surface roughness.

The effect of surface roughness in identifying pipeline locations and diameters is further investigated in reconstructed profiles. Figure 13 presents the results of single-pipeline profiles for validation. In Figure 13a,b, although the hyperbolic patterns are still convenient to recognize, the signals become fluctuated and noisy due to the rough surface, which increases the difficulty in acquiring pipeline coordinates and diameters. Some insignificant misclassified “positive” areas appear in Figure 13c,d and they will not affect pipeline identification. The pipeline area in Figure 13d is divided into three pieces by the false “negative” areas, but we can identify them as an entire section. The learning results of central coordinates and diameters are ((0.6125 m, 0.6283 m), 0.24 m) in Figure 13c and ((0.2825 m, 0.659 m), 0.355 m) in Figure 13d, respectively, with errors less than 1 cm. The proposed learning framework is applicable in pipeline identification despite the effect of surface roughness. By contrast, the reconstructed profiles after the independent SVM fail to present pipeline areas because of the low learning accuracy, as shown in Figure 13e,f, which indicates that the learning accuracy determines the profile reconstruction accuracy.

The cases of multiple pipeline identification are complicated since the hyperbolic signals have become fluctuated and noisy to ruin each other (shown in Figure 14a,b). In the reconstructed profile (Figure 14c) by our learning framework, the upper surfaces of the two pipelines are distinguishable inside the “positive” sections despite some insignificant misclassified areas. The identified central coordinates and diameters are ((1.36 m, 0.2615 m), 0.145 m) and ((0.88 m, 0.8205 m), 0.4 m), with errors less than 1.5 cm. For multiple-pipeline profiles, the two pipelines with diameters 0.05 m are not distinguishable in the “positive” sections, similar to the results in Section 3.2. This is affected by the surface roughness as the maximum roughness is 0.05 m. Upper surfaces of the residual four pipelines are convenient to recognize, and the horizontal and diameter errors are within 1 cm while the depth errors are within 3 cm. The proposed approach shows promising applicability in multiple pipeline identification. Contrarily, although the upper pipeline in Figure 14e can be identified, the reconstructed profiles after the independent SVM fail to present other pipeline areas, thereby not suitable for pipeline identification. The classification accuracy determines further reconstruction accuracy.

3.4. Applicability in Field Signals

Fourth, three practical radargrams from filed measurements are adopted as the final test set to investigate the applicability in field signals. They are fed into the trained machine learning model from Section 3.2, as the simulated noisy inputs are comparable to practical signals. To convert the three radargrams with the input format, we linearly interpolated in the profiles to 448 × 400 pixels and then rescaled them between 0 to 1 (seen in left panels in Figure 15). After processed by the trained model, the reconstructed profiles are illustrated in the right panels in Figure 15. Although hyperbolic patterns are distinguishable and can provide some location information in input profiles, it is difficult to determine the target sizes. Each pipeline in Figure 15e generates two hyperbolas, which increases the confusion of the locations by immediate identification. Contrarily, both location and diameter information can be acquired from the reconstructed profiles. In Figure 15b, two pipelines are identified inside the ‘positive’ areas where upper interface signals are visible, and their output characterized values (central coordinates and diameters) are ((3.74 m, 1.02 m), 0.34 m) and ((6.14 m, 0.96 m), 0.34 m), corresponding well with the actual diameters 0.32 m. In Figure 15d, the 5 × 2 cylinders’ locations are determined by the “positive” sections, and their identified diameters are 0.1 m or 0.1125 m, similar to the actual diameters 0.1 m. In Figure 15f, only the upper two pipelines are accurately identified with upper interfaces’ depth 1.14 m and 1.51 m and diameters 0.54 m and 0.54 m, respectively. The machine learning model failed to vertically identify the deepest pipeline since signals are weak beyond 2 m depth but our training inputs are all within 1 m depth. The depth and the diameter errors have reached 14 cm and 4 cm respectively in the third profile, and this may arise from the significantly different central frequency (250 MHz) of the third detection. Nevertheless, the proposed approach presents promising applicability in field signals.

4. Discussion

Two significant research developments resulting from our novel machine learning approach are demonstrated in the results: outputting size information and training on small datasets.

Object sizes are difficult to determine in GPR detection of subterranean sections. Although the development of radargram interpretation approaches from manual processing to machine learning is taking place, most researches (see, e.g., in [41,42,43]) on object detection only determined two characteristics: whether the objects were detectable and where the objects were. Some researches have determined the sizes of rebars [44] and small-scale voids [45] inside concrete successfully, but investigations inside the complicated underground sections become difficult. For instance, Pasolli et al. [46] attempted to estimate the buried object size, but they only utilized numerically produced data and the mean error was up to 18.6%; Luo and Lai [47] failed to determine the subsurface void sizes as the identified magnitudes significantly different from the actual sizes. In this paper, both locations and diameters of the subsurface pipelines can be extracted from the reconstructed profiles. The diameter errors are within 1 cm for numerically generated datasets (pipeline diameters: 5–40 cm) and within 4 cm for practical datasets (pipeline diameters: 10–50 cm). These acceptable errors arise from only one or two misclassified vertical or horizontal traces, which indicates a promising performance in determining pipeline sizes.

Dataset requirement is a major issue affecting applications of machine learning in GPR signal processing, as introduced in Section 1. Translation and scaling are reasonable approaches to complement the data shortage, but each profile is usually necessary to generate hundreds of duplicates that bring doubts about training reliability. Besides, producing considerable radargrams by numerical simulation and artificially labeling the large datasets are time-consuming. In this paper, a null-parameter wavelet scattering network, functionally comparable to CNN, is utilized in our machine learning framework. As the convolution kernels are predefined without parameters, we do not train the networks and thereby data requirement decreases. The classification accuracy rates are around 98% and 95% for datasets without and with noises respectively, as well as 97% for considering surface roughness. We also want to mention that the proposed learning framework is only compared with an independent SVM but not with CNN, because the purpose here is to evaluate machine learning on small datasets but the data requirement is large for CNN. Nonetheless, promising performance in both simulated GPR signals and practical radargrams demonstrates the applicability and efficiency of the proposed approach.

The practical applicability is also investigated in this paper but not extended to all aspects. Although cases considering complicated pipeline distribution are analyzed, the subsurface material is simple in both simulated and field-measured profiles, e.g., slightly variant attributes and no underground stratification. Two environmental conditions, the noises and surface roughness, have been analyzed to improve the practical applicability. The added Gaussian noises can help investigate the effects of the environmental noises and the shattered subsurface formations, while surface roughness will vary in different urban locations. Other environmental conditions (e.g., the surface material and the complicated underground formations) and detection settings (e.g., the positioning method of GPR and the bandwidth) are also important in GPR radargram interpretation, and the applicability of our learning framework will be investigated in more complicated cases in the future. Further research is planned in the following four aspects. First, we will extend our exploration depth and consider more complicated subsurface conditions; second, the practical applicability of our learning framework will be further investigated in complicated environmental conditions with different GPR detection settings; third, as the identification of upper interfaces in the reconstructed profile is manual in this paper, we will investigate an automatic approach to overcome this disadvantage; fourth, the research applications will be extended to identify other subterranean objects and even shattered underground structures.

5. Conclusions

In this paper, a wavelet scattering network based machine learning approach intended for non-intensive GPR datasets is investigated for subsurface pipeline identification. Wavelet scattering is a null-parameter convolution network for translation and rotation invariant characterization with specific wavelet approaches. It can extract signal features by predefined convolution kernels without parameters, thereby reducing data requirement. In our learning structure, the double-channel procedures, each containing a multi-layer wavelet scattering network followed by a SVM, work to determine the coordinates by recognizing the existence of pipelines on the single signal trace vertically and horizontally. GPR profiles are then reconstructed by signals inside the “positive” sections (vertical and horizontal traces contain the pipelines) for further determining pipeline locations and sizes.

Promising performance is achieved in both simulated and practical datasets. The classification accuracy rates are around 98% and 95% for datasets without and with noises respectively, as well as 97% for considering surface roughness. Upper interfaces of pipelines are convenient to identify in reconstructed profiles, and further determined locations and diameters conform well with the actual values. Pipelines with diameters of 0.05 m are difficult to identify in noisy profiles, which means their GPR patterns are sensitive to noises. These small pipelines are also unrevealed in profiles considering the rough surface since their diameters are similar to the maximum roughness. Feeding the practical radargrams as the test set into the trained learning model, “positive” sections corresponds promisingly with the object locations. The diameters of the pipelines are accurately determined despite small biases. However, the machine learning model failed to vertically identify the deepest pipeline as signals are weak beyond 2 m depth but our training inputs are all within 1 m. The depth and the diameter errors have reached 14 cm and 4 cm respectively in the third profile, and this may arise from the significantly different central frequency (250 MHz) of the third detection. Reliable detection depth of GPR is determined by the central frequency of the electromagnetic wave and the attribute of the subsurface formation, while detection resolution is limited by the wave frequency and signal bandwidth [48]. For example, high-frequency electromagnetic waves can recognize small objects, but within shallow depth due to sharp energy attenuation. Excepting the noise sensitivity of small pipelines and the failure in recognizing the deep pipeline, the proposed machine learning approach presents promising applicability in both simulated and practical GPR signals.