Gabor Filter-Based Segmentation of Railroad Radargrams for Improved Rail Track Condition Assessment: Preliminary Studies and Future Perspectives

Abstract

Ground penetrating radar (GPR) has been used for several years as a non-contact and non-destructive measurement method for rail track analysis with the aim of recording the condition of ballast and substructures. As the recorded data sets typically cover a distance of many kilometers, the evaluation of these data involves considerable effort and costs. For this reason, there is an increasing need for automated support in the evaluation of GPR measurement data. This paper presents an image segmentation pipeline based on 2D Gabor filter texture analysis, which can assist users in GPR data-based track condition assessment. Gabor filtering is used to transform a radargram image (or B-scan) into a high-dimensional, multi-resolution representation. Principal component analysis (PCA) is then applied to reduce the data content to three characteristic dimensions (namely amplitude, frequency, and local scattering) to finally obtain a segmented radargram image representing different classes of relevant image structures. From these results, quantitative measures can be derived that allow experts an improved condition assessment of the rail track.

1. Introduction

Data acquisition by means of ground penetrating radar (GPR) in railway track maintenance is an established, non-destructive measurement method for monitoring ballast conditions and other parameters [1,2,3,4]. GPR can provide measurements of the layer thickness and composition (lateral and longitudinal changes) of individual subgrade layers, moisture content and contamination condition of the ballast. It can map the lateral and longitudinal changes of ballast and subgrade layers to provide information on failure and deformation conditions with minimal disruption to train operations. In this respect, GPR can be used as an effective tool to uncover the causes of recurrent poor rail track geometries and unstable roadways [5]. It is also possible to identify and better investigate problematic railway sections by considering, for example, successive inspection campaigns that show a systematic increase in geometric fault parameters over time [6].

For this purpose, the radar measurement system is usually mounted directly on a rail vehicle, as shown in Figure 1a. The unit of transmitter and receiver is called a transducer which emits pulses of radar waves (electromagnetic waves, usually with a center frequency of 200 MHz and above) towards the ground and receives their reflections (Figure 1b). During the radar survey of a railway track, radar pulse echoes (single reflection signals referred to as “traces” or A-scans) are continuously detected and recorded at regular intervals while the vehicle is moving forward (Figure 1c). In this way, a pictorial representation of the local radar reflection properties of the substructure can be generated by combining the individually received reflection signals column by column to form a so-called radargram (or B-scan, Figure 2). This allows trained personnel to examine the subsurface structure of the track and to detect any anomalies (e.g., changes in the depth of the track planum, estimation of the degree of contaminations, and the moisture content as well as accumulated water and settlements, etc.). After manual evaluation of the recorded data, specific maintenance work can be derived. Thus, the evaluation (i.e., interpretation and categorization) of this measurement data requires a visual inspection, usually carried out by persons with geophysical training and experience. However, even for experienced persons familiar with railway radargrams, the accurate and reproducible labeling, e.g., of sub-track layer boundaries, still poses a practical challenge. On the one hand, GPR measurements can be carried out at high speeds of more than 100 km/h, so that large distances can be covered in a very short time. On the other hand, the subsequent manual evaluation of the collected image data is very time-consuming. Automatic support of this process could significantly reduce inspection times and increase objectivity in the evaluation.

Different approaches for automated support in the evaluation of GPR data in railway maintenance are reported in the literature. Frequently, these methods are based on time domain analysis or frequency domain analysis of individual GPR signals (traces) e.g., by means of 1D Fourier transform [7,8]. Combined time–frequency analysis methods are reported in [9,10,11] but also applied only to single signal traces (1D). Typical applications refer to the quantification of ballast conditions, geometric ballast properties, or the measurement of moisture content [12,13,14,15,16]. In [17,18], the application of image processing approaches supported by machine learning methods (image classification) is described to characterize mainly ballast conditions and ballast stone size distributions. Recent attempts deal with deep learning approaches for image-based classification, e.g., object detection methods [19], but with the disadvantage that a large amount of high-quality labeled ground truth data is needed in advance to achieve good results.

The aim of the method presented here is to provide automated support for the exact (pixel-wise) segmentation of relevant image regions in typical railway radargrams. The data basis for our evaluations was provided by Groundcontrol GmbH, a company that has surveyed more than 100,000 km of railway tracks worldwide using GPR. With many years of experience in the interpretation of such measurement data, experts from this company have accompanied the development of the presented method. The novelty of this work consists in the application of 2D Gabor filtering for image texture analysis and the interpretation of the results of a subsequent dimensionality reduction by means of principal component analysis (PCA). It is shown that the first three principal components can be assigned to visually interpretable image structures, which are subsequently used for a pixel-precise segmentation of relevant image areas. Technically, the presented method can be classified as a two-dimensional combined time–frequency analysis approach for segmenting radargram images. As a result, the exact subdivision of radargram image structures becomes possible in a level of detail that far exceeds the purely visual capabilities of a human observer, e.g., for determining the exact course of ground structure boundaries.

The paper is structured as follows: Section 2 presents in detail the segmentation methodology based on image texture analysis using 2-dimensional Gabor filters and principal component analysis (PCA). In Section 3, the results of the proposed method are shown using typical radargrams from GPR railway surveys. In Section 4, a refinement of the proposed method is presented, which allows to additionally consider phase information in the segmentation process. Finally, a summary and outlook are given in Section 5.

2. Radargram Segmentation Method

2.1. Overview of the Proposed Methodology

An introductory overview of the presented segmentation method is shown in Figure 3. First, the raw radargrams go through the same well-defined pre-processing steps routinely applied in GPR signal analysis (e.g., gain correction, filtering, signal whitening, start signal correction, etc.) to ensure comparability between different recordings. The results are then available in the form of an 8- or 16-bit grayscale image. Then, a multiscale 2D Gabor filtering is applied, which is described in detail in the following Section 2.2. A subsequent dimensionality reduction using PCA finally reduces the high-dimensional data representation to three characteristic feature dimensions (see Section 2.3). With this reduced representation (based on the features “amplitude”, “frequency”, and “local scattering”, as will be shown in Section 2.3), certain texture properties can then be precisely defined in a simple and objective manner. A corresponding pixel-based classification is then performed, which extracts exactly those regions in the radargram that match this texture definition. The image areas segmented in this way can then be further analyzed by means of image processing (e.g., edge detection).

2.2. Radar Image Texture Analysis Based on 2D Gabor Filters

The typical image content of railway radargrams is mainly characterized by local wave-shaped 2D structures with different spatial frequencies, orientations, and amplitudes. This motivates the use of so-called 2D Gabor filters (Figure 4), which are complex functions typically used for texture analysis in the field of digital image processing and which also can represent different spatial frequencies (and orientations) to quantitatively describe local image structures (textures).

Gabor filtering is a widely applied technique for texture analysis having optimal localization properties in both spatial and frequency domain, and they have been successfully used in many image analysis applications [20]. Frequency (or scale) and orientation are two key parameters of the Gabor filter, which detects the presence of a given spatial frequency content in an image in a given direction around a local pixel neighborhood, (i.e., Gabor filters compute one filtered value at each pixel position taking its spatial neighborhood into account). Additionally, they can be tuned to different scales and orientations to create a so-called filter bank (Figure 5).

For a radargram image $\mathrm{I}\left(x,y\right)$ with size $p\times q$, its discrete Gabor transform is given by a convolution [21]:

$${\mathrm{G}}_{mn}\left(x,y\right)={\displaystyle \sum }_{s=0}^K{\displaystyle \sum }_{t=0}^K\mathrm{I}\left(x-s,y-t\right){\mathrm{g}}_{mn}^{*}\left(s,t\right)$$

(1)

where $K$ is the filter mask size and ${\mathrm{g}}_{mn}^{*}$ is the complex conjugate of ${\mathrm{g}}_{mn}$, which is a class of self-similar wavelets generated from dilation and rotation of the following mother wavelet:

$$\mathrm{g}\left(\mathit{x},\mathit{y}\right)=\frac{1}{2\mathsf{\pi }{\sigma }_x{\sigma }_y}\exp \left(-\frac{1}{2}\left(\frac{x^2}{{\sigma }_{\mathit{x}}^2}+\frac{y^2}{{\sigma }_y^2}\right)\right)·\exp \left(\mathrm{j}2\mathsf{\pi }fx\right)$$

(2)

where $f$ is the modulation frequency and ${\sigma }_x,{\sigma }_y$ represent the standard deviations in the x and y directions. The self-similar Gabor wavelets are obtained through the generating function

$${\mathrm{g}}_{mn}\left(\mathit{x},\mathit{y}\right)={\mathrm{a}}^{-m}\mathrm{g}\left(\tilde{x},\tilde{y}\right)$$

(3)

where m and n specify the scale and orientation of the wavelet, respectively, with m = 0, 1, …, M − 1, and n = 0, 1, …, N − 1, and

$$\tilde{x}={\mathrm{a}}^{-m}\left(x\cos \theta +y\sin \theta \right)$$

(4)

$$\tilde{y}={\mathrm{a}}^{-m}\left(-x\sin \theta +y\cos \theta \right)$$

(5)

where a > 1 and θ = nπ/N.

After applying Gabor filters with different orientations n and different scales m, a set of magnitudes E at each pixel position is obtained:

$$\mathrm{E}\left(x,y,m\right)={{\displaystyle \sum }}_n\left|{\mathrm{G}}_{\mathit{mn}}(x,y)\right|$$

(6)

where m is the wavelet scale and n is the wavelet orientation.

Usually, the sum is calculated over all orientations n (from 0° to 180°) to finally have m different filter response maps (according to the number of scales) representing different frequency content, i.e., the generated magnitude maps pixel-wise represent the spatial signal energy distribution at every scale. For more specific applications, the number of angles could also be selectively adapted.

Figure 6 exemplarily shows some of the filtered results and how different image features are captured by the Gabor filters. Those image areas that structurally correspond to the Gabor filter in use will show accordingly high filter responses, i.e., high-frequency information is captured at lower scales and as scale increases, less detailed low-frequency signal content is emphasized (so, scale and spatial frequency are inversely related). As mentioned above, for a number m of different filter scales, the corresponding m filter responses are determined for each pixel position. As an example, the first three resolution levels of a radargram are shown in Figure 6b–d. Each pixel in the radargram can then be represented by a m-dimensional vector v_x,y or accordingly by a single data point in a m-dimensional feature space (Figure 6a). In general, similar local image structures (textures) in this feature space representation will form local clusters with a small Euclidean distance to each other—i.e., conversely, image structures with significantly different appearances (in terms of frequency and amplitude) will then have a feature space representation at a correspondingly larger distance and thus be easily separable from each other.

2.3. PCA of Gabor Filtered Radargrams

Subsequently, the dimensionality of this feature space representation is reduced using PCA (also known as discrete Karhunen–Loève transform) with the goal of segmenting the radargram image into different texture classes, each with a visually characteristic “appearance”. PCA represents a well-established mathematical procedure with the help of which the dimension of a feature space can be reduced without significant loss of information [22]. This means that highly correlated signals/attributes have only a low information content and the benefit, for example, a classification task is lower than that of attributes with low correlation. The goal of PCA is to de-correlate the attributes by an orthogonal basis transformation. In this way, the underlying coordinate system is rotated in the direction of the main axes, which run in the direction of the largest variance of the data points. The working principle of PCA is to look for the projection with the smallest mean-squared distance between the original data and their projections on to the new (rotated) basis system, which turns out to be equivalent to maximizing the variance of the data points. Principal components refer to the new variables constructed as a linear combination of the initial features (i.e., Gabor filter responses M in our case), such that these new variables are uncorrelated. Since the principal components are linearly independent, they are orthogonal to each other in a Cartesian coordinate system.

Based on the Gabor feature set matrix F (with size p⋅q × m, where p⋅q is the number of total pixels in the radargram and m the number of Gabor filter scales, Equation (7)), PCA returns the m principal component coefficients (representing the directions of the axes with most variance):

$$\mathrm{F}=\left(\begin{array}{ccc}{\mathrm{M}}_{1,1}& \cdots & {\mathrm{M}}_{1,m}\\ ⋮& \ddots & ⋮\\ {\mathrm{M}}_{\mathit{pq},1}& \cdots & {\mathrm{M}}_{\mathit{pq},m}\end{array}\right)$$

(7)

The actual dimensionality reduction is performed after this transformation by selecting only those principal components that meet a certain minimum variance requirement (i.e., that can explain a large amount of the total data variance). PCA compresses as much information as possible into the first principal component, the remaining into the second, and so on. Our texture analysis experiments have shown that already the first three principal components typically can explain about 97% of the signal variance in our radargrams (representing second order statistics).

Usually, these new variables do not have an interpretable meaning, being a linear combination of abstract features. In connection with the analysis of radargrams, however, these first three principal component axes can indeed be assigned a specific (visually interpretable) meaning. In this way, the first component describes the amplitude of the signal, the second the frequency, and the third is referred to as the “local scattering” property. While the first two properties have an unambiguous interpretation, the third one needs to be explained in more detail. An example is shown in Figure 7, where the top image shows coherent low-frequency radargram structures without “local scattering” and the example below shows structures with the same frequency content, but with superimposed high-frequency (most likely local wave scattering induced) noise-like patterns.

After projection of the original feature space on to the new 3D coordinate system (with the according basis axes representing amplitude, frequency, and scattering), the distribution of all radargram pixels can be visualized in the form of a 3D scatterplot (Figure 8a). In addition, the corresponding 2D projections are shown in Figure 8b–d. Specific parts of this 3D feature space now represent distinct radar signal properties, which in turn can be assigned, e.g., to characteristic radargram features (under the supervision of experts with experience in the interpretation of track radar signals). By simply defining a particular “region” in the 3D feature space (i.e., characterizing it in terms of amplitude, frequency, and scattering), the corresponding pixels (i.e., spatial position) in the radargram can be identified accordingly. This allows the expert the individual (and now objectifiable) definition of any number of specific classes in the 3D feature space (scatterplot coordinates), which, according to individual expertise, represent important subsurface structures. The black dots in Figure 8 exemplarily indicate the position of possible cluster centers that can be used for classification (i.e., the Euclidean distance to these centers determines the corresponding classification/coloring of the data points). The according visualization of these datapoints in the radargram results in a pixel-wise segmented image that automatically indicates characteristic rail track substructures, where the color indicates the corresponding class.

3. Automatic Radargram Segmentation Results

Typical results of the segmentation procedure are presented in this chapter. The examples shown are based on radargrams that were recorded with a 400 MHz GPR system at measurement speeds of approx. 100 km/h with sampling rates of 1024 samples/trace. If, for example, the task is to identify very low-frequency content ranging from specific medium- to high-amplitude values (which typically represents the so-called track planum, the boundary between the ballast and sub-ballast layer), simply the lower right part of the 3D feature space in Figure 8a must be selected and the corresponding pixels (image areas) can immediately be displayed in the segmented radargram (exemplarily indicated as blue/purple regions in Figure 9b). These dominant structures can also be recognized quite well visually in the raw radargram (Figure 9a).

High-frequency image structures with pronounced local scattering, on the other hand, typically indicates ballast fouling (i.e., when the finer materials mix with fresh ballasts due to heavy repeated train loads), which is shown as yellow/red areas in the segmentation result (Figure 9b). These structures are much less pronounced and more difficult to detect visually in the original radargram.

Another possible representation is shown in Figure 10, where five automatically extracted image segments of an examined railroad section are visualized separately. Each segment is characterized by specific signal frequency and amplitude ranges. This allows further investigations to be carried out, such as determining the signal energy or measuring the area (or “thickness”) of these specifically characterized texture segments, in order to automatically monitor the exceeding of critical threshold values (e.g., the amount of locally collected water typically represented by low-frequency and very high-amplitude regions without local scattering contributions).

Figure 11a exemplifies another advantage of this special form of automatic radargram segmentation. Due to the complex image structure in the raw images, it is difficult even for the trained human eye to determine the exact course and delineation of different relevant textures. The proposed automatic segmentation offers advantages here since, e.g., the exact course of the layer boundaries is now based on comprehensible criteria and no longer on a purely visual (subjective) assessment, which could possibly lead to different results for different assessors. Various further types of possible automated evaluations are shown in Figure 11b–d, where, e.g., upper and lower formation boundaries have been extracted automatically. Subsequently, the exact course of these contours is the basis for further evaluations, for example to determine any undulations in the boundary layers or to determine the exact depth and thickness of substructure layers. In this way, potential problem areas can be identified very quickly in an automated way, which provide useful indications for track maintenance.

4. Phase Measurements at Low-Frequency Layer Interfaces

The segmentation process shown above can also be extended to include signal phase evaluation. Generally, the phase of the reflected GPR signal is influenced by the wave impedance. It is negative when the signal is split at an interface where the impedance decreases (i.e., the relative permittivity increases). Conversely, the phase is positive when the signals are split at an interface where the velocities and wave impedances increase (i.e., the relative permittivity decreases). Thus, if the relative permittivity of a target is lower than that of the surrounding medium (e.g., air-filled void or plastic pipe), there is no phase reversal of the backscattered GPR wave—on the contrary, e.g., especially for water, a phase reversal is produced. In this way, additional information about the boundary layer conditions (e.g., water content) or about possibly buried objects (especially of metallic ones) could be derived.

Based on the segmentation process described in this paper, phase shifts in extracted very low-frequency segments of the radargram can be determined. Signal traces ${\mathrm{f}}^{\mathrm{trace}}$ with a specific phase angle are identified by applying a 1D cross-correlation (CC) with Ricker wavelets ${\mathrm{g}}_{\theta }^{\mathrm{Ricker}}$ of tunable phase from −180° to 170° in steps of 5° (Equation (8), Figure 12). The Ricker wavelet (also referred to as Mexican Hat wavelet) is used here because of its close similarity to the measured radar reflection signal. In other words, cross-correlation is used to determine that Ricker wavelet that best matches the shape of the current signal trace:

$$\mathrm{CC}(\theta )=\arg \underset{\theta }{\max }(({\mathrm{f}}^{\mathrm{trace}}{\ast \mathrm{g}}_{\theta }^{\mathrm{Ricker}})(\tau ))=\arg \underset{\theta }{\max }({\displaystyle \int }\mathrm{f}(t){\mathrm{g}}_{\theta }(t+\tau )\mathit{dt})$$

(8)

This additional signal phase information can be used for advanced classification and segmentation tasks where the identification of phase angle changes might be a valuable additional feature (e.g., for water content analysis or detection of buried objects). Figure 13a shows exemplarily an extracted low-frequency segment. Then a further subdivision of this segment is performed by defining a phase angle threshold as a distinguishing feature: those traces with a phase angle equal to approx. 0° are shown in the middle image, (Figure 13b) and those with a phase angle equal to approx. 180° are shown in the lower image (Figure 13c). As can also be perceived visually, the middle image consists of signals with a phase angle of approx. 0°, i.e., the signal maximum (bright) is clearly recognizable here in the central area of the image segment. In the lower image, on the other hand, the central peak area is continuously pronounced as a signal minimum (dark) indicating phase angles of 180°.

5. Summary and Conclusions

The presented form of automatic texture segmentation based on 2D Gabor filtering contributes to reducing the complexity of interpreting and quantifying radargrams in rail track condition assessment. Based on the presented approach, it is immediately recognizable whether, for example, anomalous characteristics of radargram structures are present. The algorithm identifies those image areas that appear similar in terms of spatial frequency, amplitude, and “local scattering”—the individual interpretation of these characteristic image structures still must be carried out by experts. However, it can support the evaluation process and significantly increase the reproducibility of results, since the same objective analysis criteria are applied to assess characteristic image structures which can then be analyzed in detail, e.g., to support the difficult process of determining critical track conditions such as ballast fouling or clay fouling. This process is accompanied by automated image measurements, such as the identification of different layer structures and the corresponding measurement of layer thicknesses and layer depths. Even the extraction of signal phase information at layer boundaries becomes feasible. In this way, typical radargram sections covering areas of several kilometers of track can be analyzed on modern computers within a few seconds. The radargram analysis presented is already routinely used in track assessment by Groundcontrol GmbH as a supporting method, and the degree of automation has already been significantly improved in this way.

The possibilities of pixel-precise segmentation will be further explored in future activities. This includes, for example, the detailed comparison with data from bore holes or trial pits. Future work should additionally show whether the segmentation proposed here can also be used for automated labeling in conjunction with modern deep learning methods (i.e., to efficiently generate learning data for supervised learning algorithms in this way), which could then in turn be used to further improve the automated interpretation of radargrams. The presented approach thus represents a further step towards advancing the reproducibility and objectifiability of GPR analysis of track substructures and thus increasing the safety of railways. However, the presented approach can presumably also be transferred to other areas of application in which automated radar signal evaluation is of importance.