OEM-HWNet: A Prior Knowledge-Guided Network for Pavement Interlayer Distress Detection Based on Computer Vision Using GPR

Abstract

Accurate detection of interlayer distress based on ground-penetrating radar has been widely adopted for in-service asphalt pavement condition assessment to improve maintenance efficiency and reduce costs. However, accurate interlayer distress locating is challenging with limited adaptability to their large-scale variations, which significantly weakens the detection performance. This study proposed a novel automatic detection network based on YOLOv5s to detect interlayer distresses in asphalt pavement named OEM-HWNet. Firstly, an object enhancement module based on prior knowledge was designed to locate the regions of interlayer distress and enhance their characteristics. Then, wavelet convolution was added to increase the receptive field of the network and enhance the ability to capture low-frequency information. Finally, an additional detection head was added to improve the detection capability of interlayer distress with different sizes. Experiments demonstrated that the proposed network achieves a mean average precision (mAP) of 89.6%, outperforming other advanced models, such as YOLOv5s, YOLOv8s, YOLOv11s, and Faster R-CNN. Incorporating prior knowledge into deep learning networks could provide an effective solution to detect interlayer distress of asphalt pavement.

1. Introduction

Due to the influence of load, temperature, water, and other factors, various hidden distresses will occur during the operation of roads, which seriously affect their service life, such as interlayer distresses, concealed cracks, water-rich voids, loosening, etc. [1]. Therefore, efficient and accurate detection of hidden distress has become one of the focuses of road maintenance. Since ground penetrating radar (GPR) has advantages of high accuracy, high efficiency, low cost, and non-destructive testing, it has been widely used in hidden distress detection of roads [2,3]. Although GPR data can reflect the internal structure distribution of roads, it requires complex processing and analysis. At present, manual processing and interpretation are mainly used in practical projects, which are highly subjective and severely limit the efficiency of distress detection. Thus, to achieve efficient detection of road hidden distresses, the automation of GPR data analysis is crucial.

Many automated detection methods of hidden distress were presented from single-channel waveform (A-scan) [4,5,6], two-dimensional profile (B-scan) [7,8,9], and three-dimensional volume (C-scan) [10,11,12,13,14], respectively, according to the type of GPR data. A-scan signals represent only local information and are susceptible to noise, while C-scan includes a huge amount of data, requiring a powerful computing platform for analysis. In addition, the annotation of three-dimensional data is particularly difficult. As a main means, the automatic analysis methods based on B-scan images have been extensively applied in detecting hidden distress of roads [15], such as interlayer distresses, cracks, voids, and loosening. These hidden distresses may be classified into hyperbolic [16,17,18,19,20] (e.g., cracks and voids) and non-hyperbolic [21,22,23] (e.g., interlayer distresses and loosening). The above work has achieved good results, but most of them focus on hyperbolic distresses in the GPR B-Scan images, while the research on interlayer distresses is scarce.

In fact, as one of the early hidden distresses of pavements [24], interlayer distresses have been a priority for pavement maintenance [25]. In particular, most of the hidden distresses detected in Jiangsu province of China are interlayer distresses [26]. Therefore, it is necessary to detect interlayer distresses in a timely manner to prevent further development of the distresses. In addition, there are also differences in interlayer distress, and this difference will affect pavement maintenance options and costs. So, it needs not only to accurately detect interlayer distresses but also to further classify interlayer distresses as interlayer debonding and interlayer water seepage. For interlayer debonding, the maintenance program only needs to consider grouting, while both underground grouting and drainage system repair are carried out for interlayer water seepage as a more serious interlayer distress. Recently, there have been some works on classifying interlayer distresses in asphalt pavement [26,27], which is insufficient for practical engineering. Detection of interlayer distress requires not only classification but also precise localization. However, accurately identifying and locating different types of interlayer distress often faces the following difficulties. Firstly, as shown in Figure 1, the shape and texture of interlayer debonding and interlayer water seepage are very similar in GPR images, and these two types of interlayer distresses may also be similar to other distresses. Secondly, the size of interlayer distresses varies greatly, which demands that the network adapts to the various sizes, and the existing object detection methods are insufficient to deal with this problem. Thirdly, many object detection methods are limited in global feature extraction and low-frequency information capture.

Existing methods for identifying subsurface road distress based on GPR B-scan images can be broadly categorized into the rule-based recognition algorithm, machine learning algorithm, and deep learning algorithm [15]. For rule-based recognition algorithms, Bugarinovic et al. introduced the Canny edge detection operator to remove unnecessary edge pixels and horizontal reflections of the road surface and soil layer to improve the detection speed and positioning robustness of the hyperbola [28]. Similarly, Harkat and Benani used a modified Hough transform method to detect hyperbolic features caused by buried cavities [29]. Although rule-based algorithms employ expert knowledge to extract features from GPR images, these empirically derived features often fail to represent the intrinsic attributes for identifying interlayer distress. Building on rule-based algorithms, machine learning algorithms establish the relationship between features extracted and hidden distresses by training classifiers such as support vector machines (SVM) [30] and K-nearest Neighbors (KNN) [31]. Also, Sun et al. [32] utilized histograms of oriented gradients (HOG) to extract crack features, followed by an SVM classifier to identify cracks. Although machine learning algorithms offer multi-dimensional feature representation to significantly improve detection accuracy and applicability compared to rule-based models [33], they fail to recognize the interlayer distress with different sizes.

In recent years, the rapid advancement of deep learning has led to its extensive application in the field of object detection [34]. These object detection methods based on deep learning may be divided into two main categories. One is two-stage methods, which rely on candidate bounding boxes, such as Faster R-CNN [35] and Cascade R-CNN [36]. The other is one-stage methods based on regression, including the YOLO series [37,38], SSD [39], DETR [40], and RetinaNet [41]. While two-stage methods often achieve higher accuracy, one-stage methods are favored for their real-time detection capabilities. Furtherly, the YOLO series, as a mainstream one-stage model, has been extensively applied in detecting various distresses in GPR B-scan images [21,22,23]. However, these methods often cannot provide accurate location information for interlayer distress and demonstrate limited adaptability to their large-scale variations.

In general, the structure of the network needs to be modified for specific scenarios. For example, many researchers have modified the structure of the YOLO series network to improve the detection accuracy of their own datasets [42,43,44,45,46]. These improvement strategies have yielded positive results. However, they all ignore the benefits of prior knowledge. In fact, the introduction of prior knowledge in the design of deep networks or learning strategies can improve detection accuracy [47], as they render a knowledge-infused algorithm more suitable to the specific scenarios, lowering the training dataset’s requirement. Moreover, it also enhances the interpretability of the network. Until now, some scholars have still been applying prior knowledge to GPR image recognition [48,49]. In this paper, shape and texture features as prior knowledge are of importance to identify interlayer distresses of asphalt pavement (see Figure 1), but the effective incorporation of specific prior knowledge to guide network architecture design remains a non-trivial challenge.

In this study, an improved YOLOv5s model was proposed to achieve interlayer distress detection of asphalt pavement with GPR images, named OEM-HWNet. In the method, shape and texture features of interlayer distress are used as prior knowledge to enhance object regions during the network learning. Firstly, by considering the requirements of real-time detection with few model parameters and high accuracy, YOLOv5s was selected as the baseline model [50]. Secondly, an object enhancement module (OEM) based on the Canny operator and morphological dilation was designed to enhance the representational ability of interlayer distresses. Thirdly, WTConv (the convolution in the wavelet domain) [51] was introduced to increase the capturing ability of low-frequency information and the receptive field. Finally, an additional detection head is incorporated to improve the detection performance of large objects. The OEM-HWNet effectively merges prior knowledge and automatic features. Figure 2 illustrates the workflow of interlayer distress detection. The primary contributions of this paper are as follows:

(1)

This paper proposes a prior knowledge-guided network for interlayer distress detection based on GPR images, named OEM-HWNet, and demonstrates its effectiveness in comparison with the state-of-the-art algorithms such as Faster R-CNN, SSD, RetinaNet, RT-DETR [52], YOLOv3 [38], YOLOv5s, YOLOv7 [37], YOLOv8s [53], and YOLOv11s [54];

(2)

The OEM, based on prior knowledge, is designed and integrated into the YOLOv5s model, which fully leverages edge position information to improve localization and feature representation abilities of the model. This module not only improves the performance of the model but also increases its interpretability of the model;

(3)

WTConv is introduced into the YOLOv5s model. This approach effectively mitigates the limitations of a convolutional neural network (CNN) in global feature extraction and enhances the model’s ability to capture low-frequency features;

(4)

The number of detection heads for YOLOv5s was extended from three (small, medium, and large) to four (small, medium, large, and huge), which effectively addresses the issue of big variations in sizes of the interlayer distresses and enhances the detection ability of large objects.

The remainder of this paper is described as follows. Section 2 introduces the proposed OEM-HWNet model and the details of module design. Section 3 describes dataset construction and the experimental details. Section 4 analyzes the experimental results. Section 5 summarizes the conclusions of the study.

2. Methodology

To clearly describe the detection model developed in this paper, this section firstly introduces the OEM-HWNet model and its main components. Then, the design of the OEM modules based on prior knowledge is described in detail. Finally, the C3WC module (WTConv embedded in the C3 module) is elaborated.

2.1. Overview of the OEM-HWNet Architecture

YOLOv5 is selected as our baseline model since it has fewer model parameters compared with the latest YOLOv11 [54] and high accuracy. As shown in Figure 3, OEM-HWNet is primarily divided into three parts: the backbone, neck, and head parts.

First, the backbone part comprises the CBS module, C3WC module, and Spatial Pyramid Pooling Fast (SPPF) module. The CBS module includes a standard convolutional layer (Conv2d), a batch normalization (BN) layer, and the SiLU activation function. The original C3 module (three convolutional layers and several Bottleneck modules) is replaced by the C3WC module, integrating the WTConv into the Bottleneck to obtain a larger receptive field and improve the ability to capture low-frequency features. The SPPF module is an optimized version of spatial pyramid pooling (SPP) [55]. Second, the neck part combines Feature Pyramid Networks (FPN) [56] and Path Aggregation Network (PANet) [57], which realize the fusion of semantic information from low level to high level. Notably, a specially designed module called OEM is inserted between the neck and the backbone part. This module focuses on learning of the interlayer distresses and inhibits interference, which significantly improves the learning ability and robustness of the model. Finally, the original YOLOv5s model includes three detection heads with sizes of 20 × 20, 40 × 40, and 80 × 80, respectively. However, these detection heads cannot fully meet the needs of distress detection in the case of interlayer distress with huge scale changes in size. To correct this, we added an additional detection head with a size of 10 × 10 to improve the model’s ability to detect large objects.

Loss function of the OEM-HWNet model consists of classification loss ($Los{s}_{cls}$), object confidence loss ($Los{s}_{obj}$), and box localization loss ($Los{s}_{box}$), which are expressed as follows:

$$Loss=Los{s}_{box}+Los{s}_{obj}+Los{s}_{cls}$$

(1)

The $Los{s}_{box}$ denotes the deviation between the truth box and predicted box. Here, the complete-IoU (CIoU) loss [58] is adopted in the localization of the bounding box, which is an improved object detection loss function. So, the $Los{s}_{box}$ is expressed as follows:

$$Los{s}_{box}=Los{s}_{CIOU}=1-IOU+{\displaystyle \frac{{\rho }^2(b,b^{gt})}{c^2}}+\alpha \upsilon$$

(2)

$$IOU=\left|{\displaystyle \frac{B\cap B^{gt}}{B\cup B^{gt}}}\right|$$

(3)

where $B$ and $B^{gt}$ denote the predicted and ground-truth box, respectively; $b$ and $b^{gt}$ denote the central points of $B$ and $B^{gt}$, $\rho \left(\cdot \right)$ is the Euclidean distance; and $c$ is the diagonal length of the smallest enclosing box covering the two boxes. The $\alpha$ is a positive trade-off parameter and $\upsilon$ measures the consistency of the aspect ratio, as follow:

$$\alpha ={\displaystyle \frac{\upsilon }{(1-IOU)+\upsilon }}$$

(4)

$$\upsilon ={\displaystyle \frac{4}{{\pi }^2}}{(arctan{\displaystyle \frac{w^{gt}}{h^{gt}}}-arctan{\displaystyle \frac{w}{h}})}^2$$

(5)

where $w$ and $w^{gt}$ represent the widths of predicted box and ground-truth box, respectively, while $h$ and $h^{gt}$ correspond to their heights. Binary cross-entropy loss [41] is employed to calculate the object confidence ($Los{s}_{obj}$) and classification loss ($Los{s}_{cls}$). The former considers both positive and negative samples, while the latter only uses positive ones. The $Los{s}_{obj}$ and $Los{s}_{cls}$ can be expressed as follows:

$$\begin{array}{c}Los{s}_{obj}={\lambda }_{obj}{\displaystyle \sum _{i=0}^{N^2}{\displaystyle \sum _{j=0}^M{I}_{ij}^{obj}[-{\widehat{C}}_{i}ln{C}_i-(1-{\widehat{C}}_i)ln(1-C_i)]}}\\ +{\lambda }_{nobj}{\displaystyle \sum _{i=0}^{N^2}{\displaystyle \sum _{j=0}^M{I}_{ij}^{nobj}[-{\widehat{C}}_{i}ln{C}_i-(1-{\widehat{C}}_i)ln(1-C_i)]}}\end{array}$$

(6)

$$Los{s}_{cls}={\displaystyle \sum _{i=0}^{N^2}{\displaystyle \sum _{j=0}^M{\displaystyle \sum _{c\subset cls}{I}_{ij}^{obj}}[-{\widehat{p}}_i(c)ln(p_i(c))-(1-{\widehat{p}}_i(c))ln(1-p_i(c))]}}$$

(7)

where $N^2$ stands for the number of grid cells; $M$ represents the number of predicted bounding boxes per grid cell; ${\lambda }_{obj}$ and ${\lambda }_{nobj}$ represent the weights of the grid cell with or without the objects, respectively; $I_{ij}^{obj}$ and $I_{ij}^{nobj}$ denote whether there is an object to be predicted in the j-th anchor box of the i-th grid cell, respectively; ${\widehat{C}}_i$ and $C_i$ are the confidence of the predicted and truth values, respectively; and ${\widehat{p}}_i(c)$, $p_i(c)$ are the predicted and truth probabilities of detecting an object belonging to interlayer distress type c in the i-th grid cell, respectively.

2.2. Prior Knowledge

2.2.1. Physical Basis

Figure 4a presents a schematic diagram of the GPR operation, where the antenna pair transmits an electromagnetic wave and receives the echo from the medium. In fact, the GPR obeying the wave equation [59] as

$${\nabla }^{2}E+\mu \sigma {\displaystyle \frac{𝜕E}{𝜕t}}+\mu \epsilon {\displaystyle \frac{{𝜕}^{2}E}{𝜕t^2}}=0$$

(8)

where $E$ is electric field strength vector, $\mu$ is magnetic permeability, $\sigma$ is conductivity, $\epsilon$ is permittivity, and $t$ is time.

Strong reflections occur when the electromagnetic wave propagates to the interlayer distresses (Figure 4b). In general, the reflection intensity is determined by the difference in the permittivity of the two mediums, namely

$$R={\displaystyle \frac{\sqrt{{\epsilon }_2}-\sqrt{{\epsilon }_1}}{\sqrt{{\epsilon }_2}+\sqrt{{\epsilon }_1}}}$$

(9)

where ${\epsilon }_1$ and ${\epsilon }_2$ are the permittivity of mediums in two adjacent layers. The greater the difference in permittivity, the stronger the reflection. When interlayer distress occurs, the permittivity of the distress is obviously different from that of the surrounding medium, so there is a strong reflected signal, as GPR B-scan images shown in Figure 4c. This principle is critical for non-destructive testing of interlayer distresses in asphalt pavement [60]. For example, interlayer debonding typically manifests as air filling between pavement layers, and the permittivity of air is 1, which is significantly different from the surrounding medium. This will result in a strong reflection, which is manifested in GPR images as the evidence for interlayer distress detection. In addition, higher-frequency antennas provide higher vertical resolution but limited penetration depth, whereas lower frequencies achieve deeper penetration at the cost of reduced resolution [59]. High-frequency antennas are essential to identify subtle changes in permittivity, while lower frequencies may fail to detect early-stage debonding. For the detection of interlayer distresses, the antenna frequency of GPR is typically set within the range of 700 MHz to 1000 MHz.

2.2.2. Object Region Segmentation

The shape and texture features are of importance to identify interlayer distresses in asphalt pavement, so these features extracted from GPR images as prior knowledge are used to segment interlayer distresses from the background. The Canny edge detection algorithm [61] utilizes dual thresholds (high and low thresholds) for object edge extraction and if any part of a contour is above the high threshold, those points are immediately output, as is the entire connected segment of contour, which contains the points and which lies above the low threshold. It has been proved to be effective in extracting the object edges [28]; however, these edges are one continuous line and cannot represent the object region. To separate the object region from the background, morphological dilation [62] is introduced as a complement to the Canny operator to connect neighboring lines to form the object region.

Specifically, the close distance between the upper and lower edges of the object region provides a clue for separating the object area from the background. The CannyMask module first uses the Canny operator to generate texture information and then uses morphological dilation to fill the gap between edges as

$$\mathrm{CannyMask}={\mathrm{Dilation}}_{(5,1)}({\mathrm{Canny}}_{(90,225)}(X))$$

(10)

where ${\mathrm{Dilation}}_{(5,1)}$ denotes the dilation operation with a kernel size of 5 × 5 and iterations of 1 to fill the gap, ${\mathrm{Canny}}_{(90,225)}$ represents the Canny operator with a low threshold of 90 and a high threshold of 225, $X$ is the input image, and $\mathrm{CannyMask}$ is the mask tensor representing object area. For the mask tensor, the pixel value of the object region is set to 255, and the background region is set to 0.

As shown in Figure 5, CannyMask can separate the object region from the background, but there are still limitations. First of all, the edge detection method based on prior knowledge cannot adapt to various complex backgrounds, which refer to non-target signals including measurement noise, normal pavement structure, and other distresses. For example, the crack shown in the yellow circle of sample 2 in Figure 5 belongs to other distresses, but this area with continuous strong gray-scale variations and hyperbolic characteristics may be prone to be confused with interlayer distresses. The regions identified by CannyMask encompass the interlayer distresses and other types of distresses, such as cracks. However, not only can CannyMask suppress most of the background interference but it also enhances the region features identified by CannyMask, which significantly contributes to improving the recognition performance of deep learning-based networks. In addition, although CannyMask can locate the object region well, it cannot further classify the object region as interlayer debonding or interlayer water seepage. It is worth noting that deep learning-based networks can solve the above classification problems, but inaccurate localization often occurs in object detection. Therefore, as an a priori feature module, CannyMask may be embedded into the deep learning network to improve the location ability of detection network.

2.3. Object Enhancement Module (OEM) Based on Prior Knowledge

An OEM based on the prior knowledge is designed as shown in Figure 3. The OEM has two inputs: the original image and the input features (the output of the previous module). Firstly, the object area of the original image is extracted through the MaxPool and subsequent CannyMask module (Figure 6a). Secondly, in order to ensure the same scale of input features, the CannyMask module is followed by the AP (AveragePool) module. Thirdly, the output of the AveragePool module is normalized, and the weight matrix is generated by the softmax function shown in Figure 6b. Finally, the output features are obtained by multiplying the weight matrix with the input features shown in Figure 6c. This module not only enhances the object region but also adds no additional parameters. The mathematical expression of this process is as follows:

$$H_n={\mathrm{AP}}^{n-1}(\mathrm{CM}[\mathrm{MP}(X)])$$

(11)

$$\mathrm{Ouput}=P_n\times Softmax(Normalization(H_n))$$

(12)

where $H_n$ represents the hand-crafted features based on the Canny operator and morphological dilation for all stages and $X$ is the original image. $\mathrm{AP}(\cdot )$, $\mathrm{CM}(\cdot )$, and $\mathrm{MP}(\cdot )$ perform AveragePool, CannyMask, and MaxPool, respectively. In Formula (12), $P_n$ represents the input features for all stages and $n$ is the stage number.

The OEM provides a new solution to some problems of the detection model, such as poor object positioning and unfocused learning. By incorporating prior knowledge, the model performance is significantly enhanced. A key advantage of this module is its parameter-free nature. Furthermore, the OEM helps to quickly locate the object region so that the learning of the model can be focused on that region, thus accelerating convergence.

2.4. C3WC Module

The size of the convolution kernel limits the global modeling capabilities of CNN. So, many researchers have attempted to increase the kernel size of CNN to mimic the global receptive field found in self-attention mechanisms. However, this method quickly hit an upper bound and saturated way before achieving a global receptive field. To address this challenge, Finder et al. proposed the WTConv based on wavelet transform, which excels at extracting features with different scales [51]. In this paper, the WTConv is integrated into the C3 module, named C3WC (Figure 7), to obtain a larger receptive field. As shown in the grid of Figure 8, a 1 × 1 convolution, which is performed on the low-frequency band of the second-level wavelet domain $X_{LL}^{(2)}$, is a response to lower frequencies of a 4 × 4 receptive field in the input $X$. The process is given by

$$Y=IWT(Conv(W,WT(X)))$$

(13)

where $X$ is the input tensor, and $W$ is the weight tensor of a k × k depth-wise kernel with four times as many input channels as $X$. In this study, we employ 5 × 5 kernel sizes for the convolutions.

Figure 8 illustrates the WTConv process with a 2-level wavelet transform. The initial step performs decomposition of the input signal into multi-scale wavelets. Subsequently, different frequency bands are processed using depth-wise convolution [63] with small convolution kernels, followed by image reconstruction through an inverse wavelet transform (IWT) to obtain the final output. The decomposition is given by

$$(X_{LL}^{(i)}, X_{HL}^{(i)}, X_{LH}^{(i)}, X_{HH}^{(i)})=\mathrm{WT}(X_{LL}^{(i-1)})$$

(14)

where $X_{LL}^{(0)}=X$ and $i$ is the current level. $X_{LL}^{(i)}$ is the low-frequency component of $X$, while $X_{LH}^{(i)}$, $X_{HL}^{(i)}$, and $X_{HH}^{(i)}$ are its horizontal, vertical, and diagonal high-frequency components. The Formulas (15) and (16) describe convolution and inverse wavelet transform (IWT), respectively.

$$(Y_{LL}^{(i)}, Y_{HL}^{(i)}, Y_{LH}^{(i)}, Y_{HH}^{(i)})=\mathrm{Conv}(W^{(i)}, (X_{LL}^{(i)}, X_{HL}^{(i)}, X_{LH}^{(i)}, X_{HH}^{(i)}\left)\right)$$

(15)

$$Z^{(i)}=\mathrm{IWT}(Y_{LL}^{(i)}+Z^{(i+1)},Y_{HL}^{(i)}, Y_{LH}^{(i)}, Y_{HH}^{(i)})$$

(16)

where $Z^{(i)}$ is the aggregated outputs from level $i$ onward. As observed in Figure 8, WTConv performs repetitive wavelet decomposition on the low-frequency band of the input image, continuously enhancing low-frequency features. WTConv has a superior capacity for capturing low-frequency information compared to standard convolution. Moreover, each level of WTConv increases the receptive field size while only marginally increasing the number of trainable parameters. This characteristic is a significant factor in integrating WTConv into the C3 module. In this study, we employ a 3-level wavelet decomposition to make a trade-off between accuracy and the number of parameters.

3. Validation Using Field Tests

3.1. Dataset

3.1.1. Data Acquisition

In interlayer distress detection of asphalt pavement within GPR images, no public dataset currently exists. To evaluate the performance of the proposed model, this study collected thousands of GPR images from several highways, including the Lianyungang-Xuzhou Highway, Huai’an-Xuzhou Highway, Yancheng–Jingjiang Highway, and Shanghai-Nanjing Highway, in Jiangsu province of China, as shown in Figure 9a.

Field data collection was conducted using the fourth-generation high-dynamic GPR, the MALA GX750 series (Figure 9b), produced by the Swedish company GuidelineGEO in Stockholm, Sweden. Its technical specifications are as follows: antenna frequency of 750 MHz, scanning speed of 1290 (traces/s), detection depth ranging from 0 to 1.5 m, air-coupling antenna mode, weight of 3.6 kg, and detection vehicle speed of 80 (km/h). After pre-processing, such as dewow, adjustment of start time, compensation for energy decay, background removal, deconvolution, and bandpass filtering, we obtained a total of 2105 GPR images (Figure 9c). Each image is with a width of 1005 pixels and a height of 210 pixels, corresponding to the length of 20 m and the time window of 20 ns, respectively.

3.1.2. Dataset Construction

Through the analysis of the above GPR images, we find that the main hidden distresses of asphalt pavement are interlayer distresses, so interlayer distresses become the research object of this paper. Further analysis of interlayer distresses shows that although the shape characteristics of interlayer distresses at different positions are similar, there are differences in vertical texture characteristics, as shown in Figure 10. By comparing and analyzing the image and the corresponding drilled sample core, combined with the experience of experts in the field, the interlayer distresses can be classified as interlayer debonding (Figure 10a) and interlayer water seepage (Figure 10b). This accurate classification of interlayer distresses is of importance to improve maintenance efficiency and reduce maintenance costs. For interlayer debonding, the pressure grouting is adopted to fill the gap between the layers of pavement [64], while interlayer water seepage is a more serious issue, requiring both pressure grouting and drainage system repair.

Figure 10 shows the GPR images and core samples of two categories of interlayer distress. In Figure 10, the red square represents the location of the distress, “a” indicates the interface of the surface layer and the upper base layer, “b” denotes the interface of the upper base layer and the lower base layer, “c” indicates the interface of the lower base layer and the subbase layer, and “d” denotes the coring position.

The characteristics of interlayer debonding and interlayer water seepage are described as follows:

• Interlayer debonding: temperature, load, or other factors may cause a continuous separation between the layers of pavement, and interlayer delamination occurs, named interlayer debonding. There is a strong reflection in the GPR image due to the difference in permittivity of the air contained between the layers and the surrounding medium. Specifically, the image features of interlayer debonding appear as black, white, and black in vertical order (Figure 10a). Interlayer debonding is abbreviated as “poor_1”;
• Interlayer water seepage: when interlayer debonding has occurred without timely maintenance, interlayer water seepage distress between the layers occurs as rainwater seeps in. Unlike other air-containing distresses, this category of distress contains water. The permittivity of water and air are different, typically 81 for water and 1 for air. As a result, Figure 10b shows a clear polarity reversal for interlayer water seepage compared to interlayer debonding. Specifically, the image features of interlayer water seepage appear as white, black, and white in vertical order. Interlayer water seepage is abbreviated as “water_1”.

We used LabelImg 1.8.6 software to annotate the 2105 GPR images. An annotated example is shown in Figure 11, where the red box indicates interlayer debonding and blue indicates interlayer water seepage.

Table 1. Basic situation of various interlayer distresses.

Distress Category	Total	Width (Pixel)			Height (Pixel)
		Max	Min	Average	Max	Min	Average
poor_l	2751	648	22	80.0	40	14	19.3
water_l	2862	574	24	82.5	27	19	20.3

summarizes the basic situation of the interlayer distresses, including the total number of samples and scale statistics of interlayer debonding, as well as interlayer water seepage. The average size indicates that the bounding boxes for interlayer debonding and interlayer water seepage are similar in size. In the field of object detection, the annotated areas less than 10% of an image belong to small objects [65]. However, the average area of interlayer distresses accounted for only 8% of the GPR images, which poses a challenge for the identification of interlayer debonding and interlayer water seepage. The maximum and minimum width values reveal that the size variability of the interlayer distresses predominantly occurs in width, with a difference of approximately 25 times. The big variation in size further increases the difficulty of detection. For all experiments conducted in this study, the dataset was partitioned into training, validation, and test sets in a ratio of 8:1:1.

3.2. Evaluation Metrics and Experimental Configuration

To accurately and fairly evaluate the performance of the OEM-HWNet, we employed several metrics, including precision ($P$), recall ($R$), and $F1$ score. The computations of these indicators are presented below, as follows:

$$P={\displaystyle \frac{TP}{TP+FP}}$$

(17)

$$R={\displaystyle \frac{TP}{TP+FN}}$$

(18)

$$F1={\displaystyle \frac{2\ast P\ast R}{P+R}}$$

(19)

where $TP$ is the true positive, $FP$ is the false positive, $FN$ is the false negative, and $TN$ is the true negative. Mean average precision ($mAP$) is typically employed for the evaluation of multi-distress detection systems. In this context, it represents the average precision for interlayer distress detection. The precision–recall ($P$–$R$) curve is illustrated with recall on the horizontal axis and precision on the vertical axis. Average Precision (AP) denotes the area under the $P$–$R$ curve, calculated at various thresholds for a specific distress category. The metrics $mAP@0.5$ can be expressed as follows:

$$mAP@0.5={\displaystyle \sum _{i=0}^n{\displaystyle {\int }_0^{1}PRdR}}\left(IO{U}_{thresh}=0.5\right)$$

(20)

In this study, the proposed interlayer distress detection model of asphalt pavement was performed on a server with an Intel(R) Xeon(R) Gold 6133 CPU and a single NVIDIA GeForce RTX 3090 GPU. The software environment configuration of the model is shown in

Table 2. Software environment.

Name	Version
Pytorch	2.1.0
CUDA	12.1
Python	3.8.18
Numpy	1.23.5

. The batch size and epoch for all experiments were set to 16 and 200, respectively. The initial learning rate was 0.01, and the learning rate decay followed a cosine schedule. The SGD [66] algorithm was selected as the optimizer with the momentum of 0.937 and decay weight of 0.0005.

4. Experimental Results

4.1. Preliminary Analysis of OEM-HWNet

4.1.1. Detection Results of OEM-HWNet

Confusion matrix is a common form of presenting and analyzing test results. The training and testing were carried out on the established GPR dataset, and the test results obtained were shown in Figure 12. The values in each cell represent the ratio of the predicted number and the true number of samples for each category. The “background” in the horizontal axis represents the situation in which the background is misjudged as the distress, whereas the “background” in the vertical axis represents the situation in which the distress is misjudged as the background. The analysis of Figure 12 is as follows. First, the main diagonal of the confusion matrix indicates that the OEM-HWNet model can identify the interlayer distresses with high accuracy, in which the scores of interlayer debonding and interlayer water seepage are 0.89 and 0.94, respectively. Then, the scores of interlayer debonding and interlayer water seepage from the “background” in the vertical axis are 0.08 and 0.05, respectively; it means that a small number of distresses are not recognized. In addition, the scores of interlayer debonding and interlayer water seepage from the “background” in the horizontal axis not only show that the interlayer distress and other distress in the background have similar characteristics but also the characteristics of interlayer debonding and interlayer water seepage are also very similar. Although interlayer disease detection is a difficult task, the OEM-HWNet still demonstrates excellent performance. To further demonstrate the detection performance of the proposed network, Figure 13 shows a comparison between the detection results and the ground-truth labels. From Figure 13, we can see that the detection results are almost consistent with the ground-truth label. This indicates that the OEM-HWNet performed well in accurately locating and identifying interlayer distresses.

4.1.2. Visualization of Attention Maps

To clearly understand the decision-making process of the OEM-HWNet network in recognizing interlayer distress, Grad-CAM [67] is employed to visualize the attention maps within the network architecture. Figure 14 presents the visualized attention maps from the output of the final layer. Notably, red indicates the areas that the network focuses on during the decision-making process. In contrast, blue indicates the less important areas. It can be observed that the red areas and the ground-truth labels are almost identical in position. This indicates that the prior knowledge-guided network can accurately locate and focus learning on interlayer distress. Although there are a few red or yellow areas appearing in the background, a clear distinction can be seen between interlayer distress and the background.

4.2. Ablation Experiments

Relative to the YOLOv5s model, the improved parts of the OEM-HWNet model include the OEM, the fourth head, and the C3WC module. To demonstrate the effectiveness of each module and combination of modules, ablation experiments were performed. The evaluation indices used in ablation experiments include average precision (AP), mean average precision (mAP) at a threshold of 0.5, parameter count (parameters), precision (P), recall (R), and F1.

The experimental results are presented in

Table 3. Ablation studies for different modules on the test set.

Model	AP		mAP@0.5	Parameters	P	R	F1
	Poor_l	Water_l		(M)
YOLOv5s (baseline)	84.0%	88.5%	86.3%	7.24	77.1%	82.3%	0.7962
+Fourth Head	84.9%	89.2%	87.0%	12.63	80.2%	79.8%	0.8000
+OEM	85.0%	89.5%	87.3%	7.24	81.3%	79.4%	0.8034
+C3WC	85.8%	89.7%	87.8%	7.70	76.2%	84.4%	0.8009
+Fourth Head + OEM	86.7%	90.2%	88.5%	12.63	81.3%	79.5%	0.8039
+Fourth Head + C3WC	85.5%	90.6%	88.1%	13.29	79.5%	81.8%	0.8063
+OEM + C3WC	87.6%	89.9%	88.8%	7.70	80.5%	80.5%	0.8050
+Fourth Head + OEM+ C3WC	87.8%	91.5%	89.6%	13.29	80.7%	82.4%	0.8154

. The analysis of Table 3 is described as follows. Firstly, adding a detection head (Fourth Head), introducing the OEM module, and incorporating the C3WC module obtains the improvements of 0.7%, 1.0%, and 1.5% in mAP, respectively. Secondly, the combination of the OEM and Fourth Head resulted in a 2.2% increase in mAP, while the joint use of the OEM and C3WC brought a mAP improvement of 2.5%. More importantly, the introduction of the OEM module does not add additional training load because of its parameter-free nature. Finally, the coupling of the three blocks resulted in a 3.3% increase in mAP. The experimental results verify that the proposed OEM-HWNet is a very competitive solution for the accurate detection of interlayer distresses.

The $P$–$R$ curves provide an intuitive representation of model performance. Figure 15 compares the $P$–$R$ curves of the different models within the dataset of this study. The larger the area under the $P$–$R$ curve, the better the performance of the model. Compared with YOLOv5s, the area under the $P$–$R$ curve of all recommended improved models has been increased, among which the OEM-HWNet model has the largest area.

4.3. Comparison with the State-of-the-Art Methods

In order to verify the effectiveness of the proposed OEM-HWNet model, the current state-of-the-art object detection models, such as Faster R-CNN, RetinaNet, SSD, RT-DETR, YOLOv3, YOLOv5, YOLOv7, YOLOv8, and YOLOv11, are chosen to compare with our model. To ensure a fair comparison, a consistent operating environment and datasets are used.

Table 4. Quantitative comparison with the state-of-the-art object detection methods.

Model	Average Precision		mAP@0.5	Size (MB)	FPS
	Poor_l	Water_l
Faster R-CNN [35]	84.4%	87.9%	86.1%	330.35	13
RetinaNet [41]	79.6%	84.2%	81.9%	257.26	15
SSD [39]	77.6%	85.1%	81.4%	106.11	42
RT-DETR [52]	84.9%	88.5%	86.7%	82.08	80
YOLOv3 [38]	85.0%	88.4%	86.7%	117.69	71
YOLOv5s [50]	84.0%	88.5%	86.3%	13.63	160
YOLOv7 [37]	83.7%	90.0%	86.9%	71.30	110
YOLOv8s [53]	83.5%	89.9%	86.7%	21.45	145
YOLOv11s [54]	82.9%	88.3%	85.6%	18.27	149
OEM-HWNet (ours)	87.8%	91.5%	89.6%	25.31	84

presents a quantitative comparison in Average Precision (AP), mean average precision (mAP), model size, and Frames Per Second (FPS) for the aforementioned models evaluated on the same test set. Firstly, the Faster R-CNN, RT-DETR, and YOLO series are similar in mAP, but YOLOv5s has the smallest size and the fastest detection speed. Therefore, we chose YOLOv5 as the baseline model. Secondly, compared with YOLOv5s, although the incorporation of the Fourth Head and C3WC modules increases the number of model parameters and leads to a decrease from 160 to 84 in FPS, the proposed OEM-HWNet model can still meet the requirement of real-time detection of interlayer distresses. Finally, the proposed OEM-HWNet model presents the highest AP and mAP for both categories of interlayer distresses, although YOLOv5s, YOLOv8s, and YOLOv11s have smaller model size and faster inference speed. In addition, for all models, the detection accuracy of interlayer water seepage is higher than that of interlayer debonding because the GPR image of interlayer water seepage has polarity reversal in the vertical direction, which makes its features more discriminative.

The qualitative assessment is crucial in understanding how these methods perform in real-time detection. Among the state-of-the-art methods, RT-DETR, YOLOv3, YOLOv5s, YOLOv7, and YOLOv8s have a higher mAP; therefore, these methods are selected to analyze furtherly. Figure 16 shows some detection results of the above methods and OEM-HWNet. As the position indicated by the yellow arrow in case 2 in Figure 16, the OEM-HWNet can detect the large-scale interlayer distress with accurate location and high confidence but YOLOv5s and YOLOv7 can not. Although RT-DETR, YOLOv3, and YOLOv8s may detect the large-scale interlayer distress, they may not detect it accurately. The results indicate that the network requires a larger receptive field for detecting large-scale distress accurately. Moreover, as shown in case 1 of Figure 16, some distresses have not been detected in the detection results of YOLOv5s, while OEM-HWNet can detect all interlayer distresses. The comparison results indicate that OEM, based on prior knowledge, can help detect model precision positioning and focused learning. In summary, OEM-HWNet, including Fourth Head, OEM, and C3WC modules, demonstrates excellent performance in interlayer distress detection.

5. Conclusions

In this study, a prior knowledge-guided network called OEM-HWNet was designed to detect the interlayer distress of asphalt pavement with GPR images. Specifically, the OEM, based on prior knowledge, significantly improves network performance without imposing additional parameters. The C3WC module was introduced to obtain a larger receptive field and enhance the low-frequency information. An additional detection head was added to enhance the detection capability of large objects. The conclusions are drawn as follows:

(1)

The existing methods may cause inaccurate locations for interlayer distress due to the interference of similar background features. Based on prior knowledge, the OEM can accurately locate the object region to allow the focusing of the learning of the model on that region without imposing additional parameters;

(2)

The proposed OEM-HWNet model achieves an average precision (AP) of 87.8% for interlayer debonding and 91.5% for interlayer water seepage, resulting in a mAP of 89.6%. Meanwhile, the comparative results with the original YOLOv5s model represent a 3.3% increase in mAP. Moreover, the results also indicate that the OEM-HWNet model surpasses other advanced models in detection accuracy;

(3)

The proposed method may be used for automatic and real-time interlayer distress detection of asphalt pavement using GPR. An extensive GPR dataset from four highways was constructed to evaluate the detection model rigorously. Future research may validate the proposed method with more testing datasets from asphalt pavement. Incorporating more interpretable prior knowledge to guide the design of the detection network is suggested.