Automatic Road-Crack Detection with Self-Supervised YOLOv7 in Drone Imagery

Abstract

Supervised object detection models rely on large training datasets to achieve good performance. However, data labeling is often an expensive and time-consuming task. To mitigate this challenge, self-supervised methods have been utilized to learn representations from unlabeled data. Accordingly, this study uses self-supervised learning to improve the YOLOv7 model. YOLOv7 was chosen because of its reliable design and lightweight E-ELAN backbone network, which helps it learn faster without influencing the gradient path. As such, YOLOv7 undergoes two separate training stages: pre-training and fine-tuning. During pre-training, new synthetic images are automatically generated by fusing different kinds of foreground road-damage objects with various background images. In addition, a contrastive loss function was incorporated into YOLOv7 to make class-specific instances cluster together. In the second stage, the pre-trained YOLOv7 model is fine-tuned using real training images. To validate the effectiveness of the proposed approach, it has been evaluated on drone images for road-damage detection from the UAPD dataset with six classes: transverse, longitudinal, oblique, pothole, alligator, and crack repair. The numerical findings showed that self-supervised learning improved YOLOv7’s performance by over eight percentage points (mAP 81.7% vs. 73.5%). The proposed approach also outperformed other detectors found in the literature, including Faster R-CNN (48.8% mAP), self-supervised DETR (20.10% mAP), YOLOv3 (68.75% mAP), and YOLOv4 (56.6% mAP).

1. Introduction

Data labeling and annotation is a time-consuming and labor-intensive process, particularly for complex types of road damage. This can create a bottleneck in the development of deep learning models and limit the amount of data that is available for training. To overcome this issue, self-supervised learning has recently emerged. It has been successfully used for improving the performance of several object detection models such as Faster-RCNN [1,2] and transformer models [3,4]. Primarily, self-supervised techniques in object detection can be divided into two distinct categories: backbone pre-training methods [1,2,5,6,7] and object detection pre-training methods [3,4,8,9]. In the former category, emphasis was placed on pre-training a backbone feature extractor, such as ResNet-50. Then, the pre-trained backbone is incorporated into object detection models such as Faster-RCNN. In the second category, the emphasis is on pre-training an object detection model using unlabeled data in which random labels are generated, and the model learns to predict those labels. For instance, Dai et al. [4] presented a self-supervised model called UP-DETR. In UP-DETR, a random crop is generated from an unlabeled image in an effort to identify its location. According to their findings, pre-training improves model performance in mAP by 7%. In InsLoc [8], two positive images are generated by randomly cropping two different views of the same object. These cropped samples were then placed randomly on two different backgrounds to create synthetic images. Consequently, the objective of the pre-training task is to determine the relative position of these objects, using a contrastive loss, which maximizes their similarity in the embedding space.

Motivated by the success of the InsLoc technique [8], this study aims to apply this concept to the YOLOv7 object detector [10] in order to enhance its performance. It is worth mentioning that YOLO models are powerful and fast object detectors that have been widely used in many real-world applications such as autonomous driving [11], surveillance [12], and robotics [13]. YOLOv7 [10] differs from other YOLO versions in that it introduces a novel backbone network called E-ELAN. Specifically, for deeper networks to converge and learn efficiently, E-ELAN considers the management of the shortest and longest gradient paths. Nevertheless, the classification performance of YOLOv7 is affected when dealing with a limited dataset with classes that have similar shape and texture pattern, such as road-crack detection as given in Figure 1. To boost the performance of YOLOv7 for handling such issues, this study adopts the InsLoc technique [8] where YOLOv7 will undergo the pre-training and fine-tuning stages. During pre-training, new synthetic images are generated automatically by combining various road-damage objects with various backgrounds. In the second stage, YOLOv7 is fine-tuned using real training images. In addition, a contrastive loss has been incorporated into the YOLOv7 loss. This will help YOLOv7 push instances from the same class to be closer to their class centroid and further apart from instances of other classes. This study has the following contributions:

-

Addressing the issue of limited drone road-damage training samples, a self-supervised YOLOv7 is introduced where new synthetic images are automatically generated for the pre-training stage of YOLOv7.

-

Incorporating a contrastive loss function into YOLOv7 in order to encourage the model to decrease the distance between instances from the same class and increase it for dissimilar instances.

-

Conducting extensive experiments on a very challenging dataset for road-damage detection from images captured by drone.

-

Obtaining further enhancement of YOLOv7 performance by using self-supervised techniques which was able to improve the accuracy by more than 8% in the mAP metric as compared with the baseline model.

Figure 1. Six different crack types as in UAPD dataset.

Figure 1. Six different crack types as in UAPD dataset.

It is important to mention that the idea of self-supervised learning is different from semi-supervised methods, which work with partially labeled data where some data are labeled and others are unlabeled, as discussed in [14].

The remainder of this paper is organized as follows. The related work of self-supervised methods in object detection and the proposed techniques in road-damage detection are presented in Section 2. Section 3 presents the architecture of the proposed self-supervised YOLOv7. Section 4 details the experiments conducted and the evaluation of the proposed approach. Section 5 concludes this study and suggests some ideas for future work.

2. Related Work

2.1. Self-Supervised Learning in Object Detection

Recent studies have promoted the use of self-supervised learning in object detection. These techniques can be divided into two categories: pre-training of the backbone network such as DetCon [6], EsViT [7], PixPro [2], DetCo [1], and DenseCL [5], and pre-training of the entire model such as InsLoc [8], and PatchReID [9], UP-DETR [4], and Detreg [3].

DetCon [6] has been introduced as part of the first category of object detection models. In the proposed DetCon model, the authors defined a new contrastive objective function that maximizes the similarity across augmentations of all objects in a scene. Unlike other methods which concentrated on learning whole-image representations, their approach focused on learning local descriptors. As a backbone network, they used ResNet-50 which was pre-trained with ImageNet, then transferred to Mask-RCNN object detector during the fine-tuning stage. In addition, DetCon [6] focused on reducing the computational cost of backbone pre-training. To overcome these cost restrictions, DetCon utilized a larger and more diverse negative set which resulted in speeding up the pre-training learning process.

In EsViT [2], a new pre-training region-matching task enables the model to capture fine-grained region dependencies. A vision transformer was adopted as an object detector and during the pre-training, the aim was to match at both the region level and view level (the whole image). To improve the visual representation at the pixel level, PixPro [2] was suggested. In PixPro [2], contrastive learning was used to measure the similarity at the pixel level. However, pre-training might become computationally expensive owing to the huge number of pixels. Additional work was introduced in DenseCL [5]. The basic notion is that it uses two contrasting procedures, the global contrastive calculator and the dense contrastive calculator. To balance the detection and classification tasks, DetCo was developed by Xie et al. [6]. DetCo achieves a good compromise between the detection and classification tasks. This balance was achieved by developing multi-level features with both global and local object representations.

In spite of this, transferring the pre-trained backbone to object detectors poses several challenges including (i) classification differing from detection in fundamental ways, (ii) architecture differences due to untrained detection heads, and (iii) detection task mismatch because the backbone network was trained on ImageNet. To overcome these limitations, researchers focused on pre-training the entire object detection network which consists of the backbone and the head as investigated in InsLoc [8], PatchReID [9], UP-DETR [4], and Detreg [3].

InsLoc [8] is an early study that introduced the localization of the bounding boxes into the pre-training stage. This work was published in 2021 by researchers from Microsoft with a study titled “Instance Localization for Self-supervised Detection Pre-training”. They proposed a self-supervised pre-training method for object detection. Interestingly, the results of the InsLoc model showed that longer pre-training time further improves detection performances. One limitation of InsLoc is that it largely sacrifices the performance of image classification. To fill this gap, a patch re-identification approach was suggested in [9]. Basically, [9] considered a simultaneous image-level and region-level representation in the feature space. Recently, more attention was given to applying self-supervised techniques to the transformer models such as UP-DETR [4] and DETReg [3]. As explained previously, UP-DETR depends on the generation of random bounding boxes with the aim of determining their locations during the pre-training phase. However, re-identifying random regions will prevent the DETR model from focusing on localizing real objects. To get around this restriction, Detreg [3] has been introduced. Unlike UP-DETR, Detreg [3] is an object-centric detection model where the researchers implemented an unsupervised region proposal generator for foreground object candidates. The conducted analysis showed significant detection improvement over both the baseline DETR model and UP-DETR.

Unfortunately, applying self-supervised techniques for YOLOv7 was not investigated. This is because YOLOv7 has a new backbone network named E-ELAN. As such, to further improve the performance of YOLOv7, this study aimed to utilize the implemented technique in InsLoc [8] to pre-train YOLOv7 for the task of road-damage detection and classification as explained in Section 3.

2.2. Road-Damage Detection

Manual road-damage detection is a labor-intensive and time-consuming task. Generally, three different technologies have been used for road-damage detection which include manual inspection by a human, using a specialized inspection vehicles, or using computer vision techniques. The main advantages and disadvantages of each technology are illustrated in Table 1. As indicated, computer vision techniques are the most cost-effective option, but their accuracy still lags behind that of competing methods. To fill this gap, many computer vision techniques were introduced for road-damage detection. For instance, the proposed study in [15] suggests a computer vision-based system for spotting potholes based on the image segmentation method, followed by calculating the damage ratio. The researchers considered improving the accuracy of the prediction based on several variables such as temperature, relative humidity, precipitation, and traffic volume. Results indicate that their model can predict and detect the occurrence of potholes; however, the accuracy of this method is dependent on the efficiency of the segmentation stage. In [16], the authors implemented CNN and showed significant advantages as compared with existing solutions. Their improvements are mainly related to accuracy and inference time. A pothole detection system for Indian roads was developed by Satti et al. [17]. Unfortunately, they employed classical edge detector methods such as Canny to localize the contour of the pothole. These techniques suffer greatly from image noise and are extremely quality-dependent. On the other hand, deep learning-based methods are more robust for computer vision problems [18].

Table 1. Road-damage detection technologies.

Technology	Advantages	Disadvantages
Manual inspection	Low technical cost	Time-consuming Labor-intensive
Inspection vehicles	High accuracy Detection of multiple type of road cracks	Expensive equipment
Computer vision	Less expensive Cutting-edge detection algorithms could be used	Lower precision

To mitigate the limitations of the classical image processing methods, automatic YOLO-based deep learning models were suggested. In [19] a lightweight YOLOv5 model was discussed where they introduced a novel backbone network suitable for mobile and limited-hardware-support devices. Their results show a speed-up of 22.3% as compared with the baseline model. Further work was carried out by Zhang et al. [20] where they applied YOLOv3 for road-damage detection from drone images. Their main contribution was a modification of the backbone network with a multi-level attention scheme. Reported results showed additional improvement of more than 3% in mAP. Another study of road-crack detection from drone images was introduced by Qiu and Lau [21]. They adopted different YOLO models including YOLOv2, YOLOv2-tiny, YOLOv3, and YOLOv4-tiny. In terms of inference time, the YOLOv2-tiny model was the most efficient for handling 150 frames per second. However, YOLOv4-tiny reported the best detection performances due to its backbone network that enriches the representation of the contextual features [22]. An improved version of YOLOv5s named YOLOv5s-M was given in [23]. The key improvements were in the neck, head, and the backbone. Their model was applied for road-damage detection in urban street-view imagery. The problem of road-damage detection from multiple countries on large-scale data collected by a smartphone was investigated [24]. For this problem, they introduced an ensemble model of YOLOv4 and YOLOv5. The results demonstrated that YOLOv5 outperformed YOLOv4. A two-stage road marking detection model based on YOLOv2 was suggested in [25]. In the first stage, YOLOv2 is used as an initial detector, and then in the second stage, a spatial transformer network verifies the localized cropped regions by YOLOv2. Their results showed an improvement in the precision rate with mAP of 3.9%. However, the second stage adds latency to the inference time, allowing only 58 frames per second to be processed. Recent work in this area was given in [26], which presented a lightweight version of YOLOv5 for detecting road damage.

Currently, YOLOv7 is getting more attention and it significantly outperforms YOLOv5 on the MS COCO dataset [10]. However, few studies were found that utilized YOLOv7 for road-crack detection [27,28]. The incorporation of a self-attention mechanism was investigated in [27], but they used their own collected dataset in the evaluation. This work is different as it introduces a self-supervised YOLOv7 and applies it to road-damage detection in drone images using the publicly available UAPD dataset.

3. The Proposed Approach

In this research YOLOv7 object detection has been used as a backbone network for the implemented self-supervised model. Figure 2 shows the basic architecture of YOLOv7. As shown, it consists of three main components: the backbone, the neck (FPN), and the head. The details of each component are described as follows.

Figure 2. YOLOv7 architecture.

3.1. The Backbone

As depicted in Figure 2, the backbone network is responsible for the process of feature extraction, which is then transmitted to the FPN stage. In YOLOv7, two new designs are proposed, the first of which is ELAN (Effective Layer Aggregation Network) and the second network is E-ELAN (Extended Efficient Layer Aggregation Network). The architecture of E-ELAN is depicted in Figure 3, where various internal operations are used, including expand, shuffle, and consolidate. These procedures were proposed as a means of enhancing the training process and the model’s capacity to learn without affecting the gradient route.

Figure 3. E-ELAN architecture.

3.2. The FPN Layer

As depicted in Figure 2, the second stage of the YOLOv7 model is denoted by the acronym FPN, which stands for Featured Pyramid Network. This stage is responsible for the creation of features at multiple levels, which are then passed on to the pyramid of features. Figure 4 depicts the FPN layer’s internal structure. This layer receives three input channels from the backbone network, labeled c5, c4, and c3, and generates three output channels, labeled P5, P4, and P3.

Figure 4. Internal channels.

3.3. The Head

In the final layer of YOLOv7, known as the “head”, the loss value is calculated. As shown in Figure 5, three distinct loss functions are employed in the training of YOLO models, which are cross-entropy loss, L1 loss, and objectness loss.

Figure 5. Loss function.

The cross-entropy is used to determine the precision of the classification during the training phase. The following equation is used to estimate the usual discrepancy between expected and actual output:

$$los{s}_{cross-entropy}= - {\displaystyle {\sum }_{i=1}^x}p\left(x\right)\mathit{log}q\left(x\right)$$

(1)

After that, the regression error of the created boxes is computed in relation to the ground-truth bounding boxes’ coordinates using L₁ loss:

$$L_1=-{\displaystyle {\sum }_{i=1}^x}\left|y_{true}-y_{predicted}\right|$$

(2)

Finally, the objectness loss function takes on the value 1 or 0. Consequently, the value will be 1 if the item is present in the grid cell and its bounding-box IoU with the ground truth is greater than 0.5, and it will be 0 otherwise.

3.4. Pre-Training of YOLOv7

To implement the self-supervised pre-training stage, we adopt the instance localization technique as a pre-text task which was proposed in [8]. The key idea is shown in Figure 6 where it generates synthetic training images by superimposing instances of the foreground on backgrounds. The objective is to distinguish the foreground from the background using information about bounding boxes. To complete this operation, one must first localize the foreground instance and then extract its features as given in the region encoding by YOLOv7 in Figure 7. Finally, the contrastive loss is added to the original YOLO loss to minimize the distance in the latent space. When the pre-training stage of YOLO7 is finished, the pre-trained model will be used in the second step which is fine-tuning using the real training images.

Figure 6. Pre-training stage of YOLOv7.

Figure 7. The computation of ${\mathrm{L}}_{NCE}$ contrastive loss function.

In this study, we modified the loss function of YOLOv7 by incorporating loss functions that were designed to attract road-damage instances from the same class and push away other instances from other classes as shown in Figure 7. As such, a contrastive loss function ${\mathrm{L}}_{NCE}$ is utilized to make instance q (first augmented positive) near to its anchor k+ (second augmented positive) while pushing it away from its negative as demonstrated in the following equations:

$$\mathrm{c}\mathrm{o}\mathrm{s}\_sim(u,v)={\displaystyle \frac{{\mathrm{u}}^{T}v}{\left|\left|u\right|\right|.\left|\right|v\left|\right|}}$$

(3)

where cos is the cosine function that computes the similarity of two vectors u and v in the embedding space.

$${\mathrm{L}}_{NCE}=-log{\displaystyle \frac{\mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{sim\left(q,k+\right)}{\tau }\right)}{\exp \left(\frac{sim\left(q,k+\right)}{\tau }\right)+\mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{sim\left(q,k-\right)}{\tau }\right)}}$$

(4)

where q is the anchor instance generated by augmentation, k+ represents another augmented positive instance, and k− represents the negative instance; $\tau$ is a hyperparameter called temperature coefficient which controls the contribution of the computed similarity measure, and it lies in the range of [0.1 to 0.5]. In this study $\tau$ was set to 0.1. Hence, large value of $\tau$ will reduce the effect of the computed similarity value from Equation (3). To minimize the effect of the incorporated ${\mathrm{L}}_{NCE}$, it has been multiplied by a small value of 0.1 to assign greater weight for other loss functions used in YOLOv7.

The flowchart in Figure 8 outlines the YOLOv7 pre-training procedure and the details of each step are provided as follows: First, a set of crack-free images were selected from the training corpus to serve as backgrounds for synthetic data generation. Positive road-crack instances were then cropped from a separate annotated dataset. The extracted crack patches were augmented using standard transformations (e.g., brightness/contrast adjustment, random cropping, scaling, and flipping) and composited onto the background images at random spatial locations. For each background, three samples were produced—two positive composites containing cracks and one negative sample without cracks—yielding three labels (bounding boxes for the two positives and a negative label used as a distractor for training the contrastive learner; see Figure 6). Finally, contrastive training was conducted on image patches derived from these samples.

Figure 8. YOLOv7 pre-training flow-chart.

4. Results and Analysis

4.1. Dataset

The drone-based asphalt pavement distress database named UADP is used for this analysis. UADP has a total of 2401 images with a resolution of 500 × 500 pixels for different types of road damage, namely transverse crack, longitudinal crack, oblique crack, pothole, alligator crack, and repair, as shown in Figure 1. Table 2 lists the number of instances for each class and it can be seen that the majority of instances are related to transverse and longitudinal crack types. It is worth mentioning that the angle with respect to the street direction is used as a standard to differentiate between longitudinal, oblique, and transverse as given in Table 3.

Table 2. UADP dataset description.

Damage Type	Longitudinal	Transverse	Alligator	Oblique	Repair	Pothole
Total instances	1264	1263	293	162	769	86

Table 3. The distinction between crack types.

Crack Type	Angle with the Street Direction
Longitudinal	0–30
Oblique	30–60
Transverse	60–90

4.2. Parameter Settings and Evaluation Measures

During model training, the collected dataset was divided randomly into training and testing with a ratio of 80:20 for training and testing, respectively. After that, 10 percent of the training set was used for validation. The validation dataset is usually used to guide the model in determining the optimal values of YOLOv7 hyperparameters during the training time. The specifications of the employed machine are given in Table 4 and Table 5 provides the parameter settings utilized in this experiment. Hyperparameter settings were adopted from prior studies evaluated on the same dataset [20]. Specifically, the training/test split was 80/20. The maximum number of training epochs for YOLOv7 was set to 300, as larger values substantially increase computational cost. A batch size of 32 was used, consistent with common practice in the literature; the momentum parameter was set to 0.3. The learning rate followed a schedule that decayed from an initial value of 1 × 10⁻⁵ to 0. Finally, all remaining settings were kept consistent with recommendations in prior work.

Table 4. Hardware and software specification.

RAM	GPUs	Operating System
48 GB for each GPU	1 NVIDIA RTX A6000 GPUs	Ubuntu 18.04 Linux

Table 5. Hyperparameters settings.

Parameters	Value
Train:test	80:20
Epochs	300
Momentum	0.3
Image size	640
Initial learning rate	${10}^{-5}$
Final learning rate	0.01
Batch size	32
Confidence threshold	0.001
Non-maximum suppression (NMS)	0.65
Gaussian blurring kernel	5 × 5
$\tau$	0.1

To numerically evaluate the performance of the proposed method, precision, recall, and mAP measures are used. The formulas for these quantities are given below:

$$Precision={\displaystyle \frac{TP}{TP+FP}}$$

(5)

$$Recall={\displaystyle \frac{TP}{TP+FN}}$$

(6)

where TP is the total of true positives, FP is the total of false positives, and FN is the total of false negatives.

$$mAP={\displaystyle \frac{{\displaystyle {\sum }_{i=0}^k}Ap\left(i\right)}{k}}$$

(7)

$$AP={\int }_0^{1}p\left(r\right)dr$$

(8)

where AP (average precision) is defined as the integral of each category’s recall rate, with upper and lower bounds of 1 and 0 respectively.

In addition, the IoU measure is used to compute the percentage of overlap between the localized box and the ground-truth bounding box as shown in Figure 9. As such, if the computed IoU value is greater than 0.5, the detection will be considered as a true positive (TP); otherwise, it will be counted as a false alarm (FP) and the object will be counted as a missing case or false negative (FN).

Figure 9. The calculation of IoU measure.

4.3. Pre-Training Stage

As mentioned in Section 3.4, we adopted the pre-training approach suggested in [6] for YOLOv7. As such, to generate synthetic images we used 10 background images cropped from the training set and resized to 500 × 500 pixels. In addition, a total of 30 random road-damage objects were used from the training images (cropped from the training set). After that, those foreground road-damage samples were exposed to both Gaussian blurring filters with a 5 × 5 square kernel and a standard deviation uniformly sampled from [0.1, 2.0]. Then, they were cropped and positioned arbitrarily within the background image as given in Figure 10. To smooth the synthetic image and reduce the contrast between the foreground object and its background, OpenCV’s Contrast Limited Adaptive Histogram Equalization (CLAHE) was used with clipLimit 2. This synthetic process resulted in a total of 1800 synthetic images used in the pre-training stage of YOLOv7. For contrastive class learning, the negative class instances consist of other road-damage types from different damage classes, whereas the positive class instances are considered from the same damage class.

Figure 10. Synthetic image generation with a road crack in red box.

4.4. Fine-Tuning Stage

The pre-trained YOLOv7 underwent a fine-tuning stage with the original training set. The results of the self-supervised YOLOv7 against the standard YOLOv7 are given in Table 6. It can be seen that the pre-training stage significantly improved the detection performance, with a more than 8% increase in mAP. This is because the generated pre-training synthetic data closely resembled the actual data as can be seen in Figure 9.

Table 6. Comparison with the baseline model.

Model	AP						mAP@0.5	t-Test p-Value
	Transverse	Longitudinal	Repair	Alligator	Pothole	Oblique
YOLOv7	0.779	0.774	0.726	0.761	0.688	0.681	0.735	-
Self-supervised YOLOv7	0.858	0.815	0.826	0.828	0.806	0.772	0.817	0.002

Furthermore, the results showed that oblique cracks reported the lowest average precision (AP). One interpretation is because this type has fewer training samples as compared to other crack categories. In addition, it has a similar pattern to the alligator cracks as shown in Figure 1. The best results were obtained for the longitudinal and transverse crack types due to the large amount of training samples. Interestingly, self-supervised YOLOv7 significantly improved all types with a maximum of 0.127 AP for potholes and a minimum value of 0.041 for longitudinal cracks. These improvements are due to the benefits of the pre-training stage as well as the incorporated contrast loss function which makes instances of the same class cluster together while instances of different classes are spread out. A t-test conducted on mAP scores across repeated runs/cross-validation folds shows that the proposed model significantly outperforms standard YOLOv7 at the 0.05 significance level, as reported in Table 6.

The confusion-matrix analysis is shown in Figure 11. It is clear that the self-supervised YOLOv7 yields fewer missing objects, as measured by the false-negative rate (FNR)—the percentage of true road-damage objects missed and classified as background. Specifically, the self-supervised YOLOv7 achieves an FNR of approximately 0.06 across all classes, whereas standard YOLOv7 has an FNR of 0.10, as shown in the bottom row of Figure 11. Most misses for the self-supervised YOLOv7 occur for alligator cracks, which are misclassified as transverse or longitudinal cracks due to similarity. By contrast, standard YOLOv7 fails to detect oblique cracks in most test cases, for the reasons noted earlier. The confusion-matrix analysis also shows that the false-positive (FP) rates of the two models are comparable. This can be mitigated by increasing the objectness confidence threshold (set to 0.01 in Table 5), albeit at the expense of a higher FNR.

Figure 11. Results of the confusion-matrix analysis (a,b).

We further examined class-wise precision–recall (PR) characteristics to quantify operating-point trade-offs (Figure 12). Average precision (AP)—the area under the PR curve—summarizes how well each class is ranked across all decision thresholds. The transverse-crack class attains the highest AP (0.858), with a long high-precision plateau through the mid-recall regime, indicating that a broad range of thresholds yields few false positives while continuing to recover additional true positives. In contrast, the oblique-crack class exhibits the lowest AP, with precision dropping steeply as recall increases; this pattern is consistent with the earlier error analysis pointing to orientation variability, class imbalance, and inter-class similarity (especially with longitudinal cracks) as the dominant failure modes. Alligator cracks achieve intermediate AP, but their PR curve shows precision erosions at higher recall, reflecting residual confusions with transverse/longitudinal cracks noted in the confusion matrix.

Figure 12. The precision–recall curve for each crack type (a–f).

For potholes, precision remains ≈ 1.0 up to recall ≳ 0.7 (Figure 12e), implying strong separability from linear crack types and robust ranking of pothole detections across thresholds. Practically, this suggests that the pothole detector can tolerate a modestly higher confidence threshold to reduce occasional false positives with negligible recall loss, whereas oblique-crack performance is more sensitive to thresholding and benefits from class-specific tuning. Taken together, these PR profiles corroborate the confusion-matrix findings: improvements from the proposed (self-supervised) model are driven primarily by gains in recall without a corresponding increase in false positives.

From an operational standpoint, PR curves support per-class threshold selection aligned to application costs (e.g., maximizing F1 or constraining precision ≥ 0.90). For oblique cracks, we recommend orientation-diverse augmentation (random rotations, perspective warps), class-balanced sampling or loss re-weighting, and rotation-aware modeling (e.g., oriented boxes or rotation-equivariant layers) to flatten the early precision drop. For alligator cracks, multi-scale training, finer tiling, and texture-aware augmentations can help reduce confusions with linear classes. Finally, reporting AP alongside precision at fixed recall levels (e.g., P@R = 0.80), class-wise F1 at the chosen operating points, and bootstrapped 95% CIs would provide a more complete and statistically robust characterization of detection quality.

4.5. Visualization of the Results

The proposed self-supervised YOLOv7 model’s qualitative results are shown in Figure 13. These examples illustrate detection performance across correctly and incorrectly localized cases, such as crack-like objects (false positives, FPs), duplicate detections of the same crack, and missed detections (FNs). Detection results are shown on the right side of Figure 13; the left side represents the ground-truth boxes. In general, the self-supervised YOLOv7 produced outstanding detection performance. For instance, the model localized transverse cracks in the second and third rows with zero false positives. However, the first and fourth rows of Figure 13 illustrate the issue of duplicate bounding boxes. This can be mitigated by decreasing the NMS IoU threshold, which controls the minimum overlap required to suppress boxes during post-processing. In this study, we follow the standard YOLOv7 settings and set the NMS IoU threshold to 0.65, as shown in Table 5.

Figure 13. Sample results from the detection of self-supervised YOLOv7.

4.6. Comparison with YOLO Variants

This section compares the performance of the proposed model with other YOLO variants. Specifically, it is compared with the recent YOLOv8 and YOLOv7-E6E. YOLOv7-E6E was chosen because it has the most complex backbone among the YOLOv7 variants and achieved the best results on MS COCO, as reported in [10]. As shown in Figure 14, the proposed model performs substantially better than YOLOv8 in overall mAP@0.5. A likely reason is that YOLOv8 is anchor-free: it first predicts an object’s center and then estimates the bounding-box width and height, whereas road cracks often lack distinct shapes or visible centers, which degrades YOLOv8’s performance compared with YOLOv7. Another observation is that self-supervised YOLOv7 outperforms YOLOv7-E6E across all road-damage types, confirming the effectiveness of the proposed self-supervised scheme in enhancing the detection performance of standard YOLOv7. The lowest performance for YOLOv7-E6E was observed for the oblique class, likely due to the similarity issues noted earlier.

Figure 14. The results of different YOLO models.

4.7. Comparison with Other Deep Learning Models

This section benchmarks the proposed model against established detectors with published UAPD results and against a recent self-supervised transformer baseline (Table 7). For the supervised baselines, YOLOv3 [20] was selected because it has been previously evaluated on UAPD, and Faster R-CNN (as reported in prior work on UAPD [29]); both underperform relative to the proposed self-supervised YOLOv7. These gains are attributable to the stronger detection head and multi-scale design of YOLOv7, together with the self-supervised pre-training stage, which improves feature quality under limited annotation.

Table 7. Comparison with other models.

Model	YOLOv3 [20]	UP-DETR [4]	Faster-RCNN [29]	Self-Supervised YOLOv7
mAP (%)	68.75	20.10	48.80	81.70

For a self-supervised transformer comparison, we considered UP-DETR [29]. Because [4] does not specify the UAPD train/test split, direct comparison to their published UAPD numbers is limited; accordingly, we trained UP-DETR using the same dataset employed to fine-tune our approach and evaluated it under our protocol. The proposed model exceeds UP-DETR by more than 60% (in mAP), indicating a substantial advantage. A plausible explanation is architectural: DETR’s set-prediction formulation excels on larger, well-delimited objects (e.g., vehicles, animals) but tends to struggle with small, thin, low-contrast structures such as pavement cracks, where dense, multi-scale anchors and convolutional inductive biases—both emphasized in YOLOv7—are advantageous. Overall, the results in Table 7 corroborate that combining YOLOv7 with self-supervised pre-training yields state-of-the-art performance on UAPD.

5. Conclusions

In conclusion, this study introduces a self-supervised YOLOv7 object detector that shows promising results in detecting various types of road damage from the UAPD dataset, captured by drones. The results indicate an improvement in the detection performance of standard YOLOv7 by more than eight percentage points in mAP. The highest accuracy is achieved when localizing transverse and longitudinal cracks. However, oblique-crack detection yields poorer results due to the small number of training samples and its similarity to other crack types. Most importantly, the results show a significant improvement in the detection accuracy of self-supervised YOLOv7 compared with baseline YOLOv7 and other deep learning models. Visual analysis further verifies that the proposed method achieves a negligible false-alarm rate.

In the future, we anticipate expanding the use of self-supervised learning to YOLO variants. In addition, the proposed approach could be applied to other industrial and medical applications that have limited training data, such as defect detection and lesion detection.