Automated Distress Detection, Classification and Measurement for Asphalt Urban Pavements Using YOLO ^†

Abstract

In pavement management, it is essential to have a good database with information on the condition of the roads that compose the corresponding network. In Chile, such a database does not currently exist, and there is no technology that can evaluate urban pavement condition in an efficient way. On this research, more than 50,000 images of 13.2 × 2.6 m of asphalt pavement from different zones of Santiago, Chile, were obtained. These images were processed, and the following distresses were labeled with two different levels of severities: patches; potholes; and transversal, longitudinal, and fatigue cracking. These data were used to train and evaluate the following object detection convolutional neural network models: YOLOv5 and YOLOv7.

1. Introduction

This study was carried out in Santiago, Chile, with the purpose of creating a database for pavement management, specifically to automate and improve the efficiency of urban pavement monitoring. Using low-cost technology, pavement images are taken and used to train a YOLO neural network to automate the detection, classification, and measurement of deterioration in urban pavements.

2. Materials and Methods

Asphalt pavement recordings were obtained by using a GoPro Hero 8 black camera mounted to a car by using a bicycle rack, as shown in Figure 1 [1].

By using the camera’s telemetry, frames were extracted to obtain an image of every section of the pavement.

Due to the position of the camera, the images obtained are not in plan view, and since the objective is to be able to measure distresses, a perspective transformation is applied [2], as shown in Figure 2. The red square was used to calibrate the transformation, since it had previously known dimensions (400 × 400 mm).

For a better understanding of the context of the distresses shown in the image, five of these images are joined together, working with a 13.55 × 2.66 m pavement section image. These images are normalized for better performance of the artificial neural network. Some examples of pavement sections are shown in Figure 3.

Asphalt pavement images from six different municipalities were obtained, with a total length of 104.0 km and a total surface of 276.7 km².

The total number of distresses labeled are shown in

Table 1. Number of distresses found in the images obtained in Santiago, Chile.

	Severity
Distress	Medium	High
Fatigue	2128	4240
Transversal Cracking	6783	4701
Longitudinal Cracking	2176	1603
Patch	296	193
Potholes	129	313

. This was performed manually with the help of some engineering students by using the rectangle labels in VGG Image Annotator [3].

It should be noted that the most common singularities found in pavement sections were also labeled in order to avoid confusion with distresses, such as manhole covers, drains, and core drilling.

The images are randomly split into 80% training, 10% validation, and 10% testing sets, while maintaining the same percentages for each type of distress. For both YOLOv5 and YOLOv7, training with 300 epochs each is carried out using the training and validation set, while for performance evaluation, the test set is used, i.e., images that have not been previously seen by the network.

The training was performed using a Lenovo Legion T5i Tower 6ta Gen with a NVIDIA GeForce^® RTX™ 3060 12 GB GDDR6 graphic card.

3. Results

Table 2. Results obtained using YOLO.

YOLO	Precision (%)	Recall (%)	mAP 0.05 (%)
v5	41.4	43.3	37.4
v7	42.2	39.9	36.8

shows the performance using both YOLOv5 and YOLOv7 with the test set. YOLOv5 and YOLOv7 took 144 and 75 h to run, respectively.

The confusion matrices are shown in Figure 4.

An example of the results obtained by evaluating the test set in YOLOv5 is shown in Figure 5.

4. Discussion

As shown in Table 2, both versions of YOLO achieved a similar performance.

As for the distresses, alligator and transverse cracking demonstrated better performance (over 50%). However, longitudinal cracking, patches, and potholes are mostly undetectable.

Although the network can identify and classify the distresses, it is unreliable in terms of severity classification, which can be seen in the diagonal of the confusion matrices in Figure 4.

Finally, YOLO was originally trained with the COCO dataset, in which objects are clearly defined, unlike distresses, where different observers might classify cracks differently.

5. Conclusions

In conclusion, there is no significant difference between the performances of the different YOLO versions. However, YOLOv7 took about half the time it took to train YOLOv5, which is a significant difference.

The pavement distresses that can be found with these results are alligator and transverse cracking; the network developed in this investigation may be useful for the detection of manhole covers and drains.