Investigation on the Process of Eliminating Abnormal Objects from the Road for the Creation of an AI Program That Can Automatically Detect Potholes ^†

Abstract

For effective pothole control on national highways, autonomous pothole identification technology utilizing artificial intelligence was deployed in Korea. There are a number of different objects on the road’s surface that resemble potholes. The YOLOv7-E6E model was used to reduce noise, before classifying these objects and potholes. In the algorithm, aberrant objects other than potholes were classified using design and learning techniques. Manhole, automobile, lane-marking, garbage, and shadow elements that are similar to potholes were learned, in order to detect them. “Etc.” was used to summarize 15 characteristics, including a broken patch, spalling, crack, ramp, license plate, leaf, and pool. In light of this, learning was conducted using a total of seven classification criteria. The test dataset had a 91% accuracy rate.

1. Introduction

Due to global warming, abnormal weather phenomena frequently occur, and the number of potholes on Korean roads has increased rapidly, due to aging roads and an increase in heavy vehicles [1,2,3]. In Korea, to respond to the increasing occurrence of potholes, two AI pothole detection devices were supplied to 18 land management offices, and the equipment is operated from time to time. To exhaustively detect the pothole, the provided equipment is designed to recognize most abnormal objects on the road pavement as potholes, and then the detection information is transmitted to the general national road pavement data management system (PDMS) [4]. However, there is a disadvantage, in that a vast amount of pothole information is transferred, and excessive human resources are required to select the presence or absence of actual potholes. Therefore, it is necessary to build a web-based AI database technology to automatically and precisely analyze the pothole data received through the detection equipment. This study aims to develop an artificial intelligence model that classifies abnormal objects from the received pothole data.

2. Algorithm to Classify Abnormal Object

2.1. AI Model Selection

AI learning was performed using about 4000 pothole data, using the ResNet, SeNet, Inception v3, and DenseNet models to select an artificial intelligence model. As a result of the learning test, the loss values of SeNet (99.7%), InceptionNet v3 (99.3%), ResNet (99.2%), and DenseNet (99.5%) were obtained. Ultimately, the SeNet model showed the highest accuracy, and was adopted as an artificial intelligence model for anomaly object classification. Figure 1 is a graph comparing the loss values of the artificial intelligence models.

Figure 1. Comparison of the AI model loss values.

2.2. Criteria for Classifying Abnormal Objects

In the transmitted image, objects that could be misinterpreted as potholes were categorized. Potholes were classified as standard objects, and five elements that could be recognized similarly to potholes in the image were, additionally, classified. Six of the elements comprised potholes, manholes, vehicles, lanes, garbage, and shadows, and a further fifteen elements, including cracks and patches, were added together to create the seventh element, “Etc.”. These seven elements determined the classification criteria.

2.3. Artificial Intelligence Learning Process

Among the 598,553 data transmitted from the pothole-detection equipment, 1670 images determined to be potholes were used as learning data. The learning was performed using 15,780 images for seven-classification learning, including potholes, manholes, cars, lanes, garbage, shadows, and “Etc.”. Table 1 shows the representative images, the number of learnings, and the learning rate of each classification item for abnormal object classification learning.

Table 1. Abnormal object learning contents.

No.	Classification of Learning Data	Learning Data Image	Number of Learning Data	Learning Rate (%)
1	Pothole		1670	10.6
2	Manhole		210	1.3
3	Lane		760	4.8
4	Car		1450	9.2
	Trash		40	0.3
	Shadow		120	0.8
	Etc.		11,530	73
Total			15,780	100

3. Results of Abnormal Object Classification Learning

As a result of inputting existing pothole data for verification into the learned artificial intelligence algorithm, it was found that 100% of potholes were detected among abnormal objects on the road. Furthermore, it showed a 91% accuracy in classifying abnormal objects using the image data. Figure 2 shows the results of pothole detection using images.

Figure 2. Pothole detection results.

4. Conclusions

For efficient pothole management, a study was conducted on developing an abnormal object classification artificial intelligence model for a server database, to detect actual potholes among a wide range of pothole images received from AI automatic pothole-detection equipment. The results are as follows.

• As a result of carrying out pothole-detection learning with 1,670 images determined to be potholes among 598,553 data transmitted from the pothole detection equipment, potholes among abnormal objects on the road were detected 100% of the time.
• Twenty-two types of abnormal objects detected on roads and spaces other than roads were classified into seven types. As a result of learning using 15,780 abnormal object image data, the accuracy of the seven types of abnormal objects learned was 91%.
• The accuracy of the anomaly object classification was improved through its learning about 30,000 additional data stored in the pothole management server. Furthermore, server filtering was implemented, using an artificial intelligence classification model, without requiring manual work in order to advance.