Next Article in Journal
Cost Estimation and Parametric Optimization for TIG Welding Joints in Dissimilar Metals Using Linear Regression Algorithm
Previous Article in Journal
A Flexible Job Shop Scheduling Method Based on Multi-Fidelity Optimization
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Proceeding Paper

Investigation on the Process of Eliminating Abnormal Objects from the Road for the Creation of an AI Program That Can Automatically Detect Potholes †

Korea Institute of Civil Engineering and Building Technology, 283, Goyang-daero, Ilsanseo-gu, Goyang-si 10223, Republic of Korea
Ministry of Land, Infrastructure and Transport, 11 Doum 6-ro, Government Complex-Sejong, Sejong City 30103, Republic of Korea
Author to whom correspondence should be addressed.
Presented at the Second International Conference on Maintenance and Rehabilitation of Constructed Infrastructure Facilities, Honolulu, HI, USA, 16–19 August 2023.
Eng. Proc. 2023, 36(1), 66;
Published: 18 September 2023


For effective pothole control on national highways, autonomous pothole identification technology utilizing artificial intelligence was deployed in Korea. There are a number of different objects on the road’s surface that resemble potholes. The YOLOv7-E6E model was used to reduce noise, before classifying these objects and potholes. In the algorithm, aberrant objects other than potholes were classified using design and learning techniques. Manhole, automobile, lane-marking, garbage, and shadow elements that are similar to potholes were learned, in order to detect them. “Etc.” was used to summarize 15 characteristics, including a broken patch, spalling, crack, ramp, license plate, leaf, and pool. In light of this, learning was conducted using a total of seven classification criteria. The test dataset had a 91% accuracy rate.

1. Introduction

Due to global warming, abnormal weather phenomena frequently occur, and the number of potholes on Korean roads has increased rapidly, due to aging roads and an increase in heavy vehicles [1,2,3]. In Korea, to respond to the increasing occurrence of potholes, two AI pothole detection devices were supplied to 18 land management offices, and the equipment is operated from time to time. To exhaustively detect the pothole, the provided equipment is designed to recognize most abnormal objects on the road pavement as potholes, and then the detection information is transmitted to the general national road pavement data management system (PDMS) [4]. However, there is a disadvantage, in that a vast amount of pothole information is transferred, and excessive human resources are required to select the presence or absence of actual potholes. Therefore, it is necessary to build a web-based AI database technology to automatically and precisely analyze the pothole data received through the detection equipment. This study aims to develop an artificial intelligence model that classifies abnormal objects from the received pothole data.

2. Algorithm to Classify Abnormal Object

2.1. AI Model Selection

AI learning was performed using about 4000 pothole data, using the ResNet, SeNet, Inception v3, and DenseNet models to select an artificial intelligence model. As a result of the learning test, the loss values of SeNet (99.7%), InceptionNet v3 (99.3%), ResNet (99.2%), and DenseNet (99.5%) were obtained. Ultimately, the SeNet model showed the highest accuracy, and was adopted as an artificial intelligence model for anomaly object classification. Figure 1 is a graph comparing the loss values of the artificial intelligence models.

2.2. Criteria for Classifying Abnormal Objects

In the transmitted image, objects that could be misinterpreted as potholes were categorized. Potholes were classified as standard objects, and five elements that could be recognized similarly to potholes in the image were, additionally, classified. Six of the elements comprised potholes, manholes, vehicles, lanes, garbage, and shadows, and a further fifteen elements, including cracks and patches, were added together to create the seventh element, “Etc.”. These seven elements determined the classification criteria.

2.3. Artificial Intelligence Learning Process

Among the 598,553 data transmitted from the pothole-detection equipment, 1670 images determined to be potholes were used as learning data. The learning was performed using 15,780 images for seven-classification learning, including potholes, manholes, cars, lanes, garbage, shadows, and “Etc.”. Table 1 shows the representative images, the number of learnings, and the learning rate of each classification item for abnormal object classification learning.

3. Results of Abnormal Object Classification Learning

As a result of inputting existing pothole data for verification into the learned artificial intelligence algorithm, it was found that 100% of potholes were detected among abnormal objects on the road. Furthermore, it showed a 91% accuracy in classifying abnormal objects using the image data. Figure 2 shows the results of pothole detection using images.

4. Conclusions

For efficient pothole management, a study was conducted on developing an abnormal object classification artificial intelligence model for a server database, to detect actual potholes among a wide range of pothole images received from AI automatic pothole-detection equipment. The results are as follows.
  • As a result of carrying out pothole-detection learning with 1,670 images determined to be potholes among 598,553 data transmitted from the pothole detection equipment, potholes among abnormal objects on the road were detected 100% of the time.
  • Twenty-two types of abnormal objects detected on roads and spaces other than roads were classified into seven types. As a result of learning using 15,780 abnormal object image data, the accuracy of the seven types of abnormal objects learned was 91%.
  • The accuracy of the anomaly object classification was improved through its learning about 30,000 additional data stored in the pothole management server. Furthermore, server filtering was implemented, using an artificial intelligence classification model, without requiring manual work in order to advance.

Author Contributions

Conceptualization, M.L.; methodology, M.L.; validation, S.H. and T.L.; investigation, Y.P.; writing-original draft preparation, S.H.; writing-review and editing, M.L., N.L. and C.K.; project administration, M.L. and N.L.; funding acquisition, M.L. and C.K. All authors have read and agreed to the published version of the manuscript.


This work was supported by consignment business 2023 Pavement Management System (PMS) by the Ministry of Land, Infrastructure and Transport (MOLIT).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

All data are new and experimented.

Conflicts of Interest

The authors declare no conflict of interest.


  1. Hong, S.P. A Study on the Improving Pothole with Snowfall in Winter. Master’s Thesis, The University of Seoul, Seoul, Republic of Korea, 2016. [Google Scholar]
  2. Dai, J.; He, K.; Sun, J. Instance-Aware Semantic Segmentation via Multi-Task Network Cascades. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 3150–3158. [Google Scholar]
  3. Kim, D.; Sim, K.; Lee, G. Object Detection by Combining Two Different CNN Algorithms and Robotic Grasping Control. J. Inst. Control Robot. Syst. 2019, 25, 811–817. [Google Scholar] [CrossRef]
  4. Ministry of Land, Infrastructure and Transport. Final-Report of the National Highway Pavement Management System 2019; Ministry of Land, Infrastructure and Transport: Sejong City, Republic of Korea, 2019. [Google Scholar]
Figure 1. Comparison of the AI model loss values.
Figure 1. Comparison of the AI model loss values.
Engproc 36 00066 g001
Figure 2. Pothole detection results.
Figure 2. Pothole detection results.
Engproc 36 00066 g002
Table 1. Abnormal object learning contents.
Table 1. Abnormal object learning contents.
No.Classification of Learning DataLearning Data ImageNumber of Learning DataLearning Rate (%)
1PotholeEngproc 36 00066 i001167010.6
2ManholeEngproc 36 00066 i0022101.3
3LaneEngproc 36 00066 i0037604.8
4CarEngproc 36 00066 i00414509.2
TrashEngproc 36 00066 i005400.3
ShadowEngproc 36 00066 i0061200.8
Etc.Engproc 36 00066 i00711,53073
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Lee, M.; Lee, T.; Park, Y.; Han, S.; Lee, N.; Kim, C. Investigation on the Process of Eliminating Abnormal Objects from the Road for the Creation of an AI Program That Can Automatically Detect Potholes. Eng. Proc. 2023, 36, 66.

AMA Style

Lee M, Lee T, Park Y, Han S, Lee N, Kim C. Investigation on the Process of Eliminating Abnormal Objects from the Road for the Creation of an AI Program That Can Automatically Detect Potholes. Engineering Proceedings. 2023; 36(1):66.

Chicago/Turabian Style

Lee, Moonsup, Taehoon Lee, Younghan Park, Seungyeon Han, Nuri Lee, and Chulki Kim. 2023. "Investigation on the Process of Eliminating Abnormal Objects from the Road for the Creation of an AI Program That Can Automatically Detect Potholes" Engineering Proceedings 36, no. 1: 66.

Article Metrics

Back to TopTop