Automated Road Defect and Anomaly Detection for Traffic Safety: A Systematic Review
Abstract
:1. Introduction
1.1. Background
1.2. Motivation and Contribution
- Selection criteria and resulting review of globally relevant articles uniquely combining automated road defects and anomalies (ARDAD) peer-reviewed research since 2000.
- Discovery of the upward and exponentially growing trend of ARDAD surveillance automation since 2000.
- Taxonomy of machine and DL approaches combined with CV, including data acquisition technology and algorithms.
- List of popular and current open access ARDAD datasets.
- Critical analysis of the current state-of-the-art ARDAD methods to highlight the shortcomings that could be addressed in future research, including increasing environmental awareness of connected/self-driving cars.
- Compliance list adopted from the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) (http://prisma-statement.org, accessed on 20 December 2022) and applied to the ARDAD research context.
2. Research Questions and Review Approach
- What are the best ML methods for improving classification performance and creating a robust detection and alert system?
- What implications does the up-to-date research have on motorists’ safety and future applications to related contexts, such as improving the environmental awareness of connected/self-driving cars?
Data Gathering and Inclusion–Exclusion Criteria
(prediction ∨ classification ∨ detection) ∧ (year ≥ 2000))
3. Datasets Reviewed
4. Literature Review
4.1. ML-Based ARDAD
4.2. Ensemble Learning for Improved Anomaly and Defect Detection
4.3. Detection Based on 3D Imaging Methods
5. Gaps, Challenges, and Limitations
- Despite setting the inclusion parameters for publishing dates between 2000 and 2023, the literature search yielded only 311 papers. Due to the focused selection criteria, the systematic review included only 116 papers (Table 1).
- Contrary to our expectations, the number of computer vision-based studies directly impacting motorist safety was lower than expected.
6. Conclusions and Future Work
Supplementary Materials
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
AI | Artificial Intelligence |
ANN | Artificial Neural Network |
ARDAD | Automated Road Defect and Anomaly Detection |
CNN | Convolutional Neural Network |
CV | Computer Vision |
DL | Deep Learning |
E2E | End-to-End |
EM | Ensemble Model |
EP | Ensemble Prediction |
IoT | Internet of Things |
ML | Machine Learning |
NMS | Non-Maximum Suppression |
PRISMA | Preferred Reporting Items for Systematic reviews and Meta-Analyses (http://prisma-statement.org, accessed on 20 December 2022) |
u-YOLO | ultralytics-You Only Look Once |
YOLO | You Only Look Once |
References
- Clevenger, A.P.; Hardy, A.; Gunson, K.; Bissonette, J. Analyses of Wildlife-Vehicle Collision Data: Applications for Guiding Decision-Making for Wildlife Crossing Mitigation and Motorist Safety; Unpublished Report; Utah State University: Logan, UT, USA, 2006. [Google Scholar]
- Dumbaugh, E.; Li, W. Designing for the safety of pedestrians, cyclists, and motorists in urban environments. J. Am. Plan. Assoc. 2010, 77, 69–88. [Google Scholar] [CrossRef]
- Khan, N.A.; Jhanjhi, N.; Brohi, S.N.; Usmani, R.S.A.; Nayyar, A. Smart traffic monitoring system using unmanned aerial vehicles (UAVs). Comput. Commun. 2020, 157, 434–443. [Google Scholar] [CrossRef]
- Justo-Silva, R.; Ferreira, A. Pavement maintenance considering traffic accident costs. Int. J. Pavement Res. Technol. 2019, 12, 562–573. [Google Scholar] [CrossRef]
- Chen, S.; Kuhn, M.; Prettner, K.; Bloom, D.E. The global macroeconomic burden of road injuries: Estimates and projections for 166 countries. Lancet Planet. Health 2019, 3, e390–e398. [Google Scholar] [CrossRef]
- Mohammed, A.A.; Ambak, K.; Mosa, A.M.; Syamsunur, D. A review of traffic accidents and related practices worldwide. Open Transp. J. 2019, 13, 65–83. [Google Scholar] [CrossRef] [Green Version]
- Manual, H.S. The Highway Safety Manual; American Association of State Highway and Transportation Officials: Washington, DC, USA, 2019.
- Munawar, H.S.; Hammad, A.W.A.; Haddad, A.; Soares, C.A.P.; Waller, S.T. Image-based crack detection methods: A review. Infrastructures 2021, 6, 115. [Google Scholar] [CrossRef]
- Mohan, A.; Poobal, S. Crack detection using image processing: A critical review and analysis. Alex. Eng. J. 2018, 57, 787–798. [Google Scholar] [CrossRef]
- Bhatt, P.M.; Malhan, R.K.; Rajendran, P.; Shah, B.C.; Thakar, S.; Yoon, Y.J.; Gupta, S.K. Image-based surface defect detection using deep learning: A review. J. Comput. Inf. Sci. Eng. 2021, 21, 014501. [Google Scholar] [CrossRef]
- Bhamare, L.; Mitra, N.; Varade, G.; Mehta, H. Study of types of road abnormalities and techniques used for their detection. In Proceedings of the 27th International Conference on Electrical, Electronics and Information Engineering (ICEEIE), Changsha, China, 13–16 August 2021. [Google Scholar]
- Nguyen, V.K.; Renault, É.; Ha, V.H. Road anomaly detection using smartphone: A brief analysis. In Proceedings of the International Conference on Mobile, Secure, and Programmable Networking, Paris, France, 18–20 June 2018. [Google Scholar]
- Avci, O.; Abdeljaber, O.; Kiranyaz, S.; Hussein, M.; Gabbouj, M.; Inman, D.J. A review of vibration-based damage detection in civil structures: From traditional methods to Machine Learning and Deep Learning applications. Mech. Syst. Signal Process. 2021, 147, 107077. [Google Scholar] [CrossRef]
- Azimi, M.; Eslamlou, A.D.; Pekcan, G. Data-Driven Structural Health Monitoring and Damage Detection through Deep Learning: State-of-the-Art Review. Sensors 2020, 20, 2778. [Google Scholar] [CrossRef]
- Xu, Y.; Ouyang, X.; Cheng, Y.; Yu, S.; Xiong, L.; Ng, C.-C.; Pranata, S.; Shen, S.; Xing, J. Dual-mode vehicle motion pattern learning for high performance road traffic anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA, 18–22 June 2018. [Google Scholar]
- Chalapathy, R.; Chawla, S. Deep learning for anomaly detection: A survey. arXiv 2019, arXiv:1901.03407. [Google Scholar]
- Fahim, M.; Sillitti, A. Anomaly Detection, Analysis and Prediction Techniques in IoT Environment: A Systematic Literature Review. IEEE Access 2019, 7, 81664–81681. [Google Scholar] [CrossRef]
- Santhosh, K.K.; Dogra, D.P.; Roy, P.P. Anomaly detection in road traffic using visual surveillance: A survey. ACM Comput. Surv. 2020, 53, 1–26. [Google Scholar] [CrossRef]
- McCall, J.; Trivedi, M. Video-Based Lane Estimation and Tracking for Driver Assistance: Survey, System, and Evaluation. IEEE Trans. Intell. Transp. Syst. 2006, 7, 20–37. [Google Scholar] [CrossRef] [Green Version]
- Martinez-Ríos, E.A.; Bustamante-Bello, M.R.; Arce-Sáenz, L.A. A Review of Road Surface Anomaly Detection and Classification Systems Based on Vibration-Based Techniques. Appl. Sci. 2022, 12, 9413. [Google Scholar] [CrossRef]
- Anand, S.; Gupta, S.; Darbari, V.; Kohli, S. Crack-pot: Autonomous road crack and pothole detection. In Digital Image Computing: Techniques and Applications (DICTA); IEEE: Canberra, Australia, 2018. [Google Scholar]
- Varona, B.; Monteserin, A.; Teyseyre, A. A deep learning approach to automatic road surface monitoring and pothole detection. Pers. Ubiquitous Comput. 2020, 24, 519–534. [Google Scholar] [CrossRef]
- Staniek, M. Detection of cracks in asphalt pavement during road inspection processes. Sci. J. Sil. Univ. Technol. 2017, 96, 175–184. [Google Scholar] [CrossRef]
- Tsa, D.M.; Wu, S.K. Automated surface inspection using gabor filters. Int. J. Adv. Manuf. Technol. 2000, 16, 474–482. [Google Scholar] [CrossRef] [Green Version]
- Tsai, D.-M.; Huang, T.-Y. Automated surface inspection for statistical textures. Image Vis. Comput. 2003, 21, 307–323. [Google Scholar] [CrossRef] [Green Version]
- Zhou, Y.; Wang, F.; Meghanathan, N.; Huang, Y. Seed-based approach for automated crack detection from pavement images. Transp. Res. Rec. 2016, 2589, 162–171. [Google Scholar] [CrossRef]
- Protopapadakis, E.; Voulodimos, A.; Doulamis, A.; Doulamis, N.; Stathaki, T. Automatic crack detection for tunnel inspection using deep learning and heuristic image post-processing. Appl. Intell. 2019, 49, 2793–2806. [Google Scholar] [CrossRef]
- Chambon, S.; Moliard, J.-M. Automatic Road Pavement Assessment with Image Processing: Review and Comparison. Int. J. Geophys. 2011, 2011, 989354. [Google Scholar] [CrossRef] [Green Version]
- Pinggera, P.; Ramos, S.; Gehrig, S.; Franke, U.; Rother, C.; Mester, R. Lost and found: Detecting small road hazards for self-driving vehicles. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Republic of Korea, 9–14 October 2016. [Google Scholar]
- Chen, J.; Xu, H.; Wu, J.; Yue, R.; Yuan, C.; Wang, L. Deer Crossing Road Detection with Roadside LiDAR Sensor. IEEE Access 2019, 7, 65944–65954. [Google Scholar] [CrossRef]
- Alam, H.; Valles, D. Debris object detection caused by vehicle accidents using UAV and deep learning techniques. In Proceedings of the IEEE 12th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), Vancouver, BC, Canada, 27–30 October 2021. [Google Scholar]
- Ramaiah, N.K.B.; America, L.H.; Kundu, S.K. Stereo Vision-Based Road Debris Detection System for Advanced Driver Assistance Systems. SAE Int. J. Transp. Saf. 2021, 10, 51–73. [Google Scholar] [CrossRef]
- Maddalena, L.; Petrosino, A. A Self-Organizing Approach to Background Subtraction for Visual Surveillance Applications. IEEE Trans. Image Process. 2008, 17, 1168–1177. [Google Scholar] [CrossRef] [PubMed]
- You, J.; Kim, Y.-K. Up-Sampling Method for Low-Resolution LiDAR Point Cloud to Enhance 3D Object Detection in an Autonomous Driving Environment. Sensors 2022, 23, 322. [Google Scholar] [CrossRef]
- Moon, H.G.; Kim, J.H. Intelligent crack detecting algorithm on the concrete crack image using neural network. In Proceedings of the 28th ISARC, Seoul, Republic of Korea, 29 June–2 July 2011. [Google Scholar]
- Riveiro, M.; Falkman, G.; Ziemke, T. Visual analytics for the detection of anomalous maritime behavior. In Proceedings of the 12th International Conference Information Visualisation, London, UK, 9–11 July 2008. [Google Scholar]
- Mukhamediev, R.I.; Symagulov, A.; Kuchin, Y.; Yakunin, K.; Yelis, M. From classical machine learning to deep neural networks: A simplified scientometric review. Appl. Sci. 2021, 11, 5541. [Google Scholar] [CrossRef]
- Tanaka, N.; Mouri, M. A detection method of cracks and structural objects of the road surface image. In Proceedings of the IAPR Conference on Machine Vision Applications (IAPR MVA 2000), Tokyo, Japan, 28–30 November 2000. [Google Scholar]
- Jouneau, E.; Carincotte, C. Particle-based tracking model for automatic anomaly detection. In Proceedings of the 18th IEEE International Conference on Image Processing, Brussels, Belgium, 11–14 September 2011. [Google Scholar]
- Kim, S.; Anagnostopoulos, G.; Barmpounakis, E.; Geroliminis, N. Visual extensions and anomaly detection in the pNEUMA experiment with a swarm of drones. Transp. Res. Part C Emerg. Technol. 2023, 147, 103966. [Google Scholar] [CrossRef]
- Hijji, M.; Iqbal, R.; Pandey, A.K.; Doctor, F.; Karyotis, C.; Rajeh, W.; Alshehri, A.; Aradah, F. 6G connected vehicle framework to support intelligent road maintenance using deep learning data fusion. IEEE Trans. Intell. Transp. Syst. 2023, 1–10. [Google Scholar] [CrossRef]
- Hegde, V.; Trivedi, D.; Alfarrarjeh, A.; Deepak, A.; Kim, S.H.; Shahabi, C. Yet another deep learning approach for road damage detection using ensemble learning. In Proceedings of the IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA, 10–13 December 2020. [Google Scholar]
- Ishtiak, T.; Ahmed, S.; Anila, M.H.; Farah, T. A convolutional neural network approach for road anomalies detection in Bangladesh with image thresholding. In Proceedings of the Third World Conference on Smart Trends in Systems Security and Sustainability (WorldS4), London, UK, 30–31 July 2019. [Google Scholar]
- Collins, T.K. Debris flows caused by failure of fill slopes: Early detection, warning, and loss prevention. Landslides 2008, 5, 107–120. [Google Scholar] [CrossRef]
- Kraus, S.; Breier, M.; Dasí-Rodríguez, S. The art of crafting a systematic literature review in entrepreneurship research. Int. Entrep. Manag. J. 2020, 16, 1023–1042. [Google Scholar] [CrossRef] [Green Version]
- Greenhalgh, T. How to read a paper: Getting your bearings (deciding what the paper is about). BMJ 1997, 315, 243–246. [Google Scholar] [CrossRef] [Green Version]
- Brereton, P.; Kitchenham, B.A.; Budgen, D.; Turner, M.; Khalil, M. Lessons from applying the systematic literature review process within the software engineering domain. J. Syst. Softw. 2007, 80, 571–583. [Google Scholar] [CrossRef] [Green Version]
- Guo, F.; Qian, Y.; Liu, J.; Yu, H. Pavement crack detection based on transformer network. Autom. Constr. 2023, 145, 104646. [Google Scholar] [CrossRef]
- Julio-Rodríguez, J.d.C.; Rojas-Ruiz, C.A.; Santana-Díaz, A.; Bustamante-Bello, M.R.; Ramirez-Mendoza, R.A. Environment classification using machine learning methods for eco-driving strategies in intelligent vehicles. Appl. Sci. 2022, 12, 5578. [Google Scholar] [CrossRef]
- Ferjani, I.; Alsaif, S.A. How to get best predictions for road monitoring using machine learning techniques. PeerJ Comput. Sci. 2022, 8, e941. [Google Scholar] [CrossRef]
- Wang, C.; Xu, S.; Yang, J. Adaboost Algorithm in Artificial Intelligence for Optimizing the IRI Prediction Accuracy of Asphalt Concrete Pavement. Sensors 2021, 21, 5682. [Google Scholar] [CrossRef]
- Chen, C.; Chandra, S.; Han, Y.; Seo, H. Deep learning-based thermal image analysis for pavement defect detection and classification considering complex pavement conditions. Remote Sens. 2021, 14, 106. [Google Scholar] [CrossRef]
- Li, Y.; Guo, T.; Xia, R.; Xie, W. Road traffic anomaly detection based on fuzzy theory. IEEE Access 2018, 6, 40281–40288. [Google Scholar] [CrossRef]
- Cabral, F.S.; Pinto, M.; Mouzinho, F.A.; Fukai, H.; Tamura, S. An automatic survey system for paved and unpaved road classification and road anomaly detection using smartphone sensor. In Proceedings of the IEEE International Conference on Service Operations and Logistics, and Informatics (SOLI), Singapore, 31 July–2 August 2018. [Google Scholar]
- Zhang, L.; Yang, F.; Zhang, Y.D.; Zhu, Y.J. Road crack detection using deep convolutional neural network. In Proceedings of the IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA, 25–28 September 2016. [Google Scholar]
- Ballinas-Hernández, A.L.; Olmos-Pineda, I.; Olvera-López, J.A. Speed Bump Detection on Roads using Artificial Vision. Res. Comput. Sci. 2019, 148, 71–82. [Google Scholar] [CrossRef]
- Makantasis, K.; Protopapadakis, E.; Doulamis, A.; Doulamis, N.; Loupos, C. Deep convolutional neural networks for efficient vision based tunnel inspection. In Proceedings of the IEEE International Conference on Intelligent Computer Communication and Processing (ICCP), Cluj-Napoca, Romania, 3–5 September 2015. [Google Scholar]
- Maya, S.; Ueno, K.; Nishikawa, T. dLSTM: A new approach for anomaly detection using deep learning with delayed prediction. Int. J. Data Sci. Anal. 2019, 8, 137–164. [Google Scholar] [CrossRef] [Green Version]
- Li, W.; Liu, Y.; Ke, L.; Peng, Y.; Ding, H.; Liu, Q.-Z. Road tunnel detection robot and method based on laser point cloud. IOP Conf. Ser. Mater. Sci. Eng. 2020, 741, 012050. [Google Scholar] [CrossRef]
- Laubis, K.; Simko, V.; Schuller, A. Road condition measurement and assessment: A crowd based sensing approach. In Proceedings of the 37th International Conference on Information Systems, Dublin, Ireland, 11–14 December 2016. [Google Scholar]
- Shi, Y.; Cui, L.; Qi, Z.; Meng, F.; Chen, Z. Automatic Road Crack Detection Using Random Structured Forests. IEEE Trans. Intell. Transp. Syst. 2016, 17, 3434–3445. [Google Scholar] [CrossRef]
- Wong, Y.D. Response-based methods to measure road surface irregularity: A state-of-the-art review. Eur. Transp. Res. Rev. 2019, 11, 43. [Google Scholar]
- Wang, K.-F.; Jia, X.; Tang, S. A survey of vision-based automatic incident detection technology. In Proceedings of the IEEE International Conference on Vehicular Electronics and Safety, Xi’an, China, 14–16 October 2005. [Google Scholar]
- Wang, H.; Fan, R.; Sun, Y.; Liu, M. Dynamic fusion module evolves drivable area and road anomaly detection: A benchmark and algorithms. IEEE Trans. Cybern. 2021, 52, 10750–10760. [Google Scholar] [CrossRef]
- Tian, B.; Morris, B.T.; Tang, M.; Liu, Y.; Yao, Y.; Gou, C.; Shen, D.; Tang, S. Hierarchical and Networked Vehicle Surveillance in ITS: A Survey. IEEE Trans. Intell. Transp. Syst. 2016, 18, 25–48. [Google Scholar] [CrossRef]
- Sattar, S.; Li, S.; Chapman, M. Road Surface Monitoring Using Smartphone Sensors: A Review. Sensors 2018, 18, 3845. [Google Scholar] [CrossRef] [Green Version]
- LaPadula, L.J. State of the Art in Anomaly Detection and Reaction: An Update; Mitre Corporation: Bedford, MA, USA, 2000. [Google Scholar]
- Dib, J.; Sirlantzis, K.; Howells, G. A Review on Negative Road Anomaly Detection Methods. IEEE Access 2020, 8, 57298–57316. [Google Scholar] [CrossRef]
- Bulusu, S.; Kailkhura, B.; Li, B.; Varshney, P.K.; Song, D. Anomalous Example Detection in Deep Learning: A Survey. IEEE Access 2020, 8, 132330–132347. [Google Scholar] [CrossRef]
- Berroukham, A.; Housni, K.; Lahraichi, M.; Boulfrifi, I. Deep learning-based methods for anomaly detection in video surveillance: A review. Bull. Electr. Eng. Inform. 2023, 12, 314–327. [Google Scholar] [CrossRef]
- Opara, J.N.; Thein, A.B.B.; Izumi, S.; Yasuhara, H.; Chun, P.-J. Defect detection on asphalt pavement by deep learning. GEOMATE J. 2021, 21, 87–94. [Google Scholar]
- Yu, J.; Kim, D.Y.; Lee, Y.; Jeon, M. Unsupervised pixel-level road defect detection via adversarial image-to-frequency transform. In Proceedings of the IEEE Intelligent Vehicles Symposium (IV), Las Vegas, NV, USA, 19 October–13 November 2020. [Google Scholar]
- Pandey, B.; Pandey, D.; Mishra, B.; Rhmann, W. A comprehensive survey of deep learning in the field of medical imaging and medical natural language processing: Challenges and research directions. J. King Saud Univ.-Comput. Inf. Sci. 2021, 34, 5083–5099. [Google Scholar] [CrossRef]
- Li, R.; Yuan, Y.; Zhang, W.; Yuan, Y. Unified Vision-Based Methodology for Simultaneous Concrete Defect Detection and Geolocalization. Comput. Civ. Infrastruct. Eng. 2018, 33, 527–544. [Google Scholar] [CrossRef]
- Cha, Y.-J.; Choi, W.; Büyüköztürk, O. Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks. Comput. Civ. Infrastruct. Eng. 2017, 32, 361–378. [Google Scholar] [CrossRef]
- Chellaswamy, C.; Famitha, H.; Anusuya, T.; Amirthavarshini, S.B. IoT based humps and pothole detection on roads and information sharing. In Proceedings of the International Conference on Computation of Power, Energy, Information and Communication (ICCPEIC), Chennai, India, 28–29 March 2018. [Google Scholar]
- Su, C.; Wang, W. Concrete Cracks Detection Using Convolutional Neural Network Based on Transfer Learning. Math. Probl. Eng. 2020, 2020, 7240129. [Google Scholar] [CrossRef]
- Zhao, H.; Qin, G.; Wang, X. Improvement of Canny algorithm based on pavement edge detection. In Proceedings of the 3rd International Congress on Image and Signal Processing, Yantai, China, 16–18 October 2010. [Google Scholar]
- Chatterjee, S.; Saeedfar, P.; Tofangchi, S.; Kolbe, L.M. Intelligent road maintenance: A machine learning approach for surface defect detection. In Proceedings of the 26th European Conference on Information Systems (ECIS), Portsmouth, UK, 23–28 June 2018. [Google Scholar]
- Vigneshwar, K.; Kumar, B.H. Detection and counting of pothole using image processing techniques. In Proceedings of the IEEE International Conference on Computational Intelligence and Computing Research (ICCIC), Chennai, India, 15–17 December 2016. [Google Scholar]
- Bustamante-Bello, R.; García-Barba, A.; Arce-Saenz, L.A.; Curiel-Ramirez, L.A.; Izquierdo-Reyes, J.; Ramirez-Mendoza, R.A. Visualizing Street Pavement Anomalies through Fog Computing V2I Networks and Machine Learning. Sensors 2022, 22, 456. [Google Scholar] [CrossRef]
- Zhou, B.; Zhao, W.; Guo, W.; Li, L.; Zhang, D.; Mao, Q.; Li, Q. Smartphone-based road manhole cover detection and classification. Autom. Constr. 2022, 140, 104344. [Google Scholar] [CrossRef]
- Alipour, M.; Harris, D.K. Increasing the robustness of material-specific deep learning models for crack detection across different materials. Eng. Struct. 2020, 206, 110157. [Google Scholar] [CrossRef]
- Fan, R.; Bocus, M.J.; Zhu, Y.; Jiao, J.; Wang, L.; Ma, F.; Cheng, S.; Liu, M. Road crack detection using deep convolutional neural network and adaptive thresholding. In Proceedings of the IEEE Intelligent Vehicles Symposium (IV), Paris, France, 9–12 June 2019. [Google Scholar]
- Ryu, S.-K.; Kim, T.; Kim, Y.-R. Image-Based Pothole Detection System for ITS Service and Road Management System. Math. Probl. Eng. 2015, 2015, 968361. [Google Scholar] [CrossRef] [Green Version]
- Hoang, N.-D.; Nguyen, Q.-L. A novel method for asphalt pavement crack classification based on image processing and machine learning. Eng. Comput. 2019, 35, 487–498. [Google Scholar] [CrossRef]
- Fakhri, S.A.; Saadatseresht, M. Road crack detection using gaussian/prewitt filter. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2019, XLII-4/W18, 371–377. [Google Scholar] [CrossRef] [Green Version]
- Park, J.W.; Lee, J.W.; Jhang, K.Y. A lane-curve detection based on an LCF. Pattern Recognit. Lett. 2003, 24, 2301–2313. [Google Scholar] [CrossRef]
- Yu, S.N.; Jang, J.-H.; Han, C.-S. Auto inspection system using a mobile robot for detecting concrete cracks in a tunnel. Autom. Constr. 2007, 16, 255–261. [Google Scholar] [CrossRef]
- Deng, W.; Zhao, Q. Road surface condition identification based on statistical pattern recognition method. In Proceedings of the 12th International IEEE Conference on Intelligent Transportation Systems, St. Louis, MO, USA, 4–7 October 2009. [Google Scholar]
- Safaei, N.; Smadi, O.; Masoud, A.; Safaei, B. An automatic image processing algorithm based on crack pixel density for pavement crack detection and classification. Int. J. Pavement Res. Technol. 2021, 15, 159–172. [Google Scholar] [CrossRef]
- Mei, Q.; Gül, M.; Azim, R. Densely connected deep neural network considering connectivity of pixels for automatic crack detection. Autom. Constr. 2020, 110, 103018. [Google Scholar] [CrossRef]
- Wang, H.; Fan, R.; Sun, Y.; Liu, M. Applying surface normal information in drivable area and road anomaly detection for ground mobile robots. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA, 25–29 October 2020. [Google Scholar]
- Zheng, Z.; Zhou, M.; Chen, Y.; Huo, M.; Sun, L.; Zhao, S.; Chen, D. A fused method of machine learning and dynamic time warping for road anomalies detection. IEEE Trans. Intell. Transp. Syst. 2020, 23, 827–839. [Google Scholar] [CrossRef]
- Khan, S.W.; Hafeez, Q.; Khalid, M.I.; Alroobaea, R.; Hussain, S.; Iqbal, J.; Almotiri, J.; Ullah, S.S. Anomaly detection in traffic surveillance videos using deep learning. Sensors 2022, 22, 6563. [Google Scholar] [CrossRef]
- Naddaf-Sh, S.; Naddaf-Sh, M.M.; Kashani, A.R.; Zargarzadeh, H. An efficient and scalable deep learning approach for road damage detection. In Proceedings of the IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA, 10–13 December 2020. [Google Scholar]
- Mahadevan, V.; Li, W.; Bhalodia, V.; Vasconcelos, N. Anomaly detection in crowded scenes. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 13–18 June 2010. [Google Scholar]
- Oliveira, H.; Correia, P.L. Automatic road crack segmentation using entropy and image dynamic thresholding. In Proceedings of the 17th European Signal Processing Conference, Glasgow, Scotland, UK, 24–28 August 2009. [Google Scholar]
- Nienaber, S.; Booysen, M.; Kroon, R. Detecting potholes using simple image processing techniques and real-world footage. In Proceedings of the 34th Annual Southern African Transport Conference SATC, Pretoria, South Africa, 6–9 July 2015. [Google Scholar]
- Amhaz, R.; Chambon, S.; Idier, J.; Baltazart, V. Automatic Crack Detection on Two-Dimensional Pavement Images: An Algorithm Based on Minimal Path Selection. IEEE Trans. Intell. Transp. Syst. 2016, 17, 2718–2729. [Google Scholar] [CrossRef] [Green Version]
- Akarsu, B.; Karaköse, M.; Parlak, K.; Erhan, A.; Sarimaden, A. A Fast and Adaptive Road Defect Detection Approach Using Computer Vision with Real Time Implementation. Int. J. Appl. Math. Electron. Comput. 2016, 4, 290. [Google Scholar] [CrossRef] [Green Version]
- Wang, X.; Che, Z.; Jiang, B.; Xiao, N.; Yang, K.; Tang, J.; Ye, J.; Wang, J.; Qi, Q. Robust Unsupervised Video Anomaly Detection by Multipath Frame Prediction. IEEE Trans. Neural Networks Learn. Syst. 2021, 33, 2301–2312. [Google Scholar] [CrossRef]
- Minhas, M.; Zelek, J. Defect detection using deep learning from minimal annotations. In Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2020), Valeta, Malta, 27 February 2020. [Google Scholar]
- Kumar, A.; Kalita, D.J.; Singh, V.P. A modern pothole detection technique using deep learning. In Proceedings of the 2nd International Conference on Data, Engineering and Applications (IDEA), Bhopal, India, 28–29 February 2020. [Google Scholar]
- Sultani, W.; Chen, C.; Shah, M. Real-world anomaly detection in surveillance videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018. [Google Scholar]
- Bibi, R.; Saeed, Y.; Zeb, A.; Ghazal, T.M.; Rahman, T.; Said, R.A.; Abbas, S.; Ahmad, M.; Khan, M.A. Edge AI-based automated detection and classification of road anomalies in VANET using deep learning. Comput. Intell. Neurosci. 2021, 2021, 6262194. [Google Scholar] [CrossRef]
- Cui, Y.; Jin, J.S.; Luo, S.; Park, M.; Au, S.S.L. Automated pattern recognition and defect inspection system. In Proceedings of the 5th International Conference on Image and Graphics, Xi’an, China, 20–23 September 2009. [Google Scholar]
- Suong, L.K.; Kwon, J. Detection of potholes using a deep convolutional neural network. J. Univers. Comput. Sci. 2018, 24, 1244–1257. [Google Scholar]
- Akagic, A.; Buza, E.; Omanovic, S.; Karabegovic, A. Pavement crack detection using Otsu thresholding for image segmentation. In Proceedings of the 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), Opatija, Croatia, 21–25 May 2018. [Google Scholar]
- Peng, L.; Chao, W.; Shuangmiao, L.; Baocai, F. Research on crack detection method of airport runway based on twice-threshold segmentation. In Proceedings of the 5th International Conference on Instrumentation and Measurement, Computer, Communication and Control (IMCCC), Qinhuangdao, China, 18–20 September 2015. [Google Scholar]
- Shankar, K.; Wang, P.; Xu, R.; Mahgoub, A.; Chaterji, S. Janus: Benchmarking commercial and open-source cloud and edge platforms for object and anomaly detection workloads. In Proceedings of the IEEE 13th International Conference on Cloud Computing (CLOUD), Beijing, China, 18–24 October 2020. [Google Scholar]
- Doshi, K.; Yilmaz, Y. Road damage detection using deep ensemble learning. In Proceedings of the IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA, 10–13 December 2020. [Google Scholar]
- Samarawickrama, Y.C.; Wickramasinghe, C.D. MATLAB based automated surface defect detection system for ceramic tiles using image processing. In Proceedings of the 6th National Conference on Technology and Management (NCTM), Malabe, Sri Lanka, 27 January 2017. [Google Scholar]
- Ayenu-Prah, A.Y.; Attoh-Okine, N.O. Evaluating Pavement Cracks with Bidimensional Empirical Mode Decomposition. EURASIP J. Adv. Signal Process. 2008, 2008, 1–7. [Google Scholar] [CrossRef] [Green Version]
- Cao, W.; Liu, Q.; He, Z. Review of Pavement Defect Detection Methods. IEEE Access 2020, 8, 14531–14544. [Google Scholar] [CrossRef]
- Zhang, Y.; Chen, C.; Wu, Q.; Lu, Q.; Zhang, S.; Zhang, G.; Yang, Y. A kinect-based approach for 3D pavement surface reconstruction and cracking recognition. IEEE Trans. Intell. Transp. Syst. 2019, 19, 3935–3946. [Google Scholar] [CrossRef]
- Silva, N.; Shah, V.; Soares, J.; Rodrigues, H. Road anomalies detection system evaluation. Sensors 2018, 18, 1984. [Google Scholar] [CrossRef] [Green Version]
- Celaya-Padilla, J.M.; Galván-Tejada, C.E.; López-Monteagudo, F.E.; Alonso-González, O.; Moreno-Báez, A.; Martínez-Torteya, A.; Galván-Tejada, J.I.; Arceo-Olague, J.G.; Luna-García, H.; Gamboa-Rosales, H. Speed bump detection using accelerometric features: A genetic algorithm approach. Sensors 2018, 18, 443. [Google Scholar] [CrossRef] [Green Version]
- Roberts, R.; Giancontieri, G.; Inzerillo, L.; Di Mino, G. Towards low-cost pavement condition health monitoring and analysis using deep learning. Appl. Sci. 2020, 10, 319. [Google Scholar] [CrossRef] [Green Version]
- Qu, Z.; Mei, J.; Liu, L.; Zhou, D.-Y. Crack detection of concrete pavement with cross-entropy loss function and improved VGG16 network model. IEEE Access 2020, 8, 54564–54573. [Google Scholar] [CrossRef]
- Mertz, C. Continuous road damage detection using regular service vehicles. In Proceedings of the ITS World Congress, Orlando, FL, USA, 16–20 October 2011. [Google Scholar]
- Medina, R.; Llamas, J.; Zalama, E.; Gómez-García-Bermejo, J. Enhanced automatic detection of road surface cracks by combining 2D/3D image processing techniques. In Proceedings of the IEEE International Conference on Image Processing (ICIP), Paris, France, 27–30 October 2014. [Google Scholar]
- Gartner, I. Gartner Hype Cycle. 2023. Available online: https://www.gartner.com/en/research/methodologies/gartner-hype-cycle (accessed on 10 May 2023).
- PRISMA. Preferred Reporting Items for Systematic Reviews and Meta-Analyses. 2020. Available online: http://www.prisma-statement.org/PRISMAStatement/Checklist.aspx (accessed on 10 January 2023).
Criteria | Included | Excluded |
---|---|---|
Date | Include the studies between 2000 and 2023 depending on the topic and availability | Exclude older versions if new versions of studies are available |
Topic | Studies that focus on ML in the context of ARDAD systems for motorist safety | Exclude studies that do not address detection in terms of computer vision or motorist safety |
Source | Scopus, IEEE, Science Direct, DOAJ, and Google Scholar | Web search of non-peer-reviewed sources, non-English publications and non-scholarly work sources. |
Peer review | Peer-reviewed conference papers, journal articles, technical reports, and web-based articles important to research questions | Studies include dissertations, thesis, posters, short papers, and abstracts. |
Research/study design | Studies focusing on video and image processing, visual defect/anomaly detection for motorist safety | Studies that do not deal with video and image processing |
Setting | Outdoor/indoor conditions with varying lighting | Permanent backgrounds or unvarying lighting conditions in surveillance scenes |
Reported Outcomes | Precise classification and detection outcome with a reasonable success rate | Unclear outcomes and below-average success rate of detection accuracy |
No. | Dataset Link | Purpose | Configuration |
---|---|---|---|
1 | Road surface anomalies kaggle.com/datasets/aminumusa/road-dataset | Detect potholes and improve road maintenance; originated from Nigerian highways | 789 good surfaces, 670 potholes images, Uniform 256 × 256 pixels |
Paper of Origin/Use: https://tinyurl.com/ydbwp39f Strength: Clear distinction between classes, real-world images Limitations: Limited variety of images specific to Nigerian highways Statistics: 14 downloads and 4 citations | |||
2 | Pothole image dataset kaggle.com/datasets/sachinpatel21/pothole-image-dataset | Detect potholes on roads | 600+ .jpg images of the potholes, web scraped |
Paper of Origin/Use: https://ieeexplore.ieee.org/abstract/document/9824637 Strength: Diverse dataset with pothole images from varied road surfaces Limitations: The collection method may have resulted in noisy or duplicate images Statistics: 3454 downloads and 2 citations | |||
3 | Debris flow data zenodo.org/record/6679461 datahub.hku.hk/articles/dataset/Dataset_and_supplementary_movies_for_geophysical_mass_flows_against_a_flexible_ring_net_barrier/20349192 | Debris and rock avalanches | 228 videos in total debris caused by an avalanche |
Paper of Origin/Use: https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2022JF006870 Strength: Detailed and high-quality data covers a range of scenarios Limitations: Some of the samples from the laboratory setting may not reflect real-world conditions Statistics: 145 downloads and 1 citation | |||
4 | Pothole and road images kaggle.com/datasets/virenbr11/pothole-and-plain-rode-images | Road defects and pothole detection | 740 images of road potholes |
Paper of Origin/Use: https://www.hindawi.com/journals/cin/2021/6262194/ Strength: A proper train-test split for potential applications in training ML models for road maintenance, traffic management, and autonomous vehicle navigation systems Limitations: Images scraped from the web may result in inconsistencies. Statistics: 1528 downloads and 3 citations | |||
5 | On-road anomalies and obstacles segmentmeifyoucan.com/datasets | Providing pixel-level annotations for the classification of anomalies and other hazardous obstacles | 467 labelled and unlabeled images of on-road anomalies and obstacles |
Paper of Origin/Use: https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/hash/d67d8ab4f4c10bf22aa353e27879133c-Abstract-round2.html Strength: The dataset builds upon the popular Cityscapes dataset, making it useful for training ML models for anomaly detection and obstacle identification in urban scenes Limitations: Low anomaly diversity in datasets limits generalisation and may cause overfitting Statistics: Unknown number of downloads and 34 citations | |||
6 | Speed hump/bump dataset data.mendeley.com/datasets/xt5bjdhy5g/1 | Detecting and classifying speed humps/bumps in real-world conditions to improve real-time applications such as self-driving cars | Total 3000 .jpg images of humps/bumps with varying conditions, such as different types of humps/bumps, illumination conditions, and geographical locations |
Paper of Origin/Use: https://www.sciencedirect.com/science/article/pii/S1877050918320295 Strength: The dataset contains diverse speed humps and bumps under varied conditions, improving model generalisation Limitations: Limited public availability, geographical scope, manual labelling, and potential biases affecting model performance Statistics: 4530 downloads and 46 citations | |||
7 | Crack-forest dataset github.com/cuilimeng/CrackForest-dataset kaggle.com/datasets/mahendrachouhanml/crackforest | Annotated road crack image database for developing and evaluating automatic road crack detection algorithms | Collection of 118 annotated images with ground truth labelling of cracks and background pixels, used for training and testing crack detection models |
Paper of Origin/Use: https://ieeexplore.ieee.org/abstract/document/7471507 Strength: A diverse set of annotated road crack images randomly shuffled with 80%-20% splits, respectively, for training and testing crack detection models Limitations: Limited number of images Statistics: 423 downloads and 749 citations | |||
8 | Pothole dataset drive.google.com/drive/folders/1vUmCvdW3-2lMrhsMbXdMWeLcEz__Ocuy https://www.kaggle.com/datasets/felipemuller5/nienaber-potholes-1-simplex | Pothole detection | Two sets of 650 annotated pothole images, with variations in complexity and some overlapping files |
Paper of Origin/Use: https://ieeexplore.ieee.org/abstract/document/7376642 Strength: Realistic pothole images with varying real-world scenarios and comprehensive annotations in two papers for pothole detection models Limitations: The possibility of duplicate image names can be problematic Statistics: 645 downloads and 102 citations | |||
9 | Road damage dataset paperswithcode.com/dataset/rdd-2020 data.mendeley.com/datasets/5ty2wb6gvg/1 | Damaged road surface detection | A total of 26,620 .jpg images of 31,000 instances of road damage from multiple countries using smartphones |
Paper of Origin/Use: https://arxiv.org/abs/2008.13101 Strength: A large, diverse dataset with annotations helpful in developing deep learning models Limitations: NA Statistics: 2346 downloads and 76 citations | |||
10 | Road anomalies epfl.ch/labs/cvlab/data/road-anomaly | Dynamic anomaly detection | 120 images with associated per-pixel labelled unusual on-road entities such as animals, rocks, traffic cones and other obstacles |
Paper of Origin/Use: https://arxiv.org/abs/2008.13101 Strength: Realistic representation of road hazards, per-pixel labels for training, and the benchmark for evaluation Limitations: The dataset is designed for a specific purpose and may not be suitable for other applications or research topics Statistics: Unknown number of downloads and 84 citations | |||
11 | Road surface potholes sites.google.com/view/pothole-600/dataset | Pothole detection and classification | 600 RGB images and pixel-level annotations collected using a ZED stereo camera; the road disparity images were estimated using Perspective Transformation—Search Range Propagation (PT-SRP) |
Paper of Origin/Use: https://link.springer.com/chapter/10.1007/978-3-030-66823-5_17 Strength: Contains annotated images that can be used for training and testing pothole detection algorithms. Stereo camera use allows for the estimation of disparity images, which helps improve the accuracy of pothole detection Limitations: The dataset was collected using a single camera setup, which may limit its generalisability to other camera setups Statistics: Unknown number of downloads and 28 citations | |||
12 | Labelled pothole dataset public.roboflow.com/object-detection/pothole kaggle.com/datasets/chitholian/annotated-potholes-dataset | Fully annotated image dataset for pothole detection | 665 images with a total of 1740 annotated potholes. 532 (80%) training images, 133 (20%) test images. |
Paper of Origin/Use: https://link.springer.com/chapter/10.1007/978-981-16-6636-0_44 Strength: It is a fully bounding box with annotated images of potholes and damaged roads Limitations: 57.1% of images are web scraped, potentially consisting of duplicate images Statistics: 2222 downloads and 1 citation | |||
13 | Pothole detection dataset kaggle.com/datasets/atulyakumar98/pothole-detection-dataset | Road surface pothole detection | 352 undamaged road images and 329 pothole images |
Paper of Origin/Use: https://ieeexplore.ieee.org/abstract/document/9850988 Strength: Diverse dataset with annotations helpful in developing deep learning models Limitations: The small size of the dataset and class imbalance may limit the generalisability of the models trained on this dataset Statistics: 4530 downloads and 8 citations | |||
14 | Road infrastructure defect dataset kaggle.com/datasets/aniruddhsharma/structural-defects-network-concrete-crack-images | Detecting cracks in the bridge decks, walls, and concrete pavements | 56,000 images of cracked and non-cracked surfaces |
Paper of Origin/Use: https://www.mdpi.com/2412-3811/7/9/107 Strength: Provides a variety of obstructions, such as shadows, surface roughness, scaling, edges, holes, and background debris Limitations: Limited to surface cracks only Statistics: 2438 downloads and 14 citations | |||
15 | Concrete bridge defects zenodo.org/record/2620293 | Concrete bridge surface defect detection | 6900 images of the defective concrete surface of 30 unique bridges, including cracks (2507), spallation (1898), efflorescence (833), exposed bars (1507) and corrosion stain (1559) |
Paper of Origin/Use: https://arxiv.org/abs/1904.08486 Strength: High-resolution images with defects in the context of 30 unique bridges and the use of a multi-stage annotation process resulting in a multilabel dataset with six categories of defects Limitations: Varied aspect ratios, scales, and resolutions of defects and even bounding boxes overlap Statistics: 27,510 downloads and 80 citations | |||
16 | Road anomaly benchmark github.com/adynathos/road-anomaly-benchmark | Anomalous object detection in autonomous driving and road traffic safety | 552 high-definition images of road anomalies and obstacles |
Paper of Origin/Use: https://arxiv.org/abs/2104.14812 Strength: Provides pixel-level annotations for identifying unseen anomalous objects and hazardous obstacles within diverse scenes Limitations: NA Statistics: Unknown number of downloads and 34 citations | |||
17 | Pothole detection datasets github.com/ruirangerfan/stereo_pothole_datasets | Pothole detection | 220 images of potholes captured using ZED stereo camera |
Paper of Origin/Use: https://ieeexplore.ieee.org/abstract/document/8809907 Strength: Contains four datasets with disparity maps, designed for pothole detection and published in a reputable journal Limitations: NA Statistics: Unknown number of downloads and 107 citations | |||
18 | pNEUMA Vision Dataset zenodo.org/record/7426506 | On-road anomaly detection | Urban trajectory 35K video frames captured using 18 swarm drones |
Paper of Origin/Use: https://www.sciencedirect.com/science/article/pii/S0968090X22003795 Strength: Extensive urban trajectory data to investigate traffic phenomena at different scales; provides comprehensive urban trajectory data Limitations: Trajectory data are limited to vehicle movement and do not include other factors such as weather, road conditions, or pedestrian behaviour Statistics: 417 downloads and 1 citation |
Research Focus | Research Areas and Applications | Reference |
---|---|---|
Inspection, defect detection, structural damage, crack detection, ML | Surface defect and damage detection | [8,9,10,13,14,28,62] |
Statistical learning, DL, intelligent environments | Anomaly detection, analysis and prediction | [12,16,17,18,20,37,63,64,65,66,67,68,69,70] |
Ref. | Detection Origin | Data Acquisition | Algorithm | Evaluation Method | Acc. (%) | Future Implications | Strength | Limitations |
---|---|---|---|---|---|---|---|---|
Kim, Anagnostopoulos [40] | On-road anomaly | Cameras mounted on a swarm of drones | Butterworth filter, pNEUMA Vision (Dataset 18) | Binary classification, neural network optimisation, precision evaluation | 91.8 | Improved traffic flow models, enhanced safety analytics, and lane-change detection | Enhanced features, diverse urban traffic use-cases | Bounding box errors, disrupted visibility, tracking failures |
Julio-Rodríguez, Rojas-Ruiz [49] | Road surface defects | Vehicle-mounted sensors | KNN, SVM, and RF | Real-world tests on prediction time and classification score | 93.20 | Improved autonomous driving, energy optimisation, and enhanced vehicle safety | A novel method, real-world tests | Limited feature applicability, idealised scenarios unsuitable for real-time |
Ferjani et al., 2022 [50] | Road surface anomalies | Lab simulations and vehicle accelerometer axis data | SVM, decision tree, and MLP | Efficacy of the ML approach using practical, real-world data | 94.00 | Improved road monitoring, enhanced traffic safety, and reduced accidents | Thorough analysis, practical advice, peer-reviewed, impactful | Feature sensitivity, limited generalisability, domain separation inefficiency |
Bustamante et al., 2022 [81] | Road anomalies | GPS, accelerometer, gyroscope, camera | Supervised KNN and ANN | Fog-computing, V2I network using ML algorithms, comparing roughness against a flat reference | 95.55 | 5G-based scalable smart urban mobility, and public spending efficiency | Innovative, data-driven, sustainable urban mobility solution | Data accuracy, privacy and security concerns due to sensors installed inside vehicles |
Zhou et al., 2022 [82] | Road Surface Condition | Smartphone camera, accelerator and gyroscope | SVM, KNN, naïve Bayes, decision tree, and RF | Average precision, loss, recall, F1-measure, and accuracy | 86.3 | Crowdsourcing-based detection system based on motorist feedbacks | High efficiency, low cost, and easy collection of data | Lower accuracy than professional equipment and is affected by shadows, road markings, reflections, and driving habits |
Alam et al., 2021 [31] | Debris object detection | Unmanned aerial vehicle (UAV)-mounted cameras | SSD and R-CNN | Mean average precision (mAP) and mean average recall (mAR) scores | 88.3 | A UAV-based fast and affordable debris detection model for urban planning | The UAV-based method improves road debris clean-up, optimises traffic safety operations | Negative drone distance impact, limited road type scope, detection accuracy affected by environmental factors |
Alipour et al., 2020 [83] | Crack detection | Images of diverse road surfaces based on material | ResNet 18-layer, ensemble learning | Accuracy, precision, recall, true negative rate, and F1 score calculated from a confusion matrix | 97.8 | Construction material independent crack detection model | Robustness of DL methods across various road surface materials | Limited defect types, domain adaptation challenges |
Taxonomy | Research Focus | Year | Ref. | Acc. (%) | Research Areas and Applications | Implications | Limitations |
---|---|---|---|---|---|---|---|
Traditional ML and statistical methods | Road surface crack, white line, joint detection | 2000 | [38] | 92.8% | Morphology operations for detecting road surface detects to safeguard traffic safety | Automating defect detection | Depend on image quality and require setting parameters |
Lane curvature detection for motorist assistance | 2003 | [88] | 99% | Lane curve and edge detection using a novel image-processing algorithm | Lane departure warning and lateral control system for vehicle control | The proposed algorithm is not effective for road elevations over 2% | |
Mobile robot for tunnel crack detection | 2007 | [89] | NA | Image processing, edge detection, graph search, Dijkstra’s algorithm, expert feedback based | A semi-automated platform for future research in defect detection | Validated in indoor experimental settings with limited application | |
Road surface condition recognition | 2009 | [90] | 90% | Road surface condition identification system for motorist safety | Enhancing vehicle active safety features by identifying road surface conditions | Requires extensive vehicle testing for index distribution on road surfaces | |
Pavement crack detection | 2010 | [78] | NA | Pavement edge detection, Canny operator, Mallat wavelet transform, quadratic optimisation | Improving pavement edge detection for faster road repairs to increase road safety | Interference from pavement markings needs further research to counter noise | |
Road surface crack detection | 2018 | [79] | 90.87% | ML-based 2D road surface image analysis from the driver’s viewpoint, crack detection, surface defect detection | A platform for cost-efficient, scalable road inspection systems to improve traffic safety | Inefficient in handling varied lighting, shadows, texture, and surface types in image analysis | |
Pavement crack detection | 2022 | [91] | 86% | Tile-based image processing method to automate the detection of cracks from 2D and 3D images of pavement and asphalt concrete surface | A platform for an automated pavement distress assessment system, reducing costs and improving the integrity | Limited crack detection, 3D image inconsistencies, false positives, threshold reliance, and width measurement issues | |
Deep learning | Automatic crack detection on a concrete bridge surface | 2011 | [35] | 90.25% | Image processing, backpropagation neural network, construction safety and management | Automated crack detection system for efficient analysis and visualisation of concrete surface cracks | Performed under similar environmental conditions and needs further evaluation; the accuracy score could be improved |
Road survey for crack detection | 2016 | [55] | 89.65% | ConvNet trained on square image patches, handcrafted feature extraction methods | A platform to build a low-cost, real-time road crack detection system | Misclassification errors in detecting cracks in some of the methods | |
Unsupervised multi-scale image fusion | 2018 | [74] | 80.7% | Automated airport runaway inspection using crack detection by multi-scale image fusion | Efficient maintenance of road infrastructures through integration within intelligent autonomous inspection systems | Issues with identical infrastructures, GPS integration and lack of real-time application support | |
Road surface cracks and defect detection | 2020 | [92] | 91.99% | Transposed convolution layer, connectivity of pixels, and densely connected layers | An automated solution for detecting cracks in roads and bridges | Poor performance with low-speed cameras; low light conditions affect performance | |
Detection of long and complicated pavement cracks | 2023 | [48] | 94.60% | Swin-transformer-based semantic segmentation method with multi-layer perceptron | Improved pavement crack detection, leading to effective maintenance strategies and traffic infrastructure systems | Heavy noise and fallen leaves coupling effect; limited experimental real-world data |
Research Focus | Year | Ref. | Acc. (%) | Research Areas and Application | Implications | Limitations |
---|---|---|---|---|---|---|
Automatic crack detection and classification | 2009 | [98] | 94.8% and 95.6% | Entropy, road crack segmentation and dynamic image thresholding | A platform for improved defect detection with cost-effective, objective rehabilitation decision support to increase traffic safety | Potential for improvements in dynamic thresholding accuracy and processing of variance in pixel intensity |
Image processing for pothole detecting | 2015 | [99] | 77.9% | Pothole detection using simple real-world images, Canny filter and contour detection | A device for vehicles that detect potholes, alerts drivers, and log pothole locations for road maintenance agencies | Limited detection range, potential for absorption of potholes into outer borders, and inability to detect potholes with no visible edges |
Crack detection on two-dimensional pavement images | 2016 | [100] | 83% | Crack detection, minimal path, Dijkstra algorithm, road surface condition analysis | An unsupervised learning algorithm for effective assessments in road in road maintenance | Potential bias due to the use of the same Dijkstra algorithm and high computation time requiring optimisation for faster processing |
Road defect detection | 2016 | [101] | 54% to 91% | Road defect detection, image processing, computer vision | A real-time road defect detection system for timely road repair and traffic safety | Issues with road colour affecting accuracy rates, real-time constraints, difficulty detecting thin cracks |
Road condition detection system | 2018 | [43] | 87% | Road Weather Information System (RWiS), Intelligent Transportation System (ITS) | Improving traffic safety by enabling autonomous cars to avoid road anomalies and control speed based on road condition | Issues with background noise filtering resulting in object shadows being detected as cracks |
Automatic road crack segmentation | 2020 | [77] | 99.11% | Morphological filter dynamic thresholding, entropy thresholding | A high-performance model for crack detection | The model presented does not address characterising crack severity |
Research Focus | Year | Acc. (%) | Ref. | Research Areas and Application | Implications | Limitations |
---|---|---|---|---|---|---|
Wildlife-vehicle collision analysis and hotspot prediction | 2006 | High p-value 0.463 | [1] | Linear nearest neighbour analysis, Ripley’s K analysis, visual analysis | Identifying hotspots to aid transportation agencies to mitigate wildlife-vehicle collision | Limited collision hotspot data for the initial method improvement |
Detection and counting of potholes | 2016 | 83.18% | [80] | K-means clustering-based segmentation, image processing, edge detection, identification, segmentation | A standalone application for pothole detection using hybrid classifiers | The study provides an analysis but no solution for pothole detection |
Detecting road hazards to help self-driving vehicles | 2016 | TPR of 63% | [29] | DBSCAN, image processing using stereo-based baseline methods, clustering | Improving self-driving vehicles to detect small road hazards and help decrease accidents caused by road debris could be reduced | Limited to stereo-based methods and specific datasets, missing real-world scenarios |
Pothole and hump detection | 2018 | NA | [76] | Internet of Things (IoT)-based road-monitoring system, honeybee optimisation (HBO), cloud-based real-time image processing | Improving traffic safety via timely alerts for motorists and facilitation of road maintenance | Study only tested two-speed scenarios (40 km and 60 km); real-world implementation not assessed |
Road anomaly detection | 2020 | F1 of 66.7% to 92.1% | [94] | Threshold detection, sliding window, KNN dynamic time warping | Large-scale data integration to create city-wide anomaly maps | Sensitive to noise, does not fully represent diverse road conditions |
Video surveillance, anomaly detection | 2011 | F1 of 55% | [39] | Particle-based tracking, a cascade of HMM and HDP-HMM models | A solution working on less structured CCTV footage, such as videos of metro systems | Manual parameter setting, inability to distinguish pedestrian and vehicle activities |
Real-world surveillance video anomaly detection | 2018 | Approx. 95% (Accidents) | [105] | Multiple instance learning (MIL), deep MIL ranking model, temporal segmentation | Improved anomaly detection by reduced reliance on manual annotations and enhanced real-world anomalous activity recognition capabilities | Potential false positives, reliance on weakly labelled data, computational complexity, and dataset diversity constraints |
Road surface monitoring | 2018 | 97% | [54] | Support vector machine, hidden Markov model (HMM) and residual network (ResNet) | Enhanced road monitoring and maintenance through smartphone-based data collection and analysis | Limited dataset, manual labelling, vehicle and smartphone variability, lack of road roughness estimation |
Anomalies detection for autonomous vehicles | 2021 | 99.21% | [106] | Image processing, AI-based edge computing for vehicular ad hoc network (VANET) | The scalable edge computing AI-based framework could improve traffic safety and autonomous driving by providing real-time road information | Limited dataset collected from online sources and the need to incorporate more types of road anomalies |
Traditional ML | Deep Learning | |
---|---|---|
Strength |
|
|
Weakness |
|
|
Research Focus | Year | Ref. | Acc. (%) | Research Area and Application | Implications | Limitations |
---|---|---|---|---|---|---|
Pavement surface reconstruction for crack recognition | 2018 | [116] | 78.27% | Microsoft Kinect fusion for crack detection, surface reconstruction for pavement serviceability analysis | Upgraded Kinect hardware and expanded data sources for enhanced pavement assessments and traffic safety | Limited hardware capabilities and Kinect field-of-view constraints hinder capabilities |
Road crack and pothole detection | 2018 | [21] | 98.93% | Using texture-based features to differentiate between crack surfaces and sound roads | Enhanced road monitoring and maintenance, reduced accidents, and improved navigation for autonomous vehicles | Inefficient restoration patches detection, issues with shadow, occlusions, and camera resolution limitations |
Road anomaly detection | 2018 | [117] | 82.51% | Principal component analysis, Fi-Ware, data mining, collaborative mobile sensing | Improve data acquisition standardisation, sensor diversity, and merging long/short bump classes to enhance real-world performance | Decreased performance in real-world conditions, data standardisation and complexity reduction not effective |
Speed bump detection | 2019 | [118] | 97.14% | Self-driving cars, artificial vision, GPS tracking | Real-time road surface monitoring, smart route optimisation, reduced fuel consumption, and continuous updates of road quality | Model uses both accelerometer and gyro data; improved performance with only one source not yet achieved |
Anomaly detection | 2019 | [17] | NA | Smart objects, intelligent transportation systems, industrial systems | Prediction/prevention and exploring data fusion techniques | Limited data access, focus on normal behaviour, high-dimensionality issues |
Asphalt pavement crack classification | 2019 | [86] | 87.50% | Asphalt pavement, crack classification, image processing, steerable filters | Image processing methods to assess crack properties, including depth and severity | Limited crack types, small image dataset, unexplored crack properties |
Road crack detection | 2019 | [84] | 98.70% | Deep learning and adaptive image segmentation | A deep neural network trained to segment positive images into semantically meaningful regions, i.e., cracks and road surface | Difficulty in properly segmenting colour images with a large number of noisy pixels |
Low-cost pavement condition health monitoring | 2020 | [119] | 93.55% | Automated detection of road pavement distresses, low-cost DL technologies | Platform for an integrated approach towards optimising urban pavement management systems | Region-specific model with manual data collection dependence, no detailed quantitative assessments |
Road damage detection | 2020 | [96] | 56.5% | EfficientDet model for crack and object detection | Improving results by setting ground rules for annotating and expanding datasets by installing cameras with optimal orientation | Limitations of the study include false positive and negative detections, misclassifications between diagonal crack classes |
Road surface monitoring and pothole detection | 2020 | [22] | 85% and 93% | Deep learning, road surface monitoring, pothole detection, crowd sensing | Adaptive method to analyse the additional type of road surfaces and apply end-user driving profiles | Limited road surfaces analysed, controlled scenarios in related works, incomplete automatic threshold adjustments |
Asphalt pavement crack detection | 2020 | [71] | 70% | YOLOV3-based asphalt pavement crack and pothole detection | Improving road maintenance efficiency and reduce infrastructure costs using AI-driven analysis | Limited by data geographical scope and weather variations, human judgment discrepancy affecting model performance |
Pavement distress and health monitoring | 2020 | [103] | 89.14%, 97.66% | Road pavement distresses detection, minimal annotations learning | Suggested trends and future work include exploring activation functions, selective layer freezing, transfer learning and different CNN architectures | Limited real-world anomalous samples and potential impact of activation function choice in transfer learning |
Modern Pothole detection technique | 2020 | [104] | NA | TensorFlow API, transfer learning, road inspection automation | Improved CNN architectures, GPS-enabled systems and Android apps, deployment on Raspberry Pi or Android devices | Limited detection in varying conditions, computational complexity, generalisability issues and integration challenges |
Object and anomaly detection | 2020 | [111] | 59.11% | Amazon Rekognition, Azure Cognitive Services and Google Vision | Choosing cloud platforms and edge devices for IoT applications based on performance and cost trade-offs for improved traffic safety | Limited number of tested platforms and the lack of real-world deployment |
Crack detection of concrete pavement | 2020 | [120] | 90.1% | Crack detection, cross-entropy loss function, VGG16 network, crack classification | Integration of algorithms with other technologies, such as drones and robots for automated inspection and maintenance | Lack of external validity in terms of deploying the proposed algorithm in real-world scenarios |
Road damage detection | 2020 | [42] | 66% | Ensemble learning, object detection, urban street analysis | An automated solution for road damage detection and classification using image analysis for smart city applications | Lack of diversity in the training dataset and limited evaluation of real-world scenarios |
Road damage detection | 2020 | [112] | 63.58% | Image classification, object detection, and ensemble models | Improving road safety and developing better road damage detection systems using smartphone and vehicle-mounted cameras | The need for high-quality data and the impact of input image size on detection performance |
Predicting the accuracy of asphalt concrete pavement | 2021 | [51] | NA | AdaBoost regression, International Roughness Index (IRI), Mechanistic-Empirical Pavement Design Guide (MEPDG) | Improving pavement design, understanding of influencing factors (including reported variables analysis) and optimising costs of road maintenance | Limitations include data bias, overfitting, lack of interpretability, and generalisation to new contexts |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Rathee, M.; Bačić, B.; Doborjeh, M. Automated Road Defect and Anomaly Detection for Traffic Safety: A Systematic Review. Sensors 2023, 23, 5656. https://doi.org/10.3390/s23125656
Rathee M, Bačić B, Doborjeh M. Automated Road Defect and Anomaly Detection for Traffic Safety: A Systematic Review. Sensors. 2023; 23(12):5656. https://doi.org/10.3390/s23125656
Chicago/Turabian StyleRathee, Munish, Boris Bačić, and Maryam Doborjeh. 2023. "Automated Road Defect and Anomaly Detection for Traffic Safety: A Systematic Review" Sensors 23, no. 12: 5656. https://doi.org/10.3390/s23125656