An Overview of Challenges Associated with Automatic Detection of Concrete Cracks in the Presence of Shadows
Abstract
:1. Introduction
Paper | Feature Enhancement/ Pre-Processing | Classification of Fracture Dimensions | Type of Neural Network | Open-Source | Image Resolution | Filter Size | Filter Type | Images in the Dataset | Accuracy, % |
---|---|---|---|---|---|---|---|---|---|
[24] | N | N | CNN | VGG16 | 256 × 256 | 5 × 5 | 3500 | 92.3 | |
[8] | Y | Y | CNN | N/A | 227 × 227 | 40,000 | 99.9 | ||
[14] | Y | Y | ANN vs. Fuzzy | N/A | 256 × 256 | Object Detection | 205 | 96.1 | |
[16] | CNN vs. SVM vs. Boost | ConvNet | 227 × 227 | 40,000 | 99.7 | ||||
[9] | Y | CNN | N/A | 256 × 256 | 40,000 | 98 | |||
[15] | Y | Image Segmentation Approach | No | N/A | Automatic Peak Detection | N/A | N/A | ||
[19] | N | N | KNN vs. Decsion Tree vs. SVM vs. Ranorest | 227 × 227 | 20,000 | ||||
[4] | Y | Y | No (Review Paper) | N/A | N/A | N/A | N/A | N/A | |
[20] | Y | N | CNN | AlexNet, LeNet-5, CIFAR10 | 1024 × 1024 | 32 × 32 | 94 | ||
[21] | Y | N | CNN | N/A | N/A | N/A | ASPP | N/A | 96.7 |
[23] | N/A | N/A | CNN-LSTM | N/A | N/A | N/A | N/A | N/A | |
[22] | N/A | N/A | New CNN vs. R-CNN vs. U-Net | N/A | 576 × 576 | N/A | N/A | N/A | 82.5 |
[17] | N | N | FCN | VGG16 | 227 × 227 | 40,000 | 90 | ||
[18] | N/A | N/A | CNN | AlexNet | 3120 × 4160 | N/A | 60,000 | 99.06 | |
[13] | Y | N | Backpropagation NN | N/A | N/A | N/A | N/A | 225 | 92 |
[25] | N | Y | R-CNN | 376 | |||||
[11] | N/A | N | U-Net | N/A | N/A | N/A | N/A | N/A | 78.12 |
[26] | AlexNet, VGGNet13, and ResNet18 | 227 × 227 | 10,000 | ||||||
[27] | N | N | CNN | N/A | 16 × 16 | N/A | N/A | 64,000 | 93 |
[28] | Y | N | SVM | N/A | N/A | N/A | N/A | N/A | 94 |
[29] | N | N | ANN | N/A | N/A | N/A | N/A | N/A | N/A |
2. The Difficulties of Automatic Concrete-Crack Detection in the Presence of Shadow Effects
- Training dataset—the dataset published in [40], consisting of 40,000 concrete images (227 × 227 pixels with RGB channels) with and without cracks. As mentioned above, this database has already been used by several researchers for the development of deep-learning methods for concrete-crack detection (see Table 1). Sample images of each type are displayed in the Figure 1. Note that no shadow effects are present in the images included in this dataset.
- Testing dataset—the dataset comprising 500 concrete images (227 × 227 pixels with RGB channels) with and without cracks with the shadow effects present in all images. Samples images of each type are depicted in the Figure 2.
3. The Challenges Associated with the Application of Shadow Removal Techniques
4. Discussion
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
NN | Neural network |
ANN | Artificial neural network |
CNN | Convolutional neural network |
R-CNN | Region-based convolutional neural network |
DSN | Deeply-supervised net |
LSTM | Long short-term memory |
SVM | Support vector machine |
KNN | K-nearest neighbors |
FCN | Fully convolutional network |
UAV | Unmanned aerial vehicle |
DSN | Deeply-supervised net |
References
- Budiansky, B.; O’connell, R.J. Elastic moduli of a cracked solid. Int. J. Solids Struct. 1976, 12, 81–97. [Google Scholar] [CrossRef]
- Aboudi, J. Stiffness reduction of cracked solids. Eng. Fract. Mech. 1987, 26, 637–650. [Google Scholar] [CrossRef]
- Dhital, D.; Lee, J.R. A fully non-contact ultrasonic propagation imaging system for closed surface crack evaluation. Exp. Mech. 2012, 52, 1111–1122. [Google Scholar] [CrossRef]
- Mohan, A.; Poobal, S. Crack detection using image processing: A critical review and analysis. Alex. Eng. J. 2018, 57, 787–798. [Google Scholar] [CrossRef]
- Nnolim, U.A. Partial Differential Equation-Based Enhancement and Crack Detection. Math. Probl. Eng. 2019, 2019, 8157205. [Google Scholar] [CrossRef]
- Nnolim, U.A. Fully adaptive segmentation of cracks on concrete surfaces. Comput. Electr. Eng. 2020, 83, 106561. [Google Scholar] [CrossRef]
- Nnolim, U.A. Automated crack segmentation via saturation channel thresholding, area classification and fusion of modified level set segmentation with Canny edge detection. Heliyon 2020, 6, e05748. [Google Scholar] [CrossRef] [PubMed]
- Kim, J.J.; Kim, A.R.; Lee, S.W. Artificial Neural Network-Based Automated Crack Detection and Analysis for the Inspection of Concrete Structures. Appl. Sci. 2020, 10, 8105. [Google Scholar] [CrossRef]
- Cha, Y.J.; Choi, W.; Büyüköztürk, O. Deep learning-based crack damage detection using convolutional neural networks. Comput.-Aided Civ. Infrastruct. Eng. 2017, 32, 361–378. [Google Scholar] [CrossRef]
- Jang, K.; Kim, N.; An, Y.K. Deep learning–based autonomous concrete crack evaluation through hybrid image scanning. Struct. Health Monit. 2019, 18, 1722–1737. [Google Scholar] [CrossRef]
- Andrushia, A.D.; Lubloy, E. Deep Learning based Thermal Crack Detection on Structural Concrete Exposed to Elevated Temperature. Adv. Struct. Eng. 2021, 24, 1369433220986637. [Google Scholar] [CrossRef]
- Ali, L.; Alnajjar, F.; Jassmi, H.A.; Gochoo, M.; Khan, W.; Serhani, M.A. Performance Evaluation of Deep CNN-Based Crack Detection and Localization Techniques for Concrete Structures. Sensors 2021, 21, 1688. [Google Scholar] [CrossRef]
- Moon, H.G.; Kim, J.H. Intelligent crack detecting algorithm on the concrete crack image using neural network. In Proceedings of the 28th International Symposium on Automation and Robotics in Construction (ISARC), Seoul, Korea, 29 June–2 July 2011; pp. 1461–1467. [Google Scholar]
- Choudhary, G.K.; Dey, S. Crack detection in concrete surfaces using image processing, fuzzy logic, and neural networks. In Proceedings of the 2012 IEEE Fifth International Conference on Advanced Computational Intelligence (ICACI), Nanjing, China, 18–20 October 2012; pp. 404–411. [Google Scholar]
- Dinh, T.H.; Ha, Q.P.; La, H.M. Computer vision-based method for concrete-crack detection. In Proceedings of the 2016 14th International Conference on control, Automation, Robotics and Vision (ICARCV), Phuket, Thailand, 13–15 November 2016; pp. 1–6. [Google Scholar]
- Chen, K.; Yadav, A.; Khan, A.; Meng, Y.; Zhu, K. Improved crack detection and recognition based on convolutional neural network. Model. Simul. Eng. 2019, 2019, 8796743. [Google Scholar] [CrossRef] [Green Version]
- Dung, C.V. Autonomous concrete-crack detection using deep fully convolutional neural network. Autom. Constr. 2019, 99, 52–58. [Google Scholar] [CrossRef]
- Li, Y.; Zhang, H.; Wang, S.; Wang, H.; Li, J. Image-based underwater inspection system for abrasion of stilling basin slabs of dam. Adv. Civ. Eng. 2019, 2019, 6924976. [Google Scholar] [CrossRef]
- Jitendra, M.S.; Srinivasu, P.N.; Shanmuk Srinivas, A.; Nithya, A.; Kandulapati, S.K. Crack detection on concrete images using classification techniques in machine learning. J. Crit. Rev. 2020, 7, 1236–1241. [Google Scholar]
- Cao, W.; Liu, Q.; He, Z. Review of pavement defect detection methods. IEEE Access 2020, 8, 14531–14544. [Google Scholar] [CrossRef]
- Vignesh, R.; Narenthiran, B.; Manivannan, S.; Murugan, R.A.; RajKumar, V. Concrete Bridge Crack Detection Using Convolutional Neural Network. In Materials, Design, and Manufacturing for Sustainable Environment; Springer: Berlin/Heidelberg, Germany, 2021; pp. 797–812. [Google Scholar]
- Zhang, C.; Chang, C.C.; Jamshidi, M. Simultaneous pixel-level concrete defect detection and grouping using a fully convolutional model. Struct. Health Monit. 2021, 20, 1475921720985437. [Google Scholar] [CrossRef]
- Zhang, Y.; Yuen, K.V. Crack detection using fusion features-based broad learning system and image processing. Comput.-Aided Civ. Infrastruct. Eng. 2021, 36, 1568–1584. [Google Scholar] [CrossRef]
- Da Silva, W.R.L.; de Lucena, D.S. Concrete cracks detection based on deep learning image classification. Proceedings 2018, 2, 489. [Google Scholar] [CrossRef] [Green Version]
- Kim, B.; Cho, S. Image-based concrete crack assessment using mask and region-based convolutional neural network. Struct. Control Health Monit. 2019, 26, e2381. [Google Scholar] [CrossRef]
- Yang, C.; Chen, J.; Li, Z.; Huang, Y. Structural Crack Detection and Recognition Based on Deep Learning. Appl. Sci. 2021, 11, 2868. [Google Scholar] [CrossRef]
- Puruncajas, B.; Vidal, Y.; Tutivén, C. Damage Detection and Diagnosis for Offshore Wind Foundations. In Proceedings of the 17th International Conference on Informatics in Control, Automation and Robotics (ICINCO), Paris, France, 7–9 July 2020; pp. 181–187. [Google Scholar]
- O’Byrne, M.; Ghosh, B.; Pakrashi, V.; Schoefs, F. Image Processing based Damage Detection of Offshore Infrastructural Elements using Texture Information. In Proceedings of the 5th European Conference on Structural Control (EACS 2012), Genoa, Italy, 18–20 June 2012. [Google Scholar]
- Guo, J.; Wu, J.; Guo, J.; Jiang, Z. A damage identification approach for offshore jacket platforms using partial modal results and artificial neural networks. Appl. Sci. 2018, 8, 2173. [Google Scholar] [CrossRef] [Green Version]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet classification with deep convolutional neural networks. Commun. ACM 2017, 60, 84–90. [Google Scholar] [CrossRef]
- Zou, Q.; Zhang, Z.; Li, Q.; Qi, X.; Wang, Q.; Wang, S. DeepCrack: Learning Hierarchical Convolutional Features for Crack Detection. IEEE Trans. Image Process. 2019, 28, 1498–1512. [Google Scholar] [CrossRef] [PubMed]
- Liu, Y.; Yao, J.; Lu, X.; Xie, R.; Li, L. DeepCrack: A deep hierarchical feature learning architecture for crack segmentation. Neurocomputing 2019, 338, 139–153. [Google Scholar] [CrossRef]
- Padalkar, M.G.; Beltrán-González, C.; Bustreo, M.; Bue, A.D.; Murino, V. A Versatile Crack Inspection Portable System based on Classifier Ensemble and Controlled Illumination. In Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy, 10–15 January 2021. [Google Scholar]
- Finlayson, G.D.; Hordley, S.D.; Drew, M.S. Removing shadows from images. In Proceedings of the European Conference on Computer Vision, Copenhagen, Denmark, 28–31 May 2002; Springer: Berlin/Heidelberg, Germany, 2002; pp. 823–836. [Google Scholar]
- Finlayson, G.D.; Hordley, S.D.; Lu, C.; Drew, M.S. On the removal of shadows from images. IEEE Trans. Pattern Anal. Mach. Intell. 2006, 28, 59–68. [Google Scholar] [CrossRef]
- Murali, S.; Govindan, V.; Kalady, S. A survey on shadow detection techniques in a single image. Inf. Technol. Control 2018, 47, 75–92. [Google Scholar] [CrossRef] [Green Version]
- Khan, S.H.; Bennamoun, M.; Sohel, F.; Togneri, R. Automatic shadow detection and removal from a single image. IEEE Trans. Pattern Anal. Mach. Intell. 2015, 38, 431–446. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Wang, J.; Li, X.; Yang, J. Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 1788–1797. [Google Scholar]
- Fan, H.; Han, M.; Li, J. Image shadow removal using end-to-end deep convolutional neural networks. Appl. Sci. 2019, 9, 1009. [Google Scholar] [CrossRef] [Green Version]
- Özgenel, Ç.F. Concrete crack images for classification. Mendeley Data 2018, 1. [Google Scholar] [CrossRef]
- Iandola, F.N.; Han, S.; Moskewicz, M.W.; Ashraf, K.; Dally, W.J.; Keutzer, K. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size. arXiv 2016, arXiv:1602.07360. [Google Scholar]
- Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar]
- Hicklin, J. Videos and Webinars. Retrieved from Mathworks Matlab. 2021. Available online: https://www.mathworks.com/videos/deep-learning-with-matlab-transfer-learning-in-10-lines-of-matlab-code-1487714838381.html (accessed on 4 February 2021).
- Murali, S.; Govindan, V. Shadow detection and removal from a single image using LAB color space. Cybern. Inf. Technol. 2013, 13, 95–103. [Google Scholar] [CrossRef] [Green Version]
- Zosso, D.; Tran, G.; Osher, S. A unifying retinex model based on non-local differential operators. In Proceedings of the Computational Imaging XI. International Society for Optics and Photonics, Burlingame, CA, USA, 3–7 February 2013; Volume 8657, p. 865702. [Google Scholar]
- Zosso, D.; Tran, G.; Osher, S.J. Non-Local Retinex—A Unifying Framework and Beyond. SIAM J. Imaging Sci. 2015, 8, 787–826. [Google Scholar] [CrossRef]
- Gong, H.; Cosker, D. Interactive removal and ground truth for difficult shadow scenes. JOSA A 2016, 33, 1798–1811. [Google Scholar] [CrossRef] [Green Version]
- Cun, X.; Pun, C.M.; Shi, C. Towards Ghost-free Shadow Removal via Dual Hierarchical Aggregation Network and Shadow Matting GAN. arXiv 2019, arXiv:1911.08718. [Google Scholar] [CrossRef]
- Yang, W.; Guo, W.; Peng, K.; Liu, L. Research on Removing Shadow in Workpiece Image Based on Homomorphic Filtering. Procedia Eng. 2012, 29, 2360–2364. [Google Scholar] [CrossRef] [Green Version]
- Fu, L.; Zhou, C.; Guo, Q.; Juefei-Xu, F.; Yu, H.; Feng, W.; Liu, Y.; Wang, S. Auto-exposure Fusion for Single-image Shadow Removal. arXiv 2021, arXiv:2103.01255. [Google Scholar]
- Hu, X.; Zhu, L.; Fu, C.W.; Qin, J.; Heng, P.A. Direction-Aware Spatial Context Features for Shadow Detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 18–23 June 2018; pp. 7454–7462. [Google Scholar]
- Hu, X.; Fu, C.W.; Zhu, L.; Qin, J.; Heng, P.A. Direction-Aware Spatial Context Features for Shadow Detection and Removal. IEEE Trans. Pattern Anal. Mach. Intell. 2019, 42, 2795–2808. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Neural Network | Prediction Time (in min) |
---|---|
AlexNet | 1490 |
SqueezeNet | 315 |
VGG16 | 10,800 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Pal, M.; Palevičius, P.; Landauskas, M.; Orinaitė, U.; Timofejeva, I.; Ragulskis, M. An Overview of Challenges Associated with Automatic Detection of Concrete Cracks in the Presence of Shadows. Appl. Sci. 2021, 11, 11396. https://doi.org/10.3390/app112311396
Pal M, Palevičius P, Landauskas M, Orinaitė U, Timofejeva I, Ragulskis M. An Overview of Challenges Associated with Automatic Detection of Concrete Cracks in the Presence of Shadows. Applied Sciences. 2021; 11(23):11396. https://doi.org/10.3390/app112311396
Chicago/Turabian StylePal, Mayur, Paulius Palevičius, Mantas Landauskas, Ugnė Orinaitė, Inga Timofejeva, and Minvydas Ragulskis. 2021. "An Overview of Challenges Associated with Automatic Detection of Concrete Cracks in the Presence of Shadows" Applied Sciences 11, no. 23: 11396. https://doi.org/10.3390/app112311396
APA StylePal, M., Palevičius, P., Landauskas, M., Orinaitė, U., Timofejeva, I., & Ragulskis, M. (2021). An Overview of Challenges Associated with Automatic Detection of Concrete Cracks in the Presence of Shadows. Applied Sciences, 11(23), 11396. https://doi.org/10.3390/app112311396