Steel Surface Defect Diagnostics Using Deep Convolutional Neural Network and Class Activation Map
Abstract
:1. Introduction
2. Background
2.1. Feature Extraction Methods
2.1.1. Gray Level Co-Occurrence Matrix
2.1.2. Histogram of Oriented Gradients
2.2. Fundamentals of CNN and CAM
3. Proposed Method
3.1. Research Outline
3.2. Network Architecture
3.3. Parametric Measures
4. Experimental Results and Discussion
4.1. Data Description
4.2. Performance of Steel Surface Defect Classification
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Song, G.W.; Tama, B.A.; Park, J.; Hwang, J.Y.; Bang, J.; Park, S.J.; Lee, S. Temperature Control Optimization in a Steel-Making Continuous Casting Process Using Multimodal Deep Learning Approach. Steel Res. Int. 2019, 90, 1900321. [Google Scholar] [CrossRef]
- Luo, Q.; He, Y. A cost-effective and automatic surface defect inspection system for hot-rolled flat steel. Robot. Comput.-Integr. Manuf. 2016, 38, 16–30. [Google Scholar] [CrossRef]
- Ghorai, S.; Mukherjee, A.; Gangadaran, M.; Dutta, P.K. Automatic defect detection on hot-rolled flat steel products. IEEE Trans. Instrum. Meas. 2012, 62, 612–621. [Google Scholar] [CrossRef]
- He, Y.; Song, K.; Meng, Q.; Yan, Y. An End-to-end Steel Surface Defect Detection Approach via Fusing Multiple Hierarchical Features. IEEE Trans. Instrum. Meas. 2019. [Google Scholar] [CrossRef]
- Choi, W.; Huh, H.; Tama, B.A.; Park, G.; Lee, S. A Neural Network Model for Material Degradation Detection and Diagnosis Using Microscopic Images. IEEE Access 2019, 7, 92151–92160. [Google Scholar] [CrossRef]
- Liu, K.; Wang, H.; Chen, H.; Qu, E.; Tian, Y.; Sun, H. Steel surface defect detection using a new Haar–Weibull-variance model in unsupervised manner. IEEE Trans. Instrum. Meas. 2017, 66, 2585–2596. [Google Scholar] [CrossRef]
- Chen, W.; Gao, Y.; Gao, L.; Li, X. A New Ensemble Approach based on Deep Convolutional Neural Networks for Steel Surface Defect classification. Procedia CIRP 2018, 72, 1069–1072. [Google Scholar] [CrossRef]
- Song, K.; Yan, Y. A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects. Appl. Surf. Sci. 2013, 285, 858–864. [Google Scholar] [CrossRef]
- Jia, H.; Murphey, Y.L.; Shi, J.; Chang, T.S. An intelligent real-time vision system for surface defect detection. In Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004, Cambridge, UK, 26 August 2004; Volume 3, pp. 239–242. [Google Scholar]
- Suvdaa, B.; Ahn, J.; Ko, J. Steel surface defects detection and classification using SIFT and voting strategy. Int. J. Softw. Eng. Appl. 2012, 6, 161–166. [Google Scholar]
- Park, J.K.; Kwon, B.K.; Park, J.H.; Kang, D.J. Machine learning-based imaging system for surface defect inspection. Int. J. Precis. Eng. Manuf.-Green Technol. 2016, 3, 303–310. [Google Scholar] [CrossRef]
- Yi, L.; Li, G.; Jiang, M. An End-to-End Steel Strip Surface Defects Recognition System Based on Convolutional Neural Networks. Steel Res. Int. 2017, 88, 1600068. [Google Scholar] [CrossRef]
- Wang, T.; Chen, Y.; Qiao, M.; Snoussi, H. A fast and robust convolutional neural network-based defect detection model in product quality control. Int. J. Adv. Manuf. Technol. 2018, 94, 3465–3471. [Google Scholar] [CrossRef]
- Masci, J.; Meier, U.; Ciresan, D.; Schmidhuber, J.; Fricout, G. Steel defect classification with max-pooling convolutional neural networks. In Proceedings of the 2012 International Joint Conference on Neural Networks (IJCNN), Brisbane, QLD, Australia, 10–15 June 2012; pp. 1–6. [Google Scholar]
- Jeon, Y.J.; Choi, D.C.; Lee, S.J.; Yun, J.P.; Kim, S.W. Defect detection for corner cracks in steel billets using a wavelet reconstruction method. JOSA A 2014, 31, 227–237. [Google Scholar] [CrossRef] [PubMed]
- Xiao, M.; Jiang, M.; Li, G.; Xie, L.; Yi, L. An evolutionary classifier for steel surface defects with small sample set. EURASIP J. Image Video Process. 2017, 2017, 48. [Google Scholar] [CrossRef]
- Gao, Y.; Gao, L.; Li, X.; Yan, X. A semi-supervised convolutional neural network-based method for steel surface defect recognition. Robot. Comput.-Integr. Manuf. 2020, 61, 101825. [Google Scholar] [CrossRef]
- Haralick, R.M.; Shanmugam, K.; Dinstein, I.H. Textural features for image classification. IEEE Trans. Syst. Man Cybern. 1973, 3, 610–621. [Google Scholar] [CrossRef] [Green Version]
- Xian, G.M. An identification method of malignant and benign liver tumors from ultrasonography based on GLCM texture features and fuzzy SVM. Expert Syst. Appl. 2010, 37, 6737–6741. [Google Scholar] [CrossRef]
- Dutta, S.; Das, A.; Barat, K.; Roy, H. Automatic characterization of fracture surfaces of AISI 304LN stainless steel using image texture analysis. Measurement 2012, 45, 1140–1150. [Google Scholar] [CrossRef]
- Okarma, K.; Fastowicz, J. No-reference quality assessment of 3D prints based on the GLCM analysis. In Proceedings of the 2016 21st International Conference on Methods and Models in Automation and Robotics (MMAR), Miedzyzdroje, Poland, 29 August–1 September 2016; pp. 788–793. [Google Scholar]
- Gadelmawla, E. A vision system for surface roughness characterization using the gray level co-occurrence matrix. NDT E Int. 2004, 37, 577–588. [Google Scholar] [CrossRef]
- Dalal, N.; Triggs, B. Histograms of Oriented Gradients for Human Detection. In International Conference on Computer Vision & Pattern Recognition (CVPR ’05); Schmid, C., Soatto, S., Tomasi, C., Eds.; IEEE Computer Society: San Diego, CA, USA, 2005; Volume 1, pp. 886–893. [Google Scholar] [CrossRef] [Green Version]
- Bertozzi, M.; Broggi, A.; Del Rose, M.; Felisa, M.; Rakotomamonjy, A.; Suard, F. A pedestrian detector using histograms of oriented gradients and a support vector machine classifier. In Proceedings of the 2007 IEEE Intelligent Transportation Systems Conference, Seattle, WA, USA, 30 September–3 October 2007; pp. 143–148. [Google Scholar]
- Pang, Y.; Yuan, Y.; Li, X.; Pan, J. Efficient HOG human detection. Signal Process. 2011, 91, 773–781. [Google Scholar] [CrossRef]
- LeCun, Y.; Bengio, Y. Convolutional networks for images, speech, and time series. In The Handbook of Brain Theory and Neural Networks; MIT Press: Cambridge, MA, USA, 1995; Volume 3361, p. 1995. [Google Scholar]
- Zhou, B.; Khosla, A.; Lapedriza, A.; Oliva, A.; Torralba, A. Learning Deep Features for Discriminative Localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016. [Google Scholar]
- Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar]
- Ioffe, S.; Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv 2015, arXiv:1502.03167. [Google Scholar]
Layer | Operation | Channel | Kernel Size | Stride | Layer | Operation | Channel | Kernel Size | Stride |
---|---|---|---|---|---|---|---|---|---|
1 | Convolution | 16 | 3 × 3 | 1 | 10 | Convolution | 128 | 3 × 3 | 1 |
2 | Convolution | 16 | 3 × 3 | 1 | 11 | Convolution | 128 | 3 × 3 | 1 |
3 | Max pooling | - | 2 × 2 | 2 | 12 | Max pooling | - | 2 × 2 | 2 |
4 | Convolution | 32 | 3 × 3 | 1 | 13 | Convolution | 256 | 3 × 3 | 1 |
5 | Convolution | 32 | 3 × 3 | 1 | 14 | Convolution | 256 | 3 × 3 | 1 |
6 | Max pooling | - | 2 × 2 | 2 | 15 | Global average pooling | 256 | - | - |
7 | Convolution | 64 | 3 × 3 | 1 | 16 | Dense | 10 | - | - |
8 | Convolution | 64 | 3 × 3 | 1 | 17 | Dense | 6 | - | - |
9 | Max pooling | - | 2 × 2 | 2 |
Class | Description | The Number of Samples | Dimension |
---|---|---|---|
1 | Rolled-in scale | 300 | 200 × 200 |
2 | Patches | 300 | 200 × 200 |
3 | Crazing | 300 | 200 × 200 |
4 | Pitted surface | 300 | 200 × 200 |
5 | Inclusion | 300 | 200 × 200 |
6 | Scratches | 300 | 200 × 200 |
Classifier | Algorithm | Feature Extraction Method | Accuracy (%) | F1-Score |
---|---|---|---|---|
SVM | ML | GLCM | 88.06 | 0.87 |
HOG | 78.61 | 0.77 | ||
GLCM+HOG | 92.22 | 0.91 | ||
Logistic regression | ML | GLCM | 90.83 | 0.90 |
HOG | 80.28 | 0.79 | ||
GLCM+HOG | 91.94 | 0.91 | ||
Proposed (CNN) | DL | Feature learning | 99.44 | 0.99 |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lee, S.Y.; Tama, B.A.; Moon, S.J.; Lee, S. Steel Surface Defect Diagnostics Using Deep Convolutional Neural Network and Class Activation Map. Appl. Sci. 2019, 9, 5449. https://doi.org/10.3390/app9245449
Lee SY, Tama BA, Moon SJ, Lee S. Steel Surface Defect Diagnostics Using Deep Convolutional Neural Network and Class Activation Map. Applied Sciences. 2019; 9(24):5449. https://doi.org/10.3390/app9245449
Chicago/Turabian StyleLee, Soo Young, Bayu Adhi Tama, Seok Jun Moon, and Seungchul Lee. 2019. "Steel Surface Defect Diagnostics Using Deep Convolutional Neural Network and Class Activation Map" Applied Sciences 9, no. 24: 5449. https://doi.org/10.3390/app9245449