Lightweight Reconstruction Network for Surface Defect Detection Based on Texture Complexity Analysis
Abstract
:1. Introduction
2. LRN-L
2.1. Texture Complexity
2.2. Lightweight Reconstruction Network Model (LRN)
2.3. Loss Function
- 1.
- L1 Loss
- 2.
- L2 Loss
- 3.
- Structural Loss
- 4.
- Loss Function of LRN
2.4. Defect Location
- 1.
- Residual Image
- 2.
- Noise Removal
- 3.
- Threshold Segmentation and Defect Location
3. Experiment
3.1. Dataset Introduction
3.2. Evaluation Index
3.3. Network Structure Comparison Experiment
3.4. Loss Function Comparison Experiment
3.5. Experiment of Texture Complexity
3.6. Experiment of Loss Function under Different Weight Factors
3.7. Comparison Experimental of Related Algorithms
4. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Deng, S.; Cai, W.; Xu, Q.; Liang, B. Defect detection of bearing surfaces based on machine vision technique. In Proceedings of the International Conference on Computer Application and System Modeling (ICCASM 2010), Taiyuan, China, 22 October 2010. [Google Scholar]
- Jian, C.; Gao, J.; Ao, Y. Automatic surface defect detection for mobile phone screen glass based on machine vision. Appl. Soft Comput. 2017, 52, 348–358. [Google Scholar] [CrossRef]
- Bulnes, F.G.; Usamentiaga, R.; Garcia, D.F.; Molleda, J. An efficient method for defect detection during the manufacturing of web materials. J. Intell. Manuf. 2016, 27, 431–445. [Google Scholar] [CrossRef]
- Jin, X.T.; Wang, Y.N.; Zhagn, H.; Liu, L.; Zhong, H.; Hei, Z.D. Deep Rail: Automatic visual detection system for railway surface defect using Bayesian CNN and attention network. Acta Autom. Sin. 2019, 45, 2312–2327. [Google Scholar]
- Li, L.F.; Ma, W.F.; Li, L.; Lu, C.J. Research on detection algorithm for bridge cracks based on deep learning. Acta Autom. Sin. 2019, 45, 1727–1742. [Google Scholar]
- Chen, S.; Hu, T.; Liu, G.; Pu, Z.; Li, M.; Du, L. Defect classification algorithm for IC photomask based on PCA and SVM. In Proceedings of the Congress on Image and Signal Processing, Sanya, China, 27 May 2008. [Google Scholar]
- Huang, J.X.; Li, D.; Ye, F.; Zhang, W.J. Detection of surface defection of solder on flexible printed circuit. Opt. Precis. Eng. 2010, 18, 2443–2453. [Google Scholar]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 2012, 25, 1097–1105. [Google Scholar] [CrossRef]
- Napoletano, P.; Piccoli, F.; Schettini, R. Anomaly detection in nanofibrous materials by CNN-based self-similarity. Sensors 2018, 18, 209. [Google Scholar] [CrossRef]
- Cha, Y.J.; Choi, W.; Suh, G.; Mahmoudkhani, S.; Büyüköztürk, O. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Comput.-Aided Civ. Infrastruct. Eng. 2018, 33, 731–747. [Google Scholar] [CrossRef]
- Gao, Y.; Gao, L.; Li, X.; Yan, X. A semi-supervised convolutional neural network-based method for steel surface defect recognition. Robot. Comput.-Integr. Manuf. 2020, 61, 1018–1025. [Google Scholar] [CrossRef]
- Zhao, Z.; Xu, G.; Qi, Y.; Liu, N.; Zhang, T. Multi-patch deep features for power line insulator status classification from aerial images. In Proceedings of the International Joint Conference on Neural Networks (IJCNN), Vancouver, BC, Canada, 24 July 2016. [Google Scholar]
- Wang, T.; Chen, Y.; Qiao, M.; Snoussi, H. A fast and robust convolutional neural network-based defect detection model in product quality control. Int. J. Adv. Manuf. Technol. 2018, 94, 3465–3471. [Google Scholar] [CrossRef]
- Xu, X.; Zheng, H.; Guo, Z.; Wu, X.; Zheng, Z. SDD-CNN: Small Data-Driven Convolution Neural Networks for Subtle Roller Defect Inspection. Appl. Sci. 2019, 9, 1364. [Google Scholar] [CrossRef]
- Weimer, D.; Scholz, R.B.; Shpitalni, M. Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection. Manuf. Technol. 2016, 65, 417–420. [Google Scholar] [CrossRef]
- Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. In Proceedings of the Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014; pp. 580–587. [Google Scholar]
- Berg, A.C.; Fu, C.Y.; Szegedy, C.; Reed, S.; Fu, C.-Y.; Berg, A.C. SSD: Single Shot MultiBox Detector. In Proceedings of the Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015. [Google Scholar] [CrossRef]
- Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You only look once: Unified, real-time object detection. Comput. Vis. Pattern Recognit. 2016, 6, 779–788. [Google Scholar]
- Chen, J.; Liu, Z.; Wang, H.; Núñez, A.; Han, Z. Automatic defect detection of fasteners on the catenary support device using deep convolutional neural network. IEEE Trans. Instrum. Meas. 2017, 67, 257–269. [Google Scholar] [CrossRef]
- Huang, Y.; Qiu, C.; Guo, Y.; Wang, X.; Yuan, K. Surface defect saliency of magnetic tile. In Proceedings of the IEEE 14th International Conference on Automation Science and Engineering, Munich, Germany, 20 August 2018. [Google Scholar]
- Long, J.; Shelhamer, E.; Darrell, T. Fully Convolutional Networks for Semantic Segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 2015, 39, 640–651. [Google Scholar]
- Yu, W.; Zhang, Y.; Shi, H. Surface Defect Inspection Under a Small Training Set Condition. In Proceedings of the International Conference on Intelligent Robotics and Applications, Shenyang, China, 8 August 2019. [Google Scholar]
- Masci, J.; Meier, U.; Cireşan, D.; Schmidhuber, J. Stacked convolutional auto-encoders for hierarchical feature extraction. In Proceedings of the International Conference on Artificial Neural Networks, Torremolinos, Spain, 8 June 2011. [Google Scholar]
- Li, Y.; Zhao, W.; Pan, J. Deformable patterned fabric defect detection with fisher criterion-based deep learning. IEEE Trans. Autom. Sci. Eng. 2016, 14, 1256–1264. [Google Scholar] [CrossRef]
- Chalapathy, R.; Menon, A.K.M.; Chawla, S. Robust, Deep and Inductive Anomaly Detection. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases; Springer: Berlin/Heidelberg, Germany, 2017; pp. 36–51. [Google Scholar]
- Yuan, J.; Zhang, Y.J. Application of sparse denoising autoencoder network with gradient difference information for abnormal action detection. Acta Autom. Sin. 2017, 43, 604–610. [Google Scholar]
- Mei, S.; Yang, H.; Yin, Z. An Unsupervised-Learning-Based Approach for Automated Defect Inspection on Textured Surfaces. IEEE Trans. Instrum. Meas. 2018, 67, 1266–1277. [Google Scholar] [CrossRef]
- Aiger, D.; Talbot, H. The phase only transform for unsupervised surface defect detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 13–18 June 2010; pp. 295–302. [Google Scholar]
- Lin, H.D. Tiny surface defect inspection of electronic passive components using discrete cosine transform decomposition and cumulative sum techniques. Image Vis. Comput 2008, 26, 603–621. [Google Scholar] [CrossRef]
- Yang, H.; Chen, Y.; Song, K.; Yin, Z. Multiscale Feature-Clustering-Based Fully Convolutional Autoencoder for Fast Accurate Visual Inspection of Texture Surface Defects. IEEE Trans. Autom. Sci. Eng. 2019, 16, 1450–1467. [Google Scholar] [CrossRef]
- Makhzani, A.; Shlens, J.; Jaitly, N.; Goodfellow, I.; Frey, B. Adversarial autoencoders. arXiv 2015, arXiv:1511.05644. [Google Scholar]
- Zhao, Z.; Li, B.; Dong, R.; Zhao, P. A Surface Defect Detection Method Based on Positive Samples. In Proceedings of the International Conference on Artificial Intelligence, Nanjing, China, 28–31 August 2018; Pacific Rim. Springer: Cham, Switzerland, 2018; pp. 473–481. [Google Scholar]
- Di, H.; Ke, X.; Peng, Z.; Dongdong, Z. Surface defect classification of steels with a new semi-supervised learning method. Opt. Lasers Eng. 2019, 117, 40–48. [Google Scholar] [CrossRef]
- Schlegl, T.; Seeböck, P.; Waldstein, S.M.; Schmidt-Erfurth, U.; Langs, G. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In Proceedings of the International Conference on Information Processing in Medical Imaging, Boone, NC, USA, 25–30 June 2017; Springer: Cham, Switzerland, 2017; Volume 6, pp. 146–157. [Google Scholar]
- Bergmann, P.; Fauser, M.; Sattlegger, D.; Steger, C. A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Los Angeles, CA, USA, 15 June 2019; pp. 9592–9600. [Google Scholar]
- Chen, Y.Q.; Duan, J.; Zhu, Y.; Qian, X. Research on the Image Complexity Based on Texture Features. Chin. Opt. 2015, 8, 407–413. [Google Scholar] [CrossRef]
- Zou, J.; Liu, C.C. Texture classification by matching co-occurrence matrices on statistical manifolds. In Proceedings of the 10th IEEE International Conference on Computer and Information Technology (CIT 2010), Bradford, UK, 29 June 2010; pp. 1–7. [Google Scholar]
- Gao, Z.Y.; Yang, X.M.; Gong, J.M.; Jin, H. Research on Image Complexity Description Methods. J. Image Graph. 2010, 15, 129–135. [Google Scholar]
- Guo, X.Y.; Li, W.S.; Qian, Y.H.; Bai, R.Y.; Jia, C.H. Computational Evaluation Methods of Visual Complexity Perception for Images. Acta Electron. Sin. 2020, 48, 819–826. [Google Scholar]
- Yang, L.; Zhou, Y.; Yang, J.; Chen, L. Variance WIE based infrared images processing. Electron. Lett. 2006, 42, 857–859. [Google Scholar] [CrossRef]
- Haralick, R.M.; Shanmugam, K. Texture features for image classification. IEEE Trans. Syst. Man Data Hiding Based Pixel Value Ordering Cybern. 1973, 3, 610–621. [Google Scholar]
- Bergmann, P.; Löwe, S.; Fauser, M.; Sattlegger, D.; Steger, C. Improving unsupervised defect segmentation by applying structural similarity to autoencoders. arXiv 2018, arXiv:1807.02011. [Google Scholar]
- Ioffe, S.; Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv 2015, arXiv:1502.03167. [Google Scholar]
- Zhao, H.; Gallo, O.; Frosio, I.; Kautz, J. Loss functions for image restoration with neural networks. IEEE Trans. Comput. Imaging 2016, 3, 47–57. [Google Scholar] [CrossRef]
- Lv, C.; Zhang, Z.; Shen, F.; Zhang, F.; Su, H. A Fast Surface Defect Detection Method Based on Background Reconstruction. Int. J. Precis. Eng. Manuf. 2019, 21, 363–375. [Google Scholar]
- Jager, M.; Knoll, C.; Hamprecht, F.A. Weakly supervised learning of a classifier for unusual event detection. IEEE Trans. Image Process. 2019, 17, 1700–1708. [Google Scholar]
- Silvestre, B.J.; Albero, A.T.; Miralles, I.; Pérez-Llorens, R.; Moreno, J. A Public Fabric Database for Defect Detection Methods and Results. Autex Res. J. 2019, 19, 363–374. [Google Scholar]
- Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Istanbul, Turkey, 17 October 2016; pp. 234–241. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26 June 2016; pp. 770–778. [Google Scholar]
- Shi, W.; Caballero, J.; Huszár, F.; Totz, J.; Aitken, A.P.; Bishop, R.; Rueckert, D.; Wang, Z. Real-time single image and video super-resolution using an efficient sub-pixel Convolutional neural network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26 June 2016; pp. 1874–1883. [Google Scholar]
- Huang, G.; Liu, Z.; Van, D.; Weinberger, K.Q. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26 June 2016; pp. 4700–4708. [Google Scholar]
- Tsai, D.M.; Huang, T.Y. Automated surface inspection for statistical textures. Image Vis. Comput. 2003, 21, 307–323. [Google Scholar]
Loss Function | L1 | L2 | LSSIM | L2 + LSSIM | LLRN | |
---|---|---|---|---|---|---|
Index | ||||||
Precision | A | 0.93 | 0.35 | 0.93 | 0.52 | 0.89 |
B | 0.84 | 0.65 | 0.96 | 0.70 | 0.87 | |
Recall | A | 0.51 | 0.38 | 0.59 | 0.5 | 0.75 |
B | 0.76 | 0.70 | 0.59 | 0.67 | 0.71 | |
F1 Measure | A | 0.66 | 0.36 | 0.72 | 0.51 | 0.82 |
B | 0.80 | 0.67 | 0.73 | 0.69 | 0.78 |
Samples | J | H | G | Q | COV | f |
---|---|---|---|---|---|---|
a | 0.025 | 3.983 | 5.720 | 0.391 | 0.032 | 3.344 |
b | 0.009 | 4.653 | 2.173 | 0.273 | 0.001 | 1.586 |
c | 0.043 | 3.439 | 0.819 | 0.692 | 0.212 | 0.8035 |
d | 0.148 | 2.343 | 0.738 | 0.765 | 0.600 | 0.569 |
e | 0.035 | 3.649 | 1.408 | 0.601 | 0.207 | 1.1005 |
f | 0.013 | 4.755 | 6.285 | 0.415 | 0.048 | 3.6185 |
g | 0.100 | 2.682 | 0.558 | 0.781 | 0.474 | 0.542 |
h | 0.042 | 3.451 | 0.648 | 0.731 | 0.172 | 0.738 |
i | 0.045 | 3.383 | 0.702 | 0.716 | 0.209 | 0.7465 |
j | 0.063 | 3.227 | 1.131 | 0.675 | 0.295 | 0.918 |
k | 0.035 | 5.273 | 1.160 | 0.664 | 0.166 | 0.997 |
l | 0.121 | 3.555 | 0.290 | 0.845 | 0.513 | 0.3885 |
m | 0.188 | 2.969 | 0.298 | 0.854 | 1.007 | 0.1455 |
n | 0.021 | 5.808 | 2.215 | 0.525 | 0.123 | 1.546 |
o | 0.074 | 4.203 | 1.386 | 0.703 | 0.254 | 1.066 |
Samples | f | Level | Precision | Recall | F1 Measure |
---|---|---|---|---|---|
a | 3.344 | H | 0.001 | 0.001 | 0.001 |
b | 1.586 | M | 0.855 | 0.799 | 0.822 |
c | 0.8035 | L | 0.68 | 0.908 | 0.777 |
d | 0.569 | L | 0.034 | 0.337 | 0.062 |
e | 1.1005 | M | 0.925 | 0.883 | 0.904 |
f | 3.6185 | H | 0.001 | 0.001 | 0.001 |
g | 0.542 | L | 0.937 | 0.742 | 0.828 |
h | 0.738 | L | 0.739 | 0.854 | 0.792 |
i | 0.7465 | L | 0.824 | 0.946 | 0.881 |
j | 0.918 | L | 0.291 | 0.431 | 0.348 |
k | 0.997 | L | 0.596 | 0.064 | 0.116 |
l | 0.3885 | L | 0.754 | 0.823 | 0.787 |
m | 0.1455 | L | 0.807 | 0.492 | 0.612 |
n | 1.546 | M | 0.935 | 0.948 | 0.941 |
o | 1.066 | M | 0.884 | 0.772 | 0.824 |
Index | Weight Factor α | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
0 | 0.15 | 0.25 | 0.35 | 0.45 | 0.55 | 0.65 | 0.75 | 0.85 | 1 | |
Precision | 0.71 | 0.69 | 0.58 | 0.28 | 0.46 | 0.53 | 0.23 | 0.89 | 0.54 | 0.62 |
Recall | 0.72 | 0.79 | 0.62 | 0.73 | 0.65 | 0.67 | 0.52 | 0.55 | 0.72 | 0.45 |
F1 Measure | 0.71 | 0.73 | 0.60 | 0.41 | 0.54 | 0.60 | 0.32 | 0.68 | 0.62 | 0.52 |
Algorithms | LCA | PHOT | MSCDAE | LRN-L | |
---|---|---|---|---|---|
Index | |||||
Recall | 1 | 0.478 | 0.133 | 0.203 | 0.799 |
2 | 0.612 | 0.318 | 0.359 | 0.946 | |
3 | 0.117 | 0.341 | 0.966 | 0.707 | |
4 | 0.641 | 0.414 | 0.881 | 0.948 | |
5 | 0.663 | 0.155 | 0.562 | 0.772 | |
Precision | 1 | 0.024 | 0.112 | 0.143 | 0.855 |
2 | 0.412 | 0.367 | 0.696 | 0.824 | |
3 | 0.002 | 0.478 | 0.444 | 0.793 | |
4 | 0.899 | 0.006 | 0.920 | 0.935 | |
5 | 0.436 | 0.324 | 0.463 | 0.884 | |
F1 Measure | 1 | 0.045 | 0.122 | 0.168 | 0.822 |
2 | 0.492 | 0.341 | 0.662 | 0.881 | |
3 | 0.004 | 0.398 | 0.608 | 0.732 | |
4 | 0.748 | 0.012 | 0.900 | 0.941 | |
5 | 0.526 | 0.210 | 0.508 | 0.824 |
Algorithms | PHOT | LCA | MSCDAE | LRN-L |
---|---|---|---|---|
Time (ms) | 450 | 430 | 9746.59 | 2.82 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Shi, H.; Li, G.; Bao, H. Lightweight Reconstruction Network for Surface Defect Detection Based on Texture Complexity Analysis. Electronics 2023, 12, 3617. https://doi.org/10.3390/electronics12173617
Shi H, Li G, Bao H. Lightweight Reconstruction Network for Surface Defect Detection Based on Texture Complexity Analysis. Electronics. 2023; 12(17):3617. https://doi.org/10.3390/electronics12173617
Chicago/Turabian StyleShi, Hui, Gangyan Li, and Hanwei Bao. 2023. "Lightweight Reconstruction Network for Surface Defect Detection Based on Texture Complexity Analysis" Electronics 12, no. 17: 3617. https://doi.org/10.3390/electronics12173617
APA StyleShi, H., Li, G., & Bao, H. (2023). Lightweight Reconstruction Network for Surface Defect Detection Based on Texture Complexity Analysis. Electronics, 12(17), 3617. https://doi.org/10.3390/electronics12173617