Wheel Hub Defects Image Recognition Base on Zero-Shot Learning
Abstract
:1. Instruction
2. Related Work
2.1. Extraction Method of Image Features
2.2. Construction of Semantic Embedding Space
2.3. Visual-Semantic Association Mapping Learning
3. Wheel Hub Defect Dataset
3.1. Image Data
3.2. Domain Knowledge Base
4. Method
4.1. Problem Description
4.2. Structure Matching Strategy
4.3. Domain Adaption Strategy
Algorithm 1: Conditional Gradient Method |
|
4.4. Recognition Anti-Bias Mechanism for GZSL
Algorithm 2: Three-step Zero-shot Learning Algorithm |
|
5. Experiment and Analysis
5.1. Experiment Preparation
5.2. Experiment Results
5.3. Experiment Analysis
- (1)
- Analysis of structure matching components
- (2)
- Domain adaptive component analysis
- (3)
- Analysis of Calibration Mechanism
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Li, W.; Li, K.; Huang, Y.; Deng, X. A New Trend Peak Algorithm with X-ray Image for Wheel Hubs Detection and Recognition. In ISICA 2015: Computational Intelligence and Intelligent Systems; Springer: Singapore, 2015; pp. 23–31. [Google Scholar]
- Zhang, J.; Hao, L.; Jiao, T.; Que, L.; Wang, M. Mathematical morphology approach to internal defect analysis of a356 aluminum alloy wheel hubs. AIMS Math. 2020, 5, 3256–3273. [Google Scholar] [CrossRef]
- Han, K.; Sun, M.; Zhou, X.; Zhang, G.; Dang, H.; Liu, Z. A new method in wheel hub surface defect detection: Object detection algorithm based on deep learning. In Proceedings of the International Conference on Advanced Mechatronic Systems, Xiamen, China, 6–9 December 2017; pp. 335–338. [Google Scholar]
- Sun, X.; Gu, J.; Huang, R.; Zou, R.; Palomares, B. Surface defects recognition of wheel hub based on improved Faster R-CNN. Electronics 2019, 8, 481. [Google Scholar] [CrossRef] [Green Version]
- Han, D.; Liu, Q.; Fan, W. A new image classification method using cnn transfer learning and web data augmentation. Expert Syst. Appl. 2018, 95, 43–56. [Google Scholar] [CrossRef]
- Jie, Z.; Wei, Y.; Jin, X.; Feng, J.; Liu, W. Deep self-taught learning for weakly supervised object localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 4294–4302. [Google Scholar]
- Ren, Z.; Zhu, Y.; Yan, K.; Chen, K.; Kang, W.; Yue, Y. A novel model with the ability of few-shot learning and quick updating for intelligent fault diagnosis. Mech. Syst. Signal Proc. 2020, 138, 106608.1–106608.21. [Google Scholar] [CrossRef]
- Luo, C.; Li, Z.; Huang, K.; Feng, J.; Wang, M. Zero-shot learning via attribute regression and class prototype rectification. IEEE Trans. Image Process. 2018, 27, 637–648. [Google Scholar] [CrossRef] [PubMed]
- Yu, L.; Feng, L.; Qian, Y.; Liu, W.; Hauptmann, A. Zero-VIRUS *: Zero-shot Vehicle Route Understanding System for Intelligent Transportation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA, 14–19 June 2020; pp. 2534–2543. [Google Scholar]
- Wen, G.; Ma, J.; Hu, Y.; Li, H.; Jiang, L. Grouping attributes zero-shot learning for tongue constitution recognition. Artif. Int. Med. 2020, 109, 101951. [Google Scholar] [CrossRef]
- Gao, Y.; Gao, L.; Li, X.; Zheng, Y. A zero-shot learning method for fault diagnosis under unknown working loads. J. Int. Manuf. 2020, 31, 899–909. [Google Scholar] [CrossRef]
- Miguel, L.; Lin, D.; Guntupalli, J.; George, D. Beyond imitation: Zero-shot task transfer on robots by learning concepts as cognitive programs. Sci. Robot. 2019, 4, 1–16. [Google Scholar] [CrossRef] [Green Version]
- Marieke, M.; Mirjam, M.; Jerzy, B.; Rainer, G.; Bandettini, P.; Nikolaus, K. Human object-similarity judgments reflect and transcend the primate-it object representation. Front. Psychol. 2013, 4, 128. [Google Scholar]
- Lee, J.; Marzelli, M.; Jolesz, F.; Yoo, S. Automated classification of fmri data employing trial-based imagery tasks. Med Image Anal. 2009, 13, 392–404. [Google Scholar] [CrossRef] [Green Version]
- Wang, G.; Forsyth, D. Joint learning of visual attributes, object classes and visual saliency. In Proceedings of the IEEE 12th International Conference on Computer Vision, Kyoto, Japan, 29 September–2 October 2009; pp. 537–544. [Google Scholar]
- Coates, A.; Ng, A. Selecting receptive fields in deep networks. In Proceedings of the 24th International Conference on Neural Information Processing Systems, Granada, Spain, 12–14 December 2011; pp. 2528–2536. [Google Scholar]
- Bengio, Y.; Courville, A.; Vincent, P. Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Ans Mach. Intell. 2013, 35, 1798–1828. [Google Scholar] [CrossRef]
- Zou, X.; Zhou, L.; Li, K.; Ouyang, A.; Chen, C. Multi-task cascade deep convolutional neural networks for large-scale commodity recognition. Neural Comp. Appl. 2020, 32, 5633–5647. [Google Scholar] [CrossRef]
- Zhao, T.; Chen, Q.; Kuang, Z.; Yu, J.; Zhang, W.; He, M. Deep mixture of diverse experts for large-scale visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2018, 41, 1072–1087. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Leong, M.; Prasad, D.; Lee, Y.; Lin, F. Semi-CNN architecture for effective spatio-temporal learning in action recognition. Appl. Sci. 2020, 10, 557. [Google Scholar] [CrossRef] [Green Version]
- Zhang, J.; Chen, Y.; Zhai, Y. Zero-shot classification based on word vector enhancement and distance metric learning. IEEE Access 2020, 8, 102292–102302. [Google Scholar] [CrossRef]
- Wang, F.; Liu, H.; Sun, F. Fabric recognition using zero-shot learning. Tsinghua Sci. Technol. 2019, 24, 3–11. [Google Scholar] [CrossRef]
- Jangra, M.; Dhull, S.; Singh, K. Ecg arrhythmia classification using modified visual geometry group network (mvggnet). J. Intell. Fuzzy Syst. 2020, 38, 1–15. [Google Scholar] [CrossRef]
- Lu, Z.; Jiang, X.; Kot, C. Deep coupled resnet for low-resolution face recognition. IEEE Signal. Proc. Lett. 2018, 526–530. [Google Scholar] [CrossRef]
- Tang, J.; Zha, Z.J.; Tao, D.; Chua, T. Semantic-gap-oriented active learning for multilabel image annotation. IEEE Trans. Image Process. 2012, 21, 2354–2360. [Google Scholar] [CrossRef]
- Amstel, V.; Brand, V.; Protic, Z.; Verhoeff, T. Transforming process algebra models into uml state machines: Bridging a semantic gap? Lect. Notes Comput. Sci. 2008, 5063, 61–75. [Google Scholar]
- Saju, A.; Bella Mary, I.T.; Vasuki, A.; Lakshmi, P. Reduction of semantic gap using relevance feedback technique in image retrieval system. Appl. Digital Inf. Web Technol. 2014, 148–153. [Google Scholar] [CrossRef]
- Chang, D.; Cho, G.; Choi, Y. Zero-shot recognition enhancement by distance-weighted contextual inference. Appl. Sci. 2020, 10, 7234. [Google Scholar] [CrossRef]
- Weston, J.; Hamel, P. Multi-tasking with joint semantic spaces for large-scale music annotation and retrieval. J. New Music Res. 2011, 40, 337–348. [Google Scholar] [CrossRef]
- Liu, G.Z. Semantic vector space model: Implementation and evaluation. J. Assoc. Inf. Sci. Technol. 2010, 48, 395–417. [Google Scholar] [CrossRef]
- Dill, S.; Eiron, N.; Gibson, D.; Gruhl, D.; Zien, J.Y. A case for automated large-scale semantic annotation. J. Web Semant. 2003, 1, 115–132. [Google Scholar] [CrossRef]
- Snel. Digital shoreline analysis system (DASA) version 4.3 transects with end-point rate calculations for sheltered shorelines between the colville river delta and point barrow for the time period 1947 to 2005. Nephrol. Dial. Transplant. 2012, 27, 3664. [Google Scholar]
- Krishnan, N.; Cook, D.J.; Wemlinger, Z. Learning a taxonomy of predefined and discovered activity patterns. J. Ambient Intell. Smart Environ. 2013, 5, 621–637. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Allen, D. Automatic one-hot re-encoding for fpls. In Proceedings of the second International Workshop on Field Programmable Logic. & Applications, Vienna, Austria, 31 August–2 September 1992; pp. 71–77. [Google Scholar]
- Liu, Z.H.; Cui, L.; Zhang, T.; Liu, Q.; Chen, E.Y.; Yang, K.M. Three dimensional mapping of ground deformation disasters caused by underground coal mining. Adv. Mater. Res. 2012, 524, 503–507. [Google Scholar] [CrossRef]
- Radovanovic, M.; Nanopoulos, A.; Ivanovic, M. Hubs in space: Popular nearest neighbors in high-dimensional data. J. Mach. Learn. Res. 2010, 11, 2487–2531. [Google Scholar]
- Shigeto, Y.; Suzuki, I.; Hara, K.; Shimbo, M.; Matsumoto, Y. Ridge regression, hubness, and zero-shot learning. In Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Porto, Portugal, 7–11 September 2015; pp. 135–151. [Google Scholar] [CrossRef] [Green Version]
- Fu, Y.; Hospedales, T.; Xiang, T.; Gong, S. Transductive multi-view zero-shot learning. IEEE Trans. Pattern Anal. Mach. Intell. 2015, 37, 2332–2345. [Google Scholar] [CrossRef] [PubMed]
- Das, D.; George, C. Sample-to-sample correspondence for unsupervised domain adaptation. Eng. Appl. Artif. Intell. 2018, 73, 80–91. [Google Scholar] [CrossRef] [Green Version]
- Chao, W.; Changpinyo, S.; Gong, B.; Sha, F. An empirical study and analysis of generalized zero-shot learning for object recognition in the wild. In Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands, 11–14 October 2016; pp. 52–68. [Google Scholar] [CrossRef]
- Chren, W.A. One-hot residue coding for low delay-power product cmos design. IEEE Trans. Circuits Syst. II Analog Digit. Signal. Process. 1998, 45, 303–313. [Google Scholar] [CrossRef]
- Juan, A.; Mateos, G.; Giannakis, G. Group-lasso on splines for spectrum cartography. IEEE Trans. Signal. Process. 2011, 59, 4648–4663. [Google Scholar]
- Bredies, K.; Lorenz, D.; Maass, P. A generalized conditional gradient method and its connection to an iterative shrinkage method. Comput. Optim. Appl. 2009, 42, 173–193. [Google Scholar] [CrossRef]
- Shek, E.; Ghani, M.; Jones, R. Simplex search in optimization of capsule formulation. J. Pharm. Ences 2015, 69, 1135–1142. [Google Scholar] [CrossRef]
- Fu, A.; Wenyin, L.; Deng, X. Detecting phishing web pages with visual similarity assessment based on earth mover’s distance (emd). IEEE Trans. Dependable Secur. Comput. 2006, 3, 301–311. [Google Scholar] [CrossRef]
- Bonneel, N.; Panne, M.; Paris, S.; Heidrich, W. Displacement interpolation using lagrangian mass transport. ACM Trans. Graph. 2011, 30, 158. [Google Scholar] [CrossRef] [Green Version]
- Sun, L.; Song, J.; Wang, Y.; Li, B. Cooperative coupled generative networks for generalized zero-shot learning. IEEE Access 2020, 8, 119287–119299. [Google Scholar]
- Farhadi, A.; Endres, I.; Hoiem, D.; Forsyth, D. Describing objects by their attributes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami, FL, USA, 20–25 June 2009; pp. 1778–1785. [Google Scholar]
- Caldwell, A.; Eller, P.; Hafych, V.; Schick, R.; Szalay, M. Integration with an adaptive harmonic mean algorithm. Int. J. Mod. Phys. A 2020, 35, 1950142. [Google Scholar] [CrossRef]
- Schnitzer, D.; Flexer, A.; Schedl, M.; Widmer, G. Local and global scaling reduce hubs in space. J. Mach. Learn. Res. 2012, 13, 2871–2902. [Google Scholar]
- Laurens, V.; Hinton, G. Visualizing data using t-SNE. J. Mach. Learn. Res. 2008, 9, 2579–2605. [Google Scholar]
Expert Defined Attribute | ||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Color of Defects | Shape of Defects | Nature of Defects | ||||||||||||||
A1 | A2 | A3 | A4 | A5 | A6 | A7 | A8 | A9 | A10 | A11 | A12 | A13 | A14 | A15 | A16 | |
D-1 | √ | √ | √ | √ | ||||||||||||
D-2 | √ | √ | √ | √ | ||||||||||||
D-3 | √ | √ | √ | √ | √ | |||||||||||
D-4 | √ | √ | √ | √ | ||||||||||||
D-5 | √ | √ | √ | √ | √ | |||||||||||
D-6 | √ | √ | √ | √ | √ | |||||||||||
D-7 | √ | √ | √ | √ | √ | |||||||||||
D-8 | √ | √ | √ | √ | √ | |||||||||||
D-9 | √ | √ | √ | √ |
Training (Seen) | Testing (Unseen) | Attribute Dimension | No. of Samples | |
---|---|---|---|---|
aPY | 20 | 10 | 65 | 15,339 |
WHD-9 | 6 | 3 | 16 | 6380 |
aPY | WHD-9 | |||||||
---|---|---|---|---|---|---|---|---|
Methods | TU | GS | GU | H | TU | GS | GU | H |
DAP[150] | 33.6 | 79.6 | 6.3 | 11.7 | 40.0 | 57.7 | 2.7 | 5.2 |
IAP[140] | 36.9 | 67.3 | 7.5 | 13.5 | 31.7 | 65.1 | 1.9 | 3.7 |
ConSE[141] | 28.7 | 92.7 | 0.0 | 0.0 | 37.2 | 83.2 | 2.1 | 4.1 |
SYNC[142] | 24.3 | 68.1 | 9.1 | 16.1 | 45.6 | 67.8 | 9.3 | 16.4 |
DeViSE[143] | 40.6 | 78.0 | 6.7 | 12.3 | 51.3 | 58.0 | 15.7 | 24.7 |
DEM[102] | 36.4 | 76.9 | 13.2 | 22.5 | 48.7 | 63.4 | 20.4 | 30.9 |
DEM+R(Our) | 31.2 | 72.3 | 17.1 | 27.6 | 50.0 | 59.7 | 23.4 | 33.6 |
DEM+RA(Our) | 37.7 | 74.0 | 32.2 | 44.8 | 54.9 | 55.1 | 47.2 | 50.8 |
DEM+RAC(Our) | 37.8 | 65.4 | 35.9 | 46.4 | 55.1 | 54.6 | 48.6 | 51.4 |
Method/Dataset | WHD-9 | aPY |
---|---|---|
DEM | 1.69 | 1.83 |
DEM+R | 1.15 | 1.37 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Sun, X.; Gu, J.; Wang, M.; Meng, Y.; Shi, H. Wheel Hub Defects Image Recognition Base on Zero-Shot Learning. Appl. Sci. 2021, 11, 1529. https://doi.org/10.3390/app11041529
Sun X, Gu J, Wang M, Meng Y, Shi H. Wheel Hub Defects Image Recognition Base on Zero-Shot Learning. Applied Sciences. 2021; 11(4):1529. https://doi.org/10.3390/app11041529
Chicago/Turabian StyleSun, Xiaohong, Jinan Gu, Meimei Wang, Yanhua Meng, and Huichao Shi. 2021. "Wheel Hub Defects Image Recognition Base on Zero-Shot Learning" Applied Sciences 11, no. 4: 1529. https://doi.org/10.3390/app11041529
APA StyleSun, X., Gu, J., Wang, M., Meng, Y., & Shi, H. (2021). Wheel Hub Defects Image Recognition Base on Zero-Shot Learning. Applied Sciences, 11(4), 1529. https://doi.org/10.3390/app11041529