# Multi-Classifier Decision-Level Fusion Classification of Workpiece Surface Defects Based on a Convolutional Neural Network

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. The Proposed Classification Model

#### 2.1. CNN Feature Extraction

#### 2.2. HOG–LBP Feature Extraction

#### 2.2.1. Histogram of Oriented Gradients: HOG Feature

#### 2.2.2. Local Binary Pattern: LBP Feature

#### 2.3. Multiple-Classifier Decision-Level Fusion

#### 2.4. Model Structure Design

## 3. Experimental Results and Discussions

#### 3.1. Datasets

#### 3.2. Hardware Platform

#### 3.3. Performance Comparison

#### 3.4. Comparative Experiments of the Proposed Model with Different Kernel Functions

#### 3.5. Effectiveness of Symmetry Ensemble Classifier

## 4. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Jeffrey, K.; Feng, C. Automated defect inspection system for CMOS image sensor with micro multi-layer non-spherical lens module. J. Manuf. Syst.
**2017**, 45, 248–259. [Google Scholar] [CrossRef] - Zhang, X.W.; Ding, Y.Q.; Lv, Y.Y. A vision inspection system for the surface defects of strongly reflected metal based on multi-class SVM. Expert Syst. Appl.
**2011**, 38, 5930–5939. [Google Scholar] - Wang, P.; Zhang, X.; Mu, Y. The Copper Surface Defects Inspection System Based on Computer Vision. In Proceedings of the Fourth International Conference on Natural Computation, Jinan, China, 18–20 October 2008; pp. 535–539. [Google Scholar]
- Zhou, S.; Chen, Y.P.; Zhang, D.L. Classification of surface defects on steel sheet using convolutional neural networks. Mater. Tehnol.
**2017**, 51, 123–131. [Google Scholar] [CrossRef] - Chu, M.X.; Jie, Z.; Liu, X.P. Multi-class classification for steel surface defects based on machine learning with quantile hyper-spheres. Chemom. Intell. Lab. Syst.
**2017**, 168, 15–27. [Google Scholar] [CrossRef] - Bandhu, A.; Roy, S.S. Classifying multi-category images using deep learning: A convolutional neural network model. In Proceedings of the IEEE International Conference on Recent Trends in Electronics, Bangalore, India, 19–20 May 2017; pp. 915–919. [Google Scholar]
- Wang, Y.; Guo, H. Weld Defect Detection of X-ray Images Based on Support Vector Machine. IETE Tech. Rev.
**2014**, 31, 137–142. [Google Scholar] [CrossRef] - Cui, D.Y.; Xia, K.W. Strip surface defects recognition based on PSO-RS&SOCP-SVM algorithm. Math. Probl. Eng.
**2017**, 2017, 1–9. [Google Scholar] - Mohamed, A.; Hamdi, M.S.; Tahar, S. Decision Tree-Based Approach for Defect Detection and Classification in Oil and Gas Pipelines. In Proceedings of the Future Technologies Conference, Vancouver, BC, Canada, 13–14 November 2018; pp. 490–504. [Google Scholar]
- Zheng, T.; Jie, G.; Zhang, H.T. Innovative method for recognizing subgrade defects based on a convolutional neural network. Constr. Build. Mater.
**2018**, 169, 69–82. [Google Scholar] - Wang, G.; Liao, T.W. Automatic identification of different types of welding defects in radiographic images. NDT E Int.
**2002**, 35, 519–528. [Google Scholar] [CrossRef] - Suvdaa, B.; Ahn, J.; Ko, J. Steel surface defects detection and classification using SIFT and voting strategy. Int. J. Softw. Eng. Appl.
**2012**, 6, 161–166. [Google Scholar] - D’Angelo, G.; Rampone, S. Feature extraction and soft computing methods for aerospace structure defect classification. Measurement
**2016**, 85, 192–209. [Google Scholar] [CrossRef] [Green Version] - Rawat, W.; Wang, Z. Deep convolutional neural networks for image classification: A comprehensive review. Neural Comput.
**2017**, 29, 2352–2449. [Google Scholar] [CrossRef] [PubMed] - Paoletti, M.E.; Haut, J.M.; Plaza, J. A new deep convolutional neural network for fast hyperspectral image classification. ISPRS J. Photogramm. Remote Sens.
**2018**, 145, 120–147. [Google Scholar] - Zhao, D.; Wang, T.Y.; Chu, F.L. Deep convolutional neural network based planet bearing fault classification. Comput. Ind.
**2019**, 107, 59–66. [Google Scholar] [CrossRef] - Gu, J.X.; Wang, Z.H.; Kuen, J. Recent advances in convolutional neural networks. Pattern Recognit.
**2018**, 77, 354–377. [Google Scholar] [CrossRef] [Green Version] - Vogado, L.H.S.; Veras, R.M.S.; Araujo, F.H.D. Leukemia diagnosis in blood slides using transfer learning in CNNs and SVM for classification. Eng. Appl. Artif. Intell.
**2018**, 72, 415–422. [Google Scholar] [CrossRef] - Han, D.M.; Liu, Q.G.; Fan, W.G. A new image classification method using CNN transfer learning and web data augmentation. Expert Syst. Appl.
**2018**, 95, 43–56. [Google Scholar] [CrossRef] - Zhong, S.S.; Fu, S.; Lin, L. A novel gas turbine fault diagnosis method based on transfer learning with CNN. Measurement
**2019**, 137, 435–453. [Google Scholar] [CrossRef] - Fahad, L.; Yassine, R. Survey on semantic segmentation using deep learning techniques. Neurocomputing
**2019**, 338, 321–348. [Google Scholar] - Tompson, J.; Goroshin, R.; Jain, A. Efficient object localization using Convolutional Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 8–10 June 2015; pp. 648–656. [Google Scholar]
- Raheja, J.L.; Kumar, S.; Chaudhary, A. Fabric defect detection based on GLCM and Gabor filter: A comparison. Optik
**2013**, 124, 6469–6474. [Google Scholar] [CrossRef] - Krishnan, K.G.; Vanathi, P.T. An efficient texture classification algorithm using integrated Discrete Wavelet Transform and local binary pattern features. Cogn. Syst. Res.
**2018**, 52, 267–274. [Google Scholar] [CrossRef] - Lahiani, H.; Neji, M. Hand gesture recognition method based on HOG-LBP features for mobile devices. Procedia Comput. Sci.
**2018**, 126, 254–263. [Google Scholar] [CrossRef] - Munir, N.; Kim, H.J.; Park, J. Convolutional neural network for ultrasonic weldment flaw classification in noisy conditions. Ultrasonics
**2019**, 94, 74–81. [Google Scholar] [CrossRef] [PubMed] - Zhang, Y.X.; You, D.Y.; Gao, X.D. Welding defects detection based on deep learning with multiple optical sensors during disk laser welding of thick plates. J. Manuf. Syst.
**2019**, 51, 87–94. [Google Scholar] [CrossRef] - Liu, R.X.; Yao, M.H.; Wang, X.B. Defects Detection Based on Deep Learning and Transfer Learning. Metall. Min. Ind.
**2015**, 7, 312–321. [Google Scholar] - Dalal, N.; Triggs, B. Histograms of Oriented Gradients for Human Detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA, 20–25 June 2005; pp. 886–893. [Google Scholar]
- Mahbod, A.; Chowdhury, M.; Smedby, Ö. Automatic brain segmentation using artificial neural networks with shape context. Pattern Recognit. Lett.
**2018**, 101, 74–79. [Google Scholar] [CrossRef] - Ojala, T.; Pietikäinen, M.; Harwood, D.A. Comparative study of texture measures with classification based on featured distributions. Pattern Recognit.
**1996**, 29, 51–59. [Google Scholar] [CrossRef]

Layer | Layer Type | Convolution Kernel/Pooling Areas | Step Size | Graphs | Size |
---|---|---|---|---|---|

C1 | convolutional layer | 11 × 11 | 4 | 96 | 55 × 55 |

S2 | pooling layer | 3 × 3 | 2 | 96 | 27 × 27 |

C3 | convolutional layer | 5 × 5 | 1 | 256 | 27 × 27 |

S4 | pooling layer | 3 × 3 | 2 | 256 | 13 × 13 |

C5 | convolutional layer | 3 × 3 | 1 | 384 | 13 × 13 |

C6 | convolutional layer | 3 × 3 | 1 | 384 | 13 × 13 |

C7 | convolutional layer | 3 × 3 | 1 | 256 | 13 × 13 |

S8 | pooling layer | 3 × 3 | 2 | 256 | 6 × 6 |

F9 | fully connected layer | − | − | 4096 | 1 × 1 |

F10 | fully connected layer | − | − | 4096 | 1 × 1 |

F11 | fully connected layer | − | − | 4 | 1 × 1 |

Layer | Layer Type | Convolution Kernel/Pooling Areas | Step Size | Graphs | Size |
---|---|---|---|---|---|

C1 | convolutional layer | 11 × 11 | 4 | 96 | 55 × 55 |

S2 | pooling layer | 3 × 3 | 2 | 96 | 27 × 27 |

C3 | convolutional layer | 5 × 5 | 1 | 256 | 27 × 27 |

S4 | pooling layer | 3 × 3 | 2 | 256 | 13 × 13 |

C5 | convolutional layer | 3 × 3 | 1 | 384 | 13 × 13 |

C6 | convolutional layer | 3 × 3 | 1 | 384 | 13 × 13 |

C7 | convolutional layer | 3 × 3 | 1 | 256 | 13 × 13 |

S8 | pooling layer | 3 × 3 | 2 | 256 | 6 × 6 |

F9 | fully connected layer | − | − | 4096 | 1 × 1 |

F10 | fully connected layer | − | − | 4096 | 1 × 1 |

T11 | HOG-LBP feature layer | − | − | 4 | 1 × 1 |

F12 | fully connected layer | − | − | 4 | 1 × 1 |

L13 | classification layer | − | − | − | − |

Type | Parameter Value |
---|---|

Pit | 500 |

Scratched surface | 500 |

Bumped surface | 500 |

Qualified surface | 500 |

Total | 2000 |

Parameter | Parameter Value |
---|---|

Resolution | 1280 × 960 |

Interface | Gigabit Ethernet |

Pixel size | 3.75 μm × 3.75 μm |

Transmission distance | 100 m |

Maximum frame rate | 40 fps |

Model | Accuracy |
---|---|

KNN | 32.85% |

MLP | 25% |

CNN | 96.55% |

CNN-SVM | 96.90% |

MDF-CNN | 97.60% |

Model | Accuracy | Time (s) |
---|---|---|

AlexNet | 96.55% | 21.44 |

VGG16 | 96.40% | 91.3 |

MDF-CNN | 97.60% | 24.47 |

Model | Accuracy |
---|---|

SVM | 50.20% |

Decision tree | 41.05% |

MDF-CNN | 97.60% |

Kernel | SVM | CNN-SVM | MDF-CNN |
---|---|---|---|

linear | 48.15% | 96.55% | 97.55% |

polynomial | 50.20% | 96.85% | 97.60% |

RBF | 25% | 25.25% | 26.35% |

Model | Accuracy |
---|---|

MDF-CNN without symmetry ensemble classifier | 97.00% |

MDF-CNN with symmetry ensemble classifier | 97.60% |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Liu, F.; Liu, Y.; Sang, H.
Multi-Classifier Decision-Level Fusion Classification of Workpiece Surface Defects Based on a Convolutional Neural Network. *Symmetry* **2020**, *12*, 867.
https://doi.org/10.3390/sym12050867

**AMA Style**

Liu F, Liu Y, Sang H.
Multi-Classifier Decision-Level Fusion Classification of Workpiece Surface Defects Based on a Convolutional Neural Network. *Symmetry*. 2020; 12(5):867.
https://doi.org/10.3390/sym12050867

**Chicago/Turabian Style**

Liu, Fen, Yuxuan Liu, and Hongqiang Sang.
2020. "Multi-Classifier Decision-Level Fusion Classification of Workpiece Surface Defects Based on a Convolutional Neural Network" *Symmetry* 12, no. 5: 867.
https://doi.org/10.3390/sym12050867