# Statistical Process Control with Intelligence Based on the Deep Learning Model

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Methodology

#### 2.1. Simulation Method of Histogram Patterns

#### 2.2. Simulation Method of Control Chart Patterns

#### 2.3. Long Short-Term Memory Network Model

**h**is the output of the network, and H is the nonlinear transformation function.

## 3. Proposed Method

## 4. Experiment and Discussion

#### 4.1. Simulation Parameters of HPs

#### 4.2. Performance Comparison of Optimization Algorithm

#### 4.3. The Influence of Batch Size on Training Process

#### 4.4. The Influence of the Number of Network Layers

#### 4.5. Comparison of HPR Results of Several Methods

- (1)
- MLP: There are three layers of MLP used in the experiment. The first layer of MLP is the input layer with 25 neurons, which are used to receive the frequency of 25 intervals of histogram. The second layer is the hidden layer with 30 neurons. The last layer is the output layer with seven neurons, corresponding to seven typical HPs. The activation function of all layers of MLP is Sigmoid function.
- (2)
- DBN: The input of DBN is the frequency of 25 intervals of histogram. The DBN in this experiment is composed of three layers of RBM. Each RBM layer is pre-trained unsupervised. After the layer by layer pre-training is completed, the supervised global optimization is carried out. Each RBM layer contains 60 neurons, and the activation function of all layers is the Sigmoid function.
- (3)
- 1D-CNN: The 1D-CNN used in this experiment is the structure recommended in [18]. The input of 1D-CNN is the frequency of 25 intervals of histogram. It consists of two convolution layers, two pooling layers and a full connection layer, and they are all one-dimensional. The number of feature maps of the two convolution layers is 6 and 12 respectively, and the convolution kernel size is 2*1 and 9*1, respectively. The activation function of the convolution layer was the rectified linear units (ReLU) function, and the output layer was the Softmax function.

#### 4.6. Simulation Parameters of CCPs

#### 4.7. Comparison of CCPR Results of Several Methods

#### 4.8. Application in Real Production Data

## 5. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Zhou, X.; Jiang, P.; Wang, X. Recognition of control chart patterns using fuzzy SVM with a hybrid kernel function. J. Intell. Manuf.
**2018**, 29, 51–67. [Google Scholar] [CrossRef] - Haghtalab, S.; Xanthopoulos, P.; Madani, K. A robust unsupervised consensus control chart pattern recognition framework. Expert Syst. Appl.
**2015**, 42, 6767–6776. [Google Scholar] [CrossRef] - Gutierrez, H.D.L.T.; Pham, D.T. Estimation and generation of training patterns for control chart pattern recognition. Comput. Ind. Eng.
**2016**, 95, 72–82. [Google Scholar] [CrossRef] [Green Version] - Wang, M.; Zan, T.; Fei, R.Y. Statistical process control with intelligence using fuzzy art neural networks. Front. Mech. Eng.
**2010**, 5, 149–156. [Google Scholar] [CrossRef] - Zan, T.; Wang, M.; Fei, R.Y. Pattern recognition for control charts using AR spectrum and fuzzy ARTMAP neural network. Adv. Mater. Res.
**2010**, 97–101, 3696–3702. [Google Scholar] [CrossRef] - Yang, W.A.; Zhou, W.; Liao, W.; Guo, Y. Identification and quantification of concurrent control chart patterns using extreme-point symmetric mode decomposition and extreme learning machines. Neurocomputing
**2015**, 147, 260–270. [Google Scholar] [CrossRef] - Roberts, S.W. Control chart tests based on geometric moving averages. Technometrics
**1959**, 1, 239–244. [Google Scholar] [CrossRef] - Nelson, L.S. Interpreting Shewhart X-bar control charts. J. Qual. Technol.
**1985**, 17, 114–116. [Google Scholar] [CrossRef] - Ducan, A.J. Quality Control and Industrial Statistics, 5th ed.; Richard D. Irwin: Homewood, IL, USA, 1986. [Google Scholar]
- Nelson, L.S. The Shewhart control chart: Test for special causes. J. Qual. Technol.
**1984**, 16, 237–239. [Google Scholar] [CrossRef] - Ranaee, V.; Ebrahimzadeh, A. Control chart pattern recognition using a novel hybrid intelligent method. Appl. Soft Comput.
**2011**, 11, 2676–2686. [Google Scholar] [CrossRef] - Cheng, C.S. A neural network approach for the analysis of control chart patterns. Int. J. Prod. Res.
**1997**, 35, 667–697. [Google Scholar] [CrossRef] - Al-Ghanim, A.M.; Ludeman, L.C. Automated unnatural pattern recognition on control charts using correlation analysis techniques. Comput. Ind. Eng.
**1997**, 32, 679–690. [Google Scholar] [CrossRef] - Swift, J.A.; Mize, J.H. Out-of-control pattern recognition and analysis for quality control charts using LISP-based systems. Comput. Ind. Eng.
**1995**, 28, 81–91. [Google Scholar] [CrossRef] - Cheng, C.; Hubele, N.F. Design of a knowledge based expert system for statistical process control. Comput. Ind. Eng.
**1992**, 22, 501–517. [Google Scholar] [CrossRef] - He, S.; He, Z.; Wang, G.A. Online monitoring and fault identification of mean shifts in bivariate processes using decision tree learning techniques. J. Intell. Manuf.
**2013**, 24, 25–34. [Google Scholar] [CrossRef] - Kuo, T.; Mital, A. Quality control expert systems: A review of pertinent literature. J. Intell. Manuf.
**1993**, 4, 245–257. [Google Scholar] [CrossRef] - Zan, T.; Liu, Z.; Wang, H.; Wang, M.; Gao, X. Control chart pattern recognition using the convolutional neural network. J. Intell. Manuf.
**2019**. [Google Scholar] [CrossRef] - Pham, D.T.; Wani, M.A. Feature-based control chart pattern recognition. Int. J. Prod. Res.
**1997**, 35, 1875–1890. [Google Scholar] [CrossRef] - Guh, R.S.; Tannock, J. A neural network approach to characterize pattern parameters in process control charts. J. Intell. Manuf.
**1999**, 10, 449–462. [Google Scholar] [CrossRef] - Al-Assaf, Y. Recognition of control chart patterns using multiresolution wavelets analysis and neural networks. Comput. Ind. Eng.
**2004**, 47, 17–29. [Google Scholar] [CrossRef] - Ranaee, V.; Ebrahimzadeh, A. Control chart pattern recognition using neural networks and efficient features: A comparative study. Pattern Anal. Appl.
**2013**, 16, 321–332. [Google Scholar] [CrossRef] - Addeh, A.; Khormali, A.; Golilarz, N.A. Control chart pattern recognition using RBF neural network with new training algorithm and practical features. Isa Trans
**2018**, 79, 202–216. [Google Scholar] [CrossRef] [PubMed] - Cheng, Z.; Ma, Y.Z. A Research about Pattern Recognition of Control Chart Using Probability Neural Network. In Proceedings of the Isecs International Colloquium on Computing, Communication, Control, & Management, Guangzhou, China, 3–4 August 2008; IEEE: Piscatway, NJ, USA. [Google Scholar]
- Awadalla, M.H.A.; Sadek, M.A. Spiking neural network-based control chart pattern recognition. Alex. Eng. J.
**2012**, 51, 27–35. [Google Scholar] [CrossRef] [Green Version] - Gauri, S.K. Control chart pattern recognition using feature-based learning vector quantization. Int. J. Adv. Manuf. Technol.
**2010**, 48, 1061–1073. [Google Scholar] [CrossRef] - Guh, R.S. Real-time recognition of control chart patterns in autocorrelated processes using a learning vector quantization network-based approach. Int. J. Prod. Res.
**2008**, 46, 3959–3991. [Google Scholar] [CrossRef] - Yang, W.A.; Zhou, W. Autoregressive coefficient-invariant control chart pattern recognition in autocorrelated manufacturing processes using neural network ensemble. J. Intell. Manuf.
**2015**, 26, 1161–1180. [Google Scholar] [CrossRef] - Hachicha, W.; Ghorbel, A. A survey of control-chart pattern-recognition literature (1991–2010) based on a new conceptual classification scheme. Comput. Ind. Eng.
**2012**, 63, 204–222. [Google Scholar] [CrossRef] - Xanthopoulos, P.; Razzaghi, T. A weighted support vector machine method for control chart pattern recognition. Comput. Ind. Eng.
**2014**, 70, 134–149. [Google Scholar] [CrossRef] - Hu, S.; Zhao, L. A Support Vector Machine Based Multi-kernel Method for Change Point Estimation on Control Chart. In Proceedings of the 2015 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Kowloon, China, 9–12 October 2015. [Google Scholar]
- Pham, D.T.; Oztemel, E. Control chart pattern recognition using learning vector quantization networks. Int. J. Prod. Res.
**1994**, 32, 721–729. [Google Scholar] [CrossRef] - Hassan, A.; Baksh, M.; Shaharoun, A.M.; Jamaluddin, H. Improved SPC chart pattern recognition using statistical features. Int. J. Prod. Res.
**2003**, 41, 1587–1603. [Google Scholar] [CrossRef] - Pelegrina, G.D.; Duarte, L.T.; Jutten, C. Blind source separation and feature extraction in concurrent control charts pattern recognition: Novel analyses and a comparison of different methods. Comput. Ind. Eng.
**2016**, 92, 105–114. [Google Scholar] [CrossRef] [Green Version] - Gauri, S.K.; Chakraborty, S. Recognition of control chart patterns using improved selection of features. Comput. Ind. Eng.
**2009**, 56, 1577–1588. [Google Scholar] [CrossRef] - Zhao, C.; Wang, C.; Hua, L.; Liu, X.; Zhang, Y.; Hu, H. Recognition of control chart pattern using improved supervised locally linear embedding and support vector machine. Procedia Eng.
**2017**, 174, 281–288. [Google Scholar] [CrossRef] - Janssens, O.; Slavkovikj, V.; Vervisch, B.; Stockman, K.; Loccufier, M.; Verstockt, S.; Van De Walle, R.; Van Hoecke, S. Convolutional Neural Network Based Fault Detection for Rotating Machinery. J. Sound Vib.
**2016**, 377, 331–345. [Google Scholar] [CrossRef] - Chen, Z.; Li, W. Multisensor feature fusion for bearing fault diagnosis using sparse autoencoder and deep belief network. IEEE Trans. Instrum. Meas.
**2017**, 66, 1693–1702. [Google Scholar] [CrossRef] - Gao, Z.; Ma, C.; Song, D.; Liu, Y. Deep quantum inspired neural network with application to aircraft fuel system fault diagnosis. Neurocomputing
**2017**, 238, 13–23. [Google Scholar] [CrossRef] - Fatemeh, A.; Antoine, T.; Marc, T. Tool condition monitoring using spectral subtraction and convolutional neural networks in milling process. Int. J. Adv. Manuf. Technol.
**2018**, 98, 3217–3227. [Google Scholar] - Zan, T.; Wang, H.; Wang, M.; Liu, Z.; Gao, X. Application of Multi-Dimension Input Convolutional Neural Network in Fault Diagnosis of Rolling Bearings. Appl. Sci.
**2019**, 9, 2690. [Google Scholar] [CrossRef] [Green Version] - Liu, E.; Chen, K.; Xiang, Z.; Zhang, J. Conductive particle detection via deep learning for ACF bonding in TFT-LCD manufacturing. J. Intell. Manuf.
**2019**. [Google Scholar] [CrossRef] - Zhao, R.; Yan, R.; Wang, J.; Mao, K. Learning to monitor machine health with convolutional bi-directional lstm networks. Sensors
**2017**, 17, 273. [Google Scholar] [CrossRef] - Zhao, R.; Wang, D.; Yan, R.; Mao, K.; Wang, J. Machine health monitoring using local feature-based gated recurrent unit networks. IEEE Trans. Ind. Electron.
**2017**, 65, 1539–1548. [Google Scholar] [CrossRef] - Rui, Z.; Ruqiang, Y.; Zhenghua, C.; Kezhi, M.; Peng, W.; Gao, R.X. Deep learning and its applications to machine health monitoring. Mech. Syst. Signal. Process.
**2019**, 115, 213–237. [Google Scholar] - Miao, Z.; Yang, M. Control Chart Pattern Recognition Based on Convolution Neural Network. In Smart Innovations in Communication and Computational Sciences. Advances in Intelligent Systems and Computing; Panigrahi, B., Trivedi, M., Mishra, K., Tiwari, S., Singh, P., Eds.; Springer: Singapore, 2019; Volume 670. [Google Scholar]
- Wu, C.; Zhu, B.; Wan, Y.; Zhao, S. Quality Control Chart Pattern Recognition Based on Bidirectional LSTM. Comput. Eng. Softw.
**2019**, 40, 89–95. (In Chinese) [Google Scholar] - Simonoff, J.S.; Udina, F. Measuring the stability of histogram appearance when the anchor position is changed. Comput. Stat. Data. Anal.
**1997**, 23, 335–353. [Google Scholar] [CrossRef] [Green Version] - Bag, M.; Gauri, S.K.; Chakraborty, S. An expert system for control chart pattern recognition. Int. J. Adv. Manuf. Technol.
**2012**, 62, 291–301. [Google Scholar] [CrossRef] - Kingma, D.; Ba, J. Adam: A method for stochastic optimization. arXiv
**2014**, arXiv:1412.6980. [Google Scholar] - Zhang, J.; Wang, P.; Yan, R.; Gao, R.X. Deep Learning for Improved System Remaining Life Prediction. Procedia CIRP
**2018**, 72, 1033–1038. [Google Scholar] [CrossRef] - Van der Maaten, L.; Hinton, G. Visualizing data using t-SNE. J. Mach. Learn. Res.
**2008**, 9, 2579–2605. [Google Scholar]

**Figure 9.**The HPR confusion matrix for (

**a**) the MLP and frequency, (

**b**) the DBN and frequency, (

**c**) the 1D-CNN and frequency, and (

**d**) the multilayer Bi-LSTM and frequency.

**Figure 10.**The CCPR confusion matrix for (

**a**) the MLP and feature set, (

**b**) the DBN and quality data, (

**c**) the 1D-CNN and quality data, and (

**d**) the multilayer Bi-LSTM and quality data.

**Figure 11.**The feature visualization results for (

**a**) the expert features, (

**b**) the features extracted by 1-CNN, and (

**c**) the features extracted by multilayer Bi-LSTM.

Patterns | Parameters | Parameter Value/Range |
---|---|---|

NOR | Mean μ, standard deviation σ | $\mu =10,\sigma =0.01$ |

BIM | Proportion a, offset distance b_{1} and b_{2} | $a\in \left[0.4,0.6\right],{b}_{1}\in \left[\sigma ,2\sigma \right],{b}_{2}\in \left[\sigma ,2\sigma \right]$ |

LI | Proportion a, offset distance b | $a\in \left[0.8,0.9\right],b\in \left[1.5\sigma ,2.5\sigma \right]$ |

RI | Proportion a, offset distance b | $a\in \left[0.8,0.9\right],b\in \left[1.5\sigma ,2.5\sigma \right]$ |

LS | The number of normal distributions m | $m\in \left\{3,4,5\right\}$ |

RS | The number of normal distributions m | $m\in \left\{3,4,5\right\}$ |

FT | Proportion a | $a\in \left[0.6,0.8\right]$ |

Initial Learning Rate | Optimization Algorithm | Batch Size | Number of Layers |
---|---|---|---|

0.05 | Adam | 200 | 3 |

MLP & Frequency | DBN & Frequency | 1D-CNN & Frequency | Multilayer Bi-LSTM & Frequency | ||||
---|---|---|---|---|---|---|---|

CRR (%) | Time (s/epoch) | CRR (%) | Time (s/epoch) | CRR (%) | Time (s/epoch) | CRR (%) | Time (s/epoch) |

96.11 | 4.16 | 98.06 | 0.74 | 99.37 | 1.37 | 99.89 | 4.67 |

Pattern | Parameters | Mathematical Representation | Parameter Value/Range |
---|---|---|---|

NOR | Mean μ, standard deviation σ | $y\left(t\right)=\mu +x\left(t\right)$ | $\mu =30,$ $\sigma =0.05$ |

UT | Gradient d | $y\left(t\right)=\mu +x\left(t\right)+d\times t$ | $d\in \left[0.15\sigma ,0.3\sigma \right]$ |

DT | Gradient d | $y\left(t\right)=\mu +x\left(t\right)-d\times t$ | $d\in \left[0.15\sigma ,0.3\sigma \right]$ |

US | Shift magnitude s | $y\left(t\right)=\mu +x\left(t\right)+v\times s$ | $s\in \left[1.5\sigma ,3\sigma \right]$ |

DS | Shift magnitude s | $y\left(t\right)=\mu +x\left(t\right)-v\times s$ | $s\in \left[1.5\sigma ,3\sigma \right]$ |

CYC | Amplitude a, Period ω | $y\left(t\right)=\mu +x\left(t\right)+v\times a\times \mathrm{sin}\left(2\pi t/\omega \right)$ | $a\in \left[1.5\sigma ,4\sigma \right],\omega \in \left\{4,5,6,7,8\right\}$ |

SYS | Magnitude g | $y\left(t\right)=\mu +x\left(t\right)+v\times g\times {\left(-1\right)}^{t}$ | $g\in \left[1\sigma ,3\sigma \right]$ |

STA | Standard deviation σ′ | $y\left(t\right)=\mu +{x}^{\prime}\left(t\right)$ | $\mu =30,$ ${\sigma}^{\prime}\in \left[0.2\sigma ,0.4\sigma \right]$ |

MIX | Magnitude m | $y\left(t\right)=\mu +x\left(t\right)+v\times m\times {\left(-1\right)}^{p}$ | $m\in \left[1.5\sigma ,2.5\sigma \right]$ |

MLP & Feature Set | DBN & Quality Data | 1D-CNN & Quality Data | Multilayer Bi-LSTM & Quality Data | ||||
---|---|---|---|---|---|---|---|

CRR (%) | Time (s/epoch) | CRR (%) | Time (s/epoch) | CRR (%) | Time (s/epoch) | CRR (%) | Time (s/epoch) |

96.40 | 2.03 | 94.04 | 0.91 | 98.44 | 1.83 | 99.26 | 6.32 |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Zan, T.; Liu, Z.; Su, Z.; Wang, M.; Gao, X.; Chen, D.
Statistical Process Control with Intelligence Based on the Deep Learning Model. *Appl. Sci.* **2020**, *10*, 308.
https://doi.org/10.3390/app10010308

**AMA Style**

Zan T, Liu Z, Su Z, Wang M, Gao X, Chen D.
Statistical Process Control with Intelligence Based on the Deep Learning Model. *Applied Sciences*. 2020; 10(1):308.
https://doi.org/10.3390/app10010308

**Chicago/Turabian Style**

Zan, Tao, Zhihao Liu, Zifeng Su, Min Wang, Xiangsheng Gao, and Deyin Chen.
2020. "Statistical Process Control with Intelligence Based on the Deep Learning Model" *Applied Sciences* 10, no. 1: 308.
https://doi.org/10.3390/app10010308