# Pattern Recognition in Multivariate Time Series: Towards an Automated Event Detection Method for Smart Manufacturing Systems

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. State of the Art

#### 2.1. Manufacturing Process

#### 2.2. Analytical Approaches

## 3. Research Methodology

#### 3.1. Principle Component Analysis (PCA)

#### 3.2. Heuristic Segmentation

#### 3.2.1. Sliding Window Segmentation

#### 3.2.2. Top-Down Segmentation

#### 3.2.3. Bottom-Up Segmentation

#### 3.3. Toeplitz inverse-Covariance Clustering (TICC)

#### 3.3.1. Problem Formulation

#### 3.3.2. Cluster assignment (Expectation)

#### 3.3.3. Updating Cluster Parameters (Maximization)

Algorithm 1: TICC (high-level) | |

initialization Cluster parameters Θ, cluster assignments P | |

repeat | |

E-step: Assign points to clusters → P | |

M-step: Update cluster parameters →Θ | |

until Stationarity | |

return Θ,P |

## 4. Results

- least absolute deviation (L1),
- least squared deviation (L2),
- linear model change based on a piecewise linear regression (Linear),
- function to detect shifts in the means and scale of a Gaussian time series (Gaussian).

## 5. Discussion

## 6. Conclusions and Future Work

_{2}-emissions and overall waste of energy) and working atmosphere (e.g., reduction of unnecessary stress from unplanned machine failure) benefits. On the other hand, those techniques may also present an opportunity for being abused when other types of time-series data, such as personal data, of workers, are analyzed. For instance, biometric and behavioral data can be monitored and analyzed automatically, potentially leading to issues with regard to privacy and other negative implications for workers. Thus, it is necessary to investigate policy implications hindering inappropriate application of such time-series analytics conflicting with individuals’ privacy rights.

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Sinha, A.; Bernardes, E.; Calderon, R.; Wuest, T. Digital Supply Networks: Transform Your Supply Chain and Gain Competitive Advantage with Disruptive Technology and Reimagined Processes; McGraw-Hill Education: New York, NY, USA, 2020. [Google Scholar]
- Özkoç, E.E. Clustering of Time-Series Data. In Data Mining—Methods, Applications and Systems; IntechOpen: London, UK, 2020. [Google Scholar] [CrossRef] [Green Version]
- Hallac, D.; Vare, S.; Boyd, S.; Leskovec, J. Toeplitz Inverse Covariance-Based Clustering of Multivariate Time Series Data. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’17), Association for Computing Machinery, New York, NY, USA, 23–27 August 2017; pp. 215–223. [Google Scholar] [CrossRef] [Green Version]
- Fontes, C.H.; Pereira, O. Pattern recognition in multivariate time series—A case study applied to fault detection in a gas turbine. Eng. Appl. Artif. Intell.
**2016**, 49, 10–18. [Google Scholar] [CrossRef] - Truong, C.; Laurent, O.; Vayatis, N. Selective Review of Offline Change Point Detection Methods. Signal Process.
**2020**, 167, 107299. [Google Scholar] [CrossRef] [Green Version] - Keogh, E.; Chu, S.; Hart, D.; Pazzani, M. An online algorithm for segmenting time series. In Proceedings of the 2001 IEEE International Conference on Data Mining, San Jose, CA, USA, 29 November–2 December 2001; pp. 289–296. [Google Scholar] [CrossRef]
- Aghabozorgi, S.; Shirkhorshidi, A.S.; Wah, T.Y. Time-series clustering—A decade review. Inf. Syst.
**2015**, 53, 16–38. [Google Scholar] [CrossRef] - Izakian, H.; Pedrycz, W.; Jamal, I. Fuzzy clustering of time series data using dynamic time warping distance. Eng. Appl. Artif. Intell.
**2015**, 39, 235–244. [Google Scholar] [CrossRef] - Glaser, B.; Strauss, A. The Discovery of Grounded Theory: Strategies for Qualitative Research; Aldine: Chicago, CA, USA, 1967. [Google Scholar]
- Wolfswinkel, J.; Furtmueller, E.; Wilderom, C. Using Grounded Theory as a Method for Rigorously Reviewing Literature. Eur. J. Inf. Syst.
**2013**, 22, 45–55. [Google Scholar] [CrossRef] - Mujumdar, A.; Hasan, M. 41 Drying of Polymers; Taylor & Francis Group, LLC: Abingdon, UK, 2016. [Google Scholar] [CrossRef]
- Munjal, S.; Kao, C. Mathematical model and experimental investigation of polycarbonate pellet drying. Polym. Eng. Sci.
**1990**, 30, 1352–1360. [Google Scholar] [CrossRef] - Rosato, D.V. Processing plastic material. In Extruding Plastics; Springer: Boston, MA, USA, 1998. [Google Scholar] [CrossRef]
- National Research Council (USA). Committee on Polymer Science and Engineering, Polymer Science and Engineering: The Shifting Research Frontiers; National Academies Press: Washington, DC, USA, 1994. [Google Scholar] [CrossRef]
- Keogh, E.J.; Kasetty, S. On the need for time series data mining benchmarks: A survey and empirical demonstration. In Proceedings of the International Conference on Knowledge Discovery and Data Mining (ACMSIGKDD), Edmonton, AB, Canada, 23–26 July 2002; pp. 23–26. [Google Scholar]
- Liao, T.W. Clustering of time series data—A survey. Pattern Recognit.
**2005**, 38, 1857–1874. [Google Scholar] [CrossRef] - Fu, T. A review on time series data mining. Eng. Appl. Artif. Intell.
**2011**, 24, 164–181. [Google Scholar] [CrossRef] - Spiegel, S.; Gaebler, J.; Lommatzsch, A.; De Luca, E.; Albayrak, S. Pattern recognition and classification for multivariate time series. In Proceedings of the Fifth International Workshop on Knowledge Discovery from Sensor Data (SensorKDD’11), Association for Computing Machinery, San Diego, CA, USA, 21 August 2011; pp. 34–42. [Google Scholar] [CrossRef]
- Garg, R.; Aggarwal, H.; Centobelli, P.; Cerchione, R. Extracting Knowledge from Big Data for Sustainability: A Comparison of Machine Learning Techniques. Sustainability
**2019**, 11, 6669. [Google Scholar] [CrossRef] [Green Version] - Sfetsos, A.; Siriopoulos, C. Time series forecasting with a hybrid clustering scheme and pattern recognition. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum.
**2004**, 34, 399–405. [Google Scholar] [CrossRef] - Harchaoui, Z.; Lévy-Leduc, C. Multiple Change-Point Estimation with a Total Variation Penalty. J. Am. Stat. Assoc.
**2010**, 105, 1480–1493. [Google Scholar] [CrossRef] [Green Version] - Haynes, K.; Fearnhead, P.; Eckley, I.A. A computationally efficient nonparametric approach for changepoint detection. Stat. Comput.
**2017**, 27, 1293–1305. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Killick, R.; Fearnhead, P.; Eckley, I.A. Optimal Detection of Changepoints with a Linear Computational Cost. J. Am. Stat. Assoc.
**2012**, 107, 1590–1598. [Google Scholar] [CrossRef] - Idé, T.; Khandelwal, A.; Kalagnanam, J. Sparse Gaussian Markov Random Field Mixtures for Anomaly Detection. In Proceedings of the 2016 IEEE 16th International Conference on Data Mining (ICDM), Barcelona, Spain, 12–15 December 2016; pp. 955–960. [Google Scholar] [CrossRef]
- Singh, R.; Pal, B.C.; Jabr, R.A. Statistical Representation of Distribution System Loads Using Gaussian Mixture Model. IEEE Trans. Power Syst.
**2010**, 25, 29–37. [Google Scholar] [CrossRef] [Green Version] - Forney, G.D. The viterbi algorithm. Proc. IEEE
**1973**, 61, 268–278. [Google Scholar] [CrossRef] - Vershynin, R. How Close is the Sample Covariance Matrix to the Actual Covariance Matrix? J. Theor. Probab.
**2012**, 25, 655–686. [Google Scholar] [CrossRef] [Green Version] - Chang, T.; Hong, M.; Liao, W.; Wang, X. Asynchronous Distributed ADMM for Large-Scale Optimization—Part I: Algorithm and Convergence Analysis. IEEE Trans. Signal Process.
**2016**, 64, 3118–3130. [Google Scholar] [CrossRef] - Karimi, D.; Salcudean, S.E. Reducing the Hausdorff Distance in Medical Image Segmentation with Convolutional Neural Networks. IEEE Trans. Med. Imaging
**2020**, 39, 499–513. [Google Scholar] [CrossRef] [PubMed] [Green Version]

**Figure 5.**Overview of heuristic segmentation approach by ruptures based on [5].

**Figure 6.**Sliding window approach: discrepancy curve based on [5].

**Figure 7.**Top-down algorithm schema based on [5].

**Figure 8.**Bottom-up algorithm schema based on [5].

**Figure 9.**Exemplary MRF structures for two distinct clusters based on [3].

**Figure 10.**Graphical formulation of the clustering assignment problem based on [3].

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Kapp, V.; May, M.C.; Lanza, G.; Wuest, T.
Pattern Recognition in Multivariate Time Series: Towards an Automated Event Detection Method for Smart Manufacturing Systems. *J. Manuf. Mater. Process.* **2020**, *4*, 88.
https://doi.org/10.3390/jmmp4030088

**AMA Style**

Kapp V, May MC, Lanza G, Wuest T.
Pattern Recognition in Multivariate Time Series: Towards an Automated Event Detection Method for Smart Manufacturing Systems. *Journal of Manufacturing and Materials Processing*. 2020; 4(3):88.
https://doi.org/10.3390/jmmp4030088

**Chicago/Turabian Style**

Kapp, Vadim, Marvin Carl May, Gisela Lanza, and Thorsten Wuest.
2020. "Pattern Recognition in Multivariate Time Series: Towards an Automated Event Detection Method for Smart Manufacturing Systems" *Journal of Manufacturing and Materials Processing* 4, no. 3: 88.
https://doi.org/10.3390/jmmp4030088