# Multivariate Time-Series Classification of Critical Events from Industrial Drying Hopper Operations: A Deep Learning Approach

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Background and State of the Art

#### 2.1. Manufacturing Process

#### 2.2. Time-Series Classification in Manufacturing—Algorithms

#### 2.2.1. Traditional Algorithms

#### 2.2.2. Deep Learning Algorithms

## 3. Methodology

#### 3.1. Data Preprocessing

#### 3.1.1. Dataset Exploration and Missing Values

#### 3.1.2. Dataset Labeling

#### 3.1.3. Addressing the Imbalance Dataset Issue

#### 3.2. Selection and Application of DL Networks

#### 3.3. Performance Metrics

## 4. Results

#### 4.1. Experimental Setup

^{®}core TM i5-3337U CPU @ 1.8 GHz.

#### 4.2. Hyperparameter Tuning

#### 4.3. Experiment Results

## 5. Discussion

#### 5.1. Event Definition and Subsequence Extraction

#### 5.2. Data Imbalance Issue

#### 5.3. Result Interpretation

## 6. Conclusions and Future Works

## Author Contributions

## Funding

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Mccormick, M.R.; Wuest, T. Challenges for Smart Manufacturing and Industry 4.0 Research in Academia: A Case Study; ResearchGate: Berlin, Germany, 2023. [Google Scholar] [CrossRef]
- Oztemel, E.; Gursev, S. Literature review of Industry 4.0 and related technologies. J. Intell. Manuf.
**2020**, 31, 127–182. [Google Scholar] [CrossRef] - Hsu, C.-Y.; Liu, W.-C. Multiple time-series convolutional neural network for fault detection and diagnosis and empirical study in semiconductor manufacturing. J. Intell. Manuf.
**2021**, 32, 823–836. [Google Scholar] [CrossRef] - Jones, S.S.; Evans, R.S.; Allen, T.L.; Thomas, A.; Haug, P.J.; Welch, S.J.; Snow, G.L. A multivariate time series approach to modeling and forecasting demand in the emergency department. J. Biomed. Inform.
**2009**, 42, 123–139. [Google Scholar] [CrossRef] - Du, Z.; Lawrence, W.R.; Zhang, W.; Zhang, D.; Yu, S.; Hao, Y. Interactions between climate factors and air pollution on daily HFMD cases: A time series study in Guangdong, China. Sci. Total Environ.
**2019**, 656, 1358–1364. [Google Scholar] [CrossRef] - Perez-D’Arpino, C.; Shah, J.A. Fast target prediction of human reaching motion for cooperative human-robot manipulation tasks using time series classification. In Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA, 26–30 May 2015; pp. 6175–6182. [Google Scholar] [CrossRef]
- Farahani, M.A.; Vahid, A.; Goodwell, A.E. Evaluating Ecohydrological Model Sensitivity to Input Variability with an Information-Theory-Based Approach. Entropy
**2022**, 24, 994. [Google Scholar] [CrossRef] - Maknickienė, N.; Rutkauskas, A.V.; Maknickas, A. Investigation of financial market prediction by recurrent neural network. Innov. Technol. Sci. Bus. Educ.
**2011**, 2, 3–8. [Google Scholar] - Martín, L.; Zarzalejo, L.F.; Polo, J.; Navarro, A.; Marchante, R.; Cony, M. Prediction of global solar irradiance based on time series analysis: Application to solar thermal power plants energy production planning. Sol. Energy
**2010**, 84, 1772–1781. [Google Scholar] [CrossRef] - Farahani, M.A.; McCormick, M.R.; Gianinny, R.; Hudacheck, F.; Harik, R.; Liu, Z.; Wuest, T. Time-series pattern recognition in Smart Manufacturing Systems: A literature review and ontology. J. Manuf. Syst.
**2023**, 69, 208–241. [Google Scholar] [CrossRef] - Muth, J.F. Optimal Properties of Exponentially Weighted Forecasts. J. Am. Stat. Assoc.
**1960**, 55, 299–306. [Google Scholar] [CrossRef] - Bagnall, A.; Lines, J.; Bostrom, A.; Large, J.; Keogh, E. The great time series classification bake off: A review and experimental evaluation of recent algorithmic advances. Data Min. Knowl. Discov.
**2017**, 31, 606–660. [Google Scholar] [CrossRef] - Berndt, D.J.; Clifford, J. Using Dynamic Time Warping to Find Patterns in Time Series. In Proceedings of the 3rd International Conference on Knowledge Discovery and Data Mining, Seattle, WA, USA, 31 July–1 August 1994; pp. 359–370. [Google Scholar]
- Box, G.E.; Jenkins, G.M.; Reinsel, G.C.; Ljung, G.M. Time Series Analysis: Forecasting and Control; John Wiley & Sons: Hoboken, NJ, USA, 2015. [Google Scholar]
- He, G.; Li, Y.; Zhao, W. An uncertainty and density based active semi-supervised learning scheme for positive unlabeled multivariate time series classification. Knowl.-Based Syst.
**2017**, 124, 80–92. [Google Scholar] [CrossRef] - Tuballa, M.L.; Abundo, M.L. A review of the development of Smart Grid technologies. Renew. Sustain. Energy Rev.
**2016**, 59, 710–725. [Google Scholar] [CrossRef] - Batal, I.; Sacchi, L.; Bellazzi, R.; Hauskrecht, M. Multivariate Time Series Classification with Temporal Abstractions. In Proceedings of the Twenty-Second International FLAIRS Conference, Sanibel Island, FL, USA, 19–21 May 2009. [Google Scholar]
- Yang, K.; Shahabi, C. An efficient k nearest neighbor search for multivariate time series. Inf. Comput.
**2007**, 205, 65–98. [Google Scholar] [CrossRef] - Hills, J.; Lines, J.; Baranauskas, E.; Mapp, J.; Bagnall, A. Classification of time series by shapelet transformation. Data Min. Knowl. Discov.
**2014**, 28, 851–881. [Google Scholar] [CrossRef] - Chang, Y.; Rubin, J.; Boverman, G.; Vij, S.; Rahman, A.; Natarajan, A.; Parvaneh, S. A Multi-Task Imputation and Classification Neural Architecture for Early Prediction of Sepsis from Multivariate Clinical Time Series. In Proceedings of the 2019 Computing in Cardiology Conference, Singapore, 8–11 September 2019. [Google Scholar] [CrossRef]
- Lines, J.; Bagnall, A. Time series classification with ensembles of elastic distance measures. Data Min. Knowl. Discov.
**2015**, 29, 565–592. [Google Scholar] [CrossRef] - Fawaz, H.I.; Forestier, G.; Weber, J.; Idoumghar, L.; Muller, P.-A. Deep learning for time series classification: A review. Data Min. Knowl. Discov.
**2019**, 33, 917–963. [Google Scholar] [CrossRef] - Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet classification with deep convolutional neural networks. Commun. ACM
**2017**, 60, 84–90. [Google Scholar] [CrossRef] - Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, E.; Vanhoucke, V.; Rabinovich, A. Going deeper with convolutions. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015; pp. 1–9. [Google Scholar] [CrossRef]
- Bahdanau, D.; Cho, K.; Bengio, Y. Neural Machine Translation by Jointly Learning to Align and Translate. arXiv
**2016**, arXiv:1409.0473. [Google Scholar] - Sutskever, I.; Vinyals, O.; Le, Q.V. Sequence to Sequence Learning with Neural Networks. arXiv
**2014**, arXiv:1409.3215. [Google Scholar] - Alayba, A.M.; Palade, V.; England, M.; Iqbal, R. A Combined CNN and LSTM Model for Arabic Sentiment Analysis. In Machine Learning and Knowledge Extraction; Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E., Eds.; Lecture Notes in Computer Science; Springer International Publishing: Cham, Switzerland, 2018; Volume 11015, pp. 179–191. ISBN 978-3-319-99739-1. [Google Scholar]
- Sainath, T.N.; Kingsbury, B.; Mohamed, A.; Dahl, G.E.; Saon, G.; Soltau, H.; Beran, T.; Aravkin, A.Y.; Ramabhadran, B. Improvements to Deep Convolutional Neural Networks for LVCSR. In Proceedings of the 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, Olomouc, Czech Republic, 8–12 December 2013; pp. 315–320. [Google Scholar] [CrossRef]
- Liu, C.-L.; Hsaio, W.-H.; Tu, Y.-C. Time Series Classification With Multivariate Convolutional Neural Network. IEEE Trans. Ind. Electron.
**2019**, 66, 4788–4797. [Google Scholar] [CrossRef] - Huang, H.-S.; Liu, C.-L.; Tseng, V.S. Multivariate Time Series Early Classification Using Multi-Domain Deep Neural Network. In Proceedings of the 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA), Turin, Italy, 1–3 October 2018; pp. 90–98. [Google Scholar] [CrossRef]
- Yazdanbakhsh, O.; Dick, S. Multivariate Time Series Classification using Dilated Convolutional Neural Network. arXiv
**2019**, arXiv:1905.01697. [Google Scholar] - Karim, F.; Majumdar, S.; Darabi, H.; Harford, S. Multivariate LSTM-FCNs for Time Series Classification. Neural Netw.
**2019**, 116, 237–245. [Google Scholar] [CrossRef] [PubMed] - Guo, Z.; Liu, P.; Yang, J.; Hu, Y. Multivariate Time Series Classification Based on MCNN-LSTMs Network. In Proceedings of the 2020 12th International Conference on Machine Learning and Computing, Shenzhen, China, 15–17 February 2020; pp. 510–517. [Google Scholar] [CrossRef]
- Zheng, Y.; Liu, Q.; Chen, E.; Ge, Y.; Zhao, J.L. Exploiting multi-channels deep convolutional neural networks for multivariate time series classification. Front. Comput. Sci.
**2016**, 10, 96–112. [Google Scholar] [CrossRef] - Lei, K.-C.; Zhang, X.D. An approach on discretizing time series using recurrent neural network. In Proceedings of the 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Madrid, Spain, 3–6 December 2018; pp. 2522–2526. [Google Scholar] [CrossRef]
- Che, Z.; Purushotham, S.; Cho, K.; Sontag, D.; Liu, Y. Recurrent Neural Networks for Multivariate Time Series with Missing Values. Sci. Rep.
**2018**, 8, 6085. [Google Scholar] [CrossRef] - Lenz, J.; Swerdlow, S.; Landers, A.; Shaffer, R.; Geller, A.; Wuest, T. Smart Services for Polymer Processing Auxiliary Equipment: An Industrial Case Study. Smart Sustain. Manuf. Syst.
**2020**, 4, 20200032. [Google Scholar] [CrossRef] - Shokoohi-Yekta, M.; Wang, J.; Keogh, E. On the Non-Trivial Generalization of Dynamic Time Warping to the Multi-Dimensional Case. In Proceedings of the 2015 SIAM International Conference on Data Mining; Society for Industrial and Applied Mathematics, Vancouver, BC, Canada, 30 April–2 May 2015; pp. 289–297. [Google Scholar] [CrossRef]
- Kapp, V.; May, M.C.; Lanza, G.; Wuest, T. Pattern Recognition in Multivariate Time Series: Towards an Automated Event Detection Method for Smart Manufacturing Systems. J. Manuf. Mater. Process.
**2020**, 4, 88. [Google Scholar] [CrossRef] - Shen, J.; Huang, W.; Zhu, D.; Liang, J. A Novel Similarity Measure Model for Multivariate Time Series Based on LMNN and DTW. Neural Process. Lett.
**2017**, 45, 925–937. [Google Scholar] [CrossRef] - Mei, J.; Liu, M.; Wang, Y.-F.; Gao, H. Learning a Mahalanobis Distance-Based Dynamic Time Warping Measure for Multivariate Time Series Classification. IEEE Trans. Cybern.
**2016**, 46, 1363–1374. [Google Scholar] [CrossRef] - Vaughan, N.; Gabrys, B. Scoring and assessment in medical VR training simulators with dynamic time series classification. Eng. Appl. Artif. Intell.
**2020**, 94, 103760. [Google Scholar] [CrossRef] - Ircio, J.; Lojo, A.; Mori, U.; Lozano, J.A. Mutual information based feature subset selection in multivariate time series classification. Pattern Recognit.
**2020**, 108, 107525. [Google Scholar] [CrossRef] - Górecki, T.; Łuczak, M. Multivariate time series classification with parametric derivative dynamic time warping. Expert Syst. Appl.
**2015**, 42, 2305–2312. [Google Scholar] [CrossRef] - Seto, S.; Zhang, W.; Zhou, Y. Multivariate Time Series Classification Using Dynamic Time Warping Template Selection for Human Activity Recognition. In Proceedings of the 2015 IEEE Symposium Series on Computational Intelligence, Cape Town, South Africa, 1–7 December 2015; pp. 1399–1406. [Google Scholar] [CrossRef]
- Łuczak, M. Univariate and multivariate time series classification with parametric integral dynamic time warping. J. Intell. Fuzzy Syst.
**2017**, 33, 2403–2413. [Google Scholar] [CrossRef] - Baydogan, M.G.; Runger, G. Learning a symbolic representation for multivariate time series classification. Data Min. Knowl. Discov.
**2015**, 29, 400–422. [Google Scholar] [CrossRef] - Lin, J.; Keogh, E.; Wei, L.; Lonardi, S. Experiencing SAX: A novel symbolic representation of time series. Data Min. Knowl. Discov.
**2007**, 15, 107–144. [Google Scholar] [CrossRef] - Schäfer, P.; Högqvist, M. SFA: A symbolic fourier approximation and index for similarity search in high dimensional datasets. In Proceedings of the 15th International Conference on Extending Database Technology, Berlin, Germany, 27–30 March 2012; pp. 516–527. [Google Scholar] [CrossRef]
- Le Nguyen, T.; Gsponer, S.; Ilie, I.; O’Reilly, M.; Ifrim, G. Interpretable time series classification using linear models and multi-resolution multi-domain symbolic representations. Data Min. Knowl. Discov.
**2019**, 33, 1183–1222. [Google Scholar] [CrossRef] - Dhariyal, B.; Le Nguyen, T.; Gsponer, S.; Ifrim, G. An Examination of the State-of-the-Art for Multivariate Time Series Classification. In Proceedings of the 2020 International Conference on Data Mining Workshops (ICDMW), Sorrento, Italy, 17–20 November 2020; pp. 243–250. [Google Scholar] [CrossRef]
- Schäfer, P.; Leser, U. Multivariate Time Series Classification with WEASEL+MUSE 2018. arXiv
**2018**, arXiv:1711.11343. [Google Scholar] - 53. Zhao, B.; Lu, H.; Chen, S.; Liu, J.; Wu, D. Convolutional neural networks for time series classification. J. Syst. Eng. Electron.
**2017**, 28, 162–169. [Google Scholar] [CrossRef] - Ruiz, A.P.; Flynn, M.; Large, J.; Middlehurst, M.; Bagnall, A. The great multivariate time series classification bake off: A review and experimental evaluation of recent algorithmic advances. Data Min. Knowl. Discov.
**2021**, 35, 401–449. [Google Scholar] [CrossRef] - Song, W.; Liu, L.; Liu, M.; Wang, W.; Wang, X.; Song, Y. Representation Learning with Deconvolution for Multivariate Time Series Classification and Visualization. In Data Science; Zeng, J., Jing, W., Song, X., Lu, Z., Eds.; Springer: Singapore, 2020; Volume 1257, pp. 310–326. ISBN 9789811579806. [Google Scholar]
- Kiangala, K.S.; Wang, Z. An Effective Predictive Maintenance Framework for Conveyor Motors Using Dual Time-Series Imaging and Convolutional Neural Network in an Industry 4.0 Environment. IEEE Access
**2020**, 8, 121033–121049. [Google Scholar] [CrossRef] - Martínez-Arellano, G.; Terrazas, G.; Ratchev, S. Tool wear classification using time series imaging and deep learning. Int. J. Adv. Manuf. Technol.
**2019**, 104, 3647–3662. [Google Scholar] [CrossRef] - Sainath, T.N.; Vinyals, O.; Senior, A.; Sak, H. Convolutional, Long Short-Term Memory, fully connected Deep Neural Networks. In Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), South Brisbane, QLD, Australia, 19–24 April 2015; pp. 4580–4584. [Google Scholar] [CrossRef]
- Hsu, E.-Y.; Liu, C.-L.; Tseng, V.S. Multivariate Time Series Early Classification with Interpretability Using Deep Learning and Attention Mechanism. In Advances in Knowledge Discovery and Data Mining; Yang, Q., Zhou, Z.-H., Gong, Z., Zhang, M.-L., Huang, S.-J., Eds.; Lecture Notes in Computer Science; Springer International Publishing: Cham, Switzerland, 2019; Volume 11441, pp. 541–553. ISBN 978-3-030-16141-5. [Google Scholar]
- Khan, M.; Wang, H.; Ngueilbaye, A.; Elfatyany, A. End-to-end multivariate time series classification via hybrid deep learning architectures. Pers. Ubiquitous Comput.
**2023**, 27, 177–191. [Google Scholar] [CrossRef] - Tripathi, A.M. Enhancing Multivariate Time Series Classification Using LSTM and Evidence Feed Forward HMM. In Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK, 19–24 July 2020; pp. 1–7. [Google Scholar] [CrossRef]
- Gruenwald, L.; Chok, H.; Aboukhamis, M. Using Data Mining to Estimate Missing Sensor Data. In Proceedings of the Seventh IEEE International Conference on Data Mining Workshops (ICDMW 2007), Omaha, NE, USA, 28–31 October 2007; pp. 207–212. [Google Scholar] [CrossRef]
- Bishop, C.M. Neural Networks for Pattern Recognition; Clarendon Press: Oxford, UK; Oxford University Press: Oxford, NY, USA, 1995; ISBN 978-0-19-853849-3. [Google Scholar]
- Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; et al. Scikit-learn: Machine learning in Python. J. Mach. Learn. Res.
**2011**, 12, 2825–2830. [Google Scholar] - Ganganwar, V. An overview of classification algorithms for imbalanced datasets. Int. J. Emerg. Technol. Adv. Eng.
**2012**, 2, 42–47. [Google Scholar] - Zheng, Z. Oversampling Method for Imbalanced Classification. Comput. Inform.
**2015**, 34, 1017–1037. [Google Scholar] - Chawla, N.V.; Bowyer, K.W.; Hall, L.O.; Kegelmeyer, W.P. SMOTE: Synthetic Minority Over-sampling Technique. J. Artif. Intell. Res.
**2002**, 16, 321–357. [Google Scholar] [CrossRef] - He, H.; Ma, Y. Imbalanced Learning: Foundations, Algorithms, and Applications; John Wiley & Sons: Hoboken, NJ, USA, 2013. [Google Scholar]
- Sun, Z.; Song, Q.; Zhu, X.; Sun, H.; Xu, B.; Zhou, Y. A novel ensemble method for classifying imbalanced data. Pattern Recognit.
**2015**, 48, 1623–1637. [Google Scholar] [CrossRef]

**Figure 1.**A view of a drying hopper with different temperature zones [39].

**Figure 4.**Three major events in case study operation. Startup procedure (

**a**), cleaning cycle (

**b**), and conveying issue (

**c**) (adapted from [37]).

**Figure 5.**Change in distribution before normalization (

**left**) and after normalization (

**right**) (created based on [64]).

**Figure 7.**Ensemble learning method [69].

Dataset Statistics | Variable Types | ||
---|---|---|---|

Number of Variables | 13 | Numeric | 12 |

Number of Observations | 264,960 | Categorical | 1 |

Missing Cells (%) | 0% | ||

Duplicate Rows (%) | 0.5% | ||

Total size in memory | 26.3 MiB |

Dataset | Class | No. of Examples | Percentage |
---|---|---|---|

Training Set | Event (class 1) | 791 | 22.39% |

Non-event (class 0) | 2742 | 77.61% | |

Test Set | Event (class 1) | 50 | 5.66% |

Non-event (class 0) | 833 | 94.34% |

Model/Layer | Hyperparameter | Values |
---|---|---|

CNN | # Filters | 8, 16 |

Filter Size | 3, 5, 7 | |

Activation Function | ReLu | |

LSTM | No. of LSTM neurons | 100 |

Dropout | Dropout rate | 0.5 |

Max Pooling | Pooling Size | 2 |

Fully connected Layer | Dense Activation Function | ReLu |

Output Activation Function | Sigmoid | |

No. of Neurons | 200 | |

Model Parameters | Batch Size | 32, 64, 128 |

Learning Rate | 0.01 | |

No. of Epochs | 100 |

Ensemble | Metrics | Best Experimental Run | Average Accuracy of 10 Runs |
---|---|---|---|

Approach 1 | TP | 832 | 0.9866 |

TN | 49 | ||

FP | 1 | ||

FN | 1 | ||

Accuracy | 0.9977 | ||

Approach 2 | TP | 833 | 0.9900 |

TN | 48 | ||

FP | 2 | ||

FN | 0 | ||

Accuracy | 0.9977 | ||

Approach 3 | TP | 833 | 0.9930 |

TN | 49 | ||

FP | 1 | ||

FN | 0 | ||

Accuracy | 0.9989 |

Metrics | Run 1 | Run 2 | Run 3 | Run 4 | Run 5 | Run 6 | Run 7 | Run8 | Run 9 | Run 10 |
---|---|---|---|---|---|---|---|---|---|---|

TP | 833 | 833 | 833 | 831 | 833 | 833 | 833 | 831 | 833 | 833 |

TN | 49 | 49 | 47 | 49 | 48 | 49 | 49 | 49 | 49 | 48 |

FP | 1 | 1 | 3 | 1 | 2 | 1 | 1 | 1 | 1 | 2 |

FN | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 2 | 0 | 0 |

Accuracy | 0.9989 | 0.9989 | 0.9966 | 0.9966 | 0.9977 | 0.9989 | 0.9966 | 0.9966 | 0.9989 | 0.9977 |

Approach and Segment | App 3, Seg 2 | App 3, Seg 2 | App 1, Seg 4 | App 2, Seg 3 | App 2, Seg 2 | App 3, Seg 2 | App 3, Seg 2 | App 2, Seg 2 | App 1, Seg 2 | App 1, Seg 2 |

Ensemble | Metrics | Best Experimental Run | Average Accuracy of 10 Runs |
---|---|---|---|

Approach 1 | TP | 833 | 0.9874 |

TN | 48 | ||

FP | 2 | ||

FN | 0 | ||

Accuracy | 0.9977 | ||

Approach 2 | TP | 832 | 0.9855 |

TN | 48 | ||

FP | 2 | ||

FN | 1 | ||

Accuracy | 0.9966 | ||

Approach 3 | TP | 831 | 0.9905 |

TN | 48 | ||

FP | 2 | ||

FN | 2 | ||

Accuracy | 0.9955 |

Metrics | Run 1 | Run 2 | Run 3 | Run 4 | Run 5 | Run6 | Run 7 | Run 8 | Run 9 | Run 10 |
---|---|---|---|---|---|---|---|---|---|---|

TP | 833 | 831 | 832 | 832 | 833 | 827 | 833 | 833 | 828 | 831 |

TN | 47 | 49 | 47 | 48 | 47 | 48 | 48 | 48 | 49 | 49 |

FP | 3 | 1 | 3 | 2 | 3 | 2 | 2 | 2 | 1 | 1 |

FN | 0 | 2 | 1 | 1 | 0 | 6 | 0 | 0 | 5 | 2 |

Accuracy | 0.9966 | 0.9966 | 0.9955 | 0.9966 | 0.9966 | 0.9909 | 0.9977 | 0.9977 | 0.9932 | 0.9966 |

Approach and Segment | App 2, Seg 3 | App 2, Seg 3 | App 1, Seg 3 | App 3, Seg 2 | App 2, Seg 2 | App 1, Seg 3 | App 1, Seg 2 | App 1, Seg 2&3 | App 2, Seg 3 | App 2, Seg 3 |

Ensemble | Metrics | Best Experimental Run | Average Accuracy of 10 Runs |
---|---|---|---|

Approach 1 | TP | 833 | 0.9875 |

TN | 48 | ||

FP | 2 | ||

FN | 0 | ||

Accuracy | 0.9977 | ||

Approach 2 | TP | 833 | 0.9826 |

TN | 48 | ||

FP | 2 | ||

FN | 0 | ||

Accuracy | 0.9977 | ||

Approach 3 | TP | 833 | 0.9807 |

TN | 47 | ||

FP | 3 | ||

FN | 0 | ||

Accuracy | 0.9966 |

Metrics | Run 1 | Run 2 | Run 3 | Run 4 | Run 5 | Run 6 | Run 7 | Run 8 | Run 9 | Run 10 |
---|---|---|---|---|---|---|---|---|---|---|

TP | 833 | 832 | 832 | 833 | 833 | 832 | 832 | 832 | 833 | 831 |

TN | 49 | 48 | 48 | 48 | 48 | 47 | 48 | 46 | 46 | 49 |

FP | 1 | 2 | 2 | 2 | 2 | 3 | 2 | 4 | 4 | 1 |

FN | 0 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 0 | 2 |

Accuracy | 0.9989 | 0.9966 | 0.9966 | 0.9977 | 0.9977 | 0.9955 | 0.9966 | 0.9943 | 0.9955 | 0.9966 |

Approach and Segment | App 2, Seg 2 | App 1, Seg 4 | App 1, Seg 3 | App 1, Seg 2 | App 3, Seg 2 | App 2, Seg 3 | App 3, Seg 2 | App 2, Seg 2 | App 2, Seg 3 | App 1, Seg 5 |

Network | Metrics | Best Experimental Run | Average Accuracy of 10 Runs |
---|---|---|---|

CNN | TP | 833 | 0.9942 |

TN | 49 | ||

FP | 1 | ||

FN | 0 | ||

Accuracy | 0.9989 | ||

LSTM | TP | 832 | 0.9830 |

TN | 49 | ||

FP | 1 | ||

FN | 1 | ||

Accuracy | 0.9977 | ||

CNN-LSTM | TP | 832 | 0.9900 |

TN | 49 | ||

FP | 1 | ||

FN | 1 | ||

Accuracy | 0.9977 |

**Table 11.**Result summary of applying machine learning algorithms on the original dataset and augmented dataset using SMOTE.

Network | Metrics | KNN | SVM | NB | DT | RF | GB | Best Algorithm |
---|---|---|---|---|---|---|---|---|

Original Dataset | TP | 833 | 822 | 817 | 813 | 820 | 822 | SVM & GB |

TN | 38 | 49 | 48 | 50 | 49 | 49 | ||

FP | 12 | 1 | 2 | 0 | 1 | 1 | ||

FN | 0 | 11 | 16 | 20 | 13 | 11 | ||

Accuracy | 0.9864 | 0.9864 | 0.9796 | 0.9773 | 0.9841 | 0.9864 | ||

Run Number | 3 | 1 | 4 | 9 | 2 | 3 | ||

Average Accuracy | 0.9742 | 0.9789 | 0.9692 | 0.9621 | 0.9735 | 0.9684 | SVM | |

Augmented Dataset with SMOTE | TP | 833 | 818 | 817 | 815 | 815 | 820 | GB |

TN | 44 | 50 | 48 | 49 | 49 | 49 | ||

FP | 6 | 0 | 2 | 1 | 1 | 1 | ||

FN | 0 | 15 | 16 | 18 | 18 | 13 | ||

Accuracy | 0.9932 | 0.9830 | 0.9796 | 0.9785 | 0.9785 | 0.9841 | ||

Run Number | 6 | 2 | 5 | 1 | 8 | 6 | ||

Accuracy | 0.9813 | 0.9766 | 0.9703 | 0.9643 | 0.9601 | 0.9732 | KNN |

Method | Ensemble Learning | ||||||||
---|---|---|---|---|---|---|---|---|---|

Approach 1 | Approach 2 | Approach 3 | |||||||

Algorithm | CNN | LSTM | CNN-LSTM | CNN | LSTM | CNN-LSTM | CNN | LSTM | CNN-LSTM |

Accuracy | 0.9866 | 0.9874 | 0.9875 | 0.9900 | 0.9855 | 0.9826 | 0.9930 | 0.9905 | 0.9807 |

Method | SMOTE | ||||||||

Approach 1 | Approach 2 | Approach 3 | |||||||

Algorithm | CNN | LSTM | CNN-LSTM | KNN | SVM | NB | DT | RF | GB |

Accuracy | 0.9942 | 0.9830 | 0.9900 | 0.9813 | 0.9766 | 0.9703 | 0.9643 | 0.9601 | 0.9732 |

Method | Original Dataset | |||||
---|---|---|---|---|---|---|

Algorithm | KNN | SVM | NB | DT | RF | GB |

Accuracy | 0.9742 | 0.9789 | 0.9692 | 0.9621 | 0.9735 | 0.9684 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Rahman, M.M.; Farahani, M.A.; Wuest, T.
Multivariate Time-Series Classification of Critical Events from Industrial Drying Hopper Operations: A Deep Learning Approach. *J. Manuf. Mater. Process.* **2023**, *7*, 164.
https://doi.org/10.3390/jmmp7050164

**AMA Style**

Rahman MM, Farahani MA, Wuest T.
Multivariate Time-Series Classification of Critical Events from Industrial Drying Hopper Operations: A Deep Learning Approach. *Journal of Manufacturing and Materials Processing*. 2023; 7(5):164.
https://doi.org/10.3390/jmmp7050164

**Chicago/Turabian Style**

Rahman, Md Mushfiqur, Mojtaba Askarzadeh Farahani, and Thorsten Wuest.
2023. "Multivariate Time-Series Classification of Critical Events from Industrial Drying Hopper Operations: A Deep Learning Approach" *Journal of Manufacturing and Materials Processing* 7, no. 5: 164.
https://doi.org/10.3390/jmmp7050164