# Deep-Learning-Based Approach to Anomaly Detection Techniques for Large Acoustic Data in Machine Operation

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Related Work

## 3. Materials and Methods

#### 3.1. Acoustic Data Measurements for Anomaly Detection

#### 3.2. Dataset

_{t}) was 51,200.

_{s}) consisted of 5 error operations that were judged to be abnormal, and one operation of the machine was confirmed normally. The classifications of data are shown in Table 2.

#### 3.3. Feature Extraction

## 4. Anomaly Detection Deep Learning Models

## 5. Results and Discussions

#### 5.1. Results of the SVM

#### 5.2. Results of K-Means

#### 5.3. Results of the KNN

#### 5.4. Results of the CNN

#### 5.5. Comparison of F1 Score and Accuracy for Four Models

#### 5.6. Discussions

## 6. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Park, Y.-J.; Fan, S.-K.S.; Hsu, C.-Y. A review on fault detection and process diagnostics in industrial processes. Processes
**2020**, 8, 1123. [Google Scholar] [CrossRef] - Tsui, K.L.; Chen, N.; Zhou, Q.; Hai, Y.; Wang, W. Prognostics and health management: A review on data driven approaches. Math. Probl. Eng.
**2015**, 2015, 793161. [Google Scholar] [CrossRef] [Green Version] - Namuduri, S.; Narayanan, B.N.; Davuluru, V.S.P.; Burton, L.; Bhansali, S. Deep Learning Methods for Sensor Based Predictive Maintenance and Future Perspectives for Electrochemical Sensors. J. Electrochem. Soc.
**2020**, 167, 037552. [Google Scholar] [CrossRef] - Mazzetto, M.; Teixeira, M.; Rodrigues, É.O.; Casanova, D. Deep learning models for visual inspection on automotive assembling line. arXiv
**2020**, arXiv:2007.01857. [Google Scholar] [CrossRef] - Jiang, Y.; Wang, W.; Zhao, C. A machine vision-based realtime anomaly detection method for industrial products using deep learning. In Proceedings of the 2019 Chinese Automation Congress (CAC), Hangzhou, China, 22–24 November 2019; pp. 4842–4847. [Google Scholar]
- Yang, J.; Li, S.; Wang, Z.; Dong, H.; Wang, J.; Tang, S. Using deep learning to detect defects in manufacturing: A comprehensive survey and current challenges. Materials
**2020**, 13, 5755. [Google Scholar] [CrossRef] - Purohit, H.; Tanabe, R.; Ichige, K.; Endo, T.; Nikaido, Y.; Suefusa, K.; Kawaguchi, Y. MIMII Dataset: Sound dataset for malfunctioning industrial machine investigation and inspection. arXiv
**2019**, arXiv:1909.09347. [Google Scholar] - Baek, W.; Kim, D.Y. An in-process inspection system to detect noise originating from within the interior trim panels of car doors. Sensors
**2020**, 20, 630. [Google Scholar] [CrossRef] [Green Version] - Pech, M.; Vrchota, J.; Bednář, J. Predictive Maintenance and Intelligent Sensors in Smart Factory. Sensors
**2021**, 21, 1470. [Google Scholar] [CrossRef] - Ma, Y.; Wang, C.; Yang, D.; Wang, C. Adaptive Extraction Method Based on Time-Frequency Images for Fault Diagnosis in Rolling Bearings of Motor. Math. Probl. Eng.
**2021**, 2021, 6687195. [Google Scholar] [CrossRef] - Ahn, H.; Choi, H.-L.; Kang, M.; Moon, S. Learning-Based Anomaly Detection and Monitoring for Swarm Drone Flights. Appl. Sci.
**2019**, 9, 5477. [Google Scholar] [CrossRef] [Green Version] - Ahn, H.; Jung, D.; Choi, H.-L. Deep generative models-based anomaly detection for spacecraft control systems. Sensors
**2020**, 20, 1991. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Glowacz, A.; Tadeusiewicz, R.; Legutko, S.; Caesarendra, W.; Irfan, M.; Liu, H.; Brumercik, F.; Gutten, M.; Sulowicz, M.; Daviu, J.A.A. Fault diagnosis of angle grinders and electric impact drills using acoustic signals. Appl. Acoust.
**2021**, 179, 108070. [Google Scholar] [CrossRef] - Mukkamala, S.; Janoski, G.; Sung, A. Intrusion detection using neural networks and support vector machines. In Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN’02 (Cat. No. 02CH37290), Honolulu, HI, USA, 12–17 May 2002; pp. 1702–1707. [Google Scholar]
- Lu, Y.; Huang, Z. A new hybrid model of sparsity empirical wavelet transform and adaptive dynamic least squares support vector machine for fault diagnosis of gear pump. Adv. Mech. Eng.
**2020**, 12, 1687814020922047. [Google Scholar] [CrossRef] - Pablo Fernández, J.; Shubitidze, F.; Shamatava, I.; Barrowes, B.E.; O’Neill, K. Realistic subsurface anomaly discrimination using electromagnetic induction and an SVM classifier. Eurasip J. Adv. Signal Process.
**2010**, 2010, 305890. [Google Scholar] [CrossRef] [Green Version] - Malhotra, P.; Ramakrishnan, A.; Anand, G.; Vig, L.; Agarwal, P.; Shroff, G. LSTM-based encoder-decoder for multi-sensor anomaly detection. arXiv
**2016**, arXiv:1607.00148. [Google Scholar] - Koizumi, Y.; Saito, S.; Uematsu, H.; Kawachi, Y.; Harada, N. Unsupervised detection of anomalous sound based on deep learning and the neyman–pearson lemma. IEEE/ACM Trans. Audiospeechand Lang. Process.
**2018**, 27, 212–224. [Google Scholar] [CrossRef] [Green Version] - Chauhan, S.; Vig, L. Anomaly detection in ECG time signals via deep long short-term memory networks. In Proceedings of the 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA), Paris, France, 19–21 October 2015; pp. 1–7. [Google Scholar]
- Maya, S.; Ueno, K.; Nishikawa, T. dLSTM: A new approach for anomaly detection using deep learning with delayed prediction. Int. J. Data Sci. Anal.
**2019**, 8, 137–164. [Google Scholar] [CrossRef] [Green Version] - Malhotra, P.; Vig, L.; Shroff, G.; Agarwal, P. Long short term memory networks for anomaly detection in time series. In Proceedings of the ESANN 2015 Proceedings, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Bruges, Belgium, 22–24 April 2015; pp. 89–94. [Google Scholar]
- Gers, F.A.; Schmidhuber, J.; Cummins, F. Learning to forget: Continual prediction with LSTM. Neural Comput.
**2000**, 12, 2451–2471. [Google Scholar] [CrossRef] - Shi, X.; Chen, Z.; Wang, H.; Yeung, D.; Wong, W.; Woo, W. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. CoRR. arXiv
**2015**, arXiv:1506.04214. [Google Scholar] - Kim, T.-Y.; Cho, S.-B. Web traffic anomaly detection using C-LSTM neural networks. Expert Syst. Appl.
**2018**, 106, 66–76. [Google Scholar] [CrossRef] - Le, N.Q.K.; Huynh, T.-T. Identifying SNAREs by incorporating deep learning architecture and amino acid embedding representation. Front. Physiol.
**2019**, 10, 1501. [Google Scholar] [CrossRef] - Bojanowski, P.; Grave, E.; Joulin, A.; Mikolov, T. Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist.
**2017**, 5, 135–146. [Google Scholar] [CrossRef] [Green Version] - Lin, S.; Clark, R.; Birke, R.; Schönborn, S.; Trigoni, N.; Roberts, S. Anomaly Detection for Time Series Using VAE-LSTM Hybrid Model. In Proceedings of the ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, 4–8 May 2020; pp. 4322–4326. [Google Scholar]
- Theodoropoulos, P.; Spandonidis, C.C.; Themelis, N.; Giordamlis, C.; Fassois, S. Evaluation of different deep-learning models for the prediction of a ship’s propulsion power. J. Mar. Sci. Eng.
**2021**, 9, 116. [Google Scholar] [CrossRef] - Le, N.Q.K.; Do, D.T.; Hung, T.N.K.; Lam, L.H.T.; Huynh, T.-T.; Nguyen, N.T.K. A computational framework based on ensemble deep neural networks for essential genes identification. Int. J. Mol. Sci.
**2020**, 21, 9070. [Google Scholar] [CrossRef] - Cortes, C.; Vapnik, V. Support-vector networks. Mach. Learn.
**1995**, 20, 273–297. [Google Scholar] [CrossRef] - MacQueen, J. Some methods for classification and analysis of multivariate observations. Berkeley Symp. Math. Stat. Probab.
**1967**, 5.1, 281–297. [Google Scholar] - Altman, N.S. An introduction to kernel and nearest-neighbor nonparametric regression. Am. Stat.
**1992**, 46, 175–185. [Google Scholar] - d’Acremont, A.; Fablet, R.; Baussard, A.; Quin, G. CNN-based target recognition and identification for infrared imaging in defense systems. Sensors
**2019**, 19, 2040. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Chung, J.; Gulcehre, C.; Cho, K.; Bengio, Y. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv
**2014**, arXiv:1412.3555. [Google Scholar] - LeCun, Y.; Bengio, Y. Convolutional networks for images, speech, and time series. In The handbook of Brain Theory and Neural Networks; MIT Press: Cambridge, MA, USA, 1995; Volume 3361, p. 1995. [Google Scholar]
- Srivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; Salakhutdinov, R. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res.
**2014**, 15, 1929–1958. [Google Scholar] - Nair, V.; Hinton, G.E. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th International Conference on International Conference on Machine Learning, Haifa, Israel, 21 June 2020. [Google Scholar]
- LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature
**2015**, 521, 436–444. [Google Scholar] [CrossRef] [PubMed] - Liu, W.; Wen, Y.; Yu, Z.; Yang, M. Large-margin softmax loss for convolutional neural networks. arXiv
**2017**, arXiv:1612.02295v4. [Google Scholar] - Powers, D.M. Evaluation: From precision, recall and F-measure to ROC, informedness, markedness and correlation. arXiv
**2020**, arXiv:2010.16061. [Google Scholar] - Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv
**2014**, arXiv:1412.6980. [Google Scholar] - Azad, M.; Khaled, F.; Pavel, M. A novel approach to classify and convert 1D signal to 2D grayscale image implementing support vector machine and empirical mode decomposition algorithm. Int. J. Adv. Res.
**2019**, 7, 328–335. [Google Scholar] [CrossRef] [Green Version] - Chen, X.-W.; Lin, X. Big data deep learning: Challenges and perspectives. IEEE Access
**2014**, 2, 514–525. [Google Scholar] [CrossRef] - Gu, J.; Wang, Z.; Kuen, J.; Ma, L.; Shahroudy, A.; Shuai, B.; Liu, T.; Wang, X.; Wang, G.; Cai, J. Recent advances in convolutional neural networks. Pattern Recognit.
**2018**, 77, 354–377. [Google Scholar] [CrossRef] [Green Version] - Bengio, Y.; Lamblin, P.; Popovici, D.; Larochelle, H. Greedy layer-wise training of deep networks. In Advances in Neural Information Processing Systems; MIT Press: Cambridge, MA, USA, 2006; pp. 153–160. [Google Scholar]
- Bengio, Y.; Courville, A.; Vincent, P. Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell.
**2013**, 35, 1798–1828. [Google Scholar] [CrossRef] [PubMed]

**Figure 1.**Experimental set-up: (

**a**) measurement environment, (

**b**) data acquisition background noise locations and environment configuration.

Name | Model | Manufacturer | Specifications |
---|---|---|---|

Chassis | NI cDAQ-9189 | NI | 8-Slot, Extended Temperature Ethernet Compact DAQ Chassis |

Noise measurement module | NI 9232 | NI | Number of input channels: 3 ch Sampling Rate: 102.4 kS/s/ch Input Voltage: ±30 V C Series Sound and Vibration Input Module |

Digital input module | NI 9421 | NI | Input Voltage: 24 V Number of input channels: 8 ch (sinking input) Measurement speed: 100 μs C series digital module |

Microphone | 130F22 | PCB | Diameter: 1/4 inch Sensitivity: 45 mV/Pa Frequency Response (±4 dB): 10 to 20,000 Hz |

# of Files | Normal | Error #1 | Error #2 | Error #3 | Error #4 | Error #5 |
---|---|---|---|---|---|---|

13,162 | 6104 | 3760 | 394 | 903 | 1274 | 727 |

Classifier | Parameters |
---|---|

SVM | random_state = 42, max_iter = 100,000 |

K-means | n_components = 300 |

KNN | k = 6 |

CNN | n_filters = 16/64/64, dropout = 0.25, optimizer = adam |

# of Training Data | # of Testing Data | # of Total Data | |
---|---|---|---|

200 Dataset | 160 | 36 | 196 |

500 Dataset | 400 | 99 | 499 |

1k Dataset | 800 | 196 | 996 |

3k Dataset | 2400 | 583 | 2983 |

10k Dataset | 8000 | 1236 | 9236 |

Precision Score | Recall Score | F1 Score | Accuracy | |
---|---|---|---|---|

200 Dataset | 0.25 | 0.83 | 0.38 | 0.36 |

500 Dataset | 0.4 | 0.64 | 0.49 | 0.43 |

1k Dataset | 0.53 | 0.82 | 0.65 | 0.5 |

3k Dataset | 0.75 | 0.7 | 0.73 | 0.56 |

10k Dataset | 0.74 | 0.73 | 0.73 | 0.58 |

Precision Score | Recall Score | F1 Score | Accuracy | |
---|---|---|---|---|

200 Dataset | 0.36 | 0.63 | 0.45 | 0.25 |

500 Dataset | 0.29 | 0.66 | 0.41 | 0.23 |

1k Dataset | 0.06 | 1 | 0.11 | 0.14 |

3k Dataset | 0.15 | 0.53 | 0.24 | 0.15 |

10k Dataset | 0.12 | 0.39 | 0.19 | 0.15 |

Precision Score | Recall Score | F1 Score | Accuracy | |
---|---|---|---|---|

200 Dataset | 0.88 | 0.59 | 0.71 | 0.56 |

500 Dataset | 0.83 | 0.65 | 0.73 | 0.6 |

1k Dataset | 0.88 | 0.74 | 0.81 | 0.68 |

3k Dataset | 0.88 | 0.69 | 0.77 | 0.68 |

10k Dataset | 0.9 | 0.72 | 0.8 | 0.7 |

Precision Score | Recall Score | F1 Score | Accuracy | |
---|---|---|---|---|

200 Dataset | 0.94 | 0.79 | 0.86 | 0.72 |

500 Dataset | 0.86 | 0.79 | 0.82 | 0.73 |

1k Dataset | 0.94 | 0.89 | 0.91 | 0.82 |

3k Dataset | 0.85 | 0.88 | 0.86 | 0.83 |

10k Dataset | 0.95 | 0.92 | 0.93 | 0.90 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Ahn, H.; Yeo, I.
Deep-Learning-Based Approach to Anomaly Detection Techniques for Large Acoustic Data in Machine Operation. *Sensors* **2021**, *21*, 5446.
https://doi.org/10.3390/s21165446

**AMA Style**

Ahn H, Yeo I.
Deep-Learning-Based Approach to Anomaly Detection Techniques for Large Acoustic Data in Machine Operation. *Sensors*. 2021; 21(16):5446.
https://doi.org/10.3390/s21165446

**Chicago/Turabian Style**

Ahn, Hyojung, and Inchoon Yeo.
2021. "Deep-Learning-Based Approach to Anomaly Detection Techniques for Large Acoustic Data in Machine Operation" *Sensors* 21, no. 16: 5446.
https://doi.org/10.3390/s21165446