# Anomaly Detection in Railway Sensor Data Environments: State-of-the-Art Methods and Empirical Performance Evaluation

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Related Work

## 3. Systematic Literature Review

#### 3.1. Research Methodology

#### 3.2. Research Context and Goal

#### 3.3. Research Questions

- RQ1:
- What anomaly detection methods have been applied to sensor data environments?
- RQ2:
- What advantages and disadvantages do these methods have?

#### 3.4. Keywords

#### 3.5. Source Database

#### 3.6. Inclusion Criteria

- The selection process strictly follows the search string (IC1).
- Only full-length articles are considered (IC2).
- The subject subject area covers computer science (IC3).
- The research is published in the English language (IC4).
- The time period covers the last five years that is 2018–2023 (IC5).

#### 3.7. Exclusion Criteria

- Articles other than English are excluded (EC1).
- The availability of the document is restricted (EC2).
- The study did not concern any anomaly detection methods (EC3).

#### 3.8. Search Execution

#### 3.9. Data Analysis and Synthesis

## 4. Anomaly Detection Methods in Sensor Data Environments

#### 4.1. Identified Methods

#### 4.2. Advantages and Disadvantages

#### 4.2.1. One-Class Support Vector Machine (OCSVM)

#### 4.2.2. Local Outlier Factor (LOF)

#### 4.2.3. Isolation Forest (IF)

#### 4.2.4. Gaussian Hidden Markov Model (Gaussian HMM)

#### 4.2.5. Naive Bayes

#### 4.2.6. Long Short-Term Memory (LSTM) Networks

#### 4.2.7. Artificial Neural Networks (ANNs)

#### 4.2.8. Support Vector Classification (SVC)

#### 4.2.9. Multi-Layer Perceptron (MLP)

#### 4.2.10. Logistic Regression

#### 4.2.11. Support Vector Regression (SVR)

#### 4.2.12. Recurrent Neural Networks (RNNs)

#### 4.2.13. 1D Convolutional Neural Networks (1D-CNNs)

#### 4.2.14. The k-Nearest Neighbors (kNN)

#### 4.2.15. Decision Trees (DTs)

#### 4.2.16. Adaptive Boosting (AdaBoost)

#### 4.2.17. eXtreme Gradient Boosting (XGBoost)

#### 4.2.18. Random Forest (RF)

#### 4.2.19. CatBoost

## 5. Performance Evaluation

#### 5.1. Input

- 0: Checked, No Anomalies Detected: This indicates that the vibration data were checked, and no anomalies were detected.
- 1: Anomaly Detection: Anomalies were detected in the vibration data, raising an alert.
- 2: Impossible to Determine: This category is used when there are issues with the data, such as missing or incorrect information.

#### 5.2. Data Analysis

#### 5.3. Data Preparation for Training Machine Learning Models

- Data Cleaning. As part of the data cleaning process, rows containing ‘vibration_value’ equal to 2 or zero were removed from the dataset. This step was essential to maintain data integrity and ensure that the training data were free from inconsistencies or missing values that could compromise the quality of the models.
- Normalization. This step aimed to subtract the mean (average) value of a feature from each data point and then divide it by the standard deviation. This method ensures that the values of different features are on a common scale, with a mean of 0 and a standard deviation of 1, making them suitable for different machine learning algorithms.
- Additional Transformations. In addition to normalization, other transformations such as one-hot encoding were applied as needed. One-hot encoding was used to convert categorical variables into a binary matrix format, allowing machine learning models to work effectively with categorical data. This technique creates binary columns for each category within a categorical variable, ensuring that categorical information is appropriately represented in a numerical format.

#### 5.4. Methods Implementation

#### 5.4.1. Local Outlier Factor (LOF)

`sklearn.neighbors.LocalOutlierFactor`with the following parameters: n_neighbors = 20, algorithm = ‘auto’, leaf_size = 30, metric = ‘minkowski’, p = 2, metric_params = None, contamination = ‘auto’, novelty = True, and n_jobs = None.

#### 5.4.2. Isolation Forest (IF)

`sklearn.ensemble.IsolationForest`with the following parameters: n_estimators = 100, max_samples = ‘auto’, contamination = ‘auto’, max_features = 1.0, bootstrap = False, n_jobs = None, random_state = None, verbose = 0, and warm_start = False.

#### 5.4.3. Gaussian Hidden Markov Model (GHMM)

`hmmlearn.hmm.GaussianHMM`with the following parameters: n_components = 2, covariance_type = ‘diag’, min_covar = 0.001, startprob_prior = 1.0, transmat_prior = 1.0, means_prior = 0, means_weight = 0, covars_prior = 0.01, covars_weight = 1, algorithm = ‘viterbi’, random_state = 42, n_iter = 50, tol = 0.01, verbose = False, params = ‘stmc’, init_params = ‘stmc’, and implementation = ‘log’.

#### 5.4.4. Naive Bayes

`sklearn.naive_bayes.GaussianNB`with the following parameters: priors = None, var_smoothing = $1\times {10}^{-9}$.

#### 5.4.5. Support Vector Classification (SVC)

`sklearn.svm.SVC`with the following parameters: C = 1.0, kernel = ‘rbf’, degree = 3, gamma = ‘scale’, coef0 = 0.0, shrinking = True, probability = False, tol = 0.001, cache_size = 200, class_weight = None, verbose = False, max_iter = −1, decision_function_shape = ‘ovr’, break_ties = False, and random_state = None.

#### 5.4.6. Support Vector Regression (SVR)

`sklearn.svm.SVR`with the following parameters: kernel = ‘rbf’, degree = 3, gamma = ‘scale’, coef0 = 0.0, tol = 0.001, C = 1.0, epsilon = 0.1, shrinking = True, cache_size = 200, verbose = False, and max_iter = −1.

#### 5.4.7. One-Class SVM

`sklearn.svm.OneClassSVM`with the following parameters: kernel = ‘rbf’, degree = 3, gamma = ‘scale’, coef0 = 0.0, tol = 0.001, nu = 0.5, shrinking = True, cache_size = 200, verbose = False, and max_iter = −1.

#### 5.4.8. Logistic Regression

`sklearn.linear_model.LogisticRegression`with the following parameters: penalty = ‘l2’, dual = False, tol = 0.0001, C = 1.0, fit_intercept = True, intercept_scaling = 1, class_weight = None, random_state = None, solver = ‘lbfgs’, max_iter = 100, multi_class = ‘auto’, verbose = 0, warm_start = False, n_jobs = None, and l1_ratio = None.

#### 5.4.9. K-Nearest Neighbors (KNN)

`Manhattan distance metric`for anomaly detection.

#### 5.4.10. Decision Tree

`sklearn.tree.DecisionTreeClassifier`with the following parameters: criterion = ‘gini’, splitter = ‘best’, max_depth = None, min_samples_split = 2, min_samples_leaf = 1, min_weight_fraction_leaf = 0.0, max_features = None, random_state = None, max_leaf_nodes = None, min_impurity_decrease = 0.0, class_weight = None, and ccp_alpha = 0.0.

#### 5.4.11. Random Forest

`sklearn.ensemble.RandomForestClassifier`with the following parameters: n_estimators = 100, criterion = ‘gini’, max_depth = None, min_samples_split = 2, min_samples_leaf = 1, min_weight_fraction_leaf = 0.0, max_features = ‘sqrt’, max_leaf_nodes = None, min_impurity_decrease = 0.0, bootstrap = True, oob_score = False, n_jobs = None, random_state = None, verbose = 0, warm_start = False, class_weight = None, ccp_alpha = 0.0, and max_samples = None.

#### 5.4.12. AdaBoost

`sklearn.ensemble.AdaBoostClassifier`with the following parameters: estimator = None, n_estimators = 50, learning_rate = 1.0, algorithm = ‘SAMME.R’, and random_state = None.

#### 5.4.13. XGBoost

`xgboost.XGBClassifier`with the default parameters.

#### 5.4.14. CatBoost

`catboost.CatBoostClassifier`with the default parameters.

#### 5.4.15. Artificial Neural Network (ANN)

#### 5.4.16. Multilayer Perceptrons (MLPs)

`sklearn.neural_network.MLPClassifier`with the following parameters: hidden_layer_sizes = (100), activation = ‘relu’, solver = ‘adam’, alpha = 0.0001, batch_size = ‘auto’, learning_rate = ‘constant’, learning_rate_init = 0.001, power_t = 0.5, max_iter = 200, shuffle = True, random_state = None, tol = 0.0001, verbose = False, warm_start = False, momentum = 0.9, nesterovs_momentum = True, early_stopping = True, validation_fraction = 0.1, beta_1 = 0.9, beta_2 = 0.999, epsilon = $1\times {10}^{-8}$, n_iter_no_change = 10, and max_fun = 15,000.

#### 5.4.17. Recurrent Neural Networks (RNNs)

#### 5.4.18. Long Short-Term Memory (LSTM)

#### 5.4.19. One-Dimensional Convolutional Neural Networks (1D-CNNs)

#### 5.5. K-Fold Cross-Validation

- Data Splitting. The dataset was first divided into five parts of approximately equal size, with each part acting as a fold. This division ensured that the distribution of data across the folds was maintained as far as possible.
- Model Training and Testing. The cross-validation process was iterated five times, with each iteration using a different fold as the test set, while the remaining four folds collectively served as the training set. Each fold had the opportunity to be the test set once, while the model was trained on the rest of the data.
- Performance Evaluation. After training on one set and testing on another, the model’s performance metrics were recorded for that particular iteration. This step ensured that the model’s performance was assessed comprehensively across different parts of the dataset.
- Average Performance. To provide a more robust and reliable estimate of the model’s performance, the performance metrics from all five iterations were averaged. This average provided a single, representative measure of the model’s effectiveness at predicting outcomes.

#### 5.6. Results

#### 5.6.1. Evaluation Metrics

#### 5.6.2. Method Comparison

- The most effective model in terms of applied metrics is CatBoost, with an impressive accuracy of 96% and an F1 Score of 94%. This suggests that CatBoost is particularly well-suited for detecting anomalies in the given dataset.
- Among the traditional machine learning models, Decision Tree, AdaBoost, XGBoost and Random Forest have relatively high accuracy percentages, ranging from 86% to 91%. These models also maintain commendable Precision and Recall, suggesting their effectiveness in balancing the identification of anomalies with minimizing false alarms.
- Models such as OCSVM, LOF, Isolation Forest, and GHMM, with lower accuracy scores of 48% and 56%, also exhibit significant weaknesses across other metrics, highlighting their limited utility for this specific anomaly detection task.
- Deep learning models show varied performance, with recursion-based and simple ANN models underperforming across all metrics, while 1D-CNN achieves the highest F1 score of 73% within this group, indicating its capability to balance Precision and Recall effectively.

## 6. Discussion

#### 6.1. Study Contributions

#### 6.2. Study Limitations

#### 6.3. Future Research

- Include more attributes. Extending the analysis to include more attributes could provide a more comprehensive understanding of the factors influencing anomaly detection.
- Hyperparameter optimization. Perform a thorough hyperparameter optimization for all methods to ensure that each is operating at peak performance.
- Test other datasets. Evaluating methods based on the other datasets will give a better understanding of their performance.
- Implement other methods. Investigate additional anomaly detection methods not considered in this study to assess their potential effectiveness in this particular domain.

#### 6.4. Unexpected Results

## 7. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Strelko, O.; Pylypchuk, O. Characteristics of unpaved roads in the late 18th century–early 19th century, and the design of the first wooden trackway as a forerunner to the Bukovyna railways. Hist. Sci. Technol.
**2021**, 11, 437–452. [Google Scholar] [CrossRef] - Kalinowski, M.; Weichbroth, P. The sensors-based artificial intelligence Train Control and Monitoring System (TCMS) for managing the railway transport fleet. Rail Veh./Pojazdy Szyn.
**2023**, 20. [Google Scholar] [CrossRef] - Chandran, P.; Asber, J.; Thiery, F.; Odelius, J.; Rantatalo, M. An investigation of railway fastener detection using image processing and augmented deep learning. Sustainability
**2021**, 13, 12051. [Google Scholar] [CrossRef] - Silva, R.; Ribeiro, D.; Bragança, C.; Costa, C.; Arêde, A.; Calçada, R. Model updating of a freight wagon based on dynamic tests under different loading scenarios. Appl. Sci.
**2021**, 11, 10691. [Google Scholar] [CrossRef] - Famurewa, S.M.; Asplund, M.; Rantatalo, M.; Parida, A.; Kumar, U. Maintenance analysis for continuous improvement of railway infrastructure performance. Struct. Infrastruct. Eng.
**2015**, 11, 957–969. [Google Scholar] [CrossRef] - WorldAtlas. Highest Railway Passenger Traffic in the World. 2022. Available online: https://www.worldatlas.com/articles/highest-railway-passenger-traffic-in-the-world.html (accessed on 15 September 2022).
- Statista. Rail Passenger Traffic in Europe from 2012 to 2019 (in Billion Passenger Kilometers). 2022. Available online: https://www.statista.com/statistics/1104588/railpassenger-traffic-europe/ (accessed on 11 September 2022).
- Statista. Projected Global Rail Freight Market Size between 2020 and 2026 (in Billion U.S. Dollars). 2022. Available online: https://www.statista.com/statistics/1261412/railfreight-market-worldwide-forecast/ (accessed on 29 August 2023).
- Nielsen, J.C.; Johansson, A. Out-of-round railway wheels-a literature survey. Proc. Inst. Mech. Eng. Part F J. Rail Rapid Transit
**2000**, 214, 79–91. [Google Scholar] [CrossRef] - Wan, T.H.; Tsang, C.W.; Hui, K.; Chung, E. Anomaly detection of train wheels utilizing short-time Fourier transform and unsupervised learning algorithms. Eng. Appl. Artif. Intell.
**2023**, 122, 106037. [Google Scholar] [CrossRef] - Liu, Y.; Bao, Y. Intelligent monitoring of spatially-distributed cracks using distributed fiber optic sensors assisted by deep learning. Measurement
**2023**, 220, 113418. [Google Scholar] [CrossRef] - Liu, Y.; Bao, Y. Review on automated condition assessment of pipelines with machine learning. Adv. Eng. Inform.
**2022**, 53, 101687. [Google Scholar] [CrossRef] - Mardini, M.T.; Iraqi, Y.; Agoulmine, N. A survey of healthcare monitoring systems for chronically ill patients and elderly. J. Med Syst.
**2019**, 43, 50. [Google Scholar] [CrossRef] - Liu, Y.; Hajj, M.; Bao, Y. Review of robot-based damage assessment for offshore wind turbines. Renew. Sustain. Energy Rev.
**2022**, 158, 112187. [Google Scholar] [CrossRef] - Saia, R. Internet of entities (IoE): A blockchain-based distributed paradigm to security. arXiv
**2018**, arXiv:1808.08809. [Google Scholar] - Latif, S.; Afzaal, H.; Zafar, N.A. Intelligent traffic monitoring and guidance system for smart city. In Proceedings of the 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), Sukkur, Pakistan, 3–4 March 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 1–6. [Google Scholar]
- Origlia, A.; Di Martino, S.; Battista, E. Rail anomalies detection: A comparative analysis of three self-supervised models on real data. Comput. Ind.
**2023**, 148, 103909. [Google Scholar] [CrossRef] - Xu, Y.; Liu, J. High-speed train fault detection with unsupervised causality-based feature extraction methods. Adv. Eng. Inform.
**2021**, 49, 101312. [Google Scholar] [CrossRef] - Park, J.; Kim, S.; Choi, J.H.; Lee, S.H. Frequency energy shift method for bearing fault prognosis using microphone sensor. Mech. Syst. Signal Process.
**2021**, 147, 107068. [Google Scholar] [CrossRef] - Islam, U.; Malik, R.Q.; Al-Johani, A.S.; Khan, M.R.; Daradkeh, Y.I.; Ahmad, I.; Alissa, K.A.; Abdul-Samad, Z.; Tag-Eldin, E.M. A Novel Anomaly Detection System on the Internet of Railways Using Extended Neural Networks. Electronics
**2022**, 11, 2813. [Google Scholar] [CrossRef] - Zuo, Y.; Thiery, F.; Chandran, P.; Odelius, J.; Rantatalo, M. Squat Detection of Railway Switches and Crossings Using Wavelets and Isolation Forest. Sensors
**2022**, 22, 6357. [Google Scholar] [CrossRef] - Steenwinckel, B.; De Paepe, D.; Hautte, S.V.; Heyvaert, P.; Bentefrit, M.; Moens, P.; Dimou, A.; Van Den Bossche, B.; De Turck, F.; Van Hoecke, S.; et al. FLAGS: A methodology for adaptive anomaly detection and root cause analysis on sensor data streams by fusing expert knowledge with machine learning. Future Gener. Comput. Syst.
**2021**, 116, 30–48. [Google Scholar] [CrossRef] - Hesser, D.F.; Altun, K.; Markert, B. Monitoring and tracking of a suspension railway based on data-driven methods applied to inertial measurements. Mech. Syst. Signal Process.
**2022**, 164, 108298. [Google Scholar] [CrossRef] - Vos, K.; Peng, Z.; Jenkins, C.; Shahriar, M.R.; Borghesani, P.; Wang, W. Vibration-based anomaly detection using LSTM/SVM approaches. Mech. Syst. Signal Process.
**2022**, 169, 108752. [Google Scholar] [CrossRef] - Cunha, B.Z.; Droz, C.; Zine, A.M.; Foulard, S.; Ichchou, M. A review of machine learning methods applied to structural dynamics and vibroacoustic. Mech. Syst. Signal Process.
**2023**, 200, 110535. [Google Scholar] [CrossRef] - Kitchenham, B.A.; Travassos, G.H.; Von Mayrhauser, A.; Niessink, F.; Schneidewind, N.F.; Singer, J.; Takada, S.; Vehvilainen, R.; Yang, H. Towards an ontology of software maintenance. J. Softw. Maint. Res. Pract.
**1999**, 11, 365–389. [Google Scholar] [CrossRef] - Kitchenham, B.; Charters, S. Procedures for performing systematic literature reviews in software engineering. Keele Univ. Durh. Univ. UK
**2007**, 33, 710. [Google Scholar] - Kitchenham, B.A.; Pfleeger, S.L.; Pickard, L.M.; Jones, P.W.; Hoaglin, D.C.; El Emam, K.; Rosenberg, J. Preliminary guidelines for empirical research in software engineering. IEEE Trans. Softw. Eng.
**2002**, 28, 721–734. [Google Scholar] [CrossRef] - Schotten, M.; Meester, W.J.; Steiginga, S.; Ross, C.A. A brief history of Scopus: The world’s largest abstract and citation database of scientific literature. In Research Analytics; Auerbach Publications: Boca Raton, FL, USA, 2017; pp. 31–58. [Google Scholar]
- Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Int. J. Surg.
**2021**, 88, 105906. [Google Scholar] [CrossRef] - Sayers, A. Tips and tricks in performing a systematic review. Br. J. Gen. Pract.
**2008**, 58, 136. [Google Scholar] [CrossRef] [PubMed] - Felizardo, K.R.; da Silva, A.Y.I.; de Souza, É.F.; Vijaykumar, N.L.; Nakagawa, E.Y. Evaluating strategies for forward snowballing application to support secondary studies updates: Emergent results. In Proceedings of the Xxxii Brazilian Symposium on Software Engineering, Carlos, Brazil, 17–21 September 2018; pp. 184–189. [Google Scholar]
- Fernandes, G.; Rodrigues, J.J.; Carvalho, L.F.; Al-Muhtadi, J.F.; Proença, M.L. A comprehensive survey on network anomaly detection. Telecommun. Syst.
**2019**, 70, 447–489. [Google Scholar] [CrossRef] - Britannica. Information Theory. Mathematics. 2023. Available online: https://www.britannica.com/science/information-theory (accessed on 11 October 2023).
- Zhu, W.; Zhong, P. A new one-class SVM based on hidden information. Knowl.-Based Syst.
**2014**, 60, 35–43. [Google Scholar] [CrossRef] - Alghushairy, O.; Alsini, R.; Soule, T.; Ma, X. A review of local outlier factor algorithms for outlier detection in big data streams. Big Data Cogn. Comput.
**2020**, 5, 1. [Google Scholar] [CrossRef] - Karczmarek, P.; Kiersztyn, A.; Pedrycz, W.; Al, E. K-Means-based isolation forest. Knowl.-Based Syst.
**2020**, 195, 105659. [Google Scholar] [CrossRef] - Lefèvre, F.; Montacié, C.; Caraty, M.J. On the influence of the delta coefficients in a HMM-based speech recognition system. In Proceedings of the Fifth International Conference on Spoken Language Processing, Sydney, Australia, 30 November–4 December 1998. [Google Scholar]
- Ganapathiraju, A.; Hamaker, J.; Picone, J. Hybrid SVM/HMM architectures for speech recognition. In Proceedings of the INTERSPEECH, Citeseer, Beijing, China, 16–20 October 2000; pp. 504–507. [Google Scholar]
- Zhang, H.; Li, D. Naïve Bayes text classifier. In Proceedings of the 2007 IEEE International Conference on Granular Computing (GRC 2007), San Jose, CA, USA, 2–4 November 2007; IEEE: Piscataway, NJ, USA, 2007; p. 708. [Google Scholar]
- Amor, N.B.; Benferhat, S.; Elouedi, Z. Naive bayes vs decision trees in intrusion detection systems. In Proceedings of the 2004 ACM Symposium on Applied Computing, Nicosia, Cyprus, 14–17 March 2004; pp. 420–424. [Google Scholar]
- Murphy, K.P. Naive bayes classifiers. Univ. Br. Columbia
**2006**, 18, 1–8. [Google Scholar] - Zeng, C.; Ma, C.; Wang, K.; Cui, Z. Parking occupancy prediction method based on multi factors and stacked GRU-LSTM. IEEE Access
**2022**, 10, 47361–47370. [Google Scholar] [CrossRef] - Tu, J.V. Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. J. Clin. Epidemiol.
**1996**, 49, 1225–1231. [Google Scholar] [CrossRef] [PubMed] - Mijwel, M.M. Artificial Neural Networks Advantages and Disadvantages. 2018, p. 21. Available online: https://www.linkedin.com/pulse/artificial-neural-networks-advantages-disadvantages-maad-m-mijwel (accessed on 15 December 2023).
- Jakkula, V. Tutorial on support vector machine (svm). Sch. EECS Wash. State Univ.
**2006**, 37, 3. [Google Scholar] - Liu, J.; Chen, X.X.; Fang, L.; Li, J.X.; Yang, T.; Zhan, Q.; Tong, K.; Fang, Z. Mortality prediction based on imbalanced high-dimensional ICU big data. Comput. Ind.
**2018**, 98, 218–225. [Google Scholar] [CrossRef] - Wu, H.; Zhao, J. An intelligent vision-based approach for helmet identification for work safety. Comput. Ind.
**2018**, 100, 267–277. [Google Scholar] [CrossRef] - de Souza, D.L.; Granzotto, M.H.; de Almeida, G.M.; Oliveira-Lopes, L.C. Fault detection and diagnosis using support vector machines-a SVC and SVR comparison. J. Saf. Eng.
**2014**, 3, 18–29. [Google Scholar] [CrossRef] - Cervantes, J.; Garcia-Lamont, F.; Rodríguez-Mazahua, L.; Lopez, A. A comprehensive survey on support vector machine classification: Applications, challenges and trends. Neurocomputing
**2020**, 408, 189–215. [Google Scholar] [CrossRef] - Yin, J.; Li, Q. A semismooth Newton method for support vector classification and regression. Comput. Optim. Appl.
**2019**, 73, 477–508. [Google Scholar] [CrossRef] - Khanna, R.; Awad, M. Efficient Learning Machines: Theories, Concepts, and Applications for Engineers and System Designers; Springer Nature: Berlin/Heidelberg, Germany, 2015. [Google Scholar] [CrossRef]
- Park, Y.S.; Lek, S. Chapter 7—Artificial Neural Networks: Multilayer Perceptron for Ecological Modeling. In Ecological Model Types; Developments in Environmental modeling; Jørgensen, S.E., Ed.; Elsevier: Amsterdam, The Netherlands, 2016; Volume 28, pp. 123–140. [Google Scholar] [CrossRef]
- Akkaya, B.; Çolakoğlu, N. Comparison of multi-class classification algorithms on early diagnosis of heart diseases. In BIS 2019 Conference Book: Recent Advances in Data Science and Business Analyst, Proceedings of the 22nd International Conference on Business Information Systems, Istanbul, Turkey, 25–28 September 2019; Springer: Berlin/Heidelberg, Germany, 2019; pp. 162–171. [Google Scholar]
- Anguita, D.; Ghio, A.; Greco, N.; Oneto, L.; Ridella, S. Model selection for support vector machines: Advantages and disadvantages of the Machine Learning Theory. In Proceedings of the 2010 International Joint Conference on Neural Networks (IJCNN), Barcelona, Spain, 18–23 July 2010; pp. 1–8. [Google Scholar] [CrossRef]
- Ackerson, J.M.; Dave, R.; Seliya, N. Applications of recurrent neural network for biometric authentication & anomaly detection. Information
**2021**, 12, 272. [Google Scholar] [CrossRef] - Brezak, D.; Bacek, T.; Majetic, D.; Kasac, J.; Novakovic, B. A comparison of feed-forward and recurrent neural networks in time series forecasting. In Proceedings of the 2012 IEEE Conference on Computational Intelligence for Financial Engineering & Economics (CIFEr), New York, NY, USA, 29–30 March 2012; pp. 1–6. [Google Scholar] [CrossRef]
- Fang, W.; Chen, Y.; Xue, Q. Survey on research of RNN-based spatio-temporal sequence prediction algorithms. J. Big Data
**2021**, 3, 97. [Google Scholar] [CrossRef] - Pearlmutter, B.A. Gradient calculations for dynamic recurrent neural networks: A survey. IEEE Trans. Neural Netw.
**1995**, 6, 1212–1228. [Google Scholar] [CrossRef] [PubMed] - Mohammed Alsumaidaee, Y.A.; Yaw, C.T.; Koh, S.P.; Tiong, S.K.; Chen, C.P.; Yusaf, T.; Abdalla, A.N.; Ali, K.; Raj, A.A. Detection of Corona Faults in Switchgear by Using 1D-CNN, LSTM, and 1D-CNN-LSTM Methods. Sensors
**2023**, 23, 3108. [Google Scholar] [CrossRef] [PubMed] - Imandoust, S.B.; Bolandraftar, M. Application of k-nearest neighbor (knn) approach for predicting economic events: Theoretical background. Int. J. Eng. Res. Appl.
**2013**, 3, 605–610. [Google Scholar] - Amra, I.A.A.; Maghari, A.Y. Students performance prediction using KNN and Naïve Bayesian. In Proceedings of the 2017 8th International Conference on Information Technology (ICIT), Amman, Jordan, 17–18 May 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 909–913. [Google Scholar]
- Sha’Abani, M.; Fuad, N.; Jamal, N.; Ismail, M. kNN and SVM classification for EEG: A review. In InECCE2019, Proceedings of the 5th International Conference on Electrical, Control & Computer Engineering, Kuantan, Pahang, Malaysia, 29 July 2019; Springer: Berlin/Heidelberg, Germany, 2020; pp. 555–565. [Google Scholar]
- Jadhav, S.D.; Channe, H. Comparative study of K-NN, naive Bayes and decision tree classification techniques. Int. J. Sci. Res. (IJSR)
**2016**, 5, 1842–1845. [Google Scholar] - Gupta, B.; Rawat, A.; Jain, A.; Arora, A.; Dhami, N. Analysis of various decision tree algorithms for classification in data mining. Int. J. Comput. Appl.
**2017**, 163, 15–19. [Google Scholar] [CrossRef] - Kalcheva, N.; Todorova, M.; Marinova, G. Naive Bayes Classifier, Decision Tree and AdaBoost Ensemble Algorithm—Advantages and Disadvantages. In Proceedings of the 6th ERAZ Conference Proceedings (Part of ERAZ Conference Collection), Online, 21 May 2020; pp. 153–157. [Google Scholar]
- Bhati, B.S.; Chugh, G.; Al-Turjman, F.; Bhati, N.S. An improved ensemble based intrusion detection technique using XGBoost. Trans. Emerg. Telecommun. Technol.
**2021**, 32, e4076. [Google Scholar] [CrossRef] - Phan, Q.T.; Wu, Y.K.; Phan, Q.D. A comparative analysis of xgboost and temporal convolutional network models for wind power forecasting. In Proceedings of the 2020 International Symposium on Computer, Consumer and Control (IS3C), Taichung City, Taiwan, 13–16 November 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 416–419. [Google Scholar]
- Hengl, T.; Nussbaum, M.; Wright, M.N.; Heuvelink, G.B.; Gräler, B. Random forest as a generic framework for predictive modeling of spatial and spatio-temporal variables. PeerJ
**2018**, 6, e5518. [Google Scholar] [CrossRef] [PubMed] - Langsetmo, L.; Schousboe, J.T.; Taylor, B.C.; Cauley, J.A.; Fink, H.A.; Cawthon, P.M.; Kado, D.M.; Ensrud, K.E.; Osteoporotic Fractures in Men (MrOS) Research Group. Advantages and disadvantages of random forest models for prediction of hip fracture risk versus mortality risk in the oldest old. JBMR Plus
**2023**, 7, e10757. [Google Scholar] [CrossRef] - Kumar, P.S.; Kumari, A.; Mohapatra, S.; Naik, B.; Nayak, J.; Mishra, M. CatBoost ensemble approach for diabetes risk prediction at early stages. In Proceedings of the 2021 1st Odisha International Conference on Electrical Power Engineering, Communication and Computing Technology (ODICON), Bhubaneswar, India, 8–9 January 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 1–6. [Google Scholar]
- Huang, G.; Wu, L.; Ma, X.; Zhang, W.; Fan, J.; Yu, X.; Zeng, W.; Zhou, H. Evaluation of CatBoost method for prediction of reference evapotranspiration in humid regions. J. Hydrol.
**2019**, 574, 1029–1041. [Google Scholar] [CrossRef] - Walfish, S. A review of statistical outlier methods. Pharm. Technol.
**2006**, 30, 82. [Google Scholar] - Zurada, J. Could decision trees improve the classification accuracy and interpretability of loan granting decisions? In Proceedings of the 2010 43rd Hawaii International Conference on System Sciences, Honolulu, HI, USA, 5–8 January 2010; IEEE: Piscataway, NJ, USA, 2010; pp. 1–9. [Google Scholar]
- Mesquita, C.T. The Pathway to a High Impact Journal and Scopus Indexation—New Achievement of the International Journal of Cardiovascular Sciences. Int. J. Cardiovasc. Sci.
**2022**, 35, 145–147. [Google Scholar] [CrossRef] - Khosravi, H.; Shafie, M.R.; Hajiabadi, M.; Raihan, A.S.; Ahmed, I. Chatbots and ChatGPT: A bibliometric analysis and systematic review of publications in Web of Science and Scopus databases. arXiv
**2023**, arXiv:2304.05436. [Google Scholar] - Padhy, P.P.; Pawar, M.; Qazi, N.; Shah, A.N.; Pawar, V. A Bibliometric Analysis of the 100 Most Cited Articles on Nitrous Oxide Conscious Sedation. Indian J. Public Health Res. Dev.
**2022**, 13. [Google Scholar] - Henríquez, C.S.; Cano, A.J.R.; Galán, J.H.; de la Fuente Robles, Y.M. The past, present, and future of accessible tourism research: A bibliometric analysis using the scopus database. J. Access. Des. All JACCES
**2022**, 12, 26–60. [Google Scholar] - Kaihan, Y.; Chin, T.A. Bibliometric Analysis on Reverse Logistics in the Scopus Database: A Guide for Future Studies. Int. J. Acad. Res. Bus. Soc. Sci.
**2021**, 11, 1715–1727. [Google Scholar] [CrossRef] [PubMed] - Okoli, C.; Schabram, K. A guide to conducting a systematic literature review of information systems research. 2015. [Google Scholar]
- Paez, A. Gray literature: An important resource in systematic reviews. J. Evid.-Based Med.
**2017**, 10, 233–240. [Google Scholar] [CrossRef] [PubMed] - Corlett, R.T. Trouble with the gray literature. Biotropica
**2011**, 43, 3–5. [Google Scholar] [CrossRef] - Curtin, M.; Fossey, E. Appraising the trustworthiness of qualitative studies: Guidelines for occupational therapists. Aust. Occup. Ther. J.
**2007**, 54, 88–94. [Google Scholar] [CrossRef] - Kitto, S.C.; Chesters, J.; Grbich, C. Quality in qualitative research. Med. J. Aust.
**2008**, 188, 243–246. [Google Scholar] [CrossRef] - Neto, A.A.; Conte, T. A conceptual model to address threats to validity in controlled experiments. In Proceedings of the 17th International Conference on Evaluation and Assessment in Software Engineering, Porto de Galinhas, Brazil, 14–16 April 2013; pp. 82–85. [Google Scholar]
- Patino, C.M.; Ferreira, J.C. Internal and external validity: Can you apply research study results to your patients? J. Bras. Pneumol.
**2018**, 44, 183. [Google Scholar] [CrossRef] [PubMed] - Lancsar, E.; Swait, J. Reconceptualising the external validity of discrete choice experiments. Pharmacoeconomics
**2014**, 32, 951–965. [Google Scholar] [CrossRef] - Pervan, G.; Maimbo, M. Designing a case study protocol for application in IS research. In Proceedings of the Ninth Pacific Asia Conference on Information Systems, PACIS, Bangkok, Thailand, 7–10 July 2005; pp. 1281–1292. [Google Scholar]
- Hajjar, S. Statistical analysis: Internal-consistency reliability and construct validity. Int. J. Quant. Qual. Res. Methods
**2018**, 6, 27–38. [Google Scholar] - O’Leary-Kelly, S.W.; Vokurka, R.J. The empirical assessment of construct validity. J. Oper. Manag.
**1998**, 16, 387–405. [Google Scholar] [CrossRef] - Balka, K.; Raasch, C.; Herstatt, C. How open is open source?–software and beyond. Creat. Innov. Manag.
**2010**, 19, 248–256. [Google Scholar] [CrossRef] - Khanjani, A.; Sulaiman, R. The process of quality assurance under open source software development. In Proceedings of the 2011 IEEE Symposium on Computers & Informatics, Kuala Lumpur, Malaysia, 20–23 March 2011; IEEE: Piscataway, NJ, USA, 2011; pp. 548–552. [Google Scholar]
- Constantino, K.; Souza, M.; Zhou, S.; Figueiredo, E.; Kästner, C. Perceptions of open-source software developers on collaborations: An interview and survey study. J. Softw. Evol. Process
**2023**, 35, e2393. [Google Scholar] [CrossRef] - Akusok, A.; Leal, L.E.; Björk, K.M.; Lendasse, A. Scikit-elm: An extreme learning machine toolbox for dynamic and scalable learning. In Proceedings of the ELM2019, Yangzhou, China, 14–16 December 2021; Springer: Berlin/Heidelberg, Germany, 2021; pp. 69–78. [Google Scholar]
- Rodríguez-Pérez, G.; Robles, G.; González-Barahona, J.M. Reproducibility and credibility in empirical software engineering: A case study based on a systematic literature review of the use of the szz algorithm. Inf. Softw. Technol.
**2018**, 99, 164–176. [Google Scholar] [CrossRef] - Lemańska-Perek, A.; Krzyżanowska-Gołąb, D.; Kobylińska, K.; Biecek, P.; Skalec, T.; Tyszko, M.; Gozdzik, W.; Adamik, B. Explainable Artificial Intelligence Helps in Understanding the Effect of Fibronectin on Survival of Sepsis. Cells
**2022**, 11, 2433. [Google Scholar] [CrossRef] - Balzategui, J.; Eciolaza, L.; Maestro-Watson, D. Anomaly detection and automatic labeling for solar cell quality inspection based on generative adversarial network. Sensors
**2021**, 21, 4361. [Google Scholar] [CrossRef] - Caiazzo, B.; Di Nardo, M.; Murino, T.; Petrillo, A.; Piccirillo, G.; Santini, S. Towards Zero Defect Manufacturing paradigm: A review of the state-of-the-art methods and open challenges. Comput. Ind.
**2022**, 134, 103548. [Google Scholar] [CrossRef] - Fragapane, G.; Eleftheriadis, R.; Powell, D.; Antony, J. A global survey on the current state of practice in Zero Defect Manufacturing and its impact on production performance. Comput. Ind.
**2023**, 148, 103879. [Google Scholar] [CrossRef]

Type | Methods | |
---|---|---|

Statistical (14) | Autoregressive Integrated Moving Average (ARIMA), Clock Drift, Covariance Matrix, Cumulative Sum, Exponential Weighted Moving Average (EWMA), Holt–Winters, Mahalanobis Distance, Markov Chains, Matrix Profile, Principal Component Analysis (PCA), Receiver Operating Characteristic Analysis, Robust Covariance, Singular Value Decomposition (SVD), and Wavelets Functions | |

Clustering (10) | Density-Based Clustering Algorithm (DBSCAN), Graph-Based Approaches, Hierarchical Affinity Propagation, K-Harmonic Means (KHM), k–Means, k–Medoids, k–NN clusters, Ordering Points to Identify the Clustering Structure (OPTICS), Self-Organizing Maps (SOM), and Subspace Clustering | |

Classification (29) | SVM (4) | SVC, SVR, One-Class SVM, SVDD |

Neural Networks (7) | Autoencoders, Bidirectional Recurrent Neural Network (BRNN), Convolutional Neural Network (CNN), Deep Neural Networks, Generative Adversarial Network (GAN), Long Short-Term Memory (LSTM), and Recurrent Neural Network (RNN) | |

Ensemble Learning (8) | AdaBoost, CatBoost, DTBagg, dBoost, Gradient Boosting, LightGBM, Random Forests, and XGBoost | |

Others (10) | Decision Trees, GaussianHMM, Isolation Forest, Levenberg–Marquardt Algorithm, Linear Discriminant Analysis, Local Outlier Factor, Logistic Regression, Naive Bayes, Quadratic Discriminant Analysis, and Synthetic Minority Oversampling Technique | |

Information Theory (2) | Entropy, Kullback—Leibler Divergence | |

Hybrid/Others (16) | Artificial Immune Systems (AIS), Autoencoder and Incremental Clustering-Enabled Anomaly Detection, Cloud-Edge Indicator of Farming Anomalies (CEIFA), Deep Transfer Learning-Based Dual Temporal Domain Adaptation, Differential Evolution, Evolutionary Computation, Flocking Algorithm (FA), Fuzzy Combination of Outlier Detection techniques (FUCOD), Genetic Algorithms (GA), Hybrid Graph Transformer Network, Incremental Learning, Local Adaptive Multivariate Smoothing (LAMS), Mixed Deep-Learning-Based Methods Particle Swarm Optimization (PSO), Rough Sets, and Social Spider Optimization (SSO) |

No | Attribute | Data Type | Description |
---|---|---|---|

1 | time | numeric | The timestamp indicating the exact time when the sensor readings were recorded |

2 | company_id | numeric | Identifier representing the company |

3 | sensor_sub_id | numeric | An identifier for the individual acoustic (vibration) sensor unit within the wheel system |

4 | battery | numeric | Information about the battery status of the sensor unit |

5 | status | nominal | Status indicator indicating the validity of the data |

6 | values | numeric | A time series of vibration data collected by the acoustic sensors |

7 | temperature_alert | nominal | Indicator for temperature-related alerts |

8 | vibration_alert | nominal | Indicator for vibration-related alerts |

9 | sensor_count | numeric | The number of sensor units |

Attribute | Example |
---|---|

time | 2080-01-07 13:08:12 |

company_id | 55 |

sensor_sub_id | 14 |

battery | 3177 |

status | valid |

values | [−1, 0, 2, 252, 239, 247, 219, 247, 220, 247, 221, 247, 220, 247, 220, 247, 219, 247, 218, 247, 220, 247, 219, 247, 219, 247] |

temperature_alert | 0 |

vibration_alert | 1 |

sensor_count | 1 |

Index\Value | 1 | 2 | 3 | … | 26 |
---|---|---|---|---|---|

1 | 0.388767 | −0.945435 | 0.308469 | … | 0.382076 |

2 | 0.516504 | 1.197810 | 0.157382 | … | 0.382076 |

3 | 0.197160 | 0.126187 | 0.526706 | … | 0.382076 |

… | … | … | … | … | … |

184 | 0.644242 | −1.659850 | −2.260009 | … | −1.885730 |

Index | Vibration Alert |
---|---|

1 | 0 |

2 | 1 |

3 | 1 |

… | … |

184 | 0 |

Fold 1 | Fold 2 | Fold 3 | Fold 4 | Fold 5 | |
---|---|---|---|---|---|

Iteration 1 | Test | Train | Train | Train | Train |

Iteration 2 | Train | Test | Train | Train | Train |

Iteration 3 | Train | Train | Test | Train | Train |

Iteration 4 | Train | Train | Train | Test | Train |

Iteration 5 | Train | Train | Train | Train | Test |

Model | Accuracy (%) | Precision (%) | Recall (%) | F1 Score | FPR (%) |
---|---|---|---|---|---|

OCSVM | 48 | 34 | 46 | 39 | 51 |

LOF | 53 | 21 | 10 | 14 | 22 |

Isolation Forest | 55 | 29 | 15 | 20 | 21 |

GaussianHMM | 56 | 32 | 15 | 20 | 18 |

Naive Bayes | 59 | 46 | 66 | 55 | 45 |

LSTM | 63 | 0 | 0 | 0 | 0 |

ANN | 64 | 50 | 7 | 13 | 4 |

SVC | 67 | 56 | 47 | 51 | 22 |

MLP | 70 | 63 | 40 | 49 | 14 |

Logistic Regression | 70 | 61 | 53 | 57 | 20 |

SVR | 70 | 58 | 66 | 62 | 28 |

RNN | 71 | 68 | 28 | 40 | 8 |

1D-CNN | 80 | 72 | 74 | 73 | 16 |

KNN | 80 | 90 | 53 | 67 | 3 |

Decision Tree | 86 | 79 | 85 | 82 | 13 |

AdaBoost | 90 | 87 | 85 | 86 | 8 |

XGBoost | 90 | 88 | 88 | 88 | 7 |

Random Forest | 91 | 90 | 82 | 86 | 5 |

CatBoost | 96 | 95 | 93 | 94 | 3 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Bałdyga, M.; Barański, K.; Belter, J.; Kalinowski, M.; Weichbroth, P.
Anomaly Detection in Railway Sensor Data Environments: State-of-the-Art Methods and Empirical Performance Evaluation. *Sensors* **2024**, *24*, 2633.
https://doi.org/10.3390/s24082633

**AMA Style**

Bałdyga M, Barański K, Belter J, Kalinowski M, Weichbroth P.
Anomaly Detection in Railway Sensor Data Environments: State-of-the-Art Methods and Empirical Performance Evaluation. *Sensors*. 2024; 24(8):2633.
https://doi.org/10.3390/s24082633

**Chicago/Turabian Style**

Bałdyga, Michał, Kacper Barański, Jakub Belter, Mateusz Kalinowski, and Paweł Weichbroth.
2024. "Anomaly Detection in Railway Sensor Data Environments: State-of-the-Art Methods and Empirical Performance Evaluation" *Sensors* 24, no. 8: 2633.
https://doi.org/10.3390/s24082633