# Recent Advances in Anomaly Detection Methods Applied to Aviation

^{*}

^{†}

^{‡}

## Abstract

**:**

## 1. Introduction

#### 1.1. Anomaly Detection

- Point anomalies. A data point that differs significantly from the rest of the data points in the dataset considered. For instance, in a time series of French temperatures in summer, a temperature of $40{\phantom{\rule{3.33333pt}{0ex}}}^{\circ}$C can be considered as an anomaly even with the undergoing climate change.
- Contextual anomalies. When a data point is an anomaly only in a particular context. The context is defined by the contextual attributes, which usually refer to time (time series) or location. For instance, in a time series of summer temperatures by country, a temperature of $40{\phantom{\rule{3.33333pt}{0ex}}}^{\circ}$C is an anomaly in France, but it might be not in hotter countries like Libya where temperatures in summer are commonly around $40{\phantom{\rule{3.33333pt}{0ex}}}^{\circ}$C. Attributes (e.g., temperature) indexed by contextual attributes (e.g., country) are called behavioural attributes. Not only anomalies in spatial data but also in time series fall into this category, e.g., $40{\phantom{\rule{3.33333pt}{0ex}}}^{\circ}$C can be an anomaly in Libya from October to April, as at this time average temperatures range from $15{\phantom{\rule{3.33333pt}{0ex}}}^{\circ}$C to $30{\phantom{\rule{3.33333pt}{0ex}}}^{\circ}$C.
- Collective anomalies. When a group of data in a dataset is an anomaly as a whole, but the individual instances in that group (or subsets of them) might be not on their own. In time series, this would correspond for instance to a situation or condition persisting over an abnormal long time. Collective anomalies can only be detected in datasets where data is related someway, i.e., sequential, spatial or graph data.

#### 1.2. Previous Surveys on Anomaly Detection

#### 1.3. Motivation and Organisation of the Survey

## 2. Taxonomy of Classical Methods in Previous Surveys

#### 2.1. Distance-Based Methods

#### 2.1.1. Nearest Neighbour-Based Methods

#### 2.1.2. Clustering-Based Methods

- In the first category, the techniques assume that normal data instances belong to a cluster whereas anomalies do not: anomalies correspond instead to the so called clustering outliers or noise. Thus, any clustering algorithm that does not force all data instances to belong to a cluster can be used. The most popular ones are density-based clustering algorithms such as DBSCAN [23], HDBSCAN [24] or OPTICS [25].
- In the second category, the assumption is that normal instances are near their closest cluster centroid whereas anomalies lie far away from them. In this case, two steps are required for anomaly detection: run an algorithm to cluster the data, and then compute an anomaly score for each data instance based on the distance to its closest cluster centroid. An example of technique in this category often cited in aerospace papers is the Inductive Monitoring System (IMS) algorithm [26].
- The third category addresses the issue with the methods in the two previous categories when clusters of anomalies are formed. This is because the assumption is now that normal instances belong to large and dense clusters and anomalies to sparse or small clusters. A threshold is thus defined on the cluster size or density to determine the anomaly cases.

#### 2.2. Ensemble-Based Methods

#### 2.3. Statistical Methods

#### 2.3.1. Gaussian Mixture Models

#### 2.3.2. Independent Component Analysis

#### 2.3.3. Regression Model-Based

#### 2.4. Domain-Based Methods

#### 2.5. Reconstruction-Based Methods

#### 2.5.1. Subspace-Based Methods

#### 2.5.2. Neural Network Methods

## 3. Recent Advances in Anomaly Detection

#### 3.1. Recent Advances in Recurrent Neural Networks

#### 3.2. Recent Advances in Convolutional Neural Networks

#### 3.3. Recent Advances in Autoencoders

#### 3.4. Recent Advances in Generative Models

#### 3.5. Recent Advances in Temporal Logic-Based Learning

## 4. Applications

#### 4.1. Anomaly Detection for Air Traffic Operations

#### 4.1.1. Domain-Based Approaches

#### 4.1.2. Distance-Based Approaches

#### 4.1.3. Reconstruction-Based Approaches

#### 4.1.4. Statistical-Based Approaches

#### 4.1.5. Temporal-Logic Learning Based Approaches

#### 4.2. Anomaly Detection for Predictive Maintenance Operations in Aviation

- NASA DASHlink open database originally designed and collected by Balaban [118] and available at https://c3.nasa.gov/dashlink/projects/85/;
- a turbofan engine degradation simulation dataset based on thermo-dynamical simulation models, introduced in [119];
- other datasets, also shared on the Prognostic Data Repository of NASA refer to bearing systems or milling machines. These do not necessarily refer to aviation problems but are still worth mentioning as they are commonly used as reference.

## 5. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## Abbreviations

AE | Autoencoder |

ARIMA | Auto Regressive Integrated Moving Averages |

CNN | Convolutional Neural Network |

DAE | Deep Autoencoder |

DBN | Deep Belief Network |

DBSCAN | Density-Based Spatial Clustering of Applications with Noise |

ELM | Extreme Learning Machines |

GAN | Generative Adversarial Network |

GLOSH | Global-Local Outlier Score from Hierarchies |

GMM | Gaussian Mixture Model |

GRU | Gated Recurrent Unit |

ICA | Independent Component Analysis |

IF | Isolation Forest |

KDE | Kernel Density Estimation |

kNN | K-Nearest Neighbours |

IMF | Inductive Monitoring System |

LOF | Local Outlier Factor |

LoOP | Local Outlier Probability |

LSTM | Long Short-Term Memory |

MKAD | Multiple Kernel Anomaly Detection |

NN | Neural Network |

OC-SVM | One-Class Support Vector Machine |

OPTICS | Ordering Points To Identify the Clustering Structure |

PCA | Principal Component Analysis |

RBM | Riemann Boltzmann Machine |

RNN | Recurrent Neural Network |

STORN | Stochastic Recurrent Network |

SVM | Support Vector Machine |

VAE | Variational Autoencoder |

VAR | Vector Auto-Regressive |

ACARS | Aircraft Communication Addressing and Reporting System |

ADS-B | Automatic Dependent Surveillance–Broadcast |

ASIAS | Aviation Safety Information Analysis and Sharing |

ATC | Air Traffic Control |

ATM | Air Traffic Management |

CBM | Condition Based Maintenance |

FDM | Flight Data Monitoring |

FOQA | Flight Operations Quality Assurance |

NAS | National Airspace System |

PHM | Prognostics and Health Management |

RUL | Remained Useful Life |

## References

- Pimentel, M.A.; Clifton, D.A.; Clifton, L.; Tarassenko, L. A review of novelty detection. Signal Process.
**2014**, 99, 215–249. [Google Scholar] [CrossRef] - Chandola, V.; Banerjee, A.; Kumar, V. Anomaly detection: A survey. ACM Comput. Surv.
**2009**, 41, 1–58. [Google Scholar] [CrossRef] - Erhan, D.; Bengio, Y.; Courville, A.; Manzagol, P.A.; Vincent, P.; Bengio, S. Why Does Unsupervised Pre-training Help Deep Learning? J. Mach. Learn. Res.
**2010**, 11, 625–660. [Google Scholar] - Markou, M.; Singh, S. Novelty detection: A review—Part 1: Statistical approaches. Signal Process.
**2003**, 83, 2481–2497. [Google Scholar] [CrossRef] - Markou, M.; Singh, S. Novelty detection: A review—Part 2: Neural network based approaches. Signal Process.
**2003**, 83, 2499–2521. [Google Scholar] [CrossRef] - Zimek, A.; Schubert, E.; Kriegel, H.P. A survey on unsupervised outlier detection in high-dimensional numerical data. Stat. Anal. Data Min.
**2012**, 5, 363–387. [Google Scholar] [CrossRef] - Aggarwal, C.C. Outlier ensembles: Position paper. ACM SIGKDD Explor. Newslett.
**2013**, 14, 49. [Google Scholar] [CrossRef] - Xu, X.; Liu, H.; Yao, M. Recent Progress of Anomaly Detection. Complexity
**2019**, 2019, 1–11. [Google Scholar] [CrossRef] - Längkvist, M.; Karlsson, L.; Loutfi, A. A review of unsupervised feature learning and deep learning for time-series modeling. Pattern Recognit. Lett.
**2014**, 42, 11–24. [Google Scholar] [CrossRef][Green Version] - Akoglu, L.; Tong, H.; Koutra, D. Graph based anomaly detection and description: A survey. Data Min. Knowl. Discov.
**2015**, 29, 626–688. [Google Scholar] [CrossRef] - Chalapathy, R.; Chawla, S. Deep Learning for Anomaly Detection: A Survey. arXiv
**2019**, arXiv:1901.03407. [Google Scholar] - Gavrilovski, A.; Jimenez, H.; Mavris, D.N.; Rao, A.H.; Shin, S.; Hwang, I.; Marais, K. Challenges and opportunities in flight data mining: A review of the state of the art. In Proceedings of the AIAA [email protected] Aerospace, San Diego, CA, USA, 4–8 January 2016. [Google Scholar] [CrossRef]
- Breunig, M.M.; Kriegel, H.P.; Ng, R.T.; Sander, J. LOF: Identifying Density-based Local Outliers. SIGMOD Rec.
**2000**, 29, 93–104. [Google Scholar] [CrossRef] - Lazarevic, A.; Ertoz, L.; Kumar, V.; Ozgur, A.; Srivastava, J. A comparative study of anomaly detection schemes in network intrusion detection. In Proceedings of the 2003 SIAM International Conference on Data Mining, San Francisco, CA, USA, 1–3 May 2003; pp. 25–36. [Google Scholar]
- Kriegel, H.P.; Kröger, P.; Schubert, E.; Zimek, A. LoOP: Local outlier probabilities. In Proceedings of the 18th ACM Conference on Information and Knowledge Management, Hong Kong, China, 2–6 November 2009; pp. 1649–1652. [Google Scholar]
- Oehling, J.; Barry, D.J. Using machine learning methods in airline flight data monitoring to generate new operational safety knowledge from existing data. Saf. Sci.
**2019**, 114, 89–104. [Google Scholar] [CrossRef] - Bay, S.D.; Schwabacher, M. Mining distance-based outliers in near linear time with randomization and a simple pruning rule. In Proceedings of the Ninth ACM SIGKDD International Conference On Knowledge Discovery and Data Mining, Washington, DC, USA, 24–27 August 2003; pp. 29–38. [Google Scholar]
- Budalakoti, S.; Srivastava, A.N.; Otey, M.E. Anomaly detection and diagnosis algorithms for discrete symbol sequences with applications to airline safety. IEEE Trans. Syst. Man Cybernet. Part C (Appl. Rev.)
**2008**, 39, 101–113. [Google Scholar] [CrossRef] - Li, L.; Gariel, M.; Hansman, R.J.; Palacios, R. Anomaly detection in onboard-recorded flight data using cluster analysis. In Proceedings of the 2011 IEEE/AIAA 30th Digital Avionics Systems Conference, Seattle, WA, USA, 16–20 October 2011; p. 4A4. [Google Scholar]
- Li, L.; Das, S.; John Hansman, R.; Palacios, R.; Srivastava, A.N. Analysis of flight data using clustering techniques for detecting abnormal operations. J. Aerosp. Inf. Syst.
**2015**, 12, 587–598. [Google Scholar] [CrossRef] - Li, L.; Hansman, R.J.; Palacios, R.; Welsch, R. Anomaly detection via a Gaussian Mixture Model for flight operation and safety monitoring. Transp. Res. Part C Emerg. Technol.
**2016**, 64, 45–57. [Google Scholar] [CrossRef] - Churchill, A.M.; Bloem, M. Clustering Aircraft Trajectories on the Airport Surface. In Proceedings of the 13th USA/Europe Air Traffic Management Research and Development Seminar, Chicago, IL, USA, 10–13 June 2019. [Google Scholar]
- Ester, M.; Kriegel, H.P.; Sander, J.; Xu, X. A Density-based Algorithm for Discovering Clusters in Large Spatial Databases with Noise. In Proceedings of the Second KDD’96 International Conference on Knowledge Discovery and Data Mining, Portland, OR, USA, 2–4 August 1996; pp. 226–231. [Google Scholar]
- Campello, R.J.; Moulavi, D.; Sander, J. Density-based clustering based on hierarchical density estimates. In Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining, Gold Coast, Australia, 14–17 April 2013; Springer: Berlin/Heidelberg, Germany, 2013; pp. 160–172. [Google Scholar]
- Ankerst, M.; Breunig, M.M.; Kriegel, H.P.; Sander, J. OPTICS: Ordering Points to Identify the Clustering Structure. In Proceedings of the 1999 ACM SIGMOD International Conference on Management of Data, Philadelphia, PA, USA, 31 May–3 June 1999; ACM: New York, NY, USA, 1999; pp. 49–60. [Google Scholar] [CrossRef]
- Iverson, D.; Martin, R.; Schwabacher, M.; Spirkovska, L.; Taylor, W.; Mackey, R.; Castle, J. General Purpose Data-Driven System Monitoring for Space Operations. J. Aerosp. Comput. Inf. Commun.
**2012**, 9. [Google Scholar] [CrossRef] - Campello, R.J.; Moulavi, D.; Zimek, A.; Sander, J. Hierarchical density estimates for data clustering, visualization, and outlier detection. ACM Trans. Knowl. Discov. Data (TKDD)
**2015**, 10, 5. [Google Scholar] [CrossRef] - Hartigan, J.A.; Wong, M.A. Algorithm AS 136: A k-means clustering algorithm. J. R. Stat. Soc. Ser. C (Appl. Stat.)
**1979**, 28, 100–108. [Google Scholar] [CrossRef] - Jolliffe, I.T. Principal Component Analysis; Springer Series in Statistics; Springer: New York, NY, USA, 1986. [Google Scholar] [CrossRef]
- Maaten, L.V.D.; Hinton, G. Visualizing data using t-SNE. J. Mach. Learn. Res.
**2008**, 9, 2579–2605. [Google Scholar] - Liu, F.T.; Ting, K.M.; Zhou, Z.H. Isolation forest. In Proceedings of the 2008 Eighth IEEE International Conference on Data Mining, Pisa, Italy, 15–19 December 2008; pp. 413–422. [Google Scholar]
- Liu, F.T.; Ting, K.M.; Zhou, Z.H. Isolation-based anomaly detection. ACM Trans. Knowl. Discov. Data (TKDD)
**2012**, 6, 3. [Google Scholar] [CrossRef] - Zenati, H.; Romain, M.; Foo, C.S.; Lecouat, B.; Chandrasekhar, V. Adversarially Learned Anomaly Detection. In Proceedings of the 2018 IEEE International Conference on Data Mining (ICDM), Singapore, 17–20 November 2018; pp. 727–736. [Google Scholar]
- Ding, Z.; Fei, M. An Anomaly Detection Approach Based on Isolation Forest Algorithm for Streaming Data using Sliding Window. IFAC Proc. Vol.
**2013**, 46, 12–17. [Google Scholar] [CrossRef] - Hariri, S.; Kind, M.C.; Brunner, R.J. Extended Isolation Forest. arXiv
**2018**, arXiv:1811.02141. [Google Scholar] - Pontoppidan, N.H.; Larsen, J. Unsupervised condition change detection in large diesel engines. In Proceedings of the 2003 IEEE XIII Workshop on Neural Networks for Signal Processing (IEEE Cat. No. 03TH8718), Toulouse, France, 17–19 September 2003; pp. 565–574. [Google Scholar]
- Jiang, X.; Wen, X.; Wu, M.; Song, M.; Tu, C. A complex network analysis approach for identifying air traffic congestion based on independent component analysis. Phys. A Stat. Mech. Appl.
**2019**, 523, 364–381. [Google Scholar] [CrossRef] - Melnyk, I.; Matthews, B.; Valizadegan, H.; Banerjee, A.; Oza, N. Vector Autoregressive Model-Based Anomaly Detection in Aviation Systems. J. Aerosp. Inf. Syst.
**2016**, 13, 161–173. [Google Scholar] [CrossRef][Green Version] - Melnyk, I.; Banerjee, A.; Matthews, B.; Oza, N. Semi-Markov Switching Vector Autoregressive Model-Based Anomaly Detection in Aviation Systems. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining-KDD ’16, San Francisco, CA, USA, 13–17 August 2016; ACM Press: San Francisco, CA, USA, 2016; pp. 1065–1074. [Google Scholar] [CrossRef][Green Version]
- Bianco, A.M.; Garcia Ben, M.; Martinez, E.; Yohai, V.J. Outlier detection in regression models with arima errors using robust estimates. J. Forecast.
**2001**, 20, 565–579. [Google Scholar] [CrossRef] - Chen, D.; Shao, X.; Hu, B.; Su, Q. Simultaneous wavelength selection and outlier detection in multivariate regression of near-infrared spectra. Anal. Sci.
**2005**, 21, 161–166. [Google Scholar] [CrossRef] - Vapnik, V. The Nature of Statistical Learning Theory; Springer Science & Business Media: New York, NY, USA, 2013. [Google Scholar]
- Schölkopf, B.; Williamson, R.C.; Smola, A.J.; Shawe-Taylor, J.; Platt, J.C. Support vector method for novelty detection. In Proceedings of the 12th International Conference on Neural Information Processing Systems, Denver, CO, USA, 29 November–4 December 1999; pp. 582–588. [Google Scholar]
- Das, S.; Matthews, B.L.; Srivastava, A.N.; Oza, N.C. Multiple kernel learning for heterogeneous anomaly detection: Algorithm and aviation safety case study. In Proceedings of the 16th ACM SIGKDD International Conference On Knowledge Discovery and Data Mining, Washington, DC, USA, 25–28 July 2010; pp. 47–56. [Google Scholar]
- Dutta, H.; Giannella, C.; Borne, K.; Kargupta, H. Distributed top-k outlier detection from astronomy catalogs using the demac system. In Proceedings of the 2007 SIAM International Conference on Data Mining, Minneapolis, MN, USA, 26–28 April 2007; pp. 473–478. [Google Scholar]
- Günter, S.; Schraudolph, N.N.; Vishwanathan, S.V.N. Fast Iterative Kernel Principal Component Analysis. J. Mach. Learn. Res.
**2007**, 8, 1893–1918. [Google Scholar] - Candès, E.J.; Li, X.; Ma, Y.; Wright, J. Robust principal component analysis? J. ACM (JACM)
**2011**, 58, 11. [Google Scholar] [CrossRef] - Ramsay, J.; Ramsay, J.; Silverman, B.; Silverman, H.; Media, S.S. Functional Data Analysis; Springer Series in Statistics; Springer: New York, NY, USA, 2005. [Google Scholar]
- Deville, J.C. Méthodes Statistiques Et Numériques De L’analyse Harmonique; Annales de l’INSEE; JSTOR: New York, NY, USA, 1974; pp. 3–101. [Google Scholar]
- Dauxois, J. Les Analyses Factorielles en Calcul Des Probabiblités Et En Statistique: Essai D’étude Synthétique. Ph.D. Thesis, Université de Toulouse, Toulouse, France, 1976. [Google Scholar]
- Dauxois, J.; Pousse, A.; Romain, Y. Asymptotic theory for the principal component analysis of a vector random function: Some applications to statistical inference. J. Multivariate Anal.
**1982**, 12, 136–154. [Google Scholar] [CrossRef][Green Version] - Jarry, G.; Delahaye, D.; Nicol, F.; Féron, E. Aircraft Atypical Approach Detection using Functional Principal Component Analysis. In Proceedings of the SESAR Innovations Days 2018, Salzburg, Austria, 3–7 December 2018. [Google Scholar]
- Huang, G.B.; Zhu, Q.Y.; Siew, C.K. Extreme learning machine: A new learning scheme of feedforward neural networks. In Proceedings of the International Joint Conference on Neural Networks (IJCNN2004), Budapest, Hungary, 25–29 July 2004; Volume 2, pp. 985–990. [Google Scholar]
- Huang, G.B.; Zhu, Q.Y.; Siew, C.K. Extreme learning machine: Theory and applications. Neurocomputing
**2006**, 70, 489–501. [Google Scholar] [CrossRef] - Huang, G.; Huang, G.B.; Song, S.; You, K. Trends in extreme learning machines: A review. Neural Netw.
**2015**, 61, 32–48. [Google Scholar] [CrossRef] [PubMed] - Janakiraman, V.M.; Nielsen, D. Anomaly detection in aviation data using extreme learning machines. In Proceedings of the 2016 International Joint Conference on Neural Networks (IJCNN), Vancouver, BC, Canada, 24–29 July 2016; pp. 1993–2000. [Google Scholar] [CrossRef]
- Malhotra, P.; Vig, L.; Shroff, G.; Agarwal, P. Long Short Term Memory Networks for Anomaly Detection in Time Series. In Proceedings of the ESANN 2015, Bruges, Belgium, 22–24 April 2015. [Google Scholar]
- Nanduri, A.; Sherry, L. Anomaly detection in aircraft data using Recurrent Neural Networks (RNN). In Proceedings of the 2016 Integrated Communications Navigation and Surveillance (ICNS), Herndon, VA, USA, 19–21 April 2016; pp. 5C2-1–5C2-8. [Google Scholar] [CrossRef]
- Ergen, T.; Mirza, A.H.; Kozat, S.S. Unsupervised and Semi-supervised Anomaly Detection with LSTM Neural Networks. arXiv
**2017**, arXiv:1710.09207. [Google Scholar] - Vinayakumar, R.; Soman, K.; Poornachandran, P. Applying convolutional neural network for network intrusion detection. In Proceedings of the 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI), Udupi, India, 13–16 September 2017; pp. 1222–1228. [Google Scholar]
- Li, Z.; Qin, Z.; Huang, K.; Yang, X.; Ye, S. Intrusion detection using convolutional neural networks for representation learning. In Proceedings of the International Conference on Neural Information Processing, Guangzhou, China, 14–18 November 2017; Springer: Cham, Switzerland, 2017; pp. 858–866. [Google Scholar]
- Kwon, D.; Natarajan, K.; Suh, S.C.; Kim, H.; Kim, J. An empirical study on network anomaly detection using convolutional neural networks. In Proceedings of the 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), Vienna, Austria, 2–6 July 2018; pp. 1595–1598. [Google Scholar]
- Malhotra, P.; Ramakrishnan, A.; Anand, G.; Vig, L.; Agarwal, P.; Shroff, G. LSTM-based encoder-decoder for multi-sensor anomaly detection. arXiv
**2016**, arXiv:1607.00148. [Google Scholar] - Zhang, C.; Song, D.; Chen, Y.; Feng, X.; Lumezanu, C.; Cheng, W.; Ni, J.; Zong, B.; Chen, H.; Chawla, N.V. A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data. arXiv
**2018**, arXiv:1811.08055. [Google Scholar] [CrossRef] - Reddy, K.K.; Sarkar, S.; Venugopalan, V.; Giering, M. Anomaly detection and fault disambiguation in large flight data: A multi-modal deep auto-encoder approach. In Proceedings of the Annual Conference of the Prognostics and Health Management Society, Denver, CO, USA, 3–6 October 2016. [Google Scholar]
- Luo, W.; Liu, W.; Gao, S. Remembering history with convolutional lstm for anomaly detection. In Proceedings of the 2017 IEEE International Conference on Multimedia and Expo (ICME), Hong Kong, China, 10–14 July 2017; pp. 439–444. [Google Scholar]
- Zenati, H.; Foo, C.S.; Lecouat, B.; Manek, G.; Chandrasekhar, V.R. Efficient gan-based anomaly detection. arXiv
**2018**, arXiv:1802.06222. [Google Scholar] - Li, D.; Chen, D.; Goh, J.; Ng, S.k. Anomaly Detection with Generative Adversarial Networks for Multivariate Time Series. arXiv
**2018**, arXiv:1809.04758. [Google Scholar] - Soelch, M.; Bayer, J.; Ludersdorfer, M.; van der Smagt, P. Variational Inference for On-line Anomaly Detection in High-Dimensional Time Series. arXiv
**2016**, arXiv:1602.07109. [Google Scholar] - Guo, Y.; Liao, W.; Wang, Q.; Yu, L.; Ji, T.; Li, P. Multidimensional time series anomaly detection: A gru-based gaussian mixture variational autoencoder approach. In Proceedings of the Asian Conference on Machine Learning, Philadelphia, PA, USA, 24–27 September 2018; pp. 97–112. [Google Scholar]
- Kong, Z.; Jones, A.; Medina Ayala, A.; Aydin Gol, E.; Belta, C. Temporal Logic Inference for Classification and Prediction from Data. In Proceedings of the 17th HSCC ’14 International Conference on Hybrid Systems: Computation and Control, Berlin, Germany, 15–17 April 2014; ACM: New York, NY, USA, 2014; pp. 273–282. [Google Scholar] [CrossRef]
- Jones, A.; Kong, Z.; Belta, C. Anomaly detection in cyber-physical systems: A formal methods approach. In Proceedings of the 53rd IEEE Conference on Decision and Control, Los Angeles, CA, USA, 15–17 December 2014; pp. 848–853. [Google Scholar]
- Kong, Z.; Jones, A.; Belta, C. Temporal logics for learning and detection of anomalous behavior. IEEE Trans. Autom. Control
**2016**, 62, 1210–1222. [Google Scholar] [CrossRef] - Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural Comput.
**1997**, 9, 1735–1780. [Google Scholar] [CrossRef] - Cho, K.; Van Merriënboer, B.; Gulcehre, C.; Bahdanau, D.; Bougares, F.; Schwenk, H.; Bengio, Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv
**2014**, arXiv:1406.1078. [Google Scholar] - Goel, H.; Melnyk, I.; Oza, N.; Matthews, B.; Banerjee, A. Multivariate Aviation Time Series Modeling: VARs vs. LSTMs. In Proceedings of the SIAM International Conference on Data Mining (SDM), Houston, TX, USA, 27–29 April 2017. [Google Scholar]
- Tax, D.M.; Duin, R.P. Support vector data description. Mach. Learn.
**2004**, 54, 45–66. [Google Scholar] [CrossRef] - Habler, E.; Shabtai, A. Using LSTM encoder-decoder algorithm for detecting anomalous ADS-B messages. Comput. Security
**2018**, 78, 155–173. [Google Scholar] [CrossRef][Green Version] - Bai, S.; Kolter, J.Z.; Koltun, V. An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling. arXiv
**2018**, arXiv:1803.01271. [Google Scholar] - Gorokhov, O.; Petrovskiy, M.; Mashechkin, I. Convolutional neural networks for unsupervised anomaly detection in text data. In Proceedings of the International Conference on Intelligent Data Engineering and Automated Learning, Guilin, China, 30 October–1 November 2017; Springer: Cham, Switzerland, 2017; pp. 500–507. [Google Scholar]
- Xingjian, S.; Chen, Z.; Wang, H.; Yeung, D.Y.; Wong, W.K.; Woo, W.c. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In Proceedings of the Neural Information Processing Systems Conference 2015, Montreal, QC, Canada, 7–12 December 2015; pp. 802–810. [Google Scholar]
- Hallac, D.; Vare, S.; Boyd, S.; Leskovec, J. Toeplitz inverse covariance-based clustering of multivariate time series data. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada, 13–17 August 2017; pp. 215–223. [Google Scholar]
- Song, D.; Xia, N.; Cheng, W.; Chen, H.; Tao, D. Deep r-th root of rank supervised joint binary embedding for multivariate time series retrieval. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK, 19–23 August 2018; pp. 2229–2238. [Google Scholar]
- Malaiya, R.K.; Kwon, D.; Kim, J.; Suh, S.C.; Kim, H.; Kim, I. An empirical evaluation of deep learning for network anomaly detection. In Proceedings of the 2018 International Conference on Computing, Networking and Communications (ICNC), Maui, HI, USA, 5–8 March 2018; pp. 893–898. [Google Scholar]
- Fu, X.; Luo, H.; Zhong, S.; Lin, L. Aircraft engine fault detection based on grouped convolutional denoising autoencoders. Chin. J. Aeronaut.
**2019**, 32, 296–307. [Google Scholar] [CrossRef] - Akerman, S.; Habler, E.; Shabtai, A. VizADS-B: Analyzing Sequences of ADS-B Images Using Explainable Convolutional LSTM Encoder-Decoder to Detect Cyber Attacks. arXiv
**2019**, arXiv:1906.07921. [Google Scholar] - Zhou, C.; Paffenroth, R.C. Anomaly Detection with Robust Deep Autoencoders. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining-KDD ’17, Halifax, NS, Canada, 13–17 August 2017; pp. 665–674. [Google Scholar] [CrossRef]
- Erhan, D.; Courville, A.; Bengio, Y. Understanding Representations Learned in Deep Architectures; Department d’ Informatique et Recherche Operationnelle, University of Montreal: Montreal, QC, Canada, 2010; Volume 1355. [Google Scholar]
- Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative adversarial nets. In Proceedings of the Neural Information Processing Systems Conference 2014, Montreal, QC, Canada, 8–13 December 2014; pp. 2672–2680. [Google Scholar]
- Schlegl, T.; Seeböck, P.; Waldstein, S.M.; Schmidt-Erfurth, U.; Langs, G. Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery. In Information Processing in Medical Imaging; Niethammer, M., Styner, M., Aylward, S., Zhu, H., Oguz, I., Yap, P.T., Shen, D., Eds.; Springer International Publishing: Cham, Switzerland, 2017; Volume 10265, pp. 146–157. [Google Scholar] [CrossRef]
- Schlegl, T.; Seeböck, P.; Waldstein, S.M.; Langs, G.; Schmidt-Erfurth, U. f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks. Med. Image Anal.
**2019**, 54, 30–44. [Google Scholar] [CrossRef] - Kingma, D.P.; Welling, M. Auto-Encoding Variational Bayes. arXiv
**2013**, arXiv:1312.6114. [Google Scholar] - An, J.; Cho, S. Variational autoencoder based anomaly detection using reconstruction probability. Spec. Lecture IE
**2015**, 2, 1–18. [Google Scholar] - Blei, D.M.; Kucukelbir, A.; McAuliffe, J.D. Variational Inference: A Review for Statisticians. J. Am. Stat. Assoc.
**2017**, 112, 859–877. [Google Scholar] [CrossRef][Green Version] - Bayer, J.; Osendorfer, C. Learning Stochastic Recurrent Networks. arXiv
**2014**, arXiv:1411.7610. [Google Scholar] - Zhai, S.; Cheng, Y.; Lu, W.; Zhang, Z. Deep Structured Energy Based Models for Anomaly Detection. arXiv
**2016**, arXiv:1605.07717. [Google Scholar] - Dilokthanakul, N.; Mediano, P.A.M.; Garnelo, M.; Lee, M.C.H.; Salimbeni, H.; Arulkumaran, K.; Shanahan, M. Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders. arXiv
**2017**, arXiv:1611.02648. [Google Scholar] - Zong, B.; Song, Q.; Min, M.R.; Cheng, W.; Lumezanu, C.; Cho, D.; Chen, H. Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection. In Proceedings of the 6th International Conference on Learning Representations, Vancouver, BC, Canada, 30 April–10 May 2018; p. 19. [Google Scholar]
- Ghosh, P.; Sajjadi, M.S.M.; Vergari, A.; Black, M.; Schölkopf, B. From Variational to Deterministic Autoencoders. arXiv
**2019**, arXiv:1903.12436. [Google Scholar] - Ustun, B.; Traca, S.; Rudin, C. Supersparse linear integer models for interpretable classification. arXiv
**2013**, arXiv:1306.6677. [Google Scholar] - Asarin, E.; Donzé, A.; Maler, O.; Nickovic, D. Parametric Identification of Temporal Properties. In Proceedings of the Second RV’11 International Conference on Runtime Verification, San Francisco, CA, USA, 27–30 September 2011; Springer: Berlin/Heidelberg, Germany, 2012; pp. 147–160. [Google Scholar] [CrossRef][Green Version]
- Maler, O.; Nickovic, D. Monitoring temporal properties of continuous signals. In Formal Techniques, Modelling and Analysis of Timed and Fault-Tolerant Systems; Springer: Grenoble, France, 2004; pp. 152–166. [Google Scholar]
- Deshmukh, R.; Hwang, I. Anomaly Detection Using Temporal Logic Based Learning for Terminal Airspace Operations. In Proceedings of the AIAA Scitech 2019 Forum, San Diego, CA, USA, 7–11 January 2019; p. 0682. [Google Scholar]
- Das, S.; Matthews, B.L.; Lawrence, R. Fleet level anomaly detection of aviation safety data. In Proceedings of the 2011 IEEE Conference on Prognostics and Health Management, Montreal, QC, Canada, 20–23 June 2011; IEEE: Denver, CO, USA, 2011; pp. 1–10. [Google Scholar] [CrossRef]
- Puranik, T.G.; Mavris, D.N. Anomaly Detection in General-Aviation Operations Using Energy Metrics and Flight-Data Records. J. Aerosp. Inf. Syst.
**2017**, 22–36. [Google Scholar] [CrossRef] - Guimin, J.; Cheng, F.; Jinfeng, Y.; Dan, L. Intelligent checking model of Chinese radiotelephony read-backs in civil aviation air traffic control. Chin. J. Aeronaut.
**2018**, 31, 2280–2289. [Google Scholar] - Zhang, X.; Chen, J.; Gan, Q. Anomaly Detection for Aviation Safety Based on an Improved KPCA Algorithm. J. Electr. Comput. Eng.
**2017**, 2017, 4890921. [Google Scholar] [CrossRef] - Olive, X.; Grignard, J.; Dubot, T.; Saint-Lot, J. Detecting Controllers’ Actions in Past Mode S Data by Autoencoder-Based Anomaly Detection. In Proceedings of the 8th SESAR Innovation Days, Salzburg, Austria, 3–7 December 2018. [Google Scholar]
- Olive, X.; Basora, L. Identifying Anomalies in past en-route Trajectories with Clustering and Anomaly Detection Methods. In Proceedings of the 13th USA/Europe Air Traffic Management Research and Development Seminar, Vienna, Austria, 17–21 June 2019. [Google Scholar]
- Dubot, T. Predicting sector configuration transitions with autoencoder-based anomaly detection. In Proceedings of the International Conference for Research in Air Transportation, Barcelona, Spain, 26–29 June 2018. [Google Scholar]
- Deshmukh, R.; Sun, D.; Hwang, I. Data-Driven Precursor Detection Algorithm for Terminal Airspace Operations. In Proceedings of the 13th USA/Europe Air Traffic Management Research and Development Seminar, Vienna, Austria, 17–21 June 2019. [Google Scholar]
- Puranik, T.; Jimenez, H.; Mavris, D. Energy-based metrics for safety analysis of general aviation operations. J. Aircraft
**2017**, 54, 2285–2297. [Google Scholar] [CrossRef] - Jia, G.; Lu, Y.; Lu, W.; Shi, Y.; Yang, J. Verification method for Chinese aviation radiotelephony readbacks based on LSTM-RNN. Electron. Lett.
**2017**, 53, 401–403. [Google Scholar] [CrossRef] - Basora, L.; Courchelle, V.; Bedouet, J.; Dubot, T. Occupancy Peak Estimation from Sector Geometry and Traffic Flow Data. In Proceedings of the 8th SESAR Innovation Days, Salzburg, Austria, 3–7 December 2018. [Google Scholar]
- Ltkepohl, H. New Introduction to Multiple Time Series Analysis; Springer: New York, NY, USA, 2007. [Google Scholar]
- Janssen, J.; Limnios, N. Semi-Markov Models and Applications; Springer: New York, NY, USA, 2013. [Google Scholar]
- Nanduri, A.; Sherry, L. Generating Flight Operations Quality Assurance (FOQA) Data from the X-Plane Simulation. In Proceedings of the 2016 IEEE Integrated Communications Navigation and Surveillance (ICNS), Herndon, VA, USA, 19–21 April 2016; p. 5C1. [Google Scholar]
- Balaban, E.; Saxena, A.; Bansal, P.; Goebel, K.F.; Curran, S. Modeling, Detection, and Disambiguation of Sensor Faults for Aerospace Applications. IEEE Sens. J.
**2009**, 9, 1907–1917. [Google Scholar] [CrossRef] - Saxena, A.; Goebel, K.; Simon, D.; Eklund, N. Damage propagation modeling for aircraft engine run-to-failure simulation. In Proceedings of the 2008 International Conference on Prognostics and Health Management, Denver, CO, USA, 6–9 October 2008; pp. 1–9. [Google Scholar] [CrossRef]
- Elattar, H.M.; Elminir, H.K.; Riad, A.M. Prognostics: A literature review. Complex Intell. Syst.
**2016**, 2, 125–154. [Google Scholar] [CrossRef] - Lei, Y.; Li, N.; Guo, L.; Li, N.; Yan, T.; Lin, J. Machinery health prognostics: A systematic review from data acquisition to RUL prediction. Mech. Syst. Signal Process.
**2018**, 104, 799–834. [Google Scholar] [CrossRef] - Atamuradov, V.; Medjaher, K.; Dersin, P.; Lamoureux, B.; Zerhouni, N. Prognostics and health management for maintenance practitioners-Review, implementation and tools evaluation. Int. J. Prognostics Health Manag.
**2017**, 8, 1–31. [Google Scholar] - Rabatel, J.; Bringay, S.; Poncelet, P. Anomaly detection in monitoring sensor data for preventive maintenance. Expert Syst. Appl.
**2011**, 38, 7003–7015. [Google Scholar] [CrossRef] - Nicchiotti, G.; Rüegg, J. Data-Driven Prediction of Unscheduled Maintenance Replacements in a Fleet of Commercial Aircrafts. In Proceedings of the European Conference of the PHM Society, Utrecht, The Netherlands, 3–6 July 2018; p. 10. [Google Scholar]
- Gugulothu, N.; Tv, V.; Malhotra, P.; Vig, L.; Agarwal, P.; Shroff, G. Predicting Remaining Useful Life using Time Series Embeddings based on Recurrent Neural Networks. arXiv
**2018**, arXiv:1709.01073. [Google Scholar] - Zhao, R.; Yan, R.; Chen, Z.; Mao, K.; Wang, P.; Gao, R.X. Deep learning and its applications to machine health monitoring. Mech. Syst. Signal Process.
**2019**, 115, 213–237. [Google Scholar] [CrossRef] - Gunning, D. Explainable artificial intelligence (xai) Program. In Proceedings of the 24th International Conference on Intelligent User Interfaces, Marina del Ray, CA, USA, 17–20 March 2019. [Google Scholar]

Recurrent Neural Networks | Section 3.1 | Stacked LSTM: [57] (2015) |

LSTM and GRU: [58] (2016) | ||

Hybrid LSTM with OC-SVM or SVDD: [59] (2017) | ||

Convolutional Neural Networks | Section 3.2 | Intrusion detection: [60] (2017), [61] (2017) |

Comparative study with other NN: [62] (2018) | ||

Advanced Autoencoders | Section 3.3 | LSTM-ED: [63] (2016) |

MSCRED: [64] (2018) | ||

Multi-modal DAE: [65] (2016) | ||

ConvLSTM-AE: [66] (2017) | ||

Generative Models | Section 3.4 | GAN: [33,67,68] (2018) |

Variational Inference: [69] (2016), [70] (2018) | ||

Temporal-logic Learning Models | Section 3.5 | Supervised model: [71] (2014) |

Unsupervised model: [72] (2014) | ||

Online model: [73] (2016) |

Section 4.1.1 Domain-based | Abnormal approaches with MKAD: [104] (2011) |

GA approach and landing anomalies with OC-SVM: [105] (2017) | |

Section 4.1.2 Distance-based | Anomalous pilot switching with SequenceMiner: [18] (2008) |

Anomalous take-off and approach operations: [19] (2011), [20] (2015) | |

Anomalous safety events with LoOP: [16] (2019) | |

Anomalous taxi paths with hierarchical clustering: [22] (2019) | |

Anomalous radiotelephony readbacks with kNN: [106] (2018) | |

Section 4.1.3 Reconstruction-based | Atypical aviation safety data with KPCA: [107] (2017) |

Atypical approaches and landings with FPCA: [52] (2018) | |

Anomalous trajectories in TMA and en-route: [108] (2018), [109] (2019) | |

Anomalous transitions between sector configurations: [110] (2018) | |

Anomalous ADS-B messages with ConvLSTM-AE: [86] (2019) | |

Section 4.1.4 Statistical-based | Anomalous flights with VARX: [38] (2016), |

Anomalous flight switches with VAR: [39] (2016) | |

Abnormal flight data with GMM: [21] (2016) | |

Anomalous air traffic congestion with ICA: [37] (2019) | |

Section 4.1.5 Temporal-logic based | Anomalous trajectories in terminal airspace with TempAD: [103,111] (2019) |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Basora, L.; Olive, X.; Dubot, T. Recent Advances in Anomaly Detection Methods Applied to Aviation. *Aerospace* **2019**, *6*, 117.
https://doi.org/10.3390/aerospace6110117

**AMA Style**

Basora L, Olive X, Dubot T. Recent Advances in Anomaly Detection Methods Applied to Aviation. *Aerospace*. 2019; 6(11):117.
https://doi.org/10.3390/aerospace6110117

**Chicago/Turabian Style**

Basora, Luis, Xavier Olive, and Thomas Dubot. 2019. "Recent Advances in Anomaly Detection Methods Applied to Aviation" *Aerospace* 6, no. 11: 117.
https://doi.org/10.3390/aerospace6110117