# Functional Kernel Density Estimation: Point and Fourier Approaches to Time Series Anomaly Detection

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

**point approach**to Functional KDE Anomaly Detection, because each curve in $\mathcal{H}$ is treated as a point. This approach then formally generates a proxy for the “probability density” over $\mathcal{H}$. Anomalous series are associated with smaller values of this density. This is distinct from considering a single time series as collection of points sampled from a distribution and using KDE upon points in the time series as has been done before [20]. This is a very simple, and seemingly effective method, with $\xi $ chosen as a hyper-parameter. We also present a

**Fourier approach**, which approximates a probability density over $\mathcal{H}$ through estimating empirical distributions for each Fourier mode with KDE. This allows us to estimate the likelihood of a given curve. Curves with lower likelihoods are more anomalous. Both methods naturally handle missing data, without interpolating. In real flight operations, sometimes it is not possible to capture and record complete information because incident data is documented from voluntary reporting, which may result in incomplete datasets. Therefore, model robustness to the impact of missing data is crucial to derive the correct understanding, which may save human lives and prevent damaged aircrafts.

`Python`and the PCA and functional boxplot methods available in

`R`); following this, we apply our techniques to data from the International Air Transport Association (IATA); finally, in Section 4, we discuss our results and present some recommendations.

## 2. Functional Kernel Density Estimation

#### 2.1. Review of Kernel Density Estimation

#### 2.2. Setup, Assumptions, and Notation

#### 2.3. Preprocessing

#### 2.4. Point Approach to Functional KDE Anomaly Detection

- Choose $\xi >0.$
- For each $x\in \mathcal{X}$, compute its score from (5) where, for example, in the case of $\mathcal{H}={L}^{2}\left([0,T]\right)$,$$\left|\right|x-{a\left|\right|}^{2}={\int}_{0}^{T}{\left|\right(x\left(t\right)-a\left(t\right)|}^{2}\mathrm{d}t.$$
- Identify anomalies as curves with the lowest score.

- To compute $I={\int}_{0}^{T}{\left|\right(x\left(t\right)-a\left(t\right)|}^{2}\mathrm{d}t$, determine all t-values where both x and a are not NaN. Call these ${t}_{0}^{*},{t}_{1}^{*},\dots ,{t}_{r-1}^{*}$.
- Define ${t}_{r}^{*}=T-{t}_{r-1}^{*}+{t}_{0}^{*}$, ${x}_{r}^{*}={x}_{0}^{*}$ and ${y}_{r}^{*}={y}_{0}^{*}$.
- Estimate the integral as$$\begin{array}{cc}\hfill I& \approx \frac{1}{2}\sum _{m=0}^{r-1}({t}_{m+1}^{*}-{t}_{m}^{*})\left(\right|x\left({t}_{m}^{*}\right)-y\left({t}_{m}^{*}\right){|}^{2}+|x\left({t}_{m+1}^{*}\right)-y\left({t}_{m+1}^{*}\right){|}^{2}).\hfill \end{array}$$

#### 2.5. Fourier Approach to Functional KDE Anomaly Detection

- Compute ${p}^{*}=min\{{P}_{1},{P}_{2},\dots ,{P}_{n}\}$.
- Compute the Discrete Fourier coefficients$${\widehat{x}}_{j}^{\left(k\right)}=\frac{1}{{P}_{k}}\sum _{m=0}^{{P}_{k}-1}exp(-2\pi ij{t}_{m}/T){x}^{\left(k\right)}\left({t}_{m}\right)$$
- For each $0\le j\le {p}^{*}-1$, use KDE to estimate the pdf over $\mathbb{C}$ for ${\widehat{x}}_{j}$, by using KDE (Equations (2)–(4)) for $\mathbb{R}$ or ${\mathbb{R}}^{2}$ when the coefficients are all purely real/imaginary or contain a mix of real and imaginary components, respectively. Call the empirical distribution ${\tilde{\zeta}}_{j}$ for each $j.$
- For any $a\in \mathcal{H}$ define an estimated pdf via$${\rho}_{F}\left[a\right]=\prod _{j=0}^{{p}^{*}-1}{\zeta}_{j}\left({\widehat{a}}_{j}\right).$$
- Let the score of $a\in \mathcal{H}$ be$${S}_{F}\left[a\right]=log{\rho}_{F}\left[a\right].$$
- Identify anomalies in $\mathcal{X}$ as those whose scores given by (8) are smallest.

## 3. Method Performance

`fbplot`function in the

`R`package

`fda`is used to obtain a center outward ordering of the time series based on the band depth concept which is a generalization to functional data of the univariate data depth concept [24]. The idea is that anomalous curves will be the ones with the largest ranks, that is, the ones that are farther away from the center. The second method is the recently proposed Functional Isolation Forest (FIF) [17], which is also depth-based and assigns a score to a curve, with higher values indicating that it is more anomalous. We used the code provided for FIF directly on GitHub [25] with the default settings given. The third is the method proposed in [26] and implemented in the

`R`package

`anomalousACM`[27]. This method works in three steps: (i) extract features (e.g., mean, variance, trend) from the time-series; (ii) use Principal Component Analysis (PCA) to identify patterns; (iii) Use a two dimensional outlier detection algorithm with the first two principal components as inputs. It will be referenced as the PCA method in what follows After testing them on synthetic data, we apply our techniques to real data to identify anomalies in time series for aviation events.

`Python`’s default interpolation scheme. For the methods proposed in this paper, we do not have to use imputation.

#### 3.1. Synthetic Data

**Scenario 1**: we define a base curve

- ${C}_{1}\left(t\right)={x}_{0}\left(t\right)\left(1+{r}_{1}\frac{{(t-{t}^{*})}^{2}}{1+{(t-{t}_{0})}^{2}}\Theta (t-{t}_{0})\right)+\u03f5\left(t\right)$, where ${r}_{1}=0.05$ and $\Theta $ denotes the Heaviside function. Thus, the function is scaled up after ${t}_{0}$.
- ${C}_{2}\left(t\right)={x}_{0}\left(t\right)+\left(1+{r}_{2}\Theta (t-{t}_{0})\right)\u03f5\left(t\right)$, where ${r}_{2}=3$. Thus, the noise is larger after ${t}_{0}.$
- ${C}_{3}\left(t\right)={x}_{0}\left(t\right)-{r}_{3}(t-{t}_{0})\Theta (t-{t}_{0})+\u03f5\left(t\right)$, where ${r}_{3}=0.05$. Thus, there is a decreasing component added to the function after ${t}_{0}.$
- ${C}_{4}\left(t\right)=2{a}_{0}\Theta (t-{t}_{0})+{c}_{0}sin({\omega}_{0}t/T)+\u03f5\left(t\right)$, i.e., the tanh is replaced by a discontinuous function.
- ${C}_{5}\left(t\right)={x}_{0}\left(t\right)+\mathcal{E}\left(t\right)$, where $\mathcal{E}\left(t\right)$ represents an exponential random variable at every t with mean $0.05$.
- ${C}_{6}\left(t\right)={a}_{0}(1+tanh\left(2{b}_{0}(t-{t}_{0})\right))+{c}_{0}sin({\omega}_{0}t/T)+\u03f5\left(t\right)$, which has a slightly steeper transition rate than the base curve.
- ${C}_{7}\left(t\right)={a}_{0}(1+tanh\left({b}_{0}(t-{t}_{0})\right)+$${c}_{0}sin((1+{r}_{7}t/T){\omega}_{0}t/T)+\u03f5\left(t\right)$, where ${r}_{7}=0.1$ so the frequency increases with time.

**Scenario 2:**we utilized the testing examples of Staerman et al. [17]. The data consist of 105 time series over $[0,1]$ with 100 time points. There are 100 regular curves defined by $x\left(t\right)=30{(1-t)}^{q}{t}^{q}$ where q is equi-spaced in $[1,1.4]$–thus there is a large family of normal curves. Then, there are 5 anomalous curves:

- ${D}_{1}\left(t\right)=30{(1-t)}^{1.2}{t}^{1.2}+\beta {\chi}_{[0.2,0.8]}$, where $\beta $ is chosen from a Normal distribution with mean 0 and standard deviation $0.3$ and ${\chi}_{I}$ is the characteristic function of I (there is a jump discontinuity at $0.2$ and $0.8$).
- ${D}_{2}\left(t\right)=30{(1-t)}^{1.6}{t}^{1.6}$, being anomalous in its magnitude.
- ${D}_{3}\left(t\right)=30{(1-t)}^{1.2}{t}^{1.2}+sin\left(2\pi t\right)$.
- ${D}_{4}\left(t\right)=30{(1-t)}^{1.2}{t}^{1.2}+2{\chi}_{\left\{\tau \right\}}$, where $\tau =0.7$ is a single point.
- ${D}_{5}\left(t\right)=30{(1-t)}^{1.2}{t}^{1.2}+\frac{1}{2}sin\left(10\pi t\right).$

`R`package

`EnvStats`[29]. In Appendix D, by considering scenarios where anomalies are present or absent, we show the validity of this approach.

#### 3.2. Aviation Safety Reports

## 4. Discussion and Conclusions

#### 4.1. Method Performance

#### 4.2. Aviation Safety Data

#### 4.3. Comments on the Models

#### 4.4. Future Work

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## Appendix A. Fourier Perspective of the Point Method

## Appendix B. Aliasing

## Appendix C. Approximate Orthogonality

**Theorem**

**A1**

**.**Let ${t}_{j}=j\Delta $ for $j=0,1,\dots ,p-1$ where $\Delta =T/p$ for $T>0$. Let ${\left\{{\tilde{t}}_{j}\right\}}_{j=0}^{{P}_{S}-1}\subset \{{t}_{0},{t}_{1},\dots ,{t}_{p-1}\}$. Define $m=p-{P}_{S}$ and define the basis vectors $\{{e}^{\left(k\right)}={\mathrm{e}}^{2\pi ik{\tilde{t}}_{j}/T},j=0,\dots ,{P}_{S}|k=0,\dots ,{P}_{S}\}.$ Then

**Proof.**

## Appendix D. Rosner Test on the Scores

`R`package

`EnvStats`[29]. Specifically, we test H

_{0}for each of the 50 trial datasets of Scenario 1 and report the proportion of time the null hypothesis is rejected at the $\alpha =0.05$ level. In this case, the proportion of rejection measures the power of the test and we wish to have the highest values possible. However, to verify the validity of the test, we also run the tests on the samples containing only the 63 normal curves. This time, we want the proportion of rejection to be close to the level $\alpha =0.05$. The results, presented in Table A1, show that this method is working. Setting aside the un-normalized Fourier approach (Fourier-U) with 10% of missing data, the proportion of rejection varies between 0.02 and 0.08 when the data contain no anomalies, showing that the test is able to maintain its prescribed level. When the data contain anomalies, the power ranges between 0.98 and 1, showing that the anomalies are detected in almost all cases. The only exception is the Fourier-U method with 10% of missing data, which never rejects ${H}_{0}$ whether or not the data contain anomalies. But this is consistent with the fact that this method had a very poor performance in this case and was not able to detect the anomalies as seen in Table 1.

**Table A1.**Proportions of time the null hypothesis ${H}_{0}$ is rejected at the $\alpha =0.05$ level. See Table 1 caption of main manuscript for -N vs -U distinction.

Method | Lost | Anomalies Present | H_{0} Reject Fraction |
---|---|---|---|

Point-N | 0% | No | 0.04 |

Point-N | 0% | Yes | 1 |

Point-U | 0% | No | 0.08 |

Point-N | 0% | Yes | 1 |

Fourier-N | 0% | No | 0.08 |

Fourier-N | 0% | Yes | 0.98 |

Fourier-U | 0% | No | 0.02 |

Fourier-U | 0% | Yes | 1 |

Point-N | 10% | No | 0.04 |

Point-N | 10% | Yes | 1 |

Point-U | 10% | No | 0.08 |

Point-N | 10% | Yes | 1 |

Fourier-N | 10% | No | 0.06 |

Fourier-N | 10% | Yes | 1 |

Fourier-U | 10% | No | 0 |

Fourier-U | 10% | Yes | 0 |

## References

- Cleophas, T.J.; Zwinderman, A.H.; Cleophas-Allers, H.I. Machine Learning in Medicine; Springer: Berlin/Heidelberg, Germany, 2013; Volume 9. [Google Scholar]
- Howedi, A.; Lotfi, A.; Pourabdollah, A. An Entropy-Based Approach for Anomaly Detection in Activities of Daily Living in the Presence of a Visitor. Entropy
**2020**, 22, 845. [Google Scholar] [CrossRef] - Chen, X.; Li, Z.; Wang, Y.; Tang, J.; Zhu, W.; Shi, C.; Wu, H. Anomaly detection and cleaning of highway elevation data from google earth using ensemble empirical mode decomposition. J. Transp. Eng. Part Syst.
**2018**, 144, 04018015. [Google Scholar] [CrossRef] - Siboni, S.; Cohen, A. Anomaly Detection for Individual Sequences with Applications in Identifying Malicious Tools. Entropy
**2020**, 22, 649. [Google Scholar] [CrossRef] - Bernikova, O.; Granichin, O.; Lemberg, D.; Redkin, O.; Volkovich, Z. Entropy-Based Approach for the Detection of Changes in Arabic Newspapers’ Content. Entropy
**2020**, 22, 441. [Google Scholar] [CrossRef][Green Version] - Saradjian, M.; Akhoondzadeh, M. Thermal anomalies detection before strong earthquakes (M > 6.0) using interquartile, wavelet and Kalman filter methods. Nat. Hazards Earth Syst. Sci.
**2011**, 11, 1099. [Google Scholar] [CrossRef] - Xu, X.; Ren, Y.; Huang, Q.; Fan, Z.Y.; Tong, Z.J.; Chang, W.J.; Liu, B. Anomaly detection for large span bridges during operational phase using structural health monitoring data. Smart Mater. Struct.
**2020**, 29, 045029. [Google Scholar] [CrossRef] - Holst, A.; Bohlin, M.; Ekman, J.; Sellin, O.; Lindström, B.; Larsen, S. Statistical anomaly detection for train fleets. AI Mag.
**2013**, 34, 33. [Google Scholar] [CrossRef][Green Version] - Das, S.; Matthews, B.L.; Lawrence, R. Fleet level anomaly detection of aviation safety data. In Proceedings of the 2011 IEEE Conference on Prognostics and Health Management, Montreal, QC, Canada, 20–23 June 2011; pp. 1–10. [Google Scholar]
- Ketabdari, M.; Giustozzi, F.; Crispino, M. Sensitivity analysis of influencing factors in probabilistic risk assessment for airports. Saf. Sci.
**2018**, 107, 173–187. [Google Scholar] [CrossRef] - Huang, Z.; Lu, X.; Duan, H. Anomaly detection in clinical processes. In AMIA Annual Symposium Proceedings; American Medical Informatics Association: Bethesda, MD, USA, 2012; Volume 2012, p. 370. [Google Scholar]
- Aydin, I.; Karakose, M.; Akin, E. Anomaly detection using a modified kernel-based tracking in the pantograph–catenary system. Expert Syst. Appl.
**2015**, 42, 938–948. [Google Scholar] [CrossRef] - Braei, M.; Wagner, S. Anomaly Detection in Univariate Time-series: A Survey on the State-of-the-Art. arXiv
**2020**, arXiv:2004.00433. [Google Scholar] - Blázquez-García, A.; Conde, A.; Mori, U.; Lozano, J.A. A review on outlier/anomaly detection in time series data. arXiv
**2020**, arXiv:2002.04236. [Google Scholar] - Martos, G.; Hernández, N.; Muñoz, A.; Moguerza, J.M. Entropy measures for stochastic processes with applications in functional anomaly detection. Entropy
**2018**, 20, 33. [Google Scholar] [CrossRef][Green Version] - Vinue, G.; Epifanio, I. Robust archetypoids for anomaly detection in big functional data. In Advances in Data Analysis and Classification; Springer: Berlin/Heidelberg, Germany, 2020; pp. 1–26. [Google Scholar]
- Staerman, G.; Mozharovskyi, P.; Clémençon, S.; d’Alché-Buc, F. Functional Isolation Forest. Proc. Mach. Learn. Res.
**2019**, 101, 332–347. [Google Scholar] - Jackson, A.C.; Lacey, S. Seasonality and Anomaly Detection in Rare Data Using the Discrete Fourier Transformation. In Proceedings of the 2019 First International Conference on Digital Data Processing (DDP), London, UK, 15–17 November 2019; pp. 13–17. [Google Scholar]
- Hyndman, R.L.; Zhang, X.; King, M.L. Bandwidth selection for multivariate kernel density estimation using mcmc. In Proceedings of the Econometric Society 2004 Australasian Meetings, Melbourne, Australia, 7–9 July 2004. number 120. [Google Scholar]
- Guo, Y.; Xu, Q.; Li, P.; Sbert, M.; Yang, Y. Trajectory shape analysis and anomaly detection utilizing information theory tools. Entropy
**2017**, 19, 323. [Google Scholar] [CrossRef] - Maniglia, S.; Rhandi, A. Gaussian measures on separable Hilbert spaces and applications. Quaderni di Matematica. 2004. Available online: http://siba-ese.unisalento.it/index.php/quadmat/issue/view/775 (accessed on 30 November 2020).
- Mobli, M.; Hoch, J.C. Nonuniform sampling and non-Fourier signal processing methods in multidimensional NMR. Prog. Nucl. Magn. Reson. Spectrosc.
**2014**, 83, 21–41. [Google Scholar] [CrossRef] [PubMed][Green Version] - Sun, Y.; Genton, M.G. Functional boxplots. J. Comput. Graph. Stat.
**2011**, 20, 316–334. [Google Scholar] [CrossRef] - Ramsay, J.O.; Graves, S.; Hooker, G. fda: Functional Data Analysis; R Package Version 5.1.5.1. Available online: https://cran.r-project.org/web/packages/fda/index.html (accessed on 30 November 2020).
- Staerman, G.; Mozharovskyi, P.; Clémençon, S.; d’Alché-Buc, F. FIF: Functional Isolation Forest. Available online: https://github.com/GuillaumeStaermanML/FIF (accessed on 15 September 2020).
- Hyndman, R.J.; Wang, E.; Laptev, N. Large-scale unusual time series detection. In Proceedings of the 2015 IEEE international conference on data mining workshop (ICDMW), Atlantic City, NJ, USA, 14–17 November 2015; pp. 1616–1619. [Google Scholar]
- Hyndman, R.J.; Wang, E.; Laptev, N. anomalousACM: Unusual Time Series Detection; R Package Version 0.1.0. Available online: https://github.com/robjhyndman/anomalous-acm (accessed on 30 November 2020).
- Rosner, B. Percentage points for a generalized ESD many-outlier procedure. Technometrics
**1983**, 25, 165–172. [Google Scholar] [CrossRef] - Millard, S.P. EnvStats: An R Package for Environmental Statistics; Springer: New York, NY, USA, 2013. [Google Scholar]
- Sturges, H.A. The choice of a class interval. J. Am. Stat. Assoc.
**1926**, 21, 65–66. [Google Scholar] [CrossRef] - Folland, G.B. Real Analysis: Modern Techniques and Their Applications; John Wiley & Sons: Hoboken, NJ, USA, 1999; Volume 40. [Google Scholar]

**Figure 1.**A visual depiction of the Point method. The curves are time series in a Hilbert space $\mathcal{H}$ but after applying KDE, there is a score associated to each point in $\mathcal{H}$. In the cartoon, curves 1 and 2 are similar and curve 3 is anomalous. (

**Left**): the time series. (

**Right**): a representation of them with associated scores in the color scale. In reality, the space is infinite dimensional and this is only a conceptual illustration.

**Figure 2.**Plot of 63 normal curves and the 7 anomalous curves ${C}_{i}\left(t\right)$, $i=1,\dots ,7$. Left: un-normalized. Right: normalized.

**Figure 3.**Histogram of scores for Point and Fourier methods for Type A Point (

**top-left**), Type A Fourier(

**top-right**), Type B Point (

**bottom-left**) and Type B Fourier (

**bottom-right**). The dashed vertical line represents the division we chose between anomalous (left of line) and normal (right of line). The Sturges estimate was used to set bin widths [30].

**Figure 4.**Plots of the time series for Type A and Type B events. Anomalous are dotted curves with markers in the legend; normal curves are solid black curves.

**Table 1.**Mean percentiles (out of 100) for curves ${C}_{1}$–${C}_{7}$ in Scenario 1. A correct classification is a percentile less than or equal to 10 (in bold in the table). The -N suffix denotes the data were normalized by the pre-processing described in Section 2.3; the -U suffixed denotes the data were un-normalized. Note that method FB is not affected by the normalization.

Method | Lost | ${\mathit{C}}_{1}$ | ${\mathit{C}}_{2}$ | ${\mathit{C}}_{3}$ | ${\mathit{C}}_{4}$ | ${\mathit{C}}_{5}$ | ${\mathit{C}}_{6}$ | ${\mathit{C}}_{7}$ |
---|---|---|---|---|---|---|---|---|

Point-N | 0% | 4.3 | 5.8 | 1.4 | 21 | 12 | 24 | 2.9 |

Point-U | 0% | 4.3 | 5.7 | 1.4 | 14 | 8.8 | 16 | 2.9 |

Fourier-N | 0% | 4.7 | 1.9 | 2.8 | 28 | 51 | 29 | 8.5 |

Fourier-U | 0% | 5.8 | 4.0 | 2.8 | 38 | 43 | 35 | 1.7 |

FIF-N | 0% | 51 | 24 | 72 | 56 | 79 | 58 | 13 |

FIF-U | 0% | 19 | 2.2 | 4.8 | 18 | 53 | 19 | 5.3 |

PCA | 0% | 7.5 | 20. | 4.3 | 7.5 | 53 | 9.4 | 11 |

FB | 0% | 4.5 | 5.6 | 1.8 | 36 | 21 | 37 | 2.5 |

Point-N | 10% | 4.3 | 6.0 | 1.4 | 23 | 18 | 29 | 2.9 |

Point-U | 10% | 4.3 | 5.7 | 1.4 | 20. | 14 | 23 | 2.9 |

Fourier-N | 10% | 4.3 | 4.5 | 2.5 | 28 | 43 | 36 | 4.0 |

Fourier-U | 10% | 45 | 59 | 50 | 46 | 49 | 53 | 49 |

FIF-N | 10% | 50. | 21 | 75 | 48 | 74 | 51 | 13 |

FIF-U | 10% | 45 | 15 | 29 | 42 | 48 | 44 | 32 |

PCA | 10% | 32 | 20. | 6.1 | 36 | 47 | 35 | 7.7 |

FB | 10% | 7.5 | 8.7 | 4.2 | 47 | 24 | 49 | 5.1 |

**Table 2.**The 95th percentile of the percentile ranks (out of 100) for curves ${C}_{1}$–${C}_{7}$ in Scenario 1. See Table 1 caption for -N vs. -U distinction.

Method | Lost | ${\mathit{C}}_{1}$ | ${\mathit{C}}_{2}$ | ${\mathit{C}}_{3}$ | ${\mathit{C}}_{4}$ | ${\mathit{C}}_{5}$ | ${\mathit{C}}_{6}$ | ${\mathit{C}}_{7}$ |
---|---|---|---|---|---|---|---|---|

Point-N | 0% | 4.3 | 6.5 | 1.4 | 47 | 42 | 62 | 2.9 |

Point-U | 0% | 4.3 | 5.7 | 1.4 | 30. | 12 | 46 | 2.9 |

Fourier-N | 0% | 8.6 | 3.6 | 4.3 | 67 | 99 | 74 | 13 |

Fourier-U | 0% | 5.7 | 4.3 | 4.2 | 84 | 94 | 77 | 2.9 |

FIF-N | 0% | 84 | 69 | 92 | 90 | 100 | 96 | 30 |

FIF-U | 0% | 57 | 7.5 | 11 | 55 | 97 | 53 | 11 |

PCA | 0% | 25 | 67 | 7.1 | 17 | 96 | 27 | 47 |

FB | 0% | 5.7 | 5.7 | 2.9 | 75 | 74 | 75 | 2.9 |

Point-N | 10% | 4.3 | 7.1 | 1.4 | 52 | 51 | 72 | 2.9 |

Point-U | 10% | 4.3 | 5.7 | 1.4 | 46 | 43 | 59 | 2.9 |

Fourier-N | 10% | 6.5 | 10. | 5.1 | 61 | 91 | 89 | 5.7 |

Fourier-U | 10% | 84 | 97 | 92 | 93 | 93 | 92 | 89 |

FIF-N | 10% | 81 | 67 | 97 | 91 | 99 | 95 | 39 |

FIF-U | 10% | 84 | 27 | 56 | 88 | 95 | 94 | 56 |

PCA | 10% | 82 | 60. | 31 | 73 | 94 | 73 | 19 |

FB | 10% | 11 | 13 | 10. | 75 | 75 | 75 | 9.4 |

**Table 3.**Percentiles (out of 100) for curves ${D}_{1}$–${D}_{5}$ in Scenario 2. A correct classification is a percentile less than or equal to $4.8$ (in bold in the table) since $5/105=4.8\%$. See Table 1 caption for -N vs. -U distinction.

Method | ${\mathit{D}}_{1}$ | ${\mathit{D}}_{2}$ | ${\mathit{D}}_{3}$ | ${\mathit{D}}_{4}$ | ${\mathit{D}}_{5}$ |
---|---|---|---|---|---|

Point-N | 74 | 0.95 | 6.7 | 73 | 1.9 |

Point-U | 83 | 0.95 | 44 | 85 | 71 |

Fourier-N | 3.8 | 4.8 | 1.9 | 2.9 | 0.95 |

Fourier-U | 1.9 | 8.6 | 30 | 0.95 | 2.9 |

FIF-N | 1.9 | 28 | 3.8 | 10 | 0.95 |

FIF-U | 1.9 | 2.9 | 3.8 | 4.8 | 0.95 |

PCA | 4.8 | 2.9 | 3.8 | 1.9 | 0.95 |

FB | 75 | 0.95 | 21 | 75 | 75 |

**Table 4.**IDs of anomalous flights for events A and B. Columnwise, the bolded IDs are common to both methods for a given event type.

Method | Type A | Type B |
---|---|---|

Point | 14, 21, 22, 23, 35 | 1, 2, 4, 5, 6, 7, 8, 9, 25, 34 |

Fourier | 14, 21, 22, 35 | 1, 2, 4, 7, 8, 9 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Lindstrom, M.R.; Jung, H.; Larocque, D. Functional Kernel Density Estimation: Point and Fourier Approaches to Time Series Anomaly Detection. *Entropy* **2020**, *22*, 1363.
https://doi.org/10.3390/e22121363

**AMA Style**

Lindstrom MR, Jung H, Larocque D. Functional Kernel Density Estimation: Point and Fourier Approaches to Time Series Anomaly Detection. *Entropy*. 2020; 22(12):1363.
https://doi.org/10.3390/e22121363

**Chicago/Turabian Style**

Lindstrom, Michael R., Hyuntae Jung, and Denis Larocque. 2020. "Functional Kernel Density Estimation: Point and Fourier Approaches to Time Series Anomaly Detection" *Entropy* 22, no. 12: 1363.
https://doi.org/10.3390/e22121363