# Functional Outlier Detection by Means of h-Mode Depth and Dynamic Time Warping

^{1}

^{2}

^{3}

^{4}

^{5}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Adapting Gaussian Mixture Models for Outlier Detection

- If the probabilistic model is adjusted including the potential outliers, they may bias the estimation of the underlying model. This is especially problematic if the outliers are assumed to be generated by a different distribution than the other data and are not only considered to be extreme realizations of the same underlying process as the others. On top of that, if the sample presents a high degree of contamination or the sample is small, this bias can greatly influence the detection.
- If the multivariate sample can be classified in several different clusters but they number of components is not well-chosen, the possibility of overfitting the probabilistic model to the data becomes a real problem. In this case, some small-sized clusters may appear overly adjusted to the outliers, which will not be identified as such.

- Initialize the values of the desired parameters ${\{{\mathbf{\omega}}_{k},{\mathbf{\mu}}_{k},{\mathsf{\Sigma}}_{k}\}}_{k=1}^{K}$,
- E-step: Evaluate the current responsibilities with the previous parameter values,
- M-step: Re-estimate the parameters using the responsibilities,
- Evaluate the log-likelihood function presented in Equation (1).

- Check if ${\mathit{u}}_{i}={\mathbf{\mu}}_{k}$ for any $k\in \{1,...,K\}$. The appearance of these singularities may maximize the log-likelihood function to infinity, since it is unbounded, and cause an overfitting of the data to isolated points. This phenomenon is well described in [28]. If this kind of anomaly is detected, the point is subtracted from the sample and the EM algorithm continues without it. Naturally, these points will be subject to close analysis since they are good candidates for potential outliers in the sample.
- For each iteration step, ${\omega}_{k}$ can be viewed as the prior probability of ${g}_{k}=1$, and $\gamma \left({g}_{k}\right)$ can be seen as the posterior probability given the observed sample. If this posterior probability is considered too low (in our applications we shall take a value of $0.1$ as the minimum weight of the mixing coefficients), we will consider that the corresponding component is either overfitting the data, or that it has detected a small subset of points which is not representative of the central trend of the data. In this case, the other calculated parameters of the components are kept and the values of means and covariances of the small cluster are reinitialized to a random value in the space.

## 3. Outlier Detection

#### 3.1. Test for Outlyingness

#### 3.2. Ordering Score in the Feature Space

#### 3.3. Proposed Detection Algorithm

Algorithm 1: FOD algorithm |

## 4. Analytical Test Cases

#### 4.1. Numerical Test Protocol

- Model 1. $Z\left(t\right)=4t+G\left(t\right)$ is the function generator for the reference set of curves. In this case, the outliers follow the distribution ${Z}_{o}\left(t\right)=4t+G\left(t\right)+2{\U0001d7d9}_{\left\{({t}_{I}<t)\right\}}$.
- Model 2. The reference model for the curve generation remains $Z\left(t\right)=4t+G\left(t\right)$, whereas the outliers are now generated from the distribution ${Z}_{o}\left(t\right)=4t+G\left(t\right)+2{\U0001d7d9}_{\left\{({t}_{I}<t<{t}_{I}+3)\right\}}$.
- Model 3. Here the reference model becomes $Z\left(t\right)=30t{(1-t)}^{3/2}+G\left(t\right)$. The outliers are generated from ${Z}_{o}\left(t\right)=30(1-t){t}^{3/2}+G\left(t\right)$.
- Model 4. For this last case, we keep the reference model as it is for Model 1 and Model 2, but the outliers simply consist of the sole deterministic part ${Z}_{o}\left(t\right)=4t$ (the Gaussian component is removed).

- The modified band depth (BD) (see Equation (3) of the Supplementary Material),
- The h-mode depth (see Equation (4) of the Supplementary Material),
- The dynamic time warping (see Equation (7) of the Supplementary Material),
- The L${}^{2}$ norm, which is one of the most intuitive and widely used metrics that can be applied to functional data. It takes the form: ${\left|\right|z\left(t\right)\left|\right|}_{2}={\left(\right)}^{{\int}_{\mathbb{R}}}1/2$.

#### 4.2. Comparison with State-of-the-Art Methodologies

#### 4.2.1. Functional Boxplots

#### 4.2.2. High-Density Regions

#### 4.2.3. Directional Detector

#### 4.2.4. Sequential Transformations

#### 4.2.5. Results

#### 4.3. Ranking Results

## 5. Industrial Test-Case Study

#### 5.1. Presentation

#### 5.2. Functional Outlier Detection

#### 5.3. Sensitivity Analysis on Outliers

## 6. Discussion and Conclusions

## Supplementary Materials

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Abbreviations

DTW | Dynamic Time Warping |

i.i.d. | independent and identically distributed |

GMM | Gaussian Mixture Models |

HDR | High-Density Regions |

LOCA | Loss of Coolant Accident |

PCT | Peak Cladding Temperature |

HSIC | Hilbert Schmidt Independence Criterion |

## References

- Chamroukhi, F.; Nguyen, H.D. Model-based clustering and classification of functional data. WIREs Data Min. Knowl. Discov.
**2019**, 9, 1298. [Google Scholar] [CrossRef] [Green Version] - Slaets, L.; Claeskens, G.; Hubert, M. Phase and Amplitude-Based Clustering for Functional Data. Comput. Stat. Data Anal.
**2012**, 56, 2360–2374. [Google Scholar] [CrossRef] - Ieva, F.; Paganoni, A.; Pigoli, D.; Vitelli, V. ECG signal reconstruction, Landmark registration and functional classification. In Proceedings of the SCO 2011, Sharable Content Objects, 7th Conference about Electronic Support of Learning, Brno, Czech Republic, 22–23 June 2011. [Google Scholar]
- Grenander, U. Stochastic processes and statistical inference. Ark. Mat.
**1950**, 1, 195–277. [Google Scholar] [CrossRef] - Ramsay, J.O. When the data are functions. Psychometrika
**1982**, 47, 379–396. [Google Scholar] [CrossRef] - Ramsay, J.; Silverman, B. Functional Data Analysis; Springer Series in Statistics; Springer: Berlin/Heidelberg, Germany, 1997. [Google Scholar]
- Ferraty, F.; Vieu, P. Nonparametric Functional Data Analysis: Theory and Practice; Springer: Berlin/Heidelberg, Germany, 2006; Volume 51. [Google Scholar]
- Besse, P.; Cardot, H.; Faivre, R.; Goulard, M. Statistical modelling of functional data. Appl. Stoch. Model. Bus. Ind.
**2005**, 21, 165–173. [Google Scholar] [CrossRef] - Febrero-Bande, M.; Galeano, P.; Gonz?lez-Manteiga, W. Outlier detection in functional data by depth measures, with application to identify abnormal NOx levels. Environmetrics
**2008**, 19, 331–345. [Google Scholar] [CrossRef] - Juang, W.C.; Huang, S.J.; Huang, F.D.; Cheng, P.W.; Wann, S.R. Application of time series analysis in modelling and forecasting emergency department visits in a medical centre in Southern Taiwan. BMJ Open
**2017**, 7, e018628. [Google Scholar] [CrossRef] [Green Version] - Sen, R.; Klüppelberg, C. Time series of functional data with application to yield curves. Appl. Stoch. Model. Bus. Ind.
**2019**, 35, 1028–1043. [Google Scholar] [CrossRef] - Santner, T.J.; Williams, B.J.; Notz, W. The Design and Analysis of Computer Experiments, 1st ed.; Springer Series in Statistics; Springer: Berlin/Heidelberg, Germany, 2003. [Google Scholar]
- Roustant, O.; Joucla, J.; Probst, P. Kriging as an alternative for a more precise analysis of output parameters in nuclear safety—Large break LOCA calculation. Appl. Stoch. Model. Bus. Ind.
**2010**, 26, 565–576. [Google Scholar] [CrossRef] - IAEA. Accident Analysis for Nuclear Power Plants with Pressurized Water Reactors; Number 30 in Safety Reports Series; International Atomic Energy Agency: Vienna, Austria, 2003. [Google Scholar]
- Geffraye, G.; Antoni, O.; Farvacque, M.; Kadri, D.; Lavialle, G.; Rameau, B.; Ruby, A. CATHARE 2 V2.5_2: A single version for various applications. Nucl. Eng. Des.
**2011**, 241, 4456–4463. [Google Scholar] [CrossRef] - Nanty, S. Stochastic Methods for Uncertainty Treatment of Functional Variables in Computer Codes: Application to Safety Studies. Ph.D. Thesis, Université Grenoble Alpes, Saint-Martin-d’Hères, France, 2015. [Google Scholar]
- Auder, B. Classification and Modelling of Computer Codes Functional Outputs: Application to Accidental Thermo-Hydraulic Computations in Pressurized Water Reactors (PWR). Ph.D. Thesis, Université Paris 6, Paris, France, 2011. [Google Scholar]
- James, G.M.; Hastie, T.J.; Sugar, C.A. Principal Component Models for Sparse Functional Data. Biometrika
**2000**, 87, 587–602. [Google Scholar] [CrossRef] [Green Version] - Aggarwal, C. Outlier Analysis, 2nd ed.; Springer: Berlin/Heidelberg, Germany, 2017. [Google Scholar]
- Schmutz, A.; Jacques, J.; Bouveyron, C.; Chèze, L.; Martin, P. Clustering multivariate functional data in group-specific functional subspaces. Comput. Stat.
**2020**, 35, 1101–1131. [Google Scholar] [CrossRef] [Green Version] - López-Pintado, S.; Sun, Y.; Lin, J.K.; Genton, M.G. Simplicial band depth for multivariate functional data. Adv. Data Anal. Classif.
**2014**, 8, 321–338. [Google Scholar] [CrossRef] - Nagy, S.; Gijbels, I.; Hlubinka, D. Depth-Based Recognition of Shape Outlying Functions. J. Comput. Graph. Stat.
**2017**, 26, 883–893. [Google Scholar] [CrossRef] - Martos, G.; Hernández, N.; Mu?oz, A.; Moguerza, J. Entropy measures for stochastic processes with applications in functional anomaly detection. Entropy
**2018**, 20, 33. [Google Scholar] [CrossRef] [Green Version] - Barreyre, C.; Laurent, B.; Loubes, J.M.; Boussouf, L.; Cabon, B. Multiple Testing for Outlier Detection in Space Telemetries. IEEE Trans. Big Data
**2020**, 6, 443–451. [Google Scholar] [CrossRef] - Arribas-Gil, A.; Romo, J. Shape outlier detection and visualization for functional data: The outliergram. Biostatistics
**2014**, 15, 603–619. [Google Scholar] [CrossRef] [Green Version] - Sguera, C.; Galeano, P.; Lillo, R. Functional outlier detection by a local depth with applications to NOx levels. Stoch. Environ. Res. Risk Assess.
**2016**, 30, 1115–1130. [Google Scholar] [CrossRef] - Moon, T. The expectation-maximization algorithm. IEEE Signal Process. Mag.
**1996**, 13, 47–60. [Google Scholar] [CrossRef] - Bishop, C.M. Pattern Recognit. Mach. Learn.; Springer: Berlin/Heidelberg, Germany, 2006. [Google Scholar]
- Ahidar-Coutrix, A. Surfaces quantile: Propriétés, convergences et applications. Ph.D. Thesis, Université de Toulouse, Toulouse, France, 2015. [Google Scholar]
- Hosseini, R.; Sra, S. Matrix Manifold Optimization for Gaussian Mixtures. In Advances in Neural Information Processing Systems; Cortes, C., Lawrence, N., Lee, D., Sugiyama, M., Garnett, R., Eds.; Curran Associates, Inc.: Red Hook, NY, USA, 2015; Volume 28. [Google Scholar]
- Genz, A. Numerical Computation Of Multivariate Normal Probabilities. J. Comput. Graph. Stat.
**2000**, 1, 141–149. [Google Scholar] - Efron, B.; Tibshirani, R. An Introduction to the Bootstrap; Macmillan: New York, NY, USA, 1994. [Google Scholar]
- Schwarz, G. Estimating the Dimension of a Model. Ann. Stat.
**1978**, 6, 461–464. [Google Scholar] [CrossRef] - Biernacki, C.; Celeux, G.; Govaert, G. Assessing a Mixture Model for Clustering with the Integrated Completed Likelihood. Pattern Anal. Mach. Intell. IEEE Trans.
**2000**, 22, 719–725. [Google Scholar] [CrossRef] - Dai, W.; Mrkvicka, T.; Sun, Y.; Genton, M. Functional outlier detection and taxonomy by sequential transformations. Comput. Stat. Data Anal.
**2020**, 149. [Google Scholar] [CrossRef] [Green Version] - López-Pintado, S.; Romo, J. On the Concept of Depth for Functional Data. J. Am. Stat. Assoc.
**2009**, 104, 718–734. [Google Scholar] [CrossRef] [Green Version] - Long, J.P.; Huang, J.Z. A Study of Functional Depths. arXiv
**2015**, arXiv:1506.01332. [Google Scholar] - Sun, Y.; Genton, M.G. Functional Boxplots. J. Comput. Graph. Stat.
**2011**, 20, 316–334. [Google Scholar] [CrossRef] - Hyndman, R.J. Rainbow plots, bagplots and boxplots for functional data. J. Comput. Graph. Stat.
**2009**, 19, 29–45. [Google Scholar] [CrossRef] [Green Version] - Dai, W.; Genton, M. Directional outlyingness for multivariate functional data. Comput. Stat. Data Anal.
**2019**, 131, 50–66. [Google Scholar] [CrossRef] [Green Version] - Cuevas, A.; Fraiman, R. On depth measures and dual statistics. A methodology for dealing with general data. J. Multivar. Anal.
**2009**, 100, 753–766. [Google Scholar] [CrossRef] [Green Version] - Iooss, B.; Marrel, A. Advanced Methodology for Uncertainty Propagation in Computer Experiments with Large Number of Inputs. Nucl. Technol.
**2019**, 205, 1588–1606. [Google Scholar] [CrossRef] [Green Version] - Da Veiga, S. Global Sensitivity Analysis with Dependence Measures. J. Stat. Comput. Simul.
**2015**, 85, 1283–1305. [Google Scholar] [CrossRef] - De Lozzo, M.; Marrel, A. New improvements in the use of dependence measures for sensitivity analysis and screening. J. Stat. Comput. Simul.
**2016**, 86, 3038–3058. [Google Scholar] [CrossRef] [Green Version]

**Figure 1.**Examples of the four analytical test cases. The blue curves correspond to the 49 ones that are generated from the main model, whereas the red one corresponds to the outlier.

**Figure 2.**Examples of IBLOCA transients simulated with CATHARE2. Only 40 curves are displayed for clarity.

**Figure 3.**IBLOCA transient curves presenting the highest degree of outlyingness (red) and the least outlying curve (blue).

**Figure 4.**Scatter plots of two input variables of CATHARE2 and the outlyingness score $\theta $. The points correspond to the bivariate plot of the values of the selected variable and the corresponding $\theta $. The red dots correspond to the simulations that have been retained as outliers.

Notation | Description |
---|---|

$G\left(t\right)$ | Centered Gaussian process of covariance function $\mathsf{\Sigma}({t}_{1},{t}_{2})=0.3exp\frac{-|{t}_{1}-{t}_{2}|}{0.3}$ |

$Z\left(t\right)$ | Functional random variable generating the main model |

${Z}_{0}$ | Functional random variable generating the outliers |

${T}_{I}$ | Random point uniformly generated in the definition domain of the function |

**Table 2.**Performance of the different algorithms on the test models. The results are expressed as a percentage (detection rates). DO: Directional Detector; FB: Functional Boxplots; HDR: High-Density Regions. The values given in bold correspond to the best results for each model.

N = 100, p = 1% | Model 1 | Model 2 | Model 3 | Model 4 |
---|---|---|---|---|

Algorithm | 100.00 | 96.94 | 100.00 | 100.00 |

DO | 59.26 | 39.51 | 100.00 | 0.00 |

FB | 2.33 | 0.00 | 100.00 | 0.00 |

HDR | 89.47 | 69.64 | 100.00 | 0.00 |

N = 100,p= 5% | ||||

Algorithm | 91.14 | 96.79 | 99.17 | 97.50 |

DO | 58.23 | 54.40 | 100.00 | 0.00 |

FB | 2.53 | 4.18 | 11.95 | 0.00 |

HDR | 48.35 | 44.8 | 49.48 | 0.00 |

N = 100,p= 10% | ||||

Algorithm | 81.50 | 75.49 | 86.67 | 92.37 |

DO | 47.25 | 45.97 | 99.63 | 0.00 |

FB | 0.75 | 1.71 | 7.41 | 0.00 |

HDR | 22.25 | 23.41 | 14.07 | 0.00 |

**Table 3.**Average ranking of the outlier curve across the 100 replications of the experiments for the selected models. The Sequential transformations procedure is presented in [35]. The Modified Band Depth is presented in [36], and the standard Integrated Depth appears in [41]. In this case, the closer the value of a method is to 100, the more outlying it will be according to the corresponding ranking measure.

N = 100, p = 1% | Model 1 | Model 2 | Model 3 | Model 4 |
---|---|---|---|---|

Algorithm | 98.36 | 97.80 | 99.57 | 93.06 |

Sequential Transformations | 98.14 | 97.34 | 99.97 | 99.88 |

Modified Band Depth | 84.00 | 61.39 | 98.49 | 1 |

Integrated Depth | 83.15 | 59.81 | 98.42 | 1 |

**Table 4.**Detected influential variables for $\theta \in \mathcal{M}$. The variables are not the actual values of the physical parameters, but

**multiplicative coefficients**that increase of decrease their importance in a scenario.

Variable | Description |
---|---|

${X}_{16}$ | Friction between the water and the discharge line of the accumulators |

${X}_{38}$ | Global heat transfer coefficient in reflood fuel/coolant |

${X}_{45}$ | Pressure drop to model the constrained flow due to the deformation of the fuel |

${X}_{62}$ | Friction coefficient between steam and water in the downcomer during the reflood phase |

${X}_{64}$ | Friction coefficient between water and steam in core during the reflood phase |

${X}_{68}$ | Heat transfer coefficient between steam and water in the downcomer |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Rollón de Pinedo, Á.; Couplet, M.; Iooss, B.; Marie, N.; Marrel, A.; Merle, E.; Sueur, R.
Functional Outlier Detection by Means of h-Mode Depth and Dynamic Time Warping. *Appl. Sci.* **2021**, *11*, 11475.
https://doi.org/10.3390/app112311475

**AMA Style**

Rollón de Pinedo Á, Couplet M, Iooss B, Marie N, Marrel A, Merle E, Sueur R.
Functional Outlier Detection by Means of h-Mode Depth and Dynamic Time Warping. *Applied Sciences*. 2021; 11(23):11475.
https://doi.org/10.3390/app112311475

**Chicago/Turabian Style**

Rollón de Pinedo, Álvaro, Mathieu Couplet, Bertrand Iooss, Nathalie Marie, Amandine Marrel, Elsa Merle, and Roman Sueur.
2021. "Functional Outlier Detection by Means of h-Mode Depth and Dynamic Time Warping" *Applied Sciences* 11, no. 23: 11475.
https://doi.org/10.3390/app112311475