# A Comprehensive Case Study of Data-Driven Methods for Robust Aircraft Sensor Fault Isolation

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Models for Sensor Fault Diagnosis

#### Sensor Fault Modeling

## 3. Sensor FI and FE Based on Primary Residuals

#### 3.1. Mahalanobis Distance-Based FI and FE

#### 3.2. Reconstruction-Based FI and FE

## 4. Sensor FI and FE Based on Transformed Residuals

#### 4.1. Transformed Residuals Based on FI and FE

#### 4.2. Transformed Residuals with Reconstruction-Based FI and FE

## 5. Bayesian Filtering for Online Fault Isolation

#### 5.1. Probabilistic (Online) Fault Isolation Method

#### 5.2. Probability Function Tuning

#### 5.2.1. Distance-Based Methods’ Tuning

#### 5.2.2. Reconstruction-Based Methods’ Tuning

## 6. Aircraft and Flight Data

#### Data Normalization

## 7. Experimental Models for Sensor FI

## 8. FI and FE Performance on the Validation Data (Offline Analysis)

#### 8.1. Fault Isolation Percentage

- ${N}_{val}$: number of validation flights.
- ${N}_{O{K}_{j,i}}\left({A}_{i}\right)$: number of samples for which the fault isolation index ${i}_{F}\left(k\right)$ correctly isolates the fault on the i-th sensor, in validation flight j, for fault amplitude ${A}_{i}$.
- ${N}_{{I}_{j}}$: total number of samples in validation flight j.

#### 8.2. Relative Fault Reconstruction Error

- ${A}_{i}$: amplitude of the fault on sensor i;
- ${\widehat{A}}_{{i}_{j}}\left(k\right)$: amplitude of the reconstructed fault at sample time k for the validation flight j.
- ${S}_{{E}_{j}}$: set of samples in validation flight j where the fault is correctly attributed to the i-th sensor.
- ${N}_{{E}_{j}}$: number of samples in the set ${S}_{{E}_{j}}$.

## 9. FI Performance on Validation Data (Online Analysis)

#### Fault Isolation Delay

- ${T}_{isolatio{n}_{{k}_{f}}}\left(\right)open="("\; close=")">{A}_{i}$: for a validation flight j and for a fault amplitude ${A}_{i}$, denotes the time from the injection of the fault at $k={k}_{f}$ and the time for which the belief associated with the i-th sensor reaches for the first time the threshold.
- ${S}_{{T}_{j}}$: set of equally-spaced instants ($k={k}_{f}$) in the validation flight j when the fault (9) is injected in the i-th sensor.
- ${N}_{{T}_{j}}$: number of samples in the set ${S}_{{T}_{j}}$.

## 10. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## Abbreviations

Maha. Pr. | Mahalanobis distance-based technique with Primary residuals |

Maha. Tr. | Mahalanobis distance-based technique with Transformed residuals |

RBE Pr. | Reconstruction-based technique with Primary residuals |

RBE Tr. | Reconstruction-based technique with Transformed residuals |

## References

- Ding, S. Model-Based Fault Diagnosis Techniques: Design Schemes, Algorithms, and Tools; Springer: Berlin/Heidelberg, Germany, 2008. [Google Scholar] [CrossRef]
- Gertler, J. Analytical Redundancy Methods in Fault Detection and Isolation-Survey and Synthesis. IFAC Proc. Vol.
**1991**, 24, 9–21. [Google Scholar] [CrossRef] - Simani, S.; Fantuzzi, C.; Patton, R. Model-Based Fault Diagnosis in Dynamic Systems Using Identification Techniques; Springer: Berlin/Heidelberg, Germany, 2003. [Google Scholar] [CrossRef] [Green Version]
- Isermann, R. Process fault detection based on modeling and estimation methods—A survey. Automatica
**1984**, 20, 387–404. [Google Scholar] [CrossRef] - Basseville, M. Detecting changes in signals and systems—A survey. Automatica
**1988**, 24, 309–326. [Google Scholar] [CrossRef] - Gertler, J. Fault Detection and Diagnosis in Engineering Systems; Routledge: London, UK, 2017. [Google Scholar]
- Patton, R.J.; Frank, P.M.; Clark, R.N. Issues of Fault Diagnosis for Dynamic Systems; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2013. [Google Scholar]
- Gao, Z.; Cecati, C.; Ding, S.X. A Survey of Fault Diagnosis and Fault-Tolerant Techniques—Part I: Fault Diagnosis with Model-Based and Signal-Based Approaches. IEEE Trans. Ind. Electron.
**2015**, 62, 3757–3767. [Google Scholar] [CrossRef] [Green Version] - Blanke, M.; Kinnaert, M.; Lunze, J.; Staroswiecki, M. Diagnosis and Fault Tolerant Control; Springer: Berlin/Heidelberg, Germany, 2016. [Google Scholar]
- Ding, S. Optimal fault detection and estimation: A unified scheme and least squares solutions. IFAC-PapersOnLine
**2018**, 51, 465–472. [Google Scholar] [CrossRef] - Li, L.; Ding, S.; Peng, X. Optimal Observer-based Fault Detection and Estimation Approaches for T-S Fuzzy Systems. IEEE Trans. Fuzzy Syst.
**2020**. [Google Scholar] [CrossRef] - Martinez-Guerra, R.; Mata-Machuca, J.L. Fault Detection and Diagnosis in Nonlinear Systems; Springer: Berlin/Heidelberg, Germany, 2016. [Google Scholar]
- Sobhani-Tehrani, E.; Khorasani, K. Fault Diagnosis of Nonlinear Systems Using a Hybrid Approach; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2009; Volume 383. [Google Scholar]
- Zhang, Q.; Zhang, X.; Polycarpou, M.M.; Parisini, T. Distributed sensor fault detection and isolation for multimachine power systems. Int. J. Robust Nonlinear Control
**2014**, 24, 1403–1430. [Google Scholar] [CrossRef] - Isermann, R. Model-based fault-detection and diagnosis—Status and applications. Annu. Rev. Control
**2005**, 29, 71–85. [Google Scholar] [CrossRef] - Ding, S. Data-Driven Design of Fault Diagnosis and Fault-Tolerant Control Systems; Springer: Berlin/Heidelberg, Germany, 2014. [Google Scholar] [CrossRef]
- Yin, S.; Ding, S.X.; Haghani, A.; Hao, H.; Zhang, P. A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark Tennessee Eastman process. J. Process Control
**2012**, 22, 1567–1581. [Google Scholar] [CrossRef] - Mrugalski, M. Advanced Neural Network-Based Computational Schemes for Robust Fault Diagnosis; Springer: Berlin/Heidelberg, Germany, 2014. [Google Scholar]
- Bocaniala, C.D.; Palade, V. Computational intelligence methodologies in fault diagnosis: Review and state of the art. In Computational Intelligence in Fault Diagnosis; Springer: Berlin/Heidelberg, Germany, 2006; pp. 1–36. [Google Scholar]
- Lei, Y.; Yang, B.; Jiang, X.; Jia, F.; Li, N.; Nandi, A.K. Applications of machine learning to machine fault diagnosis: A review and roadmap. Mech. Syst. Signal Process.
**2020**, 138, 106587. [Google Scholar] [CrossRef] - López-Estrada, F.R.; Méndez López, L.; Santos-Ruiz, I.; Valencia-Palomo, G. Detección de fallas en vehículos aéreos no tripulados mediante señales de orientación y técnicas de aprendizaje de máquina. Revista Iberoamericana de Automática e Informática Industrial RIAI
**2021**. [Google Scholar] [CrossRef] - Schaefer, R. Unmanned Aerial Vehicle Reliability Study; Office of the Secretary of Defense: Washington, DC, USA, 2003. [Google Scholar]
- Goupil, P. AIRBUS state of the art and practices on FDI and FTC in flight control system. Control Eng. Pract.
**2011**, 19, 524–539. [Google Scholar] [CrossRef] - Johnson, D.M. A review of fault management techniques used in safety-critical avionic systems. Prog. Aerosp. Sci.
**1996**, 32, 415–431. [Google Scholar] [CrossRef] - Marzat, J.; Piet-Lahanier, H.; Damongeot, F.; Walter, E. Model-based fault diagnosis for aerospace systems: A survey. Proc. Inst. Mech. Eng. Part G J. Aerosp. Eng.
**2012**, 226, 1329–1360. [Google Scholar] [CrossRef] [Green Version] - Farsoni, S.; Simani, S. Robust Fault Diagnosis and Fault Tolerant Control of Wind Turbines: Data-Driven and Model-Based Approaches; Scholars’ Press: Riga, Latvia, 2016. [Google Scholar]
- Chu, E.; Gorinevsky, D.; Boyd, S. Detecting Aircraft Performance Anomalies from Cruise Flight Data; AIAA Infotech@Aerospace: Atlanta, Georgia, 2010. [Google Scholar] [CrossRef] [Green Version]
- Li, L.; Gariel, M.; Hansman, R.J.; Palacios, R. Anomaly detection in onboard-recorded flight data using cluster analysis. In Proceedings of the 2011 IEEE/AIAA 30th Digital Avionics Systems Conference, Seattle, WA, USA, 16–20 October 2011; pp. 4A4-1–4A4-11. [Google Scholar] [CrossRef]
- Dani, M.C.; Freixo, C.; Jollois, F.; Nadif, M. Unsupervised anomaly detection for Aircraft Condition Monitoring System. In Proceedings of the 2015 IEEE Aerospace Conference, Big Sky, MT, USA, 7–14 March 2015; pp. 1–7. [Google Scholar] [CrossRef]
- Li, L.; Das, S.; John Hansman, R.; Palacios, R.; Srivastava, A.N. Analysis of Flight Data Using Clustering Techniques for Detecting Abnormal Operations. J. Aerosp. Inf. Syst.
**2015**, 12, 587–598. [Google Scholar] [CrossRef] [Green Version] - Lu, P.; Van Eykeren, L.; van Kampen, E.J.; de Visser, C.; Chu, Q. Double-model adaptive fault detection and diagnosis applied to real flight data. Control Eng. Pract.
**2015**, 36, 39–57. [Google Scholar] [CrossRef] - Fravolini, M.L.; Napolitano, M.R.; Core, G.D.; Papa, U. Experimental interval models for the robust Fault Detection of Aircraft Air Data Sensors. Control Eng. Pract.
**2018**, 78, 196–212. [Google Scholar] [CrossRef] - Fravolini, M.L.; Del Core, G.; Papa, U.; Valigi, P.; Napolitano, M.R. Data-Driven Schemes for Robust Fault Detection of Air Data System Sensors. IEEE Trans. Control Syst. Technol.
**2019**, 27, 234–248. [Google Scholar] [CrossRef] - Tecnam P92 Webpage. 2020. Available online: https://www.tecnam.com/aircraft/p92-echo-mkii/ (accessed on 22 December 2020).
- Gertler, J.J.; Kunwer, M.M. Optimal residual decoupling for robust fault diagnosis. Int. J. Control
**1995**, 61, 395–421. [Google Scholar] [CrossRef] - Basseville, M. Information criteria for residual generation and fault detection and isolation. Automatica
**1997**, 33, 783–803. [Google Scholar] [CrossRef] [Green Version] - Hu, Y.; Gertler, J. Design of Directional Residuals for Optimal Testability. IFAC Proc. Vol.
**2002**, 35, 131–136. [Google Scholar] [CrossRef] [Green Version] - Alcala, C.F.; Qin, S.J. Reconstruction-based contribution for process monitoring. Automatica
**2009**, 45, 1593–1600. [Google Scholar] [CrossRef] - Frisk, E.; Nielsen, L. Robust residual generation for diagnosis including a reference model for residual behavior. Automatica
**2006**, 42, 437–445. [Google Scholar] [CrossRef] [Green Version] - Varrier, S.; Koenig, D.; Martinez, J.J. Robust fault detection for Uncertain Unknown Inputs LPV system. Control Eng. Pract.
**2014**, 22, 125–134. [Google Scholar] [CrossRef] [Green Version] - Witczak, M.; Buciakowski, M.; Puig, V.; Rotondo, D.; Nejjari, F. An LMI approach to robust fault estimation for a class of nonlinear systems. Int. J. Robust Nonlinear Control
**2016**, 26, 1530–1548. [Google Scholar] [CrossRef] [Green Version] - Balzano, F.; Fravolini, M.; Napolitano, M.; d’Urso, S.; Crispoltoni, M.; Core, G. Air Data Sensor Fault Detection with an Augmented Floating Limiter. Int. J. Aerosp. Eng.
**2018**, 2018, 1072056. [Google Scholar] [CrossRef] [Green Version] - Leondes, C.T. Techniques in Discrete and Continuous Robust Systems; Academic Press: Cambridge, MA, USA, 1996. [Google Scholar]
- Mahalanobis, P.C. On the generalized distance in statistics. Proc. Natl. Inst. Sci.
**1936**, 2, 49–55. [Google Scholar] - De Maesschalck, R.; Jouan-Rimbaud, D.; Massart, D. The Mahalanobis distance. Chemom. Intell. Lab. Syst.
**2000**, 50, 1–18. [Google Scholar] [CrossRef] - Qin, S.J. Survey on data-driven industrial process monitoring and diagnosis. Annu. Rev. Control
**2012**, 36, 220–234. [Google Scholar] [CrossRef] - Cartocci, N.; Costante, G.; Napolitano, M.R.; Valigi, P.; Crocetti, F.; Fravolini, M.L. PCA Methods and Evidence Based Filtering for Robust Aircraft Sensor Fault Diagnosis. In Proceedings of the 2020 28th Mediterranean Conference on Control and Automation (MED), Saint-Raphaël, France, 16–19 June 2020; pp. 550–555. [Google Scholar] [CrossRef]
- Yang, F.; Xiao, D. Progress in Root Cause and Fault Propagation Analysis of Large-Scale Industrial Processes. J. Control Sci. Eng.
**2012**, 2012, 478373. [Google Scholar] [CrossRef] - Massoumnia, M. A geometric approach to the synthesis of failure detection filters. IEEE Trans. Autom. Control
**1986**, 31, 839–846. [Google Scholar] [CrossRef] - Hu, Y.; Gertler, J. Design of optimal directional residuals for linear dynamic systems. IFAC Proc. Vol.
**2003**, 36, 245–250. [Google Scholar] [CrossRef] - Lou, X.C.; Willsky, A.S.; Verghese, G.C. Optimally robust redundancy relations for failure detection in uncertain systems. Automatica
**1986**, 22, 333–344. [Google Scholar] [CrossRef] [Green Version] - Särkkä, S. Bayesian Filtering and Smoothing; Cambridge University Press: Cambridge, MA, USA, 2013; Volume 3. [Google Scholar]
- Bishop, C.M. Pattern Recognition and Machine Learning; Springer: Berlin/Heidelberg, Germany, 2006. [Google Scholar]
- Ge, Z.; Zhang, M.; Song, Z. Nonlinear process monitoring based on linear subspace and Bayesian inference. J. Process Control
**2010**, 20, 676–688. [Google Scholar] [CrossRef] - Zheng, Y.; Mao, S.; Liu, S.; Wong, D.S.; Wang, Y. Normalized Relative RBC-Based Minimum Risk Bayesian Decision Approach for Fault Diagnosis of Industrial Process. IEEE Trans. Ind. Electron.
**2016**, 63, 7723–7732. [Google Scholar] [CrossRef] - Zhu, J.; Ge, Z.; Song, Z. Distributed Parallel PCA for Modeling and Monitoring of Large-Scale Plant-Wide Processes with Big Data. IEEE Trans. Ind. Inform.
**2017**, 13, 1877–1885. [Google Scholar] [CrossRef] - Zhou, W.; Yang, W.; Wang, Y.; Zhang, H. Generalized Reconstruction-Based Contribution for Multiple Faults Diagnosis with Bayesian Decision. In Proceedings of the 2018 IEEE 7th Data Driven Control and Learning Systems Conference (DDCLS), Enshi, China, 25–27 May 2018; pp. 813–818. [Google Scholar] [CrossRef]
- Basseville, M.; Nikiforov, I.V. Detection of Abrupt Changes—Theory and Application; Prentice Hall, Inc.: Upper Saddle River, NJ, USA, 1993; p. 550. [Google Scholar]
- Chandola, V.; Banerjee, A.; Kumar, V. Anomaly Detection: A Survey. ACM Comput. Surv.
**2009**, 41. [Google Scholar] [CrossRef] - Draper, N.; Smith, H. Applied Regression Analysis, 3rd ed.; A Wiley-Interscience Publication; Wiley: New York, NY, USA, 1998. [Google Scholar]
- The Mathworks Inc. MATLAB—MathWorks; The Mathworks Inc.: Natick, MA, USA, 2020. [Google Scholar]

**Figure 2.**RMSE as function of the number of regressors in the model for the $\alpha \left(k\right)$, $\beta \left(k\right)$, and $TaS\left(k\right)$ sensors. Training (Tr) and Validation (Va) data.

**Figure 3.**Comparison of the RMSE achieved in training and validation for the 14 linear regression models.

**Figure 4.**Fault isolation percentage for the $\alpha \left(k\right)$ sensor evaluated on the validation flights as a function of the fault amplitude.

**Figure 5.**Fault isolation percentage for the $\beta \left(k\right)$ sensor evaluated on the validation flights as a function of fault amplitude.

**Figure 6.**Fault isolation percentage for the $TaS\left(k\right)$ sensor evaluated on the validation flights as a function of fault amplitude.

**Figure 7.**Relative fault reconstruction error for the $\alpha \left(k\right)$ sensor evaluated on the validation flights as a function of fault amplitude.

**Figure 8.**Relative fault reconstruction error for the $\beta \left(k\right)$ sensor evaluated on the validation flights as a function of fault amplitude.

**Figure 9.**Relative fault reconstruction error for the $TaS\left(k\right)$ sensor evaluated on the validation flights as a function of fault amplitude.

**Figure 10.**Evolution of the percent fault belief $p\left[\alpha \left(k\right)\right|{e}_{i}\left(k\right)]$ following a failure on the $\alpha \left(k\right)$ sensor, for different fault amplitudes (RBE-Pr. method).

**Figure 11.**Evolution of the percent fault belief $p\left[\beta \left(k\right)\right|{e}_{i}\left(k\right)]$ following a failure on the $\beta \left(k\right)$ sensor, for different fault amplitudes (RBE-Pr. method).

**Figure 12.**Evolution of the percent fault belief $p\left[TaS\left(k\right)\right|{e}_{i}\left(k\right)]$ following a failure on the $TaS\left(k\right)$ sensor, for different fault amplitudes (RBE-Pr. method).

**Figure 13.**Fault isolation delay for faults on the $\alpha \left(k\right)$ evaluated on the validation flights as a function of fault amplitude.

**Figure 14.**Fault isolation delay for faults on the $\beta \left(k\right)$ evaluated on the validation flights as a function of fault amplitude.

**Figure 15.**Fault isolation delay for faults on the $TaS\left(k\right)$ evaluated on the validation flights as a function of fault amplitude.

Maha. Pr. | Maha. Tr. | RBEPr. | RBE Tr. | |
---|---|---|---|---|

Computational complexity | $O\left({n}^{3}\right)$ | $O\left({n}^{2}\right)$ | $O\left({n}^{2}\right)$ | $O\left({n}^{2}\right)$ |

Memory space | $6{n}_{x}$ | $6{n}_{x}$ | $3{n}_{x}$ | $3{n}_{x}$ |

${\mathit{x}}_{0}$ | ${\mathit{u}}_{0}$ | ||||
---|---|---|---|---|---|

$\alpha $ | Angle of attack | P | Roll speed | Ap | Aileron position |

$\beta $ | Drifting angle | Q | Pitch speed | Rp | Rudder position |

TaS | True air speed | R | Yaw speed | Tp | Thrust lever position |

$\varphi $ | Roll angle | NNx | Longitudinal load factor | Pp | Pitch trim position |

$\theta $ | Pitch angle | NNy | Lateral load factor | Sp | Stabilator position |

$\psi $ | Yaw angle | NNz | Vertical load factor | Eng | Engine revolution |

ZLaser | Altitude laser | Zpos | GPS altitude |

**Table 3.**Selected regressors by the stepwise method for the $\alpha $, $\beta $, and $TaS$ models. TAS, True Air Speed.

Sensor | Selected Regressors (by the Stepwise Method) | ||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

$\alpha $ | $\beta $ | TaS | NNx | NNy | NNz | P | Q | R | ZLaser | Z | Ap | Rp | Tp | Pp | Sp | Eng | |

$\beta $ | $\alpha $ | TaS | NNx | NNy | NNz | P | R | $\varphi $ | ZLaser | Ap | Tp | ||||||

TaS | $\alpha $ | $\beta $ | NNx | NNy | NNz | P | Q | R | $\varphi $ | $\psi $ | Zpos | ZLaser | Ap | Tp | Pp | Sp | Eng |

$\mathit{\alpha}$ | $\mathit{\beta}$ | TAS | NNx | NNy | NNz | P | Q | R | $\mathit{\varphi}$ | $\mathit{\theta}$ | $\mathit{\psi}$ | Zpos | ZLaser |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|

16 | 11 | 17 | 16 | 12 | 18 | 14 | 16 | 8 | 9 | 8 | 11 | 15 | 10 |

$\mathit{\alpha}\left(\mathit{k}\right)$ (deg) | $\mathit{\beta}\left(\mathit{k}\right)$ (deg) | $\mathit{TaS}\left(\mathit{k}\right)$ (m/s) | |
---|---|---|---|

${A}_{M}$ | 2 | 5 | 3 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Cartocci, N.; Napolitano, M.R.; Costante, G.; Fravolini, M.L.
A Comprehensive Case Study of Data-Driven Methods for Robust Aircraft Sensor Fault Isolation. *Sensors* **2021**, *21*, 1645.
https://doi.org/10.3390/s21051645

**AMA Style**

Cartocci N, Napolitano MR, Costante G, Fravolini ML.
A Comprehensive Case Study of Data-Driven Methods for Robust Aircraft Sensor Fault Isolation. *Sensors*. 2021; 21(5):1645.
https://doi.org/10.3390/s21051645

**Chicago/Turabian Style**

Cartocci, Nicholas, Marcello R. Napolitano, Gabriele Costante, and Mario L. Fravolini.
2021. "A Comprehensive Case Study of Data-Driven Methods for Robust Aircraft Sensor Fault Isolation" *Sensors* 21, no. 5: 1645.
https://doi.org/10.3390/s21051645