# Performance Assessment for a Guided Wave-Based SHM System Applied to a Stiffened Composite Structure

^{1}

^{2}

^{3}

^{4}

^{5}

^{6}

^{7}

^{8}

^{9}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Need for Performance Assessment in SHM

#### 2.1. General Remarks on Performance Assessment

- 1
- Define the defect type(s) under study;
- 2
- Define the inspection conditions;
- 3
- Have a sample containing enough realistic defects;
- 4
- Have a suitable technique for validation;
- 5
- Know all relevant defect properties.

#### 2.2. Factors Influencing Performance Assessment

**Intrinsic factors**are inherent attributes of the combined structure-SHM system. Examples of intrinsic factors are the size of damage, level of information needed from the SHM system and the physics of the SHM method, here, guided waves. Intrinsic factors are widely recognized to be fundamental for the SHM system output and its reliability.

**Algorithms**have a major influence on the SHM performance as well. Every decision to develop an SHM methodology influences the end result, starting from the parameters for data acquisition (frequency, mode selection), continuing with the data pre-processing (filtering) and finalizing with the selected damage identification algorithms. With the integration of the monitoring system into the structure, intrinsic factors and algorithms become highly linked.

**Application factors**are influences specific to each application. Examples of application factors are environmental and operational conditions or available information regarding a reference state. Compared to the lab setting, where all application factors can be controlled, the monitoring ability of SHM systems in industrial applications is decreased by the application factors in most cases. It is therefore essential to include these factors in quality assessment strategies.

**Integration factors**are responsible for some of the most significant challenges of performance assessment in SHM and represent major differences between ultrasonic NDE and GW-based SHM. Having a fixed SHM system, the monitoring ability changes significantly depending on sensor positions and damage locations. It is therefore essential to include these factors when evaluating the quality of an SHM system, while at the same time, this is a major challenge for all GW-based systems used for area monitoring, in contrast to hot-spot monitoring.

## 3. Theory of Probability of Detection

- Hit/miss analysis, relying on the classic probabilistic approach where binary data are available, i.e., whether or not a flaw is found;
- $\hat{\mathit{a}}$ vs. a analysis, based on a mathematical derivation from existing correlation between signal response ($\widehat{a}$) and defect size (a), if available.

- Define a decision approach, including how the hit is achieved;
- Define a model function $f\left(a\right)=\mathrm{POD}\left(a\right)$;
- Estimate the model parameters.

- Define a decision level;
- Estimate the regression model best fitting the signal response versus the flaw dimension;
- Select an appropriate approach for obtaining confidence bounds;
- Establish a POD function model.

## 4. SHM System

#### 4.1. Brief Description of the Data Obtained from the Open Guided Waves Platform

^{2}and a 2 mm thickness. An omega stringer of 1.5 mm thickness is bonded at the center of the plate. Guided waves are sent and received by 12 piezoelectric transducers distributed in 2 rows parallel to the omega stringer. Refs. [4,40] present the data acquisition, detailed information on the manufacturing of the plate and the omega stringer, and damage scenarios.

^{2}up to 2099.3 mm

^{2}, and 13 damage sized were used. Additionally, the reference damage was de-attached and re-attached five times. This process of de-attaching and re-attaching the reference damage was performed to obtain five measurements as statistically independent as possible, resulting in a populated family of noise data. The specimen geometry, transducer positions and damage locations are sketched in Figure 6. Instead, the experimental setup is depicted in Figure 7.

#### 4.2. Damage Identification Procedure

## 5. Results

#### 5.1. Path-Dependent POD Analysis

#### 5.2. Threshold Dependency

#### 5.3. Damage Path Distance Dependency

#### 5.4. Dependency on Geometrical Placement of the Paths

#### 5.4.1. Effect of Distance Path-Damage

#### 5.4.2. Effect of Damage Orientation

#### 5.5. Discussion of Non-Sensitive Paths

#### 5.6. Numerical Analysis

^{2}at position D1 was modeled. The damage is to be understood as a material application made of structural steel. In all cases, a tone burst with a center frequency of 40 kHz and a voltage of 35 Vpp was used as the excitation signal.

^{2}. In both cases, the averaged signals calculated from the 5 available measurements per path and per damage have been determined (see Section 4). Moreover, the signals were band-pass filtered between $30\phantom{\rule{3.33333pt}{0ex}}$ kHz and $50\phantom{\rule{3.33333pt}{0ex}}$ kHz. It can be seen that the differential signal for path T4–T7 is generally stronger than for path T2–T11. Furthermore, the differential signal for the path T4–T7 shows the additional signal component at around $500\phantom{\rule{3.33333pt}{0ex}}\mu s$, which results from the reflection at the left edge of the plate. This has already been explained in the description of the wave fields. This signal part coming from the reflection explains the higher damage indicator for path T4–T7 compared to path T2–T11 and thus finally the higher sensitivity of path T4–T7.

## 6. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Mitra, M.; Gopalakrishnan, S. Guided Wave Based Structural Health Monitoring: A Review. Smart Mater. Struct.
**2016**, 25, 053001. [Google Scholar] [CrossRef] - Olisa, S.C.; Khan, M.A.; Starr, A. Review of Current Guided Wave Ultrasonic Testing (GWUT) Limitations and Future Directions. Sensors
**2021**, 21, 811. [Google Scholar] [CrossRef] [PubMed] - Cawley, P. Structural health monitoring: Closing the gap between research and industrial deployment. Struct. Health Monit.
**2018**, 17, 1225–1244. [Google Scholar] [CrossRef] [Green Version] - Moll, J.; Kexel, C.; Kathol, J.; Fritzen, C.P.; Moix-Bonet, M.; Willberg, C.; Rennoch, M.; Koerdt, M.; Herrmann, A. Guided Waves for Damage Detection in Complex Composite Structures: The Influence of Omega Stringer and Different Reference Damage Size. Appl. Sci.
**2020**, 10, 3068. [Google Scholar] [CrossRef] - Department of Defense. Department of Defense Handbook Nondestructive Evaluation System Reliability Assessment; MIL-HDBK-1823A; Department of Defense: Washington, DC, USA, 2009. [Google Scholar]
- Pfeiffer, H.; Wevers, M. Pseudo-Defects for the Validation and Tuning of Structural Health Monitoring in Plate-like Structures using Lamb Waves. In Proceedings of the EU Project Meeting on Aircraft Integratied Strucutral Health Assessment, Leuven, Belgium, 1 June 2007. [Google Scholar]
- Papatheou, E.; Manson, G.; Barthorpe, R.J.; Worden, K. The use of pseudo-faults for novelty detection in SHM. J. Sound Vib.
**2010**, 329, 2349–2366. [Google Scholar] [CrossRef] - Tschöke, K.; Gaul, T.; Schubert, L.; Mueller, I. Rechnergestützte POD-Bestimmung für SHM-Verfahren basierend auf geführten Wellen im Automobilbereich. In Proceedings of the DACH Jahrestagung, Friedrichshafen, Germany, 27–29 May 2019. [Google Scholar]
- Berthold, J.; Vrana, J.; Heckel, T.; Kanzler, D.; Jatzlau, P.; Grosse, C. Modellgestützte Bestimmung der Fehlerauffindwahrscheinlichkeit (POD) bei der Ultraschallprüfung großer Schmiedeteile. In Proceedings of the DGZfP DACH-Jahrestagung, Friedrichshafen, Germany, 27–29 May 2019. [Google Scholar]
- Cobb, A.C.; Michaels, J.E.; Michaels, T.E.; Thompson, D.O.; Chimenti, D.E. Ultrasonic structural health monitoring: A probability of detection case study. In AIP Conference Proceedings; American Institute of Physics: College Park, MD, USA, 2009. [Google Scholar] [CrossRef] [Green Version]
- Janapati, V.; Kopsaftopoulos, F.; Li, F.; Lee, S.J.; Chang, F.K. Damage detection sensitivity characterization of acousto-ultrasound-based structural health monitoring techniques. Struct. Health Monit. An Int. J.
**2016**, 15, 143–161. [Google Scholar] [CrossRef] - Aldrin, J.C.; Annis, C.; Sabbagh, H.A.; Lindgren, E.A. Best practices for evaluating the capability of nondestructive evaluation (NDE) and structural health monitoring (SHM) techniques for damage characterization. Aip Conf. Proc.
**2016**, 1706, 200002. [Google Scholar] [CrossRef] [Green Version] - Meeker, W.Q.; Roach, D.; Kessler, S.S. Statistical Methods for Probability od Detection in Structural Health Monitoring. In Proceedings of the International Workshop on Structural Health Monitoring IWSHM, Stanford, CA, USA, 10–12 September 2019. [Google Scholar]
- Kessler, S.S.; Dunn, C.T.; Swindell, P.; Meeker, W. Detection Sensitivtiy Analysis for a Guided Wave (GW) Structural Health Monitoring System. In Proceedings of the International Workshop on Structural Health Monitoring IWSHM, Stanford, CA, USA, 10–12 September 2019. [Google Scholar]
- Boller, C.; Mahapatra, D.R.; Venkat, R.S.; Ravi, N.B.; Chakraborty, N.; Shivamurthy, R.; Simon, K.M. Integration of Non-Destructive Evaluation-based Ultrasonic Simulation: A means for simulation in structural health monitoring. Struct. Health Monit. Int. J.
**2017**, 16, 611–629. [Google Scholar] [CrossRef] - Buethe, I.; Dominguez, N.; Jung, H.; Fritzen, C.P.; Ségur, D.; Reverdy, F. Path-based MAPOD using Numerical Simulations. In Smart Intelligent Aircraft Structures (SARISTU); Woelcken, P., Papadopoulos, M., Eds.; Springer: Berlin/Heidelberg, Germany, 2015; pp. 631–642. [Google Scholar]
- Memmolo, V.; Ricci, F.; Maio, L.; Boffa, N.D.; Monaco, E. Model assisted probability of detection for a guided waves based SHM technique. In Health Monitoring of Structural and Biological Systems 2016; Kundu, T., Ed.; SPIE: Bellingham, WA, USA, 2016. [Google Scholar] [CrossRef]
- Schubert Kabban, C.; Uber, R.; Lin, K.; Lin, B.; Bhuiyan, M.; Giurgiutiu, V. Uncertainty Evaluation in the Design of Structural Health Monitoring Systems for Damage Detection. Aerospace
**2018**, 5, 45. [Google Scholar] [CrossRef] [Green Version] - Tschöke, K.; Gravenkamp, H. On the numerical convergence and performance of different spatial discretization techniques for transient elastodynamic wave propagation problems. Wave Motion
**2018**, 82, 62–85. [Google Scholar] [CrossRef] - Flynn, E.B.; Todd, M.D. A Bayesian approach to optimal sensor placement for structural health monitoring with application to active sensing. Mech. Syst. Signal Process.
**2010**, 24, 891–903. [Google Scholar] [CrossRef] - Falcetelli, F.; Yue, N.; Sante, R.D.; Zarouchas, D. Probability of detection, localization, and sizing: The evolution of reliability metrics in Structural Health Monitoring. Struct. Health Monit.
**2021**, 14759217211060780. [Google Scholar] [CrossRef] - Yue, N.; Aliabadi, M. Hierarchical approach for uncertainty quantification and reliability assessment of guided wave-based structural health monitoring. Struct. Health Monit.
**2020**, 1475921720940642. [Google Scholar] [CrossRef] - Bayoumi, A.; Minten, T.; Mueller, I. Determination of Detection Probability and Localization Accuracy for a Guided Wave-Based Structural Health Monitoring System on a Composite Structure. Appl. Mech.
**2021**, 2, 996–1008. [Google Scholar] [CrossRef] - Moriot, J.; Quaegebeur, N.; Duff, A.L.; Masson, P. A model-based approach for statistical assessment of detection and localization performance of guided wave–based imaging techniques. Struct. Health Monit.
**2018**, 17, 1460–1472. [Google Scholar] [CrossRef] [Green Version] - Mueller, I.; Freitag, S.; Memmolo, V.; Moix-Bonet, M.; Möllenhoff, K.; Golub, M.; Venkat, R.S.; Lugovtsova, Y.; Eremin, A.; Moll, J.; et al. Performance Assessment for Artificial Intelligence-Based Data Analysis in Ultrasonic Guided Wave-Based Inspection: A Comparison to Classic Path-Based Probability of Detection. In Proceedings of the European Workshop on Structural Health Monitoring, Palermo, Italy, 4–7 July 2022; Rizzo, P., Milazzo, A., Eds.; Springer International Publishing: Cham, Switzerland, 2022; pp. 953–961. [Google Scholar]
- Hong, M.; Mao, Z.; Todd, M.D.; Su, Z. Uncertainty quantification for acoustic nonlinearity parameter in Lamb wave-based prediction of barely visible impact damage in composites. Mech. Syst. Signal Process.
**2017**, 82, 448–460. [Google Scholar] [CrossRef] [Green Version] - Liu, C.; Dobson, J.; Cawley, P. Efficient generation of receiver operating characteristics for the evaluation of damage detection in practical structural health monitoring applications. Proc. R. Soc. A Math. Phys. Eng. Sci.
**2017**, 473, 20160736. [Google Scholar] [CrossRef] [Green Version] - Gianneo, A.; Carboni, M.; Giglio, M. Feasibility study of a multi-parameter probability of detection formulation for a Lamb waves–based structural health monitoring approach to light alloy aeronautical plates. Struct. Health Monit.
**2017**, 16, 225–249. [Google Scholar] [CrossRef] - Tschöke, K.; Mueller, I.; Memmolo, V.; Venkat, R.S.; Golub, M.; Eremin, A.; Moix-Bonet, M.; Möllenhoff, K.; Lugovtsova, Y.; Moll, J.; et al. A Model-Assisted Case Study Using Data from Open Guided Waves to Evaluate the Performance of Guided Wave-Based Structural Health Monitoring Systems. In Proceedings of the European Workshop on Structural Health Monitoring, Palermo, Italy, 4–7 July 2022; Rizzo, P., Milazzo, A., Eds.; Springer International Publishing: Cham, Switzerland, 2022; pp. 937–944. [Google Scholar]
- Rytter, A. Vibrational based inspection of civil engineering structures. Ph.D. Thesis, Department of Building Technology and Structural Engineering, Aalborg University, Aalborg, Denmark, 1993. [Google Scholar]
- Haynes, C.; Todd, M.D.; Flynn, E.; Croxford, A. Statistically-based damage detection in geometrically-complex structures using ultrasonic interrogation. Struct. Health Monit.
**2013**, 12, 141–152. [Google Scholar] [CrossRef] - Sturges, D.J.; Gilmore, R.S.; Hovey, P.W. Estimating Probability of Detection for Subsurface Ultrasonic Inspection. In Review of Progress in Quantitative Nondestructive Evaluation; Springer Science+Business Media: New York, NY, USA, 1986; pp. 929–937. [Google Scholar] [CrossRef] [Green Version]
- Rummel, W.D. Nondestructive Inspection Reliability—History, Status and Future Path. In Proceedings of the 18th World Conference on Non-Destructive Testing, Durban, South Africa, 16–20 April 2012. [Google Scholar]
- Takahashi, K.; Pavlovic, M.; Bertovic, M.; Ewert, U.; Mueller, C.; Ronneteg, U.; Thompson, D.O.; Chimenti, D.E. Application of POD to complex industrial problems: New approaches. Aip Conf. Proc.
**2009**, 1096, 1824–1831. [Google Scholar] [CrossRef] - Rothbart, N.; Maierhofer, C.; Goldammer, M.; Hohlstein, F.; Koch, J.; Kryukov, I.; Mahler, G.; Stotter, B.; Walle, G.; Oswald-Tranta, B.; et al. Probability of detection analysis of round robin test results performed by flash thermography. Quant. Infrared Thermogr. J.
**2016**, 14, 1–23. [Google Scholar] [CrossRef] - Pavlovic, M.; Zoëga, A.; Zanotelli, C.; Kurz, J.H. Investigations to introduce the probability of detection method for ultrasonic inspection of hollow axles at Deutsche Bahn. Procedia Struct. Integr.
**2017**, 4, 79–86. [Google Scholar] [CrossRef] - Bertovic, M. Human Factors in Non-Destructive Testing (NDT): Risks and Challenges of Mechanised NDT. Doctoral Thesis, Technische Universität Berlin, Fakultät V—Verkehrs- und Maschinensysteme, Berlin, Germany, 2015. [Google Scholar] [CrossRef]
- Berens, A. NDE Reliability Data Analysis. In ASM Handbook: Nondestructive Evaluation and Quality Control, 3rd ed.; ASM Handbook; ASM International: Materials Park, OH, USA, 1993. [Google Scholar]
- Memmolo, V.; Boffa, N.; Maio, L.; Monaco, E.; Ricci, F. Damage localization in composite structures using a guided waves based multi-parameter approach. Aerospace
**2018**, 5, 111. [Google Scholar] [CrossRef] [Green Version] - Moll, J.; Kathol, J.; Fritzen, C.P.; Moix-Bonet, M.; Rennoch, M.; Koerdt, M.; Herrmann, A.S.; Sause, M.G.; Bach, M. Open Guided Waves: Online platform for ultrasonic guided wave measurements. Struct. Health Monit.
**2019**, 18, 1903–1914. [Google Scholar] [CrossRef] - Bach, M.; Pouilly, A.; Eckstein, B.; Moix-Bonet, M. Reference Damages for Verification of Probability of Detection with Guided Waves. In Proceedings of the Structural Health Monitoring 2017, Brisbane, Australia, 5–8 December 2017; DEStech Publications, Inc.: Lancaster, PA, USA, 2017. [Google Scholar] [CrossRef]
- Charnes, A.; Frome, E.L.; Yu, P.L. The Equivalence of Generalized Least Squares and Maximum Likelihood Estimates in the Exponential Family. J. Am. Stat. Assoc.
**1976**, 71, 169–171. [Google Scholar] [CrossRef]

**Figure 2.**Probability of detection curve versus flaw size, calculated using the hit/miss procedure with the function model and estimation of coefficients.

**Figure 3.**$\widehat{a}$ vs. a procedure with the regression model and the Gaussian distribution around the predicted value (illustrative data).

**Figure 4.**Probability of detection curve versus flaw size, calculated using the $\widehat{a}$ vs a procedure with the definition of Berens and integral formulation (illustrative data).

**Figure 6.**Specimen geometry with the three damage positions D1–D3 relative to the transducer locations T1–T12 ([4]). Considering path T3–T9, ${d}_{3}$ represents the distance from damage D3, while ${\theta}_{3}$ is the angle of the transducer pair path against the damage orientation, which is at 45${}^{\circ}$ respect to x-axis.

**Figure 7.**Illustration of the experimental setup arranged in a climatic chamber for the measurement campaign. The plate is instrumented with several transducers and damaged by artificial defect.

**Figure 8.**$\widehat{a}$ vs. a regression analysis. Data samples related to damage D1 and paths T3–T9 and T4–T7. The corresponding linear trend is established using $lin-lin$ (

**a**) and $log-log$ (

**b**) models.

**Figure 9.**Noise and signal response samples along the path T3–T9 and damage D1. The regression and the signal response distribution are established using $log-log$ model. The a-value corresponding to noise samples is zero (no-damage) and is moved along x-axis for a better visualization.

**Figure 10.**POD versus flaw size related to damage D1 and paths T3–T9 and T4–T7. Regression model used to estimate $\widehat{a}$ is according to Figure 8b.

**Figure 11.**Histogram for damage index values for different paths, (

**a**) T1–T7, (

**b**) T7–T11, and (

**c**) all paths, as well as those that do not cross the stringer. (

**d**) Empirical CDF of damage index values given for all paths.

**Figure 12.**Resulting threshold values ${a}_{dec}$ based on empirical cumulative distribution of damage index values given for the cumulative empirical cdf of all paths and cdfs of exemplary paths.The fields are colored according to their value; small values are colored green, and high values are colored red.

**Figure 13.**POD curves for all non-horizontal paths, which all cross the stringer over the distance. The right figure shows a detail of the left zooming in to a-values from 0 mm${}^{2}$ to 500 mm${}^{2}$.

**Figure 14.**${a}_{90|95}$ versus path absolute distance from the flaw as a 2D representation of Figure 13.

**Figure 15.**${a}_{90|95}$ values versus interrogation path. The $\mathrm{POD}\left(a\right)$ is predicted along all vertical paths. Damage D1 (

**a**), D2 (

**b**), and D3 (

**c**).

**Figure 16.**${a}_{90|95}$ versus path distance from the flaw. The $\mathrm{POD}\left(a\right)$ is predicted along all paths crossing the stringer.

**Figure 17.**Interpolating surface of ${a}_{90|95}$ values versus path-damage distance and incidence angle ($\theta $) from the flaw. The $\mathrm{POD}\left(a\right)$ is predicted along all through the paths crossing the stringer.

**Figure 18.**Visualisation of the ${a}_{90|95}$ values along different paths. A circle ∘ marks the damage position; the crosses × mark transducers. The evaluation was done for $D{I}_{Energy}$ at $f=40\phantom{\rule{3.33333pt}{0ex}}$ kHz and ${a}_{dec}=0.01$.

**Figure 19.**Exemplary representation of the wave propagation in the test specimen. The wave field of the differential signal (undamaged–damaged) is shown for a damage of size 671 mm

^{2}at position D1 and the actuator-sensor path T4–T7. The time steps $150\mathsf{\mu}$s, $250\mathsf{\mu}$s and $350\mathsf{\mu}$s were selected.

**Figure 20.**Exemplary representation of the wave propagation in the test specimen. The wave field of the differential signal (undamaged–damaged) is shown for a damage of size 671 mm

^{2}at position D1 and the actuator-sensor path T2–T11. The time steps $150\mathsf{\mu}$s, $250\mathsf{\mu}$s and $350\mathsf{\mu}$s were selected.

**Figure 21.**Exemplary differential signals of the OGW data set for a damage of size 671 mm

^{2}at position D1 and the mentioned actuator-sensor paths.

Factors | Examples |
---|---|

Intrinsic factors | Physics, SHM level, structure (geometry, material, damage types, critical damage size) |

Algorithms | Damage indices, filtering, conventional signal processing or artificial intelligence |

Application factors | Baseline, compensation techniques for operational and environmental conditions |

Integration factors | Location of damage, location (including integration) of actuators/sensors |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Mueller, I.; Memmolo, V.; Tschöke, K.; Moix-Bonet, M.; Möllenhoff, K.; Golub, M.; Venkat, R.S.; Lugovtsova, Y.; Eremin, A.; Moll, J.
Performance Assessment for a Guided Wave-Based SHM System Applied to a Stiffened Composite Structure. *Sensors* **2022**, *22*, 7529.
https://doi.org/10.3390/s22197529

**AMA Style**

Mueller I, Memmolo V, Tschöke K, Moix-Bonet M, Möllenhoff K, Golub M, Venkat RS, Lugovtsova Y, Eremin A, Moll J.
Performance Assessment for a Guided Wave-Based SHM System Applied to a Stiffened Composite Structure. *Sensors*. 2022; 22(19):7529.
https://doi.org/10.3390/s22197529

**Chicago/Turabian Style**

Mueller, Inka, Vittorio Memmolo, Kilian Tschöke, Maria Moix-Bonet, Kathrin Möllenhoff, Mikhail Golub, Ramanan Sridaran Venkat, Yevgeniya Lugovtsova, Artem Eremin, and Jochen Moll.
2022. "Performance Assessment for a Guided Wave-Based SHM System Applied to a Stiffened Composite Structure" *Sensors* 22, no. 19: 7529.
https://doi.org/10.3390/s22197529