# Towards Interpretable Machine Learning for Automated Damage Detection Based on Ultrasonic Guided Waves

^{1}

^{2}

^{3}

^{4}

^{5}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Machine Learning Approach

#### 2.1. Description of the Experimental Setup

_{1}–T

_{12}) were attached to a carbon fibre-reinforced plastic (CFRP) plate, as well as a sequentially added detachable mass (aluminium disc) at four different locations (D

_{04}, D

_{12}, D

_{16}, D

_{24}) to simulate structural damages. The impact of the simulated damages on the measurements can be considered a rough approximation of real delamination (e.g., decrease in amplitude and changes in time of flight) [12]. The exact positions of the transducers and the damage locations as well as their distance to the direct signal path (T

_{4}to T

_{9}) can be found in Table 1. Note that, in the scope of this manuscript, the term “simulated damage” denotes an experimental simulation of a damaged material and does not refer to numerical simulation.

_{4}and received by T

_{9}for all four damage locations D

_{04}, D

_{12}, D

_{16}, and D

_{24}as well the undamaged structure. Each measurement contains only one simulated damage at a time. During the experiment, the plate was subjected to several temperature cycles between 20 and 60 °C in a climatic chamber (Figure 1c) at constant humidity (50% RH, mean: ~50.1%, standard deviation ~0.3%). For studies concerning the impact of humidity on CFRP the reader is referred to Schubert et al. [13]. Note that measurements for the undamaged plate were performed on two temperature cycles instead of only one. For the pre-processing (Section 2.2) the ascending flank (20 °C to 60 °C in 0.5 °C steps) of the first temperature cycle of the undamaged plate was used as a database (DB, Figure 1c) for the optimal baseline selection (OBS) of reference signals (cf. Section 2.2), and the descending flank is labelled “undamaged group 1” (UG

_{1}). The second temperature cycle (ascending and descending flank) is labelled “undamaged group 2” (UG

_{2}). These two different groups are later used in the validation (Section 2.4).

_{4}and T

_{9}) and the other where they were located at the edge (T

_{1}and T

_{7}; Section 3.3). In the scope of this study, we focused on one transducer combination at a time to be able to interpret the ML results more easily and, more importantly, to reduce the complexity and cost of later SHM configurations. Although the performance could be increased by using the information of all sensors, the aim of this study was to gain a better understanding of which configuration is necessary to reliably detect a damaged structure.

#### 2.2. Signal Pre-Processing

- The baseline signal is stretched on the time axis to best fit the measured signal, again as determined by the RMSE.
- The stretched baseline is shifted on the time axis to achieve the best fit to the measured signal in terms of RMSE.
- The shifted baseline’s amplitude is scaled to match the measured signal in terms of RMSE.

#### 2.3. Automated Toolbox

#### 2.4. Validation Scenario

_{04}, D

_{12}, D

_{16}, D

_{24}) are included in each training set. Stratified CV cannot guarantee that the model learns general characteristics of a damaged or undamaged structure instead of only damage-specific and position-related characteristics, which only occur at the locations of the trained damages. This may result in overfitting, meaning that the ML model is trained only for specific damage locations and is then unable to identify damages at other locations. Therefore, 10-fold cross-validation is replaced by leave-one-group-out cross-validation (LOGOCV; Figure 3, right). To do so, the dataset is divided into data subsets with respect to the corresponding groups (UG

_{1}, UG

_{2}, D

_{04}, D

_{12}, D

_{16}, D

_{24}), allowing for the exclusion of each damage location from the training data once and thus making this damage location completely unknown to the ML model. The excluded group is then used to validate the performance of the trained model. To ensure that the training dataset always contains data of the undamaged sample, these measurements are split into two groups (UG

_{1,}UG

_{2}).

#### 2.5. Hyper-Parameter Selection

## 3. Results and Discussion

#### 3.1. Principle Component Analysis

_{12}and D

_{16}located in the direct signal path between T4 and T9, where waves reflected from and transmitted through the damage (resulting in decreased amplitudes) had a higher impact on the measurements. Since D

_{04}and D

_{24}were not in the direct signal path, their influence on the received signal was smaller. D

_{04}, D

_{24}, and the undamaged data formed a cluster in the centre. In addition, Figure 5b shows all pre-processed measurements coloured by the corresponding temperature. Thus, the crescent-moon shape of the signals for D

_{12}and D

_{16}was mainly due to the temperature effect, which was not fully compensated by the OBS + BSS pre-processing. Figure 5b implies that measurements of D

_{12}and D

_{16}at higher temperatures were more difficult to discriminate, as they lay closer to each other as well as to the cluster of the undamaged plate and damages D

_{04}and D

_{24}.

_{04}and D

_{24}overlapped with the undamaged data UG

_{1}and UG

_{2}in the first five PCs, which explains 72% of the variance.

#### 3.2. Results of the Automated Toolbox and Improvement of the Algorithms

_{24}, which is the location farthest from the direct path in this study (186 mm; Table 1), in combination with high temperatures (>45 °C).

#### 3.3. Influence of the Distance between Damage Location and Signal Path

_{24}, which required a considerable extrapolation since this damage location was furthest from the signal path (186 mm; Table 1), which is believed to have had a significant influence on the ML performance, especially at higher temperatures. Therefore, we performed an additional investigation of the combination of transducers 1 and 7 (Table 7), where D

_{24}lay in the direct signal path. Table 8 shows the distances of each damage location from the direct signal path for this transducer combination.

_{24}and D

_{16}were close to the signal path; thus, they were classified correctly, whereas the accuracy dropped with increasing distance between damage location and signal path. The reduced accuracies for the undamaged cases (UG

_{1}, UG

_{2}) were possibly due to features present in the damage cases being similar to features of the undamaged case; however, this needs to be investigated further.

#### 3.4. Robustness against Temperature Influences

_{1}, D

_{12}, and D

_{24}were used for training, and data from D

_{04}and the rising temperature flank of UG

_{2}for validation. The extended temperature range of these data plus the respective data from D

_{16}and the descending flank of UG

_{2}were used for testing, as shown in Figure 8a,b for 2 °C and 16 °C extrapolation, respectively.

#### 3.5. Comparison to a State-of-the-Art Neural Network

_{4}to T

_{10}with a high-pass filter (Butterworth), down sampling (factor 6), and BSS (undamaged plate at 40 °C) as pre-processing. A more detailed description as well as the architecture of the causal dilated CNN can be found in the original paper [9].

_{4}and T

_{10}for model building and replicated the grouping of Mariani et al. for training, validation, and testing data. Thus, training data contained D

_{16}, D

_{24}, and 50% of UG

_{2}; validation data contained D

_{12}and 25% of UG

_{2}; and testing data contain D

_{04}and 25% of UG

_{2}. The split of UG

_{2}into the corresponding groups was based on a training–validation–training–testing pattern with a 1.5 °C step size (e.g., data from 20 °C–21.5 °C were used for training, 22 °C–23.5 °C for validation, 24 °C–25.5 °C for training, 26 °C–27.5 °C for testing, 28 °C–29.5 °C again for training, etc.).

_{04}matches the result reported by Mariani et al.

^{®}Core™ i7 8650U CPU, which is also similar to the 5 min training time for the causal dilated CNN reported by Mariani et al. using one NVIDIA

^{®}Quadro RTX™ 6000 GPU (2000 epochs). Note, however, that the CPU used in our study only has a theoretical computational performance of 0.442 TFLOPS (tera floating-point operations per second) compared to 16.3 TFLOPS of the GPU.

## 4. Conclusions

## Supplementary Materials

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## Abbreviations

ALA | Adaptive linear approximation |

BDW | Best Daubechies wavelets |

BFC | Best Fourier coefficients |

BSS | Baseline signal stretch |

CFRP | Carbon fibre-reinforced plastic |

CNN | Convolutional neural network |

CPU | Central processing unit |

CV | Cross-validation |

D_{XX} | Damage number XX |

DB | Database |

FE | Feature extraction |

FS | Feature selection |

GPU | Graphics-processing unit |

GW | Guided waves |

HPO | Hyper-parameter optimisation |

LDA | Linear discriminant analysis |

LOGOCV | Leave-one-group-out cross-validation |

ML | Machine learning |

NN | Neural network |

OBS | Optimal baseline selection |

PC | Principle component |

PCA | Principle component analysis |

PCC | Pearson correlation coefficient |

RBF-Kernel | Radial basis function kernel |

RFE-SVM | Recursive feature elimination support vector machines* |

RH | Relative humidity |

RMSE | Root mean square error |

SHM | Structural health monitoring |

SM | Statistical moments |

SVM | Support vector machines |

T_{X} | Transducer number X |

TFLOPS | Tera floating-point operations per second |

UG | Undamaged group |

## Appendix A

**Figure A1.**(

**a**) Matrix of the first five PCs of the PCA on the raw data (undamaged plate and all simulated damage locations) coloured by their corresponding temperature with their histograms on the diagonal and the variance explained by each PC given as percentage in brackets. (

**b**) First three PCs plotted into a three-dimensional space.

## Appendix B

## Appendix C

**Figure A2.**Resulting accuracy of the validation data for various parameter combinations (parameter $C$ of the SVM and number of features selected by RELIEFF). Parameter combinations within the purple boxes achieve 100% validation accuracy. The gap between the two purple boxes consists of parameter combinations achieving an accuracy of 99.5%.

## References

- Qiu, L.; Fang, F.; Yuan, S. Improved density peak clustering-based adaptive Gaussian mixture model for damage monitoring in aircraft structures under time-varying conditions. Mech. Syst. Signal Process.
**2019**, 126, 281–304. [Google Scholar] [CrossRef] - Abdeljaber, O.; Sassi, S.; Avci, O.; Kiranyaz, S.; Ibrahim, A.A.; Gabbouj, M. Fault Detection and Severity Identification of Ball Bearings by Online Condition Monitoring. IEEE Trans. Ind. Electron.
**2018**, 66, 8136–8147. [Google Scholar] [CrossRef] [Green Version] - Munir, N.; Kim, H.-J.; Park, J.; Song, S.-J.; Kang, S.-S. Convolutional neural network for ultrasonic weldment flaw classification in noisy conditions. Ultrasonics
**2018**, 94, 74–81. [Google Scholar] [CrossRef] - De Oliveira, M.A.; Monteiro, A.V.; Filho, J.V. A New Structural Health Monitoring Strategy Based on PZT Sensors and Convolutional Neural Network. Sensors
**2018**, 18, 2955. [Google Scholar] [CrossRef] [Green Version] - Cruz, F.; Filho, E.S.; de Albuquerque, M.C.S.; Silva, I.; Farias, C.; Gouvêa, L. Efficient feature selection for neural network based detection of flaws in steel welded joints using ultrasound testing. Ultrasonics
**2016**, 73, 1–8. [Google Scholar] [CrossRef] - Bornn, L.; Farrar, C.R.; Park, G.; Farinholt, K. Structural Health Monitoring With Autoregressive Support Vector Machines. J. Vib. Acoust.
**2009**, 131, 021004. [Google Scholar] [CrossRef] [Green Version] - Roy, S.; Chang, F.; Lee, S.; Pollock, P.; Janapati, V. A novel machine-learning approach for structural state identification using ultrasonic guided waves. In Safety, Reliability, Risk and Life-Cycle Performance of Structures and Infrastructures; Deodatis, G., Ellingwood, B., Frangopol, D., Eds.; CRC Press: Boca Raton, FL, USA, 2014; pp. 321–328. [Google Scholar]
- Miorelli, R.; Kulakovskyi, A.; Chapuis, B.; D’Almeida, O.; Mesnil, O. Supervised learning strategy for classification and regression tasks applied to aeronautical structural health monitoring problems. Ultrasonics
**2021**, 113, 106372. [Google Scholar] [CrossRef] - Mariani, S.; Rendu, Q.; Urbani, M.; Sbarufatti, C. Causal dilated convolutional neural networks for automatic inspection of ultrasonic signals in non-destructive evaluation and structural health monitoring. Mech. Syst. Signal Process.
**2021**, 157, 107748. [Google Scholar] [CrossRef] - Keogh, E.; Kasetty, S. On the Need for Time Series Data Mining Benchmarks: A Survey and Empirical Demonstration. Data Min. Knowl. Discov.
**2003**, 7, 349–371. [Google Scholar] [CrossRef] - Schneider, T.; Helwig, N.; Schutze, A. Automatic feature extraction and selection for condition monitoring and related datasets. In Proceedings of the 2018 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), Houston, TX, USA, 14–17 May 2018; pp. 1–6. [Google Scholar]
- Moll, J.; Kexel, C.; Pötzsch, S.; Rennoch, M.; Herrmann, A.S. Temperature affected guided wave propagation in a composite plate complementing the Open Guided Waves Platform. Sci. Data
**2019**, 6, 1–9. [Google Scholar] [CrossRef] - Schubert, K.; Stieglitz, A.; Christ, M.; Herrmann, A. Analytical and Experimental Investigation of Environmental Influences on Lamb Wave Propagation and Damping Measured with a Piezo-Based System. Work. Struct. Health Monit.
**2012**, 49, 1–9. [Google Scholar] - Croxford, A.J.; Moll, J.; Wilcox, P.; Michaels, J.E. Efficient temperature compensation strategies for guided wave structural health monitoring. Ultrasonics
**2010**, 50, 517–528. [Google Scholar] [CrossRef] - Schneider, T.; Helwig, N.; Schütze, A. Industrial Condition Monitoring with Smart Sensors Using Automated Feature Extraction and Selection. Meas. Sci. Technol.
**2018**, 29, 094002. [Google Scholar] [CrossRef] - Dorst, T.; Robin, Y.; Schneider, T.; Schütze, A. Automated ML Toolbox for Cyclic Sensor Data. In Proceedings of the Mathematical and Statistical Methods for Metrology MSMM, Virtual, 31 May–1 June 2021. [Google Scholar]
- Bastuck, M.; Baur, T.; Schütze, A. DAV3E—A MATLAB Toolbox for Multivariate Sensor Data Evaluation. J. Sens. Sens. Syst.
**2018**, 7, 489–506. [Google Scholar] [CrossRef] [Green Version] - Olszewski, R. Generalized Feature Extraction for Structural Pattern Recognition in Time-Series Data; Carnegie Mellon University: Pittsburgh, PA, USA, 2001. [Google Scholar]
- Wold, S.; Esbensen, K.; Geladi, P. Principal Component Analysis. Chemom. Intell. Lab. Syst.
**1987**, 2, 37–52. [Google Scholar] [CrossRef] - Mörchen, F. Time Series Feature Extraction for Data Mining Using DWT and DFT; University of Marburg: Marburg, Germany, 2003. [Google Scholar]
- Daubechies, I. Ten Lectures on Wavelets; SIAM: Philadelphia, PA, USA, 1992. [Google Scholar]
- Papoulis, A.; Pillai, S.U. Probability, Random Variables, and Stochastic Processes; McGraw-Hill: Boston, MA, USA, 2002. [Google Scholar]
- Guyon, I.; Elisseeff, A. An Introduction to Variable and Feature Selection. J. Mach. Learn. Res.
**2003**, 3, 1157–1182. [Google Scholar] - Rakotomamonjy, A. Variable Selection Using SVM-Based Criteria. J. Mach. Learn. Res.
**2003**, 3, 1357–1370. [Google Scholar] - Robnik-Šikonja, M.; Kononenko, I. Theoretical and Empirical Analysis of ReliefF and RReliefF. Mach. Learn.
**2003**, 53, 23–69. [Google Scholar] [CrossRef] [Green Version] - Kononenko, I.; Šimec, E.; Robnik-Šikonja, M. Overcoming the Myopia of Inductive Learning Algorithms with RELIEFF. Appl. Intell.
**1997**, 7, 39–55. [Google Scholar] [CrossRef] - Cohen, I.; Huang, Y.; Chen, J.; Benesty, J.; Chen, Y.H.; Cohen, I. Pearson Correlation Coefficient. In Noise Reduction in Speech Processing; Springer: Berlin/Heidelberg, Germany, 2009. [Google Scholar] [CrossRef]
- Duda, R.O.; Hart, P.E.; Stork, D.G. Pattern Classification; Wiley: New York, NY, USA, 2001. [Google Scholar]
- Wainer, J. Comparison of 14 Different Families of Classification Algorithms on 115 Binary Datasets. arXiv
**2016**, arXiv:1606.00930. [Google Scholar] - Gui, G.; Pan, H.; Lin, Z.; Li, Y.; Yuan, Z. Data-driven support vector machine with optimization techniques for structural health monitoring and damage detection. KSCE J. Civ. Eng.
**2017**, 21, 523–534. [Google Scholar] [CrossRef] - Abe, S. Support Vector Machines for Pattern Classification; Springer: London, UK; New York, NY, USA, 2010. [Google Scholar]
- Schölkopf Bernhard, B. Learning With Kernels: Support Vector Machines, Regularization, Optimization, and Beyond; MIT Press: Cambridge, MA, USA, 2018. [Google Scholar]
- Liu, S.; Jiang, N. SVM Parameters Optimization Algorithm and Its Application. In Proceedings of the 2008 IEEE International Conference on Mechatronics and Automation, Takamatsu, Japan, 5–8 August 2008; pp. 509–513. [Google Scholar]
- Liu, Y.; Du, J. Parameter Optimization of the SVM for Big Data. In Proceedings of the 2015 8th International Symposium on Computational Intelligence and Design (ISCID), Hangzhou, China, 12–13 December 2015; pp. 341–344. [Google Scholar]
- Hsu, C.; Chang, C.; Lin, C.-J. A Practical Guide to Support Vector Classification; University of National Taiwan: Taipei, Taiwan, 2003. [Google Scholar]
- Schnur, C.; Moll, J.; Lugovtsova, Y.; Schütze, A.; Schneider, T. Explainable Machine Learning For Damage Detection: In Carbon Fiber Composite Plates Under Varying Temperature Conditions. In Proceedings of the ASME 2021 48th Annual Review of Progress in Quantitative Nondestructive Evaluation, Virtual, 28–30 July 2021. [Google Scholar]
- Kartikejan, P.; Sabarianand, D.; Sugantan, S. Investigation on adaptability of carbon fiber tube for serial manipulator. FME Trans.
**2019**, 47, 412–417. [Google Scholar] [CrossRef] [Green Version] - Skolaut, W. (Ed.) Maschinenbau: Ein Lehrbuch für das Ganze Bachelor-Studium; Springer: Berlin, Germany, 2014. [Google Scholar]
- Liu, D.; Tang, Z.; Bao, Y.; Li, H. Machine-Learning-Based Methods for Output Only Structural Modal Identification. Struct. Control. Health Monit.
**2021**, 28, e2843. [Google Scholar] [CrossRef] - XXu, L.; Yuan, S.; Chen, J.; Ren, Y. Guided Wave-Convolutional Neural Network Based Fatigue Crack Diagnosis of Aircraft Structures. Sensors
**2019**, 19, 3567. [Google Scholar] [CrossRef] [Green Version] - Azimi, M.; Eslamlou, A.D.; Pekcan, G. Data-Driven Structural Health Monitoring and Damage Detection through Deep Learning: State-of-the-Art Review. Sensors
**2020**, 20, 2778. [Google Scholar] [CrossRef] - Xu, Y.; Xie, L.; Zhang, X.; Chen, X.; Qi, G.-J.; Tian, Q.; Xiong, H. PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search. arXiv
**2020**, arXiv:1907.05737. [Google Scholar] - Zoph, B.; Le, Q.V. Neural Architecture Search with Reinforcement Learning. arXiv
**2017**, arXiv:1611.01578. [Google Scholar] - Liu, H.; Simonyan, K.; Yang, Y. DARTS: Differentiable Architecture Search. arXiv
**2019**, arXiv:1806.09055. [Google Scholar] - Fast Fourier Transform—MATLAB Fft—MathWorks Deutschland. Available online: https://de.mathworks.com/help/matlab/ref/fft.html#buuutyt-6 (accessed on 20 July 2021).
- Phase Angle—MATLAB Angle—MathWorks Deutschland. Available online: https://de.mathworks.com/help/matlab/ref/angle.html (accessed on 20 July 2021).
- Find K-Nearest Neighbors Using Input Data—MATLAB Knnsearch—MathWorks Deutschland. Available online: https://de.mathworks.com/help/stats/knnsearch.html (accessed on 20 July 2021).
- Fit Multiclass Models for Support Vector Machines or Other Classifiers—MATLAB Fitcecoc—MathWorks Deutschland. Available online: https://de.mathworks.com/help/stats/fitcecoc.html#bufm0tv (accessed on 9 December 2021).
- Support Vector Machine Template—MATLAB TemplateSVM—MathWorks Deutschland. Available online: https://de.mathworks.com/help/stats/templatesvm.html (accessed on 9 December 2021).

**Figure 1.**(

**a**) Schematic of the experimental setup [12]. The analysed sensor combination is indicated by circles (the red circle indicates the transmitter T4, whereas the green circle indicates the receiver T9). The considered damage positions (D

_{04}, D

_{12}, D

_{16}, D

_{24}) are indicated by filled black dots. (

**b**) 40 kHz Hann-windowed tone-burst signal with five cycles. (

**c**) Temperature of the climatic chamber for each measurement number, where the dotted lines indicate the corresponding groups of the database, undamaged and damaged measurements.

**Figure 2.**Pre-processing of the raw data to compensate for temperature-related effects by using optimal baseline selection (OBS) and baseline signal stretch (BSS).

**Figure 5.**(

**a**) Matrix of the first five PCs of the PCA on the pre-processed data (undamaged plate and all simulated damage locations) with their histograms on the diagonal and the variance explained by each PC given as a percentage in brackets. The red box indicating the scatterplot of PC 2 and PC 3 is also shown in (

**b**), where the data points are additionally coloured by their corresponding temperature.

**Figure 6.**Damage classification results of the leave-one-class-out cross-validation. The plot is divided into six sections by dotted lines. Each section represents a heat cycle with one specific damage condition (undamaged and damaged D

_{04}, D

_{12}, D

_{16}, D

_{24}).

**Figure 7.**(

**a**) 40 kHz excitation signal. (

**b**) Single-sided amplitude spectrum of the 40 kHz excitation signal. Purple bars indicate the frequencies selected by the improved algorithm of the automated toolbox.

**Figure 8.**Grouping of the data into training, validation, and test data for (

**a**) 2 °C and (

**b**) 16 °C extrapolation.

**Table 1.**Position of the transducers and the damage locations [12]. The distance of the damage locations to the direct signal path had been calculated.

Label | Position on x-Axis (mm) | Position on y-Axis (mm) | Distance to Signal Path (mm) |
---|---|---|---|

Transducer positions | |||

Transducer 4 | 210 | 470 | 0 |

Transducer 9 | 290 | 30 | 0 |

Damage positions | |||

Damage 04 | 65 | 400 | 155 |

Damage 12 | 195 | 330 | 40 |

Damage 16 | 335 | 260 | 85 |

Damage 24 | 450 | 190 | 186 |

**Table 2.**Feature extraction and selection methods of the automated toolbox [17].

Methods | Abbreviation | Literature |
---|---|---|

Feature Extraction Methods | ||

Adaptive linear approximation | ALA | [18] |

Principal component analysis | PCA | [19] |

Best Fourier coefficients | BFC | [20] |

Best Daubechies wavelets | BDW | [21] |

Statistical moments | SM | [22] |

Feature Selection Methods | ||

Recursive feature elimination support vector machines * | RFE-SVM | [23,24] |

RELIEFF * | RELIEFF | [25,26] |

Pearson correlation coefficient | PCC | [27] |

**Table 3.**Parameters and values used for the grid search approach to improve the ML model. “Number of features” means the selected features that are used for classification. Bold numbers indicate the selected hyper-parameters for Section 3.5.

Hyper-Parameter | # of Values | Values |
---|---|---|

Number of features | 31 | 1, 2, …, 10, 15, 20, …, 25, …, 50, 60, 70, …, 100, 150, …, 500 |

Regularisation parameter C | 11 | 0.1, 0.3, 1, 3.2, 10, 31.6, 100, 316.2, 1000, 3162.3, 10,000 |

**Table 4.**Overview of the testing accuracies of all 15 combinations of the automated toolbox, derived in a previous study [36]. The highest testing accuracy is shown in bold.

Testing Accuracy for Each Algorithm Combination of the Automated Toolbox | |||||
---|---|---|---|---|---|

PCA | BFC | BDW | ALA | SM | |

Pearson | 42% | 73% | 42% | 31% | 81% |

RELIEFF | 42% | 80% | 43% | 31% | 78% |

RFE-SVM | 52% | 88% | 48% | 31% | 81% |

**Table 5.**Overview of the testing accuracy and number of misclassifications of the improved algorithms (BFC, RELIEFF with Pearson pre-selection, RFE-SVM) of the toolbox for GW-based SHM.

Results of the Improved Algorithms of the Toolbox | |||||||
---|---|---|---|---|---|---|---|

Damage Case | UG_{1} | UG_{2} | D_{04} | D_{12} | D_{16} | D_{24} | Total |

Number of samples | 80 | 161 | 161 | 161 | 161 | 161 | 885 |

Misclassifications | 1 | 3 | 0 | 0 | 0 | 29 | 33 |

Accuracy | 98.7% | 98.1% | 100% | 100% | 100% | 82.0% | 96.2% |

**Table 6.**Ranked BFC features, i.e., frequencies, for transducer combinations 4 and 9 with their rank, total selections, amplitude selections, and phase selections. Ranking is based on how often the respective frequency is selected either as an amplitude or a phase feature in the six different LOGOCV models. Four frequencies are selected six times each.

Ranked Frequencies (BFC Features) | |||||
---|---|---|---|---|---|

Nr. | Rank | Frequency | Total Selections | Amplitude Selections | Phase Selections |

1 | 1 | 38.9 kHz | 10 | 4 | 6 |

2 | 2 | 42.7 kHz | 9 | 6 | 3 |

3 | 3 | 45.0 kHz | 8 | 3 | 5 |

4 | 4 | 35.9 kHz | 7 | 2 | 5 |

5 | 5 | 27.5 kHz | 6 | 6 | 0 |

6 | 5 | 36.6 kHz | 6 | 5 | 1 |

7 | 5 | 42.0 kHz | 6 | 0 | 6 |

8 | 5 | 45.8 kHz | 6 | 3 | 3 |

Label | Position on x-Axis (mm) | Position on y-Axis (mm) |
---|---|---|

Transducer 1 | 450 | 470 |

Transducer 7 | 450 | 30 |

Label | Distance from Signal Path (mm) |
---|---|

Damage 04 | 385 |

Damage 12 | 255 |

Damage 16 | 11.5 |

Damage 24 | 0 |

**Table 9.**Accuracy and number of misclassifications of the improved algorithm (BFC for feature extraction, RELIEFF for feature selection, SVM with RBF kernel for classification validated with LOGOCV) for the combination of transducers 1 (sender) and 7 (receiver).

Validation Results of the Improved Algorithm for the Combination of Transducers 1 and 7 | |||||||
---|---|---|---|---|---|---|---|

Damage case | UG_{1} | UG_{2} | D_{04} | D_{12} | D_{16} | D_{24} | Total |

Misclassifications | 4 | 39 | 133 | 68 | 0 | 0 | 244 |

Accuracy | 94.9% | 75.8% | 17.4% | 57.7% | 100% | 100% | 72% |

**Table 10.**Resulting testing accuracy over temperature extrapolation. The extrapolated temperatures were not used for the model building and only used for testing.

Resulting Testing Accuracy for a Certain Temperature Extrapolation | |||||||
---|---|---|---|---|---|---|---|

Temperature extrapolation | 2 °C | 4 °C | 6 °C | 8 °C | 10 °C | 12 °C | 14 °C |

Testing accuracy | 100% | 100% | 100% | 97.0% | 96.8% | 93.6% | 83.7% |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Schnur, C.; Goodarzi, P.; Lugovtsova, Y.; Bulling, J.; Prager, J.; Tschöke, K.; Moll, J.; Schütze, A.; Schneider, T.
Towards Interpretable Machine Learning for Automated Damage Detection Based on Ultrasonic Guided Waves. *Sensors* **2022**, *22*, 406.
https://doi.org/10.3390/s22010406

**AMA Style**

Schnur C, Goodarzi P, Lugovtsova Y, Bulling J, Prager J, Tschöke K, Moll J, Schütze A, Schneider T.
Towards Interpretable Machine Learning for Automated Damage Detection Based on Ultrasonic Guided Waves. *Sensors*. 2022; 22(1):406.
https://doi.org/10.3390/s22010406

**Chicago/Turabian Style**

Schnur, Christopher, Payman Goodarzi, Yevgeniya Lugovtsova, Jannis Bulling, Jens Prager, Kilian Tschöke, Jochen Moll, Andreas Schütze, and Tizian Schneider.
2022. "Towards Interpretable Machine Learning for Automated Damage Detection Based on Ultrasonic Guided Waves" *Sensors* 22, no. 1: 406.
https://doi.org/10.3390/s22010406