## 1. Introduction

Structural health monitoring (SHM) systems are automated methods for determining the change in the integrity of a mechanical system [

1]. For aircraft, SHM systems would ideally comprise of embedded sensors working autonomously to provide an in situ, real-time assessment of the structural health. The current assessment of U.S. Air Force airframes is maintained through the Aircraft Structural Integrity Program (ASIP) [

2]. The Aircraft Structural Integrity Program (ASIP) utilizes an inspection process that includes the potential for the use of SHM systems to detect damage, which better aligns with the 2008 Department of Defense (DoD) implemented Condition-Based Maintenance Plus (CBM+) policy, which seeks to decrease the maintenance burden by moving the assessment towards condition-based, rather than schedule-based approaches [

3]. The SHM system, though, must be validated as required by ASIP. It is the center of discussion in many communities, since the process for SHM system validation remains a complex barrier for the implementation of such systems for on board damage detection.

The validation of SHM systems is multi-faceted and compounded by the numerous conditions and variables, for which the SHM system must demonstrate reliable performance to specification. These include factors related to the following: (1) the non-destructive evaluation (NDE) method (here, it is the configuration of the SHM system itself, with respect to the sensing method utilized and its associated algorithms and models for assessment); (2) the part geometry, material, and condition; and (3) flaw characteristics, including location and orientation [

4], as well as varying environmental and operating conditions.

In order to validate the SHM system for all of these categories of factors, performing tests effectively, reliably, and simultaneously on real structures is seemingly infeasible and impractical, because of time, money, and other physical constraints. Therefore, we advocate that work that serves this purpose must begin in simulation and laboratory environments. Factors that can be accurately and mathematically computed may be studied under varying conditions to determine their effect on the SHM response. Those factors that the SHM response is most sensitive to must be considered in all SHM validation analyses, whereas other factors that are not influential on the SHM response need not be.

Indeed, many researchers have started to examine these factors using various methods, including sensitivity analyses, as well as varying factors and model-assisted approaches, with the goal of validating SHM systems for use in real (aircraft) structures [

5,

6,

7,

8,

9,

10,

11,

12,

13,

14,

15,

16,

17,

18,

19]. However, factors are either still examined individually rather than simultaneously, or many assumptions regarding the SHM system configuration and processing are fixed, so that the results are focused on the end measures—such as the SHM system validation metric, known as the probability of detection. The results from these studies are useful. However, if the design considerations or damage indices used for detection are changed, how does this affect the response? The SHM system is not only subject to the detection modality, but also to the configuration, choice of damage index from signal processing, and the environmental and load constraints. How do we know which parameters of the system design and application—from sensor choice, to the environment, to estimating of the probability of detection—was the sensitive or non-sensitive parameter(s) that could detect the damage robustly across all of the intended applications?

Therefore, we advocate for a sequential, model-assisted approach towards the validation of the SHM systems combined with the efficient statistical designs of the experiments, in order to reduce the burden of the SHM system validation. This is achieved by simultaneously examining a number of parameters while also reserving resources, to more deeply address the sources of variation that are the most influential on the SHM response, in subsequent experiments. In particular, we suggest that sensitivity on the SHM response begins, for example, with the basic design configuration and determination of what parameters, with respect to the design and possible damage indices, are the most sensitive and the most robust for the application. This information can then be used to inform subsequent experiments, which then seek to examine other factors, and finally, testing those factors experienced in the operational environment.

In this paper, we demonstrate the utility of using the simulation modeling environment to determine the SHM sensitive factors that must be considered in subsequent experiments. We demonstrate this concept by first examining the effect of the SHM system configuration and flaw location on the response of a signal from a known piezoelectric wafer active sensor (PWAS) in an aluminum plate. Mathematically, we derive the signal responses and, using statistical design of experiments (DOE), determine the statistically significant factors of the SHM system configuration and flaw locations that affect the features or damage indices that are computed from the signal. The amount of variation in the observed damage index, which is attributable to each factor, is quantified and used to identify the damage index exhibiting the most sensitivity to the system design factors and the factors exhibiting the greatest effect on the SHM system response.

As a secondary aim, we demonstrate that these results are obtainable using half of the required number of experimental runs, by invoking an efficient statistical design, namely, the half-fraction factorial. The comparison of the results between the full and reduced (half-fraction) experiment demonstrate the usefulness of this statistical design. The results from this work will demonstrate a method to examine a (large) number of factors affecting the SHM response simultaneously, using half of the required data runs so as to reduce the experimental burden and build a sequential approach to quantifying the sensitivity of the SHM system to all observable factors, in order to enable a demonstration of the system validation.

## 4. Discussion

The physical constraints in structures did not always allow optimal SHM configuration for monitoring, especially in hot spot monitoring. Knowing how alternation in the SHM configuration would affect the response was paramount to collecting and detecting quality data from which to determine whether structural cracks were forming, especially for critical structures and components. The factorial design that was utilized here was able to determine, among the SHM configuration settings, which factors were the most sensitive and produced the largest variation in the calculated damage index. By conducting these results in the simulation environment, we could save substantial costs and better inform subsequent experiments by using real material, which could focus on other factors. Furthermore, multiple damage indices could be computed and compared in order to determine which were the most robust to damage detection within the configuration. Later, if there were other damage indices that were of interest, the simulation data that were collected could be reprocessed in order to provide the same analysis for the new indices. In this manner, we could always build upon this data that were collected and reduce the need to re-run experiments, especially in samples, each time the damage index or other features were changed.

The ability of an SHM system to accurately detect and monitor damage was dependent upon the modality, physical configuration, damage indices, and features processed from the resulting signals, as well as other environmental and application specific determinants. We assumed that the application was a known PWAS in an aluminum plate and initiated the study with an examination of the sensitivity of various configuration and sensing parameters, as well as potential damage indices computable from the resultant signals. The choice of values that were given to these parameters during the SHM design had been, in the past, rather subjective. In this work, we showed that this choice did not need to be purely subjective; we demonstrated that this choice could be objectively made using the concept of sensitivity and tested with an appropriate statistical design. This current study used a binary (low/high) value assignment to each of the six parameters that were examined (namely, transmitter angle, receiver angle, frequency of excitation, sensor size, distance between excitation, and damage site and flaw size). The choice of the values that we assigned to the low and high levels of each of these factors was based upon many considerations, including the following: (1) physical options of sensor layout on a realistic riveted lapjoint; (2) our professional experience in designing SHM systems; (3) the tuning effects between guided waves and PWAS transducers; and (4) the effect of wave attenuation with distance from the point source as its energy spreads in ever increasing wavefront circles. Although the values we chose might appear somewhat subjective, we were able to determine major trends with respect to these effects on the SHM response and clearly demonstrated how such studies, which started in the simulation environment, might be able to inform subsequent studies.

The results from our analysis provided the researcher with valuable information, prior to tests in material or real structures, regarding how to best produce a response, as well as which summary measures (damage indices) of a signal were best suited for the geometry of the proposed SHM. The transmitter angle had by far the response signal, followed by the frequency and distance between excitation and the transmitter angle. These were the most critical components of the SHM system that we examined, and were logical given the geometry and considerations mentioned above, as well as the damage indices themselves, which were computed as comparisons of the baseline (no damage) signal to that of the signal in the damage state.

The SHM parameters that were significant for the damage index, based upon the correlation coefficient, contained no interactions, whereas interactions were present for the sum of the squared deviations between the baseline and observed signal. The lack of interaction for the correlation coefficient damage index was reasonable, since the correlation coefficient was scaled by the squared deviations and was perhaps a more robust measure for the SHM system. These results suggested that if the hot spot would not allow for a small transmitter angle, we still might have been able to invoke a larger response by decreasing the distance to the excitation, or, if these configuration changes were not possible, by exciting at a higher frequency. Similar responses could be achieved if using the sum of the squared deviations, however, the combined effects of the transmitter angle with frequency and distance to excitation would have to be taken into account. The difference in the peak amplitude damage index did not account for differences across the entire signal. Although simple to compute, its loss of information that was embedded in the rest of the signal was evident in its lower R^{2} value and the inconsistencies between the full factorial and half-fraction factorial statistical designs. When considering the use of a sensitive damage index, the difference in the peak amplitude remained less sensitive than the other indices that were considered.

It was, perhaps, surprising that the flaw size was not significant. However, the lack of sensitivity with respect to the sensor size had been observed previously [

12]. Clearly, the flaw size must have had some effect, otherwise there would have been no differences between the signals. It was possible that the range of flaw sizes, and in particular the smallest flaw size, was actually quite large for this application and therefore could explain why no significant effects were found. In fact, the associated wavelengths in this material (31.2 mm and 11.3 mm for 200 kHz and 550 kHz frequencies, respectively) were large in relation to the difference in crack length. Therefore, in relation to the other factors that we examined, we deduced that, although the system was sensitive to the existence of damage, it might not have had the sensitivity required to determine the size of the damage in the range that we examined. We expected that these results would change when smaller flaw sizes were considered. Finally, with respect to the parameters chosen and their results, we noted that in choosing binary low/high values, we assumed a linear trend across the values of the various parameters. If it was thought that these trends might have been be curvilinear, then three (low, medium, and high) or more values would be important. The statistical designs that include such levels did exist, as described below. In fact, we intend to address these issues in possible future work.

With respect to the statistical designs, although we focused on a specific factorial design, other designs did exist. The usefulness of the factorial design was in its ability to save experimental runs and cost by collapsing the design into half of the runs that were required to consider all of the experimental settings. In general, we demonstrated comparable results using only half of the runs in the half-fraction factorial design and we then demonstrated the efficiency of using such a design. One potential draw-back of the 2

^{k} design was in the inference between the two settings of each parameter. Since we only used two such settings, we assumed a linear trend across values of each of the various parameters (see

Figure 5 for instance). This assumption might not have held in all design settings. However, there were other factorial designs, for instance 3

^{k} factorial designs, which considered a general high, medium, and low level for each factor. If it was thought that the response might have been curvilinear as the value of the parameter increased/decreased, then the 3

^{k} design could be used to model this curvilinear trend from low to high. With such designs, the trends in the parameter values could be examined while still reducing the number of runs that were required (e.g., a half-fraction for the 3

^{k} design was possible). Such a reduction in the number of experiments was especially important considering that a 2

^{6} design was 64 experimental runs, but a full 3

^{6} design was 729 experimental runs.

It is noteworthy to remark that for the lndiffPA damage index, the normal probability plot in the half-fraction design was not as clear as is was for the other damage indices, which would be expected because of its lower sensitivity as a damage index. The plot did pick up, though, on the largest effect for lndiffPA, that being the transmitter angle. Admittedly, there was no hard and fast rule that stated that only the large effects that were found on the normal probability plot should have been be used in the ANOVA setting, as we did here. In general, this approach worked well and had produced, in both the full factorial and in a reduced design, the main significant effects for each damage index. However, the ANOVA could be conducted without this first step. In fact, if we fitted the ANOVA using a model building approach that would remove the non-significant, higher order interactions for lndiffPA, we found, as a stopping point, that the three-way interaction between the transmitter angle, receiver angle, and frequency (ABC) was significant for lndiffPA (R^{2} = 0.78). However, we utilized the normal probability plots in the spirit of the sparsity of effects principle and in order to establish appropriate degrees of freedom. As such, from only the half-fraction results, we concluded that if the damage index lndiffPA was of interest, further work would be needed in order to identify additional sources of variation—A result that might have also been concluded from the full factorial design, given the lower comparative R^{2} value. In fact, as we conducted the comparison of the damage indices to determine which were the most sensitive and robust for the SHM system, our conclusion was to work with either the damage index, based upon the correlation coefficient, or the sum of the squared deviations, as lndiffPA was much less predictive. This was a conclusion that was determined by the half-fraction factorial design. Furthermore, an R^{2} of about 97% was achievable with the correlation coefficient damage index and no interactions were present, demonstrating that this damage index was more malleable and robust for this SHM system.

Finally, we did not intend for this work to suggest that the system configuration should be determined based upon the damage index. Instead, we used different damage indices as a summary of the response signals, in order to demonstrate their ability to sensitively pick up the change in the signal, resulting from the presence of a crack, as not all damage indices were able to mathematically capture the nuances in the signal that were a result from the physics of the configuration and the damage that was present.

## 5. Conclusions

In conclusion, there is much utility when using the simulation modeling environment to determine the SHM sensitive factors that must be considered in subsequent experiments. Furthermore, with care in choosing the statistical design, an efficient and reduced number of experimental runs can provide the same information as a full experiment containing all of the factor levels that are varied simultaneously. We have found that the transmitter angle is the most sensitive SHM configuration design setting that is influential on the SHM response. Furthermore, some damage indices are more sensitive than others. The correlation coefficient appears to be the most robust as, although it is sensitive to the transmitter angle, frequency, and distance to excitation, these factors can be varied separately, as the correlation coefficient was not sensitive to interactions among these factors as the sum of squared deviations was. Furthermore, if carefully accounting for these factors, about 97% of the variation in the response can be explained. The sensor size and flaw sizes considered in this study are not significant and have no large effect on the response. In addition, choosing a simple damage index, such as the difference in the peak amplitude, results in a fair amount of variation in the response (about 30%) that is unexplained, which perhaps makes it the least useful damage index that we examined, producing inconsistent statistical results, as demonstrated by the full and half-fraction factorial statistical designs.

Finally, the methodologies and framework used in this study have larger implications for effective design and uncertainty evaluation in the design of structural health monitoring systems. With planning and care, results from a study such as this can be used to inform a subsequent experiment examining a secondary set of factors, which would build upon the these results. By reducing the number of data runs and testing the parameters in simulation environments, where possible, a sequential approach to quantifying the sensitivity of the SHM system to all observable factors may be constructed, enabling a cost- and time-efficient approach to the demonstration of the SHM system validation.