# A New Structural Health Monitoring Strategy Based on PZT Sensors and Convolutional Neural Network

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

- We developed a novel method suitable for mechanical data analysis. A method that takes advantage of the combination of the EMI-PZT-based method along with CNN.
- A way of converting PZT response based on the EMI technique to a RGB frame constitutes a novel approach;
- Frames were computed through a wide range of frequency instead of choosing only the best range in which the EMI presents higher sensitivity. This issue provides an important advantage because that task is very difficult;
- An unpublished frame dataset encompassing a total of four types of structural conditions for each PZT is introduced;
- An enhanced method which requires only a small dataset for training the CNN without using GPU. Furthermore, only three epochs are needed to yield 100% of hit rate.

## 2. Theoretical Fundamentals

#### 2.1. Structural Health Monitoring Systems

_{X}(with amplitude V

_{P}and angular frequency (ω)) will produce a current I with amplitude I

_{P}and phase Ψ. The electrical impedance of the PZT (Z

_{E}(ω)) is given as follows [4]:

_{a}(ω) and Z(ω) represent the mechanical impedances for the transducer and monitored structure, respectively. In Equation (1), ${{\overline{\mathsf{\epsilon}}}_{33}}^{\mathrm{T}},{\widehat{Y}}_{xx}^{E},{d}_{3x}^{2}$, and j represent dielectric constant, Young’s modulus, electric field constant, geometric constant and imaginary unit respectively. Note from Equation (1) that any variation in terms of the structural impedance will cause changes in the electrical impedance of the PZT patch and this, in turn, causes changes in the EMI signatures. Extra details of how PZT impedance is related to the structural condition via the EMI technique can be explored in the following references [4,7,46,47,48,49].

#### 2.2. The Convolutional Neural Network

## 3. Developed Method

#### 3.1. Phase 1: Acquisition of the EMI Signals

#### 3.2. Phase 2: Formation of the Frames

**Step 1:**The matrix containing the raw EMI data, sampled by the LabVIEW acquisition software, is read;**Step 2:**As the proposed method uses only the real part of the EMI, those samples are retrieved from the matrix into an array;**Step 3:**The EMI signatures (baseline and unknown conditions) are divided into equal parts (10 parts for each signal);**Step 4:**Those parts are used to compute Euclidian distances and generate a new array;**Step 5:**That new array is transformed into a square matrix;**Step 6:**Those obtained values (inside the array) are normalized by the maximum mean;**Step 7:**Using the colormap function (MATLAB), the normalized matrix is mapped to a colored matrix (RGB);**Step 8:**The generated image is then saved as a JPEG image. The image will be used as an input to the CNN preprocessing block (Figure 9).

^{®}, Euclidean Distances (ED) were computed from the EMI parts, as follows (Step 4):

#### 3.3. Phase 3: CNN-Based Damage Detection Method

## 4. Experimental Results

## 5. Comparison with Other State-of-the-Art Solutions

#### Advantages and Drawbacks

## 6. Conclusions and Future Work

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Morrow, D.K.; Fafard, A. World Airliner Census; FligthGlobal: London, UK, 2011; pp. 1–28. [Google Scholar]
- Boller, C. Ways and options for aircraft structural health management. Smart Mater. Struct.
**2001**, 10, 432. [Google Scholar] [CrossRef] - Brand, C.; Boller, C. Identification of Life Cycle Cost Reductions in Structures with Self-Diagnostic Devices; Daimler Chrysler Aerospace AG Munchen (Germany) Military Aircraft Div.: Munchen, Germany, 2000. [Google Scholar]
- Liang, C.; Sun, F.P.; Rogers, C.A. Coupled electro-mechanical analysis of adaptive material systems-determination of the actuator power consumption and system energy transfer. J. Intell. Mater. Syst. Struct.
**1994**, 5, 12–20. [Google Scholar] [CrossRef] - Park, G.; Cudney, H.H.; Inman, D.J. An integrated health monitoring technique using structural impedance sensors. J. Intell. Mater. Syst. Struct.
**2000**, 11, 448–455. [Google Scholar] [CrossRef] - Park, G.; Sohn, H.; Farrar, C.R.; Inman, D.J. Overview of piezoelectric impedance-based health monitoring and path forward. Shock. Vib. Dig.
**2003**, 35, 451–464. [Google Scholar] [CrossRef] - Park, G.; Inman, D.J. Structural health monitoring using piezoelectric impedance measurements. Philos. Trans. R. Soc. Lond. A Math. Phys. Eng. Sci.
**2007**, 365, 373–392. [Google Scholar] [CrossRef] [PubMed][Green Version] - Baptista, F.G.; Vieira Filho, J. A new impedance measurement system for PZT-based structural health monitoring. IEEE Trans. Instrum. Meas.
**2009**, 58, 3602–3608. [Google Scholar] [CrossRef] - Na, S.; Lee, H.K. Resonant frequency range utilized electro-mechanical impedance method for damage detection performance enhancement on composite structures. Compos. Struct.
**2012**, 94, 2383–2389. [Google Scholar] [CrossRef] - De Oliveira, M.A.; Vieira Filho, J.; Lopes, V., Jr.; Inman, D.J. A new approach for structural damage detection exploring the singular spectrum analysis. J. Intell. Mater. Syst. Struct.
**2017**, 28, 1160–1174. [Google Scholar] [CrossRef] - Ribeiro, D.M.S.; Aguiar, P.R.; Fabiano, L.F.G.; D’Addona, D.M.; Baptista, F.G.; Bianchi, E.C. Spectra measurements using piezoelectric diaphragms to detect burn in grinding process. IEEE Trans. Instrum. Meas.
**2017**, 66, 3052–3063. [Google Scholar] [CrossRef] - Mallardo, V.; Aliabadi, M.H. Optimal Sensor Placement for Structural, Damage and Impact Identification: A Review. Struct. Durab. Health Monit.
**2013**, 9, 287–323. [Google Scholar] - Lopes, V., Jr.; Park, G.; Cudney, H.H.; Inman, D.J. Impedance-based structural health monitoring with artificial neural networks. J. Intell. Mater. Syst. Struct.
**2000**, 11, 206–214. [Google Scholar] [CrossRef] - Saxena, A.; Saad, A. Evolving an artificial neural network classifier for condition monitoring of rotating mechanical systems. Appl. Soft Comput.
**2007**, 7, 441–454. [Google Scholar] [CrossRef] - Papatheou, E.; Dervilis, N.; Maguire, A.E.; Antoniadou, I.; Worden, K. A performance monitoring approach for the novel Lillgrund offshore wind farm. IEEE Trans. Ind. Electron.
**2015**, 62, 6636–6644. [Google Scholar] [CrossRef] - Na, S.; Lee, H.K. Neural network approach for damaged area location prediction of a composite plate using electromechanical impedance technique. Compos. Sci. Technol.
**2013**, 88, 62–68. [Google Scholar] [CrossRef] - Selva, P.; Cherrier, O.; Budinger, V.; Lachaud, F.; Morlier, J. Smart monitoring of aeronautical composites plates based on electromechanical impedance measurements and artificial neural networks. Eng. Struct.
**2013**, 56, 794–804. [Google Scholar] [CrossRef][Green Version] - Mallardo, V.; Khodaei, Z.S.; Aliabadi, F.M.H. A Bayesian Approach for Sensor Optimisation in Impact Identification. Materials
**2016**, 9, 946. [Google Scholar] [CrossRef] [PubMed] - Tran, V.T.; AlThobiani, F.; Ball, A.; Choi, B.-K. An application to transient current signal based induction motor fault diagnosis of Fourier–Bessel expansion and simplified fuzzy ARTMAP. Expert Syst. Appl.
**2013**, 40, 5372–5384. [Google Scholar] [CrossRef][Green Version] - Palomino, L.V.; Steffen, V.; Finzi Neto, R.M. Probabilistic neural network and fuzzy cluster analysis methods applied to impedance-based SHM for damage classification. Shock. Vib.
**2014**, 2014, 401942. [Google Scholar] [CrossRef] - Dos Anjos Lima, F.; de Souza, S.; Chavarette, F.R.; Martins Lopes, M.L.; Turra, A.E.; Lopes Júnior, V. Monitoring and fault identification in aeronautical structures using an ARTMAP-fuzzy-wavelet artificial neural network. Adv. Mater. Res.
**2014**, 1107–1112. [Google Scholar] [CrossRef] - De Oliveira, M.A.; Inman, D.J. Simplified fuzzy ARTMAP network-based method for assessment of structural damage applied to composite structures. J. Compos. Mater.
**2016**, 50, 3501–3514. [Google Scholar] [CrossRef] - Ali, J.B.; Saidi, L.; Mouelhi, A.; Chebel-Morello, B.; Fnaiech, F. Linear feature selection and classification using PNN and SFAM neural networks for a nearly online diagnosis of bearing naturally progressing degradations. Eng. Appl. Artif. Intell.
**2015**, 42, 67–81. [Google Scholar][Green Version] - De Oliveira, M.A.; Inman, D.J. Performance analysis of simplified Fuzzy ARTMAP and Probabilistic Neural Networks for identifying structural damage growth. Appl. Soft Comput.
**2017**, 52, 53–63. [Google Scholar] [CrossRef] - AlThobiani, F.; Tran, V.T.; Tinga, T. An Approach to Fault Diagnosis of Rotating Machinery Using the Second-Order Statistical Features of Thermal Images and Simplified Fuzzy ARTMAP. Engineering
**2017**, 9, 524. [Google Scholar] [CrossRef] - Gordan, M.; Razak, H.A.; Ismail, Z.; Ghaedi, K. Recent developments in damage identification of structures using data mining. Lat. Am. J. Solids Struct.
**2017**, 14, 2373–2401. [Google Scholar] [CrossRef] - De Oliveira, M.A.; Araujo, N.V.S.; da Silva, R.N.; da Silva, T.I.; Epaarachchi, J. Use of Savitzky–Golay Filter for Performances Improvement of SHM Systems Based on Neural Networks and Distributed PZT Sensors. Sensors
**2018**, 18, 152. [Google Scholar] [CrossRef] [PubMed] - Sun, M.; Han, T.X.; Liu, M.-C.; Khodayari-Rostamabad, A. Multiple instance learning convolutional neural networks for object recognition. In Proceedings of the 23rd International Conference on Pattern Recognition (ICPR), Cancun, Mexico, 4–8 December 2016; pp. 3270–3275. [Google Scholar]
- Costilla-Reyes, O.; Scully, P.; Ozanyan, K.B. Deep neural networks for learning spatio-temporal features from tomography sensors. IEEE Trans. Ind. Electron.
**2018**, 65, 645–653. [Google Scholar] [CrossRef] - Ince, T.; Kiranyaz, S.; Eren, L.; Askar, M.; Gabbouj, M. Real-time motor fault detection by 1-D convolutional neural networks. IEEE Trans. Ind. Electron.
**2016**, 63, 7067–7075. [Google Scholar] [CrossRef] - Guo, X.; Chen, L.; Shen, C. Hierarchical adaptive deep convolution neural network and its application to bearing fault diagnosis. Measurement
**2016**, 93, 490–502. [Google Scholar] [CrossRef] - Janssens, O.; Slavkovikj, V.; Vervisch, B.; Stockman, K.; Loccufier, M.; Verstockt, S.; de Walle, R.; Van Hoecke, S. Convolutional neural network based fault detection for rotating machinery. J. Sound Vib.
**2016**, 377, 331–345. [Google Scholar] [CrossRef] - Zhang, W.; Li, C.; Peng, G.; Chen, Y.; Zhang, Z. A deep convolutional neural network with new training methods for bearing fault diagnosis under noisy environment and different working load. Mech. Syst. Signal Process.
**2018**, 100, 439–453. [Google Scholar] [CrossRef] - Abdeljaber, O.; Avci, O.; Kiranyaz, S.; Gabbouj, M.; Inman, D.J. Real-time vibration-based structural damage detection using one-dimensional convolutional neural networks. J. Sound Vib.
**2017**, 388, 154–170. [Google Scholar] [CrossRef] - Xia, M.; Li, T.; Xu, L.; Liu, L.; de Silva, C.W. Fault Diagnosis for Rotating Machinery Using Multiple Sensors and Convolutional Neural Networks. IEEE/ASME Trans. Mechatron.
**2018**, 23, 101–110. [Google Scholar] [CrossRef] - Pan, J.; Zi, Y.; Chen, J.; Zhou, Z.; Wang, B. LiftingNet: A Novel Deep Learning Network with Layerwise Feature Learning from Noisy Mechanical Data for Fault Classification. IEEE Trans. Ind. Electron.
**2018**, 65, 4973–4982. [Google Scholar] [CrossRef] - Avci, O.; Abdeljaber, O.; Kiranyaz, S.; Hussein, M.; Inman, D.J. Wireless and real-time structural damage detection: A novel decentralized method for wireless sensor networks. J. Sound Vib.
**2018**, 424, 158–172. [Google Scholar] [CrossRef] - Abdeljaber, O.; Avci, O.; Kiranyaz, M.S.; Boashash, B.; Sodano, H.; Inman, D.J. 1-D CNNs for structural damage detection: verification on a structural health monitoring benchmark data. Neurocomputing
**2018**, 275, 1308–1317. [Google Scholar] [CrossRef] - Chen, F.-C.; Jahanshahi, M.R. NB-CNN: Deep Learning-Based Crack Detection Using Convolutional Neural Network and Naive Bayes Data Fusion. IEEE Trans. Ind. Electron.
**2018**, 65, 4392–4400. [Google Scholar] [CrossRef] - Kessler, S.S.; Spearing, S.M.; Atalla, M.J.; Cesnik, C.E.S.; Soutis, C. Damage detection in composite materials using frequency response methods. Compos. Part B Eng.
**2002**, 33, 87–95. [Google Scholar] [CrossRef] - Inman, D.J.; Farrar, C.R.; Junior, V.L.; Junior, V.S. Damage Prognosis: For Aerospace, Civil and Mechanical Systems; John Wiley & Sons: Hoboke, NJ, USA, 2005. [Google Scholar]
- Staszewski, W.; Boller, C.; Tomlinson, G.R. Health Monitoring of Aerospace Structures: Smart Sensor Technologies and Signal Processing; John Wiley & Sons: Hoboke, NJ, USA, 2004. [Google Scholar]
- Rytter, A. Vibrational Based Inspection of Civil Engineering Structures. Ph.D. Thesis, Department of Building Technology and Structural Engineering, Aalborg University, Aalborg, UK, 1993. [Google Scholar]
- Cortez, N.E.; Filho, J.V.; Baptista, F.G. A new microcontrolled structural health monitoring system based on the electromechanical impedance principle. Struct. Health Monit.
**2013**, 12, 14–22. [Google Scholar] [CrossRef] - Lin, B.; Giurgiutiu, V. Modeling and testing of PZT and PVDF piezoelectric wafer active sensors. Smart Mater. Struct.
**2006**, 15, 1085. [Google Scholar] [CrossRef] - Zagrai, A.N.; Giurgiutiu, V. Electro-mechanical impedance method for crack detection in thin plates. J. Intell. Mater. Syst. Struct.
**2001**, 12, 709–718. [Google Scholar] [CrossRef] - Farrar, C.R.; Worden, K. An introduction to structural health monitoring. Philos. Trans. R. Soc. Lond. A Math. Phys. Eng. Sci.
**2007**, 365, 303–315. [Google Scholar] [CrossRef] [PubMed][Green Version] - Martowicz, A.; Sendecki, A.; Salamon, M.; Rosiek, M.; Uhl, T. Application of electromechanical impedance-based SHM for damage detection in bolted pipeline connection. Nondestruct. Test. Eval.
**2016**, 31, 17–44. [Google Scholar] [CrossRef] - Huynh, T.-C.; Kim, J.-T. RBFN-based temperature compensation method for impedance monitoring in pre-stressed tendon anchorage. Struct. Control Health Monit.
**2018**, 25, e2173. [Google Scholar] [CrossRef] - Fukushima, K.; Miyake, S. Neocognitron: A self-organizing neural network model for a mechanism of visual pattern recognition. In Competition and Cooperation in Neural Nets; Springer: Berlin, Germany, 1982; pp. 267–285. [Google Scholar]
- Hubel, D.H.; Wiesel, T.N. Receptive fields and functional architecture of monkey striate cortex. J. Physiol.
**1968**, 195, 215–243. [Google Scholar] [CrossRef] [PubMed][Green Version] - LeCun, Y.; Boser, B.; Denker, J.S.; Henderson, D.; Howard, R.E.; Hubbard, W.; Jackel, L.D. Backpropagation applied to handwritten zip code recognition. Neural Comput.
**1989**, 1, 541–551. [Google Scholar] [CrossRef] - LeCun, Y.; Bottou, L.; Bengio, Y.; Haffner, P. Gradient-based learning applied to document recognition. Proc. IEEE
**1998**, 86, 2278–2324. [Google Scholar] [CrossRef][Green Version] - Gu, J.; Wang, Z.; Kuen, J.; Ma, L.; Shahroudy, A.; Shuai, B.; Liu, T.; Wang, X.; Wang, L.; Wang, G.; et al. Recent advances in convolutional neural networks. Pattern Recognit.
**2018**, 77, 354–377. [Google Scholar] [CrossRef][Green Version] - Goodfellow, I.; Bengio, Y.; Courville, A.; Bengio, Y. Deep Learning; MIT Press Cambridge: Cambridge, MA, USA, 2016; Volume 1. [Google Scholar]
- Wang, T.; Wu, D.J.; Coates, A.; Ng, A.Y. End-to-end text recognition with convolutional neural networks. In Proceedings of the 21st International Conference on Pattern Recognition (ICPR), Tsukuba, Japan, 11–15 November 2012; pp. 3304–3308. [Google Scholar]
- LeCun, Y.; Haffner, P.; Bottou, L.; Bengio, Y. Object recognition with gradient-based learning. In Shape, Contour and Grouping in Computer Vision; Springer: Berlin, Germany, 1999; pp. 319–345. [Google Scholar]
- Schmidhuber, J. Deep learning in neural networks: An overview. Neural Netw.
**2015**, 61, 85–117. [Google Scholar] [CrossRef] [PubMed][Green Version] - Sun, F.P.; Chaudhry, Z.; Liang, C.; Rogers, C.A. Truss structure integrity identification using PZT sensor–actuator. J. Intell. Mater. Syst. Struct.
**1995**, 6, 134–139. [Google Scholar] [CrossRef] - Freitas, E.S.; Baptista, F.G. Experimental analysis of the feasibility of low-cost piezoelectric diaphragms in impedance-based SHM applications. Sens. Actuators A Phys.
**2016**, 238, 220–228. [Google Scholar] [CrossRef] - De Oliveira, M.A.; Araujo, N.V.S.; Inman, D.J.; Vieira Filho, J. Kappa-PSO-FAN based method for damage identification on composite structural health monitoring. Expert Syst. Appl.
**2018**, 95, 1–13. [Google Scholar] [CrossRef] - Baptista, F.G.; Vieira Filho, J. Optimal frequency range selection for PZT transducers in impedance-based SHM systems. IEEE Sens. J.
**2010**, 10, 1297–1303. [Google Scholar] [CrossRef] - Campeiro, L.M.; da Silveira, R.Z.M.; Baptista, F.G. Impedance-based damage detection under noise and vibration effects. Struct. Health Monit.
**2018**, 17, 654–667. [Google Scholar] [CrossRef] - Baptista, F.G.; Budoya, D.E.; de Almeida, V.A.D.; Ulson, J.A.C. An experimental study on the effect of temperature on piezoelectric sensors for impedance-based structural health monitoring. Sensors
**2014**, 14, 1208–1227. [Google Scholar] [CrossRef] [PubMed] - Wandowski, T.; Malinowski, P.H.; Ostachowicz, W.M. Temperature and damage influence on electromechanical impedance method used for carbon fibre-reinforced polymer panels. J. Intell. Mater. Syst. Struct.
**2017**, 28, 782–798. [Google Scholar] [CrossRef] - Giurgiutiu, V. Structural Health Monitoring of Composite Structures with Piezoelectric-Wafer Active Sensors. AIAA J.
**2011**, 49, 565–581. [Google Scholar] [CrossRef] - Giurgiutiu, V. Structural Health Monitoring: With Piezoelectric Wafer Active Sensors, 2nd ed.; Academic Press: New York, NY, USA, 2014. [Google Scholar]
- Li, B.; Li, Z.; Zhou, J.; Ye, L.; Li, E. Damage localization in composite lattice truss core sandwich structures based on vibration characteristics. Compos. Struct.
**2015**, 126, 34–51. [Google Scholar] [CrossRef]

**Figure 2.**Developed framework for structural damage detection, based on the CNN algorithm, including all three phases.

**Figure 3.**Representation of the general diagram for the acquisition system (dimensions in millimeters) [27].

**Figure 4.**Experimental set up including: aluminum plate containing three PZT patches, DAQ (Data Acquisition) and computer running the acquisition software [27].

**Figure 6.**Division of the EMI signals for the baseline (top) and unknown (bottom) structural conditions before computing ED.

**Figure 10.**Real part of the EMI, for PZT#2, considering various structural conditions (H, D1, D2 and D3).

**Figure 11.**Set of frames formed from the EMI signatures for PZT#2: (

**a**) baseline with Healthy (H); (

**b**) baseline with damage 1 (D1); (

**c**) baseline with damage 2 (D2); (

**d**) baseline with damage 3 (D3).

**Figure 12.**Feature maps for the 1st CNN layer after applying 32 kernels into PZT#2 frames for the structural conditions: (

**a**) D1; (

**b**) H.

**Figure 13.**Feature maps for the 7th CNN layer after applying 64 kernels into PZT#2 frames for the structural conditions: (

**a**) Healthy (H); (

**b**) D1; (

**c**) D2; (

**d**) D3.

**Figure 14.**Performance analysis of the CNN for PZT#2: (

**a**) training and validation accuracy curve of the model as a function of epoch; (

**b**) training and validation loss curve of the model as a function of epoch; (

**c**) Consumption time versus number of epoch for the training phase.

**Table 1.**Distribution of number of frames, formed from the PZT-EMI signals for PZTs #1 and #2, into the dataset.

Structural Conditions | PZT #1 | PZT #2 | ||
---|---|---|---|---|

Training | Test | Training | Test | |

Healthy (H) | 36 | 24 | 36 | 24 |

Damage 1(D1) | 36 | 24 | 36 | 24 |

Damage 2(D2) | 36 | 24 | 36 | 24 |

Damage 3(D3) | 36 | 24 | 36 | 24 |

Total | 144 | 96 | 144 | 96 |

Sensors | Training Accuracy | Testing Accuracy |
---|---|---|

PZT #1 | 98% | 100% |

PZT #2 | 100% | 100% |

PZT #3 | 100% | 100% |

**Table 3.**Comparison of the CNN-based method with other NN approaches: Success rates obtained for the testing phase.

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

De Oliveira, M.A.; Monteiro, A.V.; Vieira Filho, J. A New Structural Health Monitoring Strategy Based on PZT Sensors and Convolutional Neural Network. *Sensors* **2018**, *18*, 2955.
https://doi.org/10.3390/s18092955

**AMA Style**

De Oliveira MA, Monteiro AV, Vieira Filho J. A New Structural Health Monitoring Strategy Based on PZT Sensors and Convolutional Neural Network. *Sensors*. 2018; 18(9):2955.
https://doi.org/10.3390/s18092955

**Chicago/Turabian Style**

De Oliveira, Mario A., Andre V. Monteiro, and Jozue Vieira Filho. 2018. "A New Structural Health Monitoring Strategy Based on PZT Sensors and Convolutional Neural Network" *Sensors* 18, no. 9: 2955.
https://doi.org/10.3390/s18092955