# Quantifying Information without Entropy: Identifying Intermittent Disturbances in Dynamical Systems

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Theory and Background

#### 2.1. Topological Structures of Dynamical Systems

#### 2.2. Information Theoretic Descriptions of Topological Structures for Dynamical Systems

## 3. The Information Impulse Function

## 4. Application of the IIF to Simulated Dynamic Response Data

#### 4.1. Simulation of Discontinuities in Nonlinear Dynamical Systems

#### 4.2. Example Output

## 5. Results

#### 5.1. Detecting Global Disturbances

#### 5.2. Detecting Internal Disturbances

## 6. Discussion

## Supplementary Materials

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Manini, N.; Braun, O.M.; Tosatti, E.; Guerra, R.; Vanossi, A. Friction and nonlinear dynamics. J. Phys. Condens. Matter
**2016**, 28, 293001. [Google Scholar] [CrossRef] [PubMed][Green Version] - Farazmand, M.; Sapsis, T.P. Extreme Events: Mechanisms and Prediction. Appl. Mech. Rev.
**2019**, 71. [Google Scholar] [CrossRef][Green Version] - Yan, C.; Li, P.; Liu, C.; Wang, X.; Yin, C.; Yao, L. Novel gridded descriptors of poincaré plot for analyzing heartbeat interval time-series. Comput. Biol. Med.
**2019**, 109, 280–289. [Google Scholar] [CrossRef] [PubMed] - Irving, D.; Sorrentino, F. Synchronization of dynamical hypernetworks: Dimensionality reduction through simultaneous block-diagonalization of matrices. Phys. Rev. E—Stat. Nonlinear Soft Matter Phys.
**2012**, 86. [Google Scholar] [CrossRef] [PubMed][Green Version] - Salinelli, E.; Tomarelli, F. Discrete Dynamical Models; Springer International Publishing: Vienna, Austria, 2014; ISBN 978-3-319-02291-8. [Google Scholar]
- Wen, S.F.; Shen, Y.J.; Yang, S.P.; Wang, J. Dynamical response of Mathieu–Duffing oscillator with fractional-order delayed feedback. Chaos Solitons Fractals
**2017**, 94, 54–62. [Google Scholar] [CrossRef] - Svenkeson, A.; Glaz, B.; Stanton, S.; West, B.J. Spectral decomposition of nonlinear systems with memory. Phys. Rev. E
**2016**, 93, 1–10. [Google Scholar] [CrossRef] - MacKay, R.S. Nonlinearity in complexity science. Nonlinearity
**2008**, 21, 1–10. [Google Scholar] [CrossRef][Green Version] - Yin, Y.; Duan, X. Information Transfer Among the Components in Multi-Dimensional Complex Dynamical Systems. Entropy
**2018**, 20, 774. [Google Scholar] [CrossRef][Green Version] - Allen, B.; Lippner, G.; Chen, Y.-T.; Fotouhi, B.; Momeni, N.; Nowak, M.A.; Yau, S.-T. Evolutionary dynamics on any population structure. Nat. Publ. Gr.
**2016**, 544, 227–230. [Google Scholar] [CrossRef][Green Version] - Adiga, A.; Kuhlman, C.J.; Marathe, M.V.; Ravi, S.S.; Rosenkrantz, D.J.; Stearns, R.E. Inferring local transition functions of discrete dynamical systems from observations of system behavior. Theor. Comput. Sci.
**2017**, 679, 126–144. [Google Scholar] [CrossRef] - Nepomuceno, E.G.; Perc, M. Computational chaos in complex networks. J. Complex Netw.
**2020**, 8, 1–16. [Google Scholar] [CrossRef] - Van Steen, M.; Tanenbaum, A.S. A brief introduction to distributed systems. Computing
**2016**, 98, 967–1009. [Google Scholar] [CrossRef][Green Version] - Shinbrot, T. Granular chaos and mixing: Whirled in a grain of sand. Chaos
**2015**, 25. [Google Scholar] [CrossRef] [PubMed] - Joglekar, D.M.; Mitra, M. Analysis of flexural wave propagation through beams with a breathing crack using wavelet spectral finite element method. Mech. Syst. Signal Process.
**2015**, 76–77, 576–591. [Google Scholar] [CrossRef] - Aliev, A.E.; Mayo, N.K.; Baughman, R.H.; Mills, B.T.; Habtour, E. Subwoofer and nanotube butterfly acoustic flame extinction. J. Phys. D Appl. Phys.
**2017**, 50. [Google Scholar] [CrossRef] - Liu, G.; Li, C.; Peng, Z.; Li, X.; Wu, J. Detecting remotely triggered microseismicity around Changbaishan Volcano following nuclear explosions in North Korea and large distant earthquakes around the world. Geophys. Res. Lett.
**2017**, 44, 4829–4838. [Google Scholar] [CrossRef] - Casas, J.R.; Moughty, J.J. Bridge Damage Detection Based on Vibration Data: Past and New Developments. Front. Built Environ.
**2017**, 3, 1–12. [Google Scholar] [CrossRef][Green Version] - Lang, Z.Q.; Park, G.; Farrar, C.R.; Todd, M.D.; Mao, Z.; Zhao, L.; Worden, K. Transmissibility of non-linear output frequency response functions with application in detection and location of damage in MDOF structural systems. Int. J. Non. Linear. Mech.
**2011**, 46, 841–853. [Google Scholar] [CrossRef] - Habtour, E.; Sridharan, R.; Dasgupta, A.; Robeson, M.; Vantadori, S. Phase influence of combined rotational and transverse vibrations on the structural response. Mech. Syst. Signal Process.
**2018**, 100. [Google Scholar] [CrossRef] - Habtour, E.M.; Cole, D.P.; Kube, C.M.; Henry, T.C.; Haynes, R.A.; Gardea, F.; Sano, T.; Tinga, T. Structural state awareness through integration of global dynamic and local material behavior. J. Intell. Mater. Syst. Struct.
**2019**, 30, 1355–1365. [Google Scholar] [CrossRef] - Belenky, V.; Glotzer, D.; Pipiras, V.; Sapsis, T.P. Distribution tail structure and extreme value analysis of constrained piecewise linear oscillators. Probabilistic Eng. Mech.
**2019**, 57, 1–13. [Google Scholar] [CrossRef] - Artime, O.; Ramasco, J.J.; San Miguel, M. Dynamics on networks: Competition of temporal and topological correlations. Sci. Rep.
**2017**, 7, 1–10. [Google Scholar] [CrossRef] [PubMed][Green Version] - Maack, J.; Turkington, B. Reduced models of point vortex systems. Entropy
**2018**, 20, 914. [Google Scholar] [CrossRef][Green Version] - Cliff, O.M.; Prokopenko, M.; Fitch, R. Minimising the kullback-leibler divergence for model selection in distributed nonlinear systems. Entropy
**2018**, 20, 51. [Google Scholar] [CrossRef][Green Version] - Guntu, R.K.; Yeditha, P.K.; Rathinasamy, M.; Perc, M.; Marwan, N.; Kurths, J.; Agarwal, A. Wavelet entropy-based evaluation of intrinsic predictability of time series. Chaos
**2020**, 30. [Google Scholar] [CrossRef] [PubMed] - Shaw, R. Strange attractors, chaotic behavior, and information flow. Z. Für Naturforsch. A
**1981**, 36, 80–112. [Google Scholar] [CrossRef] - Shannon, C.E. A Mathematical Theory of Communication. Bell Syst. Tech. J.
**1948**, 27, 623–666. [Google Scholar] [CrossRef] - Moon, F.C.; Holmes, P.J. A Magnetoelastic Strange Attractor. J. Sound Vib.
**1979**, 65, 275–296. [Google Scholar] [CrossRef] - Gilmore, R. Topological analysis of chaotic dynamical systems. Rev. Mod. Phys.
**1998**, 70, 1455–1529. [Google Scholar] [CrossRef][Green Version] - Das, B.; Gangopadhyay, G. Large deviation theory for the kinetics and energetics of turnover of enzyme catalysis in a chemiostatic flow. J. Chem. Phys.
**2018**, 148. [Google Scholar] [CrossRef] - Sarkar, S.; Chattopdhyay, P.; Ray, A. Symbolization of dynamic data-driven systems for signal representation. Signalimage Video Process.
**2016**, 10, 1535–1542. [Google Scholar] [CrossRef] - Ledrappier, F.; Young, L.-S. The Metric Entropy of Diffeomorphisms: Part I: Characterization of Measures Satisfying Pesin’s Entropy Formula. Ann. Math.
**1985**, 122, 509. [Google Scholar] [CrossRef] - Ledrappier, F.; Young, L.-S. The Metric Entropy of Diffeomorphisms: Part II: Relations between Entropy, Exponents and Dimension. Ann. Math.
**1985**, 122, 540. [Google Scholar] [CrossRef] - Mesón, A.M.; Vericat, F. Invariant of dynamical systems: A generalized entropy. J. Math. Phys.
**1996**, 37, 4480–4483. [Google Scholar] [CrossRef] - Bandt, C.; Pompe, B. Permutation Entropy: A Natural Complexity Measure for Time Series. Phys. Rev. Lett.
**2002**, 88, 174102. [Google Scholar] [CrossRef] - Amigó, J.M. The equality of Kolmogorov-Sinai entropy and metric permutation entropy generalized. Phys. D Nonlinear Phenom.
**2012**, 241, 789–793. [Google Scholar] [CrossRef] - Garland, J.; Jones, T.R.; Neuder, M.; Morris, V.; White, J.W.C.; Bradley, E. Anomaly detection in paleoclimate records using permutation entropy. Entropy
**2018**, 20, 931. [Google Scholar] [CrossRef][Green Version] - Yao, W.; Yao, W.; Yao, D.; Guo, D.; Wang, J. Shannon entropy and quantitative time irreversibility for different and even contradictory aspects of complex systems. Appl. Phys. Lett.
**2020**, 116. [Google Scholar] [CrossRef] - Powell, G.E.; Percival, I.C. A spectral entropy method for distinguishing regular and irregular motion of Hamiltonian systems. J. Phys. A Gen. Phys.
**1979**, 12, 2053–2071. [Google Scholar] [CrossRef] - West, B.M.; Locke, W.R.; Andrews, T.C.; Scheinker, A.; Farrar, C.R. Applying concepts of complexity to structural health monitoring. Conf. Proc. Soc. Exp. Mech. Ser.
**2019**, 6, 205–215. [Google Scholar] [CrossRef] - Islam, M.R.; Rahim, M.A.; Akter, H.; Kabir, R.; Shin, J. Optimal IMF selection of EMD for sleep disorder diagnosis using EEG signals. Acm Int. Conf. Proceeding Ser.
**2018**, 96–101. [Google Scholar] [CrossRef] - Li, D.; Cao, M.; Deng, T.; Zhang, S. Wavelet packet singular entropy-based method for damage identification in curved continuous girder bridges under seismic excitations. Sensors (Switzerland)
**2019**, 19, 4272. [Google Scholar] [CrossRef][Green Version] - Riedl, M.; Müller, A.; Wessel, N. Practical considerations of permutation entropy: A tutorial review. Eur. Phys. J. Spec. Top.
**2013**, 222, 249–262. [Google Scholar] [CrossRef] - Takeishi, N.; Kawahara, Y.; Yairi, T. Subspace dynamic mode decomposition for stochastic Koopman analysis. Phys. Rev. E
**2017**, 96, 1–11. [Google Scholar] [CrossRef] [PubMed][Green Version] - Li, Q.; Dietrich, F.; Bollt, E.M.; Kevrekidis, I.G. Extended dynamic mode decomposition with dictionary learning: A data-driven adaptive spectral decomposition of the koopman operator. Chaos
**2017**, 27, 1–25. [Google Scholar] [CrossRef] [PubMed] - Hemati, M.S.; Rowley, C.W.; Deem, E.A.; Cattafesta, L.N. De-biasing the dynamic mode decomposition for applied Koopman spectral analysis of noisy datasets. Theor. Comput. Fluid Dyn.
**2017**, 31, 349–368. [Google Scholar] [CrossRef] - Eckart C, Y.G. 311_1.Tif. Psychometrika
**1936**, 1, 211–218. [Google Scholar] - Stockwell, R.G. Localization of the complex spectrum: The s transform. IEEE Trans. Signal Process.
**1996**, 44, 993. [Google Scholar] [CrossRef]

**Figure 1.**Poincaré distance showing the topological evolution of the Duffing oscillator with parameters: $\zeta =.15$, $k=1$, and $\Omega =1.2$. Forward and backward solutions are given in grey and black, respectively.

**Figure 3.**Example IIF analysis on the acceleration time history of Duffing oscillator in chaos. Plotted in black are results without a discontinuity. Plotted in red are results containing a discontinuity. (

**a**) The sinusoidal excitation. The discontinuity at τ = 160 is almost not visible. (

**b**) The acceleration response. Due to the chaotic nature of the system, the effect of a discontinuity is evident for τ > 160 when compared to the response without a discontinuity. (

**c**) The IIF results. The IIF computed on response data containing a discontinuity produced a peak value near τ = 160 that easily distinguishable from the background. In contrast, the IIF computed on the response data without a discontinuity did not produce any large peaks.

**Figure 4.**(

**a**) Difference in excitation from the baseline to the test case. This figure shows the precise location of the discontinuity. (

**b**) Difference in response from the baseline to the test case. This figure shows that the largest difference in response due to a discontinuity does not determine the location of the peak IIF value. (

**c**) $II{F}^{R}$ values for three different window lengths of the STFT. The difference in time resolution and possible leaking in the STFT creates uncertainty in the exact peak location.

**Figure 5.**Shannon entropy values as a function of background excitation and discontinuity amplitude. The separation in the discontinuity amplitude test cases is not clear.

**Figure 6.**Permutation entropy values as a function of background excitation and discontinuity amplitude. There is no discernable separation in the discontinuity amplitude for most test cases.

**Figure 7.**IIF peak values as a function of background excitation and discontinuity amplitude. Clear separation in the discontinuity amplitude test cases shows how well the IIF is able to detect a global disturbance even when the system is chaotic. The separation between amplitude test cases is in stark contrast to the SE and PEn results shown in Figure 5 and Figure 6.

**Figure 8.**Shannon entropy against the Poincaré distance. The shape the SE values make as a function of excitation force mimics the Poincaré distance with similar inflection points near regions where the system behavior changes character.

**Figure 9.**Permutation entropy against the Poincaré distance. The shape the PEn values make as a function of excitation force also mimics the Poincaré distance with similar inflection points. However, the shape loses form past F = 0.3.

**Figure 10.**IIF peaks against the Poincaré distance. The shape the IIF peak values make as a function of excitation force mimics the Poincaré distance similarly to both the SE and PEn results, but with a smoother curve.

**Figure 11.**Shannon entropy values as a function of background excitation and discontinuity amplitude for case (ii). Inexplicably, values near F = 0.17 seem to show an increased entropy at 10% over lower values. Despite some small differences, these results closely follow those reported in Figure 6 for case (i).

**Figure 12.**Permutation entropy values as a function of background excitation and discontinuity amplitude for case (ii). Like SE, these results closely follow those reported in Figure 7 for case (i).

**Figure 15.**IIF peak values as a function of background excitation and discontinuity amplitude for case (ii). The separation in the discontinuity amplitude is apparent. The shape of the curves is very different from the global disturbance case due to the difference in sensitivity of the system to changes in stiffness. This was not the case for either SE or PEn.

**Figure 16.**IIF peak values as a function of background excitation and discontinuity amplitude for case (iii). The separation in the discontinuity amplitude is apparent. The shape of the curves is very different from the global disturbance case and case (ii) due to the difference in sensitivity of the system to changes in damping. The difference in shape between case reported in Figure 7 for case (i), Figure 15 for case (ii) and the current image for case (iii) results from the sensitivity of the IIF to the response of the system.

**Figure 17.**Difference values for cases (i), (ii), and (iii) as a function of excitation force. (

**a**) SE results (

**b**) PEn3 results (

**c**) IIF results. The low values in (

**a**) and (

**b**) contrasted with the larger, consistently positive values in (

**c**) show that the IIF is much better at distinguishing between the discontinuity levels.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Montoya, A.; Habtour, E.; Moreu, F. Quantifying Information without Entropy: Identifying Intermittent Disturbances in Dynamical Systems. *Entropy* **2020**, *22*, 1199.
https://doi.org/10.3390/e22111199

**AMA Style**

Montoya A, Habtour E, Moreu F. Quantifying Information without Entropy: Identifying Intermittent Disturbances in Dynamical Systems. *Entropy*. 2020; 22(11):1199.
https://doi.org/10.3390/e22111199

**Chicago/Turabian Style**

Montoya, Angela, Ed Habtour, and Fernando Moreu. 2020. "Quantifying Information without Entropy: Identifying Intermittent Disturbances in Dynamical Systems" *Entropy* 22, no. 11: 1199.
https://doi.org/10.3390/e22111199