# Characterizing Complex Spatiotemporal Patterns from Entropy Measures

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Methods

#### 2.1. Probabilities

#### 2.2. Entropic Forms

#### 2.3. Silhouette Score and Generalized Silhouette Score

## 3. Data

## 4. Results and Interpretation

- Input of a snapshot;
- Pre-processing for which its output is a $64\times 64$ matrix with amplitudes ranging from 0 to 255;
- Generation of three matrix data outputs: 2D histogram, 2D permutation, and 2D FFT spectra;
- For each of the three domains, the entropy measures are calculated.

## 5. Outlook

## 6. Concluding Remarks

## Author Contributions

## Funding

## Institutional Review Board Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Appendix A. Gradient Pattern Analysis

## Appendix B. Two-Dimensional Permutation Entropy

- Step 1: Obtain the coarse-grained image as an $N\times N$ matrix;
- Step 2: Apply a window of size $d\times d$ to it;
- Step 3: Carry out $d!$ reshape permutations to obtain the probabilities of each local pattern;
- Step 4: Repeat the last procedure, scanning the entire matrix;
- Step 5: Apply the probability values as input to the chosen entropy formula.

## References

- Tsallis, C. When may a system be referred to as complex? An entropic perspective. Front. Complex Syst.
**2023**, 1, 1284458. [Google Scholar] [CrossRef] - Zhang, A.; Yang, B.; Huang, L. Feature Extraction of EEG Signals Using Power Spectral Entropy. In Proceedings of the International Conference on BioMedical Engineering and Informatics, Sanya, China, 27–30 May 2008. [Google Scholar]
- Raja Abdullah, R.S.A.; Saleh, N.L.; Syed Abdul Rahman, S.M.; Zamri, N.S.; Abdul Rashid, N.E. Texture classification using spectral entropy of acoustic signal generated by a human echolocator. Entropy
**2019**, 21, 963. [Google Scholar] [CrossRef] - Mattedi, A.P.; Ramos, F.M.; Rosa, R.R.; Mantegna, R.N. Value-at-risk and Tsallis statistics: Risk analysis of the aerospace sector. Phys. A Stat. Mech. Its Appl.
**2004**, 344, 554–561. [Google Scholar] [CrossRef] - Ramos, F.M.; Rosa, R.R.; Rodrigues Neto, C.; Bolzan, M.J.A.; Abren Sá, L.D. Nonextensive thermostatistics description of intermittency in turbulence and financial markets. Nonlinear Anal. Theory Methods Appl.
**2001**, 47, 3521–3530. [Google Scholar] [CrossRef] - Ramos, F.M.; Bolzan, M.J.A.; Abreu Sá, L.D.; Rosa, R.R. Atmospheric turbulence within and above an Amazon forest. Phys. D Nonlinear Phenom.
**2004**, 193, 278–291. [Google Scholar] [CrossRef] - Brissaud, J. The meanings of entropy. Entropy
**2005**, 7, 68–96. [Google Scholar] [CrossRef] - Gotoh, T.; Kraichnan, R.H. Turbulence and Tsallis statistics. Physica D
**2004**, 193, 231–244. [Google Scholar] [CrossRef] - Abdelsamie, A.; Janiga, G.; Thévenin, D. Spectral entropy as a flow state indicator. Int. J. Heat Fluid Flow
**2017**, 68, 102–113. [Google Scholar] [CrossRef] - Mignone, A.; Bodo, G.; Massaglia, S.; Matsakos, T.; Tesileanu, O.; Zanni, C.; Ferrari, A. PLUTO: A numerical code for computational astrophysics. ApJS
**2007**, 170, 228–242. [Google Scholar] [CrossRef] - Franci, L.; Hellinger, P.; Guarrasi, M.; Chen, C.H.K.; Papini, E.; Verdini, A.; Matteini, L.; Landi, S. Three-dimensional simulations of solar wind turbulence. J. Phys. Conf. Ser.
**2018**, 1031, 012002. [Google Scholar] [CrossRef] - The Pencil Code Collaboration; Brandenburg, A.; Johansen, A.; Bourdin, P.A.; Dobler, W.; Lyra, W.; Rheinhardt, M.; Bingert, S.; Haugen, N.E.L.; Mee, A.; et al. The Pencil Code, a modular MPI code for partial differential equations and particles: Multipurpose and multiuser-maintained. J. Open Source Softw.
**2021**, 6, 2807. [Google Scholar] - Veronese, T.B.; Rosa, R.R.; Bolzan, M.J.A.; Rocha Fernandes, F.C.; Sawant, H.S.; Karlicky, M. Fluctuation analysis of solar radio bursts associated with geoeffective X-class flares. J. Atmos. Sol.-Terr. Phys.
**2011**, 73, 1311–1316. [Google Scholar] [CrossRef] - Bolzan, M.J.A.; Sahai, Y.; Fagundes, P.R.; Rosa, R.R.; Ramos, F.M.; Abalde, J.R. Intermittency analysis of geomagnetic storm time-series observed in Brazil. J. Atmos. Sol.-Terr. Phys.
**2005**, 67, 1365–1372. [Google Scholar] [CrossRef] - Lu, H.; Lü, S.J.; Zhang, M.J. Fourier spectral approximations to the dynamics of 3D fractional complex Ginzburg-Landau equation. Discret. Contin. Dyn. Syst.
**2017**, 37, 2539–2564. [Google Scholar] [CrossRef] - Pessa, A.A.B.; Ribeiro, H.V. ordpy: A Python package for data analysis with permutation entropy and ordinal network methods. Chaos Interdiscip. J. Nonlinear Sci.
**2021**, 31, 063110. [Google Scholar] [CrossRef] - Xiong, P.Y.; Jahanshahi, H.; Alcarazc, R.; Chud, Y.M.; Gómez-Aguilar, J.F.; Alsaadi, F.E. Spectral Entropy Analysis and Synchronization of a Multi-Stable Fractional-Order Chaotic System using a Novel Neural Network-Based Chattering-Free Sliding Mode Technique. Chaos Solitons Fractals
**2021**, 144, 110576. [Google Scholar] [CrossRef] - Ramos, F.M.; Rosa, R.R.; Rodrigues Neto, C.; Zanandrea, A. Generalized complex entropic form for gradient pattern analysis of spatio-temporal dynamics. Physica A
**2000**, 283, 171–174. [Google Scholar] [CrossRef] - Tsallis, C. Possible generalization of Boltzmann-Gibbs statistics. J. Stat. Phys.
**1998**, 52, 479–487. [Google Scholar] [CrossRef] - Rosa, R.R.; de Carvalho, R.R.; Sautter, R.A.; Barchi, P.H.; Stalder, D.H.; Moura, T.C.; Rembold, S.B.; Morell, D.R.F.; Ferreira, N.C. Gradient pattern analysis applied to galaxy morphology. Mon. Not. R. Astron. Soc. Lett.
**2018**, 477, L101–L105. [Google Scholar] [CrossRef] - Lesne, A. Shannon entropy: A rigorous notion at the crossroads between probability, information theory, dynamical systems and statistical physics. Math. Struct. Comput. Sci.
**2014**, 24, e240311. [Google Scholar] [CrossRef] - Li, C.; Shang, P. Multiscale Tsallis permutation entropy analysis for complex physiological time series. Physica A
**2019**, 523, 10–20. [Google Scholar] [CrossRef] - Kaufman, L.; Rousseeuw, P. Finding Groups in Data: An Introduction to Cluster Analysis; Wiley and Sons: Hoboken, NJ, USA, 2005. [Google Scholar]
- Shutaywi, M.; Kachouie, N.N. Silhouette analysis for performance evaluation in machine learning with applications to clustering. Entropy
**2021**, 23, 759. [Google Scholar] [CrossRef] [PubMed] - Timmer, J.; Koenig, M. On generating power law noise. Astron. Astrophys.
**1995**, 300, 707. [Google Scholar] - Sautter, R.A. Gradient Pattern Analysis: Enhancements and Applications Including the Influence of Noise on Pattern Formation. Ph.D. Thesis, National Institute for Space Research, São José dos Campos, Brazil, 2023. [Google Scholar]
- Sautter, R.; Rosa, R.; Pontes, J. Incremental Gradient Pattern Analysis of Stochastic Complex Ginzburg-Landau Dynamics; ResearchGate: Berlin, Germany, 2023. [Google Scholar] [CrossRef]
- Li, Y.; Perlman, E.; Wan, M.; Yang, Y.; Meneveau, C.; Burns, R.; Chen, S.; Szalay, A.; Eyink, E. A public turbulence database cluster and applications to study Lagrangian evolution of velocity increments in turbulence. J. Turbul.
**2008**, 9, N31. [Google Scholar] [CrossRef] - Bandt, C.; Pompe, B. Permutation entropy: A natural complexity measure for time series. Phys. Rev. Lett.
**2002**, 88, 174102. [Google Scholar] [CrossRef] [PubMed] - Silva, L.; Duque, J.; Felipe, J.; Murta, L.; Humeau-Heurtier, A. Two-dimensional multiscale entropy analysis: Applications to image texture evaluation. Signal Process.
**2018**, 147, 224–232. [Google Scholar] [CrossRef] - Humeau-Heurtier, A.; Omoto, A.C.M.; Silva, L.E. Bi-dimensional multiscale entropy: Relation with discrete Fourier transform and biomedical application. Comput. Biol. Med.
**2018**, 100, 36–40. [Google Scholar] [CrossRef] - Morel, C.; Humeau-Heurtier, A. Multiscale permutation entropy for two-dimensional patterns. Pattern Reg. Lett.
**2021**, 150, 139–146. [Google Scholar] [CrossRef]

**Figure 1.**Snapshots of the spatiotemporal evolution of each selected system class, listed in Table 2. Each row shows one of the simulations, rendered at time steps that show representative pattern dynamics: dynamic white noise ($\beta =0$ represented by colormap

`‘Blues’`) on the 1st row; random red noise ($\beta =2$, represented by colormap

`‘Reds’`) on the 2nd row; weak turbulence from the reaction–diffusion complex Ginzburg–Landau dynamics on the 3rd row (represented by colormap

`‘viridis’`); fully developed turbulence from JHTDB on the 4th row (represented on colormap

`‘rainbow’`) and MHD turbulence from PENCIL on the 5th row (represented by colormap

`‘cool’`).

**Figure 2.**Generalized silhouette score for all 2D metric combinations. Higher values on the heatmap indicate superior metric performance. The optimal result is achieved with the pairing of spectral Tsallis entropy and Shannon permutation entropy (${S}_{q}^{s}\times {S}_{H}^{p}$).

**Figure 3.**Optimal outcomes achieved are assessed through the generalized silhouette score criterion. The method achieves its best performance in the (${S}_{q}^{s}\times {S}_{H}^{p}$) parameter space.

**Figure 4.**Best entropy set according to the generalized silhouette score (see Figure 2) for the 3D-CGL solution over time, where the oscillatory dynamic of the system is highlighted. The color indicates the snapshot, where 500 samples are presented.

**Figure 5.**Pipeline of the method proposed in this study based on the best results found: A sequence of snapshots from the simulation of a given process (in the $2D+1$ or $3D+1$ domains) comprises the input from which entropy measurements will be obtained. To calculate the respective Shannon permutation entropy values ${S}_{H}^{p}$, the permutation values are obtained (see Appendix B). To calculate the spectral Tsallis entropy ${S}_{q}^{s}$, the respective spectra are obtained. From the calculated values, the parameter space is constructed where where it is proposed to characterize the underlying process. The space also works for classifying isolated patterns, taking as reference the distinct processes that have already been characterized.

Measure | Probability | Entropic Form | Reference |
---|---|---|---|

${S}_{H}^{h}$ | histogram | Shannon, Equation (3) | Lesne [21] |

${S}_{H}^{p}$ | permutation | Shannon, Equation (3) | Pessa, Ribeiro [16] |

${S}_{H}^{s}$ | spectral | Shannon, Equation (3) | Abdelsamie et al. [9], Abdullah et al. [3] |

${S}_{q}^{h}$ | histogram | Tsallis q-law, Equation (4) | Li and Shang [22] |

${S}_{q}^{p}$ | permutation | Tsallis q-law, Equation (4) | Li and Shang [22] |

${S}_{q}^{s}$ | spectral | Tsallis q-law, Equation (4) | This paper |

${G}_{4}$ | gradient | Complex Shannon, Equation (5) | Ramos et al. [18] |

Simulation | Process | Reference |
---|---|---|

White Dynamic Noise | Spatiotemporal stochastic | Timmer et al. [25] |

Red Dynamic Noise | Spatiotemporal stochastic | Timmer et al. [25] |

CGL ^{1} | Weak turbulence | Sautter [26], Sautter et al. [27] |

JHTDB | Fully developed turbulence | Brandenburg et al. [12] |

PENCIL | MHD turbulence | Brandenburg et al. [12] |

^{1}Our 3D simulator is public available at https://github.com/rsautter/Noisy-Complex-Ginzburg-Landau (14 January 20224).

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Barauna, L.O.; Sautter, R.A.; Rosa, R.R.; Rempel, E.L.; Frery, A.C.
Characterizing Complex Spatiotemporal Patterns from Entropy Measures. *Entropy* **2024**, *26*, 508.
https://doi.org/10.3390/e26060508

**AMA Style**

Barauna LO, Sautter RA, Rosa RR, Rempel EL, Frery AC.
Characterizing Complex Spatiotemporal Patterns from Entropy Measures. *Entropy*. 2024; 26(6):508.
https://doi.org/10.3390/e26060508

**Chicago/Turabian Style**

Barauna, Luan Orion, Rubens Andreas Sautter, Reinaldo Roberto Rosa, Erico Luiz Rempel, and Alejandro C. Frery.
2024. "Characterizing Complex Spatiotemporal Patterns from Entropy Measures" *Entropy* 26, no. 6: 508.
https://doi.org/10.3390/e26060508