# Permutation Entropy of Weakly Noise-Affected Signals

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Permutation Entropy and Its Noise-Induced Scaling Behavior

## 3. Multifractal Analysis

## 4. Assessing the Entropy Increase via Distribution Reconstruction

## 5. Discussion and Open Problems

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Keller, K.; Sinn, M. Ordinal analysis of time series. Phys. A Stat. Mech. Appl.
**2005**, 356, 114–120. [Google Scholar] [CrossRef] - Bandt, C.; Pompe, B. Permutation Entropy: A Natural Complexity Measure for Time Series. Phys. Rev. Lett.
**2002**, 88, 174102. [Google Scholar] [CrossRef] - Politi, A. Quantifying the Dynamical Complexity of Chaotic Time Series. Phys. Rev. Lett.
**2017**, 118, 144101. [Google Scholar] [CrossRef] [Green Version] - Watt, S.J.; Politi, A. Permutation entropy revisited. Chaos Soliton. Fract.
**2019**, 120, 95–99. [Google Scholar] [CrossRef] [Green Version] - Rosso, O.A.; Larrondo, H.A.; Martin, M.T.; Plastino, A.; Fuentes, M.A. Distinguishing Noise from Chaos. Phys. Rev. Lett.
**2007**, 99, 154102. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Ricci, L. Asymptotic distribution of sample Shannon entropy in the case of an underlying finite, regular Markov chain. Phys. Rev. E
**2021**, 103, 022215. [Google Scholar] [CrossRef] - Ricci, L.; Perinelli, A.; Castelluzzo, M. Estimating the variance of Shannon entropy. Phys. Rev. E
**2021**, 104, 024220. [Google Scholar] [CrossRef] - Tél, T. Fractals, Multifractals, and Thermodynamics: An Introductory Review. Z. Naturforschung A
**1988**, 43, 1154–1174. [Google Scholar] [CrossRef] - Cicirello, V.A. Classification of Permutation Distance Metrics for Fitness Landscape Analysis. In Bio-Inspired Information and Communication Technologies. BICT 2019. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering; Compagnoni, A., Casey, W., Cai, Y., Mishra, B., Eds.; Springer: Cham, Switerland, 2019; Volume 289. [Google Scholar] [CrossRef]
- Bulteau, L.; Fertin, G.; Rusu, I. Sorting by Transpositions is Difficult. SIAM J. Discret. Math.
**2012**, 26, 1148. [Google Scholar] [CrossRef] [Green Version] - Ronald, S. More Distance Functions for Order-Based Encodings. In Proceedings of the 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360), Anchorage, AK, USA, 4–9 May 1998; pp. 558–563. [Google Scholar] [CrossRef]

**Figure 1.**(

**a**) According to the rule described in the main text, a sample noiseless trajectory $(1.1,\phantom{\rule{0.166667em}{0ex}}3,\phantom{\rule{0.166667em}{0ex}}0.4,\phantom{\rule{0.166667em}{0ex}}0.9,\phantom{\rule{0.166667em}{0ex}}5)$ of dimension $m=5$ is encoded as $(3,4,1,2,5)$. The additive contribution of noise, e.g., $(-0.6,\phantom{\rule{0.166667em}{0ex}}0.3,\phantom{\rule{0.166667em}{0ex}}-1.2,\phantom{\rule{0.166667em}{0ex}}0.9,\phantom{\rule{0.166667em}{0ex}}0.12)$ as shown in (

**b**), yields the noise-affected trajectory (

**c**) given by $(0.5,\phantom{\rule{0.166667em}{0ex}}3.3,\phantom{\rule{0.166667em}{0ex}}-0.8,\phantom{\rule{0.166667em}{0ex}}1.8,\phantom{\rule{0.166667em}{0ex}}5.12)$, whose encoding is $(3,1,4,2,5)$.

**Figure 2.**Permutation entropy variation $\mathsf{\Delta}H$ as a function of the effective noise $\mathsf{\Sigma}=\sigma {m}^{\alpha}$ in the case of (

**a**) Hénon map, (

**b**) generalized 3D Hénon map, (

**c**) tent map, and (

**d**) Mackey–Glass model. The exponent $\alpha $ is 2 for the Hénon map and 2.5 for the three other dynamical systems. In each case, the line colors, ordered according to a rainbow palette, refer to different values of m, from 5 (red) to 11 (blue). The magenta line corresponds to the result of a fit of Equation (2) to all curves at different values of m. The line is upward displaced for graphical reasons, and its projection on the horizontal axis corresponds to the range used for the fit procedure. While the power law k exponents are reported on the respective graphs, the factors b are equal to −1.1, −0.4, −1.5, and −1.5, respectively.

**Figure 3.**Histograms, computed according to Equation (3), of the observed, nonvanishing probabilities of the different words for different noise amplitudes and obtained by setting $m=10$: (

**a**) Hénon map, (

**b**) tent map, (

**c**) generalized 3D Hénon map, (

**d**) Mackey–Glass model. For each dynamical system, the red line corresponds to the noiseless histogram, whereas the other lines correspond to histograms computed in the presence of different noise amplitudes $\sigma $. The line color palette veers from green, corresponding to $\sigma ={10}^{-5}$, to blue, corresponding to $\sigma ={10}^{-1}$, with multiplicative steps of $\sigma ={10}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}\approx 3.2$. The bin width is 0.8. Each gray area shows the histogram when the symbolic sequences are equiprobable so that the bin including $log{p}_{W}=-logm!\approx -15.1$ becomes populated with $10!\approx 3.6\times {10}^{6}$ symbolic sequences. This situation occurs when the dynamics is purely stochastic and thus corresponds to the distribution with $\sigma \to \infty $.

**Figure 4.**Bin population ${N}_{i}$ of the respective histograms of Figure 3 multiplied times $exp\left(\u2329log{p}_{i}\u232a\right)$, namely the probability ${p}_{W}$ corresponding to the center of the i-th logarithmic scale bin: (

**a**) Hénon map, (

**b**) tent map, (

**c**) generalized 3D Hénon map, (

**d**) Mackey–Glass model. Line and color notation is the same as explained in the caption to Figure 3. Similarly, each gray area shows the histogram when the symbolic sequences are equiprobable: the bin including $log{p}_{W}=-logm!\approx -15.1$ thus becomes populated with $10!$ symbolic sequences, each having a probability ${(10!)}^{-1}$ so that $Np=1$. This situation occurs when the dynamics is purely stochastic and thus corresponds to the distribution with $\sigma \to \infty $. For the noise amplitudes taken into account, the tent map is apparently the fastest to approach that limit behavior.

**Figure 5.**Results of the procedure of supervised distribution reconstruction: (

**a**) Hénon map, (

**b**) tent map, (

**c**) generalized 3D Hénon map, (

**d**) Mackey–Glass model. For each dynamical system, the black line corresponds to the plot $\mathsf{\Delta}H(m,\sigma )$ that, as a function of $\mathsf{\Sigma}$, is also reported in Figure 2. The orange line refers instead to the difference $\mathsf{\Delta}{H}^{\prime}(m,\sigma )$ between ${H}^{\prime}(m,\sigma )$ and $H(m,0)$, namely the entropy evaluated on the reconstructed histogram and the noiseless entropy, respectively. The related dots are colored in red (blue) if they represent a positive (negative) value of $\mathsf{\Delta}{H}^{\prime}(m,\sigma )$. The value of the reconstruction efficiency $\eta $, evaluated via Equation (4) is also reported. The number in parentheses corresponds to the related uncertainty on the least significant digit.

**Figure 6.**Escape parameter $\rho $, defined in Equation (6), as a function of the noise amplitude $\sigma $: (red) Hénon map, (orange) tent map, (blue) generalized 3D Hénon map, (magenta) Mackey–Glass model.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Ricci, L.; Politi, A.
Permutation Entropy of Weakly Noise-Affected Signals. *Entropy* **2022**, *24*, 54.
https://doi.org/10.3390/e24010054

**AMA Style**

Ricci L, Politi A.
Permutation Entropy of Weakly Noise-Affected Signals. *Entropy*. 2022; 24(1):54.
https://doi.org/10.3390/e24010054

**Chicago/Turabian Style**

Ricci, Leonardo, and Antonio Politi.
2022. "Permutation Entropy of Weakly Noise-Affected Signals" *Entropy* 24, no. 1: 54.
https://doi.org/10.3390/e24010054