Entropy of Difference

Here, we propose a new tool to estimate the complexity of a time series: the entropy of difference (ED). The method is based solely on the sign of the difference between neighboring values in a time series. This makes it possible to describe the signal as efficiently as prior proposed parameters such as permutation entropy (PE) or modified permutation entropy (mPE), but (1) reduces the size of the sample that is necessary to estimate the parameter value, and (2) enables the use of the Kullback-Leibler divergence to estimate the distance between the time series data and random signals.


I. INTRODUCTION
Permutation entropy (PE), introduced by Bandt and Pompe [1], as well as its modified version [2], are both efficient tools to measure the complexity of chaotic time series.Both methods propose to analyze time series: choosing an embedding dimension m to split the original data in a subset of m-tuples: , then to substitute to the m-tuples values by the rank of the values, resulting in a new symbolic representation of the time series.For example, consider the time series X = {0.2,0.1, 0.6, 0.4, 0.1, 0.2, 0.4, 0.8, 0.5, 1., 0.3, 0.1, • • •}.

II. ENTROPY OF DIFFERENCE-METHOD
The entropy of difference (ED) method proposes to substitute to the mtuples with strings s containing the sign ("+" or "-"), representing of the difference between subsequent elements in the m-tuples.
For the same X 4 : {{0.2, 0.1, 0.6, 0.4}, {0.1, 0.6, 0.4, 0.1}, {0.6, 0.4, 0.1, 0.2}, • • •} this leads to the representation For an m value, we have 2 m−1 strings from Again we compute, in the time series, the probability distribution p m (s) of these strings s and define the entropy of difference of order m as : The number of elements: K m to be treated, for an embedding m, are smaller for ED compared with the number of permutations π in PE or to the elements in mPE (see table I).
Furthermore the probability distribution for a string s, in a random signal : q m (s) is not constant and could be computed through the recursive equation[15] (in the following equations x and y are strings): q(+, +, +, leading to a complex probability distribution.For example for m = 9 we have 2 8 = 256 strings with the highest probability for the " + − + − + − + −" string (and its symmetric " − + − + − + − +"): q 9 (max) = 62 2835 ≈ 0.02187 (see Fig. I).These probabilities q m (s) could then be used to determine the KL-divergence between the time series probability p m (s) and the random signal.Despite the complexity of q m (s), the Shannon entropy for a random signal : − s q m (s) log 2 q m (s) increases linearly with m, with a slope ≈ 0.905.

III. CHAOTIC LOGISTIC MAP EXAMPLE
Let us illustrate the use of ED on the well know logistic map [7] Lo(x, λ) driven by the parameter λ.
It is obvious that for a range of values of λ where the time series reaches a periodic behavior (any cyclic oscillation between n different values), the ED will remain constant.The evaluation of the ED could thus be used as a new complexity parameter to determine the behavior of the time series (see FIG. 3).
For λ = 4 we know that the data are randomly distributed with a probability density given by [5] p Lo (x) = 1 We can then compute exactly the ED for an m-embedding, and the KL-divergence from a random signal.For example, for m = 2, we can determine the p + and p − by solving the inequality x < Lo(x) and x > Lo(x) respectively which implies that 0 < x < 3/4 and 3/4 < x < 1, and then In this case the logistic map produces a signal that contains twice as many increasing pairs " + " than decreasing pairs " − ".So: For m = 3 and m = 4 we can perform the same calculation: Effectively the logistic map with λ = 4 forbids the string "--" where x 1 > x 2 > x 3 .For strings of length 3 we also have also the non zero values: The probability of difference p m (s) for some string length m versus s the string binary value, where "+"→ 1 and "-"→ 0, give us the "spectrum of difference" for the distribution p (see FIG. 4)., KL m (bel20) = 0.1587 − 0.0886 m + 0.0182 m 2 with a higher curvature than the logistic map due to the fact that the spectrum of the probability p m is compatible with a constant distribution (see FIG. 6) rendering the prediction of increase or decrease signal completely random, which is not the case in any true random signal.The probability distribution of string q m for random signal is used to evaluate the Kullback-Leibler divergence versus the number of data m used to build the difference string.This KL m shows different behavior for different types of signal and can also be used also to characterize the complexity of a time series.

10 FIG. 2 :
FIG.2:The Shannon entropy of q m (s) increases linearly with m, the fit −0.799574 + 0.905206 m gives a sum of squared residuals of 1.7 10 −4 and a p-value=1.5710 −12 and 1.62 10 −30 on the fit parameter respectively.

FIG. 3 :
FIG.3:The ED 13 (strings of length 12) is plotted versus λ, with the bifurcation diagram, and the value of the Lyapunov exponent respectively.The constant value appears when the logistic map enter into a periodic regime.

FIG. 4 :
FIG.4:The spectrum of p 13 versus the string binary value (from 0 to 2 12 − 1) for the logistic map at λ = 4 and the one from a random distribution q 13

TABLE I :
K values, for different m-embedding