On Rényi Permutation Entropy

Abstract

Among various modifications of the permutation entropy defined as the Shannon entropy of the ordinal pattern distribution underlying a system, a variant based on Rényi entropies was considered in a few papers. This paper discusses the relatively new concept of Rényi permutation entropies in dependence of non-negative real number q parameterizing the family of Rényi entropies and providing the Shannon entropy for $q=1$. Its relationship to Kolmogorov–Sinai entropy and, for $q=2$, to the recently introduced symbolic correlation integral are touched.

1. Introduction

1.1. Paper Background and Motivation

Since Bandt and Pompe [1] introduced the concept of permutation entropy (PE), it has been applied in different fields from biomedicine to econophysics (e.g., Zanin et al. [2], and Amigò et al. [3]) and developed into various directions. One relatively new variant of permutation entropy is based on Rényi entropies instead of the originally used Shannon entropy and is called Rényi permutation entropy (RPE). Roughly speaking, RPE quantifies the complexity of the distribution of ordinal patterns of some length n underlying a dynamical system, where ordinal patterns describe the up and down in the dynamics. As Rényi entropies depend on a parameter $q\in [0,\infty ]$, there are also different choices of RPE depending on q.

The central aim of the paper is to discuss the asymptotics of RPE for increasing pattern length. This is motivated by the striking fact that, under certain assumptions, asymptotic PE is equal to Kolmogorov–Sinai entropy, which was first observed by Bandt et al. [4]. This paper shows that the situation for $q\ne 1$ is more complicated than that for $q=1$.

The paper is organized as follows. It first follows a short overview of first applications of the RPE. Section 2 provides the main definitions. The concepts of RPE are introduced in empirical and model-based settings. Moreover, RPE is discussed for some special q, including $q=\infty$ as a limit case. Section 3 is devoted to the asymptotics of RPE and PE. With Corollary 1, the section contains the main new result of the paper relating RPE to Kolmogorov–Sinai entropy for $q\in [0,1]$ and measures with maximal entropy. Its proof and a class of discriminating examples for $q>1$ (see Example A1) are given in Appendix A.

1.2. First Applications of RPE

To our best knowledge, the concept of RPE was first considered in the literature in 2015. In a study of monitoring the depth of anaesthesia by EEG, Liang et al. [5] systematically compared 12 entropy measures, with RPE among them. They reported that RPE had the best performance in distinguishing different anaesthesia states. Mammone et al. [6] discussed RPE in the context of absence epilepsy EEG. Their results suggested improved abilities in classifying ictal and interictal EEG by using RPE (with suitable parameters) instead of PE. Zunino et al. [7] introduced permutation min entropy, which is the limit of Rényi entropy for defining parameter q approaching to ∞ as a tool for finding temporal correlations in a time series.

Moreover, Rivero et al. [8] combined an enhanced Bayesian approach and RPE for predicting long-term time series. Following the results of Liang et al., Park et al. [9] used RPE for comparing anaesthetics given during a cesarean section with similar results as those for other entropy measures. Different variants of RPE, from weighting to multiscaling, have been applied to complex stock-market data (Zhou and Shang [10], and Chen et al. [11]). Some remarks on RPE can also be found in [12].

2. Rényi Entropies

2.1. General Entropy Concept

Given a finite index set I consisting of n elements, ${\left(p_i\right)}_{i\in I}\in {\mathbb{R}}^n$ is called a stochastic vector if $p_i\ge 0$ for all $i\in I$ and ${\sum }_{i\in I}{p}_i=1$. The Rènyi entropy $\mathrm{RE}({\left(p_i\right)}_{i\in I},q)$ of a stochastic vector ${\left(p_i\right)}_{i\in I}$ for $q\in [0,\infty ]$ is defined by

$$\begin{array}{c}\hfill \mathrm{RE}({\left(p_i\right)}_{i\in I},q)= \left\{\begin{array}{cc}\hfill -\sum _{i\in I}{p}_{i}log{p}_i& \mathrm{for} q=1\hfill \\ \hfill \frac{1}{1-q}log\sum _{i\in I}{p}_i^q& \mathrm{else}\hfill \end{array}\right..\end{array}$$

The Rényi entropy of a fixed stochastic vector monotonically decreases and is continuous with respect to q. It generalizes the Shannon entropy given in the standard case $q=1$. The larger that q is, the more the role of the largest entries in the stochastic vector is emphasized, and the smaller that q is, the more equal the role of all positive entries in the entropy formula is.

On the basis of the concept of Rényi entropies, we want to give precise definitions of RPE regarding both the empirical and the modelling viewpoint.

2.2. Empirical RPE

For $n\in \mathbb{N}$, we denote the set of permutations of $\{0,1,\dots ,n-1\}$ by ${\mathfrak{S}}_n$. A vector $(v_0,v_1,\dots ,v_{n-1})\in {\mathbb{R}}^n$ has ordinal pattern $\pi =({\pi }_0,{\pi }_1,\dots ,{\pi }_{n-1})\in {\mathfrak{S}}_n$ if

$$\begin{array}{c}\hfill v_{{\pi }_0}\ge v_{{\pi }_1}\dots \ge v_{{\pi }_{n-2}}\ge v_{{\pi }_{n-1}}\end{array}$$

and

$$\begin{array}{c}\hfill r_{l-1}>r_l if v_{r_{l-1}}=v_{r_l}.\end{array}$$

The latter requirement realises the uniqueness of ordinal patterns.

Definition 1.

The empirical Rényi permutation entropy for $q\in [0,\infty ]$ and $n\in \mathbb{N}$ of a time series ${\left(x_t\right)}_{t=0}^{N-1}$ is defined by

$$\begin{array}{ccc}\hfill \mathrm{ePE}\left({\left(x_t\right)}_{t=0}^{N-1},q,n\right)& =& \mathrm{RE}({\left(p_{\pi }\right)}_{\pi \in {\mathfrak{S}}_n},q)\hfill \\ \hfill & =& \left\{\begin{array}{cc}\hfill -\sum _{\pi \in {\mathfrak{S}}_n}{p}_{\pi }log{p}_{\pi }& \mathrm{for} q=1\hfill \\ \hfill \frac{1}{1-q}log\left(\sum _{\pi \in {\mathfrak{S}}_n}{p}_{\pi }^q\right)& \mathrm{else}\hfill \end{array}\right.\hfill \end{array}$$

with

$$\begin{array}{ccc}\hfill p_{\pi }^{\tau }& =& \frac{\#\{t\in \{0,1,\dots ,N-n\}\mid (x_t,x_{t+1},\dots ,x_{t+n-1})\mathrm{has}\mathrm{ordinal}\mathrm{pattern}\pi \}}{N-n+1}\hfill \end{array}$$

being the relative frequency of ordinal patterns π in the time series, and $0log0$ and $0^0$ being defined by 0.

2.3. RPE

On the model side, we consider a measure-preserving dynamical system $(\Omega ,\mathcal{A},\mu ,T)$, defined as a probability space $(\Omega ,\mathcal{A},\mu )$ being equipped with a $\mathcal{A}$-$\mathcal{A}$-measurable map $T:\Omega \to \Omega$ which satisfies $\mu \left(T^{-1}\left(A\right)\right)=\mu \left(A\right)$ for all $A\in \mathcal{A}$. T and the system $(\Omega ,\mathcal{A},\mu ,T)$ are called ergodic if $T^{-1}\left(A\right)=A$ for $A\in \mathcal{A}$ implies $\mu \left(A\right)\in \{0,1\}$.

Generally, the dynamics of T can be related to ordinal patterns via a real-valued random variable X on $\Omega$ by assigning $\omega \in \Omega$ the ordinal pattern of $(X\left(\omega \right),X\left(T\left(\omega \right)\right),\dots ,X\left(T^{n-1}\left(\omega \right)\right)$. Here, X is interpreted as observable modelling of a measuring process. If X (or, more generally, a collection of random variables) has certain separation properties, the ordinal patterns obtained via X (or all random variables) contain much information on the given system. In the following, however, we usually assume that $\Omega$ is a subset of $\mathbb{R}$, and the ordinal pattern $\Pi \left(x\right)$ assigned to $x\in \Omega$ is that taken from $(x,T\left(x\right),\dots ,T^n\left(x\right))$ (this is equivalent to considering an X being the identity map).

For a permutation $\pi =({\pi }_0,{\pi }_1,\cdots ,{\pi }_{n-1})\in {\mathfrak{S}}_n$, we denote the sets of all points x with ordinal pattern $\Pi \left(x\right)=\pi$ by $P_{\pi }$. From these sets, we obtain the partition

$$O{P}^{}\left(n\right)=\{P_{\pi }\ne Ø\mid \pi \in {\mathfrak{S}}_n\}$$

of $\Omega$, with this being central to considering RPE on the model side. Analogously to empirical permutation entropy, we define Rényi permutation entropy for $q\in [0,\infty ]$ and $n\in \mathbb{N}$ on the basis of $O{P}^{}\left(n\right)$ by

$$\begin{array}{ccc}\hfill \mathrm{PE}\left(T,q,n\right)& =& \mathrm{RE}({\left(\mu \left(P_{\pi }\right)\right)}_{\pi \in {\mathfrak{S}}_n},q)\hfill \\ \hfill & =& \left\{\begin{array}{cc}\hfill -\sum _{\pi \in {\mathfrak{S}}_n}\mu \left(P_{\pi }\right)log\mu \left(P_{\pi }\right)& \mathrm{for} q=1\hfill \\ \hfill \frac{1}{1-q}log\left(\sum _{\pi \in {\mathfrak{S}}_n}\mu {\left(P_{\pi }\right)}^q\right)& \mathrm{else}.\hfill \end{array}\right.\hfill \end{array}$$

2.4. Estimation

Given an orbit

$$(x_0,x_1,\dots ,x_{N-2},x_{N-1})=(x,T\left(x\right),\dots ,T^{N-2}\left(x\right),T^{N-1}\left(x\right))$$

of some $x\in \Omega$, it is natural to estimate $\mu \left(P_{\pi }\right)$ for $\pi \in {\mathfrak{S}}_n$ and $\mathrm{PE}\left(T,q,n\right)$ by $p_{\pi }$ and $\mathrm{ePE}\left({\left(x_t\right)}_{t=0}^{N-1},q,n\right)$, respectively. In the case that T is ergodic, by Birkhoff’s ergodic theorem, the corresponding estimators are asymptotically consistent. This particularly means that

$$\underset{N\to \infty }{lim}\mathrm{ePE}\left({\left(T^t\left(x\right)\right)}_{t=0}^{N-1},q,n\right) =\mathrm{PE}\left(T,q,n\right)$$

for $\mu$-almost all $x\in \Omega$.

2.5. RPE for Special Parameters q

In the following, we discuss the RPE for some special parameters q, and touch the general concept of Rényi entropies.

• $\mathit{q}=\mathbf{0}$: Rényi entropy for $q=0$ is the well-known Hartley entropy, and the RPE of a measure preserving dynamical system is no more than the logarithm of the number of ordinal patterns appearing with positive probability. Rényi entropy for $q=0$ is also called max entropy since it is maximal among Rényi entropies.
• $\mathit{q}=\mathbf{1}$: This case providing the standard (Shannon) permutation entropy has been discussed in various papers both from a theoretical and an application viewpoint. We particularly refer to the literature mentioned in several parts of this paper.
• $\mathit{q}=\mathbf{2}$: Rényi entropy for $q=2$, also called quadratic entropy or collision entropy, is used in different fields. It is obviously related to the Simpson index${\sum }_{i=1}^n{p}_i^2$ given for a stochastic vector ${\left(p_i\right)}_{i=1}^n$ and used as a diversity measure in ecology (see [13]). Given a measure-preserving dynamical system $(\Omega ,\mathcal{A},\mu ,T)$, we look at the RPE for $q=2$.

By Fubini’s theorem, it holds that

$$\begin{array}{ccc}\hfill \mathrm{PE}\left(T,2,n\right)& =& -log\left(\sum _{\pi \in {\mathfrak{S}}_n}\mu {\left(P_{\pi }\right)}^2\right)\hfill \\ \hfill & =& -log\left(\sum _{\pi \in {\mathfrak{S}}_n}{\left(\int {\mathbf{1}}_{P_{\pi }}d\mu \right)}^2\right)\hfill \\ \hfill & =& -log\left(\sum _{\pi \in {\mathfrak{S}}_n}\int \int {\mathbf{1}}_{P_{\pi }}\left({\omega }_1\right){\mathbf{1}}_{P_{\pi }}\left({\omega }_2\right)d\mu \left({\omega }_1\right)d\mu \left({\omega }_2\right)\right)\hfill \\ \hfill & =& -log\int \int \sum _{\pi \in {\mathfrak{S}}_n}\left({\mathbf{1}}_{P_{\pi }}\left({\omega }_1\right){\mathbf{1}}_{P_{\pi }}\left({\omega }_2\right)\right)d\mu \left({\omega }_1\right)d\mu \left({\omega }_2\right)\hfill \\ \hfill & =& -log\int \sum _{\pi \in {\mathfrak{S}}_n}\left({\mathbf{1}}_{P_{\pi }}\left({\omega }_1\right){\mathbf{1}}_{P_{\pi }}\left({\omega }_2\right)\right)d{\mu }^2({\omega }_1,{\omega }_2)\hfill \\ \hfill & =& -log{S}^n\hfill \end{array}$$

with

$$S^n={\mu }^2\{({\omega }_1,{\omega }_2)\in \Omega \times \Omega \mid \Pi \left({\omega }_1\right)=\Pi \left({\omega }_2\right)\}.$$

Here, ${\mathbf{1}}_A$ stands for the indicator of a set A assigning a point the value 1 if it belongs to A and value 0 otherwise, and ${\mu }^2$ denotes the product measure of $\mu$ with itself.

So, $\mathrm{PE}\left(T,2,n\right)$ is related to the probability that the ordinal patterns of length n of two independently (with respect to $\mu$) chosen points coincide.

A natural estimation of $S^n$ based on a finite orbit

$$(x_0,x_1,\dots ,x_{N-2},x_{N-1})=(x,T\left(x\right),\dots ,T^{N-2}\left(x\right),T^{N-1}\left(x\right))$$

of some $x\in \Omega$ is given by

$$\frac{2}{(N+1-n)(N-n)}\#\{(s,t)\mid 0\le s<t\le N-1,\Pi \left(x_s\right)=\Pi \left(x_t\right)\}$$

(1)

providing the relative frequency of pairs in the orbit with coinciding (completely defined) ordinal patterns. This qualifies the RPE for $q=2$ as a recurrence measure.

Quantity (1) was introduced by Caballero et al. [14] as the symbolic correlation integral in the context of a stochastic process and studied mainly in the i.i.d. case.

• $\mathit{q}=\mathbf{\infty }$: It is well-known that the Rényi entropy of a stochastic vector ${\left(p_i\right)}_{i\in I}$ for $q\to \infty$ converges to value

  $$\mathrm{RE}({\left(p_i\right)}_{i\in I},\infty ):=\underset{q\to \infty }{lim}\mathrm{RE}({\left(p_i\right)}_{i\in I},q)=-log\underset{i\in I}{max}{p}_i.$$

  (2)
  This fact can be used to reconstruct a stochastic vector up to permuting its components from its Rényi entropies for an unbounded sequence ${\left(q_n\right)}_{n\in \mathbb{N}}$ (see Appendix A). Since $\mathrm{RE}({\left(p_i\right)}_{i\in I},\infty )\le \mathrm{RE}({\left(p_i\right)}_{i\in I},q)$ for all $q\in [0,\infty ]$, number $\mathrm{RE}({\left(p_i\right)}_{i\in I},\infty )$ is called min entropy of ${\left(p_i\right)}_{i\in I}$. Applications of min entropy in the permutation entropy context can be found in Zunino et al. [7]. In the following, we further assume that $q<\infty$.

3. Asymptotics of RPE and PE

As already mentioned, there is a strong relationship between Kolmogorov–Sinai entropy and PE. The result of Takens and Verbitskiy [15] that, for $q>1$, Kolmogorov–Sinai entropy can be expressed by a limit on the basis of Rényi entropies instead of Shannon entropies suggests the question whether PE can be similarly replaced by RPE in that relationship. This question addresses the asymptotics of RPE, and the general nature of RPE is thus in the centre of this section.

3.1. Kolmogorov–Sinai Entropy via Rényi Entropies

Definitions and statements of this subsection go back to Takens and Verbitsky [15]. Many considerations of this paper are related to partitions of $\Omega$. We generally assume that, in a context where a $\sigma$-algebra on $\Omega$ is specified, such partitions are contained in it.

Let $(\Omega ,\mathcal{A},\mu ,T)$ now be a measure-preserving dynamical system, and consider a finite partition $\mathcal{P}={\left\{P_i\right\}}_{i\in I}$ of $\Omega$. For $n\in \mathbb{N}$ and multi-indices $\mathit{i}=(i_0,i_1,\dots ,i_{n-1})\in I^n$ define the sets

$$P\left(\mathit{i}\right):=\bigcap _{t=0}^{n-1}{T}^{-t}\left(P_{i_t}\right)=P_{i_0}\cap T^{-1}\left(P_{i_1}\right)\cap \dots T^{-n+1}\left(P_{i_{n-1}}\right)$$

forming the partition

$${\mathcal{P}}^{\left(n\right)}=\{P\left(\mathit{i}\right)\ne Ø|\mathit{i}\in (i_0,i_1,\dots i_{n-1})\in I^n\}$$

of $\Omega$. For $q\in [0,\infty ]$, the generalized entropy rate of T with respect to partition $\mathcal{P}$ is defined as

$$h(T,\mathcal{P},q):=\underset{n\to \infty }{lim\; inf}\frac{1}{n}H({\mathcal{P}}^{\left(n\right)},q)$$

(3)

with

$$\begin{array}{c}\hfill H(\mathcal{Q},q)=\left\{\begin{array}{cc}\hfill -\sum _{Q\in \mathcal{Q}}\mu \left(Q\right)log\mu \left(Q\right)& \mathrm{for}q=1\hfill \\ \hfill \frac{1}{1-q}log\left(\sum _{Q\in \mathcal{Q}}\mu {\left(Q\right)}^q\right)& \mathrm{else}\hfill \end{array}\right.\end{array}$$

for a finite partition $\mathcal{Q}$ of $\Omega$. Generalized Kolmogorov–Sinai entropy for $q\in [0,\infty ]$ is defined as the supremum of generalized entropy rates taken over all finite partitions:

$$h(T,q):=\underset{\mathcal{P} \mathrm{finite} \mathrm{partition} \mathrm{of} \Omega }{sup}h(T,\mathcal{P},q).$$

(Standard) Kolmogorov–Sinai entropy is given by

$$h\left(T\right):=h(T,1).$$

In the case of $q=1$, the limit inferior in (3) can be replaced by a limit and that, for $\Omega$ being an interval and $\mathcal{A}$ being the Borel $\sigma$-algebra, Kolmogorov–Sinai entropy is already determined by finite interval partitions defined as finite partitions consisting of intervals (e.g., [16]):

$$h\left(T\right)=\underset{\mathcal{P} \mathrm{finite} \mathrm{interval} \mathrm{partition} \mathrm{of} \Omega }{sup}h(T,\mathcal{P},1).$$

The following theorem of Takens and Verbitskiy [15] was originally proved for invertible systems; however, it also holds true for noninvertible systems (see Verbitskiy [17]). Assumption (i) of ergodicity can be relaxed (see Takens and Verbitskiy [18]); however, we do not go into the technical details.

Theorem 1.

Let $(\Omega ,\mathcal{A},\mu )$ be a standard-probability space and $T:\Omega \to \Omega$ an aperiodic and ergodic measure-preserving function. Then,

$$h(T,q)=h\left(T\right)$$

holds true for all $q\in [1,\infty ]$. Additionally, $h\left(T\right)>0$ implies

$$h(T,q)=\infty$$

for all $q\in [0,1]$.

Here, T is called aperiodic if the set of periodic points has measure zero with respect to $\mu$. The property of a probability space to be standard is a relatively technical one; however, it is not very restrictive since it is principally satisfied for the most common probability spaces (e.g., Walters [16]).

3.2. Kolmogorov–Sinai Entropy and RPE

In order to discuss the relationship between RPE and Kolmogorov–Sinai entropy of a measure-preserving dynamical system $(\Omega ,\mathcal{A},T,\mu )$, we define lower and upper Rényi permutation entropies $\underline{\mathrm{PE}}(T,q)$ and $\overline{\mathrm{PE}}(T,q)$ for $q\in [0,\infty ]$ as

$$\underline{\mathrm{PE}}(T,q)=\underset{n\to \infty }{lim\; inf}\frac{1}{n}H\left(O{P}^{}\left(n\right),q\right)$$

and

$$\overline{\mathrm{PE}}(T,q)=\underset{n\to \infty }{lim\; sup}\frac{1}{n}H\left(O{P}^{}\left(n\right),q\right),$$

respectively. We write $\underline{\mathrm{PE}}\left(T\right)$ and $\overline{\mathrm{PE}}\left(T\right)$ in the case of $q=1$. Both $\underline{\mathrm{PE}}(T,q)$ and $\overline{\mathrm{PE}}(T,q)$ monotonically decreasee with respect to q by the definition of Rényi entropies.

$\mathit{q}=\mathbf{1}$: The celebrated result of Bandt et al. [4] that, for piecewise continuous and monotone interval maps, the permutation and Kolmogorov-Sinai entropy coincide is the motivation for the following discussion. Here, we state the more general version of the result proved in Gutjahr and Keller [19], but afterwards return to the case of piecewise monotone interval maps. In the following, we call a subset of $\Omega \subset \mathbb{R}$ interval if it is the intersection of an interval of $\mathbb{R}$ with $\Omega$ or a one point set.

Theorem 2

([19]). Let $(\Omega ,B,\mu ,T)$ be a measure-preserving dynamical system, with $\Omega \subseteq \mathbb{R}$ being compact, and $\mathcal{B}$ being the Borel σ-algebra on Ω. If there exists a finite partition $\mathcal{M}$ or a countable partition $\mathcal{M}$ with $H\left(\mathcal{M}\right)<\infty$ of Ω into intervals, such that T is monotone on each of the intervals, then

$$\overline{\mathrm{PE}}\left(T\right)=h\left(T\right).$$

Theorem 2 covers interval maps, since a noncompact $\Omega$ can be replaced by compactification without substantially changing the structure of the given system.

$\mathit{q}>\mathbf{1}$: In light of the statement of Takens and Verbitskiy [15] mentioned above, it is a natural question whether also $\overline{\mathrm{PE}}(T,q)=h(T,q)(=h\left(T\right))$. The general answer is no. Examples with $\overline{\mathrm{PE}}(T,q)<h(T,q)$ covering all $q>1$ are given by Example A1 in Appendix B.2.

$\mathit{q}<\mathbf{1}$: We also look at case $q<1$ in the class of maps considered by Bandt et al. in [4]. For this, let $\Omega$ be an interval, $\mathcal{B}$ the Borel $\sigma$-algebra on it, and $\mathcal{M}$ be a finite partition of $\Omega$ into intervals on each of which T is monotone and continuous. For such a map, it was shown in [4] that

$$\underset{n\to \infty }{lim}\frac{1}{n}log\#\{P_{\pi }\in O{P}^{}\left(n\right)\mid P_{\pi }\ne Ø\}=\underset{n\to \infty }{lim}\frac{1}{n}log\#\{M\in {\mathcal{M}}^{\left(n\right)}\mid M\ne Ø\}$$

(4)

holds true. Using the fact that Rényi entropy monotonically decreases in q, this implies

$$\begin{array}{ccc}\hfill \overline{\mathrm{PE}}(T,q)& \le & \overline{\mathrm{PE}}(T,0)\hfill \\ & =& \underset{n\to \infty }{lim\; sup}\frac{1}{n}log\left(\sum _{P_{\pi }\in O{P}^{}\left(n\right)}\mu {\left(P_{\pi }\right)}^0\right)\hfill \\ \hfill & =& \underset{n\to \infty }{lim\; sup}\frac{1}{n}log\#\{P_{\pi }\in O{P}^{}\left(n\right)\mid \mu \left(P_{\pi }\right)>0\}\hfill \\ \hfill & \le & \underset{n\to \infty }{lim}\frac{1}{n}log\#\{P_{\pi }\in O{P}^{}\left(n\right)\mid P_{\pi }\ne Ø\}\hfill \\ \hfill & =& \underset{n\to \infty }{lim}\frac{1}{n}log\#\{M\in {\mathcal{M}}^{\left(n\right)}\mid M\ne Ø\}\hfill \end{array}$$

for all $q\in [0,\infty ]$. Let us summarize:

Proposition 1.

Let $(\Omega ,\mathcal{B},\mu ,T)$ be a measure-preserving dynamical system, with $\Omega \subseteq \mathbb{R}$ being an interval, and $\mathcal{B}$ being the Borel σ-algebra on it. Suppose that $\mathcal{M}$ is a finite partition of Ω into intervals such that T is monotone and continuous on each of the intervals. Then

$$\overline{\mathrm{PE}}(T,q)\le \underset{n\to \infty }{lim}\frac{1}{n}log\#\{M\in {\mathcal{M}}^{\left(n\right)}\mid M\ne Ø\}$$

holds true for all $q\in [0,\infty ]$.

Quantity ${lim\; sup}_{n\to \infty }\frac{1}{n}log\#\{P_{\pi }\in O{P}^{}\left(n\right)\mid P_{\pi }\ne Ø\}$ could be considered to be a topological version of permutation entropy. This is justified by (4) for T, as defined above (4), and the following: If T is continuous on all of $\Omega$, Misiurewicz and Szlenk showed that ${lim}_{n\to \infty }\frac{1}{n}log\#\{M\in {\mathcal{M}}^{\left(n\right)}\mid M\ne Ø\}$ is equal to the topological entropy of T [20]. For the definition of topological entropy and the following, see, e.g., [16].

By variation principle, the topological entropy of a map T on a compact Hausdorff space is equal to the supremum of the Kolmogorov–Sinai entropy of all Borel measures for which T is measure-preserving (e.g., [16]). Often, topological entropy is assumed by the Kolmogorov–Sinai of such a measure. Generally, given a continuous map T on a metric space, a corresponding Borel measure being measure-preserving has maximal entropy if its Kolmogorov–Sinai entropy coincides with the topological entropy of T.

On the basis of the discussion above, we show the following statement (see Appendix B.1).

Corollary 1.

Let $(\Omega ,\mathcal{B},\mu ,T)$ be a measure-preserving dynamical system, with $\Omega \subseteq R$ being an interval, and $\mathcal{B}$ being the Borel σ-algebra on Ω. Suppose that T is continuous, and there exists a finite partition of Ω into intervals, such that T is monotone on each of those intervals, and μ is a measure of maximal entropy. Then

$$\underline{\mathrm{PE}}(T,q)=\overline{\mathrm{PE}}(T,q)=\underset{P finite interval partition of \Omega }{sup}h(T,\mathcal{P},q)=h\left(T\right)$$

holds true for all $q\in [0,1]$.

4. Conclusions

In this paper, we looked more closely at the recently introduced and used Rényi variant of permutation entropy, depending on a parameter $q\in [0,\infty ]$, which is called Rényi permutation entropy (RPE) here. Giving a summary of first applications of RPE, and discussing RPE for some special parameter q, we mainly focused on the asymptotics of RPE for ordinal pattern length going to ∞. This was motivated by the fact that the usual permutation entropy (PE) often asymptotically coincides with Kolmogorov–Sinai entropy, and that, for $q>1$, Kolmogorov–Sinai entropy can be defined by Rényi entropies instead of Shannon entropies.

This paper showed that, for $q>1$, asymptotics of RPE can be different from that of PE, meaning that, for long ordinal patterns, the nature of RPE is also not the same as that of PE. One the other hand, it is interesting that, for continuous piecewise monotone interval maps with a measure of maximal entropy and $q<1$, asymptotics of RPE and PE are the same. Results indicate that the behaviour of general RPE is more specific than that of PE, although the asymptotics of PE is not completely understood. Further work for the better understanding of RPE for large pattern lengths is necessary.

The content of this paper is more or less purely mathematical, but in a certain sense, it justifies the application of RPE in dynamical systems and time series besides PE. Some of the applications mentioned at the beginning of the paper underline the benefit of using RPE. There is, however, the other interesting point that special q address special features; so, for example, $q=2$ is related to recurrence. The symbolic correlation integral related to case $q=2$ is a U-statistic, which is helpful in the statistical analysis of the corresponding entropy. Work on utilizing this fact for testing for asymmetry in temporal data is in progress.

Appendix B.1. Proof of Corollary 1

First, if a finite partition $\mathcal{P}$ of $\Omega$ is finer than another one, $\mathcal{Q}$, then $H(\mathcal{P},q)\ge H(\mathcal{Q},q)$ for all $q\in [0,\infty ]$. This is well-known and easily follows from the concavity and convexity of $x\mapsto x^q$ for $q<1$ and $q>1$, respectively, and from the convexity of $x\mapsto xlogx$ for $q=1$ on $[0,\infty ]$.

Given two finite partitions $\mathcal{P},\mathcal{Q}$ of $\Omega$, let $P\vee \mathcal{Q}=\{P\cap Q\ne Ø\mid P\in \mathcal{P},Q\in \mathcal{Q}\}$ be the largest common refinement of $\mathcal{P}$ and $\mathcal{Q}$. Further, for a partition $\mathcal{P}$ of $\Omega$ and some $Q\in \mathcal{A}$, let $\Delta \left(\mathcal{P}\right|Q):=\{P\in \mathcal{P}\mid \mu (P\cap Q)>0\}$ and $\#\Delta \left(\mathcal{P}\right|Q)$ the number of elements of $\Delta \left(\mathcal{P}\right|Q)$.

The proof of Corollary 1 is built on the two following lemmata.

Lemma A1.

Let $(\Omega ,\mathcal{A},\mu )$ be a probability space, and $\mathcal{P},\mathcal{Q}$ two finite partitions of Ω. Then

$$H(\mathcal{P}\vee \mathcal{Q},q)\le H(\mathcal{Q},q)+\left\{\begin{array}{cc}\sum _{Q\in \mathcal{Q}}\mu \left(Q\right)·log(\#\Delta \left(\mathcal{P}\right|Q))\hfill & \mathrm{for}q=1\hfill \\ \underset{Q\in \mathcal{Q}}{max}log(\#\Delta \left(\mathcal{P}\right|Q))\hfill & \mathrm{for}q\in [0,\infty ]\backslash \left\{1\right\}.\hfill \end{array}\right.$$

Proof. 

Consider function $f:[0,\infty ]\to \mathbb{R}$ with $f\left(x\right)=x^q$. Since f is concave for $q<1$ and convex for $q>1$, Jensen’s inequality implies

$$\sum _{P\in \Delta \left(\mathcal{P}\right|Q)}\frac{f\left(\mu \right(P\cap Q\left)\right)}{\#\Delta \left(\mathcal{P}\right|Q)}\left\{\begin{array}{cc}\le f\left({\displaystyle \sum _{P\in \Delta \left(\mathcal{P}\right|Q)}}\frac{\mu (P\cap Q)}{\#\Delta \left(\mathcal{P}\right|Q)}\right)\hfill & \mathrm{if}q<1\hfill \\ \ge f\left({\displaystyle \sum _{P\in \Delta \left(\mathcal{P}\right|Q)}}\frac{\mu (P\cap Q)}{\#\Delta \left(\mathcal{P}\right|Q)}\right)\hfill & \mathrm{if}q>1\hfill \end{array}\right.$$

for all $Q\in \mathcal{Q}$. Using this and the fact that function $\frac{-1}{q-1}log$ monotonically increases for $q<1$ and decreases for $q>1$ provides

$$\begin{array}{ccc}\hfill H(\mathcal{P}\vee \mathcal{Q},q)& =& \frac{-1}{q-1}log\left(\sum _{Q\in \mathcal{Q}}\sum _{P\in \mathcal{P}}\mu {(P\cap Q)}^q\right)\hfill \\ \hfill & =& \frac{-1}{q-1}log\left(\sum _{Q\in \mathcal{Q}}\sum _{P\in \Delta \left(\mathcal{P}\right|Q)}\mu {(P\cap Q)}^q\right)\hfill \\ \hfill & =& \frac{-1}{q-1}log\left(\sum _{Q\in \mathcal{Q}}\#\Delta \left(\mathcal{P}\right|Q)\sum _{P\in \Delta \left(\mathcal{P}\right|Q)}\frac{f\left(\mu \right(P\cap Q\left)\right)}{\#\Delta \left(\mathcal{P}\right|Q)}\right)\hfill \\ \hfill & \le & \frac{-1}{q-1}log\left(\sum _{Q\in \mathcal{Q}}\#\Delta \left(\mathcal{P}\right|Q)·f\left(\sum _{P\in \Delta \left(\mathcal{P}\right|Q)}\frac{\mu (P\cap Q)}{\#\Delta \left(\mathcal{P}\right|Q)}\right)\right)\hfill \\ \hfill & =& \frac{-1}{q-1}log\left(\sum _{Q\in \mathcal{Q}}\#\Delta \left(\mathcal{P}\right|Q)·f\left(\frac{\mu \left(Q\right)}{\#\Delta \left(\mathcal{P}\right|Q)}\right)\right)\hfill \\ \hfill & =& \frac{-1}{q-1}log\left(\sum _{Q\in \mathcal{Q}}\#\Delta {\left(\mathcal{P}\right|Q)}^{1-q}·\mu {\left(Q\right)}^q\right)\hfill \\ \hfill & \le & \frac{-1}{q-1}log\left({\left(\underset{Q\in \mathcal{Q}}{max}\#\Delta \left(\mathcal{P}\right|Q)\right)}^{1-q}·\sum _{Q\in \mathcal{Q}}\mu {\left(Q\right)}^q\right)\hfill \\ \hfill & =& \frac{-1}{q-1}log\left({\left(\underset{Q\in \mathcal{Q}}{max}\#\Delta \left(\mathcal{P}\right|Q)\right)}^{1-q}\right)+\frac{-1}{q-1}log\left(\sum _{Q\in \mathcal{Q}}\mu {\left(Q\right)}^q\right)\hfill \\ \hfill & =& \underset{Q\in \mathcal{Q}}{max}log(\#\Delta \left(\mathcal{P}\right|Q))+H(\mathcal{Q},q)\hfill \end{array}$$

for all $q\ne 1$.

For $q=1$, consider function $g:[0,\infty ]\to [0,\infty ]$ with $g\left(x\right)=xlog\left(x\right)$, which is convex. Therefore, by Jensen’s inequality, it follows that

$$\begin{array}{cc}\hfill & \sum _{P\in \Delta \left(\mathcal{P}\right|Q)}\mu (P\cap Q)log\left(\mu (P\cap Q)\right)\hfill \\ \hfill =& \#\Delta \left(\mathcal{P}\right|Q)\sum _{P\in \Delta \left(\mathcal{P}\right|Q)}\frac{1}{\#\Delta \left(\mathcal{P}\right|Q)}·g\left(\mu (P\cap Q)\right)\hfill \\ \hfill \ge & \#\Delta \left(\mathcal{P}\right|Q)·g\left(\sum _{P\in \Delta \left(\mathcal{P}\right|Q)}\frac{1}{\#\Delta \left(\mathcal{P}\right|Q)}·\mu (P\cap Q)\right)\hfill \\ \hfill =& \#\Delta \left(\mathcal{P}\right|Q)·g\left(\frac{\mu \left(Q\right)}{\#\Delta \left(\mathcal{P}\right|Q)}\right)\hfill \\ \hfill =& \#\Delta \left(\mathcal{P}\right|Q)·\frac{\mu \left(Q\right)}{\#\Delta \left(\mathcal{P}\right|Q)}·log\left(\frac{\mu \left(Q\right)}{\#\Delta \left(\mathcal{P}\right|Q)}\right)\hfill \\ \hfill =& \mu \left(Q\right)·\left(log\left(\mu \right(Q\left)\right)-log(\#\Delta (\mathcal{P}\left|Q\right))\right).\hfill \end{array}$$

Using the above inequality provides

$$\begin{array}{ccc}\hfill H(\mathcal{P}\vee \mathcal{Q})& =& -\sum _{Q\in \mathcal{Q}}\sum _{P\in \mathcal{P}}\mu (P\cap Q)log\left(\mu (P\cap Q)\right)\hfill \\ \hfill & =& -\sum _{Q\in \mathcal{Q}}\sum _{P\in ·\left(\mathcal{P}\right|Q)}\mu (P\cap Q)log\left(\mu (P\cap Q)\right)\hfill \\ \hfill & \le & -\sum _{Q\in \mathcal{Q}}\mu \left(Q\right)·\left(log\left(\mu \right(Q\left)\right)-log(\#\Delta (\mathcal{P}\left|Q\right))\right)\hfill \\ \hfill & =& H\left(\mathcal{Q}\right)+\sum _{Q\in \mathcal{Q}}\mu \left(Q\right)·log(\#\Delta \left(\mathcal{P}\right|Q)).\hfill \end{array}$$

 □

The following lemma was proved in Gutjahr and Keller [19] (see Lemma A1). We present it here in a slightly weakened form since we do not need the full generality.

Lemma A2.

Let $(\Omega ,\mathcal{B},\mu ,T)$ be a measure-preserving dynamical system, with $\Omega \subseteq R$ being an interval and $\mathcal{B}$ being the Borel σ-algebra on Ω. Further, let $\mathcal{P}$ be a finite interval partition of Ω. Then for all $P_{\pi }\in O{P}^{}\left(n\right)$

$$\#\Delta \left({\mathcal{P}}^{\left(n\right)}\right|P_{\pi })\le \left(\genfrac{}{}{0pt}{}{n+\#\mathcal{P}-1}{\#\mathcal{P}-1}\right)$$

holds true.

We can now finalise the proof of Corollary 1. Given the assumptions of the corollary, by Proposition 1 and the monotony of the Rényi entropy with respect to q, we have

$$\begin{array}{ccc}\hfill \underline{\mathrm{PE}}(T,q)& \le & \overline{\mathrm{PE}}(T,q)\hfill \\ & \le & \underset{n\to \infty }{lim}\frac{1}{n}log\#\{M\in {\mathcal{M}}^{\left(n\right)}\mid M\ne Ø\}\hfill \\ & =& h\left(T\right)\hfill \\ & =& \underset{P \mathrm{finite} \mathrm{interval} \mathrm{partition} \mathrm{of} \Omega }{sup}h(T,\mathcal{P})\hfill \\ & \le & \underset{P \mathrm{finite} \mathrm{interval} \mathrm{partition} \mathrm{of} \Omega }{sup}h(T,\mathcal{P},q)\hfill \end{array}$$

for all $q\in [0,1]$. So, it remains to show that

$$\underline{\mathrm{PE}}(T,q)\ge h(T,\mathcal{P},q)$$

is true for all $q\in [0,1]$ and for all finite interval partitions $\mathcal{P}$ of $\Omega$.

Fixing some $q\in [0,1]$ and some finite interval partition of $\Omega$, Lemmas A1 and A2 yield

$$\begin{array}{ccc}\hfill H({\mathcal{P}}^{\left(n\right)},q)& \le & H({\mathcal{P}}^{\left(n\right)}\vee O{P}^{}\left(n\right),q)\hfill \\ & \le & H(O{P}^{}\left(n\right),q)+\underset{\pi \in {\mathfrak{S}}_n}{max}log(\#\Delta \left({\mathcal{P}}^{\left(n\right)}\right|P_{\pi }))\hfill \\ & \le & H(O{P}^{}\left(n\right),q)+\underset{\pi \in {\mathfrak{S}}_n}{max}log\left(\left(\genfrac{}{}{0pt}{}{n+\#\mathcal{P}-1}{\#\mathcal{P}-1}\right)\right)\hfill \\ & \le & H(O{P}^{}\left(n\right),q)+\underset{\pi \in {\mathfrak{S}}_n}{max}log\left({(n+\#\mathcal{P}-1)}^{\#\mathcal{P}-1}\right)\hfill \\ & \le & H(O{P}^{}\left(n\right),q)+(\#\mathcal{P}-1)log(n+\#\mathcal{P}-1)\hfill \end{array}$$

for all $q\in [0,1]$. Hence,

$$\begin{array}{ccc}\hfill \underline{\mathrm{PE}}(T,q)& =& \underset{n\to \infty }{lim\; inf}\frac{1}{n}H\left(O{P}^{}\left(n\right),q\right)\hfill \\ \hfill & \ge & \underset{n\to \infty }{lim\; inf}\frac{1}{n}\left[H({\mathcal{P}}^{\left(n\right)},q)-(\#\mathcal{P}-1)log(n+\#\mathcal{P}-1)\right]\hfill \\ \hfill & =& \underset{n\to \infty }{lim\; inf}\frac{1}{n}H\left({\mathcal{P}}^{\left(n\right)},q\right) =h(T,\mathcal{P},q).\hfill \end{array}$$

Appendix B.2. Counter Examples

In order to give the examples after Theorem 2, we start with two lemmata.

Lemma A3.

Let $(\Omega ,\mathcal{B},\mu ,T)$ be a measure-preserving dynamical system with $\Omega \subseteq \mathbb{R}$ and $\mathcal{B}$ being the Borel σ-algebra on Ω. Further, let $\mathcal{P}={\left\{P_i\right\}}_{i\in I}$ be a finite partition of Ω. Then, for all $q\in [1,\infty ]$,

$$\overline{\mathrm{PE}}(T,q)\le h(T,\mathcal{P},q)+\underset{n\to \infty }{lim\; sup}\frac{1}{n}\underset{\mathit{i}\in I^n}{max}log(\#\Delta \left(O{P}^{}\left(n\right)\right|P\left(\mathit{i}\right)))$$

hold true.

Proof. 

Let $\mathcal{P}={\left\{P_i\right\}}_{i\in I}$ be a finite partition of $\Omega$. Then, Lemma A1 provides

$$H(O{P}^{}\left(n\right),q)\le H(O{P}^{}\left(n\right)\vee {\mathcal{P}}^{\left(n\right)},q)\le H({\mathcal{P}}^{\left(n\right)},q)+\underset{\mathit{i}\in I^n}{max}log(\#\Delta \left(O{P}^{}\left(n\right)\right|P\left(\mathit{i}\right)))$$

for all $q>1$ and $n\in \mathbb{N}$. Dividing both sides by n and taking the limit superior $n\to \infty$ finishes the proof.   □

The following lemma was shown in [22] (see Lemma 2.2.).

Lemma A4.

Let $T:\Omega \to \Omega$ be a map on $\Omega \subseteq \mathbb{R}$. Assume that $\mathcal{M}={\left\{M_i\right\}}_{i\in I}$ is a finite partition of Ω into sets on each of which T is monotone. Then, for all $n\in \mathbb{N}$ and multi-indices $\mathit{i}=(i_0,i_1,\dots ,i_{n-1})\in I^n$

$$\#\Delta \left(O{P}^{}\left(n\right)\right|M\left(\mathit{i}\right))\le 2^{\#\{s\in \{0,1,\cdots ,n-2\}\mid i_s=i_{n-1}\}}$$

holds true.

As a consequence of the two lemmata, we obtain an upper bound of RPE that is important for the following.

Corollary A1.

Let $(\Omega ,\mathcal{B},\mu ,T)$ be a measure-preserving dynamical system with $\Omega \subseteq \mathbb{R}$. Assume that $\mathcal{M}$ is a finite partition of Ω into sets on each of which T is monotone. Then, for all $q\in [0,\infty ]$

$$\overline{\mathrm{PE}}(T,q)\le h(T,\mathcal{M},q)+log2.$$

holds true.

Proof. 

We have

$$\overline{\mathrm{PE}}(T,q)\le h(T,\mathcal{M},q)+\underset{n\to \infty }{lim\; sup}\frac{1}{n}\underset{\mathit{i}\in I^n}{max}log(\#\Delta \left(O{P}^{}\left(n\right)\right|M\left(\mathit{i}\right)))$$

according to Lemma A3. Lemma A4 provides

$$\#\Delta \left(O{P}^{}\left(n\right)\right|M\left(\mathit{i}\right))\le 2^{\#\{s\in \{0,1,\cdots ,n-2\}\mid i_s=i_{n-1}\}}\le 2^{n-1}$$

for all $n\in \mathbb{N}$ and $\mathit{i}\in I^n$. Combining the above statements yields

$$\begin{array}{ccc}\hfill \overline{\mathrm{PE}}(T,q)& =& h(T,\mathcal{M},q)+\underset{n\to \infty }{lim\; sup}(n-1)/nlog2\hfill \\ & =& h(T,\mathcal{M},q)+log2.\hfill \end{array}$$

 □

We use the following fact about the Rényi entropies of stochastic vectors:

Lemma A5.

For all $q\in [1,\infty ]$ and all $c>0$, there exists a stochastic vector $\mathbf{p}$ with

$$H\left(\mathbf{p}\right)-H(\mathbf{p},q)\ge c.$$

Proof. 

Let $c>0$ and $q>1$. Choose $N\in \mathbb{N}$, such that

$$N\ge 2·\left(\frac{c}{log\left(2\right)}+\frac{q}{q-1}\right)$$

holds true. Consider stochastic vector $p=(p_1,p_2,\dots ,p_{2^N+1})\in {[0,1]}^{2^N+1}$ with

$$p_1=\frac{1}{2}$$

and

$$p_i=\frac{1}{2^{N+1}}\mathrm{for}i\in \{2,3,\dots ,2^N+1\}.$$

Then,

$$\begin{array}{ccc}\hfill H\left(\mathbf{p}\right)& =& -\sum _{i=1}^{2^N+1}{p}_{i}log\left(p_i\right)\hfill \\ & =& -2^N·\frac{1}{2^{N+1}}log\left(\frac{1}{2^{N+1}}\right)-\frac{1}{2}log\left(\frac{1}{2}\right)\hfill \\ & =& \frac{N+1}{2}log\left(2\right)+\frac{1}{2}log\left(2\right)\hfill \\ & \ge & \frac{N}{2}log\left(2\right)\hfill \end{array}$$

and

$$\begin{array}{ccc}\hfill H(\mathbf{p},q)& =& \frac{-1}{q-1}log\left(\sum _{i=1}^{2^N+1}{p}_i^q\right)\hfill \\ & =& \frac{-1}{q-1}log\left(2^N·\frac{1}{2^{(N+1)q}}+\frac{1}{2^q}\right)\hfill \\ & \le & \frac{-1}{q-1}log\left(\frac{1}{2^q}\right)\hfill \\ & =& \frac{q}{q-1}log\left(2\right).\hfill \end{array}$$

Hence,

$$H\left(\mathbf{p}\right)-H(\mathbf{p},q)\ge log\left(2\right)·\left(\frac{N}{2}-\frac{q}{q-1}\right)\ge c.$$

 □

Recall that a (one-sided) Bernoulli shift $\sigma$ is a map defined on ${\{0,1,\dots ,N-1\}}^{\infty }$ for some $N\in \mathbb{N}$ by

$$\sigma \left({\left(b_n\right)}_{n=1}^{\infty }\right)={\left(b_n\right)}_{n=2}^{\infty }$$

for ${\left(b_n\right)}_{n=1}^{\infty }\in {\{0,1,\dots ,N-1\}}^{\infty }$. Given a stochastic vector $\mathbf{p}=(p_0,p_1,\dots ,p_{N-1})\in {[0,1]}^N$, one considers the product measure ${\mu }_{\mathbf{p}}$ associated to $\mathbf{p}$. More precisely, ${\mu }_{\mathbf{p}}$ is the probability measure on the smallest $\sigma$-algebra containing all cylinder sets

$$C_k(a_1,a_2,\dots ,a_k)=\{{\left(b_n\right)}_{n=1}^{\infty }\in {\{0,1,\dots ,N-1\}}^{\infty }\mid (b_1,b_2,\dots ,b_k)=(a_1,a_2,\dots ,a_k)\}$$

for $k\in \mathbb{N};a_1,a_2,\dots ,a_k\in \{0,1,\dots ,N-1\}$ defined by

$${\mu }_{\mathbf{p}}\left(C_k(a_1,a_2,\dots ,a_k)\right)=\prod _{n=1}^k{p}_{a_n}.$$

for such cylinder sets. $\sigma$ is ${\mu }_{\mathbf{p}}$-preserving and ergodic.

Lemma A6.

Let σ be a Bernoulli shift on ${\{0,1,\dots ,N-1\}}^{\infty }$ for some $N\in \mathbb{N}$ and $\mathbf{p}=(p_0,p_1,\dots ,p_{N-1})\in {[0,1]}^N$ be a stochastic vector. Let ${\mu }_{\mathbf{p}}$ be defined as above and, for $k\in \mathbb{N}$, let

$${\mathcal{C}}_k=\{C_k(a_1,a_2,\dots ,a_k)\mid a_1,a_2,\dots ,a_k\in \{0,1,\dots ,N-1\}\}.$$

Then,

$$h(\sigma ,{\mathcal{C}}_k,q)=H(\mathbf{p},q)$$

holds true for all $q\ne 1$, $k\in \mathbb{N}$.

Proof. 

Take $k\in \mathbb{N}$ and $q\in \mathbb{R}$. Notice that

$${\mathcal{C}}_k^{\left(n\right)}=\underset{t=0}{\overset{n-1}{\bigvee }}{\sigma }^{-1}\left({\mathcal{C}}_k\right)={\mathcal{C}}_{k+n}$$

for all $n\in \mathbb{N}$. This implies that

$$\begin{array}{ccc}\hfill H\left({\mathcal{C}}_k^{\left(n\right)}\right)& =& \frac{-1}{q-1}log\left(\sum _{P\in {\mathcal{C}}_{k+n}}{\mu }_{\mathbf{p}}{\left(P\right)}^q\right)\hfill \\ & =& \frac{-1}{q-1}log\left(\sum _{a_1,\cdots ,a_{k+n}\in \{0,1,\dots ,N-1\}}{\left(\prod _{i=1}^{k+n}{p}_i\right)}^q\right)\hfill \\ & =& \frac{-1}{q-1}log\left(\sum _{a_1,\cdots ,a_{k+n}\in \{0,1,\dots ,N-1\}}\left(\prod _{i=1}^{k+n}{p}_i^q\right)\right)\hfill \\ & =& \frac{-1}{q-1}log\left({\left(\sum _{a\in \{0,1,\dots ,N-1\}}{p}_a^q\right)}^{k+n}\right)\hfill \\ & =& (k+n)·\frac{-1}{q-1}log\left(\sum _{a\in \{0,1,\dots ,N-1\}}{p}_a^q\right)\hfill \\ & =& (k+n)·H(\mathbf{p},q)\hfill \end{array}$$

for all $n\in \mathbb{N}$ and $q\ne 1$. Therefore,

$$\begin{array}{ccc}\hfill h(\sigma ,{\mathcal{C}}_k,q)& =& \underset{n\to \infty }{lim\; inf}\frac{1}{n}H\left(\underset{i=0}{\overset{n-1}{\bigvee }}{\sigma }^{-1}\left({\mathcal{C}}_k\right),q\right)\hfill \\ & =& \underset{n\to \infty }{lim\; inf}\frac{k+n}{n}·H(\mathbf{p},q)=H(\mathbf{p},q)\hfill \end{array}$$

holds true for all $q\ne 1$.  □

Example A1.

Take any $q>1$. Choose some $N\in \mathbb{N}$ and a stochastic vector $\mathbf{p}=(p_0,p_1,\dots ,p_{N-1})\in {[0,1]}^N$ such that

$$H(\mathbf{p},q)+log2<H\left(\mathbf{p}\right)$$

holds true. This is always possible due to Lemma A5. Further, let ${\mu }_{\mathbf{p}}$ be the probability measure defined before Lemma A6.

In order to obtain a map with finitely many monotone parts, identify Bernoulli shift σ on ${\{0,1,\dots ,N-1\}}^{\infty }$ with interval map T on $\Omega =[0,1]$ defined by

$$T\left(\omega \right)=N·\omega mod1$$

for all $\omega \in \Omega$. This is possible since the correspondence

$${\left(b_n\right)}_{n\in \mathbb{N}}\in \{0,1,\dots ,N-1\}\mapsto \sum _{n=1}^{\infty }{b}_n{N}^{-n}$$

turns σ into T. In this way, measure ${\mu }_{\mathbf{p}}$ can be considered as a measure on Ω with the Borel sets. One easily sees that $\mathcal{M}=\left\{\right[(i-1)/N,i/N]\mid i\in \{1,2,\dots ,N\left\}\right\}$ is a partition into intervals on each of which T is monotone and continuous. According to the above identification, $\mathcal{M}$ can be considered as the partition ${\mathcal{C}}_1$ in Lemma A6. Now

$$\begin{array}{c}\hfill \overline{\mathrm{PE}}(T,q)\le h(T,\mathcal{M},q)+log2=H(\mathbf{p},q)+log2<H\left(\mathbf{p}\right)=h(T,q)\end{array}$$

holds true. Here the first equality follows from Corollary A1, the last one from Theorem 1, and the first inequality from Lemma A6.