Linear (h,φ)-Entropies for Quasi-Power Sequences with a Focus on the Logarithm of Taneja Entropy ^†

Abstract

Conditions are highlighted for generalized entropies to allow for non-trivial time-averaged entropy rates for a large class of random sequences, including Markov chains and continued fractions. The axiomatic-free conditions arise from the behavior of the marginal entropy of the sequence. Apart from the well-known Shannon and Rényi cases, only logarithmic versions of Sharma–Taneja–Mittal entropies may fulfill these conditions. Their main properties are detailed.

1. Introduction

Quantifying the information—or uncertainty—of a sequence of random variables has been considered since the foundation of information theory in [1], where the entropy rate of a random sequence $\mathbf{X}={\left(X_n\right)}_{n\in {\mathbb{N}}^{*}}$ is defined as the limit $\mathbb{S}\left(\mathbf{X}\right)={lim}_{n\to +\infty }\mathbb{S}(X_1\dots ,X_n)/n$, with $\mathbb{S}(X_1\dots ,X_n)$ denoting the entropy of the n first coordinates of $\mathbf{X}$ for $n\in {\mathbb{N}}^{*}$ a positive integer. Shannon’s original concept of entropy naturally generalizes to various entropies, introduced in the literature for a better fit to certain complex systems through proper axiomatic requirements. Most of the classical examples given in Table 1 (first column) belong to the class of $(h,\phi )$-entropies

$${\mathbb{S}}_{h,\phi }\left(X\right)={\mathbb{S}}_{h,\phi }\left(m_X\right)=h\left(\sum _{i\in E}\phi \left(m_X\left(i\right)\right)\right),$$

(1)

where h and $\phi$ are real functions satisfying compatibility assumptions and $m_X$ is the distribution of a discrete random variable X on E; see Section 2 below and also [2,3]. When h is the identity, the corresponding $(h,\phi )$-entropy is a $\phi$-entropy, denoted by ${\mathbb{S}}_{\phi }$.

Ref. [1] established the existence and a closed-form expression for the entropy rate $\mathbb{S}\left(\mathbf{X}\right)$ of homogeneous ergodic Markov chains. This relies on the chain rule—strong additivity property—of the Shannon entropy, $\mathbb{S}(X_n,X_{n+1})=\mathbb{S}\left(X_n\right)+{\sum }_{i\in E}{m}_{X_n}\left(i\right)\mathbb{S}\left(X_{n+1}\right|X_n=i)$, where $\mathbb{S}\left(X_{n+1}\right|X_n=i)$ denotes the Shannon entropy of the distribution of $X_{n+1}$ conditional to $(X_n=i)$. This chain rule implicitly implies that, for ergodic Markov chains, the sequence of marginal entropies $\mathbb{S}\left(X_{1:n}\right)$ grows (asymptotically) linearly with n.

This type of linearity appears as an interesting property for a generalized entropy to measure—in a meaningful way—the information carried by a random process: all the variables contribute equally to the global information of the sequence. Below, an $(h,\phi )$-entropy will be said to be linear for a random sequence $\mathbf{X}={\left(X_n\right)}_{n\in {\mathbb{N}}^{*}}$ if the marginal entropy sequence ${\left({\mathbb{S}}_{h,\phi }\left(X_{1:n}\right)\right)}_n$ grows linearly with n, where $X_{1:n}=(X_1,\dots ,X_n)$. Furthermore, Ciuperca et al. [4] show that linearity is required for obtaining a non-trivial time-averaged $(h,\phi )$-entropy rate

$${\mathbb{S}}_{h,\phi }\left(\mathbf{X}\right)=\underset{n\to +\infty }{lim}\frac{1}{n}{\mathbb{S}}_{h,\phi }\left(X_{1:n}\right).$$

(2)

Self-evidently, the Shannon entropy is linear for independent and identically distributed (i.i.d.) sequences and most ergodic Markov chains. A natural attempt to prove linearity of other entropies consists in investigating some additive functional identity replacing the chain rule. Regnault et al. [5] establish the linearity of Rényi entropies for homogeneous ergodic Markov chains, based on the identity ${\mathbb{R}}_{\mathfrak{q}}(X,Y)={\mathbb{R}}_{\mathfrak{q}}\left(X\right)+{\sum }_{i\in E}{m}_X^{*\mathfrak{q}}\left(i\right){\mathbb{R}}_{\mathfrak{q}}\left(Y\right|X=i)$, where $m_X^{*\mathfrak{q}}={(m_X{\left(i\right)}^{\mathfrak{q}}/{\sum }_{j\in E}{m}_X{\left(j\right)}^{\mathfrak{q}})}_{i\in E}$ denotes the $\mathfrak{q}$-escort distribution of X; explicit weighted expressions for the Rényi entropy rates follow. Most likely, such an approach cannot be systematically generalized to any other $(h,\phi )$-entropy by lack of a proper additive functional identity.

Ciuperca et al. [4] successfully explore an axiomatic-free alternative approach, originating in analytic combinatorics. It consists of deriving the asymptotic behavior of ${\mathbb{S}}_{h,\phi }\left(X_{1:n}\right)$ from the behavior of the associated Dirichlet series

$$\Lambda (X_{1:n};s)=\Lambda (m_{1:n};s)=\sum _{x_{1:n}\in E^n}{\left(m_{1:n}\left(x_{1:n}\right)\right)}^s,$$

(3)

where $s>0$ and $m_{1:n}$ is the distribution of $X_{1:n}$. A random sequence $\mathbf{X}$ is a quasi-power (QP) sequence if $sup\{m_{1:n}\left(i_{1:n}\right):i_{1:n}\in E^n\}$ converges to zero as n tends to infinity, and if a real number ${\sigma }_0<1$, and strictly positive analytic functions c and $\lambda$, with $\lambda$ strictly decreasing and $\lambda \left(1\right)=c\left(1\right)=1$, such that for all real numbers, $s>{\sigma }_0$, some $\rho \left(s\right)\in (0,1)$ exists, such that

$$\Lambda (m_{1:n};s)=c\left(s\right)·\lambda {\left(s\right)}^{n-1}+R_n\left(s\right)$$

(4)

for all n, where $R_n$ is an analytic function, such that $|R_n\left(s\right)|=$$O\left(\rho {\left(s\right)}^{n-1}\lambda {\left(s\right)}^{n-1}\right)$; see [6]. For any $(h,\phi )$-entropy that is linear for QP sequences, the entropy rate (2) is proven to reduce to either the Shannon or Rényi entropy rate, up to a constant factor. The large class of QP sequences includes i.i.d. sequences, most ergodic Markov chains with atomic state spaces, and more complex ergodic dynamical systems, such as random continued fractions.

The present paper identifies only three types of $(h,\phi )$-entropies that are linear—up to obvious equivalence conditions—for QP sequences: Shannon and Rényi entropies, plus a third one, the logarithm of Taneja entropy, which we will call log-Taneja entropy. This result, valid for all QP sequences, extends [7], dedicated to Markov chains. As stated in Theorem 2, it relies on the asymptotic behavior of $(h,\phi )$-entropies stated in Theorem 1; in Section 4, we highlight a classification of $(h,\phi )$-entropies into five exclusive classes, depending on the growth rate of the marginal entropies: linear, over-linear, sub-linear, contracting, constant.

Further, a pertinent choice of the function h changes the class of the $\phi$-entropy. A well-known example is provided by the linear Rényi entropy that appears as the logarithm of Tsallis entropy (up to constants) while the latter is either constant or over-linear, depending on the parameter. Another interesting case considered below is the Sharma–Taneja–Mittal (STM) entropy

$${\mathbb{T}}_{\mathfrak{q},\mathfrak{p}}\left(X\right)=a_{\mathfrak{q},\mathfrak{p}}\sum _{i\in E}\left[m{\left(i\right)}^{\mathfrak{q}}-m{\left(i\right)}^{\mathfrak{p}}\right],$$

(5)

where $a_{\mathfrak{q},\mathfrak{p}}={\left(2^{1-\mathfrak{q}}-2^{1-\mathfrak{p}}\right)}^{-1}$, and $\mathfrak{q}\ne \mathfrak{p}$ are two positive parameters; see [8]. Frank and Plastino [9] show that STM entropy is the only entropy that gives rise to thermostatistics based on escort mean values, while [10] and the references therein highlight connections to statistical mechanics. The $\phi$-entropy

$$\begin{array}{ccc}\hfill {\mathbb{T}}_{\mathfrak{q}}\left(X\right)& =& -a_{\mathfrak{q}}\sum _{i\in E}m{\left(i\right)}^{\mathfrak{q}}logm\left(i\right),\hfill \end{array}$$

(6)

where $a_{\mathfrak{q}}=2^{\mathfrak{q}-1}$, called Taneja entropy in the literature, appears as the limit of ${\mathbb{T}}_{\mathfrak{q},\mathfrak{p}}\left(X\right)$ as $\mathfrak{p}$ tends to $\mathfrak{q}$, with $\mathfrak{q}>0$ fixed, and Shannon entropy is obtained for $\mathfrak{q}=1$. The logarithm transforms both of these contracting or over-linear entropies into linear ones. In other words, their associated entropy rates (2) will be non-trivial for QP sequences. Since, to our knowledge, the log-STM and log-Taneja entropies have never been considered in the literature, this paper ends by studying their main properties.

2. Quasi Power Log Entropies

Generally speaking, the $(h,\phi )$-entropies defined by (1), with $\phi :[0,1]\to {\mathbb{R}}_{+}$ and $h:{\mathbb{R}}^{+}\to \mathbb{R}$, are such that either $\phi$ is concave and h increases or $\phi$ is convex and h decreases. Moreover, with h and $\phi$, ${\mathbb{S}}_{h,\phi }$ takes only non-negative values. For most functionals of interest in the literature, h and $\phi$ are locally equivalent to products of power and logarithmic functions, which led in [4] to the definition of the class of quasi-power log entropies.

A $\phi$-entropy is a quasi-power log (QPL) if $a\in {\mathbb{R}}^{*}$, $s\in \mathbb{R}$ and $\delta \in \{0,1\}$ exist, such that $\phi$ is QPL at 0, in the sense that

$$\phi \left(x\right)\underset{0}{\sim }a{x}^s{\left(logx\right)}^{\delta }.$$

(7)

Further, an $(h,\phi )$-entropy is QPL if both (7) are satisfied and $b\in {\mathbb{R}}_{+}^{*}$, $t\in \mathbb{R}$ and $\epsilon \in \{0,1\}$ exist, such that

$$h\left(z\right)\underset{\Phi }{\sim }b{z}^t{(logz)}^{\epsilon },$$

(8)

where

$$\Phi =\left\{\begin{array}{cc}0\hfill & \mathrm{if}s>1,\hfill \\ a\hfill & \mathrm{if}s=1,\delta =0,\hfill \\ \mathcal{S}\left(a\right)\infty \hfill & \mathrm{if}s<1,\delta =0,\hfill \\ -\mathcal{S}\left(a\right)\infty \hfill & \mathrm{if}s\le 1,\delta =1,\hfill \end{array}\right.$$

with $\mathcal{S}\left(a\right)$ denoting the sign of a; see [11]. Of course, (8) makes sense only if $z^t{(logz)}^{ϵ}$ is well defined in a neighborhood of $\Phi$, inducing the following restrictions on the parameters:

$$(\delta =0)\Rightarrow (a>0) \mathrm{or} (t\in \mathbb{Z},\epsilon =0)\mathrm{and}(\delta =1)\Rightarrow (a<0) \mathrm{or} (t\in \mathbb{Z},\epsilon =0).$$

Note that for $s=0$ and $\delta =0$, the $\phi$-entropy collapses to a constant. Column 1 of

Table 1. Some classical entropies, with parameters $\mathfrak{p},\mathfrak{q}>0$. From left to right: Parameters of (7), $\Phi =lim{\mathbb{S}}_{\phi }\left(X_{1:n}\right)$, parameters of (8), $h\{\Phi \}={lim}_n{\mathbb{S}}_{h,\phi }\left(X_{1:n}\right)$, rescaling sequence, entropy rate and type (−1: contracting, 0: constant, 1: sub-linear, 2: linear, 3: over-linear).

Entropy	$[\mathit{a},\mathit{s},\mathit{\delta }]$		$\mathbf{\Phi }$		$[\mathit{b},\mathit{t},\mathit{\epsilon }]$	$\mathit{h}\left\{\mathbf{\Phi }\right\}$	${\mathit{r}}_{\mathit{n}}$	${\mathbb{H}}_{\mathit{h},\mathit{\phi }}\left(\mathbf{X}\right)$	Type
$\mathbf{\phi }\left(\mathit{x}\right),\mathit{h}\left(\mathit{z}\right)$
Shannon $\mathbb{S}$	$[-1,1,1]$		$+\infty$		$[1,1,0]$	$+\infty$	$n-1$	$-{\lambda }^{\prime }\left(1\right)$	2
$-xlogx,z$
Rényi ${\mathbb{R}}_{\mathfrak{q}}$	$[1,\mathfrak{q},0]$		$\mathfrak{q}>1$:	0	$\left[\frac{1}{1-\mathfrak{q}},0,1\right]$	$+\infty$	$n-1$	$\frac{1}{1-\mathfrak{q}}log\lambda \left(\mathfrak{q}\right)$	2
$x^{\mathfrak{q}},\frac{1}{1-\mathfrak{q}}logz$			$\mathfrak{q}<1$:	$+\infty$
Tsallis	$[1,\mathfrak{q},0]$		$\mathfrak{q}>1$:	0	$\left[\frac{1}{\mathfrak{q}-1},0,0\right]$	$\frac{1}{\mathfrak{q}-1}$	1	$\frac{1}{\mathfrak{q}-1}$	0
$x^{\mathfrak{q}},\frac{1}{\mathfrak{q}-1}(1-z)$			$\mathfrak{q}<1$:	$+\infty$	$\left[\frac{1}{1-\mathfrak{q}},1,0\right]$	$+\infty$	$\lambda {\left(\mathfrak{q}\right)}^{n-1}$	$\frac{1}{1-\mathfrak{q}}c\left(\mathfrak{q}\right)$	3
STM ${\mathbb{T}}_{\mathfrak{q},\mathfrak{p}}$ $x^{\mathfrak{q}}-x^{\mathfrak{p}},\frac{z}{\mathfrak{p}-\mathfrak{q}}$	$\mathfrak{q}>\mathfrak{p}$:	$[-1,\mathfrak{p},0]$	$\mathfrak{p}>1$:	0	$\left[\frac{1}{\mathfrak{p}-\mathfrak{q}},1,0\right]$	0	$\lambda {\left(\mathfrak{p}\right)}^{n-1}$	$\frac{1}{\mathfrak{q}-\mathfrak{p}}c\left(\mathfrak{p}\right)$	−1
			$\mathfrak{p}<1$:	$-\infty$		$+\infty$			3
	$\mathfrak{q}<\mathfrak{p}$:	$[1,\mathfrak{q},0]$	$\mathfrak{p}>1$:	0		0	$\lambda {\left(\mathfrak{q}\right)}^{n-1}$	$\frac{1}{\mathfrak{p}-\mathfrak{q}}c\left(\mathfrak{q}\right)$	−1
			$\mathfrak{p}<1$:	$+\infty$		$+\infty$			3
Taneja ${\mathbb{T}}_{\mathfrak{q}}$	$[-1,\mathfrak{q},1]$		$\mathfrak{q}>1$:	0	$[1,1,0]$	0	$(n-1)\lambda {\left(\mathfrak{q}\right)}^{n-1}$	$-c\left(\mathfrak{q}\right)\frac{{\lambda }^{\prime }\left(\mathfrak{q}\right)}{\lambda \left(\mathfrak{q}\right)}$	−1
$-x^{\mathfrak{q}}logx,z$			$\mathfrak{q}<1$:	$+\infty$		$+\infty$			3

shows the most classical entropies in the literature that are QPL.

3. Asymptotic Behavior of the Marginal Entropies of QP Sequences

The asymptotic behaviors of the sequences of marginal QPL entropies of any random sequence $\mathbf{X}={\left(X_n\right)}_{n\in {\mathbb{N}}^{*}}$ are controlled by the Dirichlet series (3) of the sequences, as made explicit in the next theorem. This special case of [11] (Theorem 1), where the asymptotic behaviors of divergence functionals are addressed, deserves to be detailed for the entropy of QP sequences. Note that Markov chains are considered in [7] in order to obtain weighted rate formulas involving Perron–Frobenius eigenvalues and eigenvectors.

Theorem 1. 

Let $\mathbf{X}={\left(X_n\right)}_{n\in {\mathbb{N}}^{*}}$ be a QP sequence. For any QPL φ-entropy,

$${\mathbb{S}}_{\phi }\left(X_{1:n}\right)\underset{n\to \infty }{\sim }{r}_n{\mathbb{H}}_{\phi }\left(\mathbf{X}\right),$$

(9)

with

$${\mathbb{H}}_{\phi }\left(\mathbf{X}\right)=ac\left(s\right){\left[\frac{{\lambda }^{\prime }\left(s\right)}{\lambda \left(s\right)}\right]}^{\delta }\mathrm{and}{r}_n={(n-1)}^{\delta }\lambda {\left(s\right)}^{n-1}.$$

(10)

Further, for any QPL $(h,\phi )$-entropy,

$${\mathbb{S}}_{h,\phi }\left(X_{1:n}\right)\underset{n\to \infty }{\sim }{r}_n{\mathbb{H}}_{h,\phi }\left(\mathbf{X}\right),$$

(11)

where

$$\begin{array}{ccc}\hfill {\mathbb{H}}_{h,\phi }\left(\mathbf{X}\right)=& & \left\{\begin{array}{cc}b{a}^{t}c{\left(s\right)}^t{\left[log\lambda \left(s\right)\right]}^{\epsilon }\hfill & if s\ne 1,\delta =0,\hfill \\ bc{\left(s\right)}^t{\left[a{\lambda }^{\prime }\left(s\right)\lambda {\left(s\right)}^{-1}\right]}^t{\left[log\lambda \left(s\right)\right]}^{\epsilon }\hfill & if s\ne 1,\delta =1,\hfill \\ b{a}^t{\left[loga\right]}^{\epsilon }\hfill & if s=1,\delta =0,\hfill \\ b{\left[a{\lambda }^{\prime }\left(1\right)\right]}^t\hfill & if s=1,\delta =1,\hfill \end{array}\right.\hfill \end{array}$$

and

$$r_n={(n-1)}^{\delta t+\epsilon }\lambda {\left(s\right)}^{t(n-1)}\mathit{if} s\ne 1,\mathit{and}{r}_n={(n-1)}^{\delta t}{\left(log(n-1)\right)}^{\delta \epsilon }\mathit{if} s=1.$$

(12)

Proof. 

For any QP sequence, the probability $m_{1:n}\left(i_{1:n}\right)$ converges to 0 as n tends to infinity, for any sequence $\left(i_n\right)\in E^{{\mathbb{N}}^{*}}$. Since $\phi$ is QPL at 0, and thanks to (4),

$$\begin{array}{c}\hfill {\mathbb{S}}_{\phi }\left(X_{1:n}\right)=\sum _{i_{1:n}\in E^n}\phi \left(m_{1:n}\left(i_{1:n}\right)\right)\underset{\infty }{\sim }\left\{\begin{array}{cc}{\displaystyle a{\Lambda }_n\left(s\right)\underset{\infty }{\sim }\lambda {\left(s\right)}^{n-1}ac\left(s\right)}\hfill & \mathrm{if}\delta =0,\hfill \\ {\displaystyle a{\Lambda }_n^{\prime }\left(s\right)\underset{\infty }{\sim }(n-1)\lambda {\left(s\right)}^{n-1}a\frac{{\lambda }^{\prime }\left(s\right)}{\lambda \left(s\right)}c\left(s\right)}\hfill & \mathrm{if}\delta =1,\hfill \end{array}\right.\end{array}$$

and (9) follows.

Further, if h satisfies (8), then for $s\ne 1$,

$$\begin{array}{c}\hfill {\mathbb{S}}_{\phi }\left(X_{1:n}\right)=\left\{\begin{array}{cc}\lambda {\left(s\right)}^{(n-1)t}b{a}^{t}c{\left(s\right)}^t\hfill & \mathrm{if}\delta =0,\epsilon =0,\hfill \\ (n-1)\lambda {\left(s\right)}^{(n-1)t}b{a}^{t}c{\left(s\right)}^{t}log\lambda \left(s\right)\hfill & \mathrm{if}\delta =0,\epsilon =1,\hfill \\ {(n-1)}^t\lambda {\left(s\right)}^{(n-1)t}bc{\left(s\right)}^t{\left[a\frac{{\lambda }^{\prime }\left(s\right)}{\lambda \left(s\right)}\right]}^t\hfill & \mathrm{if}\delta =1,\epsilon =0,\hfill \\ {(n-1)}^{t+1}\lambda {\left(s\right)}^{(n-1)t}\hfill & \\ \phantom{blabla}\times bc{\left(s\right)}^t{\left[a\frac{{\lambda }^{\prime }\left(s\right)}{\lambda \left(s\right)}\right]}^{t}log\lambda \left(s\right)\hfill & \mathrm{if}\delta =1,\epsilon =1,\hfill \end{array}\right.\end{array}$$

and for $s=1$,

$$\begin{array}{c}\hfill {\mathbb{S}}_{h,\phi }\left(X_{1:n}\right)\underset{\infty }{\sim }\left\{\begin{array}{cc}b{a}^t\hfill & \mathrm{if}\delta =0,\epsilon =0,\hfill \\ b{a}^{t}loga\hfill & \mathrm{if}\delta =0,\epsilon =1,\hfill \\ {(n-1)}^{t}b{\left[a{\lambda }^{\prime }\left(1\right)\right]}^t\hfill & \mathrm{if}\delta =1,\epsilon =0,\hfill \\ {(n-1)}^{t}log(n-1)b{\left[a{\lambda }^{\prime }\left(1\right)\right]}^t\hfill & \mathrm{if}\delta =1,\epsilon =1,\hfill \end{array}\right.\end{array}$$

and (11) follows. □

In [7,11], the first quantities in (10) and (12) are called rescaled $\phi$-entropy and $(h,\phi )$-entropy rates of $\mathbf{X}$. They constitute proper—non-degenerated—information measures associated with $\phi$ or $(h,\phi )$-entropies. The rescaled entropy rate ${\mathbb{H}}_{h,\phi }$ depends on $\mathbf{X}$ only through the parameters $[\lambda ,c]$ of the QPP; see [7] for an interpretation of these parameters for Markov chains. Note that the averaging sequence is defined up to the asymptotic equivalence $r_n^{\prime }{\sim }_{\infty }{r}_n$.

Table 1 presents (for the classical entropies) the values of $\Phi$, the averaging sequence $\mathbf{r}={\left(r_n\right)}_n$, and the rescaled entropy rates ${\mathbb{H}}_{h,\phi }\left(\mathbf{X}\right)$, depending on the values of the parameters.

Table 2. Limit marginal $\phi$-entropy $\Phi ={lim}_n{\mathbb{S}}_{\phi }\left(X_{1:n}\right)$, averaging sequence $\mathbf{r}={\left(r_n\right)}_{n\in \mathbb{N}}$, and ${\mathbb{H}}_{\phi }\left(\mathbf{X}\right)$, according to the values of a, s, $\delta$, and sign $\mathcal{S}\left(a\right)$ of a.

	$\mathit{s}<1$	$\mathit{s}=1$	$\mathit{s}>1$
$\delta =0$	$\mathcal{S}\left(a\right)\infty$	a	0	$\Phi$
	$\lambda {\left(s\right)}^{n-1}$	1	$\lambda {\left(s\right)}^{n-1}$	$r_n$
	$ac\left(s\right)$	a	$ac\left(s\right)$	${\mathbb{H}}_{\phi }\left(\mathbf{X}\right)$
$\delta =1$	$-\mathcal{S}\left(a\right)\infty$	$-\mathcal{S}\left(a\right)\infty$	0	$\Phi$
	$(n-1)\lambda {\left(s\right)}^{n-1}$	$n-1$	$(n-1)\lambda {\left(s\right)}^{n-1}$	$r_n$
	$ac\left(s\right)\frac{{\lambda }^{\prime }\left(s\right)}{\lambda \left(s\right)}$	$a{\lambda }^{\prime }\left(1\right)$	$ac\left(s\right)\frac{{\lambda }^{\prime }\left(s\right)}{\lambda \left(s\right)}$	${\mathbb{H}}_{\phi }\left(\mathbf{X}\right)$

details the dependencies for $\phi$-entropies.

4. Classification of QPL $(\mathit{h},\mathbf{\phi })$-Entropies—Dependence on $\mathit{h}$

Classification of all QPL $(h,\phi )$-entropies into five exclusive classes derives from Theorem 1, for QP sequences, according to the asymptotic behavior of the marginal entropy, or, equivalently, to the type of rescaling sequence $\mathbf{r}$. The important point is that this classification depends only on the functions $\phi$ and h—through the parameters $[s,\delta ]$ and $[t,\epsilon ]$ involved in the QPL properties (7) and (8), and not in the specific dynamics of the QP sequence $\mathbf{X}$. In particular, the classification is valid for a wide class of atomic Markov chains, including finite ones, as shown in [7].

The classification of $\phi$-entropies into four classes easily derives from either (9) or Table 2. Indeed, four types of behaviors for the sequences of marginal entropies are possible, according to the values of s and $\delta$. First, for $s=1$ and $\delta =1$ (Shannon entropy or equivalent), the marginal entropy ${\mathbb{S}}_{h,\phi }\left(X_{1:n}\right)$ increases linearly with n and, hence, $r_n=n$; it is the only linear QPL $\phi$-entropy. Second, for $s<1,$ $\delta =0,$ and $s<1,$ $\delta =1$, the rescaling sequences are respectively $\lambda {\left(s\right)}^{n-1}$ and $(n-1)\lambda {\left(s\right)}^{n-1}$. Equation (4) states that $\lambda$ is decreasing with s and that $\lambda \left(1\right)=1$. So, $\lambda \left(s\right)>1$ for $s<1$, and the marginal entropy explodes exponentially fast, up to $\Phi ={lim}_n{\mathbb{S}}_{\phi }\left(X_{1:n}\right)$; such an entropy is called over-linear or expanding. Third, for $s>1$ (with either $\delta =0$ or 1), both $r_n$ and the marginal entropy decrease (in absolute value) exponentially fast to 0; such an entropy is called contracting. Finally, for $s=1$ and $\delta =0$ (Tsallis entropy or equivalent), the marginal entropy converges to some finite value, and the rescaling sequence is constant.

The discussion easily extends to $(h,\phi )$-entropies through the asymptotic behaviors (11) of marginal entropies or the rescaling sequence $r_n$ in (12). The following classification of $(h,\phi )$-entropies ensues, shown in

Table 3. Type of the $(h,\phi )$-entropy according to the parameters in (7) and (8): $-1$ contracting, 0 constant, 1 sub-linear, $\mathbf{2}$ linear, 3 over-linear.

	$\mathit{t}<0$	$\mathit{t}=0,\mathit{\epsilon }=0$	$\mathit{t}=0,\mathit{\epsilon }=1$	$\mathit{t}>0$
$\delta =0$, $s<1$	−1	0	2	3
$\delta =0$, $s=1$	0	0	0	0
$\delta =0$, $s>1$	3	0	2	−1
$\delta =1$, $s<1$	−1	0	2	3
$\delta =1$, $s>1$	3	0	2	−1
$\delta =1$, $s=1$	$t<0$	$t=0,\epsilon =0$	$t=0,\epsilon =1$	$0<t<1$
	−1	0	1	1
	$t=1,\epsilon =0$	$t=1,\epsilon =1$	$t>1$
	2	3	3

according to parameters. The last column of Table 1 shows the classes of classical entropies.

• Contracting case:$r_n=o\left(1\right)$. The marginal entropy decreases to 0, typically exponentially fast.
• Constant case:$|r_n|=k\in {\mathbb{R}}_{+}$. The marginal entropy converges to a non-degenerate value.
• Sub-linear case:$r_n=o\left(n\right)$ and $r_n\in \omega \left(1\right)$. The marginal entropy increases slower than n, typically as $n^c$ or $n^{c}logn$, with $0<c<1$.
• Linear case:$r_n=kn$, with $k\in {\mathbb{R}}_{+}$. The marginal entropy increases linearly with the number of variables.
• Over-linear (or expanding) case:$r_n\in \omega \left(n\right)$. The marginal entropy increases faster than n, typically exponentially fast.

Transforming a $\phi$-entropy into an $(h,\phi )$-entropy with a QPL function h leads to a change of class in several cases. Any transformation h with $t<0$ exchanges the over-linear and contracting classes, while the choice of $\epsilon$ has no impact on these two classes. Taking $t=\epsilon =1$ would transform the linear Shannon entropy into an intricate entropy of the over-linear class.

Further, taking $t=0$ and $\epsilon =1$ transforms the Tsallis entropy of the constant class into the Rényi entropy of the linear class. It also transforms Sharma–Taneja–Mittal (STM) entropies (5), especially Taneja entropies (6) that are over-linear for $s=\mathfrak{q}<1$ and contracting for $s=\mathfrak{q}>1$, into entropies of the linear class.

5. Linear QPL Entropies for QP Sequences

Since both STM and Taneja entropies are positive (due to the pertinent choice of parameters), their logarithms are well defined, which allows us to derive new $(log,\phi )$-entropies and, hence, complete Shannon and Rényi entropies in the linear class.

Definition 1. 

The log-STM entropy and log-Taneja entropy are defined for any random variable X, taking values in a set E with distribution m, respectively, by ${\mathbb{L}}_{\mathfrak{q},\mathfrak{p}}\left(X\right)={\mathbb{S}}_{log,a_{\mathfrak{q},\mathfrak{p}}(x^{\mathfrak{q}}-x^{\mathfrak{p}})}\left(X\right)=log\left({\sum }_{i\in E}\left[m{\left(i\right)}^{\mathfrak{q}}-m{\left(i\right)}^{\mathfrak{p}}\right]\right),$ and ${\mathbb{L}}_{\mathfrak{q}}\left(X\right)={\mathbb{S}}_{log,-x^{\mathfrak{q}}logx}\left(X\right)=log\left[-{\sum }_{i\in E}m{\left(i\right)}^{\mathfrak{q}}logm\left(i\right)\right].$

In particular, the Rényi entropy appears as a log-STM entropy for $\mathfrak{p}=1$, up to multiplicative and additive constants, here omitted for the sake of simplicity.

The classification and ensuing discussion presented in Section 4 lead to the following characterization of the class of linear QPL $(h,\phi )$-entropies.

Theorem 2. 

Due to obvious equivalences, such as ${\phi }_1{\sim }_0{\phi }_2$ and $h_1{\sim }_{\Phi }{h}_2$, and multiplicative constants, exactly three types of linear QPL $(h,\phi )$-entropies exist for QP sequences: Shannon entropy, with $[s,\delta ,t,\epsilon ]=[1,1,1,0]$; Rényi entropies, with $[s,\delta ,t,\epsilon ]=[\mathfrak{q},0,0,1]$ and $\mathfrak{q}\ne 0$; log-Taneja entropies, with $[s,\delta ,t,\epsilon ]=[\mathfrak{q},1,0,1]$ and $\mathfrak{q}\ne 0$.

Moreover, the entropy rates associated with these entropies are either the Shannon entropy rate (for Shannon entropy) or Rényi (for both the Rényi and log-Taneja entropies).

Obviously, all log-STM entropies are also linear and equivalent to Rényi entropies since

$$a_{\mathfrak{q},\mathfrak{p}}(x^{\mathfrak{q}}-x^{\mathfrak{p}})\underset{0}{\sim }-a_{\mathfrak{q},\mathfrak{p}}{x}^{\mathfrak{p}}\mathrm{if}\mathfrak{q}>\mathfrak{p}\mathrm{and}{a}_{\mathfrak{q},\mathfrak{p}}(x^{\mathfrak{q}}-x^{\mathfrak{p}})\underset{0}{\sim }{a}_{\mathfrak{q},\mathfrak{p}}{x}^{\mathfrak{q}}\mathrm{if}\mathfrak{q}<\mathfrak{p}.$$

To our knowledge, log-Taneja entropies and log-STM entropies have never been considered in the literature, so we will now establish some of their properties.

Various properties of STM entropies are studied in [12]. STM and Taneja entropies are symmetric, continuous, and positive. Symmetry and continuity are preserved by composition by the logarithm, yielding symmetry and continuity of log-STM and log-Taneja entropies. However, ${\mathbb{L}}_{\mathfrak{q}}$ is clearly not positive.

For a number N of equiprobable states, the STM entropy is $a_{\mathfrak{q},\mathfrak{p}}(N^{1-\mathfrak{q}}-N^{1-\mathfrak{p}})$ while Taneja entropy is $a_{\mathfrak{q}}{N}^{1-\mathfrak{q}}logN$. Both increase with N except when $\mathfrak{q},\mathfrak{p}>1$. The property transfers to the logarithmic versions.

Borges and Roditi [12] show that STM-entropies are concave for $0<\mathfrak{q}<1<\mathfrak{p}$ and convex for both $0<\mathfrak{q}<1$ with $\mathfrak{p}<1$, and $\mathfrak{q},\mathfrak{p}<0$; all other cases present no definite concavity. Since the logarithm is a concave and increasing function, the log-STM entropies are also concave for $0<\mathfrak{q}<1<\mathfrak{p}$, the other cases being indefinite.

Scarfone [13] gives the maximum STM entropy distribution, while Beitollahi and Azhdari [14] compute Taneja entropy for numerous classical distributions, among which Bernoulli and geometric. Both transfer to the logarithmic versions.

STM entropies are axiomatically characterized as the family of $\phi$-entropies—with $\phi :[0,1]\to \mathbb{R}$, such that $\phi (1/2)=1/2$, satisfying the functional identity

$$\begin{array}{ccc}\hfill \sum _{i\in E}\sum _{j\in E}\phi \left(m\left(i\right)n\left(j\right)\right)& =& \sum _{i\in E}m{\left(i\right)}^{\mathfrak{q}}\sum _{j\in E}\phi \left(n\left(j\right)\right)+\sum _{j\in E}n{\left(j\right)}^{\mathfrak{p}}\sum _{i\in E}\phi \left(m\left(i\right)\right),\hfill \end{array}$$

(13)

where $m={\left(m\left(i\right)\right)}_{i\in E}$ and $n={\left(n\left(j\right)\right)}_{j\in E}$ denote arbitrary probability distributions on E; see [15] and the references therein. From a probabilistic point of view, (13) states that the entropy ${\mathbb{T}}_{\mathfrak{q},\mathfrak{p}}(X,Y)$ of a pair of independent variables, with distributions $m_X$ and $m_Y$, satisfies, for the Dirichlet series $\Lambda$ defined by (3),

$${\mathbb{T}}_{\mathfrak{q},\mathfrak{p}}(X,Y)=\Lambda (m_X;\mathfrak{q}){\mathbb{T}}_{\mathfrak{q},\mathfrak{p}}\left(Y\right)+\Lambda (m_Y;\mathfrak{p}){\mathbb{T}}_{\mathfrak{q},\mathfrak{p}}\left(X\right),$$

(14)

For any fixed $\mathfrak{q}$ and $\mathfrak{p}$ tending to $\mathfrak{q}$, (14) yields for the Taneja entropy of independent variables, ${\mathbb{T}}_{\mathfrak{q}}(X,Y)=\Lambda (m_X;\mathfrak{q}){\mathbb{T}}_{\mathfrak{q}}\left(Y\right)+\Lambda (m_Y;\mathfrak{q}){\mathbb{T}}_{\mathfrak{q}}\left(X\right).$ In particular, if $\Lambda (m_X;\mathfrak{q})=\Lambda (m_Y;\mathfrak{q})=\Lambda (m;\mathfrak{q})$, then ${\mathbb{T}}_{\mathfrak{q}}(X,Y)=\Lambda (m,\mathfrak{q})[{\mathbb{T}}_{\mathfrak{q}}\left(Y\right)+{\mathbb{T}}_{\mathfrak{q}}\left(X\right)],$ that is a “weighted additivity” rule for independent variables with equal Dirichlet series at $\mathfrak{q}$—or, equivalently, equal Rényi entropy of parameter $\mathfrak{q}$.

The axiomatic property (13) for STM entropies yields an alternative proof of the linearity of log-STM entropies for i.i.d. sequences of random variables. For the sake of simplicity, let us detail it for log-Taneja entropies. Let $\mathbf{X}=\left(X_n\right)$ be an i.i.d. sequence with common distribution m. Some simple calculation from the definition gives

$$\begin{array}{c}\hfill {\mathbb{L}}_{\mathfrak{q}}\left(m\right)=(1-\mathfrak{q}){\mathbb{R}}_{\mathfrak{q}}\left(m\right)+log\left[\frac{1}{\mathfrak{q}}\mathbb{S}\left(m^{*\mathfrak{q}}\right)+\left(1-\frac{1}{\mathfrak{q}}\right){\mathbb{R}}_{\mathfrak{q}}\left(m\right)\right],\end{array}$$

(15)

where $m^{*\mathfrak{q}}$ denotes the $\mathfrak{q}$-escort distribution associated with m. The escort transformation is known to preserve independence in the sense that ${\left(m^{\otimes n}\right)}^{*\mathfrak{q}}={\left(m^{*\mathfrak{q}}\right)}^{\otimes n}$; see, e.g., [5] and the references therein. Moreover, both the Shannon entropy $\mathbb{S}$ and Rényi entropy ${\mathbb{R}}_{\mathfrak{q}}$ are well-known to be additive for independent variables, and hence $\mathbb{S}\left({\left(m^{\otimes n}\right)}^{*\mathfrak{q}}\right)=n\mathbb{S}\left(m^{*\mathfrak{q}}\right)$ and ${\mathbb{R}}_{\mathfrak{q}}\left(m^{\otimes n}\right)=n{\mathbb{R}}_{\mathfrak{q}}\left(m\right)$. Applying (15) to $m^{\otimes n}$ thus yields

$$\begin{array}{c}\hfill {\mathbb{L}}_{\mathfrak{q}}\left(X_{1:n}\right)={\mathbb{L}}_{\mathfrak{q}}\left(m^{\otimes n}\right)=(1-\mathfrak{q})n{\mathbb{R}}_{\mathfrak{q}}\left(m\right)+log\left(n\left[\frac{1}{\mathfrak{q}}\mathbb{S}\left(m^{*\mathfrak{q}}\right)+\left(1-\frac{1}{\mathfrak{q}}\right){\mathbb{R}}_{\mathfrak{q}}\left(m\right)\right]\right).\end{array}$$

Obviously, the right-hand term of the sum is negligible with respect to n, so that ${\mathbb{L}}_{\mathfrak{q}}\left(X_{1:n}\right){\sim }_{\infty }n(1-\mathfrak{q}){\mathbb{R}}_{\mathfrak{q}}\left(m\right)$. Finally, the entropy rate of the sequence reduces to the Rényi entropy of the common distribution.