A Review of Fractional Order Entropies

Abstract

Fractional calculus (FC) is the area of calculus that generalizes the operations of differentiation and integration. FC operators are non-local and capture the history of dynamical effects present in many natural and artificial phenomena. Entropy is a measure of uncertainty, diversity and randomness often adopted for characterizing complex dynamical systems. Stemming from the synergies between the two areas, this paper reviews the concept of entropy in the framework of FC. Several new entropy definitions have been proposed in recent decades, expanding the scope of applicability of this seminal tool. However, FC is not yet well disseminated in the community of entropy. Therefore, new definitions based on FC can generalize both concepts in the theoretical and applied points of view. The time to come will prove to what extend the new formulations will be useful.

1. Introduction

In recent decades, the generalization of the concepts of differentiation [1,2,3,4] and entropy [5,6,7,8] have received considerable attention. In the first case we may cite the fractional calculus (FC) [9,10]. FC was introduced by Leibniz in the scope of mathematics by the end of the 17th century, but only recently found application in biology [11,12], physics [13,14] and engineering [15,16], among others [17,18]. The concept of entropy was introduced by Clausius [19] and Boltzmann [20] in the field of thermodynamics. Later, entropy was also explored by Shannon [21] and Jaynes [22] in the context of information theory. Meanwhile, both topics evolved considerably, motivating the formulation of fractional operators [23,24] and entropy indices [25,26,27,28,29,30,31,32,33,34,35,36,37,38]. These generalizations extend the application of the two mathematical tools and highlight certain characteristics, such as the power-law behavior, non-locality and long range memory [39,40].

This paper reviews the concept of entropy in the framework of FC. In fact, FC is not yet well disseminated among the community of entropy and, therefore, new definitions based on FC may expand the scope of this powerful tool. To the authors’ best knowledge, new entropy definitions are welcomed by the scientific community, somehow contrary to what happens with recent fractional operators. Consequently, the manuscript does not intend to assess the pros or the cons of the distinct formulations for some given problem. In a similar line of thought, the analysis of entropy-based indices proposed in the literature for comparing or characterizing some phenomena or probability distributions are outside the focus of this paper. Interested readers can obtain further information on divergence measures [41] and mutual information [42], as well as for sample [43], approximate [44], permutation [45], spectral [46], and fuzzy [47] entropies, among others [48]. Indeed, the main idea of this paper is to review the concept of fractional entropy and to present present day state of its development.

The paper is organized as follows. Section 2 presents the fundamental concepts of FC. Section 3 introduces different entropies with one, two and three parameters. Section 4 reviews the fractional-order entropy formulations. Section 5 compares the different formulations for four well-known distributions. Section 6 assesses the impact of the fractional entropies and analyses their main areas of application. Finally, Section 7 outlines the main conclusions.

2. Fractional-Order Derivatives and Integrals

FC models capture non-local effects, useful in the study of phenomena with long range correlations in time or space.

Let us consider the finite interval $[a,b]$, with $a,b\in \mathbb{R}$ and $a<b$, and let $n-1<q<n$, with $n\in \mathbb{N}$. The Euler’s gamma function is denoted by $\Gamma (·)$ and the operator $[·]$ calculates the integer part of the argument. Several definitions of fractional derivatives were formulated [24,49,50]. A small set is presented in the follow-up, which includes both historically relevant and widely used definitions:

• The left-side and the right-side Caputo derivatives,

  $$\begin{array}{c}\hfill {\displaystyle {}^C{D}_{a^{+}}^{q}f\left(t\right)=\frac{1}{\Gamma (n-q)}{\int }_a^x\frac{1}{{(x-\tau )}^{q-n+1}}\frac{d^n}{d{\tau }^n}f\left(\tau \right)d\tau ,x\ge a,}\end{array}$$

  (1)

  $$\begin{array}{c}\hfill {\displaystyle {}^C{D}_{b^{-}}^{q}f\left(x\right)=\frac{{(-1)}^n}{\Gamma (n-q)}{\int }_x^b\frac{1}{{(\tau -x)}^{q-n+1}}\frac{d^n}{d{\tau }^n}f\left(\tau \right)d\tau ,x\le b,}\end{array}$$

  (2)
• The left-side and the right-side Grünwald-Letnikov derivatives,

  $$\begin{array}{c}\hfill {}^{GL}{D}_{a^{+}}^{q}f\left(x\right)=\underset{h\to 0}{lim}{h}^{-q}\sum _{m=0}^{\left[\frac{x-a}{h}\right]}{(-1)}^m\left(\begin{array}{c}q\\ m\end{array}\right)f(x-mh),x\ge a,\end{array}$$

  (3)

  $$\begin{array}{c}\hfill {}^{GL}{D}_{b^{-}}^{q}f\left(x\right)=\underset{h\to 0}{lim}{h}^{-q}\sum _{m=0}^{\left[\frac{b-x}{h}\right]}{(-1)}^m\left(\begin{array}{c}q\\ m\end{array}\right)f(x+mh),x\le b,\end{array}$$

  (4)
• The Hadamard derivative,

  $$\begin{array}{c}\hfill {\displaystyle {}^{Ha}{D}_{+}^{q}f\left(x\right)=\frac{q}{\Gamma (1-q)}{\int }_0^x\frac{f\left(x\right)-f\left(\tau \right)}{{[log(x/\tau )]}^{q+1}}\frac{d\tau }{\tau },}\end{array}$$

  (5)
• The left-side and right-side Hilfer derivatives of type $0\le \beta \le 1$,

  $$\begin{array}{c}\hfill {}^H{D}_{a^{+}}^{q}f\left(x\right){=}^{RL}{I}_{a^{+}}^{\gamma -q}{\frac{d^n}{d{x}^n}}^{RL}{I}_{a^{+}}^{(1-\beta )(n-q)}f\left(x\right),\end{array}$$

  (6)

  $$\begin{array}{c}\hfill {}^H{D}_{b^{-}}^{q}f\left(x\right){=}^{RL}{I}_{b^{-}}^{\gamma -q}(-1){\frac{d^n}{d{x}^n}}^{RL}{I}_{b^{-}}^{(1-\beta )(n-q)}f\left(x\right),\end{array}$$

  (7)

  where ${}^{RL}{I}_{a^{+}}^q$ and ${}^{RL}{I}_{b^{-}}^q$ denote the left-side and right-side Riemann-Liouville fractional integrals of order $q>0$, respectively, defined by:

  $$\begin{array}{c}\hfill {\displaystyle {}^{RL}{I}_{a^{+}}^{q}f\left(x\right)=\frac{1}{\Gamma \left(q\right)}{\int }_a^x\frac{f\left(\tau \right)}{{(x-\tau )}^{1-q}}d\tau ,x\ge a,}\end{array}$$

  (8)

  $$\begin{array}{c}\hfill {\displaystyle {}^{RL}{I}_{b^{-}}^{q}f\left(x\right)=\frac{1}{\Gamma \left(q\right)}{\int }_x^b\frac{f\left(\tau \right)}{{(\tau -x)}^{1-q}}d\tau ,x\le b,}\end{array}$$

  (9)
• The Karcı derivative

  $${}^K{D}^{q}f\left(x\right)=\underset{h\to 0}{lim}\frac{\frac{d\{{\left[f(x+h)\right]}^q-{\left[f\left(x\right)\right]}^q\}}{dh}}{\frac{d[{(x+h)}^q-x^q]}{dh}}=\frac{\frac{d}{dx}\left[f\left(x\right)\right]·\left[f{\left(x\right)}^{q-1}\right]}{x^{q-1}}.$$

  (10)
• The Liouville, the left-side and the right-side Liouville derivatives,

  $$\begin{array}{c}\hfill {\displaystyle {}^L{D}^{q}f\left(x\right)=\frac{1}{\Gamma (1-q)}\frac{d}{dx}{\int }_{-\infty }^x\frac{f\left(\tau \right)}{{(x-\tau )}^q}d\tau ,-\infty <x<+\infty ,}\end{array}$$

  (11)

  $$\begin{array}{c}\hfill {\displaystyle {}^L{D}_{0^{+}}^{q}f\left(x\right)=\frac{1}{\Gamma (n-q)}\frac{d^n}{d{x}^n}{\int }_0^x\frac{f\left(\tau \right)}{{(x-\tau )}^{q-n+1}}d\tau ,x>0,}\end{array}$$

  (12)

  $$\begin{array}{c}\hfill {\displaystyle {}^L{D}_{0^{-}}^{q}f\left(x\right)=\frac{{(-1)}^n}{\Gamma (n-q)}\frac{d^n}{d{x}^n}{\int }_x^{+\infty }\frac{f\left(\tau \right)}{{(x-\tau )}^{q-n+1}}d\tau ,x<+\infty ,}\end{array}$$

  (13)
• The Marchaud, the left-side and the right-side Marchaud derivatives,

  $$\begin{array}{c}\hfill {\displaystyle {}^M{D}_{}^{q}f\left(x\right)=\frac{q}{\Gamma (1-q)}{\int }_{-\infty }^x\frac{f\left(x\right)-f\left(\tau \right)}{{(x-\tau )}^{q+1}}d\tau ,}\end{array}$$

  (14)

  $$\begin{array}{c}\hfill {\displaystyle {}^M{D}_{+}^{q}f\left(x\right)=\frac{q}{\Gamma (1-q)}{\int }_0^{+\infty }\frac{f\left(x\right)-f(x-\tau )}{{\tau }^{q+1}}d\tau ,}\end{array}$$

  (15)

  $$\begin{array}{c}\hfill {\displaystyle {}^M{D}_{-}^{q}f\left(x\right)=\frac{q}{\Gamma (1-q)}{\int }_0^{+\infty }\frac{f\left(x\right)-f(x+\tau )}{{\tau }^{q+1}}d\tau ,}\end{array}$$

  (16)
• The left-side and the right-side Riemann-Liouville derivatives,

  $$\begin{array}{c}\hfill {\displaystyle {}^{RL}{D}_{a^{+}}^{q}f\left(x\right)=\frac{1}{\Gamma (n-q)}\frac{d^n}{d{x}^n}{\int }_a^x\frac{f\left(\tau \right)}{{(x-\tau )}^{q-n+1}}d\tau ,x\ge a,}\end{array}$$

  (17)

  $$\begin{array}{c}\hfill {\displaystyle {}^{RL}{D}_{b^{-}}^{q}f\left(x\right)=\frac{{(-1)}^n}{\Gamma (n-q)}\frac{d^n}{d{x}^n}{\int }_x^b\frac{f\left(\tau \right)}{{(\tau -x)}^{q-n+1}}d\tau ,x\le b,}\end{array}$$

  (18)
• The Riesz derivative,

  $$\begin{array}{c}\hfill {}^R{D}_x^{q}f\left(x\right)=-\frac{1}{2cos(q\pi /2)}\frac{1}{\Gamma \left(q\right)}\frac{d^n}{d{x}^n}\left\{{\displaystyle {\int }_{-\infty }^x\frac{f\left(\tau \right)}{{(x-\tau )}^{q-n+1}}d\tau +{\int }_x^{+\infty }\frac{f\left(\tau \right)}{{(\tau -x)}^{q-n+1}}d\tau }\right\},\end{array}$$

  (19)
• The local Yang derivative,

  $$\begin{array}{c}\hfill {}^Y{D}_{-}^{q}f{\left(x\right)}_{|x=x_0}=\underset{x\to x_0}{lim}\frac{{\Delta }^q[f\left(x\right)-f\left(x_0\right]}{{(x-x_0)}^q}.\end{array}$$

  (20)

Often, the Caputo formulation is applied in physics and numerical integration, the Riemann-Liouville in calculus, and the Grünwald-Letnikov in engineering, signal processing and control. These classical definitions are the most frequently used by researchers. In what concerns the mathematical pros and cons of the Karcı and the Yang derivatives, readers may visit [23,51] and references therein. In fact, it should be noted that some formulations need some careful reflection and are the matter of some controversy, since many authors do not consider them as fractional operators [23,52,53]. Nevertheless, the debate about what it really means the term ‘fractional derivative’ is still ongoing among contemporary mathematicians [51].

3. The Concept of Entropy

Let us consider a discrete probability distribution $\mathbf{P}=\{p_1,p_2,\cdots ,p_N\}$, with ${\sum }_i{p}_i=1$ and $p_i\ge 0$. The Shannon entropy, $S^{\left(S\right)}$, of distribution $\mathbf{P}$ is defined as:

$$S^{\left(S\right)}=\sum _i{p}_{i}I\left(p_i\right)=-\sum _i{p}_{i}ln{p}_i,$$

(21)

and represents the expected value of the information content given by $I\left(p_i\right)=-ln{p}_i$. Therefore, for the uniform probability distribution we have $p_i=N^{-1}$, $N\in \mathbb{N}$, and the Shannon entropy takes its maximum value $S=lnN$, yielding the Boltzmann formula, up to a multiplicative factor, k, which denotes the Boltzmann constant.

The Rényi and Tsallis entropies are one-parameter generalizations of (21) given by, respectively:

$$S_q^{\left(R\right)}=\frac{1}{1-q}ln\left(\sum _i{p}_i^q\right),q>0,q\ne 1,$$

(22)

$$S_q^{\left(T\right)}=\frac{1}{q-1}\left(1-\sum _i{p}_i^q\right),q\in \mathbb{R}.$$

(23)

The entropies $S_q^{\left(R\right)}$ and $S_q^{\left(T\right)}$ reduce to the Shannon formulation $S^{\left(S\right)}$ when $q\to 1$. The Rényi entropy has an inverse power law equilibrium distribution [54], satisfying the zero-th law of thermodynamics [55]. It is important in statistics and ecology to quantify diversity, in quantum information to measure entanglement, and in computer science for randomness extraction. The Tsallis entropy was proposed in the scope of nonextensive statistical mechanics and has found application in the field of complex dynamics, in diffusion equations [56] and Fokker-Planck systems [57].

Other one-parameter entropies are the Landsberg-Vedral and Abe formulations [26,58]:

$$S_q^{\left(L\right)}=\frac{1}{1-q}\left(\frac{1}{{\sum }_i{p}_i^q}-1\right),$$

(24)

$${\displaystyle S_q^{\left(A\right)}=-\sum _i\frac{p_i^q-p_i^{q^{-1}}}{q-q^{-1}},q\in ]0,1].}$$

(25)

Expression (24) is related to the Tsallis entropy by $S_q^{\left(L\right)}=\frac{S_q^{\left(T\right)}}{{\sum }_i{p}_i^q}$, and is often known as normalized Tsallis entropy. Expression (25) is a symmetric modification of the Tsallis entropy, which is invariant to the exchange $q\leftrightarrow q^{-1}$, and we have $S_q^{\left(A\right)}=\frac{(q-1)S_q^{\left(T\right)}-q^{-1}{S}_{q^{-1}}^{\left(T\right)}}{q-q^{-1}}$.

The two-parameter Sharma-Mittal entropy [32] is a generalization of the Shannon, Tsallis and Rényi entropies, and is defined as follows:

$$S_{r,q}^{\left(SM\right)}=\frac{1}{1-r}\left[{\left(\sum _i{p}_i^q\right)}^{\frac{1-r}{1-q}}-1\right],q>0,q\ne 1,r\ne 1.$$

(26)

The Sharma-Mittal entropy reduces to the Rényi, Tsallis and Shannon’s formulations for the limits $r\to 1$, $r\to q$ and $\{r,q\}\to \{1,1\}$, respectively.

Examples of three-parameter formulations consist of the gamma and the Kaniadakis entropies, $S_{d,c_1,c_2}^{\left(G\right)}$ and $S_{\kappa ,\tau ,\zeta }^{\left(K\right)}$, respectively. The gamma entropy is given by [35]:

$${\displaystyle S_{d,c_1,c_2}^{\left(G\right)}=\sum _i\frac{e}{c_2-c_1}\Gamma (d+1,1-c_{1}ln{p}_i,1-c_{2}ln{p}_i),}$$

(27)

where e denotes the Napier constant, $\Gamma (a,z_1,z_2)$ represents the generalized incomplete gamma function, defined by:

$$\Gamma (a,z_1,z_2)=\Gamma (a,z_1)-\Gamma (a,z_2)={\int }_{z_1}^{z_2}{t}^{a-1}{e}^{-t}dt$$

(28)

and $\Gamma (x,y)={\int }_y^{\infty }{t}^{x-1}{e}^{-t}dt$ is the upper incomplete gamma function.

The entropy $S_{d,c_1,c_2}^{\left(G\right)}$ follows the first three Khinchin axioms [35,59,60] within the parameter regions defined by (29) and (30):

$$c_2>1>c_1>0,1-\frac{1}{c_1}<d<1-\frac{1}{c_2},$$

(29)

$$c_1>1>c_2>0,1-\frac{1}{c_2}<d<1-\frac{1}{c_1}.$$

(30)

Different combinations of the parameters yield distinct entropy formulations [35]. For example, if we set $\{d,c_1,c_2\}=\{0,1,q\}$, then we recover the Tsallis entropy, while for $\{d,c_1,c_2\}=\{0,1\pm ϵ,1\mp ϵ\}$, $ϵ\to 0$, we obtain the Shannon entropy.

The Kaniadakis entropy belongs to a class of trace-form entropies given by [38]:

$${\displaystyle S=-\sum _i{p}_i\Lambda \left(p_i\right),}$$

(31)

where $\Lambda \left(x\right)$ is a strictly increasing function defined for positive values of the argument, noting that $\Lambda (x\to 0^{+})=-\infty$. The function $\Lambda \left(x\right)$ can be viewed as a generalization of the ordinary logarithm [38] that, for three-parameter, yields:

$$\Lambda \left(x\right)={ln}_{\kappa ,\tau ,\zeta }\left(x\right)=\frac{{\zeta }^{\kappa }{x}^{\tau +\kappa }-{\zeta }^{-\kappa }{x}^{\tau -\kappa }-{\zeta }^{\kappa }+{\zeta }^{-\kappa }}{(\kappa +\tau ){\zeta }^{\kappa }+(\kappa -\tau ){\zeta }^{-\kappa }}.$$

(32)

Therefore, the Kaniadakis entropy, $S_{\kappa ,\tau ,\zeta }^{\left(K\right)}$, can be expressed as:

$${\displaystyle S_{\kappa ,\tau ,\zeta }^{\left(K\right)}=-\sum _i{p}_i{ln}_{\kappa ,\tau ,\zeta }\left(p_i\right),\kappa ,\zeta \in \mathbb{R},\left|\kappa \right|-1<\tau \le \left|\kappa \right|.}$$

(33)

The Entropy $S_{\kappa ,\tau ,\zeta }^{\left(K\right)}$ is Lesche [61] and thermodynamically [62] stable for $-\left|\kappa \right|\le \tau \le \left|\kappa \right|$. Distinct combinations of the parameters yield several entropy formulations [38]. For example, if we set $\kappa =\tau =(q-1)/2$ or $\kappa \to 0$, $\tau =0$, then expression (33) yields the Tsallis and the Shannon entropies, respectively.

Other entropies can be found in the literature [63,64], but a thorough review of all proposed formulations is out of the scope of this paper.

4. Fractional Generalizations of Entropy

It was noted [65] that the Shannon and Tsallis entropies have the same generating function ${\sum }_i{p}_i^x$ and that the difference in the Formulas (21) and (23) is just due to the adopted differentiation operator. In fact, using the standard first-order differentiation, $\frac{d}{dx}$, we obtain the Shannon entropy:

$${\displaystyle S^{\left(S\right)}=\underset{x\to -1}{lim}\frac{d}{dx}\sum _i{p}_i^{-x},}$$

(34)

while adopting the Jackson q-derivative [66], $D_{q}f\left(x\right)=\frac{f\left(qx\right)-f\left(x\right)}{qx-x}$, $0<q<1$, yields the Tsallis entropy [28]:

$${\displaystyle S_q^{\left(T\right)}=\underset{x\to -1}{lim}{D}_q\sum _i{p}_i^{-x}.}$$

(35)

Other expressions for entropy can be obtained by adopting additional differentiation operators.

In 2001, Akimoto and Suzuki [67] proposed the one-parameter fractional entropy, $S_{\alpha }^{\left(AS\right)}$, given by:

$${\displaystyle S_{\alpha }^{\left(AS\right)}=-\underset{x\to 1}{lim}\sum _i\frac{d^{\alpha }}{d{x}^{\alpha }}{e}^{xln{p}_i},}$$

(36)

where $\frac{d^{\alpha }}{d{x}^{\alpha }}{=}^{RL}{D}_{a^{+}}^{\alpha }$ is the Riemann-Liouville operator (17), with $a=0$.

The expressions (36) and (17) yield:

$${\displaystyle S_{\alpha }^{\left(AS\right)}=\sum _i\frac{\alpha -1}{\Gamma (2-\alpha )}{}_1{F}_1(1;1-\alpha ;ln{p}_i),0<\alpha <1,}$$

(37)

where ${}_1{F}_1(a;b;c)$ denotes the confluent hypergeometric function of the first kind [68]:

$${}_1{F}_1(a;b;x)=1+\frac{a}{b}\frac{x}{1!}+\frac{a(a+1)}{b(b+1)}\frac{x^2}{2!}+\frac{a(a+1)(a+2)}{b(b+1)(b+2)}\frac{x^3}{3!}+\cdots .$$

(38)

It can be shown that [67] has the concavity and non-extensivity properties. In the limit $\alpha \to 1$, it obeys positivity and gives the Shannon entropy, $S^{\left(S\right)}$.

In 2009, Ubriaco introduced a one-parameter fractional entropy, $S_{\alpha }^{\left(U\right)}$, given by [69]:

$${\displaystyle S_{\alpha }^{\left(U\right)}=\underset{x\to -1}{lim}\frac{d}{dx}\left({}^{RL}{D}_{-\infty }^{\alpha -1}\sum _i{e}^{-xln{p}_i}\right),}$$

(39)

where ${}^{RL}{D}_{-\infty }^{\alpha -1}$ is the Riemann-Liouville left-side derivative (17) with $a\to -\infty$.

Therefore, we obtain:

$${\displaystyle S_{\alpha }^{\left(U\right)}=\underset{x\to -1}{lim}\frac{d}{dx}\frac{1}{\Gamma (1-\alpha )}\sum _i{\int }_{-\infty }^x\frac{e^{-\tau ln{p}_i}}{{(x-\tau )}^{\alpha }}d\tau .}$$

(40)

Performing the integration and taking the limit $x\to -1$, it yields:

$$S_{\alpha }^{\left(U\right)}=\sum _i{p}_i{(-ln{p}_i)}^{\alpha },0\le \alpha \le 1.$$

(41)

The Ubriaco entropy (41) is thermodynamically stable and obeys the same properties of the Shannon entropy, with the exception of additivity. When $\alpha \to 1$, we recover the Shannon entropy.

In 2012, Yu et al. [70] formulated a one-parameter fractional entropy by means of the simple expression:

$$S_{\alpha }^{\left(Y\right)}={-}^{RL}{I}_0^{\alpha }(p_{i}ln{p}_i),\alpha \in {\mathbb{R}}^{+},$$

(42)

where the operator ${}^{RL}{I}_0^{\alpha }$ is the left-side Riemann-Liouville integral (8), with $a=0$. Expression (42) obeys the concavity property and is an extension and generalization of the Shannon entropy.

Another fractional entropy was derived in 2014 by Radhakrishnan et al. [71], being given by:

$${\displaystyle S_{q,\alpha }^{\left(RCJ\right)}=\sum _i{p}_i^q{(-ln{p}_i)}^{\alpha },q,\alpha >0.}$$

(43)

The two-parameter expression (43) was inspired in (41) and the entropy (44) derived by Wang in the context of the incomplete information theory [72]:

$$S_q^{\left(W\right)}={⟨-ln{p}_1⟩}_q=\sum _i{p}_i^q(-ln{p}_i),$$

(44)

where ${\sum }_i{p}_i^q=1$ and ${⟨\mathcal{O}⟩}_q={\sum }_i{p}_i^q{\mathcal{O}}_i$ denotes the q-expectation that characterizes incomplete normalization.

The entropy (43) is considered a fractional entropy in a fractal phase space in which the parameters q and $\alpha$ are associated with fractality and fractionality, respectively. In the limit, when (i) $q\to 1$ Equation (43) reduces to (41), (ii) $\alpha \to 1$ recovers $S_q^{\left(W\right)}$, and (iii) $\{q,\alpha \}=\{1,1\}$ expression (43) yields the standard Shannon formula (21).

In 2014, Machado followed a different line of thought [73], thinking of Shannon information $I\left(p_i\right)=-ln{p}_i$ as a function of order zero lying between the integer-order cases $D^{-1}I\left(p_i\right)=p_i\left(1-ln{p}_i\right)$ and $D^{1}I\left(p_i\right)=-\frac{1}{p_i}$. In the perspective of FC, this observation motivated the formulation of information and entropy of order $\alpha \in \mathbb{R}$ as [24]:

$$I_{\alpha }\left(p_i\right)=D^{\alpha }I\left(p_i\right)=-\frac{p_i^{-\alpha }}{\Gamma \left(\alpha +1\right)}\left(ln{p}_i+\tilde{\psi }\right),$$

(45)

$$S_{\alpha }^{\left(M\right)}={\displaystyle \sum _i\left[-\frac{p_i^{-\alpha }}{\Gamma \left(\alpha +1\right)}\left(ln{p}_i+\tilde{\psi }\right)\right]}{p}_i,$$

(46)

where $D^{\alpha }$ denotes a fractional derivative operator, $\tilde{\psi }=\psi \left(1\right)-\psi \left(1-\alpha \right)$ and $\psi \left(·\right)$ represent the digamma function.

The one-parameter fractional entropy (46) fails to obey some of the Khinchin axioms with exception of the case $q=0$ that leads to the Shannon entropy [74]. This behavior is in line with what occurs in FC, where fractional derivatives fail to obey some of the properties of integer-order operators [1].

Expression (46) was generalized by Jalab et al. [75] in the framework of local FC [76]. A adopting (20), the following expression was proposed:

$$S_{\alpha }^{\left(J\right)}={\displaystyle \sum _i\left[-\frac{p_i^{-i\alpha }}{\Gamma \left(i\alpha +1\right)}\left(ln{p}_i+\tilde{\psi }\right)\right]}{p}_i.$$

(47)

Equation (47) decreases from 1 to $1-\alpha$, $\alpha \in ]0,1[$. Therefore, we have:

$${\displaystyle S_{\alpha }^{\left(J\right)}\approx \sum _i\left[-\frac{p_i^{-i\alpha }}{\Gamma \left(i\alpha +1\right)}\left(ln{p}_i+\frac{1}{\alpha }\right)\right]p_i.}$$

(48)

In 2016, Karcı [77] proposed the fractional derivative (10), based on the concept of indefinite limit and the l’Hôpital’s rule. Adopting $f\left(x\right)={\sum }_i{p}_i^{-x}$, and using (10) into (34), he derived the following expression for fractional entropy [78]:

$$S_{\alpha }^{\left(K\right)}=\left|{}^K{D}^{\alpha }f\left(x\right)\right|=\left|{}^K{D}^{\alpha }\sum _i{p}_i^{-x}\right|=\sum _i{p}_i\left|{\left(\frac{p^{-x}}{x}\right)}^{\alpha -1}(-1)p^{-1}lnp\right|=\sum _i{p}_i\left|{\left(-p\right)}^{\alpha }lnp\right|.$$

(49)

In 2019, Ferreira and Machado [79] presented a new formula for the entropy based on the work of Abe [65] and Ubriaco [69]. They start by the definition of left-side Liouville fractional derivative of a function f with respect to another function g, with $g^{\prime }>0$, given by:

$${\displaystyle {}^L{D}_g^{\alpha }f\left(x\right)=\frac{1}{\Gamma (1-\alpha )g^{\prime }\left(x\right)}\frac{d}{dx}{\int }_{-\infty }^x{[g\left(x\right)-g\left(s\right)]}^{-\alpha }{g}^{\prime }\left(s\right)f\left(s\right)ds,0<\alpha \le 1.}$$

(50)

Choosing $f\left(x\right)=p_i^{-x}$ and $g\left(x\right)=e^{x+1}$ expression (50) leads to:

$$\begin{array}{cc}\hfill {}^L{D}_g^{\alpha }f\left(x\right)& {\displaystyle =\frac{1}{\Gamma (1-\alpha )e^{x+1}}\frac{d}{dx}{\int }_{-\infty }^x{[e^{x+1}-e^{s+1}]}^{-\alpha }{e}^{s+1}{e}^{-sln\left(p_i\right)}ds}\hfill \end{array}$$

(51)

$$\begin{array}{cc}\hfill & =[1-\alpha -ln\left(p_i\right)]e^{(-\alpha -1)(x+1)+x[1-ln\left(p_i\right)]+1}\frac{\Gamma (1-ln\left(p_i\right))}{\Gamma (2-\alpha -ln\left(p_i\right))}.\hfill \end{array}$$

(52)

Therefore, we have:

$${}^L{D}_g^{\alpha }f(-1)=[1-\alpha -ln\left(p_i\right)]p_i\frac{\Gamma (1-ln\left(p_i\right))}{\Gamma (2-\alpha -ln\left(p_i\right))},$$

(53)

which applying $\Gamma (x+1)=x\Gamma \left(x\right)$, for $x>0$, results in:

$${}^L{D}_g^{\alpha }f(-1)=p_i\frac{\Gamma (1-ln\left(p_i\right))}{\Gamma (1-\alpha -ln\left(p_i\right))}.$$

(54)

Using (54) into (34) gives:

$${\displaystyle S_{\alpha }^{\left(FM\right)}=\sum _i{p}_i\frac{\Gamma (1-ln\left(p_i\right))}{\Gamma (1-\alpha -ln\left(p_i\right))},0<\alpha \le 1.}$$

(55)

In 2019, Machado and Lopes [80] proposed two fractional formulations of the Rényi entropy, $S_{q,\alpha }^{\left(M{L}_1\right)}$ and $S_{q,\alpha }^{\left(M{L}_2\right)}$. Their derivation adopts a general averaging operator, instead of the linear one that is assumed for the Shannon entropy (21). Let us consider a monotonic function $f\left(x\right)$ with inverse $f^{-1}\left(x\right)$. Therefore, for a set of real values $\left\{x_i\right\}$, $i=1,2,\cdots$, with probabilities $\left\{p_i\right\}$, we can define a general mean [81] associated with $f\left(x\right)$ as:

$$f^{-1}\left({\displaystyle \sum _i{p}_{i}f\left(x_i\right)}\right).$$

(56)

Applying (56) to the Shannon entropy (21) we obtain:

$$S=f^{-1}\left(\sum _i{p}_{i}f\left(I\left(p_i\right)\right)\right),$$

(57)

where $f\left(x\right)$ is a Kolmogorov–Nagumo invertible function [82]. If the postulate of additivity for independent events is considered in (56), then only two functions $f\left(x\right)$ are possible, consisting of $f_1\left(x\right)=c·x$ and $f_2\left(x\right)=c·exp\left[(1-q)x\right]$, with $c,q\in \mathbb{R}$. For $f_1\left(x\right)$ we get the ordinary mean and we verify that $S=S^{\left(S\right)}$. For $f\left(x\right)=c·e^{(1-q)x}$ we have the expression:

$$S=\frac{1}{1-q}\sum _i{p}_i·exp\left[(1-q)I\left(p_i\right)\right],$$

(58)

which gives the Rényi entropy:

$$S_q^{\left(R\right)}=\frac{1}{1-q}ln\left(\sum _i{p}_i^q\right),q>0,q\ne 1.$$

(59)

If we combine (45) and (58), then we obtain:

$$\begin{array}{cc}\hfill S_{q,\alpha }^{\left(M{L}_1\right)}& =\frac{1}{1-q}ln\left\{\sum _i{p}_i·exp\left[(1-q)·I_{\alpha }\left(p_i\right)\right]\right\}\hfill \\ \hfill & =\frac{1}{1-q}ln\left\{\sum _i{p}_i·exp\left[(q-1)·\frac{p_i^{-\alpha }}{\Gamma \left(\alpha +1\right)}\left(ln{p}_i+\tilde{\psi }\right)\right]\right\}.\hfill \end{array}$$

(60)

On the other hand, if we rewrite (22) as:

$$\begin{array}{cc}\hfill S_q^{\left(R\right)}& =\frac{q}{1-q}ln\left[{\left(\frac{1}{N}\sum _i{p}_i^q\right)}^{\frac{1}{q}}·N^{\frac{1}{q}}\right]\hfill \\ \hfill & =\frac{q}{1-q}ln\left[{⟨p_i⟩}_g·N^{\frac{1}{q}}\right],\hfill \end{array}$$

(61)

where ${⟨p_i⟩}_g={\left(\frac{1}{N}{\sum }_i{p}_i^q\right)}^{\frac{1}{q}}$ is a generalized mean, then we obtain:

$$S_{q,\alpha }^{\left(M{L}_2\right)}=D^{\alpha }{H}_q^{\left(R\right)}=\frac{1}{N^{\frac{\alpha }{q}}}\frac{q}{1-q}\left[\frac{{⟨p_i⟩}_g^{-\alpha }}{\Gamma \left(\alpha +1\right)}\left(\frac{1}{q}lnN+ln{⟨p_i⟩}_g+\tilde{\psi }\right)\right].$$

(62)

In the limit, when $\alpha \to 0$, both $S_{q,\alpha }^{\left(M{L}_1\right)}$ and $S_{q,\alpha }^{\left(M{L}_2\right)}$ yield (22).

5. Comparison of the Fractional-Order Entropies

In this section we use the fractional entropy formulas to compute the entropy both of abstract and real-world data series.

5.1. Fractional-Order Entropy of Some Probability Distributions

We calculate the entropy of four well-known probability distributions, namely those of Poisson, Gaussian, Lévy and Weibull. We consider these cases just with the purpose of illustrating the behavior of the different formulations. Obviously other cases could be considered, but we limit the number for the sake of parsimony. Firstly, we present the results obtained with the one-parameter entropies $S_{\alpha }^{\left(AS\right)}$, $S_{\alpha }^{\left(U\right)}$, $S_{\alpha }^{\left(Y\right)}$, $S_{\alpha }^{\left(M\right)}$, $S_{\alpha }^{\left(J\right)}$, $S_{\alpha }^{\left(K\right)}$ and $S_{\alpha }^{\left(FM\right)}$. Then, we consider the two-parameter formulations $S_{q,\alpha }^{\left(RCJ\right)}$, $S_{q,\alpha }^{\left(M{L}_1\right)}$ and $S_{q,\alpha }^{\left(M{L}_2\right)}$. Table 1 summarizes the constants adopted for the distributions and the intervals of variation of the entropy parameters.

Table 1. The constants adopted for the probability distributions and the intervals of variation of the entropy parameters.

Distribution	Expression	Parameters	Domain	Order 1-par. Entropy	Order 2-par. Entropy
Poisson	$f\left(x\right)\frac{{\lambda }^z{e}^{-\lambda }}{z!}$	$\lambda =4$	$z=0,1,\dots ,50$	$\alpha \in [0,1]$	$\alpha \in [-0.6,0.6]$
Gaussian	$f\left(x\right)=\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{1}{2}{\left(\frac{x-\mu }{\sigma }\right)}^2}$	$\sigma =4$ $\mu =0$	$x\in [-2,2]$
Lévy	$f\left(x\right)=\sqrt{\frac{c}{2\pi }}\frac{e^{-\frac{c}{(2x-\mu )}}}{{(x-\mu )}^{(3/2)}}$	$c=4$ $\mu =0$	$x\in [0.1,20]$		$q\in [1.2,2.2]$
Weibull	$f\left(x\right)=\frac{k}{\lambda }{\left(\frac{x}{\lambda }\right)}^{(k-1)}{e}^{-{(x/\lambda )}^k}$	$k=1.5$ $\lambda =1$	$x\in [0.01,2.5]$

Figure 1 depicts the values of $S_{\alpha }^{\left(AS\right)}$, $S_{\alpha }^{\left(U\right)}$, $S_{\alpha }^{\left(Y\right)}$, $S_{\alpha }^{\left(M\right)}$, $S_{\alpha }^{\left(J\right)}$, $S_{\alpha }^{\left(K\right)}$ and $S_{\alpha }^{\left(FM\right)}$ versus $\alpha \in [0,1]$. We verify that in the limits, either $\alpha \to 0$ or $\alpha \to 1$, the values of the Shannon entropy are calculated as 2.087, 5.866, 4.953 and 5.309, respectively. Moreover, it can be seen that $S_{\alpha }^{\left(U\right)}$ and $S_{\alpha }^{\left(FM\right)}$ are very close to each other, $S_{\alpha }^{\left(AS\right)}$ does not obey positivity, $S_{\alpha }^{\left(J\right)}$ diverges at small values of $\alpha$, $S_{\alpha }^{\left(M\right)}$ has a maximum at values of $\alpha$ close to 0.6 and diverges as $\alpha \to 1$.

Figure 1. The values of $S_{\alpha }^{\left(AS\right)}$, $S_{\alpha }^{\left(U\right)}$, $S_{\alpha }^{\left(Y\right)}$, $S_{\alpha }^{\left(M\right)}$, $S_{\alpha }^{\left(J\right)}$, $S_{\alpha }^{\left(K\right)}$ and $S_{\alpha }^{\left(FM\right)}$ versus $\alpha \in [0,1]$ for the (a) Poisson, (b) Gaussian, (c) Lévy and (d) Weibull distributions.

Figure 2 portraits the values of $S_{q,\alpha }^{\left(RCJ\right)}$, $S_{q,\alpha }^{\left(M{L}_1\right)}$ and $S_{q,\alpha }^{\left(M{L}_2\right)}$ versus $\alpha \in [-0.6,0.6]$ and $q\in [1.2,2.2]$. We verify that in the domain considered the entropies vary slightly and do not diverge.

Figure 2. The values of $S_{q,\alpha }^{\left(RCJ\right)}$, $S_{q,\alpha }^{\left(M{L}_1\right)}$ and $S_{q,\alpha }^{\left(M{L}_2\right)}$ versus $\alpha \in [-0.6,0.6]$ and $q\in [1.2,2.2]$ for the (a–c) Poisson, (d–f) Gaussian, (g–i) Lévy and (j–l) Weibull distributions.

5.2. Fractional-Order Entropy of Real-World Data

We calculate the entropy of a real-world time series, namely the Dow Jones Industrial Average (DJIA) financial index. The DJIA raw data are available at the Yahoo Finance website (https://finance.yahoo.com/). Herein, we consider the stock closing values in the time period from 1 January 1987 up to 24 November 2018, with one-day sampling interval. Occasional missing values, as well as values corresponding to closing days, are estimated using linear interpolation. The processed DJIA time series, $x=\{x_1,x_2,\cdots ,x_T\}$, $T=12,381$ points, is used to construct a histogram of relative frequencies, $f\left(x\right)$, with $N=50$ bins equally spaced and non-overlapping, for estimating the probability distribution of x.

Figure 3a depicts the values of $S_{\alpha }^{\left(AS\right)}$, $S_{\alpha }^{\left(U\right)}$, $S_{\alpha }^{\left(Y\right)}$, $S_{\alpha }^{\left(M\right)}$, $S_{\alpha }^{\left(J\right)}$, $S_{\alpha }^{\left(K\right)}$ and $S_{\alpha }^{\left(FM\right)}$ versus $\alpha \in [0,1]$. We verify that, as shown in Section 5.1, $S_{\alpha }^{\left(U\right)}$ and $S_{\alpha }^{\left(FM\right)}$ yield similar results, $S_{\alpha }^{\left(J\right)}$ diverges for small values of $\alpha$, and $S_{\alpha }^{\left(M\right)}$ has a maximum at values of $\alpha$ close to 0.6, diverging when $\alpha \to 1$.

Figure 3. The entropy of the DJIA stock index for daily closing values in the time period from 1 January 1987 up to 24 November 2018, with one-day sampling interval: (a) $S_{\alpha }^{\left(AS\right)}$, $S_{\alpha }^{\left(U\right)}$, $S_{\alpha }^{\left(Y\right)}$, $S_{\alpha }^{\left(M\right)}$, $S_{\alpha }^{\left(J\right)}$, $S_{\alpha }^{\left(K\right)}$ and $S_{\alpha }^{\left(FM\right)}$ versus $\alpha \in [0,1]$; (b–d) $S_{q,\alpha }^{\left(RCJ\right)}$, $S_{q,\alpha }^{\left(M{L}_1\right)}$ and $S_{q,\alpha }^{\left(M{L}_2\right)}$ versus $\alpha \in [-0.6,0.6]$ and $q\in [1.2,2.2]$.

Figure 3b–d show the values of $S_{q,\alpha }^{\left(RCJ\right)}$, $S_{q,\alpha }^{\left(M{L}_1\right)}$ and $S_{q,\alpha }^{\left(M{L}_2\right)}$ versus $\alpha \in [-0.6,0.6]$ and $q\in [1.2,2.2]$, yielding results of the same type as before.

6. Impact and Applications of the Fractional-Order Entropies

To assess the impact of the fractional-order entropies on the scientific community, we consider the number of citations received by the nine papers that first proposed them. Table 1 summarizes the results obtained from the database Scopus on 7 November 2020 (www.scopus.com). We verify that those nine papers were cited 218 times by 170 distinct papers, and that the expressions proposed by Ubriaco and Machado received more attention.

To unravel the main areas of application of the fractional entropies, we use the VOSviewer (https://www.vosviewer.com/), which allows the construction and visualization of bibliometric networks [83]. The bibliometric data of the 170 papers that cite the nine papers that present the fractional-order entropies were collected from Scopus for constructing Table 2, and are the input information to the VOSviewer. The co-occurrence of the authors’ keywords in the 170 papers is analyzed, with the minimum value of co-occurrence of each keyword set to 3. Figure 4 depicts the generated map. We verify the emergence of six clusters, $\mathcal{C}=\{{\mathcal{C}}_1,\dots ,{\mathcal{C}}_6\}$. At the top, the light-blue cluster, ${\mathcal{C}}_1$, includes the fields of finance and finance time series analysis, while the light-green one, ${\mathcal{C}}_2$, encompasses a variety of areas, such as solvents, fractals, commerce, and stochastic systems, tightly connected to some entropy-based complexity measures. On the right, the dark-green cluster, ${\mathcal{C}}_3$, includes the areas of fault detection and image processing. At the bottom of the map, the red cluster, ${\mathcal{C}}_4$, highlights the fields of chromosome and DNA analysis, while the dark-blue, ${\mathcal{C}}_5$, one emphasizes some clustering and visualization techniques, as multidimensional scaling and hierarchical clustering. On the left, the magenta cluster, ${\mathcal{C}}_6$, includes keywords not related with applications.

Table 2. Citations received by the nine papers that proposed the fractional-order entropies, according to the database Scopus on 7 November 2020.

Entropy	Equation Number	Authors	Reference	N. Citations	Year
$S_{\alpha }^{\left(AS\right)}$	(37)	Akimoto and Suzuki	[67]	5	2001
$S_{\alpha }^{\left(U\right)}$	(41)	Ubriaco	[69]	88	2009
$S_{\alpha }^{\left(Y\right)}$	(42)	Yu et al.	[70]	7	2012
$S_{q,\alpha }^{\left(RCJ\right)}$	(43)	Radhakrishnan et al.	[71]	3	2014
$S_{\alpha }^{\left(M\right)}$	(46)	Machado	[73]	79	2014
$S_{\alpha }^{\left(J\right)}$	(47)	Jalab et al.	[75]	6	2019
$S_{\alpha }^{\left(K\right)}$	(49)	Karcı	[77]	16	2016
$S_{\alpha }^{\left(FM\right)}$	(55)	Ferreira and Machado	[79]	4	2019
$S_{q,\alpha }^{\left(M{L}_1\right)}$	(60)	Machado and Lopes	[80]	5	2019
$S_{q,\alpha }^{\left(M{L}_2\right)}$	(62)	Machado and Lopes	[80]	5	2019

Figure 4. The map of co-occurrence of the authors’ keywords in the 170 papers extracted from Scopus for constructing Table 2. The minimum value of co-occurrence of each keyword is 3. The clusters are represented by $\mathcal{C}=\{{\mathcal{C}}_1,\cdots ,{\mathcal{C}}_6\}$.

In summary, we conclude that the fractional entropies were applied to a considerable number of distinct scientific areas and that we may foresee a promising future for their development by exploring the synergies of the two mathematical tools. The prevalence of some proposals, from the point of view of citations, may be due to the time elapsed since their formulation. Indeed, more recent formulations had not yet sufficient time to disseminate in the community. Another reason may have to do with the type and audience of journal where they were published. Nonetheless, a full bibliometric analysis is not the leitmotif of the present paper.

7. Conclusions

This paper reviewed the concept of entropy in the framework of FC. To the best of the authors’ knowledge the fractional entropies proposed so far were included in this review. The different formulations result from the adopted (i) fractional-order operator or (ii) generating function. In general such entropies are non-extensive and converge to the classical Shannon entropy for certain values of their parameters. The fractional entropies have found applications in the area of complex systems, where the classical formulations revealed some limitations. The FC brings a shinny future in further developments of entropy and its applications.