A Review of Fractional Order Entropies

Fractional calculus (FC) is the area of calculus that generalizes the operations of differentiation and integration. FC operators are non-local and capture the history of dynamical effects present in many natural and artificial phenomena. Entropy is a measure of uncertainty, diversity and randomness often adopted for characterizing complex dynamical systems. Stemming from the synergies between the two areas, this paper reviews the concept of entropy in the framework of FC. Several new entropy definitions have been proposed in recent decades, expanding the scope of applicability of this seminal tool. However, FC is not yet well disseminated in the community of entropy. Therefore, new definitions based on FC can generalize both concepts in the theoretical and applied points of view. The time to come will prove to what extend the new formulations will be useful.

This paper reviews the concept of entropy in the framework of FC. In fact, FC is not yet well disseminated among the community of entropy and, therefore, new definitions based on FC may expand the scope of this powerful tool. To the authors' best knowledge, new entropy definitions are welcomed by the scientific community, somehow contrary to what happens with recent fractional operators. Consequently, the manuscript does not intend to assess the pros or the cons of the distinct formulations for some given problem. In a similar line of thought, the analysis of entropy-based indices proposed in the literature for comparing or characterizing some phenomena or probability distributions are outside the focus of this paper. Interested readers can obtain further information on divergence measures [41] and mutual information [42], as well as for sample [43], approximate [44], permutation [45], spectral [46], and fuzzy [47] entropies, among others [48]. Indeed, the main idea of this paper is to review the concept of fractional entropy and to present present day state of its development.

•
The left-side and the right-side Caputo derivatives, • The left-side and the right-side Grünwald-Letnikov derivatives, • The Hadamard derivative, • The left-side and right-side Hilfer derivatives of type 0 ≤ β ≤ 1, where RL I q a + and RL I q b − denote the left-side and right-side Riemann-Liouville fractional integrals of order q > 0, respectively, defined by: • The Karcı derivative • The Liouville, the left-side and the right-side Liouville derivatives, • The Marchaud, the left-side and the right-side Marchaud derivatives, • The left-side and the right-side Riemann-Liouville derivatives, • The Riesz derivative, The local Yang derivative, Often, the Caputo formulation is applied in physics and numerical integration, the Riemann-Liouville in calculus, and the Grünwald-Letnikov in engineering, signal processing and control. These classical definitions are the most frequently used by researchers. In what concerns the mathematical pros and cons of the Karcı and the Yang derivatives, readers may visit [23,51] and references therein. In fact, it should be noted that some formulations need some careful reflection and are the matter of some controversy, since many authors do not consider them as fractional operators [23,52,53]. Nevertheless, the debate about what it really means the term 'fractional derivative' is still ongoing among contemporary mathematicians [51].

The Concept of Entropy
Let us consider a discrete probability distribution P = {p 1 , p 2 , . . . , p N }, with ∑ i p i = 1 and p i ≥ 0. The Shannon entropy, S (S) , of distribution P is defined as: and represents the expected value of the information content given by I(p i ) = − ln p i . Therefore, for the uniform probability distribution we have p i = N −1 , N ∈ N, and the Shannon entropy takes its maximum value S = ln N, yielding the Boltzmann formula, up to a multiplicative factor, k, which denotes the Boltzmann constant. The Rényi and Tsallis entropies are one-parameter generalizations of (21) given by, respectively: The entropies S (R) q and S (T) q reduce to the Shannon formulation S (S) when q → 1. The Rényi entropy has an inverse power law equilibrium distribution [54], satisfying the zero-th law of thermodynamics [55]. It is important in statistics and ecology to quantify diversity, in quantum information to measure entanglement, and in computer science for randomness extraction. The Tsallis entropy was proposed in the scope of nonextensive statistical mechanics and has found application in the field of complex dynamics, in diffusion equations [56] and Fokker-Planck systems [57].
Other one-parameter entropies are the Landsberg-Vedral and Abe formulations [26,58]: Expression (24) is related to the Tsallis entropy by S , and is often known as normalized Tsallis entropy. Expression (25) is a symmetric modification of the Tsallis entropy, which is invariant to the exchange q ↔ q −1 , and we have S The two-parameter Sharma-Mittal entropy [32] is a generalization of the Shannon, Tsallis and Rényi entropies, and is defined as follows: The Sharma-Mittal entropy reduces to the Rényi, Tsallis and Shannon's formulations for the limits r → 1, r → q and {r, q} → {1, 1}, respectively.
Other entropies can be found in the literature [63,64], but a thorough review of all proposed formulations is out of the scope of this paper.

Fractional Generalizations of Entropy
It was noted [65] that the Shannon and Tsallis entropies have the same generating function ∑ i p x i and that the difference in the Formulas (21) and (23) is just due to the adopted differentiation operator. In fact, using the standard first-order differentiation, d dx , we obtain the Shannon entropy: while adopting the Jackson q-derivative [66], qx−x , 0 < q < 1, yields the Tsallis entropy [28]: Other expressions for entropy can be obtained by adopting additional differentiation operators.
In 2001, Akimoto and Suzuki [67] proposed the one-parameter fractional entropy, S (AS) α , given by: where d α dx α = RL D α a + is the Riemann-Liouville operator (17), with a = 0. The expressions (36) and (17) yield: where 1 F 1 (a; b; c) denotes the confluent hypergeometric function of the first kind [68]: It can be shown that [67] has the concavity and non-extensivity properties. In the limit α → 1, it obeys positivity and gives the Shannon entropy, S (S) .
In 2009, Ubriaco introduced a one-parameter fractional entropy, S (U) α , given by [69]: where RL D α−1 −∞ is the Riemann-Liouville left-side derivative (17) with a → −∞. Therefore, we obtain: Performing the integration and taking the limit x → −1, it yields: The Ubriaco entropy (41) is thermodynamically stable and obeys the same properties of the Shannon entropy, with the exception of additivity. When α → 1, we recover the Shannon entropy.
In 2012, Yu et al. [70] formulated a one-parameter fractional entropy by means of the simple expression: where the operator RL I α 0 is the left-side Riemann-Liouville integral (8), with a = 0. Expression (42) obeys the concavity property and is an extension and generalization of the Shannon entropy.
Another fractional entropy was derived in 2014 by Radhakrishnan et al. [71], being given by: The two-parameter expression (43) was inspired in (41) and the entropy (44) derived by Wang in the context of the incomplete information theory [72]: where ∑ i p q i = 1 and O q = ∑ i p q i O i denotes the q-expectation that characterizes incomplete normalization.
The entropy (43) is considered a fractional entropy in a fractal phase space in which the parameters q and α are associated with fractality and fractionality, respectively. In the limit, when (i) q → 1 Equation (43) In the perspective of FC, this observation motivated the formulation of information and entropy of order α ∈ R as [24]: where D α denotes a fractional derivative operator, ψ = ψ (1) − ψ (1 − α) and ψ (·) represent the digamma function. The one-parameter fractional entropy (46) fails to obey some of the Khinchin axioms with exception of the case q = 0 that leads to the Shannon entropy [74]. This behavior is in line with what occurs in FC, where fractional derivatives fail to obey some of the properties of integer-order operators [1].
Expression (46) was generalized by Jalab et al. [75] in the framework of local FC [76]. A adopting (20), the following expression was proposed: Equation (47) decreases from 1 to 1 − α, α ∈]0, 1[. Therefore, we have: In 2016, Karcı [77] proposed the fractional derivative (10), based on the concept of indefinite limit and the l'Hôpital's rule. Adopting f (x) = ∑ i p −x i , and using (10) into (34), he derived the following expression for fractional entropy [78]: In 2019, Ferreira and Machado [79] presented a new formula for the entropy based on the work of Abe [65] and Ubriaco [69]. They start by the definition of left-side Liouville fractional derivative of a function f with respect to another function g, with g > 0, given by: Choosing f (x) = p −x i and g(x) = e x+1 expression (50) leads to: Therefore, we have: which applying Γ(x + 1) = xΓ(x), for x > 0, results in: Using (54) into (34) gives: In 2019, Machado and Lopes [80] proposed two fractional formulations of the Rényi entropy, . Their derivation adopts a general averaging operator, instead of the linear one that is assumed for the Shannon entropy (21). Let us consider a monotonic function f (x) with inverse f −1 (x). Therefore, for a set of real values {x i }, i = 1, 2, . . . , with probabilities {p i }, we can define a general mean [81] associated with f (x) as: Applying (56) to the Shannon entropy (21) we obtain: where f (x) is a Kolmogorov-Nagumo invertible function [82]. If the postulate of additivity for independent events is considered in (56), then only two functions f (x) are possible, consisting of , with c, q ∈ R. For f 1 (x) we get the ordinary mean and we verify that S = S (S) . For f (x) = c · e (1−q)x we have the expression: which gives the Rényi entropy: If we combine (45) and (58), then we obtain: On the other hand, if we rewrite (22) as: where p i g = 1 N ∑ i p q i 1 q is a generalized mean, then we obtain: In the limit, when α → 0, both S (ML 1 ) q,α and S (ML 2 ) q,α yield (22).

Comparison of the Fractional-Order Entropies
In this section we use the fractional entropy formulas to compute the entropy both of abstract and real-world data series.

Fractional-Order Entropy of Some Probability Distributions
We calculate the entropy of four well-known probability distributions, namely those of Poisson, Gaussian, Lévy and Weibull. We consider these cases just with the purpose of illustrating the behavior of the different formulations. Obviously other cases could be considered, but we limit the number for the sake of parsimony. Firstly, we present the results obtained with the one-parameter entropies  Table 1 summarizes the constants adopted for the distributions and the intervals of variation of the entropy parameters. Table 1. The constants adopted for the probability distributions and the intervals of variation of the entropy parameters.

Fractional-Order Entropy of Real-World Data
We calculate the entropy of a real-world time series, namely the Dow Jones Industrial Average (DJIA) financial index. The DJIA raw data are available at the Yahoo Finance website (https://finance. yahoo.com/). Herein, we consider the stock closing values in the time period from 1 January 1987 up to 24 November 2018, with one-day sampling interval. Occasional missing values, as well as values corresponding to closing days, are estimated using linear interpolation. The processed DJIA time series, x = {x 1 , x 2 , . . . , x T }, T = 12, 381 points, is used to construct a histogram of relative frequencies, f (x), with N = 50 bins equally spaced and non-overlapping, for estimating the probability distribution of x.

Impact and Applications of the Fractional-Order Entropies
To assess the impact of the fractional-order entropies on the scientific community, we consider the number of citations received by the nine papers that first proposed them. Table 1 summarizes the results obtained from the database Scopus on 7 November 2020 (www.scopus.com). We verify that those nine papers were cited 218 times by 170 distinct papers, and that the expressions proposed by Ubriaco and Machado received more attention.
To unravel the main areas of application of the fractional entropies, we use the VOSviewer (https://www.vosviewer.com/), which allows the construction and visualization of bibliometric networks [83]. The bibliometric data of the 170 papers that cite the nine papers that present the fractional-order entropies were collected from Scopus for constructing Table 2, and are the input information to the VOSviewer. The co-occurrence of the authors' keywords in the 170 papers is analyzed, with the minimum value of co-occurrence of each keyword set to 3. Figure 4 depicts the generated map. We verify the emergence of six clusters, C = {C 1 , . . . , C 6 }. At the top, the light-blue cluster, C 1 , includes the fields of finance and finance time series analysis, while the light-green one, C 2 , encompasses a variety of areas, such as solvents, fractals, commerce, and stochastic systems, tightly connected to some entropy-based complexity measures. On the right, the dark-green cluster, C 3 , includes the areas of fault detection and image processing. At the bottom of the map, the red cluster, C 4 , highlights the fields of chromosome and DNA analysis, while the dark-blue, C 5 , one emphasizes some clustering and visualization techniques, as multidimensional scaling and hierarchical clustering. On the left, the magenta cluster, C 6 , includes keywords not related with applications.   Table 2. The minimum value of co-occurrence of each keyword is 3. The clusters are represented by C = {C 1 , . . . , C 6 }.
In summary, we conclude that the fractional entropies were applied to a considerable number of distinct scientific areas and that we may foresee a promising future for their development by exploring the synergies of the two mathematical tools. The prevalence of some proposals, from the point of view of citations, may be due to the time elapsed since their formulation. Indeed, more recent formulations had not yet sufficient time to disseminate in the community. Another reason may have to do with the type and audience of journal where they were published. Nonetheless, a full bibliometric analysis is not the leitmotif of the present paper.

Conclusions
This paper reviewed the concept of entropy in the framework of FC. To the best of the authors' knowledge the fractional entropies proposed so far were included in this review. The different formulations result from the adopted (i) fractional-order operator or (ii) generating function. In general such entropies are non-extensive and converge to the classical Shannon entropy for certain values of their parameters. The fractional entropies have found applications in the area of complex systems, where the classical formulations revealed some limitations. The FC brings a shinny future in further developments of entropy and its applications.

Conflicts of Interest:
The authors declare no conflict of interest.