Properties of Various Entropies of Gaussian Distribution and Comparison of Entropies of Fractional Processes

Abstract

We consider five types of entropies for Gaussian distribution: Shannon, Rényi, generalized Rényi, Tsallis and Sharma–Mittal entropy, establishing their interrelations and their properties as the functions of parameters. Then, we consider fractional Gaussian processes, namely fractional, subfractional, bifractional, multifractional and tempered fractional Brownian motions, and compare the entropies of one-dimensional distributions of these processes.

1. Introduction

The concept of entropy as a measure of the chaos of a dynamical system has been known for a long time, and this concept is used in numerous applications, starting with the physics of the universe and continuing with chemical reactions, hacking attacks and medical measurements.

The concept of entropy for a random variable was introduced by Shannon [1] to characterize the irreducible complexity inherent in a specific form of randomness. Nowadays, entropy measures have a wide range of potential applications across various fields [2], including information theory, machine learning, thermodynamics, information security, biology, finance, environmental sciences, social sciences, psychology and the study of complex systems. For instance, entropy is used in data compression, decision tree construction [3], statistical mechanics [4], cryptography [5], genetics [6], market analysis [7], climate analysis [8], social network analysis [9] and psychological studies [10]. Entropy measures help quantify information, predictability, complexity and other characteristics in these fields.

The notion of entropy is closely intertwined with the theory of quantum information, as developed in [11]. Recent advancements in this field can be explored in the work by Rahman et al. [12]. Entropy plays a pivotal role in practical applications, notably in signal processing and network traffic analysis. It is employed in the development of algorithms for detecting DDoS attacks [13]. Furthermore, entropy measurements are applied in medical and biological studies, where they facilitate the differentiation of pathologies and aging by quantifying physiological complexity. For instance, these concepts have been utilized to distinguish different Alzheimer’s disease states [14] and to classify signals from patients with Parkinson’s disease [15].

From a mathematical point of view, the entropy of a probability distribution is expressed in terms of its density, given the specified density, and this entropy is not difficult to calculate for specific distributions. Note, however, that there are many different approaches to determining the entropy of a probability distribution, starting with Shannon entropy, and then this concept was successively complicated and generalized by adding new parameters (Rényi, generalized Rényi, Tsallis, Sharma–Mittal entropies). The various definitions of entropy share several basic properties postulated by Alfred Rényi [16].

Rényi entropy [16] generalizes Shannon entropy by introducing an additional parameter $\alpha$ that allows for a range of entropy measures. Rényi entropy is used in quantum information theory and quantum statistical mechanics. It helps to describe the entanglement of quantum systems, the behavior of quantum phase transitions and the characterization of quantum states.

Generalized Rényi entropy extends the concept of Rényi entropy by allowing for more flexibility in the choice of the exponent. It is employed in various applications, such as describing the statistics of turbulent flows, analyzing the complexity of biological systems and studying the scaling properties of critical phenomena in condensed matter physics.

Tsallis entropy is another generalization of Shannon entropy, introduced by Constantino Tsallis [17,18]. It introduces a nonextensive parameter to describe systems that do not obey standard statistical mechanics. Tsallis entropy is relevant in the study of complex systems, self-organized criticality and in modeling systems with long-range interactions. It has been applied in various branches of physics, including astrophysics, plasma physics and high-energy particle physics.

Sharma–Mittal entropy [19,20] is a more recent entropy measure that generalizes both Shannon and Tsallis entropy. It introduces two parameters ($\alpha$ and $\beta$) to control the balance between order and disorder in a system. While it has not seen as much widespread adoption as Shannon or Tsallis entropy, it has potential applications in various areas of physics, including the study of complex systems and information theory.

In summary, these entropy measures provide different tools for quantifying the information content, complexity and uncertainty in a wide range of physical systems. Depending on the characteristics of the system being studied and the specific questions being asked, one of these entropy measures may be more appropriate and insightful than the others.

All of the indicated entropies can be successfully calculated (and they have already been calculated, for example, in [21]) in the case of a Gaussian distribution, which is the subject of this paper. However, in the presence of additional parameters of the entropy itself (not the distribution), the question immediately arises about the behavior of entropy as a function of the parameter. It is a well-known fact that the Rényi entropy as a function of the parameter decreases. However, its convexity is not a universal property and, in general, depends on the distribution ([22]). Therefore, if we concentrate on Gaussian distribution, we need to investigate the properties of the introduced entropies in as much detail as possible.

Section 2 of this paper is devoted to this issue. More precisely, we begin by revisiting the definitions of various entropies and the corresponding formulas for the entropies of a centered Gaussian distribution with variance ${\sigma }^2$. These entropies typically depend on one or two positive parameters, excluding the Shannon entropy. Our primary objective is to analyze the monotonicity and convexity properties exhibited by these entropy measures as functions of the aforementioned parameters. Additionally, we explore limiting cases in which the entropies may not be well-defined. This exploration allows us to extend the definitions of the entropies through continuity. Furthermore, we establish limiting relationships between various entropy concepts. To substantiate and complement our theoretical findings, we provide several graphical illustrations. It is worth noting that certain theoretical properties, particularly the convexity of the Tsallis entropy, are challenging to analyze analytically. In such cases, we employ numerical investigations, which offer insights into theoretical properties.

Since this paper is devoted to the entropies of the Gaussian distribution, the next logical step to consider Gaussian processes, which is what is carried out in Section 3. We restrict ourselves to fractional Gaussian processes, as these objects have numerous applications in technology, finance, economics, biology and other fields. As a rule, fractional processes contain an additional parameter, such as the Hurst index for fractional Brownian motion. Shannon entropy for stationary Gaussian processes, including fractional Gaussian noise, was considered in detail in [23]. The value of this entropy for a multidimensional Gaussian vector depends on the determinant of covariance matrix, and it is quite difficult to analyze this determinant in higher dimensions. For example, the behavior of the entropy of the vector created from fractional Gaussian noise as the function of the Hurst index was investigated in [24], where the hypothesis that the Shannon entropy increases when the Hurst index H increases from 0 to $1/2$ and decreases when when the Hurst index H increases from $1/2$ to 1 was substantiated numerically; however, analytic confirmation of this hypothesis for higher dimensions is still in progress. Taking this into account, in this paper, we decided to limit ourselves to one-dimensional distributions of fractional Gaussian processes, instead of expanding the class of processes under consideration.

Namely, we compare the entropies of the one-dimensional distributions of the following fractional processes: fractional Brownian motion, subfractional Brownian motion, Riemann–Liouville fractional Brownian motion, bifractional Brownian motion and three types of multifractional Brownian motion (moving-average, Volterra-type and harmonizable), as well as tempered fractional Brownian motions of the first and second kind. We consider normalized versions of these processes to ensure that their variances at $t=1$ are equal to 1. After this normalization, we observe that fractional Brownian motion, subfractional Brownian motion and Riemann–Liouville fractional Brownian motion share the same entropies. Similar formulas apply to bifractional Brownian motion; furthermore, its entropies can be compared to those of fractional Brownian motion depending on the values of t.

For multifractional Brownian motion, we have established that the moving-average and harmonizable versions of this process have the same entropies. These entropies can be compared with the corresponding entropies of Volterra-type multifractional Brownian motion, depending on the behavior of the Hurst function. Lastly, for two versions of tempered fractional Brownian motions, we can numerically compare their entropies depending on the ratio between the multiplicative constants involved in their definitions.

Our reason and goal of this comparison was to consider fractional processes from the point of view of quantity of information contained in their one-dimensional distributions, because these processes previously were mostly compared from the point of view of the behavior of their trajectories that is interesting in financial applications, but entropy properties are more interesting in physical applications, for example, in the calculation of the fractal dimension of a solid sample. However, there is also an application to financial models. Namely, the Hurst index of fractional processes affects the behavior of their trajectories; its decrease leads to their irregularity and vice versa. But from the point of view of entropies, the situation showed a dependence on time: near zero, more precisely from zero to one, the variance, and therefore entropy, increases when the Hurst index decreases, but when time passes through unity, the situation changes to the opposite. This means that so-called rough volatility, which corresponds to the instability of the model, plays a crucial role only on short time intervals.

This paper is organized as follows. In Section 2, we investigate the properties of the various entropies for the centered Gaussian distribution with respect to the parameters, mainly paying attention to monotonicity and convexity. Section 3 is devoted to the entropies of fractional Gaussian processes. Fractional, subfractional and bifractional Brownian motions are studied in Section 3.1, three types of multifractional processes are considered in Section 3.2 and tempered fractional Brownian motions of the first and the second kind are compared in Section 3.3. We supplement our paper with three appendices. Appendix A contains derivations of formulas for the entropies of Gaussian distributions. Appendix B includes an auxiliary lemma necessary for studying the convexity of the entropies in Section 2, while Appendix C provides definitions and properties of special functions involved in the covariance functions of tempered fractional Brownian motions.

2. Shannon, Rényi, Generalized Rényi, Tsallis and Sharma–Mittal Entropies for Normal Distribution: Properties of Entropies as Functions of Their Parameters

Since all types of entropy are considered in detail for the normal distribution, Definition 1 of all entropies considered below is provided for the distribution with density. So, let $f\left(x\right),x\in \mathbb{R}$ be a density of a probability distribution.

Definition 1. 

1. 

The Shannon entropy is given by

$${\mathbf{H}}_S=-{\int }_{\mathbb{R}}f\left(x\right)logf\left(x\right)dx.$$

2. 

The Rényi entropy with index $\alpha >0$ is given by

$${\mathbf{H}}_R\left(\alpha \right)=\frac{1}{1-\alpha }log{\int }_{\mathbb{R}}{f}^{\alpha }\left(x\right)dx.$$

3. 

The generalized Rényi entropy in the case $\alpha \ne \beta ,\alpha ,\beta >0$ is given by

$${\mathbf{H}}_{GR}(\alpha ,\beta )=\frac{1}{\beta -\alpha }log\frac{{\int }_{\mathbb{R}}{f}^{\alpha }\left(x\right)dx}{{\int }_{\mathbb{R}}{f}^{\beta }\left(x\right)dx}.$$

The generalized Rényi entropy (in the case $\alpha =\beta >0$) is given by

$${\mathbf{H}}_{GR}\left(\alpha \right)=-\frac{{\int }_{\mathbb{R}}{f}^{\alpha }\left(x\right)logf\left(x\right)dx}{{\int }_{\mathbb{R}}{f}^{\alpha }\left(x\right)dx}.$$

4. 

The Tsallis entropy with index $\alpha >0$, $\alpha \ne 1$ is given by

$${\mathbf{H}}_T\left(\alpha \right)=\frac{1}{1-\alpha }\left({\int }_{\mathbb{R}}{f}^{\alpha }\left(x\right)dx-1\right).$$

5. 

The Sharma–Mittal entropy with positive indices $\alpha \ne 1$ and $\beta \ne 1$ is defined as

$${\mathbf{H}}_{SM}(\alpha ,\beta )=\frac{1}{1-\beta }\left({\left({\int }_{\mathbb{R}}{f}^{\alpha }\left(x\right)dx\right)}^{\frac{1-\beta }{1-\alpha }}-1\right).$$

Now, let us consider the density function of normal distribution with zero mean and variance ${\sigma }^2$:

$$f\left(x\right)=\frac{1}{\sigma \sqrt{2\pi }}exp\left(-\frac{x^2}{2{\sigma }^2}\right).$$

The next proposition summarizes the formulas for various entropies for this probability density. These formulas are well known (see, e.g., [21]) and can be obtained by straightforward calculations. But for the reader’s convenience, we present their proofs in the Appendix A.

Proposition 1.

The following facts hold for the centered normal distribution with variance ${\sigma }^2$.

(1) 

The Shannon entropy equals

$${\mathbf{H}}_S=\frac{1}{2}(1+log2\pi )+log\sigma .$$

(1)

(2) 

The Rényi entropy ($\alpha >0$, $\alpha \ne 1$) equals

$${\mathbf{H}}_R\left(\alpha \right)=log\sigma +\frac{1}{2}log\left(2\pi \right)+\frac{log\alpha }{2(\alpha -1)}.$$

(2)

(3) 

The generalized Rényi entropy in the case $\alpha =\beta$ equals

$${\mathbf{H}}_{GR}\left(\alpha \right)=log\delta +\frac{1}{2\alpha }=log\sigma +\frac{1}{2}log\left(2\pi \right)+\frac{1}{2\alpha },$$

(3)

where $\delta :=\sigma \sqrt{2\pi }$.

(4) 

The generalized Rényi entropy in the case $\alpha \ne \beta$ equals

$${\mathbf{H}}_{GR}(\alpha ,\beta )=log\sigma +\frac{1}{2}log\left(2\pi \right)+\frac{log\beta -log\alpha }{2(\beta -\alpha )}.$$

(4)

(5) 

The Tsallis entropy ($\alpha >0$, $\alpha \ne 1$) equals

$${\mathbf{H}}_T\left(\alpha \right)=\frac{{\delta }^{1-\alpha }{\alpha }^{-1/2}-1}{1-\alpha }=\frac{{\theta }^{\alpha -1}{\alpha }^{-1/2}-1}{1-\alpha }=\frac{{\sigma }^{1-\alpha }{\left(2\pi \right)}^{\frac{1-\alpha }{2}}{\alpha }^{-1/2}-1}{1-\alpha },$$

(5)

where $\theta ={\delta }^{-1}={\left(\sigma \sqrt{2\pi }\right)}^{-1}$.

(6) 

The Sharma–Mittal entropy for $\alpha ,\beta \in (0,1)\cup (1,\infty )$ equals

$${\mathbf{H}}_{SM}(\alpha ,\beta )=\frac{1}{1-\beta }\left(\frac{{\left(\sqrt{2\pi }\sigma \right)}^{1-\beta }}{{\alpha }^{\frac{1-\beta }{2(1-\alpha )}}}-1\right)=\frac{1}{1-\beta }\left(\frac{{\sigma }^{1-\beta }{\left(2\pi \right)}^{\frac{1-\beta }{2}}}{{\alpha }^{\frac{1-\beta }{2(1-\alpha )}}}-1\right).$$

(6)

Now, let us compare the values of ${\mathbf{H}}_R\left(\alpha \right)$ and ${\mathbf{H}}_{GR}\left(\alpha \right)$, and along the way, we will prove one simple useful inequality, which we use in other proofs.

Lemma 1.

For any $\sigma >0$, ${\mathbf{H}}_R\left(\alpha \right)<{\mathbf{H}}_{GR}\left(\alpha \right)$ for $\alpha \in (0,1)$, and ${\mathbf{H}}_R\left(\alpha \right)>{\mathbf{H}}_{GR}\left(\alpha \right)$ for $\alpha >1$.

Proof. 

It follows from (2) and (3) that

$${\mathbf{H}}_R\left(\alpha \right)-{\mathbf{H}}_{GR}\left(\alpha \right)=\frac{log\alpha }{2(\alpha -1)}-\frac{1}{2\alpha }=\frac{log\alpha -1+1/\alpha }{2(\alpha -1)}.$$

Therefore, it suffices to prove that the numerator $f\left(\alpha \right):=log\alpha +1/\alpha -1$ is positive for any $\alpha \in (0,1)\cup (1,\infty )$. Obviously, $f\left(\alpha \right)=0$ if $\alpha =1$. Moreover, $f^{\prime }\left(\alpha \right)=-1/{\alpha }^2+1/\alpha =\frac{\alpha -1}{{\alpha }^2}<(>)0$ for $\alpha <(>)1$. This means that for any $\alpha >0$, $\alpha \ne 1$, it holds that $f\left(\alpha \right)>0$.  □

Now, we consider, step by step, the properties of the entropies introduced in Definition 1 as the functions of parameters. All entropies in this paper are considered for the centered normal distribution, but we shall recall this from time to time. Theorems 1–4 are devoted to the properties of Rényi, generalized Rényi, Tsallis and Sharma–Mittal entropy, respectively, as the functions of entropy parameters, $\alpha$ and $\beta$, if the latter parameter is present. All derivatives that are considered in the proofs of these theorems are taken in $\alpha$; therefore, we omit it in the notations of derivatives. Let us start with the properties of the Rényi entropy as the function of $\alpha$.

Remark 1.

It follows immediately from the equalities (1)–(6) that all entropies strictly increase in variance of the respective normal distribution.

Theorem 1.

The following facts hold for the centered normal distribution with variance ${\sigma }^2$ and corresponding Rényi entropy:

(1) 

As $\alpha \to 1$, the Rényi entropy converges to the Shannon entropy, and at the point $\alpha =1$, the Rényi entropy can be extended by the Shannon entropy to be continuous.

(2) 

The Rényi entropy is a decreasing and convex function of α.

Remark 2.

The continuity of the Rényi entropy at point $\alpha =1$, and the fact that it decreases in α, is common knowledge, and we provide it here in order to demonstrate how these properties are realized for the normal distribution. The convexity property is not true for all distributions. This fact was established, e.g., in [22].

Proof. 

(1) According to L’Hôpital’s rule, $\frac{log\alpha }{\alpha -1}\to 1$ and $\frac{1}{2}log2\pi +log\sigma -\frac{log\alpha }{2(1-\alpha )}\to \frac{1}{2}(1+log2\pi )+log\sigma$ as $\alpha \to 1$. Therefore, the Rényi entropy converges to the Shannon entropy as $\alpha \to 1$, and at the point $\alpha =1$, the Rényi entropy can be extended by the Shannon entropy to be continuous.

(2) Let us calculate the derivative of the Rényi entropy in $\alpha$:

$$2{\mathbf{H}}_R^{\prime }\left(\alpha \right)={\left(\frac{log\alpha }{\alpha -1}\right)}^{\prime }=\frac{(\alpha -1)/\alpha -log\alpha }{{(\alpha -1)}^2}=\frac{1-1/\alpha -log\alpha }{{(\alpha -1)}^2}.$$

It was established in the proof of Lemma 1 that $1/\alpha -1+log\alpha >0$ for all $\alpha >0$. $\alpha \ne 1$; therefore, ${\left(\frac{log\alpha }{\alpha -1}\right)}^{{}^{\prime }}<0$ for such $\alpha$, and the Rényi entropy is a strictly decreasing function in $\alpha$. Note that

$$\frac{1}{2}log2\pi +log\sigma -\frac{log\alpha }{2(1-\alpha )}\to \infty ,\alpha \to 0,$$

and

$$\frac{1}{2}log2\pi +log\sigma -\frac{log\alpha }{2(\alpha -1)}\to \frac{1}{2}log2\pi +log\sigma ,\alpha \to \infty .$$

Therefore, the Rényi entropy decreases from ∞ to $\frac{1}{2}log2\pi +log\sigma .$ Furthermore,

$$\begin{array}{cc}\hfill 2{\mathbf{H}}_R^{″}\left(\alpha \right)& ={\left(\frac{log\alpha }{\alpha -1}\right)}^{″}={\left(\frac{1-1/\alpha -log\alpha }{{(\alpha -1)}^2}\right)}^{\prime }\hfill \\ \hfill & =\frac{(1/{\alpha }^2-1/\alpha ){(\alpha -1)}^2+2(\alpha -1)(1/\alpha +log\alpha -1)}{{(\alpha -1)}^4}\hfill \\ \hfill & =\frac{(1/{\alpha }^2-1/\alpha )(\alpha -1)+2(1/\alpha +log\alpha -1)}{{(\alpha -1)}^3}\hfill \\ \hfill & =\frac{-{(\alpha -1)}^2+2(\alpha +{\alpha }^{2}log\alpha -{\alpha }^2)}{{\alpha }^2{(\alpha -1)}^3}\hfill \\ \hfill & =\frac{2{\alpha }^{2}log\alpha -3{\alpha }^2+4\alpha -1}{{\alpha }^2{(\alpha -1)}^3}.\hfill \end{array}$$

Consider the numerator. Its derivative equals

$${(2{\alpha }^{2}log\alpha -3{\alpha }^2+4\alpha -1)}^{\prime }=4\alpha log\alpha +2\alpha -6\alpha +4$$

$$=4\alpha (1/\alpha -1+log\alpha )>0,$$

for $\alpha >0$, $\alpha \ne 1$, because $1/\alpha -1+log\alpha >0$ for such $\alpha$, which was established in the proof of Lemma 1. We obtain that $2{\alpha }^{2}log\alpha -3{\alpha }^2+4\alpha -1$ is a strictly increasing function on $(0,1)$ and $(1,+\infty )$, which is zero if $\alpha =1$. This means that $2{\alpha }^{2}log\alpha -3{\alpha }^2+4\alpha -1<0$ for $0<\alpha <1$, and $2{\alpha }^{2}log\alpha -3{\alpha }^2+4\alpha -1>0$ for $\alpha >1$. Therefore,

$$\frac{2{\alpha }^{2}log\alpha -3{\alpha }^2+4\alpha -1}{{\alpha }^2{(\alpha -1)}^3}>0,$$

except one point $\alpha =1$ when it equals zero. Consequently, ${\mathbf{H}}_R^{″}\left(\alpha \right)>0$, except one point, $\alpha =1$, when it equals zero, and so the Rényi entropy is a convex function.  □

Now, we proceed with the properties of the generalized Rényi entropy for the normal distribution as the function of $\alpha$ and $\beta$.

Theorem 2.

Consider the centered normal distribution with variance ${\sigma }^2$ and corresponding generalized Rényi entropy.

(1) 

In the case $\alpha =\beta$, the generalized Rényi entropy is a decreasing and convex function of α.

(2) 

In the case $\alpha \ne \beta$, the generalized Rényi entropy ${\mathbf{H}}_{GR}(\alpha ,\beta )$ converges to the generalized Rényi entropy ${\mathbf{H}}_{GR}\left(\alpha \right)$ as $\beta \to \alpha$, and so at the point $\alpha =\beta$$,{\mathbf{H}}_{GR}(\alpha ,\beta )$, considered as the function of β for fixed α, can be extended by ${\mathbf{H}}_{GR}\left(\alpha \right)$ to be continuous.

(3) 

The generalized Rényi entropy, ${\mathbf{H}}_{GR}(\alpha ,\beta )$, considered as the function of β for fixed α, is a decreasing and convex function. The behavior in α with β fixed is symmetric.

Proof. 

(1) By (3), ${\mathbf{H}}_{GR}\left(\alpha \right)=log\delta +\frac{1}{2\alpha }$. This function decreases in $\alpha \in (0,+\infty )$ from $+\infty$ to $log\delta$ and is convex. Note that at point $\alpha =1$, it coincides with Shannon entropy.

(2) Obviously,

$$\begin{array}{c}\hfill \underset{\beta \to \alpha }{lim}{\mathbf{H}}_{GR}(\alpha ,\beta )=log\delta +\underset{\beta \to \alpha }{lim}\frac{log\beta -log\alpha }{2(\beta -\alpha )}=log\delta +\frac{1}{2\alpha }.\end{array}$$

So, ${\mathbf{H}}_{GR}(\alpha ,\beta )\to {\mathbf{H}}_{GR}\left(\alpha \right)$ as $\beta \to \alpha$, and at the point $\alpha =\beta$, ${\mathbf{H}}_{GR}(\alpha ,\beta )$ can be extended by ${\mathbf{H}}_{GR}\left(\alpha \right)$ to be continuous.

(3) Since $logx$ is a concave function, its slope function is decreasing; therefore, function

$${\mathbf{H}}_{GR}(\alpha ,\beta )=log\delta +\frac{log\beta -log\alpha }{2(\beta -\alpha )},$$

considered as the function of $\beta$ for fixed $\alpha$, is a decreasing function. To prove its convexity, we apply Lemma A1, taking

$$\psi \left(x\right)=\frac{g\left(x\right)-g\left(x_0\right)}{x-x_0},\mathrm{with}g\left(x\right)=logx,x=\beta ,x_0=\alpha .$$

Since

$$g^{‴}\left(\xi \right)={(logx)}^{‴}={\left(\frac{1}{x}\right)}^{″}={\left(-\frac{1}{x^2}\right)}^{\prime }=2x^{-3}>0,$$

the function

$${\mathbf{H}}_{GR}(\alpha ,\beta )=log\delta +\frac{log\beta -log\alpha }{2(\beta -\alpha )},$$

considered as the function of $\beta$ for fixed $\alpha$, is a convex function. The situation with $\beta$ fixed is symmetric.  □

Now, we proceed with the properties of the Tsallis entropy as the function of $\alpha$.

Theorem 3.

Consider, as before, the centered normal distribution with variance ${\sigma }^2.$

(1) 

As $\alpha \to 1$, the Tsallis entropy converges to the Shannon entropy, and at the point $\alpha =1$, the Tsallis entropy can be extended by the Shannon entropy to obtain a continuous function.

(2) 

The Tsallis entropy ${\mathbf{H}}_T\left(\alpha \right)$ decreases from $+\infty$ to $-\infty$ when α increases from 0 to $+\infty$.

(3) 

Let, as in Proposition 1, $\theta ={\delta }^{-1}={\left(\sigma \sqrt{2\pi }\right)}^{-1}$, and let $x_0$ be the unique root of the equation $x^3-\frac{3}{2}{x}^2+\frac{9}{4}x-\frac{15}{8}=0.$

(a) 

Let $\theta <1$. Then, ${\mathbf{H}}_T$ is a convex function on the whole interval $(0,+\infty )$.

(b) 

Let $1<\theta <e^{x_0}$. Then, ${\mathbf{H}}_T$ is a convex function on the interval $(0,\frac{x_0}{log\theta }).$

(c) 

Let $\theta >e^{x_0}$. Then, ${\mathbf{H}}_T$ is a concave function on the interval $(\frac{x_0}{log\theta },+\infty ).$

(d) 

For any $\theta >1$ (consequently, for any $\sigma <{\left(2\pi \right)}^{-1/2}$), there exist numbers $0<\alpha (1,\theta )<\alpha (2,\theta )<\infty$ such that ${\mathbf{H}}_T$ is a convex function on the interval $(0,\alpha (1,\theta \left)\right)$, and it is a concave function on the interval $\left(\alpha \right(2,\theta ),\infty )$.

Remark 3.

The property of decreasing is common for the Tsallis entropy if the conditions supplying the equality

$${\left({\int }_{\mathbb{R}}{f}^{\alpha }\left(x\right)dx\right)}^{″}={\int }_R{f}^{\alpha }\left(x\right){log}^{2}f\left(x\right)dx$$

and the finite values of the last integral for any $\alpha \in (0,\infty )$ are satisfied.

Proof. 

(1) Consider the numerator in the right-hand side of (A2). Its derivative equals

$${\left({\delta }^{1-\alpha }{\alpha }^{-1/2}-1\right)}^{\prime }=-\frac{2\delta \alpha log\delta +\delta }{2{\alpha }^{3/2}{\delta }^{\alpha }},$$

and

$$\underset{\alpha \to 1}{lim}\left(-\frac{2\delta \alpha log\delta +\delta }{2{\alpha }^{3/2}{\delta }^{\alpha }}\right)=-log\delta -1/2.$$

According to L’Hôpital’s rule,

$$\underset{\alpha \to 1}{lim}{\mathbf{H}}_T\left(\alpha \right)=\underset{\alpha \to 1}{lim}{\left({\delta }^{1-\alpha }{\alpha }^{-1/2}-1\right)}^{\prime }/{(1-\alpha )}^{\prime }=\frac{1}{2}(1+log2\pi )+log\sigma .$$

This means that the Tsallis entropy converges to the Shannon entropy when $\alpha \to 1$, and at the point $\alpha =1$, the Tsallis entropy can be extended by the Shannon entropy to be continuous.

(2) Now, we investigate the monotonicity of the value

$$-{\mathbf{H}}_T\left(\alpha \right)=\frac{{\theta }^{\alpha -1}{\alpha }^{-1/2}-1}{\alpha -1},\alpha >0.$$

First, let us calculate two derivatives of the function $g\left(\alpha \right)={\theta }^{\alpha -1}{\alpha }^{-1/2},\alpha >0.$ Obviously,

$$g^{\prime }\left(\alpha \right)={\theta }^{\alpha -1}(log\theta ·{\alpha }^{-1/2}-\frac{1}{2}{\alpha }^{-3/2}),$$

$$g^{″}\left(\alpha \right)={\theta }^{\alpha -1}[{log}^2\theta ·{\alpha }^{-1/2}-log\theta ·{\alpha }^{-3/2}+\frac{3}{4}{\alpha }^{-5/2}],$$

$$={\theta }^{\alpha -1}{\alpha }^{-1/2}[{log}^2\theta -{\alpha }^{-1}log\theta +\frac{3}{4}{\alpha }^{-2}].$$

It is easy to see that the quadratic function $x^2-\beta x+\frac{3}{4}{\beta }^2>0,$ where $x=log\theta$ and $\beta ={\alpha }^{-1}.$ This means that $g\left(\alpha \right)$ is convex, whence its slope function $\frac{g\left(\alpha \right)-1}{\alpha -1}$ increases when $\alpha$ increases from 0 to $+\infty$. In turn, this means that ${\mathbf{H}}_T\left(\alpha \right)$ decreases from $+\infty$ to $-\infty$ when $\alpha$ increases from 0 to $+\infty$.

(3) In order to establish the convexity of ${\mathbf{H}}_T\left(\alpha \right)$, denote, as before, $g\left(\alpha \right)={\theta }^{\alpha -1}{\alpha }^{-1/2}$ and recall that $g^{″}\left(\alpha \right)>0,\alpha >0.$ Also,

$$-{\mathbf{H}}_T\left(\alpha \right)=\frac{g\left(\alpha \right)-1}{\alpha -1}.$$

Then,

$$-{\mathbf{H}}_T^{\prime }\left(\alpha \right)=\frac{g^{\prime }\left(\alpha \right)(\alpha -1)-g\left(\alpha \right)+1}{{(\alpha -1)}^2},$$

and

$$\begin{array}{c}-{\mathbf{H}}_T^{″}\left(\alpha \right)=[g^{″}\left(\alpha \right)(\alpha -1){(\alpha -1)}^2-2(\alpha -1)[g^{\prime }\left(\alpha \right)(\alpha -1)-g\left(\alpha \right)+1]{(\alpha -1)}^{-4}\hfill \\ \hfill =2\left[\frac{1}{2}{g}^{″}\left(\alpha \right){(\alpha -1)}^2-g^{\prime }\left(\alpha \right)(\alpha -1)+g\left(\alpha \right)-1\right]{(\alpha -1)}^{-3}.\end{array}$$

(7)

According to the Taylor formula,

$$g\left(1\right)=1=g\left(\alpha \right)+g^{\prime }\left(\alpha \right)(1-\alpha )+\frac{1}{2}{g}^{″}\left(\alpha \right){(\alpha -1)}^2+\frac{1}{6}{g}^{‴}\left(\xi \right){(1-\alpha )}^3,\xi \in (1\wedge \alpha ,1\vee \alpha ).$$

Therefore,

$$-{\mathbf{H}}_T^{″}\left(\alpha \right)=-2·\frac{1}{6}{g}^{‴}\left(\xi \right){(1-\alpha )}^3{(\alpha -1)}^{-3}=\frac{1}{3}{g}^{‴}\left(\xi \right),\mathrm{and}$$

$${\mathbf{H}}_T^{″}\left(\alpha \right)=-\frac{1}{3}{g}^{‴}\left(\xi \right),\xi \in (1\wedge \alpha ,1\vee \alpha ).$$

It is easy to calculate the 3rd derivative of function g:

$$g^{‴}\left(\alpha \right)={\theta }^{(\alpha -1)}{\alpha }^{-7/2}\left(x^3-\frac{3}{2}{x}^2+\frac{9}{4}x-\frac{15}{8}\right),$$

where $x=\alpha log\theta ,\theta ={\left(\sigma \sqrt{2\pi }\right)}^{-1}.$ Function

$$h\left(x\right)=x^3-\frac{3}{2}{x}^2+\frac{9}{4}x-\frac{15}{8}$$

is increasing on $\mathbb{R}$ with the unique root $x_0\approx 1.05357.$ Consider several cases. In some of them, we can produce analytical inference about the sign of ${\mathbf{H}}_T^{″}\left(\alpha \right)$; in other cases, numerics are necessary.

(a)

Let $\theta <1$. Then, $log\theta <0$, and for all $\xi >0,x=\xi log\theta <0$; consequently, $x<x_0,g^{‴}\left(\xi \right)<0$, and ${\mathbf{H}}_T^{{}^{″}}\left(\alpha \right)>0$ for all $\alpha >0.$ This means that in the case $\theta <1$, ${\mathbf{H}}_T$ is a convex function on the whole interval $(0,+\infty )$.

(b)

Let $1<\theta <e^{x_0},\alpha <1.$ Then, $\xi \in (\alpha ,1)$ and $\xi log\theta <x_0.$ Consequently, $g^{‴}\left(\xi \right)<0$ and ${\mathbf{H}}_T^{{}^{″}}\left(\alpha \right)>0$. Similarly, let $1<\theta <e^{x_0},\alpha \in (1,\frac{x_0}{log\theta })$. Then, $\xi \in (1,\alpha )\subset (1,\frac{x_0}{log\theta }),$ and $\xi log\theta <x_0.$ Consequently, $g^{‴}\left(\xi \right)<0$ and ${\mathbf{H}}_T^{{}^{″}}\left(\alpha \right)>0$. This means that in the case $1<\theta <e^{x_0}$, ${\mathbf{H}}_T$ is a convex function on the interval $(0,\frac{x_0}{log\theta }).$

(c)

Let $\theta >e^{x_0},\alpha >1.$ Then, $\xi \in (1,\alpha )$; therefore, $\xi log\theta >x_0$, and consequently, $g^{‴}\left(\xi \right)>0$, whence ${\mathbf{H}}_T^{{}^{″}}\left(\alpha \right)<0$. Let $\theta >e^{x_0},\alpha \in \left(\frac{x_0}{log\theta },1\right).$ Then, $\alpha log\theta >x_0$ and $g^{‴}\left(\xi \right)>0,$ whence ${\mathbf{H}}_T^{{}^{″}}\left(\alpha \right)<0$. Therefore, in the case $\theta >e^{x_0}$, ${\mathbf{H}}_T$ is a concave function on the interval $(\frac{x_0}{log\theta },+\infty ).$

(d)

Analyzing the asymptotics of $g\left(\alpha \right)$, $g^{″}\left(\alpha \right)$ and $g^{″}\left(\alpha \right)$ at 0 and at $+\infty$, respectively, we obtain that for $\theta >0$

$${\mathbf{H}}_T^{″}\left(\alpha \right)\sim \frac{3}{4}{\theta }^{-1}{\alpha }^{-\frac{5}{2}}\to \infty \mathrm{as}\alpha \to 0.$$

Furthermore, for $\theta >1$ and for $\alpha \to +\infty$, it is sufficient to analyze the sign of the value

$$\begin{array}{c}-\frac{1}{2}{g}^{″}\left(\alpha \right){(\alpha -1)}^2+g^{\prime }\left(\alpha \right)(\alpha -1)+g\left(\alpha \right)-1\hfill \\ \hfill \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\sim -\frac{1}{2}{\theta }^{\alpha -1}{\alpha }^{-\frac{1}{2}}{log}^2\theta {(\alpha -1)}^2\to -\infty \mathrm{as}\alpha \to \infty .\end{array}$$

This means that ${\mathbf{H}}_T$ is convex on some interval $(0,\alpha (1,\theta \left)\right)$ and concave on some interval $\left(\alpha \right(2,\theta ),+\infty )$, where the first statement is true for any $\theta >0$, while the second is true only for $\theta >1$.

□

Remark 4.

Figure 1 and Figure 2 correspond to the behavior of Tsallis entropy for $\theta <1$. Two cases need to be investigated numerically:

$$\theta \in (1,e^{x_0}),\alpha >\frac{x_0}{log\theta }$$

and

$$\theta >e^{x_0},\alpha <\frac{x_0}{log\theta }.$$

In both cases, we already know from item $\left(d\right)$ of Theorem 3 that ${\mathbf{H}}_T$ is a convex function on the interval $(0,\alpha (1,\theta \left)\right)$ and is a concave function on the interval $\left(\alpha \right(2,\theta ),\infty )$ for some $0<\alpha (1,\theta )<\alpha (2,\theta )<\infty$. The surfaces plotted on Figure 3 and Figure 4 confirm numerically that for any $\theta >1$, there exists the unique inflection point $\alpha (0,\theta )\in \left(\alpha \right(1,\theta ),\alpha (2,\theta \left)\right)$ of ${\mathbf{H}}_T$ as the function of α. Furthermore, Figure 5 and Figure 6 give us an idea of the entropy graphs for different θ. The marked points are points of the intersection of the entropy graph with the vertical line $\nu \left(\theta \right)=\frac{x_0}{log\theta }$. Note that $e^{x_0}\approx 2.86788$; therefore, the values of θ in Figure 5 correspond to the interval $(1,e^{x_0})$, while the values in Figure 6 correspond to $(e^{x_0},\infty )$.

Moreover, we numerically compared the values of the inflection points which are the solutions of the equation $H_T^{″}\left(\alpha \right)=0$ (which is equivalent to $\frac{1}{2}{g}^{″}\left(\alpha \right){(\alpha -1)}^2-g^{\prime }\left(\alpha \right)(\alpha -1)+g\left(\alpha \right)-1=0$; see (7)) with $\nu \left(\theta \right)$. Figure 7 confirms that the unique inflection point is close to $\nu \left(\theta \right)$, slightly overcomes $\nu \left(\theta \right)$ for $\theta <e^{x_0}$ and is less than $\nu \left(\theta \right)$ for $\theta >e^{x_0}$. In the case $\theta =e^{x_0}$, we have a coincidence of inflection points with $\nu \left(\theta \right)=1.$

Now, let us study the properties of the Sharma–Mittal entropy ${\mathbf{H}}_{SM}(\alpha ,\beta )$ as the function of parameters $\alpha$ and $\beta$. It is well known (see [21]) that the Shannon, Rényi and Tsallis entropies are the limiting cases of ${\mathbf{H}}_{SM}(\alpha ,\beta )$, namely

$$\begin{array}{c}{\mathbf{H}}_{SM}(\alpha ,\beta )\to {\mathbf{H}}_R\left(\alpha \right)\mathrm{as}\beta \to 1,\\ {\mathbf{H}}_{SM}(\alpha ,\beta )\to {\mathbf{H}}_T\left(\alpha \right)\mathrm{as}\beta \to \alpha ,\\ {\mathbf{H}}_{SM}(\alpha ,\beta )\to {\mathbf{H}}_S\mathrm{as}\alpha ,\beta \to 1.\end{array}$$

Therefore, the Sharma–Mittal entropy can be extended to a continuous function of $\alpha$ and $\beta$.

Theorem 4.

Consider, as before, the centered normal distribution with variance ${\sigma }^2.$

(1) 

Let us denote

$${\theta }_1={\theta }_1\left(\alpha \right)=\frac{\sqrt{2\pi }\sigma }{{\alpha }^{\frac{1}{2(1-\alpha )}}}.$$

For any fixed $\alpha >0$, $\alpha \ne 1$, ${\mathbf{H}}_{SM}(\alpha ,\beta )$ decreases in $\beta \in (0,+\infty )$, namely as follows:

(i) 

If ${\theta }_1=1$, then ${\mathbf{H}}_{SM}(\alpha ,\beta )=0$.

(ii) 

If ${\theta }_1<1$, then ${\mathbf{H}}_{SM}(\alpha ,\beta )$ decreases from ${\theta }_1-1$ to $-\infty$.

(iii) 

If ${\theta }_1>1$, then ${\mathbf{H}}_{SM}(\alpha ,\beta )$ decreases from ${\theta }_1-1$ to 0.

(b) 

For any fixed $\alpha >0$, $\alpha \ne 1$, the function ${\mathbf{H}}_{SM}(\alpha ,\beta )$ is concave in β if ${\theta }_1<1$, and it is convex if ${\theta }_1>1$.

(b) 

For a fixed $\beta \in (0,1)\cup (1,\infty )$, ${\mathbf{H}}_{SM}(\alpha ,\beta )$ is a decreasing and convex function in α.

Proof. 

(1) We have

$${\mathbf{H}}_{SM}(\alpha ,\beta )=\frac{{\theta }_1^{1-\beta }-1}{1-\beta },\beta \in (0,1)\cup (1,\infty ).$$

Denote $x=1-\beta \in (-\infty ,0)\cup (0,1)$. Then,

$$F\left(x\right)=\frac{{\theta }_1^x-1}{x}=\frac{{\theta }_1^x-{\theta }_1^0}{x-0}$$

is a slope function for $f\left(x\right)={\theta }_1^x$, which in convex. Therefore, $F\left(x\right)$ is increasing in $x\in (-\infty ,0)\cup (0,1)$. Consequently, the statements (i)–(iii) hold.

(2) Let ${\theta }_1<1$. We can write

$$-{\mathbf{H}}_{SM}(\alpha ,\beta )=\frac{g_1\left(\beta \right)-g_1\left(1\right)}{\beta -1},\beta \in (0,1)\cup (1,\infty ).$$

where $g_1\left(\beta \right)={\theta }_1^{1-\beta }$. Since the third derivative $g_1^{‴}\left(\beta \right)=-{\theta }_1^{1-\beta }{(log{\theta }_1)}^3$ is positive for ${\theta }_1<1$, we see that $-{\mathbf{H}}_{SM}(\alpha ,\beta )$ is convex in $\beta$ by Lemma A1 from Appendix B. Hence, ${\mathbf{H}}_{SM}(\alpha ,\beta )$ is concave in $\beta$.

In the case ${\theta }_1>1$, we represent ${\mathbf{H}}_{SM}(\alpha ,\beta )$ in the following form:

$${\mathbf{H}}_{SM}(\alpha ,\beta )=\frac{g_2\left(\beta \right)-g_2\left(1\right)}{\beta -1},\beta \in (0,1)\cup (1,\infty ),$$

where $g_2\left(\beta \right)=-g_1\left(\beta \right)=-{\theta }_1^{1-\beta }$. For ${\theta }_1>1$, we have $g_2^{‴}\left(\beta \right)={\theta }_1^{1-\beta }{(log{\theta }_1)}^3>0$; hence, the desired convexity follows from Lemma A1.

(3) It is not hard to see that

$$log{\theta }_1\left(\alpha \right)={\mathbf{H}}_R\left(\alpha \right),$$

(8)

where ${\mathbf{H}}_R\left(\alpha \right)$ is the Rényi entropy, which is a decreasing function of $\alpha$ according to Theorem 1. Hence, ${\theta }_1\left(\alpha \right)$ decreases. Moreover,

$$\frac{\partial }{\partial \alpha }{\mathbf{H}}_{SM}(\alpha ,\beta )=\frac{{\theta }_1^{\prime }\left(\alpha \right)}{{\theta }_1^{\beta }\left(\alpha \right)}<0,$$

(9)

and ${\mathbf{H}}_{SM}(\alpha ,\beta )$ also decreases in $\alpha$.

In order to establish convexity, we differentiate (8) and obtain $\frac{{\theta }_1^{\prime }\left(\alpha \right)}{{\theta }_1\left(\alpha \right)}={\mathbf{H}}_R^{\prime }\left(\alpha \right)$, whence ${\theta }_1^{\prime }\left(\alpha \right)={\theta }_1\left(\alpha \right){\mathbf{H}}_R^{\prime }\left(\alpha \right)$ and

$${\theta }_1^{″}\left(\alpha \right)={\theta }_1^{\prime }\left(\alpha \right){\mathbf{H}}_R^{\prime }\left(\alpha \right)+{\theta }_1\left(\alpha \right){\mathbf{H}}_R^{″}\left(\alpha \right)>0,$$

because ${\theta }_1^{\prime }\left(\alpha \right)<0$, ${\mathbf{H}}_R^{\prime }\left(\alpha \right)<0$, and ${\mathbf{H}}_R^{″}\left(\alpha \right)>0$, according to Theorem 1 and to the previous statement. Then, by differentiation of (9), we obtain

$$\begin{array}{c}\frac{{\partial }^2}{\partial {\alpha }^2}{\mathbf{H}}_{SM}(\alpha ,\beta )=\frac{{\theta }_1^{″}\left(\alpha \right){\theta }_1^{\beta }\left(\alpha \right)-\beta {\theta }_1^{\beta -1}\left(\alpha \right){\theta }_1^{\prime }\left(\alpha \right)}{{\theta }_1^{2\beta }\left(\alpha \right)}>0,\end{array}$$

since ${\theta }_1^{″}\left(\alpha \right)>0$ and ${\theta }_1^{\prime }\left(\alpha \right)<0$. Thus, convexity is proved.  □

Remark 5.

Let us consider the equation ${\theta }_1=1$, i.e.,

$${\alpha }^{\frac{1}{2(1-\alpha )}}=\sqrt{2\pi }\sigma$$

or

$$\frac{log\alpha }{2(1-\alpha )}=log\left(\sqrt{2\pi }\sigma \right)=:\rho .$$

According to the proof of Theorem 1 (statement 3), the function $\frac{log\alpha }{2(1-\alpha )}$ increases from $-\infty$ to 0 in $\alpha \in (0,1)\cup (1,\infty )$.

Thus:

• If $\sqrt{2\pi }\sigma >1$, then ${\theta }_1>1$ and (iii) holds.
• If $\sqrt{2\pi }\sigma =1$, then (iii) holds too.
• If $\sqrt{2\pi }\sigma <1$, then let ${\alpha }_0$ be a number such that

  $${\alpha }_0^{\frac{1}{2(1-{\alpha }_0)}}=\sqrt{2\pi }\sigma .$$
  If $\alpha <{\alpha }_0$, then ${\theta }_1>1$ and (iii) holds. If $\alpha >{\alpha }_0$, then ${\theta }_1<1$ and (ii) holds. If $\alpha ={\alpha }_0$, then ${\theta }_1=1$ and (i) holds.

3. Examples of Gaussian Fractional Processes with Their Variances: Entropies of Fractional Gaussian Processes

Now, we consider several types of fractional Gaussian processes. Our goal is very simple: to compare the entropies of their marginal distributions. In order to compare their entropies and variances correctly, we normalize the variances using the normalizing coefficients so that at the point $t=1$, the variance of every process equals 1.

As already mentioned in the introduction, the entropy of vector fractional Gaussian noise is calculated using the formulas given in the book [23]. However, firstly, these calculations are based on the fact that fractional Gaussian noise is a stationary process, and secondly, using them to compare different processes, even fractional Gaussian noise with different Hurst indices, is too complicated a problem for an analytical solution. The main difficulty is that the formula for the entropy of a Gaussian vector contains the determinant of the covariance matrix, the calculation of which there are no simple proposals for at the moment, except for cumbersome standard formulas, and at the same time, they do not make it possible to compare these determinants. Therefore, having at our disposal several classes of fractional processes that model a wide variety of processes, from physics to financial mathematics, we set out to compare in as simple a way as possible the terms of the information they carry, or, more simply, to compare their entropies. The comparisons of entropies presented are based on calculating the variances of the corresponding processes, and these calculations are quite simple and understandable to a wide range of readers.

3.1. Fractional, Subfractional and Bifractional Brownian Motions

Let us start with the definition of fractional Brownian motion. This process was first introduced in [25].

Definition 2.

A centered Gaussian process $B^H=\{B_t^H,t\ge 0\}$ with a covariance function

$$Cov\left(B_t^H,B_s^H\right)=\frac{1}{2}\left(t^{2H}+s^{2H}-{|t-s|}^{2H}\right),t,s\in {\mathbb{R}}_{+}$$

is called a fractional Brownian motion (fBm) with the Hurst parameter $H\in (0,1)$.

Obviously, $Var\left(B_t^H\right)=t^{2H}$.

Definition 3

([26]). A centered Gaussian process ${\xi }^H=\{{\xi }_t^H,t\ge 0\}$ with a covariance function

$$Cov\left({\xi }_t^H,{\xi }_s^H\right)=\left(s^{2H}+t^{2H}-\frac{1}{2}\left({(s+t)}^{2H}+{|t-s|}^{2H}\right)\right),t,s\ge 0$$

is called a subfractional Brownian motion with the Hurst parameter $H\in (0,1)$

Obviously,

$$Var\left({\xi }_t^H\right)=\left(2t^{2H}-\frac{1}{2}{\left(2t\right)}^{2H}\right)=\left(2-2^{2H-1}\right)t^{2H}.$$

Let us put ${\overline{\xi }}_t^H={(2-2^{2H-1})}^{-\frac{1}{2}}{\xi }_t^H$. Then, ${\overline{\xi }}_t^H$ has the same variance as fractional Brownian motion.

Definition 4

([27] (p.71)). The process ${\mathcal{L}}^H=\{{\mathcal{L}}_t^H,t\ge 0\}$, defined by

$${\mathcal{L}}_t^H=\frac{1}{\mathsf{\Gamma }(H+\frac{1}{2})}{\int }_0^t{(t-s)}^{H-1/2}d{W}_s,t\ge 0,H\in (0,1),$$

and where $W=\{W_t,t\ge 0\}$ is a Wiener process, is called a Riemann–Liouville fractional Brownian motion.

Then

$$Var\left({\mathcal{L}}_t^H\right)=\frac{1}{{\mathsf{\Gamma }}^2(H+\frac{1}{2})}{\int }_0^t{(t-u)}^{2H-1}du=\frac{t^{2H}}{2H{\mathsf{\Gamma }}^2(H+\frac{1}{2})}.$$

Therefore, the process ${\overline{\mathcal{L}}}^H={\left(2H\right)}^{1/2}\mathsf{\Gamma }(H+\frac{1}{2}){\mathcal{L}}_t^H$ has the same variance as fractional Brownian motion and subfractional Brownian motion.

Definition 5

([28]). A centered Gaussian process $B^{H,K}=\{B_t^{H,K},t\ge 0\}$, starting from zero, with a covariance function

$$Cov\left(B_t^{H,K},B_s^{H,K}\right):=\frac{1}{2^K}\left({(t^{2H}+s^{2H})}^K-{|t-s|}^{2HK}\right)$$

is called a bifractional Brownian motion with $H\in (0,1)$ and $K\in (0,1]$.

Then

$$Var\left(B_t^{H,K}\right)=Cov\left(B_t^{H,K},B_t^{H,K}\right)=\frac{1}{2^K}\left({(t^{2H}+t^{2H})}^K\right)=t^{2HK}.$$

Obviously, at point $t=1$, variance equals $Var\left(B_1^{H,K}\right)=1$.

Proposition 2.

Let X be one of the following processes: $B^H$, ${\overline{\xi }}^H$ or ${\overline{\mathcal{L}}}^H$. Then, one has the following formulas for the entropies of $X_t$:

(1) 

The Shannon entropy equals

$${\mathbf{H}}_S^X\left(t\right)=Hlogt+\frac{1}{2}(1+log\left(2\pi \right)).$$

(10)

(2) 

The Rényi entropy ($\alpha >0$, $\alpha \ne 1$) equals

$${\mathbf{H}}_R^X(\alpha ,t)=Hlogt+\frac{1}{2}log\left(2\pi \right)+\frac{log\alpha }{2(\alpha -1)}.$$

(11)

For $\alpha =1$, we extend the Rényi entropy by the Shannon entropy continuously.

(3) 

The generalized Rényi entropy in the case $\alpha =\beta$ equals

$${\mathbf{H}}_{GR}^X(\alpha ,t)=Hlogt+\frac{1}{2}log\left(2\pi \right)+\frac{1}{2\alpha },$$

(12)

and for $\alpha =\beta =1$, we extend the generalized Rényi entropy by the Shannon entropy (and Rényi entropy with $\alpha =1$) continuously.

(4) 

The generalized Rényi entropy in the case $\alpha \ne \beta$ equals

$${\mathbf{H}}_{GR}^X(\alpha ,\beta ,t)=Hlogt+\frac{1}{2}log\left(2\pi \right)+\frac{log\beta -log\alpha }{2(\beta -\alpha )},$$

(13)

and for $\alpha =\beta$, it can be extended by the generalized Rényi entropy in the case $\alpha =\beta$ continuously.

(5) 

The Tsallis entropy ($\alpha >0$, $\alpha \ne 1$) equals

$${\mathbf{H}}_T^X(\alpha ,t)=\frac{t^{(1-\alpha )H}{\left(2\pi \right)}^{\frac{1-\alpha }{2}}{\alpha }^{-1/2}-1}{1-\alpha },$$

(14)

and for $\alpha =1$, it can be extended by the Shannon entropy continuously.

(6) 

The Sharma–Mittal entropy for $\alpha ,\beta \in (0,1)\cup (1,\infty )$ equals

$${\mathbf{H}}_{SM}^X(\alpha ,\beta ,t)=\frac{1}{1-\beta }\left(\frac{t^{(1-\beta )H}{\left(2\pi \right)}^{\frac{1-\beta }{2}}}{{\alpha }^{\frac{1-\beta }{2(1-\alpha )}}}-1\right),$$

(15)

and for $\beta =1$, it can be extended by the Rényi entropy continuously.

(7) 

The same statements hold for $X=B^{H,K}$ with $HK$ instead of H. This means that any entropy of bifractional Brownian motion with parameters H and K equals to the corresponding entropy of fBm with Hurst index $H^{\prime }=HK.$ In turn, this means that if we fix the same H in fBm and bifractional Brownian motion and take $K<1$, then

$${\mathbf{H}}_A^{B^H}(·,t)<{\mathbf{H}}_A^{B^{H,K}}(·,t),t<1,$$

$A=S,R,GR,$ and opposite inequality holds for $t>1$. For ${\mathbf{H}}_T^X(\alpha ,t)$, the situation is more involved: if $t<1,\alpha <1$ or $t>1,\alpha >1$, then

$${\mathbf{H}}_T^{B^H}(\alpha ,t)<{\mathbf{H}}_T^{B^{H,K}}(\alpha ,t),$$

and for $t<1,\alpha >1$ or $t>1,\alpha <1$, the opposite inequality holds.

Figure 8 contains graphs of various entropies of fractional Brownian motion with Hurst parameter $H=0.75$.

Remark 6.

It is interesting and natural to compare the variance of fBm $B^H=\{B_t^H,t\ge 0\}$ with the variance of the corresponding fractional Ornstein–Uhlenbeck process $X_t^H={\int }_0^t{e}^{\alpha (t-s)}d{B}_s^H$ for various H and α and consequently to compare their entropies. Consider the following cases.

$\left(i\right)$ Let $H>\frac{1}{2}$. Then, according to [29],

$$Var{X}_t^H=2H(2H-1){\int }_0^t{\int }_0^t{e}^{\alpha (2t-s-u)}{\left|s-u\right|}^{2H-2}duds.$$

If $\alpha >0$, then $e^{\alpha (2t-s-u)}>1$, and

$$Var{X}_t^H>2H(2H-1){\int }_0^t{\int }_0^t{\left|s-u\right|}^{2H-2}duds=Var{B}_t^H.$$

Similarly, if $\alpha <0$, then $Var{X}_t^H<Var{B}_t^H$.

$\left(ii\right)$ Let $H<\frac{1}{2}$. Then, the integral is understood via integration by parts [30]:

$$X_t^H=B_t^H+\alpha {\int }_0^t{e}^{\alpha (t-s)}{B}_s^{H}ds.$$

Let $\alpha >0$. Then,

$$\begin{array}{c}Var{X}_t^H=Var{B}_t^H+2\alpha {\int }_0^t{e}^{\alpha (t-s)}\mathbf{E}\left[B_t^H{B}_s^H\right]ds\hfill \\ \hfill +{\alpha }^2{\int }_0^t{\int }_0^t{e}^{\alpha (2t-s-u)}\mathbf{E}\left[B_s^H{B}_u^H\right]duds>Var{B}_t^H,\end{array}$$

(16)

because $\mathbf{E}\left[B_s^H{B}_u^H\right]=\frac{1}{2}(s^{2H}+u^{2H}-{\left|u-s\right|}^{2H})>0$ and similarly $\mathbf{E}\left[B_t^H{B}_s^H\right]>0$.

Let $\alpha <0$. Denote $\beta =-\alpha >0$. Then, our goal is to determine the sign of the value

$$\theta \left(t\right):=-2\beta {\int }_0^t{e}^{-\beta (t-s)}\mathbf{E}\left[B_t^H{B}_s^H\right]ds+{\beta }^2{\int }_0^t{\int }_0^t{e}^{-\beta (2t-s-u)}\mathbf{E}\left[B_s^H{B}_u^H\right]duds.$$

Now, we change the variables $\beta s=r$, $\beta u=q$ and take into account that $\mathbf{E}\left[B_{t/\beta }^H{B}_{s/\beta }^H\right]={\beta }^{-2H}\mathbf{E}\left[B_t^H{B}_s^H\right]$, and similarly, $\mathbf{E}\left[B_{s/\beta }^H{B}_{u/\beta }^H\right]={\beta }^{-2H}\mathbf{E}\left[B_s^H{B}_u^H\right]$, denote $\beta t=z$ and also take into account the symmetry of the integrand in the 2nd integral and, with all of this at hand, arrive at the value

$$\begin{array}{cc}\hfill \phi \left(z\right)& =-{\int }_0^z{e}^{-z+r}\left(z^{2H}+r^{2H}-{(z-r)}^{2H}\right)dr\hfill \\ \hfill & +{\int }_0^z{\int }_0^r{e}^{-2z+r+q}\left(r^{2H}+q^{2H}-{(r-q)}^{2H}\right)dqdr,\hfill \end{array}$$

whose sign is interesting for us. Obviously,

$$\begin{array}{cc}\hfill \phi \left(z\right)& =-z^{2H}{e}^{-z}\left(e^z-1\right)-e^{-z}{\int }_0^z{e}^r{r}^{2H}dr+{\int }_0^z{e}^{-v}{v}^{2H}dv+e^{-2z}{\int }_0^z{e}^r{r}^{2H}\left(e^r-1\right)dr\hfill \\ \hfill & +e^{-2z}{\int }_0^z{e}^r{\int }_0^r{e}^q{q}^{2H}dqdr-e^{-2z}{\int }_0^z{e}^{2r}{\int }_0^r{e}^{-v}{v}^{2H}dvdr.\hfill \end{array}$$

Equivalently, we can consider the sign of function

$$\begin{array}{cc}\hfill \psi \left(z\right)=e^{2z}\phi \left(z\right)& =-z^{2H}\left(e^{2z}-e^z\right)-e^z{\int }_0^z{e}^r{r}^{2H}dr+e^{2z}{\int }_0^z{e}^{-v}{v}^{2H}dv+{\int }_0^z{e}^r{r}^{2H}\left(e^r-1\right)dr\hfill \\ \hfill & +{\int }_0^z{e}^r{\int }_0^r{e}^q{q}^{2H}dqdr-{\int }_0^z{e}^{2r}{\int }_0^r{e}^{-v}{v}^{2H}dvdr.\hfill \end{array}$$

Obviously, $\psi \left(0\right)=0$. Furthermore,

$$\begin{array}{cc}\hfill {\psi }^{\prime }\left(z\right)& =-2H{z}^{2H-1}\left(e^{2z}-e^z\right)-z^{2H}\left(2e^{2z}-e^z\right)-e^z{\int }_0^z{e}^r{r}^{2H}dr-e^{2z}{z}^{2H}\hfill \\ \hfill & +2e^{2z}{\int }_0^z{e}^{-v}{v}^{2H}dv+e^z{z}^{2H}+e^z{z}^{2H}\left(e^z-1\right)+e^z{\int }_0^z{e}^q{q}^{2H}dq-e^{2z}{\int }_0^z{e}^{-v}{v}^{2H}dv\hfill \\ \hfill & =-2H{z}^{2H-1}\left(e^{2z}-e^z\right)-2z^{2H}{e}^{2z}+z^{2H}{e}^z-e^{2z}{z}^{2H}\hfill \\ \hfill & +2e^{2z}{\int }_0^z{e}^{-v}{v}^{2H}dv+e^z{z}^{2H}+e^{2z}{z}^{2H}-e^z{z}^{2H}-e^{2z}{\int }_0^z{e}^{-v}{v}^{2H}dv\hfill \\ \hfill & =-2H{z}^{2H-1}\left(e^{2z}-e^z\right)-2z^{2H}{e}^{2z}+z^{2H}{e}^z+e^{2z}{\int }_0^z{e}^{-v}{v}^{2H}dv\hfill \\ \hfill & <-2H{z}^{2H-1}\left(e^{2z}-e^z\right)-2z^{2H}{e}^{2z}+z^{2H}{e}^z+e^{2z}{z}^{2H}\left(1-e^{-z}\right)\hfill \\ \hfill & =-2H{z}^{2H-1}\left(e^{2z}-e^z\right)-z^{2H}{e}^{2z}<0.\hfill \end{array}$$

This means that $\psi \left(z\right)<0$ for all $z>0$; consequently, $\theta \left(t\right)<0$ for all $t>0$. Together with (16), this finally means that $Var{X}_t^H>Var{B}_t^H$ if $\alpha >0$, and $Var{X}_t^H<Var{B}_t^H$ if $\alpha <0$.

Remark 7.

Note that fractional Ornstein–Uhlenbeck process was generalized in the papers [31,32] to the massive-FBM and the diffusing–diffusivity-FBM. The diffusing–diffusivity-FBM is non-Gaussian, but the massive-FBM can be considered in the framework of fractional Ornstein–Uhlenbeck processes, and the calculations above can help in the comparison of the entropies.

3.2. Multifractional Brownian Motion

Let us consider various definitions of multifractional Brownian motion. The difference is in the form of their representation; the same situation that we have with standard fBm: it admits Mandelbrot–van Ness representation [33] on the whole axis, Molchan–Golosov compact interval representation [29] and spectral representation (Section 7.2.2 of [34]). However, covariance and variance functions of all fBms are the same and can differ only by normalizing multipliers (correct values of the multipliers are provided, for example, in [35]). Considering different representations of multifractional Brownian motion, the authors introduce different normalizing multipliers, basically following the form of this factor for the corresponding representation of fractional Brownian motion, but in this case, they depend on time. Below, we provide these representations and analyze the relations between them and the behavior of the normalizing multipliers as the functions on time, because their value influences the value of variance and consequently the value of the entropy. Obviously, if we renormalize the processes in order to equate their variances, the entropies become equal.

Let $H:{\mathbb{R}}_{+}\to [a,b]\subset (0,1)$ be a continuous function.

Definition 6

([36]). For $t\ge 0$, the following random function is called moving-average multifractional Brownian motion with functional parameter H:

$$\begin{array}{c}{Y}_1\left(t\right)=\frac{1}{\mathsf{\Gamma }(H_t+\frac{1}{2})}\left({\int }_{-\infty }^0[{(t-s)}^{H_t-1/2}-{(-s)}^{H_t-1/2}]dW\left(s\right)\right.\hfill \\ \hfill +\left.{\int }_0^t{(t-s)}^{H_t-1/2}dW\left(s\right)\right)=\frac{1}{\mathsf{\Gamma }(H_t+\frac{1}{2})}\left({\int }_{-\infty }^t[{(t-s)}^{H_t-1/2}-{(-s)}_{+}^{H_t-1/2}]dW\left(s\right)\right.,\end{array}$$

(17)

where W denotes the Brownian motion.

It follows from Cor. 3.4 of [33] that

$$Var\left[Y_1\left(t\right)\right]=c_1\left(H_t\right)t^{2H_t},$$

where

$$c_1\left(x\right)=\frac{1}{\mathsf{\Gamma }{(x+\frac{1}{2})}^2}\left({\int }_{-\infty }^0{\left({(1-s)}^{x-\frac{1}{2}}-{(-s)}^{x-\frac{1}{2}}\right)}^{2}ds+\frac{1}{2H}\right),x\in (0,1).$$

The function $c_1\left(x\right)$ can be written in the following form Appendix A of [35]:

$$c_1\left(x\right)=\frac{1}{2x\mathsf{\Gamma }\left(2x\right)sin\left(\pi x\right)}.$$

(18)

Let us consider the process ${\overline{Y}}_1\left(t\right)=\frac{Y_1\left(t\right)}{\sqrt{c_1\left(H_1\right)}}$. Then,

$$Var\left[{\overline{Y}}_1\left(t\right)\right]=\frac{c_1\left(H_t\right)}{c_1\left(H_1\right)}{t}^{2H_t},Var\left[{\overline{Y}}_1\left(1\right)\right]=1.$$

Remark 8.

The coefficient $\frac{1}{\mathsf{\Gamma }(H_t+1)}$ in (17) goes back to the seminal work of Mandelbrot and van Ness [33], who defined fractional Brownian motion as a fractional integral of the Wiener process (this factor ensures that a fractional integral becomes an ordinary repeated integral for integer values of $H_t-\frac{1}{2}$). However, in the literature, the moving-average multifractional Brownian motion is often defined with a different normalizing constant, namely, it is defined as ${\tilde{Y}}_1\left(t\right)=\frac{Y_1\left(t\right)}{\sqrt{c_1\left(H_t\right)}}$. In this case, we obviously have that $Var{\tilde{Y}}_1\left(t\right)=t^{2H_t}$.

Let us consider a different type of multifractional Brownian motion, introduced in [37,38]. It is based on the Molchan–Golosov compact interval representation of fBm [29]. Note that in the next definition, the class of appropriate Hurst functions is restricted to the case $H_t>\frac{1}{2}$.

Definition 7

([37]). Let $H:{\mathbb{R}}_{+}\to (\frac{1}{2},1)$. For $t\ge 0$, the following random function is called Volterra-type multifractional Brownian motion with functional parameter H:

$$Y_2\left(t\right)={\int }_0^t{s}^{1/2-H_t}\left({\int }_s^t{u}^{H_t-1/2}{(u-s)}^{H_t-3/2}du\right)dW\left(s\right),$$

where W denotes the Brownian motion.

Then, by Prop. 2 of [37],

$$Var\left[Y_2\left(t\right)\right]=c_2\left(H_t\right)t^{2H_t},$$

where

$$c_2\left(x\right)=\frac{\mathsf{\Gamma }(2-2x)\mathsf{\Gamma }{(x-\frac{1}{2})}^{2}sin\left(\pi (x-\frac{1}{2})\right)}{2\pi x(x-\frac{1}{2})},x\in (1/2,1).$$

(19)

Hence, the process ${\overline{Y}}_2\left(t\right)=\frac{Y_2\left(t\right)}{\sqrt{c_2\left(H_1\right)}}$ has the variance

$$Var\left[{\overline{Y}}_2\left(t\right)\right]=\frac{c_2\left(H_t\right)}{c_2\left(H_1\right)}{t}^{2H_t}$$

and $Var\left[{\overline{Y}}_2\left(1\right)\right]=1$.

Remark 9.

In [38], the authors defined the Volterra-type multifractional Brownian motion with a normalizing function in front of the integral, i.e., by the relation ${\tilde{Y}}_2\left(t\right)=\frac{Y_2\left(t\right)}{\sqrt{c_2\left(H_t\right)}}$. Evidently, in this case, one has $Var{\tilde{Y}}_2\left(t\right)=t^{2H_t}$.

Definition 8

([39]). The harmonizable multifractional Brownian motion with functional parameter H is defined by

$$Y_3\left(t\right)={\int }_{\mathbb{R}}\frac{e^{itu}-1}{{\left|u\right|}^{H_t+1/2}}dW\left(u\right),t\ge 0.$$

where $\tilde{W}\left(du\right)$ is the “Fourier transform” of the white noise $W\left(du\right)$ that is a unique complex-valued random measure such that for all $f\in L^2\left(\mathbb{R}\right)$

$${\int }_{\mathbb{R}}f\left(u\right)W\left(du\right)={\int }_{\mathbb{R}}\widehat{f}\left(u\right)\tilde{W}\left(du\right)a.s.;$$

see [39,40].

It is known from Prop. 4 of [41] that

$$Var\left[Y_3\left(t\right)\right]=c_3\left(H_t\right)t^{2H_t},$$

(20)

where

$$c_3\left(x\right)=\frac{\pi }{x\mathsf{\Gamma }\left(2x\right)sin\left(\pi x\right)}.$$

(21)

Define the normalized version of the harmonizable multifractional Brownian motion by ${\overline{Y}}_3\left(t\right)=\frac{Y_3\left(t\right)}{\sqrt{c_3\left(H_1\right)}}$ so that

$$Var\left[{\overline{Y}}_3\left(t\right)\right]=\frac{c_3\left(H_t\right)}{c_3\left(H_1\right)}{t}^{2H_t}\mathrm{and}Var\left[{\overline{Y}}_3\left(1\right)\right]=1.$$

Proposition 3.

One has the following formulas for the entropies of ${\overline{Y}}_i\left(t\right)$, $i=1,2,3$.

(1) 

The Shannon entropy equals

$${\mathbf{H}}_S^{{\overline{Y}}_i}\left(t\right)=H_{t}logt+\frac{1}{2}log{c}_i\left(H_t\right)-\frac{1}{2}log{c}_i\left(H_1\right)+\frac{1}{2}(1+log\left(2\pi \right)).$$

(22)

(2) 

The Rényi entropy ($\alpha >0$, $\alpha \ne 1$) equals

$${\mathbf{H}}_R^{{\overline{Y}}_i}(\alpha ,t)=H_{t}logt+\frac{1}{2}log{c}_i\left(H_t\right)-\frac{1}{2}log{c}_i\left(H_1\right)+\frac{1}{2}log\left(2\pi \right)+\frac{log\alpha }{2(\alpha -1)}.$$

(23)

(3) 

The generalized Rényi entropy in the case $\alpha =\beta$ equals

$${\mathbf{H}}_{GR}^{{\overline{Y}}_i}(\alpha ,t)=H_{t}logt+\frac{1}{2}log{c}_i\left(H_t\right)-\frac{1}{2}log{c}_i\left(H_1\right)+\frac{1}{2}log\left(2\pi \right)+\frac{1}{2\alpha }.$$

(24)

(4) 

The generalized Rényi entropy in the case $\alpha \ne \beta$ equals

$$\begin{array}{c}{\mathbf{H}}_{GR}^{{\overline{Y}}_i}(\alpha ,\beta ,t)=H_{t}logt+\frac{1}{2}log{c}_i\left(H_t\right)-\frac{1}{2}log{c}_i\left(H_1\right)\hfill \\ \hfill +\frac{1}{2}log\left(2\pi \right)+\frac{log\beta -log\alpha }{2(\beta -\alpha )}.\end{array}$$

(25)

(5) 

The Tsallis entropy ($\alpha >0$, $\alpha \ne 1$) equals

$${\mathbf{H}}_T^{{\overline{Y}}_i}(\alpha ,t)=\frac{t^{(1-\alpha )H_t}{(2\pi c_i\left(H_t\right)/c_i\left(H_1\right))}^{\frac{1-\alpha }{2}}{\alpha }^{-1/2}-1}{1-\alpha }.$$

(26)

(6) 

The Sharma–Mittal entropy for $\alpha ,\beta \in (0,1)\cup (1,\infty )$ equals

$${\mathbf{H}}_{SM}^{{\overline{Y}}_i}(\alpha ,\beta ,t)=\frac{1}{1-\beta }\left(\frac{t^{(1-\beta )H_t}{(2\pi c_i\left(H_t\right)/c_i\left(H_1\right))}^{\frac{1-\beta }{2}}}{{\alpha }^{\frac{1-\beta }{2(1-\alpha )}}}-1\right).$$

(27)

Now, let us compare entropies for various versions of multifractional Brownian motion. Recall that the Volterra-type multifractional Brownian motion $Y_2$ is well-defined only for $H_t>\frac{1}{2}$.

Proposition 4.

Let $H:{\mathbb{R}}_{+}\to (0,1)$ and $A=S,R,GR,T,SM$.

(1) 

For all $t\ge 0$, ${\mathbf{H}}_A^{{\overline{Y}}_1}(·,t)={\mathbf{H}}_A^{{\overline{Y}}_3}(·,t)$.

(2) 

Let $H_t>\frac{1}{2}$. Then

$$\begin{array}{cc}\hfill {\mathbf{H}}_A^{{\overline{Y}}_1}(·,t)& \le {\mathbf{H}}_A^{{\overline{Y}}_2}(·,t)\mathrm{if}{H}_t\le H_1,\hfill \\ \hfill {\mathbf{H}}_A^{{\overline{Y}}_1}(·,t)& \ge {\mathbf{H}}_A^{{\overline{Y}}_2}(·,t)\mathrm{if}{H}_t\ge H_1.\hfill \end{array}$$

Proof. 

$\left(i\right)$ It follows immediately from (18) and (21) that $\frac{c_3\left(x\right)}{c_1\left(x\right)}=2\pi$ for all $x\in (0,1)$. This means that for any Hurst function $H_t$,

$$Var\left[{\overline{Y}}_1\left(t\right)\right]=\frac{c_1\left(H_t\right)}{c_1\left(H_1\right)}{t}^{2H_t}=\frac{c_3\left(H_t\right)}{c_3\left(H_1\right)}{t}^{2H_t}=Var\left[{\overline{Y}}_3\left(t\right)\right],$$

i.e., the entropies for $Y_1\left(t\right)$ and $Y_3\left(t\right)$ coincide.

$\left(ii\right)$ According to Remark 1, all entropies are increasing functions of the variance. Therefore, it suffices to compare the variances of moving-average and Volterra-type multifractional Brownian motions.

For $x\in (\frac{1}{2},1)$, Formulas (18) and (19) imply that

$$\begin{array}{cc}\hfill \frac{c_2\left(x\right)}{c_1\left(x\right)}& =\frac{\mathsf{\Gamma }\left(2x\right)\mathsf{\Gamma }(2-2x)\mathsf{\Gamma }{(x-\frac{1}{2})}^{2}sin\left(\pi (x-\frac{1}{2})\right)sin\left(\pi x\right)}{\pi (x-\frac{1}{2})}\hfill \\ \hfill & =\frac{2\mathsf{\Gamma }(2x-1)\mathsf{\Gamma }(2-2x)\mathsf{\Gamma }{(x-\frac{1}{2})}^{2}sin\left(\pi (x-\frac{1}{2})\right)sin\left(\pi x\right)}{\pi }\hfill \\ \hfill & =\frac{2sin\left(\pi (x-\frac{1}{2})\right)sin\left(\pi x\right)}{sin\left(\pi \right(2x-1\left)\right)}\mathsf{\Gamma }{(x-\frac{1}{2})}^2=\mathsf{\Gamma }{(x-\frac{1}{2})}^2.\hfill \end{array}$$

Hence,

$$\frac{Var{\overline{Y}}_2\left(t\right)}{Var{\overline{Y}}_1\left(t\right)}=\frac{\mathsf{\Gamma }{(H_t-\frac{1}{2})}^2}{\mathsf{\Gamma }{(H_1-\frac{1}{2})}^2}.$$

Since the function $\mathsf{\Gamma }(x-\frac{1}{2})$ decreases for $x\in (\frac{1}{2},1)$, we see that $Var{\overline{Y}}_1\left(t\right)\le Var{\overline{Y}}_2\left(t\right)$ if and only if $H_t\le H_1$.  □

3.3. Tempered Fractional Brownian Motion

Two classes of continuous stochastic Gaussian processes, known as tempered fractional Brownian motion (TFBM) and tempered fractional Brownian motion of the second kind (TFBMII), were recently introduced in [42] and [43], respectively. These processes modify the power law kernel used in the moving-average representation of fBm by introducing exponential tempering. Unlike standard fBm, TFBMs can be defined for any Hurst parameter value $H>0$. These processes attracted the attention of researchers in various fields. Notably, a stochastic phenomenological bifurcation of the Langevin equation perturbed by TFBM was constructed in [44], revealing diverse and intriguing bifurcation phenomena. Additionally, TFBM and TFBMII are valuable as stochastic models for data exhibiting fractional Brownian motion characteristics at intermediate scales but deviating at longer scales, such as wind speed measurements.

Definition 9.

Given an independently scattered Gaussian random measure $W\left(dx\right)$ on $\mathbb{R}$ with control measure dx, for any $H>0$ and $\lambda >0$, the stochastic process $B_{H,\lambda }^I=\{B_{H,\lambda }^I\left(t\right),t\ge 0\}$ defined by the Wiener integral

$$B_{H,\lambda }^I\left(t\right):={\int }_{-\infty }^t\left[e^{-\lambda (t-x)}{(t-x)}^{H-1/2}-e^{-\lambda {(-x)}_{+}}{(-x)}_{+}^{H-1/2}\right]W\left(dx\right),$$

where $0^0=0$ is called a tempered fractional Brownian motion (TFBM).

Since TFBM ([45] (p. 7)) has the covariance function

$$Cov\left(B_{H,\lambda }^I\left(t\right),B_{H,\lambda }^I\left(s\right)\right)=\frac{1}{2}\left[{\left(C_t^I\right)}^2{t}^{2H}+{\left(C_s^I\right)}^2{s}^{2H}-{\left(C_{|t-s|}^I\right)}^2{|t-s|}^{2H}\right]$$

for any $s,t\in \mathbb{R}$, where

$${\left(C_t^I\right)}^2=\frac{2\mathsf{\Gamma }\left(2H\right)}{{\left(2\lambda t\right)}^{2H}}-\frac{2\mathsf{\Gamma }(H+1/2)}{\sqrt{\pi }}\frac{1}{{\left(2\lambda t\right)}^H}{K}_H\left(\lambda t\right),$$

(28)

where $t\ne 0$ and $K_{\nu }\left(z\right)$ is the modified Bessel function of the second kind (see Appendix C), then we have

$$Var\left(B_{H,\lambda }^I\left(t\right)\right)={\left(C_t^I\right)}^2{t}^{2H}.$$

Definition 10.

Given an independently scattered Gaussian random measure $W\left(dx\right)$ on $\mathbb{R}$ with control measure dx, for any $H>0$ and $\lambda >0$, the stochastic process $B_{H,\lambda }^{II}=\{B_{H,\lambda }^{II}\left(t\right),t\ge 0\}$ defined by the Wiener integral

$$B_{H,\lambda }^{II}\left(t\right):={\int }_{-\infty }^t{g}_{H,\lambda ,t}^{II}\left(x\right)W\left(dx\right),$$

where

$$\begin{array}{cc}\hfill g_{H,\lambda ,t}^{II}\left(x\right)& :={(t-x)}^{H-1/2}{e}^{-\lambda (t-x)}-{(-x)}_{+}^{H-1/2}{e}^{-\lambda {(-x)}_{+}}\hfill \\ \hfill & +\lambda {\int }_0^t{(s-x)}_{+}^{H-1/2}{e}^{\lambda {(s-x)}_{+}}ds,x\in \mathbb{R}.\hfill \end{array}$$

is called a tempered fractional Brownian motion of the second kind (TFBMII).

According to [45] (p. 7), TFBMII has the covariance function

$$Cov(B_{H,\lambda }^{II}\left(t\right),B_{H,\lambda }^{II}\left(s\right))=\frac{1}{2}\left[{\left(C_t^{II}\right)}^2{t}^{2H}+{\left(C_s^{II}\right)}^2{s}^{2H}-{\left(C_{|t-s|}^{II}\right)}^2{|t-s|}^{2H}\right]$$

for any $s,t\in \mathbb{R}$, where

$$\begin{array}{cc}\hfill {\left(C_t^{II}\right)}^2& =\frac{(1-2H)\mathsf{\Gamma }(H+1/2)\mathsf{\Gamma }\left(H\right){\left(\lambda t\right)}^{-2H}}{\sqrt{\pi }}\hfill \\ \hfill & \times \left[1-{}_2{F}_3(\{1,-1/2\},\{1-H,1/2,1\},{\lambda }^2{t}^2/4)\right]\hfill \\ \hfill & +\frac{\mathsf{\Gamma }(1-H)\mathsf{\Gamma }(H+1/2)}{\sqrt{\pi }H{2}^{2H}}{}_2{F}_3(\{1,H-1/2\},\{1,H+1,H+1/2\},{\lambda }^2{t}^2/4),\hfill \end{array}$$

(29)

and ${}_2{F}_3$ is the generalized hypergeometric function, defined in Appendix C. Therefore, the value of the correspondent variance equals

$$Var\left(B_{H,\lambda }^{II}\left(t\right)\right)={\left(C_t^{II}\right)}^2{t}^{2H}.$$

Let us define

$${\overline{B}}_{H,\lambda }^I\left(t\right)=\frac{B_{H,\lambda }^I\left(t\right)}{C_1^I},{\overline{B}}_{H,\lambda }^{II}\left(t\right)=\frac{B_{H,\lambda }^{II}\left(t\right)}{C_1^{II}}.$$

Then,

$$Var\left[{\overline{B}}_{H,\lambda }^I\left(t\right)\right]=\frac{{\left(C_t^I\right)}^2}{{\left(C_1^I\right)}^2}{t}^{2H},Var\left[{\overline{B}}_{H,\lambda }^{II}\left(t\right)\right]=\frac{{\left(C_t^{II}\right)}^2}{{\left(C_1^{II}\right)}^2}{t}^{2H},$$

(30)

and $Var\left[{\overline{B}}_{H,\lambda }^I\left(1\right)\right]=Var\left[{\overline{B}}_{H,\lambda }^{II}\left(1\right)\right]=1$.

According to Remark 1, in order to compare the entropies of ${\overline{B}}_{H,\lambda }^I\left(t\right)$ and ${\overline{B}}_{H,\lambda }^{II}\left(t\right)$, it suffices to compare their variances. By (30), this problem can be reduced to the investigation of the behavior of the ratio $C_t^I/C_t^{II}$. Namely, we need to compare its value at arbitrary point t to its value at $t=1$. Note also that the dependence of $C_t^I$ and $C_t^{II}$ on $\lambda$ is such that $C_t^I\left(\lambda \right)=C_1^I\left(\lambda t\right)$, $C_t^{II}\left(\lambda \right)=C_1^{II}\left(\lambda t\right)$. Therefore, it suffices to study the ratio $C_1^I/C_1^{II}$ as a function of $\lambda$. As it may be seen from Figure 9, this ratio decreases in $\lambda$ for all selected values of H. Thus, our numerical study leads to the following conjecture:

$$Var{\overline{B}}_{H,\lambda }^I\left(t\right)>Var{\overline{B}}_{H,\lambda }^{II}\left(t\right)\mathrm{if}t<1$$

and

$$Var{\overline{B}}_{H,\lambda }^I\left(t\right)<Var{\overline{B}}_{H,\lambda }^{II}\left(t\right)\mathrm{if}t>1.$$

The analytical proof of this result is challenging due to the complexity of expressions (28) and (29).

Remark 10.

For the reader’s convenience, the MATLAB scripts for the figures are published as Supplementary Material.

4. Conclusions

We examined five distinct entropy measures applied to the Gaussian distribution: Shannon entropy, Rényi entropy, generalized Rényi entropy, Tsallis entropy and Sharma–Mittal entropy. We investigated their interrelationships and analyzed their properties in terms of their dependence on specific parameters. Furthermore, our study extends to fractional Gaussian processes, encompassing fractional Brownian motion, subfractional Brownian motion, bifractional Brownian motion, multifractional Brownian motion and tempered fractional Brownian motion. We conducted a comparative analysis of the entropies associated with the one-dimensional distributions of these processes.

Entropy measures find widespread application in the analysis of fractional processes across various domains, such as signal processing, finance, climate science and image analysis. Fractional processes serve as essential models for capturing long-range dependence and self-similarity in diverse data types. Entropy plays a crucial role in quantifying the complexity and information content of signals generated by fractional processes, which proves invaluable for tasks like prediction, risk assessment and anomaly detection. In the realm of finance, entropy is employed to assess the information content and predictability of asset prices.

Our research opens up possibilities for future extensions in several directions. Potential avenues for further investigation include exploring various entropy measures for non-Gaussian processes, nonstationary processes and processes with nonstationary increments. Additionally, we can delve into the solutions of stochastic differential equations that describe the interactions of particle systems within random environments.