On the Entropy and the Maximum Entropy Principle of Uncertain Variables

Abstract

A new variance formula is developed using the generalized inverse of an increasing function. Based on the variance formula, a new entropy formula for any uncertain variable is provided. Most of the entropy formulas in the literature are special cases of the new entropy formula. Using the new entropy formula, the maximum entropy distribution for unimodel entropy of uncertain variables is provided without using the Euler–Lagrange equation.

1. Introduction

The concept of entropy was first introduced by Rudolf Clausius in the mid-19th century as a measure of the disorder or randomness in a system. In 1948, Shannon [1] inaugurated information entropy in information theory, which measures the unpredictability of the realization of a random variable. Subsequently, information entropy developed into an essential tool in information theory (see Gray [2]). Entropy was introduced into dynamical systems by Kolmogorov [3] in 1958 to investigate the conjugacy problems of the Bernoulli shift. In 2009, Liu [4] proposed logarithmic entropy in uncertainty theory to quantify the difficulty in forecasting the outcome of an uncertain variable. In 2012, Knuth et al. [5] gave an interesting application of maximal joint entropy in automated learning machines. Currently, entropy is widely used in various fields, such as probability theory, ergodic theory, dynamical systems, machine learning and data analysis, uncertainty theory, and others. We refer the readers to Martin and England [6], Walters [7], Einsiedler and Thomas [8], and Liu [9] for a detailed introduction to the entropy theory in these fields.

The maximum entropy principle was first expounded by Jaynes [10,11], where he gave a natural criterion of choice, i.e., from the set of all probability distributions compatible with several mean values of several random variables, choose the one that maximizes Shannon entropy. From then on, the methods of maximum entropy became widely applied tools for constructing the probability distribution in statistical inference [12], decision-making [13,14], communication theory [15], time-series analysis [16], and reliability theory [17,18]. In 2017, Abbas et al. [19] made an elaborate comparison between the methods of maximum entropy and the maximum log-probability from the point of the Kullback–Leibler divergence. In this paper, we will discuss the entropy and the maximum entropy principle in uncertainty theory.

Uncertainty theory, founded by Liu [20] in 2007 and refined in 2015 [9], is a branch of axiomatic mathematics that deals with human’s belief degree based on normality, duality, sub-additivity, and product axioms. Within the past few years, it has attracted numerous researchers. Peng and Iwamura [21], Liu and Lio [22] provided a sufficient and necessary characterization of an uncertainty distribution. Based on the inverse uncertainty distribution, Yao [23] introduced a new approach to computing the variance. Subsequently, Sheng and Kar [24] extended Yao’s result to the moments of the uncertain variable.

After the introduction of entropy in uncertainty theory, the entropy and maximum entropy principle were researched by numerous experts. Inspired by the maximum entropy principles in information theory and statistics, Chen and Dai [25] explored the maximum entropy principle of logarithmic entropy. According to the maximum entropy principle, using the Lagrange multipliers method, Chen and Dai [25] proved that normal uncertainty distribution is the maximal entropy distribution under the prescribed constraints on the expected value and variance. In addition, using Fubini’s theorem, Dai and Chen [26] built an entropy formula for regular uncertain variables (see Definition 2). Subsequently, there have been many extensions of entropy, the entropy formula, and the corresponding maximum entropy distribution. Yao et al. [27] proposed sine entropy. Yao et al. [27] provided an entropy formula for regular uncertain variables, and the authors also obtained the maximum entropy distribution for sine entropy. Tang and Gao [28] proposed triangular entropy and provided an entropy formula for regular uncertain variables. Ning et al. [29] applied triangular entropy to portfolio selection. Subsequently, Dai [30] suggested quadratic entropy and provided an entropy formula for regular uncertain variables. In addition, Dai [30] found the maximum entropy distribution for quadratic entropy. Here, we point out, all the entropy formulas in Dai and Chen [26], Yao et al. [27], Tang and Gao [28], and Dai [30] are based on Fubini’s theorem. Subsequently, Ma [31] proposed unimodal entropy of uncertain variables and provided an entropy formula for regular uncertain variables. Ma [31] also found the maximum entropy distribution for unimodal entropy for regular uncertain variables. Nevertheless, limited by their methods, all the entropy formulas in Dai and Chen [26], Yao et al. [27], Tang and Gao [28], Dai [30], and Ma [31] are only established for regular uncertain variables. We know most uncertain variables are not regular. Therefore, it has some significance not only in theory but also in practice to prove the entropy formulas of Dai and Chen [26], Yao et al. [27], Tang and Gao [28], Dai [30], and Ma [31] are still true for any uncertain variable. In this paper, we will provide an entropy formula that is true for any uncertain variable by using the generalized inverse of an increasing function. We emphasize that the entropy formulas of Dai and Chen [26], Yao et al. [27], Tang and Gao [28], Dai [30], and Ma [31] are special cases of our entropy formula, and our method differs completely from those methods used by Dai and Chen [26], Yao et al. [27], Dai [30], and Ma [31].

Furthermore, we point out that all the maximum entropy distributions of Chen and Dai [25], Yao et al. [27], and Dai [30] were obtained based on the Lagrange multipliers method. Using our entropy formula, we will obtain the maximum entropy distribution without utilizing the Lagrange multipliers method. We also emphasize that the maximum entropy distributions of Chen and Dai [25], Yao et al. [27], and Dai [30] are special cases of ours.

The rest of the paper is arranged as follows. In Section 2, we first collect some necessary facts in uncertainty theory and some requisite results of the generalized inverse of an increasing function. Then, we review the entropy, entropy formulas, and the maximum entropy principles in uncertainty theory. In Section 3, we present our entropy formula and our maximum entropy distribution. We also demonstrate some applications of our entropy formula and maximum entropy distribution. In Section 4, we prove our entropy formula and derive the maximum entropy distribution of unimodel entropy. In Section 5, we verify that the maximum entropy distributions of logarithmic entropy in Theorem 2, sine entropy in Theorem 5, and quadratic entropy in Theorem 8 are special cases of our maximum entropy distributions. Finally, a concise discussion and further research are given in Section 6.

2. Preliminaries

In this section, first, we will collect the necessary facts in uncertainty theory. Second, we will recall some requisite results of the generalized inverse of an increasing function. Finally, we review the entropy, entropy formulas, and the maximum entropy principles in uncertainty theory.

2.1. Some Primary Facts in Uncertainty Theory

Let $\Gamma$ be a non-empty set and suppose $\mathcal{L}$ is a $\sigma$-algebra over $\Gamma$. Each element A in $\mathcal{L}$ is called an event. To handle belief degree reasonably, Liu [20] introduced the uncertain measure.

Definition 1.

A function $\mathcal{M}:\mathcal{L}\to [0,1]$ is called an uncertain measure provided $\mathcal{M}$ satisfies the following three axioms:

1.

Normality: $\mathcal{M}\{\Gamma \}=1$;

2.

Duality: $\mathcal{M}\left\{A\right\}+\mathcal{M}\left\{A^c\right\}=1$ for each event A;

3.

Subadditivity: for each sequence of events $A_1,A_2,\dots$, one gets

$$\mathcal{M}\left\{\bigcup _{i=1}^{\infty }{A}_i\right\}\le \sum _{i=1}^{\infty }\mathcal{M}\left\{A_i\right\}.$$

The triplet $(\Gamma ,\mathcal{L},\mathcal{M})$ is called an uncertainty space. A measurable function $X:(\Gamma ,\mathcal{L},\mathcal{M})\to \mathbb{R}$ is called an uncertain variable. Let X be an uncertain variable; the function $\Phi \left(x\right)=\mathcal{M}\{X\le x\}$ is called the uncertainty distribution of X.

Definition 2

([32]). An uncertainty distribution $\Phi$ is called regular provided it is strictly increasing and continuous over $\{x:0<\Phi (x)<1\}$, and

$$\underset{x\to -\infty }{lim}\Phi \left(x\right)=0,\underset{x\to +\infty }{lim}\Phi \left(x\right)=1.$$

An uncertain variable is called regular if its uncertainty distribution is regular.

Obviously, a regular uncertainty distribution $\Phi$ is invertible. Now, we recall two regular uncertainty distributions that are most used in uncertainty theory. The first is the normal uncertainty distribution $\mathcal{N}(e,\sigma )$ [32], which is defined by

$$\Phi \left(x\right)={\left(1+exp\left(\frac{\pi (e-x)}{\sqrt{3}\sigma }\right)\right)}^{-1},x\in \mathbb{R}.$$

(1)

Here, e and $\sigma$ are real numbers with $\sigma >0$. We call an uncertain variable X normal provided its uncertainty distribution is normal and write $X\sim \mathcal{N}(e,\sigma )$ (Figure 1).

Figure 1. Normal uncertainty distribution $\mathcal{N}(e,\sigma )$.

The second is the linear uncertainty distribution $\mathcal{L}(a,b)$ [32], which is defined by

$$\Phi \left(x\right)=\left\{\begin{array}{cc}0,& x\le a\hfill \\ {\displaystyle \frac{x-a}{b-a},}& 0<x\le b\hfill \\ 1,& b<x\hfill \end{array}\right.$$

where a and b are real numbers with $a<b$ (Figure 2).

Figure 2. Linear uncertainty distribution $\mathcal{L}(a,b)$.

Liu [20] suggested the expectation to describe the mean of an uncertain variable.

Definition 3

([20]). Let X be an uncertain variable. The expectation of X is defined by

$$E\left[X\right]={\int }_0^{\infty }\mathcal{M}\{X\ge x\}dx-{\int }_{-\infty }^0\mathcal{M}\{X\le x\}dx$$

if one or two of the integrals above are finite.

Let X be an uncertain variable with finite expectation e; the variance of X is defined by

$$V\left[X\right]=E\left[{(X-e)}^2\right].$$

Given an uncertainty variable X with uncertainty distribution $\Phi$ and finite expectation e, the variance of X can be computed by (see Section 2.6 of Chapter 2 in [9])

$$V\left[X\right]={\int }_0^{+\infty }(1-\Phi (e+\sqrt{x})+\Phi (e-\sqrt{x}))dx.$$

(2)

Yao [23] provided a nice formula for calculating the variance by inverse uncertainty distribution.

Theorem 1

(Yao [23]). Assume that X is a regular uncertain variable with uncertainty distribution $\Phi$. If $E\left[X\right]$ is finite, then

$$V\left[X\right]={\int }_0^1{({\Phi }^{-1}\left(\alpha \right)-E\left[X\right])}^{2}d\alpha .$$

We will prove the variance formula of Yao is true for any uncertain variable in Theorem 9, which is one of our main tools for establishing our entropy formula and maximum entropy principle.

2.2. Necessary Results of the Generalized Inverse of an Increasing Function

To present our results, we need to review the generalized inverse of an increasing function, which is a basic tool in statistics; see Embrechts and Hofert [33], Resnick [34], Kampke-Radermacher [35]. Here, we follow the definition of Embrechts and Hofert [33].

Definition 4

([33]). Let $\Phi :\mathbb{R}\to [0,1]$ be an increasing function. The generalized inverse of $\Phi$ is defined by

$${\Phi }^{-}\left(\alpha \right)=inf\{x\in \mathbb{R}:\Phi \left(x\right)\ge \alpha \},\alpha \in (0,1],$$

(3)

with the convention that $inf\{\varnothing \}=+\infty$, where ∅ is the empty set.

Specifically, we define ${\Phi }^{-}\left(0\right)=sup\{x\in \mathbb{R}:\Phi \left(x\right)=0\}$ with the convention that $sup\{\varnothing \}=-\infty$.

Remark 1.

To stress the difference between the generalized inverse and standard inverse, we denote the generalized inverse of an increasing function $\Phi$ by ${\Phi }^{-}$. If $\Phi$ is strictly increasing and continuous, ${\Phi }^{-}$ is just the standard inverse of $\Phi$.

We also point out that, by the above definition, ${\Phi }^{-}\left(0\right)$ may be $-\infty$ and ${\Phi }^{-}\left(1\right)$ may be +∞. Here is an example used frequently in uncertainty theory.

Example 1.

Let $\Phi$ be a normal uncertainty distribution defined by (1). Then ${\Phi }^{-}$ is:

$${\Phi }^{-}\left(\alpha \right)=\left\{\begin{array}{cc}-\infty ,& \alpha =0\hfill \\ {\displaystyle e-\frac{\sqrt{3}\sigma }{\pi }ln\frac{1-\alpha }{\alpha },}& 0<\alpha <1\hfill \\ +\infty ,& \alpha =1.\hfill \end{array}\right.$$

We also need to introduce the generalized inverse of a left-continuous increasing function; see Appendix B in Iritani and Kuga [36].

Definition 5

([36]). Let G be a left-continuous and increasing function over $(0,1)$. The generalized inverse of G is defined by

$$G^{-1}\left(x\right)=sup\left\{y:G\left(y\right)\le x\right\}$$

(4)

for all real numbers x.

The succeeding interesting fact is that the generalized inverse of the generalized inverse for a right-continuous increasing function is just itself. To the best of our knowledge, this result was proved first by Iritani and Kuga [36] and was refined by Kampke-Radermacher [35].

Lemma 1

([35]). Let $\Phi :\mathbb{R}\to [0,1]$ be a right-continuous and increasing function; G is the generalized inverse of $\Phi$ defined by (3), i.e., $G={\Phi }^{-}$. Then

$$G^{-1}\left(x\right)=\Phi \left(x\right)$$

for all $x\in \mathbb{R}$, where $G^{-1}\left(x\right)$ is the generalized inverse of G defined by (4).

2.3. Entropy Formula and the Principle of Maximum Entropy

To quantify the difficulty in predicting the outcomes of an uncertain variable, Liu [4] proposed logarithm entropy.

Definition 6

([4]). Let X be an uncertain variable with uncertainty distribution $\Phi$. The logarithm entropy of X is defined by

$$H_L\left[X\right]={\int }_{-\infty }^{\infty }L\left(\Phi \left(x\right)\right)dx.$$

(5)

Here, $L\left(y\right)=-ylny-(1-y)ln(1-y)$ for $y\in (0,1)$ and $L\left(0\right)=L\left(1\right)=0$.

Using the Lagrange multipliers method, Chen and Dai [25] obtained the maximum entropy distribution for the logarithmic entropy.

Theorem 2

([25]). Suppose X is an uncertain variable with finite expectation e and variance ${\sigma }^2$. Then,

$$H_L\left[X\right]\le \frac{\sigma \pi }{\sqrt{3}},$$

(6)

and the equality holds if and only if X is a normal uncertain variable $\mathcal{N}(e,\sigma )$.

In addition, using Fubini’s theorem, Dai and Chen [26] attained an interesting entropy formula for regular uncertain variables.

Theorem 3

([26]). Suppose X is a regular uncertain variable with uncertainty distribution $\Phi$. If $H_L\left[X\right]$ is finite, we have

$$H_L\left[X\right]={\int }_0^1{\Phi }^{-1}\left(\alpha \right)ln\frac{\alpha }{1-\alpha }d\alpha .$$

(7)

Subsequently, Yao et al. [27] proposed sine entropy.

Definition 7

([27]). Let X be an uncertain variable with uncertainty distribution $\Phi$. The sine entropy of X is defined by

$$H_S\left[X\right]={\int }_{-\infty }^{\infty }S\left(\Phi \left(x\right)\right)dx.$$

(8)

Here, $S\left(y\right)=sin\left(\pi y\right)$ for $y\in [0,1]$.

Yao et al. [27] provided an entropy formula and obtained the maximum entropy distribution of sine entropy.

Theorem 4

([27]). Suppose X is a regular uncertain variable with uncertainty distribution $\Phi$. Then, we have

$$H_S\left[X\right]=\pi {\int }_0^1{\Phi }^{-1}(1-\alpha )cos\left(\pi \alpha \right)d\alpha .$$

(9)

Theorem 5

([27]). Suppose X is an uncertain variable with uncertainty distribution $\Phi$, expectation e, and variance ${\sigma }^2$. Then, we have

$$H_S\left[X\right]\le \frac{\sqrt{2}}{2}\pi \sigma$$

(10)

and equality holds if and only if

$${\Phi }^{-1}\left(\alpha \right)=e-\sigma \sqrt{2}cos\left(\pi \alpha \right),\alpha \in (0,1).$$

Tang and Gao [28] suggested triangular entropy and found an entropy formula.

Definition 8

([28]). Let X be an uncertain variable with uncertainty distribution $\Phi$. The triangular entropy of X is defined by

$$H_T\left[X\right]={\int }_{-\infty }^{\infty }T\left(\Phi \left(x\right)\right)dx,$$

(11)

where

$$T\left(y\right)=\left\{\begin{array}{cc}y,& 0\le y\le 0.5,\hfill \\ 1-y,& 0.5\le y\le 1.\hfill \end{array}\right.$$

Theorem 6

([28]). Suppose X is a regular uncertain variable with uncertainty distribution $\Phi$. Then, we have

$$H_T\left[X\right]=-{\int }_0^{0.5}{\Phi }^{-1}\left(\alpha \right)d\alpha +{\int }_{0.5}^1{\Phi }^{-1}\left(\alpha \right)d\alpha .$$

Dai [30] put forward quadratic entropy.

Definition 9

([30]). Let X be an uncertain variable with uncertainty distribution $\Phi$. The quadratic entropy of X is defined by

$$H_Q\left[X\right]={\int }_{-\infty }^{\infty }Q\left(\Phi \left(x\right)\right)dx.$$

(12)

Here, $Q\left(y\right)=y(1-y)$ for $y\in [0,1]$.

Dai [30] also provided an entropy formula and got the maximum entropy distribution of quadratic entropy.

Theorem 7

([30]). Suppose X is a regular uncertain variable with uncertainty distribution $\Phi$. If the entropy exists, then

$$H_Q\left[X\right]={\int }_0^1{\Phi }^{-1}\left(\alpha \right)(2\alpha -1)d\alpha .$$

(13)

Theorem 8

([30]). Suppose X is an uncertain variable with uncertainty distribution $\Phi$, expectation e, and variance ${\sigma }^2$. Then, we have

$$H_Q\left[X\right]\le \frac{\sigma }{\sqrt{3}}$$

(14)

and equality holds if and only if

$${\Phi }^{-1}\left(\alpha \right)=e+\sigma \sqrt{3}(2\alpha -1),\alpha \in (0,1).$$

Ma [31] suggested unimodal entropy.

Definition 10

([31]). Let X be an uncertain variable with uncertainty distribution $\Phi$. The unimodal entropy of X is defined by

$$H_U\left[X\right]={\int }_{-\infty }^{\infty }U\left(\Phi \left(x\right)\right)dx,$$

(15)

where function U is increasing on $(0,0.5]$ and decreasing on $[0.5,1)$, $U\left(0\right)=U\left(1\right)=0$.

Ma [31] also provided an entropy formula and got the maximum entropy distribution of the unimodal entropy for regular uncertain variables. The readers can refer to Liu [9], Yao et al. [27], Tang and Gao [28], Dai [30], Ma [31], and the references therein for details.

Here, we emphasize the entropy formulas of Dai and Chen [26], Yao et al. [27], Tang and Gao [28], Dai [30], and Ma [31] hold only for regular uncertain variables. We will develop an entropy formula that is true for any uncertain variable. To establish the entropy formulas, the method used by Dai and Chen [26], Yao et al. [27], Tang and Gao [28], and Dai [30] is Fubini’s theorem. We stress that Fubini’s theorem does not work in our case because the uncertainty distribution $\Phi$ may not be regular here. We need to develop new methods.

Finally, we point out, to obtain the maximum entropy distribution, the main method used by Chen and Dai [25], Yao et al. [27], and Dai [30] is the Lagrange multipliers method. Based on our entropy formula, we can obtain the maximum entropy distribution without using the Lagrange multipliers method. We also stress that the only tool we need is the Cauchy–Schwartz inequality. In the following section, we will exhibit our new entropy formula and provide the maximum entropy distribution of unimodel entropy.

3. Results

In this section, we will present our results and give some remarks. First, we point out that the variance formula of Yao [23] for regular uncertain variables is valid for any uncertain variable.

Theorem 9.

Let X be an uncertain variable with uncertainty distribution $\Phi$, finite expectation e, and finite variance. Then

$$V\left[X\right]={\int }_0^1{({\Phi }^{-}\left(\alpha \right)-e)}^{2}d\alpha ,$$

where ${\Phi }^{-}$ is the generalized inverse of $\Phi$.

We leave the verification of Theorem 9 to the next section. Based on our variance formula and some facts on the generalized inverse, we will develop an entropy formula for any uncertain variable.

Theorem 10.

Let X be an uncertain variable with finite expectation and finite variance. Assume further that unimodal function U is differentiable and satisfies

$${\int }_0^1{\left|U^{\prime }\left(\alpha \right)\right|}^{2}d\alpha <\infty .$$

(16)

Then we have

$$H_U\left[X\right]=-{\int }_0^1{\Phi }^{-}\left(\alpha \right)U^{{}^{\prime }}\left(\alpha \right)d\alpha ,$$

(17)

where ${\Phi }^{-}$ is the generalized inverse of the uncertainty distribution $\Phi$ of X.

Remark 2.

1.

The entropy formulas in Dai and Chen [26], Yao et al. [27], Tang and Gao [28], Dai [30], and Ma [31] are valid only for regular uncertain variables. Our entropy formula holds for any uncertain variable.

2.

Fubini’s theorem, which is used by Dai and Chen [26], Yao et al. [27], Tang and Gao [28], and Dai [30] to establish the entropy formula there, does not work in our setting, because the uncertainty distribution is not necessarily regular here.

We specifically point out, under the assumption that the uncertainty distribution $\Phi$ is regular, let $U\left(\alpha \right)=-\alpha ln\alpha -(1-\alpha )ln(1-\alpha )$, $U\left(\alpha \right)=sin\left(\pi t\right)$, $U\left(\alpha \right)=\alpha (1-\alpha )$,

$$U\left(\alpha \right)=\left\{\begin{array}{cc}\alpha ,& 0\le \alpha \le 0.5,\hfill \\ 1-\alpha ,& 0.5\le \alpha \le 1\hfill \end{array}\right.$$

respectively. By Theorem 10, we can regain the entropy formulas of logarithm entropy in Theorem 3, sine entropy in Theorem 4, triangular entropy in Theorem 6, and quadratic entropy in Theorem 7. As the verifications are routine, we omit them.

Using our entropy formula in Theorem 10, we obtain the maximum entropy distribution of unimodal entropy.

Theorem 11.

Let X be an uncertain variable with uncertainty distribution $\Phi$, finite expectation e, and finite variance ${\sigma }^2$. Assume further that unimodal function U is differentiable and satisfies

$${\int }_0^1{\left|U^{\prime }\left(\alpha \right)\right|}^{2}d\alpha <\infty .$$

(18)

Then, the unimodal entropy of X satisfies

$$H_U\left[X\right]\le \sigma \sqrt{{\int }_0^1{\left|U^{\prime }\left(\alpha \right)\right|}^{2}d\alpha }$$

(19)

and the equality holds if and only if

$${\Phi }^{-1}\left(\alpha \right)=e-c{U}^{\prime }\left(\alpha \right),\alpha \in (0,1)$$

(20)

where c is a real number such that

$$\left|c\right|\sqrt{{\int }_0^1{\left|U^{\prime }\left(\alpha \right)\right|}^{2}d\alpha }=\sigma .$$

(21)

Remark 3.

We stress that the maximum entropy distribution of the unimodal entropy of Ma [31] is provided only for regular uncertain variables. Here, our maximum entropy distribution of unimodal entropy in Theorem 11 is obtained for any uncertain variable.

As applications of our maximum entropy distribution in Theorem 11, we can deduce the maximum entropy distribution of logarithm entropy in Theorem 2, sine entropy in Theorem 5, and quadratic entropy in Theorem 8 immediately.

Proposition 1

(Chen and Dai [25]). Let X be an uncertain variable with uncertainty distribution $\Phi$, finite expectation e, and variance ${\sigma }^2$. Then, the logarithmic entropy of X satisfies

$$H_L\left[X\right]\le \frac{\pi \sigma }{\sqrt{3}}$$

and the equality holds if and only if

$${\Phi }^{-1}\left(\alpha \right)=e+\frac{\sigma \sqrt{3}}{\pi }ln\frac{\alpha }{1-\alpha },\alpha \in (0,1),$$

i.e., X is a normal uncertain variable $\mathcal{N}(e,\sigma )$.

Proposition 2

(Yao et al. [27]). Let X be an uncertain variable with uncertainty distribution $\Phi$, finite expectation e, and variance ${\sigma }^2$. Then, its sine entropy satisfies

$$H_S\left[X\right]\le \frac{\pi \sigma }{\sqrt{2}}$$

and the equality holds if and only if

$${\Phi }^{-1}\left(\alpha \right)=e-\sigma \sqrt{2}cos\left(\pi \alpha \right),\alpha \in (0,1).$$

Proposition 3

(Dai [30]). Let X be an uncertain variable with uncertainty distribution $\Phi$, finite expectation e, and variance ${\sigma }^2$. Then, its quadratic entropy satisfies

$$H_Q\left[X\right]\le \frac{\sigma }{\sqrt{3}}$$

and the equality holds if and only if

$${\Phi }^{-1}\left(\alpha \right)=e+\sigma \sqrt{3}(2\alpha -1),\alpha \in (0,1).$$

We leave the proofs of Corollary 1, Corollary 2, and Corollary 3 to Section 6. In the following section, we will prove Theorem 9, Theorem 10, and Theorem 11.

4. Proof of Theorem 9, Theorem 10, and Theorem 11

To prove Theorems 9 and 10, we need a result of the substitution rule on Lebesgue–Stieltjes integrals, which is a conclusion of Proposition 2 of Falkner and Teschl [37].

Lemma 2

([37]). Assume function M is increasing over $[a,b]$ and function N is left-continuous increasing over $\left[M\right(a),M(b\left)\right]$. Then, for each bounded Borel function f over $[a,b]$, one has

$${\int }_a^{b}f\left(x\right)dN\left(M\left(x\right)\right)={\int }_{M\left(a\right)}^{M\left(b\right)}f\left(M^{-1}\left(y\right)\right)dN\left(y\right),$$

(22)

where $M^{-1}$ is the generalized inverse of M defined by (4).

Here, we point out that if $N\left(y\right)=y$, this result goes back to the classical case of Lebesgue [38]. Before proving the main results, we also need a simple fact.

Lemma 3.

Suppose function g is increasing on $(0,1)$ with

$$\underset{x\to 0^{+}}{lim}g\left(x\right)=-\infty and\underset{x\to 1^{-}}{lim}g\left(x\right)=+\infty ,$$

and let L be a function increasing on $(0,a]$ and decreasing on $[a,1)$, respectively, for some a in $(0,1)$. Then

(a)

$|{\int }_0^{a}g\left(x\right)dL\left(x\right)|<+\infty$ means

$$\underset{A\to 0^{+}}{lim}g\left(A\right)(L\left(A\right)-L\left(0\right))=0;$$

(b)

$|{\int }_a^{1}g\left(x\right)dL\left(x\right)|<+\infty$ means

$$\underset{B\to 1^{-}}{lim}g\left(B\right)(L\left(1\right)-L\left(B\right))=0.$$

Because the verification of Lemma 3 is a routine computation, we omit it.

We also point out that, notwithstanding that uncertainty distribution $\Phi$ may not be right-continuous, since $\Phi$ is increasing, the assumption that $\Phi$ is right-continuous does not change the variance and the entropy of an uncertain variable. Thus, we assume uncertainty distribution $\Phi$ is right-continuous in the rest. Now, we will prove Theorem 9.

Proof of Theorem 9.

Let $Y=X-e$ and $\Psi$ be the uncertainty distribution of Y. Then we have

$$E\left[Y\right]=0,V\left[X\right]=V\left[Y\right],\mathrm{and}{\Psi }^{-}={\Phi }^{-}-e.$$

We only need to prove

$$V\left[Y\right]={\int }_0^1{\left({\Psi }^{-}\left(\alpha \right)\right)}^{2}d\alpha .$$

Recall (see equality (2))

$$V\left[Y\right]={\int }_0^{\infty }(1-\Psi \left(\sqrt{x}\right))dx+{\int }_0^{\infty }\Psi (-\sqrt{x})dx.$$

We only verify

$${\int }_0^{\infty }(1-\Psi \left(\sqrt{x}\right))dx={\int }_{\Psi \left(0\right)}^1{\left({\Psi }^{-}\left(\alpha \right)\right)}^{2}d\alpha .$$

The verification of

$${\int }_0^{\infty }\Psi (-\sqrt{x})dx={\int }_0^{\Psi \left(0\right)}{\left({\Psi }^{-}\left(\alpha \right)\right)}^{2}d\alpha$$

is similar. First, note that

$${\int }_0^{\infty }(1-\Psi \left(\sqrt{x}\right))dx=\underset{B\to 1^{-}}{lim}{\int }_0^{{\Psi }^{-}\left(B\right)}(1-\Psi \left(\sqrt{x}\right))dx.$$

Letting $y=\sqrt{x}$, we have

$$\begin{array}{cc}\hfill {\int }_0^{{\Psi }^{-}\left(B\right)}(1-\Psi \left(\sqrt{x}\right))dx& ={\int }_0^{{\Psi }^{-}\left(B\right)}(1-\Psi \left(y\right))d{y}^2\hfill \\ \hfill & ={\int }_0^{{\Psi }^{-}(\Psi \left(0\right))}(1-\Psi \left(y\right))d{y}^2+{\int }_{{\Psi }^{-}(\Psi \left(0\right))}^{{\Psi }^{-}\left(B\right)}(1-\Psi \left(y\right))d{y}^2\hfill \\ \hfill & =(1-\Psi \left(0\right)){\left({\Psi }^{-}(\Psi \left(0\right))\right)}^2+{\int }_{{\Psi }^{-}(\Psi \left(0\right))}^{{\Psi }^{-}\left(B\right)}(1-\Psi \left(y\right))d{y}^2,\hfill \end{array}$$

where the last equality is because $\Psi \left(y\right)\equiv \Psi \left(0\right)$ for $y\in \left({\Psi }^{-}(\Psi \left(0\right)),0\right)$.

By Lemma 2, let $M={\Psi }^{-}$, $f\left(x\right)=1-x$, $N\left(y\right)=y^2$, noting that $M^{-}=\Psi$ according to Lemma 1, we have

$${\int }_{{\Psi }^{-}(\Psi \left(0\right))}^{{\Psi }^{-}\left(B\right)}(1-\Psi \left(y\right))d{y}^2={\int }_{\Psi \left(0\right)}^B(1-x)d{\left({\Psi }^{-}\left(x\right)\right)}^2.$$

By integration by parts, we have

$${\int }_{\Psi \left(0\right)}^B(1-x)d{\left({\Psi }^{-}\left(x\right)\right)}^2=(1-B){\left({\Psi }^{-}\left(B\right)\right)}^2-(1-\Psi \left(0\right)){\left({\Psi }^{-}(\Psi \left(0\right))\right)}^2+{\int }_{\Psi \left(0\right)}^B{\left({\Psi }^{-}\left(x\right)\right)}^{2}dx.$$

Then, we have

$${\int }_0^{{\Psi }^{-}\left(B\right)}(1-\Psi \left(\sqrt{x}\right))dx=(1-B){\left({\Psi }^{-}\left(B\right)\right)}^2+{\int }_{\Psi \left(0\right)}^B{\left({\Psi }^{-}\left(x\right)\right)}^{2}dx.$$

Let

$$G\left(B\right)={\int }_{\Psi \left(0\right)}^B{\left({\Psi }^{-}\left(x\right)\right)}^{2}dx.$$

Note that function G is increasing over $[\Psi (0),1]$. It follows that

$$\underset{B\to 1^{-}}{lim}G\left(B\right)=\underset{n\to +\infty }{lim}G(1-\frac{1}{n}).$$

By monotone convergence theorem, we have

$$\begin{array}{cc}\hfill {\int }_{\Psi \left(0\right)}^1{\left({\Psi }^{-}\left(x\right)\right)}^{2}dx& =\underset{n\to +\infty }{lim}{\int }_{\Psi \left(0\right)}^1{\left({\Psi }^{-}\left(x\right)\right)}^2{1}_{[\Psi \left(0\right),1-\frac{1}{n}]}\left(x\right)dx\hfill \\ \hfill & =\underset{n\to +\infty }{lim}G(1-\frac{1}{n}),\hfill \end{array}$$

where $1_{[\Psi \left(0\right),1-\frac{1}{n}]}$ is indicator function of the interval $[\Psi \left(0\right),1-\frac{1}{n}]$.

Thus, we have

$$\underset{B\to 1^{-}}{lim}{\int }_{\Psi \left(0\right)}^B{\left({\Psi }^{-}\left(x\right)\right)}^{2}dx={\int }_{\Psi \left(0\right)}^1{\left({\Psi }^{-}\left(x\right)\right)}^{2}dx.$$

We need to prove

$$\underset{B\to 1^{-}}{lim}(1-B){\left({\Psi }^{-}\left(B\right)\right)}^2=0.$$

If

$$\underset{B\to 1^{-}}{lim}{\Psi }^{-}\left(B\right)<+\infty ,$$

the result is obvious. If

$$\underset{B\to 1^{-}}{lim}{\Psi }^{-}\left(B\right)=+\infty ,$$

by Lemma 3, let $g\left(x\right)={\left({\Psi }^{-}\left(x\right)\right)}^2$ and $L\left(x\right)=x$, we have

$$\underset{B\to 1^{-}}{lim}(1-B){\left({\Psi }^{-}\left(B\right)\right)}^2=0.$$

Thus, we finish the proof of Theorem 9.  □

Now, using Theorem 9, Lemma 2, and Lemma 3, we will prove Theorem 10, i.e., the entropy formula.

Proof of Theorem 10.

Recall

$$H_U\left[X\right]={\int }_{-\infty }^{\infty }U\left(\Phi \left(x\right)\right)dx.$$

Note that

$$\Phi \left(x\right)=\left\{\begin{array}{cc}0,& x\le {\Phi }^{-}\left(0\right),\hfill \\ 1,& x\ge {\Phi }^{-}\left(1\right)\hfill \end{array}\right.$$

and $U\left(0\right)=U\left(1\right)=0$. We have

$${\int }_{-\infty }^{\infty }U\left(\Phi \left(x\right)\right)dx={\int }_{{\Phi }^{-}\left(0\right)}^{{\Phi }^{-}\left(1\right)}U\left(\Phi \left(x\right)\right)dx.$$

By the monotone convergence theorem, we have

$${\int }_{{\Phi }^{-}\left(0\right)}^{{\Phi }^{-}\left(1\right)}U\left(\Phi \left(x\right)\right)dx=\underset{n\to +\infty }{lim}{\int }_{{\Phi }^{-}\left(0\right)}^{{\Phi }^{-}\left(1\right)}U\left(\Phi \left(x\right)\right)1_{[{\Phi }^{-}\left(\frac{1}{n}\right),{\Phi }^{-}(1-\frac{1}{n})]}\left(x\right)dx,$$

where

$$1_{[{\Phi }^{-}\left(\frac{1}{n}\right),{\Phi }^{-}(1-\frac{1}{n})]}\left(x\right)$$

is indicator function of the interval $[{\Phi }^{-}\left(\frac{1}{n}\right),{\Phi }^{-}(1-\frac{1}{n})]$.

Let

$$G\left(ϵ\right)=\underset{ϵ\to 0^{+}}{lim}{\int }_{{\Phi }^{-}\left(0\right)}^{{\Phi }^{-}\left(1\right)}U\left(\Phi \left(x\right)\right)1_{[{\Phi }^{-}\left(ϵ\right),{\Phi }^{-}(1-ϵ)]}\left(x\right)dx.$$

Note that $G\left(ϵ\right)$ is decreasing over $[0,1]$. We have

$$\underset{n\to +\infty }{lim}G\left(\frac{1}{n}\right)=\underset{ϵ\to 0^{+}}{lim}G\left(ϵ\right).$$

Thus

$$\begin{array}{cc}\hfill {\int }_{{\Phi }^{-}\left(0\right)}^{{\Phi }^{-}\left(1\right)}U\left(\Phi \left(x\right)\right)dx& =\underset{n\to +\infty }{lim}G\left(\frac{1}{n}\right)\hfill \\ \hfill & =\underset{ϵ\to 0^{+}}{lim}{\int }_{{\Phi }^{-}\left(ϵ\right)}^{{\Phi }^{-}(1-ϵ)}U(\Phi \left(x\right))dx.\hfill \end{array}$$

Let $f=U$, $M={\Phi }^{-}$, $N\left(x\right)=x$; by Lemma 2 and integration by parts, we have

$$\begin{array}{cc}\hfill {\int }_{{\Phi }^{-}\left(ϵ\right)}^{{\Phi }^{-}(1-ϵ)}U(\Phi \left(x\right))dx& ={\int }_{ϵ}^{1-ϵ}U\left(\alpha \right)d{\Phi }^{-}\left(\alpha \right)\hfill \\ \hfill & =U(1-ϵ){\Phi }^{-}(1-ϵ)-U\left(ϵ\right){\Phi }^{-}\left(ϵ\right)-{\int }_{ϵ}^{1-ϵ}{\Phi }^{-}\left(\alpha \right))dU\left(\alpha \right).\hfill \end{array}$$

Then, we have

$$H_U\left(X\right)=\underset{ϵ\to 0^{+}}{lim}U(1-ϵ){\Phi }^{-}(1-ϵ)-\underset{ϵ\to 0^{+}}{lim}U\left(ϵ\right){\Phi }^{-}\left(ϵ\right)-\underset{ϵ\to 0^{+}}{lim}{\int }_{ϵ}^{1-ϵ}{\Phi }^{-}\left(\alpha \right)dU\left(\alpha \right).$$

We only need to show

$$\underset{ϵ\to 0^{+}}{lim}U(1-ϵ){\Phi }^{-}(1-ϵ)=0,\underset{ϵ\to 0^{+}}{lim}U\left(ϵ\right){\Phi }^{-}\left(ϵ\right)=0$$

and

$$\underset{ϵ\to 0^{+}}{lim}{\int }_{ϵ}^{1-ϵ}{\Phi }^{-}\left(\alpha \right)dU\left(\alpha \right)={\int }_0^1{\Phi }^{-}\left(\alpha \right)U^{{}^{\prime }}\left(\alpha \right)d\alpha .$$

Noting that ${\int }_0^1{U}^{{}^{\prime }}\left(\alpha \right)d\alpha =0$, by the Cauchy–Schwartz inequality and Theorem 9, we have

$$\begin{array}{cc}\hfill \left|{\int }_0^1{\Phi }^{-}\left(\alpha \right)dU\left(\alpha \right)\right|& =\left|{\int }_0^1({\Phi }^{-}\left(\alpha \right)-e)U^{{}^{\prime }}\left(\alpha \right)d\alpha \right|\hfill \\ \hfill & \le \sqrt{{\int }_0^1{({\Phi }^{-}\left(\alpha \right)-e)}^{2}d\alpha }\sqrt{{\int }_0^1{\left(U^{{}^{\prime }}\left(\alpha \right)\right)}^{2}d\alpha }\hfill \\ \hfill & =\sigma \sqrt{{\int }_0^1{\left(U^{{}^{\prime }}\left(\alpha \right)\right)}^{2}d\alpha }<+\infty .\hfill \end{array}$$

By monotone convergence theorem, again we have

$$\underset{ϵ\to 0^{+}}{lim}{\int }_{ϵ}^{1-ϵ}{\Phi }^{-}\left(\alpha \right)dU\left(\alpha \right)={\int }_0^1{\Phi }^{-}\left(\alpha \right)U^{{}^{\prime }}\left(\alpha \right)d\alpha .$$

Let $L=U$ and $g={\Phi }^{-}$; by Lemma 3, we have

$$\underset{ϵ\to 0^{+}}{lim}U(1-ϵ){\Phi }^{-}(1-ϵ)=0\mathrm{and}\underset{ϵ\to 0^{+}}{lim}U\left(ϵ\right){\Phi }^{-}\left(ϵ\right)=0.$$

Then, we have

$$H_U\left[X\right]=-{\int }_0^1{\Phi }^{-}\left(\alpha \right)U^{{}^{\prime }}\left(\alpha \right)d\alpha .$$

Thus, we finish the proof of Theorem 10.  □

Now, based on our entropy formula, we can derive the maximum entropy distribution of unimodal entropy immediately.

Proof of Theorem 11.

Noting that ${\int }_0^1{U}^{{}^{\prime }}\left(\alpha \right)d\alpha =0$, by Theorem 10, we have

$$H_U\left[X\right]=-{\int }_0^1({\Phi }^{-}\left(\alpha \right)-e)U^{{}^{\prime }}\left(\alpha \right)d\alpha .$$

By the Cauchy–Schwartz inequality and Theorem 9, we have

$$\begin{array}{cc}\hfill H_U\left[X\right]& \le \sqrt{{\int }_0^1{({\Phi }^{-}\left(\alpha \right)-e)}^{2}d\alpha }\sqrt{{\int }_0^1{\left(U^{{}^{\prime }}\left(\alpha \right)\right)}^{2}d\alpha }\hfill \\ \hfill & =\sigma \sqrt{{\int }_0^1{\left(U^{{}^{\prime }}\left(\alpha \right)\right)}^{2}d\alpha },\hfill \end{array}$$

and “=” holds if and only if there exists some real number c such that

$${\Phi }^{-}\left(\alpha \right)-e=c{U}^{{}^{\prime }}\left(\alpha \right).$$

Then, we have

$${\int }_0^1{({\Phi }^{-}\left(\alpha \right)-e)}^{2}d\alpha =c^2{\int }_0^1{\left(U^{{}^{\prime }}\left(\alpha \right)\right)}^{2}d\alpha ,$$

i.e.,

$$\left|c\right|\sqrt{{\int }_0^1{\left|U^{\prime }\left(\alpha \right)\right|}^{2}d\alpha }=\sigma .$$

Thus, Theorem 11 is proved.  □

5. Proof of Proposition 1, Proposition 2, and Proposition 3

In this section, we use our maximum entropy distribution to deduce Proposition 1, Proposition 2, and Proposition 3, i.e., the maximum entropy distributions of logarithm entropy, sine entropy, and quadratic entropy. As the proofs of Proposition 2 and Proposition 3 are similar to that of Proposition 1, we only prove Proposition 1.

Proof of Proposition 1.

Let

$$U\left(\alpha \right)=-\alpha ln\alpha -(1-\alpha )ln(1-\alpha ),\alpha \in (0,1).$$

Then,

$$U^{{}^{\prime }}\left(\alpha \right)=ln\frac{1-\alpha }{\alpha }.$$

By Theorem 2 of Ma et al. [39], we have

$${\int }_0^1{\left(U^{{}^{\prime }}\left(\alpha \right)\right)}^{2}d\alpha =\frac{{\pi }^2}{3}.$$

By Theorem 11, we have

$$H_L\left[X\right]\le \frac{\pi \sigma }{\sqrt{3}}$$

and the equality holds if and only if

$${\Phi }^{-1}\left(\alpha \right)-e=cln\frac{\alpha }{1-\alpha },\alpha \in (0,1).$$

(23)

Since functions

$${\Phi }^{-1}\left(\alpha \right)\mathrm{and}ln\frac{\alpha }{1-\alpha }$$

are increasing for $\alpha \in (0,1)$, the real number c is positive. By equality (23), we have

$${\int }_0^1{({\Phi }^{-1}\left(\alpha \right)-e)}^{2}d\alpha =c^2{\int }_0^1{(ln\frac{\alpha }{1-\alpha })}^{2}d\alpha .$$

Thus

$$c=\frac{\sigma \sqrt{3}}{\pi }.$$

Then

$${\Phi }^{-1}\left(\alpha \right)=e+\frac{\sigma \sqrt{3}}{\pi }ln\frac{\alpha }{1-\alpha },\alpha \in (0,1),$$

i.e., $\Phi$ is a normal uncertainty distribution with expectation e and variance ${\sigma }^2$.  □

6. Discussion

In this paper, by using the generalized inverse of an increasing function, we developed a useful entropy formula that is valid for any uncertain variable and encompasses all the previous cases in the literature. In addition, based on our entropy formula, we obtained the maximum entropy distribution for unimodel entropy. We also point out that most of the entropy formulas and maximum entropy distributions of the uncertain variables in the literature are special cases of ours.

Further research can consider the following two directions:

(1)

Develop the entropy formula and the maximum entropy principle of an uncertain set;

(2)

Establish the entropy formula and research the maximum entropy distributions of uncertain random variables.