On the Entropy and the Maximum Entropy Principle of Uncertain Variables

A new variance formula is developed using the generalized inverse of an increasing function. Based on the variance formula, a new entropy formula for any uncertain variable is provided. Most of the entropy formulas in the literature are special cases of the new entropy formula. Using the new entropy formula, the maximum entropy distribution for unimodel entropy of uncertain variables is provided without using the Euler–Lagrange equation.


Introduction
The concept of entropy was first introduced by Rudolf Clausius in the mid-19th century as a measure of the disorder or randomness in a system. In 1948, Shannon [1] inaugurated information entropy in information theory, which measures the unpredictability of the realization of a random variable. Subsequently, information entropy developed into an essential tool in information theory (see Gray [2]). Entropy was introduced into dynamical systems by Kolmogorov [3] in 1958 to investigate the conjugacy problems of the Bernoulli shift. In 2009, Liu [4] proposed logarithmic entropy in uncertainty theory to quantify the difficulty in forecasting the outcome of an uncertain variable. In 2012, Knuth et al. [5] gave an interesting application of maximal joint entropy in automated learning machines. Currently, entropy is widely used in various fields, such as probability theory, ergodic theory, dynamical systems, machine learning and data analysis, uncertainty theory, and others. We refer the readers to Martin and England [6], Walters [7], Einsiedler and Thomas [8], and Liu [9] for a detailed introduction to the entropy theory in these fields.
The maximum entropy principle was first expounded by Jaynes [10,11], where he gave a natural criterion of choice, i.e., from the set of all probability distributions compatible with several mean values of several random variables, choose the one that maximizes Shannon entropy. From then on, the methods of maximum entropy became widely applied tools for constructing the probability distribution in statistical inference [12], decision-making [13,14], communication theory [15], time-series analysis [16], and reliability theory [17,18]. In 2017, Abbas et al. [19] made an elaborate comparison between the methods of maximum entropy and the maximum log-probability from the point of the Kullback-Leibler divergence. In this paper, we will discuss the entropy and the maximum entropy principle in uncertainty theory.
Uncertainty theory, founded by Liu [20] in 2007 and refined in 2015 [9], is a branch of axiomatic mathematics that deals with human's belief degree based on normality, duality, sub-additivity, and product axioms. Within the past few years, it has attracted numerous researchers. Peng and Iwamura [21], Liu and Lio [22] provided a sufficient and necessary characterization of an uncertainty distribution. Based on the inverse uncertainty distribution, Yao [23] introduced a new approach to computing the variance. Subsequently, Sheng and Kar [24] extended Yao's result to the moments of the uncertain variable.
After the introduction of entropy in uncertainty theory, the entropy and maximum entropy principle were researched by numerous experts. Inspired by the maximum entropy principles in information theory and statistics, Chen and Dai [25] explored the maximum entropy principle of logarithmic entropy. According to the maximum entropy principle, using the Lagrange multipliers method, Chen and Dai [25] proved that normal uncertainty distribution is the maximal entropy distribution under the prescribed constraints on the expected value and variance. In addition, using Fubini's theorem, Dai and Chen [26] built an entropy formula for regular uncertain variables (see Definition 2). Subsequently, there have been many extensions of entropy, the entropy formula, and the corresponding maximum entropy distribution. Yao et al. [27] proposed sine entropy. Yao et al. [27] provided an entropy formula for regular uncertain variables, and the authors also obtained the maximum entropy distribution for sine entropy. Tang and Gao [28] proposed triangular entropy and provided an entropy formula for regular uncertain variables. Ning et al. [29] applied triangular entropy to portfolio selection. Subsequently, Dai [30] suggested quadratic entropy and provided an entropy formula for regular uncertain variables. In addition, Dai [30] found the maximum entropy distribution for quadratic entropy. Here, we point out, all the entropy formulas in Dai and Chen [26], Yao et al. [27], Tang and Gao [28], and Dai [30] are based on Fubini's theorem. Subsequently, Ma [31] proposed unimodal entropy of uncertain variables and provided an entropy formula for regular uncertain variables. Ma [31] also found the maximum entropy distribution for unimodal entropy for regular uncertain variables. Nevertheless, limited by their methods, all the entropy formulas in Dai and Chen [26], Yao et al. [27], Tang and Gao [28], Dai [30], and Ma [31] are only established for regular uncertain variables. We know most uncertain variables are not regular. Therefore, it has some significance not only in theory but also in practice to prove the entropy formulas of Dai and Chen [26], Yao et al. [27], Tang and Gao [28], Dai [30], and Ma [31] are still true for any uncertain variable. In this paper, we will provide an entropy formula that is true for any uncertain variable by using the generalized inverse of an increasing function. We emphasize that the entropy formulas of Dai and Chen [26], Yao et al. [27], Tang and Gao [28], Dai [30], and Ma [31] are special cases of our entropy formula, and our method differs completely from those methods used by Dai and Chen [26], Yao et al. [27], Dai [30], and Ma [31].
Furthermore, we point out that all the maximum entropy distributions of Chen and Dai [25], Yao et al. [27], and Dai [30] were obtained based on the Lagrange multipliers method. Using our entropy formula, we will obtain the maximum entropy distribution without utilizing the Lagrange multipliers method. We also emphasize that the maximum entropy distributions of Chen and Dai [25], Yao et al. [27], and Dai [30] are special cases of ours.
The rest of the paper is arranged as follows. In Section 2, we first collect some necessary facts in uncertainty theory and some requisite results of the generalized inverse of an increasing function. Then, we review the entropy, entropy formulas, and the maximum entropy principles in uncertainty theory. In Section 3, we present our entropy formula and our maximum entropy distribution. We also demonstrate some applications of our entropy formula and maximum entropy distribution. In Section 4, we prove our entropy formula and derive the maximum entropy distribution of unimodel entropy. In Section 5, we verify that the maximum entropy distributions of logarithmic entropy in Theorem 2, sine entropy in Theorem 5, and quadratic entropy in Theorem 8 are special cases of our maximum entropy distributions. Finally, a concise discussion and further research are given in Section 6.

Preliminaries
In this section, first, we will collect the necessary facts in uncertainty theory. Second, we will recall some requisite results of the generalized inverse of an increasing function. Finally, we review the entropy, entropy formulas, and the maximum entropy principles in uncertainty theory.

Some Primary Facts in Uncertainty Theory
Let Γ be a non-empty set and suppose L is a σ-algebra over Γ. Each element A in L is called an event. To handle belief degree reasonably, Liu [20] introduced the uncertain measure.
The triplet (Γ, L, M) is called an uncertainty space. A measurable function X : (Γ, L, M) → R is called an uncertain variable. Let X be an uncertain variable; the function Φ(x) = M{X ≤ x} is called the uncertainty distribution of X.

Definition 2 ([32]
). An uncertainty distribution Φ is called regular provided it is strictly increasing and continuous over {x : An uncertain variable is called regular if its uncertainty distribution is regular.
Obviously, a regular uncertainty distribution Φ is invertible. Now, we recall two regular uncertainty distributions that are most used in uncertainty theory. The first is the normal uncertainty distribution N (e, σ) [32], which is defined by Here, e and σ are real numbers with σ > 0. We call an uncertain variable X normal provided its uncertainty distribution is normal and write X ∼ N (e, σ) ( Figure 1). The second is the linear uncertainty distribution L(a, b) [32], which is defined by where a and b are real numbers with a < b (Figure 2). Liu [20] suggested the expectation to describe the mean of an uncertain variable.

Definition 3 ([20]
). Let X be an uncertain variable. The expectation of X is defined by if one or two of the integrals above are finite.
Let X be an uncertain variable with finite expectation e; the variance of X is defined by Given an uncertainty variable X with uncertainty distribution Φ and finite expectation e, the variance of X can be computed by (see Section 2.6 of Chapter 2 in [9]) Yao [23] provided a nice formula for calculating the variance by inverse uncertainty distribution.
Theorem 1 (Yao [23]). Assume that X is a regular uncertain variable with uncertainty distribution We will prove the variance formula of Yao is true for any uncertain variable in Theorem 9, which is one of our main tools for establishing our entropy formula and maximum entropy principle.

Necessary Results of the Generalized Inverse of an Increasing Function
To present our results, we need to review the generalized inverse of an increasing function, which is a basic tool in statistics; see Embrechts and Hofert [33], Resnick [34], Kampke-Radermacher [35]. Here, we follow the definition of Embrechts and Hofert [33].
with the convention that inf{∅} = +∞, where ∅ is the empty set.
Remark 1. To stress the difference between the generalized inverse and standard inverse, we denote the generalized inverse of an increasing function Φ by Φ − . If Φ is strictly increasing and continuous, Φ − is just the standard inverse of Φ.
We also point out that, by the above definition, Φ − (0) may be −∞ and Φ − (1) may be +∞. Here is an example used frequently in uncertainty theory.
We also need to introduce the generalized inverse of a left-continuous increasing function; see Appendix B in Iritani and Kuga [36].

Definition 5 ([36]
). Let G be a left-continuous and increasing function over (0, 1). The generalized inverse of G is defined by for all real numbers x.
The succeeding interesting fact is that the generalized inverse of the generalized inverse for a right-continuous increasing function is just itself. To the best of our knowledge, this result was proved first by Iritani and Kuga [36] and was refined by Kampke-Radermacher [35].
for all x ∈ R, where G −1 (x) is the generalized inverse of G defined by (4).

Entropy Formula and the Principle of Maximum Entropy
To quantify the difficulty in predicting the outcomes of an uncertain variable, Liu [4] proposed logarithm entropy.
Using the Lagrange multipliers method, Chen and Dai [25] obtained the maximum entropy distribution for the logarithmic entropy. Theorem 2 ([25]). Suppose X is an uncertain variable with finite expectation e and variance σ 2 . Then, and the equality holds if and only if X is a normal uncertain variable N (e, σ).
In addition, using Fubini's theorem, Dai and Chen [26] attained an interesting entropy formula for regular uncertain variables.
Yao et al. [27] provided an entropy formula and obtained the maximum entropy distribution of sine entropy.

Theorem 4 ([27])
. Suppose X is a regular uncertain variable with uncertainty distribution Φ. Then, we have Theorem 5 ([27]). Suppose X is an uncertain variable with uncertainty distribution Φ, expectation e, and variance σ 2 . Then, we have and equality holds if and only if Tang and Gao [28] suggested triangular entropy and found an entropy formula.
Dai [30] also provided an entropy formula and got the maximum entropy distribution of quadratic entropy.

Theorem 7 ([30])
. Suppose X is a regular uncertain variable with uncertainty distribution Φ. If the entropy exists, then

Theorem 8 ([30])
. Suppose X is an uncertain variable with uncertainty distribution Φ, expectation e, and variance σ 2 . Then, we have and equality holds if and only if Ma [31] suggested unimodal entropy.
Ma [31] also provided an entropy formula and got the maximum entropy distribution of the unimodal entropy for regular uncertain variables. The readers can refer to Liu [9], Yao et al. [27], Tang and Gao [28], Dai [30], Ma [31], and the references therein for details.
Here, we emphasize the entropy formulas of Dai and Chen [26], Yao et al. [27], Tang and Gao [28], Dai [30], and Ma [31] hold only for regular uncertain variables. We will develop an entropy formula that is true for any uncertain variable. To establish the entropy formulas, the method used by Dai and Chen [26], Yao et al. [27], Tang and Gao [28], and Dai [30] is Fubini's theorem. We stress that Fubini's theorem does not work in our case because the uncertainty distribution Φ may not be regular here. We need to develop new methods.
Finally, we point out, to obtain the maximum entropy distribution, the main method used by Chen and Dai [25], Yao et al. [27], and Dai [30] is the Lagrange multipliers method. Based on our entropy formula, we can obtain the maximum entropy distribution without using the Lagrange multipliers method. We also stress that the only tool we need is the Cauchy-Schwartz inequality. In the following section, we will exhibit our new entropy formula and provide the maximum entropy distribution of unimodel entropy.

Results
In this section, we will present our results and give some remarks. First, we point out that the variance formula of Yao [23] for regular uncertain variables is valid for any uncertain variable. Theorem 9. Let X be an uncertain variable with uncertainty distribution Φ, finite expectation e, and finite variance. Then We leave the verification of Theorem 9 to the next section. Based on our variance formula and some facts on the generalized inverse, we will develop an entropy formula for any uncertain variable.

Theorem 10. Let X be an uncertain variable with finite expectation and finite variance. Assume further that unimodal function U is differentiable and satisfies
Then we have where Φ − is the generalized inverse of the uncertainty distribution Φ of X.
Remark 2. 1. The entropy formulas in Dai and Chen [26], Yao et al. [27], Tang and Gao [28], Dai [30], and Ma [31] are valid only for regular uncertain variables. Our entropy formula holds for any uncertain variable. 2. Fubini's theorem, which is used by Dai and Chen [26], Yao et al. [27], Tang and Gao [28], and Dai [30] to establish the entropy formula there, does not work in our setting, because the uncertainty distribution is not necessarily regular here.
We specifically point out, under the assumption that the uncertainty distribution Φ is regular, let respectively. By Theorem 10, we can regain the entropy formulas of logarithm entropy in Theorem 3, sine entropy in Theorem 4, triangular entropy in Theorem 6, and quadratic entropy in Theorem 7. As the verifications are routine, we omit them.
Using our entropy formula in Theorem 10, we obtain the maximum entropy distribution of unimodal entropy. Theorem 11. Let X be an uncertain variable with uncertainty distribution Φ, finite expectation e, and finite variance σ 2 . Assume further that unimodal function U is differentiable and satisfies Then, the unimodal entropy of X satisfies 19) and the equality holds if and only if where c is a real number such that Remark 3. We stress that the maximum entropy distribution of the unimodal entropy of Ma [31] is provided only for regular uncertain variables. Here, our maximum entropy distribution of unimodal entropy in Theorem 11 is obtained for any uncertain variable.
As applications of our maximum entropy distribution in Theorem 11, we can deduce the maximum entropy distribution of logarithm entropy in Theorem 2, sine entropy in Theorem 5, and quadratic entropy in Theorem 8 immediately. [25]). Let X be an uncertain variable with uncertainty distribution Φ, finite expectation e, and variance σ 2 . Then, the logarithmic entropy of X satisfies

Proposition 1 (Chen and Dai
and the equality holds if and only if i.e., X is a normal uncertain variable N (e, σ).
Proposition 3 (Dai [30]). Let X be an uncertain variable with uncertainty distribution Φ, finite expectation e, and variance σ 2 . Then, its quadratic entropy satisfies and the equality holds if and only if We leave the proofs of Corollary 1, Corollary 2, and Corollary 3 to Section 6. In the following section, we will prove Theorem 9, Theorem 10, and Theorem 11.

Proof of Theorem 9, Theorem 10, and Theorem 11
To prove Theorems 9 and 10, we need a result of the substitution rule on Lebesgue-Stieltjes integrals, which is a conclusion of Proposition 2 of Falkner and Teschl [37].
where M −1 is the generalized inverse of M defined by (4).
Here, we point out that if N(y) = y, this result goes back to the classical case of Lebesgue [38]. Before proving the main results, we also need a simple fact.

Lemma 3. Suppose function g is increasing on
and let L be a function increasing on (0, a] and decreasing on [a, 1), respectively, for some a in (0, 1). Then Because the verification of Lemma 3 is a routine computation, we omit it. We also point out that, notwithstanding that uncertainty distribution Φ may not be right-continuous, since Φ is increasing, the assumption that Φ is right-continuous does not change the variance and the entropy of an uncertain variable. Thus, we assume uncertainty distribution Φ is right-continuous in the rest. Now, we will prove Theorem 9.
Proof of Theorem 9. Let Y = X − e and Ψ be the uncertainty distribution of Y. Then we have We only need to prove Recall (see equality (2)) We only verify where the last equality is because Ψ(y) ≡ Ψ(0) for y ∈ (Ψ − (Ψ(0)), 0). By Lemma 2, let M = Ψ − , f (x) = 1 − x, N(y) = y 2 , noting that M − = Ψ according to Lemma 1, we have By integration by parts, we have Then, we have Note that function G is increasing over [Ψ(0), 1]. It follows that By monotone convergence theorem, we have 1 We need to prove lim Thus, we finish the proof of Theorem 9.

Proof of Theorem 10. Recall
Note that By the monotone convergence theorem, we have by Lemma 2 and integration by parts, we have Then, we have We only need to show Noting that 1 0 U (α)dα = 0, by the Cauchy-Schwartz inequality and Theorem 9, we have By monotone convergence theorem, again we have Let L = U and g = Φ − ; by Lemma 3, we have Then, we have Thus, we finish the proof of Theorem 10.
Now, based on our entropy formula, we can derive the maximum entropy distribution of unimodal entropy immediately.

Proof of Proposition 1, Proposition 2, and Proposition 3
In this section, we use our maximum entropy distribution to deduce Proposition 1, Proposition 2, and Proposition 3, i.e., the maximum entropy distributions of logarithm entropy, sine entropy, and quadratic entropy. As the proofs of Proposition 2 and Proposition 3 are similar to that of Proposition 1, we only prove Proposition 1.

Discussion
In this paper, by using the generalized inverse of an increasing function, we developed a useful entropy formula that is valid for any uncertain variable and encompasses all the previous cases in the literature. In addition, based on our entropy formula, we obtained the maximum entropy distribution for unimodel entropy. We also point out that most of the entropy formulas and maximum entropy distributions of the uncertain variables in the literature are special cases of ours.
Further research can consider the following two directions: (1) Develop the entropy formula and the maximum entropy principle of an uncertain set; (2) Establish the entropy formula and research the maximum entropy distributions of uncertain random variables.