R-Norm Entropy and R-Norm Divergence in Fuzzy Probability Spaces

Abstract

In the presented article, we define the R-norm entropy and the conditional R-norm entropy of partitions of a given fuzzy probability space and study the properties of the suggested entropy measures. In addition, we introduce the concept of R-norm divergence of fuzzy P-measures and we derive fundamental properties of this quantity. Specifically, it is shown that the Shannon entropy and the conditional Shannon entropy of fuzzy partitions can be derived from the R-norm entropy and conditional R-norm entropy of fuzzy partitions, respectively, as the limiting cases for R going to 1; the Kullback–Leibler divergence of fuzzy P-measures may be inferred from the R-norm divergence of fuzzy P-measures as the limiting case for R going to 1. We also provide numerical examples that illustrate the results.

1. Introduction

The concept of information entropy was introduced by Claude Shannon in 1948 in his article [1]. It is used in information theory [2] to quantify the amount of information or uncertainty inherent in a system. We remind that Shannon’s entropy is defined in the context of a probabilistic model. Consider a measurable partition $\mathcal{A}=\left\{E_1,E_2,\dots ,E_n\right\}$ of a probability space $(X,S,P)$ (that is a finite collection of measurable subsets $E_k\subset X,$ $k=1,2,\dots ,n,$ such that ${\displaystyle {\cup }_{k=1}^n{E}_k}=X,$ and $E_i\cap E_j=\varnothing$ whenever $i\ne j$) with probabilities $p_k=P(E_k),$ $k=1,2,\dots ,n.$ Then the Shannon entropy of $\mathcal{A}$ is defined as the number $h(\mathcal{A})=-{\displaystyle {\sum }_{k=1}^n{p}_k\cdot {\log }_b}{p}_k$ with the convention that $0\cdot {\log }_{b}0=0$ (which is justified by the fact that $\underset{p\to 0}{\lim }p\cdot {\log }_b\frac{1}{p}=0$). The base of the logarithm can be any positive real number; depending on the selected base of the logarithm, the entropy is expressed in bits ($b=2$), nats ($b=e$), or dits ($b=10$).

The extensions of Shannon’s entropy have led to several alternatives of entropy measure, of which the Rényi entropy [3] is one of the most important. The classical logical entropy (cf. [4,5]) and the entropy measure called the R-norm entropy (cf. [6,7]) are other alternative entropy measures. In this article, we will deal with the study of the R-norm entropy. If $P=\left\{p_1,p_2,\dots ,p_n\right\}$ is a probability distribution, then the R-norm entropy is defined, for every real number $R\in (0,1)\cup (1,\infty ),$ by the formula:

$$H_R(P)=\frac{R}{R-1}\left(1-{\left[{\displaystyle {\sum }_{k=1}^n{p}_k^R}\right]}^{\frac{1}{R}}\right).$$

Some results regarding the R-norm entropy measure and its generalizations can be found in [8,9,10,11,12]. The above entropy measures have found many important applications, for example, in statistics, pattern recognition, and coding theory.

In classical probability theory, partitions are defined in the context of the Cantor set theory. In solving many real-life problems, however, the partitions defined in terms of fuzzy set theory [13] are more appropriate. Therefore, many proposals have been made to generalize the classical partitions into fuzzy partitions [14,15,16,17,18,19,20]. Fuzzy partitions represent a mathematical tool for modeling random experiments that lead to unclear, vague events. Naturally, there are also many results concerning the Shannon entropy of fuzzy partitions; see e.g., [21,22,23,24,25,26,27,28,29,30,31]. We note that in [32] the results regarding the entropy of fuzzy partitions provided in [25] have been employed to introduce the notions of mutual information and Kullback–Leibler divergence for the fuzzy case. The notion of Kullback–Leibler divergence was introduced in [33] as the distance measure between two probability distributions. It plays significant roles in information theory and various disciplines such as statistics, machine learning, physics, neuroscience, computer science, linguistics, etc.

Since its inception in 1965, the theory of fuzzy sets has advanced in many mathematical disciplines and has found important applications in practice. Currently, the subjects of intense study are algebraic systems based on the theory of fuzzy sets, for example, D-posets [34,35,36], MV-algebras [37,38,39,40,41], and effect algebras [42]. Some results regarding the above entropy measures and divergence on these structures can be found e.g., in [43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58].

The aim of this article is to study the R-norm entropy of fuzzy partitions and the R-norm divergence in fuzzy probability spaces [59]. The organization of the paper is as follows. In Section 2 we provide basic definitions, terminology and some known results used in the paper. The results of the article are presented in Section 3 and Section 4. In Section 3, we define the R-norm entropy and conditional R-norm entropy of fuzzy partitions and examine their properties. In Section 4, the concept of the R-norm divergence for the case of fuzzy probability spaces is proposed and the properties of this distance measure are studied. The results presented in Section 3 and Section 4 are illustrated with numerical examples. The paper concludes in Section 5 with a brief summary.

2. Preliminaries

We begin by recalling the basic concepts and the known results used in the paper.

It is known that the concept of fuzzy set, introduced by Zadeh in 1965 [13], extends the classical set theory. In classical set theory, the membership of elements in a set is assessed in binary terms according to the condition—an element either belongs or does not belong to the considered set. By contrast, the fuzzy set is characterized by a membership function which assigns to every element a grade of membership ranging between zero and one. The mathematical model of the fuzzy set is as follows. Let X be a non-empty set. By a fuzzy subset of X we mean a mapping $a:X\to [0,1]$ (where the considered fuzzy set is identified with its membership function). The value $a(x)$ is interpreted as a grade of membership of the element $x\in X$ to the considered fuzzy set $a$.

Definition 1.

Let X be a non-empty set, and $M\subset {[0,1]}^X$ be a family of fuzzy subsets of X. The pair $(X,M)$ is called a fuzzy measurable space, if the following conditions are satisfied: (i) $1_X\in M,{\left(1/2\right)}_X\notin M;$ (ii) $a\in M\Rightarrow a^{\perp }\in M;$ (iii) if $a_n\in M,n=1,2,\dots ,$ then ${\displaystyle {\cup }_{n=1}^{\infty }{a}_n}\in M.$ The family $M\subset {[0,1]}^X$ with the properties (i)–(iii) is said to be a fuzzy $\sigma$-algebra.

Throughout the paper, the symbols ${\displaystyle {\cup }_{n=1}^{\infty }{a}_n}$ and ${\displaystyle {\cap }_{n=1}^{\infty }{a}_n}$ denote the fuzzy union and the fuzzy intersection of a sequence ${\left\{a_n\right\}}_{n=1}^{\infty }\subset M,$ respectively, by Zadeh [13], i.e., ${\displaystyle {\cup }_{n=1}^{\infty }{a}_n}={\sup }_n{a}_n,$ and ${\displaystyle {\cap }_{n=1}^{\infty }{a}_n}={\inf }_n{a}_n.$ The symbol $a^{\perp }$ denotes the complement of fuzzy set $a\in M,$ i.e., $a^{\perp }=1_X-a.$ Here, $1_X$ denotes the constant function with the value 1; analogously, the symbols ${\left(1/2\right)}_X$ and $0_X$ denote the constant functions with the value $1/2,$ and 0, respectively. Additionally, the relation $\le$ denotes the usual order relation of fuzzy subsets of X, i.e., $a\le b$ if and only if $a(x)\le b(x),$ for every $x\in X.$ The complementation $\perp :a\to a^{\perp },$ $a\in M,$ satisfies, for every $a,b\in M,$ the conditions: (i) ${(a^{\perp })}^{\perp }=a,$ and (ii) $a\le b$ implies $b^{\perp }\le a^{\perp }.$

Fuzzy subsets $a,b\in M$ with the property $a\cap b=0_X$ are said to be separated, fuzzy subsets $a,b\in M$ with the property $a\le b^{\perp }$ are said to be W-separated fuzzy sets. Any fuzzy subset $a\in M$ with the property $a\ge a^{\perp }$ is said to be a W-universum, any fuzzy subset $a\in M$ with the property $a\le a^{\perp }$ is said to be a W-empty fuzzy set. A fuzzy set from the fuzzy $\sigma$-algebra M is interpreted as a fuzzy event. W-separated fuzzy events are considered to be mutually exclusive events. A W-universum is interpreted as a certain event, a W-empty set as an impossible event. It can be shown that a fuzzy subset $a\in M$ is a W-universum if and only if there exists a fuzzy subset $b\in M$ such that $a=b\cup b^{\perp }.$

Naturally, the notion of a fuzzy measurable space generalizes the concept of a measurable space $(X,S)$ from the classical measure theory; it suffices to put $M=\left\{{\chi }_E:E\in S\right\},$ where ${\chi }_E$ is the characteristic function of the set $E\in S.$ With this procedure, the classical model can be inserted into the fuzzy one.

Definition 2

([59]). Let $(X,M)$ be a fuzzy measurable space. A map $\mu :M\to [0,1]$ is said to be a fuzzy P-measure, the following conditions being satisfied: (i) $\mu (a\cup a^{\perp })=1,$ for every $a\in M;$ (ii) if ${\left\{a_n\right\}}_{n=1}^{\infty }$ is a sequence of pairwise W-separated fuzzy sets from M, then $\mu \left({\displaystyle {\cup }_{n=1}^{\infty }{a}_n}\right)={\displaystyle {\sum }_{n=1}^{\infty }\mu \left(a_n\right)}.$ The triplet $(X,M,\mu )$ is called a fuzzy probability space.

The fuzzy P-measure $\mu :M\to [0,1]$ has the properties that correspond to properties of a classical probability measure; the proofs can be found in [59].

(P1)

$\mu (a^{\perp })=1-\mu (a),$ for every $a\in M.$

(P2)

$\mu$ is non-decreasing, i.e., if $a,b\in M$ with $a\le b,$ then $\mu (a)\le \mu (b).$

(P3)

$\mu (a\cup b)+\mu (a\cap b)=\mu (a)+\mu (b),$ for every $a,b\in M.$

(P4)

Let $b\in M.$ Then $\mu (a\cap b)=\mu (a)$ for all $a\in M$ if and only if $\mu (b)=1.$

(P5)

If $a,b\in M$ such that $a\le b^{\perp },$ then $\mu (a\cap b)=0.$

(P6)

If $a,b\in M$ such that $a\le b,$ then $\mu (b)=\mu (a)+\mu (a^{\perp }\cap b).$

Definition 3

([14]). A fuzzy partition of a fuzzy probability space $(X,M,\mu )$ is a collection $A=\left\{a_1,a_2,\dots ,a_n\right\}$ of W-separated fuzzy sets from M with the property $\mu \left({\displaystyle {\cup }_{i=1}^n{a}_i}\right)=1.$

In the system of all fuzzy partitions of $(X,M,\mu ),$ we define the refinement partial order in the following way. If A and B are two fuzzy partitions of $(X,M,\mu ),$ then we say that B is a refinement of A (and write $A\prec B),$ if for every $b\in B$ there exists $a\in A$ such that $b\le a.$ Furthermore, for every two fuzzy partitions $A=\left\{a_1,a_2,\dots ,a_n\right\},$ and $B=\left\{b_1,b_2,\dots ,b_m\right\}$ of $(X,M,\mu ),$ we put $A\vee B=$ $\left\{a_i\cap b_j;i=1,2,\dots ,n,j=1,2,\dots ,m\right\}.$ One can easily to verify that the family $A\vee B$ is a family of pairwise W-separated fuzzy sets from M; moreover, by the property (P4), we have $\mu \left({\displaystyle {\cup }_{i=1}^n{\displaystyle {\cup }_{j=1}^m(a_i\cap b_j)}}\right)=\mu \left(\left({\displaystyle {\cup }_{i=1}^n{a}_i}\right)\cap \left({\displaystyle {\cup }_{j=1}^m{b}_j}\right)\right)=\mu \left({\displaystyle {\cup }_{i=1}^n{a}_i}\right)=1.$ Thus, $A\vee B$ is a fuzzy partition of $(X,M,\mu ).$ It represents a combined experiment consisting of a realization of the experiments A and B. Evidently, it holds $A\prec A\vee B,$ and $B\prec A\vee B,$ i.e., the fuzzy partition $A\vee B$ is a common refinement of fuzzy partitions A and B. If $A_1,A_2,\dots ,A_n$ are fuzzy partitions of $(X,M,\mu ),$ then we put ${\vee }_{i=1}^n{A}_i=A_1\vee A_2\vee \dots \vee A_n.$

Definition 4.

Two fuzzy partitions $A=\left\{a_1,a_2,\dots ,a_n\right\},$ and $B=\left\{b_1,b_2,\dots ,b_m\right\}$ of a fuzzy probability space $(X,M,\mu )$ are said to be statistically independent, if $\mu (a_i\cap b_j)=\mu (a_i)\cdot \mu (b_j),$ for $i=1,2,\dots ,n,$ $j=1,2,\dots ,m.$

Example 1.

Let us consider a classical probability space $(X,S,P),$ and put $M=\left\{{\chi }_E:E\in S\right\}.$ It can be verified that the map $\mu :M\to [0,1]$ defined by $\mu ({\chi }_E)=P(E),$ for every ${\chi }_E\in M,$ is a fuzzy P-measure and the triplet $(X,M,\mu )$ is a fuzzy probability space. A classical measurable partition $\mathcal{A}=\left\{E_1,E_2,\dots ,E_n\right\}$ of a probability space $(X,S,P)$ can be eventually regarded as a fuzzy partition of $(X,M,\mu ),$ considering ${\chi }_{E_i}$ instead of $E_i.$

The Shannon entropy of fuzzy partition of a fuzzy probability space $(X,M,\mu )$ has been introduced and examined in [23], see also [25].

Definition 5

([23]). We define the entropy of a fuzzy partition $A=\left\{a_1,a_2,\dots ,a_n\right\}$ of $(X,M,\mu )$ by Shannon’s formula:

$$H^{\mu }(A)=-{\displaystyle {\sum }_{i=1}^n\mu (a_i)\cdot \log }\mu (a_i).$$

(1)

If $A=\left\{a_1,a_2,\dots ,a_n\right\},$ and $B=\left\{b_1,b_2,\dots ,b_m\right\}$ are two fuzzy partitions of $(X,M,\mu ),$ then we define the conditional entropy of A given B by the formula:

$$H^{\mu }(A/B)=-{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j=1}^m\mu (a_i\cap b_j)\cdot \log }}\frac{\mu (a_i\cap b_j)}{\mu (b_j)}$$

(2)

with the convention that $0\cdot \log \frac{0}{x}=0$ if $x\ge 0.$

The symbol log denotes the base 2 logarithm, so the Shannon entropy of fuzzy partition is expressed in bits. The entropy and the conditional entropy of fuzzy partitions have properties that correspond to properties of Shannon’s entropy of classical measurable partitions: for every fuzzy partitions $A,B,C$ of a fuzzy probability space $(X,M,\mu ),$ it holds:

(S1)

$A\prec B$ implies $H^{\mu }(A)\le H^{\mu }(B);$

(S2)

$H^{\mu }(A\vee B/C)=H^{\mu }(A/C)+H^{\mu }(B/C\vee A);$

(S3)

$A\prec B$ implies $H^{\mu }(A/C)\le H^{\mu }(B/C);$

(S4)

$A\prec B$ implies $H^{\mu }(C/A)\ge H^{\mu }(C/B);$

(S5)

$H^{\mu }(A/B)\le H^{\mu }(A)$ with the equality if and only if $A,B$ are statistically independent;

(S6)

$H^{\mu }(B\vee C/A)\le H^{\mu }(B/A)+H^{\mu }(C/A);$

(S7)

$H^{\mu }(A\vee B)=H^{\mu }(A)+H^{\mu }(B/A).$

The proofs can be found in [23,25]. We remark that in [15,16,17,18,19,20,21,22,26,27,28,29,30,31], other conceptions of fuzzy partitions and their entropy measures have been introduced. Whereas our approach is based on Zadeh’s connectives, in the referenced papers Zadeh’s connectives have been replaced by other fuzzy set operations.

We note that in [32], the concept of Kullback–Leibler divergence in the fuzzy probability space was introduced. Let $\mu ,$ $\upsilon$ be two fuzzy P-measures on a fuzzy measurable space $(X,M),$ and $A=\left\{a_1,a_2,\dots ,a_n\right\}$ be a fuzzy partition of fuzzy probability spaces $(X,M,\mu )$, $(X,M,\upsilon ).$ Then the Kullback–Leibler divergence of fuzzy P-measures $\mu ,$ $\upsilon$ with respect to $A$ is defined as the number:

$$D_A(\mu \parallel \upsilon )={\displaystyle {\sum }_{i=1}^n\mu (a_i)}\cdot \log \frac{\mu (a_i)}{\upsilon (a_i)}$$

(3)

with the convention that $x\log \frac{x}{0}=\infty$ if $x>0,$ and $0\cdot \log \frac{0}{x}=0$ if $x\ge 0$.

In the following sections, we will use the following known Minkowski inequality: for non-negative real numbers $x_1,x_2,\dots ,x_n,y_1,y_2,\dots ,y_n,$ it holds:

$${\left[{\displaystyle {\sum }_{i=1}^n{x_i}^R}\right]}^{\frac{1}{R}}+{\left[{\displaystyle {\sum }_{i=1}^n{y_i}^R}\right]}^{\frac{1}{R}}\ge {\left[{\displaystyle {\sum }_{i=1}^n{(x_i+y_i)}^R}\right]}^{\frac{1}{R}},\mathrm{for}R>1,$$

and

$${\left[{\displaystyle {\sum }_{i=1}^n{x_i}^R}\right]}^{\frac{1}{R}}+{\left[{\displaystyle {\sum }_{i=1}^n{y_i}^R}\right]}^{\frac{1}{R}}\le {\left[{\displaystyle {\sum }_{i=1}^n{(x_i+y_i)}^R}\right]}^{\frac{1}{R}},\mathrm{for}0<R<1.$$

Furthermore, we will use the Jensen inequality which states that for a real convex function $\phi ,$ real numbers $x_1,x_2,\dots ,x_m$ in its domain and non-negative real numbers ${\alpha }_1,{\alpha }_2,\dots ,{\alpha }_m$ with ${\displaystyle {\sum }_{j=1}^m{\alpha }_j=1,}$ it holds:

$$\phi \left({\displaystyle {\sum }_{j=1}^m{\alpha }_j{x}_j}\right)\le {\displaystyle {\sum }_{j=1}^m{\alpha }_j\phi (x_j}),$$

and the inequality is reversed if $\phi$ is a real concave function. The equality holds if and only if $x_1=x_2=\dots =x_m$ or $\phi$ is linear.

In addition, we will use L’Hôpital’s rule: for functions $f$ and $g$ that are differentiable on an open interval U except possibly at a point $a\in U,$ if $\underset{x\to a}{\lim }f(x)=$$\underset{x\to a}{\lim }g(x)=0,$ $g^{\prime }(x)\ne 0,$ for every x in $U$ with $x\ne a,$ and $\underset{x\to a}{\lim }\frac{f^{\prime }(x)}{g^{\prime }(x)}$ exists, then:

$$\underset{x\to a}{\lim }\frac{f(x)}{g(x)}=\underset{x\to a}{\lim }\frac{f^{\prime }(x)}{g^{\prime }(x)}.$$

3. The R-Norm Entropy of Fuzzy Partitions

In this part we define the R-norm entropy of a fuzzy partition and its conditional version and study the properties of these entropy measures. It is shown that as the limiting cases of the R-norm entropy and the conditional R-norm entropy of fuzzy partitions for R going to 1, we obtain the Shannon entropy $H^{\mu }(A),$ and the conditional Shannon entropy $H^{\mu }(A/B),$ respectively, expressed in nats.

Definition 6.

Let $A=\left\{a_1,a_2,\dots ,a_n\right\}$ be a fuzzy partition of a fuzzy probability space $(X,M,\mu ).$ The R-norm entropy of A with respect to $\mu$ is defined, for a positive real number R not equal to 1, by the formula:

$$H_R^{\mu }(A)=\frac{R}{R-1}\left(1-{\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}\right).$$

(4)

Remark 1.

For simplicity, we write $\mu {(a_i)}^R$ instead of ${(\mu (a_i))}^R.$ In the following, we will write $\log {\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}$ instead of $\log \left({\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right).$

Theorem 1.

For arbitrary fuzzy partition $A$ of a fuzzy probability space $(X,M,\mu ),$ the R-norm entropy $H_R^{\mu }(A)$ is non-negative.

Proof. 

Assume that $A=\left\{a_1,a_2,\dots ,a_n\right\}.$ We will consider two cases: the case of $R>1,$ and the case of $0<R<1.$ If $R>1,$ then $\mu {(a_i)}^R\le \mu (a_i),$ for $i=1,2,\dots ,n,$ hence ${\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\le {\displaystyle {\sum }_{i=1}^n\mu (a_i)}=1.$ This implies that ${\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}\le 1.$ Since $\frac{R}{R-1}>0$ for $R>1,$ it follows that $H_R^{\mu }(A)=$ $\frac{R}{R-1}\left(1-{\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}\right)\ge 0.$ On the other hand, for $0<R<1,$ it holds that $\mu {(a_i)}^R\ge \mu (a_i),$ for $i=1,2,\dots ,n,$ hence ${\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\ge {\displaystyle {\sum }_{i=1}^n\mu (a_i)}=1.$ It follows that ${\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}\ge 1.$ Since $\frac{R}{R-1}<0$ for $0<R<1,$ we obtain $H_R^{\mu }(A)=\frac{R}{R-1}\left(1-{\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}\right)\ge 0.$ □

Example 2.

Let $X=[0,1],$ and $a:X\to [0,1]$ be defined by $a(x)=x,x\in X.$ Consider a fuzzy measurable space $(X,M),$ where $M=\left\{1_X,0_X,a,a^{\perp },a\cup a^{\perp },a\cap a^{\perp }\right\}.$ Then it can be easily verified that the mappings $\mu :M\to \left[0,1\right],$ and $\upsilon :M\to \left[0,1\right]$ defined by the equalities $\mu \left(1_X\right)=\mu (a\cup a^{\perp })=1,$ $\mu \left(0_X\right)=\mu (a\cap a^{\perp })=0,$ $\mu (a)=\mu (a^{\perp })=\frac{1}{2},$ $\upsilon \left(1_X\right)=\upsilon (a\cup a^{\perp })=1,$ $\upsilon \left(0_X\right)=\upsilon (a\cap a^{\perp })=0,$ $\upsilon (a)=\frac{1}{3},$ $\upsilon (a^{\perp })=\frac{2}{3},$ are fuzzy P-measures and the systems $(X,M,\mu ),$ $(X,M,\upsilon )$ are fuzzy probability spaces. The sets $A=\left\{a,a^{\perp }\right\},$ $B=\left\{a\cup a^{\perp }\right\},$ $C=\left\{1_X\right\}$ are fuzzy partitions of $(X,M,\mu ),$ and $(X,M,\upsilon )$ such that $C\prec B\prec A.$ We can calculate their R-norm entropy. Evidently, $H_R^{\mu }(B)=H_R^{\mu }(C)=H_R^{\upsilon }(B)=H_R^{\upsilon }(C)=0;$ in accordance with the natural requirement, experiments resulting in a certain event have zero R-norm entropy. Furthermore, we have:

$$H_R^{\mu }(A)=\frac{R}{R-1}\left(1-{\left[{\left(\frac{1}{2}\right)}^R+{\left(\frac{1}{2}\right)}^R\right]}^{\frac{1}{R}}\right)=\frac{R}{R-1}\left(1-2^{\frac{1-R}{R}}\right),$$

$$H_R^{\upsilon }(A)=\frac{R}{R-1}\left(1-{\left[{\left(\frac{1}{3}\right)}^R+{\left(\frac{2}{3}\right)}^R\right]}^{\frac{1}{R}}\right).$$

If we put $R=\frac{1}{2},$ then $H_R^{\mu }(A)=1,$ $H_R^{\upsilon }(A)\dot{=}0.9428;$ for $R=\frac{1}{3},$ we have $H_R^{\mu }(A)=1.5,$ $H_R^{\upsilon }(A)\dot{=}1.4236;$ for $R=2,$ we have $H_R^{\mu }(A)\dot{=}0.58579,$ and $H_R^{\upsilon }(A)\dot{=}0.50928.$

Definition 7.

Let $A=\left\{a_1,a_2,\dots ,a_n\right\},$ and $B=\left\{b_1,b_2,\dots ,b_m\right\}$ be two fuzzy partitions of a fuzzy probability space $(X,M,\mu ).$ Then the conditional R-norm entropy of A given B with respect to $\mu$ is defined, for a positive real number R not equal to 1, by the formula:

$$H_R^{\mu }(A/B)=\frac{R}{R-1}\left({\left[{\displaystyle {\sum }_{j=1}^m\mu {(b_j)}^R}\right]}^{\frac{1}{R}}-{\left[{\displaystyle {\sum }_{j=1}^m{\displaystyle {\sum }_{i=1}^n\mu {(a_i\cap b_j)}^R}}\right]}^{\frac{1}{R}}\right).$$

(5)

Remark 2.

Let A be a fuzzy partition of a given fuzzy probability space $(X,M,\mu ).$ Evidently, if we put $B=\left\{b\right\},$ where $b\in M$ is a W-universum, then $H_R^{\mu }(A/B)=H_R^{\mu }(A).$

The following theorem shows the consistency of the conditional R-norm entropy $H_R^{\mu }(A/B),$ in the case of the limit of $R$ going to 1, with the conditional Shannon entropy $H^{\mu }(A/B)$ defined by the formula (2), up to a positive multiplicative constant.

Theorem 2.

Let $A=\left\{a_1,a_2,\dots ,a_n\right\},$ and $B=\left\{b_1,b_2,\dots ,b_m\right\}$ be two fuzzy partitions of a given fuzzy probability space $(X,M,\mu ).$ Then $\underset{R\to 1}{\lim }{H}_R^{\mu }(A/B)=c\cdot H^{\mu }(A/B),$ where $c=\frac{1}{\log e},$ and $H^{\mu }(A/B)=$ $-{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j=1}^m\mu (a_i\cap b_j)\cdot \log }}\frac{\mu (a_i\cap b_j)}{\mu (b_j)}.$

Proof. 

In the proof, we use L’Hôpital’s rule $\underset{R\to a}{\lim }\frac{f(R)}{g(R)}=\underset{R\to a}{\lim }\frac{f^{\prime }(R)}{g^{\prime }(R)},$ where in this case $a=1.$ For every $R\in (0,1)\cup (1,\infty ),$ we can write:

$$H_R^{\mu }(A/B)=\frac{1}{1-\frac{1}{R}}\left({\left[{\displaystyle {\sum }_{j=1}^m\mu {(b_j)}^R}\right]}^{\frac{1}{R}}-{\left[{\displaystyle {\sum }_{j=1}^m{\displaystyle {\sum }_{i=1}^n\mu {(a_i\cap b_j)}^R}}\right]}^{\frac{1}{R}}\right)=\frac{f(R)}{g(R)},$$

where $f,g$ are continuous functions defined for $R\in (0,\infty )$ in the following way:

$$f(R)={\left[{\displaystyle {\sum }_{j=1}^m\mu {(b_j)}^R}\right]}^{\frac{1}{R}}-{\left[{\displaystyle {\sum }_{j=1}^m{\displaystyle {\sum }_{i=1}^n\mu {(a_i\cap b_j)}^R}}\right]}^{\frac{1}{R}},g(R)=1-\frac{1}{R}.$$

By continuity of the function $g,$ we get $\underset{R\to 1}{\lim }g(R)=g(1)=0.$ Furthermore, by continuity of the function $f,$ and by the property (P4) of fuzzy P-measure $\mu ,$ we get $\underset{R\to 1}{\lim }f(R)=f(1)={\displaystyle {\sum }_{j=1}^m\mu (b_j)}-{\displaystyle {\sum }_{j=1}^m{\displaystyle {\sum }_{i=1}^n\mu (a_i\cap b_j)=1-{\displaystyle {\sum }_{j=1}^m\mu \left(\left({\displaystyle {\cup }_{i=1}^n{a}_i}\right)\cap b_j\right)=1-}}}{\displaystyle {\sum }_{j=1}^m\mu \left(b_j\right)=1-}1=0.$ Using L’Hôpital’s rule, this implies:

$$\underset{R\to 1}{\lim }{H}_R^{\mu }(A/B)=\frac{\underset{R\to 1}{\lim }{f}^{\prime }(R)}{\underset{R\to 1}{\lim }{g}^{\prime }(R)}$$

under the assumption that the right-hand side exists. To find the derivative of the function $f(R)$ we use the identity $b^{\alpha }=e^{\alpha \ln b}.$ Let us calculate:

$$\begin{gathered} \frac{d}{dR}f(R)=e^{\frac{1}{R}\ln {\displaystyle \sum _{j=1}^m\mu {(b_j)}^R}}\cdot \left(-\frac{1}{R^2}\cdot \ln {\displaystyle \sum _{j=1}^m\mu {(b_j)}^R}+\frac{1}{R}\cdot \frac{1}{{\displaystyle {\sum }_{j=1}^m\mu {(b_j)}^R}}{\displaystyle \sum _{j=1}^m\mu {(b_j)}^R}\cdot \ln \mu (b_j)\right) \\ -e^{\frac{1}{R}\ln {\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^m\mu {(a_i\cap b_j)}^R}}}\left(-\frac{1}{R^2}\cdot \ln {\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^m\mu {(a_i\cap b_j)}^R}}+\frac{1}{R}\cdot \frac{1}{{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^m\mu {(a_i\cap b_j)}^R}}}{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^m\mu {(a_i\cap b_j)}^R}}\cdot \ln \mu (a_i\cap b_j)\right). \end{gathered}$$

Since $\underset{R\to 1}{\lim }{g}^{\prime }(R)=\underset{R\to 1}{\lim }\frac{1}{R^2}=1,$ we obtain:

$$\begin{gathered} \underset{R\to 1}{\lim }{H}_R^{\mu }(A/B)=\underset{R\to 1}{\lim }{f}^{\prime }(R)={\displaystyle \sum _{j=1}^m\mu (b_j)\cdot \ln \mu (b_j)}-{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^m\mu (a_i\cap b_j)}}\cdot \ln \mu (a_i\cap b_j) \\ ={\displaystyle \sum _{j=1}^m{\displaystyle \sum _{i=1}^n\mu (a_i\cap b_j)}\cdot \ln \mu (b_j)}-{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^m\mu (a_i\cap b_j)}}\cdot \ln \mu (a_i\cap b_j) \\ =-{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^m\mu (a_i\cap b_j)}}\cdot \left(\frac{\log \mu (a_i\cap b_j)}{\log e}-\frac{\log \mu (b_j)}{\log e}\right)=-\frac{1}{\log e}{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^m\mu (a_i\cap b_j)}}\cdot \log \frac{\mu (a_i\cap b_j)}{\mu (b_j)}.\square \end{gathered}$$

Theorem 3.

Let $A=\left\{a_1,a_2,\dots ,a_n\right\}$ be a fuzzy partition of a fuzzy probability space $(X,M,\mu ).$ Then $\underset{R\to 1}{\lim }{H}_R^{\mu }(A)=c\cdot H^{\mu }(A),$ where $H^{\mu }(A)=-{\displaystyle {\sum }_{i=1}^n\mu (a_i)\cdot \log }\mu (a_i),$ and $c=\frac{1}{\log e}.$

Proof. 

The claim is a direct consequence of the previous theorem; it suffices to put $B=\left\{1_X\right\}.$ □

In the following, the properties of the R-norm entropy of fuzzy partitions are discussed.

Theorem 4.

For arbitrary fuzzy partitions $A,B$ and $C$ of a fuzzy probability space $(X,M,\mu ),$ it holds that:

$$H_R^{\mu }(A\vee B/C)=H_R^{\mu }(A/C)+H_R^{\mu }(B/A\vee C).$$

(6)

Proof. 

Suppose that $A=\left\{a_1,a_2,\dots ,a_n\right\},$ $B=\left\{b_1,b_2,\dots ,b_m\right\},$ $C=\left\{c_1,c_2,\dots ,c_r\right\}.$ Let us calculate:

$$\begin{gathered} H_R^{\mu }(A\vee B/C)=\frac{R}{R-1}\left({\left[{\displaystyle {\sum }_{k=1}^r\mu {(c_k)}^R}\right]}^{\frac{1}{R}}-{\left[{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j=1}^m{\displaystyle {\sum }_{k=1}^r\mu {(a_i\cap b_j\cap c_k)}^R}}}\right]}^{\frac{1}{R}}\right) \\ =\frac{R}{R-1}\left({\left[{\displaystyle {\sum }_{k=1}^r\mu {(c_k)}^R}\right]}^{\frac{1}{R}}-{\left[{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{k=1}^r\mu {(a_i\cap c_k)}^R}}\right]}^{\frac{1}{R}}\right) \\ +\frac{R}{R-1}\left({\left[{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{k=1}^r\mu {(a_i\cap c_k)}^R}}\right]}^{\frac{1}{R}}-{\left[{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j=1}^m{\displaystyle {\sum }_{k=1}^r\mu {(a_i\cap b_j\cap c_k)}^R}}}\right]}^{\frac{1}{R}}\right) \\ =H_R^{\mu }(A/C)+H_R^{\mu }(B/A\vee C).\square \end{gathered}$$

Theorem 5.

For arbitrary fuzzy partitions $A,B$ of a fuzzy probability space $(X,M,\mu ),$ it holds that:

$$H_R^{\mu }(A\vee B)=H_R^{\mu }(A)+H_R^{\mu }(B/A).$$

(7)

Proof. 

The claim is a direct consequence of the previous theorem; it suffices to put $C=\left\{1_X\right\}.$ □

In the following theorem, using the notion of conditional R-norm entropy of fuzzy partitions, chain rules for the R-norm entropy of fuzzy partitions are established.

Theorem 6.

Let $A_1,A_2,\dots ,A_n$ and C be fuzzy partitions of a fuzzy probability space $(X,M,\mu ).$ Then, for $n=2,3,\dots ,$ the following equalities hold:

(i) 

$H_R^{\mu }(A_1\vee A_2\vee \dots \vee A_n)=H_R^{\mu }(A_1)+{\displaystyle {\sum }_{i=2}^n{H}_R^{\mu }(A_i/{\vee }_{k=1}^{i-1}{A}_k)};$

(ii) 

$H_R^{\mu }({\vee }_{i=1}^n{A}_i/C)=H_R^{\mu }(A_1/C)+{\displaystyle {\sum }_{i=2}^n{H}_R^{\mu }(A_i/({\vee }_{k=1}^{i-1}{A}_k)\vee C)}.$

Proof. 

The proof can be done using mathematical induction and Theorems 4 and 5. □

In the following we prove that the R-norm entropy $H_R^{\mu }(A)$ is a concave function on the class of all fuzzy P-measures on a given fuzzy measurable space $(X,M).$

Proposition 1.

Let $\mu ,$ $\upsilon$ be two fuzzy P-measures on a given fuzzy measurable space $(X,M).$ Then, for every real number $\lambda \in [0,1],$ the map $\lambda \mu +(1-\lambda )\upsilon :X\to [0,1]$ is a fuzzy P-measure on $(X,M).$

Proof. 

It is straightforward. □

Theorem 7.

Let A be a fuzzy partition of fuzzy probability spaces $(X,M,\mu ),$ $(X,M,\upsilon ).$ Then, for every real number $\lambda \in [0,1],$ this inequality holds:

$$\lambda H_R^{\mu }(A)+(1-\lambda )H_R^{\upsilon }(A)\le H_R^{\lambda \mu +(1-\lambda )\upsilon }(A).$$

Proof. 

Let $A=\left\{a_1,a_2,\dots ,a_n\right\},$ and $\lambda \in [0,1].$ Putting $x_i=\lambda \mu (a_i),$ and $y_i=(1-\lambda )\upsilon (a_i),$ for $i=1,2,\dots ,n,$ in the Minkowski inequality, we obtain for $R>1:$

$$\lambda {\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}+(1-\lambda ){\left[{\displaystyle {\sum }_{i=1}^n\upsilon {(a_i)}^R}\right]}^{\frac{1}{R}}\ge {\left[{\displaystyle {\sum }_{i=1}^n{(\lambda \mu (a_i)+(1-\lambda )\upsilon (a_i))}^R}\right]}^{\frac{1}{R}},$$

and for $0<R<1$:

$$\lambda {\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}+(1-\lambda ){\left[{\displaystyle {\sum }_{i=1}^n\upsilon {(a_i)}^R}\right]}^{\frac{1}{R}}\le {\left[{\displaystyle {\sum }_{i=1}^n{(\lambda \mu (a_i)+(1-\lambda )\upsilon (a_i))}^R}\right]}^{\frac{1}{R}}.$$

This means that the function $\mu \mapsto {\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}$ is convex in $\mu$ for $R>1,$ and concave in $\mu$ for $0<R<1.$ Therefore, the function $\mu \mapsto 1-{\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}$ is concave in $\mu$ for $R>1,$ and convex in $\mu$ for $0<R<1.$ Evidently, $\frac{R}{R-1}>0$ for $R>1,$ and $\frac{R}{R-1}<0$ for $0<R<1.$ According to definition of the R-norm entropy $H_R^{\mu }(A),$ we obtain that for every $R\in (0,1)\cup (1,\infty ),$ the R-norm entropy $\mu \mapsto H_R^{\mu }(A)$ is a concave function on the family of all fuzzy P-measures on a given fuzzy measurable space $(X,M).$ Thus, for every $\lambda \in [0,1],$ it holds that:

$$\lambda H_R^{\mu }(A)+(1-\lambda )H_R^{\upsilon }(A)\le H_R^{\lambda \mu +(1-\lambda )\upsilon }(A).\square$$

Proposition 2.

Let $A,B$ be fuzzy partitions of a fuzzy probability space $(X,M,\mu )$ such that $A\prec B.$ Then there exists a partition $\left\{I_1,I_2,\dots ,I_n\right\}$ of the set $\left\{1,2,\dots ,m\right\}$ such that $\mu (a_i)={\displaystyle {\sum }_{j\in I_i}\mu (b_j)},$ for $i=1,2,\dots ,n.$

Proof. 

By the assumption, for every $b\in B$ there exists $a\in A$ such that $b\le a.$ Let us denote by $I_i$ the subset of the set $\left\{1,2,\dots ,m\right\}$ such that for every $j\in I_i,$ it holds that $b_j\le a_i,$ $i=1,2,\dots ,n.$ Then the set $\left\{I_1,I_2,\dots ,I_n\right\}$ is a partition of the set $\left\{1,2,\dots ,m\right\}$ and ${\cup }_{j\in I_i}{b}_j\le a_i,$ for $i=1,2,\dots ,n.$ By monotonicity of fuzzy P-measure $\mu ,$ $\mu (a_i)\ge \mu ({\cup }_{j\in I_i}{b}_j)={\displaystyle {\sum }_{j\in I_i}\mu (b_j)},$ for $i=1,2,\dots ,n.$ Summing over $i=1,2,\dots ,n,$ we get ${\displaystyle {\sum }_{i=1}^n\mu (a_i)}\ge {\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j\in I_i}\mu (b_j)}}={\displaystyle {\sum }_{j=1}^m\mu (b_j)}=1.$ Since ${\displaystyle {\sum }_{i=1}^n\mu (a_i)}=1,$ it follows that $\mu (a_i)={\displaystyle {\sum }_{j\in I_i}\mu (b_j)},$ for $i=1,2,\dots ,n.$ □

Theorem 8.

Let $A,B,C$ be fuzzy partitions of a fuzzy probability space $(X,M,\mu )$ such that $A\prec B.$ Then:

(i) 

$H_R^{\mu }(A)\le H_R^{\mu }(B);$

(ii) 

$H_R^{\mu }(A/C)\le H_R^{\mu }(B/C).$

Proof. 

(i) Assume that $A=\left\{a_1,a_2,\dots ,a_n\right\},$ $B=\left\{b_1,b_2,\dots ,b_m\right\},$ $A\prec B.$ According to Proposition 2 there exists a partition $\left\{I_1,I_2,\dots ,I_n\right\}$ of the set $\left\{1,2,\dots ,m\right\}$ such that $\mu (a_i)={\displaystyle {\sum }_{j\in I_i}\mu (b_j)},$ for $i=1,2,\dots ,n.$ For the case of $R>1,$ we obtain:

$$\mu {(a_i)}^R={\left({\displaystyle {\sum }_{j\in I_i}\mu (b_j)}\right)}^R\ge {{\displaystyle {\sum }_{j\in I_i}\mu (b_j)}}^R,\mathrm{for}i=1,2,\dots ,n,$$

and consequently:

$${\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\ge {\displaystyle {\sum }_{j=1}^m\mu {(b_j)}^R}.$$

Hence

$${\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}\ge {\left[{\displaystyle {\sum }_{j=1}^m\mu {(b_j)}^R}\right]}^{\frac{1}{R}}.$$

Since $\frac{R}{R-1}>0$ for $R>1,$ we conclude that:

$$H_R^{\mu }(A)=\frac{R}{R-1}\left(1-{\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}\right)\le \frac{R}{R-1}\left(1-{\left[{\displaystyle {\sum }_{j=1}^m\mu {(b_j)}^R}\right]}^{\frac{1}{R}}\right)=H_R^{\mu }(B).$$

For the case of $0<R<1,$ we get:

$$\mu {(a_i)}^R={\left({\displaystyle {\sum }_{j\in I_i}\mu (b_j)}\right)}^R\le {{\displaystyle {\sum }_{j\in I_i}\mu (b_j)}}^R,\mathrm{for}i=1,2,\dots ,n,$$

and consequently:

$${\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\le {\displaystyle {\sum }_{j=1}^m\mu {(b_j)}^R}.$$

Therefore:

$${\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}\le {\left[{\displaystyle {\sum }_{j=1}^m\mu {(b_j)}^R}\right]}^{\frac{1}{R}}.$$

Since $\frac{R}{R-1}<0$ for $0<R<1,$ we have:

$$H_R^{\mu }(A)=\frac{R}{R-1}\left(1-{\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}\right)\le \frac{R}{R-1}\left(1-{\left[{\displaystyle {\sum }_{j=1}^m\mu {(b_j)}^R}\right]}^{\frac{1}{R}}\right)=H_R^{\mu }(B).$$

(ii) By the assumption, for every $b\in B$ there exists $a\in A$ such that $b\le a.$ Hence, for arbitrary element $b\cap c$ of fuzzy partition $B\vee C$ there exists $a\cap c\in A\vee C$ such that $b\cap c\le a\cap c.$ This means that $A\vee C\prec B\vee C.$ Therefore, we get:

$$H_R^{\mu }(A/C)=H_R^{\mu }(A\vee C)-H_R^{\mu }(C)\le H_R^{\mu }(B\vee C)-H_R^{\mu }(C)=H_R^{\mu }(B/C).\square$$

Theorem 9.

Let $A,B$ be statistically independent fuzzy partitions of a fuzzy probability space $(X,M,\mu ).$ Then:

$$H_R^{\mu }(A/B)=H_R^{\mu }(A)-\frac{R-1}{R}{H}_R^{\mu }(A)\cdot H_R^{\mu }(B).$$

Proof. 

Let $A=\left\{a_1,a_2,\dots ,a_n\right\},$ and $B=\left\{b_1,b_2,\dots ,b_m\right\}.$ By the assumption, $\mu (a_i\cap b_j)=\mu (a_i)\cdot \mu (b_j),$ for $i=1,2,\dots ,n,j=1,2,\dots ,m.$ Therefore we can write:

$$\begin{gathered} H_R^{\mu }(A/B)=\frac{R}{R-1}\left({\left[{\displaystyle {\sum }_{j=1}^m\mu {(b_j)}^R}\right]}^{\frac{1}{R}}-{\left[{\displaystyle {\sum }_{j=1}^m{\displaystyle {\sum }_{i=1}^n\mu {(a_i\cap b_j)}^R}}\right]}^{\frac{1}{R}}\right) \\ =\frac{R}{R-1}\left({\left[{\displaystyle {\sum }_{j=1}^m\mu {(b_j)}^R}\right]}^{\frac{1}{R}}-{\left[{\displaystyle {\sum }_{j=1}^m\mu {(b_j)}^R}\right]}^{\frac{1}{R}}{\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}\right) \\ =\frac{R}{R-1}(1-{\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}-1+{\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}+{\left[{\displaystyle {\sum }_{j=1}^m\mu {(b_j)}^R}\right]}^{\frac{1}{R}}-{\left[{\displaystyle {\sum }_{j=1}^m\mu {(b_j)}^R}\right]}^{\frac{1}{R}}{\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}) \\ =\frac{R}{R-1}\left(1-{\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}\right)-\frac{R-1}{R}\left(\frac{R}{R-1}\left(1-{\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}\right)\cdot \frac{R}{R-1}\left(1-{\left[{\displaystyle {\sum }_{j=1}^m\mu {(b_j)}^R}\right]}^{\frac{1}{R}}\right)\right) \\ =H_R^{\mu }(A)-\frac{R-1}{R}{H}_R^{\mu }(A)\cdot H_R^{\mu }(B).\square \end{gathered}$$

In view of Theorems 5 and 9, the R-norm entropy does not have the property of additivity, but it satisfies the property that is called pseudo-additivity, as stated in the following theorem.

Theorem 10.

(Pseudo-additivity). Let $A,B$ be statistically independent fuzzy partitions of a fuzzy probability space $(X,M,\mu ).$ Then:

$$H_R^{\mu }(A\vee B)=H_R^{\mu }(A)+H_R^{\mu }(B)-\frac{R-1}{R}{H}_R^{\mu }(A)\cdot H_R^{\mu }(B).$$

Proof. 

The result follows by combining Theorems 5 and 9. □

4. The R-Norm Divergence of Fuzzy P-Measures

In this part, the concept of the R-norm divergence of fuzzy P-measures is defined. In order to avoid expressions like $\frac{0}{0},$ we will use in this section the following simplification: for any fuzzy partition $A=\left\{a_1,a_2,\dots ,a_n\right\}$ of a fuzzy probability space $(X,M,\mu$), we assume that $\mu (a_i)>$0, for $i=1,2,\dots ,n.$ Note that this is without loss of generality, because ${\displaystyle {\sum }_{i=1}^n\mu (a_i)=}{\displaystyle {\sum }_{i:\mu (a_i)>0}\mu (a_i)}$. We will prove basic properties of this quantity. The results are illustrated with numerical examples.

Definition 8.

Let $\mu ,$ $\upsilon$ be two fuzzy P-measures on a fuzzy measurable space $(X,M),$ and $A=\left\{a_1,a_2,\dots ,a_n\right\}$ be a fuzzy partition of fuzzy probability spaces $(X,M,\mu ),$ $(X,M,\upsilon ).$ The R-norm divergence of fuzzy P-measures $\mu ,$$\upsilon$ with respect to $A$ is defined, for a positive real number R not equal to 1, as the number:

$$D_R^A(\mu \parallel \upsilon )=\frac{R}{R-1}\left({\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}-1\right).$$

(8)

Remark 3.

It is easy to see that, for any fuzzy partition $A$ of a fuzzy probability space $(X,M,\mu ),$ we have $D_R^A(\mu \parallel \mu )=0.$

The following theorem states that the R-norm divergence $D_R^A(\mu \parallel \upsilon )$ is consistent, in the case of the limit of R going to 1, with the Kullback–Leibler divergence $D_A(\mu \parallel \upsilon )$ defined by formula (3), up to a positive multiplicative constant.

Theorem 11.

Let $A=\left\{a_1,a_2,\dots ,a_n\right\}$ be a fuzzy partition of fuzzy probability spaces $(X,M,\mu ),$ $(X,M,\upsilon ).$ Then $\underset{R\to 1}{\lim }{D}_R^A(\mu \parallel \upsilon )=c\cdot D_A(\mu \parallel \upsilon ),$ where $D_A(\mu \parallel \upsilon )={\displaystyle {\sum }_{i=1}^n\mu (a_i)}\log \frac{\mu (a_i)}{\upsilon (a_i)},$ and $c=\frac{1}{\log e}.$

Proof. 

For every $R\in (0,1)\cup (1,\infty ),$ we can write:

$$D_R^A(\mu \parallel \upsilon )=\frac{1}{1-\frac{1}{R}}\left({\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}-1\right)=\frac{f(R)}{g(R)},$$

where $f,g$ are continuous functions defined for $R\in (0,\infty )$ by the formulas:

$$f(R)={\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}-1,g(R)=1-\frac{1}{R}.$$

By continuity of the functions $f,g,$ we get $\underset{R\to 1}{\lim }f(R)=f(1)={\displaystyle {\sum }_{i=1}^n\mu (a_i)\upsilon {(a_i)}^0}-1$ $={\displaystyle {\sum }_{i=1}^n\mu (a_i)}-1$$=1-1=0,$ and $\underset{R\to 1}{\lim }g(R)=g(1)=0.$ Using L’Hôpital’s rule this implies that:

$$\underset{R\to 1}{\lim }{D}_R^A(\mu \parallel \upsilon )=\frac{\underset{R\to 1}{\lim }{f}^{\prime }(R)}{\underset{R\to 1}{\lim }{g}^{\prime }(R)}$$

under the assumption that the right-hand side exists. Let us calculate the derivative of the function $f(R)$:

$$\begin{gathered} \frac{d}{dR}f(R)=e^{\frac{1}{R}\ln {\displaystyle \sum _{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}}}\cdot (-\frac{1}{R^2}\cdot \ln {\displaystyle \sum _{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}} \\ +\frac{1}{R}\cdot \frac{1}{{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}}}{\displaystyle \sum _{i=1}^n\left(-\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}\cdot \ln \upsilon (a_i)+\upsilon {(a_i)}^{1-R}\mu {(a_i)}^R\cdot \ln \mu (a_i)\right)}). \end{gathered}$$

Since $\underset{R\to 1}{\lim }{g}^{\prime }(R)=\underset{R\to 1}{\lim }\frac{1}{R^2}=1,$ we get:

$$\begin{gathered} \underset{R\to 1}{\lim }{D}_R^A(\mu \parallel \upsilon )=\underset{R\to 1}{\lim }{f}^{\prime }(R)={\displaystyle \sum _{i=1}^n\left(\mu (a_i)\cdot \ln \mu (a_i)-\mu (a_i)\cdot \ln \upsilon (a_i)\right)} \\ ={\displaystyle \sum _{i=1}^n\left(\mu (a_i)\cdot \frac{\log \mu (a_i)}{\log e}-\mu (a_i)\cdot \frac{\log \upsilon (a_i)}{\log e}\right)} \\ =\frac{1}{\log e}{\displaystyle \sum _{i=1}^n\mu (a_i)\cdot \log \frac{\mu (a_i)}{\upsilon (a_i)}}=\frac{1}{\log e}{D}_A(\mu \parallel \upsilon ).\square \end{gathered}$$

Remark 4.

Evidently, if the Kullback–Leibler divergence $D_A(\mu \parallel \upsilon )$ is expressed in terms of a natural logarithm, then it is the limiting case of the R-norm divergence $D_R^A(\mu \parallel \upsilon )$ for R going to 1.

Let $A=\left\{a_1,a_2,\dots ,a_n\right\}$ be a fuzzy partition of fuzzy probability spaces $(X,M,\mu ),$ $(X,M,\upsilon ).$ In [32], it has been shown that for the Kullback–Leibler divergence $D_A(\mu \parallel \upsilon )$ it holds the Gibbs inequality $D_A(\mu \parallel \upsilon )\ge 0$ with the equality if and only if $\mu (a_i)=\upsilon (a_i),$ for $i=1,2,\dots ,n.$ This result allows us to interpret the Kullback–Leibler divergence $D_A(\mu \parallel \upsilon )$ as a distance measure between two fuzzy P-measures (over the same fuzzy partition). In the following theorem, we present an analogy of this result for the case of the R-norm divergence.

Theorem 12.

Let $A=\left\{a_1,a_2,\dots ,a_n\right\}$ be a fuzzy partition of fuzzy probability spaces $(X,M,\mu ),$ $(X,M,\upsilon ).$ Then $D_R^A(\mu \parallel \upsilon )\ge 0$ with the equality if and only if $\mu (a_i)=\upsilon (a_i),$ for $i=1,2,\dots ,n.$

Proof. 

We shall consider two cases: the case of $R>1,$ and the case of $0<R<1.$

Consider the case of $R>1.$ The inequality follows from Jensen’s inequality for the function $\phi$ defined by $\phi (x)=x^{1-R},$ for every $x\in [0,\infty ),$ and putting ${\alpha }_i=\mu (a_i),$ $x_i=\frac{\upsilon (a_i)}{\mu (a_i)},$ for $i=1,2,\dots ,n.$ The assumption that $R>1$ implies $1-R<0,$ hence the function $\phi$ is convex. Therefore, by Jensen’s inequality we obtain:

$$1={\left({\displaystyle {\sum }_{i=1}^n\nu (a_i)}\right)}^{1-R}={\left({\displaystyle {\sum }_{i=1}^n\mu (a_i)\frac{\nu (a_i)}{\mu (a_i)}}\right)}^{1-R}\le {{\displaystyle {\sum }_{i=1}^n\mu (a_i)\left(\frac{\nu (a_i)}{\mu (a_i)}\right)}}^{1-R}={\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\nu {(a_i)}^{1-R}},$$

(9)

and consequently:

$${\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}\ge 1.$$

Since $\frac{R}{R-1}>0$ for $R>1,$ it follows that:

$$D_R^A(\mu \parallel \upsilon )=\frac{R}{R-1}\left({\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}-1\right)\ge 0.$$

For $0<R<1,$ the function $\phi$ defined by $\phi (x)=x^{1-R},$ for every $x\in [0,\infty ),$ is concave. Hence, using the Jensen inequality, we obtain:

$$1={\left({\displaystyle {\sum }_{i=1}^n\nu (a_i)}\right)}^{1-R}={\left({\displaystyle {\sum }_{i=1}^n\mu (a_i)\frac{\nu (a_i)}{\mu (a_i)}}\right)}^{1-R}\ge {{\displaystyle {\sum }_{i=1}^n\mu (a_i)\left(\frac{\nu (a_i)}{\mu (a_i)}\right)}}^{1-R}={\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\nu {(a_i)}^{1-R}},$$

and consequently:

$${\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}\le 1.$$

Since $\frac{R}{R-1}<0$ for $0<R<1,$ we conclude that:

$$D_R^A(\mu \parallel \upsilon )=\frac{R}{R-1}\left({\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}-1\right)\ge 0.$$

The equality in (9) holds if and only if $\frac{\nu (a_i)}{\mu (a_i)}$ is constant, for $i=1,2,\dots ,n,$ i.e., if and only if $\nu (a_i)=c\mu (a_i),$ for $i=1,2,\dots ,n.$ Taking the sum over all $i=1,2,\dots ,n,$ we get ${\displaystyle {\sum }_{i=1}^n\nu (a_i)}=$$c{\displaystyle {\sum }_{i=1}^n\mu (a_i)},$ which implies that $c=1.$ Therefore, $\nu (a_i)=\mu (a_i),$ for $i=1,2,\dots ,n.$ This means that $D_R^A(\mu \parallel \upsilon )=0$ if and only if $\mu (a_i)=\upsilon (a_i),$ for $i=1,2,\dots ,n.$ □

In the example that follows, it is shown that the equality $D_R^A(\mu \parallel \upsilon )=D_R^A(\upsilon \parallel \mu )$ is not necessarily true which means that the R-norm divergence $D_R^A(\mu \parallel \upsilon )$ is not symmetrical. Therefore, it is not a metric in a true sense.

Example 3.

Consider the fuzzy probability spaces $(X,M,\mu ),$ $(X,M,\upsilon )$ defined in Example 2 and the fuzzy partition $A=\left\{a,a^{\perp }\right\}$ of $(X,M,\mu ),$ $(X,M,\upsilon ).$ Let us calculate the R-norm divergencies $D_R^A(\mu \parallel \upsilon ),$ and $D_R^A(\upsilon \parallel \mu ):$

$$\begin{gathered} D_R^A(\mu \parallel \upsilon )=\frac{R}{R-1}\left({\left[{\left(\frac{1}{2}\right)}^R{\left(\frac{1}{3}\right)}^{1-R}+{\left(\frac{1}{2}\right)}^R{\left(\frac{2}{3}\right)}^{1-R}\right]}^{\frac{1}{R}}-1\right), \\ {D}_R^A(\upsilon \parallel \mu )=\frac{R}{R-1}\left({\left[{\left(\frac{1}{3}\right)}^R{\left(\frac{1}{2}\right)}^{1-R}+{\left(\frac{2}{3}\right)}^R{\left(\frac{1}{2}\right)}^{1-R}\right]}^{\frac{1}{R}}-1\right). \end{gathered}$$

Put $R=2.$ Elementary calculations show that $D_R^A(\mu \parallel \upsilon )\dot{=}0.1213,$ and $D_R^A(\upsilon \parallel \mu )\dot{=}0.1082,$ thus $D_2^A(\mu \parallel \upsilon )\ne D_2^A(\upsilon \parallel \mu ).$ For $R=\frac{1}{3},$ we have $D_R^A(\mu \parallel \upsilon )\dot{=}0.0188,$ and $D_R^A(\upsilon \parallel \mu )$$\dot{=}0.01908,$ i.e., $D_{1/3}^A(\mu \parallel \upsilon )\ne D_{1/3}^A(\upsilon \parallel \mu ).$ This means that $D_R^A(\mu \parallel \upsilon )\ne D_R^A(\upsilon \parallel \mu ),$ in general.

Theorem 13.

Let $\mu ,$ $\upsilon$ be two fuzzy P-measures on a fuzzy measurable space $(X,M),$ and $A=\left\{a_1,a_2,\dots ,a_n\right\}$ be a fuzzy partition of fuzzy probability spaces $(X,M,\mu ),$ $(X,M,\upsilon ).$ In addition, let $\upsilon$ be uniform over $A,$ i.e., $\upsilon (a_i)=\frac{1}{n},$ for $i=1,2,\dots ,n.$ Then, it holds that:

$$H_R^{\mu }(A)=\frac{R}{R-1}\left(1-n^{\frac{1-R}{R}}\right)-n^{\frac{1-R}{R}}\cdot D_R^A(\mu \parallel \upsilon ).$$

(10)

Proof. 

Let us calculate:

$$\begin{gathered} D_R^A(\mu \parallel \upsilon )=\frac{R}{R-1}\left({\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}-1\right) \\ =\frac{R}{R-1}\left({\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R{n}^{R-1}}\right]}^{\frac{1}{R}}-1\right)=\frac{R}{R-1}\cdot n^{\frac{R-1}{R}}{\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}-\frac{R}{R-1} \\ =-n^{\frac{R-1}{R}}\cdot \frac{R}{R-1}\left(1-{\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R}\right]}^{\frac{1}{R}}\right)+n^{\frac{R-1}{R}}\cdot \frac{R}{R-1}-\frac{R}{R-1} \\ =-n^{\frac{R-1}{R}}\cdot H_R^{\mu }(A)+\frac{R}{R-1}\left(n^{\frac{R-1}{R}}-1\right). \end{gathered}$$

It follows that:

$$H_R^{\mu }(A)=\frac{R}{R-1}\left(1-n^{\frac{1-R}{R}}\right)-n^{\frac{1-R}{R}}\cdot D_R^A(\mu \parallel \upsilon ).\square$$

Example 4.

Consider the fuzzy P-measures $\mu ,$ $\upsilon$ from Example 2 and the fuzzy partition $A=\left\{a,a^{\perp }\right\}$ of fuzzy probability spaces $(X,M,\mu ),$ $(X,M,\upsilon ).$ The fuzzy P-measure $\mu$ is uniform over A. Put $R=2.$ Based on previous results, we have $D_R^A(\upsilon \parallel \mu )\dot{=}0.1082,$ and $H_R^{\upsilon }(A)\dot{=}0.50928.$ Let us calculate:

$$\frac{R}{R-1}\left(1-2^{\frac{1-R}{R}}\right)-2^{\frac{1-R}{R}}\cdot D_R^A(\upsilon \parallel \mu )\dot{=}2\left(1-2^{-\frac{1}{2}}\right)-2^{-\frac{1}{2}}\cdot 0.1082\dot{=}0.50928.$$

Thus, the Equality (10) holds.

As a direct consequence of Theorems 12 and 13, we obtain the following property of the R-norm entropy of fuzzy partitions:

Corollary 1.

For arbitrary fuzzy partition $A=\left\{a_1,a_2,\dots ,a_n\right\}$ of a fuzzy probability space $(X,M,\mu ),$ it holds that:

$$H_R^{\mu }(A)\le \frac{R}{R-1}\left(1-n^{\frac{1-R}{R}}\right)$$

with the equality if and only if the fuzzy P-measure $\mu$ is uniform over A.

Theorem 14.

Let ${\mu }_1,$ ${\mu }_2,$ $\upsilon$ be fuzzy P-measures on a fuzzy measurable space $(X,M,{\mu }_1),$ and A be a fuzzy partition of fuzzy probability spaces $(X,M,{\mu }_2),$ $(X,M,\upsilon ).$ Then, for every real number $\lambda \in [0,1],$ it holds that:

$$D_R^A(\lambda {\mu }_1+(1-\lambda ){\mu }_2\parallel \upsilon )\le \lambda D_R^A({\mu }_1\parallel \upsilon )+(1-\lambda )D_R^A({\mu }_2\parallel \upsilon ).$$

Proof. 

Assume that $A=\left\{a_1,a_2,\dots ,a_n\right\},$ and $\lambda \in [0,1].$ Putting $x_i=\lambda {\mu }_1(a_i)\upsilon {(a_i)}^{\frac{1-R}{R}},$ and $y_i=(1-\lambda ){\mu }_2(a_i)\upsilon {(a_i)}^{\frac{1-R}{R}},$ $i=1,2,\dots ,n,$ in the Minkowski inequality, we get for $R>1:$

$$\begin{gathered} {\left[{\displaystyle {\sum }_{i=1}^n{(\lambda {\mu }_1(a_i)+(1-\lambda ){\mu }_2(a_i))}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}={\left[{\displaystyle {\sum }_{i=1}^n{\left((\lambda {\mu }_1(a_i)+(1-\lambda ){\mu }_2(a_i))\upsilon {(a_i)}^{\frac{1-R}{R}}\right)}^R}\right]}^{\frac{1}{R}} \\ ={\left[{\displaystyle {\sum }_{i=1}^n{\left(\lambda {\mu }_1(a_i)\upsilon {(a_i)}^{\frac{1-R}{R}}+(1-\lambda ){\mu }_2(a_i)\upsilon {(a_i)}^{\frac{1-R}{R}}\right)}^R}\right]}^{\frac{1}{R}} \\ \le {\left[{\displaystyle {\sum }_{i=1}^n{\left(\lambda {\mu }_1(a_i)\upsilon {(a_i)}^{\frac{1-R}{R}}\right)}^R}\right]}^{\frac{1}{R}}+{\left[{{\displaystyle {\sum }_{i=1}^n\left((1-\lambda ){\mu }_2(a_i)\upsilon {(a_i)}^{\frac{1-R}{R}}\right)}}^R\right]}^{\frac{1}{R}} \\ =\lambda {\left[{\displaystyle {\sum }_{i=1}^n{\mu }_1{(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}+(1-\lambda ){\left[{\displaystyle {\sum }_{i=1}^n{\mu }_2{(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}, \end{gathered}$$

and for $0<R<1:$

$$\begin{gathered} {\left[{\displaystyle {\sum }_{i=1}^n{(\lambda {\mu }_1(a_i)+(1-\lambda ){\mu }_2(a_i))}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}} \\ \ge {\left[{\displaystyle {\sum }_{i=1}^n{\left(\lambda {\mu }_1(a_i)\upsilon {(a_i)}^{\frac{1-R}{R}}\right)}^R}\right]}^{\frac{1}{R}}+{\left[{{\displaystyle {\sum }_{i=1}^n\left((1-\lambda ){\mu }_2(a_i)\upsilon {(a_i)}^{\frac{1-R}{R}}\right)}}^R\right]}^{\frac{1}{R}} \\ =\lambda {\left[{\displaystyle {\sum }_{i=1}^n{\mu }_1{(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}+(1-\lambda ){\left[{\displaystyle {\sum }_{i=1}^n{\mu }_2{(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}. \end{gathered}$$

This means that the function $\mu \mapsto {\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}$ is convex in $\mu$ for $R>1,$ and concave in $\mu$ for $0<R<1.$ The same applies to the function $\mu \mapsto {\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}-1.$ Since $\frac{R}{R-1}>0$ for $R>1,$ and $\frac{R}{R-1}<0$ for $0<R<1,$ we conclude that the function $\mu \mapsto D_R^A(\mu \parallel \upsilon )$ is convex on the family of all fuzzy P-measures on a given fuzzy measurable space $(X,M).$ Thus, for every real number $\lambda \in [0,1],$ it holds that:

$$D_R^A(\lambda {\mu }_1+(1-\lambda ){\mu }_2\parallel \upsilon )\le \lambda D_R^A({\mu }_1\parallel \upsilon )+(1-\lambda )D_R^A({\mu }_2\parallel \upsilon ).\square$$

Theorem 15.

Let $A=\left\{a_1,a_2,\dots ,a_n\right\}$ be any fuzzy partition of a fuzzy probability space $(X,M,\mu ).$ Then:

(i) 

$0<R<1$ implies $D_R^A(\mu \parallel \upsilon )\le D_A(\mu \parallel \upsilon );$

(ii) 

$R>1$ implies $D_R^A(\mu \parallel \upsilon )\ge D_A(\mu \parallel \upsilon ),$

where

$$D_A(\mu \parallel \upsilon )={\displaystyle {\sum }_{i=1}^n\mu (a_i)}\log \frac{\mu (a_i)}{\upsilon (a_i)}.$$

Proof. 

In the proof we use the Jensen inequality for the concave function $\phi$ defined by $\phi (x)=\log x,$ for $x\in (0,\infty ),$ and putting ${\alpha }_i=\mu (a_i),$ $x_i={\left(\frac{\upsilon (a_i)}{\mu (a_i)}\right)}^{R-1},$ for $i=1,2,\dots ,n.$ Since the logarithm satisfies the condition $\log x\le x-1,$ for all real numbers $x>0,$ we get:

$$\log {\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}\le {\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}-1.$$

(11)

Suppose that $0<R<1.$ Then $\frac{R}{R-1}<0$ and using the inequality (11) and the Jensen inequality, we can write:

$$\begin{gathered} D_R^A(\mu \parallel \upsilon )=\frac{R}{R-1}\left({\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}-1\right) \\ \le \frac{R}{R-1}\log {\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}=\frac{1}{R-1}\log {\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}} \\ =\frac{1}{R-1}\log {\displaystyle {\sum }_{i=1}^n\mu (a_i){\left(\frac{\mu (a_i)}{\upsilon (a_i)}\right)}^{R-1}}\le \frac{1}{R-1}{\displaystyle {\sum }_{i=1}^n\mu (a_i)}\log {\left(\frac{\mu (a_i)}{\upsilon (a_i)}\right)}^{R-1} \\ ={\displaystyle {\sum }_{i=1}^n\mu (a_i)}\log \frac{\mu (a_i)}{\upsilon (a_i)}=D_A(\mu \parallel \upsilon ). \end{gathered}$$

The case of $R>1$ can be obtained in the similar way. □

Example 5.

Consider the fuzzy P-measures $\mu ,$ $\upsilon$ from Example 2 and the fuzzy partition $A=\left\{a,a^{\perp }\right\}$ of fuzzy probability spaces $(X,M,\mu ),$ $(X,M,\upsilon ).$ Based on the results from Example 3, we have $D_2^A(\mu \parallel \upsilon )\dot{=}0.1213,$ $D_2^A(\upsilon \parallel \mu )\dot{=}0.1082,$ $D_{1/3}^A(\mu \parallel \upsilon )\dot{=}0.0188,$ $D_{1/3}^A(\upsilon \parallel \mu )\dot{=}0.01908.$ By simple calculation we get that $D_A(\mu \parallel \upsilon )\dot{=}0.084963,$ and $D_A(\upsilon \parallel \mu )\dot{=}0.081703.$ Thus, for $R=\frac{1}{3}$ we have $D_R^A(\mu \parallel \upsilon )\le D_A(\mu \parallel \upsilon ),$ $D_R^A(\upsilon \parallel \mu )\le D_A(\upsilon \parallel \mu );$ and for $R=2$ we have $D_R^A(\mu \parallel \upsilon )\ge D_A(\mu \parallel \upsilon ),$ and $D_R^A(\upsilon \parallel \mu )\ge D_A(\upsilon \parallel \mu ),$ which is consistent with the statement in the previous theorem.

We conclude our contribution with the formulation of a chain rule for the R-norm divergence in the fuzzy case. First, we define the conditional version of the R-norm divergence of fuzzy P-measures.

Definition 9.

Let $A=\left\{a_1,a_2,\dots ,a_n\right\},$ $B=\left\{b_1,b_2,\dots ,b_m\right\}$ be two fuzzy partitions of fuzzy probability spaces $(X,M,\mu ),$ $(X,M,\upsilon ).$ Then, we define the conditional divergence of fuzzy P-measures $\mu ,$ $\upsilon$ with respect to B assuming a realization of A, for a positive real number R not equal to 1, as the number:

$$D_R^{B/A}(\mu \parallel \upsilon )=\frac{R}{R-1}\left({\left[{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j=1}^m\mu {(a_i\cap b_j)}^R\upsilon {(a_i\cap b_j)}^{1-R}}}\right]}^{\frac{1}{R}}-{\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}\right).$$

Theorem 16.

Let $A,B$ be two fuzzy partitions of fuzzy probability spaces $(X,M,\mu ),$ $(X,M,\upsilon ).$ Then

$$D_R^{A\vee B}(\mu \parallel \upsilon )=D_R^A(\mu \parallel \upsilon )+D_R^{B/A}(\mu \parallel \upsilon ).$$

Proof. 

Assume that $A=\left\{a_1,a_2,\dots ,a_n\right\},$ $B=\left\{b_1,b_2,\dots ,b_m\right\}.$ Then we have:

$$\begin{gathered} D_R^A(\mu \parallel \upsilon )+D_R^{B/A}(\mu \parallel \upsilon )=\frac{R}{R-1}\left({\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}-1\right) \\ +\frac{R}{R-1}\left({\left[{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j=1}^m\mu {(a_i\cap b_j)}^R\upsilon {(a_i\cap b_j)}^{1-R}}}\right]}^{\frac{1}{R}}-{\left[{\displaystyle {\sum }_{i=1}^n\mu {(a_i)}^R\upsilon {(a_i)}^{1-R}}\right]}^{\frac{1}{R}}\right) \\ =\frac{R}{R-1}\left({\left[{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j=1}^m\mu {(a_i\cap b_j)}^R\upsilon {(a_i\cap b_j)}^{1-R}}}\right]}^{\frac{1}{R}}-1\right)=D_R^{A\vee B}(\mu \parallel \upsilon ).\square \end{gathered}$$

5. Conclusions

In this article, we have extended the study of entropy measures and distance measures in the fuzzy case. Our goal was to introduce the concepts of R-norm entropy and R-norm divergence for the case of fuzzy probability spaces and to derive basic properties of these measures. Our results are presented in Section 3 and Section 4.

In Section 3, we have defined the R-norm entropy and conditional R-norm entropy of fuzzy partitions of a given fuzzy probability space and have examined the properties of the proposed entropy measures. In particular, it has been shown that the R-norm entropy of fuzzy partitions does not have the property of additivity, but it satisfies the property called pseudo-additivity, as stated in Theorem 10. In Theorem 6, chain rules for the R-norm entropy of fuzzy partitions are provided. Moreover, it was shown that the Shannon entropy and the conditional Shannon entropy of fuzzy partitions can be derived from the R-norm entropy and conditional R-norm entropy of fuzzy partitions, respectively, as the limiting cases for $R\to 1.$

In Section 4, the concept of R-norm divergence of fuzzy P-measures was introduced and the properties of this quantity have been proven. Specifically, it was shown that the Kullback–Leibler divergence defined and studied in [32] can be derived from the R-norm divergence of fuzzy P-measures, as the limiting case for $R\to 1.$ The result of Theorem 12 allows us to interpret the R-norm divergence as a distance measure between two fuzzy P-measures. Theorem 13 provides a relationship between the R-norm divergence and the R-norm entropy of fuzzy partitions; Theorem 15 provides a relationship between the R-norm divergence and the Kullback–Leibler divergence of fuzzy P-measures. In addition, the concavity of R-norm entropy (Theorem 7) and convexity of R-norm divergence (Theorem 14) have been demonstrated. Finally, using the suggested concept of conditional R-norm divergence of fuzzy P-measures, the chain rule for the R-norm divergence of fuzzy P-measures was established.

In the proofs, the Jensen inequality, L’Hôpital’s rule, and the Minkowski inequality were used. The results presented in Section 3 and Section 4 are illustrated with numerical examples.