Rényi Entropy and Rényi Divergence in Product MV-Algebras

Abstract

This article deals with new concepts in a product MV-algebra, namely, with the concepts of Rényi entropy and Rényi divergence. We define the Rényi entropy of order q of a partition in a product MV-algebra and its conditional version and we study their properties. It is shown that the proposed concepts are consistent, in the case of the limit of q going to 1, with the Shannon entropy of partitions in a product MV-algebra defined and studied by Petrovičová (Soft Comput. 2000, 4, 41–44). Moreover, we introduce and study the notion of Rényi divergence in a product MV-algebra. It is proven that the Kullback–Leibler divergence of states on a given product MV-algebra introduced by Markechová and Riečan in (Entropy 2017, 19, 267) can be obtained as the limit of their Rényi divergence. In addition, the relationship between the Rényi entropy and the Rényi divergence as well as the relationship between the Rényi divergence and Kullback–Leibler divergence in a product MV-algebra are examined.

1. Introduction

The Shannon entropy [1] and Kullback–Leibler divergence [2] are the most significant and most widely used quantities in information theory [3]. Due to their successful use, many attempts have been done to generalize them. It is known that their important generalizations are the Rényi entropy and Rényi divergence [4], respectively. These quantities have many significant applications; for example, in statistics, in ecology, and also in quantum information.

Shannon’s entropy is defined in the context of a probabilistic model in the following way: if we consider a probability space $(\mathsf{\Omega },S,P)$ and a measurable partition $ℰ$ $=\left\{E_1,E_2,\dots ,E_n\right\}$ of $(\mathsf{\Omega },S,P),$ then the Shannon entropy of $ℰ$ is defined as the number $H\left(ℰ\right)$$=-{\displaystyle {\sum }_{i=1}^{n}P(E_i)·\log P(E_i)}$ (with the usual convention that $P(E_i)·\log P(E_i)=0,$ for $P(E_i)=0).$ If $ℰ$$=\left\{E_1,E_2,\dots ,E_n\right\}$ and $ℱ$$=\left\{F_1,F_2,\dots ,F_m\right\}$ are two measurable partitions of $(\mathsf{\Omega },S,P),$ then the conditional Shannon entropy of $ℰ$ assuming a realization of $ℱ$ is defined as the number $H\left(ℰ/ℱ\right)$ $=-{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j=1}^{m}P(E_i\cap F_j)}}\cdot \log \frac{P(E_i\cap F_j)}{P(F_j)}$ (with the usual convention that $0\cdot \log \frac{0}{x}=0$ if $x\ge 0$). If $ℰ$ $=\left\{E_1,E_2,\dots ,E_n\right\}$ is a measurable partition of $(\mathsf{\Omega },S,P)$ with probabilities $p_i=P(E_i),$ $i=1,2,\dots ,n,$ then its Rényi entropy of order $q,$ where $q\in (0,1)\cup (1,\infty ),$ is defined as the number $H_q\left(ℰ\right)$ $=\frac{1}{1-q}\log {\displaystyle {\sum }_{i=1}^n{p}_i^q}.$ It can be shown that $\underset{q\to 1}{\lim }{H}_q$ ($ℰ$) $={\displaystyle {\sum }_{i=1}^n{p}_i}\cdot \log \frac{1}{p_i},$ thus the Shannon entropy is a limiting case of the Rényi entropy for $q\to 1$. It is known that there is no universally accepted definition of conditional Rényi entropy. The paper [5] describes three definitions of conditional Rényi entropy that can be found in the literature. In [6], it is also possible to find a brief overview of various approaches to defining the conditional Rényi entropy, and in addition, a new definition of conditional Rényi entropy was proposed. In [7], the authors introduced a general type of conditional Rényi entropy which contains some of previously defined conditional Rényi entropies as special cases. The proposed concepts have successfully been used in information theory [8], time series analysis [9], and cryptographic applications [10]. However, no one of the proposed generalizations satisfies all basic properties of Shannon conditional entropy. The selection of the definition therefore depends on the purpose of application.

The present article is devoted to the study of Rényi entropy and Rényi divergence in product MV-algebras. An MV-algebra [11] is the most useful instrument for describing multivalued processes, especially after its Mundici’s characterization as an interval in a lattice ordered group (cf. [12,13]). At present, this algebraic structure is being studied by many researchers and it is natural that there are results also regarding entropy in this structure; we refer, for instance, to [14,15]. Also, a measure theory (cf. [16]) and a probability theory (cf. [17]) were studied on MV-algebras. Of course, in some problems of probability it is necessary to introduce a product on an MV-algebra, an operation outside the corresponding group addition. The operation of a product on an MV-algebra was suggested independently in [18] from the viewpoint of mathematical logic and in [19] from the viewpoint of probability. Also, the approach from the viewpoint of a general algebra suggested in [20] seems interesting. We note that the notion of a product MV-algebra generalizes some classes of fuzzy sets; a full tribe of fuzzy sets (see e.g., [21]) presents an example of a product MV-algebra.

A suitable entropy theory of Shannon and Kolmogorov-Sinai type for the case of a product MV-algebra has been provided by Petrovičová in [22,23]. We remark that in our article [24], based on the results of Petrovičová, we proposed the notions of Kullback–Leibler divergence and mutual information of partitions in a product MV-algebra. In the present article, we continue studying entropy and divergence in a product MV-algebra, by defining and studying the Rényi entropy and the Rényi divergence.

The rest of the paper is structured as follows. In the following section, preliminaries and related works are given. Our main results are discussed in Section 3, Section 4 and Section 5. In Section 3, we define the Rényi entropy of a partition in a product MV-algebra and examine its properties. It is shown that for $q\to 1$ the Rényi entropy of order q converges to the Shannon entropy of a partition in a product MV-algebra introduced in [22]. In Section 4, we introduce the concept of conditional Rényi entropy of partitions in a product MV-algebra and study its properties. It is shown that the proposed definition of the conditional Rényi entropy is consistent, in the case of the limit of q going to 1, with the conditional Shannon entropy of partitions studied in [22], and that it satisfies the property of monotonicity and a weak chain rule. In the final part of this section, we define the Rényi information about a partition X in a partition Y as an example for the further usage of the proposed concept of the conditional Rényi entropy. Section 5 is devoted to the study of Rényi divergence in a product MV-algebra. It is shown that the Kullback–Leibler divergence in a product MV-algebra introduced by the authors in [24] can be obtained as the limit of the Rényi divergence. We illustrate the results with numerical examples. The last section contains a brief summary.

2. Preliminaries and Related Works

We start by reminding the definitions of basic terms and some of the known results that will be used in the article. We mention some works related to the subject of this article, of course, without claiming completeness.

Several different (but equivalent) axiom systems have been used to define the term of MV-algebra (cf., e.g., [19,25,26]). In this paper, we use the definition of MV-algebra given by Riečan in [27], which is based on the Mundici representation theorem. Based on Mundici’s theorem [12] (see also [13]), MV-algebras can be considered as intervals of an abelian lattice-ordered group (shortly l-group). We remind that by an l-group (cf. [28]) we understand a triplet $(G,+,\le ),$ where $(G,+)$ is an abelian group, $(G,\le )$ is a partially ordered set being a lattice and $x\le y\Rightarrow x+z\le y+z.$

Definition 1

([27]). An MV-algebra is an algebraic structure $\mathcal{A}$ $=(A,\oplus ,\ast ,0,u)$ satisfying the following conditions:

(i) 

there exists an l-group$(G,+,\le )$such that$A=[0,u]=\{x\in G;0\le x\le u\},$where 0 is the neutral element of$(G,+)$and u is a strong unit of G (i.e.,$u\in G$such that$u>0$and to every$x\in G$there exists a positive integer$n$with the property$x\le nu$;

(ii) 

$\oplus$and$\ast$are binary operations on A satisfying the following identities:$x\oplus y=(x+y)\wedge u,$$x\ast y=(x+y-u)\vee 0$.

We note that MV-algebras provide a generalization of Boolean algebras in the sense that every Boolean algebra is an MV-algebra satisfying the condition $x\oplus x=x.$ For this reason, in order to generalize the concept of probability on Boolean algebras, Mundici introduced in [29] the notion of a state on an MV-algebra in the following way. Let $\mathcal{A}$ $=(A,\oplus ,\ast ,0,u)$ be an MV-algebra. A mapping $s:A\to [0,1]$ is a state on $\mathcal{A}$ whenever $s(0)=0$ and, for every $x,y\in A,$ the following condition is satisfied: if $x\ast y=0,$ then $s(x\oplus y)=s(x)+s(y).$ Since the definitions of product MV-algebra and partition in a product MV-algebra are based on the Mundici representation theorem (i.e., the MV-algebra operation $\oplus$ is substituted by the group operation $+$ in the abelian l-group corresponding to the considered MV-algebra), in this contribution, we shall use the following (equivalent) definition of a state which is also based on the Mundici representation theorem. This means that the sum in the following definition of a state, and subsequently in what follows, denotes the sum in the abelian l-group that corresponds to the given MV-algebra.

Definition 2

([27]). A state on an MV-algebra $\mathcal{A}$$=(A,\oplus ,\ast ,0,u)$ is a mapping $s:A\to [0,1]$ with the following two properties:

(i) 

$s(u)=1;$

(ii) 

if$x,y\in A$such that$x+y\le u,$then$s(x+y)=s(x)+s(y).$

Definition 3

([19]). A product MV-algebra is an algebraic structure $(A,\oplus ,\ast ,\cdot ,0,u),$ where $(A,\oplus ,\ast ,0,u)$ is an MV-algebra and $\cdot$ is an associative and abelian binary operation on A with the following properties:

(i) 

for every$x\in A,$$u\cdot x=x;$

(ii) 

if$x,y,z\in A$such that$x+y\le u,$then$z\cdot x+z\cdot y\le u,$and$z\cdot (x+y)=z\cdot x+z\cdot y.$

For brevity, we will write $(A,\cdot )$ instead of $(A,\oplus ,\ast ,\cdot ,0,u).$ A relevant probability theory for the product MV-algebras was developed by Riečan in [30], see also [31,32]; the entropy theory of Shannon and Kolmogorov-Sinai type for the product MV-algebras was proposed in [22,23]. We present the main idea and some results of these theories that will be used in the following text.

As in [22], by a partition in a product MV-algebra $(A,\cdot ),$ we understand any n-tuple $X=(x_1,x_2,\dots ,x_n)$ of elements of A with the property $x_1+x_2+\dots +x_n=u.$ In the system of all partitions in a given product MV-algebra $(A,\cdot ),$ we define the refinement partial order $\succ$ in a standard way (cf. [33]). If $X=(x_1,x_2,\dots ,x_n),$ and $Y=(y_1,y_2,\dots ,y_m)$ are two partitions in $(A,\cdot ),$ then we write $Y\succ X$ (and we say that $Y$ is a refinement of $X),$ if there exists a partition $\left\{J(1),J(2),\dots ,J(n)\right\}$ of the set $\left\{1,2,\dots ,m\right\}$ such that $x_i={\displaystyle {\sum }_{j\in J(i)}{y}_j},$ for $i=1,2,\dots ,n.$ Further, for two finite sequences $X=(x_1,x_2,\dots ,x_n),$ and $Y=(y_1,y_2,\dots ,y_m)$ of elements of A, we put $X\vee Y=(x_i\cdot y_j;i=1,2,\dots ,n,$ $j=1,2,\dots ,m).$ If $X=(x_1,x_2,\dots ,x_n),$ and $Y=(y_1,y_2,\dots ,y_m)$ are partitions in $(A,\cdot ),$ then the system $X\vee Y$ is a partition in $(A,\cdot ),$ since ${\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j=1}^m{x}_i\cdot y_j}}$$=\left({\displaystyle {\sum }_{i=1}^n{x}_i}\right)\cdot \left({\displaystyle {\sum }_{j=1}^m{y}_j}\right)=u\cdot u=u.$ Later we shall need the following assertion:

Proposition 1.

If$X,Y,Z$are partitions in a product MV-algebra$(A,\cdot ),$then it holds$X\vee Y\succ X,$and$Y\succ X$implies$Y\vee Z\succ X\vee Z.$

Proof. 

The proof can be found in [33]. ☐

The following example shows that the model studied in this article generalizes the classical case.

Example 1.

Let us consider a probability space$(\mathsf{\Omega },S,P)$and put$A=\left\{I_E;E\in S\right\},$where$I_E:\mathsf{\Omega }\to \left\{0,1\right\}$is the indicator function of the set$E\in S.$The class A is closed with respect to the product of indicator functions and it represents a special case of product MV-algebras. The mapping$s:A\to [0,1]$defined, for every$I_E\in A,$by$s(I_E)=P(E),$is a state on the considered product MV-algebra$(A,\cdot ).$A measurable partition$\left\{E_1,E_2,\dots ,E_n\right\}$of$(\mathsf{\Omega },S,P)$can be viewed as a partition in the product MV-algebra$(A,\cdot ),$if we consider the n-tuple$\left(I_{E_1},I_{E_2},\dots ,I_{E_n}\right)$instead of$\left\{E_1,E_2,\dots ,E_n\right\}.$

Example 2.

Let$(\mathsf{\Omega },S,P)$be a probability space, and A be the class of all$S-$measurable functions$f:\mathsf{\Omega }\to [0,1],$so called full tribe of fuzzy sets (cf., e.g., [21,34]). The class A is closed with respect to the natural product of fuzzy sets and it represents a significant case of product MV-algebras. The map$s:A\to [0,1]$defined, for every$f\in A,$by$s(f)={\displaystyle {\int }_{\mathsf{\Omega }}f}dP,$is a state on the product MV-algebra$(A,\cdot ).$The notion of a partition in the product MV-algebra$(A,\cdot )$coincides with the notion of a fuzzy partition (cf. [34]).

Definition 4.

Let$s$be a state on a product MV-algebra$(A,\cdot ).$We say that partitions$X,Y$in$(A,\cdot )$are statistically independent with respect to$s,$if$s(x\cdot y)=s(x)\cdot s(y),$for every$x\in X,$and$y\in Y.$

The following definition of entropy of Shannon type was introduced in [22].

Definition 5.

Let$X=(x_1,x_2,\dots ,x_n)$be a partition in a product MV-algebra$(A,\cdot )$and let$s:A\to [0,1]$be a state. Then the entropy of$X$with respect to$s$is defined by Shannon’s formula

$$H_s(X)=-{\displaystyle {\sum }_{i=1}^{n}s(x_i)}\cdot \log s(x_i).$$

(1)

If$X=(x_1,x_2,\dots ,x_n),$and$Y=(y_1,y_2,\dots ,y_m)$are two partitions in$(A,\cdot ),$then the conditional entropy of$X$given$y_j\in Y$is defined by

$$H_s(X/y_j)=-{\displaystyle \sum _{i=1}^{n}s(x_i/y_j)}\cdot \log s(x_i/y_j),$$

where

$$s(x_i/y_j)=\{\begin{array}{ll}\frac{s(x_i\cdot y_j)}{s(y_j)},& \mathrm{if} s(y_j)>0;\\ 0,& \mathrm{if} s(y_j)=0.\end{array}$$

(2)

The conditional entropy of$X$given$Y$is defined by

$$H_s(X/Y)={\displaystyle \sum _{j=1}^{m}s(y_j)}\cdot H_s(X/y_j)=-{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^{m}s(x_i\cdot y_j)}}\cdot \log \frac{s(x_i\cdot y_j)}{s(y_j)}.$$

(3)

Here, the usual convention that $0\log \frac{0}{x}=0$ if $x\ge 0$ is used. The base of the logarithm can be any positive real number, but as a rule one takes logarithms to the base 2. The entropy is then expressed in bits. The entropy and the conditional entropy of partitions in a product MV-algebra satisfy all properties that correspond to properties of Shannon’s entropy of measurable partitions in the classical case; for more details, see [22].

In [24], the concepts of mutual information and Kullback–Leibler divergence in a product MV-algebra were introduced in the following way.

Definition 6.

Let$X,Y$be partitions in a product MV-algebra$(A,\cdot ),$and let$s:A\to [0,1]$be a state. We define the mutual information between X and Y by the formula

$$I_s(X,Y)=H_s(X)-H_s(X/Y).$$

(4)

Definition 7.

Let$s,t$be states defined on a given product MV-algebra$(A,\cdot ),$and$X=(x_1,x_2,\dots ,x_n)$be a partition in$(A,\cdot ).$Then we define theKullback–Leiblerdivergence$D_X(s\parallel t)$by the formula

$$D_X(s\parallel t)={\displaystyle {\sum }_{i=1}^{n}s(x_i)\cdot \log \frac{s(x_i)}{t(x_i)}}.$$

(5)

The logarithm in this formula is taken to the base 2 and information is measured in units of bits. We use the convention that $x\log \frac{x}{0}=\infty$ if $x>0,$ and $0\log \frac{0}{x}=0$ if $x\ge 0.$

In the proofs, we shall use the Jensen inequality which states that for a real concave function $\phi ,$ real numbers $a_1,a_2,\dots ,a_n$ in its domain and nonnegative real numbers $c_1,c_2,\dots ,c_n$ such that ${\displaystyle {\sum }_{i=1}^n{c}_i=1,}$ it holds

$$\phi \left({\displaystyle {\sum }_{i=1}^n{c}_i{a}_i}\right)\ge {\displaystyle {\sum }_{i=1}^n{c}_i\phi (a_i}),$$

(6)

and the inequality is reversed if $\phi$ is a real convex function. The equality holds if and only if $a_1=a_2=\dots =a_n$ or $\phi$ is linear.

Further, we recall the following notions.

Definition 8.

Let D be a non-empty set and$f:D\to \Re$be a real function defined on it. Then the support of$f$is defined by supp$(f)=\left\{x\in D;f(x)\ne 0\right\}.$

Definition 9.

Let$f:D\to \Re$be a real function defined on a non-empty set D. Define the$q$-norm, for$1\le q<\infty ,$or$q$-quasinorm, for$0<q<1,$of$f$as

$$\parallel f{\parallel }_q={\left({\displaystyle {\sum }_{x\in D}{\left|f(x)\right|}^q}\right)}^{\frac{1}{q}}.$$

3. The Rényi Entropy of a Partition in a Product MV-Algebra

In this section, we define the Rényi entropy of a partition in a product MV-algebra $(A,\cdot ),$ and examine its properties. In the following, we assume that $s:A\to [0,1]$ is a state.

Definition 10.

Let$X=(x_1,x_2,\dots ,x_n)$be a partition in a product MV-algebra$(A,\cdot ).$Then we define the Rényi entropy of order$q,$where$q\in (0,1)\cup (1,\infty ),$of the partition$X$with respect to$s$as the number:

$$H_q^s(X)=\frac{1}{1-q}\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}.$$

(7)

Remark 1.

In accordance with the classical theory, the log is to the base 2 and the Rényi entropy is expressed in bits. For simplicity, we write$s{(x_i)}^q$instead of${\left(s(x_i)\right)}^q$and$\log {\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^q}$instead of$\log \left({\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^q}\right).$

Let $X=(x_1,x_2,\dots ,x_n)$ be a partition in $(A,\cdot ).$ If we consider the function $s_X:X\to \Re ,$ defined, for every $x_i\in X,$ by $s_X(x_i)=s(x_i),$ then we have

$$\parallel s_X{\parallel }_q={\left({\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^q}\right)}^{\frac{1}{q}},$$

and the Formula (7) can be expressed in the following equivalent form:

$$H_q^s(X)=\frac{q}{1-q}\log \left(\parallel s_X{\parallel }_q\right).$$

(8)

Example 3.

Let$X=(x_1,x_2,\dots ,x_n)$be any partition in a product MV-algebra$(A,\cdot ).$Let$s:A\to [0,1]$be a uniform state over X, i.e.,$s(x_i)=\frac{1}{n},$for$i=1,2,\dots ,n.$Then

$$H_q^s(X)=\frac{1}{1-q}\log {\displaystyle \sum _{i=1}^n{\left(\frac{1}{n}\right)}^q}=\frac{1}{1-q}\log n^{1-q}=\log n.$$

Example 4.

Let us consider any product MV-algebra$(A,\cdot ),$and the partition$E=(u)$in$(A,\cdot )$that represents an experiment resulting in a certain event. It is easy to see that$H_q^s(E)=0$.

Remark 2.

It is possible to verify that the Rényi entropy$H_q^s(X)$is always nonnegative. Namely, for$q\in (0,1),$and$i=1,2,\dots ,n,$we have$s{(x_i)}^q\ge s(x_i),$hence${\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^q}\ge {\displaystyle {\sum }_{i=1}^{n}s(x_i)}=s\left({\displaystyle {\sum }_{i=1}^n{x}_i}\right)=s(u)=1$. It follows that$H_q^s(X)=\frac{1}{1-q}\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}\ge 0$. On the other hand, for$q\in (1,\infty ),$and$i=1,2,\dots ,n,$it holds$s{(x_i)}^q\le s(x_i),$hence${\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}\le {\displaystyle \sum _{i=1}^{n}s(x_i)}=1$. The assumption$q\in (1,\infty )$implies$\frac{1}{1-q}<0,$therefore, we get$H_q^s(X)=\frac{1}{1-q}\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}\ge 0$.

At $q=1$ the value of the quantity $H_q^s(X)$ is undefined since it generates the form $\frac{0}{0}.$ In the following theorem, it is shown that for $q\to 1$ the Rényi entropy $H_q^s(X)$ converges to the Shannon entropy of a partition X in $(A,\cdot )$ defined by the formula (1).

Theorem 1.

Let$X=(x_1,x_2,\dots ,x_n)$be any partition in a product MV-algebra$(A,\cdot ).$Then

$$\underset{q\to 1}{\lim }{H}_q^s(X)=-{\displaystyle {\sum }_{i=1}^{n}s(x_i)}\cdot \log s(x_i).$$

Proof. 

Put $f(q)=\log {\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^q},$ and $g(q)=1-q,$ for every $q\in (0,\infty ).$ The functions $f,g$ are differentiable and for every $q\in (0,1)\cup (1,\infty ),$ we have $H_q^s(X)=\frac{f(q)}{g(q)}.$ Evidently, $\underset{q\to 1}{\lim }g(q)=0,$ and $\underset{q\to 1}{\lim }f(q)=\underset{q\to 1}{\lim }\log {\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^q}=\log {\displaystyle {\sum }_{i=1}^{n}s(x_i)=\log 1=0}$. Using L’Hôpital’s rule, this yields that $\underset{q\to 1}{\lim }{H}_q^s(X)=\underset{q\to 1}{\lim }\frac{f^{\prime }(q)}{g^{\prime }(q)},$ under the assumption that the right hand side exists. It holds $\frac{d}{dq}g(q)=-1,$ and

$$\begin{array}{c}\frac{d}{dq}f(q)=\frac{1}{(\ln 2){\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^q}}{\displaystyle \sum _{i=1}^n\frac{d}{dq}}s{(x_i)}^q\\ =\frac{1}{(\ln 2){\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^q}}{\displaystyle \sum _{i=1}^{n}s{(x_i)}^q\ln }s(x_i)=\frac{1}{{\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^q}}{\displaystyle \sum _{i=1}^{n}s{(x_i)}^q\log }s(x_i).\end{array}$$

Note that the calculation of the derivative of function f is easily done by using the identity $b^{\alpha }=e^{\alpha \ln b}.$ We get

$$\underset{q\to 1}{\lim }{H}_q^s(X)=\underset{q\to 1}{\lim }\frac{-1}{{\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^q}}{\displaystyle \sum _{i=1}^{n}s{(x_i)}^q\log }s(x_i)=-{\displaystyle \sum _{i=1}^{n}s(x_i)\log }s(x_i),$$

which is the Shannon entropy of X defined by the Formula (1). ☐

In the following theorem it is proved that the function $H_q^s(X)$ is monotonically decreasing in $q.$

Theorem 2.

Let X be any partition in a given product MV-algebra$(A,\cdot ),$and$q_1,q_2\in (0,1)\cup (1,\infty ).$Then$q_1\ge q_2$implies$H_{q_1}^s(X)\le H_{q_2}^s(X).$

Proof. 

Suppose that X = x₁, x₂, …, x_n and $q_1,q_2\in (1,\infty )$ such that $q_1\ge q_2.$ Then the claim is equivalent to the inequality

$${\left({\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^{q_1}}\right)}^{\frac{1}{q_1-1}}\ge {\left({\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^{q_2}}\right)}^{\frac{1}{q_2-1}}.$$

The above inequality follows by applying the Jensen inequality to the function $\phi ,$ defined, for every $x\in \left[0,\infty \right),$ by $\phi (x)=x^{\frac{q_2-1}{q_1-1}}.$ The assumption $q_1\ge q_2$ implies $\frac{q_2-1}{q_1-1}\le 1,$ hence the function $\phi$ is concave. Putting $c_i=s(x_i),$ and $a_i=s{(x_i)}^{q_1-1},$ $i=1,2,\dots ,n,$ in the inequality (6), we get

$$\begin{array}{c}{\left({\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^{q_1}}\right)}^{\frac{1}{q_1-1}}={\left({\displaystyle {\sum }_{i=1}^{n}s(x_i)s{(x_i)}^{q_1-1}}\right)}^{\frac{q_2-1}{(q_1-1)(q_2-1)}}\\ ={\left({\left[{\displaystyle {\sum }_{i=1}^{n}s(x_i)s{(x_i)}^{q_1-1}}\right]}^{\frac{q_2-1}{q_1-1}}\right)}^{\frac{1}{q_2-1}}\ge {\left({\displaystyle {\sum }_{i=1}^{n}s(x_i)s{(x_i)}^{q_2-1}}\right)}^{\frac{1}{q_2-1}}={\left({\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^{q_2}}\right)}^{\frac{1}{q_2-1}}.\end{array}$$

The case where $q_1,q_2\in (0,1)$ is obtained in an analogous way. Finally, the case where $q_1\in (1,\infty ),$ and $q_2\in (0,1)$ is obtained by transitivity. ☐

Example 5.

Consider any product MV-algebra$(A,\cdot ),$and a state$s:A\to [0,1].$Let$x\in A$with$s(x)=p,$where$p\in (0,1).$Then$s(u-x)=1-p,$and the pair$X=(x,u-x)$is a partition in$(A,\cdot ).$If we put$p=\frac{1}{2},$then we have$H_q^s(X)=1$bit, for every$q\in (0,1)\cup (1,\infty ).$Put$p=\frac{1}{3}.$By simple calculations we get$H_{1/2}^s(X)\dot{=}0.958$bit,$H_2^s(X)\dot{=}0.848$bit,$H_{1/3}^s(X)\dot{=}0.972$bit. So, it holds$H_2^s(X)<H_{1/2}^s(X)<H_{1/3}^s(X),$which is consistent with the property proven in the previous theorem.

Theorem 3.

Let$X,Y$be partitions in a product MV-algebra$(A,\cdot )$such that$Y\succ X.$Then$H_q^s(X)\le H_q^s(Y).$

Proof. 

Suppose that $X=(x_1,x_2,\dots ,x_n),$$Y=(y_1,y_2,\dots ,y_m),$$Y\succ X.$ Then there exists a partition $\left\{J(1),J(2),\dots ,J(n)\right\}$ of the set $\left\{1,2,\dots ,m\right\}$ such that $x_i={\displaystyle {\sum }_{j\in J(i)}{y}_j},$ for $i=1,2,\dots ,n.$ Hence $s(x_i)=s\left({\displaystyle {\sum }_{j\in J(i)}{y}_j}\right)={\displaystyle {\sum }_{j\in J(i)}s(y_j}),$ for $i=1,2,\dots ,n.$

(i)

Consider the case when $q\in (1,\infty ).$ Then $s{(x_i)}^q={\left({\displaystyle {\sum }_{j\in J(i)}s(y_j})\right)}^q\ge {\displaystyle {\sum }_{j\in J(i)}s(y_j}{)}^q,$ for $i=1,2,\dots ,n,$ and consequently

$${\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^q}\ge {\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j\in J(i)}s(y_j}{)}^q}={\displaystyle {\sum }_{j=1}^{m}s{(y_j)}^q}.$$

Therefore, we get

$$\log {\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^q}\ge \log {\displaystyle {\sum }_{j=1}^{m}s{(y_j)}^q}.$$

In this case we have $\frac{1}{1-q}<0,$ hence

$$H_q^s(X)=\frac{1}{1-q}\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}\le \frac{1}{1-q}\log {\displaystyle \sum _{j=1}^{m}s{(y_j)}^q}=H_q^s(Y).$$

(ii)

Consider the case when $q\in (0,1).$ Then $s{(x_i)}^q={\left({\displaystyle {\sum }_{j\in J(i)}s(y_j})\right)}^q\le {\displaystyle {\sum }_{j\in J(i)}s(y_j}{)}^q,$ for $i=1,2,\dots ,n,$ and consequently

$${\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^q}\le {\displaystyle {\sum }_{j=1}^{m}s{(y_j)}^q}.$$

Therefore, we get

$$\log {\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^q}\le \log {\displaystyle {\sum }_{j=1}^{m}s{(y_j)}^q}.$$

In this case we have $\frac{1}{1-q}>0,$ hence

$$H_q^s(X)=\frac{1}{1-q}\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}\le \frac{1}{1-q}\log {\displaystyle \sum _{j=1}^{m}s{(y_j)}^q}=H_q^s(Y).$$

☐

As an immediate consequence of the previous theorem and Proposition 1, we obtain the following result.

Corollary 1.

For every partition$X,Y$in a product MV-algebra$(A,\cdot ),$it holds

$$H_q^s(X\vee Y)\ge \max \left[H_q^s(X),H_q^s(Y)\right].$$

(9)

Example 6.

Consider the measurable space$([0,1],ℬ),$where$ℬ$is the$\sigma -$algebra of all Borel subsets of the unit interval$[0,1].$Let A be the family of all Borel measurable functions$f:[0,1]\to [0,1].$If we define in the family A the operation$\cdot$as the natural product of fuzzy sets, then the system$(A,\cdot )$is a product MV-algebra. We define a state$s:A\to [0,1]$by the equality$s(f)={\displaystyle {\int }_0^{1}f(x)dx},$for any element$f$of A. It is easy to see that the pairs$X=(f_1,f_2),$$Y=(g_1,g_2),$where$f_1(x)=x,$$f_2(x)=1-x,$$g_1(x)=x^2,$$g_2(x)=1-x^2,$$x\in [0,1],$are partitions in$(A,\cdot )$with the state values$\frac{1}{2},\frac{1}{2}$and$\frac{1}{3},\frac{2}{3}$of the corresponding elements, respectively. The join of partitions X and Y is the system$X\vee Y=(f_1\cdot g_1,f_1\cdot g_2,f_2\cdot g_1,f_2\cdot g_2)$with the state values$\frac{1}{4},\frac{1}{4},\frac{1}{12},\frac{5}{12}$of the corresponding elements. Using Formula (7), it can be computed that$H_{1/2}^s(X\vee Y)\dot{=}1.903$bit,$H_2^s(X\vee Y)\dot{=}1.710$bit. We have$H_{1/2}^s(X)=H_2^s(X)=1$bit,$H_{1/2}^s(Y)\dot{=}0.958$bit, and$H_2^s(Y)\dot{=}0.848$bit. It can be seen that the inequality (9) applies.

Theorem 4.

If partitions$X,Y$in a product MV-algebra$(A,\cdot )$are statistically independent with respect to$s,$then$H_q^s(X\vee Y)=H_q^s(X)+H_q^s(Y).$

Proof. 

Suppose that $X=(x_1,x_2,\dots ,x_n),$ $Y=(y_1,y_2,\dots ,y_m).$ Let us calculate:

$$\begin{array}{c}{H}_q^s(X\vee Y)=\frac{1}{1-q}\log {\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^{m}s{(x_i\cdot y_j)}^q}}=\frac{1}{1-q}\log \left({\displaystyle \sum _{i=1}^{n}s{(x_i)}^q\cdot }{\displaystyle \sum _{j=1}^{m}s{(y_j)}^q}\right)\\ =\frac{1}{1-q}\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q+}\frac{1}{1-q}\log {\displaystyle \sum _{j=1}^{m}s{(y_j)}^q}=H_q^s(X)+H_q^s(Y).\end{array}$$

☐

4. The Conditional Rényi Entropy in a Product MV-Algebra

In this section, we introduce the concept of conditional Rényi entropy $H_q^s(X/Y)$ of partitions in a product MV-algebra $(A,\cdot ),$ analogously to [6]. It is shown that the proposed definition is consistent, in the case of the limit of q going to 1, with the conditional Shannon entropy defined by Equation (3). Subsequently, by using the proposed notion of conditional Rényi entropy, we define the Rényi information about a partition X in a partition Y.

Let $X=(x_1,x_2,\dots ,x_n),$ and $Y=(y_1,y_2,\dots ,y_m)$ be two partitions in a product MV-algebra $(A,\cdot ),$ and $y_j\in Y$ be fixed. If we consider the function $s_{X/y_j}:X\to \Re ,$ defined, for every $x_i\in X,$ by $s_{X/y_j}(x_i)=s(x_i/y_j),$ then we have

$$\parallel s_{X/y_j}{\parallel }_q={\left({\displaystyle \sum _{i=1}^{n}s{(x_i/y_j)}^q}\right)}^{\frac{1}{q}}.$$

Definition 11.

Let$X=(x_1,x_2,\dots ,x_n),$and$Y=(y_1,y_2,\dots ,y_m)$be partitions in$(A,\cdot ).$We define the conditional Rényi entropy of order$q,$where$q\in (0,1)\cup (1,\infty ),$of X given Y by the formula

$$H_q^s(X/Y)=\frac{q}{1-q}\log \left({\displaystyle \sum _{j=1}^{m}s(y_j)\parallel s_{X/y_j}{\parallel }_q}\right).$$

(10)

Remark 3.

In the same way as in the unconditional case, it can be verified that the conditional Rényi entropy$H_q^s(X/Y)$is always nonnegative. Let$X=(x_1,x_2,\dots ,x_n)$be any partition in$(A,\cdot ),$and$E=(u).$Since$s_{X/u}(x_i)=s(x_i/u)=s(x_i)=s_X(x_i),$for$i=1,2,\dots ,n,$it holds$\parallel s_{X/u}{\parallel }_q=\parallel s_X{\parallel }_q,$and consequently

$$H_q^s(X/E)=\frac{q}{1-q}\log \left(s(u)\parallel s_{X/u}{\parallel }_q\right)=\frac{q}{1-q}\log \left(\parallel s_X{\parallel }_q\right)=H_q^s(X).$$

Proposition 2.

Let$X=(x_1,x_2,\dots ,x_n)$be a partition in a product MV-algebra$(A,\cdot )$and let$s:A\to [0,1]$be a state. Then:

(i) 

${\displaystyle {\sum }_{i=1}^{n}s(x_i}\cdot y)=s(y),$for any element$y\in A;$

(ii) 

${\displaystyle {\sum }_{i=1}^{n}s(x_i}/y)=1,$for any element$y\in A$such that$s(y)>0$.

Proof. 

The proof of the claim (i) can be found in [33]. If $y\in A$ such that $s(y)>0,$ then using the previous equality, we get

$${\displaystyle \sum _{i=1}^{n}s(x_i/y)}=\frac{1}{s(y)}{\displaystyle \sum _{i=1}^{n}s(x_i\cdot y)}=\frac{s(y)}{s(y)}=1.$$

☐

Theorem 5.

Let$X=(x_1,x_2,\dots ,x_n),$and$Y=(y_1,y_2,\dots ,y_m)$be partitions in$(A,\cdot ).$Then

$$\underset{q\to 1}{\lim }{H}_q^s(X/Y)=-{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^{m}s(x_i\cdot y_j)}}\cdot \log \frac{s(x_i\cdot y_j)}{s(y_j)}.$$

Proof. 

For every $q\in (0,1)\cup (1,\infty ),$ we have

$$H_q^s(X/Y)=\frac{q}{1-q}\log \left({\displaystyle \sum _{j=1}^{m}s(y_j)\parallel s_{X/y_j}{\parallel }_q}\right)=-\frac{1}{1-\frac{1}{q}}\log \left({\displaystyle \sum _{j=1}^{m}s(y_j)\parallel s_{X/y_j}{\parallel }_q}\right)=-\frac{f(q)}{g(q)},$$

where $f$ and $g$ are continuous functions defined, for every $q\in (0,\infty ),$ by the equalities

$$f(q)=\log \left({\displaystyle \sum _{j=1}^{m}s(y_j)\parallel s_{X/y_j}{\parallel }_q}\right), g(q)=1-\frac{1}{q}.$$

The functions $f$ and $g$ are differentiable and evidently, $\underset{q\to 1}{\lim }g(q)=0$. Also, it can easily be verified that $\underset{q\to 1}{\lim }f(q)=0$. Indeed, if we put $\delta =\left\{j;s(y_j)>0\right\},$ then using Proposition 2, we get

$$\begin{array}{c}\underset{q\to 1}{\lim }f(q)=\log \left({\displaystyle \sum _{j=1}^{m}s(y_j)}{\displaystyle \sum _{i=1}^{n}s(x_i/y_j)}\right)=\log \left({\displaystyle \sum _{j\in \delta }s(y_j)}{\displaystyle \sum _{i=1}^{n}s(x_i/y_j)}\right)\\ =\log \left({\displaystyle \sum _{j\in \delta }s(y_j)}{\displaystyle \sum _{i=1}^n\frac{s(x_i\cdot y_j)}{s(y_j)}}\right)=\log \left({\displaystyle \sum _{j\in \delta }{\displaystyle \sum _{i=1}^{n}s(x_i\cdot y_j)}}\right)\\ =\log \left({\displaystyle \sum _{j\in \delta }s(y_j)}\right)=\log \left({\displaystyle \sum _{j=1}^{m}s(y_j)}\right)=\log 1=0.\end{array}$$

Using L’Hôpital’s rule, it follows that $\underset{q\to 1}{\lim }{H}_q^s(X/Y)=-\underset{q\to 1}{\lim }\frac{f^{\prime }(q)}{g^{\prime }(q)},$ under the assumption that the right-hand side exists. Let us calculate the derivatives of the functions f and g. We have $\frac{d}{dq}g(q)=\frac{1}{q^2},$ and $\frac{d}{dq}f(q)=\frac{h^{\prime }(q)}{h(q)\ln 2},$ where $h(q)$ is the continuous function defined, for every $q\in (0,\infty ),$ by the formula $h(q)={\displaystyle \sum _{j=1}^{m}s(y_j)\parallel s_{X/y_j}{\parallel }_q}$ with the continuous derivative $h^{\prime }$ for which it holds

$$h^{\prime }(q)={\displaystyle \sum _{j=1}^{m}s(y_j)\parallel s_{X/y_j}{\parallel }_q}\cdot \left[-\frac{1}{q^2}\ln {\displaystyle \sum _{i=1}^{n}s{(x_i/y_j)}^q}+\frac{1}{q}\frac{1}{{\displaystyle {\sum }_{i=1}^{n}s{(x_i/y_j)}^q}}{\displaystyle \sum _{i=1}^{n}s{(x_i/y_j)}^q}\ln s(x_i/y_j)\right].$$

Analogously as in the proof of Theorem 1, we used the identity $b^{\alpha }=e^{\alpha \ln b}$ to calculate the derivative of function h.

We get

$$\underset{q\to 1}{\lim }{f}^{\prime }(q)=\frac{1}{\ln 2}{\displaystyle \sum _{j=1}^{m}s(y_j)}{\displaystyle \sum _{i=1}^{n}s(x_i/y_j)}\ln s(x_i/y_j)={\displaystyle \sum _{j=1}^m{\displaystyle \sum _{i=1}^{n}s(x_i\cdot y_j)}\log \frac{s(x_i\cdot y_j)}{s(y_j)}}.$$

It follows

$$\underset{q\to 1}{\lim }{H}_q^s(X/Y)=\underset{q\to 1}{-\lim }{f}^{\prime }(q)=-{\displaystyle \sum _{i=1}^n{\displaystyle \sum _{j=1}^{m}s(x_i\cdot y_j)}}\cdot \log \frac{s(x_i\cdot y_j)}{s(y_j)},$$

which is the conditional Shannon entropy of X given Y defined by Equation (3). ☐

Theorem 6 (monotonicity).

Let X and Y be partitions in a product MV-algebra$(A,\cdot ).$Then

$$H_q^s(X/Y)\le H_q^s(X).$$

Proof. 

Let $X=(x_1,x_2,\dots ,x_n),$ $Y=(y_1,y_2,\dots ,y_m).$ Then by Proposition 2, it holds $s(x_i)={\displaystyle {\sum }_{j=1}^{m}s(x_i}\cdot y_j),$ for $i=1,2,\dots ,n.$ Suppose that $q\in (1,\infty ).$ Then, using the triangle inequality of the $q$-norm, we get

$$\begin{array}{l}{\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^q}={\displaystyle {\sum }_{i=1}^n{\left({\displaystyle {\sum }_{j=1}^{m}s(x_i}\cdot y_j)\right)}^q}={\left({\left({\displaystyle {\sum }_{i=1}^n{\left({\displaystyle {\sum }_{j=1}^{m}s(x_i}\cdot y_j)\right)}^q}\right)}^{\frac{1}{q}}\right)}^q\\ ={\left(\parallel {\displaystyle {\sum }_{j=1}^m{s}_{X\vee (y_j)}}{\parallel }_q\right)}^q\le {\left({\displaystyle {\sum }_{j=1}^m\parallel s_{X\vee (y_j)}}{\parallel }_q\right)}^q={\left({\displaystyle {\sum }_{j=1}^{m}s(y_j)\parallel s_{X/y_j}}{\parallel }_q\right)}^q.\end{array}$$

It follows that

$$\log {\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^q}\le \log {\left({\displaystyle {\sum }_{j=1}^{m}s(y_j)\parallel s_{X/y_j}}{\parallel }_q\right)}^q,$$

and consequently

$$\begin{array}{ll}{H}_q^s(X)=& \frac{1}{1-q}\log {\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^q}\ge \frac{1}{1-q}\log {\left({\displaystyle {\sum }_{j=1}^{m}s(y_j)\parallel s_{X/y_j}}{\parallel }_q\right)}^q\\ & =\frac{q}{1-q}\log \left({\displaystyle {\sum }_{j=1}^{m}s(y_j)\parallel s_{X/y_j}}{\parallel }_q\right)=H_q^s(X/Y).\end{array}$$

For the case where $q\in (0,1),$ we put $r=\frac{1}{q}.$ By writing the Rényi entropy in terms of the $\frac{1}{q}$-norm and using the triangle inequality for the $\frac{1}{q}$-norm, we get

$$\begin{array}{c}{H}_q^s(X)=\frac{1}{1-q}\log {\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^q}=\frac{r}{r-1}\log {\displaystyle {\sum }_{i=1}^{n}s{(x_i)}^{\frac{1}{r}}}=\frac{r}{r-1}\log {\displaystyle {\sum }_{i=1}^n{\left({\displaystyle {\sum }_{j=1}^{m}s(x_i}\cdot y_j)\right)}^{\frac{1}{r}}}\\ =\frac{r}{r-1}\log {\displaystyle {\sum }_{i=1}^n\parallel s_{(x_i)\vee Y}^{\frac{1}{r}}{\parallel }_r}\ge \frac{r}{r-1}\log \parallel {\displaystyle {\sum }_{i=1}^n{s}_{(x_i)\vee Y}^{\frac{1}{r}}}{\parallel }_r\\ =\frac{r}{r-1}\log {\left({\displaystyle {\sum }_{j=1}^m{\left({\displaystyle {\sum }_{i=1}^{n}s(x_i}\cdot y_j{)}^{\frac{1}{r}}\right)}^r}\right)}^{\frac{1}{r}}=\frac{r}{r-1}\log {\left({\displaystyle {\sum }_{j=1}^{m}s(y_j){\left({\displaystyle {\sum }_{i=1}^{n}s(x_i}/y_j{)}^{\frac{1}{r}}\right)}^r}\right)}^{\frac{1}{r}}\\ =\frac{1}{r-1}\log \left({\displaystyle {\sum }_{j=1}^{m}s(y_j){\left({\displaystyle {\sum }_{i=1}^{n}s(x_i}/y_j{)}^{\frac{1}{r}}\right)}^r}\right)=\frac{q}{1-q}\log \left({\displaystyle \sum _{j=1}^{m}s(y_j)\parallel s_{X/y_j}{\parallel }_q}\right)=H_q^s(X/Y).\end{array}$$

☐

Theorem 7.

If partitions$X,Y$in a product MV-algebra$(A,\cdot )$are statistically independent with respect to$s,$then$H_q^s(X/Y)=H_q^s(X).$

Proof. 

Suppose that $X=(x_1,x_2,\dots ,x_n),$ $Y=(y_1,y_2,\dots ,y_m),$ and put $\delta =\left\{j;s(y_j)>0\right\}.$ Since it holds ${\displaystyle {\sum }_{j\in \delta }s(y_j)=}{\displaystyle {\sum }_{j=1}^{m}s(y_j)}=1,$ we get

$$\begin{array}{c}{H}_q^s(X/Y)=\frac{q}{1-q}\log \left({\displaystyle \sum _{j=1}^{m}s(y_j){\left({\displaystyle \sum _{i=1}^{n}s{(x_i/y_j)}^q}\right)}^{\frac{1}{q}}}\right)=\frac{q}{1-q}\log \left({\displaystyle \sum _{j\in \delta }s(y_j){\left({\displaystyle \sum _{i=1}^n\frac{s{(x_i)}^{q}s{(y_j)}^q}{s{(y_j)}^q}}\right)}^{\frac{1}{q}}}\right)\\ =\frac{q}{1-q}\log \left({\displaystyle \sum _{j\in \delta }s(y_j){\left({\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}\right)}^{\frac{1}{q}}}\right)=\frac{1}{1-q}\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}=H_q^s(X).\end{array}$$

☐

Theorem 8.

Let X, Y be partitions in a product MV-algebra$(A,\cdot ),$and$q_1,q_2\in (0,1)\cup (1,\infty ).$Then$q_1\ge q_2$implies$H_{q_1}^s(X/Y)\le H_{q_2}^s(X/Y).$

Proof. 

Suppose that $X=(x_1,x_2,\dots ,x_n),$ $Y=(y_1,y_2,\dots ,y_m),$ and $q_1,q_2\in (1,\infty ).$ Then the claim is equivalent to the inequality

$${\left({\displaystyle \sum _{j=1}^{m}s(y_j)\parallel s_{X/y_j}{\parallel }_{q_1}}\right)}^{\frac{q_1}{q_1-1}}\ge {\left({\displaystyle \sum _{j=1}^{m}s(y_j)\parallel s_{X/y_j}{\parallel }_{q_2}}\right)}^{\frac{q_2}{q_2-1}}.$$

(11)

We prove this inequality by applying twice the Jensen inequality. First, we apply the Jensen inequality to the function ${\phi }_1$ defined, for every $x\in \left[0,\infty \right),$ by ${\phi }_1(x)=x^{\frac{q_1(q_2-1)}{q_2(q_1-1)}}.$ The assumption $q_1\ge q_2$ implies that $\frac{q_1(q_2-1)}{q_2(q_1-1)}\le 1,$ hence the function ${\phi }_1$ is concave. Therefore, if we put $c_j=s(y_j),$ and $a_j=\parallel s_{X/y_j}{\parallel }_{q_1},$ $j=1,2,\dots ,m,$ in the inequality (6), we obtain

$$\begin{array}{c}{\left({\displaystyle {\sum }_{j=1}^{m}s(y_j)\parallel s_{X/y_j}{\parallel }_{q_1}}\right)}^{\frac{q_1}{q_1-1}}={\left[{\left({\displaystyle {\sum }_{j=1}^{m}s(y_j)\parallel s_{X/y_j}{\parallel }_{q_1}}\right)}^{\frac{q_1(q_2-1)}{q_2(q_1-1)}}\right]}^{\frac{q_2}{q_2-1}}\\ \ge {\left[{\displaystyle {\sum }_{j=1}^{m}s(y_j){\left(\parallel s_{X/y_j}{\parallel }_{q_1}\right)}^{\frac{q_1(q_2-1)}{q_2(q_1-1)}}}\right]}^{\frac{q_2}{q_2-1}}={\left[{\displaystyle {\sum }_{j=1}^{m}s(y_j){\left({\displaystyle {\sum }_{i=1}^{n}s(x_i/y_j)s{(x_i/y_j)}^{q_1-1}}\right)}^{\frac{q_2-1}{q_2(q_1-1)}}}\right]}^{\frac{q_2}{q_2-1}}.\end{array}$$

Next, we apply the Jensen inequality to the function ${\phi }_2$ defined, for every $x\in \left[0,\infty \right),$ by ${\phi }_2(x)=x^{\frac{q_2-1}{q_1-1}}.$ The assumption $q_1\ge q_2$ implies $\frac{q_2-1}{q_1-1}\le 1,$ hence the function ${\phi }_2$ is concave. Put $c_i=s(x_i/y_j),$ and $a_i=s{(x_i/y_j)}^{q_1-1},$ $i=1,2,\dots ,n,$ in the inequality (6). Note that according to Proposition 2, it holds ${\displaystyle {\sum }_{i=1}^n{c}_i}=1$. By the Jensen inequality we get

$$\begin{array}{c}{\left[{\displaystyle {\sum }_{j=1}^{m}s(y_j){\left({\displaystyle {\sum }_{i=1}^{n}s(x_i/y_j)s{(x_i/y_j)}^{q_1-1}}\right)}^{\frac{q_2-1}{q_2(q_1-1)}}}\right]}^{\frac{q_2}{q_2-1}}\\ \ge {\left[{\displaystyle {\sum }_{j=1}^{m}s(y_j){\left({\displaystyle {\sum }_{i=1}^{n}s{(x_i/y_j)}^{q_2}}\right)}^{\frac{1}{q_2}}}\right]}^{\frac{q_2}{q_2-1}}= {\left({\displaystyle {\sum }_{j=1}^{m}s(y_j)\parallel s_{X/y_j}{\parallel }_{q_2}}\right)}^{\frac{q_2}{q_2-1}}.\end{array}$$

By combining the previous results, we obtain the inequality (11). Analogously, we can prove the inequality for the case where $q_1,q_2\in (0,1).$ Finally, the case where $q_1\in (1,\infty )$ and $q_2\in (0,1)$ follows by transitivity. ☐

In the following theorem, a weak chain rule for the Rényi entropy of partitions in a product MV-algebra $(A,\cdot )$ is given.

Theorem 9.

Let$X=(x_1,x_2,\dots ,x_n),$and$Y=(y_1,y_2,\dots ,y_m)$be partitions in a product MV-algebra$(A,\cdot ).$Then

$$H_q^s(X\vee Y)\le H_q^s(X/Y)+\log \beta ,$$

where$\beta =\max \left\{\frac{1}{s(y_j)};y_j\in supp(s),j=1,2,\dots ,m\right\}.$

Proof. 

Put $\delta =\left\{j;s(y_j)>0\right\}.$ The assertion follows by applying the Jensen inequality to the function $\phi$ defined by $\phi (x)=x^q,$ $x\in \left[0,\infty \right),$ and putting $c_j=s(y_j),$$a_j=\parallel s_{X/y_j}{\parallel }_q,$$j=1,2,\dots ,m.$

Let $q\in (0,1).$ Then the function $\phi$ is concave, and therefore we get

$$\begin{array}{c}{\left({\displaystyle {\sum }_{j=1}^{m}s(y_j)\parallel s_{X/y_j}{\parallel }_q}\right)}^q\ge {{\displaystyle {\sum }_{j=1}^{m}s(y_j)\left(\parallel s_{X/y_j}{\parallel }_q\right)}}^q\\ ={\displaystyle \sum _{j\in \delta }s(y_j)}{\displaystyle \sum _{i=1}^n\frac{s{(x_i\cdot y_j)}^q}{s{(y_j)}^q}}={\displaystyle \sum _{j\in \delta }{\left(\frac{1}{s(y_j)}\right)}^{q-1}}{\displaystyle \sum _{i=1}^{n}s{(x_i\cdot y_j)}^q}\ge {\displaystyle \sum _{j\in \delta }{\beta }^{q-1}}{\displaystyle \sum _{i=1}^{n}s{(x_i\cdot y_j)}^q}.\end{array}$$

It follows

$$\begin{array}{c}{H}_q^s(X/Y)=\frac{q}{1-q}\log \left({\displaystyle \sum _{j=1}^{m}s(y_j)\parallel s_{X/y_j}{\parallel }_q}\right)\ge \frac{1}{1-q}\log \left({\beta }^{q-1}{\displaystyle \sum _{j\in \delta }{\displaystyle \sum _{i=1}^{n}s{(x_i\cdot y_j)}^q}}\right)\\ =-\log \beta +\frac{1}{1-q}\log {\displaystyle \sum _{j=1}^m{\displaystyle \sum _{i=1}^{n}s{(x_i\cdot y_j)}^q}}=-\log \beta +H_q^s(X\vee Y).\end{array}$$

Consider now the case where $q\in (1,\infty ).$ Then the function $\phi$ is convex, and therefore we have

$$\begin{array}{ll}& {\left({\displaystyle {\sum }_{j=1}^{m}s(y_j)\parallel s_{X/y_j}{\parallel }_q}\right)}^q\le {{\displaystyle {\sum }_{j=1}^{m}s(y_j)\left(\parallel s_{X/y_j}{\parallel }_q\right)}}^q\\ =& {\displaystyle \sum _{j\in \delta }{\left(\frac{1}{s(y_j)}\right)}^{q-1}}{\displaystyle \sum _{i=1}^{n}s{(x_i\cdot y_j)}^q}\le {\displaystyle \sum _{j\in \delta }{\beta }^{q-1}}{\displaystyle \sum _{i=1}^{n}s{(x_i\cdot y_j)}^q}.\end{array}$$

Thus

$$q\log \left({\displaystyle {\sum }_{j=1}^{m}s(y_j)\parallel s_{X/y_j}{\parallel }_q}\right)\le (q-1)\log \beta +\log {\displaystyle {\sum }_{j=1}^m{\displaystyle {\sum }_{i=1}^{n}s{(x_i\cdot y_j)}^q.}}$$

Since $1-q<0,$ for $q\in (1,\infty ),$ we get

$$\begin{array}{c}{H}_q^s(X/Y)=\frac{q}{1-q}\log \left({\displaystyle \sum _{j=1}^{m}s(y_j)\parallel s_{X/y_j}{\parallel }_q}\right)\\ \ge \frac{q-1}{1-q}\log \beta +\frac{1}{1-q}\log {\displaystyle \sum _{j=1}^m{\displaystyle \sum _{i=1}^{n}s{(x_i\cdot y_j)}^q}}=-\log \beta +H_q^s(X\vee Y).\end{array}$$

☐

Remark 4.

Let$X=(x_1,x_2,\dots ,x_n),$and$Y=(y_1,y_2,\dots ,y_m)$be partitions in$(A,\cdot ).$Since$X\vee Y=Y\vee X,$it holds also the inequality$H_q^s(X\vee Y)\le H_q^s(Y/X)+\log \alpha ,$where$\alpha =\max \left\{\frac{1}{s(x_i)};x_i\in supp(s),i=1,2,\dots ,n\right\}.$

Corollary 2.

Let$X=(x_1,x_2,\dots ,x_n),$and$Y=(y_1,y_2,\dots ,y_m)$be partitions in$(A,\cdot ).$Then:

(i) 

$H_q^s(X\vee Y)\le H_q^s(X)+\log \beta ;$

(ii) 

$H_q^s(X\vee Y)\le H_q^s(Y)+\log \alpha ,$

where$\beta =\max \left\{\frac{1}{s(y_j)};y_j\in supp(s),j=1,2,\dots ,m\right\},$and$\alpha =\max \left\{\frac{1}{s(x_i)};x_i\in supp(s),i=1,2,\dots ,n\right\}.$

Proof. 

The claim is a direct consequence of Theorems 6 and 9. ☐

Definition 12.

Let$X,Y$be partitions in$(A,\cdot ).$We define the Rényi information of order$q,$where$q\in (0,1)\cup (1,\infty ),$about X in Y by the formula

$$I_q^s(X,Y)=H_q^s(X)-H_q^s(X/Y).$$

(12)

Theorem 10.

Let$X,Y$be partitions in$(A,\cdot ).$Then$\underset{q\to 1}{\lim }{I}_q^s(X,Y)=I_s(X,Y),$where$I_s(X,Y)$is the mutual information of partitions$X,Y$defined by Equation (4).

Proof. 

The claim is obtained as a direct consequence of Theorems 1 and 5. ☐

Theorem 11.

For arbitrary partitions$X,Y$in$(A,\cdot ),$it holds$I_q^s(X,Y)\ge 0$. Moreover, if$X,Y$are statistically independent with respect to$s,$then$I_q^s(X,Y)=0$.

Proof. 

The claim is a direct consequence of Theorems 6 and 7. ☐

5. The Rényi Divergence in a Product MV-Algebra

In this section, we introduce the concept of the Rényi divergence in a product MV-algebra $(A,\cdot ).$ We will prove basic properties of this quantity, and for illustration, we provide some numerical examples.

Definition 13.

Let$s,t$be states on a given product MV-algebra$(A,\cdot ),$and$X=(x_1,x_2,\dots ,x_n)$be a partition in$(A,\cdot )$such that$t(x_i)>0,$for$i=1,2,\dots ,n.$Then we define the Rényi divergence$D_q^X(s\parallel t)$of order$q,$where$q\in (0,1)\cup (1,\infty ),$with respect to X as the number

$$D_q^X(s\parallel t)=\frac{1}{q-1}\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}t{(x_i)}^{1-q}.$$

(13)

Remark 5.

The logarithm in the Formula (13) is taken to the base 2 and information is measured in bits. It is easy to see that$D_q^X(s\parallel s)=0$. Namely,

$$D_q^X(s\parallel s)=\frac{1}{q-1}\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}s{(x_i)}^{1-q}=\frac{1}{q-1}\log {\displaystyle \sum _{i=1}^{n}s(x_i)=}\frac{1}{q-1}\log 1=0.$$

The following theorem states that for $q\to 1$ the Rényi divergence $D_q^X(s\parallel t)$ converges to the Kullback–Leibler divergence $D_X(s\parallel t)$ defined by the Formula (5).

Theorem 12.

Let$s,t$be states on a given product MV-algebra$(A,\cdot ),$and$X=(x_1,x_2,\dots ,x_n)$be a partition in$(A,\cdot )$such that$t(x_i)>0,$for$i=1,2,\dots ,n.$Then

$$\underset{q\to 1}{\lim }{D}_q^X(s\parallel t)={\displaystyle \sum _{i=1}^{n}s(x_i)\cdot \log \frac{s(x_i)}{t(x_i)}}.$$

Proof. 

For every $q\in (0,1)\cup (1,\infty ),$ we have

$$D_q^X(s\parallel t)=\frac{1}{q-1}\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}t{(x_i)}^{1-q}=\frac{f(q)}{g(q)},$$

where $f,g$ are continuous functions defined, for every $q\in (0,\infty ),$ in the following way:

$$f(q)=\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}t{(x_i)}^{1-q}, g(q)=q-1.$$

By continuity of the functions $f,g,$ we have $\underset{q\to 1}{\lim }f(q)=f(1)=\log {\displaystyle \sum _{i=1}^{n}s(x_i)}t{(x_i)}^0=\log 1=0,$ and $\underset{q\to 1}{\lim }g(q)=g(1)=0$. Using L’Hôpital’s rule, we get that $\underset{q\to 1}{\lim }{D}_q^X(s\parallel t)=\underset{q\to 1}{\lim }\frac{f^{\prime }(q)}{g^{\prime }(q)},$ under the assumption that the right hand side exists. Since $g^{\prime }(q)=1,$ and $f^{\prime }(q)=\frac{h^{\prime }(q)}{h(q)\ln 2},$ where

$$h(q)={\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}t{(x_i)}^{1-q}, h^{\prime }(q)={\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}t{(x_i)}^{1-q}\ln \frac{s(x_i)}{t(x_i)},$$

we obtain

$$\underset{q\to 1}{\lim }{D}_q^X(s\parallel t)=\underset{q\to 1}{\lim }{f}^{\prime }(q)=\frac{1}{\ln 2}{\displaystyle \sum _{i=1}^{n}s(x_i)}\ln \frac{s(x_i)}{t(x_i)}={\displaystyle \sum _{i=1}^{n}s(x_i)}\log \frac{s(x_i)}{t(x_i)}.$$

☐

The following theorem states that the Rényi entropy $H_q^s(X)$ can be expressed in terms of the Rényi divergence $D_q^X(s\parallel t)$ of a state s from a state t that is uniform over X.

Theorem 13.

Let$s$be a state on a product MV-algebra$(A,\cdot ),$and$X=(x_1,x_2,\dots ,x_n)$be a partition in$(A,\cdot ).$If a state$t:A\to [0,1]$is uniform over X, i.e.,$t(x_i)=\frac{1}{n},$for$i=1,2,\dots ,n,$then

$$H_q^s(X)=H_q^t(X)-D_q^X(s\parallel t)=\log n-D_q^X(s\parallel t).$$

Proof. 

Let us calculate:

$$\begin{array}{c}{D}_q^X(s\parallel t)=\frac{1}{q-1}\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}t{(x_i)}^{1-q}=\frac{1}{q-1}\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}{\left(\frac{1}{n}\right)}^{1-q}\\ =\frac{1}{q-1}\log {\left(\frac{1}{n}\right)}^{1-q}+\frac{1}{q-1}\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}=\log n-H_q^s(X).\end{array}$$

On the other hand, as shown by Example 3, it holds $H_q^t(X)=\log n.$ ☐

Example 7.

Consider any product MV-algebra$(A,\cdot ),$and a state$s:A\to [0,1].$In Example 5, we dealt with the partition$X=(x,u-x)$with$s(x)=\frac{1}{3},$and we calculated the Rényi entropy$H_{1/2}^s(X)\dot{=}0.958$bit. Let$t:A\to [0,1]$be a state uniform over X, i.e.,$t(x)=t(u-x)=\frac{1}{2}.$Then the Rényidivergence$D_q^X(s\parallel t)$of order$q=\frac{1}{2}$is$D_{1/2}^X(s\parallel t)=-2\log \left(\sqrt{\frac{1}{3}\cdot \frac{1}{2}}+\sqrt{\frac{2}{3}\cdot \frac{1}{2}}\right)\dot{=}0.04186$bit, and$H_{1/2}^t(X)-D_{1/2}^X(s\parallel t)\dot{=}1-0.04186\dot{=}0.958$bit. So, the equality$H_{1/2}^s(X)=H_{1/2}^t(X)-D_{1/2}^X(s\parallel t)$holds.

Theorem 14.

Let$s,t$be states on a given product MV-algebra$(A,\cdot ),$and$X=(x_1,x_2,\dots ,x_n)$be a partition in$(A,\cdot )$such that$s(x_i)>0,$and$t(x_i)>0,$for$i=1,2,\dots ,n.$Then$D_q^X(s\parallel t)\ge 0$with the equality if and only if$s(x_i)=t(x_i),$for$i=1,2,\dots ,n.$

Proof. 

The inequality follows by applying the Jensen inequality to the function $\phi$ defined by $\phi (x)=x^{1-q},$ $x\in \left[0,\infty \right),$ and putting $c_i=s(x_i),$ $a_i=\frac{t(x_i)}{s(x_i)},$ for $i=1,2,\dots ,n.$

Let us consider the case of $q\in (1,\infty ).$ Then $1-q<0,$ therefore the function $\phi$ is convex. By the Jensen inequality we obtain

$$1={\left({\displaystyle \sum _{i=1}^{n}t(x_i)}\right)}^{1-q}={\left({\displaystyle \sum _{i=1}^{n}s(x_i)\frac{t(x_i)}{s(x_i)}}\right)}^{1-q}\le {{\displaystyle \sum _{i=1}^{n}s(x_i)\left(\frac{t(x_i)}{s(x_i)}\right)}}^{1-q}={\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}t{(x_i)}^{1-q},$$

(14)

and consequently

$$\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}t{(x_i)}^{1-q}\ge \log 1=0.$$

Since $\frac{1}{q-1}>0,$ for $q\in (1,\infty ),$ it follows that

$$D_q^X(s\parallel t)=\frac{1}{q-1}\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}t{(x_i)}^{1-q}\ge 0.$$

Let $q\in (0,1).$ Then the function $\phi$ is concave, and therefore we get ${\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}t{(x_i)}^{1-q}\le 1,$ and consequently

$$\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}t{(x_i)}^{1-q}\le \log 1=0.$$

Since $\frac{1}{q-1}<0,$ for $q\in (0,1),$ it follows that

$$D_q^X(s\parallel t)=\frac{1}{q-1}\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}t{(x_i)}^{1-q}\ge 0.$$

The equality in (14) holds if and only if $\frac{t(x_i)}{s(x_i)}$ is constant, for $i=1,2,\dots ,n,$ i.e., if and only if $t(x_i)=k\cdot s(x_i),$ for $i=1,2,\dots ,n.$ By summing over all $i=1,2,\dots ,n,$ we get ${\displaystyle {\sum }_{i=1}^{n}t(x_i)}=k\cdot {\displaystyle {\sum }_{i=1}^{n}s(x_i)},$ which implies that $k=1$. Hence $s(x_i)=t(x_i),$ for $i=1,2,\dots ,n.$ Therefore, we conclude that $D_q^X(s\parallel t)=0$ if and only if $s(x_i)=t(x_i),$ for $i=1,2,\dots ,n.$ ☐

Corollary 3.

Let$s$be a state on a product MV-algebra$(A,\cdot ),$and$X=(x_1,x_2,\dots ,x_n)$be a partition in$(A,\cdot )$such that$s(x_i)>0,$for$i=1,2,\dots ,n.$Then$H_q^s(X)\le \log n$with the equality if and only if the state s is uniform over X.

Proof. 

Let $t:A\to [0,1]$ be a state uniform over X, i.e., $t(x_i)=\frac{1}{n},$ for $i=1,2,\dots ,n.$ Then according to Theorems 14 and 13, it holds

$$0\le D_q^X(s\parallel t)=\log n-H_q^s(X),$$

which implies that $H_q^s(X)\le \log n.$ Since the equality $D_q^X(s\parallel t)=0$ applies if and only if $s(x_i)=t(x_i),$ for $i=1,2,\dots ,n,$ the equality $H_q^s(X)=\log n$ holds if and only if the state s is uniform over X. ☐

Example 8.

Let us consider any product MV-algebra$(A,\cdot ),$and states$s_1,s_2,s_3$defined on it. Let$x\in A$with$s_i(x)=p_i,$where$p_i\in (0,1),$for$i=1,2,3$. Then$s_i(u-x)=1-p_i,$for$i=1,2,3$. Further, we consider the partition$X=(x,u-x)$in$(A,\cdot ).$Putting$p_1=\frac{1}{2},$$p_2=\frac{1}{3},$$p_3=\frac{1}{4},$and$q=2,$we obtain

$$D_2^X(s_1\parallel s_2)=\log \left[{\left(\frac{1}{2}\right)}^2\cdot {\left(\frac{1}{3}\right)}^{-1}+{\left(\frac{1}{2}\right)}^2\cdot {\left(\frac{2}{3}\right)}^{-1}\right]\dot{=}0.1699 bit;$$

$$D_2^X(s_1\parallel s_3)=\log \left[{\left(\frac{1}{2}\right)}^2\cdot {\left(\frac{1}{4}\right)}^{-1}+{\left(\frac{1}{2}\right)}^2\cdot {\left(\frac{3}{4}\right)}^{-1}\right]\dot{=}0.4150 bit;$$

$$D_2^X(s_2\parallel s_3)=\log \left[{\left(\frac{1}{3}\right)}^2\cdot {\left(\frac{1}{4}\right)}^{-1}+{\left(\frac{2}{3}\right)}^2\cdot {\left(\frac{3}{4}\right)}^{-1}\right]\dot{=}0.0525 bit.$$

For$q=\frac{1}{2},$we get$D_q^X(s_1\parallel s_2)\dot{=}0.04186$bit;$D_q^X(s_1\parallel s_3)\dot{=}0.1$bit;$D_q^X(s_2\parallel s_3)\dot{=}0.0122$bit. Evidently, in both cases mentioned above,

$$D_q^X(s_1\parallel s_3)>D_q^X(s_1\parallel s_2)+D_q^X(s_2\parallel s_3).$$

This means that the triangle inequality for the Rényi divergence generally does not apply. The result means that it is not a metric in a true sense.

Theorem 15.

Let$s,t$be states on a given product MV-algebra$(A,\cdot ),$and$X=(x_1,x_2,\dots ,x_n)$be a partition in$(A,\cdot )$such that$s(x_i)>0,$and$t(x_i)>0,$for$i=1,2,\dots ,n.$Then:

(i) 

$0<q<1$implies$D_q^X(s\parallel t)\le D_X(s\parallel t);$

(ii) 

$q>1$implies$D_q^X(s\parallel t)\ge D_X(s\parallel t),$

where

$$D_X(s\parallel t)={\displaystyle \sum _{i=1}^{n}s(x_i)\cdot \log \frac{s(x_i)}{t(x_i)}}.$$

Proof. 

We prove the claims by applying the Jensen inequality to the concave function $\phi$ defined, for every $x\in (0,\infty ),$ by $\phi (x)=\log x.$ If we put $c_i=s(x_i),$ and $a_i={\left(\frac{s(x_i)}{t(x_i)}\right)}^{q-1},$ for $i=1,2,\dots ,n,$ in the inequality (6), we get

$$\begin{array}{c}\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}t{(x_i)}^{1-q}=\log {\displaystyle \sum _{i=1}^{n}s(x_i)}{\left(\frac{s(x_i)}{t(x_i)}\right)}^{q-1}\\ \ge {\displaystyle \sum _{i=1}^{n}s(x_i)}\log {\left(\frac{s(x_i)}{t(x_i)}\right)}^{q-1}=(q-1){\displaystyle \sum _{i=1}^{n}s(x_i)}\log \frac{s(x_i)}{t(x_i)}.\end{array}$$

(i)

Suppose that $0<q<1$. Then $\frac{1}{q-1}<0,$ and therefore we obtain

$$D_q^X(s\parallel t)=\frac{1}{q-1}\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}t{(x_i)}^{1-q}\le {\displaystyle \sum _{i=1}^{n}s(x_i)}\log \frac{s(x_i)}{t(x_i)}=D_X(s\parallel t).$$

(ii)

Suppose that $q>1$. Then $\frac{1}{q-1}>0,$ and we get

$$D_q^X(s\parallel t)=\frac{1}{q-1}\log {\displaystyle \sum _{i=1}^{n}s{(x_i)}^q}t{(x_i)}^{1-q}\ge {\displaystyle \sum _{i=1}^{n}s(x_i)}\log \frac{s(x_i)}{t(x_i)}=D_X(s\parallel t).$$

☐

To illustrate the result of previous theorem, let us consider the following example which is a continuation to Example 6.

Example 9.

Consider the product MV-algebra$(A,\cdot )$from Example 6 and the real functions$F_1,F_2$defined by$F_1(x)=x,F_2(x)=x^2,$for every$x\in [0,1].$On the product MV-algebra$(A,\cdot )$we define two states$s_1,s_2$by the formulas$s_i(f)={\displaystyle {\int }_0^{1}f(x)d{F}_i(x)},$$i=1,2,$for any element$f$of$A.$In addition, we consider the partition$X=\left(I_{[0,\frac{1}{3})},I_{[\frac{1}{3},1]}\right)$in$(A,\cdot ).$It can be easily calculated that it has the$s_1$-state values$\frac{1}{3},\frac{2}{3}$of the corresponding elements, and the$s_2$-state values$\frac{1}{9},\frac{8}{9}$of the corresponding elements. By using Formula (5), it can be calculated the Kullback–Leibler divergences$D_X(s_1\parallel s_2)\dot{=}0.25163$bit, and$D_X(s_2\parallel s_1)\dot{=}0.19281$bit. Further, by simple calculations we obtain:$D_2^X(s_1\parallel s_2)\dot{=}0.58496$bit,$D_2^X(s_2\parallel s_1)\dot{=}0.28951$bit,$D_{1/3}^X(s_1\parallel s_2)\dot{=}0.0707$bit, and$D_{1/3}^X(s_2\parallel s_1)\dot{=}0.07736$bit. As can be seen, for$q=\frac{1}{3},$we have$D_q^X(s_1\parallel s_2)<$$D_X(s_1\parallel s_2),$$D_q^X(s_2\parallel s_1)<D_X(s_2\parallel s_1),$and for$q=2,$we have$D_q^X(s_1\parallel s_2)>$$D_X(s_1\parallel s_2),$$D_q^X(s_2\parallel s_1)>$$D_X(s_2\parallel s_1).$The obtained results correspond to the claim of Theorem 15. Based on previous results, we can also see that the Rényi divergence$D_q^X(s\parallel t),$as well as the Kullback–Leibler divergence$D_X(s\parallel t),$is not symmetrical.

6. Conclusions

The aim of this paper was to generalize the results concerning the Shannon entropy and Kullback-Leibler divergence in a product MV-algebra given in [22] and [24] to the case of Rényi entropy and Rényi divergence. The results are contained in Section 3, Section 4 and Section 5. In Section 3, we have introduced the concept of Rényi entropy $H_q^s(X)$ of a partition X in a product MV-algebra $(A,\cdot ),$ and we examined properties of this entropy measure. In Section 4, we have defined the conditional Rényi entropy of partitions in the studied algebraic structure. It was shown that the proposed concepts are consistent, in the case of the limit of $q\to 1,$ with the Shannon entropy of partitions defined and studied in [22]. Moreover, it was shown that the Rényi entropy $H_q^s(X)$ as well as the conditional Rényi entropy $H_q^s(X/Y)$ are monotonically decreasing functions of parameter $q.$ In the final part of Section 4, we have defined the Rényi information about a partition X in a partition Y as an example for the further usage of the proposed concept of the conditional Rényi entropy. Section 5 was devoted to the study of Rényi divergence in $(A,\cdot ).$ We have proved that the Kullback–Leibler divergence of states defined on a product MV-algebra can be derived from their Rényi divergence as the limiting case for $q$ going to 1. Theorem 14 allows interpreting the Rényi divergence as a distance measure between two states (over the same partition) defined on a given product MV-algebra. In addition, we investigated the relationship between the Rényi entropy and the Rényi divergence (Theorem 13), as well as the relationship between the Rényi divergence and Kullback–Leibler divergence (Theorem 15), in a product MV-algebra.

In the proofs we used L’Hôpital’s rule, the triangle inequality of $q$-norm, and the Jensen inequality. To illustrate the results, we have provided several numerical examples.

As has been shown in Example 1, the model studied in this article generalizes the classical case; that is, the Rényi entropy and the Rényi divergence defined in this paper are a generalization of the classical concepts of Rényi entropy and Rényi divergence. On the other hand, MV-algebras enable to study more general situations. We note that MV-algebras can be for example interpreted by means of Ulam some games (see e.g., [35,36,37]). The obtained results could therefore be useful for the researches on this subject.

In Example 2, we have mentioned that the full tribe of fuzzy sets represents a special case of product MV-algebras; therefore, the results of the article can be immediately applied to this important class of fuzzy sets. We recall that by a fuzzy subset of a non-empty set $\mathsf{\Omega }$ (cf. [38]), we understand any map $f:\mathsf{\Omega }\to [0,1].$ The value $f(\omega )$ is interpreted as the degree of belongingness of $\omega \in \mathsf{\Omega }$ to the considered fuzzy set $f.$ In [39], Atanassov has generalized the fuzzy sets by introducing the idea of an intuitionistic fuzzy set, a set having with each member a degree of belongingness as well as a degree of non-belongingness. From the application point of view, it is interesting that to a given class $\mathcal{F}$ of intuitionistic fuzzy sets can be constructed an MV-algebra $\mathcal{A}$ such that $\mathcal{F}$ can be embedded to $\mathcal{A}$. Also, an operation of product on $\mathcal{F}$ can be introduced by such a way that the corresponding MV-algebra is a product MV-algebra. Therefore, all the results of this article can also be applied to the intuitionistic fuzzy case.