Logical Entropy and Logical Mutual Information of Experiments in the Intuitionistic Fuzzy Case

In this contribution, we introduce the concepts of logical entropy and logical mutual information of experiments in the intuitionistic fuzzy case, and study the basic properties of the suggested measures. Subsequently, by means of the suggested notion of logical entropy of an IF-partition, we define the logical entropy of an IF-dynamical system. It is shown that the logical entropy of IF-dynamical systems is invariant under isomorphism. Finally, an analogy of the Kolmogorov–Sinai theorem on generators for IF-dynamical systems is proved.


Introduction
The notions of entropy and mutual information are basic notions in information theory [1] and, as is known, the customary approach is based on Shannon's entropy [2].Let P = (p 1 , . . ., p n ) ∈ n be a probability distribution; Shannon's entropy of P is the number H s (P) = ∑ n i=1 s(p i ), where s : [0, 1] → [0, ∞) is the Shannon function defined by s(x) = −x log x, for every x ∈ [0, 1].Remark that it used the convention (based on continuity arguments) that 0 • log 0 = 0.The idea of Shannon's entropy was generalized in a natural way to the Kolmogorov-Sinai entropy h(T) of dynamical systems [3][4][5], which allows dynamical systems to be distinguished.Kolmogorov and Sinai applied the entropy h(T) to prove that non-isomorphic Bernoulli shifts exist.Of course, the theory of Kolmogorov-Sinai entropy has many other important applications.For this reason, various proposals were made to generalize the Kolmogorov-Sinai entropy concept.In [6], we generalized the Kolmogorov-Sinai entropy concept to the case of a fuzzy probability space [7].This structure represents an alternative mathematical model of probability theory for the situations when the considered events are fuzzy events, i.e., events described unclearly, vaguely.Further proposals for fuzzy generalizations of Shannon's and Kolmogorov-Sinai entropy are presented e.g., in [8][9][10][11][12][13][14][15][16][17].It is known that there exist many ways to define operations for modeling the union and intersection of fuzzy sets; an overview was listed in [18].We remark that while the model studied in [6] was based on Zadeh's fuzzy set operations [19], in our study [14], the Lukasiewicz fuzzy set operations were used.
In 1983, Atanassov introduced a more general fuzzy theory-intuitionistic fuzzy sets theory [44][45][46].Recall that while a fuzzy set is a mapping µ A : Ω → [0, 1] (where the considered fuzzy set is identified with its membership function µ A ), the intuitionistic fuzzy set (shortly, IF-set) is a pair A = (µ A , ν A ) : Ω → [0, 1] × [0, 1] of fuzzy sets for which the condition µ A (ω) + ν A (ω) ≤ 1, for every ω ∈ Ω, is satisfied.The function µ A is called the membership function of A, the function ν A is called the non-membership function of A. Evidently, each fuzzy set µ A can be regarded as an IF-set A = (µ A , 1 − µ A ).Each result that applies to IF-sets also applies to the case of fuzzy sets.Of course, the opposite implication is not valid, e.g., the representation theorem of IF-states does not follow by the corresponding result for fuzzy states.The theory of IF-sets represents a non-trivial generalization of the fuzzy set theory; thus, the IF-sets provide opportunities to model a larger class of real situations.We remark that a probability theory on intuitionistic fuzzy events has been elaborated in [47], see also [48].Some results about the Kolmogorov type entropy for the case of intuitionistic fuzzy sets are given e.g., in [49][50][51][52][53].
When solving some specific problems, instead of Shannon's entropy it is more appropriate to use an approach based on the concept of logical entropy [54][55][56][57].If P = (p 1 , . . ., p n ) ∈ n is a probability distribution, then the logical entropy of P is defined by the formula In [57], historical aspects of the logical entropy formula H(P) are discussed and the relationship between logical entropy and Shannon's entropy is examined.The concepts of logical conditional entropy and the logical mutual information have been introduced as well.We note that some results about the logical entropy on some of the above mentioned algebraic structures, based on fuzzy set theory, can be found e.g., in [58][59][60][61][62].
The purpose of the present work is to study the logical entropy and logical mutual information of experiments in the intuitionistic fuzzy case.The paper is organized in the following way.In the following section, basic definitions and notations are provided.In Section 3, the concept of logical entropy for the case of intuitionistic fuzzy experiments is introduced, and basic properties of the proposed measure are shown.In Section 4, we introduce the concepts of logical mutual information and conditional mutual information of intuitionistic fuzzy experiments and derive some properties of these measures.In Section 5, using the suggested concept of logical entropy, we define the logical entropy of IF-dynamical systems.It is shown that the logical entropy of IF-dynamical systems is invariant under isomorphism.Finally, an analogy of the Kolmogorov-Sinai theorem on generators for IF-dynamical systems is proved.Section 6 contains a brief summary.

Basic Definitions, Notations and Facts
In this section, we provide basic definitions, notations and facts that will be used throughout the contribution.

Definition 1. By an IF-event we will understand a pair
In the following, we will use the symbol F to denote the family of all IF-events.Analogously as in the fuzzy case, there are many possibilities to define operations for modeling the union and intersection of IF-sets (see e.g., [63][64][65]).We will use the operations ⊕, and • defined as follows.
In the family F we define the partial binary operation ⊕ in the following way Here, 1 Ω denotes the function defined by 1 Ω (ω) = 1, for every ω ∈ Ω .Similarly, we denote by 0 Ω the function defined by 0 Ω (ω) = 0, for every ω ∈ Ω .Evidently, if A, B ∈ F , then A ⊕ B exists if and only if for any A ∈ F .Further, in the family F , we define the binary operation • in the following way: if A = (µ A , ν A ), and B = (µ B , ν B ), then Evidently, A • 1 = A, for any A ∈ F .The IF-event 0 = (0 Ω , 1 Ω ) is interpreted as an impossible event; the IF-event 1 = (1 Ω , 0 Ω ) as a certain event.It can easily be verified that, for any A, B, C ∈ F , the following conditions are satisfied:

Since in the fuzzy case the inequality µ
in the family F it is natural to define the relation ≤ as follows: if A = (µ A , ν A ), and B = (µ B , ν B ) are two IF-events, then A ≤ B if and only if µ A ≤ µ B , and ν A ≥ ν B .The relation ≤ is a partial order such that 0 ≤ A ≤ 1 for all A ∈ F .Gutierrez Garcia and Rodabaugh have proved that intuitionistic fuzzy sets ordering and topology are reduced to the ordering and topology of fuzzy sets [66].Another situation is in measure theory, where the intuitionistic fuzzy case cannot be reduced to the fuzzy case.Definition 2. A map m : F → [0, 1] is said to be a state if the following conditions are satisfied: Example 1.Consider a probability space (Ω, S, P), and put It is easy to verify that the mapping m : F → [0, 1] defined, for any element A = (µ A , ν A ) of F , by the formula: is a state.Namely, for every A, B ∈ F such that A ⊕ B exists, we have: Remark 1. Riečan and Ciungu have shown in [67] that any continuous state m defined on a family F of all S-measurable IF-events has the form (1).In more detail, if a state m defined on a family F of all S-measurable IF-events is continuous (i.e., A n A implies m(A n ) m(A)), then there exist exactly one probability measure P : S → [0, 1], and exactly one α ∈ [0, 1] such that: Definition 3. By an IF-partition of F , we will understand a finite collection ξ = {A 1 , . . . ,A n } of elements of F such that ⊕ n i=1 A i exists, and m Remark 2. A classical probability space (Ω, S, P) can be regarded as a family of IF-events, if we put , is a state on F .A usual measurable partition {E 1 , . . . ,E n } of a space (Ω, S, P) (i.e., any sequence {E 1 , . . . , and the equality P( In the case that η is a refinement of ξ, we write ξ ≺ η.

Denote by M the family of all mappings
is a state, and m/F = m, i.e., m(A) = m(A), for any A ∈ F .
Proof.The proof can be found in [68].

Logical Entropy of IF-Partitions
It is obvious that each IF-partition ξ = {A 1 , . . . ,A n } represents, from the point of view of classical probability theory, a random experiment with a finite number of results A i , i = 1, 2, . . ., n, that are intuitionistic fuzzy events, with a probability distribution p i = m(A i ), i = 1, 2, . . ., n. Namely, p i ≥ 0 for i = 1, 2, . . ., n, and For that reason, we define the logical entropy of ξ = {A 1 , . . . ,A n } as the number: ( Since ∑ n i=1 m(A i ) = 1, we can also write: Remark 3. Evidently, the IF-partition η = {1} has zero logical entropy.
Example 2. Consider the measurable space (Ω, S), where Ω is the unit interval [0,1] and S is the σ−algebra of all Borel subsets of [0,1].Now, we can consider the family of all S-measurable IF-events and the state m : F → [0, 1] defined, for any element A = (µ A , ν A ) of F , by the formula: It has the m-state values 0.5, 0.5 of the corresponding elements and the logical entropy H(ξ) = 0.5 .Some basic properties of the logical entropy of IF-partitions are listed below.Theorem 3. Let ξ, η be two IF-partitions of F .Then: Proof.The property (i) is evident.We will prove the second property.Let ξ = {A 1 , . . . ,A I }, η = B 1 , . . . ,B J , ξ ≺ η.Then, for any A i ∈ ξ there exists a subset α i ⊂ {1, . . . ,J}, such that m(A i ) = ∑ j∈α i m(B j ), α i ∩ α j = ∅, for i = j, and ∪ I i=1 α i = {1, . . . ,J}.Hence, we can write: As a consequence of the inequality (a + b) 2 ≥ a 2 + b 2 , which is true for all non-negative real numbers a, b, we get: 2 , i = 1, . . ., I. Therefore: The inequality (iii) is a simple consequence of the previous property and Theorem 2. Definition 6.If ξ = {A 1 , . . . ,A I }, η = B 1 , . . ., B J are two IF-partitions of F , then the conditional logical entropy of ξ assuming a realization of the IF-experiment η is defined by the formula: Remark 4. Since m(A i • B j ) ≤ m(B j ), for the conditional logical entropy it holds that H(ξ/η) ≥ 0. If we put η = {1}, then H(ξ/η) = H(ξ).
For n = 3, using the previous equality and Theorem 6, we get: Now let us suppose that the result is true for a given n ∈ N. Then Thus, by the principle of mathematical induction, the result follows.
(ii) The proof of the second assertion is analogous; it suffices to use Theorem 6 and the principle of mathematical induction.

Logical Mutual Information of IF-Partitions
In this section, using the results of the previous parts, we define the notions of logical mutual information and logical conditional mutual information of IF-partitions and prove basic properties of these measures.We also present some numerical examples to illustrate the results.Definition 7. Let ξ, η be two IF-partitions of F .Then, we define the logical mutual information of ξ and η by the formula: Remark 7. As a simple consequence of Equation ( 6), we have: From Equation ( 9), it follows that I(ξ, η) = I(η, ξ), and I(ξ, ξ) = H(ξ).Proof.The non-negativity of logical mutual information I(ξ, η) follows from the subadditivity of logical entropy (the property (ii) of Theorem 5) and Equation (9).The second inequality is a consequence of Equation ( 9) and the property (iii) of Theorem 3.

Remark 8.
According to the previous theorem, we may say that ξ and ς are conditionally independent, assuming a realization of η, and write ξ ↔ η ↔ ς instead of ξ → η → ς.
We note that, in the classical theory, the last claim of Theorem 13 is known as the data processing inequality.

Logical Entropy of IF-Dynamical Systems
The classical dynamical system is a quadruplet (Ω, S, P, T), where (Ω, S, P) is a probability space, and T : Ω → Ω is a measure preserving map, i.e., A ∈ S implies T −1 (A) ∈ S, and P(T −1 (A)) = P(A).Define τ : S → S by the equality τ(A) = T −1 (A), for any A ∈ S.Then, τ is a mapping with the property P(τ(A)) = P(A), for any A ∈ S. In addition, It is a motivation for the following definition.Definition 10.Let F be the family of all IF-events and m : F → 0, 1 be a state.Then, the triplet (F , m, τ) will be called an IF-dynamical system, if τ : F → F is such a mapping that the following conditions are satisfied: Moreover, we have: Define τ 2 = τ • τ, and put . ., where τ 0 is the identical mapping on F .Theorem 14.Let any IF-dynamical system (F , m, τ) be given.If ξ, η are IF-partitions of F , then the following properties are satisfied: and ∪ I i=1 α i = {1, . . . ,J}.We get: However, this indicates that τξ ≺ τη.
(iii) Since m τ k (A i ) = m(A i ), for i = 1, 2, . . ., I, k = 0, 1, 2, . . ., we get: (iv) The proof is analogous to the proof of the previous property.(v) We will prove by mathematical induction.For the case of n = 2, the equality holds by Equation (7).We assume that the statement holds for a given n ∈ N and we prove it is true for n + 1.By part (iii), we have: Therefore, by Equation ( 7) and the induction assumption, we can write: The proof is complete.
Lemma 1.Let {a n } ∞ n=1 be a sequence of non-negative real numbers such that a r+s ≤ a r + a s , for every r, s ∈ N. Then lim n→∞ 1 n a n exists.
Proof.Let ψ : F 1 → F 2 be a mapping representing an isomorphism of IF-dynamical systems (F 1 , m 1 , τ 1 ), (F 2 , m 2 , τ 2 ).By Lemma 2, if ξ = {A 1 , . . . ,A n } is an IF-partition of F 1 , then the system and consequently: The opposite inequality is obtained in a similar way; it suffices to consider the inverse Indeed, since ⊕ n i=1 B i exists, according to property (ii) from Lemma 3, ⊕ n i=1 ψ −1 (B i ) exists, too.Moreover, we have: and By means of (iii) from the previous lemma, we get: Thus, according to the previous lemma, we can write: and consequently: The proof is completed.
In the final part, we prove an analogy of the Kolmogorov-Sinai theorem on generators for the studied situation.This theorem (see e.g., [69]) is the main tool for calculating the entropy of dynamical system.First, analogously as in [62], we introduce the following definition.Definition 13.Let (F , m, τ) be an IF-dynamical system and ς be an IF-partition of F .Then ς is called an m-generator of (F , m, τ) if to any IF-partition ξ of F , there exists an integer k > 0 such that ξ ≺ ∨ k i=0 τ i ς.
Proof.Let ξ be any IF-partition of F .Then, for each natural number k, we can write: Theorem 17.Let (F , m, τ) be an IF-dynamical system and ς be an m-generator of (F , m, τ).Then h(τ) = h(τ, ς).

Discussion
The purpose of the present study was to introduce the concepts of logical entropy and logical mutual information of experiments in the intuitionistic fuzzy case.Our results have been presented in Sections 3-5.
In Section 3, we defined the notions of logical entropy and logical conditional entropy for intuitionistic fuzzy experiments, and proved the basic properties of the proposed measures.It was proved that the logical entropy of intuitionistic fuzzy experiments has properties analogous to the properties of Shannon entropy of measurable partitions, in the sense of classical probability theory.In Section 4, the results of the previous part were used to develop a logical information theory for the intuitionistic fuzzy case.The concepts of logical mutual information and logical conditional mutual information of intuitionistic fuzzy experiments have been introduced, and properties of these measures were studied.Specifically, the chain rule for logical mutual information has been established, and the data processing inequality for conditionally independent IF-partitions was proved.We have also provided some numerical examples to illustrate the results.
In Section 5, the concept of logical entropy of IF-partitions was used to define the logical entropy of IF-dynamical systems.It was shown that the logical entropy of IF-dynamical systems is invariant under any isomorphism.Finally, we have provided an analogy of the Kolmogorov-Sinai theorem on generators for the intuitionistic fuzzy case.
All of the mentioned results can be immediately applied to the fuzzy case.On the other hand, it is hopeful to use the methods developed here in some more general algebraic structures.For example, we mentioned in Theorem 1 the possibility of embedding F to the family M with a state m extending the state m.Actually M is an example of an MV-algebra with a product [20][21][22][23][24][25][26][27][28].Further research ought to more fully investigate potential general applications for the methods developed in this work.