Entropy of Fuzzy Partitions and Entropy of Fuzzy Dynamical Systems

In the paper we define three kinds of entropy of a fuzzy dynamical system using different entropies of fuzzy partitions. It is shown that different definitions of the entropy of fuzzy partitions lead to different notions of entropies of fuzzy dynamical systems. The relationships between these entropies are studied and connections with the classical case are mentioned as well. Finally, an analogy of the Kolmogorov–Sinai Theorem on generators is proved for fuzzy dynamical systems.


Introduction
The notion of entropy is basic in information theory [1,2]; it is suitable for measuring the value of information which we get from a realization of the considered experiment.A customary mathematical model of a random experiment in the information theory is a measurable partition of a probability space.Partitions are standardly defined within classical, crisp sets.It turned out however, that for solving real problems partitions defined within the concept of fuzzy sets [3,4] are more suitable.That was a reason why several concepts of generalization of the classical set partition to a fuzzy partition [5][6][7][8][9][10] have been created.A fuzzy partition can serve as a mathematical model of the random experiment whose results are vaguely defined events, the so-called fuzzy events.Kolmogorov and Sinai [11] (see also [12]) used the entropy to prove the existence of non-isomorphic Bernoulli shifts (Example 1).Because the Kolmogorov and Sinai theory of entropy of classical dynamical systems has many important and interesting applications, it is reasonable to also expect similar results in the fuzzy case.
In this paper we present our results concerning the entropy of fuzzy dynamical system based on a given probability space.The results represent fuzzy generalizations of some concepts from the classical probability theory.First, we briefly repeat some basic facts from the theory of fuzzy partitions (Section 2) and the classical Kolmogorov-Sinai theory (Section 3).The presented concepts of entropy of fuzzy partitions (Riečan-Dumitrescu, Maličky, and Hudetz entropy) were used to define three kinds of entropy of a fuzzy dynamical system (Section 4).We study the relationships between these entropies and also connections with the classical case.We obtain the measure which can distinguish non-isomorphic dynamical systems more sensitively than the Kolmogorov-Sinai entropy (Theorem 4).Finally, we prove an analogy of the Kolmogorov-Sinai Theorem on generators for the case of fuzzy dynamical systems.The final section presents conclusions and some suggestions for further research.It is noted that certain basic studies on entropy of fuzzy partitions and related notions were done in [13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30][31].

Fuzzy Partitions
In our considerations the Kolmogorov name appears twice.First the Shannon entropy has been used for the distiguishing non-isomorphic dynamical systems by the Kolmogorov-Sinai entropy.We generalize the distinguishing to the fuzzy case.Secondly the whole modern probability theory and mathematical statistics with applications is based on the set theory, and this method was suggested by Kolmogorov.The main prerequisite of the Kolmogorov approach (cf.[32]) is the identification of the notion of an event with the notion of a set.So consider a non-empty set Ω, some subsets of Ω will be called events.Denote by S the family of all events.In the probability theory it is assumed that S is a σ´algebra.

Definition 1.
A family S of subsets of a non-empty set Ω is called a σ´algebra if the following conditions are satisfied: (i) Ω P S, (ii) if A P S, then Ω ´A P S, (iii) if A n P S pn " 1, 2, ...q, then Y 8 n"1 A n P S.
The couple pΩ, Sq will be called a measurable space.
Definition 2. Let pΩ, Sq be a measurable space.A mapping P : S Ñ r0, 1s is called a probability measure if the following properties are satisfied: PpA n q.
The triplet pΩ, S, Pq is called a probability space.
If we have a set A Ă Ω, and ω P Ω, then we have only two possibilities: ω P A or ω P Ω ´A.The set A can be characterized by the characteristic function χ A : Ω Ñ t0, 1u .On the other hand a fuzzy set is a mapping f : Ω Ñ r0, 1s .Analogously to the σ´algebra of sets, we consider a tribe of fuzzy sets.Definition 3. By a tribe of fuzzy subsets of a set Ω we shall mean a family F of functions f : Ω Ñ r0, 1s satisfying the following conditions: The elements of F are called fuzzy events.If S is a σ´algebra, then F " tχ A ; A P Su is a tribe.Another example of a tribe is the family F of all functions f : Ω Ñ r0, 1s measurable with respect to S. Analogously to the notion of a probability P on a σ´algebra S, P : S Ñ r0, 1s , we introduce the notion of a state m on F, m : F Ñ r0, 1s .Definition 4. Let F be a tribe.By a state on F we mean a mapping m : F Ñ r0, 1s satisfying the following conditions: One of the nicest results in the theory is the Butnariu and Klement representation theorem [33] (see also Theorem 8.1.12 in [34]).Theorem 1.Let F be a tribe and m : F Ñ r0, 1s be a state.Then there exists a probability measure P such that mp f q " ż f dP for every f P F.
Recall that P is defined on the σ´algebra T = tA Ă Ω; χ A P F }. Hence it is reasonable to consider a probability space pΩ, S, Pq and the family F of all S-measurable functions f : Ω Ñ r0, 1s .The following concept was used, for example, in [6,23].Definition 5. Let pΩ, S, Pq be a probability space, F be the family of all S-measurable functions f : Ω Ñ r0, 1s (i.e., rα, βs Ă r0, 1s ñ f ´1 prα, βsq P S ).By a fuzzy partition (more precisely F-partition) we understand any sequence f 1 , ..., f n P F such that: Evidently, if A = t f 1 , ..., f k u , B = tg 1 , ..., g l u are fuzzy partitions of pΩ, S, Pq, then the system A _ B : " f i ¨gj ; i " 1, 2, ..., k, j " 1, 2, ..., l ( is also a fuzzy partition of pΩ, S, Pq.We put A usual measurable partition tA 1 , ..., A n u of Ω (i.e., each finite sequence tA 1 , ..., A n u of measurable subsets of Ω such that Y n i"1 A i " Ω and A i X A j " Ø (i ‰ j)) can be regarded as a fuzzy partition, if we consider f i " χ A i instead of A i .Indeed:

Kolmogorov-Sinai Entropy
An inspiration for fuzzy entropy was the entropy of the classical partition.Definition 6.Let pΩ, S, Pq be a probability space, A = tA 1 , ..., A n u be an S-measurable partition of Ω.Then the Kolmogorov-Sinai entropy of A is the number: H pAq " where φ : r0, 1s Ñ is the Shannon entropy defined via: If A, B are two partitions of pΩ, S, Pq, then A _ B :" tA X B ; A P A, B P Bu.The symbol i" 1 A i has a similar meaning.Of course, the most important application of Kolmogorov-Sinai entropy has occured in dynamical systems.Definition 7. By a dynamical system we mean the quadruple pΩ, S, P, T q, where pΩ, S, Pq is a probability space and T : Ω Ñ Ω is a measure preserving transformation (i.e., T ´1pAq P S, and PpT ´1pAqq " PpAq for any A P S).
Example 1.Let X " tu 1 , ..., u k u, p 1 , ..., p k ě 0, p 1 `p2 `... `pk " 1, Ω " X N " px n q 8 n"1 : x n P X ( , S be the σ´algebra generated by the family of all subsets A Ă Ω of the form A " px n q n : x i 1 " u i 1 , x i 2 " u i 2 , ..., x i t " u i t ( , and P : S Ñ r0, 1s be the probability generated by the equalities P ` px n q n : x i 1 " u i 1 , x i 2 " u i 2 , ..., x i t " u i t (˘" p i 1 ¨pi 2 ¨... ¨pi t , and the mapping T : Ω Ñ Ω by the equality: n"1 q " py n q 8 n"1 , y n " x n`1 , n " 1, 2, .... Then pΩ, S, P, T q is a dynamical system, so-called Bernoulli shift (the independent repetition of the experiment tp 1 , ..., p k u).
Let A " tA 1 , ..., A n u be an S-measurable partition of pΩ, S, Pq.In the following, by T ´1pAq the partition T ´1pA 1 q, T ´1pA 2 q, ..., T ´1pA n q ( is denoted.
The partition A _ T ´1pAq_..._ T ´pn´1q pAq " _ n´1 i" 0 T ´ipAq represents an experiment consisting of n realizations A, T ´1pAq, . . .,T ´pn´1q pAq of experiment A. The entropy hpT, Aq of experiment A with respect to T is defined via: hpT, Aq " lim Definition 8.The Kolmogorov-Sinai entropy of dynamical system pΩ, S, P, T q is defined by the formula hpTq " sup thpT, Aqu , where the supremum is taken over all S-measurable partitions A of Ω.
If two dynamical systems are isomorphic, then they have the same entropy.It solves the existence of non-isomorphic Bernoulli shifts.Probably one of the most important results of the theory of invariant measures for practical purposes is the Kolmogorov-Sinai Theorem stating that h(T) = hpT, Aq, whenever A is a partition generating the given σ´algebra S (i.e., a measurable partition such that σpY 8  i"0 T ´ipAqq " S).In the following section, we give an analogy of this theorem for the case of fuzzy dynamical systems.

The Entropy of Fuzzy Dynamical Systems
Let us return to the fuzzy case.Let a probability space pΩ, S, Pq be given.Each fuzzy partition A " t f 1 , ..., f k u of Ω represents in the sense of the classical probability theory a random experiment with a finite number of outcomes f i , i " 1, 2, ..., k, (which are fuzzy events) with a probability distribution p i " mp f i q " ş f i dP, i " 1, 2, ..., k, since p i ě 0 for i " 1, 2, ..., k, and ř k i"1 p i " This is a motivation for the following definition.
Definition 9. Let pΩ, S, Pq be a probability space and A " t f 1 , ..., f k u be a fuzzy partition of Ω.Put mp f q " ş f dP.Then the entropy of A is given by the formula: HpAq " In the preceding section we have defined a dynamical system pΩ, S, P, T q.Now we shall define the fuzzy dynamical system.Definition 10.Let pΩ, S, Pq be a probability space, F be the family of all S-measurable functions f : Ω Ñ r0, 1s , mp f q " ş f dP.Then the quadruple pΩ, F, m, τq, where τ : F Ñ F is m-invariant (i.e., mpτp f qq " mp f q for all f P F), is called a fuzzy dynamical system.Example 2. Let T : Ω Ñ Ω be a measure P preserving map.Define τ : F Ñ F by the formula: τp f q " f ˝T for all f P F. ( Then: mpτp f qq " mp f ˝Tq " hence τ is invariant. Example 3. Let pΩ, S, P, T q be a classical dynamical system.Put F " tχ A ; A P Su .Then the system pΩ, F, m, τq, where τ : F Ñ F is defined by (1), is a fuzzy dynamical system.By this procedure the classical model can be embedded to a fuzzy one.
Then a n`m ď a n `am , for any n, m P N, and this inequality implies the existence of lim where the supremum is taken over all fuzzy partitions A Ă G .
From the following example it follows that the entropy h G pτq is a fuzzy generalization of the Kolmogorov-Sinai entropy.
Example 4. Let pΩ, S, P, T q be a dynamical system.Put G " tχ A ; A P Su , and define τ : F Ñ F by (1).Then h G pτq " hpTq is the Kolmogorov-Sinai entropy.
The main result in the Riečan-Dumitrescu entropy is the following theorem on generators (cf.[25]).Theorem 2. Let C be an S-measurable partition of Ω generating S, i.e., σpY 8  i"0 τ i pCqq " S.Then, for any fuzzy partition A " tg 1 , ..., g k u , the following inequality holds: hpτ, Aq ď hpτ, Cq `ż Of course, if G contains all constant functions, then h G pτq " 8.This defect can be removed by two other constructions, by means of the Maličky entropy and the Hudetz entropy.
In the Riečan-Dumitrescu definition we considered the entropy: for any fuzzy partition A. Instead of this number, we will use the number HpA, τ(A), . . .,τ n´1 (A)) defined as follows.If A, B are two fuzzy partitions, A = t f 1 , ..., f k u , B = tg 1 , ..., g l u , then we write A ď B if there is a partition tI 1 , ..., I k u of the set t1, 2, ..., lu such that f i " ř j:jPI i g j for any i P t1, 2, ..., ku.

Definition 12.
Let A be a fuzzy partition.Then we define HpA, τpAq, . . ., τ n´1 pAqq " inf It is noted that this approach was suggested by Maličky and Riečan in [35], but only for the case of classical dynamical systems.The above definition includes a more general case.Similarly as in Lemma 1, the following assertion can be proved.
Therefore we are able to define the entropy h G pτq of a fuzzy dynamical system (Ω, F, m, τ).
Definition 13.Let G Ă F. Then the entropy h G pτq of pΩ, F, m, τq is defined by the equality: where the supremum is taken over all fuzzy partitions A Ă G.
Now we can compare the entropy h G pτq with the Riečan-Dumitrescu entropy.
Note that the oposite inequality h G pτqď hpTq is proved for some G in [23].Let pΩ, S, P, T q be a dynamical system.In the following we shall consider a fuzzy dynamical system pΩ, F, m, τq, where the mapping τ : F Ñ F is defined by the formula τp f q " f ˝T.We shall consider the entropy suggested and studied by Hudetz in [36][37][38].Definition 14.Let A = t f 1 , ..., f k u be a fuzzy partition of Ω.Then the Hudetz entropy of A is defined by the equality: ĤpAq " Using the Hudetz entropy of fuzzy partition we will define the entropy of fuzzy dynamical systems.The possibility of this definition is based on the following theorem.
Theorem 5. Let pΩ, F, m, τq be a fuzzy dynamical system, A " t f 1 , ..., f k u be a fuzzy partition of Ω.Then the following limit exists: Ĥp_ n´1 i" 0 τ i pAqq.
Put α " pi, jq; f i ¨τp f j q ą 0 ( .Calculate: . By the principle of mathematical induction we get: where the supremum is taken over all F-partitions A Ă G.
The following theorem is a fuzzy analogy of Kolmogorov-Sinai Theorem on generators.
Proof.Let A " tg 1 , ..., g k u be a fuzzy partition of Ω .It is sufficient to prove the inequality ĥpτ, Aq ď hpτ, Cq.Based on Theorem 2 we have: According to Theorem 5: By the combination of Equations ( 2) and ( 3) we get that ĥpτ, Aq ď hpτ, Cq.

Conclusions
In this paper we study the entropy of fuzzy partitions and the entropy of fuzzy dynamical systems.The presented concepts of entropy of fuzzy partitions were used to define three kinds of entropy of a fuzzy dynamical system.The relationships between these entropies are studied.The presented measures can be considered as measures of information of experiments whose outcomes are vaguely defined events, the so-called fuzzy events.Finally, we prove an analogy of the Kolmogorov-Sinai Theorem on generators for the case of fuzzy dynamical systems.
Similarly to the set theory the fuzzy set theory has also been shown to be useful in many applications of mathematics as well as in the theoretical research.We hope that also the present text can be presented as an illustration of the fact.Of course, there exists a remarkable generalization of fuzzy set theory.It was suggested by K. Atanassov and it is named IF-set theory [39,40].Instead of one fuzzy set f : Ω Ñ r0, 1s , IF-set is a pair A " pµ A , υ A q of fuzzy sets µ A , υ A : Ω Ñ r0, 1s such that µ A `υA ď 1.The function µ A is called the membership function, the function υ A the non-membership function.If we have a fuzzy set f : Ω Ñ r0, 1s then it can be represented as an IF-set A " p f , 1 ´f q .It was reasonable to construct the probability theory on families of IF-sets (see e.g., [34,41]).
There are some results about the entropy on IF-sets (cf.[42]).Namely, any IF-set can be embedded to a suitable MV-algebra (multivalued algebra).MV-algebras (cf.[43][44][45]) play in multi-valued logic similar role as Boolean algebras in two-valued logic.There are at least two ways for further research in the area.The first way: to study the IF-entropy without using MV-algebras, and by this way to achieve some applications.The second way: to study the entropy on MV-algebras and some of its generalizations as D-posets (cf.[46]), effect algebras (cf.[47]), or A-posets (cf.[48][49][50]).

Definition 11 .
Let pΩ, F, m, τq be a fuzzy dynamical system.For any non-empty G Ă F define the Riečan-Dumitrescu entropy h G pτq of pΩ, F, m, τq by the equality: For any non-empty G Ă F define the entropy ĥG pτq of a fuzzy dynamical system pΩ, F, m, τq by the equality: