Logical Entropy of Dynamical Systems—A General Model

: In the paper by Rieˇcan and Markechová (Fuzzy Sets Syst. 96, 1998), some fuzzy modiﬁcations of Shannon’s and Kolmogorov-Sinai’s entropy were studied and the general scheme involving the presented models was introduced. Our aim in this contribution is to provide analogies of these results for the case of the logical entropy. We deﬁne the logical entropy and logical mutual information of ﬁnite partitions on the appropriate algebraic structure and prove basic properties of these measures. It is shown that, as a special case, we obtain the logical entropy of fuzzy partitions studied by Markechová and Rieˇcan (Entropy 18, 2016). Finally, using the suggested concept of entropy of partitions we deﬁne the logical entropy of a dynamical system and prove that it is the same for two dynamical systems that are isomorphic.


Introduction
The study of the concept of entropy is very important in contemporary sciences.Entropy has been applied in information theory, physics, computer sciences, statistics, chemistry, biology, sociology, general systems theory and many other fields.The classical approach in information theory is based on Shannon's entropy [1,2].In connection with the issue of the isomorphism of dynamical systems, using Shannon's entropy, Kolmogorov and Sinai have defined the entropy of dynamical systems [3][4][5].Thus, they developed a method for distinguishing non-isomorphic dynamical systems by means of which they proved the existence of non-isomorphic Bernoulli shifts.Some fuzzy generalizations of Shannon's and Kolmogorov-Sinai's entropy were introduced by Riečan [6][7][8][9] (see also [10]) and Markechová [11][12][13][14].It is known that there are many possibilities for defining operations with fuzzy sets; an overview can be found in [15].While the model of Markechová was based on the Zadeh connectives, in the model studied by Riečan, the Lukasiewicz connectives were used to define the fuzzy set operations.In [16] Riečan and Markechová introduced a general algebraic theory involving both models as special cases.
In [17][18][19] the authors deal with studying the concept of logical entropy.If P = (p 1 , . . ., p n ) ∈ n is a probability distribution, then the logical entropy of P is defined as the number h(P) = ∑ n i=1 p i (1 − p i ).More details about the concept of logical entropy can be found, e.g., in [20], where the author deals, inter alia, with historical aspects of the logical entropy formula h(P) and investigates the relationship between the logical entropy and Shannon's entropy.In the cited paper, the notions of logical conditional entropy, logical mutual information, logical cross entropy and logical divergence and their properties were also studied.In [21] Ebrahimzadeh defined and studied the notions of logical entropy of partitions and logical entropy of dynamical systems in quantum logics and in [22] the author introduced the notion of conditional logical entropy of dynamical systems on quantum logics.In the recently published paper [23], Markechová and Riečan deal with the study of logical entropy of fuzzy partitions and fuzzy dynamical systems.
In this paper, we provide analogies of the results of Riečan and Markechová given in [16] for the case of the logical entropy.We define the logical entropy and logical mutual information of finite partitions on an appropriate algebraic structure and prove basic properties of these measures.As a special case we obtain the logical entropy of fuzzy partitions introduced in [23].It is noted that some investigations concerning entropy of dynamical systems and related notions in the above setup were conducted in [24][25][26][27][28][29][30][31][32][33][34][35][36][37].
The paper is organized as follows.In the next section, we give the basic definitions and some already known results used in this paper.In Section 3, the logical entropy, conditional logical entropy and logical mutual information of finite partitions on an appropriate algebraic structure are defined and basic properties of these measures are proved.In Section 4, we define the logical entropy of a dynamical system using the suggested concept of logical entropy of finite partitions.Finally, it is shown that the logical entropy of dynamical systems is invariant under isomorphisms.Our results are summarized in the final section.

Basic Definitions and Facts
As in [16] we shall consider the algebraic structure (F, ⊕, ⊗, 1 F ), where F is a non-empty partially ordered set, ⊕ is a partial binary operation on F, ⊗ is a binary operation on F, 1 F is a fixed element of F, and two mappings m : F → [0, 1] and s : F → F, where the following conditions are satisfied: By a dynamical system we will understand the above described triplet (F, m, s).Some examples are presented in the following.
Example 1.Consider a triplet (Ω, F, m), where Ω is a non-empty set, F is a fuzzy σ-algebra of fuzzy subsets of Ω, i.e., F ⊂ [0, 1] Ω such that (i) The above described triplet (Ω, F, m) is called in the terminology of Piasecki a fuzzy probability space [38].The symbols ∪ ∞ n=1 f n = sup n f n and ∩ ∞ n=1 f n = inf n f n denote the fuzzy union and the fuzzy intersection of a sequence { f n } ∞ n=1 ⊂ F, respectively, in the sense of Zadeh [39].The partial ordering relation is defined in the following way: for every f , g ∈ F, f ≤ g if and only if f (ω) ≤ g(ω) for all ω ∈ Ω.The presented σ-additive fuzzy P-measure m has all properties analogous to properties of a classical probability measure, e.g., (i) m is a non-decreasing function, i.e., if f , g ∈ F such that f ≤ g, then m( f ) ≤ m(g); The proofs can be found in [38].The monotonicity of fuzzy P-measure m implies that this measure transforms F into the interval [0, 1].
In the set F we define the operations ⊕, ⊗ as follows: f ⊕ g is defined whenever f ≤ g ⊥ , and in this case f ⊕ g = f ∪ g; f ⊗ g = f ∩ g, for every f , g ∈ F. The set F with the operations ⊕, ⊗ and the mapping m satisfy all assumptions (F1)-(F7).Let us prove, e.g., (F5).Suppose that ⊕ n i=1 f i exists.Since Hence, ⊕ n i=1 f i is defined if and only if f i ≤ 1 Ω − f j (i = j), i.e., if { f 1 , . . . ,f n } is a system of pairwise W-separated fuzzy subsets.Since any fuzzy P-measure is additive, the equality m(⊕ n Evidently, the mapping τ : F → F fulfils the conditions (F8) and (F9).Therefore the system (F, m, τ) is a dynamical system within the meaning of our definition.
Remark 1.In the paper by Markechová [11] the Shannon entropy of fuzzy partitions in a fuzzy probability space has been defined.Recall that the notion of fuzzy partition in the given fuzzy probability space (Ω, F, m) has been introduced by Piasecki in [41] as follows: any finite sequence { f 1 , . . . ,f n } of pairwise W-separated fuzzy subsets from F is called a fuzzy partition if m(∪ n i=1 f i ) = 1.The concept of Shannon's entropy of fuzzy partitions was used to define the Kolmogorov-Sinai entropy of fuzzy dynamical systems [12].By a fuzzy dynamical system we mean a system (Ω, F, m, τ), where (Ω, F, m) is any fuzzy probability space and τ : F → F is an m− preserving σ-homomorphism.Note that a classical dynamical system (Ω, S, P, T) can be considered a fuzzy dynamical system (Ω, F, m, τ); it is sufficient to put F = {I A ; A ∈ S}, where I A is the indicator of a set A ∈ S, and define the mapping m : F → [0, 1] by m(I A ) = P(A), and the mapping τ : F → F by the formula: The above triplet (Ω, F, m) is a fuzzy probability space and τ : F → F is an m− preserving σ-homomorphism.
The recently published paper [23] deals with studying logical entropy of fuzzy partitions in a fuzzy probability space and logical entropy of fuzzy dynamical systems.
As mentioned above, operations with fuzzy sets can be introduced in various ways.While the previous model is based on Zadeh's connectives, in the following model the Lukasiewicz operations are used.Recall that the union of fuzzy subsets f , g of Ω is defined by Lukasiewicz as min( f + g, 1).
Example 2. Let (Ω, S, P) be a classical probability space, F be the family of all S-measurable functions f : Ω → [0, 1], m( f ) = Ω f dP.Further, a mapping U : F → F is given satisfying the following two conditions: In the set F we define the operation ⊕ as follows: f ⊕ g is defined whenever f + g ≤ 1, and in this case f The operation ⊗ is defined as the product of functions: f ⊗ g = f • g.It is not difficult to verify that the set F with the operations ⊕, ⊗ and the mappings m : F → [0, 1] and U : F → F satisfy all assumptions (F1)-(F9).Of course, here 1 F = 1 Ω .Let us prove, e.g., that the condition (F7) holds.
The opposite inequality is obvious.It is easy to see that a classical dynamical system (Ω, S, P, T) can be embedded to the presented fuzzy model; it is sufficient to put F = {I A ; A ∈ S}, and define the mapping U : F → F by the formula (1).Remark 2. Fuzzy analogies of Shannon's entropy and the Kolmogorov-Sinai entropy with respect to this model were introduced by Riečan in [7] (see also [10]), and independently by Dumitrescu [26][27][28].Dumitrescu considered a more general family F of fuzzy subsets of Ω and a mapping m : F → [0, 1] characterized axiomatically.However, by the representation theorem of Butnariu and Klement [42] there exists a probability measure P such that m( f ) = Ω f dP, hence, the Dumitrescu theory can be reduced to the case of Riečan.

Logical Entropy and Logical Mutual Information of Partitions in F
We shall now introduce a general algebraic theory for the case of logical entropy.We show that the proposed general theory can be applied to the two models which were described above.
. ., g p are two partitions in F, then we define: We say that B is a refinement of A, and write A ≺ B, if there exists a partition I(1), I(2), . . ., . Following (F5) we have: following (F7), (F3) and (F5), for i = 1, 2, . . ., n, we obtain: Recall that in the classical case, where partitions A, B consist of indicators only, A ∨ B is the least common refinement of A and B, i.e., every set of A or B is a union of some sets of A ∨ B. In the Markechová model, fuzzy partitions are considered wherein the fuzzy partition in the given fuzzy probability space (Ω, F, m) is defined as any finite sequence { f 1 , . . . ,f n } of pairwise W-separated fuzzy subsets from F such that m(∪ n i=1 f i ) = 1.Recall that in Example 1 ⊕ n i=1 f i is defined if and only if { f 1 , . . . ,f n } is a system of pairwise W-separated fuzzy subsets and in this case ⊕ n i=1 f i = ∪ n i=1 f i .Therefore, the notion of a fuzzy partition in a fuzzy probability space coincides with the general notion of a partition from Definition 1.If A = { f 1 , . . . ,f n }, B = g 1 , . . ., g p are two fuzzy partitions in a fuzzy probability space, then A ∨ B is defined as the system A ∨ B = f i ∩ g j ; i = 1, . . ., n, j = 1, . . ., p , which again coincides with the general notion from Definition 2 as well as the notion of a refinement.A fuzzy partition B is a refinement of A if for every fuzzy set g ∈ B there exists f ∈ A such that g ≤ f .Therefore, the general theory can be applied to the Markechová model.On the other hand, Riečan has defined a fuzzy partition as a finite system { f 1 , . . . ,f n } of fuzzy subsets of Ω such that ∑ n i=1 f i (ω) = 1, for every ω ∈ Ω, and A ∨ B as the system ), the notion of fuzzy partition in the Riečan model again coincides with the general notion from Definition 1. Evidently, the general theory can be applied to the Riečan model as well.Definition 3. Let A = { f 1 , . . . ,f n } be a partition in F. The logical entropy of A is defined as the number: Evidently, h l (A) ≥ 0, and from Equation ( 2) it immediately follows that: . ., g p be two partitions in F. The conditional logical entropy of A given B is defined by: Remark 3. Since by (F4) and (F6) it holds m(g j ) ≥ m( f i ⊗ g j ), i = 1, . . ., n, j = 1, . . ., p, we see that h l (A/B) ≥ 0.
In the following, some basic properties of the logical entropy are presented.At first, we will prove the assertions of the following lemma which will be useful in further considerations.
Theorem 2. Let A, B be two partitions in F. Then: Proof.
. ., g p .For each i = 1, 2, . . ., n, we have: Therefore: (ii) By Theorem 1 and the part (i), we get: In the following two theorems, chain rules for logical entropy of partitions in F are presented.
(i) According to Theorem 1 we have: Therefore, using Theorem 3 we get: Now let us suppose that the result is true for a given n ∈ N. Then: (ii) For n = 2, using Theorem 3 we obtain: Suppose that the result is true for a given n ∈ N. Then: Definition 5 [36].Let A, B be two partitions in F. We write A ⊂ • B if, for each f i ∈ A and for each g j ∈ B, m( f i ⊗ g j ) = m(g j ) or m f i ⊗ g j = 0. We write A .
= B if and only if A ⊂ • B and B ⊂ • A.
Theorem 5. Let A, B be partitions in F. Then: . ., g p .
(iii) It immediately follows from the property (ii).
(iv) Suppose that A ⊂ • B. Since, by definition, for i = 1, 2, . . ., n, j = 1, 2, . . ., p, m( f i ⊗ g j ) = m(g j ) or m( f i ⊗ g j ) = 0, we get: Conversely, let h l (A/B) = 0, i.e., we assume that: Since, m( f i ⊗ g j ) ≥ 0, and by (F4) and (F6) m(g j ) ≥ m( f i ⊗ g j ), for i = 1, 2, . . ., n, j = 1, 2, . . ., p, we have: This means that A ⊂ • B. (v) Since A ⊂ • B, by the part (iii) h l (A/B) = 0.So (see Theorem 1), we have: Similarly, B ⊂ • A implies: Definition 6.If A, B are two partitions in F, then the logical mutual information of A and B is defined by the formula: Remark 5.As a simple consequence of Equation ( 4) we have: Subsequently we see that I l (A, B) = I l (B, A) and I l (A, A) = h l (A).From Equation ( 6), the property (ii) of Theorem 2, and (ii) of Theorem 5 we get that 0 ≤ I l (A, B) ≤ min(h l (A), h l (B)).

Definition 7.
Let A, B and C be partitions in F. Then the logical conditional mutual information of A and B given C is defined by the formula: Theorem 6 (Chain rules for logical mutual information).Let A 1 , A 2 , . . ., A n and C be partitions in F.Then, for n = 2, 3, . . ., it holds: Proof.By Equation ( 5), Theorem 4, and Equation ( 7), we obtain: . ., g p in F are independent, i.e., m( f i ⊗ g j ) = m( f i ) • m(g j ), for i = 1, 2, . . ., n, j = 1, 2, . . ., p, then: . ., g p be independent partitions in F. By simple calculation we obtain: Corollary 1.If partitions A, B in F are independent, then: Proof.By simple calculation we get:

Logical Entropy of Dynamical Systems
Let any dynamical system (F, m, s) be given.If A = { f 1 , . . . ,f n } is a partition in F, then it is easy to verify that the system sA = {s ( f 1 ), . . ., s ( f n )} is a partition in F, too.Namely, ⊕ n i=1 s ( f i ) exists, since ⊕ n i=1 f i exists, and by (F8) and (F9) we have: . ., where s 0 is an identical mapping on F.
Theorem 8. Let (F, m, s) be a dynamical system, A, B be partitions in F. Then the following properties are satisfied: . ., g p .
, (iv) The proof is analogous to the proof of (ii).

(v)
We shall prove the assertion by mathematical induction.The assertion is valid for n = 2 according to Theorem 1 and the previous part of this theorem.Assume that the assertion holds for a given n ∈ N. Since by the part (ii) we have: by Theorem 1 and the induction assumption we get: The proof is completed.
The aim of this section is to provide the definition of entropy of dynamical system (F, m, s).The possibility of this definition is based on Proposition 3; in order to prove it we will need the assertion of Lemma 2. Lemma 2. Let {a n } ∞ n=1 be a sequence of nonnegative real numbers such that a n+m ≤ a n + a m for every n, m ∈ N. Then lim n→∞ 1 n a n exists.
Proof.The proof can be found in [43].

Proposition 3.
Let (F, m, s) be a dynamical system.Then, for every partition A in F, lim Proof.Denote a n = h l (∨ n−1 i=0 s i A).According to subadditivity of logical entropy (Theorem 2, (ii)) and the part (ii) of Theorem 8, we get: By the previous lemma lim Definition 8. Let (F, m, s) be a dynamical system, A be a partition in F. The logical entropy of s with respect to A is defined by: The logical entropy of a dynamical system (F, m, s) is defined by the formula: h l (s) =sup{h l (s, A); A is a partition in F} Theorem 9. Let (F, m, s) be a dynamical system, A, B be partitions in F. Then the following properties are satisfied:

Proof.
(i) This property is obvious.
(ii) The assumption A ≺ B implies the relation ∨ n−1 i=0 s i A ≺ ∨ n−1 i=0 s i B, n = 1, 2, . . . .Thus, according to the part (i) of Theorem 5, we have the inequality h l (∨ n−1 i=0 s i A) ≤ h l (∨ n−1 i=0 s i B), n = 1, 2, . . . .Therefore, h l (s, A) ≤ h l (s, B). (iii) By simple calculations we obtain: Definition 9. We say that two dynamical systems (F 1 , m 1 , s 1 ), (F 2 , m 2 , s 2 ) are isomorphic if there exists a bijective mapping ψ : F 1 → F 2 satisfying the following conditions: (i) the diagram: (iii) for any f , g ∈ F 1 , f ⊕ g exists if and only if ψ( f ) ⊕ ψ(g) exists, and in this case In the following theorem we show that the logical entropy of dynamical systems is invariant under any isomorphism.In order to prove it the following assertion is needed.Lemma 3. Let (F 1 , m 1 , s 1 ), (F 2 , m 2 , s 2 ) be isomorphic dynamical systems wherein a mapping ψ : F 1 → F 2 represents their isomorphism.Then, for the inverse ψ −1 : F 2 → F 1 , the following properties are satisfied: Then we have: Theorem 10.If dynamical systems (F 1 , m 1 , s 1 ), (F 2 , m 2 , s 2 ) are isomorphic, then: Proof.Let a mapping ψ : Indeed, ⊕ n i=1 ψ( f i ) exists, since ⊕ n i=1 f i exists and by Definition 9 we have: On the other hand, we have: Hence, using the conditions (i), (ii) from Definition 9 we get: and consequently: The opposite inequality is obtained in a similar way.If B = {g 1 , . . . ,g n } is a partition in F 2 , then it is easy to verify that ψ −1 (B) = ψ −1 (g 1 ), . . ., ψ −1 (g n ) is a partition in F 1 .Indeed, since ⊕ n i=1 g i exists according to the property (ii) from Lemma 3 ⊕ n i=1 ψ −1 (g i ) exists, too.Moreover, we have: and: By means of (iii) from the previous lemma we get: Thus, according to the previous lemma: Therefore: {h l (s 2 , B); B is a partition in F 2 } ⊂ {h l (s 1 , A); A is a partition in F 1 } This result implies the inequality: sup{h l (s 2 , B); B is a partition in F 2 } ≤ sup {h l (s 1 , A); A is a partition in F 1 } i.e., it holds h l (s 2 ) ≤ h l (s 1 ).The proof is completed.

Discussion
In the year 2013, classical logical entropy was discussed by Ellerman [20] as an alternative measure of information.In the paper by Markechová and Riečan [23], the notions of logical entropy and logical mutual information of fuzzy partitions in the given fuzzy probability space were introduced and studied.By means of the suggested concept of entropy of fuzzy partitions the authors have defined the logical entropy of fuzzy dynamical systems.Thereby they obtained a new tool for distinction of non-isomorphic fuzzy dynamical systems, since, as it is proved in Theorem 12 of the cited paper, the logical entropy of fuzzy dynamical systems coincides on isomorphic fuzzy dynamical systems.It is noted that the definition of fuzzy dynamical systems (Ω, F, m, τ) considered in [23] is based on Zadeh's connectives.
In this paper, we generalize the results of Markechová and Riečan concerning the logical entropy.We have introduced the general model for the logical entropy which includes-besides other cases-the mentioned case of the logical entropy of fuzzy dynamical systems (Ω, F, m, τ).We have defined the notions of logical entropy and logical mutual information of finite partitions on an appropriate algebraic structure and proved basic properties of these measures.Using the suggested concept of logical entropy of partitions, we have defined the logical entropy of dynamical systems and proved that the logical entropy of dynamical systems is invariant under any isomorphism (Theorem 10).From Example 1 it follows that the presented results are generalizations of the results of [23]; inter alia, Theorem 12 of [23] stating that the logical entropy of fuzzy dynamical systems is invariant under any isomorphism is a special case of Theorem 10.It is shown that the suggested general theory can be applied to the Riečan model of fuzzy dynamical systems (Ω, F, m, U) described in Example 2, and to the classical case as well.Note that in the Riečan approach the Lukasiewicz connectives were used to define the fuzzy set operations.Accordingly, all obtained results are valid also for the case of fuzzy dynamical systems (Ω, F, m, U) defined by Riečan.In particular, from the presented algebraic theory it follows that the logical entropy of fuzzy dynamical systems (Ω, F, m, U) is invariant under any isomorphism.
Probably one of the most important results of the theory of invariant measures for practical purposes is the Kolmogorov-Sinai Theorem on generators [43].
A fuzzy analogy of the Kolmogorov-Sinai Theorem on generators for fuzzy dynamical systems (Ω, F, m, τ) was proved in [12], and an analogy of this theorem for fuzzy dynamical systems (Ω, F, m, U) is provided in [9] (see also [10]).The aim of our further research is to provide an analogy of this theorem for the case of logical entropy.
The definition of relation ≺ in the Riečan model is the same as in Definition 2. Since ⊕ n i=1 f i exists if and only if ∑ n i=1 f i ≤ 1, and 1