Subshifts on Infinite Alphabets and Their Entropy

We analyze symbolic dynamics to infinite alphabets by endowing the alphabet with the cofinite topology. The topological entropy is shown to be equal to the supremum of the growth rate of the complexity function with respect to finite subalphabets. For the case of topological Markov chains induced by countably infinite graphs, our approach yields the same entropy as the approach of Gurevich We give formulae for the entropy of countable topological Markov chains in terms of the spectral radius in l2.


Introduction
Symbolic dynamical systems on finite alphabets are classical mathematical objects that provide a wealth of examples and have greatly influenced theoretical developments in dynamical systems. In computer science, certain symbolic systems, namely, the topological Markov chains generated by finite graphs, model the evolution of finite transition systems, and the class of sofic symbolic systems (factors of topological Markov chains) models the evolution of certain automata. The most important numerical invariant of dynamical systems is the topological entropy. For symbolic systems, the entropy equals the exponential growth rate of the number of finite words of fixed length. In the case of a topological Markov chain, the entropy equals the natural logarithm of the spectral radius of the generating graph. Considering the graph as a linear map, the spectral radius measures the rate of dilation under iterated application. On an exponential scale, this rate equals the growth of the number of finite words. This note is an attempt to generalize this meeting ground of topology, graph theory, and spectral theory to infinite alphabets, especially countably infinite alphabets. Beside its theoretical interest, this was motivated by the increasing importance of infinite state systems in computer science.
Any attempt at studying symbolic dynamics on infinite alphabets has to deal with the fact that the discrete topology, employed in the finite case, leads to shift spaces which are not compact. Most approaches attempt to compactify the respective alphabet. In this note, we endow the alphabet with a compact topology which coincides with the discrete topology in the finite case, the cofinite topology. Gurevich [1] has considered the Alexandrov compactification of the alphabet and his formula for the entropy of the respective topological Markov chains coincides with ours. The theory of Gurevich has the unpleasent feature that the closure of the set of graph walks must be taken. Still, our approach is similar to Gurevich's, since minimal open covers in the Alexandrov compactification of an infinite countable discrete space and in the cofinite topology are similar. In Gurevich's setting, the dynamical properties of the boundary of a (sofic) subshift have been studied ( [2] Section 3) (see also [3][4][5]). Another approach is due to Ott et al. [6], who considered an N-shift on words in the Alexandrov compactification which they endowed with a certain quotient topology to get rid of the introduced ∞-symbol. Their constructions have been further developed to yield topological dynamical systems which are analogous to classical Z-shifts, and in this setting, characterizations of morphisms of systems are known [7,8]. Contrary to these authors, this paper is mostly interested in entropy theory, especially its connections to subword complexity and spectral theory.
The entropy theory of this note admits a clear operational interpretation. In the case of an infinite alphabet, the number of finite words may be uncountable; hence, we identify all but finitely many letters prior to counting. The entropy is obtained by suprematizing over such identifications. Under some conditions, the entropy of a countable topological Markov chain may be computed or bounded via the spectral radius of a linear operator, analogous to the finite case. This reduces the computation of the entropy of certain symbolic systems on countable alphabets to a well-understood numerical problem.
Section 2 recalls some of the theory of symbolic dynamics on finite alphabets, Section 3 defines subshifts on infinite alphabets, Section 4 shows that the entropy equals the supremum of the exponential growth rates of the number of words in a finite subalphabet, Section 5 specializes to the case of countably infinite topological Markov chains and provides spectral formulae, Section 6 presents a proof that all nonnegative real numbers may be the entropy of a subshift on an infinite alphabet, and Section 7 collects examples.

Symbolic Dynamics on Finite Alphabets
We recall the construction of symbolic dynamical systems on a finite alphabet. Consider a finite set of letters, the alphabet, {1, ..., k} =: [k], endowed with the discrete topology. We form the product space . Special cases of subshifts are the topological Markov chains; they are induced by finite directed graphs. We treat such graphs as finite square matrices with entries from the set {0, 1}, the respective adjacency matrices. Given a graph G, whose vertex set we may enumerate as [k], its associated subshift We denote by W t (S) the words of length t in the subshift S; hence W t (S) := {a 1 , ..., a t } ∃s ∈ S with {s i , ..., s i+t−1 } = {a 1 , ..., a t } for some i ∈ Z .
The growth rate of a real nonnegative sequence {x t } ∞ t=1 is the expression where ln + (x) := max{0, ln(x)}. The above is the asymptotic exponential growth rate of the sequence. It may be zero, if the sequence grows subexponentially, or infinite, if the sequence grows superexponentially. Given a subshift S, its complexity function assigns t → |W t (S)|. The growth rate GR t |W t (S)| measures the asymptotic exponential growth rate of the number of words in the subshift. In the case of a topological Markov chain, one has [9] denotes the largest eigenvalue of the graph G. The topological entropy [10] is the chief numerical invariant associated with a topological dynamical system. Let X be a compact topological space and let f : X → X be continuous. The topological entropy of the system (X, f ) with respect to the open cover U of X is where A ∨ B := {A ∩ B} A∈A,B∈B , and #C denotes the minimal cardinality of a finite subcover of C. The topological entropy of the system (X, f ) is The entropy of a subsystem is lesser or equal to the entropy of the surrounding system. The entropy of the continuous image of a system is lesser or equal to the original entropy. It is well-known [10] that

Symbolic Dynamics on Infinite Alphabets
Let A be a set equipped with the cofinite topology, the minimal topology with the T 1 -property, which is generated by the subbasis {A \ {x}} x∈A . The respective basis consists of sets of the form A \ {x 1 , ..., x n }. It is easy to verify that this basis is the entire topology. Hence, a subset of A is open if and only if it is the complement of a finite subset. If A is finite, the cofinite topology coincides with the discrete topology. The cofinite topology turns every set into a compact separable T 1 -space. If A is infinite, its cofinite topology is hyperconnected. By Tikhonov's theorem, the product A Z is also compact. The topology of A Z has a subbasis of sets of the form

A basis for A Z is given by finite intersections of subbasic open sets, by sets of the form
In the remainder of this note, we suppose shift spaces to be equipped with the product of the cofinite topology, unless we explicitly state otherwise.

Definition 1 (Shifts and subshifts). The shift is the dynamical system
By a morphism from the subshift S ⊆ A Z to the subshift T ⊆ B Z we mean a σ-equivariant continuous surjection M : S → T. The following diagram commutes.
If S has an infinite alphabet, codings may not lead to morphisms, as the following proposition shows (See also Examples 6 and 7).
Proof. Clearly M(S) is σ-equivariant. By pulling back a subbasic open set through M, we see where the latter is open if the intersection is finite.
is finite, and therefore all sliding block codes give rise to morphisms. In fact, every morphism between subshifts on finite alphabets is a sliding block code, the Curtis-Hedlund-Lyndon theorem [11].

Entropy Theory
In the following, we restrict our attention to countably infinite alphabets. We may suppose, without loss of generality, that A = N. We denote by W t F (S) the set of words of length t in the subshift S whose letters are from the finite subalphabet F. Lemma 1. Let S be a subshift of N Z . The cover of N Z given as is open and fulfills Proof. The cover U (n) consists of open sets since [n] or might be obtained from a word in W t+1 [n] by changing its letters to letters bigger n .
These covers are minimal, since the sets of the form are properly contained in a single element of the cover. Hence The claim follows by invoking Lemma 2.

Lemma 2. Let S be a subshift of N Z . Then
for all n ∈ N and c ∈ N.
Proof. Follows by taking growth rates with respect to t on where F is a finite set. We proceed to show that the reverse inequality also holds. Proof. Note that B(n , t ) B(n, t) whenever n > n and t > t. The set N with its discrete topology is homeomorphic to a relatively discrete subset of the real line via the bijection that assigns n ∈ N to 1/n ∈ R. Its one-point Alexandrov-compactification adds the point 0 ∈ R. Therefore, the Alexandrov compactification of the discrete space N is metrized by In particular, .
By Lemma 2, we conclude GR l W 2t+1+l We are ready to prove the main result. Combining these inequalities and using Lemma 1 we obtain h(S, σ) = sup n∈N GR t W t [n] . Since |W t F | ≤ |W t F | whenever F ⊆ F and since every finite subset of N is included in some [n], we have

Remark 1.
Let S ⊆ A Z be a subshift and let F ⊆ A be a finite subset of the alphabet such that for all s ∈ S where s t ∈ F for some t ∈ Z we have s l ∈ F for all l ∈ Z. Then F Z ∩ S ⊆ S is clearly σ-invariant, and also closed, since F Z is the product of finite and therefore closed sets.
The above observation allows us to build subhifts on infinite alphabets as "disjoint unions" of subshifts on finite alphabets. Example 1. Let {F i } i∈N be a sequence of subshifts on finite alphabets. Label the alphabet of F 1 by {1, ..., f 1 }, the alphabet of F 2 by { f 1 + 1, ..., f 2 }, and so on. We have obtained a subshift of N Z whose entropy equals sup i∈N h(F i , σ).

Shifts along Infinite (Directed) Graphs
In this section, we consider countably infinite graphs. For any such graph we assume, without loss of generality, that its vertex set is N. Proposition 2. Let G be an infinite countable graph. Then the set S G := s ∈ N Z G s t ,s t+1 = 1 for all t ∈ Z is a subshift.
Proof. Clearly S G is σ-invariant. It remains to show that it is closed. Let s ∈ CoFin(N) Z \ S G . It suffices to show that s is an interior point. There are three cases.

Remark 3 (Universal topological Markov chain).
Since there are countably many finite directed graphs, their disjoint union is a countable graph. Hence, there is a countable topological Markov chain which contains all finite topological Markov chains as closed subsystems. However, there exists a more interesting universal chain. It is well-known that there exists a countable connected directed graph U such that, for any countable directed graph G, there exists an embedding G → U, an injection that is adjacency preserving, whose image is an induced subgraph of U. (The case of undirected graphs is discussed in [12]; the case of directed graphs, in the guise of 3-colored graphs, is discussed in [13]). The topological Markov chain (S U , σ) is such that, for any topological Markov chain S G on a finite alphabet, there exists a closed embedding (S G , σ) → (S U , σ). To see this, observe that the universal directed graph U contains a copy of G. Let V be the finite vertex set of G. Then V Z ∩ S U is closed and nowhere dense, since V is finite, and a σ-invariant subset of S U , since G embeds as an induced subgraph. The universal system (S U , σ) contains every chain on an infinite alphabet as an open dense subsystem, since the infinite vertex set of the subgraph is open and dense in the cofinite topology.

Lemma 5. Let G be a graph and let F be a finite induced subgraph of G. Then
for any norm − .
Proof. Since F is finite, we may, without loss of generality, use the norm

Proposition 3.
Let G be a countable graph and consider S G ⊆ N Z . Then refers to the largest eigenvalue of the subgraph induced by F.
Just as a finite graph corresponds to a linear map from a finite-dimensional vector space to itself, a countable graph, an infinite {0, 1}-matrix, corresponds to a linear map from an infinite-dimensional vector space to itself. In the infinite-dimensional setting the choice of topology becomes important. Let l 2 denote the Hilbert space of sequences Then the adjacency operator G : l 2 → l 2 is defined by We suppose that G is uniformly locally finite, i.e., there is a common upper bound for the number of successors of every vertex; therefore, the adjacency operator G : l 2 → l 2 is bounded ( [14] Theorem 3.2). The spectrum of a bounded operator B : l 2 → l 2 is Spec(B) = λ ∈ C B − λI is not invertible , and its spectral radius is the number

Proposition 4.
Let G be a uniformly locally finite directed countable graph and consider S G ⊆ N Z . Then h(S G , σ) ≤ ln + ρ(G) .

Proof.
We equip the space of linear maps from l 2 to itself with the operator norm − 2,2 . We may consider a finite subgraph F of G as F : l 2 → l 2 by filling up with zeros. For any t ∈ N we have 0 ≤ (F t ) ij ≤ (G t ) ij which implies F t 2,2 ≤ G t 2,2 . We conclude, starting with an application of Gelfand's formula, that where we have invoked Proposition 3 in the last step. Proposition 5. Let G be a uniformly locally finite countable graph that is undirected, G ij = G ji , and consider S G ⊆ N Z . Then h(S G , σ) = ln + ρ(G) .

Entropy Numbers
In this section, we show that all numbers in [0, ∞] are entropies of subshifts of N Z , in particular, entropies of topological Markov chains. This result has been obtained by Salama [15], who considered Gurevich's compactification approach, which leads to the same entropy. In fact, Salama obtained the stronger result that, given two numbers 0 ≤ α ≤ β ≤ ∞, there exists a countable graph G such that h(S G , σ) = α and where the latter is an entropy-like invariant defined by Salama.
Salama's methods are analytical, while our proof is topological. Lind [16] asked which numbers may be entropies of topological Markov chains on finite alphabets. This amounts to asking which numbers may be the spectral radii of finite directed graphs. We will need the following slight variation of a result of his.
Lemma 6 (Lind [16]). There is a dense subset D ⊂ [1, ∞) such that for every d ∈ D there exists a finite strongly connected directed graph G such that λ max Proof. Lind has shown that the set of Perron numbers, the real algebraic integers that dominate all their conjugates in absolute value, arise as spectral radii of positive integer matrices ( [16], Theorem 1), and that the Perron numbers are dense in [1, ∞) ( [16], Proposition 2). By a higher block representation ( [17], Sections 1.2-1.7), we obtain a {0, 1}-valued matrix with the same spectral radius. Since the spectral radius is realized in a strongly connected subgraph, we may choose the respective subgraph.
The following elementary lemma can be proven by considering the Rayleigh quotient ([18] Section 1.3).

Lemma 7.
Let G be a finite strongly connected graph and let S be a subgraph of G. Then λ max S < λ max G .
Proof. The full shift has infinite entropy; see Corollary 1. The infinite one-way path has entropy zero; see Example 2. Considering the interval (0, ∞) remains. Let r ∈ (1, ∞). Consider the set D from Lemma 6. Since D is a dense subset of the linearly ordered set R, there exists a monotonically increasing sequence {d i } in D such that d i → r. Let {G i } be a sequence of finite strongly connected directed graphs such that λ max G i = d i . The existence of such a sequence follows from Lemma 6. Take the disjoint union G of the topological Markov chains generated by G i , as in Example 1. Consider the subgraph of G induced by a finite subset F ⊂ N. By Lemma 7, its spectral radius is smaller or equal to the largest d i such that F intersects the vertex set of G i . We conclude that Since ln((1, ∞)) = (0, ∞), this proves the claim.
Note that the construction in the above proof yields a nonminimal subshift. There exist minimal subshifts of N Z with entropy zero-for example, the subshift obtained from the infinibonacci substitution [19]. A construction of Grillenberger [20] provides a minimal subshift of N Z with infinite entropy.

Examples
Example 2. Consider the subshift C ⊂ Z Z defined by C := s ∈ Z Z s t+1 = s t + 1 for all t ∈ Z .
We have h(C, σ) = 0 since W t {−n,n} ≤ 2n + 1. This subshift is generated by walks along an infinite graph, the infinite one-way path. Its spectral radius is 0. Example 3. Consider the subshift P ⊂ Z Z defined by P := s ∈ Z Z |s t+1 − s t | = 1 for all t ∈ Z . This subshift is generated by walks along an infinite graph, the infinite two-way path, whose spectral radius is 2.

Example 4.
Consider the lattice Z d as an undirected graph. Its spectral radius is 2d. Hence, the associated topological Markov chain has entropy ln(2) + ln(d). (This generalizes Example 3).

• •
The latter is a quotient of the former, yet the former has entropy zero, while the latter has entropy ln(2). By Proposition 1, the quotient map does not induce a continuous map between the subshifts, yet it induces an equivariant surjection whose image, the entire 2-shift, is closed. M(S G ) is continuous. Since M is a continuous map from a compact space to a Hausdorff space, its image is closed. We conclude that M is a morphism. The image M(S G ) ⊂ Z ∪ { * } Z is not generated by a graph, since whenever the symbol * appears, it must appear in a succession of evenly many * -symbols, which may not be encoded in a graph. We have ln(2) ≤ h(M(S G ), σ) < ln(2.66). This is a play on the even shift of Weiss [21].
Example 8. By considering S G ⊆ Alex(N) Z , one may adjoin many sequences that contain the symbol ∞. An example is the countable graph G on N which contains the edge (i, j) if and only if i ≤ j. Then S G \ S G equals the union of the set s ∈ N Z ∃z ∈ Z such that s t−1 ≤ s t < ∞ for all t < z and s t = ∞ for all t ≥ z with the constant sequence at ∞.