Present Address: Mathematics Department, Technion—Israel Institute of Technology, Technion City, Haifa 3200003, Israel; E-Mail:

This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

We present two examples of finite-alphabet, infinite excess entropy processes generated by stationary hidden Markov models (HMMs) with countable state sets. The first, simpler example is not ergodic, but the second is. These are the first explicit constructions of processes of this type.

For a stationary process (_{t}_{−}_{2}_{−}_{1} and the infinite future _{0}_{1} . . .. It has a long history and is widely employed as a measure of correlation and complexity in a variety of fields, from ergodic theory and dynamical systems to neuroscience and linguistics [

An important question in classifying a given process is whether the excess entropy is finite or infinite. In the former case the process is said to be

Over a finite alphabet, most of the commonly studied, simple process types are always finitary, including all independent identically distributed (IID) processes, finite-order Markov processes, and processes with finite-state hidden Markov model (HMM) presentations. However, there are also well known examples of finite-alphabet, infinitary processes. For instance, the symbolic dynamics at the onset of chaos in the logistic map and similar dynamical systems [

These latter processes, though, only admit stationary HMM presentations with uncountable state sets. Indeed, one can show that any process generated by a stationary, countable-state HMM either has positive entropy rate or consists entirely of periodic sequences, which these do not. Versions of the

Here, we present two examples of stationary, countable-state HMMs that do generate finite-alphabet, infinitary processes. To the best of our knowledge, these are the first explicit constructions of this type in the literature. Although, subsequent to our release of the earlier version of the present work [

Our first example is nonergodic, and the information conveyed from the past to the future essentially consists of the ergodic component along a given realization. This example is straightforward to construct and, though previously unpublished, others are likely aware of it or similar constructions. The second, ergodic example, though, is more involved, and both its structure and properties are novel.

To put these contributions in perspective, we note that

We denote by _{t}

_{t}_{t}_{∈ℤ} _{−}_{2}_{−}_{1} _{0}_{1} . . . :

^{t}_{−t}_{−}_{1} ^{t}_{0} _{t−}_{1}

As noted in [

where

That is, the excess entropy

Expanding the block entropy ^{t}

where

the conditional entropy in the

There are two primary types of hidden Markov models: edge-emitting (or

^{(}^{x}^{)}}

_{(}_{x}_{)}_{x}_{∈
}^{(}^{x}^{)}

Visually, a hidden Markov model may be depicted as a directed graph with labeled edges. The vertices are the states

The operation of a HMM may be thought of as a weighted random walk on the associated graph. From the current state

We denote the state at time _{t}_{t}_{t}_{t}_{t}_{+1}. The state sequence (_{t}_{t}_{t}_{t}

In either case, the process (_{t}^{(}^{x}^{)}}_{0}) = ℙ(_{t}

where for a given word _{1}_{n}^{*}^{(}^{w}^{)} is the word transition matrix ^{(}^{w}^{)} = ^{(}^{w}^{1)} · · · ^{(wn)}

^{(}^{x}^{)}}_{t}_{t}_{≥0}

_{t}_{t}_{≥0} _{t}_{t}_{≥0} ^{*}, of which there are only countably many. By stationarity, a one-sided stationary process generated by such a nonstationary HMM can be uniquely extended to a two-sided stationary process. So, in a sense, any two-sided stationary process_{t}_{t}_{∈ℤ} _{t}_{t}_{∈ℤ} _{t}_{t}

We consider now an important property known as unifilarity. This property is useful in that many quantities are analytically computable only for unifilar HMMs. In particular, for unifilar HMMs the entropy rate

^{(}^{x}^{)}}

It is well known that for any finite-state, unifilar HMM the entropy rate in the output process (_{t}

where _{σ}_{σ}_{0}_{0} =

We are unaware, though, of any proof that this is generally true for countable-state HMMs. If the entropy in the stationary distribution

_{0}_{1}_{t}|X⃗^{t}^{t}^{t}_{0}_{1}_{t−}_{1}

In the

The HMM constructed in Section 3.2 is both exact and unifilar, so Proposition 1 applies. Using this explicit formula for

We now present the two constructions of (stationary) countable-state HMMs that generate infinitary processes. In the first example the output process is not ergodic, but in the second it is.

_{i}_{i}_{i}_{2}_{3}

Intuitively, the information transmitted from the past to the future for the HPM Process is the ergodic component _{i}_{2}_{3}

For the HPM Process ℘ we will show that (i) lim_{t}_{→∞} ^{t}

Note that any word _{i,t}

No two distinct states _{ij}_{ij′}

The sets _{i,t}, i_{t}

It follows that each word _{i,t}_{ij}

Hence, for any fixed

so:

which proves Claim (i). Now, to prove Claim (ii) consider the quantity:

On the one hand, for _{t}_{t}|X⃗^{t}_{t}_{t}_{t}|X⃗^{t}_{t}_{i}_{t}_{i>t/}_{2} _{i}_{i>t/}_{2} _{i}_{i}_{i}

By inspection we see that the machine is unifilar with _{1}_{0} = 4] = 0. Since the underlying Markov chain over states (_{t}_{t}_{t}_{σ}_{σ}h_{σ}_{t}_{σ}

where ℒ_{t}_{σ}_{t}^{t}_{w}_{σ}_{σ}h_{σ}

where _{w}_{t}|X⃗^{t}

As we will show in Claim 6, concavity of the entropy function implies the quantity _{w}_{w}_{w}–h̃_{w}_{t}_{t}

A more detailed analysis with the claims and their proofs is given below. In this we will use the following notation:

ℙ_{σ}_{0} =

_{t}_{t}_{t}_{t}

Note that:

and:

These facts will be used in the proof of Claim 1.

Let

Hence, the Markov chain is recurrent and we have:

from which it follows that the chain is also positive recurrent. Note that the topology of the chain implies the first return time may not be an odd integer greater than 1.

Existence of a unique stationary distribution ^{i}^{i}

Note that for all ^{i}

Also note that for any ^{i}

Therefore, for each

ℙ(_{t}_{t}

For any state

where the final equality follows from symmetry. We prove the bounds from above and below on ℙ(_{t}

Here, (a) follows from

Here, (a) follows from

ℙ(_{t}^{t}_{t}

Applying Claim 3 we have for any

By symmetry, ℙ(_{t}^{t}_{t}_{t}_{t}^{t}_{t}

_{t},

_{w}

_{w}

_{w}_{w}

^{i}_{t}^{i}_{t}_{w}_{t}

Let the random variable _{t}_{t}_{t}_{t}_{t}_{t}^{t}_{t}_{t}_{t}^{t}_{t}_{t}^{t}_{t}_{t}

_{t}, h_{w}_{w}

For _{t}_{w}_{t}|X⃗^{t}_{σ}_{t}|S_{t}_{t}

As noted above, since the machine satisfies the conditions of Proposition 1, the entropy rate is given by

With the above decay on

Any stationary, finite-alphabet process can be presented by a stationary HMM with an uncountable state set. Thus, there exist stationary HMMs with uncountable state sets capable of generating infinitary, finite-alphabet processes. It is impossible, however, to have a finite-state, stationary HMM that generates an infinitary process. The excess entropy

The second example, the Branching Copy Process, is also ergodic—a strong restriction. It is a priori quite plausible that infinite

Following the original release of the above results [^{t}^{t}

The authors thank Lukasz Debowski for helpful discussions. Nicholas F. Travers was partially supported on a National Science Foundation VIGRE fellowship. This material is based upon work supported by, or in part by, the US Army Research Laboratory and the US Army Research Office under grant number W911NF-12-1-0288 and the Defense Advanced Research Projects Agency (DARPA) Physical Intelligence project via subcontract No. 9060-000709. The views, opinions, and findings here are those of the authors and should not be interpreted as representing the official views or policies, either expressed or implied, of the DARPA or the Department of Defense.

We prove Proposition 1 from Section 2.2, which states that the entropy rate of any countable-state, exact, unifilar HMM is given by the standard formula:

Let ℒ_{t}_{t}_{σ}_{t}^{t}_{w}_{σ}_{σ}h_{σ}_{w}_{t}|X⃗^{t}

_{t}|X⃗^{t}_{w}_{∈ℒ t} ℙ(_{w}

∑_{σ}_{σ}h_{σ}_{σ}_{∑}_{w}_{∈ℒ }_{t}_{σ}_{σ}_{w}_{∈ℒt} ℙ(_{σ}_{σ}h_{σ}_{w}_{∈ℒt} ℙ(_{w}

Thus, since we know

Now, for any for any _{t}_{w}_{w}|_{1}_{t}_{t}|X⃗^{t}_{w}_{w}

where _{t}_{t}_{t}

Combining

The authors declare no conflict of interest.

Nicholas F. Travers and James P. Crutchfield designed research; Nicholas F. Travers performed research; Nicholas F. Travers and James P. Crutchfield wrote the paper. Both authors read and approved the final manuscript.

A hidden Markov model (the _{1}_{2}}, a two symbol alphabet
_{1}_{2}) = (1

A countable-state hidden Markov model (HMM) for the Heavy-Tailed Periodic Mixture Process. The machine _{i}, i_{i}_{i}^{2} ^{(}^{x}^{)}}_{ij}_{ij}^{2} log^{2}

A countable-state HMM for the Branching Copy Process. The machine _{i}_{i}_{i}^{i},_{i}_{i}_{i}^{2}^{2}] for all _{0} _{0}_{0}_{0})] ≤ 1