What Is a Pattern in Statistical Mechanics? Formalizing Structure and Patterns in One-Dimensional Spin Lattice Models with Computational Mechanics

Abstract

This work formalizes the notions of structure and pattern for three distinct one-dimensional spin-lattice models (finite-range Ising, solid-on-solid, and three-body), using information- and computation-theoretic methods. We begin by presenting a novel derivation of the Boltzmann distribution for finite one-dimensional spin configurations embedded in infinite ones. We next recast this distribution as a stochastic process, thereby enabling us to analyze each spin-lattice model within the theory of computational mechanics. In this framework, the process’s structure is quantified by excess entropy $\mathbf{E}$ (predictable information) and statistical complexity $C_{\mu }$ (stored information), and the process’s structure-generating mechanism is specified by its $ϵ$-machine. To assess compatibility with statistical mechanics, we compare the configurations jointly determined by the information measures and $ϵ$-machines to typical configurations drawn from the Boltzmann distribution, and we find agreement. We also include a self-contained primer on computational mechanics and provide code implementing the information measures and spin-model distributions.

1. Introduction

When observing a natural system, we intuitively explain it by describing the way its components are arranged. We might say that the system displays order or randomness. We might describe systems that exhibit a blending of order and randomness as complex or structured (This paper uses two notions of structure. One refers to a system’s general type of arrangement, which we call generic structure. The other captures a more specific type of arrangement—one that exhibits patterns—which we call intrinsic structure. Throughout the paper, the intended notion will be clear from context.) [1]. Moreover, we might also regard as structured those ordered systems that have no randomness but exhibit a repetition of more than one component (a period greater than 1) [2]. Altogether, we might regard a structured system simply as one that exhibits patterns [3].

In light of this depiction, a physicist may feel compelled to bring clarity and definiteness to the notions of randomness, structure and pattern by formalizing them. Although statistical mechanics readily concretizes randomness through measures like entropy [4,5], it falls short when quantifying structure and pattern and formalizing its supporting mechanism. For instance, while magnetization is commonly treated as an indicator of structure, materials with distinct magnetic behaviors, such as paramagnets and antiferromagnets, have the same magnetization in the absence of a magnetic field: zero [2].

Furthermore, in statistical mechanics, “pattern” is not typically defined explicitly; instead, the criteria that one might regard as proxies for pattern depend on a choice of representation. This choice can enter through the observable taken as relevant, the scale at which structure is probed, or the coarse-graining scheme used to obtain a macroscopic description [6]. For the observable, one may diagnose order using magnetization or staggered magnetization [7]. For the scale, one may use correlations or structure factors evaluated at a chosen length or wavenumber [8]. For coarse-graining, one may formalize large-scale organization through a specific RG blocking or decimation prescription [9].

Faced with these limitations, the physicist may make their endeavor more concrete by posing two key questions:

• What’s a simple system in statistical mechanics that manifests structure and patterns?
• How could one extend statistical mechanics to formalize structure and patterns within such a system?

One-dimensional (1D) spin lattice models [10] (p. 67) are suitable candidates for addressing these challenges, as they compactly represent interacting magnets as spins in an evenly spaced grid, embodying both simplicity [11] and structure/patterns [12]. The simplicity stems from the spins taking discrete values (often binary) and the spin models being amenable to both analytical and numerical treatment [13,14]. The structure and patterns are evident in the model’s possible spin configurations, which exhibit regularity, randomness, and structure. For example, the 1D nearest-neighbor Ising model may have configurations rich in regularity, randomness, and structure, such as $\uparrow \downarrow \uparrow \downarrow \uparrow \downarrow$, $\uparrow \downarrow \uparrow \downarrow \downarrow \downarrow \uparrow$ and $\downarrow \uparrow \uparrow \downarrow \uparrow \uparrow$, respectively. These configurations contain repeating sequences of spins that we refer to as configuration patterns.

Mathematically, a spin model is expressed as a Hamiltonian that characterizes the energy of the spin system [10] (p. 67). Given the Hamiltonian, the usual goal is to determine the partition function and from it compute various properties of interest [15]. Among these, the Boltzmann distribution as a function of spin configurations is the least frequently computed (When the Boltzmann distribution is calculated, it is typically expressed as a function of energy [16,17] or other macroscopic properties [18,19,20], rather than directly in terms of configurations of fixed length), yet it stands out as the sole one directly addressing spin configurations, serving as a window for analyzing their structure and patterns. However, to clearly see through this window, we need to carefully consider how the distribution is formalized.

Typically, the Boltzmann distribution is defined so that each configuration, either implicitly or explicitly, represents an event of a single random variable, as indicated in Refs. [21] (p. 552) and [22]. Nonetheless, this approach is not conducive to examining how individual spins make up spin configurations. Instead, we can regard them as realizations of a partially ordered chain of random variables—a stochastic process [2].

In this process, which we call the spin process, each spin corresponds to an event of a single random variable. Given this perspective, we can now quantify the randomness, regularity, and structure of the spin process, and formalize the mechanism that generates its structure. Since randomness, regularity, and structure are ways in which a process elicits surprise, we quantify them as information—a measure of “quantifiable surprise” [23] (p. 64) or a “difference that makes a difference” [24].

In information theory, the theory of quantifiable surprise, a stochastic process’s intrinsic randomness or average randomness per symbol is quantified by its Shannon entropy rate $h_{\mu }$ (Ref. [25], pp. 74–76). The process’s regularity, as the counterpart of its randomness, can be understood as the total correlation within the process. Thus, regularity is quantified as the amount of information that is shared within the process—that is, the process’s mutual information or excess entropy E [26,27,28,29].

Because a stochastic process’s structure is effectively captured by its patterns, we quantify the process’s structure by measuring the amount of information stored in those patterns. This quantity is known as the stored information, or statistical complexity $C_{\mu }$ [30,31], and is defined as the Shannon entropy of those patterns. Calculating $C_{\mu }$, therefore, requires identifying these patterns—an inference task that effectively uncovers the process’s underlying structure-generating mechanism. We define these patterns next.

Since patterns are sought for their predictive utility, we define a pattern in the spin process setup from a prediction-based viewpoint. To do so, we split (Without loss of generality) each realization of the spin process into a left half (past) and a right half (future). Then, we define a pattern as the set of pasts that lead to the same futures (It should be highlighted that for 1D spin lattice models, the conventional time index is taken to be site location index and there is no time dependence.) [32]. By “lead to”, we mean that the conditional distribution over futures, when conditioned on any past in the set, is identical across all those pasts. This condition is known as the causal equivalence principle (This principle formalizes the implicit definition of a state commonly used in theoretical computer science when constructing machines. In this context, a state represents the information that must be retained to predict the system’s future behavior (see Appendix A).) [31,32,33], which recasts these patterns as causal states. Why the term “state”? Because this conception of pattern is consistent with the theory of computation’s definition of state as a system’s entity that “remembers a relevant portion of the system’s history” [34] (pp. 2–3). This connection points us toward the mechanism that underpins the process’s structure.

Given that a system’s structure is measured in units of information, formalizing its supporting mechanism is tantamount to unraveling how the system processes and stores information—essentially, how it computes [35]. This leads to a refined question: what is the minimal (To avoid accounting for computation not inherent to our system) abstract machine (In the 21st century, “computation” often evokes laptops, which perform useful computation—that is, computation carried out for some external task. In contrast, we focus on intrinsic computation, the computation a system performs by itself. To analyze this, we use abstract machines [34]—mathematical models that consist of states and transitions and laid the groundwork for the theory of computation,) which performs the computation inherent to the spin process. Leveraging concepts from the theory of computation (TOC), computational mechanics provides a compelling response: the set of causal states and their transitions, that is a $ϵ$-machine or Probabilistic Deterministic Finite State Machine (PDFM). Here, “probabilistic” means that state transitions include probabilities, while “deterministic” implies that when we have knowledge of a state and its associated outgoing symbol, we have complete certainty about the next state we will transition to. Several methods have been developed for inferring $ϵ$-machines [36,37,38,39,40,41,42]. Among these, Feldman and Crutchfield’s approach stands out as the only one that is both analytical and applicable to statistical mechanics [2].

In particular, Feldman and Crutchfield used this method to examine the structure of the nearest-neighbor and next-nearest neighbor Ising models. Subsequent research further developed their information-theoretic analysis of spin systems by calculating $h_{\mu }$ and $\mathbf{E}$ for the two-dimensional nearest neighbor Ising model [43], as well as decomposing the nn Ising model’s Shannon entropy rate into more refined information components [44]. Moreover, quantum $ϵ$-machine formulations revealed striking memory advantages—ranging from extreme compression when simulating long-range Ising spin chains [45] to clarifying how simplicity differs in quantum versus classical descriptions [46]. Now, the aim of this paper is to develop information measures and $ϵ$-machines for three varied one-dimensional spin-lattice models—finite-range Ising, solid-on-solid, and three-body—and to assess the consistency of these results with statistical mechanics.

These developments are timely because they broaden the rapidly evolving landscape of abstract machines used to analyze computation in physical processes in two key ways. First, they encourage the application of abstract machines—which have most often been used to study thermodynamic [47,48,49,50] and quantum [51,52,53,54] processes—to statistical mechanical processes, potentially supporting more efficient information processing in materials. Second, these developments foster the use of abstract machines that are systematically inferred from data, rather than being designed in an ad hoc manner, as has more typically been the case.

To achieve the aim of this paper, we provide a pedagogical explanation of the application of computational mechanics to the nn and nnn Ising models, along with the necessary background from statistical mechanics, measure theory, stochastic processes, and information theory. We then apply these techniques to a wider range of spin models, such as finite-range Ising models, solid-on-solid models, and three-body models. In parallel, we find that the typical patterns observed in these spin models at various parameter values match those predicted by information measures and $ϵ$-machines. This allows us to present an account of spin patterns that is clearly consistent with statistical mechanics and information/computation theory.

2. Background and Methods

This section provides an intuition-first, pedagogical introduction to the concepts and methods required for our information- and computation-theoretic analysis of 1D spin lattice models. While the underlying machinery is standard in the literature [2,33,35], we restrict attention to the ingredients strictly necessary for our purposes, motivate each formal object intuitively, and ground each one in statistical mechanics. We provide detailed explanations of what each mathematical component means and what role it plays, instead of developing an abstract-first treatment aimed at greater formal generality.

2.1. Spin Measurements: Boltzmann Distribution of Finite Chain Embedded in Infinite Chain

The Boltzmann distribution serves as an entry point for probing the structure of spin models; however, defining it for both finite and infinite configurations introduces significant difficulties. For finite configurations, the Boltzmann distribution lacks generality and often relies on numerical simulations for approximation [55,56,57]. For infinite configurations, a different issue arises: their probability is zero [58] (pp. 94–97). This defies our expectation that they occur and results in an unnormalized total probability—a sum that is zero rather than one. To balance the constructiveness of finite configurations with the generality of infinite ones, we examine a hybrid configuration: a finite spin configuration embedded in an infinite one [59]. Figure 1 illustrates the finite configuration embedded within the infinite one. The key equations leading to the embedded distribution are presented below, with detailed derivations provided in Appendix F, Appendix G, Appendix H and Appendix I.

Consider a configuration consisting of N spins, where each spin can take one of two values (↑ or ↓) and interacts only with its nearest neighbors. For convenience, the configuration is subject to periodic boundary conditions:

$$s_0\dots s_{N-1}\mathrm{where}{s}_0=s_N.$$

(1)

The system is governed by a translationally-invariant Hamiltonian, that is, a Hamiltonian whose form remains the same across spin sites. It is defined as follows:

$$E(s_i,s_{i+1})=-J{s}_i{s}_{i+1}-{\displaystyle \frac{B}{2}}(s_i+s_{i+1}).$$

(2)

Next, the corresponding transfer matrix, with components $V(s_i,s_{i+1})=e^{-\beta E\left(s_i{s}_{i+1}\right)}$, is expressed as [10] (p. 68):

$$\mathbf{V}=\left[\begin{array}{cc}{e}^{-\beta E(\uparrow ,\uparrow )}& e^{-\beta E(\downarrow ,\uparrow )}\\ e^{-\beta E(\uparrow ,\downarrow )}& e^{-\beta E(\downarrow ,\downarrow )}\end{array}\right]$$

(3)

Then, the probability distribution for this spin configuration in the thermodynamic limit $N\to \infty$ is obtained in terms of the transfer matrix components and the transfer matrix’s principal eigenvalue $\lambda$ [10] (pp. 68–69):

$$\mathrm{Pr}(s_0,\dots s_{N-1})={\displaystyle \frac{{\displaystyle \prod _{i=0}^{N-1}}V(s_i,s_{i+1})}{{\lambda }^N}}$$

(4)

Now, consider a specific finite configuration of length L embedded in an infinite one:

$$\overrightarrow{s}=s_0\dots s_{L-1}\mathrm{where}L<N$$

(5)

Although the principal eigenvectors of the transfer matrix are seldom calculated in studies of spin models, they play a crucial role in defining the embedded distribution. Therefore, we obtain the normalized principal left and right eigenvectors of the transfer matrix, as provided in Ref. [10] (pp. 72–73). For conciseness, these are expressed in terms of the magnetization m, as shown below:

$${\mathbf{u}}^{\mathcal{L}}=\left[\begin{array}{cc}\sqrt{{\displaystyle \frac{1+m}{2}}}& \sqrt{{\displaystyle \frac{1-m}{2}}}\end{array}\right]\mathrm{and}{\mathbf{u}}^{\mathcal{R}}=\left[\begin{array}{c}\sqrt{{\displaystyle \frac{1+m}{2}}}\\ \sqrt{{\displaystyle \frac{1-m}{2}}}\end{array}\right]$$

(6)

Note that for the nn Ising model, the left and right eigenvectors are identical. Hence, in all subsequent subsections of this section, we omit the left and right superscripts.

Lastly, the probability distribution for the embedded configuration [59] is given by the following:

$$\mathrm{Pr}\left(\overrightarrow{s}\right)={\displaystyle \frac{u_{s_0}^{\mathcal{L}}{u}_{s_{L-1}}^{\mathcal{R}}{\displaystyle \prod _{i=0}^{L-2}}V(s_i,s_{i+1})}{{\lambda }^{L-1}}}$$

(7)

Here, we provide the physical interpretation for each part of the equation:

• In the denominator, $\lambda$ is raised to $L-1$ as each embedded configuration has L spins and its boundaries are not periodic.
• In the numerator, the product of transfer matrix components consists of $L-1$ factors. This reflects the fact that only the spins within the bulk have neighboring spins to interact with on both their left and right sides.
• Also in the numerator, we include two extra terms: $u_{s_0}^{\mathcal{L}}$ and $u_{s_{L-1}}^{\mathcal{R}}$, which are the normalized principal eigenvector components associated with the boundary spins $s_0$ and $s_{L-1}$. Since the embedded configuration does not have periodic boundaries, these extra terms ensure that the boundary spins contribute to the system’s magnetization as much as the bulk spins. Moreover, these terms are key to normalizing the joint probabilities.

To facilitate later discussion, it will be useful to denote the component associated with spins ↑ or ↓ as $u_{\uparrow }$, and $u_{\downarrow }$, respectively. The values of these components correspond to either $u_{s_0}^{\mathcal{L}}$ or $u_{s_{L-1}}^{\mathcal{R}}$, depending on whether the orientations of the spins $s_0$ and $s_{-1}$ are up or down. For example, in a spin configuration like $\downarrow \uparrow \uparrow \uparrow$, the component for the first spin $s_0$ is $u_{s_0}^{\mathcal{L}}=u_{\downarrow }=\sqrt{{\displaystyle \frac{1-m}{2}}}$, while the component for the last spin $s_3$ is $u_{s_3}^{\mathcal{R}}=u_{\uparrow }=\sqrt{{\displaystyle \frac{1+m}{2}}}$.

Alternatively, Equation (7) can be interpreted as the probability measure of a coarse-grained configuration. The nature of this coarse-graining and its implementation, which relies on measure theory, will be discussed in the following section.

2.2. Coarse-Graining via Measure Theory

In this section, we view finite configurations embedded in infinite ones as coarse-grained versions of infinite-spin configurations. Here, “coarse-grained” means a simplified representation that retains essential features while reducing detail [60]. The procedure for arriving at these representations—that is, coarse-graining—is up to the scientist’s discretion [61]. However, when treating the spin model as a stochastic process, the conventional approach is to reduce the degrees of freedom such that only contiguous ones remain [62,63,64]. This coarse-graining is physically motivated by the observer’s inability to record infinite measurements or degrees of freedom. To define the set of coarse-grained configurations mathematically, we begin with the full set of possible configurations.

Consider the set of all possible infinite spin configurations $\mathrm{\Omega }$. An individual configuration in this set is represented as $\sigma \in \mathrm{\Omega }$. The degree of freedom at a lattice site i within a configuration $\sigma$ is denoted by ${\sigma }_i$. Thus, a configuration in terms of its degrees of freedom is given by the following:

$$\sigma ={\sigma }_0\dots {\sigma }_{N-1}$$

with

$${\sigma }_0={\sigma }_N\mathrm{and}N\to \infty$$

The set of coarse-grained configurations ${\mathrm{\Omega }}_C$ is defined as the set of infinite configurations in which the contiguous spins from ${\sigma }_0$ to ${\sigma }_{L-1}$ have fixed indices and can take any value from $\{-1,1\}$.

This can be expressed as follows:

$${\mathrm{\Omega }}_C=\{\sigma \in \mathrm{\Omega }\mid {\sigma }_0,\dots ,{\sigma }_{L-1}\mathrm{have}\mathrm{fixed}\mathrm{indices}\}.$$

Alternatively, the set of coarse-grained configurations can be defined as follows:

$${\mathrm{\Omega }}_C=\{C_1,C_2,\dots \}$$

with each coarse-grained configuration $C_j$ defined as follows:

$$C_j=\{\sigma \in \mathrm{\Omega }\mid {\sigma }_0=s_0,\dots ,{\sigma }_{L-1}=s_{L-1}\}$$

where $s_0,\dots ,s_{L-1}$ represent the fixed spin values at fixed indices $0,\dots ,L-1$. In more compact notation, this is written as follows:

$$C_j=\{\sigma \in \mathrm{\Omega }\mid {\sigma }^L=s^L\}$$

Notably, the act of coarse-graining changes our focus from individual configurations to sets, where each set $C_j$ groups configurations by their shared spin values. Figure 2, shows how the set of all possible infinite spin configurations $\mathrm{\Omega }$ is partitioned into the set of coarse-grained configurations ${\mathrm{\Omega }}_C$. Accordingly, we must adapt our notion of probability to align with this perspective, transitioning from the concept of a probability distribution to that of a probability measure, as denoted by $\mu$ [65] (pp. 331–336).

To formalize this, we introduce the concept of a sigma algebra, denoted by $\mathcal{A}$. This is a collection of all subsets of ${\mathrm{\Omega }}_C$ that can be consistently assigned probabilities or measured, meaning they are physically relevant.

The sigma algebra $\mathcal{A}$ has three key properties:

1.

Entire Set Containment:$\mathcal{A}$ includes the sample space. In this case, that is the coarse-grained set of all infinite configurations ${\mathrm{\Omega }}_C$:

$${\mathrm{\Omega }}_C\in \mathcal{A}$$

2.

Complement Closure: If a set A is in $\mathcal{A}$, then its complement ${\mathrm{\Omega }}_C\setminus A$ must also be in $\mathcal{A}$:

$$A\in \mathcal{A}\Rightarrow {\mathrm{\Omega }}_C\setminus A\in \mathcal{A}$$

3.

Countable Union Closure: If $A_1,A_2,A_3,\dots$ are in $\mathcal{A}$, then their countable union is also in $\mathcal{A}$:

$$A_1,A_2,\cdots \in \mathcal{A}\Rightarrow \bigcup _{i=1}^{\infty }{A}_i\in \mathcal{A}$$

With the concept of a sigma algebra established, we can now turn to the probability measure. This measure is analogous to a probability distribution, but applies to sets rather than individual outcomes. It extends the key constructive properties of probability distributions—namely, nonnegativity, normalization, and additivity—from finite to infinite configurations.

The probability measure is formalized as a function

$$\mu :\mathcal{A}\to [0,1],$$

which assigns a probability to each event in $\mathcal{A}$ and satisfies the following three key properties:

1.

Nonnegativity: In the same way that joint probabilities for finite configurations are never negative, the probability measure assigned to any set in $\mathcal{A}$ must also be nonnegative.

$$\mu \left(A\right)\ge 0\mathrm{for}\mathrm{every}A\in \mathcal{A}.$$

2.

Normalization: Similar to the sum of joint probabilities for all configurations equaling 1, the probability measure for the entire sample space, the set of coarse-grained configurations ${\mathrm{\Omega }}_C$, must be 1.

$$\mu \left({\mathrm{\Omega }}_C\right)=1$$

3.

Countable additivity: Mirroring the additivity of joint probabilities, which asserts that the total probability of finite configurations equals the sum of their individual probabilities, probability measures demonstrate countable additivity. This property dictates that for any countable collection of non-overlapping sets (cylinder sets) ${\left\{A_i\right\}}_{i=1}^{\infty }$, the probability of their union is the sum of the probabilities of the individual sets:

$$\mu \left(\bigcup _{i=1}^{\infty }{A}_i\right)=\sum _{i=1}^{\infty }\mu \left(A_i\right),$$

where each $A_i$ is a cylinder set corresponding to a coarse-grained configuration, and the union represents the combined event of these configurations.

The last step in constructing the spin probability measure involves assigning each spin cylinder set’s probability measure the value of its associated embedded configuration’s probability. Notably, information measures in later sections are denoted with a $\mu$ subscript, indicating that their argument is a probability measure [2].

2.3. System and Measurements: Stochastic Processes

As mentioned in the introduction, we interpret configurations as realizations of a stochastic process. This section aims to delve further into this formalism by first explaining the reasons for departing from the conventional approach.

Traditionally, a spin configuration is represented as an event s of a random variable S. For example, a configuration with all spins pointing up is depicted as follows:

$$s=\dots \uparrow \uparrow \uparrow \dots$$

(8)

However, this formalism impedes a direct examination of individual spins and their interactions. Furthermore, it leads to an unwieldy number of possible events. To address these issues, we adopt a more nuanced approach. Instead of representing a configuration as a single event, we depict it as a specific realization of events:

$$\overleftrightarrow{s}=\dots s_{-1}{s}_0{s}_1\dots$$

(9)

This realization is an instance of a stochastic process, i.e., a partially-ordered chain of random variables:

$$\overleftrightarrow{S}=\dots S_{-1}{S}_0{S}_1\dots$$

(10)

whose associated probability distribution is given by the following:

$$Pr\left(\dots S_{-1}{S}_0{S}_1\dots \right).$$

(11)

Within this framework, the all-ups spin configuration is now denoted as follows:

$$\overleftrightarrow{s}=\dots \uparrow \uparrow \uparrow \uparrow \dots$$

(12)

Without loss of generality, we can split a process into two parts: the past process, defined as follows:

$$\overleftarrow{S}=\dots S_{-1}.$$

(13)

along with its associated past realization, and the future process, defined as follows:

$$\overrightarrow{S}=S_0\dots$$

(14)

along with its associated future realization.

For simplicity, we will use the terms “past” and “future” to refer to both processes and their associated realizations, with the specific meaning inferred from the context.

The spin stochastic process will be our object of study. In the following subsection, we will elaborate on how it relates to broader categories of processes, as seen in Ref. [66].

2.3.1. Types of Processes

Stationary Process

A process in which the statistical properties of its random variables remain invariant over time. These properties include but are not limited to mean, variance, or joint distribution.

Strictly Stationary Process

A process whose joint distribution remains invariant under shifts in time. In other words, a process whose random variables are time-translation invariant. That is, a process that satisfies the following:

$$Pr\left(S_t{S}_{t+1}\dots S_{t+L-1}\right)=Pr\left(S_0{S}_1\dots S_{L-1}\right).$$

(15)

Markovian Process

A process in which the probability distribution of the next random variable depends only on the preceding one. That is, a process whose joint distribution factors as follows:

$$Pr(\overleftrightarrow{S})=\dots Pr\left(S_i\mid S_{i-1}\right)Pr\left(S_{i+1}\mid S_i\right)\dots$$

(16)

R-Order Markovian Process

A process in which the probability distribution of the next random variable depends only on the R preceding ones. That is, a process whose joint distribution is given as follows:

$$Pr(\overleftrightarrow{S})=\dots Pr\left(S_i\mid S_{i-R},\dots ,S_{i-1}\right)\dots$$

(17)

Spin Process

A process whose associated probability distribution is generated by a spin Hamiltonian model. For the models considered in this work (finite-range Ising, Solid on solid, and Three-body models), this process is strictly stationary and Markovian or R-order Markovian.

We can now define information measures of randomness, regularity, and structure for a stochastic process, starting from the basics of information theory.

2.4. Information Measures

What is information? Information can be conceived as quantifiable surprise, defined in terms of probabilities [23] (p. 64). Through this lens, an event s that is not likely to occur is deemed surprising, thus carrying high informational content. This means that the information of an event $H\left(s\right)$ is inversely proportional to its probability, that is, $H\left(s\right)\propto {\displaystyle \frac{1}{p\left(s\right)}}$. More specifically, the event’s information content—termed self-information—[23] (p. 64) is defined as follows:

$$H\left(s\right)=-{log}_{2}p\left(s\right).$$

(18)

Here, the presence of the logarithm is a convenient guarantee that the self-information possesses the additive property [67]. That is, the total surprise from combining events 1 and 2 equals the sum of their individual surprises.

The natural next step is to consider a random variable S. Its information content is known as Shannon entropy. It is defined as the weighted sum of the self-information of each possible event within the variable. Mathematically, it is expressed as follows:

$$H\left(S\right)=-\sum _{s=\pm 1}{log}_{2}p\left(s\right)$$

(19)

Following this line of reasoning, we can define the conditional entropy (Ref. [67]; Ref. [25], p. 17) as the amount of information needed to specify a random variable $S_1$ given that a random variable $S_0$ is known.

$$H\left(S_1\right|S_0)=-\sum _{s_0,s_1=\pm 1}\mathrm{Pr}(s_0,s_1){log}_2\mathrm{Pr}\left(s_1\right|s_0)$$

(20)

Moreover, we can define the joint entropy (Ref. [67]; Ref. [25], pp. 16–17) as the amount of information contained in two random variables.

$$H(S_0,S_1)=-\sum _{s_0,s_1=\pm 1}\mathrm{Pr}(s_0,s_1){log}_2\mathrm{Pr}(s_0,s_1)$$

(21)

Now, how may we define the entropy of our object of interest, that is, the stochastic process? The simplest answer would be to consider the growth entropy [68], that is, the Shannon entropy of the entire process.

$$H\left(S^L\right)=-\sum _{s_0=\pm 1}\dots \sum _{s_{L-1}=\pm 1}\mathrm{Pr}\left(s^L\right){log}_2\mathrm{Pr}\left(s^L\right)$$

(22)

However, as the length of the process increases, the growth entropy also rises and ultimately diverges when the process extends towards infinity ($L\to \infty$). This raises the question: how can we capture the total information of a stochastic process? A solution lies in the Shannon entropy rate (Ref. [68]; Ref. [25], pp. 74–76) defined as follows:

$$h_{\mu }=\underset{L\to \infty }{lim}{\displaystyle \frac{H\left(S^L\right)}{L}}$$

(23)

Again, the symbol $\mu$ signifies that the Shannon entropy rate is calculated in terms of a probability measure. Notably, this rate can be simplified for stationary, Markovian processes, such as the spin process. For a stationary process, the entropy rate reduces to the following:

$$h_{\mu }=H\left(S_L\right|S_{L-1},\dots ,S_1).$$

(24)

If the process is also Markovian, it becomes the following:

$$h_{\mu }=H\left(S_0\right|S_{-1})$$

(25)

By recasting the Shannon entropy rate as a conditional entropy, we can understand it as the amount of surprise each spin contributes. This effectively measures the process’s randomness per spin. Furthermore, for one-dimensional spin models, the Shannon entropy rate matches the Boltzmann entropy density, the more familiar form of entropy in statistical mechanics, as shown in Appendix C.

Since the regularity of the spin process is interpreted as the information shared between the process’ past and future, the regularity is defined as the process’ mutual information or excess entropy [26,27,28,29]. Mathematically, it is defined for the spin process as follows:

$$\mathbf{E}=I(\overleftarrow{S};\overrightarrow{S})=I(S_{-1};S_0).$$

(26)

Therefore,

$$\begin{array}{c}\hfill \mathbf{E}=\sum _{s_{-1},s_0=\pm 1}\mathrm{Pr}(s_{-1},s_0){log}_2\left({\displaystyle \frac{\mathrm{Pr}(s_{-1},s_0)}{\mathrm{Pr}\left(s_{-1}\right)\mathrm{Pr}\left(s_0\right)}}\right)\end{array}.$$

(27)

Notably, the excess entropy $\mathbf{E}$ can be interpreted as predictable information. That is, it quantifies the amount of information an observer has for recognizing configuration patterns, even if that information is not enough to identify them. However, in the absence of entropy rate $h_{\mu }$, $\mathbf{E}$ is sufficient to determine how much information is required for the observer to achieve synchronization with the underlying configuration patterns. Synchronization, from this purely information-theoretic perspective, refers to the observer’s ability to recognize and discern these configuration patterns.

To measure the process’s structure or statistical complexity, we need to determine the asymptotic probabilities of its patterns or causal states $\mathcal{S}$. In general, this often requires inferring the process’s $ϵ$-machine, especially for non-Markovian processes [69] (p. 37). However, for the spin process, we can calculate them directly since we have a natural definition of causal states.

Given the Markovian nature of the spin process, the next spin only depends on the previous one. Thus, the probability distribution of future spins conditioned on past ones matches the probability distribution of the future conditioned on the previous spin being up or down. Now, since the probability of a spin up and the probability of a spin down add up to 1, and they represent the probability per site throughout the process, they can be interpreted as the asymptotic probabilities of the causal states. Therefore, the statistical complexity of the spin process can be quantified as follows [2]:

$$C_{\mu }=H\left(\text{“}\mathrm{patterns}\text{”}\right)=H\left(\mathcal{S}\right)=H\left(S_0\right).$$

(28)

Therefore,

$$\begin{array}{cc}\hfill C_{\mu }& =-\sum _{s_0=\pm 1}\mathrm{Pr}\left(s_0\right){log}_2\mathrm{Pr}\left(s_0\right)\hfill \\ \hfill & =-\sum _{s_0=\pm 1}\left(u_{s_0}^L{u}_{s_0}^R\right){log}_2\left(u_{s_0}^L{u}_{s_0}^R\right)\hfill \\ \hfill & =-\sum _{s_0=\pm 1}{u}_{s_0}^2{log}_2{u}_{s_0}^2\hfill \end{array}.$$

(29)

This information can be simply related via the identity $H\left(S_0\right)=H\left(S_0\right|S_{-1})+I(S_{-1},S_0)$ as

$$C_{\mu }=R{h}_{\mu }+\mathbf{E}.$$

(30)

This relationship [69] (p. 37) formalizes our intuition that structure is a blending of randomness and regularity. Here, R denotes the neighborhood radius, which equals 1 for the nn Ising model.

Since the causal states are sufficient to predict the process’s future, and considering that prediction is tantamount to reproduction, statistical complexity can be defined as the minimum amount of information required to reproduce the stochastic process [2]. As mentioned in the introduction, if structure is viewed as quantifiable information, then this suggests that the mechanism generating the structure can be described as a machine [32,35].

2.5. Structure: Computational Mechanics

To formalize the mechanism generating a physical process’s structure, the concept of a machine must be adapted to satisfy three statistical mechanical constraints:

• Be capable of reproducing ensembles;
• Possess a well-defined notion of “state”;
• Be derivable from first principles.

Computational mechanics meets the first requirement by enhancing the simplest machine in TOC, the Deterministic Finite State Machine (DFSM), with probabilistic features while keeping its determinism intact [33,70]. The former is achieved by incorporating probabilities into the state transitions, and the latter is maintained by ensuring that the probability of transitioning to the next state, given the current state and a specific outgoing symbol, is precisely one. These modifications result in a machine known as Probabilistic Finite State Machine (PDFM) or $ϵ$-machine.

The second requirement is fulfilled by operationalizing TOC’s conceptual definition of a state—an entity that “remembers a relevant portion of the system’s history” [34] (pp. 2–3)—as a causal state. A causal state is the collection of all past realizations that when individually conditioning the process’ future yield the same conditional probability distribution [32,33]. Notably, formalizing the notion of “state” is crucial not just for conceptual clarity, but also to satisfy the third requirement. The reason for this is that without a clear understanding of what states are, the procedure for inferring them is much less clear.

To satisfy the third condition, we recast the definition of causal state as a guiding principle for inferring causal states from realizations, that is, the causal equivalence principle [32,33]. It states that two past realizations belong to the same causal state if they yield the same conditional distributions over the process’s futures. In practice, this principle allows us to construct the underlying $ϵ$-machine of an ensemble.

In summary, the key ingredients of computational mechanics are the concepts of $ϵ$-machine, causal transition, causal state, and the causal equivalence principle [33]. While we introduced them in this order to capture how they would be rediscovered conceptually, we will now present them in reverse order to delve into their mathematical details more pedagogically.

Causal equivalence principle. Two pasts are considered causally equivalent if and only if they make the same prediction over the future, i.e.,

$$\overleftarrow{s}\sim {\overleftarrow{s}}^{\prime }⟺\mathrm{Pr}(\overrightarrow{S}|\overleftarrow{s})=\mathrm{Pr}(\overrightarrow{S}|{\overleftarrow{s}}^{\prime })$$

(31)

Effectively, this principle groups pasts that lead to the same future into what are known as causal states. To formalize what we mean by “leads,” a causal state is defined as follows:

Causal state. A triple that contains the following:

1.

An event with its associated probability of the causal state random variable $\mathcal{S}$:

$${\mathcal{S}}_i\mathrm{and}\mathrm{Pr}\left({\mathcal{S}}_i\right).$$

(32)

2.

A distribution of the future conditioned on the causal event, i.e., a morph:

$${\mathcal{M}}_i=\mathrm{Pr}\left(\overrightarrow{s}\right|{\mathcal{S}}_i).$$

(33)

3.

The set of histories that lead to the same morph:

$${\mathcal{H}}_i=\{\overleftarrow{s}|\mathrm{Pr}(\overrightarrow{S}|{\mathcal{S}}_i)=\mathrm{Pr}(\overrightarrow{S}|\overleftarrow{s})\}.$$

(34)

Now, assuming that our machine is deterministic in the computational theoretic sense, we can define the causal transition as follows:

Causal transition. The probability of transitioning from state ${\mathcal{S}}_i$ to state ${\mathcal{S}}_j$ while emitting the symbol $s\in \mathcal{A}$:

$$\begin{array}{cc}\hfill T_{ij}^{\left(s\right)}& =\mathrm{Pr}({\mathcal{S}}_j,s|{\mathcal{S}}_i)\hfill \\ \hfill & =\mathrm{Pr}\left({\mathcal{S}}_j\right|s,{\mathcal{S}}_i)\mathrm{Pr}\left(s\right|{\mathcal{S}}_i)\hfill \\ \hfill & =\mathrm{Pr}\left(s\right|{\mathcal{S}}_i)\hfill \end{array}.$$

(35)

These definitions allows us to construct the minimal machine supporting a stochastic process’ structure.

$\mathbf{ϵ}$-machine or PDFM. A pair that contains the following:

• The set of causal states;
• Transition dynamic (causal transitions gathered in a matrix) [31].

For inferring $ϵ$-machines, it will be important to distinguish between two types of causal states:

• Recurrent causal states: These are states to which the machine will repeatedly transition as it operates. Consequently, their asymptotic probability is non-zero.
• Transient causal states: These are states that the machine may reach temporarily but will not return to. As a result, their asymptotic probability is zero: $\mathrm{Pr}\left({\mathcal{S}}_i\right)=0$.

Notably, the connectivity and number of transient states specify how difficult it is to identify the periodicity of configurations. In other words, these transient states reflect the computational effort required to achieve synchronization with the recurrent causal states. Here, synchronization is recast as the observer achieving certainty about the recurrent causal state it occupies, even in systems with nonzero entropy rate $h_{\mu }$. Thus, transient states offer a computational perspective on synchronization, which completes the informational interpretation provided by the excess entropy $\mathbf{E}$.

Since this is a principled approach, we can infer our machine of interest, rather than design it. For spin processes, an analytical method exists for inferring recurrent causal states [2]. Moreover, transient states can be reconstructed from these recurrent states, as detailed in Appendix B of Ref. [2]. Below, we provide a step-by-step explanation of the analytical reconstruction method for recurrent causal states.

Analytical Method to Infer $ϵ$-Machines

1.

Consider a finite configuration of length $2L$ embedded in an infinite one.

$$\overleftrightarrow{s}=s_{-L}\dots s_{-1}{s}_0{s}_1\dots s_{L-1}$$

2.

Consider the joint probability of the embedded finite configuration.

$$\mathrm{Pr}\left(\overleftrightarrow{s}\right)={\displaystyle \frac{u_{s_{-L}}^{\mathcal{L}}{u}_{s_{L-1}}^{\mathcal{R}}{\displaystyle \prod _{i=-L}^{L-2}}{V}_{s_i{s}_{i+1}}}{{\lambda }^{2L-1}}}$$

(36)

3.

Compute the conditional probability of the right half of the configuration given the left half.

$$\mathrm{Pr}\left(\overrightarrow{s}\right|\overleftarrow{s})={\displaystyle \frac{u_{s_{L-1}}^{\mathcal{R}}}{u_{s_{-1}}^{\mathcal{R}}}}{\displaystyle \frac{{\displaystyle \prod _{i=-1}^{L-2}}{V}_{s_i{s}_{i+1}}}{{\lambda }^L}}$$

4.

Notice that the only past element the conditional probability depends on is its last spin $s_{-1}$. Thus, the conditional probability is Markovian.

$$\mathrm{Pr}\left(\overrightarrow{s}\right|\overleftarrow{s})=\mathrm{Pr}\left(\overrightarrow{s}\right|\mathrm{last}\mathrm{spin})$$

5.

Identify morphs.

$$\mathrm{Pr}\left(\overrightarrow{s}\right|{\mathcal{S}}_A)=\mathrm{Pr}(\overrightarrow{s}|\mathrm{pasts}\mathrm{whose}\mathrm{last}\mathrm{spin}\mathrm{is}\uparrow )$$

$$\mathrm{Pr}\left(\overrightarrow{s}\right|{\mathcal{S}}_B)=\mathrm{Pr}(\overrightarrow{s}|\mathrm{pasts}\mathrm{whose}\mathrm{last}\mathrm{spin}\mathrm{is}\downarrow )$$

6.

Identify the number of causal states.

Since there are two morphs, there are two

causal states at most

7.

Identify sets of histories that lead to the same morph.

$$\{\overleftarrow{s}|\mathrm{last}\mathrm{spin}\mathrm{is}\uparrow \}\mathrm{and}\{\overleftarrow{s}|\mathrm{last}\mathrm{spin}\mathrm{is}\downarrow \}$$

8.

Apply the definition of causal transitions.

$$T_{\mathcal{A}\mathcal{A}}^{(\uparrow )}=\mathrm{Pr}(\uparrow |\uparrow )={\displaystyle \frac{e^{\beta (J+B)}}{\lambda }}$$

$$T_{\mathcal{A}\mathcal{B}}^{(\downarrow )}=\mathrm{Pr}(\downarrow |\uparrow )={\displaystyle \frac{e^{-\beta J}}{\lambda }}\sqrt{{\displaystyle \frac{1-m}{1+m}}}$$

$$T_{\mathcal{B}\mathcal{B}}^{(\downarrow )}=\mathrm{Pr}(\downarrow |\downarrow )={\displaystyle \frac{e^{\beta (J-B)}}{\lambda }}$$

$$T_{\mathcal{B}\mathcal{A}}^{(\uparrow )}=\mathrm{Pr}(\uparrow |\downarrow )={\displaystyle \frac{e^{-\beta J}}{\lambda }}\sqrt{{\displaystyle \frac{1+m}{1-m}}}$$

9.

Calculate asymptotic causal state probabilities using two facts:

• $\mathrm{Pr}\left(\overrightarrow{s}\right|{\mathcal{S}}_A)=\mathrm{Pr}(\overrightarrow{s}|\uparrow )$
• $\mathrm{Pr}\left(\overrightarrow{s}\right|{\mathcal{S}}_B)=\mathrm{Pr}(\overrightarrow{s}|\downarrow )$
• $\mathrm{Pr}\left({\mathcal{S}}_A\right)+\mathrm{Pr}\left({\mathcal{S}}_B\right)=1$

Since $\mathrm{Pr}(\uparrow )+\mathrm{Pr}(\downarrow )=1$, by inspection, we have the following:

• $\mathrm{Pr}\left({\mathcal{S}}_A\right)=\mathrm{Pr}(\uparrow )=u_{\uparrow }^2={\displaystyle \frac{1+m}{2}}$
• $\mathrm{Pr}\left({\mathcal{S}}_B\right)=\mathrm{Pr}(\downarrow )=u_{\downarrow }^2={\displaystyle \frac{1-m}{2}}$

10.

Build transition dynamic T.

$$\mathbf{T}=\left[\begin{array}{ccc}0& \mathrm{Pr}\left({\mathcal{S}}_A\right)& \mathrm{Pr}\left({\mathcal{S}}_B\right)\\ 0& \mathrm{Pr}\left({\mathcal{S}}_A\right|{\mathcal{S}}_A)& \mathrm{Pr}\left({\mathcal{S}}_B\right|{\mathcal{S}}_A)\\ 0& \mathrm{Pr}\left({\mathcal{S}}_A\right|{\mathcal{S}}_B)& \mathrm{Pr}\left({\mathcal{S}}_B\right|{\mathcal{S}}_B)\end{array}\right]$$

$$=\left[\begin{array}{ccc}0& {\displaystyle \frac{1+m}{2}}& {\displaystyle \frac{1-m}{2}}\\ 0& {\displaystyle \frac{e^{\beta (J+B)}}{\lambda }}& {\displaystyle \frac{e^{-\beta J}}{\lambda }}\sqrt{{\displaystyle \frac{1-m}{1+m}}}\\ 0& {\displaystyle \frac{e^{-\beta J}}{\lambda }}\sqrt{{\displaystyle \frac{1+m}{1-m}}}& {\displaystyle \frac{e^{\beta (J-B)}}{\lambda }}\end{array}\right]$$

11.

Find the left eigenvector using $\langle \pi |\mathbf{T}=\langle \pi |$.

$$\langle \pi |=(0,{\displaystyle \frac{1+m}{2}},{\displaystyle \frac{1-m}{2}})$$

Since $\mathbf{T}$ is a stochastic matrix, this is its asymptotic probability distribution vector, which contains the causal states’ probabilities, as seen in Refs. [65] (p. 330), [71] and [72] (p. 128).

12.

Build HMM representation of $ϵ$-machine using the transition matrix $\mathbf{T}$. Details of the resulting machine, for the parameter values $J_1=1.0$, $B=0.35$, and $T=1.5$, are provided in Appendix E.

2.6. Patterns as $ϵ$-Machines

The following example illustrates how computational mechanics formalizes the concept of a pattern. Consider a spin configuration such as $\uparrow \downarrow \uparrow \downarrow \uparrow \downarrow$. When asked, “What’s the pattern in this configuration?”, an intuitive answer might be $\uparrow \downarrow$. However, if presented with an ensemble of spin configurations and posed with the same question, the concept of a pattern becomes vague. To reason towards a definition of pattern for ensembles, we can ask: “What’s the key property of a pattern?” A plausible candidate is that a pattern represents a compressed form of data that enables an observer to reproduce the original content [73]. Thus, we can then ask: “What’s the object that statistically reproduces such a configuration?” The framework of computational mechanics provides the answer: The $ϵ$-machine, which can be interpreted as a physical or ensemble pattern [32,33]. Figure 3 illustrates the relationship between configuration and ensemble patterns.

From this point forward, the plotted machines will be derived using the CMPy package, which implements a tree-reconstruction method for inferring $ϵ$-machines, as described in Refs. [30,31]. The transient and recurrent states of these machines are represented in purple and green, respectively. For clarity in visualization, spins ↑ and ↓, emitted during transitions between causal states, are represented as 1 and 0, respectively. The ensembles of spin models discussed in the following section include configurations of either 4 or 6 spins. Configurations with probabilities below $1\times {10}^{-5}$ are excluded from consideration.

Based on Figure 3, one might be tempted to conclude that an $ϵ$-machine is simply a Hidden Markov Model (HMM). However, that is not the case. The difference stems from how states in HMM and $ϵ$-machines are characterized; specifically, a causal state in an $ϵ$-machine is defined as a triple, as mentioned earlier. In contrast, the conventional use of HMMs typically equates a state directly with the outcome of a random variable, treating the state as a singular entity rather than a triple. The definition of a state in computational mechanics is crucial, as it provides the foundation for inferring states from first principles rather than manually designing them [30].

3. Results and Discussion

As discussed in the previous section, the embedded Boltzmann distribution generates a vast number of spin configurations, making the structure and pattern of an arbitrary configuration unrepresentative of the system’s overall structure and patterns. However, to compare the information measures and $ϵ$-machines to the Boltzmann ensemble, it may still be useful to examine the structure and patterns of the individual configurations that are most optimally representative. To achieve this, we focus on a specific kind of configuration: typical configurations. These configurations are the most likely outcomes generated by the embedded Boltzmann distribution of a given spin model. Among these, likely typical configurations have probabilities that are significantly higher than those of non-typical configurations, whereas unlikely typical configurations have probabilities that are only slightly higher than those of non-typical configurations. The patterns present in these typical configurations are referred to as typical configuration patterns.

The patterns and structures of both typical and non-typical configurations across different spin models are shaped by various parameters [74]. To identify commonalities in how these parameters contribute to the configurations’ structure and patterns, we propose classifying them into three distinct types. To illustrate this, we will reference the nearest-neighbor (nn) Ising model as an example while defining each type of parameter.

• Randomness Parameter: This parameter governs the degree of randomness within the system. As it increases, it leads configurations to become more uniformly likely. In the nn Ising model, temperature T usually fulfills this role.
• Periodicity Parameter (Type 1): This parameter enhances periodicity and, as it varies, biases the system toward configurations that consist exclusively of a single period. In the nn Ising model, the coupling constant B exemplifies this. It induces period 1 configurations whether B is significantly positive or negative. Specifically, a high positive B biases all spins to point upwards, while a high negative B results in all spins pointing downwards.
• Periodicity Parameter (Type 2): Similarly, this parameter enhances periodicity but, as it varies, steers the system towards typical configurations with multiple distinct periods. In the nn Ising model, this role is played by the coupling constant J. A high positive J value tends to produce period 1 configurations (all spins up), akin to B, but a negative J value leads to alternating spin configurations (e.g., up-down-up-down), indicating that the typical configuration can be of period 2.

3.1. Finite-Range Ising Model

The nearest-neighbor Ising model can be generalized to a finite-range model using Dobson’s spin block method [75]. This approach consists of redefining the model’s degrees of freedom from individual spins to blocks of spins. These spin blocks are only allowed to interact with their nearest-neighbor blocks. Equivalently, in terms of spin variables, a spin $s_i$ within a spin block ${\eta }_j$ is only allowed to interact with spins within the same block and spins within the nearest-neighbor spin blocks. Notably, every spin within a block will interact with all the spins within the same block. Nonetheless, a given spin won’t necessarily interact with all the spins from the nearest spin blocks unless the nearest-neighbor Ising model is the specific model under consideration [75]. The interactions of spins within spin blocks are illustrated in Figure 4.

The spin block method expresses the Hamiltonian of two interacting spin blocks ${\eta }_j$ and ${\eta }_{j+1}$ of the finite-range Ising model as the sum of three contributions, shown in Equation (37). The first is the energy within block ${\eta }_j$, encompassing the interactions among spins within the block as well as the interactions of each spin with the magnetic field. The second contribution is the interaction energy between blocks ${\eta }_j$ and ${\eta }_{j+1}$, which is determined solely by the interactions between spins in ${\eta }_j$ and spins in ${\eta }_{j+1}$. The third contribution is the energy within block ${\eta }_{j+1}$, which, like the first, consists of the interactions between spins inside the block and the interactions of these spins with the magnetic field [75]. The reduction of the finite-range Ising model Hamiltonian to the Hamiltonians of Ising models with neighboring radii $R=1$, 2, and 3 is shown in Appendix J:

$$\begin{array}{c}\hfill E({\eta }_j,{\eta }_{j+1})={\displaystyle \frac{1}{2}}{X}_{{\eta }_j}+Y_{{\eta }_j,{\eta }_{j+1}}+{\displaystyle \frac{1}{2}}{X}_{{\eta }_{j+1}}\end{array}$$

(37)

where

• $X_{{\eta }_j}=-\left(B{\displaystyle \sum _{i=0}^{n-1}}{s}_i^j+{\displaystyle \sum _{k=1}^n}{J}_k\left({\displaystyle \sum _{i=0}^{n-k-1}}{s}_i^j{s}_{i+k}^j\right)\right)$,
• $Y_{{\eta }_j,{\eta }_{j+1}}=-{\displaystyle \sum _{k=1}^n}{J}_k\left({\displaystyle \sum _{i=0}^{k-1}}{s}_{n-i-1}^j{s}_{k-i-1}^{j+1}\right)$.

The terms in $X_{{\eta }_j}$ and $Y_{{\eta }_j,{\eta }_{j+1}}$ have the physical interpretations described below:

• $-{\displaystyle \frac{B}{2}}{\displaystyle \sum _{i=0}^{n-1}}{s}_i^j$ represents the energy contribution from the interactions between each spin in the block ${\eta }_j$ and the magnetic field B. For $B>0$, configurations tend to have all spins pointing up, while for $B<0$ all spins pointing down are favored. Therefore, B acts as a type-1 periodicity parameter.
• $-{\displaystyle \frac{1}{2}}{\displaystyle \sum _{k=1}^n}{J}_k\left({\displaystyle \sum _{i=0}^{n-k-1}}{s}_i^j{s}_{i+k}^j\right)$ represents the energy from the neighbor interactions between the spins within block ${\eta }_j$. For $J_k>0$, spins tend to align either all up or all down, favoring period-1 configurations. When $J_k<0$, spin configurations of period-$2^R$ are prone to occur. Thus, $J_k$ serves as a type-2 periodicity parameter.
• $Y_{{\eta }_j,{\eta }_{j+1}}$ denotes the energy associated with interactions between spins in neighboring blocks ${\eta }_j$ and ${\eta }_{j+1}$. Since this term shares the same form and coupling as $-{\displaystyle \frac{1}{2}}{\displaystyle \sum _{k=1}^n}{J}_k\left({\displaystyle \sum _{i=0}^{n-k-1}}{s}_i^j{s}_{i+k}^j\right)$, it leads to the same configuration patterns for corresponding values of $J_k$. Thus, $J_k$ again acts as a type-2 periodicity parameter.

The next step is to determine how effective information measures are at detecting and distinguishing configuration patterns within typical configurations of finite-range Ising models. For this, we start by considering a next-nearest neighbor Ising model with a moderately negative next-nearest-neighbor coupling $J_2=-1.2$, a very weak magnetic field $B=0.05$ and a low temperature $T=1$. Figure 5a shows the model’s information measures $h_{\mu }$, $\mathbf{E}$ and $C_{\mu }$ as a function of the nearest-neighbor coupling $J_1\in [-8,8]$. To assess the detection capability of these measures, typical configurations generated by the finite-range Boltzmann distribution at various values of $J_1$ are displayed below the horizontal axis.

For a strongly negative nearest-neighbor coupling $J_1\in [-8,-7)$, $h_{\mu }$ approaches zero, while $\mathbf{E}\approx 1$, together suggesting the presence of period-2 typical configurations. In this regime, the ensemble exclusively adopts configurations that alternate between ↑ and ↓, confirming the period-2 pattern. These resulting configurations arise from the negative coupling $J_1$, which favors antiferromagnetic behavior [76,77].

For a strongly positive nearest-neighbor coupling $J_1\in [6,8]$, all information measures approach zero, implying a period-1 typical configuration. The resulting “all-ups” pattern observed at these values is consistent with these measures. This outcome is expected, as the positive coupling $J_1$ drives the system toward ferromagnetic alignment [76,77].

For nearest-neighbor coupling $J_1=0.2$, the system exhibits $h_{\mu }\approx 0.42$, $\mathbf{E}\approx 1.17$, and reaches a maximum $C_{\mu }\approx 2$. While $1\le \mathbf{E}<1.59$ would imply period-3 configurations in the absence of entropy rate, the significant value of $h_{\mu }$ results in $C_{\mu }=2$, pointing toward period-4 configurations. Consistently, at these parameter values, we observe period-4 patterns in the typical configurations, including $\uparrow \uparrow \downarrow \downarrow$, $\uparrow \downarrow \downarrow \uparrow$, $\downarrow \uparrow \uparrow \downarrow$ and $\downarrow \downarrow \uparrow \uparrow$. Physically, this behavior can be understood as a result of the antiferromagnetic effect of $J_2$ being more dominant than the contributions from B and $J_1$.

For $J_1=-2.5$ and $J_1=2.5$, all configurations have a probability of less than $0.1$. This indicates that, at these parameter values, the system does not have a typical configuration or preferred configuration pattern. Additionally, in the regions $J_1\in [-5,-1]\cup [2,5]$, we observe that $C_{\mu }$ is not constant, but exhibits significant variation. As a result, these regions can be seen as configuration transition zones where the typical configurations are shifting to new ones as the parameter of interest varies.

Now, consider a 3-range Ising model with negative neighbor couplings of decreasing magnitude $J_1=-2.8$, $J_2=-1.3$, $J_3=-0.45$ and low temperature $T=0.2$. Figure 5b shows the model’s information measures $h_{\mu }$, $\mathbf{E}$ and $C_{\mu }$ as a function of the magnetic field $B\in [0,13]$. As in Figure 5a, typical configurations at various values of B are included below the horizontal axis.

For a weak magnetic field $B\in [0,0.75]$, we observe $h_{\mu }\approx 0$, $\mathbf{E}\approx 1$, and $C_{\mu }\approx 1$, indicating that only period-2 configurations are present, with no possibility of other configurations, even as unlikely alternatives. This is further confirmed by the exclusivity of the alternating ↑ and ↓ configurations in this region. Moreover, for a strong magnetic field $B\in [10,13]$, all information measures approach zero, indicating that the system permits only period-1 configurations, consisting entirely of ↑ spins. This is further validated by the typical configurations calculated from the Boltzmann distribution. While the information measures and configuration patterns for these field ranges resemble those in Figure 5a, they begin to differ in the intermediate range of B.

For a moderate magnetic field $B\approx 4.2$, we observe $h_{\mu }\approx 0.1$, $\mathbf{E}\approx 1.4$, and $C_{\mu }\approx 1.7$, indicating the presence of period-3 typical configurations. This is confirmed by the configurations calculated using the Boltzmann distribution. These results can be attributed to the competing effects between the antiferromagnetic couplings and the positive magnetic field [78,79,80]. Moreover, in Figure 5a, the probability of each non-typical configuration for $J_1=0.2$ is less than $0.03$, while in Figure 5b, for $B\approx 4.2$, the probability of each non-typical configuration is less than $0.01$. The lower value of $h_{\mu }$ for $B\approx 4.2$ in Figure 5b, compared to that for $J_1=0.2$ in Figure 5a, indicates that $h_{\mu }$ effectively captures the likelihood of non-typical configurations.

For a strong magnetic field $B=7.5$, the typical configurations are period-4. Therefore, compared to Figure 5a, Figure 5b shows a greater variety of periodic patterns. Moreover, although the 3-range model in Figure 5b includes spins with two additional neighbors compared to the next-nearest neighbor model in Figure 5a, it does not exhibit configuration patterns of periodicity higher than period-4. This captures how different parameters can limit or expand the diversity of configuration patterns.

Notably, there is a dip around $B=4.5$, where $B\approx |J_1+J_2+J_3|$. This suggests that, when the magnetic field and the coupling parameters are in a state of competing balance without a clear dominant effect, the configuration patterns reach a complex yet not maximally intricate compromise. That is, their periodicity is higher than that of an antiferromagnet but still below the maximum possible within the range $B\in [0,13]$.

Figure 6 shows the $ϵ$-machines for 3-range Ising models at fixed values of the coupling, temperature, and magnetic field parameters. In panel (a), the parameters are a weak magnetic field $B=0.2$, a moderate temperature $T=4$, and weak ferromagnetic couplings $J_1=1$, $J_2=1$, and $J_3=1$. In panel (b), the parameters are a strong magnetic field $B=8$, a low temperature $T=0.2$, and moderate antiferromagnetic couplings $J_1=-3$, $J_2=-2$ and $J_3=-2$.

The $ϵ$-machine in Figure 6a exhibits the maximum possible number of recurrent states, given by $2^R=2^3=8$, where R is the number of spins in a given spin block [2]. Therefore, by the definition of causal states, each spin block leads to a distinct future. This creates a one-to-one correspondence between spin blocks and causal states [2]. Additionally, it has 7 transient states, determined by $2^R-1$ [2]. This indicates that 7 spin variables must be observed before the next spin, allowing the observer to discern the precise typical configuration pattern.

Figure 6b shows fewer recurrent states compared to Figure 6a. This is due to the stronger magnetic field B and lower temperature in Figure 6b, which bias typical configurations toward a period-1 pattern. Consequently, the variety of possible typical spin configurations is reduced, limiting the range of possible futures. Moreover, Figure 6b exhibits only 3 transient states. This can also be attributed to the bias toward period-1 configurations, as fewer spins need to be observed to discern the typical configuration pattern.

Furthermore, notice that Figure 6b has reduced connectivity compared to Figure 6a. Specifically, the causal states in Figure 6a each have two outgoing transitions, while in Figure 6b, only transient states have two outgoing transitions, and recurrent states have just one. This reduced connectivity is again a result of the low temperature, which limits the diversity of configuration patterns. Moreover, it can be further understood as a consequence of the balance between the magnetic field and coupling interactions, which leads to complex but not maximally intricate configuration patterns.

Ultimately, the smaller size and reduced connectivity of the machine in Figure 6b, compared to Figure 6a, indicate that it performs less computation. Moreover, both panels in Figure 6 illustrate that the number of causal states in a spin model does not always match the number of spin blocks; this occurs only when the model operates at maximum computational capacity. Instead, the number of causal states varies based on internal factors like interaction couplings and external conditions such as the magnetic field and temperature.

3.2. Solid on Solid Model

In 1951, Burton, Frank, and Cabrera (BFC) introduced a theory on the growth of real crystals in equilibrium, built upon earlier theories of perfect crystal growth [81]. BFC posited that crystal growth is driven by the presence of steps on the crystal surface, with the rate of growth determined by kinks in these steps.

In this context, a step refers to the edge of an incomplete molecular layer on a crystal surface [81]. The interface between real crystals and their vapor is an example of a step [82]. A kink, on the other hand, is an atomic site along a surface step where the atomic alignment at that point is disrupted.

In BFC’s theory, these kinks form on the surface at a specific temperature, referred to as the roughening temperature $T_R$. This prompted BFC to quantify surface roughness per molecule by comparing the potential energy per molecule at roughening and zero temperatures, as shown in Equation (38):

$$s={\displaystyle \frac{U_R-U_0}{U_0}}$$

(38)

Here, $U_0$ and $U_R$ represent the potential energy per molecule at zero and roughening temperatures, respectively. The difference $U_R-U_0$ is referred as the configurational potential energy, and provided BFC with a gateway to model crystal surfaces as spin lattice models.

They argued that for the $\left(001\right)$ surface of a simple cubic crystal, the configurational potential energy is equivalent to the difference in potential energy between any two molecules [81]. Consequently, this allows for the crystal surface to be modeled as a two-dimensional Ising model on a square lattice where each site is labeled by integer coordinates x and y. Thus, the potential energy between two molecules is given by the following:

$$u(\mu ,{\mu }^{\prime })=U|\mu -{\mu }^{\prime }|.$$

(39)

Moreover, by focusing on kinks along the interface/step of a crystal with its vapor, the problem can be simplified in two ways. First, all molecules on the surface to the left of the interface can be treated as spin up, and those to the right as spin down [83]. Second, these two regions can be regarded as forming a one-dimensional spin chain, reducing the Ising model from 2D to 1D [83], as depicted in Figure 7. This simplification is achieved by fixing the spins along the vertical boundaries at the extreme left $x=0$ to spin value 1 and at the extreme right $x=x_{\mathrm{high}}$ to spin value $-1$. These boundary conditions create a distinct transition in the lattice, where spin values switch from 1 to $-1$. As a result, the Hamiltonian describing the configurational energy between two molecules is given by the following:

$$U|n_j-n_{j+1}|$$

(40)

where $n_j$ represents the number of leftmost up spins in row j up to the interface at column i.

If we further require that each occupied site sits directly above another occupied site—meaning no “overhangs” are allowed—then the one-dimensional spin chain meets the solid-on-solid condition [82].

Furthermore, an attractive wall potential can be incorporated into the Hamiltonian of the configurational energy. Abraham demonstrated that this potential “straightens” the interface, provided that x is restricted to lie in the right half of the plane, i.e., $0\le x\le x_{\mathrm{high}}$ [84]. Following Privman et al. [83], a simple attractive wall potential can be expressed as:

$$W{\delta }_{1,n_y}$$

(41)

Moreover, an additional external short-range potential can be included, represented as follows:

$$E\left(n_y\right)\approx c{e}^{-a{n}_y},a>0$$

(42)

The resulting Hamiltonian for this system is given by Equation (43):

$$E=\sum _{y}U|n_y-n_{y-1}|-W{\delta }_{1,n_y}+E\left(n_y\right)$$

(43)

where

• $U|n_y-n_{y-1}|$ represents the energy cost of forming a kink in the interface. $U>0$ biases the system toward period-1 configurations, while $U<0$ favors alternating spins. Therefore, U acts as a periodicity parameter of type 2.
• $-W{\delta }_{1,n_y}$ represents the energy associated with pinning the interface to the wall [83]. For $W>0$, $n_y=1$ prevails, while for $W<0$, $n_y=0$ dominates. In both cases, the system favors period-1 configurations. Thus, W serves as a type 1 periodicity parameter.
• $E\left(n_y\right)$ represents the energy contribution from an external field that influences the interface’s orientation or tilt [83]. For $E>0$, an interface made up of 1s is favored, while for $E<0$, an interface made up of 0s is preferred. Therefore, the parameters in this term function as type-1 periodicity parameters.

In what follows, we restrict $n_y\in \{0,1\}$, so that under $s_y=2n_y-1$ the SOS Hamiltonian is equivalent to a nearest-neighbor 1D Ising chain. We compute the probabilities needed for the information measures and $ϵ$-machines directly from the Boltzmann distribution associated with Equation (43) using the transfer-matrix method.

We now aim to compare how turning the pinning wall W on and off affects both the configurations and information measures of the SOS model. For this comparison, we consider an SOS model with low temperature $T=1$, external potential $V=e^{-n_y}$ and pinning wall potential $W=0\mathrm{or}1$. Figure 8 displays the information measures of the SOS model as the kink coupling U varies. In Figure 8a,b, the pinning wall W is set to 0 and 1, respectively.

In both panels of Figure 8, $C_{\mu }$ ranges from 0 to 1, indicating typical configurations of either period-1 or period-2. In both figures, even a slight increase in the kink coupling above zero causes $C_{\mu }$ to reach its peak value. This behavior aligns with the Gibbsean assumption that a low cost of forming kinks makes non-uniform configurations—that is, non-period-1 configurations—more likely to occur [81].

In Figure 8a, $C_{\mu }$ reaches its peak just below 1, whereas in Figure 8b, it peaks around 0.75. Moreover, for $0<U<5$, $C_{\mu }$ stays higher in Figure 8a than in Figure 8b. This sustained higher value of $C_{\mu }$ in Figure 8a compared to Figure 8b is in line with the SOS Hamiltonian, which suggests that biasing the interface toward the pinning wall increases the likelihood of the interface becoming flat, that is, period-1 [81].

In Figure 8a, $\mathbf{E}$ peaks around $\mathbf{E}=0.26$ at $U=1.8$, while in Figure 8b, it peaks around $\mathbf{E}=0.04$ at $U=1$. This suggests that more spins need to be observed to determine the configuration pattern of the SOS model in Figure 8a compared to Figure 8b. This is consistent with period-1 configurations being more likely in Figure 8b, as these configurations do not require observing any spins.

At the $\mathbf{E}$ peak in Figure 8a, $C_{\mu }=0.75$, while at that of Figure 8b, $C_{\mu }=0.35$. This implies that period-2 configurations are more likely to occur in Figure 8a compared to Figure 8b. This aligns with typical period-1 configurations being less prevalent and, conversely, non-typical period-2 configurations being more frequent in Figure 8a compared to Figure 8b. Moreover, this is consistent with the physical expectation that biasing the interface to be attracted to the wall increases the likelihood that it becomes flat, thereby raising the probability of a period-1 configuration.

Furthermore, in both panels of Figure 8, as the kink coupling U increases, $h_{\mu }$ decreases. This trend is expected, as the higher cost of kink formation makes non-period-1 configurations less likely, thereby reducing the uncertainty of the next observed spin. The decrease occurs more rapidly in Figure 8b compared to Figure 8a. This can be explained by the presence of the pinning wall, which further encourages the dominance of flat, period-1 configurations.

Figure 9a,b show the $ϵ$-machines corresponding to the $\mathbf{E}$ peaks of Figure 8a,b. Both $ϵ$-machines feature two recurrent states and one transient state. However, as circled in red, the probability of transitioning from state A to state B while outputting symbol 0 in Figure 9a is more than twice as high as in Figure 9b. Moreover, as circled in blue, the probability of transitioning from state B to state B while outputting symbol 0 decreases from $0.73$ in Figure 9a to $0.27$ in Figure 9b. This bias towards period-1 configurations of the machine in Figure 9b suggests that it is easier for the machine in Figure 9b to synchronize than the one in in Figure 9a, which aligns with the fact that the $\mathbf{E}$ is higher for the machine in Figure 9b while both machines have similar values of $h_{\mu }$ (approximately 0.36 for Figure 9a and 0.31 for Figure 9b). Moreover, the outgoing transition probabilities from the transient state in Figure 9b are less uniform than those in Figure 9a. This suggests that while both machines can identify the “all-ups” configuration without observing any spins, this configuration is more representative of the machine in Figure 9b than of the one in Figure 9a. Computationally, this means that the behavior of the machine in Figure 9b more closely resembles that of a single-state machine that exclusively outputs symbol 1.

Lastly, note that the machines for the SOS model in Figure 8 and the nearest-neighbor Ising model in the Appendix E share similar recurrent states, transient states, and connectivity, but have different state transition probabilities. This suggests that the $ϵ$-machines offer a constructive framework for comparing the structures of different spin models and examining their similarities and differences.

3.3. Three-Body Model

Thermal desorption is the process of heating a solid surface to release a portion of its molecules [85,86]. The defining characteristic of this process is its kinetics, which are described by the desorption rate and the desorption rate constant, as outlined in [87] and presented in Equations (44) and (45), respectively. These two key equations are directly connected to experiment, as at sufficiently high pumping rates, the desorption rate equals the desorbant’s pressure [88]:

$${\displaystyle \frac{d\theta }{dt}}=-k_d\theta$$

(44)

$$k_d=\nu \sum _i{P}_{A,i}exp\left(-{\displaystyle \frac{E_d\left(0\right)-E_i}{T}}\right)$$

(45)

Detecting the temperatures at which desorption is greatest and identifying the qualitative properties of desorption at these values is crucial for various applications [88]. To achieve these objectives, the negative desorption rate is plotted against temperature to obtain the “desorption spectrum” [88]. The peaks in this spectrum indicate the temperatures at which the desorption rate is highest. These peaks vary in width, height, and location depending on the coverage, temperature, and material examined. For the case of the desorption spectrum of CO from Ni, Pd, Pt, Rh, and Ru closed-packed faces of single crystals, two distinguishing qualitative features arise, as demonstrated by Morris et al. [89]:

• The splitting of thermal desorption peaks becomes progressively weaker as one goes from Ni to Ru.
• The integral intensities of the peaks are distinct.

While nearest-neighbor (nn) and next-nearest-neighbor (nnn) spin models had been used to model thermal desorption [90], they did not capture the aforementioned properties. Myshlyavtsev et al. addressed this limitation by incorporating a three-body term in the spin Hamiltonian, which effectively models these characteristics [91]. The resulting three-body model removes the assumption of paired interactions [90], providing a more accurate account of the CO desorption process from metal surfaces. The 1D model is exactly solvable and, if lateral interactions are anisotropic, sufficient to capture thermal desorption, making it of theoretical and practical interest, respectively [91]. The spin interactions in the 1D three-body model are illustrated in Figure 10. The Hamiltonian for this model is given in Equation (46), and the corresponding transfer matrix is detailed in Appendix K.

$$E(s_i,s_{i+1},s_{i+2})=\sum _i-J_1{s}_i{s}_{i+1}-J_2{s}_i{s}_{i+2}-J_{\mathrm{tb}}{s}_i{s}_{i+1}{s}_{i+2}$$

(46)

where

• $-J_1{s}_i{s}_{i+1}$ is the term associated with the nearest-neighbor coupling. For $J_1>0$, the model induces period-1 configurations, while for $J_1<0$, the model induces period-2 configurations. Thus, $J_1$ serves as a type-2 periodicity parameter.
• $-J_1{s}_i{s}_{i+2}$ is the energy contribution of the next-nearest-neighbour coupling. When $J_2>0$, the model tends toward period-1 configurations, whereas for $J_2<0$, it leans toward period-4 configurations. Therefore, $J_2$ acts as a periodicity parameter of type 2.
• $-J_{\mathrm{tb}}{s}_i{s}_{i+1}{s}_{i+2}$ is the expression that represents the three-body interaction. When $J_{\mathit{tb}}>0$, the configurations are biased toward a period-1 pattern, while $J_{\mathit{tb}}<0$ favors period-4 configurations. As a result, $J_{\mathrm{tb}}$ functions as a type 2 periodicity parameter.

The purpose of Figure 11 is to illustrate how turning the nearest-neighbor coupling on and off in a three-body model affects both its configurations and information measures as a parameter of interest varies. Temperature is chosen as that parameter because it plays a key role in thermal desorption applications, where the goal is to identify the temperature that maximizes desorption [88,91]. In both panels, the next-nearest-neighbor coupling $J_2$ is set to 0 to highlight the role of the nearest-neighbor coupling $J_1$, while the three-body coupling $J_{\mathrm{tb}}$ is set to $-1$. However, in Figure 11a, the nearest-neighbor coupling $J_1$ is set to 0, whereas in Figure 11b, it is set to 1.

In both Figure 11a,b, $C_{\mu }$ increases and reaches its maximum value of $C_{\mu }=2$ as the temperature T rises, but the starting values differ. In Figure 11a, $C_{\mu }$ begins around $1.9$, whereas in Figure 11b, it starts at approximately $C_{\mu }\approx 1.58$. This suggests that at low temperature values, the typical configurations in Figure 11a are period-4, and in Figure 11b, they are period-3. This difference can be attributed to the fact that Figure 11b involves competing couplings, whereas Figure 11a does not, as it only includes the three-body coupling. In particular, in both Figure 11a,b, the three-body coupling $J_{\mathrm{tb}}$ biases configurations toward a period-4 pattern. However, in Figure 11b, the ferromagnetic coupling $J_1$ also biases configurations toward a period-1 pattern. The competition leads to a compromise, resulting in period-3 configurations. This is consistent with the low-temperature typical configurations calculated using the Boltzmann distribution, which are shown below the horizontal axis in Figure 11b.

Moreover, the nearest-neighbor coupling significantly reduces the uncertainty in predicting the next spin by expanding the neighborhood of spins that each state affects. This leads to a lower $h_{\mu }$ at very low temperatures in Figure 11b compared to Figure 11a. This prevents $C_{\mu }$ in Figure 11b from being strongly influenced by $h_{\mu }$ at very low temperatures.

Furthermore, although $C_{\mu }$ is higher in Figure 11a than in Figure 11b at low temperatures, $\mathbf{E}$ is lower in Figure 11a compared to Figure 11b at the same temperatures. This implies that while typical configurations in Figure 11a at very low temperatures exhibit greater periodicity than those in Figure 11b (period-4 versus period-3), the observer must examine more spin variables to discern the configuration pattern in Figure 11b. While this might seem to suggest that patterns in Figure 11b are easier to discern than those in Figure 11a, the uncertainty per spin in Figure 11b is significantly higher. Specifically, $h_{\mu }\approx 0$ for Figure 11a, whereas $h_{\mu }\approx 0.9$ for Figure 11b. This substantial difference makes an information-theoretic approach based on excess entropy $\mathbf{E}$ insufficient for determining the ease of synchronization. We will soon address this by examining the computational properties of the three-body models.

Notably, the information measures of the three-body model reveal new features that were absent in previously studied spin models. For instance, unlike the dependence of $\mathbf{E}$ on temperature in the nearest-neighbor Ising model, where $\mathbf{E}$ decays to 0 as T increases (as shown in Ref. [2] and Appendix B), $\mathbf{E}$ for the three-body model remains nonzero even at high temperatures. Moreover, even though there is no magnetic field B in Figure 11b, the information measures are not flat across the temperature range. This suggests that a diversity of configuration patterns is possible whenever competing parameters are present, regardless of their specific nature, which further reinforces the usefulness of our classification of parameter types. Ultimately, the information measures plots in Figure 5, Figure 8 and Figure 11 suggest that different spin models give rise to distinct configuration patterns and structural behavior.

Figure 12 aims to illustrate the structural changes in the $ϵ$-machine of a three-body model with competing couplings as the temperature increases. The plots in Figure 12a and Figure 12b depict the $ϵ$-machines corresponding to Figure 11b at a very low temperature $T=0.025$ and a low temperature $T=2$, respectively.

The outgoing probabilities from the causal transient state A to the transient states B and C in Figure 12a, which are circled in red and blue, are less uniform than those in Figure 12b. This implies that the $ϵ$-machine in Figure 12a is easier to synchronize than the one in Figure 12b. At first, this may seem inconsistent with their excess entropy values, given that $\mathbf{E}\approx 1.58$ for Figure 12a and $\mathbf{E}=1$ for Figure 12b, as shown in Figure 11b. However, this apparent contradiction is resolved by observing the significantly higher value of $h_{\mu }$ in Figure 12b compared to Figure 12a, where $h_{\mu }\approx 1$ for Figure 12b and $h_{\mu }\approx 0$ for Figure 12a. As a result, while discerning configuration patterns in Figure 12a may require an additional spin, the much higher uncertainty in predicting the next spin in Figure 12b outweighs this requirement, making synchronization more challenging in Figure 12b than in Figure 12a. This uncertainty is further supported by the fact that typical configurations for Figure 12b are much less probable than those in Figure 12a. Specifically, the highest probability for a typical configuration in Figure 12a is $0.33$, whereas in Figure 12b, it is only $0.025$. This contrast highlights how the computational approach provided by $ϵ$-machines offers a more nuanced perspective on synchronization than the randomness-agnostic viewpoint of excess entropy $\mathbf{E}$.

Moreover, the recurrent part of Figure 12a is much less connected than that of Figure 12b. In the $ϵ$-machine for Figure 12a, each recurrent causal state has only one outgoing transition with probability $1.0$. In contrast, the recurrent states in Figure 12b each have two outgoing transitions, both with probabilities close to $0.50$. Furthermore, Figure 12b includes self-loops that enable it to recognize period-1 configurations consisting entirely of 0s or 1s, a feature absent in Figure 12a. This indicates that the machine in Figure 12b generates a greater variety of spin configurations compared to the one in Figure 12a. This observation is consistent with the fact that at $T=0.025$, there are only three typical configurations, whereas at $T=2$, there are six.

Lastly, the number of recurrent causal states, along with the low connectivity of the machine in Figure 12a, suggests that it can support configurations with periods of up to 3. In contrast, the machine in Figure 12b, which has the same number of recurrent causal states but higher connectivity, permits configurations with periods of up to 4. The typical configurations in Figure 11b reflect this pattern, as Figure 12b accommodates both period-4 and period-3 configurations, whereas Figure 12a only supports period-3 configurations. Ultimately, this comparison of $ϵ$-machines underscores the importance of considering not only typical configurations but also their probabilities when developing a computation-theoretic account of spin patterns.

4. Conclusions

What, then, is a pattern in statistical mechanics? If one recasts the mechanism generating a system’s structure as an information processor, the answer for the one-dimensional spin models studied here is clear: the $ϵ$-machine. To support this perspective, we began by introducing computational mechanics and its application to statistical mechanics in a conceptual manner with only the necessary amount of mathematics. We then defined typical configurations and typical configuration patterns as the most likely configurations and configuration patterns within an ensemble. Furthermore, we classified the parameters of spin models according to the type of behavior they give rise to.

Using this framework, we computed typical configurations from the embedded Boltzmann distribution and compared them to those implied by information measures and $ϵ$-machines for three different spin models: the finite-range Ising model, the SOS model, and the three-body model. Our findings confirmed consistency between the results, establishing the $ϵ$-machine as a representation of the Boltzmann distribution’s ensemble patterns. Moreover, our analysis showed that information measures and $ϵ$-machines offer a detailed and nuanced characterization of typical configuration patterns, allowing us to distinguish between them and identify their shared features.

In the finite-range Ising model, the information plots show that $C_{\mu }$ serves as a simple visual indicator of regions where no typical configurations exist. These regions, distinguished by the non-flat behavior of $C_{\mu }$, are what we refer to as transition zones. Furthermore, $C_{\mu }$ captures the fact that different parameters influence the diversity of configuration patterns, and consequently, the computational demands. For instance, a dominant antiferromagnetic $J_2$ coupling maximizes computation, while competing effects between B and antiferromagnetic $J_1$ lead to high but constrained computation.

Moreover, the $ϵ$-machines of the finite-range Ising model provide a more refined perspective on the computational differences arising from varying parameters. For instance, the high but constrained computation observed in the three-range Ising model with a high magnetic field and low temperature is represented by fewer causal states and lower connectivity compared to a system with a low magnetic field and moderate temperature. This distinction provides a more nuanced understanding of what it means for a system to require more or less computation. Additionally, the analysis shows that the number of causal states cannot simply be inferred from properties such as the number of neighbors a given spin has or the magnitude and sign of the parameters.

In the SOS model, $C_{\mu }$ allows us to quantify the reduction in computational effort caused by turning the wall on, even when the typical configuration remains unchanged. Furthermore, the observation that maximum $C_{\mu }$ occurs at very low kink coupling demonstrates that the peak of maximum $C_{\mu }$ varies depending on the specific parameter under consideration. Moreover, the $ϵ$-machines of the SOS model show that turning on the wall parameter reduces the uniformity of the outgoing transition probabilities from the start state. This indicates that the typical configuration becomes more likely as the wall parameter becomes nonzero. In computational terms, when the wall is fully activated, the machine becomes more similar to a single-state machine that solely outputs 1. More broadly, the machines from this case study, along with those of the nearest-neighbor Ising model, demonstrate that $ϵ$-machines provide a unified framework for identifying computational similarities (such as the number of states and connectivity) and differences (such as transition probabilities) between two distinct spin models.

The information measures of the three-body models, both with and without nearest-neighbor coupling, are not monotonically dependent. Specifically, a high $\mathbf{E}$ or $h_{\mu }$ is shown to not necessarily imply a high or low $C_{\mu }$. The information plots of these three body models, along with those of the finite-range Ising models, capture how different spin models produce distinct configuration patterns when the same parameter, in this case temperature, is varied. Furthermore, the $ϵ$-machines of the three-body model with nearest-neighbor coupling provide an effective framework for identifying computational similarities and differences in the spin model as a parameter, such as temperature, changes. As temperature increases, typical configurations become more periodic, but also less likely to occur. The $ϵ$-machines capture this behavior by making the outgoing probabilities from transient causal states more uniform while increasing connectivity in the recurrent portion. This suggests that as typical configuration patterns become more periodic and less likely, they also become harder to discern overall. Notably, this result highlights the limitations of an information-theoretic perspective on synchronization—while useful, it remains incomplete without a computational viewpoint. This insight sheds light on subtle structural differences between systems that, despite having the same number of recurrent and transient causal states, exhibit distinct dynamical behaviors.

Ultimately, information theory and computational mechanics provide powerful tools for defining patterns in Boltzmann ensembles and for comprehensively characterizing the typical configurations generated by the Boltzmann distribution. They also enable a unified way of examining similarities and differences in the structure and patterns of a spin model under varying parameters and across different spin models. This perspective connects the abstract formalism of information theory and automata theory with the concrete physical models of statistical mechanics, providing a constructive and effective language to describe patterns in statistical mechanics.