Potential of Entropic Force in Markov Systems with Nonequilibrium Steady State, Generalized Gibbs Function and Criticality

Abstract

In this paper, we revisit the notion of the “minus logarithm of stationary probability” as a generalized potential in nonequilibrium systems and attempt to illustrate its central role in an axiomatic approach to stochastic nonequilibrium thermodynamics of complex systems. It is demonstrated that this quantity arises naturally through both monotonicity results of Markov processes and as the rate function when a stochastic process approaches a deterministic limit. We then undertake a more detailed mathematical analysis of the consequences of this quantity, culminating in a necessary and sufficient condition for the criticality of stochastic systems. This condition is then discussed in the context of recent results about criticality in biological systems.

1. Introduction

This is part II of a series on stochastic nonlinear dynamics of complex systems. Part I [1] presents a chemical reaction kinetic perspective on complex systems in terms of a mesoscopic stochastic nonlinear kinetic approach (e.g., Delbrück–Gillespie processes) as well as a stochastic nonequilibrium thermodynamics (stoc-NET) in phase space. One particularly important feature of the theory in [1] is that it takes the abstract mathematical concepts seriously—that is, it follows what the mathematics tells us [2]. For example, it was shown that the widely employed local equilibrium assumption in the traditional macroscopic theory of NET can be eliminated when one recognizes the fine distinction between the set of random events, the $\mathcal{S}$ in a probability space $(\mathcal{S},\mathcal{F},P)$ and a random variable that is defined as an observable on the top of the measurable space, $\mathbf{x}:\mathcal{S}\to \mathbb{R}$. The local equilibrium assumption is needed only when one applies the phase space stoc-NET to physically measurable transport processes [3].

The same chemical kinetic approach can be applied to other biological systems. Biological organisms are complex systems with a large number of heterogeneous constituents, which can be thought of as “individuals”. To be able to develop a scientific theory for such a complex system with any predictive power, one must use a probabilistic treatment that classifies the individuals into “statistically identical groups” [4,5,6]. Thermodynamics and statistical mechanics provide a powerful conceptual framework, as well as a set of tools with which one can comprehend and analyze these systems. The fully developed statistical thermodynamic theory taught in college physics classes is mainly a theory of equilibrium systems. The application of its fundamental ideas, however, is not limited to just equilibrium systems or molecular processes. Stoc-NET [3,7,8,9,10], along with the information theoretical approach [6,11,12,13,14], is a further development in this area.

One of the key elements of the theory presented in [1] was the nonequilibrium steady state (NESS) potential, or “energy”, defined as the minus logarithm of the stationary probability distribution of a kinetic model. In the past, this quantity has appeared repeatedly in the literature [15,16,17,18,19], but most of the studies focus on its computation. In this paper, we attempt to illustrate its central role as a novel “law of force”, a necessary theoretical element in the stoc-NET of complex systems.

Once this connection between energy and probability is established, it is possible to formally define probabalistic quantities analagous to other physical variables such as temperature and entropy. (Notice that the term “entropy” is somewhat overloaded. In the context of probability distributions, it typically refers to the Shannon entropy $S=\int p(x)\ln p(x)\mathrm{d}x$. In statistical physics, it is more often used to refer to the Gibbs or Boltzmann entropy, both of which are defined in terms of the volume of some region in the phase space of a Hamiltonian system. These various definitions are related but not equivalent. In this work , particularly in Section 4.1, we define analogues of Gibbs and Boltzmann entropies for probability distributions.) In particular, we extend the notion of critical temperatures to the realm of stationary stochastic processes and find a necessary and sufficient condition for the existence of such criticalities. Loosely speaking, at low temperatures, the dynamics of a stochastic process are dominated by energy considerations and become nearly deterministic (i.e., the system is almost always in a ground state). At high temperatures, the dynamics are dominated by entropic considerations and become nearly uniform (i.e., the system traverses all states, regardless of energy). The former occurs for any stationary process, and the rate of approach is given by the energy. In contrast, the entropic effects typically dominate only at infinite temperatures, but some systems can reach uniformity at a finite temperature. We define such a temperature as critical.

Note that we are not presenting an alternative to existing statistical mechanical literature on criticality and phase transition. Instead, we are attempting to generalize these notions from statistical mechanics to a much broader context where the concept of criticality does not yet exist. One can certainly craft stationary distributions from an equilibrium statistical mechanics problem and apply our theory, but this will not produce results that differ from classical approaches.

The paper is organized as follows: In Section 2, we provide a brief historical review of the use of the negative logarithm of a stationary probability distribution as an energy potential. In Section 2.1, we first look at the history of using minus-log-probability to equilibrium chemical thermodynamics and briefly review Kirkwood’s fundamental idea of the potential of mean force and the notion of entropic force. In Section 2.2, we describe two recent results identifying the minus-log-probability as “energy”: a self-contained and consistent mesoscopic stoc-NET [20], and a precise agreement between its macroscopic limit and Gibbs’ theory [21,22]. These two results provide strong evidence for the validity of such an identification. In Section 2.3, we discuss the legitimacy and centrality of stationary distribution in the “entropy inequality” for a Markov process from a mathematical standpoint. In Section 3, we propose a definition of the “corresponding deterministic dynamics” of a stochastic process using power-scaling of probability densities. In Section 3, we show that the rate of convergence to this corresponding deterministic process coincides with the minus-log-probability definition of energy. With the justifications given in Section 2 and Section 3, we carry out a more detailed analysis of such a probability distribution in Section 4. In Section 4.1, terms analogous to Boltzmann’s and Gibbs’ entropy are defined, along with their corresponding microcanonical partition functions. We also discuss the relative merits of these definitions. In Section 4.2, we prove that the system has a critical temperature if and only if the Gibbs’ entropy of the system is asymptotic to the energy. In Section 4.3, we discuss several example distributions in order to emphasize some subtleties in the choice of distribution and to illustrate the connection between this theory and equilibrium statistical mechanics. Finally, in Section 5, the ideas from previous sections are related to some recent results on biological systems.

2. A Novel Law of Force: Potential of Entropic Force

In Boltzmann’s statistical mechanics, phenomenological thermodynamics is given a Newtonian mechanical basis. Based on the already well developed concepts of mechanical energy and its conservation, Boltzmann [23] derived the relation

$$p^{eq}(x)\propto e^{-U(x)/k_{B}T},$$

(1)

where $U(x)$ is the mechanical energy of a microstate x and $p^{eq}(x)$ is the probability of state x when the system is in thermal equilibrium—a concept which had also already been well established in thermodynamics via the notion of quasi-stationary processes. (It is important to distinguish between a mechanical microstate and a thermodynamic state. A thermodynamic state is a state of recurrent motion, defined by an entire level set $\mathcal{A}=\{x|U(x)=E\}$. Thus, Boltzmann [23] also introduced his celebrated entropy $S_B(E)=k_B\ln \Omega (E)$, where $S_B$ is the entropy and $\Omega (E)$ is the number of microstates consistent with a given energy E. That is, $\Omega (E)$ is the cardinality of $\mathcal{A}$. In terms of E, then $p^{eq}(E)\propto \Omega (E)e^{-E/k_{B}T}=e^{-[E-TS(E)]/k_{B}T}$.) In a thermodynamic equilibrium, there is no net transport of any kind. (In the thermodynamics before Gibbs, macroscopic transport processes were driven by either a temperature or a pressure gradient in the three-dimensional physical space. In Gibbs’ macroscopic chemical thermodynamics, a chemical equilibrium has no net flux in the abstract stoichiometric network. In the current mesoscopic, stochastic thermodynamics, an equilibrium has no net probability transport in an appropriate state space. The notion of detailed balance independently arose in physics [24,25], chemistry [26,27] and in probability theory [28].)

It is also worth noting that Boltzmann’s mathematical derivation matched the modern maximum entropy principle with the constraint of given mean value for energy, which yields an exponential law for the energy distribution. (The mathematical statements of energy conservation ${\sum }_{k=1}^N{E}_k=C$ and fixed mean energy $\frac{1}{N}{\sum }_{k=1}^N{E}_k=\overline{c}$ are equivalent when N is given.)

Inspired by Boltzmann’s law (1), generalizations of the concept of equilibrium thermodynamic potentials have been proposed in many studies. These generalizations go by a variety of names: generalized thermodynamic potential, kinetic potential, nonequilibrium potential, pseudo-potential, emergent landscape, etc. [15,16,17,18,19,29]. One of the common features of all these names is that the “potential function” is defined by applying Equation (1) in reverse. One defines a potential

$$H(x)=-\ln p^{eq}(x)$$

(2)

based on the stationary probability, which can be obtained in many statistical models and whose existence can be mathematically proven for a large class of systems. Most importantly, many systems with stationary probability have non-zero transport flux(es).

In fact, this tradition of taking (2) as a legitimate potential function started in equilibrium statistical chemical thermodynamics. Note that according to Equation (1), the term $-k_{B}T\ln p^{eq}(x)$ is simply the total mechanical energy of state x, which is known a priori. Therefore, there is no reason to define (2) in studies of a pure mechanical system. However, in statistical chemical thermodynamics, one usually does not have a full Hamiltonian function for a complex molecule at hand. It is at this juncture that the notion of a potential of mean force [30] enters the theory.

2.1. Equilibrium Potential of Mean Force

Physical chemists deal with complex molecules and force fields. Even though in molecular dynamics (MD) a molecule has a classical mechanical representation in terms of atoms as point masses, the precise potential energy is not known. The force fields in MD have therefore been under intense development over the past 50 years [31]. With such complexities, is it even possible to do statistical mechanics?

Let us first note a very important mathematical equality in connection to Equation (1). We consider a function $U(x)$ with $x=(x_1,x_2)$ where $x\in \mathcal{S}={\mathcal{S}}_1\oplus {\mathcal{S}}_2$, $x_1\in {\mathcal{S}}_1$ and $x_2\in {\mathcal{S}}_2$. Then,

$$\begin{array}{ll}Z(T)& ={\int }_{\mathcal{S}}{e}^{-U(x)/k_{B}T}\mathrm{d}x\\ & ={\int }_{{\mathcal{S}}_1}{\int }_{{\mathcal{S}}_2}{e}^{-U(x_1,x_2)/k_{B}T}\mathrm{d}{x}_1\mathrm{d}{x}_2\\ & ={\int }_{{\mathcal{S}}_1}{e}^{-\phi (x_1)/k_{B}T}\mathrm{d}{x}_1,\end{array}$$

(3)

$$\begin{array}{ccc}\hfill \phi (x_1)& =& -k_{B}T\ln {\int }_{{\mathcal{S}}_2}{e}^{-U(x_1,x_2)/k_{B}T}\mathrm{d}{x}_2.\hfill \end{array}$$

(4)

Notice that if we consider $\phi (x_1)$ as a “potential function” for the system in (coarse-grained) state $x_1$, then we can obtain the same $Z(T)$ using Equation (3), which is in the exact same form as in (1). More importantly, we see that $\phi (x_1)$ is the free energy with fluctuating $x_2$ and fixed $x_1$.

After reading the calculations above, one is naturally led to the question, “what does this potential energy function $\phi (x_1)$ defined in (4) represent?” J. G. Kirkwood answered this question in a very satisfying manner [30]: it is the potential function of a “mean force”, in equilibrium, acting on the system which is fixed at $x_1$:

$$\begin{array}{ll}-\frac{\mathrm{d}\phi (x_1)}{\mathrm{d}{x}_1}& =-\frac{{\displaystyle {\int }_{{\mathcal{S}}_2}{\left(\frac{\partial U(x_1,x_2)}{\partial x_1}\right)}_{x_2}{e}^{-U(x_1,x_2)/k_{B}T}\mathrm{d}{x}_2}}{{\displaystyle {\int }_{{\mathcal{S}}_2}{e}^{-U(x_1,x_2)/k_{B}T}\mathrm{d}{x}_2}}\\ & =-{\int }_{{\mathcal{S}}_2}{\left(\frac{\partial U(x_1,x_2)}{\partial x_1}\right)}_{x_2}{p}^{eq}\left(x_2|x_1\right)\mathrm{d}{x}_2,\end{array}$$

(5)

in which $p^{eq}(x_2|x_1)$ is the conditional equilibrium probability distribution for $x_2$ given $x_1$, and the partial derivative $-{(\partial U(x_1,x_2)/\partial x_1)}_{x_2}$ is precisely the mechanical force in the $x_1$ direction, with the given $x_2$. Averaging over the fluctuating $x_2$ with distribution $p^{eq}(x_2|x_1)$, Equation (5) is the mean force on $x_1$.

In other words, Equation (4) states that the negative logarithm of the marginal probability distribution for $x_1$ is simply the potential of mean force if one chooses the free energy of the entire system, $F(T)=-k_{B}T\ln Z(T)$, as the zero energy reference point.

$$-k_{B}T\ln {\int }_{{\mathcal{S}}_2}{p}^{eq}(x_1,x_2)\mathrm{d}{x}_2=\phi (x_1)-F(T).$$

(6)

One of the most important facts, as is clear from (4), is that the potential of mean force $\phi (x_1)$ is itself a function of temperature. In physical chemistry, one usually builds a statistical mechanical model using such a potential of mean force rather than using a mechanical energy function. That is, one uses a free energy function with certain degrees of freedom fixed and averaged over all the others.

Since $\phi (x_1)$ is temperature dependent, it has its own energy part and entropy part:

$$\phi (x_1;T)=\underset{\text{energy}}{\underbrace{\frac{\partial (\phi /T)}{\partial (1/T)}}}-T\underset{\text{entropy}}{\underbrace{\left(-\frac{\partial \phi }{\partial T}\right)}}.$$

(7)

A potential of mean force can be purely entropic. One of the best known examples is rubber elasticity, which arises from a Gaussian polymer chain [32]. If the temperature is suddenly dropped to zero, the force (and its associated energy) disappears instantly.

Observing this significant conceptual distance between chemical thermodynamics and its mechanical origin, and the essential statistical nature of Gibbs’ energy based on minus-log probability in all modeling practices, it is not surprising that some researchers who mainly work with biochemical thermodynamics strongly feel that one could reformulate statistical thermodynamics (at least in connection to energy) in terms of a “measure of information” and abandon the very term “entropy”, along with its root in mechanics [33].

2.2. Nonequilibrium Steady State Potential

For stochastic models of equilibrium systems, therefore, (2) yields a meaningful free energy function in $k_{B}T$ units. It embodies an exact coarse-graining procedure. For stochastic models of nonequilibrium steady state with non-zero transport flux, we now have sufficient evidence to suggest that

$$H(x)=-\ln p^{ss}(x),$$

(8)

where $p^{ss}$ is a stationary distribution, but may or may not be an equilibrium distribution, is also a meaningful energy function. We start with some conceptual discussions.

First, outside classical mechanics, the question “what is a force and how do we quantify it” is highly non-trival and vague. Onsager, however, introduced the notion of a thermodynamic force in his theory of irreversible processes [34]. Intuitively, a force is the cause of an action. In Newtonian mechanics, a force is the cause of a change in the vector $\frac{\mathrm{d}}{\mathrm{d}t}\overrightarrow{x}$. However, in an “overdamped world”, which emcompasses most of chemistry, biology, and society, a force is actually needed to cause a meaningful movement (i.e., a transport).

In terms of the mathematical theory of stochastic dynamics, there is a universal conception for movement or “dynamics”: Given the option to move to one of many states, a system is most likely to move to the state with the highest stationary probability. One should immediately note that this statement is highly problematic from a rigrous mathematical standpoint. Nevertheless, at least in one class of systems, the above notion is attainable: the class of systems whose dynamics have an invariant measure that is ergodic.

When discussing statistical mechanics, Montroll and Green have stated that [35] “The aim of statistical mechanics is to develop a formalism from which one can deduce the macroscopic behavior of physical systems composed of a large number of molecules from a specification of the component molecular species, the laws of force which govern intermolecular interactions, and the nature of their surroundings”. With the rise of equilibrium chemical thermodynamics, it is clear that the “laws of force” themselves can be discovered from the equilibrium distribution. In fact, most such laws of force in biophysical modeling are statistical in nature and can be seen as entropic forces.

Indeed, “[t]o date no one has succeeded in deriving the laws of nonequilibrium phenomena from the [Newtonian] equations of motion merely by allowing the number of particles involved to become infinite. However, considerable success has been achieved by introducing various statistical hypotheses” [35]. Recent studies have shown that if one identifies $H(x)$ as a “generalized Helmhotz or Gibbs energy function”, a complete and consistent mesoscopic thermodynamics can be formulated that includes nonequilibrium steady states [3,20]. Furthermore, if one passes the system from mesoscopic to macroscopic by allowing the number of particles involved and the system’s volume to become infinite, two macroscopic thermodynamic laws can be derived [21]. If the mesoscopic system is a general chemical reaction network with detailed balance, the macroscopic emergent potential was shown mathematically to be Gibbs’ function $G(x)$, where $x_i$ are the concentrations of chemical species and $\partial G/\partial x_i$ are the chemical potentials for the i-th species. The same theory also proves the existence of, and provides an equation for computing, a generalized Gibbs function for an open chemical reaction network under a chemostat, which approaches a nonequilibrium steady state.

2.3. Stationary Distribution and Entropy Inequalities of Markov Processes

Unless stated otherwise, we will exclusively deal with a denumerable state space $\mathcal{S}$ (either finite or infinite) for the remainder of the paper.

A stronger monotonicity result. The strongest version of a monotoic entropy result that we are aware of is [36,37]

$$\frac{\mathrm{d}}{\mathrm{d}t}D\left[\left\{p_x(t)\right\}\parallel \left\{q_x(t)\right\}\right]\equiv \frac{\mathrm{d}}{\mathrm{d}t}\sum _{x\in \mathcal{S}}{p}_x(t)\ln \left(\frac{p_x(t)}{q_x(t)}\right)\le 0,$$

(9)

in which $p_x(t)$ and $q_x(t)$ are two solutions to the Kolmogorov forward equation with different initial distributions. Equation (9) immediately yields a variety of related inequalities:

(i)

When $q_x(t)\equiv {\pi }_x$ $\forall t$, where $\left\{{\pi }_x\right\}$ is a stationary distribution of the Markov process, then (9) is the widely known “free energy theorem” [38,39].

(ii)

When $q_x(t)\equiv {\pi }_x$ $\forall t$, and $p_i(0)={\delta }_{i\ell }$, one has

$$\frac{\mathrm{d}}{\mathrm{d}t}\sum _{j\in \mathcal{S}}{p}_{\ell j}(t)\ln \left(\frac{p_{\ell j}(t)}{{\pi }_j}\right)\le 0\forall \ell ;$$

(10a)

therefore,

$$\frac{\mathrm{d}}{\mathrm{d}t}I\left[{\mathbf{x}}_t\parallel {\mathbf{x}}_0\right]=\frac{\mathrm{d}}{\mathrm{d}t}\sum _{\ell ,j\in \mathcal{S}}{\pi }_{\ell }{p}_{\ell j}(t)\ln \left(\frac{p_{\ell j}(t)}{{\pi }_{\ell }{\pi }_j}\right)\le 0,$$

(10b)

where $I[{\mathbf{x}}_t\parallel {\mathbf{x}}_0]$ is the mutual information between ${\mathbf{x}}_0$ and ${\mathbf{x}}_t$ of a stationary Markov process. Similarly,

$$\frac{\mathrm{d}}{\mathrm{d}t}\left(-\sum _{\ell ,j\in \mathcal{S}}{\pi }_{\ell }{p}_{\ell j}(t)\ln p_{\ell j}(t)\right)\ge 0.$$

(10c)

This result was in [40]. The term inside $(\cdots )$ is the conditional Shannon entropy $H[{\mathbf{x}}_t|{\mathbf{x}}_0]$ for the stationary ${\mathbf{x}}_t$. It is also the Kolmogorov–Sinai (KS) entropy of every t steps of the stationary ${\mathbf{x}}_t$:

$$\underset{n\to \infty }{\lim }\frac{1}{n}H\left[{\mathbf{x}}_0,{\mathbf{x}}_t,{\mathbf{x}}_{2t},\cdots {\mathbf{x}}_{nt}\right].$$

The result is more easily understood when interpreted this way: KS entropy quantifies the randomness in a “map”. The randomness does not decrease with map composition.

(iii)

When $p_x(t)\equiv {\pi }_x$ (and when we then rename $q_x(t)$ as $p_x(t)$), we have

$$\frac{\mathrm{d}}{\mathrm{d}t}\sum _{x\in \mathcal{S}}{\pi }_x\ln \frac{{\pi }_x}{p_x(t)}\le 0.$$

(11)

To explain this result more intuitively, we note that the sum in (11) can be interpreted as the information lost when predicting ${\pi }_x$ from $p_x(t)$. Roughly speaking, if $t_1<t_2$, then it takes more information to predict the distant future (${\pi }_x$) from time $t_1$ than it does from time $t_2$ because the prediction from $p_x(t_1)$ has to account for the random events that can happen within the time interval $[t_1,t_2]$.

Filtration and entropy monotonicity. Even though the original Shannon entropy used an implicit uniform prior, the necessity for an explicit prior has been widely discussed in information theory [41,42]. (The entropy with respect to an explicit prior is more accurately called the relative entropy or cross-entropy, and its expression is analagous to the free energy in statistical mechanics.) More importantly, for a continuous random variable, the logarithm of a probability density is simply ill-defined mathematically. All the various monotoic “entropy” results in the previous section provide the legitimacy of using $\left\{{\pi }_x\right\}$ as the reference measure for a Markov process. We would like to argue that this is, in fact, necessary.

We consider a Markov process in a more general setting in this section. Let the triple $(\mathcal{S},\mathcal{F},P)$ be a probability space; let $(\mathcal{I},\le )$ be a totally ordered index set; and let $(S,\Sigma )$ be a measurable space. If $X:\mathcal{I}\times \mathcal{S}\to S$ is a stochastic process, then its natural filtration of $\mathcal{F}$ with respect to X is a sequence $\left\{{\mathcal{F}}_i^{(X)}|i\in \mathcal{I}\right\}$ such that

$${\mathcal{F}}_i^{(X)}=\sigma \left\{X_j^{-1}(A)|j\in \mathcal{I},j\le i,A\in \Sigma \right\}.$$

(12)

That is, ${\mathcal{F}}_i^{(X)}$ is the smallest σ-algebra on $\mathcal{S}$ that contains all pre-images of Σ-measurable subsets of S for times j up to i. The definition given in (12) yields a monotonic relation

$${\mathcal{F}}_j^{(X)}\subseteq {\mathcal{F}}_i^{(X)}\mathrm{if}i,j\in \mathcal{I},j\le i.$$

(13)

Such a property is called non-anticipating; in other words, “when including the future, the dynamics are at least as random as up to now”.

The monotonicity in Equation (13) can be expressed in terms of Shannon’s information entropy as

$$H[X_0,X_1,\cdots ,X_i]\le H[X_0,X_1,\cdots ,X_i,X_{i+1}].$$

(14)

This inequality is true because $H[X_0,\cdots ,X_{i+1}]-H[X_0,\cdots ,X_i]$ is the conditional Shannon entropy $H[X_{i+1}|X_1,\cdots ,X_i]$, which is never negative.

Notice that Equations (13) and (14) are concerned with the sequences of $\left\{X_j|j\le i\right\}$, but the “entropy monotonicity” results in statistical physics deal with individual$X_i$ and $X_{i+1}$; and entropy has deterministic values that are different for different times. The relationship among $X_i$, $X_{i+1}$, and the filtration is shown as

$$\begin{array}{cccc}(\mathcal{S},\mathcal{F})& \underset{\phantom{\text{steppingt}}}{\overset{X_i}{\to }}& (S,\Sigma )& \underset{\phantom{\text{steppingt}}}{\overset{X_i^{-1}}{\to }}\left(\mathcal{S},F_i^{(X)}\right)\\ \parallel & & \parallel & \text{time}\text{stepping}\downarrow \\ (\mathcal{S},\mathcal{F})& \stackrel{X_{i+1}}{\to }& (S,\Sigma )& \underset{\phantom{\text{steppingt}}}{\overset{X_{i+1}^{-1}}{\to }}\left(\mathcal{S},F_{i+1}^{(X)}\right)\end{array}.$$

(15)

We now consider the information lost from $X_i$ to $X_{i+1}$ when the event ω occurs, i.e., $\ln P_{X_{i+1}}(\omega )-\ln P_{X_i}(\omega )$. Its expected value with respect to the stationary, invariant measure ${\mu }_{\pi }(\omega )$ is given by

$$\begin{array}{cc}\hfill \mathbb{E}\left[\ln P_{X_{i+1}}-\ln P_{X_i}\right]& ={\int }_{\Omega }\ln \left(\frac{\mathrm{d}{P}_{X_{i+1}}}{\mathrm{d}{P}_{X_i}}(\omega )\right)\mathrm{d}{\mu }_{\pi }(\omega )\hfill \\ & ={\int }_{\Omega }\ln \left(\frac{\mathrm{d}{P}_{X_{i+1}}}{\mathrm{d}{\mu }_{\pi }}(\omega )\right)\mathrm{d}{\mu }_{\pi }(\omega )-{\int }_{\Omega }\ln \left(\frac{\mathrm{d}{P}_{X_i}}{\mathrm{d}{\mu }_{\pi }}(\omega )\right)\mathrm{d}{\mu }_{\pi }(\omega ).\hfill \end{array}$$

(16)

If both $X_i$ and $X_{i+1}$ are real valued (i.e., $S=\mathbb{R}$) with density functions $f_{X_i}(x)$ and $f_{X_{i+1}}(x)$ respectively, then (16) becomes

$$\mathbb{E}\left[\ln P_{X_{i+1}}-\ln P_{X_i}\right]={\int }_{\mathbb{R}}\ln \left(\frac{f_{X_{i+1}}(x)}{\pi (x)}\right)\pi (x)\mathrm{d}x-{\int }_{\mathbb{R}}\ln \left(\frac{f_{X_i}(x)}{\pi (x)}\right)\pi (x)\mathrm{d}x,$$

(17)

where $\pi (x)=\mathrm{d}{\mu }_{\pi }/\mathrm{d}x$ is the density of the stationary measure. We know that Equation (17) is never negative; therefore, the mean information lost

$${\int }_{\Omega }\ln \left(\frac{\mathrm{d}{P}_{X_{i+1}}}{\mathrm{d}{P}_{X_i}}(\omega )\right)\mathrm{d}{\mu }_{\pi }(\omega )\ge 0,$$

(18)

or equivalently,

$$H\left[X^{ss}\parallel X_i\right]\ge H\left[X^{ss}\parallel X_{i+1}\right]\ge 0,$$

(19)

where $X^{ss}:\mathcal{S}\to S$ is a random variable distributed according to the stationary distribution π. This is essentially equivalent to the result in Equation (11).

Equation (18) states that information lost from $X_i$ to $X_{i+1}$, averaged with respect to the invariant density, is always greater than zero, while Equation (19) suggests that “the infinitely distant future has more information to gain from $X_i$ than from $X_{i+1}$”. There is a subtle difference between these statements and the following: “when including the future, the world is at least as random as up to now”. The reason for this, we suggest, is that (18) and (19) require the existence of the stationary measure. Knowing the existence of a stationary behavior, “the future is at least as random as now”.

3. Deterministic Correspondence and Infinite $\mathbf{\beta }$

Any representation of reality requires elements of both chance and determinism. These correspond to the stochastic and deterministic components of complex dynamics. As repeatedly pointed out in [43,44,45], it is the interaction between these two that yields self-orgranization and complex behavior. Therefore, the ability to “envision” a corresponding deterministic dynamics to some given stochastic dynamics, even when there is no obvious “system size parameter”, provides a deeper understanding of complex dynamics. The natural parameter for a stochastic differential equation (SDE) $\mathrm{d}\mathbf{x}(t)=b(\mathbf{x})\mathrm{d}t+a\mathrm{d}B(t)$ is the noise strength a; the natural parameter in classical statistical mechanics is the system’s size (or one could use the temperature); and the natural parameter in a Delbrück–Gillespie process is the system’s volume.

How can one envision such a deterministic correspondence when no obvious natural parameters exist? It is becoming increasingly common to use the modal value of a distribution as a “deterministic” counter part to the stochastic system. According to this view, a bimodal distribution corresponds to a bistable system. Note it is a widely held misconception that the mean dynamics $\langle \mathbf{x}(t)\rangle$ are the deterministic counterpart of a stochastic $\mathbf{x}(t)$. For an SDE, $\langle \mathrm{d}\mathbf{x}(t)\rangle \ne b(\langle \mathbf{x}\rangle )$ in general. More importantly, while $\langle \mathbf{x}(t)\rangle$ is a non-random function of t, it is not a trajectory of any meaningful, self-contained dynamical system. This point is best illustrated by the fact that the differential equation describing $\langle \mathbf{x}(t)\rangle$ usually depends on higher moments like $\langle {\mathbf{x}}^2(t)\rangle$. Moreover, for a discrete system, even if the mean is defined, it does not usually lie in the same space as $\mathbf{x}(t)$.

We propose the following “deterministic” counterpart for a random variable $\mathbf{x}$ with probability mass function $p_{\mathbf{x}}^{ss}$, and we will show that it is intimately related to the energy defined in (8). We will define the “deterministic” variable ${\mathbf{x}}_{\infty }$ as

$${\mathbf{x}}_{\infty }=\underset{\beta \to \infty }{\lim }{\mathbf{x}}_{\beta },$$

(20)

where

$$p_{{\mathbf{x}}_{\beta }}^{ss}(x)=\frac{p_{\mathbf{x}}^{ss}{(x)}^{\beta }}{Z(\beta )},$$

(21)

with normalization constant

$$Z(\beta )=\sum _x{p}_{\mathbf{x}}^{ss}{(x)}^{\beta }.$$

The random variable ${\mathbf{x}}_{\infty }$ will be concentrated on a finite number of states (the most probable ones of $p_{\mathbf{x}}(x)$) with a probability of 1. In particular, if $p_{\mathbf{x}}^{ss}(x)$ is unimodal, then ${\mathbf{x}}_{\infty }$ really will be a deterministic system. On the other hand, if $p_{\mathbf{x}}^{ss}(x)$ is multimodal, then there is no unique deterministic counterpart. Applying this idea to a discrete-state Markov process, the corresponding dynamics become a deterministic transformation as discussed in [46].

It is worth noting that similar definitions are often introduced formally as analogues to inverse-temperature without any discussion of deterministic correspondence, e.g., [12,13]. We spend so much time on the concept in order to emphasize that it arises naturally in a study of stochastic systems without any reference to thermodynamic concepts. The scaling factor β should not just be thought of as a formal method for introducing temperature to a system, but as a natural feature of any probabilistic system.

With this definition in hand, the obvious question becomes “how fast does the limit in (20) converge?” In the next section, we will try to make this question more rigorous. In the process, we will provide more evidence that $H(x)$ is an important quantity.

Large Deviation Principle for Infinite β

We will now investigate the rate of convergence of the limit in (20). This is a question well suited to the methods of large deviation theory. However, before we can use such methods, we need to frame the question somewhat more rigorously. Strictly speaking, we should be dealing with limits of measures rather than limits of random variables.

Let $\left(\mathcal{S},\mathcal{F},P\right)$ be a discrete probability space with probability mass function $p^{ss}$ and define the family of measures $P^{\beta }$ on $\left(\mathcal{S},\mathcal{F}\right)$ whose probability mass functions are given by

$$\begin{array}{c}p(x,\beta )=\frac{p^{ss}{(x)}^{\beta }}{Z(\beta )},\mathrm{where}\hfill \\ Z(\beta )=\sum _{x\in \mathcal{S}}{p}^{ss}{(x)}^{\beta }.\hfill \end{array}$$

(22)

As we will show later, this is always possible for $\beta \ge 1$. In addition, let $\left(S,\Sigma \right)$ be a measurable space and choose a function $\sigma :\mathcal{S}\to S$. This defines a family of S-valued random variables ${\mathcal{O}}_{\beta }$ where

$$\mathrm{Pr}\left\{{\mathcal{O}}_{\beta }=z\right\}=P^{\beta }\left(\left\{x\in \mathcal{S}|\sigma (x)=z\right\}\right),$$

(23)

where $\sigma :\mathcal{S}\to S$ is a measurable map. In particular, if σ is one-to-one, then $\mathrm{Pr}\left\{{\mathcal{O}}_{\beta }=z\right\}=p({\sigma }^{-1}(z),\beta )$. The random variables ${\mathcal{O}}_{\beta }$ are observables on the measurable space $\left(\mathcal{S},\mathcal{F}\right)$. In some cases, the distinction between such measurable processes and the underlying measure space becomes vitally important [3]. As we will see here, though, the rate of convergence of these measures is the same whether we phrase the question in terms of observables or the original measure space.

For unimodal distributions, we know that, as β goes to infinity, the distribution of ${\mathcal{O}}_{\beta }$ becomes concentrated on a single value $z^{*}\in S$. However, it is not clear a priori how the rate of this convergence depends on our choice of $\mathcal{O}$. It is conceivable that different observables could lead to different convergence rates. Moreover, we could eschew observables altogether and work solely with the measures $P^{\beta }$. In this section, we will show that the rate of convergence is identical for a wide range of observables and that it is intimately related to $H(x)$.

Case (i): Let $S=\mathbb{R}$. We will not restrict σ to be one-to-one, but we will assume that if $\sigma (x_1)=\sigma (x_2)$ for some $x_1,x_2\in \mathcal{S}$, then $p^{ss}(x_1)=p^{ss}(x_2)$. We will let $N(x)$ denote the (necessarily finite) number of elements $y\in \mathcal{S}$ such $\sigma (y)=\sigma (x)$. Finally, let $x^{*}\in \mathcal{S}$ be a state with maximal probability. We know that

$$\underset{\beta \to \infty }{\lim }\mathrm{Pr}\left\{\left|{\mathcal{O}}_{\beta }-\sigma (x^{*})\right|\ge \eta \right\}=0$$

(24)

for any $\eta \in {\mathbb{R}}^{+}$. In fact, $\mathrm{Pr}\left\{\left|{\mathcal{O}}_{\beta }-\sigma (x^{*})\right|\ge \eta \right\}$ is a non-increasing step function of η. Under reasonable conditions, we can write

$$\mathrm{Pr}\left\{\left|{\mathcal{O}}_{\beta }-\sigma (x^{*})\right|\ge \eta \right\}=e^{-\beta I_1(\eta )+o(\beta )},$$

(25)

where

$$I_1(\eta )=-\underset{\beta \to \infty }{\lim }\ln \mathrm{Pr}\left\{\left|{\mathcal{O}}_{\beta }-\sigma (x^{*})\right|\right\}.$$

(26)

If we define ${\widehat{x}}_{\eta }={\mathrm{argmax}}_{x\in \mathcal{S}}\left\{\left|\sigma (x)-\sigma (x^{*})\right|\right\}$, then we have

$$\begin{array}{cc}\hfill I_1(\eta )& =-\underset{\beta \to \infty }{\lim }\frac{1}{\beta }\ln \left(\frac{1}{Z(\beta )}\sum _{x:\left|\sigma (x)-\sigma (x^{*})\right|\ge \eta }{p}^{ss}{(x)}^{\beta }\right)\hfill \\ & =-\underset{\beta \to \infty }{\lim }\frac{1}{\beta }\ln \left(\frac{N({\widehat{x}}_{\eta })p({\widehat{x}}_{\eta },\beta )}{N(x^{*})p(x^{*},\beta )}\right)\hfill \\ & =-\ln \left(\frac{p({\widehat{x}}_{\eta },1)}{p(x^{*},1)}\right)\hfill \\ & =H({\widehat{x}}_{\eta })-H(x^{*}).\hfill \end{array}$$

Case (ii): Instead of creating a somewhat arbitrary family of observables ${\mathcal{O}}_{\beta }$, we can also work solely with the measures $P^{\beta }$. To make this more convenient, we will introduce some additional notation.

Let $\mathcal{Y}=H(\mathcal{S})\subset \mathbb{R}$ and let $y^{*}$ be the minimum value in $\mathcal{Y}$. For any $h>y^{*}$, let ${\mathcal{S}}_h=\left\{x\in \mathcal{S}|H(x)<h\right\}$ and ${\mathcal{Y}}_h=\left\{y\in \mathcal{Y}|y<h\right\}$. Let $⌊h⌋$ denote the minimum value of $\mathcal{Y}\backslash {\mathcal{Y}}_h$. Finally, define

$$Z_h(\beta )=\sum _{x\in {\mathcal{S}}_h}{p}^{ss}{(x)}^{\beta }.$$

(27)

We know that $P^{\beta }\left(\mathcal{S}\backslash {\mathcal{S}}_h\right)$ approaches zero as β goes to infinity. Much like the previous case, we would like to know how quickly this quantity decays. We have

$$P^{\beta }\left(\mathcal{S}\backslash {\mathcal{S}}_h\right)=e^{-\beta I_2(h)+o(\beta )},$$

(28)

where

$$\begin{array}{cc}\hfill I_2(h)& =-\underset{\beta \to \infty }{\lim }\frac{1}{\beta }\ln P^{\beta }\left(\mathcal{S}\backslash {\mathcal{S}}_h\right)\hfill \\ & =-\underset{\beta \to \infty }{\lim }\frac{1}{\beta }\ln \left(\frac{1-Z_h(\beta )}{Z(\beta )}\right)\hfill \\ & =-\underset{\beta \to \infty }{\lim }\frac{1}{\beta }\ln \left(\frac{{\sum }_{x:-\ln p^{ss}(x)=\lfloor h\rfloor }{p}^{ss}{(x)}^{\beta }}{{\sum }_{x:-\ln p^{ss}(x)=y^{*}}{p}^{ss}{(x)}^{\beta }}\right)\hfill \\ & =\lfloor h\rfloor -y^{*}.\hfill \end{array}$$

In fact, this is in some sense just a special case of case (i). If we choose $\sigma =H$ and let $h=\eta +y^{*}$, then $I_1$ and $I_2$ are identical.

Case (iii): One of the key insights from the theory of large deviations is that in the limit as $\beta \to \infty$ the probability $\mathrm{Pr}\left\{{\mathbf{x}}_{\beta }\notin {\mathcal{S}}_h\right\}$ is determined by one particular $x^{*}\notin {\mathcal{S}}_h$ – the one with $p(x^{*},1)\ge p(x,1)$ for all $x\notin {\mathcal{S}}_h$. Therefore, one has ${\lim }_{\beta \to \infty }p(x,\beta )\approx e^{-\beta I_3(x)}$ for any $z\in \mathcal{S}$. This is essentially the same as the WKB (Wentzel–Kramers–Brillouin) ansatz. We then have

$$\begin{array}{ccc}\hfill I_3(x)& =& -\underset{\beta \to \infty }{\lim }\frac{1}{\beta }\ln p(x,\beta )\hfill \\ & =& -\ln p(x,1)+\underset{\beta \to \infty }{\lim }\frac{1}{\beta }\ln \sum _{x\in \mathcal{S}}{p}^{\beta }(x,1)\hfill \\ & =& -\ln \left(\frac{p(x,1)}{p(x^{*},1)}\right)+\underset{\beta \to \infty }{\lim }\frac{1}{\beta }\ln \left[1+\sum _{x\in \mathcal{S},x\ne x^{*}}{\left(\frac{p(x,1)}{p(x^{*},1)}\right)}^{\beta }\right]\hfill \\ & =& -\ln \left(\frac{p(x,1)}{p(x^{*},1)}\right)\hfill \\ & =& H(x)-H(x^{*}).\hfill \end{array}$$

Whether we work with observables on top of $\left(\mathcal{S},\mathcal{F}\right)$ or if we work with the underlying measure space itself or even if we make substantial approximations as with the WKB ansatz, we always obtain essentially the same rate of convergence. The energy given in (8) describes the rate of convergence of $p^{ss}$ to its corresponding deterministic system.

4. Entropy, Energy and Criticality in Systems with Generalized Potential

The results of the previous section suggest that $H(x)=-\ln p^{ss}(x)$ is a mathematically relevant quantity and that it can reasonably be interpreted as an energy. We will now investigate some of the consequences of this definition in more detail. In particular, we will shed some light on the distinction between Gibbs and Boltzmann entropies and derive a necessary and sufficient condition for the existence of a critical temperature in stationary stochastic systems.

Let us again suppose that our system takes on possible states from a discrete (finite or countably infinite) set $\mathcal{S}$, and let $p^{ss}:\mathcal{S}\to [0,1]$ be the probability mass function describing the chance that event $x\in \mathcal{S}$ ocurrs. As above, we will define the energy of a state $x\in \mathcal{S}$ as

$$H(x)=-\ln p^{ss}(x).$$

(29)

In addition, we will avoid substantial difficulties later if we endow H with units of energy. If we do so, then we can no longer simply write $p^{ss}(x)=e^{-H(x)}$. Instead, we need to introduce another parameter β with units of inverse energy. This gives us

$$p^{ss}(x;\beta )=\frac{1}{Z(\beta )}{e}^{-\beta H(x)},$$

(30)

where the partition function $Z(\beta )$ is defined as

$$Z(\beta )=\sum _{x\in \mathcal{S}}{e}^{-\beta H(x)}.$$

(31)

Note that the partition function is necessarily a dimensionless quantity, as discussed in [47,48,49]. These distributions are precisely the probability mass functions of the measures $P^{\beta }$ defined in Section 3.

With this definition, there is a serious concern that the sum in (31) might not converge. Since $p^{ss}$ is a probability distribution, however, we do know that the sum converges for $\beta =1$ (in fact, we know that $Z(1)=1$.) We will spend much of the following sections discussing the cases where the sum in (31) diverges, but for the moment we will simply assume that $Z(\beta )$ is well-defined on some subset of $\mathbb{R}$ containing $[1,\infty )$.

In classical statistical mechanics, one typically has the mechanical energy function in hand before $p^{ss}$ and then shows that the system at finite “temperature” ${\beta }^{-1}$ has an equilibrium distribution among the states described by (30). Note that when $\beta \to \infty$, the distribution $p^{ss}(x;\beta )$ converges to a uniform probability distribution on the set of states with minimal H. For certain non-convex $H(x)$, the phenomenon of phase transition occurs [50]. This limit gives precisely the deterministic correspondence described in Section 3.

In a classical statistical mechanical problem, $\mathcal{S}$ is a continuous space describing the positions and momenta of all particles in the system, H is a Hamiltonian for this system and $\beta ={(k_{B}T)}^{-1}$ is the inverse temperature. One would then be interested in level sets with constant energy h. In particular, Gibbs’ and Boltzmann’s entropies are concerned with the phase volume and phase surface area of such level sets.

Unlike in a classical problem, though, our state space $\mathcal{S}$ is arbitrary, and, in general, may not be useful as a phase space. In particular, $\mathcal{S}$ often does not come equipped with a metric or even any sort of order. To remedy this, we will define the rank of a state x as

$$\mathfrak{R}(x)=\#|\left\{y\in \mathcal{S}|H(x)\ge H(y)\right\}|,$$

(32)

where $\#\left|·\right|$ denotes cardinality. That is, the rank of x is the number of states which have lower energy than x (or are at least as probable as x). Since $\mathfrak{R}$ depends on x only through $p^{ss}(x)$, we can unambiguously define the rank in terms of energy as $\mathcal{V}:[0,\infty )\to {\mathbb{Z}}^{+}$ as

$$\mathcal{V}(h)=\#|\left\{x\in \mathcal{S}|H(x)\le h\right\}|,$$

(33)

so that $\mathfrak{R}(x)=\mathcal{V}(H(x))$ for every $x\in \mathcal{S}$.

Notice that $\mathcal{V}$, as opposed to $\mathfrak{R}$, is no longer defined on a discrete space – it is a function of the continuous variable h. However, because $\mathcal{S}$ is discrete, $\mathcal{V}$ can be written as a non-decreasing piecewise constant function.

It is also worth noting that our assumption of a countable state space cannot be easily relaxed in this approach. If $\mathcal{S}$ were uncountable, then one could not hope to order the states by their rank. Indeed, $\mathfrak{R}$ and $\mathcal{V}$ would generally be infinite for almost all input. Such issues arise because $p^{ss}$ is, by assumption, a probability density with respect to the counting measure. We could have instead assumed that $p^{ss}$ was a density with respect to some other measure (e.g., the Lebesgue measure on $\mathcal{S}=\mathbb{R}$), but this would introduce many other subtleties later on.

4.1. Microcanonical Partition Functions and Entropy

If we take the liberty of treating the derivative of a Heaviside function as a Dirac$-\delta$ function, then we can write $\mathcal{V}$ as

$$\mathcal{V}(h)={\int }_0^h\mathrm{d}\mathcal{V}(y)={\int }_0^h\frac{{\partial }^{}\mathcal{V}}{\partial y^{}}\mathrm{d}y.$$

(34)

It is very important to note that $\partial \mathcal{V}(h)/\partial h$ has units of inverse energy. It is tempting (and often quite useful) to define

$$\Omega (h)=\#|\left\{x\in \mathcal{S}|H(x)=h\right\}|,$$

(35)

and then write

$$\mathcal{V}(h)=\sum _{n=0}^{\infty }\Omega (h_n),$$

(36)

where the sum is taken over the values $h_n\le h$ such that $\Omega (h_n)>0$. (For any finite h, note that $\mathcal{V}(h)\le e^h$ because the distribution $p^{ss}$ sums to 1. The number of distinct values $h_n\le h$ is no greater than $\mathcal{V}(h)$, so it too is finite.) However, one should keep in mind that $\mathrm{d}\mathcal{V}/\mathrm{d}h\ne \Omega (h)$. That is, $\mathrm{d}\mathcal{V}/\mathrm{d}h$ is not really just a number of states; it is a density. (This is a common point of confusion in probability theory as well. The probability of an event A should always be written as ${\int }_A\mathrm{d}F={\int }_A(\mathrm{d}F/\mathrm{d}x)\mathrm{d}x={\int }_{A}f(x)\mathrm{d}x$, where F is the cumulative probability measure and $f=\mathrm{d}F/\mathrm{d}x$ is a density with respect to some other measure. When the other measure is a counting measure, however, it is commonplace to replace the integral with a sum and use the probability mass $p(x)=(\mathrm{d}F/\mathrm{d}x)\mathrm{d}x$ instead. This is numerically correct but often leads to confusion over units.)

One of the main reasons we have introduced this notation with $\mathcal{V}$ is that it gives us a much more convenient way to write $Z(\beta )$. In particular, we can write Z without reference to the individual states x:

$$Z(\beta )=\sum _{x\in \mathcal{S}}{e}^{-\beta H(x)}={\int }_0^{\infty }{e}^{-\beta h}\mathrm{d}\mathcal{V}(h).$$

(37)

This is exactly the Laplace–Stieltjes transform of $\mathcal{V}$.

It is tempting to rewrite Z as

$$Z(\beta )={\int }_0^{\infty }{e}^{-\beta h}\left(\frac{{\partial }^{}\mathcal{V}}{\partial h^{}}\right)\mathrm{d}h={\int }_0^{\infty }{e}^{-\beta \left(h-{(k_B\beta )}^{-1}{k}_B\ln \Omega (h)\right)}\mathrm{d}h,$$

(38)

and to then identify $\partial \mathcal{V}/\partial h$ as the microcanonical partition function and $k_B\ln \Omega (h)$ as the entropy. Unfortunately, this is entirely wrong. Equation (38) relies on the identification of $\frac{\partial \mathcal{V}}{\partial h}$ with $\Omega (h)$, which is invalid. This method can be salvaged by introducing a factor $\Delta h$ with units of energy so that the (38) becomes

$$Z(\beta )={\int }_0^{\infty }\frac{1}{\Delta h}{e}^{-\beta h}\left(\Delta h\frac{{\partial }^{}\mathcal{V}}{\partial h^{}}\right)\mathrm{d}h,$$

(39)

and the entropy becomes

$$S_B=k_B\ln \left(\Delta h\frac{{\partial }^{}\mathcal{V}}{\partial h^{}}\right).$$

(40)

In fact, if we choose $\Delta h$ as a constant, then this is exactly the Boltzmann entropy. Such a solution is somewhat unsatisfying; the introduction of arbitrary constants to correct units generally suggests a deeper misunderstanding. Worse yet, there is no real reason for $\Delta h$ to be constant so long as it has the correct units.

A much more satisfying interpretation of Z arises if we integrate by parts, obtaining

$$Z(\beta )=\beta {\int }_0^{\infty }{e}^{-\beta h}\mathcal{V}(h)\mathrm{d}h=\beta {\int }_0^{\infty }{e}^{-\beta \left(h-{(k_B\beta )}^{-1}{k}_B\ln \mathcal{V}(h)\right)}\mathrm{d}h.$$

(41)

Here, we can interpret $\mathcal{V}(h)$ as the microcanonical partition function and

$$S_G=k_B\ln \mathcal{V}(h)$$

(42)

as the entropy. We have chosen the subscripts G and B to emphasize that $S_B$ corresponds to Boltzmann entropy while $S_G$ corresponds to Gibbs entropy.

There has been much debate over the relative merit of these definitions of entropy in statistical mechanics, e.g., [51,52,53,54]. While we do not claim to have resolved this question, Equations (38) and (41) suggest that Gibbs entropy is the more natural choice. Furthermore, as we will see in the next section, Gibbs entropy plays a central role in the notion of criticality.

It is worth noting that the terminology surrounding Boltzmann and Gibbs entropy is not entirely consistent. Most notably, some authors, e.g., [55,56], use the phrase “Boltzmann entropy” to refer to the logarithm of the volume of any phase space region corresponding to a suitable macrostate and use “Gibbs entropy” to refer to the quantity $\int p\ln p\mathrm{d}x$, where p is some probability density. Using this terminology, Equations (40) and (42) would both be Boltzmann entropies but would use different macrostates.

Instead, we follow the convention used in, e.g., [51,52,53,54] and use “Boltzmann entropy” to indicate the logarithm of the volume of a thin shell in phase space and “Gibbs entropy” to indicate the logarithm of the volume of the interior of such a shell. If the quantity $\int p\ln p\mathrm{d}x$ is needed, we will refer to it as Shannon entropy.

4.2. Analyticity of Z as a Function of β

The analyticity of $Z(\beta )$, which is analogous to the partition function in statistical mechanics, is intimately related to phase transitions and critical phenomena [57,58,59,60]. Our system has a critical temperature (in the statistical mechanical sense of the term) if and only if the partition function is non-analytic for some $\beta \in (0,\infty )$. Since $Z(\beta )$ is a Laplace transform, we have access to some useful theorems from classical analysis (all of which can be found in [61]).

First, there is some value ${\beta }_c\in [-\infty ,\infty ]$ such that $Z(\beta )$ converges for all $\beta \in \mathbb{C}$ with real part greater than ${\beta }_c$ and diverges for all $\beta \in \mathbb{C}$ with real part less than ${\beta }_c$. The value ${\beta }_c$ is called the abscissa of convergence.

Second, if the state space $\mathcal{S}$ is finite then Z is a sum of finitely many terms and therefore converges for any β (i.e., ${\beta }_c=-\infty$). However, if $\mathcal{S}$ is infinite, then the partition function will not be analytic for all real β. In particular, it cannot converge when $\beta =0$ because $Z(0)=\#\left|\mathcal{S}\right|$. However, by definition, we know that $Z(\beta )$ converges when $\beta =1$ since $Z(1)$ is the normalization constant of $p^{ss}$. For infinite systems, the abscissa of convergence must therefore lie somewhere in $[0,1]$.

Since the abscissa of convergence is non-negative, we have

$${\beta }_c=\underset{h\to \infty }{\mathrm{lim\; sup}}\frac{\ln \mathcal{V}(h)}{h},$$

(43)

or

$$k_B{\beta }_c=\underset{h\to \infty }{\mathrm{lim\; sup}}\frac{S_G(h)}{h}.$$

(44)

We now know that the partition function is analytic for all complex β with real part greater than ${\beta }_c$, where ${\beta }_c$ is found as in (44). However, we have not yet shown that $Z(\beta )$ cannot be extended analytically beyond $\beta ={\beta }_c$. For a general Laplace–Stieltjes transform, this might be possible (in the worst case, a Laplace transform may have a finite abscissa of convergence but still have an analytic continuation to the entire complex plane). Fortunately, since $\mathcal{V}$ is monotonic, $Z(\beta )$ has a singularity at ${\beta }_c$ (this also means that ${\beta }_c\ne 1$).

This means that the partition function $Z(\beta )$ has a singularity at some positive ${\beta }_c$ if and only if $S_G$ is asymptotic to h in the sense of (44). That is, if the Gibbs entropy of the system grows sufficiently fast as a function of energy, it can become dominant in the computation of Z (and therefore $p^{ss}$) at a finite temperature.

4.3. Examples

So far, we have let our system be very general. The arguments above apply equally well to a wide range of systems—from the single electron of a hydrogen atom (where $\mathcal{S}$ is the set of possible orbits) to the configuration of amino acids in a strand of DNA. It is not immediately clear how (44) might be influenced by the structures of $\mathcal{S}$ and $p^{ss}$. To illustrate the consequences of our result, we will look at a few examples.

First, we will investigate two so-called “non-degenerate” cases where each state has a distinct probability (i.e., $\Omega \equiv 1$). Since we only care about the rank of states, we will suffer no loss of generality by assuming that $\mathcal{S}={\mathbb{Z}}^{+}$ and that the states are ordered so that $p^{ss}(x)>p^{ss}(y)$ whenever $x<y$. As an example, consider the distribution:

$$p^{ss}(x)=2^{-x}.$$

(45)

We have

$$\begin{array}{c}H(x)=x\ln 2,\hfill \\ S_G(h)=k_B\ln h-k_B\ln \ln 2,\mathrm{and}\hfill \\ {\beta }_c=0.\hfill \end{array}$$

(46)

This distribution, therefore, does not have a nonzero critical temperature (which should not be surprising, since it is exponential).

Alternatively, consider a power law distribution.

$$p^{ss}(x)=\frac{x^{-\alpha }}{\zeta (\alpha )},$$

(47)

where $\alpha >1$ and ζ is the Riemann zeta function. This gives us

$$\begin{array}{c}H(x)=\alpha \ln x+\ln \zeta (\alpha ),\hfill \\ S_G(h)=\frac{k_B}{\alpha }\left(h-\ln \zeta (\alpha )\right)\mathrm{and}\hfill \\ {\beta }_c=\frac{1}{\alpha }.\hfill \end{array}$$

(48)

This means that power law distributions do indeed have a finite critical temperature. This result was already demonstrated in [12] but arises as a special case of our work.

These examples highlight the main feature of criticality: a system will be critical if and only if the probability of a state decays too slowly as a function of rank. That is, critical distributions are fat-tailed in “phase space”.

We observe a similar result when Ω is not identically 1 (“degenerate” distributions). For example, consider a distribution where, for each $n\in {\mathbb{Z}}^{+}$, there are $2^n$ states with stationary probability $2^{-2n}$. That is, for each $h_n=2n\ln 2$, we have $\Omega (h_n)=2^n$. In this case,

$$\mathcal{V}(h)=2\left(2^n-1\right) \mathrm{for} h_n\le h<h_{n+1},$$

(49)

and we find that ${\beta }_c=1/2$. In light of our previous examples, this should not be surprising: when written as a function of rank, $p^{ss}$ decays like $x^{-2}$ so this ${\beta }_c$ is exactly what we expect. However, it also illustrates the importance of how we label our state space.

Suppose that we observed the system given above but that we could not identify each individual state. If instead of observing $2^n$ distinct states, each with probability $2^{-2n}$, we only measured 1 state with probability $2^{-n}$, we would then calculate the probability distribution $p^{ss}(x)=2^{-x}$, for which ${\beta }_c=0$. Depending on how states are counted, the distribution could either have a finite critical temperature or not! This distinction is exactly why the partition functions in classical and quantum statistical mechanics differ by a factor of $N!$. The classical version overcounts the number of possible microstates because it assumes particles are distinguishable. Without the correction term, this would often lead to substantially different predictions between the two theories. Fortunately, we know that quantum mechanics is the correct theory and so we are able to choose the correct definition of a microstate.

In many applications, however, we do not know what a true microstate looks like. For example, imagine a particle undergoing a random walk on a lattice X and suppose that we can measure only the distance r between a particle and the origin. It would be natural to define a microstate of this system by the distance between the particle and the origin. If $X={\mathbb{Z}}^{+}$, then this is exactly correct. However, if $X=\mathbb{Z}$, then there are really two microstates for each r. Worse yet, if the lattice is two-dimensional (i.e., $X=\mathbb{Z}\times \mathbb{Z}$), then each r corresponds to a different number of microstates and this number grows without bound. As discussed in Section 2.1, we can still find a reasonable interpretation for the energy of such a system. If we treat each r as a microstate, then $H(r)$ is the potential of mean force in the radial direction. However, our notions of entropy and criticality may change drastically depending on how we define our state space.

For a slightly more involved example, consider the so-called “zipper model” (described in, e.g., [62,63,64]). This is a highly simplified model of, among other things, the conformation of a double-stranded DNA molecule. Suppose there are N base pairs along the DNA molecule (where N can be a positive integer or ∞; if $N=\infty$ then think of the molecule as having a fixed left end but extending infinitely to the right), each of which can either be linked or broken. We will assume that there is only one possible linked configuration for each base pair but that there are G possible broken configurations for each pair, where G is a positive integer. Furthermore, we will suppose that bonds are only broken from left to right. That is, it is possible for a base pair to be in one of the G broken configurations if and only if every base pair to the left is also broken. (This assumption is not entirely necessary, but makes the analysis simpler. Allowing the bonds to break from both ends does not qualitatively alter the behavior of the system, but makes the formulas that follow somewhat more complicated. On the other hand, allowing arbitrary bonds to be broken will make the state space of our system uncountable when the chain becomes infinite. As we will discuss in the next section, this has important consequences.) Suppose that the energy of a linked base pair is 0 and that the energy of any of the G broken configurations for a single base pair is $E>0$ if all base pairs to the left are broken and infinite otherwise. When $N=\infty$ and $G>1$, this system has a phase transition at ${\beta }_c=\ln G/E$. Otherwise, it has no critical temperature [63]. We will show that this critical behaivor is reproduced using (44).

The state space $\mathcal{S}$ of this system is the collection of all possible allowed configurations of linked and broken base pairs. Each configuration consists of m broken base pairs followed by $N-m$ linked base pairs, and there are $G^m$ distinct states for each m. Notice that $\mathcal{S}$ is finite whenever N is and countably infinite when $N=\infty$. The probability of each of these configurations is given by

$$p^{ss}(x;N)=\frac{1}{Q_N}{e}^{-mE},$$

(50)

where m is the number of broken base pairs in x and $Q_N$ is a constant that depends on N (but not x). Note that it is not immediately obvious from the previous assumptions that $p^{ss}(x;N)$ is well-defined, but one can show that $Q_{\infty }$ is non-zero and finite for sufficiently large E (in fact, we can solve for $Q_{\infty }$ exactly, but for our purposes, it is enough to know that it is finite).

Since $\mathcal{S}$ is finite whenever N is, we know that there is no critical temperature for $p^{ss}(·;N)$ when $N<\infty$, so consider the case where $N=\infty$. The possible energy values are $h_m=mE-\ln Q_{\infty }$ for any $m\in {\mathbb{Z}}^{+}$. The Gibbs entropy is therefore

$$S_G(h_m)=k_B\ln \left(\sum _{k=0}^m{G}^k\right)=k_B\ln \left(\frac{G^{m+1}-1}{G-1}\right),$$

(51)

if $G>1$ and $S_G(h_m)=k_B\ln \left(m+1\right)$ if $G=1$.

Applying (44), we therefore have

$${\beta }_c=\underset{m\to \infty }{\mathrm{lim\; sup}}\frac{\ln \left(G^{m+1}-1\right)-\ln \left(G-1\right)}{mE-\ln Q_{\infty }}=\underset{m\to \infty }{\lim }\frac{(m+1)\ln G}{mE}=\frac{\ln G}{E},$$

(52)

when $G>1$. If $G=1$, we have

$${\beta }_c=\underset{m\to \infty }{\mathrm{lim\; sup}}\frac{\ln \left(m+1\right)}{mE-\ln Q_{\infty }}=0.$$

(53)

These critical temperatures exactly match the known values and the mechanism for this behavior is easy to see. When $G=1$, the phase-volume $\mathcal{V}(h)$ grows linearly with h, but when $G>1$ the phase-volume grows exponentially. This allows the entropy $S_G$ to keep pace with the energy as h grows, leading to a criticality.

The preceding calculations are quite similar to those used in the equilibrium statistical mechanical approach of Kittel [63], but the procedure is very different in spirit. In Kittel’s approach, one finds $Q_N$ for arbitrary N, then uses $Q_N$ to calculate a statistic such as the expected number of broken base pairs. Finally, one takes the limit as $N\to \infty$ and demonstrates that this statistic becomes non-analytic at some finite temperature. In particular, Kittel [63] warns that “it is dangerous to write ... the partition function for $N=\infty$; the correct procedure is to evaluate the thermodynamic quantities for finite N and then to examine the limit”. In our approach, we start by finding $p^{ss}(x;\infty )$ (up to a constant). Once we have obtained this distribution, we can calculate $S_G(h)$ for the infinite system and directly obtain ${\beta }_c$. The danger that Kittel describes is still present: our method will fail if $p^{ss}(x;\infty )$ is not well-defined.

This example illustrates a general principle. If an equilibrium statistical mechanical problem has a well-defined equilibrium distribution over a countable state space, then both approaches will identify the same critical temperature.

5. Discussion

It is worth taking a moment to discuss not only what we have shown in the previous sections, but also what we have not shown. We have demonstrated that a stationary distribution over a discrete state space has a finite critical temperature if and only if the Gibbs entropy of the distribution (42) satisfies the relation (44). At such a critical temperature, entropic considerations dominate the dynamics of the stationary process (as opposed to a deterministic system where energetic considerations are dominant). The terminology used here is deliberately suggestive, but one should not take it too far. The novelty of this description of criticality is not as an alternative to (or even as a complete characterization of) the existing statistical mechanics literature on criticality and phase transitions. Instead, we have attempted to generalize some of the notions from statistical mechanics to the broader context of stationary stochastic processes where the notion of criticality does not exist at present.

In particular, notice that there are phase transitions in equilibrium statistical mechanics that do not seem to fit the description given in Section 4. The Lee–Yang theorem, for instance, describes cases where the partition function becomes zero rather than infinite, and two-dimensional Ising models can exhibit various types of phase transitions.

The key point is that we have assumed, from the outset, the existence of a well-defined stationary probability distribution on a countable state space. Such a distribution has a critical temperature ${\beta }_c$ if $Z(\beta )$ approaches either zero or infinity as $\beta \to {\beta }_c$. Because $p^{ss}(x;\beta =1)$ is a probability mass function, $Z(\beta )$ cannot become zero for any finite β. That is, a Lee–Yang type criticalities can only occur if the stationary distribution $p^{ss}$ is not well-defined for any temperature. Ising models, on the other hand, may have well-defined equilibrium distributions even in the thermodynamic limit. However, these models typically have an uncountable state space when $N\to \infty$. For such a distribution, the proofs of Section 4 do not hold as written and other types of criticalities may be present.

Our theory does agree with existing statistical mechanics in the following sense: if one can find a well-defined equilibrium distribution over a countable state space (either for given parameter values such as N and V or in the thermodynamic limit), then (44) will identify the same critical temperature as standard methods. That is, this theory will not produce any new criticalities in a classical problem.

Mora and Bialek have also discussed this approach in regards to Ising models [12]. In particular, they showed that systems where $p^{ss}(x;N)\propto \mathfrak{R}{(x)}^{-\alpha }$ follows a power law have a critical temperature given by ${\beta }_c=1/\alpha$ when N goes to infinity. Their result utilized the identification of $S_G$ with $S_B$, which becomes precise in the thermodynamic limit. In the present paper, we have shown that such an identification is unnecessary and that the critical temperature conditions are still exact in “smaller” systems. Moreover, we have found a broader condition for the existence of a critical temperature, of which the power law relationship is a special case.

After Mora and Bialek’s paper, there has been much discussion about the idea that biological systems are poised at a critical point. This idea arose because researchers obtained estimates of $p^{ss}$ for a wide range of biological systems and all appeared to follow some sort of power law. Such a distribution would indicate a non-zero abscissa ${\beta }_c$. The result from Section 4.2 does seem like it should indicate a criticality in such cases, but there are some important caveats worth considering.

First, it is notoriously difficult to calculate tail properties (such as ${\beta }_c$) from an estimated distribution. Estimates of $p^{ss}$ are necessarily based on a finite number of samples and therefore cannot give reliable information about arbitrarilly low probability events, which is required to calculate (44).

Second, and much more insidious, many biological processes are not in a true steady state. The formal analogies we have made with statistical mechanics only make sense in the context of stationary systems. If $p^{ss}$ actually varies slowly with respect to some other variable (most importantly time), then our notion of criticality does not necessarily correspond to any interesting feature of the system. For instance, Schwab, Nemenman and Mehta [65] have shown that slowly varying latent variables can give rise to apparent power law distributions, which necessarily have a non-zero ${\beta }_c$, even in conditionally independent systems.