Entropy Maximization, Time Emergence, and Phase Transition

Abstract

We survey developments in the use of entropy maximization for applying the Gibbs Canonical Ensemble to finite situations. Biological insights are invoked along with physical considerations. In the game-theoretic approach to entropy maximization, the interpretation of the two player roles as predator and prey provides a well-justified and symmetric analysis. The main focus is placed on the Lagrange multiplier approach. Using natural physical units with Planck’s constant set to unity, it is recognized that energy has the dimensions of inverse time. Thus, the conjugate Lagrange multiplier, traditionally related to absolute temperature, is now taken with time units and oriented to follow the Arrow of Time. In quantum optics, where energy levels are bounded above and below, artificial singularities involving negative temperatures are eliminated. In a biological model where species compete in an environment with a fixed carrying capacity, use of the Canonical Ensemble solves an instance of Eigen’s phenomenological rate equations. The Lagrange multiplier emerges as a statistical measure of the ecological age. Adding a weak inequality on an order parameter for the entropy maximization, the phase transition from initial unconstrained growth to constrained growth at the carrying capacity is described, without recourse to a thermodynamic limit for the finite system.

1. Introduction

Jaynes popularized the entropy maximization technique as a powerful modeling tool for working with finite systems, where results like the Central Limit Theorem or the Stirling Approximation are neither necessary nor appropriate [1,2,3]. On the basis of Jaynes’ work, this survey is designed to highlight some selected aspects of the technique that have appeared over the last thirty years, particularly driven by biological insights in parallel with more traditional topics from physics. Our examples are chosen to be as simple as possible, while still illustrating the key points we wish to convey. In particular, we avoid any reprise of the dependence of entropy and randomness on computational complexity or instrumental resolving power, as discussed in [4]. Further details may be found in the cited references and their bibliographies, but there are many opportunities for interested readers to continue the development and refinement of the topics that we raise.

In Section 2, we set the framework for most of the paper by revisiting the very well-known Gibbs Canonical Ensemble. In particular, we draw attention to an important but rarely mentioned subtlety, namely the strength of the inequality constraints in the procedure of entropy maximization by the method of Lagrange multipliers (7). For perspective, Section 3 takes a brief look at the alternative game-theoretic approach adopted by Topsøe and his school (cf. e.g., [5,6,7]). We propose a biological interpretation for the game as ecological co-evolution between a pair of species: predator and prey. The prey’s interest in randomizing the interactions with the predator, and the predator’s interest in regularizing those interactions, exactly capture the roles of the two players in the abstract game. Thus, the issue,

“The sense in assuming that Player I has the opposite aim, namely to maximize the cost function is more dubious”.

Ref. [5] (p. 198) is resolved by assigning the role of Player I to the prey and Player II to the predator. In this interpretation, the cost function measures how long it will take the predator to determine the prey’s strategy for escape from pursuit.

Section 4 reviews the standard interpretation of the entropy maximization approach to the Canonical Ensemble within statistical mechanics, where macrostates $1,\dots ,i,\dots ,r$ are identified by respective energies $E_1,\dots ,E_i,\dots ,E_r$. In preparation for the subsequent application to a phase transition in an ecological system (Section 6), we go one step beyond Baez’ advocacy of the Lagrange multiplier $\beta$ as a “coolness” parameter [8] in preference to the temperature T, arguing instead for $\tau$, the negative of the coolness $\beta$, as the best choice. Certainly, the temperature T is ill-suited to treatment of condensed matter situations where energies of states are bounded both below and above (e.g., in quantum optics, cf. [9,10]). The use of $-\beta$ as a coordinate in condensed matter physics [9] (Figure 2) then naturally leads to our preference for $\tau$, whose increase is subsequently seen to concur with the Arrow of Time. Compare (27) with (31), for example.

Section 5 uses the Canonical Ensemble for the analysis of an ecology, Lake Gibbs, where species $1,\dots ,i,\dots ,r$ with respective natural growth rates

$$E_1<\dots <E_i<\dots <E_r$$

(1)

compete within an environment having a fixed carrying capacity of N individuals. This system provides a macroscopic model of Eigen’s phenomenological rate equations [11]. While the equations may be solved using standard techniques for handling ordinary differential equations (ODEs), starting from known initial conditions [12,13], the entropy maximization technique offers a novel approach to the solution of the system of coupled ODEs, without the need for initial conditions [14]. This feature of the entropy maximization technique is especially relevant for biological applications, where one encounters existing systems whose genesis is uncertain: the classic “chicken-and-egg” dilemma!

At first glance, use of the Canonical Ensemble in biology may appear to be unrelated to the classical use case of statistical mechanics. However, following the lead of the particle physicists in using natural units with Planck’s constant set to 1 [15] (§III.2), the energies $E_i$ that appear in the statistical mechanics applications of the Canonical Ensemble are seen to have the dimensions of inverse time, exactly like the growth rates $E_i$ that appear in the ecological application. Thus, our preferred conjugate Lagrange multiplier $\tau$ becomes directly identifiable as an emergent time parameter, sharing the statistical macroscopic nature of temperature. As $\tau$ increases, the ecology of Lake Gibbs ages by moving from a diverse mix of the species $1,\dots ,r$ towards an unhealthy monoculture dominated by the most prolific species r—compare (1). The ecology could be rejuvenated by restocking the lake with a broad variety of species, thereby resetting the emergent system time $\tau$ back to a lower value.

Section 6 extends the entropy maximization treatment of the Lake Gibbs ecology: not only to cover the constrained phase analyzed in Section 5, but also the earlier unconstrained phase where each species i (for $1\le i\le r$) is growing exponentially at its natural, unchecked pace $E_i$, before the carrying capacity of the lake is reached [16]. Thus, entropy maximization is shown to handle a phase transition, for a finite system, without resort to any infinite “thermodynamic limit”. This is achieved by moving beyond the strict inequalities for the constraints on the optimization domain noted in Section 2. Along with positivity constraints for parameters $p_1,\dots ,p_r$ tracking the respective species $1,\dots ,r$, an order parameter $p_0$ subject to a weak non-negativity constraint is added (38). If the constraint is binding, i.e., $p_0=0$, then the entropy maximization reduces to its previous form for the constrained phase as described in Section 5. On the other hand, if the constraint is slack, i.e., $p_0>0$, then the entropy maximization returns the unconstrained phase where each species is growing exponentially. As a proof of concept, this basic example suggests that future research, working with richer constellations of strong and weak inequality constraints, should provide finitary entropy maximization analyses of more elaborate, multidimensional phase diagrams.

2. The Canonical Ensemble

Consider a finite, nonempty set or phase space (Figure 1) that comprises N equally likely individual elements or microstates (dots in Figure 1) with a partition

$$\Pi =\left\{C_1,\dots ,C_r\right\}$$

(2)

into the disjoint union of a family of r subsets or macrostates (boxes in Figure 1). Suppose that the macrostate $C_i$ comprises $n_i$ microstates, for $1\le i\le r$, so that

$$N=n_1+\dots +n_r.$$

(3)

The set $\Pi$ of (2) may be considered as (invoking) an experiment: take a microstate and determine the macrostate $C_i$ to which it belongs.

Figure 1. A phase space of microstates, with a partition $\Pi =\left\{C_1,\dots ,C_r\right\}$ into macrostates [17] (Figure 2).

Figure 1. A phase space of microstates, with a partition $\Pi =\left\{C_1,\dots ,C_r\right\}$ into macrostates [17] (Figure 2).

What information is gained by performing the experiment? Initially, a store of size $\log N$ would be required to tag the microstates. Suppose that we perform the experiment and obtain outcome $C_i$. In that case, knowing that the microstates are localized within $C_i$, we now only require a store of size $\log n_i$. The information gain as a result of the experiment is $\log N-\log n_i$. However, there is only a probability

$$p_i={\displaystyle \frac{n_i}{N}}$$

(4)

of obtaining outcome $C_i$. The expected information gain from the experiment is the weighted average

$$H(\Pi )=\sum _{i=1}^r{p}_i\left(\log N-\log n_i\right)=-\sum _{i=1}^r{p}_i\log p_i$$

(5)

of the information gains from each of the possible outcomes. This quantity is described as the (information-theoretic) entropy of the partition $\Pi$. It may be characterized as the expected value of the logarithm of the odds, namely $p_i^{-1}$ to one, of obtaining a particular macrostate $C_i$.

In practice, the specific partition of the phase space into macrostates is not known a priori. However, suppose that a numerical value (typically representing a dimensioned scientific quantity like an energy or a growth rate) is associated with each macrostate. A particular macrostate $C_i$ then consists of all those microstates that yield an observed numerical value $E_i$ when the experiment is performed. While the actual probabilities $p_i$ of the individual macrostates may be unknown themselves, suppose that the expected outcome

$$E=\sum _{i=1}^r{p}_i{E}_i$$

(6)

is known, say from a measurement performed on a sample of the microstates. In order to construct a model, the probabilities $p_i$ have to be assigned. If the expected value (6) is the only information available, then the truest model is the one that maximizes the entropy $H(\Pi )$ subject to the constraint (6). This model is known as Gibbs’ canonical ensemble.

The maximization problem is usually solved by the method of Lagrange multipliers [18] (Th. 3.2.2). Here, we wish to draw attention to a subtle detail of the procedure, which is rarely stated explicitly: the nonemptiness of the macrostates in the family (2) means that the optimization is taken over the set

$${\Delta }_{r-1}^{\circ }=\{(p_1,\dots ,p_r)\mid p_1+\dots +p_r=1,0<p_1,\dots ,p_r\}$$

(7)

of positive probabilities, the interior of the $(r-1)$-dimensional simplex ${\Delta }_{r-1}$. Thus, in maximization of the Lagrangrian

$$L(p_i,\kappa ,\tau )=-\sum _{i=1}^r{p}_i\log p_i+\sigma \left(1-\sum _{i=1}^r{p}_i\right)-\tau \left(E-\sum _{i=1}^r{p}_i{E}_i\right),$$

(8)

the stationarity conditions

$${\displaystyle \frac{\partial L}{\partial p_i}}=0$$

for all $1\le i\le r$ apply since the maximization is performed over ${\Delta }_{r-1}^{\circ }$. They give

$$\log p_i=-(1+\sigma )+\tau E_i$$

(9)

or

$$p_i=\exp \left(\tau E_i\right)/\exp (1+\sigma ).$$

Substitution in the completeness constraint from (3) yields

$$1=\sum _{i=1}^r{p}_i=\sum _{i=1}^r\exp \left(\tau E_i\right)/\exp (1+\sigma )$$

or

$$\exp (1+\sigma )=\sum _{i=1}^r\exp \left(\tau E_i\right).$$

(10)

Defining the partition function (or “Zustandsumme”)

$$Z\left(\tau \right)=\sum _{i=1}^r\exp \left(\tau E_i\right)$$

(11)

of $\tau$, we have

$$p_i={\displaystyle \frac{\exp \left(\tau E_i\right)}{Z\left(\tau \right)}}$$

(12)

in Gibbs’ canonical ensemble. The entropy (5) may be written as

$$H(\Pi )=-\tau E+\log Z\left(\tau \right)$$

(13)

in terms of the variable $\tau$. As another point worthy of special attention, we emphasize that the units for the multiplier $\tau$ are inverse to those for the numerical values $E_i$.

3. The Game Theoretic Approach

While the entropy maximization procedure outlined in the previous section has a number of advantages, most notably the identification of the Lagrange multiplier $\tau$ as a conjugate to the numerical values $E_i$ assigned to the macrostates, it is not the only approach. The Danish school (cf. [5,6,7], for example) have strongly advocated for an approach that involves a game between two players. In their version,

Player I chooses a consistent distribution, and Player II chooses a general code. …[T]he objective of Player II appears well motivated. [A] cost function can be interpreted as mean representation time, …and it is natural for Player II to attempt to minimize this quantity. The sense in assuming that Player I has the opposite aim, namely to maximize the cost function, is more dubious. [5] (p.198).

Here, in our simplified setting, which is chosen to avoid detailed topological and analytical concerns, we present a brief and introductory alternative account that is founded upon a more adequate, symmetrical notation and a representative interpretation of the two respective Players I and II as prey and predator in an ecological context. A parallel interpretation in a political science context might identify “people” and “government” as the two players. A relationship of this kind is implied, for example, in the work of J.C. Scott [19,20]. We refer to our description as the Predator–Prey Representation.

The ecological principle underlying the Predator–Prey Representation is that predators must seek to regularize their relationships with their prey, while the prey seek to randomize those relationships. To catch their prey, predators need encoded hunting strategies. But to evade their predators, the prey need unpredictable escape strategies.

In the model, the prey may draw from a set $\mathcal{P}\subseteq {\Delta }_{r-1}^{\circ }$ of so-called consistent probability distributions $\Pi$ on a set of evasive actions that includes no more than a finite number r of elements. Thus, a consistent distribution gives a specific option for an escape strategy. Figure 2 shows a toy model strategy where the prey is aiming to distract its predator when being pursued from behind, by swinging its tail in a pendulum-like motion. In Figure 2b, the particular consistent distribution $\Pi$ exhibited is related to (and identified with) a macrostate partition $\Pi =\left\{C_1,C_2,C_3\right\}$ of the type of phase space displayed in Figure 1. Furthermore, the phase space in this particular example may be overlaid on the classical phase space of a linear harmonic oscillator (such as a pendulum) with position variable q and momentum variable p. Thus, a microstate (such as m that appears in the macrostate $C_1$) may be considered to describe a brief video clip of a specific tail motion.

Table 1. Predator’s interpretation of prey’s tail motion. The prey’s strategy is determined by its random choice of macrostate from the consistent distribution $\Pi$ shown in Figure 2b.

Macrostate	Positions in the Macrostate	Momenta in the Macrostate
$C_1$	Left side	Any
$C_2$	Right side	Rightwards
$C_3$	Right side	Leftwards

identifies the macrostates with specific tail motion features that would be perceived by the predator in their encoding of the prey’s behavior, using the binary prefix code $\kappa$ displayed in Figure 2a.

When applying the code $\kappa$ during the chase in its attempt to learn what the prey is doing, the predator first checks if the prey’s tail is on the left side ($q<0$). If the answer is “yes” or 1, the predator has taken just 1 step to correctly identify that the prey has selected macrostate $C_1$. This event occurs with probability $2^{-1}$ within the prey’s strategy $\Pi$. On the other hand, if the answer is “no” or 0, so the tail is on the right, the predator then checks if the prey’s tail is swinging to the right ($p>0$). If the answer to this question is “yes” or 1, the predator has taken 2 steps to correctly identify that the prey has selected macrostate $C_2$, an event that occurs with probability $2^{-2}$ within the strategy $\Pi$. Finally, if the answer to the latter question is “no” or 0, the predator has taken 2 steps to correctly identify that the prey has selected macrostate $C_3$, an event that, again, occurs with probability $2^{-2}$ within strategy $\Pi$. Thus, with the code $\kappa$, the expected number of steps that the predator takes to recognize the prey’s strategy $\Pi$ is

$$\langle \Pi |\kappa \rangle ={\displaystyle \frac{1}{2}}·1+{\displaystyle \frac{1}{4}}·2+{\displaystyle \frac{1}{4}}·2={\displaystyle \frac{3}{2}},$$

(14)

matching the entropy

$$H(\Pi )=-\sum _{i=1}^3{p}_i{\log }_2{p}_i$$

of the consistent distribution $\Pi$ in bits.

In the “bra-ket” notation that appears on the left of (14), we switch the sides relative to [5] (p.196) and [7] (8) so that the distribution $\Pi$ belonging to Player I (the prey) appears first, while the code $\kappa$ belonging to Player II (the predator) appears second. In general, for a code $\kappa$ with respective code lengths ${\kappa }_1,\dots ,{\kappa }_r$ and a consistent distribution $\Pi$ with respective probabilities $p_1,\dots ,p_r$, the cost function

$$\langle \Pi |\kappa \rangle =-\sum _{i=1}^r{p}_i{\kappa }_i$$

(15)

is defined. As seen on the basis of the illustrative example from Figure 2, the cost function represents the expected time (number of questions asked and answered) taken for Player II using a binary prefix code $\kappa$ to recognize which macrostate has been chosen from the canonical distribution $\Pi$ adopted by Player I. The abstract information-theoretic game between Player I and Player II, as envisaged by the Danish school, is instantiated by the concrete ecological games that take place over multiple generations as prey and predator population pairs co-evolve. In particular, the problem raised in the earlier quotation from [5] (p. 198),

“The sense in assuming that Player I has the opposite aim, namely to maximize the cost function is more dubious”,

is clearly solved by the prey’s interest in extending the time it takes its predators to identify an escape strategy.

The full set of escape strategies $\Pi$ available to the prey is identified as the set $\mathcal{P}$ of consistent distributions. Now, consider a particular code $\kappa$ available to the predator as a hunting strategy. The risk [5] (3.11)

$$R\left(\mathcal{P}\right|\kappa )=\underset{\Pi \in \mathcal{P}}{\sup }\langle \Pi |\kappa \rangle$$

(16)

associated with that hunting strategy expresses the maximum length of time it might take the predator to identify the prey’s behavior using $\kappa$—a measure of the predator’s risk of starvation if it were to stubbornly rely on $\kappa$ as its hunting strategy. Successful predators deploy multiple hunting strategies, assembled in a set $\mathcal{K}$ of codes $\kappa$. Their risk of starvation is reduced to their minimum risk value [5] (3.12)

$$R_{\min }\left(\mathcal{P}\right|\mathcal{K})=\underset{\kappa \in \mathcal{K}}{\inf }R\left(\mathcal{P}\right|\kappa )$$

(17)

if they are able to draw on any one of these strategies.

Dually, we begin by noting that the full set of hunting strategies $\kappa$ available to the predator has been identified as the set $\mathcal{K}$ of codes. We may now consider a particular consistent distribution $\Pi$ that is available to the prey as an escape strategy. The coded entropy

$$H(\Pi |\mathcal{K})=\underset{\kappa \in \mathcal{K}}{\inf }\langle \Pi |\kappa \rangle$$

(18)

(cf. [5] (3.9)) associated with that escape strategy expresses the minimum length of time it might take a predator to identify the strategy—a measure of the prey’s risk of capture or randomization success—if it were to stubbornly rely on $\Pi$ as its escape strategy. Successful prey species deploy multiple escape strategies, assembled in their repertoire $\mathcal{P}$. Their time of freedom when pursued is maximized to the maximum coded entropy

$$H_{\max }\left(\mathcal{P}\right|\mathcal{K})=\underset{\Pi \in \mathcal{P}}{\sup }H(\Pi |\mathcal{K})$$

(19)

if they are able to draw on any one of these strategies (cf. [5] (3.10)).

Taking infima over $\mathcal{K}$ on each side of the quantified statement

$$\forall (\mathsf{P},\lambda )\in \mathcal{P}\times \mathcal{K},\langle \mathsf{P}|\lambda \rangle \le \underset{\Pi \in \mathcal{P}}{\sup }\langle \Pi |\lambda \rangle$$

gives $\forall \mathsf{P}\in \mathcal{P},{\inf }_{\kappa \in \mathcal{K}}\langle \mathsf{P}|\kappa \rangle \le {\inf }_{\kappa \in \mathcal{K}}{\sup }_{\Pi \in \mathcal{P}}\langle \Pi |\kappa \rangle$. The inequality

$$\underset{\Pi \in \mathcal{P}}{\sup }\underset{\kappa \in \mathcal{K}}{\inf }\langle \Pi |\kappa \rangle \le \underset{\kappa \in \mathcal{K}}{\inf }\underset{\Pi \in \mathcal{P}}{\sup }\langle \Pi |\kappa \rangle$$

(20)

then follows on taking the supremum over $\mathcal{P}$. It provides the final link in the full chain

$$\begin{array}{cc}\hfill \underset{\kappa \in \mathcal{K}}{\inf }\langle \Pi |\kappa \rangle & \stackrel{\left(18\right)}{=}H(\Pi |\kappa )\le \underset{\Pi \in \mathcal{P}}{\sup }H(\Pi |\mathcal{K})\stackrel{\left(19\right)}{=}{H}_{\max }\left(\mathcal{P}\right|\mathcal{K})\hfill \\ \hfill & \stackrel{\left(20\right)}{\le }{R}_{\min }\left(\mathcal{P}\right|\mathcal{K})\stackrel{\left(17\right)}{=}\underset{\kappa \in \mathcal{K}}{\inf }R\left(\mathcal{P}\right|\kappa )\le R\left(\mathcal{P}\right|\kappa )\stackrel{\left(16\right)}{=}\underset{\Pi \in \mathcal{P}}{\sup }\langle \Pi |\kappa \rangle \hfill \end{array}$$

(21)

of inequalities that summarizes the relationships between the behaviors of the prey and the predator. Reference [5] continues with a general abstract analysis of when equality is obtained. In the ecological setting, equality is to be expected for stable predator–prey population pairs that have co-evolved over multiple generation times.

4. Statistical Mechanics

After the brief excursion into the game-theoretic approach, we return to a consideration of the Lagrangian approach as presented in Section 2. In the classical applications of the canonical ensemble, one may consider the microstates as particles having a certain energy. Thus, the numerical value $E_i$ associated with macrostate $C_i$ is an energy (say in joules). The conjugate variable $\tau$, which was carefully chosen to match the non-classical applications in the subsequent sections, is connected to the temperature T (say in degrees Kelvin) by

$$\left(kT\right)\tau +1=0$$

(22)

using Boltzmann’s constant k. Baez [8] (p. 30) refers to the traditional conjugate variable $\beta =-\tau =1/kT$ as the coolness: the lower the (non-negative) temperature T, the higher the value of $\beta$. The problem with such traditional conventions, even within statistical mechanics, is that they are ill-adapted to handling negative temperatures, which are readily observed in condensed matter situations where energy levels are bounded both below and above [9,10]. In particular, the use of $-\beta$ (i.e., our $\tau$!) as an abscissa coordinate in the first figure of [9] should be noted. If $T=0$ and $\tau =0$ are avoided, (22) shows that an increase in $\tau$ conveniently corresponds to an increase in T, and vice versa.

The relation (22) gives some insight into the nature of the quantity $\tau$ in the canonical ensemble: Just like the temperature, it is a statistical property of collections of microstates. The thermodynamic entropy is

$$S=kH$$

in joules per Kelvin degree. The thermodynamic potential is the dimensionless quantity

$$\Psi =\log Z\left(\tau \right),$$

while the Helmholtz free energy is

$$F=-kT\Psi =\Psi /\tau$$

in joules. The relation (13) takes the form

$$F=E-TS.$$

The Equation (12) becomes

$$p_i=e^{-E_i/kT}/e^{-F/kT},$$

(23)

a well-known formula of kinetic theory (compare [21]). For example, it may be used to describe the distribution of atmospheric particles at different heights, according to their potential energy in the Earth’s gravitational field [21] (§40-1,2), [3] (§6.1.2(a)).

When considering physical applications of the canonical ensemble, it may prove useful to use natural units or Planck units with Planck’s constant set to 1 [15] (§III.2). Then, the energies that appear in the statistical mechanics applications of the Gibbs ensemble are seen to have the dimension (time)⁻¹. For example, the energy of a photon of light is given as the product of Planck’s constant with the frequency of the corresponding wave.

5. Time Emergence

In the statistical mechanical applications of the canonical ensemble discussed in Section 4, the coolness $\beta =1/kT$, a Lagrange conjugate of energy, emerges as a statistical property of a collection of microstates. On the other hand, using natural units, such Lagrange conjugates of energy as $\beta$ or our preferred $\tau$ should appear with the units of time. Now, following [14], we examine a model where $\tau$ does indeed represent an emergent intrinsic age of a biological system. It provides a conceptually instructive model of competition between r different species, labeled $1,\dots ,r$, as described by Eigen’s phenomenological rate equations [11]. Suppose that species i has an unconstrained growth rate of $E_i$ (say in per annum units). This means that a population of $n_i$ individuals of species i growing without constraint has a rate of change

$${\dot{n}}_i=n_i{E}_i$$

(24)

(using Newton’s dot notation for the derivative). At a Newtonian time t in an interval $[s,u]$ from a start time s to an ultimate time u, the population $n_i\left(t\right)$ is given as

$$n_i\left(t\right)=n_i\left(s\right)\exp \left(t{E}_i\right)$$

(25)

—exponential growth. Competition (as modeled by Eigen’s equations) arises when the individuals of the r species form a joint population maintained at a constant total count N. The birth of one individual is compensated for by the death of another.

Figure 3 visualizes the individuals as fish in a lake, where the food requirements of each individual fish are the same, and the food supply sustains the constant total number N. Figure 3 may also be viewed as a slightly less abstract version of Figure 1. The individual fish correspond to the microstates, which share a macrostate if they belong to the same species.

The traditional treatment to determine the population $n_i\left(t\right)$ of species i at given Newtonian times t in an interval $[s,u]$ (cf. [12,13] or [16] (§4)) involves solving the coupled quasilinear system

$${\dot{n}}_i\left(t\right)=\left(E_i-E\left(t\right)\right)n_i\left(t\right)=E_i{n}_i\left(t\right)-\sum _{j=1}^r{n}_i\left(t\right){\displaystyle \frac{E_j}{N}}{n}_j\left(t\right)$$

(26)

of ordinary differential equations. Although (26) may be seen as a special case of [3] (20 Equation (80)), the treatment in that reference, concerned as it is with equilibrium or long-term values, does not relate to our analysis, the coupling being introduced through the final, quadratic or two-point interaction term of (26). Based on an Ansatz to translate to a linear system [16] (26), the classical solutions

$$n_i\left(t\right)=n_i\left(s\right)\exp \left(t{E}_i\right)/\exp \left({\int }_s^{t}E\left(t^{\prime }\right)d{t}^{\prime }\right)$$

(27)

are valid over the Newtonian time interval $[s,u]$. Here, the initial conditions that are required to solve the system of ordinary differential equations record the population count $n_i\left(s\right)$ at $t=s$.

The function $E\left(t\right)$ appearing in the middle part of (26), whose specification follows from

$$0={\displaystyle \frac{d}{dt}}\sum _{i=1}^r{n}_i\left(t\right)=\sum _{i=1}^r{\dot{n}}_i\left(t\right)=\sum _{i=1}^r\left(E_i-E\left(t\right)\right)n_i\left(t\right)=\sum _{i=1}^r{E}_i{n}_i\left(t\right)-NE\left(t\right),$$

represents the instantaneous cull rate

$$E\left(t\right)=\sum _{i=1}^r{\displaystyle \frac{n_i\left(t\right)}{N}}{E}_i$$

(28)

required to hold the total population constant at the carrying capacity N. The argument

$${\int }_s^{t}E\left(t^{\prime }\right)d{t}^{\prime }$$

of the exponential in the denominator of the right hand side of (27) is then recognized as counting the total number of starvation victims registered over the time interval $[s,t]$.

In the approach through the canonical ensemble, we imagine going to the lake and catching a moderately sized group of fish in a net. The relative frequency $f_i$ of each species i in the catch is taken as a good approximation to the overall probability $p_i$ of catching a member of that species. In other words, we take

$$f_i={\displaystyle \frac{n_i}{N}}$$

(29)

for $1\le i\le r$ at any given time. Knowing these relative frequencies $f_i$, together with the unconstrained growth rates $E_i$ of each species, we obtain the expected average value (6) that is the premise for the canonical ensemble. The relationship (28), fundamental to our approach, equates the cull rate E to the average

$$\sum _{i=1}^r{f}_i{E}_i$$

(30)

or gross growth rate (GGR) of the population. Equation (12) yields

$$n_i\left(\tau \right)=N\exp \left(\tau E_i\right)/\sum _{j=1}^r\exp \left(\tau E_j\right),$$

(31)

in particular with

$$n_i\left(s\right)=N\exp \left(s{E}_i\right)/\sum _{j=1}^r\exp \left(s{E}_j\right)$$

(32)

as an initial condition that does not need to be specified separately in our approach. When the species are labeled by increasing unconstrained birth rates (assuming non-degeneracy), say

$$0<E_1<E_2<\dots <E_r$$

(33)

with $1<r$, then the most prolific species r will ultimately dominate.

The intrinsic time $\tau$ that appears in (31) is an emergent statistical property of the complex system. As the system ages, the proportion of the dominant species r increases, leading to a lack of biodiversity. Restocking the lake with a good mix of the various species would rejuvenate the ecosystem, resetting the system time $\tau$ independently of the relentless forward progress of the Newtonian time t.

In conjunction with the emergence of the system time $\tau$, the canonical ensemble treatment of the ecosystem with the fixed carrying capacity N has an additional feature, which will be exploited more in the following section. For this discussion, assume the start time s is set to $t=0$, with a uniform distribution at that time in which each species i has a population

$$n_i\left(0\right)={\displaystyle \frac{N}{r}}.$$

(34)

Then, the solutions (25) of the unconstrained Equation (24) for a negative time t take the form

$$n_i\left(t\right)={\displaystyle \frac{N}{r}}\exp \left(t{E}_i\right).$$

(35)

Let M denote the total fish population at any given time. Thus, $M=N$ for $t>0$, while Equation (35) gives

$$M={\displaystyle \frac{N}{r}}\sum _{i=1}^r\exp \left(t{E}_i\right)$$

(36)

for $t<0$. As a consequence, the equations

$$f_i=\exp \left(t{E}_i\right)/\sum _{j=1}^r\exp \left(t{E}_j\right)$$

(37)

for the relative frequencies, obtained for a positive time t from (12) making use of the canonical ensemble description of the constrained ecology, are equally valid during the unconstrained population growth at negative times. It is indeed remarkable that the relative frequencies extrapolate backwards from the canonical ensemble, even though the assumptions leading to the canonical ensemble do not apply in this unconstrained regime.

6. A Phase Transition

In the classical entropy-maximization treatment of the canonical ensemble (Section 2), the optimization is taken over the open set

$${\Delta }_{r-1}^{\circ }=\{(p_1,\dots ,p_r)\mid p_1+\dots +p_r=1,0<p_1,\dots ,p_r\}$$

(7) of positive probabilities, the interior of the $(r-1)$-dimensional simplex ${\Delta }_{r-1}$. In the competition model presented in Section 5, $p_i$ represents the probability of catching a fish of species i, for $1\le i\le r$, although we often prefer to use the relative frequency $f_i$ of (29) as a proxy. In this section, following [16], we now consider an additional variable $p_0$ and the subset

$${\Delta }_r^{\ast }=\{(p_0,p_1,\dots ,p_r)\mid p_0+p_1+\dots +p_r=1,0\le p_0,0<p_1,\dots ,p_r\}$$

(38)

of the r-dimensional simplex ${\Delta }_r$, maximizing the parametric entropy

$$-\sum _{i=0}^r{p}_i\log p_i$$

(39)

over the non-open set ${\Delta }_r^{\ast }$. This procedure leads to a more complete description of the ecology depicted in Figure 3 that also applies to the unconstrained phase where the total fish population M is below the carrying capacity N. In this broadened context, the regime analyzed in Section 5 is called the constrained phase. For simplicity of exposition, the phase transition is assumed to take place at system time $\tau =0$ with a uniform distribution of all the species at that time, as in (34) above. Following the analogy between time and temperature, one might regard setting the time of the phase transition to zero as analogous to the (pre-1948) Celsius scale setting of zero for a phase transition of water.

Given the various kinds of phase transition that physicists might recognize, we invoke Penrose’s general definition [22] (§28.1),

“A phenomenon of this nature, where a reduction in the ambient temperature induces an abrupt gross overall change in the nature of the stable equilibrium state of the material, is called a phase transition”,

to justify our current terminology. In the ecological setting, a “reduction in the ambient temperature” T is interpreted as an increase in the system time $\tau$ in accordance with the relation (22). Co-opting common physical terminology, we describe the variable $p_0$ in (38) as the order parameter [23]. Since the variables $p_1,\dots ,p_r$ no longer function directly as naive catch probabilities during the unconstrained phase, the way they do during the constrained phase, we refer to them as (additional) parameters in the complete history of the ecosystem, thereby leading to the terminology of (39) for the corresponding entropy. We then define

$$\mathrm{H}=-\sum _{i=1}^r{f}_i\log f_i$$

(40)

as the population entropy (Figure 4) to contrast with the parametric entropy of (39).

Since the population counts of the various fish species do not undergo a discontinuous change at the phase transition, one may wonder where the “abrupt gross overall change in the nature of the stable equilibrium state” comes in. Mathematically, it is seen in the change from the exponential growth solution (25) to the modified version (27), where the denominator with the exponentiated integral suddenly appears. For the individual fish, it means the drastic arrival of the possibility of death by starvation, where previously they were always able to live out their natural lifespans. It is also worth noting the emergence of the “long-range correlations” indicated by the addition of the final term of (26) to the original unconstrained growth Equation (24).

Heuristically, if not too literally, the order parameter $p_0$ may be associated with a ghost species 0 having a natural unconstrained growth rate $E_0=0$. Consider maximization of the parametric entropy (39) over the non-open set ${\Delta }_r^{\ast }$ of (38) subject to the equality constraint

$$D=\sum _{i=0}^r{p}_i{E}_i$$

(41)

on the parameters. The quantity D appearing in (41)—obviously motivated as a version of (6) that has been extended to include the ghosts—is discussed at the end of this section using (51). We take the Lagrangian

$$\mathcal{L}(p_i,\sigma ,\tau )=-\sum _{i=0}^r{p}_i\log p_i+\sigma \left(1-\sum _{i=0}^r{p}_i\right)-\tau \left(D-\sum _{i=0}^r{p}_i{E}_i\right)$$

(42)

in terms of the parameters $p_0,p_1,\dots ,p_r$. When the weak inequality constraint on $p_0$ from the definition (38) of ${\Delta }_r^{\ast }$ is binding or “active” [18] (p 221), i.e., $p_0=0$ and $p_0\log p_0=0$ (either by convention or as the result of the limiting procedure ${lim}_{p_0\to 0+}$), we have

$$\mathcal{L}(p_0,p_1,\dots p_r,\sigma ,\tau )=L(p_1,\dots p_r,\sigma ,\tau )$$

in terms of the original Lagrangian (8). Thus, in the situation where the order parameter $p_0$ is zero and the remaining parameters $p_i,\dots ,p_r$ are recognized as the corresponding relative frequencies, the extended description reduces to the original description. In particular, when $p_0=0$ (i.e., there are no ghosts), the parametric entropy reduces to the population entropy.

When $p_0>0$, the weak inequality constraint on $p_0$ from the definition (38) of ${\Delta }_r^{\ast }$ is slack or “inactive” [18] (p. 221); the analysis of the Lagrangian (42) for the parametric entropy proceeds in similar fashion to the analysis of the original Lagrangian (8) in Section 2. The stationarity conditions $\partial \mathcal{L}/\partial p_i=0$ for $0\le i\le r$ reduce to $\log p_i=-(1+\sigma )+\tau E_i$ or $p_i=\exp \left(\tau E_i\right)/\exp (1+\sigma )$. A substitution in the completeness constraint yields

$$1=\sum _{i=0}^r{p}_i=\sum _{i=0}^r\exp \left(\tau E_i\right)/\exp (1+\sigma )$$

or

$$\exp (1+\sigma )=\sum _{i=0}^r\exp \left(\tau E_i\right)=1+\sum _{i=1}^r\exp \left(\tau E_i\right)=1+Z\left(\tau \right)$$

using (11) for the latter term, yielding the expressions

$$p_i=\exp \left(\tau E_i\right)/\sum _{j=0}^r\exp \left(\tau E_j\right)={\displaystyle \frac{\exp \left(\tau E_i\right)}{1+Z\left(\tau \right)}}$$

(43)

for the parameters $p_0,p_1,\dots ,p_r$.

For $i=0$, the expression (43) determines the order parameter as

$$p_0={\left(1+\sum _{j=1}^r\exp \left(\tau E_j\right)\right)}^{-1}={\displaystyle \frac{1}{1+Z\left(\tau \right)}}.$$

(44)

Taking Equation (43) for $0<i\le r$, the remaining parameters may be rewritten in the form

$$p_i=p_0\exp \left(\tau E_i\right).$$

(45)

For $1\le i\le r$, a substitution of this expression into (35) yields

$$n_i={\displaystyle \frac{N{p}_i}{r{p}_0}},$$

(46)

whence (36) may be rewritten as

$$M=\sum _{j=1}^r{n}_j={\displaystyle \frac{N(1-p_0)}{r{p}_0}}$$

(47)

to determine the total population in the unconstrained phase in terms of the order parameter. The assignment $p_0\mapsto M$ represented by (47) may also be inverted to yield

$$p_0={\displaystyle \frac{N}{N+rM}}$$

(48)

as an equivalent expression of $p_0$ in terms of M. It is clear that the expressions (46) and (47), valid for negative times, will not continue to hold in the constrained regime where $p_0=0$.

In the unconstrained phase, an experiment may be conducted to determine the total population (47) and, thus, the order parameter as given by (48). A fisherman trawls a fixed volume of water and counts the number $M^{\prime }$ of fish caught in the trawl. The number of fish $N^{\prime }$ that would be caught in the trawl at the carrying capacity N is presumed to be known, so the total population M is obtained as $M^{\prime }N/N^{\prime }$. This refinement of the catch protocol is described as the trawl. Using the relation (45) that holds for $p_0>0$, the relative frequency of species i in the trawl is

$$f_i={\displaystyle \frac{\exp \left(E_i\tau \right)}{{\sum }_{j=1}^r\exp \left(E_j\tau \right)}}={\displaystyle \frac{p_i}{{\sum }_{j=1}^r{p}_j}}=p_i{(1-p_0)}^{-1}$$

(49)

recalling (37) for the first equality. The outer fragment

$$f_i=p_i{(1-p_0)}^{-1}\mathrm{or} \mathrm{equivalently}{p}_i=f_i(1-p_0)$$

(50)

of (49) is then seen to hold for the entire history, extending the previous identification $f_i=p_i$ which only holds in the constrained regime $p_0=0$. The factor $(1-p_0)$ appearing in (50) is described as the modifier for any $p_0$ within the range $\left\{0\right\}\cup \left({(1+r)}^{-1},1\right)$, and, thus, the parameters $p_1,\dots ,p_r$ are recognized as modified relative frequencies.

The quantity D that appears in the constraint (41) may now be examined. We have

$$D=\sum _{i=0}^r{p}_i{E}_i=(1-p_0)\sum _{i=1}^r{f}_i{E}_i=(1-p_0)E$$

(51)

using the second equation of (50). Since D is given as the product of the modifier with the gross growth rate (30), it is described as the modified gross growth rate. In particular, once the order parameter is known from the trawl, then the modified GGR is obtained from the unmodified GGR, which is also determined from the trawl.

In summary, the extended Lagrangian $\mathcal{L}$ given in (42) represents a maximization of the parametric entropy, subject to the completeness constraint on the parameters and the knowledge of the modified gross growth rate D that is obtained from the trawl. When $\mathcal{L}$ is maximized over the constraint set ${\Delta }_r^{\ast }$ that includes a weak inequality for the order parameter $p_0$ along with the usual strong inequalities for the remaining parameters, it enables one to use entropy maximization for the modeling of a phase transition in a finite situation without recourse to any infinitary “thermodynamic limit”.