Entropy Production and the Maximum Entropy of the Universe †

: The entropy of the observable universe has been calculated as S uni ~ 10 104 k and is dominated by the entropy of supermassive black holes. Irreversible processes in the universe can only happen if there is an entropy gap Δ𝑆 between the entropy of the observable universe S uni and its maximum entropy S max : Δ𝑆 = S max − S uni . Thus, the entropy gap Δ𝑆 is a measure of the remaining potentially available free energy in the observable universe. To compute Δ𝑆 , one needs to know the value of S max . There is no consensus on whether S max is a constant or is time-dependent. A time-dependent S max ( t ) has been used to represent instantaneous upper limits on entropy growth. However, if we define S max as a constant equal to the final entropy of the observable universe at its heat death, S max ≡ S max , HD , we can interpret T Δ𝑆 as a measure of the remaining potentially available (but not instantaneously available) free energy of the observable universe. The time-dependent slope dS uni /dt(t) then becomes the best estimate of current entropy production and T dS uni /dt(t) is the upper limit to free energy extraction.


The Entropy Gap and the Approach to 'Heat Death'
Irreversible processes in the universe can only happen if there is a nonzero entropy gap ( ) defined as the difference between the maximum entropy and present entropy of the observable universe ( ): where ≤ [1][2][3]. While it is clear that is a function of time and increases monotonically, it is unclear whether is a constant or time-dependent [4]. Before the advent of the current conventional -CDM cosmological model, there was a sense in which the universe never reached a heat death [5], [Sec. VI D], Suni never reached Smax, and eternal life was possible [6]. However, in the -CDM model (with = constant) any observable universe has a nonzero temperature known as the de Sitter temperature ~ produced by the cosmic event horizon [7]: The current temperature of the cosmic microwave background TCMB = 2.7 K. Since ~ .
where a is the scale factor of the universe, as a increases → [8]. Thus, the background temperature of the universe will equal the de Sitter temperature when the universe is ~10 30 times bigger than it is now. This will be the heat death of the universe (Figure 1) when we will have At which point, irreversible processes (including life) will no longer be possible [9,10]. A heat death is expected to occur in the -CDM universe at a time defined by the timescale for the evaporation of the largest black holes ~ years from now, when free energy has completely dissipated, and life is no longer possible [10] [Sec. IV D of 6].
The availability of free energy allows for the creation and maintenance of far-from-equilibrium dissipative structures (e.g., stars, cyclones, life forms) [2,11,12]. The rate of free energy consumption (always accompanied by entropy production) compared to the available free energy determines the longevity of these structures [13].

Estimating the Entropy Gap and Free Energy
An estimate for the remaining free energy in the universe depends on the present value of . The present entropy of the universe has been calculated in [1]. Quantifying usually involves a large representative comoving volume of the universe (typically a sphere) with approximately zero net flows of matter, energy, and entropy across its boundary, due to the largescale homogeneity of the universe [1]. The expansion of a homogeneous non-gravitationally bound fluid is isentropic [2,14,15]. Relativistic matter (i.e., CMB photons, relic neutrinos) dominates the entropy in non-gravitational degrees of freedom [1,15]. The entropy in gravitational degrees of freedom is dominated by the supermassive black holes that exist at the center of most galaxies [1]. The entropy of the comoving volume of the observable universe is of order ~ [1]. Another scheme to calculate includes the entropy of the cosmic event horizon (CEH) [1,7]. In this scheme, the representative volume is defined as the time-dependent CEH boundary which is not comoving and must account for the migration of entropy across the horizon [16]. The CEH entropy dominates the entropy budget of the universe when included, with ~ ~ . The two schemes detailed in [1] are different measures of , both adhering to the Generalized Second Law (a version of the second law formulated to include contributions from black hole entropy) [17].
There is no suitable free energy equation that applies to irreversible processes out of equilibrium. The Gibbs and Helmholtz free energies apply to reversible processes with particular state variables held constant [1]. Exergy is a generalized version of this without constant state variables (Chapter 3 of [18]).
"Available work is the difference between the exergy of states 1 and 2; the exergy of a state being defined as the optimum work achievable against a final state which is one of equilibrium with the surroundings. It is shown that exergy so defined is not a unique quantity for a given system and state, because it also depends on the kind of interaction permitted and therefore on the kind of equilibrium achieved." (p. 679 of [19]).
Therein lies a second hurdle. There are no formal thermodynamic state variables that account for the gravitational field of a system. Since the greatest contribution to entropy increase (i.e., free energy dissipation) is the accretion of matter into black holes, we require some measure of the free energy available from gravitational contraction to understand the work that can be achieved from a transition to gravitational "equilibrium".
Nonetheless, the remaining free energy is available to us over the time it will take the totality of matter to transmute into photons (either through hydrogen burning in stars or black hole evaporation) and for those photons to cool with the expansion of the universe to the de Sitter temperature (Equation (2)) (over timescales ~).

The Maximum Entropy at Heat Death-A Time-Independent Definition
There is no consensus definition for the maximum entropy of the universe [5,[20][21][22][23][24]. Two models have been identified: the entropy at heat death , taken as constant for all t, and a timedependent definition ( ).
The entropy gap between ( ) and the maximum entropy achieved at heat death , is a general measure of available free energy at time t. The maximum has been defined as a constant over all time ( Figure 2). This is endorsed by Penrose [11] who estimates it as close to the entropy of the largest possible black hole with the mass of the observable universe's present matter content. The entropy is expected to increase by a factor of ~/ as black holes evaporate into photons [25]. ( ), as described by Davies [22], considers which processes at a given point in the evolution can contribute to entropy increase. It does not account for the rate at which these processes can increase entropy but does try to account for the total amount of entropy the new entropyincreasing processes could produce. The limits on the rate of entropy production will slow as the expansion of the universe prevents further accretion of mass onto supermassive black holes in thedominated era.

The Availability of Free Energy and Dyson's Hang-Ups
The free energy content measured using , is considered available for consumption over . It is not instantaneously available at any time. Its use depends on cosmic evolution. For instance, one has to wait until inflationary expansion turns off for the resultant distribution of matter to gravitationally collapse and form structures. Hence the free energy from this structure formation is not "available" until inflation ends. However, this is not a strict enough condition on the availability of free energy. Even after structure formation begins, the rate at which the entropy can increase is limited by the virialization time of the over-densities in the early universe. Is the free energy available when the process becomes allowable (i.e., gravitational collapse)? Or is it constrained by the rate at which the process can occur (i.e., virialization)?
Here is another example of this ambiguity. Hydrogen is only able to contribute to entropy increase at high enough temperatures within stars. Prior to star formation, this hydrogen is in a metastable equilibrium until gravitational collapse creates conditions that allow for quantum tunneling and nuclear fusion. Similar to the previous structure formation example, hydrogen fusion is the newly allowed process. However, the rate at which it can contribute to entropy increase is dictated by the star formation rate. From these examples we can see that In other words, at a given time t, if we compute the entropy gap as we cannot use this ( ) as a measure of the free energy available at time t, because the dissipative structures (stars) or the background conditions (e. g. , ≫ ( = )) limit ( ).
Dyson identified features of the universe that delay or limit as "hang-ups" [26]. Limits on structure formation can be construed as a size or density hang-up: the density of matter defines the free fall time required for gravitational collapse. If the density were higher (or the expansion of the universe slower), this free fall time would decrease, and the rate at which gravitational entropy could grow would be higher. Other hang-ups include the spin hang-up-physical limits on the ability to export angular momentum limit the collapse of large, spinning structures. The thermonuclear hangup ensures there is a period of hydrogen burning (in stars) that prevents further gravitational collapse, limiting the growth of gravitational entropy.
Hang-ups describe limits on the rate of entropy increase in some important cosmological contexts. The idea of treating the constraints of fundamental physics as hang-ups to some more efficient contra-factual entropy production is not obvious. If Dyson's hang-ups limit the rate of entropy increase, what rate are we to compare it to? Should it be compared to instantaneous dissipation?

Davies'
( ) According to Davies [22], an entropy gap opens in the early universe as constraints on the maximum entropy change (Figure 2). Davies suggests ( ) increases after the end of inflation once structure formation becomes possible, and increases more quickly than , thereby generating an entropy gap where previously (during inflation) the universe was evolving at maximum entropy [15]. Even though the second law demands a monotonic increase in following inflation [10,27], this does not have to mean the remaining free energy available decreases. If the growth of ( ) exceeds the growth of ( ), additional free energy becomes available due to the processes that could not previously occur (i.e., parts of the phase space that were not accessible) [11,20]. During inflation, the universe remains at its maximum possible ( ): a de Sitter universe with large . When inflation ends and particles are distributed homogeneously, able to gravitationally collapse, Davies suggests an entropy gap opens between the potential gravitational entropy of matter and the actual gravitational entropy of matter at the time. The jump from a state of equilibrium during inflation to one out of equilibrium following inflation is explained by the nonconservation of energy since the energy density of the vacuum does not decrease as the universe inflates. This non-diluted energy is then dumped into the universe as matter and radiation during reheating.
According to Davies, the maximum entropy is set only for a particular energy-if this total energy changes the maximum can then change. In this case, the additional energy comes from the inflaton field driving inflation which generates a tremendous potential entropy in matter but reduces the entropy in gravitational degrees of freedom. The total entropy does not decrease but is "out of step" with the maximum. The universe is now able to increase its gravitational entropy , as per the Penrose picture [11,28]. Collapsing matter increases , which reaches a maximum when black holes form. The total entropy increases once a black hole evaporates, but the gravitational entropy decreases. Accounting for "frozen out" degrees of freedom which do not contribute to the increase in entropy, Davies accounts for the frozen out gravitational degrees of freedom during inflation which become available after reheating, raising the (time-dependent) ceiling (i. e. , ) on entropy increase. Davies accounts for some of the entropy-increasing processes becoming physically realizable at a particular time, but not the realistic time frames over which they can actually be realized. He is ignoring , and defining ( ) by considering only the "allowable" process and not how it relates to . Citing Frautschi's model [12], Davies may be considering gravitational collapse within a causal region of size ~, or the transmutation of matter to CMB blackbody radiation [11,29]. Regardless, Davies' ( ) assumes the maximum entropy state can be achieved instantaneously once a given process becomes "entropically favorable" [22]. Davies suggests a second gap opens when primordial hydrogen previously able to maintain thermal equilibrium cools with the expansion of the universe. It becomes favorable for the hydrogen to synthesize into the more stable (higher entropy) iron, raising the ceiling on ( ). The same assumption of instantaneous entropy production applies-the time scales involved in the formation of stars and hydrogen burning are not considered important. As soon as the potential increase in entropy becomes physically possible, the entropy gap is open.

Time-Dependent Maximum Entropy vs. Time-Dependent Entropy Production Rate
The maximum entropy at heat death is a mathematically accurate representation of the largecoarse-grained volume of the universe's phase space. However, it provides no information on how the universe will dynamically evolve to reach maximum entropy. Davies attempts to account for parts of this evolution by invoking a time-dependent maximum entropy that increases once particular dissipative processes are possible. If the entropy gap is a good measure of free energy, this suggests the free energy available from a given process over the time it takes for the process to occur is said to be "available" as soon as the process begins.
It is more accurate to say that the available free energy depends on constraints on the slope rather than on a time-dependent ( ) with no consideration of . A consequence of this approach is then, if in a universe with life that consumes free energy earlier and more quickly than one without, is the maximum value of different in these two universes? Moreover, waiting to use the available free energy at later times when the background temperature is closer to the de Sitter temperature (i.e., a lower equilibrium exhaust temperature) will allow for a greater proportion of the available free energy to be used: "Whenever a system operates irreversibly, it destroys work at a rate that is proportional to the system's rate of entropy generation." [30].
The constant of proportionality in this case is the exhaust temperature T, with the rate at which work is lost is given by If life can slow the rate of and wait for T to drop to lower values, it can minimize the lost work (Equation (6)) and maximize the amount of work achievable over [30].

Decoupled Evolution of Gravitationally-Dominated Systems
Following inflation and reheating, gravitational entropy can increase while constrained by Dyson's hang-ups until structure formation turns off during the -dominated phase of a − universe. The exponential expansion prevents uncollapsed matter from contracting, limiting increase in gravitational entropy. The collapsed matter is effectively decoupled from the expansion and can continue to increase internally [31]. These structures evolve independently of the expansion. Ordinarily, collisions between bound structures produce larger structures that further increase gravitational entropy. This resembles the interaction rate between particles that maintain equilibrium during expansion. During -domination, these collisions become increasingly rare, and hence bound structures can internally increase their (local gravitational entropy), but cannot interact externally with other bound objects to increase globally. Here, as with inflation, gravitational collapse turns off when is turned on (Figure 3).

Limits on Structure Growth as a Constraint on
-dominated expansion may represent a freeze-out for gravitationally bound structures, which could be viewed as "degrees of freedom" that could (during earlier epochs) "equilibriate" gravitationally (i.e., achieve a maximum gravitational entropy), increasing the total gravitational entropy. For relativistic particles at equilibrium, the expansion of the universe is isentropic. If the expansion rate overtakes the interaction rate between degrees of freedom, they decouple and can no longer maintain equilibrium [11]. Gravitationally bound objects (e.g., galaxy clusters), treated as individual degrees of freedom, cannot reach maximum gravitational entropy because the expansion rate overtakes their interaction rate (defined by the frequency of collisions between bound structures).
Thus, larger objects can no longer form and structure growth turns off in a -dominated phase of a − universe-a restriction on the rate of entropy increase (Figure 3) [32]. The acceleration of the expansion of the universe shuts off the flow of accreting matter into larger and larger objects and this sets a limit on the size (and thus entropy) of the largest black holes. The entropy must then increase via evaporation, which is a much slower process.

Conclusions
The entropy gap is a measure of the amount of available free energy. However, not all free energy is available at all times. The closing of the entropy gap depends on cosmic evolution and the dissipative structures that are produced. Two definitions of the maximum entropy (and by extension ) have been identified. The first is a constant defined as the entropy at heat death , . The second is time-dependent ( ), suggested by Davies, which accounts for changes in constraints (e.g., the new potential for gravitational collapse following inflation) that increase the ceiling on the maximum entropy at a given time. Davies neglects constraints on the rate of entropy increase and assumes the entropy gap can be closed instantaneously. We suggest the calculation should include limits on the rate of entropy growth . This removes the need for an ( ) since constraints on ( ) capture which processes are allowable in the universe as well as the rate at which these generate entropy (i.e., decrease free energy). One such constraint is the turn-off of structure formation during -domination, thus limiting the growth of gravitational entropy.