Next Article in Journal
Generalized Beta Distribution of the Second Kind for Flood Frequency Analysis
Next Article in Special Issue
The Mehler-Fock Transform in Signal Processing
Previous Article in Journal
Kullback–Leibler Divergence and Mutual Information of Partitions in Product MV Algebras
Previous Article in Special Issue
Projection to Mixture Families and Rate-Distortion Bounds with Power Distortion Measures

Entropy 2017, 19(6), 268; https://doi.org/10.3390/e19060268

Article
Information Geometry of Non-Equilibrium Processes in a Bistable System with a Cubic Damping
1
Department of Applied Mathematics, University of Leeds, Leeds LS2 9JT, UK
2
School of Mathematics and Statistics, University of Sheffield, Sheffield S3 7RH, UK
*
Author to whom correspondence should be addressed.
Academic Editor: Geert Verdoolaege
Received: 29 April 2017 / Accepted: 6 June 2017 / Published: 11 June 2017

Abstract

:
A probabilistic description is essential for understanding the dynamics of stochastic systems far from equilibrium, given uncertainty inherent in the systems. To compare different Probability Density Functions (PDFs), it is extremely useful to quantify the difference among different PDFs by assigning an appropriate metric to probability such that the distance increases with the difference between the two PDFs. This metric structure then provides a key link between stochastic systems and information geometry. For a non-equilibrium process, we define an infinitesimal distance at any time by comparing two PDFs at times infinitesimally apart and sum these distances in time. The total distance along the trajectory of the system quantifies the total number of different states that the system undergoes in time and is called the information length. By using this concept, we investigate the information geometry of non-equilibrium processes involved in disorder-order transitions between the critical and subcritical states in a bistable system. Specifically, we compute time-dependent PDFs, information length, the rate of change in information length, entropy change and Fisher information in disorder-to-order and order-to-disorder transitions and discuss similarities and disparities between the two transitions. In particular, we show that the total information length in order-to-disorder transition is much larger than that in disorder-to-order transition and elucidate the link to the drastically different evolution of entropy in both transitions. We also provide the comparison of the results with those in the case of the transition between the subcritical and supercritical states and discuss implications for fitness.
Keywords:
stochastic processes; Fokker–Planck equation; fluctuations and noise; non-equilibrium statistical mechanics
PACS:
05.70.Ln; 05.40.-a; 05.90.+m

1. Introduction

The spontaneous emergence of order out of disorder is one of the most fascinating phenomena in nature and laboratory experiments, attracting ever-increasing interest. Important examples include phase transition/critical phenomena in cosmology, elementary particle theory, condensed matter, chemistry, biology and social-economic movement [1,2,3,4,5]. Order is usually quantified by a non-zero value of a macroscopic observable (global mode). While triggered by an external parameter (such as temperature) or spontaneously, a macroscopic observable often does not simply evolve passively, but undergoes an indispensable interaction with fluctuations (microscopic variables). The self-regulation between macroscopic and microscopic variables leads to a dynamical equilibrium (self-organisation), which involves fluctuations as an essential part [6]. There has been accumulating evidence for relevance and important role of self-organisation in different systems such as shear flows or vortices in fluids or plasmas, pattern formation in chemical oscillators, homoeostasis in biosystems and even traffic flows [4,7,8,9,10,11,12,13,14,15,16]. In particular, self-organised shear (zonal) flows are now believed to play a crucial role in stabilising laboratory plasmas, beneficial for extracting fusion energy [7]. Due to large fluctuations involved in order-disorder transition or self-organising systems, it is essential to use statistical tools to describe these systems.
The aim of this paper is to understand order-disorder transition from the perspective of information change associated with transition and uncover a geometrical structure in a statistical space, which can be utilised to understand ever-increasing experimental/observational data. To this end, we investigate a bistable stochastic system that is often invoked as a canonical model of self-regulating systems, e.g., in electric circuits [17], in various cellular processes such as cycles, differentiation and apoptosis, regulation of heart, brain, etc. [18,19,20,21,22,23]. In this model, we calculate time-dependent Probability Density Function (PDF) and the total number of statistically different states that the system undergoes in time. The latter is defined by the dimensionless information length [24,25,26,27,28] (see Appendix A):
L ( t ) = 0 t d t 1 τ ( t 1 ) = 0 t d t 1 d x 1 p ( x , t 1 ) p ( x , t 1 ) t 1 2 ,
where p ( x , t ) is a time-dependent PDF for a stochastic variable x. In Equation (1), τ ( t ) is the time-varying “time-unit”:
E ( t ) 1 [ τ ( t ) ] 2 = d x 1 p ( x , t ) p ( x , t ) t 2 .
τ ( t ) in Equation (2) has dimensions of time and quantifies the correlation time over which the (dimensionless) information changes, thereby serving as the time unit for information change. Note that in equilibrium where p t = 0 , τ . Measuring the total elapsed time in units of τ between time t = 0 and t gives the information length in Equation (1). The latter thus establishes a distance between the initial and final PDFs in the statistical space.
We note that our information length is based on Fisher information (cf. [29]) and is a generalisation of statistical distance mainly used in equilibrium or near-equilibrium systems [30,31,32,33,34,35,36,37,38,39] to non-equilibrium systems [24,25,26,27,28]. In particular, the linear geometry of a linear Ornstein-Uhlenbeck (O-U) process was captured by the linear relation L x 0 , while the power-law geometry of a nonlinear (cubic) stochastic process was revealed by power-law scalings L x 0 n ( n 1.5–1.9) [28], where x 0 is the peak position of an initial narrow PDF. Furthermore, interesting geodesic solutions were found [27] by time-periodic modulation of the model parameters in an O-U process, which by itself does not support a geodesic solution without the modulation of parameters. As a geodesic is a unique path along which a system undergoes the minimum number of changes in the statistical states given the initial and final conditions, it is beneficial to a system where the change is costly. It is thus important to elucidate the key characteristics of a stochastic system that permits or facilitates the existence of a geodesic in general (without the modulation of model parameters). Finally, we emphasise that Equation (1) can be applied to any data; [25] constructed time-dependent PDFs from MIDI files of music and elucidated the information change in music by L and τ .
In this paper, in order to gain a key insight, we focus on a zero-dimensional (0D) model, which has one control parameter, and propose an on-quenching experiment by a sudden change of a control parameter from the critical to subcritical and from the subcritical to critical values to trigger disorder-to-order and order-to-disorder transitions, respectively. A pair of disorder-to-order and order-to-disorder transitions with the suitable choice of initial conditions then provides a simple model in which a continuous switch between ordered and disordered states can be studied in great detail. Each pair of disorder-to-order and order-to-disorder transitions models a burst (e.g., a burst in the gene expression consisting of a pair of induction and repression). Since an initial condition represents the “resting” state of a stochastic system in between the two bursts, it is important to understand the effect of different initial conditions on information change. In particular, we aim to elucidate what might be an optimal initial “resting” state that minimizes the information change, sustaining a robust geodesic solution. To this end, we provide detailed comparison of on-quenching processes in this paper with off-quenching processes, where the control parameter changes between subcritical and supercritical, reported in [40]. While some aspects are similar between the two processes, there are also important differences that will be presented and discussed here. The remainder of this paper is organized as follows. Section 2 presents our model, and Section 3 provides details on the time evolution of PDFs. We discuss information length in Section 4 and differential entropy and Fisher information in Section 5. We conclude in Section 6. Appendices contain the derivation of equations used in the main text.

2. Models

We consider the following 0D Ginzburg-Landau model [41] for a stochastic variable x:
d x d t = F ( x ) + ξ = - λ x - μ x 3 + ξ .
Here, F ( x ) = - λ x - μ x 3 is a deterministic force; ξ is a white noise with the following statistical property:
ξ ( t ) ξ ( t ) = 2 D δ ( t - t ) ,
where D is the strength of the forcing, and the angular brackets denote the average over ξ . In our study, λ is a control parameter. In the numerical computations, we will fix the value of μ ( = 1 ) with no loss of generality, while keeping track of μ in analytical calculations for clarity. λ represents the deviation from the critical value (e.g., λ T - T c for the temperature T where T c is the critical temperature). That is, our system is subcritical for λ < 0 , supercritical for λ > 0 and critical at λ = 0 . Equation (3) is the extension of our recent work [28,42] to a cyclic transition. The Fokker–Planck equation [43,44] corresponding to Equations (3) and (4) is as follows:
t p ( x , t ) = x - F ( x ) + D x p ( x , t ) .
In this paper, we consider the transition between the critical state and subcritical state by changing λ between zero and - γ ( γ > 0 ). Here, we note that we are using γ > 0 to explicitly represent a positive growth rate in the subcritical state λ < 0 . To this end, we induce the phase transition from the initial state λ = 0 by a sudden change in λ . This mimics the “on-quenching” experiment where the quenching occurs at the critical state; disorder-to-order transition (order-to-disorder transition) represents the transition from the critical to subcritical (the subcritical to critical) states. This thus contrasts to the case of the transition from the supercritical to subcritical state studied in [40] where the quenching occurs off the critical state.
Specifically, our model is described as follows:
  • Forward Process (FP): λ = - γ < 0 : at t = 0 , a unimodal PDF with a peak at x = 0 , which evolves into a bimodal PDF with peaks at x = ± γ / μ 0 as t ;
  • Backward Process (BP): λ = 0 : at t = 0 , a bimodal PDF with peaks at x = ± γ / μ 0 , which evolves into a unimodal PDF with a peak at x = 0 as t .
FP and BP have the following stationary distributions p F ( x ) and p B ( x ) , respectively:
p F ( x ) e - μ 4 D ( x 2 - γ μ ) 2 ,
p B ( x ) e - μ 4 D x 4 .
That is, FP has a stationary bimodal distribution peaked at x = ± γ μ , which can be approximated as a double Gaussian for small D as follows:
p F ( x ) β F 2 π e - β F x + γ μ 2 + e - β F x - γ μ 2 ,
where β F = γ D . Equation (8) represents the sum of the two Gaussians (double Gaussian) with the peak at ± γ μ and variance σ F = x 2 - x 2 = D 2 γ = 1 2 β F . In comparison, BP has a unimodal quartic exponential PDF centred around x = 0 in equilibrium. To model a cyclic transition between ordered and disordered states, we use p B ( x ) as an initial distribution for FP and p F ( x ) for BP, respectively. Consequently, the initial PDFs in both FP and BP are strongly out of equilibrium. We investigate time-dependent PDFs and information length during this transient relaxation. In particular, we are interested in how L depends on the deviation from the critical value ( γ ) and the strength of the stochastic noise (D). Table 1 summarizes the value of λ in Equation (3) and initial conditions for FP and BP, together with the variance σ = x 2 - x 2 of the initial and final PDFs, where the angular brackets denote the average over the stochastic noise ξ . We note that for small D, the equilibrium variance for FP, D / 2 γ , is much smaller than that for BP, 2 D / μ Γ 3 4 / Γ 1 4 .

3. Time-Evolution of PDFs

To solve Equation (5) numerically, we note first that γ , D and t can always be rescaled such that μ = 1 , thereby reducing the number of parameters that need to be explored to only γ and D. Any numerical solution also requires x to be restricted to a finite interval, which can always be rescaled to x [ - 1 , 1 ] without any loss of generality. If γ and D are chosen such that p F ( x ) is restricted well away from the boundaries | x | = 1 , then this finite interval in x is an excellent match to the analytically more convenient infinite extent. Taking γ 0 . 7 and D 10 - 3 ensures that the bimodal peaks at x = ± γ are still sufficiently far from the boundaries, and sufficiently narrow, that p = 0 can simply be imposed as the boundary condition at x = ± 1 . The details of the numerical implementation then involve second-order finite-differencing in x and t, using up to O ( 10 6 ) grid-points in space, and time-steps as small as O ( 10 - 7 ) . This spatial resolution allows D to be reduced down to 10 - 7 while still fully resolving the bimodal peaks of width O ( D 1 / 2 ) .

3.1. Overall Comparison of FP and BP

One of the most significant differences between FP and BP is the time scale on which the process evolves and settles in. There are many diagnostic quantities that could be used to quantify this, but a useful one is simply the ratio x 4 / x 2 . This can be evaluated analytically for the two end states, yielding 1.48 for p B ( x ) and 1 for p F ( x ) (taking D 1 ). The evolution in time must therefore be that FP is 1.48 1 and BP is 1 1.48 , and the question is on what time scales this happens.
As shown in Figure 1a, for FP the dependence on D is such that every reduction in D by a factor of 100 shifts the curves by a constant amount in t. That is, the ratio does not deviate significantly from 1.48 until a time c ln D - 1 has elapsed, but except for this shift the three curves are essentially identical. The numerically determined value of c is 0.355 and is very close to the factor of 1 4 γ = 0.357 in t 2 in Equation (23), discussed later. It is interesting that this value of c is exactly half of that in [40], but otherwise, the scaling with D is the same. The reason for the faster adjustment in this case is because the initial condition already starts out much broader, with a width O ( D 1 / 4 ) here, as opposed to O ( D 1 / 2 ) in [40]. We recall that x = 0 is an unstable equilibrium point when ξ = 0 , and the instability slowly builds up due to ξ (e.g., see [45,46]) and a finite width of the initial PDF until t O ( | ln D | ) . If the initial condition is already broader, then it is not surprising that the instability can develop sooner. However, this factor of half in the settling time does not mean that the evolution of PDFs in on- and off-quenching processes has any similarity. In fact, we will show that they are quite different and that the off-quenching cannot simply be made up of the two phases where effectively λ = γ 0 and λ = 0 - γ (as for the on-quenching case here).
Figure 1b shows that for BP the dependence on D is very different, with every reduction in D by a factor of 100 shifting the curves by a factor of 10; that is, time scales as D - 1 / 2 . The backward process is initially driven by the movement of the two peaks towards x = 0 before diffusion becomes crucial in forming a single peak at x = 0 ; it is this final diffusive adjustment process that requires an O ( D - 1 / 2 ) time to happen. This is in sharp contrast with the results in [40], where FP and BP both exhibited the same c ln D - 1 scaling (and even with the same value of c). In comparison with the off-quenching in [40], the on-quenching considered here thus has a forward process that is faster by a factor of two, but a backward process that is much slower, with a completely different D - 1 / 2 as opposed to ln D - 1 scaling.

3.2. PDF of Forward Process

Figure 2 shows the structure of p ( x , t ) for FP. The particular times are chosen to take the shift c ln D - 1 into account; that is, different values of D are shown at the times where they have the same ratios x 4 / x 2 . The results are seen to be identical for the three different values of D. The initial condition obviously does depend on D (as indicated also by the red curves), but once a certain broadening has occurred, in this time frame c ln D - 1 , the subsequent evolution is independent of D, relying only on the instability process (as measured by γ ). It is only in the last stages of the evolution (not shown in Figure 2), when the solution settles in to the final bimodal structure, that diffusion plays a role again and determines the O ( D 1 / 2 ) width of the peaks. Comparing with results in [40], it is interesting to note also that here a double-peak emerges essentially immediately, whereas in [40] a finite time had to elapse before the central peak split into two separate peaks. The reason for this is that here the initial condition is at a critical state with a much broader profile than the Gaussian profile at the supercritical state considered in [40].
To understand these results better, it is of value to perform analytical analysis in the limiting cases. To this end, we transform the nonlinear term in Equation (3) into a linear (anti)damping term [47] by seeking a variable y such that Equation (3) becomes d y / d t = γ y + ξ F ( y ) where F ( y ) is a function of y. We then easily show that d y / d x = γ y / ( γ x - μ x 3 ) which has the solution x = y / 1 + α y 2 ( α = μ γ ). Specifically, y satisfies:
d y d t = γ y + ξ ( 1 + α y 2 ) 3 2 .
Equation (9) provides a convenient way of computing the PDF of x through y by approximating ξ ( 1 + α y 2 ) 3 2 ξ for small y [45]. Thus, to leading order y is a Gaussian process, simply given by the Ornstein-Uhlenbeck process [44] with a negative damping. The transition probability is thus Gaussian:
p ( y , t ; y 0 , 0 ) = β 1 π e - β 1 y - y 0 2 ,
where y 0 = x 0 1 - α x 0 2 and 1 β 1 ( t ) = D γ ( e 2 γ t - 1 ) .
By using the conservation of the probability p ( x , t ) d x = p ( y , t ) d y and Equation (10), we obtain the transition probability of x as follows:
p ( x , t ; x 0 , 0 ) = 1 ( 1 - α x 2 ) 3 2 β 1 π exp - β 1 x 1 - α x 2 - x 0 1 - α x 0 2 e γ t 2 ,
which recovers the previous results [45,46] when α = 1 and x 0 = 0 .
To obtain a (marginal) PDF of x, we recall that FP has an initial PDF given by a quartic exponential:
p ( x 0 , 0 ) = M β 0 1 4 e - β 0 x 0 4 .
Here, β 0 = μ 4 D ; M = 2 Γ 1 4 - 1 is a normalisation constant where Γ ( x ) is the Gamma function. Equations (10) and (12) then give:
p ( x , t ) = M β 0 1 4 β 1 π - d x 0 1 ( 1 - α x 2 ) 3 2 exp - β 1 x 1 - α x 2 - x 0 1 - α x 0 2 e γ t 2 e - β 0 x 0 4 .
Here, 1 β 1 ( t ) = D γ ( e 2 γ t - 1 ) .
We now show that the initial quartic exponential PDF (12) undergoes roughly two stages of the time evolution: stage (i) driven by diffusion/advection with the continuous movement of the PDF peak from x = 0 towards γ μ , and then stage (ii) of settling into an equilibrium PDF with the adjustment of the PDF shape. To this end, we examine the behaviour of p ( x , t ) in Equation (13) for a sufficiently large β 1 such that:
β 1 = γ D ( e 2 γ t - 1 ) 1 .
Equation (14) will later be shown to be valid in stage (i) (e.g., for t < t 2 where t 2 is defined in Equation (23)). By using Equation (14), we can approximate the exponential function in Equation (13):
β 1 π exp - β 1 x 1 - α x 2 - x 0 1 - α x 0 2 e γ t 2 δ x 1 - α x 2 - x 0 1 - α x 0 2 e γ t = e - γ t ( 1 - α x 2 ) 3 2 ( 1 - α ¯ x 2 ) - 3 2 δ x 0 - x e - γ t 1 - α ¯ x 2 ,
where:
α ¯ α ( 1 - e - 2 γ t ) .
Then, using Equation (15) in Equation (13) gives us:
p ( x , t ) M β 0 1 4 e - γ t ( 1 - α ¯ x 2 ) 3 2 e - β 0 e - 4 γ t x 2 1 - α ¯ x 2 2 M β 0 1 4 e - γ t e - ψ ,
where ψ is defined by
ψ = β 0 e - 4 γ t x 2 1 - α ¯ x 2 2 + 3 2 ln ( 1 - α ¯ x 2 ) .
By using α ¯ = 0 at t = 0 , we can easily show that Equation (17) matches the initial PDF p ( x , 0 ) at t = 0 in Equation (12). From ψ x | x = x 1 , 2 = 0 , we find the value of x = x 1 , 2 where the PDF takes its local maximum or minimum:
x 1 = 0 , 4 β 0 e - 4 γ t x 2 2 = 3 α ¯ ( 1 - α ¯ x 2 2 ) 2 .
Since x x ln p ( x , t ) = - x x ψ > 0 at x = 0 , x = x 1 = 0 is a local minimum of p ( x , t ) for all t > 0 . On the other hand, two values of x 2 (note α ¯ x 2 2 < 1 ) represent the location of the local maximum in x > 0 and x < 0 , respectively. Thus, x = x 1 = 0 , the (global) maximum at t = 0 , becomes a local minimum for any infinitesimal time t > 0 , two peaks forming at x 2 . For instance, for α ¯ x 2 2 1 , we can show that:
x 2 2 3 D γ e 4 γ t - e 2 γ t .
This reveals the diffusive nature of the peak movement from x = 0 due to instability γ towards the equilibrium value ± γ μ . In the limit of a very small time t 1 4 γ , Equation (20) gives x 2 ± 6 D t by using e 2 γ t = 1 + 2 γ t + and β 0 = μ 4 D , showing that the initial movement of the two peaks is via random walk. Our numerical solutions confirmed the predicted scaling of x 2 D 1 / 2 in Equation (20), as well as x 2 D t for small time, followed by almost exponential increase (no figure is shown here).
To examine the evolution in more detail, we consider the characteristic time t 2 where the width of the PDF in Equation (17) becomes comparable to the peak position x 2 in Equation (20). To estimate the PDF width R in Equation (17), we use the variance of the quartic exponential function (e.g., [42]):
R 2 = Γ 3 4 Γ 1 4 1 β 0 e - 4 γ t ,
for α ¯ x 2 1 . By using β 0 = μ / 4 D , we simplify Equation (21) as:
R 2 3 e 2 γ t 4 D μ 1 / 2 .
By equating R in Equation (22) and x 2 in Equation (20) and using e 4 γ t - e 2 γ t e 4 γ t for γ t 1 , we find the characteristic time t 2 as follows:
t 2 1 4 γ ln 4 γ 2 μ D ,
where β 0 = μ 4 D was used. For γ = 0.7 and D = 10 - 3 , 10 - 5 , 10 - 7 , t 2 = 2.7 , 4.35 , 6.0 . Notably, the value of t 2 will later be shown to be very close to the other time scales t m signifying order formation. t 2 in Equation (23) marks the time when the peak position becomes comparable to the rms (Gaussian) fluctuation. For t t 2 , PDF in settling into the final equilibrium PDF is approximated by the Gaussian [45]. We have confirmed this prediction from our numerical solutions (as discussed in more detail later). Finally, we have checked that Equation (14) is valid for t 2 given in Equation (23).

3.3. PDF of Backward Process

We recall that BP starts with a bimodal PDF p F , which has two peaks at ± γ / μ , which is the final equilibrium PDF of FP. For sufficiently small D, the distance between these two peaks is much larger than the width of the PDFs and are thus well separated so that PDF is approximated as the sum of the two Gaussian (double Gaussian) PDFs given by Equation (8). The latter evolve almost independently in x > 0 and x < 0 , respectively, until t O ( 1 ) when PDFs undergo significant change in the shape with large fluctuation. Since the Gaussian evolution is completely determined by mean value and variance, we now compute mean value and variance in x > 0 or x < 0 separately by taking advantage of small fluctuations compared to mean value. Specifically, we let x = z + δ x where z = x is the mean component averaged over ξ and the initial PDF in x > 0 (or x < 0 ) while δ x is the fluctuation δ x = 0 . This gives us:
d z d t = - μ z 3 - 3 μ ( δ x ) 2 - μ z 3 ,
d σ d t = - 6 μ z 2 σ + 2 D .
The solutions to Equations (24) and (25) are as follows (c.f. [48]):
z ( t ) = z 0 ( 1 + 2 μ z 0 2 t ) 1 / 2 , σ ( t ) = σ 0 G ( t ) + 2 D G ( t ) G ( t ) ,
where G ( t ) = ( 1 + 2 μ z 0 2 t ) 3 and G = 0 t G ( t ) d t , z 0 = γ μ and σ 0 = D 2 γ (see Equation (8) and Table 1). Equations (25) and (26) will be used for computing L in the next section.
We show p ( x , t ) for BP in Figure 3. We choose the particular times again to take the D - 1 / 2 scaling into account, and show results at the times where they have the same ratios x 4 / x 2 . The initial evolution (not shown in Figure 3) consists simply of a motion toward the origin, with the width of the peaks remaining O ( D 1 / 2 ) . The scaling of this movement is found to be consistent with the prediction in Equation (26).
Once the peaks get within a distance D 1 / 4 of the origin they start to sense the presence of the potential well, and diffusion starts to collapse them to a single peak. As seen in Figure 3, if x is rescaled as D 1 / 4 , and p correspondingly rescaled as D - 1 / 4 , then the results again look the same for all three values of D. This final diffusive adjustment to the single central peak is very slow though, resulting in the D - 1 / 2 scaling in time.

3.4. Energy Diagnostics

We now elucidate the role of the linear growth term (positive feedback) and cubic damping (negative feedback) in FP in energy balance and geodesic. To this end, we multiply Equation (3) by x and take the average over ξ and initial condition to obtain the following equation:
1 2 d x 2 d t = γ x 2 - μ x 4 + D .
Here, the last term D, representing the rate of energy injection by ξ , was calculated as ξ ( t ) x ( t ) = ξ ( t ) 0 t d t 1 [ γ x ( t 1 ) - μ x ( t 1 ) 3 + ξ ( t 1 ) ] = D (also confirmed by the numerical calculations). The middle term γ x 2 - μ x 4 H represents the energy into the system or environment, depending on the sign. When H > 0 , the energy goes into the system, contributing to the increase in x 2 ; when H < 0 , the energy is dissipated in the system, increasing heat in the environment.
Figure 4a shows H = γ x 2 - μ x 4 for D = 10 - 3 , 10 - 5 , 10 - 7 . Unlike x 2 and x 4 , which each increase monotonically in time, H reaches a peak at some time t = t m , and then decreases to the negative value - D in settling in to the equilibrium PDF. The maximum H signifies when the positive feedback by the linear growth rate most dominates over the negative feedback by the nonlinear damping. It is notable that the times t m = 2.6 , 4.25 , 5.9 in Figure 4a for the maximum H are similar to the times t 2 = 2.7 , 4.35 , 6.0 given in Equation (23), with both exhibiting the same c ln D - 1 ( c = 1 / 4 γ ) scaling. t m will also be shown to be very close to the time for the maximum entropy in Section 5.
Physically, t m t 2 signifies the start of order formation. Another diagnostic for the latter is Δ = x 4 - x 2 , also shown in Figure 4b, where similar non-monotonic behaviour is prominent, with Δ peaking at the same times as H. This large fluctuation Δ signifies the phase transition from disordered to ordered states due to the development of the two peaks, which occurs on time scales increasing with c ln D - 1 as discussed above.
For BP, H = - x 4 and Δ in Figure 4c,d are monotonic during the return to the disordered state. The monotonic evolution of H and Δ for BP is also reflected in the evolution of the differential entropy in Section 5.

4. Information Length

We calculate information length in Equation (1) and explore geometric structure during phase transition. Figure 5 and Figure 6, for FP and BP respectively, show how E and L evolve in time, as well as how the total L ( t ) = L depends on γ . Since FP and BP switch between λ = 0 and - γ 0 , γ in Figure 5 and Figure 6 always refer to the non-zero value. We are especially interested also in comparing the on-quenching results computed here with the previous off-quenching results from [40], shown as the dashed lines in Figure 5 and Figure 6.

4.1. Forward Process

For FP, it is useful to consider times less than or greater than t 2 in Equation (23) separately, by approximating the time-dependent PDF as a quartic exponential and Gaussian in t < t 2 and t > t 2 , respectively. First, for t < t 2 , by ignoring the contribution from the mean value y = z compared with that from the variance, we obtain τ ( t ) in Equation (2) (see Appendix B):
1 [ τ ( t ) ] 2 1 4 β ( t ) 2 d β d t 2 .
Equations (2), (23) and (28) then give L between the time t = 0 and t t 2 :
L ( t 2 ) 1 2 ln β 0 e - 4 γ t 2 β 0 1 2 ln 3 μ D γ 2 = 1 2 ln γ 2 3 μ D .
On the other hand, during the time between t t 2 and t , the PDF settles into the double Gaussians so that we can estimate the total L between t t 2 and t by using Equation (A13) (see Appendix C) as:
L - L ( t t 2 ) 1 2 ln σ ( t 2 ) σ ( t ) 1 2 ln 2 γ 2 Γ 3 4 3 μ D Γ 1 4 ,
where σ ( t 2 ) = Γ 3 4 Γ 1 4 1 2 β 0 e - 4 γ t 2 = Γ 3 4 Γ 1 4 4 γ 2 3 μ 2 for FP (see above), α = μ γ and σ ( t ) = D 2 γ were used. Equations (29) and (30) have the same dependence on D, μ and γ . The sum of Equations (29) and (30) gives the total:
L - 1.2 + 2 + 1 2 ln γ 2 D ,
when μ = 1 and numerical values for the Γ functions are inserted.
Figure 5 shows the numerically computed E and L for FP. We see how E starts out essentially constant, corresponding to a geodesic solution [27]. This constant plateau continues until the O ( | ln D | ) equilibration time scale previously also seen in Figure 1. After this time E decreases exponentially. Comparing E here with the previous off-quenching results, we notice three differences: (a) the previous initial adjustment before the plateau regime is absent here, and the curves are essentially flat from the initial condition onward; (b) the plateau here is higher than before; (c) the equilibration and, hence, the exponential decrease in E , happen sooner.
Turning to L next, the combination that the plateau is higher, but ends sooner, has the interesting consequence that initially L is greater than in the off-quenching case, but the final values L are always lower. Figure 5c shows the variation of L with γ , and the same pattern persists throughout; L is consistently 1 less than before, with the resulting best-fit formula:
L 2.1 ln γ - 1.05 ln D .
The coefficients of ln γ and ln D are both in generally good agreement with the analytic predictions from Equation (31), which has 2.4 and - 1.2 . The constant terms, zero versus - 1.2 , match less well, but this term is also strongly affected by the best fit to the ln D term, since, e.g., | ln 10 - 7 | = 16 is already as large as the largest L values. (Note finally that the deviation from straight lines for large D and small γ has the same origin as before in [40]: the “initial” and “final” states are then so broad (large D) and so close to each other (small γ ) that they overlap, causing the dynamics to be different, but also not very interesting in this regime.)

4.2. Backward Process

Figure 6 shows corresponding results for BP. E now starts off lower than in the off-quenching case, but the final equilibration is much slower, again as seen previously in Figure 1. The result of the initially smaller E is that for small times L is a factor of two less than in the off-quenching case. See also Equation (33) below, which confirms this analytically. Because the equilibration is so slow though, there is an additional contribution to L that is not present before. Curiously, this seems to result in the final L values always being a factor of 1.5 less than in the off-quenching case. The precise origin of this particular factor, or indeed why it is always the same, independent of D, is not fully understood. As seen in Figure 6c, the results are summarized by the formula L 0.9 γ D - 1 / 2 .
To quantify this scaling, we use Equation (A13) with Equation (A24) and z 0 :
L 0 d t 1 σ d z d t c 1 ( z 0 ) 0 d t σ 0 + 2 D G μ z 0 3 0 1 2 γ d t σ 0 + 2 D t 3 - 1 2 γ μ D ,
where σ 0 = D 2 γ for BP (see Table 1) was used. The variation with γ and D is exactly as in the numerical results, whereas the constant factor is an under-estimate, 0.5 versus 0.9. Given that Equation (33) only represents the early-time contribution to L though, we would expect the true L to be larger.

5. Differential Entropy and Fisher Information

Entropy is most commonly used to describe complexity. In a continuous system, it is given by the (Gibbs) differential entropy (e.g., see [49]) defined by:
S ( t ) = - d x p ( x , t ) ln p ( x , t ) .
Here, we use units in which the Boltzmann constant K B = 1 . Unlike the usual entropy, the absolute value of the differential entropy does not have a physical meaning, only the difference between two values of the differential entropy being meaningful.
To elucidate the difference in S between the critical and subcritical states, we use equilibrium PDFs of FP and BP ( p F and p B in Equations (7) and (8), respectively) and quantify the difference between S ( t = 0 ) and S ( t ) in FP and BP. For the equilibrium of FP p F in Equation (8), we can show that the entropy Equation (34) takes the following form [49]:
S F = 1 2 1 + ln π β F + 2 β F x 0 2 1 - e r f ( β F x 0 ) - β F π 2 x 0 e - β F x 0 2 + Δ .
Here, e r f ( x ) = 2 π 0 x d u exp ( - u 2 ) is the error function; β F = γ / D ; Δ is a function of β F and x 0 , taking the value 0 Δ ln 2 . For a sufficiently narrow PDF with β F x 0 2 1 , Δ takes the maximum value ln 2 (see [49]). Since in this limit β F x 0 2 1 , e r f ( β F x 0 ) 1 , Equation (35) is simplified as:
S F 1 2 1 + ln π β F + ln 2 = 1 2 1 + ln π D γ + ln 2 .
For small values of D as used in our numerical computations, S F is negative, signifying a strongly localised PDF.
For BF, for simplicity we use the final equilibrium p B in Equation (7) or (12) and β B = γ 2 D to obtain the differential entropy S B :
S B = - - d x p B ln p B = 1 4 ln Γ 1 4 2 + 1 + ln 4 D μ .
For small D, S F 1 2 ln D π γ while S B ln 4 D μ . Thus, the difference in differential entropy between p F and p B is:
Δ S = S F - S B = 1 4 ln D μ π γ 2 ,
which is negative for small D. That is, the quartic exponential PDF at the critical state has much larger entropy than the bimodal PDF at the subcritical state.
Figure 7 shows the time evolution of (Gibbs) differential entropy defined by Equation (34). As theoretically predicted above, we see much larger difference between the initial and final states compared with the off-quenching case [40]. It is interesting to observe that for the forward process, S takes its maximum values at times 2.4, 4.0, and 5.5, very close to where H took its maximum values, and both broadly following the t 2 scaling from Equation (23).
To complement S ( t ) , we also show in Figure 7 the Fisher information defined by:
I ( t ) = 1 p p ( x , t ) x 2 d x .
As the Fisher information measures the degree of “order”, increasing as the PDF develops large gradients, it shows the opposite tendency to S ( t ) , which increases with the degree of “disorder”. In particular, the Fisher information I ( t ) in Figure 7b for FP takes the minimum value around t m where the entropy S ( t ) is maximum and starts increasing beyond t > t m . These results thus confirm that t m marks the start of the formation of order, as noted previously.

6. Conclusions

We investigated information geometry associated with order-to-disorder and disorder-to-order transitions in a 0D Ginzburg–Landau model where the formation (disappearance) of an ordered state is modelled by the transition from a unimodal (bimodal) to bimodal (unimodal) PDF of a stochastic variable x. Our 0D model permitted us to perform a detailed statistical analysis. We considered on-critical quenching with a pair of forward and backward processes FP and BP for disorder-to-order (critical to subcritical) and order-to-disorder (subcritical to critical) transitions, respectively by selecting the initial PDF of FP/BP the same as the final equilibrium PDF of BP/FP. A pair of disorder-to-order and order-to-disorder transitions models a burst, for example, in the gene expression consisting of a pair of induction and repression (e.g., see [50]). In such bistable systems, a continuous switching between ordered and disordered states is often observed, the transition occurring in bursts interspersed by a quiescent period (e.g., see [50]). For our cyclic order-disorder transition, an initial condition represents the “resting” state between the two bursts. We thus paid particular attention to the effect of initial conditions on information change by comparing on-quenching and off-quenching cases.
We showed that FP and BP exhibit strikingly different evolution of time-dependent PDFs during transient relaxation due to non-equilibrium initial PDFs. In particular, FP driven by instability undergoes the broadening of the PDF with a large increase in (anomalous) fluctuations before the transition to the ordered state accompanied by narrowing the PDF width/decreased fluctuation. This large fluctuation essentially facilitates the existence of a geodesic solution in FP. This geodesic solution is a result of the self-regulation between the positive feedback ( γ x ) and the negative feedback ( - μ x 3 ), which regulate each other, minimising the information change. In a biological context, this minimal geodesic path could be understood in terms of “fitness” in the growth phase (e.g., gene expression). This suggests that the predator-prey type self-regulation with a nonlinear interaction facilitates a geodesic. In comparison, BP is mainly driven by the macroscopic motion due to the movement of the PDF peak, with much less prominent appearance of a geodesic solution. Specifically, the information length L was found to be much larger in BP than in FP, scaling as 0 . 9 γ D - 1 / 2 for BP, but only 2 . 1 ln ( γ D - 1 / 2 ) for FP, where D is the strength of an additive stochastic noise with a short correlation time. These results demonstrate a great advantage of L in revealing different physical processes (diffusion/advection) and the different role of diffusion D in transition.
To elucidate the importance of the initial condition between two bursts in cyclic transition, we summarise the striking differences between on-quenching and off-quenching as follows: (i) for FP, double-peaks emerge essentially immediately in on-quenching compared to their appearance only after a finite time in off-quenching; (ii) for FP, the on-quenching has a equilibration time shorter by a factor of two and information length L slightly less than in off-quenching; (iii) for BP, the equilibration time is much longer in on-quenching than in off-quenching, because the final state is at critical; (iv) for BP, the information length L is nevertheless reduced by a factor of 1.5 than in off-quenching. It is worth noting that from the perspective of a system’s “fitness”, the result (ii) could be advantageous when adjusting to a changing environment is costly, and thus, the minimum total change (measured by L ) and the minimum equilibration time are beneficial (see below). We highlight that L is a “Lagrangian” measure that quantifies the total change in information content in the system over time. We discuss this further in the following.
We note that our control parameter models the effect of environment (e.g., the temperature of the heat bath, etc.), and thus, a sudden change in the control parameter represents a sudden change in the environment. The time-evolution of PDFs occurs in order for the system to reach a new equilibrium state as the equilibrium state is optimal for the given new parameter (for the new environment). On the other hand, the smaller information length represents the smaller number of different states that a system undergoes to reach this new equilibrium state. Intriguingly, these seem to be closely related to the novel concept in microbial metabolism that states evolve under the trade-off between two principles: optimality under one given condition and minimal adjustment between conditions [51]. That is, when an environment changes, the initial state (optimal for the old environment) should change to the new optimal state (the final equilibrium) by undergoing time-evolution. Additionally, the smaller the information length, the less change in the system in adjustment. Thus, our results suggest that the initial “critical” state would be more advantageous for the system in changing environment.
Finally, in future work, it is planned to extend this model to more realistic cases (e.g., 1D or 2D models, a system of coupled equations, etc.).

Author Contributions

Rainer Hollerbach and Eun-jin Kim conceived the basic ideas; Rainer Hollerbach conducted the numerical calculations; Eun-jin Kim conducted the analytical derivations; Rainer Hollerbach and Eun-jin Kim wrote the paper. Both authors have read and approved the final manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A. Relation between 𝓛 and Relative Entropy

We first show the relation between τ ( t ) in Equation (2) and the second derivative of the relative entropy (or Kullback–Leibler divergence) D ( p 1 , p 2 ) = d x p 2 ln ( p 2 / p 1 ) where p 1 = p ( x , t 1 ) and p 2 = p ( x , t 2 ) as follows:
t 1 D ( p 1 , p 2 ) = - d x p 2 t 1 p 1 p 1 ,
2 t 1 2 D ( p 1 , p 2 ) = d x p 2 ( t 1 p 1 ) 2 p 1 2 - t 1 2 p 1 p 1 ,
t 2 D ( p 1 , p 2 ) = d x t 2 p 2 + t 2 p 2 ( ln p 2 - ln p 1 ) ,
2 t 2 2 D ( p 1 , p 2 ) = d x t 2 2 p 2 + ( t 2 p 2 ) 2 p 2 + t 2 2 p 2 ( ln p 2 - ln p 1 ) .
By taking the limit where t 2 t 1 = t ( p 2 p 1 = p ) and by using the total probability conservation (e.g., d x t p = 0 ), Equations (A1) and (A3) above lead to
lim t 2 t 1 = t t 1 D ( p 1 , p 2 ) = lim t 2 t 1 = t t 2 D ( p 1 , p 2 ) = d x t p = 0 .
While Equations (A2) and (A4) give
lim t 2 t 1 = t 2 t 1 2 D ( p 1 , p 2 ) = lim t 2 t 1 = t 2 t 2 2 D ( p 1 , p 2 ) = d x ( t p ) 2 p .
See also [37] for similar derivation.
To link this to information length L , we then express D ( p 1 , p 2 ) for small d t = t 2 - t 1 as
D ( p 1 , p 2 ) = d x ( t 1 p ( x , t 1 ) ) 2 p ( d t ) 2 + O ( ( d t ) 3 ) ,
where O ( ( d t ) 3 ) is higher order term in d t . We define the infinitesimal distance (information length) d l ( t 1 ) between t 1 and t 1 + d t by
d l ( t 1 ) = D ( p 1 , p 2 ) = d x ( t p ) 2 p d t + O ( ( d t ) 3 / 2 ) .
The total change in information between time 0 and t is then obtained by summing over d t ( t 1 ) and then taking the limit of d t 0 as
L ( t ) = lim d t 0 d l ( 0 ) + d l ( d t ) + d l ( 2 d t ) + d l ( 3 d t ) + · · · d l ( t - d t ) = lim d t 0 D ( p ( x , 0 ) , p ( x , d t ) ) + D ( p ( x , d t ) , p ( x , 2 d t ) ) + · · · D ( p ( x , t - d t ) , p ( x , t ) ) 0 t d t 1 d x ( t 1 p ) 2 p .

Appendix B. Derivation of Equation (28)

For small α ¯ x 2 < 1 , we approximate p ( x , t ) in Equation (17)
p ( x , t ) M β 0 1 4 e - γ t ( 1 - α ¯ x 2 ) 3 2 e - β 0 e - 4 γ t x 2 1 - α ¯ x 2 2 M β ( t ) 1 4 e - β ( t ) x 4 ,
where the normalisation factor M and β ( t ) are
M = 2 Γ 1 4 - 1 , β ( t ) = β 0 e - γ t , β 0 = μ 4 D .
Then,
p t = d β ( t ) d t 1 4 β - x 4 p ( x , t ) .
Thus, Equation (2) becomes
1 [ τ ( t ) ] 2 = d x 1 p ( x , t ) p ( x , t ) t 2 = β ˙ 2 1 16 β 2 - 1 2 β x 4 + x 8 = β ˙ 2 4 β 2 .
Here, we used x 4 = 1 4 β and x 8 = 5 16 β 2 ; the dot denotes the time derivative. Thus, using Equations (A10) and (A11) in L ( t ) in Equation (1) gives us
L ( t ) = 0 t d t 1 1 2 β d β ( t ) d t = 1 2 ln β ( t ) β ( t = 0 ) .

Appendix C. Properties of the Sum of Two Gaussian PDFs

We recall that for a single Gaussian PDF with mean value z = x and variance σ = ( δ x ) 2 , τ in Equation (2) is given by (e.g., [26,27])
1 [ τ ( t ) ] 2 = 1 2 β ( t ) 2 d σ d t 2 + 2 β d z d t 2 = 1 2 σ ( t ) 2 d σ d t 2 + 1 σ d z d t 2 .
Here, σ = 1 / 2 β .
We now show the information length for double Gaussian PDFs which are well-separated is approximately the same as that for a single Gaussian PDF. To this end, for a double Gaussian, we let
p = p 1 + p 2 = N ( t ) [ p ˜ 1 + p ˜ 2 ] , N ( t ) = β ( t ) 2 π , p ˜ 1 = e - β ( t ) x + x 0 2 = e - β ( t ) x 1 2 , p ˜ 2 = e - β ( t ) x - x 0 2 = e - β ( t ) x 2 2 .
Here, N is the normalisation constant (e.g., N - 1 = d x ( p ˜ 1 + p ˜ 2 ) ) and x 1 = x + x 0 and x 2 = x - x 0 .
To show Equation (28), we assume x 0 is constant given by the peak location x 0 = γ μ in x > 0 while β = β ( t ) depending on time. Then, we can show
1 p ( x , t ) p ( x , t ) t 2 = N ˙ 2 N ( p ˜ 1 + p ˜ 2 ) + 2 N ˙ ( p ˜ ˙ 1 + p ˜ ˙ 2 ) + N ( p ˜ ˙ 1 + p ˜ ˙ 2 ) 2 p ˜ 1 + p ˜ 2 .
Now, we compute the various quantities in Equation (A15) as follows:
p ˜ ˙ 1 = - β ˙ x 1 2 p ˜ 1 = β ˙ β p ˜ 1 , ( p ˜ ˙ 1 ) 2 = β ˙ 2 p ˜ 1 β β p ˜ 1 .
Similarly,
p ˜ ˙ 2 = - β ˙ x 2 2 p ˜ 2 = β ˙ β p ˜ 2 , ( p ˜ ˙ 2 ) 2 = β ˙ 2 p ˜ 2 β β p ˜ 2 .
Thus, by using Equations (A16) and (A17), we calculate the last term in Equation (A15) as follows:
( p ˜ ˙ 1 + p ˜ ˙ 2 ) 2 = β ˙ 2 p ˜ 1 β β p ˜ 1 + p ˜ 2 β β p ˜ 2 + 2 β p ˜ 1 β p ˜ 2
         = β ˙ 2 ( p ˜ 1 + p ˜ 2 ) β β p ˜ 1 + ( p ˜ 1 + p ˜ 2 ) β β p ˜ 2 + G 1
      = β ˙ 2 ( p ˜ 1 + p ˜ 2 ) β β ( p ˜ 1 + p ˜ 2 ) + G 2 ,
where G 1 and G 2 are terms involving the product of p ˜ 1 and p ˜ 2 . For the PDF peaks that are well-separated and thus independent, there is no overlap between p ˜ 1 and p ˜ 2 in x, leading to d x p ˜ 1 ( x ) p ˜ 2 ( x ) = 0 . That is, in this case, d x G 1 = d x G 2 = 0 . Thus, these terms G 1 and G 2 do not contribute to Equation (2). By using these results in Equation (2), we obtain
d x 1 p ( x , t ) p ( x , t ) t 2 = N ˙ 2 N 2 + 2 β ˙ N ˙ β 1 N + N β ˙ 2 β β 1 N .
By using N = 1 2 β π , we simplify Equation (A21) as
d x 1 p ( x , t ) p ( x , t ) t 2 = β ˙ 2 2 β 2 = σ ˙ 2 2 σ 2 .
Thus, Equation (A22) is the same as Equation (A13) in the limit z = 0 . We note that Equation (28) is obtained by the time integral of Equation (A22) by using the results in Appendix B.
Next to show Equation (30), we need to consider the case where β is constant in Equation (A14) while x 0 = x 0 ( t ) depends on time. In this case, we have
( p ˜ ˙ 1 + p ˜ ˙ 2 ) 2 = 4 β 2 x 0 ˙ 2 N 2 x 2 ( p ˜ 1 + p ˜ 2 ) 2 + 2 x x 0 ( p ˜ 1 2 - p ˜ 2 2 ) + x 0 2 ( p ˜ 1 - p ˜ 2 ) 2 = 4 β 2 x 0 ˙ 2 N 2 x 2 ( p ˜ 1 + p ˜ 2 ) 2 + 2 x x 0 ( p ˜ 1 2 - p ˜ 2 2 ) + x 0 2 ( p ˜ 1 + p ˜ 2 ) 2 + G 3 ,
where G 3 is a function depending on the product of p ˜ 1 and p ˜ 2 , which vanishes upon integral over x when p ˜ 1 and p ˜ 2 are well-separated with negligible overlap. In this case,
d x 1 p ( x , t ) p ( x , t ) t 2 = 4 β 2 x 0 ˙ 2 N d x ( x + x 0 ) 2 p ˜ 1 + ( x - x 0 ) 2 p ˜ 2 = - 4 β 2 x 0 ˙ 2 N β d x ( p ˜ 1 + p ˜ 2 ) = - 4 β 2 x 0 ˙ 2 N β 1 N = 2 β x 0 ˙ 2 ,
where we used N = 1 2 β π and thus β 1 N = - 1 2 β N . Equation (A24) is the same as Equation (A13) in the opposite limit where z = x 0 and β ˙ = 0 .

References

  1. Kibble, T.W.B. Some implications of a cosmological phase transition. Phys. Rep. 1980, 67, 183–199. [Google Scholar] [CrossRef]
  2. Nagashima, Y.; Nambu, Y. Elementary Particle Physics: Quantum Field Theory and Particles; Wiley-VCH: Weinheim, Germany, 2010. [Google Scholar]
  3. Mazenko, G. Theory of unstable growth. Phys. A 1994, 204, 437–449. [Google Scholar] [CrossRef]
  4. Longo, G.; Montévil, M. From physics to biology by extending criticality and symmetry breakings. Prog. Biophys. Mol. Biol. 2011, 106, 340–347. [Google Scholar] [CrossRef] [PubMed]
  5. Bossomaier, T.; Barnett, R.; Harré, M. Information and phase transitions in socio-economic systems. Complex Adapt. Syst. Model. 2013, 1, 9. [Google Scholar] [CrossRef]
  6. Haken, H. Information and Self-Organization: A Macroscopic Approach to Complex Systems; Springer: Berlin, Germany, 2006. [Google Scholar]
  7. Kim, E.; Diamond, P.H. Zonal flows and transient dynamics of the L-H transition. Phys. Rev. Lett. 2003, 90, 185006. [Google Scholar] [CrossRef] [PubMed]
  8. Kim, E. Consistent theory of turbulent transport in two-dimensional magnetohydrodynamics. Phys. Rev. Lett. 2006, 96, 084504. [Google Scholar] [CrossRef] [PubMed]
  9. Srinivasan, K.; Young, W.R. Zonostrophic instability. J. Atmos. Sci. 2012, 69, 1633–1656. [Google Scholar] [CrossRef]
  10. Sayanagi, K.M.; Showman, A.P.; Dowling, T.E. The emergence of multiple robust zonal jets from freely evolving, three-dimensional stratified geostrophic turbulence with applications to Jupiter. J. Atmos. Sci. 2008, 65, 12. [Google Scholar] [CrossRef]
  11. Kim, E.; Liu, H.; Anderson, J. Probability distribution function for self-organization of shear flows. Phys. Plasmas 2009, 16, 052304. [Google Scholar] [CrossRef]
  12. Newton, A.P.L.; Kim, E.; Liu, H.L. On the self-organizing process of large scale shear flows. Phys. Plasmas 2013, 20, 092306. [Google Scholar] [CrossRef]
  13. Tsuchiya, M.; Giuliani, A.; Hashimoto, M.; Erenpreisa, J.; Yoshikawa, K. Emergent self-organized criticality in gene expression dynamics: Temporal development of global phase transition revealed in a cancer cell line. PLoS ONE 2015, 10, e0128565. [Google Scholar] [CrossRef] [PubMed]
  14. Tang, C.; Bak, P. Mean field theory of self-organized critical phenomena. J. Stat. Phys. 1988, 51, 797–802. [Google Scholar] [CrossRef]
  15. Jensen, H.J. Self-Organized Criticality: Emergent Complex Behavior in Physical and Biological Systems; Cambridge University Press: Cambridge, UK, 1998. [Google Scholar]
  16. Pruessner, G. Self-Organised Criticality; Cambridge University Press: Cambridge, UK, 2012. [Google Scholar]
  17. Fauve, S.; Heslot, F. Stochastic resonance in a bistable system. Phys. Lett. A 1983, 97, 5–7. [Google Scholar] [CrossRef]
  18. Angeli, D.; Ferrell, J.E.; Sontag, E.D. Detection of multistability, bifurcations, and hysteresis in a large class of biological positive-feedback systems. Proc. Natl. Acad. Sci. USA 2004, 101, 1822–1827. [Google Scholar] [CrossRef] [PubMed]
  19. Holcman, D.; Tsodyks, M. The emergence of up and down states in cortical networks. PLoS Comp. Biol. 2006, 2, e23. [Google Scholar] [CrossRef] [PubMed]
  20. Mejias, J.F.; Kappen, H.J.; Torres, J.J. Irregular dynamics in up and down cortical states. PLoS ONE 2010, 5, e13651. [Google Scholar] [CrossRef] [PubMed]
  21. Hidalgo, J.; Seoane, L.F.; Cort́es, J.M.; Munoz, M.A. Stochastic amplification of fluctuations in cortical up-states. PLoS ONE 2012, 7, e40710. [Google Scholar] [CrossRef] [PubMed]
  22. Tyagi, S. Tuning noise in gene expression. Mol. Syst. Biol. 2015, 11, 805. [Google Scholar] [CrossRef] [PubMed]
  23. Di Santo, S.; Burioni, R.; Vezzani, A.; Muñoz, M.A. Self-organized bistability associated with first-order phase transitions. Phys. Rev. Lett. 2016, 116, 240601. [Google Scholar] [CrossRef] [PubMed]
  24. Nicholson, S.B.; Kim, E. Investigation of the statistical distance to reach stationary distributions. Phys. Lett. A 2015, 379, 83–88. [Google Scholar] [CrossRef]
  25. Nicholson, S.B.; Kim, E. Entropy structures in sound: Analysis of classical music using the information length. Entropy 2016, 18, 258. [Google Scholar] [CrossRef]
  26. Heseltine, J.; Kim, E. Novel mapping in non-equilibrium stochastic processes. J. Phys. A 2016, 49, 175002. [Google Scholar] [CrossRef]
  27. Kim, E.; Lee, U.; Heseltine, J.; Hollerbach, R. Geometric structure and geodesic in a solvable model of nonequilibrium process. Phys. Rev. E 2016, 93, 062127. [Google Scholar] [CrossRef] [PubMed]
  28. Kim, E.; Hollerbach, R. Signature of nonlinear damping in geometric structure of a nonequilibrium process. Phys. Rev. E 2017, 95, 022137. [Google Scholar] [CrossRef] [PubMed]
  29. Gibbs, A.L.; Su, F.E. On choosing and bounding probability metrics. Int. Stat. Rev. 2002, 70, 419–435. [Google Scholar] [CrossRef]
  30. Wootters, W.K. Statistical distance and Hilbert space. Phys. Rev. D 1981, 23, 357–362. [Google Scholar] [CrossRef]
  31. Ruppeiner, G. Thermodynamics: A Riemannian geometric model. Phys. Rev. A 1979, 20, 1608–1613. [Google Scholar] [CrossRef]
  32. Schlögl, F. Thermodynamic metric and stochastic measures. Z. Phys. B Cond. Matt. 1985, 59, 449–454. [Google Scholar] [CrossRef]
  33. Feng, E.H.; Crooks, G.E. Far-from-equilibrium measurements of thermodynamic length. Phys. Rev. E 2009, 79, 012104. [Google Scholar] [CrossRef] [PubMed]
  34. Braunstein, S.L.; Caves, C.M. Statistical distance and the geometry of quantum states. Phys. Rev. Lett. 1994, 72, 3439–3443. [Google Scholar] [CrossRef] [PubMed]
  35. Strobel, H.; Muessel, W.; Linnemann, D.; Zibold, T.; Hume, D.B.; Pezzé, L.; Smerzi, A.; Oberthaler, M.K. Fisher information and entanglement of non-Gaussian spin states. Science 2014, 345, 424–427. [Google Scholar] [CrossRef] [PubMed]
  36. Nulton, J.; Salamon, P.; Andresen, B.; Anmin, Q. Quasistatic processes as step equilibrations. J. Chem. Phys. 1985, 83, 334–338. [Google Scholar] [CrossRef]
  37. Crooks, G.E. Measuring thermodynamic length. Phys. Rev. Lett. 2007, 99, 100602. [Google Scholar] [CrossRef] [PubMed]
  38. Sivak, D.A.; Crooks, G.E. Thermodynamic metrics and optimal paths. Phys. Rev. Lett. 2012, 8, 190602. [Google Scholar] [CrossRef] [PubMed]
  39. Salamon, P.; Nulton, J.D.; Siragusa, G.; Limon, A.; Bedeaus, D.; Kjelstrup, S. A simple example of control to minimize entropy production. J. Non-Equilib. Thermodyn. 2002, 27, 45–55. [Google Scholar] [CrossRef]
  40. Kim, E.; Hollerbach, R. Geometric structure and information change in phase transitions. Phys. Rev. E 2017, 95, 062107. [Google Scholar] [CrossRef]
  41. Bhattacharjee, J.K.; Meakin, P.; Scalapino, D.J. Nonequilibrium dynamics of N-component Ginzburg–Landau fields in zero and one dimension. Phys. Rev. A 1984, 30, 1026. [Google Scholar] [CrossRef]
  42. Kim, E.; Hollerbach, R. Time-dependent probability density function in cubic stochastic processes. Phys. Rev. E 2016, 94, 052118. [Google Scholar] [CrossRef] [PubMed]
  43. Klebaner, F. Introduction to Stochastic Calculus with Applications; Imperial College Press: London, UK, 2012. [Google Scholar]
  44. Risken, H. The Fokker–Planck Equation: Methods of Solution and Applications; Springer: Berlin, Germany, 1996. [Google Scholar]
  45. Suzuki, M. Scaling theory of transient phenomena near the instability point. J. Stat. Phys. 1977, 16, 11–32. [Google Scholar] [CrossRef]
  46. Caroli, B.; Caroli, C.; Roulet, B. Diffusion in a bistable potential: A systematic WKB treatment. J. Stat. Phys. 1979, 21, 415–437. [Google Scholar] [CrossRef]
  47. Suzuki, M. Theory of instability, nonlinear Brownian motion and formation of macroscopic order. Phys. Lett. A 1978, 67, 339–341. [Google Scholar] [CrossRef]
  48. Kubo, R.; Matsuo, K.; Kitahara, K. Fluctuation and relaxation of macrovariables. J. Stat. Phys. 1973, 9, 51–96. [Google Scholar] [CrossRef]
  49. Michalowicz, J.V.; Nichols, J.M.; Bucholtz, F. Calculation of differential entropy for a mixed Gaussian distribution. Entropy 2008, 10, 200–206. [Google Scholar] [CrossRef]
  50. Ferguson, M.L.; Le Coq, D.; Jules, M.; Aymerich, S.; Radulescue, O.; Declerck, N.; Royer, C.A. Reconciling molecular regulatory mechanisms with noise patterns of bacterial metabolic promoters in induced and repressed states. Proc. Natl. Acad. Sci. USA 2012, 109, 155–160. [Google Scholar] [CrossRef] [PubMed]
  51. Schuetz, R.; Zamboni, N.; Zampieri, M.; Heinemann, M.; Sauer, U. Multidimensional optimality of microbial metabolism. Science 2012, 336, 601–604. [Google Scholar] [CrossRef] [PubMed]
Figure 1. The ratio x 4 / x 2 as a function of t, for the three values D = 10 - 3 (red), 10 - 5 (blue) and 10 - 7 (black). (a) shows FP with t on a linear scale; (b) shows BP with t on a logarithmic scale. All six curves are for γ = 0.7 . The three dots on each curve correspond to the solutions shown in Figure 2 and Figure 3. The central dot is always when x 4 / x 2 = 1.25 . In (a), the other two dots are at t ± 1 relative to the central one; in (b), they are at t / 2 and 2 t relative to the central one.
Figure 1. The ratio x 4 / x 2 as a function of t, for the three values D = 10 - 3 (red), 10 - 5 (blue) and 10 - 7 (black). (a) shows FP with t on a linear scale; (b) shows BP with t on a logarithmic scale. All six curves are for γ = 0.7 . The three dots on each curve correspond to the solutions shown in Figure 2 and Figure 3. The central dot is always when x 4 / x 2 = 1.25 . In (a), the other two dots are at t ± 1 relative to the central one; in (b), they are at t / 2 and 2 t relative to the central one.
Entropy 19 00268 g001
Figure 2. The PDFs for FP, for (a) D = 10 - 3 ; (b) D = 10 - 5 ; (c) D = 10 - 7 . The initial condition is indicated by the red central peak. Subsequent times are indicated in green, black and blue, taking the c ln D - 1 shift into account. For (a) these are t = 1.25 , 2.25 and 3.25; for (b) t = 2.79 , 3.79 and 4.79; for (c) t = 4.43 , 5.43 and 6.43. The middle time (the black line) is always when the ratio x 4 / x 2 = 1.25 ; the green line is always t - 1 relative to that, and the black line is t + 1 . See also the three dots on each curve in Figure 1a.
Figure 2. The PDFs for FP, for (a) D = 10 - 3 ; (b) D = 10 - 5 ; (c) D = 10 - 7 . The initial condition is indicated by the red central peak. Subsequent times are indicated in green, black and blue, taking the c ln D - 1 shift into account. For (a) these are t = 1.25 , 2.25 and 3.25; for (b) t = 2.79 , 3.79 and 4.79; for (c) t = 4.43 , 5.43 and 6.43. The middle time (the black line) is always when the ratio x 4 / x 2 = 1.25 ; the green line is always t - 1 relative to that, and the black line is t + 1 . See also the three dots on each curve in Figure 1a.
Entropy 19 00268 g002
Figure 3. The PDFs for BP, for (a) D = 10 - 3 ; (b) D = 10 - 5 ; (c) D = 10 - 7 . The times are indicated in green, black and blue, taking the D - 1 / 2 shift into account. For (a), these are t = 6.4 , 12.8 and 25.6; for (b), t = 67 , 134 and 268; for (c), t = 675 , 1350 and 2700. The middle time (the black line) is always when the ratio x 4 / x 2 = 1.25 ; the green line is always t / 2 relative to that, and the black line is 2 t . See also the three dots on each curve in Figure 1b. Note finally how x and p have been rescaled according to D ± 1 / 4 .
Figure 3. The PDFs for BP, for (a) D = 10 - 3 ; (b) D = 10 - 5 ; (c) D = 10 - 7 . The times are indicated in green, black and blue, taking the D - 1 / 2 shift into account. For (a), these are t = 6.4 , 12.8 and 25.6; for (b), t = 67 , 134 and 268; for (c), t = 675 , 1350 and 2700. The middle time (the black line) is always when the ratio x 4 / x 2 = 1.25 ; the green line is always t / 2 relative to that, and the black line is 2 t . See also the three dots on each curve in Figure 1b. Note finally how x and p have been rescaled according to D ± 1 / 4 .
Entropy 19 00268 g003
Figure 4. (a) H = 0.7 x 2 - x 4 as a function of time for FP; (b) Δ = x 4 - x 2 as a function of time for FP; (c) - H = x 4 as a function of time for BP; (d) Δ = x 4 - x 2 as a function of time for BP.
Figure 4. (a) H = 0.7 x 2 - x 4 as a function of time for FP; (b) Δ = x 4 - x 2 as a function of time for FP; (c) - H = x 4 as a function of time for BP; (d) Δ = x 4 - x 2 as a function of time for BP.
Entropy 19 00268 g004
Figure 5. (a,b) show E and L , respectively, as functions of time, for γ = 0.7 ; (c) shows L as a function of γ . All three panels are for FP only. The solid lines are the on-quenching process considered here; the dashed lines are for the off-quenching process considered in [40]. D = 10 - 3 to 10 - 7 as indicated. Note the different combinations of linear and logarithmic scales to emphasize different features in different quantities.
Figure 5. (a,b) show E and L , respectively, as functions of time, for γ = 0.7 ; (c) shows L as a function of γ . All three panels are for FP only. The solid lines are the on-quenching process considered here; the dashed lines are for the off-quenching process considered in [40]. D = 10 - 3 to 10 - 7 as indicated. Note the different combinations of linear and logarithmic scales to emphasize different features in different quantities.
Entropy 19 00268 g005
Figure 6. (a,b) show E and L , respectively, as functions of time, for γ = 0.7 . (c) shows L as a function of γ . All three panels are for BP only. The solid lines are the on-quenching process considered here; the dashed lines are for the off-quenching process considered in [40]. D = 10 - 3 to 10 - 7 as indicated.
Figure 6. (a,b) show E and L , respectively, as functions of time, for γ = 0.7 . (c) shows L as a function of γ . All three panels are for BP only. The solid lines are the on-quenching process considered here; the dashed lines are for the off-quenching process considered in [40]. D = 10 - 3 to 10 - 7 as indicated.
Entropy 19 00268 g006
Figure 7. (a) Entropy S ( t ) for FP; (b) Fisher information I ( t ) for FP; (c) entropy S ( t ) for BP; (d) Fisher information I ( t ) for BP. Note how S and I are essentially opposites of each other. Again also note how the equilibration time scale is O ( ln D - 1 ) for FP, and O ( D - 1 / 2 ) for BP.
Figure 7. (a) Entropy S ( t ) for FP; (b) Fisher information I ( t ) for FP; (c) entropy S ( t ) for BP; (d) Fisher information I ( t ) for BP. Note how S and I are essentially opposites of each other. Again also note how the equilibration time scale is O ( ln D - 1 ) for FP, and O ( D - 1 / 2 ) for BP.
Entropy 19 00268 g007
Table 1. Summary of Forward Process (FP), Backward Process (BP): FP and BP have equilibrium PDFs (in the limit of t ), p F and p B , respectively. FP and BP start with the initial PDFs, p ( x , 0 ) = p B and p ( x , 0 ) = p F , respectively and reach the equilibrium PDFs, p ( x , t ) = p F  and p B , respectively. p F  and p B have equilibrium variances, D / 2 γ and 2 D / μ Γ 3 4 / Γ 1 4 , respectively. σ 0 and σ F are the initial and final variances at t = 0 and t , respectively, for each process.
Table 1. Summary of Forward Process (FP), Backward Process (BP): FP and BP have equilibrium PDFs (in the limit of t ), p F and p B , respectively. FP and BP start with the initial PDFs, p ( x , 0 ) = p B and p ( x , 0 ) = p F , respectively and reach the equilibrium PDFs, p ( x , t ) = p F  and p B , respectively. p F  and p B have equilibrium variances, D / 2 γ and 2 D / μ Γ 3 4 / Γ 1 4 , respectively. σ 0 and σ F are the initial and final variances at t = 0 and t , respectively, for each process.
Case λ p ( x , 0 ) p ( x , t ) σ 0 σ F
FP - γ p B ( x ) p F ( x ) 2 D / μ Γ 3 4 / Γ 1 4 D / 2 γ
BP0 p F ( x ) p B ( x ) D / 2 γ 2 D / μ Γ 3 4 / Γ 1 4

© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Back to TopTop