Both Gibbs’ and Onsager’s work have pointed to a new type of mathematics: random variables and stochastic processes. Gibbs’ thermodynamic quantities with thermal fluctuations are random variables, and Onsager has used extensively Gaussian-Markov processes to describe the dynamics near an equilibrium [4,5]. This approach can be traced back to the earlier work of Einstein on Brownian motion and Smoluchowski on diffusion.

Quantitative modelling in chemical engineering has been based on the Law of Mass Action [6]. The applications of this theory to enzyme kinetics gives rise to the entire kinetic modelling of biochemical reactions for individual enzymes [7] and for enzymatic reaction systems [8]. However, the theory based on the Law of Mass Action considers no fluctuations so it sits in an odd position with respect to the more general theories of Gibbs and Onsager. It does have its strength though, it is a fully dynamic theory, while the Gibbs’ is not, and it can be applied to system beyond Onsager’s linear irreversibility.

Is there a theory which can embody all the above mentioned theories? It is clear that such a theory, even very imperfect, can provide great insights into the working of biochemical reaction systems in their living environment. While a consensus has not been reached, the recent rapid rise of applications of the Gillespie algorithm seems to suggest an interesting possibility.

It might be a surprise to some, but the Gillespie algorithm (GA) is really an equation that perscribes the dynamic trajectory of a stochastic process. Mathematical equations can come in many different forms: differential equations for continuous variables changing with time, stochastic differential equations (SDE) for the trajectories of continuous random variables changing with time, and the GA is simply the equation for the trajectory of discrete random variables changing with time. For a random variable changing with time one can either characterize it by its stochastic trajectories, as by the SDE and the GA, or one can characterize its probability distribution as a function of time. The corresponding equation for the SDE is the well-known Fokker-Planck equation, and the corresponding equation for the GA is called the chemical master equation (CME).

It has been mathematically shown that the CME approach is the mesoscopic version of the Law of Mass Action [8]. It is not a competing theory to the Law of Mass Action, rather, it extends the latter to the mesoscopic chemistry and biochemistry.

We do not expect our readers to have a background in the CME. For a quick introduction see Appendix. In particular, one should learn to draw the chemical master equation graph. See Chapter 11 of [8] and a more recent [9] for more on the CME. Discussions on the GA can be found in [10].

In this article, we shall follow the CME approach to study biochemical reaction networks. We are particularly interested in such systems situated in a “living envirnment”. It turns out, one of the precise defining characteristics of the environment is the amount of chemical energy pumped into the system – similar to the battery in a radio.

While the CME approach is a new methodological advance in modelling open (driven) biochemical systems, a new concept also arises from recent studies on open (driven) biochemical systems: the nonequilibrium steady state (NESS) as a mathematical abstraction of biochemical homeostasis. In terms of the CME approach to mesoscopic chemical and biochemical reaction systems there are three, and only three, types of dynamics [11]:

To a first-order approximation, one can represent a biochemical cell or a subcellular network in homeostasis as a NESS. This is the theory being put forward by I. Prigogine, G. Nicolis and their Brussels group [12]. The NESS theory has recently gone through major development in terms of the fluctuation theorem in statistical physics, especially the stochastic version of J. Kurchan, J.L. Lebowitz and H. Spohn [13,14], and irreversible Markov processes in mathematics [15](A deep mathematical result shows that the arrow of time is a sufficient and necessary condition for the positiveness of an appropriately defined entropy production rate.). The theory of NESS also has enjoyed great appreciation through works on molecular motors [16,17].

To have a better understanding of the nature of a NESS, we list three key characteristics of a system in equilibrium steady state: First, there is no flux in each and every reaction. This is known as the principle of detailed balance [18]. Second, the system is time-reversible. One can play a “recording tape” of the system backward and will find it is statistically equivalent; There is no arrow of time [15]. The logical conseqeunce of the above statements is that any process occurred in an equilibrium system will have equal probability to run backward. Hence nothing can be accomplished. There is no energy transduction, and there is no signal processing.

For more discussions on NESS and its applications to biochemical systems and modelling, the readers are referred to [19,20].

One of the most important results in deterministic enzyme kinetic theory is the quasi-steady state approximation leading to the well-known Michaelis-Menten equation for the production of the product in Equation (1):

where E_t is the total enzyme concentration. We shall now carry out a parallel analysis for the CME (2).

As pointed out by Kepler and Elston [21], Rao and Arkin [22], and by Qian [16], the quasi-stationary approximation is best cast in terms of the conditional probability. If the dynamics in Equation (2) is such that the changes in n can reach stationarity first, due to n₀ << m₀, then one can first solve the problem of steady state conditional distribution p(n|m), then using this to solve the p(m, t). This is done as follows.

In the first step, on a fast time scale for fixed m, the Equation (2) can be written as

Here we assumed m−1 ≈ m. This immdiate gives us the conditional stationary state distribution for n:

which yields a mean value for n, with given m:

This result agrees exactly with the deterministic model.

Now the second step, let us sum over all the n for the Equation (2). With

we have

Equation (8) is the result of a quasi-statioanry approximation. It should be compared with the deterministic equation (4). It can be graphically represented as in Figure 1.

We see that at any given time, S can increase or decrease by one. The decrease of one S is from either ES → E + P or ES → E + S, with probability k ^ 2 k ^ - 1 + k ^ 2 and k ^ - 1 k ^ - 1 + k ^ 2, respectively. Hence, the number of P will increase, at any given time, with the rate of

This is exactly the CME version of Equation (4) [22]. The result shows that the waiting time between consecutive arrivial of P is exponentially distributed. This surprising result can be best understood from a mathematical theorem on the superposition of N identical, independent renewal processes [23]. For a more mathematical discussion on the subject, see [24].

Even when the number of enzymes is not large, the product arrival time distribution contains no information more than the traditional Michaelis-Menten rate constant. However, this is not the case if there is truly only a single enzyme. This will be discussed below.

Now in Equation (2), let us consider n₀ = 1. Then the equation is reduced to

This is the CME for a single molecule enzyme kinetics according to the MM in Equation (1). Often, one is only interested in the conformational states of the enzyme:

Carry out the summation on the both sides of Equation (10), we have

where

c_S is the concentration of S. Equation (11) is the stochastic model for a single enzyme molecule dynamics.

The steady state probability for the single enzyme can be easily obtained from Equation (11):

Then the steady state single enzyme turnover flux is

with k M = K - 1 + k 2 k 1 and V_max = k₂. The last expression is precisely the Michaelis-Menten formula.

The single enzyme steady-state flux J^ss is exactly the inverse of the mean time duration between two product arrivials. The time perspective is natural for single-molecule measurements on enzyme turnover [23]. Not only one can obtain the mean time duration, one can also, according to the stochastic model in Equation (11), compute the probability density function (pdf) of the time duration between two product arrivals. The pdf is exponentially distributed if there is a single rate-limiting step with in the enzyme cycle. In general, however, it is not exponentially distributed.

We now consider an enzyme kinetic scheme that is a little more complex than that in Equation (1):

With concentrations c_S and c_P for S and P being constant, as many cases in a living cell under homeostasis, we have the steady state single enzyme turnover flux

The origin of this flux is the non-equilibrium between the chemical potentials of S and P:

We see that when Δμ > 0, J^ss > 0, when Δμ < 0, J^ss < 0, and when Δμ = 0, J^ss = 0. In fact, the product Δμ × J^ss is the amount of chemical energy input to the single enzyme. If the enzyme does not do any mechanical work such as a motor protein, then all this energy becomes heat and dissipated into the aqueous solution in which the enzymatic reaction occurs.

Let us see an example of J^ss as function of Δμ, for k_i = λ, k_−i = 1, i = 1, 2, 3, and c_P = 1 while changing the c_S:

Figure 2A shows several curves. We see that their relationship is not linear over the entire range of Δμ. Only when the Δμ is very small, there is a linear region J^ss = Δμ/k_BT(λ⁴ +2λ³ +3λ² +2λ+1). This is the region where Onsager’s theory applies. In fact, the linear coefficient between Δμ and J^ss is precisely the one-way flux in the equilibrium. To show this, we note from Equations (16) and (17) that J^ss = J⁺ − J⁻ and Δμ = k_BT ln(J⁺/J⁻). Then, when J^ss <<J⁻, we have

Note that in equilibrium, J⁺ = J⁻. The last equation is known as detailed balance, which plays a central role in Onsager’s theory.

Therefore, the simple enzyme kinetics is not in the region with linear irreversibility. Onsager’s theory does not apply. Interestingly, we also note that the nonlinear curves in Figure 2A very much resemble the curret-voltage characteristics of a semiconductant diode. It will be interesting to further explore the similarities between driven biochemical and electronic systems [25].

In fact, one of the most important results in Onsager’s linear theory is the reciprocal relations [26,27] which, based on the principle of detailed balance, dictates certain symmetry in the kinetics. One of the consequences of this symmetry is that chemical kinetics near equilibrium can not oscillate. Oscillatory kinetics are associated with the complex eigenvalues of the kinetics system. For the scheme in Equation (15), the imaginary component of its eigenvalues is 4 Δ - Λ 2, where

To see an example, let us again consider k₁ = k₂ = k₃ = λ, and k₋₁ = k₋₂ = k₋₃ = 1, and c_P = 1. Then

The oscillations exist for

We see from Figure 2B that in general the c_S has to be sufficiently far away from its equilibrium value in order to have the oscillation. Chemical and biochemical oscillation is a far-from-equilibrium phenomenon [28,29].

Many kinase itself can exist in two different forms: an inactive state and an active state. Furthermore, the conversion from the former to the latter involves the binding of the E*, sometime one, sometime two. Hence, we have [31–33]:

where χ = 1, 2. We shall call χ = 1 first-order autocatalysis and χ = 2 second-order autocatalysis. Therefore, if the conversion is rapid, then the active kinase concentration is [K^‡] = K_a[K][E*]^χ. Now combining the reaction in Equation (22) with the PdPC, such a mechanism is called autocatalysis: more E* is made, more K^‡, which in turn to make more E*. Quantitatively, the rate of phosphorylation reaction catalyzed by the active kinase is:

where [X] denotes the concentration of biochemical species X. Note, however, that the same kinase K^‡ also catalyzed the reverse reaction of the phosphorylation. Hence, to be more realistic, we have

in which

contains the concentration of ATP and ADP respectively. Equation (24) is the kinetic equation for the phosphorylation reaction catalyzed by a kinase which is activated by binding χ number of E*.

Figure 3 shows four, with subtle differences, PdPC with such a positive feedback loop. Biochemical examples of this type of regulation are MPAK pathway [31], Src Family kinase pathway [32], and Rabaptin-5 mediated Rab5 activation in endocytosis [33]. We shall now establish the appropriate kinetic equations for each of these nonlinear biochemical networks.

For the kinetic scheme in Figure 3A, we have χ = 1 and

here we use x to denote the [E*], x_t = [E] + [E*]. In the equation

represent the rates for the dephosphorylation and the rate for its reverse reaction, respectively. Both are catalyzed by the enzyme phosphatase P. For simplicity, we assume both kinase and phosphatase are operating in their linear region.

Bishop and Qian [34] have carefully studied Equation (25). While this model is very simple, the issues arise from the model are important, and have not been widely discussed. It is well-known, and as we shall discuss in Section 3.4, nonlinear open chemical and biochemical reaction systems can exhibit bistability, which plays a crucial role in cellular genetic [35] and signal regulations [20,36]. In fact, it has been argued that bistability is one of the key origins that generate complex dynamic behavior [37]. Bistable chemical reaction systems have been extensively studied in the past [38]. In fact, it is relatively easy to theoretically construct reaction schemes that show bistability and bifurcation. Since bistability mathematically means two stable and one unstable fixed points in the positive quardrant, it is easy to show, according to the Law of Mass Action, that a tri-molecular reaction (as a reduced mechanism for multisteps of bimolecular reactions) is necessay.

In [34], however, we discovered a simpler bi-molecular chemical reaction system that possibly exhibits “bistability”. The bistability is in quotation marks since the mechanism is very different from that in traditional nonlinear reactions. The system is modelled in terms of a CME, and the bistability and (saddle-node) bifurcation are purely stochastic phenomenon. They only occur in reaction systems with small volume and small number of molecules.

For the kinetic scheme in Figure 3B, we again have χ = 1. However, we assume that there is a continuous biosynthesis and degradation for the E such that its concentration is sustained in the biochemical system, say at the value of a. Then, the kinetic equation for the dynamics of [E*] becomes

When δ = 0, Equation (36) is the same equation for the generic chemical reaction scheme

J. Keizer studied this model in [42] to illustrate a very interesting observation: While the deterministic kinetics of the system has a positive steady state, the steady state of its stochastic kinetics is zero. Vellela and Qian have studied this Keizer’s “paradox” [43]. It was shown that there are two very different time scales in the stochastic model: In a rather short time scale corresponding to the eigenvalues |λ₂| and above, the system rapidly settles to a quasi-stationary distribution peaking at the deterministic positive steady state. However, in a much slower time scale corresponding to the eigenvalue |λ₁|, the above probability distribution slowly decay to zero. For very large reaction system volume V, λ₁ ~ −e^−cV where c is a positive constant. Hence there is an exponentially slow decay process beyond the infinite time of the deterministic dynamics [44,45].

Keizer’s paradox and its resolution is the origin of all the multi-scale dynamics in the CME system with multi-stability. It is also clear it is intimately related to the stochastic bistability in Section 3.2.2 when the k₋₂, i.e., δ in Equation (36), is very small but nonzero. k₋₂ controls the lifetime, i.e., relative probability of the the zero state in Figure 5.

For the kinetic scheme in Figure 3C with second-order autocatalysis, we have χ = 2. We again assume that there is a continuous biosynthesis and degradation for the E such that its concentration is sustained in the biochemical system at a constant value of a. Then, the kinetic equation for the dynamics of x = [E*] becomes

On the other hand, if we assume the rate of biosynthesis is negligible, and that both kinase and phosphatase catalyzed reactions are irreversbile, then we have the kinetics

Comparing this system with that in Equation (29), the difference is in the 2E* on the left-hand-side. The kinetic equation for the fraction of E*, y = [ E * ] E t where E_t is the total amount of [E] + [E*]. is d y d t = α E t 2 y 2 ( 1 - y ) - ɛ y. The steady state exhibits a saddle-node bifurcation at θ = α E t 2 ɛ = 4:

See the orange curve in Figure 4.

Equation (38) is precisely the same kinetic equation, according to the Law of Mass Action, for the chemical reaction system

The system (41) is known as Schlögl’s model. It is the canonical example for nonlinear chemical bistability and bifurcation which has been studied for more than 30 years [46].

Qian and Reluga [47] have studied a system very similar to Equation (41) in terms of deterministic, nonlinear bifurcation theory. In particular, they established the important connection between the nonlinear bistability with nonequilibrium thermodynamics [48]. They have shown that if the concentrations of A and B are near equilibrium,

then there would be no bistability. The last equation in (42) is precisely equivalent to free energy change of ATP hydrolysis being zero. It can be easily shown, see Section 4.1, with the equilibrium condition in Equation (42), the system has only a single, unique deterministic steady state. And also, in terms of its CME, a single peak in the equilibrium probability for the number of X. This result is much more general for all nonlinear chemical and biochemical reaction systems, not just limited to the simple reaction system in (41) [36,49,50].

Vellela and Qian [36] have recently studied the Schlögl system in great detail, with a nonequilibrium steady state perspective. In particular, it was shown that the nonlinear bistability is intimately related to nonequilibrium phase transitions in statistical physics [36,51,52]. Ge and Qian [45,51] further investigated the steady state distribution according to the CME and its relationship to the steady states according to the deterministic Law of Mass Action. They have shown that in the limit of system’s volume tends infinity, i.e., the so called thermodynamic limit, the CME steady state(s) differ from that of deterministic model: A Maxwell construction like result is obtained: According to the CME, only one of the two determinsitic stable fixed point is the true global minimum, the other stable fixed point is only metastable. Hence in the thermodynamic limit, the global minimum has probability 1 while the metastable minimum has probability 0. However, the lifetime of the metastable state is infinitely long. Furthermore, using the mathematical tool of large deviation theory, [45] shows that the bistable CME system exhibits several key characteristics of nonequilibrium phase transition well-known in condensed matter physics.

For the kinetic scheme in Figure 3D, we again have χ = 2, and we again assume that there is a continuous biosynthesis for the E. However, we now consider the dynamics of both [E*] and [E], denoted by x and y, respectively. Then, the kinetic equations becomes

The system of Equations (43) is the same system for the kinetic scheme

with [A] = 1 and [B] = 1. If the β = ɛ = δ = 0, then it becomes the celebrated Schnakenberg model which is well-known to exhibit periodic chemical oscillation. Qian et al. [53] first studied its stochastic dynamics in terms of the CME. Recently, Vellela and Qian [54] again have studied this system. In particular, they have introduced a novel mathematical concept of Poincaré-Hill cycle map (PHCM) to characterize the amplitude of rotational random walk. The PHCM combines the Poincaré map from nonlinear dynamic analysis [55] with the cycle kinetic analysis developed by T.L. Hill [56,57].