This article is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

We develop the stochastic, chemical master equation as a unifying approach to the dynamics of biochemical reaction systems in a mesoscopic volume under a living environment. A living environment provides a continuous chemical energy input that sustains the reaction system in a nonequilibrium steady state with concentration fluctuations. We discuss the linear, unimolecular single-molecule enzyme kinetics, phosphorylation-dephosphorylation cycle (PdPC) with bistability, and network exhibiting oscillations. Emphasis is paid to the comparison between the stochastic dynamics and the prediction based on the traditional approach based on the Law of Mass Action. We introduce the difference between nonlinear bistability and stochastic bistability, the latter has no deterministic counterpart. For systems with nonlinear bistability, there are three different time scales: (

Quantitative modelling in terms of mathematical equations is the foundation of modern physical sciences. If one deals with mechanical motions or electromagentic issues of daily lives, he/she starts with Newton’s Second Law or Maxwell’s equations, respectively. For work on the subatomic and molecular level, we have the quantum mechanics of Heisenberg and Schrödinger for the small things, and Gibbs’ statistical mechanics for large collections of particles. The last theory on the list, Gibbs’ statistical thermodynamics, has been the foundation of molecular science [

However, it has long been recognized that Gibbs’ theory can not be applied to a system outside chemical equilibrium. In this case, and when the deviations from an equilibrium are linear, Onsager’s theory provides the unifying approach known as linear irreversible thermodynamics. However, cellular biologists have long been aware of that most living processes are not near an equilibrium, but far from it. This begs an answer to the question: What is the theory one should use in modelling a biochemical reaction system in its living environment?

Both Gibbs’ and Onsager’s work have pointed to a new type of mathematics: random variables and stochastic processes. Gibbs’ thermodynamic quantities with thermal fluctuations are random variables, and Onsager has used extensively Gaussian-Markov processes to describe the dynamics near an equilibrium [

Quantitative modelling in chemical engineering has been based on the Law of Mass Action [

Is there a theory which can embody all the above mentioned theories? It is clear that such a theory, even very imperfect, can provide great insights into the working of biochemical reaction systems in their living environment. While a consensus has not been reached, the recent rapid rise of applications of the Gillespie algorithm seems to suggest an interesting possibility.

It might be a surprise to some, but the Gillespie algorithm (GA)

It has been mathematically shown that the CME approach is the mesoscopic version of the Law of Mass Action [

We do not expect our readers to have a background in the CME. For a quick introduction see

In this article, we shall follow the CME approach to study biochemical reaction networks. We are particularly interested in such systems situated in a “living envirnment”. It turns out, one of the precise defining characteristics of the environment is the amount of chemical energy pumped into the system – similar to the battery in a radio.

While the CME approach is a new methodological advance in modelling open (driven) biochemical systems, a new concept also arises from recent studies on open (driven) biochemical systems: the

Equilibrium state with fluctuations which is well-understood according to Boltzmann’s law, and the theories of Gibbs, Einstein, and Onsager.

Time-dependent, transient processes in which systems are changing with time. In the past, this type of problems is often called “nonequilibrium problems”. As all experimentalists and computational modellers know, time-dependent kinetic experiments are very difficult to perform, and time-dependent equations are very difficult to analyze.

Nonequilibrium steady state: The system is no longer changing with time in a statistical sense,

To a first-order approximation, one can represent a biochemical cell or a subcellular network in homeostasis as a NESS. This is the theory being put forward by I. Prigogine, G. Nicolis and their Brussels group [

To have a better understanding of the nature of a NESS, we list three key characteristics of a system in equilibrium steady state: First, there is no flux in each and every reaction. This is known as the principle of detailed balance [

For more discussions on NESS and its applications to biochemical systems and modelling, the readers are referred to [

Since enzyme kinetics is the workhorse of biochemical reaction networks, let us start with the CME approach to the standard Michaelis-Menten (MM) enzyme reaction scheme:

In the CME approach to chemical and biochemical kinetics, one no longer asks what are the concentrations of _{S}_{ES}_{S}_{ES}

Assuming that the total number of substrate and product molecules is _{0} = _{S}_{P}_{ES}_{0} = _{E}_{ES}

There are three reactions in the kinetic scheme (1), hence there are six terms, three positive and three negative, in the CME (2). Note that the

One of the most important results in deterministic enzyme kinetic theory is the quasi-steady state approximation leading to the well-known Michaelis-Menten equation for the production of the product in

where _{t}

As pointed out by Kepler and Elston [_{0} << _{0}, then one can first solve the problem of steady state conditional distribution

In the first step, on a fast time scale for fixed

Here we assumed

which yields a mean value for

This result agrees exactly with the deterministic model.

Now the second step, let us sum over all the

we have

We see that at any given time,

This is exactly the CME version of

Even when the number of enzymes is not large, the product arrival time distribution contains no information more than the traditional Michaelis-Menten rate constant. However, this is not the case if there is truly only a single enzyme. This will be discussed below.

Now in _{0} = 1. Then the equation is reduced to

This is the CME for a single molecule enzyme kinetics according to the MM in

Carry out the summation on the both sides of

where

_{S}

The steady state probability for the single enzyme can be easily obtained from

Then the steady state single enzyme turnover flux is

with
_{max}_{2}. The last expression is precisely the Michaelis-Menten formula.

The single enzyme steady-state flux ^{ss}

We now consider an enzyme kinetic scheme that is a little more complex than that in

With concentrations _{S}_{P}

The origin of this flux is the non-equilibrium between the chemical potentials of

We see that when Δ^{ss}^{ss}^{ss}^{ss}

Let us see an example of ^{ss}_{i}_{−}_{i}_{P}_{S}

^{ss}_{B}^{4} +2λ^{3} +3λ^{2} +2λ+1). This is the region where Onsager’s theory applies. In fact, the linear coefficient between Δ^{ss}^{ss}^{+} − ^{−} and Δ_{B}^{+}^{−}). Then, when ^{ss}^{−}, we have

Note that in equilibrium, ^{+} = ^{−}. The last equation is known as

Therefore, the simple enzyme kinetics is not in the region with linear irreversibility. Onsager’s theory does not apply. Interestingly, we also note that the nonlinear curves in

In fact, one of the most important results in Onsager’s linear theory is the

To see an example, let us again consider _{1} = _{2} = _{3} = λ, and _{−1} = _{−2} = _{−3} = 1, and _{P}

The oscillations exist for

We see from _{S}

In a living cell, one of the most important, small biochemical regulatory networks is the phosphorylation-dephosphorylation cycle (PdPC) of an enzyme, first discovered by E.H. Fischer and E.G. Krebs in 1950s. It consists of only three players: a substrate enzyme, a kinase and a phosphatase. The phosphorylation of the substrate protein ^{5} M [

Many kinase itself can exist in two different forms: an inactive state and an active state. Furthermore, the conversion from the former to the latter involves the binding of the

where χ = 1, 2. We shall call χ = 1 first-order autocatalysis and χ = 2 second-order autocatalysis. Therefore, if the conversion is rapid, then the active kinase concentration is [^{‡}] = _{a}^{χ}. Now combining the reaction in ^{‡}, which in turn to make more

where [^{‡} also catalyzed the reverse reaction of the phosphorylation. Hence, to be more realistic, we have

in which

contains the concentration of

For the kinetic scheme in

here we use _{t}

represent the rates for the dephosphorylation and the rate for its reverse reaction, respectively. Both are catalyzed by the enzyme phosphatase

Bishop and Qian [

In [

Let _{t}

then

Let

and λ_{1} be the one of the two roots ∈ (0, 1). Since λ_{1}λ_{2} = −_{0} _{2}

The solution to

in which _{o}_{1}

If both phosphorylation and dephosphorylation reactions are irreversible, as usually assumed in cell biology (When considering kinetics, but not thermodynamics, this is indeed valid for large ATP hydrolysis free energy in a living cell), then the reaction is simplified to

where

Its steady state exhibits a transcritical bifurcation as a function of the activation signal, _{t}

Compared with the hypobolic activation curve
_{0.9}/_{0.1}, where _{0.9}) = 0.9 and _{0.1}) = 0.1.

It is interesting to point out that the curve in

where

Consider the first order autocatalytic system from

The stochastic model of this system was studied in depth by Bishop and Qian, [_{t}

where _{±}_{1} = _{±1}′/

Solving

where C is a normalization constant. For certain parameter regimes this distribution is bimodal where the bimodality appears as a sudden second peak at zero,

Note that this bistability is a purely stochastic phenomenon; it has no deterministic counterpart. The deterministic model of the same bi-molecular system in

The extrema of _{1}_{2})/(_{−1}_{−2}), of the system. If we consider _{−1}′/(_{1}′ _{t}_{2} −_{−2}) _{2}/(_{−2}_{t}_{−1} + _{−2})/(_{t}_{−1}_{−2}) with no upper bound,

For the kinetic scheme in

When

J. Keizer studied this model in [_{2}| and above, the system rapidly settles to a quasi-stationary distribution peaking at the deterministic positive steady state. However, in a much slower time scale corresponding to the eigenvalue |λ_{1}|, the above probability distribution slowly decay to zero. For very large reaction system volume _{1} ~ −^{−}^{cV}

Keizer’s paradox and its resolution is the origin of all the multi-scale dynamics in the CME system with multi-stability. It is also clear it is intimately related to the stochastic bistability in Section 3.2.2 when the _{−2}, _{−2} controls the lifetime,

For the kinetic scheme in

On the other hand, if we assume the rate of biosynthesis is negligible, and that both kinase and phosphatase catalyzed reactions are irreversbile, then we have the kinetics

Comparing this system with that in _{t}

See the orange curve in

The system (41) is known as Schlögl’s model. It is the canonical example for nonlinear chemical bistability and bifurcation which has been studied for more than 30 years [

Qian and Reluga [

then there would be no bistability. The last equation in (

Vellela and Qian [

For the kinetic scheme in

The system of

with [

So far, we have always assumed that the kinase catalysis is in its linear region, and avoided using Michaelis-Menten kinetic model for the kinase catalyzed phosphorylation. If we take the nonlinear Michaelis-Menten kinetics into account, interestingly, we discover that in this case, our model of PdPC with feedback in

In the glucolytic model, ^{‡} are the inactive and activated from of phosphofructokinase-1. One can find a nice nonlinear analysis of the deterministic model based on the Law of Mass Action in [

Nonlinear chemical reactions are the molecular basis of cellular biological processes and functions. Complex biochemical reactions in terms of enzymes and macromolecular complexes form “biochemical networks” in cellular control, regulation, and signaling. One of the central tasks of cellular systems biology is to quantify and integrate experimental observations into mathematical models that first repreduce and ultimately predict laboratory measurements. This review provides an introduction of the biochemical modeling paradigm in terms of the chemical master equation (CME) and explores the dynamical possibilities of various biochemical networks by considering models of homogenous,

The chemical master equation is a comprehensive mathematical theory that quantitatively characterize chemical and biochemical reaction system dynamics [

In recent years, due to the technological advances in optical imaging, single cell analysis, and green fluorescence proteins, experimental observations of biochemical dynamics inside single living cells have become increasingly quantitative [

Reaction kinetics of this kind are more realistically described by stochastic models that emphasize the discrete nature of molecular reactions and the randomness of their occurrences [

The master equation approach to chemical reactions began in the 1930’s with the work of M.A. Leontovich [

From a statistical mechanics point of view, each possible combination of the numbers of the chemical species defines a state of the system. The CME provides the evolution equation of the joint probability distribution function over all system states. In open chemical systems,

Continuous, diffusion approximations (also known as Fokker-Planck approximations) to the master equation were first developed by Van Kampen [

The same issue of exchanging limits is present also between a stochastic jump process and the deterministic model. It is intimately related to the time scales for “down-hill dynamics” and “up-hill dynamics” and how their dependence upon the system size ^{cV}

Kurtz carried out rigorous studies on the relation between the stochastic theory of chemical kinetics and its deterministic counterpart [

Stochastic simulations of complex chemical reaction systems were carried out as early as the 1970’s [

In the environment of a living cell, biochemical systems are operating under a driven condition, widely called an “open system” [

The nonequilibrium theory for nonlinear biochemical reactions allows the possibility of multiple steady states, and nonzero steady state flux and a nonzero entropy production rate [

The essential difference between deterministic and stochastic models is the permanence of fixed points. According to the theory of ordinary differential equation, once the system reaches a fixed point (or an attractor), it must remain there for all time. Systems with stochasticity, however, can have trajectories being pushed away from attracting fixed points by random fluctuations. Since the noise is ever-present, it can eventually push the system out of the basin of attraction of one fixed point (attractor) and into that of another. Fixed points are no longer stationary for all time; they are only temporary, or “quasistationary” [

In order to systematically understand the mesoscopic cellular biochemical dynamics, this review discussed the simplest problem that is interesting: a one dimensional system with two fixed points. The systems with only one fixed point are trivial since deterministic and stochastic models are in complete agreement when there is a unique steady state [

Logically, the next step is a one dimensional system with three fixed points, two stable with one unstable point between them [

Once the theory has been established for one dimensional systems with a single dynamic biochemical species, we turn our attention to planar systems with two dynamical species [

In all these studies one encounters the presence of a time scale that grows exponentially with the sysetm’s volume

In summary, one of the most important insights from the CME study of biochemical reaction systems in a small, cellular volume is the realization of the

We also thank Ping Ao, Hao Ge, Kyung Kim, Jie Liang, Zhilin Qu, Michael Samoilov, Herb Sauro, Melissa Vellela, Jin Wang, and Sunney Xie for many helpful discussions. The work reported here was supported in part by NSF grant No. EF0827592.

The canonical MM kinetic scheme is

Let _{0}. Then, the corresponding

We are interested in the autocatalytic reaction system

in which the concentrations of _{−1} = 0. Let _{1} and _{−1} have units of [volume][time]^{−1}, and _{2} has units of [time]^{−1}. The reaction rates in the stochastic model are related to these rates by

These reaction rates are scaled such that the units agree in the master equation (see

where _{a}

The canonical Schlögl model for chemical bistability is [

Following the _{n}_{X}

where

and

By setting the right-hand-side of

where

We now show a very interesting and important property of the _{n}^{ss}_{1}_{3}/(_{2}_{4}) = _{1}_{3}/(_{2}_{4}). Substituting this relation into

This is a Poisson distribution with the mean number of

We are now interested in the nonlinear chemical reaction system, the reversible Schnakenberg model, in a mesoscopic volume

Consider the function _{n,m}^{+} × ℤ^{+} (see

for _{n,m}^{i}_{n,m}^{i}

The factor of 1/^{2} in λ_{n,m}^{3} and _{n,m}^{3} accounts for the fact that the third reaction is trimolecular, and thus _{3} and _{−3} have units of ^{2}/

Because the CME is a set of linear ODEs, there will be a unique steady state to which the system tends, the probability steady state, ^{ss}^{ss}

Through quasi-statioanry approximation, the CME in

Simple enzyme kinetic system in ^{ss}_{1} = _{2} = _{3} = λ, _{−1} = _{−2} = _{−3} = 1, and _{P}^{ss}_{B}^{3}_{S}_{S}_{S}_{S}^{eq}

An assorted variations of the PdPC with autocatalytic feedback. The phosphorylation of the substrate ^{‡}, and the dephosphorylation is catalyzed by a phosphatase (^{‡}; In ^{‡}. The nonlinear feedback in the latter is stronger; thus they exhibit more pronounced nonlinear behavior: bistability and limit cycle oscillation.

Activation curves of PdPC with or without autocatalytic phosphorylation

_{1} = 5, _{−1} = 10, _{2} = 10, _{t}_{−2} varied. _{1} = 5, _{−1} = 10, _{2} = 10, _{−2} = 0.001, _{t}

The chemical master equation _{−1}(_{1}(_{0} −_{2}(

The chemical master equation _{n}_{a}

The chemical master equation

The chemical master equation _{n,m}^{i}_{n,m}^{i}