# Open Markov Processes: A Compositional Perspective on Non-Equilibrium Steady States in Biology

## Abstract

**:**

## 1. Introduction

## 2. Open Markov Processes

**open Markov process**, or open, continuous time, discrete state Markov chain, is a triple $(V,B,H)$ where V is a finite set of

**states**, $B\subseteq V$ is the subset of

**boundary states**and $H:{\mathbb{R}}^{V}\to {\mathbb{R}}^{V}$ is an

**infinitesimal stochastic Hamiltonian**

**population**at the $i\text{th}$ state. We call the resulting function $p:V\to [0,\infty )$ the

**population distribution**. Populations evolve in time according to the

**open master equation**

**steady state**distribution is a population distribution which is constant in time:

**closed Markov process**, or continuous time, discrete state Markov chain, is an open Markov process whose boundary is empty. For a closed Markov process, the open master equation becomes the usual master equation

**equilibrium**. We say an equilibrium $q\in {[0,\infty )}^{V}$ of a Markov process is

**detailed balanced**if

**open detailed balanced Markov process**is an open Markov process $(V,B,H)$ together with a detailed balanced equilibrium $q:V\to (0,\infty )$ on V. In Section 5, we define the “dissipation”, which depends on the detailed balanced equilibrium populations, hence we equip an open Markov process with a specific detailed balanced equilibrium of the underlying closed Markov process. Thus, if a Markov process admits multiple detailed balanced equilibria, we choose a specific one. Note that we consider only detailed balanced equilibria such that the populations of all states are non-zero. Later, it will become clear why this is important.

**net flow**of population from the $j\text{th}$ state to the $i\text{th}$ is

**net inflow**${J}_{i}\left(p\right)\in \mathbb{R}$ of a particular state to be

**Kolmogorov’s criterion**[2], namely that

**non-equilibrium steady state**is a steady state in which the net flow between at least one pair of states is non-zero. Thus, there could be population flowing between pairs of states, but in such a way that these flows still yield constant populations at all states. In a closed Markov process, the existence of non-equilibrium steady states requires that the rates of the Markov process violate Kolmogorov’s criterion. We show that open Markov processes with constant boundary populations admit non-equilibrium steady states even when the rates of the process satisfy Kolmogorov’s criterion. Throughout this paper, we use the term equilibrium to mean detailed balanced equilibrium.

## 3. Membrane Diffusion as an Open Markov Process

**boundary populations**. Given the values of ${p}_{A}$ and ${p}_{C}$, the steady state population ${p}_{B}$ compatible with these values is

## 4. The Category of Open Detailed Balanced Markov Processes

**finite set with populations**, i.e., a finite set X together with a map ${p}_{X}:X\to [0,\infty )$ assigning a population ${p}_{i}\in [0,\infty )$ to each element $i\in X$. A morphism $M:(X,{p}_{X})\to (Y,{p}_{Y})$ consists of an open detailed balanced Markov process $(V,B,H,q)$ together with

**input**and

**output**maps $i:X\to V$ and $o:Y\to V$ which

**preserve population**, i.e., ${p}_{X}=q\circ i$ and ${p}_{Y}=q\circ i$. The union of the images of the input and output maps form the boundary of the open Markov processes $B=i\left(X\right)\cup o\left(Y\right)$.

## 5. Principle of Minimum Dissipation

**Definition 1.**

**dissipation functional**of a population distribution p to be

**Definition 2.**

**principle of minimum dissipation with boundary population**$\mathit{b}$ if p minimizes $D\left(p\right)$ subject to the constraint that ${p|}_{b}=b$.

**Theorem 3.**

**Proof.**

**Definition 4.**

**steady state population-flow pair**if the flows arise from a population distribution which obeys the principle of minimum dissipation.

**Definition 5.**

**behavior**of an open detailed balanced Markov process with boundary B is the set of all steady state population-flow pairs $({p}_{B},{J}_{B})$ along the boundary.

## 6. Dissipation and Entropy Production

**relative entropy**of two distributions $p,q$ is given by

**thermodynamic flux**from j to i and

**thermodynamic force**. This quantity:

**molar fraction**of the $i\text{th}$ species with ${n}_{i}$ giving the number of moles of the $i\text{th}$ species [13]. Note that this is equal to the fraction of the population in the $i\text{th}$ state ${x}_{i}=\frac{{n}_{i}}{{\sum}_{i}{n}_{i}}=\frac{{p}_{i}}{{\sum}_{i}{p}_{i}}$. The difference in chemical potential between two states gives the force associated with the flow which seeks to reduce this difference in chemical potential

## 7. Minimum Dissipation versus Minimum Entropy Production

## 8. Discussion

## Acknowledgments

## Conflicts of Interest

## References

- Kingman, J.F.C. Markov population processes. J. Appl. Probab.
**1969**, 6, 1–18. [Google Scholar] [CrossRef] - Kelly, F.P. Reversibility and Stochastic Networks; Cambridge University Press: Cambridge, UK, 2011. [Google Scholar]
- Oster, G.; Perelson, A.; Katchalsky, A. Network thermodynamics: Dynamic modeling of biophysical systems. Q. Rev. Biophys.
**1973**, 1, 1–134. [Google Scholar] - Schnakenberg, J. Thermodynamic Network Analysis of Biological Systems; Springer: Berlin, Germany, 1981. [Google Scholar]
- Baez, J.C.; Fong, B.; Pollard, B. A compositional framework for open Markov processes. J. Math. Phys.
**2016**, 57, 033301. [Google Scholar] - Schnakenberg, J. Network theory of microscopic and macroscopic behavior of master equation systems. Rev. Mod. Phys.
**1976**, 48, 571–585. [Google Scholar] - Oster, G.; Perelson, A.; Katchalsky, A. Network thermodynamics. Nature
**1971**, 234, 393–399. [Google Scholar] - Perelson, A.; Oster, G. Chemical reaction networks. IEEE Trans. Circuits Sys.
**1974**, 21, 709–721. [Google Scholar] - Hill, T.L. Free Energy Transduction in Biology: The Steady-State Kinetic and Thermodynamic Formalism; Academic Press: New York, NY, USA, 1977. [Google Scholar]
- Hill, T.L. Free Energy Transduction and Biochemical Cycle Kinetics; Springer-Verlag: New York, NY, USA, 1989. [Google Scholar]
- Hill, T.L.; Eisenberg, E. Muscle contraction and free energy transduction in biological systems. Science
**1985**, 227, 999–1006. [Google Scholar] - Glandsorf, P.; Prigogine, I. Thermodynamic Theory of Structure, Stability and Fluctuations; Wiley-Interscience: New York, NY, USA, 1971. [Google Scholar]
- De Groot, S.R.; Mazur, P. Non-Equilibrium Thermodynamics; North-Holland Publishing Company: Amsterdam, The Netherlands, 1962. [Google Scholar]
- Lindblad, C. Non-Equilibrium Entropy and Irreversibility; D. Reidel Publishing Company: Dordrecht, the Netherland, 1983. [Google Scholar]
- Prigogine, I. Non-Equilibrium Statistical Mechanics; Interscience Publishers: New York, NY, USA, 1962. [Google Scholar]
- Prigogine, I. Etudé Thermodynamique des Phénoménes Irréversibles; Dunod: Paris, France, 1947. (In French) [Google Scholar]
- Jiang, D.; Qian, M.; Qian, M.P. Mathematical Theory of Nonequilibrium Steady States; Springer: Berlin, Germany, 2004. [Google Scholar]
- Andrieux, D.; Gaspard, P. Fluctuation theorem for currents and Schnakenberg network theory. J. Stat. Mech. Theory Exp.
**2006**, 127, 107–131. [Google Scholar] - Qian, H. Open-system nonequilibrium steady state: statistical thermodynamics, fluctuations, and chemical oscillations. J. Phys. Chem. B
**2006**, 31, 15063–15074. [Google Scholar] - Qian, H.; Beard, D.A. Thermodynamics of stoichiometric biochemical networks in living systems far from equilibrium. Biophys. Chem.
**2005**, 114, 213–220. [Google Scholar] - Qian, H.; Bishop, L. The chemical master equation approach to nonequilibrium steady-state of open biochemical systems: Linear single-molecule enzyme kinetics and nonlinear biochemical reaction networks. Int. J. Mol. Sci.
**2010**, 11, 3472–3500. [Google Scholar] - Baez, J.C.; Eberle, J. Categories in Control. Theory Appl. Categ.
**2015**, 30, 836–881. [Google Scholar] - Baez, J.C.; Fong, B. A compositional framework for passive linear networks. 2015; arxiv.org/abs/1504.05625. [Google Scholar]
- Fong, B. Decorated cospans. Theory Appl. Categ.
**2015**, 30, 1096–1120. [Google Scholar] - Baez, J.C.; Pollard, B. Relative entropy in biological systems. Entropy
**2016**, 18, 46. [Google Scholar] - Pollard, B. A Second Law for open Markov processes. Open Syst. Inf. Dyn.
**2015**, 23, 1650006. [Google Scholar] - Bruers, S.; Maes, C.; Netočný, K. On the validity of entropy production principles for linear electrical circuits. J. Stat. Phys.
**2007**, 129, 725–740. [Google Scholar] - Landauer, R. Inadequacy of entropy and entropy derivatives in characterizing the steady state. Phys. Rev. A
**1975**, 12, 636–638. [Google Scholar] - Landauer, R. Stability and entropy production in electrical circuits. J. Stat. Phys.
**1975**, 13, 1–16. [Google Scholar] - Poletinni, M.; Esposito, M. Irreversible thermodynamics of open chemical networks I: Emergent cycles and broken conservation laws. J. Chem. Phys.
**2014**, 141, 024117. [Google Scholar]

**Figure 4.**Another layer of membrane whose interior population is labeled by D and whose exterior populations are labeled by ${C}^{\prime}$ and E.

**Figure 6.**Composition of open detailed balanced Markov processes results in an open detailed balanced Markov process.

© 2016 by the author; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Pollard, B.S. Open Markov Processes: A Compositional Perspective on Non-Equilibrium Steady States in Biology. *Entropy* **2016**, *18*, 140.
https://doi.org/10.3390/e18040140

**AMA Style**

Pollard BS. Open Markov Processes: A Compositional Perspective on Non-Equilibrium Steady States in Biology. *Entropy*. 2016; 18(4):140.
https://doi.org/10.3390/e18040140

**Chicago/Turabian Style**

Pollard, Blake S. 2016. "Open Markov Processes: A Compositional Perspective on Non-Equilibrium Steady States in Biology" *Entropy* 18, no. 4: 140.
https://doi.org/10.3390/e18040140