# Information Landscape and Flux, Mutual Information Rate Decomposition and Connections to Entropy Production

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Bivariate Markov Chains

## 3. Information Landscape and Information Flux for Determining the Information Dynamics, Time-Irreversibility

## 4. Mutual Information Decomposition to Time-Reversible and Time-Irreversible Parts

## 5. Relationship between Mutual Information and Entropy Production

## 6. A Simple Case: The Blind Demon

## 7. Conclusions

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## Abbreviations

BMC | Bivariate Markov Chain |

EPR | Entropy Production Rate |

MIR | Mutual Information Rate |

SS | Steady State |

## Appendix A

## Appendix B

#### Appendix B.1 Discussions on the Generality of Mutual Information Rate Decomposition and Connections to Entropy Production in Terms of Equations (14), (17), and (18)

#### Appendix B.2. The Smart Demon

$\{{\u03f5}_{a}(x=0),{\u03f5}_{a}(x=1),{\u03f5}_{a}(x=2)\}$ | $\{{\u03f5}_{b}(x=0),{\u03f5}_{b}(x=1),{\u03f5}_{b}(x=2)\}$ | |

Group 1 | $\{0.2344,0.2730,0.4926\}$ | $\{0.4217,0.4094,0.1689\}$ |

Group 2 | $\{0.1305,0.3972,0.4723\}$ | $\{0.3358,0.0010,0.6633\}$ |

$\left\{d\right(s=a|x=0,s=a),d(s=b|x=0,s=a\left)\right\}$ | $\left\{d\right(s=a|x=1,s=b),d(s=b|x=0,s=b\left)\right\}$ | |

Group 1 | $\{0.3844,0.6156\}$ | $\{0.6811,0.3189\}$ |

Group 2 | $\{0.1072,0.8928\}$ | $\{0.7473,0.2527\}$ |

$\left\{d\right(s=a|x=1,s=a),d(s=b|x=1,s=a\left)\right\}$ | $\left\{d\right(s=a|x=1,s=b),d(s=b|x=1,s=b\left)\right\}$ | |

Group 1 | $\{0.5195,0.4805\}$ | $\{0.8088,0.1912\}$ |

Group 2 | $\{0.6595,0.3405\}$ | $\{0.1600,0.8400\}$ |

$\left\{d\right(s=a|x=2,s=a),d(s=b|x=2,s=a\left)\right\}$ | $\left\{d\right(s=a|x=2,s=b),d(s=b|x=2,s=b\left)\right\}$ | |

Group 1 | $\{0.3775,0.6225\}$ | $\{0.3340,0.6660\}$ |

Group 2 | $\{0.0232,0.9768\}$ | $\{0.0814,0.9186\}$ |

${\mathbf{R}}_{\mathbf{z}}$ | ${\mathbf{R}}_{\mathbf{x}}$ | $\mathbf{I}\mathbf{(}\mathbf{X}\mathbf{,}\mathbf{S}\mathbf{)}$ | ${\mathbf{I}}_{\mathbf{B}}\mathbf{\left(}\mathbf{X}\mathbf{,}\mathbf{S}\mathbf{\right)}$ | |
---|---|---|---|---|

Group 1 | $0.0645$ | $0.0018$ | $0.0885$ | $0.0313$ |

Group 2 | $0.5485$ | $0.1291$ | $0.3385$ | $0.2097$ |

## References

- Shannon, C.E. A mathematical theory of communication. Bell Syst. Tech. J.
**1948**, 27, 379–423. [Google Scholar] [CrossRef] - Ball, F.; Yeo, G.F. Lumpability and Marginalisability for Continuous-Time Markov Chains. J. Appl. Probab.
**1993**, 30, 518–528. [Google Scholar] [CrossRef] - Wei, W.; Wang, B.; Towsley, D. Continuous-time hidden Markov models for network performance evaluation. Perform. Eval.
**2002**, 49, 129–146. [Google Scholar] [CrossRef] - Strasberg, P.; Schaller, G.; Brandes, T.; Esposito, M. Thermodynamics of a physical model implementing a Maxwell demon. Phys. Rev. Lett.
**2013**, 110, 040601. [Google Scholar] [CrossRef] [PubMed] - Koski, J.V.; Kutvonen, A.; Khaymovich, I.M.; Ala-Nissila, T.; Pekola, J.P. On-Chip Maxwell’s Demon as an Information-Powered Refrigerator. Phys. Rev. Lett.
**2015**, 115, 260602. [Google Scholar] [CrossRef] [PubMed] - Mcgrath, T.; Jones, N.S.; Ten Wolde, P.R.; Ouldridge, T.E. Biochemical Machines for the Interconversion of Mutual Information and Work. Phys. Rev. Lett.
**2017**, 118, 028101. [Google Scholar] [CrossRef] [PubMed] - Mark, B.L.; Ephraim, Y. An EM algorithm for continuous-time bivariate Markov chains. Comput. Stat. Data Anal.
**2013**, 57, 504–517. [Google Scholar] [CrossRef] - Ephraim, Y.; Mark, B.L. Bivariate Markov Processes and Their Estimation. Found. Trends Signal Process.
**2012**, 6, 1–95. [Google Scholar] [CrossRef] - Cover, T.M.; Thomas, J.A. Elements of Information Theory, 2nd ed.; John Wiley & Sons: Hoboken, NJ, USA, 2006; ISBN 13-978-0-471-24195-9. [Google Scholar]
- Parrondo, J.M.R.; Horowitz, J.M.; Sagawa, T. Thermodynamics of information. Nat. Phys.
**2015**, 11, 131–139. [Google Scholar] [CrossRef] - Sagawa, T.; Ueda, M. Fluctuation theorem with information exchange: Role of correlations in stochastic thermodynamics. Phys. Rev. Lett.
**2012**, 109, 180602. [Google Scholar] [CrossRef] [PubMed] - Horowitz, J.M.; Esposito, M. Thermodynamics with Continuous Information Flow. Phys. Rev. X
**2014**, 4, 031015. [Google Scholar] [CrossRef] - Barato, A.C.; Hartich, D.; Seifert, U. Rate of Mutual Information Between Coarse-Grained Non-Markovian Variables. J. Stat. Phys.
**2013**, 153, 460–478. [Google Scholar] [CrossRef] - Wang, J.; Xu, L.; Wang, E.K. Potential landscape and flux framework of nonequilibrium networks: Robustness, dissipation, and coherence of biochemical oscillations. Proc. Natl. Acad. Sci. USA
**2008**, 105, 12271–12276. [Google Scholar] [CrossRef] [PubMed] - Wang, J. Landscape and flux theory of non-equilibrium dynamical systems with application to biology. Adv. Phys.
**2015**, 64, 1–137. [Google Scholar] [CrossRef] - Li, C.H.; Wang, E.K.; Wang, J. Potential flux landscapes determine the global stability of a Lorenz chaotic attractor under intrinsic fluctuations. J. Chem. Phys.
**2012**, 136, 194108. [Google Scholar] [CrossRef] [PubMed] - Schnakenberg, J. Network theory of microscopic and macroscopic behavior of master equation systems. Rev. Mod. Phys.
**1976**, 48, 571–585. [Google Scholar] [CrossRef] - Zia, R.K.P.; Schmittmann, B. Probability currents as principal characteristics in the statistical mechanics of non-equilibrium steady states. J. Stat. Mech.-Theory E
**2007**, 2007. [Google Scholar] [CrossRef] - Maes, C.; Netočný, K. Canonical structure of dynamical fluctuations in mesoscopic nonequilibrium steady states. Europhys. Lett.
**2008**, 82. [Google Scholar] [CrossRef] - Qian, M.P.; Qian, M. Circulation for recurrent markov chains. Probab. Theory Relat.
**1982**, 59, 203–210. [Google Scholar] - Zhang, Z.D.; Wang, J. Curl flux, coherence, and population landscape of molecular systems: Nonequilibrium quantum steady state, energy (charge) transport, and thermodynamics. J. Chem. Phys.
**2014**, 140, 245101. [Google Scholar] [CrossRef] [PubMed] - Zhang, Z.D.; Wang, J. Landscape, kinetics, paths and statistics of curl flux, coherence, entanglement and energy transfer in non-equilibrium quantum systems. New J. Phys.
**2015**, 17, 043053. [Google Scholar] [CrossRef] - Luo, X.S.; Xu, L.F.; Han, B.; Wang, J. Funneled potential and flux landscapes dictate the stabilities of both the states and the flow: Fission yeast cell cycle. PLoS Comput. Biol.
**2017**, 13, e1005710. [Google Scholar] [CrossRef] [PubMed] - Gray, R.; Kieffer, J. Mutual information rate, distortion, and quantization in metric spaces. IEEE Trans. Inf. Theory
**1980**, 26, 412–422. [Google Scholar] [CrossRef] - Maes, C.; Redig, F.; van Moffaert, A. On the definition of entropy production, via examples. J. Math. Phys.
**2000**, 41, 1528–1554. [Google Scholar] [CrossRef] - Gaspard, P. Time-reversed dynamical entropy and irreversibility in Markovian random processes. J. Stat. Phys.
**2004**, 117, 599–615. [Google Scholar] [CrossRef] - Feng, H.D.; Wang, J. Potential and flux decomposition for dynamical systems and non-equilibrium thermodynamics: Curvature, gauge field, and generalized fluctuation-dissipation theorem. J. Chem. Phys.
**2011**, 135, 234511. [Google Scholar] [CrossRef] [PubMed] - Polettini, M. Nonequilibrium thermodynamics as a gauge theory. Europhys. Lett.
**2012**, 97, 30003. [Google Scholar] [CrossRef]

© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Zeng, Q.; Wang, J. Information Landscape and Flux, Mutual Information Rate Decomposition and Connections to Entropy Production. *Entropy* **2017**, *19*, 678.
https://doi.org/10.3390/e19120678

**AMA Style**

Zeng Q, Wang J. Information Landscape and Flux, Mutual Information Rate Decomposition and Connections to Entropy Production. *Entropy*. 2017; 19(12):678.
https://doi.org/10.3390/e19120678

**Chicago/Turabian Style**

Zeng, Qian, and Jin Wang. 2017. "Information Landscape and Flux, Mutual Information Rate Decomposition and Connections to Entropy Production" *Entropy* 19, no. 12: 678.
https://doi.org/10.3390/e19120678