# Information Entropy Production of Maximum Entropy Markov Chains from Spike Trains

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Generalities

#### 2.1. Notation

#### 2.2. Discrete-Time Markov Chains and Spike Train Statistics

#### 2.3. Detailed Balance Equations

#### 2.4. Information Entropy Rate and Information Entropy Production

## 3. Maximum Entropy Markov Chains

#### 3.1. Inference of the Maximum Entropy Markov Process

#### 3.2. Observables and Potentials

#### Additive Observables of Spike Trains

#### 3.3. Variational Principle

#### Statistical Inference

#### 3.4. Ruelle–Perron–Frobenius Transfer Operator

#### 3.5. Maximum Entropy Markov Chain for Finite Range Potentials

#### 3.6. IEP of the Inferred Markov Maximum Entropy Process

#### 3.7. Large Deviations for Observables of Maximum Entropy Markov Chains

#### 3.8. Large Deviations for the IEP

**Remark**

**1.**

#### Gallavotti–Cohen Fluctuation Theorem

## 4. Examples

#### 4.1. Example: Discrete Time Spiking Neuronal Network Model

- ${V}_{i}\left(t\right)<\theta $, for all $k=1,\dots ,N$. This corresponds to sub-threshold dynamics.
- There exists a k such that, ${V}_{k}\left(t\right)\ge \theta $. Corresponding to firing dynamics.

#### 4.2. MEMC Example: One Observable

#### 4.3. MEMC Example: Two Observables

#### 4.4. Example: Memoryless Potentials

#### 4.5. Example: 1-Time Step Markov with Random Coefficients

#### 4.6. Example: Kinetic Ising Model with Random Asymmetric Interactions

#### 4.7. Summary

## 5. Discussion

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## Abbreviations

MEP | Maximum entropy principle |

MEMC | Maximum entropy Markov chain |

IEP | Information entropy production |

KSE | Kolmogorov–Sinai entropy |

IF | Integrate-and-Fire |

GLM | Generalized Linear model |

NESS | Non-equilibrium steady states |

## Symbol List

${\sigma}_{k}^{n}$ | Spiking state of neuron k at time n. |

${\sigma}^{n}$ | Spike pattern at time n |

${\sigma}^{{n}_{1},{n}_{2}}$ | Spike block from time ${n}_{1}$ to ${n}_{2}$. |

${A}_{T}\left(f\right)$ | Empirical Average value of the observable f considering T spike patterns. |

${\mathsf{\Sigma}}_{N}^{L}$ | Set of spike blocks of N neurons and length L. |

$\mathcal{S}\left[\mu \right]$ | Entropy of the probability measure $\mu $. |

$\mathcal{H}$ | Potential function. |

$\mathcal{P}\left[\mathcal{H}\right]$ | Free energy or topological pressure. |

## References

- Lefebvre, B.; Yger, P.; Marre, O. Recent progress in multi-electrode spike sorting methods. J. Physiol. Paris
**2016**, 4, 327–335. [Google Scholar] [CrossRef] [PubMed] - Schneidman, E.; Freedman, B.; Segev, I. Ion channel stochasticity may be critical in determining the reliability and precision of spike timing. Neural Comput.
**1998**, 10, 1679–1703. [Google Scholar] [CrossRef] [PubMed] - Rieke, F.; Warland, D.; de Ruyter van Steveninck, R.; Bialek, W. Spikes, Exploring the Neural Code; MIT Press: Cambridge, MA, USA, 1996. [Google Scholar]
- Faisal, A.; Selen, L.; Wolpert, D. Noise in the nervous system. Nat. Rev. Neurosci.
**2008**, 9, 292–303. [Google Scholar] [CrossRef] [PubMed] - Borst, A.; Theunissen, F. Information theory and neural coding. Nat. Neurosci.
**1999**, 2, 947–957. [Google Scholar] [CrossRef] [PubMed] - Rolls, E.; Treves, A. The neuronal encoding of information in the brain. Prog. Neurobiol.
**2011**, 95, 448–490. [Google Scholar] [CrossRef] [PubMed] - Cafaro, J.; Rieke, F. Noise correlations improve response fidelity and stimulus encoding. Nature
**2010**, 468, 964–967. [Google Scholar] [CrossRef] [PubMed] - Pillow, J.; Paninski, L.; Uzzell, V.; Simoncelli, E.; Chichilnisky, E. Prediction and decoding of retinal ganglion cell responses with a probabilistic spiking model. J. Neurosci.
**2005**, 25, 11003–11013. [Google Scholar] [CrossRef] [PubMed] - Shadlen, M.; Newsome, W. The variable discharge of cortical neurons: Implications for connectivity, computation, and information coding. J. Neurosci.
**1998**, 18, 3870–3896. [Google Scholar] [PubMed] - Schneidman, E.; Berry, M.J.; Segev, R.; Bialek, W. Weak pairwise correlations imply string correlated network states in a neural population. Nature
**2006**, 440, 1007–1012. [Google Scholar] [CrossRef] [PubMed] - Nirenberg, S.; Latham, P. Decoding neuronal spike trains: How important are correlations. Proc. Natl. Acad. Sci. USA
**2003**, 100, 7348–7353. [Google Scholar] [CrossRef] [PubMed] - Nirenberg, S.; Latham, P. Population coding in the retina. Curr. Opin. Neurobiol.
**1998**, 8, 488–493. [Google Scholar] [CrossRef] - Ohiorhenuan, I.E.; Mechler, F.; Purpura, K.P.; Schmid, A.M.; Hu, Q.; Victor, J.D. Sparse coding and high-order correlations in fine-scale cortical networks. Nature
**2010**, 466, 617–621. [Google Scholar] [CrossRef] [PubMed] - Panzeri, S.; Schultz, S. A unified approach to the study of temporal, correlational, and rate coding. Neural Comput.
**2001**, 13, 1311–1349. [Google Scholar] [CrossRef] [PubMed] - Ganmor, E.; Segev, R.; Schneidman, E. The architecture of functional interaction networks in the retina. J. Neurosci.
**2011**, 31, 3044–3054. [Google Scholar] [CrossRef] [PubMed] - Moore, G.; Segundo, J.; Perkel, D.; Levitan, H. Statistical signs of synaptic interaction in neurons. Biophys. J.
**1970**, 10, 876–900. [Google Scholar] [CrossRef] - Moran, J.; Desimone, R. Selective attention gates visual processing in the extrastriate cortex. Science
**1985**, 229, 782–784. [Google Scholar] [CrossRef] [PubMed] - Nasser, H.; Cessac, B. Parameter estimation for spatio-temporal maximum entropy distributions: Application to neural spike trains. Entropy
**2014**, 16, 2244–2277. [Google Scholar] [CrossRef] [Green Version] - Tang, A.; Jackson, D.; Hobbs, J.; Chen, W.; Smith, J.; Patel, H.; Prieto, A.; Petrusca, D.; Grivich, M.; Sher, A.; et al. A maximum entropy model applied to spatial and temporal correlations from cortical networks in vitro. J. Neurosci.
**2008**, 28, 505–518. [Google Scholar] - Marre, O.; El Boustani, S.; Frégnac, Y.; Destexhe, A. Prediction of spatiotemporal patterns of neural activity from pairwise correlations. Phys. Rev. Lett.
**2009**, 102, 138101. [Google Scholar] [CrossRef] [PubMed] - Vasquez, J.; Palacios, A.; Marre, O.; Berry, M.J.; Cessac, B. Gibbs distribution analysis of temporal correlation structure on multicell spike trains from retina ganglion cells. J. Physiol. Paris
**2012**, 106, 120–127. [Google Scholar] [CrossRef] [PubMed] - Cofré, R.; Cessac, B. Exact computation of the maximum entropy potential of spiking neural networks models. Phys. Rev. E
**2014**, 89, 052117. [Google Scholar] [CrossRef] [PubMed] - Jaynes, E. Information theory and statistical mechanics. Phys. Rev.
**1957**, 106, 620. [Google Scholar] [CrossRef] - Pillow, J.W.; Shlens, J.; Paninski, L.; Sher, A.; Litke, A.M.; Chichilnisky, E.J.; Simoncelli, E.P. Spatio-temporal correlations and visual signaling in a complete neuronal population. Nature
**2008**, 454, 995–999. [Google Scholar] [CrossRef] [PubMed] - Tkačik, G.; Marre, O.; Amodei, D.; Schneidman, E.; Bialek, W.; Berry, M.J. Searching for collective behavior in a large network of sensory neurons. PLoS Comput. Biol.
**2013**, 10, e1003408. [Google Scholar] [CrossRef] [PubMed] - Tkačik, G.; Mora, T.; Marre, O.; Amodei, D.; Palmer, S.E.; Berry, M.J.; Bialek, W. Thermodynamics and signatures of criticality in a network of neurons. Proc. Natl. Acad. Sci. USA
**2015**, 112, 11508–11513. [Google Scholar] [CrossRef] [PubMed] - Shi, P.; Qian, H. Irreversible stochastic processes, coupled diffusions and systems biochemistry. In Frontiers in Computational and Systems Biology; Springer: London, UK, 2010; pp. 175–201. [Google Scholar]
- Jiang, D.Q.; Qian, M.; Qian, M.P. Mathematical Theory of Nonequilibrium Steady States; Springer: Berlin/Heidelberg, Germany, 2004. [Google Scholar]
- Bowen, R. Equilibrium States and the Ergodic Theory of Anosov Diffeomorphisms, revised edition; Springer: Berlin, Germany, 2008; Volume 470. [Google Scholar]
- Fernandez, R.; Maillard, G. Chains with complete connections: General theory, uniqueness, loss of memory and mixing properties. J. Stat. Phys.
**2005**, 118, 555–588. [Google Scholar] [CrossRef] - Cessac, B.; Cofré, R. Spike train statistics and Gibbs distributions. J. Physiol. Paris
**2013**, 107, 360–368. [Google Scholar] [CrossRef] [PubMed] - Galves, A.; Löcherbach, E. Infinite systems of interacting chains with memory of variable length—A stochastic model for biological neural nets. J. Stat. Phys.
**2013**, 151, 896–921. [Google Scholar] - Gaspard, P. Time-reversed dynamical entropy and irreversibility in Markovian random processes. J. Stat. Phys.
**2004**, 117, 599–615. [Google Scholar] - Kitchens, B.P. Symbolic Dynamics: One-Sided, Two-Sided and Countable State Markov Shifts; Springer: Berlin/Heidelberg, Germany, 1998. [Google Scholar]
- Pollard, B.S. Open Markov processes: A compositional perspective on non-equilibrium steady states in biology. Entropy
**2016**, 18, 140. [Google Scholar] [CrossRef] - Ruelle, D. Thermodynamic Formalism; Addison-Wesley: Reading, MA, USA, 1978. [Google Scholar]
- Van der Straeten, E. Maximum entropy estimation of transition probabilities of reversible Markov chains. Entropy
**2009**, 11, 867–887. [Google Scholar] [CrossRef] - Chliamovitch, G.; Dupuis, A.; Chopard, B. Maximum entropy rate reconstruction of Markov dynamics. Entropy
**2015**, 17, 3738–3751. [Google Scholar] [CrossRef] - Baladi, V. Positive Transfer Operators and Decay of Correlations; World Scientific: Singapore, 2000. [Google Scholar]
- Chazottes, J.R.; Ramirez, L.; Ugalde, E. Finite type approximations of Gibbs measures on sofic subshifts. Nonlinearity
**2005**, 18, 445–463. [Google Scholar] [CrossRef] - Maldonado, C.; Salgado-García, R. Markov approximations of Gibbs measures for long-range interactions on 1D lattices. J. Stat. Mech. Theory Exp.
**2013**, 8, P08012. [Google Scholar] [CrossRef] - Gantmacher, F.R. The Theory of Matrices; AMS Chelsea Publishing: Providence, RI, USA, 1998. [Google Scholar]
- Lancaster, P. Theory of Matrices; Academic Press: Cambridge, MA, USA, 1969. [Google Scholar]
- Cessac, B. A discrete time neural network model with spiking neurons. Rigorous results on the spontaneous dynamics. J. Math. Biol.
**2008**, 56, 311–345. [Google Scholar] [CrossRef] [PubMed] - Roudi, Y.; Hertz, J. Mean field theory for non-equilibrium network reconstruction. Phys. Rev. Lett.
**2011**, 106, 048702. [Google Scholar] [CrossRef] [PubMed] - Nasser, H.; Marre, O.; Cessac, B. Spatio-temporal spike trains analysis for large scale networks using maximum entropy principle and Monte-Carlo method. J. Stat. Mech.
**2013**, 2013, P03006. [Google Scholar] [CrossRef] - Monteforte, M.; Wolf, F. Dynamical entropy production in spiking neuron networks in the balanced state. Phys. Rev. Lett.
**2010**, 105, 268104. [Google Scholar] [CrossRef] [PubMed] - Gaspard, P. Time asymmetry in nonequilibrium statistical mechanics. Adv. Chem. Phys.
**2007**, 135, 83–133. [Google Scholar] - Delvenne, J.; Libert, A. Centrality measures and thermodynamic formalism for complex networks. Phys. Rev. E
**2011**, 83, 046117. [Google Scholar] [CrossRef] [PubMed]

**Figure 1.**Plot of the average value of IEP for 500 realizations of the synaptic weight matrix for fixed $\alpha $ and $\beta $ in each case. We fix the following values of the parameters: $N=6$, $\gamma =0.2$, ${\sigma}_{b}=1$, $\theta =1$, ${I}_{i}=1$ $\forall i\in \{1,\dots ,6\}$. The components of the synaptic weight matrix ${W}_{ij}$ were drawn at random from a normalized Gaussian distribution. We plot the average value of IEP for 500 realizations of the synaptic weight matrix for fixed $\alpha $ and $\beta $ in each case.

**Figure 2.**IEP and KSE as a function of ${h}_{1}$. In this example, the detailed balance condition is only satisfied in the trivial case ${h}_{1}=0$, corresponding to the uniform distribution. In all other cases, we obtain a MEMC with positive IEP, that is a NESS.

**Figure 3.**Gallavotti–Cohen fluctuation theorem for the MEMC example with one observable at the parameter value ${h}_{1}=-1$. Left: We show the SCGF associated to $W,{\lambda}_{W}\left(k\right)$, the derivative at zero is the IEP of the MEMC, which in this case is 0.0557. This value coincides with the minimum of the rate function ${I}_{W}\left(s\right)$ at the right side of the figure.

**Figure 4.**IEP for the MEMC build from each pair of constraints for the example Section 4.3. We use the restrictions on the average values denoted by ${c}_{1}$ and ${c}_{2}$ to build the corresponding MEMC in each case. We compute the IEP for each pair. In this figure, we illustrate that the IEP is zero only when both restrictions are equal and that the IEP increases with the difference in the restrictions.

**Figure 5.**Memoryless potentials do not distinguish shuffled nor time-inverted data sets. Illustrative scheme showing three different data sets sharing the same maximum entropy distribution $\pi $. (

**A**) We illustrate three data sets: on the top is the original; in the middle is the one obtained by randomly shuffle the time-indexes of the spike patterns; and on the bottom is the data set obtained by inverting the time indexes. (

**B**) For each of these datasets, we compute the firing rate of each neuron denoted by $\langle {\sigma}_{i}\rangle $ and the pairwise correlations $\langle {\sigma}_{i}{\sigma}_{j}\rangle $ obtaining for each data set the same average values. (

**C**) The spike train statistics of these three data sets are characterized by the same time-independent maximum entropy distribution $\pi $.

**Figure 6.**IEP for the 1-time step Markov potential. The parameters ${h}_{i}$ and ${J}_{ij}$ are draw at random one time and remain fixed. We draw at random the components of 100 matrices ${\gamma}_{ij}$ from a Gaussian distribution with different values of mean and standard deviation e. We plot the average value of IEP for each case, with the respective error bars.

**Figure 7.**IEP for the Kinetic Ising model with random asymmetric interactions. We consider $N=6$. The components of field vector were drawn at random from a Gaussian $\mathcal{N}(-3,1)$ and the coupling matrix ${J}_{ij}$ were drawn at random from a Gaussian $\mathcal{N}(0,1)$. We plot the average value of IEP for 500 realizations of the synaptic coupling matrix for fixed $\alpha $ and $\beta $ in each case.

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Cofré, R.; Maldonado, C.
Information Entropy Production of Maximum Entropy Markov Chains from Spike Trains. *Entropy* **2018**, *20*, 34.
https://doi.org/10.3390/e20010034

**AMA Style**

Cofré R, Maldonado C.
Information Entropy Production of Maximum Entropy Markov Chains from Spike Trains. *Entropy*. 2018; 20(1):34.
https://doi.org/10.3390/e20010034

**Chicago/Turabian Style**

Cofré, Rodrigo, and Cesar Maldonado.
2018. "Information Entropy Production of Maximum Entropy Markov Chains from Spike Trains" *Entropy* 20, no. 1: 34.
https://doi.org/10.3390/e20010034