# Minimum and Maximum Entropy Distributions for Binary Systems with Known Means and Pairwise Correlations

^{1}

^{2}

^{3}

^{4}

^{5}

^{6}

^{7}

^{*}

## Abstract

**:**

## 1. Introduction

#### Problem Setup

## 2. Results

#### 2.1. Limits on System Growth

#### A Simple Example

#### 2.2. Bounds on Minimum Entropy

#### 2.2.1. Upper Bound on the Minimum Entropy

#### 2.2.2. Lower Bound on the Minimum Entropy

#### 2.3. Bounds on Maximum Entropy

#### 2.4. Low-Entropy Solutions

#### 2.5. Minimum Entropy for Exchangeable Distributions

#### 2.6. Implications for Communication and Computer Science

## 3. Discussion

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## Appendix A. Allowed Range of ν Given μ Across All Distributions for Large N

**Figure A1.**An example of the allowed values of ${\lambda}_{0}$ and ${\lambda}_{1}$ for the dual problem ($N=5$).

**Figure A2.**The red shaded region is the set of values for $\mu $ and $\nu $ that can be satisfied for at least one probability distribution in the $N\to \infty $ limit. The purple line along the diagonal where $\nu =\mu $ is the distribution for which only the all active and all inactive states have non-zero probability. It represents the global entropy minimum for a given value of $\mu $. The red parabola, $\nu ={\mu}^{2}$, at the bottom border of the allowed region corresponds to a wide range of probability distributions, including the global maximum entropy solution for given $\mu $ in which each neuron fires independently. We find that low entropy solutions reside at this low $\nu $ boundary as well.

## Appendix B. Minimum Entropy Occurs at Small Support

## Appendix C. The Maximum Entropy Solution

`fsolve`function from the

`SciPy`package of Python subject to constraints (A34) and (A35).

**Figure A3.**For the case of uniform constraints achievable by arbitrarily large systems, the maximum possible entropy scales linearly with system size, N, as shown here for various values of $\mu $ and $\nu $. Note that this linear scaling holds even for large correlations, provided that $\nu <\mu $. For the extreme case $\nu =\mu $, all the neurons are perfectly correlated so the entropy of the ensemble does not change with increasing N.

## Appendix D. Minimum Entropy for Exchangeable Probability Distributions

**Figure A4.**The minimum entropy for exchangeable distributions versus N for various values of $\mu $ and $\nu $. Note that, like the maximum entropy, the exchangeable minimum entropy scales linearly with N as $N\to \infty $, albeit with a smaller slope for $\nu \ne {\mu}^{2}$. We can calculate the entropy exactly for $\mu $ = 0.5 and $\nu $ = 0.25 as $N\to \infty $, and we find that the leading term is indeed linear: ${\tilde{S}}_{2}^{exch}\approx N-1/2{log}_{2}\left(N\right)-1/2{log}_{2}\left(2\pi \right)+O[{log}_{2}\left(N\right)/N]$.

## Appendix E. Construction of a Low Entropy Distribution for All Values of μ and ν

- Begin with the state with $n=3$ active neurons in a row:
- 11100

- Generate new states by inserting progressively larger gaps of 0s before each 1 and wrapping active states that go beyond the last neuron back to the beginning. This yields $N-1=4$ unique states including the original state:
- 11100
- 10101
- 11010
- 10011

- Finally, “rotate” each state by shifting each pattern of ones and zeros to the right (again wrapping states that go beyond the last neuron). This yields a total of $N(N-1)$ states:
- 11100 01110 00111 10011 11001
- 10101 11010 01101 10110 01011
- 11010 01101 10110 01011 10101
- 10011 11001 11100 01110 00111

- Note that each state is represented twice in this collection, removing duplicates we are left with $N(N-1)/2$ total states. By inspection we can verify that each neuron is active in $n(N-1)/2$ states and each pair of neurons is represented in $n(n-1)/2$ states. Weighting each state with equal probability gives us the values for $\mu $ and $\nu $ stated in Equation (A53).

## Appendix F. Another Low Entropy Construction for the Communications Regime, $\mathit{\mu}=\raisebox{1ex}{$\mathbf{1}$}\!\left/ \!\raisebox{-1ex}{$\mathbf{2}$}\right.$ & $\mathit{\nu}=\raisebox{1ex}{$\mathbf{1}$}\!\left/ \!\raisebox{-1ex}{$\mathbf{4}$}\right.$

## Appendix G. Extending the Range of Validity for the Constructions

**Figure A5.**The full shaded region includes all allowed values for the constraints $\mu $ and $\nu $ for all possible probability distributions, replotted from Figure A2. As described in Appendix E and Appendix G, one of our low-entropy constructed solutions can be matched to any of the allowed constraint values in the full shaded region, whereas the constructed solution described in Appendix F can achieve any of the values within the triangular purple shaded region. Note that even with this second solution, we can cover most of the allowed region. Each of our constructed solutions have entropies that scale as $S\sim ln\left(N\right)$.

## Appendix H. Proof of the Lower Bound on Entropy for Any Distribution Consistent with Given $\left\{{\mathit{\mu}}_{\mathit{i}}\right\}$ & $\left\{{\mathit{\nu}}_{\mathit{i}\mathit{j}}\right\}$

## References

- Pathria, R. Statistical Mechanics; Butterworth Hein: Oxford, UK, 1972. [Google Scholar]
- Russ, W.; Lowery, D.; Mishra, P.; Yaffe, M.; Ranganathan, R. Natural-like function in artificial WW domains. Nature
**2005**, 437, 579–583. [Google Scholar] [CrossRef] [PubMed] - Socolich, M.; Lockless, S.W.; Russ, W.P.; Lee, H.; Gardner, K.H.; Ranganathan, R. Evolutionary information for specifying a protein fold. Nature
**2005**, 437, 512–518. [Google Scholar] [CrossRef] [PubMed] - Mora, T.; Walczak, A.M.; Bialek, W.; Callan, C.G. Maximum entropy models for antibody diversity. Proc. Natl. Acad. Sci. USA
**2010**, 107, 5405–5410. [Google Scholar] [CrossRef] [PubMed] - Schneidman, E.; Berry, M.J.; Segev, R.; Bialek, W. Weak pairwise correlations imply strongly correlated network states in a neural population. Nature
**2006**, 440, 1007–1012. [Google Scholar] [CrossRef] [PubMed] - Shlens, J.; Field, G.D.; Gauthier, J.L.; Grivich, M.I.; Petrusca, D.; Sher, A.; Litke, A.M.; Chichilnisky, E.J. The structure of multi-neuron firing patterns in primate retina. J. Neurosci.
**2006**, 26, 8254–8266. [Google Scholar] [CrossRef] [PubMed] - Tkacik, G.; Schneidman, E.; Berry, I.; Michael, J.; Bialek, W. Ising models for networks of real neurons. arXiv, 2006; arXiv:q-bio/0611072. [Google Scholar]
- Tang, A.; Jackson, D.; Hobbs, J.; Chen, W.; Smith, J.; Patel, H.; Prieto, A.; Petrusca, D.; Grivich, M.I.; Sher, A.; et al. A maximum entropy model applied to spatial and temporal correlations from cortical networks in vitro. J. Neurosci.
**2008**, 28, 505–518. [Google Scholar] [CrossRef] [PubMed] - Shlens, J.; Field, G.D.; Gauthier, J.L.; Greschner, M.; Sher, A.; Litke, A.M.; Chichilnisky, E.J. The Structure of Large-Scale Synchronized Firing in Primate Retina. J. Neurosci.
**2009**, 29, 5022–5031. [Google Scholar] [CrossRef] [PubMed] - Ganmor, E.; Segev, R.; Schneidman, E. Sparse low-order interaction network underlies a highly correlated and learnable neural population code. Proc. Natl. Acad. Sci. USA
**2011**, 108, 9679–9684. [Google Scholar] [CrossRef] [PubMed] - Yu, S.; Huang, D.; Singer, W.; Nikolic, D. A small world of neuronal synchrony. Cereb. Cortex
**2008**, 18, 2891–2901. [Google Scholar] [CrossRef] [PubMed] - Köster, U.; Sohl-Dickstein, J. Higher order correlations within cortical layers dominate functional connectivity in microcolumns. arXiv, 2013; arXiv:1301.0050v1. [Google Scholar]
- Hamilton, L.S.; SohlDickstein, J.; Huth, A.G.; Carels, V.M.; Deisseroth, K.; Bao, S. Optogenetic activation of an inhibitory network enhances feedforward functional connectivity in auditory cortex. Neuron
**2013**, 80, 10661076. [Google Scholar] [CrossRef] [PubMed] - Bialek, W.; Cavagna, A.; Giardina, I.; Mora, T.; Silvestri, E.; Viale, M.; Walczak, A. Statistical mechanics for natural flocks of birds. arXiv, 2011; arXiv:org:1107.0604. [Google Scholar]
- Jaynes, E.T. Information Theory and Statistical Mechanics. Phys. Rev.
**1957**, 106, 620–630. [Google Scholar] [CrossRef] - Bethge, M.; Berens, P. Near-maximum entropy models for binary neural representations of natural images. In Advances in Neural Information Processing Systems; Platt, J., Koller, D., Singer, Y., Roweis, S., Eds.; MIT Press: Cambridge, MA, USA, 2008; Volume 20, pp. 97–104. [Google Scholar]
- Roudi, Y.; Nirenberg, S.H.; Latham, P.E. Pairwise maximum entropy models for studying large biological systems: When they can and when they can’t work. PLoS Comput. Biol.
**2009**, 5, e1000380. [Google Scholar] [CrossRef] [PubMed] - Nirenberg, S.H.; Victor, J.D. Analyzing the activity of large populations of neurons: How tractable is the problem? Curr. Opin. Neurobiol.
**2007**, 17, 397–400. [Google Scholar] [CrossRef] [PubMed] - Azhar, F.; Bialek, W. When are correlations strong? arXiv, 2010; arXiv:org:1012.5987. [Google Scholar]
- Tkačik, G.; Marre, O.; Amodei, D.; Schneidman, E.; Bialek, W.; Berry, M.J. Searching for collective behavior in a large network of sensory neurons. PLoS Comput. Biol.
**2014**, 10, e1003408. [Google Scholar] [CrossRef] [PubMed] - Macke, J.H.; Opper, M.; Bethge, M. Common Input Explains Higher-Order Correlations and Entropy in a Simple Model of Neural Population Activity. Phys. Rev. Lett.
**2011**, 106, 208102. [Google Scholar] [CrossRef] [PubMed] - Sylvester, J. Thoughts on inverse orthogonal matrices, simultaneous sign successions, and tessellated pavements in two or more colours, with applications to Newton’s rule, ornamental tile-work, and the theory of numbers. Philos. Mag.
**1867**, 34, 461–475. [Google Scholar] - Diaconis, P. Finite forms of de Finetti’s theorem on exchangeability. Synthese
**1977**, 36, 271–281. [Google Scholar] [CrossRef] - Shannon, C. A mathematical theory of communications, I and II. Bell Syst. Tech. J.
**1948**, 27, 379–423. [Google Scholar] [CrossRef] - Cover, T.M.; Thomas, J.A. Elements of Information Theory; Wiley: Hoboken, NJ, USA, 1991. [Google Scholar]
- De Schutter, B. Minimal state-space realization in linear system theory: An overview. J. Comput. Appl. Math.
**2000**, 121, 331–354. [Google Scholar] [CrossRef] - Shalizi, C.; Crutchfield, J. Computational mechanics: Pattern and prediction, structure and simplicity. J. Stat. Phys.
**2001**, 104, 817–879. [Google Scholar] [CrossRef] - Carter, J.; Wegman, M. Universal classes of hash functions. J. Comput. Syst. Sci.
**1979**, 18, 143–154. [Google Scholar] [CrossRef] - Sipser, M. A complexity theoretic approach to randomness. In Proceedings of the Fifteenth Annual ACM Symposium on Theory of Computing, Boston, MA, USA, 25–27 April 1983; pp. 330–335. [Google Scholar]
- Stockmeyer, L. The complexity of approximate counting. In Proceedings of the Fifteenth Annual ACM Symposium on Theory of Computing, Boston, MA, USA, 25–27 April 1983; pp. 118–126. [Google Scholar]
- Chor, B.; Goldreich, O.; Hasted, J.; Freidmann, J.; Rudich, S.; Smolensky, R. The bit extraction problem or t-resilient functions. In Proceedings of the 26th Annual Symposium on Foundations of Computer Science, Portland, OR, USA, 21–23 October 1985; pp. 396–407. [Google Scholar]
- Karp, R.; Wigderson, A. A fast parallel algorithm for the maximal independent set problem. JACM
**1985**, 32, 762–773. [Google Scholar] [CrossRef] - Luby, M. A simple parallel algorithm for the maximal independent set problem. SIAM J. Comput.
**1986**, 15, 1036–1053. [Google Scholar] [CrossRef] - Alon, N.; Babai, L.; Itai, A. A fast and simple randomized parallel algorithm for the maximal independent set problem. J. Algorithms
**1986**, 7, 567–583. [Google Scholar] [CrossRef] - Alexi, W.; Chor, B.; Goldreich, O.; Schnorr, C. RSA and Rabin functions: Certain parts are as hard as the whole. SIAM J. Comput.
**1988**, 17, 194–209. [Google Scholar] [CrossRef] - Chor, B.; Goldreich, O. On the power of two-point based sampling. J. Complex.
**1989**, 5, 96–106. [Google Scholar] [CrossRef] - Berger, B.; Rompel, J. Simulating (log cn)-wise independence in NC. JACM
**1991**, 38, 1026–1046. [Google Scholar] [CrossRef] - Schulman, L. Sample spaces uniform on neighborhoods. In Proceedings of the Twenty-Fourth Annual ACM Symposium on Theory of Computing, Victoria, BC, Canada, 4–6 May 1992; pp. 17–25. [Google Scholar]
- Luby, M. Removing randomness in parallel computation without a processor penalty. J. Comput. Syst. Sci.
**1993**, 47, 250–286. [Google Scholar] [CrossRef] - Motwani, R.; Naor, J.; Naor, M. The probabilistic method yields deterministic parallel algorithms. J. Comput. Syst. Sci.
**1994**, 49, 478–516. [Google Scholar] [CrossRef] - Koller, D.; Megiddo, N. Constructing small sample spaces satisfying given constraints. In Proceedings of the Twenty-Fifth Annual ACM Symposium on Theory of Computing, San Diego, CA, USA, 16–18 May 1993; pp. 268–277. [Google Scholar]
- Karloff, H.; Mansour, Y. On construction of k-wise independent random variables. In Proceedings of the Twenty-Sixth Annual ACM Symposium on Theory of Computing, Montreal, QC, Canada, 23–25 May 1994; pp. 564–573. [Google Scholar]
- Castellana, M.; Bialek, W. Inverse spin glass and related maximum entropy problems. Phys. Rev. Lett.
**2014**, 113, 117204. [Google Scholar] [CrossRef] [PubMed] - Fischer, K.H.; Hertz, J.A. Spin Glasses; Cambridge University Press: Cambridge, UK, 1991. [Google Scholar]
- Tanaka, T. Mean-field theory of Boltzmann machine learning. Phys. Rev. Lett. E
**1998**, 58, 2302. [Google Scholar] [CrossRef] - Hinton, G.E.; Osindero, S.; Teh, Y.W. A fast learning algorithm for deep belief nets. Neural Comput.
**2006**, 18, 1527–1554. [Google Scholar] [CrossRef] [PubMed] - Hyvärinen, A. Connections between score matching, contrastive divergence, and pseudolikelihood for continuous-valued variables. IEEE Trans. Neural Netw.
**2007**, 18, 1529–1531. [Google Scholar] [CrossRef] [PubMed] - Broderick, T.; Dudík, M.; Tkačik, G.; Schapire, R.; Bialek, W. Faster solutions of the inverse pairwise Ising problem. arXiv, 2007; arXiv:0712.2437. [Google Scholar]
- Sohl-Dickstein, J.; Battaglino, P.B.; DeWeese, M.R. New method for parameter estimation in probabilistic models: Minimum probability flow. Phys. Rev. Lett.
**2011**, 107, 220601. [Google Scholar] [CrossRef] [PubMed] - Tkačik, G.; Marre, O.; Mora, T.; Amodei, D.; Berry, M.J., II; Bialek, W. The simplest maximum entropy model for collective behavior in a neural network. J. Stat. Mech. Theory Exp.
**2013**, 2013, P03011. [Google Scholar] [CrossRef] - Toulouse, G. The frustration model. In Modern Trends in the Theory of Condensed Matter; Pekalski, A., Przystawa, J.A., Eds.; Springer: Berlin/Heidelberg, Germany, 1980; Volume 115, pp. 195–203. [Google Scholar]
- Boyd, S.; Vandenberghe, L. Convex Optimization; Cambridge University Press: Cambridge, UK, 2004. [Google Scholar]
- Rosen, J.B. Global Minimization of a Linearly Constrained Function by Partition of Feasible Domain. Math. Oper. Res.
**1983**, 8, 215–230. [Google Scholar] [CrossRef] - Candes, E.; Tao, T. Decoding by linear programming. IEEE Trans. Inf. Theory
**2005**, 51, 4203–4215. [Google Scholar] [CrossRef] - Donoho, D. Compressed sensing. IEEE Trans. Inf. Theory
**2006**, 52, 1289–1306. [Google Scholar] [CrossRef] - Donoho, D.L. Compressed Sensing. Available online: https://statweb.stanford.edu/~donoho/Reports/2004/CompressedSensing091604.pdf (accessed on 17 August 2017).
- Sarvotham, S.; Baron, D.; Baraniuk, R.G. Measurements vs. Bits: Compressed Sensing meets Information Theory. Available online: https://scholarship.rice.edu/handle/1911/20323 (accessed on 17 August 2017).
- Hao, B. Elementary Symbolic Dynamics and Chaos in Dissipative Systems; World Scientific: Singapore, 1989. [Google Scholar]
- Luby, M.; Wigderson, A. Pairwise Independence and Derandomization; Now Publishers Inc.: Breda, The Netherlands, 2006; Volume 4. [Google Scholar]
- Joffe, A. On a set of almost deterministic k-independent random variables. Ann. Probab.
**1974**, 2, 161–162. [Google Scholar] [CrossRef] - MacWilliams, F.; Sloane, N. Error Correcting Codes; North Holland: New York, NY, USA, 1977. [Google Scholar]
- Hedayat, A.; Sloane, N.; Stufken, J. Orthogonal Arrays: Theory and Applications; Springer: New York, NY, USA, 1999. [Google Scholar]
- Hall, M. Combinatorial Theory; Blaisdell Publishing Company: London, UK, 1967. [Google Scholar]
- Lancaster, H. Pairwise statistical independence. Ann. Math. Stat.
**1965**, 36, 1313–1317. [Google Scholar] [CrossRef] - Rieke, F.; Warland, D.; van Steveninck, R.d.R.; Bialek, W. Spikes: Exploring the Neural Code; The MIT Press: Cambridge, MA, USA, 1999; p. 416. [Google Scholar]
- Advani, M.; Lahiri, S.; Ganguli, S. Statistical mechanics of complex neural systems and high dimensional data. J. Stat. Mech. Theory Exp.
**2013**, 2013, P03014. [Google Scholar] [CrossRef] - Panzeri, S.; Senatore, R.; Montemurro, M.A.; Petersen, R.S. Correcting for the sampling bias problem in spike train information measures. J. Neurophysiol.
**2007**, 98, 1064–1072. [Google Scholar] [CrossRef] [PubMed] - Rolls, E.T.; Treves, A. The neuronal encoding of information in the brain. Prog. Neurobiol.
**2011**, 95, 448–490. [Google Scholar] [CrossRef] [PubMed] - Crumiller, M.; Knight, B.; Yu, Y.; Kaplan, E. Estimating the amount of information conveyed by a population of neurons. Front. Neurosci.
**2011**, 5, 90. [Google Scholar] [CrossRef] [PubMed] - Strong, S.P.; Koberle, R.; van Steveninck, R.d.R.; Bialek, W. Entropy and information in neural spike trains. Phys. Rev. Lett.
**1998**, 80, 197. [Google Scholar] [CrossRef] - Nemenman, I.; Bialek, W.; van Steveninck, R.d.R. Entropy and information in neural spike trains: Progress on the sampling problem. Phys. Rev. E
**2004**, 69, 056111. [Google Scholar] [CrossRef] [PubMed] - Borst, A.; Theunissen, F.E. Information theory and neural coding. Nat. Neurosci.
**1999**, 2, 947–957. [Google Scholar] [CrossRef] [PubMed] - Quian Quiroga, R.; Panzeri, S. Extracting information from neuronal populations: Information theory and decoding approaches. Nat. Rev. Neurosci.
**2009**, 10, 173–185. [Google Scholar] [CrossRef] [PubMed] - Gale, D.; Kuhn, H.W.; Tucker, A.W. Linear Programming and the Theory of Games; Koopmans, T.C., Ed.; Wiley: New York, NY, USA, 1951; pp. 317–329. [Google Scholar]

**Figure 1.**Minimum and maximum entropy for fixed uniform constraints as a function of N. The minimum entropy grows no faster than logarithmically with the system size N for any mean activity level $\mu $ and pairwise correlation strength $\nu $. (

**a**) In a parameter regime relevant for neural population activity in the retina [5,6] ($\mu =0.1$, $\nu =0.011$), we can construct an explicit low entropy solution (${\tilde{S}}_{2}^{con2}$) that grows logarithmically with N, unlike the linear behavior of the maximum entropy solution (${S}_{2}$). Note that the linear behavior of the maximum entropy solution is only possible because these parameter values remain within the boundary of allowed $\mu $ and $\nu $ values (See Appendix C); (

**b**) Even for mean activities and pairwise correlations matched to the global maximum entropy solution (${S}_{2}$; $\mu =\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.$, $\nu =\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$4$}\right.$), we can construct explicit low entropy solutions (${\tilde{S}}_{2}^{con}$ and ${\tilde{S}}_{2}^{con2}$) and a lower bound (${\tilde{S}}_{2}^{lo}$) on the entropy that each grow logarithmically with N, in contrast to the linear behavior of the maximum entropy solution (${S}_{2}$) and the finitely exchangeable minimum entropy solution (${\tilde{S}}_{2}^{exch}$). ${\tilde{S}}_{1}$ is the minimum entropy distribution that is consistent with the mean firing rates. It remains constant as a function of N.

**Figure 2.**Minimum and maximum entropy models for uniform constraints. (

**a**) Entropy as a function of the strength of pairwise correlations for the maximum entropy model (${S}_{2}$), finitely exchangeable minimum entropy model (${\tilde{S}}_{2}^{exch}$), and a constructed low entropy solution (${\tilde{S}}_{2}^{con}$), all corresponding to $\mu =\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.$ and $N=5$. Filled circles indicate the global minimum ${\tilde{S}}_{1}$ and maximum ${S}_{1}$ for $\mu =\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.$; (

**b**–

**d**) Support for ${S}_{2}$ (b), ${\tilde{S}}_{2}^{exch}$ (c), and ${\tilde{S}}_{2}^{con}$ (d) corresponding to the three curves in panel (a). States are grouped by the number of active units; darker regions indicate higher total probability for each group of states; (

**e**–

**h**) Same as for panels (a) through (d), but with $N=30$. Note that, with rising N, the cusps in the ${\tilde{S}}_{2}^{exch}$ curve become much less pronounced.

**Figure 3.**An example of uniform, low-level statistics that can be realized by small groups of neurons but not by any system greater than some critical size. (

**a**) Upper (red curve, ${\tilde{\nu}}_{upper}$) and lower (cyan curve, ${\tilde{\nu}}_{lower}$) bounds on the minimum value (black curve, $\tilde{\nu}$) for the pairwise correlation $\nu $ shared by all pairs of neurons are plotted as a function of system size N assuming that every neuron has mean activity $\mu =0.1$. Note that all three curves asymptote to $\nu ={\mu}^{2}=0.01$ as $N\to \infty $ (dashed blue line); (

**b**) Enlarged portion of (

**a**) outlined in grey reveals that groups of $N\le 150$ neurons can exhibit uniform constraints $\mu =0.1$ and $\nu =0.0094$ (green dotted line), but this is not possible for any larger group.

© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Albanna, B.F.; Hillar, C.; Sohl-Dickstein, J.; DeWeese, M.R.
Minimum and Maximum Entropy Distributions for Binary Systems with Known Means and Pairwise Correlations. *Entropy* **2017**, *19*, 427.
https://doi.org/10.3390/e19080427

**AMA Style**

Albanna BF, Hillar C, Sohl-Dickstein J, DeWeese MR.
Minimum and Maximum Entropy Distributions for Binary Systems with Known Means and Pairwise Correlations. *Entropy*. 2017; 19(8):427.
https://doi.org/10.3390/e19080427

**Chicago/Turabian Style**

Albanna, Badr F., Christopher Hillar, Jascha Sohl-Dickstein, and Michael R. DeWeese.
2017. "Minimum and Maximum Entropy Distributions for Binary Systems with Known Means and Pairwise Correlations" *Entropy* 19, no. 8: 427.
https://doi.org/10.3390/e19080427