Minimum and Maximum Entropy Distributions for Binary Systems with Known Means and Pairwise Correlations
AbstractMaximum entropy models are increasingly being used to describe the collective activity of neural populations with measured mean neural activities and pairwise correlations, but the full space of probability distributions consistent with these constraints has not been explored. We provide upper and lower bounds on the entropy for the minimum entropy distribution over arbitrarily large collections of binary units with any fixed set of mean values and pairwise correlations. We also construct specific low-entropy distributions for several relevant cases. Surprisingly, the minimum entropy solution has entropy scaling logarithmically with system size for any set of first- and second-order statistics consistent with arbitrarily large systems. We further demonstrate that some sets of these low-order statistics can only be realized by small systems. Our results show how only small amounts of randomness are needed to mimic low-order statistical properties of highly entropic distributions, and we discuss some applications for engineered and biological information transmission systems. View Full-Text
Scifeed alert for new publicationsNever miss any articles matching your research from any publisher
- Get alerts for new papers matching your research
- Find out the new papers from selected authors
- Updated daily for 49'000+ journals and 6000+ publishers
- Define your Scifeed now
Albanna, B.F.; Hillar, C.; Sohl-Dickstein, J.; DeWeese, M.R. Minimum and Maximum Entropy Distributions for Binary Systems with Known Means and Pairwise Correlations. Entropy 2017, 19, 427.
Albanna BF, Hillar C, Sohl-Dickstein J, DeWeese MR. Minimum and Maximum Entropy Distributions for Binary Systems with Known Means and Pairwise Correlations. Entropy. 2017; 19(8):427.Chicago/Turabian Style
Albanna, Badr F.; Hillar, Christopher; Sohl-Dickstein, Jascha; DeWeese, Michael R. 2017. "Minimum and Maximum Entropy Distributions for Binary Systems with Known Means and Pairwise Correlations." Entropy 19, no. 8: 427.
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.