Next Article in Journal
Second Law and Non-Equilibrium Entropy of Schottky Systems—Doubts and Verification–
Previous Article in Journal
Entropy Generation Rates in Two-Dimensional Rayleigh–Taylor Turbulence Mixing
Open AccessArticle

The Stochastic Complexity of Spin Models: Are Pairwise Models Really Simple?

1
The Abdus Salam International Centre for Theoretical Physics (ICTP), Strada Costiera 11, I-34014 Trieste, Italy
2
Kavli Institute for Systems Neuroscience and Centre for Neural Computation, Norges Teknisk-Naturvitenskapelige Universitet (NTNU), Olav Kyrres Gate 9, 7030 Trondheim, Norway
3
Department of Physics and Astronomy, University of Pennsylvania, 209 South 33rd Street, Philadelphia, PA 19104-6396, USA
4
Capital Fund Management, 23 rue de l’Université, 75007 Paris, France
5
Istituto Nazionale di Fisica Nucleare (INFN) Sezione di Trieste, 34100 Trieste, Italy
*
Author to whom correspondence should be addressed.
Entropy 2018, 20(10), 739; https://doi.org/10.3390/e20100739
Received: 10 August 2018 / Revised: 18 September 2018 / Accepted: 18 September 2018 / Published: 27 September 2018
(This article belongs to the Section Complexity)
Models can be simple for different reasons: because they yield a simple and computationally efficient interpretation of a generic dataset (e.g., in terms of pairwise dependencies)—as in statistical learning—or because they capture the laws of a specific phenomenon—as e.g., in physics—leading to non-trivial falsifiable predictions. In information theory, the simplicity of a model is quantified by the stochastic complexity, which measures the number of bits needed to encode its parameters. In order to understand how simple models look like, we study the stochastic complexity of spin models with interactions of arbitrary order. We show that bijections within the space of possible interactions preserve the stochastic complexity, which allows to partition the space of all models into equivalence classes. We thus found that the simplicity of a model is not determined by the order of the interactions, but rather by their mutual arrangements. Models where statistical dependencies are localized on non-overlapping groups of few variables are simple, affording predictions on independencies that are easy to falsify. On the contrary, fully connected pairwise models, which are often used in statistical learning, appear to be highly complex, because of their extended set of interactions, and they are hard to falsify. View Full-Text
Keywords: statistical inference; model complexity; minimum description length; spin models statistical inference; model complexity; minimum description length; spin models
Show Figures

Figure 1

MDPI and ACS Style

Beretta, A.; Battistin, C.; De Mulatier, C.; Mastromatteo, I.; Marsili, M. The Stochastic Complexity of Spin Models: Are Pairwise Models Really Simple? Entropy 2018, 20, 739. https://doi.org/10.3390/e20100739

AMA Style

Beretta A, Battistin C, De Mulatier C, Mastromatteo I, Marsili M. The Stochastic Complexity of Spin Models: Are Pairwise Models Really Simple? Entropy. 2018; 20(10):739. https://doi.org/10.3390/e20100739

Chicago/Turabian Style

Beretta, Alberto; Battistin, Claudia; De Mulatier, Clélia; Mastromatteo, Iacopo; Marsili, Matteo. 2018. "The Stochastic Complexity of Spin Models: Are Pairwise Models Really Simple?" Entropy 20, no. 10: 739. https://doi.org/10.3390/e20100739

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Search more from Scilit
 
Search
Back to TopTop