Multivariate Dependence beyond Shannon Information
AbstractAccurately determining dependency structure is critical to understanding a complex system’s organization. We recently showed that the transfer entropy fails in a key aspect of this—measuring information flow—due to its conflation of dyadic and polyadic relationships. We extend this observation to demonstrate that Shannon information measures (entropy and mutual information, in their conditional and multivariate forms) can fail to accurately ascertain multivariate dependencies due to their conflation of qualitatively different relations among variables. This has broad implications, particularly when employing information to express the organization and mechanisms embedded in complex systems, including the burgeoning efforts to combine complex network theory with information theory. Here, we do not suggest that any aspect of information theory is wrong. Rather, the vast majority of its informational measures are simply inadequate for determining the meaningful relationships among variables within joint probability distributions. We close by demonstrating that such distributions exist across an arbitrary set of variables. View Full-Text
Share & Cite This Article
James, R.G.; Crutchfield, J.P. Multivariate Dependence beyond Shannon Information. Entropy 2017, 19, 531.
James RG, Crutchfield JP. Multivariate Dependence beyond Shannon Information. Entropy. 2017; 19(10):531.Chicago/Turabian Style
James, Ryan G.; Crutchfield, James P. 2017. "Multivariate Dependence beyond Shannon Information." Entropy 19, no. 10: 531.
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.