Entropy Best ECR Presentation Awards at CNS*2020 Online Workshop on Methods of Information Theory in Computational Neuroscience

We are pleased to announce the two winners of the Best ECR Presentation Awards sponsored by Entropy for the CNS*2020 Online Workshop on Methods of Information Theory in Computational Neuroscience held on 21–22 July 2020.

“A Differentiable Measure of Pointwise Shared Information” by Abdullah Makkeh

Partial information decomposition (PID) of the multivariate mutual information describes the distinct ways in which a set of source variables contains information about a target variable. The groundbreaking work of Williams and Beer has shown that this decomposition cannot be determined from classic information theory without making additional assumptions, and several candidate measures have been proposed, often drawing on principles from related fields such as decision theory. None of these measures is differentiable with respect to the underlying probability mass function. Here, we present a novel measure that draws only on the principle linking the local mutual information to exclusion of probability mass. This principle is foundational to the original definition of the mutual information by Fano. We reuse this principle to define a measure of shared information based on the shared exclusion of probability mass by the realizations of source variables. Our measure is differentiable and well defined for individual realizations of the random variables. Thus, it lends itself, for example, to local learning in artificial neural networks. We show that the measure can be interpreted as local mutual information with the help of an auxiliary variable. We also show that it has a meaningful Möbius inversion on a redundancy lattice and obeys a target chain rule. We provide an operational interpretation of the measure based on the decisions that an agent should take if only given the shared information.

“Multi-Target Information Decomposition and Applications to Integrated Information Theory” by Pedro Mediano

The partial information decomposition (PID) framework allows us to decompose the information that multiple source variables have about a single target variable. In its 10 years of existence, PID has spawned numerous theoretical and practical tools to help us understand and analyze information processing in complex systems. However, the asymmetric role of sources and targets in PID hinders its application in certain contexts, like studying information sharing in multiple processes evolving jointly over time. In this work, we developed a novel extension of the PID framework to the multi-target setting, which lends itself more naturally to the analysis of multivariate dynamical systems. This new decomposition is tightly linked with integrated information theory and gives us new analysis tools as well as a richer understanding of information processing in multivariate dynamical systems. Link: https://arxiv.org/abs/1909.02297.