Next Article in Journal
Calculation of Configurational Entropy in Complex Landscapes
Next Article in Special Issue
Information Decomposition of Target Effects from Multi-Source Interactions: Perspectives on Previous, Current and Future Work
Previous Article in Journal
Image Clustering with Optimization Algorithms and Color Space
Previous Article in Special Issue
Analyzing Information Distribution in Complex Systems
Open AccessEditor’s ChoiceArticle

Pointwise Partial Information Decomposition Using the Specificity and Ambiguity Lattices

by Conor Finn 1,2,* and Joseph T. Lizier 1
1
Complex Systems Research Group and Centre for Complex Systems, Faculty of Engineering & IT, The University of Sydney, NSW 2006, Australia
2
CSIRO Data61, Marsfield NSW 2122, Australia
*
Author to whom correspondence should be addressed.
Entropy 2018, 20(4), 297; https://doi.org/10.3390/e20040297
Received: 10 July 2017 / Revised: 6 April 2018 / Accepted: 10 April 2018 / Published: 18 April 2018
What are the distinct ways in which a set of predictor variables can provide information about a target variable? When does a variable provide unique information, when do variables share redundant information, and when do variables combine synergistically to provide complementary information? The redundancy lattice from the partial information decomposition of Williams and Beer provided a promising glimpse at the answer to these questions. However, this structure was constructed using a much criticised measure of redundant information, and despite sustained research, no completely satisfactory replacement measure has been proposed. In this paper, we take a different approach, applying the axiomatic derivation of the redundancy lattice to a single realisation from a set of discrete variables. To overcome the difficulty associated with signed pointwise mutual information, we apply this decomposition separately to the unsigned entropic components of pointwise mutual information which we refer to as the specificity and ambiguity. This yields a separate redundancy lattice for each component. Then based upon an operational interpretation of redundancy, we define measures of redundant specificity and ambiguity enabling us to evaluate the partial information atoms in each lattice. These atoms can be recombined to yield the sought-after multivariate information decomposition. We apply this framework to canonical examples from the literature and discuss the results and the various properties of the decomposition. In particular, the pointwise decomposition using specificity and ambiguity satisfies a chain rule over target variables, which provides new insights into the so-called two-bit-copy example. View Full-Text
Keywords: mutual information; pointwise information; information decomposition; unique information; redundant information; complementary information; redundancy; synergy mutual information; pointwise information; information decomposition; unique information; redundant information; complementary information; redundancy; synergy
Show Figures

Figure 1

MDPI and ACS Style

Finn, C.; Lizier, J.T. Pointwise Partial Information Decomposition Using the Specificity and Ambiguity Lattices. Entropy 2018, 20, 297.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Search more from Scilit
 
Search
Back to TopTop