Next Article in Journal
Calculation of Configurational Entropy in Complex Landscapes
Next Article in Special Issue
Information Decomposition of Target Effects from Multi-Source Interactions: Perspectives on Previous, Current and Future Work
Previous Article in Journal
Image Clustering with Optimization Algorithms and Color Space
Previous Article in Special Issue
Analyzing Information Distribution in Complex Systems
Article Menu
Issue 4 (April) cover image

Export Article

Open AccessArticle
Entropy 2018, 20(4), 297; https://doi.org/10.3390/e20040297

Pointwise Partial Information Decomposition Using the Specificity and Ambiguity Lattices

1
Complex Systems Research Group and Centre for Complex Systems, Faculty of Engineering & IT, The University of Sydney, NSW 2006, Australia
2
CSIRO Data61, Marsfield NSW 2122, Australia
*
Author to whom correspondence should be addressed.
Received: 10 July 2017 / Revised: 6 April 2018 / Accepted: 10 April 2018 / Published: 18 April 2018
Full-Text   |   PDF [529 KB, uploaded 3 May 2018]   |  

Abstract

What are the distinct ways in which a set of predictor variables can provide information about a target variable? When does a variable provide unique information, when do variables share redundant information, and when do variables combine synergistically to provide complementary information? The redundancy lattice from the partial information decomposition of Williams and Beer provided a promising glimpse at the answer to these questions. However, this structure was constructed using a much criticised measure of redundant information, and despite sustained research, no completely satisfactory replacement measure has been proposed. In this paper, we take a different approach, applying the axiomatic derivation of the redundancy lattice to a single realisation from a set of discrete variables. To overcome the difficulty associated with signed pointwise mutual information, we apply this decomposition separately to the unsigned entropic components of pointwise mutual information which we refer to as the specificity and ambiguity. This yields a separate redundancy lattice for each component. Then based upon an operational interpretation of redundancy, we define measures of redundant specificity and ambiguity enabling us to evaluate the partial information atoms in each lattice. These atoms can be recombined to yield the sought-after multivariate information decomposition. We apply this framework to canonical examples from the literature and discuss the results and the various properties of the decomposition. In particular, the pointwise decomposition using specificity and ambiguity satisfies a chain rule over target variables, which provides new insights into the so-called two-bit-copy example. View Full-Text
Keywords: mutual information; pointwise information; information decomposition; unique information; redundant information; complementary information; redundancy; synergy mutual information; pointwise information; information decomposition; unique information; redundant information; complementary information; redundancy; synergy
Figures

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
SciFeed
Printed Edition Available!
A printed edition of this Special Issue is available here.

Share & Cite This Article

MDPI and ACS Style

Finn, C.; Lizier, J.T. Pointwise Partial Information Decomposition Using the Specificity and Ambiguity Lattices. Entropy 2018, 20, 297.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top