Next Article in Journal
Extraction of the Proton and Electron Radii from Characteristic Atomic Lines and Entropy Principles
Next Article in Special Issue
On Extractable Shared Information
Previous Article in Journal
Analysis of a Hybrid Thermoelectric Microcooler: Thomson Heat and Geometric Optimization
Article Menu
Issue 7 (July) cover image

Export Article

Open AccessArticle
Entropy 2017, 19(7), 318; doi:10.3390/e19070318

Measuring Multivariate Redundant Information with Pointwise Common Change in Surprisal

Institute of Neuroscience and Psychology, University of Glasgow, Glasgow G12 8QB, UK
Received: 2 May 2017 / Revised: 25 June 2017 / Accepted: 27 June 2017 / Published: 29 June 2017
View Full-Text   |   Download PDF [588 KB, uploaded 30 June 2017]   |  

Abstract

The problem of how to properly quantify redundant information is an open question that has been the subject of much recent research. Redundant information refers to information about a target variable S that is common to two or more predictor variables X i . It can be thought of as quantifying overlapping information content or similarities in the representation of S between the X i . We present a new measure of redundancy which measures the common change in surprisal shared between variables at the local or pointwise level. We provide a game-theoretic operational definition of unique information, and use this to derive constraints which are used to obtain a maximum entropy distribution. Redundancy is then calculated from this maximum entropy distribution by counting only those local co-information terms which admit an unambiguous interpretation as redundant information. We show how this redundancy measure can be used within the framework of the Partial Information Decomposition (PID) to give an intuitive decomposition of the multivariate mutual information into redundant, unique and synergistic contributions. We compare our new measure to existing approaches over a range of example systems, including continuous Gaussian variables. Matlab code for the measure is provided, including all considered examples. View Full-Text
Keywords: mutual information; redundancy; synergy; pointwise; local; surprisal; partial information decomposition; interaction information; co-information mutual information; redundancy; synergy; pointwise; local; surprisal; partial information decomposition; interaction information; co-information
Figures

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. (CC BY 4.0).

Scifeed alert for new publications

Never miss any articles matching your research from any publisher
  • Get alerts for new papers matching your research
  • Find out the new papers from selected authors
  • Updated daily for 49'000+ journals and 6000+ publishers
  • Define your Scifeed now

SciFeed Share & Cite This Article

MDPI and ACS Style

Ince, R.A.A. Measuring Multivariate Redundant Information with Pointwise Common Change in Surprisal. Entropy 2017, 19, 318.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top