Next Article in Journal
Prediction Method for Image Coding Quality Based on Differential Information Entropy
Previous Article in Journal
The Elusive Nature of Entropy and Its Physical Meaning
Previous Article in Special Issue
Consideration on Singularities in Learning Theory and the Learning Coefficient
Entropy 2014, 16(2), 968-989; doi:10.3390/e16020968
Article

Information Bottleneck Approach to Predictive Inference

Received: 3 June 2013 / Accepted: 18 June 2013 / Published: 17 February 2014
(This article belongs to the Special Issue The Information Bottleneck Method)
View Full-Text   |   Download PDF [276 KB, 24 February 2015; original version 24 February 2015]

Abstract

This paper synthesizes a recent line of work on automated predictive model making inspired by Rate-Distortion theory, in particular by the Information Bottleneck method. Predictive inference is interpreted as a strategy for efficient communication. The relationship to thermodynamic efficiency is discussed. The overall aim of this paper is to explain how this information theoretic approach provides an intuitive, overarching framework for predictive inference.
Keywords: predictive inference; information bottleneck method; dynamical systems; thermodynamic efficiency; far-from-equilibrium thermodynamics; computing engines predictive inference; information bottleneck method; dynamical systems; thermodynamic efficiency; far-from-equilibrium thermodynamics; computing engines
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Share & Cite This Article

Further Mendeley | CiteULike
Export to BibTeX |
EndNote
MDPI and ACS Style

Still, S. Information Bottleneck Approach to Predictive Inference. Entropy 2014, 16, 968-989.

View more citation formats

Related Articles

Article Metrics

For more information on the journal, click here

Comments

Cited By

[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert