Entropy 2011, 13(11), 1945-1957; doi:10.3390/e13111945

A Characterization of Entropy in Terms of Information Loss

1,2email, 3,* email and 4email
Received: 11 October 2011; in revised form: 18 November 2011 / Accepted: 21 November 2011 / Published: 24 November 2011
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract: There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the entropy of a probability measure on a finite set, this characterization focuses on the “information loss”, or change in entropy, associated with a measure-preserving function. Information loss is a special case of conditional entropy: namely, it is the entropy of a random variable conditioned on some function of that variable. We show that Shannon entropy gives the only concept of information loss that is functorial, convex-linear and continuous. This characterization naturally generalizes to Tsallis entropy as well.
Keywords: Shannon entropy; Tsallis entropy; information theory; measure-preserving function
PDF Full-text Download PDF Full-Text [246 KB, Updated Version, uploaded 25 November 2011 08:12 CET]
The original version is still available [246 KB, uploaded 24 November 2011 09:29 CET]

Export to BibTeX |

MDPI and ACS Style

Baez, J.C.; Fritz, T.; Leinster, T. A Characterization of Entropy in Terms of Information Loss. Entropy 2011, 13, 1945-1957.

AMA Style

Baez JC, Fritz T, Leinster T. A Characterization of Entropy in Terms of Information Loss. Entropy. 2011; 13(11):1945-1957.

Chicago/Turabian Style

Baez, John C.; Fritz, Tobias; Leinster, Tom. 2011. "A Characterization of Entropy in Terms of Information Loss." Entropy 13, no. 11: 1945-1957.

Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert