Entropy 2011, 13(11), 1945-1957; doi:10.3390/e13111945
Article

A Characterization of Entropy in Terms of Information Loss

John C. Baez 1,2 email, Tobias Fritz 3,* email and Tom Leinster 4 email
1 Department of Mathematics, University of California, Riverside, CA 92521, USA 2 Centre for Quantum Technologies, National University of Singapore, 117543, Singapore 3 Institut de Ciències Fotòniques, Mediterranean Technology Park, 08860 Castelldefels (Barcelona), Spain 4 School of Mathematics and Statistics, University of Glasgow, Glasgow G12 8QW, UK
* Author to whom correspondence should be addressed.
Received: 11 October 2011; in revised form: 18 November 2011 / Accepted: 21 November 2011 / Published: 24 November 2011
PDF Full-text Download PDF Full-Text [246 KB, Updated Version, uploaded 25 November 2011 08:12 CET]
The original version is still available [246 KB, uploaded 24 November 2011 09:29 CET]
Abstract: There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the entropy of a probability measure on a finite set, this characterization focuses on the “information loss”, or change in entropy, associated with a measure-preserving function. Information loss is a special case of conditional entropy: namely, it is the entropy of a random variable conditioned on some function of that variable. We show that Shannon entropy gives the only concept of information loss that is functorial, convex-linear and continuous. This characterization naturally generalizes to Tsallis entropy as well.
Keywords: Shannon entropy; Tsallis entropy; information theory; measure-preserving function

Article Statistics

Click here to load and display the download statistics.

Cite This Article

MDPI and ACS Style

Baez, J.C.; Fritz, T.; Leinster, T. A Characterization of Entropy in Terms of Information Loss. Entropy 2011, 13, 1945-1957.

AMA Style

Baez JC, Fritz T, Leinster T. A Characterization of Entropy in Terms of Information Loss. Entropy. 2011; 13(11):1945-1957.

Chicago/Turabian Style

Baez, John C.; Fritz, Tobias; Leinster, Tom. 2011. "A Characterization of Entropy in Terms of Information Loss." Entropy 13, no. 11: 1945-1957.

Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert