A Characterization of Entropy in Terms of Information Loss
John C. Baez1
Department of Mathematics, University of California, Riverside, CA 92521, USA
2
Centre for Quantum Technologies, National University of Singapore, 117543, Singapore
3
Institut de Ciències Fotòniques, Mediterranean Technology Park, 08860 Castelldefels (Barcelona), Spain
4
School of Mathematics and Statistics, University of Glasgow, Glasgow G12 8QW, UK
*
Author to whom correspondence should be addressed.
Entropy 2011, 13(11), 1945-1957; https://doi.org/10.3390/e13111945
Received: 11 October 2011 / Revised: 18 November 2011 / Accepted: 21 November 2011 / Published: 24 November 2011
There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the entropy of a probability measure on a finite set, this characterization focuses on the “information loss”, or change in entropy, associated with a measure-preserving function. Information loss is a special case of conditional entropy: namely, it is the entropy of a random variable conditioned on some function of that variable. We show that Shannon entropy gives the only concept of information loss that is functorial, convex-linear and continuous. This characterization naturally generalizes to Tsallis entropy as well.
View Full-Text
▼
Show Figures
Graphical abstract
This is an open access article distributed under the Creative Commons Attribution License
MDPI and ACS Style
Baez, J.C.; Fritz, T.; Leinster, T. A Characterization of Entropy in Terms of Information Loss. Entropy 2011, 13, 1945-1957.
Show more citation formats
Search more from Scilit
