Next Article in Journal
Loop Entropy Assists Tertiary Order: Loopy Stabilization of Stacking Motifs
Previous Article in Journal
Classes of N-Dimensional Nonlinear Fokker-Planck Equations Associated to Tsallis Entropy
Open AccessArticle

A Characterization of Entropy in Terms of Information Loss

1
Department of Mathematics, University of California, Riverside, CA 92521, USA
2
Centre for Quantum Technologies, National University of Singapore, 117543, Singapore
3
Institut de Ciències Fotòniques, Mediterranean Technology Park, 08860 Castelldefels (Barcelona), Spain
4
School of Mathematics and Statistics, University of Glasgow, Glasgow G12 8QW, UK
*
Author to whom correspondence should be addressed.
Entropy 2011, 13(11), 1945-1957; https://doi.org/10.3390/e13111945
Received: 11 October 2011 / Revised: 18 November 2011 / Accepted: 21 November 2011 / Published: 24 November 2011
There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the entropy of a probability measure on a finite set, this characterization focuses on the “information loss”, or change in entropy, associated with a measure-preserving function. Information loss is a special case of conditional entropy: namely, it is the entropy of a random variable conditioned on some function of that variable. We show that Shannon entropy gives the only concept of information loss that is functorial, convex-linear and continuous. This characterization naturally generalizes to Tsallis entropy as well. View Full-Text
Keywords: Shannon entropy; Tsallis entropy; information theory; measure-preserving function Shannon entropy; Tsallis entropy; information theory; measure-preserving function
Show Figures

Graphical abstract

MDPI and ACS Style

Baez, J.C.; Fritz, T.; Leinster, T. A Characterization of Entropy in Terms of Information Loss. Entropy 2011, 13, 1945-1957.

Show more citation formats Show less citations formats

Article Access Map

1
Only visits after 24 November 2015 are recorded.
Back to TopTop