Next Article in Journal
Loop Entropy Assists Tertiary Order: Loopy Stabilization of Stacking Motifs
Previous Article in Journal
Classes of N-Dimensional Nonlinear Fokker-Planck Equations Associated to Tsallis Entropy
Article Menu

Export Article

Open AccessArticle
Entropy 2011, 13(11), 1945-1957;

A Characterization of Entropy in Terms of Information Loss

Department of Mathematics, University of California, Riverside, CA 92521, USA
Centre for Quantum Technologies, National University of Singapore, 117543, Singapore
Institut de Ciències Fotòniques, Mediterranean Technology Park, 08860 Castelldefels (Barcelona), Spain
School of Mathematics and Statistics, University of Glasgow, Glasgow G12 8QW, UK
Author to whom correspondence should be addressed.
Received: 11 October 2011 / Revised: 18 November 2011 / Accepted: 21 November 2011 / Published: 24 November 2011
Full-Text   |   PDF [246 KB, uploaded 24 February 2015]   |  


There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the entropy of a probability measure on a finite set, this characterization focuses on the “information loss”, or change in entropy, associated with a measure-preserving function. Information loss is a special case of conditional entropy: namely, it is the entropy of a random variable conditioned on some function of that variable. We show that Shannon entropy gives the only concept of information loss that is functorial, convex-linear and continuous. This characterization naturally generalizes to Tsallis entropy as well. View Full-Text
Keywords: Shannon entropy; Tsallis entropy; information theory; measure-preserving function Shannon entropy; Tsallis entropy; information theory; measure-preserving function

Graphical abstract

This is an open access article distributed under the Creative Commons Attribution License (CC BY 3.0).

Share & Cite This Article

MDPI and ACS Style

Baez, J.C.; Fritz, T.; Leinster, T. A Characterization of Entropy in Terms of Information Loss. Entropy 2011, 13, 1945-1957.

Show more citation formats Show less citations formats

Related Articles

Article Metrics

Article Access Statistics



[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top