Information Loss in Binomial Data Due to Data Compression
Battelle Center for Mathematical Medicine, The Research Institute, Nationwide Children’s Hospital, Columbus, OH 43215, USA
Department of Pediatrics, The Ohio State University, Columbus, OH 43210, USA
Department of Statistics, The Ohio State University, Columbus, OH 43210, USA
Author to whom correspondence should be addressed.
Academic Editor: Raúl Alcaraz Martínez
Received: 1 December 2016 / Revised: 7 February 2017 / Accepted: 12 February 2017 / Published: 16 February 2017
This paper explores the idea of information loss through data compression, as occurs in the course of any data analysis, illustrated via detailed consideration of the Binomial distribution. We examine situations where the full sequence of binomial outcomes is retained, situations where only the total number of successes is retained, and in-between situations. We show that a familiar decomposition of the Shannon entropy H
can be rewritten as a decomposition into
, or the total, lost and compressed (remaining) components, respectively. We relate this new decomposition to Landauer’s principle, and we discuss some implications for the “information-dynamic” theory being developed in connection with our broader program to develop a measure of statistical evidence on a properly calibrated scale.
This is an open access article distributed under the Creative Commons Attribution License
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
Share & Cite This Article
MDPI and ACS Style
Hodge, S.E.; Vieland, V.J. Information Loss in Binomial Data Due to Data Compression. Entropy 2017, 19, 75.
Hodge SE, Vieland VJ. Information Loss in Binomial Data Due to Data Compression. Entropy. 2017; 19(2):75.
Hodge, Susan E.; Vieland, Veronica J. 2017. "Information Loss in Binomial Data Due to Data Compression." Entropy 19, no. 2: 75.
Show more citation formats
Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.
[Return to top]
For more information on the journal statistics, click here
Multiple requests from the same IP address are counted as one view.