CiteULike
Facebook
Mendeley
Twitter
This article is- freely available
- re-usable
Entropy 2011, 13(12), 2013-2023; doi:10.3390/e13122013
Article
Coincidences and Estimation of Entropies of Random Variables with Large Cardinalities
Departments of Physics and Biology and Computational and Life Sciences Initiative, Emory University, 400 Dowman Dr., Atlanta, GA 30322, USA
Received: 1 November 2011; in revised form: 8 December 2011 / Accepted: 15 December 2011 / Published: 19 December 2011
Download PDF Full-Text [246 KB, uploaded 19 December 2011 09:35 CET]
Abstract: We perform an asymptotic analysis of the NSB estimator of entropy of a discrete random variable. The analysis illuminates the dependence of the estimates on the number of coincidences in the sample and shows that the estimator has a well defined limit for a large cardinality of the studied variable. This allows estimation of entropy with no a priori assumptions about the cardinality. Software implementation of the algorithm is available.
Keywords: entropy estimation; coincidences; bias-variance tradeoff; model selection
Article Statistics
Click here to load and display the download statistics.Cite This Article
MDPI and ACS Style
Nemenman, I. Coincidences and Estimation of Entropies of Random Variables with Large Cardinalities. Entropy 2011, 13, 2013-2023.
AMA StyleNemenman I. Coincidences and Estimation of Entropies of Random Variables with Large Cardinalities. Entropy. 2011; 13(12):2013-2023.
Chicago/Turabian StyleNemenman, Ilya. 2011. "Coincidences and Estimation of Entropies of Random Variables with Large Cardinalities." Entropy 13, no. 12: 2013-2023.