Next Article in Journal
Previous Article in Journal
Entropy 2011, 13(12), 2013-2023; doi:10.3390/e13122013
Article

Coincidences and Estimation of Entropies of Random Variables with Large Cardinalities

Received: 1 November 2011; in revised form: 8 December 2011 / Accepted: 15 December 2011 / Published: 19 December 2011
Download PDF [246 KB, uploaded 19 December 2011]
Abstract: We perform an asymptotic analysis of the NSB estimator of entropy of a discrete random variable. The analysis illuminates the dependence of the estimates on the number of coincidences in the sample and shows that the estimator has a well defined limit for a large cardinality of the studied variable. This allows estimation of entropy with no a priori assumptions about the cardinality. Software implementation of the algorithm is available.
Keywords: entropy estimation; coincidences; bias-variance tradeoff; model selection entropy estimation; coincidences; bias-variance tradeoff; model selection
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Export to BibTeX |
EndNote


MDPI and ACS Style

Nemenman, I. Coincidences and Estimation of Entropies of Random Variables with Large Cardinalities. Entropy 2011, 13, 2013-2023.

AMA Style

Nemenman I. Coincidences and Estimation of Entropies of Random Variables with Large Cardinalities. Entropy. 2011; 13(12):2013-2023.

Chicago/Turabian Style

Nemenman, Ilya. 2011. "Coincidences and Estimation of Entropies of Random Variables with Large Cardinalities." Entropy 13, no. 12: 2013-2023.


Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert