Next Article in Journal
Fluctuation, Dissipation and the Arrow of Time
Previous Article in Journal
Effects of Radiation Heat Transfer on Entropy Generation at Thermosolutal Convection in a Square Cavity Subjected to a Magnetic Field
Open AccessArticle

Coincidences and Estimation of Entropies of Random Variables with Large Cardinalities

Departments of Physics and Biology and Computational and Life Sciences Initiative, Emory University, 400 Dowman Dr., Atlanta, GA 30322, USA
Entropy 2011, 13(12), 2013-2023; https://doi.org/10.3390/e13122013
Received: 1 November 2011 / Revised: 8 December 2011 / Accepted: 15 December 2011 / Published: 19 December 2011
We perform an asymptotic analysis of the NSB estimator of entropy of a discrete random variable. The analysis illuminates the dependence of the estimates on the number of coincidences in the sample and shows that the estimator has a well defined limit for a large cardinality of the studied variable. This allows estimation of entropy with no a priori assumptions about the cardinality. Software implementation of the algorithm is available. View Full-Text
Keywords: entropy estimation; coincidences; bias-variance tradeoff; model selection entropy estimation; coincidences; bias-variance tradeoff; model selection
MDPI and ACS Style

Nemenman, I. Coincidences and Estimation of Entropies of Random Variables with Large Cardinalities. Entropy 2011, 13, 2013-2023.

Show more citation formats Show less citations formats

Article Access Map by Country/Region

1
Only visits after 24 November 2015 are recorded.
Back to TopTop