Next Article in Journal
Genetic Algorithm-Based Identification of Fractional-Order Systems
Next Article in Special Issue
An Estimate of Mutual Information that Permits Closed-Form Optimisation
Previous Article in Journal
Function Identification in Neuron Populations via Information Bottleneck
Previous Article in Special Issue
Minimum Mutual Information and Non-Gaussianity through the Maximum Entropy Method: Estimation from Finite Samples
Entropy 2013, 15(5), 1609-1623; doi:10.3390/e15051609
Article

A Novel Nonparametric Distance Estimator for Densities with Error Bounds

1,* , 1
 and 2
1 Instituto de Engenharia Mecânica e Gestão Industrial, Faculdade de Engenharia, Universidade do Porto, Rua Dr. Roberto Frias s/n, 4200-465 Porto, Portugal 2 Computational Neuro Engineering Laboratory, University of Florida, EB451 Engineering Building, University of Florida, Gainesville, FL 32611, USA
* Author to whom correspondence should be addressed.
Received: 19 December 2012 / Revised: 25 April 2013 / Accepted: 28 April 2013 / Published: 6 May 2013
(This article belongs to the Special Issue Estimating Information-Theoretic Quantities from Data)
Download PDF [620 KB, uploaded 24 February 2015]

Abstract

The use of a metric to assess distance between probability densities is an important practical problem. In this work, a particular metric induced by an α-divergence is studied. The Hellinger metric can be interpreted as a particular case within the framework of generalized Tsallis divergences and entropies. The nonparametric Parzen’s density estimator emerges as a natural candidate to estimate the underlying probability density function, since it may account for data from different groups, or experiments with distinct instrumental precisions, i.e., non-independent and identically distributed (non-i.i.d.) data. However, the information theoretic derived metric of the nonparametric Parzen’s density estimator displays infinite variance, limiting the direct use of resampling estimators. Based on measure theory, we present a change of measure to build a finite variance density allowing the use of resampling estimators. In order to counteract the poor scaling with dimension, we propose a new nonparametric two-stage robust resampling estimator of Hellinger’s metric error bounds for heterocedastic data. The approach presents very promising results allowing the use of different covariances for different clusters with impact on the distance evaluation.
Keywords: generalized differential entropies; generalized differential divergences; Tsallis entropy; Hellinger metric; nonparametric estimators; heterocedastic data generalized differential entropies; generalized differential divergences; Tsallis entropy; Hellinger metric; nonparametric estimators; heterocedastic data
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Share & Cite This Article

Export to BibTeX |
EndNote


MDPI and ACS Style

Carvalho, A.R.; Tavares, J.M.R.S.; Principe, J.C. A Novel Nonparametric Distance Estimator for Densities with Error Bounds. Entropy 2013, 15, 1609-1623.

View more citation formats

Article Metrics

Comments

Citing Articles

[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert