Next Article in Journal
Community Detection Method Based on Node Density, Degree Centrality, and K-Means Clustering in Complex Network
Previous Article in Journal
OTEC Maximum Net Power Output Using Carnot Cycle and Application to Simplify Heat Exchanger Selection
Open AccessArticle

Convergence Rates for Empirical Estimation of Binary Classification Bounds

1
School of Computing and Information Science, University of Maine, Orono, ME 04469, USA
2
Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109, USA
3
Department of Mathematics and Statistics, Utah State University, Logan, UT 84322, USA
*
Author to whom correspondence should be addressed.
Entropy 2019, 21(12), 1144; https://doi.org/10.3390/e21121144
Received: 5 November 2019 / Accepted: 15 November 2019 / Published: 23 November 2019
Bounding the best achievable error probability for binary classification problems is relevant to many applications including machine learning, signal processing, and information theory. Many bounds on the Bayes binary classification error rate depend on information divergences between the pair of class distributions. Recently, the Henze–Penrose (HP) divergence has been proposed for bounding classification error probability. We consider the problem of empirically estimating the HP-divergence from random samples. We derive a bound on the convergence rate for the Friedman–Rafsky (FR) estimator of the HP-divergence, which is related to a multivariate runs statistic for testing between two distributions. The FR estimator is derived from a multicolored Euclidean minimal spanning tree (MST) that spans the merged samples. We obtain a concentration inequality for the Friedman–Rafsky estimator of the Henze–Penrose divergence. We validate our results experimentally and illustrate their application to real datasets. View Full-Text
Keywords: classification; Bayes error rate; Henze–Penrose divergence; Friedman–Rafsky test statistic; convergence rates; bias and variance trade-off; concentration bounds; minimal spanning trees classification; Bayes error rate; Henze–Penrose divergence; Friedman–Rafsky test statistic; convergence rates; bias and variance trade-off; concentration bounds; minimal spanning trees
Show Figures

Figure 1

MDPI and ACS Style

Sekeh, S.Y.; Noshad, M.; Moon, K.R.; Hero, A.O. Convergence Rates for Empirical Estimation of Binary Classification Bounds. Entropy 2019, 21, 1144.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop