Next Article in Journal
Pricing Interval European Option with the Principle of Maximum Entropy
Previous Article in Journal
Entropy-Based Clustering Algorithm for Fingerprint Singular Point Detection
Previous Article in Special Issue
Gender Diversity in STEM Disciplines: A Multiple Factor Problem
Article Menu
Issue 8 (August) cover image

Export Article

Open AccessArticle

Geometric Estimation of Multivariate Dependency

Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109, USA
*
Author to whom correspondence should be addressed.
Current address: School of Computing and Information Science, University of Maine, Orono, ME 04469, USA
Entropy 2019, 21(8), 787; https://doi.org/10.3390/e21080787
Received: 21 May 2019 / Revised: 2 August 2019 / Accepted: 8 August 2019 / Published: 12 August 2019
(This article belongs to the Special Issue Women in Information Theory 2018)
PDF [518 KB, uploaded 12 August 2019]

Abstract

This paper proposes a geometric estimator of dependency between a pair of multivariate random variables. The proposed estimator of dependency is based on a randomly permuted geometric graph (the minimal spanning tree) over the two multivariate samples. This estimator converges to a quantity that we call the geometric mutual information (GMI), which is equivalent to the Henze–Penrose divergence. between the joint distribution of the multivariate samples and the product of the marginals. The GMI has many of the same properties as standard MI but can be estimated from empirical data without density estimation; making it scalable to large datasets. The proposed empirical estimator of GMI is simple to implement, involving the construction of an minimal spanning tree (MST) spanning over both the original data and a randomly permuted version of this data. We establish asymptotic convergence of the estimator and convergence rates of the bias and variance for smooth multivariate density functions belonging to a Hölder class. We demonstrate the advantages of our proposed geometric dependency estimator in a series of experiments.
Keywords: Henze–Penrose mutual information; Friedman–Rafsky test statistic; geometric mutual information; convergence rates; bias and variance tradeoff; optimization; minimal spanning trees Henze–Penrose mutual information; Friedman–Rafsky test statistic; geometric mutual information; convergence rates; bias and variance tradeoff; optimization; minimal spanning trees
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
SciFeed

Share & Cite This Article

MDPI and ACS Style

Yasaei Sekeh , S.; Hero, A.O. Geometric Estimation of Multivariate Dependency. Entropy 2019, 21, 787.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top