An Estimator of Mutual Information and its Application to Independence Testing
Department of Mathematics, Graduate School of Science, Osaka University, Toyonaka, Osaka 560-0043, Japan
Academic Editor: Raúl Alcaraz Martínez
Received: 22 February 2016 / Revised: 14 March 2016 / Accepted: 23 March 2016 / Published: 29 March 2016
This paper proposes a novel estimator of mutual information for discrete and continuous variables. The main feature of this estimator is that it is zero for a large sample size n
if and only if the two variables are independent. The estimator can be used to construct several histograms, compute estimations of mutual information, and choose the maximum value. We prove that the number of histograms constructed has an upper bound of O
) and apply this fact to the search. We compare the performance of the proposed estimator with an estimator of the Hilbert-Schmidt independence criterion (HSIC), though the proposed method is based on the minimum description length (MDL) principle and the HSIC provides a statistical test. The proposed method completes the estimation in O
) time, whereas the HSIC kernel computation requires O
) time. We also present examples in which the HSIC fails to detect independence but the proposed method successfully detects it.
This is an open access article distributed under the Creative Commons Attribution License
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
Share & Cite This Article
MDPI and ACS Style
Suzuki, J. An Estimator of Mutual Information and its Application to Independence Testing. Entropy 2016, 18, 109.
Suzuki J. An Estimator of Mutual Information and its Application to Independence Testing. Entropy. 2016; 18(4):109.
Suzuki, Joe. 2016. "An Estimator of Mutual Information and its Application to Independence Testing." Entropy 18, no. 4: 109.
Show more citation formats
Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.
[Return to top]
For more information on the journal statistics, click here
Multiple requests from the same IP address are counted as one view.