Two Measures of Dependence
Signal and Information Processing Laboratory, ETH Zurich, 8092 Zurich, Switzerland
Author to whom correspondence should be addressed.
Received: 5 July 2019 / Revised: 2 August 2019 / Accepted: 5 August 2019 / Published: 8 August 2019
PDF [955 KB, uploaded 8 August 2019]
Two families of dependence measures between random variables are introduced. They are based on the Rényi divergence of order
and the relative
-entropy, respectively, and both dependence measures reduce to Shannon’s mutual information when their order
is one. The first measure shares many properties with the mutual information, including the data-processing inequality, and can be related to the optimal error exponents in composite hypothesis testing. The second measure does not satisfy the data-processing inequality, but appears naturally in the context of distributed task encoding.
This is an open access article distributed under the Creative Commons Attribution License
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
Share & Cite This Article
MDPI and ACS Style
Lapidoth, A.; Pfister, C. Two Measures of Dependence. Entropy 2019, 21, 778.
Lapidoth A, Pfister C. Two Measures of Dependence. Entropy. 2019; 21(8):778.
Lapidoth, Amos; Pfister, Christoph. 2019. "Two Measures of Dependence." Entropy 21, no. 8: 778.
Show more citation formats
Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.
[Return to top]
For more information on the journal statistics, click here
Multiple requests from the same IP address are counted as one view.