Next Article in Journal
Gallager Exponent Analysis of Coherent MIMO FSO Systems over Gamma-Gamma Turbulence Channels
Next Article in Special Issue
Strongly Convex Divergences
Previous Article in Journal
Classification of Actigraphy Records from Bipolar Disorder Patients Using Slope Entropy: A Feasibility Study
Previous Article in Special Issue
On Relations Between the Relative Entropy and χ2-Divergence, Generalizations and Applications
 
 
Article

A Two-Moment Inequality with Applications to Rényi Entropy and Mutual Information

by 1,2
1
Department of Electrical and Computer Engineering, Duke University, Durham, NC 27708, USA
2
Department of Statistical Science, Duke University, Durham, NC 27708, USA
Entropy 2020, 22(11), 1244; https://doi.org/10.3390/e22111244
Received: 14 September 2020 / Accepted: 6 October 2020 / Published: 1 November 2020
This paper explores some applications of a two-moment inequality for the integral of the rth power of a function, where 0<r<1. The first contribution is an upper bound on the Rényi entropy of a random vector in terms of the two different moments. When one of the moments is the zeroth moment, these bounds recover previous results based on maximum entropy distributions under a single moment constraint. More generally, evaluation of the bound with two carefully chosen nonzero moments can lead to significant improvements with a modest increase in complexity. The second contribution is a method for upper bounding mutual information in terms of certain integrals with respect to the variance of the conditional density. The bounds have a number of useful properties arising from the connection with variance decompositions. View Full-Text
Keywords: information inequalities; mutual information; Rényi entropy; Carlson–Levin inequality information inequalities; mutual information; Rényi entropy; Carlson–Levin inequality
Show Figures

Figure 1

MDPI and ACS Style

Reeves, G. A Two-Moment Inequality with Applications to Rényi Entropy and Mutual Information. Entropy 2020, 22, 1244. https://doi.org/10.3390/e22111244

AMA Style

Reeves G. A Two-Moment Inequality with Applications to Rényi Entropy and Mutual Information. Entropy. 2020; 22(11):1244. https://doi.org/10.3390/e22111244

Chicago/Turabian Style

Reeves, Galen. 2020. "A Two-Moment Inequality with Applications to Rényi Entropy and Mutual Information" Entropy 22, no. 11: 1244. https://doi.org/10.3390/e22111244

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop