A Lower Bound on the Differential Entropy of Log-Concave Random Vectors with Applications
Center for the Mathematics of Information, California Institute of Technology, Pasadena, CA 91125, USA
Department of Electrical Engineering, California Institute of Technology, Pasadena, CA 91125, USA
Author to whom correspondence should be addressed.
Received: 18 January 2018 / Revised: 6 March 2018 / Accepted: 6 March 2018 / Published: 9 March 2018
PDF [496 KB, uploaded 12 March 2018]
We derive a lower bound on the differential entropy of a log-concave random variable X
in terms of the p
-th absolute moment of X
. The new bound leads to a reverse entropy power inequality with an explicit constant, and to new bounds on the rate-distortion function and the channel capacity. Specifically, we study the rate-distortion function for log-concave sources and distortion measure
, and we establish that the difference between the rate-distortion function and the Shannon lower bound is at most
bits, independently of r
and the target distortion d
. For mean-square error distortion, the difference is at most
bit, regardless of d
. We also provide bounds on the capacity of memoryless additive noise channels when the noise is log-concave. We show that the difference between the capacity of such channels and the capacity of the Gaussian channel with the same noise power is at most
bit. Our results generalize to the case of a random vector X
with possibly dependent coordinates. Our proof technique leverages tools from convex geometry.
This is an open access article distributed under the Creative Commons Attribution License
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
Share & Cite This Article
MDPI and ACS Style
Marsiglietti, A.; Kostina, V. A Lower Bound on the Differential Entropy of Log-Concave Random Vectors with Applications. Entropy 2018, 20, 185.
Marsiglietti A, Kostina V. A Lower Bound on the Differential Entropy of Log-Concave Random Vectors with Applications. Entropy. 2018; 20(3):185.
Marsiglietti, Arnaud; Kostina, Victoria. 2018. "A Lower Bound on the Differential Entropy of Log-Concave Random Vectors with Applications." Entropy 20, no. 3: 185.
Show more citation formats
Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.
[Return to top]
For more information on the journal statistics, click here
Multiple requests from the same IP address are counted as one view.