# Rényi Entropy Power Inequalities via Normal Transport and Rotation

^{1}

^{2}

## Abstract

**:**

## 1. Introduction

**Notation**

**1.**

**Lemma**

**1**(Normal Transport).

**Lemma**

**2**

## 2. Preliminary Definitions and Properties

**Definition**

**1**(Conjugate Exponent).

**Remark**

**1.**

$p>1$ | or | $0<p<1$ |

${p}^{\prime}>1$ | ${p}^{\prime}<0$ |

**Definition**

**2**(Rényi Entropy).

**Lemma**

**3**(Scaling Property).

**Proof.**

**Lemma**

**4**(Rényi Entropy of the Normal).

**Proof.**

**Definition**

**3**

**Lemma**

**5**(Monotonicity Property).

**Proof.**

**Remark**

**2.**

## 3. An Information Inequality

**Theorem**

**1**(Information Inequality).

**Proof.**

**Remark**

**3.**

**Corollary**

**1**(Conditional Information Inequality)

**Proof.**

**Theorem**

**2**(Transformational Invariance).

**Proof.**

**Remark**

**4.**

## 4. First Version of the Rényi EPI

**Definition**

**4**(Exponent Triple).

$p,q,r>1$ | or | $0<p,q,r<1$ |

${p}^{\prime},{q}^{\prime},{r}^{\prime}>1$ | ${p}^{\prime},{q}^{\prime},{r}^{\prime}<0$ | |

${r}^{\prime}<{p}^{\prime}$, ${r}^{\prime}<{q}^{\prime}$ | $|{r}^{\prime}|<|{p}^{\prime}|$, $|{r}^{\prime}|<|{q}^{\prime}|$ | |

$r>p$, $r>q$ | $r<p$, $r<q$ |

**Lemma**

**6**(Transformational Invariance for Two Independent Variables).

**Proof.**

**Lemma**

**7.**

**Proof.**

**Theorem**

**3**

**Proof.**

**Corollary**

**2**(Rényi EPI for Several Variables).

**Proof.**

## 5. Recent Versions of the Rényi EPI

**Definition**

**5**

- a maximum possible value of c in Equation (57) since the inequality is automatically satisfied for all positive constants ${c}^{\prime}<c$.
- a minimum possible value of $\alpha $ in Equation (58) since the inequality is automatically satisfied for all positive exponents ${\alpha}^{\prime}>\alpha $; in fact, since Equation (58) is homogeneous by scaling the variables ${X}_{i}\mapsto a{X}_{i}$ as in Equation (56), one may suppose without loss of generality that the r.h.s. of Equation (58) is $=1$; then, ${N}_{r}\left({X}_{i}\right)<1$, hence ${{N}_{r}^{\phantom{2}}}^{{\alpha}^{\prime}}\left({X}_{i}\right)<{{N}_{r}^{\phantom{2}}}^{\alpha}\left({X}_{i}\right)$ for all i and $1\phantom{\rule{3.33333pt}{0ex}}=\phantom{\rule{3.33333pt}{0ex}}\sqrt[\alpha ]{{\sum}_{i=1}^{m}{{N}_{r}^{}}^{\alpha}\left({X}_{i}\right)}\ge \sqrt[{\alpha}^{\prime}]{{\sum}_{i=1}^{m}{{N}_{r}^{}}^{{\alpha}^{\prime}}\left({X}_{i}\right)}$.

**Lemma**

**8.**

**Proof.**

#### 5.1. Rényi Entropy Power Inequalities for Orders >1

**Theorem**

**4**

**Proof.**

**Remark**

**5.**

**Remark**

**6.**

**Theorem**

**5.**

**Proof.**

**Remark**

**7.**

**Proof.**

#### 5.2. Rényi Entropy Power Inequalities for Orders <1 and Log-Concave Densities

**Definition**

**6**(Log-Concave Density).

**Lemma**

**9.**

**Proof.**

**Corollary**

**3**

**Proof.**

**Corollary**

**4**

**Theorem**

**7.**

**Proof.**

**Theorem**

**8.**

**Proof.**

**Remark**

**8.**

**Theorem**

**9.**

**Proof.**

## 6. Conclusions

## Funding

## Conflicts of Interest

## References

- Shannon, C.E. A Mathematical Theory of Communication. Bell Syst. Tech. J.
**1948**, 27, 623–656. [Google Scholar] [CrossRef] - Rioul, O. Information Theoretic Proofs of Entropy Power Inequalities. IEEE Trans. Inf. Theory
**2011**, 57, 33–55. [Google Scholar] [CrossRef] [Green Version] - Dembo, A.; Cover, T.M.; Thomas, J.A. Information Theoretic Inequalities. IEEE Trans. Inf. Theory
**1991**, 37, 1501–1518. [Google Scholar] [CrossRef] - Cover, T.M.; Thomas, J.A. Elements of Information Theory, 2nd ed.; Wiley: Hoboken, NJ, USA, 2006. [Google Scholar]
- Rényi, A. On Measures of Information and Entropy. In Proceedings of the Fourth Berkeley Symposium on Mathematics, Statistics and Probability, Berkeley, CA, USA, 20 June–30 July 1960; University of California Press: Berkeley, CA, USA, 1960; Volume 1, pp. 547–561. [Google Scholar]
- Campbell, L.L. A Coding Theorem and Rényi’s Entropy. Inf. Control
**1965**, 8, 423–429. [Google Scholar] [CrossRef] - Ben-Bassat, M.; Raviv, J. Rényi’s Entropy and the Probability of Error. IEEE Trans. Inf. Theory
**1978**, 24, 324–331. [Google Scholar] [CrossRef] - Arimoto, S. Information Measures and Capacity of Order α for Discrete Memoryless Channels. In Topics in Information Theory, Colloquia mathematica Societatis János Bolyai 16; Csiszár, I., Elias, P., Eds.; North Holland: Amsterdam, The Netherlands, 1977; pp. 41–52. [Google Scholar]
- Arikan, E. An Inequality on Guessing and its Application to Sequential Decoding. IEEE Trans. Inf. Theory
**1996**, 42, 99–105. [Google Scholar] [CrossRef] [Green Version] - Erdogmus, D.; Hild, K.E.; Principe, J.C.; Lazaro, M.; Santamaria, I. Adaptive Blind Deconvolution of Linear Channels Using Rényi’s Entropy with Parzen Window Estimation. IEEE Trans. Signal Process.
**2004**, 52, 1489–1498. [Google Scholar] [CrossRef] - Savaré, G.; Toscani, G. The Concavity of Rényi Entropy Power. IEEE Trans. Inf. Theory
**2014**, 60, 2687–2693. [Google Scholar] [CrossRef] - Madiman, M.; Melbourne, J.; Xu, P. Forward and Reverse Entropy Power Inequalities in Convex Geometry. In Convexity and Concentration; Carlen, E., Madiman, M., Werner, E., Eds.; IMA Volumes in Mathematics and its Applications; Springer: Berlin, Germany, 2017; Volume 161, pp. 427–485. [Google Scholar]
- Bobkov, S.G.; Chistyakov, G.P. Entropy Power Inequality for the Rényi Entropy. IEEE Trans. Inf. Theory
**2015**, 61, 708–714. [Google Scholar] [CrossRef] - Ram, E.; Sason, I. On Rényi Entropy Power Inequalities. IEEE Trans. Inf. Theory
**2016**, 62, 6800–6815. [Google Scholar] [CrossRef] [Green Version] - Bobkov, S.G.; Marsiglietti, A. Variants of the Entropy Power Inequality. IEEE Trans. Inf. Theory
**2017**, 63, 7747–7752. [Google Scholar] [CrossRef] [Green Version] - Li, J. Rényi Entropy Power Inequality and a Reverse. Stud. Math.
**2018**, 242, 303–319. [Google Scholar] [CrossRef] - Marsiglietti, A.; Melbourne, J. On the Entropy Power Inequality for the Rényi Entropy of Order [0, 1]. arxiv, 2018; arXiv:1710.00800. [Google Scholar]
- Rioul, O. Yet Another Proof of the Entropy Power Inequality. IEEE Trans. Inf. Theory
**2017**, 63, 3595–3599. [Google Scholar] [CrossRef] [Green Version] - Rosenblatt, M. Remarks on a Multivariate Transformation. Ann. Math. Stat.
**1952**, 23, 470–472. [Google Scholar] [CrossRef] - Knothe, H. Contributions to the theory of convex bodies. Mich. Math. J.
**1957**, 4, 39–52. [Google Scholar] [CrossRef] - Rioul, O. Optimal Transportation to the Entropy-Power Inequality. In Proceedings of the IEEE Information Theory and Applications Workshop (ITA 2017), San Diego, CA, USA, 12–17 February 2017. [Google Scholar]
- Bryc, W. The Normal Distribution—Characterizations with Applications; Lecture Notes in Statistics; Springer: Berlin, Germany, 1995. [Google Scholar]
- Rioul, O. Optimal Transport to Rényi Entropies. In Proceedings of the 3rd Conference on Geometric Science of Information (GSI 2017), Paris, France, 7–9 November 2017; Lecture Notes in Computer Science. Springer: Berlin, Germany, 2017. [Google Scholar]
- Bercher, J.F. Source Coding with Escort Distributions and Rényi Entropy Bounds. Phys. Lett. A
**2009**, 373, 3235–3238. [Google Scholar] [CrossRef] [Green Version] - Bercher, J.F. On Generalized Cramér-Rao Inequalities, Generalized Fisher Information and Characterizations of Generalized q-Gaussian Distributions. J. Phys. A Math. Theor.
**2012**, 45, 255303. [Google Scholar] [CrossRef] [Green Version] - Beck, C.; Schlögl, F. Thermodynamics of Chaotic Systems: An Introduction; Cambridge University Press: Cambridge, UK, 1993. [Google Scholar]
- Bercher, J.F.; (ESIEE Paris, Marne-la-Vallée, France). Private communication, 2018.
- Fradelizi, M.; Madiman, M.; Wang, L. Optimal Concentration of Information Content for Log-Concave Densities. In High Dimensional Probability VII: The Cargèse Volume; Houdré, C., Mason, D.M., Reynaud-Bouret, P., Rosiński, J., Eds.; Birkhäuser: Basel, Switzerland, 2016. [Google Scholar]

© 2018 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Rioul, O.
Rényi Entropy Power Inequalities via Normal Transport and Rotation. *Entropy* **2018**, *20*, 641.
https://doi.org/10.3390/e20090641

**AMA Style**

Rioul O.
Rényi Entropy Power Inequalities via Normal Transport and Rotation. *Entropy*. 2018; 20(9):641.
https://doi.org/10.3390/e20090641

**Chicago/Turabian Style**

Rioul, Olivier.
2018. "Rényi Entropy Power Inequalities via Normal Transport and Rotation" *Entropy* 20, no. 9: 641.
https://doi.org/10.3390/e20090641