Abstract
Recently, Savaré-Toscani proved that the Rényi entropy power of general probability densities solving the p-nonlinear heat equation in  is a concave function of time under certain conditions of three parameters , which extends Costa’s concavity inequality for Shannon’s entropy power to the Rényi entropy power. In this paper, we give a condition  of  under which the concavity of the Rényi entropy power is valid. The condition  contains Savaré-Toscani’s condition as a special case and much more cases. Precisely, the points  satisfying Savaré-Toscani’s condition consist of a two-dimensional subset of , and the points satisfying the condition  consist a three-dimensional subset of . Furthermore,  gives the necessary and sufficient condition in a certain sense. Finally, the conditions are obtained with a systematic approach.
  1. Introduction
In 1948, Claude Elwood Shannon [] first introduced his mathematical theory of information. In particular, he presented the concept of entropy as a measure for information. On this foundation, Alfréd Rényi [] then built one of his contributions in 1961. At the center, he introduced a new notion of entropy that included that of Shannon as a special case, and this is called Rényi entropy.
The p-th Rényi entropy [,] of a probability density function  is defined as
      
      
        
      
      
      
      
    
      for , . The p-th Rényi entropy power is given by
      
      
        
      
      
      
      
    
      where  is a real-valued parameter. The Rényi entropy for  is defined as the limit of  as . It follows from definition (1) that
      
      
        
      
      
      
      
    
      which is Shannon’s entropy. Thus, the Rényi entropy power of index , given by (2). coincides with Shannon’s entropy power
      
      
        
      
      
      
      
    
Shannon’s entropy power inequality (EPI) is one of the most important information inequalities [], which has many proofs, generalizations, and applications [,,,,,,,,]. In particular, Costa presented a stronger version of the EPI in his seminal paper [].
Let  be the n-dimensional random vector introduced by Costa [,,,] and  the probability density of , which solves the heat equation in the whole space ,
      
      
        
      
      
      
      
    
Costa’s differential entropy is defined to be
      
      
        
      
      
      
      
    
Related to EPI, Costa [] proved that the Shannon entropy power  is a concave function in t; that is,  and . Several new proofs and generalizations for Costa’s EPI were given in [,,,].
Savaré-Toscani [] proved that the concavity of entropy power is a property which is not restricted to the Shannon entropy power (3) in connection with the heat Equation (4), but it holds for the p-th Rényi entropy power (2). They put it in connection with the solution to the nonlinear heat equation
      
        
      
      
      
      
    
      posed in the whole space  and  and show that  and  hold if  satisfy certain conditions.
In this paper, we give a generalization for the concavity of the p-th Rényi entropy power (CREP). Precisely, we give a propositional logic formula  such that if  satisfy this formula, then the CREP holds. The condition  extends the parameter range of the CREP given by Savaré-Toscani [] and contains many more cases. Precisely, the points  satisfying the condition given in [] consist of a two-dimensional subset of  and the points satisfying the condition  consist of a three-dimensional subset of . Furthermore,  gives the necessary and sufficient condition for CREP to be valid in a certain sense.
The formula  is obtained using a systematic procedure which can be considered as a parametric version of that given in [,,,], where parameters  exist in the formulas. The procedure reduces the proof of the CREP to check the semi-positiveness of a quadratic form whose coefficients are polynomials in the parameters . In principle, a necessary and sufficient condition for the parameters to satisfy this property can be computed with the quantifier elimination []. In this paper, the problem is in a special form and an explicit proof is given.
2. Proof Procedure
In this section, we present a procedure to prove the CREP. To make the paper concise, we only give those steps that are needed in this paper.
2.1. Notations
Let  be a set of variables depending on t and
        
      
        
      
      
      
      
    
Let  and . To simplify the notations, we use u to denote  in the rest of the paper. Denote
        
      
        
      
      
      
      
    
        as the set of all derivatives of u with respect to the differential operators ,  as the set of polynomials in parameters , and
        
      
        
      
      
      
      
    
        as the set of polynomials in  with coefficients in . For , we say v has order . For a monomial  with , its degree, order, and total order are defined to be , , and , respectively.
A polynomial in  is called a kth-order differentially homogenous polynomial or simply a kth-order differential form, if all its monomials have degree k and total order k. Let  be the set of all monomials which have degree k and total order k. Then, the set of kth-order differential forms is an -linear vector space generated by , which is denoted as . We use Gaussian elimination in  by treating the monomials as variables. We always use the lexicographic order for the monomials defined in [,,].
2.2. Sketch of the Proof
In this section, we give the procedure to prove the CREP. The property  can be easily proved []. We focus on proving . The procedure consists of four steps.
In step 1, we reduce the proof of CREP into the proof of an integral inequality, as shown by the following lemma, the proof of which is given in Section 2.4.
Lemma 1. 
The proof of  can be reduced to show
      
        
      
      
      
      
    under the condition , where  is a fourth-order differential form in  and
      
        
      
      
      
      
    
In step 2, we compute the constraints, which are relations satisfied by the probability density u of . Since  in (7) is a fourth-order differential form, we need only the constraints which are fourth-order differential forms. A fourth-order differential form R is called an equational or inequality constraint if
        
      
        
      
      
      
      
    
The method to compute the constraints is given in Section 2.3. Suppose that the equational and inequality constraints are respectively
        
      
        
      
      
      
      
    
      
        
      
      
      
      
    
In step 3, we find a propositional formula  such that when  and  satisfy ,
        
      
        
      
      
      
      
    
        where S is a sum of squares (SOS). Details of this step and the formula  are given in Section 3.
To summarize the proof procedure, we have
Theorem 1. 
The CREP is true if  is valid.
Proof.  
By Lemma 1, we have the following proof for CREP:
          
      
        
      
      
      
      
    Equality S1 is true, because  are equational constraints. Inequality S2 is true, because  are inequality constraints. Inequality S3 is true, because S is an SOS and hence  under the condition .    □
2.3. The Equational Constraints
In this section, we show how to find the second-order equational constraints. A second-order equational constraint is a fourth-order differential form in  such that . We need the following property.
Property 1. 
Let  and  be an th-order derivative of u. If  is a smooth, strictly positive and rapidly decaying probability density, then
      
        
      
      
      
      
    with.
When , Property 1 follows from []. While , we make the assumption that  also satisfies Property 1.
Using Property 1, we can compute 28 second-order equational constraints using the method given in [,,]:
      
        
      
      
      
      
    
        where  can be found in the Appendix A. Note that  are variables taking values in .
We use an example to show how to obtain these constraints. Starting from a monomial  with degree 4 and total order 4, using integral by parts, we have
        
      
        
      
      
      
      
    
Then,
        
      
        
      
      
      
      
    
We then obtain a 2th-order constraint: . The other 27 constraints in  are obtained in the same way.
2.4. Proof of Lemma 1
We first prove several lemmas.
Lemma 2. 
      
        
      
      
      
      
    
      
        
      
      
      
      
    
Proof.  
□
Lemma 3. 
We have
      
        
      
      
      
      
    
Proof.  
Integrating by parts [], we have
          
      
        
      
      
      
      
    
□
By Cauchy–Schwarz inequality, we have
Lemma 4.  
      
        
      
      
      
      
    
Remark 1. 
Furthermore, we have
        
      
        
      
      
      
      
    
        where
        
      
        
      
      
      
      
    
For convenience, introduce the notation . Then, by calculating the differentiation formulas in (25) and substituting , we have , where
        
      
        
      
      
      
      
    
        which is a fourth-order differential form.
3. A Generalized Version of CREP
In this section, we prove a generalized CREP using the procedure given in Section 2.
Theorem 2. 
The proof of the above theorem consists of three steps, which are given in the following three subsections.
3.1. Reduce to a Finite Problem
We first give an inequality constraint. Denote . Then, based on the trace inequality , we give an inequality constraint:
      
        
      
      
      
      
    
        where .
Problem 1. 
Since  and , we have
        
      
        
      
      
      
      
    
        where .
From (30), in order to solve Problem 1, it suffices to solve
Problem 2. 
Find a formula  such that  under the conditions , , and .
3.2. Simplify the Problem with the Equational Constraints
In this section, we simplify  in Problem 2 with the equational constraints  in (15). Note that the subscripts a and b are fixed and are treated as symbols.
Our goal is to reduce  into a quadratic form in certain new variables. The new variables are all the monomials in  with degree 2 and total order 2:
      
        
      
      
      
      
    
        where  is defined in (8).
We simplify the constraints in (15) as follows. A quadratic monomial in  is called a quadratic monomial. Write monomials in  as quadratic monomials if possible. Performing Gaussian elimination to  by treating the monomials as variables, and according to a monomial order such that a quadratic monomial is less than a non-quadratic monomial, we obtain
        
      
        
      
      
      
      
    
        where  is the set of quadratic forms in ,  is the set of non-quadratic forms, and . We obtain  and , where
        
      
        
      
      
      
      
    
      
        
      
      
      
      
    
We now simplify  using  and . Eliminating the non-quadratic monomials in  using , and performing further reduction by , we have
        
      
        
      
      
      
      
    
In order for  to be true, we need to eliminate the monomial  from , which can be done with  as follows.
        
      
        
      
      
      
      
    
        where
        
      
        
      
      
      
      
    
3.3. Compute
From (32), in order to solve Problem 2, it suffices to solve
Problem 3. 
Find a propositional formula  such that
      
        
      
      
      
      
    
In principle, Problem 3 can be solved with the quantifier elimination []. In this paper, the problem is special, and an explicit proof is given.
By the knowledge of linear algebra,  is equivalent to , , , where
        
      
        
      
      
      
      
    
        and  and  are defined in (37). Furthermore,  is assumed in . Thus, the following lemma is proved.
Lemma 5. 
We have
      
        
      
      
      
      
    
We present an explicit formula for  in (34). First, we introduce the following parameters.
        
      
        
      
      
      
      
    
We define  in (34) using Table 1, where * means ⌀. Define  to be the formula in the i-th row and the j-th column in Table 1. Then, we denote
        
      
        
      
      
      
      
    
       
    
    Table 1.
    The description for  in (34).
  
For example,  is , which means that if  satisfy , then there exists a  such that (33) is true and the CREP is valid. , which means that there exist no values for  such that (33), and the CREP is true in this case.
We now give the main result of the paper, which implies Theorem 2. The proof for the theorem can be found in Section 3.5.
3.4. Compare with Existing Results
We show that our result includes the result proved in [] and more essential results.
In [], the CREP was proved under the conditions  and . Obviously, the result proved in [] is a special case of  in Table 1.
We can also prove the result in [] directly as follows. Set  and  in (31), we obtain . In addition, the condition  implies . So, when  and , the CREP is proved based on our proof procedure.
We can use the SDP code in ([], Appendix B) to verify the result in Table 1 for given values of . For instance, for , the condition  is satisfied naturally. With the SDP code in [], we obtain  with . Thus, the CREP is proved when . This case  is included in  in Table 1. Note that  is not satisfied for these parameters, and thus our condition  is strictly larger than those given in []. More precisely, the points  satisfying the conditions  given in [] consist of a two-dimensional subset of , while the points satisfying the condition  consist of a three-dimensional subset of , as shown by the following result.
Property 2. 
The points satisfying the condition  consist of a three-dimensional subset of .
Proof.  
We show that the points satisfying  consist of a three-dimensional subset of .
From Table 1, we have , where , , . Under the condition , we can reduce the inequality  to the form  and reduce the inequality  to the form . Thus, . Since  under the condition ,  defines a three-dimensional subset of .    □
3.5. Proof of Theorem 3
In order to make the proof precise, we introduce the following parameters:
      
        
      
      
      
      
    
We first treat the three inequalities , , . Firstly,  is equivalent to . Secondly, since the roots of  are  and , we have  if  or ; and  if . In order to analyze , we first compute
        
      
        
      
      
      
      
    
Therefore,  can be divided into four cases:  if  and ;  if  and ;  if  and ;  if  and . Finally,  is equivalent to .
Based on the above analysis and (34),  can be divided into six cases: 
      
        
      
      
      
      
    
The special cases , and  need to be considered differently.
Below, we give a detailed analysis of the above six cases, which leads to the results in Table 1. We first have the following formulas:
      
        
      
      
      
      
    
      
        
      
      
      
      
    
      
        
      
      
      
      
    
      
        
      
      
      
      
    
      
        
      
      
      
      
    
      
        
      
      
      
      
    
      
        
      
      
      
      
    
      
        
      
      
      
      
    
      
        
      
      
      
      
    
Firstly, we have the following formulas which eliminate .
        
      
        
      
      
      
      
    
We divide the proof into several cases, first according to the values of  and then according to the values of n.
, if  or ;
, if ;
 or , if .
According to the vales of p, we consider seven cases below.
Case 1.1:  and . In this case, from (42) and (44), we have  and . Hence, .
We now eliminate  from . By (49),  is equivalent to . ) is equivalent to (). Therefore, in this case, , and  is proved.
Case 1.2:  and . When , we have . Then, . Because  and , we have . By (44), we have . When , we have  and . Thus .
We now eliminate  from . By (49),  is equivalent to .  is equivalent to  and . Therefore, in this case, , and  is proved.
Case 1.3:  and , . This case is divided into two sub-cases.
Case 1.3.1:  and . By (43) and (44), we have  and . Hence, .
We now eliminate  from . Similar to Case 1.1, we have ,  is proved.
Case 1.3.2:  and , . By (43)–(46), we have , (), () and (). Hence . This case is further divided into two sub-cases.
Case 1.3.2.1: If , then , and . Thus, we need , which yields . Thus, .
We now eliminate  from . Like Case 1.1, we have , and  is proved
Case 1.3.2.2: If , then , and . Thus, we need , which yields . By (47), we know  results in , which yields . Thus, .
We now eliminate  from .  is equivalent to  and . ) is equivalent to . Therefore, in this case, , and  is proved.
Case 1.4:  and . When , we have . Then . Because  and , we have . By (44), we have .
Case 1.4.1: Similar to Case 1.3.2.1, . When , we have . Thus .
We now eliminate  from .  is equivalent to .  is equivalent to  and . Therefore, in this case , and  is proved.
Case 1.4.2: Similar to Case 1.3.2.2, . When , we have . Thus, .
We now eliminate  from .  is equivalent to  and .  is equivalent to . Therefore, in this case, , and  is proved.
Case 1.5:  and . By (42) and (44), we have  and (). Hence .
Case 1.5.1: Similar to Case 1.3.2.1, we have .
We now eliminate  from . Like Case 1.3.2.1, we have , and  is proved
Case 1.5.2: Similar to Case 1.3.2.2, we have .
We now eliminate  from . Like Case 1.3.2.2, we have , and  is proved.
Case 1.6:  and . When , we have . Then  if . By (45), we know . And . Thus .
We now eliminate  from .  is equivalent to  and .  is equivalent to . Therefore, in this case , and  is proved
Case 1.7:  and .
Case 1.7.1: If we select , by (42), we have . Thus, . So, we need , which yields . By (47), we know  results in , which yields  with . Thus .
We now eliminate  from . Like Case 1.3.2.2, we have , and  is proved.
Case 1.7.2: If we select , by (41), we have , which yields a contradiction.
Case 2: . From (40), we have  in this case, and from (35),  simplifies to one case: , if  and . Since p satisfies , we need only consider the following cases.
Case 2.1:  and . By (41), (43) and (44), we have ,  and (). Then, we need , which yields  by (46). Because  means  with , we have .
We now eliminate  from .  is equivalent to  and .  is equivalent to . Therefore, in this case, , and  is proved.
Case 2.2:  and . In Case 1.5, we know that  with , and , which yields a contradiction.
Case 2.3:  and . In Case 1.6, we know that  with , and , which yields a contradiction.
Case 2.4:  and . We have  based on . Then, we have () and ( if ). So, we need . By (42), we have , which yields a contradiction.
Case 3: . When , we have ,  and . Thus,  and .
Case 3.1: If , then . Furthermore, . Thus, , and  are proved.
Case 3.2: If , then . Then, we need , which yields . And  implies  with . Thus, , and ,  are proved.
4. Conclusions
This paper is an extension of the work [,,] to the case where the entropy power involves parameters. The basic idea is to prove entropy power inequalities in a systematic way. Precisely, the concavity of Rényi entropy power is considered, where the probability density  solves the nonlinear heat equation with two parameters p and . Our procedure reduces the proof of the CREP to checking the semi-positiveness of a quadratic form (33) whose coefficients are polynomials in the parameters . In principle, a necessary and sufficient condition on parameters  for this can be computed with the quantifier elimination []. Some interesting works [,] can help to understand our approach in this paper.
Based on the above method, we give a sufficient condition  for the CREP, which extends the parameter’s range of the CREP given by Savaré-Toscani []. By Theorem 3, our results give the necessary and sufficient condition for the CREP under certain conditions. However, in the general case, Theorem 1 only gives a sufficient condition for the following reasons: Problem 1 may not be equivalent to Problem 2, and more constraints may exist.
For future research, it is interesting to see whether the three conjectures about Costa’s differential entropy studied in [] can be generalized to this more general case.
Author Contributions
Conceptualization, L.G.; Formal analysis, L.G., C.-M.Y. and X.-S.G.; Funding acquisition, L.G., C.-M.Y. and X.-S.G.; Investigation, L.G. and X.-S.G.; Methodology, L.G.; Project administration, X.-S.G.; Resources, L.G.; Software, L.G.; Supervision, X.-S.G. All authors have read and agreed to the published version of the manuscript.
Funding
This work is supported by NSFC 11688101 and NKRDP 2018YFA0704705, Beijing Natural Science Foundation (No. Z190004), and the Fundamental Research Funds for the Central Universities 2021NTST32.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
Not applicable.
Conflicts of Interest
The authors declare no conflict of interest.
Appendix A. Constraints in (15)
In this appendix, we give the constraints in (15), where .
References
- Shannon, C.E. A mathematical theory of communications. Bell Syst. Tech. J. 1948, 27, 379–423. [Google Scholar] [CrossRef] [Green Version]
 - Rényi, A. On measures of entropy and information. In Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics; University of California Press: Berkeley, CA, USA, 1961; pp. 547–561. [Google Scholar]
 - Cover, T.M. Elements of Information Theory; John Wiley & Sons: New York, NY, USA, 1999. [Google Scholar]
 - Gardner, R.J. The Brunn-Minkowski inequality. Bull. Am. Math. Soc. 2002, 39, 355–405. [Google Scholar] [CrossRef] [Green Version]
 - Stam, A.J. Some inequalities satisfied by the quantities of information of Fisher and Shannon. Inf. Control 1959, 2, 101–112. [Google Scholar] [CrossRef] [Green Version]
 - Blachman, N.M. The convolution inequality for entropy powers. IEEE Trans. Inf. Theory 1965, 11, 267–271. [Google Scholar] [CrossRef]
 - Lieb, E.H. Proof of an entropy conjecture of Wehrl. Commun. Math. Phys. 1978, 62, 35–41. [Google Scholar] [CrossRef]
 - Verdú, S.; Guo, D. A simple proof of the entropy-power inequality. IEEE Trans. Inf. Theory 2006, 52, 2165–2166. [Google Scholar] [CrossRef]
 - Rioul, O. Information theoretic proofs of entropy power inequalities. IEEE Trans. Inf. Theory 2011, 57, 33–55. [Google Scholar] [CrossRef] [Green Version]
 - Bergmans, P.P. A simple converse for broadcast channels with additive white Gaussian noise. IEEE Trans. Inform. Theory 1974, 20, 279–280. [Google Scholar] [CrossRef]
 - Zamir, R.; Feder, M. A generalization of the entropy power with respect to added Gaussian noise. IEEE Trans. INf. Theory 1993, 39, 1723–1728. [Google Scholar] [CrossRef]
 - Liu, T.; Viswanath, P. An extremal inequality motivated by multiterminal information-theoretic problems. IEEE Trans. Inf. Theory 2007, 53, 1839–1851. [Google Scholar] [CrossRef] [Green Version]
 - Wang, L.; Madiman, M. A new approach to the entropy power inequality, via rearrangements. In Proceedings of the 2013 IEEE International Symposium on Information Theory, Istanbul, Turkey, 7–12 July 2013. [Google Scholar]
 - Costa, M.H.M. A new entropy power ineqaulity. IEEE Trans. Inf. Theory 1985, 31, 751–760. [Google Scholar] [CrossRef]
 - Guo, L.; Yuan, C.M.; Gao, X.S. Prove Costa’s entropy power inequality and high order inequality for differential entropy with semidefinite programming. arXiv 2020, arXiv:2004.08543. [Google Scholar]
 - Guo, L.; Yuan, C.M.; Gao, X.S. Lower bound on derivatives of Costa’s differential entropy. arXiv 2020, arXiv:2007.10145. [Google Scholar]
 - Guo, L.; Yuan, C.; Gao, X. Lower bound for derivatives of Costa’s differential entropy. In Proceedings of the IEEE International Symposium on Information Theory (ISIT), Melbourne, Australia, 12–20 July 2021; pp. 1516–1521. [Google Scholar]
 - Dembo, A. Simple proof of the concavity of the entropy power with respect to added Gaussian noise. IEEE Trans. Inf. Theory 1989, 35, 887–888. [Google Scholar] [CrossRef]
 - Villani, C. A short proof of the concavity of entropy power. IEEE Trans. Inf. Theory 2000, 46, 1695–1696. [Google Scholar] [CrossRef] [Green Version]
 - Toscani, G. A concavity property for the reciprocal of Fisher information and its consequences on Costa’s EPI. Phys. A Stat. Mech. Appl. 2015, 432, 352. [Google Scholar] [CrossRef] [Green Version]
 - Cheng, F.; Geng, Y. Higher order derivatives in Costa’s entropy power inequality. IEEE Trans. Inf. Theory 2015, 61, 5892–5905. [Google Scholar] [CrossRef]
 - Savaré, G.; Toscani, G. The concavity of Rényi entropy power. IEEE Trans. Inf. Theory 2014, 60, 2687–2693. [Google Scholar] [CrossRef] [Green Version]
 - Zhang, X.; Anantharam, V.; Geng, Y. Gaussian optimality for derivatives of differential entropy using linear matrix inequalities. Entropy 2018, 20, 182. [Google Scholar] [CrossRef] [Green Version]
 - Caviness, B.F.; Johnson, J.R. Quantifier Elimination and Cylindrical Algebraic Decomposition; Springer: New York, NY, USA, 1998. [Google Scholar]
 - Treves, F. Topological Vector Spaces, Distributions and Kernels; Pure and Applied Mathematics; Elsevier: New York, NY, USA, 2016. [Google Scholar]
 - Abbas, I.A.; Marin, M. Analytical solutions of a two-dimensional generalized thermoelastic diffusions problem due to laser pulse. Iran. J. Sci. Technol. Trans. Mech. Eng. 2018, 42, 57–71. [Google Scholar] [CrossRef]
 - Marin, M.; Othman, M.I.A.; Seadawy, A.R.; Carstea, C. A domain of influence in the Moore–Gibson–Thompson theory of dipolar bodies. J. Taibah Univ. Sci. 2020, 14, 653–660. [Google Scholar] [CrossRef]
 
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.  | 
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).