The Ordering of Shannon Entropies for the Multivariate Distributions and Distributions of Eigenvalues

Abstract

In this paper, we prove the Shannon entropy inequalities for the multivariate distributions via the notion of convex ordering of two multivariate distributions. We further characterize the multivariate totally positive of order 2 ($MT{P}_2$) property of the distribution functions of eigenvalues of both central Wishart and central MANOVA models, and of both noncentral Wishart and noncentral MANOVA models under the general population covariance matrix set-up. These results can be directly applied to both the comparisons of two Shannon entropy measures and the power monotonicity problem for the MANOVA problem.

1. Introduction

Let ${\mathbf{X}}_1,{\mathbf{X}}_2,\cdots$ be independent random vectors with a common continuous distribution function $F(\mathbf{x})$ which has density function $f(\mathbf{x}),\mathbf{x}\in R^p$ with respect to Lebesgue measure. The Shannon entropy for $f(\mathbf{x})$ is denoted by

$$\begin{array}{c}\hfill {\mathrm{I}}_S(f)={\int }_{-\infty }^{\infty }\{-logf(\mathbf{x})\}f(\mathbf{x})d\mathbf{x}.\end{array}$$

(1)

We further use $\mathcal{F}$ to denote all density functions whose Shannon entropies exist. That is

$$\begin{array}{c}\hfill \mathcal{F}=\{f:{\mathrm{I}}_S(f)<\infty \}.\end{array}$$

(2)

Entropy inequality has been studied by several authors, such as Karlin and Rinott ([1,2], and references therein), who used the notions of majorization and Schur functions to study the Shannon (differential) entropy comparisons. Zografos and Nadarajah [3] pointed out that some comparisons of Shannon entropies can be induced by the affine transformation, those results show agreement on the concern that Shannon entropy is not scale invariant. Tsai [4] used the notion of convex ordering to investigate the Shannon inequalities for the univariate distributions. The novel finding of his approach is to adopt the difference of two Shannon entropy measures as a new measure, which is essentially symmetric. This new measure has some advantages over the well-known Kullback-Leibler divergence (relative entropy). The former one is designed for small or moderate deviation, however, the latter one is quite often used for large deviation. The measure of the difference between two Shannon entropies enjoys the finite sample property.

The key point of this method is to transform the convex ordering into the Lorenz ordering, which nicely depicts the stochastic ordering of two normalized distributions with the same starting and ending points, respectively. The presentation of the difference between two Shannon entropy measures can be represented in terms of a standard form of Kullback-Leibler divergence, however, it is interesting to note that the original Kullback-Leibler divergence does not have such kind of symmetric representation. Furthermore, the new measure is monotone likelihood ratio of two normalized density functions. This new measure provides us with some advantages for statistical inference.

The ordering of Shannon entropies can naturally occur when two underline distributions have the convex ordering or concave ordering relationship via the help of Lorenz ordering. In this paper, one of the main goals is to extend the result of Tsai [4] to the multivariate case, which is studied in Section 3. Please note that the notions of convex ordering of two distributions, monotone likelihood ratio ($MLR$) and totally positive of order 2 ($T{P}_2$) are essentially equivalent. We then adopt the notion of multivariate totally positive of order 2 ($MT{P}_2$) or multivariate reverse rule of order 2 ($MR{R}_2$) to get a more complete theory on Shannon entropy comparisons. Eigenvalues problem has become quite a hot topic at present. We further study the $MT{P}_2$ property of the distribution functions of eigenvalues of both central Wishart and central MANOVA models, and of both noncentral Wishart and noncentral MANOVA models under the general population covariance matrix set-up, respectively, in Section 4. The results can be directly applied to the comparisons of two Shannon entropy measures as well as the power monotonicity problem for the MANOVA problem, which has been open for a long time in the literature (Perlman and Olkin [5]). For a high-dimensional Wishart matrix with unknown scale times identity matrix as the population matrix, the $T{P}_2$ property of limiting empirical density function of eigenvalues is also studied in the Section 4.5. Mixture density functions, such as the noncentral chi-square density function and those presented in Section 4.3 and Section 4.4, play an important role in statistical inference. We also connected this notion with that of well-known Fisher’s multivariate analysis of variance in the final remark section.

2. The Univariate Distributions

Tsai [4] adopted the notion of convex ordering of distribution functions to study the orderings of Shannon entropy measures. For two univariate distributions $F,G\in \mathcal{F}$, it is said that F is c-$ordered$ (convex ordered) with respect to $G(F{\le }_{c}G)$ if and only if $G^{-1}F$ is convex on the interval where $0<F(x)<1$ (van Zwet [6]). The transformation of Barlow and Doksum [7] is to transform the convex ordering of distributions F and G to the stochastic ordering of distributions, it essentially can be viewed as a kind of Lorenz ordering (Gastwirth [8]).

Let $C_F(x)=G^{-1}F(x)$, then note that $d{C}_F(x)/dx=f(x)/g(G^{-1}F(x)),x\in (-\infty ,\infty )$. It can also be rewritten as $f(F^{-1}(u))/g(G^{-1}(u)),whereu=F(x),u\in [0,1]$. We adopt the notion of Barlow and Doksum [7] to denote that $H_F(u)={\int }_0^{G^{-1}(u)}f(F^{-1}G(t))dt,u\in [0,1]$, which can be viewed as the inverse function of the transformation considered by Barlow and Doksum [7]. Please note that the notation in Barlow and Doksum [7] is originally defined that $F(0)=0$ and $F^{-1}(0)=0$, we use the general notations that $F(-\infty )=0$ and $F^{-1}(0)=-\infty$ in this paper. Obviously, $H_G(u)=u,u\in [0,1]$. Under the assumption that $F(x){\le }_{c}G(y)$, Barlow and Doksum [7] pointed out that both $H_F(u)$ and $H_G(u),u\in [0,1]$ are two distribution functions. Let $h_F$ and $h_G$ be the corresponding density functions of $H_F$ and $H_G$, respectively, then we may see that $h_F(u)=f(F^{-1}(u))/g(G^{-1}(u))$ and $h_G(u)=1,u\in [0,1]$. Then, it is easy to note that the difference of two Shannon entropies can be represented as the Kullback-Leibler divergence of the uniform density function $h_G$ and the density function $h_F$.

Please note that if $F(x){\le }_{c}G(y)$, then $h_F(u)$ is nondecreasing in $u\in [0,1]$, namely, it enjoys the property of monotone likelihood ratio. In addition, it gets across with the uniform density function $h_G(u)=1,u\in [0,1]$ at most once, the sign of the difference ($h_F(u)-h_G(u),u\in [0,1]$) of two density functions is from negative to nonnegative. Also note that ${\int }_0^1(h_F(v)-1)dv=0$, thus we have that $H_F(u)\le u=H_G(u),u\in [0,1]$. Similarly, if $F(x){\le }_{c}G(y)$, then $F(x)\le G(y)$, and $C_F(x)$ is a convex function in $x,x\in (-\infty ,\infty )$ (for the details see Shaked and Shanthikumar [9]). Namely, the convex ordering implies the stochastic ordering, however, the vice versa may not be true. We provide a simple counterexample for this. For $0\le x\le 1$, let $F(x)=x,G(x)={(3x-2)}^3/9+8/9$, then $F^{-1}(u)=u,G^{-1}(u)=({}^3\sqrt{9u-8}+2)/3,u\in [0,1]$. Thus, $f(F^{-1}(u))=1,g(G^{-1}(u))={(9u-8)}^{2/3}$ and $h_F(u)={(9u-8)}^{2/3}$. It is easy to see that $F(u)\le G(u)$. However, $h_F(u)$ is not nondecreasing in u, $u\in [0,1]$, that means that $F(x){\le }_{c}G(y)$ will not hold in this example. As such, we may conclude that if $F(x){\le }_{c}G(y)$, then $F(x)\le G(y)$. However, if $F(x)\le G(y)$, then $F(x){\le }_{c}G(y)$ may not hold generally.

Theorem 1

(Tsai [4]). If $F(x){\le }_{c}G(y)$, then $I_S(f)\ge I_S(g)$.

With the convex ordering assumption, i.e., $F(x){\le }_{c}G(y)$, the corresponding Lorenz ordering plays an important role for the proof of Theorem 1. To check whether the condition $F(x){\le }_{c}G(y)$ holds or not, one should make simultaneously check whether the condition $F(x)\le G(y)$ holds or not. Similarly, define that $F(x){\le }_{concave}G(x)$ if and only if $F^{-1}G(x)$ is concave function in $x,x\in (-\infty ,\infty )$, then we have the following corollary.

Corollary 1.

If $F(x){\le }_{concave}G(y)$, then $I_S(f)\le I_S(g)$.

Proof. 

It can be easily followed by Theorem 1 via changing the sign of $log{h}_F(u)$. □

Proposition 1.

Composition lemma (Karlin [10]). If K is $T{P}_2$ and L is $T{P}_2$ and ν is a σ-finite measure, then the convolution $M(x,y)={\int }_{Z}K(x,z)L(z,y)d\nu (z)$ is $T{P}_2$.

It can be proved similarly. We also note that if K is $T{P}_2$ and L is $R{R}_2$, then M is $R{R}_2$.

3. The Multivariate Distributions

By convex ordering transformation, we can neatly transform the difference of two Shannon entropy measures into the Lorenz ordering. In this section, we intend to extend this result to the multivariate case. It is natural to do so in the sense of conditional convex ordering for the multivariate case. Let $\mathbf{X}={(X_1,\dots ,X_p)}^{\top }$ and $\mathbf{Y}={(Y_1,\dots ,Y_p)}^{\top }$ be random vectors, and f and g be the density function of $\mathbf{X}$ and $\mathbf{Y}$, respectively. Please note that the ${MTP}_2$ holds if and only if all the pairwise ${TP}_2$ hold (Fact 4.3.2 of Tong [11]), namely to examine the ${MTP}_2$ is the same as to examine all the pairwise ${TP}_2$ for any fixed pair. Hence without loss of generality, we may take $p=2$. Please note that $f(x_1,x_2)=f_1(x_1)f_{2|1}(x_2|x_1)$, thus $f(x_1,x_2)$ is $T{P}_2$ if and only if $f_1(x_1)f_{2|1}(x_2|x_1)$ is $T{P}_2$. By the product property that the product function is $T{P}_2$ if two functions are each $T{P}_2$, respectively. Thus we may conclude that $f(x_1,x_2)$ is $T{P}_2$ if and only if both $f_1(x_1)$ and $f_{2|1}(x_2|x_1)$ are $T{P}_2$. That is to say that $F(x_1,x_2){\le }_{c}G(y_1,y_2)$ iff $F_1(x_1){\le }_c{G}_1(y_1)$ and $F_{2|1}(x_2|x_1){\le }_c{G}_{2|1}(y_2|y_1)$. Continue this process, for the general p case, we have the following.

Proposition 2.

Let $\mathbf{c}={(c_1,\dots ,c_p)}^{\top }\in R^p$, then (i) $P\{Y_1>c_1\}{\le }_{c}P\{X_1>c_1\}$, and (ii) $P\{Y_j>c_j|Y_1=y_1,\dots ,Y_{j-1}=y_{j-1}\}{\le }_{c}P\{X_j>c_j|X_1=x_1,\dots ,X_{j-1}=x_{j-1}\}$ for all $x_1\le y_1,\dots ,x_{j-1}\le y_{j-1}$ and for all $j=2,\dots ,p$ if and only if $F(\mathbf{x}){\le }_{c}G(\mathbf{y})$.

From the results of Proposition 2, we are now in a position to use the convex ordering assumption for which $F_{j|1,\dots ,j-1}(x_j|x_1,\dots ,x_{j-1}){\le }_c{G}_{j|1,\dots ,j-1}(y_j|y_1,\dots ,y_{j-1})$ to get each corresponding Lorenz ordering, $\forall j=1,\dots ,p$, where $j=1$ denotes that $F_1(x_1){\le }_c{G}_1(y_1)$. With some elementary arguments, we then have the following.

Theorem 2.

If $F(\mathbf{x}){\le }_{c}G(\mathbf{y})$, then $I_S(f)\ge I_S(g)$.

Proof. 

For simplicity we may consider the situation when $p=2$ first. Let $g_{12}(y_1,y_2)$ be the joint density function of random variables $Y_1$ and $Y_2$, $g_{2|1}(y_2|y_1)$ be the corresponding conditional density function of $Y_2$ given $Y_1=y_1$ and $G_{2|1}(y_2|y_1)$ be the corresponding distribution function. Let $f_{12}(x_1,x_2),f_{2|1}(x_2|x_1)$ and $F_{2|1}(x_2|x_1)$ be defined similarly for the variables $X_1$ and $X_2$. Also let $U_1=F_1(x),U_2|u_1=F_{2|1}(x_2|x_1)(V_1=G_1(y),V_2|v_1=G_{2|1}(y_2|y_1))$. Please note that

$$\begin{array}{cc}\hfill {\mathrm{I}}_S(f_{12})-{\mathrm{I}}_S(g_{12})& ={\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }\{-log{f}_{12}(x_1,x_2)\}{f}_{12}(x_1,x_2)d{x}_{1}d{x}_2\hfill \\ & -{\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }\{-log{g}_{12}(y_1,y_2)\}{g}_{12}(y_1,y_2)d{y}_{1}d{y}_2\hfill \\ & ={\int }_0^1{\int }_0^1\{-log[f_1(F_1^{-1}(u_1))f_{2|1}(F_{2|1}^{-1}(u_2|u_1))]\}d{u}_{1}d{u}_2\hfill \\ & -{\int }_0^1{\int }_0^1\{-log[g_1(G_1^{-1}(v_1))g_{2|1}(G_{2|1}^{-1}(v_2|v_1))]\}d{v}_{1}d{v}_2\hfill \\ & ={\int }_0^1{\int }_0^1-log[\frac{f_1(F_1^{-1}(u_1))f_{2|1}(F_{2|1}^{-1}(u_2|u_1))}{g_1(G_1^{-1}(u_1))g_{2|1}(G_{2|1}^{-1}(u_2|u_1))}]d{u}_{1}d{u}_2\hfill \\ & ={\int }_0^1-log{h}_{F_1}(u_1)d{u}_1+{\int }_0^1[{\int }_0^1-log{h}_{F_{2|1}}(u_2|u_1)d{u}_2]d{u}_1,\hfill \end{array}$$

(3)

where $h_{F_1}(u_1)=f_1(F_1^{-1}(u_1))/g_1(G_1^{-1}(u_1)),h_{G_1}(u_1)=1$ and $h_{F_{2|1}}(u_2|u_1)=f_{2|1}(F_{2|1}^{-1}(u_2|u_1))/g_{2|1}(G_{2|1}^{-1}(u_2|u_1)),h_{G_{2|1}}(u_2|u_1)=1$. Furthermore, by Proposition 2, we may note that $F(x_1){\le }_{c}G(y_1)$, and thus via the arguments in Section 2, we have that $h_{F_1}(u_1),u_1\in [0,1]$ is the density function. Similarly, we also have that $F_{2|1}(x_2|x_1){\le }_c{G}_{2|1}(y_2|y_1)$, and then $h_{F_{2|1}}(u_2|u_1),u_2\in [0,1]$ is the density function. Thus by the information inequality, the first term of the right hand side of Equation (3) is non-negative and the second term insight bracket is non-negative too, and hence the second term of the right hand side of Equation (3) is non-negative. Thus we may conclude that ${\mathrm{I}}_S(f_{12})\ge {\mathrm{I}}_S(g_{12})$.

Let $U_i|u_1,\dots ,u_{i-1}=F_{i|1,\dots ,i-1}(x_i|x_1,\dots ,x_{i-1})$, $V_i|v_1,\dots ,v_{i-1}=G_{i|1,\dots ,i-1}(y_i|y_1,\dots ,i_{p-1})$ and $h_{F_{i|1,\dots ,i-1}}(u_i|u_1,\dots ,u_{i-1})=f_{i|1,\dots ,i-1}(F_{i|1,\dots ,i-1}^{-1}(u_i|u_1,\dots ,u_{i-1}))/g_{i|1,\dots ,i-1}(G_{i|1,\dots ,i-1}^{-1}(v_i|v_1,\dots ,v_{i-1})),i=2,\dots ,p$. Continuing the processes, finally for general p we have

$$\begin{array}{cc}\hfill {\mathrm{I}}_S(f)-{\mathrm{I}}_S(g)& ={\int }_{-\infty }^{\infty }\{-logf(\mathbf{x})\}f(\mathbf{x})d\mathbf{x}-{\int }_{-\infty }^{\infty }\{-logg(\mathbf{y})\}g(\mathbf{y})d\mathbf{y}\hfill \\ & ={\int }_0^1-log{h}_{F_1}(u_1)d{u}_1+{\int }_0^1[{\int }_0^1-log{h}_{F_{2|1}}(u_2|u_1)d{u}_2]d{u}_1\hfill \\ & +\dots +{\int }_0^1\dots {\int }_0^1[{\int }_0^1-log{h}_{F_{p|1,\dots ,p-1}}(u_p|u_1,\dots ,u_{p-1})d{u}_p]d{u}_{p-1}\dots d{u}_1\hfill \\ & \ge 0.\hfill \end{array}$$

(4)

Hence, the theorem follows. □

For two arbitrary densities $f_g,f_h$, the Kullback-Leibler information discrimination is defined by $K(f_g,f_h)={\int }_{-\infty }^{\infty }{f}_g(x)log\{f_g(x)/f_h(x)\}dx$. It is non-negative for all $f_g,f_h$, and is equal to zero if $f_g=f_h$ almost everywhere. Let $\mathcal{E}[X]$ denote the expectation of random variable X. If $F(\mathbf{x}){\le }_{c}G(\mathbf{y})$, from Equation (4) we have the following representation:

$$\begin{array}{cc}\hfill {\mathrm{I}}_S(f)-{\mathrm{I}}_S(g)& =\mathrm{K}(1,h_{F_1})+{\mathcal{E}}_{U_1}[\mathrm{K}(1,h_{F_{2|1}})]+\dots +{\mathcal{E}}_{U_1,\dots ,U_{p-1}}[\mathrm{K}(1,h_{F_{p|1,\dots ,p-1}})],\hfill \end{array}$$

(5)

where ${\mathcal{E}}_{U_1,\dots ,U_{i-1}}$ denotes the expectation of independent uniform densities of $(U_1,\dots ,U_{i-1})$, $U_i\in [0,1],i=1,\dots ,p-1$.

Corollary 2.

If $F(\mathbf{x}){\le }_{concave}G(\mathbf{y})$, then $I_S(f)\le I_S(g)$.

As we have seen, the proof of Theorem 2 is surprisingly easy. However, to characterize the property of convex ordering for two multivariate distributions is sometimes much more complicated via the conditional approach than to characterize the $MT{P}_2$ property directly. The convex ordering of two multivariate distributions is essentially equivalent to the notion of $MT{P}_2$, so does the concave ordering to the $MR{R}_2$. Karlin and Rinott ([12,13]) had studied several $MT{P}_2$ and $MR{R}_2$ distributions, respectively. Some other often seen models related to Theorem 2 and Corollary 2 are exampled in the following.

Example 1.

Let $\mathbf{y}=\mathbf{A}\mathbf{x}$, where $\mathbf{A}$ is a nonsingular matrix. Then $I_S(g)=log|\mathbf{A}|+I_S(f)$, where $|\mathbf{A}|$ denotes the determinant of matrix $\mathbf{A}$ (Zografos and Nadarajah [3]). Thus we have $I_S(f)\ge I_S(g)$ iff $|\mathbf{A}|\le 1$.

Example 2.

Let ${\mathcal{B}}^p=\{{(x_1,\dots ,x_p)}^{\top }:x_i>0,i=1,\dots ,p,{\displaystyle {\sum }_{i=1}^p}{x}_i\le 1\}$ be the p-dimensional simplex, and let

$$\begin{array}{c}\hfill f(\mathbf{x},a_1,\dots ,a_p;b_1,\dots ,b_p)=c{\displaystyle \prod _{i=1}^p}{\displaystyle x_i^{a_i-1}}{(1-{\displaystyle \sum _{k=1}^i}{x}_k)}^{b_i-1}\end{array}$$

be the generalized Dirichlet density function with parameters, $a_i>0,b_i>0,i=1,\cdots ,p$ and

$$\begin{array}{c}\hfill c={\displaystyle \prod _{i=1}^p}\frac{\mathsf{\Gamma }({\displaystyle {\sum }_{k=i}^p}(a_k+b_k-1)+1)}{\mathsf{\Gamma }(a_i)\mathsf{\Gamma }(b_i+{\displaystyle {\sum }_{k=i}^p}(a_k+b_k-1))}.\end{array}$$

It is easy to note that ${\displaystyle x_i^{a_i-1}}$ is $T{P}_2$ in $(x_i,a_i)$, ${(1-{\displaystyle {\sum }_{k=1}^i}{x}_k)}^{b_i-1}$ is $R{R}_2$ in $(x_i,b_i)$ and ${(1-{\displaystyle {\sum }_{k=1}^i}{x}_k)}^{b_i-1}$ is pairwisely $R{R}_2$ in $(x_i,x_j)$ when $b_i\ge 1$. Then, by Proposition 2, we have (i) ${\displaystyle {\prod }_{i=1}^p}{\displaystyle x_i^{a_i-1}}$ is $MT{P}_2$ and (ii) ${\displaystyle {\prod }_{i=1}^p}{(1-{\displaystyle {\sum }_{k=1}^i}{x}_k)}^{b_i-1}$ is $MR{R}_2$ when $b_i\ge 1,\forall i=1,\dots ,p$. Thus, we may conclude that the generalized Dirichlet density function is $MR{R}_2$ when $b_i\ge 1,\forall i=1,\dots ,p$. In addition, Corollary 2 is applicable.

Example 3.

Gupta and Richards [14] considered the multivariate Liouville distribution which the density function is of the form

$$\begin{array}{c}\hfill f(\mathbf{x},\mathbf{\theta })=c(\mathbf{\theta })g({\displaystyle \sum _{i=1}^p}{x}_i){\displaystyle \prod _{i=1}^p}{\displaystyle x_i^{{\theta }_i}},\end{array}$$

where $g:R^{+}\to R^{+}$ is continuous, and $\mathbf{\theta },\mathbf{x}\in R^{+p}$ and

$$\begin{array}{c}\hfill c^{-1}(\mathbf{\theta })=\frac{{\displaystyle {\prod }_{i=1}^p}\mathsf{\Gamma }({\theta }_i)}{\mathsf{\Gamma }({\displaystyle {\sum }_{i=1}^p}{\theta }_i)}{\displaystyle {\int }_0^{\infty }}{t}^{{\displaystyle {\sum }_{i=1}^p}{\theta }_i-1}g(t)dt.\end{array}$$

Please note that ${\displaystyle {\prod }_{i=1}^p}{\displaystyle x_i^{{\theta }_i}}$ is $MT{P}_2$. By Proposition 2, it is easy to see that $f(\mathbf{x},\mathbf{\theta })$ is $MT{P}_2$ ($MR{R}_2$) if and only if $g(x_1+x_2)$ is $T{P}_2(R{R}_2)$ in $(x_1,x_2)$ on $R^{+2}$. The $MR{R}_2$ property of some special cases of multivariate Liouville distributions, such as Dirichlet distributions and inverted Dirichlet distributions, are studied by Karlin and Rinott [13].

Example 4.

In biostatistics, many studies are under the setup of well-known Cox proportional hazard model (Cox [15]), and its basic model is assumed to be that $\overline{G}(\mathbf{x})={(\overline{F}(\mathbf{x}))}^{\lambda },\lambda >0$, where $\overline{G}(\mathbf{x})=1-G(\mathbf{x})$. Without loss of generality, we may assume $p=2$. Write $\overline{G}(\mathbf{x})={\overline{G}}_1(x_1){\overline{G}}_{2|1}(x_2|x_1)$ and $\overline{F}(\mathbf{x})={\overline{F}}_1(x_1){\overline{F}}_{2|1}(x_2|x_1)$. Let $G_1(x)=u_1,u_1\in [0,1]$, then we have $F_1({\displaystyle G_1^{-1}}(u_1))=1-{(1-u_1)}^{1/\lambda }$, thus we have $d{F}_1({\displaystyle G_1^{-1}}(u_1))/d{u}_1={(1-u_1)}^{(1/\lambda -1)}/\lambda$, which is nondecreasing in $u_1$ if $\lambda \ge 1$. As such, we have ${\displaystyle G_1^{-1}}(u_1){\le }_c{\displaystyle F_1^{-1}}(u_1),u_1\in [0,1]$ when $\lambda \ge 1$. Namely, when $\lambda \ge 1$ we have $F_1(x_1){\le }_c{G}_1(x_1)$. Let $G_{2|1}(x_2|x_1)=u_2$, similar arguments as above, we may have that $F_{2|1}(x_2|x_1){\le }_c{G}_{2|1}(x_2|x_1)$ when $\lambda \ge 1$. Thus, by Proposition 2, we may have that $F(\mathbf{x}){\le }_{c}G(\mathbf{x})$ when $\lambda \ge 1$. This procedure can be directly generalized to the general p case by mathematical induction. As such, for the positive disease gene case $\lambda \le 1$, then by Theorem 2 we may conclude that $I_S(f)\le I_S(g)$. Similarly, for the negative disease gene case $\lambda \ge 1$, then $I_S(f)\ge I_S(g)$.

Example 5.

The density function of eigenvalues for a multivariate beta matrix is of the form

$$\begin{array}{c}\hfill f(\mathbf{x})=c{\displaystyle \prod _{i=1}^p}{\displaystyle x_i^{\frac{1}{2}(n_1-p+1)-1}}{(1-x_i)}^{\frac{1}{2}(n_2-p+1)-1}\prod _{1\le i<j\le p}|x_i-x_j|,\end{array}$$

where $0<x_i<1,i=1,\dots ,p,n_1,n_2>p$, and

$$\begin{array}{c}\hfill c={\displaystyle \prod _{i=1}^p}\frac{\mathsf{\Gamma }(\frac{3}{2})\mathsf{\Gamma }(\frac{1}{2}(n_1+n_2-p+i))}{\mathsf{\Gamma }(1+\frac{i}{2})\mathsf{\Gamma }(\frac{1}{2}(n_1-p+i))\mathsf{\Gamma }(\frac{1}{2}(n_1-p+i))}\end{array}$$

(Dumitriu [16]; Peddada and Richards [17]). Please note that the Vandermonde determinant ${\prod }_{1\le i<j\le p}|x_i-x_j|$ is $MT{P}_2$, for the details see Dykstra and Hewett [18]. Using Proposition 2, it is easy to see that ${\displaystyle {\prod }_{i=1}^p}{\displaystyle x_i^{(n_1-p+1)/2-1}}$ is $T{P}_2$ in ($x_i,n_1-p+1$), and ${\displaystyle {\prod }_{i=1}^p}{(1-x_i)}^{(n_2-p+1)/2-1}$ is $R{R}_2$ in ($x_i,n_2-p+1$). Thus, ${\displaystyle {\prod }_{i=1}^p}{\displaystyle x_i^{(n_1-p+1)/2-1}}{(1-x_i)}^{(n_2-p+1)/2-1}$ is $MR{R}_2$. Then, we may conclude that the density function, which is the product of these functions, is $MR{R}_2$. In addition, Corollary 2 is applicable.

4. The Central Wishart and Central MANOVA Models, and Noncentral Wishart and Noncentral MANOVA Models

Theorem 2 states that if the density function has the $MT{P}_2$ property, then the corresponding ordering of Shannon entropies holds, namely the $MT{P}_2$ (or $MR{R}_2)$ property of density function ensures the ordering of Shannon entropies. The $MT{P}_2$ also implies the power monotonicity property of MANOVA tests, for which based on the monotone function of eigenvalues. Eigenvalues play an important role in statistical inference. Example 5 deals with the density function of eigenvalues of MANOVA models (or Jacobi ensembles) when the population covariance matrix is assumed to be an identity matrix $(\Sigma =\mathbf{I})$. Two other often seen models are (i) Gaussian (or Hermite) ensembles,

$$\begin{array}{c}\hfill f(\mathbf{x})={(2\pi )}^{-\frac{p}{2}}{\displaystyle \prod _{i=1}^p}\frac{\mathsf{\Gamma }(\frac{3}{2})}{\mathsf{\Gamma }(1+\frac{i}{2})}{e}^{-\frac{1}{2}{\displaystyle {\sum }_1^p}{\displaystyle x_i^2}}\prod _{1\le i<j\le p}|x_i-x_j|,\end{array}$$

and (ii) Wishart models (or Laguerre ensembles),

$$\begin{array}{c}\hfill f(\mathbf{x})=2^{-\frac{p(p+1)}{2}}{\displaystyle \prod _{i=1}^p}\frac{\mathsf{\Gamma }(\frac{3}{2})}{\mathsf{\Gamma }(1+\frac{i}{2})\mathsf{\Gamma }(\frac{1}{2}(n_1-p+1))}{\displaystyle \prod _{i=1}^p}{\displaystyle x_i^{\frac{1}{2}(n_1-p+1)-1}}{e}^{-\frac{1}{2}{\displaystyle {\sum }_1^p}{x}_i}\prod _{1\le i<j\le p}|x_i-x_j|\end{array}$$

(Dumitriu [16]). From similar arguments to Example 5, it is easy to see that the density function $f(\mathbf{x})$ is $MT{P}_2$ either for (i) (the identity population covariance matrix is an M-matrix) or for (ii) (gamma type density function), respectively, we omit the details.

For statistical inference, generally the population covariance matrix is unknown. In this section, we further study the $MT{P}_2$ property of the distribution functions of eigenvalues of central Wishart and MANOVA models, and of noncentral Wishart and MANOVA models (James [19]; Muirhead [20]) under the general population covariance matrix set-up.

4.1. Type ${}_0{F}_0$, Exponential: Eigenvalues of a Wishart Matrix

Suppose that the columns of a matrix $p\times n$ matrix $\mathbf{X}$ are independently normally distributed with covariance matrix $\mathbf{\Sigma }$ and $E(\mathbf{X})=\mathbf{0}$. Let $\mathbf{L}=\mathrm{diag}(l_1,\dots ,l_p)$, where $l_i$ be the ith largest eigenvalue of $\mathbf{X}{\mathbf{X}}^{\top }$. Similarly, let $\mathbf{\Delta }=\mathrm{diag}({\delta }_1,\dots ,{\delta }_1)$, where ${\delta }_i$ be the ith largest eigenvalue of $\mathbf{\Sigma }$. Write $\mathbf{\Sigma }=\mathbf{Q}\mathbf{\Delta }{\mathbf{Q}}^{\top }$ and $\mathbf{X}{\mathbf{X}}^{\top }=\mathbf{U}\mathbf{L}{\mathbf{U}}^{{}^{\prime }}$, where $\mathbf{Q},\mathbf{U}\in \mathcal{O}(p)$ with $\mathcal{O}(p)$ being the group of $p\times p$ orthogonal matrices. Also denote $(d\mathbf{U})$ the m-form of $\mathbf{U}$ with $m=\frac{1}{2}p(p+1)$ and $\mathrm{tr}$ denotes the trace operator, then

$$\begin{array}{cc}\hfill {}_0{F}_0(-\frac{1}{2}{\mathbf{\Sigma }}^{-1},\mathbf{X}{\mathbf{X}}^{\top })& ={\int }_{\mathcal{O}(p)}{e}^{-\frac{1}{2}\mathbf{tr}(\mathbf{Q}{\mathbf{\Delta }}^{-1}{\mathbf{Q}}^{\top }\mathbf{U}\mathbf{L}{\mathbf{U}}^{\top })}(d\mathbf{U})\hfill \\ & ={\int }_{\mathcal{O}(p)}{e}^{-\frac{1}{2}\mathbf{tr}({\mathbf{\Delta }}^{-1}\mathbf{H}\mathbf{L}{\mathbf{H}}^{\top })}(d\mathbf{H})\hfill \\ & ={}_0{F}_0(-\frac{1}{2}{\mathbf{\Delta }}^{-1},\mathbf{L}),\hfill \end{array}$$

(6)

where $\mathbf{H}={\mathbf{Q}}^{\top }\mathbf{U}\in \mathcal{O}(p)$ and noting that $(d\mathbf{U})=(d\mathbf{H})$. Due to the invariant property, the density function of $\mathbf{L}$ depends only upon $\mathbf{\Delta }$, which is of the form

$$\begin{array}{c}\hfill {\varphi }_{\delta }(l_1,\dots ,l_p)={|\mathbf{\Delta }|}^{-\frac{1}{2}n}{}_0{F}_0(-\frac{1}{2}{\mathbf{\Delta }}^{-1},\mathbf{L}){\varphi }_0(l_1,\dots ,l_p),\end{array}$$

(7)

where

$$\begin{array}{c}\hfill {\varphi }_0(l_1,\dots ,l_p)=\frac{{\pi }^{\frac{1}{2}{p}^2}}{2^{\frac{1}{2}pn}{\mathsf{\Gamma }}_p(\frac{1}{2}n){\mathsf{\Gamma }}_p(\frac{1}{2}p)}{|\mathbf{L}|}^{\frac{1}{2}(n-p-1)}\prod _{i<j}(l_i-l_j)\end{array}$$

(8)

with ${\mathsf{\Gamma }}_p(a)={\pi }^{\frac{1}{4}p(p-1)}{\displaystyle {\sum }_{i=1}^p}\mathsf{\Gamma }(a-\frac{1}{2}(i-1))$.

Theorem 3.

${}_0{F}_0(-\frac{1}{2}{\mathbf{\Delta }}^{-1},\mathbf{L})$ is $MT{P}_2$.

Proof. 

Please note that

$$\begin{array}{cc}\hfill \frac{\partial \log {}_0{F}_0(-\frac{1}{2}{\mathbf{\Delta }}^{-1},\mathbf{L})}{\partial \mathbf{L}}& =\frac{{\int }_{\mathcal{O}(p)}{e}^{-\frac{1}{2}\mathbf{tr}({\mathbf{\Delta }}^{-1}\mathbf{H}\mathbf{L}{\mathbf{H}}^{\top })}(-\frac{1}{2}{\mathbf{\Delta }}^{-1}\mathbf{H}{\mathbf{H}}^{\top })(d\mathbf{H})}{{\int }_{\mathcal{O}(p)}{e}^{-\frac{1}{2}\mathbf{tr}({\mathbf{\Delta }}^{-1}\mathbf{H}\mathbf{L}{\mathbf{H}}^{\top })}(d\mathbf{H})}\hfill \\ & =\frac{{\int }_{\mathcal{O}(p)}{e}^{-\frac{1}{2}\mathbf{tr}({\mathbf{\Delta }}^{-1}\mathbf{H}\mathbf{L}{\mathbf{H}}^{\top })}(-\frac{1}{2}{\mathbf{\Delta }}^{-1})(d\mathbf{H})}{{\int }_{\mathcal{O}(p)}{e}^{-\frac{1}{2}\mathbf{tr}({\mathbf{\Delta }}^{-1}\mathbf{H}\mathbf{L}{\mathbf{H}}^{\top })}(d\mathbf{H})}\hfill \\ & =-\frac{1}{2}{\mathbf{\Delta }}^{-1}.\hfill \end{array}$$

Thus, it is easy to see that

$$\begin{array}{c}\hfill \frac{{\partial }^2\log {}_0{F}_0(-\frac{1}{2}{\mathbf{\Delta }}^{-1},\mathbf{L})}{\partial {\mathbf{L}}^{{}^{\prime }}\partial \mathbf{L}}=\mathbf{0}\end{array}$$

and

$$\begin{array}{c}\hfill \frac{{\partial }^2\log {}_0{F}_0(-\frac{1}{2}{\mathbf{\Delta }}^{-1},\mathbf{L})}{\partial \mathbf{\Delta }\partial \mathbf{L}}=\frac{1}{2}{\mathbf{\Delta }}^{-2},\end{array}$$

respectively. Please note that $\mathbf{\Delta }$ is a positive diagonal matrix, which is clearly positive definite. Hence the theorem follows. □

By the result of Dykstra and Hewett [18], it is easy to note that ${\varphi }_0(l_1,\dots ,l_p)$ enjoys the $MT{P}_2$ property. Hence, by Theorem 3 we may conclude that the density function ${\varphi }_{\delta }(l_1,\dots ,l_p)$ of eigenvalues of $\mathbf{X}{\mathbf{X}}^{\top }$, which is the product of two $MT{P}_2$ functions (Karlin and Rinott [12]), is $MT{P}_2$.

4.2. Type ${}_1{F}_0$, Binomial Series: Eigenvalues When ${\mathbf{\Sigma }}_1\ne {\mathbf{\Sigma }}_2$

Suppose that the $p\times n_1$ matrix variate $\mathbf{X}$ with $n_1$ independent columns distributed as $N(\mathbf{0},{\mathbf{\Sigma }}_1)$ and $p\times n_2$ matrix variate $\mathbf{Y}$ with $n_2$ independent columns distributed as $N(\mathbf{0},{\mathbf{\Sigma }}_2)$. Also let $\mathbf{F}=\mathrm{diag}(f_1,\dots ,f_p)$ and $\mathsf{\Omega }=\mathrm{diag}({\omega }_1,\dots ,{\omega }_p)$, where $f_i$ is the ith largest eigenvalue of

$$\begin{array}{c}\hfill |\mathbf{X}{\mathbf{X}}^{\top }-f\mathbf{Y}{\mathbf{Y}}^{\top }|=0,\end{array}$$

(9)

and

$$\begin{array}{c}\hfill |{\mathbf{\Sigma }}_1-\omega {\mathbf{\Sigma }}_2|=0.\end{array}$$

(10)

Let

$$\begin{array}{c}\hfill {}_1{F}_0(a;-\mathbf{S},\mathbf{T})={\int }_{\mathcal{O}(p)}{|\mathbf{I}+\mathbf{S}\mathbf{H}\mathbf{T}{\mathbf{H}}^{\top }|}^{-a}(d\mathbf{H}),\end{array}$$

(11)

the equation is well defined for all $\mathbf{S}$ and $\mathbf{T}$ being positive definite. Without loss of generality, we may take both $\mathbf{S}$ and $\mathbf{T}$ are positive diagonal matrices. Then the joint density function of eigenvalues $f_1,\dots ,f_p$ is of the form

$$\begin{array}{c}\hfill {\varphi }_{\omega }(f_1,\cdots ,f_p)={|\mathsf{\Omega }|}^{-\frac{1}{2}{n}_1}{}_1{F}_0(\frac{1}{2}(n_1+n_2);-{\mathsf{\Omega }}^{-1},\mathbf{F}){\varphi }_{*}(f_1,\dots ,f_p),\end{array}$$

(12)

where

$$\begin{array}{c}\hfill {\varphi }_{*}(f_1,\dots ,f_p)=\frac{{\pi }^{\frac{1}{2}{p}^2}{\mathsf{\Gamma }}_p(\frac{1}{2}(n_1+n_2))}{{\mathsf{\Gamma }}_p(\frac{1}{2}{n}_1){\mathsf{\Gamma }}_p(\frac{1}{2}{n}_2){\mathsf{\Gamma }}_p(\frac{1}{2}p)}{|\mathbf{F}|}^{\frac{1}{2}(n_1-p-1)}\prod _{i<j}(f_i-f_j).\end{array}$$

(13)

Theorem 4.

${}_1{F}_0(\frac{1}{2}n;-{\mathsf{\Omega }}^{-1},\mathbf{F})$ is $MT{P}_2$, where $n=n_1+n_2$.

Proof. 

Please note that ${}_1{F}_0(\frac{1}{2}n;-{\mathsf{\Omega }}^{-1},\mathbf{F})={\int }_{\mathcal{O}(p)}|\mathbf{I}+{\mathsf{\Omega }}^{-1}\mathbf{H}\mathbf{F}{\mathbf{H}}^{\top }{|}^{-n/2}(d\mathbf{H})={|\mathsf{\Omega }|}^{n/2}{\int }_{\mathcal{O}(p)}{|\mathsf{\Omega }+\mathbf{H}\mathbf{F}{\mathbf{H}}^{\top }|}^{-n/2}(d\mathbf{H})$, thus

$$\begin{array}{cc}\hfill \frac{\partial \log {}_1{F}_0(\frac{1}{2}n;-{\mathsf{\Omega }}^{-1},\mathbf{F})}{\partial \mathbf{F}}& =\frac{{\int }_{\mathcal{O}(p)}-\frac{1}{2}n{|\mathsf{\Omega }+\mathbf{H}\mathbf{F}{\mathbf{H}}^{\top }|}^{-\frac{1}{2}n}{(\mathsf{\Omega }+\mathbf{H}\mathbf{F}{\mathbf{H}}^{\top })}^{-1}(d\mathbf{H})}{{\int }_{\mathcal{O}(p)}{|\mathsf{\Omega }+\mathbf{H}\mathbf{F}{\mathbf{H}}^{\top }|}^{-\frac{1}{2}n}(d\mathbf{H})}.\hfill \end{array}$$

Write $B(\mathsf{\Omega },\mathbf{F})={\int }_{\mathcal{O}(p)}{|\mathsf{\Omega }+\mathbf{H}\mathbf{F}{\mathbf{H}}^{\top }|}^{-n/2}(d\mathbf{H})>0$, then after some calculations

$$\begin{array}{cc}\hfill \frac{{\partial }^2\log {}_1{F}_0(\frac{1}{2}n;-{\mathsf{\Omega }}^{-1},\mathbf{F})}{\partial \mathsf{\Omega }\partial \mathbf{F}}& =\{\frac{1}{4}{n}^2{\int }_{\mathcal{O}(p)}|\mathsf{\Omega }+\mathbf{H}\mathbf{F}{\mathbf{H}}^{\top }{|}^{-\frac{1}{2}n}{(\mathsf{\Omega }+\mathbf{H}\mathbf{F}{\mathbf{H}}^{\top })}^{-2}(d\mathbf{H})\times B(\mathsf{\Omega },\mathbf{F})\hfill \\ & -\frac{1}{4}{n}^2[{\int }_{\mathcal{O}(p)}|\mathsf{\Omega }+\mathbf{H}\mathbf{F}{\mathbf{H}}^{\top }{{|}^{-\frac{1}{2}n}{(\mathsf{\Omega }+\mathbf{H}\mathbf{F}{\mathbf{H}}^{\top })}^{-1}(d\mathbf{H})]}^2\hfill \\ & +\frac{1}{2}n{\int }_{\mathcal{O}(p)}|\mathsf{\Omega }+\mathbf{H}\mathbf{F}{\mathbf{H}}^{\top }{|}^{-\frac{1}{2}n}{(\mathsf{\Omega }+\mathbf{H}\mathbf{F}{\mathbf{H}}^{\top })}^{-2}(d\mathbf{H})\times B(\mathsf{\Omega },\mathbf{F})\}/B^2(\mathsf{\Omega }.\mathbf{F})\hfill \end{array}$$

By Schwartz inequality, we then have

$$\begin{array}{cc}\hfill \frac{{\partial }^2\log {}_1{F}_0(\frac{1}{2}n;-{\mathsf{\Omega }}^{-1},\mathbf{F})}{\partial \mathsf{\Omega }\partial \mathbf{F}}& \ge \frac{\frac{1}{2}n{\int }_{\mathcal{O}(p)}{|\mathsf{\Omega }+\mathbf{H}\mathbf{F}{\mathbf{H}}^{\top }|}^{-\frac{1}{2}n}{(\mathsf{\Omega }+\mathbf{H}\mathbf{F}{\mathbf{H}}^{\top })}^{-2}(d\mathbf{H})}{B(\mathsf{\Omega },\mathbf{F})}\hfill \\ & =\frac{1}{2}n{\mathcal{E}}_f[{(\mathsf{\Omega }+\mathbf{H}\mathbf{F}{\mathbf{H}}^{\top })}^{-2}|\mathbf{F};\mathsf{\Omega }]\hfill \\ & \ge \mathbf{0}\hfill \end{array}$$

with probability one, where $f(\mathbf{H}|\mathbf{F};\mathsf{\Omega })=|\mathsf{\Omega }+\mathbf{H}\mathbf{F}{\mathbf{H}}^{\top }{|}^{-n/2}/B(\mathsf{\Omega },\mathbf{F})$ can be viewed as the conditional density function of $\mathbf{H}$ given $\mathbf{F}$. Please note that the matrix $\mathbf{H}\mathbf{F}{\mathbf{H}}^{\top }$ is positive definite with probability one, thus the conditional expectation ${\mathcal{E}}_f[{(\mathsf{\Omega }+\mathbf{H}\mathbf{F}{\mathbf{H}}^{\top })}^{-2}|\mathbf{F};\mathsf{\Omega }]$ is positive definite with probability one. Hence, the theorem follows. □

Please note that ${\varphi }_{*}(f_1,\dots ,f_p)$ is $MT{P}_2$. Thus, by Theorem 4 we have that the density function ${\varphi }_{\omega }(f_1,\dots ,f_p)$ is $MT{P}_2$.

4.3. Type ${}_0{F}_1$, Bessel: Noncentral Means with Known Covariance

Suppose that the columns of a matrix $p\times n$ matrix $\mathbf{X}$ are independently normally distributed with covariance matrix $\mathbf{\Sigma }$ and $E(\mathbf{X})=\mathbf{M}$. Let $w_i$ be the eigenvalues of $|\mathbf{X}{\mathbf{X}}^{\top }-w\mathbf{\Sigma }|=0$ and $\mathbf{W}=\mathrm{diag}(w_1,\dots ,w_p)$, then its density function depends only upon $\mathsf{\Omega }=\mathrm{diag}({\omega }_1,\dots ,{\omega }_p)$, where ${\omega }_i$ are the eigenvalues of $|\mathbf{M}{\mathbf{M}}^{\top }-\omega \mathbf{\Sigma }|=0$, and is

$$\begin{array}{cc}\hfill {\psi }_{\omega }(w_1,\dots ,w_p;\mathsf{\Omega })=e^{-\frac{1}{2}\mathbf{tr}\mathsf{\Omega }}{}_0{F}_1(\frac{1}{2}n;& \frac{1}{4}\mathsf{\Omega },\mathbf{W}){\psi }_0(w_1,\dots ,w_p),\hfill \end{array}$$

(14)

where

$$\begin{array}{c}\hfill {\psi }_0(w_1,\cdots ,w_p)=\frac{{\pi }^{\frac{1}{2}{p}^2}}{2^{\frac{1}{2}pn}{\mathsf{\Gamma }}_p(\frac{1}{2}n){\mathsf{\Gamma }}_p(\frac{1}{2}p)}{e}^{-\frac{1}{2}\mathbf{trw}}{|\mathbf{W}|}^{\frac{1}{2}(n-p-1)}\prod _{i<j}(w_i-w_j).\end{array}$$

(15)

Furthermore, from Equation (32) of James [19], we have the inverse Laplace transform

$$\begin{array}{cc}\hfill {}_0{F}_1(\frac{1}{2}n;\mathsf{\Omega },\mathbf{W})& =\frac{2^{\frac{1}{2}p(p-1)}{\mathsf{\Gamma }}_p(\frac{1}{2}n)}{{(2\pi i)}^{\frac{1}{2}p(p+1)}}{\int }_{\mathcal{R}(\mathbf{T})>0}{e}^{tr\mathbf{T}}{|\mathbf{T}|}^{-\frac{1}{2}n}{}_0{F}_0({\mathbf{T}}^{-1}\mathsf{\Omega },\mathbf{W})(d\mathbf{T}).\hfill \end{array}$$

(16)

Without loss of generality, we may assume that $\mathbf{T}=\mathrm{diag}(t_1,\dots ,t_p)$ and also let ${\mathbf{T}}^{-\mathbf{1}}\mathsf{\Omega }={\mathbf{U}}^{-1}$, then the Equation (16) becomes

$$\begin{array}{c}{}_0{F}_1(b;\mathsf{\Omega },\mathbf{W})\hfill \\ =\frac{2^{\frac{1}{2}p(p-1)}{\mathsf{\Gamma }}_p(\frac{1}{2}n)}{{(2\pi i)}^{\frac{1}{2}p(p+1)}}{\int }_{\mathcal{R}(\mathbf{U})>0}{e}^{tr(\mathsf{\Omega }\mathbf{U})}{|\mathsf{\Omega }\mathbf{U}|}^{-\frac{1}{2}n}{}_0{F}_0({\mathbf{U}}^{-1},\mathbf{W}){|\mathsf{\Omega }|}^p(d\mathbf{U})\hfill \\ =\frac{2^{\frac{1}{2}p(p-1)}{\mathsf{\Gamma }}_p(\frac{1}{2}n)}{{(2\pi i)}^{\frac{1}{2}p(p+1)}}{\int }_{\mathcal{R}(\mathbf{U})>0}g(u_1,\dots ,u_p;\mathsf{\Omega }){}_0{F}_0({\mathbf{U}}^{-1},\mathbf{W})(d\mathbf{U}),\hfill \end{array}$$

(17)

where

$$\begin{array}{c}\hfill g(u_1,\dots ,u_p;\mathsf{\Omega })=e^{{\displaystyle {\sum }_{i=1}^p}{\omega }_i{u}_i}{\displaystyle \prod _{i=1}^p}{\displaystyle {\omega }_i^{-\frac{1}{2}n+p}}{\displaystyle \prod _{i=1}^p}{\displaystyle u_i^{-\frac{1}{2}n}}.\end{array}$$

(18)

Theorem 5.

${}_0{F}_1(\frac{1}{2}n;\mathsf{\Omega },\mathbf{W})$ is $MT{P}_2$.

Proof. 

By Equation (18), after some straightforward calculations, we have

$$\begin{array}{c}\hfill \frac{{\partial }^2\log g(u_1,\dots ,u_p;\mathsf{\Omega })}{\partial {\omega }_i\partial u_i}=1>0,\forall i=1,\dots ,p.\end{array}$$

Thus, $g(u_1,\dots ,u_p;\mathsf{\Omega })$ is $T{P}_2$ pairwise in $(u_i,{\delta }_i)$. Namely, $g(u_1,\dots ,u_p;\mathsf{\Omega })$ is $MT{P}_2$. By the composition lemma (Proposition 1) of Karlin [10], thus the theorem follows. □

Please note that ${\psi }_0(w_1,\dots ,w_p)$ is $MT{P}_2$, thus by Theorem 5, we may conclude that the density function ${\psi }_{\omega }(w_1,\dots ,w_p;\mathsf{\Omega })$ is $MT{P}_2$.

4.4. Type ${}_1{F}_1$, Confluent Hypergeometric: Noncentral Eigenvalues

Suppose that the $p\times n_1$ matrix variate $\mathbf{X}$ with $n_1$ independent columns distributed as $N(\mathbf{M},\mathbf{\Sigma })$ and $p\times n_2$ matrix variate $\mathbf{Y}$ with $n_2$ independent columns distributed as $N(\mathbf{0},\mathbf{\Sigma })$ for $n_1\ge p$. Let ${\displaystyle {\omega }_i^{*}}$ be the eigenvalues of $|\mathbf{M}{\mathbf{M}}^{\top }-{\omega }^{*}\mathbf{\Sigma }|=0$. Write ${\mathsf{\Omega }}^{*}=\mathrm{diag}({\displaystyle {\omega }_1^{*}},\dots ,{\displaystyle {\omega }_p^{*}})$, then the joint density function of eigenvalues $f_i$ of $|\mathbf{X}{\mathbf{X}}^{\top }-f\mathbf{Y}{\mathbf{Y}}^{\top }|=0$, (i) for $n_1\ge p$, is of the form

$$\begin{array}{c}{\displaystyle {\varphi }_{{\omega }^{*}}^{*}}(f_1,\dots ,f_p;{\mathsf{\Omega }}^{*})\hfill \\ =e^{-\frac{1}{2}\mathbf{tr}{\mathsf{\Omega }}^{*}}{}_1{F}_1(\frac{1}{2}(n_1+n_2);\frac{1}{2}{n}_2;\frac{1}{2}{\mathsf{\Omega }}^{*},{(\mathbf{I}+{\mathbf{F}}^{-1})}^{-1}){\varphi }^{*}(f_1,\dots ,f_p),\hfill \end{array}$$

(19)

where

$$\begin{array}{c}\hfill {\varphi }^{*}(f_1,\dots ,f_p)=\frac{{\pi }^{\frac{1}{2}{p}^2}{\mathsf{\Gamma }}_p(\frac{1}{2}(n_1+n_2))}{{\mathsf{\Gamma }}_p(\frac{1}{2}{n}_1){\mathsf{\Gamma }}_p(\frac{1}{2}{n}_2){\mathsf{\Gamma }}_p(\frac{1}{2}p)}\frac{{|\mathbf{F}|}^{\frac{1}{2}(n_1-p-1)}}{{|\mathbf{I}+\mathbf{F}|}^{\frac{1}{2}(n_1+n_2)}}\prod _{i<j}(f_i-f_j).\end{array}$$

(20)

Theorem 6.

${}_1{F}_1(\frac{1}{2}n;\frac{1}{2}{n}_2;\frac{1}{2}{\mathsf{\Omega }}^{*},{(\mathbf{I}+{\mathbf{F}}^{-1})}^{-1})$ is $MR{R}_2$, where $n=n_1+n_2$.

Proof. 

Please note that

$$\begin{array}{cc}\hfill {}_1{F}_0(\frac{1}{2}n;{\mathsf{\Omega }}^{*-1},{(\mathbf{I}+{\mathbf{F}}^{-1})}^{-1})& ={\int }_{\mathcal{O}(p)}{|\mathbf{I}-{\mathsf{\Omega }}^{*-1}\mathbf{H}{(\mathbf{I}+{\mathbf{F}}^{-1})}^{-1}{\mathbf{H}}^{\top }|}^{-\frac{1}{2}n}(d\mathbf{H})\hfill \\ & =|{\mathsf{\Omega }}^{*}{|}^{\frac{1}{2}n}{\int }_{\mathcal{O}(p)}{|{\mathsf{\Omega }}^{*}-\mathbf{H}{(\mathbf{I}+{\mathbf{F}}^{-1})}^{-1}{\mathbf{H}}^{\top }|}^{-\frac{1}{2}n}(d\mathbf{H}).\hfill \end{array}$$

Let $\mathbf{G}={(\mathbf{I}+{\mathbf{F}}^{-1})}^{-1}$ and $C({\mathsf{\Omega }}^{*},\mathbf{G})={\int }_{\mathcal{O}(p)}{|{\mathsf{\Omega }}^{*}-\mathbf{H}\mathbf{G}{\mathbf{H}}^{\top }|}^{-n/2}(d\mathbf{H})>0$, then

$$\begin{array}{cc}& \frac{{\partial }^2\log {}_1{F}_0(\frac{1}{2}n;{\mathsf{\Omega }}^{*-1},\mathbf{G})}{\partial {\mathsf{\Omega }}^{*}\partial \mathbf{G}}\hfill \\ & =-\{\frac{1}{4}{n}^2{\int }_{\mathcal{O}(p)}|{\mathsf{\Omega }}^{*}-\mathbf{H}\mathbf{G}{\mathbf{H}}^{\top }{|}^{-\frac{1}{2}n}{({\mathsf{\Omega }}^{*}-\mathbf{H}\mathbf{G}{\mathbf{H}}^{\top })}^{-2}(d\mathbf{H})\times C({\mathsf{\Omega }}^{*},\mathbf{G})\hfill \\ & -\frac{1}{4}{n}^2[{\int }_{\mathcal{O}(p)}|{\mathsf{\Omega }}^{*}-\mathbf{H}\mathbf{G}{\mathbf{H}}^{\top }{{|}^{-\frac{1}{2}n}{({\mathsf{\Omega }}^{*}-\mathbf{H}\mathbf{G}{\mathbf{H}}^{\top })}^{-1}(d\mathbf{H})]}^2\hfill \\ & +\frac{1}{2}n{\int }_{\mathcal{O}(p)}|{\mathsf{\Omega }}^{*}-\mathbf{H}\mathbf{G}{\mathbf{H}}^{\top }{|}^{-\frac{1}{2}n}{({\mathsf{\Omega }}^{*}-\mathbf{H}{\mathbf{H}}^{\top })}^{-2}(d\mathbf{H})\times C({\mathsf{\Omega }}^{*},\mathbf{G})\}/C^2({\mathsf{\Omega }}^{*},\mathbf{G}).\hfill \end{array}$$

By Schwartz inequality, then

$$\begin{array}{cc}& \frac{{\partial }^2\log {}_1{F}_0(\frac{1}{2}n;{\mathsf{\Omega }}^{*-1},\mathbf{G})}{\partial {\mathsf{\Omega }}^{*}\partial \mathbf{G}}\hfill \\ & \le -\frac{1}{2}n{\int }_{\mathcal{O}(p)}{|{\mathsf{\Omega }}^{*}-\mathbf{H}\mathbf{G}{\mathbf{H}}^{\top }|}^{-\frac{1}{2}n}{({\mathsf{\Omega }}^{*}-\mathbf{H}\mathbf{G}{\mathbf{H}}^{\top })}^{-2}(d\mathbf{H})/C({\mathsf{\Omega }}^{*},\mathbf{G})\hfill \\ & =-\frac{1}{2}n{\mathcal{E}}_g[{({\mathsf{\Omega }}^{*}+\mathbf{H}\mathbf{G}{\mathbf{H}}^{\top })}^{-2}|\mathbf{G};{\mathsf{\Omega }}^{*}]\hfill \\ & \le \mathbf{0}\hfill \end{array}$$

with probability one, where $g(\mathbf{H}|\mathbf{G};{\mathsf{\Omega }}^{*})={|\mathsf{\Omega }+\mathbf{H}\mathbf{G}{\mathbf{H}}^{\top }|}^{-n/2}/C({\mathsf{\Omega }}^{*},\mathbf{G})$ can be viewed as the conditional density function of $\mathbf{H}$ given $\mathbf{G}$. Please note that the matrix $\mathbf{H}\mathbf{G}{\mathbf{H}}^{\top }$ is positive definite with probability one, thus the conditional expectation ${\mathcal{E}}_g[{({\mathsf{\Omega }}^{*}+\mathbf{H}\mathbf{G}{\mathbf{H}}^{\top })}^{-2}|\mathbf{G};{\mathsf{\Omega }}^{*}]$ is positive definite with probability one. Then, we have that ${}_1{F}_0(\frac{1}{2}n;{\mathsf{\Omega }}^{*-1},\mathbf{G})$ is $MR{R}_2$ in $({\mathsf{\Omega }}^{*},\mathbf{G})$. Also, it is easy to note that $\mathbf{G}$ is the monotone increasing function of $\mathbf{F}$, thus, ${}_1{F}_0(\frac{1}{2}n;{\mathsf{\Omega }}^{*-1},{(\mathbf{I}+{\mathbf{F}}^{-1})}^{-1})$ is $MR{R}_2$ in $({\mathsf{\Omega }}^{*},\mathbf{F})$.

Next, consider

$$\begin{array}{c}{}_1{F}_1(\frac{1}{2}n,\frac{1}{2}{n}_2;{\mathsf{\Omega }}^{*},{(\mathbf{I}+{\mathbf{F}}^{-1})}^{-1})\hfill \\ =\frac{2^{\frac{1}{2}p(p-1)}{\mathsf{\Gamma }}_p(\frac{1}{2}{n}_2)}{{(2\pi i)}^{\frac{1}{2}p(p+1)}}{\int }_{\mathcal{R}(\mathbf{T})>0}{e}^{tr\mathbf{T}}{|\mathbf{T}|}^{-\frac{1}{2}{n}_2}{}_1{F}_0(\frac{1}{2}n;{\mathbf{T}}^{-1}{\mathsf{\Omega }}^{*},{(\mathbf{I}+{\mathbf{F}}^{-1})}^{-1})(d\mathbf{T}).\hfill \end{array}$$

(21)

Let ${\mathbf{T}}^{-\mathbf{1}}{\mathsf{\Omega }}^{*}={\mathbf{U}}^{-1}$, then

$$\begin{array}{cc}& {}_1{F}_1(\frac{1}{2}n,\frac{1}{2}{n}_2;{\mathsf{\Omega }}^{*},{(\mathbf{I}+{\mathbf{F}}^{-1})}^{-1})\hfill \\ & =\frac{2^{\frac{1}{2}p(p-1)}{\mathsf{\Gamma }}_p(\frac{1}{2}{n}_2)}{{(2\pi i)}^{\frac{1}{2}p(p+1)}}{\int }_{\mathcal{R}(\mathbf{U})>0}{e}^{tr({\mathsf{\Omega }}^{*}\mathbf{U})}|{\mathsf{\Omega }}^{*}{\mathbf{U}|}^{-\frac{1}{2}{n}_2}{}_1{F}_0(\frac{1}{2}n;{\mathbf{U}}^{-1},\mathbf{W}){|{\mathsf{\Omega }}^{*}|}^p(d\mathbf{U})\hfill \\ & =\frac{2^{\frac{1}{2}p(p-1)}{\mathsf{\Gamma }}_p(\frac{1}{2}{n}_2)}{{(2\pi i)}^{p(p+1)/2}}{\int }_{\mathcal{R}(\mathbf{U})>0}{g}^{*}(u_1,\dots ,u_p;{\mathsf{\Omega }}^{*}){}_1{F}_0(\frac{1}{2}n;{\mathbf{U}}^{-1},\mathbf{W})(d\mathbf{U}),\hfill \end{array}$$

where

$$\begin{array}{c}\hfill g^{*}(u_1,\dots ,u_p;{\mathsf{\Omega }}^{*})=e^{{\displaystyle {\sum }_{i=1}^p}{\displaystyle {\omega }_i^{*}}{u}_i}{\displaystyle \prod _{i=1}^p}{\displaystyle {{\omega }^{*}}_i^{-\frac{1}{2}{n}_2+p}}{\displaystyle \prod _{i=1}^p}{\displaystyle u_i^{-\frac{1}{2}{n}_2}}.\end{array}$$

(22)

After some straightforward calculations, we have

$$\begin{array}{c}\hfill \frac{{\partial }^2\log g^{*}(u_1,\dots ,u_p;{\mathsf{\Omega }}^{*})}{\partial {\displaystyle {\omega }_i^{*}}\partial u_i}=1>0,u_i,\forall i=1,\dots ,p.\end{array}$$

Thus, $g^{*}(u_1,\dots ,u_p;{\mathsf{\Omega }}^{*})$ is $T{P}_2$ pairwise in $(u_i,{\displaystyle {\omega }_i^{*}})$. Namely, $g^{*}(u_1,\dots ,u_p;{\mathsf{\Omega }}^{*})$ is $MT{P}_2$. By the composition lemma (Proposition 1) of Karlin [10], then the theorem follows. □

Similar to arguments in Example 5, we may have that ${\varphi }^{*}(f_1,\dots ,f_p)$ is $MR{R}_2$. Thus by Theorem 6, we may conclude that the density function ${\displaystyle {\varphi }_{{\omega }^{*}}^{*}}(f_1,\dots ,f_p;{\mathsf{\Omega }}^{*})$, which is the product of two $MR{R}_2$ functions (Karlin and Rinott [13]), is $MR{R}_2$.

(ii) For $p\ge n_1$ the joint density function of eigenvalues $f_1,\dots ,f_p$ is of the form

$$\begin{array}{c}{\displaystyle {\varphi }_{2{\omega }^{*}}^{*}}(f_1,\dots ,f_p;{\mathsf{\Omega }}^{*})\hfill \\ =e^{-\frac{1}{2}\mathbf{tr}{\mathsf{\Omega }}^{*}}{}_1{F}_1(\frac{1}{2}(n_1+n_2);\frac{1}{2}p;\frac{1}{2}{\mathsf{\Omega }}^{*},{(\mathbf{I}+{\mathbf{F}}^{-1})}^{-1}){\displaystyle {\varphi }_2^{*}}(f_1,\dots ,f_p),\hfill \end{array}$$

(23)

where

$$\begin{array}{c}{\displaystyle {\varphi }_2^{*}}(f_1,\dots ,f_p)\hfill \\ =\frac{{\pi }^{\frac{1}{2}{\displaystyle n_1^2}}{\mathsf{\Gamma }}_{n_1}(\frac{1}{2}(n_1+n_2))}{{\mathsf{\Gamma }}_{n_1}(\frac{1}{2}{n}_1){\mathsf{\Gamma }}_{n_1}(\frac{1}{2}(n_1+n_2-p)){\mathsf{\Gamma }}_{n_1}(\frac{1}{2}p)}\frac{{|\mathbf{F}|}^{\frac{1}{2}(p-n_1-1)}}{{|\mathbf{I}+\mathbf{F}|}^{\frac{1}{2}(n_1+n_2)}}\prod _{i<j}(f_i-f_j).\hfill \end{array}$$

(24)

Similar to arguments in Theorem 6, we may conclude that the density function ${\displaystyle {\varphi }_{2{\omega }^{*}}^{*}}(f_1,\dots ,f_p;{\mathsf{\Omega }}^{*})$ is $MR{R}_2$.

4.5. High-Dimensional Wishart Matrices

Suppose that the columns of a matrix $p\times n$ matrix $\mathbf{X}$ are independently normally distributed with covariance matrix $\mathbf{\Sigma }$ and $E(\mathbf{X})=\mathbf{0}$. Let $\mathbf{L}=\mathrm{diag}(l_1,\dots ,l_p)$, where $l_i$ are the ith largest eigenvalue of $\mathbf{X}{\mathbf{X}}^{\top }$. Let $\mathbf{\Sigma }={\sigma }^2\mathbf{I}$ and $c={lim}_{n\to \infty }p/n,c\in (0,1)$. When dimension p is large, the limiting distribution of sample spectral eigenvalues has the well-known Mar$\check{c}$enko-Pastur distribution $F_{c,{\sigma }^2}$ (M-P law) with index c and scale parameter $\sigma$, which the density function is of the form

$$\begin{array}{c}\hfill f_{c,{\sigma }^2}(x)=\frac{1}{2\pi xc{\sigma }^2}\sqrt{(b{\sigma }^2-x)(x-a{\sigma }^2)},a{\sigma }^2\le x\le b{\sigma }^2,\end{array}$$

where $a={(1-\sqrt{c})}^2$ and $b={(1+\sqrt{c})}^2$ (Mar$\check{c}$enko and Paster [21]). After some algebraic manipulations, we can show that

$$\begin{array}{cc}\hfill \frac{{\partial }^2\log f_{c,{\sigma }^2}(x)}{\partial x\partial {\sigma }^2}& =\frac{1}{4\pi c}\frac{(a+b)x^2-4abx{\sigma }^2+(a+b)ab{\sigma }^4}{{(b{\sigma }^2-x)}^2{(x-a{\sigma }^2)}^2}\hfill \\ & \ge 0.\hfill \end{array}$$

Thus, the Mar$\check{c}$enko-Pastur density function is $T{P}_2$ in $(x,{\sigma }^2)$, and hence Theorem 1 is applicable.

5. Remarks

We adopted the notion of convex ordering of two distributions to prove the ordering of Shannon entropies in Theorem 2. Please note that the notions of convex ordering of two distributions, the monotone likelihood ratio and totally positive of order 2 ($MT{P}_2$) are essentially equivalent. For many density functions such as those discussed in Section 3 and Section 4, it seems easier to characterize the $MT{P}_2$ property of density functions than that of convex ordering of distribution functions. In practice, we suggest use the notion of $MT{P}_2$ for the comparisons of Shannon entropy measures.

The difference of two Shannon entropies is an intrinsic distribution measure as we emphasized in this paper. As a result, the Shannon entropy measures can be ordered when two underline distributions have the relationship of convex ordering (i.e., the $MT{P}_2$) property. The monotonicity of power function, which is discussed basically under the same distributions but with different ordered parameters set up, comes out to be a special case of the results of the comparison of two Shannon entropies which can be even discussed under two totally different distribution functions.

For the problem of hypothesis testing $H_0:F(x){=}_{c}G(x)$ against $H_1:F(x){\le }_{c}G(x)$, test statistic based on the sample version of the difference of two Shannon entropies enjoys the optimal property. The above hypothesis testing problem is equivalent to the hypothesis problem of testing $H_0:H_F(u)=u,u\in [0,1]$ against $H_1:H_F(u)$ is convex on $[0,1]$. Let $U_{1:n}<U_{2:n}<\dots <U_{n:n}$ be the order statistics from $H_F$, then the most powerful test is based on the statistics ${\displaystyle {\sum }_{i=1}^n}log{h}_F(U_{i:n})$, which is the empirical version of the difference of two Shannon entropies. Since $h_F(u)$ is monotone increasing in u, thus the test has the property of power monotonicity.

Noncentral chi-square distributions play an important role for statistical inference. For the univariate case, a noncentral chi-square or gamma density which are typically Poisson mixture of central chi-square or gamma type densities. Beyond the monotonicity property, we may aware of the following: let $f_{\alpha }(x,\beta )$ denote the univariate gamma density function with the shape parameter $\alpha$ and the scale parameter $\beta$, and its distribution function is denoted by $F_{\alpha }(x,\beta )$. For the fixed scale parameter gamma densities, van Zwet [6] showed the result of convex ordering of gamma distributions, i.e., if $0<{\alpha }_1\le {\alpha }_2$ then $F_{{\alpha }_2}(x,\beta ){\le }_c{F}_{{\alpha }_1}(x,\beta )$. Namely, $f_{\alpha }(x,\beta )$ is $T{P}_2$ in $(x,\alpha )$ when $\beta$ is fixed. Thus, by Theorem 1 if $0<{\alpha }_1\le {\alpha }_2$ then ${\mathrm{I}}_S(f_{{\alpha }_1})\le {\mathrm{I}}_S(f_{{\alpha }_2})$ when $\beta$ is fixed. Similarly, when $\alpha$ is fixed $f_{\alpha }(x,\beta )$ is $T{P}_2$ in $(x,\beta )$. The above results tell us that the larger degrees of freedom is, the larger Shannon entropy measure is when both noncentralities are kept the same. For any two test statistics (or estimators) which are noncentral chi-square distributed with the same noncentrality, but with different degrees of freedom, then the Pitman efficiency (based on the Kullback-Leibler divergence) makes no difference of comparisons. However, our new measure of the difference of two Shannon entropies is nonzero, which means that the new measure makes a difference of comparisons. For the MANOVA models, the density functions of eigenvalues which are Bessel of multivariate gamma type density or confluent hypergeometric of multivariate beta type density such as those discussed in Section 4.3 and Section 4.4, respectively, our results can be directly applied to the tests with monotone acceptance regions to enjoy the power monotonicity for those density functions with the $MT{P}_2$ ($MLR$) property.

The conceived mixture model arises also for the finite case. A multi-sample model where for some $G(G>1)$, there are G densities $f_g(\mathbf{x}),g=1,\dots ,G$ and a sample of size $n_g$ is drawn from the density $f_g(\mathbf{x})$, so that the pooled sample relates to the mixture density with $w_g=n_g/{\displaystyle {\sum }_{g=1}^G}{n}_g$. For the finite mixture case, it is hard to find out the explicit form of Shannon entropy of mixture distribution, even under the multinormal set-up. Suppose that there are G groups, the random vector $\mathbf{X}$ has the mixture density function ${\displaystyle f_G^{*}}(\mathbf{x})={\displaystyle {\sum }_{g=1}^G}{w}_g{f}_g(\mathbf{x})$, where ${\displaystyle {\sum }_{g=1}^G}{w}_g=1$. Then the distribution function is ${\displaystyle F_G^{*}}(\mathbf{x})=P\{\mathbf{X}\le \mathbf{x}\}={\displaystyle {\int }_{-\infty }^{\mathbf{x}}}{\displaystyle {\sum }_{g=1}^G}{w}_g{f}_g(\mathbf{y})d\mathbf{y}={\displaystyle {\sum }_{g=1}^G}{w}_g{\displaystyle {\int }_{-\infty }^{\mathbf{x}}}{f}_g(\mathbf{y})d\mathbf{y}={\displaystyle {\sum }_{g=1}^G}{w}_g{F}_g(\mathbf{x})$. The $w_g$ are nonnegative and they add up to 1, so the mixture density represents a convex combination of the component densities. We find that the decomposability of Shannon entropy still holds in a different way for the continuous random vectors.

Theorem 7.

$I_S({\displaystyle f_G^{*}})={\displaystyle {\sum }_{g=1}^G}{w}_g{I}_S(f_g)+{\displaystyle {\sum }_{g=1}^G}{w}_{g}K(f_g,{\displaystyle f_G^{*}})$.

Proof. 

Please note that

$$\begin{array}{cc}\hfill {\mathrm{I}}_S({\displaystyle f_G^{*}})& ={\displaystyle {\int }_{-\infty }^{\infty }}-log{\displaystyle f_G^{*}}(\mathbf{x})d{\displaystyle F_G^{*}}(\mathbf{x})\hfill \\ & =-{\displaystyle {\int }_{-\infty }^{\infty }}log({\displaystyle \sum _{g=1}^G}{w}_g{f}_g(\mathbf{x}))d[{\displaystyle \sum _{i=1}^G}{w}_i{F}_i(\mathbf{x})]\hfill \\ & =-{\displaystyle \sum _{g=1}^G}{w}_g{\displaystyle {\int }_{-\infty }^{\infty }}{f}_g(\mathbf{x})log({\displaystyle \sum _{i=1}^G}{w}_i{f}_i(\mathbf{x}))d\mathbf{x}\hfill \\ & =-{\displaystyle \sum _{g=1}^G}{w}_g{\displaystyle {\int }_{-\infty }^{\infty }}{f}_g(\mathbf{x})log{f}_g(\mathbf{x})d\mathbf{x}+{\displaystyle \sum _{g=1}^G}{w}_g{\displaystyle {\int }_{-\infty }^{\infty }}{f}_g(\mathbf{x})log(\frac{f_g(\mathbf{x})}{{\displaystyle {\sum }_{i=1}^G}{w}_i{f}_i(\mathbf{x})})d\mathbf{x}\hfill \\ & ={\displaystyle \sum _{g=1}^G}{w}_g{\mathrm{I}}_S(f_g)+{\displaystyle \sum _{g=1}^G}{w}_{g}K(f_g,{\displaystyle f_G^{*}}).\hfill \end{array}$$

 □

It follows from Theorem 7 that ${\mathrm{I}}_S({\displaystyle f_G^{*}})\ge {\displaystyle {\sum }_{g=1}^G}{w}_g{\mathrm{I}}_S(f_g)$. We further note that for a mixture model, the Shannon entropy has the decomposability property similar to well-known Fisher’s MANOVA model. Please note that the first term on the right hand side represents an average of the individual entropies (i.e., analogous to the within group sum of squares) while the second term is nonnegative and represents the between group distances.

For a high-dimensional Wishart matrix with an unknown scale times the identity matrix being the population covariance matrix, the property of $T{P}_2$ property (i.e., the $MLR$ property) for the limiting empirical density function of eigenvalues is studied in the Section 4.5. Although the Mar$\check{c}$enko-Pastur equation (Silverstein [22]) provides the link of limiting empirical spectral distribution, F, to the limiting behavior of the population spectral distribution, H, one can expect to retrieve the information of H from F. However, the difficulty lies within the fact that the relationship between H and F is entangled. Whether the result studied in Section 4.5 can be extended to the general population covariance matrix case or not remains to be clarified. Another focus of random matrices in the literature is related to the Gaussian divisible ensembles. The Gaussian divisible ensembles are matrices of the form ${\mathbf{H}}_t=e^{-t/2}{\mathbf{H}}_0+{(1-e^{-t})}^{1/2}{\mathbf{H}}_G$ where $t>0$ is a parameter, ${\mathbf{H}}_0$ is a Wigner matrix and ${\mathbf{H}}_G$ is an independent Gaussian orthogonal ensemble matrix. The eigenvalue distribution of the Gaussian divisible ensembles are the same as that of the solution of a matrix valued Ornstein-Uhlenbeck process ${\mathbf{H}}_t$ for any time $t\ge 0$. The dynamics of the eigenvalues of ${\mathbf{H}}_t$ is given by the system of so-called Dyson’s Brownian motion (Dyson [23]). The treatment of the sample covariance matrix is analogous, but the formulas change slightly (see Erd$\stackrel{ʺ}{o}$s, L. et al. [24], for the details). It is interesting to further study that: under the general model, whether the corresponding limiting spectral density function of eigenvalues, obtained via the convolution of the limiting density function of eigenvalues of initial matrix with the Mar$\check{c}$enko-Pastur density function, is $T{P}_2$ or not. We pose those problems as a project of future study.