Testing Differential Gene Networks under Nonparanormal Graphical Models with False Discovery Rate Control

Abstract

The nonparanormal graphical model has emerged as an important tool for modeling dependency structure between variables because it is flexible to non-Gaussian data while maintaining the good interpretability and computational convenience of Gaussian graphical models. In this paper, we consider the problem of detecting differential substructure between two nonparanormal graphical models with false discovery rate control. We construct a new statistic based on a truncated estimator of the unknown transformation functions, together with a bias-corrected sample covariance. Furthermore, we show that the new test statistic converges to the same distribution as its oracle counterpart does. Both synthetic data and real cancer genomic data are used to illustrate the promise of the new method. Our proposed testing framework is simple and scalable, facilitating its applications to large-scale data. The computational pipeline has been implemented in the R package DNetFinder, which is freely available through the Comprehensive R Archive Network.

1. Background

Inferring the structural change of a network under different conditions is essential in many problems arising in biology, medicine, and other scientific fields. For instance, in genomics, it is often of importance to study the structural change of a genetic pathway between diseased and normal groups. In the field of brain mapping, it is critical to identify the difference in brain connectivity between groups (for example, the brain connectivity network of normal subjects and patients often possess different structures). Most of these applications have relied on the prevailing Gaussian graphical models (GGMs) because of its good interpretability and computational convenience, and there is a rich and growing literature on learning differential networks under GGMs. To name a few, Guo et al. (2015) [1] introduced a joint estimation for multiple GGMs by a group lasso approach, under the assumption that the GGMs being studied are sparse and only differ in a small portion of edges. Danaher et al. (2014) [2] proposed a fused graphical lasso method which is free from the sparsity assumption on condition-specific networks and only requires the sparsity of the differential network. Zhao et al. (2014) [3] constructed a new estimator which directly estimates the differential network defined as $\mathbf{\Delta }={\mathbf{\Sigma }}_X^{-1}-{\mathbf{\Sigma }}_Y^{-1}$, where ${\mathbf{\Sigma }}_X^{-1}$ and ${\mathbf{\Sigma }}_Y^{-1}$ represent the two condition-specific precision matrices and $\mathbf{\Delta }$, ${\mathbf{\Sigma }}_X^{-1}$, ${\mathbf{\Sigma }}_Y^{-1}$ have the same dimension. Liu (2017) [4] presented a new test to simultaneously study structural similarities and differences between multiple high-dimensional GGMs, which adopts the partial correlation coefficients to characterize the potential changes of dependency strength between two variables.

Most of the aforementioned algorithms were based upon penalized likelihood maximization. Although some algorithms were consistent under certain regularity conditions, they failed to control the false discovery rate (FDR) of the substructure detection as it is difficult to choose a tuning parameter to control the FDR at the desired level [1,2,3]. One exception is Liu (2017), who introduced a hierarchical testing framework to adjust for the multiplicity. Liu’s test was constructed to asymptotically control the FDR while keeping satisfactory statistical power. Simulation studies in [4] have shown that this new test exhibits substantial power gains over existing methods such as graphical lasso. One major drawback that limits the application of Liu’s test is the Gaussian assumption, which is often violated in practice especially in genomics. For instance, some digital measurements of gene expression level such as RNA-Seq data often greatly deviate from normality even after log-transformation or other variance-stabilizing transformations. In this paper, we aim to extend Liu’s work to a more flexible semiparametric framework, namely the nonparanormal graphical models (NPNGMs), where the random variables are assumed to follow a multivariate normal distribution after a set of monotonically increasing transformations. We use a novel rank-based multiple testing method to detect the structural difference between multiple networks from non-Gaussian data. The method is computationally efficient and asymptotically controls the FDR at a desired level. To begin with, we give the formal definition of nonparanormal distribution:

Definition 1.

A random vector $\mathit{Y}=(Y_1,Y_2,...,Y_p)$ follows a nonparanormal distribution if there exists a set of univariate and monotonically increasing transformations, $\mathit{f}=(f_1,...,f_p)$, such that:

$$(X_1,...,X_p)\equiv (f_1\left(Y_1\right),...,f_p\left(Y_p\right))\sim N(\mathit{\mu },\mathbf{\Sigma }),$$

where μ and Σ denote the mean and covariance matrix in the multivariate normal distribution, respectively. The distribution of $\mathit{Y}$ depends on three parameters and it can be generally written as $\mathit{Y}\sim NPN(\mathit{\mu },\mathbf{\Sigma },\mathit{f}).$

By Definition 1 and Sklar’s theorem, it is easy to verify that when the transformation functions $f_j^{\prime }s$ are all differentiable, the nonparanormal distribution $NPN(\mathit{\mu },\mathbf{\Sigma },\mathit{f})$ is equivalent to a Gaussian copula [5]. As graphical models, the NPNGMs are much more flexible than GGMs in modeling non- Gaussian data while retaining the interpretability of the latter. Some recent studies have established the estimation and properties of high dimensional nonparanormal graphical models. For example, Liu et al. (2009) [5], who first studied high-dimensional NPNGMs, bridged the estimations of GGMs and NPNGMs by a nonparametric and truncated (Winsorized) estimator of the unknown transformation functions. Xue and Zou (2012) [6] proposed to use an adjusted Spearman’s correlation to estimate the structure of high-dimensional NPNGMs, and they showed that the rank-based estimator achieves the same rate of convergence as its oracle counterpart (i.e., assuming known transformation functions). Despite the advances in single NPNGM estimation, to the best of our knowledge, the inference of differential substructure between multiple NPNGMs has not been studied. In this paper, we tackled this problem by embedding the Winsorized estimator into the testing framework of Liu (2017). Under some regularity conditions, we showed that the new test statistic converges to the same distribution as its oracle counterpart does [4].

We begin with the notations and problem formulation. For a vector $\mathit{a}=(a_1,...,a_p)$, we define its ${\ell }_0$ norm as ${\parallel \mathit{a}\parallel }_{{\ell }_0}={\sum }_{i=1}^{p}I\{a_i\ne 0\}$, its ${\ell }_1$ norm as ${\parallel \mathit{a}\parallel }_{{\ell }_1}={\sum }_{i=1}^p\left|a_i\right|$, its ${\ell }_2$ norm as ${\parallel \mathit{a}\parallel }_{{\ell }_2}=\sqrt{{\sum }_{i=1}^p{a}_i^2}$, and its ${\ell }_{\infty }$ norm as ${\parallel \mathit{a}\parallel }_{{\ell }_{\infty }}={max}_i\left|a_i\right|$. For a matrix $\mathit{A}=\left(a_{ij}\right)\in {\mathbb{R}}^{p\times q}$, we define its ${\ell }_0$ norm as ${\parallel \mathit{A}\parallel }_0={\sum }_{i,j}I\{a_{ij}\ne 0\}$, its ${\ell }_1$ norm as ${\parallel \mathit{A}\parallel }_1={\sum }_{i,j}\left|a_{ij}\right|$, its Frobenius norm as ${\parallel \mathit{A}\parallel }_F=\sqrt{{\sum }_{i,j}{a}_{ij}^2}$ and its ${\ell }_{\infty }$ norm as ${\parallel \mathit{A}\parallel }_{\infty }={max}_{i,j}\left|a_{ij}\right|$. Let ${\mathit{A}}_{i,-j}$ denote the ith row of $\mathit{A}$ with its jth entry being removed and ${\mathit{A}}_{-i,j}$ denote the jth column with its ith entry being removed. We use ${\mathit{A}}_{-i,-j}$ to denote a $(p-1)\times (q-1)$ matrix by removing the ith row and the jth column. For square matrix $\mathit{B}$, we let ${\lambda }_{max}\left(\mathit{B}\right)$ and ${\lambda }_{min}\left(\mathit{B}\right)$ denote the largest and smallest eigenvalues of $\mathit{B}$ respectively. In addition, for a given sequence of random variable $\{X_n,n=1,2,...\}$ and a constant sequence $\{a_n,n=1,2,...\}$, $X_n=o_p\left(a_n\right)$ denotes that $X_n/a_n$ converges to zero in probability as n approaches to infinity and $X_n=O_p\left(a_n\right)$ denotes that $X_n/a_n$ is stochastically bounded. If there are positive constants c and C such that $c\le X_n/a_n\le C$ for all $n\ge 1$, we write $X_n\sim a_n$.

To formulate the problem, we let $k\in \{1,2,...,K\}$ be the index of class, p be the dimension, and $(Y_1^{\left(k\right)},...,Y_{n_k}^{\left(k\right)})$ be a sample of size $n_k$ for class k where $Y_m^{\left(k\right)}={(Y_{m1}^{\left(k\right)},...,Y_{mp}^{\left(k\right)})}^T\in {\mathbb{R}}^p$, $m\in \{1,...,n_k\}$. Under $Y_m^{\left(k\right)}\sim NPN({\mathit{\mu }}^{\left(k\right)},{\mathbf{\Sigma }}^{\left(k\right)},{\mathit{f}}^{\left(k\right)})$, we test the following hypothesis:

$$\begin{array}{cc}\hfill H_{0ij}:& {\rho }_{ij·}^{\left(1\right)}={\rho }_{ij·}^{\left(2\right)}=...={\rho }_{ij·}^{\left(K\right)},\hfill \\ \hfill H_{aij}:& {\rho }_{ij·}^{\left(k\right)}\ne {\rho }_{ij·}^{\left(k^{\prime }\right)},\mathrm{for}\mathrm{some}k,k^{\prime }\in \{1,...,K\},\hfill \end{array}$$

where $1\le i,j\le p$, ${\left\{{\mathbf{\Sigma }}^{\left(k\right)}\right\}}^{-1}={\mathbf{\Omega }}^{\left(k\right)}=\left({\omega }_{ij}^{\left(k\right)}\right)$, and ${\rho }_{ij·}^{\left(k\right)}$ represents the partial correlation coefficient between $X_i^{\left(k\right)}$ and $X_j^{\left(k\right)}$ given ${\mathit{X}}^{\left(k\right)}\setminus (X_i^{\left(k\right)},X_j^{\left(k\right)})$, $(X_{m1}^{\left(k\right)},...,X_{mp}^{\left(k\right)})=(f_1^{\left(k\right)}\left(Y_{m1}^{\left(k\right)}\right),...,f_p^{\left(k\right)}\left(Y_{mp}^{\left(k\right)}\right))$. The edge $(i,j)$ is a differential edge if ${\rho }_{ij·}^{\left(k\right)}\ne {\rho }_{ij·}^{\left(k^{\prime }\right)}$ for some $k,k^{\prime }\in \{1,...,K\}$, and the differential network is defined as the set of all differential edges. As a well-known result in statistics, ${\rho }_{ij·}^{\left(k\right)}=-{\omega }_{ij}^{\left(k\right)}/\sqrt{{\omega }_{ii}^{\left(k\right)}{\omega }_{jj}^{\left(k\right)}}$. Here, we consider an equivalent alternative of the hypothesis testing above. Similar as in [4], let

$$S_{ij}(\mathbf{\Omega })=\sqrt{\sum _{1\le k<k^{\prime }\le K}{({\rho }_{ij·}^{\left(k\right)}-{\rho }_{ij·}^{\left(k^{\prime }\right)})}^2},$$

(1)

then the hypothesis testing can be simplified as

$$\begin{array}{cc}\hfill H_{0ij}:& S_{ij}(\mathbf{\Omega })=0,\hfill \\ \hfill H_{aij}:& S_{ij}(\mathbf{\Omega })>0.\hfill \end{array}$$

As $S_{ij}(\mathbf{\Omega })=S_{ji}(\mathbf{\Omega })$, we define ${\mathcal{H}}_0=\{H_{0ij},1\le i<j\le p\}$ and ${\mathcal{H}}_a=\{H_{aij},1\le i<j\le p\}$, and the total numbers of tests are $p(p-1)/2$, i.e., $card\left({\mathcal{H}}_0\right)=card\left({\mathcal{H}}_a\right)=p(p-1)/2$.

The rest of this paper is structured as follows: In Section 2, we introduce the new test statistic and multiple testing procedure. In Section 3 we perform a simulation study to evaluate the finite sample performance of the proposed test in terms of FDR control and statistical power. We then apply the new method to a rich genomic data to study the genetic difference between four breast cancer subtypes. We discuss the strength and shortcomings of the test in Section 5. Technical proof of the asymptotic results is provided in Appendix A.

2. Statistical Methods

2.1. Winsorized Estimator of the Latent Gaussian Variables

In practice, the transformation functions ${\mathit{f}}^{\left(k\right)}=(f_1^{\left(k\right)},...,f_p^{\left(k\right)})$ in the nonparanormal distribution are unknown. However, one can use a Winsorized estimator to approximate ${\mathit{f}}^{\left(k\right)}$, i.e., to impute the latent Gaussian variables (oracle data) ${(X_{m1}^{\left(k\right)},...,X_{mp}^{\left(k\right)})}_{1\le m\le n_k}$. To illustrate the Winsorized estimator, we define the following quantile function:

$${\widehat{h}}_j^{\left(k\right)}\left(t\right)={\Phi }^{-1}\left({\tilde{F}}_j^{\left(k\right)}\left(t\right)\right),1\le j\le p,$$

where ${\tilde{F}}_j^{\left(k\right)}$ is some estimator of the cumulative distribution function of $Y_j^{\left(k\right)}$, and a natural choice for ${\tilde{F}}_j^{\left(k\right)}$ would be the empirical cumulative distribution function (eCDF)

$${\widehat{F}}_j^{\left(k\right)}\left(t\right)=\frac{1}{n_k}\sum _{m=1}^{n_k}I\{Y_{mj}^{\left(k\right)}\le t\}.$$

One major drawback of the eCDF above is that under high dimensionality, the variance of ${\widehat{F}}_j^{\left(k\right)}\left(t\right)$ could be too large. To overcome the problem, Liu et al. (2009) considered a truncated (Winsorized) estimator as follows:

$${\tilde{F}}_j^{\left(k\right)}=\left\{\begin{array}{cc}{\delta }_n\hfill & {\widehat{F}}_j^{\left(k\right)}\left(t\right)<{\delta }_n\hfill \\ {\widehat{F}}_j^{\left(k\right)}\left(t\right)\hfill & {\delta }_n\le {\widehat{F}}_j^{\left(k\right)}\left(t\right)\le 1-{\delta }_n,\hfill \\ 1-{\delta }_n\hfill & {\widehat{F}}_j^{\left(k\right)}\left(t\right)>1-{\delta }_n\hfill \end{array}\right.$$

where ${\delta }_n$ serves as the truncation parameter that should be carefully chosen. Liu et al. (2009) [5] suggested ${\delta }_n=1/\left(4n^{1/4}\sqrt{\pi logn}\right)$ to balance the bias and variance of eCDF, and so we will use this value in our calculations. To estimate the transformation functions and impute the latent Gaussian variable $\mathit{X}$, we define

$$X_{mj}^{\left(k\right)*}={\tilde{f}}_j^{\left(k\right)}\left(Y_{mj}^{\left(k\right)}\right)={\widehat{\mu }}_j^{\left(k\right)}+{\widehat{\sigma }}_j^{\left(k\right)}{\tilde{h}}_j^{\left(k\right)}\left(Y_{mj}^{\left(k\right)}\right),$$

where ${\tilde{h}}_j^{\left(k\right)}\left(t\right)$, ${\widehat{\mu }}_j^{\left(k\right)}$ and ${\widehat{\sigma }}_j^{\left(k\right)}$ are given below:

$${\tilde{h}}_j^{\left(k\right)}\left(t\right)={\Phi }^{-1}\left({\tilde{F}}_j^{\left(k\right)}\left(t\right)\right),$$

$${\widehat{\mu }}_j^{\left(k\right)}=\frac{1}{n_k}\sum _{m=1}^{n_k}{Y}_{mj}^{\left(k\right)},$$

$${\widehat{\sigma }}_j^{\left(k\right)}=\sqrt{\frac{1}{n_k}\sum _{m=1}^{n_k}{(Y_{mj}^{\left(k\right)}-{\widehat{\mu }}_j^{\left(k\right)})}^2}.$$

The Winsorized estimator $X_{mj}^{\left(k\right)*}$ generally works well in approximating the unknown $X_{mj}^{\left(k\right)}$, and it could be used to estimate the oracle sample covariance. Let ${\widehat{\mathbf{\Sigma }}}^{\left(k\right)}$ be the sample covariance matrix by the oracle data, and ${\tilde{\mathbf{\Sigma }}}^{\left(k\right)}$ be the sample covariance matrix by $(X_1^{\left(k\right)*},...,X_p^{\left(k\right)*})$, that is

$${\tilde{\mathbf{\Sigma }}}^{\left(k\right)}=\frac{1}{n_k}\sum _{m=1}^{n_k}(X_m^{\left(k\right)*}-{\tilde{\mathit{\mu }}}^{\left(k\right)}){(X_m^{\left(k\right)*}-{\tilde{\mathit{\mu }}}^{\left(k\right)})}^T,$$

where ${\tilde{\mathit{\mu }}}^{\left(k\right)}=(1/n_k){\sum }_{m=1}^{n_k}{X}_m^{\left(k\right)*}$. Liu et al. (2009) established the following consistency results under mild regularity conditions:

$${\parallel {\tilde{\mathbf{\Sigma }}}^{\left(k\right)}-{\widehat{\mathbf{\Sigma }}}^{\left(k\right)}\parallel }_{\infty }=O_p\left(\sqrt{\frac{logp{log}^2{n}_k}{n_k^{1/2}}}\right).$$

When estimating the precision matrix ${\mathbf{\Omega }}^{\left(k\right)}$, one can consider a modified graphical lasso based on imputed data, i.e.,

$${\tilde{\mathbf{\Omega }}}_{glasso}^{\left(k\right)}=arg\underset{\mathbf{\Omega }}{min}\left\{tr(\mathbf{\Omega }{\tilde{\mathbf{\Sigma }}}^{\left(k\right)})-log|\mathbf{\Omega }|+{\lambda \parallel \mathbf{\Omega }\parallel }_1\right\}.$$

(2)

Liu et al. (2009) showed the following convergence, which elucidated the asymptotic equivalence between the oracle data and imputed data in the structural estimation of NPNGM

$${\parallel {\tilde{\mathbf{\Omega }}}_{glasso}^{\left(k\right)}-{\mathbf{\Omega }}^{\left(k\right)}\parallel }_F=O_p\left(\sqrt{\frac{(\parallel {\mathbf{\Omega }}^{\left(k\right)}{\parallel }_0+p)logp{log}^2{n}_k}{n_k^{1/2}}}\right).$$

2.2. Asymptotic Results for a Single Class

To extend Liu’s test to a nonparanormal case, we first consider the problem of single GGM estimation based on oracle data, i.e., ${(X_{m1}^{\left(k\right)},...,X_{mp}^{\left(k\right)})}_{1\le m\le n_k}\sim N({\mathit{\mu }}_k,{\mathbf{\Sigma }}_k)$, in the following regression framework

$$X_{mj}^{\left(k\right)}={\alpha }_j^{\left(k\right)}+{\mathit{X}}_{m,-j}^{{\left(k\right)}^{\prime }}{\mathit{\beta }}_j^{\left(k\right)}+{ϵ}_{mj}^{\left(k\right)}.$$

(3)

It is not hard to show that the regression coefficients ${\mathit{\beta }}_j^{\left(k\right)}=({\beta }_{j,1}^{\left(k\right)},...,{\beta }_{j,j-1}^{\left(k\right)},{\beta }_{j,j+1}^{\left(k\right)},{\beta }_{j,p}^{\left(k\right)})$ and the error term ${ϵ}_{mj}^{\left(k\right)}$ satisfy

$${\mathit{\beta }}_j^{\left(k\right)}=-{\left({\omega }_{jj}^{\left(k\right)}\right)}^{-1}{\mathbf{\Omega }}_{-j,j}^{\left(k\right)},\mathrm{cov}({ϵ}_{mi}^{\left(k\right)},{ϵ}_{mj}^{\left(k\right)})=\frac{{\omega }_{ij}^{\left(k\right)}}{{\omega }_{ii}^{\left(k\right)}{\omega }_{jj}^{\left(k\right)}}.$$

As the oracle data ${(X_{m1}^{\left(k\right)},...,X_{mp}^{\left(k\right)})}_{1\le m\le n_k}$ in Equation (3) are generally unknown, we consider a new regression model based on Winsorized imputations:

$$X_{mj}^{\left(k\right)*}={\widehat{\alpha }}_j^{\left(k\right)}+{\mathit{X}}_{m,-j}^{\left(k\right){*}^{\prime }}{\widehat{\mathit{\beta }}}_j^{\left(k\right)}+{ϵ}_{mj}^{\left(k\right)*}.$$

(4)

In solving the problem of single GGM estimation, Liu (2017) proposed an elegant test based on a bias-corrected sample covariance. This has motivated us to construct the following new statistic

$$S_{ij}^{\left(k\right)*}=\sqrt{\frac{1}{n_k{r}_{ii}^{\left(k\right)*}{r}_{jj}^{\left(k\right)*}}}\left(\sum _{m=1}^{n_k}{ϵ}_{mi}^{\left(k\right)*}{ϵ}_{mj}^{\left(k\right)*}+\sum _{m=1}^{n_k}{\left\{{ϵ}_{mi}^{\left(k\right)*}\right\}}^2{\widehat{\beta }}_{i,j}^{\left(k\right)}+\sum _{m=1}^{n_k}{\left\{{ϵ}_{mj}^{\left(k\right)*}\right\}}^2{\widehat{\beta }}_{j,i}^{\left(k\right)}\right),$$

(5)

where $r_{ij}^{\left(k\right)*}=(1/n_k){\sum }_{m=1}^{n_k}{ϵ}_{mi}^{\left(k\right)*}{ϵ}_{mj}^{\left(k\right)*}$. By letting ${\overline{\mathit{ϵ}}}^{\left(k\right)}=(1/n_k){\sum }_{m=1}^{n_k}{\mathit{ϵ}}_m^{\left(k\right)}$, ${\left({\widehat{\sigma }}_{ij,ϵ}^{\left(k\right)}\right)}_{1\le i,j\le p}=(1/n_k)$${\sum }_{m=1}^{n_k}({\mathit{ϵ}}_m^{\left(k\right)}-{\overline{\mathit{ϵ}}}^{\left(k\right)}){({\mathit{ϵ}}_m^{\left(k\right)}-{\overline{\mathit{ϵ}}}^{\left(k\right)})}^{\prime }$, $b_{ij}^{\left(k\right)}={\omega }_{ii}^{\left(k\right)}{\widehat{\sigma }}_{ii,ϵ}^{\left(k\right)}+{\omega }_{jj}^{\left(k\right)}{\widehat{\sigma }}_{jj,ϵ}^{\left(k\right)}-1$, we will prove that, under mild conditions (see a detailed proof in Appendix A)

$$S_{ij}^{\left(k\right)*}+b_{ij}^{\left(k\right)}\frac{{\omega }_{ij}^{\left(k\right)}}{{\omega }_{ii}^{\left(k\right)}{\omega }_{jj}^{\left(k\right)}}\stackrel{D}{\to }N\left(0,1+\frac{{\left\{{\omega }_{ij}^{\left(k\right)}\right\}}^2}{{\omega }_{ii}^{\left(k\right)}{\omega }_{jj}^{\left(k\right)}}\right).$$

(6)

Similar as in [4], the estimated coefficients ${\widehat{\mathit{\beta }}}_j^{\left(k\right)}$ must satisfy the following conditions:

$${\parallel {\widehat{\mathit{\beta }}}_j^{\left(k\right)}-{\mathit{\beta }}_j^{\left(k\right)}\parallel }_{{\ell }_1}=O_p\left(a_n^{\left(k\right)}\right),$$

$$min\left\{{\lambda }_{max}^{1/2}\left({\mathbf{\Sigma }}^{\left(k\right)}\right){\parallel {\widehat{\mathit{\beta }}}_j^{\left(k\right)}-{\mathit{\beta }}_j^{\left(k\right)}\parallel }_{{\ell }_2},\underset{1\le j\le p}{max}\sqrt{{({\widehat{\mathit{\beta }}}_j^{\left(k\right)}-{\mathit{\beta }}_j^{\left(k\right)})}^T{\widehat{\mathbf{\Sigma }}}_{-j,-j}^{\left(k\right)}({\widehat{\mathit{\beta }}}_j^{\left(k\right)}-{\mathit{\beta }}_j^{\left(k\right)})}\right\}=O_p\left(b_n^{\left(k\right)}\right),$$

where

$$a_n^{\left(k\right)}=o\left(\sqrt{logp/n_k}\right),\mathrm{and}{b}_n^{\left(k\right)}=o\left(n_k^{-1/4}\right).$$

(7)

Equation (6) is our main result, which is essentially a counterpart of Proposition 3.1 in [4]. The detailed proof is given in Appendix A. The asymptotic result we obtained here suggested that, by an appropriate choice of regression coefficients ${\widehat{\mathit{\beta }}}_j^{\left(k\right)}$, Liu’s test can be readily extended to a nonparanormal framework by Winsorized imputation. Under GGMs, the condition (7) can be satisfied by several popular shrinkage estimators including lasso estimator and Dantzig selector. For the choice of ${\mathit{\beta }}_j^{\left(k\right)}$ under NPNGMs, one can use the rank-based method introduced by Xue and Zou (2012) [6]. Xue and Zou (2012) showed that the rank-based estimator (e.g., rank-based lasso and rank-based Dantzig selector) achieved exactly the same convergence rate as its oracle counterpart, therefore, it also satisfies our condition (7).

2.3. Multiple Testing Procedure for FDR Control

Now we introduce the multiple testing procedure for FDR control based on the single-class result from Equation (6). As suggested in [4], the partial correlation coefficient can be well estimated by a thresholding estimator

$${\widehat{\rho }}_{ij.}^{\left(k\right)}=S_{ij}^{\left(k\right)}I\left\{\left|S_{ij}^{\left(k\right)}\right|\ge 2\sqrt{\frac{logp}{n_k}}\right\},$$

and we define the following two-sample test statistics

$$S_{ij}^{(k,k^{\prime })}=\frac{S_{ij}^{\left(k\right)}-S_{ij}^{\left(k^{\prime }\right)}}{\sqrt{\frac{1}{n_k}{(1-{\left\{{\widehat{\rho }}_{ij.}^{\left(k\right)}\right\}}^2)}^2+\frac{1}{n_{k^{\prime }}}{(1-{\left\{{\widehat{\rho }}_{ij.}^{\left(k^{\prime }\right)}\right\}}^2)}^2}}.$$

In the multi-sample case ${\mathit{S}}_{ij}={\left(S_{ij}^{(k,k^{\prime })}\right)}_{1\le k<k^{\prime }\le K}$, we consider a sum squared test statistics

$${\mathbb{S}}_{ij}=\sqrt{\sum _{k<k^{\prime }}{\left\{S_{ij}^{(k,k^{\prime })}\right\}}^2}.$$

Motivated by [4] (Equations 2.6 and 2.7) and [7], we define the following statistic

$$T_{ij}={\Phi }^{-1}P\left(\sqrt{{\sum }_{i=1}^M{\lambda }_i{Z}_i^2}\le {\mathbb{S}}_{ij}\right),$$

and constant $A=(P_0-{\widehat{P}}_0)/Q_0$, where $Z_i,i=1,...,M$ represent a sequence of M i.i.d. standard normal random variables, $P_0=2\Phi \left(1\right)-1$, ${\widehat{P}}_0=2{\sum }_{1\le i<j\le p}I\left\{\right|T_{ij}|\le 1\}/(p^2-p)$, $Q_0=\sqrt{2}\varphi \left(1\right)$ and $A\left(t\right)=(1+|A|\frac{\left|t\right|\varphi \left(t\right)}{\sqrt{2}(1-\Phi \left(t\right))}{)}^{-1}$. For a given $0<{\alpha }_0<1$, let

$$t\left({\alpha }_0\right)=inf\left\{t\in \mathbb{R},1-\varphi \left(t\right)\le \frac{{\alpha }_{0}A\left(t\right)max\{1,{\sum }_{1\le i<j\le p}I\{T_{ij}\ge t\}\}}{(p^2-p)/2}\right\}.$$

the FDR can be controlled at level $\alpha$, if we reject $H_{0ij}:S_{ij}(\mathbf{\Omega })=0$ when $T_{ij}\ge t\left({\alpha }_0\right)$. One may refer to [7] for the detailed proof about this testing procedure.

Our proposed computational pipeline consisted of three steps: (1) Winsorized imputation for the latent Gaussian variables; (2) rank-based estimation of regression coefficients, and (3) multiple testing with FDR control. On the whole, we put forward a simple procedure to estimate the structural difference between multiple nonparanormal graphical models. The computational pipeline for a two-sample comparison has been implemented in the R package DNetFinder, which can be downloaded from the Comprehensive R Archive Network (CRAN).

3. Numerical Study

We performed a simulation study to evaluate the finite sample performance of the proposed procedure. In particular, we evaluated the empirical false discovery rate (eFDR) as well as the statistical power under two classes, i.e., $K=2$. The dimension and sample size were set to be $p=200$ and $n_1=n_2=100$. We consider two commonly used graph-generating models including the band graph and Erdos–Rényi (ER) graph, and two estimators for regression coefficients including lasso estimator and Dantzig selector. Detailed set-up for precision matrices ${\mathbf{\Omega }}_1$ and ${\mathbf{\Omega }}_2$ are given below:

• Band graph:${\mathbf{\Omega }}_1={\left({\omega }_{ij}\right)}_{1\le i,j\le p}$ was obtained by the following assignments

  $${\omega }_{ij}=\left\{\begin{array}{cc}1\hfill & |i-j|=0\hfill \\ 0.6\hfill & |i-j|=1\hfill \\ 0\hfill & |i-j|\ge 2\hfill \end{array}\right..$$
  We then randomly picked 50 edges in ${\mathbf{\Omega }}_1$ as the differential edges and changed their signs in ${\mathbf{\Omega }}_2$. To ensure positive definiteness, we added $max\left(\right|{\lambda }_{min}\left({\mathbf{\Omega }}_1\right)|,|{\lambda }_{min}\left({\mathbf{\Omega }}_2\right)\left|\right)+0.05$, to the diagonal of ${\mathbf{\Omega }}_1$ and ${\mathbf{\Omega }}_2$.
• Erdos–Rényi (ER) graph: Each node pair ($i,j$) were randomly connected with probability 5%. A correlation coefficient is generated for each edge in the network from a two-part uniform distribution $[-1/2,-1/4]\cup [1/4,1/2]$. To ensure positive-definiteness, we shrunk the correlations by a factor of 5 and the diagonals were set to be one for ${\mathbf{\Omega }}_1$. We then randomly selected 5% of the edges as the differential edges, and changed their signs in ${\mathbf{\Omega }}_2$.

For each graph, we generated the latent Gaussian data (oracle data) from $N(\mathbf{0},{\mathbf{\Omega }}^{-1})$, $\mathbf{\Omega }\in \{{\mathbf{\Omega }}_1,{\mathbf{\Omega }}_2\}$, and a Winsorized estimator with truncation parameter ${\delta }_n=1/\left(4n^{1/4}\sqrt{\pi logn}\right)$ was used to implement our test. The performance of the proposed method was then evaluated in two aspects: false discovery rate control and statistical power. In particular, we compared the results based on oracle data and imputed data by the Winsorized estimator. Two estimators including the lasso estimator and Dantzig selector were used to estimate coefficients $\widehat{\mathit{\beta }}$. For oracle data, we applied the R package flare to calculate the solution path over a sequence of 20 candidate $\lambda$’s and tune by Akaike information criterion (AIC). For imputed data, we adopted the rank-based methods introduced by [6], i.e., the rank-based lasso and rank-based Dantzig selector. The simulation was repeated for 100 times for each FDR level ($\alpha \in \{0.05,0.10,0.15,...,0.50\}$) and the average empirical FDR and statistical power were summarized.

Figure 1 and Figure 2 compared the empirical false discovery rate (eFDR) with the desired level $\alpha$ under the band graph and ER graph. It can be seen that the empirical FDR based on imputed data is close to the one by oracle data, both close to the desired level of $\alpha$, suggesting that the FDRs were controlled quite well for both cases. The lasso estimator works almost equally well as Dantzig selector in both settings.

Figure 3 and Figure 4 summarized the statistical power of the test for the band graph and ER graph. As can be seen, the power for ER graph is substantially lower than the band graph, indicating that the complexity and denseness of the underlying differential network may significantly decrease the power of our test. The test based on oracle data performs slightly better than the imputed data, which is due to the loss of information during Winsorized imputation. Similar as we observed from Figure 1 and Figure 2, the lasso estimator works almost equally well as Dantzig selector.

In addition, we compared the proposed test with a direct estimator, recently developed by Zhang (2019) [8]. The direct estimator is a rank-based estimator and can be solved by a parametric simplex algorithm. We simulated the data from the Erdos–Rényi (ER) graph with different sample sizes ($n=25,50,100,150$) and numbers of dimensions ($p=40,60,90,120$). As the direct estimator does not control the false discovery rate, we set the FDR level at 0.05 for our proposed test. Figure 5 summarized the empirical FDR and statistical power under different sample sizes (with dimension fixed at 100) and different dimensions (with sample size fixed at 100). It can be seen that the two methods have comparable performance and our proposed test achieves lower FDR but slightly lower statistical power. However, it is noteworthy that the direct estimator is computationally expensive and becomes impractical when the dimensions exceed 150.

Table 1. Running time of the proposed test and direct estimator (in seconds).

n/p	40	60	90	120
25	0.88 (7.0)	1.59 (110)	3.79 (1936)	6.49 (23,066)
50	1.19 (7.7)	1.93 (127)	4.15 (1973)	6.83 (23,119)
100	1.87 (9.1)	2.61 (146)	5.00 (2016)	7.80 (23,153)
200	2.11 (11)	3.04 (165)	6.22 (2055)	9.61 (23,201)

summarized the running time of the two methods, where it can be seen that our test is much faster than the direct estimator, especially for relatively high dimensions. For instance, when $p=120$, the direct estimator takes hours while our test takes less than 10 seconds. As the core part of the proposed algorithm is the estimation of regression coefficients, the time complexity is the same as the linear regression. For instance, with LASSO and $p>n$, the time complexity is $O\left(n{p}^2\right)$, while the direct estimator by Zhang (2019) has a time complexity $O\left(n{p}^4\right)$.

4. A Genomic Application

In this part, we applied the proposed test to the Cancer Genome Atlas data (TCGA, [9]) to study the different roles of the cell cycle pathway in the two subtypes of breast cancer including luminal A subtype and basal-like subtype. The cell cycle pathway is known to play a critical role in the initiation and progression of many human cancers including breast cancer and ovarian cancer [10,11]. For instance, the cell cycle pathway provided by KEGG (Kyoto Encyclopedia of Genes and Genomes, [12]) contains 128 important genes that co-regulate cell proliferation, including ATM, RB1, CCNE1, and MYC. Abnormal regulation among these genes may cause the over-proliferation of cells and an accumulation of tumor cell numbers [11].

The transcriptome profiling data for breast cancer were downloaded through the Genomic Data Commons portal [13] in January 2017. The expression level of each gene was quantified by the count of reads mapped to the gene. The quantifications were done by software HTSeq of version 0.9.1 [14]. In our analysis, we excluded 43 subjects including 12 male subjects and 31 subjects with >1% missing values. In addition, we removed the effects due to different age groups and batches using a median- matching and variance-matching strategy [10,15,16]. For example, the batch effect can be removed in the following way:

$$g_{ijk}^{*}=M_i+(g_{ijk}-M_{ij})\frac{{\widehat{\sigma }}_{g_i}}{{\widehat{\sigma }}_{g_{ij}}},$$

where $g_{ijk}$ refers to the expression value for gene i from sample k in batch j ($j=1,2,...,J;k=1,2,...,n_j$), $M_{ij}$ represents the median of $g_{ij}=(g_{ij1},...,g_{ij{n}_j})$, $M_i$ refers to the median of $g_i=(g_{i1},...,g_{iJ})$, ${\widehat{\sigma }}_{g_i}$ and ${\widehat{\sigma }}_{g_{ij}}$ stand for the standard deviations of $g_i$ and $g_{ij}$, respectively.

The remaining 959 breast cancer samples were further classified into five subtypes according to two molecular signatures, namely PAM50 [17] and SCMOD2 [18]. The two classifications were implemented separately using R package genefu [19] and we obtained 530 subjects with concordant classification by two classifiers. The resulting set contains 221 subjects in the luminal A group, 119 in the luminal B group, 74 in the her2-enriched group, 105 in the basal-like group, and 11 in the normal-like group. For illustration purposes, we conducted two pairwise comparisons (1) Luminal A vs basal-like and (2) Luminal B vs basal-like.

To balance the bias and variance, we choose the same truncation parameter in Winsorized imputation as in our simulation study

$${\delta }_n^{\left(k\right)}=\frac{1}{4n_k^{1/4}\sqrt{\pi log{n}_k}},$$

where $k\in \{1,2\}$, $n_1=221$, $n_2=105$. The proposed test based on the Winsorized estimator was then conducted for each gene pair with different FDR cutoffs. Figure 6 and Figure 7 summarized all the identified differential edges under FDR levels $\alpha =0.05,0.10,0.15,0.20$, with all isolated genes being removed. Our results suggested a list of important genes that play different roles in different breast cancer subtypes. For instance, in Figure 6, genes CCNB1 and PRKDC contribute to several differential edges. According to recent studies, gene CCNB1 is a prognostic biomarker for certain subtypes of breast cancer and it is closely associated with hormone therapy resistance [20]. It has also been reported in the literature that the PRKDC regulates chemosensitivity and is a potential prognostic and predictive marker of response to adjuvant chemotherapy in breast cancer patients [21]. Our findings about several other genes including CHEK2 and CDC7 also confirmed some existing reports [22,23]. As we observed from the two examples, as the desired FDR level increases, the resulting differential network tends to be denser and denser (Figure 8 showed the correlation between FDR and the number of differential edges). In practice, users should consider the trade-off between the accuracy (FDR) and number of new hypotheses (number of differential edges) and choose an appropriate FDR [24].

5. Discussion

Detecting the differential substructure on multiple graphical models is a fundamental and challenging problem in statistics. Liu (2017) studied the problem under the Gaussian framework and introduced an elegant hierarchical test based on the estimation of single GGM. Unlike most existing methods, Liu’s approach asymptotically controlled the false discovery rate at a nominal level, which guarantees the quality of the estimated differential network. In this work, we further extended Liu’s test to a more flexible semiparametric framework, namely the nonparanormal graphical models. Our test is built upon a Winsorized estimator of the unknown transformation functions and it enjoys similar asymptotic properties as its oracle counterpart does.

Although the new test holds great promise in many applications such as genetic network modeling, it has some practical limitations. First, as we see from the theoretical derivation, the good performance of the test relied on the sparsity assumption on the differential network. Although the sparsity assumption is reasonable in many cases, it still could be violated in some applications. For instance, some genetic pathways may exhibit a global change of gene–gene regulations between different phenotypes. When the differential network is dense or locally dense, the method may fail to control the FDR. To solve the problem, a new test needs to be defined to evaluate the level of the sparseness of the change between two conditions. However, there is still a gap on the literature of this topic.

Second, one key assumption in NPNGMs is that the transformed variables follow a joint Gaussian distribution. This assumption also needs to be checked in real-world applications. Under low dimensions, one can employ some popular normality tests, including the Anderson–Darling test and Shapiro–Wilk test, on the imputed data or other normal scores. However, most of these tests fail to detect non-normality for high-dimension data. The normality test under high dimension is still an open and challenging problem and we left it for future research.

It is also noteworthy to mention that the new test relied on an accurate estimator for the coefficients $\mathit{\beta }$. Motivated by [6], we chose two popular estimators including lasso estimator and Dantzig selector based on the adjusted Spearman’s rank, which satisfies Condition (7). In fact, some other estimators also satisfy the conditions, for instance, the rank-based adaptive lasso [6,25] and square-root lasso estimator [6,26]. These estimators can also be incorporated into our testing framework.

6. Conclusions

We have introduced a novel statistical test to detect the structural difference between the two nonparanormal graphical models. The proposed test dropped the Gaussian assumption and can be potentially applied to many non-Gaussian data for differential network analysis. For instance, some digital gene expression data (e.g., RNA-seq data) do not follow Gaussian distribution even after log transformation or other variance-stabilizing transformations. In such cases, one can model the data with a nonparanormal graphical model and apply our test to find differential edges between two or multiple phenotypic conditions. The proposed test may also be used to detect the difference between normal and disease populations in the brain connectivity network.