Improved Test for High-Dimensional Mean Vectors and Covariance Matrices Using Random Projection

Abstract

This paper proposes an improved random projection-based method for testing high-dimensional two-sample mean vectors and covariance matrices. For mean testing, the proposed approach incorporates training data to guide the construction of projection matrices toward the estimated mean difference, thereby substantially enhancing the power of the projected Hotelling’s $T^2$ statistic. We introduce three aggregation strategies—maximum, average, and percentile-based—to ensure stable performance across multiple projections. For covariance testing, the method employs data-driven projections aligned with the leading eigenvector of the sample covariance matrix to amplify the differences between matrices. Aggregation strategies—maximum-, average-, and percentile-based for the mean problem and minimum and average p-values for the covariance problem—are developed to further stabilize performance across repeated projections. An application to gene expression data is provided to illustrate the method. Extensive simulation studies show that the proposed method performs favorably compared to a recent state-of-the-art technique, particularly in detecting sparse signals, while maintaining control of the Type-I error rate.

1. Introduction

We consider two-sample hypothesis testing problems involving high-dimensional mean vectors and covariance matrices. In high-dimensional settings—where the number of variables p exceeds the sample size n—traditional methods such as Hotelling’s $T^2$ test become invalid due to the singularity of the sample covariance matrix. Such scenarios are common in modern applications, ranging from genomics to finance, where high-dimensional data are collected with relatively small sample sizes. These challenges have motivated the development of new statistical tools tailored to the high-dimensional regime. A notable contribution is the random projection method introduced by Lopes et al. [1], which has gained considerable attention due to its simplicity, flexibility, and effectiveness for dimension reduction.

Let ${\mathit{X}}_1,\dots ,{\mathit{X}}_{n_1}$ be independent and identically distributed (i.i.d.) random vectors from a p-dimensional normal distribution ${\mathrm{N}}_p({\mathit{\mu }}_1,\Sigma )$, and let ${\mathit{Y}}_1,\dots ,{\mathit{Y}}_{n_2}$ be i.i.d. from ${\mathrm{N}}_p({\mathit{\mu }}_2,\Sigma )$. We are interested in testing the hypothesis:

$$H_0:{\mathit{\mu }}_1={\mathit{\mu }}_2.$$

(1)

The likelihood ratio test (LRT) statistic for testing the null hypothesis in (1) is given by

$$LRT={\displaystyle \frac{n_1{n}_2}{n_1+n_2}}{(\overline{\mathit{X}}-\overline{\mathit{Y}})}^{\top }{\mathit{S}}^{-1}(\overline{\mathit{X}}-\overline{\mathit{Y}}),$$

(2)

where

• $\overline{\mathit{X}}={\displaystyle \frac{1}{n_1}}{\sum }_{i=1}^{n_1}{\mathit{X}}_i$ and $\overline{\mathit{Y}}={\displaystyle \frac{1}{n_2}}{\sum }_{i=1}^{n_2}{\mathit{Y}}_i$ are the sample mean vectors,
• ${\mathit{S}}_X={\displaystyle \frac{1}{n_1-1}}{\sum }_{i=1}^{n_1}({\mathit{X}}_i-\overline{\mathit{X}}){({\mathit{X}}_i-\overline{\mathit{X}})}^{\top }$,
• ${\mathit{S}}_Y={\displaystyle \frac{1}{n_2-1}}{\sum }_{i=1}^{n_2}({\mathit{Y}}_i-\overline{\mathit{Y}}){({\mathit{Y}}_i-\overline{\mathit{Y}})}^{\top }$, and 
• $\mathit{S}$ is the pooled sample covariance matrix:

  $$\mathit{S}={\displaystyle \frac{(n_1-1){\mathit{S}}_X+(n_2-1){\mathit{S}}_Y}{n_1+n_2-2}}.$$

It is well known that the LRT performs poorly in high-dimensional settings, particularly when the dimension p increases, while the sample size n remains fixed or grows slowly. Moreover, the test statistic becomes undefined when $p>n$ due to the singularity of the sample covariance matrix. These limitations have led to the development of alternative testing procedures, several of which are reviewed below.

Many studies address the issue of singularity by modifying the sample covariance matrix. Bai and Saranadasa [2] showed that the performance of the classical Hotelling’s $T^2$ test deteriorates when the dimension p approaches the sample size n. They proposed a test based on the squared Euclidean norm $\parallel \overline{X}-\overline{Y}{\parallel }^2$, which avoids inverting the sample covariance matrix. Their test statistic was shown to be asymptotically normal under mild conditions, including the existence of finite fourth moments and a high-dimensional asymptotic regime where $p/n\to y>0$.

Chen and Qin [3] extended the work of Bai and Saranadasa [2] by developing a more general framework. Notably, they established the asymptotic normality of their test statistic without requiring a specific relationship between p and n; in particular, the ratio $p/n$ may diverge to infinity.

A projection-based method was proposed by Lopes et al. [1], in which high-dimensional data are projected onto a lower-dimensional subspace of dimension $k<n$. This dimensionality reduction enables the use of the classical Hotelling’s $T^2$ test in the reduced space. Their test statistic takes the following form:

$$\begin{array}{c}\hfill T_k^2={\displaystyle \frac{n_1{n}_2}{n_1+n_2}}{\left[{\mathit{R}}_k^{\top }(\overline{\mathit{X}}-\overline{\mathit{Y}})\right]}^{\top }{\left({\mathit{R}}_k^{\top }\mathit{S}{\mathit{R}}_k\right)}^{-1}\left[{\mathit{R}}_k^{\top }(\overline{\mathit{X}}-\overline{\mathit{Y}})\right],\end{array}$$

(3)

where ${\mathit{R}}_k$ is a projection matrix that maps the original p-dimensional data to a k-dimensional subspace.  Lopes et al. [1] derived the asymptotic power function of their test and recommended choosing $k=\lfloor n/2\rfloor$, where $\lfloor a\rfloor$ denotes the integer part of a. Building on this framework, Srivastava et al. [4] proposed an exact F-test version of the projection method, which can be implemented via Monte Carlo simulation. They also analyzed the asymptotic power of their test in the regime where both p and n tend to infinity. The projection method offers several advantages: it is conceptually simple, computationally efficient, and imposes no restrictions on the relationship between p and n, making it suitable for high-dimensional applications.

An alternative approach based on random subspaces was proposed by Thulin [5]. Instead of testing the equality of the full mean vectors, this method repeatedly selects $k<n_1+n_2-2$ components at random from the sample mean vectors and tests the equality of the corresponding sub-vectors. This procedure is repeated multiple times, and multiple testing corrections, such as the Bonferroni adjustment, are applied to control the overall Type-I error rate. A notable feature of this method is that the test statistic is invariant under linear transformations of the marginal distributions. Permutation tests are used to approximate the null distribution.

Instead of relying on the sample covariance matrix, Chen et al. [6] proposed a one-sample test based on improved estimators of $tr(\Sigma )$ and $tr\left({\Sigma }^2\right)$, under the assumption that $tr\left({\Sigma }^4\right)=o\left({tr}^2\left({\Sigma }^2\right)\right)$.

In the two-sample covariance testing context, let ${\mathit{X}}_1,\dots ,{\mathit{X}}_{n_1}$ be i.i.d. from ${\mathrm{N}}_p(\mathbf{0},{\Sigma }_1)$ and ${\mathit{Y}}_1,\dots ,{\mathit{Y}}_{n_2}$ be i.i.d. from ${\mathrm{N}}_p(\mathbf{0},{\Sigma }_2)$. We test:

$$H_0^2:{\Sigma }_1={\Sigma }_2.$$

(4)

The LRT statistic is given by

$$T_2=-2log\left({\displaystyle \frac{|{\mathit{S}}_1{|}^{n_1/2}·{\left|{\mathit{S}}_2\right|}^{n_2/2}}{{\left|w_1{\mathit{S}}_1+w_2{\mathit{S}}_2\right|}^{(n_1+n_2)/2}}}\right),$$

(5)

where ${\mathit{S}}_1$ and ${\mathit{S}}_2$ are the sample covariance matrices of ${\mathit{X}}_i$ and ${\mathit{Y}}_i$, respectively, and $w_j=n_j/(n_1+n_2)$ for $j=1,2$.

This statistic also suffers from singularity issues when $p>n_j$, rendering the test unstable in high-dimensional scenarios.

In summary, significant progress has been made in the field of testing high-dimensional covariance matrices. Broadly, three main approaches have emerged. The first involves the use of random matrix theory to study the limiting distributions of the extreme eigenvalues of the sample covariance matrix, as developed in Bai [7] and Bai and Yin [8]. The second focuses on constructing consistent and improved estimators of the population covariance matrix; notable contributions include the works of Bickel and Levina [9,10] and Li and Chen [11]. The third approach involves regularized covariance estimation, where the estimator is obtained by solving a maximum likelihood problem under a constraint on the condition number, as proposed in Won et al. [12].

In parallel, several works have explored the use of random projections for testing two-sample means in high-dimensional settings; see, for example, Lopes et al. [1], Srivastava et al. [4].

In this study, we extend the projection-based method of Lopes et al. [1] by investigating random projections with nonzero means while guided by training data. The approach leverages the training data to identify and emphasize directions that are most discriminative. As a result, it enhances the power of the projected Hotelling’s $T^2$ test by combining the strengths of both non-random (data-driven) and random projections.

This method is appealing for three reasons: it is conceptually simple, straightforward to implement, and computationally efficient. Moreover, it does not require any specific relationship between the dimensionality p and sample size n unless otherwise specified.

The remainder of the paper is organized as follows. Section 2 presents the proposed methodology. Section 3 provides extensive simulation studies comparing the proposed tests with recent approaches. A real-world application to Acute Lymphoblastic Leukemia (ALL) gene expression data is presented in Section 4. Section 5 concludes with a summary and discussion.

2. Random Projection

2.1. A Brief Review

Random projection is an emerging and powerful technique for handling high-dimensional data. It reduces the dimensionality of the data from p to k, where $k<min(n,p)$, while preserving the validity and applicability of conventional statistical procedures. This approach enables efficient computation and facilitates hypothesis testing in settings where classical methods may break down due to the high dimensionality. Several researchers have successfully applied random projection techniques to hypothesis testing problems, particularly for moderate to large values of k.

Given a random projection matrix $\mathit{R}\in {\mathbb{R}}^{p\times k}$ that is independent of the data, observations ${\mathit{X}}_i$ and ${\mathit{Y}}_i$ are projected onto a lower-dimensional space via

$$\begin{array}{c}\hfill {\mathit{X}}_i^R={\mathit{R}}^{\top }{\mathit{X}}_i,i=1,\dots ,n_1,\mathrm{and}{\mathit{Y}}_i^R={\mathit{R}}^{\top }{\mathit{Y}}_i,i=1,\dots ,n_2,\end{array}$$

where ${\mathit{X}}_i^R$ and ${\mathit{Y}}_i^R\in {\mathbb{R}}^k$ are the projected observations. Conditional on $\mathit{R}$, ${\mathit{X}}_i^R$ and ${\mathit{Y}}_i^R$ follow ${\mathrm{N}}_k({\mathit{R}}^{\top }{\mathit{\mu }}_1,{\mathit{R}}^{\top }\Sigma \mathit{R})$ and ${\mathrm{N}}_k({\mathit{R}}^{\top }{\mathit{\mu }}_2,{\mathit{R}}^{\top }\Sigma \mathit{R})$, respectively. For notational convenience, we suppress the subscript when referring to univariate normal distributions.

Before developing test statistics based on the projected data, we briefly discuss the construction of the projection matrix $\mathit{R}=\left(r_{ij}\right)$. In theory, the entries $r_{ij}$ can be drawn from any distribution with mean zero. A common and theoretically well-justified choice is to take $\mathit{R}$ with i.i.d. standard normal entries and then orthonormalize the columns so that ${\mathit{R}}^{\top }\mathit{R}={\mathit{I}}_k$. This ensures that the projected data preserves important geometric and statistical properties. For computational efficiency, Srivastava et al. [4] proposed a method based on “one permutation + one random projection”, inspired by the ideas of “very sparse random projections” in  Li et al. [13] and “one permutation hashing” in Li et al. [14].

In the context of two-sample testing, Srivastava et al. [4] introduced RAPTT (RAndom Projection T-Test), an exact testing procedure for high-dimensional mean vectors under multivariate normality. In their approach, the Hotelling’s $T^2$ statistic is computed on the projected data:

$$\begin{array}{c}\hfill T_{\mathit{R}}^2={\left({\displaystyle \frac{1}{n_1}}+{\displaystyle \frac{1}{n_2}}\right)}^{-1}{(\overline{\mathit{X}}-\overline{\mathit{Y}})}^{\top }{\mathit{R}}^{\top }{\left(\mathit{R}\mathit{S}\mathit{R}\right)}^{-1}{\mathit{R}}^{\top }(\overline{\mathit{X}}-\overline{\mathit{Y}}),\end{array}$$

(6)

where $\overline{\mathit{X}}$ and $\overline{\mathit{Y}}$ are the sample means of the two groups, and $\mathit{S}$ is the pooled sample covariance matrix. To increase statistical power, RAPTT aggregates the results over multiple random projections. Let $p_i$ denote the p-value obtained from the projected Hotelling test using projection matrix ${\mathit{R}}_i$ for $i=1,\dots ,m$. The null hypothesis is rejected if

$${\displaystyle \frac{1}{m}}\sum _{i=1}^m{p}_i<{\mathit{\mu }}_{\alpha },$$

where ${\mathit{\mu }}_{\alpha }$ is a threshold chosen such that

$$P\left({\displaystyle \frac{1}{m}}\sum _{i=1}^m{p}_i<{\mathit{\mu }}_{\alpha }|H_0\right)=\alpha .$$

In a related line of work, Wu and Li [15] extended the random projection methodology to high-dimensional covariance matrix testing. Their results demonstrate that even using one-dimensional projection vectors can be sufficient and effective for certain classes of structured covariance matrices, offering a computationally attractive alternative to full-dimensional methods.

2.2. An Improved Test for Equality of Two Mean Vectors

The random projection approach reduces high-dimensional data to a lower-dimensional subspace. However, this transformation may introduce distortion, potentially affecting the accuracy of the test statistic. To mitigate this issue and enhance the power of the test, we propose using a random projection matrix whose columns are generated with a nonzero mean aligned with the estimated difference between the group means.

Let ${\mathit{X}}_1,\dots ,{\mathit{X}}_{n_1}$ follow a p-dimensional normal distribution ${\mathrm{N}}_p({\mathit{\mu }}_1,\Sigma )$, and let ${\mathit{Y}}_1,\dots ,{\mathit{Y}}_{n_2}$ follow a p-dimensional normal distribution ${\mathrm{N}}_p({\mathit{\mu }}_2,\Sigma )$. Suppose that there are reference samples ${\left\{{\mathit{X}}_i^0\right\}}_{i=1}^{m_1}$ and ${\left\{{\mathit{Y}}_i^0\right\}}_{i=1}^{m_2}$, referred to as training samples. Let ${\mathit{X}}_1^0,\dots ,{\mathit{X}}_{m_1}^0\sim {\mathrm{N}}_p({\mathit{\mu }}_1,\Sigma )$ and ${\mathit{Y}}_1^0,\dots ,{\mathit{Y}}_{m_2}^0\sim {\mathrm{N}}_p({\mathit{\mu }}_2,\Sigma )$. Let ${\overline{\mathit{X}}}^0$, ${\overline{\mathit{Y}}}^0$, ${\mathit{S}}_X^0$, and ${\mathit{S}}_Y^0$ denote the sample mean vectors and sample covariance matrices computed from the training samples ${\left\{{\mathit{X}}_i^0\right\}}_{i=1}^{m_1}$ and ${\left\{{\mathit{Y}}_i^0\right\}}_{i=1}^{m_2}$, respectively.

To test $H_0$ in (1), we consider the projected statistic

$$\begin{array}{c}\hfill T_{\mathit{P}}^2={(1/n_1+1/n_2)}^{-1}{(\overline{\mathit{X}}-\overline{\mathit{Y}})}^{\top }\mathit{P}{\left({\mathit{P}}^{\top }{\mathit{S}}_p\mathit{P}\right)}^{-1}{\mathit{P}}^{\top }(\overline{\mathit{X}}-\overline{\mathit{Y}}),\end{array}$$

(7)

where ${\mathit{S}}_p={\displaystyle \frac{(n_1-1){\mathit{S}}_X+(n_2-1){\mathit{S}}_Y+(m_1-1){\mathit{S}}_X^0+(m_2-1){\mathit{S}}_Y^0}{n_1+n_2+m_1+m_2-4}}$ is the pooled sample covariance matrix computed from both training and test samples, and the columns of the $p\times k$ projection matrix $\mathit{P}$ are generated independently from ${\mathrm{N}}_p({\theta }^0,\mathit{I})$, where ${\theta }^0={\overline{\mathit{X}}}^0-{\overline{\mathit{Y}}}^0$. This construction enables $\mathit{P}$ to amplify the mean difference, thereby increasing the power of the test.

Lemma 1.

Given the projection matrix $\mathit{P}$, the statistic $T_{\mathit{P}}^2$ follows a Hotelling’s $T^2$ distribution.

Proof. 

Note that ${\left\{{\mathit{X}}_i\right\}}_{i=1}^{n_1}$, ${\left\{{\mathit{Y}}_i\right\}}_{i=1}^{n_2}$, ${\left\{{\mathit{X}}_i^0\right\}}_{i=1}^{m_1}$, and ${\left\{{\mathit{Y}}_i^0\right\}}_{i=1}^{m_2}$ are independent samples. Under normality, $\overline{\mathit{X}}$, $\overline{\mathit{Y}}$, ${\overline{\mathit{X}}}^0$, and ${\overline{\mathit{Y}}}^0$ are independent of ${\mathit{S}}_X$, ${\mathit{S}}_Y$, ${\mathit{S}}_X^0$, and ${\mathit{S}}_Y^0$. Thus, the projection matrix $\mathit{P}$, which depends only on ${\overline{\mathit{X}}}^0-{\overline{\mathit{Y}}}^0$, is independent of ${\mathit{S}}_p$. Moreover, the pooled sample covariance matrix ${\mathit{S}}_p$ follows a Wishart distribution with $n_1+n_2+m_1+m_2-4$ degrees of freedom. Hence, by Theorem 5.8 in Härdle and Simar [16], given the projection matrix $\mathit{P}$, $T_{\mathit{P}}^2$ follows a Hotelling’s $T^2$ distribution with degrees of freedom k and $n_1+n_2+m_1+m_2-4$.    □

Theorem 1.

Let $c_{\alpha }$ be chosen such that $F_{k,n_1+n_2+m_1+m_2-3-k}\left(c_{\alpha }\right)=1-\alpha$, where $F_{k,n_1+n_2+m_1+m_2-3-k}$ is the F-distribution function with degrees of freedom k and $n_1+n_2+m_1+m_2-3-k$. Then,

$$\begin{array}{c}\hfill P\left({\displaystyle \frac{n_1+n_2+m_1+m_2-3-k}{k(n_1+n_2+m_1+m_2-4)}}{T}_{\mathit{P}}^2\ge c_{\alpha }|H_0\right)=\alpha .\end{array}$$

(8)

Proof. 

$$\begin{array}{cc}\hfill & P\left({\displaystyle \frac{n_1+n_2+m_1+m_2-3-k}{k(n_1+n_2+m_1+m_2-4)}}{T}_{\mathit{P}}^2\ge c_{\alpha }|H_0\right)\hfill \\ \hfill & =E\left[P\left({\displaystyle \frac{n_1+n_2+m_1+m_2-3-k}{k(n_1+n_2+m_1+m_2-4)}}{T}_{\mathit{P}}^2\ge c_{\alpha }\left|\right|H_0,\mathit{P}\right)\right]\hfill \\ \hfill & =E\left[\alpha \right]=\alpha .\hfill \end{array}$$

The second equality follows from Lemma 1.    □

As discussed in Wu and Li [15], a single random-projection Hotelling test may suffer from low power, as different projections can yield contradictory results. To address this, a common remedy is to use multiple random projections. Specifically, let ${\mathit{P}}_1,{\mathit{P}}_2,\dots ,{\mathit{P}}_m$ be m independent projection matrices, and let $T_{{\mathit{P}}_j}^2$ denote the statistic in (7) for projection ${\mathit{P}}_j$. We consider the following three aggregate statistics:

(1)

Maximum: $T_1={max}_{1\le j\le m}{T}_{{\mathit{P}}_j}^2$;

(2)

Average: $T_2={\displaystyle \frac{1}{m}}{\sum }_{j=1}^m{T}_{{\mathit{P}}_j}^2$;

(3)

100p-th Percentile: $T_3=T_{{\mathit{P}}_{(\lceil mp\rceil )}}^2$, where $T_{{\mathit{P}}_{\left(1\right)}}^2\le T_{{\mathit{P}}_{\left(2\right)}}^2\le \dots \le T_{{\mathit{P}}_{\left(m\right)}}^2$ and $\lceil x\rceil$ denotes the smallest integer not less than x.

The null hypothesis $H_0:{\mathit{\mu }}_1={\mathit{\mu }}_2$ is rejected if $T_i\ge c_i\left(\alpha \right)$, where $c_i\left(\alpha \right)$ is chosen such that $P\left(T_i\ge c_i\left(\alpha \right)|H_0\right)=\alpha$. These aggregate statistics help stabilize the performance of the test by leveraging information across multiple projections, thus improving robustness and power. For the numerical results, we select $p=0.95$.

2.3. An Improved Test for Equality of Two Covariance Matrices

In the two-sample covariance test, let ${\mathit{X}}_1,\dots ,{\mathit{X}}_{n_1}$ be i.i.d. observations from a p-dimensional normal distribution ${\mathrm{N}}_p(\mathbf{0},{\Sigma }_1)$, and let ${\mathit{Y}}_1,\dots ,{\mathit{Y}}_{n_2}$ be i.i.d. observations from ${\mathrm{N}}_p(\mathbf{0},{\Sigma }_2)$. In this section, we develop an improved test adopting the idea in Wu and Li [15] using one-dimensional random projections.

We project the two samples $\left\{{\mathit{X}}_i\right\}$ and $\left\{{\mathit{Y}}_i\right\}$ using one-dimensional $p\times 1$ random vectors ${\mathit{R}}_j,j=1,\dots ,m$. Given ${\mathit{R}}_j$, the projected data ${\mathit{X}}_i^j={\mathit{R}}_j^{\top }{\mathit{X}}_i$ and ${\mathit{Y}}_i^j={\mathit{R}}_j^{\top }{\mathit{Y}}_i$ follow univariate normal distributions with variances ${\sigma }_1={\mathit{R}}_j^{\top }{\Sigma }_1{\mathit{R}}_j$ and ${\sigma }_2={\mathit{R}}_j^{\top }{\Sigma }_2{\mathit{R}}_j$, respectively. The testing problem in (4) reduces to test if ${\sigma }_1-{\sigma }_2=0$.

Suppose reference (or training) samples are available to estimate $\nu$ (if not, part of the data can be used as the training samples). Let $s_i^j={\mathit{R}}_j^{\top }{\mathit{S}}_i{\mathit{R}}_j$, $i=1,2$ and $F_j=s_2^j/s_1^j,j=1,\dots ,m$. Regardless of ${\mathit{R}}_j$, it is always true that ${\sigma }_1-{\sigma }_2=0$ under $H_0^2$. When $H_0^2$ is not true, we then want to maximize the difference between ${\sigma }_1$ and ${\sigma }_2$ for a better power. As a result, we generate each random vector ${\mathit{R}}_j$ from a p-variate normal distribution ${\mathrm{N}}_p(\nu ,\mathit{I})$, where $\nu$ is the eigenvector associated with the largest eigenvalue for ${\mathit{S}}_2$ to amplify the difference between ${\Sigma }_{\mathbf{1}}$ and ${\Sigma }_{\mathbf{2}}$.

Theorem 2.

Let $f_{\alpha }$ be chosen such that $F_{n_2-1,n_1-1}\left(f_{\alpha }\right)=1-\alpha$, where $F_{n_2-1,n_1-1}$ is the F-distribution function with degrees of freedom $n_2-1$ and $n_1-1$. Then, for $j=1,2,\dots ,m,$

$$\begin{array}{c}\hfill P\left(F_j\ge f_{\alpha }|H_0^2\right)=\alpha .\end{array}$$

(9)

Proof. 

The proof follows the argument similar to those used in the proof of Theorem 1 and is therefore omitted.    □

Let $p_j$ be the p-value of the statistic $F_j$ using ${\mathit{R}}_j$. Rather than using $max{F}_j$, as suggested in Wu and Li [15], we consider two test statistics given by

(1)

Min_p: $W_1={min}_{1\le j\le m}\left\{p_j\right\}$;

(2)

Ave_p: $W_2={\displaystyle \frac{1}{m}}{\sum }_{j=1}^m{p}_j$.

The null hypothesis $H_0^2$ is rejected if $W_i\ge w_i\left(\alpha \right)$, where $w_i\left(\alpha \right)$ is chosen such that $P\left(W_i\ge w_i\left(\alpha \right)|H_0^2\right)=\alpha$, $i=1,2$.

3. Numerical Results

3.1. Comparing Two Mean Vectors

3.1.1. Setup and Simulation Design

To evaluate the performance of the proposed projection-based testing procedures, we conduct a series of simulation studies under high-dimensional settings. In all simulations, we set the sample sizes $n_1=n_2=50$ and the dimension $p=200$. Without loss of generality, we let ${\mathit{\mu }}_1=\mathbf{0}$.

We consider two types of covariance structures for the underlying distributions:

• Independent Structure: ${\Sigma }_1={\mathit{I}}_p$, where ${\mathit{I}}_p$ is the $p\times p$ identity matrix.
• Toeplitz Structure: ${\Sigma }_2$ is a symmetric Toeplitz matrix defined by the autocorrelation sequence $(1,0.4,0,\dots ,0)$, so that

  $${\Sigma }_2=\left(\begin{array}{ccccc}1& 0.4& 0& \dots & 0\\ 0.4& 1& 0.4& \ddots & ⋮\\ 0& 0.4& \ddots & \ddots & 0\\ ⋮& \ddots & \ddots & 1& 0.4\\ 0& \dots & 0& 0.4& 1\end{array}\right).$$

Under the null hypothesis, both mean vectors are set to zero, i.e., ${\mathit{\mu }}_1={\mathit{\mu }}_2=\mathbf{0}$. To investigate the power under alternatives, we consider two scenarios for ${\mathit{\mu }}_2$:

• Dense alternative: 75% of the components of ${\mathit{\mu }}_2$ are nonzero.
• Sparse alternative: 1% of the components of ${\mathit{\mu }}_2$ are nonzero.

The nonzero components are sampled from $\mathrm{N}(1,1)$ and rescaled to ensure

$${\displaystyle \frac{1}{2}}{({\mathit{\mu }}_1-{\mathit{\mu }}_2)}^{\top }{\Sigma }^{-1}({\mathit{\mu }}_1-{\mathit{\mu }}_2)=1.$$

When no reference data are available, we divide the samples into training and testing sets. Let $m_1=m_2$ denote the training sample sizes per group. The projection matrix $\mathit{P}$ is constructed from columns independently drawn from ${\mathrm{N}}_p({\theta }_0,{\mathit{I}}_p)$, where ${\theta }_0={\overline{\mathit{X}}}_0-{\overline{\mathit{Y}}}_0$ is the sample mean difference from the training data. Critical values are estimated from the combined simulated null distributions based on 1000 simulation runs for each covariance matrix.

3.1.2. Simulation Results

We evaluate three aggregation strategies over m random projections:

(1)

Maximum (Max): $T_1={max}_{1\le j\le m}{T}_{P_j}^2$;

(2)

Average (Ave): $T_2={\displaystyle \frac{1}{m}}{\sum }_{j=1}^m{T}_{P_j}^2$;

(3)

95th Percentile (95): $T_3=T_{P_{(\lceil 0.95m\rceil )}}^2$.

Type-I Error Control

Empirical sizes under the null hypothesis are presented in Figure 1, Figure 2, Figure 3, Figure 4, Figure 5 and Figure 6. Across all configurations—varying $m_1$, k, and $m=10$ or 100—the empirical sizes are generally controlled at the nominal level $\alpha =0.05$, with minor fluctuations attributable to sampling variability.

Power Comparisons

We now present the empirical power of the three proposed tests—Max, Ave, and 95th percentile (95)—in various scenarios. The simulations evaluate the performance of these tests across a range of projection dimensions k, training sample sizes $m_1$, number of projections m, and both dense and sparse alternatives under two different covariance structures (${\Sigma }_1=\mathit{I}$ and a Toeplitz matrix ${\Sigma }_2$).

Case 1: Dense Alternative with ${\Sigma }_1=\mathit{I}$ and $m=10$

Figure 7, Figure 8 and Figure 9 show power curves for training sizes $m_1=1,\dots ,30$ and projection dimensions $k=1,\dots ,47$. Using training data significantly improves power for small k. For example, when $k=2$, the power increases more than twofold compared to RAPTT (horizontal red dotted line) without training. The Ave statistic generally outperforms Max and 95, especially for moderate k.

Case 2: Sparse Alternative with ${\Sigma }_1=\mathit{I}$ and $m=10$

Figure 10, Figure 11 and Figure 12 show that, under sparse alternatives, the Max and 95 statistics are more powerful for small k, while Ave dominates for larger k. Training samples consistently enhance power, particularly in low dimensions. These results highlight that in sparse settings, we can replace the maximum by the 95th percentile to detect sparse signals in lower-dimensional projections (small k), while maintaining satisfactory performance for moderate to large k.

Case 3: Dense Alternative with ${\Sigma }_2$ (Toeplitz) and $m=100$

Figure 13, Figure 14 and Figure 15 provide power curves for the more complex covariance structure ${\Sigma }_2$. Similar trends are observed: Ave performs best overall, while Max is slightly superior for small k and small $m_1$.

Case 4: Sparse Alternative with ${\Sigma }_2$ and $m=100$

Figure 16, Figure 17 and Figure 18 present results under the sparse alternative with ${\Sigma }_2$. As expected, the Max test excels at small k and small training sample sizes. It consistently outperforms both Ave and 95 when projection dimensionality is kept low due to its ability to capture strong, isolated differences.

As k increases, the performance of Max deteriorates, and Ave regains superiority. The 95 test continues to provide a good compromise between power and robustness, especially for intermediate values of k.

Although the overall power of the tests under sparse alternatives is lower compared to dense ones, the use of training samples still yields meaningful power gains at small k. The gap between RAPTT and our methods is narrower in this case, but the benefit of informed projections remains visible.

Finally, to demonstrate the consistency of the proposed test, the mean vector ${\mathit{\mu }}_2$ is generated from $\mathrm{N}(1,1)$ and then rescaled so that

$${\displaystyle \frac{1}{2}}{({\mathit{\mu }}_1-{\mathit{\mu }}_2)}^{\top }{\Sigma }^{-1}({\mathit{\mu }}_1-{\mathit{\mu }}_2)=c.$$

To evaluate performance across a range of signal strengths c, we consider various combinations of projection dimensions k and training sample sizes $m_1$. The results in

Table 1. Power curves for selected values of $m_1$ and $k=1$ under ${\Sigma }_1$ and $m=10$.

	RAPTT		Max		Ave		95
c	m1 = 10	m1 = 20	m1 = 10	m1 = 20	m1 = 10	m1 = 20	m1 = 10	m1 = 20
0	0.040	0.050	0.046	0.064	0.042	0.050	0.046	0.064
0.5	0.072	0.094	0.086	0.090	0.096	0.090	0.086	0.090
1	0.172	0.148	0.158	0.136	0.224	0.168	0.158	0.136
1.5	0.316	0.306	0.300	0.240	0.412	0.358	0.300	0.240
2	0.480	0.492	0.448	0.426	0.600	0.558	0.448	0.426
2.5	0.676	0.676	0.650	0.548	0.784	0.742	0.650	0.548
3	0.818	0.792	0.778	0.716	0.876	0.852	0.778	0.716
3.5	0.880	0.870	0.844	0.806	0.926	0.924	0.844	0.806
4	0.932	0.952	0.936	0.892	0.974	0.974	0.936	0.892
4.5	0.978	0.972	0.964	0.936	0.992	0.990	0.964	0.936
5	0.990	0.984	0.982	0.978	0.996	0.996	0.982	0.978
5.5	0.998	0.994	0.996	0.996	1.000	1.000	0.996	0.996
6	1.000	1.000	1.000	1.000	1.000	1.000	1.000	1.000

and 

Table 2. Power curves for selected values of $m_1$ and $k=25$ under ${\Sigma }_2$ and $m=100$.

	RAPTT		Max		Ave		95
c	m1 = 10	m1 = 20	m1 = 10	m1 = 20	m1 = 10	m1 = 20	m1 = 10	m1 = 20
0	0.064	0.052	0.044	0.046	0.058	0.044	0.042	0.034
0.5	0.212	0.178	0.112	0.122	0.222	0.172	0.212	0.144
1	0.510	0.388	0.296	0.216	0.500	0.384	0.446	0.316
1.5	0.770	0.632	0.454	0.380	0.766	0.632	0.692	0.552
2	0.888	0.788	0.664	0.542	0.898	0.774	0.858	0.698
2.5	0.972	0.930	0.796	0.746	0.974	0.918	0.942	0.858
3	0.988	0.982	0.898	0.874	0.988	0.982	0.978	0.952
3.5	1.000	0.994	0.970	0.920	0.998	0.990	1.000	0.974
4	1.000	0.998	0.996	0.974	1.000	0.998	1.000	0.998
4.5	1.000	1.000	0.996	0.984	1.000	1.000	1.000	0.996
5	1.000	1.000	0.996	1.000	1.000	1.000	1.000	1.000
5.5	1.000	1.000	1.000	1.000	1.000	1.000	1.000	1.000
6	1.000	1.000	1.000	1.000	1.000	1.000	1.000	1.000

demonstrate that the proposed test procedures are consistent. Specifically, the power of all tests increases monotonically with the signal strength c, approaching one as c grows large.

3.2. Comparing Two Covariance Matrices

3.2.1. Setup and Simulation Design

We adopt the models in Wu and Li [15] for evaluating our proposed method. The three models considered are:

Model a:

${\left\{{\mathit{X}}_i\right\}}_{i=1}^{n_1}$ and ${\left\{{\mathit{Y}}_i\right\}}_{i=1}^{n_2}$ are independently generated as follows: let $Z_{ij}$ be generated from a standard normal distribution, and then construct ${\mathit{X}}_i=(X_{i1},\dots ,X_{ip})$ by letting

$$X_{ij}=Z_{ij}+{\theta }_1{Z}_{ij+1},$$

and ${\mathit{Y}}_i=(Y_{i1},\dots ,Y_{ip})$ by letting

$$Y_{ij}=Z_{ij}+{\theta }_1{Z}_{ij+1}+{\theta }_2{Z}_{ij+2},$$

with ${\theta }_1=0.5$ and ${\theta }_2=0.5$.

Next, consider population covariance matrices of the following form:

$$\begin{array}{c}\hfill {\Sigma }_1=\left(\begin{array}{ccccc}{d}_1& {\rho }_1& \dots & {\rho }_1^{p-2}& {\rho }_1^{p-1}\\ {\rho }_1& d_1& \dots & {\rho }_1^{p-3}& {\rho }_1^{p-2}\\ ⋮& ⋮& \ddots & ⋮& ⋮\\ {\rho }_1^{p-2}& {\rho }_1^{p-3}& \dots & d_1& {\rho }_1\\ {\rho }_1^{p-1}& {\rho }_1^{p-2}& \dots & {\rho }_1& d_1\end{array}\right) \mathrm{and} {\Sigma }_2=\left(\begin{array}{ccccc}{d}_2& {\rho }_2& \dots & {\rho }_2^{p-2}& {\rho }_2^{p-1}\\ {\rho }_2& d_2& \dots & {\rho }_2^{p-3}& {\rho }_2^{p-2}\\ ⋮& ⋮& \ddots & ⋮& ⋮\\ {\rho }_2^{p-2}& {\rho }_2^{p-3}& \dots & d_2& {\rho }_2\\ {\rho }_2^{p-1}& {\rho }_2^{p-2}& \dots & {\rho }_2& d_2\end{array}\right).\end{array}$$

(10)

Model b:

$(d_1,{\rho }_1,d_2,{\rho }_2)=(1.2,0.6,1,0.5)$

Model c:

$(d_1,{\rho }_1,d_2,{\rho }_2)=(1.1,0.24,1,0.2)$

Critical values are estimated from the combined simulated null distributions based on 1000 simulation runs for each covariance matrix in Models a, b, and c.

3.2.2. Simulation Results

We evaluated the performance of the proposed tests for comparing two covariance matrices under three distinct models (Models a, b, and c). We also include the test statistic $max{F}_j$ from Wu and Li [15] for comparison. This allows us to evaluate the relative performance of our p-value-based approaches against the previously suggested maximum-based method.

Type-I Error Control

Under the null hypothesis ${\Sigma }_1={\Sigma }_2$, we select one representative scenario from each model to evaluate the sizes of the proposed tests. Specifically, we choose ${\theta }_1=0.5$ and ${\theta }_2=0$ for Model a, $d_1=d_2=1.2$ and ${\rho }_1={\rho }_2=0.6$ for Model b, and $d_1=d_2=1.1$ and ${\rho }_1={\rho }_2=0.24$ for Model c. The empirical sizes under these settings are reported in

Table 3. Empirical sizes for training sizes from 0 to 20 (step size 2), $m=10$ under three models.

	Minp			Avep			max Fj
m1	Model a	Model b	Model c	Model a	Model b	Model c	Model a	Model b	Model c
0	0.0420	0.0530	0.0590	0.0590	0.0580	0.0560	0.0420	0.0530	0.0590
2	0.0490	0.0520	0.0600	0.0500	0.0440	0.0410	0.0490	0.0520	0.0600
4	0.0410	0.0320	0.0450	0.0490	0.0470	0.0580	0.0410	0.0320	0.0450
6	0.0670	0.0530	0.0590	0.0640	0.0620	0.0590	0.0670	0.0530	0.0590
8	0.0500	0.0630	0.0630	0.0520	0.0600	0.0530	0.0500	0.0630	0.0630
10	0.0640	0.0710	0.0730	0.0450	0.0390	0.0430	0.0640	0.0710	0.0770
12	0.0430	0.0430	0.0420	0.0590	0.0470	0.0560	0.0430	0.0430	0.0420
14	0.0510	0.0520	0.0390	0.0490	0.0490	0.0440	0.0510	0.0520	0.0390
16	0.0620	0.0560	0.0520	0.0550	0.0610	0.0590	0.0620	0.0560	0.0520
18	0.0390	0.0440	0.0440	0.0480	0.0400	0.0550	0.0390	0.0440	0.0440
20	0.0480	0.0430	0.0480	0.0430	0.0450	0.0420	0.0480	0.0430	0.0480

and 

Table 4. Empirical size for training sizes from 0 to 20 (step size 2), $m=100$ under three models.

	Minp			Avep			max Fj
m1	Model a	Model b	Model c	Model a	Model b	Model c	Model a	Model b	Model c
0	0.0400	0.0460	0.0430	0.0610	0.0530	0.0500	0.0400	0.0460	0.0430
2	0.0660	0.0630	0.0610	0.0640	0.0480	0.0530	0.0660	0.0630	0.0610
4	0.0460	0.0330	0.0420	0.0720	0.0580	0.0660	0.0460	0.0330	0.0420
6	0.0440	0.0510	0.0490	0.0740	0.0470	0.0570	0.0440	0.0510	0.0490
8	0.0540	0.0420	0.0490	0.0770	0.0420	0.0440	0.0540	0.0420	0.0490
10	0.0620	0.0630	0.0650	0.0720	0.0580	0.0580	0.0620	0.0630	0.0650
12	0.0390	0.0610	0.0540	0.0760	0.0600	0.0670	0.0390	0.0610	0.0540
14	0.0590	0.0590	0.0540	0.0700	0.0490	0.0600	0.0590	0.0590	0.0540
16	0.0450	0.0380	0.0380	0.0800	0.0530	0.0670	0.0450	0.0380	0.0380
18	0.0420	0.0390	0.0460	0.0790	0.0610	0.0620	0.0420	0.0390	0.0460
20	0.0560	0.0510	0.0580	0.0820	0.0580	0.0520	0.0560	0.0510	0.0580

for $m=10$ and $m=100$, respectively.

The empirical sizes reported in Table 3 demonstrate that the proposed tests effectively control the Type I error across all training sizes and models. Most values remain close to the nominal level of 0.05. The empirical sizes reported in Table 4 show that the proposed tests generally maintain appropriate Type I error control across all training sizes and models when $m=100$. Most values remain close to the nominal level of 0.05. While a few entries reach values near 0.08, these deviations are modest. Overall, the results suggest that the tests remain well-calibrated under the null hypothesis.

Power Comparisons

We computed the empirical power of the tests across varying training sample sizes (from 0 to 20 in increments of 2) and two projection settings ($m=10$ and $m=100$). The results are summarized in

Table 5. Empirical powers for training sizes from 0 to 20 (step size 2), $m=10$ under three models.

	Minp			Avep			max Fj
m1	Model a	Model b	Model c	Model a	Model b	Model c	Model a	Model b	Model c
0	0.2380	0.1150	0.2280	0.5860	0.2400	0.5970	0.2340	0.1100	0.2560
2	0.2090	0.1060	0.2100	0.6280	0.2790	0.5930	0.2280	0.1370	0.2730
4	0.2320	0.1140	0.2410	0.6040	0.2590	0.5790	0.2130	0.0950	0.2210
6	0.2000	0.0930	0.2010	0.5070	0.2120	0.5210	0.2190	0.1330	0.2310
8	0.2300	0.1040	0.2350	0.5730	0.2490	0.5510	0.2320	0.1270	0.2360
10	0.1660	0.0860	0.1520	0.5520	0.2600	0.5660	0.2170	0.1110	0.2360
12	0.1810	0.0880	0.1980	0.5000	0.2010	0.5150	0.1920	0.0970	0.1980
14	0.1790	0.1150	0.2090	0.4620	0.2010	0.4910	0.1860	0.1190	0.2160
16	0.1970	0.1040	0.2130	0.5050	0.2230	0.4570	0.2190	0.1170	0.2040
18	0.1600	0.0990	0.1800	0.4430	0.1840	0.4280	0.1390	0.0880	0.1570
20	0.1580	0.1000	0.1700	0.3960	0.1760	0.4370	0.1690	0.0800	0.1820

and 

Table 6. Empirical powers for training sizes from 0 to 20 (step size 2), $m=100$ under three models.

	Minp			Avep			max Fj
m1	Model a	Model b	Model c	Model a	Model b	Model c	Model a	Model b	Model c
0	0.2970	0.1300	0.3150	0.9990	0.8050	0.9980	0.3210	0.1560	0.3330
2	0.2890	0.1390	0.3120	0.9990	0.8350	0.9990	0.3720	0.1730	0.3760
4	0.3490	0.1720	0.4120	0.9970	0.7970	0.9970	0.2860	0.1190	0.3030
6	0.2840	0.1180	0.3070	0.9980	0.7910	0.9990	0.2510	0.1090	0.2670
8	0.2410	0.1330	0.2860	0.9960	0.7840	0.9990	0.2620	0.1340	0.2760
10	0.2690	0.1490	0.2730	0.9950	0.7600	0.9970	0.2910	0.1370	0.3040
12	0.1880	0.0940	0.2220	0.9950	0.7770	0.9960	0.2850	0.1450	0.2840
14	0.2590	0.1210	0.2790	0.9900	0.7500	0.9940	0.2470	0.1260	0.2540
16	0.2440	0.1200	0.2630	0.9860	0.7320	0.9900	0.2170	0.1100	0.2150
18	0.2350	0.1200	0.2430	0.9840	0.6610	0.9910	0.2140	0.1030	0.2310
20	0.2310	0.1380	0.2350	0.9760	0.6630	0.9890	0.2360	0.1140	0.2260

.

The ${\mathbf{Ave}}_p$ statistic consistently outperformed both ${\mathbf{Min}}_p$ and $max{F}_j$ across all models and settings, particularly for larger m. This advantage likely stems from the non-sparse nature of the covariance matrix differences, where averaging over projections can capture a broader structure.

For $m=10$ (Table 5), all three statistics demonstrated modest gains in power with small training sample sizes. However, ${\mathbf{Ave}}_p$ remained superior in most scenarios, while $max{F}_j$ and ${\mathbf{Min}}_p$ performed comparably. The results for $m=100$ (Table 6) revealed similar patterns, though with more substantial power gains compared to the $m=10$ case. Increasing the number of projections to $m=100$ significantly enhanced the power of all three ${\mathbf{Ave}}_p$, ${\mathbf{Min}}_p$, and $max{F}_j$ statistics.

We demonstrate the power consistency of the proposed tests using Model a, with parameters ${\theta }_1=0$ and varying ${\theta }_2$. In this setting, the Frobenius norm of ${\Sigma }_2-{\Sigma }_1$ increases as ${\theta }_2$ increases, reflecting greater deviation from the null hypothesis. As shown in Figure 19, Figure 20, Figure 21 and Figure 22 the empirical power of the tests approaches 1 as ${\theta }_2$ increases for both $m=10$ and $m=100$.

4. Application to Acute Lymphoblastic Leukemia (ALL) Data

We apply the proposed method to the Acute Lymphoblastic Leukemia (ALL) dataset, which has been previously analyzed by Chen et al. [6]. ALL is a common type of cancer characterized by the overproduction of lymphocytes in the bone marrow. The dataset comprises gene expression profiles from 128 individuals, each with 12,625 features.

4.1. Data Preparation

To align with biological interpretability, we subset the data to include only individuals classified as either BCR/ABL or NEG based on molecular biology annotations (mol.biol column). This results in two groups: BCR/ABL ($n_1=37$ individuals) and NEG ($n_2$ = 42 individuals). We focus on Gene Ontology (GO) terms across three domains—Molecular Function (MF), Biological Process (BP), and Cellular Component (CC)—to test for differences in covariance matrices between the two groups.

4.2. Procedures

• Initial Screening: We follow the same procedure in Chen et al. [6] to perform an initial screening using the genefilter package (Bioconductor version 3.6) to retain 2391 genes for analysis.
• Gene Set Selection: GO term-based gene sets are extracted, excluding those with fewer than 2 genes to ensure meaningful multivariate analysis. This yields 3468, 571, and 803 GO terms for BP, CC, and MF, respectively. The largest gene set contains 1644 genes.
• Projected Test: For each gene set, we apply the covariance matrix test Ave_p test. For those gene sets that fail to reject, we then apply the test Ave with $m=10$ random projections to compare mean vectors between the BCR/ABL and NEG groups. We consider two training sample size settings: $m_1=m_2=0$ (no training data) and $m_1=m_2=2$ (with training data) as suggested by Table 5.
• Multiple Testing Correction: p-values are adjusted using the Benjamini-Hochberg procedure to control the false discovery rate (FDR) at 5%.

4.3. Results

The analysis reveals significant differences in covariance structures between the two groups:

• Case 1: $m_1=m_2=0$

• BP: 208 (6.00%) gene sets are significant ($p<0.05$ after FDR correction)
• CC: 25 (4.38%) gene sets are significant
• MF: 27 (3.36%) gene sets are significant

• Case 2: $m_1=m_2=2$

• BP: 206 (5.94%) gene sets are significant ($p<0.05$ after FDR correction)
• CC: 28 (4.90%) gene sets are significant
• MF: 32 (3.99%) gene sets are significant

The results indicate that the proposed method can detect differences in covariance structures in high-dimensional gene expression data. The version using training samples performs slightly better than the one without, identifying more significant gene sets in some cases. These findings suggest that incorporating training data may improve sensitivity and that accounting for covariance heterogeneity is important in genomic analyses.

5. Summary and Conclusions

This paper introduces an improved random projection framework for high-dimensional two-sample testing of both mean vectors and covariance matrices. By aligning projection directions with estimated parameters obtained from training samples, our approach enhances the power of traditional projection-based tests while maintaining the Type I error rate at the nominal level.

When no external training data are available, the method divides the observed data into training and test subsets. Although this reduces the test sample size, simulations show that the resulting power gains often outweigh this drawback. We recommend using no more than one-third of the available data as training samples.

For mean vector testing, which is among the proposed aggregation strategies, the Ave statistic provides the most consistent performance across settings, especially for dense signals. The Max statistic is more powerful for detecting sparse alternatives, particularly in low-dimensional projections. The 95th percentile statistic offers a good balance between the two, effectively capturing both sparse and dense signals.

Two p-value-based statistics were introduced for testing the equality of two covariance matrices. Across all three models examined, the average p-value statistic Ave_p consistently outperformed the minimum p-value approach Min_p. The gain in power from using a training sample is subtle due to the large number of unknown parameters in the covariance matrices. Future research on high-dimensional covariance matrix inference remains open.