Adaptive CoCoLasso for High-Dimensional Measurement Error Models

Abstract

A significant portion of theoretical and empirical studies in high-dimensional regression have primarily concentrated on clean datasets. However, in numerous practical scenarios, data are often corrupted by missing values and measurement errors, which cannot be ignored. Despite the substantial progress in high-dimensional regression with contaminated covariates, methods that achieve an effective trade-off among prediction accuracy, feature selection, and computational efficiency remain significantly underexplored. We introduce adaptive convex conditioned Lasso (Adaptive CoCoLasso), offering a new approach that can handle high-dimensional linear models with error-prone measurements. This estimator combines a projection onto the nearest positive semi-definite matrix with an adaptively weighted ${\ell }_1$ penalty. Theoretical guarantees are provided by establishing error bounds for the estimators. The results from the synthetic data analysis indicate that the Adaptive CoCoLasso performs strongly in prediction accuracy and mean squared error, particularly in scenarios involving both additive and multiplicative noise in measurements. While the Adaptive CoCoLasso estimator performs comparably or is slightly outperformed by certain methods, such as Hard, in reducing the number of incorrectly identified covariates, its strength lies in offering a more favorable trade-off between prediction accuracy and sparse modeling.

1. Introduction

High-dimensional statistical learning has found extensive applications across diverse fields, including artificial intelligence, genomics, molecular biology, and economics. Numerous effective methods leveraging sparse learning through regularization have been developed to facilitate statistical inference in high-dimensional settings. These methods are well-documented in various studies, such as [1,2,3,4,5,6,7,8,9,10,11,12,13], among others. However, most of the previous research has focused on error-free data. In practice, measurement errors are prevalent in applications such as surveys with missing or inaccurate data due to non-responses, voting systems affected by imprecise instruments or systematic biases, and sensor networks corrupted by communication failures or environmental interference. Challenges involving noisy, incomplete, or corrupted data are frequently encountered. Naively applying methods designed for clean datasets to those affected by measurement errors can lead to inconsistent and imprecise estimates, which in turn result in inaccurate conclusions, particularly in high-dimensional settings. Therefore, developing robust methods for model selection and estimation that explicitly account for measurement errors in high-dimensional problems is of paramount importance.

In recent years, sparse modeling in high-dimensional models with measurement errors has garnered widespread attention. For instance, Ref. [14] proposed minimizing regularized least squares while accounting for additive measurement errors in the covariate matrices of partially linear models. In high-dimensional linear sparse regression, Ref. [15] developed a Lasso-type estimator that utilizes an unbiased approximation to replace the corrupted Gram matrix. However, incorporating measurement error information often leads to non-convex likelihood functions, complicating the solution of the associated optimization problems. To address this challenge, Ref. [16] proposed the nearest positive semi-definite projection matrix as an approximation for the unbiased Gram matrix estimate. Using this matrix as a foundation, they introduced the convex conditioned Lasso (CoCoLasso), which reformulates the objective function as a convex optimization problem to facilitate efficient sparse learning in error-prone high-dimensional linear models.

Although CoCoLasso demonstrates superior computational efficiency due to its convex optimization framework, the use of the ${\ell }_1$ penalty imposes the same level of shrinkage on all coefficients, which can introduce biases. This often results in overfitting by selecting an overly complex model to minimize prediction error [1,12,17]. To address the biases and overfitting introduced by the ${\ell }_1$ penalty, Ref. [18] proposed balanced estimation, which is based on the nearest positive semi-definite matrix and incorporates combined ${\ell }_1$ and concave regularization. Although balanced estimation achieves an ideal trade-off between prediction accuracy and variable selection, it suffers from certain limitations. Firstly, the non-convex nature of the concave regularization results in increased computational complexity, rendering its application in high-dimensional settings difficult. Secondly, the selection of tuning parameters for the concave penalty is often difficult and may lead to suboptimal performance in practice.

To address these issues, we propose Adaptive CoCoLasso, which combines the nearest positive semi-definite projection with an adaptive ${\ell }_1$ penalty. The Adaptive CoCoLasso estimator not only preserves the computational efficiency of convex optimization but also achieves precise estimation and feature selection in the presence of both additive and multiplicative measurement errors. By imposing higher penalties on zero coefficients and lower penalties on nonzero coefficients, the Adaptive CoCoLasso minimizes estimation bias and enhances variable selection accuracy. Furthermore, error bounds for the Adaptive CoCoLasso estimator are established, and a theorem guarantees the consistency of support recovery.

This paper makes two primary contributions. First, we propose the Adaptive CoCoLasso estimator for high-dimensional linear regression models where the design matrix is affected by measurement errors, aiming to ensure precise estimation and accurate variable selection. By applying stronger penalties to zero coefficients and weaker penalties to nonzero coefficients, the method effectively mitigates overfitting when dealing with additive and multiplicative measurement errors. In addition, we establish theoretical guarantees for the proposed method by deriving oracle inequalities for prediction and estimation errors and proving the consistency of support recovery. Extensive simulation studies demonstrate the effectiveness of our approach.

The structure of this paper is as follows. Section 2 outlines the model setup and introduces the proposed Adaptive CoCoLasso estimator. Section 3 presents the theoretical properties, including bounds on oracle estimation errors. Section 4 evaluates the finite-sample performance of the proposed method through simulation studies. We provide all the proofs in the Appendix A.

Notation 1. 

For a vector $\mathbf{x}={(x_1,\dots ,x_p)}^{\top }$, the ${\ell }_q$ norm is defined as ${\parallel \mathbf{x}\parallel }_q={\left({\sum }_{j=1}^p{|x_j|}^q\right)}^{1/q}$ for $q\in (0,\infty )$, and the ${\ell }_{\infty }$ norm is provided by ${\parallel \mathbf{x}\parallel }_{\infty }={max}_{1\le i\le p}|x_i|$. For a matrix $\mathbf{A}=\left(a_{ij}\right)\in {\mathbb{R}}^{p\times q}$, the following matrix norms are defined: ${\parallel \mathbf{A}\parallel }_1={max}_{1\le j\le q}{\sum }_{i=1}^p|a_{ij}|$, ${\parallel \mathbf{A}\parallel }_{\infty }={max}_{1\le i\le p}{\sum }_{j=1}^q|a_{ij}|$, ${\parallel \mathbf{A}\parallel }_{max}={max}_{i,j}|a_{ij}|$, and ${\parallel \mathbf{A}\parallel }_2={\mathsf{\Lambda }}_{max}{\left({\mathbf{A}}^{\top }\mathbf{A}\right)}^{1/2}$, where ${\mathsf{\Lambda }}_{min}\left(\mathbf{A}\right)$ and ${\mathsf{\Lambda }}_{max}\left(\mathbf{A}\right)$ denote the smallest and largest eigenvalues of $\mathbf{A}$, respectively.

2. Adaptive CoCoLasso for Error-Prone Models

2.1. Model Setting

Consider the high-dimensional linear regression model

$$\mathbf{y}=\mathbf{X}\mathit{\beta }+\mathit{\epsilon },$$

(1)

where $\mathbf{y}={(y_1,\dots ,y_n)}^{\top }$ represents the n-dimensional response vector, $\mathbf{X}={({\mathbf{X}}_1,\dots ,{\mathbf{X}}_n)}^{\top }\in {\mathbb{R}}^{n\times p}$ denotes the fixed design matrix, $\mathit{\beta }={({\beta }_1,\dots ,{\beta }_p)}^{\top }$ is the unknown p-dimensional regression coefficient vector, $\mathit{\epsilon }\sim N(0,{\sigma }^2{\mathbf{I}}_n)$ is an n-dimensional error vector independent of $\mathbf{X}$, and ${\mathbf{I}}_n$ is the $n\times n$ identity matrix (the Gaussian distribution is assumed for simplicity in analysis. However, similar theoretical results hold under the sub-Gaussian assumption provided that the tail probability of $\mathit{\epsilon }$ decays exponentially). Measurement errors in the design matrix are common in various applications, leading to the observation of a corrupted covariate matrix $\mathbf{W}\in {\mathbb{R}}^{n\times p}$ rather than the true matrix $\mathbf{X}$.

Two classical cases are associated with measurement errors in the design matrix $\mathbf{X}$. For cases of additive errors, the observed covariates are represented as $\mathbf{W}=\mathbf{X}+\mathbf{A}$, where the rows of the additive error matrix $\mathbf{A}={\left(a_{ij}\right)}_{n\times p}$ are independently and identically distributed (i.i.d.) with a mean vector $\mathbf{0}$ and a covariance matrix ${\mathsf{\Sigma }}_a$. For cases of multiplicative errors, the observed covariates follow $\mathbf{W}=\mathbf{X}\odot \mathbf{M}$, where ⊙ denotes the Hadamard product and the rows of the multiplicative error matrix $\mathbf{M}={\left(m_{ij}\right)}_{n\times p}$ have mean vector ${\mathbf{\mu }}_m$ and covariance matrix ${\mathsf{\Sigma }}_m$. Missing data can be treated as a special case of this model, where the entries of $\mathbf{M}$ are Bernoulli random variables with success probability $1-{\pi }_j$, representing the probability of observing the j-th covariate, and ${\pi }_j$ denotes the missingness rate for the j-th covariate. To ensure model identifiability, the covariance matrix ${\mathsf{\Sigma }}_a$ (for additive errors) or the pair $({\mathit{\mu }}_m,{\mathsf{\Sigma }}_m)$ (for multiplicative errors) is assumed to be known, as in [16,18].

2.2. Adaptive CoCoLasso

In high-dimensional settings where the dimensionality p exceeds the sample size n, the true coefficient vector $\mathit{\beta }$ is often assumed to be sparse. Specifically, the support set $S=\{j:{\beta }_j\ne 0\}$, representing the indices of truly relevant predictors, has size $s=|S|$ satisfying $s=o\left({\displaystyle \frac{n}{logp}}\right)$. This sparsity assumption ensures model identifiability by requiring that only a small subset of predictors are nonzero, i.e., $s\ll n$. Let $S^C$ denote the complementary set of S. In the context of clean data, penalized least squares methods are widely employed for sparse estimation of the true coefficient vector $\mathit{\beta }={({\beta }_1,\dots ,{\beta }_p)}^{\top }$ in high-dimensional linear models. The loss function depends on $\mathsf{\Sigma }$ and $\mathit{\rho }$, where $\mathsf{\Sigma }={\displaystyle \frac{1}{n}}{\mathbf{X}}^{\top }\mathbf{X}$ represents the Gram matrix and $\mathit{\rho }={\displaystyle \frac{1}{n}}{\mathbf{X}}^{\top }\mathbf{y}$ denotes the marginal correlation vector of $(\mathbf{X},\mathbf{y})$, respectively. When the covariate matrix is affected by errors, [15] proposed unbiased estimators $\widehat{\mathsf{\Sigma }}$ and $\tilde{\mathit{\rho }}$ to approximate the unobservable quantities $\mathsf{\Sigma }$ and $\mathit{\rho }$. Specifically, these estimators can be expressed as

$${\widehat{\mathsf{\Sigma }}}_{add}={\displaystyle \frac{1}{n}}{\mathbf{W}}^{\top }\mathbf{W}-{\mathsf{\Sigma }}_a,{\tilde{\mathit{\rho }}}_{add}={\displaystyle \frac{1}{n}}{\mathbf{W}}^{\top }\mathbf{y}$$

(2)

for the additive error cases and

$${\widehat{\mathsf{\Sigma }}}_{mult}={\displaystyle \frac{1}{n}}{\mathbf{W}}^{\top }\mathbf{W}\oslash ({\mathsf{\Sigma }}_m+{\mathit{\mu }}_m{\mathit{\mu }}_m^{\top }),{\tilde{\mathit{\rho }}}_{mult}={\displaystyle \frac{1}{n}}{\mathbf{W}}^{\top }\mathbf{y}\oslash {\mathit{\mu }}_m$$

(3)

for the multiplicative error cases, where ⊘ denotes element-wise division.

However, the unbiased surrogate $\widehat{\mathsf{\Sigma }}$ is generally not positive semi-definite in high-dimensional scenarios. Consequently, $\widehat{\mathsf{\Sigma }}$ may possess a negative eigenvalue, resulting in the term ${\widehat{\mathit{\beta }}}^{\top }\widehat{\mathsf{\Sigma }}\widehat{\mathit{\beta }}$ lacking a lower bound and causing the loss function to lose convexity. To resolve this problem, the unbiased surrogate $\widehat{\mathsf{\Sigma }}$ is replaced by its nearest positive semi-definite projection matrix, defined as $\tilde{\mathsf{\Sigma }}=arg{min}_{\mathsf{\Sigma }\ge 0}{\parallel \mathsf{\Sigma }-\widehat{\mathsf{\Sigma }}\parallel }_{max}$, which can be efficiently solved using the alternating direction method of multipliers (ADMM). By definition and the triangle inequality, it follows that

$$\begin{array}{cc}\hfill \parallel \tilde{\mathsf{\Sigma }}-\widehat{\mathsf{\Sigma }}{\parallel }_{max}& \le \parallel \tilde{\mathsf{\Sigma }}{-\mathsf{\Sigma }\parallel }_{max}+\parallel \mathsf{\Sigma }-\widehat{\mathsf{\Sigma }}{\parallel }_{max}\le 2{\parallel \mathsf{\Sigma }-\widehat{\mathsf{\Sigma }}\parallel }_{max},\hfill \end{array}$$

(4)

indicating that $\tilde{\mathsf{\Sigma }}$ serves as an approximation to $\mathsf{\Sigma }$ with accuracy comparable to that of the unbiased estimate $\widehat{\mathsf{\Sigma }}$.

Following the processing of $\tilde{\mathsf{\Sigma }}$ and $\mathit{\rho }$, Ref. [16] introduced the CoCoLasso method, which employs the ${\ell }_1$ penalty for regularization. However, the $L_1$ penalty often selects overly large models to minimize prediction risk. Motivated by [19], we adopt a weighted $L_1$ penalty to develop a convex objective function, where the weights are determined by an initial estimator. Suppose $\tilde{\mathit{\beta }}={({\tilde{\beta }}_1,\dots ,{\tilde{\beta }}_p)}^{\top }$ is an initial estimator of $\mathit{\beta }$, which provides preliminary estimates of the regression coefficients. Based on this initial estimator, we define the weight vector $\mathit{\omega }={({\omega }_1,\dots ,{\omega }_p)}^{\top }$, where ${\omega }_j={|{\tilde{\beta }}_j|}^{-1}$ for $j=1,\dots ,p$. These weights enable the penalty to adapt to the relative importance of each variable, assigning larger penalties to coefficients with smaller initial estimates and smaller penalties to coefficients with larger initial estimates. Specifically, the proposed Adaptive CoCoLasso estimator $\widehat{\mathit{\beta }}$ is defined as the optimal solution to the following optimization problem, computed after obtaining $\tilde{\mathsf{\Sigma }}$ and $\tilde{\mathit{\rho }}$ as provided in (2) and (3):

$$\begin{array}{cc}\hfill \widehat{\mathit{\beta }}& =arg\underset{\mathit{\beta }\in {\mathbb{R}}^p}{min}\left\{{\displaystyle \frac{1}{2}}{\mathit{\beta }}^{\top }\tilde{\mathsf{\Sigma }}\mathit{\beta }-{\tilde{\mathit{\rho }}}^{\top }\mathit{\beta }+2\lambda \sum _{j=1}^p{\omega }_j|{\beta }_j|\right\},\hfill \end{array}$$

(5)

where $\lambda =c_{0}s{\left[{\displaystyle \frac{logp}{n}}\right]}^{1/2}$ for some positive constant $c_0$. Let $n^{-1/2}\tilde{\mathbf{W}}\in {\mathbb{R}}^{p\times p}$ be the Cholesky factor of $\tilde{\mathsf{\Sigma }}$, such that $n^{-1}{\tilde{\mathbf{W}}}^{\top }\tilde{\mathbf{W}}=\tilde{\mathsf{\Sigma }}$, and define $\tilde{\mathbf{y}}\in {\mathbb{R}}^p$ to satisfy $n^{-1}{\tilde{\mathbf{W}}}^{\top }\tilde{\mathbf{y}}=\tilde{\mathit{\rho }}$. Then, the proposed Adaptive CoCoLasso estimator $\widehat{\mathit{\beta }}$ defined in (5) can be reformulated equivalently as the global minimizer of the following optimization problem

$$\begin{array}{cc}\hfill \widehat{\mathit{\beta }}& =arg\underset{\mathit{\beta }\in {\mathbb{R}}^p}{min}\left\{{\displaystyle \frac{1}{2n}}\parallel \tilde{\mathbf{y}}-\tilde{\mathbf{W}}{\mathit{\beta }\parallel }_2^2+2\lambda \sum _{j=1}^p{\omega }_j|{\beta }_j|\right\}.\hfill \end{array}$$

(6)

For comparison, CoCoLasso solves the following optimization problem:

$${\widehat{\mathit{\beta }}}^{\mathrm{CoCoLasso}}=arg\underset{\mathit{\beta }\in {\mathbb{R}}^p}{min}\left\{{\displaystyle \frac{1}{2n}}\parallel \tilde{\mathbf{y}}-\tilde{\mathbf{W}}{\mathit{\beta }\parallel }_2^2+\lambda \sum _{j=1}^p|{\beta }_j|\right\}.$$

Here, the penalty term $\lambda {\sum }_{j=1}^p|{\beta }_j|$ introduces sparsity by shrinking some coefficients to exactly zero, effectively performing variable selection. However, since the same penalty $\lambda >0$ is applied to all coefficients, it tends to introduce bias, particularly for larger coefficients. Unlike CoCoLasso, Adaptive CoCoLasso incorporates data-driven weights ${\omega }_j$, where ${\omega }_j={|{\tilde{\beta }}_j|}^{-1}$, into the penalty term ${\sum }_{j=1}^p{\omega }_j|{\beta }_j|,$ adjusting the penalty to reflect the relative importance of each variable. This weighting scheme enables the Adaptive CoCoLasso to handle cases where variables have vastly different scales or signal strengths. Assigning smaller penalties to variables with larger estimated coefficients avoids over-penalizing important predictors, improving both variable selection accuracy and coefficient estimation. Additionally, the Adaptive CoCoLasso enhances the recovery of weak signals and reduces bias introduced by uniform penalties in traditional CoCoLasso.

3. Theoretical Properties

In this section, we rigorously derive the statistical error bounds for the Adaptive CoCoLasso estimate under ${\ell }_1$ and ${\ell }_2$ norms and establish theoretical guarantees for exact support recovery with high probability. Before discussing the theoretical results, we outline four technical assumptions.

Condition 1. 

The distributions of $\widehat{\mathsf{\Sigma }}$ and $\tilde{\mathit{\rho }}$ are identified by a set of parameters $\mathit{\theta }$. Then, there exist generic constants C and c and positive functions ξ and ${ϵ}_0$ depending on ${\mathit{\beta }}_S,{\mathit{\beta }}_{S^C}$, and ${\sigma }^2$ such that, for every $ϵ\le {ϵ}_0$, $\widehat{\mathsf{\Sigma }}$ and $\tilde{\mathit{\rho }}$ satisfy the following probability statements:

$$\begin{array}{cc}\hfill \mathbb{P}(|{\widehat{\mathsf{\Sigma }}}_{ij}-{\mathsf{\Sigma }}_{ij}|\ge ϵ)& \le Cexp(-cn{ϵ}^2{\xi }^{-1}),foranyi,j=1,\dots ,p;\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill \mathbb{P}(|{\tilde{\mathit{\rho }}}_j-{\mathit{\rho }}_j|\ge ϵ)& \le Cexp(-cn{s}^{-2}{ϵ}^2{\xi }^{-1}),foranyj=1,\dots ,p.\hfill \end{array}$$

(8)

Condition 2. 

For some positive constant κ, assume

$$0<\kappa =\underset{\parallel {\mathbf{v}}_{S^C}{\parallel }_1\le 3{\parallel {\mathbf{v}}_S\parallel }_1}{min}{\displaystyle \frac{{\mathbf{v}}^{\top }\mathsf{\Sigma }\mathbf{v}}{{\parallel \mathbf{v}\parallel }_2^2}},$$

where $\mathbf{v}={({\mathbf{v}}_S^{\top },{\mathbf{v}}_{S^C}^{\top })}^{\top }\in {\mathbb{R}}^p$, with ${\mathbf{v}}_S$ and ${\mathbf{v}}_{S^C}$ representing the subvectors corresponding to the support set S and its complement $S^C$, respectively.

Condition 3. 

The minimum signal strength satisfies ${min}_{j\in S}|{\beta }_j|\ge C\sqrt{s}\lambda ,$ where $C>0$ is a positive constant.

Condition 4. 

The initial estimator $\tilde{\mathit{\beta }}$ satisfies $\parallel \tilde{\mathit{\beta }}{-\mathit{\beta }\parallel }_2=O_P\left(\sqrt{s}\lambda \right)$ with $\lambda =c_{0}s{\left[{\displaystyle \frac{logp}{n}}\right]}^{1/2}$.

Condition 1, known as the closeness condition proposed by [16], requires that surrogates $\widehat{\mathsf{\Sigma }}$ (and consequently $\tilde{\mathsf{\Sigma }}$) and $\tilde{\mathit{\rho }}$ achieve sufficient element-wise closeness to $\mathsf{\Sigma }$ and $\mathit{\rho }$, respectively. This condition has already been proven in [16] for typical additive and multiplicative measurement error cases, with $\widehat{\mathsf{\Sigma }}$ and $\tilde{\mathit{\rho }}$ defined in Equations (2) and (3). Condition 2, the restricted eigenvalue (RE) condition, ensures the stability and non-degeneracy of the design matrix on sparse predictor subsets. A similar RE condition was used in [20] to derive statistical error bounds for the clean Lasso estimator. Condition 3 specifies the minimum signal strength, ensuring that the true signal is large enough relative to the regularization parameter $\lambda$ to differentiate significant predictors from noise. This condition is commonly assumed in high-dimensional regression settings to guarantee consistent variable selection and accurate estimation [21,22].

Condition 4 ensures that the initial estimator $\tilde{\mathit{\beta }}$ approximates the true parameter $\mathit{\beta }$ with an ${\ell }_2$ error rate of $O_P\left(s\sqrt{{\displaystyle \frac{slogp}{n}}}\right)$, providing the accuracy needed to construct adaptive weights that capture the underlying sparsity and improve the efficiency of the Adaptive CoCoLasso estimator. For clean covariates, commonly used initial estimators include Lasso [10], which leverages sparsity in high-dimensional settings, and ridge regression [23], which addresses multicollinearity effectively. Another widely used approach is the marginal regression estimator, which achieves zero-consistency under a partial orthogonality condition and is derived by fitting univariate regressions for each predictor separately [19]. When faced with a design matrix with measurement errors, CoCoLasso can be used as the initial estimator as it is specifically designed for measurement error models. Alternatively, the estimator introduced in [15], which provides theoretical guarantees for high-dimensional regression with noisy or missing data, can act as an initial estimator.

Theorem 1. 

Under Conditions 1–4, the Adaptive CoCoLasso estimator $\widehat{\mathit{\beta }}$ satisfies the following oracle inequalities with high probability $1-O\left(p^{-c_1}\right)$ for some positive constant $c_1$:

$$\parallel \widehat{\mathit{\beta }}{-\mathit{\beta }\parallel }_2=O\left(\sqrt{s}\lambda \right),{\parallel \widehat{\mathit{\beta }}-\mathit{\beta }\parallel }_1=O\left(s\lambda \right).$$

Here, $\lambda =c_{0}s{\left[{\displaystyle \frac{logp}{n}}\right]}^{1/2}$, and the constants hidden in the $O(·)$ notation depend on the restricted eigenvalue constant κ, the noise variance ${\sigma }^2$, and the probabilistic bounds specified in Condition 1.

Moreover, with high probability $1-O\left(p^{-c_2}\right)$, it holds that

$$\mathbb{P}\left(\mathrm{supp}\left(\widehat{\mathit{\beta }}\right)=\mathrm{supp}\left(\mathit{\beta }\right)\right)\to 1\mathrm{as}n\to \infty ,$$

where $\mathrm{supp}\left(\mathit{\beta }\right)=\{j:{\beta }_j\ne 0\}$ denotes the support set of $\mathit{\beta }$, representing the indices of its nonzero components. Similarly, $\mathrm{supp}\left(\widehat{\mathit{\beta }}\right)$ represents the support set of the Adaptive CoCoLasso estimator $\widehat{\mathit{\beta }}$.

Theorem 1 establishes the theoretical properties of the Adaptive CoCoLasso estimator under high-dimensional linear models with measurement errors. Specifically, it guarantees oracle inequalities for the estimation errors in both ${\ell }_2$ and ${\ell }_1$ norms, showing that the errors scale with $\sqrt{s}\lambda$ and $s\lambda$, respectively. The constants in these bounds depend on key factors such as the restricted eigenvalue condition, noise variance, and probabilistic bounds. Tail probability is influenced by measurement errors, as described by the component $\zeta$ in Condition 1. Additionally, the theorem guarantees consistent support recovery, meaning that the true set of relevant predictors can be identified with high probability as the sample size n and dimensionality p increase. All proofs are provided in Appendix A.

4. Numerical Studies

Within this section, we utilize synthetic datasets to evaluate the effectiveness of the Adaptive CoCoLasso (A-CoCoLasso) estimator on finite samples. The comparison includes various alternative estimators, such as CoCoLasso [16], balanced estimation with $L_1$ regularization and smoothly clipped absolute deviation (B-SCAD), balanced estimation with $L_1$ regularization and hard-thresholding (B-Hard) [18], and the standalone hard-thresholding method. The Adaptive CoCoLasso weights were computed using CoCoLasso regression coefficients. The CoCoLasso and Adaptive CoCoLasso estimators were implemented using the LARS algorithm. All the simulation studies were performed using R, focusing on both additive and multiplicative measurement errors to evaluate the performance of the proposed method. In all the numerical experiments, the penalty parameter $\lambda$ was determined through 10-fold cross-validation.

To evaluate the aforementioned estimators, we employed the performance metrics introduced in [18]. The first two metrics are the count of correctly selected covariates (C) and the count of incorrectly selected covariates (IC). These are defined as $C=\mathrm{True}\mathrm{Positives}\left(\mathrm{TP}\right)={\sum }_{j\in S}\mathbf{I}({\widehat{\beta }}_j\ne 0)$ and $IC=\mathrm{False}\mathrm{Positives}\left(\mathrm{FP}\right)={\sum }_{j\notin S}\mathbf{I}({\widehat{\beta }}_j\ne 0)$, respectively. The third and fourth metrics are the prediction error (PE) and the mean squared error (MSE), provided by $\mathrm{PE}\left(\widehat{\mathit{\beta }}\right)={(\mathit{\beta }-\widehat{\mathit{\beta }})}^{\top }{\mathsf{\Sigma }}_X(\mathit{\beta }-\widehat{\mathit{\beta }})$ and $\mathrm{MSE}\left(\widehat{\mathit{\beta }}\right)={\parallel \mathit{\beta }-\widehat{\mathit{\beta }}\parallel }_2^2$, respectively. These metrics collectively assess feature selection accuracy through C and IC while evaluating predictive performance and estimation performance via PE and MSE, thus providing a comprehensive comparison framework.

4.1. Additive Error Cases

Example 1. 

We followed the simulation setup in [18] and generated 100 datasets, each containing $n=100$ observations from the linear model $\mathbf{y}=\mathbf{X}\mathit{\beta }+\mathit{\epsilon }$, with $p=250$, $\mathit{\beta }={(3,1.5,0,0,2,0,\dots ,0)}^{\top }$, and $\sigma =3$. The components of $\mathbf{X}$ and $\mathit{\epsilon }$ were independently sampled from a multivariate normal distribution $\mathcal{N}(\mathbf{0},{\mathsf{\Sigma }}_X)$. We considered two covariance structures for ${\mathsf{\Sigma }}_X$: the autoregressive structure, where ${\mathsf{\Sigma }}_X={(0.5^{|i-j|})}_{1\le i,j\le p}$, and the compound symmetry structure, where ${\mathsf{\Sigma }}_X=0.3\mathbf{1}{\mathbf{1}}^{\top }+0.7\mathbf{I}$. The contaminated covariates $\mathbf{W}$ were obtained as $\mathbf{W}=\mathbf{X}+\mathbf{A}$, where the rows of $\mathbf{A}$ were independently drawn from $N(0,{\tau }^2)$ with $\tau =0.75$ and $\tau =1.25$, respectively. The results for the five estimators are summarized in Table 1 and Table 2.

Table 1. Means and standard errors (in parentheses) of four performance metrics for five methods under additive error cases over 100 replications in the autoregressive structure.

Measure	A-CoCoLasso	CoCoLasso	Hard	B-SCAD	B-Hard
$\tau =0.75$
C	2.93 (0.03)	2.94 (0.03)	2.27 (0.07)	2.85 (0.04)	2.69 (0.05)
IC	3.89 (0.40)	12.54 (0.75)	0.14 (0.04)	8.85 (0.53)	0.71 (0.18)
PE	1.68 (0.12)	3.65 (0.17)	3.01 (0.27)	2.82 (0.16)	2.26 (0.25)
MSE	2.01 (0.15)	3.64 (0.18)	3.67 (0.33)	3.06 (0.17)	2.58 (0.28)
$\tau =1.25$
C	2.79 (0.04)	2.72 (0.05)	1.78 (0.08)	2.55 (0.05)	2.14 (0.08)
IC	5.30 (0.60)	13.36 (0.86)	0.25 (0.06)	8.20 (0.47)	0.83 (0.24)
PE	5.40 (0.24)	8.69 (0.24)	6.51 (0.39)	7.69 (0.32)	6.45 (0.29)
MSE	5.01 (0.21)	7.53 (0.21)	6.11 (0.34)	6.60 (0.26)	5.79 (0.33)

Table 2. Means and standard errors (in parentheses) of four performance metrics for five methods under additive error cases over 100 replications in the compound symmetric structure.

Measure	A-CoCoLasso	CoCoLasso	Hard	B-SCAD	B-Hard
$\tau =0.75$
C	2.77 (0.04)	2.69 (0.05)	1.86 (0.08)	2.62 (0.05)	2.31 (0.07)
IC	8.37 (0.50)	14.95 (0.73)	0.45 (0.07)	8.88 (0.41)	1.89 (0.29)
PE	3.01 (0.15)	4.62 (0.19)	5.04 (0.41)	3.43 (0.20)	4.16 (0.31)
MSE	3.79 (0.20)	6.05 (0.25)	6.15 (0.49)	4.40 (0.25)	5.43 (0.42)
$\tau =1.25$
C	2.46 (0.06)	2.24 (0.07)	1.20 (0.07)	2.07 (0.07)	1.64 (0.07)
IC	12.88 (0.44)	18.84 (0.64)	1.24 (0.13)	13.59 (0.45)	3.64 (0.27)
PE	6.80 (0.21)	8.46 (0.22)	10.43 (0.52)	8.01 (0.30)	8.60 (0.31)
MSE	8.39 (0.26)	10.81 (0.28)	11.59 (0.62)	9.91 (0.36)	11.04 (0.44)

The results in Table 1 and Table 2 highlight the comparative performance of the five methods under additive measurement errors in both the autoregressive and compound symmetric structures. Comparing A-CoCoLasso with CoCoLasso, A-CoCoLasso consistently achieved a higher number of C and fewer instances of IC. For example, in the autoregressive structure with $\tau =0.75$, A-CoCoLasso attained C = 2.93, which is closer to the ideal value of 3, compared to CoCoLasso (C = 2.94) while significantly reducing IC from 12.54 (CoCoLasso) to 3.89. Furthermore, A-CoCoLasso demonstrated better prediction error (PE = 1.68) and mean squared error (MSE = 2.01) compared to CoCoLasso (PE = 3.65; MSE = 3.64). Similarly, when compared to Hard, A-CoCoLasso achieved a higher C (2.93 vs. 2.27) and better overall estimation and prediction performance. These results indicate that A-CoCoLasso provides more accurate variable selection and estimation than both CoCoLasso and Hard.

In comparison to the balanced estimation methods (B-SCAD and B-Hard), A-CoCoLasso also demonstrated superior performance, particularly in reducing IC while maintaining C closer to the ideal value. For example, in the compound symmetric structure with $\tau =1.25$, A-CoCoLasso achieved C = 2.46, outperforming both B-SCAD (C = 2.07) and B-Hard (C = 1.64). Additionally, A-CoCoLasso maintained a competitive IC of 12.88, which is lower than that of B-SCAD (IC = 13.59), while achieving better prediction and estimation accuracy (PE = 6.80; MSE = 8.39) compared to both B-SCAD (PE = 8.01; MSE = 9.91) and B-Hard (PE = 8.60; MSE = 11.04). These results demonstrate that A-CoCoLasso not only balances the trade-off between correctly identifying covariates and excluding noise variables but also provides robust estimation and prediction accuracy under various settings with additive measurement errors.

Example 2. 

To investigate the performance of the methods under ultra-high-dimensional settings with additive measurement errors, we adopted a similar setting to that in [7]. The coefficient vector was specified as $\mathit{\beta }={(1,-0.5, 0.7, -1.2, -0.9, 0.3, 0.55, 0, \dots , 0)}^{\top }$. The sample size, dimensionality, and noise level were set as $n=80$, $p=1000$, and $\sigma =1$, respectively, reflecting an ultra-high-dimensional setting. The variability in the additive errors was characterized by standard deviation values of $\tau =0.25$ and $0.5$. Table 3 summarizes the performance of the five methods under this setting.

Table 3. Means and standard errors (in parentheses) of four performance metrics for five methods under additive error cases over 100 replications in the ultra-high-dimensional autoregressive structure.

Measure	A-CoCoLasso	CoCoLasso	Hard	B-SCAD	B-Hard
$\tau =0.25$
C	3.95 (0.07)	3.96 (0.07)	3.14(0.11)	4.30 (0.09)	3.88 (0.08)
IC	12.7 (1.48)	31.32 (2.03)	0.40 (0.07)	21.69 (1.52)	4.82 (1.49)
PE	0.90 (0.03)	1.37 (0.04)	0.95 (0.04)	0.82 (0.06)	0.88 (0.05)
MSE	1.55 (0.03)	2.19 (0.05)	1.70 (0.08)	1.45 (0.05)	1.52 (0.06)
$\tau =0.5$
C	3.56 (0.07)	3.54 (0.08)	2.36 (0.11)	3.69 (0.09)	3.49 (0.09)
IC	11.05 (1.26)	24.66 (1.32)	0.37 (0.10)	21.84(0.99)	5.05 (0.96)
PE	1.21 (0.03)	1.76 (0.04)	1.36 (0.06)	1.27 (0.04)	1.24 (0.05)
MSE	2.01 (0.04)	2.72 (0.04)	2.26 (0.08)	2.07 (0.06)	2.01 (0.06)

Table 3 presents the performance of the five methods under ultra-high-dimensional settings with additive measurement errors for $\tau =0.25$ and $\tau =0.5$. In terms of the number of correctly identified covariates (C), A-CoCoLasso achieved a competitive performance compared to CoCoLasso and B-SCAD, with values close to the ideal benchmark of 4 under $\tau =0.25$ (C = 3.95), and slightly reduced performance under $\tau =0.5$ (C = 3.56). In contrast, Hard demonstrated considerably lower values for C (3.14 for $\tau =0.25$ and 2.36 for $\tau =0.5$), indicating weaker variable selection ability.

When examining the number of incorrectly identified covariates (IC), A-CoCoLasso substantially outperformed CoCoLasso and B-SCAD, maintaining much lower IC values (12.7 for $\tau =0.25$ and 11.05 for $\tau =0.5$) compared to CoCoLasso (31.32 and 24.66, respectively) and B-SCAD (21.69 and 21.84, respectively). Hard achieved the smallest IC but at the cost of reduced C, highlighting its conservative nature. In terms of PE and MSE, A-CoCoLasso remained competitive, with PE = 0.90 and MSE = 1.55 for $\tau =0.25$, and PE = 1.21 and MSE = 2.01 for $\tau =0.5$, showing comparable or better results than Hard and B-Hard while maintaining a balanced variable selection performance.

4.2. Multiplicative Error Cases

Example 3. 

We evaluated the performance of Adaptive CoCoLasso and other competing methods, including CoCoLasso, Hard, and balanced estimation, under multiplicative measurement errors. The true model remained the same as in the additive error setup, as described in Example 1. To simulate the multiplicative errors, we generated $\mathbf{W}=\mathbf{X}\odot \mathbf{M}$, where the components of $\mathbf{M}={\left(m_{ij}\right)}_{n\times p}$ followed a log-normal distribution. Specifically, $log\left(m_{ij}\right)$ independently followed the same distribution as $N(0,{\tau }^2)$, with $\tau =0.25$ and $\tau =0.75$. Table 4 and Table 5 present the outcomes for the multiplicative error scenarios.

Table 4. Means and standard errors (in parentheses) of four performance metrics for five methods under multiplicative error cases over 100 replications in the autoregressive structure.

Measure	A-CoCoLasso	CoCoLasso	Hard	B-SCAD	B-Hard
$\tau =0.25$
C	3.00 (0.00)	3.00 (0.00)	2.67 (0.05)	2.99 (0.01)	2.69 (0.02)
IC	4.78 (0.58)	14.50 (0.93)	0.13 (0.05)	8.46 (0.55)	0.51 (0.06)
PE	0.70 (0.05)	1.90 (0.09)	1.02 (0.10)	0.99 (0.06)	0.65 (0.08)
MSE	0.82 (0.06)	1.82 (0.10)	1.41 (0.14)	1.07 (0.07)	0.77 (0.08)
$\tau =0.75$
C	2.90 (0.03)	2.85 (0.04)	2.11 (0.07)	2.79 (0.04)	2.56 (0.06)
IC	5.17 (0.50)	12.98(0.70)	0.27 (0.06)	9.10 (0.58)	0.88 (0.18)
PE	2.36 (0.15)	5.05 (0.20)	3.76 (0.25)	3.63 (0.21)	2.98 (0.23)
MSE	2.63 (0.16)	4.84 (0.18)	4.50 (0.31)	3.80 (0.20)	3.37 (0.26)

Table 5. Means and standard errors (in parentheses) of four performance metrics for five methods under multiplicative error cases over 100 replications in the compound symmetric structure.

Measure	A-CoCoLasso	CoCoLasso	Hard	B-SCAD	B-Hard
$\tau =0.25$
C	3.00 (0.00)	2.97 (0.02)	2.86 (0.03)	2.97 (0.02)	2.93 (0.03)
IC	8.11 (0.60)	14.79 (0.75)	0.16 (0.05)	8.69 (0.57)	2.03 (0.62)
PE	1.23 (0.07)	2.22 (0.10)	0.76 (0.09)	1.06 (0.06)	1.12 (0.13)
MSE	1.53 (0.09)	2.93 (0.14)	0.96 (0.12)	1.37 (0.08)	1.47 (0.18)
$\tau =0.75$
C	2.67 (0.05)	2.54 (0.06)	1.89 (0.07)	2.55 (0.05)	2.30 (0.06)
IC	9.88 (0.53)	15.76 (0.62)	0.57 (0.09)	10.79 (0.50)	2.77 (0.33)
PE	4.23 (0.18)	6.25 (0.22)	4.85 (0.33)	4.16 (0.21)	4.63 (0.28)
MSE	5.27 (0.24)	8.11 (0.29)	5.83 (0.41)	5.28 (0.27)	6.16 (0.38)

The results in Table 4 and Table 5 demonstrate that A-CoCoLasso exhibits strong performance under multiplicative measurement errors across both autoregressive and compound symmetric structures. Compared to the other methods, A-CoCoLasso achieved a desirable balance between correctly identifying covariates and maintaining low false discovery rates while also demonstrating competitive prediction and estimation accuracy. Its robustness under varying levels of multiplicative error ($\tau =0.25$ and $\tau =0.75$) further highlights its adaptability and effectiveness in handling challenging high-dimensional scenarios with multiplicative measurement errors.

Example 4. 

We examined the performance of Adaptive CoCoLasso in ultra-high-dimensional settings with multiplicative measurement errors. To maintain comparability with the additive error scenarios, the simulation setup remained largely consistent with that in Example 2, except that the standard deviation values of the multiplicative errors were specified as $\tau =0.1$ and $\tau =0.2$, ensuring a comparable signal-to-noise ratio. The performance of the five methods is summarized in Table 6.

Table 6. Means and standard errors (in parentheses) of four performance metrics for five methods under multiplicative error cases over 100 replications in the ultra-high-dimensional autoregressive structure.

Measure	A-CoCoLasso	CoCoLasso	Hard	B-SCAD	B-Hard
$\tau =0.1$
C	4.18 (0.09)	4.22 (0.08)	3.51 (0.11)	4.56 (0.09)	4.03 (0.09)
IC	14.55 (1.55)	34.79 (3.78)	0.49 (0.11)	27.42 (3.70)	12.87 (4.12)
PE	0.78 (0.02)	1.21 (0.04)	0.83 (0.05)	0.76 (0.04)	0.82 (0.04)
MSE	1.37 (0.03)	1.99 (0.05)	1.50 (0.08)	1.35 (0.05)	1.44 (0.05)
$\tau =0.2$
C	3.57 (0.08)	4.00 (0.08)	3.21 (0.07)	4.29 (0.08)	3.92 (0.08)
IC	4.26 (1.55)	26.81 (1.45)	0.43 (0.09)	20.82 (0.94)	4.88 (1.09)
PE	0.92 (0.04)	1.28 (0.04)	0.93 (0.05)	0.81 (0.04)	0.87 (0.04)
MSE	1.60 (0.05)	2.10 (0.05)	1.66 (0.07)	1.44 (0.05)	1.50 (0.05)

Table 6 presents the performance of five methods under ultra-high-dimensional settings with multiplicative measurement errors for $\tau =0.1$ and $\tau =0.2$. The results demonstrate that A-CoCoLasso achieved a desirable balance between correctly identifying covariates (C) and maintaining a low number of incorrectly identified covariates (IC) while delivering competitive prediction and estimation accuracy. For $\tau =0.1$, A-CoCoLasso shows robust performance, with C = 4.18 and IC = 14.55, outperforming CoCoLasso in terms of IC (34.79) while maintaining similar predictive performance (PE = 0.78 vs. PE = 1.21 for CoCoLasso). As $\tau$ increased to 0.2, A-CoCoLasso remained effective with C = 3.57 and IC = 4.26, again demonstrating a significant reduction in IC compared to CoCoLasso (26.81). Additionally, A-CoCoLasso achieved comparable PE and MSE values to the best-performing methods, such as B-SCAD, while demonstrating better variable selection than Hard. Overall, these results highlight A-CoCoLasso’s ability to effectively balance variable selection and prediction accuracy under challenging ultra-high-dimensional multiplicative error settings.

5. Discussion

This paper introduces the Adaptive CoCoLasso estimator, designed to balance prediction accuracy and feature selection in high-dimensional linear regression with measurement errors, effectively addressing both additive and multiplicative cases. The proposed method combines two key techniques: the nearest positive semi-definite projection matrix for correcting measurement errors by reconstructing the covariate matrix and an adaptive weighted ${\ell }_1$ penalty, which enhances sparsity and variable selection by assigning data-driven weights to the coefficients. Unlike combined ${\ell }_1$ and concave regularization, which introduces computational challenges due to its non-convex nature and parameter tuning difficulties, Adaptive CoCoLasso retains the computational efficiency of convex optimization while providing robust estimation performance. The methodology leverages the LARS algorithm to solve the penalized optimization problem, ensuring scalability to high-dimensional settings. The theoretical analysis and simulation results show that the Adaptive CoCoLasso achieves robust prediction and estimation performance, effectively addressing overfitting and the challenges posed by contaminated data.

Future work could focus on extending the Adaptive CoCoLasso estimator to address statistical inference challenges, such as constructing confidence intervals and performing hypothesis testing. A major difficulty in these extensions arises from the unknown true covariate matrix, which not only makes predicting the response vector challenging but also hinders accurate noise level estimation, even with a reliable coefficient estimator. These issues fall outside the scope of this paper and represent intriguing directions for future research.

Appendix A.1. Proof of Theorem 1

Proof. 

Denote the estimation error by $\mathbf{\nu }=\widehat{\mathit{\beta }}-\mathit{\beta }$, where $\widehat{\mathit{\beta }}$ is the global minimizer defined in the Adaptive CoCoLasso method. For simplicity of notation, we let $\lambda \parallel \widehat{\mathit{\beta }}{\parallel }_{1,\mathit{\omega }}$ denote $\lambda {\sum }_{j=1}^p{\omega }_j|{\beta }_j|$, where $\parallel \widehat{\mathit{\beta }}{\parallel }_{1,\mathit{\omega }}={\sum }_{j=1}^p{\omega }_j|{\beta }_j|$ represents the weighted ${\ell }_1$ norm. Based on the definition of $\widehat{\mathit{\beta }}$ in (6), the following inequality holds

$${\displaystyle \frac{1}{2}}{\widehat{\mathit{\beta }}}^{\top }\tilde{\mathsf{\Sigma }}\widehat{\mathit{\beta }}-{\tilde{\mathit{\rho }}}^{\top }\widehat{\mathit{\beta }}+2\lambda \parallel \widehat{\mathit{\beta }}{\parallel }_{1,\mathit{\omega }}\le {\displaystyle \frac{1}{2}}{\mathit{\beta }}^{\top }\tilde{\mathsf{\Sigma }}\mathit{\beta }-{\tilde{\mathit{\rho }}}^{\top }\mathit{\beta }+2\lambda {\parallel \mathit{\beta }\parallel }_{1,\mathit{\omega }}.$$

By simple calculations and the definition of the ${\ell }_{\infty }$ norm, we have

$${\displaystyle \frac{1}{2}}{\mathbf{\nu }}^{\top }\tilde{\mathsf{\Sigma }}\mathbf{\nu }+2\lambda \parallel \widehat{\mathit{\beta }}{\parallel }_{1,\mathit{\omega }}\le {\parallel \mathbf{\nu }\parallel }_1\parallel \tilde{\mathit{\rho }}-\tilde{\mathsf{\Sigma }}{\mathit{\beta }\parallel }_{\infty }+2\lambda {\parallel \mathit{\beta }\parallel }_{1,\mathit{\omega }}.$$

(A1)

For better clarity, the proof will be divided into six steps, deriving bounds on the prediction based on the inequality above.

Step 1. We derive a bound for the term $\parallel \tilde{\mathit{\rho }}-\tilde{\mathsf{\Sigma }}{\mathit{\beta }\parallel }_{\infty }$. By applying the triangle inequality, we obtain

$$\parallel \tilde{\mathit{\rho }}-\tilde{\mathsf{\Sigma }}{\mathit{\beta }\parallel }_{\infty }\le \parallel \tilde{\mathit{\rho }}{-\mathit{\rho }\parallel }_{\infty }{+\parallel \mathit{\rho }-\mathsf{\Sigma }\mathit{\beta }\parallel }_{\infty }+{\parallel (\tilde{\mathsf{\Sigma }}-\mathsf{\Sigma })\mathit{\beta }\parallel }_{\infty }.$$

(A2)

For the first part of the inequality (A2) on the right-hand side, $\parallel \tilde{\mathit{\rho }}{-\mathit{\rho }\parallel }_{\infty }$, Condition 1 implies that

$$\mathbb{P}(|{\tilde{\mathit{\rho }}}_j-{\mathit{\rho }}_j|\ge \lambda /6)\le Cexp(-cn{s}^{-2}{\lambda }^2{\xi }^{-1}),\mathrm{for}\mathrm{any}j=1,\dots ,p.$$

Applying the union bound,

$$\mathbb{P}\left(\parallel \tilde{\mathit{\rho }}{-\mathit{\rho }\parallel }_{\infty }\ge \lambda /6\right)\le \sum _{j=1}^p\mathbb{P}\left(|{\tilde{\mathit{\rho }}}_j-{\mathit{\rho }}_j|\ge \lambda /6\right)\le p·Cexp\left(-cn{s}^{-2}{\lambda }^2{\xi }^{-1}\right).$$

Thus, we have $\parallel \tilde{\mathit{\rho }}{-\mathit{\rho }\parallel }_{\infty }=O_P\left(s\sqrt{{\displaystyle \frac{logp}{n}}}\right).$ Next, we derive a bound for the second term ${\parallel \mathit{\rho }-\mathsf{\Sigma }\mathit{\beta }\parallel }_{\infty }$. Note that

$$\mathit{\rho }-\mathsf{\Sigma }\mathit{\beta }={\displaystyle \frac{1}{n}}{\mathbf{X}}^{\top }\mathbf{y}-\mathsf{\Sigma }\mathit{\beta }={\displaystyle \frac{1}{n}}{\mathbf{X}}^{\top }(\mathbf{X}\mathit{\beta }+\mathit{\epsilon })-\mathsf{\Sigma }\mathit{\beta }={\displaystyle \frac{1}{n}}{\mathbf{X}}^{\top }\mathit{\epsilon }.$$

Under the assumption that $\mathit{\epsilon }\sim N(0,{\sigma }^2{\mathbf{I}}_n)$, invoking Lemma A1, we have the probability bound

$$\mathbb{P}\left({\parallel \mathit{\rho }-\mathsf{\Sigma }\mathit{\beta }\parallel }_{\infty }>\lambda /6\right)\le p·Cexp\left(-cn{\lambda }^2{\sigma }^{-2}\right),$$

for some constants $C>0$ and $c>0$. Therefore, with high probability, ${\parallel \mathit{\rho }-\mathsf{\Sigma }\mathit{\beta }\parallel }_{\infty }=O_P\left(\sqrt{{\displaystyle \frac{logp}{n}}}\right).$ The third component $\parallel (\tilde{\mathsf{\Sigma }}-\mathsf{\Sigma }){\mathit{\beta }\parallel }_{\infty }$ can be bounded as follows. Using Condition 1, for all $i,j=1,\dots ,p$, we have

$$\mathbb{P}\left(|{\tilde{\mathsf{\Sigma }}}_{ij}-{\mathsf{\Sigma }}_{ij}|\ge ϵ\right)\le Cexp\left(-cn{ϵ}^2{\xi }^{-1}\right),$$

where $C>0$ and $c>0$ are constants. Combining Lemma A2, we have

$$\parallel \tilde{\mathsf{\Sigma }}{-\mathsf{\Sigma }\parallel }_{max}=O_P\left(\sqrt{{\displaystyle \frac{logp}{n}}}\right).$$

For the sparse vector $\mathit{\beta }$ with support size $|S|=s$, we additionally obtain

$$\parallel (\tilde{\mathsf{\Sigma }}-\mathsf{\Sigma }){\mathit{\beta }\parallel }_{\infty }\le \parallel \tilde{\mathsf{\Sigma }}{-\mathsf{\Sigma }\parallel }_{max}{\parallel \mathit{\beta }\parallel }_1.$$

Using ${\parallel \mathit{\beta }\parallel }_1\le \sqrt{s}{\parallel \mathit{\beta }\parallel }_2$ and assuming ${\parallel \mathit{\beta }\parallel }_2=O\left(1\right)$, it follows that

$$\parallel (\tilde{\mathsf{\Sigma }}-\mathsf{\Sigma }){\mathit{\beta }\parallel }_{\infty }=O_P\left(s\sqrt{{\displaystyle \frac{logp}{n}}}\right).$$

Combining the three parts above, we have

$$\parallel \tilde{\mathit{\rho }}-\tilde{\mathsf{\Sigma }}{\mathit{\beta }\parallel }_{\infty }\le O_P\left(s\sqrt{{\displaystyle \frac{logp}{n}}}\right)+O_P\left(\sqrt{{\displaystyle \frac{logp}{n}}}\right)+O_P\left(s\sqrt{{\displaystyle \frac{logp}{n}}}\right).$$

Since the sparsity s dominates in high-dimensional settings, the final bound is

$$\parallel \tilde{\mathit{\rho }}-\tilde{\mathsf{\Sigma }}{\mathit{\beta }\parallel }_{\infty }=O_P\left(s\sqrt{{\displaystyle \frac{logp}{n}}}\right).$$

(A3)

Step 2. Decompose the error term $\mathbf{\nu }$ into components on the support set $S=\{j:{\beta }_j\ne 0\}$ and its complement $S^C$, so $\mathbf{\nu }={\mathbf{\nu }}_S+{\mathbf{\nu }}_{S^C}.$ The weighted ${\ell }_1$ norms can be written as

$$\parallel \widehat{\mathit{\beta }}{\parallel }_{1,\mathit{\omega }}=\parallel {\widehat{\mathit{\beta }}}_S{\parallel }_{1,\mathit{\omega }}+\parallel {\widehat{\mathit{\beta }}}_{S^C}{\parallel }_{1,\mathit{\omega }}{,\parallel \mathit{\beta }\parallel }_{1,\mathit{\omega }}={\parallel {\mathit{\beta }}_S\parallel }_{1,\mathit{\omega }}.$$

Substituting these into inequality (A1) yields

$${\displaystyle \frac{1}{2}}{\mathbf{\nu }}^{\top }\tilde{\mathsf{\Sigma }}\mathbf{\nu }+2\lambda \parallel {\widehat{\mathit{\beta }}}_{S^C}{\parallel }_{1,\mathit{\omega }}\le {\mathbf{\nu }}^{\top }(\tilde{\mathit{\rho }}-\tilde{\mathsf{\Sigma }}\mathit{\beta })+2\lambda {\parallel {\mathbf{\nu }}_S\parallel }_{1,\mathit{\omega }}.$$

(A4)

For the first term on the left-hand side of (A4), we have

$${\mathbf{\nu }}^{\top }\tilde{\mathsf{\Sigma }}\mathbf{\nu }={\mathbf{\nu }}^{\top }\mathsf{\Sigma }\mathbf{\nu }+{\mathbf{\nu }}^{\top }\Delta \mathbf{\nu },$$

where $\Delta =\tilde{\mathsf{\Sigma }}-\mathsf{\Sigma }$ and $\mathsf{\Sigma }={\displaystyle \frac{1}{n}}{\mathbf{X}}^{\top }\mathbf{X}$. By applying Condition 2 for any $\mathbf{\nu }$ satisfying $\parallel {\mathbf{\nu }}_{S^C}{\parallel }_1\le 3{\parallel {\mathbf{\nu }}_S\parallel }_1$, we have

$${\mathbf{\nu }}^{\top }\mathsf{\Sigma }\mathbf{\nu }\ge n\kappa {\parallel {\mathbf{\nu }}_S\parallel }_2^2,$$

where $\kappa >0$ is the restricted eigenvalue constant. For the error term $\Delta =\tilde{\mathsf{\Sigma }}-\mathsf{\Sigma }$, Condition 1 ensures that

$${\parallel \Delta \parallel }_{max}\le O_P\left(\sqrt{{\displaystyle \frac{logp}{n}}}\right).$$

We can bound $|{\mathbf{\nu }}^{\top }\Delta \mathbf{\nu }|$ as follows: $|{\mathbf{\nu }}^{\top }{\Delta \mathbf{\nu }|\le \parallel \Delta \parallel }_{max}{\parallel \mathbf{\nu }\parallel }_1^2.$ From the sparsity assumption, ${\parallel \mathbf{\nu }\parallel }_1=\parallel {\mathbf{\nu }}_S{\parallel }_1+\parallel {\mathbf{\nu }}_{S^C}{\parallel }_1\le 4{\parallel {\mathbf{\nu }}_S\parallel }_1$. Applying the Cauchy–Schwarz inequality $\parallel {\mathbf{\nu }}_S{\parallel }_1\le \sqrt{s}{\parallel {\mathbf{\nu }}_S\parallel }_2,$ where $s=|S|$, the size of the support set, we obtain

$$|{\mathbf{\nu }}^{\top }\Delta \mathbf{\nu }|\le O_P\left(\sqrt{{\displaystyle \frac{logp}{n}}}\right)·(4\sqrt{s}\parallel {\mathbf{\nu }}_S{{\parallel }_2)}^2.$$

Combining the bounds for ${\mathbf{\nu }}^{\top }\mathsf{\Sigma }\mathbf{\nu }$ and ${\mathbf{\nu }}^{\top }\Delta \mathbf{\nu }$, we have

$${\mathbf{\nu }}^{\top }\tilde{\mathsf{\Sigma }}\mathbf{\nu }={\mathbf{\nu }}^{\top }\mathsf{\Sigma }\mathbf{\nu }+{\mathbf{\nu }}^{\top }\Delta \mathbf{\nu }\ge n\kappa \parallel {\mathbf{\nu }}_S{\parallel }_2^2-O_P\left(s\sqrt{{\displaystyle \frac{logp}{n}}}\right){\parallel {\mathbf{\nu }}_S\parallel }_2^2.$$

When $n\gg s\sqrt{logp}$, the second term is asymptotically negligible. Redefining the restricted eigenvalue constant as ${\kappa }^{\prime }=\kappa /2>0$, we ensure that the first term on the left-hand side of (A4) satisfies ${\mathbf{\nu }}^{\top }\tilde{\mathsf{\Sigma }}\mathbf{\nu }\ge n{\kappa }^{\prime }{\parallel {\mathbf{\nu }}_S\parallel }_2^2.$

Bound the first term ${\mathbf{\nu }}^{\top }(\tilde{\mathit{\rho }}-\tilde{\mathsf{\Sigma }}\mathit{\beta })$ on the right-hand side of (A4) and we have

$${\mathbf{\nu }}^{\top }(\tilde{\mathit{\rho }}-\tilde{\mathsf{\Sigma }}\mathit{\beta })=\sum _{j\in S}{\nu }_j({\tilde{\mathit{\rho }}}_j-{\tilde{\mathsf{\Sigma }}}_j^{\top }\mathit{\beta })+\sum _{j\notin S}{\nu }_j({\tilde{\mathit{\rho }}}_j-{\tilde{\mathsf{\Sigma }}}_j^{\top }\mathit{\beta }).$$

Using the bound in (A3) $\parallel \tilde{\mathit{\rho }}-\tilde{\mathsf{\Sigma }}{\mathit{\beta }\parallel }_{\infty }\le O_P\left(s\sqrt{{\displaystyle \frac{logp}{n}}}\right),$ we have

$$|{\mathbf{\nu }}^{\top }(\tilde{\mathit{\rho }}-\tilde{\mathsf{\Sigma }}\mathit{\beta }){|\le \parallel \mathbf{\nu }\parallel }_1·O_P\left(s\sqrt{{\displaystyle \frac{logp}{n}}}\right).$$

For the sparsity structure of $\mathbf{\nu }$, ${\parallel \mathbf{\nu }\parallel }_1=\parallel {\mathbf{\nu }}_S{\parallel }_1+{\parallel {\mathbf{\nu }}_{S^C}\parallel }_1$. Under the sparsity constraint $\parallel {\mathbf{\nu }}_{S^C}{\parallel }_1\le 3{\parallel {\mathbf{\nu }}_S\parallel }_1$, we obtain ${\parallel \mathbf{\nu }\parallel }_1\le 4{\parallel {\mathbf{\nu }}_S\parallel }_1.$ Further, by the Cauchy–Schwarz inequality $\parallel {\mathbf{\nu }}_S{\parallel }_1\le \sqrt{s}{\parallel {\mathbf{\nu }}_S\parallel }_2,$ we have

$$|{\mathbf{\nu }}^{\top }(\tilde{\mathit{\rho }}-\tilde{\mathsf{\Sigma }}\mathit{\beta })|\le 4\sqrt{s}{\parallel {\mathbf{\nu }}_S\parallel }_2·O_P\left(s\sqrt{{\displaystyle \frac{logp}{n}}}\right).$$

Simplifying, this yields

$$|{\mathit{\nu }}^{\top }(\tilde{\mathit{\rho }}-\tilde{\mathsf{\Sigma }}\mathit{\beta })|\le O_P\left(\sqrt{s}\lambda \right){\parallel {\mathit{\nu }}_S\parallel }_2.$$

(A5)

Substituting bounds (A5) and

$${\mathit{\nu }}^{\top }\tilde{\mathsf{\Sigma }}\mathit{\nu }\ge n{\kappa }^{\prime }{\parallel {\mathit{\nu }}_S\parallel }_2^2$$

into inequality (A4), we obtain

$${\displaystyle \frac{1}{2}}n{\kappa }^{\prime }\parallel {\mathit{\nu }}_S{\parallel }_2^2+2\lambda \parallel {\widehat{\mathit{\beta }}}_{S^C}{\parallel }_{1,\mathit{\omega }}\le O_P\left(\sqrt{s}\lambda \right)\parallel {\mathit{\nu }}_S{\parallel }_2+2\lambda {\parallel {\mathit{\nu }}_S\parallel }_{1,\mathit{\omega }}.$$

(A6)

Step 3. To bound $\parallel {\widehat{\mathit{\beta }}}_{S^C}{\parallel }_{1,\mathit{\omega }}$ on the left-hand side of inequality (A6), we observe that $\parallel {\widehat{\mathit{\beta }}}_{S^C}{\parallel }_{1,\mathit{\omega }}={\sum }_{j\in S^C}{\omega }_j|{\widehat{\beta }}_j|,$ where the weights $\mathit{\omega }=({\omega }_1,\dots ,{\omega }_p)$ are defined based on the initial estimator $\tilde{\mathit{\beta }}$, with ${\omega }_j={|{\tilde{\beta }}_j|}^{-1}$ for $j=1,\dots ,p$. Under Condition 4, the initial estimator satisfies $\parallel \tilde{\mathit{\beta }}{-\mathit{\beta }\parallel }_2=O_P\left(\sqrt{s}\lambda \right)$, and the sparsity assumption implies that ${\beta }_j=0$ for $j\in S^C$. By the consistency of $\tilde{\mathit{\beta }}$, we have $|{\tilde{\beta }}_j|=O_P\left(\sqrt{s}\lambda \right)$ for $j\in S^C$, leading to ${\omega }_j=O_P\left({\displaystyle \frac{1}{\sqrt{s}\lambda }}\right)$. Thus, the penalty term $2\lambda {\omega }_j|{\beta }_j|$ in the objective function (6) becomes $2\lambda {\omega }_j|{\beta }_j|=O_P\left(\lambda ·{\omega }_j·|{\beta }_j|\right)=O_P\left({\displaystyle \frac{1}{\sqrt{s}}}|{\beta }_j|\right).$ Under Condition 3, as $\lambda {\omega }_j$ becomes large for $j\in S^C$, the penalty ensures $\mathbb{P}({\widehat{\mathit{\beta }}}_j=0)\to 1$ as $n\to \infty$. To bound $\parallel {\mathit{\nu }}_S{\parallel }_{1,\mathit{\omega }}$ on the right-hand side of (A6), we note that the weighted ${\ell }_1$ norm satisfies $\parallel {\mathit{\nu }}_S{\parallel }_{1,\mathit{\omega }}={\sum }_{j\in S}{\omega }_j|{\nu }_j|,$ where ${\omega }_j={|{\tilde{\beta }}_j|}^{-1}$, and ${\tilde{\beta }}_j$ denotes the initial estimator. Under Condition 4, the weights satisfy $|{\omega }_j|=O_P\left(1\right)$ for $j\in S$ since ${\tilde{\beta }}_j$ is consistent and ${\beta }_j\ne 0$. Thus, the weighted ${\ell }_1$ norm satisfies $\parallel {\mathit{\nu }}_S{\parallel }_{1,\mathit{\omega }}={\sum }_{j\in S}{\omega }_j|{\nu }_j|\le C\parallel {\mathit{\nu }}_S{\parallel }_1$ for some constant $C>0$. Substituting, we obtain

$${\displaystyle \frac{1}{2}}n{\kappa }^{\prime }\parallel {\mathit{\nu }}_S{\parallel }_2^2\le \parallel {\mathit{\nu }}_S{\parallel }_1·O_P\left(s\sqrt{{\displaystyle \frac{logp}{n}}}\right)+2C\lambda {\parallel {\mathit{\nu }}_S\parallel }_1.$$

(A7)

Step 4. We derive the ${\ell }_2$ norm bound for the estimation error $\mathit{\nu }=\widehat{\mathit{\beta }}-\mathit{\beta }$. Using the Cauchy–Schwarz inequality, the ${\ell }_1$ norm and ${\ell }_2$ norm satisfy $\parallel {\mathit{\nu }}_S{\parallel }_1\le \sqrt{s}{\parallel {\mathit{\nu }}_S\parallel }_2$, where $s=|S|$ is the sparsity level. Substituting this, inequality (A7) can be rewritten as

$${\displaystyle \frac{1}{2}}n{\kappa }^{\prime }\parallel {\mathit{\nu }}_S{\parallel }_2^2\le \sqrt{s}\parallel {\mathit{\nu }}_S{\parallel }_2·O_P\left(s\sqrt{{\displaystyle \frac{logp}{n}}}\right)+2C\lambda \sqrt{s}{\parallel {\mathit{\nu }}_S\parallel }_2.$$

Factoring out $\parallel {\mathit{\nu }}_S{\parallel }_2$, we have

$$\parallel {\mathit{\nu }}_S{\parallel }_2\left({\displaystyle \frac{1}{2}}n{\kappa }^{\prime }{\parallel {\mathit{\nu }}_S\parallel }_2-\sqrt{s}{O}_P\left(s\sqrt{{\displaystyle \frac{logp}{n}}}\right)-2C\lambda \sqrt{s}\right)\le 0.$$

For $\parallel {\mathit{\nu }}_S{\parallel }_2>0$, the term in parentheses must be non-positive. We have

$$\parallel {\mathit{\nu }}_S{\parallel }_2\le {\displaystyle \frac{2\sqrt{s}}{n{\kappa }^{\prime }}}{O}_P\left(s\sqrt{{\displaystyle \frac{logp}{n}}}\right)+{\displaystyle \frac{4C\sqrt{s}\lambda }{n{\kappa }^{\prime }}}.$$

Substituting $\lambda =O\left(s\sqrt{{\displaystyle \frac{logp}{n}}}\right)$, we obtain $\parallel {\mathit{\nu }}_S{\parallel }_2\le {\displaystyle \frac{\sqrt{s}}{n{\kappa }^{\prime }}}{O}_P\left(s\sqrt{{\displaystyle \frac{logp}{n}}}\right).$ Simplifying, the ${\ell }_2$ norm of the error satisfies $\parallel {\mathit{\nu }}_S{\parallel }_2\le C_{1}s\sqrt{{\displaystyle \frac{slogp}{n}}},$ where $C_1>0$ is a constant depending on ${\kappa }^{\prime }$. Hence, we obtain

$$\parallel \widehat{\mathit{\beta }}{-\mathit{\beta }\parallel }_2={\parallel {\mathit{\nu }}_S\parallel }_2\le C_1\sqrt{s}\lambda .$$

(A8)

where $\lambda =s\sqrt{{\displaystyle \frac{logp}{n}}}$ and $C_1$ is some positive constant.

Step 5. We derive the ${\ell }_1$ norm bound for the estimation error $\mathit{\nu }=\widehat{\mathit{\beta }}-\mathit{\beta }$. The ${\ell }_1$ norm of $\mathit{\nu }$ decomposes as ${\parallel \mathit{\nu }\parallel }_1=\parallel {\mathit{\nu }}_S{\parallel }_1+{\parallel {\mathit{\nu }}_{S^C}\parallel }_1$. By the sparsity assumption, $\parallel {\mathit{\nu }}_{S^C}{\parallel }_1\le 3{\parallel {\mathit{\nu }}_S\parallel }_1$, we have ${\parallel \mathit{\nu }\parallel }_1\le 4{\parallel {\mathit{\nu }}_S\parallel }_1.$ By the Cauchy–Schwarz inequality, the ${\ell }_1$ norm on the support set S satisfies $\parallel {\mathit{\nu }}_S{\parallel }_1\le \sqrt{s}{\parallel {\mathit{\nu }}_S\parallel }_2$, where $s=|S|$ is the sparsity level. Substituting the ${\ell }_2$ norm bound from inequality (A8), $\parallel {\mathit{\nu }}_S{\parallel }_2\le C_1\sqrt{s{\displaystyle \frac{slogp}{n}}}$, where $C_1>0$ depends on ${\kappa }^{\prime }$, n, and p. Substituting into the inequality for $\parallel {\mathit{\nu }}_S{\parallel }_1$, we have $\parallel {\mathit{\nu }}_S{\parallel }_1\le s\sqrt{s}·C_1\sqrt{{\displaystyle \frac{slogp}{n}}}=C_1{\displaystyle \frac{s^{2}logp}{n}}$. Combining the bounds for ${\mathit{\nu }}_S$ and ${\mathit{\nu }}_{S^C}$, the ${\ell }_1$ norm satisfies ${\parallel \mathit{\nu }\parallel }_1\le 4{\parallel {\mathit{\nu }}_S\parallel }_1\le 4C_1{\displaystyle \frac{s^{2}logp}{n}}$. Defining $C_2=4C_1$, we conclude

$$\parallel \widehat{\mathit{\beta }}{-\mathit{\beta }\parallel }_1={\parallel \mathit{\nu }\parallel }_1\le C_{2}s\lambda ,$$

where $\lambda =s\sqrt{{\displaystyle \frac{logp}{n}}}$, $C_2>0$ depends on ${\kappa }^{\prime }$, s, n, and p.

Step 6. We aim to show that $\widehat{\mathit{\beta }}$ correctly identifies the support set of the true regression coefficients $\mathit{\beta }$, such that $\mathbb{P}\left(\mathrm{supp}\left(\widehat{\mathit{\beta }}\right)=\mathrm{supp}\left(\mathit{\beta }\right)\right)\to 1\mathrm{as}n\to \infty .$ For a vector $\mathit{\nu }=\widehat{\mathit{\beta }}-\mathit{\beta }$, we have ${\parallel \mathit{\nu }\parallel }_{\infty }\le {\parallel \mathit{\nu }\parallel }_2.$ From inequality (A8), the ${\ell }_2$ norm of the error satisfies $\parallel \widehat{\beta }{-\beta \parallel }_2\le C_1\sqrt{s}\lambda .$ Then, we conclude that $\parallel \widehat{\beta }{-\beta \parallel }_{\infty }\le O_P\left(\sqrt{s}\lambda \right).$ For each $j\in S$, the minimum signal strength, Condition 3 implies $|{\beta }_j|\ge C\sqrt{s}\lambda .$ With high probability, the estimation error satisfies $|{\widehat{\beta }}_j-{\beta }_j|\le O_P\left(\sqrt{s}\lambda \right).$ For $j\in S$, combining this with the minimum signal strength, Condition 3 provides $|{\widehat{\beta }}_j|\ge C{c}_0\sqrt{s}\lambda -O_P\left(\sqrt{s}\lambda \right).$ For sufficiently large n, where $C{c}_0>1$, this ensures $|{\widehat{\beta }}_j|>0$. For $j\notin \mathrm{supp}\left(\mathit{\beta }\right)$, the estimation error simplifies to $|{\widehat{\beta }}_j|=|{\widehat{\beta }}_j-{\beta }_j|=|{\widehat{\beta }}_j|$. The ${\ell }_{\infty }$ norm error bound satisfies $\parallel \widehat{\mathit{\beta }}{-\mathit{\beta }\parallel }_{\infty }\le O_P\left(\sqrt{s}\lambda \right)$, which implies $|{\widehat{\beta }}_j|\le O_P\left(\sqrt{s}\lambda \right)$. In addition, by Condition 3, there is $|{\widehat{\beta }}_j|\ge C\sqrt{s}\lambda$. This implies $O_P\left(\sqrt{s}\lambda \right)\ge |{\widehat{\beta }}_j|\ge C\sqrt{s}\lambda$. This leads to a contradiction. Therefore, ${\widehat{\beta }}_j=0$ for $j\notin \mathrm{supp}\left(\mathit{\beta }\right)$. Combining the cases for $j\in S$ and $j\notin S$, we conclude that $\mathbb{P}\left(\mathrm{supp}\left(\widehat{\mathit{\beta }}\right)=\mathrm{supp}\left(\mathit{\beta }\right)\right)\to 1\mathrm{as}n\to \infty .$ □

Appendix A.2. Proof of Lemmas

Lemma A1. 

Let $\mathbf{X}\in {\mathbb{R}}^{n\times p}$ denote a fixed design matrix and $\mathit{\epsilon }\sim N(0,{\sigma }^2{\mathbf{I}}_n)$ represent an n-dimensional error vector, where ${\mathbf{I}}_n$ is the $n\times n$ identity matrix. For any $t>0$, we define $z=\sigma \sqrt{{\displaystyle \frac{t^2+2logp}{n}}}$. Under these settings, the following inequality holds

$$\mathbb{P}(\parallel n^{-1}{\mathbf{X}}^{\top }\mathit{\epsilon }{\parallel }_{\infty }\le z)\ge 1-2exp\{-t^2/2\}.$$

Proof. 

Given that $\mathit{\epsilon }\sim N(0,{\sigma }^2{\mathbf{I}}_n)$, for $t>0$ and $z=\sigma \sqrt{{\displaystyle \frac{t^2+2logp}{n}}}$, we have

$$1-\mathbb{P}(\parallel n^{-1}{\mathbf{X}}^{\top }{\mathit{\epsilon }\parallel }_{\infty }\le z)=\mathbb{P}(\parallel n^{-1}{\mathbf{X}}^{\top }\mathit{\epsilon }{\parallel }_{\infty }>z)=\mathbb{P}\left(\underset{1\le j\le p}{max}{\displaystyle \frac{|{\mathit{\epsilon }}^{\top }{\mathbf{X}}_j|}{n}}>z\right),$$

where ${\mathbf{X}}_j$ is the j-th column of $\mathbf{X}$. Using the union bound,

$$\mathbb{P}\left(\underset{1\le j\le p}{max}{\displaystyle \frac{|{\mathit{\epsilon }}^{\top }{\mathbf{X}}_j|}{n}}>z\right)\le \sum _{j=1}^p\mathbb{P}\left({\displaystyle \frac{|{\mathit{\epsilon }}^{\top }{\mathbf{X}}_j|}{\sqrt{n}\sigma }}>\sqrt{t^2+2logp}\right).$$

Since ${\displaystyle \frac{{\mathit{\epsilon }}^{\top }{\mathbf{X}}_j}{\sqrt{n}\sigma }}\sim N(0,1)$,

$$\mathbb{P}\left({\displaystyle \frac{|{\mathit{\epsilon }}^{\top }{\mathbf{X}}_j|}{\sqrt{n}\sigma }}>\sqrt{t^2+2logp}\right)=2\mathbb{P}\left({\displaystyle \frac{{\mathit{\epsilon }}^{\top }{\mathbf{X}}_j}{\sqrt{n}\sigma }}>\sqrt{t^2+2logp}\right).$$

Thus,

$$\mathbb{P}\left(\underset{1\le j\le p}{max}{\displaystyle \frac{|{\mathit{\epsilon }}^{\top }{\mathbf{X}}_j|}{n}}>z\right)\le 2pexp\left\{-\left({\displaystyle \frac{t^2}{2}}+logp\right)\right\}.$$

Simplifying,

$$\mathbb{P}(\parallel n^{-1}{\mathbf{X}}^{\top }\mathit{\epsilon }{\parallel }_{\infty }>z)\le 2exp\left(-{\displaystyle \frac{t^2}{2}}\right).$$

The proof of Lemma A1 is now complete.□

Lemma A2. 

For any $ϵ>0$, $\mathbb{P}(\parallel \tilde{\mathsf{\Sigma }}{-\mathsf{\Sigma }\parallel }_{max}\ge ϵ)\le p^2{max}_{i,j}\mathbb{P}(|{\widehat{\mathsf{\Sigma }}}_{i,j}-{\mathsf{\Sigma }}_{i,j}|\ge ϵ/2).$

Proof. 

From the inequality

$$\parallel \tilde{\mathsf{\Sigma }}{-\mathsf{\Sigma }\parallel }_{max}\le \parallel \tilde{\mathsf{\Sigma }}-\widehat{\mathsf{\Sigma }}{\parallel }_{max}+\parallel \widehat{\mathsf{\Sigma }}{-\mathsf{\Sigma }\parallel }_{max}\le 2{\parallel \widehat{\mathsf{\Sigma }}-\mathsf{\Sigma }\parallel }_{max},$$

it follows that $\mathbb{P}(\parallel \tilde{\mathsf{\Sigma }}{-\mathsf{\Sigma }\parallel }_{max}\ge ϵ)\le \mathbb{P}(\parallel \widehat{\mathsf{\Sigma }}-\mathsf{\Sigma }{\parallel }_{max}\ge ϵ/2).$ The result is obtained by applying the union bound over $\mathbb{P}(|{\widehat{\mathsf{\Sigma }}}_{i,j}-{\mathsf{\Sigma }}_{i,j}|\ge ϵ/2)$ for all $i,j$. □