Statistical Inference for High-Dimensional Heteroscedastic Partially Single-Index Models

Abstract

In this study, we propose a novel penalized empirical likelihood approach that simultaneously performs parameter estimation and variable selection in heteroscedastic partially linear single-index models with a diverging number of parameters. It is rigorously proved that the proposed method possesses the oracle property: (i) with probability tending to 1, the zero components are consistently estimated as zero; (ii) the estimators for nonzero coefficients achieve asymptotic efficiency. Furthermore, the penalized empirical log-likelihood ratio statistic is shown to asymptotically follow a standard chi-squared distribution under the null hypothesis. This methodology can be naturally applied to pure partially linear models and single-index models in high-dimensional settings. Simulation studies and real-world data analysis are conducted to examine the properties of the presented approach.

1. Introduction

Consider the following partially linear single-index model (PLSIM)

$$Y_i={\theta }^{\top }{X}_i+g\left(Z_i^{\top }\gamma \right)+{\epsilon }_i,E({\epsilon }_i|X_i,Z_i)=0i=1,\cdots ,n,$$

(1)

where $X_i\in {\mathbb{R}}^{\mathrm{p}}$ and $Z_i\in {\mathbb{R}}^{\mathrm{r}}$ are covariates, $g(·)$ denotes an unknown function, $Y_i$ is a response variable, $\theta \in {\mathbb{R}}^{\mathrm{p}}$ and $\gamma \in {\mathbb{R}}^{\mathrm{r}}$ are parameter vectors, and ${\epsilon }_i$ is an independent random error. Let $\mathrm{Var}({\epsilon }_i|X_i,Z_i)=v(X_i,Z_i)>0$, where the function $v(X,Z)$ expresses potential heteroscedasticity. For model (1), the scale of the index parameter $\gamma$ is generally not uniquely identifiable. That is, if the index parameter is multiplied by a nonzero constant while the nonparametric function is divided by the same constant, the model’s predictions remain unchanged. To ensure identifiability and avoid non-unique representations, it is standard practice to impose an identifiability constraint on the index parameter $\gamma$, for example, fixing one of its components to 1 or restricting it to have unit norm. We fix the first component of $\gamma$ to 1 and denote the remaining components of Z as $Z_{-1}$. Model (1) is a combination of a partially linear model (PLM) and single-index model (SIM). As far as we know, Carroll et al. [1] introduced PLSIMs and developed a backfitting algorithm for estimation in their generalized form. Since their introduction, they have attracted much research attention, and extensive research has focused on estimating $g(·)$ and the unknown parameter vectors $\theta$ and $\gamma$. For PLSIMs, Yu and Ruppert [2] showed a spline estimation approach via the penalty function; Zhu and Xue [3] proposed an empirical likelihood (EL) method; Xia and Härdle [4] introduced a semiparametric estimation procedure; Liang et al. [5] introduced an estimation method using profile least squares; etc.

In recent years, the analysis of high-dimensional data has evolved into a major frontier of statistical research, with applications spanning internet portals, hyperspectral imagery, financial applications, and high-throughput genomic data analysis within computational biology. See, e.g., Ma and Zhu [6], Fang et al. [7], Hao and Yin [8], Liu et al. [9], etc. Ma and Zhu [6] introduced efficient estimators for heteroscedastic PLSIMs allowing high-dimensional setting. While their methodology accommodated high-dimensional covariates, it restricted their dimensions to be fixed rather than allowing them to diverge as the sample size increases. Fang et al. [7] proposed EL estimators for high-dimensional PLSIMs, permitting the covariate dimension to diverge as the sample size increases.

In high-dimensional analyses, PLSIMs suffer from the inclusion of irrelevant covariates, leading to inefficient parameter estimators and reduced prediction accuracy. Pruning these non-informative variables from the sparse true model is thus a natural imperative. This has motivated the proliferation and rigorous study of various variable selection methods in contemporary statistics. Various variable selection methods have been investigated, including AIC and BIC (Breiman [10]), the LASSO penalty (Tibshirani [11]), the SCAD penalty (Fan and Li [12]), and so on. For PLSIMs, several variable selection methods were proposed. Xie and Huang [13] proposed a variable selection method for PLMs, which are special cases of PLSIMs, and proved that this method possesses the oracle property. Wang and Zhu [14] established nearly necessary and sufficient conditions for estimator consistency in SIMs under high-dimensional (“large p small n”) settings, which are special cases of PLSIMs. Zhang et al. [15] developed a method for variable selection and parameter estimation in high-dimensional PLSIMs. Lai et al. [16] studied variable selection for heteroscedastic PLSIMs by using the efficient score function. These methods, however, are largely constrained to scenarios where the covariate dimensions remain fixed.

Within nonparametric frameworks, Owen [17] developed the EL method for statistical inference. This approach retains likelihood methodology while eliminating parametric distributional assumptions. Since this method was proposed, it has been successfully extended to various circumstances, including linear models [18], generalized linear models [19], heteroscedastic PLMs [20], SIMs [21], and network data [22], among others. Chen et al. [23] demonstrated that EL remains valid when the data dimension diverges. In high-dimensional data settings, Tang and Leng [24] studied a variable selection method by penalized empirical likelihood (PEL) for linear regression models, and Leng and Tang [25] investigated a PEL method for general estimating equations. To our knowledge, applications to heteroscedastic PLSIMs have been scarcely explored, especially for variable selection in high-dimensional settings.

Empirical likelihood is a data-driven, nonparametric methodology that retains the merits of parametric likelihood while offering robustness and flexibility in incorporating auxiliary information to obtain estimates and construct confidence sets. Motivated by the PEL method for high-dimensional estimating equations in Tang and Leng [25], we aim to explore a variable selection approach using PEL for a PLSIM in a heteroscedastic high-dimensional situation, where dimensionality $p\to \infty$ and $r\to \infty$, as $n\to \infty$. For model (1), the PEL ratio is constructed using semiparametric efficient estimating equations, incorporating the semiparametric efficient score for the heteroscedastic PLSIM. We prove that PEL has the oracle property and excels at generating sparse models without requiring a prespecified parametric likelihood. Although existing variable selection techniques (e.g., Lai et al. [16]) also attain the oracle property, specifying a high-dimensional distribution remains theoretically challenging. Furthermore, the PEL ratio statistic satisfies Wilks’ theorem, converging to a chi-squared distribution under some regularity conditions, which facilitates hypothesis testing and produces range-respecting confidence regions. As a robust alternative to parametric likelihood ratios in high-dimensional settings, PEL combines the adaptive ability and statistical efficiency inherent in nonparametric likelihood methods, complementing the existing methods.

The rest of this article is organized as follows. Section 2 outlines methods of variable selection, parameter estimation, and asymptotic properties for high-dimensional heteroscedastic PLSIMs using PEL. Section 3 extends the method to PLMs and SIMs as special examples. In Section 4, we exhibit simulation results, and an application of the proposed method is stated in Section 5. Lemmas and technical proofs are shown in Appendix A.

2. Penalized Empirical Likelihood for PLSIM

Denote the weight function as $w(X_i,Z_i)={\left[E({\epsilon }_i^2\mid X_i,Z_i)\right]}^{-1}$, $i=1,\cdots ,n$. Using the kernel function $K_h\left(u\right)=h^{-1}K(u/h)$ with bandwidth $h\to 0$, the nonparametric estimators are defined as follows:

$$\begin{array}{ccc}\hfill \widehat{E}\{\widehat{w}(X,Z)|Z_i^{\top }\gamma \}& =& {\displaystyle \frac{{\sum }_{i\ne j}{K}_{h_3}(Z_i^{\top }\gamma -Z_j^{\top }\gamma )\widehat{w}(X_i,Z_i)}{{\sum }_{i\ne j}{K}_{h_3}(Z_i^{\top }\gamma -Z_j^{\top }\gamma )}},\hfill \\ \hfill \widehat{E}\{\widehat{w}(X,Z)Z_{-1,i}|Z_i^{\top }\gamma \}& =& {\displaystyle \frac{{\sum }_{i\ne j}{K}_{h_3}(Z_i^{\top }\gamma -Z_j^{\top }\gamma )\widehat{w}(X_i,Z_i)Z_{-1,i}}{{\sum }_{i\ne j}{K}_{h_3}(Z_i^{\top }\gamma -Z_j^{\top }\gamma )}},\hfill \\ \hfill \widehat{E}\{\widehat{w}(X,Z)X|Z_i^{\top }\gamma \}& =& {\displaystyle \frac{{\sum }_{i\ne j}{K}_{h_3}(Z_i^{\top }\gamma -Z_j^{\top }\gamma )\widehat{w}(X_i,Z_i)X_i}{{\sum }_{i\ne j}{K}_{h_3}(Z_i^{\top }\gamma -Z_j^{\top }\gamma )}},\hfill \end{array}$$

$$\widehat{w}(X_i,Z_i)=\sum _{i\ne j}{K}_{h_2}({\eta }_i-{\eta }_j)/\sum _{i\ne j}{K}_{h_2}({\eta }_i-{\eta }_j)e_i^2,$$

$$\widehat{g}\left(Z_i^{\top }\gamma \right)=\sum _{i\ne j}{K}_{h_1}(Z_i^{\top }\gamma -Z_j^{\top }\gamma )(Y_i-X_i^{\top }\theta )/\sum _{i\ne j}{K}_{h_1}(Z_i^{\top }\gamma -Z_j^{\top }\gamma )$$

and

$$\begin{array}{ccc}\hfill {\widehat{g}}^{\prime }\left(Z_i^{\top }\gamma \right)& =& h_1^{-1}\left\{\sum _{i\ne j}{K}_{h_1}^{\prime }(Z_i^{\top }\gamma -Z_j^{\top }\gamma )(Y_i-X_i^{\top }\theta )\sum _{i\ne j}{K}_{h_1}(Z_i^{\top }\gamma -Z_j^{\top }\gamma )\right\}\hfill \\ & -& \sum _{i\ne j}{K}_{h_1}(Z_i^{\top }\gamma -Z_j^{\top }\gamma )(Y_i-X_i^{\top }\theta )\hfill \\ & \times & \sum _{i\ne j}{K}_{h_1}^{\prime }(Z_i^{\top }\gamma -Z_j^{\top }\gamma )\}/{\left\{\sum _{i\ne j}{K}_{h_1}(Z_i^{\top }\gamma -Z_j^{\top }\gamma )\right\}}^2.\hfill \end{array}$$

We propose the following estimating equations for PLSIM:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\displaystyle \frac{1}{\sqrt{n}}}{\displaystyle \sum _{i=1}^n}{\tilde{\epsilon }}_i\widehat{w}(X_i,Z_i)[X_i-{\displaystyle \frac{\widehat{E}\{\widehat{w}(X,Z)X|Z_i^{\top }\gamma \}}{\widehat{E}\{\widehat{w}(X,Z)|Z_i^{\top }\gamma \}}}]=0,\hfill \\ {\displaystyle \frac{1}{\sqrt{n}}}{\displaystyle \sum _{i=1}^n}{\tilde{\epsilon }}_i\widehat{w}(X_i,Z_i){\widehat{g}}^{\prime }\left(Z_i^{\top }\gamma \right)[Z_{-1,i}-{\displaystyle \frac{\widehat{E}\{\widehat{w}(X,Z)Z_{-1}|Z_i^{\top }\gamma \}}{\widehat{E}\{\widehat{w}(X,Z)|Z_i^{\top }\gamma \}}}]=0,\hfill \end{array}\right.\end{array}$$

(2)

where ${\widehat{w}}_i\equiv \widehat{w}(X_i,Z_i)$ and ${\tilde{\epsilon }}_i=Y_i-{\theta }^{\top }{X}_i-\widehat{g}\left(Z_i^{\top }\gamma \right)$ denotes the residual term for $i=1,\cdots ,n$.

Attempting to estimate $\mathrm{var}(\epsilon \mid X,Z)$ or $w(X,Z)$ by applying nonparametric regression to the residuals and the covariates (X, Z) poses a significant challenge, as this constitutes a high-dimensional problem that is highly susceptible to the curse of dimensionality. To simplify the estimation of $w(X,Z)$, we assume there exists a function ${\eta }_i=\eta (X_i,Z_i)$ satisfying $\mathrm{var}({\epsilon }_i\mid X_i,Z_i)=\mathrm{var}({\epsilon }_i\mid {\eta }_i)$, where ${\eta }_i$ is a known low-dimensional function of the covariates $(X_i,Z_i)$, $i=1,\cdots ,n$. For instance, $\eta$ could take the form ${\theta }^{\top }X$, implying that the error variance depends solely on a linear combination of X. Alternatively, $\eta$ could be ${\gamma }^{\top }Z$, indicating dependence only on Z, or it could represent a combination of both or other structures. A similar assumption is also present in Ma and Zhu [6]. In practice, a reasonable approximation of $\eta$ can be obtained using standard procedures for modeling heteroscedasticity, based on residuals from a preliminary model fit. It should be noted, however, that this assumption can be relaxed to incorporate intermediate multivariate frameworks, such as additive structures, thereby preserving univariate convergence rates while maintaining considerable flexibility in variance modeling.

Define

$$S_{eff}=w\epsilon {\left(X^{\top }-{\displaystyle \frac{E(w{X}^{\top }|Z^{\top }\gamma )}{E(w|Z^{\top }\gamma )}},\left\{Z_{-1}^{\top }-{\displaystyle \frac{E(w{Z}_{-1}^{\top }|Z^{\top }\gamma )}{E(w|Z^{\top }\gamma )}}\right\}{g}^{\prime }\left(Z^{\top }\gamma \right)\right)}^{\top }.$$

(3)

According to Ma and Zhu [6], $S_{eff}$ is the semiparametric efficient score, and the estimator ${({\widehat{\theta }}^{\top },{\widehat{\gamma }}^{\top })}^{\top }$, based on (2), is doubly robust and efficient with fixed p and r. Double robustness means that a consistent estimator of the target parameter can be obtained as long as one of the two models is correctly specified. For example, we can use an estimator $\widehat{g}(·)$ that is inconsistent for $g(·)$; as long as the conditional expectation is consistently estimated, i.e., $\widehat{E}(·\mid Z^{\top }\gamma )$ converges to $E(·\mid Z^{\top }\gamma )$, then expression (3) will yield consistent estimators for $\theta$ and $\gamma$. The converse also holds: if $\widehat{g}(·)$ is consistent for $g(·)$, then consistency of the estimators in (3) is maintained even if the model for the conditional expectation is misspecified. However, the doubly robust and efficient property of the estimator achieved by solving (2) is no longer valid when p and r tend to infinity as $n\to \infty$.

Our goal is, under high-dimensional sparse setting, to develop new estimation and variable selection approaches for heteroscedastic model (1) by using the PEL method. In order to construct the PEL function, we need to propose an auxiliary random vector using $S_{eff}$. Define

$${\xi }_i(\theta ,\gamma )=w_i{\epsilon }_i{\left(X_i^{\top }-{\displaystyle \frac{E(w_i{X}_i^{\top }|Z_i^T\gamma )}{E(w_i|Z_i^{\top }\gamma )}},\left\{Z_{-1,i}^{\top }-{\displaystyle \frac{E(w_i{Z}_{-1,i}^{\top }|Z_i^{\top }\gamma )}{E(w_i|Z_i^{\top }\gamma )}}\right\}{g}^{\prime }\left(Z_i^{\top }\gamma \right)\right)}^{\top }.$$

We have $E\left\{{\xi }_i(\theta ,\gamma )\right\}=0$ for $i=1,\cdots ,n$. Let $q=(q_1\cdots q_n)$ satisfying ${\sum }_{i=1}^n{q}_i=1$, $q_i\ge 0$. For ${({\theta }^{\top },{\gamma }^{\top })}^{\top }$, the EL function is written as

$$L(\theta ,\gamma )=sup\{\prod _{i=1}^n\left(n{q}_i\right):\sum _{i=1}^n{q}_i=1,q_i\ge 0,\sum _{i=1}^n{q}_i{\xi }_i(\theta ,\gamma )=0\}.$$

(4)

Since $L(\theta ,\gamma )$ contains unknown functions, it cannot be directly used for statistical inference on ${({\theta }^{\top },{\gamma }^{\top })}^{\top }$. A natural approach to solving this issue is to substitute the unknown functions in $L(\theta ,\gamma )$ with their corresponding estimator provided above. For ${({\theta }^{\top },{\gamma }^{\top })}^{\top }$, redefine the estimated EL function as

$$\tilde{L}(\theta ,\gamma )=sup\{\prod _{i=1}^n\left(n{q}_i\right):\sum _{i=1}^n{q}_i=1,q_i\ge 0,\sum _{i=1}^n{q}_i{\widehat{\xi }}_i(\theta ,\gamma )=0\},$$

(5)

where ${\tilde{\epsilon }}_i=Y_i-{\theta }^{\top }{X}_i-\widehat{g}\left(Z_i^{\top }\gamma \right)$ and

$${\widehat{\xi }}_i(\theta ,\gamma )={\widehat{w}}_i{\tilde{\epsilon }}_i{\left(X_i^{\top }-{\displaystyle \frac{\widehat{E}({\widehat{w}}_i{X}_i^{\top }|Z_i^{\top }\gamma )}{\widehat{E}({\widehat{w}}_i|Z_i^{\top }\gamma )}},\left\{Z_{-1,i}^{\top }-{\displaystyle \frac{\widehat{E}({\widehat{w}}_i{Z}_{-1,i}^{\top }|Z_i^{\top }\gamma )}{\widehat{E}({\widehat{w}}_i|Z_i^{\top }\gamma )}}\right\}{\widehat{g}}^{\prime }\left(Z_i^{\top }\gamma \right)\right)}^{\top }.$$

Define the PEL estimator ${({\widehat{\theta }}^{\top },{\widehat{\gamma }}^{\top })}^{\top }$ as the maximizer of

$$log\left\{\tilde{L}(\theta ,\gamma )\right\}-n\sum _{i=1}^p{p}_{\tau }(|{\theta }_i|)-n\sum _{i=1}^r{p}_{\nu }(|{\gamma }_i|),$$

(6)

where $p_{\tau }\left(t\right)$ and $p_{\nu }\left(t\right)$ are the penalty functions with tuning parameters $\tau$ and $\nu$, respectively.

Many commonly used penalty functions have been studied, i.e., the $L_1$ penalty (Donoho and Johnstone [26]); $L_2$ penalty (Hoerl and Kennard [27]); LASSO penalty (Tibshirani [11]); and SCAD penalty (Fan and Li [12]). It is well known that the SCAD penalty has the oracle property. Therefore, in this article, we consider PEL for a heteroscedastic PLSIM by using the SCAD penalty. Its first derivative satisfies

$$p_{\nu }^{\prime }\left(t\right)=\nu \{{\displaystyle \frac{{(a\nu -t)}_{+}}{(a-1)\nu }}I(t>\nu )+I(t\le \nu )\},$$

where $I(·)$ denotes an indicator function and a is a constant with $a>2$.

Combining the Lagrange multiplier method and Equation (5), we have

$$q_i={\displaystyle \frac{1}{n}}{\displaystyle \frac{1}{1+{\lambda }^{\top }{\widehat{\xi }}_i(\theta ,\gamma )}},$$

(7)

and $\lambda$ satisfies

$${\displaystyle \frac{1}{n}}\sum _{i=1}^n{\displaystyle \frac{{\widehat{\xi }}_i(\theta ,\gamma )}{1+{\lambda }^{\top }{\widehat{\xi }}_i(\theta ,\gamma )}}=0.$$

(8)

By substituting Equation (7) into $\tilde{L}(\theta ,\gamma )$, we can show that maximizing (6) corresponds to minimizing

$${\tilde{\ell }}_p(\theta ,\gamma )=2\sum _{i=1}^{n}log\{1+{\lambda }^{\top }{\widehat{\xi }}_i(\theta ,\gamma )\}+n\sum _{i=1}^p{p}_{\tau }(|{\theta }_i|)+n\sum _{i=1}^r{p}_{\nu }(|{\gamma }_i|).$$

(9)

Therefore, ${({\widehat{\theta }}^T,{\widehat{\gamma }}^{\top })}^{\top }$ can also be defined to be the minimization of (9).

Let ${\mathcal{A}}_1=\{j:{\theta }_{0j}\ne 0\}$ and ${\mathcal{A}}_2=\{j:{\gamma }_{0j}\ne 0\}$, and denote the cardinalities of ${\mathcal{A}}_1$ and ${\mathcal{A}}_2$ as $d_1$ and $d_2$, where ${\theta }_0$ and ${\gamma }_0$ are the true values of $\theta$ and $\gamma$ respectively. Without loss of generality, we write $\theta ={({\theta }_1^{\top },{\theta }_2^{\top })}^{\top }$, where ${\theta }_1\in {\mathbb{R}}^{{\mathrm{d}}_1}$ and ${\theta }_2\in {\mathbb{R}}^{\mathrm{p}-{\mathrm{d}}_1}$ represent $\theta$’s nonzero and zero components, respectively. $\gamma ={({\gamma }_1^{\top },{\gamma }_2^{\top })}^{\top }$ can be similarly partitioned, where ${\gamma }_1\in {\mathbb{R}}^{{\mathrm{d}}_2}$ and ${\gamma }_2\in {\mathbb{R}}^{\mathrm{r}-{\mathrm{d}}_2}$. Analogously, the true parameter values ${\theta }_0$ and ${\gamma }_0$ can be decomposed as ${\theta }_0={({\theta }_{10}^{\top },0)}^{\top }$ and ${\gamma }_0={({\gamma }_{10}^{\top },0)}^{\mathrm{T}}$. For notational purposes, let $I_p={(H_1^{\top },H_2^{\top })}^{\top }$ and $I_{r-1}={(H_3^{\top },H_4^{\top })}^{\top }$, where $H_1\in {\mathbb{R}}^{d_1\times p}$, $H_2\in {\mathbb{R}}^{(p-d_1)\times p}$, $H_3\in {\mathbb{R}}^{d_2\times (r-1)}$, and $H_4\in {\mathbb{R}}^{(r-1-d_2)\times (r-1)}$.

To derive asymptotic properties of the proposed PEL estimator, the following conditions are necessary.

Condition 1. $K_h(·)$ is symmetric with $K_h^{\prime }(·)$ continuous on $[-1,1]$.

Condition 2. The bandwidth $h_i$, for $i=1,2,3$, satisfies the following asymptotic assumption: (1) $n{h}_1^8\to 0$, $n{h}_1^4\to \infty$, $h_2=O(1/\sqrt[5]{n})$ and $h_3=O(1/\sqrt[5]{n})$; (2) ${log}^2\left(n\right)/\left(n{h}_i\right)\to 0$, ${log}^4\left(n\right)/\left(n{h}_1{h}_i\right)\to 0$ and $h_1^4{log}^2\left(n\right)/h_i\to 0$.

Condition 3. Let $\mathrm{Var}\left(X_i\right)={\Sigma }_{xi}$ and $\mathrm{Var}\left(Z_i\right)={\Sigma }_{zi}$, $i=1\cdots n$. For ${\Sigma }_{xi}$ and ${\Sigma }_{zi}$, their eigenvalues satisfy $C_1\le {\Gamma }_1\left({\Sigma }_{xi}\right)\le \cdots \le {\Gamma }_p\left({\Sigma }_{xi}\right)\le C_2$ and $C_1\le {\Gamma }_1\left({\Sigma }_{zi}\right)\le \cdots \le {\Gamma }_r\left({\Sigma }_{zi}\right)\le C_2$ for some constants $0<C_1<C_2$, $i=1\cdots n$. In addition, $E({\epsilon }^{4+\delta }|X,Z)<\infty$, where $\delta$ is a positive constant.

Condition 4. Let $v(·)$ and $\eta =\eta (X,Z)$ satisfy $E({\epsilon }^2|X,Z)=v\left(\eta \right)$, where $0<C_1<v(·)<C_2<\infty$, and $C_1$ and $C_2$ are positive constants. Moreover, $\mathrm{Var}(X_i|\eta (X_i,Z_i))$ is positive definite with a bounded spectrum.

Condition 5. There exist $v_1(X,Z)$ values satisfying

$${\displaystyle \frac{{\partial }^{2}E(X|Z^{\top }\gamma )}{\partial {\gamma }_i\partial {\gamma }_j}},{\displaystyle \frac{{\partial }^{2}E(Z|Z^{\top }\gamma )}{\partial {\gamma }_i\partial {\gamma }_j}},{\displaystyle \frac{{\partial }^{2}E(w|Z^{\top }\gamma )}{\partial {\gamma }_i\partial {\gamma }_j}},{\displaystyle \frac{{\partial }^{2}E(wZ|Z^{\top }\gamma )}{\partial {\gamma }_i\partial {\gamma }_j}},$$

$${\displaystyle \frac{{\partial }^{2}E(wX|Z^{\top }\gamma )}{\partial {\gamma }_i\partial {\gamma }_j}}<v_1(X,Z),E{v}_1^2<\infty ,(i,j=1,\cdots ,p).$$

There also exist $v_2(X,Z)$ values such that

$${\displaystyle \frac{{\partial }^3\eta (X,Z)}{\partial {\pi }_i\partial {\pi }_j\partial {\pi }_k}}<v_2(X,Z),E{v}_2^2<\infty ,$$

where ${(X^{\top },Z^{\top })}^{\top }={({\pi }_1\cdots {\pi }_{p+r})}^{\top }$ and $i,j,k=1,\cdots ,p+r$. There exist $v_3(X,Z)$ values satisfying

$${\displaystyle \frac{{\partial }^{4}g\left(Z^{\top }\gamma \right)}{\partial {\gamma }_i\partial {\gamma }_j\partial {\gamma }_k\partial {\gamma }_l}},{\displaystyle \frac{{\partial }^{4}v\left(\eta \right)}{\partial {\eta }_{i_1}\partial {\eta }_{j_1}\partial {\eta }_{k_1}\partial {\eta }_{l_1}}}<v_3(X,Z),E{v}_3^2<\infty ,$$

where $\eta \in R^{p_1}$, $i,j,k,l=1,\cdots ,p$ and $i_1,j_1,k_1,l_1=1\cdots p_1$.

Condition 6. Assume $\eta$ and $Z^{\top }\gamma$ possess densities, denoted by $f_{\eta }\left(\eta \right)$ and $f_{Z^{\top }\gamma }\left(Z^{\top }\gamma \right)$, respectively, which are bounded away from zero and infinity. There exist $v_4(X,Z)$ values such that

$${\displaystyle \frac{{\partial }^2{f}_{Z^{\top }\gamma }\left(Z^{\top }\gamma \right))}{\partial {\gamma }_i\partial {\gamma }_j}},{\displaystyle \frac{{\partial }^2{f}_{\eta }\left(\eta \right)}{\partial {\eta }_k\partial {\eta }_l}}<v_4(X,Z),E{v}_4^2<\infty ,(i,j=1\cdots p;k,l=1\cdots p_1).$$

Condition 7. As $n\to \infty$, we assume $p\to \infty$ and $p/\sqrt[5]{n}\to 0$, and $r\to \infty$ and $r/\sqrt[5]{n}\to 0$.

Condition 8. All random elements X, $\epsilon$, $\epsilon X$, and $\epsilon Z$ have finite fourth moments.

Condition 9. Define

$${\xi }_n(\theta ,\gamma )=w\epsilon {\left(X^{\top }-{\displaystyle \frac{E(w{X}^{\top }|Z^{\top }\gamma )}{E(w|Z_i^{\top }\gamma )}},\left\{Z^{\top }-{\displaystyle \frac{E(w{Z}^{\top }|Z^{\top }\gamma )}{E(w|Z^{\top }\gamma )}}\right\}{g}^{\prime }\left(Z^{\top }\gamma \right)\right)}^{\top }.$$

As $n\to \infty$, we assume the following moments are uniformly bounded by a positive constant C: $E(\parallel {\xi }_n(\theta ,\gamma )/\sqrt{p}{\parallel }^4)<C$, $E(\parallel Z{X}^{\top }{\parallel }^4)<C$, $E(\parallel X{X}^{\top }{\parallel }^4)<C$. Furthermore, we assume $E(\parallel X{Z}^{\top }{\parallel }^4)<\infty$.

Condition 10. As $n\to \infty$, $\tau {(p/n)}^{{\displaystyle \frac{1}{2}}}\to \infty$, $\nu {(r/n)}^{1/2}\to \infty$, $\underset{j\in {\mathcal{A}}_1}{min}{\theta }_{0j}/\tau \to \infty$, and $\underset{j\in {\mathcal{A}}_2}{min}{\gamma }_{0j}/\nu \to \infty$.

Condition 11. Assume $\underset{j\in {\mathcal{A}}_1}{max}{P}_{\tau }^{\prime }(|{\theta }_{0j}|)=o\left\{{\left(np\right)}^{-1/2}\right\}$, $\underset{j\in {\mathcal{A}}_2}{max}{P}_{\nu }^{\prime }(|{\gamma }_{0j}|)=o\left\{{\left(nr\right)}^{-1/2}\right\}$, $\underset{j\in {\mathcal{A}}_1}{max}{P}_{\tau }^{″}(|{\theta }_{0j}|)=o\left\{{\left(p\right)}^{-1/2}\right\}$, and $\underset{j\in {\mathcal{A}}_2}{max}{P}_{\nu }^{″}(|{\gamma }_{0j}|)=o\left\{{\left(r\right)}^{-1/2}\right\}$.

Conditions 1–6 support the existence of estimators ${({\widehat{\theta }}^T,{\widehat{\gamma }}^{\top })}^{\top }$. These conditions also ensure that the functions $w(X,Z)$, $g\left(Z^{\top }\gamma \right)$, and $g^{\prime }\left(Z^{\top }\gamma \right)$ and the conditional expectations $E\{\widehat{w}(X,Z)Z_{-1,i}|Z_i^{\top }\gamma \}$, $E\{\widehat{w}(X,Z)X|Z_i^{\top }\gamma \}$, and $E\{\widehat{w}(X,Z)|Z_i^{\top }\gamma \}$ can be estimated with maintained precision. Moreover, these conditions also guarantee that nonparametric estimation does not alter the asymptotic behavior of the empirical likelihood ratio. As a result, the estimated PEL ratio $\tilde{L}(\theta ,\gamma )$ converges to the same asymptotic distribution as the standard PEL ratio $L(\theta ,\gamma )$. Conditions 1–6 were also used by Ma and Zhu [6] as sufficient conditions for the double-robustness property of the estimators. Condition 7 serves as a technical requirement. Since determining the minimum upper bound is quite challenging, this condition is necessary, and the resulting bounds in the stochastic analysis are conservative. Condition 8 guarantees that the asymptotic variance exists for the estimator of the increasing-dimensional parameters ${({\theta }^T,{\gamma }^{\top })}^{\top }$. Condition 9 restricts the tail probability behavior of the estimating equation. Condition 10 requires that the weakest signal remains stronger than the penalty parameter, and Condition 11 helps limit the influence of the penalty on the nonzero components. Conditions 10 and 11 are satisfied by a range of penalty functions, including those discussed in Fan and Li [12]. Due to the considerable theoretical challenges in establishing asymptotic properties for PEL methods in the context of diverging covariate dimensions, these conditions are intentionally stringent rather than minimally sufficient, and the resulting stochastic bounds are conservative.

In the following theorem, we will show the theoretical properties of the PEL estimator ${({\widehat{\theta }}^{\top },{\widehat{\gamma }}^{\top })}^{\top }$.

Theorem 1.

As $n\to \infty$, under Conditions 1–11, we have

(1) 

${\widehat{\theta }}_2=0$ and ${\widehat{\gamma }}_2=0$, with probability tending to 1;

(2) 

$\sqrt{n}B{I}_B^{-1/2}\{{({\widehat{\theta }}_1^{\top },{\widehat{\gamma }}_1^{\top })}^{\top }-{({\theta }_{10}^{\top },{\gamma }_{10}^{\top })}^{\top }\}\stackrel{L}{\to }N(0,G)$, where $B\in {\mathbb{R}}^{({\mathrm{q}}_1+{\mathrm{q}}_2)\times (\mathrm{p}+\mathrm{r}-1)}$, $B{B}^{\top }\to G$ and G is a $(q_1+q_2)\times (q_1+q_2)$ matrix with fixed $q_1$ and $q_2$,

$$\begin{array}{c}\hfill H_0=\left(\begin{array}{cc}{H}_1& 0\\ 0& H_3\end{array}\right), H=\left(\begin{array}{cc}{H}_2& 0\\ 0& H_4\end{array}\right),\end{array}$$

$$\begin{array}{c}\hfill B=\left(\begin{array}{cc}{B}_1& 0\\ 0& B_2\end{array}\right),V=\left(\begin{array}{cc}{V}_{11}& V_{12}\\ V_{21}& V_{22}\end{array}\right), U=\left(\begin{array}{cc}{U}_{11}& U_{12}\\ U_{21}& U_{22}\end{array}\right)\end{array}$$

$$V_{11}=E\left\{wX{X}^T-{\displaystyle \frac{E(wX|Z^T\gamma )E(w{X}^T|Z^T\gamma )}{E(w|Z^T\gamma )}}\right\},$$

$$V_{12}=E\left[g^{\prime }\left(Z^T\gamma \right)\left\{wX{Z}_{-1}^T-{\displaystyle \frac{E(wX|Z^T\gamma )E(w{Z}_{-1}^T|Z^T\gamma )}{E(w|Z^T\gamma )}}\right\}\right],$$

$$V_{21}=E\left[g^{\prime }\left(Z^T\gamma \right)\left\{w{Z}_{-1}{X}^T-{\displaystyle \frac{E(w{Z}_{-1}|Z^T\gamma )E(w{X}^T|Z^T\gamma )}{E(w|Z^T\gamma )}}\right\}\right],$$

$$V_{22}=E\left[g^{\prime }{\left(Z^T\gamma \right)}^2\left\{w{Z}_{-1}{Z}_{-1}^T-{\displaystyle \frac{E(w{Z}_{-1}|Z^T\gamma )E(w{Z}_{-1}^T|Z^T\gamma )}{E(w|Z^T\gamma )}}\right\}\right],$$

$$U_{11}=E\left\{{\displaystyle \frac{\partial {\xi }_i(\theta ,\gamma )}{\partial {\theta }^{\top }}}{V}^{-1}{\displaystyle \frac{\partial {\xi }_i(\theta ,\gamma )}{\partial \theta }}\right\},U_{12}=E\left\{{\displaystyle \frac{\partial {\xi }_i(\theta ,\gamma )}{\partial {\theta }^{\top }}}{V}^{-1}{\displaystyle \frac{\partial {\xi }_i(\theta ,\gamma )}{\partial \gamma }}\right\},$$

$$U_{21}=E\left\{{\displaystyle \frac{\partial {\xi }_i(\theta ,\gamma )}{\partial {\gamma }^{\top }}}{V}^{-1}{\displaystyle \frac{\partial {\xi }_i(\theta ,\gamma )}{\partial \theta }}\right\},U_{12}=E\left\{{\displaystyle \frac{\partial {\xi }_i(\theta ,\gamma )}{\partial {\gamma }^{\top }}}{V}^{-1}{\displaystyle \frac{\partial {\xi }_i(\theta ,\gamma )}{\partial \gamma }}\right\},$$

$I_B=H_0{U}^{-1}V{U}^{-1}{H}_0^{\top }-H_0{U}^{-1}{H}^{\top }{\left(H{U}^{-1}{H}^{\top }\right)}^{-1}{H}_2{U}^{-1}V{A}^{-1}{H}_2^{\top }{\left(H{U}^{-1}{H}^{\top }\right)}^{-1}H{U}^{-1}{H}_0^{\top }$, and $\stackrel{L}{\to }$ denotes convergence in distribution.

In Theorem 1, B projects the diverging dimensional parameter vector ${({\theta }_1^{\top },{\gamma }_1^{\top })}^{\top }$ onto a fixed $(q_1+q_2)$-dimensional subspace.

Remark 1.

Theorem 1 proves that the proposed estimator satisfies the oracle property. Specifically, the components of ${\theta }_{20}$ and ${\gamma }_{20}$ are estimated as zero, and the PEL estimator of the nonzero components ${\theta }_{10}$ and ${\gamma }_{10}$ is efficient, with probability tending to 1.

Next, we will describe the construction of confidence regions and hypothesis testing for ${({\theta }^{\top },{\gamma }^{\top })}^{\top }$ using the PEL method. Consider testing the following null and alternative hypotheses:

$$H_0:L_n{({\theta }_0^{\top },{\gamma }_0^{\top })}^{\top }=0\mathrm{vs}.H_1:L_n{({\theta }_0^{\top },{\gamma }_0^{\top })}^{\top }\ne 0,$$

where $L_n\in {\mathbb{R}}^{({\mathrm{q}}_1+{\mathrm{q}}_2)\times (\mathrm{p}+\mathrm{r}-1)}$ satisfies that, for the fixed $q_1$ and $q_2$,

$$\begin{array}{c}\hfill L_n=\left(\begin{array}{cc}{L}_{n1}& 0\\ 0& L_{n2}\end{array}\right),L_n{L}_n^{\top }=\left(\begin{array}{cc}{I}_{q_1}& 0\\ 0& I_{q_2}\end{array}\right),\end{array}$$

$L_{n1}\in {\mathbb{R}}^{{\mathrm{q}}_1\times \mathrm{p}}$, $L_{n2}\in {\mathbb{R}}^{{\mathrm{q}}_2\times \mathrm{r}-1}$, and $I_{q_1}$ and $I_{q_2}$ are the $q_1$-dimensional and $q_2$-dimensional identity matrixes, respectively. Not only can we use this type of hypothesis to test the hypothesis for individual and multiple components of ${({\theta }_0^{\top },{\gamma }_0^{\top })}^{\top }$, but we can also use it to test the hypotheses about linear functions of ${({\theta }_0^{\top },{\gamma }_0^{\top })}^{\top }$.

Similar to the EL ratio for the PLSIM in [3], we can construct the PEL ratio statistic as

$$\tilde{\ell }\left(L_n\right)=-\{{\tilde{\ell }}_p(\widehat{\theta },\widehat{\gamma })-\underset{(\theta ,\gamma ):L_n{({\theta }^{\top },{\gamma }^{\top })}^{\top }=0}{min}{\tilde{\ell }}_p(\theta ,\gamma )\}.$$

(10)

The following theorem shows the properties of the PEL ratio statistic for model (1).

Theorem 2.

As $n\to \infty$, under the null hypothesis and Conditions 1–11, we have

$$\tilde{\ell }\left(L_n\right)\stackrel{L}{\to }{\chi }_{q_1+q_2}^2.$$

The standard PEL ratio, under some regular conditions, converges in law to a chi-square distribution. This is one of the most important properties of the PEL method, and similar conclusions can be found in Fang et al. [7] and Tang and Leng [24], among others. Theorem 2 shows that, under Conditions 1–11, the estimated PEL ratio $\tilde{L}(\theta ,\gamma )$ converges to the same asymptotic distribution as the standard PEL ratio $L(\theta ,\gamma )$. This result provides a convenient approach for testing hypotheses and constructing data-driven confidence regions without any shape constraints. Combined with the oracle property of the PEL method established in Theorem 1, these findings demonstrate the robustness and efficiency of the PEL method for PLSIMs.

Confidence regions for ${({\theta }^{\top },{\gamma }^{\top })}^{\top }$ can be constructed using Theorem 2; that is,

$${\mathcal{I}}_{\alpha }=\{{({\theta }^{\top },{\gamma }^{\top })}^{\top }:-\{{\tilde{\ell }}_p(\widehat{\theta },\widehat{\gamma })-\underset{(\theta ,\gamma ):L_n{({\theta }^{\top },{\gamma }^{\top })}^{\top }=0}{min}{\tilde{\ell }}_p(\theta ,\gamma )\}\le {\chi }_{q_1+q_2,(1-\alpha )}^2\},$$

(11)

where ${\chi }_{q_1+q_2,(1-\alpha )}^2$ is the $1-\alpha$ quantile of the ${\chi }_{q_1+q_2}^2$ distribution with $q_1+q_2$ degrees of freedom. Here ${\mathcal{I}}_{\alpha }$ provides an asymptotic confidence region for ${({\theta }^{\top },{\gamma }^{\top })}^{\top }$ with confidence level $1-\alpha$, i.e., as $n\to \infty$, $P(L_n(\theta ,\gamma )\in {\mathcal{I}}_{\alpha })\to 1-\alpha$.

Remark 2.

In this article, we develop a PEL method for simultaneous variable selection and parameter estimation in high-dimensional sparse heteroscedastic PLSIMs, and this requires $n>p$ and $n>r$. When this is violated in practice, one can first adopt SIS, proposed by Fan and Lv [28], to reduce the dimensionality to a moderate level below sample size.

3. Penalized Empirical Likelihood for PLM and SIM

For two special cases of model (1), we develop PEL estimators for parameters $\theta$ and $\gamma$. If $\gamma =1$, model (1) reduces to a heteroscedastic PLM, and it can be written as

$$Y_i={\theta }^{\top }{X}_i+g\left(Z_i\right)+{\epsilon }_i,fori=1,\cdots ,n.$$

(12)

Consider the PEL method for the high-dimensional PLM. Redefine the EL function for $\theta$ and ${\widehat{\xi }}_{1i}\left(\theta \right)$ as

$$\tilde{L}\left(\theta \right)=sup\{\prod _{i=1}^n\left(n{q}_i\right):\sum _{i=1}^n{q}_i=1,q_i\ge 0,\sum _{i=1}^n{q}_i{\widehat{\xi }}_{1i}\left(\theta \right)=0\},$$

(13)

and

$${\widehat{\xi }}_{1i}\left(\theta \right)={\widehat{w}}_i\left\{Y_i-X_i^T\theta -\widehat{g}\left(Z_i\right)\right\}\left\{X_i-{\displaystyle \frac{\widehat{E}({\widehat{w}}_i{X}_i|Z_i)}{\widehat{E}({\widehat{w}}_i|Z_i)}}\right\},i=1,\cdots ,n.$$

(14)

The PEL function for the model (12) can be written as

$${\tilde{\ell }}_p\left(\theta \right)=2\sum _{i=1}^{n}log\{1+{\lambda }^{\top }{\widehat{\xi }}_{1i}\left(\theta \right)\}+n\sum _{i=1}^p{p}_{\tau }(|{\theta }_i|).$$

(15)

We state the similar results as follows.

Corollary 1.

As $n\to \infty$, under Conditions 1–11, we have

(1) 

${\widehat{\theta }}_2=0$, with probability tending to 1;

(2) 

$\sqrt{n}{B}_1{I}^{-1/2}\{({\widehat{\theta }}_1-\left({\theta }_{10}\right)\}\stackrel{L}{\to }N(0,G)$, where $B_1\in {\mathbb{R}}^{{\mathrm{q}}_1\times \mathrm{p}}$, $B_1{B}_1^{\top }\to G_1$, $G_1$ is a $q_1\times q_1$ matrix with fixed $q_1$ and $\stackrel{L}{\to }$ stands for convergence in distribution.

Consider testing the following null and alternative hypotheses:

$$H_0:L_{n1}{\theta }_0=0\mathrm{vs}.H_1:L_{n1}{\theta }_0\ne 0,$$

where $L_{n1}\in {\mathbb{R}}^{{\mathrm{q}}_1\times \mathrm{p}}$ satisfies that, for the fixed $q_1$, $L_{n1}{L}_{n1}^{\top }=I_{q_1}$, where $I_{q_1}$ is a $q_1$-dimensional identity matrix. The PEL ratio statistic can be constructed as follows:

$$\tilde{\ell }\left(L_{n1}\right)=-\{{\tilde{\ell }}_p\left(\widehat{\theta }\right)-\underset{\theta :L_{n1}\theta =0}{min}{\tilde{\ell }}_p\left(\theta \right)\}.$$

(16)

Corollary 2.

As $n\to \infty$, under the null hypothesis and Conditions 1–11, we have

$$\tilde{\ell }\left(L_{n1}\right)\stackrel{L}{\to }{\chi }_{q_1}^2.$$

Next, we consider the following SIM with a diverging number of parameters, which is another special case of model (1).

$$Y_i=g\left(Z_i^{\top }\gamma \right)+{\epsilon }_i,E({\epsilon }_i|X_i,Z_i)=0i=1,\cdots ,n.$$

(17)

Redefine ${\widehat{\xi }}_{2i}\left(\gamma \right)$ as

$${\widehat{\xi }}_{2i}\left(\gamma \right)={\widehat{w}}_i(Y_i-\widehat{g}\left(Z_i^{\top }\gamma \right))\left\{Z_{-1,i}-{\displaystyle \frac{\widehat{E}({\widehat{w}}_i{Z}_{-1,i}|Z_i^{\top }\gamma )}{\widehat{E}({\widehat{w}}_i|Z_i^{\top }\gamma )}}\right\}{\widehat{g}}^{\prime }\left(Z_i^{\top }\gamma \right),i=1,\cdots ,n,$$

and rewrite the PEL ratio (9) as

$${\tilde{\ell }}_p\left(\gamma \right)=2\sum _{i=1}^{n}log\{1+{\lambda }^{\top }{\widehat{\xi }}_{2i}\left(\gamma \right)\}+n\sum _{i=1}^r{p}_{\nu }(|{\gamma }_i|).$$

(18)

Corollary 3.

As $n\to \infty$, nnder Conditions 1–11, we have

(1) 

${\widehat{\gamma }}_2=0$, with probability tending to 1;

(2) 

$\sqrt{n}{B}_2{I}^{-1/2}\{({\widehat{\gamma }}_1-\left({\gamma }_{10}\right)\}\stackrel{L}{\to }N(0,G)$, where $B_2\in {\mathbb{R}}^{{\mathrm{q}}_2\times \mathrm{r}-1}$, $B_2{B}_2^{\top }\to G_2$, and $G_2$ is a $q_2\times q_2$ matrix with fixed $q_2$.

Consider testing the following null and alternative hypotheses:

$$H_0:L_{n2}{\gamma }_0=0\mathrm{vs}.H_1:L_{n2}{\gamma }_0\ne 0,$$

where $L_{n2}\in {\mathbb{R}}^{{\mathrm{q}}_2\times \mathrm{r}}$ satisfies that, for the fixed $q_2$, $L_{n2}{L}_{n2}^{\top }=I_{q_2}$ and $I_{q_2}$ is a $q_2$-dimensional identity matrix. The PEL ratio statistic for $\gamma$ can be written as follows:

$$\tilde{\ell }\left(L_{n2}\right)=-\{{\tilde{\ell }}_p\left(\widehat{\gamma }\right)-\underset{\gamma :L_{n2}\gamma =0}{min}{\tilde{\ell }}_p\left(\gamma \right)\}.$$

(19)

Corollary 4.

As $n\to \infty$, under the null hypothesis and Conditions 1–11, we have

$$\tilde{\ell }\left(L_{n2}\right)\stackrel{L}{\to }{\chi }_{q_2}^2.$$

4. Simulations

First, we describe how to solve the optimization problems introduced by the PEL. The minimizer of the PEL ratio is obtained through the local quadratic approximation algorithm. For the PEL estimator, we minimize (9) using a nested optimization algorithm. The following steps outline the nested algorithm for calculating the PEL estimator by minimizing Equation (9).

• Step 1: We use the estimation procedure (a relatively simple but inefficient estimation method) described in Section 2 of Ma and Zhu [6] to obtain an initial estimator ${({\theta }^{0\top },{\gamma }^{0\top })}^{\top }$.
• Step 2: Obtain $\widehat{g}\left(Z_i^{\top }\gamma \right)$, ${\widehat{g}}^{\prime }\left(Z_i^{\top }\gamma \right)$, $\widehat{w}(X_i,Z_i)$, $\widehat{E}\{\widehat{w}(X,Z)|Z_i^{\top }\gamma \}$, $\widehat{E}\{\widehat{w}(X,Z)X|Z_i^{\top }\gamma \}$, and $\widehat{E}\{\widehat{w}(X,Z)Z_{-1}|Z_i^{\top }\gamma \}$ described above using fixed values of ${({\theta }^{\top },{\gamma }^{\top })}^{\top }$.
• Step 3: Obtain the auxiliary random vector ${\widehat{\xi }}_i(\theta ,\gamma )$.
• Step 4: Use Newton’s method to minimize (9) with respect to $\lambda$ for fixed values of ${({\theta }^{\top },{\gamma }^{\top })}^{\top }$.
• Step 5: Use the local quadratic approximation algorithm to minimize (9) with respect to ${({\theta }^{\top },{\gamma }^{\top })}^{\top }$ for fixed values of $\lambda$ obtained from Step 4.
• Step 6: Iterate Steps 4 and 5 until ${({\theta }^{\top },{\gamma }^{\top })}^{\top }$ converges.

Assume that ${({\theta }^{0\top },{\gamma }^{0\top })}^{\top }$ is an initial value of ${({\theta }^{\top },{\gamma }^{\top })}^{\top }$, and ${\theta }_j^{\left(k\right)}$ and ${\gamma }_l^{\left(k\right)}$ are the k-th step estimators of ${\theta }_j$ and ${\gamma }_l$, respectively. When ${\theta }_j^{\left(k\right)}$ ($|{\theta }_j^{\left(k\right)}|<\varsigma$) or ${\gamma }_l^{\left(k\right)}$ (where $|{\gamma }_l^{\left(k\right)}| <\varsigma$) are very close to 0, we set ${\widehat{\theta }}_j^{\left(k\right)}=0$ or ${\widehat{\gamma }}_l^{\left(k\right)}=0$, where $\varsigma$ is a predefined small positive tolerance. If ${\theta }_j^{\left(k\right)}\ne 0$, $p_{\tau }(|{\theta }_j|)$ can be locally approximated by $p_{\tau }(|{\theta }_j^{\left(k\right)}|)+{\displaystyle \frac{1}{2}}\{p_{\tau }^{\prime }(|{\theta }_j^{\left(k\right)}|)/|{\theta }_j^{\left(k\right)}|\}\{{\theta }_j^2-{\left({\theta }_j^{\left(k\right)}\right)}^2\}$. Similarly, we can use $p_{\nu }(|{\gamma }_l^{\left(k\right)}|)+{\displaystyle \frac{1}{2}}\{p_{\nu }^{\prime }(|{\gamma }_l^{\left(k\right)}|)/|{\gamma }_l^{\left(k\right)}|\}\{{\gamma }_l^2-{\left({\gamma }_l^{\left(k\right)}\right)}^2\}$ to approximate $p_{\nu }(|{\gamma }_l|)$ when ${\gamma }_l^{\left(k\right)}\ne 0$. These procedures are repeated until $\parallel {({\theta }^{(j+1)\top },{\gamma }^{(j+1)\top })}^{\top }-{({\theta }^{j\top },{\gamma }^{j\top })}^{\top }\parallel <{\varsigma }_1$, where ${\varsigma }_1$ is a very small positive number.

Next, we present simulation studies to illustrate the properties of the PEL inference for a heteroscedastic PLSIM.

Example 1.

For the PLSIM, we generated $X_1$ from a Poisson distribution with parameter 2, $X_p$ from a binomial distribution with a success probability 0.6, $X_j$ from the uniform distribution $U(0,1)$ for $j=2,\cdots ,(p-1)$, and $Z_k$ from the normal distribution with mean 0 and variance 1. Using ${(X^{\top },Z^{\top })}^{\top }$, we generated responses from $Y\sim N(X^T\theta +exp\left(Z^T\gamma \right),\mathit{Var}\left(Y\right)=|Z^T\gamma |)$. Let $\theta ={(2,\cdots ,1,0)}^{\top }$ and $\gamma ={(1,1.5,-2,\cdots ,0)}^{\top }$. We consider dimensions $p=10,20$ and $r=10,20$, and sample sizes $n=50,100$, and 200, respectively. We applied the cross-validation method to select the penalty parameters τ and ν. In order to compare the influence of the kernel function, we consider using the cosine kernel, defined as ${\displaystyle \frac{\pi }{4}}cos\left({\displaystyle \frac{\pi t}{2}}\right)·\mathbf{I}(t\le 1)$, and the Epanechnikov kernel, given by $K\left(t\right)={\displaystyle \frac{3}{4}}{(1-t^2)}_{+}$, respectively. In accordance with Condition 2, the bandwidth was set to $n^{-1/5}$, resulting in $h\approx 0.45$ at $n=50$, $h\approx 0.4$ at $n=100$, and $h\approx 0.35$ at $n=200$. Furthermore, to examine the robustness of the bandwidth selection, a grid search algorithm was also employed. For each of these settings, we repeated the simulation 500 times, and the results are reported in

Table 1. Variable selection results for the PEL method.

$(\mathit{p},\mathit{r})$	n	Kernel Function	Mean Count of Zero Coefficients
			Correct	Incorrect
(10,10)	50	Epanechnikov	(4.47 [55.9%], 3.82 [54.6%])	(2.51, 3.24)
		Cosine	(4.35 [54.4%], 3.69 [52.7%])	(2.38, 3.47)
(10,10)	100	Epanechnikov	(6.93 [86.6%], 5.63 [80.4%])	(0.52, 0.76)
		Cosine	(6.86 [85.8%], 5.58 [79.7%])	(0.55, 0.74)
(10,10)	200	Epanechnikov	(7.32 [91.5%], 6.27 [89.5%])	(0.34, 0.51)
		Cosine	(7.29 [91.1%], 6.24 [89.1%])	(0.37, 0.58)
(20,20)	50	Epanechnikov	(9.71 [53.9%], 8.95 [52.6%])	(4.83, 5.95)
		Cosine	(9.58 [53.2%], 8.79 [51.7%])	(4.92, 6.14)
(20,20)	100	Epanechnikov	(12.69 [70.5%], 11.81 [69.4%])	(3.58, 5.32)
		Cosine	(12.66 [70.3%], 11.67 [68.6%])	(3.71, 5.45)
(20,20)	200	Epanechnikov	(15.81 [87.8%], 14.71 [86.5%])	(0.42, 0.87)
		Cosine	(15.28 [84.9%], 14.56 [85.6%])	(0.45, 0.93)

and

Table 2. Estimation mean and standard deviations of the PEL estimators (the values in parentheses are the corresponding standard deviations).

$(\mathit{p},\mathit{r})$	n	Kernel Function	${\mathit{\theta }}_1$	${\mathit{\theta }}_{(\mathit{p}-1)}$	${\mathit{\gamma }}_2$	${\mathit{\gamma }}_3$
(10,10)	50	Epanechnikov	4.35 (2.616)	2.74 (1.738)	3.91 (2.237)	−4.96 (3.521)
		Cosine	4.27 (2.853)	2.59 (1.802)	4.14 (2.394)	−4.58 (3.475)
(10,10)	100	Epanechnikov	1.94 (0.128)	0.96 (0.106)	1.56 (0.132)	−1.93 (0.103)
		Cosine	2.06 (0.131)	0.94 (0.114)	1.43 (0.146)	−2.09 (0.115)
(10,10)	200	Epanechnikov	1.95 (0.087)	1.03 (0.082)	1.56 (0.105)	−2.04 (0.103)
		Cosine	2.02 (0.094)	1.06 (0.091)	1.43 (0.113)	−2.07 (0.107)
(20,20)	50	Epanechnikov	4.92 (3.587)	3.17 (3.264)	5.89 (4.316)	−6.83 (4.763)
		Cosine	4.53 (3.951)	2.96 (3.728)	6.34 (4.512)	−6.71 (4.625)
(20,20)	100	Epanechnikov	2.97 (1.438)	1.53 (0.896)	2.16 (1.048)	−2.79 (0.951)
		Cosine	3.18 (1.502)	1.61 (0.925)	2.12 (1.073)	−2.85 (1.027)
(20,20)	200	Epanechnikov	2.09 (0.135)	1.05 (0.107)	1.54 (0.139)	−1.96 (0.114)
		Cosine	2.12 (0.139)	1.07 (0.119)	1.43 (0.147)	−2.08 (0.121)

.

From Table 1 and Table 2, we observe that (1) for fixed p and n, as the sample size increases, the accuracy of variable selection improves and the standard deviation of the estimation decreases; (2) the choice of kernel function has a relatively minor influence on the results. Overall, the Epanechnikov kernel performs slightly better in estimation than the cosine kernel.

Example 2.

To consider the performance of the presented method for dependent covariates, we generated predictors by ${(X,Z)}^T\sim \mathrm{N}(0,\Sigma )$, where ${\sigma }_{ij}=0.3^{|i-j|}$, and generated responses by $Y\sim N(X^T\theta +exp\left(Z^T\gamma \right),\mathit{Var}\left(Y\right)=|Z^T\gamma |)$. Let $\theta ={(1,\cdots ,-1,0)}^{\top }$ and $\gamma ={(1,1,2,1,\cdots ,0)}^{\top }$. We consider dimensions $p=20,30$ and $r=20,30$, and sample sizes of $n=200,400$, respectively. We applied the Epanechnikov kernel functions $K\left(t\right)={\displaystyle \frac{3}{4}}{(1-t^2)}_{+}$, and applied the cross-validation method to select the penalty parameters τ and ν. According to Condition 2, the bandwidth was set to be $n^{-1/5}$, which means that $h\approx 0.35$ when $n=200$, and $h\approx 0.3$ when $n=400$. Because Lai et al. [16] also studied parameter estimation and variable selection for a heteroscedastic PLSIM, we computed their estimator (PVS) in this simulation study for the purpose of comparison. For each of these settings, we repeated the simulation 500 times, and the results are reported in

Table 3. Variable selection results for the PEL and PVS methods.

$(\mathit{p},\mathit{r})$	n	Method	Mean Count of Zero Coefficients
			Correct	Incorrect
(20,20)	200	PEL	(15.21 [84.5%], 13.45 [84.1%])	(0.37, 1.03)
		PVS	(14.75 [81.9%], 13.17 [82.3%])	(0.45, 1.26)
	400	PEL	(15.89 [88.3%], 14.02 [87.6%])	(0.35, 0.85)
		PVS	(15.46 [85.8%], 13.74 [85.9%])	(0.41, 0.92)
(20,30)	200	PEL	(15.24 [84.6%], 22.56 [86.7%])	(0.39, 1.27)
		PVS	(14.98 [83.2%], 22.12 [85.1%])	(0.38, 1.35)
	400	PEL	(15.79 [87.7%], 23.31 [89.7%])	(0.34, 1.14)
		PVS	(15.13 [84.1%], 22.85 [87.9%])	(0.43, 1.28)
(30,20)	200	PEL	(24.38 [87.1%], 13.51 [84.4%])	(0.38, 1.24)
		PVS	(23.85 [85.2%], 12.97 [81.1%])	(0.44, 1.31)
	400	PEL	(25.14 [89.7%], 14.13 [88.3%])	(0.33, 1.17)
		PVS	(24.73 [88.3%], 13.54 [84.6%])	(0.39, 1.26)
(30,30)	200	PEL	(24.25 [86.6%], 23.36 [89.8%])	(0.40, 1.31)
		PVS	(23.72 [84.7%], 22.68 [87.2%])	(0.47, 1.38)
	400	PEL	(25.27 [90.0%], 23.15 [89.1%])	(0.38, 1.19)
		PVS	(24.54 [87.6%], 22.82 [87.7%])	(0.45, 1.25)

and

Table 4. Estimation mean and standard deviations of the PEL and PVS estimators (the Values in parentheses are the corresponding standard deviations).

$(\mathit{p},\mathit{r})$	n	Method	${\mathit{\theta }}_1$	${\mathit{\theta }}_{(\mathit{p}-1)}$	${\mathit{\gamma }}_2$	${\mathit{\gamma }}_3$	${\mathit{\gamma }}_4$
(20,20)	200	PEL	1.05 (0.132)	−1.04 (0.101)	1.06 (0.143)	2.07 (0.127)	0.95 (0.106)
		PVS	1.12 (0.145)	−1.08 (0.134)	0.93 (0.175)	2.11 (0.142)	1.08 (0.131)
	400	PEL	0.98 (0.102)	−1.03 (0.091)	1.04 (0.126)	1.98 (0.108)	1.06 (0.113)
		PVS	1.07 (0.134)	−1.06 (0.114)	0.96 (0.135)	2.08 (0.125)	1.05 (0.117)
(20,30)	200	PEL	0.94 (0.137)	−1.05 (0.109)	1.10 (0.128)	2.06 (0.121)	1.07 (0.149)
		PVS	1.86 (0.153)	−1.12 (0.142)	0.92 (0.136)	1.96 (0.147)	1.13 (0.151)
	400	PEL	0.96 (0.125)	−1.04 (0.098)	1.08 (0.124)	2.03 (0.119)	1.09 (0.128)
		PVS	1.91 (0.138)	−1.09 (0.117)	0.92 (0.225)	1.95 (0.124)	1.92 (0.137)
(30,20)	200	PEL	1.09 (0.141)	−1.13 (0.135)	1.07 (0.127)	1.94 (0.130)	1.08 (0.143)
		PVS	0.89 (0.159)	−1.08 (0.162)	1.09 (0.143)	1.91 (0.129)	1.13 (0.156)
	400	PEL	1.06 (0.098)	−0.96 (0.107)	1.03 (0.126)	2.05 (0.115)	1.04 (0.112)
		PVS	1.86 (0.116)	−1.58 (0.154)	0.94 (0.139)	1.97 (0.129)	1.07 (0.123)
(30,30)	200	PEL	1.14 (0.129)	−1.13 (0.138)	0.91 (0.142)	2.09 (0.138)	1.12 (0.128)
		PVS	0.79 (0.148)	−1.18 (0.145)	0.88 (0.157)	1.90 (0.141)	0.93 (0.135)
	400	PEL	1.09 (0.114)	−1.07 (0.126)	0.93 (0.119)	2.10 (0.128)	1.09 (0.119)
		PVS	1.15 (0.123)	−1.15 (0.131)	1.05 (0.125)	1.89 (0.133)	0.92 (0.121)

.

From Table 3 and Table 4, it can be observed that (1) both estimators (PEL and PVS) yield estimates close to the true parameter values, with PEL exhibiting slightly smaller standard deviations than PVS; (2) in terms of variable selection, PEL produces, on average, fewer false zeros than PVS. Furthermore, the PEL method is a nonparametric methodology that retains the advantages of parametric likelihood while possessing double robustness. In contrast, the PVS method is a semiparametric efficient method and does not possess double robustness. For instance, when the model is misspecified, the performance of the PVS estimator is adversely affected. Thus, the proposed PEL method demonstrates favourable performance and outperforms the PVS method.

5. Real Data Application

We will demonstrate the proposed methodology through application of a PLSIM to the AIDS Clinical Trials Group Protocol 175 (ACTG175) dataset (Hammer et al. [29]; https://www.nejm.org/doi/full/10.1056/NEJM199610103351501#tab-contributors (accessed on 5 August 2025)), previously examined by Lai and Wang [30]. The CD4 glycoprotein functions as an essential T-cell receptor (TCR) coreceptor that facilitates antigen-presenting cell interactions, establishing CD4 cell count as the primary immunological endpoint for comparing antiretroviral treatment effects during predefined observation periods in HIV clinical research. ACTG175 evaluates four distinct antiretroviral regimens: didanosine (ddI), zidovudine (ZDV) monotherapy, ZDV+ddI, and ZDV+zalcitabine, utilizing a balanced randomization design to assign 2138 eligible participants across therapeutic arms. The trial results demonstrate that structured antiretroviral interventions effectively reduce disease progression risks among clinically asymptomatic individuals with intermediate-stage HIV infection.

We aimed to construct a PLSIM to analyze subject responses under zidovudine (ZDV) monotherapy. Our analysis utilizes a curated subset of the ACTG175 cohort comprising 320 patients with complete CD4 endpoint data, derived from an initial pool of 521 subjects exhibiting baseline CD4 counts between 200–500 cells/mm³. The response variable Y (CD496) quantifies CD4 cell counts at $96\pm 5$ weeks post-treatment. Predictors include the following:

Linear component: Discrete covariates $x_1$ (drugs (history of IV drug use (0 = no, 1 = yes))), $x_2$ (str2 (antiretroviral history (0 = naive, 1 = experienced))), $x_3$ (gender (0 = F, 1 = M)), $x_4$ (symptom (symptomatic indicator (0 = asymp, 1 = symp))), $x_5$ (race (0 = White, 1 = non-white)), $x_6$ (hemo (hemophilia (0 = no, 1 = yes))), $x_7$ (homo (homosexual activity (0 = no, 1 = yes))), and $x_8$ (karnof (Karnofsky score (on a scale of 0–100))).

Single-index component: Continuous covariates $z_1$ (CD80 (baseline CD8 count)), $z_2$ (CD820 (CD8 count at 20 ± 5 weeks)), $z_3$ (CD420 (CD4 count at 20 ± 5 weeks)), $z_4$ (CD40 (baseline CD4 count)), $z_5$ (wtkg (weight)), and $z_6$ (age (age (yrs) at baseline)).

After standardizing Y, we specify the following heteroscedastic PLSIM:

$$Y={\theta }^{\top }X+g\left(Z^{\top }\gamma \right)+\epsilon ,$$

where $X^{\top }=(x_1,\cdots ,x_8)$ and $Z^{\top }=(z_1,\cdots ,z_6)$. The test for heteroscedasticity confirms that the model is homoscedastic. We applied the Epanechnikov kernel function in this data analysis. According to Condition 2, the bandwidth was set to $n^{-1/5}$, which means that $h\approx 0.32$. We required $\gamma ={({\gamma }_1,\cdots ,{\gamma }_6)}^{\top }$ to have unit length to ensure identifiability. We compared our results to the PVS method in [16], and the results are summarized in

Table 5. Estimations and confidence intervals of the methods of PEL and PVS (the values in parentheses are the corresponding confidence intervals).

Variable	Method
	PEL	PVS
homo	0.193 ([0.065, 0.317])	0.131 ([0.001, 0.262])
str2	−0.322 ([−0.517, −0.016])	−0.045 ([−0.324, 0.234])
age	0.528 ([0.314, 0.735])	0.153 ([−0.118, 0.424])
CD820	−0.691 ([−0.478, −0.893])	−0.208 ([−0.413, 0.003])
cD40	0.255 ([0.009, 0.452])	0.446 ([0.134, 0.758])
cD420	0.423 ([0.126, 0.674])	0.857 ([0.536, 1.178])

, with residual plots shown in Figure 1. The residual sums of squares estimated by PEL and PVS were 186 and 194, respectively. From these results, homosexuality exhibits a significant positive linear association with Y. The estimated coefficient for antiretroviral therapy (str2) is negative, indicating a beneficial effect of this treatment for asymptomatic patients with HIV. CD820 shows a negative nonlinear relationship with Y, while age, CD40, and CD420 are positively associated with Y. These factors play important roles in antiretroviral regimens. Moreover, both methods yield similar results, but the approach in [16] produces a slightly larger standard error.