Penalized Exponentially Tilted Likelihood for Growing Dimensional Models with Missing Data

Abstract

This paper develops a penalized exponentially tilted (ET) likelihood to simultaneously estimate unknown parameters and select variables for growing dimensional models with missing response at random. The inverse probability weighted approach is employed to compensate for missing information and to ensure the consistency of parameter estimators. Based on the penalized ET likelihood, we construct an ET likelihood ratio statistic to test the contrast hypothesis of parameters. Under some wild conditions, we obtain the consistency, asymptotic properties, and oracle properties of parameter estimators and show that the constrained penalized ET likelihood ratio statistic for testing the contrast hypothesis possesses the Wilks’ property. Simulation studies are conducted to validate the finite sample performance of the proposed methodologies. Thyroid data taken from the First People’s Hospital of Yunnan Province is employed to illustrate the proposed methodologies.

1. Introduction

Estimating equations (EEs) are widely used to make statistical inference on data with unknown distribution but moment conditions. Generally, it can be formulated as $g(x,y;\beta )=(g_1(x,y;\beta ),g_2(x,y;\beta ),\dots ,g_r(x,y;\beta ))$, satisfying the following moment restrictions, $E[g(x,y;\beta \left)\right]=0$, where $\{x,y\}$ are response variables or covariates and $\beta$ is an unknown p-dimensional parameter vector. There is considerable literature on the statistical inference of EEs. For example, Godambe [1] discussed the properties (e.g., unbiasedness) of unknown parameter estimators in EEs. Carroll et al. [2] applied EEs to measurement error models. Hardin & Hilbe [3] extended EEs to a more general case. For the fixed and finite values of p and r, various methods have been developed to make statistical inference on EEs. For example, see the generalized method of moments (GMM) [4], the empirical likelihood (EL) method [5,6,7], and the exponentially tilted (ET) likelihood method [8]. In particular, Newey & Smith [9] showed that EL- and ET-likelihood-based estimators of unknown parameters in EEs belong to a class of generalized EL (GEL) estimators. The GEL estimator, as an alternative to the GMM estimator, possesses the Wilks’ property, Bartlett correction, and well-defined high-order asymptotic properties, which are superior to GMM estimation [9,10].

Estimating equations for growing dimensional models has extensive applications in statistics, economics, finance, and other related fields. When p and r grow to infinity, it usually assumes that these models are sparse, meaning that only a small number of covariates contribute to the response variable. Hence, many penalized approaches have been developed to simultaneously estimate unknown parameters and select important covariates. Introducing penalty functions into likelihood is a widely adopted approach; there are many choices for penalty functions, such as least absolute shrinkage and selection operator (Lasso) [11], adaptive Lasso [12], group Lasso, smoothly clipped absolute deviation (SCAD) [13], and folded concave penalty (FCP) [14], among others. For example, Hastie et al. [15] showed that introducing penalty functions into likelihood can adjust the trade-off between bias and variance. Lam & Fan [16] studied the profile-kernel likelihood method in linear regression models. Zou & Zhang [17] established the oracle property of the adaptive elastic-net under the divergent parameter model. Caner & Zhang [18] extended Zou & Zhang’s work to GMMs. Tang et al. [8] proposed an exponentially tilted (ET) likelihood inference for EEs with growing dimensional models, studied the asymptotic properties of ET likelihood estimators, and showed that the ET likelihood estimators of unknown parameters are robust enough to model misspecification and the ET likelihood ratio statistic for testing parameters and possess the Wilks’ property under some wild conditions. The aforementioned literature only focuses on situations where the data are completely observed. However, missing data are widely encountered in many applications, such as biomedical studies and clinical trials, due to various reasons, such as data collection errors, non-response, or technical limitations.

In data analysis, the imputation of missing data is critical when the missing data mechanism is not missing completely at random, in that the complete case analysis, which is conducted using only the observations with completely observed variables, may lead to severe bias. To address the issue, many methods have been developed to make statistical inference on missing data models via the imputation of missing data in various fields, including economics, social sciences, healthcare, and machine learning. For example, Wang et al. [19] investigated the estimation problem of regression coefficients in linear models with missing covariates. Ref. [20] applied locally weighted kernel polynomial regression methods to generalized linear models with missing predictors. Scharfstein et al. [21] studied the nonignorable drop-out problem in semiparametric nonresponse models. FitzGerald [22] proposed a weighted approach to addressing missing data in generalized estimating equations with binary outcomes. Wang et al. [23] presented a general approach to handling missing data in a longitudinal study based on expected EEs. Zhou et al. [24] redefined the EE imputed method using the kernel regression method to mitigate the biased effects of missing data. Tang et al. [25] developed an EL inference on parameters in generalized EEs with nonignorable missing response data. Tang et al. [8] proposed a nonparametric imputation method based on the propensity score in a general class of semiparametric models for missing data. Qi et al. [26] reformulated the EE imputed method via the kernel regression method to handle the problem of nonignorable missing data and utilized GMM to simultaneously estimate model parameters and tilting parameters in propensity score functions. Shao & Wang [27] investigated the nonignorable missing data with a semiparametric exponential tilting propensity and applied the inverse propensity weighting approach to estimating population parameters. Tang et al. [28] proposed a new feature screening procedure in ultrahigh-dimensional partially linear models with missing responses at random for longitudinal data based on the profile marginal kernel-assisted EE imputation technique. Liu and Yuan [29] proposed adaptive EL estimators to improve the estimation efficiency of existing adaptive estimators of the mean response with nonignorable nonresponse data. The aforementioned literature on the imputation of missing data mainly focus on EL inference for fixed-dimensional EEs. Although Tang et al. [8] considered an ET likelihood inference on EEs with growing dimensional models to address the shortcoming of EL inference that is not robust to the misspecification of EEs, to our knowledge, there is little work on ET likelihood inference on growing dimensional models with missing responses at random.

In this paper, we conduct statistical inference on growing dimensional models with missing data and propose a penalized ET likelihood to simultaneously estimate parameters and select variables. We consider the case where data including response variables and covariates are subject to missing at random, impute information for missing data via the inverse probability weighted (IPW) approach, and develop the ET likelihood estimators of parameters based on the EE imputed method together with penalized ET likelihood. When the penalty function satisfies certain conditions, we obtain the oracle property and asymptotic properties of parameter estimators and show that the constrained penalized ET likelihood ratio statistic for testing the contrast hypothesis asymptotically follows the centred chi-squared distribution. Based on the test statistics, we construct confidence intervals of parameters. We conduct simulation studies for several settings to verify the finite sample performance of the proposed methodologies, including the robustness and oracle properties of parameter estimators. Thyroid data taken from the First People’s Hospital of Yunnan Province is used to select important variables associated with the thyroid nodules disease.

The rest of this paper is organized as follows. Section 2 constructs the penalized ET likelihood and develops the ET likelihood estimators of parameters and the penalized ET likelihood ratio statistic for testing te contrast hypothesis. Section 3 studies the oracle properties and asymptotic properties of parameter estimators and shows the Wilks’ property of the asymptotic distribution of the constrained penalized ET likelihood ratio statistic for testing the contrast hypothesis under some regularization conditions. Section 4 conducts simulation studies to assess the finite sample performance of the proposed methodologies. An example is illustrated in Section 5. A brief discussion is given in Section 6.

2. Model and Notation

Let ${\{X_i,Y_i\}}_{i=1}^n$ be n independent and identically distributed observations of random variables $\{X,Y\}$ coming from an unknown distribution $F(x,y)$, where $X_i$’s are the $d_x$-dimensional vector and $Y_i$’s are the $d_y$-dimensional vector. It is assumed that $X_i$’s are always observed, while $Y_i$’s are subject to missingness for $i=1,\dots ,n$. Let ${\delta }_i$ be an indicator of $Y_i$, i.e., ${\delta }_i=1$ if $Y_i$ is observed and ${\delta }_i=0$ if $Y_i$ is missing for $i=1,\dots ,n$. It is assumed that ${\delta }_i$ and ${\delta }_j$ are independent for any $i\ne j$. Here, we assume a missing at random (MAR) mechanism for missing value of $Y_i$, i.e., ${\delta }_i$ only depends on $X_i$, such that $\mathrm{Pr}({\delta }_i=1\mid X_i)\doteq {\pi }_i(X_i;\gamma )$ for $i=1,\dots ,n$, where $\gamma$ is a q-dimensional unknown parameter vector.

To meet the identifiability condition of the considered model, the idea of instrumental variables [30] is adopted. Thus, it is assumed that $X_i$ can be decomposed into $X_i={(U_i^{\top },Z_i^{\top })}^{\top }$, where $Z_i$ is a $d_z$-dimensional vector of instrumental variables. Under the above assumption, the missingness data mechanism model can be expressed as $\mathrm{Pr}({\delta }_i=1\mid X_i)=\mathrm{Pr}({\delta }_i=1\mid U_i)={\pi }_i(U_i;\gamma )$, which can be formulated by a logistic regression. That is,

$$\begin{array}{c}\hfill {\delta }_i|U_i\sim \mathcal{B}\left({\pi }_i(U_i;\gamma )\right)\end{array}$$

(1)

where $\mathcal{B}\left(a\right)$ represents the Bernoulli distribution with the success probability a,

$$\begin{array}{c}\hfill \mathrm{logit}\left\{{\pi }_i(U_i;\gamma )\right\}={\gamma }_c+U_i^{\top }{\gamma }_u,\end{array}$$

(2)

where $\mathrm{logit}\left(a\right)=log\{a/(1-a\left)\right\}$, $\gamma =({\gamma }_c,{\gamma }_u^{\top })={({\gamma }_1,\dots ,{\gamma }_q)}^{\top }$ is the q-dimensional vector of unknown parameters associated with propensity score function ${\pi }_i(U_i;\gamma )$. To estimate unknown parameters, we consider the following calibration conditions:

$$\begin{array}{c}\hfill \phi (U_i,Z_i;\gamma )=\left\{{\displaystyle \frac{{\delta }_i}{{\pi }_i(U_i;\gamma )}}-1\right\}d\left(Z_i\right),i=1,2,\dots ,n,\end{array}$$

(3)

where $d\left(Z_i\right)$ is any users-specified function or vector on $Z_i$. If ${\pi }_i(U_i;\gamma )$ is correctly specified, we have $E[\phi (U_i,Z_i;\gamma )]=0$. Stock [31] discussed the conditions for the valid instrumental variables, along with some common pitfalls on the applications of instrumental variables.

Let ${\beta }_0$ be a p-dimensional vector of true parameters satisfying $E\left\{g(X_i,Y_i;{\beta }_0)\right\}=0$. Here, $g(X,Y;\beta )={(g_1(X,Y;\beta ),\dots ,g_r(X,Y;\beta ))}^{\top }$ represents r unconditional moment restrictions with $r\ge p$. To estimate parameter vector $\gamma$ associated with the missingness data mechanism, we assume that $\phi (U_i,Z_i;\gamma )={({\phi }_1(U_i,Z_i;\gamma ),\dots ,{\phi }_m(U_i,Z_i;\gamma ))}^{\top }$ satisfies $E\left\{\phi (U_i,Z_i;{\gamma }_0)\right\}=0$ for $i=1,\dots ,n$, where ${\gamma }_0$ is the true values of parameter $\gamma$. Our key purpose is to make statistical inference on $\beta$ and $\gamma$ for growing dimensional estimating equations and unknown parameters, i.e., $\{r,m,p,q\}$ are allowed to diverge with the sample size n.

To make statistical inference on unknown parameter vectors $\beta$ and $\gamma$, we consider the inverse probability weighted (IPW) method, which is an important method for handling missing data and has been widely applied to various missing data analyses. For example, Rosenbaum & Rubin [32] provided a detailed argument on the application of propensity scores for adjusting weights and reducing the bias of selection. Imai & Ratkovic [33] developed the IPW method to estimate causal effects. Robins et al. [34] proposed a semiparametric approach based on inverse probability weighted EEs to handle missing data. Robins & Rotnitzky [35] discussed the application of the IPW method for improving parameter estimation efficiency in the presence of missing data. Hirano et al. [36] applied the IPW method to efficiently estimate the average effect of treatments in the presence of missing data. Inspired by the aforementioned literature, we consider the following IPW-based EEs:

$$\begin{array}{c}\hfill g^{ipw}(X_i,Y_i,U_i;\beta ,\gamma )={\displaystyle \frac{{\delta }_i}{{\pi }_i(U_i;\gamma )}}g(X_i,Y_i;\beta ).\end{array}$$

(4)

Combining the calibration conditions (3) and (4) leads to the following modified estimating equations:

$$\begin{array}{c}\hfill G(D_i;\beta ,\gamma )=(g_1^{ipw}(D_i;\beta ,\gamma ),\dots ,g_r^{ipw}(D_i;\beta ,\gamma ),{\phi }_1(U_i;\gamma ),\dots ,{\phi }_m(U_i;\gamma )),\end{array}$$

(5)

where $D_i=\{X_i,Y_i,U_i\}$ for $i=1,\dots ,n$.

Let $t=r+m$ represent the dimension of the modified estimating equations and $\theta ={({\beta }^{\top },{\gamma }^{\top })}^{\top }$ be an s-dimensional vector of the parameters associated with the modified estimating equations, where $s=p+q$. When $t\ge s$, the estimators of parameters $\theta$ can be directly obtained by solving estimating Equation (5), but this is not necessarily feasible, especially when $t>s$, i.e., the number of estimating equations is greater than the number of parameters, implying that estimating equations provide external available information, which may lead to the over-identification problem. Hansen [4] showed that this external available information can improve estimation efficiency in the presence of over-identification. Due to the fact that the ET likelihood is more robust than the EL likelihood in the presence of model misclassification [8], here we utilize the ET likelihood method to estimate parameters $\theta$. Following the literature [8], the ET likelihood for $\theta$ based on the data $\{D_i:i=1,\dots ,n\}$ can be defined as

$$\begin{array}{c}\hfill L\left(\theta \right)=inf\left\{\sum _{i=1}^n{w}_{i}log\left(w_i\right):w_i\ge 0,\sum _{i=1}^n{w}_i=1,\sum _{i=1}^n{w}_{i}G(D_i;\theta )=0\right\}.\end{array}$$

(6)

With the Lagrange multiplier method, we obtain

$$w_i={\displaystyle \frac{exp\left\{{\lambda }^{\top }G(D_i;\theta )\right\}}{{\displaystyle \sum _{j=1}^n}exp\left\{{\lambda }^{\top }G(D_j;\theta )\right\}}},$$

where $\lambda$ is the Lagrange multiplier. Also, the global minimizer of $w_i$ is reached when $w_i=n^{-1}$ when there are no constrained conditions. Thus, the profiled log-ET likelihood ratio function of $\theta$ based on the data $\{D_i:i=1,\dots ,n\}$ can be expressed as

$$\begin{array}{c}\hfill l_n(\lambda ,\theta )=-\{L\left(\theta \right)+log\left(n\right)\}=log\left[{\displaystyle \frac{1}{n}}\sum _{i=1}^{n}exp\left\{{\lambda }^{\top }G(D_i;\theta )\right\}\right].\end{array}$$

(7)

Let

$$\begin{array}{cc}\hfill U_{n1}(\lambda ,\theta )=& \partial l_n(\lambda ,\theta )/\partial \lambda =n^{-1}\sum _{i=1}^{n}exp\left\{{\lambda }^{\top }G(D_i;\theta )\right\}G(D_i;\theta ),\hfill \\ \hfill U_{n2}(\lambda ,\theta )=& \partial l_n(\lambda ,\theta )/\partial \theta =n^{-1}\sum _{i=1}^{n}exp\left\{{\lambda }^{\top }G(D_i;\theta )\right\}{\lambda }^{\top }\{\partial G(D_i;\theta )/\partial \theta \},\hfill \end{array}$$

and

$$\begin{array}{c}\hfill {\widehat{\theta }}^{\mathrm{ET}}=arg\underset{\theta \in \Theta }{max}\underset{\lambda \in {\widehat{\Lambda }}_n\left(\theta \right)}{inf}{l}_n(\lambda ,\theta ),\end{array}$$

where ${\widehat{\Lambda }}_n\left(\theta \right)=\{\lambda :{\lambda }^{\top }G(D_i;\theta )\in \epsilon ,i=1,\dots ,n\}$, in which $\epsilon$ is an open ball containing zero. Under some regularity conditions, ${\widehat{\theta }}^{\mathrm{ET}}$ can be obtained by simultaneously solving $U_{n1}(\lambda ,\theta )=0$ and $U_{n2}(\lambda ,\theta )=0$, and the Lagrange multiplier $\lambda$ satisfies $U_{n1}(\lambda ,{\widehat{\theta }}^{\mathrm{ET}})=0$.

In growing dimensional estimating equations, there is a well-known ill-posed problem [8]. To address this issue, we impose a sparsity assumption on parameter vectors $\theta$ and $\gamma$ so that only a very small portion of their coordinates are non-zero, which means only a small number of covariates contribute to the response variable. Under the sparsity assumption, let ${\mathcal{A}}_{\beta }=\{j:{\beta }_{0j}\ne 0\}$ and $d_{\beta }=|{\mathcal{A}}_{\beta }|$ be the cardinality of ${\mathcal{A}}_{\beta }$, where ${\beta }_{0j}$ represents the jth component of the true value ${\beta }_0$ of parameter vector $\beta$. Without loss of generality, let $\beta ={({\beta }_1^{\top },{\beta }_2^{\top })}^{\top }$, where ${\beta }_1\in {\mathbb{R}}^{d_{\beta }}$ and ${\beta }_2\in {\mathbb{R}}^{p-d_{\beta }}$ correspond to the non-zero and zero components of $\beta$, respectively. Similarly, let ${\mathcal{A}}_{\gamma }=\{j:{\gamma }_{0j}\ne 0\}$, its cardinality be $d_{\gamma }=|{\mathcal{A}}_{\gamma }|$, and $\gamma ={({\gamma }_1^{\top },{\gamma }_2^{\top })}^{\top }$, where ${\gamma }_1\in {\mathbb{R}}^{d_{\gamma }}$ and ${\gamma }_2\in {\mathbb{R}}^{q-d_{\gamma }}$ correspond to non-zero and zero components of $\gamma$, respectively. Also, let ${\mathcal{A}}_{\theta }=\{j:{\theta }_{0j}\ne 0\}$, $d_{\theta }=|{\mathcal{A}}_{\theta }|$, $\theta ={({\theta }_1^{\top },{\theta }_2^{\top })}^{\top }$, where ${\theta }_1\in {\mathbb{R}}^{d_{\theta }}$ and ${\theta }_2\in {\mathbb{R}}^{s-d_{\theta }}$, where $d_{\theta }=d_{\beta }+d_{\gamma }$. Here, we consider simultaneously selecting non-zero components of both parameters of $\gamma$ associated with the missingness data mechanism and parameter $\beta$ related to growing dimensional EEs. To this end, we define the following penalized log-ET likelihood ratio function of $\theta$ based on the data $\{D_i:i=1,\dots ,n\}$:

$$\begin{array}{c}\hfill l_{pn}\left(\theta \right)=\log \left[{\displaystyle \frac{1}{n}}\sum _{i=1}^{n}exp\left\{{\lambda }^{\top }G(D_i;\theta )\right\}\right]-\sum _{j=1}^p{p}_{{\nu }_1}(|{\beta }_j|)-\sum _{k=1}^q{p}_{{\nu }_2}(|{\gamma }_k|),\end{array}$$

(8)

where $p_{\nu }(·)$ is some proper penalty function, with the tuning parameter $\nu$ controlling the trade-off between bias and model complexity. There are many choices for penalty functions, such as Lasso, adaptive Lasso, SCAD, FCP, and MCP. When the penalty function is non-convex, the calculation of the penalized ET likelihood estimation of parameters $\theta$ is challenging [37]. In particular, for the SCAD penalty function, the nested optimization algorithm is adopted to minimize the objective function $l_{pn}\left(\theta \right)$ given in (8) by using the local quadratic approximation method of Fan & Li [13] to approximate the nonconvex penalty $p_{\nu }(|{\theta }_j|)$ for $j=1,\dots ,p+q$. The computational complexity of optimizing Equation (8) is $\mathrm{O}\left(nt{s}^3\mathcal{T}\right)$, where $\mathcal{T}$ is the number of iterations.

For the selection of tuning parameter ${\nu }_k$ ($k=1,2$), there are many widely used methods, such as Akaike information criterion (AIC), Bayesian information criterion (BIC), cross validation (CV), and generalized CV, among others. Similarly to [8], we consider the Bayesian information criterion,

$$\begin{array}{c}\hfill \mathrm{BIC}\left(\nu \right)=-2l\left({\widehat{\theta }}_{\nu }\right)+{\mathcal{B}}_{n}log\left(n\right)d{f}_{\nu }\end{array}$$

to select the tuning parameter ${\nu }_k$, where $\nu =\{{\nu }_1,{\nu }_2\}$, $l\left(\theta \right)=l_n(\lambda ,\theta )$ is given in Equation (7), ${\widehat{\theta }}_{\nu }$ is the penalized ET estimator of $\theta$ depending on the tuning parameter $\nu$, $\nu$ is the tuning parameter, $d{f}_{\nu }$ is the number of non-zero coefficients in $\theta$ implying the “degrees of freedom” of the estimated EEs, and ${\mathcal{B}}_n$ is the scaling factor that diverges to infinity at a slow rate as $s\to \infty$. Similarly to [8], when s is fixed, we simply take ${\mathcal{B}}_n=1$; otherwise, we set ${\mathcal{B}}_n=max(loglogs,1)$. By the Bayesian information criterion, it can be seen that the continuously increasing ${\mathcal{B}}_n$ is used to offset the effect of the continuously increasing dimension s on parameter estimation.

3. Asymptotic Properties

3.1. Asymptotic Properties for Correctly Specified EEs

This section focuses on the consistent and oracle properties of parameter estimator ${\widehat{\theta }}^{PET}$, and on testing the contrast hypothesis. To this end, we need the following assumptions.

Assumption 1.(i) The probability density function $f(x,y)$ is bounded for any $x\in \mathbb{X}$ and $y\in \mathbb{Y}$, where $\mathbb{X}$ and $\mathbb{Y}$ are the supports of x and y, respectively. (ii) The second derivatives of $f(x,y)$ are continuous and bounded.

Assumption 2.(i) The propensity score function $\pi (u;\gamma )$ satisfies $\pi (u;\gamma )>c_0$ for any $u\in \mathbb{U}$, where $\mathbb{U}$ is the support of u, and $c_0>0$ is a constant. (ii) The second derivatives of $\pi (u;\gamma )$ are continuous and bounded.

Assumption 3.

Parameter vector $\theta ={({\beta }^{\top },{\gamma }^{\top })}^{\top }$ is recognizable, which means that there exists a unique solution, ${\theta }_0={({\beta }_0^{\top },{\gamma }_0^{\top })}^{\top }\in \Theta$ to $E[G(D_i;{\theta }_0)]=0$ for $D_i\in \{\mathbb{X},\mathbb{Y},\mathbb{U}\}$, $i=1,\dots ,n$.

Assumption 4.

Let ${\mathcal{D}}_{n\beta }=\{\beta :\Vert \beta -{\beta }_0\Vert \le C\sqrt{r/n}\}$, ${\mathcal{D}}_{n\gamma }=\{\gamma :\Vert \gamma -{\gamma }_0\Vert \le C\sqrt{m/n}\}$, and ${\mathcal{D}}_{n\theta }=\{\theta :\Vert \theta -{\theta }_0\Vert \le C\sqrt{t/n}\}$, where $C>0$ is a proper constant. There are three constants, $C_1^g>0$, ${\mathcal{K}}_1^g>0$, and ${\mathcal{K}}_2^g>0$, and two functions, ${\mathbb{K}}_1^g\left({\tilde{D}}_i\right)$ and ${\mathbb{K}}_2^g\left({\tilde{D}}_i\right)$, satisfying $E{[{\mathbb{K}}_1^g\left({\tilde{D}}_i\right)]}^2\le {\mathcal{K}}_1^g$ and $E{[{\mathbb{K}}_2^g\left({\tilde{D}}_i\right)]}^2\le {\mathcal{K}}_2^g$, such that for $\beta \in {\mathcal{D}}_{n\beta }$, (i) $|E[n^{-1}{\sum }_{i=1}^n{g}^{\top }({\tilde{D}}_i;\beta )g({\tilde{D}}_i;\beta )]|\le C_1^g$, with probability tending to one; (ii) ${sup}_{j}E{[g_j({\tilde{D}}_i;\beta )]}^2$$\le C_1^g$ and ${sup}_{j,l}E{[g_j({\tilde{D}}_i;\beta )g_l({\tilde{D}}_i;\beta )]}^2\le C_1^g$; (iii) ${sup}_{j,l,\beta }|\partial g_j({\tilde{D}}_i;\beta )/\partial {\beta }_l|\le {\mathbb{K}}_1^g\left({\tilde{D}}_i\right)$ and ${sup}_{j,l_1,l_2,\beta }|{\partial }^2{g}_j({\tilde{D}}_i;\beta )/\partial {\beta }_{l_1}\partial {\beta }_{l_2}|\le {\mathbb{K}}_2^g\left({\tilde{D}}_i\right)$, where ${\tilde{D}}_i=\{X_i,Y_i\}$.

Assumption 5.

$t=o\left(n^{1/2-1/\iota }\right)$ for some $\iota >2$ and $t=O\left(s\right)$.

Assumptions 1 and 2 are the widely used conditions in the missing data literature. Assumption 1 implies that the probability density functions $f\left(u\right)$ and $f\left(z\right)$ are bounded for any $u\in \mathbb{U}$ and $z\in \mathbb{Z}$, respectively, where $\mathbb{Z}$ is the support of z. Assumption 3 ensures the existence and consistency of the solution that can be obtained with (7). Assumption 4 is a common condition for sample moments. Also, it follows from Assumptions 1(i) and 2 and (3) that the EEs $\phi (U_i;\gamma )$ and $g^{ipw}(D_i;\theta )$ satisfy Assumption 4 for $\gamma \in {\mathcal{D}}_{n\gamma }$ and $\theta \in {\mathcal{D}}_{n\theta }$. There are constants $C_1^G>0$, ${\mathcal{K}}_1^G>0$, ${\mathcal{K}}_2^G>0$ and functions ${\mathbb{K}}_1^G\left(D_i\right)$ and ${\mathbb{K}}_2^G\left(D_i\right)$ satisfying $E{[{\mathbb{K}}_1^G\left(D_i\right)]}^2\le {\mathcal{K}}_1^G$ and $E{[{\mathbb{K}}_2^G\left(D_i\right)]}^2\le {\mathcal{K}}_2^G$, such that, for $\theta \in {\mathcal{D}}_{n\theta }$, (i) $|E[n^{-1}{\sum }_{i=1}^n{G}^{\top }(D_i;\theta )G(D_i;\theta )]|\le C_1^G$ with probability tending to one; (ii) ${sup}_{j}E{[G_j(D_i;\theta )]}^2\le C_1^G$ and ${sup}_{j,l}E{[G_j(D_i;\theta )G_l(D_i;\theta )]}^2\le C_1^G$; (iii) ${sup}_{j,l,\theta }|\partial G_j(D_i;\theta )/\partial {\theta }_l|\le {\mathbb{K}}_1^G\left(D_i\right)$ and ${sup}_{j,l_1,l_2,\theta }|{\partial }^2{G}_j(D_i;\theta )/\partial {\theta }_{l_1}\partial {\theta }_{l_2}|\le {\mathbb{K}}_2^G\left(D_i\right)$, which is used to ensure the boundedness of EEs $G(D_i;\theta )$ and their derivatives [8]. Here, we assume that t and s are allowed to be divergent when $n\to \infty$. But Assumption 5 imposes certain limitations on the divergence rate of t and s.

Lemma 1.

Under Assumptions 1–5, we have

$$\begin{array}{c}\hfill l_{pn}\left(\theta \right)=-{\displaystyle \frac{1}{2}}{(\theta -\dot{\theta })}^{\top }\mathcal{V}(\theta -\dot{\theta })+o_p(t/n),\end{array}$$

where $\mathcal{V}=V_0+\mathcal{M}{\Sigma }^{-1}{\mathcal{M}}^{\top }$, $\dot{\theta }={\mathcal{V}}^{-1}(V_0{\theta }_0+\mathcal{M}{\Sigma }^{-1}{\mathcal{M}}^{\top }{\widehat{\theta }}^{\mathrm{ET}})$, ${\widehat{\theta }}^{\mathrm{ET}}$ is the ET likelihood estimator of θ, and $V_0$ is a diagonal matrix with the jth diagonal element being $V_0^{jj}$ for $j=1,\dots ,p+q$.

Similarly to [8], the jth non-zero diagonal component $V_0^{jj}$ can be evaluated with the following quadratic approximation of $p_{\nu }(|{\theta }_j|)$ at ${\theta }_{0j}$: $p_{\nu }(|{\theta }_j|)=p_{\nu }(|{\theta }_{0j}|)+{\dot{p}}_{\nu }(|{\theta }_j^{\xi }|){\theta }_j^{\xi }({\theta }_j-{\theta }_{0j})/|{\theta }_j^{\xi }|\approx p_{\nu }(|{\theta }_{0j}|)+{\dot{p}}_{\nu }(|{\theta }_j^{\xi }|)({\theta }_j^2-{\theta }_{0j}^2)/(2|{\theta }_j^{\xi }|)$ for ${\theta }_j\in \{{\theta }_j:|{\theta }_j-{\theta }_{0j}|\le C\sqrt{t/n}\}$, which yields $V_0^{jj}={\dot{p}}_{\nu }(|{\theta }_j^{\xi }|)/|{\theta }_j^{\xi }|$, where ${\theta }_j^{\xi }$ is a point lying in the line between ${\theta }_j$ and ${\theta }_{0j}$. Lemma 1 provides a quadratic form of the penalized ET likelihood ratio function, which is approximately an ellipsoid, with $\dot{\theta }$ being the center. Lemma 1 can be used to obtain the consistency and asymptotic properties of the parameter estimator for $\theta$. To obtain the oracle property of the parameter estimator for $\theta$, we also need the following assumptions:

Assumption 6.

For any $ϵ>0$, ${inf}_{\beta \in {\Theta }_{\beta }^W}||E[g({\tilde{D}}_i;\beta )]||\ge {\psi }_1(d_{\beta },r){\psi }_2\left(ϵ\right)>0$ and ${inf}_{\gamma \in {\Theta }_{\gamma }^W}||E[\phi (U_i;\gamma )]||\ge {\psi }_1(d_{\gamma },m){\psi }_2\left(ϵ\right)>0$, where ${\Theta }_{\beta }^W=\{\beta \in {\Theta }_{\beta }:||\beta -{\beta }_0||\ge ϵ\}$, ${\Theta }_{\gamma }^W=\{\gamma \in {\Theta }_{\gamma }:||\gamma -{\gamma }_0||\ge ϵ\}$, ${\Theta }_{\beta }$ is the projection of θ-parameter space Θ onto β, ${\Theta }_{\gamma }$ is the projection of θ-parameter space Θ onto γ, ${\psi }_2(·)$ is some proper positive function, and the positive function ${\psi }_1(·,·)$ satisfies $lim{inf}_{d_{\beta },r\to \infty }{\psi }_1(d_{\beta },r)>0$.

Assumption 7.

Let ${\Sigma }_g\left(\beta \right)=E[\{g_i\left(\beta \right)-g\left(\beta \right)\}{\{g_i\left(\beta \right)-g\left(\beta \right)\}}^{\top }]$. There are constants $C_2^g$ and $C_3^g$, such that $0<C_2^g\le {\mathbb{EV}}_1\left\{{\Sigma }_g\left(\beta \right)\right\}\le \cdots \le {\mathbb{EV}}_t\left\{{\Sigma }_g\left(\beta \right)\right\}\le C_3^g<\infty$ for $\forall \beta \in {\mathcal{D}}_{n\beta }$, where $g_i\left(\beta \right)=g({\tilde{D}}_i;\beta )$, $g\left(\beta \right)=E[g({\tilde{D}}_i;\beta )]$, ${\mathbb{EV}}_j\left(A\right)$ represents the jth eigenvalue of matrix A.

Assumption 8.

There is a positive function $\eta \left(\nu \right)>0$, such that $\eta \left(\nu \right)=O\left(\nu \right)$. For ${\theta }_0\in \Theta$, ${max}_j|{\theta }_{0j}|<\infty$, and $\eta \left(\nu \right)$ satisfies $\eta \left(\nu \right)/{min}_{j\in \mathcal{A}}|{\theta }_{0j}|\to 0$, where ${\theta }_{0j}$ represents the jth component of the true value ${\theta }_0$ for parameter vector θ.

Assumption 9.

The penalty function $p_{\nu }(·)$ satisfies ${\mathit{max}}_j{\dot{p}}_{\nu }(|{\theta }_{0j}|)=o\left\{{\left(ns\right)}^{-1/2}\right\}$ and ${\mathit{max}}_j{\ddot{p}}_{\nu }(|{\theta }_{0j}|)=o\left(s^{-1/2}\right)$, and ${max}_j{\dot{p}}_{\nu }(|{\theta }_j|)/\eta \left(\nu \right)\to 0$ and $s/[\eta \left(\nu \right){min}_j{\dot{p}}_{\nu }(|{\theta }_j|)]\to 0$ for $\theta \in {\mathcal{D}}_{n\theta }$ as $n\to \infty$, where ${\dot{p}}_{\nu }(·)$ and ${\ddot{p}}_{\nu }(·)$ are the first- and second-order derivatives of $p_{\nu }(·)$, respectively.

Assumption 6 is the identification condition for the parameter space, which grows to infinity. Assumption 7 ensures that ${\Sigma }_g\left(\beta \right)$ is a strictly positive definite matrix. It follows from Assumptions 1(i) and 2(i) and (3) that ${\Sigma }_{\phi }\left(\gamma \right)=E[\{{\phi }_i\left(\gamma \right)-\phi \left(\gamma \right)\}{\{{\phi }_i\left(\gamma \right)-\phi \left(\gamma \right)\}}^{\top }]$ and ${\Sigma }_{g^{ipw}}\left(\theta \right)=E[\{g_i^{ipw}\left(\theta \right)-g^{ipw}\left(\theta \right)\}{\{g_i^{ipw}\left(\theta \right)-g^{ipw}\left(\theta \right)\}}^{\top }]$ are also strictly positive definite matrices, where ${\phi }_i\left(\gamma \right)=\phi (U_i;\gamma )$, $\phi \left(\gamma \right)=E[\phi (U_i;\gamma )]$, $g_i^{ipw}\left(\theta \right)=g^{ipw}(D_i;\theta )$ and $g^{ipw}\left(\theta \right)=E[g^{ipw}(D_i;\theta )]$. The tuning parameter $\nu$ associated with the penalty function controls the variable selection process to some extent. Without loss of generality, we assume that there exists a positive function $\eta \left(\nu \right)$ related to $\nu$, such that $\widehat{\mathcal{A}}=\{j:|{\widehat{\theta }}_j^{\mathrm{PET}}|>\eta \left(\nu \right)\}$, where ${\widehat{\theta }}_j^{\mathrm{PET}}$ is the jth component of the penalized ET estimator ${\widehat{\theta }}^{\mathrm{PET}}$. In addition, if parameter vector $\theta$ quickly converges to zero, the estimators of parameters will be subject to certain disturbances, which results in the relatively large biases and variances of parameter estimators. Therefore, Assumption 8 limits the convergence rate of parameter vector $\theta$. Assumption 9 has been adopted in the regularization literature. The widely used penalty functions, such as the SCAD penalty function, satisfy Assumption 9.

Theorem 1.

Let ${\check{\theta }}^{\mathit{PET}}={({\check{\theta }}_1^{\mathit{PET}\top },{\check{\theta }}_2^{\mathit{PET}\top })}^{\top }$ be the penalized ET likelihood estimator of θ for a correctly specified model. Under Assumptions 1–9, we have

(i) 

$||{\check{\theta }}_1^{\mathit{PET}}-{\theta }_{10}||\stackrel{\mathcal{P}}{⟶}0$ and ${\check{\theta }}_2^{\mathit{PET}}\stackrel{\mathcal{P}}{⟶}0$ as $n\to \infty$, where the notation $\stackrel{\mathcal{P}}{⟶}$ represents the convergence in probability.

(ii) 

$\sqrt{n}{A}_n{\Gamma }_1^{-1/2}({\check{\theta }}^{\mathit{PET}}-{\theta }_0)\stackrel{\mathcal{L}}{⟶}\mathcal{N}(0,\mathcal{V})$, where $A_n\in {\mathbb{R}}^{s\times t}$ satisfies $A_n{A}_n^{\top }\to \mathcal{V}$ as $n\to \infty$, $\mathcal{V}$ is a $s\times s$ nonnegative symmetric matrix, and $\stackrel{\mathcal{L}}{⟶}$ denotes convergence in distribution.

When the model correctly specified, Theorem 1 shows that the penalized ET likelihood estimator ${\check{\theta }}^{\mathrm{PET}}$ of $\theta$ possesses the consistency property for non-zero components of ${\theta }_0$, and zero components of ${\theta }_0$ are estimated as zero, with probability tending to one by the penalized ET likelihood method. Also, Theorem 1 shows that the distribution of the penalized ET likelihood estimator ${\check{\theta }}^{\mathrm{PET}}$ asymptotically converges to the normal distribution. In particular, it follows from ${\check{\theta }}^{\mathrm{PET}}=({\check{\beta }}^{\mathrm{PET}\top },{\check{\gamma }}^{\mathrm{PET}\top })$ that $\sqrt{n}{A}_{n,\beta }{\Gamma }_{1,\beta }^{-1/2}({\check{\beta }}^{\mathrm{PET}}-{\beta }_0)\stackrel{\mathcal{L}}{⟶}\mathcal{N}(0,{\mathcal{V}}_{\beta })$ and $\sqrt{n}{A}_{n,\gamma }{\Gamma }_{1,\gamma }^{-1/2}({\check{\gamma }}^{\mathrm{PET}}-{\gamma }_0)\stackrel{\mathcal{L}}{⟶}\mathcal{N}(0,{\mathcal{V}}_{\gamma })$, where $A_{n,\beta }$, ${\Gamma }_{1,\beta }$, and ${\mathcal{V}}_{\beta }$ are the projections of matrices $A_n$, ${\Gamma }_1$, and $\mathcal{V}$ onto the $\beta$ space, respectively, and $A_{n,\gamma }$, ${\Gamma }_{1,\gamma }$, and ${\mathcal{V}}_{\gamma }$ are the projections of matrices $A_n$, ${\Gamma }_1$, and $\mathcal{V}$ onto the $\gamma$ space, respectively.

We consider the following null and alternative hypotheses for testing the contrast of parameters $\theta$:

$$\begin{array}{c}\hfill H_0:{\mathcal{C}}_n\theta =0\mathrm{versus}{H}_1:{\mathcal{C}}_n\theta \ne 0,\end{array}$$

where ${\mathcal{C}}_n\in R^{e\times s}$ satisfies ${\mathcal{C}}_n{\mathcal{C}}_n^{\top }=I_e$ for some fixed integer e and $I_e$ is the $e\times e$ identity matrix. The aforementioned hypothesis includes some special cases. For example, $H_{0j}$: ${\theta }_j=0$ for $j=1,\dots ,s$, or $H_0$: ${\theta }_j-{\theta }_k=0$ for $j\ne k\in \{1,\dots ,s\}$, and so on. The penalized ET likelihood ratio statistic for testing the null hypothesis $H_0:{\mathcal{C}}_n\theta =0$ is given as

$$\begin{array}{c}\hfill \tilde{l}\left({\mathcal{C}}_n\right)=2n\left\{l_{pn}\left({\widehat{\theta }}^{\mathrm{PETO}}\right)-\underset{\theta ,{\mathcal{C}}_n\theta =0}{max}{l}_{pn}\left(\theta \right)\right\},\end{array}$$

(9)

where ${\widehat{\theta }}^{\mathrm{PETO}}=arg{max}_{\theta \in \Theta }{inf}_{\lambda \in {\widehat{\Lambda }}_n\left(\theta \right)}{l}_{pn}\left(\theta \right)$. Similarly to the empirical likelihood literature, we can show that $\tilde{l}\left({\mathcal{C}}_n\right)$ converges to ${\chi }^2$-distribution as $n\to \infty$ under some regularity conditions.

3.2. Asymptotic Properties for Misspecified EEs

Section 3.1 discusses the asymptotic properties of estimators for parameters $\theta$ when EEs are correctly specified. However, in practical applications, it is rather difficult to correctly specify EEs. That is, misspecified EEs are usually encountered in some applications. Thus, the aforementioned asymptotic properties do not hold for misspecified EEs. To address this issue, here we investigate the asymptotic properties of estimators for parameters $\theta$ obtained with misspecified EEs. To this end, we need the following assumptions.

Assumption 10.(i) The EEs $g(X_i;\beta )$ and penalty function $p_{\nu }(|\theta |)$ are continuous. (ii) The second-order derivatives of $g(X_i;\beta )$ and $\pi (U_i;\gamma )$ are continuous with respect to β and γ in the neighbourhoods ${\mathbb{N}}_{\beta }^{*}$ and ${\mathbb{N}}_{\gamma }^{*}$ of ${\beta }^{*}$ and ${\gamma }^{*}$, respectively, where ${\beta }^{*}$ and ${\gamma }^{*}$ are the “pseudo-truth” values of β and γ, which are interior points of ${\Theta }_{\beta }$ and ${\Theta }_{\gamma }$, respectively.

Assumption 11.

There are constant ${\mathcal{K}}_3^G<\infty$ and functions ${\mathbb{K}}_3^G\left(D_i\right)$ satisfying the condition $E{\left\{{\mathbb{K}}_3^G\left(D_i\right)\right\}}^2\le {\mathcal{K}}_3^G$, such that ${sup}_{\theta \in \Theta }{sup}_{\lambda \in {\widehat{\Lambda }}_n\left(\theta \right)}exp\left\{{\lambda }^{\top }G(D_i;\theta )\right\}<{\mathbb{K}}_3^G\left(D_i\right)$, where ${\widehat{\Lambda }}_n\left(\theta \right)$ is a compact set, such that ${\lambda }^{*}\left(\theta \right)\in \mathrm{int}\left({\widehat{\Lambda }}_n\left(\theta \right)\right)$.

Assumption 12.(i) For all $\theta \in {\mathbb{R}}^s$, the conditional expectation $E\{G(D_i;\theta )|G(D_i;{\theta }_0)=G(d;{\theta }_0)\}=g(d;{\theta }_0)$ exists. (ii) The expectation function $E[exp\left\{{\lambda }^{*\top }\left(\theta \right)G(D_i;\theta )\right\}]$ is convex and maximized at the “pseudo-truth” value ${\theta }^{*}$ of θ, where ${\theta }^{*}$ is the interior point of Θ.

Assumption 13.

There is a function $h\left(D_i\right)$ satisfying

$$E\left[\underset{\theta \in {\mathbb{N}}_{\theta }^{*}}{sup}\underset{\lambda \in {\widehat{\Lambda }}_n\left(\theta \right)}{sup}exp\left\{l_1{\lambda }^{\top }G(D_i;\theta )\right\}{h}^{l_2}\left(D_i\right)\right]<\infty$$

for $l_1,l_2=0,1,2$, where ${\mathbb{N}}_{\theta }^{*}$ is the neighbourhood of ${\theta }^{*}$, such that $||G(D_i;\theta )||\le h\left(D_i\right)$, $||{\partial }_{\theta }G(D_i;\theta )||\le h\left(D_i\right)$, $||{\partial }_{{\theta }_{j_1}{\theta }_{j_2}}G(D_i;\theta )||\le h\left(D_i\right)$ for any $\theta \in {\mathbb{N}}_{\theta }^{*}$ and $j_1,j_2=1,\dots ,s$, where ${\partial }_{\theta }G(D_i;\theta )\doteq \partial G(D_i;\theta )/\partial \theta$ and ${\partial }_{{\theta }_{j_1}{\theta }_{j_2}}G(D_i;\theta )\doteq {\partial }^{2}G(D_i;\theta )/\partial {\theta }_{j_1}\partial {\theta }_{j_2}$.

Assumption 14.(i) As $n\to \infty$ and $\nu \to 0$, $lim{inf}_{\theta \to 0^{+}}{\dot{p}}_{\nu }\left(\theta \right)/\nu >0$ and $||\lambda \left(\theta \right)||=o\left(\nu \right)$ for $\lambda \left(\theta \right)\in \mathrm{int}\left({\widehat{\Lambda }}_n\left(\theta \right)\right)$. (ii) There is a constant $C_p$, such that ${max}_{j\in {\mathcal{A}}_{\theta }}{\dot{p}}_{\nu }(|{\theta }_{0j}|)\le C_p\nu$ and ${max}_{j\in {\mathcal{A}}_{\theta }}{\ddot{p}}_{\nu }(|{\theta }_{0j}|)\le C_p\nu$.

Assumptions 10–13 are some regularity conditions on the EEs $g(X_i,Y_i;\beta )$, $G(D_i;\theta )$, and the propensity score functions $\pi (U_i;\gamma )$ required for the consistency and asymptotic normality of the penalized ET likelihood estimator when EEs are misspecified [8]. Assumptions 11–13 are slightly stronger than Assumptions 1 and 2. In fact, we only need to make corresponding assumptions on the EEs $g(X_i,Y_i;\beta )$ and propensity score functions $\pi (U_i;\gamma )$ based on Assumptions 1 and 2 and Equation (3), and can then obtain Assumptions 11–13. Assumption 14 gives the relationship between the order of Lagrange multipliers of the ET likelihood function and the order of tuning parameter associated with the penalty function, and presents the constraint for the first- and second-order derivatives of the penalty function in terms of the tuning parameter $\nu$, which is used to ensure the sparsity of the penalized ET likelihood estimator in the presence of EE misspecification.

Theorem 2.

Let ${\widehat{\theta }}^{\mathrm{PET}}={({\widehat{\theta }}_1^{\mathit{PET}\top },{\widehat{\theta }}_2^{\mathit{PET}\top })}^{\top }$ be the penalized ET likelihood estimator of θ in the presence of EE misspecification. Under Assumptions 10–14, as $n\to \infty$, we have

(i) 

(Consistency) ${\widehat{\theta }}^{\mathit{PET}}\stackrel{\mathcal{P}}{⟶}{\theta }^{*}$, where $\stackrel{\mathcal{P}}{⟶}$ denotes convergence in probability;

(ii) 

(Oracle property) ${\widehat{\theta }}_2^{\mathit{PET}}=0$ with probability tending to one;

(iii) 

(Asymptotic normality) $\sqrt{n}({\widehat{\zeta }}^{\mathit{PET}}-{\zeta }^{*})\stackrel{\mathcal{L}}{⟶}\mathcal{N}(0,{\Delta }^{-1}\mathsf{\Omega }{\Delta }^{-\top })$, where $\zeta ={({\lambda }^{\top },{\theta }^{\top },{\tau }^{\top })}^{\top }$, $\Delta =E\left\{{\partial }_{\zeta }\tilde{S}(\lambda ,\theta ,\tau )\right\}{|}_{\zeta ={\zeta }^{*}}$, $\mathsf{\Omega }=E\left\{\tilde{S}({\lambda }^{*},{\theta }^{*},{\tau }^{*}){\tilde{S}}^{\top }({\lambda }^{*},{\theta }^{*},{\tau }^{*})\right\}$, τ is the Lagrange multiplier, vector $\tilde{S}(\lambda ,\theta ,\tau )$ is the first-order derivatives of the penalized log-ET likelihood ratio function for misspecified EEs under constraint $M_2\theta ={\theta }_2=0$ in which $M_2$ is the projection matrix, and $\stackrel{\mathcal{L}}{⟶}$ denotes convergence in distribution.

Theorem 2 indicates that there is no convergence rate due to the misspecification of EEs; the penalized ET likelihood estimator of $\theta$ converges to its pseudo-true value rather than to its true value. Also, Theorem 2 is shown to possess the oracle property under some regularity conditions. The consistency of the estimator holds only when the penalized log-ET likelihood ratio function converges to its population form, satisfying some continuity and boundness assumptions. Moreover, Theorem 3 establishes the asymptotic normality of estimators for the non-zero components of $\theta$.

Theorem 3.

Under Assumptions 10–14, we have $\tilde{l}\left({\mathcal{C}}_n\right)\tilde{M}\stackrel{\mathcal{L}}{⟶}{\chi }_e^2$, where $\tilde{M}=K_1{({\mathcal{S}}_1-{\mathcal{S}}_2)}^{-1}$ with $K_1$, ${\mathcal{S}}_1$, and ${\mathcal{S}}_2$ defined in Appendix A and ${\chi }_e^2$ denotes the chi-squared distribution with e degrees of freedom.

Theorem 3 extends the results of the growing dimensional unconditional moment models given in [8] to the MAR case. Theorem 3 can be used to test hypotheses and construct the confidence regions of parameters $\theta$. In particular, the $100(1-\alpha )\%$ approximate confidence region for ${\mathcal{C}}_n\theta$ based on the constrainedly penalized ET likelihood ratio statistic can be written as

$$C{R}_{\alpha }=\left\{\varsigma :2n\tilde{M}\left(l_{pn}\left(\widehat{\theta }\right)-\underset{\theta ,{\mathcal{C}}_n\theta =\varsigma }{max}{l}_{pn}\left(\theta \right)\right)\le {\chi }_e^2(1-\alpha )\right\},$$

where ${\chi }_e^2(1-\alpha )$ is the $(1-\alpha )$-quantile of the chi-squared distribution with e degrees of freedom.

4. Simulation Studies

In this section, some simulation studies are conducted to investigate the finite sample performance of the proposed methodologies.

Experiment 1 (Linear Regression Model). The data $\{Y_i:i=1,\dots ,n\}$ are generated with the following linear regression model: $Y_i={\beta }_c+X_i^{\top }{\beta }_x+{ϵ}_i$, where $X_i={(U_i^{\top },Z_i^{\top })}^{\top }$ is the p-dimensional covariate, ${\beta }_c$ is the constant term, ${\beta }_x$ is the p-dimensional vector of regression coefficients, the instrumental variables $Z_i={(z_{i,1},\dots ,z_{i,p-q})}^{\top }$ are drawn from the multivariate normal distribution $\mathcal{N}(0,\Sigma )$, i.e., $Z_i\sim \mathcal{N}(0,\Sigma )$, covariates $U_i={(u_{i,1},\dots ,u_{i,q})}^{\top }$ and $u_{i,j}\sim \mathcal{N}(10z_{i,j},1)$ for $j=1,\dots ,q$ ($2q=p$), and ${ϵ}_i$’s follow the standard normal distribution $\mathcal{N}(0,1)$ for $i=1,\dots ,n$. The true value ${\beta }_0$ of $\beta ={({\beta }_c,{\beta }_x^{\top })}^{\top }\in {\mathbb{R}}^{p+1}$ is taken as ${\beta }_0={(1.0,0.6,0.5,0.0,\dots ,0.0)}^{\top }$, which implies that there are two non-zero elements and $p-2$ zero elements in ${\beta }_x$, corresponding to two active and $p-2$ inactive covariates. The true value of the $(k,l)$th component of $\Sigma =\left({\rho }_{kl}\right)$ is taken to be ${\rho }_{kl}=0.5^{|k-l|}$ for $k,l=1,\dots ,p$. Here, we assume that covariates $x_i$ are completely observed, while responses $y_i$ are subject to missingness. To this end, we regard ${\delta }_i$ as the missing indicator for $Y_i$, i.e., ${\delta }_i=0$ if $Y_i$ is missing and ${\delta }_i=1$ if $Y_i$ is observed, and denote ${\pi }_i(U_i;\gamma )=\mathrm{Pr}({\delta }_i=1|U_i)$. To create missing data for response $y_i$, we consider the following propensity score functions:

(M1) $\mathrm{logit}\left\{{\pi }_i(U_i;\gamma )\right\}={\gamma }_c+U_i^{\top }{\gamma }_u$, where $U_i={(u_{i1},\dots ,u_{iq})}^{\top }$ are simulated from the above normal distribution, and $\gamma ={({\gamma }_c,{\gamma }_u^{\top })}^{\top }={({\gamma }_0,{\gamma }_1,\dots ,{\gamma }_q)}^{\top }$, logit($b)=log\{b/(1-b)\}$;

(M2) ${\pi }_i(U_i;\gamma )=0.5$ if ${\gamma }_c+U_i^{\top }{\gamma }_u\le a$, and ${\pi }_i(U_i;\gamma )=0.5$ if ${\gamma }_c+U_i^{\top }{\gamma }_u>a$;

(M3) ${\pi }_i(U_i;\gamma )=\Phi ({\gamma }_c+U_i^{\top }{\gamma }_u)$, where $\Phi (·)$ is the cumulative distribution function of the standard normal distribution.

The true value ${\gamma }_0$ of $\gamma \in {\mathbb{R}}^{q+1}$ is taken as ${\gamma }_0={(1.0,0.8,0.5,0.0,\dots ,0.0)}^{\top }$, which indicates that there are three non-zero components and $q-2$ zero elements in ${\gamma }_0$ corresponding to two active predictors and $q-2$ inactive predictors. To illustrate the proposed method, we consider the following unconditional moments: $g(D_i;\beta )=Z_i(Y_i-{\beta }_c-Z_i^{\top }{\beta }_x)$ and

$$\phi (U_i,Z_i;\gamma )=\left({\displaystyle \frac{{\delta }_i}{{\pi }_i(U_i;\gamma )}}-1\right)Z_i,i=1,\dots ,n$$

with ${\pi }_i(U_i;\gamma )$ specified in model (M1), which implies that (M1) is a correctly specified model and (M2) and (M3) are the misspecified models that are used to show the robustness of the proposed methods, and $E\left\{g(D_i;{\beta }_0)\right\}=0$ and $E\left\{\phi (U_i,Z_i;{\theta }_0)\right\}=0$.

We consider five combinations of $(n,p+1,q+1)$: $(n,p,q)=(50,9,5)$, $(100,13,7)$, $(200,17,9)$, $(500,23,12)$, and $(500,101,51)$, where p is the even-integer part of ${\left(5n\right)}^{{\displaystyle \frac{2}{5}}}$ or $9{\left(n\right)}^{{\displaystyle \frac{2}{5}}}-8$, and $q=p/2$. The tuning parameter $\tau$ is selected with the BIC, and the SCAD is taken as the penalty function $p_{\nu }(·)$. For each of the five combinations for $(n,p,q)$, we simulate 1000 datasets, which are used to estimate parameters $\theta =\{\beta ,\gamma \}$ via the proposed method. We calculate the bias, standard deviation (SD), and root mean square error (RMSE) of parameters $\theta$ for assessing the accuracy of the proposed method, where bias=$T^{-1}{\sum }_{t=1}^T{\theta }_j^{\left(t\right)}-{\theta }_{0j}$ (i.e., the difference between the true value and the mean of the estimates based on $T=1000$ replications), SD=$\sqrt{T^{-1}{\sum }_{t=1}^T{({\theta }_j^{\left(t\right)}-{\overline{\theta }}_j)}^2}$ (i.e., the standard deviation of $T=1000$ estimates), and $\mathrm{RMSE}=\sqrt{T^{-1}{\sum }_{t=1}^T{({\theta }_j^{\left(t\right)}-{\theta }_{0j})}^2}$ (i.e., the root mean square error of estimates obtained with $T=1000$ replications and its true value) and ${\overline{\theta }}_j=T^{-1}{\sum }_{t=1}^T{\theta }_j^{\left(t\right)}$, in which ${\theta }_j^{\left(t\right)}$ is the penalized ET likelihood estimate of the jth component of $\theta$ for the tth replication for $j=1,\dots ,p+q+2$. To assess the oracle property of the proposed variable selection method, we evaluate the average number of correctly estimated zero components (denoted as ‘TP’) and the average number of incorrectly estimated zero components (denoted as ‘FP’). Also, to assess the model fitting effect, we calculate the average number of under-fitting (denoted as ‘UF’), correctly-fitting (denoted as ‘CF’), and over-fitting (denoted as ‘OF’) for 1000 repeated simulations, where under-fitting refers to the case that some zero-components are incorrectly estimated as non-zero, over-fitting refers to the case that some non-zero-components are incorrectly estimated to be zero and all zero-components are correctly estimated to be zero, while correctly-fitting refers to the case that all zero-components are correctly estimated to be zero and all non-zero-components are correctly estimated as non-zero. The results for bias, SD, and RMSE values of non-zero parameters in $\theta$ and variable selection are given in

Table 1. Bias, SD, and RMSE values of non-zero parameters and performance of variable selection in Experiment 1.

		Bias			SD			RMSE
$(\mathit{n},\mathit{p}+\mathbf{1})$	Case	${\widehat{\mathit{\beta }}}_{\mathit{c}}$	${\widehat{\mathit{\beta }}}_{\mathit{x},\mathbf{1}}$	${\widehat{\mathit{\beta }}}_{\mathit{x},\mathbf{2}}$	${\widehat{\mathit{\beta }}}_{\mathit{c}}$	${\widehat{\mathit{\beta }}}_{\mathit{x},\mathbf{1}}$	${\widehat{\mathit{\beta }}}_{\mathit{x},\mathbf{2}}$	${\widehat{\mathit{\beta }}}_{\mathit{c}}$	${\widehat{\mathit{\beta }}}_{\mathit{x},\mathbf{1}}$	${\widehat{\mathit{\beta }}}_{\mathit{x},\mathbf{2}}$	TP	FP	UF	CF	OF
(50,9)	M1	−0.0256	0.0811	−0.0285	0.841	0.863	0.313	0.842	0.867	0.315	5.779	0.141	0.196	0.697	0.107
	M2	−0.0465	−0.0255	0.1355	0.537	0.501	0.870	0.539	0.502	0.880	5.576	0.165	0.334	0.563	0.103
	M3	−0.0001	−0.0132	0.0427	0.851	0.793	0.822	0.851	0.793	0.823	5.689	0.139	0.262	0.639	0.099
(100,13)	M1	0.0464	−0.0228	−0.0195	0.750	0.146	0.135	0.751	0.148	0.137	9.785	0.101	0.174	0.752	0.074
	M2	−0.0714	−0.0357	−0.0548	0.378	0.741	0.708	0.384	0.742	0.710	9.584	0.142	0.297	0.611	0.092
	M3	0.0487	−0.0118	−0.0230	0.803	0.221	0.210	0.805	0.221	0.211	9.752	0.095	0.196	0.736	0.068
(200,17)	M1	−0.0175	−0.0044	−0.0042	0.153	0.088	0.088	0.154	0.088	0.089	13.905	0.039	0.082	0.881	0.037
	M2	−0.0293	−0.0193	−0.0158	0.186	0.141	0.125	0.188	0.143	0.126	13.811	0.081	0.137	0.794	0.069
	M3	−0.0234	−0.0288	−0.0081	0.168	0.218	0.118	0.169	0.220	0.118	13.850	0.068	0.116	0.825	0.059
(500,23)	M1	0.0000	0.0000	0.0001	0.005	0.005	0.005	0.005	0.005	0.005	19.975	0.000	0.022	0.978	0.000
	M2	−0.0050	0.0001	0.0014	0.094	0.084	0.077	0.094	0.084	0.077	19.929	0.015	0.061	0.927	0.012
	M3	−0.0003	−0.0004	0.0003	0.009	0.009	0.009	0.009	0.009	0.009	19.970	0.000	0.028	0.972	0.000
(500,101)	M1	−0.0004	−0.0005	−0.0001	0.050	0.010	0.010	0.050	0.010	0.010	97.934	0.000	0.040	0.960	0.000
	M2	−0.0068	−0.0013	−0.0036	0.096	0.073	0.069	0.096	0.074	0.069	97.703	0.024	0.145	0.834	0.021
	M3	−0.0044	0.0043	−0.0031	0.077	0.076	0.046	0.077	0.076	0.046	97.902	0.012	0.063	0.925	0.012
		Bias			SD			RMSE
$(\mathit{n},\mathit{q}+\mathbf{1})$	Case	${\widehat{\mathit{\gamma }}}_{\mathit{c}}$	${\widehat{\mathit{\gamma }}}_{\mathit{u},\mathbf{1}}$	${\widehat{\mathit{\gamma }}}_{\mathit{u},\mathbf{2}}$	${\widehat{\mathit{\gamma }}}_{\mathit{c}}$	${\widehat{\mathit{\gamma }}}_{\mathit{u},\mathbf{1}}$	${\widehat{\mathit{\gamma }}}_{\mathit{u},\mathbf{2}}$	${\widehat{\mathit{\gamma }}}_{\mathit{c}}$	${\widehat{\mathit{\gamma }}}_{\mathit{u},\mathbf{1}}$	${\widehat{\mathit{\gamma }}}_{\mathit{u},\mathbf{2}}$	TP	FP	UF	CF	OF
(50,5)	M1	−0.0950	0.0261	−0.0159	0.369	0.786	0.296	0.381	0.786	0.297	1.905	0.176	0.093	0.747	0.160
	M2	−0.0680	−0.0516	−0.0246	0.570	0.526	0.513	0.574	0.528	0.513	1.864	0.174	0.128	0.732	0.140
	M3	0.0055	−0.0391	−0.0298	0.786	0.481	0.459	0.786	0.483	0.460	1.871	0.103	0.123	0.790	0.087
(100,7)	M1	−0.0410	0.0457	0.0849	0.229	0.795	0.791	0.232	0.796	0.796	3.923	0.112	0.075	0.825	0.100
	M2	−0.0368	0.0230	−0.0212	0.445	0.852	0.410	0.446	0.852	0.411	3.818	0.141	0.161	0.727	0.112
	M3	−0.0687	−0.1052	−0.0268	0.727	0.712	0.322	0.730	0.720	0.323	3.900	0.138	0.094	0.785	0.121
(200,9)	M1	−0.0263	−0.0240	−0.0074	0.199	0.138	0.163	0.200	0.140	0.163	5.955	0.066	0.042	0.893	0.065
	M2	−0.0324	0.0002	0.0036	0.556	0.551	0.544	0.556	0.551	0.544	5.920	0.062	0.072	0.873	0.055
	M3	−0.0229	−0.0188	−0.0089	0.175	0.148	0.110	0.177	0.150	0.110	5.954	0.066	0.040	0.899	0.061
(500,12)	M1	0.0001	−0.0005	−0.0000	0.006	0.006	0.006	0.006	0.006	0.006	8.981	0.000	0.019	0.981	0.000
	M2	−0.0143	−0.0147	−0.0127	0.119	0.111	0.080	0.120	0.112	0.081	8.971	0.058	0.028	0.917	0.055
	M3	−0.0002	0.0001	−0.0003	0.008	0.008	0.008	0.008	0.008	0.008	8.989	0.000	0.011	0.989	0.000
(500,51)	M1	−0.0195	−0.0138	−0.0090	0.134	0.107	0.065	0.135	0.108	0.065	47.957	0.052	0.035	0.915	0.050
	M2	−0.0068	−0.0073	−0.0049	0.092	0.092	0.060	0.093	0.092	0.060	47.862	0.026	0.083	0.894	0.023
	M3	−0.0173	−0.0160	−0.0023	0.168	0.109	0.053	0.169	0.110	0.053	47.900	0.032	0.070	0.901	0.029

. An examination of Table 1 shows that (i) bias values of all the parameter estimates are relatively small and SD and RMSE values of all the parameter estimates are almost identical; (ii) as the sample size n increases, the average number of correctly estimated zero components tends towards $100\%$ and the average number of incorrectly estimated zero components tends towards 0, implying that the proposed variable selection method performs well; (iii) as the sample size n increases, UF and OF tend towards 0 and CF tends towards $100\%$; and (iv) although UF, CF, and OF values corresponding to misspecified models (e.g., M2 and M3) are larger than those corresponding to the correctly specified model (e.g., M1), their differences are relatively small, implying that the proposed penalized ET likelihood method is robust to the misspecified model.

Experiment 2 (logistic model). The data $\{Y_i:i=1,\dots ,n\}$ are generated from the following logistic model: $Y_i\sim \mathrm{Bernoulli}\left(p_i\right)$ with $\mathrm{logit}\left(p_i\right)={\beta }_c+X_i^{\top }{\beta }_x$, where $X_i={(U_i^{\top },Z_i^{\top })}^{\top }$ is the p-dimensional covariates, the instrumental variables $Z_i\sim {\mathcal{N}}_p(0,\mathbf{I})$, where $\mathbf{I}$ is a $p\times p$ identity matrix, and $\beta ={({\beta }_c,{\beta }_x^{\top })}^{\top }$. The q-dimensional covariates $U_i={(u_{i,1},\dots ,u_{i,q})}^{\top }$ and $u_{i,j}\sim \mathcal{N}(z_{i,j},100)$ for $j=1,\dots ,q$ ($2q=p$). Similarly to experiment 1, we assume that $X_i$’s are fully observed, while $Y_i$’s are subject to missingness and missing indicators ${\delta }_i$ for responses $Y_i$ are simulated from the Bernoulli distribution with the propensity score function ${\pi }_i(U_i;\gamma )$ and $\mathrm{logit}\left\{{\pi }_i(U_i;\gamma )\right\}={\gamma }_c+U_i^{\top }{\gamma }_u$, where $\gamma ={({\gamma }_c,{\gamma }_u^{\top })}^{\top }$. Here, we consider the same true value of $\{\theta ,\gamma \}$ as those given in experiment 1, four combinations of triples $(n,p+1,q+1)$: $(n,p+1,q+1)=(50,9,5)$, $(100,13,7)$, $(200,17,9)$, and $(500,101,51)$, where p is the even-integer part of ${\left(5n\right)}^{{\displaystyle \frac{2}{5}}}$ or $9n^{{\displaystyle \frac{2}{5}}}-8$ and $q=p/2$. As an illustration of the proposed method, we consider the following unconditional moment constraints:

$$g(D_i;\beta )={(1,Z_i)}^{\top }\left\{Y_i-{\displaystyle \frac{exp({\beta }_c+Z_i^{\top }{\beta }_x)}{1+exp({\beta }_c+Z_i^{\top }{\beta }_x)}}\right\},i=1,\dots ,n.$$

The missing mechanism model for fitting data is given in Equation (1). We select the tuning parameter $\tau$ with the BIC criterion and take the SCAD as the penalty function. For simplicity, let ${\omega }^1=n^{-1}{\sum }_{i=1}^n{(1,Z_i)}^{-\top }log({\tilde{p}}_i/(1-{\tilde{p}}_i))$, ${\omega }^2=n^{-1}{\sum }_{i=1}^n{(1,U_i)}^{-\top }log({\tilde{\pi }}_i/(1-{\tilde{\pi }}_i))$ and $\omega ={({\omega }^{1\top },{\omega }^{2\top })}^{\top }$ is a s-dimensional vector, where ${\tilde{p}}_i=n^{-1}{\sum }_{i=1}^n{y}_i$, ${\tilde{\pi }}_i=n^{-1}{\sum }_{i=1}^n{\delta }_i$, ${\omega }_j$ is the jth component of $\omega$, $I\left(A\right)$ is an indicator function of the event A, and ${\left\{B\right\}}_{+}=\max (0,B)$. For comparison, we consider the hard-threshold (HT) estimator ${\widehat{\theta }}_j^{\mathrm{HT}}={\omega }_{j}I(|{\omega }_j|<{\varsigma }_1)$, the soft-threshold (ST) estimator ${\widehat{\theta }}_j^{\mathrm{ST}}=\mathrm{sign}\left({\omega }_j\right)\{|{\omega }_j{|-{\varsigma }_2\}}_{+}$, and the penalized least squares estimator with the SCAD penalty function (denoted as ‘PLS’). In implementing the proposed method (denoted as ‘PETL’), we evaluate the tuning parameters $\{{\varsigma }_1,{\varsigma }_2\}$ with the five-fold cross-validation method. The corresponding results of non-zero parameters in $\beta$ and $\gamma$ and variable selection for 1000 replications are given in

Table 2. Bias, SD, and RMSE values of non-zero parameters and performance of variable selection in Experiment 2.

		Bias			SD			RMSE
$(\mathit{n},\mathit{p}+\mathbf{1})$	Case	${\widehat{\mathit{\beta }}}_{\mathit{c}}$	${\widehat{\mathit{\beta }}}_{\mathit{x},\mathbf{1}}$	${\widehat{\mathit{\beta }}}_{\mathit{x},\mathbf{2}}$	${\widehat{\mathit{\beta }}}_{\mathit{c}}$	${\widehat{\mathit{\beta }}}_{\mathit{x},\mathbf{1}}$	${\widehat{\mathit{\beta }}}_{\mathit{x},\mathbf{2}}$	${\widehat{\mathit{\beta }}}_{\mathit{c}}$	${\widehat{\mathit{\beta }}}_{\mathit{x},\mathbf{1}}$	${\widehat{\mathit{\beta }}}_{\mathit{x},\mathbf{2}}$	TP	FP	UF	CF	OF
(50,9)	PETL	−0.0200	−0.0327	−0.0237	0.843	0.218	0.220	0.844	0.221	0.222	5.760	0.124	0.208	0.698	0.094
	HT	0.0013	−0.0486	0.0221	0.958	0.591	0.910	0.958	0.593	0.911	4.103	0.330	0.717	0.200	0.083
	ST	−0.0346	−0.0349	0.0358	0.923	0.848	0.884	0.924	0.849	0.885	2.961	0.222	0.854	0.114	0.032
	PLS	−0.1264	−0.0767	−0.0232	0.762	0.974	0.732	0.772	0.977	0.732	4.507	0.365	0.699	0.214	0.087
(100,13)	PETL	0.0477	−0.0238	−0.0195	0.749	0.227	0.218	0.750	0.228	0.219	9.792	0.089	0.167	0.767	0.066
	HT	−0.1052	−0.0493	0.0476	0.482	0.421	0.813	0.494	0.424	0.815	7.994	0.238	0.530	0.365	0.105
	ST	−0.0732	−0.0083	−0.0501	0.538	0.497	0.489	0.543	0.497	0.491	4.475	0.157	0.774	0.196	0.030
	PLS	−0.0250	−0.0364	0.0230	0.868	0.565	0.832	0.869	0.566	0.833	8.799	0.326	0.523	0.342	0.135
(200,17)	PETL	0.0063	0.0006	0.0017	0.165	0.081	0.080	0.165	0.081	0.080	13.909	0.000	0.074	0.926	−0.000
	HT	−0.0558	−0.0372	−0.0278	0.277	0.479	0.200	0.283	0.480	0.202	12.867	0.143	0.287	0.616	0.097
	ST	−0.0303	−0.0310	−0.0268	0.191	0.375	0.376	0.193	0.377	0.377	8.295	0.090	0.603	0.363	0.034
	PLS	−0.0275	−0.0138	−0.0102	0.182	0.136	0.123	0.184	0.137	0.123	13.485	0.083	0.206	0.734	0.060
(500,101)	PETL	0.0000	−0.0004	−0.0001	0.008	0.008	0.008	0.008	0.008	0.008	97.984	0.000	0.012	0.988	0.000
	HT	−0.0286	−0.0124	−0.0181	0.181	0.121	0.119	0.183	0.122	0.120	94.261	0.096	0.138	0.781	0.081
	ST	−0.0075	−0.0018	−0.0054	0.118	0.080	0.081	0.118	0.080	0.081	74.997	0.021	0.450	0.536	0.014
	PLS	−0.0537	−0.0404	−0.0346	0.236	0.158	0.200	0.242	0.163	0.203	95.283	0.195	0.305	0.565	0.130
		Bias			SD			RMSE
$(\mathit{n},\mathit{q}+\mathbf{1})$	Case	${\widehat{\mathit{\gamma }}}_{\mathit{c}}$	${\widehat{\mathit{\gamma }}}_{\mathit{u},\mathbf{1}}$	${\widehat{\mathit{\gamma }}}_{\mathit{u},\mathbf{2}}$	${\widehat{\mathit{\gamma }}}_{\mathit{c}}$	${\widehat{\mathit{\gamma }}}_{\mathit{u},\mathbf{1}}$	${\widehat{\mathit{\gamma }}}_{\mathit{u},\mathbf{2}}$	${\widehat{\mathit{\gamma }}}_{\mathit{c}}$	${\widehat{\mathit{\gamma }}}_{\mathit{u},\mathbf{1}}$	${\widehat{\mathit{\gamma }}}_{\mathit{u},\mathbf{2}}$	TP	FP	UF	CF	OF
(50,5)	PETL	−0.0852	0.0307	−0.0106	0.331	0.785	0.247	0.342	0.785	0.247	1.904	0.152	0.094	0.768	0.138
	HT	−0.1180	−0.0948	−0.0058	0.622	0.565	0.694	0.633	0.572	0.694	1.332	0.331	0.505	0.342	0.153
	ST	−0.0448	−0.0687	−0.0474	0.885	0.683	0.649	0.886	0.687	0.650	1.076	0.201	0.691	0.250	0.059
	PLS	−0.1135	−0.0745	−0.0625	0.776	0.762	0.763	0.784	0.766	0.766	1.586	0.274	0.345	0.489	0.166
(100,7)	PETL	−0.0476	−0.0422	−0.0232	0.299	0.267	0.247	0.302	0.270	0.248	3.923	0.127	0.074	0.814	0.112
	HT	−0.0459	−0.0020	−0.0367	0.785	0.802	0.395	0.787	0.802	0.397	3.203	0.233	0.461	0.424	0.115
	ST	0.0010	−0.0369	−0.0412	0.810	0.800	0.485	0.810	0.801	0.487	1.767	0.191	0.765	0.201	0.034
	PLS	−0.1247	−0.1587	−0.1148	0.870	0.794	0.778	0.879	0.810	0.786	3.548	0.343	0.316	0.459	0.225
(200,9)	PETL	−0.0161	−0.0239	−0.0121	0.144	0.140	0.097	0.145	0.142	0.097	5.955	0.058	0.040	0.902	0.058
	HT	−0.0386	−0.0296	−0.0288	0.261	0.233	0.225	0.264	0.235	0.227	5.433	0.125	0.276	0.635	0.089
	ST	−0.0331	−0.0295	−0.0151	0.205	0.179	0.131	0.208	0.182	0.132	4.041	0.103	0.540	0.420	0.040
	PLS	−0.0366	−0.0219	−0.0215	0.207	0.164	0.132	0.210	0.166	0.133	5.772	0.106	0.150	0.764	0.086
(500,51)	PETL	−0.0009	0.0003	0.0021	0.009	0.009	0.033	0.009	0.009	0.033	47.991	0.000	0.007	0.993	0.000
	HT	−0.0259	−0.0283	−0.0235	0.167	0.161	0.113	0.169	0.163	0.116	46.242	0.108	0.127	0.779	0.094
	ST	−0.0366	−0.0220	−0.0045	0.334	0.132	0.082	0.336	0.134	0.082	35.787	0.052	0.447	0.527	0.026
	PLS	−0.0434	−0.0359	−0.0240	0.226	0.179	0.127	0.230	0.183	0.130	46.535	0.146	0.298	0.602	0.100

.

An examination of Table 2 shows that (i) the proposed method and the competing methods have relatively small Bias values and negligible differences between SD and RMSE values, regardless of the sample size and the dimensionality, which implies that the compared four methods can well estimate non-zero parameters; (ii) the proposed method has larger TP and CF values and smaller FP and UF values than the compared three methods, regardless of the sample size and the dimensionality, which implies that the proposed method behaves better than the compared three methods; (iii) as the sample size increases, the TP and CF values of the proposed method approach $p-3$, while its FP, UF, and OF values approach 0 and its CF values are close to one; (iv) the HT and ST methods for variable selection perform poorly due to the lack of the penalty function; and (v) when the sample size is small, the PLS method for variable selection behaves unsatisfactorily, even if the SCAD penalty function is used. In a word, the proposed method outperforms the other three methods, regardless of parameter estimation and variable selection.

Experiment 3 (structural equation model). To investigate the performance of the proposed method, we consider a more complicated model (e.g., structural equation model), which is specified by

$$Y_i=B{X}_i+{ϵ}_i,X_i=C{X}_i+{\epsilon }_i,i=1,\dots ,n$$

where $Y_i$ is an e-dimensional vector of manifest variables, $X_i={(U_i^{\top },Z_i^{\top })}^{\top }$ is a f-dimensional vector of latent variables, where $U_i=(u_{i,1},\dots ,u_{i,h})$ is an h-dimensional vector, $Z_i=(z_{i,1},\dots ,z_{i,f-h})$ is a $f-h$-dimensional instrumental variables and $f=2h$, B is a $e\times f$ factor loading matrix, C is a $f\times f$ coefficient matrix used to identify the dependence structure between latent variables, and ${ϵ}_i$ and ${\epsilon }_i$ are measurement errors. It is assumed that ${ϵ}_i$’s follow the multivariate normal distribution with zero mean and covariance ${\Sigma }_{ϵ}=\mathrm{diag}({\alpha }_1,\dots ,{\alpha }_e)$, i.e., ${ϵ}_i\sim \mathcal{N}(0,{\Sigma }_{ϵ})$, ${\epsilon }_i={({\epsilon }_{u,i}^{\top },{\epsilon }_{z,i}^{\top })}^{\top }$, where ${\epsilon }_{u,i}$ is an h-dimensional zero vector and ${\epsilon }_{z,i}$’s follow the multivariate normal distribution $\mathcal{N}(0,{\Sigma }_{\epsilon })$, where ${\Sigma }_{\epsilon }=\mathrm{diag}({\zeta }_1,\dots ,{\zeta }_{f-h})$ and $e=2f$. The dataset ${\left\{Y_i\right\}}_{i=1}^n$ is generated from the structural equation model introduced above, with the following specification of B and C:

$$B=\left(\begin{array}{ccc}{\beta }_{1,1}& \cdots & {\beta }_{1,f}\\ ⋮& \ddots & ⋮\\ {\beta }_{e,1}& \cdots & {\beta }_{e,f}\end{array}\right),C=\left(\begin{array}{cc}0& C_u\\ 0& C_z\end{array}\right)$$

where

$$C_u=\left(\begin{array}{ccccc}1& 0.5& \cdots & 0.0& 0.0\\ -0.5& 1& \cdots & 0.0& 0.0\\ ⋮& ⋮& \ddots & ⋮& ⋮\\ 0.0& 0.0& \cdots & 1& 0.5\\ 0.0& 0.0& \cdots & -0.5& 1\end{array}\right),C_z=\left(\begin{array}{ccccc}1& 0.8& \cdots & 0.0& 0.0\\ -0.8& 1& \cdots & 0.0& 0.0\\ ⋮& ⋮& \ddots & ⋮& ⋮\\ 0.0& 0.0& \cdots & 1& 0.8\\ 0.0& 0.0& \cdots & -0.8& 1\end{array}\right)$$

are the $h\times (f-h)$ and $(f-h)\times (f-h)$ matrices, respectively. Here, we take the true values of ${\zeta }_m$ and ${\alpha }_k$ as $0.8$ for $m=1,\dots ,f-h$ and $k=1,\dots ,e$, and set the true value of ${\beta }_{j_1,j_2}$ to be $0.6$ for $|j_1-j_2|=1$ and 0.0 otherwise, which indicates that the number of non-zero parameters is $2(f-1)$. Because ${\beta }_{k,l}$ measures the dependence of manifest variables $Y_{i,k}$ on latent variable $X_{i,l}$, we only regard ${\beta }_{k,l}$ as a parameter of interest, which indicates that $p=ef=2f^2$. To create the missing values of components of $Y_i$ for $i=1,\dots ,n$, we consider the missingness data mechanism model given in Equation (1). The true value of the q-dimensional parameter vector $\gamma ={({\gamma }_c,{\gamma }_u^{\top })}^{\top }$ in Equation (1) is taken as $\gamma =(0.8,-0.8,0.0,\dots ,0.0)$ and $q=h+1$. As an illustration of the proposed method, we consider the following unconditional moment constraints:

$$\begin{array}{c}\hfill g(Y_i;\beta )=\mathrm{vech}\{Y_i{Y}_i^{\top }-B{(I-C)}^{-1}{\Sigma }_{ϵ}{(I-C)}^{-\top }{B}^{\top }+{\Sigma }_{\epsilon }\},\end{array}$$

(10)

which satisfy $E[g(Y_i;{\beta }_0)]=0$ for $i=1,\dots ,n$, where ${\beta }_0$ is the true value of $\beta$, and $\mathrm{vech}\left(H\right)$ represents the half-vectorization of matrix H. In this case, the number of unconditional moment constraints is $t=f(2f+1)+h$, which shows that $g(Y_i;\beta )$’s are over-identification conditions (i.e., $t>s$). Under the settings specified above, we consider the following three combinations of the number of parameters of interest p, the number of parameters q in missingness data mechanism model (1), and the sample size n: $(n,p,q)=(117,8,2),(241,32,3),(602,72,4)$, where n is taken as the integer of ${((p+80)/18)}^3$. For comparison, we also consider the following two competing methods: generalized method of moments (GMM) and penalized empirical likelihood (PEL). For each of combinations of $(n,p,q)$, we generate 1000 datasets. For each of the simulated 1000 datasets, we utilize the proposed method and the considered two competing methods to estimate parameters and select variable selection. In implementing the proposed method, we use BIC criterion to select the tuning parameters $\tau$. Bias, standard deviation (SD), and root mean square error (RMSE) values of the non-zero component of C for 1000 replications are reported in

Table 3. Bias, SD, and RMSE values of parameter estimates for penalized ET likelihood (PETL), GMM, and PEL in Experiment 3.

		PETL					GMM					PEL
$(\mathit{n},\mathit{p})$	Par	Bias	SD	RMSE	AL	CP	Bias	SD	RMSE	AL	CP	Bias	SD	RMSE	AL	CP
(117,8)	${\widehat{\beta }}_{12}$	−0.0132	0.142	0.143	1.462	97.4	−0.0323	0.316	0.317	2.646	88.4	0.0065	0.549	0.549	1.279	81.2
	${\widehat{\beta }}_{21}$	−0.0208	0.154	0.155	1.952	97.8	−0.0349	0.320	0.322	2.659	88.7	−0.0253	0.155	0.157	1.501	93.7
	${\widehat{\beta }}_{32}$	−0.0233	0.161	0.162	1.945	97.5	−0.0245	0.331	0.332	1.197	81.0	−0.0229	0.151	0.152	1.269	93.5
(241,32)	${\widehat{\beta }}_{12}$	−0.0064	0.069	0.069	0.878	97.1	−0.0319	0.262	0.264	2.912	91.6	−0.0101	0.105	0.105	1.312	93.5
	${\widehat{\beta }}_{21}$	−0.0029	0.068	0.068	0.618	97.6	−0.0290	0.146	0.148	2.994	94.4	−0.0147	0.109	0.110	1.936	94.9
	${\widehat{\beta }}_{23}$	−0.0043	0.066	0.066	0.612	96.8	−0.0360	0.160	0.164	2.945	92.5	−0.0135	0.208	0.208	1.960	96.2
	${\widehat{\beta }}_{32}$	0.0032	0.174	0.174	1.177	97.7	−0.0214	0.259	0.259	1.981	94.4	−0.0095	0.099	0.099	1.908	93.8
	${\widehat{\beta }}_{34}$	0.0000	0.172	0.172	0.884	95.7	−0.0332	0.149	0.153	1.959	93.2	−0.0131	0.114	0.115	1.885	92.5
	${\widehat{\beta }}_{43}$	0.0103	0.177	0.178	1.178	97.6	−0.0242	0.267	0.268	1.942	92.5	−0.0034	0.232	0.232	1.935	94.8
	${\widehat{\beta }}_{54}$	−0.0046	0.076	0.076	0.875	96.2	−0.0240	0.136	0.138	2.915	91.7	−0.0098	0.231	0.231	1.927	94.3
(602,72)	${\widehat{\beta }}_{12}$	−0.0007	0.029	0.029	0.598	99.8	−0.0106	0.117	0.117	1.394	92.9	−0.0082	0.073	0.074	1.427	95.1
	${\widehat{\beta }}_{21}$	0.0009	0.030	0.030	0.595	99.3	−0.0053	0.058	0.058	1.384	92.3	−0.0120	0.084	0.085	1.429	95.2
	${\widehat{\beta }}_{23}$	0.0006	0.030	0.030	0.599	99.7	−0.0106	0.078	0.079	0.579	90.2	−0.0071	0.066	0.066	1.008	95.0
	${\widehat{\beta }}_{32}$	−0.0001	0.029	0.029	0.442	98.1	−0.0059	0.061	0.061	0.584	91.8	−0.0084	0.073	0.074	0.999	93.7
	${\widehat{\beta }}_{34}$	−0.0001	0.031	0.031	0.315	99.8	−0.0068	0.063	0.064	0.986	93.0	−0.0061	0.060	0.060	0.601	94.5
	${\widehat{\beta }}_{43}$	0.0005	0.030	0.030	0.450	99.8	−0.0073	0.118	0.118	0.600	94.3	−0.0056	0.060	0.061	1.003	94.7
	${\widehat{\beta }}_{45}$	0.0061	0.132	0.132	0.443	91.2	−0.0057	0.061	0.061	0.591	92.8	−0.0087	0.071	0.071	0.606	94.9
	${\widehat{\beta }}_{54}$	−0.0021	0.129	0.129	0.439	91.4	−0.0082	0.069	0.069	1.426	95.1	−0.0072	0.066	0.066	1.009	95.0
	${\widehat{\beta }}_{56}$	0.0010	0.030	0.030	0.439	97.5	−0.0069	0.063	0.064	0.990	93.4	−0.0083	0.073	0.074	1.003	94.2
	${\widehat{\beta }}_{65}$	0.0033	0.029	0.030	0.442	98.3	−0.0042	0.051	0.051	1.386	92.4	−0.0044	0.050	0.050	1.440	96.0
	${\widehat{\beta }}_{76}$	0.0006	0.029	0.029	0.440	98.1	−0.0075	0.066	0.066	0.994	93.4	−0.0047	0.057	0.057	1.007	95.2

. Also, the TP, FP, UF, CF, and OF values for identifying non-zero components of C are given in

Table 4. TP, FP, UF, CF, and OF values of variable selection for penalized ET likelihood (PETL), GMM, and PEL in Experiment 3.

	PETL					GMM					PEL
$(\mathit{n},\mathit{p},\mathit{q})$	TP	FP	UF	CF	OF	TP	FP	UF	CF	OF	TP	FP	UF	CF	OF
(117,8,2)	4.878	0.114	0.112	0.794	0.094	4.703	0.156	0.256	0.630	0.114	4.783	0.138	0.195	0.696	0.109
(241,32,3)	24.840	0.034	0.122	0.851	0.027	24.404	0.358	0.362	0.441	0.197	24.354	0.141	0.343	0.566	0.091
(602,72,4)	60.866	0.000	0.081	0.919	0.000	60.277	0.118	0.225	0.689	0.086	60.253	0.128	0.228	0.686	0.086

.

An examination of Table 3 and Table 4 shows that (i) SD values of the parameter estimates are almost the same as their corresponding RMSE values for the considered three methods, regardless of the sample sizes, which implies that the considered three methods can estimate parameters well; (ii) the RMSE value decreases as the sample size increases for the considered three methods, which indicates that increasing the sample size can improve the performance of the considered three estimation methods; (iii) the proposed penalized ET likelihood estimation behaves better than the other two competing methods, in that the total RMSE value of the former is smaller than that of the latter, while the GMM estimation has the same performance as the PEL estimation, in that they have the same total RMSE value, indicating that the GMM and PEL methods have the same asymptotic properties [24]; (iv) and the PETL method behaves better than the other two methods in identifying non-zero components of C, in that the former has smaller FP, UF, and OF values and a larger CF value than the latter, and the TP value of the former is closer to the number of zero components in C (i.e., ${(f-1)}^2+1$) than that of the latter, regardless of the sample sizes.

5. Real Examples

As an illustration of the proposed method, here we consider thyroid data taken from the First People’s Hospital of Yunnan Province. In the data, thyroid nodule refers to any growth of thyroid cells that form a lump within the thyroid. Most thyroid nodules do not cause any symptoms. Mostly, a nodule cannot cause pain, difficulty swallowing or breathing, hoarseness, or symptoms of hyperthyroidism; these are usually defined as benign nodules. Here, we explore the relationship between malignant thyroid nodules and variables obtained with ultrasound (US) instruments and clinical characteristics, such as nodular echogenecity ($x_1$), thyroid background echogenecity ($x_2$), anterior cervical muscles echogenecity ($x_3$), echo ratio between nodular and anterior cervical muscles ($x_4$), nodule size ($x_5$), longitudinal diameter ($x_6$), horizontal diameter ($x_7$), aspect ratio ($x_8$), age ($x_9$), sex (1 = male, 0 = female; $x_{10}$), number of nodules (1 = more, 0 = less; $x_{11}$), shape (1 = irregularities, 0 = rules; $x_{12}$), margins (1 = unclear, 0 = clear; $x_{13}$), margins (1 = angled, 0 = non-angled; $x_{14}$), echogenecity (1 = uniform, 0 = uneven; $x_{15}$), internal structural solidity (1 = yes, 0 = no; $x_{16}$), internal structure cystic (1 = yes, 0 = no; $x_{17}$), internal structure cystic solid mixture (1 = yes, 0 = no; $x_{18}$), sound halo (1 = yes, 0 = no; $x_{19}$), rear attenuation (1 = yes, 0 = no; $x_{20}$), lateral acoustic shadow (1 = yes, 0 = no; $x_{21}$), bloodstream (1 = yes, 0 = no; $x_{22}$), blood supply branch (1 = yes, 0 = no; $x_{23}$), blood supply strip (1 = yes, 0 = no; $x_{24}$), blood supply punctate (1 = yes, 0 = no; $x_{25}$), and calcification (1 = yes, 0 = no; $x_{26}$). Thus, we take the aforementioned 26 variables as covariates, denoted as $x={(x_1,\dots ,x_{26})}^{\top }$, and regard the indicator of malignant thyroid nodules (1 = malignant, 0 = benign) as response variable y, i.e., $y=1$ if patient has the malignant thyroid nodules, and $y=0$ otherwise. In this study, the sample size is $n=2430$ and covariates in x are fully observed, while roughly $40\%$ of observations of response variable y are subject to missingness. Here, we assume that the missingness data mechanism for y is missing at random, in that y can be unobserved due to the presence of some covariates rather than judgement of benign or malignant thyroid nodules. Under the above assumption, we consider the following logistic regression model:

$$\mathrm{logit}\left\{\mathrm{Pr}(y=1|x)\right\}={\beta }_0+x_1{\beta }_1+\cdots +x_{26}{\beta }_{26},$$

which leads to the following unconditional moment constraints:

$$g(x_i,y_i;\theta )={(1,x_i)}^{\top }\left\{y_i-{\displaystyle \frac{exp({\beta }_0+x_{i1}{\beta }_1+\cdots +x_{i,26}{\beta }_{26})}{1+exp({\beta }_0+x_{i1}{\beta }_1+\cdots +x_{i,26}{\beta }_{26})}}\right\},$$

which satisfy $E\left\{g(x_i,y_i;\theta )\right\}=0$, where $\theta ={({\beta }_0,{\beta }_1,\dots ,{\beta }_{26})}^{\top }$ and $x_i={(x_{i,1},\dots ,x_{i,26})}^{\top }$ for $i=1,\dots ,n$. Let $U_i=(x_{i,1},x_{i,4},x_{i,8},x_{i,14},x_{i,15},x_{i,18},x_{i,25})$, and other components of $x_i$ are taken as instrumental variables (denoted as $U_i$), in that the components in $U_i$ can be indirectly determined by the components in $Z_i$; for example, when the blood supply is neither branching nor strip-shaped, it can be determined as point-shaped. For missing value of $y_i$’s, we consider a logistic regression model for propensity score function ${\pi }_i(U_i;\gamma )$, i.e., $\mathrm{logit}\left\{{\pi }_i(U_i;\gamma )\right\}={\gamma }_c+U_i^{\top }{\gamma }_u$ with $\gamma ={({\gamma }_c,{\gamma }_1,\dots ,{\gamma }_q)}^{\top }$ and $q=7$. To apply the preceding proposed method to the considered dataset, we consider the following unconditional moments for estimating unknown parameters in $\gamma$:

$$\phi (U_i,Z_i;\gamma )=\left({\displaystyle \frac{{\delta }_i}{{\pi }_i(U_i;\gamma )}}-1\right)Z_i,i=1,\dots ,n.$$

In utilizing the proposed method, we take the tuning parameter $\tau$ with the BIC criterion and consider three penalty functions, Lasso, Adaptive Lasso (represented by ALasso), and SCAD for comparison. We calculate the PETL estimates of unknown parameters and their corresponding $95\%$ confidence intervals.

An examination of

Table 5. PETL estimates (Est) and 95% confidence intervals (CI) of parameters under three penalty functions in real example.

	Lasso		ALasso		SCAD
Par.	Est	CI	Est	CI	Est	CI
${\widehat{\beta }}_1$	−2.864	(−3.515,−2.073)	−2.850	(−3.248,−2.455)	−2.929	(−3.223,−2.630)
${\widehat{\beta }}_2$	0.000	(−0.744,0.771)	−0.195	(−0.869,0.375)	−0.012	(−0.116,0.081)
${\widehat{\beta }}_3$	−0.082	(−0.927,0.801)	−0.046	(−0.256,0.152)	−0.134	(−0.241,−0.031)
${\widehat{\beta }}_4$	0.000	(−0.523,0.584)	0.040	(−0.640,0.614)	0.084	(−0.435,0.599)
${\widehat{\beta }}_5$	0.000	(−0.471,0.597)	0.000	(−0.656,0.600)	0.000	(−0.296,0.302)
${\widehat{\beta }}_6$	0.000	(−0.699,0.801)	0.128	(−0.505,0.727)	0.111	(−0.191,0.406)
${\widehat{\beta }}_7$	0.000	(−0.381,0.404)	0.282	(−0.282,0.871)	0.283	(−0.011,0.582)
${\widehat{\beta }}_8$	−3.118	(−3.643,−2.520)	−3.104	(−3.728,−2.491)	−3.139	(−3.636,−2.629)
${\widehat{\beta }}_9$	0.000	(−1.056,0.742)	0.429	(−0.223,1.006)	0.469	(0.169,0.769)
${\widehat{\beta }}_{10}$	0.000	(−0.592,0.625)	0.000	(−0.416,0.407)	0.000	(−0.505,0.496)
${\widehat{\beta }}_{11}$	5.920	(5.374,6.541)	5.801	(5.607,5.984)	5.865	(5.371,6.371)
${\widehat{\beta }}_{12}$	6.205	(5.570,6.824)	6.343	(5.817,6.897)	6.207	(5.904,6.505)
${\widehat{\beta }}_{13}$	4.952	(4.428,5.586)	4.893	(4.530,5.304)	4.939	(4.641,5.243)
${\widehat{\beta }}_{14}$	4.284	(3.599,5.111)	4.338	(3.789,4.867)	4.481	(3.987,4.990)
${\widehat{\beta }}_{15}$	5.617	(5.211,6.096)	5.777	(5.179,6.373)	5.727	(5.628,5.820)
${\widehat{\beta }}_{16}$	0.000	(−0.789,0.741)	0.000	(−0.389,0.405)	0.000	(−0.300,0.306)
${\widehat{\beta }}_{17}$	0.000	(−0.357,0.442)	0.000	(−0.433,0.393)	0.000	(−0.097,0.095)
${\widehat{\beta }}_{18}$	0.000	(−0.770,0.797)	0.000	(−0.425,0.412)	0.000	(−0.527,0.497)
${\widehat{\beta }}_{19}$	0.000	(−0.512,0.566)	0.000	(−0.615,0.584)	0.000	(−0.099,0.093)
${\widehat{\beta }}_{20}$	1.633	(1.078,2.248)	1.231	(0.813,1.618)	1.230	(0.700,1.736)
${\widehat{\beta }}_{21}$	0.000	(−0.784,0.797)	0.000	(−0.614,0.563)	0.000	(−0.103,0.100)
${\widehat{\beta }}_{22}$	8.851	(8.037,9.757)	8.603	(7.998,9.211)	8.478	(8.173,8.781)
${\widehat{\beta }}_{23}$	0.000	(−0.390,0.413)	2.993	(2.369,3.586)	2.774	(2.248,3.263)
${\widehat{\beta }}_{24}$	0.000	(−0.363,0.467)	2.901	(2.345,3.468)	2.635	(2.112,3.134)
${\widehat{\beta }}_{25}$	0.000	(−0.364,0.354)	3.118	(2.753,3.498)	2.971	(2.867,3.090)
${\widehat{\beta }}_{26}$	5.647	(5.060,6.238)	5.452	(5.055,5.836)	5.461	(4.923,5.955)

shows that the three penalty functions consistently identify ${\beta }_2,{\beta }_4,{\beta }_5,{\beta }_6,{\beta }_7,{\beta }_{10},{\beta }_{16},{\beta }_{17},{\beta }_{18},{\beta }_{19},{\beta }_{21}$ as zero, in that their corresponding 95% CIs contain zero, and consistently detect ${\beta }_{11},{\beta }_{12},{\beta }_{13},{\beta }_{14},{\beta }_{15},{\beta }_{20},{\beta }_{22},{\beta }_{26}$ as non-zero, in that their corresponding 95% CIs do not contain zero (i.e., number of nodules, shape, margins, echogenecity, rear attenuation, bloodstream, and calcification are positively associated with malignant thyroid nodules), while ${\beta }_3$ and ${\beta }_9$ are identified as no effect on malignant thyroid nodules by Lasso and ALasso penalty functions but there ia a negative effect on malignant thyroid nodules only by SCAD. Also, ${\beta }_{23}$, ${\beta }_{24}$, and ${\beta }_{25}$ are detected as on effect on malignant thyroid nodules only by Lasso. The PETL estimates of non-zero parameters obtained with three penalty functions are almost identical. The above results indicate that the Lasso penalty function tends to shrink more parameters to zero, which helps us better identify important variables. However, it is also possible for this to be over-fitted, and this may cause more important variables to be ignored. Combining the above results leads to the following conclusions. First, it follows from a negative estimate of ${\beta }_1$ that there is a negative correlation between nodular echogenicity and malignant thyroid nodules, i.e., hypoechogenicity often indicates that the thyroid nodule is malignant. In addition, it follows from ${\widehat{\beta }}_8,{\widehat{\beta }}_{13},{\widehat{\beta }}_{14},{\widehat{\beta }}_{26}$ that aspect ratio, margins, and microcalcifications are closely related to malignant thyroid nodules, which is consistent with the conclusion of [38]. Second, by ${\widehat{\beta }}_9$ obtained with the SCAD penalty function, there is a positive correlation between age and malignant thyroid nodules. The coefficients associated with nodule size $x_5$, sex $x_{10}$, and internal structure (solidity $x_{16}$, cystic $x_{17}$, cystic solid mixture $x_{18}$) are shrunk to zero, implying that these covariates are not associated with malignant thyroid nodules, which is consistent with the conclusion regarding the role of predicting telomerase reverse transcriptase (TERT) promoter mutations in follicular thyroid cancer [39,40]. Third, there are correlations between certain data types, but their interaction effects have not been explored well, in that some of these data types represent certain interaction effects; for example, the aspect ratio represents the interaction effect between longitudinal diameter and horizontal diameter. By ${\widehat{\beta }}_6$, ${\widehat{\beta }}_7$, and ${\widehat{\beta }}_8$, although longitudinal diameter $x_6$ and horizontal diameter $x_7$ are not correlated with malignant thyroid nodules, their interaction effect (aspect ratio $x_8$) has a negative effect on malignant thyroid nodules.

6. Discussion

This paper studies growing dimensional unconditional moment models in the presence of missing responses at random, constructs the modified unconditional moment models via the inverse probability weighted approach to deal with missing data, utilizes logistic regression models to describe propensity score functions, employs calibration conditions to establish estimating equations for estimating parameters in logistic regression models, and proposes a penalized log-ET likelihood ratio function to estimate unknown parameters in growing dimensional unconditional moment models and logistic regression models by combining the modified unconditional moment models and calibration conditions. To test contrasting hypotheses, we construct a constrainedly penalized ET likelihood ratio statistic, which is theoretically shown to asymptotically follow the centred chi-squared distribution that extends Wilks’ theory to growing unconditional moment models with missing response at random. Empirical results demonstrate that the proposed method has better estimation properties and more accurate variable selection than other methods, such as GMM and penalized empirical likelihood methods. Also, empirical results evidence that the proposed penalized ET likelihood method is robust to misspecified moment models. The thyroid data taken from the First People’s Hospital of Yunnan Province are used to illustrate the proposed method, and parameter estimates show the consistency with [38,39,40].

The proposed method has the following merits. First, it can simultaneously estimate unknown parameters associated with growing dimensional unconditional moment models and propensity score functions and select important variables. Second, novel unified equations are constructed to jointly rather than separately estimate unknown parameters by incorporating growing dimensional unconditional moment models and calibration conditions, which largely enhances the efficiency of the algorithm. Third, under some regularity conditions, we show the oracle property and asymptotic property of parameter estimators.

This paper focuses on missing at random of response variable and logistic regression models of propensity score functions. It is possible to extend the proposed method to nonignorable missing responses together with unknown forms of propensity score functions based on the neural network method to approximate the unknown propensity score functions.