A Bootstrap Method for a Multiple-Imputation Variance Estimator in Survey Sampling

Abstract

Rubin’s variance estimator of the multiple imputation estimator for a domain mean is not asymptotically unbiased. Kim et al. derived the closed-form bias for Rubin’s variance estimator. In addition, they proposed an asymptotically unbiased variance estimator for the multiple imputation estimator when the imputed values can be written as a linear function of the observed values. However, this needs the assumption that the covariance of the imputed values in the same imputed dataset is twice that in the different imputed datasets. In this study, we proposed a bootstrap variance estimator that does not need this assumption. Both theoretical argument and simulation studies show that it was unbiased and asymptotically valid. The new method was applied to the Hox pupil popularity data for illustration.

1. Introduction

Surveys are popular tools for empirical research in many fields, such as the social sciences, marketing, and public health. One of the goals of survey data analysis is the estimation of subpopulations, which are called domains. The estimator of the domain mean $\theta$ usually involves weights in survey data analysis [1,2,3,4,5], which is written as

$$\widehat{\theta }={{\displaystyle \sum }}_{\left\{i\in A\right\}}{\alpha }_i{y}_i$$

(1)

where A represents the set of indices of the sample elements, $y_i$ are the observed sample values in A, and ${\alpha }_i$ is the known weight function that is free of $y_i$. The ${\alpha }_i$ can be used to adjust the non-response items or for calibration purposes.

Imputation is commonly used in survey data analysis to handle the non-response items in survey samples. It replaces the non-response items in the survey sample with substituted values. Single imputation leads to underestimating the variances of the estimators [6]. To overcome this, a multiple imputation method [6,7] was proposed to account for the variances in the analysis when substituted values are used instead of the true observations. The basic idea of multiple imputations is to impute each non-response value M times to create M imputed datasets. Then, each imputed dataset is analyzed using standard data analysis techniques. Finally, Rubin’s rule is used to obtain Rubin’s estimator and Rubin’s variance estimator by combining the results from each imputed dataset.

Research on multiple imputations is motivated by a Bayesian framework. Rubin [6] claimed that multiple imputations can provide a valid frequentist inference in many applications, assuming “proper” imputation [6] or congeniality condition [8] for both the imputation model and the analysis model. However, Fay [9,10] and Binder and Sun [11] found that it is difficult to satisfy the conditions for “proper” imputation for general complex sampling schemes. In a parametric model setting, Wang and Robin [12] and Nielsen [13] showed that Rubin’s variance estimator is unbiased when the complete-sample estimator is the maximum likelihood estimator. Then, Robin and Wang [14] showed the bias of Rubin’s variance estimator for nonparametric inference procedures and proposed a new variance estimator. Yang and Kim [15] showed the bias of Rubin’s variance estimator for the method of moments estimator. They also proposed a new variance estimator when the complete-sample estimator is the method of moments estimator or maximum likelihood estimator. However, these proposed variance estimators cannot be directly used when the complete-sample estimator is given as (1) for the domain mean. For estimator (1), Kim et al. [16] derived the closed-form bias of Rubin’s variance estimator and proposed a new variance estimator when the imputed values can be expressed as a linear function of the response items. However, it requires an assumption that the covariance of the imputed values in the same imputed datasets is twice that of the different imputed datasets. Kim et al. [16] did not discuss how to check this assumption. Our simulation study results presented in Section 3.1 for the domain mean estimation showed that Kim’s variance estimator is biased, which indicated that this assumption may not be satisfied.

In this study, we proposed a bootstrap variance estimator without this assumption. Traditional bootstrapping [17] is usually used to estimate the properties of an estimator by sampling from an approximating distribution. For independent and identically distributed sample data with n observations, it obtains bootstrap samples, each with n observations, by randomly drawing observations with replacement from the original sample. It was used for variance estimation in the presence of single imputation for missing data analysis [18,19,20,21,22,23,24,25]. Recently, researchers [26,27,28,29,30,31] applied the traditional bootstrap for multiple imputations. They applied multiple imputations for each bootstrap sample to obtain the parameter estimator. Then, the variance of the estimator is estimated by the sample variance of the estimators from the bootstrap samples. Therefore, they need to refit the model M $\times$ B times, where M is the number of imputed samples and B is the number of bootstrap samples. Obviously, the traditional bootstrap methods are computationally intensive. In this study, we proposed a computationally efficient approach for a bootstrap procedure. We used the bootstrap samples with single imputation and multiple imputation samples to estimate the variance. Therefore, it only needed to refit the model M + B times to estimate the variance of the multiple imputation estimator with the complete-sample estimator in the form of (1).

2. Methods

2.1. Rubin’s Method

We consider data generated from a superpopulation model, i.e., a random sample from an infinite population. The sampling mechanism and the response mechanism are assumed to be ignorable under the superpopulation model in the sense of Rubin [6]. Let ${\mathrm{y}}_1,\cdots ,{ \mathrm{y}}_{\mathrm{n}}$ be the complete sample for the random variable Y. Without loss of generality, assume that the first r observations are observed and the remaining n − r observations are missing, i.e., $Y_{\mathrm{obs}}=\left\{{\mathrm{y}}_1,\cdots ,{ \mathrm{y}}_{\mathrm{r}}\right\}$ and $Y_{\mathrm{mis}}=\left\{{\mathrm{y}}_{\mathrm{r}+1},\cdots ,{ \mathrm{y}}_{\mathrm{n}}\right\}$. Denote ${\mathrm{R}}_{\mathrm{i}}$ as the indicator of whether the ith observation is observed. If ${\mathrm{R}}_{\mathrm{i}}=1$, ${\mathrm{y}}_{\mathrm{i}}$ is observed and if ${\mathrm{R}}_{\mathrm{i}}=0$, ${\mathrm{y}}_{\mathrm{i}}$ is missing. The multiple imputation method uses the following three steps to estimate the parameter domain mean $\mathsf{\theta }$ for the complete-sample estimator in the form of (1).

Step 1. Create $m=1,\cdots , M,$ imputed datasets, where the observations $y_i^{\ast m}$, $i=1,\cdots , n,$ are imputed only for the missing values; otherwise, they remain as observed.

Step 2. Apply the complete-sample estimation procedure to each imputed dataset. Let ${\widehat{\theta }}_n^m$ be the imputed-sample estimator of $\theta$ for the mth imputed dataset and ${\widehat{V}}_n^m$ be the imputed-sample variance estimator of ${\widehat{\theta }}_n^m$.

Step 3. Combine the results from the imputed datasets using Rubin’s rule [6]. Rubin’s estimator of $\theta$ is ${\widehat{\theta }}_{MI}=M^{-1}{\displaystyle \sum }_{m=1}^M{\widehat{\theta }}_n^m$, and Rubin’s variance estimator is

$${\widehat{V}}_{MI}({\widehat{\theta }}_{MI})=W_M+(1+M^{-1})B_M$$

(2)

where $W_M=M^{-1}{\displaystyle \sum }_{m=1}^M{\widehat{V}}_n^m$ is the within-imputation variance and $B_M={(M-1)}^{-1}{\displaystyle \sum }_{m=1}^M{({\widehat{\theta }}_n^m-{\widehat{\theta }}_{MI})}^2$ is the between-imputation variance.

Rubin’s variance estimator is based on the decomposition

$$var({\widehat{\theta }}_{MI})=var({\widehat{\theta }}_n)+var({\widehat{\theta }}_{MI}-{\widehat{\theta }}_n)+2cov({\widehat{\theta }}_{MI}-{\widehat{\theta }}_n, {\widehat{\theta }}_n)$$

(3)

where ${\widehat{\theta }}_n$ is the complete-sample estimator of $\theta$. In Rubin’s variance estimator (2), $W_M$ estimates the first term of (3) and $(1+M^{-1})B_M$ estimates the second term of (3). Kim et al. [16] showed that for estimators in the form of (1), $(1+M^{-1})B_M$ is an asymptotically unbiased estimator of the second term of (3) based on the covariance assumption, that is, the covariance between $y_i^{\ast m_1}$ and $y_j^{\ast m_1}$ is twice that of the covariance between $y_i^{\ast m_1}$ and $y_j^{\ast m_2}$ for $i\ne j$ and $m_1\ne m_2$. Therefore, if $W_M$ is an unbiased estimator of the first term of (3), the asymptotic bias of Rubin’s variance estimator is

$$bias({\widehat{V}}_{MI})=-2cov({\widehat{\theta }}_{MI}-{\widehat{\theta }}_n, {\widehat{\theta }}_n).$$

(4)

Kim et al. [16] derived the closed-form bias of (4) for the estimators in the form of (1) when the imputed values can be expressed as a linear function of the observed items. We can see that Kim’s method [16] requires the covariance assumption. In this study, we proposed a bootstrap variance estimator for ${\widehat{\theta }}_{MI}$ that does not need this assumption, and hence, this new method has a much wider range of applications.

2.2. The Bootstrap Estimation Method

2.2.1. Assumptions

We employed the regularity assumptions for the data and the imputation mechanism in Kim et al. [16] for the newly proposed method.

(C.1) Both the complete-sample point estimator and variance estimator are asymptotically unbiased, i.e.,

$E({\widehat{\theta }}_n)=\theta +O(n^{-1})$ and $E({\widehat{V}}_n)=var({\widehat{\theta }}_n)+O(n^{-1})$, where ${\widehat{V}}_n$ is the complete-sample variance estimator of ${\widehat{\theta }}_n$.

(C.2) The conditional expectations of the imputed values are the same as those of the real values, i.e.,

$E(y_i^{\ast m}|A, A_R)=E(y_i|A, A_R), m=1,\cdots , M$, where $A_R$ is the set of indices of the observed items.

(C.3) The conditional variances of the imputed values are asymptotically the same as those of the real values, i.e.,

$$\underset{i,j}{\max }|cov(y_i^{\ast m},y_j^{\ast m}|A, A_R)-cov(y_i, y_j|A, A_R)|=o(1), m=1,\cdots , M.$$

(C.4) Conditional on $A, A_R$ and ${\mathit{Y}}_{obs}$, the imputed values $y_j^{\ast m}$ of $y_j, j\in A_k$ are conditionally independent and identically distributed, where $A_k$ is the set of indices of the missing items.

Assumptions (C.1) and (C.2) ensure that the imputed-sample estimators of the form (1) are approximately unbiased. Assumption (C.3) ensures that the covariances of the imputed values are asymptotically the same as those of the original values. Therefore, we can treat the imputed values similarly to the original values in the inference procedure. Assumption (C.4) ensures both the estimators ${\widehat{\theta }}_n^m , m=1, \cdots , M$, and the naive variance estimators ${\widehat{V}}_n^m , m=1, \cdots , M$, are identically distributed. Therefore, the imputed values and the imputed samples can be treated equally in the inference procedure.

2.2.2. Variance Decomposition

Instead of the decomposition (3), we decomposed the variance as

$$var({\widehat{\theta }}_{MI})=V_1+V_2+V_3+V_4,$$

(5)

where $V_1=var({\displaystyle \sum }_{i\in A_R}{\alpha }_i{y}_i)$, $V_2=M^{-1}E\{var({\displaystyle \sum }_{j\in A_k}{\alpha }_j{y}_j^{\ast m}|{\mathit{Y}}_{obs})$, $V_3=var\{E({\displaystyle \sum }_{j\in A_k}{\alpha }_j{y}_j^{\ast m}|{\mathit{Y}}_{obs})$, and $V_4=2cov({\displaystyle \sum }_{j\in A_k}{\alpha }_j{y}_j^{\ast m}, {\displaystyle \sum }_{j\in A_R}{\alpha }_i{y}_i)$. $V_1$ is the sum of the variances for the observed observations. The sum of $V_2$ and $V_3$ is the sum of the variances for the imputed observations based on the law of total variance. $V_4$ is the sum of the covariances of the observed observations and the imputed observations.

2.2.3. Bootstrap Variance Estimator and Its Properties

To estimate (5), we generated bootstrap data as follows. For $b=1,\cdots , B$:

Step 1: Take a random sample of size n with a replacement from the original sample, which consists of independent and identical observations Denote the bootstrap sample as $(y_1^b, R_1^b),\cdots ,(y_n^b,R_n^b)$. Denote the observed items as ${\mathit{Y}}_{obs}^b=\{y_1^b,\cdots , y_{r^{\prime }}^b\}$ and the missing items as ${\mathit{Y}}_{mis}^b=\left\{y_{r^{\prime }+1}^b,\cdots , y_n^b\right\}$ in the bootstrap method. Let $A^b$, $A_R^b$, and $A_k^b$ represent the set of indices of the sample elements, the set of indices of the observed items, and the set of indices of the missing items in the bth bootstrap sample, respectively.

Step 2: With the resultant bootstrap sample, calculate the imputed values $y_j^{\ast b}$ for missing items $y_j^b$ using a single imputation method, which is the same as the multiple imputation method with M = 1. The imputed-sample estimator in the bth bootstrap sample is ${\widehat{\theta }}^b={\displaystyle \sum }_{i\in A^b}{\alpha }_i^b{y}_i^{\ast b}$.

We estimated $V_1$ and $V_2$ using multiple imputed samples and estimated $V_3$ and $V_4$ using bootstrap samples. $V_1$ can be estimated from the imputed-sample output based on condition (C.1), i.e., ${\widehat{V}}_1=r{(nM)}^{-1}{\displaystyle \sum }_{m=1}^M{\widehat{V}}_n^m. V_2$ is estimated using the empirical conditional variance, which is ${\widehat{V}}_2=M^{-1}{(M-1)}^{-1}{\displaystyle \sum }_{m=1}^M{({\displaystyle \sum }_{j\in A_k}{\alpha }_j{y}_j^{\ast m}-H_1)}^2,$ where $H_1=M^{-1}{\displaystyle \sum }_{m=1}^M{\displaystyle \sum }_{j\in A_k}{\alpha }_j{y}_j^{\ast m}$. $V_3$ is estimated using ${\widehat{V}}_3={(B-1)}^{-1}{\displaystyle \sum }_{b=1}^B{({\displaystyle \sum }_{j\in A_k^b}{\alpha }_j^b{\eta }_j^b-H_2)}^2$, where ${\eta }_j^b\equiv E(y_j^{\ast b}|{\mathit{Y}}_{obs}^b)$ and $H_2=B^{-1}{\displaystyle \sum }_{b=1}^B{\displaystyle \sum }_{j\in A_k^b}{\alpha }_j^b{\eta }_j^b.$ $V_4$ is estimated using ${\widehat{V}}_4=2{\displaystyle \sum }_{j\in A_k}{\alpha }_j{\displaystyle \sum }_{i\in A_R}{\alpha }_i{(B-1)}^{-1}{\displaystyle \sum }_{b=1}^B({\overline{\eta }}_j^b-H_3)({\overline{Y}}_{r^{\prime }}^b-H_4)$, where ${\overline{\eta }}_j^b={(n-r^{\prime })}^{-1}{\displaystyle \sum }_{j\in A_k^b}{\eta }_j^b$, ${\overline{Y}}_{r^{\prime }}^b={(r^{\prime })}^{-1}{\displaystyle \sum }_{i\in A_R^b}{y}_i^b$, $H_3=B^{-1}{\displaystyle \sum }_{b=1}^B{\overline{\eta }}_j^b$, and $H_4=B^{-1}{\displaystyle \sum }_{b=1}^B{\overline{Y}}_{r^{\prime }}^b$. Therefore, the bootstrap variance estimator is $\widehat{V}={\widehat{V}}_1+{\widehat{V}}_2+{\widehat{V}}_3+{\widehat{V}}_4$.

The newly proposed bootstrap variance estimator is an asymptotically unbiased estimator for the variance of the multiple imputation estimator. Under the conditions (C.1)–(C.4), ${\widehat{V}}_1$ is an asymptotically unbiased estimator of $V_1$. ${\widehat{V}}_2$ is an asymptotically unbiased estimator because $E({\widehat{V}}_2)=E\{E({\widehat{V}}_2|{\mathit{Y}}_{obs})\}$. ${\widehat{V}}_3$ is an asymptotically unbiased estimator of $V_3$ because it is based on the Glivenko–Cantelli Theorem:

$E({\widehat{V}}_3)=E[E\left\{{({\displaystyle \sum }_{j\in A_k^b}{\alpha }_j^b{\eta }_j^b)}^2|{\mathit{Y}}_{obs}\right\}-{\left\{E({\displaystyle \sum }_{j\in A_k^b}{\alpha }_j^b{\eta }_j^b|{\mathit{Y}}_{obs})\right\}}^2]$,

where ${\eta }_j^b=E(y_j^{\ast b}|{\mathit{Y}}_{obs})$.

For ${\widehat{V}}_4$, we derived

$E\{cov( {\overline{Y}}_{r^{\prime }}^b,{\overline{\eta }}_j^b|A, A_R, {\mathit{Y}}_{obs})\}=cov(y_i, y_j^{\ast m}|A, A_R)+O(\frac{1}{r})$,

and

$E(\frac{1}{B-1}{\displaystyle \sum }_{b=1}^B({\overline{\eta }}_j^b-H_3)({\overline{Y}}_{r^{\prime }}^b-H_4))$

=$E\{\frac{1}{B-1}E({\displaystyle \sum }_{b=1}^B(({\overline{\eta }}_j^b-{\mu }_1)+({\mu }_1-H_3))(({\overline{Y}}_{r^{\prime }}^b-{\mu }_2)+({\mu }_2-H_4))|{\mathit{Y}}_{obs})\}$

=$E\{cov({\overline{\eta }}_j^b, {\overline{Y}}_{r^{\prime }}^b|{\mathit{Y}}_{obs})\}$,

where ${\mu }_1\equiv E({\overline{\eta }}_j^b|{\mathit{Y}}_{obs})$ and ${\mu }_2\equiv E({\overline{Y}}_{r^{\prime }}^b|{\mathit{Y}}_{obs})$. Thus, ${\widehat{V}}_4$ is an asymptotically unbiased estimator of $V_4$.

To account for the uncertainty in the variance estimator with a small-to-moderate imputation size and bootstrap replicates, a t-distribution is used to calculate $100(1-\alpha )\%$ confidence intervals for $\theta$, i.e., ${\widehat{\theta }}_{MI}\pm t_{v,1-\alpha /2}{\widehat{V}}^{1/2}$. The degrees of freedom $v$ is an approximate number along the lines of the Satterthwaite [32] method, which is derived as

$$v=\frac{{\widehat{V}}^2}{{(M-1)}^{-1}{\widehat{V}}_2^2+{(B-1)}^{-1}[{\widehat{V}}_3^2+{\widehat{V}}_5^2+{\widehat{V}}_6^2+{\widehat{V}}_7^2+2(1-{\pi }^2){\widehat{V}}_5({\widehat{V}}_3-{\widehat{V}}_7)-2{\widehat{V}}_3{\widehat{V}}_7-2{\widehat{V}}_5{\widehat{V}}_6{\pi }^2]},$$

in which $\pi$ is the response rate of the interested domain, ${\widehat{V}}_5={(B-1)}^{-1}{\displaystyle \sum }_{b=1}^B{(H_5+H_6)}^2$, ${\widehat{V}}_6={(B-1)}^{-1}{\displaystyle \sum }_{b=1}^B{H}_5^2$, and ${\widehat{V}}_7={(B-1)}^{-1}{\displaystyle \sum }_{b=1}^B{H}_6^2$, where $H_5={\displaystyle \sum }_{i\in A_R}{\alpha }_i({\overline{Y}}_{r^{\prime }}^b-H_4)$ and $H_6={\displaystyle \sum }_{j\in A_k}{\alpha }_j({\overline{\eta }}_j^b-H_3)$. The detailed derivation of $v$ is provided in the Appendix A.

3. Examples and Results

Simulations and a real data analysis were conducted to investigate the performance of the proposed bootstrap variance estimator ($\widehat{V}$). We compared it with Rubin’s variance estimator (${\widehat{V}}_{MI}$), Kim’s method (${\widehat{V}}_k$) [16], and the traditional bootstrapping method (${\widehat{V}}_B$). As in Yang and Kim [15], we report the relative bias (Rbias) of the variance estimators, mean width (mwidth), and coverage probabilities of the 95% confidence intervals (95cov) for $\theta$. The relative bias was calculated using $(E(\tilde{V})-var({\widehat{\theta }}_{MI}))/var({\widehat{\theta }}_{MI})\times 100\%$, where $\tilde{V}$ is a variance estimator. The 95% confidence intervals were calculated based on a t-distribution with $v$ degrees of freedom. We considered different values of B, M, and n to investigate their effects. These results were based on 5000 Monte Carlo runs for each setting. The R code used for these simulations is available from the first author upon request.

3.1. Simulation 1: Domain Mean Estimation

We simulated a sample $y_1,\cdots ,y_n$ from $N(\mu ,{\sigma }^2)$, where $\mu =2, {\sigma }^2=4$, and n = 500 or 1000. Then non-respondents were generated based on the response rate $\pi$ and using the ampute function in the mice package of R. We considered two response rates, which were 0.8 and 0.5. To indicate whether an observation belonged to a domain $Z$, we generated $z_i$ from Bernoulli distribution with probability d, which was 0.2 or 0.6 in the simulation study. If $z_i$= 1, this meant that $y_i$ was in domain Z and was 0 otherwise. The $z_i, i=1,\cdots , n$, were always observed.

Now we would like to estimate the mean $\theta$ of the domain Z. Without loss of generality, denote the first r observations as respondents $(y_1,\cdots , y_r)$ and the remaining (n − r) observations as the non-respondents $(y_{r+1},\cdots , y_n)$. Then, the complete-sample estimator of $\theta$ is

$${\widehat{\theta }}_n={\displaystyle \sum }_{i=1}^n\frac{z_i}{{{\displaystyle \sum }}_{i=1}^n{z}_i}{y}_i.$$

Let $n_d={\displaystyle \sum }_{i=1}^n{z}_i$ be the size of the domain Z and $r_d={\displaystyle \sum }_{i=1}^r{z}_i$ be the number of response items in the domain Z. It is obvious that ${\alpha }_i=z_i/n_d.$

To handle the missing observations, we used the multiple imputation method with M = 10 or 30. The mth imputed-sample estimator is

$${\widehat{\theta }}_n^m={\displaystyle \sum }_{i=1}^n\frac{z_i}{{{\displaystyle \sum }}_{i=1}^n{z}_i}{y}_i^{\ast m}.$$

If $y_i$ is missing, the conditional expectation of the imputed value is ${\eta }_i\equiv E(y_i^{\ast m}|y_1,\cdots , y_r)={\overline{y}}_r$ and ${\overline{y}}_r={\displaystyle \sum }_{i=1}^r{y}_i/r$.

Now we used the newly proposed bootstrap method with B = 200 or 500 in Section 3 to estimate the variance of the multiple imputation estimator. Denote the bootstrap sample as $\left\{y_1^b,\cdots , y_n^b\right\}$ in which the response items are ${\mathit{Y}}_{obs}^b=\left\{y_1^b,\cdots , y_{r^{\prime }}^b\right\}$, the non-response items are ${\mathit{Y}}_{mis}^b=\left\{y_{r^{\prime }+1}^b,\cdots , y_n^b\right\}$, and the corresponding indicators for domain Z are $z_i^b, i=1,\cdots ,n.$ Let $n_d^b$ be the size of the domain Z in the bth bootstrap sample. Then, we calculate

${\widehat{V}}_1=\frac{r_d}{n_d^2(n-1)M}{\displaystyle \sum }_{m=1}^M{\displaystyle \sum }_{i=1}^n{(y_i^{\ast m}-{\overline{y}}^m)}^2$, where ${\overline{y}}^m=\frac{1}{n}{\displaystyle \sum }_{i=1}^n{y}_i^{\ast m}$;

${\widehat{V}}_2=\frac{1}{M(M-1)}{\displaystyle \sum }_{m=1}^M{({\displaystyle \sum }_{j=r+1}^n\frac{z_j}{n_d}{y}_j^{\ast m}-H_1)}^2$, where $H_1=\frac{1}{M}{\displaystyle \sum }_{m=1}^M{\displaystyle \sum }_{j=r+1}^n\frac{z_j}{n_d}{y}_j^{\ast m}$;

${\widehat{V}}_3=\frac{1}{B-1}{\displaystyle \sum }_{b=1}^B{({\displaystyle \sum }_{j=r^{\prime }+1}^n\frac{z_j^b}{n_d^b}{\overline{Y}}_{r^{\prime }}^b-H_2)}^2$, where $H_2=\frac{1}{B}{\displaystyle \sum }_{b=1}^B{\displaystyle \sum }_{j=r^{\prime }+1}^n\frac{z_j^b}{n_d^b}{\overline{Y}}_{r^{}}^b$ and

${\overline{Y}}_{r^{\prime }}^b=\frac{{{\displaystyle \sum }}_{i=1}^{r^{\prime }}{y}_i^b}{r^{\prime }}$;

${\widehat{V}}_4=2{\displaystyle \sum }_{j=r+1}^n\frac{z_j}{n_d}{\displaystyle \sum }_{i=1}^r\frac{z_i}{n_d}{(B-1)}^{-1}{\displaystyle \sum }_{b=1}^B({\overline{Y}}_{r^{\prime }}^b-H_3)({\overline{Y}}_z^b-H_4)$, where ${\overline{Y}}_z^b=\frac{{{\displaystyle \sum }}_{i=1}^{r^{\prime }}{z}_i^b{y}_i^b}{{{\displaystyle \sum }}_{i=1}^{r^{\prime }}{z}_i^b}$,

$H_3=B^{-1}{\displaystyle \sum }_{b=1}^B{\overline{Y}}_{r^{\prime }}^b$ and $H_4=B^{-1}{\displaystyle \sum }_{b=1}^B{\overline{Y}}_z^b$.

To compare the newly proposed method with the method proposed by Kim et al. [16], we used the following formula derived by them [16]:

${\widehat{V}}_k=W_M+(1+M^{-1})B_M+2n_d^{-2}{m}_d(1-r^{-1}{r}_d){\widehat{\sigma }}^2$, where $m_d=n_d-r_d$ and ${\widehat{\sigma }}^2={(n-1)}^{-1}{M}^{-1}{\displaystyle \sum }_{m=1}^M{\displaystyle \sum }_{i=1}^n{(y_i^{\ast m}-{\overline{y}}^m)}^2$.

The simulation results are summarized in Table 1. They show that the newly proposed method $\widehat{V}$ had a smaller relative bias than both Rubin’s method ${\widehat{V}}_{MI}$ and Kim’s method ${\widehat{V}}_k$, especially for cases with a small response rate. We notice that the relative bias was larger for the proposed method when the sample size became larger. This may have been due to the smaller value of $var({\widehat{\theta }}_{MI})$ for the larger sample size. Other methods have this pattern as well. The width of the confidence interval of Rubin’s method was usually larger, while that of Kim’s method was usually smaller than that of the newly proposed method. The coverage rate of the newly proposed method was close to the nominal level. However, Rubin’s method had a larger coverage rate and Kim’s method had a smaller coverage rate than the nominal level. These results indicated that the covariance assumption required in Kim’s method may not be satisfied, and hence, $(1+M^{-1})B_M$ may not unbiasedly estimate $var({\widehat{\theta }}_{MI}-{\widehat{\theta }}_n)$ for the domain mean estimation. The newly proposed method $\widehat{V}$ had a similar performance to the traditional bootstrap method ${\widehat{V}}_B$. However, it was much faster than the traditional bootstrap method ${\widehat{V}}_B$, especially for cases with large n, B, and M. For example, for a case with n = 1000, B = 500, M = 30, and 5000 Monte Carlo runs, the new method needed about half an hour, while the traditional bootstrap method needed about four hours. In addition, the values of B considered in these examples had no significant effects on these methods. Larger values of n and M resulted in smaller widths of the confidence intervals.

3.2. Simulation 2: Linear Regression

The variance estimation method proposed in this study can not only be applied to the domain mean estimation but also to any other estimator that has the form of (1). In this example, we showed its performance for the mean estimation of the response variable in the linear regression model. We considered the model used in Yang and Kim [15] as follows:

$$y_i=\beta x_i+e_i, i=1, \cdots , n,$$

Table 1. Simulation results for the domain mean estimation.

					Rbias (%)				Mwidth $(\times {10}^2)$				95cov $(\times {10}^2)$
n	B	M	$\mathit{\pi }$	d	$\widehat{\mathit{V}}$	${\widehat{\mathit{V}}}_{\mathit{M}\mathit{I}}$	${\widehat{\mathit{V}}}_{\mathit{K}}$	${\widehat{\mathit{V}}}_{\mathit{B}}$	$\widehat{\mathit{V}}$	${\widehat{\mathit{V}}}_{\mathit{M}\mathit{I}}$	${\widehat{\mathit{V}}}_{\mathit{K}}$	${\widehat{\mathit{V}}}_{\mathit{B}}$	$\widehat{\mathit{V}}$	${\widehat{\mathit{V}}}_{\mathit{M}\mathit{I}}$	${\widehat{\mathit{V}}}_{\mathit{K}}$	${\widehat{\mathit{V}}}_{\mathit{B}}$
500	500	10	0.8	0.2	0.6	146.2	−12.5	0.5	65	102	58	65	95	99	89	95
				0.6	1.6	21.6	−4.9	1.5	48	53	46	48	95	97	95	95
			0.5	0.2	−1.3	237.7	−41.1	0.7	61	113	36	61	95	99	58	96
				0.6	−0.7	18.1	−40.5	−0.6	55	61	42	56	95	96	85	95
		30	0.8	0.2	1.0	156.8	−12.6	0.9	63	100	57	62	95	99	92	95
				0.6	1.3	21.6	−5.3	1.3	47	52	46	47	95	97	95	95
			0.5	0.2	−2.2	261.4	−54.3	0.6	58	109	33	57	95	100	63	95
				0.6	−1.4	17.8	−42.2	−1.4	54	59	41	54	95	97	86	95
	200	10	0.8	0.2	0.5	146.2	−12.5	0.2	65	102	58	65	95	99	89	95
				0.6	1.6	21.6	−4.9	1.5	48	53	46	48	95	97	95	95
			0.5	0.2	−1.4	237.7	−41.1	0.7	62	113	36	61	95	99	58	96
				0.6	−0.8	18.1	−40.5	−0.7	55	61	42	56	95	96	85	95
		30	0.8	0.2	1.1	156.8	−12.6	1.0	63	100	57	62	95	99	92	95
				0.6	1.3	21.6	−5.3	1.3	47	52	46	47	95	97	95	95
			0.5	0.2	−2.2	261.4	−54.3	0.6	61	108	33	57	96	100	63	95
				0.6	−1.3	−42.2	17.8	−1.3	54	59	41	54	95	97	86	95
1000	500	10	0.8	0.2	−1.1	141.4	13.1	−1.1	46	72	42	46	95	99	89	95
				0.6	−4.0	53.4	−9.7	−4.0	34	37	33	34	95	97	94	95
			0.5	0.2	−3.7	255.4	−44.3	−3.2	43	80	25	44	95	99	57	95
				0.6	−0.3	19.5	−39.2	−0.3	39	44	30	40	95	97	85	95
		30	0.8	0.2	−1.5	148.6	−14.6	−1.6	44	71	41	44	95	99	92	95
				0.6	−3.7	15.7	−9.8	−3.6	33	37	32	34	95	96	94	95
			0.5	0.2	−3.9	251.2	−55.3	−3.3	41	77	23	40	94	99	63	94
				0.6	−0.1	20.0	−40.8	−0.1	38	42	29	39	95	97	85	95
	200	10	0.8	0.2	−1.0	141.4	−13.1	−1.0	46	72	42	46	95	99	89	95
				0.6	−3.9	15.3	−9.7	−4.0	34	37	33	34	94	97	94	95
			0.5	0.2	−3.8	225.4	−44.3	−3.3	44	80	25	44	96	99	57	95
				0.6	−0.3	19.5	−39.2	−0.6	39	44	30	40	95	97	85	95
		30	0.8	0.2	−1.6	148.6	14.6	−1.5	45	71	41	44	95	99	92	95
				0.6	−3.7	15.7	−9.8	−3.6	33	37	32	34	95	96	94	94
			0.5	0.2	−4.0	251.2	−55.3	−3.3	42	77	23	40	95	99	63	94
				0.6	−0.1	20.0	−40.8	−0.1	38	42	29	39	95	97	85	95

Table 1. Simulation results for the domain mean estimation.

					Rbias (%)				Mwidth $(\times {10}^2)$				95cov $(\times {10}^2)$
n	B	M	$\mathit{\pi }$	d	$\widehat{\mathit{V}}$	${\widehat{\mathit{V}}}_{\mathit{M}\mathit{I}}$	${\widehat{\mathit{V}}}_{\mathit{K}}$	${\widehat{\mathit{V}}}_{\mathit{B}}$	$\widehat{\mathit{V}}$	${\widehat{\mathit{V}}}_{\mathit{M}\mathit{I}}$	${\widehat{\mathit{V}}}_{\mathit{K}}$	${\widehat{\mathit{V}}}_{\mathit{B}}$	$\widehat{\mathit{V}}$	${\widehat{\mathit{V}}}_{\mathit{M}\mathit{I}}$	${\widehat{\mathit{V}}}_{\mathit{K}}$	${\widehat{\mathit{V}}}_{\mathit{B}}$
500	500	10	0.8	0.2	0.6	146.2	−12.5	0.5	65	102	58	65	95	99	89	95
				0.6	1.6	21.6	−4.9	1.5	48	53	46	48	95	97	95	95
			0.5	0.2	−1.3	237.7	−41.1	0.7	61	113	36	61	95	99	58	96
				0.6	−0.7	18.1	−40.5	−0.6	55	61	42	56	95	96	85	95
		30	0.8	0.2	1.0	156.8	−12.6	0.9	63	100	57	62	95	99	92	95
				0.6	1.3	21.6	−5.3	1.3	47	52	46	47	95	97	95	95
			0.5	0.2	−2.2	261.4	−54.3	0.6	58	109	33	57	95	100	63	95
				0.6	−1.4	17.8	−42.2	−1.4	54	59	41	54	95	97	86	95
	200	10	0.8	0.2	0.5	146.2	−12.5	0.2	65	102	58	65	95	99	89	95
				0.6	1.6	21.6	−4.9	1.5	48	53	46	48	95	97	95	95
			0.5	0.2	−1.4	237.7	−41.1	0.7	62	113	36	61	95	99	58	96
				0.6	−0.8	18.1	−40.5	−0.7	55	61	42	56	95	96	85	95
		30	0.8	0.2	1.1	156.8	−12.6	1.0	63	100	57	62	95	99	92	95
				0.6	1.3	21.6	−5.3	1.3	47	52	46	47	95	97	95	95
			0.5	0.2	−2.2	261.4	−54.3	0.6	61	108	33	57	96	100	63	95
				0.6	−1.3	−42.2	17.8	−1.3	54	59	41	54	95	97	86	95
1000	500	10	0.8	0.2	−1.1	141.4	13.1	−1.1	46	72	42	46	95	99	89	95
				0.6	−4.0	53.4	−9.7	−4.0	34	37	33	34	95	97	94	95
			0.5	0.2	−3.7	255.4	−44.3	−3.2	43	80	25	44	95	99	57	95
				0.6	−0.3	19.5	−39.2	−0.3	39	44	30	40	95	97	85	95
		30	0.8	0.2	−1.5	148.6	−14.6	−1.6	44	71	41	44	95	99	92	95
				0.6	−3.7	15.7	−9.8	−3.6	33	37	32	34	95	96	94	95
			0.5	0.2	−3.9	251.2	−55.3	−3.3	41	77	23	40	94	99	63	94
				0.6	−0.1	20.0	−40.8	−0.1	38	42	29	39	95	97	85	95
	200	10	0.8	0.2	−1.0	141.4	−13.1	−1.0	46	72	42	46	95	99	89	95
				0.6	−3.9	15.3	−9.7	−4.0	34	37	33	34	94	97	94	95
			0.5	0.2	−3.8	225.4	−44.3	−3.3	44	80	25	44	96	99	57	95
				0.6	−0.3	19.5	−39.2	−0.6	39	44	30	40	95	97	85	95
		30	0.8	0.2	−1.6	148.6	14.6	−1.5	45	71	41	44	95	99	92	95
				0.6	−3.7	15.7	−9.8	−3.6	33	37	32	34	95	96	94	94
			0.5	0.2	−4.0	251.2	−55.3	−3.3	42	77	23	40	95	99	63	94
				0.6	−0.1	20.0	−40.8	−0.1	38	42	29	39	95	97	85	95

Where n = 500 or 1000, $\beta =0.1$, $x_i\sim \exp (1)$, and $e_i\sim N(0, {\sigma }_e^2)$ with ${\sigma }_e^2=0.25$. In this data setup, we assumed that $x_i$ were observed fully. The missingness in $y_i$ was controlled by ${\delta }_i$, which followed ${\delta }_i\sim Ber(p_i)$ and $p_i=1/\left\{exp(-{\varphi }_0-{\varphi }_1{x}_i)\right\}$. Two scenarios were considered, namely, $({\varphi }_0,{\varphi }_1)$ = (−1.5, 2) and $({\varphi }_0,{\varphi }_1)$ = (3, −3), which yielded the approximate average response rate of 0.5. We were interested in estimating $\theta =E(y_i)$. The normal imputation procedure [33] was used to generate the imputed values for the missing observations. Let ${\mathit{Y}}_{obs}=\left\{y_1,\cdots , y_r\right\}$ be the observed observations and ${\mathit{Y}}_{mis}=\left\{y_{r+1},\cdots , y_n\right\}$ be the missing observations. In the simulations, we set B = 200 or 500 and M = 10 or 30. Then, the complete-sample estimator was ${\widehat{\theta }}_n={\displaystyle \sum }_{i=1}^n{y}_i/n$.

Therefore, ${\alpha }_i=1/n$. It is a special case of the estimator with form (1), that is, ${\alpha }_i, i=1,\cdots ,n,$ were all same. The mth imputed-sample estimator is ${\widehat{\theta }}_n^m={\displaystyle \sum }_{i=1}^n{y}_i^{\ast m}/n$.

If $y_i$ was missing, the conditional expectation of the imputed value was ${\eta }_i\equiv E(y_i^{\ast m}|y_1,\cdots ,y_r)=x_i{({\mathit{X}}_r^T{\mathit{X}}_r)}^{-1}{\mathit{X}}_r^T{\mathit{Y}}_{obs}$ for $m=1, \cdots ,M$, where ${\mathit{X}}_r$ was a $r\times 2$ matrix with rows ${\mathit{X}}_i=\left[1 x_i\right], i=1,\cdots ,r$. To estimate the variance of the multiple imputation estimator, we used the bootstrap procedure described in Section 3. Denote the bootstrap sample as $\left\{y_1^b,\cdots ,y_n^b\right\}$, the response items as ${\mathit{Y}}_{obs}^b=\left\{y_1^b,\cdots ,y_{r^{\prime }}^b\right\}$ and the non-response items as ${\mathit{Y}}_{mis}^b=\left\{y_{r^{\prime }+1}^b,\cdots ,y_n^b\right\}$ in the bootstrap sample. The corresponding independent variable for $y_i^b$ is $x_i^b$. Then, we calculated

${\widehat{V}}_1=\frac{r}{n^2(n-1)M}{\displaystyle \sum }_{m=1}^M{\displaystyle \sum }_{i=1}^n{(y_i^{\ast m}-{\overline{y}}^m)}^2,$ where ${\overline{y}}^m=n^{-1}{\displaystyle \sum }_{i=1}^n{y}_i^{\ast m}$;

${\widehat{V}}_2=\frac{1}{M(M-1)}{\displaystyle \sum }_{m=1}^M{({\displaystyle \sum }_{j=r+1}^n\frac{1}{n}{y}_j^{\ast m}-H_1)}^2$, where $H_1=\frac{1}{M}{\displaystyle \sum }_{m=1}^M{\displaystyle \sum }_{j=r+1}^n\frac{1}{n}{y}_j^{\ast m}$;

${\widehat{V}}_3=\frac{1}{B-1}{\displaystyle \sum }_{b=1}^B{({\displaystyle \sum }_{j=r^{\prime }+1}^n\frac{1}{n}{\eta }_j^b-H_2)}^2$, where $H_2=\frac{1}{B}{\displaystyle \sum }_{b=1}^B{\displaystyle \sum }_{j=r^{\prime }+1}^n\frac{1}{n}{\eta }_j^b$ and

${\eta }_j^b=x_j^b{({\mathit{X}}_r^{bT}{\mathit{X}}_r^b)}^{-1}{\mathit{X}}_r^{bT}{\mathit{Y}}_{obs}^b$;

${\widehat{V}}_4=2{\displaystyle \sum }_{j=r+1}^n\frac{1}{n}{\displaystyle \sum }_{i=1}^r\frac{1}{n}{(B-1)}^{-1}{\displaystyle \sum }_{b=1}^B({\overline{\eta }}_j^b-H_3)({\overline{Y}}_{r^{\prime }}^b-H_4)$, where ${\overline{\eta }}_j^b={(n-r^{\prime })}^{-1}{\displaystyle \sum }_{j=r^{\prime }+1}^n{\eta }_j^b$, ${\overline{Y}}_{r^{\prime }}^b={(r^{\prime })}^{-1}{\displaystyle \sum }_{i=1}^{r^{\prime }}{y}_i^b,H_3=B^{-1}{\displaystyle \sum }_{b=1}^B{\overline{\eta }}_j^b$, and $H_4=B^{-1}{\displaystyle \sum }_{b=1}^B{\overline{Y}}_{r^{\prime }}^b$.

We derived the variance estimator based on Kim’s method [16]:

${\widehat{V}}_k=W_M+(1+M^{-1})B_M+\frac{2}{n^2}{\mathbf{1}}^T{\mathit{X}}_{n-r}^T{({\mathit{X}}_r^T{\mathit{X}}_r)}^{-1}{\mathit{S}}_r{\widehat{\sigma }}^2$,

where $\mathbf{1}$ is a vector with elements 1 of length (n − r); ${\mathit{X}}_{n-r}$ is an (n − r)$\times$ 2 matrix with rows ${\mathit{X}}_j=\left[1 x_j\right], j=r+1,\cdots , n$; ${\mathit{S}}_r= [r {\displaystyle \sum }_{i=1}^r{x}_i]$ is a 2 $\times$ 1 matrix; and ${\widehat{\sigma }}^2={(n-1)}^{-1}{M}^{-1}{\displaystyle \sum }_{m=1}^M{\displaystyle \sum }_{i=1}^n{(y_i^{\ast m}-{\overline{y}}^m)}^2$.

The simulation results are shown in

Table 2. Simulation results for the linear regression mean estimation.

				Rbias (%)				$\mathbf{Mwidth} (\times {10}^2)$				$95\mathbf{cov} (\times {10}^2)$
Scen	n	B	M	$\widehat{\mathit{V}}$	${\widehat{\mathit{V}}}_{\mathit{M}\mathit{I}}$	${\widehat{\mathit{V}}}_{\mathit{K}}$	${\widehat{\mathit{V}}}_{\mathit{B}}$	$\widehat{\mathit{V}}$	${\widehat{\mathit{V}}}_{\mathit{M}\mathit{I}}$	${\widehat{\mathit{V}}}_{\mathit{K}}$	${\widehat{\mathit{V}}}_{\mathit{B}}$	$\widehat{\mathit{V}}$	${\widehat{\mathit{V}}}_{\mathit{M}\mathit{I}}$	${\widehat{\mathit{V}}}_{\mathit{K}}$	${\widehat{\mathit{V}}}_{\mathit{B}}$
1	500	500	10	0.7	0.4	0.4	0.1	21	22	22	22	95	95	95	96
			30	0.8	0.2	0.3	0.1	20	20	20	20	95	95	95	95
		200	10	0.6	0.4	0.4	−0.1	21	22	22	22	95	95	95	96
			30	0.7	0.2	0.3	0.0	20	20	20	20	95	95	95	95
	1000	500	10	−1.8	−2.4	−2.3	−2.3	14	15	15	16	94	95	95	96
			30	−2.4	−2.9	−2.9	−2.9	14	14	14	14	94	95	95	95
		200	10	−2.0	−2.4	−2.4	−2.4	14	15	15	16	94	95	95	96
			30	−2.4	−2.9	−2.9	−2.8	14	14	14	14	94	95	95	95
2	500	500	10	−6.0	−5.0	−5.0	−5.0	13	14	14	14	94	94	94	95
			30	−5.0	−4.0	−4.0	−4.1	13	13	13	13	94	95	95	95
		200	10	−6.0	−5.0	−5.0	−5.0	13	14	14	14	94	94	94	95
			30	−5.0	−4.0	−4.0	−4.0	13	13	13	13	94	95	95	95
	1000	500	10	−1.0	0.4	0.4	0.0	9	10	10	10	95	95	95	96
			30	−0.6	0.8	0.8	0.6	9	9	9	9	95	95	95	95
		200	10	−1.1	0.4	0.4	−0.1	9	10	10	10	95	95	95	96
			30	−0.8	0.8	0.8	0.7	9	9	9	9	95	95	95	95

. We noticed that the biases of both Rubin’s method and Kim’s method were close to zero and the coverage rates were close to the nominal level. This indicated that the covariance assumption was satisfied and the bias term (4) was close to zero. The newly proposed method and the traditional bootstrap method had a similar performance as Rubin’s method and Kim’s method. This showed that they performed as well as the other two methods when their assumptions were satisfied. The newly proposed method was much faster than the traditional bootstrapping method. For a case with n = 1000, B = 500, M = 30, and 5000 Monte Carlo runs, the new method needed about half an hour, while the traditional bootstrap method needed about eight hours. In addition, same as in example 1, the B considered in example 2 had no significant effects, while larger n and M values led to smaller widths of the confidence intervals.

3.3. A Real Data Analysis

In order to illustrate the proposed method, we applied the newly proposed method to the Hox pupil popularity data [34]. The dataset collects information about pupil popularity, sex, school, and teacher experience. Pupil popularity is measured using a self-rating scale that ranges from 0 (very unpopular) to 10 (very popular). The sex denotes the gender of the students, which is boy or girl. The school variable indicates which schools the students are from. It includes 100 schools. The teacher experience records the teacher’s teaching experience in years. There are 2000 observations in the dataset.

Before doing any analysis on the dataset, we usually show descriptive statistics first. We were interested in the average pupil popularity for the domain third school. We selected the first 50 schools. Therefore, the data set had 998 observations. The values for the variable school are always observed. For the variable pupil popularity, there were 405 missing values. For the third school, there were 18 observations for pupil popularity and six were missing among them. Therefore, the domain proportion was 1.8% and the total response rate was approximately 60%. Then, the method used in example 1 in Section 4 was used to calculate the average pupil popularity of the third school. We used M = 30 and B = 500. Rubin’s point estimator was 6.41. The four methods used in the simulation study were used to estimate the variance of Rubin’s point estimator. The bootstrap variance estimator $\widehat{V}$ was 0.059, with the corresponding 95% confidence interval of (5.935, 6.887). Rubin’s variance estimator ${\widehat{V}}_{MI}$ was 0.109, with the corresponding 95% confidence interval of (5.762, 7.060). Kim’s variance estimator ${\widehat{V}}_k$ was 0.053, with the corresponding 95% confidence interval of (5.961, 6.862). The traditional bootstrap variance estimator ${\widehat{V}}_B$ was 0.066, with the corresponding 95% confidence interval of (5.932, 6.946). The results showed that the new proposed bootstrap variance estimator results were close to the traditional bootstrap variance estimator results. We observed that Rubin’s variance estimator ${\widehat{V}}_{MI}$ was larger than the bootstrap variance estimator $\widehat{V}$ and Kim’s variance estimator ${\widehat{V}}_k$ was smaller than the bootstrap variance estimator $\widehat{V}$. The results from the simulation studies indicated that the covariance assumption may not have been satisfied In addition, Rubin’s variance estimator may have resulted in a larger coverage probability and Kim’s variance estimator may have resulted in a lower coverage probability of the confidence interval than the nominal level.

4. Discussion and Conclusions

In this study, we proposed a new bootstrap variance estimator for a multiple imputation estimator with the complete-sample estimator in the form of (1). It does not need the covariance assumption used in Kim’s method, and thus, it has a wider range of applications. The simulations showed that it performed similarly to the traditional bootstrap method but greatly saved computational time. Moreover, the new method performed well for both situations with the covariance assumption satisfied or not. In addition, Kim’s method can only be applied to a situation where the imputed values can be written as a linear function of the observed values. However, the new method has no such restriction, and it can be applied to the multiple imputation estimator with any form of imputed values. Further research can be conducted to investigate its performance in such situations.

This newly proposed method can be extended to a more general class of estimators with the form ${\displaystyle \sum }_{i\in A}g(y_i)$, such as a maximum likelihood estimator, where the proportion of $y$ is less than any constant and the quantile of $y$. Future research can be conducted to investigate the performance when using these estimators.