Variational Bayesian Quantile Regression with Non-Ignorable Missing Response Data

Abstract

For non-ignorable missing response variables, the mechanism of whether the response variable is missing can be modeled through logistic regression. In Bayesian computation, the lack of a conjugate prior for the logistic function poses a significant challenge. Introducing a new Pólya-Gamma variable and employing lower-bound approximation are two common methods for parameter inference in conjugate Bayesian logistic regression. It can be observed that these two methods yield essentially the same variational posterior in the calculation of the variational Bayesian posterior. This paper applies a popular Bayesian spike-and-slab LASSO prior for variable selection in quantile regression with non-ignorable missing response variables, which demonstrates good performance in both simulations and practical applications.

1. Introduction

In many application fields, such as economics, sociology, and biomedicine, some subjects may have missing responses or predictors due to various reasons, including study dropout, unwillingness of study participants to answer certain questions in the questionnaire, and information loss caused by uncontrollable factors. Statistical inference for missing data problems is quite challenging.

Rubin [1] categorized missing data into three mechanisms: missing completely at random (MCAR), where the missingness process is independent of the observed and missing quantities; missing at random (MAR), where the missingness process depends on the observed quantities but not on the missing quantities; and non-ignorable missingness or not missing at random (NMAR), where the missingness process depends on both the observed and missing quantities. In missing data analysis, the NMAR assumption may be more reasonable than the classical MAR assumption.

Bayesian parameter estimation methods are often used to address the estimation problem of parametric models involving missing data for several reasons. First, Markov chain Monte Carlo (MCMC) methods widely used in statistical computing, such as the Gibbs algorithm [2] and the Metropolis–Hastings (MH) algorithm [3,4], can be employed to estimate the posterior distributions of parameters, nonparametric functions, and missing data. Second, compared to the setting without missing data, Bayesian methods with missing data only require an additional step in the Gibbs sampler. Therefore, Bayesian methods can easily handle missing data without the need for new statistical inference techniques [5]. Third, some prior information can be directly incorporated into the analysis, resulting in more accurate parameter estimation when good prior assumptions are available. Fourth, sampling-based Bayesian methods do not rely on asymptotic theory and may provide more reliable statistical inference even in small sample situations. In recent years, there have been many studies on NMAR data analysis, such as those by Lee and Tang [6], Tang and Zhao [7], and Xu and Tang [8].

Furthermore, variable selection can be viewed as a special case of model selection, which is achieved through spike-and-slab priors in Bayesian variable selection. This paper chooses the spike-and-slab LASSO prior [9] for parameter estimation and variable selection. The missingness mechanism of the response variable can be obtained through logistic regression, and variational Bayesian algorithms are considered for model parameter estimation. In the calculation of variational posteriors, due to the lack of a conjugate prior for logistic regression, we cannot compute the variational posteriors by specifying a reasonable variational family. Introducing Pólya-Gamma latent variables [10] and employing lower-bound approximation [11] are two methods that can yield conjugate posteriors for Bayesian logistic regression. In the posterior computation process of variational Bayesian methods, these two methods will have the same variational posteriors. Considering the characteristics of the spike-and-slab prior, it is unnecessary to calculate the complex variational lower bound, and the algorithm can still converge.

In this study, we propose a variational Bayesian quantile regression algorithm to address the challenges posed by non-ignorable response missing data. Unlike traditional methods, the variational Bayesian approach offers an efficient and scalable solution by transforming the posterior inference problem into an optimization task, ensuring faster convergence and reducing computational burden. The quantile regression framework provides a robust alternative to mean-based models, capturing the conditional distribution of the response variable across different quantiles, which is particularly valuable when the data exhibit heteroscedasticity or skewness. While Li’s paper [12] also considers missing covariates and response variables, it does not address variable selection. In our work, we employ a prior that enables variable selection, allowing for effective variable screening alongside parameter estimation. Additionally, the convergence criteria of our algorithm differ from Li’s. Li uses the minimal change in the variational lower bound as the stopping condition, which involves more complex computations. In contrast, our approach is more computationally efficient. Moreover, the proposed method incorporates variable selection through a Bayesian shrinkage prior, effectively identifying significant predictors while accounting for the missing data mechanism. This combination of variational inference, quantile regression, and variable selection not only enhances estimation accuracy but also offers a flexible and computationally efficient tool for analyzing complex missing data structures. These features highlight the novelty and practical relevance of our approach.

This article is organized as follows: Section 2 introduces the model, prior, and variational Bayesian logistic regression used in this paper; Section 3 proposes the corresponding variational Bayesian algorithm for data with non-ignorable missing responses; Section 4 conducts simulation studies for the proposed algorithm; Section 5 applies the algorithm to real data analysis; and relevant conclusions are presented in Section 6.

2. Model Prior and Variational Bayesian Logistic Regression

In this paper, the response variable $y_i$, $i=1, 2, \cdots , n$ may be missing, while all covariates (explanatory variables) $X_i=\left(x_{i1},\cdots ,x_{ip}\right)$ are completely observable. The incomplete observations are as follows:

$${(X_i,y}_i,r_i),i=1, 2, \cdots , n.$$

(1)

where $r_i\in \left\{0,1\right\}$ determines whether $y_i$ is missing. When $r_i=1$, $y_i$ is missing. Let $\mathit{y}=\{{\mathit{y}}_o,{\mathit{y}}_m\}$, $\mathit{r}=(r_1,{ r}_2, \cdots ,{ r}_n)$, where ${\mathit{y}}_o$ and ${\mathit{y}}_m$ represent the observed response variables and the missing response variables, respectively. Let $p\left(r_i\mid y_i,X_i,\phi \right)$ denote the conditional distribution of $r_i$ given $y_i$, $X_i$, and φ, where φ is the unknown parameter vector in the conditional probability function $p\left(r_i\mid y_i,X_i,\phi \right)$. The missingness mechanism of the data is completely determined by this conditional distribution.

We consider the following non-ignorable missingness mechanism:

$$p\left(r_i\mid y_i,X_i,\phi \right)={\left\{p\left(r_i=1\mid y_i,X_i,\phi \right)\right\}}^{r_i}{\left\{1-p\left(r_i=1\mid y_i,X_i,\phi \right)\right\}}^{1-r_i}$$

(2)

where $p\left(r_i\mid y_i,X_i,\phi \right)$ can be modeled through logistic regression:

$$\mathrm{l}\mathrm{o}\mathrm{g}\mathrm{i}\mathrm{t}\left(p\left(r_i\mid y_i,X_i,\phi \right)\right)={\phi }_{01}+{\phi }_{11}{y}_i+{\phi }_{21}{x}_{i1}+\cdots +{\phi }_{2p}{x}_{ip}=W_i^T\mathit{\phi }$$

(3)

where $\mathit{\phi }={\left({\phi }_{01},{\phi }_{11},{\phi }_{21},\cdots ,{\phi }_{2p}\right)}^T$ are the logistic regression model parameters, and $W_i^T=\left(1,y_i,x_{i1},\cdots ,x_{ip}\right)$ are the covariates of the logistic regression model.

The quantile regression model with non-ignorable missing responses. For $0<\tau <1$, it is constrained such that the $\tau$-th quantile equals zero. The $\tau$-th quantile regression model is expressed as follows:

$$Q_{\tau }\left(y_i\mid x_i\right)=X_i^T\beta ,i=1,\cdots ,n.$$

(4)

$$p\left(r_i=1\mid y_i,x_i,\mathit{\phi }\right)={\displaystyle \frac{e^{W_i^T\mathit{\phi }}}{1+e^{w_i^T\mathit{\phi }}}}=\sigma \left(W_i^T\mathit{\phi }\right)$$

(5)

where β = (β₁, …, β_p)^T are the quantile regression model parameters to be estimated when $r_i=1,y_i$ is missing.

2.1. Spike-and-Slab LassoLASSO Prior

The spike-and-slab LassoLASSO (SSL) prior [9] is represented as follows:

$${\beta }_j\mid {\gamma }_j={\gamma }_j{\phi }_1\left({\beta }_j\right)+\left(1-{\gamma }_j\right){\phi }_0\left({\beta }_j\right),j=1,2,\dots ,p,$$

(6)

$${\phi }_1\left({\beta }_j\right)={\displaystyle \frac{{\lambda }_1}{2}}\exp \left(-{\lambda }_1\left|{\beta }_j\right|\right),{\phi }_0\left({\beta }_j\right)={\displaystyle \frac{{\lambda }_0}{2}}\exp \left(-{\lambda }_0\left|{\beta }_j\right|\right).$$

(7)

where λ₁ is chosen to be a smaller value, while λ₀ should be chosen to be a larger value. In the Bayesian framework, the Laplace distribution is not conjugate, but it can be hierarchically represented using the normal distribution (⋅, ⋅) and the exponential distribution Exp(⋅):

$$p\left({\beta }_j\mid {\tau }_{1j}^2,{\gamma }_j=1\right)~N\left(0,{\tau }_{1j}^2\right), {p(\tau }_{1j}^2\left|{\lambda }_1^2\right)~Exp\left({\displaystyle \frac{{\lambda }_1^2}{2}}\right)$$

(8)

$$p\left({\beta }_j∣{\tau }_{0j}^2,{\gamma }_j=0\right)~N\left(0,{\tau }_{0j}^2\right), {p(\tau }_{0j}^2\left|{\lambda }_0^2\right)~Exp\left({\displaystyle \frac{{\lambda }_0^2}{2}}\right)$$

(9)

Figure 1 describes four types of spike-and-slab priors with a mixing proportion of ${\gamma }_j=1/2$. They are the Normal mixture (where both ${\phi }_1\left({\beta }_j\right)$ and ${\phi }_0\left({\beta }_j\right)$ are normal distributions), the Normal and Point-mass mixture (where ${\phi }_0\left({\beta }_j\right)$ is a point-mass function at 0 and ${\phi }_1\left({\beta }_j\right)$ is a normal distribution), the Laplace and Point-mass mixture (where ${\phi }_0\left({\beta }_j\right)$ is a point-mass function at 0 and ${\phi }_1\left({\beta }_j\right)$ is a Laplace distribution), and the SSL (where both are Laplace distributions). It can be seen that the Normal mixture cannot adequately penalize smaller coefficients, making it difficult to achieve variable selection. On the other hand, the point-mass spike-and-slab prior has an over-shrinkage problem and may miss important variables. Therefore, the SSL prior can be considered as a balance between the two.

Penalized priors (such as LASSO) and spike-and-slab priors are common priors in Bayesian variable selection. When ${\lambda }_1={\lambda }_0=\lambda ,$ the SSL prior degenerates to the LassoLASSO prior. When ${\lambda }_0 \to \infty$, ${\phi }_0\left({\beta }_j\right)\to 0$. That is, in the limit case, SSL can be transformed into the “gold standard” point-mass spike-and-slab prior. Thus, SSL integrates the penalized likelihood (LassoLASSO) and the spike-and-slab prior. SSL is also adaptive (there is no spike-and-slab adaptive LassoLASSO), and the proof of its adaptivity can be found in the discussion by Rockova and George [9]. SSL uses the spike component to encourage sparsity by shrinking many regression coefficients to zero, while the slab component captures larger signals, enabling simultaneous variable selection and parameter estimation. Unlike traditional LassoLASSO with fixed regularization, SSL automatically adjusts the sparsity parameter based on the data, reducing the need for manual tuning and providing multiplicity correction, which lowers the false positive rate.

2.2. Bayesian Logistic Regression Based on Pólya-Gamma Latent Variables

The logistic function does not have a conjugate prior, which poses a challenge for Bayesian inference in logistic regression. Polson et al. [10] proposed a new data augmentation strategy for Bayesian inference in logistic regression models.

If ω∼P (b, c), where P (b, c) denotes the Pólya-Gamma distribution with parameters (b, c), its expectation is

$$\mathbb{E}\left(\omega \right)={\displaystyle \frac{b}{2c}}\tanh \left({\displaystyle \frac{c}{2}}\right)={\displaystyle \frac{b}{2c}}\left({\displaystyle \frac{e^c-1}{e^c+1}}\right)$$

The probability density of ω has the following property:

$$p(\omega \mid b,c)\propto \mathrm{e}\mathrm{x}\mathrm{p}\left(-{\displaystyle \frac{c^2}{2}}\omega \right)p(\omega \mid b,0)$$

If ω∼(b, 0) and p(ω) represents its probability density, then

$${\displaystyle \frac{{\left(e^{\psi }\right)}^a}{{\left(1+e^{\psi }\right)}^b}}=2^{-b}{e}^{\kappa \psi }{\int }_0^{\mathrm{\infty }} e^{-\omega {\psi }^2/2}p(\omega )d\omega$$

where $b>0, \kappa =a-b/2$. Applying this result to (5) (see Section 3.1 of Polson et al. [10]), we obtain that

$$L\left(\mathit{\phi },\mathit{\omega }\right)={\prod }_{i=1}^n{\displaystyle \frac{{\left(e^{{W_i}^T\mathit{\phi }}\right)}^{r_i}}{1+e^{{W_i}^T\mathit{\phi }}}}\propto {\prod }_{i=1}^n \mathrm{e}\mathrm{x}\mathrm{p}\left\{\left(r_i-{\displaystyle \frac{1}{2}}\right)\left({W_i}^T\mathit{\phi }\right)-{\displaystyle \frac{{\omega }_i}{2}}{\left({W_i}^T\mathit{\phi }\right)}^2\right\}p\left({\omega }_i\mid 1,0\right)$$

where $\mathit{\omega }=({\omega }_1,\dots ,{\omega }_n)$, ${\omega }_i~PG(1,0)$, and its probability density is denoted as $p\left({\omega }_i\mid 1,0\right)$.

Assuming a prior $p\left(\mathit{\phi }\right)$ for $\mathit{\phi }$, the posterior of $\mathit{\phi }$ is as follows:

$$q(\mathit{\phi }\mid \mathit{\omega },\mathit{y})\propto p\left(\mathit{\phi }\right){\prod }_{i=1}^n \mathrm{e}\mathrm{x}\mathrm{p}\left\{{\kappa }_i{W_i}^T\mathit{\phi }-{\omega }_i{\left({W_i}^T\mathit{\phi }\right)}^2/2\right\}$$

where ${\kappa }_i=r_i-1/2$. If $p\left(\mathit{\phi }\right)$ is a Gaussian prior, then the posterior $q(\mathit{\phi }\mid \mathit{\omega },\mathit{y})$ is conjugate with the prior. The posterior density of the variable ${\omega }_i$ is

$$q({\omega }_i|\mathit{\phi })\propto \mathrm{e}\mathrm{x}\mathrm{p}\left\{-{\displaystyle \frac{{\omega }_i}{2}}{\left({W_i}^T\mathit{\phi }\right)}^2\right\}p\left({\omega }_i\mid 1,0\right)$$

Then, the posterior of ${\omega }_i$ is $PG(1,{W_i}^T\mathit{\phi })$, and $\mathbb{E}\left({\omega }_i|\mathit{\phi }\right)={\displaystyle \frac{1}{2{W_i}^T\mathit{\phi }}}\tanh \left({\displaystyle \frac{{W_i}^T\mathit{\phi }}{2}}\right)$.

2.3. Bayesian Logistic Regression Based on Lower-Bound Approximation

The log-likelihood of $\left(5\right)$ is as follows:

$$l\left(\mathit{\phi }\right)=\log L\left(\mathit{\phi }\right)=\log {\prod }_{i=1}^n{\displaystyle \frac{{(e^{{W_i}^T\mathit{\phi }})}^{r_i}}{1+e^{{W_i}^T\mathit{\phi }}}}={\sum }_{i=1}^n r_i{W_i}^T\mathit{\phi }-g({W_i}^T\mathit{\phi })$$

where $g\left(t\right)=\log (1+e^t)$, $t\in \mathbb{R}$. We take the logarithm of the logistic function:

$$\log \sigma \left(t\right)=\log {(1+e^{-t})}^{-1}={\displaystyle \frac{t}{2}}-\log (e^{{\displaystyle \frac{t}{2}}}+e^{-{\displaystyle \frac{t}{2}}})$$

Jaakkola and Jordan [11] approximate $\log (e^{{\displaystyle \frac{t}{2}}}+e^{-{\displaystyle \frac{t}{2}}})$ using a first-order Taylor expansion:

$$\begin{gathered} \log \sigma \left(t\right)\ge {\displaystyle \frac{t}{2}}-\log \left(e^{{\displaystyle \frac{\eta }{2}}}+e^{-{\displaystyle \frac{\eta }{2}}}\right)-{\displaystyle \frac{1}{4\eta }}\tanh \left({\displaystyle \frac{\eta }{2}}\right)\left(t^2-{\eta }^2\right) \\ ={\displaystyle \frac{t-\eta }{2}}+\log \sigma \left(\eta \right)-{\displaystyle \frac{1}{4\eta }}\tanh \left({\displaystyle \frac{\eta }{2}}\right)\left(t^2-{\eta }^2\right) \end{gathered}$$

Substituting $-g\left({W_i}^T\mathit{\phi }\right)=\log \sigma \left({{-W}_i}^T\mathit{\phi }\right)$ into $l\left(\mathit{\phi }\right)$:

$$l\left(\mathit{\phi }\right)\ge {\sum }_{i=1}^n\log \sigma \left({\eta }_i\right)-{\displaystyle \frac{{\eta }_i}{2}}+\left(r_i-{\displaystyle \frac{1}{2}}\right){W_i}^T\mathit{\phi }-{\displaystyle \frac{1}{4{\eta }_i}}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}\left({\eta }_i/2\right)\left({\left({W_i}^T\mathit{\phi }\right)}^2-{\eta }_i^2\right)=:f(\mathit{\phi },\eta )$$

We want the lower bound $f(\mathit{\phi },\eta )$ of $l\left(\mathit{\phi }\right)$ to be as large as possible. For a given $\mathit{\phi }$, the lower bound $f\left(\mathit{\phi },\eta \right)=f\left(\eta \right)$, so we maximize $f\left(\eta \right)$. We define a function $f_a(\mathbb{R}\to \mathbb{R},a\ge 0)$:

$$f_a(x)=\mathrm{l}\mathrm{o}\mathrm{g}\sigma \left(x\right)-{\displaystyle \frac{x}{2}}-{\displaystyle \frac{1}{4x}}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}(x/2)\left(a^2-x^2\right)$$

Then, $f_a\left(x\right)$ is symmetric about $x=0$, and $f_a\left(x\right)$ reaches its maximum at $x=\pm a$. The proof of this conclusion can be found in Kolyan Ray et al.’s work [13]. Therefore, when ${\eta }_i={W_i}^T\mathit{\phi }$, the lower bound $f\left(\eta \right)$ reaches its maximum value.

The posterior of $\mathit{\phi }$ is as follows:

$$\begin{gathered} q\left(\mathit{\phi }\right)=\mathrm{c}p\left(\mathit{\phi }\right)L\left(\mathit{\phi }\right)\approx \mathrm{c}p\left(\mathit{\phi }\right)\mathrm{e}\mathrm{x}\mathrm{p}\left\{f\left(\mathit{\phi },\eta \right)\right\} \\ \propto p(\mathit{\phi }){\prod }_{i=1}^n \mathrm{e}\mathrm{x}\mathrm{p}\left\{\left(r_i-{\displaystyle \frac{1}{2}}\right){W_i}^T\mathit{\phi }-{\displaystyle \frac{1}{4{\eta }_i}}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}\left({\eta }_i/2\right){\left({W_i}^T\mathit{\phi }\right)}^2\right\} \\ =p(\mathit{\phi }){\prod }_{i=1}^n \mathrm{e}\mathrm{x}\mathrm{p}\left\{\left(r_i-{\displaystyle \frac{1}{2}}\right){W_i}^T\mathit{\phi }-\mathbb{E}\left({\omega }_i|\mathit{\phi }\right){\left({W_i}^T\mathit{\phi }\right)}^2/2\right\} \end{gathered}$$

Comparing the posterior q($\mathit{\phi }$) under these two methods, we find that whether using the lower bound approximation or introducing the new variable Pólya-Gamma method, $\mathit{\phi }$ will have essentially the same variational posterior.

3. Variational Bayesian Algorithm for Bayesian Quantile Regression with Non-Ignorable Missing Responses

3.1. Hierarchical Model and Prior

In Bayesian quantile regression, it is often assumed that $y_i$ follows an asymmetric Laplace (AL) distribution, where

$$f\left(y_i\mid \mathit{\beta },\sigma \right)={\displaystyle \frac{\tau (1-\tau )}{\sigma }}\exp \left\{-{\rho }_{\tau }\left({\displaystyle \frac{y-X_{ij}^T\mathit{\beta }}{\sigma }}\right)\right\}.$$

where ${\rho }_{\tau }\left(u\right)=u(\tau -I(u<0\left)\right)$, $I(\cdot )$ is an indicator function.according According to the findings of Kozumi and Kobayashi [14], the asymmetric Laplace (AL) distribution is expressed as a mixture of a normal distribution and an exponential distribution.

$$\left\{\begin{array}{c}{y}_i|\beta ,\sigma ,e_i~N\left(X_i^T\mathit{\beta }+k_1{e}_i,\sigma k_2{e}_i\right),\\ e_i|\sigma ~ Exp\left({\displaystyle \frac{1}{\sigma }}\right),\end{array}\right.$$

(10)

where $k_1=(1-2\tau )/\left(\tau \left(1-\tau \right)\right)$, $k_2=2/\left(\tau \left(1-\tau \right)\right)$.

The Bayesian quantile regression model with non-ignorable missing responses has the following hierarchical representation:

$$\begin{gathered} y_i\sim N\left(X_i^T\beta +k_1{e}_i,k_2\sigma e_i\right), when r_i=1, y_i is missing. \\ {r}_i|{\omega }_i\propto \mathrm{e}\mathrm{x}\mathrm{p}\left\{\left(r_i-{\displaystyle \frac{1}{2}}\right)\left({W_i}^T\mathit{\phi }\right)-{\displaystyle \frac{{\omega }_i}{2}}{\left({W_i}^T\mathit{\phi }\right)}^2\right\}, \\ {\omega }_i\sim PG\left(0,1\right),i=1,2,\cdots ,n. \\ {e}_i\mid \sigma \sim exp\left({\displaystyle \frac{1}{\sigma }}\right). \end{gathered}$$

SSL prior can be hierarchically represented using the normal distribution and the exponential distribution:

$$\begin{gathered} {\beta }_j\left|{\tau }_{1j}^2,{\gamma }_j=1\sim N\left(0,{\tau }_{1j}^2\right),{\tau }_{1j}^2\right|{\lambda }_1^2\sim Exp\left({\lambda }_1^2/2\right),j=1,2,\cdots ,p. \\ {\beta }_j\left|{\tau }_{0j}^2,{\gamma }_j=0\sim N\left(0,{\tau }_{0j}^2\right),{\tau }_{0j}^2\right|{\lambda }_0^2\sim Exp\left({\lambda }_0^2/2\right). \\ {\gamma }_j\mid \pi \sim Bernoulli\left(\pi \right) \end{gathered}$$

The prior of hyperparameters:

$$\begin{gathered} \sigma \sim IG\left(a_{{\sigma }_0},b_{{\sigma }_0}\right),\pi \sim Beta\left(a_0,b_0\right) \\ {\lambda }_1^2\sim Ga\left(c_1,d_1\right),{\lambda }_0^2\sim Ga\left(c_0,d_0\right) \end{gathered}$$

where $\mathrm{G}\mathrm{a}\left(a,b\right)$ and $\mathrm{I}\mathrm{G}(a,b)$ represent the gamma distribution and inverse gamma distribution, respectively, with shape parameter $a$ and scale parameter $b$. $Beta\left(·,·\right)$ isas beta distribution. If $p\left(\mathit{\phi }\right)$ is a Gaussian prior:

$$\mathit{\phi }\sim N\left({\mu }_{{\mathit{\phi }}_0},{\Sigma }_{{\mathit{\phi }}_0}\right)$$

In the incomplete data setting, the missing data ${\mathit{y}}_m$ introduce additional complexity because the posterior distribution of the model parameters $(\mathit{\beta },\sigma ,\mathit{\phi })$ is a marginal distribution:

$$q\left(\mathit{\beta },\sigma ,\mathit{\phi }|{\mathit{y}}_o,X,\mathit{r}\right)=\int q\left(\mathit{\beta },\sigma ,\mathit{\phi },{\mathit{y}}_m|{\mathit{y}}_o,X,\mathit{r}\right)d{\mathit{y}}_m$$

This marginalization often results in a complicated and intractable posterior distribution since the integral rarely has a closed-form solution. By augmenting the data with ${\mathit{y}}_m$, the posterior becomes a complete-data posterior:

$$q\left(\mathit{\beta },\sigma ,\mathit{\phi },{\mathit{y}}_m|{\mathit{y}}_o,X,\mathit{r}\right)$$

which is typically easier to work with because—conditionally on the imputed ${\mathit{y}}_m$—the model often simplifies to standard forms like conjugate priors, making Bayesian updating more straightforward.

3.2. Coordinate Ascent Variational Inference Algorithms

Let $\mathit{\theta }\in \mathsf{\Theta }$, $D$ is the observed data, and $p\left(\mathit{\theta }\right|D)$ is the conditional posterior distribution of $\mathit{\theta }$. Variational Bayesian methods aim to find a distribution $q\left(\mathit{\theta }\right)$ within a given family of distributions $\mathcal{F}$, minimizing the Kullback–Leibler (KL) divergence between $q\left(\mathit{\theta }\right)$ and $p\left(\mathit{\theta }\right|D)$. Denoting $p\left(\mathit{\theta }\right)$ as the prior distribution of the parameters, we have

$$\log p\left(D\right)=\int \log {\displaystyle \frac{p\left(\mathit{\theta }\right)p\left(D|\mathit{\theta }\right)}{q\left(\mathit{\theta }\right)}}q\left(\mathit{\theta }\right)d\mathit{\theta }+\int \log {\displaystyle \frac{q\left(\mathit{\theta }\right)}{p\left(\mathit{\theta }|D\right)}}q\left(\mathit{\theta }\right)d\mathit{\theta }.$$

In this equation, the first term is the evidence lower bound observed (ELBO), and the second term is the Kullback–Leibler (KL) divergence between $q\left(\mathit{\theta }\right)$ and $p\left(\mathit{\theta }\right|D)$, denoted as $\mathrm{K}\mathrm{L}\left(q\right|\left|p\right)$. The logarithm of the likelihood of the observed data, $\log p\left(D\right)$, is independent of the choice of $q\left(\mathit{\theta }\right)$.

Therefore, minimizing $\mathrm{K}\mathrm{L}\left(q\right|\left|p\right)$ is equivalent to maximizing the evidence lower bound:

$$\widehat{q}\left(\mathit{\theta }\right)=\arg \underset{q\left(\mathit{\theta }\right)\in \mathcal{F}}{\max }\left\{E_{q\left(\mathit{\theta }\right)}\left[ \log p\left(D,\mathit{\theta }\right)\right]-E_{q\left(\mathit{\theta }\right)}\left[ \log q\left(\mathit{\theta }\right)\right]\right\}.$$

For ease of solving $\widehat{q}\left(\mathit{\theta }\right)$, consider mean-field Variational Bayesian Bayes (MFVB). Coordinate Ascent Variational Inference (CAVI) is one of the most commonly used VB algorithms. CAVI iteratively optimizes each factor of the mean-field variational density while keeping the other factors fixed, ultimately increasing the ELBO until a local optimum is reached. Let $\widehat{q}\left(\mathit{\theta }\right)=\prod _{t=1}^T{q}_t\left({\theta }_t\right)$, and denote ${\mathit{\theta }}_{-t}=({\theta }_1,\dots ,{\theta }_{t-1},{\theta }_{t+1},...,{\theta }_T)$. According to the results of Blei et al. [15], the optimal variational posterior is given by

$$q^{*}\left({\theta }_t\right)\propto \exp \left\{E_{{\mathit{\theta }}_{-t}}\left[\log p\left({\theta }_t|D,{\mathit{\theta }}_{-t}\right)\right]\right\}.$$

3.3. Variational Posterior

In this paper, the parameters under consideration are

$$\mathit{\theta }=\left(\mathit{\beta },{\left\{{\tau }_{0j}^2,{\tau }_{1j}^2,{\gamma }_j\right\}}_{j=1}^p,\pi ,{\lambda }_1^2,{\lambda }_2^2,{\left\{e_i,{\omega }_i\right\}}_{i=1}^n,\sigma ,\mathit{\phi },y_m\right)$$

The derivation of the variational posterior is provided in Appendix A, and the variational posterior of β is

$$q\left(\mathit{\beta }\right)\sim N\left(\mathit{\mu },\mathit{\Sigma }\right),$$

The variational posterior of ${\tau }_{1j}^2$ is

$$q\left({\tau }_{1j}^2\right)\sim GIG\left({\displaystyle \frac{1}{2}},a_{1j},b_{1j}\right),$$

The variational posterior of ${\tau }_{0j}^2$ is

$$q({\tau }_{0j}^2)\sim GIG({\displaystyle \frac{1}{2}},a_{0j},b_{0j}),$$

The Variationalvariational posterior of ${\gamma }_j$ is:

$$q\left({\gamma }_j\right)\sim \mathrm{Bernoulli}\left({\varphi }_j\right),$$

The Variationalvariational posterior of $\pi$ is

$$\mathrm{q}\left(\pi \right)~\mathit{Beta}\left(a,b\right),$$

The Variationalvariational posterior of ${\mathsf{\lambda }}_1^2$ is

$$\mathrm{q}\left({\lambda }_1^2\right)~\Gamma (\widetilde{c_1},\widetilde{d_1}),$$

The Variationalvariational posterior of ${\mathsf{\lambda }}_0^2$ is

$$\mathrm{q}\left({\lambda }_0^2\right)~\Gamma (\widetilde{c_0},\widetilde{d_0}),$$

The Variationalvariational posterior of $e_i$:

$$q(e_i)~GIG({\displaystyle \frac{1}{2}},a_e={\displaystyle \frac{{\left(y_i-{X_i}^T\mathbb{E}\left[\mathit{\beta }\right]\right)}^2}{\sigma k_2}},b_e=({k_1}^2+2k_2)/(\sigma k_2)).$$

The Variationalvariational posterior of $\sigma$:

$$q\left(\sigma \right)~IG(a_{\sigma }={\displaystyle \frac{3n}{2}}+r_0,b_{\sigma }),$$

The Variationalvariational posterior ($\mathit{\phi }$):

$$q\left(\mathit{\phi }\right)~\mathrm{N}({\mathsf{\mu }}_{\mathit{\phi }},{\mathsf{\Sigma }}_{\mathit{\phi }}),$$

The Variationalvariational posterior of ${\omega }_i$ is

$$q\left({\omega }_i\right)~PG\left(1,{\widehat{c}}_i\right),$$

For a missing individual $i$, denoted as $y_{im}$, suppose there are $n_m$ individuals missing in total, the Variationalvariational posterior is

$$q\left(y_{im}\right)\sim N({\mu }_y,{\sigma }_y),$$

The CAVI algorithm is used to update the parameters in the posterior distributions until convergence. A common convergence criterion is that the ELBO no longer changes (or the change is small enough to be ignored). Since we focus on variable selection and the ELBO involves many complex expectations and entropy calculations, we calculate the entropy of $\mathit{\varphi }=({\varphi }_1, \cdots , {\varphi }_p)$ until it no longer changes (or the change is very small, less than a given threshold). It can be seen that, when ϕ tends to 0, the algorithm converges. Please refer to Algorithm 1.

E(ϕ) = ϕ log(ϕ) + (1_p − ϕ) log (1_p − ϕ)

Algorithm 1: Variational Bayesian algorithm with non-negligible missing response

Variational Bayesian parameter initialization;

Setting initial values$: \delta ={10}^{-3},J=1000,t=1,\mathrm{E}\mathrm{n}\mathrm{t}(\varphi {)}^{\left(0\right)}=0$;

while  $\left|\mathrm{E}\mathrm{n}\mathrm{t}(\mathit{\varphi }{)}^{\left(t\right)}-\mathrm{E}\mathrm{n}\mathrm{t}(\mathit{\varphi }{)}^{(t-1)}\right|⩾\delta$ and $1⩽t⩽J$ do

${\mathsf{\Sigma }}^{\left(t\right)}\leftarrow {\left({\displaystyle \frac{{\mathrm{E}}^{(\mathrm{t}-1)}[1/\sigma ]}{k_2}}{\sum }_{i=1}^n {\displaystyle \frac{X_i^{\tau }{X}_i}{\mathrm{E}\left[e_i\right]}}+{\mathbb{E}}^{(t-1)}\left[D_{\tau }^{-1}\right]\right)}^{-1};$

${\mathit{\mu }}^{\left(t\right)}\leftarrow \mathsf{\Sigma }\left({\displaystyle \frac{{\mathrm{E}}^{(\mathrm{t}-1)}[1/\sigma ]}{k_2}}{\sum }_{i=1}^n X_i^T\left({\mathbb{E}}^{(t-1)}\left[{\displaystyle \frac{1}{e_i}}\right]\left(y_i-k_1\right)\right)\right);$

$a_{1j}^{\left(t\right)}\leftarrow {\mathbb{E}}^{(t-1)}\left[{\gamma }_j\right]{\mathbb{E}}^{(t-1)}\left[{\beta }_j^2\right],b_{1j}^{\left(t\right)}\leftarrow {\mathbb{E}}^{(t-1)}\left[{\gamma }_j\right]{\mathbb{E}}^{(t-1)}\left[{\lambda }_1^2\right];$

$a_{0j}^{\left(t\right)}\leftarrow {\mathbb{E}}^{(t-1)}\left[\left(1-{\gamma }_j\right)\right]{\mathbb{E}}^{(t-1)}\left[{\beta }_j^2\right],b_{0j}^{\left(t\right)}={\mathbb{E}}^{(t-1)}\left[\left(1-{\gamma }_j\right)\right]{\mathbb{E}}^{(t-1)}\left[{\lambda }_0^2\right];$

$\mathrm{l}\mathrm{o}\mathrm{g}\mathrm{i}\mathrm{t}{\left({\varphi }_j\right)}^{\left(t\right)}\leftarrow {\mathbb{E}}^{(t-1)}\left[{\displaystyle \frac{1}{2}}\mathrm{l}\mathrm{o}\mathrm{g}\left({\displaystyle \frac{{\tau }_{0j}^2}{{\tau }_{1j}^2}}\right)+{\displaystyle \frac{{\beta }_1^2}{2}}\left({\displaystyle \frac{1}{{\tau }_{0j}^2}}-{\displaystyle \frac{1}{{\tau }_{1j}^2}}\right)+\mathrm{l}\mathrm{o}\mathrm{g}\left({\displaystyle \frac{{\lambda }_1^2}{{\lambda }_0^2}}\right)+{\displaystyle \frac{{\lambda }_1^2}{2}}{\tau }_{0j}^2-{\displaystyle \frac{{\lambda }_1^2}{2}}{\tau }_{1\mathrm{j}}^2+\mathrm{l}\mathrm{o}\mathrm{g}\left({\displaystyle \frac{\pi }{1-\pi }}\right)\right]$;

$a^{\left(t\right)}\leftarrow a_0+{\sum }_{j=1}^p {\mathbb{E}}^{(t-1)}\left[{\gamma }_j\right],b^{\left(t\right)}\leftarrow b_0+p-{\sum }_{j=1}^p {\mathbb{E}}^{(t-1)}\left[{\gamma }_j\right];$

${\tilde{c}}_1^{\left(t\right)}\leftarrow c_1+{\sum }_{j=1}^p {\mathbb{E}}^{(t-1)}\left[{\gamma }_j\right],{\tilde{d}}_1^{\left(t\right)}\leftarrow d_1+{\sum }_{j=1}^p {\displaystyle \frac{{\mathrm{E}}^{(j-1)}\left[{\gamma }_j\right]{\mathbb{E}}^{(i-1)}\left[r_1^2\right]}{2}};$

${\tilde{c}}_0^{\left(t\right)}\leftarrow c_0+p-{\sum }_{j=1}^p {\mathbb{E}}^{(t-1)}\left[{\gamma }_j\right],{\tilde{d}}_0^{\left(t\right)}\leftarrow d_0+{\sum }_{j=1}^p {\displaystyle \frac{{\mathbb{E}}^{(t-1)}\left[\left(1-{\gamma }_j\right)\right]{\mathrm{E}}^{(t-1)}\left[{\tau }_0^2\right]}{2}};$

$a_e^{\left(t\right)}\leftarrow {\displaystyle \frac{{\left(y_1-X_i^T{\mathrm{E}}^{(t-1)}\left[\beta \right]\right)}^2}{\sigma k_2}},b_e^{\left(t\right)}\leftarrow \left({k_1}^2+2k_2\right)/\left(\sigma k_2\right);$

$a_{\sigma }^{\left(t\right)}\leftarrow {\displaystyle \frac{3n}{2}}+a_{{\sigma }_0},b_{\sigma }^{\left(t\right)}\leftarrow b_{{\sigma }_0}+{\displaystyle \frac{1}{2k_2}}{\sum }_{i=1}^n {\displaystyle \frac{{\left(y_i-X_i^T{\mathrm{E}}^{(t-1)}\left[\beta \right]-k_1{\mathbb{E}}^{(t-1)}\left[e_i\right]\right)}^2}{{\mathbb{E}}^{\left[\right(-1)}\left[e_i\right]}}+{\sum }_{i=1}^n {\mathbb{E}}^{(t-1)}\left[e_i\right];$

${\mathsf{\Sigma }}_{\phi }^{\left(t\right)}\leftarrow {\left(W{\mathbb{E}}^{(t-1)}[\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}(\omega \left)\right]W^T+{\mathsf{\Sigma }}_{{\phi }_0}^{-1}\right)}^{-1};$

${\mu }_{\phi }^{\left(t\right)}\leftarrow {\mathsf{\Sigma }}_{\phi }\left(W\left(\mathit{r}-{\displaystyle \frac{1}{2}}{\mathbf{1}}_n\right)+{\mathsf{\Sigma }}_{{\phi }_0}^{-1}{\mu }_{{\phi }_0}\right);$

${\hat{c}}_i^{\left(t\right)}\leftarrow \sqrt{{\mathbb{E}}^{(t-1)}\left[{\left(w_i^T\phi \right)}^2\right]};$

${\sigma }_y^{\left(t\right)}\leftarrow {\left({\phi }_1^2{\mathbb{E}}^{(t-1)}\left[{\omega }_{im}\right]+{\mathbb{E}}^{(t-1)}\left[{\displaystyle \frac{1}{2\sigma k_2{e}_{im}}}\right]\right)}^{-1};$

${\mu }_y^{\left(t\right)}\leftarrow {\sigma }_y\left({\phi }_1\left(r_{im}-{\displaystyle \frac{1}{2}}\right)+{\mathbb{E}}^{(t-1)}\left[{\displaystyle \frac{X_{mm}^T\beta +k_1{e}_{mm}}{\sigma k_2{e}_{im}}}\right]\right);$

Update $\mathrm{E}\mathrm{n}\mathrm{t}(\mathit{\varphi }{)}^{\left(t\right)},t\leftarrow t+1$;

end while

4. Simulation Study

In this section, we generate simulated data as follows:

$${\mathrm{y}}_{\mathrm{i}}={\mathrm{X}}_i^{\mathrm{T}}\mathsf{\beta }+{\mathsf{\epsilon }}_{\mathrm{i}}, i=1,2,\cdots ,n$$

Let $x_{ij}$ is the jth element of $X_i$ and $x_{ij}$∼(0, 1), ε_i are independently and identically distributed. We consider the following distributions for ε_i: (1) normal distribution ε_i∼(0, 1); (2) Cauchy distribution ε_i∼C(0, 1); (3) t-distribution with 3 degrees of freedom ε_i∼t(3). The values of β are set as follows:

Simulation 1: β = (3, 1.5, 0, 0, 2, 0, 0, 0), a sparse model;

Simulation 2: β = (5, 5, 5, 5, 5, 5, 5, 0), a dense model;

Simulation 3: β = (5, 0, 0, 0, 0, 0, 0, 0), an ultra-sparse model.

The missing data mechanism M₀ ${\phi }_{01}={\phi }_{11}={\phi }_{21}=\cdots ={\phi }_{28}=0.1$

$$\mathrm{l}\mathrm{o}\mathrm{g}\mathrm{i}\mathrm{t}\left(p\left(r_i\mid y_i,X_i,\phi \right)\right)=0.1+0.1y_i+0.1x_{i1}+\cdots +0.1x_{i8},$$

We generate n = 50 data points. To reduce the influence of priors on the results, the hyperparameters in the priors are set as $a_0$ = $b_0$ = 0.01; $a_{{\sigma }_0}$= $b_{{\sigma }_0}$= 0.01; $c_1$ = $d_1$ = 0.01; $c_0$ = $d_0$ = 0.01; μ_φ0 is a vector consisting of the means of non-missing data, and Σφ₀ is a diagonal matrix of the variances of non-missing data. We use the mean squared error (MSE) of β_j estimates and running time (in seconds, T) to measure the estimation accuracy and computational efficiency of each method. The running time is obtained using the tic and toc functions in the “tictoc” package in R (unit: seconds). For each setting, we conduct 100 simulations. The tables report the average results of 100 simulations, with standard deviations in parentheses. The quantiles considered are τ = 0.2, 0.5, 0.8. In R (version 4.3.1), there are no publicly available software packages for methods related to non-ignorable missing response variables. Therefore, we only present the parameter estimation and variable selection results of the algorithm proposed in this chapter.

Table 1. Comparison of parameter estimation and running time for 3 simulations when ε∼N(0, 1).

Simulation	τ	$\widehat{\mathit{\beta }}$1	$\widehat{\mathit{\beta }}$2	$\widehat{\mathit{\beta }}$3	$\widehat{\mathit{\beta }}$4	$\widehat{\mathit{\beta }}$5	$\widehat{\mathit{\beta }}$6	$\widehat{\mathit{\beta }}$7	$\widehat{\mathit{\beta }}$8	T
Simulation 1	β	3.000	1.500	0.000	0.000	2.000	0.000	0.000	0.000
	τ = 0.2	3.492	1.036	0.153	0.159	2.374	−0.460	−0.246	−0.374	0.041
		(0.242)	(0.214)	(0.023)	(0.025)	(0.140)	(0.211)	(0.060)	(0.139)
	τ = 0.5	2.986	1.478	−0.008	0.002	2.058	0.047	−0.028	0.042	0.039
		(0.059)	(0.022)	(0.034)	(0.051)	(0.033)	(0.071)	(0.045)	(0.027)
	τ = 0.8	2.696	1.294	−0.004	0.308	1.735	−0.075	−0.174	−0.068	0.040
		(0.517)	(0.728)	(0.611)	(0.615)	(0.429)	(0.390)	(0.381)	(0.533)
Simulation 2	β	5.000	5.000	5.000	5.000	5.000	5.000	5.000	0.000
	τ = 0.2	5.015	4.808	5.044	4.985	5.005	4.904	4.977	−0.001	0.012
		(0.466)	(0.564)	(0.238)	(0.317)	(0.384)	(0.502)	(0.433)	(0.205)
	τ = 0.5	4.978	4.947	4.995	4.956	4.993	4.979	5.013	0.006	0.011
		(0.019)	(0.022)	(0.019)	(0.029)	(0.019)	(0.024)	(0.033)	(0.027)
	τ = 0.8	5.046	4.864	4.899	4.774	5.002	4.971	4.874	0.107	0.012
		(0.503)	(0.389)	(0.325)	(0.346)	(0.360)	(0.364)	(0.428)	(0.285)
Simulation 3	β	5.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
	τ = 0.2	5.312	−0.169	−0.045	−0.144	0.445	0.030	−0.215	0.073	0.019
		(0.526)	(0.410)	(0.413)	(0.285)	(0.437)	(0.109)	(0.320)	(0.209)
	τ = 0.5	4.993	0.009	−0.054	−0.015	−0.115	−0.063	−0.008	−0.019	0.012
		(0.010)	(0.008)	(0.013)	(0.023)	(0.016)	(0.019)	(0.022)	(0.010)
	τ = 0.8	5.025	−0.242	0.080	−0.038	0.042	−0.058	−0.109	−0.241	0.019
		(0.326)	(0.582)	(0.142)	(0.324)	(0.153)	(0.430)	(0.611)	(0.510)

,

Table 2. Comparison of parameter estimation and running time for 3 simulations when ε∼C(0, 1).

Simulation	τ	$\widehat{\mathit{\beta }}$1	$\widehat{\mathit{\beta }}$2	$\widehat{\mathit{\beta }}$3	$\widehat{\mathit{\beta }}$4	$\widehat{\mathit{\beta }}$5	$\widehat{\mathit{\beta }}$6	$\widehat{\mathit{\beta }}$7	$\widehat{\mathit{\beta }}$8	T
Simulation 1	β	3.000	1.500	0.000	0.000	2.000	0.000	0.000	0.000
	τ = 0.2	2.890	1.360	0.186	−0.010	2.157	−0.029	−0.255	0.178	0.024
		(0.621)	(0.210)	(0.343)	(0.267)	(0.137)	(0.199)	(0.170)	(0.219)
	τ = 0.5	3.032	1.591	−0.224	−0.071	2.255	−0.304	0.284	−0.246	0.029
		(0.291)	(0.317)	(0.131)	(0.299)	(0.137)	(0.129)	(0.331)	(0.229)
	τ = 0.8	3.037	1.872	−0.404	−0.182	2.122	0.242	−0.194	0.099	0.023
		(0.259)	(0.262)	(0.433)	(0.425)	(0.137)	(0.129)	(0.323)	(0.219)
Simulation 2	β	5.000	5.000	5.000	5.000	5.000	5.000	5.000	0.000
	τ = 0.2	4.752	4.825	4.778	5.298	4.587	4.818	4.742	0.321	0.016
		(0.225)	(0.310)	(0.399)	(0.255)	(0.324)	(0.119)	(0.390)	(0.219)
	τ = 0.5	4.645	4.769	4.544	4.642	4.774	5.281	4.908	4.945	0.018
		(0.276)	(0.218)	(0.233)	(0.295)	(0.247)	(0.112)	(0.184)	(0.105)
	τ = 0.8	5.182	5.543	4.750	5.552	4.931	5.339	4.718	0.280	0.018
		(0.326)	(0.433)	(0.447)	(0.215)	(0.337)	(0.179)	(0.290)	(0.219)
Simulation 3	β	5.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
	τ = 0.2	4.970	0.018	−0.243	−0.154	0.230	0.244	−0.238	0.251	0.021
		(0.373)	(0.220)	(0.383)	(0.275)	(0.267)	(0.119)	(0.391)	(0.119)
	τ = 0.5	5.411	−0.117	0.185	0.238	0.355	0.163	0.193	201	0.020
		(0.284)	(0.310)	(0.217)	(0.195)	(0.197)	(0.199)	(0.110)	(0.199)
	τ = 0.8	4.879	0.119	0.295	0.423	0.148	0.253	0.165	0.114	0.026
		(0.371)	(0.388)	(0.313)	(0.385)	(0.237)	(0.169)	(0.333)	(0.288)

and

Table 3. Comparison of MSE and running time for 3 simulations when ε∼t(3).

Simulation	τ	$\widehat{\mathit{\beta }}$1	$\widehat{\mathit{\beta }}$2	$\widehat{\mathit{\beta }}$3	$\widehat{\mathit{\beta }}$4	$\widehat{\mathit{\beta }}$5	$\widehat{\mathit{\beta }}$6	$\widehat{\mathit{\beta }}$7	$\widehat{\mathit{\beta }}$8	T
Simulation 1	β	3.000	1.500	0.000	0.000	2.000	0.000	0.000	0.000
	τ = 0.2	2.923	1.449	0.145	0.210	1.762	0.324	0.181	0.149	0.037
		(0.164)	(0.120)	(0.213)	(0.185)	(0.125)	(0.129)	(0.410)	(0.189)
	τ = 0.5	3.018	1.409	0.012	−0.222	2.020	−0.036	0.056	0.161	0.036
		(0.186)	(0.220)	(0.198)	(0.117)	(0.237)	(0.119)	(0.298)	(0.119)
	τ = 0.8	3.245	1.418	0.129	−0.176	2.195	0281	−0.124	0.115	0.022
		(0.121)	(0.210)	(0.213)	(0.285)	(0.237)	(0.109)	(0.320)	(0.209)
Simulation 2	β	5.000	5.000	5.000	5.000	5.000	5.000	5.000	0.000
	τ = 0.2	4.850	5.362	4.987	4.863	5.081	5.079	4.552	0.171	0.007
		(0.123)	(0.245)	(0.265)	(0.134)	(0.239)	(0.221)	(0.179)	(0.238)
	τ = 0.5	4.997	4.857	4.952	4.930	5.007	5.028	5.028	−0.074	0.011
		(0.298)	(0.210)	(0.256)	(0.235)	(0.277)	(0.219)	(0.335)	(0.349)
	τ = 0.8	4.812	4.977	4.577	4.737	4.793	5.020	4.746	0.210	0.012
		(0.253)	(0.203)	(0.323)	(0.335)	(0.177)	(0.187)	(0.289)	(0.219)
Simulation 3	β	5.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
	τ = 0.2	5.151	−0.127	−0.029	0.022	0.039	−0.129	0.014	−0.134	0.016
		(0.253)	(0.137)	(0.128)	(0.134)	(0.267)	(0.214)	(0.197)	(0.298)
	τ = 0.5	5.171	0.029	−0.086	0.084	−0.097	−0.201	−0.474	0.277	0.012
		(0.203)	(0.110)	(0.278)	(0.235)	(0.117)	(0.279)	(0.220)	(0.289)
	τ = 0.8	5.256	0.306	−0.200	−0.012	−0.005	0.254	−0.288	0.136	0.017
		(0.184)	(0.212)	(0.196)	(0.187)	(0.117)	(0.109)	(0.120)	(0.109)

show the average results of β estimates using the algorithm proposed in this chapter for Simulations 1, 2, and 3, respectively, when ε follows a normal distribution, Cauchy distribution, and t-distribution. The values in parentheses are the average MSE of β over 100 simulations. The results indicate that our algorithm can provide good estimates of β. When ε follows a Cauchy distribution, the estimation error of β is slightly larger, but the difference is not significant. This is consistent with the characteristics of the Cauchy distribution for ε.

In the algorithm proposed in this chapter, the values of ϕ = (ϕ₁, ⋯, ϕ_p) determine the variable selection. When ϕ_j ≥ 1/2, the j-th variable is selected; otherwise, the j-th variable is not selected.

Table 4. Variable selection results for 3 simulations when ε∼N(0, 1).

Simulation	τ	β1	β2	β3	β4	β5	β6	β7	β8
Simulation 1	β	3.000	1.500	0.000	0.000	2.000	0.000	0.000	0.000
	τ = 0.2	1	1	0	0	1	0	0	0
	τ = 0.5	1	1	0	0	1	0	0	0
	τ = 0.8	1	1	0	0	1	0	0	0
Simulation 2	β	5.000	5.000	5.000	5.000	5.000	5.000	5.000	0.000
	τ = 0.2	1	1	1	1	1	1	1	0
	τ = 0.5	1	1	1	1	1	1	1	0
	τ = 0.8	1	1	1	1	1	1	1	0
Simulation 3	β	5.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
	τ = 0.2	1	0	0	0	0	0	0	0
	τ = 0.5	1	0	0	0	0	0	0	0
	τ = 0.8	1	0	0	0	0	0	0	0

,

Table 5. Variable selection results for 3 simulations when ε∼C(0, 1).

Simulation	τ	β1	β2	β3	β4	β5	β6	β7	β8
Simulation 1	β	3.000	1.500	0.000	0.000	2.000	0.000	0.000	0.000
	τ = 0.2	1	1	0	0	1	0	0	0
	τ = 0.5	1	1	0	0	1	0	0	0
	τ = 0.8	1	1	0	0	1	0	0	0
Simulation 2	β	5.000	5.000	5.000	5.000	5.000	5.000	5.000	0.000
	τ = 0.2	1	1	1	1	1	1	1	0
	τ = 0.5	1	1	1	1	1	1	1	0
	τ = 0.8	1	1	1	1	1	1	1	0
Simulation 3	β	5.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
	τ = 0.2	1	0	0	0	0	0	0	0
	τ = 0.5	1	0	0	0	0	0	0	0
	τ = 0.8	1	0	0	0	0	0	0	0

and

Table 6. Variable selection results for 3 simulations when ε∼t(3).

Simulation	τ	β1	β2	β3	β4	β5	β6	β7	β8
Simulation 1	β	3.000	1.500	0.000	0.000	2.000	0.000	0.000	0.000
	τ = 0.2	1	1	0	0	1	0	0	0
	τ = 0.5	1	1	0	0	1	0	0	0
	τ = 0.8	1	1	0	0	1	0	0	0
Simulation 2	β	5.000	5.000	5.000	5.000	5.000	5.000	5.000	0.000
	τ = 0.2	1	1	1	1	1	1	1	0
	τ = 0.5	1	1	1	1	1	1	1	0
	τ = 0.8	1	1	1	1	1	1	1	0
Simulation 3	β	5.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000
	τ = 0.2	1	0	0	0	0	0	0	0
	τ = 0.5	1	0	0	0	0	0	0	0
	τ = 0.8	1	0	0	0	0	0	0	0

show the variable selection results when ε follows a normal distribution, Cauchy distribution, and t-distribution, respectively. A value of 1 indicates that the variable is selected, while 0 indicates that it is not selected. It can be observed that all important covariates are identified.

We stop the proposed algorithm when the entropy of ϕ no longer changes (or changes very little, less than a given threshold). Figure 2, Figure 3 and Figure 4 show the entropy calculation results of ϕ when ε follows a normal distribution, Cauchy distribution, and t-distribution, respectively. It can be seen that, in all simulations, our algorithm converges within 100 iterations, which is far fewer than the number of samples in sampling-based MCMC algorithms. Therefore, the algorithm we propose has high computational efficiency.

Variational Bayesian (VB) inference is often considered superior to MCMC in terms of speed, scalability, and efficiency. Unlike MCMC, which relies on iterative sampling and can be computationally expensive, VB transforms the inference problem into an optimization task, leading to faster and more predictable convergence. VB produces deterministic results, avoiding the Monte Carlo error inherent in MCMC, and is more memory-efficient as it does not require storing large numbers of posterior samples. It is particularly well-suited for large-scale models, where MCMC may struggle due to slow sampling and high computational costs. Additionally, VB tends to offer more interpretable posterior distributions by approximating them with simpler parametric families. However, while VB is faster and more scalable, it may underestimate uncertainty due to its reliance on approximations, whereas MCMC remains more flexible and accurate for complex posterior distributions.

5. Real Data Analysis

In this section, we analyze data from HIV-positive patients in the AIDS Clinical Trials Group (ACTG175) study [16], which can be obtained using the command data (ACTG175) in the R package “BART”. The ACTG175 dataset contains 27 variables, and 2139 HIV-positive patients were randomly assigned to the following four groups: (1) 532 received zidovudine treatment; (2) 522 received didanosine treatment; (3) 524 received a combination of zidovudine and didanosine; and (4) 561 received a combination of zidovudine and zalcitabine. These patients were monitored at weeks 2, 4, and 8 after the start of the experiment and then every 12 weeks thereafter. Monitoring ended when the CD4 T-cell count declined by 50% or more, or the patient died.

We are interested in the relationship between the dependent variable Y, the CD4 T-cell count at 96 ± 5 weeks, and the covariates age (X₁), weight (X₂), baseline CD4 T-cell count (X₃), CD4 T-cell count at 20 ± 5 weeks (X₄), baseline CD8 T-cell count (X₅), and CD8 T-cell count at 20 ± 5 weeks (X₆). Due to patient death or dropout, the dependent variable Y has missing records, with missing rates of 39.66%, 36.21%, 35.69%, and 37.43% for the four groups, respectively. Relevant medical research indicates that CD4 T-cell count is related to disease progression, and patients with lower CD4 T-cell counts are more likely to drop out of the study. This suggests that the missingness of Y (CD4 T-cell count at 96 ± 5 weeks) is related to the CD4 T-cell count. In summary, the missingness of Y is not random, and we can establish the following model:

Q_τ (y_i ∣ x_i) = β₁x_i1 + ⋯ + β₆x_i6, i = 1, ⋯, n.

$$p\left(r_i=1\mid y_i,x_i,\mathit{\phi }\right)={\displaystyle \frac{e^{{\phi }_0+{\phi }_1{y}_i+{\phi }_{21}{x}_{i1}+\cdots +{\phi }_{26}{x}_{i6}}}{1+e^{{\phi }_0+{\phi }_1{y}_i+{\phi }_{21}{x}_{i1}+\cdots +{\phi }_{26}{x}_{i6}}}}$$

Table 7. Results of the AIDS data analysis.

		β1	β2	β3	β4	β5	β6
(1)	QR	0.050	−0.028	0.128	0.531	−0.008	−0.107
	Est	0.052	−0.030	0.134	0.550	−0.010	−0.111
	CI	(−0.060, 0.134)	(−0.158, 0.045)	(−0.040, 0.204)	(0.453, 0.612)	(−0.135, 0.090)	(−0.195, 0.019)
(2)	QR	0.048	−0.035	0.125	0.652	0.122	−0.166
	Est	0.055	−0.031	0.129	0.620	0.125	−0.169
	CI	(−0.041, 0.141)	(−0.105, 0.043)	(−0.002, 0.200)	(0.519, 0.650)	(0.005, 0.274)	(−0.301, −0.139)
(3)	QR	0.040	0.081	0.201	0.513	0.060	−0.211
	Est	0.036	0.079	0.196	0.507	0.062	−0.209
	CI	(−0.054, 0.121)	(−0.003, 0.159)	(0.112, 0.309)	(0.398, 0.614)	(−0.070, 0.227)	(−0.337, −0.098)
(4)	QR	0.059	0.064	0.271	0.515	−0.095	−0.063
	Est	0.065	0.062	0.262	0.511	−0.098	−0.061
	CI	(−0.030, 0.149)	(−0.038, 0.152)	(0.131, 0.465)	(0.398, 0.615)	(−0.188, 0.037)	(−0.187, 0.069)

summarizes the coefficients and 95% confidence intervals for the quantile regression of the four treatment groups when = 0.5. The QR row shows the results of quantile regression estimation when there is no missing data. It can be seen that the proposed method effectively imputes missing data. Figure 5 shows that our algorithm achieves convergence in the analysis of the AIDS data. From the estimated coefficients of the quantile regression, age and weight factors do not significantly influence the observed CD4 T-cell count at 96 ± 5 weeks. The observed value at 20 ± 5 weeks has a significant positive effect on the CD4 T-cell count at 96 ± 5 weeks and is the main influencing factor for the 96 ± 5-week measurement. The influence of the baseline measurement at a longer interval on the 96 ± 5-week measurement is relatively small and not even significant in groups (1) and (2) (corresponding to variable selection with $\varphi <1/2$). Interestingly, the CD8 T-cell count at 20 ± 5 weeks has a significant negative effect in groups (2) and (3), indicating that higher previous CD4 levels and lower CD8 levels contribute to an increase in CD4 T-cell count. This finding is similar to the use of the CD4/CD8 ratio as an indicator of antiretroviral therapy efficacy in existing studies, suggesting that a higher baseline ratio favors CD4 T-cell count recovery and immune function reconstruction. The results of this study also suggest that, in the mid-term assessment of HIV infection treatment efficacy, attention should be paid not only to CD4 T-cell levels but also to CD8 T-cell levels. With unchanged baseline CD4 levels, a low CD4/CD8 ratio may negatively impact long-term treatment outcomes.

6. Conclusions

In this paper, we consider a Bayesian quantile regression model with non-ignorable missing response data, where the non-ignorable missing mechanism is specified by a logistic function. The model parameters of the quantile regression are assigned a spike-and-slab LassoLASSO prior, which enables effective parameter estimation and variable selection without requiring further processing of the parameter posteriors. Bayesian variable selection based on the spike-and-slab prior is computationally expensive, and variational Bayes is a popular Bayesian computational method. Since the logistic function does not have a conjugate prior, we introduce a new Pólya-Gamma variable in the logistic function, making the parameter posteriors of the logistic function conjugate. Finally, we propose a variational algorithm for Bayesian quantile regression with non-ignorable missing response data, which performs well in both simulation studies and real data analysis.