Prioritizing Longitudinal Gene–Environment Interactions Using an FDR-Assisted Robust Bayesian Linear Mixed Model

Abstract

Analysis of longitudinal data in high-dimensional gene–environment interaction studies have been extensively conducted using variable selection methods. Despite their success, these studies have been consistently challenged by the lack of uncertainty quantification procedures to identify main and interaction effects under longitudinal phenotypes that follow heavy-tailed distributions due to disease heterogeneity. In this article, to improve statistical rigor of variable selection-based G × E analysis, we propose to apply the robust Bayesian linear mixed-effect model with a false discovery rate (FDR) control procedure to tackle these challenges. The Bayesian mixed model adopts a robust likelihood function to account for skewness in longitudinal phenotypic measurements, and it imposes spike-and-slab priors to detect important main and interaction effects. Leveraging the parallelism between spike-and-slab priors and the Bayesian approach to hypothesis testing, we perform variable selection and uncertainty quantification through a Bayesian false discovery rate (FDR)-assisted procedure. Numerical analyses have demonstrated the advantage of our proposal over alternative approaches. A case study of a longitudinal cancer prevention study with high-dimensional lipid measures yields main and interaction effects with important biological implications.

1. Introduction

Gene–environment interactions provide additional insights in explaining variations in complex diseases traits beyond main genetic and environmental effects [1]. With large-scale omics features, the dissection of gene–environment interactions becomes a high-dimensional data analysis problem, which can generally be addressed through two frameworks: hypothesis testing and variable selection [2,3,4]. Hypothesis testing–based methods are usually of marginal nature by analyzing one or a small number of markers at a time through statistical tests, with p-values being generated for each test to perform statistical inference. Multiple testing procedures then follow to detect significant interactions. Due to the computational advantage to conduct each test and inferential power to identify significant interactions, this branch of methods dominates in genome-wide association studies (or GWAS) [5,6], including biobank-scale studies [7,8,9,10]. On the other hand, variable selection methods, especially those based on regularization, have achieved success in addressing challenging issues that are not widely considered in testing based studies, including accommodating structured sparsity, such as the hierarchy between main and interaction effects, and accounting for robustness of the G × E analysis [4].

Due to the heterogeneity of complex diseases, disease phenotypes generally follow heavy-tailed distributions with outliers. Robust penalization methods have therefore been developed to safeguard against outliers to improve accuracy in identifying important omics features in bioinformatics studies [11]. From a modeling perspective, robust regularized variable selection is of the form of “robust unpenalized loss function + penalty function”, where the robustness comes from loss functions that can downweigh the influence from outlying observations. Typical choices of robust loss functions include Huber loss [12], quantile check loss [13], rank-based loss [14], among many others [11]. Subsequently, corresponding robust penalization methods have been widely developed [15,16,17].

Addressing robustness is also a critical issue in gene–environment interaction studies, which usually involve analyzing complex disease traits [4]. In the literature, robust penalization methods have been extensively developed for interaction studies [18,19,20,21], where robust loss functions are adopted to accommodate data heterogeneity with penalty functions tailored for interaction analysis being developed. Although these studies have yielded gene–environment interactions with meaningful biological implications, there is a lack of statistical inference procedures that can lead to uncertainty quantification measures. This is partly attributable to the difficulty of developing inference measures with robust loss functions, which are often non-differentiable and violate the standard assumptions required for conventional inference procedures under non-robust sparse models with Gaussian model errors [22,23,24,25]. From both frequentist and Bayesian perspectives, some recent studies have been proposed to provide uncertainty quantification measures for G × E analysis [26,27,28], which is parallel to the trend of improving statistical rigor of sparse models via developing high-dimensional inference procedures [22,23,24]. In particular, robust fully Bayesian analysis has been considered since robust Bayesian inference utilizing posterior samples drawn from MCMC can be readily performed [29,30,31].

In longitudinal gene–environment interaction studies where disease phenotypes are repeatedly measured across a sequence of time points, overcoming the aforementioned challenge in statistical inference is further complicated by the defining characteristics of longitudinal data, whereby repeated measures are correlated and demonstrate subject–specific clustering patterns [32]. In the literature, generalized estimating equation (or GEE) is one of the major frameworks for longitudinal data analysis [33], where extensions to high-dimensional settings have been widely considered. For example, Wang et al. (2012) [34] have developed penalized GEE with valid inference through confidence intervals that can be validated on finite samples. However, vulnerability to outliers is a well-known disadvantage of GEE [35]. Consequently, the main effect model-based penalized GEE and inference procedures cannot be directly applied when robustness in interaction studies is a concern. Moreover, addressing data heterogeneity through standard transformation procedures, such as the Box–Cox transformation, is not very effective for longitudinal data. Applying transformations to repeated measures distorts the correlation structure among the original longitudinal observations. Therefore, performing a data transformation step prior to using non-robust GEE-based methods is generally not preferable in longitudinal studies. In spite of attempts to improve robustness through quadratic inference functions [36] in tailored G × E analysis [37,38], uncertainty quantification measures are not available with variable selection procedures, and robust analysis methods with inferential guarantees are still in pressing need for longitudinal studies.

In addition to GEE, the mixed-effect model is another major framework to analyze longitudinal data [35,39]. Since robustifying fully Bayesian analysis in cross-sectional studies has produced valid inference procedures in the presence of data heterogeneity [25], in longitudinal studies, we leverage robust, fully Bayesian methods for gene–environment interaction analyses. Specifically, we are interested in adopting a robust Bayesian linear mixed-effect model that captures inter-connections among repeated measures through incorporating the random effects [35,40], and models G × E interactions through fixed effects. Our primary interest in this study is prioritizing, or ranking, important main omics effects and G × E interactions. Consequently, although a plethora of shrinkage priors exist [41], we focus on two-component spike-and-slab priors, which account for large effects through slab components in terms of continuous densities such as the normal or Laplace distributions, while shrinking regression coefficients corresponding to small effects to zero via a spike component as a point mass at zero [42,43]. The two-group nature of spike-and-slab priors naturally results in a posterior inclusion probability (PIP) indicating the proportion of regression coefficients representing main or interaction effects being sampled from the slab component. Therefore, a larger PIP suggests a stronger evidence supporting the significance of the effects and can be used as a ranking tool to prioritize the importance of detected effects in longitudinal interaction studies.

In a cross-sectional G × E analysis of type 2 diabetes data with high-dimensional single nucleotide polymorphism (SNP) features, Lu et al. (2021) [29] have developed a robust marginal analysis method to generate a ranked list of G × E interactions using PIP. In frequentist studies, marginal penalization methods that select important markers based on non-zero shrinkage estimates have also been considered [44]. Compared to the frequentist regularization method, Lu et al. (2021) [29] offers the advantages of robustness and uncertainty quantification through PIPs. Although the results are promising, a caveat is the lack of a valid rule for determining cut-offs to claim important identification effects.

With spike-and-slab priors, median probabilities have been shown to be effective for the purpose of variable selection with theoretical and empirical reasons [25,45,46]. As our primary goal is to produce a ranked list of appropriate length, it may be reasonable to adopt an adaptive threshold. In this longitudinal gene–environment interaction studies, we consider calculating the threshold for labeling significant main and interaction effects utilizing Bayesian false discovery rates (FDRs) [47,48]. Suppose that ${\mathrm{PIP}}_j$ is the PIP included as a slab component for the regression coefficient corresponding to the jth main or interaction effect, then $(1-{\mathrm{PIP}}_j)$ indicates the probability of being a false positive if the jth effect is claimed as an important one but is actually not associated with disease traits. Therefore, $(1-{\mathrm{PIP}}_j)$ can be interpreted as estimates for Bayesian q–values or local FDRs [49,50,51]. Morris et al. (2008) [48] and Zhang et al. (2014) [47], among others, have provided procedures to determine the cut-off adaptively with the FDR control.

Combined, our proposal consists of the following two steps. First, we identify longitudinal gene–environment interactions and corresponding PIPs with the robust Bayesian linear mixed model that accommodates trait heterogeneity, intra-clustering of repeated measures, and high-dimensionality of longitudinal omics data. Then, the Bayesian FDR procedure described above is applied to adaptively choose a threshold to prioritize main and interaction effects as well as claim significance of findings. It is worth noting that Lu et al. (2021) [29] have used ROC curves and the top 100 ranked effects to summarize variable selection results. While informative, ROC curves are only feasible in simulations where the ground truth is available, and the choice of the top 100 effects is somewhat arbitrary. In comparison, the adaptive threshold determined from our study is well-established according to the published literature. Extensive numerical experiments are performed to demonstrate the utility of the proposed methods. In addition, we analyze longitudinal data with high-dimensional lipidomics features and repeatedly measured the body weight of CD-1 mice that can be retrieved from a cancer prevention study [52].

2. Materials and Methods

2.1. The Robust Linear Mixed Model

In a longitudinal gene–environment interaction study with n subjects and k repeated measures per subject, we denote $Y_{ij}$ as the phenotype of interest for the ith subject measured at the jth time point ($1\le i\le n$, $1\le j\le k$). At time point j, let ${\mathbf{G}}_{ij}={(G_{ij1},\dots ,G_{ijp})}^{\top }$ be the p-dimensional vector for genetic factors and ${\mathbf{E}}_{ij}={(E_{ij1},\dots ,E_{ijq})}^{\top }$ be the q-dimensional vector for environmental factors. Then, we consider the following robust longitudinal gene-environment interaction mixed-effect model:

$$Y_{ij}={\mathbf{T}}_{ij}^{\top }{\beta }_0+{\mathbf{G}}_{ij}^{\top }{\beta }_1+{\mathbf{E}}_{ij}^{\top }{\beta }_2+{({\mathbf{G}}_{ij}\otimes {\mathbf{E}}_{ij})}^{\top }{\beta }_3+{\mathit{U}}_{ij}^{\top }{\mathit{\alpha }}_i+{\epsilon }_{ij},$$

(1)

where the fixed effect includes a 3 by 1 vector ${\mathit{\beta }}_0$ that is associated with baseline time effects ${\mathbf{T}}_{ij}={(1,j,j^2)}^{\top }$, while ${\beta }_1$, ${\beta }_2$, and ${\beta }_3$, which are the p-, q-, and $pq$-dimensional regression coefficient vectors, represent the genetic and environmental main effects as well as their interactions, respectively. Specifically, the longitudinal gene–environment interactions can be expressed as a Kronecker product in terms of an $pq$-dimensional vector:

$$G_{ij}\otimes E_{ij}={\left[G_{ij1}{E}_{ij1},G_{ij1}{E}_{ij2},\dots ,G_{ij1}{E}_{ijq},G_{ij2}{E}_{ij1},\dots ,G_{ijp}{E}_{ijq}\right]}^{\top }.$$

The mixed model (1) accommodates correlations among repeated measures through random effects ${\mathit{\alpha }}_i$. Under the random intercept–slope model, ${\mathit{\alpha }}_i$ is a 2 by 1 vector associated with ${\mathbf{U}}_{ij}={(1,j)}^{\top }$, where j is the time point ($1\le j\le k$). ${\mathbf{U}}_{ij}^{\top }{\mathit{\alpha }}_i$ leads to a scalar in model (1). Under the random intercept model, ${\mathbf{U}}_{ij}=1$, and ${\mathit{\alpha }}_i$ reduces to a scalar accordingly. The model errors ${ϵ}_{ij}$ are assumed to be independent and follow heavy-tailed distributions.

Under standard, low-dimensional mixed-effects models, model selection between a random-intercept model and a random intercept-and-slope model can be performed using information criteria such as AIC or BIC, or by comparing marginal likelihoods [35]. However, in high-dimensional settings, these model selection criteria are confounded by the subset of selected features, and the issue therefore remains an open problem. We adopt the random intercept-and-slope model throughout this study and justify our choice based on the case study.

Model (1) differs from published works such as [37,53] in that it does not arise from a repeated-measure one-way ANOVA design. Therefore, unlike the group-based dummy variable structure used for environmental factors in [53], the ${\mathbf{E}}_{ij}$ terms in model (1) do not have such an implication, as they are not the result of dummy coding for categorical predictors.

2.2. A Bayesian Formulation of the Robust Linear Mixed Model

To ensure the robustness of model (1), we assume that model errors ${ϵ}_{ij}$ follow identical and independent Laplace distributions. Specifically,

$$f({ϵ}_{ij}\mid \tau )={\displaystyle \frac{\tau }{4}}exp\left\{-{\displaystyle \frac{\tau }{2}}\left|{ϵ}_{ij}\right|\right\},i=1,\dots ,n,j=1,\dots ,k,$$

where the scale and location parameters of the Laplace distribution are ${\displaystyle \frac{2}{\tau }}$ and 0, respectively. The L1 norm in the exponent of the kernel, $|{ϵ}_{ij}|$, is critical for a robust linear mixed model since this formulation downweights the importance of outliers and heavy-tailed model errors. Given the fixed and random effects in model (1), we can obtain the conditional distribution of $Y_{ij}$ as

$$f(Y_{ij}\mid {\mu }_{ij},{\alpha }_i,U_{ij},\tau )={\displaystyle \frac{\tau }{4}}exp\left(-{\displaystyle \frac{\tau }{2}}\left|Y_{ij}-{\mu }_{ij}-U_{ij}^{\top }{\alpha }_i\right|\right).$$

where the mean function ${\mu }_{ij}$ is defined as ${\mu }_{ij}={\mathbf{T}}_{ij}^{\top }{\beta }_0+{\mathbf{G}}_{ij}^{\top }{\beta }_1+{\mathbf{E}}_{ij}^{\top }{\beta }_2+{({\mathbf{G}}_{ij}\otimes {\mathbf{E}}_{ij})}^{\top }{\beta }_3.$ As Laplace model errors can be expressed as ${ϵ}_{ij}={\tau }^{-1}\xi \sqrt{v_{ij}}{z}_{ij}$, where $v_{ij}\sim \mathrm{Exp}\left(1\right)$, $z_{ij}\sim \mathcal{N}(0,1)$, and $\xi =\sqrt{8}$ [54], we formulate the linear mixed model (1) utilizing the following hierarchical structure:

$$Y_{ij}={\mu }_{ij}+U_{ij}^{\top }{\alpha }_i+\xi {\tau }^{-1/2}\sqrt{{\tilde{v}}_{ij}}{z}_{ij},$$

$${\tilde{v}}_{ij}\mid \tau \stackrel{\mathrm{ind}}{\sim }\tau exp(-\tau {\tilde{v}}_{ij}),z_{ij}\stackrel{\mathrm{ind}}{\sim }\mathcal{N}(0,1),i=1,\dots ,n\mathrm{and}j=1,\dots ,k.$$

2.3. Robust Sparse Bayesian Linear Mixed Model

Under the above mixed model for longitudinal gene–environment interactions, the high dimensionality of genetic factors ${\mathbf{G}}_{ij}$ necessitates variable selection to identify main and interaction effects that are associated with disease phenotypes. The low-dimensional environmental factors ${\mathbf{E}}_{ij}$ are not subject to selection, and we only consider selecting genetic factors and gene–environment interactions. Therefore, our proposed model respects weak hierarchical structures between main and interaction effects [4]. From a frequentist variable selection perspective, regularized loss function for the above gene–environment interaction problem can be expressed as

$$\sum _{i=1}^n\sum _{j=1}^k|Y_{ij}-{\mu }_{ij}-U_{ij}^{\top }{\alpha }_i|+{\lambda }_1\sum _{m=1}^p\left|{\beta }_{1m}\right|+{\lambda }_2\sum _{s=1}^{pq}\left|{\beta }_{3s}\right|,$$

where ${\lambda }_1,{\lambda }_2>0$ are tuning parameters for penalized selection on the main and interaction effects. To enable uncertainty quantification in the presence of heavy-tailed model errors, we consider an Bayesian approach and imposing appropriate shrinkage priors on p- and $pq$-dimensional fixed effects ${\mathit{\beta }}_1$ and ${\mathit{\beta }}_3$, respectively. While variable selection is the focus here, it is also acknowledged that statistical testing based approaches can be developed within mixed effects as well for association analysis [55,56].

We show brief prior elicitation procedures as follows. For main effects ${\beta }_{1m}$$(m=1,\dots ,p)$, we first consider assigning Laplacian shrinkage priors such that $\pi \left({\beta }_{1m}\right|\tau ,{\lambda }_1)=(\tau {\lambda }_1/2)\exp \{-\tau {\lambda }_1|{\beta }_{1m}\left|\right\}$. To make the representation more concise, let ${\eta }_1=\tau {\lambda }_1$ so we can assign Laplacian shrinkage priors that can be hierarchically expressed as a scale mixture of normals [57]:

$${\beta }_{1m}\mid r_{1m}\stackrel{\mathrm{ind}}{\sim }N(0,r_{1m}),r_{1m}\stackrel{\mathrm{ind}}{\sim }{\displaystyle \frac{{\eta }_1^2}{2}}exp\left(-{\displaystyle \frac{{\eta }_1^2}{2}}{r}_{1m}\right).$$

Since Laplacian priors do not lead to exact zero in posterior estimates for fixed effect, we further incorporate an extra layer of point mass spike-and-slab priors in the Laplacian hierarchy as

$$\begin{array}{cc}\hfill & {\beta }_{1m}\mid {\varphi }_{1m},r_{1m}\stackrel{\mathrm{ind}}{\sim }{\varphi }_{1m}N(0,r_{1m})+(1-{\varphi }_{1m}){\delta }_0\left({\beta }_{1m}\right),\hfill \\ \hfill & {\varphi }_{1m}\stackrel{\mathrm{ind}}{\sim }\mathrm{Bernoulli}\left({\pi }_1\right),r_{1m}\stackrel{\mathrm{ind}}{\sim }\mathrm{Gamma}\left(1,{\displaystyle \frac{{\eta }_1^2}{2}}\right)\hfill \end{array}$$

(2)

where the spike component ${\delta }_0\left({\beta }_{1m}\right)$ is a point mass at 0, promoting exact sparsity in estimating main effect. On the other hand, the slab component $N(0,r_{1m})$ models large signals. The binary indicator ${\varphi }_{1m}\in [0,1]$ specifies whether the spike or slab component is active. When ${\varphi }_{1m}=1$, the prior in Equation (2) reduces to the univariate Laplace prior, suggesting that the mth genetic variant has a main effect. Otherwise, if ${\varphi }_{1m}=0$, there is no main effect. Similarly, we impose the following priors on ${\beta }_{3s}$$(s=1,\dots ,pq)$, regression coefficients representing interaction effects, as

$$\begin{array}{cc}\hfill & {\beta }_{3s}\mid {\varphi }_{3s},r_{3s}\stackrel{\mathrm{ind}}{\sim }{\varphi }_{3s}N(0,r_{3s})+(1-{\varphi }_{3s}){\delta }_0\left({\beta }_{3s}\right),\hfill \\ \hfill & {\varphi }_{3s}\stackrel{\mathrm{ind}}{\sim }\mathrm{Bernoulli}\left({\pi }_2\right),r_{3s}\stackrel{\mathrm{ind}}{\sim }\mathrm{Gamma}\left(1,{\displaystyle \frac{{\eta }_2^2}{2}}\right)\hfill \end{array}$$

(3)

where the latent mixture proportion ${\pi }_2\in [0,1]$ and the same rationale as in the prior specified on main effect follows.

Hyper priors to retain conjugacy have been assigned as ${\pi }_1\sim \mathrm{Beta}(a_1,b_1)$ and ${\pi }_2\sim \mathrm{Beta}(a_2,b_2)$. In addition, Gamma priors are assumed as ${\eta }_1^2\sim \mathrm{Gamma}(c_1,d_1)$, ${\eta }_2^2\sim \mathrm{Gamma}(c_2,d_2)$, and $\tau \sim \mathrm{Gamma}(w_1,w_2)$; an inverse-Gamma prior is assigned to ${\varphi }^2\sim \mathrm{Inverse}\text{-}\mathrm{Gamma}(e_1,e_2)$. We set the hyperparameters as $a_1=b_1=a_2=b_2=c_1=d_1=c_2=d_2=w_1=w_2=e_1=e_2=1$.

For the random effect ${\mathit{\alpha }}_i$, a multivariate normal prior under the random intercept-and-slope model is adopted: ${\mathit{\alpha }}_i\sim \mathrm{MVN}(0,{\varphi }^2\mathit{I})$.

2.4. Robust Variable Selection with Bayesian False Discovery Rates

With spike-and-slab priors [42,43], Bayesian variable selection can be performed by using all the MCMC samples. Consider the ($p+pq$) main and interaction effects with the hierarchical structures specified in (2) and (3), respectively. Let us assume that G posterior samples have been drawn after excluding burn-ins. At the gth MCMC iteration, the uth main or interaction effect is selected if its binary indicator ${\varphi }_u^{\left(g\right)}=1$, ($1\le u\le p+pq$). Otherwise, it is not selected, i.e., ${\varphi }_u^{\left(g\right)}=0$. Hence, the posterior inclusion probability (PIP) of selecting the uth main or interaction effect across G MCMC iterations is

$$p_u={\displaystyle \frac{1}{G}}\sum _{g=1}^G{\varphi }_{1m}^{\left(g\right)},,1\le u\le p+pq.$$

With a slight abuse of notation, we use ${\beta }_u$ to represent the uth main or interaction effect. Under a two-group spike-and-slab formulation, variable selection can be viewed as Bayesian hypothesis testing with $H_{0u}:{\beta }_u=0$ versus $H_{1u}:{\beta }_u\ne 0$. The complement of the PIP, $1-p_u=Pr(H_{0u}\mid \mathrm{data})$, is the Bayesian local false discovery rate (local FDR) for the uth effect, or the posterior probability that the effect is null. This quantity represents the expected contribution of the uth main or interaction effect to the global false discovery proportion if it is declared significant.

Let $\gamma \in (0,1)$ be a threshold applied to the PIPs. All main and interaction effects satisfying $p_u>\gamma$ are declared as discoveries. For this selected set of effects, the Bayesian FDR is defined as the average of their local FDRs:

$$\mathrm{FDR}\left(\gamma \right)={\displaystyle \frac{{\sum }_{u=1}^{p+pq}(1-p_u)\mathbb{I}(p_u>\gamma )}{{\sum }_{u=1}^{p+pq}\mathbb{I}(p_u>\gamma )}}.$$

(4)

where the denominator is the total number of discovery of significant genetic main effects and G × E interactions, while the numerator sums their probabilities as false discoveries (i.e., posterior null probabilities). Thus, $\mathrm{FDR}\left(\gamma \right)$ is the expected proportion of false discoveries among all declared significant effects. This Bayesian FDR definition has been widely adopted in the published literature [47,48,58].

Instead of using a fixed cut-off such as the median probability threshold $\gamma =0.5$ [25,45], we determine an adaptive threshold that yields a desired global FDR level $c\in (0,1)$. Let the sorted PIPs be $p_{\left[1\right]}\ge p_{\left[2\right]}\ge \cdots \ge p_{[p+pq]}$. These ordered PIPs serve as candidate thresholds. If we select the top u effects, the Bayesian FDR corresponding to the threshold $\gamma =p_{\left[u\right]}$ becomes

$$\mathrm{FDR}\left(p_{\left[u\right]}\right)=1-{\displaystyle \frac{1}{u}}\sum _{i=1}^u{p}_{\left[i\right]}.$$

Because the PIPs are sorted in decreasing order, adding an additional effect (moving from u to $u+1$) reduces average PIP among the selected effects, thereby increasing the expected false discovery proportion. Consequently, $\mathrm{FDR}\left(p_{\left[u\right]}\right)$ increases monotonically with u. The adaptive procedure therefore identifies the largest set of effects for which the expected false discovery proportion does not exceed the desired level. Formally, we choose

$$u^{*}=max\left\{u:\mathrm{FDR}\left(p_{\left[u\right]}\right)\le c\right\},\widehat{\gamma }=p_{\left[u^{*}\right]}.$$

The resulting cut-off $\widehat{\gamma }$ ensures that the expected proportion of false discoveries among the selected main and interaction effects is controlled at level c. In this way, the threshold is determined directly from the posterior evidence and adapts automatically to the sparsity level and signal strength in the data.

2.5. Gibbs Sampling Algorithm

Based on the elicited prior from the section above, we can write the joint density as follows:

$$\begin{array}{cc}\hfill \pi & ({\mathit{\beta }}_0,{\mathit{\beta }}_1,{\mathit{\beta }}_2,{\mathit{\beta }}_3,r_{1m},r_{3s},{\eta }_1^2,{\eta }_2^2,{\pi }_1,{\pi }_2,\tau \mid Y_{ij})\propto {\left({\tau }^{-1}\right)}^{-{\displaystyle \frac{nk}{2}}}{\left(\sum _{i=1}^n\sum _{j=1}^k{\tilde{v}}_{ij}\right)}^{-{\displaystyle \frac{1}{2}}}\hfill \\ \hfill & \times exp\left\{-{\displaystyle \frac{\tau }{2{\xi }^2}}\sum _{i=1}^n\sum _{j=1}^k{\left(Y_{ij}-{\mu }_{ij}-U_{ij}^{\top }{\alpha }_i\right)}^2/{\tilde{v}}_{ij}\right\}\hfill \\ \hfill & \times exp\left(-{\displaystyle \frac{1}{2}}{\mathit{\beta }}_0^{\top }{\mathrm{\Sigma }}_0^{-1}{\mathit{\beta }}_0\right)\times exp\left(-{\displaystyle \frac{1}{2}}{\mathit{\beta }}_2^{\top }{\mathrm{\Sigma }}_1^{-1}{\mathit{\beta }}_2\right)\hfill \\ \hfill & \times \prod _{m=1}^p\left[(1-{\pi }_1){\left(2\pi r_{1m}\right)}^{-1/2}exp\left(-{\displaystyle \frac{1}{2r_{1m}}}{\beta }_{1m}^2\right)\mathbb{I}\{{\beta }_{1m}\ne 0\}+{\pi }_1{\delta }_0\left({\beta }_{1m}\right)\right]\hfill \\ \hfill & \times {\tau }^{nk}exp\left(-\tau \sum _{i=1}^n\sum _{j=1}^k{\tilde{v}}_{ij}\right)\times {\tau }^{w_1-1}exp(-w_2\tau )\hfill \\ \hfill & \times {\left({\eta }_1^2\right)}^{c_1-1}exp(-d_1{\eta }_1^2)\times \prod _{m=1}^p\left[{\displaystyle \frac{{\eta }_1^2}{2}}exp\left(-{\displaystyle \frac{{\eta }_1^2}{2}}{r}_{1m}\right)\right]\hfill \\ \hfill & \times {\pi }_1^{a_1-1}{(1-{\pi }_1)}^{b_1-1}\hfill \\ \hfill & \times \prod _{s=1}^{pq}\left[(1-{\pi }_2){\left(2\pi r_{3s}\right)}^{-1/2}exp\left(-{\displaystyle \frac{1}{2r_{3s}}}{\beta }_{3s}^2\right)\mathbb{I}\{{\beta }_{3s}\ne 0\}+{\pi }_2{\delta }_0\left({\beta }_{3s}\right)\right]\hfill \\ \hfill & \times {\left({\eta }_2^2\right)}^{c_2-1}exp(-d_2{\eta }_2^2)\times \prod _{s=1}^{pq}\left[{\displaystyle \frac{{\eta }_2^2}{2}}exp\left(-{\displaystyle \frac{{\eta }_2^2}{2}}{r}_{3s}\right)\right]\times {\pi }_2^{a_2-1}{(1-{\pi }_2)}^{b_2-1}\hfill \\ \hfill & \times {\left({\varphi }^2\right)}^{-nh/2}\times \prod _{i=1}^n\prod _{h=1}^{2}exp\left(-{\displaystyle \frac{1}{2{\varphi }^2}}{\alpha }_{ih}^2\right)\times {\left({\varphi }^2\right)}^{e_1-1}exp(-e_2{\varphi }^2)\hfill \end{array}$$

(5)

We obtain the full conditional distribution as follows:

• Define ${\mu }_{ij}^{\left(0\right)}={\mu }_{ij}+{\mathit{U}}_{\mathit{ij}}^{\top }{\mathit{\alpha }}_{\mathit{i}}-{\mathbf{T}}_{\mathit{ij}}^{\top }{\beta }_{\mathbf{0}}$. Then, the full conditional distribution of ${\mathit{\beta }}_{\mathbf{0}}$ follows $\mathrm{MVN}({\mathit{\mu }}_{{\mathit{\beta }}_{\mathbf{0}}},{\mathrm{\Sigma }}_{{\mathit{\beta }}_{\mathbf{0}}})$ with mean

  $${\mathit{\mu }}_{{\mathit{\beta }}_{\mathbf{0}}}={\mathrm{\Sigma }}_{{\mathit{\beta }}_{\mathbf{0}}}\tau {\xi }^{-2}\sum _{i=1}^n\sum _{j=1}^k{\tilde{v}}_{ij}^{-1}{\mathbf{T}}_{\mathit{ij}}(Y_{ij}-{\mu }_{ij}^{\left(0\right)}),$$

  and variance

  $${\mathrm{\Sigma }}_{{\mathit{\beta }}_{\mathbf{0}}}={\left({\mathrm{\Sigma }}_{\mathbf{0}}^{-1}+\tau {\xi }^{-2}\sum _{i=1}^n\sum _{j=1}^k{\tilde{v}}_{ij}^{-1}{\mathbf{T}}_{\mathit{ij}}{\mathbf{T}}_{\mathit{ij}}^{\top }\right)}^{-1}.$$
• Denote the partial residual after excluding environmental main effect as ${\mu }_{ij}^{\left(2\right)}={\mu }_{ij}+{\mathit{U}}_{\mathit{ij}}^{\top }{\mathit{\alpha }}_{\mathit{i}}-{\mathit{E}}_{\mathit{ij}}^{\top }{\mathit{\beta }}_{\mathbf{2}}$, then the full conditional distribution of ${\mathit{\beta }}_{\mathbf{2}}$ is $\mathrm{MVN}({\mathit{\mu }}_{{\mathit{\beta }}_{\mathbf{2}}},{\mathrm{\Sigma }}_{{\mathit{\beta }}_{\mathbf{2}}})$ with mean

  $${\mathit{\mu }}_{{\mathit{\beta }}_{\mathbf{2}}}={\mathrm{\Sigma }}_{{\mathit{\beta }}_{\mathbf{2}}}\tau {\xi }^{-2}\sum _{i=1}^n\sum _{j=1}^k{\tilde{v}}_{ij}^{-1}{\mathit{E}}_{\mathit{ij}}(Y_{ij}-{\mu }_{ij}^{\left(2\right)}),$$

  and variance

  $${\mathrm{\Sigma }}_{{\mathit{\beta }}_{\mathbf{2}}}={\left({\mathrm{\Sigma }}_{\mathbf{1}}^{-1}+\tau {\xi }^{-2}\sum _{i=1}^n\sum _{j=1}^k{\tilde{v}}_{ij}^{-1}{\mathit{E}}_{\mathit{ij}}{\mathit{E}}_{\mathit{ij}}^{\top }\right)}^{-1}.$$
• To obtain the full conditional distribution of ${\beta }_{1m}$, which represents the main genetic effects, we first define ${\mu }_{ij}^{\left(1m\right)}={\mu }_{ij}+{\mathit{U}}_{\mathit{ij}}^{\top }{\mathit{\alpha }}_{\mathit{i}}-G_{ijm}{\beta }_{1m}$ and $l_{1m}=\pi ({\beta }_{1m}=0|\mathrm{rest})$. Then, the posterior distribution of ${\beta }_{1m}$ given the rest of the parameters follows a spike-and-slab distribution:

  $${\beta }_{1m}|\mathrm{rest}\sim (1-l_{1m})\mathrm{N}({\mu }_{{\beta }_{1m}},{\mathrm{\Sigma }}_{{\beta }_{1m}})+l_{1m}{\delta }_0\left({\beta }_{1m}\right),$$

  where

  $$\begin{array}{cc}\hfill {\mu }_{{\beta }_{1m}}& ={\mathrm{\Sigma }}_{{\beta }_{1m}}\tau {\xi }^{-2}\sum _{i=1}^n\sum _{j=1}^k{\tilde{v}}_{ij}^{-1}{G}_{ijm}(Y_{ij}-{\mu }_{ij}^{\left(1m\right)}),\hfill \\ \hfill {\mathrm{\Sigma }}_{{\beta }_{1m}}& ={(\tau {\xi }^{-2}\sum _{i=1}^n\sum _{j=1}^k{\tilde{v}}_{ij}^{-1}{G}_{ijm}^2+{\displaystyle \frac{1}{r_{1m}}})}^{-1}.\hfill \end{array}$$

  and the posterior mixture proportion is

  $$l_{1m}={\displaystyle \frac{{\pi }_1}{{\pi }_1+(1-{\pi }_1){\left(r_{1m}\right)}^{-\frac{1}{2}}{\left({\mathrm{\Sigma }}_{{\beta }_{1m}}\right)}^{\frac{1}{2}}\exp \left\{\frac{1}{2}{\mathrm{\Sigma }}_{{\beta }_{1m}}{\left(\tau {\xi }^{-2}{\sum }_{i=1}^n{\sum }_{j=1}^k{\tilde{v}}_{ij}^{-1}{G}_{ijm}(Y_{ij}-{\mu }_{ij}^{\left(1m\right)})\right)}^2\right\}}}.$$
  The posterior distribution of ${\beta }_{1m}$ is a mixture component distrinbution consisting of a normal distribution and a point mass at 0. At each MCMC iteration, ${\beta }_{1m}$ is drawn from $\mathrm{N}({\mu }_{{\beta }_{1m}},{\mathrm{\Sigma }}_{{\beta }_{1m}})$ with probability $(1-l_{1m})$, and set to 0 otherwise. If ${\beta }_{1m}$ = 0, then we have ${\varphi }_{1l}=0$. Otherwise, ${\varphi }_{1l}=1$.
• We show the full conditional distribution of ${\beta }_{3s}$ denoting the effect size of $G{E}_{ijs}$, the interaction between the omics features and environmental factors, $(s=1,\dots ,pq)$. With a partial residual ${\mu }_{ij}^{\left(3s\right)}={\mu }_{ij}+{\mathit{U}}_{\mathit{ij}}^{\top }{\mathit{\alpha }}_{\mathit{i}}-G{E}_{ijs}{\beta }_{3s}$ and $l_{3s}=\pi ({\beta }_{3s}=0|\mathrm{rest})$, the posterior distribution of ${\beta }_{3s}$ is expressible as follows:

  $${\beta }_{3s}|\mathrm{rest}\sim (1-l_{3s})\mathrm{N}({\mu }_{{\beta }_{3s}},{\mathrm{\Sigma }}_{{\beta }_{3s}})+l_{3s}{\delta }_0\left({\beta }_{3s}\right),$$

  where mean vector and covariance are

  $$\begin{array}{cc}\hfill {\mu }_{{\beta }_{3s}}& ={\mathrm{\Sigma }}_{{\beta }_{3s}}\tau {\xi }^{-2}\sum _{i=1}^n\sum _{j=1}^k{\tilde{v}}_{ij}^{-1}G{E}_{ijs}(Y_{ij}-{\mu }_{ij}^{\left(3s\right)}),\hfill \\ \hfill {\mathrm{\Sigma }}_{{\beta }_{3s}}& ={\left(\tau {\xi }^{-2}\sum _{i=1}^n\sum _{j=1}^k{\tilde{v}}_{ij}^{-1}G{E}_{ijs}^2+{\displaystyle \frac{1}{r_{3s}}}\right)}^{-1},\hfill \end{array}$$

  respectively, and the posterior mixture proportion is

  $$l_{3s}={\displaystyle \frac{{\pi }_2}{{\pi }_2+(1-{\pi }_2){\left(r_{3s}\right)}^{-\frac{1}{2}}{\left({\mathrm{\Sigma }}_{{\beta }_{3s}}\right)}^{\frac{1}{2}}\exp \left\{\frac{1}{2}{\mathrm{\Sigma }}_{{\beta }_{3s}}{\left(\tau {\xi }^{-2}{\sum }_{i=1}^n{\sum }_{j=1}^k{\tilde{v}}_{ij}^{-1}G{E}_{ijs}(Y_{ij}-{\mu }_{ij}^{\left(3s\right)})\right)}^2\right\}}}.$$
• The full conditional distributions of ${(r_{1m})}^{-1}$ and ${(r_{3s})}^{-1}$ are

  $$\begin{array}{c}\hfill {(r_{1m})}^{-1}|\mathrm{rest}\sim \left\{\begin{array}{cc}\mathrm{Inverse}\text{-}\mathrm{Gamma}(1,{\displaystyle \frac{{\eta }_1^2}{2}})\hfill & \mathrm{if}{\beta }_{1m}=0\hfill \\ \mathrm{Inverse}\text{-}\mathrm{Gaussian}({\eta }_1^2,\sqrt{{\displaystyle \frac{{\eta }_1^2}{{\beta }_{1m}^2}}})\hfill & \mathrm{if}{\beta }_{1m}\ne 0,\hfill \end{array}\right.\end{array}$$

  and

  $$\begin{array}{c}\hfill {(r_{3s})}^{-1}|\mathrm{rest}\sim \left\{\begin{array}{cc}\mathrm{Inverse}\text{-}\mathrm{Gamma}(1,{\displaystyle \frac{{\eta }_2^2}{2}})\hfill & \mathrm{if}{\beta }_{3s}=0\hfill \\ \mathrm{Inverse}\text{-}\mathrm{Gaussian}({\eta }_2^2,\sqrt{{\displaystyle \frac{{\eta }_2^2}{{\beta }_{3s}^2}}})\hfill & \mathrm{if}{\beta }_{3s}\ne 0.\hfill \end{array}\right.\end{array}$$
• The full conditional distributions of ${\eta }_1^2$ and ${\eta }_2^2$ are

  $$\begin{array}{c}\hfill {\eta }_1^2|\mathrm{rest}\sim \mathrm{Gamma}(c_1+p,d_1+{\displaystyle \frac{{\sum }_{m=1}^p{r}_{1m}}{2}}),\end{array}$$

  and

  $$\begin{array}{c}\hfill {\eta }_2^2|\mathrm{rest}\sim \mathrm{Gamma}(c_2+pq,d_2+{\displaystyle \frac{{\sum }_{s=1}^{pq}{s}_{3s}}{2}}).\end{array}$$
• The full conditional distributions of ${\pi }_1$ and ${\pi }_2$ are

  $$\begin{array}{c}\hfill {\pi }_1|\mathrm{rest}\sim \mathrm{Beta}(a_1+\sum _{m=1}^p{\mathit{I}}_{\{{\beta }_{1m}=0\}},b_1+\sum _{m=1}^p{\mathit{I}}_{\{{\beta }_{1m}\ne 0\}}).\end{array}$$

  and

  $$\begin{array}{c}\hfill {\pi }_2|\mathrm{rest}\sim \mathrm{Beta}(a_2+\sum _{s=1}^{pq}{\mathit{I}}_{\{{\beta }_{3s}=0\}},b_2+\sum _{s=1}^{pq}{\mathit{I}}_{\{{\beta }_{3s}\ne 0\}}).\end{array}$$
• The full conditional distribution of ${\tilde{v}}_{ij}^{-1}$:

  $$\begin{array}{c}\hfill {\tilde{v}}_{ij}^{-1}|\mathrm{rest}\sim \mathrm{Inverse}\text{-}\mathrm{Gaussian}(\sqrt{{\displaystyle \frac{2{\xi }^2}{{(Y_{ij}-{\mu }_{ij}-{\mathit{U}}_{\mathit{ij}}^{\top }{\alpha }_{\mathit{i}})}^2}}},2\tau ).\end{array}$$
• The full conditional distribution of $\tau$:

  $$\begin{array}{c}\hfill \tau |\mathrm{rest}\sim \mathrm{Gamma}(w_1+{\displaystyle \frac{3nk}{2}},w_2+\sum _{i=1}^n\sum _{j=1}^k{\tilde{v}}_{ij}+\sum _{i=1}^n\sum _{j=1}^k{(2{\xi }^2)}^{-1}{\tilde{v}}_{ij}^{-1}{(Y_{ij}-{\mu }_{ij}-{\mathit{U}}_{\mathit{ij}}^{\top }{\mathit{\alpha }}_{\mathit{i}})}^2.\end{array}$$
• The full conditional distribution of random effects ${\mathit{\alpha }}_{\mathit{i}}$ is

  $$\begin{array}{c}\hfill {\mathit{\alpha }}_{\mathit{i}}|\mathrm{rest}\sim \mathrm{MVN}({\mathit{\mu }}_{{\mathit{\alpha }}_{\mathit{i}}},{\mathrm{\Sigma }}_{{\mathit{\alpha }}_{\mathit{i}}}),\end{array}$$

  where

  $$\begin{array}{cc}\hfill {\mathit{\mu }}_{{\mathit{\alpha }}_{\mathit{i}}}& ={\displaystyle \frac{\tau }{{\xi }^2}}{\mathrm{\Sigma }}_{{\mathit{\alpha }}_{\mathit{i}}}\sum _{j=1}^k{\displaystyle \frac{{\mathit{U}}_{\mathit{ij}}(Y_{ij}-{\mu }_{ij})}{{\tilde{v}}_{ij}}},\mathrm{and}{\mathrm{\Sigma }}_{{\mathit{\alpha }}_{\mathit{i}}}={\left({\displaystyle \frac{\tau }{{\xi }^2}}\sum _{j=1}^k{\displaystyle \frac{{\mathit{U}}_{\mathit{ij}}{\mathit{U}}_{\mathit{ij}}^{\top }}{{\tilde{v}}_{ij}}}+{\displaystyle \frac{1}{{\varphi }^2}}\mathit{I}\right)}^{-1}.\hfill \end{array}$$
• Finally, the full conditional distribution of ${\varphi }^2$ is

  $$\begin{array}{c}\hfill {\varphi }^2|\mathrm{rest}\sim \mathrm{Inverse}\text{-}\mathrm{Gamma}(n+e_1,e_2+{\displaystyle \frac{1}{2}}\sum _{i=1}^n{\mathit{\alpha }}_{\mathit{i}}^{\top }{\mathit{\alpha }}_{\mathit{i}}).\end{array}$$

The Gibbs sampler, as a Markov Chain Monte Carlo (MCMC) method, generates samples from the posterior distribution of a set of parameters by iteratively sampling from the conditional distributions of each parameter, given the current values of the others. We summarize the Gibbs sampling Algorithm 1 based on the derivations of posterior distributions. In the context of Bayesian inference, the algorithm begins by initializing all parameters, then proceeds through a sequence of steps where, at each iteration, one parameter is sampled from its conditional distribution, conditioned on the most recent values of the other parameters. This process is repeated for a large number of iterations (e.g., 10,000 in out numeric studies). The rationale behind Gibbs sampling is that, although the joint posterior distribution of all the parameters may be complex, the full conditional distributions of each parameter, given the others, are easy to sample using MCMC. By repeating this process for many iterations, the algorithm generates posterior samples of model parameters that can be used to approximate the posterior distribution of the parameters.

Algorithm 1. Gibbs Sampler for Bayesian Inference

Initialize: Set initial values for ${\mathit{\beta }}_{\mathbf{0}},{\mathit{\beta }}_{\mathbf{1}},{\mathit{\beta }}_{\mathbf{2}},{\mathit{\beta }}_{\mathbf{3}},r_{1m},r_{3s},{\eta }_1^2,{\eta }_2^2,{\pi }_1,{\pi }_2,\tau$. Set iteration counter $t=0$. While $t<10000$  do: (3.1) ${\mathit{\beta }}_{\mathbf{0}}\sim \mathrm{MVN}({\mu }_{{\mathit{\beta }}_{\mathbf{0}}},{\mathrm{\Sigma }}_{{\mathit{\beta }}_{\mathbf{0}}}).\mathrm{Sampling}{\mathit{\beta }}_{\mathbf{0}}.$ (3.2) ${\mathit{\beta }}_{\mathbf{2}}\sim \mathrm{MVN}({\mu }_{{\mathit{\beta }}_{\mathbf{2}}},{\mathrm{\Sigma }}_{{\mathit{\beta }}_{\mathbf{2}}}).\mathrm{Sampling}{\mathit{\beta }}_{\mathbf{2}}.$ (3.3) ${\beta }_{1m}\sim (1-l_{1m})\mathrm{N}({\mu }_{{\beta }_{1m}},{\mathrm{\Sigma }}_{{\beta }_{1m}})+l_{1m}{\delta }_0\left({\beta }_{1m}\right).\mathrm{Sampling}{\beta }_{1m}.$ (3.4) ${\beta }_{3s}\sim (1-l_{3s})\mathrm{N}({\mu }_{{\beta }_{3s}},{\mathrm{\Sigma }}_{{\beta }_{3s}})+l_{3s}{\delta }_0({\beta }_{3s}).\mathrm{Sampling}{\beta }_{3s}.$ (3.5) $r_{1m}\sim \mathrm{Inverse}\text{-}\mathrm{Gamma}(1,{\displaystyle \frac{{\eta }_1^2}{2}})\mathrm{or}\mathrm{Inverse}\text{-}\mathrm{Gaussian}({\eta }_1^2,\sqrt{{\displaystyle \frac{{\eta }_1^2}{{\beta }_{1m}^2}}}).\mathrm{Sampling}{r}_{1m}.$ (3.6) $r_{3s}\sim \mathrm{Inverse}\text{-}\mathrm{Gamma}(1,{\displaystyle \frac{{\eta }_2^2}{2}})\mathrm{or}\mathrm{Inverse}\text{-}\mathrm{Gaussian}({\eta }_2^2,\sqrt{{\displaystyle \frac{{\eta }_2^2}{{\beta }_{3s}^2}}}).\mathrm{Sampling}{r}_{3s}.$ (3.7) ${\eta }_1^2\sim \mathrm{Gamma}(c_1+p,d_1+{\displaystyle \frac{{\sum }_{m=1}^p{r}_{1m}}{2}}).\mathrm{Sampling}{\eta }_1^2.$ (3.8) ${\eta }_2^2\sim \mathrm{Gamma}(c_2+pq,d_2+{\displaystyle \frac{{\sum }_{s=1}^{pq}{r}_{3s}}{2}}).\mathrm{Sampling}{\eta }_2^2.$ (3.9) ${\pi }_1\sim \mathrm{Beta}(a_1+{\sum }_{m=1}^p{\mathbb{I}}_{\{{\beta }_{1m}=0\}},b_1+{\sum }_{m=1}^p{\mathbb{I}}_{\{{\beta }_{1m}\ne 0\}}).\mathrm{Sampling}{\pi }_1.$ (3.10) ${\pi }_2\sim \mathrm{Beta}(a_2+{\sum }_{s=1}^{pq}{\mathbb{I}}_{\{{\beta }_{3s}=0\}},b_2+{\sum }_{s=1}^{pq}{\mathbb{I}}_{\{{\beta }_{3s}\ne 0\}}).\mathrm{Sampling}{\pi }_2.$ (3.11) ${\tilde{v}}_{ij}^{-1}\sim \mathrm{Inverse}\text{-}\mathrm{Gaussian}(\sqrt{{\displaystyle \frac{2{\xi }^2}{{(Y_{ij}-{\mu }_{ij}-{\mathit{U}}_{\mathit{ij}}^{\top }{\mathit{\alpha }}_{\mathit{i}})}^2}}},2\tau ).\mathrm{Sampling}{\tilde{v}}_{ij}^{-1}.$ (3.12) $\tau \sim \mathrm{Gamma}(w_1+{\displaystyle \frac{3nk}{2}},w_2+{\sum }_{i=1}^n{\sum }_{j=1}^k{\tilde{v}}_{ij}+{\sum }_{i=1}^n{\sum }_{j=1}^k{\displaystyle \frac{{(Y_{ij}-{\mu }_{ij}-{\mathit{U}}_{\mathit{ij}}^{\top }{\mathit{\alpha }}_{\mathit{i}})}^2}{2{\xi }^2}}).\mathrm{Sampling}\tau .$ (3.13) ${\mathit{\alpha }}_{\mathit{i}}\sim \mathrm{MVN}({\mu }_{{\alpha }_i},{\mathrm{\Sigma }}_{{\alpha }_i}).\mathrm{Sampling}{\mathit{\alpha }}_{\mathit{i}}.$ (3.14) ${\varphi }^2\sim \mathrm{Inverse}\text{-}\mathrm{Gamma}(n+e_1,e_2+{\displaystyle \frac{1}{2}}{\sum }_{i=1}^n{\mathit{\alpha }}_{\mathit{i}}^{\top }{\alpha }_{\mathit{i}}).\mathrm{Sampling}{\varphi }^2.$ Increment $t=t+1$. Output: Posterior samples for ${\mathit{\beta }}_{\mathbf{0}},{\mathit{\beta }}_{\mathbf{1}},{\mathit{\beta }}_{\mathbf{2}},{\mathit{\beta }}_{\mathbf{3}},r_{1m},r_{3s},{\eta }_1^2,{\eta }_2^2,{\pi }_1,{\pi }_2,\tau ,{\mathit{\alpha }}_{\mathit{i}},{\varphi }^2$.

3. Results

3.1. Simulation

We assess the performance of the FDR-assisted robust Bayesian linear mixed model with spike-and-slab priors, or RBLSS, with three alternative methods. A direct competitor is its non-robust counterpart, the FDR-assisted Bayesian linear mixed model with spike-and-slab priors, which is termed as BLSS. We also consider robust Bayesian linear mixed model, or RBL, without using spike-and-slab priors. Laplacian shrinkage priors are placed on high-dimensional genetic main effects and interactions. The non-robust variant of RBL is termed as BL.

The mixed model (1) has been adopted to generate longitudinal phenotypes with sample size n = 500 and three time points (i.e., k = 3). The number of genetic factors is p = 50 and 100, with q = 3 environmental factors, leading to a total dimension of main genetic effects and interactions as $p+pq$ = 200 and 400, respectively. Environmental factors are simulated from multivariate normal distributions with marginal mean 0, marginal variance 1, and an AR–1 correlation with $\rho$ = 0.5. The non-zero fixed effects from model (1), consisting of those that are not subject to selection (i.e., ${\mathit{\beta }}_0$ and ${\mathit{\beta }}_2$) as well as those corresponding to important main and interaction effects from ${\mathit{\beta }}_1$ and ${\mathit{\beta }}_3$, are generated from a uniform distribution Unif [0.4, 0.8]. A total of 8 main G effects and 12 $G\times E$ interactions are assumed to be associated with the longitudinal response. Under the random intercept-and-slope model (1), each component of the random effects is simulated from a standard normal distribution. The random error are generated from the following: (1) N(0, 1) (Error 1), (2) t-distribution with 2 degrees of freedom (t(2)) (Error2), (3) LogNormal(0,2) (Error3), (4) Laplace(0,$\sqrt{2}$) (Error4). Except N(0,1), all the rest are heavy-tailed model errors.

In addition, we simulate genetic factors according to the three settings below. In Setting 1, continuous genetic factors are generated from the same multivariate Gaussian distribution adopted to generate environmental factors. The AR structure is considered in computing the gene correlation, under which gene j and k have correlation ${\rho }^{|j-k|}$ where $\rho =0.5$, with variance 1. Since the single nucleotide polymorphism (SNP) data are categorical, we further consider two more realistic settings to generate high-dimensional genetic factors.

Specifically, in Setting 2, we mimic the categorical nature of SNP data by dichotomizing the gene expression values according to quantile levels ${\displaystyle \frac{1}{3}}$ and ${\displaystyle \frac{2}{3}}$, with the 3 levels (0, 1, 2) corresponding to genotypes (0 = aa, 1 = Aa, and 2 = AA). In Setting 3, SNP data are simulated based on a pairwise linkage disequilibrium (LD) pattern. Given LD and the minor allele frequency (MAF) of two adjacent SNPs, such as SNP 1 and SNP 2, one can compute four hapolytypes, which leads to a conditional genotype probability matrix of SNP 2 given SNP 1. Then, in general, SNP genotypes can be simulated according to the conditional distribution of current genotypes based on the previous genotype data. Please refer to Ren et al. (2023) [30] for more details about the data-generating procedure.

For the two methods with spike-and-slab priors, we apply the Bayesian FDR procedure described in Section 2.4 with a global FDR of 0.05 after model fitting. The 95% marginal credible intervals are applied to identify important findings under methods without using spike-and-slab priors. Variable selection accuracy is assessed based on the number of true positives (TPs), false positives (FPs), F1 score, and Matthews correlation coefficient (MCC), where F1 and MCC are defined as $\mathrm{F}1={\displaystyle \frac{2TP}{2TP+FP+FN}}$ and $\mathrm{MCC}={\displaystyle \frac{TP·TN-FP·FN}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}}}$, respectively. In terms of estimation accuracy, among selected main and interaction effects, we compute ${\mathit{\ell }}_1$ distance between the estimated and true fixed effects.

Table 1. Assessment under Setting 1 with $(n,p,q,k)=(400,50,3,3)$ and 100 replicates.

Error	Method	TP	FP	F1	MCC	Estimation Error
$N(0,1)$	RBLSS	19.38 (0.78)	1.36 (1.12)	0.95 (0.03)	0.95 (0.04)	2.21 (0.55)
	RBL	18.66 (1.12)	3.10 (1.99)	0.89 (0.05)	0.88 (0.05)	14.08 (1.09)
	BLSS	19.32 (0.84)	0.80 (0.86)	0.96 (0.03)	0.96 (0.04)	2.10 (0.53)
	BL	18.76 (1.10)	2.04 (1.71)	0.92 (0.04)	0.91 (0.05)	13.29 (1.13)
$t\left(2\right)$	RBLSS	18.66 (0.94)	0.72 (0.73)	0.95 (0.03)	0.94 (0.04)	2.68 (0.61)
	RBL	17.32 (1.30)	1.30 (1.13)	0.90 (0.05)	0.89 (0.05)	15.41 (0.99)
	BLSS	11.68 (3.99)	0.42 (0.64)	0.71 (0.18)	0.72 (0.15)	6.27 (2.44)
	BL	12.62 (3.52)	2.96 (2.44)	0.70 (0.16)	0.68 (0.15)	27.65 (8.11)
$logN(0,1)$	RBLSS	19.10 (1.04)	0.88 (0.82)	0.96 (0.04)	0.95 (0.04)	2.40 (0.72)
	RBL	18.08 (1.18)	1.56 (1.39)	0.91 (0.05)	0.90 (0.05)	14.65 (1.35)
	BLSS	16.06 (2.11)	0.46 (0.81)	0.88 (0.08)	0.87 (0.07)	3.95 (1.15)
	BL	15.18 (1.99)	2.12 (1.86)	0.81 (0.07)	0.80 (0.07)	20.49 (2.36)
$Laplace(0,\sqrt{2}$)	RBLSS	16.38 (2.01)	0.90 (0.99)	0.88 (0.08)	0.87 (0.08)	3.61 (1.09)
	RBL	15.50 (2.03)	1.46 (1.03)	0.84 (0.07)	0.83 (0.08)	15.70 (1.44)
	BLSS	14.12 (2.62)	0.36 (0.60)	0.81 (0.09)	0.81 (0.09)	4.21 (1.33)
	BL	14.40 (2.19)	2.06 (1.63)	0.79 (0.09)	0.77 (0.09)	19.80 (1.70)

shows the performance in terms of the above metrics under Setting 1 across four error distributions. With 50 genetic factors and 3 environmental variables, there are a total of 150 main and interaction effects. Under the $N(0,1)$ model error, the two FDR-assisted methods, RBLSS and BLSS, are the top-performing approaches, with BLSS showing a slight advantage. Specifically, BLSS achieves a lower average FP of 0.80 (sd 0.86) compared to 1.36 (sd 1.12) for RBLSS, while maintaining a similar number of true positives. BLSS also attains the highest F1 score and Matthews correlation coefficient (MCC), both approximately 0.96, outperforming the other methods, particularly RBL and BL. Additionally, it yields the lowest estimation error of 2.10 (sd 0.53). In the presence of skewed model errors, Table 1 indicates that the advantage shifts to RBLSS. For example, under t(2) error, RBLSS has the highest F1 score of 0.95 (sd 0.03) and MCC of 0.94 (sd 0.04), as well as the lowest estimation error, 2.68 (sd 0.61). In terms of variable selection, RBL is the second-best method, achieving both an F1 score and MCC of around 0.90. However, it yields a much larger estimation error of 15.41 (sd 0.99). When the number of main genetic effects increases to 100 under Setting 1,

Table 2. Assessment under Setting 1 with $(n,p,q,k)=(400,100,3,3)$ and 100 replicates.

Error	Method	TP	FP	F1	MCC	Estimation Error
$N(0,1)$	RBLSS	19.16 (0.84)	1.56 (1.37)	0.94 (0.04)	0.94 (0.05)	2.46 (0.75)
	RBL	17.02 (1.53)	2.34 (1.61)	0.86 (0.05)	0.86 (0.05)	23.47 (1.41)
	BLSS	19.40 (0.73)	0.58 (0.76)	0.97 (0.02)	0.97 (0.02)	2.06 (0.48)
	BL	17.38 (1.34)	1.04 (1.18)	0.90 (0.05)	0.90 (0.05)	21.15 (1.28)
$t\left(2\right)$	RBLSS	17.28 (1.33)	0.46 (0.68)	0.91 (0.04)	0.91 (0.04)	3.26 (0.80)
	RBL	13.68 (1.87)	0.66 (0.92)	0.79 (0.07)	0.80 (0.06)	24.41 (1.25)
	BLSS	10.16 (3.37)	0.48 (0.84)	0.65 (0.17)	0.68 (0.14)	6.86 (2.29)
	BL	7.86 (2.84)	1.58 (2.14)	0.52 (0.17)	0.56 (0.15)	44.21(14.99)
$logN(0,1)$	RBLSS	19.12 (0.90)	0.64 (0.78)	0.96 (0.04)	0.96 (0.04)	2.31 (0.71)
	RBL	16.42 (1.46)	0.96 (0.95)	0.88 (0.05)	0.88 (0.05)	23.73 (1.66)
	BLSS	16.04 (1.68)	0.40 (0.67)	0.88 (0.06)	0.88 (0.05)	4.06 (0.95)
	BL	12.28 (1.95)	1.08 (1.24)	0.73 (0.08)	0.74 (0.08)	33.93 (3.81)
$Laplace(0,\sqrt{2}$)	RBLSS	17.00 (1.50)	0.86 (0.99)	0.90 (0.05)	0.89 (0.05)	3.67 (1.06)
	RBL	14.16 (1.84)	1.20 (1.29)	0.80 (0.07)	0.80 (0.07)	25.37 (1.50)
	BLSS	15.18 (1.80)	0.36 (0.69)	0.85 (0.06)	0.85 (0.06)	4.27 (1.21)
	BL	12.12 (2.22)	1.08 (0.97)	0.73 (0.09)	0.73 (0.08)	32.41 (2.11)

shows that the pattern remains consistent with that in Table 1. FDR-assisted RBLSS outperforms the other three in both variable selection and estimation error.

Table 1 shows the following patterns that warrant more thorough discussions. First, under $N(0,1)$ error, the non-robust BLSS is comparable to, or even slightly outperforms, the robust RBLSS, provided that the variability in point estimates of the F1 score, MCC, and estimation errors is ignored. In general, the suboptimal performance of robust statistical methods under Gaussian model errors is well-documented, as these methods penalize or downweight extreme observations that are infrequent in such cases [12]. This leads to unnecessary penalization of observations, resulting in decreased efficiency due to the trade-off between robustness and efficiency.

Table 1 also shows that, under Laplace(0, $\sqrt{2}$) error, the advantage of RBLSS over BLSS diminishes compared to the case with $t\left(2\right)$ error. The main reason for this is that the Laplace(0, $\sqrt{2}$) distribution generates fewer extreme observations than the $t\left(2\right)$ distribution and behaves more similarly to the $N(0,1)$ distribution. Specifically, the Laplace distribution has a sharper peak and exponentially decaying tails, making it more concentrated around the mean, similar to $N(0,1)$, but less likely to producing extreme values than the $t\left(2\right)$, which has a wider, less sharp peak and polynomially decaying tails.

Furthermore, Table 1 and additional numerical results demonstrate that regardless of model errors, RBL and BL, the methods utilizing Laplacian shrinkage priors instead of spike-and-slab priors for shrinkage estimation, yield consistently higher estimation error. This is because Laplacian shrinkage priors cannot shrink regression coefficients to exactly zero. As a result, for a sparse regression coefficient vector, they produce much larger estimation errors (and false positive signals) compared to RBLSS and BLSS, which can achieve exact zeros using spike-and-slab priors. Such a phenomenon suggests that, in certain high-dimensional settings, the choice of an aggressive sparsity-inducing prior can outweigh the impact of selecting a robust likelihood in terms of model performance.

We have also visualized the results in Table 1 and Table 2 by generating box-plots of F1 score, MCC and estimation error across 100 replicates in Figure 1, Figure 2 and Figure 3, respectively. In the box-plots, the central line represents the median, the box denotes the interquartile range, and the whiskers and points indicate variability and potential outliers. Figure 2 and Figure 3 demonstrate that for non-robust BLSS and BL, variable selection accuracy in terms of F1 score and MCC deteriorates dramatically under t(2) error which is more skewed than the other three model errors. Although the two non-robust methods show slight improvement under $logN(0,1)$ (Error 3) and Laplace $(0,\sqrt{2})$ (Error 4), which are less heavy-tailed than t(2), the advantage of the robust methods, especially RBLSS, are still evident. By comparing RBL and BLSS, Figure 1, Figure 2 and Figure 3 demonstrate an interesting phenomenon that under Laplace $(0,\sqrt{2})$ (Error 4) and p = 100, the non-robust BLSS can even dominate robust RBL in both identification and estimation. In fact, even under skewed model errors, it is not surprising to observe better estimation performance from non-robust spike-and-slab models compared to robust models without spike-and-slab priors, as the former induces exact sparsity in posterior estimates, and is more consistent with the sparse nature of high-dimensional omics data. However, it is impressive that BLSS also has a advantage over RBL in variable selection.

Setting 2 differs from Setting 1 in that all high-dimensional omics features are categorical. Therefore, it is more similar in structure to SNP data. Results under Setting 2 are shown in

Table A1. Assessment under Setting 2 with $(n,p,q,k)=(400,50,3,3)$ and 100 replicates.

Error	Method	TP	FP	F1	MCC	Estimation Error
$N(0,1)$	RBLSS	19.78 (0.46)	1.52 (1.23)	0.96 (0.03)	0.95 (0.03)	2.01 (0.55)
	RBL	19.48 (0.68)	3.30 (1.84)	0.91 (0.04)	0.90 (0.05)	14.14 (1.05)
	BLSS	19.88 (0.33)	0.94 (0.84)	0.97 (0.02)	0.97 (0.02)	1.72 (0.41)
	BL	19.66 (0.56)	1.96 (1.68)	0.95 (0.04)	0.94 (0.04)	13.35 (1.00)
$t\left(2\right)$	RBLSS	17.52 (1.37)	0.62 (0.81)	0.92 (0.05)	0.91 (0.05)	2.98 (0.78)
	RBL	16.48 (1.54)	1.04 (1.01)	0.88 (0.05)	0.87 (0.05)	14.56 (1.06)
	BLSS	12.12 (3.33)	0.28 (0.54)	0.73 (0.14)	0.75 (0.12)	5.22 (1.76)
	BL	12.76 (2.43)	3.04 (2.51)	0.71 (0.11)	0.69 (0.12)	23.70 (6.15)
$logN(0,1)$	RBLSS	19.36 (0.83)	0.86 (0.81)	0.96 (0.03)	0.96 (0.04)	2.22 (0.65)
	RBL	18.56 (0.97)	2.10 (1.46)	0.91 (0.04)	0.90 (0.04)	14.13 (1.09)
	BLSS	16.96 (1.46)	0.44 (0.73)	0.91 (0.05)	0.90 (0.05)	3.56 (1.01)
	BL	16.40 (1.59)	2.66 (1.89)	0.84 (0.06)	0.82 (0.07)	20.08 (2.44)
$Laplace(0,\sqrt{2}$)	RBLSS	17.92 (1.23)	1.10 (0.89)	0.92 (0.05)	0.91 (0.05)	3.18 (0.77)
	RBL	16.66 (1.36)	1.96 (1.28)	0.86 (0.05)	0.85 (0.06)	15.82 (1.02)
	BLSS	17.16 (1.33)	0.72 (0.57)	0.90 (0.04)	0.90 (0.05)	3.31 (0.72)
	BL	16.42 (1.16)	2.62 (1.72)	0.84 (0.05)	0.83 (0.06)	19.22 (1.39)

and

Table A2. Assessment under Setting 2 with $(n,p,q,k)=(400,100,3,3)$ and 100 replicates.

Error	Method	TP	FP	F1	MCC	Estimation Error
$N(0,1)$	RBLSS	19.40 (0.67)	1.84 (1.58)	0.94 (0.04)	0.94 (0.04)	2.29 (0.65)
	RBL	16.88 (1.19)	2.20 (1.47)	0.86 (0.04)	0.86 (0.05)	23.05 (1.42)
	BLSS	19.48 (0.65)	0.76 (0.82)	0.97 (0.03)	0.97 (0.03)	1.91 (0.47)
	BL	17.48 (1.11)	1.06 (1.11)	0.91 (0.04)	0.90 (0.04)	20.96 (1.34)
$t\left(2\right)$	RBLSS	18.16 (1.13)	0.70 (0.81)	0.93 (0.04)	0.93 (0.04)	2.84 (0.85)
	RBL	15.06 (1.68)	1.00 (0.97)	0.83 (0.06)	0.83 (0.06)	24.32 (1.44)
	BLSS	11.90 (4.57)	0.14 (0.40)	0.71 (0.23)	0.75 (0.17)	5.65 (2.60)
	BL	8.80 (3.57)	1.46 (1.85)	0.56 (0.20)	0.60 (0.17)	42.45(17.73)
$logN(0,1)$	RBLSS	19.34 (0.66)	0.94 (0.93)	0.96 (0.03)	0.96 (0.03)	2.21 (0.55)
	RBL	17.06 (1.38)	1.62 (1.24)	0.88 (0.05)	0.88 (0.05)	24.31 (1.57)
	BLSS	17.66 (1.39)	0.66 (0.75)	0.92 (0.05)	0.92 (0.05)	3.47 (1.01)
	BL	13.64 (1.85)	1.88 (2.13)	0.77 (0.08)	0.77 (0.08)	34.34 (4.74)
$Laplace(0,\sqrt{2}$)	RBLSS	17.04 (1.62)	1.10 (1.05)	0.89 (0.06)	0.89 (0.06)	3.50 (0.98)
	RBL	14.18 (1.47)	1.34 (1.14)	0.80 (0.05)	0.80 (0.05)	25.21 (1.52)
	BLSS	15.54 (1.79)	0.48 (0.79)	0.86 (0.06)	0.86 (0.06)	3.87 (0.85)
	BL	12.46 (1.82)	1.32 (1.11)	0.74 (0.08)	0.74 (0.08)	32.00 (2.15)

, with Figure A1, Figure A2, Figure A3 in Appendix A. The superior performance of RBLSS over the alternatives in variable selection and estimation under heavy-tailed model errors remains consistent. The previously noted pattern reappears in the comparison between BLSS and RBL. As indicated in Table A2 and Figure A1, Figure A2, Figure A3, the non-robust BLSS dominates robust RBL in both identification and shrinkage estimation under heavy-tailed model errors such as $logN$(0,1) and $Laplace(0,\sqrt{2})$. In Appendix A,

Table A3. Assessment under Setting 3 with $(n,p,q,k)=(400,50,3,3)$ and 100 replicates.

Error	Method	TP	FP	F1	MCC	Estimation Error
$N(0,1)$	RBLSS	19.56 (0.73)	1.76 (0.85)	0.95 (0.03)	0.94 (0.04)	2.30 (0.60)
	RBL	19.00 (1.07)	3.68 (1.99)	0.89 (0.06)	0.88 (0.06)	14.39 (1.07)
	BLSS	19.84 (0.37)	0.94 (0.74)	0.97 (0.02)	0.97 (0.02)	1.90 (0.45)
	BL	19.02 (0.98)	2.00 (1.32)	0.93 (0.04)	0.92 (0.04)	13.54 (0.88)
$t\left(2\right)$	RBLSS	17.28 (1.69)	0.70 (0.79)	0.91 (0.06)	0.90 (0.06)	3.28 (0.88)
	RBL	15.88 (1.73)	1.42 (1.18)	0.85 (0.06)	0.84 (0.06)	15.22 (1.25)
	BLSS	11.22 (4.16)	0.28 (0.50)	0.69 (0.21)	0.73 (0.13)	5.90 (1.85)
	BL	11.48 (3.56)	2.66 (2.87)	0.66 (0.18)	0.66 (0.14)	24.50 (4.82)
$logN(0,1)$	RBLSS	19.08 (0.90)	0.78 (0.91)	0.96 (0.03)	0.95 (0.03)	2.38 (0.70)
	RBL	18.10 (1.16)	1.46 (1.39)	0.92 (0.05)	0.91 (0.05)	14.58 (1.28)
	BLSS	15.94 (2.03)	0.54 (0.86)	0.87 (0.07)	0.87 (0.07)	4.26 (1.19)
	BL	15.50 (1.63)	2.22 (1.80)	0.82 (0.07)	0.81 (0.08)	21.61 (3.12)
$Laplace(0,\sqrt{2}$)	RBLSS	17.88 (1.45)	0.76 (0.69)	0.92 (0.05)	0.92 (0.05)	3.24 (0.92)
	RBL	16.68 (1.57)	1.48 (1.33)	0.87 (0.06)	0.86 (0.06)	15.97 (1.21)
	BLSS	16.60 (1.55)	0.38 (0.60)	0.90 (0.05)	0.89 (0.05)	3.61 (0.91)
	BL	15.96 (1.59)	2.20 (1.54)	0.84 (0.06)	0.82 (0.07)	20.12 (1.66)

and

Table A4. Assessment under Setting 3 with $(n,p,q,k)=(400,100,3,3)$ and 100 replicates.

Error	Method	TP	FP	F1	MCC	Estimation Error
$N(0,1)$	RBLSS	19.34 (0.69)	1.40 (1.12)	0.95 (0.03)	0.95 (0.04)	2.24 (0.58)
	RBL	17.38 (1.18)	2.40 (1.65)	0.87 (0.05)	0.87 (0.05)	23.69 (1.40)
	BLSS	19.58 (0.61)	0.48 (0.65)	0.98 (0.02)	0.98 (0.03)	1.84 (0.44)
	BL	17.98 (1.20)	1.00 (0.97)	0.92 (0.04)	0.92 (0.04)	21.22 (1.17)
$t\left(2\right)$	RBLSS	17.28 (1.40)	0.78 (1.06)	0.91 (0.05)	0.90 (0.05)	3.32 (0.93)
	RBL	13.58 (1.93)	1.20 (1.37)	0.78 (0.07)	0.78 (0.07)	25.01 (1.93)
	BLSS	10.44 (3.39)	0.22 (0.46)	0.66 (0.17)	0.69 (0.14)	6.52 (2.15)
	BL	8.24 (3.01)	1.84 (2.38)	0.53 (0.17)	0.57 (0.15)	43.24(14.10)
$logN(0,1)$	RBLSS	19.32 (0.91)	0.70 (0.84)	0.97 (0.03)	0.96 (0.03)	2.30 (0.66)
	RBL	15.58 (2.04)	1.26 (1.19)	0.84 (0.07)	0.84 (0.07)	24.06 (1.51)
	BLSS	15.30 (2.73)	0.48 (0.68)	0.85 (0.10)	0.85 (0.09)	4.55 (1.69)
	BL	11.90 (2.24)	1.24 (1.30)	0.71 (0.09)	0.72 (0.09)	34.67 (4.79)
$Laplace(0,\sqrt{2}$)	RBLSS	15.94 (1.60)	0.92 (0.80)	0.86 (0.06)	0.86 (0.06)	4.03 (1.02)
	RBL	12.56 (1.64)	1.36 (1.24)	0.74 (0.07)	0.74 (0.07)	25.56 (1.63)
	BLSS	14.22 (1.79)	0.50 (0.68)	0.82 (0.07)	0.82 (0.06)	4.66 (1.06)
	BL	10.68 (1.96)	1.10 (1.07)	0.67 (0.09)	0.68 (0.08)	32.21 (2.23)

demonstrate the performance metrics evaluated under Setting 3 where LD correlations have been adopted to simulate SNP data. The same conclusion about the superiority of RBLSS can be drawn.

We perform additional analyses to illustrate that the proposed method achieves satisfactory performance in finite-sample FDR control. In our study, we follow Müller et al. (2007) [58] to define Bayesian FDR, as shown in Equation (4) in Section 2.4. At the 0.05 FDR level for all three simulation settings, we have computed the Bayesian FDR of RBLSS and BLSS using Equation (4) over 100 replicates, and generated the corresponding violin plots, which are shown in Figure 4, as well as Figure A4 and Figure A5 in Appendix A, respectively. These plots indicate that finite sample FDR control has been achieved. Moreover, sensitivity analysis has been conducted to show that RBLSS is insensitive to difference choices of hyperparameters. Please refer to

Table A5. Hyperparameter sensitivity analysis of Beta priors on ${\pi }_1$ and ${\pi }_2$ in RBLSS under Setting 1 with $(n,p,q,k)=(400,50,3,3)$ and 100 replicates.

Error	Prior	TP	FP	F1	MCC	Estimation Error
$N(0,1)$	$\mathrm{Beta}(0.5,0.5)$	19.24 (0.98)	1.60 (1.31)	0.94 (0.04)	0.94 (0.05)	2.39 (0.64)
	$\mathrm{Beta}(1,1)$	19.18 (0.96)	1.42 (1.03)	0.94 (0.04)	0.94 (0.04)	2.40 (0.59)
	$\mathrm{Beta}(2,2)$	19.32 (0.89)	1.50 (1.15)	0.95 (0.04)	0.94 (0.04)	2.40 (0.61)
	$\mathrm{Beta}(1,5)$	19.26 (0.99)	1.44 (1.01)	0.95 (0.04)	0.94 (0.04)	2.37 (0.57)
	$\mathrm{Beta}(5,1)$	19.36 (0.85)	1.92 (1.24)	0.94 (0.04)	0.93 (0.04)	2.46 (0.67)
$t\left(2\right)$	$\mathrm{Beta}(0.5,0.5)$	18.70 (1.15)	0.70 (0.93)	0.95 (0.04)	0.94 (0.04)	2.70 (0.66)
	$\mathrm{Beta}(1,1)$	18.58 (1.20)	0.66 (0.75)	0.95 (0.04)	0.94 (0.04)	2.74 (0.70)
	$\mathrm{Beta}(2,2)$	18.78 (0.93)	0.72 (0.90)	0.95 (0.03)	0.95 (0.04)	2.66 (0.61)
	$\mathrm{Beta}(1,5)$	18.50 (1.05)	0.76 (0.89)	0.94 (0.04)	0.94 (0.04)	2.80 (0.62)
	$\mathrm{Beta}(5,1)$	18.90 (0.97)	1.04 (1.24)	0.95 (0.04)	0.94 (0.05)	2.80 (0.79)
$logN(0,1)$	$\mathrm{Beta}(0.5,0.5)$	19.10 (1.02)	0.88 (0.82)	0.96 (0.04)	0.95 (0.04)	2.43 (0.69)
	$\mathrm{Beta}(1,1)$	19.14 (0.88)	0.86 (0.93)	0.96 (0.03)	0.95 (0.04)	2.41 (0.60)
	$\mathrm{Beta}(2,2)$	19.16 (1.09)	0.90 (0.95)	0.96 (0.04)	0.95 (0.05)	2.39 (0.74)
	$\mathrm{Beta}(1,5)$	19.18 (0.85)	0.72 (0.95)	0.96 (0.04)	0.96 (0.04)	2.40 (0.77)
	$\mathrm{Beta}(5,1)$	19.26 (0.88)	1.22 (0.93)	0.95 (0.03)	0.95 (0.04)	2.46 (0.70)
$Laplace(0,\sqrt{2})$	$\mathrm{Beta}(0.5,0.5)$	16.46 (2.17)	0.88 (1.06)	0.88 (0.08)	0.87 (0.09)	3.55 (1.16)
	$\mathrm{Beta}(1,1)$	16.22 (2.06)	0.78 (1.00)	0.87 (0.08)	0.87 (0.08)	3.65 (1.17)
	$\mathrm{Beta}(2,2)$	16.48 (1.92)	0.72 (0.78)	0.88 (0.07)	0.88 (0.07)	3.51 (1.06)
	$\mathrm{Beta}(1,5)$	16.28 (1.95)	0.72 (0.78)	0.88 (0.07)	0.87 (0.07)	3.59 (1.00)
	$\mathrm{Beta}(5,1)$	16.70 (1.95)	1.14 (0.95)	0.88 (0.07)	0.87 (0.08)	3.69 (1.18)

and

Table A6. Hyperparameter sensitivity analysis of Gamma priors on ${\eta }_1^2$, ${\eta }_2^2$ and ${\tau }^2$ in RBLSS under Setting 1 with $(n,p)=(400,50)$ and 100 replicates.

Error	Prior	TP	FP	F1	MCC	Estimation Error
$N(0,1)$	$\mathrm{Gamma}(0.1,1)$	19.28 (0.88)	1.34 (1.17)	0.95 (0.04)	0.94 (0.05)	2.35 (0.63)
	$\mathrm{Gamma}(1,1)$	19.18 (0.96)	1.42 (1.03)	0.94 (0.04)	0.94 (0.04)	2.40 (0.59)
	$\mathrm{Gamma}(5,1)$	19.36 (0.78)	1.94 (1.30)	0.94 (0.03)	0.93 (0.04)	2.41 (0.55)
	$\mathrm{Gamma}(1,2)$	19.24 (0.87)	1.26 (1.01)	0.95 (0.03)	0.95 (0.04)	2.29 (0.53)
	$\mathrm{Gamma}(1,5)$	18.92 (1.05)	1.02 (1.02)	0.95 (0.04)	0.94 (0.05)	2.38 (0.69)
$t\left(2\right)$	$\mathrm{Gamma}(0.1,1)$	18.72 (1.03)	0.68 (1.00)	0.95 (0.04)	0.95 (0.04)	2.73 (0.68)
	$\mathrm{Gamma}(1,1)$	18.58 (1.20)	0.66 (0.75)	0.95 (0.04)	0.94 (0.04)	2.74 (0.70)
	$\mathrm{Gamma}(2,2)$	18.66 (1.24)	1.12 (1.14)	0.94 (0.05)	0.93 (0.05)	2.85 (0.79)
	$\mathrm{Gamma}(1,5)$	18.44 (1.13)	0.66 (0.80)	0.94 (0.04)	0.94 (0.04)	2.80 (0.73)
	$\mathrm{Gamma}(5,1)$	18.24 (1.13)	0.46 (0.71)	0.94 (0.04)	0.94 (0.04)	2.86 (0.75)
$logN(0,1)$	$\mathrm{Gamma}(0.1,1)$	19.26 (0.88)	0.78 (0.82)	0.96 (0.03)	0.96 (0.04)	2.37 (0.64)
	$\mathrm{Gamma}(1,1)$	19.14 (0.88)	0.86 (0.93)	0.96 (0.03)	0.95 (0.04)	2.41 (0.60)
	$\mathrm{Gamma}(2,2)$	19.16 (0.82)	1.08 (1.01)	0.95 (0.03)	0.95 (0.04)	2.42 (0.69)
	$\mathrm{Gamma}(1,5)$	19.02 (0.96)	0.84 (0.87)	0.95 (0.03)	0.95 (0.04)	2.44 (0.63)
	$\mathrm{Gamma}(5,1)$	18.90 (1.28)	0.60 (0.73)	0.96 (0.04)	0.95 (0.05)	2.45 (0.84)
$Laplace(0,\sqrt{2})$	$\mathrm{Gamma}(0.1,1)$	16.42 (2.00)	0.84 (0.93)	0.88 (0.07)	0.87 (0.08)	3.65 (1.25)
	$\mathrm{Gamma}(1,1)$	16.22 (2.06)	0.78 (1.00)	0.87 (0.08)	0.87 (0.08)	3.65 (1.17)
	$\mathrm{Gamma}(2,2)$	16.82 (1.91)	1.02 (0.89)	0.89 (0.07)	0.88 (0.07)	3.51 (1.08)
	$\mathrm{Gamma}(1,5)$	16.14 (2.06)	0.78 (0.95)	0.87 (0.08)	0.86 (0.08)	3.75 (1.18)
	$\mathrm{Gamma}(5,1)$	15.82 (1.87)	0.76 (0.82)	0.86 (0.06)	0.86 (0.07)	3.90 (1.13)

in Appendix A for more details.

3.2. Case Study

We analyze the longitudinal lipidomics data generated using weight–controlled CD1 mice from a repeated measure one-way ANOVA study [52]. The weight of 60 female CD-1 mice repeatedly measured on a weekly basis over 11 weeks is the phenotype of interest. King et al. (2015) [52] have profiled 176 plasma neutral lipid species, focusing on diacylglycerols. After excluding species with measurements below detection limits in most samples, the dataset contains 31 lipid features that are used as genetic factors. The environmental factor is treatment regarding exercise and/or dietary restriction with the following four treatment groups: control (sedentary with ad libitum feeding), AE (treadmill exercise with ad libitum feeding), PE (treadmill exercise with pair feeding), and DCR (sedentary with 20% dietary calorie restriction). With 31 lipid (or G) factors and 1 environmental factors, there are additionally 31 G × E interactions. Our goal of analysis is to identify top-ranked main and interaction effects with FDR control. A comparison based on the Bayesian Information Criterion (BIC), excluding lipid features, has shown that the random intercept-and-slope model (BIC = 2496.4) has outperformed the random intercept model (BIC = 2822.7). Accordingly, the random intercept-and-slope model has been adopted for the follow-up analysis.

Here, we apply FDR-assisted RBLSS and BLSS to detect important effects.

Table 3. PIP of top-ranked main and interaction effects identified by RBLSS with an FDR of $\alpha$ = 0.05.

Lipid	Main PIP	Interaction PIP
C18:2/16:1	0.9597	-
C18:2/18:1	0.9752	-
C22:7/16:0	-	0.9087
C16:0/18:1	-	0.9501
C18:3/18:1	-	1.0000
C18:2/20:4	-	0.9248

shows top-ranked main and interaction effects identified by RBLSS with posterior inclusion probabilities (PIP) with an FDR of $\alpha$ = 0.05. In total, there are two main effects and four interaction effects. The structure of a phospholipid has the following meanings. For example, C18:2/20:4 refers to a phospholipid consisting of two fatty acid chains: one with 18 carbons and two double bonds, and the other with 20 carbons and four double bonds.

Table 3 provides additional insight regarding the findings reported in Fan et al. [53], where the four treatment groups, control, AE, PE, and DCR, are modeled as three binary indicators, and lipid-by-treatment interactions are represented as group-level interactions. In contrast, in this study, treatment is modeled as a single covariate, with G × E interactions represented by individual predictors. The modeling strategy used in this study facilitates FDR-assisted variable selection, as both main effects and interaction effects are treated equally as individual predictors in our model. Due to the difference in modeling environmental factors, we do not expect all findings are identical from both studies. However, we believe that main or interaction effects overlapping between the two studies warrant particular attention, as they are strongly associated with the longitudinal response regardless of whether treatment is modeled as a group or a categorical predictor. The interaction between C18:2/20:4 and environmental factor has a PIP of 0.9248, which is consistent with the group-level modeling of treatment in Fan et al. (2025) [53], where group-level G × E involving C18:2/20:4 has also been reported using the median thresholding rule [45].

The phospholipids identified by RBLSS with high posterior inclusion probabilities for main effects, C18:2/16:1 (PIP = 0.9597) and C18:2/18:1 (PIP = 0.9792), are broadly involved in lipid metabolism, membrane composition, and energy regulation. These processes are expected to vary across the four treatment groups in our study. Phospholipids containing linoleic acid (C18:2) and related acyl chains are linked to pathways responsive to changes in energy intake and physical activity, including shifts in membrane remodeling and metabolic signaling [59,60]. Thus, the fact that RBLSS selects C18:2/16:1 and C18:2/18:1 as the top main effects is consistent with broad biological expectations about lipids that differentiate overall metabolic states across these treatment conditions.

In addition, RBLSS identifies several lipid–treatment interaction effects with large PIPs, including C22:7/16:0 (0.9087), C16:0/18:1 (0.9501), C18:3/18:1 (1.0000), and C18:2/20:4 (0.9248). These interactions indicate that the associations between these lipid species and the weight differ across the four groups, rather than being constant shifts. Long-chain and polyunsaturated species such as those involving C18:3, C20:4, and C22:7 are known to participate in adaptive responses to exercise and caloric restriction, while saturated and monounsaturated chains like 16:0 and 18:1 are central to membrane and energy-storage dynamics [59,61]. While a full mechanistic explanation is beyond the scope of this paper, the fact that RBLSS detects plausible lipid main effects and lipid–treatment interactions under realistic experimental conditions indicates that the method effectively captures biologically meaningful structure in high-dimensional lipidolomics data.

We have listed significant findings using non-robust BLSS with the same FDR level of $\alpha$ = 0.05 in

Table 4. PIP of top-ranked main and interaction effects identified by BLSS with an FDR of $\alpha$ = 0.05.

Lipid	Main PIP	Interaction PIP
C20:6/16:0	0.9863	-
C18:3/18:1	-	1.0000
C18:2/20:4	-	0.9578

. In total, there are one important main effects (C20:6/16:0) and two important G×E interactions involving phospholipids C18:3/18:1 and C18:2/20:4, respectively. We notice that the total number of pinpointed significant effects under non-robust method is less than robust RBLSS. This pattern in the case study aligns with our simulation studies, which indicate that RBLSS outperforms BLSS by yielding a substantially larger number of significant findings.

4. Discussion

For hypothesis testing and variable selection, the two major frameworks of G × E analysis, their pros and cons carry over from cross-sectional to longitudinal studies. While penalized variable selection methods are generally more flexible to handle defining characteristics of G × E, such as hierarchical structures between main and interaction effects [20,62], resilience to outliers through robustification, and diverse forms of environmental factors (and thus G × E interactions) [4], statistical test-based approaches excel at providing rigorous inference, typically in terms of p-values, such as those in GWAS [32,63]. In this article, we propose to apply FDR-assisted procedures with robust Bayesian linear mixed model to prioritize important main and interaction effects in the presence of longitudinal traits with outliers, thereby offering a new approach to improve the rigor of variable selection–based interaction analysis. This method depends on previous work exploring FDR and Bayesian multiple comparisons rules [58] and studies alike [48,51]. Our numerical studies show that it is promising to couple FDR-assisted procedure with robust Bayesian interaction analysis using mixed-effect models.

Our study can be extended in the following aspects. First, the proposed interaction models respect a weak hierarchical structure without imposing shrinkage estimation on environmental effects [20,21]. To impose strong hierarchy, we conjecture that the strategy implemented or discussed in [20,30] can be generalized to sparse Bayesian mixed-effect models. Such a strategy applies heavier penalization on interaction effects to ensure that, whenever an interaction is selected, the corresponding genetic main effects are also included, thereby preserving strong hierarchy. Second, the current model assumes independence among main and interaction effects. It is intriguing to explore the performance in GWAS-alike settings where genetic features are highly correlated due to linkage disequilibrium. Benjamini and Yekutieli (2001) [64] suggest that if dependence among data is ignored, FDR-based approaches lead to inferior inference results. To better accommodate correlations among genetic predictors, we conjecture that conducting Bayesian FDR analysis after incorporating network information or Markov random field priors will further improve the model performance with correlated features [65]. Last but not least, our proposals also suffer from typical disadvantage of fully Bayesian analysis in that it is computationally intensive and not scalable to genome scale applications. Variational Bayesian methods, among others, can substantially accelerate computations and make the approach more suitable for large-scale genomics studies.