Nonparametric FBST for Validating Linear Models ^†

Abstract

In Bayesian analysis, testing for linearity requires placing a prior to the entire space of potential regression functions. This poses a problem for many standard tests, as assigning positive prior probability to such a hypothesis is challenging. The Full Bayesian Significance Test (FBST) sidesteps this issue, standing out for also being logically coherent and offering a measure of evidence against $H_0$, although its application to nonparametric settings is still limited. In this work, we use Gaussian process priors to derive FBST procedures that evaluate general linearity assumptions, such as testing the adherence of data and performing variable selection to linear models. We also make use of pragmatic hypotheses to verify if the data might be compatible with a linear model when factors such as measurement errors or utility judgments are accounted for. This contribution extends the theory of the FBST, allowing for its application in nonparametric settings and requiring, at most, simple optimization procedures to reach the desired conclusion.

1. Introduction

Although linear models are widespread in the scientific literature, their validity is rarely tested in its full complexity. Generally, linearity is tested as a particular case of a more general parametric model [1] or compared to a finite selection of models—each with their own prior specification—through measures such as the Deviance Information Criterion [2]. In actuality, testing the adherence of linear models to data requires (i) assigning a nonparametric prior to the set of regression functions and (ii) devising a procedure that highlights the evidence against the linear model hypothesis based on the data and the prior. Devising a test solely based on the posterior probability of the hypothesis in this case is seldom advised, as it imposes positive prior probability to the set of linear models when there are countless nonlinear functions arbitrarily close to any element of it.

The Full Bayesian Significance Test (FBST, [3]) is the testing framework used throughout this work. The FBST does not violate the likelihood principle, does not require setting positive prior probabilities to hypotheses, and provides a measure of evidence against $H_0$, along with other desirable characteristics. With the exceptions of Corrêa Filho [4] and Liu et al. [5], the FBST has not been applied to nonparametrics, still requiring new theoretical developments to systematically embrace such settings.

Bridging the gaps above, this paper provides a nonparametric FBST formulation that tests the adherence of linear models to data. By using a Gaussian Process (GP, [6]) as a prior to model the regression function, we propose FBST procedures that depend on whether the covariates’ domain $\mathcal{X}$ is finite or infinite. Furthermore, we lay out FBST procedures for hypotheses that include negligible deviations from $H_0$, known as pragmatic hypotheses [7], useful to evaluate if $H_0$ is approximately instead of precisely compatible with the data.

In Figure 1, we illustrate how the FBST operates when applied to $H_0$ and its pragmatic version, $Pg\left(H_0\right)$. For $\alpha \in (0,1)$, the posterior is used to obtain the $(1-\alpha )100\%$ Highest Posterior Density (HPD) region, the smallest credible region with probability $(1-\alpha )$ of containing the quantity of interest. The hypothesis is rejected if it does not intersect with the HPD. This procedure is meaningful even when $H_0$ is precise, that is, when $\mathbb{P}\left(H_0\right)=0$.

Even though our contribution makes exclusive use of the FBST, this does not imply that it is the only valid framework for the problem. While no other testing procedure as general as ours has been proposed, Mulder [8] uses Bayes factors to test if a single covariate may be nonlinearly related to the response variable and Lassance et al. [9] (Section 3.1) test the pragmatic version of the linear model hypothesis through its posterior probability.

This work is organized as follows. In Section 2, the required background knowledge is provided. Our findings are presented in Section 3, leading in Section 4 to an application that puts all the FBST procedures to use. Lastly, Section 5 describes how to enhance the FBST further and establishes potential future research. All proofs can be found in Appendix A.

2. Materials and Methods

2.1. Full Bayesian Significance Test (FBST)

The FBST is composed of three steps [3]. For $H_0:\theta \in {\Theta }_0\subset \Theta$, where $\Theta$ is the parameter space, these steps are as follows:

• Delimit the set of elements in $\Theta$ that are more likely than those in ${\Theta }_0$. That is, if $f\left(\theta \right|\mathcal{D})$ is the posterior density of $\theta$ given the data $\mathcal{D}$,

  $$T:=\left\{\theta \in \Theta :f\left(\theta \right|\mathcal{D})\ge \underset{\theta \in {\Theta }_0}{sup}f\left(\theta \right|\mathcal{D})\right\}.$$
• Obtain $\text{e-value}:=1-\mathbb{P}(\theta \in T|\mathcal{D})=1-{\int }_{T}f\left(\theta \right|\mathcal{D})d\theta ,$ the Bayesian evidence value.
• Reject $H_0$ if $\text{e-value}\le \alpha$ for a previously specified significance level $\alpha \in (0,1)$.

In this paper, we use a procedure equivalent to the FBST: reject $H_0$ if the Highest Posterior Density (HPD) region is such that $HPD\cap {\Theta }_0=\varnothing$. The HPD is the smallest region with posterior probability of $1-\alpha$, obtained by finding the value $f^{\ast }$ such that

$$\mathbb{P}(\theta \in HPD|\mathcal{D})=1-\alpha ,HPD:=\left\{\theta \in \Theta :f\left(\theta \right|\mathcal{D})\ge f^{\ast }\right\}.$$

When $\theta |\mathcal{D}$ is normally distributed, the HPD region is equivalent to the credible interval symmetric around the posterior mean. For its multivariate counterpart, $\mathit{\theta }|\mathcal{D}\sim N_k(\mathit{\mu },\Sigma )$, we have that ${\Sigma }^{-1/2}(\mathit{\theta }-\mathit{\mu })\sim N_k(\mathbf{0},\mathbb{I})$, and thus ${(\mathit{\theta }-\mathit{\mu })}^{\prime }{\Sigma }^{-1}(\mathit{\theta }-\mathit{\mu })\sim {\chi }_k^2$, where ${\chi }_k^2$ stands for the chi-squared distribution with k degrees of freedom. Therefore, if $q_{(1-\alpha )}(·)$ is the $(1-\alpha )$100% quantile function, the HPD is given by the following ellipsoid ([10], Result 4.7):

$$\{\mathit{\theta }\in {\mathbb{R}}^k:{(\mathit{\theta }-\mathit{\mu })}^{\prime }{\Sigma }^{-1}(\mathit{\theta }-\mathit{\mu })\le q_{(1-\alpha )}\left({\chi }_k^2\right)\}.$$

(1)

2.2. Gaussian Processes (GPs)

A GP is a nonparametric family of priors used to model functions in regression settings. The random function $g:\mathcal{X}\to \mathbb{R}$ behaves according to a GP if

$$g\left(\mathit{X}\right)\sim N\left(m\right(\mathit{X}),K(\mathit{X},\mathit{X}\left)\right),\forall \mathit{X}\subset \mathcal{X},$$

where $m(·)$ and $K(·,·)$, respectively, determine the mean and covariance of the process. When the response variable Y is such that $Y=g\left(\mathit{x}\right)+ϵ$ for $ϵ\sim N(0,{\sigma }^2)$, that is,

$$L(g,{\sigma }^2|\mathit{y},\mathit{X})={\left(2\pi {\sigma }^2\right)}^{-n/2}exp\left\{-{\displaystyle \frac{1}{2{\sigma }^2}}\sum _{i=1}^n{\left(y_i-g\left({\mathit{x}}_i\right)\right)}^2\right\},$$

then the GP is conjugate and its posterior is such that

$$\begin{array}{cc}\hfill g\left({\mathit{X}}^{\prime }\right)& |\mathit{y},\mathit{X},{\sigma }^2\sim N(\mu \left({\mathit{X}}^{\prime }\right),\Sigma ({\mathit{X}}^{\prime },{\mathit{X}}^{\prime })),\hfill \\ \hfill \mu \left({\mathit{X}}^{\prime }\right)& :=m\left({\mathit{X}}^{\prime }\right)+K(\mathit{X},{\mathit{X}}^{\prime }){(K(\mathit{X},\mathit{X})+{\sigma }^2\mathbb{I})}^{-1}(\mathit{y}-m\left(\mathit{X}\right)),\hfill \\ \hfill \Sigma ({\mathit{X}}^{\prime },{\mathit{X}}^{\prime })& :=K({\mathit{X}}^{\prime },{\mathit{X}}^{\prime })-K(\mathit{X},{\mathit{X}}^{\prime }){(K(\mathit{X},\mathit{X})+{\sigma }^2\mathbb{I})}^{-1}K({\mathit{X}}^{\prime },\mathit{X}).\hfill \end{array}$$

The choice of m, K, and ${\sigma }^2$ reflect positions on the mean, smoothness, and variation surrounding the GP. In Section 4, we use the specifics of the application to choose them. For more general settings, one may assume that the uncertainty of m and K is reducible to a finite number of parameters. Then, one can either set priors to such parameters directly [11] or plug point estimates for them based on the maximum partial likelihood [12].

Conditionally on ${\sigma }^2$, the HPD region of the GP can be analytically obtained for any finite set ${\mathit{X}}^{\prime }={({\mathit{x}}_1,{\mathit{x}}_2,\cdots ,{\mathit{x}}_m)}^{\prime }$. Since the marginals of the posterior GP are also normally distributed, Equation (1) entails that the $(1-\alpha )100\%$ HPD region for $g\left({\mathit{X}}^{\prime }\right)|\mathit{y},\mathit{X}$ is

$$\{h\in \mathbb{H}:{(h\left({\mathit{X}}^{\prime }\right)-\mu \left({\mathit{X}}^{\prime }\right))}^{\prime }\Sigma {({\mathit{X}}^{\prime },{\mathit{X}}^{\prime })}^{-1}(h\left({\mathit{X}}^{\prime }\right)-\mu \left({\mathit{X}}^{\prime }\right))\le q_{(1-\alpha )}\left({\chi }_m^2\right)\}.$$

(2)

It is also possible to obtain an HPD set for the GP without setting ${\mathit{X}}^{\prime }$. Let ${\mathbb{P}}_g$ and ${\mathbb{P}}_{g|\mathit{y},\mathit{X},{\sigma }^2}$, respectively, be the prior and posterior probability measures of the GP defined on a measurable space $(\mathcal{G},\mathbb{G})$. Hence,

$${\mathbb{P}}_{g|\mathit{y},\mathit{X},{\sigma }^2}\left(A\right)={\displaystyle \frac{{\int }_{A}L(g,{\sigma }^2|\mathit{y},\mathit{X})d{\mathbb{P}}_g\left(h\right)}{{\int }_{\mathcal{G}}L(g,{\sigma }^2|\mathit{y},\mathit{X})d{\mathbb{P}}_g\left(h\right)}},\forall A\subset \mathbb{G}.$$

Since ${\mathbb{P}}_{g|\mathit{y},\mathit{X},{\sigma }^2}\ll {\mathbb{P}}_g$, the Radon–Nikodym derivative of the GP for $h\in \mathcal{G}$ is such that

$${\displaystyle \frac{d{\mathbb{P}}_{g|\mathit{y},\mathit{X},{\sigma }^2}}{d{\mathbb{P}}_g}}\left(h\right)\propto exp\left(-{\displaystyle \frac{1}{2{\sigma }^2}}\sum _{i=1}^n{(y_i-h\left({\mathit{x}}_i\right))}^2\right)=exp\left(-{\displaystyle \frac{1}{2{\sigma }^2}}{(\mathit{y}-h\left(\mathit{X}\right))}^{\prime }(\mathit{y}-h\left(\mathit{X}\right))\right),$$

(3)

i.e., it suffices to evaluate h only on the values of $\mathit{X}$ in the sample. To account for repeated lines in $\mathit{X}$, let ${\mathit{X}}^{\ast }$ be the matrix with all unique observations from $\mathit{X}$ and $n^{\ast }$ be the number of lines of ${\mathit{X}}^{\ast }$. Defining $D_{n_{{\mathit{X}}^{\ast }}}$ as a diagonal matrix that counts how many times each $\mathit{x}\in {\mathit{X}}^{\ast }$ appears in $\mathit{X}$ and ${\overline{\mathit{y}}}_{{\mathit{X}}^{\ast }}$ as the vector of means of all elements of $\mathit{y}$ related to each $\mathit{x}$,

$${\displaystyle \frac{d{\mathbb{P}}_{g|\mathit{y},\mathit{X},{\sigma }^2}}{d{\mathbb{P}}_g}}\left(h\right)\propto exp\left(-{\displaystyle \frac{1}{2{\sigma }^2}}{(h\left({\mathit{X}}^{\ast }\right)-{\overline{\mathit{y}}}_{{\mathit{X}}^{\ast }})}^{\prime }{D}_{n_{{\mathit{X}}^{\ast }}}(h\left({\mathit{X}}^{\ast }\right)-{\overline{\mathit{y}}}_{{\mathit{X}}^{\ast }})\right).$$

(4)

Thus, for a constant $c_{\alpha }$, a Weighted Residual Sum of Squares (WRSS) defines the HPD:

$$HP{D}_{(1-\alpha )}=\left\{h\in \mathcal{G}:WRSS\left(h\right)\le c_{\alpha }\right\},WRSS\left(h\right):={(h\left({\mathit{X}}^{\ast }\right)-{\overline{\mathit{y}}}_{{\mathit{X}}^{\ast }})}^{\prime }{D}_{n_{{\mathit{X}}^{\ast }}}(h\left({\mathit{X}}^{\ast }\right)-{\overline{\mathit{y}}}_{{\mathit{X}}^{\ast }}).$$

(5)

2.3. Pragmatic Hypotheses

The pragmatic hypothesis enlarges $H_0$ to a set deemed as practically equivalent. The implementation uses the notion of negligible deviations from $H_0$. The degree to which the hypothesis is enlarged depends on the choice of a threshold $\epsilon$, and factors such as the scale of measurement errors or expert’s utility judgments could help set it (see Section 4 for a practical example and (Lassance et al. [9], Section 4) for suggestions). Formally, for a hypothesis space $\mathbb{H}$, let $d(·,·)$ be the dissimilarity function from which one can express how much of a departure from $H_0$ is reasonable. Then, the pragmatic hypothesis is given by

$$Pg(H_0,d,\epsilon ):=\bigcup _{h_0\in H_0}\left\{h\in \mathbb{H}:d(h_0,h)\le \epsilon \right\}=\left\{h\in \mathbb{H}:\underset{h_0\in H_0}{inf}d(h_0,h)\le \epsilon \right\},$$

(6)

that is, the pragmatic hypothesis contains all elements $h\in \mathbb{H}$ such that, for some element $h_0\in H_0$, $d(h_0,h)\le \epsilon$. In this work, we assume that $\mathbb{H}=\mathcal{G}$ is a space of functions of the type $h:\mathcal{X}\to \mathbb{R}$. Further specifications on $\mathbb{H}$ are presented in Section 3. When $d(·,·)$ and $\epsilon$ are implicit, we use $Pg\left(H_0\right)$ to denote the pragmatic hypothesis.

3. Results

Throughout this work, we use the modeling assumptions in Section 2.2 for the data $(\mathit{y},\mathit{X})$ and the regression function $g(·)$, and assume that the hypothesis of interest is

$$H_0:g\left(\mathit{x}\right)=\mathit{b}\left(\mathit{x}\right)\mathit{\beta },\forall \mathit{x}\in \mathcal{X},\mathit{\beta }\in {\mathbb{R}}^k,$$

(7)

where $\mathit{b}\left(\mathit{x}\right)=(b_1\left(\mathit{x}\right),b_2\left(\mathit{x}\right),\cdots ,b_k\left(\mathit{x}\right))\subset \mathbb{H}$ is a linearly independent set of linear functions and $\mathcal{X}$ is the covariates’ domain. The choice of $\mathit{b}$ determines the test performed, such as evaluating linear models ($\mathit{b}\left(\mathit{x}\right):=\mathit{x}$) or doing variable selection ($\mathit{b}\left(\mathit{x}\right):={\mathit{x}}_{-i}$).

Our findings are divided in two settings: those applicable to $H_0$ and those to $Pg\left(H_0\right)$. In both cases, we explore when $\mathcal{X}$ is a finite or an infinite set. The finite case provides a closed-form solution for the FBST of $H_0$ and a solution for the pragmatic hypothesis that requires a univariate optimization procedure. When $\mathcal{X}$ is infinite, testing $H_0$ or $Pg\left(H_0\right)$ also requires determining $c_{\alpha }$ in the HPD of Equation (5), which is achieved by noting that the $WRSS$ can be expressed as a linear combination of noncentral chi-squared random variables; therefore, $c_{\alpha }$ is the $(1-\alpha )$ quantile of a generalized chi-squared distribution [13].

Theorem 1 (FBST of the linear model hypothesis).

Let $H_0$ be the hypothesis in Equation (7) and $g(·)|\mathit{y}, \mathit{X}\sim GP(\mu (·),\Sigma (·,·))$. Then,

• When $\mathcal{X}$ is a finite set, the FBST does not reject $H_0$ if and only if

  $${\left(\mathit{b}\left(\mathcal{X}\right)\widehat{\mathit{\beta }}-\mu \left(\mathcal{X}\right)\right)}^{\prime }\Sigma {(\mathcal{X},\mathcal{X})}^{-1}\left(\mathit{b}\left(\mathcal{X}\right)\widehat{\mathit{\beta }}-\mu \left(\mathcal{X}\right)\right)\le q_{(1-\alpha )}\left({\chi }_{\left|\mathcal{X}\right|}^2\right),$$

  where $\widehat{\mathit{\beta }}={\left(\mathit{b}{\left(\mathcal{X}\right)}^{\prime }\Sigma {(\mathcal{X},\mathcal{X})}^{-1}\mathit{b}\left(\mathcal{X}\right)\right)}^{-1}\mathit{b}{\left(\mathcal{X}\right)}^{\prime }\Sigma {(\mathcal{X},\mathcal{X})}^{-1}\mu \left(\mathcal{X}\right)$ and $\left|\mathcal{X}\right|$ is the size of $\mathcal{X}$.
• When $\mathcal{X}$ is an infinite set, the FBST does not reject $H_0$ if and only if

  $$\begin{array}{cc}\hfill & {\overline{\mathit{y}}}_{{\mathit{X}}^{\ast }}^{\prime }\mathit{M}{\overline{\mathit{y}}}_{{\mathit{X}}^{\ast }}\le c_{\alpha },\mathit{M}=D_{n_{{\mathit{X}}^{\ast }}}-D_{n_{{\mathit{X}}^{\ast }}}\mathit{b}\left({\mathit{X}}^{\ast }\right){\left(\mathit{b}{\left({\mathit{X}}^{\ast }\right)}^{\prime }{D}_{n_{{\mathit{X}}^{\ast }}}\mathit{b}\left({\mathit{X}}^{\ast }\right)\right)}^{-1}\mathit{b}{\left({\mathit{X}}^{\ast }\right)}^{\prime }{D}_{n_{{\mathit{X}}^{\ast }}}.\hfill \end{array}$$

Before presenting the FBST for the pragmatic version of Equation (7), we specify $\mathbb{H}$ and provide the infimum when the dissimilarity function in Equation (6) is the $L^2$ distance in the probability space of $\mathit{X}$. The hypothesis space $\mathbb{H}$ is such that

$$h\in \mathbb{H}⟺{\mathbb{E}}_{\mathit{X}}\left(h^2\right)={\int }_{\mathcal{X}}h{\left(\mathit{x}\right)}^{2}d\mathbb{P}\left(\mathit{x}\right)<\infty .$$

(8)

As for the infimum, it is described in the following Lemma:

Lemma 1 (Infimum of the dissimilarity on the linear model set).

Let Equation (8) denote the hypothesis space and $H_0$ be the hypothesis in Equation (7). If $d(h_0,h):=\sqrt{{\mathbb{E}}_{\mathit{X}}\left[{(h_0-h)}^2\right]}$, then $d(H_0,h)=d(\mathit{b}\times {\tilde{\mathit{\beta }}}_h,h),\forall h\in \mathbb{H}$, where

$$\begin{array}{cc}\hfill {\tilde{\mathit{\beta }}}_h=A_{\mathit{b}}^{-1}\times {\mathit{h}}_{\mathit{b}},A_{\mathit{b}}& =\left(\begin{array}{cccc}\mathbb{E}\left[b_1^2\left(\mathit{X}\right)\right]& \mathbb{E}\left[b_2\left(\mathit{X}\right)b_1\left(\mathit{X}\right)\right]& \cdots & \mathbb{E}\left[b_k\left(\mathit{X}\right)b_1\left(\mathit{X}\right)\right]\\ \mathbb{E}\left[b_1\left(\mathit{X}\right)b_2\left(\mathit{X}\right)\right]& \mathbb{E}\left[b_2^2\left(\mathit{X}\right)\right]& \cdots & \mathbb{E}\left[b_k\left(\mathit{X}\right)b_2\left(\mathit{X}\right)\right]\\ ⋮& ⋮& \ddots & ⋮\\ \mathbb{E}\left[b_1\left(\mathit{X}\right)b_k\left(\mathit{X}\right)\right]& \mathbb{E}\left[b_2\left(\mathit{X}\right)b_k\left(\mathit{X}\right)\right]& \cdots & \mathbb{E}\left[b_k^2\left(\mathit{X}\right)\right]\end{array}\right),\hfill \\ \hfill {\mathit{h}}_{\mathit{b}}^{\prime }& =\left(\mathbb{E}\left[h\left(\mathit{X}\right)b_1\left(\mathit{X}\right)\right],\mathbb{E}\left[h\left(\mathit{X}\right)b_2\left(\mathit{X}\right)\right],\cdots ,\mathbb{E}\left[h\left(\mathit{X}\right)b_k\left(\mathit{X}\right)\right]\right).\hfill \end{array}$$

Theorem 2 (FBST of the pragmatic version of $H_0$).

Let $\mathbb{H}$ be given by Equation (8) and define $d(h_0,h):=\sqrt{{\mathbb{E}}_{\mathit{X}}\left[{(h_0-h)}^2\right]}$. Assume that ${\sum }_{\mathit{x}\in {\mathit{X}}^{\ast }}\mathbb{P}\left(\mathit{x}\right)>0$. Then,

• When $\mathcal{X}$ is a finite set, the FBST does not reject $Pg\left(H_0\right)$ if and only if

  $$\exists s\in (0,1):1-\mu {\left(\mathcal{X}\right)}^{\prime }\left({\displaystyle \frac{{\epsilon }^2}{1-s}}{\mathit{N}}^{-1}+{\displaystyle \frac{1}{s}}\Sigma (\mathcal{X},\mathcal{X})q_{(1-\alpha )}\left({\chi }_{\left|\mathcal{X}\right|}^2\right)\right)\mu \left(\mathcal{X}\right)<0,$$

  where $\mathit{N}:=D_{\mathbb{P}\left(\mathcal{X}\right)}\left[\mathbb{I}-\mathit{b}\left(\mathcal{X}\right){\left(\mathit{b}{\left(\mathcal{X}\right)}^{\prime }{D}_{\mathbb{P}\left(\mathcal{X}\right)}\mathit{b}\left(\mathcal{X}\right)\right)}^{-1}\mathit{b}{\left(\mathcal{X}\right)}^{\prime }{D}_{\mathbb{P}\left(\mathcal{X}\right)}\right]$ and $D_{\mathbb{P}\left(\mathcal{X}\right)}$ is a diagonal matrix formed by the vector $\mathbb{P}\left(\mathit{x}\right),\mathit{x}\in \mathcal{X}$.
• When $\mathcal{X}$ is an infinite set, the FBST does not reject $Pg\left(H_0\right)$ if and only if

  $$\exists s\in (0,1):1-{\overline{\mathit{y}}}_{{\mathit{X}}^{\ast }}^{\prime }\left({\displaystyle \frac{{\epsilon }^2}{1-s}}{\mathit{M}}^{-1}+{\displaystyle \frac{1}{s}}{D}_{n_{{\mathit{X}}^{\ast }}}{c}_{\alpha }\right){\overline{\mathit{y}}}_{{\mathit{X}}^{\ast }}<0,$$

  where $\mathit{M}:=D_{\mathbb{P}\left({\mathit{X}}^{\ast }\right)}\left[\mathbb{I}-\mathit{b}\left({\mathit{X}}^{\ast }\right){\left(\mathit{b}{\left({\mathit{X}}^{\ast }\right)}^{\prime }{D}_{\mathbb{P}\left({\mathit{X}}^{\ast }\right)}\mathit{b}\left({\mathit{X}}^{\ast }\right)\right)}^{-1}\mathit{b}{\left({\mathit{X}}^{\ast }\right)}^{\prime }{D}_{\mathbb{P}\left({\mathit{X}}^{\ast }\right)}\right]$ and $D_{\mathbb{P}\left({\mathit{X}}^{\ast }\right)}$ is a diagonal matrix formed by the vector $\mathbb{P}\left(\mathit{x}\right),\mathit{x}\in {\mathit{X}}^{\ast }$.

In the infinite case of $\mathcal{X}$ in Theorem 2, an appealing choice for the distribution of $\mathit{X}$ is based on the posterior of a Dirichlet process [14]. For a concentration parameter $\tau$ and a centering distribution $\pi$, set a Dirichlet process $P\sim DP(\tau ,\pi )$ such that $\mathit{X}|P\sim P$. Then,

$$P|\mathit{x}\sim DP\left({\displaystyle \frac{\tau \pi }{\tau +n}}+{\displaystyle \frac{{\sum }_{i=1}^n{\delta }_{{\mathit{x}}_i}}{\tau +n}},\tau +n\right)⟹\mathbb{P}({\mathit{X}}_{new}\in A|\mathit{X}=\mathit{x})={\displaystyle \frac{\tau \pi \left(A\right)}{\tau +n}}+{\displaystyle \frac{{\sum }_{i=1}^n\mathbb{I}({\mathit{x}}_i\in A)}{\tau +n}},$$

where ${\delta }_{{\mathit{x}}_i}=\mathbb{I}({\mathit{x}}_i\in A)$ for $A\subseteq \mathcal{X}$. With this choice, one can ensure that positive probability will always be assigned to all $\mathit{x}\in {\mathit{X}}^{\ast }$. Moreover, $\tau$ can leverage the weight of the prior on the FBST, with higher values of $\tau$ leading to a higher chance of not rejecting $Pg\left(H_0\right)$.

4. Application: Water Droplet Experiment

The dataset from Duguid [15] provides a setting where small water droplets (ranging from 3 to 9 μm) are free falling through a tube that keeps factors such as temperature and humidity constant. As a droplet falls, a camera takes a picture at every 0.5 s, ceasing activity after 7 s. One of the objectives of the study was to evaluate Fick’s law, which in this setting implies that—when time is a covariate—the decrease in radius of the droplet can be described through a linear model. The two hypotheses of interest are

$$\left\{\begin{array}{c}{H}_0^1:g\left(t\right)={\mathit{\beta }}_0+{\mathit{\beta }}_{1}t,\forall t\in \{0s,0.5s,\cdots ,7s\},({\mathit{\beta }}_0,{\mathit{\beta }}_1)\in {\mathbb{R}}^2;\hfill \\ H_0^2:g\left(t\right)={\mathit{\beta }}_0,\forall t\in \{0s,0.5s,\cdots ,7s\},{\mathit{\beta }}_0\in \mathbb{R};\hfill \end{array}\right.$$

with the first hypothesis testing the validity of Fick’s law for this case and the second one verifying if time can be removed as a covariate.

We use the GP in Section 2.2 to model the data, with the following prior settings:

$${\sigma }^2=0.01,m\left(t\right)={\displaystyle \frac{3+9}{2}}=6,\forall t\in \mathcal{X},K(t_1,t_2)=exp\left\{-{\displaystyle \frac{1}{2}}\left|\right|t_1-t_2{\left|\right|}_2\right\},(t_1,t_2)\subset \mathcal{X}.$$

As shown in Figure 2a, this choice leads to functions that obey the 3–9 $\mathsf{\mu }$m restriction without becoming too restrictive as a consequence. In Figure 2b, we observe that the posterior draws resemble a linear model except on $t=0$, due to the missing observation.

In

Table 1. e-value of $H_0$ under finite and infinite $\mathcal{X}$ for the water droplet experiment.

	Hypothesis
Assumption on $\mathit{t}$	${\mathit{H}}_{\mathbf{0}}^{\mathbf{1}}:\mathit{g}\left(\mathit{t}\right)={\mathit{\beta }}_{\mathbf{0}}+{\mathit{\beta }}_{\mathbf{1}}\mathit{t}$	${\mathit{H}}_{\mathbf{0}}^{\mathbf{2}}:\mathit{g}\left(\mathit{t}\right)={\mathit{\beta }}_{\mathbf{0}}$
t is discrete and finite	0.0446	0
t is continuous	0.0068	0

, we present the e-value for both hypotheses of interest, assuming that $\mathcal{X}$ is either finite or infinite. Since small e-values provide strong evidence against $H_0$, with $\alpha =0.05$ we conclude that both $H_0^1$ and $H_0^2$ should be rejected, i.e., Fick’s law would fail.

While this analysis shows that Fick’s law is not exactly valid, it might still provide an adequate approximation, motivating the use of pragmatic hypotheses. This requires setting the threshold $\epsilon$, which is detailed below.

In the original experiment, the radius of the droplets was obtained indirectly through Stoke’s law, that is,

$$V_T\left(t\right)={\displaystyle \frac{g{\left(t\right)}^2}{K_s}}⟹g\left(t\right)=\sqrt{V_T\left(t\right)\times K_s},$$

(9)

where $V_T$ is the terminal velocity and $K_s=8.446$. Since the mean velocity ($V_M$) was used in (9) instead of $V_T$, there are two sources of measurement error: the estimate of $V_M$ (maximum error of $\delta =0.14$) and switching $V_T$ for $V_M$ in (9) (maximum error of $\eta =0.3555$ ([9], Example 1.3)). We conclude that the margin of error of the radius is

$$\begin{array}{cc}\hfill ϵ:& =\underset{t\in T}{max}|g\left(t\right)-y\left(t\right)|=\underset{t\in T}{max}\left\{\left|\sqrt{K_s{V}_T\left(t\right)}-y\left(t\right)\right|\right\}\hfill \\ \hfill & =\underset{t\in T}{max}\left\{\left|\sqrt{K_s(V_M\left(t\right)-\delta -\eta )}-y\left(t\right)\right|,\left|\sqrt{K_s(V_M\left(t\right)+\delta +\eta )}-y\left(t\right)\right|\right\}\approx 0.6218.\hfill \end{array}$$

While $ϵ$ relates to the $l_{\infty }$ distance, Lemma 1 uses the $l_2$ distance. To obtain an estimate of the latter from the former, we use Proposition 6.11 of Folland [16], which implies that

$$\sqrt{{\displaystyle \frac{1}{n}}\sum _{i=1}^n{(y\left({\mathit{x}}_i\right)-g\left({\mathit{x}}_i\right))}^2}\le \sqrt{{\displaystyle \frac{1}{n}}}ϵ⟹\underset{i\in \{1,2,\cdots ,n\}}{max}|y\left({\mathit{x}}_i\right)-g\left({\mathit{x}}_i\right)|\le ϵ,$$

thus $\epsilon \approx 0.6218/\sqrt{15}\approx 0.1606$.

Table 2. e-value of $Pg(H_0,d,0.1606)$ under finite and infinite $\mathcal{X}$ for the water droplet experiment.

	Original Hypothesis
Assumption on $\mathit{t}$	${\mathit{H}}_{\mathbf{0}}^{\mathbf{1}}:\mathit{g}\left(\mathit{t}\right)={\mathit{\beta }}_{\mathbf{0}}+{\mathit{\beta }}_{\mathbf{1}}\mathit{t}$	${\mathit{H}}_{\mathbf{0}}^{\mathbf{2}}:\mathit{g}\left(\mathit{t}\right)={\mathit{\beta }}_{\mathbf{0}}$
$t\in \{0,0.5,1,\cdots ,7\}$	1	0
$t|P\sim P,P\sim DP(1,U(0,7))$	1	0

presents the e-values for the pragmatic hypotheses $Pg(H_0^i,d,0.1606)$, $i\in \{1,2\}$. We assume either that $\mathcal{X}=\{0, 0.5, 1, \cdots , 7\}$ (original setting, discrete uniform) or that $t|P\sim P$, with $P\sim DP(1,U(0,7\left)\right)$ (continuous uniform as centering distribution). Contrary to Table 1, the first hypothesis is not rejected, demonstrating that Fick’s law provides a good approximation of the phenomenon.

5. Discussion

Regarding the results of the application (Section 4), we believe to have demonstrated the importance of using pragmatic hypotheses whenever reasonable. While choosing $\epsilon$ is not a simple task in nonparametric settings, there are strategies available for deriving it [9]. Furthermore, while the e-value is not a measure of evidence against $H_0^c$ [3], combining it with a pragmatic hypothesis allows one to perform the Generalized FBST (GFBST, [17]), which can discriminate “evidence of absence” from “absence of evidence”, along with many other desirable properties.

One of the main limitations of this work is in the strategy of performing variable selection. While the aforementioned GFBST allows for multiple testing without the necessity of correcting $\alpha$, variable selection is only possible through Equation (7) if the linear model hypothesis is not rejected. Therefore, one future research direction is developing tests that evaluate conditional independence without assuming a specific functional form for the relationship between variables.