Bayesian Bootstrap in Multiple Frames

Abstract

Multiple frames are becoming increasingly relevant due to the spread of surveys conducted via registers. In this regard, estimators of population quantities have been proposed, including the multiplicity estimator. In all cases, variance estimation still remains a matter of debate. This paper explores the potential of Bayesian bootstrap techniques for computing such estimators. The suitability of the method, which is compared to the existing frequentist bootstrap, is shown by conducting a small-scale simulation study and a case study.

1. Introduction

For several decades, Multiple Frame (MF) surveys, introduced by Hartley [1,2], and related estimators [3,4,5,6] have been receiving increasing attention. For these estimators, incorrect frame membership attribution is a potential drawback. The multiplicity estimator [7], further developed by Singh and Mecatti [8] and involving the number of frames to which units belong, bypasses the problem.

The topic is still relevant in statistical practice, as witnessed by the recent contributions of Lohr and Raghunathan [9], Wu and Thompson [10], and Lohr [11] and by targeted works regarding, for instance, calibration in dual frames [12], inference in the case of ordinal data [13], kernel-based methods [14], and empirical likelihood estimation in dual frames [15]. Moreover, variance estimation still represents one of the main challenges for scholars and practitioners [16,17]. In the special case of dual frames, useful proposals have been elaborated [18,19,20] based on linearization and jackknife methods. Another option in dual and multiple frames consists in applying bootstrap methods [21,22].

Originally introduced by Efron [23], bootstrap has been widely applied in survey sampling for variance estimation and data imputation; overviews of bootstrap methods in survey sampling can be found in Shao [24] and Lahiri [25]. Regarding complex surveys, two important contributions are those of Rao and Wu [26] and Sitter [27], where the performance of bootstrap methods in survey sampling is compared to linearization and jackknife. Although most of the methods for variance estimation in survey sampling are based on the frequentist proposal, Bayesian bootstrap (BB) methods, originally introduced by Rubin [28], have also been developed: see Lo [29], who introduced them in finite populations and discussed the case of stratified samples, Aitkin [30] for an application in complex surveys, and Carota [31] for a discussion about the choice of priors.

This paper aims at exploring the potential of BB techniques in estimating the variance in multiple frame surveys. In particular, we develop a new BB-based algorithm, which allows the estimation of the variance of the multiplicity estimator. The main advantage of the BB algorithms is that they allow the estimated variance to be obtained without any evaluation of second-order inclusion probabilities. A related contribution can be found in Lohr [21], who proposes two frequentist bootstrap algorithms (named separate and combined, respectively) based on the Rao and Wu [26] rescaling technique. From a different perspective, Dong et al. [32] used BB with the aim of taking multiple complex surveys into account (without considering multiplicities), while Aidara [22] applied frequentist bootstrap (FB) in quasi-random sequences to estimate the variance of the multiplicity estimator [7].

The paper is organized as follows: Section 2 introduces the problem of variance estimation in multiple frames, while Section 3 illustrates the peculiarities of our non-parametric Bayesian proposal. A small-scale simulation study was performed in Section 4, while a case study appears in Section 5. Some concluding remarks are contained in the final section.

2. Multiple Frames and Variance Estimation

Multiple-frame sampling refers to surveys in which two or more frames are available and samples are drawn (usually independently) from each frame. This solution is preferred over the single sampling frame approach whenever a coverage improvement is needed, e.g., for dealing with elusive populations or for cost reduction purposes. The simplest case with two frames (A and B) is depicted in Figure 1.

More generally, the situation can be expounded as follows: let $U_1,\dots ,U_q,\dots ,U_{r}eal-timeQ$ be a collection of $Q\ge 2$ frames. The sample data collected from a generic frame $U_q$ can be classified into $D_q$ disjoint domains $U_{1\left(q\right)},\dots ,U_{d\left(q\right)},\dots ,U_{\left(D_q\left(q\right)\right)}$. The potential number of non-empty domains allowed for each frame is defined as $D_q=2^{(Q-1)}$.

Table 1. Partition of multi-frame samples into frame-specific domains: example with $Q=3$: $\{\mathrm{A},\mathrm{B},\mathrm{C}\}$.

Frame A	Frame B	Frame C
a(A)	b(B)	c(C)
ab(A)	ab(B)	ac(C)
ac(A)	bc(B)	bc(C)
abc(A)	abc(B)	abc(C)

reports the case of $Q=3$ frames: $\{\mathrm{A},\mathrm{B},\mathrm{C}\}$ where four domains can be identified in each frame. For instance, for frame A, the domains $D_A$ are as follows: $\{\mathrm{a}\left(\mathrm{A}\right),\mathrm{ab}\left(\mathrm{A}\right),\mathrm{ac}\left(\mathrm{A}\right),\mathrm{abc}\left(\mathrm{A}\right)\}$.

The population is completely covered by two or more frames that may be overlapping, and this multiplicity should be taken into account when proposing an estimator. A relevant discussion can be found in Lohr and Rao [16]. A solution is proposed by Mecatti [7] and included in Singh and Mecatti [8] and Mecatti and Singh [33]: the multiplicity estimator for the total Y has the following form:   

$$\widehat{Y}=\sum _{q=1}^Q\sum _{k\in s_q}\frac{y_k}{m_k{\pi }_k},$$

(1)

where $s_q$ is a sample extracted from frame $U_q$ under a given sampling design, $y_k$ represents the individual target characteristic of interest, $m_k$ is the multiplicity factor (i.e., the number of frames in which a given individual is sampled), and ${\pi }_k$ is the first order inclusion probability for the k-th primary sampling unit (psu). Estimator (1) does not need to assign the k-th unit to any $U_q$. It belongs to the Horvitz–Thompson (HT) class, for which a closed-form for the variance can be computed, which depends on second-order inclusion probabilities. In fact, following Singh and Mecatti [8] and Mecatti and Singh [33], the variance of (1) is as follows:

$$V\left(\widehat{Y}\right)=\sum _{q=1}^Q\left[\sum _{k\in U_q}{y}_k^2{m}_k^{-2}\frac{1-{\pi }_{k\left(q\right)}}{{\pi }_{k\left(q\right)}}+\sum _{k\ne k^{\prime }}\sum _{\in U_q}\frac{y_k{m}_k^{-1}{y}_{k^{\prime }}{m}_{k^{\prime }}^{-1}}{{\pi }_{k\left(q\right)}{\pi }_{k^{\prime }\left(q\right)}}({\pi }_{k{k}^{\prime }\left(q\right)}-{\pi }_{k\left(q\right)}{\pi }_{k^{\prime }\left(q\right)})\right],$$

(2)

where, for each frame, all first-order (${\pi }_{k\left(q\right)}$) and second-order (${\pi }_{k{k}^{\prime }\left(q\right)}$) inclusion probabilities must be specified. In the case of simple random sampling from each frame, the variance of (1) reduces to the following:

$$V\left(\widehat{Y}\right)=\sum _{q=1}^Q\frac{N_q-n_q}{n_q(N_q-1)}\left[N_q\sum _{k\in U_q}{y}_k^2{m}_k^{-2}-{\left(\sum _{k\in U_q}{y}_k{m}_k^{-1}\right)}^2\right],$$

(3)

where $N_q$ and $n_q$ are the population and sample size of frame q, respectively.

2.1. Variance Estimation for the Multiplicity Estimator

The Sen–Yates–Grundy estimator of (2) is as follows [33]:

$$\widehat{V}\left(\widehat{Y}\right)=\frac{1}{2}\sum _{q=1}^Q\sum _{k\ne k^{\prime }}\sum _{k\in s_q}\frac{{\pi }_{k^{\prime }\left(q\right)}{\pi }_{k^{\prime }\left(q\right)}-{\pi }_{k{k}^{\prime }\left(q\right)}}{{\pi }_{k{k}^{\prime }\left(q\right)}}{\left(\frac{y_k{m}_k^{-1}}{{\pi }_{k\left(q\right)}}-\frac{y_{k^{\prime }}{m}_{k^{\prime }}^{-1}}{{\pi }_{k^{\prime }\left(q\right)}}\right)}^2.$$

(4)

In cases of simple random sampling of each frame, the estimator of (3) is [7].

$$\widehat{V}\left(\widehat{Y}\right)=\sum _{q=1}^Q\frac{N_q(N_q-n_q)}{n_q^2(N_q-1)}\left[N_q\sum _{k\in s_q}{y}_k^2{m}_k^{-2}-\frac{N_q}{n_q}{\left(\sum _{k\in s_q}{y}_k{m}_k^{-1}\right)}^2\right].$$

(5)

Estimator (4) needs the first- and second-order inclusion probabilities of the sampled units. The last quantities are unknown and not trivial to estimate, especially in complex surveys.

When second-order inclusion probabilities are not available, and non-linear methods to estimate the variance are required, resampling techniques can be used to obtain an estimate of (2). The most used resampling methods for variance estimation in survey sampling [34] are the balanced repeated replications [35], the jackknife [36], and the bootstrap [25]. The jackknife has been introduced by Lohr and Rao [17,19] in dual frames, and it is further developed by Lohr and Rao [16] in multiple frames. In addition, jackknife has been recently proposed to estimate variance in the case of ordinal data in multiple frames [13].

2.2. Frequentist Bootstrap for Variance Estimation

The frequentist bootstrap (FB) is currently applied to variance estimation in (complex) survey sampling [26,27,37]. See Mashreghi et al. [38] for a general overview. In multiple frames, the technique is introduced by Lohr [21] using the rescaling bootstrap of Rao and Wu [26] and developed by [39] to obtain confidence intervals in the case of pseudo-empirical likelihood-based estimator. Two possible procedures can be carried out in this regard. The former jointly resamples psus from all available frames. As per the latter, an algorithm is implemented to resample from each frame separately. In this case, a different number of iterations may be set in each frame. Regarding the multiplicity estimator (1), Aidara [22] applies the algorithm of [21] in a three-frame context using quasi Monte Carlo methods to improve bootstrap convergence.

In agreement with the mentioned literature, Algorithm 1 below summarizes the general procedure, sketched, in the two-frame case, by Lohr [21]. In a given frame q, for each bootstrap iteration $(n_{h\left(q\right)}-1)$, psus are sampled from stratum h in frame q. Defining $x_{h\left(q\right)k}\left(b\right)$ as how many times the psu k in the stratum h is drawn at the b-th bootstrap iteration, the sampling weight $w_{h\left(q\right)k}$ is scaled according to the following scheme.

$$w_{h\left(q\right)k}\left(b\right)=w_{h\left(q\right)k}\frac{n_{h\left(q\right)}}{n_{h\left(q\right)}-1}{x}_{h\left(q\right)k}\left(b\right).$$

(6)

Algorithm 1 Frequentist bootstrap

for each frame q do

for each bootstrap iteration b do

for each stratum $h\left(q\right)$ do

(a) generate a synthetic sample $s_{h\left(q\right)}^{*}$ of size $n_{h\left(q\right)}-1$ using SRSWR

(b) adjust unit-specific sampling weights using Equation (6)

end for

estimate population total using the q-th row of Equation (7)

end for

estimate bootstrap variance of the frame using (8)

end for

aggregate frame-specific variances (9)

Similarly to the jackknife technique, the variance estimator can be expressed as a function of weights in (6) as follows. Indeed, for a given $\widehat{\tau }$ estimator of interest, for each iteration b, the following Q-elements vector is constructed:

$$\begin{array}{c}{\widehat{\tau }}_{\left(1\right)}^{*}\left(b\right)=g(w_{\left(1\right)}\left(b\right),w_{\left(2\right)},\dots ,w_{\left(Q\right)})\\ \cdots \\ {\widehat{\tau }}_{\left(q\right)}^{*}\left(b\right)=g(w_{\left(1\right)},\dots ,w_{\left(q\right)}\left(b\right),\dots ,w_{\left(Q\right)})\\ \cdots \\ {\widehat{\tau }}_{\left(Q\right)}^{*}\left(b\right)=g(w_{\left(1\right)},w_{\left(2\right)},\dots ,w_{\left(Q\right)}\left(b\right))\end{array}$$

(7)

where g(.) is a duly specified function.

Assuming that g(.) has the functional form (1), for each frame, a bootstrap-based variance estimator can be computed as follows:

$${\widehat{V}}_{\left(q\right)}^{*}\left(\widehat{Y}\right)=\frac{1}{B_q}\sum _{b=1}^{B_q}{({\widehat{\tau }}_{\left(q\right)}^{*}\left(b\right)-\widehat{Y})}^2.$$

(8)

where $B_q$ is the total number of bootstrap iterations, which can be different for each frame. Finally, the variance estimator of (1) is obtained bt aggregating separate estimators (8):

$$\begin{array}{c}{\widehat{V}}^{*}\left(\widehat{Y}\right)=\sum _{q=1}^Q{\widehat{V}}_{\left(q\right)}^{*}\left(\widehat{Y}\right)\\ \end{array}$$

(9)

3. Bayesian Bootstrap in Multiple Frames

In what follows, we propose a non-parametric Bayesian approach for variance estimation in multi-frame sampling designs. Bayesian bootstrap (BB) constitutes an additional method to approach variance estimation based on resampling and is a little explored opportunity in the case of multiple frame surveys. While the classical bootstrap resamples from the observed sampled values (in the “naive” case), BB starts from the posterior distribution of the sampled units [28]. It was introduced by Lo [29] for survey sampling in finite populations and, more recently, discussed by Aitkin [30] and Carota [31]. In case of multiple surveys, contributions can be found in Dong et al. [32,40].

Broadly speaking, BB can be defined in terms of the Dirichlet–Multinomial compound model. Let ${\left\{y\right\}}_1^N$ be the values of a characteristic attributed to an exchangeable population, where $J\le N$ includes the values that can be sampled according to a vector of probabilities $\mathit{\theta }=({\theta }_1,{\theta }_2,\dots ,{\theta }_J)$. In the simplest case, the prior distributions for the parameters $\theta$ are assumed to be uniformly distributed, i.e., flat priors [41]. We assume that $\mathit{\theta }\sim F\left(\mathit{\alpha }\right)$, where $F\left(\mathit{\alpha }\right)$ is the Dirichlet prior distribution over the parameter of the population generating process, as follows.

$$\left\{\begin{array}{c}{y}_i|\mathit{\theta }\sim Multinomial({\theta }_1,{\theta }_2,\dots ,{\theta }_J)\\ \mathit{\theta }|\mathit{\alpha }\sim Dirichlet({\alpha }_1,{\alpha }_2,\dots ,{\alpha }_J)\end{array}\right.$$

(10)

Thus, according to the Bayes theorem, the posterior predictive distribution can be derived [32] as follows:

$$\begin{array}{ccc}\hfill p\left(Y\right|y)& =& \int \frac{p(Y,\mathit{\theta },y)}{p\left(y\right)}d\mathit{\theta }\hfill \\ & =& \frac{{\int }_0^1\dots {\int }_0^{1}p\left(Y\right|y,\mathit{\theta })p\left(y\right|\mathit{\theta })p\left(\mathit{\theta }\right)d{\theta }_1\dots d{\theta }_J}{{\int }_0^1\dots {\int }_0^{1}p\left(y\right|\mathit{\theta })p\left(\mathit{\theta }\right)d{\theta }_1\dots d{\theta }_J}\hfill \\ & =& \frac{{\int }_0^1\dots {\int }_0^{1}p\left(Y\right|y,\mathit{\theta })p\left(y\right|\mathit{\theta })p\left(\theta \right)d{\theta }_1\dots d{\theta }_J}{{\int }_0^1\dots {\int }_0^{1}p\left(y\right|\mathit{\theta })p\left(\mathit{\theta }\right)d{\theta }_1\dots d{\theta }_J}\hfill \\ & =& \frac{{\int }_0^1\dots {\int }_0^1{\prod }_{j=1}^J{\theta }_j^{N_j-n_j}{\prod }_{j=1}^J{\theta }_j^{n_j}{\prod }_{j=1}^J{\theta }_j^{{\alpha }_j-1}{\theta }_1\dots d{\theta }_J}{{\int }_0^1\dots {\int }_0^1{\prod }_{j=1}^J{\theta }_j^{n_j}{\prod }_{j=1}^J{\theta }_i^{{\alpha }_j-1}d{\theta }_1\dots d{\theta }_J}\hfill \\ & =& \frac{{\prod }_{j=1}^J\mathrm{\Gamma }(N_j+{\alpha }_j)/\mathrm{\Gamma }\left({\alpha }_j\right)}{\mathrm{\Gamma }(N+{\alpha }_0)/\mathrm{\Gamma }\left({\alpha }_0\right)}{\left(\frac{{\prod }_{j=1}^J\mathrm{\Gamma }(n_j+{\alpha }_j)}{\mathrm{\Gamma }\left(n+{\alpha }_0\right)}\right)}^{-1},\hfill \end{array}$$

(11)

where $N={\sum }_{j=1}^J{N}_j$ and $n={\sum }_{j=1}^J{n}_j$ represent the total number of elements in the population and in the sample, respectively. In addition, ${\alpha }_0={\sum }_{j=1}^J{\alpha }_j$ and $\mathrm{\Gamma }(.)$ stands for the Gamma function.

Resampling from the posterior predictive (12) is not trivial at all. Consequently, the suggestion for practical implementation is to leverage Pólya’s urn scheme to simulate such a distribution [29]. In a nutshell, the Pólya’s urn scheme contains J values ${\alpha }_j$ with $j=1,\cdots ,J$. Initially, a value j is randomly sampled from the urn and reinserted together with another value of the same type. After a sufficiently large number of iterations, the distribution converges to $F\left(\mathit{\alpha }\right)\sim Dirichlet({\alpha }_1,\dots ,{\alpha }_J)$ [42].

The Proposed Algorithm

As mentioned above, BB relies on generating a Dirichlet-based posterior predictive distribution via Pólya’s urn scheme. For a single frame and non-complex sampling, the application of the BB is straightforward, but the presence of multiple frames and multi-strata sampling designs [29] induces a further degree of complexity that must be considered.

Our proposal in this regard is summarized in Algorithm 2. The generation of a synthetic population represents its core, starting from generating synthetic samples. A number of $(N-n)$ elements is generated using Pólya’s urn scheme. In the generic synthetic sample, we can identify $(N_j-n_j)$ as the number of draws of units belonging to the same group. This corresponds to the draw from the Dirichlet-based posterior predictive distribution $p\left(Y\right|y)$ of Equation (12).

Algorithm 2 Bayesian bootstrap

for each frame q do

for each bootstrap iteration $b\left(q\right)$ do

for each stratum $h\left(q\right)$ do

(a) generate a synthetic sample $s_{h\left(q\right)}^{*}$ of size $(N_{h\left(q\right)}-n_{h\left(q\right)})$ using the

Pólya Urn model on the original sample $s_{h\left(q\right)}$

(b) construct $C_{h\left(q\right)}$ by concatenating the original sample $s_{h\left(q\right)}$ with $s_{h\left(q\right)}^{*}$ (12)

(c) $n_{h\left(q\right)}$-sized sampled is drawn from $C_{h\left(q\right)}$

(d) adjust unit-specific sampling weights using Equation (13)

end for

estimate population total using the q-th row of Equation (14)

end for

estimate bootstrap variance of the frame using Equation (15)

end for

aggregate frame-specific variances (16)

Therefore, in any frame q, for the variable of interest y in any stratum $h\left(q\right)$, the final bootstrapped population is obtained by concatenating the $n_{h\left(q\right)}$ units of the original sample and the $N_{h\left(q\right)}-n_{h\left(q\right)}$ bootstrapped units. Then, the BB-based population values are obtained for each $h\left(q\right)$ as follows:

$$\begin{array}{c}{C}_{h\left(q\right)}=\{y_1,\dots ,y_{n_{h\left(q\right)}}\}\cup \{y_1^{*},\dots ,y_{N_{h\left(q\right)}-n_{h\left(q\right)}}^{*}\}\hfill \end{array}$$

(12)

where $y^{*}$s represent values sampled from the Dirichlet-based posterior predictive distribution. The population in (12) is then used to resample $n_{h\left(q\right)}$ units, which constitute the frame and stratum-specific bootstrap sample. Distinct from the FB-based weights in (6), in BB, the weights are obtained as follows.

$$w_{h\left(q\right)k}^{BB}\left(b\right)=w_{h\left(q\right)k}{x}_{h\left(q\right)k}\left(b\right).$$

(13)

Then, for a given $\widehat{\tau }$ estimator, for each iteration b, the following Q-elements vector is constructed using weights: (13):

$$\begin{array}{c}{\widehat{\tau }}_{(1)}^{*BB}\left(b\right)=g(w_{\left(1\right)}^{BB}\left(b\right),w_{\left(2\right)},\dots ,w_{\left(Q\right)})\\ \cdots \\ {\widehat{\tau }}_{\left(q\right)}^{*BB}\left(b\right)=g(w_{\left(1\right)},\dots ,w_{\left(q\right)}^{BB}\left(b\right),\dots ,w_{\left(Q\right)})\\ \cdots \\ {\widehat{\tau }}_{\left(Q\right)}^{*BB}\left(b\right)=g(w_{\left(1\right)},w_{\left(2\right)},\dots ,w_{\left(Q\right)}^{BB}\left(b\right))\end{array}$$

(14)

where g(.) is the function of the previous paragraph in the frequentist case.

Assuming that g(.) has the functional form (1), for each frame, a BB-based variance estimator can be computed as follows.

$${\widehat{V}}_{\left(q\right)}^{*BB}\left(\widehat{Y}\right)=\frac{1}{B_q}\sum _{b=1}^{B_q}{({\widehat{\tau }}_{\left(q\right)}^{*BB}\left(b\right)-\widehat{Y})}^2.$$

(15)

Similarly to the FB case, the variance estimator can be obtained as follows.

$$\begin{array}{c}{\widehat{V}}^{*BB}\left(\widehat{Y}\right)={\displaystyle \sum _{q=1}^Q}{\widehat{V}}_{\left(q\right)}^{*BB}\left(\widehat{Y}\right)\end{array}$$

(16)

4. Simulation Study

In this section, a small-scale simulation study was performed to assess the proposed methodology via a comparison between BB and FB.

4.1. Set-Up

As a first Data Generating Process (DGP), we consider a three-frame design with simple random sampling in each frame (${\mathrm{DGP}}_1$). Following Mecatti, we generate M Monte Carlo pseudo-populations of $N=2400$ elements from a Gamma distribution with parameters (1.5, 2) such that the population total is $Y=7200$. Each element of the population is then randomly assigned to a frame via Bernoulli trials. We state two alternative expected values for each frame, $p=0.4$ and $p=0.6$, and ensure overlapping between frames and non-empty frames. Three possible sampling fractions ($f_q=n_q/N_q$) are considered: 0.05, 0.15, and 0.40.

Secondly, we consider a more complex sample design with stratification (DGP${}_2$). Two strata are generated according to $N_1=N_2=1200$; the individual values of the characteristic under study are sampled from the following.

-

A Gamma distribution with parameters (1.5, 2);

-

A Gamma distribution with parameters (2, 4).

The population total is now equal to $Y=$ 13,200, while the same Bernoulli trials (to construct frames) and sampling fractions of DGP${}_1$ are used.

The number of Monte Carlo simulations is set equal to $M=500$, and the number of boostrap replications is $B=399$. In agreement with previous studies [7,16,22], two performance indicators were used: Relative Bias (RB) and the Coefficient of Variation (CV). The RB is computed as follows:

$$RB=\frac{1}{M}\sum _{m=1}^M\frac{({\widehat{V}}_m^{*}-MSE)}{MSE}·100$$

(17)

where the ${\widehat{V}}_m^{*}$ is either the BB- or the FB-based variance estimate for the m-th sample, according to the theoretical choice performed. The expression for the CV is as follows.

$$CV=\frac{\sqrt{\frac{1}{M}{\sum }_{m=1}^M{({\widehat{V}}_m^{*}-MSE)}^2}}{MSE}$$

(18)

The reference MSE in (17) and (18) is computed via 10,000 Monte Carlo simulations as follows:

$$MSE=\frac{1}{10,000}\sum _{m=1}^{10,000}{({\widehat{Y}}_m-Y)}^2$$

where ${\widehat{Y}}_m$ is the estimate computed for the m-th synthetic sample.

4.2. Main Results

For DGP${}_1$,

Table 2. Performance indicators for DGP${}_1$ considering $p=0.4$.

	RB		CV
${\mathit{f}}_{\mathit{q}}$	BB	FB	BB	FB
0.05	−8.328	−6.198	0.288	0.279
0.15	−7.575	−6.459	0.173	0.168
0.40	−13.828	−11.546	0.166	0.150

and

Table 3. Performance indicators for DGP${}_1$ considering $p=0.6$.

	RB		CV
${\mathit{f}}_{\mathit{q}}$	BB	FB	BB	FB
0.05	−1.216	−3.541	0.316	0.312
0.15	−0.217	−0.095	0.196	0.181
0.40	−7.141	−5.935	0.137	0.130

illustrate the results about performance indicators, i.e., the RB (17) and the CV (18), with $p=0.4$ and $p=0.6$, respectively.

In terms of RB, both tables show how the BB and the FB have a similar counterintuitive behaviour, achieving the poorest results for the highest sampling fraction (0.4). Furthermore, in the case of $p=0.6$ (Table 3), a lower value of the indicator is associated with the BB at the lowest sampling fraction (0.05), while it performs poorly when the sampling fraction is higher. With an intermediate value of sampling fraction (0.15), still considering $p=0.6$, the performances in terms of RB are very satisfactory for both methods: the values are very close to zero.

As per the CV, the two methods show comparable results in both Table 2 and Table 3, even if FB slightly outperforms BB, with a decreasing behaviour of the indicator as the sampling fraction increases, either for $p=0.4$ or $p=0.6$.

The results for the more complex DGP${}_2$ witness the reliability of the BB with respect to the FB, as shown in

Table 4. Performance indicators for DGP${}_2$ with $p=0.4$.

	RB		CV
${\mathit{f}}_{\mathit{q}}$	BB	FB	BB	FB
0.05	−21.996	62.258	0.375	1.088
0.15	−12.827	56.689	0.236	0.755
0.40	−12.849	49.886	0.174	0.605

and

Table 5. Performance indicators for DGP${}_2$ with $p=0.6$.

	RB		CV
${\mathit{f}}_{\mathit{q}}$	BB	FB	BB	FB
0.05	−14.944	74.108	0.432	1.211
0.15	−2.479	68.636	0.266	0.848
0.40	−2.010	58.881	0.159	0.688

. In particular, FB severely overestimates the variability for each p and $f_q$, as denoted by the high positive values of its RB. Conversely, BB, which slightly underestimates the variability of the estimator, performs well, especially with $p=0.6$ (Table 5) and when the sampling fraction is not too small ($f_q=0.15$ and $f_q=0.40$). In terms of CV, even if both methods show a decreasing trend according to the increase in sampling fraction, BB definitively outperforms FB.

5. Case Study

To stress the advantages of using the BB in multiple frame surveys, the proposed algorithm is applied to the two-frame dataset included as a running example in the R package Frames2 [43]. Two households populations are considered with $N_A=1735$ and $N_B=1191$ and with an intersection of $N_{ab}=601$ such that $N=2325$. The first population is organized in $H=6$ strata with the following sizes: $N_{h\left(A\right)}=\{727,375,113,186,115,219\}$. Two samples are selected without replacement in the following manner:

-

$n_A=105$ by simple random sampling in each stratum:

$$n_{h\left(A\right)}=\{15,20,15,20,15,20\};$$

-

$n_B=135$ by simple random sampling.

The available variables are three types of expenditures: Feeding, Clothing, and Leisure (in Euros). The number of bootstrap replications is set equal to $B=999$.

Table 6. Resampling-based variance estimates (divided by ${10}^6$) for the case study.

Variable	FB	BB	(FB − BB)/FB
Feeding	348.24	280.12	19.56%
Clothing	6.33	5.40	14.74%
Leisure	2.50	1.88	24.52%

summarizes the main results where the estimated variances are divided by ${10}^6$.

Empirical results confirms that the BB-based variance estimates are lower than the competing ones computed with FB under stratified sampling in multiple frames. In particular, BB exhibits a relative percentage difference (with respect to FB) that ranges (approximately) between 15% and 25%.

6. Discussion and Conclusions

The novelty of the present paper is the proposal of a Bayesian non-parametric technique (BB) to estimate the variance of a multiple frame estimator for a parameter of interest, namely the population total. The BB is proposed to construct the first-order inclusion probabilities in a non-frequentist manner and without modifying design-based properties. BB is also compared with frequentist bootstrap (FB), suggested by [21] and applied by [22]. The motivation for using resampling methods in multiple frames is due to the fact that they do not require the estimation of second-order inclusion probabilities.

Results of a small-scale simulation study show that the BB and FB perform similarly under simple random sampling in each frame, with a slight advantage in favor of the FB except when the sampling fraction is very low. However, this result should be considered as a benchmark, since under simple random sampling in each frame, a closed-form for the variance estimator is currently available [8,33].

Under a more complex sampling design like stratification, FB becomes practically unusable, severely overestimating the variability of the estimator in the context of multiple frames. Few previous experiments of FB in multiple frames have been performed [21,22], but they are not directly comparable with our findings due to different DGPs as a starting point for the simulation studies. Possible issues related to the application of FB to stratified samples in multiple frames should be investigated in further studies even from a theoretical perspective. Conversely, BB exhibits satisfactory performance, especially when the sampling fraction is not very low. A case study also reveals the suitability of BB in the context of dual frames with stratification.

The results here presented need future investigation with more intensive Monte Carlo simulations, alternative Data Generating Processes, different population parameters, further estimators in the context of multiple frames in addition to (1), and other complex sampling design, e.g., cluster sampling. In addition, BB is insensitive to the choice of prior for totals or means; its sensitivity for non-linear estimators should be deepened using tools similar to those provided by Aitkin [30] and Carota [31]. Finally, a relevant advantage for scholars and practitioners would be the implementation of BB (and the FB) in (possibly open source) statistical software.