# Sample Size Requirements for Calibrated Approximate Credible Intervals for Proportions in Clinical Trials

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

## 2. Methodology

#### 2.1. Exact and Approximate Intervals

#### 2.2. A Measure of Discrepancy and Predictive Analysis

- (i)
- draw N samples ${{\mathit{x}}_{\mathit{n}}}^{\left(1\right)},\dots ,{{\mathit{x}}_{\mathit{n}}}^{\left(N\right)}$ from ${f}_{n}(\xb7;{\theta}_{d})$;
- (ii)
- compute $\tilde{\ell}\left({{\mathit{x}}_{\mathit{n}}}^{\left(j\right)}\right)$ and $\tilde{u}\left({{\mathit{x}}_{\mathit{n}}}^{\left(j\right)}\right)$, for $j=1,\dots ,N$;
- (iii)
- compute $P\left({{\mathit{x}}_{\mathit{n}}}^{\left(j\right)}\right)$, for $j=1,\dots ,N$;
- (iv)
- set ${e}_{n}^{P}\simeq \frac{{\sum}_{j=1}^{N}P\left({{\mathit{x}}_{\mathit{n}}}^{\left(j\right)}\right)}{N}$;with a large number of draws, e.g., $N=10000$.

## 3. Examples: The Beta-Binomial Model

#### 3.1. Credible Intervals for a Proportion

`hdi()`function of the

`HDInterval`package of

`R`, [27], which simply requires the

`R`function

`qbeta()`in input. Conversely, closed-form expressions for approximate intervals are easily obtained as follows. Recalling that $\widehat{\theta}={\overline{x}}_{n}$ and ${I}_{n}\left(\theta \right)=\frac{n}{\theta (1-\theta )}$, from Equation (3) the bounds of the likelihood approximate interval are

#### 3.2. Credible Intervals for the Log-Odds

- (i)
- draw ${\theta}^{\left(1\right)},\dots ,{\theta}^{\left(M\right)}$ from the posterior Beta density, where M is a large number;
- (ii)
- compute ${\psi}^{\left(j\right)}=g\left({\theta}^{\left(j\right)}\right)$, for $j=1,\dots ,M$;
- (iii)
- use the
`R`function`HDInterval::hdi`with the MC draws ${\psi}^{\left(1\right)},\dots ,{\psi}^{\left(M\right)}$ in input.

## 4. Application to Clinical Trials

- Effect of sample size. As expected, the values of ${e}_{n}^{P}$ decrease as n increases and depend on the specific choices of $\alpha $, $\beta $ and ${\theta}_{d}$ as commented in the following remarks.
- Effect of prior sample size. For each value of n, the larger $\alpha +\beta $, the greater the values of ${e}_{n}^{P}$. In fact, as the prior becomes more and more concentrated around the prior mean $0.54$, the weight of the prior in the posterior distribution increases with respect to the role of the likelihood. This makes the discrepancy between Bayesian exact intervals and their likelihood approximation more striking. Moreover, when the uniform non-informative prior is considered, the smallest values of ${e}_{n}^{P}$ are observed (see solid line in Figure 2). As a consequence, larger values of the prior sample size imply greater values of ${n}_{P}^{\u2606}$, as shown in Table 1.
- Effect of the difference between design value and prior mean. When the distance between ${\theta}_{d}$ and the prior mean $\alpha /(\alpha +\beta )$ is relatively large and, at the same time, the prior sample size $\alpha +\beta $ dominates n, the posterior mode and the maximum likelihood estimate are well separated. In other words, Equation (4) does not provide a good approximation of the posterior density of $\theta $. This explains the larger values of ${e}_{n}^{P}$, in the right panel of Figure 2, where $|{\theta}_{d}-\mathbb{E}\left(\theta \right)|=0.35$, with respect to those observed in the left panel, where $|{\theta}_{d}-\mathbb{E}\left(\theta \right)|=0.09$. As before, the effect of the difference between design value and prior mean on ${e}_{n}^{P}$ also reflects on the values of the optimal sample sizes reported in Table 1. For instance, under the most informative prior, if $|{\theta}_{d}-\mathbb{E}\left(\theta \right)|=0.09$, then ${n}_{P}^{\u2606}=182$; conversely, when $|{\theta}_{d}-\mathbb{E}\left(\theta \right)|=0.35$, a huge number of experimental units (e.g., ${n}_{P}^{\u2606}=2911$) is required to have a sufficiently small expected discrepancy.
- Comparison with ${n}_{B}^{\u2606}$. As expected, the trend of ${n}_{B}^{\u2606}$ w.r.t. to $(\alpha ,\beta )$ and ${\theta}_{d}$ is consistent with that of ${n}_{P}^{\u2606}$.
- Comparison with ALC. For each ${\theta}_{d}$, ${n}_{L}^{\u2606}$ becomes slightly smaller when the prior sample size gets larger and the corresponding posterior is more concentrated (see Table 1). Conversely, since approximate intervals do not depend on the prior, ${n}_{\tilde{L}}^{\u2606}$ is not affected by the choice of prior hyperparameters. Furthermore, when the design value is closer to the boundary of the parameter space, the posterior distribution and, consequently, its approximation, become more concentrated, yielding shorter intervals. Hence the values of ${n}_{L}^{\u2606}$ and of ${n}_{\tilde{L}}^{\u2606}$ are uniformly smaller for ${\theta}_{d}=0.80$ than for ${\theta}_{d}=0.45$.It is interesting to note the opposite impact of the prior sample size $\alpha +\beta $ on ${n}_{P}^{\u2606}$ and ${n}_{B}^{\u2606}$ on the one hand, and on ${n}_{L}^{\u2606}$ on the other hand. In fact, larger values of $\alpha +\beta $ determine shorter intervals and smaller values of ${n}_{L}^{\u2606}$. On the contrary, when ${\theta}_{d}\ne \mathbb{E}\left(\theta \right)$, a more concentrated prior implies a more remarkable discrepancy between the posterior and its likelihood approximation and, consequently, yields greater values of ${n}_{P}^{\u2606}$ and ${n}_{B}^{\u2606}$.

## 5. Conclusions

- Other models. The methodology proposed in the paper can be easily extended to other models and setups relevant to clinical trials applications. A natural extension is to two-arms designs for the comparison of two proportions (difference or log odds ratio), in which the additional issue of units allocation arises [32]. For a predictive approach to allocation based on the control of posterior variances, see for instance [33]. See also [5] for related ideas in the Poisson model.
- Probability vs. Expectation. In Section 2.2 we propose to summarize the predictive distribution of the discrepancy using the expected value w.r.t. ${f}_{n}(\xb7|{\theta}_{d})$. An alternative is to take into account the whole probability distribution of P and to determine the smallest n such that $\mathbb{P}[P\left({\mathbf{X}}_{\mathit{n}}\right)>{\u03f5}_{P}]$ is sufficiently small.
- Design prior. For simplicity in this article we have performed preposterior calculations using the sampling distribution ${f}_{n}(\xb7|{\theta}_{d})$. An alternative is to consider the so-called two–priors approach [23,24,30,34]) which avoids local optimality by replacing the design value with the design prior.
- Decision-theoretic approach. The approach proposed in the paper is performance-based. Alternatively one could follow some previous works and rephrase the problem in a decision-theoretic framework and define a measure of discrepancy based on the posterior expected loss of C and $\tilde{C}$. We will elaborate on this in the future.

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Lee, J.; Chu, C.T. Bayesian clinical trials in action. Stat. Med.
**2012**, 31, 2955–2972. [Google Scholar] [PubMed] - Yin, G.; Lam, C.K.; Shi, H. Bayesian randomized clinical trials: From fixed to adaptive design. Contemp. Clin. Trials
**2017**, 59, 77–86. [Google Scholar] [CrossRef] [PubMed] - Bittl, J.A.; He, Y. Bayesian analysis. A practical approach to interpret clinical trials and create clinical practice guidelines. Circ. Cardiovasc. Qual. Outcomes
**2017**, 10, e003563. [Google Scholar] [CrossRef] [PubMed] - Spiegelhalter, D.J.; Abrams, K.R.; Myles, J.P. Bayesian Approaches to Clinical Trials and Health-Care Evaluation; Statistics in Practice; Wiley: Chichester, UK, 2004. [Google Scholar]
- De Santis, F.; Gubbiotti, S. A note on the progressive overlap of two alternative Bayesian intervals. Commun. Stat. Theory Methods
**2019**, 1–18. [Google Scholar] [CrossRef] - Lesaffre, E.; Lawson, A.B. Bayesian Biostatistics; Wiley: Chichester, UK, 2012. [Google Scholar]
- Gelman, A.; Carlin, J.B.; Stern, H.S.; Dunson, D.B.; Vehtari, A.; Rubin, D.B. Bayesian Data Analysis, 3rd ed.; Chapman & Hall/CRC Texts in Statistical Science; Taylor & Francis: Boca Raton, FL, USA, 2013. [Google Scholar]
- Kalbfleisch, J.G. Probability and Statistical Inference. Volume 2: Statistical Inference, 2nd ed.; Springer: New York, NY, USA, 1985. [Google Scholar]
- Robert, C.P. The Bayesian Choice: From Decision-Theoretic Foundations to Computational Implementation, 2nd ed.; Springer: New York, NY, USA, 2007. [Google Scholar]
- Meeker, W.Q.; Hahn, G.J.; Escobar, L.A. Statistical Intervals: A Guide for Practitioners and Researchers; Wiley: Hoboken, NJ, USA, 2017. [Google Scholar]
- Brutti, P.; De Santis, F.; Gubbiotti, S. Bayesian—Frequentist sample size determination: A game of two priors. Metron
**2014**, 72, 133–151. [Google Scholar] [CrossRef] - Adcock, C.J. Sample size determination: A review. J. R. Stat. Soc. Ser. D Stat.
**1997**, 46, 261–283. [Google Scholar] [CrossRef] - Joseph, L.; du Berger, R.; Belisle, P. Bayesian and mixed Bayesian/likelihood criteria for sample size determination. Stat. Med.
**1997**, 16, 769–781. [Google Scholar] [CrossRef] - Joseph, L.; Wolfson, D. Interval-based versus decision theoretic criteria for the choice of sample size. J. R. Stat. Soc. Ser. Stat.
**1997**, 46, 145–149. [Google Scholar] [CrossRef] - Brutti, P.; De Santis, F. Robust Bayesian sample size determination for avoiding the range of equivalence in clinical trials. J. Stat. Plan. Inference
**2008**, 138, 1577–1591. [Google Scholar] [CrossRef] - Cao, J.; Lee, J.J.; Alber, S. Comparison of Bayesian sample size criteria: ACC, ALC, and WOC. J. Stat. Plan. Inference
**2009**, 139, 4111–4122. [Google Scholar] [CrossRef][Green Version] - Gubbiotti, S.; De Santis, F. A Bayesian method for the choice of the sample size in equivalence trials. Aust. New Zealand J. Stat.
**2011**, 53, 443–460. [Google Scholar] [CrossRef] - Joseph, L.; Wolfson, D.B.; Berger, R.D. Sample Size Calculations for Binomial Proportions via Highest Posterior Density Intervals. J. R. Stat. Soc. Ser. D Stat.
**1995**, 44, 143–154. [Google Scholar] [CrossRef] - M’Lan, C.E.; Joseph, L.; Wolfson, D.B. Bayesian sample size determination for binomial proportions. Bayesian Anal.
**2008**, 3, 269–296. [Google Scholar] [CrossRef] - De Santis, F.; Fasciolo, M.C.; Gubbiotti, S. Predictive control of posterior robustness for sample size choice in a Bernoulli model. Stat. Methods Appl.
**2013**, 22, 319–340. [Google Scholar] [CrossRef] - De Santis, F. Sample size determination for robust Bayesian analysis. J. Am. Stat. Assoc.
**2006**, 101, 278–291. [Google Scholar] [CrossRef] - Brutti, P.; De Santis, F.; Gubbiotti, S. Robust Bayesian sample size determination in clinical trials. Stat. Med.
**2008**, 27, 2290–2306. [Google Scholar] [CrossRef] [PubMed] - Joseph, L.; Belisle, P. Bayesian consensus-based sample size criteria for binomial proportions. Stat. Med.
**2019**, 38, 4566–4573. [Google Scholar] [CrossRef] [PubMed] - Brutti, P.; De Santis, F.; Gubbiotti, S. Predictive measures of the conflict between frequentist and Bayesian estimators. J. Stat. Plan. Inference
**2014**, 148, 111–122. [Google Scholar] [CrossRef] - De Santis, F.; Gubbiotti, S. A decision-theoretic approach to sample size determination under several priors. Appl. Stoch. Model. Bus. Ind.
**2017**, 33, 282–295. [Google Scholar] [CrossRef] - Casella, G.; Berger, R. Statistical Inference; Duxbury: Belmont, CA, USA, 2001. [Google Scholar]
- R Core Team. R: A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2018. [Google Scholar]
- Sacchi, S.; Marcheselli, R.; Bari, A.; Buda, G.; Molinari, A.L.; Baldini, L.; Vallisa, D.; Cesaretti, M.; Musto, P.; Ronconi, S.; et al. Safety and efficacy of lenalidomide in combination with rituximab in recurrent indolent non-follicular lymphoma: Final results of a phase II study conducted by the Fondazione Italiana Linfomi. Haematologica
**2016**, 101, e196–e199. [Google Scholar] [CrossRef][Green Version] - Zhou, H.; Lee, J.J.; Yuan, Y. BOP2: Bayesian optimal design for phase II clinical trials with simple and complex endpoints. Stat. Med.
**2017**, 36, 3302–3314. [Google Scholar] [CrossRef] [PubMed] - Sambucini, V. Bayesian predictive monitoring with bivariate binary outcomes in phase II clinical trials. Comput. Stat. Data Anal.
**2019**, 132, 18–30. [Google Scholar] [CrossRef] - Morita, S.; Thall, P.F.; Muller, P. Determining the effective sample size of a parametric prior. Biometrics
**2008**, 64, 595–602. [Google Scholar] [CrossRef] [PubMed] - M’Lan, C.E.; Joseph, L.; Wolfson, D.B. Bayesian Sample Size Determination for Case-Control Studies. J. Am. Stat. Assoc.
**2006**, 101, 760–772. [Google Scholar] [CrossRef][Green Version] - De Santis, F.; Perone Pacifico, M.; Sambucini, V. Optimal predictive sample size for case-control studies. Appl. Stat.
**2004**, 53, 427–441. [Google Scholar] [CrossRef] - Wang, F.; Gelfand, A.E. A simulation-based approach to Bayesian sample size determination for performance under a given model and for separating models. Statist. Sci.
**2002**, 17, 193–208. [Google Scholar] [CrossRef]

**Figure 1.**Posterior density, given a prior density of hyperparameters $(\alpha ,\beta )=(10.8,9.2)$, and likelihood approximation, given ${\overline{x}}_{n}=0.45$ (top row) and ${\overline{x}}_{n}=0.8$ (bottom row) for $n=10$ (left column) and $n=100$ (right column). Exact credible intervals (HPD: Highest Posterior Density) are denoted by empty circles, likelihood approximated credible intervals (LNA: Likelihood Normal Approximation) are denoted by black circles. The probability that $\theta $ belongs to the approximate interval under the exact posterior distribution is highlighted in grey.

**Figure 2.**Plots of ${e}_{n}^{P}$ as a function of n for several values of the prior hyperparameters $(\alpha ,\beta )$, with ${\theta}_{d}=0.45$ (

**left column**) and ${\theta}_{d}=0.8$ (

**right column**).

**Figure 3.**Plots of ${e}_{n}^{P}$ as a function of n for several values of the prior hyperparameters $(\alpha ,\beta )$ with ${\theta}_{d}=0.45$ (

**left panel**) and ${\theta}_{d}=0.8$ (

**right panel**), when the logodds $\psi $ is the parameter of interest.

**Table 1.**Optimal sample sizes for several choices of the prior hyperameters and of the design values, given ${\u03f5}_{P}={\u03f5}_{B}=0.01$ and ${\u03f5}_{L}=0.1$.

${\mathit{\theta}}_{\mathit{d}}$ | $(\mathit{\alpha},\mathit{\beta})$ | $(1,1)$ | $(2.7,2.3)$ | $(5.4,4.6)$ | $(10.8,9.2)$ |
---|---|---|---|---|---|

$0.45$ | ${n}_{P}^{\u2606}$ | 49 | 80 | 119 | 182 |

${n}_{B}^{\u2606}$ | 42 | 96 | 180 | 347 | |

${n}_{L}^{\u2606}$ | 265 | 262 | 257 | 247 | |

${n}_{\tilde{L}}^{\u2606}$ | 267 | 267 | 267 | 267 | |

$0.80$ | ${n}_{P}^{\u2606}$ | 35 | 118 | 646 | 2911 |

${n}_{B}^{\u2606}$ | 91 | 228 | 482 | 992 | |

${n}_{L}^{\u2606}$ | 170 | 169 | 169 | 167 | |

${n}_{\tilde{L}}^{\u2606}$ | 172 | 172 | 172 | 172 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

De Santis, F.; Gubbiotti, S.
Sample Size Requirements for Calibrated Approximate Credible Intervals for Proportions in Clinical Trials. *Int. J. Environ. Res. Public Health* **2021**, *18*, 595.
https://doi.org/10.3390/ijerph18020595

**AMA Style**

De Santis F, Gubbiotti S.
Sample Size Requirements for Calibrated Approximate Credible Intervals for Proportions in Clinical Trials. *International Journal of Environmental Research and Public Health*. 2021; 18(2):595.
https://doi.org/10.3390/ijerph18020595

**Chicago/Turabian Style**

De Santis, Fulvio, and Stefania Gubbiotti.
2021. "Sample Size Requirements for Calibrated Approximate Credible Intervals for Proportions in Clinical Trials" *International Journal of Environmental Research and Public Health* 18, no. 2: 595.
https://doi.org/10.3390/ijerph18020595