# Optimal Weighted Multiple-Testing Procedure for Clinical Trials

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. The O’Brien and Fleming Procedure

#### 2.1.1. Statement of the O’Brien and Fleming Procedure

#### 2.1.2. Evaluation of the Stopping Bound

#### 2.2. Optimal Allocation

## 3. The Proposed Procedure OWMP

#### 3.1. The New Methodology

- For $i=1,\dots ,K-1$, ${n}_{i}$ is calculated as ${n}_{i}$= round ($w{e}_{i}$× N), if ${n}_{1}$ is even. Otherwise ${n}_{1}$ = round ($w{e}_{i}\times $ N) + 1; because equal allocation is used in the first stage. Furthermore, equal allocation is used in other stages when the optimal ratio equals zero or one.
- For the last stage $K$, ${n}_{K}=N-{\displaystyle \sum}_{k=1}^{K-1}{n}_{k}$ to cover the rounding that is used in the previous stages.

#### 3.2. Type I Error and Power to Validate the OWMP

#### 3.2.1. Testing Type I Error Algorithm

#### 3.2.2. Result of Testing Type I Error

#### 3.2.3. Testing Power Algorithm

#### 3.2.4. Result of Testing Power

#### 3.3. Calculating Rejection Rates for Each Stage

#### 3.3.1. Calculating Rejection Rates for Each Stage When ${H}_{a}$ Is True, and the Difference Is Presented

#### 3.3.2. Calculating Rejection Rates for Each Stage When ${H}_{0}$ Is True, and the Difference Is Not Presented

## 4. Examples

#### 4.1. Example 1: Real Life Example

#### 4.2. Example 2: Computational Example

## 5. Discussion

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Glancey, J.L.; Upadhyaya, S.K. An improved technique for agricultural implement draught analysis. Soil Tillage Res.
**1995**, 35, 175–182. [Google Scholar] [CrossRef] - Chow, S.-C.; Chang, M. Adaptive design methods in clinical trials—A review. Orphanet J. Rare Dis.
**2008**, 3, 11. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Jennison, C.; Turnbull, B.W. Group Sequential Methods with Applications to Clinical Trials; CRC Press: Boca Raton, FL, USA, 1999. [Google Scholar] [CrossRef]
- Bretz, F.; Maurer, W.; Brannath, W.; Posch, M. A graphical approach to sequentially rejective multiple test procedures. Stat. Med.
**2009**, 28, 586–604. [Google Scholar] [CrossRef] [PubMed] - Fu, Y. Step-Down Parametric Procedures for Testing Correlated Endpoints in a Group-Sequential Trial. Stat. Biopharm. Res.
**2018**, 10, 18–25. [Google Scholar] [CrossRef] [Green Version] - Shun, Z. Stopping Boundaries Adjusted for Sample Size Reestimation and Negative Stop. J. Biopharm. Stat.
**2002**, 12, 485–502. [Google Scholar] [CrossRef] - Urach, S.; Posch, M. Multi-arm group sequential designs with a simultaneous stopping rule. Stat. Med.
**2016**, 35, 5536–5550. [Google Scholar] [CrossRef] - Lookman, T.; Balachandran, P.V.; Xue, D.; Hogden, J.; Theiler, J. Statistical inference and adaptive design for materials discovery. Curr. Opin. Solid State Mater. Sci.
**2017**, 21, 121–128. [Google Scholar] [CrossRef] - Shuster, J.J.; Neu, J. A Pocock approach to sequential meta-analysis of clinical trials. Res. Synth. Methods
**2013**, 4, 269–279. [Google Scholar] [CrossRef] [Green Version] - Motzer, R.J.; Escudier, B.; Oudard, S.; Hutson, T.E.; Porta, C.; Bracarda, S.; Grünwald, V.; Thompson, J.A.; Figlin, R.A.; Hollaender, N.; et al. Efficacy of everolimus in advanced renal cell carcinoma: A double-blind, randomised, placebo-controlled phase III trial. Lancet
**2008**, 372, 449–456. [Google Scholar] [CrossRef] - Goldberg, R.M.; Sargent, D.; Morton, R.F.; Fuchs, C.S.; Ramanathan, R.K.; Williamson, S.K.; Findlay, B.P.; Pitot, H.C.; Alberts, S.R. A Randomized Controlled Trial of Fluorouracil Plus Leucovorin, Irinotecan, and Oxaliplatin Combinations in Patients with Previously Untreated Metastatic Colorectal Cancer. J. Clin. Oncol.
**2004**, 22, 23–30. [Google Scholar] [CrossRef] - Bailey, R.C.; Moses, S.; Parker, C.B.; Agot, K.; Maclean, I.; Krieger, J.N.; Williams, C.F.; Campbell, R.T.; Ndinya-Achola, O.J. Male circumcision for HIV prevention in young men in Kisumu, Kenya: A randomized controlled trial. Lancet
**2007**, 369, 643–656. [Google Scholar] [CrossRef] - Marcus, R.; Davies, A.; Ando, K.; Klapper, W.; Opat, S.; Owen, C.; Phillips, E.; Sangha, R.; Schlag, R.; Seymour, J.F.; et al. Obinutuzumab for the First-Line Treatment of Follicular Lymphoma. N. Engl. J. Med.
**2017**, 377, 1331–1344. [Google Scholar] [CrossRef] - Lui, K.-J. The Performance of the O’Brien-Fleming Multiple Testing Procedure in the Presence of Intraclass Correlation. Biometrics
**1994**, 50, 232–236. [Google Scholar] [CrossRef] [PubMed] - Hammouri, H. Review and Implementation for the O’Brien Fleming Multiple Testing Procedure; Virginia Commonwealth University: Richmond, VA, USA, 2013. [Google Scholar] [CrossRef]
- O’Brien, P.C.; Fleming, T.R. A Multiple Testing Procedure for Clinical Trials. Biometrics
**1979**, 35, 549–556. [Google Scholar] [CrossRef] [PubMed] - Thompson, W.R. On the Likelihood that One Unknown Probability Exceeds Another in View of the Evidence of Two Samples. Biometrika
**1933**, 25, 285–294. [Google Scholar] [CrossRef] - Robbins, H. Some Aspects of the Sequential Design of Experiments. Bull. Am. Math. Soc.
**1952**, 58, 527–535. [Google Scholar] [CrossRef] [Green Version] - Zelen, M. Play the Winner Rule and the Controlled Clinical Trial. J. Am. Stat. Assoc.
**1969**, 64, 131–146. [Google Scholar] [CrossRef] - Sverdlov, O.; Rosenberger, W.F. On Recent Advances in Optimal Allocation Designs in Clinical Trials. J. Stat. Theory Pract.
**2013**, 7, 753–773. [Google Scholar] [CrossRef] - Lehmacher, W.; Wassmer, G. Adaptive Sample Size Calculations in Group Sequential Trials. Biometrics
**1999**, 55, 1286–1290. [Google Scholar] [CrossRef] - Liu, Q.; Chi, G.Y.H. On Sample Size and Inference for Two-Stage Adaptive Designs. Biometrics
**2001**, 57, 172–177. [Google Scholar] [CrossRef] - Al Garni, H.Z.; Awasthi, A. Al Garni, H.Z.; Awasthi, A. A Monte Carlo Approach Applied to Sensitivity Analysis of Criteria Impacts on Solar PV Site Selection. In Handbook of Probabilistic Models, 1st ed.; Samui, P., Bui, D.T., Chakraborty, S., Deo, R.C., Eds.; Butterworth-Heinemann: Oxford, UK, 2020. [Google Scholar] [CrossRef]
- Martinez, W.L.; Martinez, A.R. Computational Statistics Handbook with MATLAB, 2nd ed.; CRC Press: Boca Raton, FL, USA, 2007. [Google Scholar] [CrossRef]
- Morris, T.P.; White, I.R.; Crowther, M.J. Using simulation studies to evaluate statistical methods. Stat. Med.
**2019**, 38, 2074–2102. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Rosenberger, W.F.; Stallard, N.; Ivanova, A.; Harper, C.N.; Ricks, M.L. Optimal adaptive designs for binary response trials. Biometrics
**2001**, 57, 909–913. [Google Scholar] [CrossRef] [PubMed] - Lewis, R.J.; Berry, D.A. Group Sequential Clinical Trials: A Classical Evaluation of Bayesian Decision-Theoretic Designs. J. Am. Stat. Assoc.
**1994**, 89, 1528–1534. [Google Scholar] [CrossRef] - Schulz, K.F.; Grimes, A.D. Multiplicity in randomised trials II: Subgroup and interim analyses. Lancet
**2005**, 365, 1657–1661. [Google Scholar] [CrossRef] - Pocock, S.J. When (Not) to Stop a Clinical Trial for Benefit. JAMA
**2005**, 294, 2228–2230. [Google Scholar] [CrossRef] - Glimm, E.; Maurer, W.; Bretz, F. Hierarchical testing of multiple endpoints in group-sequential trials. Stat. Med.
**2010**, 29, 219–228. [Google Scholar] [CrossRef] - Tamhane, A.C.; Mehta, C.R.; Liu, L. Testing a Primary and a Secondary Endpoint in a Group Sequential Design. Biometrics
**2010**, 66, 1174–1184. [Google Scholar] [CrossRef] [Green Version] - Tamhane, A.C.; Wu, Y.; Mehta, C.R. Adaptive extensions of a two-stage group sequential procedure for testing primary and secondary endpoints (I): Unknown correlation between the endpoints. Stat. Med.
**2012**, 31, 2027–2040. [Google Scholar] [CrossRef] - Ye, Y.; Li, A.; Liu, L.; Yao, B. A group sequential Holm procedure with multiple primary endpoints. Stat. Med.
**2013**, 32, 1112–1124. [Google Scholar] [CrossRef] - Xi, D.; Tamhane, A.C. Allocating recycled significance levels in group sequential procedures for multiple endpoints. Biom. J.
**2015**, 57, 90–107. [Google Scholar] [CrossRef] - Lui, K.-J. A Simple Generalization of the O’Brien and Fleming Group Sequential Test Procedure to More Than Two Treatment Groups. Biometrics
**1993**, 49, 1216–1219. [Google Scholar] [CrossRef]

**Figure 3.**The algorithm to calculate the values of the Type I error and power using Monte Carlo simulations with the OWMP.

**Figure 4.**Power values of OWMP with α = 0.05, ${P}_{1}$ = 0.1, and ${P}_{2}$ = 0.15, 0.2, 0.25, 0.3.

**Figure 5.**Power values for OWMP with α = 0.01, ${P}_{1}$ = 0.1, and ${P}_{2}$ = 0.15, 0.2, 0.25, 0.3.

$\mathit{\alpha}$ | Number of Stages (K) | ||||
---|---|---|---|---|---|

1 | 2 | 3 | 4 | 5 | |

0.5 | 0.462 | 0.656 | 0.750 | 0.785 | 0.819 |

0.1 | 2.67 | 2.859 | 2.907 | 2.979 | 3.087 |

0.09 | 2.866 | 3.031 | 3.073 | 3.147 | 3.283 |

0.08 | 3.077 | 3.197 | 3.24 | 3.338 | 3.467 |

0.07 | 3.294 | 3.363 | 3.437 | 3.546 | 3.663 |

0.06 | 3.576 | 3.652 | 3.683 | 3.853 | 3.889 |

0.05 | 3.869 | 3.928 | 3.940 | 4.170 | 4.149 |

0.04 | 4.289 | 4.231 | 4.264 | 4.477 | 4.584 |

0.03 | 4.800 | 4.722 | 4.700 | 4.964 | 5.045 |

0.02 | 5.490 | 5.392 | 5.462 | 5.555 | 5.789 |

0.01 | 6.667 | 6.574 | 6.503 | 6.864 | 6.838 |

0.005 | 7.885 | 7.818 | 7.442 | 7.890 | 8.037 |

0.001 | 10.062 | 10.240 | 10.202 | 11.060 | 10.600 |

$\mathit{\alpha}$ | Number of Stages (K) | ||||
---|---|---|---|---|---|

1 | 2 | 3 | 4 | 5 | |

0.5 | 0.4547 | 0.6546 | 0.7439 | 0.8013 | 0.8431 |

0.1 | 2.7042 | 2.8195 | 2.9247 | 3.0047 | 3.0650 |

0.09 | 2.8730 | 2.9817 | 3.0877 | 3.1668 | 3.2275 |

0.08 | 3.0633 | 3.1646 | 3.2700 | 3.3498 | 3.4114 |

0.07 | 3.2814 | 3.3754 | 3.4799 | 3.5594 | 3.6220 |

0.06 | 3.5348 | 3.6207 | 3.7251 | 3.8034 | 3.8669 |

0.05 | 3.8399 | 3.9152 | 4.0191 | 4.0961 | 4.1602 |

0.04 | 4.2177 | 4.2809 | 4.3836 | 4.4599 | 4.5243 |

0.03 | 4.7099 | 4.7622 | 4.8587 | 4.9341 | 5.0008 |

0.02 | 5.4106 | 5.4537 | 5.5396 | 5.6148 | 5.6827 |

0.01 | 6.6393 | 6.6618 | 6.7353 | 6.8021 | 6.8764 |

0.005 | 7.8863 | 7.9019 | 7.9529 | 8.0094 | 8.0803 |

0.001 | 10.8280 | 10.8527 | 10.8618 | 10.9263 | 10.9820 |

P | Number of Stages (K) | |||||

1 | 2 | 3 | 4 | 5 | ||

0.1 | 0.0503 | 0.0486 | 0.0458 | 0.0442 | 0.0415 | |

0.2 | 0.0499 | 0.0487 | 0.0478 | 0.0457 | 0.0430 | |

0.3 | 0.0499 | 0.0488 | 0.0476 | 0.0461 | 0.0433 | |

0.4 | 0.0487 | 0.0485 | 0.0477 | 0.0463 | 0.0438 | |

0.5 | 0.0499 | 0.0500 | 0.0497 | 0.0453 | 0.0422 |

P | Number of Stages (K) | |||||

1 | 2 | 3 | 4 | 5 | ||

0.1 | 0.0093 | 0.0088 | 0.0082 | 0.0079 | 0.0075 | |

0.2 | 0.0098 | 0.0098 | 0.0094 | 0.0089 | 0.0084 | |

0.3 | 0.0101 | 0.0100 | 0.0094 | 0.0091 | 0.0088 | |

0.4 | 0.0104 | 0.0100 | 0.0096 | 0.0092 | 0.0089 | |

0.5 | 0.0094 | 0.0099 | 0.0097 | 0.0094 | 0.0090 |

**Table 5.**Power values for the OWMP with $\alpha =0.05$, ${P}_{1}=0.1,\mathrm{and}{P}_{2}=0.15,0.2,0.25,\mathrm{and}0.3$.

P_{1} | Number of Stages (K) | ||||||

0.1 | P_{2} | n | 1 | 2 | 3 | 4 | 5 |

0.15 | 1366 | 0.8020 | 0.7959 | 0.7890 | 0.7857 | 0.7822 | |

0.2 | 394 | 0.8046 | 0.7977 | 0.7917 | 0.7858 | 0.7786 | |

0.25 | 200 | 0.8164 | 0.8062 | 0.8008 | 0.7953 | 0.7892 | |

0.3 | 120 | 0.8133 | 0.8013 | 0.7931 | 0.7795 | 0.7726 |

**Table 6.**Power values for the OWMP with $\alpha =0.01$, ${P}_{1}$ = 0.1, and ${P}_{2}$ = 0.15, 0.2, 0.25, 0.3.

P_{1} | Number of Stages (K) | ||||||

0.1 | P_{2} | N | 1 | 2 | 3 | 4 | 5 |

0.15 | 2032 | 0.8010 | 0.7994 | 0.7955 | 0.7918 | 0.7875 | |

0.2 | 588 | 0.8037 | 0.8016 | 0.7943 | 0.7897 | 0.7840 | |

0.25 | 296 | 0.8113 | 0.8079 | 0.8002 | 0.7941 | 0.7888 | |

0.3 | 182 | 0.8085 | 0.8057 | 0.8006 | 0.7926 | 0.7841 |

**Table 7.**Sample sizes and percentages of rejections of ${H}_{0}$ occurring at stage $i$ with 500,000 iterations. Where ${P}_{1}$ = 0.1 and ${P}_{2}$ = 0.2 with α = 0.01, and the sample size equals 588.

i | $NumberofStages$(K) | ||||

2 | 3 | 4 | 5 | ||

First Stage | 488(35%) | 338 (3%) | 300(0%) | 180 (0%) | |

Second Stage | 588 (65%) | 488 (55%) | 430 (25%) | 330 (4%) | |

Third Stage | 588 (42%) | 530 (49%) | 450 (35%) | ||

Fourth Stage | 588 (26%) | 530 (38%) | |||

Fifth Stage | 588 (23%) |

**Table 8.**Based on 5% of the 500,000 iterations, values for sample sizes and acceptance rates at stage $i$ when ${H}_{0}$ is true.

i | $NumberofStages$(K) | ||||

2 | 3 | 4 | 5 | ||

First Stage | 250 (1%) | 220 (0%) | 200 (0%) | 160 (0%) | |

Second Stage | 300 (99%) | 270 (12%) | 250 (1%) | 210 (0%) | |

Third Stage | 300 (88%) | 280 (23%) | 250 (4%) | ||

Fourth Stage | 300 (76%) | 280 (26%) | |||

Fifth Stage | 300 (70%) |

$\mathit{i}$ | Critical Values | n_{1} | n_{2} | X_{1} | X_{2} | Total Sample Size | Usual Chi-Square | $\frac{\mathit{i}}{\mathit{K}}{\mathit{\chi}}_{\left(\mathit{i}\right)}^{2}$ |
---|---|---|---|---|---|---|---|---|

Result of Case 1 ($K=1)$ (${we}_{1}=$ 100%) | ||||||||

1 | 3.84 | 150 | 150 | 91 | 14 | 300 | 86.9 | 86.9 |

Result of Case 2 ($K=2)$ (${we}_{1}=$ 60%, ${we}_{2}=$ 40%) | ||||||||

1 | 3.92 | 90 | 90 | 37 | 13 | 180 | 15.95 | 7.975 |

Result of Case 3 ($K=3)$ (${we}_{1}=$ 40%, ${we}_{2}=$ 35%, ${we}_{3}=$ 25%) | ||||||||

1 | 4.02 | 60 | 60 | 19 | 13 | 120 | 1.534 | 0.5113 |

2 | 4.02 | 118 | 108 | 63 | 13 | 226 | 41.201 | 27.467 |

Result of Case 4 ($K=4)$ (${we}_{1}=$ 40%, ${we}_{2}=$ 25%, ${we}_{3}=$ 20%, ${we}_{4}=$ 15%) | ||||||||

1 | 4.02 | 60 | 60 | 19 | 13 | 120 | 1.534 | 0.3835 |

2 | 4.02 | 102 | 94 | 47 | 13 | 196 | 25.951 | 12.976 |

Result of Case 5 ($K=5)$ (${we}_{1}=$ 30%, ${we}_{2}=$ 25%, ${we}_{3}=$ 20%, ${we}_{4}=$ 15%, ${we}_{5}=$ 10%) | ||||||||

1 | 4.16 | 45 | 45 | 13 | 11 | 90 | 0.227 | 0.0454 |

2 | 4.16 | 85 | 81 | 32 | 13 | 166 | 9.791 | 0.0226 |

3 | 4.16 | 121 | 105 | 64 | 13 | 226 | 41.074 | 24.644 |

$\mathit{i}$ | Critical Values | n_{1} | n_{2} | X_{1} | X_{2} | Total Sample Size | Chi-Square | $\frac{\mathit{i}}{\mathit{K}}{\mathit{\chi}}_{\left(\mathit{i}\right)}^{2}$ |
---|---|---|---|---|---|---|---|---|

Result of Case 1 ($K=1)$ (${we}_{1}=$ 100%) | ||||||||

1 | 3.84 | 200 | 200 | 46 | 79 | 400 | 12.7 | 12.7 |

Result of Case 2 ($K=2)$ (${we}_{1}=$ 70%, ${we}_{2}=$ 30%) | ||||||||

1 | 3.92 | 140 | 140 | 32 | 53 | 280 | 7.508 | 3.73 |

2 | 3.92 | 192 | 208 | 43 | 82 | 400 | 13.473 | 13.473 |

Result of Case 3 ($K=3)$ (${we}_{1}=$ 45%, ${we}_{2}=$ 35%, ${we}_{3}=$ 20%) | ||||||||

1 | 4.02 | 90 | 90 | 22 | 32 | 180 | 2.646 | 0.88 |

2 | 4.02 | 153 | 167 | 36 | 64 | 320 | 8.134 | 5.42 |

Result of Case 4 ($K=4)$ (${we}_{1}=$ 40%, ${we}_{2}=$ 25%, ${we}_{3}=$ 20%, ${we}_{4}=$ 15%) | ||||||||

1 | 4.02 | 80 | 80 | 19 | 29 | 160 | 2.976 | 0.74 |

2 | 4.02 | 125 | 135 | 29 | 51 | 260 | 6.475 | 3.23 |

3 | 4.02 | 164 | 176 | 38 | 68 | 340 | 9.431 | 7.07 |

Result of Case 5 ($K=5)$ (${we}_{1}=$ 30%, ${we}_{2}=$ 25%, ${we}_{3}=$ 20%, ${we}_{4}=$ 15%, ${we}_{5}=$ 10%) | ||||||||

1 | 4.16 | 60 | 60 | 16 | 21 | 120 | 0.977 | 0.19 |

2 | 4.16 | 107 | 113 | 25 | 41 | 220 | 4.368 | 1.74 |

3 | 4.16 | 143 | 157 | 32 | 59 | 300 | 8.184 | 4.91 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Hammouri, H.; Alquran, M.; Abdel Muhsen, R.; Altahat, J.
Optimal Weighted Multiple-Testing Procedure for Clinical Trials. *Mathematics* **2022**, *10*, 1996.
https://doi.org/10.3390/math10121996

**AMA Style**

Hammouri H, Alquran M, Abdel Muhsen R, Altahat J.
Optimal Weighted Multiple-Testing Procedure for Clinical Trials. *Mathematics*. 2022; 10(12):1996.
https://doi.org/10.3390/math10121996

**Chicago/Turabian Style**

Hammouri, Hanan, Marwan Alquran, Ruwa Abdel Muhsen, and Jaser Altahat.
2022. "Optimal Weighted Multiple-Testing Procedure for Clinical Trials" *Mathematics* 10, no. 12: 1996.
https://doi.org/10.3390/math10121996