Hypothesis Testing for Two-Arm Proportions with Two Binary Endpoints

Abstract

Many studies require evidence that a new treatment improves efficacy and maintains or improves safety. Composite endpoints can obscure trade-offs and complicate interpretation. We propose a single-stage hypothesis test that directly evaluates two binary endpoints against a concurrent control, offering a transparent alternative to composite endpoints. The test rejects only if the observed improvements on both endpoints exceed pre-specified paired thresholds. The joint distribution of efficacy and safety is modeled with a four-category multinomial, yielding probabilities for all the outcome combinations. This enables exact computation of rejection probabilities and identification of least-favorable parameter configurations to control type I error at the nominal level while retaining adequate power. Design tables map the target significance level and power, together with predefined effect sizes for each endpoint, to the required sample size and decision thresholds. Simulations and one case study illustrate design selection and interpretation. The proposed test provides an exact and practical tool for early-phase trials with dual binary endpoints, particularly when efficacy and safety must be evaluated simultaneously.

1. Introduction

Clinical trials increasingly require evidence that new treatments not only improve efficacy but also maintain or enhance safety. Regulatory agencies such as the U.S. Food and Drug Administration [1] explicitly emphasize the importance of multiple endpoints to ensure that therapeutic benefit is not achieved at the expense of unacceptable adverse effects. In many modern therapeutic areas—such as oncology, immunotherapy, and cardiovascular medicine—both efficacy and toxicity are regarded as co-primary outcomes, reflecting the dual objective of maximizing patient benefit while minimizing harm. Composite endpoints are frequently used to summarize multiple outcomes into a single measure; however, they can obscure trade-offs between efficacy and safety and complicate interpretation [2]. Consequently, there is growing interest in transparent and statistically rigorous frameworks that jointly evaluate efficacy and safety without conflating their effects.

From a methodological standpoint, the prevailing paradigm for co-primary binary endpoints is the intersection–union testing (IUT) framework, in which overall success is declared only when significant improvement is demonstrated on all the endpoints [3,4]. This approach provides strong control of the familywise type I error rate and aligns with regulatory expectations. A natural modeling strategy is to represent two binary outcomes jointly as a four-category multinomial variable, explicitly accounting for their correlation and enabling exact computation of joint probabilities and rejection regions.

A substantial body of research has focused on the design of clinical trials with dual binary endpoints. The seminal work of Bryant and Day [5] introduced a two-stage single-arm phase II design that simultaneously monitors efficacy and toxicity under a multinomial model. Their framework defined admissible regions for early termination but was limited to settings in which the experimental treatment was compared with a fixed standard rather than against a control treatment. Building on this idea, Conaway and Petroni [6] extended the single-arm framework to bivariate sequential settings, still assuming fixed standard values. Subsequent developments by Stallard, Thall, and Whitehead [7], Ivanova et al. [8], and Chen and Chi [9] further advanced early-phase designs with dual binary endpoints under single-arm or fixed-standard settings. Thall and Cheng [10] proposed a related decision-theoretic design for evaluating efficacy and safety in which clinically meaningful improvements of an experimental treatment over a standard treatment are elicited from the physician. Their testing procedure was developed using a large-sample normal approximation, whereas the present study provides an exact two-arm hypothesis testing framework under a multinomial model, allowing rejection probabilities and type I error rates to be computed exactly. Therefore, despite substantial progress, a methodological gap remains for exact two-arm hypothesis tests that jointly evaluate efficacy and safety when a clear decision is reached.

The present study addresses this methodological gap by proposing an exact frequentist hypothesis test for two-arm trials with two binary endpoints—efficacy and safety. The goal of the proposed method is twofold. First, it aims to directly evaluate the simultaneous improvement of both endpoints relative to a concurrent control rather than against fixed pre-specified standard values. Second, it ensures exact control of the type I error rate at the nominal significance level by analytically identifying least favorable parameter configurations (LFCs) or generalized least favorable parameter configurations (GLFCs). The LFCs and GLFCs are obtained by using monotonicity results to identify boundary parameter settings and then exactly evaluating the rejection probabilities under the multinomial model.

To achieve these objectives, the joint outcomes of efficacy and safety are modeled using a four-category multinomial distribution, which enables complete enumeration of all possible response combinations across treatment arms. Within this framework, the study focuses on the following four key methodological questions:

• How can we construct a single-stage rejection rule that declares superiority only when both endpoints exceed pre-specified paired thresholds?
• How can rejection probabilities be computed exactly under the four-category multinomial model?
• How can LFCs or GLFCs be identified to guarantee exact control of the type I error rate and adequate power?
• How can design tables be developed to map desired significance levels, power, and effect sizes to the required sample size and decision thresholds?

Since the two binary endpoints generate four possible joint response categories, the proposed formulation is naturally connected to the classical analysis of $2\times 2$ contingency tables; see, for example, Everitt [11] (Section 2.8). In particular, the condition $p_{11}{p}_{22}=p_{12}{p}_{21}$ corresponds to the classical no-association structure for a $2\times 2$ table, a testing problem discussed by Kendall and Stuart [12] (Section 33.16). The present work builds on this classical framework but focuses on a different objective: constructing an exact two-arm hypothesis testing procedure for simultaneous efficacy and safety improvement relative to a concurrent control. This work makes several distinct contributions. First, it extends the single-arm joint-endpoint framework of Bryant and Day [5] and Chen and Chi [9] to a two-arm comparative design, allowing direct inference about relative efficacy and safety between treatments rather than against fixed standards. Second, the proposed test provides an exact frequentist formulation that allows analytical computation of rejection probabilities and formal identification of least favorable configurations. Finally, the framework produces design tables that map nominal type I error, power and effect sizes to required sample sizes and thresholds, facilitating practical implementation with simulations and exact calculations.

More recently, Homma and Yoshida [13] and Jung et al. [14] considered two-arm clinical trial designs with two binary co-primary endpoints under joint binary-outcome frameworks. Homma and Yoshida [13] developed exact power and sample size calculations for two-arm superiority trials with two co-primary binary endpoints by combining the bivariate binomial distribution with endpoint-wise testing procedures, such as Fisher’s exact test, Fisher’s mid-p test, Pearson’s chi-square test, the Z-pooled exact unconditional test, and Boschloo’s exact unconditional test. Thus, their framework is primarily built on marginal tests for each endpoint within an intersection–union structure, whereas the present study directly formulates a joint multinomial hypothesis testing procedure for simultaneous efficacy and safety improvement based on the paired treatment–control differences in $(p_e,p_s)$.

Jung et al. [14] proposed a related two-arm, two-stage phase II design for two binary co-primary endpoints, which allows early termination for futility and is therefore useful for screening ineffective treatments. Their approach is closer in spirit to the present work because it also models correlated binary endpoints jointly. However, their operating characteristics are calibrated at pre-specified null and alternative configurations: the type I error rate is evaluated under selected point configurations of the null hypothesis, and the power is calculated at a specified alternative point. In contrast, the proposed method identifies least favorable or generalized least favorable configurations for type I error control and power evaluation so that the resulting design guarantees the desired operating characteristics over the corresponding parameter regions rather than only at selected design points.

By uniting the intersection–union test principle and multinomial modeling, this work offers an exact and transparent testing framework for clinical trials requiring simultaneous improvement in efficacy and safety. The proposed testing procedure is calibrated to guarantee the desired type I error control and power requirements over the corresponding parameter regions rather than only at selected point configurations. It provides statistical rigor through precise error control and clinical interpretability by maintaining separate explicit evaluation of efficacy and safety. ChatGPT-5.5 was used to assist in converting simulation output tables into LaTeX format.

2. Formulation for Two Endpoints

We consider a procedure for comparing an experimental treatment with a control. Let ${\pi }_0$ denote the control treatment and ${\pi }_1$ the experimental treatment. Each treatment yields two binary endpoints, referred to as efficacy and safety, although other binary outcomes could be accommodated within the same framework. For treatment j ($j=0,1$), let $X_e^j$ and $X_s^j$ denote the numbers of successes in efficacy and safety among n patients, with corresponding success probabilities $p_e^j$ and $p_s^j$. The two endpoints within a treatment arm may be correlated, whereas responses across different treatments are assumed to be independent. In this paper, the association between the efficacy and safety endpoints is characterized through a pre-specified odds ratio $\varphi$. This specification is needed because, under the four-category multinomial model, the marginal efficacy probability, the marginal safety probability, and the odds ratio together determine the joint cell probabilities $(p_{11},p_{12},p_{21},p_{22})$; conversely, the joint cell probabilities determine the two marginal probabilities and the odds ratio. Thus, a working specification of the association structure is required to fully define the multinomial probabilities used in the design. In practice, information on the control treatment may be available from historical studies or previous clinical experience, so the marginal response rates and the association between efficacy and safety can often be estimated or elicited for the control arm. For the experimental treatment, however, the association structure is typically less well established at the design stage. Following common practice in phase II designs with dual binary endpoints, we therefore use a common odds ratio across treatment arms as a working design assumption. A similar specification of the association structure through a pre-specified odds ratio has also been used in the dual-endpoint phase II trial literature; see, for example, Chen and Chi [9].

To simultaneously assess clinical efficacy and safety, one may formulate the null hypothesis as stating that the new treatment is not sufficiently better than the control in terms of either efficacy or safety, while the alternative hypothesis requires improvement in both dimensions. Specifically, we consider the following hypothesis testing problem:

$$H_0:p_e^1\le p_e^0\mathrm{or}{p}_s^1\le p_s^0\text{vs.}{H}_1:p_e^1>p_e^0\mathrm{and}{p}_s^1>p_s^0.$$

(1)

Based on the null and alternative hypotheses in (1), the following probability requirements are imposed:

$$P(\mathrm{Reject}{H}_0\mid H_0\mathrm{is}\mathrm{true})\le \alpha ,$$

(2)

and

$$P(\mathrm{Reject}{H}_0\mid H_a\mathrm{is}\mathrm{true})\ge 1-\beta .$$

(3)

Here, condition (2) controls the type I error probability at the nominal level $\alpha$, which represents the maximum acceptable probability of falsely declaring the experimental treatment superior when it does not achieve simultaneous improvement in both efficacy and safety. In potential clinical applications, the choice of $\alpha$ reflects the tolerance for such a false-positive conclusion and may depend on the trial phase, disease severity, available alternatives, and regulatory considerations. Condition (3) requires the power of the test to be at least $1-\beta$ under clinically meaningful alternatives, where $\beta$ denotes the maximum acceptable probability of failing to detect a treatment that improves both endpoints by the pre-specified effect sizes. Thus, $\alpha$ controls the risk of incorrectly advancing an insufficient treatment, whereas $1-\beta$ measures the probability of correctly identifying a treatment with meaningful joint efficacy–safety benefit.

Joint Probability of Two Endpoints

Let $j=0,1$ index the two treatment arms, where $j=0$ corresponds to the control treatment ${\pi }_0$ and $j=1$ corresponds to the experimental treatment ${\pi }_1$. For patient i in treatment arm j, let $(E_{ji},S_{ji})$ denote the two binary endpoints, where $E_{ji}=1$ indicates efficacy success and $S_{ji}=1$ indicates safety success. Thus, each patient falls into one of four joint response categories, $(1,1)$, $(1,0)$, $(0,1)$, or $(0,0)$, corresponding, respectively, to efficacious and safe, efficacious but unsafe, nonefficacious but safe, and neither efficacious nor safe.

Accordingly, for treatment j, we denote the vector of cell counts by

$${\overrightarrow{X}}^j=(X_{11}^j,X_{12}^j,X_{21}^j,X_{22}^j),$$

where

$$X_{11}^j={\displaystyle \sum _{i=1}^n}I(E_{ji}=1,S_{ji}=1),X_{12}^j={\displaystyle \sum _{i=1}^n}I(E_{ji}=1,S_{ji}=0),$$

$$X_{21}^j={\displaystyle \sum _{i=1}^n}I(E_{ji}=0,S_{ji}=1),X_{22}^j={\displaystyle \sum _{i=1}^n}I(E_{ji}=0,S_{ji}=0).$$

We assume that this vector follows a multinomial distribution with total sample size n and cell probabilities

$${\overrightarrow{p}}^j=(p_{11}^j,p_{12}^j,p_{21}^j,p_{22}^j),$$

for $j=0,1$. Here, $p_{kl}^j$ ($k,l=1,2$) denotes the probability corresponding to each efficacy–safety combination in treatment arm j. The sample proportions $X_{kl}^j/n$ provide empirical estimates of the corresponding cell probabilities $p_{kl}^j$, for $k,l=1,2$.

This notation is equivalent to the standard three-dimensional contingency-table notation $n_{klj}$ for observed frequencies and $p_{klj}$ for cell probabilities, where the third index corresponds to the treatment arm. We retain notations $X_{kl}^j$ and $p_{kl}^j$ because, in clinical trial testing and ranking-and-selection designs, n is conventionally used for the sample size, while X denotes observed response counts.

The marginal probability of observing efficacy under treatment j is

$$p_e^j=p_{11}^j+p_{12}^j,$$

which aggregates all outcomes classified as efficacious.

Similarly, the marginal safety probability is

$$p_s^j=p_{11}^j+p_{21}^j,$$

representing the chance that a patient satisfies the safety criterion regardless of efficacy.

The corresponding marginal cell counts are

$$X_e^j=X_{11}^j+X_{12}^j,X_{e^c}^j=X_{21}^j+X_{22}^j,$$

and

$$X_s^j=X_{11}^j+X_{21}^j,X_{s^c}^j=X_{12}^j+X_{22}^j,$$

where $e^c$ and $s^c$ denote the complements of efficacy success and safety success, respectively.

Outcomes for treatment ${\pi }_j$ is listed in

Table 1. Observed outcomes for treatment ${\pi }_j$.

		Safety
		Yes	No	Sum
Efficacy	Yes	$X_{11}^j$	$X_{12}^j$	$X_e^j$
	No	$X_{21}^j$	$X_{22}^j$	$X_{e^c}^j$
	Sum	$X_s^j$	$X_{s^c}^j$

.

For each treatment arm, the dependence between efficacy and safety is quantified by the treatment-specific odds ratio

$${\varphi }_j={\displaystyle \frac{p_{11}^j{p}_{22}^j}{p_{12}^j{p}_{21}^j}},j=0,1.$$

The odds ratio is a classical measure of association for $2\times 2$ contingency tables; see, for example, Everitt [11] (Section 2.8). Equivalently, the condition $p_{11}^j{p}_{22}^j=p_{12}^j{p}_{21}^j$ corresponds to the classical no-association case in treatment arm j, which has long been studied as a testing problem for $2\times 2$ tables [12] (Section 33.16). For multiple $2\times 2$ tables, such as those arising after stratification by treatment arm, the treatment-specific log odds ratio is a standard way to describe conditional association. The closely related problems of partial association and conditional independence in stratified $2\times 2$ tables have been studied in the classical contingency-table literature; see Birch [15] and Kendall and Stuart [12] (Section 33.62). These formulations are also closely connected to the common-odds-ratio, or homogeneous-association, setting used for multiple $2\times 2$ tables. This classical formulation provides a natural justification for using the odds ratio as an association measure for the two binary endpoints. Following its use in bivariate binary-endpoint designs such as Conaway and Petroni [6], we use the odds ratio to parameterize the association between efficacy and safety.

In the present paper, we adopt the working assumption of a known common odds ratio across the two treatment arms, namely

$${\varphi }_0={\varphi }_1=\varphi .$$

This common-odds-ratio assumption is not a mathematical necessity but a design simplification that allows the four cell probabilities in each treatment arm to be determined by the two marginal probabilities $p_e^j$ and $p_s^j$ together with a common association parameter $\varphi$. It is also related to the classical common-association framework for several $2\times 2$ contingency tables, often expressed in terms of the log odds ratio; see Kendall and Stuart [12] (Section 33.62). In applications, $\varphi$ may be specified using prior studies, pilot data, or sensitivity analyses. The value of $\varphi$ has a direct interpretation in terms of the association between the two binary endpoints. When $\varphi =1$, the efficacy and safety endpoints are independent within a treatment arm, corresponding to the no-association case discussed above. When $\varphi >1$, the two endpoints are positively associated: patients who achieve efficacy are more likely to also achieve safety, and patients who fail to achieve efficacy are more likely to also fail to achieve safety. Equivalently, for fixed marginal probabilities, larger values of $\varphi$ place more probability mass on the concordant cells $(1,1)$ and $(0,0)$ and less probability mass on the discordant cells $(1,0)$ and $(0,1)$. When $0<\varphi <1$, the two endpoints are negatively associated, indicating a tendency toward discordant outcomes; for example, achieving efficacy may be less likely to occur together with achieving safety. The boundary case $\varphi =0$ represents an extreme form of negative association and is mainly included for sensitivity assessment rather than as a typical clinical setting. Two situations are considered:

Case 1: $\varphi =1$ (independence) When the two endpoints are independent, the joint distribution of $(X_e,X_s)$ factorizes as

$$P(X_e=x_e,X_s=x_s\mid p_e,p_s,\varphi =1)=b(n,p_e,x_e)b(n,p_s,x_s),$$

where $b(·)$ denotes the binomial pmf.

Case 2: $\varphi \ne 1$ (dependence) For specified $p_e$, $p_s$, and $\varphi$, Bryant and Day [5] derived the corresponding four cell probabilities:

$$p_{11}={\displaystyle \frac{a-\sqrt{a^2+d}}{2(\varphi -1)}},$$

(4)

$$p_{12}=p_e-p_{11},$$

(5)

$$p_{21}=p_s-p_{11},$$

(6)

$$p_{22}=1-p_{11}-p_{12}-p_{21},$$

(7)

with

$$a=1+(\varphi -1)(p_e+p_s),d=-4\varphi (\varphi -1)p_e{p}_s.$$

Given these cell probabilities, the joint probability of $(X_e,X_s)$ is

$$\begin{array}{c}\hfill P(X_e=x_e,X_s=x_s\mid p_e,p_s,\varphi )={\displaystyle \sum _{i=max(0,x_e+x_s-n)}^{min(x_e,x_s)}}{\displaystyle \frac{n!}{i!(x_e-i)!(x_s-i)!(n-x_e-x_s+i)!}}\\ \hfill \times p_{11}^i{p}_{12}^{x_e-i}{p}_{21}^{x_s-i}{p}_{22}^{n-x_e-x_s+i}.\end{array}$$

(8)

Remark 1.

When $\varphi =1$, expression (8) continues to hold, with the simplifying identity

$$p_{11}=p_e{p}_s.$$

3. Fixed Sample Size Design

Consider fixed design constants n, e, and s. The decision rule is constructed as follows. Here, n denotes the sample size per treatment arm, while e and s are positive integer thresholds representing the minimum required excess numbers of successes for efficacy and safety, respectively, in the experimental treatment compared with the control. These thresholds are chosen together with n so that the resulting test satisfies the desired type I error and power requirements.

Procedure H:

Collect n observations from each of the two treatments (the control and the new treatment). For treatment i $(i=0,1)$, let $X_e^i$ and $X_s^i$ denote the observed numbers of successes for the efficacy and safety endpoints, respectively. With positive thresholds e and s, the rule in Procedure H is given by:

• Reject the null hypothesis if both inequalities $X_e^1-X_e^0\ge e$ and $X_s^1-X_s^0\ge s$ hold.
• Otherwise (i.e., if either $X_e^1-X_e^0<e$ or $X_s^1-X_s^0<s$), retain the null hypothesis.

Probability Requirements:

The design constants n, e, and s for Procedure H are chosen so that the procedure meets the following probabilistic criteria:

$$\underset{H_0}{sup}P\left(X_e^1-X_e^0\ge e,X_s^1-X_s^0\ge s|\varphi \right)\le \alpha ,$$

(9)

$$\underset{H_1}{inf}P\left(X_e^1-X_e^0\ge e,X_s^1-X_s^0\ge s|\varphi \right)\ge 1-\beta .$$

(10)

The left-hand side of (9) represents the probability of incorrectly rejecting $H_0$ and is therefore controlled at the type I error level $\alpha$. Likewise, the left-hand side of (10) corresponds to the probability of correctly declaring the new treatment superior, which equals the power $1-\beta$ under $H_1$.

To guarantee a meaningful distinction between the null and alternative spaces, we require a minimum effect separation that specifies how far the true parameters under $H_1$ must lie from those under $H_0$. In particular, we assume the alternative satisfies

$$p_e^1\ge p_e^0+{\delta }_e,p_s^1\ge p_s^0+{\delta }_s,$$

(11)

where ${\delta }_e$ and ${\delta }_s$ denote the smallest clinically relevant differences in efficacy and safety. These effect-size constraints ensure that the testing procedure can reliably discriminate between $H_0$ and $H_1$ when the new treatment exhibits meaningful improvement.

• We now derive the values of the procedure parameters such that the probability constraints (9) and (10) are satisfied.

Theorem 1

(Monotonicity of the rejection probability). Let ϕ denote the odds ratio. The rejection probability

$$P\left(\mathit{Reject}{H}_0\right)=P\left(X_e^1-X_e^0\ge e,X_s^1-X_s^0\ge s|\varphi \right)$$

satisfies the following monotonicity properties with respect to the marginal efficacy and safety probabilities.

1.

Non-increasing in ${\mathit{p}}_{\mathit{e}}^{\mathbf{0}}$: For fixed $p_e^1,p_s^0,p_s^1,$ and ϕ,

$$P\left(\mathit{Reject}{H}_0\right)\mathit{decreases}(\mathit{or}\mathit{remains}\mathit{constant})\mathit{as}{p}_e^0\mathit{increases}.$$

2.

Non-increasing in ${\mathit{p}}_{\mathit{s}}^{\mathbf{0}}$: For fixed $p_e^0,p_e^1,p_s^1,$ and ϕ,

$$P\left(\mathit{Reject}{H}_0\right)\mathit{decreases}(\mathit{or}\mathit{remains}\mathit{constant})\mathit{as}{p}_s^0\mathit{increases}.$$

3.

Non-decreasing in ${\mathit{p}}_{\mathit{e}}^{\mathbf{1}}$: For fixed $p_e^0,p_s^0,p_s^1,$ and ϕ,

$$P\left(\mathit{Reject}{H}_0\right)\mathit{increases}(\mathit{or}\mathit{remains}\mathit{constant})\mathit{as}{p}_e^1\mathit{increases}.$$

4.

Non-decreasing in ${\mathit{p}}_{\mathit{s}}^{\mathbf{1}}$: For fixed $p_e^0,p_e^1,p_s^0,$ and ϕ,

$$P\left(\mathit{Reject}{H}_0\right)\mathit{increases}(\mathit{or}\mathit{remains}\mathit{constant})\mathit{as}{p}_s^1\mathit{increases}.$$

The detailed proof is provided in the Appendix A.

Theorem 1 immediately implies that, when $p_e^0$ and $p_s^0$ are known, the power is minimized at the boundary of the effect-size constraints, namely when

$$p_e^1=p_e^0+{\delta }_e,p_s^1=p_s^0+{\delta }_s.$$

Theorem 2

(Least favorable configuration for power control under $\varphi =1$). When the odds ratio satisfies $\varphi =1$ and the critical values satisfy $e\le n{\delta }_e$ and $s\le n{\delta }_s$, the minimal power under the alternative is attained at the boundary of the effect-size constraints, specifically when

$$p_e^1=p_e^0+{\delta }_e,p_s^1=p_s^0+{\delta }_s,p_e^0={\displaystyle \frac{1-{\delta }_e}{2}},p_s^0={\displaystyle \frac{1-{\delta }_s}{2}}.$$

The proof is provided in Appendix A.

Theorem 3.

If the null-hypothesis parameters $p_e^0$ and $p_s^0$ are fixed and known, then the maximal type I error is attained at one of the following configurations:

$$p_e^1=1,p_s^1=p_s^0,$$

or

$$p_e^1=p_e^0,p_s^1=1.$$

This result follows directly from the monotonicity properties in Theorem 1. To attain the maximal type I error, the rejection probability should be made as large as possible while the null hypothesis remains true. Since the rejection probability is non-decreasing in $p_e^1$ and $p_s^1$, the maximum occurs on the boundary of the null hypothesis: either $p_s^1=p_s^0$ with $p_e^1$ as large as possible or $p_e^1=p_e^0$ with $p_s^1$ as large as possible.

Consequently, Theorem 3 implies that the constraint in (9) can be equivalently expressed as

$$\begin{array}{cc}\hfill \max & \{P\left(X_e^1-X_e^0\ge e,X_s^1-X_s^0\ge s|p_e^1=p_e^0,p_s^1=1,\varphi \right),\hfill \\ & P\left(X_e^1-X_e^0\ge e,X_s^1-X_s^0\ge s|p_e^1=1,p_s^1=p_s^0,\varphi \right)\}\le \alpha .\hfill \end{array}$$

(12)

Theorem 4

(Least favorable configuration for type I error control under $\varphi =1$). When the odds ratio satisfies $\varphi =1$ and the critical values satisfy $e,s>0$, the maximal type I error under the null hypothesis is attained at one of two configurations,

$$p_e^1=p_e^0={\displaystyle \frac{1}{2}},p_s^1=1,p_s^0=0,$$

or, symmetrically,

$$p_s^1=p_s^0={\displaystyle \frac{1}{2}},p_e^1=1,p_e^0=0.$$

The proof is provided in the Appendix A.

The above results characterize the least favorable configurations for both power and type I error control in Procedure H. Theorem 1 shows that the rejection probability is monotone in the marginal efficacy and safety probabilities. This monotonicity reduces the search for worst-case configurations to boundary points of the parameter space. Theorem 2 identifies the least favorable alternative for power calculation when $\varphi =1$. Theorems 3 and 4 characterize the worst-case null configurations for type I error control.

A key implication is that, when $\varphi =1$, the efficacy and safety endpoints are independent. In this case, a universal least favorable configuration (LFC) exists. Therefore, the design parameters $(n,e,s)$ can be determined without specifying the nuisance parameters $p_e^0$ and $p_s^0$. By contrast, when $\varphi \ne 1$, the two endpoints are dependent. A universal LFC is generally unavailable in this setting. Thus, determining $(n,e,s)$ requires the baseline values $p_e^0$ and $p_s^0$ to be specified.

If $p_e^0$ and $p_s^0$ are known in advance, the resulting design is less conservative. It may also require a substantially smaller sample size. This reduction is illustrated in the numerical results reported in the subsequent tables.

4. Tables and Discussion

In this section, we report the design parameters required to implement the proposed fixed-sample procedure. Throughout, we assume a common association structure between the two binary endpoints across the control and the tested treatment; i.e., all the treatments share the same odds ratio $\varphi$. Under this assumption, bivariate binary outcomes can be characterized by the marginal success probabilities and the odds ratio, which jointly determine the $2\times 2$ cell probabilities used in simulation.

We consider the following configurations: $\varphi \in \{0,1,2,4,8\}$, ${\delta }_e\in \{0.1,0.2,0.3\}$, and ${\delta }_s\in \{0.1,0.2,0.3\}$. The target operating characteristics are specified by

$$(\mathrm{Power},\mathrm{Type}\mathrm{I}\mathrm{error})\in \left\{\right(0.75,0.15),(0.85,0.15\left)\right\}.$$

The type I error level of 0.15 is used here for illustrative purposes in an exploratory early-phase setting, where a less stringent error level may be considered acceptable for screening promising treatments. In practice, the choice of type I error rate and power should be determined according to the clinical objective, regulatory context, and input from clinical investigators.

For each configuration, we determine the minimum required sample size per treatment, n, together with the corresponding critical values $(e,s)$ such that the probability constraints in (9) and (10) are satisfied. The reported operating characteristics in the tables are estimated by Monte Carlo simulation using 100,000 replications for each configuration. For example, when $\varphi =1$, ${\delta }_e={\delta }_s=0.2$, and the target operating characteristics are $(\mathrm{Power},\mathrm{Type}\mathrm{I}\mathrm{error})=(0.75,0.15)$, we evaluate candidate sample sizes n sequentially. For each n, all feasible integer threshold pairs $(e,s)$ are checked by Monte Carlo estimation of the corresponding rejection probabilities. The smallest sample size for which at least one pair $(e,s)$ satisfies both requirements is selected. In this setting, based on 100,000 Monte Carlo replications, $n=63$ with $(e,s)=(7,7)$ gives an estimated power of $0.75241$ and an estimated type I error of $0.12335$, so it satisfies the desired constraints.

The simulation procedure used to determine $(n,e,s)$ can be summarized as follows:

• Specify the target type I error level $\alpha$, target power $1-\beta$, effect sizes ${\delta }_e$ and ${\delta }_s$, odds ratio $\varphi$, and, when needed, baseline control rates $p_e^0$ and $p_s^0$.
• For a candidate sample size n, enumerate all feasible positive integer threshold pairs $(e,s)$.
• For each pair $(e,s)$, estimate the type I error and power using 100,000 Monte Carlo replications under the corresponding LFC or GLFC.
• Retain the threshold pairs $(e,s)$ that satisfy both the type I error constraint and the power constraint.
• Increase n sequentially until at least one feasible pair $(e,s)$ is found.
• Select the smallest such n. If multiple threshold pairs satisfy the constraints for this minimum n, choose the pair with the smallest thresholds $(e,s)$.

The numerical study is divided into two cases.

Case 1: $\varphi =1$ with unknown baseline control rates. When $\varphi =1$, the efficacy and safety endpoints are independent. In this case, we assume that the baseline control rates $p_e^0$ and $p_s^0$ are unknown. By Theorem 2, the minimum power under the alternative is attained at a specific least favorable configuration when $\varphi =1$. Similarly, by Theorem 4, the maximal type I error under the null is attained at a least favorable configuration. These least favorable configurations together make it possible to calibrate the design parameters $(n,e,s)$ without specifying $p_e^0$ and $p_s^0$. The resulting designs are reported in

Table 2. Target power $=0.75$, target type I error $=0.15$, $\varphi =1$, and $p_e^0,p_s^0$ unknown.

${\mathit{\delta }}_{\mathit{e}}$	${\mathit{\delta }}_{\mathit{s}}$	${\mathit{p}}_{\mathit{e}}^0$	${\mathit{p}}_{\mathit{s}}^0$	n	e	s	Power	Type I Error
0.1	0.1	0.45	0.45	235	12	12	0.75452	0.14436
	0.2	0.45	0.40	156	10	10	0.75276	0.14103
	0.3	0.45	0.35	154	10	10	0.75091	0.13948
0.2	0.1	0.40	0.45	156	10	10	0.75417	0.14103
	0.2	0.40	0.40	63	7	7	0.75241	0.12335
	0.3	0.40	0.35	46	6	6	0.75714	0.12566
0.3	0.1	0.35	0.45	153	16	10	0.75072	0.13869
	0.2	0.35	0.40	46	6	6	0.75521	0.12566
	0.3	0.35	0.35	29	5	5	0.76641	0.11852

and

Table 3. Target power $=0.85$, target type I error $=0.15$, $\varphi =1$, and $p_e^0,p_s^0$ unknown.

${\mathit{\delta }}_{\mathit{e}}$	${\mathit{\delta }}_{\mathit{s}}$	${\mathit{p}}_{\mathit{e}}^0$	${\mathit{p}}_{\mathit{s}}^0$	n	e	s	Power	Type I Error
0.1	0.1	0.45	0.45	312	14	14	0.85141	0.13987
	0.2	0.45	0.4	225	12	12	0.85153	0.13912
	0.3	0.45	0.35	224	12	17	0.85112	0.13859
0.2	0.1	0.4	0.45	225	12	12	0.85081	0.13912
	0.2	0.4	0.4	76	7	7	0.85473	0.14583
	0.3	0.4	0.35	56	6	6	0.85093	0.14930
0.3	0.1	0.35	0.45	224	14	12	0.85054	0.13859
	0.2	0.35	0.4	56	6	6	0.85183	0.14930
	0.3	0.35	0.35	34	5	5	0.85353	0.13750

, corresponding to the two target operating characteristics $(0.75,0.15)$ and $(0.85,0.15)$, respectively.

Case 2: known baseline control rates. We next consider the case where the baseline control rates are known, with

$$p_e^0,p_s^0\in \{0.2,0.4,0.6\},$$

yielding all possible combinations of $(p_e^0,p_s^0)$. For this case, we fix

$${\delta }_e={\delta }_s=0.2$$

and consider $\varphi \in \{0,1,2,4,8\}$. Using the generalized least favorable configuration (GLFC) characterized by Theorems 1 and 3, we determine the corresponding minimum sample size n and critical values $(e,s)$ for each parameter setting. The results are presented in

Table 4. Target power $=0.75$, target type I error $=0.15$, ${\delta }_e={\delta }_s=0.2$, and $\varphi \in \{0,1,2,4,8\}$.

${\mathit{p}}_{\mathit{s}}^0$	${\mathit{p}}_{\mathit{e}}^0$	$\mathit{\varphi }$	n	e	s	Power	Type I Error
0.2	0.2	0	48	5	5	0.75495	0.12505
		1	47	5	5	0.75505	0.12295
		2	46	5	5	0.75396	0.11976
		4	45	5	5	0.75135	0.11693
		8	45	5	5	0.76221	0.11774
	0.4	0	53	6	5	0.75059	0.13623
		1	52	6	5	0.75415	0.13770
		2	51	6	5	0.75134	0.13305
		4	50	6	5	0.75022	0.12924
		8	50	6	5	0.75559	0.13262
	0.6	0	52	6	5	0.76262	0.13648
		1	50	6	5	0.75010	0.13051
		2	50	6	5	0.75735	0.13080
		4	50	6	5	0.76275	0.13154
		8	49	6	5	0.75629	0.12770
0.4	0.2	0	54	5	6	0.76188	0.13951
		1	52	5	6	0.75205	0.13523
		2	52	5	6	0.76051	0.13511
		4	51	5	6	0.75904	0.13437
		8	50	5	6	0.75575	0.13115
	0.4	0	58	6	6	0.75214	0.14951
		1	57	6	6	0.75607	0.14663
		2	56	6	6	0.75733	0.14505
		4	55	6	6	0.75392	0.14443
		8	54	6	6	0.75765	0.14107
	0.6	0	57	6	6	0.76124	0.14765
		1	55	6	6	0.75283	0.14256
		2	54	6	6	0.75011	0.14013
		4	54	6	6	0.76298	0.14010
		8	53	6	6	0.75934	0.13641
0.6	0.2	0	52	5	6	0.76290	0.13554
		1	50	5	6	0.75125	0.13153
		2	50	5	6	0.75556	0.13236
		4	49	5	6	0.75186	0.12765
		8	49	5	6	0.75558	0.12815
	0.4	0	57	6	6	0.76215	0.14689
		1	55	6	6	0.75404	0.14279
		2	55	6	6	0.76194	0.14152
		4	54	6	6	0.75950	0.14007
		8	53	6	6	0.76019	0.13837
	0.6	0	55	6	6	0.76030	0.14275
		1	54	6	6	0.76235	0.13779
		2	53	6	6	0.75781	0.13831
		4	52	6	6	0.75438	0.13652
		8	51	6	6	0.75506	0.13410

and

Table 5. Target power $=0.85$, target type I error $=0.15$, ${\delta }_e={\delta }_s=0.2$, and $\varphi \in \{0,1,2,4,8\}$.

${\mathit{p}}_{\mathit{s}}^0$	${\mathit{p}}_{\mathit{e}}^0$	$\mathit{\varphi }$	n	e	s	Power	Type I Error
0.2	0.2	0	58	5	5	0.85928	0.14710
		1	57	5	5	0.85588	0.14647
		2	56	5	5	0.85261	0.14387
		4	55	5	5	0.85124	0.14296
		8	55	5	5	0.85713	0.14121
	0.4	0	71	7	6	0.85598	0.13195
		1	70	7	6	0.85405	0.13131
		2	70	7	6	0.85598	0.13254
		4	69	7	6	0.85427	0.12807
		8	68	7	6	0.85286	0.12577
	0.6	0	68	7	6	0.85027	0.12720
		1	67	7	6	0.85180	0.12478
		2	67	7	6	0.85443	0.12593
		4	67	7	6	0.85791	0.12723
		8	66	7	6	0.85109	0.12466
0.4	0.2	0	71	6	7	0.85386	0.13086
		1	70	6	7	0.85311	0.13067
		2	70	6	7	0.85510	0.13046
		4	69	6	7	0.85481	0.13033
		8	68	6	7	0.85297	0.12787
	0.4	0	77	7	7	0.85411	0.14499
		1	76	7	7	0.85368	0.14162
		2	75	7	7	0.85262	0.13935
		4	74	7	7	0.85120	0.13735
		8	73	7	7	0.85252	0.13745
	0.6	0	74	7	7	0.85361	0.13814
		1	73	7	7	0.85441	0.13641
		2	72	7	7	0.85028	0.13375
		4	72	7	7	0.85345	0.13483
		8	72	7	7	0.85966	0.13623
0.6	0.2	0	68	6	7	0.85159	0.12723
		1	67	6	7	0.85175	0.12547
		2	67	6	7	0.85383	0.12457
		4	67	6	7	0.85748	0.12530
		8	66	6	7	0.85126	0.12497
	0.4	0	74	7	7	0.85541	0.13949
		1	73	7	7	0.85350	0.13631
		2	72	7	7	0.85037	0.13508
		4	72	7	7	0.85189	0.13507
		8	71	7	7	0.85313	0.13311
	0.6	0	72	7	7	0.85746	0.13336
		1	71	7	7	0.85887	0.13302
		2	70	7	7	0.85303	0.13095
		4	69	7	7	0.85038	0.12930
		8	69	7	7	0.85707	0.13020

, corresponding to $(\mathrm{Power},\mathrm{Type}\mathrm{I}\mathrm{error})=(0.75,0.15)$ and $(0.85,0.15)$, respectively.

The design parameters $(n,e,s)$ are obtained via simulation. Specifically, for each candidate value of n, we search over feasible integer thresholds $(e,s)$ and select the smallest n for which at least one pair $(e,s)$ meets the constraints. When multiple $(e,s)$ pairs satisfy the requirements for the same minimal n, we adopt the pair that yields the largest empirical power while maintaining the type I error constraint. The resulting parameters provide a direct lookup table for implementing the procedure under the considered settings.

When the baseline control rates $p_e^0$ and $p_s^0$ are unknown, Table 2 and Table 3 summarize, under $\varphi =1$, the minimum per-treatment sample size n and the corresponding critical values $(e,s)$ that achieve the desired operating characteristics. As expected, larger effect thresholds (i.e., larger ${\delta }_e$ and/or ${\delta }_s$) generally lead to smaller required sample sizes since stronger separation between the null and alternative hypotheses makes it easier to satisfy the power constraint while controlling type I error. These tabulated values enable straightforward implementation of the procedure without re-running the design search for each new study.

In addition, Table 4 and Table 5 report the case where the baseline control rates $p_e^0$ and $p_s^0$ are known in advance and ${\delta }_e={\delta }_s=0.2$. The results show that, as the odds ratio $\varphi$ increases, the required sample size n is non-increasing across all the reported configurations, and the overall variation in n is relatively small. This indicates that the proposed design is not highly sensitive to the dependence parameter $\varphi$. The sensitivity results in Table 4 and Table 5 further suggest that the proposed design is relatively robust to the choice of $\varphi$ over a clinically relevant range. In particular, the required sample size changes only slightly for moderate values of $\varphi$, indicating that moderate misspecification of the working odds ratio has limited impact on the resulting design. The case $\varphi =0$ represents an extreme boundary scenario and is included mainly for sensitivity assessment rather than as a typical clinical setting. Moreover, for the setting ${\delta }_e={\delta }_s=0.2$, the sample sizes in Table 2 and Table 3 are generally no smaller than those in Table 4 and Table 5, reflecting the fact that the designs obtained under unknown baseline control rates are more conservative. In particular, in Table 2, the required sample size is $n=63$, which exceeds all the corresponding values reported in Table 4 for ${\delta }_e={\delta }_s=0.2$. In Table 3, the required sample size is $n=76$, which is larger than nearly all the corresponding values in Table 5; the only exception is one configuration with $n=77$ when $\varphi =0$ and $p_e^0=p_s^0=0.4$. Overall, these findings suggest that the values reported in Table 2 and Table 3 provide a reasonably conservative design benchmark. Therefore, when $p_e^0$ and $p_s^0$ are unknown and $\varphi \ne 1$, or even when $\varphi$ itself is unknown, the design calibrated under the $\varphi =1$ least favorable configuration can still serve as a practical and reliable approximation for trial planning.

To further examine the robustness of the proposed design to possible misspecification of the odds ratio $\varphi$, we conducted an additional sensitivity calculation using the first configuration in Table 3, where $p_e^0=p_s^0=0.2$. For this setting, the design obtained under $\varphi =8$ is $(n,e,s)=(45,5,5)$. We then fixed this design and estimated the corresponding power and type I error rate under several different values of the true odds ratio. When $\varphi =0$, the estimated power is $0.7132$ and the estimated type I error rate is $0.11864$; when $\varphi =1$, the estimated power is $0.73223$ and the estimated type I error rate is $0.11806$; and, when $\varphi =2$, the estimated power is $0.74129$ and the estimated type I error rate is $0.11623$. The cases $\varphi =4$ and $\varphi =8$ lead to the same design parameters in the table and therefore do not require separate design calibration. These results suggest that, when the misspecification of $\varphi$ is moderate, its impact on the rejection probabilities is relatively small. Even in the boundary case $\varphi =0$, which represents an extreme scenario on the odds-ratio scale, the type I error rate remains below the nominal level $0.15$, while the reduction in power is moderate. Moreover, when $p_e^0$ and $p_s^0$ are unknown and the design from Table 2 with ${\delta }_e={\delta }_s=0.2$ is used, namely $(n,e,s)=(63,7,7)$, the estimated power and type I error rate under $\varphi =0$ and $p_e^0=p_s^0=0.2$ are $0.77839$ and $0.07363$, respectively. Thus, when there is substantial uncertainty about the value of $\varphi$, the designs reported in Table 2 and Table 3 can serve as conservative and practical choices for trial planning.

The design tables also reveal several structural patterns among the design parameters. First, as expected, a higher target power generally requires a larger sample size because stronger probability guarantees require more information from each treatment arm. Second, the clinically meaningful effect sizes ${\delta }_e$ and ${\delta }_s$ have a substantial impact on the required sample size. When either the efficacy threshold or the safety threshold becomes larger, the required sample size tends to decrease since larger treatment differences are easier to detect. Conversely, smaller values of ${\delta }_e$ or ${\delta }_s$ lead to more demanding designs and therefore require larger sample sizes. Third, for fixed baseline rates and effect-size thresholds, the required sample size tends to be non-increasing as the odds ratio $\varphi$ increases. This pattern indicates that a stronger positive association between efficacy and safety can reduce the amount of information needed to jointly demonstrate improvement on both endpoints. Overall, the numerical results are consistent with intuition: sample size is most sensitive to target power, the type I error requirement, and clinically meaningful thresholds ${\delta }_e$ and ${\delta }_s$, while the dependence parameter $\varphi$ appears to have little impact over the practical range considered in the tables.

5. Example

This example considers an experimental trial involving an immunotherapy-based treatment for elderly patients (≥75 years old) diagnosed with advanced non-small-cell lung cancer (NSCLC). The trial compares the immunotherapy strategy, PD1-A (anti-PD-1 monotherapy), against the standard chemotherapy regimen, consisting of carboplatin and pemetrexed, which serves as the control treatment.

The goal of the trial is to determine whether PD1-A demonstrates superiority over the control treatment in terms of both efficacy and safety. Specifically, the study targets an improvement of at least $0.30$ in the response rate and a reduction of at least $0.20$ in high-grade toxicity, corresponding to ${\delta }_e=0.30$ and ${\delta }_s=0.20$. If PD1-A fails to show meaningful improvement over the control, the standard chemotherapy will be selected.

Under this setting, the least favorable configuration for rejecting the null hypothesis occurs when

$$p_e^0={\displaystyle \frac{1-{\delta }_e}{2}},p_s^0={\displaystyle \frac{1-{\delta }_s}{2}},p_e^1={\displaystyle \frac{1+{\delta }_e}{2}},p_s^1={\displaystyle \frac{1+{\delta }_s}{2}}.$$

By Theorem 4, the least favorable configuration (LFC) for the type I error is attained when

$$p_e^1=p_e^0={\displaystyle \frac{1}{2}},p_s^1=1,p_s^0=0,$$

or, symmetrically,

$$p_s^1=p_s^0={\displaystyle \frac{1}{2}},p_e^1=1,p_e^0=0.$$

When the probability constraints are set at power $=0.85$ and type I error $=0.15$, Table 2 indicates that the fixed-sample procedure requires $n=56$ observations per treatment arm, with associated critical values $e=6$ and $s=6$. Therefore, the total number of observations required to meet the design criteria is $2\times 56=112$.

After the trial is completed, the procedure is applied by comparing the observed numbers of efficacy and safety successes between the PD1-A arm and the control arm. Let $X_e^1$ and $X_s^1$ denote the observed efficacy and safety successes in the PD1-A arm, and let $X_e^0$ and $X_s^0$ denote the corresponding observed successes in the control arm. With the selected critical values $e=6$ and $s=6$, the null hypothesis is rejected only if both $X_e^1-X_e^0\ge 6$ and $X_s^1-X_s^0\ge 6$. Otherwise, the trial does not provide sufficient evidence that PD1-A is superior to the control treatment in both efficacy and safety. For instance, if the observed differences are $X_e^1-X_e^0=7$ and $X_s^1-X_s^0=6$, then both criteria are met and the null hypothesis is rejected; if either difference is less than 6, the null hypothesis is retained.

6. Conclusions

This paper develops an exact frequentist fixed sample size hypothesis testing procedure for two-arm clinical trials with two binary endpoints, motivated by settings in which a new treatment must demonstrate improvement in both efficacy and safety relative to a concurrent control. Instead of relying on a composite endpoint, the proposed method evaluates the two endpoints jointly and rejects the null hypothesis only when the observed treatment differences exceed pre-specified paired thresholds on both dimensions. By modeling the joint efficacy–safety outcomes through a four-category multinomial distribution, the procedure allows calculation of rejection probabilities and provides a clear framework for benefit–risk evaluation.

The theoretical results characterize the least favorable configurations for both power and type I error control. The monotonicity properties of the rejection probability reduce the search for worst-case configurations to the boundary of the parameter space. When the two endpoints are independent ($\varphi =1$), the least favorable configurations can be identified explicitly, allowing the design parameters $(n,e,s)$ to be determined without specifying the baseline control rates $p_e^0$ and $p_s^0$. When $\varphi \ne 1$, a universal least favorable configuration is generally unavailable, and calibration requires the baseline control rates to be specified. The numerical results further suggest that the required sample size is not highly sensitive to the odds ratio across the settings considered.

The proposed framework can be implemented through design tables that map the target power, type I error rate, and clinically meaningful effect sizes to the required sample size and decision thresholds. These tables provide a convenient tool for study planning and reduce the need for repeated design searches in new applications. In particular, when the baseline control rates are unknown, designs calibrated under the $\varphi =1$ least favorable configuration may provide a practical approximation even when the true odds ratio differs from 1.

Several extensions of the proposed framework are possible. One limitation of the proposed framework is that the odds ratio $\varphi$ is treated as a pre-specified working design parameter and is assumed to be common across the control and experimental treatments. Although sensitivity analyses indicate that the design is relatively robust over practical ranges of $\varphi$, future work may consider adaptive or data-driven approaches that allow different association structures between efficacy and safety across treatment arms. Second, although the present study focuses on a fixed-sample design, the relatively large sample sizes required in some settings suggest that sequential or curtailed versions of the proposed procedure may provide meaningful gains in efficiency.

Overall, the proposed procedure offers a rigorous and interpretable approach for two-arm trials with co-primary binary endpoints. By preserving separate assessments of efficacy and safety while controlling type I error and maintaining adequate power, the method provides a useful alternative to composite-endpoint analyses and helps to bridge the gap between early-phase bivariate endpoint designs and comparative two-arm confirmatory testing.