Confidence Intervals for Two Proportions—A Generalized Estimation and Information Assessment

Abstract

This paper develops profile score confidence intervals (i.e., z-standard intervals) by inverting orthogonalized score estimators for comparing two independent binomial proportions in rate difference, rate ratio, or odds ratio and utilizes a generalized estimation framework of Vos and Wu (Inf. Geom. 8: 99–123, 2025) to evaluate different confidence interval methods. The orthogonalized score estimators dominate other generalized estimators in $\Lambda$-efficiency for distinguishing parameter values around the truth, so the z-standard intervals become more efficient and acquire coverage closer to the nominal level than other types of confidence intervals. In addition, the degrees of freedom and small-sample corrections (applied to the profile nuisance parameter estimates) are expected to improve the coverage of the z-standard intervals and help maintain them above the nominal level. Computational algorithms are developed to find the z-standard intervals using R’s polyroot function. Numerical studies are conducted to compare both coverage and endpoints of different types of confidence intervals.

1. Introduction

Building on extensive work in the comparative analysis of two independent binomial proportions (see, e.g., [1,2,3]), this paper presents a novel framework for assessing different confidence interval methods. Three parameters of interest—the rate difference (RD), rate ratio (RR), and odds ratio (OR)—are commonly used for inference in two-by-two comparative trials and the relative performance of their corresponding confidence intervals requires a careful evaluation. Our proposed approach uses a generalized estimation and information assessment developed by Vos and Wu [4] to provide a new perspective, specifically on the properties of the profile score confidence intervals. The analysis considers the classic setting where data are represented in a two-by-two contingency table (e.g., Table 1) with fixed group sample sizes $n_1$ and $n_2$.

Table 1. 2 × 2 contingency table.

	Sample 1	Sample 2	Total
Success	$x_1$	$x_2$	k
Failure	$n_1-x_1$	$n_2-x_2$	$N-k$
Total	$n_1$	$n_2$	N

Due to the discrete nature of the data, existing methods, such as the classic Wald confidence interval, often generate wide intervals to ensure coverage. Sometimes, corrections for small samples, such as the well-known plus-four (or Wilson) confidence interval covered in most introductory statistics textbooks, are implemented and further exacerbate the over-coverage issue. In this paper, we propose the profile score confidence intervals by inverting the orthogonalized score estimators and demonstrate their superior information and coverage properties.

1.1. Generalized Estimation and Information Framework

Following the generalized estimation framework of Vos and Wu [4], we consider a two-parameter model defined on a manifold of distributions. For the Bernoulli distribution with success parameter $p\in (0,1)$, the sample space for the number of successes in $n\ge 1$ trials is $\left\{0,1,2,\dots ,n\right\}$ and the sampling measure is $f_n(x;p)=\left({\displaystyle \genfrac{}{}{0pt}{}{n}{x}}\right)p^x{(1-p)}^{n-x}$. Thus, for the data in Table 1, the sample space is $\mathcal{X}=\left\{0,1,\dots ,n_1\right\}\times \left\{0,1,\dots ,n_2\right\}$ and the two-parameter manifold of the joint distributions of the numbers of successes from the two samples is

$$M=\left\{m:m(x_1,x_2)=f_{n_1}(x_1;p_1)f_{n_2}(x_2;p_2),0<p_1,p_2<1\right\}.$$

(1)

In general, let M be a two-dimensional manifold of distributions on a common support $\mathcal{X}$, and let $\underline{\theta }:M\to \underline{\Theta }\subset {\mathbb{R}}^2$ be a global parameterization of the distributions $m\in M$ such that $\underline{\theta }={(\theta ,\tau )}^t$. For example, ${(\theta ,\tau )}^t$ can be any reparameterization of ${(p_1,p_2)}^t$ in the Bernoulli example (1). For any distribution $m^{\prime }\in M$, $M_{{\tau }^{\prime }}=\{m\in M:\tau \left(m\right)=\tau \left(m^{\prime }\right)\}$ and $M_{{\theta }^{\prime }}=\{m\in M:\theta \left(m\right)=\theta \left(m^{\prime }\right)\}$ are regular submanifolds, where ${\tau }^{\prime }=\tau \left(m^{\prime }\right)$ and ${\theta }^{\prime }=\theta \left(m^{\prime }\right)$. The parameter space for the submanifold $M_{{\tau }^{\prime }}$ is ${\underline{\Theta }}_{{\tau }^{\prime }}=\{\theta \in \mathbb{R}:{(\theta ,{\tau }^{\prime })}^t\in \underline{\Theta }\}$ and similarly, the parameter space for $M_{{\theta }^{\prime }}$ is ${\underline{\Theta }}_{{\theta }^{\prime }}=\{\tau \in \mathbb{R}:{({\theta }^{\prime },\tau )}^t\in \underline{\Theta }\}$.

In statistical applications, the two parameters $\theta$ and $\tau$ are usually assigned distinct inferential roles. The parameter of interest $\theta$ represents the attribute relevant to the research question, while the nuisance parameter $\tau$ is necessary to specify the full distribution but is not of direct inferential concern. A key objective is to ensure that inferences about both parameters are invariant to reparameterization. For example, if $\theta$ represents the log odds ratio, inferences should be invariant to a change in parameterization to the odds ratio or any reparameterization that are diffeomorphisms. This principle of parameter invariance is equally important for both the parameter of interest and the nuisance parameter. The inference for $\theta$ must also not change if a different nuisance parameterization is considered. Parameter invariance allows us to choose parameterizations that are convenient for computations.

A generalized estimator for $\theta$ is a real-valued function g with domain $\mathcal{X}\times \underline{\Theta }$ having the following properties. For any fixed value ${\tau }^{\prime }$ of the nuisance parameter and almost every $x\in \mathcal{X}$, $g(x,·,{\tau }^{\prime })$ is a smooth function on the parameter space ${\underline{\Theta }}_{{\tau }^{\prime }}$. To achieve nuisance parameterization invariance, we use the residual of g after projecting it onto the score function for the nuisance parameter, $\tilde{s}=\partial l/\partial \tau$. This projection yields the invariant estimator $g^{\perp }$, defined as:

$$g^{\perp }=g-E\left[g\tilde{s}\right]V{\left(\tilde{s}\right)}^{-1}\tilde{s},$$

where E and V denote the expectation and variance operators, respectively. This invariant estimator is then standardized to create ${\overline{g}}^{\perp }=V{\left(g^{\perp }\right)}^{-1/2}{g}^{\perp }$. For each distribution m, the function ${\overline{g}}_m^{\perp }={\overline{g}}^{\perp }\circ \underline{\theta }\left(m\right)$ represents a standardized distribution with mean of zero and variance of one.

According to Vos and Wu [4], the generalized estimator ${\overline{g}}^{\perp }$ possesses both distributional and geometric properties. For a given distribution $m\in M_{{\tau }^{\prime }}$, the distribution of ${\overline{g}}_m^{\perp }$ has a mean of zero and a variance of one, and is uncorrelated with the nuisance parameter score, ${\tilde{s}}_m$. Geometrically, within the submanifold $M_{{\tau }^{\prime }}$ (defined by a fixed nuisance parameter ${\tau }^{\prime }$), ${\overline{g}}^{\perp }$ and $\tilde{s}$ are orthogonal vector fields. This implies that for any point m on the one-dimensional submanifold $M_{{\tau }^{\prime }}$, ${\overline{g}}_m^{\perp }$ is a direction vector that is orthogonal to the tangent space of the nuisance parameter submanifold at that point.

When the generalized estimator g is the score for the parameter of interest, $s=\partial l/\partial \theta$, we use the notation ${\overline{s}}^{\perp }$ for ${\overline{g}}^{\perp }$. To compare K estimates, we can plot the functions ${\overline{g}}_k^{\perp }$ for $k=1,\dots ,K$ on the parameter space ${\underline{\Theta }}_{{\tau }^{\prime }}$ for a fixed nuisance parameter value ${\tau }^{\prime }$. This approach is particularly feasible in the binomial setting, where the finite number of sample outcomes allows for an exhaustive comparison. While the choice of ${\tau }^{\prime }$ may depend on the observed data, it must be held constant across all estimates during a given comparison.

While generalized estimators can serve as estimating functions in the sense of Liang and Zeger [5], they are fundamentally distinct. Generalized estimators offer direct means for constructing confidence intervals and conducting hypothesis tests, but more importantly, they provide a meaningful summary of the information captured by the data regarding different parameter values. The (nuisance orthogonalized Fisher) information for the parameter of interest, $\theta$, utilized by an estimator g is quantified by its standardized version, $\overline{g}$, as $\Lambda \left(g\right)=E^2[\partial \overline{g}/\partial \theta ]$ (or $\Lambda \left(g\right)=E^2[\partial {\overline{g}}^{\perp }/\partial \theta ]$ in the presence of nuisance parameters). When comparing generalized estimators using $\Lambda$-information, the Cramer-Rao lower bound for the variance of point estimators becomes the $\Lambda$-information upper bound. This upper bound is the orthogonalized Fisher information, which the orthogonalized score attains $\Lambda \left({\overline{s}}^{\perp }\right)$. There is a simple relationship between the $\Lambda$-information of g and the orthogonalized score

$$\Lambda \left(g\right)={\rho }^2\Lambda \left({\overline{s}}^{\perp }\right),$$

(2)

where $\rho =E\left[{\overline{g}}^{\perp }{\overline{s}}^{\perp }\right]$ is the correlation between ${\overline{g}}^{\perp }$ and ${\overline{s}}^{\perp }$. The $\Lambda$-efficiency of any g is the ratio of its $\Lambda$-information to the upper bound

$${\mathrm{Eff}}^{\Lambda }=\Lambda \left(g\right)/\Lambda \left({\overline{s}}^{\perp }\right)={\rho }^2,$$

where the last equation follows from (2). We leverage this generalized estimation framework to develop profile score estimators for the RD, RR, and OR, and demonstrate their optimality in terms of $\Lambda$-information.

1.2. Existing Confidence Interval Methods

In the two-by-two comparative trials, let $q_i=1-p_i$ for $i=1,2$. The maximum likelihood estimators (MLEs) for the probabilities are ${\widehat{p}}_i=x_i/n_i$ and ${\widehat{q}}_i=1-{\widehat{p}}_i$. The classic Wald confidence intervals rely on the asymptotic normality of the MLEs. For RR and OR, the Wald confidence intervals are given in the logarithm of these parameters.

• Rate Difference (RD) For the rate difference, $\delta =p_1-p_2$, the MLE is $\widehat{\delta }={\widehat{p}}_1-{\widehat{p}}_2$ and the

Wald confidence interval is found by solving

$$(\widehat{\delta }-\delta )=\pm z_{\alpha /2}\widehat{\sigma }\left(\widehat{\delta }\right),$$

where the estimated variance is ${\widehat{\sigma }}^2\left(\widehat{\delta }\right)=[{\widehat{p}}_1{\widehat{q}}_1/n_1+{\widehat{p}}_2{\widehat{q}}_2/n_2]$, and $z_{\alpha }$ is the upper $\alpha$

quantile of the standard normal distribution.

• Log Rate Ratio (RR) For the log rate ratio, $\xi =log(p_1/p_2)$, the MLE is $\widehat{\xi }=log({\widehat{p}}_1/{\widehat{p}}_2)$

and the Wald confidence interval is found by solving

$$(\widehat{\xi }-\xi )=\pm z_{\alpha /2}\widehat{\sigma }\left(\widehat{\xi }\right),$$

where the estimated variance is ${\widehat{\sigma }}^2\left(\widehat{\xi }\right)=[{\widehat{q}}_1/\left(n_1{\widehat{p}}_1\right)+{\widehat{q}}_2/\left(n_2{\widehat{p}}_2\right)]$.

• Log Odds Ratio (OR) For the log odds ratio, $\theta =log(\left(p_1{q}_2\right)/\left(q_1{p}_2\right))$, the MLE is

$\widehat{\theta }=log(\left({\widehat{p}}_1{\widehat{q}}_2\right)/\left({\widehat{q}}_1{\widehat{p}}_2\right))$ and the Wald confidence interval is found by solving

$$(\widehat{\theta }-\theta )=\pm z_{\alpha /2}\widehat{\sigma }\left(\widehat{\theta }\right),$$

where the estimated variance is ${\widehat{\sigma }}^2\left(\widehat{\theta }\right)=[1/\left(n_1{\widehat{p}}_1{\widehat{q}}_1\right)+1/\left(n_2{\widehat{p}}_2{\widehat{q}}_2\right)]$.

Wald confidence intervals are known to perform well for large samples but are often unreliable for small or moderate sample sizes [6]. Issues also arise with boundary conditions when the MLEs ${\widehat{p}}_1$ or ${\widehat{p}}_2$ are close to 0 or 1. Common approaches to address these issues include adding a small constant, such as 0.5 or 1, to each cell of the contingency table, as suggested by Agresti and Caffo [6]. For the RD, a continuity correction is sometimes applied by subtracting and adding $(1/n_1+1/n_2)/2$ to the interval endpoints (See, for example, [3], p. 60), while Hauck and Anderson [7] used $1/(2min(n_1,n_2))$ as the continuity correction factor.

To improve small-sample performance, Miettinen and Nurminen [8] proposed a score-based method that inverts a chi-squared test for the null hypothesis (e.g., $H_0:\delta ={\delta }_0$). For the RD, the confidence interval is found by solving

$$(\widehat{\delta }-{\delta }_0)=\pm z_{\alpha /2}{\widehat{\sigma }}_0\left(\widehat{\delta }\right)$$

for ${\delta }_0$, where the variance estimate ${\widehat{\sigma }}_0^2\left(\widehat{\delta }\right)=[{\tilde{p}}_1{\tilde{q}}_1/n_1+{\tilde{p}}_2{\tilde{q}}_2/n_2]N/(N-1)$ is based on the restricted MLEs that satisfy the null hypothesis ${\tilde{p}}_1-{\tilde{p}}_2={\delta }_0$. Similarly, for the RR, their approach involves solving

$$({\widehat{p}}_1-{\widehat{p}}_2{e}^{{\xi }_0})=\pm z_{\alpha /2}{\widehat{\sigma }}_0({\widehat{p}}_1-{\widehat{p}}_2{e}^{{\xi }_0})$$

for ${\xi }_0$ (or for $e^{{\xi }_0}$), where the variance estimate ${\widehat{\sigma }}_0^2({\widehat{p}}_1-{\widehat{p}}_2{e}^{{\xi }_0})=N/(N-1)[{\tilde{p}}_1{\tilde{q}}_1/n_1+e^{2{\xi }_0}{\tilde{p}}_2{\tilde{q}}_2/n_2]$ is based on the restricted MLEs such that ${\tilde{p}}_1={\tilde{p}}_2{e}^{{\xi }_0}$. Because the variance terms depend on the unknown parameter values, these solutions typically require an iterative algorithm. Miettinen and Nurminen [8] did not develop a similar interval for the OR due to the complexity of the variance term, instead favoring a likelihood-based approach. Other related methods include Mee [9]’s confidence interval, which omits the degrees of freedom (DF) correction factor $N/(N-1)$ from the variance term, and Beal [10]’s RD interval, which is similar to Miettinen and Nurminen [8]’s approach.

For the RD, Newcombe [11] proposed a hybrid score confidence interval based on the one-sample Wilson score confidence limits, which was later discussed by Barker et al. [12]. An exact unconditional confidence interval for the RD was proposed by Santner and Snell [13] and further discussed by Agresti [14], Chan and Zhang [15], Agresti and Min [16], and Santner et al. [17]. Other confidence interval methods for the RR include the likelihood ratio confidence interval [8,18] and the exact unconditional confidence interval [13,14,16,17,19], while other confidence interval methods for the OR include the likelihood ratio confidence interval [20], the median unbiased (Midp) conditional confidence interval [21], and the well-known Fisher’s exact (conditional) confidence interval. All these confidence interval methods have been implemented in SAS version 9.4 and later through its Proc Freq procedure (SAS Institute Inc. 2023, Cary, NC, USA).

The methods of Miettinen and Nurminen [8], Mee [9], and Beal [10] for the RD share the same generalized estimator format, $g=\widehat{\delta }-\delta$, and thus have the same $\Lambda$-efficiency as the Wald estimator. The difference among these methods lies mainly in how the nuisance parameter in the variance term is estimated. However, the small-sample correction, such as adding 0.5 or 1 to each cell count, introduces a new estimator with a lower $\Lambda$-efficiency but potentially higher small-sample coverage.

A key contribution of the generalized estimation framework by Vos and Wu [4] is the $\Lambda$-efficiency as a local property that quantifies an estimator’s ability to distinguish parameter values around the truth. This framework also reveals that for the RR, the generalized estimator $g={\widehat{p}}_1-{\widehat{p}}_2{e}^{\xi }$ is the score estimator. Thus, this estimator dominates other ones, such as the small-sample corrected Wald estimator, $g=\widehat{\xi }-E\left[\widehat{\xi }\right]$, in terms of $\Lambda$-efficiency. The same holds for the OR, where the score estimator developed in this paper dominates other estimators, including the small-sample corrected Wald estimator, $g=\widehat{\theta }-E\left[\widehat{\theta }\right]$.

2. Profile Score Confidence Intervals

This section details the derivation of the profile score confidence intervals for the RD, RR, and OR within the context of a one-parameter exponential family by inverting the orthogonalized score estimators. Utilizing a one-parameter exponential family enables us to provide a unified derivation for all three parameters of interest. Our main focus is on the analysis of 2 × 2 Comparative Trials.

2.1. Analysis of 2 × 2 Comparative Trials

Consider a one-parameter exponential family with probability density or mass function $f\left(x\right|\eta )=h\left(x\right)exp\left[\eta T\left(x\right)-A\left(\eta \right)\right]$, where $\eta$ is the natural parameter. For this family, the mean of the sufficient statistic $T\left(X\right)$ is given by $\mu =E\left[T\left(X\right)\right]=A^{\prime }\left(\eta \right)$ and its variance is $I=V\left(T\left(X\right)\right)=A^{\prime \prime }\left(\eta \right)$ (see, e.g., [22] (pp. 109–137) for more details). The Bernoulli(p) distribution is a member of this family with $\eta =log(p/(1-p\left)\right)$, $\mu =p$, and $I=p(1-p)$.

Assume we have independent samples $\{x_{11},\dots ,x_{1n_1}\}$ and $\{x_{21},\dots ,x_{2n_2}\}$, respectively, from two distributions within this exponential family, denoted $f_{n_i}(x;{\eta }_i)$ for $i=1,2$. We define the sufficient statistics as the sample means ${\overline{T}}_i={\sum }_{j}T\left(x_{ij}\right)/n_i$. The log-likelihood function for the sufficient statistics, parameterized by a parameter of interest $\theta$ and a nuisance parameter $\tau$ is given by:

$$l(\theta ,\tau |{\overline{T}}_1,{\overline{T}}_2)\propto {\eta }_1{n}_1{\overline{T}}_1-n_{1}A\left({\eta }_1\right)+{\eta }_2{n}_2{\overline{T}}_2-n_{2}A\left({\eta }_2\right),$$

where ${(\theta ,\tau )}^t$ can be any reparameterization of ${({\eta }_1,{\eta }_2)}^t$.

The scores for $\theta$ and $\tau$ are found by differentiating the log-likelihood:

$$\begin{array}{ccc}\hfill s& =& {\displaystyle \frac{\partial l}{\partial \theta }}=n_1({\overline{T}}_1-{\mu }_1){\eta }_{1/\theta }+n_2({\overline{T}}_2-{\mu }_2){\eta }_{2/\theta },\hfill \end{array}$$

(3)

$$\begin{array}{ccc}\hfill \tilde{s}& =& {\displaystyle \frac{\partial l}{\partial \tau }}=n_1({\overline{T}}_1-{\mu }_1){\eta }_{1/\tau }+n_2({\overline{T}}_2-{\mu }_2){\eta }_{2/\tau },\hfill \end{array}$$

(4)

where ${\eta }_{i/\theta }=\partial {\eta }_i/\partial \theta$ and ${\eta }_{i/\tau }=\partial {\eta }_i/\partial \tau$ are used for notational convenience. The covariance between the two score functions is

$$\begin{array}{ccc}\hfill E\left[s\tilde{s}\right]& =& n_1^{2}E{({\overline{T}}_1-{\mu }_1)}^2{\eta }_{1/\theta }{\eta }_{1/\tau }+n_2^{2}E{({\overline{T}}_2-{\mu }_2)}^2{\eta }_{2/\theta }{\eta }_{2/\tau }\hfill \\ & =& n_1{I}_1{\eta }_{1/\theta }{\eta }_{1/\tau }+n_2{I}_2{\eta }_{2/\theta }{\eta }_{2/\tau }\hfill \end{array}$$

(5)

and the variance of the nuisance score function is

$$V\left(\tilde{s}\right)=n_1{I}_1{\eta }_{1/\tau }^2+n_2{I}_2{\eta }_{2/\tau }^2.$$

According to Vos and Wu [4], the orthogonalized score estimator for $\theta$ is

$$s^{\perp }=s-E\left[s\tilde{s}\right]V{\left(\tilde{s}\right)}^{-1}\tilde{s}\propto {\displaystyle \frac{{\overline{T}}_1-{\mu }_1}{{\eta }_{1/\tau }{I}_1}}-{\displaystyle \frac{{\overline{T}}_2-{\mu }_2}{{\eta }_{2/\tau }{I}_2}}$$

with its standardized version

$${\overline{s}}^{\perp }={\displaystyle \frac{({\overline{T}}_1-{\mu }_1)/\left({\eta }_{1/\tau }{I}_1\right)-({\overline{T}}_2-{\mu }_2)/\left({\eta }_{2/\tau }{I}_2\right)}{\sqrt{1/\left(n_1{\eta }_{1/\tau }^2{I}_1\right)+1/\left(n_2{\eta }_{2/\tau }^2{I}_2\right)}}}.$$

(6)

The generalized score estimator ${\overline{s}}^{\perp }$ is orthogonal to the nuisance tangent space and achieves the information upper bound. A profile score confidence interval for $\theta$ can then be obtained by solving ${\overline{s}}^{\perp }=\pm z_{\alpha /2}$, using the profile likelihood estimate for the nuisance parameter $\widehat{\tau }$ (by solving the nuisance parameter score equation $\tilde{s}=0$). This method is called the z-standard interval.

Theorem 1 (Parameter Invariance).

If the parameter of interest θ and the nuisance parameter τ are two different smooth functions of the parameters ${\eta }_1$ and ${\eta }_2$ for the two populations, the standardized score estimator (6) for θ when orthogonalized to the nuisance tangent space is parameterization invariant to both θ and τ.

Proof. 

The inverse Jacobian rule shows that

$${\displaystyle \frac{\partial ({\eta }_1,{\eta }_2)}{\partial (\theta ,\tau )}}={\left({\displaystyle \frac{\partial \theta }{\partial {\eta }_1}}{\displaystyle \frac{\partial \tau }{\partial {\eta }_2}}-{\displaystyle \frac{\partial \theta }{\partial {\eta }_2}}{\displaystyle \frac{\partial \tau }{\partial {\eta }_1}}\right)}^{-1}\left(\begin{array}{cc}\partial \tau /\partial {\eta }_2& -\partial \tau /\partial {\eta }_1\\ -\partial \theta /\partial {\eta }_2& \partial \theta /\partial {\eta }_1\end{array}\right)$$

(7)

and the chain rule shows that

$${\displaystyle \frac{\partial \theta }{\partial \tau }}={\displaystyle \frac{\partial \theta }{\partial {\eta }_1}}{\eta }_{1/\tau }+{\displaystyle \frac{\partial \theta }{\partial {\eta }_2}}{\eta }_{2/\tau }=0.$$

If another nuisance parameter $\nu =\nu (\theta ,\tau )$ was used, we will write

$${\eta }_{i/\nu }={\eta }_{i/\theta }{\displaystyle \frac{\partial \theta }{\partial \nu }}+{\eta }_{i/\tau }{\displaystyle \frac{\partial \tau }{\partial \nu }}={\eta }_{i/\tau }{\displaystyle \frac{\partial \tau }{\partial \nu }},$$

and this holds true for any parameter of interest $\theta$. When ${\eta }_{1/\nu }$ and ${\eta }_{2/\nu }$ are substituted into (6), the $\partial \tau /\partial \nu$ part in the numerator and denominator cancels out and the estimator remains unchanged. Thus, (6) is parameterization invariant to the nuisance parameter $\tau$.

To show (6) is parameterization invariant to the parameter of interest $\theta$, we just need to rewrite it as

$${\overline{s}}^{\perp }={\displaystyle \frac{({\overline{T}}_1-{\mu }_1)(\partial \theta /\partial {\eta }_1)/I_1+({\overline{T}}_2-{\mu }_2)(\partial \theta /\partial {\eta }_2)/I_2}{\sqrt{{(\partial \theta /\partial {\eta }_1)}^2/\left(n_1{I}_1\right)+{(\partial \theta /\partial {\eta }_2)}^2/\left(n_2{I}_2\right)}}}$$

(8)

following (7). If another parameter of interest $\nu =\nu (\theta ,\tau )$ was used, we will write

$${\displaystyle \frac{\partial v}{\partial {\eta }_i}}={\displaystyle \frac{\partial \nu }{\partial \theta }}{\displaystyle \frac{\partial \theta }{\partial {\eta }_i}}+{\displaystyle \frac{\partial \nu }{\partial \tau }}{\displaystyle \frac{\partial \tau }{\partial {\eta }_i}}={\displaystyle \frac{\partial \nu }{\partial \theta }}{\displaystyle \frac{\partial \theta }{\partial {\eta }_i}},$$

and this holds true for any nuisance parameter $\tau$. When $\partial \nu /\partial {\eta }_1$ and $\partial \nu /\partial {\eta }_2$ are substituted into (8), the $\partial \nu /\partial \theta$ part in the numerator and denominator cancels out and the estimator remains unchanged. □

As shown in Cox and Reid [23], for each parameter of interest, there exists an orthogonal nuisance parameterization. For the mean difference $\delta ={\mu }_1-{\mu }_2$, ${\tau }_{\delta }=n_1{\eta }_1+n_2{\eta }_2$ is an orthogonal nuisance parameterization because it leads to orthogonal scores with $E\left[s\tilde{s}\right]=0$. To see this, we just need to use the inverse Jacobian rule (7) and substitute these partial derivatives into (5). Similarly, for the log mean ratio, $\xi =log({\mu }_1/{\mu }_2)$, ${\tau }_{\xi }=n_{1}A\left({\eta }_1\right)+n_{2}A\left({\eta }_2\right)$ is an orthogonal nuisance parameterization, while for the natural parameter difference $\theta ={\eta }_1-{\eta }_2$, ${\tau }_{\theta }=n_1{\mu }_1+n_2{\mu }_2$ is an orthogonal nuisance parameterization. Due to the parameterization invariant properties of (6), these nuisance parameters ${\tau }_{\delta }$, ${\tau }_{\xi }$, and ${\tau }_{\theta }$ can be used equivalently for all three parameters of interest $\delta$, $\xi$, and $\theta$. For each parameter of interest, the standardized estimator (6) simplifies to a unique form, regardless of the nuisance parameterization.

• Mean Difference For the mean difference, $\delta ={\mu }_1-{\mu }_2$, the standardized score estimator

simplifies to

$${\overline{s}}^{\perp }={\displaystyle \frac{{\overline{T}}_1-{\overline{T}}_2-\delta }{\sqrt{I_1/n_1+I_2/n_2}}}.$$

(9)

The information upper bound achieved by this estimator is $\Lambda \left({\overline{s}}^{\perp }\right)=1/(I_1/n_1+$

$I_2/n_2)$. The nuisance parameter score equation simplifies to $n_1({\overline{T}}_1-{\mu }_1)/I_1+n_2({\overline{T}}_2-$

${\mu }_2)/I_2=0$.

• Log Mean Ratio For the log mean ratio, $\xi =log({\mu }_1/{\mu }_2)$, the standardized score estimator

simplifies to

$${\overline{s}}^{\perp }={\displaystyle \frac{{\overline{T}}_1-e^{\xi }{\overline{T}}_2}{\sqrt{I_1/n_1+e^{2\xi }{I}_2/n_2}}}.$$

(10)

The information upper bound achieved by this estimator is $\Lambda \left({\overline{s}}^{\perp }\right)={\mu }_1^2/(I_1/n_1+$

$e^{2\xi }{I}_2/n_2)$. The nuisance parameter score equation simplifies to $n_1({\overline{T}}_1-{\mu }_1){\mu }_1/I_1+$

$n_2({\overline{T}}_2-{\mu }_2){\mu }_2/I_2=0$.

• Natural Parameter Difference For the natural parameter difference, $\theta ={\eta }_1-{\eta }_2$, the

standardized score estimator simplifies to

$${\overline{s}}^{\perp }={\displaystyle \frac{({\overline{T}}_1-{\mu }_1)/I_1-({\overline{T}}_2-{\mu }_2)/I_2}{\sqrt{1/\left(n_1{I}_1\right)+1/\left(n_2{I}_2\right)}}}.$$

(11)

The information upper bound achieved by this estimator is $\Lambda \left({\overline{s}}^{\perp }\right)=1/(1/\left(n_1{I}_1\right)+$

$1/\left(n_2{I}_2\right))$. The nuisance parameter score equation simplifies to $n_1({\overline{T}}_1-{\mu }_1)+n_2({\overline{T}}_2-$

${\mu }_2)=0$. This equation leads to a simplified profile nuisance parameter estimate

${\widehat{\tau }}_{\theta }=n_1{\overline{T}}_1+n_2{\overline{T}}_2$, which is independent of the parameter of interest and greatly

simplifies the computation.

The standardized score estimator ${\overline{s}}^{\perp }$ is parameter invariant and has $\Lambda$-information that achieves the Fisher information bound. As $\Lambda$-information is a local property, we consider finite samples in Section 3.2 to assess the extent to which the theoretical optimality of $\Lambda$-efficiency translates to superior confidence intervals.

2.2. Analysis of Independence

Although this paper focuses on comparing two independent proportions arising from two-by-two comparative trials, it is worthwhile to consider other related designs, such as the design that samples from a single combined population and inferences for independence between two categorical variables. In this latter design, the analysis is conditioned only on the total sample size, N, but not on the group sample sizes $n_1$ and $n_2$.

In a combined population, the proportions of individuals in the exposure and control groups, respectively, $\pi$ and $1-\pi$, are themselves unknown, introducing an additional nuisance parameter, $\pi \in (0,1)$. Assuming that the probability of an individual being sampled is independent of their outcome, the log-likelihood function for this model, parameterized by the parameter of interest (e.g., $\theta ={\eta }_1-{\eta }_2$) and the two nuisance parameters, $\tau$ and $\pi$, can be expressed as:

$$\begin{array}{c}l(\theta ,\tau ,\pi |{\overline{T}}_1,{\overline{T}}_2,n_1,n_2)\hfill \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} \propto {\eta }_1{n}_1{\overline{T}}_1-n_{1}A\left({\eta }_1\right)+{\eta }_2{n}_2{\overline{T}}_2-n_{2}A\left({\eta }_2\right)+n_{1}log\left(\pi \right)+n_{2}log(1-\pi ).\hfill \end{array}$$

A key assumption of this model is that the parameter of interest, $\theta$, and the nuisance parameter, $\tau$, are fundamentally unrelated to the nuisance parameter, $\pi$. Consequently, the score function s for $\theta$ and the score function ${\tilde{s}}_{\tau }=\tilde{s}$ for $\tau$ remain unchanged from (3) and (4), respectively. The score for the nuisance parameter $\pi$ is given by the partial derivative

$${\tilde{s}}_{\pi }={\displaystyle \frac{\partial l}{\partial \pi }}={\displaystyle \frac{n_1}{\pi }}-{\displaystyle \frac{n_2}{1-\pi }},$$

so the covariances among the score functions satisfy $E\left[s{\tilde{s}}_{\tau }\right]=E\left[s{\tilde{s}}_{\pi }\right]=E\left[{\tilde{s}}_{\tau }{\tilde{s}}_{\pi }\right]=0$.

After orthogonalization to the nuisance scores and standardization, the standardized score estimator for $\theta$ retains the same structural form as (6). The only adjustment required is to replace the variance term in the denominator with its expected value. However, the estimators (9)–(11) are still their profile versions (after substituting the expected variances for their MLEs) to find the z-standard intervals in the same manner as in the two population case.

2.3. Analysis of Randomized Controlled Trials

In randomized controlled trials, however, a conditional analysis is performed by fixing not only the group sample sizes, $n_1$ and $n_2$, but also the total number of “successes”, $k=x_1+x_2$, and “failures”, $N-k$. This approach yields a conditional probability mass function for the number of successes in the first group, $x_1$, given the fixed marginal totals.

The conditional probability mass function for $x_1$ is given by

$$p\left(x_1\right|n_1,n_2,k,p_1,p_2)=\left({\displaystyle \genfrac{}{}{0pt}{}{n_1}{x_1}}\right)\left({\displaystyle \genfrac{}{}{0pt}{}{n_2}{k-x_1}}\right)e^{\theta x_1}/f_k\left(\theta \right),$$

where $\theta$ represents the log odds ratio and $f_k\left(\theta \right)$ is the normalizing constant defined as:

$$f_k\left(\theta \right)=\sum _{i_1=0}^k\left({\displaystyle \genfrac{}{}{0pt}{}{n_1}{i_1}}\right)\left({\displaystyle \genfrac{}{}{0pt}{}{n_2}{k-i_1}}\right)e^{\theta i_1}.$$

A significant advantage of this approach is that the conditional log-likelihood function $l\left(\theta \right|x_1)\propto x_1\theta -log\left(f_k\left(\theta \right)\right)$ depends only on the parameter of interest, $\theta$, and contains no nuisance parameters. This simplifies inference substantially.

The score function for $\theta$ is derived as the first derivative of the log-likelihood: $s=x_1-f_k^{\prime }\left(\theta \right)/f_k\left(\theta \right)$ and the variance of the score function is $V\left(s\right)=f_k^{\prime \prime }\left(\theta \right)/f_k\left(\theta \right)-{\left(f_k^{\prime }\left(\theta \right)/f_k\left(\theta \right)\right)}^2.$ The z-standard interval for $\theta$ can be found by solving $\overline{s}=\pm z_{\alpha /2}$, where $\overline{s}=V{\left(s\right)}^{-1/2}s$.

This conditional likelihood approach is particularly well-suited for the odds ratio. For this design, an alternative and often preferred method for constructing confidence intervals is Fisher’s exact (or Midp) confidence interval for the odds ratio. Using other parameterizations, such as the RD or RR, would reintroduce nuisance parameters, leading to more complex calculations than the odds ratio.

3. Numerical Studies

This section first discusses algorithms in computing the z-standard intervals for the RD, RR, and OR. R’s polyroot function is utilized for polynomial root finding. Special attentions are given to the profile likelihood estimates of the nuisance parameters, and a couple of correction factors are proposed. Second, this section uses numerical studies to establish the superiority of the z-standard intervals, especially with the DF and small-sample corrections, in finite-sample coverage, in the sense of maintaining coverage close to the nominal level.

3.1. Computational Details

For comparative analysis of two independent proportions or analysis of independence in a two-by-two table, the z-standard interval for the log odds ratio $\theta$ (or equivalently the odds ratio $e^{\theta }$) can be constructed by solving

$${\displaystyle \frac{x_1-n_1{p}_1}{n_1{p}_1{q}_1}}-{\displaystyle \frac{x_2-n_2{p}_2}{n_2{p}_2{q}_2}}=\pm z_{\alpha /2}\sqrt{{\displaystyle \frac{1}{n_1{p}_1{q}_1}}+{\displaystyle \frac{1}{n_2{p}_2{q}_2}}},$$

(12)

where $\tau ={\tau }_{\theta }=n_1{p}_1+n_2{p}_2$ is the chosen nuisance parameter. The profile likelihood estimate of the nuisance parameter is simply $\widehat{\tau }=x_1+x_2$ regardless of $\theta$. When the observed data correspond to the extreme cases $(x_1,x_2)=(0,0)$ or $(n_1,n_2)$, nothing can be learned about $\theta$ and thus we always report $(-\infty ,+\infty )$ as the z-standard interval for $\theta$ or (0, $+\infty$) as the z-standard interval for the OR. In comparison, the SAS Proc Freq procedure does not report any confidence intervals for these extreme cases.

A similar DF correction as that of Miettinen and Nurminen [8] to the z-standard intervals can be achieved by multiplying the critical value $z_{\alpha /2}$ with the factor $\sqrt{N/(N-1)}$. The DF correction is expected to widen the z-standard intervals and improve their coverage. However, the small-sample correction like those in Agresti and Caffo [6] is only recommended for the profile likelihood estimate of the nuisance parameter via the mean sufficient statistics:

$$\tilde{\tau }\left(c\right)={\displaystyle \frac{n_1(x_1+c)}{n_1+2c}}+{\displaystyle \frac{n_2(x_2+c)}{n_2+2c}}$$

(13)

for some small constant $c>0$. The small-sample correction is not only expected to widen the z-standard intervals but also help with the extreme cases. Although (12) is the same as that of Miettinen and Nurminen [8] for the OR, our treatment of the nuisance parameter and its small-sample correction is unique.

When both $p_i$ and $q_i$ for $i=1,2$ are nonzero, (12) simplifies to a 6-degree polynomial equation in $p_1$, where $p_2=(\tilde{\tau }-n_1{p}_1)/n_2$. This equation has six real or complex roots, from which an appropriate interval can be determined by testing $0<p_1,p_2<1$. When R’s polyroot function is used, the middle two real roots correspond to the z-standard interval except for the two extreme cases. The z-standard interval for $\theta$ or $e^{\theta }$ is then obtained from those of $p_1$ and $p_2$. The advantage of using R’s polyroot function is that it has no convergence issue and executes very fast compared to other root-finding procedures.

Similarly, the z-standard interval for the rate difference $\delta$ can be found from solving

$${\displaystyle \frac{x_1}{n_1}}-{\displaystyle \frac{x_2}{n_2}}-\delta =\pm z_{\alpha /2}\sqrt{{\displaystyle \frac{p_1{q}_1}{n_1}}+{\displaystyle \frac{p_2{q}_2}{n_2}}},$$

(14)

where the profile likelihood estimate of the nuisance parameter can be obtained from solving

$${\displaystyle \frac{x_1-n_1{p}_1}{p_1{q}_1}}+{\displaystyle \frac{x_2-n_2{p}_2}{p_2{q}_2}}=0.$$

(15)

For the convenience of comparison, we adopt the same nuisance parameter $\tau =n_1{\mu }_1+n_2{\mu }_2$ but it works equivalently if ${\tau }_{\delta }$ was chosen instead. But no matter which nuisance parameter is used, its profile likelihood estimate depends on the parameter of interest $\delta$. Starting from an initial nuisance parameter value, e.g., $\widehat{\tau }=x_1+x_2$, we can solve (14) for an initial interval for $\delta$. After that, an iterative algorithm between (14) and (15) can be carried out until convergence to find the z-standard interval for $\delta$. In our numeric studies, the iteration often converges within 30 steps when a threshold of ${10}^{-9}$ is used for changes in successive parameter values. The numerical procedure experiences no issue with extreme cases because the RD is not a ratio. In comparison, the classic Wald confidence interval will report (0, 0) for the two extreme cases, which we do not think provides any practical guidance to the true RD.

Lastly, the z-standard interval for the log rate ratio $\xi$ (or equivalently the rate ratio $e^{\xi }$) can be found from solving

$${\displaystyle \frac{x_1}{n_1}}-e^{\xi }{\displaystyle \frac{x_2}{n_2}}=\pm z_{\alpha /2}\sqrt{{\displaystyle \frac{p_1{q}_1}{n_1}}+e^{2\xi }{\displaystyle \frac{p_2{q}_2}{n_2}}},$$

(16)

where the profile likelihood estimate of the nuisance parameter can be obtained from solving

$${\displaystyle \frac{x_1-n_1{p}_1}{q_1}}+{\displaystyle \frac{x_2-n_2{p}_2}{q_2}}=0.$$

(17)

We again adopt the same nuisance parameter $\tau =n_1{\mu }_1+n_2{\mu }_2$ and use a similar iterative algorithm as previously described to find the z-standard interval for $\xi$ or the RR. This procedure may have convergence issues when $x_2=0$, in which case we assign $+\infty$ to the upper limit, or when $x_1=x_2=0$, in which case we report $(0,+\infty )$ as the z-standard interval for the RR.

The z-standard intervals for the RD and RR should be the same as those of Mee [9], and similar DF and small-sample corrections as those for the OR can be applied (to the profile nuisance parameter estimates via the sufficient statistics).

Figure 1 shows three examples for comparing two independent binomial proportions with $n_1=20$ and $n_2=30$, respectively. The numbers of successes are $x_1=1$ and $x_2=2$ in the first example, $x_1=5$ and $x_2=8$ in the second example, and $x_1=11$ and $x_2=20$ in the third example. For the log odds ratio, the profile likelihood estimates of the nuisance parameter are $\widehat{\tau }=3$, 13, and 31, respectively, in the three examples and the z-standard intervals are represented by the horizontal bold-faced segments. For the rate difference and log rate ratio, however, the profile likelihood estimates of the nuisance parameter are represented by the bold-faced curves, and the z-standard intervals are the projections of the curves to the horizontal axis. Figure 1 also includes some horizontal grayed segments representing what the z-standard intervals could be if different nuisance parameter values had been specified a priori, where the gray shaded area represents nuisance parameter values that are within one standard deviation of $\widehat{\tau }$.

Figure 1. Three examples of z-standard intervals for the log(OR), RD, and log(RR) comparing two independent binomial populations. The sample sizes are $(n_1,n_2)=(20,30)$ for all examples, and the numbers of successes are $(x_1,x_2)=(1,2)$ for the first example, $(x_1,x_2)=(5,8)$ for the second example, and $(x_1,x_2)=(11,20)$ for the third example. For the log(OR), the bold-faced segments represent the z-standard intervals at the profiled nuisance parameter estimates $\widehat{\tau }=x_1+x_2$. For the RD and log(RR), the bold-faced curves depict the corresponding profile nuisance parameter estimates at each parameter of interest value. The projection of each curve to the horizontal axis gives the corresponding z-standard interval. Other grayed horizontal segments in the plots represent the potential z-standard intervals at each pre-specified nuisance parameter value, where the gray shaded area represents nuisance parameter values that are within one standard deviation of $\widehat{\tau }$. The red dashed segments/curves represent z-standard intervals with the small-sample correction ($c=0.5$).

In the generalized estimation framework of Vos and Wu [4], the parameter spaces for both $p_1$ and $p_2$ are $(0,1)$ so the parameter space for the nuisance parameter $\tau$ is $(0,n_1+n_2)$. When the data correspond to the extreme cases $(x_1,x_2)=(0,0)$ or $(n_1,n_2)$, the estimators considered in this paper do not exist without the small-sample correction. In addition, as shown in Figure 1, different nuisance parameter values could greatly impact the z-standard intervals. As a remediation, we could report the widest interval for nuisance parameter values within one standard deviation of $\widehat{\tau }$. This remediation is often conservative, but it could sometimes report intervals that are too narrow, e.g., in the RD case when $x_1=1$ and $x_2=2$. Our numerical studies in Section 3.2 suggest that the small-sample correction ($c=0.5$) to the profile likelihood estimate of the nuisance parameter provides a reasonable remediation, as represented by the red dashed segments/curves in Figure 1.

3.2. Coverage and Confidence Interval Width

Consider two independent binomial distributions with $n_1=20$ and $n_2=30$ at 600 combinations of true log odds ratio $\theta$ and nuisance parameter $\tau$ values. These combinations cover $\theta$ values ranging from −3 to 3 and $\tau$ values at 5, 10, 20, 30, 40, and 45. Figure 2 compares the coverage of the z-standard intervals of the log(OR) with (Z-adj) and without (Z-std) the DF and small-sample corrections ($c=0.5$) to those of the Wald confidence intervals with (Wald-adj) and without (Wald) the small-sample correction ($c=0.5$).

Figure 2. Coverage of the z-standard intervals of the log(OR) with (Z-adj) and without (Z-std) the DF and small-sample corrections ($c=0.5$) compared to those of the Wald confidence intervals with (Wald-adj) and without (Wald) the small-sample correction ($c=0.5$) under a nominal level of 0.95. Two independent binomial distributions are used with $n_1=20$ and $n_2=30$ at 600 combinations of true log odds ratio $\theta$ and nuisance parameter $\tau$ values. These combinations cover $\theta$ values ranging from −3 to 3 and $\tau$ values at 5, 10, 20, 30, 40, and 45. When $x_1=0$ or $n_1$ or when $x_2=0$ or $n_2$, the Wald confidence intervals for the log(OR) are ($-\infty$, $+\infty$).

Figure 2 shows the coverage of each type of confidence intervals at a nominal level of 0.95. The discrete nature of the problem makes a zig-zag pattern in the coverage as the true parameter value crosses the interval limits. For maintaining coverage, the Wald methods tend to report wide confidence intervals, especially when the observed data $(x_1,x_2)$ are near the extremes. The DF and small-sample corrections in (13) are expected to widen the z-standard intervals and improve their coverage. Using more aggressive small-sample corrections, e.g., for some $c\ge 1$, will lead to more aggressive coverage corrections, but we may need to consider the trade-off because coverage varies with the underlying true parameter values as shown in the figure.

Observations from Figure 2 include (1) the coverage of the Wald confidence intervals are always higher than those of the z-standard intervals, (2) the DF and small-sample corrections improve the coverage of the z-standard intervals as expected, and (3) the small-sample correction to the Wald confidence intervals may significantly lower their coverage when $\theta$ is far away from zero. But the most important takeaway is that the z-standard intervals are the best in maintaining the coverage close to the nominal level of 0.95 and the DF and small-sample corrections can help them stay above the nominal level. Note that the small-sample correction to the z-standard intervals is only applied to the profile likelihood estimate of the nuisance parameter $\tau$. For this reason, it acquires the predictable effect of producing wider intervals. In comparison, the small-sample correction to the Wald intervals is applied to the confidence intervals themselves and has varying effects.

To better understand these confidence intervals, their endpoints are compared in the left plot of Figure 3 with the endpoints of the z-standard intervals serving as the references. Because of the difficulties with the Wald intervals, Figure 3 only includes the confidence intervals for $1\le x_1\le n_1-1$ and $1\le x_2\le n_2-1$. The plot shows that (1) the DF and small-sample corrections widen the z-standard intervals as expected, (2) the Wald confidence intervals are always wider than and contain the z-standard intervals, and (3) the small-sample corrected Wald intervals are shifted relative to the z-standard intervals. In fact, for small $x_1$ values, they tend to shift to the right, while for large $x_1$ values, they tend to shift to the left. These findings echo those from the coverage.

Figure 3. A comparison of confidence interval endpoints generated in the coverage studies with $n_1=20$ and $n_2=30$ as presented in Figure 2, Figure 3, Figure 4 and Figure 5 using those of the z-standard intervals (Z-std) as references. The color scale corresponds to the $x_1$ values so that the red color represents small $x_1$ values, while the blue color represents large $x_1$ values. When both endpoints of a particular confidence interval are higher than those of the corresponding z-standard interval (i.e., showing positive differences), the interval is shifted to the right relative to the z-standard interval. When both endpoints show negative differences, the interval is shifted to the left relative to the z-standard interval. When the right endpoint shows a positive difference and the left endpoint shows a negative difference, the interval contains the z-standard interval.

Next, consider a similar setting consisting of two independent binomial distributions with $n_1=20$ and $n_2=30$ at 600 combinations of true rate difference $\delta$ and nuisance parameter $\tau$ values. These combinations cover $\delta$ values ranging from −0.9833 to 0.9833 and $\tau$ values at 5, 10, 20, 30, 40, and 45. Under this setting, not all $\delta$ values are permissible for a given $\tau$ value. For example, when $\tau =5$, only $\delta \in (-5/n_2,5/n_1)$ are permissible. Figure 4 shows a comparison of the coverage of the four types of confidence intervals for the RD over the permissible ranges of $\delta$. The z-standard intervals are still the best in maintaining the coverage close to the nominal level, and the DF and small-sample corrections can help them stay above the nominal level. The coverage of the Wald intervals can change drastically, while those of the small-sample corrected Wald intervals are relatively more stable.

Figure 4. Coverage of the z-standard intervals of the RD with (Z-adj) and without (Z-std) the DF and small-sample corrections ($c=0.5$) compared to those of the Wald confidence intervals with (Wald-adj) and without (Wald) the small-sample correction ($c=0.5$) under a nominal level of 0.95. Two independent binomial distributions were used with $n_1=20$ and $n_2=30$ at 600 combinations of true rate difference $\delta$ and nuisance parameter $\tau$ values. These combinations cover $\delta$ values ranging from −0.9833 to 0.9833 and $\tau$ values at 5, 10, 20, 30, 40, and 45.

The endpoint comparisons in the middle plot of Figure 3 show that the DF and small-sample corrections still, as expected, widen the z-standard intervals. However, both Wald confidence intervals with and without the small-sample correction are shifted relative to the z-standard intervals. Particularly, for small $x_1$ values, they tend to shift to the left, while for large $x_1$ values, they tend to shift to the right.

Next, Figure 5 shows the coverage of the four types of confidence intervals for the log(RR) at 600 combinations of true log rate ratio $\xi$ and nuisance parameter $\tau$ values and the right plot in Figure 3 shows a comparison of their endpoints. These combinations cover $\xi$ values ranging from −4.5985 to 5.0006 and $\tau$ values at 5, 10, 20, 30, 40, and 45. This setting also faces an issue that not all $\xi$ values are permissible. Figure 5 shows that the coverage of all confidence intervals may change drastically when $\xi$ is far away from 0. In these cases, the small-sample corrected Wald confidence intervals performed even worse than their original counterparts, and even the z-standard intervals with or without the corrections may fail. These results suggest that inference for the RR may not be the best choice in certain situations where extreme parameter values are expected. This may be because of the asymmetrical nature of RR and the effect of small denominators.

Figure 5. Coverage of the z-standard intervals of the log(RR) with (Z-adj) and without (Z-std) the DF and small-sample corrections ($c=0.5$) compared to those of the Wald confidence intervals with (Wald-adj) and without (Wald) the small sample correction ($c=0.5$) under a nominal level of 0.95. Two independent binomial distributions were used with $n_1=20$ and $n_2=30$ at 600 combinations of true log rate ratio $\xi$ and nuisance parameter $\tau$ values. These combinations cover $\xi$ values ranging from −4.5985 to 5.0006 and $\tau$ values at 5, 10, 20, 30, 40, and 45. When $x_1=0$ or $x_2=0$, Wald confidence intervals for the log(RR) are ($-\infty$, $+\infty$).

The right plot in Figure 3 shows that the Wald confidence intervals tend to be wider than the z-standard intervals for small $x_1$ values and narrower for large $x_1$ values. On the other hand, the small-sample corrected Wald confidence intervals are shifted relative to the z-standard intervals so that they tend to shift to the right for small $x_1$ values and to the left for large $x_1$ values.

Finally, Table 2 contains a more comprehensive comparison of coverage probabilities between the z-standard intervals with (Z-adj) and without (Z-std) the DF and small-sample corrections ($c=0.5$) and many existing ones. The existing methods under comparison include Wald confidence interval with (Wald-adj) and without (Wald) the small-sample correction ($c=0.5$) and all confidence interval methods implemented by the Proc Freq procedure in SAS 9.4. For the RD, these include Agresti and Caffo [6], which is a Wald interval with a small-sample correction of $c=1$, the exact unconditional confidence interval, the Hauck and Anderson [7] confidence interval with a continuity correction factor of $1/(2min(n_1,n_2))$, and the Newcombe [11] hybrid score confidence interval. For the RR, these include the likelihood ratio and the exact unconditional confidence interval, while for the OR, these include Fisher’s exact (conditional), likelihood ratio, and Midp (conditional) confidence intervals.

Table 2. A comparison of coverage probabilities among different confidence interval methods for the RD, RR, and OR for three sample size settings where $(n_1,n_2)$ = (20, 30), (30, 30), or (30, 20). Coverage probabilities are calculated at 600 combinations of the true parameter of interest and nuisance parameter values. These combinations cover $\theta$ values ranging from −3 to 3, $\delta$ values ranging from about −0.98 to 0.98, $\xi$ values ranging from about −4.6 to 5.0, and $\tau$ values at 5, 10, 20, 30, 40, and 45. The z-standard intervals (Z-adj and Z-std) and the Wald confidence intervals (Wald-adj and Wald) with and without the corrections are generated using R code, while the other confidence intervals are generated using the SAS Proc Freq procedure applied to the entire sample space. The SAS Proc Freq procedure does not provide any confidence intervals for the two extreme cases where $(x_1,x_2)=(0,0)$ or $(n_1,n_2)$. In these cases, either the widest possible intervals are reported or an R package (R 4.5.1) (if available for a particular method) is implemented, such as the contingencytables package for the Newcombe hybrid score method.

Confidence Interval Method	${\mathit{n}}_1=20,{\mathit{n}}_2=30$ Median (Min, Max)	${\mathit{n}}_1=30,{\mathit{n}}_2=30$ Median (Min, Max)	${\mathit{n}}_1=30,{\mathit{n}}_2=20$ Median (Min, Max)
Rate difference
Z-std	0.950 (0.929, 0.981)	0.949 (0.929, 0.978)	0.950 (0.930, 0.981)
Z-adj	0.961 (0.939, 0.992)	0.955 (0.936, 0.983)	0.960 (0.943, 0.992)
Wald	0.932 (0.331, 0.955)	0.939 (0.452, 0.950)	0.932 (0.453, 0.955)
Wald-adj	0.948 (0.926, 0.999)	0.948 (0.927, 0.997)	0.948 (0.927, 0.997)
Agresti-Caffo	0.957 (0.936, 0.992)	0.954 (0.938, 0.986)	0.957 (0.937, 0.986)
Exact (unconditional)	0.968 (0.953, 0.992)	0.984 (0.952, 1.000)	0.987 (0.952, 1.000)
Hauck-Anderson	0.961 (0.552, 1.000)	0.960 (0.453, 0.981)	0.961 (0.453, 0.983)
Newcombe (hybrid)	0.957 (0.881, 0.986)	0.953 (0.903, 0.982)	0.961 (0.453, 0.983)
Rate ratio
Z-std	0.952 (0.905, 0.983)	0.951 (0.905, 0.976)	0.952 (0.900, 0.978)
Z-adj	0.960 (0.905, 0.986)	0.956 (0.905, 0.980)	0.959 (0.900, 0.986)
Wald	0.958 (0.872, 0.999)	0.959 (0.919, 0.999)	0.958 (0.871, 0.999)
Wald-adj	0.948 (0.741, 0.999)	0.949 (0.740, 0.998)	0.950 (0.739, 0.999)
Exact (unconditional)	0.977 (0.952, 0.999)	1.000 (0.963, 1.000)	1.000 (0.964, 1.000)
Likelihood ratio	0.946 (0.870, 0.994)	0.948 (0.872, 0.993)	0.945 (0.863, 0.994)
Odds ratio
Z-std	0.953 (0.926, 0.972)	0.952 (0.933, 0.971)	0.953 (0.926, 0.972)
Z-adj	0.961 (0.939, 0.983)	0.957 (0.942, 0.983)	0.961 (0.939, 0.983)
Wald	0.971 (0.947, 0.998)	0.968 (0.943, 0.997)	0.971 (0.947, 0.998)
Wald-adj	0.962 (0.875, 0.996)	0.960 (0.905, 0.996)	0.962 (0.875, 0.996)
Fisher’s exact	0.980 (0.969, 0.993)	0.976 (0.967, 0.993)	0.980 (0.969, 0.993)
Likelihood ratio	0.943 (0.900, 0.984)	0.943 (0.904, 0.982)	0.943 (0.900, 0.984)
Midp (conditional)	0.959 (0.942, 0.987)	0.956 (0.942, 0.986)	0.959 (0.942, 0.987)

The results in Table 2 confirm that the z-standard interval is the best in maintaining the coverage close to the nominal level, and the DF and small-sample corrections can help its coverage stay above the nominal level. For the RD, the Wald confidence interval undercovers and the small-sample correction helps improve their coverage, while the method of Agresti and Caffo [6] performs reasonably well. For the RR, all confidence intervals may undercover except the exact unconditional method, which overcovers. For the OR, the Wald and Fisher’s exact confidence intervals may overcover, and the likelihood ratio confidence interval may undercover, while the Midp interval performs reasonably well.

4. Discussion

This paper develops the z-standard confidence intervals for the rate difference, rate ratio, and odds ratio using the generalized estimation framework of Vos and Wu [4]. The orthogonalized score estimator dominates other estimators in $\Lambda$-efficiency for distinguishing parameter values around a neighborhood of their truth. As a result, the z-standard intervals are typically narrower than others and better at maintaining coverage close to the nominal level. Moreover, the DF and small-sample corrections are expected to help them maintain coverage above and closely around the nominal level. Numerical studies in this paper suggest that the less aggressive small-sample correction, i.e., $c=0.5$, applied only to the profile likelihood estimate of the nuisance parameter via the mean sufficient statistics, may suffice. The computation of the z-standard intervals can be conveniently and efficiently achieved using R’s polyroot function. Thus, the z-standard interval developed in this paper represents a superior choice for inference practice in two-way contingency tables.

While this paper covers the three most frequently used study designs, i.e., the 2 × 2 comparative design, the analysis of independence design, and the randomized controlled design. The proposed method via the generalized estimation framework of Vos and Wu [4] may be applicable to other designs, such as the longitudinal design, which generates correlated proportions. Such more complicated designs may be of great future research interest.