Confidence Intervals of Risk Ratios for the Augmented Logistic Regression with Pseudo-Observations

Abstract

The augmented logistic regression proposed by Diaz-Quijano directly provides risk ratios with an augmented dataset with the pseudo-observations. However, the standard errors of regression coefficients cannot be accurately estimated using either the ordinary model variance estimator or the robust variance estimator, as neither method appropriately accounts for the pseudo-observations. In this study, we proposed two resampling strategies based on the bootstrap and jackknife methods to construct improved variance estimators for the augmented logistic regression. Both procedures can reflect the overall uncertainty of the augmented dataset involving the pseudo-observations and require only standard software, making them feasible for a wide range of clinical and epidemiological researchers. We validated these proposed methods through comprehensive simulation studies, which demonstrated that both the bootstrap- and jackknife-based variance estimators provided smaller standard error estimates and correspondingly narrower 95% confidence intervals, whereas the robust variance estimator remained biased. Additionally, we applied the proposed methods to real-world binary data, confirming their practical utility.

1. Introduction

Logistic regression has been widely used as a multivariable regression analysis method in clinical and epidemiological studies. However, the resultant odds ratio estimates cannot be interpreted as effect measures directly, except for the cases that they become approximations of risk ratio estimates under the rare disease assumption [1,2]. Due to the serious limitations, recent guidelines [3] recommend using interpretable effect measures in reporting results of clinical studies. Several effective alternative regression analysis methods have been developed in recent studies, e.g., the modified Poisson and least-squares regressions [4,5] for providing consistent estimators of risk ratios and risk differences based on the quasi-likelihood theory [6]. The logistic regression analysis, augmenting pseudo-observations, was proposed by Diaz-Quijano [7] to provide consistent estimators of risk ratios. This method is simply implementable using standard statistical packages of logistic regression after augmenting pseudo-observations to the original dataset and has also been used in many applications. It is also interpreted as a special case of logistic regression of case–cohort studies that can provide consistent estimators of risk ratios [8]. In this article, we call this method an augmented logistic regression analysis.

For the augmented logistic regression, since artificial pseudo-observations are augmented, the standard errors (SEs) of regression coefficients cannot be estimated by the ordinary model variance estimator. Dias-Quijano [7] proposed an ad-hoc method that multiplies inflation factors to the ordinary SE estimates, which are defined as the ratios between the ordinary SE estimates and those resulting from the binomial regression with log-link function. However, this method has serious problems in that the binomial log-linear model cannot provide maximum likelihood (ML) estimates frequently [9,10], because the predictive values of this model can exceed the range [0, 1], and the consistency of the adjusted SE estimator was not shown. As an alternative effective approach, Dwivedi et al. [11] proposed using the robust variance estimator [12] to explain the influences of artificial pseudo-observations in the estimating function. Schouten et al. [8] also proposed adopting the robust variance estimator for the logistic regression analysis in case–cohort studies [13]. However, they did not provide rigorous mathematical explanations and numerical evidence that the robust variance estimator provides valid SE estimates. For the logistic regression of case–cohort studies, Noma [14] provided simulation-based evidence that the robust variance estimator generally provides biased SE estimates and inaccurate confidence intervals of risk ratios. The augmented logistic regression is founded on the same principle as Schouten et al.’s [8] logistic regression for case–cohort studies; the SE estimation could have similar problems.

In this study, we conducted extensive simulation studies and showed that the robust variance estimator of the augmented logistic regression also has a certain bias. Then, we proposed alternative consistent variance estimators for the augmented logistic regression to address this issue, and provided new methods to construct accurate confidence intervals of risk ratios. These methods applied resampling methods to account for the uncertainties caused by using the artificial pseudo-observations. Through numerical evidence from simulations, the proposed SE estimators could unbiasedly estimate actual SEs, and the confidence intervals also provided accurate interval estimates consistently. In addition, we demonstrated the practical effectiveness of the proposed methods via applications to real-world data from the National Child Development Study (NCDS) [15]. We also provided example R codes to implement the proposed methods in the Supplementary Materials.

2. Materials and Methods

2.1. The Augmented Logistic Regression

Consider a cohort with $n$ participants. Let $\mathit{X}={\left(X_1,\cdots ,X_p\right)}^{\top }$ denotes the explanatory variables and $Y$ the binary outcome. A conventional multivariable regression analysis method to obtain risk ratios within the framework of a generalized linear model is the binomial regression model with a log-link function,

$$\mathrm{l}\mathrm{o}\mathrm{g}\mathrm{Pr}\left(Y=1\right)={\beta }_0+{\beta }_1{X}_1+\cdots +{\beta }_p{X}_p$$

where $\mathit{\beta }={\left({\beta }_0,\cdots ,{\beta }_p\right)}^{\top }$ corresponds to the risk ratios. However, the ML estimate of $\mathit{\beta }$ cannot be defined in many cases, because the values of the regression function do not fall within [0, 1] [9,10].

The augmented logistic regression (Algorithm 1) is a pseudo-likelihood estimation method of $\mathit{\beta }$ in the binomial regression model. It avoids the computational difficulty. The estimating procedure is simply summarized as follows [7]:

Algorithm 1 Augmented logistic regression

For the $n_1$ participants whose outcomes are $Y=1$, generate copies with the same explanatory variables and outcome variable $Y=0$. Then, add the artificial pseudo-observations to the original dataset. Fit the ordinary logistic regression model to the augmented dataset,

$\mathrm{l}\mathrm{o}\mathrm{g}\mathrm{i}\mathrm{t}\mathrm{Pr}\left(Y=1\right)={\beta }_0+{\beta }_1{X}_1+\cdots +{\beta }_p{X}_p$

Then, the ML estimator $\widehat{\mathit{\beta }}$ of $\mathit{\beta }$ from the augmented logistic regression becomes a consistent estimator of the regression coefficients of the binomial regression model above; thus, they correspond to log-transformed risk ratio estimates obtained from a multivariable regression analysis. This logistic regression analysis is also interpreted as a special case of that for case–cohort studies proposed by Schouten et al. [8]; the ordinary cohort study is formulated as a case–cohort study whose sampling probability of subcohort is 100%.

Dias-Quijano [7] provided an ad-hoc method to estimate the SE of the pseudo-likelihood estimator of $\mathit{\beta }$. This method calculated the ratios between the ordinary SE estimates obtained from the logistic regression and those resulting from the binomial regression with the log-link function. However, the consistency of the adjusted SE estimator was not shown, and the latter SE estimates are often not calculable. To address this issue, Dwivedi et al. [11] proposed applying the robust variance estimator [12]. In their approach, the duplicated observations are treated as correlated clusters, and the intra-cluster correlation is accounted for through the robust variance estimator. Schouten et al. [8] also proposed using the robust variance estimator for the logistic regression analysis of case–cohort studies by the same concept. However, the pseudo-observations are artificially generated, and they do not actually exist. The model-based robust variance estimator is defined for the augmented dataset, assuming all of the participants are sampled from the source population [12], and then the statistical uncertainty cannot be properly evaluated. Noma [14] showed that the robust variance estimator has substantial bias for the cases of case–cohort studies by simulation studies. In Section 4, we conducted simulation studies to assess the validity of the robust variance estimator for the augmented logistic regression and showed that the SEs are also not unbiasedly estimated.

2.2. Variance Estimation and Confidence Intervals

2.2.1. Bootstrapping Approach

To provide consistent variance estimators, the duplications should be appropriately accounted for. Noma [14] proposed a bootstrap-based variance estimator that addresses the duplication issue by embedding the duplication mechanism within the bootstrap resampling procedure. We first propose applying this bootstrap-based approach to the augmented logistic regression analysis. For convenience, we denote the $n_1$ participants in the cohort whose outcomes are $Y=1$ as cases and $n_0$ participants whose outcomes are $Y=0$ as non-cases.

Through the bootstrap algorithm (Algorithm 2), the overall uncertainty of the regression coefficients estimator is properly explained, and the SEs are consistently estimated.

Algorithm 2 Bootstrap-based variance estimator

Perform a bootstrap resampling from the $n_1$ case samples. Perform a bootstrap resampling from the $n_0$ non-case samples. Create $n_1$ artificial pseudo-observations for the bootstrap case samples. Combine these three datasets and fit the logistic regression to the augmented bootstrap samples. Repeat the processes 1–4 and calculate bootstrap samples of the model coefficient estimates B times, ${\widehat{\mathit{\beta }}}_{\left(1\right)},\dots ,{\widehat{\mathit{\beta }}}_{\left(B\right)}$. Compute the empirical variances of the bootstrap samples ${\widehat{\mathit{\beta }}}_{\left(1\right)},\dots ,{\widehat{\mathit{\beta }}}_{\left(B\right)}$, and the bootstrap variance estimates are obtained.

To construct bootstrap confidence intervals for practical analyses, the percentile method [16,17,18] is a straightforward method, e.g., for calculating 95% confidence intervals, the confidence limits are calculated by the 2.5th and 97.5th percentiles. An alternative effective approach is the bias-corrected and accelerated (BCa) method [16,17,18]. The BCa method improves the percentile method by correcting for the bias and accounting for the skewness of the bootstrap distribution. It provides more accurate confidence intervals, particularly in the presence of bias or asymmetry. The $100\times \left(1-\alpha \right)\%$ confidence interval of ${\beta }_k (k=1,\dots ,p)$ based on the BCa method is provided as follows:

$$\left[{{\widehat{G}}_k}^{-1}\left\{\Phi \left(z_{0,k}+{\displaystyle \frac{z_{0,k}+z_{\alpha /2}}{1-a\left(z_{0,k}+z_{\alpha /2}\right)}}\right)\right\}, {{\widehat{G}}_k}^{-1}\left\{\Phi \left(z_{0,k}+{\displaystyle \frac{z_{0,k}+z_{1-\alpha /2}}{1-a\left(z_{0,k}+z_{1-\alpha /2}\right)}}\right)\right\}\right]$$

where ${{\widehat{G}}_k}^{-1}(·)$ denotes the cumulative distribution function of the bootstrap distribution of ${\widehat{\beta }}_k(k=1,\dots ,p)$, $\Phi (·)$ denotes the cumulative distribution function of the standard normal distribution, and $z_{\alpha }$ denotes the quantile at probability $\alpha$ of the standard normal distribution. $z_{0,k}$ denotes the bias correcting factor given by $z_{0,k}={\Phi }^{-1}\left\{{\widehat{G}}_k({\widehat{\beta }}_k)\right\}$, and $a$ denotes the acceleration estimated using the jackknife influence function [19].

2.2.2. Jackknife Approach

The jackknife method can also be considered as an alternative effective approach for the variance estimation [20,21]. It can reflect the overall uncertainty of the augmented dataset involving artificial pseudo-observations. The jackknife algorithm (Algorithm 3) for the augmented logistic regression is given as follows.

Algorithm 3 Jackknife-based variance estimator

For each observation $i=1,\cdots n$, remove the $i$th observation from the dataset. Based on the leave-one-out dataset, create the augmented dataset by adding pseudo-observations. Fit the logistic regression to the augmented dataset. Repeat processes 1–3 for all n observations and obtain n jackknife replicants of the coefficient estimates, ${\stackrel{\mathbf{~}}{\mathit{\beta }}}_{\left(1\right)},\dots ,{\stackrel{\mathbf{~}}{\mathit{\beta }}}_{\left(n\right)}$. Compute the jackknife estimate of the covariance matrix of $\widehat{\mathit{\beta }}$,

${\widehat{V}}_{jack}\left[\widehat{\mathit{\beta }}\right]={\displaystyle \frac{n-1}{n}}{\displaystyle \sum _{i=1}^n}\left({\stackrel{\mathbf{~}}{\mathit{\beta }}}_{\left(i\right)}-{\overline{\stackrel{\mathbf{~}}{\mathit{\beta }}}}_{jack}\right){\left({\stackrel{\mathbf{~}}{\mathit{\beta }}}_{\left(i\right)}-{\overline{\stackrel{\mathbf{~}}{\mathit{\beta }}}}_{jack}\right)}^T$

$\mathrm{where} {\overline{\stackrel{\mathbf{~}}{\mathit{\beta }}}}_{\mathrm{j}\mathrm{a}\mathrm{c}\mathrm{k}}=\left({\sum }_{i=1}^n{\stackrel{\mathbf{~}}{\mathit{\beta }}}_{\left(i\right)}\right)/n$.

Through the jackknife algorithm, a consistent variance estimator can also be obtained, since the overall uncertainty of the augmented dataset is explained. Using the jackknife-type SE estimates, Wald-type confidence intervals can be constructed.

3. Results

3.1. Simulations

To evaluate the validity of the proposed methods, we conducted simulation studies. The simulation settings were based on the NCDS [15], which was designed to evaluate the effect of a binary exposure on a binary outcome. We considered three factors in constructing 27 simulation scenarios: outcome event frequency, number of explanatory variables, and sample size. The outcome event frequency was set to 10%, 20%, and 40%. The number of explanatory variables was set to 3, 5, and 10. The sample size was set to 500, 2000, and 10,000. Details of the simulation settings are described in the Supplementary Materials (Table S1).

We compared the performance of four methods in the simulation analyses. These included the augmented logistic regression with the robust variance estimator, the bootstrap-based variance estimator (using both the percentile and BCa methods), and the jackknife-based variance estimator. We replicated the simulations 2000 times for each scenario and evaluated the SE estimates, the coverage probabilities, and the widths of the 95% confidence interval. The number of bootstrap resamples was set to 2000 consistently.

The simulation results are presented in Figure 1, Figure 2 and Figure 3. As a reference in Figure 1, we calculated the empirical standard deviations of the standard error estimates across 2000 simulations. Overall, the proposed bootstrap- and jackknife-based variance estimators produced mean SE estimates that were smaller than those from the robust variance estimator. Moreover, they were also closer to the empirical standard deviations across all scenarios (Figure 1). Furthermore, the coverage probabilities of the proposed estimators were close to the nominal 95% level, while those of the robust variance estimator exhibited over-coverage across all scenarios (Figure 2). The widths of the 95% confidence intervals based on the robust variance estimator were wider than those obtained using the proposed methods, whose widths were generally comparable to each other (Figure 3). These findings suggested that the robust variance estimator was biased, whereas the proposed bootstrap- and jackknife-based estimators provided more precise estimates while preserving adequate coverage rates.

3.2. Applications

As an illustrative analysis, we applied the proposed methods in analyses by the augmented logistic regression of the NCDS data. We primarily aimed to assess the effect of educational attainment on hourly wage in this analysis [15]. We considered a dichotomized outcome of the average hourly wage of individual participants (based on whether the individual’s hourly wage was above the sample mean) and the following fifteen explanatory variables: whether the individual had received any education, self-identified as white, employment status of mother, school type, math score at age 7, math score at age 11, reading score at age 7, reading score at age 11, father’s years of education, mother’s years of education, age of father, age of mother, number of siblings, the interaction between father’s years of education and father’s age, and the interaction between mother’s years of education and mother’s age. The frequency of outcome events was 44.2% (1610 cases/3642 participants) [15], indicating that odds ratios cannot be interpreted as approximations of risk ratios.

Table 1. Results of the augmented logistic regression analysis of the NCDS dataset.

			95% Confidence Interval
Method	Estimate	SE	Lower	Upper
Robust variance	1.836	0.157	1.552	2.172
Bootstrap (Percentile method)	1.836	0.111	1.629	2.072
Bootstrap (BCa method)	1.836	0.111	1.631	2.073
Jackknife approach	1.836	0.114	1.625	2.074

presents the results of the augmented logistic regression analyses using the robust variance estimator, bootstrap, and jackknife approach. The number of resampling of the bootstrap approaches was 2000. The results demonstrated that the SE estimates obtained using the bootstrap- and jackknife-based estimators were smaller than those obtained using the robust variance estimator. This result was consistent with that of our simulation study (Section 4), indicating that the SE estimate obtained using the robust variance estimator was biased. Comparing within the proposed methods, the SE estimates and the widths of confidence intervals were comparable across all of these methods.

4. Discussion and Conclusions

Logistic regression has been widely used in clinical and epidemiological research; however, the resultant odds ratio does not have any epidemiologic interpretation as an effect measure and can only be an approximate estimate of risk ratios, particularly when the outcome is not rare [1,2]. The augmented logistic regression provides a straightforward and useful approach for directly estimating risk ratios without relying on the rare disease assumption, but poses challenges in variance estimation due to the uncertainty of pseudo-observations. Our simulation studies demonstrated that the ordinary robust variance estimator could provide biased variance estimates and inadequate confidence intervals. To address this problem, we proposed alternative variance estimators based on the bootstrap and jackknife resampling. Through the extensive simulation studies, we showed that these proposed methods provide improved SE estimates and confidence intervals.

In the simulation studies, although the robust variance estimator exhibited over-coverage across all scenarios, the SE estimates obtained using the proposed methods were smaller than those from the robust variance estimator and more accurately reflected the uncertainty of the augmented dataset. The proposed methods provided appropriate coverage probabilities and reasonable confidence interval widths, demonstrating superior precision. In the application, the SE estimates obtained using the robust variance estimator were larger, whereas those obtained from the proposed methods were more precise. These findings were consistent with the results of the simulation studies, further supporting the practical utility of the proposed methods.

The augmented logistic regression can be readily implemented using standard statistical software. Because it is based on the logistic regression model [7], it is particularly accessible for research groups, even in settings with limited access to specialized tools. In this study, we addressed the biased variance estimation issue of the augmented logistic regression by applying Noma’s bootstrap-based approach [14] and further extending it to include the jackknife-based alternative. As expected, the bootstrap-based estimator performed well, and our simulation results indicated that the jackknife-based estimator achieved comparable accuracy, supporting its practical use alongside the bootstrap method. Because jackknife resampling removes only one observation at a time, it generally requires far fewer replicates and, therefore, much less computing time than the thousands of bootstrap replicates often needed. This lower computational burden represents a clear advantage when analyzing large datasets or working in resource-constrained research environments.

In conclusion, the proposed methods using the bootstrap and jackknife resampling approaches provide effective alternatives to the conventional robust variance estimator for the augmented logistic regression. They provide more accurate and precise evaluation of effect measures, and thereby would facilitate broader adoption of adequate effect measures reporting in the practice of clinical and epidemiologic studies.