Asymptotic Relative Efficiency of Parametric and Nonparametric Survival Estimators

Abstract

The dominance of non- and semi-parametric methods in survival analysis is not without criticism. Several studies have highlighted the decrease in efficiency compared to parametric methods. We revisit the problem of Asymptotic Relative Efficiency ($ARE$) of the Kaplan–Meier survival estimator compared to parametric survival estimators. We begin by generalizing Miller’s approach and presenting a formula that enables the estimation (numerical or exact) of $ARE$ for various survival distributions and types of censoring. We examine the effect of follow-up time and censoring on $ARE$. The article concludes with a discussion about the reasons behind the lower and time-dependent $ARE$ of the Kaplan–Meier survival estimator.

1. Introduction

In medical statistics, survival analysis is often performed using the nonparametric Kaplan–Meier estimator [1] together with the semiparametric Proportional Hazards Model by Cox [2]. Although the theory of parametric estimators of survival is well-developed [3], their application has been limited in biomedical research. Miller [4] was one of the first to criticize this practice and highlight the loss of efficiency compared to parametric methods. Miller’s work was continued by Klein and Moeschberger [5] and Aranda-Ordaz [6] and was partly rebutted by Meier and collaborators [7] and recently touched upon by Jullum and Hjort [8]. Cheng and Lin [9] proposed a new maximum likelihood estimator under the competing risks model with proportional hazards and assessed its efficacy against the Kaplan–Meier estimator.

In this paper, we revisit the problem of the Asymptotic Relative Efficiency ($ARE$) of the Kaplan–Meier estimator compared to parametric estimators of survival. We begin by extending Miller’s method for calculating the Asymptotic Relative Efficiency ($ARE$) and present a formula that enables the estimation (numerical or exact) of $ARE$ for various survival and censoring distributions, as well as different censoring types. We examine how censoring and the type of censoring affect $ARE$ and discuss the reasons behind the lower and time-dependent $ARE$ of the Kaplan–Meier estimator compared to parametric estimators of survival.

2. Notation and Estimators

We assume that the survival times for an event of interest, $X_1,\dots ,X_n$, are distributed independently and identically according to the distribution function $F\left(x\right)$ and the survival function $S\left(x\right)=1-F\left(x\right)$. We do not always have complete information for all subjects due to loss to follow-up or insufficient follow-up time. The times to censoring are denoted by $C_1,\dots C_n$, and are assumed to be independent and identical, and survival function $\overline{G}\left(c\right)=1-G\left(c\right)$. Further censoring occurs when the study period ends at the pre-determined time $\tau$; all remaining subjects at risk are censored. Thus, the actual observed time for the subject j is $T_j=min\left(X_j,C_j\wedge \tau \right)$. We assume independence between failure and censoring time. We define ${\delta }_j=I\left\{X_j\le C_j\wedge \tau \right\}$ as an event indicator, taking values 1 if an event is recorded prior to censoring, otherwise 0. We consider two competing estimators for the survival function $S\left(t\right)$, the nonparametric Kaplan–Meier estimator, and the parametric maximum likelihood estimator, noted hereafter with the subscripts ${}_{km}$ and ${}_{ml}$. Their respective variances will be noted as $v_{km}$ and $v_{ml}$. In addition, we define $\mathcal{N}\left(t\right)={\sum }_j^{n}I(T_j\le t,{\delta }_j=1)$, the process that counts the number of events in $(0,t]$ and $\mathcal{R}\left(t\right)={\sum }_j^{n}I(T_j>t)$ the number at risk at time t. Although the index is not used to simplify notation through the whole of the text, we generally deal with sequences of estimators indexed by the sample size n.

2.1. The Kaplan–Meier Estimator

The naïve nonparametric survival function estimator $S\left(t\right)=n^{-1}\mathcal{N}\left(t\right)$ is only feasible when there is no censoring in the data, i.e., $P(X\le C)=1$. In this case, the survival at $\forall t$ is a binomial probability with variance $n^{-1}S\left(t\right)(1-S\left(t\right))$.

The nonparametric product limit or Kaplan–Meier estimator [1] that can incorporate censored data is given by

$$\begin{array}{c}\hfill S_{km}\left(t\right)=\prod _{T_i\le t}\left. [1-\frac{{\delta }_i}{\mathcal{R}\left(T_i\right)}\right],\end{array}$$

(1)

where $T_i$ is the ordered sequence of observed event times and t is the absolute time. The Kaplan–Meier is asymptotically unbiased and has as a limiting distribution

$$\begin{array}{c}\hfill \sqrt{n}\left\{{\widehat{S}}_{km}\left(t\right)-S\left(t\right)\right\}\stackrel{D}{\to }N(0,v_{km}),\end{array}$$

(2)

with asymptotic variance

$$\begin{array}{c}\hfill v_{km}\left(t\right)=\frac{1}{n}{\left\{1-F\left(t\right)\right\}}^2{\int }_0^t\frac{f\left(u\right)}{{\left\{1-F\left(u\right)\right\}}^2\left\{1-G\left(u\right)\right\}}dt.\end{array}$$

(3)

This requires knowledge of the survival and censoring distribution, thus is not applicable in real-life situations and its nonparametric variant, the Greenwood estimator, is used:

$$\begin{array}{c}\hfill v_{km}\left(t\right)=S_{km}^2\left(t\right)\sum _{T_i\le t}\frac{{\delta }_i}{\mathcal{R}\left(T_i\right)\left. [\mathcal{R}\left(T_i\right)-{\delta }_i\right]}.\end{array}$$

(4)

There are alternatives to the Greenwood variance estimator, but as Klein [10] concluded, the variance-bias trade-off favours the Greenwood estimator.

The Parametric Survival Estimator

Next, we establish the notation for the maximum likelihood estimators. The parametric survival function is given by ${\widehat{S}}_{ml}(t,\widehat{\theta })$, where $\widehat{\theta }=arg{max}_{\theta }{l}_n\left(\theta \right)$ with $l_n\left(\theta \right)={\sum }_{i=1}^{n}logf(X_i,\theta )$ being the log-likelihood function of the data

$$\begin{array}{c}\hfill l(\theta \mid T_i,{\delta }_i)=\prod _{i=1}^n{\left\{f\left(T_i\right)\overline{G}\left(T_i\right)\right\}}^{{\delta }_i}{\left\{(1-F\left(T_i\right))g\left(T_i\right)\right\}}^{1-{\delta }_i}.\end{array}$$

(5)

As the censoring distribution G does not depend on $\theta$, censoring does not contribute to the estimation of ${\theta }_{ml}$ and it is factored out during maximization of the likelihood. On the other hand, the variance and Fisher information of ${\theta }_{ml}$ does account for censoring [11,12], with

$$\begin{array}{c}\hfill J^T\left(\theta \right)={\int }_0^{\infty }{\left\{\frac{\partial }{\partial \theta }logf\right\}}^{2}f\overline{G}dx=J^X\left(\theta \right)P(X<C),\end{array}$$

(6)

with the superscript ${}^T$ indicating that calculation refers to the observed follow-up time, while the superscript ${}^X$ is the possibly unobserved survival times. Here, $P(X<C)={\int }_0^{\infty }F\left(u\right)g\left(u\right)du$ [13]. In addition we define the score function $U(t,\theta )$ as the first derivative of $logf(X_i,\theta )$ with regard to $\theta$.

Under mild regularity conditions, the maximum likelihood estimator is consistent and converges in probability ${\widehat{\theta }}_{ml}\stackrel{\mathcal{P}}{\to }\theta$ as the sample size $n\to \infty$ and is asymptotically normally distributed,

$$\begin{array}{c}\hfill \sqrt{n}\left\{{\widehat{S}}_{ml}(\widehat{\theta };t)-S({\theta }_0;t)\right\}\stackrel{D}{\to }N(0,v_{ml}),\end{array}$$

(7)

with variance $v_{ml}$

$$\begin{array}{c}\hfill v_{ml}\left(t\right)=\frac{1}{n}{\left\{\frac{\partial S_{ml}(t,\theta )}{\partial \theta }\right\}}^t{J}^{-1}\left\{\frac{\partial S_{ml}(t,\theta )}{\partial \theta }\right\},\end{array}$$

(8)

where J is the Fisher information matrix.

3. Asymptotic Relative Efficiency

For the statistic of interest $S\left(t\right)$ and for consistent estimators $S_{km}\left(t\right)$ and $S_{ml}\left(t\right)$, the Asymptotic Relative Efficiency is given by

$$\begin{array}{c}\hfill ARE(S_{km}\left(t\right),S_{ml}\left(t\right),F)=\frac{v_{ml}\left(t\right)}{v_{km}\left(t\right)}.\end{array}$$

(9)

From the Cramér–Rao theorem, we know that $ARE(S_{km}{S}_{ml},F)\le 1$, and $ARE$ are the needed factor change in the sample size so that the competing estimators perform equally [14].

Theorem 1. 

Under the assumption that $S_{km}\left(t\right)\stackrel{}{\to }S\left(t\right)$ and $S_{ml}\left(t\right)\stackrel{}{\to }S\left(t\right)$ as $n\stackrel{}{\to }\infty$, Equation (9) with censoring considered can be defined as

$$\begin{array}{c}\hfill ARE(S_{km}\left(t\right),S_{ml}\left(t\right),F)=\\ \hfill {\left\{\frac{\partial S_{ml}(t,\theta )}{\partial \theta }\right\}}^t{\left\{J^X\left(\theta \right)P(X<C)S\left(t\right)(1-S\left(t\right))\phi \left(t\right)\right\}}^{-1}\left\{\frac{\partial S_{ml}(t,\theta )}{\partial \theta }\right\}.\end{array}$$

Proof. 

The naïve nonparametric survival function estimator $\widehat{S}\left(t\right)=n^{-1}\mathcal{N}\left(t\right)$ it is feasible only when there is no censoring in the data, i.e., $P(X\le C)=1$. In this case, the survival at $\forall t$ is a binomial probability with variance $n^{-1}S\left(t\right)(1-S\left(t\right))$. As information is lost due to censoring of the binomial variance being inflated by factor $\phi \left(t\right)$ [15], where

$$\begin{array}{c}\hfill \phi \left(t\right)=\frac{S\left(t\right)}{1-S\left(t\right)}{\int }_0^t\frac{\alpha \left(u\right)}{S\left(u\right)\overline{G}\left(u\right)}du\end{array}$$

(10)

and $n^{-1}\widehat{S}\left(t\right)\left(1-\widehat{S}\left(t\right)\right)\phi \left(t\right)$ is estimated by the Greenwood variance estimator. Censoring causes a loss of information with $J^T\left(\theta \right)=J^X\left(\theta \right)P(X<C)$ [11,12], where $P(X<C)={\int }_0^{\infty }F\left(u\right)g\left(u\right)du$ [13]. The Delta-method variance of the maximum likelihood survival estimate, considering the proportion of censoring, is

$$\begin{array}{c}\hfill \frac{1}{n{J}^X\left(\theta \right)P(X<C)}{\left\{\frac{\partial S_{ml}(t,\theta )}{\partial \theta }\right\}}^2,\end{array}$$

(11)

while the variance of the nonparametric Kaplan–Meier estimate with variance inflation due censoring is given by

$$\begin{array}{c}\hfill \frac{1}{n}{S}_{km}\left(t\right)(1-S_{km}\left(t\right))\phi \left(t\right).\end{array}$$

(12)

By plugging these two estimators into Equation for $ARE$, we conclude the proof. □

4. ARE as Correlation between Estimates

Crámer [16] showed that $ARE$ equals the square of the correlation between the competing estimates, ${\rho }^2$, with

$$\begin{array}{c}\hfill \widehat{\rho }(S_{km}\left(t\right),S_{ml}\left(t\right))=\frac{v_c\left(t\right)}{\sqrt{v_{km}\left(t\right)v_{ml}\left(t\right)}},\end{array}$$

(13)

where $v_c=Cov(S_{km}\left(t\right),S_{ml}\left(t\right))$. A necessary and sufficient condition for Crámer’s $ARE$ to equal $ARE$ as defined in Equation (9) is that $v_c$ equals the variance of the most efficient estimator. As both $S_{km}\left(t\right)$ and $S_{ml}\left(t\right)$ are consistent estimates of $S\left(t\right)$, then for any constant $\alpha$, a linear combination of the two in the form of $\alpha S_{km}\left(t\right)(1-\alpha )S_{ml}\left(t\right)$ is also unbiased and has variance

$$\begin{array}{c}\hfill {\{1-\alpha \}}^2{v}_{km}\left(t\right)+2\alpha \{1-\alpha \}{v}_c\left(t\right)+{\alpha }^2{v}_{ml}\left(t\right).\end{array}$$

(14)

This variance is minimum at $\alpha =0$ and the derivative at $\alpha =0$, thus $v_c\left(t\right)=v_{ml}\left(t\right)$. This of course does not imply that $v_c\left(t\right)$ and $v_{ml}\left(t\right)$ will return equal estimates in finite samples but they have the same limit as $n\to \infty$. The estimator for $v_c\left(t\right)$ is presented in the Appendix.

5. ARE for Exponential Survival and Exponential Censoring

In this section, we provide closed form solutions based on Theorem 1 for exponential survival and censoring times. Besides the existence of a closed form solution for $ARE$, the $ARE$ of the exponential distribution can be considered as a lower bound to many distributions used in survival analysis such as Weibull, Gamma, or Generalized Gamma, or any other parametric extension of the exponential distribution. The addition of extra parameters to the model results in the diagonal elements of the inverse of the information matrix being always greater than the corresponding elements of the simpler model. In the following, we assume exponential survival times with hazard $\lambda$ and exponential censoring times with hazard $\gamma$. The survival function for exponential survival times is $S_{\lambda }\left(t\right)=e^{-t\lambda }$ with variance $Var\left(S_{\lambda }\left(t\right)\right)=t^2{\left(e^{-t\lambda }\right)}^2{\lambda }^2{n}^{-1}$. Moreover, as F is known to be an exponential distribution, the event probability for the binomial distribution is $p\left(t\right)=e^{-\lambda t}$. In this case, we have a closed form solution for Theorem 1,

$$\begin{array}{c}\hfill ARE\left(t\right)=\frac{{\left(t\lambda \right)}^2}{(e^{\lambda t}-1)P(X<C)\phi \left(t\right)}.\end{array}$$

(15)

We will now consider four scenarios: (1) no censoring, (2) random censoring, (3) type I censoring with fixed censoring times, and (4) hybrid, type I, and random censoring times. For the sake of simplicity, we will denote $ARE$ with subscripts 1, 2, 3, and 4 for each scenario.

5.1. No-Censoring

If the survival time is recorded for every study participant, then $\phi \left(t\right)=P(x<Y)=1$ giving

$$\begin{array}{c}\hfill AR{E}_1\left(t\right)=\frac{{\left(t\lambda \right)}^2}{(e^{\lambda t}-1)}.\end{array}$$

(16)

In this case, the maximum efficacy of the nonparametric estimator is 64.76% and happens at $t=1.593{\lambda }^{-1}$, i.e., 1.6 times the mean survival time.

5.1.1. Random Censoring Times

In this scenario, the observed time for subject j is $T_j=min\left(X_j,C_j\right)$ and ${\delta }_j=I\left\{X_j\le C_j\right\}$. Additionally, we have

$$\begin{array}{c}\hfill P(X<C)=\frac{\lambda }{\lambda +\gamma }\mathrm{and}\phi \left(t\right)=\frac{\lambda }{\lambda +\gamma }\frac{\left(e^{t\left(\lambda +\gamma \right)}-1\right)}{\left(e^{t\lambda }-1\right)}.\end{array}$$

(17)

Combing the above with Equation (15), the asymptotic relative efficiency is

$$\begin{array}{c}\hfill AR{E}_2\left(t\right)=\frac{{\left\{t(\lambda +\gamma )\right\}}^2}{(e^{t(\lambda +\gamma )}-1)}.\end{array}$$

(18)

The maximum efficacy of the nonparametric estimator is observed at $t=1.593{(\lambda +\gamma )}^{-1}$, and as for the non-censored case, it equals 64.76%. As $1.593{(\lambda +\gamma )}^{-1}\le 1.593{\left(\lambda \right)}^{-1}$, this maximum is reached earlier, and in the beginning of the follow-up period, censoring increases the efficiency of the Kaplan–Meier estimator (Figure 1).

5.1.2. Type I or Fixed Censoring

Here, the observed time for subject j is $T_j=min\left(X_j,\tau \right)$ and ${\delta }_j=I\left\{X_j\le \tau \right\}$. In this case, $\forall t<\tau$$\phi \left(t\right)=1$ and $P(X<\tau )=(1-e^{-\lambda \tau })$, resulting in

$$\begin{array}{c}\hfill AR{E}_3\left(t\right)=\frac{{\left(t\lambda \right)}^2}{(e^{t\lambda }-1)(1-e^{-\lambda \tau })}.\end{array}$$

(19)

As $(1-e^{-\lambda \tau })\le 1$, the $ARE$ of the nonparametric variance estimator exceeds the $ARE$ of the nonparametric variance estimator when there is no censoring and ${lim}_{\tau \to \infty }AR{E}_2\left(t\right)=AR{E}_1\left(t\right)$.

5.1.3. Hybrid Type I and Random Censoring

Here, just as for Type I censoring, follow-up is stopped at a pre-assigned time point $\tau$, but in addition participants might leave the study prior to $\tau$ for reasons unrelated to the studied disease. Thus, $T_j=min\left(X_j,C_j\wedge \tau \right)$. The minimum of two exponentially distributed variables with parameters $\lambda$ and $\gamma$ is also exponentially distributed with parameter $\lambda +\gamma$, thus

$$\begin{array}{c}\hfill AR{E}_4\left(t\right)=\frac{{\left(t(\lambda +\gamma )\right)}^2}{(e^{t(\lambda +\gamma )}-1)(1-e^{-(\lambda +\gamma )\tau })}.\end{array}$$

(20)

Type I (or fixed) censoring has no impact on the Greenwood variance estimator, but it does reduce the information available in the data for the estimation of the parametric variance with $P(X<C\wedge \tau )$ and improves the $ARE$ of the Kaplan–Meier estimator up $\tau$ by $P{(X<C\wedge \tau )}^{-1}$. At relatively short follow-up times (i.e., less than the average survival time) and high survival rates, the point-wise ARE of the Kaplan–Meier estimator at $\tau$ and close to $\tau$ can exceed 64% (Figure 2). Of course, if there is additional random censoring, the heavier the censoring the greater the asymptotic relative efficiency of the parametric survival estimator (see Appendix B). For the effect of additional dimensions to the parametric estimators, see Appendix C.

5.2. Efficiency in Finite Samples

While comparing the asymptotic variances of different estimates is essentially comparing confidence intervals and is invariant under nonlinear transformation of the target parameter, Cox [17] highlighted that such stability of interpretation does not always hold true for small-sample comparisons. Serfling [14] noted that asymptotic relative efficiencies pertain to large sample comparisons and need not reliably indicate small sample performance, and exemplified with the relative efficiency of median vs mean as an estimator of location for normal distribution. Identical trends can be seen in the survival analysis as well. For the closely related Nelson–Allen estimator, Peña and Rohatgi [18] concluded that, for sample sizes under 30 asymptotic approximations can be misleading, specially at the extremes of the follow-up times. With the use of a simulation study, we looked at the ratio of variances’ small sample characteristics. We assumed survival times are exponential with a rate of $1/365$ and with sample sizes ranging from 25 to 1000. We have fitted a parametric and the non-parametric Kaplan–Meier estimator to the data and survival times of 50, 180, and 365.

Table 1. Ratio of variances for Kaplan–Meier and parametric survival estimates for exponential survival times with a median survival time of 365 as a function of sample size and the procentual deviation for $ARE$ in parenthesis.

N	t = 50	t = 180	t = 365
ARE	12.78	38.15	58.20
25	16.35 (27.92%)	40.63 (19.4%)	59.15 (7.45%)
50	14.89 (16.49%)	39.32 (9.15%)	58.67 (3.70%)
75	14.03 (9.77%)	39.09 (7.35%)	58.48 (2.21%)
100	13.65 (6.79%)	38.65 (3.91%)	58.31 (0.88%)
125	13.57 (6.17%)	38.69 (4.22%)	58.51 (2.44%)
150	13.35 (4.45%)	38.61 (3.59%)	58.41 (1.66%)
200	13.22 (3.43%)	38.44 (2.26%)	58.34 (1.11%)
250	13.11 (2.57%)	38.27 (0.93%)	58.30 (0.37%)
1000	12.89 (0.85%)	38.23 (0.62%)	58.23 (0.25%)

summarises the results of 1000 such simulations. The findings imply that the Kaplan–Meier estimator’s small sample relative effectiveness depends on both the sample size and the survival time. In finite samples, the ratio of variances typically surpasses $ARE$, and this is particularly clear at the start of the follow-up when $ARE$ is low. This can be somewhat explained by how reliable the estimation is. Outliers in small samples can inflate variances and skew parametric survival estimates [6]. The finite sample bias of the maximum likelihood parameter [19,20] and the Kaplan–Meier estimator complicates comparisons, and the likely comparison of Mean Squared Errors should be preferred over a comparison of variances [21].

6. Discussion

With a focus on the impact of censoring and different forms of censoring on $ARE$, we compared the asymptotic relative efficiency of the Kaplan–Meier survival estimator vs. parametric survival estimators in this work. Due to the existence of closed-form solutions as well as historical considerations, we have used the exponential distribution as an example. Previous papers on the subject noted that the maximum $ARE$ of the Kaplan–Meier estimator in the case of exponential survival times is 64% [4,8,22]. However, the maximum $ARE$ can be close to 1, as we have demonstrated in studies using type I censoring. This is particularly clear when the average survival time is longer than the follow-up time. Heuristically, the amount of information available for estimation and the way in which this information is used are the two main factors that account for the Kaplan–Meier estimator’s relatively low $ARE$ in comparison to parametric estimators. The Kaplan–Meier survival and the Greenwood variance estimator only use data up to time t and censor anything after that for any given time-point t. All available data are utilized by parametric estimators. The way in which these data are used is the second factor that needs to be taken into account. The parameter vector of the assumed distribution is estimated by parametric models. Since the parameters of the distribution and time are mapped one-to-one to parametric survival probabilities, the correlation between survival estimates at any given two time points is 1. Naturally, knowledge of the survival probabilities at $t_1$ also provides knowledge of the Kaplan–Meier estimates at $t_2$. If $t_1<t_2$, then the two survival probabilities are correlated and, by definition, $S\left(t_1\right)>S\left(t_2\right)$. However, as the time interval between $t_1$ and $t_2$ increases and as the percentage of censored observations increases, the correlation between $S\left(t_1\right)$ and $S\left(t_2\right)$ decreases (see Appendix D). Asymptotic Relative Efficiency was long associated with significance testing, and the alternative formulation as the limiting correlation coefficient between estimators was first applied in [23,24]. With the help of simulation, we have illustrated that the realizations, on average, are equal between the ratio of variances and the Crámer correlation estimate of $ARE$. From the perspective of correlation, we can look at $ARE$ as the amount of (linear) information that the Kaplan–Meier estimate contains about the form of the parametric model. Near zero $ARE$ or correlation at the beginning of the follow-up likely suggests that the Kaplan–Meier estimate confers limited knowledge about possible parametric estimators, which might complicate the data driven choice of parametric estimators [8,25,26]. While theoretically this is a problem at follow-up times where the survival probability approaches zero, in practice, at that time, survival estimates are rarely meaningful from an interpretation point of view [27].

In this paper, we offered a more nuanced view of the question raised by Miller [4], the price (i.e., loss of efficiency) of the routine use of the Kaplan–Meier estimator. We looked into the different censoring types and their effect of the $ARE$. Contrary to previous studies, we suggest that the price of the Kaplan–Meier estimator is acceptable. As we have shown, studies with Type I censoring have relatively high efficiency. We have also seen that adding extra dimensions to parametric estimators increases the efficiency of the Kaplan–Meier estimator. While the exponential distribution is widely used in theoretical works and research planning phases, its relatively rigid form makes it an unlikely candidate for a parametric model. Looking at $ARE$ from the perspective of correlation suggests that applying the framework of Jullum and Hjort [8,25] to select parametric models for studies with follow-up times shorter than the mean survival time could be difficult. It is likely that, for the foreseeable future, the Kaplan–Meier estimator will be the method of choice in survival analysis.