A Brief Overview of Restricted Mean Survival Time Estimators and Associated Variances

Abstract

Restricted Mean Survival Time ($RMST$) experiences a renaissance and is advocated as a model-free, easy to interpret alternative to proportional hazards regression and hazard rates with implication in causal inference. Estimation of $RMST$ and associated variance is mainly done by numerical integration of Kaplan–Meier curves. In this paper we briefly review the two main alternatives to the Kaplan–Meier method; analysis based on pseudo-observations, and the flexible parametric survival method. Using computer simulations, we assess the efficacy of the three methods compared to a fully parametric approach where the distribution of survival times is known. Thereafter, the three methods are directly compared without any distributional assumption for the survival data. Generally, flexible parametric survival methods outperform both competitors, however the differences are small.

1. Introduction

In 1958, Kaplan and Meier [1] published their paper on the product limit estimator for survival probabilities, an estimator that can handle censored and/or truncated survival data. The easy to use and interpretable nature of the method, made it an instant success. The Kaplan–Meier estimator have since then become the dominant method to summarize survival data and parametric methods have lost relevance. This prompted Miller [2] to criticize this over-reliance of applied scientist on the Kaplan–Meier curve. Miller established, both theoretically and numerically, the loss of efficacy of the Kaplan–Meier curve compared to parametric methods. Jullum and Hjort [3] further explored this aspect and corroborated Miller’s findings. Meier et al. [4] have addressed some of Miller’s critique, by highlighting the difficulties in choosing the right parametric form. In addition to the Kaplan–Meier curve, Meier et al. [4] have addressed the efficacy of a functional of the Kaplan–Meier curve: the Restricted Mean Survival Time ($RMST$), the average survival time up to a given time point. Theoretical aspects of $RMST$ (or Kaplan–Meier integrals) are well studied [5,6,7] and currently $RMST$ is experiencing a renaissance [8,9,10] being hailed as a model-free, easy to interpret statistic with implications in causal inference [11]. Not needing to establish the proper parametric model has certain advantages. However, generally non-parametric methods have lower efficacy than parametric ones. Generally, practical applications do not consider parametric estimators for $RMST$ and use either numerical integration of the Kaplan–Meier curve [12], pseudo-observations [13,14,15] or flexible parametric survival methods [16]. In this paper, we briefly summarize parametric and non-parametric estimators for $RMST$ and associated variance estimators. We will use Monte Carlo simulations to examine their efficacy.

2. Restricted Survival Times

We assume that survival times to an event of interest $X_1,\dots ,X_n$ are independently and identically distributed ($iid$) according to the cumulative distribution function $F\left(x\right)$ and survival function $S\left(x\right)$. Similarly we assume $C_1,\dots ,C_n$ to be $iid$ censoring times according to the distribution function $G\left(c\right)$. Additionally, we assume that follow-up times are restricted, by an upper threshold $\tau$ set by the researcher. Thus the actual observed time for subject j is $T_j=min\left(X_j,C_j,\tau \right)$. Additionally, we have ${\delta }_j=I\left\{X_j\le min(C_j,\tau )\right\}$ as event indicator that takes a value of 1 if the event of interest is recorded at the given time, 0 otherwise. We assume independence between failure and censoring time. We let $t_{\left(1\right)}\le ...\le t_{\left(n\right)}$ denote the ordered observed survival times and ${\delta }_{\left(1\right)},...,{\delta }_{\left(n\right)}$ their associated indicator values. Survival times are summarized with the help of the survival function $S\left(t\right)=P(T>t)$ and the $\tau$-restricted mean survival times ar given by ${\int }_0^{\tau }S\left(t\right)dt$ [12].

From Andersen et al. [14] we know that $\sqrt{n}\left({\int }_0^{\tau }\widehat{S}\left(t\right)dt-{\int }_0^{\tau }S\left(t\right)dt\right)\stackrel{\mathcal{D}}{\to }\mathcal{N}(0,{\sigma }^2)$ for some variance ${\sigma }^2>0$. In the next sections we review, then evaluate, different estimators for ${\int }_0^{\tau }S\left(t\right)dt$ and ${\sigma }^2$.

3. Estimators of RMST and Associated Variance

The statistical literature describes several parametric and non-parametric ways of estimating ${\int }_0^{\tau }S\left(t\right)dt$ and its associated variance.

3.1. Parametric Methods

For parametric estimation, we need to make assumptions about the distribution of the unobserved survival times, X. Once the distribution function is established, the Restricted Mean Survival Time $\left({\mu }_{\tau }\right)$ is estimated as

$$\begin{array}{c}\hfill {\mu }_{\tau }={\int }_0^{\tau }S(\theta ,t)dt.\end{array}$$

where $\theta$ is the parameter vector for the assumed distribution function.

In the following, we outline three possibilities for parametric estimation

• Likelihood and $\delta$-method based approach.
  Estimation of ${\int }_0^{\tau }S(\theta ,t)dt$ requires knowledge of $\theta$. Estimates for $\theta$ under censoring can be obtained with maximum likelihood. Given the censoring indicator $I_{\{X_j\le C_j\}}$ and $Y_j=min(X_j,C_j)$ we can define the likelihood function for $\theta$ as

  $$\begin{array}{c}\hfill L\left(\theta \right|Y,\delta )=\prod _{j=1}^{n}f{(Y_j,\theta )}^{I_{\{X_j\le C_j\}}}{\left[1-F(Y_j,\theta )\right]}^{1-I_{\{X_j\le C_j\}}}.\end{array}$$
  For most cases there are no closed-form solutions for $\widehat{\theta }$ and $Var\left(\widehat{\theta }\right)$, however, numerical estimates are enough. Once estimates for $\theta$ are available, ${\int }_0^{\tau }S(\theta ,t)dt$ can be estimated either by using $\widehat{\theta }$ in closed form equations, or as plug-ins in numerical integration. The variance for ${\int }_0^{\tau }S(\theta ,t)dt$ by the $\delta$-method is given by

  $$\begin{array}{c}\hfill Var\left({\mu }_{\tau }\right)=\sum _i{\left(\frac{\delta {\mu }_{\tau }}{\delta {\theta }_i}\right)}^{2}Var\left({\theta }_i\right)+\sum _i\sum _{i\ne j}\frac{\delta {\mu }_{\tau }}{\delta {\theta }_i}\frac{\delta {\mu }_{\tau }}{\delta {\theta }_j}Cov({\theta }_i,{\theta }_j)\end{array}$$

  where ${\theta }_i$ is the $i^{th}$ element of the population parameter.
• M-estimation (or estimating equation)
  M-estimation seeks solution to the vector equation ${\sum }_{i=1}^n\mathit{\psi }({\mathit{T}}_{\mathit{i}},\widehat{\mathit{\theta }})=0$. Here, $T_i$ are independently and identically distributed restricted survival times, $\widehat{\mathit{\theta }}$ is a p-dimensional parameter and $\mathit{\psi }$ is a known $(p\times 1)$-function that does not depend on i or n [17]. M-estimates are asymptotically normal with variance $n^{-1}V\left({\mathit{\theta }}_{\mathbf{0}}\right)$. $V\left({\mathit{\theta }}_{\mathbf{0}}\right)$ is a sandwich variance estimator given by

  $$\begin{array}{c}\hfill \mathit{V}\left({\mathit{\theta }}_{\mathbf{0}}\right)=\mathit{A}{\left({\mathit{\theta }}_{\mathbf{0}}\right)}^{-1}\mathit{B}\left({\mathit{\theta }}_{\mathbf{0}}\right){\left\{\mathit{A}{\left({\mathit{\theta }}_{\mathbf{0}}\right)}^{-1}\right\}}^T.\end{array}$$

  (1)

  where,

  $$\begin{array}{c}\hfill \mathit{A}\left({\mathit{\theta }}_{\mathbf{0}}\right)=\mathbb{E}\left[-\frac{\partial }{\partial {\mathit{\theta }}^T}\mathit{\psi }({\mathit{T}}_{\mathit{i}},\mathit{\theta })\right]\end{array}$$

  (2)

  and

  $$\begin{array}{c}\hfill \mathit{B}\left({\mathit{\theta }}_{\mathbf{0}}\right)=\mathbb{E}\left[\mathit{\psi }({\mathit{T}}_{\mathit{i}},\widehat{\mathit{\theta }})\mathit{\psi }{({\mathit{T}}_{\mathit{i}},\mathit{\theta })}^T\right].\end{array}$$

  (3)
  Wang [18] discusses alternative formulations for $\mathit{\psi }$ with regards to the strictly non-negative and often skewed nature of survival data. For an in-depth discussion on the M-estimators the reader is referred to [17,19].
• The second cumulant
  The variance equivalent to the second cumulant of a probability distribution of $T_j$. If there is no censoring present $Var\left(T\right)={\mathbb{E}}_{\tau }\left[T^2\right]-{\mathbb{E}}_{\tau }{\left[T\right]}^2$ and can be estimated as

  $$\begin{array}{c}\hfill Var\left(T\right)=2{\int }_0^{\tau }tS\left(t\right)dt-{\left[{\int }_0^{\tau }tS\left(t\right)dt\right]}^2\end{array}$$

  leading to $Var\left({\mu }_t\right)=n^{-1}Var\left(T\right)$.
  This estimator is not practical due to censoring. When we have censoring in the data Rosyton & Parmar [20] suggested multiplying $n^{-1}Var\left(T\right)$ with a positive scaling factor $\varphi$, so that $\varphi =1$ if no censoring and $\varphi >1$ otherwise. The scaling factor $\varphi$ can be estimated with help of Monte Carlo simulation.
  Alternatively, the variance can be estimated with the Stute estimator [6]. Adopting the notation of Stute [21], (i.e., $\int xdF\left(x\right)=\int xf\left(x\right)dx$) and $(1-H)=(1-F)(1-G)$ and $\phi$ a transformation of X so that $n^{-1}{\sum }_i\phi \left(X_i\right)\to \int \phi dF\equiv S^{\phi }={\mu }_{\tau }$. Here, $Var[\int \phi d{\widehat{F}}_n]=n^{-1}{\sigma }_1^2$ where

  $$\begin{array}{c}\hfill {\sigma }_1^2={\int }_0^{\tau }\frac{{\phi }^2}{1-G}dF-{\left[{\int }_0^{\tau }\phi dF\right]}^2-{\int }_0^{\tau }{\left[{\int }_x^{\tau }\phi dF\right]}^2\frac{1-F\left(x\right)}{{[1-H\left(x\right)]}^2}G\left(dx\right).\end{array}$$
  Both approaches require assumptions about the distribution of the survival times and censoring times. These two estimators of variance will not be further evaluated, however they do represent an important aspect of $RMST$.

3.2. Flexible Parametric Survival Methods

In 2002, Royston and Parmar [22] introduced flexible parametric survival models as a simple way to deal with non-proportional hazards, and as a way of easy visualization of the hazard function. In a later paper the authors exemplified the utility of the model in estimation of $RMST$ [16]. The model is defined through the log cumulative hazard function as

$$\begin{array}{c}\hfill lnH(t,\mathit{z})=ln{H}_0\left(t\right)+{\mathit{z}}^T\left(\beta \right),\end{array}$$

(4)

where the baseline ($H_0$) is modeled as a restricted cubic spline in log time, $ln{H}_0\left(t\right)=s(lnt)$, and where $\mathit{z}$ is a covariate vector. The splines $s(lnt)$ are linear combination of basis functions and regression parameters. The model requires a number of knots and a probability distribution function. This later is often assumed to be the Weibull distribution, which we used in this article, with 2 knots. $RMST$ is estimated by predicting the log cumulative hazard function using Equation (4) at a suitable grid of time points equally spaced between 0 and $\tau$. Thereafter the cumulative hazard is transformed into the survival function ${\widehat{S}}_{RP}\left(t\right)$, and ${\mu }_{\tau }={\int }_0^{\tau }{\widehat{S}}_{RP}\left(t\right)dt$. The variance is estimated with the $\delta$-method or by bootstrapping.

While this approach is fully parametric and the survival function is completely specified, the model is ought to be robust against distribution misspecification. Integration proceeds with the formula of the survival function of the assumed distribution, but with numerical integration for the area under ${\widehat{S}}_{RP}\left(t\right)$ between 0 and $\tau$. Additionally, the splines and the piece-wise modeling between the knots offer great flexibility to the model.

3.3. Non-Parametric Methods

3.3.1. The Kaplan–Meier-Method

The Kaplan–Meier method is perhaps the most well-known and used method for estimation of $RMST$. In 1975, Meier [12] proved that the substitution of the Kaplan–Meier estimator in ${\int }_0^{\tau }S\left(t\right)dt$ provides an (approximately) unbiased estimate that is asymptotically normal.

The Kaplan–Meier estimator is given by

$$\begin{array}{c}\hfill {\widehat{S}}_{KM}\left(t\right)=\prod _{T_i\le t}\left[1-\frac{{\delta }_i}{R_i}\right]\end{array}$$

where $T_i$ is the ordered follow up time, $R_i$ is the number of subjects at risk prior to $T_i$ and ${\delta }_i$ the number of events that happened at time $T_i$. The $RMST$ is estimated as

$$\begin{array}{c}\hfill {\mu }_{\tau }={\int }_0^{\tau }{\widehat{S}}_{KM}\left(t\right)dt.\end{array}$$

As far as we know, Irwin in 1949 [23], was the first to provide an estimator for the variance based on life tables. Later, Meier [12] in 1975 proved that reducing the data to the life table format was unnecessary and established the following variance estimator

$$\begin{array}{c}\hfill V\left({\mu }_{\tau }\right)\approx 2{\int }_0^{\tau }{\int }_u^{\tau }S\left(u\right)S\left(v\right)C\left(u\right)dvdu,\end{array}$$

where $C\left(u\right)={\sum }_i\left[(1-S\left(u_i\right))/nS\left(u_i\right)\right]$ is the sum of the squared coefficient of variation for $S\left(u\right)$. Klein & Moeschberger [24] provided a variant with the Greenwood variance estimator of survival probabilities as plug-ins and popularized Meier’s variance estimator in the following form:

$$\begin{array}{c}\hfill \widehat{V}\left({\mu }_{\tau }\right)=\sum _{i=1}^D{\left[{\int }_{t_i}^{\tau }{\widehat{S}}_{KM}\left(t\right)dt\right]}^2\frac{{\delta }_i}{Y_i(Y_i-{\delta }_i)},\end{array}$$

where $D={\sum }_i{\delta }_i$. The Aalen - Johansen variance estimator of survival probabilities could also serve as plug-in in Meier’s estimator, or re-sampling methods can be used as well. However, as Eaton et al. [25] noted based on Monte Carlo simulations, the estimator with the Greenwood plug-in give variance estimates closest to empirical and asymptotic variances.

3.3.2. Pseudo-Observations

Pseudo-observations is a method that is based on the pseudo-values from a jackknife statistic constructed from simple summary statistic estimates which are then used in a generalized estimating equation to obtain estimates of the model parameters [13,15].

The parameter of interest is ${\mu }_{\tau }$. If there is no censoring, then ${\mu }_{\tau }=n^{-1}{\sum }_i{T}_i$. Under censorship, an approximately unbiased estimator for ${\mu }_{\tau }$ is ${\int }_0^{\tau }\widehat{S}\left(t\right)dt$. The pseudo-observation $T_j^{*}$ associated with the $j^{th}$ observed time is estimated as

$$\begin{array}{c}\hfill T_j^{*}=n{\int }_0^{\tau }\widehat{S}\left(t\right)dt-(n-1){\int }_0^{\tau }{\widehat{S}}^{-j}\left(t\right)dt\end{array}$$

where $S^{-j}$ is the survival curve estimated withouth the $j^{th}$ observation. The assumption is that $\mu \left(T^{*}\right)\approx \mu \left(T\right)$ and ${\sigma }^2\left(T^{*}\right)\approx {\sigma }^2\left(T\right)$. The most straightforward estimators are $\widehat{\mu }\left(T^{*}\right)=n^{-1}{\sum }_j{T}_j^{*}$ and ${\sigma }^2\left(T^{*}\right)={(n-1)}^{-1}{\sum }_j{(T_j^{*}-{\overline{T}}^{*})}^2$. The approach preferred by Andersen et al. [13] is based on M-estimators. The usual assumption is that the underlying distribution is symmetric about the location parameter and the corresponding function $\mathit{\psi }$ is an odd function for the location parameter (mean in our case, ${\theta }_1$) and an even function for the targeted scale parameter (variance, ${\theta }_2$) defined as

$$\begin{array}{c}\hfill \sum _{i=1}^n\mathit{\psi }(T_j^{*},{\widehat{\theta }}_1,{\widehat{\theta }}_2)=\left(\begin{array}{c}(T_j^{*}-{\theta }_1)\\ {(T_j^{*}-{\theta }_1)}^2-{\theta }_2\end{array}\right)=\left(\begin{array}{c}0\\ 0\end{array}\right)\end{array}$$

Solving for $\mathit{V}\left({\mathit{\theta }}_{\mathbf{0}}\right)$ (Equation (1)) starts with solving $\mathit{A}\left({\mathit{\theta }}_{\mathbf{0}}\right)$ (Equation (2)) and $\mathit{B}\left({\mathit{\theta }}_{\mathbf{0}}\right)$ (Equation (3)). In this case $\mathit{A}\left({\mathit{\theta }}_{\mathbf{0}}\right)=\mathit{I}\left({\mathit{\theta }}_{\mathbf{0}}\right)$, the identity matrix. Solving for $\mathit{B}\left({\mathit{\theta }}_{\mathbf{0}}\right)$ gives

$$\begin{array}{c}\hfill \mathit{B}\left({\mathit{\theta }}_{\mathbf{0}}\right)=\left(\begin{array}{cc}{\sigma }^2& {\mu }_3\\ {\mu }_3& {\mu }_4-{\sigma }^4\end{array}\right)\end{array}$$

where, ${\mu }_k$ is the $k^{th}$ central moment. The variance estimator $\mathit{V}\left({\mathit{\theta }}_{\mathbf{0}}\right)=\mathit{I}{\left({\mathit{\theta }}_{\mathbf{0}}\right)}^{-1}\mathit{B}\left({\mathit{\theta }}_{\mathbf{0}}\right){\left\{\mathit{I}{\left({\mathit{\theta }}_{\mathbf{0}}\right)}^{-1}\right\}}^T=\mathit{B}\left({\mathit{\theta }}_{\mathbf{0}}\right).$ The variability estimates by $\mathit{V}\left({\mathit{\theta }}_{\mathbf{0}}\right)$ are robust to model misspecification. For symmetric distributions (i.e., $f(\mu +x)=f(\mu -x)$) the off-diagonal elements of $\mathit{V}\left({\mathit{\theta }}_{\mathbf{0}}\right)$ are zero. For asymmetric distributions (i.e., ${\mu }_3>0$) there is an asymptotic variance inflation due to estimating variance in addition to the mean [19].

4. Simulation Studies

We used Monte Carlo simulations to gain insight of the small sample proprieties of the estimators. We used the $\mathbf{R}$ computing environment (R 3.6.0) [26] for calculations and illustrations. Codes are available from the corresponding author.

4.1. Relative Efficacy under Parametric Assumption

A first set of simulation studies was used to assess the relative efficacy of non-parametric methods and flexible parametric survival methods compared to the parametric estimates of variance. The relative efficiency (eff) of the estimators was assessed as the ratio between the estimated variances of the different estimators and the estimator with the minimum variance

$$\begin{array}{c}\hfill \mathit{eff}=\frac{min\langle {\sigma }_M^2\rangle }{{\sigma }_M^2},\end{array}$$

where M is the model considered for estimation. We were interested in seeing how relative efficacy varies as a function of (i) sample size, (ii) censoring percentage and (iii) restriction time. To facilitate calculation and possibility of obtaining closed form solutions for the parametric variance estimates, we assumed that survival times were exponentially distributed with rate, $\lambda =1/365$. Similarly, we assumed that censoring times were exponentially distributed with rate $\gamma$.

In the first set of simulations we set the sample size to 50 and 100 with censoring percentages at 50% and 75%. Given the assumed censoring percentages ($cp$) we estimated $\gamma$, the rate parameter for the censoring distribution, as the solution for the equation

$$\begin{array}{c}\hfill cp=1-\lambda {(\lambda +\gamma )}^{-1}\end{array}$$

(5)

where $\lambda {(\lambda +\gamma )}^{-1}$ is an estimate for $P\left(X<C\right)={\int }_0^{t}C\left(t\right)f\left(t\right)dt$ [27]. Every setting was simulated 1000 times and we estimated ${\mu }_{\tau }$ and $Var\left({\mu }_{\tau }\right)$ with the $\delta$-method, M-estimation, the Kaplan–Meier method, flexible parametric survival and pseudo-observations with $\tau =365$.

Table 1. Efficacy ($\mathit{eff}$) and 95 % confidence interval coverage probabilities of the different Restricted Mean Survival Time ($RMST$) estimators as a function of sample size (n) and censoring percentage ($cp$) at restriction time of 365 days, with the expected ${\mu }_{\tau }$ of 230 days.

	$\mathit{\delta }$-Method	M-Estimator	Kaplan–Meier	Flexible Parametric	Pseudo-Obs
n = 50, CP = 50 %
$\widehat{{\mu }_{\tau }}$	230.1	230.1	230.7	231.2	230.7
$\widehat{Var\left({\mu }_{\tau }\right)}$	368.0	362.1	437.5	429.3	446.9
eff	0.98	-	0.82	0.84	0.81
Coverage	0.94	0.94	0.93	0.93	0.94
n = 50, CP = 75 %
$\widehat{{\mu }_{\tau }}$	230.2	230.2	230.2	228.0	228.4
$\widehat{Var\left({\mu }_{\tau }\right)}$	724.6	712.9	792.1	794.6	1068.7
eff	0.98	-	0.90	0.90	0.66
Coverage	0.93	0.92	0.89	0.89	0.90
n = 100, CP = 50 %
$\widehat{{\mu }_{\tau }}$	230.9	230.9	231.4	231.6	231.4
$\widehat{Var\left({\mu }_{\tau }\right)}$	185.2	182.2	221.2	217.0	223.4
eff	0.98	-	0.83	0.84	0.81
Coverage	0.96	0.95	0.95	0.95	0.95
n = 100, CP = 75 %
$\widehat{{\mu }_{\tau }}$	229.9	229.9	229.4	229.1	229.1
$\widehat{Var\left({\mu }_{\tau }\right)}$	367.0	365.1	421.8	406.3	485.6
eff	0.99	-	0.86	0.89	0.75
Coverage	0.95	0.94	0.92	0.94	0.94

presents the results of the simulation study. The expected ${\mu }_{365}=230$ days. Generally, all methods provided unbiased estimates of ${\mu }_{365}$, with small deviations.

Of the 5 tested variance estimators, M-estimation proved to have the smallest estimates. The $\delta$-method variance differed only marginally from the M-estimation. Of the three practically usable methods, the Kaplan–Meier method, pseudo-observations and flexible parametric survival, the later had the highest relative efficacy, irrespective of the setting. To assess if the lower variance estimate of the flexible parametric survival, compared with the Kaplan–Meier method and pseudo-observation affects negatively statistical inference, we assessed the coverage portability of the approximate 95% Wald-type confidence intervals. There is no indication that the lower variance estimate of the flexible parametric survival method negatively affects statistical inference compared to Kaplan–Meier method and pseudo-observation. Although this simulation is not comprehensive we can conclude that the used Wald-type statistics in statistical inference for $RMST$ should be used with caution when sample sizes are small and censoring percentage is high.

To gain further insights of the relative efficacy of the variance estimator a second set of simulation studies was run. First we simulated survival times with $\lambda =1/365$ and $\gamma =1/365$ with a sample size of 100 and a censoring percentage of 50%. We varied $\tau$ between 50 and 650 days and estimated ${\mu }_{\tau }$ and $Var\left({\mu }_{\tau }\right)$ with M-estimation, the Kaplan–Meier method, flexible parametric survival and pseudo-observations.

All estimators returned unbiased ${\mu }_{\tau }$ estimates. As Figure 1a shows, the Kaplan–Meier method, flexible parametric survival and pseudo-observations have lower efficacy compared to parametric estimate of the variance. The estimated increases with longer restriction times and declines after a maximum is reached approximately at 600 days, when around 80% of the subjects should have experienced the event of interest.

Similarly, we have assessed how the censoring percentage affects efficacy when everything else is kept constant. We simulated survival times with $\lambda =1/365$, sample size of 100 and set the restriction time at 365 days.

We estimated $\gamma$, the censoring rate with Equation (5). The pattern was similar to the previous setting. The efficacy increased up to around 65% censoring rate, thereafter declined again.

4.2. Estimation with Unknown Distribution Function

Practitioners rarely, if ever, know the distribution of survival times, which makes parametric estimation difficult, if not impossible. Here, we aim to assess the precision, efficacy and confidence internal coverage, for the Kaplan–Meier method, pseudo-observations and flexible parametric survival methods. For the former two, distributional assumptions are less important. For flexible parametric survival methods with an assumed Weibull survival function, deviations from the assumptions might affect the outcome. In a first set of simulations, we simulated survival times with the log-logistic distribution, a distribution used in accelerated failure models. The hazard of the log-logistic distribution can take non-monotonic forms depending on the shape parameter. We assumed shape of $1.5$ and a scale of 151, giving a mean survival time of approximately 365 days. Thereafter, we simulated survival times with a mixture distribution with $f\left(t\right)=0.5exp(1/265)+0.5exp(1/465)$. In both cases we assumed exponential censoring with a rate of $\gamma =1/365$.

Table 2. Efficacy and 95% confidence interval probabilities of the different $RMST$ estimators at the mean survival time of 365 days, with the expected ${\mu }_{\tau }$ of 182 days for the Log-logistic distribution and 225 days for the exponential distribution.

	Kaplan–Meier	Flexible Parametric	Pseudo-Obs
Log-logistic, n = 50
$\widehat{{\mu }_{\tau }}$	183.4	183.9	183.4
$\widehat{Var\left({\mu }_{\tau }\right)}$	402.6	397.6	413.9
Coverage	0.93	0.94	0.93
Log-logistic, n = 100
$\widehat{{\mu }_{\tau }}$	182.2	182.7	182.2
$\widehat{Var\left({\mu }_{\tau }\right)}$	201.7	199.4	204.2
Coverage	0.94	0.94	0.94
Exp-mix, n = 50
$\widehat{{\mu }_{\tau }}$	224.5	224.6	224.5
$\widehat{Var\left({\mu }_{\tau }\right)}$	444.4	435.4	454.3
Coverage	0.95	0.95	0.95
Exp-mix, n = 100
$\widehat{{\mu }_{\tau }}$	225.5	225.7	225.5
$\widehat{Var\left({\mu }_{\tau }\right)}$	224.8	220.5	227.1
Coverage	0.96	0.95	0.96

summarizes the results of the simulation studies with 1000 runs. Generally, flexible parametric survival methods outperform both the Kaplan–Meier method and pseudo-observations without compromising the coverage probability of the 95% confidence intervals.

4.3. Parametric Estimation under Model Misspecification

As we have seen in the previous subsection, parametric estimation outperforms non-parametric estimation, at least when the correct model is specified. However, this is rarely possible. In this subsection, we assess the parametric estimators and associated variances under model misspecification. In a first set of simulations, we simulated survival times with the log-logistic distribution as described above (Section 4.2). When the distribution of survival times is unknown, it is common to assume an Exponential distribution. While the Kaplan - Meier method was practically unbiased, the parametric estimator overestimated ${\mu }_{\tau }$. Of the two variance estimator the $\delta$-method had the smallest estimates. This is expected as sandwich variance estimates are robust against model misspecification at the expanse of efficacy (

Table 3. Bias and accuracy of the parametric estimation with variance estimated by the $\delta$-method and M-estimation, together with comparative numbers for the Kaplan - Meier method under model misspecification.

	$\widehat{{\mathit{\mu }}_{\mathit{\tau }}}$	$\widehat{\mathit{V}\mathit{a}\mathit{r}\left({\mathit{\mu }}_{\mathit{\tau }}\right)}$	Bias	MSE
$\tau$ = 50
Parametric($\delta$-met)	45.11	0.35	−1.48	2.54
Parametric (M-est)	45.11	0.39	−1.48	2.58
Kaplan–Meier	46.63	0.91	0.04	0.91
$\tau$ = 365
Parametric ($\delta$-met)	188.24	191.42	6.04	227.96
Parametric (M-est)	188.24	213.10	6.04	249.63
Kaplan–Meier	182.54	203.06	0.35	203.18

).

See Table 3. In a second set of simulation, we simulated survival times according to a Weibull distribution with shape (k) and scale ($\lambda$) parameters. We assumed exponential censoring with rate $1/365$. We varied the scale parameter between 0.5 and 3. The Shape parameter was set so that the mean of Weibull survival times should equal 365, the mean of exponential survival time with rate $1/365$. We estimated $\lambda$ with numerical root finding, by solving the equation $\lambda \Gamma (1+k^{-1})=365$. At $k=1$, $\lambda =365$ and the Weibull distribution simplifies to an Exponential distribution with rate 1/365. At these values parametric $RMTS$ estimation assuming Exponential survival distribution returned, as expected, nearly unbiased estimates. However, as k deviated from 1, the bias increased. The sign and magnitude of bias is dependent on the direction and magnitude of deviation from $k=1$ (Figure 2).

5. Discussion

In this paper we presented the main estimators for $RMST$ and associated variances. Additionally, with the help of simulation studies, we examined their small sample properties. As expected, the results of our simulations corroborated that parametric methods of estimation are more effective than non-parametric estimation. This is in line with previous findings about the Kaplan–Meier curve [2] or semi-parametric Cox-regression [3]. Naturally, just as Meier et al. [4] concluded, establishing the proper parametric form is difficult. Since the work of Meier et al., important advancements have been made in the field. The Focused Information Criterion ($FIC$) have been proposed for comparing general non-nested parametric models with a non-parametric alternative [28]. Application of $FIC$ requires advanced statistical/technical knowledge and it is possible that survival times do not follow a known and well-established distribution. Thus, it is likely that in practical applications, parametric estimators for $RMST$ will be rare.

The numerical results and the patterns of efficacy as a function of censoring percentage are similar to the ones reported by Meier et al. [4] and by Jullum and Hjort [3]. Independently of the employed method, we could see that a small sample size and a high percentage of censoring, result in genuine departure from nominal coverage values of the approximate Wald-type confidence interval. This departure from the nominal coverage of the 95% confidence intervals is not a fallacy of the $RMST$ and associated variance estimators, more it is a sign of the inappropriateness of Wald-type intervals and statistics. Lawrence et al. [29] described this deviation from the normal distribution in small samples and proposed an approximation to the correct null distribution using the Cornish–Fisher expansion. In medium to large samples the normal approximation is expected to hold. Our simulation suggests that in samples $\ge 100$ statistical inference will be correct. This value of $\ge 100$ should not be taken as a proposed threshold. This is data dependent. Schall [30] proposed a bootstrap based framework to assess if the coverage probability of confidence intervals approximates the intended nominal coverage. Alternatively, one could either correct null distribution as Lawrence et al. [29] proposed or employ computer intensive methods. Jackknife estimate, that is behind numerous important theoretical results about $RMST$ [21,31,32] could be a natural option or bootstrapping that has a straightforward application in survival analysis [33,34]. Nevertheless, bootstrapping $RMST$ requires extra care. The parameter space for $RMST$ is $(0,\tau ]$ and $RMST$ can be close to $\tau$. Bootstrapping parameters on the boundary of the parameter space can return inconsistent results [35,36]. This is not a surprising result. While we assume normal distribution for ${\mu }_{\tau }$, its parameter space is restricted to $(0,\tau ]$. A Wald-type 95% confidence interval would require that 2.5% of the distribution would be at either side of the interval. As ${\mu }_{\tau }\to \tau$ the right tail of the assumed normal distribution will include more and more values that are larger than $\tau$. Thus the confidence interval might include values outside the parameter space. If ${\mu }_{\tau }\sim N(\widehat{{\mu }_{\tau }},\widehat{Var}\left({\mu }_{\tau }\right))$, then $P({\mu }_{\tau }\ge \tau )$ determines the values of the ${\mu }_{\tau }$ under the assumed distribution that falls outside of $(0,\tau ]$. $P({\mu }_{\tau }\ge \tau )$ is directly influenced by $\widehat{Var}\left({\mu }_{\tau }\right))$ and decreases with rate $\sqrt{n}$ as the sample size increases. In either case, further studies are needed to establish routines of statistical inference. Extra care is also needed in inferring from $RMST$ based studies with Wald-type confidence intervals or Z-scores. Applied studies, while they do present $RMST$ and associated confidence intervals, are interested in comparing different treatment regimens. This is done by calculating difference between two $RMST$ estimates $\mathcal{D}$, or the ratio between them $\mathcal{R}$. Neither $\mathcal{D}$ or $\mathcal{R}$ is affected by the boundary value problems in realistic situations. The parameter space for $\mathcal{D}$ is $(-\tau ,\tau )$ and the $\mathcal{D}$ approaches the boundaries if and only if $RMST$ for one arm is close to zero and the $RMST$ for the competing treatment should be close to $\tau$. Under similar circumstances $\mathcal{R}$ approaches to its boundary values of $(0,\infty )$.

One aspect that practitioners face when planing a clinical trial with $RMST$ as their method of choice is deciding upon $\tau$. The choice of the time window $(0,t]$ may be prespecified at the design stage of the study based on clinical considerations or the choice of time point $\tau$ could be data-dependent after the data is collected [37]. The choice of $\tau$ has direct implications on statistical power [38] and as we highlighted statistical power can be maximized by considering the intricate relationship between the choice of $\tau$ and the censoring percentage.

Although we observed differences in the magnitude of the variance between the Kaplan - Meier method, pseudo-observations and flexible parametric survival methods the practical implications are likely limited. The differences between the variance estimators might be numerically large however when compared to the magnitude of the $RMST$ estimate, these differences will have limited practical implications. Thus, we refrain from suggesting one or the other method as the preferred one. All three methods have arguments that spoke for them. The Kaplan- Meier method is probably the most used and best known. Its application is simple and straightforward and require limited programming knowledge to implement on the top of Kaplan–Meier curve estimator, if not readily available in the preferred software. The flexible parametric survival method takes the idea behind the Kaplan–Meier method one step further by assuming a parametric form and fully define a smoothed version of the survival curve. Estimation assumes definition of the survival distribution. Additionally, to increase flexibility and model fit, the survival function can be modeled as a natural cubic spline function of log time.

Standard statistical software provide readily available routines for estimation [39,40]. An approach similar in philosophy was recently proposed by Luo et al. [41] based on piecewise exponential survival and censoring times. This approach does not require smoothing of the survival curve, however the exponential assumption facilitates design and monitoring of survival trials based on $RMST$. Both estimation with the Kaplan–Meier method and flexible parametric survival method allows adjustment of the survival curves for covariates. However, interpretation for continuous variables is difficult. The third approach, pseudo-observations, is also readily available in most statistical software [42,43,44]. With pseudo-observation covariates can be regressed on restricted survival times with linear regression. While the estimates of the Cox-regression or flexible parametric survival regression are hazard rates, regression on pseudo-observation returns regression coefficients on the time scale, i.e. they directly quantify the change in restricted survival times as a function of change in the covariates.

In conclusion, we suggest researchers interested in using $RMST$ to analyse survival data to adopt the method that best fit their purpose. If ease of implementation is paramount, then the Kaplan–Meier method is preferred. To increase statistical power, flexible parametric survival models could be the proper option as they show superior efficacy compared to the other practical applicable methods. Clinical studies that aim to assess the effect of treatments/exposures adjusted for relevant covariates could take benefit from considering pseudo-observations as their tool of preference. Naturally, parametric method of estimation and inference can be considered. However researchers should be extra cautious about violation of assumptions.