Mixture Modeling of Time-to-Event Data in the Proportional Odds Model

Abstract

Subgroup analysis with survival data are most essential for detailed assessment of the risks of medical products in heterogeneous population subgroups. In this paper, we developed a semiparametric mixture modeling strategy in the proportional odds model for simultaneous subgroup identification and regression analysis of survival data that flexibly allows the covariate effects to differ among several subgroups. Neither the membership or the subgroup-specific covariate effects are known a priori. The nonparametric maximum likelihood method together with a pair of MM algorithms with monotone ascent property are proposed to carry out the estimation procedures. Then, we conducted two series of simulation studies to examine the finite sample performance of the proposed estimation procedure. An empirical analysis of German breast cancer data is further provided for illustrating the proposed methodology.

1. Introduction

In some clinical trials, a substantial proportion of patients respond favorably to a new treatment while the others may eventually relapse. Subgroup analyses aim to classify the patients into a few homogeneous groups and tailor a disease treatment specifically for each subgroup to optimize the treatment effect. In recent years, subgroup identification has received increasing attention in a wide range of fields such as clinical trials, public management, econometrics, and social science. For example, Refs. [1,2] conducted subgroup analysis in econometrics and marketing, while Refs. [3,4] implemented the subgroup analysis in epidemiology and biology, respectively.

Statistical methods for subgroup analysis have also been greatly developed recently. Among them, a finite mixture model has been recognized as an important tool and has been widely used for analyzing data from a heterogeneous population [5]. For example, there are many studies on the Gaussian mixture model for data clustering and classification [6,7,8]. Ref. [9] introduced a structured logistic-normal mixture model to identify subgroups in randomized clinical trials with differential treatment effects. Refs. [10,11] extended the mixture model-based approach to generalized linear models. Bayesian approaches for mixture regression models are studied by [12]. Moreover, nonparametric mixture models have also been under study in recent years. Ref. [13] studied a nonparametric mixture model for cure rate estimation. Ref. [14] studied a semiparametric accelerated failure time mixture model for estimation of a biological treatment effect on a latent subgroup of interest in randomized clinical trials. Ref. [15] proposed a semiparametric Logistic–Cox mixture model for subgroup analysis when the interested outcome is event time with right censoring.

Mixture models are deeply connected to the expectation–maximization (EM) algorithm. The EM algorithm is a popular approach for maximum likelihood estimation in incomplete data problems, of which finite mixtures are canonical examples because the unobserved labels of the individuals (as in unsupervised clustering) give a direct interpretation of missing data [16]. Actually, the EM algorithm is a special member of the general family of MM algorithms [17]. The MM algorithm possesses great flexibility in solving optimization problems because the basic idea of MM algorithm is to convert a difficult optimization problem into a series of simpler ones. The MM algorithm has been a powerful tool for optimization problems and enjoys its greatest vogue in computational statistics. Thus far, the MM algorithm has been widely used in many statistical optimization problems. We can find applications of MM principle in a broad range of statistical contexts, including the Bradley–Terry model [18], quantile regression [19], variable selection [20,21], the proportional odds model [22], the shared frailty model [23], distance majorization [24] and so on. The key property of MM principle is that it can decompose a high-dimensional objective function into separable low-dimensional functions by the construction of surrogate function. In this paper, we introduce the general MM principle to the semiparametric mixture of proportional odds model for simultaneous subgroup identification and regression analysis.

The rest of the paper is organized as follows. We first review the MM algorithm in Section 2. In Section 3, we present the latent proportional odds model and develop a pair of estimation procedures for the proposed model using the MM algorithm. In Section 4, we provide two parts of simulation studies to select the number of subgroups and assess the finite-sample performances of the proposed methods. We further provide an application of the German breast cancer study data to illustrate the practical utilities of the proposed methods in Section 5.

2. MM Principle

The MM algorithm is an important and powerful tool for optimization problems and enjoys its greatest vogue in computational statistics. For example, $\ell \left(\mathit{\alpha }\right|Y_{obs})$ is the objective log-likelihood function, $\mathit{\alpha }={({\alpha }_1,\cdots ,{\alpha }_q)}^T\in \mathbf{\Theta }$ are the vector of parameters to be estimated, and $\mathbf{\Theta }$ is the parameter space. The maximum likelihood estimate of $\mathit{\alpha }$ is $\widehat{\mathit{\alpha }}=\arg {\max }_{\mathit{\alpha }\in \mathbf{\Theta }}\ell \left(\mathit{\alpha }\right|Y_{obs})$. The MM principle provides a general frame for constructing iterative algorithms with monotone convergence, which involves double duty. In maximization problems, the first M stands for minorize and the second M for maximize. The minorization step first constructs a surrogate function $Q\left(\mathit{\alpha }\right|{\mathit{\alpha }}^{\left(k\right)})$ such that

$$Q\left(\mathit{\alpha }\right|{\mathit{\alpha }}^{\left(k\right)})\le \ell \left(\mathit{\alpha }\right|Y_{obs}),\forall \mathit{\alpha },{\mathit{\alpha }}^{\left(k\right)}\in \mathbf{\Theta },Q\left({\mathit{\alpha }}^{\left(k\right)}\right|{\mathit{\alpha }}^{\left(k\right)})=\ell \left({\mathit{\alpha }}^{\left(k\right)}\right|Y_{obs}),$$

(1)

where ${\mathit{\alpha }}^{\left(k\right)}$ denotes the current estimate of $\mathit{\alpha }$ in the k-th iteration. The maximization step then updates ${\mathit{\alpha }}^{\left(k\right)}$ by ${\mathit{\alpha }}^{(k+1)}$, which maximizes the surrogate function $Q(·|{\mathit{\alpha }}^{\left(k\right)})$ instead of $\ell \left(\mathit{\alpha }\right|Y_{obs})$, that is,

$${\mathit{\alpha }}^{(k+1)}=\arg \underset{\mathit{\alpha }\in \mathbf{\Theta }}{\max }Q\left(\mathit{\alpha }\right|{\mathit{\alpha }}^{\left(k\right)}).$$

Since

$$\ell \left({\mathit{\alpha }}^{(k+1)}\right|Y_{obs})\ge Q\left({\mathit{\alpha }}^{(k+1)}\right|{\mathit{\alpha }}^{\left(k\right)})\ge Q\left({\mathit{\alpha }}^{\left(k\right)}\right|{\mathit{\alpha }}^{\left(k\right)})=\ell \left({\mathit{\alpha }}^{\left(k\right)}\right|Y_{obs}),$$

the constructed MM algorithm can increase the objective function at each iteration and possess the ascent property driving the objective optimization function $\ell \left(\mathit{\alpha }\right|Y_{obs})$ uphill.

3. Proportional Odds Model with Individual-Specific Covariate Effects

Let T be time to event. The proportional odds model postulates that

$${\lambda }_i(t\mid X)=\frac{{\lambda }_0\left(t\right)\exp \left(X_i^{\top }\mathit{\beta }\right)}{1+{\Lambda }_0\left(t\right)\exp \left(X_i^{\top }\mathit{\beta }\right)},$$

where ${\lambda }_i\left(t\right)$ is the hazard function of $T_i$ given the covariates $X_i$. Let the conditional survival function of T be $S\left(t\right|X)=P(T>t\left|X\right)$. We know that $\lambda \left(t\right|X)=-\frac{d(-\log S(t\left|X\right))}{dt}$. In the proportional odds model, $\mathit{\beta }$ is the regression coefficients, quantifying the effect of the covariates X on the time to event T through the conditional hazard function. It is assumed to be the same for all subjects in the population. In practice, however, subjects may come from different subgroups, the covariate effects may differ and therefore it is more appropriate to assume the following proportional odds model with individual-specific covariate effects:

$${\lambda }_i(t\mid X)=\frac{{\lambda }_0\left(t\right)\exp \left(X_i^{\top }{\mathit{\beta }}_{\mathit{i}}\right)}{1+{\Lambda }_0\left(t\right)\exp \left(X_i^{\top }{\mathit{\beta }}_{\mathit{i}}\right)}.$$

In this model, we assume that the covariate effects ${\mathit{\beta }}_i$ for the subject i may differ. For both parsimony and better interpretation, it is reasonable to assume that ${\mathit{\beta }}_i={\mathit{\beta }}_{0,m}$ with probability ${\pi }_m,m=1,\dots M$. In other words, there are only M different subgroups for the covariate effects ${\mathit{\beta }}_i$, where ${\mathit{\beta }}_{0,m},m=1,\dots ,M$ are M different regression coefficients. It is of our interest to estimate the number of groups M, ${\mathit{\beta }}_{0,m},m=1,\dots ,M$ and ${\pi }_m,m=1,\dots ,M$. Note that ${\displaystyle \sum _{m=1}^M}{\pi }_m=1$.

3.1. Heterogeneity Regression Pursuit via MM Algorithm

The joint density function of $(T,\delta )$ can be written as

$$f(t,\delta |X)=\sum _{m=1}^M{\pi }_m{f}_m(t,\delta |X)$$

where

$$f_m(t,\delta |X)={\left\{\frac{{\lambda }_0\left(t\right)\exp \left(X^{\top }{\mathit{\beta }}_m\right)}{1+{\Lambda }_0\left(t\right)\exp \left(X^{\top }{\mathit{\beta }}_m\right)}\right\}}^{\delta }\frac{1}{1+{\Lambda }_0\left(t\right)\exp \left(X^{\top }{\mathit{\beta }}_m\right)}$$

denotes the density function of the m-th subgroup, $m=1,2,\dots ,M$, ${\mathit{\beta }}_m$ is the corresponding effect parameter of X in the m-th subgroup. Given the observed data $Y_{obs}=({\left\{t_i\right\}}_{i=1}^n,{\left\{d_i\right\}}_{i=1}^n,{\left\{X_i\right\}}_{i=1}^n)$, we have the observed log-likelihood function as

$$\begin{array}{c}\hfill \ell ({\Lambda }_0,\mathit{\beta },\mathit{\pi }|Y_{obs})=\sum _{i=1}^n\log \left\{\sum _{m=1}^M{\pi }_m{f}_m(t_i,{\delta }_i|X_i)\right\}.\end{array}$$

where ${\Lambda }_0\left(t\right)={\sum }_i^{n}I(t_i⩽t){\lambda }_0\left(t_i\right),\mathit{\beta }={({\mathit{\beta }}_1^T,\dots ,{\mathit{\beta }}_M^T)}^T,\mathit{\pi }=({\pi }_1,\dots ,{\pi }_M)$. Given the parameters in the k-th iteration and denoting

$${\upsilon }_{mi}^{\left(k\right)}=\frac{{\pi }_m^{\left(k\right)}·f_m^{\left(k\right)}(t_i,{\delta }_i|X_i)}{{\sum }_{m=1}^K{\pi }_m^{\left(k\right)}·f_m^{\left(k\right)}(t_i,{\delta }_i|X_i)},$$

then we can rewrite $\ell ({\Lambda }_0,\mathit{\beta },\mathit{\pi }|Y_{obs})$ as

$$\begin{array}{c}\hfill \ell ({\Lambda }_0,\mathit{\beta },\mathit{\pi }|Y_{obs})=\sum _{i=1}^n\log \left\{\sum _{m=1}^M{\upsilon }_{mi}^{\left(k\right)}·\frac{{\pi }_m·f_m(t_i,{\delta }_i|X_i)}{{\upsilon }_{mi}^{\left(k\right)}}\right\}.\end{array}$$

(2)

By the continuous version of Jensen’s inequality as $\phi \left({\int }_{\Omega }f\left(x\right)·g\left(x\right)\mathrm{d}x\right)\ge {\int }_{\Omega }\phi \left(f\left(x\right)\right)·g\left(x\right)\mathrm{d}x$, we can transfer the function $\phi (·)$ outside the integral to the inside of the integral, where $g\left(x\right)$ is a density function. Inspired by this feature, we construct a density function ${\upsilon }_{mi}^{\left(k\right)}$ in Equation (2) which plays the role of function $g\left(x\right)$, the rest of the part ${\pi }_m·f_m(t_i,{\delta }_i|X_i)/{\upsilon }_{mi}^{\left(k\right)}$ plays the role of function $f\left(x\right)$. By the following calculation,

$$\sum _{i=1}^n\log \left\{\sum _{m=1}^M{\upsilon }_{mi}^{\left(k\right)}·\frac{{\pi }_m·f_m(t_i,{\delta }_i|X_i)}{{\upsilon }_{mi}^{\left(k\right)}}\right\}⩾\sum _{i=1}^n\sum _{m=1}^M{\upsilon }_{mi}^{\left(k\right)}·\left\{\log {\pi }_m+\log f_m(t_i,{\delta }_i|X_i)\right\},$$

the logarithmic function on the outside is transferred to the inside of the integral, which breaks down the product terms into a summation. Hence, we construct the surrogate function for $\ell ({\Lambda }_0,\mathit{\beta },\mathit{\pi }|Y_{obs})$ as

$$\begin{array}{cc}\hfill Q({\Lambda }_0,\mathit{\beta },\mathit{\pi }|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})& =\sum _{i=1}^n\sum _{m=1}^M{\upsilon }_{im}^{\left(k\right)}·\left\{\log {\pi }_m+\log f_m(t_i,{\delta }_i|X_i)\right\},\hfill \\ & \widehat{=}Q\left(\mathit{\pi }\right|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})+Q({\Lambda }_0,\mathit{\beta }|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)}),\hfill \end{array}$$

where

$$Q\left(\mathit{\pi }\right|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})=\sum _{i=1}^n\sum _{m=1}^M{\upsilon }_{im}^{\left(k\right)}·\log {\pi }_m,$$

(3)

and

$$\begin{array}{cc}\hfill & Q({\Lambda }_0,\mathit{\beta }|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})\hfill \\ \hfill & =\sum _{i=1}^n\sum _{m=1}^M{\upsilon }_{im}^{\left(k\right)}\log f_m(t_i,{\delta }_i|X_i),\hfill \\ \hfill & =\sum _{i=1}^n{\delta }_i\log {\lambda }_0\left(t_i\right)+\sum _{i=1}^n\sum _{m=1}^M{\upsilon }_{im}^{\left(k\right)}{\delta }_i{X}_i^{\top }{\mathit{\beta }}_m-\sum _{i=1}^n\sum _{m=1}^M{\upsilon }_{im}^{\left(k\right)}({\delta }_i+1)\log \left[1+{\Lambda }_0\left(t_i\right)\exp \left(X_i^{\top }{\mathit{\beta }}_m\right)\right].\hfill \end{array}$$

(4)

The surrogate function $Q({\Lambda }_0,\mathit{\beta },\mathit{\pi }|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})$ separates the parameters $\mathit{\pi }$ and $({\Lambda }_0,\mathit{\beta })$ into (3) and (4), respectively. All the parameters ${\left\{{\pi }_m\right\}}_{m=1}^K$ in (3) are separated from each other so that updating ${\pi }_m$ is as straightforward as

$$\begin{array}{c}\hfill {\widehat{\pi }}_m=\frac{{\sum }_{i=1}^n{\upsilon }_{im}^{\left(k\right)}}{n},m=1,\dots ,M.\end{array}$$

(5)

To update $({\Lambda }_0,\mathit{\beta })$, we apply the supporting hyperplane inequality to Equation (4) to release the object x from the logarithmic function,

$$-\log \left(x\right)\ge -\log \left(x_0\right)-\frac{x-x_0}{x_0},$$

we have

$$\begin{array}{cc}\hfill -\log \left[1+{\Lambda }_0\left(t_i\right)\exp \left(X_i^{\top }{\mathit{\beta }}_m\right)\right]\ge & -\log \left(A_{im}^{\left(k\right)}\right)\hfill \\ & -\frac{1+{\Lambda }_0\left(t_i\right)\exp \left(X_i^{\top }{\mathit{\beta }}_m\right)-A_{im}^{\left(k\right)}}{A_{im}^{\left(k\right)}},\hfill \end{array}$$

where $A_{im}^{\left(k\right)}=1+{\Lambda }_0^{\left(k\right)}\left(t_i\right)\exp \left(X_i^{\top }{\mathit{\beta }}_m^{\left(k\right)}\right)$. Then, we obtain the following surrogate function for $Q({\Lambda }_0,\mathit{\beta }|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})$,

$$\begin{array}{cc}& Q_1({\Lambda }_0,\mathit{\beta }|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})\hfill \\ & =\sum _{i=1}^n{\delta }_i\log {\lambda }_0\left(t_i\right)+\sum _{i=1}^n\sum _{m=1}^M{\upsilon }_{im}^{\left(k\right)}{\delta }_i{X}_i^{\top }{\mathit{\beta }}_m-\sum _{i=1}^n\sum _{m=1}^M{\upsilon }_{im}^{\left(k\right)}({\delta }_i+1)\frac{{\Lambda }_0\left(t_i\right)\exp \left(X_i^{\top }{\mathit{\beta }}_m\right)}{A_{im}^{\left(k\right)}}.\hfill \end{array}$$

3.2. Profile MM Method

Following [25,26], we consider the profile estimation approach and first profile out ${\Lambda }_0$ in $Q_1({\Lambda }_0,\mathit{\beta }|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})$ for any given $\mathit{\beta }$. This leads to the estimate of ${\Lambda }_0$ given $\mathit{\beta }$ as

$${\widehat{\lambda }}_0\left(t_i\right)=\frac{{\delta }_i}{{\sum }_{j=1}^{n}I\left(t_j⩾t_i\right){\sum }_{m=1}^M{\upsilon }_{jm}^{\left(k\right)}\left({\delta }_j+1\right)\exp \left(X_j^{\top }{\mathit{\beta }}_m\right)/A_{jm}^{\left(k\right)}}.$$

(6)

Substituting (6) into $Q_1({\Lambda }_0,\mathit{\beta }|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})$ yields the function

$$\begin{array}{cc}& Q_2\left(\mathit{\beta }\right|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})\hfill \\ & =\sum _{i=1}^n\sum _{m=1}^M{\upsilon }_{im}^{\left(k\right)}{\delta }_i{X}_i^{\top }{\mathit{\beta }}_m-\sum _{i=1}^n{\delta }_i\log \left[\sum _{j=1}^{n}I\left(t_j⩾t_i\right)\sum _{m=1}^M{\upsilon }_{jm}^{\left(k\right)}\left({\delta }_j+1\right)\exp \left(X_j^{\top }{\mathit{\beta }}_m\right)/A_{jm}^{\left(k\right)}\right].\hfill \end{array}$$

We use the supporting hyperplane inequality again to deal with $Q_2\left(\mathit{\beta }\right|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})$, then we obtain the follwing $Q_3\left(\mathit{\beta }\right|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})$ where all ${\mathit{\beta }}_m(m=1,\cdots ,M)$ are separated from each other,

$$\begin{array}{cc}& Q_3\left(\mathit{\beta }\right|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})\hfill \\ & =\sum _{i=1}^n\sum _{m=1}^M{\upsilon }_{im}^{\left(k\right)}{\delta }_i{X}_i^{\top }{\mathit{\beta }}_m-\sum _{i=1}^n{\delta }_i\frac{{\sum }_{j=1}^{n}I\left(t_j⩾t_i\right){\sum }_{m=1}^M{\upsilon }_{jm}^{\left(k\right)}\left({\delta }_j+1\right)\exp \left(X_j^{\top }{\mathit{\beta }}_m\right)/A_{jm}^{\left(k\right)}}{B_i^{\left(k\right)}}\hfill \\ & =\sum _{m=1}^M\left\{\sum _{i=1}^n{\upsilon }_{im}^{\left(k\right)}{\delta }_i{X}_i^{\top }{\mathit{\beta }}_m-\sum _{i=1}^n{\delta }_i\frac{{\sum }_{j=1}^{n}I\left(t_j⩾t_i\right){\upsilon }_{jm}^{\left(k\right)}\left({\delta }_j+1\right)\exp \left(X_j^{\top }{\mathit{\beta }}_m\right)/A_{jm}^{\left(k\right)}}{B_i^{\left(k\right)}}\right\}\hfill \\ & =\sum _{m=1}^M{Q}_3\left({\mathit{\beta }}_m\right|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)}),\hfill \end{array}$$

where $B_i^{\left(k\right)}={\sum }_{j=1}^{n}I\left(t_j⩾t_i\right){\sum }_{m=1}^M{\upsilon }_{jm}^{\left(k\right)}\left({\delta }_j+1\right)\exp \left(X_j^{\top }{\mathit{\beta }}_m^{\left(k\right)}\right)/A_{jm}^{\left(k\right)}$. Finally, the estimate of each ${\mathit{\beta }}_m$ can be obtained by one step Newton iteration.

3.3. Non-Profile MM Method

For the above profile MM method, the estimate of ${\Lambda }_0$ is highly related to the estimate of $\mathit{\beta }$ because we treat nonparametric component ${\Lambda }_0$ as a function of $\mathit{\beta }$ in the profile step. Inspired by the parameter-separable property of the MM principle, we further separate the nonparametric part ${\Lambda }_0$ with the $\mathit{\beta }$ according to the decomposition rules. That is, we use the following inequality of arithmetic and geometric means to the function $Q_1({\Lambda }_0,\mathit{\beta }|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})$ as

$$-\prod _{i=1}^n{x}_i^{a_i}\ge -\sum _{i=1}^n\frac{a_i}{{\parallel \mathrm{a}\parallel }_1}{x}_i^{{\parallel \mathbf{a}\parallel }_1}.$$

Here, we let $x_1={\Lambda }_0\left(t_i\right)/{\Lambda }_0^{\left(k\right)}\left(t_i\right)$ and $x_2=\exp \left(X_i^{\top }{\mathit{\beta }}_m\right)/\exp \left(X_i^{\top }{\mathit{\beta }}_m^{\left(k\right)}\right)$, then we have

$$-\frac{{\Lambda }_0\left(t_i\right)\exp \left(X_i^{\top }{\mathit{\beta }}_m\right)}{{\Lambda }_0^{\left(k\right)}\left(t_i\right)\exp \left(X_i^{\top }{\mathit{\beta }}_m^{\left(k\right)}\right)}\ge -\frac{{\Lambda }_0^2\left(t_i\right)}{2{\Lambda }_0^{2\left(k\right)}\left(t_i\right)}-\frac{\exp \left(2X_i^{\top }{\mathit{\beta }}_m^{\left(k\right)}\right)}{2\exp \left(2X_i^{\top }{\mathit{\beta }}_m^{\left(k\right)}\right)}.$$

That is,

$$-{\Lambda }_0\left(t_i\right)\exp \left(X_i^{\top }{\mathit{\beta }}_m\right)\ge -\frac{\exp \left(X_i^{\top }{\mathit{\beta }}_m^{\left(k\right)}\right)}{2{\Lambda }_0^{\left(k\right)}\left(t_i\right)}{\Lambda }_0^2\left(t_i\right)-\frac{{\Lambda }_0^{\left(k\right)}\left(t_i\right)}{2\exp \left(X_i^{\top }{\mathit{\beta }}_m^{\left(k\right)}\right)}\exp \left(2X_i^{\top }{\mathit{\beta }}_m\right).$$

Substituting the above inequality back to $Q_1({\Lambda }_0,\mathit{\beta }|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})$, we may obtain

$$\begin{array}{cc}& Q_4({\Lambda }_0,\mathit{\beta }|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})\hfill \\ & =\sum _{i=1}^n{\delta }_i\log {\lambda }_0\left(t_i\right)+\sum _{i=1}^n\sum _{m=1}^M{\upsilon }_{im}^{\left(k\right)}{\delta }_i{X}_i^{\top }{\mathit{\beta }}_m\hfill \\ & -\sum _{i=1}^n\sum _{m=1}^M{\upsilon }_{im}^{\left(k\right)}({\delta }_i+1)\left[\frac{\exp \left(X_i^{\top }{\mathit{\beta }}_m^{\left(k\right)}\right)}{2{\Lambda }_0^{\left(k\right)}\left(t_i\right)}{\Lambda }_0^2\left(t_i\right)+\frac{{\Lambda }_0^{\left(k\right)}\left(t_i\right)}{2\exp \left(X_i^{\top }{\mathit{\beta }}_m^{\left(k\right)}\right)}\exp \left(2X_i^{\top }{\mathit{\beta }}_m\right)\right]/A_{im}^{\left(k\right)}\hfill \\ & \widehat{=}{Q}_4\left({\Lambda }_0\right|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})+Q_4\left(\mathit{\beta }\right|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)}),\hfill \end{array}$$

where

$$Q_4\left({\Lambda }_0\right|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})=\sum _{i=1}^n{\delta }_i\log {\lambda }_0\left(t_i\right)-\sum _{i=1}^n\sum _{m=1}^M{\upsilon }_{im}^{\left(k\right)}({\delta }_i+1)\frac{\exp \left(X_i^{\top }{\mathit{\beta }}_m^{\left(k\right)}\right)}{2{\Lambda }_0^{\left(k\right)}\left(t_i\right)}{\Lambda }_0^2\left(t_i\right)/A_{im}^{\left(k\right)}$$

and

$$\begin{array}{cc}& Q_4\left(\mathit{\beta }\right|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})\hfill \\ & =\sum _{i=1}^n\sum _{m=1}^M{\upsilon }_{im}^{\left(k\right)}{\delta }_i{X}_i^{\top }{\mathit{\beta }}_m-\sum _{i=1}^n\sum _{m=1}^M\frac{{\upsilon }_{im}^{\left(k\right)}({\delta }_i+1){\Lambda }_0^{\left(k\right)}\left(t_i\right)}{2\exp \left(X_i^{\top }{\mathit{\beta }}_m^{\left(k\right)}\right)}\exp \left(2X_i^{\top }{\mathit{\beta }}_m\right)/A_{im}^{\left(k\right)}.\hfill \end{array}$$

It is observed that the parameters ${\Lambda }_0$ and ${\mathit{\beta }}_m$ are completely separated, then the corresponding parameter estimators can be obtained by differentiating them separately. Letting $\partial Q_4\left({\Lambda }_0\right|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})/\partial {\Lambda }_0=0$, we obtain the estimate of ${\Lambda }_0$ by

$${\widehat{\lambda }}_0\left(t_i\right)=\frac{{\delta }_i}{{\sum }_{j=1}^{n}I\left(t_j⩾t_i\right){\sum }_{m=1}^M{\upsilon }_{jm}^{\left(k\right)}\left({\delta }_j+1\right)\exp \left(X_j^{\top }{\mathit{\beta }}_m\right)/A_{jm}^{\left(k\right)}}.$$

To update ${\mathit{\beta }}_m$, we calculate the first and second derivatives of $Q_4\left(\mathit{\beta }\right|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})$ as follows:

$$\begin{array}{cc}& Q_{4{\mathit{\beta }}_m}^{\prime }\left(\mathit{\beta }\right|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})\hfill \\ & =\sum _{i=1}^n\sum _{m=1}^M{\upsilon }_{im}^{\left(k\right)}{\delta }_i{X}_i^{\top }-\sum _{i=1}^n\sum _{m=1}^M\frac{{\upsilon }_{im}^{\left(k\right)}({\delta }_i+1){\Lambda }_0^{\left(k\right)}\left(t_i\right)}{\exp \left(X_i^{\top }{\mathit{\beta }}_m^{\left(k\right)}\right)}\exp \left(2X_i^{\top }{\mathit{\beta }}_m\right)X_i^{\top }/A_{im}^{\left(k\right)}\hfill \end{array}$$

and

$$Q_{4{\mathit{\beta }}_m}^{″}\left(\mathit{\beta }\right|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})=-\sum _{i=1}^n\sum _{m=1}^M\frac{{\upsilon }_{im}^{\left(k\right)}({\delta }_i+1){\Lambda }_0^{\left(k\right)}\left(t_i\right)}{\exp \left(X_i^{\top }{\mathit{\beta }}_m^{\left(k\right)}\right)}\exp \left(2X_i^{\top }{\mathit{\beta }}_m\right)X_i^{\top }{X}_i/A_{im}^{\left(k\right)}.$$

Then, ${\mathit{\beta }}_m$ can be estimated by

$${\mathit{\beta }}_m^{(k+1)}={\mathit{\beta }}_m^{\left(k\right)}-Q_{4{\mathit{\beta }}_m}^{″}{\left({\mathit{\beta }}_m^{\left(k\right)}\right|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)})}^{-1}{Q}_{4{\mathit{\beta }}_m}^{\prime }\left({\mathit{\beta }}_m^{\left(k\right)}\right|{\Lambda }_0^{\left(k\right)},{\mathit{\beta }}^{\left(k\right)},{\mathit{\pi }}^{\left(k\right)}).$$

4. Simulation Study

According to the estimation equation derived in previous sections, we simulate the data to analyze the estimation result at finite sample size. As the number of groups M in the mixture of proportional odds model is unknown and will be estimated by a data-driven manner. Here, we use the modified Bayesian information criterion (BIC [19]) to choose the number of components M by minimizing the criterion function:

$$BI{C}_M=-2\ast \ell ({\widehat{\Lambda }}_0,\widehat{\mathit{\beta }},\widehat{\mathit{\pi }}|Y_{obs})+M\ast q\ast \log \left(n\right).$$

where n is the sample size and q is the dimension of ${\mathit{\beta }}_m$. Note that this is strictly related to the marginal likelihood computation as can be seen in [27,28,29].

Scenario 1. We generate clustered right-censored data from a mixture of proportional odds model with two subgroups and two covariates

$${\lambda }_i(t\mid X)=\frac{{\lambda }_0\left(t\right)\exp \left(X_i^{\top }{\mathit{\beta }}_{\mathit{i}}\right)}{1+{\Lambda }_0\left(t\right)\exp \left(X_i^{\top }{\mathit{\beta }}_{\mathit{i}}\right)},$$

where the two covariates $X_{i1}$ and $X_{i2}$ are independent and follow the standard normal distribution, ${\Lambda }_0\left(t\right)={(t/2)}^2$, We randomly assign the sample size n into two subgroups with equal probabilities, i.e., we let $P(i\in G_1)=P(i\in G_2)=0.5$ so that ${\mathit{\beta }}_i={(3,-1)}^{\top }$ for $i\in G_1$, ${\mathit{\beta }}_i={(-3,2)}^{\top }$ for $i\in G_2$. We choose different sample sizes $n=150,250,500$ and set the censoring proportion at 30% to assess their performance of the proposed estimation procedures.

Table 1 reports the mean and median of the estimator $\widehat{M}$ and the proportion of $\widehat{M}$ equal to the true number of subgroups based on 500 replications. Table 2 reports the empirical bias, mean square error (MSE), and standard error (s.d.) of the estimators $\widehat{\pi }$, ${\mathit{\beta }}_1$, and ${\mathit{\beta }}_2$ based on 500 replications. We found that the mean of $\widehat{M}$ gradually approaches the true number of subgroups 2, and the median of $\widehat{M}$ remains at 2, and the proportion of correctly identifying the true number of subgroups is close to 1 with the increase of sample size. Moreover, our methods can estimate the parameters well with small empirical bias, small MSE, and small standard error, even at small sample sizes.

Table 1. The mean, median, standard error (s.d.), and the proportion (Pro) of $\widehat{M}$ in Scenario 1.

Method	n	Mean	Median	Pro
Profile MM	150	2	2	1
	250	2.03	2	0.97
	500	2	2	1
Non-profile MM	150	2.03	2	0.97
	250	2.03	2	0.97
	500	2.005	2	0.995

Table 2. Parameter estimation results in Scenario 1.

n	Parameter	True	Profile MM			Non-Profile MM
			BIAS	MSE	s.d.	BIAS	MSE	s.d.
150	${\pi }_1$	0.5	$-0.0014$	0.0029	0.0543	$-0.0035$	0.0032	0.0565
	${\beta }_{11}$	3	0.0162	0.1731	0.4162	0.0235	0.1812	0.4255
	${\beta }_{12}$	$-1$	$-0.0216$	0.0977	0.3122	0.0047	0.0925	0.3045
	${\beta }_{21}$	$-3$	$-0.0085$	0.1785	0.4228	$-0.0233$	0.1913	0.4372
	${\beta }_{22}$	2	$-0.0011$	0.1234	0.3516	0.0332	0.1267	0.23548
250	${\pi }_1$	0.5	0.0013	0.0019	0.0441	0.0018	0.0017	0.0417
	${\beta }_{11}$	3	$-0.0106$	0.0911	0.3019	$-0.0112$	0.1038	0.3222
	${\beta }_{12}$	$-1$	$-0.0119$	0.0551	0.2347	$-0.0134$	0.0487	0.2206
	${\beta }_{21}$	$-3$	$-0.0100$	0.1068	0.3270	$-0.0067$	0.1030	0.3212
	${\beta }_{22}$	2	0.0032	0.0744	0.2730	$-0.0059$	0.0724	0.2693
500	${\pi }_1$	0.5	0.0002	0.0008	0.0287	$-0.0014$	0.0008	0.0277
	${\beta }_{11}$	3	0.0089	0.0431	0.2076	0.0085	0.0462	0.2150
	${\beta }_{12}$	$-1$	$-0.0082$	0.0240	0.1550	0.0043	0.0220	0.1483
	${\beta }_{21}$	$-3$	$-0.0096$	0.0485	0.2202	$-0.0158$	0.0434	0.2079
	${\beta }_{22}$	2	0.0076	0.0318	0.1784	$-0.0053$	0.0349	0.1870

Scenario 2. We generate right-censored data from a proportional odds model with three covariates

$${\lambda }_i(t\mid X)=\frac{{\lambda }_0\left(t\right)\exp \left(X_i^{\top }{\mathit{\beta }}_{\mathit{i}}\right)}{1+{\Lambda }_0\left(t\right)\exp \left(X_i^{\top }{\mathit{\beta }}_{\mathit{i}}\right)},$$

where the three covariates $X_{i1},X_{i2}$ and $X_{i3}$ are independent and follow the standard normal distribution. We set $\mathit{\beta }={(1,-3,2)}^{\top }$ and ${\Lambda }_0\left(t\right)={(t/2)}^2$ for all subjects. Note that the model corresponds to the latent proportional odds model with the true number of subgroups M being 1. We set the censoring proportion at 30% and choose different sample sizes $n=250,500$ to assess their performance of the proposed estimation procedures.

Table 3 reports the mean and median of the estimator $\widehat{M}$ and the proportion of $\widehat{M}$ equal to the true number of subgroups based on 200 replications. Table 4 reports the empirical bias, mean square error (MSE), and standard error (s.d.) of the estimators $\mathit{\beta }$ based on 500 replications. Based on the profile MM method, we observed that the median of $\widehat{M}$ is equal to the true number 1, the mean also gets closer to 1, and the empirical percentage of $\widehat{M}$ is close to 1 as the sample size increases. Based on the non-profile MM method, we found that the mean and median of $\widehat{M}$ are both the true number 1, and the proportion of $\widehat{M}$ is 1 when the sample sizes are 250 and 500. Furthermore, our methods show excellent performance in parameter estimation. We obtain great estimates of $\mathit{\beta }$ under different sample sizes.

Table 3. The mean, median, and the proportion (Pro) of $\widehat{M}$ in Scenario 2.

Method	n	Mean	Median	Pro
Profile MM	250	1.005	1	0.995
	500	1	1	1
Non-profile MM	250	1	1	1
	500	1	1	1

Table 4. Parameter estimation results in Scenario 2.

n	Parameter	True	Profile MM			Non-Profile MM
			BIAS	MSE	s.d.	BIAS	MSE	s.d.
250	${\beta }_1$	1	0.0021	0.0185	0.1361	$-0.0049$	0.0206	0.1436
	${\beta }_2$	$-3$	0.0194	0.0517	0.2268	0.0078	0.0449	0.2121
	${\beta }_3$	2	$-0.0190$	0.0319	0.1779	0.0060	0.0309	0.1760
500	${\beta }_1$	1	$-0.0012$	0.0093	0.0966	0.0004	0.0097	0.0986
	${\beta }_2$	$-3$	$-0.0014$	0.02439	0.1563	0.0001	0.0247	0.1574
	${\beta }_3$	2	0.0114	0.0167	0.1288	$-0.0013$	0.0149	0.1221

Scenario 3. We generate clustered right-censored data from a mixture of proportional odds model with two subgroups and two correlated covariates

$${\lambda }_i(t\mid X)=\frac{{\lambda }_0\left(t\right)\exp \left(X_i^{\top }{\mathit{\beta }}_{\mathit{i}}\right)}{1+{\Lambda }_0\left(t\right)\exp \left(X_i^{\top }{\mathit{\beta }}_{\mathit{i}}\right)},$$

where the two covariates are generated from a multivariate normal distribution with mean zero and a first-order autoregressive structure ${\rho }^{|r-s|}$ for $r,s=1,2.$ Set ${\Lambda }_0\left(t\right)={(t/2)}^2$, sample size $n=200$. Then, we randomly assign the sample size n into two subgroups with equal probabilities, i.e., we let $P(i\in G_1)=P(i\in G_2)=0.5$ so that ${\mathit{\beta }}_i={(3,-1)}^{\top }$ for $i\in G_1$, ${\mathit{\beta }}_i={(-3,2)}^{\top }$ for $i\in G_2$. We choose different values of $\rho$ with $\rho =0.2,0.8$ and set the censoring proportion at 30% to assess their performance of the proposed estimation procedures.

Table 5 reports the mean and median of the estimator $\widehat{M}$ and the proportion of $\widehat{M}$ equal to the true number of subgroups based on 500 replications. Table 6 reports the empirical bias, mean square error (MSE), and standard error (s.d.) of the estimators $\widehat{\pi }$, ${\mathit{\beta }}_1$, and ${\mathit{\beta }}_2$ based on 500 replications. In Table 5, the results of the profile MM method and non-profile MM method are basically consistent, the proportions of $\widehat{M}$ are very close to 1 and the smaller the value of $\rho$, the larger the value of Pro. it shows that our proposed methods can accurately identify the number of subgroups. In Table 6, the estimation results at a smaller value of $\rho$ perform better and more stably than the results at a larger value of $\rho$ for both the profile MM method and the non-profile MM method.

Table 5. The mean, median, and the proportion (Pro) of $\widehat{M}$ in Scenario 3.

Method	$\mathit{\rho }$	Mean	Median	Pro
Profile MM	0.2	2.005	2	0.995
	0.8	2.015	2	0.985
Non-profile MM	0.2	2.005	2	0.995
	0.8	2.015	2	0.985

Table 6. Parameter estimation results in Scenario 3.

$\mathit{\rho }$	Parameter	Profile MM			Non-Profile MM
		BIAS	MSE	s.d.	BIAS	MSE	s.d.
0.2	${\pi }_1$	0.0003	0.0023	0.0487	$-0.0036$	0.0023	0.0484
	${\beta }_{11}$	0.0285	0.1228	0.3502	$-0.0146$	0.1356	0.3689
	${\beta }_{12}$	0.0161	0.0625	0.2502	0.0095	0.0822	0.2873
	${\beta }_{21}$	$-0.0356$	0.1194	0.3446	$-0.0021$	0.1351	0.3684
	${\beta }_{22}$	$-0.0149$	0.0866	0.2945	$-0.0059$	0.0918	0.3037
0.8	${\pi }_1$	0.0023	0.0043	0.0661	$-0.0022$	0.0039	0.0630
	${\beta }_{11}$	0.0251	0.2466	0.4972	$-0.0131$	0.2413	0.4923
	${\beta }_{12}$	$-0.0155$	0.1753	0.4195	$-0.0005$	0.1648	0.4070
	${\beta }_{21}$	$-0.0990$	0.3442	0.5797	$-0.0209$	0.2590	0.5098
	${\beta }_{22}$	0.0965	0.2601	0.5020	0.0108	0.2128	0.4624

5. Real Data Analysis

Now, we apply the proposed method to analyze the German Breast Cancer Study data which can be available from R package “pec”. The data contain the observations of 686 women where the censoring rate is 56.41%. In order to analyze whether there is heterogeneity in the data, we consider “tgrade(I vs. III, II vs. III )” and “pnodes” as explanatory variables of interest, where “tgrade” indicates tumor grade which is an ordered factor at levels I vs. III or II vs. III, “pnodes” indicates the number of positive lymph nodes. Then, we use the BIC criterion function to determine the number of subgroups M. In Table 7, we report the maximum log-likelihood values (LL), the BIC values (BIC), and the estimated parameters under the number of subgroups $M=1,2,3$. Based on the results in Table 7, we found that the optimal M is 1 by comparing the BIC values. The estimated regression coefficients are detailed in Table 7.

Table 7. Estimation results for breast cancer data.

Method	M	LL	BIC	Estimated Parameters
Profile MM	1	$-2049.165$	4117.923	$\widehat{\mathit{\beta }}=(-1.3489,-0.3919,-0.0937)$
	2	$-2046.936$	4133.057	$\widehat{{\pi }_1}=0.7012$, ${\widehat{\mathit{\beta }}}_1=(-1.2093,-0.0392,0.0630)$
				$\widehat{{\pi }_2}=0.2988$, ${\widehat{\mathit{\beta }}}_2=(-1.6508,-1.2284,0.2415)$
	3	$-2044.792$	4148.361	$\widehat{{\pi }_1}=0.2680$, ${\widehat{\mathit{\beta }}}_1=(-1.6517,-1.4040,0.2596)$
				$\widehat{{\pi }_2}=0.0757$, ${\widehat{\mathit{\beta }}}_2=(-2.1672,0.5394,0.5748)$
				$\widehat{{\pi }_3}=0.6563$, ${\widehat{\mathit{\beta }}}_3=(-1.1245,-0.0477,0.0658)$
Non-profile MM	1	$-2049.165$	4117.923	$\widehat{\mathit{\beta }}=(-1.3489,-0.3918,-0.0937)$
	2	$-2046.936$	4133.057	$\widehat{{\pi }_1}=0.7012$, ${\widehat{\mathit{\beta }}}_1=(-1.2093,-0.0393,0.0630)$
				$\widehat{{\pi }_2}=0.2988$, ${\widehat{\mathit{\beta }}}_2=(-1.6508,-1.2282,0.2415)$
	3	$-2044.792$	4148.361	$\widehat{{\pi }_1}=0.2680$, ${\widehat{\mathit{\beta }}}_1=(-1.6516,-1.4038,0.2596)$
				$\widehat{{\pi }_2}=0.0757$, ${\widehat{\mathit{\beta }}}_2=(-2.1672,0.5392,0.5748)$
				$\widehat{{\pi }_3}=0.6563$, ${\widehat{\mathit{\beta }}}_3=(-1.1245,-0.0477,0.0658)$

6. Conclusions

In this work, we introduce the MM algorithm into a semiparametric mixture modeling strategy in the proportional odds model for subgroup analysis of survival data that flexibly allows the covariate effects to differ among several subgroups. Both proposed MM methods to the semiparametric mixture of proportional odds model are able to conduct simultaneous subgroup identification and regression analysis, which provides a general frame for constructing iterative algorithms with monotone convergence. The main advantage of our MM algorithm is that it can separate the nonparametric baseline hazard rate with other regression parameters and can help to avoid matrix inversion in high-dimensional regression analysis, which makes the estimation process more efficient. Furthermore, our algorithm can mesh well with the existing quasi-Newton acceleration and other simple off-the-shelf accelerators to further boost the estimation process. Such estimation procedures derived for the semiparametric mixture proportional odds model can be easily extended to other semiparametric or nonparametric mixture models. Although our proposed MM algorithms are developed for the mixture of proportional odds models, a parallel approach can essentially be developed for the more general mixture of transformation models. We will investigate this in our future work.