# Weighted Trajectory Analysis and Application to Clinical Outcome Assessment

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

**Binary Condition**The event must be binary in nature or coded into binary form (0 for non-occurrence, 1 for occurrence). It is not possible to capture grades or stages of severity. For example, death is naturally binary (0 for alive, 1 for dead), but an outcome variable such as toxicity (measured in grades from zero to four) must be coded into binary form by setting a threshold for event occurrence, such as arbitrarily defining an event as any toxicity exceeding grade two;**Descent Condition**Event occurrence always produces a drop in the KM curve (a consequence of plotting probability). It is not possible to track the trajectory of conditions that can both improve and worsen over time. For example, patients experiencing rising toxicity due to chemotherapy require additional interventions to tolerate therapy. The interventions may initially improve symptoms and reduce toxicity grade but fail to sustain benefits in subsequent treatment cycles. For a KM estimate following the above example, this complex trajectory would be simplified to an event occurrence the first time toxicity increases beyond grade two;**Finality Condition**Once a patient experiences the event of interest, they are omitted from any subsequent analysis.

## 2. Methodology and Theory

#### 2.1. Kaplan–Meier Estimator

#### 2.2. Weighted Trajectory Analysis

- Assesses outcomes defined by various ordinal grades (or stages) of clinical severity;
- Permits continued analysis of participants following changes in the variable of interest;
- Demonstrates the ability of an intervention to both prevent the exacerbation of outcomes and improve recovery, as well as the time course of these effects.

#### 2.3. Mathematical Overview of Weighted Trajectory Analysis

#### 2.4. The Logrank Test

- Let ${t}_{1}<{t}_{2}<\dots <{t}_{K}$ be K distinct failure times observed in the data;
- ${n}_{j}^{A}$ is the number of patients in group A at risk at ${t}_{j}$, where $j=1,2,\dots ,\mathit{K}$;
- ${n}_{j}^{B}$ is the number of patients in group B at risk at ${t}_{j}$, where $j=1,2,\dots ,\mathit{K}$;
- ${n}_{j}={n}_{j}^{A}+{n}_{j}^{B}$ is the total number of patients at risk at ${t}_{j}$, where $j=1,2,\dots ,\mathit{K}$;
- ${d}_{j}^{A}$ is the number of patients who experienced the (binary) event in group A at ${t}_{j}$;
- ${d}_{j}^{B}$ is the number of patients who experienced the (binary) event in group B at ${t}_{j}$;
- ${d}_{j}={d}_{j}^{A}+{d}_{j}^{B}$ is the total number of patients who experienced the (binary) event at ${t}_{j}$;
- ${S}^{A}\left(t\right)$ and ${S}^{B}\left(t\right)$ are the survival functions for group A and B, respectively.

Observed to fail at ${t}_{j}$ | At risk at ${t}_{j}$ | ||

Group A | ${d}_{j}^{A}$ | ${n}_{j}^{A}-{d}_{j}^{A}$ | ${n}_{j}^{A}$ |

Group B | ${d}_{j}^{B}$ | ${n}_{j}^{B}-{d}_{j}^{B}$ | ${n}_{j}^{B}$ |

${d}_{j}$ | ${n}_{j}-{d}_{j}$ | ${n}_{j}$ |

#### 2.5. The Weighted Logrank Test—Analytical Method

- Let L be the total number of possible values taken by the change variable ${d}_{i,j}^{A}$. When a severity score takes values from 0 to 4, $L=9$;
- Let W be the ordered non-decreasing list of the L possible change values. When a severity score takes values from 0 to 4, $W=(-4,-3,-2,-1,0,1,2,3,4)$;
- Let ${w}_{l}$ be the lth element of $\mathit{W}$;
- Let ${d}_{j}^{A,l}$ be the number of subjects in group A at ${t}_{j}$ whose change values equal ${w}_{l}$:$${d}_{j}^{A,l}=\sum _{i=1}^{{n}_{j}^{A}}{d}_{i,j}^{A}I({d}_{i,j}^{A}={w}_{l})$$
- Let ${d}_{j}^{B,l}$ be the number of subjects in group B at ${t}_{j}$ whose change values equal ${w}_{l}$;
- ${d}_{j}^{\left(l\right)}={d}_{j}^{A,l}+{d}_{j}^{B,l}$ is the total number of patients whose change values equal ${w}_{l}$ at ${t}_{j}$.

Observed values of ${d}_{i,j}$ (${w}_{l}$) | −4 | −3 | −2 | −1 | 0 | 1 | 2 | 3 | 4 | At risk at ${t}_{j}$ | |

Group A | ${d}_{j}^{A,1}$ | ${d}_{j}^{A,2}$ | ${d}_{j}^{A,3}$ | ${d}_{j}^{A,4}$ | ${d}_{j}^{A,5}$ | ${d}_{j}^{A,6}$ | ${d}_{j}^{A,7}$ | ${d}_{j}^{A,8}$ | ${d}_{j}^{A,9}$ | ${n}_{j}^{A}-{\sum}_{l=1}^{L}{d}_{j}^{A,l}$ | ${n}_{j}^{A}$ |

Group B | ${d}_{j}^{B,1}$ | ${d}_{j}^{B,2}$ | ${d}_{j}^{B,3}$ | ${d}_{j}^{B,4}$ | ${d}_{j}^{B,5}$ | ${d}_{j}^{B,6}$ | ${d}_{j}^{B,7}$ | ${d}_{j}^{B,8}$ | ${d}_{j}^{B,9}$ | ${n}_{j}^{B}-{\sum}_{l=1}^{L}{d}_{j}^{B,l}$ | ${n}_{j}^{B}$ |

${d}_{j}^{\left(1\right)}$ | ${d}_{j}^{\left(2\right)}$ | ${d}_{j}^{\left(3\right)}$ | ${d}_{j}^{\left(4\right)}$ | ${d}_{j}^{\left(5\right)}$ | ${d}_{j}^{\left(6\right)}$ | ${d}_{j}^{\left(7\right)}$ | ${d}_{j}^{\left(8\right)}$ | ${d}_{j}^{\left(9\right)}$ | ${n}_{j}-{\sum}_{l=1}^{L}{d}_{j}^{\left(l\right)}$ | ${n}_{j}$ |

#### 2.6. The Weighted Logrank Test—Computational Method

- Determine transition probabilities using msm to load into n-fold simulations blind to treatment assignment;
- Generate a distribution of the null hypothesis using the test statistic (Equation (23));
- Calculate a test statistic from the clinical data and then determine a p-value by comparison to the distribution of the null hypothesis.

#### 2.7. GEE Longitudinal Analysis

## 3. Simulation Study One—Toxicity

- Treatment group: randomly assigned as zero or one with the constraint of having an equal number of patients allocated to each group;
- Duration: the number of days a patient remains within the trial was programmed as a random value within a uniform distribution of 0 to 50 days;
- Toxicity grade: computed for each patient on a daily basis for the extent of their assigned duration. To model the trajectory of toxicity grade over time, we made the following simplifying assumptions:
- (a)
- On any given day, patients can rise or fall by a single toxicity grade;
- (b)
- Transitions in toxicity grade are random, but a larger hazard ratio suggests a greater chance of exacerbation and lower chance of recovery;
- (c)
- A patient is censored once their pre-assigned duration within the trial has elapsed or they reach maximum toxicity, in this case representing death, whichever occurs first.

#### 3.1. Kaplan–Meier Estimator: Toxicity Trial

#### 3.2. Weighted Trajectory Analysis: Simulated Trial

#### 3.3. Thousandfold Power Comparison—KM Estimation vs. WTA

#### 3.4. Thousandfold Power Comparison—KM Estimation, WTA (Analytic and Computational), GEE

## 4. Simulation Study Two—Schizophrenia

#### Thousandfold Power Comparison—WTA vs. GEE

## 5. Illustrative Real-World Example

#### 5.1. Immune Checkpoint Inhibitor Therapy for Melanoma

#### 5.1.1. Kaplan–Meier Estimator: Anti-PD-1 vs. Combination Therapy

#### 5.1.2. Weighted Trajectory Analysis: Anti-PD-1 vs. Combination Therapy

#### 5.2. Rose/Trio-012 Trial

Treatment Outcome | Definition |
---|---|

Complete response (CR) | Disappearance of all target lesions. Any pathological lymph nodes (whether target or non-target) must show reduction in short axis to <10 mm |

Partial response (PR) | At least a 30% decrease in the sum of diameters of target lesions, taking as reference the baseline sum diameters |

Progressive disease (PD) | At least a 20% increase in the sum of diameters of target lesions, taking as reference the smallest sum in the study (this includes the baseline sum that is the smallest in the study). In addition to the relative increase of 20%, the sum must also demonstrate an absolute increase of at least 5 mm (note: the appearance of one or more new lesions is also considered progression) |

Stable disease (SD) | Neither sufficient shrinkage to qualify as PR nor sufficient increase to qualify as PD, taking as reference the smallest sum of diameters in the study |

#### 5.2.1. Kaplan–Meier: Ramucirumab vs. Placebo + Docetaxel

#### 5.2.2. RECIST Endpoints: Ramucirumab vs. Placebo + Docetaxel

#### 5.2.3. Weighted Trajectory Analysis: Ramucirumab vs. Placebo in Addition to Docetaxel

## 6. Discussion

## Author Contributions

## Funding

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Kaplan, E.L.; Meier, P. Nonparametric Estimation from Incomplete Observations. J. Am. Stat. Assoc.
**1958**, 5, 457–481. [Google Scholar] [CrossRef] - Peto, R.; Pike, M.; Armitage, P.; Breslow, N.E.; Cox, D.R.; Howard, S.V.; Mantel, N.; McPherson, K.; Peto, J.; Smith, P.G. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design. Br. J. Cancer
**1976**, 34, 585–612. [Google Scholar] [CrossRef] [PubMed] - Peto, R.; Pike, M.; Armitage, P.; Breslow, N.E.; Cox, D.R.; Howard, S.V.; Mantel, N.; McPherson, K.; Peto, J.; Smith, P.G. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. II. Analysis and examples. Br. J. Cancer
**1977**, 35, 1–39. [Google Scholar] [CrossRef] [PubMed] - Oken, M.M.; Creech, R.H.; Tormey, D.C.; Horton, J.; Davis, T.E.; McFadden, E.T.; Carbone, P.P. Toxicity and response criteria of the Eastern Cooperative Oncology Group. Am. J. Clin. Oncol.
**1982**, 5, 649–655. [Google Scholar] [CrossRef] [PubMed] - American Heart Association. Classes of Heart Failure. Published 2 June 2022. Available online: https://www.heart.org/en/health-topics/heart-failure/what-is-heart-failure/classes-of-heart-failure (accessed on 29 September 2022).
- U.S. Department of Health and Human Services. Common Terminology Criteria for Adverse Events (CTCAE) Version 5.0. Published 27 November 2017. Available online: https://ctep.cancer.gov/protocoldevelopment/electronic_applications/docs/ctcae_v5_quick_reference_5x7.pdf (accessed on 23 March 2020).
- Liang, K.; Zeger, S.L. Longitudinal data analysis using generalized linear models. Biometrika
**1986**, 73, 13–22. [Google Scholar] [CrossRef] - Python Software Foundation. Python Language Reference, Version 3.7. Available online: http://www.python.org (accessed on 16 March 2020).
- Davidson-Pilon, C. Lifelines: Survival analysis in Python. J. Open Source Softw.
**2019**, 4, 1317. [Google Scholar] [CrossRef] - IBM Corp. IBM SPSS Statistics for Windows; Version 26.0; IBM Corp.: Armonk, NY, USA, 2017. [Google Scholar]
- Wang, D.Y.; Salem, J.E.; Cohen, J.V.; Chandra, S.; Menzer, C.; Ye, F.; Zhao, S.; Das, S.; Beckermann, K.E.; Ha, L.; et al. Fatal toxic effects associated with immune checkpoint inhibitors: A systematic review and meta-analysis. JAMA Oncol.
**2018**, 4, 1721–1728. [Google Scholar] [CrossRef] [PubMed] - Larkin, J.; Chiarion-Sileni, V.; Gonzalez, R.; Grob, J.J.; Cowey, C.L.; Lao, C.D.; Schadendorf, D.; Dummer, R.; Smylie, M.; Rutkowski, P.; et al. Combined nivolumab and ipilimumab or monotherapy in untreated melanoma. N. Engl. J. Med.
**2015**, 373, 23–34. [Google Scholar] [CrossRef] [PubMed] - Larkin, J.; Chiarion-Sileni, V.; Gonzalez, R.; Grob, J.J.; Rutkowski, P.; Lao, C.D.; Cowey, L.; Schadendorf, D.; Wagstaff, J.; Dummer, R.; et al. Five-year survival with combined nivolumab and ipilimumab in advanced melanoma. N. Engl. J. Med.
**2019**, 381, 1535–1546. [Google Scholar] [CrossRef] [PubMed] - Mackey, J.R.; Ramos-Vazquez, M.; Lipatov, O.; McCarthy, N.; Krasnozhon, D.; Semiglazov, V.; Manikhas, A.; Gelmon, K.; Konecny, G.; Webster, M.; et al. Primary results of ROSE/TRIO-12, a randomized placebo-controlled phase III trial evaluating the addition of ramucirumab to first-line docetaxel chemotherapy in metastatic breast cancer. J. Clin. Oncol.
**2015**, 33, 141–148. [Google Scholar] [CrossRef] - Schwartz, L.H.; Litière, S.; De Vries, E.; Ford, R.; Gwyther, S.; Mandrekar, S.; Shankar, L.; Bogaerts, J.; Chen, A.; Dancey, J.; et al. RECIST 1.1-Update and clarification: From the RECIST committee. Eur. J. Cancer
**2016**, 62, 132–137. [Google Scholar] [CrossRef] [PubMed] - Robins, J.M.; Rotnitzky, A.; Zhao, L.P. Analysis of Semiparametric Regression Models for Repeated Outcomes in the Presence of Missing Data. J. Am. Stat. Assoc.
**1995**, 90, 106–121. [Google Scholar] [CrossRef]

**Figure 1.**The Kaplan Meier estimator plot for a randomly generated chemotherapy toxicity trial of 300 patients with 1:1 allocation. An event was considered the onset of chemotherapy toxicity (beyond stage zero) and patients were censored once their assigned duration had been reached. The hazard ratio between treatment arms was 1.25:1.

**Figure 2.**The weighted trajectory analysis plot for a randomly generated chemotherapy toxicity trial of 300 patients with 1:1 allocation. The weighted health status of both groups dropped due to increasing morbidity from chemotherapy toxicity following randomization. The hazard ratio between treatment arms was 1.25:1.

**Figure 3.**Thousandfold simulations of power as a function of sample size for both KM estimation and WTA across several hazard ratios. WTA demonstrated consistently higher power, reflecting a smaller sample size requirement during trial design. The type I error rate of WTA was approximately 0.025, indicating the method was conservative. The type I error approached 0.05 within the limit of larger trials with more distinct failure times.

**Figure 4.**Chemotherapy toxicity simulation study: 1000-fold simulations of power as a function of sample size for KM estimation, the GEE, and WTA in both its analytical and computational forms. WTA outperformed KM estimation and the GEE with consistently higher power and, thus, a smaller sample size requirement. In addition, the computational approach with WTA outperformed the analytical approach in return for a more time- and resource-intensive methodology. The computational approach also met a standard type I error rate of 0.05 that was robust to changes in trial size.

**Figure 5.**Schizophrenia disease course simulation study: 1000-fold simulations of power as a function of sample size for the GEE and WTA in its analytical form. WTA again outperformed the GEE and demonstrated a type I error rate of 0.037, closer to the 0.05 standard due to the larger size of each trial.

**Figure 7.**The Kaplan–Meier estimator plot for immunotherapy-related toxicities associated with an increase in ALT. An event was considered the onset of a nonzero toxicity grade.

**Figure 8.**Weighted trajectory analysis plot for immunotherapy-related toxicities associated with an increase in ALT. The weighted health status of the combination group initially diverged from the anti-PD-1 group but subsequent recovery led to similar longitudinal outcomes.

**Figure 9.**Figure 2A,C from Mackey et al.’s 2014 paper comparing ramucirumab to a placebo added to standard docetaxel chemotherapy [14]. The figures provide patient outcomes using KM estimates of progression-free survival (PFS) and overall survival (OS), respectively.

**Figure 10.**Weighted trajectory analysis of the original ROSE/TRIO-012 dataset using an ordinal scale that merges RECIST criteria with mortality. The trajectory of patient outcomes demonstrates that partial and complete response initially outweighed progressive disease and mortality for the first few chemotherapy cycles. Following this peak, patient prognosis was generally poor, as both treatment arms experienced growing disease burden and death.

Feature | Kaplan–Meier Estimator | Weighted Trajectory Analysis |
---|---|---|

Event | Outcome with binary coding. A patient must begin at “0” and is removed from analysis following an event (“1”) | An event is a change in clinical severity and does not remove a patient from further analysis. Must be discrete with a finite range that depends on the variable of interest |

Variable of interest | Death, metastases, local recurrence, stroke, and more. Can include variables outside of medicine, such as postgraduate employment | Graded/staged outcomes: ECOG performance, toxicities, NYHA heart failure class, questionnaire scores, and more; also includes variables outside of medicine |

Trajectory | Survival function always decreases | Bidirectional: severity function can decrease or increase |

Censoring | Removes patients from subsequent analysis (for withdrawal, discharge, loss to follow up, etc.) | |

Test for significance | Logrank test | Weighted logrank test |

Y-axis | Survival probability | Weighted health status |

X-axis | Time (discrete: days, weeks, months, etc.) | |

Y-intercept | 1.0 | Between 0 and 1.0, inclusive |

Patient ID | Treatment Arm | Duration | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|

1 | 1 | 10 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 0 | |

2 | 1 | 10 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 1 | |

3 | 0 | 11 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

4 | 1 | 6 | 0 | 0 | 0 | 0 | 0 | 0 | |||||

5 | 0 | 13 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 |

6 | 1 | 9 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ||

7 | 0 | 18 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 2 | 2 |

8 | 1 | 6 | 0 | 0 | 0 | 0 | 0 | 0 | |||||

9 | 1 | 29 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 |

10 | 0 | 4 | 0 | 0 | 0 | 0 |

Outcome | Score |
---|---|

Complete response (CR) | 0 |

Partial response (PR) | 1 |

Stable disease (SD) | 2 |

Progressive disease (PD) | 3 |

Death | 4 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Chauhan, U.; Zhao, K.; Walker, J.; Mackey, J.R.
Weighted Trajectory Analysis and Application to Clinical Outcome Assessment. *BioMedInformatics* **2023**, *3*, 829-852.
https://doi.org/10.3390/biomedinformatics3040052

**AMA Style**

Chauhan U, Zhao K, Walker J, Mackey JR.
Weighted Trajectory Analysis and Application to Clinical Outcome Assessment. *BioMedInformatics*. 2023; 3(4):829-852.
https://doi.org/10.3390/biomedinformatics3040052

**Chicago/Turabian Style**

Chauhan, Utkarsh, Kaiqiong Zhao, John Walker, and John R. Mackey.
2023. "Weighted Trajectory Analysis and Application to Clinical Outcome Assessment" *BioMedInformatics* 3, no. 4: 829-852.
https://doi.org/10.3390/biomedinformatics3040052