Real-World Data Validation of NAPOLI-1 Nomogram for the Prediction of Overall Survival in Metastatic Pancreatic Cancer

Simple Summary The nomogram derived from the pivotal phase III NAPOLI-1 study could predict the overall survival in gemcitabine-refractory metastatic pancreatic cancer treated with liposomal irinotecan plus fluorouracil and leucovorin. However, the NAPOLI-1 nomogram has not been validated in a real-world setting and therefore the applicability of the NAPOLI-1 nomogram in daily practice remains unknown. In the current study, we validated the NAPOLI-1 nomogram in a multicenter real-world cohort and confirmed that the NAPOLI-1 nomogram could predict the prognosis of gemcitabine-refractory metastatic pancreatic cancer in daily practice and may help clinical decision making. We further found that the relative dose intensity at 6 weeks was an independent prognostic factor beyond the NAPOLI-1 nomogram, which highlighted the importance of optimal dose delivery regardless of the baseline condition. Abstract Background: The nomogram derived from the pivotal phase III NAPOLI-1 study demonstrated a significant ability to predict median overall survival (OS) in gemcitabine-refractory metastatic pancreatic ductal adenocarcinoma (PDAC) treated with liposomal irinotecan plus fluorouracil and leucovorin (nal-IRI+5-FU/LV). However, the NAPOLI-1 nomogram has not been validated in a real-world setting and therefore the applicability of the NAPOLI-1 nomogram in daily practice remains unknown. This study aims to evaluate the NAPOLI-1 nomogram in a multicenter real-world cohort. Methods: The NAPOLI-1 nomogram was applied to a previously established cohort of metastatic PDAC patients treated with nal-IRI+5-FU/LV in nine participating centers in Taiwan. Patients were divided into three risk groups according to the NAPOLI-1 nomogram. The survival impact of relative dose intensity at 6 weeks (RDI at 6 weeks) in different risk groups was also investigated. Results: Of the 473 included patients, the median OSs of patients classified as low (n = 156), medium (n = 186), and high (n = 131) risk were 10.9, 6.3, and 4.3 months, respectively (p < 0.0001). The survival impact of RDI at 6 weeks remained significant after stratification by risk groups, adjustment with Cox regression, inverse probability weighting, or propensity score matching. Conclusions: Our results support the usefulness of the NAPOLI-1 nomogram for risk stratification in gemcitabine-refractory metastatic PDAC treated with nal-IRI+5-FU/LV in daily practice. We further showed that the RDI at 6 weeks is an independent prognostic factor beyond the NAPOLI-1 nomogram.

A nomogram, usually derived from complicated models, is a user-friendly tool to help physicians quickly and easily understand the prognosis of a patient and aid clinical decision-making [16][17][18]. A well-established nomogram will not only help clinicians with risk stratification and decision-making but may also serve as a comparison benchmark to compare different studies. The NAPOLI-1 study-derived nomogram consisted of eight independent dichotomized parameters identified in multivariate Cox regression for overall survival in the NAPOLI-1 population, including baseline Karnofsky performance score, baseline albumin level, baseline neutrophil-lymphocyte ratio, the presence of liver metastasis, baseline CA19-9, stage IV at diagnosis, body mass index (BMI) and receiving nal-IRI+5-FU/LV combination treatment [1,19]. Based on the NALOPI-1 nomogram, a risk score ranging from 0 to 680 was calculated for each patient. The NAPOLI-1 population was then divided into three risk tertiles, low risk, intermediate risk, and high risk, with cut-off point scores of >370, 260-370, and <260, respectively. The NAPOLI-1 nomogram demonstrated significant discriminatory power to predict overall survival. Patients in high-, intermediate-, and low-risk groups had a median OS of 2.9, 5.3, and 8.5 months, respectively. However, the NAPOLI-1 nomogram was not validated in a real-world setting and therefore the applicability of the NAPOLI-1 nomogram in daily practice remains uncertain.
The per-protocol (PP) analysis of the NAPOLI-1 study defined the PP population as relative dose intensity at 6 weeks (RDI at 6 weeks) ≥ 80% and the median OS of the PP population was significantly better than that of non-PP populations (8.9 vs. 4.4 months, respectively) [20]. Our previous real-world multicenter study further set another cut-point of 60% and divided patients into three groups based on their RDI at 6 weeks: >80%, 60-80%, and <60%. We found that RDI at 6 weeks was an independent prognostic factor of OS even after adjustment of covariates by multivariable Cox regression [21]. In this study, we validate the NAPOLI-1 nomogram in the same large real-world multicenter cohort and further address how RDI at 6 weeks impacts the survival of patients in different risk groups.

Materials and Methods
A real-world multicenter cohort of patients with pancreatic cancer treated with nal-IRI+5-FU/LV has been previously described [21]. The inclusion and exclusion criteria of the current study were set to accord with the NAPOLI-1 population. The inclusion criteria included (1) prior use of gemcitabine; (2) presence of metastatic disease while receiving nal-IRI+5-FU/LV; patients with an initial diagnosis of American Joint Committee on Cancer (AJCC) stage I-III are eligible if they developed metastasis after gemcitabine-based treatment; and (3) Karnofsky performance score ≥ 70 (equivalent to ECOG performance score 0-1) while receiving nal-IRI+5-FU/LV. Patients with the following conditions were excluded: (1) using nal-IRI without 5-FU or in combination with S1; (2) receiving nal-IRI+5-FU/LV plus other agents; and (3) receiving nal-IRI+5-FU/LV before the reimbursement in August 2018. This retrospective study was approved by the Institutional Review Board (IRB) with a waiver of informed consent and followed the Declaration of Helsinki. The IRB approval numbers of each participating institute were as follows: Chang Gung Memorial Hospital, 202100783B0; China Medical University Hospital, CMUH109-REC2-176; Chung Shan Medical University Hospital, CS2-21095; National Cheng Kung University Hospital, A-ER-109-477; National Taiwan University Hospital, 201911042RINC; Kaohsiung Medical University Hospital, KMUHIRB-E(I)-20210150; Taipei Veterans General Hospital, 2021-08-001AC; and Tri-Service General Hospital, B202105057.
Descriptive statistics are presented as medians or percentages. The difference in proportions between groups was compared by Fisher's exact test. The Kolmogorov-Smirnov test was used to check the normality of data distribution.
The reverse Kaplan-Meier method was used to estimate the median duration of followup. The definition of PFS was the time between the beginning of nal-IRI+5-FU/LV to the date of either radiological or clinical progression, death, intolerance, loss to follow-up, or data cut-off. OS was defined as the interval between the beginning of nal-IRI+5-FU/LV and death, loss to follow-up, or data cut-off. Intolerance, loss to follow-up, or data cut-off will be censored in PFS data while the loss to follow-up or data cut-off will be censored in OS data. The Kaplan-Meier method was applied for the calculation of PFS and OS while the log-rank test was used for survival comparison between groups.
Three parameters in the NAPOLI-1 nomogram, albumin, neutrophil-lymphocyte ratio, and baseline CA19-9, were not available in some patients and were imputed by using the R multivariate imputation by chained equation (MICE) package [22]. The area under the time-dependent receiver operating characteristic curve (AUC) with inverse probability of censoring weighting (IPCW) technique was estimated to evaluate the prediction accuracy of the NAPOLI-1 nomogram. In terms of investigating the effect of cumulative dose, Cox regression was used to estimate the hazard ratio (HR) and 95% CI with the nomogram score as an adjusted covariate. Propensity score weighting and matching methods were also applied to further balance the distribution of nomogram scores among cumulative dose groups. The inverse probability of weights (IPW) was computed from the generalized propensity scores (GPSs) by using multinomial logistic regression with cumulative dose groups as the outcome variable and the nomogram as the covariate. The 3 groups were matched at 1:1:1 with the same nomogram score. When more than one participant had the same score, patients were randomly selected. We also conducted these statistical analyses on patients without any missing values (n = 264) as a sensitivity analysis. All variables with p < 0.05 were statistically significant. All statistical analyses were performed using R version 4.0.5 (R Core Team, Vienna, Austria) and SAS version 9.4 (SAS Institute Inc., Cary, NC, USA).

Patient Demographics
A total of 696 patients treated with nal-IRI+5-FU/LV were identified from 9 participating centers in Taiwan, as previously described [21]. Among them, 523 patients met the inclusion criteria. After excluding the 50 patients whose treatment had a combination that included oxaliplatin (n = 19), immunotherapy (n = 4) or other agents (n = 3), or whose treatment used S1 instead of 5-FU (n = 3) and those who received nal-IRI before reimbursement in Aug. 2018 (n = 21), finally, 473 patients fulfilled the inclusion and exclusion criteria and were included in the current study ( Figure 1). cumulative dose groups. The inverse probability of the generalized propensity scores (GPSs) by using m cumulative dose groups as the outcome variable and 3 groups were matched at 1:1:1 with the same nom participant had the same score, patients were random statistical analyses on patients without any missing ysis. All variables with p < 0.05 were statistically sig performed using R version 4.0.5 (R Core Team, Vienn Institute Inc., Cary, NC, USA).

Model Performance
In the current study, the AUCs of the NAPOLI-1 nomogram for median OS pred tion range from 0.698 to 0.738 ( Figure 3A,C). Most of the parameters used in the NAPO 1 nomogram remain significant in the multivariable analysis for the current RWD pop lation (Supplementary Table S1). Another multivariable analysis was performed to f ther explore whether any factor other than the NAPOLI-1 nomogram had a signific impact on survival by selecting a covariate with a p-value < 0.05 in univariate analy RDI at 6 weeks was found to be the most significant factor beyond the NAPOLI-1 nom gram (Supplementary Table S2). Adding RDI at 6 weeks to the NAPOLI-1 nomogram i proved the AUC mainly at 3 months and 6 months, which reached values of 0.822 a 0.731, respectively ( Figure 3B,D). In the sensitivity analysis, the AUC of 264 patients wi In the sensitivity analysis which excluded 209 patients with missing values in the nomogram, the distribution of NAPOLI-1 nomogram risk classification, median PFS, and median OS of the remaining 264 patients without missing value imputation were similar to those of the entire cohort (Supplementary Figure S1).

Model Performance
In the current study, the AUCs of the NAPOLI-1 nomogram for median OS prediction range from 0.698 to 0.738 ( Figure 3A,C). Most of the parameters used in the NAPOLI-1 nomogram remain significant in the multivariable analysis for the current RWD population (Supplementary Table S1). Another multivariable analysis was performed to further explore whether any factor other than the NAPOLI-1 nomogram had a significant impact on survival by selecting a covariate with a p-value < 0.05 in univariate analysis. RDI at 6 weeks was found to be the most significant factor beyond the NAPOLI-1 nomogram (Supplementary Table S2). Adding RDI at 6 weeks to the NAPOLI-1 nomogram improved the AUC mainly at 3 months and 6 months, which reached values of 0.822 and 0.731, Cancers 2023, 15, 1008 7 of 13 respectively ( Figure 3B,D). In the sensitivity analysis, the AUC of 264 patients without missing value imputation was similar to that of the entire cohort (Supplementary Figure S2).

Relative Dose Intensity at 6 Weeks Is an Independent Prognostic Factor
The distribution of cumulative dose during the first 6 weeks was s ferent risk groups ( Figure 4A). The distribution of nomogram risk s among the three dose intensity groups ( Figure 4B). Linear regression statistically significant correlation between nomogram risk score and 6 dose (Spearman's correlation coefficient = 0.041, p = 0.37) ( Figure 4C consistent in that RDI at 6 weeks was independent of the baseline cond

Relative Dose Intensity at 6 Weeks Is an Independent Prognostic Factor
The distribution of cumulative dose during the first 6 weeks was similar among different risk groups ( Figure 4A). The distribution of nomogram risk scores was similar among the three dose intensity groups ( Figure 4B). Linear regression demonstrated no statistically significant correlation between nomogram risk score and 6-week cumulative dose (Spearman's correlation coefficient = 0.041, p = 0.37) ( Figure 4C). All results were consistent in that RDI at 6 weeks was independent of the baseline condition.
In the sensitivity analysis, the dose distribution, survival difference before and after covariate adjustment, stratification, weighting, and matching in the 264 patients without missing value imputation were quite similar to those of the entire cohort ( Supplementary  Figures S3 and S4, Table S3).    In the sensitivity analysis, the dose distribution, survival difference before and after covariate adjustment, stratification, weighting, and matching in the 264 patients without missing value imputation were quite similar to those of the entire cohort (Supplementary Figures S3 and S4, Table S3).

Real-World Safety Profile
The most common ≥ grade 3 hematological toxicities were neutropenia (23.3%) and anemia (19.5%); while the most common non-hematological toxicities were hypokalemia (12.7%) and increased bilirubin (7.4%) ( Table 3). There was no significant difference in safety profiles among the three different risk groups.

Discussion
Our study is the first study to evaluate and validate the NAPOLI-1 nomogram in a real-world setting and support its use as a risk stratification tool, which will not only help physicians and patients to make decisions but can also be implemented in the comparison of results among parallel studies, including real-world data and clinical trials.
In the per-protocol (PP) analysis of the NAPOLI-1 study, patients in PP populations (defined as RDI at 6 weeks ≥ 80%) had significantly better median OS as compared to non-PP populations (8.9 vs. 4.4 months, respectively) [20]. One may argue that the longer survival of patients with higher RDI at 6 weeks might result from better baseline condition as reflected by the higher percentage of patients with Karnofsky performance score ≥ 90 in PP than in non-PP populations (62.1% and 49%, respectively). In our previous study, we showed that RDI at 6 weeks remained an important prognostic factor after confounder adjustment [21]. In the current study, there was no significant correlation between NAPOLI-1 nomogram risk score, an indicator of baseline condition, and RDI at 6 weeks ( Figure 3C). In addition, RDI at 6 weeks significantly impacted the survival of different risk groups ( Figure 5A-C), even after weighting ( Figure 4D), matching ( Figure 4E), and Cox regression adjustment (Table 2). Through different methodologies, the results highlighted the importance of dose delivery regardless of the baseline condition.
In the post hoc analysis of the NAPOLI-1 study, the median OS of Asian patients receiving nal-IRI+5-FU/LV was 8.9 months compared to the 6.1 months of the entire nal-IRI+5-FU/LV-treated population [23]. Of note, the median OS of patients with a low and intermediate risk score in the current study was numerically better than that of patients with nal-IRI+5-FU/LV in the NAPOLI-1 study (10.9 vs. 9.0 months and 6.3 vs. 5.4 months, respectively), but similar survival for patients with a high risk score (4.3 vs. 4.3 months) [19]. In a recent population pharmacokinetic study, Asian patients had a significantly higher average concentration of un-encapsulated SN38 (uSN38 C avg ), a pharmacokinetic parameter associated with prolonged OS, compared to Caucasians in the nal-IRI+5-FU/LV arm of the NAPOLI-1 study [24]. Interestingly, the uSN38 concentration was similar in Caucasian patients with and without UGT1A1*28/*28 homozygosity, but significantly higher in Asian patients with UGT1A1*6/*6 homozygosity or UGT1A1*6/*28 compound heterozygosity than in those with UGT1A1*6 single heterozygosity or wild-type individuals [24,25]. Since the allele frequency of UGT1A1*6 only ranged from 15-30% with UGT1A1*6 homozygosity or compound heterozygosity in 9-10% of the Asian population, the mechanisms for the ethnic differences in nal-IRI pharmacokinetics and survival after nal-IRI+5-FU/LV are likely multifactorial and warrant further investigation. This study has some limitations. First, three parameters in the nomogram, baseline albumin, neutrophil-lymphocyte ratio, and baseline CA 19-9 level were not available in all patients and imputed by the MICE procedure. The handling of missing values is a huge discipline and each method had its advantages and disadvantages. Originally proposed by Donald Rubin in 1987, the multiple imputation procedure has been greatly improved and become more widely used even in clinical trials published in The Lancet and The New England Journal of Medicine [26][27][28]. However, the results from the sensitivity analyses of the 264 patients without missing values were consistent with all study results, which justifies the imputation procedure. Second, in the original NAPOLI-1 nomogram, nal-IRI+5-FU/LV treatment was one of the eight dichotomized parameters. The inclusion of such a parameter can be redundant if the goal of the study was intra-study risk categorization for a patient cohort with homogenous treatment. On the other hand, it is reasonable to do so if inter-studies survival outcome comparison is the main theme, such as RWD versus clinical trial results in this study. Third, although all baseline characteristics other than ethnicity were comparable between the current study and NAPOLI-1, the practice pattern was very different in that more patients in our cohort had prior exposure to fluorouracil-containing treatment than did participants in NAPOLI-1 (77.4% vs. 42.7%, respectively). Due to the delayed reimbursement of nab-paclitaxel and FOLFIRINOX, the gemcitabine/fluorouracil-based triplet, including the GOFL regimen (gemcitabine, oxaliplatin, 5-FU, and leucovorin) and SLOG regimen (S1, leucovorin, oxaliplatin, and gemcitabine), were among the most commonly used first-line regimens for advanced pancreatic cancer in Taiwan before 2020 [29][30][31][32][33][34]. Although the current study suggests previous more aggressive gemcitabine-based treatments, such as GOFL and SLOG, might not impact the therapeutic efficacy of second-line nal-IRI+5-FU/LV, the issue still deserves further exploration with NAPOLI-1 nomogram risk group-stratified comparison.

Conclusions
In the current study, we employed several strategies including stratification, multivariable adjustment, weighting, and matching methods. The results were consistent among these methods, which indicates the robustness of our findings in using the NAPOLI-1 nomogram as a measurable confounding factor. Our study not only validated the NAPOLI-1 nomogram for risk stratification in a real-world setting but also identified RDI at 6 weeks as an independent prognostic factor beyond the NAPOLI-1 nomogram.
Supplementary Materials: The following supporting information can be downloaded at: https: //www.mdpi.com/article/10.3390/cancers15041008/s1, Figure S1: Sensitivity analysis of survival in patients without missing values in the NAPOLI-1 nomogram (n = 264); Figure S2: Sensitivity analysis of model performance in patients without missing values in the NAPOLI-1 nomogram (n = 264); Figure S3: Sensitivity analysis of dose distribution in patients without missing values in the NAPOLI-1 nomogram (n = 264); Figure S4: Sensitivity analysis of survival in patients without missing values in the NAPOLI-1 nomogram (n = 264). Table S1: Multivariable Cox regression of factors in the NAPOLI-1 nomogram (n = 473); Table S2: Multivariable Cox regression of other covariates with NAPOLI-1 nomogram risk score (n =473); Table S3: Model in patients without missing values (n = 264). Informed Consent Statement: Patient consent was waived due to the retrospective nature of the study.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author.

Conflicts of Interest:
Sz-Chi Chiu is employed by PharmaEngine, Inc. The remaining authors declare no conflict of interest.