Predictive Nomogram for Early Recurrence after Pancreatectomy in Resectable Pancreatic Cancer: Risk Classification Using Preoperative Clinicopathologic Factors

The survival of patients with pancreatic ductal adenocarcinoma (PDAC) is closely related to recurrence. It is necessary to classify the risk factors for early recurrence and to develop a tool for predicting the initial outcome after surgery. Among patients with resected resectable PDAC at Samsung Medical Center (Seoul, Korea) between January 2007 and December 2016, 631 patients were classified as the training set. Analyses identifying preoperative factors affecting early recurrence after surgery were performed. When the p-value estimated from univariable Cox’s proportional hazard regression analysis was <0.05, the variables were included in multivariable analysis and used for establishing the nomogram. The established nomogram predicted the probability of early recurrence within 12 months after surgery in resectable PDAC. One thousand bootstrap resamplings were used to validate the nomogram. The concordance index was 0.665 (95% confidence interval [CI], 0.637–0.695), and the incremental area under the curve was 0.655 (95% CI, 0.631–0.682). We developed a web-based calculator, and the nomogram is freely available at http://pdac.smchbp.org/. This is the first nomogram to predict early recurrence after surgery for resectable PDAC in the preoperative setting, providing a method to allow proceeding to treatment customized according to the risk of individual patients.


Introduction
Pancreatic ductal adenocarcinoma (PDAC) is one of the most lethal diseases worldwide, and is the fifth leading cause of cancer deaths in Korea [1]. Although surgical resection is an essential factor in providing a cure, only a minority of PDAC cases are diagnosed at a stage that can still benefit from surgical resection [2]. Further, even in patients considered eligible for surgical resection, early recurrence within 12 months after surgery has been reported to occur in 50% to 60% and the 5-year survival rate has been reported to be only 20% to 30% [3].
The resectability criteria for PDAC without distant metastasis were proposed by the National Comprehensive Cancer Network (NCCN), and tailored therapeutic strategies according to the 2 of 11 classification are applied to improve the prognosis [4]. However, even in cases classified as resectable according to the NCCN guideline, high recurrence rate and low survival rate after surgery have been reported. For that reason, this current guideline recommends considering neoadjuvant therapy in high-risk patients of resectable disease. But, the high-risk features are presented with an ambiguous definition, such as very highly elevated CA19-9, large primary tumor, large regional lymph nodes, excessive weight loss, or extreme pain. The survival of patients with PDAC is closely related to recurrence, and early recurrence after surgery is one of the typical characteristics of PDAC. Therefore, in order to distinguish high-risk patients, it is necessary to classify the risk factors for early recurrence and to develop a tool for predicting the initial outcome after surgery beyond the criteria for determining the feasibility of surgical resection.
In the present study, we focused on the prediction of early recurrence, not survival, and we attempted to predict the postoperative outcome before the initiation of treatment. The purpose of this study was to develop a risk prediction model for early recurrence of PDAC using preoperative factors to clarify the high-risk features in patients with resectable disease.

Patients Database
Between January 2007 and December 2016, a total of 833 consecutive patients with PDAC underwent pancreatectomy with a curative intent at Samsung Medical Center (Seoul, Korea). Their electronic medical records were retrospectively reviewed from a prospectively maintained electronic database system (MDB © , Seoul, Korea). This study was approved by the Institutional Review Board of Samsung Medical Center (approval no. 2018- . Because this study aimed to extract preoperative factors affecting early recurrence after surgery in resectable PDAC, we excluded patients with initially borderline resectable/unresectable cancers according to the NCCN guideline [4], as well as patients who were lost to follow-up. Of the 753 patients with resectable PDAC after exclusion, 631 had all considered preoperative factors without omission and their data were used as the training set. Our Institutional Review Board (IRB) waived the need for written informed consent from the participants.
Follow-up data were also obtained from the records, and the disease-free survival (DFS) was measured from the time of surgery until the detection of a recurrence. For postoperative surveillance, contrast-enhanced abdominoperineal computed tomography (CT) and carbohydrate antigen 19-9 (CA19-9) levels were examined every 3 months during the first 2 years postoperatively and every 6 months thereafter at our institute. The diagnosis of recurrence was based on progressive soft tissue growth at specific sites and elevated CA19-9 levels [5]. When lesions of potential recurrent disease were detected, 18 F-fluorodeoxyglucose positron emission tomography, chest CT, and/or biopsy were performed to confirm the diagnosis of recurrence. Because recurrent lesions of PDAC often have located in inaccessible area, most of recurrences were diagnosed by imaging. However, biopsy was performed in some patients who were not accurately diagnosed by imaging. Although a clear definition of the term 'early recurrence' is currently lacking, a previous study concluded that a recurrence-free interval of 12 months is the optimal threshold for differentiating between early and late recurrence [6]. Accordingly, early recurrence was defined as recurrence within the first year after surgery in the present study.

Preoperative Data
Tumor markers including carcinoembryonic antigen (CEA) level, CA19-9 level, white blood cell count, and platelet count were collected using the measurements that were closest to the operation and within at least 1 month before the surgery. Inflammation-based prognostic scores, including neutrophil-lymphocyte ratio (NLR) and platelet-lymphocyte ratio (PLR), were calculated [7]. NLR and PLR were divided into two groups based on the 75% quantile. Tumor size, tumor location, and abutment degree to the portal vein (PV)-superior mesenteric vein (SMV) were measured using preoperative CT scans.

Data Analysis and Statistical Methods
The chi-square test was used for categorical variables and Student's t-test or the Mann-Whitney U-test was used for continuous variables. Disease-specific survival (DSS) and DFS were depicted using Kaplan-Meier curves. Cox proportional hazard regression analysis was used for estimating the effect of preoperative risk factors for early recurrence. A p-value of <0.05 was considered statistically significant. The performance of survival models for early recurrence was evaluated using the concordance index (c-index), the integrated area under the curve (iAUC), and a calibration plot using the bootstrap samples from the training set. Further, a nomogram-based prediction of DFS was developed. First, preoperative risk factors for early recurrence were selected from the training set. If the p-value was <0.05 in a univariable analysis, variables were included in multivariable analysis and in the nomogram. Second, calibration curves with a thousand bootstrap replications were plotted, as well as for the observed empirical versus the predicted probability. Finally, based on the optimal cut-off value obtained from the Youden index, the positive and negative likelihood ratios were calculated, which led to the development of a Fagan's nomogram for estimating the posttest probability of a patient to have a disease [8]. Recurrence analysis was executed using SAS version 9.4 (SAS Institute Inc., Cary, NC, USA). The nomogram was established based on the results of multivariable Cox's regression analysis using R 3.5.1 (Vienna, Austria; http://www.R-project.org/).

Demographic Features of Preoperative Factors
The clinicopathological characteristics of the overall patients (n = 753) and the training set (n = 631) are shown in Table 1. Early recurrence was identified in 394 (52.3%) of overall 753 patients, 45 (11.4%) of whom were diagnosed by biopsy. In the overall patients, the CA 19-9 level and NLR were significantly higher (p < 0.001 and p = 0.016, respectively), and the measured tumor size on CT was larger (p < 0.001) in the early recurrence group. The tumor differentiation was also significantly different (p < 0.001) between the two groups. In the training set (n = 631), the four factors mentioned above still showed statistically significant differences. Additionally, the PLR also showed a significant difference (p = 0.011).

Survival Analysis
The median DSS in the overall patients and in the training set were 23.8 and 23.6 months, respectively ( Figure 1). The median DFS and 1-year DFS rate in the overall patients were 10.4 months and 46.0%, respectively. In the training set, the median DFS and 1-year DFS rate were 10.4 months and 45.6%, respectively.

Survival Analysis
The median DSS in the overall patients and in the training set were 23.8 and 23.6 months, respectively ( Figure 1). The median DFS and 1-year DFS rate in the overall patients were 10.4 months and 46.0%, respectively. In the training set, the median DFS and 1-year DFS rate were 10.4 months and 45.6%, respectively.

Preoperative Risk Factors and Establishment of a Nomogram
Analyses identifying the preoperative factors affecting early recurrence after surgery were performed in the training set (Table 2). When the p-value estimated from univariable Cox's proportional hazard regression analysis was <0.05, the variables were included in multivariable analysis and used for establishing a nomogram, including logCEA, logCA19-9, NLR, PLR, tumor size on CT, PV-SMV abutment, and tumor differentiation. Considering the hazard ratio estimated from multivariable analysis for each factor, the total points were summed and the early recurrence probabilities were calculated based on these points. The nomogram based on the Cox model is shown in Figure 2. The nomogram predicted the probability that a patient will have a recurrence within 12 months after surgery for resectable PDAC. One thousand bootstrap resamplings were used to validate the established nomogram ( Figure 3). The c-index was 0.665 (95% confidence interval [CI], 0.637-0.695), and the iAUC was 0.655 (95% CI, 0.631-0.682). We developed a web-based calculator, and the nomogram is freely available at http://pdac.smchbp.org/.

Postoperative Outcomes
The postoperative outcomes are described in Table 3. The early recurrence group had significantly more advanced T and N stages than the no or late recurrence group (all p < 0.001). However, there were no significant differences between the two groups in factors that could affect the prognosis or the results of this study, such as resection margin status or adjuvant therapy (p = 0.246 and p = 0.338, respectively). Postoperative complications, which were graded by Clavien-Dindo complication classification [9,10], were not also significantly different between two groups. In terms of recurrence patterns, there were more systemic recurrences in the early recurrence group (p < 0.001).

Postoperative Outcomes
The postoperative outcomes are described in Table 3. The early recurrence group had significantly more advanced T and N stages than the no or late recurrence group (all p < 0.001). However, there were no significant differences between the two groups in factors that could affect the prognosis or the results of this study, such as resection margin status or adjuvant therapy (p = 0.246 and p = 0.338, respectively). Postoperative complications, which were graded by Clavien-Dindo complication classification [9,10], were not also significantly different between two groups. In terms of recurrence patterns, there were more systemic recurrences in the early recurrence group (p < 0.001).

Postoperative Outcomes
The postoperative outcomes are described in Table 3. The early recurrence group had significantly more advanced T and N stages than the no or late recurrence group (all p < 0.001). However, there were no significant differences between the two groups in factors that could affect the prognosis or the results of this study, such as resection margin status or adjuvant therapy (p = 0.246 and p = 0.338, respectively). Postoperative complications, which were graded by Clavien-Dindo complication classification [9,10], were not also significantly different between two groups. In terms of recurrence patterns, there were more systemic recurrences in the early recurrence group (p < 0.001).  [9,10].; † Recurrence in the remnant pancreas or soft tissue around the pancreaticojejunostomy site, such as along the celiac or superior mesenteric artery.; ‡ Including single distant metastasis, multiple metastasis, and peritoneal seeding.

Discussion
Patients with resected PDAC are very likely to die of their disease because PDAC is notorious for aggressive invasion, early metastasis, and subsequent poor clinical outcomes. To date, many researchers have studied the factors that can predict the prognosis of PDAC and have made efforts to improve patient survival. However, the studies have shown inconsistent results depending on the research institute and the study cohort, and the predictors have not been able to directly contribute to improving prognosis because most of them are unmodifiable factors [3,11,12]. To complement the inconsistency of the prognostic factors of PDAC and to more accurately predict the survival possibility, predictive nomograms developed by analysing the contribution of each factor have been reported [13][14][15][16]. Although these outcomes may contribute to predicting the overall outcomes of a patient, there are still limitations in contributing to prognosis improvement. Therefore, we focused on the prediction of early recurrence, not survival. To transform prognostic factors with inconsistencies and unmodifiable characteristics into a tool that can be applied clinically, we developed a risk prediction nomogram for early recurrence of resectable PDAC in the preoperative setting ( Figure 2).
As neoadjuvant therapy (NAT) for PDAC progresses, its role is also evolving. Previous studies have shown that NAT downstages some cases of initially borderline resectable or even locally advanced PDAC to a point wherein they become eligible for surgery [17][18][19][20]. Furthermore, some centres have taken the view that nearly all patients who appear to have potentially resectable PDAC should be considered for NAT on account of the inaccuracy of imaging, high rates of positive margins, and poor survival [21]. A recent meta-analysis showed that NAT for resectable PDAC seemed to improve overall survival in intention-to-treat and per-protocol analyses; however, the study showed that the overall resection rate was significantly lower with NAT than with upfront surgery (66.0 versus 81.3%, p < 0.001) [22]. Although the meta-analysis involved many considerations because it included both Cancers 2020, 12, 137 8 of 11 resectable and borderline resectable PDAC cases and most of the studies utilized older chemotherapy regimens, the low resection rate should be particularly carefully considered in patients with initially resectable PDAC. Low resection rates may be a basis for avoiding unnecessary surgery in some patients; however, they might lead to a loss of treatment opportunity in others. Therefore, we believe that it is timely to consider criteria that will be the clear boundary for NAT conducted in the setting of resectable PDAC.
In the present study, we devised a tool for predicting posttreatment outcomes before starting treatment, which can be used to determine the treatment direction. To optimize the predictive value of the designed nomogram, we determined the cut-off value using the Youden index. When the cut-off value of the nomogram-predicted probability of early recurrence (within 12 months) was set to 0.71 (Figure 4), the estimated diagnostic sensitivity and specificity were 54% and 91%, respectively. The likelihood ratios of positive and negative test results were calculated as 5.9 and 0.51, respectively. If a patient with an estimated early recurrence rate (pretest probability) of 52.61% tests positive, the posttest probability that the patient truly has early recurrence would be approximately 86.76% (green line). Alternatively, if the patient tests negative, the posttest probability would be approximately 36.15% (red line). In this way, it is expected that the nomogram can be used not only for prediction but also as a tool for determining the direction of actual treatment and customized treatment.
Cancers 2020, 12, x 8 of 11 seemed to improve overall survival in intention-to-treat and per-protocol analyses; however, the study showed that the overall resection rate was significantly lower with NAT than with upfront surgery (66.0 versus 81.3%, p < 0.001) [22]. Although the meta-analysis involved many considerations because it included both resectable and borderline resectable PDAC cases and most of the studies utilized older chemotherapy regimens, the low resection rate should be particularly carefully considered in patients with initially resectable PDAC. Low resection rates may be a basis for avoiding unnecessary surgery in some patients; however, they might lead to a loss of treatment opportunity in others. Therefore, we believe that it is timely to consider criteria that will be the clear boundary for NAT conducted in the setting of resectable PDAC.
In the present study, we devised a tool for predicting posttreatment outcomes before starting treatment, which can be used to determine the treatment direction. To optimize the predictive value of the designed nomogram, we determined the cut-off value using the Youden index. When the cutoff value of the nomogram-predicted probability of early recurrence (within 12 months) was set to 0.71 (Figure 4), the estimated diagnostic sensitivity and specificity were 54% and 91%, respectively. The likelihood ratios of positive and negative test results were calculated as 5.9 and 0.51, respectively. If a patient with an estimated early recurrence rate (pretest probability) of 52.61% tests positive, the posttest probability that the patient truly has early recurrence would be approximately 86.76% (green line). Alternatively, if the patient tests negative, the posttest probability would be approximately 36.15% (red line). In this way, it is expected that the nomogram can be used not only for prediction but also as a tool for determining the direction of actual treatment and customized treatment. For example, an endoscopic ultrasound-guided biopsy-proven poorly differentiated 3.3-cm lesion in the pancreatic head with SMV abutment, white blood cell count of 5400/μL (neutrophil count of 69% and lymphocyte count of 19.4%), platelet count of 250/μL, CEA level of 11.2 ng/mL, and CA19-9 level of 486 U/mL would yield a total of 136 points and the estimated early recurrence probability would be 84.4% (Figure 2). The estimated probability of 84.4% exceeds the cut-off value of 0.71, and For example, an endoscopic ultrasound-guided biopsy-proven poorly differentiated 3.3-cm lesion in the pancreatic head with SMV abutment, white blood cell count of 5400/µL (neutrophil count of 69% and lymphocyte count of 19.4%), platelet count of 250/µL, CEA level of 11.2 ng/mL, and CA19-9 level of 486 U/mL would yield a total of 136 points and the estimated early recurrence probability would be 84.4% (Figure 2). The estimated probability of 84.4% exceeds the cut-off value of 0.71, and this patient may be classified as high-risk of early recurrence. We could then recommend neoadjuvant therapy to this high-risk patient with resectable PDAC. This study has several limitations. First, measurement errors may have occurred, because there might be inaccuracies in measuring the tumor size on CT scans. Although the magnitude of the error does not always match, we used preoperative values in the present study and confirmed that this approach had sufficient prognostic value. Likewise, because we used only preoperative factors, postoperative factors that could affect the prognosis were not considered. However, as shown in Table 3, the resection margin and adjuvant therapy, which may artificially affect the results, did not differ between the two groups. Second, there was a possibility of selection bias because this study was conducted in a training set with only patients who did not have omitted data. However, most of the factors did not show any differences, and the present study did not aim to investigate the characteristics of consecutive patients but to investigate the characteristics of individual patients according to the recurrence classification. Third, although the nomogram was devised using a large number of patients at a large tertiary centre and validation using bootstrapping was attempted, it was not possible to obtain enough patient numbers to perform sufficient validation. Verification will be necessary through validation, and it is worthwhile to test newly developed tools through deep learning using artificial intelligence beyond the statistical limits. In addition, in order to overcome the limitations of retrospective study with the characteristics of selection bias and incomplete data, we think that further research on prospective validation and controlled study, as well as external validation, will be needed.

Conclusions
In the present study, we attempted to develop a nomogram for predicting early recurrence after surgery using only preoperative clinicopathologic data. For now, this nomogram is meaningful as a tool that can be more widely applied because it uses easily accessible data. This tool is expected to be able to identify patients who are classified as morphologically resectable but have high-risk features of early recurrence, and it can provide a method to allow proceeding to treatment customized according to the risk of individual patients, by predicting early recurrence before surgery rather than simply predicting prognosis. Further, more advanced forms of nomograms using more specific and advanced data, such as biomarkers, as well as the development of tools using deep learning with artificial intelligence are expected in the future.