Evaluating NSQIP Outcomes According to the Clavien–Dindo Classification: A Model to Estimate Global Outcome Measures Following Hepatopancreaticobiliary Surgery

Abstract

Background: The National Surgical Quality Improvement Program (NSQIP) database provides one of the largest repositories of surgical outcome data—guiding local, national, and international quality improvement and research. We aim to describe a model to estimate Clavien–Dindo complication (CDC) rates from NSQIP data to enable comprehensive outcome measurement, allowing an NSQIP-based surrogate measure for longer-term outcomes. Methods: This is a validation study of a model to estimate CDCs from NSQIP data for pancreaticoduodenectomy (PD) and hepatic resection (HR). The primary objective of this study is to evaluate whether our method to estimate CDCs ≥ 3 outcomes from NSQIP data results in similar serious complication rates to large benchmark studies on outcomes following PD and HR. Secondary outcomes evaluate whether specific NSQIP outcomes provide adequate information to estimate CDC grades I-V following PD and HR. Results: We evaluated 20,575 patients undergoing PD, with 71.3% having pancreatic ductal adenocarcinoma. Comparing CDCs ≥ 3 complications for NSQIP and benchmark PD patients, we estimated a 23.2% rate with our model, which was significantly lower than the reported 27.6% in the benchmark study (p < 0.001). Additionally, the benchmark reported higher complication rates for every CDC grade compared to our estimates using NSQIP PD patients (p < 0.001). Further, we evaluated 29,809 patients within NSQIP undergoing HR, where most patients with a diagnosis listed had colorectal cancer metastases (30.8%). Compared to the benchmark HR study (n = 2159), the NSQIP patients were less likely to have hepatic resection for malignancy (57.7% vs. 84.0%; p < 0.001). Comparing CDCs ≥ 3 complications following HR demonstrated that rates were clinically similar (13.0% vs. 15.8%) but statistically different between the benchmark study and NSQIP data (p < 0.001). Additionally, the NSQIP patients had lower rates of estimated complications for nearly all CDC grades (p < 0.001). Conclusions: This is the first reported method to estimate aggregate morbidity from NSQIP data. Results demonstrate that despite differences in this and comparator cohorts, this model may underestimate CDC grade 1–2 complications but provide similar rates of CDCs ≥ 3 complications compared to benchmark studies. Future studies to validate or modify this estimation method are warranted and may allow extrapolation of short-term NSQIP measures to oncologic, quality of life, and long-term outcomes.

1. Introduction

The National Surgical Quality Improvement Program (NSQIP) database provides one of the largest repositories of surgical outcome data—informing local, national, and international quality improvement and research [1]. Despite its rich demographic and outcome data, NSQIP is limited by 30-day post-operative evaluation, with no studies currently available relating the impact of these outcome measures on longer-term morbidity and mortality [2]. On the other hand, the Clavien–Dindo Classification (CDC) is a well-established measure of overall surgical morbidity with growing evidence relating each to long-term quality of life, oncologic outcomes, and survival following hepatopancreaticobiliary (HPB) surgery [3,4,5,6,7,8,9,10,11,12,13,14,15]. An accurate model to estimate CDC from NSQIP outcomes would provide valuable information for all future NSQIP studies by enabling comprehensive outcome measurement relating to long-term outcomes.

NSQIP data is collected and reviewed by trained nurse reviewers with data for >150 data points per patient, with additional outcomes collected for patients undergoing pancreatic and hepatic resections [2]. Data is collected across 874 hospitals in Canada and the United States. Outcomes are most often assessed in binary fashion, with some specific pancreatic and hepatic outcomes also outlining interventions provided for each complication. On the other hand, the CDC provides a measure of overall complication severity by assessing complications according to their intervention and grading patient outcomes based on the most severe complication from a patient’s hospital stay [3]. By creating and validating a model to estimate CDC from NSQIP outcomes, future NSQIP studies would be able to report CDC as a consistent aggregate outcome measure that is associated with long-term outcomes.

The aim of this study is to generate a model to estimate CDC from NSQIP data as a comprehensive outcome measure with evaluation against prior benchmark HPB studies. If accurate, this information will provide a validation for CDC estimation from NSQIP data and enable assessment of a surrogate marker for long-term outcomes that can be utilized in future studies.

2. Methods

2.1. Study Design, Objectives, and Data Source

This is a validation study of a model to estimate CDC from NSQIP data for pancreaticoduodenectomy (PD) and hepatic resection (HR). The primary objective of this study is to evaluate whether our method to estimate CDC ≥ 3 outcomes from NSQIP data results in similar serious complication rates to large benchmark studies on outcomes following pancreaticoduodenectomy and liver resection. CDC ≥ 3 will be utilized as the primary outcome, as it is the most common measure associated with long-term outcomes [10,11,12,14,16].

Secondary outcomes will evaluate whether specific NSQIP outcomes provide adequate information to estimate CDC grades I–V following PD and HR. Since the CCI is calculated from CDC grading, we also demonstrate a method to evaluate CCI from NSQIP data, with discussion on its accuracy. Finally, we also intend to provide the statistical software code to estimate CDC and the CCI from NSQIP data as an open source following this study to be used in future studies.

We hypothesize that serious complications defined by CDC ≥ 3 can be estimated from NSQIP data for PD and HR, resulting in similar outcomes to previously published large benchmark studies. To accomplish this study, the NSQIP hepatic-targeted and pancreatic-targeted databases and general database for the years 2016–2021 were merged. This study was exempt from ethics approval as this was a retrospective cohort study that utilized anonymized patient information from a nationally collected database.

2.2. Study Population, Comparators, and Variable Definitions

Two unique analyses were completed for this study to evaluate our CDC estimation model for PD and HR. PD and HR were selected as they represent the largest group of hepatopancreaticobiliary surgical procedures within the NSQIP and therefore allowed maximizing the sample size of this study’s evaluation. Estimated CDC was compared to large studies evaluating outcomes following pancreaticoduodenectomy [17] and hepatic resection [18]. These studies were chosen because they were the largest studies available that detailed outcomes and their associated CDC grading.

To enable comparison for pancreaticoduodenectomy, we identified patients using current procedural terminology (CPT) codes 48152 (Whipple-type procedure), 48153 (pylorus-sparing, Whipple-type procedure), 48150 (Whipple), and 48154 (pylorus-sparing, Whipple-type procedure). Only oncologic pancreaticoduodenectomy was included for comparison because the comparator study by Russell et al. (2023) only evaluated patients with malignant etiology [11,17]. Otherwise, we included and defined the proportion of patients undergoing minimally invasive surgery (MIS) and vascular resection.

To identify patients undergoing hepatic resection, we defined included patients using current procedural terminology (CPT) codes 47120 (hepatectomy, partial lobectomy), 47125 (left lobectomy), 47130 (right lobectomy), and 47122 (trisegmentectomy). Patients undergoing hepatic resection requiring bile duct reconstruction and those with MIS resection were included and defined within the demographics.

To improve outcome assessment, included patient demographics were compared to historic controls within benchmark studies. In terms of demographics, patients were evaluated based on their indication for surgery, age, sex, body mass index (BMI), functional status prior to surgery, and their American Society of Anesthesiologists (ASA) score. Patient comorbidities were compared, including cardiopulmonary conditions such as congestive heart failure (CHF), hypertension, chronic obstructive pulmonary disease (COPD), and active smoking status. Other demographics included the presence of diabetes, dialysis dependence, chronic steroid use, and diagnosis of a preoperative bleeding disorder. Oncologic factors characterized in this study were preoperative neoadjuvant therapy, tumor size (for hepatic resections), and tumor invasion (described as T3 or T4 tumor status). Finally, preoperative sepsis (none, systemic inflammatory response syndrome, or sepsis) was described. For PD we also evaluated pancreatic texture (soft vs. intermediate/hard), whether they had preoperative biliary stenting, and whether they required venous or arterial resection during their procedure. For HR we also analyzed surgery type (partial lobectomy, right lobectomy, left lobectomy, or trisegmentectomy) and whether patients received biliary reconstruction (i.e., hepaticojejunostomy).

Outcomes were defined according to NSQIP predetermined definitions [19]. In this study we evaluated operative time, length of post-operative hospital stay, discharge destination (home, rehabilitation, increased level of care), unplanned reoperations, readmission to hospital within 30 days, and if the patients were still admitted to hospital > 30 days post-operatively. Infectious post-operative complications were extracted, including wound complications (superficial surgical site infection (SSI), deep SSI, organ space SSI, wound disruption/dehiscence), pneumonia, and urinary tract infection. Other complications included respiratory complications (unplanned intubation, pulmonary embolism), cardiovascular complications (myocardial infarction, cardiac arrest), acute renal failure, renal insufficiency, sepsis, septic shock, deep vein thrombosis (DVT), cerebral vascular accidents (CVA), bleeding, and mortality within 30 days of operation. Outcomes were defined according to definitions within the NSQIP participant user file [19]. For pancreaticoduodenectomy cases, we also considered post-operative pancreatic fistula (POPF) and post-operative acute pancreatitis described according to the updated International Study Group for Pancreatic Surgery (ISGPS) definitions [20,21]. Unique to the NSQIP hepatic data, we also evaluated any post-operative bile leak, including any bile leak requiring drain maintenance, percutaneous drain placement, or reoperation. We also characterized any post-hepatectomy liver failure and graded these complications according to the International Study Group of Liver Surgery [22].

The datasets presented in this article are not readily available because they are part of the National Surgical Quality Improvement Program. Requests to access the datasets should be directed to the American College of Surgeons.

2.3. Model to Estimate CDC from NSQIP Complications

In order to estimate CDC from NSQIP outcomes, each complication was assessed and associated with a CDC outcome. For most outcomes, treatment of each complication was not outlined by the NSQIP. In these scenarios, treatments were assumed based on expected/accepted treatments for these complications as described in

Table 1. Complications and their associated Clavien–Dindo Classification applied during this study for pancreaticoduodenectomy and hepatic resection.

Complication	Clavien–Dindo Classification	Rationale/Proposed Treatment
Assessment for pancreaticoduodenectomy
Superficial SSI	1	Bedside wound exploration
Deep incision SSI	1	Bedside wound exploration
Readmission	1	Conservative management
Grade A POPF	1	Conservative management
Post-operative acute pancreatitis grade A	1	Conservative management
Renal insufficiency	1	Conservative management
Delayed gastric emptying	2	May require parenteral nutrition
Grade B POPF	2	May require antibiotics
Post-operative acute pancreatitis grade B	2	May require parenteral nutrition
Pneumonia	2	Antibiotics
Sepsis	2	Antibiotics
DVT requiring therapy	2	Anticoagulant
Pulmonary embolism	2	Anticoagulant
Transfusion intraoperative or post-operative	2	Transfusion
Urinary tract infection	2	Antibiotics
Organ space SSI	3A	May require drainage
Grade C POPF	3A	May require drainage
Post-operative acute pancreatitis grade C	3A
Wound dehiscence	3B	Require reoperation
Return to OR	3B	Reoperation
Cardiac arrest	4A	Classified as organ failure
CVA	4A	May require intensive care
Myocardial infarction	4A	Classified as organ failure
Acute renal failure	4A	Classified as organ failure
Unplanned intubation	4A	Classified as organ failure
Persistent intubation >48 h	4A	Classified as organ failure
Septic shock	4B	Often multiorgan failure
Mortality	5	Mortality
Assessment for hepatic resection
Superficial SSI	1	Bedside wound exploration
Deep incision SSI	1	Bedside wound exploration
Readmission	1	Conservative management
Renal insufficiency	1	Conservative management
Bile leakage—spontaneous wound drainage	2	Antibiotics
Bile leakage—drain maintained after post-operative day 3	2	Antibiotics
Liver failure Grade B	2
Pneumonia	2	Antibiotics
Sepsis	2	Antibiotics
DVT requiring therapy	2	Anticoagulant
Pulmonary embolism	2	Anticoagulant
Transfusion intraoperative or post-operative	2	Transfusion
Urinary tract infection	2	Antibiotics
Organ space SSI	3A	May require drain
Bile leakage requiring percutaneous drain	3A	Require drain
Wound dehiscence	3B	Require reoperation
Return to OR	3B	Reoperation
CVA	4A	May require intensive care
Cardiac arrest	4A	Classified as organ failure
Myocardial infarction	4A	Classified as organ failure
Acute renal failure	4A	Classified as organ failure
Unplanned intubation	4A	Classified as organ failure
Persistent intubation >48 h	4A	Classified as organ failure
Liver failure Grade C	4B	Often multiorgan failure
Septic shock	4B	Often multiorgan failure
Mortality	5	Mortality

with associated CDCs. In cases where complication treatments were provided (i.e., POPF, transfusion, post-operative acute pancreatitis), the exact CDC score could be applied [3]. Following the determination of the CDC, the CCI could be calculated for each patient as previously described [4]. The model was validated by having an additional author compare their suggested CDC grade with discrepancies resolved in mutual discussion. The coding for this analysis is available for PD and HR in Supplementary Material S1 and S2, respectively.

2.4. Statistical Analysis

Due to large sample sizes, data was assumed to be normally distributed and was presented as mean ± standard deviation for continuous variables, and differences between groups were evaluated using regression. Categorical variables were presented as counts and percentages, with differences analyzed using chi-squared tests. Significance was considered as p < 0.05. Inter-rater reliability of the CDC grading model was determined with weighted Kappa’s coefficient with a linear weighting system assigned for the CDC grades 1, 2, 3A, 3B, 4A, 4B, and 5.

3. Results

3.1. Demographic Variables

3.1.1. Pancreaticoduodenectomy

We evaluated 20,575 patients undergoing PD, of whom 71.3% had pancreatic ductal adenocarcinoma (PDAC), 21.2% had ampullary/duodenal malignancy, and 7.5% had cholangiocarcinoma (CC;

Table 2. Demographics of patients undergoing pancreaticoduodenectomy evaluated within the NSQIP and comparator benchmark study.

	NSQIP n = 20,575 n (%)	RAW n = 1348 n (%)	p-Value
Diagnosis			<0.001
PDAC	14,659 (71.3)	792 (58.8)
AA	4371 (21.2)	364 (27.0)
CC	1545 (7.5)	192 (14.2)
Demographics
Age, years (mean ± sd)	66.7 (10.6)	66.0 (9.8)	0.018
Female	9530 (46.3)	587 (42.4)	0.047
BMI in kg/m2 (mean ± sd)	27.2 (5.6)	25.5 (4.4)	<0.001
Functional status			-
Independent	20,344 (98.9)
Partially dependent	193 (0.9)	-
Totally dependent	10 (0.1)
ASA category			-
1	50 (0.2)
2	3449 (16.8)	-
3	15,221 (74.0)
4	1838 (8.9)
5	0 (0)
ASA > 2	17,059 (83.0)	467 (33.7)	<0.001
Smoker	3146 (15.3)	-	-
Comorbidities
Diabetes	5927 (28.8)	277 (20.6)	<0.001
Severe COPD	792 (3.9)	-	-
CHF	174 (0.9)	-	-
Hypertension	11,023 (53.6)	-	-
Sepsis		-	-
None	20,263 (98.5)
SIRS	226 (1.1)
Sepsis	81 (0.4)
Dialysis-dependent	59 (0.3)	-	-
Chronic steroid use	663 (3.2)	-	-
Bleeding disorder	645 (3.1)	-	-
Disease and Operative Factors
Weight loss > 10%	2494 (15.3)	-	-
Neoadjuvant	5651 (33.6)	61 (4.6)	<0.001
Soft pancreas	5240 (39.8)	-	-
Preoperative biliary stent	10,469 (64.4)	875 (63.3)	0.681
Concomitant venous reconstruction	2664 (13.0)	205 (15.5)	0.017
Concomitant arterial reconstruction	863 (4.2)	25 (1.9)	<0.001
Minimally invasive surgical approach	4009 (14.3)	-	-

Abbreviations: AA, ampullary adenocarcinoma; ASA, American Society of Anesthesiologists; BMI, body mass index; CHF, congestive heart failure; COPD, chronic obstructive pulmonary disease; PDAC, pancreatic ductal adenocarcinoma; SIRS, systemic inflammatory response syndrome. The RAW study represents the comparator study by Russell et al. (2023) [17].

). Patients undergoing PD were compared to the Russell et al. (2023) study that evaluated 1348 patients from the Recurrence After Whipple’s (RAW) study and had a higher portion of patients with PDAC (Table 2) [11,17]. Patients in our analysis and the comparator had statistically different but clinically similar age, proportion female, and BMI (Table 2). Otherwise, patients analyzed in this study were more likely to be ASA > 2 (83.0% vs. 33.7%; p < 0.001) and had a higher rate of diabetes (28.8% vs. 20.6%; p < 0.001). Oncologically, patients in our study were more likely to receive neoadjuvant therapy, less likely to require venous resection, and more likely to require arterial resection. Rates of preoperative biliary stenting were similar (64.4% vs. 63.3%; p = 0.681). Our analysis also evaluated several demographic characteristics not included in the comparator study. Most patients in our analysis were functionally independent non-smokers, and few patients had severe COPD or CHF, dialysis dependence, or chronic steroid use (Table 2). Additionally, over half of patients had hypertension, and 39.8% were found to have a soft pancreatic texture during their operation. There were 4009 (14.3%) patients in the NSQIP analysis who received an MIS approach, and this was not described in the comparator study. Kappa’s weighted coefficient was 0.95, suggesting strong agreement for CDC grading for the model.

3.1.2. Hepatic Resection

We also evaluated 29,809 patients undergoing HR, where most patients with a diagnosis listed had colorectal cancer metastases (30.8%), followed by hepatocellular carcinoma and cholangiocarcinoma (

Table 3. Demographics of patients undergoing hepatic resection evaluated within the NSQIP and comparator benchmark study.

	NSQIP n = 29,809 n (%)	LiverGroup n = 2159 n (%)	p-Value *
Diagnosis			<0.001
Cholangiocarcinoma	3192 (10.7)	274 (12.7)
Hepatocellular carcinoma	4825 (16.2)	410 (17.0)
Colorectal liver metastases	9180 (30.8)	1070 (43.0)
Other/Not reported	12,612 (42.3)	405 (18.8)
Malignancy	17,197 (57.7)	2050 (84.0)	<0.001
Surgery
Wedge resection	20,823 (69.9)	873 (37)
Left hepatectomy	2575 (8.6)	228 (10)	<0.001
Right hepatectomy	4040 (13.6)	372 (16.0)
Trisegmentectomy	2371 (7.9)	25 (1)
Demographics
Age, years (mean ± sd)	59.9 (13.7)	63.0 (12.6) *	<0.001
Female	14,559 (48.8)
BMI in kg/m2 (mean ± sd)	28.7 (6.3)	26.0 (4.7) *	<0.001
Functional Status			-
Independent	29,472 (98.9) 243
Partially dependent	243 (0.8)	-
Totally dependent	16 (0.1)
ASA Category			-
1	252 (0.9)
2	6635 (22.3)	-
3	20,597 (69.2)
4	2263 (7.6)
5	0 (0)
Smoker	4175 (14.0)	-	-
Comorbidities
Diabetes	5673 (19.0)	277 (20.6)	<0.001
Severe COPD	1028 (3.5)	170 (7)	<0.001
CHF	203 (0.7)	-	-
Hypertension	14,061 (47.2)	-	-
Sepsis		-	-
None	29,352 (98.5)
SIRS	305 (1.0)
Sepsis	128 (0.4)
Dialysis-dependent	120 (0.4)	-	-
Chronic steroid use	1110 (3.7)	-	-
Bleeding disorder	1013 (3.4)	-	-
Operative and Disease Factors
Weight loss > 10%	830 (3.5)	-	-
Tumor size
<2 cm	3371 (11.3)	-
2–5 cm	6297 (21.1)	-
>5 cm	4650 (15.6)	-
Not reported	15,491 (52.0)	-
Neoadjuvant	7407 (30.6)	61 (4.6)	<0.001
Biliary reconstruction	1477 (5.0)	81 (4)	0.018
Minimally invasive surgical approach	5271 (25.5)	597 (25.1)	0.709

Abbreviations: ASA, American Society of Anesthesiologists; BMI, body mass index; CHF, congestive heart failure; COPD, chronic obstructive pulmonary disease; SIRS, systemic inflammatory response syndrome. * Data provided as median with interquartile range and mean and standard deviation were estimated according to Wan et al. (2014) [23]. The LiverGroup study represents the comparator study by The LiverGroup.org Collaborative [18].

). Within our NSQIP-analyzed patients, 57.7% received liver resection for malignancy, with most (69.9%) having wedge resection, followed by right hepatectomy, left hepatectomy, and trisegmentectomy (Table 3). Patients were aged 59.9 with a BMI of 28.7, and 48.8% were female. Nearly all patients were functionally independent, and a complete evaluation of their comorbidities can be found in Table 3. Patients undergoing HR were compared to the LiverGroup Collaborative study [18], which evaluated 2159 patients. The rate of procedures performed using a minimally invasive approach was similar between groups (25.5% vs. 25.1%; p = 0.709). Otherwise, most demographics, including diagnoses, surgeries performed, comorbidities, and oncologic parameters, including receipt of neoadjuvant therapy, were different between groups (Table 3). Kappa’s weighted coefficient was 1.00, suggesting strong agreement for CDC grading for the model (both raters classified complications similarly).

3.2. Post-Operative Outcomes Bivariate Analysis

3.2.1. Pancreaticoduodenectomy

Upon evaluating 30-day outcomes following PD, most complications were found to be significantly different between the NSQIP patients analyzed in our study and the comparator patients from the Russell et al. study (Figure 1). Notably, length of stay and operative duration were significantly shorter in our comparator study. Evaluating CDC ≥ 3 outcomes, we estimated a 23.2% rate with our model, which was significantly lower than the reported 27.6% in the study by Russell et al. [17]. The study by Russell et al. (2023) had higher complication rates for every CDC grade compared to our estimates using NSQIP data (

Table 4. Thirty-day post-operative outcomes for patients undergoing pancreaticoduodenectomy comparing patients within the NSQIP database and the RAW study.

	NSQIP n = 20,575 n (%)	RAW n = 1348 n (%)	p-Value
Outcomes and Complications
Operative duration in minutes (mean ± sd)	378.3 (131.6)	204.6 (126.1)	<0.001
Length of stay in days (mean ± sd)	9.6 (5.9)	4.5 (3.9)	<0.001
Length of stay > 30 days	644 (3.1)	-	-
Readmission	3316 (16.1)	134 (10.0)	<0.001
Delayed gastric emptying	2710 (16.0)	167 (12.4)	<0.001
Superficial SSI	1400 (6.8)	115 (8.5)	0.015
Deep SSI	182 (0.9)	-	-
Organ space SSI	3347 (16.3)	179 (13.3)	0.004
Wound disruption	268 (1.3)	-	-
Sepsis	1970 (9.6)	-	-
Septic shock	631 (3.1)	-	-
Pneumonia	793 (3.9)	96 (7.1)	<0.001
Unplanned intubation	670 (3.3)	-	-
Persistent intubation > 48 h	523 (2.5)	-	-
DVT	574 (2.8)	25 (1.8)	<0.001
Pulmonary embolism	235 (1.1)	15 (1.1)	0.922
Acute renal failure	223 (1.1)	-	-
Acute kidney injury	102 (0.6)	33 (2.4)	<0.001
Urinary tract infection	462 (2.3)	20 (1.5)	0.065
Cerebral vascular accidents	58 (0.3)	-	-
Myocardial infarction	267 (1.3)	3 (0.2)	<0.001
Cardiac arrest	259 (1.3)	-	-
Bleed	4115 (20.0)	248 (18.4)	0.153
Unplanned reoperation	1140 (5.5)	74 (5.5)	0.937
Surgery-Specific Complications
POPF
A	757 (4.5)	102 (7.6)
B	2050 (12.2)	85 (6.3)	<0.001
C	179 (1.1)	23 (1.7)
POPF (any grade)	2986 (17.8)	108 (15.6)	0.037
Post-operative acute pancreatitis
A	6720 (95.3)	-	-
B	66 (0.9)
C	145 (2.1)
Postoperative Morbidity
CDC ≥ 3	4770 (23.2)	373 (27.6)	<0.001
Comprehensive Complication Index (mean ± sd)	25.1 (26.3)	-	-
Clavien–Dindo Complications			<0.001
1	4696 (22.8)	328 (24.5)
2	4768 (23.2)	640 (47.8)
3A	2506 (12.2)	142 (10.6)
3B	800 (3.9)	98 (7.3)
4A	583 (2.8)	59 (4.4)
4B	467 (2.3)	20 (1.5)
5	414 (2.0)	53 (4.0)

Abbreviations: CDC, Clavien–Dindo Classification; DVT, deep vein thromboembolism; MI, myocardial infarction; POPF, post-operative pancreatic fistula; SSI, surgical site infection. The RAW study represents the comparator study by Russell et al. (2023) [17].

) [11,17].

3.2.2. Hepatic Resection

With regard to a comparison of outcomes using the NSQIP to the study by the Liver Group Collaborative, significant differences were noted across most measured domains (Figure 2). However, despite statistical differences, operative time (240.1 vs. 223.3 min), readmission rates (9.8% vs. 10.0%), and post-operative bile leaks (5.1% vs. 8.4%) were clinically similar between groups. Additionally, rates of post-hepatectomy liver failure and reoperation were similar between groups (

Table 5. Thirty-day post-operative outcomes for patients undergoing hepatectomy comparing patients within the NSQIP database and the LiverGroup study.

	NSQIP n = 29,809 n (%)	LiverGroup n = 2159 n (%)	p-Value
Outcomes and Complications
Operative duration in minutes (mean ± sd)	240.1 (124.8)	223.3 (125.9) *	<0.001
Length of stay in days (mean ± sd)	5.8 (4.6)	8.0 (4.4) *	<0.001
Length of stay > 30 days	343 (1.2)	-	-
Readmission	2907 (9.8)	240 (10.0)	0.040
Superficial SSI	907 (3.0)	-
Deep SSI	90 (0.3)	-	-
Organ space SSI	2204 (7.4)	-
Wound disruption	165 (0.6)	-	-
Sepsis	1022 (3.4)	-	-
Septic shock	473 (1.6)	-	-
Pneumonia	901 (3.0)	-
Urinary tract infection	514 (1.7)	-
Any infection	6276 (21.1)	114 (5.2)	<0.001
Unplanned intubation	584 (2.0)	-	-
Persistent intubation > 48 h	515 (1.7)	-	-
DVT	471 (1.6)	-
Pulmonary embolism	309 (1.0)	-
Acute renal failure	260 (0.9)	-	-
Acute kidney injury	165 (0.7)
Urinary tract infection	514 (1.7)
Cerebral vascular accidents	55 (0.2)		-
Myocardial infarction	265 (0.9)
Cardiac arrest	213 (0.7)		-
Bleed	5025 (16.9)	146 (6.8)	<0.001
Unplanned reoperation	918 (3.1)	74 (5.5)	0.368
Surgery-Specific Complications
Post-hepatectomy liver failure
A	370 (1.5)	42 (2)	0.106
B	378 (1.6)	25 (1)
C	267 (1.1)	22 (1)
Any post-operative bile leak	1518 (5.1)	182 (8.4)	<0.001
Bile leak with wound drainage or drain maintenance	798 (2.7)	-	-
Bile leak requiring percutaneous drainage	720 (2.4)	-	-
Postoperative Morbidity
CDC ≥ 3	3875 (13.0)	341 (15.8)	<0.001
Comprehensive Complication Index (mean ± sd)	13.6 (23.9)	-	-
Clavien–Dindo Complications			<0.001
1	1366 (4.6)	214 (9.0)
2	4609 (15.5)	420 (18.0)
3A	1842 (6.2)	185 (8.0)
3B	634 (2.1)	86 (4.0)
4A	549 (1.9)	42 (2)
4B	408 (1.4)	10 (1)
5	442 (1.5)	46 (2)

Abbreviations: CDC, Clavien–Dindo Classification; DVT, deep vein thromboembolism; MI, myocardial infarction; SSI, surgical site infection. * Data provided as median with interquartile range and mean and standard deviation were estimated according to Wan et al. (2014) [23]. The LiverGroup study represents the comparator study by The LiverGroup.org Collaborative [18].

). With regard to CDC ≥ 3 complications, rates were clinically similar (13.0% vs. 15.8%) but statistically different (p < 0.001). Additionally, this NSQIP cohort had lower rates for nearly all CDC grades (Table 5).

4. Discussion

This study presents a method to translate NSQIP complications to CDC grading. Compared to historical controls, the NSQIP cohorts had significant outcome differences, including different rates of CDC ≥ 3 complications, but in the context of notable demographic differences. However, the generalizability of this approach to the complete body of literature is limited and warrants ongoing evaluation. The intention of this study is to describe a model to estimate NSQIP complication severity according to the CDC grading system to allow for aggregate outcome measurement in future NSQIP studies.

With regard to this study’s primary outcome, the rate of estimated CDC ≥ 3 complications from NSQIP data is statistically different than reported rates in our comparator studies for PD and HR [17,18]. However, for PD, other large contemporary studies have demonstrated rates of CDC ≥ 3 that are both higher and lower than our evaluated cohort, ranging from 15.9 to 28.0% [6,10,11,17,24,25,26]. Similarly, for HR, rates of CDC ≥ 3 complications range from 11.6 to 16.0% [13,18,27,28]. Our rates for CDC ≥ 3 complications after PD (23.2%) and HR (13.0%) fall near the middle of these ranges. A strength of our estimation model included that many of the complications that were designated as CDC ≥ 3 were directly measured within the NSQIP; for example, reoperation, septic shock, acute renal failure, and mortality are directly measured by the NSQIP and can be accurately graded according to CDC. Additionally, pancreas and hepatic outcomes, including POPF, post-operative pancreatitis, and post-operative liver failure, are evaluated with defined severity grading or with their associated intervention within NSQIP and allow for accurate CDC ≥ 3 determination. Thus, there is support from the literature that estimating CDC from NSQIP data may be feasible and thus could be used for future contemporary studies.

Future studies could consider a repeated measure study design to compare the estimated and measured CDCs to better assess our estimation method. Similar approaches have previously been used to validate estimation of aggregate comorbidity scoring using ASA scoring [29]. Importantly, it should not be assumed that multi-source estimation is inferior to direct measurement; in fact, more recent studies of using estimated ASA have suggested that it may be more accurate than the inputted ASA score within the NSQIP due to bias towards higher scoring by anesthetists in certain scenarios. Similarly, imputation of albumin using multiple NSQIP data points has been demonstrated to be robust and provides additional evidence to support estimation of certain outcomes from the included data within the NSQIP. Studies such as those previously conducted to validate ASA and albumin estimation may be able to validate this CDC estimation technique or be utilized to improve NSQIP data collection by modifying specific definitions.

By comparing the NSQIP cohort CDC grading to benchmark studies in the field, we are also able to gain additional value from the individual complications and outcomes obtained from both cohorts. Russell et al. (Table 2 of their study) detail specific CDC grades for each complication in their study—their data support that complications such as UTI (95% grade II), DVT (83% grade II), sepsis (70% grade II), and PE (66% grade II) have predictable CDCs and can be reasonably estimated. On the other hand, complications such as DGE (88% grade I-II), pneumonia (85% grade I-II), and SSI (95% grade I-II) are variably categorized as CDC I or II, and estimating them as a single grade may contribute to inaccuracy when calculating minor complications. However, it should also be noted that studies report rates of CDCs I-II from 29.6 to 76.8% following pancreaticoduodenectomy and 14.4–30% following hepatic resection, highlighting a more widespread issue with evaluation of low-grade complications [6,15,17,18,24,30]. As such, when using this estimation method, categorization of CDCs as I and II combined may be more accurate. Even then, the NSQIP likely underevaluates low-grade complications since it does not currently capture outcomes including ileus (95% grade I-II), chyle leak (87% grade I-II), and cardiac arrhythmia (84% grade I-II). Consideration for including these outcomes within the NSQIP could be discussed in the future. Given that multiple CDCs grade I or II can pose a similar effect on the CCI as a single higher-grade complication, this remains a limitation in the presented analysis. Our method of estimating CDCs likely has a higher risk of inaccurately categorizing low-grade complications, and as such, may underestimate the CCI as an aggregate outcome.

As seen from the results, there may be significant value in the NSQIP complication data beyond comparisons with center-specific quality metrics; its use in estimating and measuring CDCs can be of value to clinicians and health care policy. As discussed above, CDC is an aggregate measure of morbidity that has been shown to be associated with quality of life, oncologic outcomes, and long-term outcomes. Furthermore, more recent studies have demonstrated the value of the comprehensive complication index (CCI) as a continuous aggregate measure of morbidity that can be measured from CDCs [6,7,8]. Finally, an additional benefit of accurately evaluating CDC ≥ 3 complications is that it allows users to evaluate ideal outcomes after PD and HR. Ideal outcomes for both PD and HR have been established through consensus as targets for high-functioning centers to achieve and include that patients should not have any CDC ≥ 3 complications; other measures of ideal outcomes are already captured by the NSQIP and would allow further aggregate assessment of outcomes [26,27]. Overall, describing a method to meaningfully measure aggregate morbidity following HPB surgery from NSQIP data may allow future studies an outcome measure that has been associated with oncologic and long-term outcomes. Given the extensive number of procedures captured by the NSQIP, it would be useful to study how well this model, after being adjusted for procedure, would predict morbidity in other high-risk procedures; currently the model is not validated beyond pancreatic and liver surgery or beyond oncologic surgery. The results from this study build a foundation on which administrative data from large databases can be assessed for cumulative markers of morbidity that can help with prognostication and identification of patients who may be at higher risk. Future studies would benefit from prospective validation of such metrics, as well as further assessing if the CCI measured from NSQIP data is similar to what is observed in the literature.

Despite these considerations, important limitations of this study should be recognized. Firstly, although outcomes collected by NSQIP are conducted by trained nurse data collectors and audited regularly, it is possible that input error exists, which may limit the accuracy of estimating CDCs from NSQIP outcomes. Additionally, even if this method of estimating CDCs were to be validated in the future, it would only represent an extrapolation of data for longer-term outcomes and should be evaluated carefully. Furthermore, a significant limitation of this study, as discussed above, is the demographic differences noted between our NSQIP patient cohort and the comparator study’s. As well, there may be changes in perioperative practices both temporally (different years of cohorts) and from center to center that could impact the results seen. Overall, this study should be highlighted as an exploratory analysis with further evaluation required in follow-up studies.

Despite these limitations, this is the first method reported to estimate aggregate morbidity from NSQIP data. Future studies to validate or modify this estimation method are warranted and may allow extrapolation of short-term NSQIP measures to oncologic, quality of life, and long-term outcomes. Alternatively, discussion to integrate a measure of complication severity within large databases such as the NSQIP may be warranted to better understand morbidity for each patient. The role of machine learning may also enhance low-grade complication detection from dense administrative data and should be considered in future models.

5. Conclusions

This study offers a method to estimate CDC as an aggregate measure of morbidity using NSQIP data. Compared to specific historical controls, demographics and outcomes are statistically different; however, within the context of available literature, this method to estimate serious complications (CDC ≥ 3) may be reasonable, and future validation studies are warranted. A validated model to grade CDCs would allow NSQIP studies to estimate long-term or oncologic outcomes.