Do Publicly Available Risk Calculators Apply to Adult Spinal Deformity Surgery?
by
, , , , , ,
Anthony Pajak
1,2,
Justin T. Samuel
1,
Tejas Subramanian
1,3,
Robert K. Merrill
1,
Izzet Akosman
1,3,
John C. Clohisy
1,
Jerry Du
1,
Bo Zhang
1,
Jonathan Elysee
1,
Pratyush Shahi
1,
David N. Kim
1,
Yusef Jordan
1,
Rachel L. Knopp
1,
Francis C. Lovecchio
1 and
Han Jo Kim
1,*
1
Department of Spine Surgery, Hospital for Special Surgery, 535 E. 71st Street, New York, NY 10021, USA
2
UMass Chan Medical School, 55 N Lake Avenue, Worcester, MA 01655, USA
3
Weill Cornell Medical Center, 525 East 68th Street, New York, NY 10065, USA
*
Author to whom correspondence should be addressed.
J. Clin. Med. 2025, 14(24), 8618; https://doi.org/10.3390/jcm14248618
Submission received: 15 July 2025
/
Revised: 2 September 2025
/
Accepted: 28 September 2025
/
Published: 5 December 2025
(This article belongs to the Special Issue Spine Surgery: Clinical Advances and Future Directions)
Abstract
Background/Objectives: The American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) and SpineSageTM risk calculators are automated online tools to predict short-term complications for surgical procedures. The objective of this study was to assess the validity of ACS-NSQIP and SpineSageTM risk calculators to predict short-term complications after adult spinal deformity (ASD) surgery. Methods: We included ASD patients who had surgery between 2017 and 2020 (≥5 levels, single-stage, posterior-only). Patient factors were entered into the risk calculators to generate probabilities for 30-day outcomes. Calibration and discrimination were assessed using Brier scores and C-statistics, respectively. Results: A total of 198 patients were included (67 male, 131 female) who underwent posterior spinal fusion for ASD surgery. The ACS-NSQIP risk calculator had strong calibration for all complications (Brier score < 0.09) except non-home discharge (Brier score 0.2). Discrimination was poor for all complications except surgical site infection (C-statistic 0.86), venous thromboembolism (C-statistic 0.84), and readmission (C-statistic 0.7). The SpineSageTM risk calculator had strong calibration for all complications (Brier score < 0.09) aside from the “any complications” subset (Brier score 0.36). The discrimination capacity was poor for all complications (C-statistic < 0.7). Conclusions: The ACS-NSQIP calculator had strong calibration and poor discrimination for most complications. The SpineSageTM calculator had strong calibration for most complications and a poor discrimination capacity for all complications. NSQIP calculation deficits may be due to the reliance on a single CPT code to calculate risk. The deficient discriminatory capacity of the SpineSageTM calculator may be due to the inclusion of common perioperative occurrences as complications.
1. Introduction
Shared decision-making has become a cornerstone of patient care throughout all specialties in medicine, consistently demonstrating improved patient centered outcomes and satisfaction [1,2]. To effectively engage in shared decision-making, the surgeon and patient must adequately understand the risks associated with undergoing a procedure or treatment. The risk of complications following adult spinal deformity (ASD) surgery can be as high as 67.4% [3]. Patient age, comorbidities, and surgical invasiveness have been reported to increase these risks [4,5,6,7]. In addition to guiding presurgical counseling, surgical risk calculators may also help foster a cost-conscious healthcare landscape through the stratification of high-risk ASD patients [8]. Various automated, publicly available online risk calculators aim to use patient demographic and surgical characteristics to estimate the risk of developing postoperative complications. However, a 2024 systematic review highlighted the lack of externally validated risk stratification models specific for spine surgery [9]. Two such tools are the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) and SpineSageTM risk calculators.
The ACS-NSQIP risk calculator has been deemed an adequate predictive tool for a subset of complications following anterior lumbar interbody fusion (ALIF) and primary cervical fusion procedures [10,11]. However, the calculator was deficient in predicting specific complications in a more granular manner. Few investigations have validated this tool for ASD corrective surgery [12]. The SpineSageTM risk calculator is specific to spine surgery and has been shown to more accurately predict the risk of serious medical complications in comparison to the ACS-NSQIP risk calculator following single-stage spine surgery [13]. However, minimal literature exists evaluating its ability to predict complications following surgery for ASD [14]. Accordingly, the primary outcome of this study is to validate the ability of the ACS-NSQIP and SpineSageTM risk calculators to predict short term complications after ASD surgery by a single surgeon.
2. Materials and Methods
2.1. Patient Population and Data Collection
This was a single center retrospective cohort study approved by the Institutional Review Board (IRB). Patients were retrospectively queried from a prospectively maintained single surgeon registry. The patient inclusion criteria were as follows:
Inclusion criteria:
- Adult patients (age > 18 years) with spinal deformity.
- Patients undergoing primary, posterior-only, single-stage fusion procedures.
- Fusion involving more than five levels, with or without extension to the pelvis.
- Surgery performed between 1 January 2017 and 31 December 2020.
Exclusion criteria:
- Age < 18 years.
- Neuromuscular, infectious, or post-traumatic deformities.
- Incomplete preoperative demographic and comorbidity data.
- Insufficient 30-day postoperative follow-up to assess complications of interest.
- Concurrent surgeries outside the spine.
The following subsections demonstrate the patient data that was collected in order to perform risk calculations utilizing the 2022 versions of the ACS-NSQIP and SpineSageTM calculators. The data was collected and managed using REDCap (Research Electronic Data Capture) [15] hosted at Weill Cornell Medicine Clinical and Translational Science Center supported by the National Center for Advancing Translational Science of the National Institute of Health under award number UL1 TR002384:
- Demographic details: age, gender, body mass index (BMI), American Society of Anesthesiologists (ASA) class, Charlson Comorbidity Index (CCI), and any individual comorbidities (hypertension, diabetes, kidney disease, heart disease, liver disease, vascular disease, autoimmune disease, rheumatological disease, dyslipidemia, cancer, peptic ulcer disease, osteoarthritis/osteoporosis, peripheral neuropathy, bleeding/hypercoagulable disorder, anxiety/depression).
- Operative details: number of levels fused, 3-column osteotomy, operative time, estimated blood loss (EBL).
- Peri and postoperative details: length of stay, any complications (pneumonia, cardiac, surgical site infection, urinary tract infection, sepsis, venous thromboembolism, renal failure, blood loss requiring transfusion, dural tear), readmission, reoperation, non-home discharge.
2.2. Risk Calculators Data Collection
Patient characteristics and demographics were entered into each risk calculator. CPT codes for the ACS-NSQIP calculator were selected to best match the type and attributes of the surgical procedures performed. The CPT codes included 22843 (posterior instrumentation 7–12 levels), 22844 (posterior instrumentation 13+ levels), 22206 (thoracic three column osteotomy), and 22207 (lumbar three column osteotomy), all used when appropriate. Sample output report sheets of the ACS-NSQIP and SpineSageTM risk calculators are displayed in Figure 1 and Figure 2, respectively. Risk profiles were recorded for each patient. The SpineSageTM risk calculator also attempts to capture surgical complexity by categorizing the risk profile based on a surgical invasiveness score developed by Mirza et al. [16]. We calculated this invasiveness score and used the appropriate risk profile.
![Jcm 14 08618 g001]( "Jcm 14 08618 g001")
![Jcm 14 08618 g002]( "Jcm 14 08618 g002")
Figure 1.
Sample outputs of the ACS-NSQIP surgical risk calculator report screen. Estimated risks for 11 complication categories, length of stay, and discharge to a nursing or rehab facility are displayed following the input of the CPT code for the planned procedure and patient specific variables.
Figure 1.
Sample outputs of the ACS-NSQIP surgical risk calculator report screen. Estimated risks for 11 complication categories, length of stay, and discharge to a nursing or rehab facility are displayed following the input of the CPT code for the planned procedure and patient specific variables.
Figure 2.
Sample outputs of the SpineSageTM surgical risk calculator report screen. Estimated risks for major complication, all complications, infection, and dural tear are displayed based on surgical invasiveness scores and patient specific variables.
Figure 2.
Sample outputs of the SpineSageTM surgical risk calculator report screen. Estimated risks for major complication, all complications, infection, and dural tear are displayed based on surgical invasiveness scores and patient specific variables.
2.3. Statistical Analysis
All statistical analyses were performed using Posit Team (2023). RStudio: Integrated Development Environment for R. Posit Software version 2023.03.0, PBC, Boston, MA. URL http://www.posit.co/ (accessed on 14 July 2025). The risk calculators were evaluated for calibration and discrimination ability using the Brier score and C-statistic, respectively. Briefly, the Brier score is used to determine the difference between the predicted outcome and the actual outcome for each individual observation, with a perfect score being 0 and the worst possible score being 1 [17]. The c-statistic measures discrimination and is also known as the area under the curve (AUC) of the receiver operating characteristic curve (ROC). Briefly, discrimination is the chance that a random participant who did in fact develop the outcome (complication) had a higher risk probability than a participant who did not develop the outcome of interest. There are various interpretations of the output, but generally a value of greater than or equal to 0.7 is considered acceptable [18].
We also divided patients into cohorts of those who experienced a complication as defined by each risk calculator and those who did not and compared various demographics that were input into the risk calculators. The definition of a complication for the NSQIP risk calculator can be found on the website (https://riskcalculator.facs.org/RiskCalculator/ (accessed on 14 July 2025)). The definitions of medical complications of the SpineSageTM risk calculator are in accordance with those reported by Lee et al. [14]. A summary of the ACS-NSQIP and SpineSageTM specific complication and outcome of interest definitions is listed in Table 1.
All continuous variables were assessed for normality with a Shapiro–Wilk’s normality test. Normally distributed continuous variables were compared with a two-tailed independent sample t-test, while non-normally distributed variables were compared with a Wilcoxon rank sum test. Categorical variables were compared between groups with a chi-square test or a Fisher’s exact test when applicable. Statistical significance was taken at p-value < 0.05.
3. Results
3.1. Patient Cohort Demographics and Complication Profiles
A total of 198 patients met the inclusion criteria, comprising 67 (34%) males and 131 (66%) females (Table 2). The mean age was 61.0 ± 15.68 years, and the mean BMI was 27 ± 6.07. Most patients were classified as ASA class II (69%) or ASA class III (30%) with a mean CCI of 2.3 ± 1.61 (Table 2). Patients had an average of 9 ± 3.26 levels fused, with 26% undergoing a three-column osteotomy. The mean operative time was 260 ± 81.88 min, and the mean estimated blood loss was 1.1 ± 0.80 L.
Patient demographics and the ACS-NSQIP-defined complication profiles are summarized in Table 2. Although the ACS-NSQIP and SpineSageTM calculators apply different thresholds for defining complications, the demographic patterns of patients with and without complications were broadly comparable. Across both definitions, operative time, estimated blood loss (EBL), number of levels fused, and length of stay were all higher for those that had complications. The ASA class was significantly higher for those that had complications under the ACS-NSQIP definition, while age, hypertension, and lung disease were significantly higher in those who had complications under the SpineSageTM definition. Patient demographics and the SpineSageTM defined complication profiles are summarized in Table 3.
3.2. ACS-NSQIP Calculator
Brier scores for overall, overall serious complications, pneumonia, cardiac complications, surgical site infection (SSI), urinary tract infection (UTI), venous thromboembolism (VTE), renal failure, readmission, re-operation, and death were calculated. All demonstrated good calibration in the prediction of event risk utilizing the ACS-NSQIP calculator. Non-home discharge was the single variable for which the ACS-NSQIP Brier score demonstrated less calibration (Brier score 0.2). Discriminatory performance was poor (<0.7) for all outcomes except SSI, VTE, and readmission (Table 4). The median predicted length of stay (5 days ± 1.5) was statistically shorter than the observed length of stay (5.4 ± 3) (p = 0.03).
3.3. SpineSageTM Calculator
The SpineSageTM demonstrated proficient calibration for predicting serious complications, infection (Brier score 0.02 for both), and dural tear (Brier score 0.1). However, the discriminatory performance was poor for all complications (C-statistic < 0.7). Moreover, the predicted event rates differed substantially from observed incidences, with the exception of dural tear (Table 5).
4. Discussion
Understanding the risk patients face when undergoing certain procedures and treatments is integral to the shared decision-making process. Accurate predictive models can provide both surgeons and patients with a clearer understanding of potential complications. In this retrospective review, we evaluated the performance of the ASC-NSQIP and SpineSageTM risk calculators to predict short term complications after adult spinal deformity surgery. The ACS-NSQIP risk calculator had good accuracy, also termed calibration, as determined by the Brier score for all complications (Brier < 0.1) except for non-home discharge (Brier = 0.2). In terms of discrimination, also known as the area under the ROC, the calculator performed well (C-statistic ≥ 0.7) only for predicting surgical site infection, venous thromboembolism, and readmission. The SpineSageTM risk calculator demonstrated strong calibration for major complications, SSI, and dural tear (Brier ≤ 0.1), but the discrimination capacity was poor for all predicted outcomes (C-statistic < 0.7).
With regard to adult spinal deformity specifically, only a few studies have investigated the performance of the NSQIP risk calculator [12,19,20]. A study by Pierce et al. assessed the calculator using the Brier score and found that the calculator was accurate (Brier < 0.25) for predicting all the evaluated complications [12]. This supports our results based on Brier score alone. Interestingly, the least accurate outcome predicted in their cohort was non-home discharge (Brier = 0.19), similarly to our cohort (Brier = 0.2). A Brier score of 0.25 is equal to randomness, and so while these results technically suggest better than random calibration, they are not good scores. This is further supported by both Pierce et al. and our results showing that the predicted incidence of non-home discharge is vastly different from the actual incidence of non-home discharge [12]. A 2024 cohort study of ASD patients demonstrated similar findings, with poor prediction of non-home discharge, SSI, reoperation, and overall complications [13]. The predicted and actual incidence in both studies for several of the complications are within one percentage point. This suggests the calculator may be beneficial as a benchmarking tool in counseling patients on the incidence of complications in a population with similar demographics and surgical characteristics as their own. The SpineSageTM has not been extensively studied in adult spinal deformity surgery but has been shown to have good discrimination and superior predictive capacity for major complications in other single stage spinal surgery applications [19,20]. Consistent with our findings, Jadresic et al. studied the SpineSageTM tool on a heterogeneous population and found that the discrimination was poor for detecting major complication (Brier = 0.19), SSI (Brier = 0.08), and dural tear (Brier = 0.14), only being proficient for any complication (Brier = 0.19) [21].
The current literature assessing the validity of the NSQIP and SpineSageTM risk calculators is not ubiquitous. McCarthy et al. evaluated the NSQIP risk calculator for elective cervical and lumbar fusions [11]. They only evaluated the calculator using the area under the ROC (c-statistic) and found in the lumbar fusion cohort it only had good discrimination for predicting pneumonia. It is important to note, though, they set a more stringent cutoff of >0.8. If their cutoff for acceptable performance was set at the same as ours, ≥0.7, the calculator had good discrimination for acute renal failure as well. Both pneumonia and acute renal failure only occurred at a <1% incidence, and therefore the authors appropriately caution the results may not be fully reflective of the model’s discriminatory capacity. Another study by Neassig et al. highlights important limitations of the NSQIP risk calculator when predicting complications for adult spinal deformity patients, namely that it lacks granularity of each patient and procedure [19]. When patients were stratified by frailty, there was a 71% incidence of actual complications but only a 17% predicted incidence, with a Brier score of 0.37. These findings clearly demonstrate the calculator is inaccurate and should be used judiciously in certain patient populations. They saw similar results when stratifying patients by high T1 pelvic angle (TPA) or pelvic incidence–lumbar lordosis (PI–LL), for every complication assessed. These findings demonstrate that the complexity of a surgical procedure simply cannot be captured by the NSQIP risk calculator. Similarly, we found that the NSQIP risk calculator demonstrated acceptable discrimination only for SSI, VTE, and readmission. All of which are outcomes largely influenced by systemic factors rather than the detailed clinical variables that the calculator fails to capture. With respect to SpineSageTM, conflicting findings have been reported as well. Kasparek et al. evaluated the risk calculator on a heterogeneous patient population of cervical, thoracic, and lumbar patients indicated for degenerative, traumatic, infectious, and neoplastic conditions. They reported a predicted (14.7%) and actual (16.1%) incidence of overall complications and a strong discrimination with a c-statistic of 0.71 for all and 0.85 for major complications [22].
Accurate predictive models must demonstrate both excellent calibration and discrimination; adequacy in one or the other domain is insufficient. Few studies have investigated the NSQIP and SpineSageTM tools in elective spine surgery. Future studies should assess the validity of the NSQIP and SpineSageTM risk calculators exclusively in adult spinal deformity patient populations in order to draw fair comparisons. To remediate the reported deficits in discriminatory capacity of the NSQIP tool, future developments must incorporate specific surgical parameters to adequately account for the invasiveness of ASD surgery, thereby improving predictive modeling for patients. The SpineSageTM calculator attempts to overcome the previously mentioned flaws by instituting a spine specific surgical invasiveness score developed by Mirza et al. [16]. This score incorporates the number of levels decompressed and instrumented from both a posterior and anterior approach and is input into the calculator to generate a more accurate risk profile. SpineSageTM has been shown to have good discrimination for more heterogenous cohorts of spine procedures [20,22]. We feel the inaccuracies in SpineSageTM may come from two specific drawbacks. The first is that EBL > 3L and transfusion occurrence are considered complications by the calculator. For corrective spinal deformity procedures, these are not uncommon occurrences. Our cohort had a 62% incidence of transfusions, all of which were technically considered a complication according to the calculator. Additionally, the surgical invasiveness score will be very high for deformity correction procedures, which may overestimate the prediction of complications by the calculator. While deformity procedures are certainly invasive, the magnitude of increase in the invasiveness score simply by instrumenting additional levels may not correlate with an actual increased risk of developing complications. These two points highlight that the calculator may not be suited for adult deformity patients. Future developments should tailor reasonable definitions of complications and invasiveness in accordance with spinal deformity correction.
A major limitation of our study is that it was a single surgeon cohort. Including more surgeons would make the calculator more generalizable and less specific to one person. The study was retrospective in nature, which predisposes it to inherent bias and possible inaccuracy in data collection. Lastly, we did not have an occurrence of certain medical complications, which limited our ability to evaluate the NSQIP calculator with respect to predicting those specific complications.
5. Conclusions
Our results demonstrate that the ACS-NSQIP calculator had strong calibration and poor discrimination for most complications. The SpineSageTM calculator had strong calibration for most complications and a poor discrimination capacity for all complications. For the ACS-NSQIP risk calculator, this likely relates to the use of a single CPT code for procedure input, which underestimates the complexity of adult spinal deformity. While the SpineSageTM risk calculator attempts to capture the complexity of the surgical procedure, it may be set back by including a variety of common occurrences after such procedures in its definition of a complication. Thus, these calculators may be more appropriate as benchmarking tools rather than utilized for individual patient risk counseling.
Author Contributions
Conceptualization, F.C.L., P.S., R.K.M. and H.J.K.; methodology, F.C.L. and R.K.M.; software, R.K.M. and J.D.; validation, J.D.; formal analysis, J.D. and B.Z.; investigation, A.P., J.T.S., I.A., D.N.K. and F.C.L.; resources, R.L.K. and H.J.K.; data curation, A.P., Y.J., J.T.S., T.S., I.A., J.C.C., B.Z., D.N.K. and J.E.; writing—original draft preparation, A.P., J.T.S. and T.S.; writing—review and editing, A.P., P.S., F.C.L. and H.J.K.; visualization, A.P. and R.K.M.; supervision, R.K.M., J.E., R.L.K., F.C.L. and H.J.K.; project administration, J.C.C. and R.L.K.; funding acquisition, F.C.L. and H.J.K. All authors have read and agreed to the published version of the manuscript.
Funding
No direct funding was received for this study. However, the study used REDCap (Research Electronic Data Capture) hosted at the Weill Cornell Medicine Clinical and Translational Science Center supported by the National Center for Advancing Translational Science of the National Institute of Health (NIH) under award number UL1 TR002384.
Institutional Review Board Statement
The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board of HOSPITAL FOR SPECIAL SURGERY (2021-1875). The IRB was approved on 25 October 2021.
Informed Consent Statement
No informed consent was required for this study, as it was all retrospective analysis.
Data Availability Statement
Patient data were compiled from patient charts; the data were then placed into tables for analysis. The data from this study are available upon reasonable request.
Acknowledgments
The authors would like to acknowledge the ACS-NSQIP Surgical Risk Calculator and the SpineSageTM risk prediction platform for providing publicly accessible tools that supported this study’s risk estimation processes. These calculators were used to generate individualized perioperative risk profiles, CPT-based predictions, and surgical invasiveness scoring as described in the Methods section. Screenshots taken from the home pages of both calculators were used, with permission where required, as Figure 1 and Figure 2 to illustrate the interfaces and output formats referenced in this work. These resources contributed only data estimation functions; no external organizations participated in study design, data interpretation, or manuscript preparation. The authors take full responsibility for the accuracy and interpretation of all results derived from these tools.
Conflicts of Interest
The authors declare no conflicts of interest relating to the study. However, these are the following disclosures: Francis Lovecchio—consulting: SeaSpine; consulting: SI—Bone Han Jo Kim—royalties: Acuity Surgical; consultant: Alphatec Spine, Inc.; royalties: Aspen Medical; stock ownership: Globus; consultant: Highridge Medical; royalties: Highridge Medical; ownership interest: HS2, LLC; royalties: K2 Medical, LLC; speakers’ bureau: K2 Medical, LLC; consultant: Mirus; consultant: SI—Bone; ownership interest: SPINE STUD; royalties: Stryker.
References
- Sepucha, K.R.; Vo, H.; Chang, Y.; Dorrwachter, J.M.; Dwyer, M.; Freiberg, A.A.; Talmo, C.T.; Bedair, H. Shared Decision-Making Is Associated with Better Outcomes in Patients with Knee but Not Hip Osteoarthritis: The DECIDE-OA Randomized Study. J. Bone Jt. Surg. Am. 2022, 104, 62–69. [Google Scholar] [CrossRef] [PubMed]
- Wilson, C.D.; Probe, R.A. Shared Decision-making in Orthopaedic Surgery. J. Am. Acad. Orthop. Surg. 2020, 28, e1032–e1041. [Google Scholar] [CrossRef]
- Lafage, R.; Bass, R.D.; Klineberg, E.; Smith, J.S.; Bess, S.; Shaffrey, C.; Burton, D.C.; Kim, H.J.; Eastlack, R.; Mundis, G., Jr.; et al. International Spine Study Group. Complication Rates Following Adult Spinal Deformity Surgery: Evaluation of the Category of Complication and Chronology. Spine 2024, 49, 829–839. [Google Scholar] [CrossRef]
- Akıntürk, N.; Zileli, M.; Yaman, O. Complications of adult spinal deformity surgery: A literature review. J. Craniovertebr. Junction Spine 2022, 13, 17–26. [Google Scholar] [CrossRef]
- Iorio, J.A.; Reid, P.; Kim, H.J. Neurological complications in adult spinal deformity surgery. Curr. Rev. Musculoskelet. Med. 2016, 9, 290–298. [Google Scholar] [CrossRef]
- Soroceanu, A.; Burton, D.C.; Oren, J.H.; Smith, J.S.; Hostin, R.; Shaffrey, C.I.; Akbarnia, B.A.; Ames, C.P.; Errico, T.J.; Bess, S.; et al. Medical Complications After Adult Spinal Deformity Surgery: Incidence, Risk Factors, and Clinical Impact. Spine 2016, 41, 1718–1723. [Google Scholar] [CrossRef]
- Dinizo, M.; Dolgalev, I.; Passias, P.G.; Errico, T.J.; Raman, T. Complications After Adult Spinal Deformity Surgeries: All Are Not Created Equal. Int. J. Spine Surg. 2021, 15, 137–143. [Google Scholar] [CrossRef]
- Yeramaneni, S.; Robinson, C.; Hostin, R. Impact of spine surgery complications on costs associated with management of adult spinal deformity. Curr. Rev. Musculoskelet. Med. 2016, 9, 327–332. [Google Scholar] [CrossRef]
- Lauinger, A.R.; Blake, S.; Fullenkamp, A.; Polites, G.; Grauer, J.N.; Arnold, P.M. Prediction models for risk assessment of surgical site infection after spinal surgery: A systematic review. N. Am. Spine Soc. J. 2024, 19, 100518. [Google Scholar] [CrossRef]
- Narain, A.S.; Kitto, A.Z.; Braun, B.; Poorman, M.J.; Curtin, P.; Slavin, J.; Whalen, G.; DiPaola, C.P.; Connolly, P.J.; Stauff, M.P. Does the ACS NSQIP Surgical Risk Calculator Accurately Predict Complications Rates After Anterior Lumbar Interbody Fusion Procedures? Spine 2021, 46, E655–E662. [Google Scholar] [CrossRef]
- McCarthy, M.H.; Singh, P.; Maslak, J.P.; Nayak, R.; Jenkins, T.J.; Patel, A.A.; Hsu, W.K. Can the American College of Surgeons Risk Calculator Predict 30-Day Complications After Cervical Spine Surgery? Clin. Spine Surg. 2019, 32, 357–362. [Google Scholar] [CrossRef]
- Pierce, K.E.; Kapadia, B.H.; Naessig, S.; Ahmad, W.; Vira, S.; Paulino, C.; Gerling, M.; Passias, P.G. Validation of the ACS-NSQIP Risk Calculator: A Machine-Learning Risk Tool for Predicting Complications and Mortality Following Adult Spinal Deformity Corrective Surgery. Int. J. Spine Surg. 2021, 15, 1210–1216. [Google Scholar] [CrossRef] [PubMed]
- Willoughby, J.E.; Baker, J.F. Utility of Surgical Risk Calculators in Spine Surgery in Patients Aged Over 80 Years: Analysis of SpineSage and ACS NSQIP. Glob. Spine J. 2022, 12, 21925682221074659. [Google Scholar] [CrossRef] [PubMed]
- Lee, M.J.; Cizik, A.M.; Hamilton, D.; Chapman, J.R. Predicting medical complications after spine surgery: A validated model using a prospective surgical registry. Spine J. 2014, 14, 291–299. [Google Scholar] [CrossRef] [PubMed]
- Harris, P.A.; Taylor, R.; Thielke, R.; Payne, J.; Gonzalez, N.; Conde, J.G. Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. J. Biomed. Inform. 2009, 42, 377–381. [Google Scholar] [CrossRef]
- Mirza, S.K.; Deyo, R.A.; Heagerty, P.J.; Konodi, M.A.; Lee, L.A.; Turner, J.A.; Goodkin, R. Development of an index to characterize the “invasiveness” of spine surgery: Validation by comparison to blood loss and operative time. Spine 2008, 33, 2651–2662. [Google Scholar] [CrossRef]
- Brier, G.W. Verification of Forecasts Expressed in Terms of Probability. Mon. Weather Rev. 1950, 78, 1–3. [Google Scholar] [CrossRef]
- Steyerberg, E.W. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating, 2nd ed.; Springer: Cham, Switzerland, 2019. [Google Scholar]
- Naessig, S.; Pierce, K.; Ahmad, W.; Passfall, L.; Krol, O.; Kummer, N.A.; Williamson, T.; Imbo, B.; Tretiakov, P.; Moattari, K.; et al. External Validation of the National Surgical Quality Improvement Program Calculator Utilizing a Single Institutional Experience for Adult Spinal Deformity Corrective Surgery. Int. J. Spine Surg. 2023, 17, 168–173. [Google Scholar] [CrossRef]
- Im, J.; Soliman, M.A.R.; Aguirre, A.O.; Quiceno, E.; Burns, E.; Khan, A.M.A.; Kuo, C.C.; Baig, R.A.; Khan, A.; Hess, R.M.; et al. American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator as a Predictor of Postoperative Outcomes After Adult Spinal Deformity Surgery: A Retrospective Cohort Analysis. Neurosurgery 2025, 96, 338–345. [Google Scholar] [CrossRef]
- Jadresic, M.C.; Baker, J.F. Predicting Complications of Spine Surgery: External Validation of Three Models. Spine J. 2022, 22, 1801–1810. [Google Scholar] [CrossRef]
- Kasparek, M.F.; Boettner, F.; Rienmueller, A.; Weber, M.; Funovics, P.T.; Krepler, P.; Windhager, R.; Grohs, J. Predicting medical complications in spine surgery: Evaluation of a novel online risk calculator. Eur. Spine J. 2018, 27, 2449–2456. [Google Scholar] [CrossRef]
Table 1.
Complication and outcome of interest definitions as defined by the ACS-NSQIP and SpineSageTM risk calculators.
Table 1.
Complication and outcome of interest definitions as defined by the ACS-NSQIP and SpineSageTM risk calculators.
| Complication | ACS-NSQIP Definition | SpineSageTM Definition |
|---|---|---|
| Any Complication | Superficial incisional SSI, deep incisional SSI, organ space SSI, wound disruption, pneumonia, unplanned intubation, PE, ventilator > 48 h, progressive renal insufficiency, acute renal failure, UTI, stroke, cardiac arrest, myocardial infarction, DVT, return to the operating room, systemic sepsis | GI bleeding, ileus, obstruction, pancreatitis, perforation, peritonitis, other GI occurrence, CVA/TIA, cerebral perfusion, delirium, diabetes insipidus, electrolyte change, meningitis, SAH/intracerebral hemorrhage, seizure, alcohol withdrawal, narcotic withdrawal, coagulopathy, EBL > 3 L, transfusion occurrence, Foley catheter trauma, renal insufficiency, urinary retention, other neurologic event |
| Serious/Major Complications | Cardiac arrest, myocardial infarction, pneumonia, progressive renal insufficiency, acute renal failure, PE, DVT, return to the operating room, deep incisional SSI, organ space SSI, systemic sepsis, unplanned intubation, UTI, wound disruption | Cardiac arrest, myocardial infarction, myocardial ischemia, acute respiratory distress syndrome, postoperative hypoxia, pulmonary embolus, respiratory arrest, GI perforation, CVA/TIA, meningitis, SAH/intracerebral hemorrhage |
| Pneumonia | Pneumonia | Infection of the lung parenchyma confirmed by fever, sputum or bronchial cultures, CXR, and requiring treatment |
| Cardiac Complications | Cardiac arrest or MI | Air embolism, cardiac arrest, arrhythmia, CHF, hypertension, hypotension, myocardial infarction, inappropriate or inadequate fluid therapy, myocardial ischemia, thermoregulation, other cardiac occurrence |
| Surgical Site Infection | Surgical site infection | - |
| Urinary Tract Infection | Urinary tract infection | The presence of large amounts of bacteria (>100,000 organisms/mL) in the upper or lower urinary tract associated with symptoms or requiring treatment |
| Venous Thromboembolism | Venous thromboembolism | The presence of thrombosis of the iliac, femoral, popliteal, or other veins confirmed by imaging studies (duplex scan, CT, or MR) with or without swelling, warmth, erythema, or tenderness |
| Renal Failure/Insufficiency | Progressive renal insufficiency or acute renal failure | Operational definition: Failure of the kidneys characterized by rapid decline in glomerular filtration rate (hours to days), retention of nitrogenous waste products, and perturbation of extracellular fluid volume and electrolyte and acid–base homeostasis; criteria: Serum Cr > 2 above baseline |
| Readmission | Readmission | - |
| Reoperation | Return to OR | - |
| Non-Home Discharge | Discharge to nursing or rehab facility | - |
| Death | Death | Death |
| Length of Stay (median days) | Predicted length of hospital stay | - |
| Dural Tear | - | Dural tear |
| Infection | - | Surgical site infection |
Table 2.
Patient demographics and complication profiles as defined by the ACS-NSQIP risk calculator.
Table 2.
Patient demographics and complication profiles as defined by the ACS-NSQIP risk calculator.
| ACS-NSQIP Defined Complications | ||||
|---|---|---|---|---|
| Variable | Total (198) | No Complication (n = 180) | Any Complication (n = 18) | p-Value |
| Sex (n, %) | 0.878 | |||
| Sex: Male | 67 (33.8%) | 62 (34.4%) | 5 (27.8%) | |
| Sex: Female | 131 (66.2%) | 118 (65.6%) | 13 (72.2%) | |
| Age (mean, sd) | 60.68 (15.68) | 60.98 (15.72) | 57.67 (15.40) | 0.251 |
| BMI (mean, sd) | 27.11 (6.07) | 26.93 (6.10) | 28.97 (5.51) | 0.096 |
| Current/Former Smoker (n, %) | 4 (2.0%) | 4 (2.2%) | 0 (0.0%) | 0.645 |
| Medical Comorbidities (n, %) | ||||
| Hypertension | 38 (19.2%) | 35 (19.4%) | 3 (16.7%) | 0.717 |
| Diabetes | 16 (8.1%) | 15 (8.3%) | 1 (5.6%) | 0.825 |
| Kidney Disease | 4 (2.0%) | 3 (1.7%) | 1 (5.6%) | 0.382 |
| Lung Disease | 6 (3.0%) | 5 (2.8%) | 1 (5.6%) | 0.346 |
| Congestive Heart Failure | 2 (1.0%) | 2 (1.1%) | 0 (0.0%) | 0.732 |
| Myocardial Infarction | 3 (1.6%) | 3 (1.7%) | 0 (0.0%) | 0.855 |
| Coronary Artery Disease | 8 (4.3%) | 6 (3.3%) | 2 (11.1%) | 0.275 |
| Cerebrovascular Disease | 4 (2.0%) | 4 (2.2%) | 0 (0.0%) | 0.813 |
| Peripheral Vascular Disease | 2 (1.0%) | 2 (1.1%) | 0 (0.0%) | 0.9 |
| Autoimmune Disease | 8 (4.3%) | 8 (4.4%) | 0 (0.0%) | 0.658 |
| Dyslipidemia | 48 (25.5%) | 43 (23.9%) | 5 (27.8%) | 0.924 |
| Cancer | 18 (9.6%) | 17 (9.4%) | 1 (5.6%) | 0.859 |
| Liver Disease | 2 (1.0%) | 2 (1.1%) | 0 (0.0%) | 0.9 |
| Peptic Ulcer Disease | 3 (1.6%) | 3 (1.7%) | 0 (0.0%) | 0.855 |
| Gastroesophageal Reflux Disease | 47 (23.7%) | 43 (23.9%) | 4 (22.2%) | 0.984 |
| Osteoarthritis | 20 (10.6%) | 18 (10.0%) | 2 (11.1%) | 0.982 |
| Osteoporosis | 30 (16%) | 26 (14.4%) | 4 (22.2%) | 0.668 |
| Rheumatologic Disease | 7 (3.7%) | 7 (3.9%) | 0 (0.0%) | 0.694 |
| Anxiety | 25 (13.3%) | 25 (13.9%) | 0 (0.0%) | 0.239 |
| Depression | 14 (7.4%) | 14 (7.8%) | 0 (0.0%) | 0.471 |
| Peripheral Neuropathy | 3 (1.6%) | 3 (1.7%) | 0 (0.0%) | 0.855 |
| Hypercoagulability Disorder | 4 (2.0%) | 3 (1.7%) | 1 (5.6%) | 0.53 |
| Bleeding Disorder | 3 (1.6%) | 3 (1.7%) | 0 (0.0%) | 0.855 |
| American Society of Anesthesiologists Class (n, %) | 0.027 | |||
| ASA I | 3 (1.6%) | 2 (1.1%) | 1 (5.6%) | |
| ASA II | 136 (68.7%) | 128 (71.1%) | 8 (44.4%) | |
| ASA III | 59 (29.8%) | 50 (27.8%) | 9 (50.0%) | |
| Charlson Comorbidity Index (mean, standard deviation) | 2.31 (1.61) | 2.36 (1.63) | 1.89 (1.41) | 0.318 |
| Operative and Perioperative Data | ||||
| Three-Column Osteotomy (n, %) | 52 (26.3%) | 47 (26.1%) | 5 (27.8%) | 1 |
| Operative Time (mean hours, standard deviation) | 259.95 (81.88) | 255.41 (79.33) | 305.35 (95.07) | 0.028 |
| Estimated Blood Loss (mean, standard deviation) | 1070.36 (801.3) | 1033.45 (793.46) | 1433.33 (809.68) | 0.029 |
| Number of Levels Fused (mean, standard deviation) | 8.96 (3.26) | 8.82 (3.04) | 10.39 (4.82) | 0.405 |
| Length of Stay (mean, standard deviation) | 6.20 (3.27) | 6.05 (3.32) | 7.70 (2.29) | 0.001 |
Note: Bold indicates that p < 0.05, the data is statistically significant.
Table 3.
Patient demographics and complication profiles as defined by the SpineSageTM risk calculator.
Table 3.
Patient demographics and complication profiles as defined by the SpineSageTM risk calculator.
| SpineSageTM Defined Complications | ||||
|---|---|---|---|---|
| Variable | Total (198) | No Complication (n = 63) | Any Complication (n = 135) | p-Value |
| Sex (n, %) | ||||
| Sex: Male | 67 (33.8%) | 26 (41.3%) | 41 (30.4%) | 0.284 |
| Sex: Female | 131 (66.2%) | 37 (58.7%) | 94 (69.6%) | |
| Age (mean, standard deviation) | 60.68 (15.68) | 56.43 (18.16) | 62.66 (14.01) | 0.024 |
| Body Mass Index (mean, standard deviation) | 27.11 (6.07) | 27.65 (6.49) | 26.85 (5.87) | 0.532 |
| Current or Former Smoker (n, %) | 4 (2%) | 1 (1.6%) | 3 (2.2%) | 0.37 |
| Medical Comorbidities (n, %) | ||||
| Hypertension | 38 (19.2%) | 5 (7.9%) | 33 (24.4%) | 0.021 |
| Diabetes | 16 (8.1%) | 3 (4.8%) | 13 (9.6%) | 0.303 |
| Kidney Disease | 4 (2%) | 2 (3.2%) | 2 (1.5%) | 0.223 |
| Lung Disease | 6 (3%) | 3 (4.8%) | 3 (2.2%) | 0.035 |
| Congestive Heart Failure | 2 (1%) | 0 (0.0%) | 2 (1.5%) | 0.256 |
| Myocardial Infarction | 3 (1.6%) | 1 (1.6%) | 2 (1.5%) | 1 |
| Coronary Artery Disease | 8 (4.3%) | 2 (3.2%) | 6 (4.4%) | 0.967 |
| Cerebrovascular Disease | 4 (2%) | 0 (0.0%) | 4 (3%) | 0.4 |
| Peripheral Vascular Disease | 2 (1%) | 1 (1.6%) | 1 (0.7%) | 1 |
| Autoimmune Disease | 8 (4.3%) | 4 (6.3%) | 4 (3%) | 0.463 |
| Dyslipidemia | 48 (25.5%) | 18 (28.6%) | 30 (22.2%) | 0.434 |
| Cancer | 18 (9.6%) | 5 (7.9%) | 13 (9.6%) | 0.896 |
| Liver Disease | 2 (1%) | 1 (1.6%) | 1 (0.7%) | 1 |
| Peptic Ulcer Disease | 3 (1.6%) | 1 (1.6%) | 2 (1.5%) | 1 |
| Gastroesophageal Reflux Disease | 47 (23.7%) | 15 (23.8%) | 32 (23.7%) | 1 |
| Osteoarthritis | 20 (10.6%) | 8 (12.7%) | 12 (8.9%) | 0.571 |
| Osteoporosis | 30 (16%) | 8 (12.7%) | 22 (16.3%) | 0.646 |
| Rheumatological Disease | 7 (3.7%) | 2 (3.2%) | 5 (3.7%) | 1 |
| Anxiety | 25 (13.3%) | 8 (12.7%) | 17 (12.6%) | 1 |
| Depression | 14 (7.4%) | 6 (9.5%) | 8 (5.9%) | 0.539 |
| Peripheral Neuropathy | 3 (1.6%) | 1 (1.6%) | 2 (1.5%) | 1 |
| Hypercoagulability Disorder | 4 (2%) | 0 (0.0%) | 4 (3%) | 0.4 |
| Bleeding Disorder | 3 (1.6%) | 0 (0.0%) | 3 (2.2%) | 0.568 |
| American Society of Anesthesiologists Class (n, %) | 0.099 | |||
| ASA Class I | 3 (1.6%) | 1 (1.6%) | 2 (1.5%) | |
| ASA Class II | 136 (68.7%) | 49 (77.8%) | 87 (64.4%) | |
| ASA Class III | 59 (29.9%) | 13 (20.6%) | 46 (34.1%) | |
| Charlson Comorbidity Index (mean, standard deviation) | 2.31 (1.61) | 2.02 (1.82) | 2.45 (1.49) | 0.026 |
| Operative and Perioperative Data | ||||
| Three-Column Osteotomy | 52 (26.3%) | 12 (19.0%) | 40 (29.6%) | 0.161 |
| Operative Time (hours, standard deviation) | 259.95 (81.88) | 219.49 (73.91) | 278.83 (78.74) | <0.001 |
| Estimated Blood Loss (mean, standard deviation) | 1070.36 (801.30) | 876.29 (619.66) | 1160.83 (860.40) | 0.026 |
| Number of Levels Fused (mean, standard deviation) | 8.96 (3.26) | 7.81 (2.31) | 9.50 (3.50) | 0.004 |
| Length of Stay (mean, standard deviation) | 6.20 (3.27) | 5.07 (2.27) | 6.73 (3.53) | 0.001 |
Note: Bold indicates that p < 0.05, the data is statistically significant.
Table 4.
Complication predictions produced by the ACS-NSQIP risk calculator.
| Variable | Predicted Probability | Actual Incidence | Brier Score | C-Statistic |
|---|---|---|---|---|
| Any Complication | 15.2 | 9.1 | 0.09 | 0.57 |
| Serious Complications | 13.2 | 8.6 | 0.08 | 0.57 |
| Pneumonia | 1.4 | 0.5 | 0.005 | 0.51 |
| Cardiac | 0.8 | 0 | 0.0001 | - |
| Surgical Site Infection | 2.2 | 2 | 0.02 | 0.86 |
| Urinary Tract Infection | 3.2 | 3.5 | 0.03 | 0.56 |
| Venous Thromboembolism | 2.6 | 0.5 | 0.006 | 0.84 |
| Renal Failure | 0.23 | 0 | 0.00002 | - |
| Readmission | 6.1 | 3 | 0.03 | 0.7 |
| Reoperation | 3.7 | 2.5 | 0.02 | 0.57 |
| Non-Home Discharge | 41.6 | 10.2 | 0.2 | 0.58 |
| Death | 0.4 | 0 | 0.00005 | - |
| Length of Stay (median days) | 5 | 5.4 | - | - |
Note: Values for all complications represent predicted probability and actual incidence as a percentage. Only length of stay is in days and calculated as a median. Medical complications with no c-statistic and low brier score reports did not occur in our cohort and were omitted from analysis.
Table 5.
Complication predictions produced by the SpineSageTM risk calculator.
| Variable | Predicted Probability | Actual Incidence | Brier Score | C-Statistic |
|---|---|---|---|---|
| Any Complication (transfusions) | 32 | 74.7 (62.1) | 0.36 | 0.58 |
| Serious Complications | 7.2 | 1 | 0.02 | 0.5 |
| Infection | 12.3 | 2 | 0.02 | 0.6 |
| Dural Tear | 15.7 | 11.1 | 0.1 | 0.54 |
Note: Values for all complications represent predicted probability and actual incidence as a percentage.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
MDPI and ACS Style
Pajak, A.; Samuel, J.T.; Subramanian, T.; Merrill, R.K.; Akosman, I.; Clohisy, J.C.; Du, J.; Zhang, B.; Elysee, J.; Shahi, P.; et al. Do Publicly Available Risk Calculators Apply to Adult Spinal Deformity Surgery? J. Clin. Med. 2025, 14, 8618. https://doi.org/10.3390/jcm14248618
AMA Style
Pajak A, Samuel JT, Subramanian T, Merrill RK, Akosman I, Clohisy JC, Du J, Zhang B, Elysee J, Shahi P, et al. Do Publicly Available Risk Calculators Apply to Adult Spinal Deformity Surgery? Journal of Clinical Medicine. 2025; 14(24):8618. https://doi.org/10.3390/jcm14248618
Chicago/Turabian StylePajak, Anthony, Justin T. Samuel, Tejas Subramanian, Robert K. Merrill, Izzet Akosman, John C. Clohisy, Jerry Du, Bo Zhang, Jonathan Elysee, Pratyush Shahi, and et al. 2025. "Do Publicly Available Risk Calculators Apply to Adult Spinal Deformity Surgery?" Journal of Clinical Medicine 14, no. 24: 8618. https://doi.org/10.3390/jcm14248618
APA StylePajak, A., Samuel, J. T., Subramanian, T., Merrill, R. K., Akosman, I., Clohisy, J. C., Du, J., Zhang, B., Elysee, J., Shahi, P., Kim, D. N., Jordan, Y., Knopp, R. L., Lovecchio, F. C., & Kim, H. J. (2025). Do Publicly Available Risk Calculators Apply to Adult Spinal Deformity Surgery? Journal of Clinical Medicine, 14(24), 8618. https://doi.org/10.3390/jcm14248618
Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.
