Procalcitonin and Risk Prediction for Diagnosing Bacteremia in Hospitalized Patients: A Retrospective, National Observational Study

Bacteremia is associated with significant morbidity and mortality. Timely, appropriate therapy may improve clinical outcomes, and therefore, determining which patients benefit from more comprehensive diagnostic strategies (i.e., direct specimen testing) could be of value. We performed an assessment of procalcitonin (PCT) and clinical characteristics in the discrimination of bacteremic hospitalizations. We analyzed 71,105 encounters and 14,846 visits of patients with bacteremia alongside 56,259 without an admission. The area under the receiver—operating characteristic (AUROC) curve for the prediction of bacteremia via procalcitonin was 0.782 (95% CI 0.779–0.787). The prediction modeling of clinical factors with or without PCT resulted in a similar performance to PCT alone. However, the clinically predicted risk of bacteremia stratified by PCT thresholds allowed the targeting of high-incidence bacteremia groups (e.g., ≥50% positivity). The combined use of PCT and clinical characteristics could be useful in diagnostic stewardship by targeting further advanced diagnostic testing in patients with a high predicted probability of bacteremia.


Introduction
Sepsis results in significant morbidity and mortality and, specifically, is associated with 20% of all global deaths annually [1].Early appropriate antimicrobial therapy has been established as paramount to improving clinical outcomes in septic patients [2].Blood cultures are routinely used to direct appropriate therapy in patients with sepsis, but while useful, they are incrementally limited in their clinical impact when considering time to positivity and turnaround time in conventional microbiology processing.
Molecular rapid diagnostic testing (RDT) has been shown to improve the management of patients with sepsis and bloodstream infections (BSIs) [3].Similarly, the direct specimen RDT has been shown to have the potential to allow for more rapid tailoring of antibiotics in patients with sepsis [4,5].However, ubiquitous testing among sepsis patients is limited by the laboratory burden in relation to testing costs and staffing resource limitations when performing a related volume of testing.Therefore, targeting testing through diagnostic stewardship to select patients with a high likelihood of BSIs may allow for the application of advanced diagnostics to direct specimens in an effective and efficient manner [6].
Procalcitonin (PCT), a 116-amino-acid precursor to calcitonin, while normally produced by the thyroid in healthy individuals, peaks 12 to 24 h at several-fold higher levels and is produced by a variety of other organs in response to bacterial infection [7].PCT has been FDA-approved for prognostic use in ICU-admitted sepsis patients and as an aid in antibiotic decision-making in lower respiratory tract infections [8].Additionally, research on PCT has shown promise in predicting BSIs either alone or in combination with clinical prediction models [9,10].However, research has been limited due to its single center and/or smaller sample size, thus limiting conclusions on clinical utility.Further research may inform the diagnostic stewardship of direct specimen rapid diagnostics [6].
Our objective was to characterize the performance of PCT when predicting bacterial bloodstream infections across a national inpatient cohort in the US, along with evaluating the additional utility of clinical predictors to identify BSIs, as both approaches could inform diagnostic stewardship efforts toward the effective use of direct specimen RDTs.

Data Source
Hospital encounters were derived from the Premier Healthcare Database (PHD; Premier Inc., Charlotte, NC, USA) [11].The PHD, which represents 20% of acute care hospitalization nationwide, includes retrospective, observational patient-level encounters with demographics as well as the 10th revision of the International Statistical Classification of Diseases and Related Health Problems (ICD-10), including diagnostic and procedure codes present on admission, invoiced items per day including medications, general laboratory and microbiology data with collection and result alongside the day and times, and facility characteristics.The PHD includes both inpatient and outpatient encounters for over one billion visits over 20 years from currently over 1000 geographically diverse hospitals of varying sizes in both rural and urban areas.This database is compliant with the Health Insurance Portability and Accountability Act.As this study used a fully de-identified database, it was exempt from ethics review under US 45 CFR 46.101(b)4.13[12].

Study Population
Patients were included if they underwent hospital admission from 1 January 2017 through to 31 December 2019.Adults (≥18 years of age) were selected if they had one or more blood cultures and a PCT level obtained on day one of admission.Encounters were classified by the admission of blood cultures into bacteremic encounters for bacterial etiologies or negative encounters based on culture-based microbiology results.Common contaminants (e.g., coagulase-negative Staphylococcus spp., Micrococcus spp., etc.) were excluded due to the absence of data on vitals (e.g., heart rate and temperature) provided by the PHD, and therefore, we were unable to adjudicate likely contamination by standardized definitions [13,14].Fungal and mycobacterial bloodstream infections were also excluded due to the discrimination performance of PCT being specific and robust for typical bacterial infections [15,16].

Outcomes and Definitions
Descriptive data were analyzed for the cohort, including baseline patient and hospital characteristics, in addition to patient encounter characteristics.Chronic disease comorbidities were evaluated using the Charlson Comorbidity Index, which was derived from standardized ICD-10 codes [17].Similarly, admission diagnoses were evaluated using ICD-10 codes (Supplementary Table S1).Laboratory values were selected based on their Logical Observation Identifiers Names and Codes (LOINC ® ; Supplementary Table S2) and evaluated based on clinically relevant abnormalities.Maximum values of lactate and serum creatinine on admission were assessed, while minimum values of albumin and platelets were evaluated.WBCs were assessed for the presence of leukopenia (<4000 cells/mm 3 ) or leukocytosis (>12,000 cells/mm 3 ).Inpatient mortality was determined by hospital encounter discharge records.
The discrimination performance of PCT for diagnosing bacteremia was assessed for the overall cohort.Subgroups were analyzed among those admitted with a diagnosis (overall cohort, pneumonia, pyelonephritis, and febrile neutropenia) and were evaluated based on relevance from previous studies [9,10].Likewise, age, comorbidities, admission diagnosis, and laboratory-based clinical predictors of bacteremia on admission were exam-ined through the derivation of a regression model based on previously described prediction models [9,10].The predictors evaluated included an admission or present on admission diagnosis (pyelonephritis, pneumonia, febrile neutropenia), PCT level, elevated serum creatinine (>2 mg/dL), elevated lactate (>2 mmol/L), leukocytosis or leukopenia, low platelets (<150,000/mm 3 ), and low albumin (≤3 g/dL) [9,10].The erythrocyte sedimentation rate (ESR), C-reactive protein (CRP), and lactate were not included, as they are not routinely used in practice in the majority of facilities for sepsis patients (i.e., missing data for at least 40% of the study population) and were thus prohibitive to imputation.All time-varying predictors (i.e., laboratory values) were assessed based on day one of admission labs.

Statistical Analysis
Patient, hospital, and encounter-related characteristics were summarized using the median (IQR) and number (percentage).The performance of PCT to predict bacteremia was assessed using sensitivity, specificity, and the area under the receiver operating characteristic curve (AUROC).An AUC of 0.5 reflects the discrimination capacity consistent with chance, while 1 reflects perfect discrimination.The optimized cutoff for PCT was determined using Youden's index value.
Additional encounter predictors were evaluated along with PCT in an unconditional logistic regression model for the evaluation of bacteremia [9].Multivariable logistic regression was performed with the least absolute shrinkage and selection operator (LASSO), a penalized least squares method, with candidate variables for model development and model selection based on a lambda shrinkage parameter with best performance in a ten-fold cross-validation [18].Final variables were modeled with logistic regression, as LASSO could bias final model coefficients toward zero based on its soft thresholding property [18][19][20].The sample size was assessed using the pmsampsize package for prediction modeling using 14 candidate parameters with a potential prevalence of 5% bacteremia and an average derivation model of 0.79 c-statistic from previous models [21].The estimated required samples and events were at least 2125 and 107, respectively.Single imputation was performed if data were missing.Model performance was evaluated for discrimination (C-statistic) and calibration (agreement between predicted and observed using calibration slopes).

Results
We identified 71,105 patients meeting the eligibility criteria of at least one procalcitonin level and blood culture obtained on the day of admission; 14,846 were bacteremic, and 56,259 were non-bacteremic encounters.Patient and hospital characteristics among bacteremic and non-bacteremic encounters were overall similar (Table 1).The median age of the cohort was 68 years (IQR 56-79), and 49.8% were female.The Charlson Comorbidity Index was the same across groups (median 3, IQR 2-4); however, bacteremic patients had a higher proportion of diagnoses for chronic renal disease (33.2% vs. 28.6%)and chronic hepatic disease (11.4% vs. 7.1%) compared to non-bacteremic patients.The majority of patient encounters occurred in hospitals with ≤500 beds (36.1%) and in urban settings (92.3%).
The clinical characteristics of patient encounters can be found in Table 2.The majority of encounters occurred in hospitalist or internal medicine wards as the admission service (78.0%).Among admission or present-on-admission diagnoses, as described in previous derivation or validation models for predicting bacteremia, pneumonia, and sepsis were common, occurring in 42.1% and 42.7% of encounters, respectively.Notably, bacteremia occurred less commonly among pneumonic patients (34.8% vs. 44.0%)and was more common among sepsis patients (80.3% vs. 32.8%).Abnormal laboratory values were more common among bacteremic patients, and specifically, the median PCT was 2.44 (IQR 0.45-13.81)among bacteremic patients and 0.19 (0.07-0.73) among non-bacteremic patients.Finally, the length of stay (4 vs. 6 days) and mortality (6.0% vs. 11.0%)were lower among non-bacteremic patients than bacteremic patients.The diagnostic performance of PCT for bacteremia had good discrimination with an AUROC of 78.2% (95% CI, 77.9 to 78.7).Discrimination performance decreased among pneumonia and pyelonephritis admission or present-on-admission diagnoses (Figure 1).Sensitivity-balanced (72.9%) and specificity-balanced (70.2%) discrimination performance was maximized at a cutoff threshold of 0.53 ug/mL.Additionally, a regression model was developed (Table 3) based on clinical characteristics and lab results including PCT, which reflected moderate discrimination performance (C-statistic 80.7%) and calibration (Supplementary Figure S1).In the absence of PCT in this model, discrimination performance was slightly lower (C-statistic 79.4%), and calibration was similar.Prediction model regression equations can be found in the Supplementary Materials (Table S3).
The diagnostic performance of PCT for bacteremia had good discrimination with an AUROC of 78.2% (95% CI, 77.9 to 78.7).Discrimination performance decreased among pneumonia and pyelonephritis admission or present-on-admission diagnoses (Figure 1).Sensitivity-balanced (72.9%) and specificity-balanced (70.2%) discrimination performance was maximized at a cutoff threshold of 0.53 ug/mL.Additionally, a regression model was developed (Table 3) based on clinical characteristics and lab results including PCT, which reflected moderate discrimination performance (C-statistic 80.7%) and calibration (Supplementary Figure S1).In the absence of PCT in this model, discrimination performance was slightly lower (C-statistic 79.4%), and calibration was similar.Prediction model regression equations can be found in the Supplementary Materials (Table S3).Bacteremia incidence varied when evaluated using PCT-only stratified cutoffs, a prediction model with PCT stratifications, and a prediction model without PCT (Table 4).For PCT only, levels of >2 to <10 ng/dL and >10 ng/dL were associated with a bacteremia incidence of 40% or more compared to 20.9% for the overall cohort.Similarly, using the clinical prediction model without PCT, a medium (25-75%) and high (>75%) risk of bacteremia predicted the probability of bacteremia incidence in patients as 42% or more.When using both PCT and the clinical prediction model with stratified cutoffs together, a bacteremia incidence of 53% or more was noted with medium risk and PCT > 2 ng/dL, while all high-risk patients had an incidence of 70% or higher.

Discussion
In our comprehensive cohort study of 71,105 hospitalized patients from whom we obtained blood cultures and procalcitonin, we demonstrated a moderate diagnostic discrimination performance when predicting bacteremia for both procalcitonin testing and clinical prediction modeling.Our findings showcase a moderate level of performance for both methodologies.When considering the immense challenge and critical importance of timely bacteremia detection, the potential utility of either approach in guiding advanced diagnostic testing from blood cultures becomes evident.While either approach may be beneficial in driving advanced diagnostic testing from blood cultures, the use of both approaches at stratified ranges can allow increased incidence risk groups who are most likely to benefit from advanced diagnostic testing to be targeted.These evaluations reflect a diagnostic stewardship approach of enriching patient selection for advanced diagnostic testing to ensure the right test for the right patient at the right time [25].Furthermore, we observed that PCT discrimination performance across admission diagnoses was mostly comparable, with the exception of pyelonephritis and pneumonia, which had lower performance.This insight underscores the complexities and variables inherent in bacteremia diagnostics.Additionally, we found high proportions (60.3%) of bacteremic patients with a PCT > 10 ng/dL, reinforcing the value of PCT as a diagnostic marker.
Previous studies of clinical prediction modeling when predicting bacteremia have resulted in an AUROC of 0.60-0.83,while pooled estimates of PCT have reflected an AU-ROC of 0.79 [9,10].We observed discrimination performance to be similar among both PCT and clinical prediction models.In a systematic review and meta-analysis of procalcitonin for diagnostic accuracy on bacteremia for 58 studies and 16,514 patients (3420 with bloodstream infection), cut-off values for PCT varied between 0.10 ng/dL and 17 ng/dL (median 0.5 ng/dL) [9].Similarly, in our overall cohort, we observed 0.53 ng/dL as the threshold for balancing sensitivity and specificity.This balance assumes that false positives and false negatives are equally important, which might not be applicable to all clinical scenarios [26].For instance, a febrile neutropenia patient admitted to an ICU on pressors could benefit from a low threshold that increases sensitivity.By contrast, a young adult floor patient without comorbidities likely has a threshold favoring increased specificity.Additionally, when used to evaluate advanced diagnostic testing decisions, this threshold may be a function of prevalence for disease or a potential for therapy modification based on advanced diagnostic.Moreover, continuous prediction models without specific thresholds could help facilitate individual-level decision-making in certain use cases, allowing for personalized medicine [26].As we look ahead, there is a compelling need for research that dives deep into crafting prediction models tailored to specific advanced diagnostics and patient populations, ensuring that the chosen thresholds resonate with the unique demands of each clinical context.
Recent research has evaluated the unique characteristics of patients with lower PCT levels (<2, 0.5 to 2, and <0.5 ng/dL) and bacteremia [27], noting that these patients are difficult to identify based on observed attributes.In the context of a clinical presentation that is consistent with potential bacteremia, these PCT levels could make appropriate workup and diagnosis challenging.We found that bacteremia incidence was 9.1% and 25.4% for <0.5 and 0.5 to <2 ng/dL, respectively, but higher likelihoods could be targeted through clinical prediction modeling to >24% and >40% for these cutoffs if restricted to medium-and high-risk-prediction-model patients, reflecting the synergistic value of PCT and clinical prediction models for determining patients who are likely bacteremic.This combined approach not only augments diagnostic accuracy but also underscores the importance of a multi-faceted strategy when managing complex clinical scenarios.
The realm of direct specimen testing often enters intricate territories.Direct specimen testing can be problematic, as testing is driven by often subjective judgments of who benefits from testing or their perceived pre-test probability of a bloodstream infection [4,6].If optimal testing is not approached through diagnostic stewardship, a lower diagnostic yield and avoidable increased costs without meaningful clinical impacts may result.To recognize these inherent challenges and address this issue, we performed a retrospective evaluation of the diagnostic accuracy of PCT and a validation of clinical prediction for bacteremia.Our findings illuminate a promising path: harnessing the insights from these tools offers a strategic direction for advanced testing.This aligns with previous research in this area, which has found PCT to be an avenue for improving direct specimen testing with SeptiFast real-time PCR (Roche Diagnostics GmbH, Mannheim, Germany) in a more cost-effective manner [6].While we evaluated a retrospective national cohort using conventional culture diagnostics, we believe our data build on this literature and aforementioned systematic reviews through our combination approach of PCT and clinical prediction models, which allows a variety of bacteremia incidence groups to be targeted with increased precision.As the implementation of advanced diagnostics can occur across varied, heterogenous settings with competing needs, limited resources, and clinical practice, the implementation and prioritization of advanced diagnostics should be tailored.Therefore, our suggested approach can allow flexibility based on these needs.In an envisioned diagnostic framework, patients exhibiting a predicted incidence of bacteremia greater than 50% could be selectively prioritized for advanced direct specimen testing.This strategic direction could harness the synergistic power of both PCT and clinical risk prediction models, as illustrated in a hypothetical diagnostic pathway in Supplementary Table S4.To truly realize the potential of these integrated approaches, it is imperative that future research engages in subsequent external validations of the prediction model.Furthermore, future investigations focusing on these approaches with the practical clinical utility of specific advanced diagnostic methodologies could be instrumental in refining and substantiating our proposed strategies.
Our study has several limitations.As our data were retrospective, the blood cultures, procalcitonin, and other labs were not systematically collected and thus subject to clinical variability and bias.However, the performance reported herein likely reflects pragmatic real-world utility.While our study is derived from 1200 diverse hospitals and is likely nationally representative, generalization to specific hospitals should be undertaken with caution and proceed with local validations.In contrast with our present study, most previous studies are limited to single centers in a specific geographic setting with samples of 200-2000, and, therefore, our data likely reflect a typically expected discrimination performance.In our research, we did not incorporate data related to erythrocyte sedimentation rate (ESR) or the C-reactive protein (CRP) as predictive markers for bacteremia.The rationale behind this exclusion stemmed from several considerations.First, these particular biomarkers, while valuable in assessing systemic inflammation, have only sporadically been incorporated into bacteremia prediction models in prior studies [10].This limited inclusion might reflect their broader role in indicating inflammation rather than specifically identifying bacterial infections.Second, existing research using extensive national US cohorts underscores a notable trend: approximately 80% of clinical encounters do not involve the use of these biomarkers for this purpose, suggesting a prevailing clinical practice pattern that often omits ESR and CRP in the bacteremia diagnostic process in the US [28].While our study sought to mirror current practices and prevailing research trends, we acknowledge the potential utility of these markers.Future investigations might benefit from a comprehensive evaluation of ESR and CRP, potentially revealing additional insights into their diagnostic relevance in bacteremia.We also did not analyze pediatric patients.We focused on adults as labeling-supported and clear cutoffs that exist to guide evaluations and analysis [8].By contrast, pediatric patients, particularly neonates and young infants, may have elevated PCT in the absence of bacterial infection [29].While we believe our approach of either clinical prediction, PCT, or a combination thereof offers merit in predicting bacteremia and driving advanced diagnostic testing in pediatrics as well, specific models and thresholds should be evaluated in future research for this population.Finally, our goal was to find reliable predictors to drive advanced diagnostic testing, and the current study only provides a potential approach toward that goal.Future research should evaluate the use of PCT, clinical prediction models, or a combination thereof as a diagnostic stewardship and pre-analytical strategy to drive advanced diagnostic testing.

Conclusions
In our study, spanning 71,105 hospitalized patients, the pivotal role of procalcitonin (PCT) and clinical prediction modeling in bacteremia diagnosis is underscored.Both individually and in tandem, these methods offer significant diagnostic accuracy.Elevated PCT levels in a notable 60.3% of bacteremic patients highlight its diagnostic potential.Furthermore, the combined strength of PCT and clinical models points to a nuanced, tailored approach for patient selection in advanced diagnostic testing-epitomizing diagnostic stewardship.While direct specimen testing brings its own set of challenges, our research underscores the necessity for a more integrated diagnostic approach.As the landscape of bacteremia diagnostics evolves, our findings set the stage for future research, emphasizing the importance of refining and validating these tools in diverse clinical contexts.

Figure 1 .
Figure 1.AUROC curves overall and for patient admission diagnosis.

Table 1 .
Patient and hospital characteristics.

Table 2 .
Clinical characteristics of patient encounters.

Table 3 .
Predictive model for bacteremia.

Table 3 .
Cont.Variable selection by LASSO depends on performance in the model for a ten-fold cross-validation; therefore, all variables in the final logistic model may not be significant. *

Table 4 .
Bacteremia incidence via PCT only and clinical prediction models.