The Integration of qSOFA with Clinical Variables and Serum Biomarkers Improves the Prognostic Value of qSOFA Alone in Patients with Suspected or Confirmed Sepsis at ED Admission.

Background: The prognostic value of quick sepsis-related organ failure assessment (qSOFA) outside intensive care units has been criticized. Therefore, we aimed to improve its ability in predicting 30-day all-cause mortality, and in ruling out the cases at high risk of death among patients with suspected or confirmed sepsis at emergency department (ED) admission. Methods: This study is a secondary analysis of a prospective multicenter study. We built three predictive models combining qSOFA with the clinical variables and serum biomarkers that resulted in an independent association with 30-day mortality, in both 848 undifferentiated patients (Group 1) and in 545 patients definitively diagnosed with sepsis (Group 2). The models reaching the highest negative predictive value (NPV) with the minimum expenditure of biomarkers in Group 1 and in Group 2 were validated in two cohorts of patients initially held out due to missing data. Results: In terms of the area under the receiver-operating characteristic curve, all six models significantly exceeded qSOFA in predicting prognosis. An “extended” qSOFA (eqSOFA1) in Group 1 and an eqSOFA2 integrated with C-reactive protein and mid-regional proadrenomedullin (eqSOFA2+CRP+MR-proADM) in Group 2 reached the best NPV (0.94 and 0.93, respectively) and ease of use. eqSOFA1 and eqSOFA2+CRP+MR-proADM performed equally well in both the inception and validation cohorts. Conclusions: We have derived and validated two prognostic models that outweigh qSOFA in predicting mortality and in identifying the low risk of death among patients with suspected or confirmed sepsis at ED admission.


Introduction
Sepsis is a major cause of mortality worldwide. At emergency department (ED) admission, the proper assessment of sepsis outcome is vitally important to aid clinical decision making and to optimize the allocation of critical care resources. In 2016, the SEPSIS-3 Task Force proposed quick sepsis-related organ failure assessment (qSOFA), a simple and rapid score, to help discriminate sepsis survivors from non-survivors with a performance at least comparable with that of a sequential organ failure assessment (SOFA) score [1]. Several studies have demonstrated the usefulness of qSOFA-based prognostication in patients with suspected sepsis, especially when compared with that of the systemic inflammatory response syndrome (SIRS) criteria [2]. However, it was not always clearly stated in those works if patients were included who were initially suspected to have infection, but were adjudicated not to have infection on expert review. This finding, on one hand, makes those studies' interpretation tricky, and on the other hand, it may raise concerns about their generalizability. Given the above premises, the prognostic value of qSOFA at ED admission has been questioned, especially when compared with more complex warning systems [2][3][4].

Methods
This study is a secondary analysis of a prospective multicenter study conducted in 5 Italian EDs during a 2-year period [22]. Its original purpose was to derive and validate a predictive algorithm to rule out infection in 1132 undifferentiated patients with suspected sepsis who fulfilled at least 2 SIRS criteria at ED admission. The latter cohort included both patients who received a definitive diagnosis of infection and those who were deemed not infected (ni-SIRS) at the end of clinical work-up. Exclusion criteria were age less than 18 years, absence of informed consent, and pregnancy. Etiology, severity, and source of infection were adjudicated according to a predefined classification system (available in the Supplement). For this secondary analysis, we reviewed the three clinical variables required to calculate the qSOFA score, and the first venous lactate concentration measured at each respective site in all undifferentiated patients with suspected sepsis. The primary outcome of the present study was 30-day all-cause mortality. The secondary outcome was ruling out the high risk of death.

Biomarker Measurements
Blood samples for CRP, procalcitonin, sPLA2GIIA, presepsin, sIL2Rα, and sTREM-1 measurements were collected at ED admission in the entire cohort of undifferentiated patients with suspected sepsis, as previously reported [22]. MR-proADM and sTNFR-1 were measured specifically for this secondary analysis in the remaining frozen aliquots of serum. In detail, MR-proADM was assayed by an automated Kryptor analyzer (Kriptor Brahms AG, Hennigsdorg Germany) in the 1132 undifferentiated patients with suspected sepsis. Among these patients, in those who were definitively diagnosed with sepsis (729), sTNFR-1 determinations were carried out by commercial enzyme-linked immunosorbent assay kits following the manufacturer's instructions (R&D).

Statistical Analysis
Categorical data are summarized as percentages and numbers, normally distributed continuous data as mean ± standard deviation, and non-normally distributed variables as median and interquartile range. The unpaired Student t test was used, when appropriate, for the comparison of normally distributed data, while the Mann-Whitney test was used for non-normal continuous data. The χ2 test or the Fisher test was used, when appropriate, to compare categorical variables expressed as proportions. Unadjusted odds ratios (ORs) were calculated to assess the independent effect of qSOFA score, clinical variables, and serum biomarkers on 30-day mortality.
To estimate a multivariable model, two strategies were implemented and compared: one based on a least absolute shrinkage and selection operator (LASSO) [23], i.e., a penalized logistic regression, and the other on a standard logistic full-model approach, in both cases starting from all variables significant at univariable analysis. For the lasso-based procedure, hyperparameter λ (penalty weight) were tuned with a default grid search implemented in the R "glmnet" library, in a 10-fold cross-validation. The same conclusions (in terms of the selected variables) were obtained with the two strategies. We therefore reported as results the odds ratios and corresponding 95% CI, estimated by the standard logistic regression model. We built three nested-predictive models, including the qSOFA score, the most significant clinical variables, and the biomarkers selected by the LASSO method. This was done both in undifferentiated patients with suspected sepsis at ED admission and in those, among them, definitively diagnosed with sepsis.
Independent predictors were incrementally added to the qSOFA score starting from the easiest and cheapest to collect in clinical practice (i.e., from the clinical variables), through the conventional biomarkers and ending with the experimental biomarkers (unavailable for prime time). For the undifferentiated patients with suspected sepsis, the three predictive models were built as follows: Model 1 included the qSOFA score and clinical variables; Model 2 encompassed the qSOFA score, clinical variables, and conventional biomarkers; and Model 3 incorporated all of the above items and the experimental biomarkers. The same procedure was applied for the patients definitively diagnosed with sepsis (the models were termed Model 4, Model 5, and Model 6, respectively). Discrimination power was quantified using C statistics (area under the receiver-operating characteristic curve [AUROC]), and the De Long test was calculated to compare model AUROCs. According to the Youden Index method, different cut-offs were estimated between models on the estimated predicted probabilities of event, and compared in terms of sensitivity, specificity, positive predictive values (PPVs), negative predictive values (NPVs), and positive/negative likelihood ratios (LRs). The tests between NPVs were performed using the generalized score statistics [24]. All p values were two sided and considered significant when <0.05. All analyses were performed using the IBM SPSS statistic 24 software and the R software, R Core Team (2019), using the libraries "rms," "Hmisc,", "glmnet", "OptimalCutPoints," and "DTComPair".

Group 1: Undifferentiated Patients with Suspected Sepsis at ED Admission.
Of the 1132 undifferentiated patients with suspected sepsis, 304 were excluded because of missing data ( Figure S1 available in the Supplement). We did not find a statistically significant difference between complete and incomplete cases except for median Charlson Index, sTREM-1, and presepsin concentrations (higher in complete than in noncomplete data, p = 0.01). Baseline characteristics of the 828 patients in Group 1 and their etiology of SIRS at the end of clinical work up are summarized in Table 1. Abbreviations: OR = odds ratio, IQR = interquartile range, SOFA = sequential organ failure assessment, Qsofa = quick sepsis-related organ failure assessment, RR = respiratory rate, ni-SIRS = non infective systemic inflammatory response syndrome, d-SIRS = debatable systemic inflammatory response syndrome, sIL2Rα = soluble IL-2 receptor α, sTREM-1 = soluble triggering receptor expressed on myeloid cell-1, sPLA 2 GIIA = soluble phospholipase A 2 group IIA, and MR-proADM = mid-regional proadrenomedullin.ˆData are n • (%) unless otherwise specified.

Group 2: Patients with Suspected Sepsis at ED Admission who Received a Definitive Diagnosis of Sepsis at the end of Clinical Work-Up.
A total of 729 patients with suspected sepsis were definitively diagnosed with sepsis. There were 184 incomplete cases ( Figure S1, available in the Supplement). Only the median concentration of sTREM-1 was significantly higher in patients with complete data than in those with incomplete data (p = 0.03). Table 3 summarizes the baseline characteristics of patients included in Group 2 (545), their etiology, and source of infection. Abbreviations: OR= odds ratio, IQR= interquartile range, SOFA= sequential organ failure assessment, qSOFA=quick sepsis-related organ failure assessment, RR=respiratory rate, LRTI=lower respiratory tract infections, BSI=blood stream infections, MR-proADM=mid-regional proadrenomedullin, sIL2Rα=soluble IL-2 receptor α, sTREM-1=soluble triggering receptor expressed on myeloid cell-1, sPLA 2 GIIA= soluble phospholipase A 2 group IIA, and sTNFR-1-= soluble tumor necrosis factor receptor-1 (sTNFR-1). Data are n • (%) unless otherwise specified.
A total of 122 patients died within 30 days from ED admission (23%). In Group 2, we did not find a statistically significant difference between the AUROC of the qSOFA and SOFA score in predicting sepsis outcome (0.66; 95% CI 0.61-0.71 and 0.65; 95% CI 0.59-0.69, respectively; p = 0.5639).
The same method of analysis was applied in Group 2 as in Group 1. The only difference was that, in Group 2, sTNFR-1 was added to the panel of biomarkers already tested in Group 1. This choice was elicited by the recent findings obtained in favor of using sTNFR-1 for sepsis prognostication in the intensive care unit (ICU) [17]. In accord with the final list of the independent predictors of 30-day mortality (Table S2, available in the Supplement), the three predictive models were built as follows: Model 4 included gender, age, Charlson Index, qSOFA, body temperature < 36 • C, and heart rate >90/min (eqSOFA 2 ); Model 5 added CRP to eqSOFA 2 (eqSOFA 2 +CPR); and Model 6 enclosed eqSOFA 2 , CRP, and MR-proADM (eqSOFA 2 +CRP+MR-proADM). Sensitivity, specificity, NPV, PPV, LR-, and LR+ of the three models are reported in Table 4.

Discussion
Patients initially suspected to have an infection, but who were deemed ni-SIRS on expert review, were excluded from the first prospective study, which was conducted to validate the prognostic accuracy of qSOFA in patients with suspected sepsis at ED admission [25]. In our study, 10% of these patients had an ni-SIRS result at the end of clinical work-up. These epidemiological findings resembled that of previous works performed in the same setting [26,27], and reinforced the need to evaluate the prognostic performance of qSOFA in a population that mirrors "real life" patients with suspected sepsis at ED admission. Therefore, we assessed the predictivity of qSOFA not only in patients definitively diagnosed with sepsis (Group 2), but also in undifferentiated patients with suspected sepsis (Group 1). Interestingly, the prognostic accuracy shown by qSOFA in Group 1 and in Group 2 was similarly far from perfection (AUROC 0.67 and 0.66, respectively). To corroborate this finding, we have to highlight that an AUROC for mortality under 0.7 was also reached by qSOFA in the largest prospective study ever performed to assess its prognostic accuracy in patients with suspected sepsis at ED admission 3 .
As stated in the method section, the first step to enhance the predictivity of qSOFA entailed its combination with the clinical variables independently associated with the outcome in each cohort. In Group 1, this meant building a predictive model (eqSOFA 1 ) based on gender, age, Charlson Index score, qSOFA score, and three of the SIRS criteria (body temperature <36 • C, heart rate >90/minute, and white blood cell count>12,000/mm 3 ). The same clinical variables, except for white blood cell count >12,000/mm 3 , composed the eqSOFA 2 in Group 2. It follows that, eqSOFA is not a new warning score, but rather the synthesis, adjusted for age, gender, and comorbidities, of two existing ones. For the first time in the literature, the SIRS criteria and qSOFA were proven to act synergistically to predict outcome in patients with suspected sepsis at ED admission; furthermore, studies do not exist that have derived a biomarker-enriched model predictive of prognosis specifically for undifferentiated patients with suspected sepsis (eqSOFA 1 ) and for those, among them, definitively diagnosed with sepsis (eqSOFA 2 ). Notably, the prognostic performance of both eqSOFA 1 and eqSOFA 2 exceeded that of qSOFA (AUROC 0.79 and 0.73, respectively; all p < 0.001). When eqSOFA 1 and eqSOFA 2 were enriched with the biomarkers that were proved to be independently associated with the outcome in each group, their predictive ability of 30-day mortality was further enhanced. Of the six prognostic models derived in this study, eqSOFA 1 +CRP+lactate+MR-proADM in Group 1, and eqSOFA 2 +CRP+MR-proADM in Group 2 exhibited the highest AUROC for mortality with the maximum expenditure of biomarkers (0.83 and 0.80, respectively). A similar performance was reached by Mikacenic et al in the largest study ever conducted to derive and validate a biomarker-enriched model to predict sepsis outcome in the ICU (AUROC 0.79) [16]. Among all of the putative biomarkers, only IL-8 and sTNFR-1 were included in that "robust" model. We may speculate that if the prognostic performance of eqSOFA 1 +CRP+lactate+MR-proADM and eqSOFA 2 +CRP+MR-proADM was to be confirmed in the ICU setting, the key issue in favor of using our models would be that both of these model could be promptly implemented in clinical practice with CRP, lactate, and MR-proADM, unless IL-8 and sTNFR-1, are already available for prime time.
Seeing as the introduction of qSOFA, as compared with SIRS-based prognostication, has enhanced the specificity of the prediction of mortality associated with sepsis, but at the expense of sensitivity 2 , our analysis focused on predictive models that exhibited the highest NPV, possibly, with the minimum expenditure of biomarkers. The main goal of such an approach was to identify the models with the best ability to rule out the outcome, while easing the financial burden of the use of biomarkers in the health care system. The proper identification of subjects at low risk of death is particularly pressing in Europe, where, as in other countries with poor availability of ICU beds, most of the patients with suspected sepsis are initially managed in the ED and then transferred to general wards [28]. It would be desirable to identify a screening tool that could support ED physicians in the safe admission of patients with suspected sepsis outside the ICU, to avoid the risk of a less than optimum resuscitation. Unfortunately, qSOFA did not meet this need in either Group 1 (NPV = 0.84; 95% CI 0.82-0.90) or Group 2 (NPV = 0.80; 95% CI 0.79-0.87). Both eqSOFA 1 and eqSOFA 2 outperformed qSOFA in ruling out poor prognosis (all p < 0.001). However, depending on the etiology of SIRS, eqSOFA could be employed as a "stand alone" model for the safe identification of subjects at low risk of death among patients with suspected sepsis at ED admission. In fact, while the NPV of eqSOFA 1 was sufficiently high to support its use for a safe ruling out of patients at high risk of mortality (0.94; 95% CI 0.91-0.97), the NPV eqSOFA 2 was too low for this purpose (0.85; 95% CI 0.81-0.89). The contribution of biomarkers to further improve the NPV of eqSOFA was not straightforward in either group. In fact, the ability of eqSOFA 1 to rule out poor prognosis was not ameliorated by its enrichment with CRP, lactate, and MR-proADM (0.94), whereas CRP, and MR-proADM significantly increased the NPV of eqSOFA 2 to 0.93 (p < 0.001). Outstandingly, eqSOFA 1 and eqSOFA 2 +CRP+MR-proADM obtained the highest NPV to identify the patients at low risk of death, using as few biomarkers as possible, and without the need to maximize their threshold score [17]. We want to emphasize that eqSOFA 1 in Group 1 and eqSOFA 2 +CRP+MR-prodADM in Group 2 reached such a performance without worsening the corresponding PPV of qSOFA (Group 1 = 0.36 and 0.40, respectively; Group 2 = 0.40 and 0.42, respectively).
These findings can be translated to clinical practice as follows: eqSOFA 1 could be particularly helpful for recognizing subjects at low risk of death among patients with suspected sepsis at ED admission, for whom it is often challenging to pose a definitive diagnosis of sepsis, because the early presentation of many acute pathologies closely mimics that of sepsis, and the results of the microbiological work-up are usually not available. When the diagnosis of sepsis is already established (which occurs mainly in general wards), eqSOFA 2 +CRP+MR-proADM should be used in place of eqSOFA 1 , to rule out patients at high risk of mortality with the same high predictivity exhibited by eqSOFA 1 in undifferentiated patients.
This is the largest sample to date of patients with suspected sepsis at ED admission, who had all eight biomarkers measured. It is worth nothing that, for the first time in the literature, this panel of conventional and experimental biomarkers (including two of the most promising, MR-proADM and sTNFR-1) was compared side by side with qSOFA in the same study for sepsis prognostication. In the light of these premises, some interesting conclusions can be drawn. First, MR-proADM, as standalone biomarker to predict outcome, exhibited an AUROC higher than that of both qSOFA and all the other biomarkers independently from etiology of SIRS (Table S3 and Table S4). Second, among all the biomarkers assayed in our study, only CRP and MR-proADM were independently associated with 30-day mortality in either Group 1 or Group 2. Third, CRP and MR-prodADM proved to be addictive to eqSOFA in predicting prognosis in both cohorts. Fourth, lactate resulted an independent biomarker of prognosis only in Group 1: it contributed with eqSOFA 1 , CRP and MR-proADM to build the model with the highest AUROC to predict mortality (qSOFA 1 +CRP+lactate+MR-proADM). Fifth, biomarkers were useless in improving the NPV of eqSOFA 1 ; however, in Group 2, CRP and MR-proADM were necessary to significantly increase the ability of eqSOFA 2 in ruling out adverse outcome.
The study does have limitations. A total of 304 undifferentiated patients with suspected sepsis did not enter the final analysis. To mitigate the concerns related to the generalizability of eqSOFA 1 and eqSOFA+CRP+MR-proADM 2 , we validated our models in two cohorts of patients initially held out due to missing data. The details are discussed in the Supplement. In summary, in terms of AUROC, the prognostic performance of eqSOFA 1 and eqSOFA 2 +CRP+MR-proADM in the inception (0.79 and 0.80 [ Figure S2], respectively) and validation cohorts (0.80 and 0.82 [ Figure S3], respectively) was similar (p = 0.82, p = 0.81, respectively). Moreover, we did not assess the correlation of eqSOFA 1 and eqSOFA 2 +CRP+MR-proADM with the length of hospital stay, disease progression, or requirement for ICU care after ED admission.
In conclusion, we have derived and validated two prognostic models that were proven to be more robust than qSOFA and SOFA in predicting mortality and in ruling out the high risk of death among patients with suspected sepsis at ED admission, either undifferentiated (eqSOFA 1 ) or definitively diagnosed with sepsis (eqSOFA 2 +CRP+MR-proADM). The two predictive models, which were composed of clinical variables routinely measured at ED admission, and encompassed biomarkers that are already available for prime time, need to be externally validated prior to any clinical application.
Supplementary Materials: The following are available online at http://www.mdpi.com/2077-0383/9/4/1205/s1, Figure S1. Flow chart describing the design of the inception (Group 1 and Group 2) and validation cohorts. Figure S2. Receiver operating characteristic curve of "extended" quick sepsis-related organ failure assessment derived in Group 1 (eqSOFA1) to predict 30-day mortality in the validation cohort. Figure S3. Receiver operating characteristic curve of the "extended" quick sepsis-related organ failure assessment derived in Group 2 integrated with serum concentrations of C-reactive protein and mid -regional proadrenomedullin (eqSOFA2+CRP+MR-proADM) to predict 30-day mortality in the validation cohort. Table S1. Independent predictors of 30-day mortality in Group 1: multivariable analysis. Table S2. Independent predictors of 30-day mortality in Group 2: multivariable analysis. Table S3. Prognostic performance of the biomarkers assayed in Group 1. Table S4. Prognostic performance of the biomarkers assayed in Group 2.