1. Introduction
The recent COVID-19 pandemic put our healthcare system under extreme pressure, including through the overwhelming requirement for intensive care unit (ICU) beds. With increasing infection rates, the number of COVID-19 patients requiring critical care beds increased, far exceeding the maximum number of ICU beds available with care providers [
1]. Due to the burden of COVID-19 cases, many elective surgeries were indefinitely postponed, and much of the workload was focused on emergency cases and the care of COVID-19 patients. Hospital beds that would normally be used for elective surgeries had to be utilised for managing COVID-19 cases [
2]. Category 5a COVID-19 was defined as a critically ill COVID-19 patient with multiorgan involvement requiring assisted ventilation, which was provided at least via high-flow nasal cannula (HFNC) support. During the peak of the pandemic, many patients rapidly developed severe hypoxic respiratory failure and acute respiratory distress syndrome, part of category 5a, requiring ICU admission. At the height of the pandemic, in August 2021, the Malaysian Ministry of Health reported a nationwide peak of 924 ventilated COVID-19 patients and 1648 ICU beds in use [
3]. Therefore, there was a need for reliable disease severity scoring tools that could be applied to assist in the appropriate direction of patient care, as well as for systematically utilising ICU beds and treatments based on the severity of illness.
The Sequential Organ Failure Assessment (SOFA) is a simple mortality prediction score used in critically ill patients based on the degree of dysfunction in six organ systems (
Appendix A and
Appendix B) [
4,
5]. It assesses the degree of organ dysfunction and, consequently, the mortality risk of patients in the ICU. The worst parameter score is calculated at admission and every 24 h until discharge.
Several retrospective studies investigated the association between SOFA and COVID-19 [
6,
7,
8,
9,
10]. Gowda et al. [
6] suggested that the SOFA scoring system may be better suited for assessing COVID-19 patients compared to the Acute Physiological and Chronic Health Evaluation score, particularly for those requiring HFNC. Leoni et al. [
7] found that SOFA and other factors, namely, age, obesity, procalcitonin and partial pressure of the oxygen/fractional inspired oxygen (PaO
2/FiO
2) ratio, were independently associated with 28-day mortality in critically ill COVID-19 patients. Yang et al. [
8] concluded that the SOFA score could be used to evaluate severity and 60-day mortality in severely ill COVID-19 patients. Fayed et al. [
9] found that a SOFA score of less than 2 was associated with 100% survival, while a score of more than 11 was associated with 100% mortality. Ramlan et al. [
10] found that the SOFA score demonstrated very strong discrimination and good calibration in predicting 28-day mortality in critically ill COVID-19 patients.
The abnormal and uncontrolled production of multiple cytokines, known as cytokine release syndrome (CRS), has been observed in critically ill patients with COVID-19 pneumonia and represents a major factor contributing to COVID-19 mortality [
11,
12]. CRS, which could cause multiorgan dysfunction syndrome, might occur from the time of diagnosis up to 5–7 days afterwards. After this window, the majority (approximately 80%) of patients tend to improve, whereas the remaining 20% patients progress to severe pneumonia, with approximately 2% mortality. We embarked on this study to determine the accuracy of the SOFA score in predicting COVID-19 mortality in patients with category 5a illness and above.
2. Materials and Methods
This retrospective study was reviewed and approved by the research committee of the Department of Anaesthesiology, Faculty of Medicine, Universiti Kebangsaan Malaysia (UKM), prior to approval from both the UKM Research Ethics Committee (approval code: FF-2022-256, approved on 28 July 2022) and the Medical Research and Ethics Committee. This study was also registered in the National Medical Research Register as ID-22-01303-J9U (11R).
After obtaining approval from the directors of Serdang Hospital and Hospital Canselor Tuanku Muhriz, (HCTM) UKM, which were tertiary COVID-19 referral centres, a retrospective review of patients admitted to the COVID-19 ICUs of both hospitals between January and December 2021 was performed. The inclusion criteria were patients aged 18 years old and above, with COVID-19 category 5a and above, who required ventilatory support for either HFNC, non-invasive ventilation, or invasive ventilation. Any pregnant COVID-19 patients admitted to the ICU were excluded from this study. Patients with incomplete data were also excluded.
The data of the SOFA scores were recorded at three time points: on day 1, day 3 and day 5 of each patient’s COVID-19 ICU stay. These 3 time points of evaluation were chosen because CRS was expected to occur within these days [
11,
12]. The following data were obtained from existing records: gender, race, comorbidities, clinical frailty score (CFS) on admission, and early morning biochemical parameters related to the SOFA score. Specific comorbidities included in the data collection were diabetes mellitus (DM), hypertension (HPT), chronic kidney disease (CKD) or end-stage renal failure (ESRF), ischaemic heart disease (IHD), dyslipidaemia, obesity, bronchial asthma, chronic lung disease and stroke.
The SOFA score comprised 0 to 4 points assigned to each of 6 organ systems based on the ratio of PaO
2 to the fraction of inspired O
2 (PFR), the Glasgow Coma Scale (GCS) score, the mean arterial pressure (MAP), the serum creatinine level, the bilirubin level, and the platelet count (
Appendix A). Higher scores indicated worse organ function. The CFS is a 9-point scale that quantifies frailty based on function in individual patients, with a higher score denoting an increased the level of frailty.
The SOFA scoring trend and the highest score were determined. The SOFA scoring trend was a change in SOFA scores over time between the evaluated times. An increased SOFA score was recorded when the subsequent score on day 3 or 5 was higher than that on day 1. A maintained score was recorded when there was no score change over the three time points. A decreased SOFA score was defined as when the subsequent score on day 3 or 5 was lower than that on day 1. Patients who did not survive until day 3, thus only having SOFA scores for day 1, were excluded from data analysis.
Apart from the trend, the highest score (excluding those who only had a SOFA score for one evaluation time point) was also determined for each patient and the percentage of mortality was compared with the patient outcomes. The main outcome variables were defined as either survivors or non-survivors. Patients were grouped as survivors when they survived up to 28 days after hospital discharge. Non-survivors were defined as patients who died during the same hospitalisation, either in the COVID-19 ICU or after transfer to general ward within 28 days of hospitalisation. The extent of and changes in the scoring trend were also analysed statistically using an independent-sample t test and Pearson chi-squared test, and the cut-off SOFA scores between survivors and non-survivors were also derived via receiver operating characteristic (ROC) analysis.
The sample size was calculated based on the Krejcie–Morgan (1970) [
13] formula. Based on the population size of COVID-19 patients admitted to both units in 2021 (599 in Serdang Hospital and 336 in HCTM) and a 5% margin of error, at a 95% confidence interval (95% CI), the calculated sample size was 383. Considering the 5% dropout rate, 400 patients were recruited. The data were analysed using the Statistical Package for the Social Sciences (SPSS) for Windows version 26.0 (IBM Corp, Armonk, NY, USA). Descriptive statistics was used to describe the demographic and clinical characteristics of the patients. The distribution of continuous variables was explored using skewness, kurtosis, and histograms. Continuous variables were presented using mean ± standard deviation (SD) if the data were normally distributed or median [25th percentile, 75th percentile] for skewed data. Categorical variables were presented as frequency with percentage. The association between the changes in SOFA score trends with mortalities were explored using independent-sample
t tests, the Pearson chi-squared test, Fisher’s Exact test and the Mann–Whitney U test. A repeated-measure ANOVA with Bonferroni correction for pairwise correction was performed to compare the changes in the SOFA score from day 1 to day 3 and day 5. Moreover, receiver operating characteristic (ROC) analyses were used to explore the ability of SOFA scores in predicting mortality and to determine an optimal SOFA cut-off score for mortality. All the tests were two-sided, and statistical significance was denoted by
p < 0.05. The factors associated with non-survivors (deaths) were explored using logistic regression (LR). Univariable model simple LR was first used for variable exploration to investigate the factors significant for the outcome death, and variables with
p < 0.250 were chosen in the variable selection process for the multivariable model using the forward LR selection method. The model fit of the final model was checked using the Hosmer–Lemeshow goodness-of-fit test, a classification table, and the area under the ROC curve. All the tests were two-sided, and statistical significance was denoted by
p < 0.05.
3. Results
In this study, 400 patients were recruited. A total of 16 patients were excluded from the analyses because of missing data, leaving 384, which were equally distributed between Serdang Hospital and HCTM, as shown in
Figure 1. Out of the 384 patients, 27 were excluded from analysis because they only had SOFA scores for day 1. Of the remaining 357 patients, almost half (49.3%) died.
Table 1 shows the patients’ demographic, comorbidity and clinical characteristic data. Patients’ gender and ethnicity did not affect their survival. Patients who died were significantly older, with significantly more comorbidities. The severity of a patient’s frailty affected their survival. The post hoc analysis revealed that those who survived were more likely to have low CFSs. For all three time points used in the SOFA score evaluation, patients who survived had significantly lower mean SOFA scores than those who died. Furthermore, as the day passed in the ICU, SOFA scores improved among the patients who survived, whereas SOFA scores remained high with minimal changes in those who died. Post hoc analysis revealed that those who were alive had a significantly decreased SOFA score on day 5 compared to day 1, while patients who died had a significantly increased SOFA score on day 5 compared to day 1. Although the increase in SOFA score from day 1 to day 5 was small and remained high for non-survivors compared to survivors, it still represented a significant rise when analysed further (
Table 2).
The trends of change in the SOFA score over the three time points among the 357 patients were further categorised into increased scores (positive value), maintained scores (unchanged value) and decreased scores (negative value), and associations with mortality are shown in
Table 3. There were significant differences in the SOFA score trend across time between survivors and non-survivors. When analysed further, there was a statistically significant relationship between the mortality outcome and the SOFA score trend.
Table 4 shows the association between all 176 non-survivors, demographic parameters, clinical characteristics, comorbidities and mortalities. Based on logistic regression analysis, the factors significant in the univariable exploration were age, length of ICU stay, CFS, DM, HPT, CKD/ESRF, IHD, highest SOFA score, number of comorbidities and endotracheal intubation. These significant factors were entered into the multivariable models. Following the variable selection process, the significant factors associated with mortality after eliminating confounding effects were age, length of ICU stay, CFS, IHD, highest SOFA score and endotracheal intubation. There was a 3% increase in the odds of mortality with each additional year of the patient’s age and 5% increase with each additional day of their ICU stay. A unit increase in the CFS increased the odds of mortality by 56%, while a unit increase in the highest SOFA score increased the odds of mortality by 39%. The presence of IHD as a comorbidity increased the odds of mortality by almost 4 times. Patients who underwent endotracheal intubation had 5 times higher odds of mortality compared to non-intubated patients.
The ROC curve analysis of the 357 patients (
Figure 2) was performed to explore the ability of the SOFA score to predict mortality outcomes. For any diagnostic technique to be meaningful, the AUC must be greater than 0.5 or 0.8 and be fair and acceptable [
13]. The observed area under the curve (AUC) was fair [AUC (95% CI): 0.796 (0.750–0.841)], with a cut-off SOFA score of ≥9 derived from our population data, yielding sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of 75.57% (95% CI: 71.11–80.03%), 67.40% (95% CI: 62.54–72.27), 69.27% (95% CI: 64.48–74.06) and 73.94% (95% CI: 69.39–78.49), respectively [
13,
14]. A total of 75.57% of patients who died had a SOFA score of nine or higher, while 67.40% of patients who survived had scores below nine. Of those who had SOFA scores of at least nine, 69.27% died, while for those with scores less than nine, 73.94% survived.
Another ROC curve analysis was performed to explore the utility of the SOFA score in predicting mortality outcomes among patients with different SOFA trends (increased, maintained and decreased score trends), as shown in
Figure 3 and
Figure 4. The observed AUC was good for assessing increased [AUC (95% CI): 0.856 (0.786–0.926)], maintained [AUC (95% CI): 0.800 (0.701–0.900)] and decreased [AUC (95% CI): 0.779 (0.712–0.847)] SOFA scores. A cut-off SOFA score of ≥9 yielded sensitivity, specificity, PPV and NPV of 83.95% (95% CI: 77.18–90.72), 65.63% (95% CI: 56.87–74.38), 86.08% (95% CI: 79.69–92.46) and 61.76% (95% CI: 52.80–70.72) for patients with increased SOFA scores, as well as 81.85% (95% CI: 76.04–87.60), 60.34% (95% CI: 53.01–67.68), 49.45% (95% CI: 41.96–56.94) and 87.50% (95% CI: 82.54–92.46) for patients with decreased SOFA scores. Moreover, for patients who maintained their SOFA scores during their ICU stay, a cut-off of ≥7 yielded sensitivity, specificity, PPV and NPV of 87.50% (95% CI: 79.91–95.09), 51.52% (95% CI: 40.05–62.98), 68.63% (95% CI: 57.98–79.27) and 77.27% (95% CI: 67.66–86.89), respectively.
4. Discussion
Introduced by Vincent et al. in 1996, the SOFA score was initially designed to sequentially assess the severity of organ dysfunction in critically ill septic patients [
5]. Although COVID-19 mainly affects the respiratory system, studies have shown that it can induce the activation of the complement and coagulation systems, thus leading to multiorgan system failure and contributing to higher SOFA scores [
15]. In our study, 49.3% mortality was reported as an outcome for critically ill COVID-19 patients in 2021, and this group showed a higher SOFA score trend, with a mean SOFA score of 10.71 compared to 7.15 in the survivors’ group. This finding is comparable with a study conducted by Fayed et al. [
9], where 47% mortality was reported out in 111 critically ill COVID-19 patients, with the majority demonstrating a SOFA score of >11.
In terms of demographic characteristics, age was a consistent predictor of mortality in the majority of worldwide COVID-19 cases, similarly to our study [
16]. Gender, on the other hand, did not influence survival in our study, whereas in the study of Isamaeili Tarki et al. [
16], there was a significant predominance of male patients among the non-survivors. Looking at the comorbidities, Isamaeili Tarki et al. [
16] also detailed that diabetes mellitus and renal disease were significantly more prevalent in the non-survivor group (
p < 0.001 for both). Isamaeili Tarki et al. [
16] noted no differences in the prevalence of HPT, coronary artery disease, cerebrovascular disease and pulmonary disease between groups. In our study, HPT, renal disease, IHD and living with obesity were significant comorbidities associated with higher mortality, after eliminating confounding effects.
While several studies have been performed since the start of the COVID-19 pandemic, our study focused on the changes in SOFA score trends, i.e., whether they increased, were maintained or decreased. We also evaluated the highest SOFA score at 3 time points during the CRS period, with the relationship with mortality considered as an outcome. By looking at the highest SOFA score, we found that a unit increase in the score increased the odds of mortality by 39%. Our study observed a significant association between changes in SOFA score mortality outcome, as increased mortality was observed in those with an increasing SOFA score. For the non-survivor group, the SOFA score remained high at all three time points, although the increase from day 1 to day 5 was small. However, this increase was deemed statistically significant when evaluated further. Among the survivor group, the reduction in SOFA scores over the days spent in the ICU was significant, and post hoc analysis revealed that the most significant decrease in SOFA score was between day 1 and day 5, meaning that clinical improvement was seen within 5 days. In other words, 3 days was not long enough to prognosticate such patients. Gruyters et al. [
17], on the other hand, reported that the highest SOFA score recorded during a patient’s ICU stay was most strongly associated with ICU mortality, while a significant increase in SOFA score in the first week was found in both groups, but the increase was less pronounced in survivors. Ismaeili Tarki et al. [
16] also reported that ICU-admitted patients with a greater-than-average mean SOFA score (one point higher) were found to be around 3.8 times more likely to die (
p < 0.001), and if the highest reported SOFA score of a patient throughout hospitalisation was another point higher, the patient was 2.7 times more likely to die (
p < 0.001) [
16].
As the ROC curve closer to the left upper corner is better at discriminating between true positive (mortality) and false positive (survivor) results, we found that our ROC curve demonstrated the optimum sensitivity, or true positive rate, and specificity at a cut-off SOFA score of ≥9. Our study showed that the AUC for the SOFA score predicting mortality in critically ill COVID-19 patients was 0.796, indicative of the appropriateness of the SOFA score for predicting mortality. However, when different SOFA trends were considered for ROC curve analysis, no differences were seen, as the observed AUC was good for all aspects, including maintained and decreased SOFA scores. In all groups, the SOFA score demonstrated a strong sensitivity and NPV, being effective at identifying patients likely to die and determining that those with lower scores were more likely to survive. The SOFA score cut-off in this study was effective but not flawless in predicting mortality, with a balance between detecting high-risk cases and not misclassifying too many low-risk patients. Fayed et al. reported that the AUC for SOFA for predicting mortality in patients with severe respiratory distress from COVID-19 pneumonia was 0.883, with a cut-off value of 5, and their study also comparatively suggested that a SOFA score of ≥5 can predict severity and mortality in patients with COVID-19 pneumonia [
9].
Besides the assessment of the score trends, patients’ premorbid conditions also have to be taken into consideration and should form part of triaging and resource allocation in ICUs. COVID-19 Decision Support Tools introduced by the NHS incorporated the CFS as one of the criteria for ICU admission triaging, and this system is sensitive and specific for predicting treatment limitation [
18]. Wilkinson et al. [
19] also stated that the frailty score is a good ICU triaging method, as it has been studied prospectively in multiple studies in many countries and is independently associated with intensive care outcomes. In our study, we evaluated other associated factors contributing to a higher risk of mortality. Based on the logistic regression analysis conducted in this study, significant factors associated with non-survivors after eliminating confounding effects included age, length of ICU stay, CFS, mechanical ventilation, high SOFA scores and underlying IHD, with the final factor increasing the odds of morbidity in non-survivors by almost four times. Hu et al. [
20] found that gender, age, IHD, HPT, thrombocytopenia and mechanical ventilation were independent risk factors for death in patients with severe and critical COVID-19 during hospitalisation. Khatib et al. [
21] performed a more detailed analysis of variables in their study, finding that, in their multiple logistic regression analysis, independent predictors of mortality in critically ill COVID-19 patients included age, chronic kidney disease, a higher neutrophil-to-lymphocyte ratio on admission, a low platelet count, a higher serum ferritin level on admission, and a higher serum bilirubin level on admission [
21].
COVID-19 infection can cause significant lung injury, as well as damage to the skin, blood system, endocrine system, neurological system, kidneys, liver and heart, potentially leading to a hyperglycaemic state, gastrointestinal manifestations, a high thrombotic state, acute coronary syndrome and arrhythmia [
22]. Rod et al. [
23] showed that 60 predictors can be used to assess the severity of COVID-19 infections, of which 7 are considered to be highly correlated and consistent, including the SOFA score, D-dimer, C-reactive protein (CRP), body temperature, albumin and diabetes mellitus. The result of the study revealed that the SOFA score and other factors are independent risk factors for in-hospital death.
When ICU resources are overwhelmed, they should be allocated to patients more likely to survive. Patients with the lowest SOFA score trends likely have the highest likelihood of survival with appropriate care and should, therefore, be given the highest priority for resources. Our analysis identified significant differences in patients’ outcomes with the SOFA score trend, showing that those who survived had decreased SOFA score trends compared to non-survivors. Thus, the SOFA score might be sufficiently significant to be used as a triaging tool in an ICU setting for COVID-19 patients. For our patients, the criteria for admission during the COVID-19 ICU bed shortage were based, in general, on a combination of age, comorbidities and frailty score.
Apart from the diagnostic and prognostic properties of the SOFA score, its interpretation can guide the healthcare worker to make decisions as part of resource allocation, especially in times of crisis. In our study, the aim was to prove that this scoring system is reliable for use as a prognostic indicator for mortality. Our study has proven that the SOFA score trend can assist us in terms of the prognostication of critically ill patients, which will guide us for resource allocation, triaging patients, and facilitating end-of-life care discussions with families. Further studies could be conducted to validate the prediction by utilising the SOFA-based model.
In terms of limitations, we did not extract other factors, including prognostic scores, such as the Acute Physiology and Chronic Health Evaluation (APACHE) II score; therapeutic management; other inflammatory biomarkers, such as CRP; and the time of treatment initiation. We did not collect data on days 2 and 4, which might have highlighted some changes. Furthermore, this study collected data from only two tertiary centres and could have benefited from including more centres. Future studies with more centres with better design, fewer limitations and incorporating other factors may provide a better insight into the predictive ability of the SOFA-based model.