Role of SpO2/FiO2 Ratio and ROX Index in Predicting Early Invasive Mechanical Ventilation in COVID-19. A Pragmatic, Retrospective, Multi-Center Study

The ability of COVID-19 to compromise the respiratory system has generated a substantial proportion of critically ill patients in need of invasive mechanical ventilation (IMV). The objective of this paper was to analyze the prognostic ability of the pulse oximetry saturation/fraction of inspired oxygen ratio (SpO2/FiO2) and the ratio of SpO2/FiO2 to the respiratory rate–ROX index–as predictors of IMV in an emergency department in confirmed COVID-19 patients. A multicenter, retrospective cohort study was carried out in four provinces of Spain between March and November 2020. The discriminative power of the predictive variable was assessed through a prediction model trained using a derivation sub-cohort and evaluated by the area under the curve (AUC) of the receiver operating characteristic (ROC) on the validation sub-cohort. A total of 2040 patients were included in the study. The IMV rate was 10.1%, with an in-hospital mortality rate of 35.3%. The performance of the SpO2/FiO2 ratio was better than the ROX index–AUC = 0.801 (95% CI 0.746–0.855) and AUC = 0.725 (95% CI 0.652–0.798), respectively. In fact, a direct comparison between AUCs resulted in significant differences (p = 0.001). SpO2 to FiO2 ratio is a simple and promising non-invasive tool for predicting risk of IMV in patients infected with COVID-19, and it is realizable in emergency departments.


Introduction
Coronavirus disease 2019 (COVID-19) has caused a major disruption in the normal operations of healthcare systems, demanding a reorientation of all routine care towards strategic medicine [1,2]. One of the primary challenges during the current pandemic has been the assessment of a large pool of patients affected by a new disease with no 2 of 12 clear specific symptomatology and few treatment options, but accompanied by a nonproportional increase in resources or hospitals capacity [3]. The overall front-line emergency system has been dramatically altered, from family centers and emergency medical services to emergency departments (ED) [4,5].
COVID-19 s ability to compromise the respiratory system has generated a proportion of critically ill patients who develop novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and who require invasive mechanical ventilation (IMV), intensive care unit admission, and who suffer from high associated mortality [6]. The identification of the so-called silent hypoxemia, characterized by a dissociation between oxygen saturation (SpO2) and ventilatory dynamics [7,8], has been strongly emphasized recently. Moreover, some studies have shown that delayed intubation may even worsen [9]. These observations highlight the need to identify objective measurements of early predictors of mechanical ventilation [10,11]. Under normal conditions, arterial blood gases can determine true oxygenation status but, in the current pandemic situation, non-invasive solutions are needed to guide actions as early as during hospital triage [12,13].
Typically, the ratio of SpO2/FiO2 (oxygen saturation measured by pulse oximetry/fraction of inspired oxygen) to respiratory rate-the ROX index-has been used as a predictor of high-flow nasal cannula failure/need for intubation [14]; however, its use has been extended to other clinical contexts, as is the case with COVID-19 [15,16]. As an alternative, the SpO2/FiO2 ratio has been proposed. Both can be performed at any time and under any clinical conditions, both continuously and non-invasively [17,18].
A study has suggested that the SpO2/FiO2 ratio could be used as a prognostic marker in the management of COVID-19 patients, with the objective of early improvement in the adjustment of treatments in the intensive care unit (ICU) [19]. It has also been evaluated as a possible index for triage upon admission of patients admitted for acute respiratory symptoms, particularly in the case of suspected COVID-19 [20]. Moreover, if respiratory rate is available, ROX index, (SpO2/FiO2)/respiratory rate) can also be obtained [15,21]; its implementation is an additional tool to help emergency services assess the clinical severity of COVID-19 patients, and it also serves as a safety measure during hospital discharges (10). It has even been shown to be potentially useful in guiding the decision to intubate COVID-19 patients, especially those with moderate to acute respiratory failure under non-ICU conditions [22].
The primary purpose of this study was to analyze the prognostic ability of both SpO2/FiO2 and the ROX indices in confirmed COVID-19 patients, during initial contact with ED, as predictors of prompt deterioration, as objectified by requirement IMV. The relationship between the performance of both parameters to the age of patients was also analyzed.

Study Design and Settings
A multicenter retrospective cohort study was carried out in the provinces of Palencia, Salamanca, Segovia, and Valladolid (Spain) between March and November 2020. The study involved ten advanced life support units, fifty-one basic life support units, and eight hospitals (three tertiary university hospitals, four general district hospitals, and one local hospital). All the facilities involved depend on the public health system-Sanidad de Castilla Y León (SACYL)-which is the major healthcare operator responsible for regional responses to the COVID-19 pandemic.
The institutional review board at the Hospital Universitario Rio Hortega (reference: PI 138/20) approved the study protocol, which was conducted in accordance with the Declaration of Helsinki, and we followed the Strengthening the Reporting of Observational studies in Epidemiology (STROBE) statement [23]. The institutional review board granted a waiver on the obligation of collecting informed consent from participants. The study handled data from de-identified subjects; all patients were anonymized.

Population
From all calls for medical assistance to the 1-1-2 emergency number, participants with suspected COVID-19 infection and those transferred with high priority by ambulance to the corresponding ED were identified. Thereafter, patients aged between 18 and 80 with a positive SARS-CoV-2 test, as shown by polymerase chain reaction (PCR), were recruited for the study.
The exclusion criteria included: patients under 18 or over 80 years, cases without analytical evidence of COVID-19 infection, patients with unknown comorbidities, and patients in whom it was not possible to collect parameters to calculate the analyzed indices.

Outcome
The primary outcome was the requirement for IMV. This outcome was adopted as an objective endpoint. In accordance with the recommendations of scientific societies [1,24,25], the upper cut-off age was set at 80 years, since these patients should preferably receive a high-concentration oxygen mask, high-flow oxygen therapy, or non-IMV of IMV. All IMV cases were re-checked by the head researcher.

Measures and Data Abstraction
Demographic covariates (age, sex, rural or urban area, and origin of nursery homes) were obtained from the standardized emergency medical services (EMS) medical records. Vital signs (respiratory rate, pulse oximetry saturation, systolic and diastolic arterial pressure, heart rate, and temperature) were recorded by an emergency registered nurse at the triage box upon initial access to the patient. Pulse oximetry saturation, systolic and diastolic arterial pressure, heart rate, and temperature were measured using the Connex ® Vital Signs Monitor (Welch Allyn Inc., Skaneateles Falls, NY, USA). Respiratory rate was calculated by counting respiratory cycles for 30 s. In cases of irregular or very shallow breathing, measurements were made by direct auscultation with a stethoscope. Glasgow coma scale and fraction of inspired oxygen whourere also recorded at the triage box.
Pulse oximetry saturation, fraction of inspired oxygen, and respiratory rate were used to calculate the SpO2/FiO2 ratio and the ROX index.
The following data were obtained by reviewing the electronic medical records: PCR test positive, hospital-inpatient and hospitalization time, ICU-admission date and stay period, IMV, in-hospital mortality, out-hospital mortality, aggregate mortality, and comorbidities to calculate the Charlson comorbidity index (CCI).
The hospital follow-up period for in-hospital patients lasted up to 120 days (patient of the cohort with the longest hospital stay).

Primary Data Analysis
A database with anonymized records was specifically created for this research. Prior to analysis, the case registry was checked for logic and range, removing duplicate or ambiguous entries. None of the recorded variables had more than 5% missing values. Once all the parameters were entered into the database, scores were calculated using XLSTAT ® BioMED for Microsoft Excel ® version 14.4.0 software (Microsoft Inc., Redmond, WA, USA).

Statistical Analyses
Absolute values and percentages were used to represent categorical variable; for continuous variables, median and interquartile ranges (IQR) were used because they did not follow a normal distribution (normal distribution was assessed by Shapiro-Wilk test). The univariable comparison between each independent variable and the main outcome (IMV), or age range, was assessed by the Mann-Whitney U test or chi-squared test, when appropriate.
To compare both indices, the whole cohort was randomly split into training and validation subsets; the training sample was used to build the model and the validation subset was used to determine the predictive validity of each score. In particular, the discriminatory validity of the scales for the primary outcome was assessed by the area under the receiver operating characteristic (ROC) curve (AUC), calculating the confidence interval (95% CI) obtained by resampling (or bootstrapping) 2000 iterations for each case. Moreover, the specificity, sensitivity, positive predictive value, negative predictive value, positive likelihood ratio, and negative likelihood ratio were calculated for each case. With the objective of comparing ROCs, a Delong's test was used. Finally, four groups of patients were derived from the following age ranges: <40, 41-55, 56-70, >70.
Data were analyzed using our own codes and base functions in R, version 4.0.3 (http://www.R-project.org; the R Foundation for Statistical Computing, Vienna, Austria).

Cohort Overview
A total of 2040 patients were included in the study ( Figure 1). The median age was 67 years (IQR: 55-75 years). Table 1 shows the global demographic characteristics and the comparison between IMV and non-IMV patients. The admission rate was 71.2% (1453 cases), with 10.1% (207 cases) of IMV. The in-hospital mortality rate was 35.3% (73 cases) in IMV vs. 9.7% (178 cases) of patients who did not require IMV. No differences in CCI between patients with or without IMV were found. or age range, was assessed by the Mann-Whitney U test or chi-squared test, when appropriate.
To compare both indices, the whole cohort was randomly split into training and validation subsets; the training sample was used to build the model and the validation subset was used to determine the predictive validity of each score. In particular, the discriminatory validity of the scales for the primary outcome was assessed by the area under the receiver operating characteristic (ROC) curve (AUC), calculating the confidence interval (95% CI) obtained by resampling (or bootstrapping) 2000 iterations for each case. Moreover, the specificity, sensitivity, positive predictive value, negative predictive value, positive likelihood ratio, and negative likelihood ratio were calculated for each case. With the objective of comparing ROCs, a Delong's test was used. Finally, four groups of patients were derived from the following age ranges: <40, 41-55, 56-70, >70.
Data were analyzed using our own codes and base functions in R, version 4.0.3 (http://www.R-project.org; the R Foundation for Statistical Computing, Vienna, Austria).

Cohort Overview
A total of 2040 patients were included in the study ( Figure 1). The median age was 67 years (IQR: 55-75 years). Table 1 shows the global demographic characteristics and the comparison between IMV and non-IMV patients. The admission rate was 71.2% (1453 cases), with 10.1% (207 cases) of IMV. The in-hospital mortality rate was 35.3% (73 cases) in IMV vs. 9.7% (178 cases) of patients who did not require IMV. No differences in CCI between patients with or without IMV were found.     8%), with both ICU-admission and IMV rates being extremely low (2.4%), without comorbidities, and with an associated mortality less than 1%. Range #2 (41-55 years, 18%, 367 cases) showed no differences in terms of sex, with a relatively low proportion of nursing home patients, a rate of ICU-admission and IMV of 8.4% and 6.8%, respectively, and with an aggregate mortality of 3.5% (13 cases). Range #3 (56-70 years, 33.7%, 687 cases) was mostly composed of males (61.39%), with 11.1% (76 cases) coming from nursing homes, with a mortality of 11.5% (79 cases), and with the highest ICU-admission and IMV rates of 13.4% and 13.2%, respectively. Finally, range #4 (older than 71 years, 40.2%, 821 cases) represented most of the cohort with the highest nursing home origin (23.8%), with ICUadmission and IMV rates of 11.9% and 10.6%, which are smaller than range #3; however, range #4 had the highest mortality rate in the cohort (26.3%). There is an inverse association between both the SpO2/FiO2 ratio and the ROX index and age, as both indexes decrease with increasing age; there is also a direct association between age and mortality (in-hospital and out-hospital), hospitalization time, and the presence of comorbidities.

SpO2/FiO2 Ratio and ROX Index Discrimination
The discriminatory validity of the SpO2/FiO2 ratio was better than the ROX index, with an AUC = 0.801 (95% CI 0.746-0.855) and an AUC = 0.725 (95% CI 0.652-0.798), respectively. In fact, the direct comparison between AUCs yielded a significant difference (p = 0.001). Additionally, the observed number of cases for the value of each index is shown in Figure 2A for the case of SpO2/FiO2 ratio and Figure 2B for the case of ROX index, which also shows the predicted probability of IMV according to the value of the index. Supplementary Figure S1 showed both AUCs (SpO2/FiO2 ratio in grey and ROX index in black); further details of the discriminatory capacity can be found in the Supplementary  Table S1. spectively. In fact, the direct comparison between AUCs yielded a significant difference (p = 0.001). Additionally, the observed number of cases for the value of each index is shown in Figure 2A for the case of SpO2/FiO2 ratio and Figure 2B for the case of ROX index, which also shows the predicted probability of IMV according to the value of the index. Supplementary Figure S1 showed both AUCs (SpO2/FiO2 ratio in grey and ROX index in black); further details of the discriminatory capacity can be found in the Supplementary Table S1.

SpO2/FiO2 Ratio and Age Association
As the SpO2/FiO2 ratio performed better than the ROX index, the possible role of age on SpO2/FiO2 ratio performance was assessed by determining discriminatory capacity in the four age ranges. As shown in Figure 3, the AUC decreases with age, resulting in an AUC = 0.925 (95% CI 0.836-1) in the youngest group and an AUC = 0.711 (95% CI 0.655-0.767) in the oldest group.

SpO2/FiO2 Ratio and Age Association
As the SpO2/FiO2 ratio performed better than the ROX index, the possible ro on SpO2/FiO2 ratio performance was assessed by determining discriminatory cap the four age ranges. As shown in Figure 3, the AUC decreases with age, resulti AUC = 0.925 (95% CI 0.836-1) in the youngest group and an AUC = 0.711 (95% C 0.767) in the oldest group.
Lastly, using previous SpO2/FiO2 ratio cutoff points considered for the de tion of mortality risk groups (18)

Discussion
To our knowledge, this is the first multicenter, derivation-validation, progn hort study comparing the diagnostic accuracy of the SpO2/FiO2 ratio with the RO to predict IMV as a primary outcome in COVID-19 patients transferred by ambu an ED. In the current study, we found that the SpO2/FiO2 ratio had better accur Lastly, using previous SpO2/FiO2 ratio cutoff points considered for the determination of mortality risk groups (18), we assessed the percentages of IMV for each group; the high-risk group (50-100) presented 44%, the intermediate group (101-426) presented 25%, and the low risk group (427-476) presented 5% of patients on IMV.

Discussion
To our knowledge, this is the first multicenter, derivation-validation, prognostic cohort study comparing the diagnostic accuracy of the SpO2/FiO2 ratio with the ROX index to predict IMV as a primary outcome in COVID-19 patients transferred by ambulance to an ED. In the current study, we found that the SpO2/FiO2 ratio had better accuracy than the ROX index in predicting IMV.
In the final cohort, IMV rate was 10.1%, a percentage such as that observed in other studies (13%) [26]. Patient in-hospital mortality with IMV was 35.3%, a percentage slightly lower than the one already reported [26]; this is perhaps due to the lower percentage of elderly patients. IMV was higher for patients not living in nursing homes, in older patients (68 vs. 66 years, p < 0.001), and in women, in accordance with previous studies [27]. Interestingly, CCI was not a critical factor between IMV and non-IMV patients.
The COVID-19 pandemic has created significant pressure on healthcare systems, and the burden of patient suffering has also affected the availability of resources such as ventilators [27]. Given this exceptional situation, consensus works from ethics groups of medical societies have been published, providing guidelines and recommendations to deal with resource shortages; these interventions assist decision-making criteria for the adoption of patient prioritization [1,24,25]. This certainly shows the necessity of a proper triage of patients to optimize available resources.
Different studies have shown that the early warning scores usually applied, such as the Modified Early Warning Score and Quick Sequential Organ Failure Assessment score, were inadequate to accurately predict respiratory failure in COVID-19 patients [28,29]. On the one hand, recent data indicated that the ROX index had moderate utility in predicting IMV in patients infected with COVID-19, especially in cases with moderate to acute respiratory failure [30]. Suliman et al. [12] showed that AUC value, as a predictor of IMV, on the first day of admission in non-ICU-conditions was 0.897; however, this was in a small study of 69 patients with pneumonia, so studies with larger sample sizes and patients with varying degrees of severity will be necessary to support these results. On the other hand, prepandemic studies showed that, in acute hypoxemic respiratory failure [31], the value of the SpO2/FiO2 ratio serves as control during noninvasive mechanical ventilation [32], or as a proxy measure for the calculation of the sepsis-related organ failure assessment score when partial pressure of oxygen in arterial blood is not available [33]; however, evidence related to COVID-19 is more limited. In this sense, Catoire et al. has suggested that the SpO2/FiO2 ratio is a reliable tool for hypoxemia screening in triage among patients admitted to the ED with respiratory symptoms [20]. Likewise, Lu et al. have shown that it could be used to improve the early adjustment of treatments-such as non-invasive or invasive ventilators, high-flow nasal cannula oxygen therapy, and extracorporeal membrane oxygenation-in the ICU [19]. Nonetheless, it has not yet been postulated as a clear predictor of orotracheal intubation. There are some novel models that predict, with similar AUCs as the one from SpO2/FiO2 ratio described here, the need for IMV in COVID-19 patients, but these scores require a blood sample that cannot be available immediately upon ED admission [34].
During the pandemic, a great deal of emphasis has been placed on so-called silent hypoxemia, which is characterized by a dissociation between SpO 2 and ventilatory dynamics [35,36]. The ventilatory response to hypoxemia is highly variable and is usually observed oin people who practically do not increase their ventilation when their oxygen decreases [7,8,37]. This unique pathophysiology of the disease could explain a decrease in the diagnostic accuracy of the ROX index when compared to SpO2/FiO2, though this could also be affected by an underestimated/incorrect evaluation of respiratory rates.
Assessment of the oxygenation status of critical patients is of utmost importance to plan interventions and prognosis. We demonstrated the prognostic value of SpO2/FiO2 in COVID-19 patients where its decreasing trajectory was directly associated with age and an increased risk of deterioration and mortality. These results support those published by Lu et al. and Catoire et al. [19,20].
A variable of particular interest to consider when interpreting the SpO2/FiO2 ratio is age [38,39]. The SpO2/FiO2 ratio is more clearly predicted in younger patients (<40 years); a reason for this could be found by the fact that pre-existing co-morbidities may not represent a limitation for orotracheal intubation, which translates into a higher correlation between the ratio value and the requirement of IMV. The index should be used with greater caution in the group over 70 years old.
To determine to what extent the ratio could be useful as a tool to predict outcomes, we have chosen to classify the patients into three groups, with SpO2/FiO2 cut-off points previously considered for the determination of mortality risk groups [18]. Our results show that we must carry out a closer study of intermediate (SpO2/FiO2 101-426) and high-risk (SpO2/FiO2 50-100) patients, as these groups represent an increased requirement for IMV (25% and 44%, respectively) as compared to the low-risk group (SpO2/FiO2: 427-476, with 5% of patients with IMV).
Avoiding emerging procedures during the COVID-19 pandemic is important due to the risk of viral transmission to health personnel [28]. Therefore, a highly specific model can avoid unnecessary intubation [40]. This lends even more value to the performance of the SpO2/FiO2 ratio, since we obtained a specificity of 89.49 (95% CI: 87.65-91.32). These data reaffirm the predictive capacity of the SpO2/FiO2 ratio, a crucial piece of information during the extreme phases of the current pandemic, when it was necessary, on numerous occasions, to select the order of intubation or, in the worst scenario, to decide which patients most fulfilled the criteria for IMV. SpO2/FiO2 can help in the critical decision-making process, representing a straightforward tool to implement in the system that can clarify required interventions.

Limitations
Our investigation is not free of limitations. First, SARS-CoV-2 virological status in our study is based on the presence of a positive PCR. The number of false positives is likely to be small, but we should assume the existence of false negatives. Second, because the standardization of intubation was not decided a priori, one could argue that the outcome of mechanical ventilation was somewhat subjective, which could be a function of local practices; since all eight hospitals cooperated in previous common studies, the guidelines for airway and ventilation management were similar. Third, the population sample collected during this period cannot be considered representative of a standard sample of patients transferred by the EMS due to the very character of the pandemic. Likewise, a patient selection bias may be present in this manuscript due to non-probability sampling methods. To minimize bias, the study involved units working in urban and rural areas all over 24 h and 7 days a week. Fourth, the data extractors were not blind. To ensure that the results were not a matter of interpretation, a two-step verification between an associate investigator from each hospital and the principal investigator was performed in those cases requiring invasive mechanical ventilation. Previously validated indices were used to minimize possible interobserver bias among researchers. Finally, it should be noted that we are dealing with a retrospective study, with the methodological drawbacks that this may entail. As soon as the pandemic allows health systems to work in an ordinary way, it is necessary to carry out prospective studies to determine the true nature of the biomarkers analyzed here.

Conclusions
The current study suggested that the SpO2/FiO2 ratio is a simple, non-invasive, and promising tool for predicting the risk of IMV in patients infected with COVID-19. It can also be performed in an ED. The timely identification of these cases could help to improve survival rates, together with the reasonable and appropriate cost-effective allocation of resources. The use of the SpO2/FiO2 ratio can help in performing an early estimation of the degree of hypoxemia in patients infected with COVID-19, even in patients who are seriously ill or at high risk of clinical deterioration, but with a low initial suspicion of infection; thus, overall survival might be improved.
In summary, the SpO2/FiO2 ratio presents a good combination of precision, noninvasiveness, and speed, all of which are especially useful in scenarios in which a blood gas analyzer is not available. This could help EMS personnel to assess clinical severity in complex decision-making processes during ED procedures.
Supplementary Materials: The following are available online at https://www.mdpi.com/article/10 .3390/biomedicines9081036/s1, Figure S1: Area under the receiver operating characteristic (ROC) curve (AUC) for SpO2/FiO2 ratio in grey and ROX index in black, Table S1: Further details of the predictive capacity of SpO2/FiO2 ratio and ROX index.  Informed Consent Statement: Patient consent was waived due to the fact that the study handled data from de-identified subjects; all patients were anonymized.

Data Availability Statement:
The data of the study are available to other researchers upon reasonable request from the corresponding author.