Prognostic Performance of Bedside Lung Ultrasound Score (LUSS) and ROX Index in Hypoxemic Respiratory Failure Due to COVID-19

Background: Noninvasive ventilation, mainly helmet CPAP, was widely used during the COVID-19 pandemic, even outside of intensive care units. Both the ROX index and the LUS score (LUSS) have been proposed as tools to predict negative outcomes in patients with hypoxemia treated with noninvasive ventilation (NIV) outside of ICUs. We aim to evaluate whether the combination of LUSS with the ROX index improves the predictive performance of these indices in patients with hypoxemia due to COVID-19 pneumonia, treated with NIV outside of ICUs. Methods: This is a monocentric prospective observational study conducted at the university teaching hospital Fondazione IRCCS San Gerardo dei Tintori (Monza, Italy) from February to April 2021. LUSS and ROX were collected at the same time in noninvasively ventilated patients outside of the ICU. An LUS exam was performed by 3 emergency medicine attending physicians with at least 5 years’ experience in point-of-care ultrasonography using a 12-zone system. To evaluate the accuracy of the prognostic indices in predicting a composite outcome (endotracheal intubation and mortality), ROC curves were used. A logistic multivariable model was used to explore the predictors of the composite outcome of endotracheal intubation and in-hospital mortality. An unadjusted Kaplan–Meier analysis was used to explore the association with the composite outcome of survival without invasive mechanical ventilation at the 30-day follow-up by stratifying the 3 indices by their best cut-offs. Results: A total of 79 patients were included in the statistical analysis and stratified into 2 groups based on the presence of a negative outcome, which was reported in 24 patients out of 79 (30%). A great proportion of patients (66 patients—84%) were treated with helmet CPAP. All three indices (LUSS, ROX and LUSS/ROX) were independently associated with negative outcomes in the multivariable analyses. Although the comparison between the AUROC of LUSS or ROX versus LUSS/ROX did not reveal a statistically significant difference, we observed a trend toward a higher accuracy for predicting negative outcomes using the LUSS/ROX index as compared to using LUSS. With the Kaplan–Maier approach, all three indices stratified by the best cut-off reported a significant association with the outcome of 30-day survival without mechanical ventilation. Conclusions: A multimodal noninvasive approach that combines ultrasound (i.e., LUSS) and a bedside clinical evaluation (i.e., the ROX index) may help clinicians to predict outcomes and to identify patients who would benefit the most from invasive respiratory support.


Introduction
Coronavirus disease (COVID-19) rapidly spread worldwide and constituted a public health emergency. The clinical presentation is highly variable, with most people with the infection expressing mild-to-moderate symptoms and about 20% of patients developing severe respiratory failure [1]. Acute respiratory failure due to COVID-19 may cause profound hypoxemia, chest X-ray infiltrates and dyspnea with a high morbidity and mortality [2].
Noninvasive ventilation (NIV) was widely used during the COVID-19 pandemic, when an overwhelming number of patients needed respiratory assistance. The rapid shortage of intensive care unit (ICU) beds forced healthcare professionals to apply different types of noninvasive respiratory support, including helmet continuous positive airway pressure (CPAP), noninvasive pressure support ventilation (NPPV) and high-flow nasal oxygen (HFNO), not only in ICUs but also in emergency departments and regular wards [3,4].
The respiratory rate-oxygenation (ROX) index has been proposed as an easy tool to identify patients treated with noninvasive ventilation at high risk of worsening. The ROX index has been shown to predict treatment failure and the need for endotracheal intubation within 12 h from HFNO onset [5]. Recently, the role of the ROX index in identifying patients with COVID-19 with dyspnea at high evolutionary risk has been demonstrated [6].
Ultrasonography is a feasible bedside and radiation-free tool to monitor the clinical course of respiratory failure at any time. Lung ultrasound (LUS) is widely used as a diagnostic technique and as a guide in quick therapeutic decision making in critical care settings. It is an interesting tool in the evaluation of respiratory failure etiology [7] and has been recently demonstrated to be a useful monitoring device for the clinical history of ARDS [8][9][10].
During the COVID-19 pandemic, the clinical use of the LUS score (LUSS) increased. The LUS findings described in COVID-19 pneumonia include vertical pneumogenic artifacts (B-line patterns), subpleural consolidation and alterations of the pleural line, similar to those described in patients with non-COVID pneumonia [11][12][13]. In COVID-19 respiratory failure, LUS provides risk stratification, as it has been demonstrated to be a good predictor of death, ICU admission and endotracheal intubation [14][15][16]. In this context, LUS may help clinicians to predict the clinical course of patients in and out of ICUs.
The aim of the present study is to evaluate whether the effect of incorporating the LUS score (LUSS), a semiquantitative score that measures lung aeration loss caused by different pathological conditions [7,17], in the already known ROX index improves its predictive performance in patients with hypoxemic respiratory failure due to COVID-19 pneumonia, treated with NIV outside of ICUs. We defined the LUSS/ROX index as the ratio between the LUS score, performed using a 12-zone scoring system that includes an ultrasound of the lung parenchyma and pleural line patterns, and the ROX index.

Materials and Methods
This is a monocentric prospective observational study conducted at the university teaching hospital Fondazione IRCCS San Gerardo dei Tintori, Monza, Italy, which served as a referral center for COVID-19 pneumonia during the COVID-19 pandemic. Data were collected from February to April 2021. The study was conducted in accordance with the Declaration of Helsinki. Patients' consent was waived due to the observational nature of the study and according to the Institutional Review Board approval. Data were collected as part of the STORM study (Spallanzani Institute approval number 84/2020; NCT04424992).
The inclusion criteria were as follows: • Age ≥ 18 years; • Confirmed COVID-19 pneumonia (positive test result using reverse-transcriptase polymerase chain reaction, RT-PCR, assay of a nasopharyngeal swab); Alert of the hospital Medical Emergency Team for clinical worsening: PaO 2 /FiO 2 ratio < 300 mmHg despite oxygen therapy, respiratory rate > 22 breaths per minute and/or respiratory distress.
The exclusion criteria were hemodynamic instability requiring vasoactive drugs and comorbidities with a life expectancy < 12 months. All eligible patients were enrolled. The following variables were collected: sex, age, body mass index (BMI), sequential organ failure assessment (SOFA) score, C-reactive protein (PCR) and applied oxygen support devices (i.e., HFNO, CPAP helmet and Venturi mask).

LUS Protocol
An LUS exam was performed within 4 days of hospital admission using a pre-defined step-by-step protocol as part of routine patient care management. A total of 3 emergency medicine attending physicians with at least 5 years' experience in point-of-care ultrasonography collected the US parameters. The patient was preferably examined in the sitting position or, when this position could not be maintained, in the semirecumbent position. We performed LUS applying a six-zone method for each lung. We obtained 12 zones, if considering the whole thorax. We explored the anterior, antero-lateral and posterior areas of the thorax. A 12-4 Mhz linear-array transducer was used (Lumify L12-4, (c) Philips, Amsterdam, The Netherlands), adjusting the portable sonography machine in the lung preset to a reasonable depth.
We graded each patient by performing a 12-zone scoring system (lung ultrasound score, LUSS), including an ultrasound of the lung parenchyma and pleural line patterns, according to the current literature that describes the utility of LUS in predicting the clinical course of patients with COVID-19 [11,[14][15][16][17].

•
Lung parenchyma score: A score of 0 was assigned for no B-lines in a single intercostal space, a score of 1 was assigned for multiple spaced or isolated B-lines, a score of 2 was assigned for diffuse coalescent B-lines, and a score of 3 was assigned for lung consolidations. • Pleural line score: A score of 0 was assigned for a normal, continuous, hyperechoic pleural line; a score of 1 was assigned for a discontinuous, irregular pleural line; and a score of 2 was assigned for a broken or a blurred pleural line.
The final LUSS was the sum of the points in all 12 regions, and it ranged from 0 to 60 [12]. If the maximum score was assigned to the lung (36 points) and to the pleura (24 points), an overall maximum score of 60 was obtained.

Clinical Data
At the time of the ultrasonography study, the respiratory rate, peripheral oxygen saturation (SpO 2 ), oxygen device support used, fraction of inspired oxygen (FiO 2 ) and positive end-expiratory pressure (PEEP), if applied, were obtained. Thus, an arterial gas sample was collected. The ROX index was calculated as the ratio between an oxygenation parameter, assessed using SpO 2 /FiO 2 , and the respiratory rate at the same time of the LUSS assessment.
We defined LUSS/ROX as the ratio between the LUS score and the ROX index, evaluated at the same time.
The patients were followed up at 30 days or until hospital discharge, and the need for intubation and status at hospital discharge (alive, transferred to another hospital or deceased) were recorded. The main study outcome was considered a composite outcome of the need for tracheal intubation within 30 days from the day of hospital admission and in-hospital mortality from any cause despite invasive or noninvasive ventilatory support.

Statistical Analyses
Normality was assessed using the Shapiro-Wilk test, and continuous data are expressed as mean ± SD or median [IQR], as appropriate. Categorical data are expressed as count (proportion). Comparisons between two groups of normally distributed variables were performed using the independent samples t-test, while non-normally distributed variables were compared using the Mann-Whitney U test. The correlations between the qualitative and quantitative variables were obtained using univariable and multivariable logic regression. The factors found to be associated with negative outcomes using univariable logic regression based on p < 0.1 and considered clinically relevant were included in the multivariable models. Thus, in the multivariable analysis, we included baseline patient characteristics (age and BMI); variables describing the respiratory illness severity (respiratory rate, PaO 2 /FiO 2 and set PEEP); and ROX or, alternatively, LUSS or LUSS/ROX. Adjusted models were ranked by using the Akaike information criterion (AIC). AIC provides information on both the simplicity and goodness of fit of a model. As we compared multivariable models that included the same number of independent variables for the same set of patients, the best-fit model was represented by the lowest AIC. To evaluate the accuracy of the ROX index, LUSS and their ratio (LUSS/ROX) in predicting negative outcomes, receiver-operating characteristic (ROC) curves were used. We tested the difference between the area under the ROC areas (AUROCs) of ROX, LUSS and the LUS/ROX index by using the command "roccomp". The best cut-off value to predict NIV failure or death was defined by Youden's index calculation. A Kaplan-Meier survival analysis at the 30-day follow-up-available in all patients-was performed to determine the probability of survival without invasive mechanical ventilation in the overall population stratified by the best cut-offs of LUSS, the ROX index and the LUSS/ROX index to predict the composite outcome. Differences between the stratified populations was assessed using the log-rank test. The significance level was set to 5% (two-sided). Stata/MP version 17 (Copyright 1985-2021 StataCorp LLC, College Station, TX, USA) was used for the statistical analysis.

Study Population
We enrolled 79 patients admitted to the Emergency Department of the university teaching hospital Fondazione IRCCS San Gerardo dei Tintori (Monza, Italy) from February to April 2021. All enrolled patients were included in the statistical analysis. All patients had COVID-19 pneumonia and were treated with noninvasive respiratory support outside of the ICU. We stratified the patients into 2 groups based on the presence of the composite outcome of endotracheal intubation and mortality, which was described in 24 patients out of 79 (30%). Table 1 summarizes the main demographic variables, comorbidities and respiratory parameters of our cohort. We observed more immunosuppressed patients and a trend of a higher BMI in the negative-outcome group. Organ dysfunction severity expressed by the SOFA score was homogeneous between the two groups. We did not observe a difference in laboratory biomarkers at baseline between the study groups.

Ventilatory Variables and Outcomes
A great proportion of patients (66 patients-84%) were treated with helmet CPAP. In all of them, the in-flow gas was delivered through a free-flow system and a PEEP valve with a median set PEEP of 8 cmH 2 O [IQR, [5][6][7][8][9][10]. A Venturi mask was used in 10 (13%) patients, and 3 patients (4%) were treated with HFNO. The median FiO 2 was 0.6 [IQR, 0.5-0.8]. Mild hypoxemia was observed in the overall population (PaO 2 /FiO 2 of 245 ± 101 mmHg). The patients who failed NIV or died had a significantly lower PaO 2 /FiO 2 ratio, a higher respiratory rate and PEEP, and a lower ROX index. LUSS and the LUSS/ROX index were significantly higher in the patients with a negative outcome.

Multivariable Analyses
The multivariable analyses (Table 2) showed that all three indices (ROX, LUSS and LUSS/ROX) were independently associated with negative outcomes. When comparing the models, the Akaike information criterion (AIC) favored the model exploring LUSS/ROX over the model assessing LUSS alone.  Figure 1 describes the optimal cut-off point (obtained using the highest Youden's index), the sensitivity, the specificity and the area under the receiver-operating characteristic curve (AUROC) for the LUSS, the ROX index and the LUSS/ROX index. Although the comparison between the AUC of LUSS/ROX versus the ROX index and that versus LUSS did not reveal a statistically significant difference (p = 0.436 and p = 0.186, respectively), we observed a trend toward a higher accuracy for predicting a negative outcome using the LUSS/ROX index as compared with LUSS ( Figure 1). A comparison between the AUCs of the single indices of LUSS versus the ROX index was not significant (p = 0.621).

Prognostic Performance of the 3 Indices
Diagnostics 2023, 13, x FOR PEER REVIEW 6 of 10 models, the Akaike information criterion (AIC) favored the model exploring LUSS/ROX over the model assessing LUSS alone.  Figure 1 describes the optimal cut-off point (obtained using the highest Youden's index), the sensitivity, the specificity and the area under the receiver-operating characteristic curve (AUROC) for the LUSS, the ROX index and the LUSS/ROX index. Although the comparison between the AUC of LUSS/ROX versus the ROX index and that versus LUSS did not reveal a statistically significant difference (p = 0.436 and p = 0.186, respectively), we observed a trend toward a higher accuracy for predicting a negative outcome using the LUSS/ROX index as compared with LUSS ( Figure 1). A comparison between the AUCs of the single indices of LUSS versus the ROX index was not significant (p = 0.621).

Unadjusted Survival Analysis at 30-Day Follow-Up
The Kaplan-Meier plots showed the probability of survival without invasive mechanical ventilation according to the LUSS, ROX and LUSS/ROX best cut-off groups (Figure 2). A statistically significant difference in survival without invasive mechanical ventilation Diagnostics 2023, 13, 1361 7 of 10 was observed by stratifying the population using all three indices. We observed a higher probability of survival without mechanical ventilation over the 30-day follow-up by using the combined LUSS/ROX best cut-off (p = 0.0000 by log-rank test) (Figure 2c) than by using the best cut-off of LUSS (Figure 2a).

Unadjusted Survival Analysis at 30-Day Follow-up
The Kaplan-Meier plots showed the probability of survival without invasive mechanical ventilation according to the LUSS, ROX and LUSS/ROX best cut-off groups (Figure 2). A statistically significant difference in survival without invasive mechanical ventilation was observed by stratifying the population using all three indices. We observed a higher probability of survival without mechanical ventilation over the 30-day follow-up by using the combined LUSS/ROX best cut-off (p = 0.0000 by log-rank test) (Figure 2c) than by using the best cut-off of LUSS (Figure 2a).

Discussion
In this monocentric prospective observational study conducted in noninvasively ventilated patients outside of the ICU, we observed the following:

−
All three indices (ROX, LUSS and LUSS/ROX) were independently associated with a negative outcome in the univariable analysis, and this association was confirmed in the multivariable analyses. − Although the comparison between the AUC of LUSS or ROX versus LUSS/ROX did not reveal a statistically significant difference, we observed a trend toward a better outcome prediction using the LUSS/ROX index as compared to using LUSS; furthermore, the AUROC of the combined LUSS/ROX index improved the low sensitivity of LUSS and the low specificity of the ROX index. − Using the Kaplan-Meier approach, a statistically significant difference in survival without invasive mechanical ventilation was observed by stratifying the population using all three indices. We observed a higher probability of survival without mechanical ventilation over the 30-day follow-up by using the combined LUSS/ROX best cutoff (p = 0.0000 by log-rank test) (Figure 2c) when compared with LUSS best cut-off (p = 0.0015 by log-rank test) (Figure 2a).
Because of the overwhelming number of patients requiring respiratory support during the COVID-19 pandemic, a great proportion of hospitalized patients were treated outside of ICUs using noninvasive ventilatory support [3]. The helmet CPAP was applied in 83% of our patients, reflecting the common use of the device by healthcare professionals, the efficacy of PEEP in improving gas exchange and the advantage of reducing the risk of environmental contamination, in line with the literature [3]. In these patients, strict clinical monitoring is essential to predict treatment failure, to avoid self-inflicted lung injury and to not further defer intubation when required.

Discussion
In this monocentric prospective observational study conducted in noninvasively ventilated patients outside of the ICU, we observed the following: − All three indices (ROX, LUSS and LUSS/ROX) were independently associated with a negative outcome in the univariable analysis, and this association was confirmed in the multivariable analyses. − Although the comparison between the AUC of LUSS or ROX versus LUSS/ROX did not reveal a statistically significant difference, we observed a trend toward a better outcome prediction using the LUSS/ROX index as compared to using LUSS; furthermore, the AUROC of the combined LUSS/ROX index improved the low sensitivity of LUSS and the low specificity of the ROX index. − Using the Kaplan-Meier approach, a statistically significant difference in survival without invasive mechanical ventilation was observed by stratifying the population using all three indices. We observed a higher probability of survival without mechanical ventilation over the 30-day follow-up by using the combined LUSS/ROX best cut-off (p = 0.0000 by log-rank test) (Figure 2c) when compared with LUSS best cut-off (p = 0.0015 by log-rank test) (Figure 2a).
Because of the overwhelming number of patients requiring respiratory support during the COVID-19 pandemic, a great proportion of hospitalized patients were treated outside of ICUs using noninvasive ventilatory support [3]. The helmet CPAP was applied in 83% of our patients, reflecting the common use of the device by healthcare professionals, the efficacy of PEEP in improving gas exchange and the advantage of reducing the risk of environmental contamination, in line with the literature [3]. In these patients, strict clinical monitoring is essential to predict treatment failure, to avoid self-inflicted lung injury and to not further defer intubation when required.
In this context, a lung ultrasound (LUS), well known for its feasibility at the bedside, noninvasiveness and radiation sparing effect, has been proposed to evaluate the respiratory failure severity [12,13]. LUS has been demonstrated to predict negative outcomes (i.e., endotracheal intubation and mortality) in patients with COVID-19 at the ED [14,15,18,19] and at ICU [15] admission. In our study, we evaluated the performance of a 12-zone scoring system (LUSS) in predicting respiratory deterioration in patients treated with noninvasive ventilation outside of ICUs and followed up by the Medical Emergency Team. We identified 33 as the threshold above which the risk of an adverse outcome increases, and LUSS was shown to predict negative outcomes with an AUC of 0.71 and a sensitivity of 80% (Figure 1a). Our cut-off is higher than the one described in the previous literature using the same scoring system (33 in our study vs. 12 described by L. Ji et al.) [18]. This finding may be related to a more severe respiratory disease in our population or to the presence of pulmonary co-existing conditions, considering that comorbidities potentially influencing image acquisition (i.e., heart failure, non-COVID 19 interstitial pneumonia and pleural effusion) were excluded by Li J. et al. in the design of their study [18]. LUSS showed a good level of accuracy in the stratification risk of patients treated with NIV in our cohort, constituted mainly of patients undergoing helmet CPAP, confirming the recent literature [14,15].
Another useful tool for the prognostication of noninvasively ventilated patients is the ROX index. In recent studies, it has been observed that the ROX index may help clinicians to identify patients at high risk of NIV failure during treatment with both HFNO [18] and helmet CPAP [6,19]. The threshold below which the risk of treatment failure is high and the ROX index is accurate for predicting negative outcomes in our study is similar to the one described in patients with hypoxemia and COVID-19 by Colaianni-Alfonso et al. after 12 h of CPAP treatment [6], as shown in Figure 1b. However, our cut-off is higher than the one proposed for non-COVID-19 patients receiving HFNC (3.85) [5]. This result may be related to the effect of positive airway pressure on arterial oxygenation and, thus, on peripheric saturation, which may not always be predictive of treatment success, especially in patients with COVID-19 [20].
We explored whether combining the LUS score with the ROX index, using the ratio between the two values obtained at the same time, improve the predictive performance of the two indices when used alone in patients undergoing NIV. We observed an independent association with negative outcomes for all three indices (Table 2), and the Akaike information criterion (AIC) favored the model including LUSS/ROX over the model including LUSS. Although the comparison between the three AUCs did not reach statistical significance, probably due to the small sample size, a trend toward a higher accuracy for predicting negative outcomes using LUSS/ROX was shown with a higher gain in the prognostic performance of LUSS/ROX when compared with the LUS score. The Kaplan-Meier analysis showed that using the best cut-off of the combined LUSS/ROX index could identify subgroups of patients significantly associated with the outcome of survival without mechanical ventilation (Figure 2c). Considering that the same PEEP level may generate different effects in different patients (the maintenance of recruitment vs. overdistention) [21] and that the real pressure within the CPAP helmet could be higher than the set PEEP level because of HEPA and HMEF filters [22], incorporating the set PEEP in the ROX index does not increase its capacity to discriminate between patients who would succeed with NIV and those who would fail with it, as recently demonstrated [23]. During NIV, when invasive monitoring (i.e., tidal volume and transpulmonary pressure) is lacking, the ultrasonography gives the clinician a chance to evaluate the effect of a therapeutic maneuver to directly assess the change in lung aeration, as already described in critically ill patients with ARDS [24]. Furthermore, ultrasonography also allows one to unveil the cause of sudden or progressive clinical worsening. Thus, it is reasonable to think that combining a straight clinical evaluation, provided by the ROX index, with a simple ultrasound scoring system (LUSS) may result in a gain in risk stratification and outcome prediction.
Although our study describes the novelty of the application of LUSS and LUSS/ROX during the follow-up of noninvasively ventilated patients outside of ICUs, it must be acknowledged that our research may have some limitations. First, it was a single-center study with a relatively limited sample size, which included only patients with COVID-19 in a limited-resource setting. Furthermore, repeated measures of ROX, LUSS and LUSS/ROX over time were not available, so we could not investigate whether the prognostic performances of these indices change over time. Third, although the LUS exam was performed by an expert clinician using a standardized procedure, it remains an operator-dependent method. In our study, only one clinician performed the LUS exam for each patient; thus, we could not test inter-observer agreement.

Conclusions
In conclusion, these preliminary findings suggest that a multimodal noninvasive approach, including a lung ultrasound and a bedside clinical evaluation with the combined LUSS/ROX index, may help clinicians to predict a higher risk of adverse outcomes. Furthermore, this would allow one to identify patients who would benefit from NIV and to not further defer invasive support, when required, in a COVID-19 population mainly ventilated with helmet CPAP.

Institutional Review Board Statement:
The study was conducted in accordance with the Declaration of Helsinki. Data were collected as part of the STORM study (Spallanzani Institute approval number 84/2020; NCT04424992).

Informed Consent Statement:
Patients' consent was waived due to the observational nature of the study and according to the Institutional Review Board approval. Data Availability Statement: Data are available from the corresponding author upon reasonable request.

Conflicts of Interest:
The authors declare no conflict of interest.