Comparison of Nine Early Warning Scores for Identification of Short-Term Mortality in Acute Neurological Disease in Emergency Department

(1) Background: The aim was screening the performance of nine Early Warning Scores (EWS), to identify patients at high-risk of premature impairment and to detect intensive care unit (ICU) admissions, as well as to track the 2-, 7-, 14-, and 28-day mortality in a cohort of patients diagnosed with an acute neurological condition. (2) Methods: We conducted a prospective, longitudinal, observational study, calculating the EWS [Modified Early Warning Score (MEWS), National Early Warning Score (NEWS), VitalPAC Early Warning Score (ViEWS), Modified Rapid Emergency Medicine Score (MREMS), Early Warning Score (EWS), Hamilton Early Warning Score (HEWS), Standardised Early Warning Score (SEWS), WHO Prognostic Scored System (WPSS), and Rapid Acute Physiology Score (RAPS)] upon the arrival of patients to the emergency department. (3) Results: In all, 1160 patients were included: 808 patients were hospitalized, 199 cases (17%) required ICU care, and 6% of patients died (64 cases) within 2 days, which rose to 16% (183 cases) within 28 days. The highest area under the curve for predicting the need for ICU admissions was obtained by RAPS and MEWS. For predicting mortality, MREMS obtained the best scores for 2- and 28-day mortality. (4) Conclusions: This is the first study to explore whether several EWS accurately identify the risk of ICU admissions and mortality, at different time points, in patients with acute neurological disorders. Every score analyzed obtained good results, but it is suggested that the use of RAPS, MEWS, and MREMS should be preferred in the acute setting, for patients with neurological impairment.


Introduction
Acute neurological diseases were about 10% to 20% of the medical admissions at Emergency Departments (ED) before the coronavirus disease   [1][2][3], accounting for the third cause of admissions after cardiovascular and respiratory diseases [4]. Common neurological diseases include a variety of major disorders such as stroke, headache, and seizures [2][3][4]; traumatic brain injuries exhibit a significant incidence as well [5].
The reorganization of medical assistance in consequence of the effort focused on the ongoing COVID-19 pandemic, together with the reduction in patients, who are avoiding the hospital, have entailed a decrease in interventions related with neurological problems in many countries [6,7]. These indirect effects of the COVID-19 pandemic have increased the risk of mortality of these neurological diseases, which have previously shown a high mortality risk worldwide (6.55 million deaths by stroke in 2019) [8,9], with a subsequent overload of ED and Intensive Care Units (ICU).
Until now, many studies have focused on possible strategies to decrease the crowding of ED and ICUs. In this context, it was recently demonstrated that the implementation of assistance protocols in ED with the use of Early Warning Scores (EWS) could help to reduce absolute in-hospital mortality rate [10].
When the focus was moved to neurological diseases, particularly acute cerebrovascular diseases, it was demonstrated that a model including age, stroke severity, Early Warning Score, and Performance Status was found to be valid to predict one-year mortality [11]. Furthermore, the onset severity of neurological diseases could be misleading (many clinical patterns are similar and confounding at onset), and a part of those could present an unexpected rapid clinical deterioration. Keeping in mind this aspect, it appears crucial to develop and fix strategies to better predict not only mortality but also the risk of rapid clinical deterioration in neurological diseases.
Protocol assistance for a coordinated response is necessary to assess the severity of the prognostic and the best initial decision [12,13], which certainly involves the use of Early Warning Scores (EWS).
Most of the studies use only one score to evaluate acute neurological disorders (such as stroke [14][15][16][17][18], traumatic brain injury [5,19], acute hypertensive intracerebral haemorrhage [20], or patients with medium and long-term deterioration [20,21]). A wide variety of EWS is now available, and it is crucial to determine which are the best ones for each pathology, from the clinical context and from the phenomenon that is wanted to be predicted [12,13].
To the best of our knowledge, until now no studies performed a direct comparison between the different EWS. The aim of the study was screening the performance of nine Early Warning Scores to identify patients at high-risk of premature impairment and to detect intensive care unit-admissions, as well as to track the 2-, 7-, 14-, and 28-day mortality in a cohort of patients diagnosed with an acute neurological condition.

Study Design and Setting
We conducted a prospective, longitudinal, observational study with adult patients transferred with high priority by ambulance to the EDs of five hospitals of the Health Public System of Castilla y León (Spain) between 1 January 2019 and 30 August 2021. The study conforms to the broader EQUATOR (Improving the Quality and Transparency of Health Research) guidelines [11] The

Population
The sample was selected from all patients over 18 years of age transferred with high priority by ambulance to the EDs of the selected hospitals with a diagnosis of neurological disorders (ischaemic stroke, seizures, haemorrhage, confusion syndrome, degenerative disease, headache, vertigo, tumour, infection, neuromediated syncope, coma). For this study, all patients who were minors or who met any of the following exclusion criteria were excluded: cardiorespiratory arrest, death before or during transfer, pregnancy, discharge in situ, psychiatric pathology, and/or diagnosis of terminal illness.
During the investigation, an attempt was made to minimize the risk of bias by excluding patients without follow-up through the clinical history, from whom the vital signs necessary for the calculation of the scales could not be obtained, or who did not give their informed consent. A family member or legal guardian was contacted to obtain informed consent for patients with an altered clinical condition or level of consciousness.

Outcomes
The principal outcome was the presence of high-risk of premature impairment (intensive care unit-admissions). Secondly, the performance of the scores for 2-, 7-, 14-, and 28-day mortality was evaluated.

Early Warning Scores Selection
For the study, the clinical variables were analyzed on arrival at the patient's emergency department, which allowed the calculation of the EWS indices, specifically the Modified Early Warning Score (MEWS), National Early Warning Score (NEWS), VitalPAC Early Warning Score (ViEWS), Modified Rapid Emergency Medicine Score (MREMS), Early Warning Score (EWS), Hamilton Early Warning Score (HEWS), Standardised Early Warning Score (SEWS), WHO Prognostic Scored System (WPSS), and Rapid Acute Physiology Score (RAPS).
The MEWS scale was first used as a tool to assess the state of the hospitalized patient; however, its usefulness has spread as a predictor of ICU admissions and predictions of hospital mortality [22,23]. NEWS was categorized by several studies as the most reliable scale to measure a patient's deterioration at the prehospital level [24,25]. MREMS is a scale derived from REMS (Rapid Emergency Medicine Score), which seeks to predict hospital mortality in real time, adding the variables age and oxygen saturation, of which, for example, RAPS lacks, which comes from the Acute Physiology score and Chronic Health Evaluation II (APACHE-II) and has been shown to be a good rapid prediction tool [26,27]. HEWS [28] was designed to improve the early detection of all hospitalized patients at risk for deterioration, particularly among those with suspected infection. In SEWS, the inclusion of oxygen saturation is shown to have a significant relationship with short-to medium-term mortality [29]. Finally, WPSS can classify patients into five risk groups with different survivals and probabilities evolution [30].

Collection of the Parameters
For these scores, it was necessary to collect information on respiratory rate, oxygen saturation (SpO 2 ), need for supplemental oxygen, heart rate, temperature, age, systolic blood pressure (SBP), mean arterial pressure (MAP), and level of consciousness using the Glasgow Coma Scale (GCS) or, alternatively, the AVPU scale (alert, voice, pain, no response).
During triage, the emergency personnel oversaw collecting the clinical and evaluation data to calculate the scores of the different EWS. SpO 2 , SBP, MAP, temperature, and heart rate measurements were obtained using the Connex ® Vital Signs Monitor (Welch Allyn, Inc., Skaneateles Falls, NY, USA). Respiratory rate was measured by directly observing the total number of complete respiratory cycles for one minute. The patient's level of consciousness was assessed using the GCS. All data were included anonymously in an electronic database created for this study.
The rest of the hospital variables (hospitalized patients, ICU admissions, mortality rates, diagnosis of the corresponding group according to the International Classification of Diseases 11th Revision) were collected by an associate researcher from each hospital, by reviewing the electronic medical records 30 days after the index event.

Statistical Analysis
Quantitative variables are presented as a median and interquartile range (IQR), while for categorical variables absolute and relative frequencies were used. Quantitative variables were analyzed using a normality test (Kolmogorov Smirnov) and were compared with a non-parametric Mann-Whitney U test. Effect sizes (ES) were calculated with the Rosenthal r test and classified according to these parameters: Trivial (<0. To compare the predictive capacity of the EWS, the area under the curve (AUC) of the receiver operating characteristic (ROC) was calculated, to predict the need for ICU admission and 2-, 7-, 14-, and 28-day mortality. Using the Youden test, we selected the cut-off point with the best results on each scale for sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio, negative likelihood ratio, and diagnostic accuracy (DA). We used the chi-squared test to generate contingency tables that would allow us to establish association relationships through odds ratios. The accuracy of the scores was compared using DeLong's test.
All analyses were performed with XLSTAT ® BioMED software for Microsoft Excel ® version 14.4.0 (Microsoft Inc., Redmond, WA, USA) and IBM SPSS Statistics version 20.0 (IBM Corp., Armonk, NY, USA). In all tests, a confidence level of 95% and a p-value below 0.05 were considered significant.

Results
Between 1 January 2019 and 30 August 2021, we recorded a total of 1160 cases of adult patients with neurological disease who were referred to the ED of the five participating hospitals. Figure 1 show the flowchart. number of complete respiratory cycles for one minute. The patient's level of consciousnes was assessed using the GCS. All data were included anonymously in an electronic databas created for this study.
The rest of the hospital variables (hospitalized patients, ICU admissions, mortalit rates, diagnosis of the corresponding group according to the International Classificatio of Diseases 11th Revision) were collected by an associate researcher from each hospita by reviewing the electronic medical records 30 days after the index event.

Statistical Analysis
Quantitative variables are presented as a median and interquartile range (IQR), while fo categorical variables absolute and relative frequencies were used. Quantitative variables wer analyzed using a normality test (Kolmogorov Smirnov) and were compared with a non-pa ametric Mann-Whitney U test. Effect sizes (ES) were calculated with the Rosenthal r test an classified according to these parameters: Trivial (<0. To compare the predictive capacity of the EWS, the area under the curve (AUC) of th receiver operating characteristic (ROC) was calculated, to predict the need for ICU admissio and 2-, 7-, 14-, and 28-day mortality. Using the Youden test, we selected the cut-off point wit the best results on each scale for sensitivity, specificity, positive predictive value (PPV), neg tive predictive value (NPV), positive likelihood ratio, negative likelihood ratio, and diagnost accuracy (DA). We used the chi-squared test to generate contingency tables that woul allow us to establish association relationships through odds ratios. The accuracy of th scores was compared using DeLong's test.
All analyses were performed with XLSTAT ® BioMED software for Microsoft Exce version 14.4.0 (Microsoft Inc., Redmond, WA, USA) and IBM SPSS Statistics version 20 (IBM Corp, Armonk, NY, USA). In all tests, a confidence level of 95% and a p-value below 0.05 were considered significant.

Results
Between 1 January 2019 and 30 August 2021, we recorded a total of 1160 cases of adu patients with neurological disease who were referred to the ED of the five participating ho pitals. Figure 1 show the flowchart.  The median age was 71 years (IQR: 53-82 years, range 18-97), and 46% (537 patients) were females. The main reasons for medical check-up were ischaemic stroke (369 cases, 32%) and seizure (282 cases, 24%), and their priority of care according to hospital triage was mainly level II (50%) or level III emergency care (40%). In total, 808 patients were hospitalized, with ICU care required in 199 cases (17%). The mortality of the patient ranged from 6% (64 cases) within 2 days, to 10% (114 cases) within 7 days, to 13% (145 cases) within 14 days, to 15% (173 cases) within 21 days, to 16% (183 cases) within 28 days.
Tables 1 and 2 compare the clinical variables from initial assessment and the hospital care data of the patients with the variables resulting from this study: the requirement of ICU care and patient mortality at 2-, 7-, 14-, and 28-days.     Considering the clinical deterioration of patients with different types of neurological diseases, we found that the scales with the best capacity to predict the need for ICU care were the RAPS with an AUC of 0.790 (95% CI: 0.751-0.829) and the MEWS with an AUC of 0.789 (95% CI: 0.750-0.828) ( Table 3). Regarding the capacity to predict mortality, the MREMS obtained the best score for 2-day mortality with an AUC of 0.929 (95% CI: 0.885-0.973) and for 28-day mortality with an AUC of 0.856 (95% CI: 0.820-0.891) ( Table 3). Table 3. AUROC, cut-off points for combined sensitivity and specificity with the best score (Youden's test) for the different scales analyzed (for intensive care unit, 2-, and 28-day mortality).

Scores
Intensive Care Unit    3 and Table S1 shows the best cut-off values for each EWS, according to sensitivity and specificity (Youden test) for each of the studied events (ICU, 2-, 7-, 14-, 28-day mortality), which allowed to discern their predictive value and diagnostic accuracy. Comparisons of the AUC of the different scoring systems (Tables S2 and S3) show that they are significantly different with RAPS and MEWS, having the highest efficacy to predict the ICU admissions and MREMS for 2-and 28-day mortality.

Discussion
To the best of our knowledge, this is the first study to explore whether several EWS accurately identify the risk of ICU admissions and mortality at different time points in patients with acute neurological disorders. Our findings showed that both RAPS and MEWS performed well. However, regarding mortality prediction, MREMS showed the best capacity to predict it both in the short term and in the long term. Previous studies demonstrated that the use of combined scores, in which EWS was one of the factors included, were able to predict one-year mortality in haemorrhagic and ischaemic stroke patients, with a similar AUC of 85% [11]. Liljehult et al. retrospectively analyzed a cohort of 1113 stroke patients and evaluated the performance of a combined score that included EWS. On the other hand, our total cohort was similar, but we analyzed a not-homogeneous population that included different neurological conditions, so the two studies are not directly comparable. Other studies drew similar conclusions about the usefulness of EWS in mortality and ICU hospitalization prediction in ED, but most are focused on a particular neurological condition as well as traumatic brain injury, brain neoplasm, brain haemorrhage, etc. [19].
Importantly, when applied to neurological disorders, we found a difference in performance of EWS that is mainly based on similar parameters with different relative weight. In this aspect, we must underline that EWS has been developed and used for areas such as ED and ICU, in which the stratification of patients into risk groups helps rapid evaluation, triage, and early management [31][32][33]. Furthermore, EWS can improve the care of critically ill patients, who must be transported from acute settings to hospitals, segregating those with a good prognosis from those with a bad one. Importantly, most of the patients included were diagnosed with ischaemic stroke, epilepsy, and cerebral haemorrhage that included traumatic and not-traumatic etiology, so that our results are mainly affected by these three conditions.
Particularly, in the present cohort, 32% of the patients were diagnosed with stroke, representing the most frequent pathology analyzed. In Spain, stroke care is based on a "Hub and Spoke" system that, together with multimodal imaging, was demonstrated to improve access to revascularization treatment [34]. Revascularization could be achieved mainly by intravenous thrombolysis and/or mechanical thrombectomy: the last one is usually available in Hub Hospitals, but not in Spoke Hospitals, and is indicated for large vessel occlusions [35]. Large vessel occlusion strokes frequently reduce GCS scores, and their general clinical conditions are often worse than those observed in patients with stroke of small vessel occlusions, leading to higher scores in EWS [36]. Keeping in mind this point the use of EWS, particularly of the scales that we demonstrated to have best performances in terms of mortality and ICU risk, could permit to differentiate the transport destiny, bringing "high risk" stroke patients directly to Hub Hospital, reducing treating times and improving the quality of stroke care, permitting a rapid access to advanced imaging and, if necessary, to mechanical thrombectomy. In other words, the use of RAPS, MEWS, and MREMS could help to fasten and ameliorate acute stroke treatment, reducing pre-hospital timings.
It should also be considered that the management of traumatic brain injuries and cerebral haemorrhages depend on the presence of neurosurgery department: interestingly, not all the patients with these conditions are potential candidates for neurosurgery. So, the use of EWS could stratify the patients who are at risk of a rapid clinical deterioration, to anticipate transport to a tertiary hospital (that offers 24 h on-call neurosurgery) or rapid on-site access to neurosurgery, when available.

Limitations
The major limitations depend on the design and the sample size of the study. Firstly, we conceived and developed an observational study that it is known to increase the risk of a selection bias. However, the fact that several neurological conditions were included could have significantly reduced it. Another important limitation is the sample size: we included more than 1000 patients, but the majority of those were affected by strokes, haemorrhages, and seizures, so that the weight of the final analysis mostly depends on these conditions, hence, de facto reducing our cohort to 50% of the total.
In the sample distribution, according to the need for intensive care of the patients, we found that age groups were significantly different; this situation may be due to confounding bias with older patients, who often have limited ICU care [37,38], despite of the impact of hospitalization in older adults is bigger, especially for polimedication [39]. In total, 90% of the patients admitted to the ICU presented as a base pathology ischaemic stroke, seizures, or haemorrhages. We consider it interesting to carry out future studies to address the predictive capacity of EWS, discriminating between "unplanned ICU admission" and "direct ICU admission from the EDs" in patients with these pathologies. Another limitation consists of the lack of a longer follow-up, which could add interesting information about the presence of residual functional deterioration and long-term mortality. All these limitations did not allow more meaningful conclusions about the role of EWS in stroke patients, and more studies with specific analysis are warranted.

Conclusions
In conclusion, this study suggests that the use of RAPS, MEWS, and MREMS should be preferred in the acute setting in patients with neurological impairment. High scores indicate worse prognosis and the potential necessity of more intensive and specific treatments.
Supplementary Materials: The following supporting information can be downloaded at: https://www. mdpi.com/article/10.3390/jpm12040630/s1. Table S1: AUROC, cut-off points for combined sensitivity and specificity with best score (Youden's test) for the different scales analyzed (for 7-, 14-and 21-day mortality); Table S2: Comparison of the AUROC of the different scores (ICU admissions); Table S3: Comparison of the AUROC of the different scores (2-day mortality).  Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.

Data Availability Statement: Not applicable.
Acknowledgments: The authors thank the nursing and medical staff from the ambulances and the emergency departments of all hospitals that belong to the public health system of Castilla and León (SACYL), for their assistance in facilitating data collection.

Conflicts of Interest:
The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.