Clinical Validation of Cardiac Arrest Hospital Prognosis (CAHP) Score and MIRACLE2 Score to Predict Neurologic Outcomes after Out-of-Hospital Cardiac Arrest

Background. Out-of-hospital cardiac arrest (OHCA) remains a challenge for emergency physicians, given the poor prognosis. In 2020, MIRACLE2, a new and easier to apply score, was established to predict the neurological outcome of OHCA. Objective. The aim of this study is to compare the discrimination of MIRACLE2 score with cardiac arrest hospital prognosis (CAHP) score for OHCA neurologic outcomes. Methods. This retrospective cohort study was conducted between January 2015 and December 2019. Adult patients (>17 years) with cardiac arrest who were brought to the hospital by an emergency medical service crew were included. Deaths due to trauma, burn, drowning, resuscitation not initiated due to pre-ordered “do not resuscitate” orders, and patients who did not achieve return of spontaneous circulation were excluded. Receiver operating characteristic curve analysis with Youden Index was performed to calculate optimal cut-off values for both scores. Results. Overall, 200 adult OHCA cases were analyzed. The threshold of the MIRACLE2 score for favorable neurologic outcomes was 5.5, with an area under the curve (AUC) value of 0.70 (0.61–0.80, p < 0.001); the threshold of the CAHP score was 223.4, with an AUC of 0.77 (0.68–0.86, p < 0.001). On setting the MIRACLE2 score cut-off value, we documented 64.7% sensitivity (95% confidence interval [CI], 56.9–71.9%), 66.7.0% specificity (95% CI, 48.2–82.0%), 90.8% positive predictive value (PPV; 95% CI, 85.6–94.2%), and 27.2% negative predictive value (NPV; 95% CI, 21.4–33.9%). On establishing a CAHP cut-off value, we observed 68.2% sensitivity (95% CI, 60.2–75.5%), 80.6% specificity (95% CI, 62.5–92.6%), 94.6% PPV (95% CI, 88.6%–98.0%), and 33.8% NPV (95% CI, 23.2–45.7%) for unfavorable neurologic outcomes. Conclusions. The CAHP score demonstrated better discrimination than the MIRACLE2 score, affording superior sensitivity, specificity, PPV, and NPV; however, the CAHP score remains relatively difficult to apply. Further studies are warranted to establish scores with better discrimination and ease of application.


Introduction
Out-of-hospital cardiac arrest (OHCA) remains a challenge, given the poor prognosis of OHCA. According to previous studies, only 2-11% of patients survive until hospital discharge in the Asia-Pacific area [1][2][3]. Several prehospital factors can influence OHCA outcomes, such as bystander cardiopulmonary resuscitation (CPR), automated external defibrillator (AED) use, the presence of a witness, emergency medical service (EMS) response time, location of OHCA occurrence, and level of intervention by emergency medical technicians (EMTs) [4][5][6][7]. Additionally, some post-resuscitation care can impact the prognosis of Healthcare 2022, 10, 578 2 of 10 OHCA, such as target temperature management (TTM), coronary angiography, extracorporeal membrane oxygenation (ECMO) intervention for ECMO facilitated resuscitation, and the level of hospital care [8][9][10][11][12][13][14]. However, patients who need TTM and ECMO-facilitated resuscitation are typically deeply sedated or comatose due to brain injury and therefore, difficult to evaluate post-resuscitation. Therefore, early prediction of OHCA prognosis remains crucial.
Previous studies have attempted to develop clinical decision rules to predict the outcome of OHCA, such as the OHCA score, TTM, and cardiac arrest hospital prognosis (CAHP) score [15][16][17]. Among available scores, the CAHP score seemed to present relatively better sensitivity and specificity to predict the neurologic outcome of OHCA [18,19]; however, the score can be relatively complex and clinical use is limited in emergency department (ED) settings. Recently, Pareek et al. have developed a practical risk score to predict the neurologic outcome of OHCA, termed MIRACLE2, which is substantially easier to apply [20]. The MIRACLE2 score consists of 7 items: missed witness OHCA, initial rhythm, reactivity of pupils, age, changing rhythm, low blood pH, and epinephrine administration. Compared to the CAHP score, MIRACLE2 exhibited a better receiver operating curve (ROC) along with a superior area under the curve (AUC). However, it should be noted that only OHCAs attributed to a primary cardiac cause were included in the MIRACLE2 study. Accordingly, we aimed to examine the clinical value of CAHP and MIRACLE2 scores for predicting neurological outcomes for OHCA. The primary outcome is to compare MIRACLE2 and CAHP scores in predicting neurological outcomes in OHCA.

Study Population
This retrospective observational study was conducted at a tertiary academic medical center with more than 2500 acute beds and an average of 72,000 adult ED visits annually. The medical records of adult patients (>17 years) were extracted from the ED administrative database from January 2015 to December 2019, using the International Classification of Diseases (ICD) Tenth Revision coding system (ICD code: I46). Two trained emergency physicians (EPs) reviewed electronic charts to identify patients with OHCA. OHCA due to trauma, burns, drowning, resuscitation not started due to pre-ordered "do not resuscitate" (DNR) orders, and patients who did not achieve return of spontaneous circulation (ROSC) after resuscitation were excluded.

Data Collection
Data were collected in two parts. The first part was collected from the EMS database, as described in our previous study [2]. Briefly, the EMS is a single-tiered system in Taiwan, and ambulance records were electronically stored in the EMS command center of each province. The EMS database included pre-hospital information, such as age, sex, witnessed OHCA, location of cardiac arrest (public or residential), initial rhythm, duration from cardiac arrest to CPR, and hospitals where patients were sent. The second part was collected from the ED administrative database of Kaohsiung Chang Gung Memorial Hospital. Data from the second part included disease underlying OHCAs, pupil reactivity, biochemistry data, epinephrine dose used, time of ROSC, and patient outcomes. A favorable neurologic outcome was defined as a cerebral performance category (CPC) of 1 to 2. CPC was measured at the time of hospital discharge. The present study was approved by our hospital's institutional review board (number: 202100739B0) and was performed in accordance with the ethical standards of the 1964 Declaration of Helsinki and its later amendments.

Statistical Analysis
For independent variables, the results of continuous variables are presented as mean ± standard deviation (SD). The independent t-test and Mann-Whitney test were used to examine the difference in the distribution of continuous variables, and the chisquare test for independence was used to assess differences in the distribution of categorical Healthcare 2022, 10, 578 3 of 10 variables between groups. ROC analysis with Youden Index was used to calculate the optimal cut-off value for CAHP and MIRACLE2 scores, which predict favorable neurologic outcomes. Logistic regression was used to analyze the statistically significant relationship between OHCA scores and outcomes. Univariate analysis was first performed, then a multivariate regression model was established to adjust for confounding factors. Odds ratios (ORs), 95% confidence intervals (CIs), and p-values were calculated using a logistic regression.
We assessed the classification performance of CAHP and MIRACLE2 scores using 2 × 2 contingency tables to generate estimates for sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). We calculated the 95% CIs for each proportion using the method described by Newcombe [16]. Statistical significance was set at p value < 0.05. All statistical analyses were performed using SPSS version 25.0 (IBM Corp., Armonk, NY, USA).

Results
The STROBE checklist for statistical reporting is presented in Supplementary Table S1. Figure 1 shows that 1043 adults (>17 years) were sent to our ED due to OHCA during the study period. We excluded patients who died due to burns, trauma, or drowning (n = 86); resuscitation not started due to a prescribing DNR order (n = 104); and patients who did not achieve ROSC (n = 643). Furthermore, 10 patients were excluded due to incomplete data, four cases due to missing "reactive pupil," and six due to uncertain "collapse in BLS duration."

Statistical Analysis
For independent variables, the results of continuous variables are presented as mean ± standard deviation (SD). The independent t-test and Mann-Whitney test were used to examine the difference in the distribution of continuous variables, and the chi-square test for independence was used to assess differences in the distribution of categorical variables between groups. ROC analysis with Youden Index was used to calculate the optimal cutoff value for CAHP and MIRACLE2 scores, which predict favorable neurologic outcomes. Logistic regression was used to analyze the statistically significant relationship between OHCA scores and outcomes. Univariate analysis was first performed, then a multivariate regression model was established to adjust for confounding factors. Odds ratios (ORs), 95% confidence intervals (CIs), and p-values were calculated using a logistic regression.
We assessed the classification performance of CAHP and MIRACLE2 scores using 2 × 2 contingency tables to generate estimates for sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). We calculated the 95% CIs for each proportion using the method described by Newcombe [16]. Statistical significance was set at p value < 0.05. All statistical analyses were performed using SPSS version 25.0 (IBM Corp., Armonk, NY, USA).

Results
The STROBE checklist for statistical reporting is presented in Supplementary Table S1. Figure 1 shows that 1043 adults (>17 years) were sent to our ED due to OHCA during the study period. We excluded patients who died due to burns, trauma, or drowning (n = 86); resuscitation not started due to a prescribing DNR order (n = 104); and patients who did not achieve ROSC (n = 643). Furthermore, 10 patients were excluded due to incomplete data, four cases due to missing "reactive pupil," and six due to uncertain "collapse in BLS duration."  After excluding subjects, 200 patients with OHCA were analyzed in the present study. Demographic characteristics, pre-hospital factors, treatment in the ED, comorbidities, and calculated CAHP and MIRACLE2 scores are listed in Table 1. Overall, 33 patients with OHCA were discharged with favorable neurological outcomes. Patients who were dis-Healthcare 2022, 10, 578 4 of 10 charged with favorable neurologic outcomes were associated with lower CAHP (p < 0.001) and MIRACLE2 scores (p < 0.001), lower epinephrine dose (p = 0.003), initial shockable rhythm (p = 0.011), received primary percutaneous cardiovascular interventions (PCI, p < 0.001), shorter time from collapse to basic life support (BLS), shorter time from BLS to ROSC (p < 0.001), and presented higher pH values (p = 0.005). The results of the univariate analyses of prognostic factors are shown in Supplementary Table S2. Tables 2 and 3 Figure S1 show the results of the ROC curve analysis and the calculated optimal cut-off values of MIRACLE2 and CAHP scores to predict favorable neurologic outcomes. The threshold of the MIRACLE2 score for favorable neurologic outcome was 5.5, with an AUC of 0.704 (0.610-0.797, p < 0.001), and the threshold of the CAHP score was 223.4, with an AUC of 0.773(0.688-0.858, p < 0.001).

Discussion
In the present study, we compared a new score, termed MIRACLE2, with a frequently used score, CAHP, to predict neurologic outcomes in patients with OHCA. We found that CAHP was superior to MIRACLE2 in terms of AUC value, sensitivity, specificity, PPV, and NPV. Furthermore, we observed that the epinephrine dose employed during CPR, patients deemed suitable for PCI, MIRACLE2 score, and CAHP score were significantly associated with OHCA neurological outcomes.
Numerous scores have been designed to predict OHCA outcomes, such as OHCA, TTM, and CAHP scores [16,17,20,21]. A previous study has compared TTM, OHCA, and CAHP scores to predict the prognosis of OHCA. The authors reported that the AUC value of the TTM score was better than those of OHCA and CAHP scores [21]. Another study has compared OHCA, CAHP, Acute Physiology and Chronic Health Evaluation (APACHE) II, and Simplified Acute Physiology (SAPS) II scores to predict the prognosis of cardiac arrest [18]. Isenschmid et al. have reported the superior prognostic performance characteristics of the CAHP score than those of the OHCA, APACHE II, and SAPS II scores. Kim et al. have examined patients with OHCA who received TTM and compared the original CAHP, C-GRApH, and OHCA scores. The authors discovered that the CAHP score showed the best AUC when compared with OHCA and C-GRApH scores [22]. Notably, Pareek et al. have examined OHCAs attributed to suspected primary cardiac causes, treated at King's College Hospital, and established a score to predict the outcome of OHCA, i.e., the MIRACLE2 score. Comparing MIRACLE2, TTM, OHCA, and CAHP scores, the authors revealed that the MIRACLE2 score displayed superior AUC, sensitivity, and PPV for predicting the neurologic outcome of OHCA [20]. However, in the present study, we found that both CAHP and MIRACLE2 scores were associated with the neurologic outcome of OHCA. We observed that the CAHP score afforded better discrimination ability in predicting neurologic outcomes than the MIRACLE2 score. In addition, the sensitivity, specificity, and PPV of the CAHP score were superior to those of the MIRACLE2 score. The discrepancy between our results and those reported by Pareek et al. could be attributed to differences in study groups examined. Notably, Pareek et al. included only OHCA patients with suspected primary cardiac causes. In the present study, we included OHCAs attributed to all underlying causes. Accordingly, 76.4% of patients underwent coronary angiography in the study reported by Pareek et al., whereas only 19% of patients with OHCA underwent coronary angiography in our cohort.
Although the discrimination ability of the CAHP score was better in our study, clinical validation was relatively limited. The duration of the "no-flow" interval (from collapse to CPR initiation) was occasionally difficult to estimate. In East Asian countries, the rates of witnessed OHCA were approximately 60% and 64% of cardiac arrest cases in Japan and Korea, respectively. The rates of witnessed OHCA were 81.1% and 91% in North America and France, respectively [23,24]. In our study, the rate of witnessed OHCA cases was 77.5%. Furthermore, the "low-flow" interval (from CPR to ROSC) was occasionally missed as the time of CPR initiation was omitted or ECMO was employed. Conversely, almost all items of MIRACLE2 can be obtained at hospitals, and we could easily calculate the MIRACLE2 score. Recently, a few studies have attempted to adjust the CAHP score to improve the ease of application. Wang et al. have excluded the duration of the "no-flow interval" from the CAHP score and observed that the AUC for neurological outcome was still acceptable [25]. However, large-scale studies validating the discrimination ability of such scores are lacking. Additional studies are required to establish scores with better clinical validation and discrimination.
Herein, we observed that a high epinephrine dose was related to a poor neurologic outcome. Previous studies have examined the relationship between the neurologic outcomes of patients with OHCA and pre-hospital epinephrine use, but the results were inconsistent. According to Nakahara et al., early epinephrine use during the pre-hospital phase, especially within 10 min of collapse, could be associated with favorable neurologic outcomes in bystander-witnessed patients with OHCA, regardless of cardiogenic or non-cardiogenic OHCA [26]. Ran et al. have reported a similar finding [27]. However, in a study by Loomba et al., pre-hospital epinephrine use was shown to be unrelated to favorable neurologic outcomes, with similar results reported in numerous other previous studies [28][29][30]. The differences between observed results might be explained by the duration of pre-hospital resuscitation time, and a longer pre-hospital CPR time also indicates greater epinephrine use. Kaji et al. have found that epinephrine use can be associated with favorable neurologic outcomes only when patients with OHCA receive dosages lower than 1.5 mg [31].
In the present study, patients with OHCA who underwent PCI exhibited favorable neurologic outcomes. Previous studies have reported similar results, although with slightly different patient selection settings. Bergman et al. included patients with OHCA presenting ROSC who were admitted to the hospital. The authors observed favorable neurologic outcomes in patients with OHCA presenting initial ventricular fibrillation (VF) rhythm, who then underwent PCI [32]. In Shavelle's [33] study, although the survival rate of patients with OHCA was lower in the group with ST-elevation myocardial infarction (STEMI) than in the group without STEMI, the proportion of survival with favorable neurologic outcomes exceeded 50% (73%). Zimmermann et al. [34] have further highlighted that initiation of advanced cardiovascular life support (ACLS) within 6 min of collapse is a predictor of favorable neurologic outcomes in STEMI-complicated patients with OHCA. This result might be explained by the fact that VF is a common underlying cause of cardiogenic OHCA [35], and VF and cardiogenic OHCA are independent factors for favorable OHCA outcomes [36].
Some studies have focused on the impact of extracorporeal cardiopulmonary resuscitation (ECPR) for managing OHCA; however, the results remain inconclusive. Napp et al. selected 40 patients with OHCA presenting potentially favorable circumstances (witnessed collapse, initial shockable rhythm, without major comorbidities, bystander CPR, and age < 75 years) who received ECPR and analyzed the outcomes. The authors found that only three patients were discharged with favorable neurologic outcomes, and no significant differences were observed when compared with previous studies [37]. In contrast, Lunz et al. have examined 423 patients treated with ECPR at 5 different European intensive care units and found that 80 patients (19%) were discharged with favorable neurological outcomes [38]. In addition, Beyea et al. reviewed 75 studies and demonstrated that the ratios of favorable neurologic outcomes in the ECPR and conventional CPR were 8.3% to 41.6% and 1.5% to 9.1%, respectively [39]. However, a meta-analysis could not be performed owing to the significant heterogeneity in the study by Beyea et al. In the current study, we observed that ECMO did not afford a statistically significant difference. The case number was considerably small as only 15 patients received ECMO; further studies could help determine the impact of ECMO intervention.
The present study has some limitations. First, this was a retrospective observational study restricted to a single city. Second, the database was limited to a single-tiered EMS system, and the results may be different for other cities with different EMS systems. Third, some OHCA patients would be missed if they were brought to the hospital by family or healthcare facilities, not by EMS. Fourth, prehospital interventions, such as advanced airway management and mechanical devices for reanimation, were not recorded in this database, and these interventions might influence the outcome of OHCA. Furthermore, a history of PCI, arrhythmias, coronary artery bypass graft, and lactate values were not recorded, which might impact the prognosis of OHCA.

Conclusions
In the present study, we compared MIRACLE2 and CAHP scores to predict neurological outcomes for OHCA. We found that the CAHP score had better discrimination ability than the MIRACLE2 score. In addition, the CAHP score afforded a superior sensitivity, specificity, PPV, and NPV. We also found that epinephrine dose during CPR, patients suitable for PCI, and MIRACLE2 and CAHP scores were all significantly associated with OHCA neurological outcomes. However, the CAHP score was relatively difficult to apply despite the superior discrimination. Further investigations could help establish new scores with better discrimination and ease of application.

Supplementary Materials:
The following supporting information can be downloaded at: https:// www.mdpi.com/article/10.3390/healthcare10030578/s1, Table S1: STROBE Statement-Checklist of items that should be included in reports; Table S1: Univariate analyses for favorable neurologic outcome. Data Availability Statement: The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.