Predictive Model of Good Clinical Outcomes in Patients Undergoing Coronary Angiography after Out-of-Hospital Cardiac Arrest: A Prospective, Multicenter Observational Study Conducted by the Korean Cardiac Arrest Research Consortium

This observational study aimed to develop novel nomograms that predict the benefits of coronary angiography (CAG) after resuscitating patients with out-of-hospital cardiac arrest (OHCA) regardless of the electrocardiography findings and to perform an external validation of these models. Data were extracted from a prospective, multicenter registry of resuscitated patients with OHCA (October 2015–June 2018). New nomograms were developed based on variables associated with survival discharge and neurologic outcomes; their analysis included 723 and 709 patients, respectively. Patient age (p < 0.001), prehospital defibrillation by emergency medical technicians (EMTs) (p = 0.003), prehospital return of spontaneous circulation (ROSC) (p = 0.02), and time from collapse to ROSC (p < 0.001) were associated with survival discharge. Patient age (p < 0.001), prehospital defibrillation by EMTs (p < 0.001), and time from collapse to ROSC (p < 0.001) were associated with neurologic outcomes. The new nomogram had a good predictive performance, with an area under the curve (AUC) of 0.8832 (95% confidence interval (CI): 0.8358–0.9305) for survival discharge and an AUC of 0.9048 (95% CI: 0.8627–0.9469) for neurologic outcomes. Novel nomograms that predict survival discharge and good neurological outcomes after CAG in patients with OHCA were developed and validated; they can be quickly and easily applied to identify patients who will benefit from CAG.


Introduction
The management of out-of-hospital cardiac arrest (OHCA) remains a challenge worldwide [1,2]. The current European Resuscitation Council and American Heart Association guidelines recommend that all patients with cardiac arrest with a suspected cardiac cause of arrest and ST-segment elevation on electrocardiography (ECG) undergo emergent coronary angiography (CAG) [3,4]. In addition, these guidelines state that emergent CAG is reasonable for some adult patients with OHCA and suspected cardiac origin who do not have ST-segment elevation on ECG (such as those who are electrically or hemodynamically unstable) [3,4].
Several previous observational studies and meta-analyses including patients without ST-segment elevation on ECG reported a correlation between CAG and improved neurological outcomes [5][6][7]. However, the Coronary Angiography after Cardiac Arrest without ST Segment Elevation trial, conducted by Lemkes et al., reported no improvement in the survival rates of patients resuscitated after OHCA with an initial shockable rhythm in whom no ST segment elevation or signs of shock were present. Additionally, while coronary artery disease was found in 65% of patients who underwent CAG, most of whom were in a stable state, only approximately 15% of the patients had acute unstable lesions, and thrombotic occlusion was reported in only 5% of them [8]. Therefore, not all patients who undergo successful resuscitation after cardiac arrest benefit from CAG.
The objective of this study was to develop novel nomograms that can predict the prognosis of patients who undergo CAG regardless of the ECG findings after OHCA and to perform an external validation of the novel nomograms. The nomograms developed in this study provide a visual representation of a statistical predictive model that produces numerical probabilities of clinical events, which is more exact than the conventional method that uses odds ratios to predict prognosis [9].

Study Design and Setting
This observational study utilized data from a prospective, multicenter registry (Korean Cardiac Arrest Research Consortium (KoCARC) registry) of patients with OHCA who were resuscitated between October 2015 and June 2018. The patients received cardiopulmonary resuscitation (CPR) in accordance with advanced cardiac life support recommendations; targeted temperature management (TTM), extracorporeal CPR (E-CPR), CAG, and percutaneous coronary intervention (PCI) were conducted according to each institution's protocol. The KoCARC is a multicenter, nationwide collaborative research network of 62 participating secondary or tertiary hospitals that was established to comprehend the multitude of studies conducted in the field of OHCA and to reinforce the cooperative effort in conducting these studies [10]. The KoCARC registry was designed to continuously enroll patients with non-traumatic OHCA, who were transported to the emergency departments (EDs) of the participating hospitals. This study excluded patients with OHCA with obvious non-cardiac etiologies, such as trauma, drowning, poisoning, burns, asphyxia, or hanging. In addition, the registry excluded patients with OHCA who had terminal illnesses documented by medical records, patients under hospice care, pregnant women, and patients with pre-documented do-not-attempt-resuscitation cards [10]. This study was approved by the Institutional Review Board of Yonsei University College of Medicine, Severance Hospital (no. 4-2015-1162) as well as the institutional review boards of each of the participating hospitals. The KoCARC registry database was registered at clinicaltrials.gov as protocol number NCT03222999.

Korean Emergency Medical Service (EMS) System
The Korean EMS system is a basic-to-intermediate level ambulance service operated by 16 provincial headquarters of the National Fire Department and a single-tiered, fire-based EMS system [11]. The most qualified emergency medical technician (EMT) performs CPR with an automated external defibrillator, evaluates the cardiac rhythm at scene, provides advanced airway management, and administers intravenous fluids. EMTs are not allowed to declare death or stop CPR on the scene unless the patient regains a pulse in the field or during transport to an ED; therefore, all EMS-assessed patients are transported to the nearest ED. In the case of prehospital EMS in Korea, E-CPR is not implemented [12].

Study Population
The KoCARC registry included data of 874 patients, who underwent CAG regardless of ECG findings (both ST elevation and non-ST elevation) after the return of spontaneous circulation (ROSC), registered from October 2015 to June 2018. In this study, patients aged <18 years, those transferred from other hospitals after ROSC, and those with unknown primary or secondary outcomes at hospital discharge were excluded. For time-related variables (including the time required for emergency medical service (EMS) response), values that were more than two standard deviations (SDs) from the mean were excluded from the data analysis [13].

Data Collection
All data of the KoCARC registry were anonymized and collected using a web-based case report form (CRF) by the research coordinator at each institution. The CRF comprised seven research fields classified with variables related to OHCA. Each field contained core and optional variables. To improve data quality, the web-based data entry system filtered out outliers or inaccurate values. The research coordinator of each participating hospital was responsible for ensuring data accuracy. The research committee sought to improve the quality of the data by regularly monitoring the data and providing continuous feedback regarding inaccurate data [10].
The following data were retrieved from the KoCARC registry: patient demographics, comorbidities (hypertension, diabetes mellitus, and dyslipidemia), prehospital characteristics (witnessed arrest, place of arrest, bystander cardiopulmonary resuscitation [CPR], prehospital defibrillation by bystanders or EMS personnel, primary ECG rhythm at the scene, and epinephrine use by EMS providers), hospital characteristics (ROSC at ED arrival, initial ECG rhythm at hospital, and total dose of epinephrine used at hospital), post-ROSC characteristics (E-CPR, TTM, coronary angiographic finding, PCI, and vasopressor use in the hospital), cardiac markers, and time intervals (time from EMS call to scene arrival, time from collapse to ED arrival, time from collapse to ROSC, and time from ED arrival to CAG).

Outcome Measurements
The primary endpoint was survival at hospital discharge. The secondary outcome was a favorable neurologic recovery at hospital discharge, defined as a cerebral performance category (CPC) score of 1 or 2.

Statistical Analyses
All statistical analyses were performed using the R software, version 3.4.3 for Windows (The R Foundation for Statistical Computing, Vienna, Austria). Continuous variables are presented as means ± SDs, and categorical variables are presented as frequencies (percentages). Independent two-sample t-tests were used to compare continuous variables, and the chi-squared test or Fisher's exact test was used to compare categorical variables, as appropriate.
The study population was randomly allocated to either a training or validation set at a 7:3 ratio using the split sample validation method. In the training set, a univariate logistic regression analysis was performed to evaluate the prognostic ability of each variable for the primary and secondary outcomes. Then, a multivariable logistic regression analysis with variables with <20% missing data that had a p-value < 0.05 on the univariate binary logistic regression was performed. The variables that were identified as independently associated with survival or neurologic outcomes at hospital discharge were used to create the novel nomograms using the training data set. The performances of the nomograms were evaluated with respect to discrimination and calibration [14]. The predictive accuracies (discriminations) of the models were assessed using area under the receiver-operating characteristic curve (AUC) values and 95% confidence intervals (CIs), which quantify the level of concordance between the predicted probabilities and actual chance of the event of interest occurring. The accuracy of the novel nomograms was internally validated using the bootstrap method using resampling 1000 times of simple random sampling with replacement. The calibration of the nomograms represents how accurate the predicted probabilities are compared with the observed outcome frequencies using graphic representations (calibration curves). A curve along a 45-degree line indicates a perfect calibration model in which the predicted probabilities are identical to the actual outcomes. The calibration curves are presented as apparent and bias-corrected calibration plots using the bootstrapping methods with 500 re-samples. The predictive accuracy and calibration of the novel nomograms generated with the training set were subsequently tested in the validation set. In addition, to verify whether our novel nomogram would be applicable to the patients irrespective of the CAG findings, we performed the sensitivity analysis.
According to the CAG findings, the group was divided into patients without actual lesions (normal coronary, insignificant lesion) and patients with actual lesions or decreased blood flow (notable stenosis, vasospasm). All reported p-values are two-sided, and statistical significance was set at p < 0.05.

Results
Within the study period, 7576 patients with OHCA were registered in the KoCARC registry. Of these, 874 patients underwent CAG after ROSC. Patients who transferred from other hospitals (n = 108), were aged <18 years (n = 1), or had missing data regarding mortality (n = 17) or CPC score (n = 14) at hospital discharge and patients with data sentations (calibration curves). A curve along a 45-degree line indicates a perfect calibration model in which the predicted probabilities are identical to the actual outcomes. The calibration curves are presented as apparent and bias-corrected calibration plots using the bootstrapping methods with 500 re-samples. The predictive accuracy and calibration of the novel nomograms generated with the training set were subsequently tested in the validation set. In addition, to verify whether our novel nomogram would be applicable to the patients irrespective of the CAG findings, we performed the sensitivity analysis. According to the CAG findings, the group was divided into patients without actual lesions (normal coronary, insignificant lesion) and patients with actual lesions or decreased blood flow (notable stenosis, vasospasm). All reported p-values are two-sided, and statistical significance was set at p < 0.05.

Results
Within the study period, 7576 patients with OHCA were registered in the KoCARC registry. Of these, 874 patients underwent CAG after ROSC. Patients who transferred from other hospitals (n = 108), were aged <18 years (n = 1), or had missing data regarding mortality (n = 17) or CPC score (n = 14) at hospital discharge and patients with data outliers of time variables (n = 25) were excluded from the study. The final analyses included 723 patients for the survival outcome group and 709 patients for the neurological outcome group (Figure 1). Among the 723 patients in the survival outcome group, we could confirm the CAG findings in 519 patients in the KoCARC registry: normal coronary findings in 84 patients (16.2%), an insignificant lesion in 69 patients (13.3%), significant stenosis in 303 patients (58.4%), and vasospasm in 63 patients (12.1%). Of the 723 patients in the survival outcome group, 273 (168 survivors/105 deaths) underwent PCI. Among the 709 patients in the neurological outcome group, we could confirm the CAG findings in 508 patients: normal coronary, 83 patients (16.3%); insignificant lesion, 67 patients (13.2%); significant stenosis, 296 patients (58.3%); and vasospasm, 62 patients (12.2%). Among the 709 patients in the neurological outcome group, 268 (144 good neurologic outcomes/124 poor neurologic outcomes) underwent PCI.  The training set for the survival outcome analyses included 496 patients, and the validation set included 227 patients. The baseline characteristics are summarized in Table 1. In the univariate logistic regression analysis, patient age, medical history, bystander CPR, shockable rhythm at the scene, prehospital defibrillation by bystanders or EMS providers, epinephrine use by EMS providers, prehospital ROSC, initial shockable rhythm at hospital arrival, total dose of epinephrine used during CPR in the hospital, diastolic blood pressure, vasopressor use in the hospital, and time-related variables (time from EMS call to scene arrival, time from collapse to ROSC, and time from ED arrival to CAG) were identified as significantly associated with survival discharge after CAG (Table 1). Among them, patient age (p < 0.001), prehospital defibrillation by EMS providers (p = 0.003), prehospital ROSC (p = 0.02), and time from collapse to ROSC (p < 0.001) were identified as independent prognostic factors for survival discharge using multivariable logistic regression analysis ( Table 2). These four prognostic factors were included in the novel nomogram for predicting survival discharge after CAG. The time from collapse to ROSC (min) was the factor that contributed the most to the prediction of survival discharge after performing CAG, followed by patient age (Figure 2a). The total points of the variables in the nomogram allowed for the prediction of the survival discharge probability of the patients in the CAG group. For example, a 65-year-old (25 points) patient who was defibrillated by EMS providers (20 points) with ROSC before hospital arrival (17.5 points) and a time from collapse to ROSC of 10 min (92.5 points) would have a total score of 155 and a predicted probability of survival discharge of approximately 90%. The discriminative ability of the novel nomogram was good, with an AUC of 0.8832 (95% CI: 0.8358-0.9305). The AUC of the internally validated nomogram was 0.8881 (95% CI: 0.8386-0.9339), with a statistical power similar to that of the initial nomogram. In the validation set, the discrimination was excellent, with an AUC of 0.9002 (95% CI: 0.869-0.9315) (Figure 2b). The calibration plot of the nomogram demonstrated an excellent agreement between the predicted and observed probabilities of survival discharge in both sets (Figure 2c). The sensitivity analysis in patients with or without actual lesions showed that the AUC values of the novel nomogram were 0.8557 (95% CI: 0.7832-0.9281) and 0.9439 (95% CI: 0.8824-0.9999), which were not significantly different from that of the initial nomogram (p = 0.53 and p = 0.13, respectively).
The training set for the neurologic outcome analyses included 489 patients, and the validation set included 220 patients. The baseline characteristics and variables found to be associated with favorable neurologic outcomes are summarized in Table 3. Patient age (p < 0.001), prehospital defibrillation by EMS providers (p < 0.001), and time from collapse to ROSC (p < 0.001) were identified as independent prognostic factors for neurologic outcomes ( Table 2).
These three prognostic factors were included in the novel nomogram for predicting neurologic outcomes at hospital discharge. The time from collapse to ROSC was the most contributing factor in this nomogram (Figure 3a). The AUC of the nomogram was 0.9048 (95% CI: 0.8627-0.9469), and the internally validated AUC was 0.9070 (95% CI: 0.8554-0.9509). In the validation set, the discrimination of the nomogram was good, with an AUC of 0.8742 (95% CI: 0.8383-0.9101) (Figure 3b). The calibration plot of the nomogram demonstrated a good agreement in both sets (Figure 3c). The sensitivity analysis in patients with or without actual lesions showed that the AUC values of the novel nomogram were 0.8906 (95% CI: 0.8234-0.9579) and 0.9490 (95% CI: 0.8872-0.9999), which were not significantly different to that of the initial nomogram (p = 0.73 and p = 0.25, respectively). that of the initial nomogram. In the validation set, the discrimination was excellent, with an AUC of 0.9002 (95% CI: 0.869-0.9315) (Figure 2b). The calibration plot of the nomogram demonstrated an excellent agreement between the predicted and observed probabilities of survival discharge in both sets (Figure 2c). The sensitivity analysis in patients with or without actual lesions showed that the AUC values of the novel nomogram were 0.8557 (95% CI: 0.7832-0.9281) and 0.9439 (95% CI: 0.8824-0.9999), which were not significantly different from that of the initial nomogram (p = 0.53 and p = 0.13, respectively). The training set for the neurologic outcome analyses included 489 patients, and the validation set included 220 patients. The baseline characteristics and variables found to be associated with favorable neurologic outcomes are summarized in Table 3. Patient age (p < 0.001), prehospital defibrillation by EMS providers (p < 0.001), and time from collapse to ROSC (p < 0.001) were identified as independent prognostic factors for neurologic outcomes (Table 2).

Discussion
In this study, novel nomograms that predict survival discharge and good neurological outcomes for performing CAG in patients with OHCA were developed and validated. To the best of our knowledge, this is the first study to create a nomogram that determines if CAG will be beneficial by considering various variables in patients with OHCA. These nomograms were based on patient age, prehospital ROSC, prehospital defibrillation by EMS providers, and time from collapse to ROSC, which are readily available data at the prehospital and in-hospital stages of treatment. Therefore, they can be used in an emergency situation and easily applied to each patient individually.

Discussion
In this study, novel nomograms that predict survival discharge and good neurological outcomes for performing CAG in patients with OHCA were developed and validated. To the best of our knowledge, this is the first study to create a nomogram that determines if CAG will be beneficial by considering various variables in patients with OHCA. These nomograms were based on patient age, prehospital ROSC, prehospital defibrillation by EMS providers, and time from collapse to ROSC, which are readily available data at the prehospital and in-hospital stages of treatment. Therefore, they can be used in an emergency situation and easily applied to each patient individually.
The time from collapse to ROSC was the most contributing variable in the nomograms predicting survival discharge and good neurologic outcomes at hospital discharge after CAG in patients with OHCA. The time from collapse to ROSC may be associated with ischemia. Cardiac ischemia and adenosine triphosphate depletion are believed to worsen over time after cardiac arrest [15]. Longer ischemic periods result in worsened neurological prognoses, and ultimately, increased probabilities of death. Therefore, the time from collapse to ROSC affects the severity of hypoxic-ischemic brain injuries and is one of the major causes of death after cardiac arrest [16,17]. Thus, the time from collapse to ROSC is an important factor for determining the benefits of CAG. Patient age and prehospital defibrillation by EMS providers are also common variables to both nomograms. Several previous studies have reported that a younger age and the use of automated external defibrillators at the scene are associated with better outcomes for patients with OHCA [18][19][20]. Patient age has more influence on the survival discharge in this study. Prehospital ROSC was included on the survival discharge nomogram and not on the neurologic outcome nomogram. A study conducted by David et al. reported that cardiac arrest survival is rare without prehospital ROSC [21]. Another study reported that patients with an initial shockable rhythm or EMS-witnessed arrest in the absence of field ROSC had a survival rate of 0.5% [22]. The results of these previous studies support the nomograms developed in this study.
Recent studies have reported that performing CAG early improved the neurological outcomes and survival rate regardless of ECG findings in survivors of OHCA [23,24]. One study showed that early CAG may be associated with improved survival in patients with bystander-witnessed OHCA with shockable rhythms [25]. However, other studies have reported no association between early CAG and survival in patients with non-ST elevation after cardiac arrest [26,27]. In this study, the time from ED arrival to CAG was approximately 10 times longer in the survival group than in the deceased group (2042.5 ± 4903.8 min vs. 204.6 ± 727.3 min; p < 0.001) and approximately five times longer in the good neurologic outcome group than in the poor neurologic outcome group (2184.0 ± 4916.1 min vs. 483.7 ± 2535.2 min; p < 0.001). These results should not be interpreted to mean that delayed or late CAG results in better outcomes than early CAG. Rather, these results may be attributed to the fact that severely ill patients often receive early CAG, resulting in decreased survival rates and poor neurological outcomes among patients who receive early CAG. In contrast, hemodynamically stable patients who have regained consciousness may receive delayed CAG. The results of previous studies combined with those of this study indicate that it is difficult to decide whether to perform CAG simply based on time because of the various clinical situations that act as confounding factors. Therefore, the novel nomogram developed in this study is a good alternative for deciding whether or not to perform CAG.
This study is not without limitations. First, despite the constant efforts of the KoCARC registry committee to improve the reliability and accuracy of the data, some patients had inaccurate or missing data, which could not be included in this study. Second, only data regarding survival and neurologic prognoses at the time of hospital discharge were collected, although long-term data were not collected in this study. Third, some outlier data for time-related variables were excluded to increase the reliability of the study. Fourth, this study did not include the data of some variables such as echocardiographic finding, which was important to identify clinically relevant characteristics of the patients. Fifth, in the case of patients who underwent E-CPR, different results were obtained, but the analysis was not conducted owing to the small number of patients who underwent E-CPR in this study. Finally, data regarding ST segment elevation on the initial ECG were not available in the KoCARC registry. For the practical application of the newly developed nomogram, further validations including patients with suspected cardiac causes of arrest and non-ST segment elevation are needed.

Conclusions
Novel nomograms that predict survival discharge and good neurological outcomes of patients with OHCA undergoing CAG were developed and validated in this study. These nomograms use data that can be obtained during the prehospital and hospital resuscitation period and can be quickly and easily applied to decide if CAG should be performed in resuscitated patients with OHCA.

Informed Consent Statement:
This study was a retrospective study, and consent wavier has been approved from our IRB because it was not possible to obtain patient consent.

Data Availability Statement:
Restrictions apply to the availability of these data. Data was obtained from KoCARC and are available with the permission of KoCARC.