New Quality Indicator for Treatment of Acute Myocardial Infarction

Abstract

Introduction: Crude mortality is commonly used as a quality indicator (QI) for the treatment of acute myocardial infarction (AMI), but has important limitations including its dependence on the local case-mix. We aimed to explore the feasibility of a novel approach using risk adjustment according to the Global Registry of Acute Coronary Events (GRACE). Methods: In 1471 consecutive patients admitted with AMI to a Swiss university hospital in 2012 and 2013, we quantified working hours needed by a trained healthcare professional to complement the available administrative dataset by detailed medical review of all available medical records to: 1) differentiate the subtypes of AMI in order to separate type 1 (including type 4) AMIs from type 2 and postoperative AMIs (GRACE is only validated for type 1 AMI); 2) add all medical variables required to calculate the GRACE score. Results: Detailed medical review identified 93 additional patients (6.7%) with AMI as the main diagnosis, who were missed in the administrative dataset. Complete data for the calculation of the GRACE score could be obtained for 1233 patients (93.8%). In both years, observed crude mortality was significantly lower than the expected in-hospital mortality using the GRACE model (2012 [n = 613]: crude mortality 6.0%, mean GRACE mortality 8.3% [95% CI 7.2–9.4%]; 2013 [n = 620]: crude mortality 5.8%, mean GRACE mortality 9.4% [95%CI 8.3–10.6%]). Overall, the number of working hours required to retrospectively complement the administrative dataset was 1150 hours (575 h per year). Conclusion: Assessment of risk-adjusted in-hospital mortality in AMI is feasible, provides important insights regarding treatment results while improving comparability between hospitals, but is very time-consuming if done retrospectively. Prospective documentation of the GRACE score within the electronic medical records would help to reduce the effort needed to obtain this novel QI. Further multicentre studies are warranted.

Introduction

Quality indicators (QIs) play a central role in the evaluation of healthcare provided in hospitals [1]. Since the introduction of the diagnosis-related group (DRG) classification system (SwissDRG in Switzerland), governmental agencies and health insurers, as well as the press, increasingly focus on QIs in order to assess, compare and comment on the cost-effectiveness and standard of care provided by healthcare providers. In Switzerland, the performance of care providers is assessed and compared using a catalogue called Swiss Inpatient Quality Indicators (CH-IQI) [2]. In most centres of care, these data are acquired routinely and on an automated basis in order to maintain a cost-effective way of evaluation. The results are published every year for the public.

Crude in-hospital mortality, adjusted by age and gender, and in certain cases the Charlson comorbidity index are currently used to assess quality of care in acute myocardial infarction (AMI) [3,4,5]. Unfortunately, this approach has multiple shortcomings: for example, the local case mix for certain diseases can vary widely between centres of care, either because of their regional status as a central hub for interventional and intensive care, or because of the quality of life in a certain region [6]. Therefore, for acute diseases including AMI, risk adjustment solely based on gender and age seems incomplete, as variables including vital signs at presentation outperform by far age and gender in the prediction of in-hospital death [7,8,9]. Appropriate risk adjustment for these variables would seem mandatory, but has until now not been implemented.

Our aim was to explore the feasibility of a novel approach that uses the Global Registry of Acute Coronary Events (GRACE) risk score, an internationally validated [10,11,12] and accepted [13,14] tool for risk adjustment and estimation of in-hospital mortality, as a QI for treatment of AMI.

Methods

Study Design and Patient Population

We retrospectively identified all patients aged 18 years and above admitted with AMI to the University Hospital Basel, Switzerland, in 2012 and 2013. The patient population was obtained from either the International Classification of Diseases (ICD)-coded administrative hospital database, or, in order to complement this dataset, a full-text search of the digital patient files using keywords related to AMI.

For cases derived from the ICD-coded administrative database, all patients with primary diagnosis categorised under ICD-10, I21.x (“ST elevation and non-ST elevation myocardial infarction”) were included. For the full-text-based search using the digital patient archive, the following keywords were used to identify AMIs: “myocardial infarction”, “STEMI”, “NSTEMI”, “AMI”. A detailed medical review of all cases obtained was performed and mainly consisted of differentiation by subtype of AMI (type 1, type 2, post-interventional or other). This was done in order to ensure that the obtained list of cases was complete. AMI type 4 (stent thrombosis) was included in this analysis and added to the other AMI type 1 patients. The categorisation into type 1 AMI was performed according to the universal definition of AMI [15].

All type 1 AMIs were further categorised into non-ST elevation myocardial infarction (NSTEMI) and ST elevation myocardial infarction (STEMI). Next, the required variables for the assessment of the GRACE risk score were obtained. Cases with one or more variables undocumented or unobtainable by retrospective review had to be excluded, as the GRACE risk score tool requires all variables for calculation.

Table 1. Distribution of GRACE variables in patient population (total n = 1233 patients).

shows the baseline characteristics of the patient population.

In addition, we quantified the working hours required by a trained healthcare professional to complete the medical review and assessment of the GRACE risk score, in order to assess the feasibility of this novel method. The endpoint of the study was defined as inhospital death, according to standards in outcome QI.

Risk Adjustment with the GRACE Risk Score

The GRACE risk score version 1 [10] was used for risk adjustment, and consists of the following variables: age, systolic blood pressure, heart rate, Killip class, serum creatinine level, changes in the ST segment on the electrocardiogram (ECG) at admission, elevated cardiac troponin (Roche hs-cTnT assay), and cardiac arrest. For calculation of the expected percentage of deaths, the in-hospital / 6 months method was used. All variables were required in order to perform risk adjustment; patient cases with one or more variables missing had to be excluded from the calculation.

Variable Data Collection

Variables were either obtained from the electronic patient records or from handwritten case documentation. Wherever handwritten documentation was not available in electronic/scanned form, the hospital archives were searched manually. Age, serum levels of creatinine and troponin were available as part of datasets and collected by means of an automated method. Systolic blood pressure and the variable ‘cardiac arrest at admission’ were obtained from handwritten monitoring sheets. Heart rate was either obtained from the admission ECG or monitoring sheet (whichever was timed first). ST-segment changes in the ECG were gathered from case history. For Killip class, the data were obtained from the case presentation at admission. When not explicitly stated in the case documentation, the Killip class [16] was evaluated retrospectively using available patient history (clinical examination and presentation, radiological findings).

Statistical Analysis

Continuous variables are presented as medians with the interquartile range, and categorical variables as numbers and percentages.

Firstly, the probability of in-hospital death as calculated with the GRACE risk score tool was assessed for all cases. Next, a mean probability of in-hospital death, adjusted using the GRACE risk score, was calculated for each year individually, and was compared with the crude mortality rate for both years by use of a 95% confidence interval (CI). Significance between actual and risk-adjusted (expected) mortality was tested using the chi-square test. A p-value <0.05 was considered statistically significant. All statistical analyses were performed using Microsoft Excel 2013 and IBM SPSS for Windows version 19.0 (SPSS Inc, Chicago Il).

Results

Patient Flow and Baseline Characteristics

The ICD-coded administrative hospital database contained 1471 cases with a diagnosis coded under ICD-10, I21.x for the years 2012 and 2013 (Figure 1). The full-text search using AMI-related keywords returned 1460 results for the same years. In total, 1314 cases were identified as having the primary diagnosis of type 1 AMI.

These consisted of:

a)

1221 (93.3%) cases from the ICD-coded dataset; 250 cases were excluded from the ICD-10-coded dataset, mainly because they were type 2 AMIs, post- or peri- procedural events, or older events requiring further care.

b)

93 (6.7%) additional cases were identified using the full-text search and were miscategorised in the ICD- 10-coded dataset. However, most type 1 cases from the full-text search matched those from the ICD-10-coded database.

In 1233 cases (93.8%), all variables required for risk adjustment using the GRACE risk score were obtainable, whereas 81 patients had to be excluded because of one or more missing variables.

Table 2. Distribution of GRACE (Global Registry of Coronary Events) score in patient population

shows the distribution of GRACE scores in our patient population.

Risk-Adjusted Mortality for 2012 and 2013

Analogous to the assessment of CH-IQI, which is performed on a yearly basis, the results were stratified by year for risk adjustment using the GRACE score.

For 2012, after excluding patients with insufficient available data for risk adjustment (n = 61 [9.5%], of which 3 were in-hospital deaths), 613 patients were available for risk adjustment using the GRACE risk score. Their mean crude mortality was 6.0%, while the mean mortality expected after risk adjustment using the GRACE score for the same population was at 8.3% (95% CI 7.2–9.4%, p = 0.044). The crude mortality for 2012 was therefore significantly lower than the expected mortality rate using the GRACE risk score.

For 2013, 620 type 1 AMI cases were eligible for risk adjustment, while 20 (3.1%, of which 3 were deaths) were excluded owing to one or more missing variables. Crude mortality was 5.8%, while the mean expected mortality using GRACE risk score was 9.4 (95% CI 8.3–10.6%, p = 0.003). Therefore, for 2013 also, the crude mortality was lower than the expected mortality using the GRACE risk score (

Total	Cases identified with AMI type 1	1314
	From administrative data only	1221 (93.0%)
	From additional full-text search	93 (7.0%)
	Data available for risk-adjustment in (%)	1233 (93.8%)
2012	Cases identified with AMI type 1	674
	Data available for risk-adjustment in (%)	613 (90.5%)
	Crude mortality	6.0%
	Expected mortality using risk-adjustment with GRACE score (%)	8.3% (7.2–9.4%, p* = 0.044)
2013	Cases identified with AMI type 1	640
	Data available for risk-adjustment in (%)	620 (96.9%)
	Crude mortality	5.8%
	Expected mortality using risk-adjustment with GRACE score (%)	9.4% (8.3–10.6%, p* = 0.003)

AMI = acute myocardial infarction; GRACE score = Global Registry of Acute Coronary Events score. * Significance between actual (crude) and risk-adjusted mortality was tested using the chi-square test. p-values <0.05 were considered statistically significant.

).

Assessment of Working Hours Required

For all steps required in order to complete the risk adjustment (medical review, complementation of dataset, obtaining of variables), a total of 1150 working hours was required (575 hours per year and 55 minutes of work per case). This mostly consisted of (a) medical review of the full-text-based results, (b) identification of AMI subtype, and (c) the retrospective collection of variables which were not electronically available and which had to be obtained from the hospital archives. Only once all the variables were available, the calculation of the probability for in-hospital mortality using the GRACE risk model could be performed (Figure 1).

Discussion

We explored the feasibility of a novel approach using the GRACE risk score as an accurate, internationally validated [10,11,12] and accepted [13,14] tool for risk adjustment of in-hospital mortality as a QI for the treatment of AMI. This concept was developed based on the observation that the risk adjustment performed by CH-IQI using solely gender and age is rather insufficient for AMI.

Data Collection

We report three major findings during data collection. First, detailed medical review revealed a small, but relevant, discrepancy between the administrative database and the adjudicated cases of AMI. This observation highlights an important source of error for the assessment of the quality of care [17] and is supported by a recent report from Groene et al. [18].

Second, using GRACE adjusted mortality is feasible, with 93.8% of patients having all variables necessary for the calculation of the GRACE score available. While we consider the IT structure of our University hospital, and therefore this rate, representative of large hospitals in Switzerland or Europe, it is important to acknowledge that the rate may be lower in hospitals without electronic patient records.

Third, our retrospective approach for variable collection required substantial work. It is unlikely that these investments will be made by many institutions in the current reimbursement system. The high cost of the retrospective approach clearly argues in favour of prospectively documenting all GRACE variables (Figure 2). Although our study cannot exactly quantify the additional working hours needed to document all required variables electronically at admission, we think it is fair to hypothesise that their number would be far lower than observed in our retrospective approach. As long as the hospital reimbursement system and/or the administrative database do not distinguish type 1 from type 2 AMI, the exact classification of type 1 AMI will likely remain burdensome and an additional step.

Risk-Adjusted Mortality Using the GRACE Method

Risk adjustment using the GRACE score offers several advantages in comparison with the currently performed basic risk adjustment, which should improve comparability of quality of care across hospitals:

First, risk adjustment using the seven variables featured in the GRACE risk score is more appropriate than a risk adjustment by age and gender alone. The GRACE score features seven variables that showed the highest odds ratios for in-hospital death in AMI [9], the strongest variables for in-hospital death being cardiac arrest at admission and ST segment deviation in the ECG. Risk adjustment with the GRACE method therefore incorporates the case mix and especially variations in presentation in the acute scenario. Additional risk adjustment using the Charlson comorbidity index is, in our opinion, not sufficient for AMI, as it does not consider the vital, cardiac or laboratory parameters of the patient at admission, which will have a far greater impact on outcome and mortality.

Second, the GRACE risk score is a validated tool for the risk adjustment and mortality prediction in AMI patients, and recommended as such by the European Society of Cardiology [12,13]. In 2014, the GRACE risk score version 2.0 was released, a revised version of the original calculator, offering an even more simplified and therefore more effective tool as a quality indicator [19]. However, at the time of data collection, version 2.0 had not yet been externally validated, therefore we chose to perform the calculations using version 1.0.

Discharge treatment and door-to-balloon time are important additional QIs that also provide direct indications as to where process refinements should be focused.

Considerations for Clinical Use and Limitations

As discussed above, a prospective method of data collection for risk adjustment would be advantageous, less time-consuming and therefore more cost-effective than a retrospective approach. As it relies on clinical findings that leave little room for intra-observer variability and since the calculation is performed with an online tool, we find it to be a highly reproducible method. For clinical use, electronic and automatic calculation of GRACE scores would be recommended to fully optimise the process (Figure 3). Although our own electronic dataset does not currently allow such collection and automatic calculation of the GRACE score, internal discussions on implementing such methods into the hospital software have been held.

While further improvements can be made to the approach described in our study, this advanced method will still require additional efforts compared with the simplified method used by CH-IQI.

Performance was better than predicted by the GRACE model for both years, as well as all patient groups, and percutaneous coronary intervention seemed to be the most influential variable. The comparison of predicted versus crude mortality allows review of the performance of a hospital. It is conceivable that, due to the high standard of patient care in Switzerland, a small degree of overestimation of mortality with the GRACE score may occur. The GRACE risk score was developed using a multicentre, multinational cohort, and outcomes including mortality may vary with the geographical location, especially with differences in clinical practice [20,21].

In our university hospital, the standard cardiac troponin assay is the hs-cTnT assay. The GRACE risk score was derived and validated using less sensitive assays, which may have further influenced the results. As the prognostic accuracy of hs-cTnT is even higher as that of less sensitive assays, it seems fair to assume that this effect might have been small [13].

In conclusion, assessment of risk-adjusted in-hospital mortality in AMI is feasible, provides important insights regarding treatment results and could improve comparability between healthcare centres. If done retrospectively, it is very time-consuming, relying heavily on the quality and availability of the required data. Prospective documentation of the GRACE score within the electronic medical records would help to reduce substantially the effort needed to obtain this novel QI and improve reliability of data. Further studies in a multicentre setting are required in order to confirm the advantages of using the GRACE score as a quality indicator and tool for comparison of performance.

Disclosure Statement

No financial support and no other potential conflict of interest relevant to this article was reported.