Next Article in Journal
A Rare Complication of Ascariasis: A Case of Acute Interstitial Nephritis
Next Article in Special Issue
Heart Rate Recovery as a Predictor of Long-Term Adverse Events after Negative Exercise Testing in Patients with Chest Pain and Pre-Test Probability of Coronary Artery Disease from 15% to 65%
Previous Article in Journal
Imaging Outcomes of Emergency MRI in Patients with Suspected Cerebral Venous Sinus Thrombosis: A Retrospective Cohort Study
Previous Article in Special Issue
The Role of Cardiac Biomarkers in the Diagnosis of Hypertensive Emergency
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Maximizing Equity in Acute Coronary Syndrome Screening across Sociodemographic Characteristics of Patients

1
Department of Emergency Medicine, Stanford University, Palo Alto, CA 94304, USA
2
Tulane University School of Medicine, New Orleans, LA 70112, USA
3
Quantitative Sciences Unit, Stanford University, Palo Alto, CA 94304, USA
*
Author to whom correspondence should be addressed.
Diagnostics 2023, 13(12), 2053; https://doi.org/10.3390/diagnostics13122053
Submission received: 17 April 2023 / Revised: 22 May 2023 / Accepted: 6 June 2023 / Published: 14 June 2023
(This article belongs to the Special Issue Cardiovascular Diseases: Diagnosis and Management)

Abstract

:
We compared four methods to screen emergency department (ED) patients for an early electrocardiogram (ECG) to diagnose ST-elevation myocardial infarction (STEMI) in a 5-year retrospective cohort through observed practice, objective application of screening protocol criteria, a predictive model, and a model augmenting human practice. We measured screening performance by sensitivity, missed acute coronary syndrome (ACS) and STEMI, and the number of ECGs required. Our cohort of 279,132 ED visits included 1397 patients who had a diagnosis of ACS. We found that screening by observed practice augmented with the model delivered the highest sensitivity for detecting ACS (92.9%, 95%CI: 91.4–94.2%) and showed little variation across sex, race, ethnicity, language, and age, demonstrating equity. Although it missed a few cases of ACS (7.6%) and STEMI (4.4%), it did require ECGs on an additional 11.1% of patients compared to current practice. Screening by protocol performed the worst, underdiagnosing young, Black, Native American, Alaskan or Hawaiian/Pacific Islander, and Hispanic patients. Thus, adding a predictive model to augment human practice improved the detection of ACS and STEMI and did so most equitably across the groups. Hence, combining human and model screening––rather than relying on either alone––may maximize ACS screening performance and equity.

1. Introduction

Using predictive models in clinical care has high potential to improve care quality and equity, especially as the numbers of evidence-based standards for diagnosis and treatment increase. Such improvements can only be achieved, however, when existing biases in practice and clinical data are understood and accounted for in clinical implementation [1]. We explore this dynamic within emergency care, specifically screening for acute coronary syndrome (ACS) to capture patients with ST-elevation myocardial infarction (STEMI).
Emergency departments (EDs) have screening protocols to identify patients who present on arrival with symptoms concerning ACS [2]. As per the guidelines adopted internationally, these patients should receive an electrocardiogram (ECG) within 10 min of arrival at the ED to identify the severe sub-diagnosis called STEMI [3,4]. An early ECG, performed within 10 min, allows clinicians to diagnose and treat STEMI quickly [5]. For every minute that STEMI diagnosis is delayed, the interventions are less effective, and the risks of heart failure and mortality are increased [4,6,7]. In current practice, ACS screening is often performed manually by non-clinical staff. Once they determine an ECG should be performed, it is immediately reviewed by a physician to determine the next step in the patient’s care pathway.
In order to ensure usability, current screening uses simple criteria such as age and chief complaint to encourage consistent application [6]. This approach has been noted to underdiagnose minority groups and women [8,9,10,11]. In particular, a prior investigation showed that Black patients have a 51% greater chance of delayed ACS diagnosis than White patients, and female patients have a 36% greater chance of delay than male patients [4]. Furthermore, oversimplified screening is unable to capture variations in risk that are needed for a more precise prediction of ACS, leading to disparities in timely interventions, including thrombolysis or percutaneous coronary intervention (PCI) for those with STEMI. This compromises a critical step in the STEMI chain of survival [12].
Prior work has shown that predictive modeling can augment current screening practices to shorten the time to diagnosis for more patients [13]. This was evidenced by higher sensitivity for identifying ACS and an increased proportion of those subsequently diagnosed with STEMI due to earlier screening without increasing the number of ECGs performed [7]. It is not clear, however, whether a model using the same data as manual practice will improve screening quality for all patients or exacerbate existing disparities for women and minority groups [14]. In addition, the literature suggests the best use of predictive models in screening and diagnostic care is to augment human performance, but this has not been tested for this clinical screening challenge [15,16,17].
To explore this, we evaluated four ACS screening approaches and their performance across sex, age, race, ethnicity, and language groups. First, we measured the proportion of patients in a large, urban ED who received ECGs within 10 min of arrival (observed human practice). Next, we simulated three alternative screening approaches in the same population. We asked what would have happened if the clinical protocols were applied perfectly, if a statistical predictive model screened for ACS, and if the observed human practice was augmented with the predictive model.
For ACS screening and STEMI diagnosis, missing any case is unacceptable [18,19,20]. Identifying those with ACS risk is the early clinical goal [4,6,20], so we took the sensitivity of screening approaches as our primary measurement. Our objective was to quantify differences in these screening approaches by demographic subgroup, with the ultimate goal of informing the discussion on how using predictive algorithms in clinical practice can alleviate bias.

2. Methods and Materials

2.1. Study Design

This was a simulation of comparative ACS screening approaches vs. manual screening that occurred within actual care. For this, we used a 5-year retrospective electronic health record cohort including emergency department (ED) visits at Stanford University from 1 January 2015 to 31 December 2020 at one large urban hospital. We obtained an ethics review via the Stanford Institutional Review Board before initiating the study and collecting data (IRB number: 56066).

2.2. Screening Approaches

ACS screening is designed to identify patients for an “early ECG,” which is an ECG within 10 min of arrival to the ED. A “positive screen” is a patient who is identified as requiring an early ECG. We restricted our screening approaches to using only the data elements that (1) are typically available during ED registration and (2) are part of this institution’s current screening guidelines: age, chest pain, and other ACS-associated symptoms [15]. We describe the method for applying the screening criteria below.

2.2.1. Screen 1: Observed Human Practice

Registration clerks use arrival intake data such as age and chief complaint to determine whether a patient should receive an early ECG according to the ED’s screening protocol. To measure how the screening criteria are actually implemented in practice, positive screens were all patients who actually received an ECG within 10 min of arrival, regardless of what the protocol suggested.

2.2.2. Screen 2: Per Clinical Protocol

Using the existing ED screening protocols, positive screens included patients aged 65 years or older or those reporting chest pain or another ACS-associated symptom upon arrival. We applied these criteria to identify those who would require an early ECG if the protocols were strictly followed.

2.2.3. Screen 3: Predictive Model

Screening via the predictive model used the same ED arrival screening criteria as predictive independent variables, with a final diagnosis of ACS as the dependent variable. Additional modeling details are in the Statistical Analysis section below.

2.2.4. Screen 4: Model-Augmented Human Practice

Across industries, there are recommendations for artificial intelligence and predictive modeling intended to augment human performance; these recommendations encourage the use of models to bridge gaps in human prediction activity [11,21]. We explored this approach by combining the positive screens from observed practice with those who were screened positive by the predictive model. In other words, patients who were positive on either observed practice (Screen 1) or screening by the model (Screen 3) were considered positive under model-augmented human performance (Screen 4). Hence, the model served as a fail-safe: it was able to add positive screens but not remove a positive screening determination from practice.

2.3. Demographic Subgroups

We included demographic subgroups for which the literature has previously described variation in the timeliness of care [4,8,9,10,11,22,23]. These include sex (male or female), age (18–29, 30–49, 50–64, 65–80, and >80), race (Asian, Black/African American, Native American, Alaskan, or Hawaiian/Pacific Islander, White, other race, and unknown/refused), ethnicity (Hispanic or Non-Hispanic), and language (English, Spanish, or other language).

2.4. Outcomes

For patients, the primary outcome was a final hospital diagnosis of ACS as per methods previously published and validated, which use international classification of disease billing codes [22,24,25]. We also examined cases of STEMI to understand the impact of false negatives and effective screening capture of ACS patients for an early ECG to identify those with STEMI.
For the screening methods, our outcomes are test characteristics: sensitivity (primary), specificity, number of ECGs required, number of missed ACS cases, and number of missed STEMI cases.

2.5. Statistical Analysis

We present counts and percent distributions for each demographic characteristic among the full study population, the subset with confirmed ACS, and the subset with confirmed STEMI. To quantify incidence, we also present the observed number of ACS and STEMI cases per 10,000 for each demographic group.
For the predictive model, we fit a logistic regression model that uses the same data elements available at registration (chest pain, other ACS symptoms, age, and sex) to predict ACS. We calculated each of the measurement outcomes directly from standard 2 × 2 contingency tables that compared screening status with true ACS status within each demographic subgroup. Supplement A contains more details on the modeling and calculation of measurement outcomes.
Comparing each test characteristic across all demographic groups would run a high risk of false discovery due to multiple comparisons. As a result, we did not include hypothesis testing in this descriptive exploration. Rather, we calculated each measure and provided 95% confidence intervals around sensitivity and specificity to facilitate interpretation.

3. Results

The cohort included 279,132 ED patients, of whom 1397 had a final hospital diagnosis of ACS. Of those with ACS, 225 had a final diagnosis of STEMI (Table 1). Compared to the total population of patients, those with ACS and STEMI were older, more often male, more often White or Asian, and less often Hispanic/Latino (columns D and G in Table 1). The highest rates for ACS and STEMI per 10,000 patients were observed among patients over 65, men, and non-Hispanic/Latino patients (Table 1).

3.1. Screening Criteria/Predictor Prevalence

Figure 1 includes the screening characteristics or predictors used to identify those at risk of ACS. The distribution of these characteristics across subgroups of age, sex, race, ethnicity, and language suggests variation in the distribution of risk across the subgroups. Chest pain (Panel A) showed the least variation across the age subgroups. It ranged from 8–10% in most subgroups, although it was markedly lower among the very young (5%, 18–29 years) and the elderly (6%, >80 years). Other ACS-associated symptoms (Panel B) were more prevalent than chest pain in the general population, ranging from 33% to 60% among the subgroups. They were more commonly present among those over the age of 65 (51–60%) and those who spoke neither English nor Spanish (50%).
The subgroups varied most in proportion over age 65 (Panel C), especially by race, language, and ethnicity. Among all ED patients, the White and Asian subgroups were the oldest, with one-third or more over age 65. The other racial groups were much younger, with only 16% to 18% over age 65. These sharp differences in age structure by race also appeared within the ACS and STEMI populations. Figure 2 shows that strictly applying the cutoff at age 65 would miss most cases of ACS (Panel A) and STEMI (Panel B) among Black, NAAH/PI, and other race patients. The concentration of cases in the under-65 age group is even more pronounced for STEMI than for ACS for all groups except White patients (Figure 2, with point estimates and confidence interval detail in Supplement B).

3.2. Screening Performance Outcomes

Table 2 shows the overall measurement properties for each screening method. Observed human practice (Screen 1) had 73% sensitivity and 78% specificity for ACS, with 22% of all patients requiring an ECG. This approach missed 27% of ACS cases and 15% of STEMI cases. In comparison, screening per clinical protocol (Screen 2) had the worst overall performance, with low sensitivity (56%) and correspondingly high proportions of missed ACS cases (45%) and missed STEMI cases (53%). The predictive model (Screen 3) had higher sensitivity (82%) than observed human practice, with similar specificity (78%). The model’s operating point was selected to require the same proportion of ECGs as human practice (22%), but it missed fewer ACS cases (18% as opposed to 27%); the proportion of STEMI cases missed was the same (15%). Finally, model-augmented human practice (Screen 4) had the highest sensitivity, at 92%, and the fewest missed cases of ACS (8%) and STEMI (4%). This screen would require 33% of patients to receive an ECG, however, as opposed to 22% under current practice. This would represent a 57% relative increase in the number needed to screen.

3.3. Screening Performance Variation across Subgroups

Figure 3 shows the sensitivity and specificity for all four screens among the total sample and by demographic subgroup. Screening by observed human practice (Screen 1) shows low variability in sensitivity across subgroups, ranging from 66% to 77%. This screen is slightly less sensitive in women compared to men, in Hispanic/Latino patients compared to others, among speakers of languages other than English, and in those over 80 years old compared to younger patients. For specificity, screening in observed human practice was lower among non-Hispanic patients, speakers of languages other than English or Spanish, and patients older than age 65.
The per clinical protocol screen had the lowest sensitivity in nearly all groups by far, though it had correspondingly higher specificity compared to the other screening methods. This screen had the most variability in sensitivity across the subgroups, driven by its sharp age cut-off at 65, such that sensitivity is 0% for those under 65, 92% for those aged 65 to 80, and 96% for those over 80. Sensitivity varied across racial groups from as low as 34% among NAAH/PI patients and 37% among Blacks to over 60% among White and Asian patients, driven largely by the different age structures by race as shown in Figure 2.
Screening by predictive model (Screen 3) exhibited higher sensitivity compared to observed practice for all subgroups except people under age 65, where its sensitivity was lower but comparable. Overall, the specificity of the predictive model follows a similar pattern by subgroup to observed practice by race, ethnicity, and language. It has lower specificity for men (71%) compared to women (84%), however, and its specificity at older ages is quite low (9% over age 80).
Finally, screening with model-augmented human performance (Screen 4) had the highest sensitivity across all subgroups and screening methods, except for the over-80 subgroup, where it matched the 99% sensitivity of the predictive model. This screen showed little variation by sex, race, or language and most variation by age, ranging from 87% for patients under 50 to 99% for the oldest group. Across most subgroups, however, this consistently high sensitivity is reflected in an approximately 10 percent reduction in specificity (compared to observed human practice) among most subgroups. There were marked reductions in specificity among those over 80 and those who did not speak English or Spanish. Further exploration revealed that this latter population was far older than the average (results not shown) and mostly consisted of Asian language speakers.

3.4. Screening Gain When Human Performance Is Augmented with the Model

Figure 4 reveals the measurement gains from combining observed human practice (Screen 1) with the predictive model (Screen 3). The resulting model-augmented human performance method (Screen 4) demonstrated superior sensitivity in our tests. Panel A shows the overall incidence of ACS by subgroup, which varies most strongly by age. As discussed above, the variations by race and language are partly driven by differences in the age structure of these groups. Panel B shows the percentage of patients who screened positive under Screen 4, which dictates the number of ECGs required. The predictive model added most screen positives to the older age categories, making up for the lower sensitivity of observed practice at those ages. In the over-80 age group, observed human practice identified 35% of patients as being at ACS risk, and the predictive model added a full 57% of patients to that, raising the screen positives to 92% total in that group under Screen 4. The rise in sensitivity from adding the predictive model to observed practice for ACS is shown in Panel C and for STEMI in Panel D. Setting aside the very young ages of 18–29, which had only 6 cases of ACS and only 1 STEMI, the predictive model was able to add to the sensitivity of all subgroups, ranging from +9% to +33% for ACS and +2% to +22% for STEMI.

4. Discussion

Despite using the same data to determine who should receive an early ECG, the four screening approaches yielded distinct results. Observed human practice (Screen 1) had reasonable sensitivity and was fairly equitable across demographic subgroups (Figure 3). In contrast, screening per clinical protocol (Screen 2) had marked inequity in performance, largely driven by subgroup differences in age structure (Figure 2). As anticipated, the predictive model (Screen 3) had higher sensitivity than observed practice (Screen 1) for many subgroups, and it was far higher among the oldest patients. Overall, model-augmented human performance (Screen 4) had the highest sensitivity of all the screens in every subgroup. While it increased the total number of ECGs from 22% to 33% of all ED patients, it managed to identify 92% of ACS cases and 96% of STEMI cases. Compared to observed human practice, this augmented model required ECGs on an additional 12% of patients, leading to an additional 19% of the ACS cases and 10% of the STEMI cases being captured.

4.1. Variation in Age Distributions by Subgroup

Many of our results showing inequity across the subgroups were driven by differences in age structure, particularly by racial group. Among the subgroups we considered, rates of ACS per 10,000 vary most strongly by age, ranging up to 120 for patients over 80—by far the highest incidence of any group (Figure 4). This strong relationship means that if a subgroup has a different age structure than another, as shown in Figure 2 for racial groups, then any screen based on age will behave differently in that group. For example, 33% of non-Hispanic/Latino patients in our sample were aged 65 or older, compared to only 13% of all Hispanic/Latino patients. In an unexpected skew, a full 64% of “other” language speakers were 65 or older, compared to just 26% of English speakers and 20% of Spanish speakers. The variations in age structures by group are the product of social, medical, and historical circumstances that are well beyond the scope of this study. The consequence of potentially missed ACS screening is de facto inequities that arise in practice due to the race, ethnicity, and language of patients. Fortunately, multivariable models that include these types of characteristics, including nonlinear terms such as interactions, may result in better sensitivity and specificity of screening methods. The fairness and consequences of including race in clinical prediction models are widely debated [26,27]; in future research, we plan to investigate this question for our specific use case.

4.2. Challenge of Screening via Current Human Practice

Although observed practice did not have the highest sensitivity, it exhibited limited variability across demographic groups and fewer missed STEMI cases compared to the clinical protocols alone. We surmise that those performing manual screening leverage or respond to other available information, such as visual or linguistic cues about a patient’s condition, that influence their selection of patients [28].

4.3. Advantages of Predictive Modeling

The predictive model for Screen 3 included the same data elements used in observed care and in the clinical protocol (age, chest pain, and other ACS symptoms), yet it resulted in higher sensitivity in most subgroups. Its superior sensitivity to Screen 2 was largely due to the treatment of age: model 3 encoded age as a continuous variable rather than a blunt dichotomy at age 65. We cannot compare the working of the model to observed care because we have no insight into how the registration clerks made their decisions. We can, however, show that the model and the clerks identified somewhat different groups of patients for early ECGs.

4.4. Added Value of Model-Augmented Human Practice

Model-augmented human practice (Screen 4) had the highest sensitivity across all groups and above all other approaches. This was accompanied by a reduction in specificity across all groups due to a notable increase in false positives. Although increased testing from false positive screens is not desired, missed screening (false negative) delays diagnosis and is far less acceptable given the medical gravity of the delay in care. Indeed, prior work has suggested over-testing may be appropriate in certain populations that carry high risk due to the associated co-morbidity burden and communication delays associated with the need for translation [29]. Furthermore, additional ECGs are a relatively low-cost, rapidly performed diagnostic test.
We found that adding the predictive model to clinical practice would trigger ECGs for an additional 11% of patients (Table 2) while increasing sensitivity by 19 and 10 percentage points for ACS and STEMI, respectively (Figure 4, Panels C and D). This would result in a 72% [(374 − 106)/374] increase in the detection of ACS cases, and a 70% [(33 − 10)/33] increase in the detection of STEMI cases. (Table 2).
Augmented practice counterbalanced the demographic groups for which observed practice had lower sensitivity and greater variation in sensitivity. In particular, the patients added by Screen 3 were predominantly women, over the age of 80, and those who did not speak English or Spanish. This suggests that collaboration between humans and modeling can deliver better screening performance as well as improved equity. This accords with prior studies that observed that humans and predictive modeling have distinct strengths. Whereas models can be more consistent, humans can be more intuitive, emotional, and culturally sensitive in ways that we are just learning to quantify [16]. This suggests that the synergy of strengths from combining human performance with predictive modeling can optimize benefits for care delivery.

5. Limitations

The importance of considering chest pain as a predictive chief complaint and the challenge of balancing the consideration of ACS-associated chief complaints with a high false negative rate are universal. Thus, the patterns we observe are likely to be generalizable. Nevertheless, there are limitations that should be noted when interpreting our results.
Our study used a large sample of patients from a diverse patient population to conceptually explore ACS prediction performance, variation, and equity, but our subjects were drawn from a single-center study. Since the distribution of patient demographics of our study population may vary from other EDs, the prevalence and distribution of the predictive characteristics therefore may vary at other sites.
Our observations of Native American, Alaskan, or Hawaiian/Pacific Islander patients are novel and likely permitted by this being a population with notable representation in our regions; as such, we note that analogous disparities seen among black patients and women have been previously reported [4].
Finally, we report only one way in which models can augment care: by having a model run concurrently and augment human decision-making. There are other ways for models to augment human performance. For example, models being interactive for the screener, such as models suggesting specific questions that the person screening can ask to better ascertain the patient’s condition. These additional explorations could be the basis of future work.
We present this study as a conceptual simulation of screening options including augmented human practice using retrospective data. The next steps to better understand the implications of these models would be to perform prospective research through the implementation of these models into electronic health records. The move from desktop to bedside requires extensive work but will be critical to the future success of predictive models in healthcare. To build on our findings, we encourage research into these questions using prospective designs to account for data that may be missing at the time of risk calculation that are completed at a later time during the encounter. Our data were collected well after the 10-min time window to perform an ECG. The data could have been entered after those 10 min and therefore not appear as missing in our dataset. Therefore, it will be important to prospectively capture this data to identify whether the appropriate data are available within 10 min of arrival for the predictive model to be usable. This kind of time-sensitive missingness is not well represented in archived clinical data, but patterns associated with systematically missing data vs. ‘missingness at random’ may influence the performance of structured screening approaches more than human-driven practice [30].

6. Conclusions

We found that augmented screening (observed practice supplemented with the predictive model) had the best sensitivity (92.4%) and missed fewer ACS patients (7.6% vs. 26.8%) and STEMIs (4.4% vs. 14.7%) than observed human practice. It also showed little variation in sex, race, ethnicity, language, and age, demonstrating improved equity compared to other approaches. Augmented screening increased sensitivity by 20.8%, required ECGs to be performed on an additional 11.1% of patients, and reduced specificity by 14%. The increase in ECGs is likely within the tolerable range, however, given the marked gain in ACS case detection. Furthermore, it suggests that combining human and model screening––rather than relying on either one alone––may improve both overall performance and equity in ACS screening.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/diagnostics13122053/s1, Supplement A: Approach to the Predictive Model Screening for Acute Coronary Syndrome to Identify those who need an Electrocardiogram to Identify STEMI; Supplement B: Confidence Intervals and Point Estimates Associated with Figure 2: Sensitivity and Specificity of ACS screening Outcomes for Each Approach Across Sex, Race, Ethnicity, Language, and Age Demographic Subgroups.

Author Contributions

M.Y.A.B.Y. conceptualized the study, designed the analysis with K.M., provided clinical scientific oversight, and drafted the manuscript. A.G.-N. curated the study dataset. K.M. completed the data analysis. M.A.P. was the project manager for the study. G.B., S.M.B., D.K., A.G.-N. and R.K. participated in the study design, data analysis planning, and results interpretation. All authors made meaningful edits to the manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

The research reported in this publication was supported by the Stanford University Vice Provost and Dean of Research Office’s (VPDoR) Research on Racial Equity and Justice award number 257231. The content is solely the responsibility of the authors.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board (or Ethics Committee) of Stanford University (protocol code: 56066 and date of approval: 6 December 2020).

Informed Consent Statement

Patient consent was waived due to minimal risk to the patient due to our procedures involving the secondary use of clinical data.

Data Availability Statement

A de-identified version of this data may be made available upon request from the corresponding author, for the conduct of research, after review of the requestor’s intent and adherence with data sharing regulations.

Acknowledgments

Special thanks are given to Jon-Michael Knapp for his helpful comments on the manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Rajkomar, A.; Hardt, M.; Howell, M.D.; Corrado, G.; Chin, M.H. Ensuring fairness in machine learning to advance health equity. Ann. Intern. Med. 2018, 169, 866–872. [Google Scholar] [CrossRef]
  2. O’gara, P.T.; Kushner, F.G.; Ascheim, D.D.; Casey, D.E.; Chung, M.K.; De Lemos, J.A.; Ettinger, S.M.; Fang, J.C.; Fesmire, F.M.; Franklin, B.A.; et al. 2013 ACCF/AHA guideline for the management of ST-elevation myocardial infarction: A report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. J. Am. Coll. Cardiol. 2013, 61, e78–e140. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Coyne, C.J.; Testa, N.; Desai, S.; Lagrone, J.; Chang, R.; Zheng, L.; Kim, H. Improving door-to-balloon time by decreasing door-to-ECG time for walk-in STEMI patients. West. J. Emerg. Med. 2015, 16, 184. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Yiadom, M.Y.; Gong, W.; Patterson, B.W.; Baugh, C.W.; Mills, A.M.; Gavin, N.; Podolsky, S.R.; Salazar, G.; Mumma, B.E.; Tanski, M.; et al. Fallacy of Median Door-to-ECG Time: Hidden Opportunities for STEMI Screening Improvement. J. Am. Heart Assoc. 2022, 11, e024067. [Google Scholar] [CrossRef] [PubMed]
  5. Yiadom, M.Y.; Gong, W.; Bloos, S.; Liu, D. 287 Influence of Time to Diagnosis on Time to Percutaneous Coronary Intervention for Emergency Department ST Elevation Myocardial Infarction (STEMI) Patients: Door to ECG Matters. Ann. Emerg. Med. 2022, 80, S125. [Google Scholar] [CrossRef]
  6. Gho, J.; Postema, P.G.; Conijn, M.; Bruinsma, N.; de Jong, J.S.S.G.; Bezzina, C.R.; Wilde, A.A.M.; Asselbergs, F.W. Heart failure following STEMI: A contemporary cohort study of incidence and prognostic factors. Open Heart 2017, 4, e000551. [Google Scholar] [CrossRef] [Green Version]
  7. McNamara, R.L.; Wang, Y.; Herrin, J.; Curtis, J.P.; Bradley, E.H.; Magid, D.J.; Peterson, E.D.; Blaney, M.; Frederick, P.D.; Krumholz, H.M.; et al. Effect of door-to-balloon time on mortality in patients with ST-segment elevation myocardial infarction. J. Am. Coll. Cardiol. 2006, 47, 2180–2186. [Google Scholar] [CrossRef] [Green Version]
  8. Lehmann, J.B.; Wehner, P.S.; Lehmann, C.U.; Savory, L.M. Gender bias in the evaluation of chest pain in the emergency department. Am. J. Cardiol. 1996, 77, 641–644. [Google Scholar] [CrossRef]
  9. Jacobi, J.A.; Parikh, S.V.; McGuire, D.K.; Delemos, J.A.; Murphy, S.A.; Keeley, E.C. Racial disparity in clinical outcomes following primary percutaneous coronary intervention for ST elevation myocardial infarction: Influence of process of care. J. Interv. Cardiol. 2007, 20, 182–187. [Google Scholar] [CrossRef]
  10. Milner, K.A.; Vaccarino, V.; Arnold, A.L.; Funk, M.; Goldberg, R.J. Gender and age differences in chief complaints of acute myocardial infarction (Worcester Heart Attack Study). Am. J. Cardiol. 2004, 93, 606–608. [Google Scholar] [CrossRef]
  11. Dey, S.; Flather, M.D.; Devlin, G.; Brieger, D.; Gurfinkel, E.P.; Steg, P.G.; Fitzgerald, G.; Jackson, E.A.; Eagle, K.A. Sex-related differences in the presentation, treatment and outcomes among patients with acute coronary syndromes: The Global Registry of Acute Coronary Events. Heart 2009, 95, 20–26. [Google Scholar] [CrossRef] [PubMed]
  12. Ornato, J.P. The ST-segment–elevation myocardial infarction chain of survival. Circulation 2007, 116, 6–9. [Google Scholar] [CrossRef] [Green Version]
  13. Bunney, G.; Sundaram, V.; Graber-Naidich, A.; Brown, I.; McCoy, A.; Freeze, B.; Berger, D.; Wright, A.; Yiadom, M.Y.A.B. Beyond Chest Pain: Incremental Value of Other ACS Chief Complaints, Age, and Sex to Identify Patients at Risk of ACS for an Early ECG. Am. J. Emerg. Med. 2023, 67, 70–78. [Google Scholar] [CrossRef]
  14. Bracic, A.; Callier, S.L.; Price, W.N. Exclusion cycles: Reinforcing disparities in medicine. Science 2022, 377, 1158–1160. [Google Scholar] [CrossRef] [PubMed]
  15. De Cremer, D.; Kasparov, G. AI should augment human intelligence, not replace it. Harv. Bus. Rev. 2021, 18, 18. [Google Scholar]
  16. Wilson, H.J.; Daugherty, P.R. Collaborative intelligence: Humans and AI are joining forces. Harv. Bus. Rev. 2018, 96, 114–123. [Google Scholar]
  17. Reddy, S.; Fox, J.; Purohit, M.P. Artificial intelligence-enabled healthcare delivery. J. R. Soc. Med. 2019, 112, 22–28. [Google Scholar] [CrossRef]
  18. Yiadom, M.Y.; Baugh, C.W.; McWade, C.M.; Liu, X.; Song, K.J.; Patterson, B.W.; Jenkins, C.A.; Tanski, M.; Mills, A.M.; Salazar, G.; et al. Performance of emergency department screening criteria for an early ECG to identify ST-segment elevation myocardial infarction. J. Am. Heart Assoc. 2017, 6, e003528. [Google Scholar] [CrossRef] [Green Version]
  19. Gallagher, S.; Knight, C.; Wragg, A. Medicolegal pitfalls in the management of chest pain. Clin. Risk 2010, 16, 161–168. [Google Scholar] [CrossRef]
  20. Kwok, C.S.; Bennett, S.; Azam, Z.; Welsh, V.; Potluri, R.; Loke, Y.K.; Mallen, C.D. Misdiagnosis of acute myocardial infarction: A systematic review of the literature. Crit. Pathw. Cardiol. 2021, 20, 155–162. [Google Scholar] [CrossRef]
  21. Davenport, T.H.; Glover, W.J. Artificial intelligence and the augmentation of health care decision-making. NEJM Catal. 2018, 19, 4. [Google Scholar]
  22. Bloos, S.M.; Kaur, K.; Lang, K.; Gavin, N.; Mills, A.M.; Baugh, C.W.; Patterson, B.W.; Podolsky, S.R.; Salazar, G.; Mumma, B.E.; et al. Comparing the Timeliness of Treatment in Younger vs. Older Patients with ST-Segment Elevation Myocardial Infarction: A Multi-Center Cohort Study. J. Emerg. Med. 2021, 60, 716–728. [Google Scholar] [CrossRef] [PubMed]
  23. Yiadom, M.Y.; Baugh, C.W.; Jenkins, C.A.; Tanski, M.; Mumma, B.E.; Vogus, T.J.; Miller, K.F.; Jackson, B.E.; Lehmann, C.U.; Dorner, S.C.; et al. Outcome Differences Associated with STEMI Diagnostic Delay: Disparities on the Frontlines of STEMI Care. Circ. Cardiovasc. Qual. Outcomes 2018, 11 (Suppl. S1), A185. [Google Scholar] [CrossRef]
  24. Yiadom, M.Y.; Mumma, B.E.; Baugh, C.W.; Patterson, B.W.; Mills, A.M.; Salazar, G.; Tanski, M.; Jenkins, C.A.; Vogus, T.J.; Miller, K.F.; et al. Measuring outcome differences associated with STEMI screening and diagnostic performance: A multicentred retrospective cohort study protocol. BMJ Open 2018, 8, e022453. [Google Scholar] [CrossRef]
  25. Yiadom, M.Y.; Olubowale, O.O.; Jenkins, C.A.; Miller, K.F.; West, J.L.; Vogus, T.J.; Lehmann, C.U.; Antonello, V.D.; Bernard, G.R.; Storrow, A.B.; et al. Understanding timely STEMI treatment performance: A 3-year retrospective cohort study using diagnosis-to-balloon-time and care subintervals. J. Am. Coll. Emerg. Physicians Open 2021, 2, e12379. [Google Scholar] [CrossRef]
  26. Vyas, D.A.; Eisenstein, L.G.; Jones, D.S. Hidden in Plain Sight—Reconsidering the Use of Race Correction in Clinical Algorithms. N. Engl. J. Med. 2020, 383, 874–882. [Google Scholar] [CrossRef] [PubMed]
  27. Mhasawade, V.; Zhao, Y.; Chunara, R. Machine learning and algorithmic fairness in public and population health. Nat. Mach. Intell. 2021, 3, 659–666. [Google Scholar] [CrossRef]
  28. Mongillo, G.; Shteingart, H.; Loewenstein, Y. Race against the machine. Proc. IEEE 2014, 102, 542–543. [Google Scholar] [CrossRef] [Green Version]
  29. Khare, R.K.; Nannicelli, A.P.; Powell, E.S.; Seivert, N.P.; Adams, J.G.; Holl, J.L. Use of risk assessment analysis by failure mode, effects, and criticality to reduce door-to-balloon time. Ann. Emerg. Med. 2013, 62, 388–398. [Google Scholar] [CrossRef]
  30. Groenwold, R.H. Informative missingness in electronic health record systems: The curse of knowing. Diagn. Progn. Res. 2020, 4, 8. [Google Scholar] [CrossRef]
Figure 1. Prevalence of key screening criteria/predictors by demographic group. NAAH/PI = Native American, Alaskan, or Hawaiian/Pacific Islander. This figure presents the prevalence of the screening criteria used as guidance for human practice and as predictors in the model.
Figure 1. Prevalence of key screening criteria/predictors by demographic group. NAAH/PI = Native American, Alaskan, or Hawaiian/Pacific Islander. This figure presents the prevalence of the screening criteria used as guidance for human practice and as predictors in the model.
Diagnostics 13 02053 g001
Figure 2. Acute coronary syndrome (ACS) and STEMI patient age by race. Using ‘age >65 years’ as a screening criterion to identify those at risk of ACS introduces substantial inequity. Panel A shows that Black and NAAH/PI ACS patients are generally younger than White, Asian, or other race patients, and many are under age 65. The concentration of cases in the younger age group is even more pronounced among STEMI patients (Panel B), including those of Asian or other race. The sharp cut-off at age 65 for screening inadvertently disadvantages sub-groups with younger age structures, which can fall along racial lines in the US. NAAH/PI = Native American, Alaskan, or Hawaiian/Pacific Islander.
Figure 2. Acute coronary syndrome (ACS) and STEMI patient age by race. Using ‘age >65 years’ as a screening criterion to identify those at risk of ACS introduces substantial inequity. Panel A shows that Black and NAAH/PI ACS patients are generally younger than White, Asian, or other race patients, and many are under age 65. The concentration of cases in the younger age group is even more pronounced among STEMI patients (Panel B), including those of Asian or other race. The sharp cut-off at age 65 for screening inadvertently disadvantages sub-groups with younger age structures, which can fall along racial lines in the US. NAAH/PI = Native American, Alaskan, or Hawaiian/Pacific Islander.
Diagnostics 13 02053 g002
Figure 3. Sensitivity and specificity of screening approaches for acute coronary syndrome by demographic subgroup. NAAH/PI = Native American, Alaskan, or Hawaiian/Pacific Islander. screening per clinical protocol had the widest variation in sensitivity across the demographic groups, and observed human practice had lower variation. When observed practice was augmented by the predictive model, sensitivity increased for all sub-groups except those >80 years of age, which already had a sensitivity of nearly 100%. Increased ACS case detection with the model-augmented human practice screen results in a lower specificity by about 10 percentage points compared to observed human practice alone. Sensitivity and specificity are not equally important, however, given the high intolerance for missed ACS and the low burden of performing additional early ECGs.
Figure 3. Sensitivity and specificity of screening approaches for acute coronary syndrome by demographic subgroup. NAAH/PI = Native American, Alaskan, or Hawaiian/Pacific Islander. screening per clinical protocol had the widest variation in sensitivity across the demographic groups, and observed human practice had lower variation. When observed practice was augmented by the predictive model, sensitivity increased for all sub-groups except those >80 years of age, which already had a sensitivity of nearly 100%. Increased ACS case detection with the model-augmented human practice screen results in a lower specificity by about 10 percentage points compared to observed human practice alone. Sensitivity and specificity are not equally important, however, given the high intolerance for missed ACS and the low burden of performing additional early ECGs.
Diagnostics 13 02053 g003
Figure 4. ACS and STEMI incidence, positive ACS screens, and true positive case capture. NAAH/PI = Native American, Alaskan, or Hawaiian/Pacific Islander. Panel A presents the incidence of ACS across the demographic subgroups, illustrating differential risk. Panel B presents the percent of the total ED population who screened positive under observed human practice (yellow) and the positive screens that would be added under the model-augmented human practice approach (orange). Panels C and D present the proportion of ACS patients in each subgroup identified via observed human practice (green) and those that would be added with model-augmented human practice screening (dark green).
Figure 4. ACS and STEMI incidence, positive ACS screens, and true positive case capture. NAAH/PI = Native American, Alaskan, or Hawaiian/Pacific Islander. Panel A presents the incidence of ACS across the demographic subgroups, illustrating differential risk. Panel B presents the percent of the total ED population who screened positive under observed human practice (yellow) and the positive screens that would be added under the model-augmented human practice approach (orange). Panels C and D present the proportion of ACS patients in each subgroup identified via observed human practice (green) and those that would be added with model-augmented human practice screening (dark green).
Diagnostics 13 02053 g004
Table 1. Demographic characteristics of the total emergency department compared to ACS and STEMI patients with prevalence.
Table 1. Demographic characteristics of the total emergency department compared to ACS and STEMI patients with prevalence.
TotalACS PatientsSTEMI Patients
Column:ABCDEFGH
NColumn
%
NColumn
%
Cases per 10KNColumn %Cases per 10K
Total 279,132100%1397100%50225100%8.1
Age
Group
18 to 2958,40721%60%110%0.2
30 to 4981,81129%15111%18.53214%3.9
50 to 6461,22822%41129%67.18337%13.6
65 to 8049,87918%49435%996730%13.4
Over 8027,80710%33524%120.54219%15.1
SexFemale154,48555%55340%35.86629%4.3
Male124,64745%84460%67.715971%12.8
Race
(In order of group size)
White123,80644%69450%56.112154%9.8
Other82,86930%29321%35.43917%4.7
Asian38,63114%25018%64.74118%10.6
Black22,5968%937%41.294%4
NAAH/PI82163%473%57.252%6.1
Unknown/
Refused
27451%181%65.684%29.1
Hispanic/
Latino
Yes68,53425%20015%29.23114%4.5
No207,77675%117785%56.618886%9
LanguageEnglish233,55684%113381%48.519185%8.2
Spanish29,70411%1249%41.7167%5.4
Other15,6316%14010%89.6188%11.5
NAAH/PI = Native American, Alaskan, or Hawaiian/Pacific Islander.
Table 2. Comparing screening approaches to identify patients at risk of ACS for an early ECG: sensitivity, specificity, ECGs required, and missed ACS and STEMI screens.
Table 2. Comparing screening approaches to identify patients at risk of ACS for an early ECG: sensitivity, specificity, ECGs required, and missed ACS and STEMI screens.
Among All Patients (N = 279,132)Among
ACS Cases
(N = 1397)
Among
STEMI Cases
(N = 225)
ScreenSensitivity
(95% CI)
Specificity
(95% CI)
Positive Screens
N
Requiring ECGs
%
Requiring ECGs
N
Missed
%
Missed
N
Missed
%
Missed
1. Observed practice73.2%
(70.8–75.5%)
78.3%
(78.2–78.5%)
61,15621.9%37426.8%3314.7%
2. Screening by protocol55.5%
(52.9–58.2%)
83.4%
(83.3–83.6%)
46,81816.8%62144.5%11952.9%
3. Screening by the predictive model81.9%
(79.8–83.9%)
78.1%
(77.9–78.2%)
62,01122.2%25318.1%3314.7%
4. Observed practice augmented with a predictive model92.4%
(90.9–93.7%)
67.3%
(67.2–67.5%)
91,99433.0%1067.6%104.4%
Screening per protocol had the lowest sensitivity at 55.5%. Observed practice augmented by the predictive model achieved the highest sensitivity of 92.4% and missed the fewest ACS and STEMI cases.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Bunney, G.; Bloos, S.M.; Graber-Naidich, A.; Pasao, M.A.; Kabeer, R.; Kim, D.; Miller, K.; Yiadom, M.Y.A.B. Maximizing Equity in Acute Coronary Syndrome Screening across Sociodemographic Characteristics of Patients. Diagnostics 2023, 13, 2053. https://doi.org/10.3390/diagnostics13122053

AMA Style

Bunney G, Bloos SM, Graber-Naidich A, Pasao MA, Kabeer R, Kim D, Miller K, Yiadom MYAB. Maximizing Equity in Acute Coronary Syndrome Screening across Sociodemographic Characteristics of Patients. Diagnostics. 2023; 13(12):2053. https://doi.org/10.3390/diagnostics13122053

Chicago/Turabian Style

Bunney, Gabrielle, Sean M. Bloos, Anna Graber-Naidich, Melissa A. Pasao, Rana Kabeer, David Kim, Kate Miller, and Maame Yaa A. B. Yiadom. 2023. "Maximizing Equity in Acute Coronary Syndrome Screening across Sociodemographic Characteristics of Patients" Diagnostics 13, no. 12: 2053. https://doi.org/10.3390/diagnostics13122053

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop