Predictive Value of Clinical Risk Assessment Tools and Guidelines for 10-Year Coronary Heart Disease Risk in Practice-Based Primary Care

Summary

Background: Risk for developing myocardial infarction derived from risk tables in primary care subjects in Switzerland may over- or underestimate risk. Coronary calcifications may improve the performance of risk tables.

Methods: We used coronary calcium score percentiles >50 (CS% >50) as a surrogate marker for 10-year myocardial infarction risk in a prospective cross-sectional monocenter study. CS% >50 was compared to several risk charts, was used to reclassify subjects in the intermediate risk category assessed by Framingham risk scores (FRS), and was used to calculate a cohort specific correction factor for FRS and PROCAM. Results from risk charts were entered into the Bayes formula as the pretest risk estimates. Subjects in our cohort were 100 primary care patients with no known history of cardiovascular disease randomly selected from three primary care practices.

Results: The sensitivity of FRS to detect CS% >50 was 47%, and specificity was 85%. NCEP III and Swiss guidelines had sensitivities of 53% and 67%, respectively (p = ns), and specificities of 66% and 67% (p <0.05). In 21 subjects with intermediate risk as assessed by FRS, CS% >50-derived post-test probabilities shifted 16 subjects (76%) into the low-risk category and 5 subjects (24%) into the high-risk category. Cohort-specific correction factors were 0.68 for FRS and 0.64 for PROCAM.

Conclusions: For our Swiss German cohort, FRS and PROCAM tended to overestimate risk. A biological risk marker (coronary calcifications) may allow improvement of risk prediction in primary care subjects with intermediate risk and also helps in the calculation of cohort-specific correction factors.

Key words: myocardial infarction; coronary calcification; cardiovascular risk charts

Introduction

Coronary heart disease (CHD) presents as myocardial infarction or sudden death in 60% of the cases [1]. Therefore, early detection of individuals at risk for CHD is crucial to promote risk-modifying behaviors and initiate appropriate medical management [2,3,4,5,6]. Accordingly, cardiovascular risk assessment tools such as the Framingham risk charts [7, 10] and PROCAM [9] are used to guide preventive measures on both the North American and European continent [7,8,9,10,11]. These risk assessment tools, although useful, miss a substantial portion of patients who are at risk of an event, and these tools also have not been validated in several European countries where there might be a greater prevalence of low-risk populations [6]. One example of such low-risk countries is Switzerland, where life style and genetic heritage make a “French paradox” very likely. Hence, measurements of atherosclerosis burden may provide additional information to allow the refinement of risk assessment.

In this study, we used a very sensitive biological marker for 10-year risk of developing hard coronary events (fatal or non-fatal myocardial infarction) obtained from percentiles of coronary calcifications above the 50th percentile (CS% >50), a strong marker of CHD risk [12,13,14,15]. The first objective of the study was to compare CS% >50 with the risk obtained from several coronary risk charts. The second objective was to improve risk prediction in the group of intermediate-risk patients, where about one-third of coronary events occur within 10 years [9]. The third objective was to determine an adjustment for risk in a lowerrisk population by calculating posttest probabilities based on the widely accepted Bayes formula that would correct results from coronary risk charts. These calculations may be helpful for working groups that are developing new guidelines to create region-specific correction factors.

Material and methods

• Patient selection

The Ethics Committee of the Canton Solothurn, Switzerland, approved this study. Our study subjects were randomly selected from a Swiss German primary care setting. Three primary care physicians participated in this prospective cross-sectional study. Their practices are located in Olten and Trimbach, Switzerland, an area with approximately 20 000 total rural and urban inhabitants. The three physicians were asked to recruit 100 subjects by randomly selecting subjects from consecutive patients entering their practices between the hours of 9:00 a.m. and 10:00 a.m. Subjects from age 35 to 75 years were included if they had no known cardiovascular disease, severe claustrophobia, or chronic arrhythmias (such as atrial fibrillation and frequent premature ventricular contractions); they were excluded if they had a life expectancy of less than two years or were unwilling or unable to give informed written consent. Women of childbearing age who were not using birth control were also excluded from the study.

Of the original 304 subjects that were screened, 88 patients (29%) were excluded because their ages were under 35 years or over 75 years; 47 patients (15%) were excluded because they had a history of vascular disease (such as previous myocardial infarction or stroke), frequent arrhythmias, or severe claustrophobia, because they had a life expectancy of less than two years, or because they were unable to understand German; and 69 patients (23%) were excluded because they did not consent to participate. A total of 100 primary care patients were entered into the study.

• Coronary imaging method

All imaging studies were performed centrally at the RODIAG-Institute, Olten, Switzerland. Coronary calcium screening was performed with a multi-detector computed tomography scanner (MDCT, Aquilion, Toshiba, Tokyo, Japan) with prospective triggering (starting at 50% of the R wave). Images (2-mm slice thickness) were transferred to a workstation for quantification of coronary calcium using NetraMD Software (ScImage Palo Alto, California, USA). High risk for fatal and non-fatal myocardial infarction was defined as coronary calcium score percentile (CS%) >50 (ten-year risk for fatal and non-fatal myocardial infarction 20%). Percentiles were obtained by comparing the calculated scores with the published scores of 10 122 asymptomatic subjects screened by electron beam computed tomography (EBCT) [19]. Interobserver variability for coronary calcium score percentile calculation was 0.2 ± 1.3% in 96 subjects randomly selected from a database of 240 subjects in our imaging center and the interscan variability was 6 ± 7%.

• Assessment of risk and statistical methods

Risk comparisons were based for all tests on estimates for the occurrence of fatal and non-fatal myocardial infarction within 10 years. For the calculation of risk according to the Framingham Risk Score (FRS) [16], the following variables were initially considered: age, gender, smoking status, diabetes mellitus, and hypertension. According to the recommendations of the Adult Treatment Panel III (ATP-III), risk was then calculated for each patient on the basis of age, gender, smoking status, total cholesterol, HDL cholesterol, and systolic blood pressure [10].

The PROCAM algorithm, which is only valid for men aged from 35 to 65 years, was applied to the 45 men in our cohort who were between the ages of 35 and 65, also using the variables LDL-cholesterol, HDL-cholesterol, triglycerides, systolic blood pressure, smoking habits, diabetes mellitus, and family history for premature CAD [9]. Total cholesterol, triglycerides, HDL-cholesterol, and glucose were measured after an 8-hour fast.

Coronary calcium percentiles (CS%) used for this study were originally generated from a large US group of subjects having included 5684 males and 4438 women with positive Agatston scores only. Therefore, percentiles were calculated using only an Agatston Score >0. This was done to potentially reduce the effect of referral bias. In fact, referring large numbers of patients with a low pretest likelihood of disease may skew the percentiles toward lower values if all 0 scores were included [19]. Consequently, in our study, we used these CS% to adjust risk estimates derived from FRS and PROCAM, in order to reduce the effects of referral bias inherent in CS% having included large number of subjects with very low risk for coronary events. Using posterior probabilities derived from the Bayes formula performed adjustment. Eg, a posterior probability would be left unchanged, if a test had a sensitivity and specificity of 50%, which equals the likelihood of coin flipping. We used the published sensitivities and specificities for risk of future myocardial infarction derived from CS% that was observed in 676 physician referred asymptomatic middle aged men and women with an intermediate annual risk of 1.7% for myocardial infarction [20]. From this paper, posterior probabilities were calculated from sensitivities and specificities for CS% (Table 3). Finally, the risk for myocardial infarction was adjusted by using correction factors that were calculated by dividing the post-test risk values derived from CS% by the pre-test risk values derived from either FRS (all subjects) or PROCAM (men only).

Data were compiled in a Microsoft^® Office Excel data sheet (Microsoft, Redmond, WA, USA) and further analysed using GB-STAT Cohne^® version 9.0 (Dynamic Microsystems, 2000, Silver Spring, MD, USA) using Chi2-statistics. For statistical analysis, the level of significance was set at <0.05.

Results

Patient characteristics are provided in Table 1.

• Risk derived from a Framingham Risk Scoreand PROCAM

Using the Framingham Risk Scores (FRS), risk estimates to detect 10-year risk for fatal and non-fatal myocardial infarction showed 24 subjects to be at low risk (5–9%), 35 subjects at very low risk (<5%), 21 subjects at intermediate (10–19%), and 20 subjects at high risk (020%) (Figure 1). The mean 10-year FRS risk for the entire cohort was 10.6%. Similar calculations performed in 45 men assessed by the PROCAM score showed a mean 10-year PROCAM risk of 10.8%.

• Coronary calcium percentiles

Sixty-five subjects had no detectable coronary calcifications on MDCT. In the 35 subjects with coronary calcium, 15 had a percentile value >50 (Figure 2).

• Performance of different risk assessment strategies tod etect CS% > 50

Table 2 summarises the sensitivities, specificities, positive and negative predictive values, and accuracies of the various cardiovascular risk assessment tools. It also includes the FRS and PROCAM values that were a result of applying the correction factors calculated from the results of this study for these risk tables.

Swiss guidelines (sensitivity 67%, Chi2 = 1.22, p = 0.269) tended (p = ns) to show increased sensitivity in comparison to FRS (sensitivity 47%), but not NCEP III (sensitivity 53%, Chi2 = 0.18, p = 0.715).

Swiss guidelines and NCEP III both showed lower specificities of 67% (Chi2 = 7.23, p = 0.007) and 66% (Chi2 = 7.12, p = 0.008), respectively, when compared to FRS. The specificity of FRS for prediction of a CS% 050 was 85%.

• Coronary calcium percentiles as a possible modifier of FRS-derived risk estimates

The Bayes formula was used to compute posttest probabilities, with the FRS risk estimates as the pre-test value [20]. Five (24%) of the 21 intermediate-risk subjects were reclassified into the high-risk group, and 16 (76%) were reclassified into the low-risk group. Eleven (55%) of the 20 subjects with high risk as defined by FRS were reclassified into the low-risk group, and 2 subjects of the high-risk group were reclassified into the intermediate-risk group. In the low-risk group, only 2 (3%) of 51 subjects were reclassified as high-risk subjects; the remaining 48 (94%) subjects remained in the low-risk group.

• Coronary calcium percentiles as a possible marker to calculate population-specific correction factors

By adjusting FRS risk estimates on the basis of the post-test probability risk values determined by coronary calcium score percentiles (Table 3), the average FRS 10-year risk for myocardial infarction decreased from 10.6 to 7.2% in this cohort, yielding a correction factor of 0.68.

Table 1. Clinical characteristics of patients in the study group (n = 100).

Table 1. Clinical characteristics of patients in the study group (n = 100).

Table 2. Performance of different risk assessment tools to detect CS% > 50.

Table 2. Performance of different risk assessment tools to detect CS% > 50.

The PROCAM algorithm showed a sensitivity of 67% and a specificity of 87% to predict a CS% >50 in 12 men. When PROCAM risk estimates were adjusted on the basis of post-test risk values determined from coronary calcium score percentiles (Table 3), the PROCAM 10-year risk for myocardial infarction decreased from 10.8 to 6.9%, yielding a correction factor of 0.64.

Discussion

The European guidelines for coronary risk estimates were published in 1998 [7] and were based on data from the Framingham study cohort. Risk estimation in the Framingham paradigm includes soft (angina) as well as hard CHD events and has been shown to markedly overestimate risk in an Italian cohort [6]. The European Task Force was aware of these issues and called for national adjustments to their guidelines for cardiovascular risk assessment [7]. Since the publication of these European guidelines, there has been a shift of paradigm in risk prediction. Prediction of soft events has been removed from the recent NCEP III guidelines [10], from the PROCAM algorithm [9], and from the new European guidelines termed EU-SCORE [25]. As a consequence, the specificity of the PROCAM algorithm [9] increased to 95%. This high specificity of PROCAM is associated with a very high accuracy (91%), but also, unfortunately, with a very low sensitivity (33%) for detecting future myocardial infarctions [9]. The EU-SCORE guidelines also show low sensitivity and high specificity for global cardiovascular mortality risk [25].

Table 3. Sensitivities and specificities for CS% to detect 10-year risk for the combined end point of fatal and non-fatal myocardial infarction (modified from [20]).

Table 3. Sensitivities and specificities for CS% to detect 10-year risk for the combined end point of fatal and non-fatal myocardial infarction (modified from [20]).

This is, to our knowledge, the first study investigating coronary risk using coronary calcium in a low risk population. Our study cohort allowed us to calculate coronary risk estimates derived from a rather limited number of patients in small towns (Olten and Trimbach, Switzerland). The Swiss population is at relatively low risk for coronary events, comparable to Mediterranean countries such as Italy and France [22], but there are no data on coronary risk in the Swiss German part of Switzerland, a region that did not participate in the MONICA project [24]. In addition, none of the widely used risk assessment charts [8–11, 16] have been validated for a Swiss or, more specifically, for a Swiss German population, and, in general, very few data exist for low-risk populations. Therefore, our goal was to find a reasonable surrogate marker of outcome. We have chosen a coronary calcium score percentile (CS%) above 50 for this purpose because a CS% >50 has been shown in a USA cohort to be a highly sensitive (93%), although not very specific (specificity 52%) biological or “imaging” marker for detecting the 10-year risk for myocardial infarction [19].

In our study we first compared CS% >50 with several risk charts for the occurrence of 10-year risk of hard coronary events (fatal or non-fatal myocardial infarction). Since the original Framingham cohort includes also risk for soft coronary events (eg, angina) [7], we have chosen a modification of FRS, which calculates risk for myocardial infarction only [16]. Swiss guidelines for the treatment of cholesterol published in 1999 [8] show a better sensitivity for detecting CS% >50 than FRS (67 versus 47%, p = ns), but at a significant loss of specificity (67 versus 85%, p = 0.007) when compared to FRS. This may lead to overtreatment and most probably increased cost of treatment. When we applied NCEP III guidelines [10] to our population, we also observed a lower sensitivity (53%) than the Swiss guidelines and a lower specificity than FRS (66 versus 85%, p = 0.008). Therefore, the application of NCEP III guidelines is inadequate in Swiss primary care subjects.

We evaluated CS% >50 as an assessment tool for intermediate risk patients in comparison to other risk assessment tools. Twenty nine percent of myocardial infarctions have been observed to occur in intermediate-risk patients [9]. For this purpose, we reclassified patients assessed with FRS [16] according to their coronary calcium percentiles using post-test probabilities that are based on the Bayes formula. As we hypothesised on the basis of results from previous studies [6, 23], that risk in our group of subjects may be overestimated by risk charts, 16 (76%) out of 21 intermediate-risk subjects could be reclassified into the low-risk and only 5 (24%) had to be reclassified into the high-risk category. Therefore, these results, albeit from a small study, show that most subjects classified as intermediaterisk subjects as defined by FRS are in fact lowrisk subjects.

In the third segment of this study, we estimated correction factors that could be used regionally to adjust the risk values obtained from risk charts through use of our chosen biological marker of risk, CS% >50. In the PROCAM Pocket Guide 2003 based on the MONICA data [24], a correction factor of 0.61 has been proposed for the Swiss Italian population and a correction factor of 0.55 for the Swiss French population in Switzerland [24]. However, no correction factor is available for the Swiss German population in Switzerland, which did not participate in the MONICA project [24]. Our estimated FRS 10-year myocardial infarction risk was 10.6% for the entire cohort of 100 subjects. Our post-test risk estimate after coronary calcium scoring was 7.2%, which yielded a correction factor of 0.68. By using this correction factor, the accuracy of detecting risk with CS% >50 increased from 79 to 82%, and the specificity increased from 85 to 93%, although at the expense of sensitivity (20 versus 47%, all p = ns). The correction factor of 0.68 is comparable to the correction factors derived from MONICA data for the Swiss Italian population (0.61) and the Swiss French population (0.55).

We also applied our calculations to PROCAM risk estimates of 10-year risk for developing a myocardial infarction. In the men in the PROCAM group, the PROCAM algorithm performed with a sensitivity of 67% and a specificity of 87% for detecting CS% >50. Our estimated PROCAM 10-year risk for myocardial infarction (Table 3) decreased from 10.8 to 6.9%, yielding a correction factor of 0.64. By using this correction factor, the accuracy of detecting risk with CS% >50 increased from 84 to 87%, however sensitivity was 0%. By using a correction factor of 0.70 instead of 0.64, however, accuracy increased to 89% and sensitivity to 33% with maintained specificity (97%). Importantly, using a correction factor of 0.7 for the male PROCAM algorithm, we observed the highest positive predictive value (67%), which has implications for the adequacy of intensive primary prevention of myocardial infarction in the setting of primary care.

Our study has some limitations. The first limitation was that the gold standard (ie, event rates of myocardial infarction in a prospective cohort study) was not available for the study. We substituted a surrogate marker of risk for myocardial infarction for this gold standard and used a database to calculate CS% that reduces the effects of a referral bias [19]. The absence of coronary calcium is correlated with a very low event rate for fatal and non-fatal myocardial infarction in middle-age asymptomatic subjects (2 events/1000 subjects/year) [14, 21]. Therefore, it is reasonable to argue that the absence of coronary calcium in intermediate-risk subjects as defined by FRS and PROCAM may help identify the actual low-risk subjects. The risk that is related to coronary calcification may be even lower in low-risk populations, such as the Swiss population, than it is in the USA cohort, from which our CS% >50 surrogate marker was originally obtained [19], although this remains to be proven. If this were shown to be true, this risk relationship would further support reclassification of intermediate-risk subjects into low-risk subjects in low-risk populations. Further, the extrapolated 10-year event rate for fatal and nonfatal myocardial infarction in the USA cohort was 17%, which corresponds to an intermediate risk population [19] and increases the applicability of this risk assessment tool to our Swiss study population. Certainly, risk prediction needs to be further improved. Unfortunately, post-test risk calculations with additional markers of risk, such as the ankle-arm-index [17] or carotid intima-to-media thickness, do not improve the prediction of myocardial infarction events [18].

The second limitation was that we had to adopt data from the EBCT risk prognostication database [19] into MDCT imaging because a similar normal database allowing for calculation of accurate post-test probabilities [20] is not available for either MDCT or EBCT in Europe. When the results from the Rotterdam Study [26] and the Heinz Nixdorf Recall Study [27] are available, these other European data could possibly be substituted for the CS% >50 used in our study. However, the European data will be outcome data and, therefore, inaccurate because medical intervention reduces risk in subjects that have already been identified as having high-risk coronary calcifications. This is true for all event studies that assess risk tools.

In conclusion, using the CS% >50 marker may allow improvement in risk prediction in physician referred intermediate-risk patients, where, according to PROCAM, about one-third of coronary events occur within 10 years [9]. Despite the fact that CS% >50 is a very sensitive (“oversensitive”) biological marker of risk, its use in this study pointed to reclassification of 76% of the FRS intermediate-risk subjects in our cohort to a low-risk category for hard coronary events.

Coronary calcifications also may help to identify the patients who are at higher than the expected intermediate risk, eg, by using CS% >75. New guidelines or risk assessment tools such as PROCAM or the EU-SCORE have higher specificity and accuracy, but they entail an unacceptable loss of sensitivity. This might leave up to two-thirds of subjects, who would have needed preventive therapy, undetected even in a low-risk population.

Our CS% >50-derived correction factors show an overestimation of risk in the cohort with the use of FRS and PROCAM. Therefore, such an algorithm may be helpful for calculating correction factors in other areas in Europe for which data from prospective observational studies are lacking.