Comparison of Machine Learning Computed Tomography-Based Fractional Flow Reserve and Coronary CT Angiography-Derived Plaque Characteristics with Invasive Resting Full-Cycle Ratio

Background: The aim is to compare the machine learning-based coronary-computed tomography fractional flow reserve (CT-FFRML) and coronary-computed tomographic morphological plaque characteristics with the resting full-cycle ratio (RFRTM) as a novel invasive resting pressure-wire index for detecting hemodynamically significant coronary artery stenosis. Methods: In our single center study, patients with coronary artery disease (CAD) who had a clinically indicated coronary computed tomography angiography (cCTA) and subsequent invasive coronary angiography (ICA) with pressure wire-measurement were included. On-site prototype CT-FFRML software and on-site CT-plaque software were used to calculate the hemodynamic relevance of coronary stenosis. Results: We enrolled 33 patients (70% male, mean age 68 ± 12 years). On a per-lesion basis, the area under the receiver operating characteristic curve (AUC) of CT-FFRML (0.90) was higher than the AUCs of the morphological plaque characteristics length/minimal luminal diameter4 (LL/MLD4; 0.80), minimal luminal diameter (MLD; 0.77), remodeling index (RI; 0.76), degree of luminal diameter stenosis (0.75), and minimal luminal area (MLA; 0.75). Conclusion: CT-FFRML and morphological plaque characteristics show a significant correlation to detected hemodynamically significant coronary stenosis. Whole CT-FFRML had the best discriminatory power, using RFRTM as the reference standard.


Introduction
Assessment of coronary artery stenosis with coronary computed tomographic angiography (cCTA) and additional computed tomographic morphological plaque characteristics derived from cCTA datasets leads to a more detailed evaluation of suspicious stenosis [1]. Two studies published recently showed that the joined assessment of some computed tomographic morphological plaque characteristics and cCTA could improve the detection of hemodynamically relevant stenosis [2,3]. In addition, the functional relevance of a lesion may be assessed by cCTA-based fractional flow reserve (CT-FFR) [4]. Several studies have shown a significant correlation between non-invasive CT-FFR and invasive fractional flow reserve (FFR) [2,5,6]. Using an on-site prototype with a machine-learning algorithm (CT-FFR ML ) to reduce computation time was the consequent next step in technical development [7][8][9][10]. CT-FFR ML was compared in previously published studies to the invasive gold standards invasive FFR [8] and instantaneous wave-free ratio (iwFR) [11] and showed promising results.
Invasive diagnostic tools currently used to assess the hemodynamic relevance of coronary artery stenosis during invasive coronary angiography (ICA) are invasive FFR and iwFR [12,13]. These two methods are both recommended by the 2018 European Society of Cardiology (ESC) guidelines on myocardial revascularization as a class IA recommendation [12,14,15].
Resting full-cycle ratio (RFR TM ) is a recently developed invasive parameter to determine the hemodynamic relevance of coronary artery stenosis. Compared to invasive FFR, it is not necessary to induce hyperemia for RFR TM , resulting in improved patient comfort. The advantage of this novel resting index approach may consist of detecting lower values during systole, while iwFR measures values during diastole. RFR TM showed an excellent correlation to the gold standard iwFR in recent studies [16,17].
The aim of our study is to compare the machine learning-based coronary-computed tomography fractional flow reserve (CT-FFR ML ) and coronary-computed tomographic morphological plaque characteristics calculated based on coronary-computed tomography angiography (cCTA) with the resting full-Cycle ratio (RFR TM ).

Study Design and Patient Population
The local Institutional Review Board approved the study protocol (No. 2015-583N-MA). Written informed consent was obtained from all patients. Patients with suspected coronary artery disease (CAD) underwent a clinically indicated cCTA and ICA with pressure wire measurement. The CAD consortium clinical score was applied to determine the pre-test probability for CAD and estimate the probability based on age, sex, symptoms, and cardiovascular risk factors [18]. The acquired cCTA datasets were used to calculate CT-FFR ML and computed tomographic morphological plaque characteristics. Four patients had to be excluded due to inadequate image quality. Patients were included if they had at least one coronary artery with stenosis of undetermined hemodynamic relevance, defined as a 40% to 70% diameter stenosis by visual assessment [15]. These lesions were further assessed by pressure wire measurements (Verrata TM pressure wire, Volcano Corporation, Koninklijke Philips N.V. Amsterdam, The Netherlands) [14]. After the procedures, the raw data of the iwFR measurement were extracted and analyzed with the RFR TM algorithm by an external core lab (Abbott GmbH & Co. KG; Wiesbaden, Germany). Exclusion criteria for cCTA were electrocardiographic signs of acute myocardial ischemia, unstable angina with elevated serum cardiac biomarkers, patients with renal insufficiency, and known contrast agent allergies. The exclusion criteria for CT-FFR ML calculation and computed tomographic plaque characteristics analysis included left main or complex bifurcation stenosis, aneurysms, severe diffuse disease, chronic total occlusion, previous percutaneous coronary stent implantation or coronary artery bypass graft (CABG), or inadequate image quality. Data for baseline characteristics were obtained from electronic medical records.

Acquisition and Analysis of cCTA Datasets
A third-generation dual-source CT (Siemens Somatom FORCE, Siemens Healthineers, Forchheim, Germany) was used for imaging. A specified regime of medication consisting of sublingual nitroglycerin (0.8 mg) and intravenous beta-blockers were given prior to the scan if deemed necessary by a radiologist. Initially, 80 mL iodinated contrast material (Iomeron 400; Bracco Imaging S.p.A., Milan, Italy) was administered using a power injector (Stellant D; Medrad, Warrendale, PA, USA) at a flow rate of 5 mL/s followed by a 50 mL saline chaser. Coronary artery stenosis was analyzed and quantified using on-site software (Coronary Plaque Analysis 2.0. syngo. via FRONTIER, Siemens Healthineers). The grading was done in accordance with the guidelines of the Society of Cardiovascular Computed Tomography [19].

Analysis of Machine Learning Computed Tomography-Based Fractional Flow Reserve and Computed Tomographic Morphological Plaque Characteristics
The Leiden-risk score that combines different plaque characteristics to optimize risk stratification was calculated to characterize the complexity and severity of CAD [20]. To quantify coronary calcification, the Agatston score was calculated using a software application, according to the Agatston scoring convention (CaScore, Siemens Healthineers) [21]. The cCTA datasets were used for morphological plaque analysis calculated by on-site software (Coronary Plaque Analysis 2.0.). The target lesion was defined as an area with atherosclerotic changes between non-affected proximal and distal sections without any atherosclerotic changes. Discrepancies in plaque density measured in Hounsfield units (HU) were used to analyze the plaque morphology [2]. The software automatically calculated the parameters lesion length (LL), vessel volume (VV), and total plaque volume (TPV) [3]. LL was defined as the dimension of the plaque between areas free of atherosclerotic plaque. Minimal luminal area (MLA) was measured automatically by the on-site prototype software as the lowest value for the luminal area within the lesion. Corrected coronary opacification (CCO) was defined as the difference between the lowest mean HU attenuation at the proximal extent of stenosis and the lowest mean HU attenuation in the coronary artery distal to the lesion and was calculated manually [22]. The minimal luminal diameter (MLD) was measured, although manually, as the area with the narrowest luminal diameter in the lesion [2]. The remodeling index (RI) was calculated as the ratio of the smallest cross-sectional area of the atherosclerotic lesion over the respective proximal luminal area [23]. On-site CT-FFR prototype software based on a machine-learning algorithm (Siemens cFFR, version 3.1; Siemens Healthineers, currently not commercially available) was used and installed on a regular workstation (Syngo VE36A; Siemens Healthineers) for the CT-FFR ML calculation. An experienced cardiovascular radiologist checked the semi-automatically calculated vessel-specific centerlines for the accurate recognition of suspected coronary artery stenosis. The exact percentage of manual centerline corrections was initially not documented. However, the algorithm has captured excellently in most cases, and the rate of manual corrections was fewer than 5%. The on-site prototype software calculated blood flow and showed the hyperemic state in the coronary vessels based on patient-specific physiological conditions and flow dynamic models. The technical specification of the used CT-FFR ML algorithm is described in detail in a previously published study [8]. After on-site calculation, the CT-FFR ML software created a patient-specific anatomic color-coded 3-dimensional mesh of the coronary artery tree and aortic root. A CT-FFR ML cut off value of ≤ 0.80 was used for the detection of hemodynamically relevant coronary artery stenosis [24].

Invasive Coronary Angiography and Resting Full Cycle Ratio Measurement
A coronary-pressure guide wire (Verrata TM pressure wire, Volcano Corporation) was introduced in the respective coronary artery [12,15]. The pressure sensor was placed distal to the stenosis. The optimal diastolic interval with minimized and constant microvascular resistance for pressure measurement was calculated by dedicated software (Volcano Corporation). IwFR values ≤ 0.89 were considered as diagnostic relevant for lesion-specific ischemia [12,15,25]. The raw data of the iwFR measurement was extracted from the console and an external core lab (Abbott GmbH & Co. KG) retrospectively calculated the RFR TM with the new algorithm, using a hyperemia-free resting measurement for the coronary pressure at the lowest point of resting diastolic-pressure-to-aortic-pressure ratio. The maximal relative pressure difference in the whole cardiac cycle (either in systole or diastole) in five consecutive cardiac cycles was measured. The cut-off value for hemodynamic relevance for RFR TM was equal to the value for iwFR ≤ 0.89 [16,17].

Statistical Analysis
SAS (version 9.4, SAS Institute Inc., Cary, NC, USA) was used for all analyses. For quantitative variables, mean values and standard deviations were calculated. Categorical variables are presented as percentages, whereas continuous variables are presented as either mean ± standard deviation (SD) or median with interquartile range (IQR) and were analyzed with the independent sample t-test. For qualitative factors, absolute and relative frequencies are presented. In order to quantify the degree to which two measurement methods agree, interclass correlation (ICC) was estimated using a two-way ANOVA with one of the methods being equivalent to 100% consistency. Additionally, Kappa coefficients were calculated to assess the degree of agreement for binary factors. Sensitivity, specificity, positive predictive value, and negative predictive value were evaluated on a per-lesion and per-patient level for CT-FFR ML and coronary-computed tomographic morphological plaque characteristics calculated based cCTA, using RFR TM (≤0.89) as the reference standard to detect lesion-specific ischemia. The area under the receiver operating characteristics curve (AUC) was determined and compared for cCTA, CT-FFR ML , and RFR TM as a metric of overall diagnostic performance. For computed tomographic morphological plaque characteristics that showed statistically significant correlation with the RFR TM values, sensitivity, specificity, positive predictive value, and negative predictive value were calculated. A p-value of ≤0.05 was considered statistically significant. The AUC was determined for the significant CT-FFR ML and morphological plaque characteristics as a metric of overall diagnostic performance.

Results
We enrolled 33 patients (70% male, mean age 68 ± 12 years) with acquired cCTA datasets, CT-FFR ML and morphological plaque characteristics calculations, and RFR TM measurements. On average, we calculated a pretest probability of 57% ± 19%, which was calculated with the CAD consortium clinical score [18]. Baseline characteristics of patients are presented in Table 1. Unless otherwise specified, data are numbers of patients with percentages in parentheses. Data are mean ± standard deviation (SD) or interquartile range. + Pretest probability calculated with the CAD consortium clinical score [18]. * Defined as blood pressure > 140 mmHg systolic, >90 mmHg diastolic, or use of an antihypertensive medication. † Defined as a total cholesterol level of >200 mg/dL or use of antilipidemic medication; cCTA = coronary computed tomography angiography; CT-FFR ML = fractional flow reserve derived from coronary computed tomography angiography based on machine learning algorithm; RFR TM = resting full-cycle ratio.
Forty-four vessel lesions in 33 patients were analyzed. Fourteen (32%) were identified as hemodynamically relevant stenoses by RFR TM (RFR TM ≤ 0.89), whereas CT-FFR ML classified only 13 (30%) as hemodynamically significant coronary artery stenoses (CT-FFR ML ≤ 0.80). The mean calculation time for the CT-FFR ML was 11 ± 2 min, and the mean calculation time for the analysis of the morphological plaque markers was 15 ± 5 min. An example case is presented in Figure 1. The cCTA-derived quantitative markers, LL/MLD 4 , MLD, the degree of luminal diameter stenosis, MLA, CT-FFR ML , and RI, showed statistically significant differences between obstructive and non-obstructive lesions compared to the reference standard RFR TM (Table 2).  Lesion length (LL), total plaque volume (TPV), vessel volume (VV), and corrected coronary opacification (CCO) did not achieve statistical significance and were unable to detect lesion-specific ischemia ( Table 2). We further assessed the sensitivity, specificity, positive predictive values and negative predictive values and accuracy for CT-FFR ML and morphological plaque characteristics with statistical significance as well as the area under the receiver operating characteristic curve (AUC).
On a per-lesion and per-patient level, for CT-FFR ML , the parameter with the highest discriminatory power, the sensitivity, specificity, positive predictive value, and a negative predictive value are demonstrated in Table 3.
When compared to RFR TM , the diagnostic accuracy for detecting hemodynamically significant coronary artery stenoses of CT-FFR ML and LL/MLD 4 on a per-lesion and a per-patient level was 93% (95% CI: 81-98%) and 91% (95% CI: 76-98%) and 75% (95% CI: 60-87%) and 73% (95% CI: 55-87%), respectively. For MLD, the degrees of accuracy of luminal diameter stenosis, MLA, RI, cCTA (>50%), and cCTA (>70%) are also listed in Table 3. Table 3. Diagnostic performance of fractional flow reserve derived from coronary-computed tomography angiography based on machine learning algorithm and coronary-computed tomographic plaque characteristic on a per-lesion and per-patient level using RFR TM as the reference standard.  The ROC curve analysis on a per lesion level for CT-FFR ML and coronary-computed tomographic morphological plaque characteristics resulted in an AUC of 0.90 for CT-FFR ML , 0.80 for LL/MLD 4 , 0.77 for MLD, 0.75 for the degree of luminal diameter stenosis, 0.77 for MLA, 0.76 for RI, 0.69 for cCTA (<50%), and likewise, 0.69 for cCTA (<70%). The AUCs on a per-patient level are outlined in Table 3.

Discussion
In this study, the aim is to compare CT-FFR ML and coronary computed tomographic morphological plaque characteristics calculated with the RFR TM as a novel invasive resting pressure-derived index for detecting hemodynamically significant coronary artery stenosis.
Based on the comparatively moderate specificity of cCTA, a tendency may exist to overestimate the severity of coronary artery stenosis by cCTA examination [26][27][28]. In order to compensate for this disadvantage of cCTA, which is caused by its low specificity for the detection of hemodynamically relevant stenosis, techniques have been developed in recent years to improve the quantification of suspicious coronary artery stenosis.
The coronary computed tomographic morphological plaque characteristics calculated based on cCTA were able to prove in various studies and the information gained from plaque parameters led to an improvement in the diagnostic accuracy of the cCTA for the detection of hemodynamically relevant stenosis [2,3,22]. Our results demonstrate that cCTA morphological plaque characteristics, including LL/MLD 4 , MLD, the degree of luminal diameter stenosis, MLA, and RI, have discriminatory power to differentiate between hemodynamically relevant and non-relevant coronary artery stenoses.
In line with previous studies, LL/MLD 4 , MLD, the degree of luminal diameter stenosis, and MLA were also identified as having significant discriminatory power in our study [3,22]. The present analysis supports findings from Tesche et al. [3], Wang et al. [22], and Baumann et al. [29], that LL, TPV, and VV do not show statistically relevant differences between flow obstructing and non-flow obstructing coronary artery stenoses. However, our findings concerning the discriminatory power of RI and CCO are discordant with the results of the studies mentioned above regarding cCTA plaque characteristics. We could demonstrate that the RI belongs to the morphological plaque characteristics with significant discriminatory power. Previous studies have described the opposite results when examining the discriminatory power of RI [3,22]. Even when using CCO, our results differed from the studies mentioned. We could further demonstrate that CCO did not show statistically significant differences between relevant and non-relevant coronary artery lesions. In contrast to these studies mentioned above, with the exception of one study, which used the invasive FFR as the reference standard, we utilized the RFR TM as a novel invasive resting index and to detect hemodynamically significant coronary artery stenosis. We could demonstrate that LL/MLD 4 , an anatomic plaque marker, had the best diagnostic power to determine the hemodynamic relevance of coronary lesions on a per-lesion level (AUC: 0.80; 95% CI: 0.64-0.96%). Our results are in accordance with the findings of a previous study, including Baumann et al. [29] (AUC: 0.84) and Wang et al. [22] (AUC: 0.90) that also demonstrated the utility of LL/MLD 4 . We are also able to support the findings in terms of MLD (AUC: 0.81 and AUC: 0.80). In addition to our previously published results [29], it can be assumed that some plaque markers derived from regular anatomical cCTA may assist in the detection of hemodynamically relevant coronary stenosis. Specifically, LL/MLD 4 , a morphological plaque marker, with its high specificity and accuracy, may resemble a strong predictor.
In addition to morphological plaque characteristics calculated by cCTA, suspicious coronary lesions may be further assessed by functional cCTA predictors. As an example, CT-FFR is gaining importance for non-invasive assessment of suspicious coronary artery stenosis. Previous trials, including DISCOVER-FLOW, DeFACTO, and NXT, have demonstrated a good correlation between CT-FFR and invasive FFR [24,30,31]. These results led to the approval of the CT-FFR off-site algorithm by the US Food and Drug Administration (FDA) in 2015.
Because of the time-consuming calculation time by the off-site CT-FFR algorithm, an on-site CT-FFR ML algorithm was developed [7]. This on-site CT-FFR ML algorithm shortened calculation time and has been successfully validated since [10]. In our study, we utilized this on-site CT-FFR ML algorithm, which is currently not commercially available. In addition, both off-site and on-site CT-FFR algorithms were compared to invasive gold standard FFR in the above-mentioned studies. We could demonstrate similar results in terms of diagnostic accuracy (accuracy on a per-lesion level: 93%, 95% CI: 81-98%) and a high diagnostic performance for the CT-FFR ML (AUC: 0.90, 95% CI: 0.75-1.00%) using the RFR TM as a novel invasive resting pressure-derived index for detecting hemodynamically significant coronary artery stenosis.
Instead of using the invasive gold standard FFR, the novel resting pressure-derived index RFR TM was used as the reference standard to classify suspicious coronary artery stenosis. Initial studies have demonstrated that both CT-FFR ML and morphological plaque characteristics compare well to iwFR as the reference standard for detecting hemodynamically significant coronary artery stenosis regarding their diagnostic accuracy [11,29,32]. Thus, we used the rest index RFR TM as the reference standard, as RFR TM demonstrated an excellent correlation to iwFR [16]. Therefore, this is the first study that compares CT-FFR ML and morphological plaque characteristics based on cCTA with the RFR TM as a novel invasive resting pressure-derived index for detecting hemodynamically significant coronary artery stenosis.
Our findings are to be evaluated and considered in light of the following limitations. First, it should be noted that our monocentric study cohort of a total number of 33 enrolled patients with 44 lesions is relatively small. Second, we did not perform any comparisons to conventional invasive FFR as the invasive gold standard.
Third, all patients had suspected but chronic coronary syndrome with an indication for cCTA, which is just a subgroup of patients who could possibly have an advantage of improved diagnostic tools. We treated patients according to existing guidelines, stating that invasive pressure wire measuring is only indicated in patients lacking evidence of ischemia. Finally, there are no follow up data regarding any adverse events or survival rates stratified by diagnostic parameter or treatment.

Conclusions
In contrast to prior studies using invasive FFR or iwFR as the reference standard, this is the first investigation to our knowledge to compare CT-FFR ML and morphological plaque characteristics against the invasive reference standard RFR TM as a novel resting index without the use of intra-arterial adenosine administration. On-site CT-FFR ML and computed tomography morphological plaque characteristics (LL/MLD 4 , RI, MLD, MLA, degree of luminal diameter stenosis) showed a significant correlation to detected hemodynamically significant coronary artery stenosis. Whole CT-FFR ML had the best discriminatory power using RFR TM as the reference standard. CCO, LL, TPV, and VV failed to detect hemodynamically relevant coronary artery stenosis.  Acknowledgments: Supported by Siemens Healthineers for providing CT-FFR ML software and morphological plaque software for research purposes, which is currently not commercially available. We acknowledge financial support by the Baden-Württemberg Ministry of Science, Research and the Arts and by Ruprecht-Karls-Universität Heidelberg.
Conflicts of Interest: S.B. receives research support from Philips Volcano. C.T. has received research support and speaker honoraria from Siemens. D.L. receives consultant honoraria from Actelion. All other authors declare that they have no financial disclosures. The presented CT-FFR ML software and morphological plaque software is provided by Siemens and is currently not commercially available.