“Vox Populi” Fractional Flow Reserve (vpFFR)—Leveraging Wisdom of the Crowd for the Assessment of Hemodynamic Severity of Intermediate Coronary Lesions
Abstract
1. Introduction
2. Materials and Methods
2.1. Patient Selection
2.2. FFR Measurements
2.3. vpFFR Derivation
2.4. QCA
2.5. Statistical Analysis
3. Results
3.1. Diagnostic Performance of vpFFR, 2D-, and 3D-QCA
3.2. The Influence of Operator Experience on vpFFR Accuracy
3.3. The Influence of Vessel Type on vpFFR Accuracy
4. Discussion
Limitations
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
| FFR | Fractional Flow Reserve |
| ORBITA | The Objective Randomized Blinded Investigation with optimal medical Therapy of Angioplasty in Stable Angina |
| ROC | Receiver Operating Characteristic |
| QCA | Quantitative Coronary Angiography |
| vpFFR | vox populi Fractional Flow Reserve |
References
- Vrints, C.; Andreotti, F.; Koskinas, K.C.; Rossello, X.; Adamo, M.; Ainslie, J.; Banning, A.P.; Budaj, A.; Buechel, R.R.; Chiariello, G.A.; et al. 2024 ESC Guidelines for the Management of Chronic Coronary Syndromes: Developed by the Task Force for the Management of Chronic Coronary Syndromes of the European Society of Cardiology (ESC) Endorsed by the European Association for Cardio-Thoracic Surgery (EACTS). Eur. Heart J. 2024, 45, 3415–3537. [Google Scholar] [CrossRef]
- Virani, S.S.; Newby, L.K.; Arnold, S.V.; Bittner, V.; Brewer, L.C.; Demeter, S.H.; Dixon, D.L.; Fearon, W.F.; Hess, B.; Johnson, H.M.; et al. 2023 AHA/ACC/ACCP/ASPC/NLA/PCNA Guideline for the Management of Patients With Chronic Coronary Disease. JACC 2023, 82, 833–955. [Google Scholar] [CrossRef]
- Tonino, P.A.L.; Bruyne, B.D.; Pijls, N.H.J.; Siebert, U.; Ikeno, F.; Veer, M.v.; Klauss, V.; Manoharan, G.; Engstrøm, T.; Oldroyd, K.G.; et al. Fractional Flow Reserve versus Angiography for Guiding Percutaneous Coronary Intervention. N. Engl. J. Med. 2009, 360, 213–224. [Google Scholar] [CrossRef] [PubMed]
- Schlundt, C.; Bietau, C.; Klinghammer, L.; Wiedemann, R.; Rittger, H.; Ludwig, J.; Achenbach, S. Comparison of Intracoronary Versus Intravenous Administration of Adenosine for Measurement of Coronary Fractional Flow Reserve. Circ. Cardiovasc. Interv. 2015, 8, e001781. [Google Scholar] [CrossRef] [PubMed]
- Odanovic, N.; Parikh, R.V.; Shah, S.M. Finding the Optimal Strategy for Nonculprit Revascularization in Acute Myocardial Infarction: The Role of Angiography-Derived Physiology. Circ. Cardiovasc. Interv. 2024, 17, e014129. [Google Scholar] [CrossRef] [PubMed]
- Li, W.; Takahashi, T.; Sehatbakhsh, S.; Parikh, M.A.; Garcia-Garcia, H.M.; Fearon, W.F.; Kobayashi, Y. Diagnostic Performances of Nonhyperemic Pressure Ratios and Coronary Angiography-Based Fractional Flow Reserve against Conventional Wire-Based Fractional Flow Reserve. Coron. Artery Dis. 2024, 35, 83–91. [Google Scholar] [CrossRef]
- Westra, J.; Tu, S.; Campo, G.; Qiao, S.; Matsuo, H.; Qu, X.; Koltowski, L.; Chang, Y.; Liu, T.; Yang, J.; et al. Diagnostic Performance of Quantitative Flow Ratio in Prospectively Enrolled Patients: An Individual Patient-Data Meta-Analysis. Catheter. Cardiovasc. Interv. 2019, 94, 693–701. [Google Scholar] [CrossRef]
- Tu, S.; Ding, D.; Chang, Y.; Li, C.; Wijns, W.; Xu, B. Diagnostic Accuracy of Quantitative Flow Ratio for Assessment of Coronary Stenosis Significance from a Single Angiographic View: A Novel Method Based on Bifurcation Fractal Law. Catheter. Cardiovasc. Interv. 2021, 97, 1040–1047. [Google Scholar] [CrossRef]
- Oshima, A.; Ninomiya, K.; Sekiguchi, M.; Yoshiura, D.; Gonda, Y.; Horiuchi, Y.; Taniwaki, M.; Yuzawa, H.; Komiyama, K.; Tanaka, J.; et al. Diagnostic Performance of Angiography-Based Vessel-Fractional Flow Reserve Compared with Various Wire-Based Physiological Assessments. Catheter. Cardiovasc. Interv. 2025, 106, 207–217. [Google Scholar] [CrossRef]
- Yang, G.; Li, L.; Peng, X.; Tang, G.; Zheng, N.; Zhao, Y.; Li, H.; Zhang, H.; Sun, F.; Ai, H. Accuracy and Reproducibility of Coronary Angiography-Derived Fractional Flow Reserve in the Assessment of Coronary Lesion Severity. Int. J. Gen. Med. 2023, 16, 3805–3814. [Google Scholar] [CrossRef]
- Ninomiya, K.; Serruys, P.W.; Kotoku, N.; Zhou, J.; Kageyama, S.; Masuda, S.; Revaiah, P.C.; Wang, B.; He, X.; Tsai, T.-Y.; et al. Anonymous Comparison of Various Angiography-Derived Fractional Flow Reserve Software With Pressure-Derived Physiological Assessment. JACC Cardiovasc. Interv. 2023, 16, 1778–1790. [Google Scholar] [CrossRef]
- Tang, X.; Dai, N.; Zhang, B.; Cai, H.; Huo, Y.; Yang, M.; Jiang, Y.; Duan, S.; Shen, J.; Zhu, M.; et al. Comparison of 2D-QCA, 3D-QCA and Coronary Angiography Derived FFR in Predicting Myocardial Ischemia Assessed by CZT-SPECT MPI. J. Nucl. Cardiol. 2023, 30, 1973–1982. [Google Scholar] [CrossRef] [PubMed]
- Ding, D.; Yang, J.; Westra, J.; Chen, Y.; Chang, Y.; Sejr-Hansen, M.; Zhang, S.; Christiansen, E.H.; Holm, N.R.; Xu, B.; et al. Accuracy of 3-Dimensional and 2-Dimensional Quantitative Coronary Angiography for Predicting Physiological Significance of Coronary Stenosis: A FAVOR II Substudy. Cardiovasc. Diagn. Ther. 2019, 9, 481–491. [Google Scholar] [CrossRef] [PubMed]
- Vox Populi|Nature. Available online: https://www.nature.com/articles/075450a0 (accessed on 17 October 2024).
- Revisiting Francis Galton’s Forecasting Competition. Available online: https://projecteuclid.org/journals/statistical-science/volume-29/issue-3/Revisiting-Francis-Galtons-Forecasting-Competition/10.1214/14-STS468.full (accessed on 17 October 2024).
- Hasan, E.; Duhaime, E.; Trueblood, J.S. Boosting Wisdom of the Crowd for Medical Image Annotation Using Training Performance and Task Features. Cogn. Res. Princ. Implic. 2024, 9, 31. [Google Scholar] [CrossRef] [PubMed]
- Dissanayake, I.; Nerur, S.; Singh, R.; Lee, Y. Medical Crowdsourcing: Harnessing the “Wisdom of the Crowd” to Solve Medical Mysteries. J. Assoc. Inf. Syst. 2019, 20, 1589–1610. [Google Scholar] [CrossRef]
- Krockow, E.M.; Kurvers, R.H.J.M.; Herzog, S.M.; Kämmer, J.E.; Hamilton, R.A.; Thilly, N.; Macheda, G.; Pulcini, C. Harnessing the Wisdom of Crowds Can Improve Guideline Compliance of Antibiotic Prescribers and Support Antimicrobial Stewardship. Sci. Rep. 2020, 10, 18782. [Google Scholar] [CrossRef]
- Kattan, M.W.; O’Rourke, C.; Yu, C.; Chagin, K. The Wisdom of Crowds of Doctors: Their Average Predictions Outperform Their Individual Ones. Med. Decis. Mak. 2016, 36, 536–540. [Google Scholar] [CrossRef]
- Foley, M.; Rajkumar, C.A.; Ahmed-Jushuf, F.; Nour, D.; Fung, C.H.; Seligman, H.; Pathimagaraj, R.H.; Petraco, R.; Sen, S.; Nijjer, S.; et al. The Ability of Contemporary Cardiologists to Judge the Ischemic Impact of a Coronary Lesion Visually. Cardiovasc. Revasc. Med. 2024, 59, 60–66. [Google Scholar] [CrossRef]
- Nobre Menezes, M.; Francisco, A.R.G.; Carrilho Ferreira, P.; Jorge, C.; Torres, D.; Cardoso, P.; Duarte, J.A.; Marques da Costa, J.; Infante de Oliveira, E.; Pinto, F.J.; et al. Comparative Analysis of Fractional Flow Reserve and Instantaneous Wave-Free Ratio: Results of a Five-Year Registry. Rev. Port. Cardiol. 2018, 37, 511–520. [Google Scholar] [CrossRef]
- Kobayashi, Y.; Collet, C.; Achenbach, S.; Engstrøm, T.; Assali, A.; Shlofmitz, R.; Fournier, S.; Kirtane, A.J.; Ali, Z.A.; Kornowski, R.; et al. Diagnostic Performance of Angiography-Based Fractional Flow Reserve by Patient and Lesion Characteristics. EuroIntervention 2021, 17, e294–e300. [Google Scholar] [CrossRef]
- Westra, J.; Sejr-Hansen, M.; Koltowski, L.; Mejía-Rentería, H.; Tu, S.; Kochman, J.; Zhang, Y.; Liu, T.; Campo, G.; Hjort, J.; et al. Reproducibility of Quantitative Flow Ratio: The QREP Study. EuroIntervention 2022, 17, 1252. [Google Scholar] [CrossRef]





| Total (n = 116) | |
|---|---|
| Age | 65.3 (±8.5) |
| Sex | |
| Female | 32 (27.6%) |
| Male | 84 (72.4%) |
| BMI (average) | 27.5 (±4.3) |
| Hypertension | 115 (99.1%) |
| Hyperlipidemia | 111 (95.7%) |
| Diabetes mellitus | 49 (42.2%) |
| Smoking status | |
| Never smoker | 47 (40.5%) |
| Former smoker | 34 (29.3%) |
| Current smoker | 35 (30.2%) |
| Family history of CAD | 54 (46.6%) |
| Prior revascularization of CAD | |
| Prior PCI | 40 (34.5%) |
| Prior CABG | 1 (0.9%) |
| PAD | 16 (13.8%) |
| CVA | 13 (11.1%) |
| CKD | 12 (10.3%) |
| HFrEF (EF < 40%) | 20 (17.2%) |
| Symptoms | |
| None | 43 (37.1%) |
| CCS I | 22 (19.0%) |
| CCS II | 23 (19.8%) |
| CCS III | 13 (11.2%) |
| CCS IV | 4 (3.4%) |
| Dyspnea | 11 (9.5%) |
| Vessels | |
| LAD or branches | 93 (59.6%) |
| LCX or branches | 26 (16.7%) |
| RCA or branches | 35 (22.4%) |
| RI or branches | 2 (1.3%) |
| FFR | 0.83 ± 0.08 |
| ≤0.80 | 59 (37.8%) |
| >0.80 | 97 (62.2%) |
| Diameter stenosis by 2D-QCA | 46% ± 9% |
| ≥0.50 | 54 (34.6%) |
| <0.50 | 102 (65.4%) |
| Operator ID | Operator Experience/ Level of Training | Number of Guesses | Mean FFR in the Subgroup | Standard Deviation of Mean FFR | Mean Guess | Standard Deviation of Mean Guess | Pearson’s Correlation Coefficient | p Value (One Tailed) |
|---|---|---|---|---|---|---|---|---|
| 1 | Fellow | 114 | 0.83 | 0.09 | 0.82 | 0.05 | 0.36 | <0.001 |
| 2 | Fellow | 21 | 0.81 | 0.09 | 0.82 | 0.06 | 0.23 | 0.153 |
| 3 | Fellow | 26 | 0.82 | 0.08 | 0.80 | 0.04 | 0.07 | 0.377 |
| 4 | Fellow | 77 | 0.83 | 0.08 | 0.83 | 0.06 | 0.35 | 0.001 |
| 5 | Fellow | 22 | 0.83 | 0.09 | 0.83 | 0.05 | 0.41 | 0.029 |
| 6 | Fellow | 12 | 0.82 | 0.10 | 0.82 | 0.05 | 0.26 | 0.210 |
| 7 | Fellow | 6 | 0.81 | 0.11 | 0.81 | 0.06 | 0.93 | 0.004 |
| 8 | Early Career IC | 21 | 0.83 | 0.08 | 0.80 | 0.07 | 0.61 | 0.002 |
| 9 | Fellow | 29 | 0.82 | 0.07 | 0.82 | 0.05 | 0.18 | 0.180 |
| 10 | Fellow | 63 | 0.83 | 0.09 | 0.81 | 0.06 | 0.43 | <0.001 |
| 11 | Early Career IC | 124 | 0.83 | 0.08 | 0.84 | 0.06 | 0.60 | <0.001 |
| 12 | Experienced IC | 6 | 0.83 | 0.09 | 0.82 | 0.10 | 0.90 | 0.007 |
| 13 | Experienced IC | 76 | 0.84 | 0.08 | 0.83 | 0.06 | 0.31 | 0.003 |
| 14 | Experienced IC | 14 | 0.82 | 0.09 | 0.82 | 0.05 | 0.66 | 0.005 |
| 15 | Experienced IC | 4 | 0.87 | 0.11 | 0.83 | 0.10 | 0.88 | 0.058 |
| 16 | Experienced IC | 15 | 0.82 | 0.08 | 0.81 | 0.04 | 0.33 | 0.116 |
| 17 | Experienced IC | 41 | 0.82 | 0.09 | 0.83 | 0.07 | 0.57 | <0.001 |
| 18 | Experienced IC | 53 | 0.82 | 0.09 | 0.82 | 0.05 | 0.53 | <0.001 |
| 19 | Experienced IC | 1 | 0.71 | - | 0.80 | - | - | - |
| 20 | Early Career IC | 55 | 0.84 | 0.08 | 0.84 | 0.07 | 0.68 | <0.001 |
| vpFFR (n = 156) | 2D-QCA (n = 156) | 3D-QCA (n = 132) | |
|---|---|---|---|
| Sensitivity | 55.9% | 49.2% | 28.0% |
| Specificity | 83.5% | 74.2% | 64.6% |
| PPV | 67.3% | 53.7% | 32.5% |
| NPV | 75.7% | 70.6% | 59.5% |
| Diagnostic accuracy | 73.0% | 64.7% | 50.7% |
| AUC | 0.777 | 0.625 | 0.454 |
| Pearson correlation coefficient | 0.560 | −0.262 | −0.010 |
| p value for Pearson correlation coefficient | <0.001 | 0.001 | 0.915 |
| Vessel | Number of Vessels | Pearson’s Correlation Coefficient | p Value for Pearson’s Correlation Coefficient (One-Tailed) | Binary Diagnostic Accuracy | Area Under ROC Curve |
|---|---|---|---|---|---|
| Fellow | 370 | 0.34 | <0.001 | 63.5% | 0.67 |
| Early career IC | 200 | 0.61 * | <0.001 | 75.0% | 0.84 * ¥ |
| Experienced IC | 210 | 0.48 | <0.001 | 65.7% | 0.72 |
| 5-Operator Agreement (n = 68) | 4-Operator Agreement (n = 43) | 3-Operator Agreement (n = 45) | |
|---|---|---|---|
| Sensitivity | 61.9% | 52.6% | 31.6% |
| Specificity | 93.6% | 70.8% | 69.2% |
| PPV | 81.3% | 58.8% | 42.9% |
| NPV | 84.6% | 65.4% | 58.1% |
| Diagnostic accuracy | 83.8% | 62.8% | 53.3% |
| Vessel | Number of Vessels | Pearson’s Correlation Coefficient | p Value for Pearson’s Correlation Coefficient (One-Tailed) | Binary Diagnostic Accuracy | Area Under ROC Curve |
|---|---|---|---|---|---|
| LAD | 93 | 0.46 | <0.001 | 66.7% | 0.71 |
| LCx | 26 | 0.22 | 0.140 | 84.6% | 0.69 |
| RCA | 35 | 0.48 | 0.002 | 80.0% | 0.75 |
| RI | 2 | - | - | 100% | - |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
Share and Cite
Odanovic, N.; Misevic, V.; Obradovic, A.; Bojic, V.; Krupnikovic, K.; Mandic, A.; Furtula, M.; Borzanovic, D.; Lazarevic, N.; Zivkovic, S.; et al. “Vox Populi” Fractional Flow Reserve (vpFFR)—Leveraging Wisdom of the Crowd for the Assessment of Hemodynamic Severity of Intermediate Coronary Lesions. Diagnostics 2026, 16, 269. https://doi.org/10.3390/diagnostics16020269
Odanovic N, Misevic V, Obradovic A, Bojic V, Krupnikovic K, Mandic A, Furtula M, Borzanovic D, Lazarevic N, Zivkovic S, et al. “Vox Populi” Fractional Flow Reserve (vpFFR)—Leveraging Wisdom of the Crowd for the Assessment of Hemodynamic Severity of Intermediate Coronary Lesions. Diagnostics. 2026; 16(2):269. https://doi.org/10.3390/diagnostics16020269
Chicago/Turabian StyleOdanovic, Natalija, Vojko Misevic, Aleksa Obradovic, Vanja Bojic, Kosta Krupnikovic, Aleksandar Mandic, Matija Furtula, Dusan Borzanovic, Nikola Lazarevic, Stefan Zivkovic, and et al. 2026. "“Vox Populi” Fractional Flow Reserve (vpFFR)—Leveraging Wisdom of the Crowd for the Assessment of Hemodynamic Severity of Intermediate Coronary Lesions" Diagnostics 16, no. 2: 269. https://doi.org/10.3390/diagnostics16020269
APA StyleOdanovic, N., Misevic, V., Obradovic, A., Bojic, V., Krupnikovic, K., Mandic, A., Furtula, M., Borzanovic, D., Lazarevic, N., Zivkovic, S., Ilic, I., Dobric, M., & Shah, S. M. (2026). “Vox Populi” Fractional Flow Reserve (vpFFR)—Leveraging Wisdom of the Crowd for the Assessment of Hemodynamic Severity of Intermediate Coronary Lesions. Diagnostics, 16(2), 269. https://doi.org/10.3390/diagnostics16020269

