Contribution of an Artificial Intelligence Tool in the Detection of Incidental Pulmonary Embolism on Oncology Assessment Scans
Abstract
:1. Introduction
2. Methods and Materials
2.1. Study Design
2.2. Imaging Feature and Analysis
2.2.1. CT Scan Data
2.2.2. AI System
2.3. Statistical Analysis
3. Results
4. Discussion
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Hasenberg, U.; Paul, T.; Feuersenger, A.; Goyen, M.; Kröger, K. Cancer Patients and Characteristics of Pulmonary Embolism. Eur. J. Radiol. 2009, 69, 478–482. [Google Scholar] [CrossRef] [PubMed]
- Kearon, C. Natural History of Venous Thromboembolism. Circulation 2003, 107, I-22. [Google Scholar] [CrossRef] [PubMed]
- Gimbel, I.A.; Mulder, F.I.; Bosch, F.T.M.; Freund, J.E.; Guman, N.; van Es, N.; Kamphuisen, P.W.; Büller, H.R.; Middeldorp, S. Pulmonary Embolism at Autopsy in Cancer Patients. J. Thromb. Haemost. 2021, 19, 1228–1235. [Google Scholar] [CrossRef]
- Prandoni, P. The Long-Term Clinical Course of Acute Deep Venous Thrombosis. Ann. Intern. Med. 1996, 125, 1. [Google Scholar] [CrossRef] [PubMed]
- Carson, J.L.; Kelley, M.A.; Duff, A.; Weg, J.G.; Fulkerson, W.J.; Palevsky, H.I.; Schwartz, J.S.; Thompson, B.T.; Popovich, J.; Hobbins, T.E.; et al. The Clinical Course of Pulmonary Embolism. N. Engl. J. Med. 1992, 326, 1240–1245. [Google Scholar] [CrossRef]
- Mulder, F.I.; Horvàth-Puhó, E.; van Es, N.; Van Laarhoven, H.W.M.; Pedersen, L.; Moik, F.; Ay, C.; Büller, H.R.; Sørensen, H.T. Venous Thromboembolism in Cancer Patients: A Population-Based Cohort Study. Blood J. Am. Soc. Hematol. 2021, 137, 1959–1969. [Google Scholar] [CrossRef]
- van Es, N.; Bleker, S.M.; Di Nisio, M. Cancer-Associated Unsuspected Pulmonary Embolism. Thromb. Res. 2014, 133, S172–S178. [Google Scholar] [CrossRef]
- Schoepf, U.J.; Costello, P. CT Angiography for Diagnosis of Pulmonary Embolism: State of the Art. Radiology 2004, 230, 329–337. [Google Scholar] [CrossRef]
- Albrecht, M.H.; Bickford, M.W.; Nance, J.W.; Zhang, L.; De Cecco, C.N.; Wichmann, J.L.; Vogl, T.J.; Schoepf, U.J. State-of-the-Art Pulmonary CT Angiography for Acute Pulmonary Embolism. Am. J. Roentgenol. 2017, 208, 495–504. [Google Scholar] [CrossRef]
- Smith, S.B.; Geske, J.B.; Maguire, J.M.; Zane, N.A.; Carter, R.E.; Morgenthaler, T.I. Early Anticoagulation Is Associated with Reduced Mortality for Acute Pulmonary Embolism. Chest 2010, 137, 1382–1390. [Google Scholar] [CrossRef]
- Barragán-Montero, A.; Javaid, U.; Valdés, G.; Nguyen, D.; Desbordes, P.; Macq, B.; Willems, S.; Vandewinckele, L.; Holmström, M.; Löfman, F. Artificial Intelligence and Machine Learning for Medical Imaging: A Technology Review. Phys. Medica 2021, 83, 242–256. [Google Scholar] [CrossRef] [PubMed]
- Patel, S.; Kazerooni, E.A.; Cascade, P.N. Pulmonary Embolism: Optimization of Small Pulmonary Artery Visualization at Multi–Detector Row CT. Radiology 2003, 227, 455–460. [Google Scholar] [CrossRef] [PubMed]
- Hosny, A.; Parmar, C.; Quackenbush, J.; Schwartz, L.H.; Aerts, H.J. Artificial Intelligence in Radiology. Nat. Rev. Cancer 2018, 18, 500–510. [Google Scholar] [CrossRef]
- Cacciamani, G.E.; Sanford, D.I.; Chu, T.N.; Kaneko, M.; De Castro Abreu, A.L.; Duddalwar, V.; Gill, I.S. Is Artificial Intelligence Replacing Our Radiology Stars? Not Yet! Eur. Urol. Open Sci. 2023, 48, 14–16. [Google Scholar] [CrossRef]
- Batra, K.; Xi, Y.; Al-Hreish, K.M.; Kay, F.U.; Browning, T.; Baker, C.; Peshock, R.M. Detection of Incidental Pulmonary Embolism on Conventional Contrast-Enhanced Chest CT: Comparison of an Artificial Intelligence Algorithm and Clinical Reports. Am. J. Roentgenol. 2022, 219, 895–902. [Google Scholar] [CrossRef]
- Topff, L.; Ranschaert, E.R.; Bartels-Rutten, A.; Negoita, A.; Menezes, R.; Beets-Tan, R.G.H.; Visser, J.J. Artificial Intelligence Tool for Detection and Worklist Prioritization Reduces Time to Diagnosis of Incidental Pulmonary Embolism at CT. Radiol. Cardiothorac. Imaging 2023, 5, e220163. [Google Scholar] [CrossRef]
- Langius-Wiffen, E.; de Jong, P.A.; Mohamed Hoesein, F.A.; Dekker, L.; van den Hoven, A.F.; Nijholt, I.M.; Boomsma, M.F.; Veldhuis, W.B. Added Value of an Artificial Intelligence Algorithm in Reducing the Number of Missed Incidental Acute Pulmonary Embolism in Routine Portal Venous Phase Chest CT. Eur. Radiol. 2024, 34, 367–373. [Google Scholar] [CrossRef]
- Wildman-Tobriner, B.; Ngo, L.; Mammarappallil, J.G.; Konkel, B.; Johnson, J.M.; Bashir, M.R. Missed Incidental Pulmonary Embolism: Harnessing Artificial Intelligence to Assess Prevalence and Improve Quality Improvement Opportunities. J. Am. Coll. Radiol. 2021, 18, 992–999. [Google Scholar] [CrossRef] [PubMed]
- Wiklund, P.; Medson, K.; Elf, J. Incidental Pulmonary Embolism in Patients with Cancer: Prevalence, Underdiagnosis and Evaluation of an AI Algorithm for Automatic Detection of Pulmonary Embolism. Eur. Radiol. 2023, 33, 1185–1193. [Google Scholar] [CrossRef]
- Cheikh, A.B.; Gorincour, G.; Nivet, H.; May, J.; Seux, M.; Calame, P.; Thomson, V.; Delabrousse, E.; Crombé, A. How Artificial Intelligence Improves Radiological Interpretation in Suspected Pulmonary Embolism. Eur. Radiol. 2022, 32, 5831–5842. [Google Scholar] [CrossRef]
- O’Neill, T.J.; Xi, Y.; Stehel, E.; Browning, T.; Ng, Y.S.; Baker, C.; Peshock, R.M. Active Reprioritization of the Reading Worklist Using Artificial Intelligence Has a Beneficial Effect on the Turnaround Time for Interpretation of Head CT with Intracranial Hemorrhage. Radiol. Artif. Intell. 2021, 3, e200024. [Google Scholar] [CrossRef] [PubMed]
- Soffer, S.; Klang, E.; Shimon, O.; Barash, Y.; Cahan, N.; Greenspana, H.; Konen, E. Deep Learning for Pulmonary Embolism Detection on Computed Tomography Pulmonary Angiogram: A Systematic Review and Meta-Analysis. Sci. Rep. 2021, 11, 15814. [Google Scholar] [CrossRef] [PubMed]
- Meyer, H.-J.; Wienke, A.; Surov, A. Incidental Pulmonary Embolism in Oncologic Patients-a Systematic Review and Meta-Analysis. Support. Care Cancer 2021, 29, 1293–1302. [Google Scholar] [CrossRef] [PubMed]
- Eng, J.; Krishnan, J.A.; Segal, J.B.; Bolger, D.T.; Tamariz, L.J.; Streiff, M.B.; Jenckes, M.W.; Bass, E.B. Accuracy of CT in the Diagnosis of Pulmonary Embolism: A Systematic Literature Review. Am. J. Roentgenol. 2004, 183, 1819–1827. [Google Scholar] [CrossRef]
- Kligerman, S.J.; Mitchell, J.W.; Sechrist, J.W.; Meeks, A.K.; Galvin, J.R.; White, C.S. Radiologist Performance in the Detection of Pulmonary Embolism: Features That Favor Correct Interpretation and Risk Factors for Errors. J. Thorac. Imaging 2018, 33, 350–357. [Google Scholar] [CrossRef]
- Aidoc’s 6th FDA Clearance for AI Solution. Available online: https://www.aidoc.com/about/news/fda-incidental-pulmonary-embolism/ (accessed on 1 April 2024).
- Avicenna.AI—Avicenna.AI Secures Two FDA Clearances For Stroke Assessment And Opportunistic PE. Available online: https://avicenna.ai/fda-clearances-aspects-and-ipe/ (accessed on 20 September 2024).
- Baumgartner, C.; Klok, F.A.; Carrier, M.; Limacher, A.; Moor, J.; Righini, M.; Beer, J.-H.; Peluso, M.; Rakovic, D.; Huisman, M.V. Clinical Surveillance vs. Anticoagulation for Low-Risk patiEnts with Isolated SubSegmental Pulmonary Embolism: Protocol for a Multicentre Randomised Placebo-Controlled Non-Inferiority Trial (SAFE-SSPE). BMJ Open 2020, 10, e040151. [Google Scholar] [CrossRef]
- Vallée, A.; Quint, R.; Brun, A.L.; Mellot, F.; Grenier, P.A. A Deep Learning-Based Algorithm Improves Radiology Residents’ Diagnoses of Acute Pulmonary Embolism on CT Pulmonary Angiograms. Eur. J. Radiol. 2024, 171, 111324. [Google Scholar] [CrossRef]
- Hanna, T.N.; Steenburg, S.D.; Rosenkrantz, A.B.; Pyatt, R.S.; Duszak, R.; Friedberg, E.B. Emerging Challenges and Opportunities in the Evolution of Teleradiology. Am. J. Roentgenol. 2020, 215, 1411–1416. [Google Scholar] [CrossRef]
N | 3050 Patients |
Age (mean ± SD) | 60.86 ± 12.44 |
Sex | |
Male (%) | 65.20% |
Female (%) | 34.80% |
Characteristics of PEs | |
Prevalence of PEs | 1.3% (39 pts) |
Primary Tumors (33 pts) | |
Digestive | 27.7% (9 pts) |
Thoracix | 24.2% (8 pts) |
Gynecological | 15.2% (5 pts) |
Urological | 9.1% (3 pts) |
Head and Neck | 9.1% (3 pts) |
Breast | 9.1% (3 pts) |
Others | 6 pts (2 pts) |
Time between interpretation and exam | |
Mean ± SD | 8.13 ± 15.48 |
95% CI | [3.21–13.05] |
CONFUSION MATRIX Version 1 | CONFUSION MATRIX Version 2 | ||||||
---|---|---|---|---|---|---|---|
CINA-iPE | CINA-iPE | ||||||
iPE | NOT iPE | ALL | iPE | NOT iPE | ALL | ||
GT | iPE | 39 | 0 | 39 | 36 | 1 | 37 |
NOT iPE | 194 | 2816 | 3010 | 68 | 2942 | 3010 | |
ALL | 233 | 2816 | 3049 | 104 | 2943 | 3047 | |
Sensitivity | 100.0% | 97.3% | |||||
Specifcity | 93.6% | 97.7% | |||||
Accuracy | 93.6% | 97.7% | |||||
PPV | 16.7% | 34.6% | |||||
NPV | 100.0% | 100.0% |
Author | AI Model | Number of Patients | Number of CT Scans | Population | iPE Prevalence (%) | Se | Sp | PPV | NPV | Missed PEs (%) |
---|---|---|---|---|---|---|---|---|---|---|
Batra et al (2022) [15] | AIDOC | 2555 | 3003 | All comers | 1.3% | 82.5% | 99.8% | 86.8% | 99.8% | 10% (4 PEs) |
Topff et al. (2023) [16] | AIDOC | 6447 | 11,736 | Cancer pts | 1.3% | 91.6% | 99.7% | 80.9% | 99.9% | 44.8% (47 PEs) |
Langius-Wiffen et al. (2023) [17] | AIDOC | 3089 | 3089 | All comers | 2.2% | 95.5% | 99.6% | 85.3% | 99.9% | 37.3% (25 PEs) |
Wildman-Tobriner et al. (2021) [18] | AIDOC | 4087 (CAP)–4779 (AP) | 11,913 | All comers | 0.66% | 62%/ 61.2% | 99.97%/ 99.98% | 96.1%/ 96.8% | 99.5%/ 99.7% | 38% (49 PEs) |
Wiklund et al. (2022) [19] | AIDOC | 1004 | 1892 | Cancer pts | 4% | 90.7% | 99.8% | 95.6% | 99.6% | 81.5% (53 PEs) |
Our Study (2024) | CINA-IPE | 3049 | 3049 | Cancer pts | 1.3% | 97.3% | 97.74% | 34.62% | 99.97% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ammari, S.; Camez, A.O.; Ayobi, A.; Quenet, S.; Zemmouri, A.; Mniai, E.M.; Chaibi, Y.; Franciosini, A.; Clavel, L.; Bidault, F.; et al. Contribution of an Artificial Intelligence Tool in the Detection of Incidental Pulmonary Embolism on Oncology Assessment Scans. Life 2024, 14, 1347. https://doi.org/10.3390/life14111347
Ammari S, Camez AO, Ayobi A, Quenet S, Zemmouri A, Mniai EM, Chaibi Y, Franciosini A, Clavel L, Bidault F, et al. Contribution of an Artificial Intelligence Tool in the Detection of Incidental Pulmonary Embolism on Oncology Assessment Scans. Life. 2024; 14(11):1347. https://doi.org/10.3390/life14111347
Chicago/Turabian StyleAmmari, Samy, Astrid Orfali Camez, Angela Ayobi, Sarah Quenet, Amir Zemmouri, El Mehdi Mniai, Yasmina Chaibi, Angelo Franciosini, Louis Clavel, François Bidault, and et al. 2024. "Contribution of an Artificial Intelligence Tool in the Detection of Incidental Pulmonary Embolism on Oncology Assessment Scans" Life 14, no. 11: 1347. https://doi.org/10.3390/life14111347
APA StyleAmmari, S., Camez, A. O., Ayobi, A., Quenet, S., Zemmouri, A., Mniai, E. M., Chaibi, Y., Franciosini, A., Clavel, L., Bidault, F., Muller, S., Lassau, N., Balleyguier, C., & Assi, T. (2024). Contribution of an Artificial Intelligence Tool in the Detection of Incidental Pulmonary Embolism on Oncology Assessment Scans. Life, 14(11), 1347. https://doi.org/10.3390/life14111347