Economic Evaluation of Artificially Intelligent (AI) Diagnostic Systems: Cost Consequence Analysis of Clinician-Friendly Interpretable Computer-Aided Diagnosis (ICADX) Tested in Cardiology, Obstetrics, and Gastroenterology, from the HosmartAI Horizon 2020 Project
Abstract
1. Introduction
2. Methods
3. Results
- Scenario 1: Cost Consequence Analysis of Preterm Births Early Diagnosis
- 2.
- Scenario 2: Cost Consequence Analysis of Echocardiography
- 3.
- Scenario 3. Cost Consequence Analysis of CCTA Scenario
- 4.
- Scenario 4. Cost Consequence Analysis of Capsule Endoscopy Scenario
Sensitivity Analysis
4. Discussion
5. Limitations
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
AI | Artificially Intelligent |
CCA | Cost Consequence Analysis |
ICADX | Clinician-Friendly Interpretable Computer-Aided Diagnosis |
CCTA | Coronary Computed Tomography Angiography |
CE | Capsule Endoscopy |
DHI | Digital Health Interventions |
OB-GYN | Obstetrics gynecology |
DRG | Diagnosis-Related Group |
SUS | System Usability Score |
MAE | Mean Absolute Error |
HosmartAI | The Smart Hospital with AI Technologies |
ROI | Return on Investment |
CVD | Cardiovascular Diseases |
QALYs | Quality-Adjusted Life-Years |
EU | European Union |
References
- Sun, F.; Schoelles, K.M.; Coates, V.H. Assessing the Utility of Genetic Tests. J. Ambul. Care Manag. 2013, 36, 222–232. [Google Scholar] [CrossRef] [PubMed]
- Huang, E.P.; Lin, F.I.; Shankar, L.K. Beyond Correlations, Sensitivities, and Specificities. Acad. Radiol. 2017, 24, 1036–1049. [Google Scholar] [CrossRef]
- Pepe, M.S. The Statistical Evaluation of Medical Tests for Classification and Prediction; Oxford University Press: Oxford, UK, 2003. [Google Scholar]
- Zhou, X.H.; Obuchowski, N.A.; McClish, D.K. Statistical Methods in Diagnostic Medicine; John Wiley & Sons: Hoboken, NJ, USA, 2014. [Google Scholar]
- Khalifa, M.; Albadawy, M. AI in diagnostic imaging: Revolutionising accuracy and efficiency. Comput. Methods Programs Biomed. Update 2024, 5, 100146. [Google Scholar] [CrossRef]
- Najjar, R. Redefining Radiology: A Review of Artificial Intelligence Integration in Medical Imaging. Diagnostics 2023, 13, 2760. [Google Scholar] [CrossRef] [PubMed]
- Hameed, B.M.Z.; Prerepa, G.; Patil, V.; Shekhar, P.; Zahid Raza, S.; Karimi, H.; Paul, R.; Naik, N.; Modi, S.; Vigneswaran, G.; et al. Engineering and clinical use of artificial intelligence (AI) with machine learning and data science advancements: Radiology leading the way for future. Ther. Adv. Urol. 2021, 13, 175628722110448. [Google Scholar] [CrossRef]
- Srivastav, S.; Chandrakar, R.; Gupta, S.; Babhulkar, V.; Agrawal, S.; Jaiswal, A.; Prasad, R.; Wanjari, M.B. ChatGPT in Radiology: The Advantages and Limitations of Artificial Intelligence for Medical Imaging Diagnosis. Cureus 2023, 15, e41435. [Google Scholar] [CrossRef]
- Lijmer, J.G.; Bossuyt, P.M.M. Various randomized designs can be used to evaluate medical tests. J. Clin. Epidemiol. 2009, 62, 364–373. [Google Scholar] [CrossRef]
- Mauskopf, J.A.; Paul, J.E.; Grant, D.M.; Stergachis, A. The Role of Cost-Consequence Analysis in Healthcare Decision-Making. PharmacoEconomics 1998, 13, 277–288. [Google Scholar] [CrossRef]
- NICE Medical Technologies Evaluation Programme Methods Guide Process and Methods [Internet]. 2017. Available online: https://www.nice.org.uk/process/pmg33/resources/medical-technologies-evaluation-programme-methods-guide-pdf-72286774205893 (accessed on 5 January 2023).
- Geitona, M.; Hatzikou, M.; Steiropoulos, P.; Alexopoulos, E.C.; Bouros, D. The cost of COPD exacerbations: A university hospital—Based study in Gr. Respir. Med. 2011, 105, 402–409. [Google Scholar] [CrossRef]
- Apostolidis, G.; Kakouri, A.; Dimaridis, I.; Vasileiou, E.; Gerasimou, I.; Charisis, V.; Hadjidimitriou, S.; Lazaridis, N.; Germanidis, G.; Hadjileontiadis, L. A web-based platform for studying the impact of artificial intelligence in video capsule endoscopy. Health Inform. J. 2024, 30, 14604582241296072. [Google Scholar] [CrossRef]
- Kyparissidis-Kokkinidis, I.; Rigas, E.S.; Logaras, S.; Tsakiridis, E.; Dagklis, I.; Lagakis, P.; Billis, A.; Bamidis, P.D. An Explainable AI-Based Decision Support Tool to Predict Preterm Birth. In Proceedings of the 2024 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI), Houston, TX, USA, 10–13 November 2024; IEEE: Piscataway, NJ, USA, 2024; pp. 1–8. [Google Scholar]
- Kyparissidis Kokkinidis, I.; Rigas, E.S.; Logaras, E.; Samaras, A.; Rampidis, G.P.; Giannakoulas, G.; Kouskouras, K.; Billis, A.; Bamidis, P.D. Towards an explainable ai-based tool to predict the presence of obstructive coronary artery disease. In Proceedings of the 26th Pan-Hellenic Conference on Informatics, New York, NY, USA, 25–27 November 2022; pp. 335–340. [Google Scholar]
- Rigas, E.S.; Lagakis, P.; Karadimas, M.; Logaras, E.; Latsou, D.; Hatzikou, M.; Poulakidas, A.; Billis, A.; Bamidis, P.D. Semantic interoperability for an AI-based applications platform for smart hospitals using HL7 FHIR. J. Syst. Softw. 2024, 215, 112093. [Google Scholar] [CrossRef]
- Chatzikou, M. The Evaluation of Key Performance Indicators for Medical and Digital Technologies in Healthcare. In Intelligent Technologies for Healthcare Business Applications [Internet]; Springer: Vienna, Austria, 2024; pp. 93–117. [Google Scholar] [CrossRef]
- Pennazio, M.; Rondonotti, E.; Despott, E.J.; Dray, X.; Keuchel, M.; Moreels, T.; Sanders, D.S.; Spada, C.; Carretero, C.; Cortegoso Valdivia, P.; et al. Small-bowel capsule endoscopy and device-assisted enteroscopy for diagnosis and treatment of small-bowel disorders: European Society of Gastrointestinal Endoscopy (ESGE) Guideline–Update 2022. Endoscopy 2023, 55, 58–95. [Google Scholar] [CrossRef] [PubMed]
- Rondonotti, E.; Pennazio, M.; Toth, E.; Koulaouzidis, A. How to read small bowel capsule endoscopy: A practical guide for everyday use. Endosc. Int. Open 2020, 8, E1220–E1224. [Google Scholar] [CrossRef] [PubMed]
- Bharadwaj, P.; Nicola, L.; Breau-Brunel, M.; Sensini, F.; Tanova-Yotova, N.; Atanasov, P.; Lobig, F.; Blankenburg, M. Unlocking the Value: Quantifying the ROI of Hospital AI. J. Am. Coll. Radiol. 2024, 21, 1677–1685. [Google Scholar] [CrossRef]
- Ayorinde, A.; Mensah, D.O.; Walsh, J.; Ghosh, I.; Ibrahim, S.A.; Hogg, J.; Peek, N.; Griffiths, F. Healthcare Professionals’ Experience of Using Artificial Intelligence: A Systematic Review with Narrative Synthesis (Preprint). J. Med. Internet Res. 2024, 26, e55766. [Google Scholar] [CrossRef]
- Malani, S.N.; Shrivastava, D.; Raka, M.S. A Comprehensive Review of the Role of Artificial Intelligence in Obstetrics and Gynecology. Cureus 2023, 15, e34891. [Google Scholar] [CrossRef]
- Dhali, A.; Kipkorir, V.; Maity, R.; Srichawla, B.S.; Biswas, J.; Rathna, R.B.; Bharadwaj, H.R.; Ongidi, I.; Chaudhry, T.; Morara, G.; et al. Artificial Intelligence–Assisted Capsule Endoscopy Versus Conventional Capsule Endoscopy for Detection of Small Bowel Lesions: A Systematic Review and Meta-Analysis. J. Gastroenterol. Hepatol. 2025, 40, 1105–1118. [Google Scholar] [CrossRef]
- Meltzer, A.C.; Ward, M.J.; Gralnek, I.M.; Pines, J.M. The cost-effectiveness analysis of video capsule endoscopy compared to other strategies to manage acute upper gastrointestinal hemorrhage in the ED. Am. J. Emerg. Med. 2014, 32, 823–832. [Google Scholar] [CrossRef]
- Jiang, X.; Ming, W.K.; You, J.H. The Cost-Effectiveness of Digital Health Interventions on the Management of Cardiovascular Diseases: Systematic Review. J. Med. Internet Res. 2019, 21, e13166. [Google Scholar] [CrossRef]
- Vithlani, J.; Hawksworth, C.; Elvidge, J.; Ayiku, L.; Dawoud, D. Economic evaluations of artificial intelligence-based healthcare interventions: A systematic literature review of best practices in their conduct and reporting. Front. Pharmacol. 2023, 14, 1220950. [Google Scholar] [CrossRef]
- Khanna, N.N.; Maindarkar, M.A.; Viswanathan, V.; Fernandes, J.F.E.; Paul, S.; Bhagawati, M.; Ahluwalia, P.; Ruzsa, Z.; Sharma, A.; Kolluri, R.; et al. Economics of Artificial Intelligence in Healthcare: Diagnosis vs. Treatment. Healthcare 2022, 10, 2493. [Google Scholar] [CrossRef] [PubMed]
- Wang, Y.; Liu, C.; Hu, W.; Luo, L.; Shi, D.; Zhang, J.; Yin, Q.; Zhang, L.; Han, X.; He, M. Economic evaluation for medical artificial intelligence: Accuracy vs. cost-effectiveness in a diabetic retinopathy screening case. NPJ Digit. Med. 2024, 7, 43. [Google Scholar] [CrossRef] [PubMed] [PubMed Central]
- Sanders, G.D.; Neumann, P.J.; Basu, A.; Brock, D.W.; Feeny, D.; Krahn, M.; Kuntz, K.M.; Meltzer, D.O.; Owens, D.K.; Prosser, L.A.; et al. Recommendations for Conduct, Methodological Practices, and Reporting of Cost-effectiveness Analyses. JAMA 2016, 316, 1093. [Google Scholar] [CrossRef] [PubMed]
- Gomes, M.; Murray, E.; Raftery, J. Economic Evaluation of Digital Health Interventions: Methodological Issues and Recommendations for Practice. PharmacoEconomics 2022, 40, 367–378. [Google Scholar] [CrossRef]
- Wilkinson, T.; Wang, W.; Friedman, J.; Görgens, M. Health, Nutrition and Population Global Practice & Development Research Group. In Policy Research Working Paper 10407 A Framework for the Economic Evaluation of Digital Health Interventions; Health, Nutrition and Population Global Practice & Development Research Group: Washington, DC, USA, 2023. [Google Scholar]
Scenario | Clinical Area | AI Function |
---|---|---|
1. Early diagnosis of preterm births | Obstetrics | Early risk detection of preterm birth using maternal clinical data |
2. Automated estimation of LV-EF and LV-GLS | Cardiology (Echocardiography) | Automatic measurement of cardiac function metrics (LV-EF, LV-GLS) |
3. AI-assisted detection of coronary stenosis (CCTA) | Cardiology (Imaging) | AI analysis of CCTA data and clinical biomarkers to detect stenosis |
4. Automated capsule endoscopy interpretation | Gastroenterology | Detection and classification of small bowel abnormalities |
Cost Consequence Analysis—Obstetrics Scenario | ||||
---|---|---|---|---|
Cost/Outcome Categories | HosmartAI Intervention (Annual Cost) | Current Practice (Annual Cost) | Difference | Discounted 5-Year Total (EUR) |
Cost of AI technology (personnel) | 5000 EUR | 0 EUR | 5000 EUR | 23,585 EUR |
Cost of maintenance | 6972 EUR | 0 EUR | 6972 EUR | 32,897 EUR |
Cost of AI infrastructure | 170 EUR | 0 EUR | 170 EUR | 802 EUR |
Cost of prematurity (age category 1500–1999 g) with co-morbidity (T25Μγ) per case | 0 EUR | 3158 EUR | −3158 EUR | −14,895 EUR |
Cost of prematurity (age category 1500–1999 g) without comorbidity (DRGT25X) | 2646 EUR | 0 EUR | 2646 EUR | 12,481 EUR |
Cost of averted severity prematurity (n = 195 preterm births annually) annual cost | 515,970 EUR | 615,810 EUR | −99,840 EUR | −470,955 EUR |
Consequence Categories | HosmartAI Intervention | Current Practice | Difference | Discounted 5-Year Total |
Clinical performance | 0.83 | 0.75 | 0.08 | 0.377 |
Duration of diagnosis PTB > 75% (duration for experienced physician > 2 years) in weeks | 34 weeks | 28 weeks | 6 weeks | 28.3 weeks |
Cost Consequence Analysis—Echocardiography Scenario | ||||
---|---|---|---|---|
Cost/Outcome Categories | HosmartAI Intervention (Annual Cost) | Current Practice (Annual Cost) | Difference | Discounted 5-Year Total (EUR) |
Cost of AI technology (personnel) | 5000 EUR | 0 EUR | 5000 EUR | 23,585 EUR |
Cost of maintenance | 3000 EUR | 0 EUR | 3000 EUR | 14,151 EUR |
Cost of AI infrastructure | 300 EUR | 0 EUR | 300 EUR | 1415 EUR |
Physician cost of LV-EF measurement (n = 2880 patients annually) | 634 EUR | 1382 EUR | −749 EUR | −3534 EUR |
Physician cost of LV-GLS measurement (n = 1440 patients annually) | 475 EUR | 734 EUR | −259 EUR | −1222 EUR |
Total cost per year | 9409 EUR | 2116 EUR | 7292 EUR | 34,395 EUR |
Consequence Categories | HosmartAI Intervention | Current Practice | Difference | Discounted 5-Year Total |
System usability (SUS) | 75.00% | - | 75.00% | Not time-dependent |
Mean absolute error (MAE) of automatic measurement of LV-EF | 5.55 | - | 5.55 | Constant |
Accuracy of the automated analysis | 3.03 | - | 3.03 | Constant |
Diagnostic accuracy for a low-experience physician (low-experience physician: <5 years) (Youden’s J index) | 0.80 | 0.54 | 0.26 | 1.23 |
Diagnostic accuracy for a highly experienced physician (highly experienced physician: >5 years) (Youden’s J index) | 0.82 | 0.64 | 0.18 | 0.85 |
Average time for measurement of LV-EF by a highly experienced physician (in min) | 1.00 | 2.00 | −1.00 | −4.72 |
Average time for measurement of LV-EF by a low-experience physician (in min) | 1.00 | 1.00 | 0.00 | - |
Average time for measurement of LV-GLS by a low-experience physician (in min) | 1.50 | 4.00 | −2.50 | −11.79 |
Average time for measurement of LV-GLS by a highly experienced physician (in min) | 1.50 | 3.00 | −1.50 | −7.08 |
Cost Consequence Analysis—CCTA Scenario | ||||
---|---|---|---|---|
Cost/Outcome Categories | HosmartAI Intervention (Annual Cost) | Current Practice (Annual cost) | Difference | Discounted 5-Year Total (EUR) |
Cost of AI technology (personnel) | 5000 EUR | 0 EUR | 5000 EUR | 23,585 EUR |
Cost of maintenance | 6972 EUR | 0 EUR | 6972 EUR | 32,897 EUR |
Cost of AI infrastructure | 170 EUR | 0 EUR | 170 EUR | 802 EUR |
CCTA examination | 127 EUR | 286 EUR | −159 EUR | - |
CCTA examination (n = 239 patients annually) | 29,718 EUR | 66,924 EUR | −37,206 EUR | −175,594 EUR |
Total cost per year | 41,860 EUR | 67,210 EUR | −25,350 EUR | −118,310 EUR |
Consequence Categories | HosmartAI Intervention | Current Practice | Difference | Discounted 5-Year Total |
Clinical performance | 84.00% | 77.00% | 7.00% (0.07) | 0.33 |
Duration of diagnosis (duration for experienced physician > 2 years) | 3 examinations | 2 examinations | 1 examination | −4.72 fewer exams |
Duration of diagnosis (duration for experienced physician < 2 years) | 3 examinations | 2 examinations | 1 examination | −4.72 fewer exams |
Cost-Consequences Analysis—Capsule Endoscopy Scenario | ||||
---|---|---|---|---|
Cost/Outcomes Categories | HosmartAI Intervention (Annual Cost) | Current Practice (Annual Cost) | Difference | Discounted 5-Year Total (EUR) |
Cost of AI technology (personnel) | 5000 EUR | 0 EUR | 5000 EUR | 23,585 EUR |
Cost of maintenance | 3000 EUR | 0 EUR | 3000 EUR | 14,151 EUR |
Cost of capsule | 60,000 EUR | 60,000 EUR | 0 EUR | 0 EUR |
Small bowel capsule endoscopy review | 0 EUR | 1674 EUR | −1674 EUR | −7897 EUR |
Cost of infrastructure | 300 EUR | 0 EUR | 300 EUR | 1415 EUR |
Total cost per year | 68,300 EUR | 61,674 EUR | 6626 EUR | 31,254 EUR |
Consequence categories | HosmartAI Intervention | Current Practice | Difference | Discounted 5-year Total |
System usability (SUS) | 76.40% | 70% | 6% | Not time-dependent |
Sensitivity of automated detection of small bowel abnormalities | 0.89 | 0.90 | −0.01 | Slight reduction |
Average time for completion of small bowel VCE reading (in min) | 60.00 | 240 | −180.00 | −849 min |
Number of personnel involved in screening/examination | 1 senior doctor | 1 junior doctor and 1 senior doctor | junior doctor −1 | Workload Impact (less resources) |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Chatzikou, M.; Latsou, D.; Apostolidis, G.; Billis, A.; Charisis, V.; Rigas, E.S.; Bamidis, P.D.; Hadjileontiadis, L. Economic Evaluation of Artificially Intelligent (AI) Diagnostic Systems: Cost Consequence Analysis of Clinician-Friendly Interpretable Computer-Aided Diagnosis (ICADX) Tested in Cardiology, Obstetrics, and Gastroenterology, from the HosmartAI Horizon 2020 Project. Healthcare 2025, 13, 1661. https://doi.org/10.3390/healthcare13141661
Chatzikou M, Latsou D, Apostolidis G, Billis A, Charisis V, Rigas ES, Bamidis PD, Hadjileontiadis L. Economic Evaluation of Artificially Intelligent (AI) Diagnostic Systems: Cost Consequence Analysis of Clinician-Friendly Interpretable Computer-Aided Diagnosis (ICADX) Tested in Cardiology, Obstetrics, and Gastroenterology, from the HosmartAI Horizon 2020 Project. Healthcare. 2025; 13(14):1661. https://doi.org/10.3390/healthcare13141661
Chicago/Turabian StyleChatzikou, Magda, Dimitra Latsou, Georgios Apostolidis, Antonios Billis, Vasileios Charisis, Emmanouil S. Rigas, Panagiotis D. Bamidis, and Leontios Hadjileontiadis. 2025. "Economic Evaluation of Artificially Intelligent (AI) Diagnostic Systems: Cost Consequence Analysis of Clinician-Friendly Interpretable Computer-Aided Diagnosis (ICADX) Tested in Cardiology, Obstetrics, and Gastroenterology, from the HosmartAI Horizon 2020 Project" Healthcare 13, no. 14: 1661. https://doi.org/10.3390/healthcare13141661
APA StyleChatzikou, M., Latsou, D., Apostolidis, G., Billis, A., Charisis, V., Rigas, E. S., Bamidis, P. D., & Hadjileontiadis, L. (2025). Economic Evaluation of Artificially Intelligent (AI) Diagnostic Systems: Cost Consequence Analysis of Clinician-Friendly Interpretable Computer-Aided Diagnosis (ICADX) Tested in Cardiology, Obstetrics, and Gastroenterology, from the HosmartAI Horizon 2020 Project. Healthcare, 13(14), 1661. https://doi.org/10.3390/healthcare13141661